nitishKumar
/
Skill
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

ESD: add Peak Power(W) param (template v2, skill v2)

main
library-manager 2026-07-10 07:57:00 +00:00
parent 1537551e36
commit 9c7684f1b2
No known key found for this signature in database
13 changed files with 1479 additions and 370 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

419
SKILL.md
View File

@ -1,270 +1,271 @@
---
name: datasheet-extractor
name: library-manager
description: >-
Extract key parameters from component datasheet PDFs into the Vecmocon master
template (18 component types incl. resistors, capacitors, inductors, diodes,
transistors, ICs, protection, power modules, connectors, sensors, and more).
Column A = MPN_make_typeid (make = first word of manufacturer, typeid from the
taxonomy). Writes ONE master workbook (Components_Master.xlsx) with sheets only for the
types whose datasheets were provided, plus a Meta sheet at the end (template version,
total components, date, time), assembles
a per-part DFS folder (datasheet, Altium footprint, Altium symbol), and pushes
design files to the Gitea DFS repo, Excel outputs to the Parameters repo, and skill
files plus templates to the Skill_Assets repo. Also appends new parameters to a
template as a new version. Use WHENEVER the user uploads component datasheets and
asks to extract parameters, build the parameter sheets or component library, or push
datasheets to Gitea. ALWAYS trigger when the user types "\datasheet".
Manage Vecmocon's component library: extract parameters from a component datasheet PDF into
the per-typeid Excel template (125 type-IDs across 18 classes - resistors, capacitors,
diodes, transistors, ICs, connectors, sensors, etc.). The user gives a datasheet named
as its MPN plus the make. The skill checks Gitea for a duplicate MPN_make and hard-stops if
present, classifies the part to a typeid, confirms or updates that typeid's template
(per-typeid version, changelog, backfill), fills a per-part workbook, loops on human
verification, then takes the Altium symbol (.SchLib) and footprint (.PcbLib), fills the
Library/Footprint columns, assembles a part folder (xlsx, datasheet, symbol, footprint) and
pushes it to the components repo under its Class. Use WHENEVER the user uploads a component
datasheet, builds a library entry, adds a parameter to a type template, or pushes a part to
Gitea. ALWAYS trigger on "\datasheet", "\library", or "\library-manager", or any
component-library / datasheet-extraction task.
---
# Datasheet Extractor
# Library Manager
Read component datasheets, fill the customer's master template, assemble each part's
design-file folder, and publish everything to three Gitea repos. Be careful and honest:
put every value in the right column and unit, and flag what a datasheet does not state.
**Pushing is the default.** At the end of a run, push every output to Gitea without asking
for confirmation — that's the whole job. Only pause when there's a genuine need for the
user: a **login wall** while fetching a footprint/symbol, or an **MPN that already exists**
in Gitea (discard vs replace, because replace overwrites data). Nothing else warrants a
prompt.
Turn one component datasheet into a verified, versioned library entry in Gitea. The guiding
idea is honesty and traceability: every value lands in the right column and unit, anything
the datasheet doesn't state stays blank, and nothing reaches Gitea until a human has
confirmed it. This is an **interactive** skill — it asks at the few points where a person's
judgement or an upload is genuinely needed, and does everything else on its own.
## Inputs
- **Datasheet PDFs**, one per part. The filename is the part's MPN (search that exact
number inside a series datasheet to read the right variant).
- No project/version is needed — repos use a **flat layout**.
- **A datasheet PDF whose filename is the MPN** (e.g. `BAT46WJ.pdf`). If it's a series
datasheet, search that exact MPN inside to read the correct variant.
- **The make** (manufacturer), given by the user. The `make` tag is the first word of the
manufacturer, alphanumerics only (Texas Instruments → `Texas`, Nexperia → `Nexperia`).
- **Later in the flow**, after verification: an Altium **symbol** (`.SchLib`) and
**footprint** (`.PcbLib`) file, provided by the user.
## The identifier: `MPN_make_typeid`
Column A of every sheet (header `MPN_make_type`) and each DFS folder name is:
Every part folder, every per-part workbook, and column A of every sheet (`MPN_make_type`)
use the same tag:
```
<MPN>_<make>_<typeid> e.g. BAT46WJ_Nexperia_SCH
```
- **make** = the **first word of the manufacturer** name (Texas Instruments → `Texas`,
Nexperia → `Nexperia`, STMicroelectronics → `STMicroelectronics`).
- **typeid** = the Type ID for the part's subclass, from the taxonomy in
`references/taxonomy.md` (full source `assets/template/Type_ID.xlsx`). e.g. Schottky →
`SCH`, TVS → `TVS`, MOSFET → `MOS`, LDO → `LDO`, Common-mode choke → `CMC`.
`typeid` is the part's type-ID code from the taxonomy (`references/taxonomy.md`, full source
`assets/template/Type_ID.xlsx`) — Schottky → `SCH`, MOSFET → `MOS`, LDO → `LDO`. In the new
template **each typeid is its own sheet** (125 of them). The broader **Class** (Diode,
Transistor, IC …) is used only to organise the components repo into top-level folders.
## Gitea layout (two repos)
```
skill repo/ this skill's own files (updated versions land here too)
components repo/
<Class>/ e.g. Diode, IC, Transistor, Resistor, ...
<MPN>_<make>_<typeid>/ e.g. BAT46WJ_Nexperia_SCH
<MPN>_<make>_<typeid>.xlsx this part's own one-row parameter sheet
<MPN>_data.pdf the datasheet
<symbol>.SchLib user-provided
<footprint>.PcbLib user-provided
```
There is **no single master workbook** — each part carries its own sheet inside its folder.
Connection + repo names live in `config/gitea.env` (`SKILL_REPO`, `COMPONENTS_REPO`), so runs
need no per-session token. If the host is unreachable, the git steps fail clearly and write
nothing.
## Workflow
1. **Read each datasheet.** Get MPN (from filename), manufacturer, and the parameters.
2. **Classify** to one of the 18 **types** (= template sheet names) and to a **subclass**
from `references/taxonomy.md`; record its **typeid**. The subclass name goes in the
`Class` column where the sheet has one.
3. **Extract values** into that type's columns (headers come from
`assets/template/template.xlsx`). Convert to each header's unit; leave blank if the
datasheet doesn't state it (an honest blank beats a guess).
4. **Build the master workbook** with `scripts/fill_templates.py` — a single
`Components_Master.xlsx` that mirrors the template but keeps **only the sheets for the
types you actually extracted** (the components whose datasheets were given), each with
column A = `MPN_make_typeid`, and a single **Meta** sheet appended at the end. (See
Producing outputs.)
5. **Assemble each part's DFS folder** `MPN_make_typeid/` containing `MPN_data` (the
datasheet), `MPN_fp` (footprint), `MPN_sym` (symbol). See Footprint & symbol.
6. **Push everything to Gitea — automatically.** At the end of a run, push all outputs to
all three repos with no confirmation step: DFS folders → **DFS**, per-type sheets →
**Parameters** (merged in), skill files + templates → **Skill_Assets**. The only reasons
to pause are a login wall or an already-present MPN (discard vs replace); otherwise it
all just goes. See Pushing to Gitea.
Run these in order. Each `python`/`bash` command is a helper in `scripts/`.
## Producing outputs
### 1. Duplicate check first — before any real work
Collect what you extracted into `parts.json` (keys match template headers; `typeid` and
`subclass` from the taxonomy; `manufacturer` used for the make tag):
```json
{"parts":[
{"type":"Diode","mpn":"BAT46WJ","manufacturer":"Nexperia","typeid":"SCH","subclass":"Schottky",
"values":{"Description":"100 V 250 mA Schottky, SOD323F","Forward Voltage(V)":"0.71","Package":"SOD323F"}}
]}
```
The part's presence is keyed on **MPN + make** (typeid not known yet). If it already exists,
stop; re-doing an existing part would only risk overwriting good data.
```bash
python scripts/fill_templates.py parts.json \
--template assets/template/template.xlsx --dest <outputs-dir>
python scripts/gitea_components.py check-mpn --mpn <MPN> --make <make>
```
Produces a **single** `Components_Master.xlsx`. It starts from a copy of the reference
template (so every kept sheet keeps its exact headers, styling, widths and freeze), fills
this run's parts into the matching sheet (column A = `MPN_make_typeid`), then **drops every
type sheet with no parts** — so the file holds only the components whose datasheets were
provided. A single **Meta** sheet is appended **at the end** with: Template Version, Total
Components, Date, Time, and a per-sheet count breakdown. Deliver this one file to the user;
the Parameters repo stores the same master, accumulated across runs (see Pushing to Gitea).
`EXISTS …` (exit 3) → **stop and tell the user the part is already present in Gitea. End
here.** `ABSENT` (exit 0) → continue.
Sheet naming follows the template (Diode, Resistor, …), not `Sheet1/Sheet2`. If the user
supplies a new reference template, drop it in at `assets/template/template.xlsx` and the
master mirrors whatever sheets/headers it has.
### 2. Classify → typeid (and its Class)
## Footprint & symbol (for the DFS folder)
Read the datasheet, identify the part, and match it to the closest subclass in
`references/taxonomy.md`; record its **typeid** (= the template sheet name). The **Class**
(for the components-repo folder) comes from the same taxonomy row —
`scripts/common.py:class_folder(typeid)` returns it (e.g. `SCH` → `Diode`).
For each part, the DFS folder needs `MPN_fp` (footprint) and `MPN_sym` (symbol) beside
`MPN_data` (the datasheet). **Deliver these as Altium files** — an Altium PCB footprint
(`.PcbLib`) and schematic symbol (`.SchLib`), or a single integrated library (`.IntLib`)
that carries both — because that's what the design team actually consumes. Get them in
this order:
### 3. Confirm the typeid's template (and add parameters if asked)
1. **Auto-download via the user's pre-logged-in Chrome** (`scripts/fetch_cad.py`). The user
keeps signed-in CAD-site windows open; the script attaches to that running Chrome and
reuses those live sessions — it never logs in, and no password is ever handled. One-time
setup: the user quits Chrome, relaunches it with `--remote-debugging-port=9222
--user-data-dir="$HOME/cad-chrome"`, and signs into the sites. Then, per part:
Check whether that typeid has a sheet in `assets/template/template.xlsx`.
- **No sheet for this typeid** → ask the user to upload the template sheet for it. Add it to
`assets/template/template.xlsx`, then push the updated skill files to the skill repo (see
*Pushing the skill repo*). Then continue.
- **Sheet exists** → print **all** of that sheet's parameters (its column headers) in the
chat and ask the user whether any new parameters should be added.
- **No** → go to step 4.
- **Yes** → collect the new parameter name(s), then:
```bash
python scripts/fetch_cad.py --mpn BAT46WJ --dest <DFS_folder> \
--sites snapeda,ultralibrarian,componentsearchengine,digikey
python scripts/append_parameter.py --typeid <typeid> \
--param "New Parameter(unit)" [--param "Another(unit)"] \
--desc "why these were added"
```
It tries each site in order, downloads the Altium model, and drops it into the DFS folder
named `MPN_fp` / `MPN_sym` (or `MPN_cad` for a bundle). **It never solves CAPTCHAs or
defeats bot-detection** — if a site shows a login wall, CAPTCHA, or bot-check, that
adapter stops and reports `manual` with the URL. Note: this runs on the *user's* machine
(where the browsers are), and some sites' terms restrict automated downloads — treat it
as a convenience with the manual fallback below.
This appends the column(s) at the end of that typeid's sheet, **bumps that typeid's
Template Version and Skill Version together** (v1→v2 — see *Per-typeid versioning*),
and writes one entry to the global changelog `assets/template/CHANGELOG.md`. Push the
updated `template.xlsx` + `versions.json` + `CHANGELOG.md` to the **skill** repo, and
make sure the components repo's copies stay consistent.
2. **Manual fallback** (script returned `manual`/`notfound`, or no browser session): **share
the user the direct link** where the model lives and ask them to download it and send it
back; then attach it into the DFS folder with the agreed naming (`MPN_fp` / `MPN_sym`).
Never type, store, or ask for their password, and never write credentials into this skill
— it's pushed to Skill_Assets, and a secret in a repo is a real leak (same reason the
`GIT_TOKEN` is blanked before syncing).
Then ask: **should this change apply to the parts of this typeid already in Gitea?**
- **No** → go to step 4 (only the current part gets the new column).
- **Yes → backfill** (see *Backfilling existing parts*), then tell the user the previous
sheets were updated, and go to step 4.
Whichever path, once you have a candidate **cross-check** its pad/pin count and key
package dimensions against the datasheet's package drawing before trusting it — library
models are sometimes wrong, or drawn for a different variant of the series.
### 4. Extract and fill the per-part workbook
3. **If no good model exists, generate the footprint/symbol yourself** as Altium files
from the datasheet's package drawing and pinout — but only when you can do it reliably
— then **ask the user to verify** it against the datasheet before it's trusted, since a
generated land pattern that's slightly off is worse than an obvious blank.
Read every parameter the datasheet actually states into that typeid's columns, converting to
each header's unit. **Leave blanks where the datasheet is silent — an honest blank beats a
guess.** Collect them into a small `part.json`:
4. **If none of that is possible**, still create the folder with the datasheet and **tell
the user that MPN needs a manual footprint/symbol build**, leaving a clearly-named
placeholder or listing it so nothing silently goes missing.
Keep the datasheet's extension on `MPN_data` (e.g. `BAT46WJ_data.pdf`). Name the
footprint/symbol files `MPN_fp` / `MPN_sym` with the Altium extension the model uses
(`.PcbLib` / `.SchLib`, or a single `.IntLib` when the source bundles both).
## Pushing to Gitea
Connection + repos are pre-configured in `config/gitea.env` (host, user, token, and the
three repos), so runs need no per-session token. Layout is **flat** (no project/version).
**Every run ends by pushing all three repos — no "shall I push?" step.** The user has
already asked for this to be automatic; don't re-ask. The reconcile step self-guards
against the one risky case (overwriting an existing part), so a straight push is safe by
default.
The three repos split into two kinds, and they're pushed differently:
- **DFS** and **Parameters** are **MPN-indexed** — DFS has one `MPN_make_typeid/` folder
per part; Parameters holds a **single master workbook** (`Components_Master.xlsx`) whose
type sheets each have one row per part (keyed on column A). Because a re-run can collide
with a part that's already there, these go through `scripts/gitea_reconcile.py`, which is
dedup-aware and **merge-safe** (it never blows away parts it isn't touching). See
*Handling parts already in Gitea* below.
- **Skill_Assets** holds the skill itself + templates — it is **not** per-part, so there's
nothing MPN-level to reconcile. Push it with `scripts/push_to_gitea.sh` as before.
```bash
# Skill files + templates -> Skill_Assets (flat copy; no dedup needed)
bash scripts/push_to_gitea.sh --repo "$SKILL_ASSETS_REPO" --src <skill-dir> --message "Sync skill assets"
```json
{"mpn":"BAT46WJ","manufacturer":"Nexperia","typeid":"SCH",
"values":{"Description":"100 V 250 mA Schottky, SOD323F","Forward Voltage(V)":"0.71",
"Reverse Voltage(V)":"100","Forward Current(A)":"0.25","Package":"SOD323F"}}
```
The push script clones the repo, copies the source contents in (flat), commits, and
pushes; the repo must already exist. If the host is unreachable (e.g. Cowork without the
domain allowlisted) it fails clearly and leaves the staged files for a manual push.
**Push the skill files organised** to Skill_Assets: keep the skill's own structure
(`SKILL.md`, `scripts/`, `assets/`, `references/`, `config/`) — but do **not** push the
real token. Before syncing skill assets, blank the `GIT_TOKEN` line in the copy you push,
or push everything except `config/gitea.env`.
> Why not push Parameters with the flat script? `fill_templates.py` writes a master
> workbook holding **only this run's** rows. Copied flat over the repo, it would overwrite
> the stored master and delete every previously-stored part. `gitea_reconcile.py` opens the
> repo's master, merges this run's rows into the right sheets, and writes it back — so only
> the parts you decided on change, and the Meta sheet (totals, date, time) refreshes.
## Handling parts already in Gitea (discard vs replace)
A part's identity is its tag `MPN_make_typeid` (same tag = same part). A re-run that
brings in a *new* MPN is not a conflict — it just gets added, and you push without asking.
The prompt only exists for the case where an MPN **already lives in DFS/Parameters**,
because "replace" overwrites data that's already on the server.
The simplest path is a single push. It clones the repos, adds everything new, and stops
**only** if it hits an existing MPN it can't resolve on its own:
```bash
python scripts/gitea_reconcile.py --parts parts.json --dfs-src <dfs-stage> \
--template assets/template/template.xlsx --push
python scripts/fill_templates.py part.json \
--template assets/template/template.xlsx --dest <stage>/<tag>/
```
- **No existing MPNs** → it adds all the folders/rows and pushes. No prompt. Done.
- **An MPN already exists** → it refuses (listing which) so you can ask the user, per MPN,
**discard** (keep what's in Gitea, drop the new copy) or **replace** (overwrite that MPN's
DFS folder *and* its Parameters row with the fresh extraction). Every other folder/row is
always left alone — the Parameters sheet is merged, never wholesale-overwritten.
This writes `<tag>.xlsx` — just that typeid's sheet, one row — with column A = the tag,
**Skill Version (col B)** and **Template Version (col C)** stamped from this typeid's current
versions, and the four design columns left blank for now.
Feed the user's answers back as a small JSON map (`tag → replace|discard`):
### 5. Human verification loop
Deliver the filled workbook to the user and ask them to verify it. If they report an error
or say it isn't right, **go back to step 4, re-read the datasheet more carefully, re-fill,
and hand it back.** Repeat until the user confirms it's verified. Nothing is pushed until
this passes — the engineer is the ground truth for the numbers.
### 6. Symbol + footprint → the design columns
Once verified, ask the user to upload the **symbol (`.SchLib`)** and **footprint
(`.PcbLib`)** files. Copy them into the staging part folder **under their proper names** (so
the Path columns match the files that actually get stored — strip any upload-staging prefix
the environment may have added), then read all four design values in one shot:
```bash
# decisions.json e.g. {"BAT46WJ_Nexperia_SCH": "replace", "1N4148_onsemi_SCH": "discard"}
python scripts/gitea_reconcile.py --parts parts.json --dfs-src <dfs-stage> \
--template assets/template/template.xlsx --decisions decisions.json --push
cp <uploaded_symbol> <stage>/<tag>/<symbol_name>.SchLib
cp <uploaded_footprint> <stage>/<tag>/<footprint_name>.PcbLib
python scripts/altium_refs.py design \
--symbol <stage>/<tag>/<symbol_name>.SchLib \
--footprint <stage>/<tag>/<footprint_name>.PcbLib > design.json
```
**For fully unattended runs**, skip the prompt entirely with a standing policy — every
conflict resolved the same way, no questions:
This produces (verified against real Ultra-Librarian exports):
- **Library Ref** = the component name **inside** the `.SchLib` (e.g. `CGA3E3X7R1H474K080AE`)
- **Library Path** = the `.SchLib` file name
- **Footprint Ref** = the **base** pattern inside the `.PcbLib`; Altium ships IPC density
variants (`-L` / `-M` / `-N`) alongside the base, and the base is the one used (e.g.
`CAP_CGA3_TDK`, not `CAP_CGA3_TDK-L`)
- **Footprint Path** = the `.PcbLib` file name
These Ref names come from *inside* the files and can differ from the MPN or filename. If a Ref
comes back `null` (or a footprint shows several unrelated candidates), ask the user to confirm
the name from Altium's properties and edit `design.json`. Then re-fill so the columns land:
```bash
python scripts/gitea_reconcile.py --parts parts.json --dfs-src <dfs-stage> \
--template assets/template/template.xlsx --on-conflict replace --push
python scripts/fill_templates.py part.json \
--template assets/template/template.xlsx --dest <stage>/<tag>/ --design design.json
```
`--on-conflict replace` (or `discard`) applies to all conflicts; an explicit per-MPN entry
in `--decisions` still wins over the blanket policy. Use `--report conflicts.json` (without
`--push`) if you ever want to preview conflicts first. Host unreachable → it fails clearly,
same as the push script.
### 7. Assemble the part folder
## Updating a template (new parameter)
The staging folder `<tag>/` should now hold the four files: the per-part `<tag>.xlsx`, the
datasheet (name it `<MPN>_data.<ext>`), the symbol, and the footprint.
When the user wants a new parameter on a type, **append it at the end** of that type's
sheet and bump the version:
### 8. Push to the components repo, under the part's Class
```bash
python scripts/append_parameter.py --type Diode --param "Reverse Recovery Time(ns)" \
--template assets/template/template.xlsx
python scripts/gitea_components.py push-part --folder <stage>/<tag> --typeid <typeid>
```
This adds the column at the end and increments `assets/template/VERSION` (v1 → v2 → …).
Then **push the updated `assets/template/template.xlsx` + `VERSION` to the Skill_Assets
repo** so the new template version is archived. New masters record the new version in the
Meta sheet automatically.
This places the folder at `components/<Class>/<tag>/` — creating the Class folder if it
doesn't exist yet, or pushing into it if it does — and commits and pushes. Confirm to the
user where it landed.
## Per-typeid versioning
Versioning is **per typeid**, not global. Each typeid carries its own `template_version` and
`skill_version` in `assets/template/versions.json` (both start at 1). When a parameter is
added to a typeid, that typeid gets a new template, so its `template_version` bumps — and on
the back of that its `skill_version` bumps too (v1→v2). **Only that typeid moves**; every
other typeid keeps its versions. Those two numbers are exactly what `fill_templates` stamps
into that typeid's rows (cols B and C), so a row always records the template/skill version it
was built against. `append_parameter.py` does the bump; `common.py` is the single source for
reading and writing these numbers.
## The changelog
`append_parameter.py` maintains one **global** changelog at `assets/template/CHANGELOG.md`
(newest entry on top), recording for each change: date, typeid + class, the template and
skill version transitions, the parameter(s) added, and the description. Push it to the skill
repo alongside `template.xlsx` and `versions.json` whenever it changes.
## Backfilling existing parts
When the user wants a newly-added parameter applied to parts of that typeid already in Gitea:
```bash
python scripts/gitea_components.py checkout --dest work/
python scripts/gitea_components.py list-type --typeid <typeid> --root work/ --json
```
`list-type` lists every existing part of that typeid with the files in its folder — including
its datasheet, which is co-located. For each one: read that datasheet, re-extract the values
(including the new parameter), and rebuild its per-part sheet in place:
```bash
python scripts/fill_templates.py <that_part>.json \
--template assets/template/template.xlsx --dest work/<Class>/<that_tag>/
```
Because `fill_templates` uses the current template and current versions, each rebuilt sheet
picks up the new column and the bumped version automatically. When all are done, push once
and tell the user the previous sheets were updated:
```bash
python scripts/gitea_components.py commit-push --root work/ --message "backfill <param> into <typeid>"
```
## Pushing the skill repo
When skill files change (a new typeid template, a parameter add, a version/changelog bump),
push the skill's own files — but **never push the real token**. Blank the `GIT_TOKEN` line in
the copy you push, or exclude `config/gitea.env`:
```bash
bash scripts/push_to_gitea.sh --repo "$SKILL_REPO" --src <skill-dir> --message "Sync skill files"
```
## Resources
- `assets/template/template.xlsx` — master template, one sheet per type (source of headers,
styling, and sheet order the master workbook mirrors).
- `assets/template/template.xlsx` — the master template: one sheet per **typeid** (125),
source of every sheet's headers, styling and order. Columns A/B/C are always
`MPN_make_type` / `Skill Version` / `Template Version`; `Library Ref/Path`,
`Footprint Ref/Path` and `Manufacturer` sit near the end.
- `assets/template/Type_ID.xlsx` + `references/taxonomy.md` — Class → Subclass → Type ID.
- `assets/template/VERSION` — current template version (integer; the Meta sheet shows `vN`).
- `scripts/fill_templates.py` — build the single master workbook `Components_Master.xlsx`
(only sheets for the extracted types + a trailing Meta sheet); also exposes the shared
helpers (`part_to_row`, `template_headers`, `sheet_rows`, `read_all_rows`, `build_master`,
`MASTER_NAME`) the reconcile step reuses so the Gitea master is built and merged identically.
- `scripts/gitea_reconcile.py` — add new MPNs and push automatically; only stops on an MPN
that already exists (discard/replace), or run unattended with `--on-conflict replace`.
Merge-safe: merges into the repo's master workbook, other rows/sheets/folders untouched.
- `scripts/push_to_gitea.sh` — push a folder's contents to a Gitea repo (flat). Used for
Skill_Assets (not MPN-indexed).
- `scripts/append_parameter.py` — append a parameter to a template + bump version.
- `scripts/fetch_cad.py` — download Altium footprint/symbol by attaching to the user's
pre-logged-in Chrome (remote-debugging port); tries sites in order, defers on
CAPTCHA/login (never bypasses), and names files `MPN_fp`/`MPN_sym`. Runs on the user's
machine; needs `selenium`.
- `config/gitea.env` — host, user, token, and the DFS / Parameters / Skill_Assets repos (**secret**).
- `assets/template/versions.json` — per-typeid `template_version` + `skill_version`.
- `assets/template/CHANGELOG.md` — global version/parameter changelog (created on first add).
- `scripts/common.py` — taxonomy loader (`load_taxonomy`, `class_folder`), version store
(`get_versions`, `version_labels`, `bump_versions`), and the tag helper (`part_tag`).
- `scripts/fill_templates.py` — build one per-part `<tag>.xlsx` (version-stamped); reused for
backfill.
- `scripts/append_parameter.py` — append parameter(s) to a typeid, bump its versions, write
the changelog.
- `scripts/altium_refs.py` — read Library/Footprint Ref from `.SchLib`/`.PcbLib`.
- `scripts/gitea_components.py` — `check-mpn`, `checkout`, `list-type`, `place-part`,
`commit-push`, `push-part` against the components repo.
- `scripts/push_to_gitea.sh` — push a folder's contents to a Gitea repo (used for the skill
repo).
- `config/gitea.env` — host, user, token, and the `SKILL_REPO` / `COMPONENTS_REPO` names
(**secret** — do not push the token).

7
assets/template/CHANGELOG.md Normal file
View File

@ -0,0 +1,7 @@
# library-manager - template/version changelog
## 2026-07-10 07:56 - ESD (Diode)
- Template v1 -> v2, Skill v1 -> v2
- Added parameter(s): "Peak Power(W)"
- Description: Add Peak Power (Ppk, 8x20us) capture for ESD protection diodes

BIN
assets/template/template.xlsx
View File

Binary file not shown.

502
assets/template/versions.json Normal file
View File

@ -0,0 +1,502 @@
{
"ADC": {
"skill_version": 1,
"template_version": 1
},
"AFE": {
"skill_version": 1,
"template_version": 1
},
"AMP": {
"skill_version": 1,
"template_version": 1
},
"ANC": {
"skill_version": 1,
"template_version": 1
},
"ANE": {
"skill_version": 1,
"template_version": 1
},
"ANP": {
"skill_version": 1,
"template_version": 1
},
"ARR": {
"skill_version": 1,
"template_version": 1
},
"BJT": {
"skill_version": 1,
"template_version": 1
},
"BMS": {
"skill_version": 1,
"template_version": 1
},
"BPK": {
"skill_version": 1,
"template_version": 1
},
"BRG": {
"skill_version": 1,
"template_version": 1
},
"BUZ": {
"skill_version": 1,
"template_version": 1
},
"CBB": {
"skill_version": 1,
"template_version": 1
},
"CBK": {
"skill_version": 1,
"template_version": 1
},
"CCO": {
"skill_version": 1,
"template_version": 1
},
"CER": {
"skill_version": 1,
"template_version": 1
},
"CFR": {
"skill_version": 1,
"template_version": 1
},
"CHG": {
"skill_version": 1,
"template_version": 1
},
"CHL": {
"skill_version": 1,
"template_version": 1
},
"CLF": {
"skill_version": 1,
"template_version": 1
},
"CLI": {
"skill_version": 1,
"template_version": 1
},
"CLK": {
"skill_version": 1,
"template_version": 1
},
"CMC": {
"skill_version": 1,
"template_version": 1
},
"CMP": {
"skill_version": 1,
"template_version": 1
},
"CNI": {
"skill_version": 1,
"template_version": 1
},
"CPL": {
"skill_version": 1,
"template_version": 1
},
"CTC": {
"skill_version": 1,
"template_version": 1
},
"CTX": {
"skill_version": 1,
"template_version": 1
},
"CWB": {
"skill_version": 1,
"template_version": 1
},
"DAC": {
"skill_version": 1,
"template_version": 1
},
"DCD": {
"skill_version": 1,
"template_version": 1
},
"DCM": {
"skill_version": 1,
"template_version": 1
},
"DIG": {
"skill_version": 1,
"template_version": 1
},
"DRV": {
"skill_version": 1,
"template_version": 1
},
"DSG": {
"skill_version": 1,
"template_version": 1
},
"DSW": {
"skill_version": 1,
"template_version": 1
},
"ELE": {
"skill_version": 1,
"template_version": 1
},
"ESD": {
"skill_version": 2,
"template_version": 2
},
"FAN": {
"skill_version": 1,
"template_version": 1
},
"FBD": {
"skill_version": 1,
"template_version": 1
},
"FFC": {
"skill_version": 1,
"template_version": 1
},
"FIX": {
"skill_version": 1,
"template_version": 1
},
"FLM": {
"skill_version": 1,
"template_version": 1
},
"FRD": {
"skill_version": 1,
"template_version": 1
},
"FSR": {
"skill_version": 1,
"template_version": 1
},
"FUS": {
"skill_version": 1,
"template_version": 1
},
"GAN": {
"skill_version": 1,
"template_version": 1
},
"GDT": {
"skill_version": 1,
"template_version": 1
},
"HDR": {
"skill_version": 1,
"template_version": 1
},
"HSK": {
"skill_version": 1,
"template_version": 1
},
"IDC": {
"skill_version": 1,
"template_version": 1
},
"IFC": {
"skill_version": 1,
"template_version": 1
},
"IGBT": {
"skill_version": 1,
"template_version": 1
},
"IND": {
"skill_version": 1,
"template_version": 1
},
"INV": {
"skill_version": 1,
"template_version": 1
},
"ISO": {
"skill_version": 1,
"template_version": 1
},
"JFET": {
"skill_version": 1,
"template_version": 1
},
"LCD": {
"skill_version": 1,
"template_version": 1
},
"LDO": {
"skill_version": 1,
"template_version": 1
},
"LED": {
"skill_version": 1,
"template_version": 1
},
"LOG": {
"skill_version": 1,
"template_version": 1
},
"MCU": {
"skill_version": 1,
"template_version": 1
},
"MEM": {
"skill_version": 1,
"template_version": 1
},
"MFR": {
"skill_version": 1,
"template_version": 1
},
"MMO": {
"skill_version": 1,
"template_version": 1
},
"MOR": {
"skill_version": 1,
"template_version": 1
},
"MOS": {
"skill_version": 1,
"template_version": 1
},
"MTR": {
"skill_version": 1,
"template_version": 1
},
"NTC": {
"skill_version": 1,
"template_version": 1
},
"OBC": {
"skill_version": 1,
"template_version": 1
},
"OLE": {
"skill_version": 1,
"template_version": 1
},
"OSC": {
"skill_version": 1,
"template_version": 1
},
"PBT": {
"skill_version": 1,
"template_version": 1
},
"PBZ": {
"skill_version": 1,
"template_version": 1
},
"PLY": {
"skill_version": 1,
"template_version": 1
},
"PMU": {
"skill_version": 1,
"template_version": 1
},
"POT": {
"skill_version": 1,
"template_version": 1
},
"PSU": {
"skill_version": 1,
"template_version": 1
},
"PTC": {
"skill_version": 1,
"template_version": 1
},
"PWC": {
"skill_version": 1,
"template_version": 1
},
"PWR": {
"skill_version": 1,
"template_version": 1
},
"RCM": {
"skill_version": 1,
"template_version": 1
},
"REC": {
"skill_version": 1,
"template_version": 1
},
"RFI": {
"skill_version": 1,
"template_version": 1
},
"RFM": {
"skill_version": 1,
"template_version": 1
},
"RLP": {
"skill_version": 1,
"template_version": 1
},
"RLR": {
"skill_version": 1,
"template_version": 1
},
"RLS": {
"skill_version": 1,
"template_version": 1
},
"RSF": {
"skill_version": 1,
"template_version": 1
},
"RSN": {
"skill_version": 1,
"template_version": 1
},
"RSW": {
"skill_version": 1,
"template_version": 1
},
"RSY": {
"skill_version": 1,
"template_version": 1
},
"SAW": {
"skill_version": 1,
"template_version": 1
},
"SCH": {
"skill_version": 1,
"template_version": 1
},
"SCM": {
"skill_version": 1,
"template_version": 1
},
"SCU": {
"skill_version": 1,
"template_version": 1
},
"SEN": {
"skill_version": 1,
"template_version": 1
},
"SFY": {
"skill_version": 1,
"template_version": 1
},
"SHA": {
"skill_version": 1,
"template_version": 1
},
"SHT": {
"skill_version": 1,
"template_version": 1
},
"SIC": {
"skill_version": 1,
"template_version": 1
},
"SIM": {
"skill_version": 1,
"template_version": 1
},
"SPK": {
"skill_version": 1,
"template_version": 1
},
"SPR": {
"skill_version": 1,
"template_version": 1
},
"SSR": {
"skill_version": 1,
"template_version": 1
},
"STE": {
"skill_version": 1,
"template_version": 1
},
"SUP": {
"skill_version": 1,
"template_version": 1
},
"SVO": {
"skill_version": 1,
"template_version": 1
},
"SVR": {
"skill_version": 1,
"template_version": 1
},
"SWI": {
"skill_version": 1,
"template_version": 1
},
"SWT": {
"skill_version": 1,
"template_version": 1
},
"TAN": {
"skill_version": 1,
"template_version": 1
},
"TBK": {
"skill_version": 1,
"template_version": 1
},
"TFR": {
"skill_version": 1,
"template_version": 1
},
"TFT": {
"skill_version": 1,
"template_version": 1
},
"TPD": {
"skill_version": 1,
"template_version": 1
},
"TVS": {
"skill_version": 1,
"template_version": 1
},
"USB": {
"skill_version": 1,
"template_version": 1
},
"VAR": {
"skill_version": 1,
"template_version": 1
},
"VRF": {
"skill_version": 1,
"template_version": 1
},
"WWR": {
"skill_version": 1,
"template_version": 1
},
"XCV": {
"skill_version": 1,
"template_version": 1
},
"XFM": {
"skill_version": 1,
"template_version": 1
},
"XTL": {
"skill_version": 1,
"template_version": 1
},
"ZEN": {
"skill_version": 1,
"template_version": 1
}
}

15
config/gitea.env
View File

@ -1,10 +1,13 @@
# Gitea connection for the datasheet-extractor skill.
# Gitea connection for the library-manager skill.
# WARNING: contains a secret (GIT_TOKEN) in plaintext - keep this skill PRIVATE.
# Use a token scoped to these repos (repository: write) and rotate periodically.
GIT_HOST=gitea.vecmocon.com
GIT_USER=nitishKumar
GIT_TOKEN=
# Target repos (flat layout, no project/version grouping):
DFS_REPO=nitishKumar/DFS
PARAMS_REPO=nitishKumar/Parameters
SKILL_ASSETS_REPO=nitishKumar/Skill_Assets
GIT_TOKEN=451bff1dc32202cbc0a371f8e5645079466d2120
# Target repos — TWO now (set these to the exact repo names on your Gitea):
# SKILL_REPO : holds this skill's own files (SKILL.md, scripts, assets, ...).
# COMPONENTS_REPO : holds components, one folder per Class (Diode, IC, ...); inside each,
# one MPN_make_typeid/ folder per part with { xlsx, datasheet, symbol,
# footprint }. (Replaces the old DFS + Parameters repos.)
SKILL_REPO=nitishKumar/Skill_Assets
COMPONENTS_REPO=nitishKumar/Components

5
references/taxonomy.md
View File

@ -1,7 +1,10 @@
# Component taxonomy — Class → Subclass (Type) → Type ID
Use the **Type ID** as the `typeid` in `MPN_make_typeid`. Match each part to the closest
subclass. The output sheet name is the **Class** (template.xlsx sheet). Full source:
subclass. In the current template **each Type ID is its own sheet** (125 sheets), so the
template sheet name **is the Type ID** (e.g. `SCH`, `MOS`, `LDO`). The **Class** is used only
to organise the components repo into top-level folders (e.g. `SCH` lives under `Diode/`);
`scripts/common.py:class_folder()` maps a typeid to its folder name. Full source:
`assets/template/Type_ID.xlsx`.

BIN
scripts/__pycache__/common.cpython-311.pyc Normal file
View File

Binary file not shown.

169
scripts/altium_refs.py Normal file
View File

@ -0,0 +1,169 @@
#!/usr/bin/env python3
"""Read the internal *reference name* out of Altium symbol (.SchLib) and footprint (.PcbLib)
files, and assemble the four design-column values for a part.
Why this exists: the value that belongs in the sheet's **Library Ref** / **Footprint Ref**
column is NOT the file name or the MPN - it is the component/footprint name as Altium stores
it *inside* the library (what an engineer sees in the file's properties). We read it directly
so no one has to open Altium by hand.
Verified against real Vecmocon files (Ultra Librarian export, e.g. CGA3E3X7R1H474K080AE):
- .SchLib: the component is a top-level OLE *storage* whose name is the Library Ref, e.g.
``CGA3E3X7R1H474K080AE``. (Confirmed also in the FileHeader as ``LibRef0=...``.)
- .PcbLib: each footprint is a top-level storage. An Ultra Librarian export ships the base
pattern plus IPC density variants, e.g. ``CAP_CGA3_TDK`` (nominal), ``CAP_CGA3_TDK-L``
(least/L), ``CAP_CGA3_TDK-M`` (most/M). The **base** pattern (the one the others suffix
with ``-L`` / ``-M`` / ``-N``) is the Footprint Ref -> ``CAP_CGA3_TDK``.
The four values that get written to the sheet:
Library Ref = name inside the .SchLib (read here)
Library Path = the .SchLib file name (just the basename)
Footprint Ref = base pattern inside the .PcbLib (read here)
Footprint Path = the .PcbLib file name (just the basename)
Usage:
# one file -> its reference name
python altium_refs.py CGA3E3X7R1H474K080AE.SchLib
python altium_refs.py CGA3E3X7R1H474K080AE.PcbLib --json
# both files -> ready-to-use design map for fill_templates --design
python altium_refs.py design --symbol X.SchLib --footprint Y.PcbLib > design.json
Programmatic: from altium_refs import extract_ref, build_design
If a ref comes back None or a footprint has several unrelated candidates, ask the engineer to
confirm the name shown in Altium's properties.
"""
import argparse, json, os, re
import olefile
# Entry names that are container metadata, never a component/footprint reference.
META = {
"fileheader", "additionalfileheader", "storage", "library", "root entry",
"componentparamstoc", "fileversioninfo", "sectionkeys", "data", "header",
"uniqueidprimitiveinformation", "pintextdata", "textframe", "models",
"modelsnoembed", "arcs6", "pads6", "vias6", "tracks6", "texts6", "fills6",
"regions6", "componentbodies6", "designatoroverride", "parameters",
"primitiveguids", "widestrings", "layerkindmapping", "padvialibrary",
"embeddedfonts", "textures", "fileversion",
}
def _decode(raw):
for enc in ("utf-16-le", "latin-1", "utf-8"):
try:
return raw.decode(enc, errors="ignore")
except Exception:
continue
return ""
def _storage_names(ole):
"""Distinct top-level storage names that aren't container metadata."""
names = []
for entry in ole.listdir(streams=True, storages=True):
top = entry[0]
if top.lower() in META or top in names:
continue
names.append(top)
return names
def _from_header_records(ole):
"""Backstop: scan header streams for LibReference / PatternName records."""
hits = []
for entry in ole.listdir(streams=True):
if entry[-1].lower() not in ("fileheader", "header", "data"):
continue
try:
txt = _decode(ole.openstream(entry).read())
except Exception:
continue
for key in ("LIBREFERENCE", "PATTERNNAME", "PATTERN"):
for m in re.finditer(rf"{key}=([^\x00|]+)", txt, re.IGNORECASE):
val = m.group(1).strip()
if val and val not in hits:
hits.append(val)
return hits
def _pick_base(candidates):
"""Choose the primary reference. When several names share a base and differ only by a
trailing density suffix (Altium IPC variants: -L / -M / -N / -Least / -Most / -Nominal),
the base (a prefix of the others) is the real one. Otherwise take the first."""
if not candidates:
return None
bases = [c for c in candidates
if any(o != c and o.startswith(c + "-") for o in candidates)]
if bases:
return sorted(bases, key=len)[0]
# no prefix relationship: drop obvious -L/-M/-N variants if a suffix-less peer exists
plain = [c for c in candidates if not re.search(r"-(L|M|N|Least|Most|Nominal)$", c)]
return (plain or candidates)[0]
def extract_ref(path):
"""Return (ref, candidates): ref = best single reference name (base pattern) or None."""
if not olefile.isOleFile(path):
return None, []
ole = olefile.OleFileIO(path)
try:
candidates = _storage_names(ole) or _from_header_records(ole)
finally:
ole.close()
return _pick_base(candidates), candidates
def kind_of(path):
ext = os.path.splitext(path)[1].lower()
return {".schlib": "symbol", ".pcblib": "footprint"}.get(ext, "unknown")
def build_design(symbol_path, footprint_path):
"""Assemble the four design-column values from the two uploaded files."""
lib_ref, lib_cands = extract_ref(symbol_path)
fp_ref, fp_cands = extract_ref(footprint_path)
return {
"Library Ref": lib_ref,
"Library Path": os.path.basename(symbol_path),
"Footprint Ref": fp_ref,
"Footprint Path": os.path.basename(footprint_path),
"_candidates": {"symbol": lib_cands, "footprint": fp_cands},
}
def main():
ap = argparse.ArgumentParser()
sub = ap.add_subparsers(dest="cmd")
d = sub.add_parser("design", help="emit the 4-field design map from both files")
d.add_argument("--symbol", required=True)
d.add_argument("--footprint", required=True)
ap.add_argument("file", nargs="?", help=".SchLib or .PcbLib (single-file mode)")
ap.add_argument("--json", action="store_true")
a = ap.parse_args()
if a.cmd == "design":
design = build_design(a.symbol, a.footprint)
warn = [k for k in ("Library Ref", "Footprint Ref") if not design[k]]
print(json.dumps(design, ensure_ascii=False, indent=2))
if warn:
print(f"# WARNING: could not read {', '.join(warn)} - confirm from Altium", flush=True)
return
if not a.file:
ap.error("give a file, or use: design --symbol X.SchLib --footprint Y.PcbLib")
ref, cands = extract_ref(a.file)
if a.json:
print(json.dumps({"kind": kind_of(a.file), "ref": ref, "candidates": cands}, ensure_ascii=False))
return
if ref is None:
print(f"Could not read a reference from {a.file}. Open it in Altium and use the "
"name shown in properties.")
else:
print(ref)
if len(cands) > 1:
print(f"(variants present: {cands}; using base '{ref}')")
if __name__ == "__main__":
main()

120
scripts/append_parameter.py
View File

@ -1,27 +1,99 @@
#!/usr/bin/env python3
"""Append a new parameter (column) to the end of a type's sheet in the master
template, and bump the template VERSION. After running, push the updated
assets/template/template.xlsx + VERSION to the Skill_Assets repo.
"""Add one or more new parameters to a single typeid's template sheet, bump that typeid's
versions, and record the change in the global changelog.
Usage: python append_parameter.py --type Diode --param "Reverse Recovery Time(ns)" \
--template assets/template/template.xlsx
Versioning is **per typeid**: adding a parameter to (say) SCH means there's a new SCH
template, so SCH's template_version bumps v1->v2, and on the back of that its skill_version
bumps v1->v2 too. Only SCH moves; every other typeid keeps its versions. Those two numbers
are what fill_templates later stamps into the SCH sheet's rows (columns B and C).
The new column is appended at the END of the sheet (as requested), styled like the other
headers. Then versions.json is bumped and one entry is written to CHANGELOG.md.
python append_parameter.py --typeid SCH \
--param "Reverse Recovery Time(ns)" --param "Diode Capacitance(pF)" \
--desc "Added Trr and Cd for SMPS rectifier derating"
After running, push the updated template.xlsx + versions.json + CHANGELOG.md to the SKILL
repo (and, per the workflow, the same change flows to the components repo copies).
"""
import argparse, os, openpyxl
ap=argparse.ArgumentParser()
ap.add_argument("--type", required=True); ap.add_argument("--param", required=True)
ap.add_argument("--template", required=True)
a=ap.parse_args()
wb=openpyxl.load_workbook(a.template)
if a.type not in wb.sheetnames:
raise SystemExit(f"no sheet '{a.type}' in template; sheets: {wb.sheetnames}")
ws=wb[a.type]
headers=[ws.cell(1,c).value for c in range(1, ws.max_column+1)]
if a.param in headers:
raise SystemExit(f"'{a.param}' already exists in {a.type}")
ws.cell(1, ws.max_column+1, a.param) # append at the end
wb.save(a.template)
vf=os.path.join(os.path.dirname(a.template), "VERSION")
v=int(open(vf).read().strip()) if os.path.exists(vf) else 1
v+=1; open(vf,"w").write(str(v))
print(f"Appended '{a.param}' to {a.type}. Template is now v{v}. "
f"Push {os.path.basename(a.template)} + VERSION to the Skill_Assets repo.")
import argparse, datetime, os
import openpyxl
from copy import copy
from common import (TEMPLATE_XLSX, CHANGELOG_MD, bump_versions, load_taxonomy)
def _style_header(new_cell, ref_cell):
"""Make an appended header look like the sheet's existing headers."""
for attr in ("font", "fill", "border", "alignment", "number_format"):
try:
setattr(new_cell, attr, copy(getattr(ref_cell, attr)))
except Exception:
pass
def append_params(template_path, typeid, params):
wb = openpyxl.load_workbook(template_path)
if typeid not in wb.sheetnames:
raise SystemExit(f"no sheet '{typeid}' in template; is it a valid typeid?")
ws = wb[typeid]
headers = [ws.cell(1, c).value for c in range(1, ws.max_column + 1)]
added = []
for p in params:
if p in headers:
print(f" ! '{p}' already exists in {typeid} - skipping")
continue
col = ws.max_column + 1
cell = ws.cell(1, col, p)
_style_header(cell, ws.cell(1, 1))
headers.append(p)
added.append(p)
if not added:
raise SystemExit("nothing to add (all parameters already present)")
wb.save(template_path)
return added
def write_changelog(typeid, added, old, new, desc, when=None):
when = when or datetime.datetime.now()
folder = (load_taxonomy().get(typeid) or {}).get("folder", "?")
lines = [
f"## {when.strftime('%Y-%m-%d %H:%M')} - {typeid} ({folder})",
f"- Template v{old['template_version']} -> v{new['template_version']}, "
f"Skill v{old['skill_version']} -> v{new['skill_version']}",
"- Added parameter(s): " + ", ".join(f'\"{p}\"' for p in added),
]
if desc:
lines.append(f"- Description: {desc}")
entry = "\n".join(lines) + "\n\n"
header = "# library-manager - template/version changelog\n\n"
prior = ""
if os.path.exists(CHANGELOG_MD):
prior = open(CHANGELOG_MD, encoding="utf-8").read()
if prior.startswith(header):
prior = prior[len(header):]
open(CHANGELOG_MD, "w", encoding="utf-8").write(header + entry + prior) # newest on top
return CHANGELOG_MD
def main():
ap = argparse.ArgumentParser()
ap.add_argument("--typeid", required=True, help="e.g. SCH, MOS, LDO")
ap.add_argument("--param", required=True, action="append",
help="new column header; repeat for several")
ap.add_argument("--desc", default="", help="what/why - goes in the changelog")
ap.add_argument("--template", default=TEMPLATE_XLSX)
a = ap.parse_args()
added = append_params(a.template, a.typeid, a.param)
old, new = bump_versions(a.typeid)
cl = write_changelog(a.typeid, added, old, new, a.desc)
print(f"Added to {a.typeid}: {', '.join(added)}")
print(f"{a.typeid}: template v{old['template_version']}->v{new['template_version']}, "
f"skill v{old['skill_version']}->v{new['skill_version']}")
print(f"Changelog updated: {cl}")
print("Now push template.xlsx + versions.json + CHANGELOG.md to the SKILL repo.")
if __name__ == "__main__":
main()

163
scripts/common.py Normal file
View File

@ -0,0 +1,163 @@
#!/usr/bin/env python3
"""Shared helpers for the library-manager skill.
One place for the three things every script needs to agree on:
- the taxonomy (typeid -> Class -> components-repo folder name), read from
``assets/template/Type_ID.xlsx`` so there is a single source of truth;
- the **per-typeid version store** (``assets/template/versions.json``) that holds each
typeid's own ``template_version`` and ``skill_version`` - both start at 1 and both bump
together the moment a parameter is added to that typeid (and nothing else moves);
- the part **tag** ``<MPN>_<make>_<typeid>`` that names every folder and column-A cell.
Keeping these here means fill_templates / append_parameter / gitea_components can't drift
apart on how a version is read or a folder is named.
"""
import json, os, re
import openpyxl
SKILL_ROOT = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
TEMPLATE_DIR = os.path.join(SKILL_ROOT, "assets", "template")
TEMPLATE_XLSX= os.path.join(TEMPLATE_DIR, "template.xlsx")
TYPE_ID_XLSX = os.path.join(TEMPLATE_DIR, "Type_ID.xlsx")
VERSIONS_JSON= os.path.join(TEMPLATE_DIR, "versions.json")
CHANGELOG_MD = os.path.join(TEMPLATE_DIR, "CHANGELOG.md")
# Fixed template columns that live on every sheet and are NOT extracted from a datasheet.
COL_TAG = "MPN_make_type" # column A header (the part tag lives here)
COL_SKILL_VER = "Skill Version" # column B (per-typeid skill version, stamped by us)
COL_TEMPLATE_VER = "Template Version" # column C (per-typeid template version, stamped by us)
DESIGN_COLS = ("Library Ref", "Library Path", "Footprint Ref", "Footprint Path")
# Raw taxonomy Class name -> clean, filesystem-safe folder name for the components repo.
# These match Vecmocon's established sheet-name convention (the old 18-class template).
CLASS_FOLDER = {
"Resistor": "Resistor",
"Capacitor": "Capacitor",
"Inductor / Magnetics": "Inductor_Magnetics",
"Diode": "Diode",
"Transistor": "Transistor",
"Integrated Circuit (IC)": "IC",
"Protection Device": "Protection",
"Power Conversion Module": "Power Conversion Module",
"Relay / Contactor": "Relay_Contactor",
"Switch / Button": "Switch_Button",
"Connector": "Connector",
"Antenna / RF": "Antenna_RF",
"Crystal / Oscillator / Timing": "Crystal_Oscillator",
"Battery / Cell": "Battery_Cell",
"Audible / Indicator": "Audible_Indicator",
"Display / HMI": "Display_HMI",
"Sensor (discrete / module)": "Sensor",
"Thermal / Cooling": "Thermal_Cooling",
}
def norm(s):
return re.sub(r"[^a-z0-9]", "", str(s).lower())
# ---------------------------------------------------------------- taxonomy
def load_taxonomy(type_id_xlsx=TYPE_ID_XLSX):
"""Return {typeid: {"class": rawclass, "subclass": name, "folder": foldername}}.
Reads the Type_ID.xlsx. A real mapping row has all three of Class / Subclass / Type ID;
the group-header rows (only a Class label like 'Resistor (12 types)') are skipped
because they have no Type ID.
"""
wb = openpyxl.load_workbook(type_id_xlsx, read_only=True, data_only=True)
ws = wb[wb.sheetnames[0]]
out = {}
for row in ws.iter_rows(min_row=1, values_only=True):
cells = (list(row) + [None, None, None])[:3]
cls, sub, tid = (str(c).strip() if c is not None else "" for c in cells)
if not tid or tid.lower() in ("type id",):
continue
folder = CLASS_FOLDER.get(cls, re.sub(r"\s*/\s*", "_", re.sub(r"\s*\(.*?\)", "", cls)).strip().replace(" ", "_"))
out[tid] = {"class": cls, "subclass": sub, "folder": folder}
return out
def class_folder(typeid, taxonomy=None):
"""The components-repo top-level folder name for a typeid (e.g. SCH -> 'Diode')."""
tax = taxonomy or load_taxonomy()
info = tax.get(typeid)
return info["folder"] if info else None
# ---------------------------------------------------------------- part tag
def make_tag(manufacturer):
"""The 'make' token = first word of the manufacturer, stripped to alphanumerics."""
w = re.split(r"\s+", str(manufacturer).strip())
first = w[0] if w and w[0] else "NA"
return re.sub(r"[^A-Za-z0-9]", "", first) or "NA"
def part_tag(part):
"""Identity of a part: <MPN>_<make>_<typeid>. Same tag = same part."""
make = part.get("make") or make_tag(part.get("manufacturer", ""))
return f'{part.get("mpn","")}_{make}_{part.get("typeid","NA")}'
def mpn_make_prefix(mpn, make):
"""The <MPN>_<make>_ prefix used for the early duplicate check (typeid not yet known)."""
return f"{mpn}_{make}_"
# ---------------------------------------------------------------- version store
def _load_versions():
if os.path.exists(VERSIONS_JSON):
try:
return json.load(open(VERSIONS_JSON, encoding="utf-8"))
except Exception:
pass
return {}
def get_versions(typeid):
"""Return {'template_version': int, 'skill_version': int} for a typeid (default 1/1)."""
v = _load_versions().get(typeid) or {}
return {"template_version": int(v.get("template_version", 1)),
"skill_version": int(v.get("skill_version", 1))}
def version_labels(typeid):
"""Return ('vN', 'vN') = (skill_version, template_version) strings for stamping cells."""
v = get_versions(typeid)
return f'v{v["skill_version"]}', f'v{v["template_version"]}'
def bump_versions(typeid):
"""Increment BOTH template_version and skill_version for one typeid; persist and return
(old, new) each as {'template_version','skill_version'}. Only this typeid changes."""
data = _load_versions()
cur = data.get(typeid) or {"template_version": 1, "skill_version": 1}
old = {"template_version": int(cur.get("template_version", 1)),
"skill_version": int(cur.get("skill_version", 1))}
new = {"template_version": old["template_version"] + 1,
"skill_version": old["skill_version"] + 1}
data[typeid] = new
json.dump(data, open(VERSIONS_JSON, "w", encoding="utf-8"), indent=2, sort_keys=True)
return old, new
def init_versions(force=False):
"""Seed versions.json with 1/1 for every typeid in the taxonomy (idempotent)."""
data = {} if force else _load_versions()
for tid in load_taxonomy():
data.setdefault(tid, {"template_version": 1, "skill_version": 1})
json.dump(data, open(VERSIONS_JSON, "w", encoding="utf-8"), indent=2, sort_keys=True)
return data
if __name__ == "__main__":
import sys
if len(sys.argv) > 1 and sys.argv[1] == "init":
d = init_versions(force="--force" in sys.argv)
print(f"versions.json seeded with {len(d)} typeids at {VERSIONS_JSON}")
else:
tax = load_taxonomy()
print(f"taxonomy: {len(tax)} typeids, {len(set(v['folder'] for v in tax.values()))} class folders")

214
scripts/fill_templates.py
View File

@ -1,143 +1,105 @@
#!/usr/bin/env python3
"""Fill the reference template into ONE master workbook (Components_Master.xlsx).
"""Build ONE per-part Excel sheet from the master template.
The master contains ONLY the type sheets that actually have parts (the components whose
datasheets were provided) - built from a copy of the template so each kept sheet keeps its
exact headers, styling, widths and freeze - plus a single **Meta** sheet appended at the
END with: Template Version, Total Components, Date, Time, and a per-sheet count breakdown.
Empty template sheets are dropped. Column A = <MPN>_<make>_<typeid>.
The new Vecmocon layout stores each component on its own: a folder ``MPN_make_typeid/``
holding that part's own single-sheet workbook (plus its datasheet, symbol and footprint).
There is no accumulating master workbook any more - so this script takes one part and writes
``<MPN>_<make>_<typeid>.xlsx``, a copy of just that typeid's template sheet (headers, styling,
widths, freeze preserved) with a single filled data row.
Shared helpers (part_tag, part_to_row, template_headers, sheet_rows, read_all_rows,
build_master, resolve_version, MASTER_NAME) are imported by gitea_reconcile.py so the Gitea
master is built and merged the exact same way.
What gets stamped automatically (never read from a datasheet):
- column A ``MPN_make_type`` = the part tag ``<MPN>_<make>_<typeid>``
- column B ``Skill Version`` = this typeid's skill version (vN, from versions.json)
- column C ``Template Version``= this typeid's template version (vN, from versions.json)
python fill_templates.py parts.json --template <template.xlsx> --dest <dir> [--version v1]
The four design columns (Library Ref/Path, Footprint Ref/Path) are filled from a --design
map when the symbol/footprint step provides them, and left blank otherwise. Everything else
comes from the extracted ``values``.
# one part, values only (design columns left blank for now)
python fill_templates.py part.json --template assets/template/template.xlsx --dest out/
# later, same part with the Altium-derived design fields merged in
python fill_templates.py part.json --template ... --dest out/ --design design.json
``part.json`` is either a single part object or {"parts": [ ... ]} (each part built to its
own file). A part = {"mpn","manufacturer" (or "make"),"typeid","values":{header:value,...}}.
"""
import argparse, json, os, re, datetime
import argparse, json, os
import openpyxl
from openpyxl.styles import Font, PatternFill, Border, Side, Alignment
from openpyxl.utils import get_column_letter
from common import (norm, part_tag, version_labels,
COL_TAG, COL_SKILL_VER, COL_TEMPLATE_VER, DESIGN_COLS)
GREEN="B6D7A8"; GRAY="BFBFBF"
MASTER_NAME="Components_Master.xlsx"; META_SHEET="Meta"
def norm(s): return re.sub(r'[^a-z0-9]', '', str(s).lower())
ALIAS={"induactanceuh":"inductanceuh"} # template header typo -> value key
def make_tag(manufacturer):
w=re.split(r'\s+', str(manufacturer).strip()); first=w[0] if w and w[0] else "NA"
return re.sub(r'[^A-Za-z0-9]', '', first) or "NA"
def part_tag(part):
"""Identity of a part: <MPN>_<make>_<typeid>. Same tag = same part."""
make=part.get("make") or make_tag(part.get("manufacturer",""))
return f'{part.get("mpn","")}_{make}_{part.get("typeid","NA")}'
def template_headers(template_path, typeid):
"""Canonical headers for a typeid sheet, or None if the template has no such sheet."""
wb = openpyxl.load_workbook(template_path, read_only=True)
if typeid not in wb.sheetnames:
return None
ws = wb[typeid]
return [ws.cell(1, c).value for c in range(1, ws.max_column + 1)]
def template_headers(template_path, ctype):
"""Canonical data headers for a type sheet, or None if the template has no such sheet."""
wb=openpyxl.load_workbook(template_path)
if ctype not in wb.sheetnames: return None
ws=wb[ctype]; return [ws.cell(1,c).value for c in range(1, ws.max_column+1)]
def template_types(template_path):
return [s for s in openpyxl.load_workbook(template_path).sheetnames if s!=META_SHEET]
def part_to_row(part, headers, design=None):
"""Map one part to {header: value}, following the fixed-column rules above."""
vals = {norm(k): v for k, v in part.get("values", {}).items()}
dmap = {norm(k): v for k, v in (design or {}).items()}
tag = part_tag(part)
sver, tver = version_labels(part.get("typeid", "NA"))
row = {}
for h in headers:
key = norm(h)
if key == norm(COL_TAG): row[h] = tag
elif key == norm(COL_SKILL_VER): row[h] = sver
elif key == norm(COL_TEMPLATE_VER): row[h] = tver
elif key == norm("Manufacturer"): row[h] = part.get("manufacturer", vals.get("manufacturer", ""))
elif h in DESIGN_COLS: row[h] = dmap.get(key, vals.get(key, ""))
else: row[h] = vals.get(key, "")
return tag, row
def part_to_row(part, headers):
"""Return (tag, {header: value}) for one part, following the column rules."""
vals={norm(k):v for k,v in part.get("values",{}).items()}; tag=part_tag(part); row={}
for c,h in enumerate(headers, start=1):
key=ALIAS.get(norm(h),norm(h))
if c==1: row[h]=tag
elif key=="class": row[h]=part.get("subclass","")
elif key=="manufacturer":row[h]=part.get("manufacturer", vals.get("manufacturer",""))
else: row[h]=vals.get(key,"")
return tag,row
def sheet_rows(ws, width):
"""Existing data rows in a sheet -> (headers, {tag: {header: value}}), bounded to `width`."""
headers=[ws.cell(1,c).value for c in range(1,width+1)]; rows={}
for r in range(2, ws.max_row+1):
tag=ws.cell(r,1).value
if tag in (None,""): continue
rows[tag]={headers[c-1]: ws.cell(r,c).value for c in range(1,width+1)}
return headers, rows
def read_all_rows(master_path, template_path):
"""Read every type sheet of an existing master -> {ctype: {tag: rowdict}} (skips Meta)."""
wb=openpyxl.load_workbook(master_path); types=set(template_types(template_path)); out={}
for name in wb.sheetnames:
if name not in types: continue
th=template_headers(template_path,name); w=len(th) if th else 0
if not w: continue
_,rows=sheet_rows(wb[name], w)
if rows: out[name]=rows
return out
def _thin(): s=Side(style="thin",color=GRAY); return Border(left=s,right=s,top=s,bottom=s)
def _hdr(cell):
cell.font=Font(name="Calibri",bold=True); cell.fill=PatternFill("solid",fgColor=GREEN)
cell.border=_thin(); cell.alignment=Alignment(horizontal="center",vertical="center")
def _write_rows(ws, headers, rows_by_tag):
for r in range(2, ws.max_row+1):
for c in range(1, len(headers)+1): ws.cell(r,c).value=None
r=2
for tag,row in rows_by_tag.items():
for c,h in enumerate(headers, start=1): ws.cell(r,c, row.get(h,""))
r+=1
def _build_meta_sheet(wb, version, counts, when):
if META_SHEET in wb.sheetnames: del wb[META_SHEET]
ws=wb.create_sheet(META_SHEET) # appended at the end
for r,(k,v) in enumerate([("Template Version",version),
("Total Components",sum(counts.values())),
("Date",when.strftime("%Y-%m-%d")),
("Time",when.strftime("%H:%M:%S"))], start=1):
a=ws.cell(r,1,k); a.font=Font(name="Calibri",bold=True)
a.fill=PatternFill("solid",fgColor=GREEN); a.border=_thin()
ws.cell(r,2,v).border=_thin()
r=6
for c,t in ((1,"Sheet"),(2,"Components")): _hdr(ws.cell(r,c,t))
for ctype,n in counts.items():
r+=1; ws.cell(r,1,ctype).border=_thin(); ws.cell(r,2,n).border=_thin()
ws.column_dimensions["A"].width=20; ws.column_dimensions["B"].width=16
return ws
def resolve_version(template_path, version):
if version: return version
vf=os.path.join(os.path.dirname(template_path),"VERSION")
return "v"+open(vf).read().strip() if os.path.exists(vf) else "v1"
def build_master(template_path, rows_by_type, version, dest_dir):
"""Write the single master workbook from a copy of the template. `rows_by_type` =
{ctype: {tag: rowdict}} is the COMPLETE desired contents. Only sheets present (with at
least one row) are kept; all other template sheets are dropped. A Meta sheet is appended
last. Returns the output path."""
def build_part_sheet(part, template_path, dest_dir, design=None):
"""Write ``<tag>.xlsx`` = just this part's typeid sheet with one filled row. Returns
(path, tag). Reused for backfill: re-extract a part and call this again to regenerate its
sheet against the current template (new columns) and current stamped versions."""
typeid = part["typeid"]
headers = template_headers(template_path, typeid)
if headers is None:
raise SystemExit(f"no template sheet for typeid '{typeid}'")
wb = openpyxl.load_workbook(template_path)
for name in list(wb.sheetnames): # keep only this typeid's sheet
if name != typeid:
del wb[name]
ws = wb[typeid]
tag, row = part_to_row(part, headers, design)
for r in range(2, ws.max_row + 1): # clear any stray rows below the header
for c in range(1, len(headers) + 1):
ws.cell(r, c).value = None
for c, h in enumerate(headers, start=1):
ws.cell(2, c, row.get(h, ""))
os.makedirs(dest_dir, exist_ok=True)
wb=openpyxl.load_workbook(template_path); when=datetime.datetime.now(); counts={}
for ctype in template_types(template_path):
ws=wb[ctype]; rows=rows_by_type.get(ctype) or {}
if rows:
th=template_headers(template_path,ctype); width=len(th) if th else ws.max_column
_write_rows(ws, [ws.cell(1,c).value for c in range(1,width+1)], rows)
counts[ctype]=len(rows)
_build_meta_sheet(wb, version, counts, when) # append first so a sheet always remains
for ctype in template_types(template_path):
if ctype not in counts: del wb[ctype] # drop empty type sheets
out=os.path.join(dest_dir, MASTER_NAME); wb.save(out); return out
out = os.path.join(dest_dir, f"{tag}.xlsx")
wb.save(out)
return out, tag
def _iter_parts(data):
return data["parts"] if isinstance(data, dict) and "parts" in data else [data]
def main():
ap=argparse.ArgumentParser()
ap.add_argument("parts_json"); ap.add_argument("--template", required=True)
ap.add_argument("--dest", required=True); ap.add_argument("--version", default=None)
a=ap.parse_args()
data=json.load(open(a.parts_json, encoding="utf-8"))
version=resolve_version(a.template, a.version); by={}
for p in data.get("parts", []):
headers=template_headers(a.template, p["type"])
if headers is None: print(f"! no template sheet for type '{p['type']}' - skipping"); continue
tag,row=part_to_row(p, headers); by.setdefault(p["type"],{})[tag]=row
out=build_master(a.template, by, version, a.dest)
total=sum(len(v) for v in by.values())
print(f"master: {out} ({total} part(s) across {len(by)} sheet(s) + Meta, template {version})")
ap = argparse.ArgumentParser()
ap.add_argument("part_json")
ap.add_argument("--template", required=True)
ap.add_argument("--dest", required=True)
ap.add_argument("--design", default=None, help="JSON map of design columns to merge in")
a = ap.parse_args()
data = json.load(open(a.part_json, encoding="utf-8"))
design = json.load(open(a.design, encoding="utf-8")) if a.design else None
for p in _iter_parts(data):
out, tag = build_part_sheet(p, a.template, a.dest, design)
sver, tver = version_labels(p.get("typeid", "NA"))
print(f"wrote {out} (tag {tag}, skill {sver}, template {tver})")
if __name__=="__main__":
if __name__ == "__main__":
main()

227
scripts/gitea_components.py Normal file
View File

@ -0,0 +1,227 @@
#!/usr/bin/env python3
"""Talk to the two Gitea repos of the new library-manager layout.
Repos (from config/gitea.env):
SKILL_REPO - this skill's own files (pushed with push_to_gitea.sh, not here).
COMPONENTS_REPO - one folder per Class (Diode, IC, Transistor, ...); inside each Class,
one MPN_make_typeid/ folder per part holding { xlsx, datasheet, symbol,
footprint }.
Subcommands (all accept --local <dir> to run against a local folder instead of cloning, for
testing or offline work):
check-mpn --mpn BAT46WJ --make Nexperia
Is any part with this MPN+make already in the components repo? (typeid-agnostic - the
early duplicate gate, run BEFORE classifying.) Prints EXISTS/<tag> or ABSENT; exit 0
if absent, 3 if it already exists (so a shell can hard-stop).
checkout --dest work/
Clone the components repo to a working dir you can browse and edit in place.
list-type --typeid SCH [--root work/] [--json]
List every existing part of a typeid and the files in its folder (for backfill: the
datasheet to re-read is in there). Needs a checked-out --root (or --local).
place-part --folder staging/BAT46WJ_Nexperia_SCH --typeid SCH --root work/
Copy an assembled part folder into work/<Class>/<tag>/ (creates the Class folder if it
is missing). Then commit-push.
commit-push --root work/ [--message "..."]
Commit everything in the checked-out clone and push.
push-part --folder staging/BAT46WJ_Nexperia_SCH --typeid SCH [--message "..."]
Convenience: clone -> place-part -> commit-push in one go (the common single-part path).
Host unreachable (e.g. sandbox without the domain allowlisted) -> clones fail clearly and
nothing is written.
"""
import argparse, os, re, shutil, subprocess, sys, tempfile
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
from common import class_folder, load_taxonomy, mpn_make_prefix
CFG_DEFAULT = os.path.join(os.path.dirname(__file__), "..", "config", "gitea.env")
# ------------------------------------------------------------------ gitea plumbing
def load_env(cfg):
env = {}
if os.path.exists(cfg):
for line in open(cfg):
line = line.strip()
if not line or line.startswith("#") or "=" not in line:
continue
k, v = line.split("=", 1)
env[k.strip()] = v.strip()
for k in ("GIT_HOST", "GIT_USER", "GIT_TOKEN", "SKILL_REPO", "COMPONENTS_REPO"):
if os.environ.get(k):
env[k] = os.environ[k]
return env
def repo_url(env, repo):
host = re.sub(r"^https?://", "", env["GIT_HOST"]).rstrip("/")
cred = (env.get("GIT_USER", "") + ":" if env.get("GIT_USER") else "") + env["GIT_TOKEN"]
return f"https://{cred}@{host}/{repo}.git"
def clone(env, dest):
url = repo_url(env, env["COMPONENTS_REPO"])
r = subprocess.run(["git", "clone", url, dest], capture_output=True, text=True)
if r.returncode != 0:
err = (r.stderr or "").replace(env["GIT_TOKEN"], "***")
sys.exit(f"clone COMPONENTS failed (host reachable / token scope / repo exists?):\n{err[:400]}")
return dest
def commit_push(root, token, msg):
for args in (["config", "user.email", "datasheet-bot@local"],
["config", "user.name", "library-manager"],
["checkout", "-B", "main"], ["add", "-A"]):
subprocess.run(["git", "-C", root] + args, capture_output=True)
if subprocess.run(["git", "-C", root, "diff", "--cached", "--quiet"]).returncode == 0:
print("nothing new to push.")
return
subprocess.run(["git", "-C", root, "commit", "-m", msg], capture_output=True)
r = subprocess.run(["git", "-C", root, "push", "-u", "origin", "main"],
capture_output=True, text=True)
print((r.stdout + r.stderr).replace(token or "", "***").strip() or "pushed.")
def _root(env, args):
"""Return (root_dir, is_temp_clone). Honour --local / --root, else clone."""
if getattr(args, "local", None):
return args.local, False
if getattr(args, "root", None):
return args.root, False
tmp = tempfile.mkdtemp(prefix="components_")
return clone(env, tmp), True
# ------------------------------------------------------------------ folder logic
def find_part_dirs(root, prefix="", typeid=None):
"""Yield (class_folder, part_folder_name, abspath) for part folders under any Class dir
that match a name prefix and/or a typeid suffix."""
if not os.path.isdir(root):
return
for cls in sorted(os.listdir(root)):
cdir = os.path.join(root, cls)
if not os.path.isdir(cdir) or cls.startswith("."):
continue
for name in sorted(os.listdir(cdir)):
pdir = os.path.join(cdir, name)
if not os.path.isdir(pdir):
continue
if prefix and not name.startswith(prefix):
continue
if typeid and not name.endswith(f"_{typeid}"):
continue
yield cls, name, pdir
def cmd_check_mpn(env, args):
root, _ = _root(env, args)
prefix = mpn_make_prefix(args.mpn, args.make)
matches = [(cls, name) for cls, name, _ in find_part_dirs(root, prefix=prefix)]
if matches:
for cls, name in matches:
print(f"EXISTS\t{cls}/{name}")
sys.exit(3)
print("ABSENT")
def cmd_checkout(env, args):
dest = clone(env, args.dest)
print(f"components repo checked out at {dest}")
def cmd_list_type(env, args):
root, _ = _root(env, args)
parts = list(find_part_dirs(root, typeid=args.typeid))
if args.json:
import json
out = [{"class": cls, "tag": name,
"folder": pdir,
"files": sorted(os.listdir(pdir))} for cls, name, pdir in parts]
print(json.dumps(out, indent=2))
return
if not parts:
print(f"no existing parts of typeid {args.typeid}")
return
for cls, name, pdir in parts:
files = ", ".join(sorted(os.listdir(pdir)))
print(f"{cls}/{name} -> {files}")
def place_part(root, folder, typeid):
cls = class_folder(typeid)
if not cls:
sys.exit(f"unknown typeid '{typeid}' (not in taxonomy)")
tag = os.path.basename(os.path.normpath(folder))
cls_dir = os.path.join(root, cls)
# A repo may seed classes as 0-byte placeholder FILES (git can't store empty dirs).
# Replace such a placeholder with a real class directory before adding the part.
if os.path.exists(cls_dir) and not os.path.isdir(cls_dir):
os.remove(cls_dir)
os.makedirs(cls_dir, exist_ok=True)
dest = os.path.join(cls_dir, tag)
if os.path.isdir(dest):
shutil.rmtree(dest)
shutil.copytree(folder, dest)
return cls, tag, dest
def cmd_place_part(env, args):
root, _ = _root(env, args)
cls, tag, dest = place_part(root, args.folder, args.typeid)
print(f"placed {tag} -> {cls}/{tag}")
def cmd_commit_push(env, args):
commit_push(args.root, env.get("GIT_TOKEN", ""), args.message or "library-manager: update components")
def cmd_push_part(env, args):
tmp = tempfile.mkdtemp(prefix="components_")
clone(env, tmp)
cls, tag, _ = place_part(tmp, args.folder, args.typeid)
print(f"placed {tag} -> {cls}/{tag}")
commit_push(tmp, env.get("GIT_TOKEN", ""), args.message or f"library-manager: add {tag}")
def main():
ap = argparse.ArgumentParser()
ap.add_argument("--config", default=CFG_DEFAULT)
sub = ap.add_subparsers(dest="cmd", required=True)
p = sub.add_parser("check-mpn"); p.add_argument("--mpn", required=True)
p.add_argument("--make", required=True); p.add_argument("--local"); p.add_argument("--root")
p = sub.add_parser("checkout"); p.add_argument("--dest", required=True)
p = sub.add_parser("list-type"); p.add_argument("--typeid", required=True)
p.add_argument("--root"); p.add_argument("--local"); p.add_argument("--json", action="store_true")
p = sub.add_parser("place-part"); p.add_argument("--folder", required=True)
p.add_argument("--typeid", required=True); p.add_argument("--root"); p.add_argument("--local")
p = sub.add_parser("commit-push"); p.add_argument("--root", required=True)
p.add_argument("--message")
p = sub.add_parser("push-part"); p.add_argument("--folder", required=True)
p.add_argument("--typeid", required=True); p.add_argument("--message")
args = ap.parse_args()
env = load_env(args.config)
if args.cmd in ("check-mpn", "checkout", "commit-push", "push-part") or \
(args.cmd in ("list-type", "place-part") and not getattr(args, "local", None) and not getattr(args, "root", None)):
for k in ("GIT_HOST", "GIT_TOKEN", "COMPONENTS_REPO"):
if not env.get(k):
sys.exit(f"missing {k} in env/config")
{"check-mpn": cmd_check_mpn, "checkout": cmd_checkout, "list-type": cmd_list_type,
"place-part": cmd_place_part, "commit-push": cmd_commit_push, "push-part": cmd_push_part}[args.cmd](env, args)
if __name__ == "__main__":
main()

4
scripts/push_to_gitea.sh
View File

@ -20,7 +20,7 @@ while [[ $# -gt 0 ]]; do case "$1" in
*) echo "unknown arg: $1" >&2; exit 2;; esac; done
[[ -z "$REPO" || -z "$SRC" || -z "$HOST" || -z "$TOKEN" ]] && { echo "need --repo, --src, and GIT_HOST/GIT_TOKEN" >&2; exit 2; }
[[ -d "$SRC" ]] || { echo "ERROR: src not found: $SRC" >&2; exit 2; }
[[ -z "$MSG" ]] && MSG="Update from datasheet-extractor"
[[ -z "$MSG" ]] && MSG="Update from library-manager"
HOST="${HOST#http://}"; HOST="${HOST#https://}"; HOST="${HOST%/}"
CRED="${USER_:+$USER_:}${TOKEN}"
URL="https://${CRED}@${HOST}/${REPO}.git"
@ -30,7 +30,7 @@ git clone "$URL" "$WORK" 2>&1 | scrub || { echo "ERROR: clone failed (repo must
cd "$WORK"; git checkout -B main >/dev/null 2>&1 || true
DEST="$WORK${SUBDIR:+/$SUBDIR}"; mkdir -p "$DEST"
cp -r "$SRC/." "$DEST/"
git config user.email "datasheet-bot@local"; git config user.name "datasheet-extractor"
git config user.email "datasheet-bot@local"; git config user.name "library-manager"
git add -A
if git diff --cached --quiet; then echo "Nothing new for $REPO."; else
git commit -m "$MSG" >/dev/null; git push -u origin main 2>&1 | scrub