nitishKumar
/
Skill
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

ESD: add Peak Power(W) param (template v2, skill v2)

main
library-manager 2026-07-10 07:57:00 +00:00
parent 1537551e36
commit 9c7684f1b2
No known key found for this signature in database
13 changed files with 1479 additions and 370 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

411
SKILL.md
View File

@ -1,270 +1,271 @@
--- ---
name: datasheet-extractor name: library-manager
description: >- description: >-
Extract key parameters from component datasheet PDFs into the Vecmocon master Manage Vecmocon's component library: extract parameters from a component datasheet PDF into
template (18 component types incl. resistors, capacitors, inductors, diodes, the per-typeid Excel template (125 type-IDs across 18 classes - resistors, capacitors,
transistors, ICs, protection, power modules, connectors, sensors, and more). diodes, transistors, ICs, connectors, sensors, etc.). The user gives a datasheet named
Column A = MPN_make_typeid (make = first word of manufacturer, typeid from the as its MPN plus the make. The skill checks Gitea for a duplicate MPN_make and hard-stops if
taxonomy). Writes ONE master workbook (Components_Master.xlsx) with sheets only for the present, classifies the part to a typeid, confirms or updates that typeid's template
types whose datasheets were provided, plus a Meta sheet at the end (template version, (per-typeid version, changelog, backfill), fills a per-part workbook, loops on human
total components, date, time), assembles verification, then takes the Altium symbol (.SchLib) and footprint (.PcbLib), fills the
a per-part DFS folder (datasheet, Altium footprint, Altium symbol), and pushes Library/Footprint columns, assembles a part folder (xlsx, datasheet, symbol, footprint) and
design files to the Gitea DFS repo, Excel outputs to the Parameters repo, and skill pushes it to the components repo under its Class. Use WHENEVER the user uploads a component
files plus templates to the Skill_Assets repo. Also appends new parameters to a datasheet, builds a library entry, adds a parameter to a type template, or pushes a part to
template as a new version. Use WHENEVER the user uploads component datasheets and Gitea. ALWAYS trigger on "\datasheet", "\library", or "\library-manager", or any
asks to extract parameters, build the parameter sheets or component library, or push component-library / datasheet-extraction task.
datasheets to Gitea. ALWAYS trigger when the user types "\datasheet".
--- ---
# Datasheet Extractor # Library Manager
Read component datasheets, fill the customer's master template, assemble each part's Turn one component datasheet into a verified, versioned library entry in Gitea. The guiding
design-file folder, and publish everything to three Gitea repos. Be careful and honest: idea is honesty and traceability: every value lands in the right column and unit, anything
put every value in the right column and unit, and flag what a datasheet does not state. the datasheet doesn't state stays blank, and nothing reaches Gitea until a human has
confirmed it. This is an **interactive** skill — it asks at the few points where a person's
**Pushing is the default.** At the end of a run, push every output to Gitea without asking judgement or an upload is genuinely needed, and does everything else on its own.
for confirmation — that's the whole job. Only pause when there's a genuine need for the
user: a **login wall** while fetching a footprint/symbol, or an **MPN that already exists**
in Gitea (discard vs replace, because replace overwrites data). Nothing else warrants a
prompt.
## Inputs ## Inputs
- **Datasheet PDFs**, one per part. The filename is the part's MPN (search that exact - **A datasheet PDF whose filename is the MPN** (e.g. `BAT46WJ.pdf`). If it's a series
number inside a series datasheet to read the right variant). datasheet, search that exact MPN inside to read the correct variant.
- No project/version is needed — repos use a **flat layout**. - **The make** (manufacturer), given by the user. The `make` tag is the first word of the
manufacturer, alphanumerics only (Texas Instruments → `Texas`, Nexperia → `Nexperia`).
- **Later in the flow**, after verification: an Altium **symbol** (`.SchLib`) and
**footprint** (`.PcbLib`) file, provided by the user.
## The identifier: `MPN_make_typeid` ## The identifier: `MPN_make_typeid`
Column A of every sheet (header `MPN_make_type`) and each DFS folder name is: Every part folder, every per-part workbook, and column A of every sheet (`MPN_make_type`)
use the same tag:
``` ```
<MPN>_<make>_<typeid> e.g. BAT46WJ_Nexperia_SCH <MPN>_<make>_<typeid> e.g. BAT46WJ_Nexperia_SCH
``` ```
- **make** = the **first word of the manufacturer** name (Texas Instruments → `Texas`, `typeid` is the part's type-ID code from the taxonomy (`references/taxonomy.md`, full source
Nexperia → `Nexperia`, STMicroelectronics → `STMicroelectronics`). `assets/template/Type_ID.xlsx`) — Schottky → `SCH`, MOSFET → `MOS`, LDO → `LDO`. In the new
- **typeid** = the Type ID for the part's subclass, from the taxonomy in template **each typeid is its own sheet** (125 of them). The broader **Class** (Diode,
`references/taxonomy.md` (full source `assets/template/Type_ID.xlsx`). e.g. Schottky → Transistor, IC …) is used only to organise the components repo into top-level folders.
`SCH`, TVS → `TVS`, MOSFET → `MOS`, LDO → `LDO`, Common-mode choke → `CMC`.
## Gitea layout (two repos)
```
skill repo/ this skill's own files (updated versions land here too)
components repo/
<Class>/ e.g. Diode, IC, Transistor, Resistor, ...
<MPN>_<make>_<typeid>/ e.g. BAT46WJ_Nexperia_SCH
<MPN>_<make>_<typeid>.xlsx this part's own one-row parameter sheet
<MPN>_data.pdf the datasheet
<symbol>.SchLib user-provided
<footprint>.PcbLib user-provided
```
There is **no single master workbook** — each part carries its own sheet inside its folder.
Connection + repo names live in `config/gitea.env` (`SKILL_REPO`, `COMPONENTS_REPO`), so runs
need no per-session token. If the host is unreachable, the git steps fail clearly and write
nothing.
## Workflow ## Workflow
1. **Read each datasheet.** Get MPN (from filename), manufacturer, and the parameters. Run these in order. Each `python`/`bash` command is a helper in `scripts/`.
2. **Classify** to one of the 18 **types** (= template sheet names) and to a **subclass**
from `references/taxonomy.md`; record its **typeid**. The subclass name goes in the
`Class` column where the sheet has one.
3. **Extract values** into that type's columns (headers come from
`assets/template/template.xlsx`). Convert to each header's unit; leave blank if the
datasheet doesn't state it (an honest blank beats a guess).
4. **Build the master workbook** with `scripts/fill_templates.py` — a single
`Components_Master.xlsx` that mirrors the template but keeps **only the sheets for the
types you actually extracted** (the components whose datasheets were given), each with
column A = `MPN_make_typeid`, and a single **Meta** sheet appended at the end. (See
Producing outputs.)
5. **Assemble each part's DFS folder** `MPN_make_typeid/` containing `MPN_data` (the
datasheet), `MPN_fp` (footprint), `MPN_sym` (symbol). See Footprint & symbol.
6. **Push everything to Gitea — automatically.** At the end of a run, push all outputs to
all three repos with no confirmation step: DFS folders → **DFS**, per-type sheets →
**Parameters** (merged in), skill files + templates → **Skill_Assets**. The only reasons
to pause are a login wall or an already-present MPN (discard vs replace); otherwise it
all just goes. See Pushing to Gitea.
## Producing outputs ### 1. Duplicate check first — before any real work
Collect what you extracted into `parts.json` (keys match template headers; `typeid` and The part's presence is keyed on **MPN + make** (typeid not known yet). If it already exists,
`subclass` from the taxonomy; `manufacturer` used for the make tag): stop; re-doing an existing part would only risk overwriting good data.
```bash
python scripts/gitea_components.py check-mpn --mpn <MPN> --make <make>
```
`EXISTS …` (exit 3) → **stop and tell the user the part is already present in Gitea. End
here.** `ABSENT` (exit 0) → continue.
### 2. Classify → typeid (and its Class)
Read the datasheet, identify the part, and match it to the closest subclass in
`references/taxonomy.md`; record its **typeid** (= the template sheet name). The **Class**
(for the components-repo folder) comes from the same taxonomy row —
`scripts/common.py:class_folder(typeid)` returns it (e.g. `SCH` → `Diode`).
### 3. Confirm the typeid's template (and add parameters if asked)
Check whether that typeid has a sheet in `assets/template/template.xlsx`.
- **No sheet for this typeid** → ask the user to upload the template sheet for it. Add it to
`assets/template/template.xlsx`, then push the updated skill files to the skill repo (see
*Pushing the skill repo*). Then continue.
- **Sheet exists** → print **all** of that sheet's parameters (its column headers) in the
chat and ask the user whether any new parameters should be added.
- **No** → go to step 4.
- **Yes** → collect the new parameter name(s), then:
```bash
python scripts/append_parameter.py --typeid <typeid> \
--param "New Parameter(unit)" [--param "Another(unit)"] \
--desc "why these were added"
```
This appends the column(s) at the end of that typeid's sheet, **bumps that typeid's
Template Version and Skill Version together** (v1→v2 — see *Per-typeid versioning*),
and writes one entry to the global changelog `assets/template/CHANGELOG.md`. Push the
updated `template.xlsx` + `versions.json` + `CHANGELOG.md` to the **skill** repo, and
make sure the components repo's copies stay consistent.
Then ask: **should this change apply to the parts of this typeid already in Gitea?**
- **No** → go to step 4 (only the current part gets the new column).
- **Yes → backfill** (see *Backfilling existing parts*), then tell the user the previous
sheets were updated, and go to step 4.
### 4. Extract and fill the per-part workbook
Read every parameter the datasheet actually states into that typeid's columns, converting to
each header's unit. **Leave blanks where the datasheet is silent — an honest blank beats a
guess.** Collect them into a small `part.json`:
```json ```json
{"parts":[ {"mpn":"BAT46WJ","manufacturer":"Nexperia","typeid":"SCH",
{"type":"Diode","mpn":"BAT46WJ","manufacturer":"Nexperia","typeid":"SCH","subclass":"Schottky", "values":{"Description":"100 V 250 mA Schottky, SOD323F","Forward Voltage(V)":"0.71",
"values":{"Description":"100 V 250 mA Schottky, SOD323F","Forward Voltage(V)":"0.71","Package":"SOD323F"}} "Reverse Voltage(V)":"100","Forward Current(A)":"0.25","Package":"SOD323F"}}
]}
``` ```
```bash ```bash
python scripts/fill_templates.py parts.json \ python scripts/fill_templates.py part.json \
--template assets/template/template.xlsx --dest <outputs-dir> --template assets/template/template.xlsx --dest <stage>/<tag>/
``` ```
Produces a **single** `Components_Master.xlsx`. It starts from a copy of the reference This writes `<tag>.xlsx` — just that typeid's sheet, one row — with column A = the tag,
template (so every kept sheet keeps its exact headers, styling, widths and freeze), fills **Skill Version (col B)** and **Template Version (col C)** stamped from this typeid's current
this run's parts into the matching sheet (column A = `MPN_make_typeid`), then **drops every versions, and the four design columns left blank for now.
type sheet with no parts** — so the file holds only the components whose datasheets were
provided. A single **Meta** sheet is appended **at the end** with: Template Version, Total
Components, Date, Time, and a per-sheet count breakdown. Deliver this one file to the user;
the Parameters repo stores the same master, accumulated across runs (see Pushing to Gitea).
Sheet naming follows the template (Diode, Resistor, …), not `Sheet1/Sheet2`. If the user ### 5. Human verification loop
supplies a new reference template, drop it in at `assets/template/template.xlsx` and the
master mirrors whatever sheets/headers it has.
## Footprint & symbol (for the DFS folder) Deliver the filled workbook to the user and ask them to verify it. If they report an error
or say it isn't right, **go back to step 4, re-read the datasheet more carefully, re-fill,
and hand it back.** Repeat until the user confirms it's verified. Nothing is pushed until
this passes — the engineer is the ground truth for the numbers.
For each part, the DFS folder needs `MPN_fp` (footprint) and `MPN_sym` (symbol) beside ### 6. Symbol + footprint → the design columns
`MPN_data` (the datasheet). **Deliver these as Altium files** — an Altium PCB footprint
(`.PcbLib`) and schematic symbol (`.SchLib`), or a single integrated library (`.IntLib`)
that carries both — because that's what the design team actually consumes. Get them in
this order:
1. **Auto-download via the user's pre-logged-in Chrome** (`scripts/fetch_cad.py`). The user Once verified, ask the user to upload the **symbol (`.SchLib`)** and **footprint
keeps signed-in CAD-site windows open; the script attaches to that running Chrome and (`.PcbLib`)** files. Copy them into the staging part folder **under their proper names** (so
reuses those live sessions — it never logs in, and no password is ever handled. One-time the Path columns match the files that actually get stored — strip any upload-staging prefix
setup: the user quits Chrome, relaunches it with `--remote-debugging-port=9222 the environment may have added), then read all four design values in one shot:
--user-data-dir="$HOME/cad-chrome"`, and signs into the sites. Then, per part:
```bash ```bash
python scripts/fetch_cad.py --mpn BAT46WJ --dest <DFS_folder> \ cp <uploaded_symbol> <stage>/<tag>/<symbol_name>.SchLib
--sites snapeda,ultralibrarian,componentsearchengine,digikey cp <uploaded_footprint> <stage>/<tag>/<footprint_name>.PcbLib
python scripts/altium_refs.py design \
--symbol <stage>/<tag>/<symbol_name>.SchLib \
--footprint <stage>/<tag>/<footprint_name>.PcbLib > design.json
``` ```
It tries each site in order, downloads the Altium model, and drops it into the DFS folder This produces (verified against real Ultra-Librarian exports):
named `MPN_fp` / `MPN_sym` (or `MPN_cad` for a bundle). **It never solves CAPTCHAs or
defeats bot-detection** — if a site shows a login wall, CAPTCHA, or bot-check, that
adapter stops and reports `manual` with the URL. Note: this runs on the *user's* machine
(where the browsers are), and some sites' terms restrict automated downloads — treat it
as a convenience with the manual fallback below.
2. **Manual fallback** (script returned `manual`/`notfound`, or no browser session): **share - **Library Ref** = the component name **inside** the `.SchLib` (e.g. `CGA3E3X7R1H474K080AE`)
the user the direct link** where the model lives and ask them to download it and send it - **Library Path** = the `.SchLib` file name
back; then attach it into the DFS folder with the agreed naming (`MPN_fp` / `MPN_sym`). - **Footprint Ref** = the **base** pattern inside the `.PcbLib`; Altium ships IPC density
Never type, store, or ask for their password, and never write credentials into this skill variants (`-L` / `-M` / `-N`) alongside the base, and the base is the one used (e.g.
— it's pushed to Skill_Assets, and a secret in a repo is a real leak (same reason the `CAP_CGA3_TDK`, not `CAP_CGA3_TDK-L`)
`GIT_TOKEN` is blanked before syncing). - **Footprint Path** = the `.PcbLib` file name
Whichever path, once you have a candidate **cross-check** its pad/pin count and key These Ref names come from *inside* the files and can differ from the MPN or filename. If a Ref
package dimensions against the datasheet's package drawing before trusting it — library comes back `null` (or a footprint shows several unrelated candidates), ask the user to confirm
models are sometimes wrong, or drawn for a different variant of the series. the name from Altium's properties and edit `design.json`. Then re-fill so the columns land:
3. **If no good model exists, generate the footprint/symbol yourself** as Altium files
from the datasheet's package drawing and pinout — but only when you can do it reliably
— then **ask the user to verify** it against the datasheet before it's trusted, since a
generated land pattern that's slightly off is worse than an obvious blank.
4. **If none of that is possible**, still create the folder with the datasheet and **tell
the user that MPN needs a manual footprint/symbol build**, leaving a clearly-named
placeholder or listing it so nothing silently goes missing.
Keep the datasheet's extension on `MPN_data` (e.g. `BAT46WJ_data.pdf`). Name the
footprint/symbol files `MPN_fp` / `MPN_sym` with the Altium extension the model uses
(`.PcbLib` / `.SchLib`, or a single `.IntLib` when the source bundles both).
## Pushing to Gitea
Connection + repos are pre-configured in `config/gitea.env` (host, user, token, and the
three repos), so runs need no per-session token. Layout is **flat** (no project/version).
**Every run ends by pushing all three repos — no "shall I push?" step.** The user has
already asked for this to be automatic; don't re-ask. The reconcile step self-guards
against the one risky case (overwriting an existing part), so a straight push is safe by
default.
The three repos split into two kinds, and they're pushed differently:
- **DFS** and **Parameters** are **MPN-indexed** — DFS has one `MPN_make_typeid/` folder
per part; Parameters holds a **single master workbook** (`Components_Master.xlsx`) whose
type sheets each have one row per part (keyed on column A). Because a re-run can collide
with a part that's already there, these go through `scripts/gitea_reconcile.py`, which is
dedup-aware and **merge-safe** (it never blows away parts it isn't touching). See
*Handling parts already in Gitea* below.
- **Skill_Assets** holds the skill itself + templates — it is **not** per-part, so there's
nothing MPN-level to reconcile. Push it with `scripts/push_to_gitea.sh` as before.
```bash ```bash
# Skill files + templates -> Skill_Assets (flat copy; no dedup needed) python scripts/fill_templates.py part.json \
bash scripts/push_to_gitea.sh --repo "$SKILL_ASSETS_REPO" --src <skill-dir> --message "Sync skill assets" --template assets/template/template.xlsx --dest <stage>/<tag>/ --design design.json
``` ```
The push script clones the repo, copies the source contents in (flat), commits, and ### 7. Assemble the part folder
pushes; the repo must already exist. If the host is unreachable (e.g. Cowork without the
domain allowlisted) it fails clearly and leaves the staged files for a manual push.
**Push the skill files organised** to Skill_Assets: keep the skill's own structure The staging folder `<tag>/` should now hold the four files: the per-part `<tag>.xlsx`, the
(`SKILL.md`, `scripts/`, `assets/`, `references/`, `config/`) — but do **not** push the datasheet (name it `<MPN>_data.<ext>`), the symbol, and the footprint.
real token. Before syncing skill assets, blank the `GIT_TOKEN` line in the copy you push,
or push everything except `config/gitea.env`.
> Why not push Parameters with the flat script? `fill_templates.py` writes a master ### 8. Push to the components repo, under the part's Class
> workbook holding **only this run's** rows. Copied flat over the repo, it would overwrite
> the stored master and delete every previously-stored part. `gitea_reconcile.py` opens the
> repo's master, merges this run's rows into the right sheets, and writes it back — so only
> the parts you decided on change, and the Meta sheet (totals, date, time) refreshes.
## Handling parts already in Gitea (discard vs replace)
A part's identity is its tag `MPN_make_typeid` (same tag = same part). A re-run that
brings in a *new* MPN is not a conflict — it just gets added, and you push without asking.
The prompt only exists for the case where an MPN **already lives in DFS/Parameters**,
because "replace" overwrites data that's already on the server.
The simplest path is a single push. It clones the repos, adds everything new, and stops
**only** if it hits an existing MPN it can't resolve on its own:
```bash ```bash
python scripts/gitea_reconcile.py --parts parts.json --dfs-src <dfs-stage> \ python scripts/gitea_components.py push-part --folder <stage>/<tag> --typeid <typeid>
--template assets/template/template.xlsx --push
``` ```
- **No existing MPNs** → it adds all the folders/rows and pushes. No prompt. Done. This places the folder at `components/<Class>/<tag>/` — creating the Class folder if it
- **An MPN already exists** → it refuses (listing which) so you can ask the user, per MPN, doesn't exist yet, or pushing into it if it does — and commits and pushes. Confirm to the
**discard** (keep what's in Gitea, drop the new copy) or **replace** (overwrite that MPN's user where it landed.
DFS folder *and* its Parameters row with the fresh extraction). Every other folder/row is
always left alone — the Parameters sheet is merged, never wholesale-overwritten.
Feed the user's answers back as a small JSON map (`tag → replace|discard`): ## Per-typeid versioning
Versioning is **per typeid**, not global. Each typeid carries its own `template_version` and
`skill_version` in `assets/template/versions.json` (both start at 1). When a parameter is
added to a typeid, that typeid gets a new template, so its `template_version` bumps — and on
the back of that its `skill_version` bumps too (v1→v2). **Only that typeid moves**; every
other typeid keeps its versions. Those two numbers are exactly what `fill_templates` stamps
into that typeid's rows (cols B and C), so a row always records the template/skill version it
was built against. `append_parameter.py` does the bump; `common.py` is the single source for
reading and writing these numbers.
## The changelog
`append_parameter.py` maintains one **global** changelog at `assets/template/CHANGELOG.md`
(newest entry on top), recording for each change: date, typeid + class, the template and
skill version transitions, the parameter(s) added, and the description. Push it to the skill
repo alongside `template.xlsx` and `versions.json` whenever it changes.
## Backfilling existing parts
When the user wants a newly-added parameter applied to parts of that typeid already in Gitea:
```bash ```bash
# decisions.json e.g. {"BAT46WJ_Nexperia_SCH": "replace", "1N4148_onsemi_SCH": "discard"} python scripts/gitea_components.py checkout --dest work/
python scripts/gitea_reconcile.py --parts parts.json --dfs-src <dfs-stage> \ python scripts/gitea_components.py list-type --typeid <typeid> --root work/ --json
--template assets/template/template.xlsx --decisions decisions.json --push
``` ```
**For fully unattended runs**, skip the prompt entirely with a standing policy — every `list-type` lists every existing part of that typeid with the files in its folder — including
conflict resolved the same way, no questions: its datasheet, which is co-located. For each one: read that datasheet, re-extract the values
(including the new parameter), and rebuild its per-part sheet in place:
```bash ```bash
python scripts/gitea_reconcile.py --parts parts.json --dfs-src <dfs-stage> \ python scripts/fill_templates.py <that_part>.json \
--template assets/template/template.xlsx --on-conflict replace --push --template assets/template/template.xlsx --dest work/<Class>/<that_tag>/
``` ```
`--on-conflict replace` (or `discard`) applies to all conflicts; an explicit per-MPN entry Because `fill_templates` uses the current template and current versions, each rebuilt sheet
in `--decisions` still wins over the blanket policy. Use `--report conflicts.json` (without picks up the new column and the bumped version automatically. When all are done, push once
`--push`) if you ever want to preview conflicts first. Host unreachable → it fails clearly, and tell the user the previous sheets were updated:
same as the push script.
## Updating a template (new parameter)
When the user wants a new parameter on a type, **append it at the end** of that type's
sheet and bump the version:
```bash ```bash
python scripts/append_parameter.py --type Diode --param "Reverse Recovery Time(ns)" \ python scripts/gitea_components.py commit-push --root work/ --message "backfill <param> into <typeid>"
--template assets/template/template.xlsx
``` ```
This adds the column at the end and increments `assets/template/VERSION` (v1 → v2 → …). ## Pushing the skill repo
Then **push the updated `assets/template/template.xlsx` + `VERSION` to the Skill_Assets
repo** so the new template version is archived. New masters record the new version in the When skill files change (a new typeid template, a parameter add, a version/changelog bump),
Meta sheet automatically. push the skill's own files — but **never push the real token**. Blank the `GIT_TOKEN` line in
the copy you push, or exclude `config/gitea.env`:
```bash
bash scripts/push_to_gitea.sh --repo "$SKILL_REPO" --src <skill-dir> --message "Sync skill files"
```
## Resources ## Resources
- `assets/template/template.xlsx` — master template, one sheet per type (source of headers, - `assets/template/template.xlsx` — the master template: one sheet per **typeid** (125),
styling, and sheet order the master workbook mirrors). source of every sheet's headers, styling and order. Columns A/B/C are always
`MPN_make_type` / `Skill Version` / `Template Version`; `Library Ref/Path`,
`Footprint Ref/Path` and `Manufacturer` sit near the end.
- `assets/template/Type_ID.xlsx` + `references/taxonomy.md` — Class → Subclass → Type ID. - `assets/template/Type_ID.xlsx` + `references/taxonomy.md` — Class → Subclass → Type ID.
- `assets/template/VERSION` — current template version (integer; the Meta sheet shows `vN`). - `assets/template/versions.json` — per-typeid `template_version` + `skill_version`.
- `scripts/fill_templates.py` — build the single master workbook `Components_Master.xlsx` - `assets/template/CHANGELOG.md` — global version/parameter changelog (created on first add).
(only sheets for the extracted types + a trailing Meta sheet); also exposes the shared - `scripts/common.py` — taxonomy loader (`load_taxonomy`, `class_folder`), version store
helpers (`part_to_row`, `template_headers`, `sheet_rows`, `read_all_rows`, `build_master`, (`get_versions`, `version_labels`, `bump_versions`), and the tag helper (`part_tag`).
`MASTER_NAME`) the reconcile step reuses so the Gitea master is built and merged identically. - `scripts/fill_templates.py` — build one per-part `<tag>.xlsx` (version-stamped); reused for
- `scripts/gitea_reconcile.py` — add new MPNs and push automatically; only stops on an MPN backfill.
that already exists (discard/replace), or run unattended with `--on-conflict replace`. - `scripts/append_parameter.py` — append parameter(s) to a typeid, bump its versions, write
Merge-safe: merges into the repo's master workbook, other rows/sheets/folders untouched. the changelog.
- `scripts/push_to_gitea.sh` — push a folder's contents to a Gitea repo (flat). Used for - `scripts/altium_refs.py` — read Library/Footprint Ref from `.SchLib`/`.PcbLib`.
Skill_Assets (not MPN-indexed). - `scripts/gitea_components.py` — `check-mpn`, `checkout`, `list-type`, `place-part`,
- `scripts/append_parameter.py` — append a parameter to a template + bump version. `commit-push`, `push-part` against the components repo.
- `scripts/fetch_cad.py` — download Altium footprint/symbol by attaching to the user's - `scripts/push_to_gitea.sh` — push a folder's contents to a Gitea repo (used for the skill
pre-logged-in Chrome (remote-debugging port); tries sites in order, defers on repo).
CAPTCHA/login (never bypasses), and names files `MPN_fp`/`MPN_sym`. Runs on the user's - `config/gitea.env` — host, user, token, and the `SKILL_REPO` / `COMPONENTS_REPO` names
machine; needs `selenium`. (**secret** — do not push the token).
- `config/gitea.env` — host, user, token, and the DFS / Parameters / Skill_Assets repos (**secret**).

7
assets/template/CHANGELOG.md Normal file
View File

@ -0,0 +1,7 @@
# library-manager - template/version changelog
## 2026-07-10 07:56 - ESD (Diode)
- Template v1 -> v2, Skill v1 -> v2
- Added parameter(s): "Peak Power(W)"
- Description: Add Peak Power (Ppk, 8x20us) capture for ESD protection diodes

BIN
assets/template/template.xlsx
View File

Binary file not shown.

502
assets/template/versions.json Normal file
View File

@ -0,0 +1,502 @@
{
"ADC": {
"skill_version": 1,
"template_version": 1
},
"AFE": {
"skill_version": 1,
"template_version": 1
},
"AMP": {
"skill_version": 1,
"template_version": 1
},
"ANC": {
"skill_version": 1,
"template_version": 1
},
"ANE": {
"skill_version": 1,
"template_version": 1
},
"ANP": {
"skill_version": 1,
"template_version": 1
},
"ARR": {
"skill_version": 1,
"template_version": 1
},
"BJT": {
"skill_version": 1,
"template_version": 1
},
"BMS": {
"skill_version": 1,
"template_version": 1
},
"BPK": {
"skill_version": 1,
"template_version": 1
},
"BRG": {
"skill_version": 1,
"template_version": 1
},
"BUZ": {
"skill_version": 1,
"template_version": 1
},
"CBB": {
"skill_version": 1,
"template_version": 1
},
"CBK": {
"skill_version": 1,
"template_version": 1
},
"CCO": {
"skill_version": 1,
"template_version": 1
},
"CER": {
"skill_version": 1,
"template_version": 1
},
"CFR": {
"skill_version": 1,
"template_version": 1
},
"CHG": {
"skill_version": 1,
"template_version": 1
},
"CHL": {
"skill_version": 1,
"template_version": 1
},
"CLF": {
"skill_version": 1,
"template_version": 1
},
"CLI": {
"skill_version": 1,
"template_version": 1
},
"CLK": {
"skill_version": 1,
"template_version": 1
},
"CMC": {
"skill_version": 1,
"template_version": 1
},
"CMP": {
"skill_version": 1,
"template_version": 1
},
"CNI": {
"skill_version": 1,
"template_version": 1
},
"CPL": {
"skill_version": 1,
"template_version": 1
},
"CTC": {
"skill_version": 1,
"template_version": 1
},
"CTX": {
"skill_version": 1,
"template_version": 1
},
"CWB": {
"skill_version": 1,
"template_version": 1
},
"DAC": {
"skill_version": 1,
"template_version": 1
},
"DCD": {
"skill_version": 1,
"template_version": 1
},
"DCM": {
"skill_version": 1,
"template_version": 1
},
"DIG": {
"skill_version": 1,
"template_version": 1
},
"DRV": {
"skill_version": 1,
"template_version": 1
},
"DSG": {
"skill_version": 1,
"template_version": 1
},
"DSW": {
"skill_version": 1,
"template_version": 1
},
"ELE": {
"skill_version": 1,
"template_version": 1
},
"ESD": {
"skill_version": 2,
"template_version": 2
},
"FAN": {
"skill_version": 1,
"template_version": 1
},
"FBD": {
"skill_version": 1,
"template_version": 1
},
"FFC": {
"skill_version": 1,
"template_version": 1
},
"FIX": {
"skill_version": 1,
"template_version": 1
},
"FLM": {
"skill_version": 1,
"template_version": 1
},
"FRD": {
"skill_version": 1,
"template_version": 1
},
"FSR": {
"skill_version": 1,
"template_version": 1
},
"FUS": {
"skill_version": 1,
"template_version": 1
},
"GAN": {
"skill_version": 1,
"template_version": 1
},
"GDT": {
"skill_version": 1,
"template_version": 1
},
"HDR": {
"skill_version": 1,
"template_version": 1
},
"HSK": {
"skill_version": 1,
"template_version": 1
},
"IDC": {
"skill_version": 1,
"template_version": 1
},
"IFC": {
"skill_version": 1,
"template_version": 1
},
"IGBT": {
"skill_version": 1,
"template_version": 1
},
"IND": {
"skill_version": 1,
"template_version": 1
},
"INV": {
"skill_version": 1,
"template_version": 1
},
"ISO": {
"skill_version": 1,
"template_version": 1
},
"JFET": {
"skill_version": 1,
"template_version": 1
},
"LCD": {
"skill_version": 1,
"template_version": 1
},
"LDO": {
"skill_version": 1,
"template_version": 1
},
"LED": {
"skill_version": 1,
"template_version": 1
},
"LOG": {
"skill_version": 1,
"template_version": 1
},
"MCU": {
"skill_version": 1,
"template_version": 1
},
"MEM": {
"skill_version": 1,
"template_version": 1
},
"MFR": {
"skill_version": 1,
"template_version": 1
},
"MMO": {
"skill_version": 1,
"template_version": 1
},
"MOR": {
"skill_version": 1,
"template_version": 1
},
"MOS": {
"skill_version": 1,
"template_version": 1
},
"MTR": {
"skill_version": 1,
"template_version": 1
},
"NTC": {
"skill_version": 1,
"template_version": 1
},
"OBC": {
"skill_version": 1,
"template_version": 1
},
"OLE": {
"skill_version": 1,
"template_version": 1
},
"OSC": {
"skill_version": 1,
"template_version": 1
},
"PBT": {
"skill_version": 1,
"template_version": 1
},
"PBZ": {
"skill_version": 1,
"template_version": 1
},
"PLY": {
"skill_version": 1,
"template_version": 1
},
"PMU": {
"skill_version": 1,
"template_version": 1
},
"POT": {
"skill_version": 1,
"template_version": 1
},
"PSU": {
"skill_version": 1,
"template_version": 1
},
"PTC": {
"skill_version": 1,
"template_version": 1
},
"PWC": {
"skill_version": 1,
"template_version": 1
},
"PWR": {
"skill_version": 1,
"template_version": 1
},
"RCM": {
"skill_version": 1,
"template_version": 1
},
"REC": {
"skill_version": 1,
"template_version": 1
},
"RFI": {
"skill_version": 1,
"template_version": 1
},
"RFM": {
"skill_version": 1,
"template_version": 1
},
"RLP": {
"skill_version": 1,
"template_version": 1
},
"RLR": {
"skill_version": 1,
"template_version": 1
},
"RLS": {
"skill_version": 1,
"template_version": 1
},
"RSF": {
"skill_version": 1,
"template_version": 1
},
"RSN": {
"skill_version": 1,
"template_version": 1
},
"RSW": {
"skill_version": 1,
"template_version": 1
},
"RSY": {
"skill_version": 1,
"template_version": 1
},
"SAW": {
"skill_version": 1,
"template_version": 1
},
"SCH": {
"skill_version": 1,
"template_version": 1
},
"SCM": {
"skill_version": 1,
"template_version": 1
},
"SCU": {
"skill_version": 1,
"template_version": 1
},
"SEN": {
"skill_version": 1,
"template_version": 1
},
"SFY": {
"skill_version": 1,
"template_version": 1
},
"SHA": {
"skill_version": 1,
"template_version": 1
},
"SHT": {
"skill_version": 1,
"template_version": 1
},
"SIC": {
"skill_version": 1,
"template_version": 1
},
"SIM": {
"skill_version": 1,
"template_version": 1
},
"SPK": {
"skill_version": 1,
"template_version": 1
},
"SPR": {
"skill_version": 1,
"template_version": 1
},
"SSR": {
"skill_version": 1,
"template_version": 1
},
"STE": {
"skill_version": 1,
"template_version": 1
},
"SUP": {
"skill_version": 1,
"template_version": 1
},
"SVO": {
"skill_version": 1,
"template_version": 1
},
"SVR": {
"skill_version": 1,
"template_version": 1
},
"SWI": {
"skill_version": 1,
"template_version": 1
},
"SWT": {
"skill_version": 1,
"template_version": 1
},
"TAN": {
"skill_version": 1,
"template_version": 1
},
"TBK": {
"skill_version": 1,
"template_version": 1
},
"TFR": {
"skill_version": 1,
"template_version": 1
},
"TFT": {
"skill_version": 1,
"template_version": 1
},
"TPD": {
"skill_version": 1,
"template_version": 1
},
"TVS": {
"skill_version": 1,
"template_version": 1
},
"USB": {
"skill_version": 1,
"template_version": 1
},
"VAR": {
"skill_version": 1,
"template_version": 1
},
"VRF": {
"skill_version": 1,
"template_version": 1
},
"WWR": {
"skill_version": 1,
"template_version": 1
},
"XCV": {
"skill_version": 1,
"template_version": 1
},
"XFM": {
"skill_version": 1,
"template_version": 1
},
"XTL": {
"skill_version": 1,
"template_version": 1
},
"ZEN": {
"skill_version": 1,
"template_version": 1
}
}

15
config/gitea.env
View File

@ -1,10 +1,13 @@
# Gitea connection for the datasheet-extractor skill. # Gitea connection for the library-manager skill.
# WARNING: contains a secret (GIT_TOKEN) in plaintext - keep this skill PRIVATE. # WARNING: contains a secret (GIT_TOKEN) in plaintext - keep this skill PRIVATE.
# Use a token scoped to these repos (repository: write) and rotate periodically. # Use a token scoped to these repos (repository: write) and rotate periodically.
GIT_HOST=gitea.vecmocon.com GIT_HOST=gitea.vecmocon.com
GIT_USER=nitishKumar GIT_USER=nitishKumar
GIT_TOKEN= GIT_TOKEN=451bff1dc32202cbc0a371f8e5645079466d2120
# Target repos (flat layout, no project/version grouping): # Target repos — TWO now (set these to the exact repo names on your Gitea):
DFS_REPO=nitishKumar/DFS # SKILL_REPO : holds this skill's own files (SKILL.md, scripts, assets, ...).
PARAMS_REPO=nitishKumar/Parameters # COMPONENTS_REPO : holds components, one folder per Class (Diode, IC, ...); inside each,
SKILL_ASSETS_REPO=nitishKumar/Skill_Assets # one MPN_make_typeid/ folder per part with { xlsx, datasheet, symbol,
# footprint }. (Replaces the old DFS + Parameters repos.)
SKILL_REPO=nitishKumar/Skill_Assets
COMPONENTS_REPO=nitishKumar/Components

5
references/taxonomy.md
View File

@ -1,7 +1,10 @@
# Component taxonomy — Class → Subclass (Type) → Type ID # Component taxonomy — Class → Subclass (Type) → Type ID
Use the **Type ID** as the `typeid` in `MPN_make_typeid`. Match each part to the closest Use the **Type ID** as the `typeid` in `MPN_make_typeid`. Match each part to the closest
subclass. The output sheet name is the **Class** (template.xlsx sheet). Full source: subclass. In the current template **each Type ID is its own sheet** (125 sheets), so the
template sheet name **is the Type ID** (e.g. `SCH`, `MOS`, `LDO`). The **Class** is used only
to organise the components repo into top-level folders (e.g. `SCH` lives under `Diode/`);
`scripts/common.py:class_folder()` maps a typeid to its folder name. Full source:
`assets/template/Type_ID.xlsx`. `assets/template/Type_ID.xlsx`.

BIN
scripts/__pycache__/common.cpython-311.pyc Normal file
View File

Binary file not shown.

169
scripts/altium_refs.py Normal file
View File

@ -0,0 +1,169 @@
#!/usr/bin/env python3
"""Read the internal *reference name* out of Altium symbol (.SchLib) and footprint (.PcbLib)
files, and assemble the four design-column values for a part.
Why this exists: the value that belongs in the sheet's **Library Ref** / **Footprint Ref**
column is NOT the file name or the MPN - it is the component/footprint name as Altium stores
it *inside* the library (what an engineer sees in the file's properties). We read it directly
so no one has to open Altium by hand.
Verified against real Vecmocon files (Ultra Librarian export, e.g. CGA3E3X7R1H474K080AE):
- .SchLib: the component is a top-level OLE *storage* whose name is the Library Ref, e.g.
``CGA3E3X7R1H474K080AE``. (Confirmed also in the FileHeader as ``LibRef0=...``.)
- .PcbLib: each footprint is a top-level storage. An Ultra Librarian export ships the base
pattern plus IPC density variants, e.g. ``CAP_CGA3_TDK`` (nominal), ``CAP_CGA3_TDK-L``
(least/L), ``CAP_CGA3_TDK-M`` (most/M). The **base** pattern (the one the others suffix
with ``-L`` / ``-M`` / ``-N``) is the Footprint Ref -> ``CAP_CGA3_TDK``.
The four values that get written to the sheet:
Library Ref = name inside the .SchLib (read here)
Library Path = the .SchLib file name (just the basename)
Footprint Ref = base pattern inside the .PcbLib (read here)
Footprint Path = the .PcbLib file name (just the basename)
Usage:
# one file -> its reference name
python altium_refs.py CGA3E3X7R1H474K080AE.SchLib
python altium_refs.py CGA3E3X7R1H474K080AE.PcbLib --json
# both files -> ready-to-use design map for fill_templates --design
python altium_refs.py design --symbol X.SchLib --footprint Y.PcbLib > design.json
Programmatic: from altium_refs import extract_ref, build_design
If a ref comes back None or a footprint has several unrelated candidates, ask the engineer to
confirm the name shown in Altium's properties.
"""
import argparse, json, os, re
import olefile
# Entry names that are container metadata, never a component/footprint reference.
META = {
"fileheader", "additionalfileheader", "storage", "library", "root entry",
"componentparamstoc", "fileversioninfo", "sectionkeys", "data", "header",
"uniqueidprimitiveinformation", "pintextdata", "textframe", "models",
"modelsnoembed", "arcs6", "pads6", "vias6", "tracks6", "texts6", "fills6",
"regions6", "componentbodies6", "designatoroverride", "parameters",
"primitiveguids", "widestrings", "layerkindmapping", "padvialibrary",
"embeddedfonts", "textures", "fileversion",
}
def _decode(raw):
for enc in ("utf-16-le", "latin-1", "utf-8"):
try:
return raw.decode(enc, errors="ignore")
except Exception:
continue
return ""
def _storage_names(ole):
"""Distinct top-level storage names that aren't container metadata."""
names = []
for entry in ole.listdir(streams=True, storages=True):
top = entry[0]
if top.lower() in META or top in names:
continue
names.append(top)
return names
def _from_header_records(ole):
"""Backstop: scan header streams for LibReference / PatternName records."""
hits = []
for entry in ole.listdir(streams=True):
if entry[-1].lower() not in ("fileheader", "header", "data"):
continue
try:
txt = _decode(ole.openstream(entry).read())
except Exception:
continue
for key in ("LIBREFERENCE", "PATTERNNAME", "PATTERN"):
for m in re.finditer(rf"{key}=([^\x00|]+)", txt, re.IGNORECASE):
val = m.group(1).strip()
if val and val not in hits:
hits.append(val)
return hits
def _pick_base(candidates):
"""Choose the primary reference. When several names share a base and differ only by a
trailing density suffix (Altium IPC variants: -L / -M / -N / -Least / -Most / -Nominal),
the base (a prefix of the others) is the real one. Otherwise take the first."""
if not candidates:
return None
bases = [c for c in candidates
if any(o != c and o.startswith(c + "-") for o in candidates)]
if bases:
return sorted(bases, key=len)[0]
# no prefix relationship: drop obvious -L/-M/-N variants if a suffix-less peer exists
plain = [c for c in candidates if not re.search(r"-(L|M|N|Least|Most|Nominal)$", c)]
return (plain or candidates)[0]
def extract_ref(path):
"""Return (ref, candidates): ref = best single reference name (base pattern) or None."""
if not olefile.isOleFile(path):
return None, []
ole = olefile.OleFileIO(path)
try:
candidates = _storage_names(ole) or _from_header_records(ole)
finally:
ole.close()
return _pick_base(candidates), candidates
def kind_of(path):
ext = os.path.splitext(path)[1].lower()
return {".schlib": "symbol", ".pcblib": "footprint"}.get(ext, "unknown")
def build_design(symbol_path, footprint_path):
"""Assemble the four design-column values from the two uploaded files."""
lib_ref, lib_cands = extract_ref(symbol_path)
fp_ref, fp_cands = extract_ref(footprint_path)
return {
"Library Ref": lib_ref,
"Library Path": os.path.basename(symbol_path),
"Footprint Ref": fp_ref,
"Footprint Path": os.path.basename(footprint_path),
"_candidates": {"symbol": lib_cands, "footprint": fp_cands},
}
def main():
ap = argparse.ArgumentParser()
sub = ap.add_subparsers(dest="cmd")
d = sub.add_parser("design", help="emit the 4-field design map from both files")
d.add_argument("--symbol", required=True)
d.add_argument("--footprint", required=True)
ap.add_argument("file", nargs="?", help=".SchLib or .PcbLib (single-file mode)")
ap.add_argument("--json", action="store_true")
a = ap.parse_args()
if a.cmd == "design":
design = build_design(a.symbol, a.footprint)
warn = [k for k in ("Library Ref", "Footprint Ref") if not design[k]]
print(json.dumps(design, ensure_ascii=False, indent=2))
if warn:
print(f"# WARNING: could not read {', '.join(warn)} - confirm from Altium", flush=True)
return
if not a.file:
ap.error("give a file, or use: design --symbol X.SchLib --footprint Y.PcbLib")
ref, cands = extract_ref(a.file)
if a.json:
print(json.dumps({"kind": kind_of(a.file), "ref": ref, "candidates": cands}, ensure_ascii=False))
return
if ref is None:
print(f"Could not read a reference from {a.file}. Open it in Altium and use the "
"name shown in properties.")
else:
print(ref)
if len(cands) > 1:
print(f"(variants present: {cands}; using base '{ref}')")
if __name__ == "__main__":
main()

118
scripts/append_parameter.py
View File

@ -1,27 +1,99 @@
#!/usr/bin/env python3 #!/usr/bin/env python3
"""Append a new parameter (column) to the end of a type's sheet in the master """Add one or more new parameters to a single typeid's template sheet, bump that typeid's
template, and bump the template VERSION. After running, push the updated versions, and record the change in the global changelog.
assets/template/template.xlsx + VERSION to the Skill_Assets repo.
Usage: python append_parameter.py --type Diode --param "Reverse Recovery Time(ns)" \ Versioning is **per typeid**: adding a parameter to (say) SCH means there's a new SCH
--template assets/template/template.xlsx template, so SCH's template_version bumps v1->v2, and on the back of that its skill_version
bumps v1->v2 too. Only SCH moves; every other typeid keeps its versions. Those two numbers
are what fill_templates later stamps into the SCH sheet's rows (columns B and C).
The new column is appended at the END of the sheet (as requested), styled like the other
headers. Then versions.json is bumped and one entry is written to CHANGELOG.md.
python append_parameter.py --typeid SCH \
--param "Reverse Recovery Time(ns)" --param "Diode Capacitance(pF)" \
--desc "Added Trr and Cd for SMPS rectifier derating"
After running, push the updated template.xlsx + versions.json + CHANGELOG.md to the SKILL
repo (and, per the workflow, the same change flows to the components repo copies).
""" """
import argparse, os, openpyxl import argparse, datetime, os
ap=argparse.ArgumentParser() import openpyxl
ap.add_argument("--type", required=True); ap.add_argument("--param", required=True) from copy import copy
ap.add_argument("--template", required=True) from common import (TEMPLATE_XLSX, CHANGELOG_MD, bump_versions, load_taxonomy)
a=ap.parse_args()
wb=openpyxl.load_workbook(a.template)
if a.type not in wb.sheetnames: def _style_header(new_cell, ref_cell):
raise SystemExit(f"no sheet '{a.type}' in template; sheets: {wb.sheetnames}") """Make an appended header look like the sheet's existing headers."""
ws=wb[a.type] for attr in ("font", "fill", "border", "alignment", "number_format"):
try:
setattr(new_cell, attr, copy(getattr(ref_cell, attr)))
except Exception:
pass
def append_params(template_path, typeid, params):
wb = openpyxl.load_workbook(template_path)
if typeid not in wb.sheetnames:
raise SystemExit(f"no sheet '{typeid}' in template; is it a valid typeid?")
ws = wb[typeid]
headers = [ws.cell(1, c).value for c in range(1, ws.max_column + 1)] headers = [ws.cell(1, c).value for c in range(1, ws.max_column + 1)]
if a.param in headers: added = []
raise SystemExit(f"'{a.param}' already exists in {a.type}") for p in params:
ws.cell(1, ws.max_column+1, a.param) # append at the end if p in headers:
wb.save(a.template) print(f" ! '{p}' already exists in {typeid} - skipping")
vf=os.path.join(os.path.dirname(a.template), "VERSION") continue
v=int(open(vf).read().strip()) if os.path.exists(vf) else 1 col = ws.max_column + 1
v+=1; open(vf,"w").write(str(v)) cell = ws.cell(1, col, p)
print(f"Appended '{a.param}' to {a.type}. Template is now v{v}. " _style_header(cell, ws.cell(1, 1))
f"Push {os.path.basename(a.template)} + VERSION to the Skill_Assets repo.") headers.append(p)
added.append(p)
if not added:
raise SystemExit("nothing to add (all parameters already present)")
wb.save(template_path)
return added
def write_changelog(typeid, added, old, new, desc, when=None):
when = when or datetime.datetime.now()
folder = (load_taxonomy().get(typeid) or {}).get("folder", "?")
lines = [
f"## {when.strftime('%Y-%m-%d %H:%M')} - {typeid} ({folder})",
f"- Template v{old['template_version']} -> v{new['template_version']}, "
f"Skill v{old['skill_version']} -> v{new['skill_version']}",
"- Added parameter(s): " + ", ".join(f'\"{p}\"' for p in added),
]
if desc:
lines.append(f"- Description: {desc}")
entry = "\n".join(lines) + "\n\n"
header = "# library-manager - template/version changelog\n\n"
prior = ""
if os.path.exists(CHANGELOG_MD):
prior = open(CHANGELOG_MD, encoding="utf-8").read()
if prior.startswith(header):
prior = prior[len(header):]
open(CHANGELOG_MD, "w", encoding="utf-8").write(header + entry + prior) # newest on top
return CHANGELOG_MD
def main():
ap = argparse.ArgumentParser()
ap.add_argument("--typeid", required=True, help="e.g. SCH, MOS, LDO")
ap.add_argument("--param", required=True, action="append",
help="new column header; repeat for several")
ap.add_argument("--desc", default="", help="what/why - goes in the changelog")
ap.add_argument("--template", default=TEMPLATE_XLSX)
a = ap.parse_args()
added = append_params(a.template, a.typeid, a.param)
old, new = bump_versions(a.typeid)
cl = write_changelog(a.typeid, added, old, new, a.desc)
print(f"Added to {a.typeid}: {', '.join(added)}")
print(f"{a.typeid}: template v{old['template_version']}->v{new['template_version']}, "
f"skill v{old['skill_version']}->v{new['skill_version']}")
print(f"Changelog updated: {cl}")
print("Now push template.xlsx + versions.json + CHANGELOG.md to the SKILL repo.")
if __name__ == "__main__":
main()

163
scripts/common.py Normal file
View File

@ -0,0 +1,163 @@
#!/usr/bin/env python3
"""Shared helpers for the library-manager skill.
One place for the three things every script needs to agree on:
- the taxonomy (typeid -> Class -> components-repo folder name), read from
``assets/template/Type_ID.xlsx`` so there is a single source of truth;
- the **per-typeid version store** (``assets/template/versions.json``) that holds each
typeid's own ``template_version`` and ``skill_version`` - both start at 1 and both bump
together the moment a parameter is added to that typeid (and nothing else moves);
- the part **tag** ``<MPN>_<make>_<typeid>`` that names every folder and column-A cell.
Keeping these here means fill_templates / append_parameter / gitea_components can't drift
apart on how a version is read or a folder is named.
"""
import json, os, re
import openpyxl
SKILL_ROOT = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
TEMPLATE_DIR = os.path.join(SKILL_ROOT, "assets", "template")
TEMPLATE_XLSX= os.path.join(TEMPLATE_DIR, "template.xlsx")
TYPE_ID_XLSX = os.path.join(TEMPLATE_DIR, "Type_ID.xlsx")
VERSIONS_JSON= os.path.join(TEMPLATE_DIR, "versions.json")
CHANGELOG_MD = os.path.join(TEMPLATE_DIR, "CHANGELOG.md")
# Fixed template columns that live on every sheet and are NOT extracted from a datasheet.
COL_TAG = "MPN_make_type" # column A header (the part tag lives here)
COL_SKILL_VER = "Skill Version" # column B (per-typeid skill version, stamped by us)
COL_TEMPLATE_VER = "Template Version" # column C (per-typeid template version, stamped by us)
DESIGN_COLS = ("Library Ref", "Library Path", "Footprint Ref", "Footprint Path")
# Raw taxonomy Class name -> clean, filesystem-safe folder name for the components repo.
# These match Vecmocon's established sheet-name convention (the old 18-class template).
CLASS_FOLDER = {
"Resistor": "Resistor",
"Capacitor": "Capacitor",
"Inductor / Magnetics": "Inductor_Magnetics",
"Diode": "Diode",
"Transistor": "Transistor",
"Integrated Circuit (IC)": "IC",
"Protection Device": "Protection",
"Power Conversion Module": "Power Conversion Module",
"Relay / Contactor": "Relay_Contactor",
"Switch / Button": "Switch_Button",
"Connector": "Connector",
"Antenna / RF": "Antenna_RF",
"Crystal / Oscillator / Timing": "Crystal_Oscillator",
"Battery / Cell": "Battery_Cell",
"Audible / Indicator": "Audible_Indicator",
"Display / HMI": "Display_HMI",
"Sensor (discrete / module)": "Sensor",
"Thermal / Cooling": "Thermal_Cooling",
}
def norm(s):
return re.sub(r"[^a-z0-9]", "", str(s).lower())
# ---------------------------------------------------------------- taxonomy
def load_taxonomy(type_id_xlsx=TYPE_ID_XLSX):
"""Return {typeid: {"class": rawclass, "subclass": name, "folder": foldername}}.
Reads the Type_ID.xlsx. A real mapping row has all three of Class / Subclass / Type ID;
the group-header rows (only a Class label like 'Resistor (12 types)') are skipped
because they have no Type ID.
"""
wb = openpyxl.load_workbook(type_id_xlsx, read_only=True, data_only=True)
ws = wb[wb.sheetnames[0]]
out = {}
for row in ws.iter_rows(min_row=1, values_only=True):
cells = (list(row) + [None, None, None])[:3]
cls, sub, tid = (str(c).strip() if c is not None else "" for c in cells)
if not tid or tid.lower() in ("type id",):
continue
folder = CLASS_FOLDER.get(cls, re.sub(r"\s*/\s*", "_", re.sub(r"\s*\(.*?\)", "", cls)).strip().replace(" ", "_"))
out[tid] = {"class": cls, "subclass": sub, "folder": folder}
return out
def class_folder(typeid, taxonomy=None):
"""The components-repo top-level folder name for a typeid (e.g. SCH -> 'Diode')."""
tax = taxonomy or load_taxonomy()
info = tax.get(typeid)
return info["folder"] if info else None
# ---------------------------------------------------------------- part tag
def make_tag(manufacturer):
"""The 'make' token = first word of the manufacturer, stripped to alphanumerics."""
w = re.split(r"\s+", str(manufacturer).strip())
first = w[0] if w and w[0] else "NA"
return re.sub(r"[^A-Za-z0-9]", "", first) or "NA"
def part_tag(part):
"""Identity of a part: <MPN>_<make>_<typeid>. Same tag = same part."""
make = part.get("make") or make_tag(part.get("manufacturer", ""))
return f'{part.get("mpn","")}_{make}_{part.get("typeid","NA")}'
def mpn_make_prefix(mpn, make):
"""The <MPN>_<make>_ prefix used for the early duplicate check (typeid not yet known)."""
return f"{mpn}_{make}_"
# ---------------------------------------------------------------- version store
def _load_versions():
if os.path.exists(VERSIONS_JSON):
try:
return json.load(open(VERSIONS_JSON, encoding="utf-8"))
except Exception:
pass
return {}
def get_versions(typeid):
"""Return {'template_version': int, 'skill_version': int} for a typeid (default 1/1)."""
v = _load_versions().get(typeid) or {}
return {"template_version": int(v.get("template_version", 1)),
"skill_version": int(v.get("skill_version", 1))}
def version_labels(typeid):
"""Return ('vN', 'vN') = (skill_version, template_version) strings for stamping cells."""
v = get_versions(typeid)
return f'v{v["skill_version"]}', f'v{v["template_version"]}'
def bump_versions(typeid):
"""Increment BOTH template_version and skill_version for one typeid; persist and return
(old, new) each as {'template_version','skill_version'}. Only this typeid changes."""
data = _load_versions()
cur = data.get(typeid) or {"template_version": 1, "skill_version": 1}
old = {"template_version": int(cur.get("template_version", 1)),
"skill_version": int(cur.get("skill_version", 1))}
new = {"template_version": old["template_version"] + 1,
"skill_version": old["skill_version"] + 1}
data[typeid] = new
json.dump(data, open(VERSIONS_JSON, "w", encoding="utf-8"), indent=2, sort_keys=True)
return old, new
def init_versions(force=False):
"""Seed versions.json with 1/1 for every typeid in the taxonomy (idempotent)."""
data = {} if force else _load_versions()
for tid in load_taxonomy():
data.setdefault(tid, {"template_version": 1, "skill_version": 1})
json.dump(data, open(VERSIONS_JSON, "w", encoding="utf-8"), indent=2, sort_keys=True)
return data
if __name__ == "__main__":
import sys
if len(sys.argv) > 1 and sys.argv[1] == "init":
d = init_versions(force="--force" in sys.argv)
print(f"versions.json seeded with {len(d)} typeids at {VERSIONS_JSON}")
else:
tax = load_taxonomy()
print(f"taxonomy: {len(tax)} typeids, {len(set(v['folder'] for v in tax.values()))} class folders")

204
scripts/fill_templates.py
View File

@ -1,143 +1,105 @@
#!/usr/bin/env python3 #!/usr/bin/env python3
"""Fill the reference template into ONE master workbook (Components_Master.xlsx). """Build ONE per-part Excel sheet from the master template.
The master contains ONLY the type sheets that actually have parts (the components whose The new Vecmocon layout stores each component on its own: a folder ``MPN_make_typeid/``
datasheets were provided) - built from a copy of the template so each kept sheet keeps its holding that part's own single-sheet workbook (plus its datasheet, symbol and footprint).
exact headers, styling, widths and freeze - plus a single **Meta** sheet appended at the There is no accumulating master workbook any more - so this script takes one part and writes
END with: Template Version, Total Components, Date, Time, and a per-sheet count breakdown. ``<MPN>_<make>_<typeid>.xlsx``, a copy of just that typeid's template sheet (headers, styling,
Empty template sheets are dropped. Column A = <MPN>_<make>_<typeid>. widths, freeze preserved) with a single filled data row.
Shared helpers (part_tag, part_to_row, template_headers, sheet_rows, read_all_rows, What gets stamped automatically (never read from a datasheet):
build_master, resolve_version, MASTER_NAME) are imported by gitea_reconcile.py so the Gitea - column A ``MPN_make_type`` = the part tag ``<MPN>_<make>_<typeid>``
master is built and merged the exact same way. - column B ``Skill Version`` = this typeid's skill version (vN, from versions.json)
- column C ``Template Version``= this typeid's template version (vN, from versions.json)
python fill_templates.py parts.json --template <template.xlsx> --dest <dir> [--version v1] The four design columns (Library Ref/Path, Footprint Ref/Path) are filled from a --design
map when the symbol/footprint step provides them, and left blank otherwise. Everything else
comes from the extracted ``values``.
# one part, values only (design columns left blank for now)
python fill_templates.py part.json --template assets/template/template.xlsx --dest out/
# later, same part with the Altium-derived design fields merged in
python fill_templates.py part.json --template ... --dest out/ --design design.json
``part.json`` is either a single part object or {"parts": [ ... ]} (each part built to its
own file). A part = {"mpn","manufacturer" (or "make"),"typeid","values":{header:value,...}}.
""" """
import argparse, json, os, re, datetime import argparse, json, os
import openpyxl import openpyxl
from openpyxl.styles import Font, PatternFill, Border, Side, Alignment from common import (norm, part_tag, version_labels,
from openpyxl.utils import get_column_letter COL_TAG, COL_SKILL_VER, COL_TEMPLATE_VER, DESIGN_COLS)
GREEN="B6D7A8"; GRAY="BFBFBF"
MASTER_NAME="Components_Master.xlsx"; META_SHEET="Meta"
def norm(s): return re.sub(r'[^a-z0-9]', '', str(s).lower())
ALIAS={"induactanceuh":"inductanceuh"} # template header typo -> value key
def make_tag(manufacturer): def template_headers(template_path, typeid):
w=re.split(r'\s+', str(manufacturer).strip()); first=w[0] if w and w[0] else "NA" """Canonical headers for a typeid sheet, or None if the template has no such sheet."""
return re.sub(r'[^A-Za-z0-9]', '', first) or "NA" wb = openpyxl.load_workbook(template_path, read_only=True)
def part_tag(part): if typeid not in wb.sheetnames:
"""Identity of a part: <MPN>_<make>_<typeid>. Same tag = same part.""" return None
make=part.get("make") or make_tag(part.get("manufacturer","")) ws = wb[typeid]
return f'{part.get("mpn","")}_{make}_{part.get("typeid","NA")}' return [ws.cell(1, c).value for c in range(1, ws.max_column + 1)]
def template_headers(template_path, ctype):
"""Canonical data headers for a type sheet, or None if the template has no such sheet."""
wb=openpyxl.load_workbook(template_path)
if ctype not in wb.sheetnames: return None
ws=wb[ctype]; return [ws.cell(1,c).value for c in range(1, ws.max_column+1)]
def template_types(template_path): def part_to_row(part, headers, design=None):
return [s for s in openpyxl.load_workbook(template_path).sheetnames if s!=META_SHEET] """Map one part to {header: value}, following the fixed-column rules above."""
vals = {norm(k): v for k, v in part.get("values", {}).items()}
def part_to_row(part, headers): dmap = {norm(k): v for k, v in (design or {}).items()}
"""Return (tag, {header: value}) for one part, following the column rules.""" tag = part_tag(part)
vals={norm(k):v for k,v in part.get("values",{}).items()}; tag=part_tag(part); row={} sver, tver = version_labels(part.get("typeid", "NA"))
for c,h in enumerate(headers, start=1): row = {}
key=ALIAS.get(norm(h),norm(h)) for h in headers:
if c==1: row[h]=tag key = norm(h)
elif key=="class": row[h]=part.get("subclass","") if key == norm(COL_TAG): row[h] = tag
elif key=="manufacturer":row[h]=part.get("manufacturer", vals.get("manufacturer","")) elif key == norm(COL_SKILL_VER): row[h] = sver
elif key == norm(COL_TEMPLATE_VER): row[h] = tver
elif key == norm("Manufacturer"): row[h] = part.get("manufacturer", vals.get("manufacturer", ""))
elif h in DESIGN_COLS: row[h] = dmap.get(key, vals.get(key, ""))
else: row[h] = vals.get(key, "") else: row[h] = vals.get(key, "")
return tag, row return tag, row
def sheet_rows(ws, width):
"""Existing data rows in a sheet -> (headers, {tag: {header: value}}), bounded to `width`."""
headers=[ws.cell(1,c).value for c in range(1,width+1)]; rows={}
for r in range(2, ws.max_row+1):
tag=ws.cell(r,1).value
if tag in (None,""): continue
rows[tag]={headers[c-1]: ws.cell(r,c).value for c in range(1,width+1)}
return headers, rows
def read_all_rows(master_path, template_path): def build_part_sheet(part, template_path, dest_dir, design=None):
"""Read every type sheet of an existing master -> {ctype: {tag: rowdict}} (skips Meta).""" """Write ``<tag>.xlsx`` = just this part's typeid sheet with one filled row. Returns
wb=openpyxl.load_workbook(master_path); types=set(template_types(template_path)); out={} (path, tag). Reused for backfill: re-extract a part and call this again to regenerate its
for name in wb.sheetnames: sheet against the current template (new columns) and current stamped versions."""
if name not in types: continue typeid = part["typeid"]
th=template_headers(template_path,name); w=len(th) if th else 0 headers = template_headers(template_path, typeid)
if not w: continue if headers is None:
_,rows=sheet_rows(wb[name], w) raise SystemExit(f"no template sheet for typeid '{typeid}'")
if rows: out[name]=rows wb = openpyxl.load_workbook(template_path)
return out for name in list(wb.sheetnames): # keep only this typeid's sheet
if name != typeid:
def _thin(): s=Side(style="thin",color=GRAY); return Border(left=s,right=s,top=s,bottom=s) del wb[name]
def _hdr(cell): ws = wb[typeid]
cell.font=Font(name="Calibri",bold=True); cell.fill=PatternFill("solid",fgColor=GREEN) tag, row = part_to_row(part, headers, design)
cell.border=_thin(); cell.alignment=Alignment(horizontal="center",vertical="center") for r in range(2, ws.max_row + 1): # clear any stray rows below the header
for c in range(1, len(headers) + 1):
def _write_rows(ws, headers, rows_by_tag): ws.cell(r, c).value = None
for r in range(2, ws.max_row+1): for c, h in enumerate(headers, start=1):
for c in range(1, len(headers)+1): ws.cell(r,c).value=None ws.cell(2, c, row.get(h, ""))
r=2
for tag,row in rows_by_tag.items():
for c,h in enumerate(headers, start=1): ws.cell(r,c, row.get(h,""))
r+=1
def _build_meta_sheet(wb, version, counts, when):
if META_SHEET in wb.sheetnames: del wb[META_SHEET]
ws=wb.create_sheet(META_SHEET) # appended at the end
for r,(k,v) in enumerate([("Template Version",version),
("Total Components",sum(counts.values())),
("Date",when.strftime("%Y-%m-%d")),
("Time",when.strftime("%H:%M:%S"))], start=1):
a=ws.cell(r,1,k); a.font=Font(name="Calibri",bold=True)
a.fill=PatternFill("solid",fgColor=GREEN); a.border=_thin()
ws.cell(r,2,v).border=_thin()
r=6
for c,t in ((1,"Sheet"),(2,"Components")): _hdr(ws.cell(r,c,t))
for ctype,n in counts.items():
r+=1; ws.cell(r,1,ctype).border=_thin(); ws.cell(r,2,n).border=_thin()
ws.column_dimensions["A"].width=20; ws.column_dimensions["B"].width=16
return ws
def resolve_version(template_path, version):
if version: return version
vf=os.path.join(os.path.dirname(template_path),"VERSION")
return "v"+open(vf).read().strip() if os.path.exists(vf) else "v1"
def build_master(template_path, rows_by_type, version, dest_dir):
"""Write the single master workbook from a copy of the template. `rows_by_type` =
{ctype: {tag: rowdict}} is the COMPLETE desired contents. Only sheets present (with at
least one row) are kept; all other template sheets are dropped. A Meta sheet is appended
last. Returns the output path."""
os.makedirs(dest_dir, exist_ok=True) os.makedirs(dest_dir, exist_ok=True)
wb=openpyxl.load_workbook(template_path); when=datetime.datetime.now(); counts={} out = os.path.join(dest_dir, f"{tag}.xlsx")
for ctype in template_types(template_path): wb.save(out)
ws=wb[ctype]; rows=rows_by_type.get(ctype) or {} return out, tag
if rows:
th=template_headers(template_path,ctype); width=len(th) if th else ws.max_column
_write_rows(ws, [ws.cell(1,c).value for c in range(1,width+1)], rows) def _iter_parts(data):
counts[ctype]=len(rows) return data["parts"] if isinstance(data, dict) and "parts" in data else [data]
_build_meta_sheet(wb, version, counts, when) # append first so a sheet always remains
for ctype in template_types(template_path):
if ctype not in counts: del wb[ctype] # drop empty type sheets
out=os.path.join(dest_dir, MASTER_NAME); wb.save(out); return out
def main(): def main():
ap = argparse.ArgumentParser() ap = argparse.ArgumentParser()
ap.add_argument("parts_json"); ap.add_argument("--template", required=True) ap.add_argument("part_json")
ap.add_argument("--dest", required=True); ap.add_argument("--version", default=None) ap.add_argument("--template", required=True)
ap.add_argument("--dest", required=True)
ap.add_argument("--design", default=None, help="JSON map of design columns to merge in")
a = ap.parse_args() a = ap.parse_args()
data=json.load(open(a.parts_json, encoding="utf-8")) data = json.load(open(a.part_json, encoding="utf-8"))
version=resolve_version(a.template, a.version); by={} design = json.load(open(a.design, encoding="utf-8")) if a.design else None
for p in data.get("parts", []): for p in _iter_parts(data):
headers=template_headers(a.template, p["type"]) out, tag = build_part_sheet(p, a.template, a.dest, design)
if headers is None: print(f"! no template sheet for type '{p['type']}' - skipping"); continue sver, tver = version_labels(p.get("typeid", "NA"))
tag,row=part_to_row(p, headers); by.setdefault(p["type"],{})[tag]=row print(f"wrote {out} (tag {tag}, skill {sver}, template {tver})")
out=build_master(a.template, by, version, a.dest)
total=sum(len(v) for v in by.values())
print(f"master: {out} ({total} part(s) across {len(by)} sheet(s) + Meta, template {version})")
if __name__ == "__main__": if __name__ == "__main__":
main() main()

227
scripts/gitea_components.py Normal file
View File

@ -0,0 +1,227 @@
#!/usr/bin/env python3
"""Talk to the two Gitea repos of the new library-manager layout.
Repos (from config/gitea.env):
SKILL_REPO - this skill's own files (pushed with push_to_gitea.sh, not here).
COMPONENTS_REPO - one folder per Class (Diode, IC, Transistor, ...); inside each Class,
one MPN_make_typeid/ folder per part holding { xlsx, datasheet, symbol,
footprint }.
Subcommands (all accept --local <dir> to run against a local folder instead of cloning, for
testing or offline work):
check-mpn --mpn BAT46WJ --make Nexperia
Is any part with this MPN+make already in the components repo? (typeid-agnostic - the
early duplicate gate, run BEFORE classifying.) Prints EXISTS/<tag> or ABSENT; exit 0
if absent, 3 if it already exists (so a shell can hard-stop).
checkout --dest work/
Clone the components repo to a working dir you can browse and edit in place.
list-type --typeid SCH [--root work/] [--json]
List every existing part of a typeid and the files in its folder (for backfill: the
datasheet to re-read is in there). Needs a checked-out --root (or --local).
place-part --folder staging/BAT46WJ_Nexperia_SCH --typeid SCH --root work/
Copy an assembled part folder into work/<Class>/<tag>/ (creates the Class folder if it
is missing). Then commit-push.
commit-push --root work/ [--message "..."]
Commit everything in the checked-out clone and push.
push-part --folder staging/BAT46WJ_Nexperia_SCH --typeid SCH [--message "..."]
Convenience: clone -> place-part -> commit-push in one go (the common single-part path).
Host unreachable (e.g. sandbox without the domain allowlisted) -> clones fail clearly and
nothing is written.
"""
import argparse, os, re, shutil, subprocess, sys, tempfile
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
from common import class_folder, load_taxonomy, mpn_make_prefix
CFG_DEFAULT = os.path.join(os.path.dirname(__file__), "..", "config", "gitea.env")
# ------------------------------------------------------------------ gitea plumbing
def load_env(cfg):
env = {}
if os.path.exists(cfg):
for line in open(cfg):
line = line.strip()
if not line or line.startswith("#") or "=" not in line:
continue
k, v = line.split("=", 1)
env[k.strip()] = v.strip()
for k in ("GIT_HOST", "GIT_USER", "GIT_TOKEN", "SKILL_REPO", "COMPONENTS_REPO"):
if os.environ.get(k):
env[k] = os.environ[k]
return env
def repo_url(env, repo):
host = re.sub(r"^https?://", "", env["GIT_HOST"]).rstrip("/")
cred = (env.get("GIT_USER", "") + ":" if env.get("GIT_USER") else "") + env["GIT_TOKEN"]
return f"https://{cred}@{host}/{repo}.git"
def clone(env, dest):
url = repo_url(env, env["COMPONENTS_REPO"])
r = subprocess.run(["git", "clone", url, dest], capture_output=True, text=True)
if r.returncode != 0:
err = (r.stderr or "").replace(env["GIT_TOKEN"], "***")
sys.exit(f"clone COMPONENTS failed (host reachable / token scope / repo exists?):\n{err[:400]}")
return dest
def commit_push(root, token, msg):
for args in (["config", "user.email", "datasheet-bot@local"],
["config", "user.name", "library-manager"],
["checkout", "-B", "main"], ["add", "-A"]):
subprocess.run(["git", "-C", root] + args, capture_output=True)
if subprocess.run(["git", "-C", root, "diff", "--cached", "--quiet"]).returncode == 0:
print("nothing new to push.")
return
subprocess.run(["git", "-C", root, "commit", "-m", msg], capture_output=True)
r = subprocess.run(["git", "-C", root, "push", "-u", "origin", "main"],
capture_output=True, text=True)
print((r.stdout + r.stderr).replace(token or "", "***").strip() or "pushed.")
def _root(env, args):
"""Return (root_dir, is_temp_clone). Honour --local / --root, else clone."""
if getattr(args, "local", None):
return args.local, False
if getattr(args, "root", None):
return args.root, False
tmp = tempfile.mkdtemp(prefix="components_")
return clone(env, tmp), True
# ------------------------------------------------------------------ folder logic
def find_part_dirs(root, prefix="", typeid=None):
"""Yield (class_folder, part_folder_name, abspath) for part folders under any Class dir
that match a name prefix and/or a typeid suffix."""
if not os.path.isdir(root):
return
for cls in sorted(os.listdir(root)):
cdir = os.path.join(root, cls)
if not os.path.isdir(cdir) or cls.startswith("."):
continue
for name in sorted(os.listdir(cdir)):
pdir = os.path.join(cdir, name)
if not os.path.isdir(pdir):
continue
if prefix and not name.startswith(prefix):
continue
if typeid and not name.endswith(f"_{typeid}"):
continue
yield cls, name, pdir
def cmd_check_mpn(env, args):
root, _ = _root(env, args)
prefix = mpn_make_prefix(args.mpn, args.make)
matches = [(cls, name) for cls, name, _ in find_part_dirs(root, prefix=prefix)]
if matches:
for cls, name in matches:
print(f"EXISTS\t{cls}/{name}")
sys.exit(3)
print("ABSENT")
def cmd_checkout(env, args):
dest = clone(env, args.dest)
print(f"components repo checked out at {dest}")
def cmd_list_type(env, args):
root, _ = _root(env, args)
parts = list(find_part_dirs(root, typeid=args.typeid))
if args.json:
import json
out = [{"class": cls, "tag": name,
"folder": pdir,
"files": sorted(os.listdir(pdir))} for cls, name, pdir in parts]
print(json.dumps(out, indent=2))
return
if not parts:
print(f"no existing parts of typeid {args.typeid}")
return
for cls, name, pdir in parts:
files = ", ".join(sorted(os.listdir(pdir)))
print(f"{cls}/{name} -> {files}")
def place_part(root, folder, typeid):
cls = class_folder(typeid)
if not cls:
sys.exit(f"unknown typeid '{typeid}' (not in taxonomy)")
tag = os.path.basename(os.path.normpath(folder))
cls_dir = os.path.join(root, cls)
# A repo may seed classes as 0-byte placeholder FILES (git can't store empty dirs).
# Replace such a placeholder with a real class directory before adding the part.
if os.path.exists(cls_dir) and not os.path.isdir(cls_dir):
os.remove(cls_dir)
os.makedirs(cls_dir, exist_ok=True)
dest = os.path.join(cls_dir, tag)
if os.path.isdir(dest):
shutil.rmtree(dest)
shutil.copytree(folder, dest)
return cls, tag, dest
def cmd_place_part(env, args):
root, _ = _root(env, args)
cls, tag, dest = place_part(root, args.folder, args.typeid)
print(f"placed {tag} -> {cls}/{tag}")
def cmd_commit_push(env, args):
commit_push(args.root, env.get("GIT_TOKEN", ""), args.message or "library-manager: update components")
def cmd_push_part(env, args):
tmp = tempfile.mkdtemp(prefix="components_")
clone(env, tmp)
cls, tag, _ = place_part(tmp, args.folder, args.typeid)
print(f"placed {tag} -> {cls}/{tag}")
commit_push(tmp, env.get("GIT_TOKEN", ""), args.message or f"library-manager: add {tag}")
def main():
ap = argparse.ArgumentParser()
ap.add_argument("--config", default=CFG_DEFAULT)
sub = ap.add_subparsers(dest="cmd", required=True)
p = sub.add_parser("check-mpn"); p.add_argument("--mpn", required=True)
p.add_argument("--make", required=True); p.add_argument("--local"); p.add_argument("--root")
p = sub.add_parser("checkout"); p.add_argument("--dest", required=True)
p = sub.add_parser("list-type"); p.add_argument("--typeid", required=True)
p.add_argument("--root"); p.add_argument("--local"); p.add_argument("--json", action="store_true")
p = sub.add_parser("place-part"); p.add_argument("--folder", required=True)
p.add_argument("--typeid", required=True); p.add_argument("--root"); p.add_argument("--local")
p = sub.add_parser("commit-push"); p.add_argument("--root", required=True)
p.add_argument("--message")
p = sub.add_parser("push-part"); p.add_argument("--folder", required=True)
p.add_argument("--typeid", required=True); p.add_argument("--message")
args = ap.parse_args()
env = load_env(args.config)
if args.cmd in ("check-mpn", "checkout", "commit-push", "push-part") or \
(args.cmd in ("list-type", "place-part") and not getattr(args, "local", None) and not getattr(args, "root", None)):
for k in ("GIT_HOST", "GIT_TOKEN", "COMPONENTS_REPO"):
if not env.get(k):
sys.exit(f"missing {k} in env/config")
{"check-mpn": cmd_check_mpn, "checkout": cmd_checkout, "list-type": cmd_list_type,
"place-part": cmd_place_part, "commit-push": cmd_commit_push, "push-part": cmd_push_part}[args.cmd](env, args)
if __name__ == "__main__":
main()

4
scripts/push_to_gitea.sh
View File

@ -20,7 +20,7 @@ while [[ $# -gt 0 ]]; do case "$1" in
*) echo "unknown arg: $1" >&2; exit 2;; esac; done *) echo "unknown arg: $1" >&2; exit 2;; esac; done
[[ -z "$REPO" || -z "$SRC" || -z "$HOST" || -z "$TOKEN" ]] && { echo "need --repo, --src, and GIT_HOST/GIT_TOKEN" >&2; exit 2; } [[ -z "$REPO" || -z "$SRC" || -z "$HOST" || -z "$TOKEN" ]] && { echo "need --repo, --src, and GIT_HOST/GIT_TOKEN" >&2; exit 2; }
[[ -d "$SRC" ]] || { echo "ERROR: src not found: $SRC" >&2; exit 2; } [[ -d "$SRC" ]] || { echo "ERROR: src not found: $SRC" >&2; exit 2; }
[[ -z "$MSG" ]] && MSG="Update from datasheet-extractor" [[ -z "$MSG" ]] && MSG="Update from library-manager"
HOST="${HOST#http://}"; HOST="${HOST#https://}"; HOST="${HOST%/}" HOST="${HOST#http://}"; HOST="${HOST#https://}"; HOST="${HOST%/}"
CRED="${USER_:+$USER_:}${TOKEN}" CRED="${USER_:+$USER_:}${TOKEN}"
URL="https://${CRED}@${HOST}/${REPO}.git" URL="https://${CRED}@${HOST}/${REPO}.git"
@ -30,7 +30,7 @@ git clone "$URL" "$WORK" 2>&1 | scrub || { echo "ERROR: clone failed (repo must
cd "$WORK"; git checkout -B main >/dev/null 2>&1 || true cd "$WORK"; git checkout -B main >/dev/null 2>&1 || true
DEST="$WORK${SUBDIR:+/$SUBDIR}"; mkdir -p "$DEST" DEST="$WORK${SUBDIR:+/$SUBDIR}"; mkdir -p "$DEST"
cp -r "$SRC/." "$DEST/" cp -r "$SRC/." "$DEST/"
git config user.email "datasheet-bot@local"; git config user.name "datasheet-extractor" git config user.email "datasheet-bot@local"; git config user.name "library-manager"
git add -A git add -A
if git diff --cached --quiet; then echo "Nothing new for $REPO."; else if git diff --cached --quiet; then echo "Nothing new for $REPO."; else
git commit -m "$MSG" >/dev/null; git push -u origin main 2>&1 | scrub git commit -m "$MSG" >/dev/null; git push -u origin main 2>&1 | scrub