diff --git a/SKILL.md b/SKILL.md index 1f68148..1b9a087 100644 --- a/SKILL.md +++ b/SKILL.md @@ -2,16 +2,16 @@ name: library-manager description: >- Manage Vecmocon's component library. Extract parameters from a component datasheet PDF into the - per-typeid Excel template (125 type-IDs across 18 classes). Checks Gitea for a duplicate - MPN_make, classifies to a typeid, updates that typeid's template (versioning, changelog, - backfill), fills a per-part workbook, loops on human verification, reads the Altium - symbol/footprint refs, assembles a part folder and pushes to Gitea. Can UPDATE an existing part - instead of hard-stopping on a duplicate, and can WRITE the mandatory SOP parameters directly - into a .SchLib symbol (Value, Manufacturer, Operating Temperature, Tolerance, RoHS, Datasheet, - ...) while stripping Ultra-Librarian defaults. Every Description follows Vecmocon's strict + per-typeid Excel template, check Gitea for a duplicate MPN_make, classify to a + typeid, update that typeid's template (versioning, changelog, backfill), fill a per-part + workbook, verify with a human, read the Altium symbol/footprint refs, and push a part folder to + Gitea. Can UPDATE an existing part instead of hard-stopping on a duplicate. WRITES the mandatory + SOP parameters (from the verified Excel) directly into the .SchLib symbol and COMPILES an Altium + integrated library (.IntLib) bundling symbol + footprint, so every part folder holds five files + (workbook, datasheet, .SchLib, .PcbLib, .IntLib). Every Description follows Vecmocon's strict Altium Description Format. Use whenever the user uploads a datasheet, builds/updates a library - entry, adds a template parameter, fills/updates .SchLib parameters, or pushes a part to Gitea. - ALWAYS trigger on "\datasheet", "\library", "\library-manager", or "\schlib". + entry, adds a parameter, fills .SchLib parameters, builds an integrated library, or pushes to + Gitea. ALWAYS trigger on "\datasheet", "\library", "\library-manager", or "\schlib". --- # Library Manager @@ -75,10 +75,16 @@ library repo/ __/ e.g. BAT46WJ_Nexperia_SCH __.xlsx this part's own one-row parameter sheet _data.pdf the datasheet - .SchLib user-provided + .SchLib user-provided, with all parameters written in .PcbLib user-provided + .IntLib integrated library (symbol + footprint, compiled by the skill) ``` +**Every part folder holds five files** — the workbook, the datasheet, the `.SchLib` (with the +full parameter set written into it), the `.PcbLib`, and the compiled `.IntLib`. The `.SchLib` +parameter fill and the `.IntLib` build are **not optional** — they run on every part before the +push (see *Mandatory symbol parameters* and *Build the integrated library* below). + There is **no single master workbook** — each part carries its own sheet inside its folder. Connection + repo names live in `config/gitea.env` (`SKILL_REPO`, `LIBRARY_REPO`), so runs need no per-session token. If the host is unreachable, the git steps fail clearly and write @@ -286,16 +292,23 @@ python scripts/fill_templates.py part.json \ --template assets/template/template.xlsx --dest // --design design.json ``` -Once you have the symbol and its Library Ref, you can also produce the **mandatory symbol -parameters** the SOP requires on the `.SchLib` (Manufacturer, Manufacturer Part, Value, -Tolerance, Operating Temperature, RoHS, Datasheet, Process, Vecmocon Part Code, …) — see -*Mandatory symbol parameters* below. This is optional per run but is how the datasheet values -land in the Altium symbol's properties. +Once you have the symbol and its Library Ref, **always** write the full parameter set into the +`.SchLib` from the verified per-part Excel — see *Mandatory symbol parameters* below. **This is +compulsory on every run; do not ask the engineer whether to do it — just do it.** It is how the +verified workbook values (and the Description) land in the Altium symbol's properties, and the +`.IntLib` is then built from this enriched symbol. -### 7. Assemble the part folder +### 7. Assemble the part folder — build the `.IntLib`, then five files -The staging folder `/` should now hold the four files: the per-part `.xlsx`, the -datasheet (name it `_data.`), the symbol, and the footprint. +The staging folder `/` should hold the per-part `.xlsx`, the datasheet (name it +`_data.`), the enriched symbol (`.SchLib` with parameters written in, step 6), and the +footprint (`.PcbLib`). Now **compile the integrated library** from the enriched symbol + footprint +so the folder carries all **five** files — see *Build the integrated library* below: + +```bash +python scripts/build_intlib.py --schlib //.SchLib \ + --pcblib //.PcbLib --out //.IntLib +``` ### 8. Push to the library repo, under the part's Class @@ -313,8 +326,9 @@ user where it landed. The SOP (§5) requires every schematic symbol to carry a fixed parameter set in its Altium properties — `Manufacturer`, `Manufacturer Part`, `Value`, `Tolerance`, `Operating Temperature`, `ROHS`, `Datasheet`, `Process`, `Vecmocon Part Code`, and the two second-source -fields — with the **Comment** set to the MPN. The skill can stamp these onto the symbol from -the datasheet. +fields — with the **Comment** set to the MPN. The skill stamps these onto the symbol from the +verified workbook. **This step is compulsory on every part build — never ask whether to fill the +`.SchLib` parameters; always do it** before assembling the folder and building the `.IntLib`. This runs **as its own task too**, not only inside new-part creation: whenever the user hands over one or more `.SchLib` files and wants their parameters filled/updated (e.g. "\schlib", "add @@ -370,6 +384,40 @@ round-trip, fall back to `scripts/altium_params.py` (emits an Altium DXP script parameters from inside Altium). Always have the engineer confirm the file opens in Altium — the skill writes Altium's own binary format, so Altium is the final validator. +## Build the integrated library (.IntLib) + +Every part also gets a compiled **integrated library** — one file that bundles the schematic +symbol (with the parameters written in) and its footprint, so the component resolves in Altium +with no separate `.PcbLib` to locate. `scripts/build_intlib.py` compiles it in pure Python, no +Altium needed, and it's the **fifth file** in every part folder. + +```bash +python scripts/build_intlib.py --schlib //.SchLib \ + --pcblib //.PcbLib --out //.IntLib +``` + +How it works and what it needs: + +- Feed it the **enriched `.SchLib`** (after `schlib_write.py` has written the parameters in) so the + integrated symbol carries the full parameter set and the correct ComponentDescription. +- The symbol **must contain a footprint model link** (an Altium RECORD=45 `ModelName` / + `ModelType=PCBLIB` in its `Data` stream — Ultra-Librarian and Altium exports include this). The + builder reads that link to know which footprint to bind, and errors clearly if it's absent — in + that case the symbol has no footprint assigned, so fix the symbol (or re-export it) first. +- It builds the `.IntLib` as an OLE compound file with five streams — the embedded `.schlib` and + `.pcblib` (zlib-compressed at Altium's default level), plus `LibCrossRef.Txt`, `Parameters .bin`, + and `Version.Txt` — reusing a bundled container skeleton + (`assets/templates/intlib_container.IntLib`) for the exact directory layout Altium expects, and a + **FAT-first** compound-file writer (`build_intlib.write_cfb`). Both the outer container and the + embedded symbol are written FAT-first — this matters: a FAT-last layout re-opens fine in olefile + and even standalone in Altium, but Altium's **IntLib extractor** throws "Stream read error" on it. + The two embedded libraries are compressed at zlib's **default level** (`0x789c`); Altium's + decompressor rejects other levels (e.g. level-9 `0x78da`). +- The builder self-validates: it re-opens the output, decompresses both embedded libraries, and + confirms they round-trip and that the cross-reference names the symbol + footprint. Even so, + Altium is the final validator — have the engineer open the `.IntLib` once (or, as a guaranteed + fallback, compile a `.LibPkg` in Altium from the same `.SchLib` + `.PcbLib`). + ## Per-typeid versioning Versioning is **per typeid**, not global. Each typeid carries its own `template_version` and @@ -544,6 +592,12 @@ plain flat push, but it does not merge the changelog or blank the token, so pref `Manufacturer_Part_Number` defaults). Primary path; see `references/schlib_parameters.md`. - `scripts/altium_params.py` — fallback: generate an Altium DelphiScript that stamps the same parameters onto a `.SchLib` from inside Altium (DXP → Run Script). +- `scripts/build_intlib.py` — compile a component's `.SchLib` + `.PcbLib` into an Altium + **integrated library** (`.IntLib`) in pure Python (FAT-first OLE writer + Altium-level zlib); + the fifth file in every part folder. Needs the enriched `.SchLib` (parameters written) with a + footprint model link. Uses `assets/templates/intlib_container.IntLib` as the container skeleton. +- `assets/templates/intlib_container.IntLib` — a known-good single-component `.IntLib` reused + purely as the OLE container skeleton by `build_intlib.py` (all its streams are overwritten). - `scripts/gitea_components.py` — `check-mpn`, `find-part` (locate an existing part to update), `checkout`, `list-type`, `place-part`, `commit-push`, `push-part` (library repo), and `push-skill` (skill repo: token-blanked push + append-only changelog merge). diff --git a/assets/CHANGELOG.xlsx b/assets/CHANGELOG.xlsx index a31adbf..721e445 100644 Binary files a/assets/CHANGELOG.xlsx and b/assets/CHANGELOG.xlsx differ diff --git a/assets/templates/intlib_container.IntLib b/assets/templates/intlib_container.IntLib new file mode 100644 index 0000000..2cb35c5 Binary files /dev/null and b/assets/templates/intlib_container.IntLib differ diff --git a/scripts/build_intlib.py b/scripts/build_intlib.py new file mode 100644 index 0000000..59b7dca --- /dev/null +++ b/scripts/build_intlib.py @@ -0,0 +1,277 @@ +#!/usr/bin/env python3 +"""Compile an Altium **integrated library** (.IntLib) from a .SchLib + .PcbLib, in pure Python. + +An .IntLib is an OLE compound file with five streams: + SchLib/0.schlib 0x02 + zlib() + PCBLib/0.pcblib 0x02 + zlib() + LibCrossRef.Txt symbol -> footprint cross-reference index (length-prefixed strings) + Parameters .bin the Components-panel search index (parameter string) + Version.Txt constant \x00 + uint32(2) + +We don't hand-roll a compound-file writer: we take a known-good single-component .IntLib as a +**container template** and swap in this part's five stream contents, then rebuild the OLE with the +same directory tree (reusing schlib_write.read_container / rebuild, already proven to emit +Altium-valid OLEs). The symbol->footprint linkage lives inside the embedded .SchLib itself +(RECORD=45 model link), so the compiled library resolves the footprint with no external PcbLib. +""" +import argparse, json, os, re, struct, sys, zlib +import olefile +sys.path.insert(0, os.path.join(os.path.dirname(os.path.abspath(__file__)))) +from schlib_write import read_container + +FREESECT = 0xFFFFFFFF; ENDOFCHAIN = 0xFFFFFFFE; FATSECT = 0xFFFFFFFD +SEC = 512; MINI = 64; CUTOFF = 4096 + + +def _pad(b, n): + return b + b"\x00" * ((-len(b)) % n) + + +def write_cfb(entries, content): + """Serialise an OLE/CFB compound file from raw 128-byte directory entries + {sid: bytes}, + reproducing Altium's on-disk SECTOR ORDER: [FAT][directory][miniFAT][mini-stream][big streams]. + (schlib_write.rebuild is spec-valid but places the FAT last; Altium's IntLib reader is strict + about big streams and expects the conventional FAT-first layout, so we match it here.) + Streams < 4096 B live in the mini-stream; >= 4096 B get their own FAT sectors.""" + minis = [sid for sid, c in content.items() if len(c) < CUTOFF] + bigs = [sid for sid, c in content.items() if len(c) >= CUTOFF] + + # mini-stream + mini-FAT + ministream = b""; mini_start = {} + for sid in minis: + mini_start[sid] = len(ministream) // MINI + ministream += _pad(content[sid], MINI) + n_mini = len(ministream) // MINI + minifat = [ENDOFCHAIN] * n_mini + for sid in minis: + st = mini_start[sid]; cnt = (len(content[sid]) + MINI - 1) // MINI or 1 + for k in range(cnt - 1): + minifat[st + k] = st + k + 1 + minifat_bytes = _pad(b"".join(struct.pack(" n_fat * SEC: + n_fat += 1 + total = n_fat + non_fat + if n_fat > 109: + sys.exit("too many FAT sectors for a header-only DIFAT") + + # sector assignment — FAT first, matching Altium + fat_secs = list(range(0, n_fat)); cur = n_fat + dir_secs = list(range(cur, cur + n_dir)); cur += n_dir + mf_secs = list(range(cur, cur + n_mf)); cur += n_mf + ms_secs = list(range(cur, cur + n_ms)); cur += n_ms + big_secs = {} + for sid in bigs: + big_secs[sid] = list(range(cur, cur + big_k[sid])); cur += big_k[sid] + + # patch directory entries (start sector + 64-bit size) + def setentry(e, start, size): + e[116:120] = struct.pack(" 254: + raise ValueError(f"string too long for single-byte length prefix: {s[:40]!r}...") + return struct.pack("