schlib: symbol now carries ALL filled workbook params (new schlib_params_from_xlsx.py; schlib_write transliterates non-ANSI glyphs like Ω->Ohm); docs updated (by admin)

SKILL.md

@@ -286,11 +286,25 @@ python scripts/fill_templates.py part.json \
     --template assets/template/template.xlsx --dest <stage>/<tag>/ --design design.json
 ```
 
-Once you have the symbol and its Library Ref, you can also produce the **mandatory symbol
+Once you have the symbol and its Library Ref, you can also write the symbol's Altium parameters
-parameters** the SOP requires on the `.SchLib` (Manufacturer, Manufacturer Part, Value,
+onto the `.SchLib`. **House rule: the symbol carries _every_ parameter the workbook was filled
-Tolerance, Operating Temperature, RoHS, Datasheet, Process, Vecmocon Part Code, …) — see
+with from the datasheet — not just the fixed SOP set.** Build the parameter set straight from the
-*Mandatory symbol parameters* below. This is optional per run but is how the datasheet values
+finished `<tag>.xlsx` and write it in:
-land in the Altium symbol's properties.
+
+```bash
+python scripts/schlib_params_from_xlsx.py --xlsx <stage>/<tag>/<tag>.xlsx \
+    --component <LibraryRef> --set "Process=<Reflow|Wave>" --set "Datasheet=<url-or-doc-ref>" \
+    --out params.json
+python scripts/schlib_write.py --schlib <in>.SchLib --params params.json --out <stage>/<tag>/<sym>.SchLib
+```
+
+`schlib_params_from_xlsx.py` keeps every filled datasheet column (exact header as the parameter
+name), drops the identity/version/model-link columns, renames `Rohs compliance → ROHS` and
+`Operating Temp(°C) → Operating Temperature`, and merges in the SOP-only fields (`Manufacturer
+Part` = MPN automatically; `Process` / `Datasheet` / `Value` / `Vecmocon Part Code` via `--set`).
+See *Mandatory symbol parameters* below and `references/schlib_parameters.md` for the full method
+(incl. the ANSI glyph transliteration, e.g. `Ω → Ohm`). This is how the datasheet values land in
+the Altium symbol's properties.
 
 ### 7. Assemble the part folder
 
@@ -313,8 +327,11 @@ user where it landed.
 The SOP (§5) requires every schematic symbol to carry a fixed parameter set in its Altium
 properties — `Manufacturer`, `Manufacturer Part`, `Value`, `Tolerance`, `Operating
 Temperature`, `ROHS`, `Datasheet`, `Process`, `Vecmocon Part Code`, and the two second-source
-fields — with the **Comment** set to the MPN. The skill can stamp these onto the symbol from
+fields — with the **Comment** set to the MPN. **On top of that fixed minimum, the symbol also
-the datasheet.
+carries every other parameter the part's workbook was filled with from the datasheet** (e.g. a
+CMC's `Rated Current(A)`, `DC Resistance(mΩ)`, `Package`, `ESD Withstand Voltage(kV)`): the
+Altium properties mirror the `<tag>.xlsx` row. The skill can stamp all of these onto the symbol
+from the workbook + datasheet.
 
 This runs **as its own task too**, not only inside new-part creation: whenever the user hands
 over one or more `.SchLib` files and wants their parameters filled/updated (e.g. "\schlib", "add
@@ -340,12 +357,19 @@ Leave any genuinely-unknown field blank — the SOP hides blank parameters, so a
 empty until filled. The full method for the second-source search is in
 `references/schlib_parameters.md`.
 
-Collect the values into a `params.json` and write them into the symbol:
+Build the `params.json` from the finished workbook (so it carries all the filled datasheet
+parameters), then write it into the symbol:
 
 ```bash
+python scripts/schlib_params_from_xlsx.py --xlsx <stage>/<tag>/<tag>.xlsx \
+    --component <LibraryRef> --set "Process=<Reflow|Wave>" --set "Datasheet=<url-or-doc-ref>" \
+    [--set "Value=<value>"] [--set "Vecmocon Part Code=<code>"] --out params.json
 python scripts/schlib_write.py --schlib <in>.SchLib --params params.json --out <stage>/<tag>/<sym>.SchLib
 ```
 
+(For a bare `.SchLib`-only task with no workbook, you can still hand-write `params.json` — the
+shape is in `references/schlib_parameters.md`.)
+
 Deliver the resulting `.SchLib`; the engineer opens it in Altium once to confirm it loads, then
 **Saves to Server** with a revision note. The full parameter set, each value's source, the
 `params.json` shape (incl. the `remove` list), and the mini-stream size caveat are in
@@ -523,9 +547,14 @@ plain flat push, but it does not merge the changelog or blank the token, so pref
 - `scripts/append_parameter.py` — append parameter(s) to a typeid, bump its versions, write
   the changelog.
 - `scripts/altium_refs.py` — read Library/Footprint Ref from `.SchLib`/`.PcbLib`.
-- `scripts/schlib_write.py` — write the SOP mandatory parameters **directly into a `.SchLib`**
+- `scripts/schlib_params_from_xlsx.py` — build the symbol `params.json` from the finished
-  (pure-Python OLE rebuild; removes the Ultra-Librarian `Manufacturer_Name` /
+  per-part `<tag>.xlsx`, so the `.SchLib` carries **every filled datasheet parameter** (not just
-  `Manufacturer_Part_Number` defaults). Primary path; see `references/schlib_parameters.md`.
+  the SOP minimum); skips identity/version/model-link columns, renames a couple to their SOP
+  names, and merges in the SOP-only fields (`Manufacturer Part`, `Process`, `Datasheet`, …).
+- `scripts/schlib_write.py` — write the parameters **directly into a `.SchLib`** (pure-Python OLE
+  rebuild; removes the Ultra-Librarian `Manufacturer_Name` / `Manufacturer_Part_Number` defaults;
+  transliterates non-ANSI unit glyphs, e.g. `Ω → Ohm`). Primary path; see
+  `references/schlib_parameters.md`.
 - `scripts/altium_params.py` — fallback: generate an Altium DelphiScript that stamps the same
   parameters onto a `.SchLib` from inside Altium (DXP → Run Script).
 - `scripts/gitea_components.py` — `check-mpn`, `find-part` (locate an existing part to

references/schlib_parameters.md

@@ -5,6 +5,50 @@ parameters in its component properties (the panel shown in Altium: *Properties
 This file defines that set, where each value comes from, and how the skill stamps them onto the
 `.SchLib` symbol.
 
+## The symbol mirrors the workbook (fill everything the datasheet gave)
+
+**House rule: the symbol's Altium properties must carry _every_ parameter that was filled into
+the part's per-typeid workbook from the datasheet — not just the fixed SOP §5 set.** So a CMC
+symbol also gets `Rated Current(A)`, `Rated Voltage(V)`, `DC Resistance(mΩ)`, `Package`,
+`ESD Withstand Voltage(kV)`, … — whatever that typeid's sheet holds and the datasheet filled.
+The SOP set below is the **minimum**; the workbook is the **source of truth** for the rest.
+
+Build the parameter set straight from the finished `<tag>.xlsx` with
+`scripts/schlib_params_from_xlsx.py`, which reads the one data row and keeps every **non-empty**
+column, then hand its output to `schlib_write.py`:
+
+```bash
+python scripts/schlib_params_from_xlsx.py --xlsx <stage>/<tag>/<tag>.xlsx \
+    --component <LibraryRef> \
+    --set "Process=Reflow" --set "Datasheet=<url-or-doc-ref>" \
+    [--set "Value=<value>"] [--set "Vecmocon Part Code=<code>"] \
+    --out params.json
+python scripts/schlib_write.py --schlib <in>.SchLib --params params.json --out <stage>/<tag>/<sym>.SchLib
+```
+
+What the builder does:
+
+- **Keeps every filled datasheet column** as a symbol parameter, using the **exact sheet header
+  as the parameter name** (e.g. `Rated Current(A)`) so the symbol and the workbook stay
+  traceably identical. Empty columns are left out (the SOP hides blank parameters).
+- **Never writes** the identity / versioning / model-link columns — `MPN_make_type`,
+  `Skill Version`, `Template Version`, `Library Ref/Path`, `Footprint Ref/Path` (Library Ref is
+  the symbol's own name and the footprint is the linked PCB model, not a text property).
+- **Renames** the two columns whose Altium/SOP name differs — `Rohs compliance → ROHS`,
+  `Operating Temp(°C) → Operating Temperature` — value copied through unchanged.
+- **Merges in the SOP-only fields the sheet doesn't hold**: `Manufacturer Part` (= the MPN,
+  recovered from the tag) is added automatically; pass `Process`, `Datasheet`, and (if known)
+  `Value` / `Vecmocon Part Code` via `--set` or a `--sop` JSON. A non-empty override wins over a
+  sheet value; an empty one is ignored.
+
+Glyph note: Altium stores parameter text as single-byte ANSI, so `schlib_write.py` transliterates
+the few unit glyphs that aren't representable — the ohm sign `Ω → Ohm` (so `DC Resistance(mΩ)`
+lands as `DC Resistance(mOhm)`) and a Greek micro `μ → u`; `±`, `°` and the latin-1 micro sign
+pass through unchanged. So a couple of symbol parameter names are the ASCII form of the sheet
+header — expected, not a mismatch.
+
+The rest of this file describes the SOP §5 minimum set and where each value comes from.
+
 How the parameters get in: the skill writes them **directly into the `.SchLib` in pure Python**
 via `scripts/schlib_write.py` — it rebuilds the OLE compound file around the enlarged component
 `Data` stream while preserving every other byte (all other streams, the directory tree, the

scripts/schlib_params_from_xlsx.py

@@ -0,0 +1,127 @@
+#!/usr/bin/env python3
+"""Build a schlib_write params.json from a part's per-typeid Excel sheet, so the .SchLib symbol
+carries **every parameter that was filled from the datasheet** — not only the fixed SOP §5 set.
+
+Vecmocon's rule (per the engineer): the schematic symbol's Altium properties must mirror the
+per-part workbook. So this reads the one data row of the part's typeid sheet, keeps every
+**non-empty** datasheet parameter, drops the bookkeeping/identity/model-link columns that don't
+belong in the symbol's parameter list, renames the few columns whose Altium/SOP name differs,
+and merges in the SOP-only fields that don't live in the sheet (Manufacturer Part = MPN, Process,
+Datasheet, Value, Vecmocon Part Code). The result is a params.json consumed by schlib_write.py.
+
+Columns never written as symbol parameters (identity / versioning / Altium model links — those
+are the symbol's Library Ref and the linked PCB footprint, not text properties):
+  MPN_make_type, Skill Version, Template Version,
+  Library Ref, Library Path, Footprint Ref, Footprint Path
+
+Column-name -> Altium/SOP parameter-name renames (value copied through unchanged):
+  "Rohs compliance"    -> "ROHS"
+  "Operating Temp(°C)" -> "Operating Temperature"
+Everything else keeps its exact sheet header as the parameter name (e.g. "Rated Current(A)",
+"ESD Withstand Voltage(kV)", "Package", "Description", "Manufacturer") so the symbol and the
+workbook stay traceably identical.
+
+Usage:
+  python schlib_params_from_xlsx.py --xlsx PART.xlsx [--component LIBREF] \
+      [--sop sop.json] [--set "Process=Reflow" --set "Datasheet=onsemi EMI8141/D"] \
+      [--out params.json]
+
+--sop / --set carry the SOP-only fields the sheet doesn't hold. --set wins over --sop; both are
+overlaid on top of the sheet-derived params, but a sheet value is only overridden by a non-empty
+override. Empty override values are ignored (SOP hides blanks — a gap simply stays empty).
+"""
+import argparse, json, os, sys
+import openpyxl
+
+# Columns that must never become symbol text parameters.
+SKIP_COLS = {
+    "MPN_make_type", "Skill Version", "Template Version",
+    "Library Ref", "Library Path", "Footprint Ref", "Footprint Path",
+}
+# Sheet header -> Altium/SOP parameter name (value unchanged).
+RENAME = {
+    "Rohs compliance": "ROHS",
+    "Operating Temp(°C)": "Operating Temperature",
+}
+DEFAULT_REMOVE = ["Manufacturer_Name", "Manufacturer_Part_Number"]
+
+
+def _clean(v):
+    if v is None:
+        return ""
+    return str(v).strip()
+
+
+def _mpn_from_tag(tag):
+    """tag = <MPN>_<make>_<typeid>; typeid + make are the last two tokens, so the MPN is the
+    remainder — robust even when the MPN itself contains underscores."""
+    parts = tag.rsplit("_", 2)
+    return parts[0] if len(parts) == 3 else tag
+
+
+def build(xlsx, component=None, sop=None, sets=None):
+    wb = openpyxl.load_workbook(xlsx, read_only=True, data_only=True)
+    # the parameter sheet is the typeid sheet (first sheet that isn't Version History)
+    sheet = next((s for s in wb.sheetnames if s != "Version History"), wb.sheetnames[0])
+    ws = wb[sheet]
+    rows = list(ws.iter_rows(values_only=True))
+    if len(rows) < 2:
+        sys.exit(f"{xlsx}: sheet '{sheet}' has no data row")
+    headers = [_clean(h) for h in rows[0]]
+    values = list(rows[1])
+    row = {h: _clean(values[i]) if i < len(values) else "" for i, h in enumerate(headers) if h}
+
+    tag = row.get("MPN_make_type", "")
+    mpn = _mpn_from_tag(tag) if tag else ""
+
+    params = {}
+    for h in headers:
+        if not h or h in SKIP_COLS:
+            continue
+        val = row.get(h, "")
+        if val == "":
+            continue                       # honest blank — leave it out
+        params[RENAME.get(h, h)] = val
+
+    # SOP-only fields the sheet doesn't carry. Manufacturer Part = MPN by default.
+    overlay = {}
+    if mpn:
+        overlay["Manufacturer Part"] = mpn
+    if sop:
+        overlay.update({k: _clean(v) for k, v in sop.items()})
+    for kv in (sets or []):
+        if "=" in kv:
+            k, v = kv.split("=", 1)
+            overlay[k.strip()] = v.strip()
+    for k, v in overlay.items():
+        if v != "":                        # non-empty override wins; empty is ignored
+            params[k] = v
+
+    out = {"comment": mpn, "parameters": params, "remove": DEFAULT_REMOVE}
+    if component:
+        out["component"] = component
+    return out
+
+
+def main():
+    ap = argparse.ArgumentParser()
+    ap.add_argument("--xlsx", required=True, help="the part's per-typeid .xlsx")
+    ap.add_argument("--component", help="symbol Library Ref (targets one component; omit -> all)")
+    ap.add_argument("--sop", help="JSON file of SOP-only fields (Process, Datasheet, Value, ...)")
+    ap.add_argument("--set", dest="sets", action="append", default=[],
+                    help='SOP-only field as "Name=Value" (repeatable; overrides --sop)')
+    ap.add_argument("--out", help="write params.json here (default: stdout)")
+    a = ap.parse_args()
+    sop = json.load(open(a.sop, encoding="utf-8")) if a.sop else None
+    res = build(a.xlsx, component=a.component, sop=sop, sets=a.sets)
+    text = json.dumps(res, indent=2, ensure_ascii=False)
+    if a.out:
+        open(a.out, "w", encoding="utf-8").write(text + "\n")
+        print(f"wrote {a.out}  ({len(res['parameters'])} parameters"
+              + (f", component={a.component}" if a.component else "") + ")")
+    else:
+        print(text)
+
+
+if __name__ == "__main__":
+    main()

scripts/schlib_write.py

@@ -29,7 +29,7 @@ IMPORTANT: this writes Altium's own binary format from outside Altium. It is val
 re-open as a well-formed OLE with every other stream byte-identical, but ALWAYS open the result
 in Altium once to confirm it loads before relying on it.
 """
-import argparse, json, struct, sys, hashlib
+import argparse, json, struct, sys, hashlib, unicodedata
 import olefile
 
 FREESECT=0xFFFFFFFF; ENDOFCHAIN=0xFFFFFFFE; FATSECT=0xFFFFFFFD
@@ -90,10 +90,30 @@ def _uid(name):
     return (h+"ABCDEFGH")[:8]
 
 
+# Altium stores parameter text as single-byte ANSI (≈ latin-1). A few unit glyphs that show up
+# in the Excel column names (the ohm sign Ω, a Greek micro µ) aren't representable there, so map
+# them to ASCII before encoding; ±, °, and the latin-1 micro sign (0xB5) pass through unchanged.
+_GLYPH={'Ω':'Ohm','Ω':'Ohm','μ':'u'}
+
+def _enc(s):
+    s=''.join(_GLYPH.get(c,c) for c in s)
+    try:
+        return s.encode('latin-1')
+    except UnicodeEncodeError:                      # transliterate any remaining non-ANSI glyph
+        out=[]
+        for c in s:
+            try:
+                c.encode('latin-1'); out.append(c)
+            except UnicodeEncodeError:
+                d=unicodedata.normalize('NFKD',c).encode('ascii','ignore').decode('ascii')
+                out.append(d or '_')
+        return ''.join(out).encode('latin-1')
+
+
 def _param_record(idx, name, value):
     s=(f"|RECORD=41|IndexInSheet={idx}|OwnerPartId=1|Justification=4|FontID=2|IsHidden=T"
        f"|Text={value}|Name={name}|UniqueID={_uid(name)}")
-    payload=s.encode('latin-1')+b'\x00'
+    payload=_enc(s)+b'\x00'
     return struct.pack('<I',len(payload))+payload
 
 
@@ -240,6 +260,7 @@ def write_params(schlib, params_json, out):
         if e[-1].lower()=="data":
             t=ole.openstream(e).read().decode('latin-1','ignore')
             for nm in fields:
+                nm=_enc(nm).decode('latin-1')      # compare against how the name was actually written
                 if f"|Name={nm}|" not in t and f"|Name={nm}\x00" not in t and f"Name={nm}" not in t:
                     ok=False
     ole.close()