Skill/SKILL.md

---
name: datasheet-extractor
description: >-
  Extract key parameters from component datasheet PDFs into the Vecmocon master
  template (18 component types incl. resistors, capacitors, inductors, diodes,
  transistors, ICs, protection, power modules, connectors, sensors, and more).
  Column A = MPN_make_typeid (make = first word of manufacturer, typeid from the
  taxonomy). Writes one Excel file per type with a Meta sheet holding the template
  version, assembles a per-part DFS folder (datasheet, Altium footprint, Altium symbol), and pushes
  design files to the Gitea DFS repo, Excel outputs to the Parameters repo, and skill
  files plus templates to the Skill_Assets repo. Also appends new parameters to a
  template as a new version. Use WHENEVER the user uploads component datasheets and
  asks to extract parameters, build the parameter sheets or component library, or push
  datasheets to Gitea. ALWAYS trigger when the user types "\datasheet".
---

# Datasheet Extractor

Read component datasheets, fill the customer's master template, assemble each part's
design-file folder, and publish everything to three Gitea repos. Be careful and honest:
put every value in the right column and unit, and flag what a datasheet does not state.

**Pushing is the default.** At the end of a run, push every output to Gitea without asking
for confirmation — that's the whole job. Only pause when there's a genuine need for the
user: a **login wall** while fetching a footprint/symbol, or an **MPN that already exists**
in Gitea (discard vs replace, because replace overwrites data). Nothing else warrants a
prompt.

## Inputs

- **Datasheet PDFs**, one per part. The filename is the part's MPN (search that exact
  number inside a series datasheet to read the right variant).
- No project/version is needed — repos use a **flat layout**.

## The identifier: `MPN_make_typeid`

Column A of every sheet (header `MPN_make_type`) and each DFS folder name is:

```
<MPN>_<make>_<typeid>        e.g.  BAT46WJ_Nexperia_SCH
```

- **make** = the **first word of the manufacturer** name (Texas Instruments → `Texas`,
  Nexperia → `Nexperia`, STMicroelectronics → `STMicroelectronics`).
- **typeid** = the Type ID for the part's subclass, from the taxonomy in
  `references/taxonomy.md` (full source `assets/template/Type_ID.xlsx`). e.g. Schottky →
  `SCH`, TVS → `TVS`, MOSFET → `MOS`, LDO → `LDO`, Common-mode choke → `CMC`.

## Workflow

1. **Read each datasheet.** Get MPN (from filename), manufacturer, and the parameters.
2. **Classify** to one of the 18 **types** (= template sheet names) and to a **subclass**
   from `references/taxonomy.md`; record its **typeid**. The subclass name goes in the
   `Class` column where the sheet has one.
3. **Extract values** into that type's columns (headers come from
   `assets/template/template.xlsx`). Convert to each header's unit; leave blank if the
   datasheet doesn't state it (an honest blank beats a guess).
4. **Build the Excel outputs** with `scripts/fill_templates.py` — one `<Type>.xlsx` per
   type present, column A = `MPN_make_typeid`, plus a **Meta** sheet holding the template
   version. (See Producing outputs.)
5. **Assemble each part's DFS folder** `MPN_make_typeid/` containing `MPN_data` (the
   datasheet), `MPN_fp` (footprint), `MPN_sym` (symbol). See Footprint & symbol.
6. **Push everything to Gitea — automatically.** At the end of a run, push all outputs to
   all three repos with no confirmation step: DFS folders → **DFS**, per-type sheets →
   **Parameters** (merged in), skill files + templates → **Skill_Assets**. The only reasons
   to pause are a login wall or an already-present MPN (discard vs replace); otherwise it
   all just goes. See Pushing to Gitea.

## Producing outputs

Collect what you extracted into `parts.json` (keys match template headers; `typeid` and
`subclass` from the taxonomy; `manufacturer` used for the make tag):

```json
{"parts":[
  {"type":"Diode","mpn":"BAT46WJ","manufacturer":"Nexperia","typeid":"SCH","subclass":"Schottky",
   "values":{"Description":"100 V 250 mA Schottky, SOD323F","Forward Voltage(V)":"0.71","Package":"SOD323F"}}
]}
```

```bash
python scripts/fill_templates.py parts.json \
  --template assets/template/template.xlsx --dest <outputs-dir>
```

Produces one `<Type>.xlsx` per type present (only types with parts), each with a **Meta**
sheet (Template Version, date, type, part count). Deliver these files to the user, and
push them to the Parameters repo.

## Footprint & symbol (for the DFS folder)

For each part, the DFS folder needs `MPN_fp` (footprint) and `MPN_sym` (symbol) beside
`MPN_data` (the datasheet). **Deliver these as Altium files** — an Altium PCB footprint
(`.PcbLib`) and schematic symbol (`.SchLib`), or a single integrated library (`.IntLib`)
that carries both — because that's what the design team actually consumes. Get them in
this order:

1. **Scout which sources actually hold this part first — before any login.** Search
   across SnapEDA, Ultra Librarian, Component Search Engine, DigiKey (its "PCB Symbol,
   Footprint & 3D Model" links), and the manufacturer's own CAD downloads, and for each
   note whether *this exact MPN* has a downloadable **Altium** symbol + footprint, and
   whether grabbing it needs an account. The point is to find where the model lives before
   asking the user to do anything.

2. **Surface what you found, then handle the login.** Tell the user which sources have the
   part in Altium form — e.g. "BAT46WJ: SnapEDA and Ultra Librarian both have an Altium
   symbol + footprint; DigiKey lists a model too" — so they can pick. Then get the file:
   - If you're driving the user's own signed-in browser (e.g. Claude in Chrome) and it's
     already logged into that source, just download — no password ever changes hands.
   - Otherwise **ask the user to sign in to the source they picked** (or to grab the file
     and drop it in), then continue. Never type, store, or ask for their password, and
     never write credentials into this skill — it's pushed to Skill_Assets, and a secret in
     a repo is a real leak (same reason the `GIT_TOKEN` is blanked before syncing).

   Once you have a candidate, **cross-check** its pad/pin count and key package dimensions
   against the datasheet's package drawing before trusting it — library models are
   sometimes wrong, or drawn for a different variant of the series.

3. **If no good model exists, generate the footprint/symbol yourself** as Altium files
   from the datasheet's package drawing and pinout — but only when you can do it reliably
   — then **ask the user to verify** it against the datasheet before it's trusted, since a
   generated land pattern that's slightly off is worse than an obvious blank.

4. **If none of that is possible**, still create the folder with the datasheet and **tell
   the user that MPN needs a manual footprint/symbol build**, leaving a clearly-named
   placeholder or listing it so nothing silently goes missing.

Keep the datasheet's extension on `MPN_data` (e.g. `BAT46WJ_data.pdf`). Name the
footprint/symbol files `MPN_fp` / `MPN_sym` with the Altium extension the model uses
(`.PcbLib` / `.SchLib`, or a single `.IntLib` when the source bundles both).

## Pushing to Gitea

Connection + repos are pre-configured in `config/gitea.env` (host, user, token, and the
three repos), so runs need no per-session token. Layout is **flat** (no project/version).

**Every run ends by pushing all three repos — no "shall I push?" step.** The user has
already asked for this to be automatic; don't re-ask. The reconcile step self-guards
against the one risky case (overwriting an existing part), so a straight push is safe by
default.

The three repos split into two kinds, and they're pushed differently:

- **DFS** and **Parameters** are **MPN-indexed** — DFS has one `MPN_make_typeid/` folder
  per part, Parameters has one row per part (keyed on column A) inside each `<Type>.xlsx`.
  Because a re-run can collide with a part that's already there, these go through
  `scripts/gitea_reconcile.py`, which is dedup-aware and **merge-safe** (it never blows
  away parts it isn't touching). See *Handling parts already in Gitea* below.
- **Skill_Assets** holds the skill itself + templates — it is **not** per-part, so there's
  nothing MPN-level to reconcile. Push it with `scripts/push_to_gitea.sh` as before.

```bash
# Skill files + templates -> Skill_Assets (flat copy; no dedup needed)
bash scripts/push_to_gitea.sh --repo "$SKILL_ASSETS_REPO" --src <skill-dir> --message "Sync skill assets"
```

The push script clones the repo, copies the source contents in (flat), commits, and
pushes; the repo must already exist. If the host is unreachable (e.g. Cowork without the
domain allowlisted) it fails clearly and leaves the staged files for a manual push.

**Push the skill files organised** to Skill_Assets: keep the skill's own structure
(`SKILL.md`, `scripts/`, `assets/`, `references/`, `config/`) — but do **not** push the
real token. Before syncing skill assets, blank the `GIT_TOKEN` line in the copy you push,
or push everything except `config/gitea.env`.

> Why not push Parameters with the flat script? `fill_templates.py` writes a `<Type>.xlsx`
> holding **only this run's** rows. Copied flat over the repo, it would overwrite the file
> and delete every previously-stored part of that type. `gitea_reconcile.py` merges into
> the repo's existing sheet instead, so only the parts you decided on change.

## Handling parts already in Gitea (discard vs replace)

A part's identity is its tag `MPN_make_typeid` (same tag = same part). A re-run that
brings in a *new* MPN is not a conflict — it just gets added, and you push without asking.
The prompt only exists for the case where an MPN **already lives in DFS/Parameters**,
because "replace" overwrites data that's already on the server.

The simplest path is a single push. It clones the repos, adds everything new, and stops
**only** if it hits an existing MPN it can't resolve on its own:

```bash
python scripts/gitea_reconcile.py --parts parts.json --dfs-src <dfs-stage> \
  --template assets/template/template.xlsx --push
```

- **No existing MPNs** → it adds all the folders/rows and pushes. No prompt. Done.
- **An MPN already exists** → it refuses (listing which) so you can ask the user, per MPN,
  **discard** (keep what's in Gitea, drop the new copy) or **replace** (overwrite that MPN's
  DFS folder *and* its Parameters row with the fresh extraction). Every other folder/row is
  always left alone — the Parameters sheet is merged, never wholesale-overwritten.

Feed the user's answers back as a small JSON map (`tag → replace|discard`):

```bash
# decisions.json  e.g.  {"BAT46WJ_Nexperia_SCH": "replace", "1N4148_onsemi_SCH": "discard"}
python scripts/gitea_reconcile.py --parts parts.json --dfs-src <dfs-stage> \
  --template assets/template/template.xlsx --decisions decisions.json --push
```

**For fully unattended runs**, skip the prompt entirely with a standing policy — every
conflict resolved the same way, no questions:

```bash
python scripts/gitea_reconcile.py --parts parts.json --dfs-src <dfs-stage> \
  --template assets/template/template.xlsx --on-conflict replace --push
```

`--on-conflict replace` (or `discard`) applies to all conflicts; an explicit per-MPN entry
in `--decisions` still wins over the blanket policy. Use `--report conflicts.json` (without
`--push`) if you ever want to preview conflicts first. Host unreachable → it fails clearly,
same as the push script.

## Updating a template (new parameter)

When the user wants a new parameter on a type, **append it at the end** of that type's
sheet and bump the version:

```bash
python scripts/append_parameter.py --type Diode --param "Reverse Recovery Time(ns)" \
  --template assets/template/template.xlsx
```

This adds the column at the end and increments `assets/template/VERSION` (v1 → v2 → …).
Then **push the updated `assets/template/template.xlsx` + `VERSION` to the Skill_Assets
repo** so the new template version is archived. New outputs will record the new version in
their Meta sheet automatically.

## Resources

- `assets/template/template.xlsx` — master template, one sheet per type (source of headers).
- `assets/template/Type_ID.xlsx` + `references/taxonomy.md` — Class → Subclass → Type ID.
- `assets/template/VERSION` — current template version (integer; Meta sheet shows `vN`).
- `scripts/fill_templates.py` — build per-type Excel outputs (+ Meta sheet); also exposes
  the row/style helpers (`part_to_row`, `read_type_rows`, `write_type_workbook`) that the
  reconcile step reuses so merged sheets look identical to freshly generated ones.
- `scripts/gitea_reconcile.py` — add new MPNs and push automatically; only stops on an MPN
  that already exists (discard/replace), or run unattended with `--on-conflict replace`.
  Merge-safe: other rows/folders untouched.
- `scripts/push_to_gitea.sh` — push a folder's contents to a Gitea repo (flat). Used for
  Skill_Assets (not MPN-indexed).
- `scripts/append_parameter.py` — append a parameter to a template + bump version.
- `config/gitea.env` — host, user, token, and the DFS / Parameters / Skill_Assets repos (**secret**).