149 lines
7.4 KiB
Markdown
149 lines
7.4 KiB
Markdown
---
|
|
name: datasheet-extractor
|
|
description: >-
|
|
Extract key parameters from component datasheet PDFs into the Vecmocon master
|
|
template (18 component types incl. resistors, capacitors, inductors, diodes,
|
|
transistors, ICs, protection, power modules, connectors, sensors, and more).
|
|
Column A = MPN_make_typeid (make = first word of manufacturer, typeid from the
|
|
taxonomy). Writes one Excel file per type with a Meta sheet holding the template
|
|
version, assembles a per-part DFS folder (datasheet, footprint, symbol), and pushes
|
|
design files to the Gitea DFS repo, Excel outputs to the Parameters repo, and skill
|
|
files plus templates to the Skill_Assets repo. Also appends new parameters to a
|
|
template as a new version. Use WHENEVER the user uploads component datasheets and
|
|
asks to extract parameters, build the parameter sheets or component library, or push
|
|
datasheets to Gitea. ALWAYS trigger when the user types "\datasheet".
|
|
---
|
|
|
|
# Datasheet Extractor
|
|
|
|
Read component datasheets, fill the customer's master template, assemble each part's
|
|
design-file folder, and publish everything to three Gitea repos. Be careful and honest:
|
|
put every value in the right column and unit, and flag what a datasheet does not state.
|
|
|
|
## Inputs
|
|
|
|
- **Datasheet PDFs**, one per part. The filename is the part's MPN (search that exact
|
|
number inside a series datasheet to read the right variant).
|
|
- No project/version is needed — repos use a **flat layout**.
|
|
|
|
## The identifier: `MPN_make_typeid`
|
|
|
|
Column A of every sheet (header `MPN_make_type`) and each DFS folder name is:
|
|
|
|
```
|
|
<MPN>_<make>_<typeid> e.g. BAT46WJ_Nexperia_SCH
|
|
```
|
|
|
|
- **make** = the **first word of the manufacturer** name (Texas Instruments → `Texas`,
|
|
Nexperia → `Nexperia`, STMicroelectronics → `STMicroelectronics`).
|
|
- **typeid** = the Type ID for the part's subclass, from the taxonomy in
|
|
`references/taxonomy.md` (full source `assets/template/Type_ID.xlsx`). e.g. Schottky →
|
|
`SCH`, TVS → `TVS`, MOSFET → `MOS`, LDO → `LDO`, Common-mode choke → `CMC`.
|
|
|
|
## Workflow
|
|
|
|
1. **Read each datasheet.** Get MPN (from filename), manufacturer, and the parameters.
|
|
2. **Classify** to one of the 18 **types** (= template sheet names) and to a **subclass**
|
|
from `references/taxonomy.md`; record its **typeid**. The subclass name goes in the
|
|
`Class` column where the sheet has one.
|
|
3. **Extract values** into that type's columns (headers come from
|
|
`assets/template/template.xlsx`). Convert to each header's unit; leave blank if the
|
|
datasheet doesn't state it (an honest blank beats a guess).
|
|
4. **Build the Excel outputs** with `scripts/fill_templates.py` — one `<Type>.xlsx` per
|
|
type present, column A = `MPN_make_typeid`, plus a **Meta** sheet holding the template
|
|
version. (See Producing outputs.)
|
|
5. **Assemble each part's DFS folder** `MPN_make_typeid/` containing `MPN_data` (the
|
|
datasheet), `MPN_fp` (footprint), `MPN_sym` (symbol). See Footprint & symbol.
|
|
6. **Push to the three Gitea repos** (flat): DFS folders → **DFS**, Excel outputs →
|
|
**Parameters**, skill files + templates → **Skill_Assets**. See Pushing to Gitea.
|
|
|
|
## Producing outputs
|
|
|
|
Collect what you extracted into `parts.json` (keys match template headers; `typeid` and
|
|
`subclass` from the taxonomy; `manufacturer` used for the make tag):
|
|
|
|
```json
|
|
{"parts":[
|
|
{"type":"Diode","mpn":"BAT46WJ","manufacturer":"Nexperia","typeid":"SCH","subclass":"Schottky",
|
|
"values":{"Description":"100 V 250 mA Schottky, SOD323F","Forward Voltage(V)":"0.71","Package":"SOD323F"}}
|
|
]}
|
|
```
|
|
|
|
```bash
|
|
python scripts/fill_templates.py parts.json \
|
|
--template assets/template/template.xlsx --dest <outputs-dir>
|
|
```
|
|
|
|
Produces one `<Type>.xlsx` per type present (only types with parts), each with a **Meta**
|
|
sheet (Template Version, date, type, part count). Deliver these files to the user, and
|
|
push them to the Parameters repo.
|
|
|
|
## Footprint & symbol (for the DFS folder)
|
|
|
|
For each part, the DFS folder needs `MPN_fp` (footprint) and `MPN_sym` (symbol) beside
|
|
`MPN_data` (the datasheet). Get them like this, in order:
|
|
|
|
1. **Search the web** for the part's footprint and schematic symbol by MPN (e.g. SnapEDA,
|
|
Ultra Librarian, Component Search Engine, the manufacturer's CAD models). If found,
|
|
**cross-check** the footprint's pad/pin count and dimensions against the datasheet's
|
|
package drawing before using it.
|
|
2. **If not found, generate** the footprint/symbol yourself from the datasheet's package
|
|
and pinout **only if you can do so reliably**, then cross-check with the user before
|
|
trusting it.
|
|
3. **If neither is possible**, still create the folder with the datasheet, and **tell the
|
|
user that the footprint and/or symbol needs a manual build** for that MPN (leave a
|
|
clearly-named placeholder or omit and list it).
|
|
|
|
Keep the datasheet's extension on `MPN_data` (e.g. `BAT46WJ_data.pdf`). Name footprint/
|
|
symbol files `MPN_fp` / `MPN_sym` with whatever extension the source/format uses.
|
|
|
|
## Pushing to Gitea
|
|
|
|
Connection + repos are pre-configured in `config/gitea.env` (host, user, token, and the
|
|
three repos), so runs need no per-session token. Layout is **flat** (no project/version).
|
|
Use `scripts/push_to_gitea.sh` (git over HTTPS), once per repo:
|
|
|
|
```bash
|
|
# 1) design files -> DFS repo (staging dir holds MPN_make_typeid/ folders)
|
|
bash scripts/push_to_gitea.sh --repo "$DFS_REPO" --src <dfs-stage> --message "Add datasheets/fp/sym"
|
|
# 2) Excel outputs -> Parameters repo
|
|
bash scripts/push_to_gitea.sh --repo "$PARAMS_REPO" --src <outputs-dir> --message "Update parameters"
|
|
# 3) skill files + templates -> Skill_Assets repo
|
|
bash scripts/push_to_gitea.sh --repo "$SKILL_ASSETS_REPO" --src <skill-dir> --message "Sync skill assets"
|
|
```
|
|
|
|
`$DFS_REPO`, `$PARAMS_REPO`, `$SKILL_ASSETS_REPO` come from `config/gitea.env`. The
|
|
script clones the repo, copies the source contents in (flat), commits, and pushes; the
|
|
repos must already exist. If the host is unreachable (e.g. Cowork without the domain
|
|
allowlisted) it fails clearly and leaves the staged files for a manual push.
|
|
|
|
**Push the skill files organised** to Skill_Assets: keep the skill's own structure
|
|
(`SKILL.md`, `scripts/`, `assets/`, `references/`, `config/`) — but do **not** push the
|
|
real token. Before syncing skill assets, blank the `GIT_TOKEN` line in the copy you push,
|
|
or push everything except `config/gitea.env`.
|
|
|
|
## Updating a template (new parameter)
|
|
|
|
When the user wants a new parameter on a type, **append it at the end** of that type's
|
|
sheet and bump the version:
|
|
|
|
```bash
|
|
python scripts/append_parameter.py --type Diode --param "Reverse Recovery Time(ns)" \
|
|
--template assets/template/template.xlsx
|
|
```
|
|
|
|
This adds the column at the end and increments `assets/template/VERSION` (v1 → v2 → …).
|
|
Then **push the updated `assets/template/template.xlsx` + `VERSION` to the Skill_Assets
|
|
repo** so the new template version is archived. New outputs will record the new version in
|
|
their Meta sheet automatically.
|
|
|
|
## Resources
|
|
|
|
- `assets/template/template.xlsx` — master template, one sheet per type (source of headers).
|
|
- `assets/template/Type_ID.xlsx` + `references/taxonomy.md` — Class → Subclass → Type ID.
|
|
- `assets/template/VERSION` — current template version (integer; Meta sheet shows `vN`).
|
|
- `scripts/fill_templates.py` — build per-type Excel outputs (+ Meta sheet).
|
|
- `scripts/push_to_gitea.sh` — push a folder's contents to a Gitea repo (flat).
|
|
- `scripts/append_parameter.py` — append a parameter to a template + bump version.
|
|
- `config/gitea.env` — host, user, token, and the DFS / Parameters / Skill_Assets repos (**secret**).
|