Sync datasheet-extractor skill (SKILL.md, scripts, assets, references, config; token blanked)
datasheet-extractor
commit
7fc1ae0fa7
|
|
@ -0,0 +1,148 @@
|
||||||
|
---
|
||||||
|
name: datasheet-extractor
|
||||||
|
description: >-
|
||||||
|
Extract key parameters from component datasheet PDFs into the Vecmocon master
|
||||||
|
template (18 component types incl. resistors, capacitors, inductors, diodes,
|
||||||
|
transistors, ICs, protection, power modules, connectors, sensors, and more).
|
||||||
|
Column A = MPN_make_typeid (make = first word of manufacturer, typeid from the
|
||||||
|
taxonomy). Writes one Excel file per type with a Meta sheet holding the template
|
||||||
|
version, assembles a per-part DFS folder (datasheet, footprint, symbol), and pushes
|
||||||
|
design files to the Gitea DFS repo, Excel outputs to the Parameters repo, and skill
|
||||||
|
files plus templates to the Skill_Assets repo. Also appends new parameters to a
|
||||||
|
template as a new version. Use WHENEVER the user uploads component datasheets and
|
||||||
|
asks to extract parameters, build the parameter sheets or component library, or push
|
||||||
|
datasheets to Gitea. ALWAYS trigger when the user types "\datasheet".
|
||||||
|
---
|
||||||
|
|
||||||
|
# Datasheet Extractor
|
||||||
|
|
||||||
|
Read component datasheets, fill the customer's master template, assemble each part's
|
||||||
|
design-file folder, and publish everything to three Gitea repos. Be careful and honest:
|
||||||
|
put every value in the right column and unit, and flag what a datasheet does not state.
|
||||||
|
|
||||||
|
## Inputs
|
||||||
|
|
||||||
|
- **Datasheet PDFs**, one per part. The filename is the part's MPN (search that exact
|
||||||
|
number inside a series datasheet to read the right variant).
|
||||||
|
- No project/version is needed — repos use a **flat layout**.
|
||||||
|
|
||||||
|
## The identifier: `MPN_make_typeid`
|
||||||
|
|
||||||
|
Column A of every sheet (header `MPN_make_type`) and each DFS folder name is:
|
||||||
|
|
||||||
|
```
|
||||||
|
<MPN>_<make>_<typeid> e.g. BAT46WJ_Nexperia_SCH
|
||||||
|
```
|
||||||
|
|
||||||
|
- **make** = the **first word of the manufacturer** name (Texas Instruments → `Texas`,
|
||||||
|
Nexperia → `Nexperia`, STMicroelectronics → `STMicroelectronics`).
|
||||||
|
- **typeid** = the Type ID for the part's subclass, from the taxonomy in
|
||||||
|
`references/taxonomy.md` (full source `assets/template/Type_ID.xlsx`). e.g. Schottky →
|
||||||
|
`SCH`, TVS → `TVS`, MOSFET → `MOS`, LDO → `LDO`, Common-mode choke → `CMC`.
|
||||||
|
|
||||||
|
## Workflow
|
||||||
|
|
||||||
|
1. **Read each datasheet.** Get MPN (from filename), manufacturer, and the parameters.
|
||||||
|
2. **Classify** to one of the 18 **types** (= template sheet names) and to a **subclass**
|
||||||
|
from `references/taxonomy.md`; record its **typeid**. The subclass name goes in the
|
||||||
|
`Class` column where the sheet has one.
|
||||||
|
3. **Extract values** into that type's columns (headers come from
|
||||||
|
`assets/template/template.xlsx`). Convert to each header's unit; leave blank if the
|
||||||
|
datasheet doesn't state it (an honest blank beats a guess).
|
||||||
|
4. **Build the Excel outputs** with `scripts/fill_templates.py` — one `<Type>.xlsx` per
|
||||||
|
type present, column A = `MPN_make_typeid`, plus a **Meta** sheet holding the template
|
||||||
|
version. (See Producing outputs.)
|
||||||
|
5. **Assemble each part's DFS folder** `MPN_make_typeid/` containing `MPN_data` (the
|
||||||
|
datasheet), `MPN_fp` (footprint), `MPN_sym` (symbol). See Footprint & symbol.
|
||||||
|
6. **Push to the three Gitea repos** (flat): DFS folders → **DFS**, Excel outputs →
|
||||||
|
**Parameters**, skill files + templates → **Skill_Assets**. See Pushing to Gitea.
|
||||||
|
|
||||||
|
## Producing outputs
|
||||||
|
|
||||||
|
Collect what you extracted into `parts.json` (keys match template headers; `typeid` and
|
||||||
|
`subclass` from the taxonomy; `manufacturer` used for the make tag):
|
||||||
|
|
||||||
|
```json
|
||||||
|
{"parts":[
|
||||||
|
{"type":"Diode","mpn":"BAT46WJ","manufacturer":"Nexperia","typeid":"SCH","subclass":"Schottky",
|
||||||
|
"values":{"Description":"100 V 250 mA Schottky, SOD323F","Forward Voltage(V)":"0.71","Package":"SOD323F"}}
|
||||||
|
]}
|
||||||
|
```
|
||||||
|
|
||||||
|
```bash
|
||||||
|
python scripts/fill_templates.py parts.json \
|
||||||
|
--template assets/template/template.xlsx --dest <outputs-dir>
|
||||||
|
```
|
||||||
|
|
||||||
|
Produces one `<Type>.xlsx` per type present (only types with parts), each with a **Meta**
|
||||||
|
sheet (Template Version, date, type, part count). Deliver these files to the user, and
|
||||||
|
push them to the Parameters repo.
|
||||||
|
|
||||||
|
## Footprint & symbol (for the DFS folder)
|
||||||
|
|
||||||
|
For each part, the DFS folder needs `MPN_fp` (footprint) and `MPN_sym` (symbol) beside
|
||||||
|
`MPN_data` (the datasheet). Get them like this, in order:
|
||||||
|
|
||||||
|
1. **Search the web** for the part's footprint and schematic symbol by MPN (e.g. SnapEDA,
|
||||||
|
Ultra Librarian, Component Search Engine, the manufacturer's CAD models). If found,
|
||||||
|
**cross-check** the footprint's pad/pin count and dimensions against the datasheet's
|
||||||
|
package drawing before using it.
|
||||||
|
2. **If not found, generate** the footprint/symbol yourself from the datasheet's package
|
||||||
|
and pinout **only if you can do so reliably**, then cross-check with the user before
|
||||||
|
trusting it.
|
||||||
|
3. **If neither is possible**, still create the folder with the datasheet, and **tell the
|
||||||
|
user that the footprint and/or symbol needs a manual build** for that MPN (leave a
|
||||||
|
clearly-named placeholder or omit and list it).
|
||||||
|
|
||||||
|
Keep the datasheet's extension on `MPN_data` (e.g. `BAT46WJ_data.pdf`). Name footprint/
|
||||||
|
symbol files `MPN_fp` / `MPN_sym` with whatever extension the source/format uses.
|
||||||
|
|
||||||
|
## Pushing to Gitea
|
||||||
|
|
||||||
|
Connection + repos are pre-configured in `config/gitea.env` (host, user, token, and the
|
||||||
|
three repos), so runs need no per-session token. Layout is **flat** (no project/version).
|
||||||
|
Use `scripts/push_to_gitea.sh` (git over HTTPS), once per repo:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# 1) design files -> DFS repo (staging dir holds MPN_make_typeid/ folders)
|
||||||
|
bash scripts/push_to_gitea.sh --repo "$DFS_REPO" --src <dfs-stage> --message "Add datasheets/fp/sym"
|
||||||
|
# 2) Excel outputs -> Parameters repo
|
||||||
|
bash scripts/push_to_gitea.sh --repo "$PARAMS_REPO" --src <outputs-dir> --message "Update parameters"
|
||||||
|
# 3) skill files + templates -> Skill_Assets repo
|
||||||
|
bash scripts/push_to_gitea.sh --repo "$SKILL_ASSETS_REPO" --src <skill-dir> --message "Sync skill assets"
|
||||||
|
```
|
||||||
|
|
||||||
|
`$DFS_REPO`, `$PARAMS_REPO`, `$SKILL_ASSETS_REPO` come from `config/gitea.env`. The
|
||||||
|
script clones the repo, copies the source contents in (flat), commits, and pushes; the
|
||||||
|
repos must already exist. If the host is unreachable (e.g. Cowork without the domain
|
||||||
|
allowlisted) it fails clearly and leaves the staged files for a manual push.
|
||||||
|
|
||||||
|
**Push the skill files organised** to Skill_Assets: keep the skill's own structure
|
||||||
|
(`SKILL.md`, `scripts/`, `assets/`, `references/`, `config/`) — but do **not** push the
|
||||||
|
real token. Before syncing skill assets, blank the `GIT_TOKEN` line in the copy you push,
|
||||||
|
or push everything except `config/gitea.env`.
|
||||||
|
|
||||||
|
## Updating a template (new parameter)
|
||||||
|
|
||||||
|
When the user wants a new parameter on a type, **append it at the end** of that type's
|
||||||
|
sheet and bump the version:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
python scripts/append_parameter.py --type Diode --param "Reverse Recovery Time(ns)" \
|
||||||
|
--template assets/template/template.xlsx
|
||||||
|
```
|
||||||
|
|
||||||
|
This adds the column at the end and increments `assets/template/VERSION` (v1 → v2 → …).
|
||||||
|
Then **push the updated `assets/template/template.xlsx` + `VERSION` to the Skill_Assets
|
||||||
|
repo** so the new template version is archived. New outputs will record the new version in
|
||||||
|
their Meta sheet automatically.
|
||||||
|
|
||||||
|
## Resources
|
||||||
|
|
||||||
|
- `assets/template/template.xlsx` — master template, one sheet per type (source of headers).
|
||||||
|
- `assets/template/Type_ID.xlsx` + `references/taxonomy.md` — Class → Subclass → Type ID.
|
||||||
|
- `assets/template/VERSION` — current template version (integer; Meta sheet shows `vN`).
|
||||||
|
- `scripts/fill_templates.py` — build per-type Excel outputs (+ Meta sheet).
|
||||||
|
- `scripts/push_to_gitea.sh` — push a folder's contents to a Gitea repo (flat).
|
||||||
|
- `scripts/append_parameter.py` — append a parameter to a template + bump version.
|
||||||
|
- `config/gitea.env` — host, user, token, and the DFS / Parameters / Skill_Assets repos (**secret**).
|
||||||
Binary file not shown.
|
|
@ -0,0 +1 @@
|
||||||
|
1
|
||||||
Binary file not shown.
Binary file not shown.
|
After Width: | Height: | Size: 12 KiB |
|
|
@ -0,0 +1,10 @@
|
||||||
|
# Gitea connection for the datasheet-extractor skill.
|
||||||
|
# WARNING: contains a secret (GIT_TOKEN) in plaintext - keep this skill PRIVATE.
|
||||||
|
# Use a token scoped to these repos (repository: write) and rotate periodically.
|
||||||
|
GIT_HOST=gitea.vecmocon.com
|
||||||
|
GIT_USER=nitishKumar
|
||||||
|
GIT_TOKEN=
|
||||||
|
# Target repos (flat layout, no project/version grouping):
|
||||||
|
DFS_REPO=nitishKumar/DFS
|
||||||
|
PARAMS_REPO=nitishKumar/Parameters
|
||||||
|
SKILL_ASSETS_REPO=nitishKumar/Skill_Assets
|
||||||
|
|
@ -0,0 +1,167 @@
|
||||||
|
# Component taxonomy — Class → Subclass (Type) → Type ID
|
||||||
|
|
||||||
|
Use the **Type ID** as the `typeid` in `MPN_make_typeid`. Match each part to the closest
|
||||||
|
subclass. The output sheet name is the **Class** (template.xlsx sheet). Full source:
|
||||||
|
`assets/template/Type_ID.xlsx`.
|
||||||
|
|
||||||
|
|
||||||
|
## Resistor (12 types)
|
||||||
|
- **FIX** — Thick-film chip resistor — Std chip R, RC/CRCW/RK73
|
||||||
|
- **TFR** — Thin-film chip resistor — Precision, low tempco
|
||||||
|
- **MFR** — Metal-film resistor — Axial MELF/leaded
|
||||||
|
- **CFR** — Carbon-film resistor — Leaded general purpose
|
||||||
|
- **MOR** — Metal-oxide resistor — High-power leaded
|
||||||
|
- **WWR** — Wirewound resistor — High-power / low-ohm
|
||||||
|
- **SHT** — Current-sense / shunt — mΩ sense resistor
|
||||||
|
- **ARR** — Resistor array / network — Isolated/bussed array
|
||||||
|
- **POT** — Potentiometer / trimmer — Variable resistor
|
||||||
|
- **FSR** — Fusible / safety resistor — Fails open safely
|
||||||
|
- **NTC** — NTC thermistor — Inrush / temp sense
|
||||||
|
- **PTC** — PTC thermistor — Temp sense / self-heat
|
||||||
|
|
||||||
|
## Capacitor (7 types)
|
||||||
|
- **CER** — Ceramic MLCC — X7R/X5R/C0G chip
|
||||||
|
- **ELE** — Aluminium electrolytic — Bulk / DC-link
|
||||||
|
- **TAN** — Tantalum — Compact bulk
|
||||||
|
- **PLY** — Aluminium-polymer — Low-ESR polymer
|
||||||
|
- **FLM** — Film (MKT/MKP) — DC-link / snubber
|
||||||
|
- **SFY** — Safety (Class-X / Y) — Mains X2/Y suppression
|
||||||
|
- **SUP** — Supercapacitor / EDLC — Backup energy
|
||||||
|
|
||||||
|
## Inductor / Magnetics (7 types)
|
||||||
|
- **PWR** — Power inductor — SMD/shielded
|
||||||
|
- **FBD** — Ferrite bead — EMI suppression
|
||||||
|
- **CMC** — Common-mode choke — CM noise filter
|
||||||
|
- **RFI** — RFI suppression choke — Line filter
|
||||||
|
- **XFM** — Transformer — Isolation / flyback
|
||||||
|
- **CTX** — Current transformer — Current sensing
|
||||||
|
- **CPL** — Coupled inductor — SEPIC / multi-winding
|
||||||
|
|
||||||
|
## Diode (10 types)
|
||||||
|
- **REC** — Rectifier (general) — 50/60Hz rectifier
|
||||||
|
- **FRD** — Fast-recovery — SMPS rectifier
|
||||||
|
- **SCH** — Schottky — Low-Vf rectifier
|
||||||
|
- **SIC** — SiC Schottky — HV fast, low-loss
|
||||||
|
- **ZEN** — Zener — Voltage clamp/ref
|
||||||
|
- **TVS** — TVS — Transient suppression
|
||||||
|
- **ESD** — ESD protection — Port ESD clamp
|
||||||
|
- **SWI** — Switching / small-signal — 1N4148/BAV
|
||||||
|
- **BRG** — Bridge rectifier — Full-bridge module
|
||||||
|
- **LED** — LED (indicator) — Light-emitting
|
||||||
|
|
||||||
|
## Transistor (7 types)
|
||||||
|
- **BJT** — BJT — NPN/PNP small-signal
|
||||||
|
- **MOS** — MOSFET (Si) — Switching FET
|
||||||
|
- **SCM** — SiC MOSFET — HV traction switch
|
||||||
|
- **GAN** — GaN FET — HF power switch
|
||||||
|
- **IGBT** — IGBT — HV power switch
|
||||||
|
- **JFET** — JFET — Analog/RF
|
||||||
|
- **DIG** — Digital / bias-resistor — Built-in bias R
|
||||||
|
|
||||||
|
## Integrated Circuit (IC) (21 types)
|
||||||
|
- **MCU** — Microcontroller — MCU/SoC
|
||||||
|
- **LDO** — LDO regulator — Linear regulator
|
||||||
|
- **DCD** — DC-DC converter IC — Buck/boost controller
|
||||||
|
- **PMU** — PMIC / power management — Multi-rail PMIC
|
||||||
|
- **BMS** — Battery management (BMS AFE) — Cell monitor/balance
|
||||||
|
- **DRV** — Gate / motor driver — MOSFET/IGBT/motor driver
|
||||||
|
- **AMP** — Amplifier / op-amp — Signal amplifier
|
||||||
|
- **CMP** — Comparator — Threshold compare
|
||||||
|
- **VRF** — Voltage reference — Precision reference
|
||||||
|
- **ADC** — ADC — Analog-to-digital
|
||||||
|
- **DAC** — DAC — Digital-to-analog
|
||||||
|
- **ISO** — Isolator / optocoupler — Galvanic isolation
|
||||||
|
- **XCV** — Transceiver (CAN/RS-485) — Bus transceiver
|
||||||
|
- **AFE** — Analog front end — Sensor/metering AFE
|
||||||
|
- **MEM** — Memory (Flash/EEPROM) — Non-volatile memory
|
||||||
|
- **LOG** — Logic gate — Gates/translators/buffers
|
||||||
|
- **SEN** — Sensor IC — On-chip sensor
|
||||||
|
- **IFC** — Interface / expander — I/O/level/expander
|
||||||
|
- **CLK** — Clock / RTC — Timing / real-time clock
|
||||||
|
- **SVR** — Supervisor / reset — Voltage supervisor
|
||||||
|
- **MTR** — Energy metering — Power metering IC
|
||||||
|
|
||||||
|
## Protection Device (5 types)
|
||||||
|
- **FUS** — Fuse — One-time fuse
|
||||||
|
- **RSF** — Resettable fuse / PPTC — Polyfuse
|
||||||
|
- **VAR** — Varistor / MOV — Surge clamp
|
||||||
|
- **GDT** — Gas-discharge tube — Surge diverter
|
||||||
|
- **CBK** — Circuit breaker / RCBO — Resettable breaker
|
||||||
|
|
||||||
|
## Power Conversion Module (7 types)
|
||||||
|
- **DCM** — DC-DC converter module — Point-of-load brick
|
||||||
|
- **IDC** — Isolated DC-DC module — Isolated brick
|
||||||
|
- **INV** — Inverter (DC-AC) — Motor/grid inverter
|
||||||
|
- **OBC** — On-board charger (OBC) — AC-DC EV charger
|
||||||
|
- **CHG** — Battery charger module — Charger assembly
|
||||||
|
- **PSU** — AC-DC power supply (SMPS) — Mains SMPS
|
||||||
|
- **RCM** — Rectifier module — Power rectifier block
|
||||||
|
|
||||||
|
## Relay / Contactor (5 types)
|
||||||
|
- **RLS** — Signal relay — Low-current relay
|
||||||
|
- **RLP** — Power relay — High-current relay
|
||||||
|
- **SSR** — Solid-state relay — Semiconductor relay
|
||||||
|
- **RLR** — Reed relay — Reed-switch relay
|
||||||
|
- **CTC** — Contactor — HV/HC contactor
|
||||||
|
|
||||||
|
## Switch / Button (5 types)
|
||||||
|
- **SWT** — Tactile switch — PCB tact
|
||||||
|
- **PBT** — Push button — Panel/PCB button
|
||||||
|
- **DSW** — DIP / slide switch — Config switch
|
||||||
|
- **RSW** — Rocker / toggle — Power switch
|
||||||
|
- **RSY** — Rotary switch — Selector
|
||||||
|
|
||||||
|
## Connector (7 types)
|
||||||
|
- **CWB** — Wire-to-board — JST/Molex
|
||||||
|
- **CBB** — Board-to-board — Stacking
|
||||||
|
- **HDR** — Pin header / socket — 2.54/1.27 header
|
||||||
|
- **FFC** — FFC / FPC — Flat-flex
|
||||||
|
- **USB** — USB / data — USB/RJ45
|
||||||
|
- **PWC** — Power / high-current — Power connector
|
||||||
|
- **TBK** — Terminal block — Screw/spring terminal
|
||||||
|
|
||||||
|
## Antenna / RF (5 types)
|
||||||
|
- **ANC** — Chip antenna — SMD antenna
|
||||||
|
- **ANP** — PCB / trace antenna — Etched antenna
|
||||||
|
- **ANE** — External / whip antenna — Cable/whip
|
||||||
|
- **SAW** — SAW filter — RF band filter
|
||||||
|
- **RFM** — RF / wireless module — BLE/WiFi/LTE module
|
||||||
|
|
||||||
|
## Crystal / Oscillator / Timing (4 types)
|
||||||
|
- **XTL** — Crystal — Quartz crystal
|
||||||
|
- **OSC** — Crystal oscillator (XO) — Packaged oscillator
|
||||||
|
- **MMO** — MEMS oscillator — MEMS timing
|
||||||
|
- **RSN** — Ceramic resonator — Low-cost resonator
|
||||||
|
|
||||||
|
## Battery / Cell (6 types)
|
||||||
|
- **CLI** — Li-ion cell — Cylindrical/pouch
|
||||||
|
- **CLF** — LiFePO4 cell — LFP cell
|
||||||
|
- **CCO** — Coin / button cell — Backup coin cell
|
||||||
|
- **CNI** — NiMH cell — NiMH
|
||||||
|
- **BPK** — Battery pack — Assembled pack
|
||||||
|
- **CHL** — Cell holder — Coin/AA holder
|
||||||
|
|
||||||
|
## Audible / Indicator (4 types)
|
||||||
|
- **BUZ** — Magnetic buzzer — Driven buzzer
|
||||||
|
- **PBZ** — Piezo buzzer — Self/ext-drive piezo
|
||||||
|
- **SPK** — Speaker — Audio speaker
|
||||||
|
- **IND** — Indicator lamp / pilot light — Panel indicator
|
||||||
|
|
||||||
|
## Display / HMI (4 types)
|
||||||
|
- **DSG** — 7-segment LED display — Numeric display
|
||||||
|
- **OLE** — OLED module — OLED graphic
|
||||||
|
- **LCD** — LCD module — Char/graphic LCD
|
||||||
|
- **TFT** — TFT display — Color TFT
|
||||||
|
|
||||||
|
## Sensor (discrete / module) (6 types)
|
||||||
|
- **STE** — Temperature sensor — NTC/IC temp
|
||||||
|
- **SCU** — Current sensor (Hall) — Isolated current
|
||||||
|
- **SVO** — Voltage / isolated sensor — HV sensing
|
||||||
|
- **SHA** — Hall / position sensor — Position/speed
|
||||||
|
- **SIM** — IMU / accelerometer — Motion sensing
|
||||||
|
- **SPR** — Pressure sensor — Pressure/altitude
|
||||||
|
|
||||||
|
## Thermal / Cooling (3 types)
|
||||||
|
- **FAN** — Fan — Cooling fan
|
||||||
|
- **HSK** — Heatsink — Thermal dissipation
|
||||||
|
- **TPD** — Thermal pad / interface — TIM
|
||||||
|
|
@ -0,0 +1,27 @@
|
||||||
|
#!/usr/bin/env python3
|
||||||
|
"""Append a new parameter (column) to the end of a type's sheet in the master
|
||||||
|
template, and bump the template VERSION. After running, push the updated
|
||||||
|
assets/template/template.xlsx + VERSION to the Skill_Assets repo.
|
||||||
|
|
||||||
|
Usage: python append_parameter.py --type Diode --param "Reverse Recovery Time(ns)" \
|
||||||
|
--template assets/template/template.xlsx
|
||||||
|
"""
|
||||||
|
import argparse, os, openpyxl
|
||||||
|
ap=argparse.ArgumentParser()
|
||||||
|
ap.add_argument("--type", required=True); ap.add_argument("--param", required=True)
|
||||||
|
ap.add_argument("--template", required=True)
|
||||||
|
a=ap.parse_args()
|
||||||
|
wb=openpyxl.load_workbook(a.template)
|
||||||
|
if a.type not in wb.sheetnames:
|
||||||
|
raise SystemExit(f"no sheet '{a.type}' in template; sheets: {wb.sheetnames}")
|
||||||
|
ws=wb[a.type]
|
||||||
|
headers=[ws.cell(1,c).value for c in range(1, ws.max_column+1)]
|
||||||
|
if a.param in headers:
|
||||||
|
raise SystemExit(f"'{a.param}' already exists in {a.type}")
|
||||||
|
ws.cell(1, ws.max_column+1, a.param) # append at the end
|
||||||
|
wb.save(a.template)
|
||||||
|
vf=os.path.join(os.path.dirname(a.template), "VERSION")
|
||||||
|
v=int(open(vf).read().strip()) if os.path.exists(vf) else 1
|
||||||
|
v+=1; open(vf,"w").write(str(v))
|
||||||
|
print(f"Appended '{a.param}' to {a.type}. Template is now v{v}. "
|
||||||
|
f"Push {os.path.basename(a.template)} + VERSION to the Skill_Assets repo.")
|
||||||
|
|
@ -0,0 +1,100 @@
|
||||||
|
#!/usr/bin/env python3
|
||||||
|
"""Fill the master multi-type template (assets/template/template.xlsx) from parts.json.
|
||||||
|
|
||||||
|
Per component type present, writes one <Type>.xlsx containing the filled type sheet
|
||||||
|
plus a Meta sheet (template version, date). Column A ("MPN_make_type") is set to
|
||||||
|
<MPN>_<make>_<typeid>, where make = first word of the manufacturer and typeid comes
|
||||||
|
from the taxonomy (Type_ID.xlsx), supplied per part as "typeid".
|
||||||
|
|
||||||
|
parts.json:
|
||||||
|
{
|
||||||
|
"parts": [
|
||||||
|
{"type": "Diode", "mpn": "BAT46WJ", "manufacturer": "Nexperia",
|
||||||
|
"typeid": "SCH", "subclass": "Schottky",
|
||||||
|
"values": {"Description": "...", "Forward Voltage(V)": "0.71", ...}}
|
||||||
|
]
|
||||||
|
}
|
||||||
|
Usage: python fill_templates.py parts.json --template <template.xlsx> --dest <dir> [--version v1]
|
||||||
|
"""
|
||||||
|
import argparse, json, os, re, datetime
|
||||||
|
import openpyxl
|
||||||
|
from openpyxl.styles import Font, PatternFill, Alignment, Border, Side
|
||||||
|
|
||||||
|
GREEN="B6D7A8"; GRAY="BFBFBF"
|
||||||
|
def norm(s): return re.sub(r'[^a-z0-9]', '', str(s).lower())
|
||||||
|
ALIAS={"induactanceuh":"inductanceuh"} # template header typo -> value key
|
||||||
|
|
||||||
|
def make_tag(manufacturer):
|
||||||
|
w = re.split(r'\s+', str(manufacturer).strip())
|
||||||
|
first = w[0] if w and w[0] else "NA"
|
||||||
|
return re.sub(r'[^A-Za-z0-9]', '', first) or "NA"
|
||||||
|
|
||||||
|
def style_header(cell):
|
||||||
|
cell.font=Font(name="Calibri", bold=True); cell.fill=PatternFill("solid", fgColor=GREEN)
|
||||||
|
s=Side(style="thin", color=GRAY); cell.border=Border(left=s,right=s,top=s,bottom=s)
|
||||||
|
cell.alignment=Alignment(horizontal="center", vertical="center", wrap_text=True)
|
||||||
|
|
||||||
|
def main():
|
||||||
|
ap=argparse.ArgumentParser()
|
||||||
|
ap.add_argument("parts_json"); ap.add_argument("--template", required=True)
|
||||||
|
ap.add_argument("--dest", required=True); ap.add_argument("--version", default=None)
|
||||||
|
a=ap.parse_args()
|
||||||
|
data=json.load(open(a.parts_json, encoding="utf-8"))
|
||||||
|
version=a.version
|
||||||
|
if not version:
|
||||||
|
vf=os.path.join(os.path.dirname(a.template), "VERSION")
|
||||||
|
version="v"+open(vf).read().strip() if os.path.exists(vf) else "v1"
|
||||||
|
os.makedirs(a.dest, exist_ok=True)
|
||||||
|
master=openpyxl.load_workbook(a.template)
|
||||||
|
by={}
|
||||||
|
for p in data.get("parts", []):
|
||||||
|
by.setdefault(p["type"], []).append(p)
|
||||||
|
|
||||||
|
for ctype, parts in by.items():
|
||||||
|
if ctype not in master.sheetnames:
|
||||||
|
print(f"! no template sheet for type '{ctype}' - skipping"); continue
|
||||||
|
msheet=master[ctype]
|
||||||
|
headers=[msheet.cell(1,c).value for c in range(1, msheet.max_column+1)]
|
||||||
|
out=openpyxl.Workbook(); ws=out.active; ws.title=ctype[:31]
|
||||||
|
for c,h in enumerate(headers, start=1):
|
||||||
|
ws.cell(1,c,h); style_header(ws.cell(1,c))
|
||||||
|
ws.row_dimensions[1].height=30
|
||||||
|
r=2
|
||||||
|
for p in parts:
|
||||||
|
vals={norm(k):v for k,v in p.get("values", {}).items()}
|
||||||
|
make=p.get("make") or make_tag(p.get("manufacturer",""))
|
||||||
|
tid=p.get("typeid","NA")
|
||||||
|
colA=f'{p.get("mpn","")}_{make}_{tid}'
|
||||||
|
for c,h in enumerate(headers, start=1):
|
||||||
|
key=norm(h); key=ALIAS.get(key,key)
|
||||||
|
if c==1:
|
||||||
|
ws.cell(r,c,colA)
|
||||||
|
elif key=="class":
|
||||||
|
ws.cell(r,c,p.get("subclass",""))
|
||||||
|
elif key=="manufacturer":
|
||||||
|
ws.cell(r,c,p.get("manufacturer", vals.get("manufacturer","")))
|
||||||
|
else:
|
||||||
|
ws.cell(r,c,vals.get(key,""))
|
||||||
|
r+=1
|
||||||
|
# column widths
|
||||||
|
from openpyxl.utils import get_column_letter
|
||||||
|
for c,h in enumerate(headers, start=1):
|
||||||
|
w=32 if norm(h)=="description" else (24 if c==1 else max(11,min(18,len(str(h))+2)))
|
||||||
|
ws.column_dimensions[get_column_letter(c)].width=w
|
||||||
|
ws.freeze_panes="A2"
|
||||||
|
# Meta sheet
|
||||||
|
meta=out.create_sheet("Meta")
|
||||||
|
rows=[("Template Version", version),
|
||||||
|
("Generated", datetime.date.today().isoformat()),
|
||||||
|
("Component Type", ctype),
|
||||||
|
("Parts in file", len(parts)),
|
||||||
|
("Source", "datasheet-extractor skill")]
|
||||||
|
for i,(k,v) in enumerate(rows, start=1):
|
||||||
|
meta.cell(i,1,k).font=Font(bold=True); meta.cell(i,2,v)
|
||||||
|
meta.column_dimensions["A"].width=20; meta.column_dimensions["B"].width=40
|
||||||
|
fn=re.sub(r'\s+','_',ctype)+".xlsx"
|
||||||
|
out.save(os.path.join(a.dest, fn))
|
||||||
|
print(f"{ctype}: {len(parts)} row(s) -> {fn} (template {version})")
|
||||||
|
|
||||||
|
if __name__=="__main__":
|
||||||
|
main()
|
||||||
|
|
@ -0,0 +1,38 @@
|
||||||
|
#!/usr/bin/env bash
|
||||||
|
# Push the CONTENTS of a source dir to a Gitea repo (flat, at repo root or a subdir).
|
||||||
|
# Connection (GIT_HOST/GIT_USER/GIT_TOKEN) comes from env or config/gitea.env.
|
||||||
|
# Usage: bash push_to_gitea.sh --repo <owner/name> --src <dir> [--subdir <path>] [--message "msg"]
|
||||||
|
set -euo pipefail
|
||||||
|
HERE="$(cd "$(dirname "$0")" && pwd)"
|
||||||
|
CFG="$HERE/../config/gitea.env"
|
||||||
|
if [[ -f "$CFG" ]]; then
|
||||||
|
while IFS='=' read -r k v; do
|
||||||
|
[[ "$k" =~ ^[A-Z_]+$ ]] || continue
|
||||||
|
case "$k" in GIT_HOST|GIT_USER|GIT_TOKEN)
|
||||||
|
[[ -z "${!k:-}" ]] && export "$k=$v" ;; esac
|
||||||
|
done < "$CFG"
|
||||||
|
fi
|
||||||
|
HOST="${GIT_HOST:-}" USER_="${GIT_USER:-}" TOKEN="${GIT_TOKEN:-}"
|
||||||
|
REPO="" SRC="" SUBDIR="" MSG=""
|
||||||
|
while [[ $# -gt 0 ]]; do case "$1" in
|
||||||
|
--repo) REPO="$2"; shift 2;; --src) SRC="$2"; shift 2;;
|
||||||
|
--subdir) SUBDIR="$2"; shift 2;; --message) MSG="$2"; shift 2;;
|
||||||
|
*) echo "unknown arg: $1" >&2; exit 2;; esac; done
|
||||||
|
[[ -z "$REPO" || -z "$SRC" || -z "$HOST" || -z "$TOKEN" ]] && { echo "need --repo, --src, and GIT_HOST/GIT_TOKEN" >&2; exit 2; }
|
||||||
|
[[ -d "$SRC" ]] || { echo "ERROR: src not found: $SRC" >&2; exit 2; }
|
||||||
|
[[ -z "$MSG" ]] && MSG="Update from datasheet-extractor"
|
||||||
|
HOST="${HOST#http://}"; HOST="${HOST#https://}"; HOST="${HOST%/}"
|
||||||
|
CRED="${USER_:+$USER_:}${TOKEN}"
|
||||||
|
URL="https://${CRED}@${HOST}/${REPO}.git"
|
||||||
|
scrub(){ sed -e "s/${TOKEN}/***/g"; }
|
||||||
|
WORK="$(mktemp -d)"
|
||||||
|
git clone "$URL" "$WORK" 2>&1 | scrub || { echo "ERROR: clone failed (repo must exist + host reachable + token scope)"; exit 3; }
|
||||||
|
cd "$WORK"; git checkout -B main >/dev/null 2>&1 || true
|
||||||
|
DEST="$WORK${SUBDIR:+/$SUBDIR}"; mkdir -p "$DEST"
|
||||||
|
cp -r "$SRC/." "$DEST/"
|
||||||
|
git config user.email "datasheet-bot@local"; git config user.name "datasheet-extractor"
|
||||||
|
git add -A
|
||||||
|
if git diff --cached --quiet; then echo "Nothing new for $REPO."; else
|
||||||
|
git commit -m "$MSG" >/dev/null; git push -u origin main 2>&1 | scrub
|
||||||
|
fi
|
||||||
|
echo "Pushed to https://${HOST}/${REPO}${SUBDIR:+/$SUBDIR} ($(find "$SRC" -type f | wc -l | tr -d ' ') file(s))"
|
||||||
Loading…
Reference in New Issue