diff --git a/SKILL.md b/SKILL.md index b69b961..b643e09 100644 --- a/SKILL.md +++ b/SKILL.md @@ -20,8 +20,18 @@ description: >- Turn one component datasheet into a verified, versioned library entry in Gitea. The guiding idea is honesty and traceability: every value lands in the right column and unit, anything the datasheet doesn't state stays blank, and nothing reaches Gitea until a human has -confirmed it. This is an **interactive** skill — it asks at the few points where a person's -judgement or an upload is genuinely needed, and does everything else on its own. +confirmed it. + +## Ask, don't assume + +This is a deliberately **interactive** skill. At every decision point, **ask the user and +wait for an answer — do not assume a default and proceed.** In particular, always confirm: +the **make**; the **typeid/classification** you inferred; whether any **new parameters** +should be added to the template; the extracted **values** (the verification loop); that the +**symbol/footprint actually match this part**; whether to **apply a new parameter to existing +parts** (backfill); and **before every push to Gitea** (the part, the skill files, the +changelog). When something is ambiguous, ask a specific question rather than guessing. It is +always better to ask one more question than to write the wrong thing into the library. ## Inputs @@ -108,9 +118,10 @@ Check whether that typeid has a sheet in `assets/template/template.xlsx`. This appends the column(s) at the end of that typeid's sheet, **bumps that typeid's Template Version and Skill Version together** (v1→v2 — see *Per-typeid versioning*), - and writes one entry to the global changelog `assets/template/CHANGELOG.md`. Push the - updated `template.xlsx` + `versions.json` + `CHANGELOG.md` to the **skill** repo, and - make sure the components repo's copies stay consistent. + and writes one row to the global changelog `assets/CHANGELOG.xlsx`. Then **ask + the user before pushing**, and sync the updated skill files + changelog to the skill repo + with `push-skill` (see *Pushing the skill repo*) — that merges the new changelog row onto + the one already in Gitea rather than overwriting it. Then ask: **should this change apply to the parts of this typeid already in Gitea?** - **No** → go to step 4 (only the current part gets the new column). @@ -206,10 +217,16 @@ reading and writing these numbers. ## The changelog -`append_parameter.py` maintains one **global** changelog at `assets/template/CHANGELOG.md` -(newest entry on top), recording for each change: date, typeid + class, the template and -skill version transitions, the parameter(s) added, and the description. Push it to the skill -repo alongside `template.xlsx` and `versions.json` whenever it changes. +`append_parameter.py` maintains one **global** changelog as an Excel workbook at +`assets/CHANGELOG.xlsx` (sheet `Changelog`, styled green header). Every time a +typeid's template/version changes, one row is appended with columns +**Date | Typeid | Skill Version | Template Version | Description** — the version columns hold +the new versions, and Description is your note (or the parameter(s) added if you gave none). + +The changelog lives in the **skill repo in Gitea** as well. When you push (via `push-skill`), +the new local rows are **merged onto the changelog already in Gitea** — appended, never +overwritten — so the Gitea copy is the growing, authoritative history across machines and +sessions, and the merged file is copied back locally so the two stay in sync. ## Backfilling existing parts @@ -240,13 +257,19 @@ python scripts/gitea_components.py commit-push --root work/ --message "backfill ## Pushing the skill repo When skill files change (a new typeid template, a parameter add, a version/changelog bump), -push the skill's own files — but **never push the real token**. Blank the `GIT_TOKEN` line in -the copy you push, or exclude `config/gitea.env`: +**ask the user first**, then push the skill's own files to the skill repo with `push-skill`: ```bash -bash scripts/push_to_gitea.sh --repo "$SKILL_REPO" --src --message "Sync skill files" +python scripts/gitea_components.py push-skill --message "Sync skill files + changelog" ``` +`push-skill` clones the skill repo, copies the skill files in with the **`GIT_TOKEN` blanked +out** (the real token never leaves the machine), and **merges** `CHANGELOG.xlsx` — appending +this run's new rows onto the changelog already in Gitea so earlier entries are preserved — then +writes the merged changelog back locally. (The older `push_to_gitea.sh` still exists for a +plain flat push, but it does not merge the changelog or blank the token, so prefer +`push-skill` for the skill repo.) + ## Resources - `assets/template/template.xlsx` — the master template: one sheet per **typeid** (125), @@ -255,7 +278,8 @@ bash scripts/push_to_gitea.sh --repo "$SKILL_REPO" --src --message " `Footprint Ref/Path` and `Manufacturer` sit near the end. - `assets/template/Type_ID.xlsx` + `references/taxonomy.md` — Class → Subclass → Type ID. - `assets/template/versions.json` — per-typeid `template_version` + `skill_version`. -- `assets/template/CHANGELOG.md` — global version/parameter changelog (created on first add). +- `assets/CHANGELOG.xlsx` — global version/parameter changelog (created on first add; + merged into the skill repo's copy in Gitea by `push-skill`). - `scripts/common.py` — taxonomy loader (`load_taxonomy`, `class_folder`), version store (`get_versions`, `version_labels`, `bump_versions`), and the tag helper (`part_tag`). - `scripts/fill_templates.py` — build one per-part `.xlsx` (version-stamped); reused for @@ -264,7 +288,8 @@ bash scripts/push_to_gitea.sh --repo "$SKILL_REPO" --src --message " the changelog. - `scripts/altium_refs.py` — read Library/Footprint Ref from `.SchLib`/`.PcbLib`. - `scripts/gitea_components.py` — `check-mpn`, `checkout`, `list-type`, `place-part`, - `commit-push`, `push-part` against the components repo. + `commit-push`, `push-part` (components repo), and `push-skill` (skill repo: token-blanked + push + append-only changelog merge). - `scripts/push_to_gitea.sh` — push a folder's contents to a Gitea repo (used for the skill repo). - `config/gitea.env` — host, user, token, and the `SKILL_REPO` / `COMPONENTS_REPO` names diff --git a/assets/template/template.xlsx b/assets/template/template.xlsx index 07164c3..40bc1a8 100644 Binary files a/assets/template/template.xlsx and b/assets/template/template.xlsx differ diff --git a/assets/template/versions.json b/assets/template/versions.json index 98e2211..ca1840f 100644 --- a/assets/template/versions.json +++ b/assets/template/versions.json @@ -148,8 +148,8 @@ "template_version": 1 }, "ESD": { - "skill_version": 2, - "template_version": 2 + "skill_version": 1, + "template_version": 1 }, "FAN": { "skill_version": 1, diff --git a/config/gitea.env b/config/gitea.env index eb9e970..5615cfc 100644 --- a/config/gitea.env +++ b/config/gitea.env @@ -3,11 +3,11 @@ # Use a token scoped to these repos (repository: write) and rotate periodically. GIT_HOST=gitea.vecmocon.com GIT_USER=nitishKumar -GIT_TOKEN=451bff1dc32202cbc0a371f8e5645079466d2120 +GIT_TOKEN= # Target repos — TWO now (set these to the exact repo names on your Gitea): # SKILL_REPO : holds this skill's own files (SKILL.md, scripts, assets, ...). # COMPONENTS_REPO : holds components, one folder per Class (Diode, IC, ...); inside each, # one MPN_make_typeid/ folder per part with { xlsx, datasheet, symbol, # footprint }. (Replaces the old DFS + Parameters repos.) -SKILL_REPO=nitishKumar/Skill_Assets +SKILL_REPO=nitishKumar/skill COMPONENTS_REPO=nitishKumar/Components diff --git a/scripts/append_parameter.py b/scripts/append_parameter.py index c68383d..b64d773 100644 --- a/scripts/append_parameter.py +++ b/scripts/append_parameter.py @@ -8,19 +8,23 @@ bumps v1->v2 too. Only SCH moves; every other typeid keeps its versions. Those t are what fill_templates later stamps into the SCH sheet's rows (columns B and C). The new column is appended at the END of the sheet (as requested), styled like the other -headers. Then versions.json is bumped and one entry is written to CHANGELOG.md. +headers. Then versions.json is bumped and one entry is written to CHANGELOG.xlsx. python append_parameter.py --typeid SCH \ --param "Reverse Recovery Time(ns)" --param "Diode Capacitance(pF)" \ --desc "Added Trr and Cd for SMPS rectifier derating" -After running, push the updated template.xlsx + versions.json + CHANGELOG.md to the SKILL +After running, push the updated template.xlsx + versions.json + CHANGELOG.xlsx to the SKILL repo (and, per the workflow, the same change flows to the components repo copies). """ import argparse, datetime, os import openpyxl from copy import copy -from common import (TEMPLATE_XLSX, CHANGELOG_MD, bump_versions, load_taxonomy) +from openpyxl.styles import Font, PatternFill, Border, Side, Alignment +from common import (TEMPLATE_XLSX, CHANGELOG_XLSX, bump_versions) + +CHANGELOG_HEADERS = ["Date", "Typeid", "Skill Version", "Template Version", "Description"] +GREEN = "B6D7A8"; GRAY = "BFBFBF" def _style_header(new_cell, ref_cell): @@ -54,26 +58,50 @@ def append_params(template_path, typeid, params): return added +def _thin(): + s = Side(style="thin", color=GRAY) + return Border(left=s, right=s, top=s, bottom=s) + + +def _new_changelog_book(): + """Create the changelog workbook with a styled header row.""" + wb = openpyxl.Workbook() + ws = wb.active + ws.title = "Changelog" + for c, h in enumerate(CHANGELOG_HEADERS, start=1): + cell = ws.cell(1, c, h) + cell.font = Font(name="Calibri", bold=True) + cell.fill = PatternFill("solid", fgColor=GREEN) + cell.border = _thin() + cell.alignment = Alignment(horizontal="center", vertical="center") + for col, w in zip("ABCDE", (18, 10, 14, 16, 60)): + ws.column_dimensions[col].width = w + ws.freeze_panes = "A2" + return wb + + def write_changelog(typeid, added, old, new, desc, when=None): + """Append one row to the Excel changelog for a typeid's template/version change. + Columns: Date | Typeid | Skill Version | Template Version | Description. The version + columns show the NEW versions; Description is the user's note (or the added params).""" when = when or datetime.datetime.now() - folder = (load_taxonomy().get(typeid) or {}).get("folder", "?") - lines = [ - f"## {when.strftime('%Y-%m-%d %H:%M')} - {typeid} ({folder})", - f"- Template v{old['template_version']} -> v{new['template_version']}, " - f"Skill v{old['skill_version']} -> v{new['skill_version']}", - "- Added parameter(s): " + ", ".join(f'\"{p}\"' for p in added), - ] - if desc: - lines.append(f"- Description: {desc}") - entry = "\n".join(lines) + "\n\n" - header = "# library-manager - template/version changelog\n\n" - prior = "" - if os.path.exists(CHANGELOG_MD): - prior = open(CHANGELOG_MD, encoding="utf-8").read() - if prior.startswith(header): - prior = prior[len(header):] - open(CHANGELOG_MD, "w", encoding="utf-8").write(header + entry + prior) # newest on top - return CHANGELOG_MD + detail = desc or ("Added parameter(s): " + ", ".join(added)) + if os.path.exists(CHANGELOG_XLSX): + wb = openpyxl.load_workbook(CHANGELOG_XLSX) + ws = wb["Changelog"] if "Changelog" in wb.sheetnames else wb.active + else: + wb = _new_changelog_book() + ws = wb.active + row = [when.strftime("%Y-%m-%d %H:%M"), typeid, + f"v{new['skill_version']}", f"v{new['template_version']}", detail] + r = ws.max_row + 1 + for c, v in enumerate(row, start=1): + cell = ws.cell(r, c, v) + cell.border = _thin() + cell.alignment = Alignment(vertical="center", + wrap_text=(c == 5), horizontal="left") + wb.save(CHANGELOG_XLSX) + return CHANGELOG_XLSX def main(): @@ -92,7 +120,7 @@ def main(): print(f"{a.typeid}: template v{old['template_version']}->v{new['template_version']}, " f"skill v{old['skill_version']}->v{new['skill_version']}") print(f"Changelog updated: {cl}") - print("Now push template.xlsx + versions.json + CHANGELOG.md to the SKILL repo.") + print("Now push template.xlsx + versions.json + CHANGELOG.xlsx to the SKILL repo.") if __name__ == "__main__": diff --git a/scripts/common.py b/scripts/common.py index dffc88b..8c231ea 100644 --- a/scripts/common.py +++ b/scripts/common.py @@ -21,7 +21,7 @@ TEMPLATE_DIR = os.path.join(SKILL_ROOT, "assets", "template") TEMPLATE_XLSX= os.path.join(TEMPLATE_DIR, "template.xlsx") TYPE_ID_XLSX = os.path.join(TEMPLATE_DIR, "Type_ID.xlsx") VERSIONS_JSON= os.path.join(TEMPLATE_DIR, "versions.json") -CHANGELOG_MD = os.path.join(TEMPLATE_DIR, "CHANGELOG.md") +CHANGELOG_XLSX = os.path.join(SKILL_ROOT, "assets", "CHANGELOG.xlsx") # Fixed template columns that live on every sheet and are NOT extracted from a datasheet. COL_TAG = "MPN_make_type" # column A header (the part tag lives here) diff --git a/scripts/gitea_components.py b/scripts/gitea_components.py index 868163b..6e32464 100644 --- a/scripts/gitea_components.py +++ b/scripts/gitea_components.py @@ -32,6 +32,12 @@ testing or offline work): push-part --folder staging/BAT46WJ_Nexperia_SCH --typeid SCH [--message "..."] Convenience: clone -> place-part -> commit-push in one go (the common single-part path). + push-skill [--src ] [--message "..."] + Push this skill's files to the SKILL repo (token blanked out) and MERGE the changelog: + the new rows in the local CHANGELOG.xlsx are appended onto the repo's existing one, so + previous entries in Gitea are preserved, not overwritten. The merged changelog is + written back locally so local and Gitea stay in sync. + Host unreachable (e.g. sandbox without the domain allowlisted) -> clones fail clearly and nothing is written. """ @@ -65,12 +71,13 @@ def repo_url(env, repo): return f"https://{cred}@{host}/{repo}.git" -def clone(env, dest): - url = repo_url(env, env["COMPONENTS_REPO"]) +def clone(env, dest, repo=None): + repo = repo or env["COMPONENTS_REPO"] + url = repo_url(env, repo) r = subprocess.run(["git", "clone", url, dest], capture_output=True, text=True) if r.returncode != 0: err = (r.stderr or "").replace(env["GIT_TOKEN"], "***") - sys.exit(f"clone COMPONENTS failed (host reachable / token scope / repo exists?):\n{err[:400]}") + sys.exit(f"clone {repo} failed (host reachable / token scope / repo exists?):\n{err[:400]}") return dest @@ -190,6 +197,90 @@ def cmd_push_part(env, args): commit_push(tmp, env.get("GIT_TOKEN", ""), args.message or f"library-manager: add {tag}") +# ------------------------------------------------------------------ skill repo + changelog + +CHANGELOG_REL = os.path.join("assets", "CHANGELOG.xlsx") + + +def _changelog_rows(path): + """(workbook, sheet, [row-tuples]) for a changelog xlsx; ([]) if the file is absent.""" + import openpyxl + if not os.path.exists(path): + return None, None, [] + wb = openpyxl.load_workbook(path) + ws = wb["Changelog"] if "Changelog" in wb.sheetnames else wb.active + rows = [tuple("" if c is None else str(c) for c in r) + for r in ws.iter_rows(min_row=2, values_only=True)] + return wb, ws, rows + + +def merge_changelog(local_path, repo_path): + """Append the local changelog's rows onto the repo's existing changelog (union, order + preserved: repo history first, then any local rows not already there). This is what keeps + Gitea's changelog as the growing source of truth instead of being overwritten. Writes the + merged file to repo_path and returns how many new rows were added.""" + import shutil + if not os.path.exists(local_path): + return 0 + if not os.path.exists(repo_path): # repo has none yet -> seed from local + os.makedirs(os.path.dirname(repo_path), exist_ok=True) + shutil.copy(local_path, repo_path) + _, _, rows = _changelog_rows(repo_path) + return len(rows) + repo_wb, repo_ws, repo_rows = _changelog_rows(repo_path) + _, _, local_rows = _changelog_rows(local_path) + seen = set(repo_rows) + added = 0 + for row in local_rows: + if row not in seen: + repo_ws.append(list(row)) + seen.add(row) + added += 1 + repo_wb.save(repo_path) + return added + + +def _copy_skill_files(src, dst): + """Copy the skill's own files into a repo clone, minus junk and the changelog (merged + separately) and WITHOUT the real token (config/gitea.env is copied with GIT_TOKEN blanked).""" + skip = {".git", "__pycache__"} + for root, dirs, files in os.walk(src): + dirs[:] = [d for d in dirs if d not in skip] + rel = os.path.relpath(root, src) + for f in files: + if f.endswith((".bak.xlsx",)) or "OLD-18class" in f: + continue + relpath = os.path.normpath(os.path.join(rel, f)) + if relpath == CHANGELOG_REL: # changelog handled by merge, not copy + continue + s = os.path.join(root, f) + d = os.path.join(dst, relpath) + os.makedirs(os.path.dirname(d), exist_ok=True) + if relpath == os.path.join("config", "gitea.env"): + text = open(s, encoding="utf-8").read() + text = re.sub(r"(?m)^(GIT_TOKEN=).*$", r"\1", text) # blank the secret + open(d, "w", encoding="utf-8").write(text) + else: + shutil.copy2(s, d) + + +def cmd_push_skill(env, args): + """Push this skill's files to the SKILL repo and MERGE the changelog into the repo's + existing one (append-only, so previous entries are preserved). The merged changelog is + also written back to the local skill assets so local and Gitea stay in sync.""" + src = os.path.abspath(args.src) + tmp = tempfile.mkdtemp(prefix="skill_") + clone(env, tmp, env["SKILL_REPO"]) + _copy_skill_files(src, tmp) + local_cl = os.path.join(src, CHANGELOG_REL) + repo_cl = os.path.join(tmp, CHANGELOG_REL) + added = merge_changelog(local_cl, repo_cl) + if os.path.exists(repo_cl): # sync merged history back to local + shutil.copy(repo_cl, local_cl) + print(f"changelog: merged (+{added} new row(s)) into the skill repo's CHANGELOG.xlsx") + commit_push(tmp, env.get("GIT_TOKEN", ""), args.message or "library-manager: sync skill files + changelog") + + def main(): ap = argparse.ArgumentParser() ap.add_argument("--config", default=CFG_DEFAULT) @@ -212,15 +303,25 @@ def main(): p = sub.add_parser("push-part"); p.add_argument("--folder", required=True) p.add_argument("--typeid", required=True); p.add_argument("--message") + p = sub.add_parser("push-skill") + p.add_argument("--src", default=os.path.join(os.path.dirname(__file__), ".."), + help="skill directory to push (defaults to this skill)") + p.add_argument("--message") + args = ap.parse_args() env = load_env(args.config) - if args.cmd in ("check-mpn", "checkout", "commit-push", "push-part") or \ - (args.cmd in ("list-type", "place-part") and not getattr(args, "local", None) and not getattr(args, "root", None)): - for k in ("GIT_HOST", "GIT_TOKEN", "COMPONENTS_REPO"): - if not env.get(k): - sys.exit(f"missing {k} in env/config") + needs = {"GIT_HOST", "GIT_TOKEN"} + if args.cmd == "push-skill": + needs.add("SKILL_REPO") + elif args.cmd in ("check-mpn", "checkout", "commit-push", "push-part") or \ + (args.cmd in ("list-type", "place-part") and not getattr(args, "local", None) and not getattr(args, "root", None)): + needs.add("COMPONENTS_REPO") + for k in sorted(needs): + if not env.get(k): + sys.exit(f"missing {k} in env/config") {"check-mpn": cmd_check_mpn, "checkout": cmd_checkout, "list-type": cmd_list_type, - "place-part": cmd_place_part, "commit-push": cmd_commit_push, "push-part": cmd_push_part}[args.cmd](env, args) + "place-part": cmd_place_part, "commit-push": cmd_commit_push, "push-part": cmd_push_part, + "push-skill": cmd_push_skill}[args.cmd](env, args) if __name__ == "__main__":