library-manager: sync skill files (rename + Excel changelog + push-skill)

SKILL.md

@@ -20,8 +20,18 @@ description: >-
 Turn one component datasheet into a verified, versioned library entry in Gitea. The guiding
 idea is honesty and traceability: every value lands in the right column and unit, anything
 the datasheet doesn't state stays blank, and nothing reaches Gitea until a human has
-confirmed it. This is an **interactive** skill — it asks at the few points where a person's
+confirmed it.
-judgement or an upload is genuinely needed, and does everything else on its own.
+
+## Ask, don't assume
+
+This is a deliberately **interactive** skill. At every decision point, **ask the user and
+wait for an answer — do not assume a default and proceed.** In particular, always confirm:
+the **make**; the **typeid/classification** you inferred; whether any **new parameters**
+should be added to the template; the extracted **values** (the verification loop); that the
+**symbol/footprint actually match this part**; whether to **apply a new parameter to existing
+parts** (backfill); and **before every push to Gitea** (the part, the skill files, the
+changelog). When something is ambiguous, ask a specific question rather than guessing. It is
+always better to ask one more question than to write the wrong thing into the library.
 
 ## Inputs
 
@@ -108,9 +118,10 @@ Check whether that typeid has a sheet in `assets/template/template.xlsx`.
 
     This appends the column(s) at the end of that typeid's sheet, **bumps that typeid's
     Template Version and Skill Version together** (v1→v2 — see *Per-typeid versioning*),
-    and writes one entry to the global changelog `assets/template/CHANGELOG.md`. Push the
+    and writes one row to the global changelog `assets/CHANGELOG.xlsx`. Then **ask
-    updated `template.xlsx` + `versions.json` + `CHANGELOG.md` to the **skill** repo, and
+    the user before pushing**, and sync the updated skill files + changelog to the skill repo
-    make sure the components repo's copies stay consistent.
+    with `push-skill` (see *Pushing the skill repo*) — that merges the new changelog row onto
+    the one already in Gitea rather than overwriting it.
 
     Then ask: **should this change apply to the parts of this typeid already in Gitea?**
     - **No** → go to step 4 (only the current part gets the new column).
@@ -206,10 +217,16 @@ reading and writing these numbers.
 
 ## The changelog
 
-`append_parameter.py` maintains one **global** changelog at `assets/template/CHANGELOG.md`
+`append_parameter.py` maintains one **global** changelog as an Excel workbook at
-(newest entry on top), recording for each change: date, typeid + class, the template and
+`assets/CHANGELOG.xlsx` (sheet `Changelog`, styled green header). Every time a
-skill version transitions, the parameter(s) added, and the description. Push it to the skill
+typeid's template/version changes, one row is appended with columns
-repo alongside `template.xlsx` and `versions.json` whenever it changes.
+**Date | Typeid | Skill Version | Template Version | Description** — the version columns hold
+the new versions, and Description is your note (or the parameter(s) added if you gave none).
+
+The changelog lives in the **skill repo in Gitea** as well. When you push (via `push-skill`),
+the new local rows are **merged onto the changelog already in Gitea** — appended, never
+overwritten — so the Gitea copy is the growing, authoritative history across machines and
+sessions, and the merged file is copied back locally so the two stay in sync.
 
 ## Backfilling existing parts
 
@@ -240,13 +257,19 @@ python scripts/gitea_components.py commit-push --root work/ --message "backfill
 ## Pushing the skill repo
 
 When skill files change (a new typeid template, a parameter add, a version/changelog bump),
-push the skill's own files — but **never push the real token**. Blank the `GIT_TOKEN` line in
+**ask the user first**, then push the skill's own files to the skill repo with `push-skill`:
-the copy you push, or exclude `config/gitea.env`:
 
 ```bash
-bash scripts/push_to_gitea.sh --repo "$SKILL_REPO" --src <skill-dir> --message "Sync skill files"
+python scripts/gitea_components.py push-skill --message "Sync skill files + changelog"
 ```
 
+`push-skill` clones the skill repo, copies the skill files in with the **`GIT_TOKEN` blanked
+out** (the real token never leaves the machine), and **merges** `CHANGELOG.xlsx` — appending
+this run's new rows onto the changelog already in Gitea so earlier entries are preserved — then
+writes the merged changelog back locally. (The older `push_to_gitea.sh` still exists for a
+plain flat push, but it does not merge the changelog or blank the token, so prefer
+`push-skill` for the skill repo.)
+
 ## Resources
 
 - `assets/template/template.xlsx` — the master template: one sheet per **typeid** (125),
@@ -255,7 +278,8 @@ bash scripts/push_to_gitea.sh --repo "$SKILL_REPO" --src <skill-dir> --message "
   `Footprint Ref/Path` and `Manufacturer` sit near the end.
 - `assets/template/Type_ID.xlsx` + `references/taxonomy.md` — Class → Subclass → Type ID.
 - `assets/template/versions.json` — per-typeid `template_version` + `skill_version`.
-- `assets/template/CHANGELOG.md` — global version/parameter changelog (created on first add).
+- `assets/CHANGELOG.xlsx` — global version/parameter changelog (created on first add;
+  merged into the skill repo's copy in Gitea by `push-skill`).
 - `scripts/common.py` — taxonomy loader (`load_taxonomy`, `class_folder`), version store
   (`get_versions`, `version_labels`, `bump_versions`), and the tag helper (`part_tag`).
 - `scripts/fill_templates.py` — build one per-part `<tag>.xlsx` (version-stamped); reused for
@@ -264,7 +288,8 @@ bash scripts/push_to_gitea.sh --repo "$SKILL_REPO" --src <skill-dir> --message "
   the changelog.
 - `scripts/altium_refs.py` — read Library/Footprint Ref from `.SchLib`/`.PcbLib`.
 - `scripts/gitea_components.py` — `check-mpn`, `checkout`, `list-type`, `place-part`,
-  `commit-push`, `push-part` against the components repo.
+  `commit-push`, `push-part` (components repo), and `push-skill` (skill repo: token-blanked
+  push + append-only changelog merge).
 - `scripts/push_to_gitea.sh` — push a folder's contents to a Gitea repo (used for the skill
   repo).
 - `config/gitea.env` — host, user, token, and the `SKILL_REPO` / `COMPONENTS_REPO` names

assets/template/versions.json

@@ -148,8 +148,8 @@
     "template_version": 1
   },
   "ESD": {
-    "skill_version": 2,
+    "skill_version": 1,
-    "template_version": 2
+    "template_version": 1
   },
   "FAN": {
     "skill_version": 1,

config/gitea.env

@@ -3,11 +3,11 @@
 # Use a token scoped to these repos (repository: write) and rotate periodically.
 GIT_HOST=gitea.vecmocon.com
 GIT_USER=nitishKumar
-GIT_TOKEN=451bff1dc32202cbc0a371f8e5645079466d2120
+GIT_TOKEN=
 # Target repos — TWO now (set these to the exact repo names on your Gitea):
 #   SKILL_REPO      : holds this skill's own files (SKILL.md, scripts, assets, ...).
 #   COMPONENTS_REPO : holds components, one folder per Class (Diode, IC, ...); inside each,
 #                     one MPN_make_typeid/ folder per part with { xlsx, datasheet, symbol,
 #                     footprint }. (Replaces the old DFS + Parameters repos.)
-SKILL_REPO=nitishKumar/Skill_Assets
+SKILL_REPO=nitishKumar/skill
 COMPONENTS_REPO=nitishKumar/Components

scripts/append_parameter.py

@@ -8,19 +8,23 @@ bumps v1->v2 too. Only SCH moves; every other typeid keeps its versions. Those t
 are what fill_templates later stamps into the SCH sheet's rows (columns B and C).
 
 The new column is appended at the END of the sheet (as requested), styled like the other
-headers. Then versions.json is bumped and one entry is written to CHANGELOG.md.
+headers. Then versions.json is bumped and one entry is written to CHANGELOG.xlsx.
 
   python append_parameter.py --typeid SCH \
       --param "Reverse Recovery Time(ns)" --param "Diode Capacitance(pF)" \
       --desc "Added Trr and Cd for SMPS rectifier derating"
 
-After running, push the updated template.xlsx + versions.json + CHANGELOG.md to the SKILL
+After running, push the updated template.xlsx + versions.json + CHANGELOG.xlsx to the SKILL
 repo (and, per the workflow, the same change flows to the components repo copies).
 """
 import argparse, datetime, os
 import openpyxl
 from copy import copy
-from common import (TEMPLATE_XLSX, CHANGELOG_MD, bump_versions, load_taxonomy)
+from openpyxl.styles import Font, PatternFill, Border, Side, Alignment
+from common import (TEMPLATE_XLSX, CHANGELOG_XLSX, bump_versions)
+
+CHANGELOG_HEADERS = ["Date", "Typeid", "Skill Version", "Template Version", "Description"]
+GREEN = "B6D7A8"; GRAY = "BFBFBF"
 
 
 def _style_header(new_cell, ref_cell):
@@ -54,26 +58,50 @@ def append_params(template_path, typeid, params):
     return added
 
 
+def _thin():
+    s = Side(style="thin", color=GRAY)
+    return Border(left=s, right=s, top=s, bottom=s)
+
+
+def _new_changelog_book():
+    """Create the changelog workbook with a styled header row."""
+    wb = openpyxl.Workbook()
+    ws = wb.active
+    ws.title = "Changelog"
+    for c, h in enumerate(CHANGELOG_HEADERS, start=1):
+        cell = ws.cell(1, c, h)
+        cell.font = Font(name="Calibri", bold=True)
+        cell.fill = PatternFill("solid", fgColor=GREEN)
+        cell.border = _thin()
+        cell.alignment = Alignment(horizontal="center", vertical="center")
+    for col, w in zip("ABCDE", (18, 10, 14, 16, 60)):
+        ws.column_dimensions[col].width = w
+    ws.freeze_panes = "A2"
+    return wb
+
+
 def write_changelog(typeid, added, old, new, desc, when=None):
+    """Append one row to the Excel changelog for a typeid's template/version change.
+    Columns: Date | Typeid | Skill Version | Template Version | Description. The version
+    columns show the NEW versions; Description is the user's note (or the added params)."""
     when = when or datetime.datetime.now()
-    folder = (load_taxonomy().get(typeid) or {}).get("folder", "?")
+    detail = desc or ("Added parameter(s): " + ", ".join(added))
-    lines = [
+    if os.path.exists(CHANGELOG_XLSX):
-        f"## {when.strftime('%Y-%m-%d %H:%M')} - {typeid}  ({folder})",
+        wb = openpyxl.load_workbook(CHANGELOG_XLSX)
-        f"- Template v{old['template_version']} -> v{new['template_version']}, "
+        ws = wb["Changelog"] if "Changelog" in wb.sheetnames else wb.active
-        f"Skill v{old['skill_version']} -> v{new['skill_version']}",
+    else:
-        "- Added parameter(s): " + ", ".join(f'\"{p}\"' for p in added),
+        wb = _new_changelog_book()
-    ]
+        ws = wb.active
-    if desc:
+    row = [when.strftime("%Y-%m-%d %H:%M"), typeid,
-        lines.append(f"- Description: {desc}")
+           f"v{new['skill_version']}", f"v{new['template_version']}", detail]
-    entry = "\n".join(lines) + "\n\n"
+    r = ws.max_row + 1
-    header = "# library-manager - template/version changelog\n\n"
+    for c, v in enumerate(row, start=1):
-    prior = ""
+        cell = ws.cell(r, c, v)
-    if os.path.exists(CHANGELOG_MD):
+        cell.border = _thin()
-        prior = open(CHANGELOG_MD, encoding="utf-8").read()
+        cell.alignment = Alignment(vertical="center",
-        if prior.startswith(header):
+                                   wrap_text=(c == 5), horizontal="left")
-            prior = prior[len(header):]
+    wb.save(CHANGELOG_XLSX)
-    open(CHANGELOG_MD, "w", encoding="utf-8").write(header + entry + prior)  # newest on top
+    return CHANGELOG_XLSX
-    return CHANGELOG_MD
 
 
 def main():
@@ -92,7 +120,7 @@ def main():
     print(f"{a.typeid}: template v{old['template_version']}->v{new['template_version']}, "
           f"skill v{old['skill_version']}->v{new['skill_version']}")
     print(f"Changelog updated: {cl}")
-    print("Now push template.xlsx + versions.json + CHANGELOG.md to the SKILL repo.")
+    print("Now push template.xlsx + versions.json + CHANGELOG.xlsx to the SKILL repo.")
 
 
 if __name__ == "__main__":

scripts/common.py

@@ -21,7 +21,7 @@ TEMPLATE_DIR = os.path.join(SKILL_ROOT, "assets", "template")
 TEMPLATE_XLSX= os.path.join(TEMPLATE_DIR, "template.xlsx")
 TYPE_ID_XLSX = os.path.join(TEMPLATE_DIR, "Type_ID.xlsx")
 VERSIONS_JSON= os.path.join(TEMPLATE_DIR, "versions.json")
-CHANGELOG_MD = os.path.join(TEMPLATE_DIR, "CHANGELOG.md")
+CHANGELOG_XLSX = os.path.join(SKILL_ROOT, "assets", "CHANGELOG.xlsx")
 
 # Fixed template columns that live on every sheet and are NOT extracted from a datasheet.
 COL_TAG          = "MPN_make_type"     # column A header (the part tag lives here)

scripts/gitea_components.py

@@ -32,6 +32,12 @@ testing or offline work):
   push-part   --folder staging/BAT46WJ_Nexperia_SCH --typeid SCH [--message "..."]
        Convenience: clone -> place-part -> commit-push in one go (the common single-part path).
 
+  push-skill  [--src <skill-dir>] [--message "..."]
+       Push this skill's files to the SKILL repo (token blanked out) and MERGE the changelog:
+       the new rows in the local CHANGELOG.xlsx are appended onto the repo's existing one, so
+       previous entries in Gitea are preserved, not overwritten. The merged changelog is
+       written back locally so local and Gitea stay in sync.
+
 Host unreachable (e.g. sandbox without the domain allowlisted) -> clones fail clearly and
 nothing is written.
 """
@@ -65,12 +71,13 @@ def repo_url(env, repo):
     return f"https://{cred}@{host}/{repo}.git"
 
 
-def clone(env, dest):
+def clone(env, dest, repo=None):
-    url = repo_url(env, env["COMPONENTS_REPO"])
+    repo = repo or env["COMPONENTS_REPO"]
+    url = repo_url(env, repo)
     r = subprocess.run(["git", "clone", url, dest], capture_output=True, text=True)
     if r.returncode != 0:
         err = (r.stderr or "").replace(env["GIT_TOKEN"], "***")
-        sys.exit(f"clone COMPONENTS failed (host reachable / token scope / repo exists?):\n{err[:400]}")
+        sys.exit(f"clone {repo} failed (host reachable / token scope / repo exists?):\n{err[:400]}")
     return dest
 
 
@@ -190,6 +197,90 @@ def cmd_push_part(env, args):
     commit_push(tmp, env.get("GIT_TOKEN", ""), args.message or f"library-manager: add {tag}")
 
 
+# ------------------------------------------------------------------ skill repo + changelog
+
+CHANGELOG_REL = os.path.join("assets", "CHANGELOG.xlsx")
+
+
+def _changelog_rows(path):
+    """(workbook, sheet, [row-tuples]) for a changelog xlsx; ([]) if the file is absent."""
+    import openpyxl
+    if not os.path.exists(path):
+        return None, None, []
+    wb = openpyxl.load_workbook(path)
+    ws = wb["Changelog"] if "Changelog" in wb.sheetnames else wb.active
+    rows = [tuple("" if c is None else str(c) for c in r)
+            for r in ws.iter_rows(min_row=2, values_only=True)]
+    return wb, ws, rows
+
+
+def merge_changelog(local_path, repo_path):
+    """Append the local changelog's rows onto the repo's existing changelog (union, order
+    preserved: repo history first, then any local rows not already there). This is what keeps
+    Gitea's changelog as the growing source of truth instead of being overwritten. Writes the
+    merged file to repo_path and returns how many new rows were added."""
+    import shutil
+    if not os.path.exists(local_path):
+        return 0
+    if not os.path.exists(repo_path):                 # repo has none yet -> seed from local
+        os.makedirs(os.path.dirname(repo_path), exist_ok=True)
+        shutil.copy(local_path, repo_path)
+        _, _, rows = _changelog_rows(repo_path)
+        return len(rows)
+    repo_wb, repo_ws, repo_rows = _changelog_rows(repo_path)
+    _, _, local_rows = _changelog_rows(local_path)
+    seen = set(repo_rows)
+    added = 0
+    for row in local_rows:
+        if row not in seen:
+            repo_ws.append(list(row))
+            seen.add(row)
+            added += 1
+    repo_wb.save(repo_path)
+    return added
+
+
+def _copy_skill_files(src, dst):
+    """Copy the skill's own files into a repo clone, minus junk and the changelog (merged
+    separately) and WITHOUT the real token (config/gitea.env is copied with GIT_TOKEN blanked)."""
+    skip = {".git", "__pycache__"}
+    for root, dirs, files in os.walk(src):
+        dirs[:] = [d for d in dirs if d not in skip]
+        rel = os.path.relpath(root, src)
+        for f in files:
+            if f.endswith((".bak.xlsx",)) or "OLD-18class" in f:
+                continue
+            relpath = os.path.normpath(os.path.join(rel, f))
+            if relpath == CHANGELOG_REL:               # changelog handled by merge, not copy
+                continue
+            s = os.path.join(root, f)
+            d = os.path.join(dst, relpath)
+            os.makedirs(os.path.dirname(d), exist_ok=True)
+            if relpath == os.path.join("config", "gitea.env"):
+                text = open(s, encoding="utf-8").read()
+                text = re.sub(r"(?m)^(GIT_TOKEN=).*$", r"\1", text)   # blank the secret
+                open(d, "w", encoding="utf-8").write(text)
+            else:
+                shutil.copy2(s, d)
+
+
+def cmd_push_skill(env, args):
+    """Push this skill's files to the SKILL repo and MERGE the changelog into the repo's
+    existing one (append-only, so previous entries are preserved). The merged changelog is
+    also written back to the local skill assets so local and Gitea stay in sync."""
+    src = os.path.abspath(args.src)
+    tmp = tempfile.mkdtemp(prefix="skill_")
+    clone(env, tmp, env["SKILL_REPO"])
+    _copy_skill_files(src, tmp)
+    local_cl = os.path.join(src, CHANGELOG_REL)
+    repo_cl = os.path.join(tmp, CHANGELOG_REL)
+    added = merge_changelog(local_cl, repo_cl)
+    if os.path.exists(repo_cl):                        # sync merged history back to local
+        shutil.copy(repo_cl, local_cl)
+    print(f"changelog: merged (+{added} new row(s)) into the skill repo's CHANGELOG.xlsx")
+    commit_push(tmp, env.get("GIT_TOKEN", ""), args.message or "library-manager: sync skill files + changelog")
+
+
 def main():
     ap = argparse.ArgumentParser()
     ap.add_argument("--config", default=CFG_DEFAULT)
@@ -212,15 +303,25 @@ def main():
     p = sub.add_parser("push-part"); p.add_argument("--folder", required=True)
     p.add_argument("--typeid", required=True); p.add_argument("--message")
 
+    p = sub.add_parser("push-skill")
+    p.add_argument("--src", default=os.path.join(os.path.dirname(__file__), ".."),
+                   help="skill directory to push (defaults to this skill)")
+    p.add_argument("--message")
+
     args = ap.parse_args()
     env = load_env(args.config)
-    if args.cmd in ("check-mpn", "checkout", "commit-push", "push-part") or \
+    needs = {"GIT_HOST", "GIT_TOKEN"}
-       (args.cmd in ("list-type", "place-part") and not getattr(args, "local", None) and not getattr(args, "root", None)):
+    if args.cmd == "push-skill":
-        for k in ("GIT_HOST", "GIT_TOKEN", "COMPONENTS_REPO"):
+        needs.add("SKILL_REPO")
-            if not env.get(k):
+    elif args.cmd in ("check-mpn", "checkout", "commit-push", "push-part") or \
-                sys.exit(f"missing {k} in env/config")
+         (args.cmd in ("list-type", "place-part") and not getattr(args, "local", None) and not getattr(args, "root", None)):
+        needs.add("COMPONENTS_REPO")
+    for k in sorted(needs):
+        if not env.get(k):
+            sys.exit(f"missing {k} in env/config")
     {"check-mpn": cmd_check_mpn, "checkout": cmd_checkout, "list-type": cmd_list_type,
-     "place-part": cmd_place_part, "commit-push": cmd_commit_push, "push-part": cmd_push_part}[args.cmd](env, args)
+     "place-part": cmd_place_part, "commit-push": cmd_commit_push, "push-part": cmd_push_part,
+     "push-skill": cmd_push_skill}[args.cmd](env, args)
 
 
 if __name__ == "__main__":