tidy-repos-and-skills

Imported from jasper-tms/shell-configs (claude/skills/tidy-repos-and-skills/SKILL.md). Install upstream with npx skills add jasper-tms/shell-configs --skill tidy-repos-and-skills. Copyright stays with the author.

Tidy repos and skills (nightly maintenance)

This skill is the playbook for the autonomous nightly task ~/cron_tasks/tidy-repos-and-skills/, which runs headless (claude -p) overnight. It pulls every repo on this machine and keeps two kinds of skill index in sync with the actual SKILL.md files on disk:

• a per-folder INDEX.md in each version-controlled agent-skills folder, and
• the single global ~/.claude/skills/_SKILL_LISTING.md (names + locations).

The same steps are useful interactively whenever an index looks stale, so the skill is loadable by any agent, not only the cron job.

This task commits on its own; the wrapper pushes main

This is the deliberate exception to the normal finishing-tasks-in-repos convention (which writes a commit_*.sh script for Jasper to run). The nightly task is autonomous and self-authorized to commit its index fixes directly. Still follow the message conventions (under 73 characters, start with a verb).

Pushing main is not your job. A global gate-git-push.sh PreToolUse hook denies an agent git push to main/master/prod in this headless run, so do not attempt one. After you exit, the run.sh wrapper pushes the protected branch for the repos this task maintains (its git push runs in the wrapper shell, not an agent tool call, so the hook never gates it). Just commit and leave main to the wrapper. (You may still push a feature branch yourself if a repo happens to be on one - the hook allows that.)

Which folders are indexed: the INDEX.md itself is the opt-in

There is no hand-maintained list of indexed folders. A skills folder participates in this task — its INDEX.md is regenerated, its skills appear in the global _SKILL_LISTING.md, and its frontmatter is validated — if and only if it already contains an INDEX.md. Creating that file once (a single build_index.py run) opts a new folder in; deleting it opts out. So the set of indexed folders lives on disk, as the set of existing INDEX.md files, not in this skill. This is deliberate: it keeps the general "how to tidy" (this skill, useful to anyone) separate from the specific "what we tidy" (whatever is on this machine's disk).

Two folders that do contain real skills are deliberately left without an INDEX.md, so they never participate:

• ~/repos/jasper-tms/swiss-table-tennis-chat/skills/ — the chatbot app's own runtime skills, loaded by the app itself, not agent skills.
• ~/.claude/skills/ — a symlink farm plus the unversioned third-party runpodctl, not in any repo.

To bring a new folder in, run build_index.py on it once and add a section for it to _SKILL_LISTING.md; from then on this task keeps both current.

Regenerating an INDEX.md: use build_index.py

Never hand-transcribe an INDEX.md. This skill ships build_index.py, which reads each SKILL.md's frontmatter and deterministically rewrites the folder's INDEX.md (skills sorted by name, so diffs stay stable). This skill is intentionally not symlinked into ~/.claude/skills, so refer to build_index.py by its real path next to this SKILL.md. Run it per folder:

build_index=~/repos/jasper-tms/shell-configs/claude/skills/tidy-repos-and-skills/build_index.py
python3 "$build_index" ~/repos/jasper-tms/raspberry-pi/agent-skills

The generated file is repo-relative and clone-portable:

# Skill index for `raspberry-pi/agent-skills/`
Each line below lists a single skill as `- <skill-name>: <skill-description>`. Each skill can be found alongside this INDEX.md file at `<skill-name>/SKILL.md`
- <skill-name>: <description>
...

Validating skill frontmatter

The Agent Skills spec makes both name: and description: mandatory, and requires each skill's name: to match its parent directory name. This skill ships validate_skill_frontmatter.py, which enforces all three across every skill the global listing knows about. It reads the folder set straight from ~/repos/jasper-tms/raspberry-pi/agent-skills/_SKILL_LISTING.md (no hardcoded folder list to drift out of date), so a skill added anywhere the listing covers gets checked automatically, and it reuses build_index.py's frontmatter parser. Like build_index.py, it sits next to this SKILL.md and is intentionally not symlinked into ~/.claude/skills, so refer to it by its real path. Run it and route any output into the report's Needs attention:

validate=~/repos/jasper-tms/shell-configs/claude/skills/tidy-repos-and-skills/validate_skill_frontmatter.py
python3 "$validate"

It checks the raspberry-pi _SKILL_LISTING.md by default; pass a different listing path as an argument to check another. It prints one problem per line (or all listed skills valid) and exits nonzero when any listed skill is invalid.

Absent folders (repos not cloned on this machine) are skipped. A listed but no SKILL.md on disk line means the listing is stale - normally the reconcile step below has already removed such an entry; if it persists, the listing points at a skill that no longer exists.

Watch for asserted-state drift

Beyond the fixed checks above, stay alert for the general failure they are all instances of: a place that asserts some state should be true, which has silently drifted from what is actually true. The index files assert "these skills exist"; frontmatter asserts "name: matches the folder"; a README asserts "the cron line is 0 3 * * *" or "this file is the source of truth"; a committed crontab.txt asserts "this is the live crontab" (the machine-specific crontab -l vs crontab.txt diff, which lives in this task's run.sh prompt, not here). These drift because the assertion and the reality are edited at different times by different people or agents.

You cannot enumerate every such assertion in advance — new ones appear with every refactor. So when, in the course of the nightly pull and index work, you happen to notice a doc, README, comment, or config that clearly states something now contradicted by reality (a stale schedule, a "source of truth" that has moved, a path or filename that no longer exists, a count that no longer matches), record it under Needs attention with the file and the specific contradiction. Do not auto-fix it: the correct direction is usually ambiguous (fix the assertion, or fix the reality?), and that judgment is a human's. Reporting it is the whole value — a drifted assertion is invisible until someone trusts it.

If a particular assertion turns out to be worth checking every single night, that is a signal to promote it to a deterministic check (like the frontmatter validator above, or the crontab diff in run.sh) rather than relying on noticing it.

Nightly sequence

Run these in order. Keep a running note of everything worth reporting.

1. Pull every repo, rebasing where needed

Run pullrepos (it pulls all repos under ~/repos in parallel, times out and retries hangs, and prints a summary of any repos whose GitHub credentials aren't cached). pullrepos uses a plain git pull, which does not resolve a repo that has unpushed local commits and an advanced remote (non-fast-forward). So after pullrepos, sweep every repo and repair those:

for gitdir in ~/repos/*/.git ~/repos/*/*/.git; do
    repo="${gitdir%/.git}"
    git -C "$repo" rev-parse --abbrev-ref --symbolic-full-name @{u} \
        >/dev/null 2>&1 || continue          # skip: no upstream
    # Behind or diverged from upstream? rebase local commits on top.
    counts=$(git -C "$repo" rev-list --left-right --count HEAD...@{u} 2>/dev/null) || continue
    behind=$(printf '%s' "$counts" | awk '{print $2}')
    [ "${behind:-0}" -gt 0 ] || continue
    git -C "$repo" pull --rebase --autostash || {
        git -C "$repo" rebase --abort 2>/dev/null
        echo "REBASE CONFLICT (left for manual fix): $repo"
    }
done

Record, for the report: repos that were rebased, repos that hit a rebase conflict (aborted, untouched), and any credential failures from pullrepos.

2. Regenerate every INDEX.md on disk

Find every folder that already has an INDEX.md beside a set of skills and rebuild it. A folder without one is opted out, so it is skipped (see "Which folders are indexed"):

build_index=~/repos/jasper-tms/shell-configs/claude/skills/tidy-repos-and-skills/build_index.py
find ~/repos -maxdepth 6 -type f -name INDEX.md \
    -not -path '*/.git/*' -not -path '*/node_modules/*' | while read -r index; do
    folder=$(dirname "$index")
    ls "$folder"/*/SKILL.md >/dev/null 2>&1 || continue  # not a skill index; skip
    python3 "$build_index" "$folder"
done

Then, per repo, git diff --stat the INDEX.md files to see what actually changed. A changed INDEX.md means a skill was added, removed, or had its description edited. Some indexed folders live in other people's repos (for example scoreTec/agent-skills, whose own scoretec-tidy-up skill defers index regeneration to this task); regenerating an already-correct index is a no-op, so a commit happens only when a skill really changed.

3. Reconcile the global _SKILL_LISTING.md

~/.claude/skills/_SKILL_LISTING.md (real file: ~/repos/jasper-tms/raspberry-pi/agent-skills/_SKILL_LISTING.md) lists every skill by name under its real-folder heading, plus symlink/consumer notes. It carries names and locations only - no descriptions (those live in the INDEX.md files). This step needs judgment, which is why the task is agentic:

• Every folder that has an INDEX.md (i.e. every indexed folder) should have a section in the listing. If one is missing — a folder was opted in but never listed — add a heading with its real path and its skill names, and note it under Needs attention so a human can write the section's consumer/symlink annotation.
• Within each section, every skill directory that exists on disk must appear under the heading. Add any missing; remove any whose directory is gone.
• Skip any skill directory git ignores (a privately symlinked skill hidden via .git/info/exclude), matching build_index.py, so private skills never leak into the listing.
• Preserve the file's structure and its per-skill annotations (e.g. (NOT symlinked into ~/.claude/skills)); only add/remove skill lines (and, when a section is genuinely missing, whole sections), don't reflow existing prose.

4. Validate skill frontmatter

Run the check from "Validating skill frontmatter" above (now that the listing is reconciled against disk). Add any problems it prints - a missing mandatory name: or description:, or a name: that doesn't match its folder - to the report's Needs attention section. These need a human to fix the offending SKILL.md; the nightly task does not rewrite skill frontmatter itself.

5. Commit per repo (the wrapper pushes main)

For each repo touched in steps 1-3, commit the changed files with a verb-first message under 73 characters (e.g. Refresh skill INDEX.md files, Sync _SKILL_LISTING.md with skills on disk). Do not git push a main/master/prod branch - the hook blocks it and the wrapper pushes it for you after you exit (see "This task commits on its own; the wrapper pushes main"). The wrapper records what it pushed, or any rejected push, into the report itself, so you do not need to. If a repo happens to be on a feature branch, you may push that yourself; never force-push.

6. Write the report file

Write your final summary to the path in the TIDY_REPORT_FILE environment variable. The first line is the machine-readable status the wrapper keys on:

• STATUS: quiet - everything pulled cleanly, no index drift, nothing committed, nothing needs attention. The wrapper sends no email.
• STATUS: report - anything changed or anything needs attention. The wrapper emails the rest of the file.

Keep the report terse - it is read at a glance, and a routine success must not cost more than one line. After the status line, write:

1. A header line: tidy-repos-and-skills - <YYYY-MM-DD>.

2. Exactly one line per step below, each prefixed ✓ (fine) or ⚠ (needs attention). A ✓ line is at most one line long: state the outcome, naming what changed in parentheses only if something did. Do not narrate how you did it, which skills you followed, or which sub-steps had nothing to report - a clean step is just ✓ <step>: OK.

   Use these descriptive step labels verbatim (keep the label; only the part after the colon varies):

   • ✓ Pull all repos: OK - or ✓ Pull all repos: OK (rebased: <repos>).
   • ✓ Regenerate agent-skills INDEX.md: no change - or ... : updated <repos>.
   • ✓ Reconcile global _SKILL_LISTING.md: no change - or ... : <what changed>.
   • ✓ Validate skill frontmatter: valid - or ⚠ Validate skill frontmatter: <problems>.
   • ✓ Check crontab vs source: in sync - or ⚠ Check crontab vs source: <diff summary>.

   (The wrapper appends its own push-outcome line after you exit - do not write one, and do not report on pushing main.)

3. A ⚠ Needs attention (<n>): block only if something needs attention: a rebase conflict, a credential failure, a frontmatter problem, a crontab diff, or asserted-state drift you noticed (see "Watch for asserted-state drift"). This is the one place to expand - give each item the file/command and the specific contradiction, enough to act on without re-investigating. Every step line you marked ⚠ above must appear here in full. Omit the whole block when nothing needs attention.

Anything needing attention makes the overall status STATUS: report (the wrapper also flips it to report if its push was rejected). If you could not finish, still write the file with STATUS: report, whatever ✓/⚠ step lines you completed, and a ⚠ Needs attention note on how far you got - a missing report file makes the wrapper send a generic failure email.

Example of a run with one real issue (this is the whole email):

STATUS: report
tidy-repos-and-skills - 2026-09-10

✓ Pull all repos: OK (rebased: SportID)
✓ Regenerate agent-skills INDEX.md: updated reaction-test
✓ Reconcile global _SKILL_LISTING.md: no change
✓ Validate skill frontmatter: valid
✓ Check crontab vs source: in sync

⚠ Needs attention (1):
- uv-sources clean filter not configured on this Pi; checkout/rebase leaves
  [tool.uv.sources] staged, and a commit could leak local absolute paths.
  Fix: using-uv INSTALL.md steps 4-5.

A fully clean run is STATUS: quiet and sends no email at all - so every email that does arrive has at most a handful of ✓ lines plus, when it matters, the ⚠ Needs attention block.

Testing without spamming Jasper

run.sh --dry-run runs the whole task but the email step only prints what it would send. Use it to verify behavior without mailing anyone.