Imported from xzzpig/pi-extensions (
.agents/skills/pi-upstream-subtree/SKILL.md). Install upstream withnpx skills add xzzpig/pi-extensions --skill pi-upstream-subtree. Copyright stays with the author.
Pi Upstream Subtree
Manage upstream-derived packages under packages/pi-* without hiding the
source ref or synchronization commit. The JSON record in subtrees/ is the
source of truth consumed by the direnv runtime helper; the helper validates
records against the schema, enforces ref consistency, and exports
PI_UPSTREAM_* variables. All synchronization is performed by the explicit
git commands below.
Tracking and commit invariants
Synchronize upstream content only with git subtree:
- First import:
git subtree add --prefix="packages/<name>" --squash <remote> <commit>. - Existing subtree:
git subtree pull --prefix="packages/<name>" --squash <remote> <commit>.
Never replace the subtree prefix through git archive, ordinary git merge,
cp, rsync, extraction, generated patches, or a manual file replacement.
Those approaches can reproduce the files while losing the squash parent and
git-subtree-dir / git-subtree-split trailers required for future pulls. If
git subtree cannot complete safely, stop and resolve that blocker rather than
falling back to a file-copy update.
An explicit user request to add, import, pull, synchronize, or update an
upstream subtree authorizes the local commits required by this workflow,
including the synchronization commit created by git subtree. It does not
require an immediate follow-up commit for fork adaptations, metadata, version,
or changelog changes. Keep those repository-owned changes uncommitted until all
post-synchronization work and verification are complete, then commit them
together when a follow-up commit is needed. Otherwise, leave them in the
worktree for review. The authorization does not extend to push, publication,
unrelated commits, or history rewrites.
Establish the root and inspect state
ROOT="${PI_EXTENSIONS_ROOT:-$(git rev-parse --show-toplevel)}"
cd "$ROOT"
direnv allow
git status --short --branch
add, pull, record, split, and push require a clean worktree. Do not
fold unrelated changes into a subtree synchronization; stop or handle them
under separate authorization before running the workflow.
Metadata contract
Store one JSON record at subtrees/<plugin-name>.json. The shape is defined
in schemas/subtree-metadata.schema.json
and documented in subtrees/AGENTS.md. A valid
record must contain:
name: local record/directory name, unscoped, matching^[a-z0-9][a-z0-9-]*$. Pi plugin packages use thepi-*form; an upstream-derived support library that is not a Pi plugin may keep the plain upstream-derived name (e.g.sandbox-runtime). This is subtree bookkeeping only — NOT the published npm name (see "Forked package npm naming").prefix: exactlypackages/<name>.upstreamPath(optional): relative directory inside a monorepo source. Omit it when the plugin is the source repository root.source: upstream Git source.remote: exactlyupstream-<name>.ref: branch, tag, or commit-ish.upstreamCommit: 40-character commit recorded at synchronization.squash:true.lastSyncedAt: ISO timestamp.notes(optional): short summary of the fork; the maintenance lists live in the two arrays below, and capability intent lives inopenspec/.reapplyOnSync(optional): array of strings, one entry per adaptation a sync must re-apply by hand. Omit the field rather than writing an empty array — the audit rejects an empty list.doNotReintroduce(optional): array of strings, one entry per decision never to re-introduce (a dropped divergence, or a deliberate non-change).knownDebt(optional): array of accepted divergences, each{kind, reason, path?, recordedAt?}withkindinnoise | deleted | lint | test | other.noiseanddeletedentries require apath(exact or shell glob) so the audit can reconcile findings; the field is validated by the schema and consumed byaudit-fork-divergence.sh. Record debt there rather than innotes, and delete the entry once the divergence is gone.
Do not put credentials in source. Copy
subtrees/template.json.example only as a shape reference; do not commit it as
an active record.
Helper behavior on load
When direnv loads the project, env/ensure-upstreams.mjs validates every
active record against the schema, ensures the upstream-* remote exists with
the recorded URL, records the accepted ref in remote.<name>.pi-ref, and
refuses an unreviewed ref change. A direct ref edit in the JSON without a
corresponding pull --ref produces a failure on the next direnv reload.
After adding or changing a record, run:
direnv reload
Forked package npm naming (二开包命名)
Every package imported from upstream is a locally forked (二开) package. It
keeps the unscoped pi-* name for the directory, subtree prefix, and metadata
record, but the npm package name in package.json MUST be the scoped
@xzzpig/pi-* form:
| Where | Format | Example |
|---|---|---|
| Directory / prefix | packages/pi-* |
packages/pi-tool-display |
Metadata name |
pi-* |
pi-tool-display |
npm package.json name |
@xzzpig/pi-* |
@xzzpig/pi-tool-display |
This makes every forked package installable as pi install npm:@xzzpig/pi-*,
keeps the local fork clearly distinguishable from the upstream package on npm,
and matches the repo-wide convention in README.md. Upstream ships its own
name (e.g. @gotgenes/pi-permission-system); the fork MUST NOT keep it.
Rename the npm name after the import, before unrelated local changes. Keep the
rename and its required README, manifest, and code adaptations together as one
post-import change set, and verify the complete change set before creating a
follow-up commit:
# name=pi-tool-display → npm name @xzzpig/pi-tool-display
jq --arg n "@xzzpig/${name}" '.name = $n' "packages/${name}/package.json" \
> "packages/${name}/package.json.tmp" && \
mv "packages/${name}/package.json.tmp" "packages/${name}/package.json"
# Fix README, manifest, and code references to the upstream name.
Keep the scope OUT of subtrees/*.json records: the schema validates name
against the unscoped pi-* pattern and rejects @xzzpig/pi-* there.
Fork divergence discipline (二开分歧纪律) — MANDATORY
Fork-divergence discipline now lives in its own skill,
pi-fork-divergence. Following it is
required, not optional, for any secondary development (二开) of an
upstream-derived package.
Before writing, adapting, or reviewing fork code, and before every sync-time
adaptation or follow-up commit, load and obey pi-fork-divergence:
- Put new logic in fork-only files and leave only a minimal seam in upstream files; never reformat upstream files or leave large fork blocks in them.
- Keep fork tests and docs in fork-only files.
- Keep the
subtrees/<name>.jsonnotes,reapplyOnSyncanddoNotReintroduceentries current, declare every accepted divergence in itsknownDebtarray, then run the whitespace audit before committing any diff that touches an upstream file.
Do not commit fork code that has not passed that discipline. This skill owns
only the import, pull, metadata, ref, conflict-resolution, and
synchronization mechanics; the discipline itself is authoritative in
pi-fork-divergence.
Add an upstream plugin
The following sequence creates the remote, imports the subtree, and writes the metadata record. Replace variables with the actual plugin name, source, ref, and optional version label.
# Set variables once.
name=pi-upstream-plugin
source=https://github.com/example/pi-upstream-plugin.git
ref=main
version=v1.2.3
# Validate naming.
# Validate naming (pi-* for plugins; plain names allowed for support libraries).
echo "$name" | grep -qE '^[a-z0-9][a-z0-9-]*$' || exit 1
# Create remote, fetch, and import.
git remote add "upstream-${name}" "$source"
git fetch "upstream-${name}" "$ref"
commit=$(git rev-parse FETCH_HEAD)
git subtree add --prefix="packages/${name}" --squash "upstream-${name}" "$commit"
test "$(git rev-list --parents -n 1 HEAD | wc -w)" -eq 3
git show -s --format=%B HEAD^2 | grep -Fx "git-subtree-dir: packages/${name}"
git show -s --format=%B HEAD^2 | grep -Fx "git-subtree-split: ${commit}"
# Record accepted ref.
git config --local "remote.upstream-${name}.pi-ref" "$ref"
# Write metadata record.
cat > "subtrees/${name}.json" <<EOF
{
"\$schema": "../schemas/subtree-metadata.schema.json",
"name": "${name}",
"prefix": "packages/${name}",
"source": "${source}",
"remote": "upstream-${name}",
"ref": "${ref}",
"version": ${version:+"\"${version}\""}${version:-null},
"upstreamCommit": "${commit}",
"squash": true,
"lastSyncedAt": "$(date -Iseconds)"
}
EOF
# Leave metadata uncommitted while completing post-import adaptations.
After the import, rename the npm package to @xzzpig/<name> (see "Forked
package npm naming" above) and adapt the Pi manifest when the upstream layout
is not already a valid package. Keep the metadata, naming, manifest, and other
repository-owned changes uncommitted while running the post-import checks; do
not create a follow-up commit immediately after the subtree merge. Confirm the
helper accepts the record:
direnv reload
Pull upstream changes
Upstream subdirectory
When a plugin lives below the root of a monorepo, set upstreamPath in its
metadata. upstreamCommit MUST remain the exact commit from source; never
replace it with the derived split commit. The split commit belongs only in the
git-subtree-split trailer and makes the local prefix synchronizable.
For each update, use this sequence before the regular record update:
- Fetch the tag or branch through the metadata remote and save its root commit.
- In a clean local mirror of
source, create a split branch forupstreamPathat that root commit. Do not rungit subtree splitagainst the local fork, because it would include local divergence. - Fetch that split branch through a separate local transport remote, then use
git subtree pull --squashfrom the split branch into the localprefix. The metadataremoteremains the source remote, never the local split transport. - Verify the split tree equals
<upstreamCommit>:<upstreamPath>, verify the squash trailer names the split commit, and update metadata with the root commit,upstreamPath, ref, version, and timestamp.
name=pi-upstream-plugin
ref=v1.2.3
upstream_path=packages/pi-upstream-plugin
source_mirror=/absolute/path/to/clean/upstream-mirror
split_remote=local-pi-upstream-plugin-split
git fetch "upstream-${name}" "$ref"
upstream_commit=$(git rev-parse FETCH_HEAD)
# The mirror's origin must be the same source as the metadata record.
git -C "$source_mirror" fetch origin "$ref"
test "$(git -C "$source_mirror" rev-parse FETCH_HEAD)" = "$upstream_commit"
split_branch="split-${name}-${upstream_commit:0:12}"
git -C "$source_mirror" subtree split \
--prefix="$upstream_path" \
--branch="$split_branch" \
"$upstream_commit"
split_commit=$(git -C "$source_mirror" rev-parse "$split_branch")
test "$(git -C "$source_mirror" rev-parse "${split_commit}^{tree}")" = \
"$(git -C "$source_mirror" rev-parse "${upstream_commit}:${upstream_path}")"
# A local transport remote is only a carrier for the derived split branch.
if git remote get-url "$split_remote" >/dev/null 2>&1; then
test "$(git remote get-url "$split_remote")" = "$source_mirror"
else
git remote add "$split_remote" "$source_mirror"
fi
git fetch "$split_remote" "$split_branch"
git subtree pull --prefix="packages/${name}" --squash "$split_remote" "$split_branch"
test "$(git rev-list --parents -n 1 HEAD | wc -w)" -eq 3
git show -s --format=%B HEAD^2 | grep -Fx "git-subtree-dir: packages/${name}"
git show -s --format=%B HEAD^2 | grep -Fx "git-subtree-split: ${split_commit}"
Then write upstreamPath: "$upstream_path" and upstreamCommit: "$upstream_commit" into the record. For a ref change, set
remote.upstream-${name}.pi-ref to the new ref before direnv reload.
Same ref
name=pi-upstream-plugin
ref=main # same as record
git fetch "upstream-${name}" "$ref"
commit=$(git rev-parse FETCH_HEAD)
If $commit equals the current upstreamCommit in the record, the subtree
is already synchronized. Otherwise, pull through git subtree and verify that
its squash parent preserves tracking trailers while the synchronization merge
remains HEAD:
git subtree pull --prefix="packages/${name}" --squash "upstream-${name}" "$commit"
test "$(git rev-list --parents -n 1 HEAD | wc -w)" -eq 3
git show -s --format=%B HEAD^2 | grep -Fx "git-subtree-dir: packages/${name}"
git show -s --format=%B HEAD^2 | grep -Fx "git-subtree-split: ${commit}"
Then update the record:
version=v1.2.4 # optional
jq --arg c "$commit" --arg v "$version" --arg t "$(date -Iseconds)" \
'.upstreamCommit = $c | .version = $v | .lastSyncedAt = $t' \
"subtrees/${name}.json" > "subtrees/${name}.json.tmp" &&
mv "subtrees/${name}.json.tmp" "subtrees/${name}.json"
Leave the record uncommitted while applying fork adaptations, version changes, and changelog updates. Run the full verification workflow first. If a local follow-up commit is needed, create one coherent commit only after those checks pass; otherwise, report the expected uncommitted files for review.
Fork package versioning
subtrees/<name>.json records the upstream release version, while
packages/<name>/package.json and root versions.json record the independent
@xzzpig/<name> fork release. Never copy the upstream package's version into
the fork manifest automatically.
After every upstream sync, compare the old and new upstream release:
- An upstream major or minor release with user-visible changes increments the
fork's minor version and resets its patch component (
0.xforks use the minor component for breaking upstream changes). - An upstream patch-only release increments the fork's patch version.
- Update
package.json,versions.json, and any package-local lockfile version fields together, then add a local changelog entry.
Keep these version and changelog edits in the same uncommitted post-sync change set as the metadata and fork adaptations until verification passes.
Ref change (explicit override)
Do not edit ref in the JSON and reload direnv; the helper rejects it. To
switch to a different ref:
new_ref=develop
git fetch "upstream-${name}" "$new_ref"
commit=$(git rev-parse FETCH_HEAD)
git subtree pull --prefix="packages/${name}" --squash "upstream-${name}" "$commit"
test "$(git rev-list --parents -n 1 HEAD | wc -w)" -eq 3
git show -s --format=%B HEAD^2 | grep -Fx "git-subtree-dir: packages/${name}"
git show -s --format=%B HEAD^2 | grep -Fx "git-subtree-split: ${commit}"
git config --local "remote.upstream-${name}.pi-ref" "$new_ref"
Then update the JSON record: set ref to the new value, upstreamCommit to
the new commit, and lastSyncedAt to the current timestamp. Keep the result
uncommitted with the other post-sync changes until adaptation and verification
are complete; do not create an immediate metadata-only commit.
Conflict resolution
- Let the subtree pull stop; do not delete or regenerate the metadata.
- Resolve conflicts under
packages/<name>while preserving the local@xzzpig/pi-*npm name and thepi-*package contract (never adopt the upstream's own package name). Do not replace the subtree with copied files. - Complete the subtree merge, including the conflict resolutions, and verify its squash-parent trailers before creating any additional repository-owned fork or metadata commit.
- Record the exact resolved upstream commit:
git fetch "upstream-${name}" "$ref"
tip=$(git rev-parse FETCH_HEAD)
git merge-base --is-ancestor <resolved-commit> "$tip" # must succeed
Update the JSON record with the resolved commit and optional version. Keep the
record and any additional post-merge adaptations uncommitted during
verification. Run direnv reload and type-check the target package.
Split and push
Produce a subtree commit for review or contribution to upstream:
git subtree split --prefix="packages/${name}"
After reviewing the split output, push to a target branch:
git subtree push --prefix="packages/${name}" "upstream-${name}" main
Treat push as an explicit publishing action. The direnv helper never
invokes it.
Verification
For every upstream change, run verification while repository-owned follow-up changes are still uncommitted. A dirty worktree containing only the expected metadata, fork adaptation, version, and changelog files is valid at this stage:
git status --short --branch
# npm name must be the scoped fork name, e.g. @xzzpig/pi-tool-display
jq -e --arg n "@xzzpig/${name}" '.name == $n' "packages/${name}/package.json"
pnpm --filter "@xzzpig/${name}" run typecheck 2>/dev/null || true
pnpm exec prettier --check .
direnv reload
Then run the mandatory fork-divergence audit from the
pi-fork-divergence skill, which owns the
audit command and its output semantics (checked=… noise=… deleted=… declared=… undeclared=… stale=…). Treat an audit that prints nothing as a
failure, not a pass.
For a successful git subtree pull, verify its second parent before a later
commit moves HEAD. Uncommitted working-tree edits do not prevent this check:
git show -s --format='%P%n%B' HEAD
git show -s --format=%B HEAD^2 | grep -Fx "git-subtree-dir: packages/${name}"
git show -s --format=%B HEAD^2 | grep -Fx "git-subtree-split: ${commit}"
Use an isolated local upstream repository to test new workflow behavior. Cover first add, repeated direnv initialization, upstream updates, conflict resolution, ref change tracking, trailer preservation, and schema enforcement.
