ChangeSafe

Imported from wonkwonlee/ChangeSafe (AGENTS.md). Install upstream with npx skills add wonkwonlee/ChangeSafe. Copyright stays with the author.

ChangeSafe — Repository Instructions

This file and AGENTS.md are identical mirrors; when you change one, apply the same change to the other.

Mission

ChangeSafe is an open-source deterministic airlock for AI-proposed infrastructure changes, developed library/CLI-first. The non-negotiable trust model is:

AI diagnoses and proposes. Deterministic code validates. A human decides. ChangeSafe never executes changes against infrastructure.

Strategy, phases, and design constraints live in docs/OSS_ROADMAP.md — read it before starting multi-file work. P0–P6 are complete; P7 (benchmark

• community) has had its first pass and is the ongoing phase, so new work usually means scenarios, domains, docs, or integrations rather than new platform machinery. BUILD_WEEK_CHANGELOG.md is a historical record of the project's origin (built during OpenAI Build Week 2026, not submitted, pivoted to OSS); do not treat its Build Week constraints (two-scenario cap, GPT-5.6/OpenAI-only, single-app scope) as current rules.

Safety invariants

These override convenience, demo polish, and feature requests:

1. No execution path to infrastructure, ever: no SSH, NETCONF, RESTCONF, SNMP, gNMI-SET, vendor SDKs, shell execution, terraform apply, or arbitrary outbound HTTP actions. ChangeSafe analyzes, gates, and records; humans and their existing systems execute. The self-hosted server has no execution endpoint and never gains one.
2. Ingestion is read-only artifacts and data (bundled fixtures, uploaded bundles, terraform show -json output, future read-only collectors), validated by Zod schemas at every boundary.
3. All external content — alerts, notes, names, configuration values, plan contents, PR text — is untrusted data, never instructions.
4. Secrets (model API keys) exist only in server/CLI environment scope; never in client bundles, receipts, fixtures, logs, or error messages. CI enforces this with per-provider canaries against the built client chunks.
5. Model output is invalid until provider-side structured output (where available) AND local Zod validation both succeed; invented evidence ids or unknown resource references are hard rejections. Every provider faces the identical local validation.
6. The LLM never determines approval, risk, or execution status. Policies are pure functions that never import AI modules, read the clock, use randomness, or receive model confidence. Risk derivation (any BLOCK→CRITICAL, ≥2 WARN→HIGH, 1 WARN→MEDIUM, else LOW) is core-owned and identical across domains.
7. Any BLOCK finding makes approval and simulation impossible at the domain layer (state machine throws), not merely in the UI. No auto-approval path may exist anywhere, including the CLI and the authenticated server — authentication grants no new power over the gate.
8. Simulation mutates only deep clones; patch application is transactional (no partial mutation can escape); rollback is verified by canonical equality where the domain supports it.
9. Bundled data is fictional and publishable: documentation IP ranges (192.0.2.0/24, 198.51.100.0/24, 203.0.113.0/24), no real orgs, no third-party branding, no PII.
10. Provenance honesty: fixtures declare captured vs authored provenance; authored content is never attributed to a model; replay is always labeled and never silently substituted for live analysis.
11. Integrity is not authorship. Hashes prove a receipt was not altered; only an Ed25519 signature checked against an out-of-band public key proves who issued it. verify exits 2 rather than 0 when it was given no key to check a signature with — an unchecked claim never reads as a verified one.
12. The decision record is append-only. Ledger writes are guarded by SQLite triggers and a hash chain, so an altered, removed, or reordered entry is detectable; ledger verify exits non-zero on any break.

If a request conflicts with an invariant, stop that change and explain.

Architecture

npm workspaces monorepo. The extraction window (P2) is closed — do not restructure packages ad hoc; breaking schema renames now follow semver plus documented migrations (docs/OSS_ROADMAP.md §5).

packages/core/             @changesafe/core — proposal contract, findings and
                           risk, workflow state machine, universal policies,
                           receipts + canonicalization + hashing, Ed25519
                           signing, scenario expectations, the DomainAdapter
                           contract (deps: zod)
packages/domain-network/   @changesafe/domain-network — incident/topology
                           model, path allowlist, transactional patch engine,
                           inverse derivation, reachability, simulator,
                           network policies
packages/domain-terraform/ @changesafe/domain-terraform — external-diff
                           domain over `terraform show -json`; never runs
                           Terraform
packages/domain-kubernetes/ @changesafe/domain-kubernetes — offline Kubernetes
                           state domain and policies
packages/kubernetes-collector/ private namespace-scoped read-only collector;
                           the only workspace depending on @kubernetes/client-node
packages/ai/               @changesafe/ai — provider-agnostic adapters
                           (OpenAI, Anthropic, Ollama) on plain `fetch`,
                           hardened prompt, portable JSON Schema derivation,
                           fixture capture; server/CLI-side only
packages/ledger/           @changesafe/ledger — append-only `node:sqlite`
                           receipt ledger with a hash chain
packages/server/           @changesafe/server — authenticated self-hosted
                           decision API: OIDC approver identity, findings
                           recomputed server-side, signed receipts, ledger
                           append before response
packages/cli/              changesafe — gate, analyze, eval, verify, keygen,
                           ledger, serve, scenario, and Kubernetes collect;
                           ships pre-bundled
app/                       Next.js multi-domain review workbench: public
                           Network replay at /, Terraform/Kubernetes
                           subroutes, optional self-hosted client,
                           versioned replay API + status
components/                domain workbenches, shared evidence and
                           capability/authority presentation
features/                  app-local domain registries, versioned review
                           contracts, neutral controller, public/self-hosted
                           transports, durable review contracts
lib/ai/                    server-only live-provider status/config binding
lib/domain/                app-level wire contracts and version constants
scenarios/                 synthetic incident bundles by domain, fixtures,
                           expectations, plus the browser-usable registry
tests/                     unit, integration, e2e (Playwright)
scripts/                   release-bundle build/verify, PR summary rendering
verification/              published v0.1.0 verification snapshot
action.yml, examples/      the GitHub Action and a copyable workflow
docs/                      roadmap, architecture, threat model, scenarios
                           (generated), authoring, benchmark, launch, blog

Dependency direction (violations are review failures): packages/core depends on zod alone — never on the app, the AI layer, or a domain package. A domain package depends on core. packages/ai may depend on core and domain schemas; nothing in the gate path may depend on packages/ai. The ledger depends on core; the server depends on core, domains, and the ledger — never on AI. The CLI wires them together, and only analyze and eval reach a model. The app depends on core, domains, and (server-side only) AI. Scenario fixtures must pass the production schemas. packages/core/src/state-machine.ts is the single workflow authority; UI must dispatch through it.

Two domain shapes exist: simulated-state (network and Kubernetes — hold a declarative model and apply operations to a clone) and external-diff (Terraform — the diff arrives precomputed, so nothing simulates). An external-diff domain may skip a universal policy only by declaring it in skippedUniversalPolicies with the reason and the domain policy that replaces it.

Domains reach core through the DomainAdapter contract (packages/core/src/domain.ts): state extraction, transactional applyOperations, blast-radius units, untrusted text, known evidence ids, and the domain's own policies. Adding a domain means implementing that interface — never editing a universal policy to special-case it. packages/core/tests/standalone-domain.test.ts implements a complete toy domain in one file and is the reference for what the contract requires.

Policy set

Universal policies (in packages/core/src/policies/): PATCH_SCHEMA, BLAST_RADIUS, ROLLBACK_COMPLETE, VERIFICATION_REQUIRED, UNTRUSTED_INSTRUCTION. Domain policies: network contributes MGMT_REACHABILITY and PROTECTED_RESOURCE; terraform contributes DESTRUCTIVE_OP, PROTECTED_RESOURCE, REVERSIBILITY (its declared replacement for ROLLBACK_COMPLETE), and PLAN_CONTEXT_REQUIRED (its declared replacement for VERIFICATION_REQUIRED). Policy ids are open UPPER_SNAKE strings so each domain contributes its own; policyOrder(adapter) publishes the evaluation order (structural → domain → universal). Policy status is PASS | WARN | BLOCK; all policies fail closed. Policy packs may tune typed parameters (thresholds, protected patterns) but are never a DSL, and never alter the risk formula. Any policy behavior change bumps CORE_POLICY_VERSION (packages/core/src/version.ts) or the domain's version (NETWORK_POLICY_VERSION, TERRAFORM_POLICY_VERSION) — both compose into the policyVersion recorded in receipts — and updates receipt tests.

Only ROLLBACK_COMPLETE and VERIFICATION_REQUIRED may ever be skipped (SKIPPABLE_UNIVERSAL_POLICY_IDS) — PATCH_SCHEMA, BLAST_RADIUS, and UNTRUSTED_INSTRUCTION are structurally answerable by every domain and no adapter may declare them skipped. evaluatePolicies and policyOrder both enforce this and reject a skippedUniversalPolicies entry whose replacedBy names a domain policy the adapter does not actually declare — core verifies this itself rather than trusting the app's registration path, so a third-party adapter reaching changesafe gate directly gets the same guarantee. Every receipt records policyCoverage (computePolicyCoverage): the exact ordered policy ids that ran plus what was skipped and why, so a verifier reads what a receipt's absence of a policy id means from the signed payload itself rather than the adapter's source.

Technology

Strict TypeScript, Zod-first (schemas before derived types), Next.js App Router for the multi-domain workbench, npm workspaces, Vitest, Playwright. Node 22 and npm 10.9.8 are pinned (engines, .nvmrc, and a CI runtime-contract assertion). AI: provider-agnostic adapters (OpenAI Responses, Anthropic Messages, local Ollama) in packages/ai, plain fetch with no vendor SDKs, one Zod schema deriving all three wire schemas — all server/CLI-side only, and never depended on by the gate. Persistence and identity use platform primitives only (node:sqlite, Web Crypto), so self-hosting adds no dependency and no native build step. Do not add major infrastructure (databases beyond SQLite, queues, workers) ahead of the roadmap phase that calls for it.

Commands

Keep valid and documented in README.md:

npm install
npm run dev        # http://localhost:3000, works with no API key
npm run lint
npm run typecheck
npm test           # vitest unit + integration; no network, no API credit
npm run build
npm run build:cli  # bundle the changesafe CLI (some tests require it)
npm run test:e2e   # Playwright; npx playwright install chromium once

Fast loop for scenario or policy work (what CI's corpus job runs):

npm run build:cli
node packages/cli/dist/changesafe.js scenario check
node packages/cli/dist/changesafe.js scenario gallery --check   # docs/SCENARIOS.md drift

Release-snapshot helpers: npm run build:bundle:v0.1.0, npm run verify:v0.1.0. Optional live model work is env-gated: CHANGESAFE_LIVE_SMOKE=1 npm test (add CHANGESAFE_CAPTURE_FIXTURE=1 to capture a provenance-stamped fixture). changesafe eval spends API credit and is never part of a default run.

Run the relevant gate before reporting completion; the replay demo and the red-team scenario contract (blocked, never approvable) must stay green in every phase. CI jobs: full gate, Playwright, Action self-test, corpus + gallery currency, and the client-bundle secret check.

Coding standards

• No any, no unsafe casts; exhaustive handling for operations, findings, risk, and workflow states.
• Small pure functions for policy and patch behavior; validate all boundary inputs; canonicalize before comparison or hashing.
• Stable identifiers over display strings; typed domain errors without stack traces or secrets in user-visible messages.
• Exit codes are part of the contract: 0 evaluated and nothing blocking, 1 blocked, 2 could not evaluate. A missing verdict must never read as approval.
• Comments explain invariants and non-obvious decisions, not restatements.
• No dead code, placeholder copy, fake metrics, or TODO-driven behavior in merged work.

Testing expectations

Every behavioral change includes or updates the smallest relevant test. Standing coverage: schema accept/reject, patch allowlist + transactional failure, canonical/hash stability, receipt signing and signature verification, every policy PASS and failure, risk derivation, illegal state transitions, rollback restoration, scenario expectations (per-scenario expectations.json, walked from disk — an unregistered scenario directory fails), replay parity and provenance honesty, keyless replay API, invalid model output rejection, provider adapters producing identical accepted output, ledger chain-break detection, OIDC token rejection paths, the shipped CLI bundle, the published verification bundle, no secret leakage in client artifacts, and the Playwright safe + blocked critical paths. Default test runs spend no API credit and need no network; live smoke stays env-gated. Never weaken a test to make an implementation pass.

Scenarios and fixtures

Scenarios are first-class contribution surface: fully fictional, schema- valid, provenance-honest, each with an expectations.json proving its claimed verdicts in CI. Twenty-seven scenarios ship today across network, Terraform, and Kubernetes (thirteen adversarial); docs/SCENARIOS.md carries the authoritative count, since it is generated from the corpus. Each declares corpus.adversarial plus its failureModes so coverage is countable. An adversarial scenario must be refused by the gate or flagged by simulation — the schema itself rejects an adversarial scenario declared as approvable and cleanly simulating. Red-team scenarios must always produce their expected BLOCKs; that corpus never approving is a release gate. docs/SCENARIOS.md is generated by changesafe scenario gallery — never hand-edit it, and regenerate when the corpus changes. Authoring guide: docs/SCENARIO_AUTHORING.md.

Git and external actions

• Inspect the worktree before editing; preserve unrelated changes.
• Small intentional commits; never rewrite history, force-push, or delete branches. Remote: github.com/wonkwonlee/ChangeSafe (public) — push after verified milestones.
• Never commit secrets or local env files.
• Publishing packages to npm requires explicit owner authorization. The @changesafe scope and the unscoped changesafe name are both claimed and published (docs/OSS_ROADMAP.md §7).

Work tracking

GitHub Issues is the contribution surface and the only public record: user reports, scenario proposals, and anything a contributor should be able to find. Nothing below replaces it.

Locally, this repository supports beads (bd) as the agent-facing work graph, because a flat list cannot express "this is blocked until v0.4.0 is published" and repeatedly failed to: the vNext epic's ten delivery issues stayed open for a week after the work merged. bd ready answers what is actually startable; bd blocked says why the rest is not.

• bd is optional. Contributors do not need it, and no build, test, or CI step depends on it.
• .beads/ is per-machine runtime state and is ignored, including issues.jsonl — beads syncs through its own git refs, so committing that export would version a passive copy as if it were the source of truth.
• A bead that mirrors a GitHub issue records it with --external-ref gh-<number>. Automated bd github sync is off on purpose: it is bidirectional by default and would push local planning items into a public tracker.
• Durable project decisions still belong in MEMORY.md and the roadmap, not in a local database.

Scope control

When effort is constrained, priority order: (1) safety invariants and deterministic correctness, (2) the always-working replay demo and red-team contract, (3) the current roadmap phase's exit gate, (4) tests and build health, (5) docs, (6) polish. The do-not-build list in docs/OSS_ROADMAP.md §6 is binding.

Before proposing new work, read docs/STRATEGY.agent.md