roci

Imported from jakobmoellerdev/roci (AGENTS.md). Install upstream with npx skills add jakobmoellerdev/roci. Copyright stays with the author.

AGENTS.md

Guidance for AI coding agents working in this repository.

Project

roci — a Rust implementation of the OCI Distribution Specification (an OCI registry). See README.md for goals and the feature roadmap.

Design docs (read before non-trivial work)

• ARCHITECTURE.md — component/crate model, build flavors (minimal/full), storage subsystem, background scheduler, scaling (vertical + horizontal scale-out), architectural invariants.
• SECURITY.md — build/runtime hardening, authn/authz matrix, content trust, security invariants.
• PLAN.md — phased master build plan; each phase gated on the OCI conformance suite.
• RESEARCH.md — peer-reviewed + industry evidence (CAS/dedup/GC, consistent hashing, lazy-pull, P2P, index structures, telemetry overhead) with a Sources table; cite it when justifying a storage or scale-out decision.

These are reverse-engineered from and cite zot's design as prior art: Architecture, Storage, Security Posture, Scale-out. When a design question arises, consult these docs first; they mark [roci divergence] where roci intentionally differs from zot. Respect the stated invariants — do not violate them without updating the design doc and flagging it.

Keeping docs consistent (maintenance contract)

Each doc has one owner; when your change touches its concern, update it in the same change (see PLAN.md §Document maintenance contract for the full table):

• Design decisions or corrections → consolidate into ARCHITECTURE.md (and SECURITY.md for security). Do not scatter design rationale into PLAN or README; mark evidence-backed refinements [refined from RESEARCH …].
• Freshly gathered research → consolidate into RESEARCH.md with a Sources row, then cite it by key from the doc that acts on it. Never inline a new source table elsewhere.
• PLAN.md (build steps) and the README.md high-level feature roadmap MUST be kept updated whenever a decision adds/changes a build step or a user-facing capability — a design/security decision is not "done" until PLAN and the README reflect it.
• Respect the stated invariants; changing one requires updating its owning doc and flagging the change.
• docs/ (the VitePress site) MUST be kept in sync with any user-facing or design change: the Roadmap page mirrors the README roadmap, and the design overviews under docs/design/ link the canonical root docs. See § Documentation site (docs/) below.

Documentation site (docs/)

The public documentation site is a VitePress app in docs/, deployed to GitHub Pages by .github/workflows/docs.yml on every push to main (pull requests build-only, validating the site and its internal links).

• Work in docs/. npm install once, then npm run docs:dev for a hot-reloading preview and npm run docs:build to reproduce the CI build (VitePress fails the build on dead internal links, so a green build means links resolve). Commit the docs/package-lock.json when dependencies change — the workflow uses npm ci.
• Design pages don't duplicate. The overviews under docs/design/ (architecture, security, storage, research) are summaries that link the canonical root docs (ARCHITECTURE.md, SECURITY.md, RESEARCH.md), which remain the source of truth. The build plan has no site overview — PLAN.md is the single source and is linked directly (e.g. from docs/guide/configuration.md). When you change a canonical doc, update the matching overview in the same change so the site does not drift; never fork design rationale into the site.
• Roadmap mirrors the README. docs/roadmap.md mirrors the README feature roadmap. A capability whose status changes MUST be updated in both README.md and docs/roadmap.md in one change (this extends the maintenance contract above).
• Guide pages track behavior. When a user-facing capability changes (CLI flags, config surface, container usage, local-dev tasks), update the relevant page under docs/guide/ alongside the code and the README.
• Branding is not re-authored here. Theming maps the assets/BRAND.md palette onto VitePress variables in docs/.vitepress/theme/brand.css; the logo/favicon/social-card in docs/public/ are copies of the source-of-truth SVGs in assets/. When the brand assets change, re-copy them (cp assets/{logo.svg,logo-dark.svg,favicon.svg,icon.svg,social-card.svg} docs/public/) — never recolor gradients or hand-edit the copies.
• Workflow hygiene. Keep docs.yml actionlint- and zizmor-clean like every other workflow: pin actions by commit SHA, persist-credentials: false on checkout, default-deny permissions with only the deploy job holding pages: write + id-token: write.

OCI specs (local reference)

The authoritative OCI specs are vendored locally as git submodules, each pinned to v1.1.1. Read them from the local checkout — do not fetch them from the web:

spec/distribution-spec/spec.md    # OCI Distribution Spec (registry API)
spec/image-spec/spec.md           # OCI Image Spec
spec/image-spec/image-layout.md   # OCI Image Layout (on-disk storage format)
spec/docker-registry-api-v2.md    # Docker Registry HTTP API V2 (de-facto companion: bearer-token auth, worked pull/push/delete examples)

The full submodules live at spec/distribution-spec/ and spec/image-spec/.

If a file is missing or empty (fresh clone), populate the submodules first:

git submodule update --init --depth 1

spec/docker-registry-api-v2.md is a vendored snapshot of Docker Hub's Registry API V2 reference (source URL and fetch date in its header). It is authoritative only for the de-facto Docker bearer-token auth flow and worked client examples; for the protocol itself the OCI specs above win. Refresh it by re-fetching the source URL, not by hand-editing.

When implementing or verifying any registry endpoint, error code, media type, or workflow, cite and follow these specs as the source of truth. The submodules are pinned; do not bump their commits without an explicit instruction.

CI & local development

• Before pushing / opening a PR, run just ci. It runs the required CI checks locally — actionlint (workflow lint), fmt, clippy (both flavors — the sole gate compiling the minimal flavor), test+coverage (one instrumented build), build (a dev-convenience compile of both flavors; the minimal-flavor CI compile is the clippy job, the full flavor is the coverage build), deps-guard, and conformance (the OCI distribution suite). Green just ci ⇒ green required CI. See README.md "Developing locally" for the full recipe table.
• Line coverage is enforced at a 95% floor. ci.yml's test + coverage job compiles the workspace once (instrumented) and runs the tests under cargo llvm-cov nextest, then scripts/coverage.sh asserts line coverage stays at or above 95% (lcov), excluding the thin roci-cli/src/main.rs entrypoint (its logic lives in the fully-covered library). It gates on the lcov line metric — not --fail-under-lines, whose region-derived metric penalizes async .await state-machine arms that always execute. Uncovered lines are always listed at gate time for triage (the handful that remain are unreachable-in-CI defensive syscall-error arms in the beneath-root storage path); a PR dropping below the floor fails. Inspect region gaps with just coverage-report. The job also emits cobertura.xml and reports it to GitHub via actions/upload-code-coverage (needs code-quality: write), feeding GitHub's PR coverage gate ("Restrict code coverage").
• Coverage is measured on Linux; macOS is NOT authoritative. cargo llvm-cov's line attribution differs between platforms: macOS collapses many match/if let error arms and let-else { … continue } skip arms and closures as covered that Linux CI counts as uncovered. So a local just coverage printing 100.00% on darwin can still fail the Linux CI gate (observed: local 100% / 5101 lines vs CI 99.18% / 4152 lines — CI even instruments a different line count). Before pushing coverage-sensitive work, reproduce the Linux number — the authoritative gate is Linux. Bind-mounting into Docker Desktop is often blocked by file-sharing; instead run the gate in-image with a throwaway Dockerfile that COPYs a git archive HEAD of the tree, cargo install cargo-nextest cargo-llvm-cov (do not download the get.nexte.st/latest/linux tarball — it is x86_64 and fails 255 on an arm64 host/image), then runs cargo llvm-cov --no-report nextest --workspace --all-features + cargo llvm-cov report --lcov. Or read the failed CI job's uploaded coverage artifact (gh run download <run-id> -n coverage) — its lcov.info lists the exact DA:<line>,0 lines Linux flags. Fix the gap deterministically: add a test that unconditionally executes each flagged branch (foreign/corrupt/unreadable inputs for let-else continue skips; a metadata-store miss to hit an index.json fallback), and for a genuinely unreachable defensive arm (e.g. a seek/rewind/metadata error on an already-opened regular file) remove the arm rather than write an untestable test — never rely on macOS collapsing it.
• Security & static analysis: three gates run in CI — actionlint (a job in ci.yml, lints workflow YAML + shellchecks run: blocks), zizmor.yml (GitHub Actions security auditor, SARIF to code scanning), and codeql.yml (CodeQL SAST for Rust, build-mode: none, tuned for speed via .github/codeql/config.yml: scoped to crates/, default query suite, threads: 0). CodeQL runs on every push to main and every PR (no path filter): the code_scanning repo ruleset requires a CodeQL result to merge, and a path-filtered run is skipped on docs-only PRs — leaving the required check waiting forever — so it must always run to keep the gate satisfiable. Rust supports only build-mode: none (CodeQL extracts from source, never compiling), so there is no cargo artifact to reuse; instead init enables dependency-caching: true (caches the Cargo dependency resolution CodeQL models) and trap-caching: true (caches per-source TRAP extractor output so unchanged files skip re-extraction), both action-managed via the Actions cache (base branch populates, PR/fork runs restore read-only). Do not hand-drive overlay/incremental analysis with CODEQL_OVERLAY_DATABASE_MODE + a self-managed actions/cache base database: that plumbing is unsupported alongside the codeql-action, produced overlay-base databases missing base-database-oids.json, and failed init on every cache hit. Overlay is left to the action, which enables it automatically when the CLI/server support it. Run just lint-workflows and just zizmor locally; CodeQL runs only on GitHub. Keep workflows actionlint- and zizmor-clean (pin third-party actions by commit SHA, persist-credentials: false on checkout, least-privilege permissions).
• CI is fork-safe and uses a read-only token. ci.yml, audit.yml, and conformance.yml default to permissions: contents: read and use no secrets, so autonomous-agent PRs (including from forks) run the full gate without a write token. The test + coverage job additionally grants code-quality: write + pull-requests: read to report coverage — fork PRs lack this and the upload step skips gracefully. zizmor.yml/codeql.yml need security-events: write for SARIF but execute no PR code. Never add a step needing a write token or secret to the build/test path. Privileged merge automation belongs in auto-merge.yml (base-repo context, no PR code).
• Agentic PRs: apply the automerge label to request auto-merge. Auto-merge still requires green CI and a code-owner approval (SECURITY.md policy) — the label does not bypass review.
• Toolchain is pinned in rust-toolchain.toml. Do not hardcode a different toolchain in a workflow; bump the pin (and the matching dtolnay/rust-toolchain@<version> refs in .github/actions/setup-rust/action.yml and ci.yml) in one change.
• Build-flavor invariants (ARCHITECTURE.md 1 & 2): every new extension crate MUST be named roci-ext-* (or roci-cluster) and be reachable from roci-cli only behind a cargo feature, never as an unconditional dependency. just deps-guard / the minimal-deps-guard CI job enforces this.
• Every crate carries #![forbid(unsafe_code)] unless it is an audited zero-copy module explicitly exempted (SECURITY.md invariant 7).
• Conformance: just conformance builds the Go conformance binary from the pinned spec/distribution-spec submodule and runs it against a locally-started roci; run just init first if the submodules are absent.
• Pre-commit hook: run just hooks once to install .git/hooks/pre-commit (or pre-commit install with the framework via .pre-commit-config.yaml). When Rust sources are staged it runs fmt, clippy (both flavors), the 100% coverage gate, and the full OCI conformance suite (scripts/conformance.sh, shared with just conformance — requires Go 1.17+ and the pinned submodule); it also runs workflow lint/security when workflows are staged, and regenerates COVERAGE.md + the README badge, re-staging them so the coverage report stays in sync with each commit. A commit is blocked unless coverage and conformance pass.
• Container image (Containerfile): hardened static musl binary on scratch, nonroot UID, read-only root FS with only the storage volume writable. The container CI workflow builds and smoke-tests on native per-arch runners (no QEMU): PRs build linux/arm64 only (on ubuntu-24.04-arm) to save time; main and v* tags build linux/amd64 + linux/arm64, merge a multi-arch manifest, and push to GHCR (main → :main; a release tag → :X.Y.Z, :X.Y, :latest; both → :<sha>) with a signed build-provenance attestation (actions/attest-build-provenance) + embedded SBOM/SLSA provenance. It also builds+attests a standalone static binary per Linux arch. Test locally with just container. Container images are Linux-only; darwin ships as release binaries (release.yml, native macOS runners, draft GitHub Release on v* tags — publish the draft after verifying assets), not images. When changing Containerfile, keep it scratch-based and nonroot — do not add a shell or run as root.