Imported from jakobmoellerdev/roci (
AGENTS.md). Install upstream withnpx skills add jakobmoellerdev/roci. Copyright stays with the author.
AGENTS.md
Guidance for AI coding agents working in this repository.
Project
roci — a Rust implementation of the OCI Distribution Specification (an OCI registry). See README.md for goals and the feature roadmap.
Design docs (read before non-trivial work)
ARCHITECTURE.md— component/crate model, build flavors (minimal/full), storage subsystem, background scheduler, scaling (vertical + horizontal scale-out), architectural invariants.SECURITY.md— build/runtime hardening, authn/authz matrix, content trust, security invariants.PLAN.md— phased master build plan; each phase gated on the OCI conformance suite.RESEARCH.md— peer-reviewed + industry evidence (CAS/dedup/GC, consistent hashing, lazy-pull, P2P, index structures, telemetry overhead) with a Sources table; cite it when justifying a storage or scale-out decision.
These are reverse-engineered from and cite zot's design as prior art: Architecture, Storage, Security Posture, Scale-out. When a design question arises, consult these docs first; they mark [roci divergence] where roci intentionally differs from zot. Respect the stated invariants — do not violate them without updating the design doc and flagging it.
Keeping docs consistent (maintenance contract)
Each doc has one owner; when your change touches its concern, update it in the same change (see PLAN.md §Document maintenance contract for the full table):
- Design decisions or corrections → consolidate into
ARCHITECTURE.md(andSECURITY.mdfor security). Do not scatter design rationale into PLAN or README; mark evidence-backed refinements[refined from RESEARCH …]. - Freshly gathered research → consolidate into
RESEARCH.mdwith a Sources row, then cite it by key from the doc that acts on it. Never inline a new source table elsewhere. PLAN.md(build steps) and theREADME.mdhigh-level feature roadmap MUST be kept updated whenever a decision adds/changes a build step or a user-facing capability — a design/security decision is not "done" until PLAN and the README reflect it.- Respect the stated invariants; changing one requires updating its owning doc and flagging the change.
docs/(the VitePress site) MUST be kept in sync with any user-facing or design change: the Roadmap page mirrors the README roadmap, and the design overviews underdocs/design/link the canonical root docs. See § Documentation site (docs/) below.
Documentation site (docs/)
The public documentation site is a VitePress app in docs/, deployed to GitHub Pages by .github/workflows/docs.yml on every push to main (pull requests build-only, validating the site and its internal links).
- Work in
docs/.npm installonce, thennpm run docs:devfor a hot-reloading preview andnpm run docs:buildto reproduce the CI build (VitePress fails the build on dead internal links, so a green build means links resolve). Commit thedocs/package-lock.jsonwhen dependencies change — the workflow usesnpm ci. - Design pages don't duplicate. The overviews under
docs/design/(architecture,security,storage,research) are summaries that link the canonical root docs (ARCHITECTURE.md,SECURITY.md,RESEARCH.md), which remain the source of truth. The build plan has no site overview —PLAN.mdis the single source and is linked directly (e.g. fromdocs/guide/configuration.md). When you change a canonical doc, update the matching overview in the same change so the site does not drift; never fork design rationale into the site. - Roadmap mirrors the README.
docs/roadmap.mdmirrors the README feature roadmap. A capability whose status changes MUST be updated in bothREADME.mdanddocs/roadmap.mdin one change (this extends the maintenance contract above). - Guide pages track behavior. When a user-facing capability changes (CLI flags, config surface, container usage, local-dev tasks), update the relevant page under
docs/guide/alongside the code and the README. - Branding is not re-authored here. Theming maps the
assets/BRAND.mdpalette onto VitePress variables indocs/.vitepress/theme/brand.css; the logo/favicon/social-card indocs/public/are copies of the source-of-truth SVGs inassets/. When the brand assets change, re-copy them (cp assets/{logo.svg,logo-dark.svg,favicon.svg,icon.svg,social-card.svg} docs/public/) — never recolor gradients or hand-edit the copies. - Workflow hygiene. Keep
docs.ymlactionlint- andzizmor-clean like every other workflow: pin actions by commit SHA,persist-credentials: falseon checkout, default-denypermissionswith only the deploy job holdingpages: write+id-token: write.
OCI specs (local reference)
The authoritative OCI specs are vendored locally as git submodules, each pinned to v1.1.1. Read them from the local checkout — do not fetch them from the web:
spec/distribution-spec/spec.md # OCI Distribution Spec (registry API)
spec/image-spec/spec.md # OCI Image Spec
spec/image-spec/image-layout.md # OCI Image Layout (on-disk storage format)
spec/docker-registry-api-v2.md # Docker Registry HTTP API V2 (de-facto companion: bearer-token auth, worked pull/push/delete examples)
The full submodules live at spec/distribution-spec/ and spec/image-spec/.
If a file is missing or empty (fresh clone), populate the submodules first:
git submodule update --init --depth 1
spec/docker-registry-api-v2.md is a vendored snapshot of Docker Hub's Registry API V2 reference (source URL and fetch date in its header). It is authoritative only for the de-facto Docker bearer-token auth flow and worked client examples; for the protocol itself the OCI specs above win. Refresh it by re-fetching the source URL, not by hand-editing.
When implementing or verifying any registry endpoint, error code, media type, or workflow, cite and follow these specs as the source of truth. The submodules are pinned; do not bump their commits without an explicit instruction.
CI & local development
- Before pushing / opening a PR, run
just ci. It runs the required CI checks locally —actionlint(workflow lint),fmt,clippy(both flavors — the sole gate compiling the minimal flavor),test+coverage(one instrumented build),build(a dev-convenience compile of both flavors; the minimal-flavor CI compile is the clippy job, the full flavor is the coverage build),deps-guard, andconformance(the OCI distribution suite). Greenjust ci⇒ green required CI. SeeREADME.md"Developing locally" for the full recipe table. - Line coverage is enforced at a 95% floor.
ci.yml'stest + coveragejob compiles the workspace once (instrumented) and runs the tests undercargo llvm-cov nextest, thenscripts/coverage.shasserts line coverage stays at or above 95% (lcov), excluding the thinroci-cli/src/main.rsentrypoint (its logic lives in the fully-covered library). It gates on the lcov line metric — not--fail-under-lines, whose region-derived metric penalizes async.awaitstate-machine arms that always execute. Uncovered lines are always listed at gate time for triage (the handful that remain are unreachable-in-CI defensive syscall-error arms in the beneath-root storage path); a PR dropping below the floor fails. Inspect region gaps withjust coverage-report. The job also emitscobertura.xmland reports it to GitHub viaactions/upload-code-coverage(needscode-quality: write), feeding GitHub's PR coverage gate ("Restrict code coverage"). - Coverage is measured on Linux; macOS is NOT authoritative.
cargo llvm-cov's line attribution differs between platforms: macOS collapses manymatch/if leterror arms andlet-else { … continue }skip arms and closures as covered that Linux CI counts as uncovered. So a localjust coverageprinting100.00%on darwin can still fail the Linux CI gate (observed: local 100% / 5101 lines vs CI 99.18% / 4152 lines — CI even instruments a different line count). Before pushing coverage-sensitive work, reproduce the Linux number — the authoritative gate is Linux. Bind-mounting into Docker Desktop is often blocked by file-sharing; instead run the gate in-image with a throwawayDockerfilethatCOPYs agit archive HEADof the tree,cargo install cargo-nextest cargo-llvm-cov(do not download theget.nexte.st/latest/linuxtarball — it is x86_64 and fails 255 on an arm64 host/image), then runscargo llvm-cov --no-report nextest --workspace --all-features+cargo llvm-cov report --lcov. Or read the failed CI job's uploadedcoverageartifact (gh run download <run-id> -n coverage) — itslcov.infolists the exactDA:<line>,0lines Linux flags. Fix the gap deterministically: add a test that unconditionally executes each flagged branch (foreign/corrupt/unreadable inputs forlet-else continueskips; a metadata-store miss to hit anindex.jsonfallback), and for a genuinely unreachable defensive arm (e.g. aseek/rewind/metadataerror on an already-opened regular file) remove the arm rather than write an untestable test — never rely on macOS collapsing it. - Security & static analysis: three gates run in CI —
actionlint(a job inci.yml, lints workflow YAML + shellchecksrun:blocks),zizmor.yml(GitHub Actions security auditor, SARIF to code scanning), andcodeql.yml(CodeQL SAST for Rust,build-mode: none, tuned for speed via.github/codeql/config.yml: scoped tocrates/, default query suite,threads: 0). CodeQL runs on every push tomainand every PR (no path filter): thecode_scanningrepo ruleset requires a CodeQL result to merge, and a path-filtered run is skipped on docs-only PRs — leaving the required check waiting forever — so it must always run to keep the gate satisfiable. Rust supports onlybuild-mode: none(CodeQL extracts from source, never compiling), so there is no cargo artifact to reuse; insteadinitenablesdependency-caching: true(caches the Cargo dependency resolution CodeQL models) andtrap-caching: true(caches per-source TRAP extractor output so unchanged files skip re-extraction), both action-managed via the Actions cache (base branch populates, PR/fork runs restore read-only). Do not hand-drive overlay/incremental analysis withCODEQL_OVERLAY_DATABASE_MODE+ a self-managedactions/cachebase database: that plumbing is unsupported alongside thecodeql-action, produced overlay-base databases missingbase-database-oids.json, and failediniton every cache hit. Overlay is left to the action, which enables it automatically when the CLI/server support it. Runjust lint-workflowsandjust zizmorlocally; CodeQL runs only on GitHub. Keep workflowsactionlint- andzizmor-clean (pin third-party actions by commit SHA,persist-credentials: falseon checkout, least-privilege permissions). - CI is fork-safe and uses a read-only token.
ci.yml,audit.yml, andconformance.ymldefault topermissions: contents: readand use no secrets, so autonomous-agent PRs (including from forks) run the full gate without a write token. Thetest + coveragejob additionally grantscode-quality: write+pull-requests: readto report coverage — fork PRs lack this and the upload step skips gracefully.zizmor.yml/codeql.ymlneedsecurity-events: writefor SARIF but execute no PR code. Never add a step needing a write token or secret to the build/test path. Privileged merge automation belongs inauto-merge.yml(base-repo context, no PR code). - Agentic PRs: apply the
automergelabel to request auto-merge. Auto-merge still requires green CI and a code-owner approval (SECURITY.mdpolicy) — the label does not bypass review. - Toolchain is pinned in
rust-toolchain.toml. Do not hardcode a different toolchain in a workflow; bump the pin (and the matchingdtolnay/rust-toolchain@<version>refs in.github/actions/setup-rust/action.ymlandci.yml) in one change. - Build-flavor invariants (
ARCHITECTURE.md1 & 2): every new extension crate MUST be namedroci-ext-*(orroci-cluster) and be reachable fromroci-clionly behind a cargo feature, never as an unconditional dependency.just deps-guard/ theminimal-deps-guardCI job enforces this. - Every crate carries
#![forbid(unsafe_code)]unless it is an audited zero-copy module explicitly exempted (SECURITY.mdinvariant 7). - Conformance:
just conformancebuilds the Go conformance binary from the pinnedspec/distribution-specsubmodule and runs it against a locally-started roci; runjust initfirst if the submodules are absent. - Pre-commit hook: run
just hooksonce to install.git/hooks/pre-commit(orpre-commit installwith the framework via.pre-commit-config.yaml). When Rust sources are staged it runs fmt, clippy (both flavors), the 100% coverage gate, and the full OCI conformance suite (scripts/conformance.sh, shared withjust conformance— requires Go 1.17+ and the pinned submodule); it also runs workflow lint/security when workflows are staged, and regeneratesCOVERAGE.md+ the README badge, re-staging them so the coverage report stays in sync with each commit. A commit is blocked unless coverage and conformance pass. - Container image (
Containerfile): hardened static musl binary onscratch, nonroot UID, read-only root FS with only the storage volume writable. ThecontainerCI workflow builds and smoke-tests on native per-arch runners (no QEMU): PRs buildlinux/arm64only (onubuntu-24.04-arm) to save time;mainandv*tags buildlinux/amd64+linux/arm64, merge a multi-arch manifest, and push to GHCR (main→:main; a release tag →:X.Y.Z,:X.Y,:latest; both →:<sha>) with a signed build-provenance attestation (actions/attest-build-provenance) + embedded SBOM/SLSA provenance. It also builds+attests a standalone static binary per Linux arch. Test locally withjust container. Container images are Linux-only; darwin ships as release binaries (release.yml, native macOS runners, draft GitHub Release onv*tags — publish the draft after verifying assets), not images. When changingContainerfile, keep it scratch-based and nonroot — do not add a shell or run as root.
