Imported from xberg-io/xberg (
.ai-rulez/skills/test-corpus/SKILL.md). Install upstream withnpx skills add xberg-io/xberg --skill test-corpus. Copyright stays with the author.
Test corpus
test_documents is a git submodule (.gitmodules → xberg-io/test_documents). Its binary
fixtures — 654 objects, ~581 MiB per test_documents/corpus.lock.json — are not committed.
They are materialised from a public GCS bucket over anonymous HTTPS, with no credentials and no
SDK.
python3 test_documents/scripts/fetch_corpus.py # everything
python3 test_documents/scripts/fetch_corpus.py --include 'pdf/**'Fetched files are gitignored inside the submodule, so neither the submodule nor the
superproject goes dirty. CI does the same thing with xberg-io/actions/fetch-test-documents@v1,
narrowing with include: where the job's fixture surface is known (see ci-rust.yaml).
Without the fetch
Tests that read from test_documents fail with missing-file errors, not assertion failures.
Read the message before concluding the suite regressed.
Never include_bytes! a corpus fixture
Baking corpus bytes in at compile time turns a missing fixture into a build failure for the whole crate — this is what broke the workspace-wide clippy run in CI. Use the canonical helper instead:
// crates/xberg/src/utils/mod.rs — #[cfg(test)], pub(crate)
let Some(bytes) = crate::utils::read_test_fixture("images/test.heic") else { return; };read_test_fixture prints a greppable SKIP: fixture … not available line naming the missing
path and returns None. It lives in utils deliberately: utils compiles unconditionally,
while extraction::image and extraction::email are feature-gated.
A None is "not run", never "passed".
A git worktree is not a valid control
git worktree add does not populate submodules, so a control worktree has an empty
test_documents. Every fixture-guarded test skips silently and the run reports green — a
vacuous control inverts the verdict. The tell is the runtime: finished in 0.00s means the
binary did nothing.
Two more worktree traps: a relative [patch.crates-io] path dependency (the root
Cargo.toml currently patches liter-llm to ../liter-llm/…) resolves relative to the
worktree, not the main checkout; and a shared CARGO_TARGET_DIR will thrash.
Working setup:
git worktree add /tmp/ctl <ref> --detach
rmdir /tmp/ctl/test_documents && ln -sfn <main-tree>/test_documents /tmp/ctl/test_documents
CARGO_TARGET_DIR=/tmp/ctl-target cargo test ...When the goal is only build isolation, prefer a dedicated CARGO_TARGET_DIR in the main
checkout over a worktree — it avoids all three traps.
Pushing a corpus change
Bucket-managed fixtures are published from inside the test_documents submodule with the
authenticated gcloud storage wrapper. Preview the exact object and manifest changes first:
cd test_documents
python3 scripts/publish_corpus.py --bucket xberg-test-documents --dry-run
python3 scripts/publish_corpus.py --bucket xberg-test-documents
python3 scripts/verify_corpus.py --bucket xberg-test-documentsThe publisher writes content-addressed objects, refreshes corpus.lock.json, skips objects that
already exist, refuses tracked corpus binaries, and probes bucket write access. Publish the object
before committing or pushing the refreshed lock file; CI can verify public reads but cannot publish
working-tree binaries.
Push the submodule commit before the superproject gitlink. A local-only submodule commit
builds and tests green on the machine that made it and turns every CI workflow red at
checkout with Fetched in submodule path 'test_documents', but it did not contain <sha>.
Check with git branch -r --contains <sha> inside the submodule.
Not the same thing as fixtures/
fixtures/ at the repo root is committed and safe to include_bytes!. Only
test_documents/ is bucket-fetched.
