event-recon

Imported from SovaDash/event-recon (AGENTS.md). Install upstream with npx skills add SovaDash/event-recon. Copyright stays with the author.

AGENTS.md

Rules for anyone — human or model — changing this repository.

The one structural rule

event_recon/core/ must not import verifiers. Ever. The core is the asset; the adapters are disposable wiring. CI enforces this with a grep, and a violation is a build failure, not a style note.

event_recon/core/      pure domain, generation, grading. No harness imports.
event_recon/adapters/  thin per-harness wiring. verifiers_v1.py is ~440 lines.
event_recon/toolsets/  the five tools the agent gets. Imports verifiers.

Four modules in core/ exist because a review found something, and each one's docstring says what. Read them before changing anything nearby:

• attacks.py — the strongest workspace-only policy we could build, shipped so CI measures it. CHEAT_CEILING bounds single-axis fixtures; this bounds the composite, which is the number a buyer prices on. It must never solve a task.
• leakcheck.py — nothing about the private split may appear in a tracked file. The seed formula leaked into a datasheet once; the constants were rotated and this exists so it cannot recur.
• zones.py — R9 makes the IANA database an input to the graded answer, so it is pinned rather than inherited from the host.
• ablations.py — per-rule near-miss solvers, reported in two columns because only reporting the upper bound was misleading.

The two-implementation rule

core/generator.py implements the accounting contract forward (simulate a clean world). core/reconciler.py implements it backward (derive what is owed from raw events). Both were written from INTEGRATION.md, and they must never share a helper. If they share code, a generator bug becomes an invisible grader bug and nothing catches it. The reconciler imports record types from core/world.py and the pinned zone table, and derives every monetary quantity itself; tests/test_contract_rules.py keeps the two R7 and R10 constructions separate but agreeing, which is the case that once slipped through.

The gate is scripts/gate_crosscheck.py: both implementations must agree on the balance projection for every task in the split. It has already caught two real contract defects. When they disagree, fix the contract or the implementation that is wrong — never change one to match the other.

Ground truth

There is no answer file in this repository and none is generated. A task is a seed; everything graded is recomputed from it at scoring time. Do not add a cache, a fixture directory, or a "golden output" file. If you find yourself wanting one, you are about to make the environment leakable.

core/reconciler.py, the injection manifest and data/private/ must never appear in a path the agent can read. core/workspace.py::assert_no_leakage proves this per task and CI runs it.

Before you publish anything

python scripts/gate_private_leak.py --require-private --history   # where private/ exists
mypy --strict private/event_recon_private               # same: CI cannot see this package
ruff check private/ && ruff format --check private/      # same, and named for the same reason
PYTHONPATH=private pytest private/tests                  # same: the held-back split's guards
python -m event_recon.adapters.cli gates                 # the whole public split, unsampled
PYTHONPATH=private python -m event_recon.adapters.cli gates \
    --family-module event_recon_private                  # the same eight gates, held-back split
PYTHONPATH=private python scripts/gate_crosscheck.py \
    --family-module event_recon_private                  # both implementations, held-back split
python scripts/write_reports.py --limit 30 --rescore 100 --oracle-runs 200
uv build --wheel && python scripts/gate_private_leak.py \
    --require-private --artifact dist/*.whl                # the artifact itself, not its inputs

The private line is not a duplicate of the one above it. For nine versions the held-back split -- the half that is actually sold -- was gated by event_recon_private.cli gates, which ran four of the eight gate families: oracle, null, the cheat probes and the composite bounds. It never ran the generator/reconciler cross-check, workspace leakage, determinism or the canary, never evaluated BOUNDED_PROBES at all, and appeared nowhere on this checklist, so it ran when somebody remembered. That runner has been deleted, so there is nothing left to run by mistake -- verified against the shared path on the same eight tasks first, where oracle, null, cheat max, sweep max, composite max and the held-back injection count all agree exactly. Its one unique measurement, the count of held-back families actually injected, is reported by the shared path now, on the [families] line.

--family-module runs the same code path as the public gate rather than a parallel implementation, which is the point: the two cannot drift, and every gate added later reaches both splits for free. It also inherits _parallel, so the run is minutes rather than the forty the serial loop took.

pytest private/tests is on this list for the same reason as the two lines above it. It holds the guard the private side did not have: that data/tasks.jsonl is what private_split() produces today. Both copies of that file sat five generator versions stale -- 120 of 120 task ids denoting tasks the generator no longer builds -- while the public split's equivalent guard had been green the whole time.

The cross-check line is separate because that gate is a script rather than a cli gates subcommand, and it is the one AGENTS.md credits with catching two real contract defects -- so running it only against the public split meant the two implementations were never required to agree about a held-back family at all. First run of the combination: 120/120 tasks agreeing over 3049 partner projections; 3084 after the seed rotation.

The precedent for why this matters is in that runner's own comment. When the composite bounds were finally run against the private split they came back at 0.1582 against a bound of 0.15 -- a breach that had been sitting there the whole time, invisible because no gate looked.

The last line is the only check that opens the thing that actually gets uploaded. Every other mechanism checks an input: .gitignore decides what git and the Hub archive walker see, pyproject.toml decides what hatchling packs, .dockerignore decides what a build context copies. Four arguments that the artifact should be clean, and no evidence that it is. --artifact reads member bytes rather than names -- a leak that mattered would be a seed inside a file, not a file called private -- and it is fire-tested: planting one seed into a copy of the wheel makes it exit 1, and it never echoes the value it matched.

--history is not optional on that first line, and for a long time it was missing. The two halves of the leak scan are orthogonal: the live values are readable only where private/ exists, and history is scanned only when asked for. CI runs --history on a runner with no private/, so it checks the past against markers and digests but never against a real seed; this checklist used to run --require-private without --history, so it checked real seeds against the working tree but never against the past. The combination — the live private values against every blob every ref reaches — ran nowhere, which is exactly the question leakcheck.check's docstring says publishing asks: making a repository public publishes its history, and this one keeps tags on the history it rewrote. First run of the combination: 353 historical blobs against 242 live values, no hits. (The value count is two spellings of each of the 120 seeds plus two of the canary. It was 126 when the seeds were a base and a stride, and the underscored spelling was covered for those two constants only -- the base was seed zero, so removing them without adding the underscored seeds would have silently dropped the only such coverage there was.)

Lines two, three and four are here rather than in [tool.mypy] files, a bare ruff check . and [tool.pytest] testpaths because private/ is gitignored: mypy fails outright on a path that does not exist, so listing it there would break CI on every checkout without the private split, and ruff honours .gitignore, so ruff check . walks straight past the package and reports the repository clean. Both defaults are right for CI and both mean the one directory a buyer actually receives is the one directory nothing checks. Naming it explicitly is what closes that, from the single machine that can see it. When this was first written down the package had 22 --strict errors, an unused import and an unformatted file.

The reports are generated, not written: the numbers in reports/determinism.md and reports/difficulty_calibration.md come out of scripts/write_reports.py, so re-run it after any change that could move one. A README quoting a number the code no longer produces is the fastest way to lose a diligence conversation.

That rule went unenforced and both files went three bumps stale -- difficulty_calibration.md last regenerated at generator 1.9.0 and determinism.md at 1.10.0, while the code produced 1.11.0 and core/tasks.py's own history recorded the composite mean moving 0.3409 to 0.3421 underneath them. Neither carried a version, so there was nothing to compare. Both are stamped now and tests/test_isolation.py::test_the_generated_reports_were_generated_at_this_generator_version fails when the stamp and GENERATOR_VERSION disagree, which turns "re-run it" from a habit into a gate. write_reports.py also exits non-zero when it writes a **FAIL** verdict; it returned 0 whatever it found, so a breach was a bold word in a file that ships.

That is a claim about the report's tables, and it was overstated for a long time. A handful of figures sit in the template's prose, interpolated from nothing, and one of them — a tier-1 identification recall quoted as 0.96 — was true when it was typed and silently false afterwards. It is measured now. The ones that remain are labelled where they appear: reports/determinism.md has a "Recorded once, not re-measured" section, and the four numbers behind invariants 5, 6 and 7 are quoted from the runs that prompted those changes and are cited to core/generator.py rather than to a report. If you add a number to a template, either interpolate it or say in the text that it is not recomputed.

Changing the contract

core/contract.py holds the normative text. Changing a rule changes what every task is worth, so:

1. amend the text first, then both implementations;
2. re-run scripts/gate_crosscheck.py and the oracle gate, which is a CLI subcommand rather than a script: python -m event_recon.adapters.cli gates;
3. bump GENERATOR_VERSION in core/tasks.py if any seed's world changes.

Every assertion in tests/ must trace to a numbered rule in the contract, and every numbered rule must be exercised by an assertion. tests/test_contract_coverage.py enforces both directions.

Running things

uv pip install -e .
python -m event_recon.adapters.cli gates          # oracle, null, cheat, determinism, leakage
python -m event_recon.adapters.cli solve --seed 4100000 --tier 1
uv run eval event-recon -n 5                      # needs a model endpoint
uv run validate event-recon --only-gold --runtime.type subprocess

--runtime.type subprocess is not optional: validate defaults to the Prime runtime and exits with "not authenticated with prime" without it. That last line is the oracle gate run through the framework, and nothing in this environment needs a container.

Do not

• Use an LLM judge for anything checkable programmatically. There is none in this repo and there is no reason to add one.
• Add a reward component that a policy which does nothing can max out. The non-destructiveness component is scaled by ledger correctness for exactly this reason — share * ledger_score in core/grader.py::score_non_destructiveness, so it is worth zero to a submission that repairs nothing. It was gated on the first repair once; that step function made the first repair worth roughly five times the second, and it was removed deliberately. Read the docstring before touching the weights.
• Give the agent a shell. The default harness is tool-only on purpose — see the EventReconHarness docstring.
• Publish data/private/, its seeds, or the private-only defect families. This has already gone wrong once: an early reports/datasheet.md printed the private seed formula, which is a full compromise — the seeds are the tasks, and a solver holding them scores 0.40 per task without reconciling anything. The constants were rotated and scripts/gate_private_leak.py now checks every tracked file, statically everywhere and against the live values wherever private/ exists. Run it before you publish anything, and never quote a private constant in a document, a commit message, or a comment.
• Remove /private/ or /data/private/ from .gitignore, even having convinced yourself that pyproject.toml already excludes them. It does, from the sdist and the wheel — and a prime env push archive is neither. The Prime CLI walks the tree itself and honours the root .gitignore, so those two lines, and not the packaging config, are what keeps the private split off the Hub. .dockerignore is a third, separate copy of the same decision, because a Docker build context ignores .gitignore too. Three mechanisms for three ways the tree gets copied; none of them is redundant with another.