Imported from ActiveInferenceInstitute/COGANT (
cogant/docs/evaluation/AGENTS.md). Install upstream withnpx skills add ActiveInferenceInstitute/COGANT --skill evaluation. Copyright stays with the author.
AGENTS.md - Evaluation module
This directory is the human-facing evaluation companion to the
machine-readable artifacts in cogant/evaluation/. Public pages here must
describe the current checked-in evidence, not release archaeology.
Purpose and ownership
Evaluation pages are maintained as current status surfaces. Stable filenames may be kept for inbound links, but their content should be rewritten when a new ledger, metric contract, or validation gate replaces an earlier one. Do not leave an active page carrying an unsupported headline with a small correction banner.
The source of truth for numeric release claims is
cogant/evaluation/METRICS.yaml, regenerated by
uv run --directory cogant python ../tools/regenerate_metrics.py from the
project root. Per-target roundtrip details come from
cogant/evaluation/dataset/roundtrip_results.jsonl.
File map
| File | Purpose | Update trigger |
|---|---|---|
README.md |
Evaluation index grouped by readiness, roundtrip, empirical, calibration, and references | When a file is added, removed, or retitled |
AGENTS.md |
This file - maintenance rules | When evaluation-document policy changes |
V1.0_READINESS.md |
Current gate-by-gate v1.0 readiness assessment | At each readiness review |
FINAL_REPORT.md |
Consolidated current evaluation report | When a major evaluation milestone closes |
SCOPING_REPORT.md |
Current project scope and remaining release gaps | When scope or readiness changes |
R&D_LOG.md |
Current research and development status | When major evidence or validation status changes |
ACTIVE_INFERENCE_MAPPING.md |
Code patterns to Active Inference roles | When role vocabulary or mapping rules change |
ISOMORPHISM_THEOREM.md |
Role-preservation and invariant proof sketch | When the theory contract changes |
CALIBRATION.md |
Confidence and rule-calibration backlog | When calibration defaults or review corpus status changes |
CONSTRAINT_FIX.md |
Constraint-role recovery mechanism and current effect | When reverse synthesis behavior changes |
FIRST_INFERENCE.md |
Executable Active Inference loop notes | When runtime inference behavior changes |
ROUNDTRIP_EVAL.md |
Native v0.6 roundtrip evaluation results | When roundtrip results are re-measured |
ROUNDTRIP_VALIDATION.md |
Validation contract for roundtrip claims | When validation methodology changes |
ROUNDTRIP_IMPROVEMENT.md |
Current reverse-synthesis status | When role-preservation behavior changes |
CROSS_LANG_ROUNDTRIP.md |
Cross-language roundtrip study | When language-pair evidence changes |
REAL_WORLD_EVAL.md |
External-repository forward-pipeline fixture | When external-repo runs are refreshed |
EMPIRICAL_CLAIM.md |
Stated empirical claims and supporting evidence | When claims are added, narrowed, or withdrawn |
BENCHMARK_VS_PRIOR.md |
Baseline comparisons and open calibration work | When baselines are re-run |
INCREMENTAL_BENCHMARK.md |
Incremental rescanning benchmarks | When incremental analysis changes |
SCALING_ANALYSIS.md |
Current scaling behavior and regression watch | When a new scale run is measured |
GNN_VALIDATION_REPORT.md |
Validation results for generated GNN packages | When validator rules or output change |
MUTATION_REPORT.md |
Mutation-testing results | When mutation testing is re-run |
LITERATURE.md |
Bibliography-style references | When a cited paper is added or corrected |
RELATED_WORK.md |
Comparison with adjacent tools and research | When a comparable tool or comparison changes |
Editing rules
- Prefer current, source-backed statements over correction notes.
- If a number appears in
METRICS.yaml, cite or inject that value instead of duplicating a literal unless the page is deliberately explaining a fixed measured fixture. - Keep volatile facts in
METRICS.yaml, JSON artifacts, or a documented fixture file when the project has an injection path. - After link edits, run:
uv run --directory cogant python docs/verify_doc_links.py- After manuscript-facing numeric edits, run the manuscript audits from the project root:
uv run python tools/audit_manuscript_numbers.py
uv run python tools/audit_manuscript_crossrefs.py
uv run python tools/audit_manuscript_citations.py
uv run python tools/claim_ledger.py --manuscript-dir manuscript --output-dir /tmp/cogant_claim_ledger --fail-on-literal-numbers