Imported from ivanRajacic/horizon-scout (
.claude/skills/question-orchestrator/SKILL.md). Install upstream withnpx skills add ivanRajacic/horizon-scout --skill question-orchestrator. Copyright stays with the author.
/question-orchestrator
Fill quota cells of the M5 bank with drafted, adversarially attacked, judged, staged-for-human-review questions.
Arguments: $ARGUMENTS
Format: [output_dir] - optional directory for the working journal and the two canonical outputs. Default: eval/drafts/.
Or: a path to a packet file (a .json whose top-level kind is "packet") - then run in packet mode (see below): all setup is already done, skip straight to dispatching drafters.
Run this orchestrator session at medium reasoning effort. Never high. You route, you pick candidates, you relay messages; none of that needs high-effort reasoning, and the pilot's biggest cost leak was a high session effort amplifying re-reads of held evidence. Effort is set at LAUNCH (claude --effort medium "<prompt>"); a session cannot change its own.
Permissions this skill depends on. Unlike its prompt-only predecessor, this pipeline calls the shell and relays messages, so a batch stalls on a permission prompt without these in .claude/settings.local.json. If a run hangs with no MCP activity, check for a prompt before you check anything else:
mcp__horizon-draft the seven read-only tools
Task spawning drafters, critics, judges
SendMessage relaying to the warm drafter, critic, and judge
Read, Glob, Grep skills, brief, staged files
Write(eval/drafts/**) the journal's batch header (line 0) only
Bash(./.venv/Scripts/python.exe -m src.cli:*) the deterministic nodes (gap-report, next-ids,
validate-record, journal-append, write-batch)
What this skill is
A message bus with three jobs of its own: negotiate cells with the user, pick candidates, relay messages between nodes. It judges nothing, drafts nothing, attacks nothing, and formats nothing - each of those belongs to a node that exists for it. (In packet mode the first two jobs were done in advance by /draft-questions; only the relaying remains, plus the two decisions that cannot be made in advance - which candidate to fall back to when one is abandoned, and what lesson to relay to the next drafter.)
The design principle is that authority is split from execution:
- the drafter authors and verifies facts, and is gated by a deterministic precheck it cannot talk its way past;
- the critic attacks and reports typed findings, with no verdict and no kill power;
- the judge rules on every finding and decides fate, with no tools and no ability to investigate;
- you carry messages between them, and never substitute your own judgement for the judge's.
Everything mechanical - the gap report, id assignment, schema validation, the cross-batch collision sweep, and both output files - is a CLI command, not a model. If you find yourself counting, comparing, or formatting, you are doing a node's job.
The graph
SETUP [D] gap-report -> [H] user picks cells -> [A] you pick 3 candidates/slot
-> [D] next-ids -> [D] health probe (topical batches only)
PER SLOT budget: 3 candidates x (1 draft + 1 fix) = 6 drafter passes max
[A] DRAFTER ground / draft / execute / reference
|__ [D] precheck_record (MCP) <- internal gate; cannot emit a package
| until it returns ok
v RECORD + PRECHECK + CHECKLIST + EVIDENCE + WHY-GOOD + HISTORY
[D] validate-record schema gate; fail -> message the warm drafter
v
[A] CRITIC adversarial only; findings [class, HIGH|MID|LOW, evidence,
v fix_direction?]. Warm per slot, protocol re-draws keyed to
the diff. Blind to the budget and to the judge.
[A] JUDGE warm across this slot's rounds, no MCP tools.
Rules UPHELD/DISMISSED/RECORDED on every HIGH and MID first,
then ACCEPT | FIX <targets> | ABANDON <why>
|-- FIX -> back to the warm DRAFTER (1 per candidate)
|-- ABANDON -> next candidate, or fail the slot
\-- ACCEPT -> journal line
CLOSE-OUT [D] write-batch (runs batch-crosscheck, writes both files)
-> [H] user ticks APPROVE/REJECT -> [D] promote-drafts -> bank.jsonl
[D] deterministic (a CLI command or an MCP tool), [A] an agent, [H] the user. Every agent-to-agent edge physically transits you as the bus - that is what keeps the judge warm across rounds without nesting agents inside each other.
Who reads what
Keep these boundaries; they are what the design buys.
| Node | Sees | Never sees |
|---|---|---|
| drafter | its own cell, candidate block, and on a fix round the judge's named targets | other slots, the critic's raw report, the budget |
| critic | record + evidence + the drafter's precheck result; on a re-attack, its own prior transcript plus what changed |
the budget, other slots, what the judge wants, the judge's rulings on its findings |
| judge | its own slot's full typed state, warm across rounds | any other slot, the batch tally, MCP tools |
| you | everything | - |
A re-attack after a fix goes to the SAME warm critic (SendMessage), with mandatory protocol re-draws keyed to what changed - see step 3. The isolation that actually matters, and that stays absolute: the critic must never see the budget or what the judge wants - a critic that knows a fix round already ran starts calibrating severity to what it thinks you can afford, and one that sees rulings starts arguing with the judge instead of attacking the draft. The old rule dispatched a fresh critic per round on the theory that a cold agent is what keeps the attack honest; the measured runs said otherwise - the load-bearing property is not that the agent is cold, it is that the anti-anchoring protocols (BLIND-SOLVE, OWN-WORDING) get a fresh draw, which a warm critic produces on demand when told to and will not produce spontaneously. (Honest caveat: a warm critic accumulates its own transcript, so per-round token counts stop being independent and the saving is smaller than round-count x average-round-cost.)
Typed slot state - the working journal
One append-only JSONL at <output_dir>/draft-batch-journal-<YYYY-MM-DD>.jsonl, one line per transition, latest line per question_id wins. Append the moment a transition happens - this is the crash-recovery layer, and it is also the only input to the writer, so anything not in it does not reach the report.
Every slot transition goes through journal-append - never a hand-written script, never an Edit. You write exactly one line yourself: the batch header, at setup. After that:
./.venv/Scripts/python.exe -m src.cli journal-append <journal> --id hyb-08 --status REVIEWING --payload - <<'PAYLOAD'
{"findings": [ ...only the fields that changed... ]}
PAYLOAD
The payload holds ONLY what changed; the command merges it over the slot's latest line, enforces the envelope (kind, question_id, status, cell), keeps record opaque, and refuses HTML entities (< etc.) at the boundary instead of letting them into the bank. A pure status flip needs no payload at all. This is what makes real slot concurrency possible: the previous run hand-marshalled eighteen Python scripts, each re-quoting record, evidence, findings and history verbatim on the orchestrator's single thread while every other slot sat idle.
Line 0 is the batch header:
{"kind": "batch", "date": "2026-07-25",
"started": "2026-07-25T14:03:00", // `date +%Y-%m-%dT%H:%M:%S`, at setup
"order": "<the cells and counts the user asked for, in words>",
"model": "claude-opus-5", // REQUIRED - see below
"budgets": {"candidates_per_slot": 3, "passes_budget": 6},
"versions": {
"corpus_profile": {"version": "cp3", "content_hash": "..."},
"schema_docs": {"version": "sd2", "content_hash": "..."},
"bank_brief": {"version": "bb1", "content_hash": "..."},
"index": {"fingerprint": "...", "n_vectors": 190248}}}
model is required and you must not guess it. The three role agents do not
pin a model in their frontmatter - they inherit the model this orchestrator
session is running as - so the repo state does not record who authored a batch.
The header is the only place it lives. Write the model id this session was
launched with (claude --model <id>); if the packet carries a model field,
that is authoritative and you copy it verbatim. Never write a placeholder and
never infer one from the agent files. write-batch prints it on the report's
provenance line, and a report reading model: unrecorded means this step was
skipped - the run is then not comparable to any other and has to be re-labelled
by hand from the transcripts.
Every later line is a slot:
{
"kind": "slot",
"question_id": "sql-18", // primary key, survives candidate swaps
"status": "DRAFTING|REVIEWING|JUDGING|FIXING|ACCEPTED|FAILED|BLOCKED",
"terminal_reason": null, // why, on FAILED / BLOCKED
"cell": {"route": "sql", "level": "L2", "subtype": "rank", "term_style": null},
"candidates": [{"id": "sql-07", "topic": "...", "block": "..."}, ...], // all 3
"candidate_index": 0,
"budget": {"passes_spent": 2, "passes_budget": 6, "fix_rounds_this_candidate": 1},
"record": { /* the drafter's RECORD, verbatim, OPAQUE */ },
"evidence": "...", // verbatim from the drafter
"why_good": "...", // verbatim; the report needs it
"checklist": "...", // verbatim; traceability
"findings": [{
"round": 1, "class": "MISSED-GOLD", "severity": "HIGH",
"claim": "...", "evidence": "...", "fix_direction": null,
"ruling": "UPHELD", "ruling_why": "...", // filled in from the judge
"recorded": false // true once the judge rules it RECORDED:
// real, accepted, rides into the report,
// and a later re-discovery of the class on
// this candidate is noted, never re-ruled
}],
"defect_classes_seen": [{"candidate": 0, "class": "MISSED-GOLD", "round": 1}],
"judge_decisions": [{"round": 1, "disposition": "FIX",
"targets": ["MISSED-GOLD"], "rationale": "..."}],
"history": ["...", "..."]
}
The envelope is always valid; record is opaque and may be schema-invalid mid-run. That is deliberate: it preserves the never-validated-mid-run rule while giving every node a typed contract. kind, question_id, status, and cell must be present and well-formed on every line - the writer refuses a journal whose envelopes are broken, and refusing at close-out is much better than a malformed report.
Resume is manual. Hand the journal to an agent to continue; this skill adds no automatic resume. The journal is a working file, not a canonical output, and is disposable after promote.
Closed-slot discipline: once a transition is journaled, do not re-quote, re-summarize, or re-reason over that slot's evidence on later turns. The writer reads the journal; you do not need to hold it.
Packet mode
When the argument is a packet file, skip steps 1-4 of the procedure entirely - no gap-report, no cell negotiation with a user, no get_corpus_profile, no next-ids, no health probe. All of that was done once by /draft-questions, which wrote the packet and launched this session; the servers were probed immediately before launch. Your first MCP call in packet mode belongs to a drafter, not to you.
The packet is one JSON file:
{
"kind": "packet",
"output_dir": "eval/drafts/groupA/", // this group's own directory - use it for
// the journal and both outputs
"order": "<what this group is drafting, in words>",
"siblings": "<one line: what the other groups are working on - so an abandoned
candidate is never swapped for a topic a sibling tab already has>",
"versions": { /* exactly the batch header's versions block: corpus_profile,
schema_docs, bank_brief, index - already resolved */ },
"slots": [
{
"question_id": "vec-25", // pre-assigned; never call next-ids
"cell": {"route": "vector", "level": "L2",
"subtype": "...", "term_style": "..."},
"candidates": [ // all three, in the order to try them
{"topic": "...",
"block": "...", // the corpus-profile candidate block, verbatim
"bucket_map": "good for: ... / thin for: ... / texture: ..."}
// x3
]
},
{
"question_id": "adv-01", // an ADVERSARIAL slot
"cell": {"route": "sql", "level": "ADV", "subtype": "zero-match"},
"parents": [ // three, in the order to try them;
{"twin_id": "sql-08", // `parents` replaces `candidates`
"record": { /* the parent's COMPLETE bank entry, from pick-parents */ }}
// x3
]
}
// one per slot, up to three
]
}
In packet mode:
- Read the packet, write the batch header (line 0) into
<output_dir>/draft-batch-journal-<date>.jsonlusing the packet'sversionsverbatim, the packet'smodelverbatim (or, if the packet has none, the model id this session was launched with - never a placeholder), anddate +%Y-%m-%dT%H:%M:%Sforstarted, then go straight to step 5, dispatching every slot's first drafter at once. - The candidate blocks and bucket-map lines in the packet are what you pass to drafters - do not re-pull them from the corpus profile. For an ADV slot, the parent record plays exactly that role: pass it verbatim, and never substitute a parent of your own choosing.
- Everything from step 5 on is identical to a normal run: the per-slot loop, the stop rules,
write-batch, the trace, the final message. The final message is your report; the launching session collects it, so make the two output paths and the tally unmissable. - The two decisions that stay yours: when a candidate is abandoned, move to the next candidate in the packet's order, and relay the distilled lesson (the trap, never the verdict or content) to the next drafter, exactly as step 5 says. Never invent a candidate not in the packet - if all three die, the slot fails, same as always.
- No user is watching by default. Never stop to ask; journal, decide within the budget, and report at the end. (A user CAN interject in the tab mid-run - honour what they say - but do not wait for them.)
Procedure
1. Gap report, user picks (plain text)
./.venv/Scripts/python.exe -m src.cli gap-report
It reads the allocation table LIVE from horizon-scout.md and prints filled + staged / target per route x level, the adversarial line, the subtype spread, the term_style balance for vector and hybrid, the next free id per cell, and which bank questions are still available as adversarial parents. Print it as-is; do not recompute anything from it.
Then ask the user, in plain text (never the multiple-choice window), which cells and how many questions each. Wait. Their picks become the batch order. Never exceed a cell's target without the user saying so explicitly. ambiguous and compositional cells stay out of this batch - their skills exist, but they are interactive-only (compositional permanently so, by design). Adversarial cells ARE draftable here; see step 2b for how their subject is chosen.
2. Candidates - three per slot
- Call
mcp__horizon-draft__get_corpus_profileforfrontierplus each needed route section. An{"error": ...}result means the profile is unbuilt - STOP and tell the user to run/explore-corpusfirst; this skill does not draft from thin air. - For each ordered slot pick three candidates whose
recommend:matches the cell, in the order you would try them. Prefer buckets the frontier marksmappedbut not yetmined- regions we deliberately explored and have not yet drawn from, which is what keeps the bank spread across the corpus. Break ties on least-covered axes, and skip candidates whose axis values or named entities are already used by bank questions or by candidates picked for this batch. Candidates are advisory seeds; the drafters re-verify everything. - If a candidate carries a
bucket:line, pull that bucket's## Corpus mapentry and pass itsgood for:/thin for:/texture:lines along with the candidate block. That is the region knowledge exploration paid for: it tells the drafter what shape of question the region can actually support before it starts grounding.
Three, not one plus a spare: the stop rules below can kill a candidate for a reason that has nothing to do with the drafter, and a slot with only one fallback fails on the second unlucky pick.
2b. Parents - three per ADVERSARIAL slot
An ADV slot's subject is not a corpus-profile topic, it is a bank question. Each adversarial question is a perturbation of an answerable one, which is both the control the refusal cells need and the proof that the emptiness sits next to something real.
./.venv/Scripts/python.exe -m src.cli pick-parents --n 9
The node is deterministic and owns everything with a right answer: it excludes questions already used as some other ADV question's twin (in the bank or in any staged draft), excludes ids named by --exclude (what sibling tabs have claimed), and orders what remains as a round robin over route, then subtype, then level, so parents off the top of the list are different kinds of question rather than three of a kind. Ask for 3 x <number of ADV slots> and hand each slot three, in the order returned. Do not re-rank them and do not substitute one you like better.
unanswerable is the exception: it derives from nothing and takes no parent. If the order names that subtype, the slot's "candidates" are the subtype alone and the drafter authors from scratch.
3. Assign ids
./.venv/Scripts/python.exe -m src.cli next-ids --sql N --vector N --hybrid N --adversarial N
Counts the bank AND every staged draft file, so a staged id is taken even before promotion. Assign one id per slot up front so parallel drafters never collide. Failed slots leave id gaps - harmless.
4. Health probe (topical batches only)
If the batch contains any topical slot (vector, hybrid, or topical ADV - the routes that need the llama-servers), run one mcp__horizon-draft__search_corpus("probe", k=1, snippet_chars=0) as an environment health check, not authoring. Expect the first pooled call after server start to be slow - it is the embedder and reranker loading their models, serialised behind one call, and it can exceed the 120 s tool timeout. That is a cold start, not an outage: give the first probe a longer window (run it in the background and wait) rather than killing and re-issuing it - the 2026-07-25 run lost ~7 minutes to exactly that stop-and-retry dance. Only a probe that returns an ERROR RESULT (or hangs long past any plausible model load, ~5 minutes) means the servers are down: then do not dispatch any topical slot, journal those slots BLOCKED with terminal_reason: "retrieval servers down", run only the SQL slots, and report the blocked slots to the user. A SQL-only batch skips this probe.
5. The per-slot loop
Concurrency: unlimited - every slot runs at once. However many slots the batch has, that many chains are in flight: 4 questions means 4 drafters, then their critics and judges as each slot's rounds arrive. There is no agent cap to schedule around; the bound on a batch is its slot count. The shared resources still physically serialize underneath (the MCP server is one stdio process over a single read-only DuckDB connection; the embedder and reranker share one GPU) - that shows up as slower individual calls, not as a reason to hold a slot back. Watch for MCP and rerank contention on the first wide topical run and report timeouts if you see them.
Budget per slot: 3 candidates, 6 drafter passes, 1 fix round per candidate.
Slots run concurrently; the loop below is written per-slot only for readability. Dispatch every slot's first drafter BEFORE handling any return, then handle returns in the order they arrive - never in slot order, and never blocking slot B's dispatch on slot A's bookkeeping (with journal-append, a transition is one cheap call, so there is no bookkeeping excuse left). The 2026-07-25 run's wall clock (97 m) exceeded its total agent-time (86 m) - there were stretches with NO agent running - because every transition was a hand-written marshalling script on this single thread. Under real 4-wide dispatch the floor is the longest single slot chain (~41 m on that run's shape), and agent-time should EXCEED wall clock - that is what real concurrency looks like.
Retry = resume, not respawn. When an agent dies mid-flight on a transport error (ENOTFOUND, connection reset), the preferred retry is to SendMessage the same agent with a short "you were cut off after X - carry on" note: its transcript is intact and it resumes exactly where it stopped, keeping work already paid for (a completed grounding pass is ~140k tokens). Respawn only when the transcript is unusable. Either way it consumes the slot's single retry.
Per slot, per candidate:
- DRAFTER. Spawn a
question-drafter. Prompt: the pre-assignedquestion_id, the cell (route/level/subtype, term_style if topical), the candidate block verbatim plus its bucket map lines, and the instruction to follow the route's drafting skill in orchestrated mode. For an ADV slot the subject is the parent instead of a candidate block: give thetwin_idand the parent's complete record verbatim, and point the drafter at/draft-adversarial-questionin orchestrated mode - the costume route decides how the question is phrased, not which skill runs. JournalDRAFTING.DRAFT-FAILED - retrieval servers down-> the outage path (below). Do NOT consume a candidate.- Any other
DRAFT-FAILED-> journal it intohistory, advancecandidate_index, go to the next candidate.
- validate-record. Pipe the returned RECORD straight in on stdin - no temp file, and a quoted heredoc means a record containing quotes or
$cannot break the shell:./.venv/Scripts/python.exe -m src.cli validate-record - <<'RECORD' {...the drafter's RECORD line, verbatim...} RECORD- Valid -> continue.
- Invalid -> send the exact validator output to the SAME warm drafter (SendMessage) and re-validate its correction. A schema round costs a pass but is NOT the candidate's fix round - it is a contract failure, not a quality one. One schema round only; still invalid -> abandon the candidate.
- CRITIC. For a candidate's first round spawn a
question-reviewerin draft mode. Prompt: aDRAFT:block containing the RECORD JSON, the EVIDENCE section, and the PRECHECK result, all verbatim. For a re-attack after a fix, SendMessage the SAME warm critic with the updated package, a plain statement of WHAT CHANGED (the diff surface, e.g. "the question text and filter were reworded" or "only the reference answer changed"), and anyevidence_carried_forwarddisclosure the drafter made - never the judge's rulings, never the budget. The critic's own skill keys its mandatory protocol re-draws to that diff statement, so state it accurately. JournalREVIEWING, then journal the parsed findings. Tell the critic which classes are alreadyRECORDEDon this candidate so a re-discovery comes back as a note, not a finding to re-rule.STATUS SKIPPED - retrieval servers down-> the outage path. Do NOT consume a candidate.STATUS REVIEW-FAILEDor a dead agent -> retry that agent once (resume-from-transcript preferred - see step 5's retry note), then journal the slotFAILEDwith the reason.
- JUDGE. For the slot's first round spawn a
question-judge; for every later round SendMessage the SAME judge (it is warm across the slot's rounds - that is what lets it see a repeat class without being told). Payload: this slot's typed state only - never another slot's. JournalJUDGING, then write the rulings back onto the findings and append the decision tojudge_decisions.- ACCEPT -> journal
ACCEPTED. Slot done. - FIX
<targets>-> relay the judge's targets and direction to the SAME warm drafter (SendMessage - never respawn for rectification, its context is warm). JournalFIXING, then loop back to step 2 with the returned package. If the returned fix package carries anevidence_carried_forwarddisclosure (measurements NOT re-run because the edit did not invalidate them), pass that disclosure to the critic verbatim in the re-attack prompt - the critic re-measures rather than trusting it, and that only works if it is told. - ABANDON -> journal the reason into
historyandterminal_reason, advancecandidate_index, next candidate. Relay the lesson to the next candidate's drafter: the trap, never the verdict and never the content. Distil the abandonment into one transferable instruction ("check where your topic sits in the euroSciVoc path before wording the scope; word it so it has exactly one executable reading") and put it in the fresh drafter's prompt. The verdict would prejudice a node that must judge independently; the dead question's content would anchor the new drafter on it; the lesson is a known trap and travels free. Candidates or passes exhausted -> journal the slotFAILEDwith the judge's reason. If the judge flagged the cell suspect (the cross-candidate stop rule), record that on the slot and repeat it in the close-out message.
- ACCEPT -> journal
You do not overrule the judge. There is no override valve any more and no reviewer_override in batch mode - with a real judge there is nothing to override, and the old valve existed only because the node that ruled was also the node that paid. (The field stays in the schema for the interactive "confirm anyway" path, and bank.py still tallies it.) Your only discretion is in relaying: you may sharpen the judge's FIX direction as you pass it on, never reverse its disposition.
Outage path. A servers-down signal from a drafter or a critic is a channel signal, not a quality signal, and it never reaches the judge. Stop dispatching topical slots, journal the remaining topical slots BLOCKED with terminal_reason: "retrieval servers down", finish any SQL slots, and report which slots were blocked. Never advance the candidate index on an outage - a fresh candidate on dead servers is the same bet, not a better one.
6. Close out
./.venv/Scripts/python.exe -m src.cli write-batch <journal path>
This runs batch-crosscheck and writes both canonical outputs: <output_dir>/draft-bank-<date>.jsonl (one RECORD line per accepted slot, byte-identical, promotable as-is) and <output_dir>/draft-report-<date>.md (the self-contained human-review report, with the machine-parsed Draft-bank-file: header and one Decision: [ ] APPROVE [ ] REJECT per accepted question). The report restates every accepted question's gold, evidence, reference, why-good, findings with the judge's rulings, and history, so the user can decide each one cold.
The writer refuses to overwrite an existing pair - it may hold an unpromoted earlier batch. If it refuses, ask the user, then re-run with --suffix -2 (both names stay paired).
The cross-check is the sweep no per-slot node can run: the critic's near-duplicate check reads get_bank_questions, which returns the promoted bank, so two parallel slots can converge and nothing notices. Its output is flags on the report, never a gate and never a redraft - the user adjudicates at promote time. Do not act on a flag yourself.
Then trace what the batch cost:
./.venv/Scripts/python.exe -m src.cli agent-trace --orchestrator --steps --since <the header's "started">
Per agent: turns, steps, active time, span, output tokens, input tokens with the cache share, and tool calls - read from each agent's own transcript, so the slots you ran concurrently are never confused with each other. Pass --since or a long session picks up an earlier batch's agents. It rolls up two ways that matter here and nowhere else: by agent type, which is what a drafter costs against a critic against a judge, and by slot, which is what one question cost across the three agents that made it (the slot id is read from the spawn descriptions, so keep writing them as Draft hyb-09 ..., Attack draft hyb-09 round 2, Judge slot hyb-09).
--steps is the part that matters for warm agents. A drafter's fix round, a critic's re-attack and each round the warm judge ruled on are separate lines, so "the judge cost 38 minutes" resolves into four rulings totalling three minutes plus 35 minutes of waiting on the bus. Read the two time columns together: active is work, span is lifetime. Summed active far below summed span means agents sat warm and idle - and if the batch's wall clock also exceeded summed active, the bus was the bottleneck, which is the exact failure step 5 warns about.
This goes in the final message, not into the report: write-batch writes the canonical outputs, and those must not depend on a harness-internal transcript format that is not ours to control.
7. Final message
Both paths, the tally (accepted / failed with reasons / blocked), any cell the judge flagged suspect, id gaps left, the cross-check flag count, the agent-trace table with its by-type and by-slot rollups, and:
./.venv/Scripts/python.exe -m src.cli validate-bank # optional pre-check
./.venv/Scripts/python.exe -m src.cli promote-drafts <report path>
Stop rules
The judge enforces these off the typed state; you supply the state and honour the disposition.
| Stop | Trigger | Action |
|---|---|---|
| within-candidate | the same defect class upheld again after a fix round that targeted that class (a fix that did not take - not a fresh critic re-finding a RECORDED note that no fix ever targeted) |
abandon it, next candidate |
| cross-candidate | the same class kills two candidates in one slot |
fail the slot, flag the cell as suspect |
| budget | 6 drafter passes spent, or all 3 candidates used | fail the slot |
A failed slot is a normal outcome, not an error. An empty cell is cheaper than a corrupt bank entry.
Standing rules
- Two canonical outputs, plus one working journal. The staged jsonl and the report are written by
write-batch, never by you. The journal is yours to append to and nothing else. Never writeeval/bank.jsonl, never skills, never agents, never the corpus profile. - Relay, do not adjudicate. You never author, never attack, never rule, and never open the MCP tools to form a finding of your own. Your MCP use is limited to setup: the corpus profile and the health probe.
- Records are byte-identical. What the drafter returned is what gets staged. There is no sanctioned edit any more - the
reviewer_overridecarve-out is gone with the override valve. - Every slot accounted for. N ordered slots in, N rows out - accepted, failed, or blocked. The writer enforces this; do not hide a slot by leaving it out of the journal.
- Bounded everything (except width). Concurrency is unbounded - one chain per slot, all slots at once. Per slot the bounds stand: 3 candidates; 1 fix round per candidate (8 drafter tool calls max); 1 schema round; 1 retry per dead agent (resume-from-transcript preferred, respawn as fallback); 6 drafter passes. When a bound is hit, record it and move on - never loop.
- The quota is the user's. Cells and counts come from the gap-report conversation; never top up beyond the order because a candidate looked promising.
- Existing output files are never overwritten without asking.