Imported from Shuhei-uni/P4P_knowledgeWiki (.agents/skills/design-experiment/SKILL.md). Install upstream with npx skills add Shuhei-uni/P4P_knowledgeWiki --skill design-experiment. Copyright stays with the author.

Design Experiment

Formalize an approved simulation direction into evidence-complete work, not new cases. The phase-root CONTEXT.md owns candidate selection; this skill owns comparison logic, evidence design, figure planning, and faithful scientific formalization.

A useful design may be one decisive setup or a campaign of linked setups whose combined evidence answers a question that no single run can. The unit of design is the scientific strategy, not automatically one simulation.

Respect the mandatory phase lifecycle

This skill has two modes, but they are not interchangeable entry points inside an autonomous phase.

DISCOVERY DESIGN
→ discovery runs and analysis
→ DISCOVERY_EVIDENCE = PASS
→ HYPOTHESIS_DEFINITION = PASS
→ HYPOTHESIS-TEST DESIGN

Discovery is the normal first experiment-design stage of a new phase. Hypothesis-test mode is legal only when the phase-root phase-state.yaml shows DISCOVERY_EVIDENCE == PASS and HYPOTHESIS_DEFINITION == PASS, unless the human phase handoff supplied equivalent verified discovery evidence and verify-phase-transition accepted it.

Do not bypass the lifecycle because one candidate looks obvious.

Start from the uncertainty

Read the phase-root CONTEXT.md before designing. Require a named candidate or conditional qualification path with Human status: approved, its origin, decision gate, and declared claim limit. If any is absent, return to phase-planner / phase-grill; do not invent it here.

Begin with the approved question the phase still cannot answer.

Understand what is known, what is only suspected, what competing explanations are plausible, and what observation would actually change the current understanding.

Do not begin from an available parameter list and ask what can be swept. Begin from the uncertainty and ask what evidence could resolve it.

Reasoning, literature, previous experience, and prior simulations can shape the hypothesis. Unless there is genuinely equivalent prior evidence, they do not establish what a new simulation will do.

Check approved candidates for prior experiment collisions

Before formalizing an approved candidate, inspect retained Project history across all phases, not only the current phase. Search by scientific substance: physical mechanism, modelling choice, formulation, boundary condition, numerical change, initialization, operating regime, intended question, and comparison logic.

For every approved candidate record:

• what was already tested;
• what happened, including failed, non-converged, partial, rejected, or inconclusive outcomes;
• the exact scientific delta;
• why existing evidence/additional analysis does not already answer the question;
• novelty: NEW, PARTIAL REPEAT, REPLICATION, or REDUNDANT.

Reject REDUNDANT candidates. A failed historical run still counts; rerun only when a concrete correction or unresolved delta makes the new attempt scientifically different. Return a rejected approved candidate to the human; do not substitute a new one.

This collision check applies equally to mainline, discovery, and speculative work.

Discovery mode — earn a hypothesis

Use discovery mode to determine what deserves qualification, not to produce the final phase claim.

Use explore-experiment-space only to organize approved context candidates into a contrastive campaign. The declared human-approved decision gate determines the smallest adequate screen; it may contain a few tests when they genuinely distinguish the stated alternatives. Roughly 500–1,000 iterations per case is a useful project ballpark when sufficient to expose comparative behaviour, not a convergence criterion.

Design discovery for:

• breadth across plausible mechanisms/branches;
• comparability;
• early response shape and direction;
• diagnostics that distinguish explanations;
• enough evidence to formulate a falsifiable hypothesis.

Every discovery strategy must state what it is screening and what outcome would justify deeper qualification.

Mandatory discovery output

The discovery design must define the evidence needed so the later analysis can produce:

candidate hypothesis
supporting discovery observations
strongest competing explanation
why discovery alone is insufficient for the final claim
what a long qualification run would need to establish

A discovery campaign has not succeeded merely because its cases completed. It succeeds when every approved screen has completed, its declared evidence has been compared, and its decision gate either authorizes a named qualification path or returns the decision to the human.

Before implementation, scientific-phase-loop must obtain DISCOVERY_DESIGN == PASS from verify-phase-transition.

Hypothesis-test mode — design for a strong statement

Use hypothesis-test mode only after the hypothesis contract is verified.

Start from the hypothesis contract, not from the most promising discovery setup. Preserve:

• one clear falsifiable hypothesis/question;
• discovery evidence that motivated it;
• strongest competing explanation or material alternative;
• observations that would support it;
• observations that would weaken/reject it;
• important assumptions and claim limits.

Write the intended claim form before designing the run

State the form of strong statement the evidence should be capable of supporting, without predicting which answer will occur.

For example:

Under the specified model and boundary conditions, the carrier solution is or is not demonstrably bounded and phase-mass-closed over the declared qualification window.

Then work backward from that statement. If a piece of evidence would be required to make the statement defensible, instrument it before the solve.

Long qualification depth is mandatory

For ordinary steady iteration-based full-geometry hypothesis qualification in this project, the planned horizon must be at least 10,000 iterations by default.

For slow inventory, routing, stationarity, or convergence questions, 10k–30k or another deliberately justified longer horizon may be appropriate.

A shorter hypothesis run is allowed only when:

• the human explicitly approves the exception; or
• the experiment uses a scientifically equivalent non-iteration qualification basis appropriate to the model/question.

Do not relabel a 500–1,000 iteration discovery screen as hypothesis-test.

When the intended claim depends on steady/stationary/bounded/reference behaviour, include a deliberate continuation or cold save/reopen qualification window when needed to distinguish a durable state from a favourable transient.

Hypothesis qualification contract

Record at minimum:

hypothesis: ...
discovery_basis: ...
competing_explanation: ...
intended_statement_form: ...
would_support: ...
would_weaken: ...
qualification_horizon: ...
qualification_window: ...
restart_or_continuation_requirement: ...
required_numerical_evidence: ...
required_physical_evidence: ...
required_core_figures: ...

If numerical credibility is necessary to the claim, scaled residual/equation histories or another explicitly justified equivalent are required evidence, not optional post-run decoration.

Missing required evidence after the run means the evidence gate fails; do not design a contract whose requirements the implementation cannot actually capture.

Use the live fleet as a constraint and opportunity

Use the current resource envelope from fluent-fleet-orchestration before finalizing runnable work.

Know which servers are reachable, which exact parents/recovery states are available locally or through OneDrive, and which sessions the active scientific goal can take over under its authority envelope.

Let fleet state influence campaign shape and execution efficiency, not the scientific question itself.

When two or more servers are usable, use the capacity only for approved context items. bold-probe-research may support an approved speculative candidate; idle capacity never authorizes a new bold lane.

Keep setup identity server-neutral. Placement is resolved later.

Record working assumptions without hiding missing facts

Keep hypotheses and assumptions distinct:

• hypothesis — actively tested;
• working assumption — accepted temporarily so the test can proceed;
• missing human-owned fact — cannot be replaced by an assumption unless the phase contract explicitly authorizes a surrogate class.

Use accepted-for-now, questioned, and materially-challenged where useful.

If an assumption would materially determine the answer rather than merely bound it, surface that before compute. If it crosses a human boundary, the correct result is HUMAN_REQUIRED, not an invented target or plant parameter.

Design the smallest useful strategy

Prefer the smallest strategy that can produce the needed learning at the required evidence depth.

In discovery the smallest valid strategy is the approved contrastive screen that can satisfy its declared gate. Add breadth only when the human approves the additional candidates in CONTEXT.md.

In hypothesis-test mode this may be one deep run, a controlled pair, or a very small linked campaign. “Smallest” must not be used to shrink the qualification horizon until the intended claim is no longer supportable.

Design evidence and figures before the run

Analysis planning is part of experiment design.

For each hypothesis/comparison decide what result would let you judge it and what evidence must therefore exist when the run finishes. Prefer behaviour over the run rather than only an endpoint.

Require a small core figure set

Default:

• discovery: usually 1–3 reusable core figures;
• hypothesis qualification: usually 2–5 core figures supporting distinct reasoning steps.

The first core figure should normally be the most direct visual answer to the experiment question. Generic residual dashboards are supporting numerical evidence unless convergence itself is the question.

For every core figure specify:

Figure and evidence rules

• one scientific message per figure;
• use iteration/physical time for evolving quantities unless another axis is more informative;
• avoid unfocused spaghetti plots and unnecessary dual axes;
• preserve raw oscillation/drift even when showing reductions;
• contours must answer a spatial question;
• define units, sign conventions, phase/zone scope, and comparison windows;
• if a decisive history cannot be reconstructed from final data, require its monitor/report before the run;
• define derived metrics before execution.

The later analysis agent should be able to create the key figures without inventing the scientific story after seeing the data.

Challenge approved strategies

When several approved strategies exist, compare them without generating a new one. Use arena, independent subagents, or literature-focused swarm only to challenge or research a context-approved candidate.

Call question-experiment before formalizing an approved strategy.

For hypothesis qualification, independent challenge is mandatory. The reviewer must check not only scientific value, interpretability, and cost, but also whether the proposed evidence/horizon could actually support the intended strong statement and whether a human lock or material missing fact is being bypassed.

question-experiment recommends a strategy. It does not grant lifecycle permission.

Formalize

Formalize the named approved context candidate for the current mode, then call create-setup. Record its context path, candidate ID, gate ID, and human approval in the resulting setup. Do not change its scientific purpose, intentional delta, invariants, claim limit, or evidence objective.

Before discovery implementation, require DISCOVERY_DESIGN == PASS from verify-phase-transition.

Before hypothesis qualification launch, require HYPOTHESIS_RUN_READY == PASS after setup creation, implementation readback/save-reopen/smoke, instrumentation verification, horizon verification, and the runtime-specific self-wake contract are all in place.

Do not write predicted results into setup records. The setup defines what will be tested, what evidence will judge it, and how that evidence should be visualised. Simulation data decides what happened.