Imported from yanqian/ai-helloworld (
.agent-harness/AGENTS.md). Install upstream withnpx skills add yanqian/ai-helloworld --skill .agent-harness. Copyright stays with the author.
AGENTS.md
This repository uses a long-running AI agent harness.
Agents must behave like stateless workers:
- No durable memory between sessions.
- All durable state lives in repository files.
- Every session reconstructs context from files and git history.
Core Principle
Never rely on chat history. Always rely on project state.
Required Startup Protocol
Before planning, coding, evaluating, or resuming work, agents must:
-
Read
progress.md. -
Read
feature_list.json. -
Check recent work:
git log --oneline -20 -
Run:
./init.sh
Roles
Initializer
Use this role when bootstrapping a new repository from the template.
Responsibilities:
- Create or adapt
SPEC.md. - Create or adapt
feature_list.json. - Create or adapt
progress.md. - Create or adapt
init.sh. - Initialize git when the repository has no git history.
- Do not implement business logic during initialization.
- Before a project minspec exists,
init.shmay only verify the harness and must not claim project business behavior is runnable.
Planning Agent
Use this role for new requirements before implementation begins.
Responsibilities:
- Read
AGENTS.md. - Read
SPEC.md. - Read
feature_list.json. - Normalize the new requirement into explicit SPEC additions.
- Decompose broad requirements into independently verifiable feature entries.
- Use
docs/project-recovery-init.mdwhen planning a fresh project or runnable skeleton. - Append new feature entries to
feature_list.json. - Preserve all existing feature IDs, order, status fields, attempts, errors, and unknown fields.
Strict rules:
- Do not modify existing feature IDs.
- Do not reorder existing features.
- Do not reset existing feature state.
- Only append new feature entries unless explicitly instructed otherwise.
- Use
docs/spec-normalization.mdbefore feature decomposition. - Do not turn vague requirements into feature entries without explicit goal, included scope, excluded scope, core flows, constraints, ambiguities or assumptions, required capabilities, implementation paths, and verification surface.
- Do not collapse unrelated or independently verifiable work into one feature.
- Use
docs/feature-decomposition.mdto decide whether to split or intentionally merge work. - If a minspec has been accepted and the project has no runnable skeleton, append a project recovery feature before product behavior depends on runtime assumptions.
- Ensure
feature_list.jsonremains valid JSON. - Ensure feature IDs remain unique.
Coding Agent
Use this role to implement exactly one feature.
Default entrypoint:
-
For one-feature implementation and evaluation, use the orchestrator first:
make work -
Use manual or interactive Coding Agent work only as an explicit fallback when role adapters are not configured, unavailable, or the user explicitly asks for manual work.
-
Manual fallback must be recorded in
progress.mdorruns/and must not bypass evaluator pass, evaluator evidence, attempts, failure records, or final./init.shverification.
Responsibilities:
- Follow the startup protocol.
- Implement only the selected feature.
- Keep the system runnable.
- Update
progress.md. - Update only the selected feature in
feature_list.json. - Preserve unknown feature fields and feature ordering.
- Do not stage or commit during orchestrated runs.
- Do not modify unrelated pre-existing working-tree changes.
The Coding Agent must not mark unrelated features as done.
Evaluator Agent
Use this role to verify one feature.
Responsibilities:
- Follow the startup protocol.
- Inspect the implementation for the selected feature.
- Run relevant tests or harness checks.
- Verify the feature against its description and acceptance criteria.
Strict rules:
- Do not implement new features.
- Do not mark unrelated features done.
- Do not accept incomplete work.
- Prevent premature completion.
- If verification fails, explain the exact failure.
The Evaluator Agent must output exactly one of:
EVAL_PASS: Fxxx
EVAL_FAIL: Fxxx: <reason>
Orchestrator
orchestrator.py owns the default one-feature work entrypoint and optional unattended feature execution.
Responsibilities:
- Follow the startup protocol before doing anything else.
- Pick one unfinished feature per round.
- Mark the selected feature
status="in_progress". - Increment the selected feature's
attempts. - Run a Coding Agent prompt for that feature.
- Run an Evaluator Agent prompt for that feature.
- Mark the feature done only after evaluator pass.
- Mark the feature failed or blocked after coding or evaluation failure.
- Write a failure run record when unattended coding or evaluation fails.
The template orchestrator is vendor-neutral. By default it supports --dry-run prompt preview. To execute real agents, replace the role adapters:
scripts/run-coding-agent.shscripts/run-evaluator-agent.sh
The orchestrator sends each role prompt to the corresponding adapter on stdin.
If either role adapter is missing, not executable, or still the template adapter, real orchestrator work must fail closed with clear configuration guidance before silently treating manual state edits as completion.
Default adapters delegate to scripts/run-agent-provider.py. Configure them by copying agent-provider.example.json to agent-provider.json and selecting an explicit provider. Do not guess between Codex, Claude Code, Cursor Agent, or custom providers; missing, ambiguous, or unavailable provider setup is a capability gap.
State Files
Repository Knowledge Map
AGENTS.md is the entry point, not the whole manual.
Use these durable knowledge files:
docs/README.mdfor the repository knowledge index.docs/architecture.mdfor structure and boundaries.docs/testing.mdfor verification layers.docs/agent-provider-configuration.mdfor explicit Codex, Claude Code, Cursor Agent, and custom provider configuration.docs/spec-normalization.mdfor converting vague requirements into clear SPEC additions before feature decomposition.docs/feature-decomposition.mdfor requirement-to-feature splitting rules.docs/project-recovery-init.mdfor separating harness verification from project recovery.docs/evaluator-evidence.mdfor requiring durable evaluator pass evidence before feature completion.docs/commit-messages.mdfor linking commits to feature IDs.docs/external-behavior.mdfor CLI, API, runtime, and tool-output verification rules.docs/capability-gaps.mdfor missing tools, permissions, dependencies, generators, environment setup, and required project capabilities.docs/example-boundaries.mdfor rules that keep default examples from becoming project implementation shortcuts.docs/agent-workflow.mdfor planning, coding, evaluation, continuation, and run artifacts.docs/failure-domains.mdfor failure classification and harness improvement rules.QUALITY.mdfor evaluator criteria.runs/for per-run evidence and handoff records.
feature_list.json
feature_list.json is the machine-readable feature scope and state.
Feature entries must be independently verifiable units of work. Planning Agents must split broad requirements when multiple user-visible behaviors, capability dependencies, implementation boundaries, risk domains, or verification surfaces are present. If a broad requirement is intentionally represented by one feature, the planning output must explain why it remains independently verifiable.
Each feature must include:
idstring, such asF001titlestringdescriptionstringacceptancearray of stringspassesbooleanstatusstring:todo,in_progress,done, orblockedattemptsintegerlast_errorstring- optional metadata such as
priority
Rules:
passes=truemeans complete.passes=falsemeans incomplete.statusis orchestration state and must not conflict withpasses.status=blockedmeans temporarily skipped after repeated failures.attemptsis incremented when the orchestrator starts a round for that feature.- Agents must not delete unknown fields.
- Agents must preserve unknown fields.
State Safety Rules
- Do not overwrite the entire
feature_list.jsonunnecessarily. - Update only the current feature during Coding Agent work.
- Preserve feature ordering and existing fields.
- Do not remove metadata fields.
- Do not reset existing fields such as
passes,status,attempts, orlast_errorunless explicitly instructed.
Spec Normalization
Planning Agents must normalize new requirements before appending feature entries. The normalized SPEC addition must define goal, included scope, excluded scope, core flows, constraints, ambiguities or assumptions, required capabilities, project-owned implementation paths, and verification surface.
If a requirement cannot be normalized without guessing, the Planning Agent must ask for clarification, record explicit assumptions, mark planning risks, or append capability, blocker, or follow-up features before product behavior work. Vague requirements must not become executable features merely because they sound actionable.
progress.md
progress.md must include:
- Current system status.
- Last completed feature.
- Next feature.
- Known issues.
- Recovery notes when useful.
The Coding Agent updates progress.md after implementation work.
Failure Improvement Loop
Failures must improve the harness when they reveal a weak loop.
When a feature fails, is blocked, or exposes repeated evaluator feedback:
- Record the concrete failure in
last_error. - Record run evidence in
runs/. - Assign one primary failure domain from
docs/failure-domains.md. - Assess whether the failure requires a harness improvement.
- Convert harness failures into durable changes such as docs, prompts, scripts, schemas, contract tests, smoke tests, or a new feature entry.
- If no harness improvement is required, record why the failure is only an implementation issue.
Repeated failures in the same domain must not remain only retries. They should become an explicit harness improvement feature or a committed harness rule/test change.
Capability Gap Handling
A missing tool, permission, generator, dependency, service, credential, runtime setting, CI resource, or verification fixture is a capability gap when it is required to implement or verify the selected feature durably.
Rules:
- Verify the gap with a primary source, real command output, official documentation, or captured logs before relying on it.
- Do not bypass a capability gap by hand-writing generated artifacts, weakening acceptance criteria, skipping verification, or making local-only environment changes that future agents and CI cannot reproduce.
- Resolve the gap with a durable project capability such as setup documentation, scripts, adapters, fixtures, CI configuration, dependency declarations, or tests.
- If the durable capability is out of scope for the selected feature, mark the feature blocked or append a follow-up feature instead of treating the workaround as completion.
- Temporary workarounds must be recorded in
runs/,progress.md, orlast_error, and the feature must not be marked complete until the capability is provided or explicitly scoped out.
Example Boundaries
Default examples are harness fixtures and teaching references. They are not the default implementation surface for project-level requirements.
Rules:
- Do not satisfy a project feature by modifying
examples/tiny-cli,examples/go-server, or another default example unless the selected feature explicitly targets that example. - Put project requirements in project-owned source, contract, documentation, and test paths; update
./init.shto verify those paths. - Use examples as references only. If fresh project setup removes or replaces default examples, record that as setup work and update verification accordingly.
- Evaluators must reject project-level work that passes only because an example was repurposed.
Project Recovery Init
init.sh is the root project recovery entry point. In hidden layout, .agent-harness/scripts/init.sh verifies harness health; root ./init.sh starts as a thin harness wrapper only before a project minspec exists.
Once a minspec is accepted, Planning Agents must add a runnable-skeleton feature that turns root ./init.sh into the project recovery contract. That contract must install or verify dependencies, start required services itself, run at least one real smoke test against an endpoint or core function, print clear logs, be idempotent, avoid manual steps and TODO placeholders, and exit non-zero on failure.
Harness verification alone must not be treated as project completion after the accepted scope requires runnable business behavior.
External Behavior Verification
When implementation depends on behavior outside this repository's own code, agents must verify that behavior before relying on it.
Examples include:
- CLI tools and their flags, stdin/stdout/stderr behavior, exit codes, signals, working directory, environment variables, and timeout behavior.
- Third-party APIs, webhooks, SDKs, protocol payloads, callback formats, and version-specific fields.
- Runtime and platform behavior such as process management, filesystem semantics, shell behavior, permissions, networking, deployment platforms, and operating-system differences.
- Model or tool output schemas, streamed event formats, JSONL event fields, and approval or permission protocols.
Rules:
- Do not infer unknown external behavior from intuition or local mocks.
- Prefer primary sources: official help output, official documentation, real minimal commands, real sample payloads, or captured logs from the target tool.
- Treat mocks and fake children as tests of this repository's state machine only; they do not prove the external tool or platform behaves that way.
- When changing process semantics such as argv, stdio, cwd, env, timeout, signal handling, or shell mode, verify the real command behavior or document why direct verification is not possible.
- When depending on structured output fields, verify with real-shaped output from the source and add regression tests using those captured shapes.
- If behavior remains uncertain, state the uncertainty explicitly in
SPEC.md,progress.md, or implementation notes, and choose the safer default.
External Tool Schema Rules
When implementing behavior that parses output from external tools such as Codex CLI JSONL, Claude JSON, Cursor logs, deployment CLIs, test runners, or webhook payloads:
- Do not invent or assume field names from naming convention alone.
- Prefer real captured local output or official documentation as fixtures.
- Add regression tests using real-shaped output for every trusted schema field.
- If the schema is unknown, fail closed instead of extracting identifiers from assistant prose, command output, source files, documentation, or log tails.
Work Rules
- Only one feature per Coding Agent run.
- Always keep the system runnable.
- Always run
./init.shbefore declaring success. - The Coding Agent updates state and progress for its target feature.
- The Evaluator Agent verifies without implementation changes.
- The orchestrator is the default entrypoint for one-feature implementation and evaluation.
- Manual Coding Agent work is an explicit fallback only when adapters are unavailable or the user requests it.
- The orchestrator owns unattended feature state transitions.
- From the evaluator-evidence baseline onward, do not mark a feature done unless
runs/containsEVAL_PASS: Fxxxevidence for that feature.
Commit Message Rules
When the user explicitly asks to commit approved feature work:
- Follow
docs/commit-messages.md. - Include the selected feature ID first in the commit subject using the format
Fxxx <Action> <concise summary>. - Verify every referenced feature ID exists in
feature_list.json. - Use batch feature IDs only when the user explicitly approved a batch commit.
- Use
No-feature: <summary>only for explicitly non-feature work.
Anti-Patterns
- Doing multiple features in one Coding Agent run.
- Collapsing unrelated or independently verifiable requirements into one over-bundled feature.
- Turning vague requirements into feature entries without spec normalization.
- Relying on previous chat instead of repository files.
- Skipping
./init.sh. - Leaving broken code.
- Bypassing missing required capabilities with hand-written generated code, skipped verification, weakened scope, or local-only environment changes.
- Treating harness verification as project recovery after a minspec requires a runnable skeleton.
- Implementing project-level requirements inside default examples instead of project-owned source and tests.
- Treating manual Coding Agent work as the default path when orchestrator adapters should be used.
- Silently falling back from orchestrator adapter failure to hand-edited feature completion.
- Coding Agent committing during orchestrated runs.
- Committing approved feature work without the feature ID in the commit subject.
- Marking a feature done without evaluator evidence in
runs/. - Evaluator Agent accepting incomplete work.
- Marking a feature done without evaluator pass.
Goal
Make the system:
- Recoverable at any time.
- Runnable at any time.
- Continuously improvable.
- Resistant to premature completion.
Verification
./init.sh is the default verification entry point. It must be deterministic and fail with a non-zero exit code when the repository is not in a usable state.
Feature-level validation uses:
scripts/validate-feature.sh F001
Optional orchestration preview uses:
python3 orchestrator.py --dry-run
Default one-feature work uses:
make work