gremlins

Imported from xbrianh/gremlins (gremlins/AGENTS.md). Install upstream with npx skills add xbrianh/gremlins --skill gremlins. Copyright stays with the author.

gremlins/

Orchestration package for background gremlins. Owns the plan / implement / review / address pipelines, the fleet manager (fleet/), the chain-step decision agent (handoff.py), and the launcher (launcher.py).

Module layout

• cli/ — subcommand entry points. __init__.py is the top-level dispatch; one file per subcommand group: launch.py, resume.py, fleet.py. Bare invocation prints fleet status.
• spawn/pipeline.py — python -m gremlins.spawn.pipeline <gremlin_id> <pipeline_path> [args...]. Spawned by the launcher; wraps executor.run.run_pipeline and writes terminal state on exit.
• spawn/child.py — python -m gremlins.spawn.child <spec_path>. Spawned by the parallel runner to run a single stage in a fresh process (lands with #690).
• runner.py — run_stages sequencer (with resume_from) + SIGINT/SIGTERM handlers that reap model subprocess children.
• state.py — session-dir resolution, set_stage / write_bail_file / patch_state.
• utils/git.py — setup_workdir / stage_gremlins_overlay (overlay staging around a worktree). The git operations themselves live in the native _gremlins_core.utils.git module.
• fleet/ — fleet manager package: status listing + stop / land / close / rm / log ops. See fleet/AGENTS.md for the per-module breakdown.
• clients/protocol.py — CompletedRun dataclass.
• clients/stream.py — stream_events + _emit_event (stream-json parser and stderr renderer).
• pipeline/ — Pipeline dataclass + Pipeline.from_yaml(path) classmethod; resolve_pipeline_path; supports parallel stage groups. pipeline/loader.py holds STAGE_TYPES, the explicit dispatch table mapping type-name strings to Stage classes. YAML expansion (resolving include:, prompt:, and type: <name> macros) is handled by the Rust expand_pipeline in _gremlins_core.schemas.
• pipelines/ — bundled YAML pipeline files (local.yaml, gh.yaml); lookup target for resolve_pipeline_path.
• stages/composite.py — StageAttrs (common stage attributes) + get_client_from_dict + child_state for composite stages.
• stages/ — per-stage bodies: plan, review_code, github_address_pull_request_reviews, verify, github_wait_copilot, github_wait_ci, handoff.
• executor/run.py — run_main. Drives the local pipeline.
• executor/pipeline.py — StageRunner. Sequences stages for a pipeline run.
• prompts/ — externalized prompt templates (plan, implement, review lenses, etc).

Conventions

Reuse gremlins/utils/

Before shelling out to subprocess, git, or gh, check gremlins/utils/:

• utils/proc.py — run, run_or_raise, run_async, etc. Use instead of subprocess.run.
• utils/git.py — setup_workdir / stage_gremlins_overlay. For git operations (in_git_repo, head_sha, current_branch, worktree helpers, …) import from _gremlins_core.utils.git. Use these instead of shelling git directly.

GitHub calls belong in exec/shell stages (YAML cmds:), not in Python. Add reusable GitHub shell interactions to gremlins/recipes/stages/ rather than Python. For non-GitHub helpers, add to utils/ rather than duplicating subprocess plumbing in the consumer.

No module-level globals or registration side-effects

Do not introduce new module-level mutable state or register_* side-effect APIs. Pass dependencies into constructors instead. A new registry that needs extension should take its resolver map as a constructor argument, not expose a module-level register_x() that mutates a global dict.

No speculative plugin hooks

Don't add extension points (registries, plugin loaders, scheme-resolver maps) for hypothetical second consumers. Hand-curate the built-in set; generalize only when a real second user lands. Three concrete lines beat a premature plugin API.

Stage definitions (stage-definitions:)

A top-level stage-definitions: map in a pipeline YAML lets you name a reusable stage dict, scoped to that file. Call it with type: <name> just like any primitive.

stage-definitions:
  normalize:
    type: exec
    options:
      cmds: ["ruff format .", "git add -A", "git diff --cached --quiet || git commit -m 'normalize'"]

stages:
  - { type: plan, ... }
  - name: post-plan-normalize
    type: normalize
    out: { post-plan-commits: git://range }
  - { type: implement, ... }
  - name: post-implement-normalize
    type: normalize
    out: { post-implement-commits: git://range }

Resolution order for an unknown type: value:

1. STAGE_TYPES (primitives and container types in pipeline/loader.py)
2. Inline stage-definitions: in the current file
3. Discoverable pipeline files (resolve_pipeline_name) — expands the same way as include:

Call-site-owns-IO rule: definitions must not declare out: keys. Each call site declares its own out: (and in:, name:), exactly as if writing a bare primitive stage. The preprocessor merges name, in, and out from the call site onto the definition's stage dict. For multi-primitive definitions (those with a stages: list, e.g. implement), the call-site in: is applied to the first inner stage and out: to the last inner stage.

No parameters yet: definitions contribute only fixed fields (type, options, prompt, etc.). There is no mechanism to pass options into a definition from the call site; add that only when a real use case demands it.

Entry points

Testability seam: Client

Every stage that invokes a model takes an injected client through the Client class (in clients/client.py). Production code passes Client.parse("openai:gpt-4o") or Client("openai", "gpt-4o"); tests pass FakeClient(fixtures={label: <jsonl-or-list>}). The fake records each run(...) call into self.calls for assertion. Never have a stage spawn a model subprocess directly — go through the injected client so tests can intercept.

FakeClient looks fixtures up by label. Stages that re-enter the same logical step within one process (e.g. resumed implement) must use distinct labels per phase.

Byte-stable strings — DO NOT change

These values are persisted to state.json files and read by other writers (session-summary.sh hook, liveness.sh sourced from session-summary.sh, the fleet manager that inlines an equivalent classifier in fleet/state.py, the launcher). Renaming any of them silently breaks cross-process consumers. Source of truth: the bail-class tokens are parsed from the BAIL: <class>: <detail> marker by stages/agent.rs; stage-name vocab is defined in the pipeline YAML.

• Bail classes (state.json.bail_class): reviewer_requested_changes, security, secrets, other. Convention tokens only — no Python constant enumerates them.
• Stage names (state.json.stage): stable within a pipeline definition. The authoritative list for any pipeline is its YAML file. resolve_pipeline_path checks .gremlins/pipelines/<name>.yaml (project-scoped) first, then bundled gremlins/pipelines/<name>.yaml; --pipeline accepts either a bare name (resolved this way) or a direct path.

Recovering from a child bail

When a child bails in a boss chain, the boss halts. The operator must put the child into a well-defined state, then resume the boss. The boss reads only the child's state.json to decide what to do — no gh calls, no git inspection.

Child states the boss recognizes

The three operator commands

• gremlins resume <child-id> — re-spawn the bailed child at its bailed stage. Use after pushing a fix to the PR or editing the worktree.
• gremlins ack <child-id> — assert the child's work is already in main. Writes external_outcome=landed. Use after manually merging the child's PR.
• gremlins skip <child-id> — give up on the child's work. Writes external_outcome=abandoned. Use when the child's plan was wrong and you want the handoff agent to plan something different.

Operator recovery flows

# Keep this child going: address the PR review manually, then:
gremlins resume <child-id>
gremlins resume <boss-id>

# Manually merged the PR:
gh pr merge <pr> --squash
gremlins ack <child-id>
gremlins resume <boss-id>

# Give up on this child's work, re-plan:
gremlins skip <child-id>
gremlins resume <boss-id>

Two commands per recovery. If the boss was resumed with no operator decision recorded, it prints the three options above and exits non-zero — it never silently re-handoffs and spawns a near-duplicate child.

Stage and bail bookkeeping

state.set_stage writes stage info to state.json atomically via patch_state. state.write_bail_file writes bail_{attempt}.json to the state dir. When a stage detects a recorded bail (via state.json), it raises a Bail exception. Both helpers no-op without GREMLINS_GREMLIN_ID and never raise — stage / bail bookkeeping must not crash a running gremlin.

Tests

uv pip install -e ".[dev]"
python -m pytest

make test runs the same thing. Tests live at the top-level ./tests/ (sibling to this package), discovered via [tool.pytest.ini_options] testpaths = ["tests"] in the repo's pyproject.toml.