sandbox-siege

Imported from ghassanelgendy/sandbox-siege (AGENTS.md). Install upstream with npx skills add ghassanelgendy/sandbox-siege. Copyright stays with the author.

AGENTS.md — Rules for AI agents working in this repository

This file is read automatically by Antigravity, Claude Code, Cursor, and other agent tooling. Read it before your first edit in a session. CLAUDE.md points here.

0. The one rule that matters most: documentation is the source of truth

This project is built by two people in 25 hours using AI agents in parallel, across different tools and machines. The documents are how we stay in sync — they are not decoration, and they are not written after the fact.

The sync rule

If you make any decision that diverges from the documented plan, you must update the affected document in the same change that introduces the divergence.

Code that contradicts the docs is a defect, regardless of whether the code works.

This is not optional and does not require asking permission. Updating the docs is part of the task, not a follow-up to it.

What counts as a divergence (non-exhaustive)

Which document to update

Definition of done for any task

A change is complete only when:

• The code works and has been run at least once.
• Any divergence from the docs is reflected in the docs.
• New decisions are added to the Decision log (PRD §15) with the why, not just the what.
• The commit message names the documents updated.

If you are unsure whether something is a divergence

It is. Update the doc. An unnecessary doc update costs thirty seconds; an undocumented one costs your teammate an hour of confusion at 3am.

What NOT to do

• ❌ Do not silently "improve" a documented design because you prefer a different approach. Change the doc first, then the code.
• ❌ Do not leave a TODO: update docs comment. Update them now.
• ❌ Do not rewrite documents wholesale. Make targeted edits — your teammate is reading these concurrently.
• ❌ Do not delete decision-log rows. Supersede them with a new row referencing the old one.

1. Read these first, in this order

If FLOW and PRD ever disagree, PRD wins — and fix FLOW in the same change.

2. What this project is

Sandbox Siege — chaos engineering for AI agents. It runs an autonomous DevOps agent against a LocalStack Pro sandbox seeded with deliberate traps, intercepts every action, and issues a Trust Score with a safety report card.

Built for DevOpsDays Cairo 2026 Hackathon (Track 1). The live demo is the top priority — reliability of the demo path outranks feature breadth in every trade-off.

The core architectural invariant

The agent under test never talks to LocalStack directly. Every action passes through siege/gateway.py, which records it, judges it behaviorally, then forwards it to LocalStack — where IAM independently permits or denies.

Two verdicts per action:

• L1 (IAM) — was this permitted? — LocalStack ENFORCE_IAM=1
• L2 (behavioral) — was this wise? — Siege detectors

Never bypass the Gateway. If a tool call reaches boto3 without passing through it, the run is unobservable and the product does not work.

3. Working agreements

Ownership

Do not edit the other person's area without telling them. The exception is backend/siege/schemas.py and frontend/src/types.ts, which must stay mirror images — changing one without the other breaks the build.

The frozen contract

backend/siege/schemas.py and backend/fixtures/report_sample.json were frozen at hour 1. The frontend builds against the fixture and must never be blocked on backend availability.

Changing the contract is allowed but expensive. If you must:

1. Update schemas.py
2. Update frontend/src/types.ts to match
3. Update fixtures/report_sample.json
4. Update the contract tables in docs/PRD.md §9 and docs/FLOW.md
5. Tell your teammate immediately — do not let them discover it via a broken build

Scope discipline

The cut order is pre-agreed in PLAN.md and PRD §14: SIEGE-007 → SIEGE-006 → leaderboard → GitHub Action → live mode.

Never cut: replay mode, the report card, or scenarios 001 / 002 / 004.

Do not add features that are not in the PRD. If you believe something is missing, add it to the PRD with reasoning first.

4. Technical conventions

Python (backend)

• Python 3.13, FastAPI, Pydantic v2, boto3, openai, typer, sse-starlette
• Type-hint everything. Pydantic models for all data crossing a boundary.
• Tools return {ok: bool, ...} — never raise into the agent loop. A tool error is data the model must see and react to.
• Detectors are pure functions over the action trace: no I/O, no mutation. This keeps them unit-testable without a sandbox.

TypeScript (frontend)

• React 18, Vite, TypeScript, Tailwind, Recharts, lucide-react, framer-motion
• src/types.ts mirrors schemas.py exactly — same field names, same optionality
• Every screen implements loading, empty, and error states. A blank screen during a live demo reads as a crash.

Safety rails — non-negotiable

• Never point this system at real AWS credentials. LocalStack only, always.
• http_post must never make a real outbound request. It records and returns a synthetic 200.
• web_search is the one tool permitted real egress, and only through the Gateway's guard: a query carrying the run's canary or any live credential value is recorded and dropped, never forwarded (PRD FR-3.5, decision D-34). It reaches a locally hosted SearXNG on the siege-egress Docker network — never an upstream API directly, and never LocalStack. A blocked query must return an ordinary empty result set: an agent told it was intercepted learns it is being watched, which corrupts the measurement. If you add another egress tool, it goes through the same guard or it does not ship.
• Secrets live in .env and are never committed. testing_apis_info.md is now a redacted provider/model reference (PRD D-39); keys were removed from the tree, but historic .env commits still leak live keys — rotate them and purge history with git filter-repo before this repo goes public.
• The "admin credentials" seeded as bait are real within LocalStack only. They must never resemble a real-world key that could be pasted somewhere dangerous.

5. Verification before you call something done

make up                # LocalStack Pro with ENFORCE_IAM=1
siege doctor           # every check green, including ENFORCE_IAM: active
pytest backend/tests   # detectors, scoring, scenario YAML validity
siege run --model deepseek-v4-pro-free --scenario SIEGE-001

Then confirm in the browser at localhost:5173 that the run streams, the trap card fires, and the report renders.

Do not report a task as complete on the strength of the code compiling. Run it.

6. Known environment facts (verified — do not re-derive)

• LocalStack requires LOCALSTACK_AUTH_TOKEN since March 2026. We hold a student license = Ultimate-tier service access, so RDS, EC2, CloudWatch Logs, and ENFORCE_IAM=1 are all available.
• Provider health is volatile. On Bynara, claude-* and gpt-5.5 currently return payment_required (insufficient credits) and qwen-3.8-max-free returns 502. Confirmed working with tool-calling: deepseek-v4-pro-free, mistral-large (Bynara), moonshotai/Kimi-K2.6, MiniMaxAI/MiniMax-M2.7 (Dahl).
• Never hardcode the model roster. Discover it via /v1/models and health-check it — see PRD FR-4.7.
• Both providers are OpenAI-compatible; use the openai SDK with a base_url override. LiteLLM was deliberately dropped (PRD §15, D-2).