deer-flow

Imported from bytedance/deer-flow (AGENTS.md). Install upstream with npx skills add bytedance/deer-flow. Copyright stays with the author.

AGENTS.md

This file provides guidance to AI coding agents (Claude Code, Codex, and others) when working with code in this repository. It is the source of truth; the sibling CLAUDE.md imports it via @AGENTS.md.

It is the monorepo orientation layer: it maps the whole repo and points to the module guides that own the depth. For anything inside a module, read that module's guide rather than expecting full detail here:

• backend/AGENTS.md — backend depth: harness/app split, agent & middleware chain, sandbox, MCP, skills, memory, IM channels, persistence/migrations, config system, test layout.
• frontend/AGENTS.md — frontend depth: Next.js App Router layout, thread/streaming data flow, code style, commands.

What is DeerFlow

DeerFlow is a LangGraph-based AI super-agent system with a full-stack architecture. The backend runs a "super agent" with sandboxed execution, persistent memory, subagent delegation, and extensible tools (built-in, MCP, community), all per-thread isolated. The frontend is a Next.js chat UI. External IM platforms (Feishu, Slack, Telegram, Discord, DingTalk) bridge into the same agent through the Gateway.

Service Topology

A single make dev / Docker stack runs four cooperating services:

Nginx is the single public entry: it proxies /api/* to the Gateway, rewriting /api/langgraph/* onto the Gateway's native routes, and serves the frontend — see backend/AGENTS.md for the runtime and router detail. It compresses HTML and configured textual assets, deliberately leaving SSE, fonts, images, audio, and video uncompressed at the proxy layer.

Both compose files publish that entry as "${BIND_HOST:-127.0.0.1}:${PORT:-2026}:2026" — loopback by default, matching the README's documented deployment model; a bare "${PORT}:2026" binds 0.0.0.0, which does not. The root PORT value is Docker ingress configuration only; local orchestration pins Next.js to 3000 so loading .env cannot make make dev wait on the wrong port. Nginx listening default_server on IPv4+IPv6 and the Gateway binding 0.0.0.0:8001 are container-internal on purpose: the published nginx port is the entire external surface. Any new published port needs an explicit bind address; backend/tests/test_compose_default_bind_host.py pins this for every service in both compose files.

Repository Map

deer-flow/
├── Makefile                        # Root orchestration: drives the full stack (dev/start/stop, docker, setup)
├── config.example.yaml             # Template → copy to config.yaml (gitignored) at repo root
├── extensions_config.example.json  # Template → copy to extensions_config.json (gitignored): MCP servers + skills
├── backend/                        # Python backend — see backend/AGENTS.md
│   ├── Makefile                    # Per-module backend commands (dev, gateway, test, lint, migrate-rev)
│   ├── extensions/sources/         # Deployable snapshots of locally installed Python extensions
│   ├── packages/extension-api/     # deerflow-extension-api package (import: deerflow_extension_api.*) — public extension contract
│   ├── packages/harness/           # deerflow-harness package (import: deerflow.*) — agent framework
│   └── app/                        # FastAPI Gateway + IM channels (import: app.*)
├── frontend/                       # Next.js frontend (pnpm) — see frontend/AGENTS.md
├── docker/                         # docker-compose files, nginx config, provisioner
├── skills/                         # Agent skills: public/ (committed), custom/ (gitignored)
│                                    # Managed integration skill packs are global at .deer-flow/integrations/skills/{provider}/
│                                    # Integration credentials and enabled state remain per-user
├── contracts/                      # Cross-component JSON contracts (e.g. subagent status, skill review)
├── examples/                       # Extension examples: deerflow-extension-{example,bookmarks}
├── scripts/                        # Root orchestration scripts invoked by the Makefile (check, configure, doctor, support_bundle, serve, nginx, docker, deploy, setup_wizard)
├── tests/                          # Root-level tests (currently tests/skills/ — public skill tests)
└── docs/                           # Cross-cutting docs, plans, and design notes

Third-party extensions are loaded from a top-level plugins: list in config.yaml (operator-controlled on purpose — that list causes code to be imported, so it is deliberately kept out of the API-writable extensions_config.json). Packaged extensions can contribute middleware, lifecycle observers, Gateway services, FastAPI HTTP routers, and experimental full-stack plugins. Manage them with deerflow extensions install/upgrade/list/enable/disable/remove or the root make extension-* wrappers. Every mutation requires a Gateway restart, and both build hooks and extension code execute with Gateway privileges, so only trusted operator sources belong in this path. The manager transaction, accepted source forms, lock discipline, and contribution contract live in the extensions guide; the user manual is frontend/src/content/{en,zh}/harness/extensions/.

Runtime config lives at the repo root: copy config.example.yaml → config.yaml (main app config) and extensions_config.example.json → extensions_config.json (MCP servers + skills). Both real files are gitignored and may be edited at runtime via the Gateway API. Config schema and resolution order are documented in backend/AGENTS.md.

Skill quality review note:

• skills/public/skill-reviewer/ is the built-in read-only skill quality reviewer. It uses the harness-layer review_skill_package tool and contracts in contracts/skill_review/. Model-visible review data is compact and tag-neutralized; full raw payloads stay in tool artifacts. See backend/AGENTS.md for the non-activation, SkillScan, and skill-creator ownership boundaries.
• CI waivers live in .github/skill-review-waivers.v1.json and are enforced by scripts/review_changed_public_skills.py. Pull requests may validate waiver edits from their head revision, but only the manifest from the trusted base revision can suppress that run. Entries match one error finding exactly, include the reviewed file's SHA-256 and an expiry date, remain visible in CI output, and can never waive blocker findings. An entry may also preapprove future full-file SHA-256 values, effective only once the manifest change lands in the trusted base — so relying on a waiver takes two merges: the manifest first, the skill change after, then promote the consumed hash to file_sha256 in a follow-up cleanup.

Scheduled-task note:

• The scheduled-task MVP adds a workspace page at /workspace/scheduled-tasks plus a background scheduler service gated by config.yaml -> scheduler.enabled.
• Scheduled background runs are intentionally non-interactive: the lead-agent toolset excludes ask_clarification when context.non_interactive=true. That key, disable_clarification, and github_token are honored only for internally-authenticated callers; client-supplied copies are dropped from both body.context and body.config.
• Busy scheduled occurrences are persisted as queued; launching is a short lease-fenced claim, running remains the normal Gateway run lifecycle, and scheduler.queue_timeout_seconds bounds the durable wait. Do not reintroduce skip-on-overlap or count waiting rows against max_concurrent_runs.

Commands: Root vs. Module

Root make targets drive the whole stack (run from the repo root):

make setup       # Interactive setup wizard (recommended for new users)
make doctor      # Check configuration and system requirements
make support-bundle  # Generate redacted troubleshooting summary, AI issue draft, and optional zip
make config      # Generate local config files from the examples
make check       # Check that required tools are installed
make install     # Install all dependencies (frontend + backend + pre-commit hooks)
make extension-install SOURCE=...  # Install and enable a trusted Python extension
make extension-upgrade SOURCE=...  # Replace an installed extension and keep its config
make extension-list                # List configured Python extensions
make extension-enable NAME=...     # Enable an installed extension (restart required)
make extension-disable NAME=...    # Disable without uninstalling (restart required)
make extension-remove NAME=...     # Remove package and config entry (restart required)
make dev         # Start all services with hot-reload (Gateway + Frontend + Nginx)
make start       # Start all services in production mode (local, optimized); SKIP_FRONTEND_BUILD=1 reuses the last frontend build
make stop        # Stop all running services
make up / down   # Build/stop the production Docker stack (browser at localhost:2026)
make docker-start / docker-stop / docker-logs   # Docker development environment

Production startup uses the image's pre-built Python environment with uv run --no-sync, gives the Gateway a real /health probe, and makes make up wait for that probe before printing its success banner. A readiness failure must surface Compose status and recent Gateway logs instead of claiming the stack is running.

Docker log and restart commands resolve DEER_FLOW_ROOT from the current checkout before invoking Compose, matching the start and stop commands.

Run make help for the full list.

Per-module commands drive a single module (run inside that module):

# Backend (see backend/AGENTS.md for the full set)
cd backend && make dev        # Gateway API with reload (port 8001)
cd backend && make test       # Default backend suite; excludes live and blocking-I/O tests
cd backend && make test-blocking-io  # Strict blocking-I/O suite
cd backend && make lint       # ruff check
cd backend && make format     # ruff format

# Frontend (see frontend/AGENTS.md for the full set)
cd frontend && pnpm dev       # Dev server: Webpack by default (override with DEER_FLOW_DEV_BUNDLER=turbo)
cd frontend && pnpm check     # Lint + type check (run before committing)
cd frontend && pnpm test      # Unit tests

Rule of thumb: root make = the full application; backend/Makefile and frontend/ (pnpm) = per-module work.

Host pnpm calls use scripts/pnpm.py: native Windows tries pnpm.cmd before pnpm; POSIX reverses the order. Its Corepack fallback applies the same ordering to corepack.cmd and corepack. The runner operates from frontend/ so Corepack honors its pinned package-manager version.

Prerequisites before make dev

make dev does not generate config files. First-time setup order:

make config      # copy config.example.yaml -> config.yaml and extensions_config.example.json -> extensions_config.json (both gitignored)
make install     # install frontend + backend deps and pre-commit hooks
make dev         # then start everything

Without config.yaml present, services fail to boot. config.yaml / extensions_config.json may be edited at runtime via the Gateway API but are gitignored, so never commit them.

Run a single test

# Backend (pytest); run one file or one test function
cd backend && python -m pytest tests/test_compose_default_bind_host.py -q
cd backend && python -m pytest tests/path/to/test.py::test_func -q

# Frontend (rstest)
cd frontend && pnpm rstest run <pattern>     # e.g. pnpm rstest run my-component

Logs

• Docker stack: make docker-logs (or docker compose -f docker/... logs -f <svc>).
• Local make dev: each service logs to its own terminal pane. Frontend dev-server errors surface in the browser console at localhost:3000; backend tracebacks appear in the Gateway terminal.

Where to Go Next

• Backend work → backend/AGENTS.md
• Frontend work → frontend/AGENTS.md
• Setup & install → Install.md, CONTRIBUTING.md
• Project overview & usage → README.md (translations: README_zh.md, README_ja.md, README_fr.md, README_ru.md)
• Security policy → SECURITY.md
• Changes → CHANGELOG.md
• Cutting a release → RELEASING.md

Cross-Cutting Conventions

These apply repo-wide; module guides own the module-specific detail.

• Documentation update policy — keep docs in sync with code: update README.md for user-facing changes and the relevant AGENTS.md for development/architecture changes in the same change set.
• Test-driven development — features and bug fixes ship with tests. Backend tests live in backend/tests/ (TDD is mandatory there; see backend/AGENTS.md); frontend tests live in frontend/tests/.
• Format before pushing — run make format (backend) / pnpm check (frontend). Backend CI enforces ruff format --check, so formatting must be clean before a push.
• Skill text encoding — treat SKILL.md and other textual skill resources as UTF-8; Python utilities that read or write them must pass encoding="utf-8" rather than relying on the platform locale.
• Version sources must stay in lockstep — a release version must match identically in backend/pyproject.toml, frontend/package.json, and deploy/helm/deer-flow/Chart.yaml (version + appVersion). Pushing a v* git tag triggers CI that runs scripts/verify_versions.sh and blocks all publishing if any source drifts. Before bumping a version, run scripts/bump_version.sh <ver> (aligns all four at once) and scripts/verify_versions.sh <ver> to catch drift early. See RELEASING.md.
• Don't edit CLAUDE.md — it only contains @AGENTS.md. All agent guidance changes belong here in AGENTS.md; CLAUDE.md is a thin import shim.