askplanner

Imported from guo-shaoge/askplanner (AGENTS.md). Install upstream with npx skills add guo-shaoge/askplanner. Copyright stays with the author.

askplanner_v2

Go relay for TiDB SQL query tuning. Receives questions (CLI or Lark bot) → forwards to Codex CLI → returns answer. This project handles session management, prompt loading, and I/O; all reasoning happens inside Codex CLI.

Architecture

cmd/askplanner (CLI REPL)  ─┐
cmd/larkbot (bootstrap)    ─┤
cmd/askplanner_usage       ─┤
cmd/askplanner_migrate_userdata ─┤
                             ├→ internal/larkbot/app (Feishu bot app lifecycle)
                             │       → internal/larkbot/handler (message routing)
                             │       → internal/larkbot/thread_context (topic history prefetch for new sessions)
                             │       → internal/attachments (user file library)
                             │       → internal/clinic (slow query prefetch)
                             │       → internal/workspace (per-user repo workspace)
                             ├→ internal/codex/responder (session mgmt)
                             │       → internal/codex/runner (exec codex CLI)
                             │            → codex exec ... (external binary)
                             │                 → answer (reply file or JSON stdout)
                             ├→ internal/usage (dashboard collector, question event store, HTTP pages/APIs)

Key Files

contrib/ Submodules

Codex CLI WorkDir = project root, so it reads contrib/ via shell commands (rg, cat, etc.).

Build & Run

make all          # build cli, larkbot, usage dashboard, and migrate-userdata tool
make larkbot      # larkbot only
make migrate-userdata
make fmt          # go fmt ./...
make lint         # run golangci-lint with the pinned repo version

Requires: Go 1.23+, codex CLI in PATH, git submodules initialized.

Lint

make lint

Lint uses golangci-lint via go run github.com/golangci/golangci-lint/v2/cmd/golangci-lint@v2.11.4, so contributors do not need a separate global install.

Environment Variables

Workspace

Each Lark bot user gets an isolated workspace so Codex CLI can explore TiDB source, docs, and skills independently per user. The workspace subsystem uses shared git bare mirrors plus per-user git worktrees to avoid cloning the full repositories for every user.

Directory Layout

<WORKSPACE_ROOT>/                        (default: .askplanner/workspaces)
├── mirrors/                             # shared bare mirrors (all users share these)
│   ├── tidb.git                         #   git clone --mirror of TiDB
│   ├── agent-rules.git                  #   git clone --mirror of agent-rules
│   └── tidb-docs.git                    #   git clone --mirror of tidb-docs
├── users/<sanitized_user_key>/
│   ├── root/                            # Codex CLI WorkDir for this user
│   │   ├── contrib/
│   │   │   ├── tidb/                    #   git worktree → mirrors/tidb.git
│   │   │   ├── agent-rules/             #   git worktree → mirrors/agent-rules.git
│   │   │   └── tidb-docs/               #   git worktree → mirrors/tidb-docs.git
│   │   ├── user-files → <uploads>/<key> #   symlink to user's uploaded attachments
│   │   └── clinic-files → <clinic>/<key>#   symlink to user's Clinic snapshots
│   └── data/
│       └── workspace.json               #   metadata: refs, SHAs, env hash, last active time
├── locks/                               # per-user flock files
│   ├── <user_key>.lock                  #   exclusive lock for workspace mutations
│   └── gc.lock                          #   GC sweep lock
└── .trash/

Key Operations

agent-rules mirror is also synced on a separate background timer (AGENT_RULES_SYNC_INTERVAL_MIN), and its worktrees track the latest default branch automatically (TrackLatest=true).

Concurrency

All workspace mutations (SwitchRepo, Sync, Reset) and reads (Ensure) acquire a per-user exclusive flock (locks/<user_key>.lock). This serializes all operations for the same user. GC Sweep uses a non-blocking lock and skips users whose lock is held.

Environment Hash

computeEnvironmentHash() produces a SHA256 from the workspace root path and all repo states (name|requestedRef|resolvedSHA|trackingLatest). This hash is stored in workspace.json and attached to every Codex session record.

When the hash changes (e.g., user switches branches), canResume() in the responder returns false with reason "environment_changed", forcing a new Codex session. This guarantees the AI never continues a conversation based on stale source code.

/ws Commands

/ws status                              # show current workspace state
/ws switch <repo> <ref> [-- question]   # switch repo to a branch/tag/SHA, optionally ask a question
/ws sync [repo|all]                     # pull latest and re-checkout
/ws reset [repo|all]                    # revert to default branch

Session Management

• Keys: cli:default (CLI), lark:thread:* / lark:chat:*:user:* (bot)
• Resume if: same prompt hash, same work dir, same environment hash, turns < max, TTL not expired
• For Lark topic messages with non-empty thread_id, the relay prefetches earlier visible thread messages and injects them only into the initial prompt of a new bot session; resume prompts do not repeat thread history
• On resume failure: auto-starts new session with last 6 turns as context
• Editing prompt invalidates all sessions (hash changes)

User Data Migration

Use bin/askplanner_migrate_userdata when you need to move persisted askplanner state to a new machine, a new deployment root, or a backup directory.

make migrate-userdata
./bin/askplanner_migrate_userdata --source .askplanner /path/to/askplanner-backup

Behavior:

• Copies the askplanner data directory tree to the destination.
• Preserves regular files such as sessions.json, usage_questions.jsonl, logs, attachment files, Clinic snapshots, and workspace metadata.
• Rewrites absolute workspace symlinks like user-files and clinic-files so they point at the migrated destination tree.
• Skips managed git data under workspaces/mirrors/.
• Skips per-user managed repo worktrees under workspaces/users/*/root/contrib/{tidb,tidb-docs,agent-rules} because those can be recreated by Ensure//ws sync.

Operational notes:

• Stop the running askplanner processes before migrating so session, log, and usage files are not being written during the copy.
• The destination must be different from the source and cannot be nested inside the source directory.
• After migration, start askplanner with the migrated paths, for example by pointing CODEX_SESSION_STORE, USAGE_QUESTIONS_PATH, LOG_FILE, and WORKSPACE_ROOT at the new root if you are not using the default .askplanner layout.

Usage Dashboard (Agent Fast Path)

Core principle

The usage tool is a local observability layer with two metric types:

• Cumulative metrics (users/questions, top users, question details) from append-only question events (USAGE_QUESTIONS_PATH).
• Snapshot/recent metrics (active sessions, recent requests/errors, workspace/session breakdown) from sessions.json, workspace metadata, and log tail.

Do not treat all cards as the same source of truth. Cumulative and snapshot metrics are intentionally mixed on the same page.

Main entrypoints and call flow

Runtime entrypoints:

• Dashboard server bootstrap: cmd/askplanner_usage/main.go
• HTTP handlers/pages: internal/usage/http.go
• Aggregation and pagination: internal/usage/collector.go
• Event write path: internal/usage/events.go

Question event write flow:

1. CLI/Lark question handling starts a span via QuestionTracker.Begin(...).
2. The span is finalized as success, short_circuit, or error.
3. Finalized events are appended to usage_questions.jsonl.
4. Append failures are non-fatal (must not break answer path).

Key data structures

internal/usage/events.go:

• QuestionEvent
  • Identity: event_id
  • Time/source identity: asked_at, source, user_key, conversation_key
  • Payload: question, status, duration_ms, model, error
  • Provenance: workspace_env_hash, question_fingerprint
• QuestionTracker / QuestionSpan
  • Tracks one logical user question lifecycle.
  • Guarantees at-most-one append per span.
• QuestionStore
  • JSONL append + load.
  • Uses file lock to serialize writes.

internal/usage/collector.go:

• Snapshot
  • Homepage payload (/api/usage): summary, breakdown arrays, trend arrays, top_users, recent tables.
• Summary
  • Includes total_users, total_questions, active_users_24_hours, active_users_7_days, etc.
• UserSummary
  • Per-user aggregate: cumulative + 24h/7d counts, last question/time.
• QuestionsPage / UsersPage
  • Paginated API responses for /api/questions and /api/users.

API and filters

Pages:

• /: dashboard
• /questions: detailed paginated question list

JSON APIs:

• /api/usage: aggregated dashboard snapshot
• /api/users?page=&page_size=: paginated user aggregates
• /api/questions?page=&page_size=&user_key=&source=&status=&q=&from=&to=: paginated question events

Filter semantics:

• from / to: date-based bounds (YYYY-MM-DD).
• source: normalized to cli|lark|other.
• status: success|short_circuit|error.
• q: case-insensitive substring search on question/user/conversation key.

Semantics and caveats

• usage_questions.jsonl is the source of truth for cumulative user/question metrics.
• total_users is distinct user_key count from question events (not session count).
• CLI traffic is normalized as virtual user cli:default.
• Recent request/error trend cards come from log tail parsing, not from question events.

Codex CLI Invocation

# new
codex exec --sandbox read-only --json --model <model> -c model_reasoning_effort="medium" -o <reply> - < <prompt>
# resume
codex exec resume --json --model <model> -c model_reasoning_effort="medium" -o <reply> <session_id> - < <prompt>

Answer: read from reply file (-o), fallback to final_answer in JSON stdout.

Conventions

• Module: lab/askplanner
• Standard log package, env-var-only config, no external deps beyond Lark SDK
• All paths relative to project root
• Before finishing changes, run make fmt and keep make lint clean