Imported from SemperSupra/WineBot (AGENTS.md). Install upstream with npx skills add SemperSupra/WineBot. Copyright stays with the author.

WineBot Agent Map

This document is a navigation aid for autonomous agents working on the WineBot codebase.

1. System Architecture

WineBot is a containerized Windows application runtime (Wine 10.0) with an X11 display stack, controlled via a Python FastAPI.

Core Layers

2. Key File Map

3. Environment Variables

4. Common Tasks

Docker runtime on this host

Docker Desktop has been removed. Docker Engine v29.6.1 runs inside WSL2 Ubuntu; use explicit WSL commands from Windows:

wsl -d Ubuntu docker <args>
wsl -d Ubuntu docker compose <args>

Do not run bare docker from Windows PowerShell unless you know the user's PowerShell profile loaded the local proxy functions. Use docker compose, not docker-compose. See docs/DOCKER_ENGINE_ON_WSL2.md.

How to run tests?

# Rapid local feedback (Watch mode)
./scripts/bin/dev-watch.sh

# UI/UX Policy Compliance
wsl -d Ubuntu docker compose -f compose/docker-compose.yml --profile interactive --profile test run --rm test-runner pytest tests/e2e/test_ux_quality.py

# Unit tests
wsl -d Ubuntu docker compose -f compose/docker-compose.yml --profile interactive --profile test run --rm test-runner scripts/ci/test.sh

How to apply config changes?

# 1. Edit config
scripts/winebotctl config set KEY VALUE

# 2. Apply (Restarts container)
scripts/winebotctl config apply

How to trace input?

scripts/winebotctl input trace start --layer windows
scripts/winebotctl input trace events --source client --limit 50

How to debug input issues?

1. Enable traces: WINEBOT_INPUT_TRACE=1 etc.
2. Check logs/input_events_*.jsonl in session dir.
3. Run scripts/diagnose-input-suite.sh inside container.

5. Programmatic Interaction

Agents should use the following API patterns for reliable control.

Automated Input

POST /input/mouse/click

Performs a mouse click at specific coordinates.

Payload:

{
  "x": 100,
  "y": 100,
  "button": 1,
  "window_title": "Notepad",
  "relative": true
}

Features:

• Validation: Clicks are validated against the current SCREEN resolution to prevent out-of-bounds errors.
• Window Targeting: Providing window_title or window_id logs the target for better traceability.
• Relative Clicking: If relative: true, coordinates are calculated relative to the specified window's top-left corner.
• Non-blocking: The call is asynchronous and will not stall the system during execution.

Health & Discovery

• GET /health: Use this to verify system readiness. Check security_warning for potential exposure.
• GET /health/invariants: Use this to verify runtime lifecycle/control/config invariants.
• mDNS Discovery: WineBot broadcasts _winebot-session._tcp.local.. Agents on the same network can discover instances automatically.

6. Responsible Automation (Agent Ethics)

To ensure system stability and reliability, agents must adhere to the following constraints:

1. Avoid UI Feedback Loops: Do not programmatically click on transient UI elements like Toast notifications or status badges. This can lead to non-deterministic state transitions.
2. Action Throttling: Enforce a minimum "Politeness" delay of at least 100ms between discrete API actions (e.g., clicks or keypresses) to allow the Wine/X11 stack to settle.
3. Graceful Termination: Always attempt to call POST /lifecycle/shutdown before exiting to ensure video artifacts are finalized and resources are reaped.
4. Least Privilege: Do not attempt to modify files outside of /wineprefix or /artifacts. The apps and automation directories are mounted as Read-Only for safety.

7. Diagnosing Input Issues

Is the input pipeline healthy?

# Check keyboard injection works
scripts/winebotctl input key "test" --window-title "Notepad"

# Check trace layers
scripts/winebotctl input trace status --layer x11
scripts/winebotctl input trace status --layer windows

# Run full diagnostic suite (requires interactive mode)
scripts/diagnostics/diagnose-input-suite.sh

# Run 5-layer trace bisect
scripts/diagnostics/diagnose-input-trace.sh --layers x11,windows

# Analyze keyboard latency
python3 scripts/diagnostics/analyze-trace-latency.py --mode keyboard

Keyboard events not reaching the app?

The Wine desktop shell (explorer.exe /desktop) intercepts X11 keyboard events. The /input/key endpoint uses AHK Send by default, bypassing this barrier. Verify the backend in the response: {"backend": "ahk", "status": "sent"}.

If key events arrive at X11 but not Windows:

# Enable Windows trace
POST /input/trace/windows/start
# Send a test key
POST /input/key {"keys": "Test", "window_title": "Notepad"}
# Query Windows trace for the key
GET /input/events?source=windows&origin=agent&limit=50
# Look for key_down/key_up events with matching trace_id

Input latency investigation

# Start all trace layers
scripts/diagnostics/diagnose-input-trace.sh --layers x11,windows
# Send several keystrokes
scripts/winebotctl input key "test1"
scripts/winebotctl input key "test2"
# Analyze latency
python3 scripts/diagnostics/analyze-trace-latency.py --mode keyboard

See docs/tracing.md for the full trace event schema and cross-layer correlation guide.

8. Window Identification Systems

WineBot has two distinct window ID systems that do not map to each other:

Rule: When calling /input/key with a window_title, the AHK backend uses AHK's native WinActivate/WinWaitActive with the title string — NOT the X11 ID. This is correct because X11 window IDs are not AHK HWNDs and cannot be used with ahk_id.

When calling /input/mouse/click, the xdotool backend uses X11 window IDs, which ARE correct for X11-level operations.

To discover window titles:

GET /health/windows   # Returns X11 IDs and titles for all windows

To target a specific window in a script, use its title (visible in the window list), not its numeric ID. The API resolves the title to the correct system internally.

Cross-system coordination

In scripts and agents that use both mouse and keyboard:

1. Get window list via GET /health/windows — titles are the source of truth
2. Mouse clicks: Use window_title from the window list (xdotool resolves to X11 ID)
3. Keyboard input: Use the same window_title (AHK resolves by Wine title matching)
4. Dialogs: After launching a dialog (Save As, Open), switch window_title to the dialog's title. Example: "Save As" not "Notepad" after Ctrl+S opens the save dialog.

See docs/known-limitations.md for the full catalog of platform constraints including comdlg32 dialog keyboard limitations, AHK Send character escaping rules, /run/python Linux vs Windows behavior, and tool selection guidance.