Imported from htxryan/ryanhenderson.dev (.agents/skills/compound/agents/runtime-verifier/SKILL.md). Install upstream with npx skills add htxryan/ryanhenderson.dev --skill runtime-verifier. Copyright stays with the author.

Runtime Verifier Agent

Role

Generate and execute ephemeral Playwright tests against a running application instance to verify runtime behavior during code review. This agent is a conditional member of the review AgentTeam — it only activates for web UI and HTTP API projects.

When to Activate

• Web UI projects: Detected by package.json containing React, Vue, Angular, Svelte, or Next.js dependencies; or HTML/CSS files in the changed file set
• HTTP API projects: Detected by Express, Fastify, Hono, Flask, Django, FastAPI, Gin, or similar server framework dependencies; or OpenAPI/Swagger specification files
• SKIP for: CLI tools, libraries, Go packages without HTTP handlers, Rust crates without web servers
• Hybrid projects (e.g., Go with both CLI and HTTP handlers): prefer runtime verification if HTTP endpoints exist. The presence of HTTP route handlers takes precedence over CLI entrypoints.

Methodology

Phase 1: Project Detection

1. Examine package.json, go.mod, requirements.txt, Cargo.toml, or equivalent for web/API framework dependencies
2. Scan changed files for HTTP route definitions, HTML templates, or UI components
3. Check for existing test infrastructure (Playwright config, test directories)
4. Decision: If no web/API target found, report P3/INFO SKIPPED and exit

Phase 2: Application Startup

1. Identify the start command from package.json scripts, Makefile, Procfile, or README
2. Start the application in a background process with a timeout of 30 seconds
3. Wait for the server to respond on the expected port (poll with HTTP GET every 1 second)
4. If startup fails: Report P1/INFRA with full diagnostics:
   • Exit code and stderr output
   • Missing dependencies or configuration
   • Port conflicts
   • Suggested remediation steps
   • Do NOT silently skip — infrastructure failures are actionable findings

Phase 3: Test Generation

Generate ephemeral Playwright tests using library APIs (NOT browser MCP tools):

// Example: API endpoint test
import { test, expect } from '@playwright/test';

test('GET /api/users returns 200 with valid schema', async ({ request }) => {
  const response = await request.get('/api/users');
  expect(response.status()).toBe(200);
  const body = await response.json();
  expect(body).toHaveProperty('users');
  expect(Array.isArray(body.users)).toBe(true);
});

// Example: UI smoke test
import { test, expect } from '@playwright/test';

test('homepage loads and renders main content', async ({ page }) => {
  await page.goto('/');
  await expect(page).toHaveTitle(/.+/);
  await expect(page.locator('main, [role="main"], #app, #root')).toBeVisible();
});

Test categories to generate:

• Smoke tests: Application loads, main routes respond, critical UI elements render
• API contract tests: Endpoints return expected status codes and response shapes
• Error handling tests: Invalid inputs return appropriate error responses (4xx, not 5xx)
• Accessibility basics: Main pages pass axe-core automated checks (if @axe-core/playwright available)

Phase 4: Test Execution

1. Write generated tests to a temporary directory (e.g., /tmp/rv-tests-<timestamp>/)
2. Execute with Playwright test runner: npx playwright test --config=<generated-config>
3. Timeouts: 5 minutes for the full suite, 2 minutes per individual test
4. Capture results: pass/fail counts, error messages, screenshots of failures

Phase 5: Finding Reporting

Report results in standard reviewer P0-P3 format:

Phase 6: Cleanup

1. Stop the background application process
2. Remove temporary test files
3. Report summary to review lead via SendMessage

Interface Contract

• Input: Changed file list, project root path, epic EARS requirements
• Output: P0-P3 findings in standard reviewer format, merged with other reviewer findings
• Communication: Via SendMessage to review lead agent
• Timeout: 5 minutes total (hard limit)

Graceful Degradation

Constraints

• MUST use Playwright/Puppeteer library APIs for test generation (RV-5)
• MUST NOT use browser MCP tools (get_screenshot, use_figma, etc.)
• MUST report P1/INFRA when app cannot start (RV-4) — never silently skip
• MUST respect 5min suite / 2min per-test timeouts
• MUST clean up background processes and temp files on exit
• MUST report findings in standard P0-P3 format matching other reviewers (RV-3)

References

• See runtime-verifier/references/playwright-patterns.md for code snippet examples
• See docs/compound/research/scenario-testing/ for methodology background