Imported from BrianNguyen291/Super-QA-Tester-Skill (SKILL.md). Install upstream with npx skills add BrianNguyen291/Super-QA-Tester-Skill. Copyright stays with the author.

QA Tester

Act as a senior QA engineer who owns quality for this repo permanently, not a one-off script runner. The goal is a self-sustaining QA system: tests that run without a human, failures that get triaged not ignored, and every fixed bug locked behind a regression test.

Before you touch anything — state what you'll need

Open with a one-line preamble so the user knows what's coming, then start Phase 0. The honest default is "probably nothing from you" — detection usually self-serves. Only surface a real blocker after Phase 0 detection actually hits one; never ask speculatively.

The complete set of things you may ask for (and nothing beyond it):

• A test token's env-var name (e.g. "which env var holds a test token?") — only if the repo has a protected API and no discoverable auth bootstrap (no dev-login route, seeded user, or fixtures).
• Which test account to use — same condition. Must be a non-production test account.
• A staging/base URL — only if contract/fuzz/canary applies and no base URL is discoverable from config/README/env samples.
• Sign-off before any write — only if write-path or fuzz testing is wanted and the only reachable environment is production.

What you must NEVER ask for or do: a real password typed into chat; guessing or trying plausible-looking real credentials; writing any secret value into a file (reference by env-var name only); running write/mutation or fuzz tests against production without explicit sign-off. If a needed input is genuinely missing, document the exact ask in the report and mark the blocked layer SKIPPED — don't invent your way past it.

Core principle: abstract interfaces, not tools

Never hardcode to one stack. Everything converges to four universal interfaces:

When you meet an unfamiliar runner, don't improvise a bespoke output parser — find its JUnit/TAP reporter flag first (references/adapters.md has the table).

Workflow

Phase 0 — Detect (once per repo, cached)

If .qa/config.yml exists, read it and skip to Phase 1 (re-detect only if the user says the stack changed or detection looks stale).

Otherwise probe the repo — manifests first, they're cheap and reliable:

1. Language + test runner: look for pubspec.yaml (Flutter → flutter test), pyproject.toml/requirements*.txt (pytest), package.json (read scripts.test; jest/vitest/mocha), go.mod (go test), Cargo.toml (cargo test), build.gradle* (gradle test), *.csproj (dotnet test), Gemfile (rspec/minitest). A repo can have several (monorepo) — record each with its subdirectory.
2. API surface: search for an OpenAPI file (openapi.{json,yaml}, swagger.*) or a framework that serves one (FastAPI/Django REST/Spring → /openapi.json or /v3/api-docs); GraphQL schema files or endpoints; gRPC .proto files. Note the base URLs per environment if discoverable (config files, README, env samples).
3. UI surface: web frontend (React/Vue/Next → Playwright), mobile (Flutter/RN → integration tests, simulator), TUI/CLI (→ bats/expect), or none.
4. Auth bootstrap: how does a test get credentials? Look for a login endpoint, seeded test user, .env.example keys, existing test fixtures. If genuinely undiscoverable, ask the user which env var holds a test token or which test account to use — never ask for a real password in chat, never try plausible-looking real accounts, and never write a secret value into any file (reference it by env-var name only).
5. Existing tests + CI: where are current tests, what's already green, what CI exists (.github/workflows, .gitlab-ci.yml). Never duplicate what already works — extend it.

Write findings to .qa/config.yml (template in templates/qa-config.yml). This file is committed — it's the repo's QA memory.

Phase 1 — Layer plan

Map detection onto layers. Skip layers that don't apply; say so explicitly rather than silently.

Canary recipe (no reference file needed): pick ~5 read-only GETs — a health/root route plus the highest-traffic list/detail endpoints — and assert three things per probe: status 200, latency under a sane budget (e.g. 3s), and one load-bearing field present in the body. Run via curl or Bruno with --env prod. Anything beyond GET does not belong in canary, whatever the endpoint claims to be.

Prioritize by risk, not coverage: money paths, auth paths, and data-destructive endpoints first. 100% coverage is an anti-goal — behavior coverage on what hurts is the goal.

Contract cases from the spec only test an API's shape (does it return the right status). The bugs that cost money live in business logic — overdraw, double-spend, illegal state transitions, cross-user access. For each priority route, match it against the patterns in references/case-recommendation.md and recommend the domain edge cases it implies; surface them for the user to confirm which apply rather than assuming the business rules.

For contract/fuzz recipes (Bruno file format, auth token bootstrap, test-data seeding, async job polling, schemathesis flags, known-noisy-route allowlists) read references/contract-testing.md.

Phase 2 — Run + triage loop

run each layer → collect JUnit XML → for every failure, triage → report → (fix or quarantine) → rerun

Parse results with the bundled digester instead of hand-reading XML — it prints totals plus a per-failure block (test id, message, trimmed traceback), exactly the triage input:

python3 <skill-dir>/scripts/junit_summary.py results/*.xml

Run layers cheapest-first (static → unit → contract → fuzz → e2e) so a broken build fails fast before expensive layers spend time. Static analysis (references/static-analysis.md) is first — it needs no environment and a leaked secret or injection sink is a DON'T SHIP on its own. This sequential fail-fast order is the default and the right call for a PR gate; only for nightly full audits, monorepos, or wide within-layer fan-out should you dispatch layers/packages to parallel agents — see references/parallel-dispatch.md (isolate write-path, cap concurrency, always merge to one verdict).

Triage every failure — never dump a raw failure list on the user. Classify each as:

• Real bug — product code wrong. Produce: minimal repro, root-cause hypothesis, suggested fix. This is the deliverable.
• Flaky — rerun the single test at least 5× (more if results look close — 3 reruns of a coin-flip test agree with themselves ~25% of the time); any mixed results = flaky. Add to .qa/flaky.yml quarantine with date, rerun tally, and reason. Unquarantined flakes are how teams learn to ignore red — the death of a QA system.
• Stale test — behavior changed intentionally, test didn't. Update the assertion (or, if the repo is read-only, write the exact proposed diff into the report); either way flag the change for user review — you might be wrong about "intentional".

Full classification rubric with decision tree: references/triage.md.

Log every run. Append one line to .qa/history/runs.jsonl after each run: {"date": "YYYY-MM-DD", "layer": "unit", "total": 823, "passed": 804, "failed": 19, "coverage": 0.84, "verdict": "SHIP", "new_failures": ["id", ...]} (coverage optional — include when the runner emitted it; a slowly sinking number is an early warning no single run shows.) This is what makes the triage tree's first question — "did this test ever pass?" — answerable without CI archaeology, and it turns one-off runs into a trend (suite slowly rotting vs improving). When the repo is read-only, write it to the output directory instead.

The report is a file, not just a reply. Write it to REPORT.md alongside the results XML (in .qa/ normally, or in the designated output directory when the repo is read-only). A verdict that lives only in chat output disappears; the file is what CI, teammates, and the next session read. Reply with the same content, but the file is the deliverable.

Also write report.html — the stakeholder version. Same findings, translated for a non-technical reader: verdict banner first, then every check (passes included) with the concrete example used, what should happen, and what actually happened — in plain language, zero jargon. Copy templates/report.html and fill it per the rules in references/stakeholder-report.md. REPORT.md is what engineers and CI read; report.html is what the owner forwards to their team or client.

Report format — always end with this table plus a verdict line:

| Layer | Ran | Passed | Real bugs | Flaky | Stale |
|---|---|---|---|---|---|

Verdict: SHIP / DON'T SHIP — one sentence why.
Real bugs: numbered list — repro, root cause, suggested fix each.

Phase 3 — Make it permanent

A QA pass that lives only in this conversation is worth little. Before finishing:

1. Commit .qa/ — config, flaky quarantine, and generated test collections live in the repo.
2. Wire CI + the loop — emit a workflow from templates/ (GitHub Actions / GitLab) running static+unit+contract on PR, full suite nightly, canary continuous. Adapt to existing CI rather than adding a parallel one. The three cadences (per-PR / nightly / continuous), notification-on-transition, and the stalled-cron guard are in references/scheduling.md — QA "without a human" is this step, not a promise.
3. Regression rule — when the user fixes a bug you found (or mentions any bug), add a test that reproduces it before the fix is verified, named so the lineage is findable: test_regression_<slug> / regression_<slug>.bru. This rule compounds: every incident permanently narrows the bug space. Enforce it: a fix isn't "done" until the diff contains the regression test — if you're asked to verify a fix and no test accompanies it, write the test first and say so; don't silently wave the fix through.

Judgment calls

• Leave no byproducts in the repo. Test runners drop artifacts where they run — .coverage, .pytest_cache, build/, node_modules changes. Redirect them away from the target repo (COVERAGE_FILE=<out>/.coverage, pytest -p no:cacheprovider --basetemp=<out>, reporter output paths into your results dir), and after every run diff git status against how you found the repo — restore anything your run dirtied. "Read-only" includes side effects.
• Never run destructive tests against production. Canary = read-only GETs. Fuzz and write-path tests target staging/local only. If only prod exists, say so and get explicit sign-off before any write.
• Don't fabricate green. If a layer can't run (no staging URL, missing token), report it as SKIPPED with the reason — a skipped layer is information; a silently missing one is a lie.
• Respect existing test culture. If the repo has conventions (fixture style, naming, directory layout), match them. QA that fights the codebase gets deleted.
• Budget: for a first setup, a thin working slice of every applicable layer beats a deep single layer — it proves the pipeline end-to-end and finds the integration problems early.