doit.testit

Prompt file imported from seanbarlow/theme-switcher (.claude/commands/doit.testit.md). Fill in {{arguments}} before use. Copyright stays with the author.

User Input

{{arguments}}

You MUST consider the user input before proceeding (if not empty).

Load Project Context

Before proceeding, load the project context to inform your responses:

doit context show

If the command fails or doit is not installed: Continue without context, but note that alignment with project principles cannot be verified.

Use loaded context to:

• Reference constitution principles when making decisions
• Consider roadmap priorities
• Identify connections to related specifications

Code Quality Guidelines

Before generating or modifying code:

1. Search for existing implementations - Use Glob/Grep to find similar functionality before creating new code
2. Follow established patterns - Match existing code style, naming conventions, and architecture
3. Avoid duplication - Reference or extend existing utilities rather than recreating them
4. Check imports - Verify required dependencies already exist in the project

Context already loaded (DO NOT read these files again):

• Constitution principles and tech stack
• Roadmap priorities
• Current specification
• Related specifications

Artifact Storage

• Temporary scripts: Save to .doit/temp/{purpose}-{timestamp}.sh (or .py/.ps1)
• Status reports: Save to specs/{feature}/reports/{command}-report-{timestamp}.md
• Create directories if needed: Use mkdir -p before writing files
• Note: .doit/temp/ is gitignored - temporary files will not be committed

Outline

1. Setup: Run .doit/scripts/bash/check-prerequisites.sh --json from repo root and parse FEATURE_DIR and AVAILABLE_DOCS list. All paths must be absolute.

2. Load test context:

   • REQUIRED: Read spec.md for requirements (FR-XXX identifiers)
   • REQUIRED: Read tasks.md for test file locations
   • IF EXISTS: Read plan.md for test strategy and coverage goals
   • IF EXISTS: Read contracts/ for API test expectations

3. Detect test framework:

   • Check for test configuration files:
     • Python: pytest.ini, pyproject.toml [tool.pytest], setup.cfg, conftest.py
     • JavaScript/TypeScript: jest.config.js/ts, vitest.config.js/ts, mocha.opts, .mocharc.*
     • Java: pom.xml (maven-surefire), build.gradle (test task)
     • Go: *_test.go files present
     • Ruby: Rakefile, .rspec, spec/ directory
     • C#/.NET: *.csproj with test SDK, xunit, nunit references
     • Rust: Cargo.toml with [dev-dependencies] test crates
   • Determine test command:
     • pytest: pytest -v --tb=short
     • jest: npm test or npx jest
     • vitest: npm test or npx vitest run
     • go: go test ./...
     • maven: mvn test
     • gradle: ./gradlew test
     • dotnet: dotnet test
     • cargo: cargo test

4. Execute test suite:

   • Run detected test command
   • Capture stdout/stderr
   • Parse test results:
     • Total tests run
     • Passed tests
     • Failed tests
     • Skipped tests
     • Test duration
   • Capture any coverage reports if generated

5. Generate test report:

   ## Automated Test Results
   
   **Framework**: [detected framework]
   **Command**: [test command used]
   **Duration**: [total time]
   
   ### Summary
   | Metric | Count |
   |--------|-------|
   | Total | X |
   | Passed | X |
   | Failed | X |
   | Skipped | X |
   
   ### Failed Tests
   | Test | File | Error |
   |------|------|-------|
   | test_name | path/to/test.py | AssertionError: ... |
   
   ### Coverage (if available)
   | File | Coverage |
   |------|----------|
   | src/module.py | 85% |

6. Map tests to requirements:

   • Parse test names and docstrings for FR-XXX references

   • Match tests to requirements from spec.md

   • Generate requirement coverage matrix:

     ## Requirement Coverage
     
     | Requirement | Description | Tests | Status |
     |-------------|-------------|-------|--------|
     | FR-001 | User login | test_login, test_auth | COVERED |
     | FR-002 | Password reset | - | NOT COVERED |
     | FR-003 | Session timeout | test_session_expiry | COVERED |

   • Calculate coverage percentage: (covered requirements / total requirements) * 100

7. Generate manual testing checklist:

   • Extract acceptance criteria from spec.md that cannot be automated

   • Create checklist format:

     ## Manual Testing Checklist
     
     ### UI/UX Tests
     - [ ] MT-001: Verify login form displays correctly on mobile
     - [ ] MT-002: Verify error messages are user-friendly
     
     ### Integration Tests
     - [ ] MT-003: Verify third-party payment processing
     - [ ] MT-004: Verify email notifications are received
     
     ### Edge Cases
     - [ ] MT-005: Verify behavior with slow network
     - [ ] MT-006: Verify recovery from server timeout

8. Record manual test results:

   • If {{arguments}} contains --manual:
     • Present each manual test item
     • Ask for PASS/FAIL/SKIP result
     • Record notes for failed tests
     • Track completion progress
   • Otherwise, output checklist for later completion

9. Generate test-report.md in FEATURE_DIR:

   # Test Report: [Feature Name]
   
   **Date**: [timestamp]
   **Branch**: [current branch]
   **Test Framework**: [detected]
   
   ## Automated Tests
   
   ### Execution Summary
   | Metric | Value |
   |--------|-------|
   | Total Tests | X |
   | Passed | X |
   | Failed | X |
   | Skipped | X |
   | Duration | Xs |
   
   ### Failed Tests Detail
   [list of failed tests with errors]
   
   ### Code Coverage
   [coverage summary if available]
   
   ## Requirement Coverage
   
   | Requirement | Tests | Status |
   |-------------|-------|--------|
   | FR-XXX | test_xxx | COVERED |
   ...
   
   **Coverage**: X/Y requirements (Z%)
   
   ## Manual Testing
   
   ### Checklist Status
   | Test ID | Description | Status |
   |---------|-------------|--------|
   | MT-001 | ... | PENDING |
   ...
   
   ## Recommendations
   
   1. [Fix failing tests before merge]
   2. [Add tests for uncovered requirements]
   3. [Complete manual testing checklist]
   
   ## Next Steps
   
   - Fix any failing tests
   - Complete manual testing if not done
   - Run `/doit.checkin` when all tests pass

10. Report: Output path to test-report.md and summary:

    • Automated test pass rate
    • Requirement coverage percentage
    • Manual test completion status
    • Overall readiness for merge

Key Rules

• Use absolute paths for all file operations
• If no test framework detected, report and suggest adding tests
• Continue generating report even if some tests fail
• Preserve test output for debugging
• Map requirements using FR-XXX pattern matching in test names/docstrings

Error Recovery

No Test Framework Detected

The test runner could not find a compatible testing framework in the project.

ERROR | If no test framework is detected:

1. Check if test dependencies are installed: pip list | grep pytest (or equivalent for your framework)
2. If missing, install the test framework: pip install pytest (or as specified in dev dependencies)
3. Check if test configuration exists: ls pytest.ini pyproject.toml setup.cfg for pytest config sections
4. Verify: pytest --version confirms the framework is available

Prevention: Ensure test dependencies are listed in dev requirements and installed before running tests

If the above steps don't resolve the issue: manually specify the test command in the template or project configuration.

Test Execution Failure

One or more tests failed during the test run.

ERROR | If tests fail during execution:

1. Review the failure output to identify failing tests and error messages
2. Run the failing test in isolation for detailed output: pytest tests/path/to/failing_test.py -x -v --tb=long
3. Check if the failure is in your changes or pre-existing: git stash && pytest tests/ && git stash pop
4. Fix the failing code or test, then re-run: pytest tests/ -x --tb=short
5. Verify: pytest tests/ --tb=short shows all tests passing

If the above steps don't resolve the issue: run /doit.reviewit to get a code review that may identify the root cause.

Missing Test Dependencies

Required packages for testing are not installed.

ERROR | If test dependencies are missing:

1. Check for a requirements file: ls requirements*.txt **/requirements*.txt setup.py pyproject.toml
2. Install dev dependencies: pip install -e ".[dev]" or pip install -r requirements-dev.txt
3. If specific packages are missing, install them: pip install {package-name}
4. Verify: pip list | grep {package-name} confirms installation

Prevention: Run pip install -e ".[dev]" at the start of each development session

If the above steps don't resolve the issue: check if the dependency version is compatible with your Python version.

Coverage Threshold Not Met

Tests pass but code coverage is below the expected threshold.

WARNING | If coverage is below the target threshold:

1. Review the coverage report to identify uncovered files and lines
2. Check which files have the lowest coverage: review the coverage output or HTML report
3. Add tests for the most critical uncovered paths first
4. Re-run with coverage: pytest tests/ --cov=src/ --cov-report=term-missing
5. Verify: coverage percentage meets or exceeds the project threshold

If the above steps don't resolve the issue: document the coverage gap in the review notes and address it in a follow-up task.

Next Steps

After completing this command, display a recommendation section based on the outcome:

On Success (all tests pass)

Display the following at the end of your output:

---

## Next Steps

┌─────────────────────────────────────────────────────────────────────────┐
│  Workflow Progress                                                      │
│  ● specit → ● planit → ● taskit → ● implementit → ● testit → ○ checkin │
└─────────────────────────────────────────────────────────────────────────┘

**Recommended**: Run `/doit.reviewit` for a code review before finalizing.

**Alternative**: Run `/doit.checkin` to merge your changes if code review is not needed.

On Failure (tests fail)

If some tests fail:

---

## Next Steps

┌─────────────────────────────────────────────────────────────────────────┐
│  Workflow Progress                                                      │
│  ● specit → ● planit → ● taskit → ● implementit → ◐ testit → ○ checkin │
└─────────────────────────────────────────────────────────────────────────┘

**Status**: N tests failed out of M total.

**Recommended**: Run `/doit.implementit` to fix the failing tests.