Imported from yu-iskw/data-agent-ctl (
.claude/skills/test-and-fix/SKILL.md). Install upstream withnpx skills add yu-iskw/data-agent-ctl --skill test-and-fix. Copyright stays with the author.
Test and Fix Loop
Purpose
An autonomous loop for the agent to identify, analyze, and fix failing unit tests using pytest.
Loop Logic
- Identify: Run
make testto identify failing tests. - Analyze: Examine the
pytestoutput to determine:- The failing test file and line number.
- The expected vs actual values (assertion errors).
- Tracebacks for runtime errors.
- Fix: Apply the minimum necessary change to either the source code (if it's a bug) or the test code (if the test is outdated).
- Verify: Re-run
make test(oruv run pytest path/to/failing_test.pyfor speed).- If passed: Move to the next failing test or finish if all are resolved.
- If failed: Analyze the new failure and repeat the loop.
Termination Criteria
- All tests pass (as reported by
make test). - Reached max iteration limit (default: 5).
- The error persists after multiple distinct fix attempts, indicating a need for human intervention.
Examples
Scenario: Fixing a logic error
make testfails insrc/your_package/tests/test_dummy.pydue to an assertion or import error.- Agent inspects the failing test and the implementation under
src/your_package/. - Agent applies the minimum fix in source or test so behavior matches the intended contract.
make testpasses.
Resources
- Pytest Documentation: Official documentation for the pytest framework.
- Testing conventions for this repo: AGENTS.md (Testing section).
