Imported from bbolek-ap/new-ai-native (.claude/skills/automation-candidate-criteria/SKILL.md). Install upstream with npx skills add bbolek-ap/new-ai-native --skill automation-candidate-criteria. Copyright stays with the author.

Automation Candidate Criteria

Before extracting test cases, understand what qualifies as an automation candidate. Not everything in the execution plan needs a test.

✅ Automate: Functional Guarantees (testable business outcomes)

These add real value and must be automated:

1. Domain rules — business logic constraints that must be enforced

   • "Resource cannot be modified if status is immutable"
   • "Quantity cannot exceed maximum allowed limit"
   • "Entity cannot transition from Draft directly to Final"

2. State transitions — valid and invalid paths through application state

   • "Request moves to Approved after validation succeeds"
   • "Operation is rejected when precondition is not met"

3. Business events — what the system publishes as a consequence of actions

   • "Success event is published when action completes"
   • "Compensation event is sent when workflow step fails"

4. API contracts — endpoint behaviour for each business scenario

   • Each endpoint × valid status code combination
   • Error responses with correct format for known failure modes
   • Auth boundary enforcement (forbidden, unauthorized)

5. Async workflows — saga steps, message routing, compensation

   • "Next step executes after previous step completes and publishes confirmation"
   • "Rollback handler runs when failure event is received"
   • "Timeout triggers retry logic after configured duration"

6. Cross-service interactions — contracts and data consistency at integration boundaries

   • "Service reads external response and updates internal state correctly"
   • "Event published by one service is consumed by another as specified"

7. Non-functional thresholds — measurable, testable performance or scalability targets

   • "Response time ≤ 200ms p95"
   • "System handles 1000 concurrent requests"

❌ Do Not Automate: Technical Verification & Setup

These do NOT add functional value — they verify that environment is correct, not that the system works:

1. Initial setup, infrastructure prerequisites, and infrastructure code — these must work before tests can run, so they are implicitly verified by the fact that tests execute at all. This includes stories where code is written in a general-purpose language (C#, TypeScript) but whose purpose is environment setup rather than business behaviour delivery.

   • "Application starts without error"
   • "Database connection pool is initialized"
   • "Message queue is accessible"
   • "Data migrations complete successfully"
   • "Migration runner creates expected schema"
   • "Migration runner is idempotent when run twice"
   • Any other "smoke test" that confirms the environment is ready

   Why: If setup fails, tests cannot execute. A passing test run IS the proof that setup worked. Do not waste automation effort on setup verification. Migration runners are infrastructure code even when written in C#. Their purpose is to prepare the environment for functional tests. A test suite that runs and passes proves schemas exist — if a schema were missing, every downstream test touching that schema would fail. You do not need a dedicated schema-presence test.

2. File and asset presence checks — verification that files exist in the right place

   • "Static stylesheet is deployed at expected path"
   • "Configuration file is readable from expected location"
   • "Health endpoint returns success status"
   • "Required dependency is available on filesystem"

   Why: These verify deployment correctness, not application behaviour. They are infrastructure/DevOps concerns, not functional guarantees.

3. Static analysis and linting — checks that code meets syntactic or style standards

   • "All imports are used"
   • "No undefined variables"
   • "Naming conventions are followed"

   Why: Static analysis tools already do this. Tests should verify behaviour, not run linters.

4. Schema and metadata correctness — checks that types/classes are defined as expected

   • "Handler can be resolved by dependency container"
   • "Enum constant is defined in type"
   • "Method implements expected interface correctly"
   • "Required field is defined on entity"

   Why: The fact that code compiles and tests execute proves schema is correct. If a field didn't exist, the code would not compile.

5. One-off data migrations or seed operations — operations that apply once and never repeat

   • "System administrator is created during initialization"
   • "Reference data is seeded into database"
   • "Default values are populated during setup"

   Why: These are validation steps, not functional tests. If the migration is wrong, it will be caught when business logic that depends on it fails. If the business logic tests pass, the migration worked.

6. Dependency resolution and composition — checks that the DI container works

   • "Component can be constructed by dependency container"
   • "Service receives correct implementation instance"
   • "Factory method produces correct type instance"

   Why: The DI container is third-party code. Verify your domain logic uses the dependency correctly, not that the container resolves it. If resolution fails, the application won't start.

7. Integration with third-party frameworks — checks that frameworks work as documented

   • "ORM can persist and retrieve an entity"
   • "Message broker can publish and receive a message"
   • "JSON serializer can convert objects to/from JSON"
   • "Middleware processes requests in correct sequence"

   Why: These test third-party code, not your domain. Verify that YOUR code uses the framework correctly (e.g., "aggregate is persisted correctly" → you wrote the code; "ORM can persist" → third-party code). If framework usage is wrong, business logic tests will fail.

8. Repeated assertions on N identical instances of the same pattern — if a story creates N services, handlers, or runners that all follow the same structural pattern, do not write N identical test cases (one per instance). Test the pattern once with a representative instance.

   • VerifyContractsMigratorCreatesContractsSchema, VerifyNominationsMigratorCreatesNominationsSchema, … (×4)
   • VerifyContractsHandlerValidatesRequest, VerifyNominationsHandlerValidatesRequest, … (×N)

   Why: Enumerating a pattern across N instances adds no additional confidence. If the implementation is correct for one instance, it is correct for all — the shared code guarantees it. The only cases worth testing separately are instances with genuinely different behaviour. Different names or connection strings are parameters, not behaviour differences. Use a parameterized test or test one representative case.

Rule of thumb: If the test would still pass after removing all your business logic (replacing it with a stub that returns a hardcoded value), the test is not testing your domain — delete it.

Automation Candidate Decision Tree

When deciding whether to automate a test case:

Does the test verify application BEHAVIOUR 
(what the system does when users/systems interact with it)?
  ├─ YES → Automate (unless justified as Manual)
  └─ NO → Skip automation

Identifying the zone is not a second decision — the skip is already decided. The zone tells you where on the map you are so you can document the justification and explain it to others.