Instruction file imported from BetterThanTomorrow/calva (.github/instructions/workspace-smoke-testing-memory.instructions.md). Copyright stays with the author.

Smoke Testing Memory

Patterns for reliable smoke testing and debugging workflows.

Use Containerized Test Runner

Always use the containerized test runner to reproduce CI conditions exactly:

npm run smoke-test-containerized

Why: The containerized runner can use Insiders even if it is being used on the host machine.

Test Both Development and Production Scenarios

Smoke tests should cover both:

• Development mode: extensionDevelopmentPath for source code testing
• Production mode: VSIX package testing (what users actually install)

Use npm Scripts for Containerized Testing

Always use npm scripts rather than running Docker commands directly:

• npm run smoke-test-containerized - for containerized testing

This ensures proper environment setup and dependency management.

Environment Variable Debugging

When debugging CI failures related to environment variables:

• Use exact CI argument reproduction: npm run smoke-test-containerized -- --ci
• Check JavaScript variable access syntax: process.env.VARIABLE_NAME not process.env[VARIABLE_NAME]
• Verify variable scoping in launch scripts

Data-Oriented Smoke Test Debugging

Follow the general debugging workflow for smoke test issues:

1. Gather context - understand CI vs local environment differences
2. Reproduce precisely - use containerized runner with exact CI conditions
3. Analyze data flow - trace through launch.js and test setup
4. Apply targeted fix - address root cause in environment/configuration
5. Validate thoroughly - test with both development and CI scenarios

Targeted Test Suite Execution

Implement CLI filters for selective test execution to accelerate debugging of flaky tests. Pass filters through an environment variable like CALVA_E2E_SUITE_FILTER, selecting only matching *-test.js files. This enables running specific suites (e.g., session-management-test or jack-in-test) on demand, avoiding full suite reruns and ensuring faster, more deterministic flake reproduction.