qa

Imported from matchojecki/logistiko-celny (.ai/qa/AGENTS.md). Install upstream with npx skills add matchojecki/logistiko-celny --skill qa. Copyright stays with the author.

QA Integration Testing Instructions

Quick Start

# Run all integration tests headlessly (zero token cost, CI-ready)
yarn test:integration

# Run tests matching a module/category path fragment
npx playwright test --config .ai/qa/tests/playwright.config.ts sales

# Run all tests in ephemeral containers (no dev server needed, Docker required)
yarn test:integration:ephemeral

# Run tests from an interactive menu in persisted ephemeral environment
yarn test:integration:ephemeral:interactive

# Start isolated ephemeral app only (for MCP/manual exploration)
yarn test:integration:ephemeral:start

# View HTML report
yarn test:integration:report

Preferred local workflow for short iterations:

1. Start yarn test:integration:ephemeral:start
2. Reuse the running environment from .ai/qa/ephemeral-env.json
3. Use /integration-tests against that URL

Discovery troubleshooting:

• If Playwright reports No tests found, run npx playwright test --config .ai/qa/tests/playwright.config.ts --list first.
• Keep testIgnore entries in .ai/qa/tests/playwright.config.ts scoped to absolute paths under projectRoot; avoid loose relative globs such as .codex/** that can match parent workspace paths.

Directory Structure

.ai/qa/
├── AGENTS.md                    # This file
├── scenarios/                   # OPTIONAL — markdown test case descriptions
│   ├── TC-AUTH-001-*.md         #   Human-readable, used as input for test generation
│   ├── TC-CAT-001-*.md         #   NOT required — tests can be generated directly
│   └── ...
├── tests/                       # Playwright config/helpers + legacy test location
│   ├── playwright.config.ts
│   ├── .gitignore               # Ignores test-results/
│   ├── helpers/
│   │   ├── auth.ts              # Login helper
│   │   └── api.ts               # API call helper
└── ...

packages/<package>/src/modules/<module>/__integration__/   # Preferred test location
apps/mercato/src/modules/<module>/__integration__/         # App-specific modules
packages/enterprise/modules/<module>/__integration__/      # Optional enterprise overlay tests

Reusable Helpers

Use shared helpers from @open-mercato/core/helpers/integration/*. These are published in the npm package and available to both monorepo and standalone app developers.

Legacy path: @open-mercato/core/modules/core/__integration__/helpers/* still works in the monorepo via re-exports but is NOT available from npm. New code should use the @open-mercato/core/helpers/integration/* path.

Import pattern from module tests:

import { login } from '@open-mercato/core/helpers/integration/auth';
import { apiRequest, getAuthToken } from '@open-mercato/core/helpers/integration/api';

Barrel import (subset): @open-mercato/core/testing/integration re-exports the most common helpers as a single import for convenience.

Scenarios Are Optional

Markdown test scenarios (.ai/qa/scenarios/TC-*.md) are optional reference material. Tests can be generated through any of these paths:

Two Testing Modes

1. Executable Tests (Playwright TypeScript) — Preferred

Pre-written tests discovered from module __integration__ folders (with legacy .ai/qa/tests/ support) run headlessly via yarn test:integration. Zero token cost, CI-ready.

yarn test:integration

2. Manual AI-Driven Tests (Playwright MCP)

An AI agent reads a scenario or spec and executes it interactively via Playwright MCP. Useful for exploratory testing and for creating new executable tests.

Interactive Ephemeral Runner

Use interactive mode as the default local workflow when you want one ephemeral app/database session and multiple test runs without repeating full bootstrap.

yarn test:integration:ephemeral:interactive

What you can do from the menu:

• Run all tests
• Run one selected .spec.ts file
• Refresh the discovered test list
• Open Playwright HTML report
• Quit and clean up the environment

Useful flags:

• --workers <n>
• --retries <n>
• --verbose
• --screenshots
• --no-screenshots

Environment state:

• Active ephemeral environment is written to .ai/qa/ephemeral-env.json
• Default app port is 5001 when available
• If 5001 is busy, a free fallback port is used and written to .ai/qa/ephemeral-env.json
• File is cleared automatically when the ephemeral environment is stopped

How to Create New Tests

Option A: Use /integration-tests Skill (Recommended)

The skill reads the related spec, explores the running app via Playwright MCP, and produces executable tests automatically. It optionally generates a markdown scenario for documentation.

/integration-tests

Option B: Manual Workflow

Step 1 — Understand What to Test

Read one of:

• A spec from .ai/specs/*.md or .ai/specs/enterprise/*.md
• A scenario from .ai/qa/scenarios/TC-*.md (if one exists)
• A feature description from the user

Step 2 — Explore via Playwright MCP

Always check .ai/qa/ephemeral-env.json first and reuse an existing running environment.

If no active environment exists, start interactive mode first:

yarn test:integration:ephemeral:start

Use isolated app mode only for MCP/manual exploration without the menu:

yarn test:integration:ephemeral:start

Use base_url from .ai/qa/ephemeral-env.json to avoid interference with any other local app instance.

Walk through the test flow interactively to discover actual selectors:

mcp__playwright__browser_navigate({ url: "http://127.0.0.1:<ephemeral-port>/login" })
mcp__playwright__browser_snapshot()
mcp__playwright__browser_click({ element: "Submit button", ref: "..." })

For each test step:

1. Execute the action via Playwright MCP
2. Snapshot to identify actual element locators (roles, labels, text)
3. Verify the expected result matches reality
4. Note the locator strategy that works

Step 3 — Write the TypeScript Test

Create the test in the module where behavior lives:

• packages/<package>/src/modules/<module>/__integration__/TC-{CATEGORY}-{XXX}.spec.ts
• apps/mercato/src/modules/<module>/__integration__/TC-{CATEGORY}-{XXX}.spec.ts
• packages/create-app/template/src/modules/<module>/__integration__/TC-{CATEGORY}-{XXX}.spec.ts
• packages/enterprise/modules/<module>/__integration__/TC-{CATEGORY}-{XXX}.spec.ts for overlay tests only
• Nested subfolders inside __integration__ are supported

UI test template:

import { test, expect } from '@playwright/test';
import { login } from './helpers/auth';

/**
 * TC-{CATEGORY}-{XXX}: {Title}
 * Source: .ai/qa/scenarios/TC-{CATEGORY}-{XXX}-{slug}.md (if exists)
 */
test.describe('TC-{CATEGORY}-{XXX}: {Title}', () => {
  test.beforeEach(async ({ page }) => {
    await login(page, 'admin');
  });

  test('should {main scenario}', async ({ page }) => {
    await page.goto('/backend/...');
    await page.getByRole('button', { name: '...' }).click();
    await expect(page.getByText('...')).toBeVisible();
  });
});

API test template:

import { test, expect } from '@playwright/test';
import { getAuthToken, apiRequest } from './helpers/api';

/**
 * TC-{CATEGORY}-{XXX}: {Title}
 * Source: .ai/qa/scenarios/TC-{CATEGORY}-{XXX}-{slug}.md (if exists)
 */
test.describe('TC-{CATEGORY}-{XXX}: {Title}', () => {
  let token: string;

  test.beforeAll(async ({ request }) => {
    token = await getAuthToken(request);
  });

  test('should {main scenario}', async ({ request }) => {
    const response = await apiRequest(request, 'GET', '/api/...', { token });
    expect(response.ok()).toBeTruthy();
    const body = await response.json();
    expect(body).toHaveProperty('...');
  });
});

Step 4 — Verify

Run the test to confirm it passes:

npx playwright test --config .ai/qa/tests/playwright.config.ts <path-to-test-file>

Conditional Metadata (Folder + Test)

Use optional metadata to skip tests when required modules are not enabled.

• Folder-level metadata:
  • Add meta.ts or index.ts under any __integration__/ subfolder
  • Supported keys: dependsOnModules, requiredModules, requiresModules
• Per-test metadata:
  • Add the same keys inside the .spec.ts file, or create sibling TC-*.meta.ts
• Inheritance:
  • Metadata is inherited from __integration__/ root through nested subfolders, then test-level metadata is applied
• Behavior:
  • If any declared dependency module is not enabled, that folder/test is excluded from discovery and run

Example folder metadata:

export const integrationMeta = {
  description: 'Sales flows requiring currencies module',
  dependsOnModules: ['sales', 'currencies'],
}

MUST Rules for Executable Tests

• Use Playwright locators: getByRole, getByLabel, getByText, getByPlaceholder — avoid CSS selectors
• If a matching scenario exists, reference it in a comment (e.g., Source: .ai/qa/scenarios/TC-AUTH-001-*.md)
• Keep tests independent — each test handles its own login
• Keep tests data-independent — do not rely on seeded/demo records being present
• Create required fixtures per test (prefer API setup), and always clean up created data in finally/teardown
• Ensure tests are deterministic/stable across retries and run order (no cross-test state coupling)
• Keep reusable helpers centralized in packages/core/src/helpers/integration/ (importable as @open-mercato/core/helpers/integration/*), and re-export from module-local helper files when needed
• One .spec.ts file per test case
• MUST NOT leave broken tests — fix or skip with test.skip() and a reason

How to Test Manually (AI-Driven via Playwright MCP)

UI Testing

Use Playwright MCP to execute UI test scenarios. The browser automation handles navigation, form interactions, and visual verification.

# Example: Navigate and interact
mcp__playwright__browser_navigate({ url: "http://127.0.0.1:<ephemeral-port>/backend/login" })
mcp__playwright__browser_snapshot()
mcp__playwright__browser_fill_form({ fields: [...] })
mcp__playwright__browser_click({ element: "Submit button", ref: "..." })

Workflow:

1. Navigate to the target URL
2. Take a snapshot to identify element refs
3. Interact with elements (click, type, fill forms)
4. Verify expected results via snapshots or assertions

API Testing (cURL)

Use cURL for direct API endpoint testing.

# Login and get token
curl -X POST http://127.0.0.1:<ephemeral-port>/api/auth/login \
  -H "Content-Type: application/json" \
  -d '{"email": "admin@acme.com", "password": "secret"}'

# Authenticated request
curl -X GET http://127.0.0.1:<ephemeral-port>/api/customers/companies \
  -H "Authorization: Bearer <token>" \
  -H "Content-Type: application/json"

Default Credentials

These accounts are created via mercato init command:

Note: Superadmin has access to all features across all tenants. Admin has full access within their organization. Employee has limited access based on role configuration.

Results Presentation

For yarn test:integration (Headless)

Results are automatically generated:

• Console: Pass/fail summary with list reporter
• JSON: test-results/results.json — machine-readable for CI
• HTML: test-results/html/ — interactive report (open with yarn test:integration:report)

For AI-Driven Tests (Manual)

Present test results in a table format:

Test Run Summary

Summary Statistics

Failed Tests Detail

For each failed test, include:

• Test ID: TC-XXX-XXX
• Failure Step: Step number where failure occurred
• Expected: What should have happened
• Actual: What actually happened
• Screenshot/Evidence: If applicable

How to Manage Scenarios (Optional)

Scenarios live in .ai/qa/scenarios/ and serve as documentation. They are NOT required for test generation.

Naming Convention

TC-[CATEGORY]-[XXX]-[title].md

• TC: Test Case prefix
• CATEGORY: Module category code (see category codes below)
• XXX: 3-digit sequential number
• title: Kebab-case descriptive title

Category Codes

Scenario Template

# Test Scenario [NUMBER]: [TITLE]

## Test ID
TC-[CATEGORY]-[XXX]

## Category
[Category Name]

## Priority
[High/Medium/Low]

## Type
[UI Test / API Test]

## Description
[Brief description of what this test validates]

## Prerequisites
- [Prerequisite 1]
- [Prerequisite 2]

## API Endpoint (for API tests)
`[METHOD] /api/path`

## Test Steps
| Step | Action | Expected Result |
|------|--------|-----------------|
| 1 | [Action] | [Expected] |
| 2 | [Action] | [Expected] |

## Expected Results
- [Final expected outcome 1]
- [Final expected outcome 2]

## Edge Cases / Error Scenarios
- [Edge case 1]
- [Edge case 2]

Best Practices

1. One scenario per file: Keep tests atomic and focused
2. Clear prerequisites: List all setup requirements
3. Specific steps: Each step should be actionable
4. Measurable results: Expected results should be verifiable
5. Include edge cases: Document error scenarios and boundary conditions
6. Set priority: High for critical paths, Medium for standard flows, Low for edge cases