Build Test Framework from Scratch

Prompt file imported from gokultw/playwright_ts_framework (.github/prompts/build-test-framework.prompt.md). Fill in {{params_ENV}} before use. Copyright stays with the author.

Before generating anything, ask the user the following two questions and wait for their answers:

1. Testing tool: Which test automation tool would you like to use?
   • Options: Playwright, Cypress, WebdriverIO, Selenium
2. Language: Which programming language would you like to use?
   • Options: TypeScript, JavaScript, Python, Java

Once the user provides both answers, use them throughout this prompt wherever [TOOL] and [LANGUAGE] appear, and proceed to build the framework.

Build a complete end-to-end test automation framework from scratch using the chosen tool and language.

The application under test is Swag Labs at https://www.saucedemo.com/.

Application Overview

Swag Labs is a demo e-commerce app. Explore and use the following pages and user accounts:

Pages:

Test Accounts (password for all: secret_sauce):

Framework Requirements

1. Project Structure

Generate the following folder layout:

project-root/
├── config/
│   ├── environments/
│   │   ├── dev.json           # { "baseUrl": "https://www.saucedemo.com" }
│   │   ├── staging.json       # same schema, different URL if needed
│   │   └── prod.json
│   └── config_reader.[ext]        # Reads active environment config
├── pages/                         # Page Object Model classes
│   ├── base_page.[ext]
│   ├── login_page.[ext]           # https://www.saucedemo.com/
│   ├── inventory_page.[ext]       # /inventory.html
│   ├── product_detail_page.[ext]  # /inventory-item.html
│   ├── cart_page.[ext]            # /cart.html
│   ├── checkout_info_page.[ext]   # /checkout-step-one.html
│   ├── checkout_overview_page.[ext] # /checkout-step-two.html
│   └── checkout_complete_page.[ext] # /checkout-complete.html
├── tests/                         # Test spec files
│   ├── login_test.[ext]
│   ├── inventory_test.[ext]
│   └── checkout_test.[ext]
├── test-data/                     # JSON test data files
│   ├── login.data.json
│   ├── inventory.data.json
│   └── checkout.data.json
├── utilities/
│   ├── json_reader.[ext]
│   ├── screenshot_helper.[ext]
│   ├── logger.[ext]
│   ├── faker_helper.[ext]
│   ├── date_utils.[ext]
│   └── string_utils.[ext]
├── logs/
├── fixtures/
│   ├── base_fixture.[ext]
│   └── auth_fixture.[ext]
├── ci/
│   ├── Jenkinsfile                # Declarative Jenkins pipeline
│   └── .github/
│       └── workflows/
│           └── test.yml               # GitHub Actions workflow
├── [TOOL].config.[ext]
└── [package-manager config]

[ext] = file extension for the chosen language (e.g. ts, js, py, java)

2. Multiple Environment Support

• Load environment from an ENV environment variable (e.g. ENV=staging npx [tool] test)
• Config reader must resolve baseUrl, apiUrl, credentials, and any env-specific values
• Fallback to dev if ENV is not set
• Each environment config file (dev.json, staging.json, prod.json) must follow a consistent schema

3. Config Reader

Create config/config_reader.[ext] using the idioms of the chosen language:

• Reads the active environment JSON file at runtime
• Exposes a config object/class accessible throughout the framework
• For typed languages (TypeScript, Java), use a typed model/dataclass; for Python use a dataclass or dict
• Must not hard-code environment values anywhere except the JSON files

4. Page Object Model (POM)

Create a BasePage class with shared utilities: navigate(), waitForElement(), getTitle(), etc. Each feature page extends BasePage. Page classes must encapsulate all selectors and interaction methods. No raw selectors or direct browser calls should appear in test files.

Implement the following pages based on Swag Labs:

LoginPage (https://www.saucedemo.com/)

• Selectors: #user-name, #password, #login-button, .error-message-container
• Methods: login(username, password), getErrorMessage(), isLoginPageDisplayed()

InventoryPage (/inventory.html)

• Selectors: .inventory_list, .inventory_item, .inventory_item_name, .btn_inventory, .shopping_cart_badge, [data-test="product-sort-container"]
• Methods: getProductCount(), addProductToCartByName(name), removeProductFromCartByName(name), sortProductsBy(option), getCartBadgeCount(), openProductByName(name)

ProductDetailPage (/inventory-item.html)

• Selectors: .inventory_details_name, .inventory_details_price, .inventory_details_desc, #add-to-cart, #back-to-products
• Methods: getProductName(), getProductPrice(), addToCart(), goBackToInventory()

CartPage (/cart.html)

• Selectors: .cart_item, .cart_item_label, .cart_quantity, #checkout, #continue-shopping, #remove
• Methods: getCartItems(), removeItemByName(name), proceedToCheckout(), continueShopping()

CheckoutInfoPage (/checkout-step-one.html)

• Selectors: #first-name, #last-name, #postal-code, #continue, #cancel, .error-message-container
• Methods: fillShippingInfo(firstName, lastName, postalCode), continue(), cancel(), getErrorMessage()

CheckoutOverviewPage (/checkout-step-two.html)

• Selectors: .cart_item, .summary_subtotal_label, .summary_tax_label, .summary_total_label, #finish, #cancel
• Methods: getOrderItems(), getSubtotal(), getTax(), getTotal(), finishCheckout(), cancel()

CheckoutCompletePage (/checkout-complete.html)

• Selectors: .complete-header, .complete-text, #back-to-products
• Methods: getConfirmationHeader(), getConfirmationText(), backToProducts()

5. JSON Test Data Readers

Create utilities/json_reader.[ext] using idiomatic patterns for the chosen language:

• Reads test data from test-data/*.json files
• Returns typed/structured data objects (dataclass, POJO, interface, dict — whichever fits the language)
• Supports reading by key (e.g. get_data('login', 'validUser') in Python; getData("login", "validUser") in Java/TS)

Generate the following test data files with real Swag Labs data:

test-data/login.data.json

{
  "validUser": { "username": "standard_user", "password": "secret_sauce" },
  "lockedUser": { "username": "locked_out_user", "password": "secret_sauce" },
  "problemUser": { "username": "problem_user", "password": "secret_sauce" },
  "performanceUser": { "username": "performance_glitch_user", "password": "secret_sauce" }
}

test-data/inventory.data.json

{
  "expectedProductCount": 6,
  "firstProduct": "Sauce Labs Backpack",
  "sortOptions": ["az", "za", "lohi", "hilo"]
}

test-data/checkout.data.json

{
  "shippingInfo": {
    "note": "Fields below are placeholders — runtime values are generated via faker_helper",
    "firstName": "FAKER",
    "lastName": "FAKER",
    "postalCode": "FAKER"
  }
}

5b. Faker — Dynamic Test Data

Create utilities/faker_helper.[ext] as the single Faker wrapper for the entire framework.

Language library to use:

Methods to implement:

Requirements:

• All faker calls must go through faker_helper — no direct faker imports in tests or page objects
• Seed faker with a fixed value when FAKER_SEED environment variable is set, to allow reproducible runs in CI
• Log each generated value at DEBUG level (e.g. DEBUG: Generated firstName = "Alice")
• Update CheckoutInfoPage.fillShippingInfo() to accept data from faker_helper or from JSON — both paths must work
• In tests/checkout_test.[ext], use faker_helper to generate shipping info dynamically instead of static JSON values

5c. Date Utils

Create utilities/date_utils.[ext] with the following methods:

Language equivalents:

Requirements:

• getCurrentTimestamp() must be used by screenshot_helper and logger for all file naming — no inline new Date() calls elsewhere
• Format strings must follow ISO 8601 by default; accept an optional override
• All methods must be pure functions (no side effects)
• Provide unit-testable examples for each method in comments

5d. String Utils

Create utilities/string_utils.[ext] with the following methods:

Usage in Swag Labs context:

• Use stripCurrencySymbol + toNumber in CheckoutOverviewPage to assert subtotal/tax/total arithmetic
• Use sanitize in every Page Object getter that returns UI text before it is used in an assertion
• Use contains in LoginPage.getErrorMessage() assertions
• Use truncate in logger when logging long page titles or URLs

Requirements:

• All methods must be pure functions with no side effects
• No method should throw on null/undefined input — return a safe default instead
• Import string_utils in BasePage so all page objects inherit access without separate imports

6. Screenshots

Create utilities/screenshot_helper.[ext]:

• Capture full-page screenshot with a timestamped filename
• Auto-capture screenshot on test failure
• Save under test-results/screenshots/[test-name]-[timestamp].png
• Hook into the failure handler appropriate for the chosen tool/language (e.g. afterEach / onTestFailed / pytest fixture / JUnit @AfterEach)

7. Setup & Teardown Fixtures

Create fixture files under fixtures/ covering all three scopes — global, suite, and test level:

Global (run once per entire test session):

• Launch browser / WebDriver instance
• Load environment config
• Initialise any shared state (e.g. authenticated session token)

Suite-level (run once per test file/class):

• Navigate to the starting URL
• Seed or reset test data where needed
• Log in and store session/cookie state to avoid repeated logins

Test-level (run before & after every individual test):

• Create a fresh page/context per test to ensure isolation
• Inject Page Object instances so tests receive ready-to-use objects
• Capture screenshot and attach to report on failure (afterEach / teardown)
• Clear cookies, local storage, or DB state after each test

Language equivalents to implement:

Requirements:

• Base fixture must be extended/inherited by all feature fixtures — no duplication
• Fixtures must accept the active environment config so base URL and credentials are injected, not hard-coded
• Screenshot-on-failure must be wired inside the teardown, not inside individual tests
• Provide a working example fixture file and show how a test imports and uses it

8. Logger

Create utilities/logger.[ext] as the single shared logger for the entire framework:

Log levels to support: DEBUG, INFO, WARN, ERROR

• INFO — test step progress, navigation events, config values loaded
• DEBUG — raw request/response data, element lookup details
• WARN — retried actions, slow responses, skipped steps
• ERROR — exceptions, assertion failures, unexpected states

Output targets:

• Console (stdout) with coloured level labels when running locally
• Rolling file output to logs/test-run-[timestamp].log
• Log level controlled by a LOG_LEVEL environment variable; default to INFO

Language library to use:

Requirements:

• Single logger instance imported across all modules — no per-file logger setup
• Logger must be initialised once during global setup (tie into the base fixture)
• Log every test start and end (name, status) automatically from the fixture — not from individual tests
• Log config values on startup (mask credentials — never log passwords or tokens in plain text)
• Page Object methods must call logger.info("Performing action: ...") at each step
• On test failure, log the error message and stack trace at ERROR level before the screenshot is taken
• Provide a working example showing logger import and usage in a Page Object and in a fixture

9. Sample Tests

Write the following specs using real Swag Labs flows:

tests/login_test.[ext]

1. Successful login with standard_user → lands on /inventory.html
2. Login with locked_out_user → error message "Sorry, this user has been locked out."
3. Login with empty username/password → appropriate validation error

tests/inventory_test.[ext]

1. Verify 6 products are displayed after login
2. Add "Sauce Labs Backpack" to cart → cart badge shows 1
3. Sort products by price low-to-high → verify first item is cheapest

tests/checkout_test.[ext]

1. Full end-to-end: login → add product → cart → fill shipping info using faker_helper (random first name, last name, postal code generated at runtime) → overview → finish → confirm order complete

All specs must:

• Use a fixture for browser/driver setup and teardown
• Use Page Object methods (no raw selectors in tests)
• Read credentials and test data from JSON files
• Use baseUrl from environment config
• Have logger logging test start, each step, and any failure
• Auto-capture screenshot on failure via the teardown fixture
• Tag every test with @smoke or @regression (or both) so CI pipelines can filter by suite

Tagging convention per language:

Login tests 1 & 2 → @smoke + @regression; login test 3 → @regression only All inventory tests → @regression; checkout E2E → @smoke + @regression

10. CI/CD Pipelines

10a. Test Tagging — Run Commands

Document the exact CLI commands to run tagged suites in README comments inside the config file:

# Smoke only
[tool-specific tag filter command for @smoke]

# Regression only  
[tool-specific tag filter command for @regression]

# All tests
[tool-specific run all command]

10b. Jenkins Pipeline

Create ci/Jenkinsfile as a declarative pipeline with the following stages:

pipeline {
  agent any

  parameters {
    choice(name: 'TEST_SUITE', choices: ['smoke', 'regression', 'all'], description: 'Which test suite to run')
    choice(name: 'ENV', choices: ['dev', 'staging', 'prod'], description: 'Target environment')
  }

  environment {
    ENV          = "{{params_ENV}}"
    LOG_LEVEL    = 'INFO'
    FAKER_SEED   = '12345'
  }

  stages {
    stage('Checkout') { /* git checkout */ }
    stage('Install Dependencies') { /* npm ci / pip install / mvn install */ }
    stage('Run Tests') {
      /* Execute based on TEST_SUITE param:
         smoke     -> run only @smoke tagged tests
         regression -> run only @regression tagged tests
         all       -> run full suite */
    }
    stage('Publish Report') {
      /* Publish HTML test report using publishHTML plugin */
    }
  }

  post {
    always {
      archiveArtifacts artifacts: 'test-results/**/*,logs/**/*,playwright-report/**/*', allowEmptyArchive: true
      junit allowEmptyResults: true, testResults: 'test-results/**/*.xml'
    }
    failure {
      /* Optional: send email or Slack notification */
    }
  }
}

Requirements:

• TEST_SUITE and ENV must be build parameters, not hard-coded
• Install step must use a lock/frozen dependency file (package-lock.json, requirements.txt, pom.xml)
• Artifacts must include: HTML report, screenshots, log files, JUnit XML results
• Pipeline must work on both Linux agents and Docker agents
• Do not store credentials in the Jenkinsfile — use Jenkins Credentials binding or environment variables injected at runtime

10c. GitHub Actions Workflow

Create .github/workflows/test.yml with the following structure:

name: Test Suite

on:
  push:
    branches: [main, develop]
  pull_request:
    branches: [main]
  workflow_dispatch:
    inputs:
      test_suite:
        description: 'Test suite to run'
        required: true
        default: 'smoke'
        type: choice
        options: [smoke, regression, all]
      environment:
        description: 'Target environment'
        required: true
        default: 'dev'
        type: choice
        options: [dev, staging, prod]

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - name: Checkout code
      - name: Set up [LANGUAGE runtime]   # Node / Python / Java
      - name: Install dependencies
      - name: Run smoke tests             # when test_suite == smoke
      - name: Run regression tests        # when test_suite == regression
      - name: Run all tests               # when test_suite == all
      - name: Upload test artifacts
        uses: actions/upload-artifact@v4
        if: always()
        with:
          name: test-results-${{ github.run_id }}
          path: |
            test-results/
            logs/
            playwright-report/
          retention-days: 14
      - name: Publish HTML Report         # Optional: use peaceiris/actions-gh-pages or similar

Requirements:

• workflow_dispatch inputs must mirror Jenkins parameters (suite + environment)
• On push / pull_request default to running smoke against dev
• Use if: always() on the artifact upload step so results are uploaded even on test failure
• Environment variables (ENV, LOG_LEVEL, FAKER_SEED) must be set from workflow inputs, not hard-coded
• Secrets (credentials, API keys) must be read from GitHub Actions Secrets (${{ secrets.SECRET_NAME }})
• Artifact retention set to 14 days
• Matrix strategy is optional but include a commented-out example for multi-browser runs

Implementation Notes

• Use strict typing where the language supports it (TypeScript interfaces, Java POJOs, Python dataclasses)
• Prefer async/await for JS/TS; use synchronous patterns where idiomatic (Python, Java)
• Use the language's standard dependency manager: npm/yarn for JS/TS, pip/poetry for Python, Maven/Gradle for Java
• All config and data file paths must be resolved relative to the project root
• Do not hard-code URLs, credentials, or selectors in test files
• Follow naming conventions of the chosen language (camelCase for TS/Java, snake_case for Python)
• Add logs/ to .gitignore; never commit log files
• Never log sensitive values (passwords, tokens, PII) in plain text — mask or omit them
• Set FAKER_SEED in CI environments for reproducible faker-generated data; leave unset locally for true randomness

Generate all files with complete, working code. Add // TODO: comments where the user must supply application-specific selectors or values.