Imported from jeremylongshore/tons-of-skills-marketplace (skills/.curated/exa-prod-checklist/SKILL.md). Install upstream with npx skills add jeremylongshore/tons-of-skills-marketplace --skill exa-prod-checklist. Copyright stays with the author (MIT).

Exa Production Checklist

Prerequisites

• A named service/policy/data owner, approved model/provider route, production rollback plan, and passing staging evidence.

Instructions

1. Complete evidence gates for identity/secrets, data handling, policy controls, source/citation requirements, observability, rate limits, and rollback.
2. Validate an approved non-sensitive canary and verify its expected routing/guardrail behavior.
3. Block release when any security, policy, data, ownership, or recovery gate is unverified.

Output

• A production-readiness receipt with evidence, owners, exceptions, canary result, and rollback path.

Examples

Release an approved staging configuration to a small canary using sanitized queries, verify policy/source/citation and alert behavior, then observe the defined window. Restore the previous configuration if any guardrail or SLO fails; do not widen automation to conceal a failure.

Overview

Complete checklist for deploying Exa search integrations to production. Covers API key management, error handling verification, performance baselines, monitoring, and rollback procedures.

Pre-Deployment Checklist

Security

• Production API key stored in secret manager (not env file)
• Different API keys for dev/staging/production
• .env files in .gitignore
• Git history scanned for accidentally committed keys
• API key has minimal scopes needed

Code Quality

• All tests passing (unit + integration)
• No hardcoded API keys or URLs
• Error handling covers all Exa HTTP codes (400, 401, 402, 403, 429, 5xx)
• requestId captured from error responses
• Rate limiting/exponential backoff implemented
• Content moderation enabled (moderation: true) for user-facing search

Performance

• Search type appropriate for latency SLO (fast/auto/neural)
• numResults minimized per use case (3-5 for most)
• maxCharacters set on text and highlights
• Result caching enabled (LRU or Redis)
• Request queue with concurrency limit (respect 10 QPS default)

Monitoring

• Search latency histogram instrumented
• Error rate counter by status code
• Cache hit/miss rate tracked
• Daily search volume tracked (for budget)
• Alerts configured for latency > 3s, error rate > 5%

Deploy Procedure

Step 1: Pre-Flight Verification

set -euo pipefail
echo "=== Exa Pre-Flight ==="

# 1. Verify production API key works
HTTP_CODE=$(curl -s -o /dev/null -w "%{http_code}" \
  -X POST https://api.exa.ai/search \
  -H "x-api-key: $EXA_API_KEY_PROD" \
  -H "Content-Type: application/json" \
  -d '{"query":"pre-flight check","numResults":1}')
echo "API Status: $HTTP_CODE"
[ "$HTTP_CODE" = "200" ] || { echo "FAIL: API key invalid"; exit 1; }

# 2. Verify tests pass
npm test || { echo "FAIL: Tests failing"; exit 1; }

echo "Pre-flight PASSED"

Step 2: Health Check Endpoint

import Exa from "exa-js";

const exa = new Exa(process.env.EXA_API_KEY);

app.get("/health/exa", async (_req, res) => {
  const start = performance.now();
  try {
    const result = await exa.search("health check", { numResults: 1 });
    const latencyMs = Math.round(performance.now() - start);
    res.json({
      status: "healthy",
      latencyMs,
      resultCount: result.results.length,
      timestamp: new Date().toISOString(),
    });
  } catch (err: any) {
    res.status(503).json({
      status: "unhealthy",
      error: err.message,
      errorCode: err.status,
      latencyMs: Math.round(performance.now() - start),
    });
  }
});

Step 3: Gradual Rollout

set -euo pipefail
# Deploy canary (10% traffic)
kubectl apply -f k8s/production.yaml
kubectl rollout pause deployment/exa-service

echo "Canary deployed. Monitor for 10 minutes..."
echo "Check: /health/exa endpoint, error rates, latency"

# After monitoring, resume to full rollout
# kubectl rollout resume deployment/exa-service

Post-Deployment Verification

set -euo pipefail
# Verify production endpoint
curl -sf https://your-app.com/health/exa | python3 -m json.tool

# Check error rates (if Prometheus available)
curl -s "localhost:9090/api/v1/query?query=rate(exa_search_error[5m])" 2>/dev/null

Rollback Procedure

set -euo pipefail
# Immediate rollback
kubectl rollout undo deployment/exa-service
kubectl rollout status deployment/exa-service
echo "Rollback complete. Verify /health/exa endpoint."

Alert Thresholds

Error Handling

Resources

• Exa API Documentation
• Exa Error Codes

Next Steps

For version upgrades, see exa-upgrade-migration. For incident response, see exa-incident-runbook.