Imported from giveitlegs/vacationdeals.to (
.claude/skills/vacdeals-evergreen/SKILL.md). Install upstream withnpx skills add giveitlegs/vacationdeals.to --skill vacdeals-evergreen. Copyright stays with the author.
VacationDeals.to — Evergreen Operations Runbook
Run these phases IN ORDER. Obey the guardrails, then report a consolidated summary.
GUARDRAILS (learned the hard way)
- Agent waves ≤ 5 at a time. A 13-agent batch was killed mid-run by a session limit (2026-07-30). Dispatch writer/research/rate swarms in waves of ≤5, insert + commit each wave before the next. Agents must write once and stop (no self-expansion loops — those burned the tokens that tripped the limit).
- Guarded deploys: always
pnpm buildon the VPS BEFOREpm2 restart. If the build fails, do NOT restart — the site stays up on the old build. Deploy recipe:cd /var/www/vacationdeals && git pull -q origin main && set -a && source .env && set +a && pnpm build && pm2 restart vacationdeals-web --update-env. - Cost: DataForSEO creds have been 401 (refresh at app.dataforseo.com); treat volume data as optional. The only metered spend is the AI-reviews cron (guarded to 2 runs/day) + any DataForSEO. Do not exceed a $25 all-in run without asking.
- Programmatically re-validate agent output (word floors, FAQ counts, unique slugs) — a writer once shipped 10 under-length posts while claiming it validated them.
PHASE 1 — Full 3-layer backup (crown-jewel = deal_price_history)
ssh root@72.60.126.82 "/usr/local/bin/vacdeals-db-backup.sh >> /var/log/vacdeals-dbbackup.log 2>&1 && ls -t /root/db-backups/*.pgdump | head -1"
# then scp the newest dump to backups/db/ (local Dropbox mirror — it drifts stale)
# then: bash scripts/backup-to-private-github.sh (private GitHub repo)
Verify deal_price_history row count + span before/after (it should only grow).
PHASE 2 — Recrawl all sources + maintenance
ssh root@72.60.126.82 "nohup /var/www/vacationdeals/scripts/run-with-env.sh /usr/bin/npx tsx src/scrape-wave.ts --wave=all > /var/log/vacdeals-fullscrape-$(date +%Y%m%d).log 2>&1 &"
# wait for all 5 waves green, then run maintenance (paths are relative to apps/scraper):
# run-with-env.sh npx tsx src/check-deal-health.ts (zombie sweep + dead-URL)
# run-with-env.sh npx tsx ../../scripts/nightly-data-quality.ts --fix
# run-with-env.sh npx tsx src/verify-prices.ts --limit=50
PHASE 3 — Rate spot-check swarm (≤5 agents)
Dump active deals to a scratch PSV, then dispatch ≤5 agents grouped by source.
Each samples deals across its sources, WebFetches the partner lander (curl fallback
for JS/403), and returns a table: deal_id|scraper_key|db_price|live_price|verdict|notes
(MATCH/MISMATCH/GONE/FETCH_BLOCKED). Fix confirmed data errors in the DB (stale
prices, credit-as-price, expired/repurposed URLs, placeholder original_price=50000).
Target ≥20 rates checked. Confirms scrapers work AND published price == brand-site price.
Known-walled sources (curl/WebFetch always FETCH_BLOCKED): vegas-timeshare
(sgcaptcha), holiday-inn (403 WAF), hyatt (JS-rendered price), branson-travel-group
(Cloudflare/sgcaptcha). Their CRON scrapers DO work (Playwright/desktop-UA paths) — as of
2026-09-23 all four had fresh active deals — so FETCH_BLOCKED here is a verification gap,
NOT a data problem. Don't deactivate them on FETCH_BLOCKED alone. To actually verify their
prices, give the agent handling that chunk the stealth-browser-mcp tools (spawn_browser →
navigate → get_page_content), which pass these challenges; fall back to reading the DB
scraped_at freshness as a liveness proxy when the browser isn't available.
PHASE 4 — Deep-research swarm: new vacpack sites (≤2 agents)
Dispatch research agents to find NEW timeshare-preview sites we don't scrape (exclude
the ~48 current sources — see CLAUDE.md/scraper list). Each returns a ranked list:
url | example packages+prices | presentation required? | Cheerio vs browser | bot-blocking.
Relay the ranked candidates; building crawlers is a follow-up (don't auto-build).
PHASE 5 — 50 bespoke Google-Discover blog posts (≤5 agents, last wave)
Follow the STYLE-SPEC in research/blog-batches/weird-batch-2026-07/STYLE-SPEC.md
(BLUF box, 8-10 AEO FAQs, humanization in BODY ONLY, casual first-person, real
vacpack data). NEW angles each time — check existing slugs first
(SELECT slug FROM blog_posts) to avoid dupes. Discover-oriented = curiosity-gap
titles, timely/story hooks, strong first image; lightly SEO'd (one keyword woven in).
Author as JSON to a new research/blog-batches/<batch>/ dir, validate (unique slugs,
≥900 on-page words incl. FAQs, ≥6 FAQs), commit + push, insert on the VPS, rebuild
(guarded) so the sitemap picks them up, verify a sample renders 200 + FAQPage schema.
Insert command gotcha: insert-blog-batch-json.ts imports the @vacationdeals/db
workspace pkg. npx tsx AND pnpm exec tsx from the repo root BOTH fail
(MODULE_NOT_FOUND / "tsx not found"). Run it from apps/scraper (which has both tsx
and the db symlink) with ABSOLUTE paths:
cd /var/www/vacationdeals/apps/scraper && set -a && source /var/www/vacationdeals/.env && set +a && ./node_modules/.bin/tsx /var/www/vacationdeals/scripts/insert-blog-batch-json.ts /var/www/vacationdeals/research/blog-batches/<batch> [--dry-run].
Always --dry-run first (validates required fields, ≥5 FAQs, category enum, skips
existing slugs), then run for real.
Sitemap-cache trap (learned 2026-08-11): sitemap.ts has revalidate=3600, so new
posts appear in /sitemap.xml within the hour on their own — DO NOT try to force it by
deleting .next/server/app/sitemap.xml*; that removes the compiled route and makes the
sitemap 500 (MODULE_NOT_FOUND), needing a full TURBO_FORCE=true pnpm build to recover.
If you must refresh it now, a normal guarded rebuild + pm2 restart regenerates it
cleanly. The per-slug blog pages go live immediately (fresh ISR), only the sitemap lags.
FINISH
- Update
docs/NEXT-ENHANCEMENTS.md(dated entry) + relevant memory. - Report a consolidated summary: backup status, recrawl/maintenance results, rate spot-check verdicts + fixes, new-site candidates, blogs published + live count.