core

Instruction file imported from idfortytwo/car-aggregator (.github/instructions/core.instructions.md). Copyright stays with the author.

Core

Read docs/architecture.md before changing anything here. Core is shared infrastructure: a change here affects every source and both interfaces. Prefer solving a problem in a plugin over changing core.

Database

• All SQL lives in core/db/repo.py. Nowhere else executes SQL, ever.
• Always parameter-bound (cur.execute("... WHERE id = ?", (listing_id,))). String interpolation into SQL is never acceptable.
• One function per query, named for intent: insert_listing, get_unnotified_encumbered.
• Schema changes are new numbered files in core/db/migrations/. Never edit an applied migration.
• UNIQUE(source_id, external_id) on listings and UNIQUE(listing_id, channel) on notifications are correctness guarantees, not optimizations. The second one is what prevents duplicate pings — do not drop it or defeat it with a blind ON CONFLICT.
• Timestamps: ISO-8601 UTC strings. verdicts is append-only history keyed by content_hash + prompt_version.

Pipeline

Order is fixed: fetch → normalize → dedup → stage 1 prefilter → stage 2 judge → persist → notify. Steps after dedup must stay pure functions of the ad text plus prompt version, so reclassification is reproducible without re-fetching.

• Dedup: unchanged content_hash → touch last_seen_at and stop. Never re-judge unchanged text (it costs money and risks a duplicate notification).
• Only Verdict.ENCUMBERED reaches a notifier.
• Write the notifications row after a confirmed successful send, never before.

Failure handling

A failing source must never crash the scheduler. Catch at the source boundary, store the message in sources.last_error, increment consecutive_errors, back off exponentially, continue with other sources. Failures must be visible in the web UI — never swallow an error into an empty result.

except Exception is allowed only at the two designed boundaries (per-listing parse, per-source run), always with exc_info=True. Everywhere else, catch narrowly.

Service layer

core/service.py is the only thing web/ and bot/ may call. Its functions take typed arguments and return typed data — no framework objects (Request, Update) in signatures, so they are testable with no server and no bot running.

record_feedback writes to feedback and appends a case to evals/cases.yaml. That loop is how the classifier improves; keep it intact.

General

• Type-annotate everything; mypy must pass on core/.
• Module logger, never print(). Never log secrets, tokens, or full headers.
• Async in the fetch/pipeline path; no blocking I/O inside async def (use asyncio.to_thread if unavoidable).
• Text normalization happens once, in core/text.py. Keep the original for display and the normalized form for hashing and matching; never .lower() text shown to the user.