Imported from mkreyman/loopctl (.claude/skills/knowledge-wiki/SKILL.md). Install upstream with npx skills add mkreyman/loopctl --skill knowledge-wiki. Copyright stays with the author.

name: knowledge-wiki description: Use when working on loopctl's knowledge/retrieval product — the shared Knowledge Wiki, agent memory, the context retriever, hybrid (curated + RAG) search, the novelty/dedup gate, embeddings, conflict resolution, or KB curation permissions. Covers the four agent information surfaces and how to pick between them. Triggers on: knowledge, wiki, knowledge_create, knowledge_search, hybrid_search, novelty, dedup, proposal gate, embedding, vector search, conflict, curation, memory_remember, memory_recall, retrieve_, context retriever, entity, RAG, progressive disclosure, OKF, corpus_search, corpus tier, verbatim text, source_ref.

Knowledge Wiki & Retrieval Surfaces

The knowledge/retrieval stack is loopctl's product core (it is the second brain). It is large (lib/loopctl/knowledge/) and split across four distinct agent-facing surfaces plus a hybrid retrieval capability. This skill routes you to the right surface and the load-bearing invariants; full references live in docs/agent-memory.md, docs/context-retriever.md, docs/knowledge-hybrid-retrieval.md, docs/user_stories/epic_43_corpus_tier/README.md. AGENTS.md carries the same four-surface routing for quick recall.

Four surfaces — pick by WHAT THE DATA IS

Rule of thumb: quote me the exact text of an authoritative document? → corpus_search, then open the file it points at. what did we learn about X? → knowledge_search. live structured business row? → retrieve_*. worth another agent reading? → knowledge_create. a fact only I need? → memory_remember. Scope is key-derived server-side — you never pass tenant_id/subject_id.

knowledge_search never covers the corpus tables, so an empty wiki result says NOTHING about whether a document is indexed — check corpus_list before concluding it is not. And corpus_search is deliberately absent from /api/v1/recall: verbatim spec chunks auto-injected into every session are exactly the pollution the separate tables prevent.

Invariants (cited)

1. Novelty / dedup gate on create — Knowledge.propose_article/3 → the private gate_proposal/4 (both in knowledge.ex; the four gate_proposal/4 clauses match on assessment.verdict). SIX outcomes, not four — :duplicate, :low_novelty, :unknown, :novel, :deduplicated (created: false, returned when an idempotency-key match is resolved from the existing row — the %Article{} = existing -> deduplicated_result(existing, assessment) clause in skip_low_novelty/4's else block, and create_article/3's own key lookup), and :skipped_low_novelty (created: false, article may be nil). An idempotency-key dedup is deliberately a 200 no-op rather than a 409 on a changed body — the fleet's harvest sourcers re-run with stable keys BY DESIGN, so refusing would break every harvest — so the API SIGNALS the discard instead: Knowledge.dedup_drift/2 compares the submitted payload against the stored row on the SAME trim normalization same_content?/2 uses, and EVERY deduplicated: true render emits content_drift/title_drift — the idempotency and title-collision branches via ArticleController.dedup_response/3, and the gate's :duplicate verdict in its own render_proposal/4 clause, which is the DEFAULT create path and the one that shipped without them. Three asymmetries that are easy to get wrong: an ABSENT body key is not drift but a PRESENT non-string one is (the key path short-circuits before Article.create_changeset/2, so a broken extraction is never validated and must not be answered "in sync"); title_drift is SUPPRESSED when the submitted title equals previous_title, because the nightly :generic_title retitle moved the STORED side and a compliant sourcer would otherwise PATCH the placeholder back every night. That condition rides on a COLUMN castable from nowhere: Article.update_changeset/2 CLEARS previous_title on any ordinary title edit, so the suppression ends the moment a human curates the title and cannot be re-armed or erased through metadata (the consolidation_title_generated marker is advisory — one metadata PATCH replaces the whole map, so nothing behavioural may read it); and the note says READ before it says update, because drift is symmetric and only one of the two sides is the caller's. The gate's :duplicate branch takes neighbor_drift_note/2 INSTEAD: its row is a near-NEIGHBOUR matched by SIMILARITY — VectorSearch.nearest/4 has NO self-exclusion, so it may be a stranger's article or the caller's own earlier capture — so the flags are true by construction and the note says read-then-decide: PATCH only your own prior capture, else merge or re-send under a different title with force: true (the same title answers 409 title_conflict) — and for the same reason that branch emits neither the Logger.warning nor drift telemetry metadata. It reports only; nothing in the create path branches on it. The DURABLE trace of a drifted discard is that Logger.warning; the flags also ride the :deduplicated write telemetry as metadata (never a new outcome atom — IngestionWriteStats.column_for/1 SKIPS unmapped outcomes), but the ingestion_write_stats rollup does NOT break them out, so drift cannot be counted from it. A caller matching only the first four falls through on either of the last two, both reachable. :duplicate returns the canonical neighbor without creating; :low_novelty is created but forced to status: "draft" (the :low_novelty gate_proposal/4 clause) with novelty stamped into metadata.proposal_novelty (stamp_proposal_metadata/2) so a smarter consumer decides — UNLESS the caller passes on_low_novelty: :skip (for an UNATTENDED writer whose drafts nothing would review), which creates NOTHING and returns :skipped_low_novelty with the near-neighbor (skip_low_novelty/4). That skip is decided LAST: an invalid project_id, an idempotency_key match, and an exact active-title collision are all still answered normally rather than dropped. Two branches that are easy to miss: :duplicate falls through to create if the canonical neighbor vanished between assess and now (the :error arm of its canonical_neighbor/3 case), and :unknown creates only BY DEFAULT — a caller passing on_gate_unavailable: :skip gets {:error, :gate_unavailable} and nothing is created (the :unknown clause). The assessor is config-injected (proposal_assessor/0 → Loopctl.Knowledge.ProposalGate) — do not hardcode it.

2. Hybrid search provenance — Loopctl.Knowledge.hybrid_search/3 (in knowledge.ex). :curated wins ONLY when a governed curated source's absolute (never pool-relative) confidence (absolute_score/1) clears a scale-matched threshold AND beats the best retrieved candidate by a margin (hybrid_curated_threshold_and_margin/1; the pure decision is resolve_provenance/4) AND is authoritative (not superseded/conflicted — the caller passes only list_curated_sources/2-filtered scores). Otherwise :retrieved. Both branches return identical results/meta key sets — callers branch on meta.provenance alone. A sparse pool must never let a near-but-wrong curated doc win.

3. All heavy KB reads route through Loopctl.HeavyRead — semantic search, novelty, suggest-links, distant-pairs, enumeration (knowledge.ex:12). See the tenancy-rls skill for the pool/pgbouncer reasoning; never run these on Repo/AdminRepo.

4. KB-content curation is agent-role because it is NON-DESTRUCTIVE + audited (#331) — knowledge_create, knowledge_update (ID-preserving in-place edit), knowledge_archive/knowledge_delete (soft delete → status: :archived, row retained), and knowledge_resolve_conflict in all dispositions. Non-destructive is not the same as reversible, and archive is the case that separates them (#605/#606): :archived is TERMINAL — Article's @valid_transitions has no {:archived, _} and there is no unarchive function, so the only way back is a user+ PATCH with an explicit status. Nothing is destroyed and everything is audited, which is what earns agent role; nothing automated restores it, which is why an unattended writer must reach for unpublish ({:published, :draft}, undone by publish) instead — or now for suppress, the REVERSIBLE retrieval tombstone (knowledge_suppress/knowledge_unsuppress, POST /api/v1/articles/:id/suppress, Knowledge.suppress_article/3, agent role). Suppression sets articles.suppressed_at plus a required reason and an actor, leaves status, body, embedding and links untouched, keeps the article resolvable BY ID (get_article/3, knowledge_get) so the act is inspectable, and excludes it from every ranked read path through the one predicate in Loopctl.Knowledge.Suppression. When you add a read path, that predicate is not optional and not your judgement call: test/loopctl/knowledge/suppression_guard_test.exs scans lib/ for published-status filter sites and fails on any that neither applies it nor is named in Suppression.exempt_sites/0 with a category and a reason. Pick between the three retraction verbs by what you need AFTERWARDS — suppress (undoable, silent about status), unpublish (undoable, but asserts the article is a draft), archive (not undoable by any agent call). The :user set is single-article unpublish plus ALL the SET-BASED bulk ops — bulk_publish, bulk_unpublish and the ENTIRE bulk_delete action, soft path included (article_workflow_controller.ex:59-61). Both criteria matter: set-based blast radius (one call mutates an unbounded set) AND irreversibility (bulk_delete carries a hard-delete path) — see the controller @moduledoc (:13-21). Single-article ops are agent-role precisely BECAUSE nothing they do destroys a row, so never drop that property when reasoning about a new op. drafts/publish are :orchestrator (:55). bulk_delete takes no model-visible confirm argument, on either path (#779). A confirm flag is authorization the caller writes for itself: the same request that asks for the mutation carries its own approval, so nothing outside the caller ever sees the proposal. Both high-blast-radius paths — the irreversible HARD delete over any selector, and the SOFT archive of a tag selector — return a server-minted proposal the caller REPLAYS instead: a dry_run freezes the id-set into a single-use, TTL-bounded, tenant-scoped, TYPED BulkDeleteToken, and the run executes exactly that frozen set. The type is what stops an archive proposal being replayed as a delete, or the reverse, and on BOTH ops it carries a keyed digest of the SELECTOR so a token is not spendable on a set the caller never named; over the frozen bound, where there is no token, the confirm_hash is keyed on the OP for the same reason. A request carrying confirm is 400 confirm_removed, refused rather than ignored; a tag call with neither dry_run nor a replay credential is 400 dry_run_required (a selector matching nothing needs no proposal — it stays a 200 no-op on either path). The article_ids and source archives are unchanged — each names a set the caller already holds. When adding a destructive op here, mint a proposal; never add a confirm flag. Agent edits are visibility-scoped: an agent can only touch an article it can see. (See chain-of-custody.) Recording a verdict is agent-role; AUTHORIZING the unattended RETIREMENT is not. The conflict PAIR is manufacturable — the queue is fed by a mechanical similarity threshold — so annotate_conflict/3 GRANTS a :supersede's confidence from the recorder's role (grant_confidence/3, :actor_role resolved server-side from the key) instead of accepting it from params: an agent asking for :high is recorded at :medium with the ask kept in requested_confidence, and a :high supersede must carry evidence. :merge is NEVER capped — it synthesizes a new DRAFT and retires nothing, so capping it disabled the disposition and bought no safety. Scope any new gate to what it actually protects. A verdict nothing will act on does NOT settle the pair: conflict_unresolved_subquery/0 settles on executable_resolution/0, so a capped or unattributed row leaves its pair in GET /knowledge/conflicts to be re-recorded. A pair the system never flagged is REACHABLE but not self-judgeable (#730). Knowledge.assert_conflict/3 (POST /api/v1/knowledge/conflicts, agent+) creates the :potential_conflict link a caller cannot create directly, stamped auto_generated: false, asserted: true and carrying a REQUIRED evidence — the case it exists for is a session that just wrote a correction, whose pair is minutes old and may never be similar enough to be auto-flagged. Three properties keep the kb-02 guard intact, and each has a mutation-verified test: an assertion never reaches curated suppression (open_conflict_subquery/1 and article_in_open_conflict?/2 still require auto_generated, or an agent could retract any article from the governed answer path by disputing it); the asserter may not record the verdict (validate_not_self_asserted/2 → 409 self_asserted_conflict, fail-closed on an unknown recorder, and refusing an ANCESTOR/DESCENDANT dispatch of the asserting one — siblings are separation, exactly as the L4 gates read lineage; re-checked in apply_flagged_resolution/3 on the PRINCIPALS stamped at assert and verdict time, with the audit label evaluated IN ADDITION, never as an else-branch, since only the LAST verdict's principal is on the row); and an assertion never overwrites a system flag — fetch_conflict_flag/3 prefers auto_generated on a tie. Pre-settling is closed on BOTH sides: the self-refusal covers every disposition, and conflict_unresolved_subquery/0 — the QUEUE only — settles a flag with a verdict that POSTDATES it, so two principals cannot dismiss a pair against a genuine system flag raised over it later. Curated suppression deliberately keeps the older predicate-free judged_pair_subquery/0: the automatic drain skips any pair that already carries a verdict row, so a re-flagged judged pair counted as unjudged would withhold both articles with nothing able to release them. Ids are cast (cast_distinct_pair/2) before any query interpolates them, and visibility is checked BEFORE existence so an invisible id and a nonexistent one are one answer. Hiding a pair behind a row that will never apply is the black hole to avoid. Corroboration covers BOTH duplicate signals (Consolidation.corroborated?/3), and the winner is the OLDEST member, not the longest. An idempotency_key AND a normalized title are both caller-controlled, so corroborating content the same party wrote proves nothing — age is the one input a later writer cannot manufacture. Scoring is keyed by {drift_signal, member_id} — a group scored under the other signal's normalized key finds nothing and withholds (fail-closed). 5b. Selection decides WHAT is returned; the deterministic sort decides the ORDER — Loopctl.Knowledge.Diversity.select/4 (#792), shared by Memory.recall_context/2 and Knowledge.hybrid_search/3. Exact content-hash dedup, containment-in-history, near-duplicate removal at cosine >= 0.95 against the ALREADY-SELECTED set (never against the query), then MMR. Four rules that must move together: every drop is REFILLED from an OVER-FETCHED pool rather than leaving a hole, and a drop that CANNOT be refilled is not taken at all (containment re-admits its highest-ranked repeats rather than returning an empty half a caller reads as "the KB has nothing"); a candidate whose vector cannot be loaded is never dropped, because unmeasurable is not similar; MMR must never be allowed to set the render order, or the cache-friendly byte-identical block it feeds is gone; and a PINNED candidate is split out ahead of every drop stage, or a curated answer can vanish from a page whose meta.provenance still names it. lambda: 1.0 reproduces the pre-#792 selection exactly — but only for a caller whose :score descends with its OWN ranking, so rank the candidates on the scale you ordered them by, never on a per-lane absolute score. Full pipeline: docs/agent-memory.md, "Diversity selection on the knowledge half".

5. Heat must not rank on a signal heat produces — Knowledge.heat_index/2 (knowledge.ex:11967; the counted set is @heat_read_access_types, :11837). The heat index is the one retrieval route that takes NO query, so its misses are uncorrelated with embedding similarity — which is worth nothing if its ordering is something a caller or the route itself generates. It has been violated FOUR times, each differently — and once by a FIX for one of the others — so treat any new input to the ranking as guilty until checked:

   • #563 counted search/context — one row per RESULT of a ranked query, so heat became a tally of past ranker output and re-coupled to the embedding similarity it exists to escape.
   • #567 counted raw event rows, so one key could pin its own article with a knowledge_get loop; then counted DISTINCT KEYS, which under v2's per-dispatch ephemeral keys counted DISPATCHES. A reader is coalesce(k.agent_id, e.api_key_id), and ties break on distinct read DAYS — never count(e.id), which hands the tie straight back to the counter a loop inflates.
   • #569 counted the hop knowledge_progressive_drill makes FROM this index — the tool its own meta.drill names — so being SHOWN produced the rank that showed it. Every drill now records the uncounted drill type. Derive that label from the READ PATH, never from a caller-declared origin — a declaration binds only the clients that send it, and an older MCP release or a raw HTTP call then re-opens the loop.
   • #572 was #569's own fix, half-applied. It exempted tenant articles only: a system canonical's drill stayed COUNTED, because get_article/3 filtered on tenant_id and the drill was the sole path to a canon body, so excluding it would have frozen every canonical at heat 0. Sound about the canon, wrong about the index — ranking a counted class against an uncounted one means one heat column measures two different things, and since drilling is the DOCUMENTED path, following the docs raised only canonicals, self-reinforcingly. get_article/3 resolves canonicals now, so the canon has a caller-named read of its own. The corollary, and the reason this keeps recurring: never rank counted and uncounted read paths on one number. When an item lacks an uncounted-origin read path, give it one — do not count the loop it already has. drill still counts in RetrievalMetrics.compute_followed_through/2, which asks whether a body was DELIVERED. The two access-type sets diverge on purpose; do not unify them. Adding a value to ArticleAccessEvent.@access_types requires the same value in Analytics.@valid_access_types — there is no DB CHECK, those two allowlists ARE the enforcement.

6. The idem- tag namespace is reserved, enforced at write time, and is NOT an idempotency mechanism — Loopctl.Knowledge.IdempotencyTag, enforced from the one place both changesets converge (validate_tag_format/2 in article.ex), so it binds every writer (API controllers, ContentIngestionWorker, ReviewKnowledgeWorker, OKF import) rather than one call site — the same reasoning as Coordination's reserved claim: key prefix. A tag claiming the prefix must be idem-<family>-<digest> with a 12- or 40-char lowercase hex digest: both eras, because the sourcers' suffix was truncated from a full sha1 to 12 and a rule that knew only the current form is the drift bug that made pre-truncation captures invisible. Malformed → 422 naming the remedy; never a silent re-prefix (rewriting a caller's tags makes the response body a lie). The 422 is what a CALLER sees on POST/PATCH /api/v1/articles. Every MACHINE path drops the tag instead, because a whole batch must not die on one string a model or a foreign document produced: OKF sanitize_tags/1 drops a tag that sanitizing COERCED into the namespace (discriminating on whether the string CHANGED — a well-formed reserved tag arriving byte-identical is the article's own capture identity and MUST survive, or a native bundle's round trip and the merge path silently delete it); ReviewKnowledgeWorker and ContentIngestionWorker strip a malformed reserved tag from extractor output pre-insert (a review's articles share one Multi whose :insert_failed changeset discards the job permanently, and an ingested article whose changeset is invalid is dropped WHOLE — body and all — before insert_all); and Memory.sanitize_graduation_tags/1 filters one out, since a failed graduation insert still STAMPS the memory graduated and burns its one shot. Any new tag rule on the Article changeset must be mirrored in those four, and in KnowledgeMocWorker's @excluded_prefixes, which suppresses idem- so a capture id can never become a published Index: hub (its match is by PREFIX: url- does NOT cover idem-url-…). The reservation is FORWARD-looking: pre-reservation tags carry the bare <family>-<digest> form with no prefix to match on, so reads need the independent shape discriminator legacy?/1, and the bare form is still WRITABLE until the client half (mkreyman/claude-config#222) adopts the reserved form. legacy?/1 requires BOTH a known source family and a hex digest — a bare <anything>-<hex> also describes commit-<sha> and release-202604150930, and promoting one of those fabricates a capture identity that --drop-legacy then makes irreversible. mix loopctl.reserve_idempotency_tags promotes the corpus (dry-run default; --drop-legacy is the second pass, only after clients switch). What this is not: a tag is caller-controlled data, so reserving its namespace is defense in depth, not authority. The server-guaranteed key is the articles.idempotency_key column with its per-tenant unique index — prefer it, and do not treat a tag as proof of capture identity.

7. The nightly consumers are watched by something OUTSIDE them (#765 item 6) — Loopctl.Knowledge.IngestionHealth.detect_consumer_stalled_scan/1, flagged by the hourly IngestionHealthWorker as :consumer_stalled anomalies. Now that drafts, :duplicate_capture, :generic_title and the conflict judge all have automatic consumers, the failure left is a pass that COMPLETES and applies nothing — byte-identical, in the audit event and the summary line, to a clean corpus. Three properties are load-bearing and easy to break while tuning it:

   • It is not hosted in KnowledgeLintWorker. The failure includes the pass dying (#761: six nights killed inside the judge, no audit event written at all), and a detector inside the pass cannot report that. For the same reason the streak has a PASS half counted over NIGHTS (consumer_pass_source_type/0) rather than only a per-class streak counted over EVENTS, which would freeze exactly when the system broke.
   • Quiet must stay quiet, but a REFUSAL is only quiet beside a DISPOSITION. A run offered nothing with every gate open neither starts nor extends a streak, and a deliberate refusal — a duplicate_groups_uncorroborated withhold, a generic_titles_abstained, a {:skip, :curated} — is subtracted from the work signal rather than counted as a queue, since the scan re-proposes those same items every night. SATURATION is the opposite reading and must stay visible: refusing EVERY offer while applying none is what a dead extraction provider or a dead vectorisation input looks like (generated_title/3 folds a provider error, a raise and an exit into abstained), so it reads as hard-blind. A candidate needs actionable work waiting, a hard-blind step (apply_failed / scan_failed / the -1 sentinel / a *_budget_exhausted that cut in before the first application / refusal saturation), or a PAUSED gate whose work is corroborated independently — by_class for the consolidation classes, DraftConsumer.tenant_ids_holding_drafts/1 (ONE read over the paused tenants named by the caller, never per tenant and never unqualified, on the 3-connection admin pool) for drafts. drain_disabled and no_embedding_key report offered: 0 whether the queue is full or empty, so treating either as evidence on its own alarms forever on healthy installs. Suspended tenants are excluded at the READ, like every sibling detector: suspension is what STOPS the pass.
   • Recovery needs POSITIVE evidence (recovered_keys), never mere absence from the candidate list. Two routes make the claim: the consumer APPLIED something, or a full window offered nothing with no gate blind and none PAUSED (the queue is demonstrably empty). A tenant with too few completed runs is UNEVALUATED rather than recovered. The empty-queue route is not optional: the auto-close is the only path that stamps last_event_at, and an operator clearing a stuck row by hand does not — resolved_episode_suppression?/1 then silences that key forever. Two windows, deliberately: the READ runs to audit_retention_days/0 so a pass dead for months is still flagged, while consumer_history_days/0 (double the longer threshold, clamped to that retention) bounds only the per-class STREAK — which is why consumer_stall_runs/0 is clamped to HALF the retention, so the derived window still fits inside the evidence.

Three dedup mechanisms, three different questions — do not read one's coverage as another's gap

They are not redundant and they are not substitutes. Before proposing that one "should" cover what another does, work out which question you are asking:

The fourth is the odd one out and the confusion is worth naming: the first three decide what enters the CORPUS, Diversity decides what enters one ANSWER. It never writes, never archives and never suppresses — the duplicates it drops stay published and will be returned by the next query that does not put them side by side. So "the novelty gate should have caught this" is not a reply to a redundant recall page, and vice versa.

Never backfill the column from the tag. The tag is shared by every atomic note of one capture — measured 2026-08-22 on the hosted corpus at 15.1 articles per capture — so a raw copy collides on articles_tenant_idempotency_key_idx for all but one row per capture. Worse than failing: any rows that DID land are then grouped by Consolidation.idempotency_drift_groups/1, classified :duplicate_capture, and auto-unpublished (#608). If you need a per-article key derived from a capture, the digest can only be a PREFIX plus a stable per-article discriminator — exactly what content_ingestion_worker.ex does with ingest:<hash>:<chunk>:<article>.

A low idempotency_key coverage number is not a defect, and #733 is the worked example. Measured 2026-08-22: the column is set on 813 of 86,431 articles (0.94%), which reads alarming until you ask which population it is supposed to cover. Every loopctl-internal writer that should set one already does — the ingestion worker, KnowledgeMocWorker (moc:<tag>), StructuralLinks (structural-hub-<source>), and memory graduation. Of the 16,223 articles (18.8%) carrying no identity at all, 14,935 have no source_type: they are agent-authored findings, insights and patterns written about the session's own work. They were never captured FROM anything, so capture identity is meaningless for them and the novelty gate is their mechanism, by design.

The outcome is what to measure, and it was clean: 0 duplicate titles across 80,130 published articles, and 0 intra-capture duplicates. One historical incident exists — a YouTube talk captured three times on 2026-07-02 — and it had already been remediated, its 36 redundant copies archived with exactly one published copy per title left standing. Note also what that incident proves about scope: all three copies shared ONE capture tag, so a source-level tag structurally could not have caught it. Only a per-article key could, which is the case the column exists for and the reason clients that publish many notes per capture are the ones worth adopting it.

Ranking must never key on HOW a document got in (owner decision, 2026-08-21)

Loopctl.Knowledge.RankingPriors carried two PROVENANCE priors — a source_type table ("human/reviewed provenance over raw automated ingests") and a first-party/third-party split keyed on the sourcers' capture tags (book-/url-/yt-/doc- and the bare kind tags). Both are removed. Mark's reasoning, which outlives the measurement: "if we heavily favor the internally produced knowledge, we would never learn anything new and unexpectedly useful... agents improve and I don't want less intelligent agents to decide what to pick for more intelligent future agents. I want the decision of what knowledge to use and how to combine it to be done on the receiving side."

The prior's own evidence did not survive checking. It cited a 26.7x reads-per-article gap — a statistic whose denominator is the harvest's own volume (~96% of the corpus), so it falls ~1/N mechanically. The measure that actually answers "is this material worse?" is surfaced-to-opened conversion, and rank-stratified on the live corpus it converged by rank 3 and INVERTED by rank 4 (first-party 1.95% vs third-party 1.96% at rank 4; 1.57% vs 1.74% at rank 5). Its discriminator-verification claim (98.5%/0.4%) had also gone stale.

What is still allowed, so this is a rule and not a mood: relevance itself (RRF — that IS the retrieval); DELIBERATE EDITORIAL ACTS (verdict-kill, :superseded, curation); FORM rather than origin (the MOC-hub demotion — a navigation stub is not an answer whoever wrote it); and @category_authority, KEPT by explicit owner decision on the same date because a category is an editorial classification, not an ingestion method.

What is forbidden is a weight keyed on a document's sourcer, capture tag, source_type, or ingestion batch. Two failure modes no measurement catches: a provenance prior is a CLOSED LOOP (demote unread material → it stays unread → cite the ratio as proof) and a RATCHET (a weight shipped once by one model constrains every future receiver). Guarded by test/loopctl/knowledge/ranking_priors_test.exs — "provenance priors are GONE".

The golden-question eval below cannot catch a reintroduction. Measured 2026-08-21: 1 of 124 docs in priv/retrieval_eval/golden.jsonl carries a harvest marker and NONE carry a source_type, so removing both priors moved every metric by exactly +0.000. That green is near-vacuous for this class of change — the unit guard is the real one.

What would overturn this: the owner saying so, or a conversion measurement that is rank-stratified, uses a discriminator verified against the CURRENT corpus, and still shows a durable gap. Reads-per-article is not that measurement.

Recency ranks AUTHORED age, never last-mutation time (#791)

RankingPriors.recency_decay/2 measures from RankingPriors.recency_timestamp/1 — articles.content_changed_at, falling back to updated_at only when that is null. Never read :updated_at for a recency purpose: it is bumped by ANY write to the row (a re-embed / content-hash refresh via Knowledge.update_embedding/4, a link write, a suppression flip), so ranking on it meant one bulk re-embed reset the apparent freshness of the whole corpus at once — silently, since nothing reported it.

Three things hold it together, and each has been mutation-verified:

• :content_changed_at is castable from NOWHERE. It advances only in Article.stamp_content_changed_at/1, on a real :body change. A ranking input a caller can write is a caller who can pin its own article at maximum freshness — the same rule that moved the MOC-hub signal off tags. A title edit, a status flip, a curation mark and the nightly :generic_title retitle all leave it alone.
• Every ranking-fed lane must PROJECT it. recency_timestamp/1 fails open to updated_at, so a lane whose select omits the column does not crash — it silently ranks its lane-ONLY candidates on the poisoned field. Guarded, alongside idempotency_key, by the @ranking_lanes source scan in test/loopctl/knowledge/ranking_priors_test.exs.
• The legacy corpus is BACKFILLED, and the nil-fallback is only a safety net. Seeding content_changed_at := updated_at freezes each pre-#791 row's apparent age into a field no later write moves; leaving it NULL would keep coalesce(content_changed_at, updated_at) reading the poisoned field forever, so the next bulk re-embed — or any update_all that stamps updated_at, e.g. BulkOps' status transitions — would still flatten the whole corpus. (inserted_at is the wrong seed the other way: it ages every edited article and floods the staleness lint.) It runs OUTSIDE the ADD COLUMN transaction in bounded batches, because one whole-table UPDATE under ACCESS EXCLUSIVE churns the HNSW index and can outrun the release_command budget. What still resolves through the fallback is a lane whose select omits the column, or a row the bounded loop did not reach.

The golden-question eval cannot catch a regression here either: RetrievalEval seeds content_changed_at equal to updated_at, so both fields agree by construction and the metrics move by +0.000 whichever one the prior reads. The unit and integration guards are the real ones.

Importance ranks USAGE, and only upward (#790)

RankingPriors.importance_factor/4 multiplies a fused score by clamp(1 + strength * importance_signal(read_days), 1.0, 1.1), where read_days is articles.read_day_count — the distinct days the article was opened inside the last Knowledge.heat_default_window_days/0. Usage is the authority on importance, which heat_index/2 had asserted since #554 while heat never reached ranking.

Four properties hold it together, and each is mutation-verified:

• It is one-sided upward IN SCORE, which is not the same as rank-neutral. An article with no recorded usage gets a factor of EXACTLY 1.0, so its SCORE is untouched. Its POSITION is not: ranking is ordinal, so promoting a used article moves an unread one down the list relative to it, and unread bulk-harvested material is the population at the floor. What bounds that is the 1.1 ceiling and the fact that nothing is ever scored down — a two-sided prior would drop both and reach the 2026-08-21 decision's own stated failure by another road, burying ~96% of the corpus, which then stays unread. Never give this factor a reachable sub-1.0 branch; demotion belongs to demotion_factor/1 and to deliberate editorial acts. The carve-out in CLAUDE.md's 2026-08-21 section is NOT owner-ratified and says so — do not cite it as permission, and that is why :knowledge_importance_prior_enabled SHIPS false. Enabling it is Mark's call and a one-line config flip; the nightly stamp collects the column either way.
• The signal is distinct read DAYS, capped PER PRINCIPAL. Not raw reads (the #567/#569/#572 pinning defect — a knowledge_get loop inflates a count and cannot inflate a day) and not distinct READERS (heat_counts_query/5: "under a fleet sharing one key EVERY article ties at 1"). Days defeat a same-day loop and NOT a daily one, so an article's count is least(days, Importance.solo_reader_day_cap/0 * distinct principals) where a principal is coalesce(agent_id, api_key_id): one principal buys just over half the band, six are needed to reach saturation, and the value is still DISTINCT DAYS rather than reader-days. The cap is proportional and never lifted — a binary "two readers switch it off" is defeated by one extra read, since a caller can mint a child dispatch inside its own subtree and the documented MCP config already ships two keys. Drills and search impressions stay uncounted because Loopctl.Knowledge.Importance reads Knowledge.heat_read_access_types/0 rather than restating the list — a second, wider copy of that list is how three of the four heat regressions happened.
• It is a STORED COLUMN because heat is not available DB-free. heat_index/2 is a HeavyRead aggregate plus an AdminRepo projection under a pool bound; RankingPriors is pure and merge_results/5 must stay DB-free. So the value is stamped nightly by Importance.stamp/2 inside the KnowledgeLintWorker pass (Day 5-II: stamp at consolidation time, never per write, and never by LLM-scoring an article), written with update_all so it moves neither updated_at nor content_changed_at, and projected onto every ranking lane — guarded by the same @ranking_lanes source scan content_changed_at uses, and castable from nowhere for the same reason.
• The CEILING is what bounds it against relevance, not the strength. A cross-lane RRF consensus winner scores ~2x a single-lane hit, so importance could only flip one at a ceiling of 2.0. At 1.1 it breaks ties and nothing more; that bound has its own test.

A tenant's stamp never touches a SYSTEM CANONICAL (NULL tenant_id): one column on a row several tenants read must not have one tenant's usage decide its rank for the others. A canonical's read_day_count is therefore permanently NULL, which means NOT MEASURED rather than read on zero days — and the column cannot tell those apart. So a canonical is scored at the MEDIAN factor of the pool's measured candidates (RankingPriors.pool_importance_default_factor/4), which places it at the centre of the population it is ranked against instead of at its floor: ranking a measured class against an unmeasurable one on a single number is the #569/#572 defect with the direction reversed. Do NOT turn the prior off for the whole pool instead — the canon is the bulk of this corpus, so one canonical anywhere in a ~200-row fused candidate set would disable the prior for a page that never held it, and the keyword lane cannot hold a canonical while the side-table semantic lane can, so the prior would apply on the DEGRADED response and not on the healthy one. meta.importance_strength is therefore the configured weight and does not move with pool membership. Restore full measurability by making canonicals measurable PER TENANT, never by stamping the shared column.

The golden-question eval cannot catch a regression here: RetrievalEval seeds no article_access_events, so every golden doc's read_day_count is NULL and the factor is 1.0 on every candidate — the metrics move by +0.000 by construction. The unit, integration and stamp tests are the real guards.

Ranking changes are gated by the golden-question eval (#469)

Any change to search_combined/3 ranking (weights, fusion, recency/authority) must ship with a delta from mix loopctl.retrieval.eval — recall@k / MRR / nDCG against the committed golden set (priv/retrieval_eval/golden.jsonl) and baseline (priv/retrieval_eval/baseline_v1.json). The retrieval-eval CI job runs it in both the embeddings and keyword-only arms and gates deploy. How to add a labeled question, re-baseline, and read the per-question winners/losers table: docs/runbooks/retrieval_eval.md. Its semantic lane is a SYNTHETIC (provider-free) stand-in — a regression instrument, not an absolute quality score.

Latency & observability — the semantic-search hot path

Ranking quality is gated above; this is the LATENCY side. Semantic search / novelty / suggest-links are the latency-critical path — the #172 full-corpus-scan incident is why the read SHAPE, not the number, is the load-bearing invariant. To monitor and keep it healthy over time:

• Metric. Prometheus histogram loopctl_heavy_read_repo_query_duration_bucket{endpoint="semantic_search"} (buckets ms 10/50/100/250/500/1000/2500/5000/10000; siblings vector_search, memory_recall), defined in lib/loopctl_web/telemetry.ex, scraped by Fly managed Prometheus off the internal port 9568 (/metrics). Metrics table + no-leak label rules: docs/runbooks/knowledge-scale.md.
• p95: histogram_quantile(0.95, sum by (le) (rate(<bucket>[24h]))). Fly Prometheus auth is the FlyV1-token-not-Bearer gotcha — wiki 6dd01e58.
• Low-traffic caveat (READ THIS before trusting a p95). Prod serves ~2 semantic searches/hr, so ANY window's p95 is dominated by the occasional cold-cache / autostop-resume outlier on the 512MB machines — one cold query swings it by seconds. Read p50 AND the bucket distribution, never the sparse p95 in isolation. The CI plan-shape gate is the real regression gate; the prod p95 is an observation, not a pass/fail number.
• Plan-shape invariant (the actual gate). The request-path inner ANN (Knowledge.semantic_side_table_pool_query/4) stays filter-after-ANN: a pure index-ordered top-k over article_embeddings on the per-dimension index (article_embeddings_hnsw_dim_<dim>_idx), with NO join or distance predicate inside it (wiki bd4a26b6; #172). CI asserts this on a seeded corpus without enable_seqscan=off (test/loopctl/knowledge/embedding_dimension_plan_scale_test.exs, US-41.1 AC-41.1.12(i)). Re-check prod with AdminRepo.explain(:all, Knowledge.semantic_side_table_pool_query(...)) via fly ssh console -a loopctl -C "/app/bin/loopctl rpc ..." — the plan MUST show Index Scan using article_embeddings_hnsw_dim_<dim>_idx, never a Seq Scan reaching the vector relation.
• Read-routing flag. SystemConfig "embedding_side_table_reads" (read via Embeddings.side_table_reads_enabled?/0) routes reads to the dimension-tagged side table vs the legacy articles.embedding column. Flipping it is a single reversible SystemConfig.put/2; changes reach every node within 60s via the per-minute SystemConfigRefreshWorker. The side table is a NARROW relation (the ANN fetches only article_id from a lean heap), so it reads measurably FASTER than the wide legacy column at equal recall — adding the relation IMPROVED latency, it did not cost it. Cutover prod EXPLAIN + p95 artifact: GH #464. A cache MISS answers the in-code default 0 = the LEGACY column, so the boot prime is ORDERED, not merely fast: the supervised one-shot Loopctl.SystemConfig.CachePrimer primes synchronously inside its start_link/1 and is listed in Loopctl.Application.children/0 after Loopctl.AdminRepo and before Oban and the Endpoint (GH #588). Never make it a Task child or a handle_continue/2 — both return to the supervisor immediately and restore the boot window in which every vector read silently used the legacy column. Production resolves the decision through Loopctl.Embeddings.ReadPathBehaviour (default impl Loopctl.Embeddings.SystemConfigReadPath, which also owns the flag-key string); config/test.exs points it at Loopctl.MockEmbeddingReadPath. Tests must stub that mock per-process (Loopctl.DataCase.stub_embedding_read_path/0, called for you by stub_all_defaults/0) and must NEVER write the flag — it is :persistent_term-cached VM-globally, so a write leaks across the whole node. Loopctl.ConfigEmbeddingReadPathTest fails the build on a second writer and on any :embedding_read_path config outside config/test.exs.
• Reverting the flag is now an UNINDEXED read path on a cut-over install (GH #578). The legacy articles_embedding_hnsw_idx (657 MB / 26 scans on prod 2026-08-04, vs 658 MB / 1,695 scans for the live side-table index, pg_stat_user_indexes) is retired: with shared_buffers at 1536 MB the two ~657 MB indexes evicted each other, and a cold vector search measured 8,044 ms of which 7,926 ms was blk_read_time. articles.embedding is still dual-WRITTEN — it stays the backfill/reconciliation source, and the column was NOT dropped — but on an install where the drop has run, setting the flag back to 0 puts reads on a column with no ANN index: a seq scan + top-N sort over the corpus, which trips HeavyRead's per-read SET LOCAL statement_timeout. That cancel is a raised Postgrex.Error (57014) rendering 504 db_statement_timeout — NOT {:error, :heavy_read_overloaded} (only the TenantGate concurrency shed produces that tuple) and so NOT the labelled keyword degrade, which matches the shed alone. Semantic search returns no results at all. A revert is therefore an INCIDENT action, not a routine toggle — rebuild the index FIRST (an explicit CREATE INDEX CONCURRENTLY, or down/0 of 20260805120000_drop_legacy_articles_embedding_hnsw_index.exs; raise maintenance_work_mem well above the 64 MB default or the ~657 MB build silently falls back to the slow on-disk path, wiki 753fbf69) — and if you rebuilt via a rollback, flip the flag to 0 BEFORE the next migrate runs, since the rollback makes that migration PENDING again and the next deploy would re-drop the index you just rebuilt. The drop migration is GUARDED on that same flag read straight from system_configs, so an install still reading the legacy column keeps its index and a fresh self-hosted install is unaffected. Loopctl.Embeddings.LegacyRetirement discovers legacy indexes BY COLUMN, so an install that cuts over AFTER the migration ran still gets the leftover named in its scan map. mix loopctl.embeddings revert REFUSES while the legacy index is absent (capability-detected, --force to override) and mix loopctl.embeddings status reports its presence — but that refusal binds only where mix exists (source checkout / self-hosted from source). A release ships no mix, so on the hosted instance a revert goes through bin/loopctl eval or plain SQL and the guard never runs; there the runbook's rebuild-first ordering is the only thing standing between you and a tenant-wide semantic-search outage. Operator procedure: docs/runbooks/embedding-dimension-cutover.md — Retiring the legacy articles ANN index (the drop, the baseline above, and the pg_stat_statements query for the AFTER reading) and Reverting.
• Bulk (re)embed / backfill is a live-DB hazard. Unthrottled it 504s the live wiki — per-row HNSW index maintenance saturates the small Fly Postgres and starves concurrent heavy-read searches past their statement_timeout. Throttled id-range keyset pattern: wiki 7a4187fd.

Anti-patterns

• Writing a private, task-local fact via knowledge_create (pollutes shared KB) — use memory_remember.
• Curating a live operational row into the wiki instead of exposing it via a Context Retriever entity.
• Branching caller logic on which subsystem answered instead of meta.provenance.
• Bypassing propose_article's gate to force-create a near-duplicate.
• A heavy vector read on Repo/AdminRepo (starves the admin pool) — route through HeavyRead.
• Treating hybrid_search confidence as pool-relative — it is absolute, scale-matched, margin-gated.
• Re-baselining the retrieval eval to turn a red gate green (the numbers move only with a reviewed ranking change).

Related

• tenancy-rls — the HeavyRead/HeavyReadRepo pool every KB read uses; RLS scoping.
• chain-of-custody — the role model and the #331 KB-content carve-out.
• Ecto query composition behind list_*/search: the global patterns-ecto skill.