Imported from jsnyder/quorum (AGENTS.md). Install upstream with npx skills add jsnyder/quorum. Copyright stays with the author.

[quorum] recent context, 2026-05-12 12:06pm CDT

Legend: 🎯session 🔴bugfix 🟣feature 🔄refactor ✅change 🔵discovery ⚖️decision 🚨security_alert 🔐security_note Format: ID TIME TYPE TITLE Fetch details: get_observations([IDs]) | Search: mem-search skill

Stats: 50 obs (17,106t read) | 1,280,203t work | 99% savings

May 12, 2026

82687 12:46a ⚖️ AST Judge system architecture finalized with two-phase implementation plan 82688 " ⚖️ Judge verdict separated from calibrator_action to distinguish machine vs human feedback 82689 " ⚖️ Two-layer caching strategy minimizes judge latency on repeat analysis runs 82690 " ⚖️ Nano/mini class models selected for judge to optimize cost of structured JSON classification 82692 12:52a 🔵 Codebase structure mapped: Finding/FeedbackEntry fields and analysis.rs construction sites identified 82693 " 🔵 Finding construction patterns examined: all fields set explicitly, no rule_id yet, ~22 fields per constructor 82696 1:02a 🟣 Added rule_id field to Finding struct for rule tracking 82697 1:57a 🟣 ast-grep findings now populated with standardized rule_id 82698 2:04a 🔵 Mapped 61 rule_id: None sites across analysis.rs security scanning functions 82700 2:13a ✅ Add rule_id field to Finding struct 82701 2:15a 🟣 Implement rule_id tracking for AST-derived findings 82702 " 🟣 Implement per-rule precision tracking via --by-rule stats dimension 82703 2:23a 🟣 Rule metadata system with precision tier and judge requirement support 82704 2:36a ✅ load_rules() now returns rule metadata alongside rules 82705 2:38a 🟣 Judge telemetry metrics added to TelemetryEntry 82706 " 🟣 Judge feature gated by QUORUM_JUDGE environment variable 82707 2:58a ✅ Integrate judge metrics into FileReviewResult 82708 3:04a 🟣 Judge stage wired into review pipeline for AST rule validation 82709 7:58a 🟣 Judge System for AST-Derived Findings: Full Implementation Complete 82710 " ✅ Judge Implementation Branch Prepared and Pushed to Remote 82711 10:43a ✅ Judge Implementation Patch Generated for Review 82712 " 🔵 Installed Quorum Binary at v0.20.0 82713 " ✅ Release Build Started for v0.21.0 82714 " 🔵 GitHub Actions CI Running on Judge Implementation PR 82716 10:45a 🔴 Clippy Collapsible-If Error Blocking PR on Local Reproduction 82715 " 🔵 CI Validation Reveals Clippy Warning and Test Failure on Judge PR 82717 10:46a ✅ Judge telemetry fields added to TelemetryEntry test assertions 82718 10:58a 🔴 Clone precision tier in ast_grep Finding construction 82719 " 🔄 Separate mutation from filtering in judge_findings Phase 3 82720 11:01a ✅ Language-qualified metadata keys and improved judge orchestration 82721 " 🟣 Comprehensive test suite validation for new judge fields 82722 11:02a ✅ Committed hardening fixes addressing metadata key collision and judge logic 82736 11:20a 🔴 Judge system silently disabled due to metadata key format mismatch 82737 " ✅ Standardized metadata key format to "ast-grep:{language}/{rule_id}" across codebase 82738 " 🔴 Optional+Rejected findings confidence not clamped to 0.05 82739 " 🔴 Duplicate LLM verdict matching in judge Phase 2 82740 " ✅ Test assertion updated to use qualified metadata keys 82741 " ✅ Error handling for JSON serialization in main.rs 82742 " ✅ Guard against empty QUORUM_JUDGE_MODEL environment variable 82743 11:26a ✅ Filed true positive (TP) feedback verdicts for 4 critical bug fixes via quorum feedback CLI 82744 " ✅ Completed filing all 10 CodeRabbit-identified bug fixes as true positive (TP) calibration feedback 82745 11:27a ✅ Filed additional TP verdicts for test coverage improvements (records 4170-4171) 82746 " ✅ Filed FP (false positive) verdicts for 6 intentionally-skipped CodeRabbit findings (records 4172-4177) 82747 " 🔵 Calibration system enforces role-based verdict permissions: wontfix verdicts require project owner, external agents can file tp/fp/partial only 82748 " ✅ Filed final calibration feedback verdicts: 3 partial (4178-4180) and 1 FP (4181) from CodeRabbit review 82749 11:28a ✅ Project owner filed 3 wontfix verdicts (4182-4184) that external agent could not file 82750 " ✅ Filed TP verdict for compute_confidence Optional+Rejected gap identified by CodeRabbit (record 4185) S1874 Wire the LLM client into the judge stage to enable lower-precision rule support via judge verdicts. Started by exploring codebase architecture and integration points. (May 12 at 11:38 AM) S1879 Judge LLM wiring design specification: complete architecture document with trait definition, OpenAiClient integration, error handling, and testing approach ready for approval before Phase 4 TDD implementation (May 12 at 11:40 AM) 82755 11:42a 🔵 Judge system architecture fully implemented but LLM client not wired 82756 " 🔵 Identified async/sync boundary challenge for judge LLM integration 82757 " ⚖️ Judge LLM wiring implementation approach and phased roadmap S1877 Finalize judge LLM wiring implementation plan: testing strategy and success criteria for validating judge effectiveness on speculative rules (May 12 at 11:49 AM) S1875 Wire LLM client into judge stage for AST-derived findings: comprehensive architecture & design phase with strategic decisions on async/sync bridge, trait design, batching, and client reuse (May 12 at 11:49 AM) S1876 Wire LLM client into judge stage for lower-precision AST-derived findings: complete architecture phase with strategic decisions on async/sync bridge, trait abstraction, call batching, and client reuse strategy (May 12 at 11:49 AM) S1878 Judge LLM wiring: complete planning phase with all architectural decisions finalized; ready for Phase 2 implementation (worktree and TDD) (May 12 at 11:49 AM) S1880 Judge LLM wiring planning phase complete: comprehensive design specification finalized and ready for implementation approval; moving toward Phase 2 (worktree) and Phase 4 (TDD implementation) (May 12 at 11:51 AM) S1882 Implement LLM judge system for quorum code review tool to filter false positives from speculative AST-grep rules by obtaining verdicts (Approved/Rejected/Uncertain) from Claude LLM (May 12 at 11:54 AM) S1881 Implement a judge system for AST-derived findings in quorum code review tool, wiring up LLM client to filter false positives via judge verdicts for speculative rules. Design review phase to identify critical correctness issues before implementation. (May 12 at 11:58 AM) S1883 Design and validate judge system architecture for quorum code review tool to wire LLM client into AST-based speculative finding evaluation, addressing critical Rust async trait object safety constraints (May 12 at 12:06 PM) Investigated: Two comprehensive expert design reviews via PAL chat: (1) gpt-5.4 full architecture review covering 6 major concerns (object safety, error handling, batching, async migration, reference vs Arc, edge cases); (2) o3-pro deep-dive on async trait object safety question with Rust 1.93 language feature status and four implementation options with trade-offs

Learned: Async trait object safety: async fn in traits NOT object-safe in Rust 1.93 — compiler expands to hidden associated types (Self::call<'a>) that vary per implementor, breaking v-table unification. RFC 3498 (return-type-notation) not merged for stable; unlikely by 1.93. Reviewer was correct: &dyn JudgeLlm does not compile without boxing. Best solution for this scenario: async-trait crate (zero-effort ergonomics, allocation cost negligible vs multi-second LLM call). Alternatives: (B) generics with monomorphization (zero overhead, lose dynamic dispatch), (C) manual Pin (verbose, error-prone), (D) enum + generics hybrid (scales poorly past handful of variants). Architecture confirmations: use Arc over &OpenAiClient (soundness, future parallelism); add tracing::warn! for LLM error observability; implement batch size guardrails (warn >20 findings, max 50); document breaking change (all tests → #[tokio::test])

Completed: Design phase complete. Both expert reviews converged on same technical decisions: async-trait for object safety, Arc for reference model, observable error handling, batch size caps. No blockers remain. Codex CLI (v0.128.0) confirmed available as potential code generation assistant during implementation

Next Steps: Transition to implementation Phase 2 (TDD). Next immediate steps: (1) Create git worktree for judge-llm-wiring feature branch; (2) Write unit tests first (RED phase) with MockJudge concrete type, test verdict parsing, confidence clamping, cache behavior, async runtime integration; (3) Implement JudgeLlm trait with #[async_trait] annotation, OpenAiJudge wrapper struct holding Arc, async judge_findings refactor; (4) Wire pipeline integration point (review_file async context); (5) Add observability (tracing::warn for LLM errors, batch size warnings); (6) Smoke test on real quorum findings

Access 1280k tokens of past work via get_observations([IDs]) or mem-search skill.