Imported from joshaustintech/Erebor (AGENTS.md). Install upstream with npx skills add joshaustintech/Erebor. Copyright stays with the author.

System Overview

High-throughput text ingestion and summarization platform built with Java 25 and Spring Boot 4, designed to scale from a handful of weekly entries to millions per hour while maintaining low-latency, accurate summaries.

Architecture (High Level)

1. Monorepo Layout (Gradle multi-project): shared library (common models/utilities), backend service, frontend web (Elm + Thymeleaf templates served via Spring WebMVC).
2. Ingestion API (backend): Authenticated REST/stream endpoints accepting raw text payloads; enforces quotas and schema validation.
3. Queue Layer: Apache Kafka for durable, horizontally scalable buffering and ordered, replayable streams.
4. Processing Workers: Spring Boot backend components consuming from Kafka, normalizing text, deduplicating, and tagging metadata.
5. Storage: PostgreSQL for all non-customer-facing data (control-plane + normalized events; Flyway-managed schema, MyBatis mappers, partitioning/TTLs as needed); Elasticsearch/OpenSearch for customer-facing summary search; object storage for raw archives.
6. Summarization Service: Parallel summarization pipeline (batch + streaming) using Spring AI with a local Gemma 4 E2B model for summarization, producing incremental and roll-up outputs with quality gates, writing searchable results to Elasticsearch.
7. Delivery: APIs and webhooks to retrieve on-demand summaries, periodic digests, and health signals backed by Elasticsearch indices; frontend renders via Thymeleaf + Elm SPA components.
8. Observability & SRE: Micrometer metrics with Prometheus/Grafana, structured logging, SLOs on ingestion latency and summary freshness, autoscaling hooks, and backpressure controls.

Code Rules

1. Language/Runtime: Java 25, Spring Boot 4; prefer records, virtual threads (Project Loom), and reactive only when needed.
2. Packaging: Modular Spring Boot starters per bounded context; avoid cyclic dependencies; keep DTOs contract-stable.
3. Data Contracts: Version all public payloads; use JSON Schema for validation; never break backward compatibility without a migration path.
4. Testing: Minimum: unit tests for business logic, contract tests for APIs, integration tests for queue and DB adapters; prefer Testcontainers-backed integration over mocks; CI must be green before merge.
5. Performance: Treat ingestion and summarization as latency-budgeted paths; add benchmarks for hot code and enforce backpressure at queue consumers.
6. Resilience: Idempotent consumers; at-least-once processing by default; dead-letter topics and retry policies are required for every consumer group.
7. Observability: Emit structured logs and Micrometer metrics; no silent catches; alarms on lag, failure rates, and summary staleness; prefer Prometheus exporters over OpenTelemetry.
8. Security: Require auth on all endpoints; redact secrets in logs; principle of least privilege for DB/queue credentials.
9. Configuration: Externalize via environment/profiles; never hardcode endpoints or secrets; support local dev overrides.
10. Model Management: Pin Gemma 4 E2B model version; verify GPU/CPU requirements; add load/health checks; guardrails for prompt injection and max token limits.
11. Portability: Build and test multi-arch images (amd64/arm64) to run locally (kind/Docker) and on K3s Raspberry Pi clusters; use distroless or alpine bases where compatible.
12. Documentation: Maintain a living architecture log in docs/DECISIONS.md; keep README, ROADMAP, and AGENTS current when interfaces or workflows change.