backend

Imported from Sunagatov/Memora (backend/AGENTS.md). Install upstream with npx skills add Sunagatov/Memora --skill backend. Copyright stays with the author.

backend/AGENTS.md

Purpose

Backend is Memora's source of truth.

Tech stack

• Kotlin
• Spring Boot 4 (note: spring.data.mongodb.uri is error-level deprecated; use spring.mongodb.uri)
• Jackson (JSON)
• bcrypt (password hashing)
• MongoDB (via Spring Data MongoDB 5; in-memory stores are used only in tests)

Responsibilities

• item lifecycle
• auth/session handling
• telegram ingest API
• bot-facing failure notification delivery
• review queue APIs
• category CRUD baseline
• future client-agnostic business logic

Structural rule

Prefer feature/domain/area packages.

Current areas:

• auth
• capture
• category
• common
• config
• health
• item
• review
• transcription — voice transcription pipeline: VoiceTranscriptionService (interface), OpenAiCompatibleVoiceTranscriptionService, OpenAiAudioTranscriptionClient, TelegramVoiceDownloader, TranscriptionAudioPreparer
• item/ai — deterministic AI adapter for local/dev/test fallback; LangChain4j-backed OpenAI-compatible adapter when MEMORA_AI_MODE=openai for real V1 polishing, with safe fallback title derivation when a provider response leaves title blank

Do not drift back into a broad global technical-layer structure.

Item lifecycle / status machine

RECEIVED
  ↓ (text processing succeeds)
AI_PROCESSED_UNREVIEWED
  ↓ approve               → HUMAN_APPROVED
  ↓ edit-and-approve      → HUMAN_EDITED_APPROVED
  ↓ reject                → REJECTED

RECEIVED
  ↓ (voice transcription + AI succeeds)
AI_PROCESSED_UNREVIEWED

RECEIVED
  ↓ (voice transcription fails after retries)
TRANSCRIPTION_FAILED

AI_PROCESSING_FAILED     (text processing fails after retries)

TRANSCRIPTION_FAILED | AI_PROCESSING_FAILED
  ↓ retry               → RECEIVED (re-enqueued)

HUMAN_APPROVED | HUMAN_EDITED_APPROVED
  ↓ direct PATCH        → HUMAN_EDITED_APPROVED (stays approved)

Any status
  ↓ deleteToTrash       → DELETED

State guards

• approve: only AI_PROCESSED_UNREVIEWED
• reject: only AI_PROCESSED_UNREVIEWED
• edit-and-approve: only AI_PROCESSED_UNREVIEWED
• retry: only TRANSCRIPTION_FAILED or AI_PROCESSING_FAILED
• updateItem (PATCH): only HUMAN_APPROVED or HUMAN_EDITED_APPROVED
• deleteToTrash: any status

Filter params — ItemListQueryRequest

All three list endpoints share the same query param model:

• keyword — searches title, cleanedText, rawTranscript, rawInputText, answer
• type — IDEA | THOUGHT | QUESTION | REMINDER | OTHER
• status — exact ItemStatus enum value
• priority — URGENT_IMPORTANT | URGENT_NOT_IMPORTANT | NOT_URGENT_IMPORTANT | NOT_URGENT_NOT_IMPORTANT | NOT_APPLICABLE
• category, subcategory — partial or full 2-level path (each level optional)
• createdFrom, createdTo — ISO date (YYYY-MM-DD), inclusive range via UTC day boundaries
• sort — format field-direction
  • fields: createdAt, title, category
  • directions: asc, desc
  • default if omitted: createdAt-desc

Warning: Frontend and backend must use the same param names. The historical bug was dateFrom/dateTo vs createdFrom/createdTo. Current correct name is createdFrom/createdTo.

Category invariants

• Exactly 2 levels: category, subcategory — both required, non-blank
• CategoryService.rename() cascades categoryPath on all linked items
• CategoryService.rename() does NOT update aiCategoryPath — original AI output is preserved
• CategoryService.delete() blocked if any item uses that path
• CategoryService.delete() blocked for the configured default category path
• Duplicate path creation rejected
• CategoryService.requireExistingPath() validates category exists before assigning to item

Failure notification contract

• notificationId format: "${item.id}:${item.updatedAt.epochSecond}" — re-derived each poll
• Only items with telegramTrace.telegramChatId produce notifications
• retryContext separates manual retry counts from configured automatic attempts: "manualTranscriptionRetries=N, manualAiRetries=N, autoTranscriptionAttempts=N, autoAiAttempts=N"
• Once acknowledged via /delivered, notificationStore.isDelivered() suppresses re-delivery
• If item is retried and fails again, updatedAt changes → new notificationId → re-deliverable

Security

• Session auth filter: rejects all /api/** except /api/health, /api/auth/**, /api/capture/telegram/**
• Bot auth filter: requires X-Memora-Bot-Token header for all /api/capture/telegram/**
• Bot auth filter rejects blank configured tokens instead of accepting blank headers
• Spring Security config uses anyRequest().permitAll() — actual enforcement is in custom filters

Config keys (application.yml / env vars)

• BACKEND_ALLOWED_ORIGIN (default: http://localhost:5173)
• BACKEND_APP_PASSWORD_HASH (bcrypt hash of app password)
• BACKEND_APP_PASSWORD (optional local plaintext override; rejected by production validation)
• BACKEND_SESSION_DAYS (default: 30)
• BACKEND_COOKIE_SECURE (default: true; set false only for local HTTP dev)
• BACKEND_BOT_INGEST_TOKEN
• MEMORA_OWNER_TELEGRAM_USER_ID (String, compared to request.telegramUserId)
• DEFAULT_CATEGORY_PATH (format: Level1/Level2; legacy Level1/Level2/Level3 tolerated with level 3 ignored, default: Default/General)
• MEMORA_STORAGE_MODE (default: mongo; in-memory is for tests/local only)
• MEMORA_TRANSCRIPTION_AUTO_RETRY_ATTEMPTS (default: 3)
• MEMORA_AI_AUTO_RETRY_ATTEMPTS (default: 2)
• MONGODB_URI (default: mongodb://localhost:27017/memora) — resolves via ${MONGODB_URI} placeholder in spring.mongodb.uri; env var SPRING_MONGODB_URI also maps to spring.mongodb.uri
• MEMORA_TRANSCRIPTION_API_BASE_URL (default: https://api.openai.com; prod: http://whisper-worker:8000)
• MEMORA_TRANSCRIPTION_API_KEY (no default; prod: placeholder — whisper does not validate this)
• MEMORA_TRANSCRIPTION_MODEL (default: gpt-4o-mini-transcribe; current prod in Vault: Systran/faster-whisper-medium)
• MEMORA_TRANSCRIPTION_LANGUAGE (optional ISO-639-1 language hint)
• MEMORA_TRANSCRIPTION_PROMPT (optional domain hint forwarded to the transcription endpoint when non-blank)
• MEMORA_TRANSCRIPTION_TIMEOUT_SECONDS (default: 120)
• MEMORA_TRANSCRIPTION_MAX_AUDIO_BYTES (default: 26214400)
• MEMORA_TRANSCRIPTION_MAX_DURATION_SECONDS (default: 600)
• MEMORA_AI_MODE (default: deterministic; openai is required for real V1 AI polishing and enables the LangChain4j-backed text AI adapter)
• MEMORA_AI_API_KEY (required only in openai AI mode)
• MEMORA_AI_API_BASE_URL (default: https://api.openai.com)
• MEMORA_AI_MODEL (default: gpt-4o-mini)
• MEMORA_AI_TIMEOUT_SECONDS (default: 60)
• MEMORA_AI_FALLBACK_TO_DETERMINISTIC (default: true; local/dev-only escape hatch)
• MEMORA_VALIDATE_PRODUCTION_CONFIG (default: false; also active for prod/production Spring profiles and rejects unsafe AI fallback/config)

Production-AI safety anchor points:

• config/ProductionConfigValidator.kt enforces production-like fail-fast validation
• BackendHardeningTests.kt covers deterministic mode rejection, blank AI config rejection, fallback rejection, and local deterministic allowance when production validation is off

Rules

• do not let Telegram-specific concepts define domain logic
• prefer explicit services/use cases over framework-driven magic
• keep controllers thin
• keep status transitions explicit
• keep code runnable
• do not move deployment/runtime concerns here
• keep category paths exactly 2 levels in V1
• preserve original AI output separately from latest human-facing item values
• keep direct PATCH /api/items/{itemId} approved-only
• use edit-and-approve for reviewable edits
• keep unified Telegram ingest at one backend endpoint with nested voice payload
• reject backend ingest when MEMORA_OWNER_TELEGRAM_USER_ID is blank; owner-only capture must not silently become allow-all
• MongoDB is the production store; in-memory stores are test-only
• Mongo-backed session storage is the default; in-memory session storage follows MEMORA_STORAGE_MODE=in-memory
• async processing catches non-runtime exceptions from HTTP/IO and maps them to visible failure states; interrupted processing re-interrupts the thread
• direct backend ./gradlew bootRun does not auto-load .env.local; document local startup assumptions in backend/README.md, and keep Vault local task truth in Vault/apps/memora/backend/

Testing patterns

• All service tests in FoundationServicesTests.kt (unit-style, in-memory stores)
• directExecutor() runs ItemProcessingService synchronously — enables state assertions immediately after ingest()
• testProperties() helper provides valid bcrypt hash and sane defaults
• MemoraBackendApplicationTests — Spring context load test only
• Note on voice tests: voice ingest → TRANSCRIPTION_FAILED is still correct in tests — no real whisper endpoint in test context. In production, transcription succeeds. Do not change this test expectation.

Tests that must remain green:

• owner-only ingest (non-owner throws)
• xor validation (both text+voice rejected; neither rejected)
• text ingest → AI_PROCESSED_UNREVIEWED
• voice ingest → TRANSCRIPTION_FAILED (test-only; production uses real whisper → AI_PROCESSED_UNREVIEWED)
• category rename cascades to item.categoryPath but not aiCategoryPath
• approved edits stay approved
• approved query with keyword/status/sort filters
• failure notifications: exposed once, suppressed after ack
• direct PATCH rejected for non-approved item
• edit-and-approve rejected for already-approved item
• retry rejected for reviewable item
• category delete blocked when non-empty
• QUESTION answer failure stays visible and can be regenerated
• category proposal can be approved and reused
• regenerate-all preserves original ai* snapshot fields
• approved question answer can be cleared/rejected/deleted without removing the item

Current validation

cd backend && ./gradlew compileKotlin
cd backend && ./gradlew test