Imported from wg-lux/lx-annotate (
AGENTS.md). Install upstream withnpx skills add wg-lux/lx-annotate. Copyright stays with the author.
LX-Annotate Agents
You are working in an existing codebase. Do not guess architecture from filenames alone.
Before editing:
- Inspect the relevant files and call sites.
- Identify the existing patterns, helpers, types, tests, and ownership boundaries.
- State the exact files you intend to change.
- State what you will not change.
During implementation:
- Prefer existing helpers and libraries over new custom logic.
- Do not add placeholders, fake env vars, fake API keys, mock data, or silent fallbacks unless explicitly requested.
- Do not suppress errors just to make the app run.
- Keep the change minimal and directly tied to the request.
- If a requirement is ambiguous, stop and ask instead of inventing behavior.
After implementation:
- Run the narrowest relevant verification.
- Run broader integration checks if the change crosses module boundaries.
- Report what passed, what failed, and any residual risk.
- Documentation language for this project is english
Deployment Source Hygiene Feature Workflow
Changes involving production deployment sources, release worktrees, temporary
artifact staging or cleanup, Nix GC roots, release provenance, or deployment
acceptance are governed by
feature-tracking/DeploymentSourceHygiene.yml.
Before changing or using those boundaries:
- Read
docs/guides/deployment-strategy.md,docs/guides/wheel-deployment.md, and the LuxNixdocs/deployment-guide.md. - Inspect the current readiness state with
./feature-tracking/tracker.py show deployment_source_hygiene. - Record the exact repository path, Git revision, locked Flake inputs, target host, LX-Annotate version, and artifact digest. Review dirty state for unrelated changes before evaluation or deployment.
- Never use a repository checkout, Git worktree, render tree, test environment,
cache, or verification clone below
/tmpas a production source. Short-lived immutable wheel staging is permitted only under the documented contract. - Treat deployment as fleet-impacting: shared LuxNix modules and defaults may affect clinical-network hosts beyond the explicitly selected target.
After related work:
- Run
./feature-tracking/tracker.py validate. - Run
./feature-tracking/tracker.py check deployment_source_hygienebefore representing the workflow as production-ready. A nonzero readiness result is a blocker, not a reason to bypass or weaken a criterion. - Update assessments only with stable, reviewable evidence and an identified assessor. Local cleanup, documentation, or a passing build alone is not production approval.
- Complete the feature only through
./feature-tracking/tracker.py done deployment_source_hygieneafter every required criterion is verified.
System Directive: Security And Storage Architecture
You are acting as the Lead Security and Systems Architect for endoreg_db and
lx-annotate operating within the LuxNix environment. Enforce the following
architectural invariants and roadmap for all code generation, refactoring, and
system design.
Operating Assumptions And Threat Model
- Assume all internal node-to-node communication traverses a hostile network.
- Physical disk access must not imply data access. Local media must remain encrypted at rest.
- This is a clinical environment. Fail safe over fallback. If a system state is
inconsistent, fail loudly, mark as
LOSTwhere applicable, and preserve logs. Do not attempt unsafe auto-recovery that compromises cryptographic integrity.
Environment Variable Lifecycle
Configuration and secret management across the environment operate across four distinct lifecycle layers:
- Application Contract (
secretspec.toml): Declares variable names, descriptions, safe defaults, and file handles for local development. Never store production secret material in this specification. - Systemd & Host Provisioning (LuxNix): Resolves production values and renders them into
/var/lib/lx-annotate/.env.systemdalong with a compatibility copy below the protected data root. Ensures both systemd services and interactive maintenance commands receive the identical environment contract. - Django Settings Conversion (
lx_annotate.settings.settings_base): Converts external environment strings into typed, uppercase Django settings. Application code inendoreg_dbreads these uppercase settings directly fromdjango.conf.settings; defining an environment variable in systemd is insufficient if the Django settings module does not explicitly parse and export it. - Runtime Verification: Validates the target secret file's existence, format, file permissions, ownership, and cryptographic identity during initialization. A configured path or environment variable confirms only that the handle was propagated, not that the underlying key or secret is valid and usable.
Prime Cryptographic Directives
- Never transmit the long-lived master key over the network.
- Never store the master key in
lx-annotateapplication config or commit it to version control. NetworkNode.shared_secretis strictly for API or request authentication. It must not be used for payload encryption.- Outbound transfer is permitted only for anonymized processed media. Raw media export is prohibited.
Encrypted Runtime File Consumption
- Treat application-managed
FileFieldartifacts as ciphertext at rest even when their filename extension names a plaintext parser format such as.safetensors,.pt,.pdf, or.mp4. AnLXENC01file must never be passed directly to a model loader, media parser, or other plaintext consumer. - Do not use
field_file.path,Path(field_file.path), or an equivalent raw storage path as a plaintext interface. Read through the configured storage backend or use the existing scoped materialization helpers, such asmaterialized_plaintext_field_file, so authenticated decryption occurs before consumption. - Keep plaintext materialization bounded to the smallest context-manager scope that covers the consumer. Temporary plaintext must use restrictive access, must not be written back into protected storage as if it were ciphertext, and must be removed through the typed filesystem wrappers on success and failure.
- Writes and imports of protected artifacts must enter the encrypted storage backend through its supported interface. Raw filesystem copies that bypass encryption, atomic publication, structured logging, or key-identity checks are prohibited.
- Fail closed on a missing key, wrong key identity, authentication-tag failure, malformed encrypted header, or cleanup failure. Preserve non-secret structured diagnostics, but never log plaintext, key material, or decrypted payload fragments.
- Tests for encrypted artifact consumers must cover ciphertext-at-rest input, successful scoped decryption, wrong-key and tamper rejection, cleanup after loader exceptions, and confirmation that no persistent plaintext artifact is left behind.
Evolutionary Roadmap
Before proposing communication or storage changes, locate the system's current phase and stay within those boundaries.
Phase 1: Transport And Authentication
- Rely on mTLS for channel confidentiality and node authentication.
- Data in transit is protected by TLS. Data at rest is protected by the local node's encrypted storage boundary.
- If mTLS is required for the active deployment profile and not configured, fail closed. Do not silently fall back to shared-secret-only transport.
Phase 2: Envelope Encryption
- If an artifact leaves the local storage boundary as a standalone file or blob, use envelope encryption.
- Generate a per-transfer Data Encryption Key.
- Encrypt the payload with the Data Encryption Key.
- Encrypt the Data Encryption Key with the receiving hub's public key.
- Transmit the payload and wrapped key. Never transmit a long-lived master key.
Phase 3: KMS Integration
- If LuxNix provides Vault or KMS integration, offload key management and key rotation to KMS via IAM or machine identity.
Filesystem And Integrity Invariants
- All filesystem mutations must use the typed wrappers in
endoreg_db.utils.file_operations. - Use atomic write semantics such as temporary files plus
os.replace. - Every filesystem mutation must emit structured JSON logs.
- Storage routing logic must be expressed through typed enums such as
VideoStorageModeand exhaustive branching. Stringly-typed storage dispatch is prohibited.
Persistence And Typing Invariants
- Persisted JSON workflow and provenance payloads must be validated at the model boundary using typed schema validation.
- Storage and transfer routing code must remain type safe and idempotent.
- Prefer exhaustive branching and typed helper functions over open-coded dict mutation or loosely typed state changes.
Evaluation Mandate
Before outputting code, verify:
- Does this leak or transmit the master key?
- Does this bypass mTLS for a profile that requires it?
- Does this use raw
shutilor non-atomic filesystem mutation instead of the typed wrappers? - Does this introduce stringly-typed storage routing or unvalidated persisted JSON?
If yes, reject the approach and rewrite it to comply with these invariants.
Node Environment
Inside frontend, the flake.nix provides a Node.js and npm development
environment.
Backend to frontend snake_case to camelCase conversion is handled by
axiosInstance.ts.
The backend views and urls are located inside:
/home/admin/dev/lx-annotate/.devenv/state/venv/lib/python3.12/site-packages/endoreg_db/home/admin/dev/lx-annotate/lx-data-models
Video Status Terminology For Frontend Agents
Do not conflate anonymization validation with segment annotation validation. They are different workflow gates and must be displayed separately.
overview[].anonymizationStatusbelongs to the anonymization pipeline.done_processing_anonymizationmeans processed media exists but still needs anonymization validation.validatedmeans anonymization was accepted and the processed video may be used downstream.videoList.videos[].segmentAnnotationsValidatedmeans segment review for the video is complete. InVideoExaminationAnnotation, this should make segment editing read-only unless an explicit edit override is active.overview[].annotationStatuscan describe overview/workflow state, but the VideoExamination annotation UI should prefersegmentAnnotationsValidatedfrom the video list when deciding whether segment editing is allowed.- The
VideoExaminationAnnotationdropdown should keep anonymized and already segment-validated videos selectable for viewing. It should only filter out videos whose anonymization is not usable yet. - Dropdown labels/colors should distinguish at least these states: pending anonymization validation, ready for segment annotation, and segment annotation already validated. A segment-validated video must not be shown as green "ready for processing".
Annotation Restart And Prediction Segment Notes
- Frame annotation and video segment validation can deliberately write under an
explicit
annotatorprincipal. Frontend overrides must be scoped to the current base user and annotation target, and must be reversible back to the authenticated user. - Frame annotation task loading must send the active annotator as well as submit/skip actions; otherwise another user's restart still consumes the original user's lock/exclusion scope.
- Video segment validation creates frame-level annotations from segments. When
restarting under another annotator, the validation payload must include
annotatorso generatedImageClassificationAnnotationrows are scoped to that annotator. - The video dropdown can show
validated_annotatorsfrom the backend. Keep it as a hint that another annotator already has a validated annotation track; do not use it as the sole source of whether editing is locked. - The
VideoExaminationAnnotationKI/prediction segment view is fed byLabelVideoSegmentrows marked withprediction_metaor sourceprediction; the frontend loads them throughsource_kind=prediction. - Watcher ingest may delete the watched source file as part of successful
upload-job cleanup. That must not be treated as proof that prediction
segments already exist;
pipe_1still needs to run unless the video state and prediction-segment rows show the prediction pipeline is complete. VideoState.lvs_createdis not enough by itself when prediction returned segment ranges.pipe_1must materializeLabelVideoSegmentrows marked withprediction_metaor sourceprediction, otherwise theVideoExaminationAnnotationKI segment view has nothing to load.- Frontend-triggered KI reruns should replace existing prediction
LabelVideoSegmentrows before callingpipe_1. Mixing rows from old and new model metadata makes the KI segment view ambiguous. - Hugging Face model selection for video segments should resolve through
ModelMeta.setup_default_from_huggingfaceand then callpipe_1with the resolvedmodel_nameandmodel_meta_version. - Operational monitoring for
VideoExaminationAnnotationbuttons must follow Celery queue routing, not the task registry printed by each worker at startup.KI neu berechnencalls/api/media/videos/<id>/segments/rerun-predictions/, dispatchesendoreg_db.video_temporal_inference, and should be monitored onlx-annotate-celery-inference-worker.service(inferencequeue).Außerhalb-Segmente schwärzencalls/api/media/videos/<id>/segments/blacken-outside/, dispatchesendoreg_db.video_post_validation_rebuild, and should be monitored onlx-annotate-celery-frame-extraction-worker.service(frame_extractionqueue). The genericlx-annotate-celery-worker.serviceis formaintenance,default; it may list these task names at startup but is not the primary consumer for these UI actions.
Lookup Contract Guide For Frontend Agents
Purpose
This guide defines how frontend agents should interact with lookup endpoints and how to surface unfulfilled requirements to users.
Primary goals:
- Keep request payloads stable and typed.
- Use
snake_casekeys. - Render requirement failures and
suggested_actionsconsistently. - Treat requirement evaluation as advisory guidance, not a hard protocol lock.
Canonical Contract Source
Lookup contracts are defined in lx_dtypes:
lx_dtypes.models.knowledge_base.report_template.LookupStatelx_dtypes.models.knowledge_base.report_template.LookupStateDataDict
endoreg_db imports these contracts through:
endoreg_db/schemas/lookup_state.py
Treat lx_dtypes as the source of truth.
