ai-security-monitor

Imported from samiulsun/ai-security-monitor (AGENTS.md). Install upstream with npx skills add samiulsun/ai-security-monitor. Copyright stays with the author.

AGENTS.md — AI Security Monitor

Compact cheat-sheet and persistent project context for OpenCode sessions. If a fact is obvious from filenames or a standard framework default, it’s omitted.

Project Overview

AI Security Monitor is a full-stack security monitoring demo for academic cloud-computing projects. It analyzes login attempts in real time using a rule-based engine and an IsolationForest ML anomaly detector, then visualizes the results in a React dashboard.

Core flows:

1. User submits credentials on /login.
2. FastAPI backend checks the IP block list, validates credentials, and runs the SecurityDetector.
3. Results (attack type, severity, risk score) are returned and displayed.
4. The dashboard at /dashboard polls for stats, alerts, logs, and charts.

Architecture

User/Browser
    ↓
Nginx Reverse Proxy (port 80)
    ├── /api/*  → FastAPI Backend (port 8000)
    └── /*      → React/Vite Frontend (port 80 internal)
                    ↓
            PostgreSQL (persistent database)

Background services:

• LogMonitor is disabled by default in cloud deployments to prevent duplicate ingestion across replicas. A dedicated ingestion worker can opt in to tailing /var/log/nginx/access.log; synthetic traffic is separately opt-in.
• SecurityDetector retrain runs periodically on synthetic events.

See CLOUD_DEPLOYMENT.md and AWS_DEPLOYMENT.md for cloud target architectures.

Technology Stack

No lint/format/typecheck tooling is configured. CI only runs tests and the frontend build.

Repository Structure

ai-security-monitor/
├── .github/workflows/ci.yml
├── .opencode/              # OpenCode persistent context
├── backend/
│   ├── app/
│   │   ├── api/            # auth, alerts, logs, stats, simulate routers
│   │   ├── core/           # config, database, logging
│   │   ├── ml/             # SecurityDetector + IsolationForest
│   │   ├── services/       # LogMonitor
│   │   └── main.py         # FastAPI entrypoint
│   ├── migrations/         # SQL migration files
│   ├── tests/
│   ├── Dockerfile
│   ├── start.sh            # DEBUG-aware uvicorn startup
│   └── requirements.txt
├── frontend/
│   ├── src/
│   ├── Dockerfile
│   └── package.json
├── nginx/
│   └── nginx.conf
├── scripts/
│   ├── deploy.bat          # Windows self-hosted runner
│   ├── deploy.sh           # Linux/macOS self-hosted runner
│   └── simulate_attacks.py # CLI attack simulator
├── docker-compose.yml
├── .env.example
├── AGENTS.md
├── AWS_DEPLOYMENT.md
└── CLOUD_DEPLOYMENT.md

Development Commands

Backend only

cd backend
python -m venv venv
source venv/bin/activate
pip install -r requirements.txt
# Ensure DATABASE_URL points to PostgreSQL, e.g.:
# export DATABASE_URL=postgresql+asyncpg://postgres:postgres@localhost:5432/security
uvicorn app.main:app --reload

• Entrypoint: app.main:app
• Runs on http://localhost:8000
• Tables and indexes are created automatically on startup.
• Logs written to backend/logs/app.log (must be writable).

Frontend only

cd frontend
npm install
npm run dev

• Dev server: http://localhost:3000
• Vite proxies /api/* to http://localhost:8000.
• Production build: npm run build → frontend/dist/

Frontend has no test framework configured.

Test Commands

cd backend
pytest tests/ -v

• pytest.ini sets pythonpath = .; tests must run from backend/.
• tests/conftest.py injects the project root into sys.path.

Docker Commands

Run everything locally

docker compose up --build

• Main app: http://localhost
• Direct backend / Swagger: http://localhost:8000/api/docs
• Stop: docker compose down
• Reset including DB volume: docker compose down -v

Services

Named volumes: postgres-data, backend-logs, nginx-logs.

Deployment Commands

Local / self-hosted

# Linux/macOS
./scripts/deploy.sh

# Windows
scripts/deploy.bat

Cloud

See:

• CLOUD_DEPLOYMENT.md — general VM and managed-container options.
• AWS_DEPLOYMENT.md — EC2 + Docker Compose and ECS Fargate + RDS + ALB.

Environment Configuration

Copy .env.example to .env and fill in values. .env is gitignored.

Key variables:

Never commit actual secrets.

Important Technical Decisions

1. PostgreSQL over SQLite — Replaced SQLite with PostgreSQL to support horizontal scaling and cloud-managed databases. Used the lightweight databases library with asyncpg instead of SQLAlchemy/Alembic to keep the academic demo simple.
2. Redis-backed shared state — Redis stores sliding-window counters and publishes security events for SSE. Heavy background processing is still intentionally outside the API process.
3. Simulation endpoints gated — Added ENABLE_SIMULATION_ENDPOINTS so destructive demo endpoints can be disabled in production.
4. Structured JSON logging — LOG_FORMAT=json outputs cloud-friendly logs with request IDs.
5. Health checks — /api/health for liveness; /api/health/ready checks PostgreSQL connectivity.
6. Nginx as single entry point — All traffic goes through Nginx; /api/* is proxied to backend, everything else to frontend.

Current Project Status

• ✅ PostgreSQL migration complete
• ✅ Docker Compose with PostgreSQL working
• ✅ Health checks (/api/health, /api/health/ready)
• ✅ Structured JSON logging with request IDs
• ✅ Frontend polling with error backoff
• ✅ Cloud deployment guides (general + AWS)
• ✅ CI/CD with PostgreSQL service, security scans, Docker build
• ✅ Redis-backed shared counters and event streaming
• ✅ Database-backed bcrypt authentication, JWTs, and RBAC
• ❌ Background workers not implemented (uses FastAPI BackgroundTasks)
• ❌ No frontend tests
• ❌ HTTPS/TLS not configured in local Nginx (use ALB/ACM in AWS)

Known Issues

• Log ingestion ownership — Keep ENABLE_LOG_MONITOR=false on every horizontally scaled API replica. Run one dedicated ingestion worker if Nginx log monitoring is required.
• ML training buffer — The detector model is process-local and retrained from baseline data; production-grade model training should move to a dedicated worker/model store.
• Demo credentials — Demo users are seeded from environment-configured passwords for local/course use; replace them with rotated deployment secrets or an external identity provider in production.
• README.md is outdated — Still mentions SQLite and a standalone /api/simulate/xss endpoint. AGENTS.md and AWS_DEPLOYMENT.md reflect the current state.

Remaining Enhancements

• (Optional) Move heavy simulations/ML retraining to Celery workers.
• (Optional) Add HTTPS/TLS to local Nginx config.
• (Optional) Add frontend/component tests.
• (Optional) Update README.md to match the current PostgreSQL/cloud-ready architecture.

Important Warnings

• Never commit .env or any secrets. It is gitignored, but double-check before pushing.
• Disable simulation endpoints in production by setting ENABLE_SIMULATION_ENDPOINTS=false.
• Nginx rate limits will trigger HTTP 429 during high-rate simulations through http://localhost. Use http://localhost:8000 for simulator traffic.
• Run backend commands from backend/ so pythonpath = . resolves app.* correctly.
• ML cold start — The IsolationForest model trains on startup; first launch may take a moment.

Quick Reference

API endpoints (all under /api)

• POST /api/auth/login — login + security analysis
• GET /api/auth/me — stubbed current user
• GET /api/alerts — alerts
• GET /api/stats/* — dashboard statistics
• GET /api/logs — login attempts
• GET /api/logs/events — raw log events
• POST /api/simulate/* — attack simulation (dev only)
• DELETE /api/simulate/reset — wipe all data (dev only)
• GET /api/health — liveness
• GET /api/health/ready — readiness (PostgreSQL check)

Test credentials (demo only)

Demo users are seeded into PostgreSQL from DEMO_*_PASSWORD environment settings. Do not use demo passwords in production.

Attack simulator CLI

scripts/simulate_attacks.py requires requests (not in requirements.txt):

pip install requests
python scripts/simulate_attacks.py --mode all