External security review pack

Everything an independent reviewer needs to assess the OpenSmartRoute control plane against the threat model in Security model: scope, trust boundaries, the evidence the project already produces, how to reproduce it, the questions we want answered and how a report is published. It exists so that an independent review can be run by a third party without a briefing call.

1. Engagement

2. System under review

caller ── HTTPS ──> platform/web (Next.js, proxies /api,/v1,/mcp) ──> platform/api (FastAPI, osr_platform)
                                                                        │  API keys, sessions, plans, metering,
                                                                        │  workspaces, audit, admin
                                                                        └──> opensmartroute (SDK: signals,
                                                                             policy, strategies, learners)
                                                                                └──> targets (Azure OpenAI,
                                                                                     MCP tools, agents, humans)
self-hosted: caller ──> osr serve (src/opensmartroute/server.py) ──> opensmartroute ──> targets
library:     your code ──> Router / RouterBuilder ──> targets

Trust boundaries, from the outside in:

1. Internet -> web / API. Anonymous: landing, docs, /api/v1/models, /rankings, /stats/public, /status, /estimate (rate limited), /signup (5 per IP per day). Everything else needs an API key (osr_live_, hashed at rest) or a browser session (osr_sess_, 30 days, SSO via PKCE with a signed state). Admin routes need X-Admin-Token. Cross-workspace reads are 404 by construction (db.usage_by_request(account, request_id) scopes every trace / event lookup).
2. osr serve. Optional bearer / X-API-Key middleware (serve --token, --require-auth, OSR_SERVER_AUTH_TOKENS); OPEN_PATHS are the probes, /metrics, /whoami and the OpenAPI UI. The Helm chart and the Docker image are documented as "front it with your gateway".
3. Router -> request text. InputGuard (size, gadget and steering scrub, optional PII redaction) runs before signals; Policy runs before scoring and cannot be outweighed; the LLM judge sees a fenced, sanitised prompt and its output is parsed item by item.
4. Router -> catalogue. Imported MCP tool descriptions, A2A cards and SKILL.md files pass the injection lexicon and the learned gadget score; MCP manifests can be signed (HMAC-SHA256 / Ed25519).
5. Router -> execution. OriginPolicy (parameter provenance), ResourceLimiter (steps, calls, depth, tokens, cost, wall-clock), breakers and budgets sit between a decision and a handler call; OutputGuard (secrets, PII, relayed injection - flag / redact / block) sits between the handler's answer and the caller.
6. Router -> state. JSON only, atomic writes, hashed keys, versioned schema, quarantine of corrupt state, optional AES-256-GCM at rest, hash-chained audit with verify().
7. Platform -> workspace. API keys carry scopes (route, execute, teach, manage), an expiry and per-key limits (daily / monthly budget, allowed targets, requests per minute); the workspace policy and tenant budgets apply on top; GET /api/v1/governance/report renders the controls in force, the key posture and the guard verdicts of the last 24 h as the evidence pack a reviewer would otherwise assemble.

3. Threat model

SECURITY.md, "Threat model & controls" lists T1-T19 with the OWASP Top 10 / OWASP LLM Top 10 mapping and the implementing code; "Threat-model delta: 0.3 -> 0.4" lists the residual risk we already know about per threat. The reviewer is asked to confirm, refute or extend that table rather than start from a blank page.

4. Evidence the project produces

5. Questions for the reviewer

Ordered by the impact a positive finding would have.

1. Rerouting (T1, T2). Can a request author steer the decision to a chosen target through the text alone - gadgets the learned detector misses, steering the scrub does not remove, or judge-prompt escapes past the sanitiser? Success = a reproducible prompt that moves a target's propensity by more than the ensemble's stated confidence margin on examples/targets.yaml.
2. Policy bypass (T3). Does any strategy, plan slot, fallback, cascade or exclude/candidates path reach a target that Policy.check rejected (PII to a pii_allowed: false target, region or tenant constraint)? The platform's /api/v1/policy overlay is in scope.
3. Cross-workspace access (platform). Any read or write that crosses account_id: traces, events, activity, savings, API keys, invites, SSO connections, marketplace listings, admin routes without the admin token, IDOR on request_id.
4. Authentication (platform). API-key hashing and comparison, session fixation / expiry, SSO PKCE and state signature, the RFC 8628 device flow (slow_down, code entropy, approval binding), signup rate limiting, the OpenAI-compatible /v1 proxy's key handling.
5. Catalogue poisoning (T13, T14). An MCP tool description, A2A card or SKILL.md that passes the import filter and changes routing or smuggles a parameter past OriginPolicy.
6. Learner poisoning (T5). Feedback (/feedback, Outcome) sequences that move a target's learned quality faster than the documented prior / confidence ramp allows, or state files that load and are trusted when they should be quarantined.
7. Resource amplification (T6, T15). Inputs or plans that escape InputGuard limits, ResourceLimiter caps or the knapsack meters; unbounded growth in any in-memory map or state store.
8. Data at rest and in logs (T4, T16). Any code path that writes request text to logs, traces, cache keys, feedback, audit or metrics; weaknesses in EncryptedStateStore (nonce reuse, key rotation).
9. Supply chain (T11). Release workflow, attestation coverage, Docker base image, the install.sh / install.ps1 bootstrap scripts served from the web app, the npm package @opensmartroute/sdk.
10. Answer-side leakage (T19). A model answer that carries a credential, personal data or an instruction past OutputGuard in redact / block mode - credential shapes outside SECRET_PATTERNS, PII entities the Redactor misses, injection phrasing under injection_threshold, or the streamed path where the guard can only flag after the fact.

6. Known gaps (do not re-report)

• The SDK does not authenticate callers; osr serve is unauthenticated unless --require-auth or tokens are configured, and the Helm chart says so.
• The gadget detector ships trained on synthetic gadgets; the injection lexicon is English-centric.
• A compromised LLM judge can still bias ranks among legitimate candidate ids.
• A valid-looking poisoned learner snapshot is indistinguishable from real learning; write access to the state store is the boundary.
• Audit verification is only meaningful if the log file is write-once at the OS level.
• Encryption at rest is opt-in; key management is the operator's (load_secret).
• The output guard under redact / block never lets an unscreened token out of a stream: by default it buffers the streamed answer (buffer_streams, the caller sees the screened text chunked after the fact); with buffer_streams=False a StreamScreen holds back the newest OSR_SERVER_OUTPUT_GUARD_HOLD_CHARS characters and releases only cleared text, so a secret split across chunks is caught - the trade-off is that an injection phrase longer than the hold may have partly left before block stops the stream. flag inspects a stream after the fact. The hosted platform defaults to flag (OSR_PLATFORM_OUTPUT_GUARD), osr serve to off.

7. Review log

A completed review adds a row here, a "Threat-model delta" row in Security model for anything that changed, and a CHANGELOG entry for every fix. Until a row exists, the independent report is an open adoption item, not a shipped control.

8. Privacy: data flow and retention (hosted platform)

Reviewed 2026-10-09 against the code; the retention figures below are the defaults a fresh deployment runs with and a test holds this table to the constants, so a change in either shows up as a red build. The customer-facing summary is the privacy policy; PLATFORM.md, "Privacy and data handling" has the request-path diagram.

8.1 Where personal data goes

8.2 Retention

Deletion runs in the retention sweep (every 6 hours, once at start); a workspace on Team or Enterprise can set its own shorter request-row retention, which wins over the deployment default.

8.3 Rights

GET /api/v1/me/export returns the user's profile, identities, sessions and history and the workspace's plan, members, key prefixes, tenants, policy, SSO, usage, requests, outcomes, request logs, evaluation reports and - since this review - conversations with their turns and remembered facts. POST /api/v1/me/delete removes the user (sessions, identities, memberships; workspaces they alone owned are disabled with their keys); POST /api/v1/workspace/delete removes an organization and everything scoped to it, memory included; DELETE /api/v1/activity/logs erases kept payloads on their own.

8.4 Findings of this review

• Fixed. log_payloads: redacted masked PII but not credentials: a pasted API key reached the log row verbatim. Redacted mode now runs the output guard's secret patterns as well.
• Fixed. The export did not carry conversations or memories although the privacy policy promises the dashboard removes them; it does now (workspace.memory).
• Open. Conversations have no age-based retention; a workspace that never deletes keeps them until it is deleted. The per-thread and per-subject caps bound the volume, not the age.
• Open. Journey events are unlinked, not deleted, when a workspace is deleted (the anonymous visitor rows stay for their 400 days); the privacy policy says so, a reviewer may want it tightened.

9. Findings of the 2026-10-14 registration review

• Fixed. A domain rule cannot keep up with a bot that mints a subdomain per account. The signup guard now scores the shape of the registration (digits woven into the local part, a letters-and-digits subdomain label, a word-and-hex display name, a long consonant run) and refuses what adds up to a machine (block_generated, on by default); the per-domain daily cap counts by the registered domain; cc.cd and the tempmail.plus family join the temporary-inbox list. Evidence: platform/api/tests/test_signup_guard.py pins the nine observed registrations as refused and a list of ordinary addresses as accepted.
• Fixed. Clearing the accounts already made meant one rule and one disable per row. The Users page's bulk actions Block their e-mail domains / Block their addresses write the blocklist rule and disable the user in one move, audited per user.
• Fixed. The bot challenge (Turnstile / hCaptcha) was configurable on the API but the web app's Content Security Policy would have blocked its script and iframe in production; both providers are now in script-src, frame-src and connect-src.
• Open (accepted). POST /api/v1/signup answers 409 for an address that already has an account, which lets a caller test whether an address is registered. Password reset (202 either way) and sign-in do not. The 409 is kept because the form shows the API key on the spot and sends the person to sign in otherwise; the per-address sign-up limit (OSR_PLATFORM_SIGNUP_PER_IP_PER_DAY, 5 by default) and the IP blocklist bound the rate at which a list can be tested. A deployment that must not disclose membership runs with OSR_PLATFORM_SIGNUP=false and invitations.
• Operator action, not code. The hosted deployment runs without a bot challenge; set OSR_PLATFORM_CAPTCHA_PROVIDER=turnstile with the site key and secret, a per-domain cap and auto_block_after on the Access page.