Security Model

OpenSmartRoute is an LLM control plane: it decides who answers. That makes its integrity a security property in its own right (Shafran et al., "Rerouting LLM Routers", 2025). This document lists assets, threats, and the controls implemented, mapped to OWASP Top 10 / OWASP LLM Top 10.

Assets#

1. Routing decisions (integrity, availability)
2. Request content (confidentiality – may contain PII / secrets)
3. Learner state (integrity – poisoning changes future decisions)
4. Configuration & credentials of downstream targets
5. Audit trail

Threat model & controls#

#	Threat	OWASP ref	Control	Code
T1	Confounder gadgets – attacker appends token soup to force routing to the expensive model (cost DoS) or to a weaker model (quality attack)	LLM01, LLM04	Head/tail naturalness analysis; signals computed on cleaned head; optional reject; learned gadget detector (InputGuard(learned=True), trained with osr train --gadget); router-directed sentences ("this is extremely complex", [complexity=1]) are scrubbed before signal extraction	security.InputGuard, security.gadget.GadgetDetector, security.injection.strip_steering, GuardMiddleware
T2	Prompt injection into LLM judge – request text tells the judge which target to pick	LLM01	Request wrapped in a fenced block; role markers and code fences neutralised; control/zero-width chars stripped; judge output parsed as strict JSON, item by item: non-object items, unparseable / NaN scores and ids not in the candidate list are dropped, scores clipped to [0,1]; judge only enabled below a confidence threshold; judge failures carry zero confidence so the ensemble ignores them	security.sanitize_for_prompt, LLMJudgeStrategy
T3	Policy bypass – PII to a cloud model, EU data to US region, tenant reading another tenant's targets	A01 Broken access control	Policy evaluated before scoring and cannot be outweighed; tenant middleware; allow/deny lists	policy.Policy, TenantMiddleware
T4	Data leakage via logs / cache / feedback	A09 Logging failures, LLM06	Logs contain hash+length only; cache keys are SHA-256; Redactor masks PII before routing; feedback store records ids, not text	LoggingTelemetry, CacheMiddleware, Redactor, FeedbackStore
T5	Learner poisoning – fake outcomes to steer routing	LLM03 Training-data poisoning	Outcomes only accepted via learn() from your code path (auth is your boundary); graded rewards clipped to [0,1]; Bayesian priors + confidence ramps limit the impact of few samples; Page–Hinkley drift alerts on sudden shifts; state files atomic and hash-addressable; corrupt / incompatible state is quarantined on load instead of being trusted or crashing (AutoLearner.quarantined); federated merge() adds only evidence beyond the prior	AutoLearner, math.estimators, learning.merge_learners
T6	Resource exhaustion – huge prompts, unbounded state	A04 Insecure design / DoS	max_chars/max_tokens guard; every in-memory map is bounded; per-target rate limits and budgets; breaker isolates failing targets	InputGuard, realtime
T7	Path traversal in state store	A01	Keys hashed to filenames; writes are atomic to a fixed root	FileStateStore
T8	Secrets in config	A02 Cryptographic failures / A05 Misconfig	Targets YAML never contains credentials; load_secret() reads env or *_FILE	security.load_secret
T9	Deserialisation	A08 Software & data integrity	Only JSON / yaml.safe_load; no pickle anywhere	TargetRegistry.from_file, learners
T10	Audit tampering	A09	Hash-chained audit records; the chain resumes from the last record across restarts (no second genesis) and FileAuditSink.verify(path) re-walks it naming the first edited, removed or unparseable line	FileAuditSink
T11	Supply chain	A06 Vulnerable components	Zero runtime dependencies in core; optional extras pinned by minimum version; CI runs bandit, ruff -S, mypy	pyproject.toml, CI
T12	Jailbreak content reaching a model	LLM01	jailbreak_risk ≥ 0.5 → only safety-tagged targets or humans are admissible; run_safety_suite() / osr safety regression-tests that safety routing survives paraphrase, gadgets and steering	Policy.check, security.safety
T13	Poisoned catalogue entries – MCP tool descriptions, A2A agent cards or skill files carry instructions to the router or model	LLM01, A08	Instruction-injection lexicon + learned gadget score on every imported description; poisoned tools dropped at import time and reported; signed MCP manifests (HMAC-SHA256 or Ed25519, freshness window) so only vetted catalogues load	security.injection, adapters.mcp.tools_from_mcp, sign_manifest / verify_manifest
T14	Parameter provenance – a tool result or retrieved document smuggles a value into a state-changing tool call ("send the refund to this account")	LLM01, A01	OriginPolicy: sensitive parameters of state-changing tools must originate in the user turn; violations raise OriginViolation before execution	security.provenance
T15	Resource amplification in agentic plans – runaway loops, recursive tool calls, token / cost blow-up	A04, LLM10	Per-task caps on steps, tool calls, depth, tokens, cost and wall-clock enforced as middleware; multi-resource knapsack bandit checks hard meters before commitment and audits every pruned arm	security.limits.ResourceLimiter, ResourceLimitMiddleware, math.bandits.MultiKnapsackBandit
T16	Learner state at rest – snapshots reveal usage patterns or are modified on disk	A02	AES-256-GCM EncryptedStateStore with key rotation (cryptography extra); versioned store refuses to load unknown schema versions	enterprise.stores.EncryptedStateStore, VersionedStateStore

Secure defaults#

• Core has no network access; it never calls a model unless you wire an LLMJudgeStrategy or a target handler.
• Logging is content-free, and so is tracing: span and event attributes carry ids, numbers, short labels and a text digest + length (observability.text_digest), never the request text. The hosted platform reads traces per workspace only (GET /api/v1/trace/{request_id} is 404 for another workspace's id).
• The LLM judge is off by default and, when on, only consulted below a confidence threshold.
• Redactor is opt-in (GuardMiddleware(redact=True)); when on, the original text is kept only in request.context["_pii_map"] for the executor and is never persisted by the SDK.

What OpenSmartRoute does not do#

• Authenticate callers. Put it behind your API gateway / service mesh.
• Encrypt state at rest by default. Use disk/volume encryption or wrap your store in enterprise.stores.EncryptedStateStore.
• Guarantee the downstream target's safety. Routing to a "safe" target is a policy you write.

Threat-model delta: 0.3 -> 0.4#

What changed for operators upgrading, and which residual risks remain:

Area	0.3	0.4	Residual risk
Gadget / steering (T1)	heuristic head/tail analysis	+ learned detector, + steering-sentence scrub before signal extraction	detector is trained on synthetic gadgets; re-train on your traffic (osr train --gadget)
Judge injection (T2)	fenced prompt, strict JSON	+ per-item filtering, id allow-list, NaN / type guards, zero-confidence failures	a compromised judge can still bias ranks among legitimate ids; keep escalate_llm_judge_below <= 0.5
Catalogue poisoning (T13)	none	injection lexicon + gadget score on imported descriptions; signed manifests	lexicon is English-centric; sign manifests for anything you did not author
Parameter provenance (T14)	none	OriginPolicy for state-changing tools	policy is per tool / parameter; unlisted tools are not checked
Resource amplification (T15)	per-target rate limits and budgets	+ per-task caps (steps, calls, depth, tokens, cost, wall-clock), knapsack meters before commitment	caps are enforced in the routing process; a handler that ignores the returned budget can still spend
Learner integrity (T5)	atomic writes	+ quarantine of corrupt / incompatible state, versioned schema, federated merge adds evidence only	a valid-looking poisoned snapshot is indistinguishable from real learning; restrict write access to the store
Audit (T10)	hash chain per process	+ chain continuity across restarts, verify()	the log file itself must be write-once at the OS level to make verification meaningful
State at rest (T16)	plain JSON	+ AES-256-GCM store with rotation	key management is yours (load_secret)
Deployment	none	non-root container, hardened Helm chart (read-only rootfs, drop all capabilities, NetworkPolicy)	the chart exposes /route unauthenticated inside the cluster; front it with your gateway

External review of this delta is outstanding; SECURITY_REVIEW.md is the pack a reviewer works from and its review log is where the report is linked (tracked in ROADMAP.md).

Hardening checklist for operators#

• Run osr serve behind TLS termination and authn/z (OIDC / mTLS).
• Enable GuardMiddleware(redact=True) for any tenant that may send PII.
• Set TargetConstraints.pii_allowed=False and data_boundary on every cloud target.
• Configure HealthRegistry.configure(rate_per_s=…, budget_limit=…) per expensive target.
• Ship MetricsTelemetry to your monitoring; alert on AutoLearner.drifted and breaker opens.
• Rotate and restrict access to the state directory; back it up (it is your learned policy).
• Alert on AutoLearner.quarantined (corrupt state was replaced) and run FileAuditSink.verify() on the audit log from a read-only replica.
• Sign MCP manifests you import (osr mcp-manifest) and fail closed on verify_manifest errors.
• Run osr safety --learned-guard in CI so safety routing regressions block a release.
• Pin the package version; watch the security advisories on this repo.

Reporting#

See SECURITY.md.