osr-security-hardening

Threat model: the router sits between untrusted text (users, retrieved documents, tool output) and expensive or state-changing targets. Defend the input, the catalogue, and the execution budget.

1. Input guard (every deployment)

from opensmartroute.security import InputGuard, GuardMiddleware
guard = InputGuard(max_chars=32_000, max_tokens_est=8_000, gadget_threshold=0.6,
                   reject_on_gadget=False, learned=False, strip_steering=True)
app = RouterBuilder(reg).with_defaults().with_middleware(GuardMiddleware(guard, redact=True)).build()

• guard.inspect(text) -> GuardReport(ok, reasons, gadget_suspected, clean_text, score, steering_removed).
• GuardMiddleware raises SecurityError when not ok; with redact=True PII is replaced before routing (original in request.context["_original_text"], map in context["_pii_map"]).
• learned=True uses GadgetDetector.default(); train your own with osr train --gadget gadget.json and GadgetDetector.load.
• Pure functions for ad-hoc checks: injection_risk(text), inspect_injection(text), strip_steering(text), sanitize_for_prompt(text), shannon_entropy(text).
• moderate(inputs, flag_pii=False) / moderate_text(text) run the guard, the injection lexicon and the Redactor and answer in the OpenAI moderation shape (prompt_injection, router_steering, confounder_gadget, pii, oversize, control_characters); POST /v1/moderations on osr serve and the platform exposes it, so client.moderations.create screens prompts before a model is paid for.
• Answers get their own check: GuardMiddleware(output="redact") / RouterBuilder.with_guard(output=...) / harden(output_guard=...) / osr serve --output-guard flag|redact|block attach an OutputGuard that the executor runs after every model / agent handler - leaked secrets (SECRET_PATTERNS: API keys, private keys, JWTs), PII and instruction injection in the answer are flagged in the trace (output_guard.flagged), redacted to <SECRET_n> / <EMAIL_n>, or the answer is refused (SecurityError). Streamed answers are only flagged.

2. Data boundaries and PII in the catalogue

• Targets declare constraints.data_boundary: public|private|on_prem and pii_allowed. Requests set RequestConstraints(data_boundary=..., contains_pii=...); the PIISignal also detects PII.
• Policy rejects mismatches (DataBoundaryRule, pii rule) - verify in trace.policy_rejections.
• Pair with a rule: when: {contains_pii: true} prefer: [llm-onprem] pin: true.
• Tenant isolation: constraints.tenants on targets + TenantMiddleware(require=True).

3. Budgets for tools, agents and workflows

from opensmartroute.security import ResourceLimits, ResourceLimiter, ResourceLimitMiddleware, apply_limits
limiter = ResourceLimiter(ResourceLimits(max_steps=50, max_tool_calls=100, max_depth=4,
                                         max_total_tokens=500_000, max_cost_usd=5.0, max_wall_s=900.0))
apply_limits(registry, limiter, kinds=("tool", "agent", "workflow"))   # wraps handlers per task_id
builder.with_middleware(ResourceLimitMiddleware(limiter))

Exceeding a limit raises ResourceLimitExceeded (a SecurityError). Budgets key on request.context["task_id"].

Per-target spend and rate caps live in the catalogue: metadata: {rate_limit: {per_minute: 600, burst: 60}, budget: {usd: 50, per: hour}, breaker: {failure_threshold: 3}} on any target is applied by RouterBuilder.with_health() / harden(health=True) / osr serve --health (HealthRegistry.apply_limits); an exhausted bucket or budget rejects the target in the policy and the request spills to the next candidate.

4. Tool-parameter provenance (indirect injection)

from opensmartroute.security import OriginPolicy, OriginRule, mark_untrusted, apply_origin_policy
mark_untrusted(request, retrieved_text, source="retrieved")         # tag text you did not author
policy = OriginPolicy(rules=[OriginRule(state_changing=["delete_*", "send_*"], allowed_origins=["user"])])
apply_origin_policy(registry, policy)                                # wraps tool handlers

A state-changing tool whose arguments were copied from untrusted text raises OriginViolation.

5. MCP catalogues

• tools_from_mcp(payload, max_description_risk=0.6, guard=InputGuard()) drops tools whose descriptions look like steering ("ignore previous instructions ...").
• Sign catalogues offline and verify at load: osr mcp-manifest tools.json --key-env OSR_MCP_KEY --server name --out manifest.json; then tools_from_manifest(manifest, key, require_signature=True, max_age_s=86400). --algorithm ed25519 needs the crypto extra.
• Plans offer tools to the model and run() executes the model's calls to them (T17 in docs/SECURITY.md): only tools that passed the policy filter and tool_quality_floor for the request are offered, calls to anything else are handed back unrun, rounds are capped by tool_rounds, and OriginPolicy / ResourceLimiter apply to the tool handler. OSR_ROUTING_TOOL_SLOTS=0 (or Router(reg, tool_slots=0)) turns the slot off.
• Network-sourced catalogue and knowledge (T18): MarketplaceSync follows only the kinds / query / slugs you set over https; article feeds (OSR_SLM_FEEDS) are byte-capped, parsed without entity expansion and can only make a priced model eligible - never set a quality prior or skip the promotion gates.

6. Secrets

load_secret("OPENAI_API_KEY") reads the env var or the file named by OPENAI_API_KEY_FILE (Kubernetes secret mounts). Never put secrets in targets.yaml; handlers read them at call time. Telemetry never logs raw text.

7. Red-team suite

osr -t targets.yaml safety [--learned-guard] [--json] or run_safety_suite(app.route) runs SafetyCases (jailbreaks, PII exfiltration, boundary hops, cost bombs) and reports pass_rate and by_category. Add it next to osr eval --min-accuracy in CI.

Review checklist

• GuardMiddleware first in the chain, redact=True when any target is pii_allowed: false.
• Every private/on-prem target has data_boundary and regions set; requests carry tenant.
• apply_limits on all tool/agent/workflow targets; max_cost_usd matches the tenant budget.
• MCP tools loaded from a signed manifest; max_description_risk left at or below 0.6.
• Secrets via load_secret; audit sink enabled; osr safety passes in CI.