Loading
Model router and benchmarks
Unify routes to the model endpoint that best balances quality, speed and cost from its live benchmarks, behind one API key. OpenSmartRoute uses your outcomes rather than public benchmarks, weighs cost and quality per request under hard policy, routes to more than models and runs self-hosted.
“Benchmarks say what is good in general; your outcomes say what is good for you.”
Our reading of public documentation; corrections welcome through the support page.
| Capability | OpenSmartRoute | Unify |
|---|---|---|
| Routes to LLMs | Yes | Yes |
| Routes to agents, skills, tools, workflows and humans | Yes | No |
| Hard policy before scoring (region, data boundary, PII, tenant, cost) | Yes | No |
| Per-request trace with ranked candidates and rejections | Yes | No |
| Learns from your outcomes (per target, per tenant)Routing from public benchmark scores; no learning from your reported outcomes | Yes | Partial |
| Multi-step plans (persona -> skill -> model) | Yes | No |
| Self-hosted, air-gapped, zero runtime dependencies | Yes | No |
| Prompt-injection and steering-gadget defences on the router | Yes | No |
| Output guard on responses (PII, secrets, injection: flag, redact or block) | Yes | No |
| Savings ledger, bill analysis and routing auditEvery request records baseline and routed cost; upload last month's provider export to see what routing would have saved before you switch | Yes | No |
| OpenAI-compatible proxy (chat, embeddings, Responses, moderations, audio, images)Chat through an OpenAI-compatible endpoint; other routes follow the provider | Yes | Partial |
| Anthropic Messages API (/v1/messages) | Yes | No |
| Virtual keys: per-key budgets, rate limits, allowed targets, scopes and expiry | Yes | No |
| Per-target rate limits, budgets and circuit breakers | Yes | No |
| Python and TypeScript SDKs | Yes | Yes |
| MCP server for IDEs and agents | Yes | No |
| Browser extension and desktop app for the people on the teamPrice, personal-data and writing checks on the device before a prompt is sent, on ChatGPT, Claude, Gemini and in any desktop app | Yes | No |
| Provider marketplace with creditsModels by the token: every catalogue model on offer per provider with the price you pay, one key, a prepaid credits wallet and provider preferences (order, only, ignore, max_price) | Yes | Yes |
| Request/response observability UI for every callDecision trace, tokens, cost and outcome per request; prompt and answer stored when the workspace turns on request logging (redacted or full) | Yes | No |
Run both for a week on the same traffic and report every outcome to the router: the rankings page shows each target's success with its confidence bound, and the savings page shows what it cost against your baseline.
osr -t targets.yaml audit traffic.jsonl --baseline llm-frontier --monthly 3000000 --markdown
# or, in production: every reported outcome moves the ranking
curl -s $OSR/api/v1/feedback -H "Authorization: Bearer $OSR_API_KEY" \
-d '{"request_id": "...", "target_id": "llm-mid", "success": true, "quality": 0.9}'Run the Routing Audit on a week of anonymised logs: savings versus your current mix, policy violations your routing allowed, and where agents or humans would have been the better answer.
Other comparisons: OpenRouter, LiteLLM, Portkey, Cloudflare AI Gateway, Kong AI Gateway, Helicone, Not Diamond, Martian, Building it yourself.