Loading
Learned model router
Not Diamond recommends the best model for a prompt from a router trained on large evaluation data, and can train a custom router on your evaluation set. OpenSmartRoute learns from the outcomes of your own traffic, per target and per tenant, routes across agents, tools and people as well as models, enforces hard policy first, and runs wherever you do.
“Learned routing you own, for more than models.”
Our reading of public documentation; corrections welcome through the support page.
| Capability | OpenSmartRoute | Not Diamond |
|---|---|---|
| Routes to LLMs | Yes | Yes |
| Routes to agents, skills, tools, workflows and humans | Yes | No |
| Hard policy before scoring (region, data boundary, PII, tenant, cost) | Yes | No |
| Per-request trace with ranked candidates and rejections | Yes | No |
| Learns from your outcomes (per target, per tenant)A custom router trained on an evaluation set you upload; not online learning from production outcomes per tenant | Yes | Partial |
| Multi-step plans (persona -> skill -> model) | Yes | No |
| Self-hosted, air-gapped, zero runtime dependencies | Yes | No |
| Prompt-injection and steering-gadget defences on the router | Yes | No |
| Output guard on responses (PII, secrets, injection: flag, redact or block) | Yes | No |
| Savings ledger, bill analysis and routing auditEvery request records baseline and routed cost; upload last month's provider export to see what routing would have saved before you switch | Yes | No |
| OpenAI-compatible proxy (chat, embeddings, Responses, moderations, audio, images)A recommendation API and client helpers; you call the provider yourself | Yes | Partial |
| Anthropic Messages API (/v1/messages) | Yes | No |
| Virtual keys: per-key budgets, rate limits, allowed targets, scopes and expiry | Yes | No |
| Per-target rate limits, budgets and circuit breakers | Yes | No |
| Python and TypeScript SDKs | Yes | Yes |
| MCP server for IDEs and agents | Yes | No |
| Browser extension and desktop app for the people on the teamPrice, personal-data and writing checks on the device before a prompt is sent, on ChatGPT, Claude, Gemini and in any desktop app | Yes | No |
| Provider marketplace with creditsModels by the token: every catalogue model on offer per provider with the price you pay, one key, a prepaid credits wallet and provider preferences (order, only, ignore, max_price) | Yes | No |
| Request/response observability UI for every callDecision trace, tokens, cost and outcome per request; prompt and answer stored when the workspace turns on request logging (redacted or full) | Yes | No |
Evaluate any learned router against yours with the same file: `osr eval` prints accuracy against your labels, the cost/quality frontier and calibration, so the choice is a number, not a pitch.
# labelled rows: {"text": "...", "expected_target": "llm-mid"} per line
osr -t targets.yaml eval labelled.jsonl --frontier --calibration --min-accuracy 0.85
# train the router on your own outcomes and ship the model with the catalogue
osr -t targets.yaml train outcomes.jsonl --out models.jsonRun the Routing Audit on a week of anonymised logs: savings versus your current mix, policy violations your routing allowed, and where agents or humans would have been the better answer.
Other comparisons: OpenRouter, LiteLLM, Portkey, Cloudflare AI Gateway, Kong AI Gateway, Helicone, Martian, Unify, Building it yourself.