v1.0.0Open source, Apache-2.0430 models from 51 vendors in the catalogue
The right model, agent or human for every request. With the reason.
OpenSmartRoute sits in front of your LLMs, agents, tools and people. Send a request; get the best target under your cost, latency and data constraints, together with why it was chosen. Report the outcome and the next decision is better.
curl -LsSf https://opensmartroute.ai/install.sh | shMost routed, last 7 days
Rankings- requests routed
- 18
- targets on this deployment
- 17
- models in the catalogue
- 430
- 212
- tokens routed
- 18
- requests, all time
- 212
- tokens, last 24 h
- 100.0%
- success rate
- 430
- models in the catalogue
- 2
- accounts routing here
Measured on this deployment, updated continuously. No prompt text is stored; counters aggregate anonymised usage records.
Why teams switch
Lower bills, better answers, and proof of both.
No estimates you have to take on faith: the audit measures savings on your own traffic, and every routed request explains itself.
70%
of spend is typically over-served
Pay frontier prices only when the request needs it
Most traffic is simple. The router quotes every capable target and sends each request to the cheapest one that meets your quality bar - the audit replays a week of your own logs and prints the exact savings before you commit.
Compute your ROI100%
of decisions carry the reason
Trust it, because it shows its work
Every decision returns the per-strategy scores, the policy rejections and the confidence. Feed outcomes back and the learners improve; the hash-chained audit trail makes each choice reproducible for compliance.
Watch a live decision$0
to run it yourself, forever
Open source, no lock-in
Apache-2.0, zero runtime dependencies. Run it as a Python library, a sidecar container or this hosted platform; your catalogue, rules and learned state move with you either way.
Read the docsThe same router, your industry's rules
- Customer support
- Small models answer FAQs; angry or complex tickets escalate to a frontier model or a human.
- Software engineering
- Code requests go to the best coding model; everything else stops paying coding-model prices.
- Financial services
- Requests with account data pin to your on-prem model; the audit trail satisfies the regulator.
- Healthcare
- PII never leaves your boundary - a policy rule, not a promise. Everything else routes on cost.
- Legal
- Long contracts go to large-context models; a confidence floor routes low-certainty answers to counsel.
- Retail & marketing
- Bulk copy runs on cheap models within a per-tenant budget; brand-critical work gets the best.
rules:
- name: phi-stays-onprem
when: { contains_pii: true }
prefer: [llm-onprem]
avoid: [llm-frontier, llm-mid]
weight: 1.0
- name: clinical-review
when: { domains: [medical], min_complexity: 0.6 }
prefer: [clinician-review]
pin: trueConstraints and preferences live in two YAML files - targets.yaml and rules.yaml - so switching industries is editing text, not retraining a model.
How it works
Two files, one call, a decision you can defend.
The same catalogue and rules drive the Python SDK, the sidecar container and this hosted platform.
- 01
Describe your destinations
Declare each LLM, agent, skill, persona, tool or human once, with capabilities, complexity band, cost, latency and a quality prior. Import MCP tool catalogues, A2A agent cards and SKILL.md packages directly.
targets.yamlyaml# targets.yaml - one catalogue for every destination targets: - id: llm-frontier kind: llm capabilities: domains: [math, coding, legal, finance] actions: [reasoning, planning, code_generation] min_complexity: 0.5 constraints: { pii_allowed: false } cost: { usd_per_1k_tokens: 0.010 } latency_ms: 1800 quality_prior: 0.92 - id: coding-agent kind: agent capabilities: domains: [coding] actions: [code_generation, code_review, debugging] cost: { usd_per_1k_tokens: 0.004 } latency_ms: 4000 - id: llm-onprem kind: llm data_boundary: on_prem regions: [eu, in] constraints: { pii_allowed: true } - 02
State what must never happen
Rules prefer, avoid or pin targets when signals match. Per-request and per-tenant constraints restrict region, data boundary, cost and latency. Policies run before scoring, so no weight can trade them away.
rules.yamlyaml# rules.yaml - preferences on top of hard policy rules: - name: pii-stays-onprem when: { contains_pii: true } prefer: [llm-onprem, human-escalation] avoid: [llm-frontier, llm-mid, llm-small] weight: 1.0 - name: hard-coding-to-agent when: { domains: [coding], min_complexity: 0.45 } prefer: [coding-agent] weight: 0.7 - name: escalation-intent when: { actions: [escalate] } prefer: [human-escalation] pin: true - 03
Route, read the trace, send feedback
Every decision returns the winner, the runner-ups with a per-strategy breakdown, and which policy rejected what. Report the outcome and the learners adjust the next decision.
POST /api/v1/routejson{ "target": { "id": "coding-agent", "kind": "agent" }, "confidence": 0.87, "elapsed_ms": 4.2, "signals": { "task_type": "coding", "complexity": 0.71, "contains_pii": false, "jailbreak_risk": 0.02 }, "policy_rejections": { "llm-small": "complexity 0.71 above max_complexity 0.45" }, "ranked": [ { "id": "coding-agent", "utility": 0.91 }, { "id": "llm-frontier", "utility": 0.78 }, { "id": "llm-mid", "utility": 0.61 } ] }
The routing SLM
The router is itself a small model.
OpenSmartRoute ships a routing SLM: a small model trained on preference data - public benchmarks and your own logged decisions - that predicts which target wins a request before it runs. It joins the strategy ensemble as one more voice, and the autopilot keeps it current without anyone retraining anything by hand.
How the SLM works# corpus: ten public preference datasets + your own traffic$ osr collect --cache-dir data --tier frontier=llm-frontier --tier mid=llm-mid --tier small=llm-small# train the routing SLM - no GPU; numpy makes it ten times faster$ osr slm train --cache-dir data --out slm.json --report$ osr slm eval slm.json holdout.jsonl # accuracy 0.94 on RouteLLM's labels# serve with it; it retrains itself on drift$ osr -t targets.yaml --slm slm.json serve --autopilot
- holdout accuracy on RouteLLM's GPT-4-judge battles (cheap-model-good-enough labels), vs 0.86 for always picking the cheaper tier; across all ten sources the model is a smaller step over the strongest static baseline
- 0.94
- real preference rows from ten public datasets (LMArena, RouteLLM, RewardBench, UltraFeedback ...)
- 18,459
- of learned state as plain JSON - readable by the zero-dependency router
- 4 MB
- to retrain on the full corpus in-process, without pausing serving
- ~1 min
holdout accuracy on RouteLLM's GPT-4-judge battles (cheap-model-good-enough labels), vs 0.86 for always picking the cheaper tier; across all ten sources the model is a smaller step over the strongest static baseline
real preference rows from ten public datasets (LMArena, RouteLLM, RewardBench, UltraFeedback ...)
of learned state as plain JSON - readable by the zero-dependency router
to retrain on the full corpus in-process, without pausing serving
Autopilot
A Page-Hinkley drift monitor watches the outcomes you report. When routing quality drifts, the server collects fresh evidence, retrains and swaps the model in place - while requests keep flowing.
Champion vs challenger
Every candidate model is evaluated against the incumbent on a held-out slice. A challenger is promoted only when it wins; a bad batch of data can never make routing worse.
Distilled from the ensemble
osr slm train --distill compresses the full strategy ensemble - rules, similarity, learners - into the one small model, so the edge deployment routes like the fleet does.
The model landscape
Quality per dollar is not a straight line.
430 hosted models from 51 vendors, list prices from $0.01 to over $50 per million tokens and a 56-model benchmark set. The router's job is to land each request on the frontier for your constraints rather than on the most expensive point.
- Models
- 430
- 51 vendors
- Median list price
- $0.85
- blended, per 1M tokens
- Best value on the frontier
- Gemini 3.8 Flash
- $0.75 batch · index 47, within 10 of the top
- Released last 30 days
- 46
- newest Sep 4, 2026
Price against quality
Blended list price (3 input : 1 output tokens) per million against the intelligence index. Composite of reasoning, knowledge and instruction-following evaluations.
- OpenAI
- Anthropic
- DeepSeek
- Qwen
- Z.ai
- MiniMax
- Other vendors
- Pareto frontier
On the frontier: Claude Fable 5.1 (batch), Claude Opus 5 (batch), GPT-5.6 Sol (batch), Gemini 3.8 Flash (batch), GLM 5.3 Flash, DeepSeek V4 Flash 0731, Ling 3.0 Flash.
Benchmark leaders
56 models publish the intelligence index.
Library, sidecar or hosted API
Use it your way.
Embed the SDK in a Python service, run the container beside your app, or call this platform. The OpenAI-compatible endpoint means existing clients switch by changing one base URL.
- Decorator SDK
- Custom strategies, policy rules, signals, middleware and telemetry sinks registered with a decorator.
- Adapters
- MCP tool catalogues, A2A agent cards, LangGraph nodes and the Microsoft Agent Framework.
- Enterprise builder
- Auto-learning, health, budgets, tenants, guard, audit and Redis, SQL or encrypted state.
- MCP server
- Expose route, estimate and explain as tools to Claude, Cursor and every other MCP client.
pip install "opensmartroute[yaml]"
from opensmartroute import (
Router, RouteRequest, RequestConstraints, Objective,
load_targets, load_rules,
)
from opensmartroute.strategies import default_strategies
router = Router(
load_targets("targets.yaml"),
strategies=[load_rules("rules.yaml"), *default_strategies()],
)
req = RouteRequest(
"Prove that the square root of two is irrational.",
constraints=RequestConstraints(max_cost_per_1k=0.02),
objective=Objective(quality=1.0, cost=0.4),
)
d = router.route(req, plan=True)
print(d.target.id, f"{d.confidence:.2f}")
print(d.trace.explain()) # per-strategy scores and rationales
print(d.trace.policy_rejections) # who was filtered out, and whyEditions
Same router, two depths.
The community edition is the open-source SDK. The enterprise edition adds the RouterBuilder stack for multi-tenant, high-volume and regulated environments. Both run as a library, a container or on this platform.
| Capability | Community Apache-2.0, free forever | Enterprise self-hosted or managed |
|---|---|---|
| Routing decisions with full trace | ||
| OpenAI-compatible proxy (model="auto") | ||
| Hard constraints: PII, region, data boundary, cost, latency | ||
| Plans (persona -> skill -> model) and execution | ||
| Outcome feedback to the learners | in-process | persisted |
| IRT, Bradley-Terry, LinUCB and Markov auto-learning | ||
| Health circuit breakers and budgets | ||
| Per-tenant constraints and isolation | ||
| Guard middleware (injection, gadget, PII redaction) | ||
| Hash-chained audit trail | ||
| Metrics, tracing and per-request traces | ||
| Redis, SQL and encrypted state stores |
Get started
Three steps to your first routed request.
No SDK required for the hosted API; the Python package is there when you want to embed the router.
- 01
Create an account
Email and password, no card. The Free plan includes 500 routed decisions a day with the full explanation trace.
Sign up - 02
Mint an API key
Keys are shown once and stored hashed. Rotate or revoke them from the dashboard; every request is attributed to the key that made it.
Open keys - 03
Point your client here
Change one base URL. model="auto" lets the router decide; pass a list of candidates for scored fallbacks; set stream=true for SSE.
Try the playground
curl -s https://api.opensmartroute.ai/v1/chat/completions \
-H "Authorization: Bearer $OSR_API_KEY" \
-H "Content-Type: application/json" \
-d '{"model":"auto","messages":[{"role":"user","content":"Hello"}]}'The response is a standard chat completion. reply.model is the target that answered, and the opensmartroute metadata carries the request id, plan and cost so you can send feedback later.
Prefer the CLI
curl -LsSf https://opensmartroute.ai/install.sh | shosr loginOr the container
docker run --rm -p 8000:8000 -v ./examples:/config ghcr.io/isathish/opensmartroute:latestRoute your first request in under a minute.
Free plan, no card. Five hundred decisions a day with the full trace.