Skip to content
OpenSmartRoute

v1.0.0Open source, Apache-2.0

The right model, agent or human for every request. With the reason.

OpenSmartRoute sits in front of your LLMs, agents, tools and people. Send a request; get the best target under your cost, latency and data constraints, together with why it was chosen. Report the outcome and the next decision is better.

curl -LsSf https://opensmartroute.ai/install.sh | sh
route
Try
212
tokens routed
18
requests, all time
212
tokens, last 24 h
100.0%
success rate
430
models in the catalogue
2
accounts routing here

Measured on this deployment, updated continuously. No prompt text is stored; counters aggregate anonymised usage records.

Why teams switch

Lower bills, better answers, and proof of both.

No estimates you have to take on faith: the audit measures savings on your own traffic, and every routed request explains itself.

70%

of spend is typically over-served

Pay frontier prices only when the request needs it

Most traffic is simple. The router quotes every capable target and sends each request to the cheapest one that meets your quality bar - the audit replays a week of your own logs and prints the exact savings before you commit.

Compute your ROI

100%

of decisions carry the reason

Trust it, because it shows its work

Every decision returns the per-strategy scores, the policy rejections and the confidence. Feed outcomes back and the learners improve; the hash-chained audit trail makes each choice reproducible for compliance.

Watch a live decision

$0

to run it yourself, forever

Open source, no lock-in

Apache-2.0, zero runtime dependencies. Run it as a Python library, a sidecar container or this hosted platform; your catalogue, rules and learned state move with you either way.

Read the docs

The same router, your industry's rules

Customer support
Small models answer FAQs; angry or complex tickets escalate to a frontier model or a human.
Software engineering
Code requests go to the best coding model; everything else stops paying coding-model prices.
Financial services
Requests with account data pin to your on-prem model; the audit trail satisfies the regulator.
Healthcare
PII never leaves your boundary - a policy rule, not a promise. Everything else routes on cost.
Legal
Long contracts go to large-context models; a confidence floor routes low-certainty answers to counsel.
Retail & marketing
Bulk copy runs on cheap models within a per-tenant budget; brand-critical work gets the best.
rules.yaml - healthcareyaml
rules:
  - name: phi-stays-onprem
    when: { contains_pii: true }
    prefer: [llm-onprem]
    avoid: [llm-frontier, llm-mid]
    weight: 1.0

  - name: clinical-review
    when: { domains: [medical], min_complexity: 0.6 }
    prefer: [clinician-review]
    pin: true

Constraints and preferences live in two YAML files - targets.yaml and rules.yaml - so switching industries is editing text, not retraining a model.

How it works

Two files, one call, a decision you can defend.

The same catalogue and rules drive the Python SDK, the sidecar container and this hosted platform.

  1. 01

    Describe your destinations

    Declare each LLM, agent, skill, persona, tool or human once, with capabilities, complexity band, cost, latency and a quality prior. Import MCP tool catalogues, A2A agent cards and SKILL.md packages directly.

    targets.yamlyaml
    # targets.yaml - one catalogue for every destination
    targets:
      - id: llm-frontier
        kind: llm
        capabilities:
          domains: [math, coding, legal, finance]
          actions: [reasoning, planning, code_generation]
          min_complexity: 0.5
        constraints: { pii_allowed: false }
        cost: { usd_per_1k_tokens: 0.010 }
        latency_ms: 1800
        quality_prior: 0.92
    
      - id: coding-agent
        kind: agent
        capabilities:
          domains: [coding]
          actions: [code_generation, code_review, debugging]
        cost: { usd_per_1k_tokens: 0.004 }
        latency_ms: 4000
    
      - id: llm-onprem
        kind: llm
        data_boundary: on_prem
        regions: [eu, in]
        constraints: { pii_allowed: true }
  2. 02

    State what must never happen

    Rules prefer, avoid or pin targets when signals match. Per-request and per-tenant constraints restrict region, data boundary, cost and latency. Policies run before scoring, so no weight can trade them away.

    rules.yamlyaml
    # rules.yaml - preferences on top of hard policy
    rules:
      - name: pii-stays-onprem
        when: { contains_pii: true }
        prefer: [llm-onprem, human-escalation]
        avoid: [llm-frontier, llm-mid, llm-small]
        weight: 1.0
    
      - name: hard-coding-to-agent
        when: { domains: [coding], min_complexity: 0.45 }
        prefer: [coding-agent]
        weight: 0.7
    
      - name: escalation-intent
        when: { actions: [escalate] }
        prefer: [human-escalation]
        pin: true
  3. 03

    Route, read the trace, send feedback

    Every decision returns the winner, the runner-ups with a per-strategy breakdown, and which policy rejected what. Report the outcome and the learners adjust the next decision.

    POST /api/v1/routejson
    {
      "target": { "id": "coding-agent", "kind": "agent" },
      "confidence": 0.87,
      "elapsed_ms": 4.2,
      "signals": {
        "task_type": "coding", "complexity": 0.71,
        "contains_pii": false, "jailbreak_risk": 0.02
      },
      "policy_rejections": {
        "llm-small": "complexity 0.71 above max_complexity 0.45"
      },
      "ranked": [
        { "id": "coding-agent",  "utility": 0.91 },
        { "id": "llm-frontier",  "utility": 0.78 },
        { "id": "llm-mid",       "utility": 0.61 }
      ]
    }

The routing SLM

The router is itself a small model.

OpenSmartRoute ships a routing SLM: a small model trained on preference data - public benchmarks and your own logged decisions - that predicts which target wins a request before it runs. It joins the strategy ensemble as one more voice, and the autopilot keeps it current without anyone retraining anything by hand.

How the SLM works
from logs to a self-tuning router
# corpus: ten public preference datasets + your own traffic
$ osr collect --cache-dir data --tier frontier=llm-frontier --tier mid=llm-mid --tier small=llm-small
# train the routing SLM - no GPU; numpy makes it ten times faster
$ osr slm train --cache-dir data --out slm.json --report
$ osr slm eval slm.json holdout.jsonl # accuracy 0.94 on RouteLLM's labels
# serve with it; it retrains itself on drift
$ osr -t targets.yaml --slm slm.json serve --autopilot
holdout accuracy on RouteLLM's GPT-4-judge battles (cheap-model-good-enough labels), vs 0.86 for always picking the cheaper tier; across all ten sources the model is a smaller step over the strongest static baseline
0.94

holdout accuracy on RouteLLM's GPT-4-judge battles (cheap-model-good-enough labels), vs 0.86 for always picking the cheaper tier; across all ten sources the model is a smaller step over the strongest static baseline

real preference rows from ten public datasets (LMArena, RouteLLM, RewardBench, UltraFeedback ...)
18,459

real preference rows from ten public datasets (LMArena, RouteLLM, RewardBench, UltraFeedback ...)

of learned state as plain JSON - readable by the zero-dependency router
4 MB

of learned state as plain JSON - readable by the zero-dependency router

to retrain on the full corpus in-process, without pausing serving
~1 min

to retrain on the full corpus in-process, without pausing serving

Autopilot

A Page-Hinkley drift monitor watches the outcomes you report. When routing quality drifts, the server collects fresh evidence, retrains and swaps the model in place - while requests keep flowing.

Champion vs challenger

Every candidate model is evaluated against the incumbent on a held-out slice. A challenger is promoted only when it wins; a bad batch of data can never make routing worse.

Distilled from the ensemble

osr slm train --distill compresses the full strategy ensemble - rules, similarity, learners - into the one small model, so the edge deployment routes like the fleet does.

The model landscape

Quality per dollar is not a straight line.

430 hosted models from 51 vendors, list prices from $0.01 to over $50 per million tokens and a 56-model benchmark set. The router's job is to land each request on the frontier for your constraints rather than on the most expensive point.

Explore the catalogue
Models
430
51 vendors
Median list price
$0.85
blended, per 1M tokens
Best value on the frontier
Gemini 3.8 Flash
$0.75 batch · index 47, within 10 of the top
Released last 30 days
46
newest Sep 4, 2026

Price against quality

Blended list price (3 input : 1 output tokens) per million against the intelligence index. Composite of reasoning, knowledge and instruction-following evaluations.

Library, sidecar or hosted API

Use it your way.

Embed the SDK in a Python service, run the container beside your app, or call this platform. The OpenAI-compatible endpoint means existing clients switch by changing one base URL.

Decorator SDK
Custom strategies, policy rules, signals, middleware and telemetry sinks registered with a decorator.
Adapters
MCP tool catalogues, A2A agent cards, LangGraph nodes and the Microsoft Agent Framework.
Enterprise builder
Auto-learning, health, budgets, tenants, guard, audit and Redis, SQL or encrypted state.
MCP server
Expose route, estimate and explain as tools to Claude, Cursor and every other MCP client.
route.pypython
pip install "opensmartroute[yaml]"

from opensmartroute import (
    Router, RouteRequest, RequestConstraints, Objective,
    load_targets, load_rules,
)
from opensmartroute.strategies import default_strategies

router = Router(
    load_targets("targets.yaml"),
    strategies=[load_rules("rules.yaml"), *default_strategies()],
)

req = RouteRequest(
    "Prove that the square root of two is irrational.",
    constraints=RequestConstraints(max_cost_per_1k=0.02),
    objective=Objective(quality=1.0, cost=0.4),
)
d = router.route(req, plan=True)
print(d.target.id, f"{d.confidence:.2f}")
print(d.trace.explain())          # per-strategy scores and rationales
print(d.trace.policy_rejections)  # who was filtered out, and why

Editions

Same router, two depths.

The community edition is the open-source SDK. The enterprise edition adds the RouterBuilder stack for multi-tenant, high-volume and regulated environments. Both run as a library, a container or on this platform.

Capability
Community
Apache-2.0, free forever
Enterprise
self-hosted or managed
Routing decisions with full trace
OpenAI-compatible proxy (model="auto")
Hard constraints: PII, region, data boundary, cost, latency
Plans (persona -> skill -> model) and execution
Outcome feedback to the learnersin-processpersisted
IRT, Bradley-Terry, LinUCB and Markov auto-learning
Health circuit breakers and budgets
Per-tenant constraints and isolation
Guard middleware (injection, gadget, PII redaction)
Hash-chained audit trail
Metrics, tracing and per-request traces
Redis, SQL and encrypted state stores

Get started

Three steps to your first routed request.

No SDK required for the hosted API; the Python package is there when you want to embed the router.

  1. 01

    Create an account

    Email and password, no card. The Free plan includes 500 routed decisions a day with the full explanation trace.

    Sign up
  2. 02

    Mint an API key

    Keys are shown once and stored hashed. Rotate or revoke them from the dashboard; every request is attributed to the key that made it.

    Open keys
  3. 03

    Point your client here

    Change one base URL. model="auto" lets the router decide; pass a list of candidates for scored fallbacks; set stream=true for SSE.

    Try the playground
Your first requestbash
curl -s https://api.opensmartroute.ai/v1/chat/completions \
  -H "Authorization: Bearer $OSR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model":"auto","messages":[{"role":"user","content":"Hello"}]}'

The response is a standard chat completion. reply.model is the target that answered, and the opensmartroute metadata carries the request id, plan and cost so you can send feedback later.

Prefer the CLI

curl -LsSf https://opensmartroute.ai/install.sh | shosr login

Or the container

docker run --rm -p 8000:8000 -v ./examples:/config ghcr.io/isathish/opensmartroute:latest

Route your first request in under a minute.

Free plan, no card. Five hundred decisions a day with the full trace.