Cost estimates, recommended models and the MCP server

Quote a request before sending it, recommended models per use case, and the MCP server that puts the router in VS Code, Cursor, Claude and any agent.

docs/MCP.md

Three ways to ask the router "what will this cost and what should I use?" before spending tokens: a quote API and page, a recommended-models digest, and a Model Context Protocol server that puts all of it inside VS Code, Cursor, Claude, Windsurf or any agent framework. The same pieces exist in the SDK (opensmartroute.estimate, opensmartroute.mcp_server, osr estimate, osr mcp) and on the hosted platform (POST /api/v1/estimate, GET /api/v1/models/recommended, /mcp, the /estimate page and Dashboard -> Integrations).

1. Quotes#

A quote routes the request exactly as route() would - signals, policy, strategies, learned quality - and then prices every ranked candidate instead of executing one:

Tokens. Input tokens come from the text (word / digit / punctuation / CJK / code heuristic, estimate_tokens) plus the history (estimate_messages_tokens, 4 tokens of framing per message). Output tokens are the caller's output_tokens, else the signals' predicted answer length, else DEFAULT_OUTPUT_TOKENS (256); output_tokens_source says which.
Prices. usd_per_1k_input / usd_per_1k_output on the target when declared, else the blended usd_per_1k_tokens; a fixed usd_per_call is added for agents, tools and humans. On the platform a target that resolves to a reference model is priced from the live catalogue (priced_from: "catalogue"), so quotes follow vendor list prices.
Picks. recommended is the router's decision; best_quality the highest quality estimate; cheapest and fastest are chosen among candidates within quality_tolerance (0.1) of the best that fit their context window, so a free-but-useless target never wins. savings_usd is the gap between the dearest candidate and the recommended one.
Nothing is executed and the text is never stored.

from opensmartroute.estimate import estimate

q = estimate(router, "Summarise this contract in five bullets", output_tokens=300)
print(q.input_tokens, q.output_tokens, q.output_tokens_source)   # 9 300 caller
print(q.recommended.name, q.recommended.cost_usd)                # Small model 0.000062
for t in q.targets:                                              # every candidate, ranked
    print(t.rank, t.target_id, f"${t.cost_usd:.6f}", t.quality_estimate, t.latency_ms, t.fits_context)
print(q.cheapest.target_id, q.best_quality.target_id, q.fastest.target_id, q.savings_usd)
q.to_dict()                                                      # the JSON the API returns

estimate(router, request, *, output_tokens=None, kinds=None, prices=None, quality_tolerance=0.1, decision=None) accepts a RouteRequest or a string, a kinds filter and a prices(target) -> (usd_per_1k_in, usd_per_1k_out) | None hook for live price feeds; pass decision= to quote what an earlier route() already ranked.

osr -t targets.yaml estimate "Translate this contract into German" --monthly 100000
osr -t targets.yaml estimate "..." --output-tokens 500 --cost-weight 1.0 --json

`POST /api/v1/estimate`#

Same body as POST /api/v1/route (text, history, objective, constraints, kinds) plus output_tokens and monthly_requests. Without a key the endpoint is rate limited per client (20 per minute, 8 000 characters) and ignores tenant rules; with a key it is metered as estimate, applies your plan and tenants and returns X-Request-Id.

curl -s https://<platform>/api/v1/estimate -H "Authorization: Bearer $OSR_KEY" \
  -H "Content-Type: application/json" \
  -d '{"text": "Summarise this contract...", "monthly_requests": 50000}'

{
  "request_id": "…", "input_tokens": 9, "output_tokens": 256, "output_tokens_source": "signals",
  "total_tokens": 265, "confidence": 0.71, "savings_usd": 0.00209,
  "recommended": {"target_id": "llm-small", "name": "Small model", "vendor": "openai", "model": "openai/gpt-4.1-nano",
                  "input_cost_usd": 0.0000009, "output_cost_usd": 0.0001, "cost_usd": 0.000103,
                  "latency_ms": 400, "quality_estimate": 0.62, "fits_context": true, "priced_from": "catalogue",
                  "rationale": {"semantic": "…", "cost": "…"}},
  "cheapest": {"…": "…"}, "best_quality": {"…": "…"}, "fastest": {"…": "…"},
  "targets": [{"…": "…"}],
  "signals": {"complexity": 0.31, "domains": ["legal"], "task_type": "summarise", "contains_pii": false, "…": "…"},
  "policy_rejections": {},
  "monthly": {"requests": 50000, "recommended_usd": 5.15, "cheapest_usd": 5.15, "best_quality_usd": 109.6,
              "dearest_usd": 109.6, "savings_usd": 104.45, "tokens": 13250000},
  "anonymous": false
}

The /estimate page is the same call with a form: paste a prompt, choose what to optimise for, set the expected answer length and monthly volume, and read the four picks, the signals the router saw and the full candidate table with input / output cost bars.

2. Recommended models#

GET /api/v1/models/recommended (no key) quotes a representative prompt per use case - chat, code, reasoning, summarise, extraction, pii - against the live catalogue and returns, for each, the recommended, cheapest, best_quality and fastest pick with cost per request, quality estimate, latency, 30-day traffic, the reference model's intelligence index and the reasons (why). A leaderboard counts how often each target was the router's pick. The digest is recomputed at most once a minute and follows the deployment's targets, prices and learned quality rather than a hand-maintained list. It renders on /models ("Recommended per use case") and on /estimate.

3. MCP server#

What it exposes#

Tool	Purpose	Side effects
`route`	Pick the best target: decision, confidence, alternatives, signals, ranked candidates with reasons	none
`estimate`	The quote above for a request	none
`recommend`	One-paragraph recommendation for a task; `priority` = balanced, cost, quality or speed	none
`explain`	Human-readable trace: signals, policy filtering, per-strategy breakdown	none
`list_targets`	The catalogue (optionally one `kind`) with prices, latency, quality prior, boundary	none
`feedback`	Report an outcome (`request_id`, `target_id`, `success`, `quality`, `cost_usd`, `latency_ms`)	learners update
`ask`	Route and run the chosen target / plan; returns the answer, tokens and cost	executes
`marketplace_search`, `marketplace_get`	Search listings; fetch one with its manifest and snippets	none

Resources: osr://targets (the catalogue as JSON) and osr://stats (feedback statistics). The server speaks MCP 2025-06-18 (also accepts 2025-03-26 and 2024-11-05) over JSON-RPC 2.0: initialize, ping, tools/list, tools/call, resources/list, resources/read, prompts/list, logging/setLevel; notifications get no reply; unknown methods return -32601, unknown tools -32602, and tool failures come back as isError results, never as protocol errors.

ask and marketplace_* appear only when the server was given an execute / marketplace hook: osr serve and the platform provide both; a bare MCPServer(router) exposes the read-only tools and feedback.

Hosted platform#

GET /mcp describes the server (tools, resources, protocol version, connection snippets) without a key. POST /mcp takes one JSON-RPC message or a batch with the same Authorization: Bearer <api key> (or X-API-Key) header as the REST API; notifications return 202. Every routed call carries the workspace tenant and is metered as mcp against the plan's quotas; feedback needs the feedback feature and ask the plan feature (Pro and above). Quota, plan and routing errors surface as tool errors such as 429: rate limit exceeded. Dashboard -> Integrations renders ready-to-paste configuration for each client with the workspace's key.

HTTP transport (the client talks to https://<platform>/mcp directly):

// VS Code: .vscode/mcp.json
{"servers": {"opensmartroute": {"type": "http", "url": "https://<platform>/mcp",
                                "headers": {"Authorization": "Bearer osr_..."}}}}
// Cursor: .cursor/mcp.json
{"mcpServers": {"opensmartroute": {"url": "https://<platform>/mcp",
                                   "headers": {"Authorization": "Bearer osr_..."}}}}
// Windsurf: ~/.codeium/windsurf/mcp_config.json
{"mcpServers": {"opensmartroute": {"serverUrl": "https://<platform>/mcp",
                                   "headers": {"Authorization": "Bearer osr_..."}}}}

claude mcp add --transport http opensmartroute https://<platform>/mcp --header "Authorization: Bearer osr_..."

Stdio bridge for clients that only launch local processes (Claude Desktop and older clients): the CLI forwards each line to the platform with your key.

{"mcpServers": {"opensmartroute": {"command": "osr",
                                   "args": ["mcp", "--url", "https://<platform>/mcp", "--api-key", "osr_..."]}}}

Self-hosted and local#

osr serve mounts the same server at POST /mcp (and GET /mcp for the description) next to /route and /v1, with ask backed by the configured providers. Without a service, the CLI runs the server over stdio straight from a catalogue - useful for a laptop IDE setup:

osr -t targets.yaml -r rules.yaml mcp               # newline-delimited JSON-RPC on stdin/stdout
osr -t targets.yaml mcp --list-tools                 # what the IDE will see
osr mcp --url https://<platform>/mcp --api-key osr_... # bridge to a hosted platform

{"servers": {"opensmartroute": {"type": "stdio", "command": "osr",
                                "args": ["-t", "targets.yaml", "-r", "rules.yaml", "mcp"]}}}

In Python#

from opensmartroute.mcp_server import MCPServer, serve_stdio

server = MCPServer(router, execute=my_execute, marketplace=my_marketplace, prices=my_prices)
server.tools()                                   # tool specs (tools/list)
server.call("estimate", {"text": "…"})           # {"content": [...], "structuredContent": {...}}
reply = server.handle({"jsonrpc": "2.0", "id": 1, "method": "tools/call",
                       "params": {"name": "recommend", "arguments": {"task": "…", "priority": "cost"}}})
serve_stdio(server)                              # block on stdin/stdout

execute(request) -> dict enables ask; marketplace(action, args) with action in search / get enables the marketplace tools; prices is the quote hook above. RemoteMCP(url, api_key) and bridge_stdio(url, api_key) are the client side used by osr mcp --url. Everything is standard library only; osr serve needs the server extra (FastAPI).

Things to ask your assistant#

"Estimate what this prompt will cost and which model to use."
"Recommend the cheapest model that can refactor this file."
"Explain how OpenSmartRoute would route this request."
"Search the marketplace for a SQL reporting skill and show me its manifest."
"Route and answer this with the best model, then record that the answer was good."