Cost estimates, recommended models and the MCP server

Three ways to ask the router "what will this cost and what should I use?" before spending tokens: a quote API and page, a recommended-models digest, and a Model Context Protocol server that puts all of it inside VS Code, Cursor, Claude, Windsurf or any agent framework. The same pieces exist in the SDK (opensmartroute.estimate, opensmartroute.mcp_server, osr estimate, osr mcp) and on the hosted platform (POST /api/v1/estimate, GET /api/v1/models/recommended, /mcp, the /estimate page and Dashboard -> Integrations).

1. Quotes

A quote routes the request exactly as route() would - signals, policy, strategies, learned quality - and then prices every ranked candidate instead of executing one:

• Tokens. Input tokens come from the text (word / digit / punctuation / CJK / code heuristic, estimate_tokens, calibrated against billed gpt-4.1 prompt tokens on twelve text types - mean ratio 1.02, every type within about 20 %) plus the history (estimate_messages_tokens, 4 tokens of framing per message). Output tokens are the caller's output_tokens, else the signals' predicted answer length, else DEFAULT_OUTPUT_TOKENS (256); output_tokens_source says which.
• Prices. usd_per_1k_input / usd_per_1k_output on the target when declared, else the blended usd_per_1k_tokens; a fixed usd_per_call is added for agents, tools and humans. On the platform a target that resolves to a reference model is priced from the live catalogue (priced_from: "catalogue"), so quotes follow vendor list prices.
• Picks. recommended is the router's decision; best_quality the highest quality estimate; cheapest and fastest are chosen among candidates within quality_tolerance (0.1) of the best that fit their context window, so a free-but-useless target never wins. savings_usd is the gap between the dearest candidate and the recommended one.
• Nothing is executed and the text is never stored.

from opensmartroute.estimate import estimate

q = estimate(router, "Summarise this contract in five bullets", output_tokens=300)
print(q.input_tokens, q.output_tokens, q.output_tokens_source)  # 9 300 caller
print(q.recommended.name, q.recommended.cost_usd)  # Small model 0.000062
for t in q.targets:  # every candidate, ranked
    print(t.rank, t.target_id, f"${t.cost_usd:.6f}", t.quality_estimate, t.latency_ms, t.fits_context)
print(q.cheapest.target_id, q.best_quality.target_id, q.fastest.target_id, q.savings_usd)
q.to_dict()  # the JSON the API returns

estimate(router, request, *, output_tokens=None, kinds=None, prices=None, quality_tolerance=0.1, decision=None) accepts a RouteRequest or a string, a kinds filter and a prices(target) -> (usd_per_1k_in, usd_per_1k_out) | None hook for live price feeds; pass decision= to quote what an earlier route() already ranked.

osr -t targets.yaml estimate "Translate this contract into German" --monthly 100000
osr -t targets.yaml estimate "..." --output-tokens 500 --cost-weight 1.0 --json

POST /api/v1/estimate

Same body as POST /api/v1/route (text, history, objective, constraints, kinds) plus output_tokens and monthly_requests. Without a key the endpoint is rate limited per client (20 per minute, 8 000 characters) and ignores tenant rules; with a key it is metered as estimate, applies your plan and tenants and returns X-Request-Id.

curl -s https://<platform>/api/v1/estimate -H "Authorization: Bearer $OSR_KEY" \
  -H "Content-Type: application/json" \
  -d '{"text": "Summarise this contract...", "monthly_requests": 50000}'

{
  "request_id": "…", "input_tokens": 9, "output_tokens": 256, "output_tokens_source": "signals",
  "total_tokens": 265, "confidence": 0.71, "savings_usd": 0.00209,
  "recommended": {"target_id": "llm-small", "name": "Small model", "vendor": "openai", "model": "openai/gpt-4.1-nano",
                  "input_cost_usd": 0.0000009, "output_cost_usd": 0.0001, "cost_usd": 0.000103,
                  "latency_ms": 400, "quality_estimate": 0.62, "fits_context": true, "priced_from": "catalogue",
                  "rationale": {"semantic": "…", "cost": "…"}},
  "cheapest": {"…": "…"}, "best_quality": {"…": "…"}, "fastest": {"…": "…"},
  "targets": [{"…": "…"}],
  "signals": {"complexity": 0.31, "domains": ["legal"], "task_type": "summarise", "contains_pii": false, "…": "…"},
  "policy_rejections": {},
  "monthly": {"requests": 50000, "recommended_usd": 5.15, "cheapest_usd": 5.15, "best_quality_usd": 109.6,
              "dearest_usd": 109.6, "savings_usd": 104.45, "tokens": 13250000},
  "anonymous": false
}

The /estimate page is the same call with a form: paste a prompt, choose what to optimise for, set the expected answer length and monthly volume, and read the four picks, the signals the router saw and the full candidate table with input / output cost bars.

2. Recommended models

GET /api/v1/models/recommended (no key) quotes a representative prompt per use case - chat, code, reasoning, summarise, extraction, pii - against the live catalogue and returns, for each, the recommended, cheapest, best_quality and fastest pick with cost per request, quality estimate, latency, 30-day traffic, the reference model's intelligence index and the reasons (why). A leaderboard counts how often each target was the router's pick. The digest is recomputed at most once a minute and follows the deployment's targets, prices and learned quality rather than a hand-maintained list. It renders on /models ("Recommended per use case") and on /estimate.

3. MCP server

An MCP client - VS Code, Cursor, Claude, Windsurf or an agent framework - calls the router's tools over JSON-RPC and gets a decision (or a quote, or the executed answer) back:

sequenceDiagram
    participant C as MCP client (Cursor / Claude / agent)
    participant S as MCPServer(router)
    participant R as Router
    C->>S: tools/list
    S-->>C: route, estimate, recommend, explain, list_targets, feedback, ask
    C->>S: tools/call route "translate this contract"
    S->>R: route(request)
    R-->>S: decision + trace
    S-->>C: target, confidence, alternatives, reasons
    Note over C,R: ask also runs the chosen target and returns the answer, tokens and cost

What it exposes

Resources: osr://targets (the catalogue as JSON) and osr://stats (feedback statistics). The server speaks MCP 2025-06-18 (also accepts 2025-03-26 and 2024-11-05) over JSON-RPC 2.0: initialize, ping, tools/list, tools/call, resources/list, resources/read, prompts/list, logging/setLevel (validated, applied to the package logger); notifications get no reply; unknown methods return -32601, unknown tools -32602, and every tool failure - invalid arguments, no admissible target, a provider that is down, a marketplace listing that does not exist - comes back as an isError result the model can read and act on, never as a protocol error.

ask and marketplace_* appear only when the server can honour them: osr serve and the platform provide both; osr mcp (local, stdio) offers ask when something in the catalogue can run - an LLM with a chat handler (OSR_LLM_BASE_URL) or a bound manifest endpoint such as an A2A agent or MCP server synced from the marketplace - and the marketplace tools through MarketplaceClient.mcp_hook (--no-marketplace leaves them out); a bare MCPServer(router) exposes the read-only tools and feedback.

Hosted platform

GET /mcp describes the server (tools, resources, protocol version, connection snippets) without a key. POST /mcp takes one JSON-RPC message or a batch with the same Authorization: Bearer <api key> (or X-API-Key) header as the REST API; notifications return 202. Every routed call carries the workspace tenant and is metered as mcp against the plan's quotas; feedback needs the feedback feature and ask the plan feature (Pro and above). Quota, plan and routing errors surface as tool errors such as 429: rate limit exceeded. Dashboard -> Integrations renders ready-to-paste configuration for each client with the workspace's key.

HTTP transport (the client talks to https://<platform>/mcp directly):

// VS Code: .vscode/mcp.json
{"servers": {"opensmartroute": {"type": "http", "url": "https://<platform>/mcp",
                                "headers": {"Authorization": "Bearer osr_..."}}}}
// Cursor: .cursor/mcp.json
{"mcpServers": {"opensmartroute": {"url": "https://<platform>/mcp",
                                   "headers": {"Authorization": "Bearer osr_..."}}}}
// Windsurf: ~/.codeium/windsurf/mcp_config.json
{"mcpServers": {"opensmartroute": {"serverUrl": "https://<platform>/mcp",
                                   "headers": {"Authorization": "Bearer osr_..."}}}}

claude mcp add --transport http opensmartroute https://<platform>/mcp --header "Authorization: Bearer osr_..."

Stdio bridge for clients that only launch local processes (Claude Desktop and older clients): the CLI forwards each line to the platform with your key.

{"mcpServers": {"opensmartroute": {"command": "osr",
                                   "args": ["mcp", "--url", "https://<platform>/mcp", "--api-key", "osr_..."]}}}

Streamable HTTP: sessions, streams and notifications

The endpoint implements the Streamable HTTP transport of the MCP specification, not only request / reply:

• Sessions. The reply to initialize carries an Mcp-Session-Id; send it back on later calls. The id is signed with the deployment secret and bound to the workspace, so every replica accepts it without shared state; an id of another workspace, an expired one (24 hours) or one that was ended answers 404 - re-initialize. DELETE /mcp with the header ends the session (204).
• Streamed replies. Send Accept: text/event-stream on POST /mcp and the answer is a server-sent-events stream: notifications/progress for a _meta.progressToken (ask reports routing, running on <target>, recording), notifications/message log lines at the level set with logging/setLevel, then the JSON-RPC result as the last message event. Without that header the reply is plain JSON as before.
• Notification stream. GET /mcp with Accept: text/event-stream and your credential opens the server's own stream: notifications/tools/list_changed and notifications/resources/list_changed when the catalogue, your installed listings or the tools change, a : ping comment every 15 seconds, closed after max_s seconds (default and ceiling OSR_PLATFORM_MCP_STREAM_MAX_S, one hour). Every frame has an id; reconnect with Last-Event-ID and the stream starts with a list_changed so a change during the gap is not missed. initialize advertises listChanged: true for tools and resources because of it.
• Guards. A browser Origin must be one of the deployment's (the web app, the API's own origin, OSR_PLATFORM_CORS_ORIGINS, loopback) - the DNS-rebinding protection the specification asks for; non-browser clients send none. An MCP-Protocol-Version header must name a revision the server speaks (2025-06-18, 2025-03-26, 2024-11-05), else 400. Bodies are capped at 256 KB.

Signing in with your identity provider (OAuth 2.1, on-behalf-of)

Besides workspace API keys, /mcp accepts a bearer token your organization's identity provider issued for this resource - the OAuth 2.1 resource-server flow MCP clients implement (VS Code, Claude, Cursor ...): the client is refused with 401 and WWW-Authenticate: Bearer resource_metadata="https://<platform>/.well-known/oauth-protected-resource/mcp", reads that document (RFC 9728: resource, authorization_servers, scopes_supported), signs the person in at the authorization server and comes back with a JWT. The platform verifies the signature against the issuer's JWKS (RS256 / PS256 / ES256 families, keys refreshed on rotation), the issuer, the audience (https://<platform>/mcp, or the app's client id / api://<client id>), expiry and not-before, then maps the token to the platform user: a linked single sign-on identity whose subject is the token's sub or oid first, else the user whose address the token names (preferred_username, email, upn; OSR_PLATFORM_MCP_OAUTH_EMAIL_MATCH=false turns that off). Someone the platform does not know gets 403 with a pointer to sign in on the site once. The call then runs in the person's workspace, metered and audited like a keyed one, with their role. The endpoint is reachable both on the API host and on the web origin (https://<site>/mcp, proxied with the session, protocol-version and stream headers intact); the metadata, the challenge and the accepted audience name whichever URL the client used, and Dashboard -> Integrations shows the key-less configuration when the deployment has an identity provider.

Configuration defaults from the microsoft single sign-on provider (OSR_PLATFORM_SSO_MICROSOFT_*): its tenant names the issuer (https://login.microsoftonline.com/<tenant>/v2.0; a multi-tenant app keeps Entra's {tenantid} placeholder and matches any tenant), its client id the audience, its credentials the on-behalf-of client. Any OIDC issuer works explicitly:

On-behalf-of. A model behind the organization's own identity - Azure OpenAI or Azure AI Foundry with Entra authentication, a corporate gateway - can be a provider whose credential is obo:<scope> instead of a key (obo:https://cognitiveservices.azure.com/.default), in the deployment file, the operator console or a workspace's own providers. When a call that arrived with an identity provider's token reaches such a provider, the platform exchanges that token for one with the scope - Entra's jwt-bearer grant with requested_token_use=on_behalf_of for a Microsoft issuer, the RFC 8693 token exchange for any other - and calls the model with Authorization: Bearer <downstream token>: the person's own permissions apply, no shared key exists, and the exchange is cached per user and scope until shortly before it expires and recorded in the audit log (mcp.obo_exchange, no token material). A request that came with an API key has nobody to act for: an obo: provider is unavailable to it and the fallback loop moves on to the next executable target. The health probe of such a provider checks reachability without a credential.

// VS Code: no key in the file - the editor signs you in at the authorization server
{"servers": {"opensmartroute": {"type": "http", "url": "https://<platform>/mcp"}}}

Self-hosted and local

osr serve mounts the same server at POST /mcp (and GET /mcp for the description) next to /route and /v1, with ask backed by the configured providers - and the same Streamable HTTP transport as the hosted platform: a signed Mcp-Session-Id on initialize (bound to the access token it was issued to; DELETE /mcp ends it), event-stream replies with notifications/progress and log messages when the client accepts text/event-stream, a GET /mcp notification stream that announces tools/list_changed / resources/list_changed when the catalogue moves (a marketplace sync, a /targets write; max_s bounds it, ceiling OSR_SERVER_MCP_STREAM_MAX_S), the protocol-version check and the browser-origin guard (loopback always passes; OSR_SERVER_MCP_ORIGINS names other origins). Without a service, the CLI runs the server over stdio straight from a catalogue - useful for a laptop IDE setup; set OSR_LLM_BASE_URL (and OSR_LLM_API_KEY) and ask runs the routed model for real, exactly as osr serve would:

osr -t targets.yaml -r rules.yaml mcp               # newline-delimited JSON-RPC on stdin/stdout
osr -t targets.yaml mcp --list-tools                 # what the IDE will see
osr -t targets.yaml mcp --marketplace-url https://api.example.test   # marketplace_* against another marketplace
osr -t targets.yaml mcp --no-marketplace             # local tools only
osr mcp --url https://<platform>/mcp --api-key osr_... # bridge to a hosted platform
osr mcp --remote --list-tools                        # tools of the platform you signed in to (osr login)

{"servers": {"opensmartroute": {"type": "stdio", "command": "osr",
                                "args": ["-t", "targets.yaml", "-r", "rules.yaml", "mcp"]}}}

When the IDE's environment does not have osr on PATH (a project virtualenv, uv run), use the interpreter instead: "command": "python", "args": ["-m", "opensmartroute", "-t", "targets.yaml", "mcp"]. Both transports are exercised in the test suite with the official mcp Python client: initialize, tools/list, tools/call, resources/list over stdio against python -m opensmartroute ... mcp, and over Streamable HTTP against osr serve's POST /mcp with a bearer token.

In Python

from opensmartroute.mcp_server import MCPServer, serve_stdio

server = MCPServer(router, execute=my_execute, marketplace=my_marketplace, prices=my_prices)
server.tools()  # tool specs (tools/list)
server.call("estimate", {"text": "…"})  # {"content": [...], "structuredContent": {...}}
reply = server.handle(
    {
        "jsonrpc": "2.0",
        "id": 1,
        "method": "tools/call",
        "params": {"name": "recommend", "arguments": {"task": "…", "priority": "cost"}},
    }
)
serve_stdio(server)  # block on stdin/stdout

execute(request) -> dict enables ask; marketplace(action, args) with action in search / get / target (the last returns a RouteTarget for a slug and powers the route tool's marketplace candidates) enables the marketplace tools; prices is the quote hook above. RemoteMCP(url, api_key) and bridge_stdio(url, api_key) are the client side used by osr mcp --url. Everything is standard library only; osr serve needs the server extra (FastAPI).

Importing other MCP servers as targets

The router is also an MCP client: a server's tools, resources and prompts become routable targets.

from opensmartroute.adapters import connect_mcp, connect_mcp_http

client, targets = connect_mcp(["npx", "-y", "@modelcontextprotocol/server-filesystem", "."], resources=True)
http, remote = connect_mcp_http("https://tools.example.com/mcp", api_key="...", resources=True, prompts=True)
for t in [*targets, *remote]:
    registry.upsert(t)

connect_mcp spawns a stdio server; connect_mcp_http (adapters.HttpMCPClient) speaks the Streamable HTTP transport (JSON or SSE replies, Mcp-Session-Id) to remote servers over https. Tools become tool targets whose handler runs tools/call; with resources=True every resource becomes a read-only tool target that reads it (resources_from_mcp), with prompts=True every prompt becomes an instructions-only skill slot whose handler renders the template (prompts_from_mcp). Tool descriptions are still scored for injection risk at import time and signed manifests still verify as before.

Things to ask your assistant

• "Estimate what this prompt will cost and which model to use."
• "Recommend the cheapest model that can refactor this file."
• "Explain how OpenSmartRoute would route this request."
• "Search the marketplace for a SQL reporting skill and show me its manifest."
• "Route and answer this with the best model, then record that the answer was good."