Loading
Edge AI gateway
Cloudflare AI Gateway sits at the edge in front of your providers: logs, caching, rate limits, retries and fallbacks with almost no setup. OpenSmartRoute is the decision behind it - which capability, under which policy, at which price, explained and learned - and it runs wherever your data may go, including nowhere near a public edge.
“Cloudflare carries the request; OpenSmartRoute decides where it should go.”
Our reading of public documentation; corrections welcome through the support page.
| Capability | OpenSmartRoute | Cloudflare AI Gateway |
|---|---|---|
| Routes to LLMs | Yes | Yes |
| Routes to agents, skills, tools, workflows and humans | Yes | No |
| Hard policy before scoring (region, data boundary, PII, tenant, cost) | Yes | No |
| Per-request trace with ranked candidates and rejections | Yes | No |
| Learns from your outcomes (per target, per tenant) | Yes | No |
| Multi-step plans (persona -> skill -> model) | Yes | No |
| Self-hosted, air-gapped, zero runtime dependencies | Yes | No |
| Prompt-injection and steering-gadget defences on the router | Yes | No |
| Output guard on responses (PII, secrets, injection: flag, redact or block)Content guardrails on prompts and responses through a moderation model; no secret or PII redaction of the answer | Yes | Partial |
| Savings ledger, bill analysis and routing auditEvery request records baseline and routed cost; upload last month's provider export to see what routing would have saved before you switch | Yes | No |
| OpenAI-compatible proxy (chat, embeddings, Responses, moderations, audio, images)A unified endpoint for chat across providers; the rest follows each provider's own API | Yes | Partial |
| Anthropic Messages API (/v1/messages) | Yes | Yes |
| Virtual keys: per-key budgets, rate limits, allowed targets, scopes and expiryGateway-level rate limits and stored provider keys; no per-key budget, target allow-list or scopes | Yes | Partial |
| Per-target rate limits, budgets and circuit breakersRetries and fallback order per gateway; no budget or breaker per target | Yes | Partial |
| Python and TypeScript SDKsThe provider's own SDK pointed at the gateway URL | Yes | Partial |
| MCP server for IDEs and agents | Yes | No |
| Browser extension and desktop app for the people on the teamPrice, personal-data and writing checks on the device before a prompt is sent, on ChatGPT, Claude, Gemini and in any desktop app | Yes | No |
| Provider marketplace with creditsModels by the token: every catalogue model on offer per provider with the price you pay, one key, a prepaid credits wallet and provider preferences (order, only, ignore, max_price) | Yes | No |
| Request/response observability UI for every callDecision trace, tokens, cost and outcome per request; prompt and answer stored when the workspace turns on request logging (redacted or full) | Yes | Yes |
Keep the gateway for logs and caching and make its upstream the OpenSmartRoute proxy with model osr/auto: the edge sees every call, the router chooses the target and enforces your policy before any provider is reached.
# the gateway's upstream is the router, not a provider
client = OpenAI(base_url=f"{GATEWAY_URL}/openai", api_key=OSR_API_KEY)
client.chat.completions.create(
model="osr/auto:private", # never a public endpoint for this call
messages=messages,
extra_body={"osr": {"constraints": {"region": "eu"}}},
)Run the Routing Audit on a week of anonymised logs: savings versus your current mix, policy violations your routing allowed, and where agents or humans would have been the better answer.
Other comparisons: OpenRouter, LiteLLM, Portkey, Kong AI Gateway, Helicone, Not Diamond, Martian, Unify, Building it yourself.