Loading
90 days of the enterprise, pro and team plans free, no card needed. See the plans
Open-source LLM gateway
Bifrost is Maxim AI's Go gateway: one OpenAI-compatible endpoint over the major providers with virtual keys, budgets, semantic caching, load balancing, an MCP gateway and very low per-request overhead, paired with Maxim's evaluation and observability suite. OpenSmartRoute is the decision in front of execution: which model, agent, tool or person under your policy, with a ranked trace, learning from your outcomes, a response cache and a savings ledger, self-hosted with zero dependencies.
“Bifrost forwards the model call your code already addressed; OpenSmartRoute decides which capability should get it.”
Our reading of public documentation; corrections welcome through the support page.
| Capability | OpenSmartRoute | Bifrost (Maxim AI) |
|---|---|---|
| Routes to LLMs | Yes | Yes |
| Routes to agents, skills, tools, workflows and humans | Yes | No |
| Hard policy before scoring (region, data boundary, PII, tenant, cost)Budgets, allow-lists and rate limits per virtual key and team; no data-boundary or PII gating per request | Yes | Partial |
| Per-request trace with ranked candidates and rejections | Yes | No |
| Learns from your outcomes (per target, per tenant) | Yes | No |
| Multi-step plans (persona -> skill -> model) | Yes | No |
| Self-hosted, air-gapped, zero runtime dependenciesA single Go binary or container, self-hosted; semantic caching and the console need their own stores | Yes | Partial |
| Prompt-injection and steering-gadget defences on the router | Yes | No |
| Output guard on responses (PII, secrets, injection: flag, redact or block)Governance plugins and guardrail integrations; no built-in secret or PII redaction of the answer | Yes | Partial |
| Savings ledger, bill analysis and routing auditEvery request records baseline and routed cost; upload last month's provider export to see what routing would have saved before you switch | Yes | No |
| OpenAI-compatible proxy (chat, embeddings, Responses, moderations, audio, images) | Yes | Yes |
| Anthropic Messages API (/v1/messages) | Yes | Yes |
| Virtual keys: per-key budgets, rate limits, allowed targets, scopes and expiry | Yes | Yes |
| Per-target rate limits, budgets and circuit breakersLoad balancing, retries and fallbacks per provider; budgets live on keys and teams rather than on a target | Yes | Partial |
| Python and TypeScript SDKsWorks through the OpenAI, Anthropic and Google SDKs pointed at the gateway; a Go SDK of its own | Yes | Partial |
| MCP server for IDEs and agentsAn MCP gateway that exposes your MCP servers' tools to the models it proxies; not an MCP server an IDE connects to for routing | Yes | Partial |
| Browser extension and desktop app for the people on the teamPrice, personal-data and writing checks on the device before a prompt is sent, on ChatGPT, Claude, Gemini and in any desktop app | Yes | No |
| Provider marketplace with creditsModels by the token: every catalogue model on offer per provider with the price you pay, one key, a prepaid credits wallet and provider preferences (order, only, ignore, max_price) | Yes | No |
| Request/response observability UI for every callDecision trace, tokens, cost and outcome per request; prompt and answer stored when the workspace turns on request logging (redacted or full) | Yes | Yes |
Keep Bifrost as the execution layer for the providers you already wired, and make its upstream for `model: auto` the OpenSmartRoute proxy - or register Bifrost as one target so the router sends it the requests that may go to a public model and keeps the rest on-prem, in an agent or with a person. The OpenAI, Anthropic and Gemini SDKs keep working unchanged against either.
# the gateway's upstream for routed traffic is the router, not a provider
client = OpenAI(base_url=f"{OSR}/v1", api_key=OSR_API_KEY)
client.chat.completions.create(model="auto", messages=messages,
extra_headers={"X-OSR-Cache-TTL": "600"}) # the hosted response cache answers repeats for free
# or: Bifrost as one target in the catalogue
- id: bifrost-public
kind: llm
description: Public models through the Bifrost gateway
capabilities: { domains: [general, coding], max_complexity: 1.0 }
constraints: { data_boundary: public, pii_allowed: false }
cost: { usd_per_1k_tokens: 0.006 }Run the Routing Audit on a week of anonymised logs: savings versus your current mix, policy violations your routing allowed, and where agents or humans would have been the better answer.
Other comparisons: OpenRouter, LiteLLM, Portkey, Cloudflare AI Gateway, Kong AI Gateway, Helicone, Not Diamond, Martian, Unify, Requesty, Building it yourself.
Photo: Mario Gogh
