Loading
Glossary
An AI gateway is a single API endpoint placed in front of several AI providers that handles keys, rate limits, retries, logging and - in a routing gateway - the decision about which model answers each request.
Teams that call several AI vendors end up with several SDKs, several key stores, several logging formats and several places to retry. An AI gateway collapses that into one endpoint: applications call it with one credential, and it holds the provider keys, enforces rate limits, retries on failure, records usage and presents one response shape. Most gateways speak the OpenAI chat-completions dialect because most clients already do.
A plain gateway forwards to the model the caller named. A routing gateway adds the decision: when the caller says `model: "auto"` the gateway reads the request, scores the candidates and picks one; when the caller names a model the gateway still applies policy, prices the call and falls back to the next candidate if the provider fails. The difference matters when models, prices or policies change - a plain gateway needs every application to change with them; a routing gateway changes a catalogue entry.
Governance lives naturally in a gateway because every request passes through it. Data-boundary and region rules, per-key and per-tenant budgets, allowed-model lists and payload-logging settings can be enforced before any provider is called, and the same place can keep the audit trail. That is why enterprise teams tend to adopt a gateway before they adopt routing, and why the two are usually the same product.
Questions people ask
OpenSmartRoute is open source and the free plan keeps the full trace of every decision. Type a request in the playground and read the ranked candidates.
Free plan, no card. Fifteen thousand decisions a month with the full trace.