Loading
API gateway with AI plugins
Kong adds AI plugins to the API gateway many platform teams already run: an OpenAI-compatible proxy across providers, token-based rate limiting, prompt guards and load balancing between models. OpenSmartRoute is the routing brain those plugins call: a cost/quality objective per request, hard policy before scoring, a trace for every decision and learning from your outcomes.
“Kong guards the door; OpenSmartRoute picks the room.”
Our reading of public documentation; corrections welcome through the support page.
| Capability | OpenSmartRoute | Kong AI Gateway |
|---|---|---|
| Routes to LLMs | Yes | Yes |
| Routes to agents, skills, tools, workflows and humans | Yes | No |
| Hard policy before scoring (region, data boundary, PII, tenant, cost)Static routing rules per plugin config; no data-boundary or PII gating of the decision | Yes | Partial |
| Per-request trace with ranked candidates and rejections | Yes | No |
| Learns from your outcomes (per target, per tenant) | Yes | No |
| Multi-step plans (persona -> skill -> model) | Yes | No |
| Self-hosted, air-gapped, zero runtime dependenciesSelf-hosted with the gateway's own database and footprint | Yes | Partial |
| Prompt-injection and steering-gadget defences on the routerPattern-based prompt guard; no learned gadget detector on the decision | Yes | Partial |
| Output guard on responses (PII, secrets, injection: flag, redact or block)Prompt guard and PII sanitiser plugins on the request; response screening is limited | Yes | Partial |
| Savings ledger, bill analysis and routing auditEvery request records baseline and routed cost; upload last month's provider export to see what routing would have saved before you switch | Yes | No |
| OpenAI-compatible proxy (chat, embeddings, Responses, moderations, audio, images)Chat and embeddings across providers through the ai-proxy plugins | Yes | Partial |
| Anthropic Messages API (/v1/messages)Anthropic as an upstream provider behind the OpenAI-shaped route | Yes | Partial |
| Virtual keys: per-key budgets, rate limits, allowed targets, scopes and expiryConsumers and token-based rate limits; no per-key budget or target allow-list | Yes | Partial |
| Per-target rate limits, budgets and circuit breakersLoad balancing and health checks per upstream; no spend budget per model | Yes | Partial |
| Python and TypeScript SDKsAny OpenAI client pointed at the gateway route | Yes | Partial |
| MCP server for IDEs and agentsMCP traffic can be proxied; the gateway is not an MCP server for routing | Yes | Partial |
| Browser extension and desktop app for the people on the teamPrice, personal-data and writing checks on the device before a prompt is sent, on ChatGPT, Claude, Gemini and in any desktop app | Yes | No |
| Provider marketplace with creditsModels by the token: every catalogue model on offer per provider with the price you pay, one key, a prepaid credits wallet and provider preferences (order, only, ignore, max_price) | Yes | No |
| Request/response observability UI for every callGateway analytics and logs; no per-request decision trace | Yes | Partial |
Declare one upstream service - the OpenSmartRoute proxy - behind the ai-proxy route. Kong keeps authentication, rate limiting and plugins; the router keeps the decision, the policy and the trace.
# kong.yaml (declarative): one upstream, the router decides the model
services:
- name: osr
url: https://osr.internal/v1
routes:
- name: chat
paths: [/v1/chat/completions]
# clients send model: "osr/auto:cheap" (or a target id) and the router picks the providerRun the Routing Audit on a week of anonymised logs: savings versus your current mix, policy violations your routing allowed, and where agents or humans would have been the better answer.
Other comparisons: OpenRouter, LiteLLM, Portkey, Cloudflare AI Gateway, Helicone, Not Diamond, Martian, Unify, Building it yourself.