AI gateway and observability
OpenSmartRoute vs Portkey
Portkey (like Helicone and Kong AI Gateway) sees every call: logs, guardrails, caching and retries. OpenSmartRoute chooses the route across kinds with a cost/quality objective and learns from outcomes; run it behind the gateway you already have.
“They see the traffic; we choose the route.”
Choose Portkey when
- Observability UX is the purchase
- Enterprise gateway features are needed today
Choose OpenSmartRoute when
- Routing across models, agents, tools and humans
- An explicit cost/quality objective per request
- Learned routing that shifts after outcomes
- Decision explainability for audit
Capability by capability
Our reading of public documentation; corrections welcome via GitHub issues.
| Capability | OpenSmartRoute | Portkey |
|---|---|---|
| Routes to LLMs | Yes | Yes |
| Routes to agents, skills, tools, workflows and humans | Yes | No |
| Hard policy before scoring (region, data boundary, PII, tenant, cost)Guardrails on the payload; routing rules are static configs | Yes | Partial |
| Per-request trace with ranked candidates and rejections | Yes | No |
| Learns from your outcomes (per target, per tenant) | Yes | No |
| Multi-step plans (persona -> skill -> model) | Yes | No |
| Self-hosted, air-gapped, zero runtime dependencies | Yes | Partial |
| Prompt-injection and steering-gadget defences on the routerGuardrail plugins on content, not on the routing decision | Yes | Partial |
| Savings ledger and Routing Audit | Yes | No |
| OpenAI-compatible proxy | Yes | Yes |
| Provider marketplace with credits | No | No |
| Request/response observability UI for every callDecision traces and usage; pair with your APM for payload-level tracing | Partial | Yes |
Decision API behind the gateway
Point the gateway's routing config at a single upstream: the OpenSmartRoute proxy with model osr/auto. Observability stays where it is.
client = OpenAI(base_url=f"{GATEWAY}/v1")
client.chat.completions.create(model="osr/auto:eu", messages=messages)Prove it on your traffic
Run the Routing Audit on a week of anonymised logs: savings versus your current mix, policy violations your routing allowed, and where agents or humans would have been the better answer.
Other comparisons: OpenRouter, LiteLLM, Building it yourself.