The hosted platform provides a managed environment accessible via the /platform/dashboard interface. Users can view their own endpoints and keys at /platform/dashboard/providers to manage connections to Azure, Bedrock, Gemini, or Vertex AI. The hosted version simplifies the operational burden, allowing teams to focus on routing strategies rather than infrastructure maintenance. Both modes support the same core routing features, including trace generation, quota management, and budget enforcement, differing primarily in where the execution and billing data reside.

Cost Management and Spend Visibility

A key feature of OpenSmartRoute is the ability to track and optimize AI spending across multiple platforms. The browser extension provides an "AI spend page" that displays observed calls and spend from the router's quote for the same requests. This view highlights the gap between actual costs and the router's optimized quotes, identifying models with the widest price differences. The page breaks down costs by model, site, and task, offering per-call history via the GET /api/v1/usage/observed endpoint. Rows are retained for 90 days and deleted with the workspace, ensuring data freshness without long-term storage bloat.

Only calls that the catalogue can price count towards the gap calculation, ensuring that both sides describe the same requests. This transparency helps organizations avoid vendor lock-in by comparing costs across different model providers. The platform guide on growth and SEO mentions that the vocabulary used for search keywords is added to every public page, ensuring that ads and pages agree on the target audience. This alignment supports better cost optimization by ensuring that the routing decisions match the business goals defined in the go-to-market plan.

Configuration Examples

To connect a custom endpoint, users can define a destination in the content API. The following example demonstrates how to configure a destination for a self-hosted CMS that runs on a private network.

{
  "kind": "cms",
  "name": "internal-blog",
  "url": "https://internal-blog.example.com",
  "auto_publish": true,
  "schedule": "immediate",
  "post_status": "publish",
  "credentials_encrypted": true
}

For model routing, the POST /api/v1/route endpoint accepts an execute: true flag to trigger the chosen model. The router then forwards the request to the configured endpoint, such as bedrock-runtime.<region>.amazonaws.com/openai/v1 for Amazon Bedrock or generativelanguage.googleapis.com/v1beta/openai for Google Gemini.

Comparison of Routing Strategies

The following table compares the primary model providers supported by OpenSmartRoute, highlighting their wire formats and verification status.

Frequently Asked Questions

How does OpenSmartRoute handle credentials for private endpoints?

Credentials are stored encrypted using AES-GCM under the deployment secret, bound to the workspace. They are never returned in API responses, ensuring that private model endpoints remain secure even when accessed through the router.

Can I mix different model providers in a single workspace?

Yes, every workspace connects its own endpoints and keys. Users can map catalogue targets to models on different clouds, such as Azure for inference and Bedrock for training, all within the same routing configuration.

What is the retention period for usage data in the spend report?

Rows in the AI spend page are kept for 90 days and are deleted with the workspace. This ensures that historical data remains accessible for analysis without consuming excessive storage resources.

Next Steps

To begin routing your AI workloads, start by reviewing the model catalogue to see which providers are available for your use case. For detailed implementation steps, visit the models page to explore the supported endpoints and pricing tiers.

biz it: A Guide to Open Source AI Routing

Learn how OpenSmartRoute helps engineers deploy open-source LLMs. Discover self-hosted options, model rankings, and SDK usage for your projects.