Install the platform

The hosted platform at opensmartroute.ai needs no installation - sign up, create a key, follow the platform quickstart. This page is for running the whole platform yourself: the routing API with its OpenAI-compatible endpoint, the web app (public site, dashboard, operator console), the routing SLM service and the state behind them. The same images and the same OSR_PLATFORM_* settings run everywhere; only the way they are started differs.

Just the Python library or the osr command line: README - Install.

What you get

Every install path deploys the same product:

• osr-platform-api - the FastAPI platform: POST /api/v1/route, the OpenAI-compatible /v1 proxy, MCP server, accounts, keys, workspaces, tenants, governance, marketplace, billing, notifications, support tickets and the operator API. In cluster mode it also runs once per domain (docs, rankings, marketplace, providers, onboarding, accounts, billing, admin, mcp, openai, routing) behind the API gateway, plus the routing SLM service.
• osr-platform-web - the Next.js web app: the public site and documentation, /platform (sign-in, dashboard, playground, publish flow) and /platform/admin (the operator console).
• State: PostgreSQL (always), Redis and Kafka (cluster mode), a data volume for the audit chain and the routing SLM.
• Model providers: Azure OpenAI deployments on Azure; anything OpenAI-compatible everywhere (connected in the console under Providers, or by workspaces with their own keys under Model providers); optionally a self-hosted Ollama server for the private / specialist targets.

Release images: crosrua2c6onyidguw.azurecr.io/osr-platform-api:1.2.0 and crosrua2c6onyidguw.azurecr.io/osr-platform-web:1.2.0 (anonymous pull; :latest follows the newest release, :1.2 the newest patch). The web image is built without a baked hostname: it reads OSR_PUBLIC_WEB_URL at run time and the browser uses the origin it is talking to, so one image serves any domain.

Azure Marketplace

The offer OpenSmartRoute platform is an Azure Application (solution template): it deploys into a resource group of your subscription, you own every resource, nothing is shared with the publisher and there is no software charge on the Azure bill - you pay the Azure resources it creates.

Deploy

1. In the Azure portal open Marketplace, search for OpenSmartRoute, choose the plan and Create.
2. Basics - subscription, a new or existing resource group, the region (Azure OpenAI and Container Apps must be offered there; the default gpt-4.1 family is), a deployment name, the first operator username and password (the console sign-in), the edition (Enterprise: tenants, audit, statistics, persisted learners; Community: the lightest footprint).
3. Size and state - Cluster mode (Redis + Kafka, the API split into domain services scaled on demand; off = one API replica with PostgreSQL only, right for a trial) and Managed state (Azure Database for PostgreSQL with point-in-time restore, Azure Cache for Redis, Event Hubs - recommended for production; off = state containers on Azure Files, cheapest).
4. Hostnames - leave empty to use the Container Apps hostnames, or enter your web and API hostnames (and let the template create the Azure DNS zone). TLS certificates are issued in a second run once DNS resolves (below).
5. E-mail - Azure Communication Services or SMTP for verification, reset, invitation, alert and support-ticket mail. Without a transport every message waits in the console's Mail page; nothing is lost.
6. Models and learning - the self-hosted model server (Ollama, CPU or serverless GPU), the autopilot (retrain the routing model from your own feedback) and the marketplace catalogue refresh.
7. Sign-in and access - password sign-in, self-serve signups per IP (0 = operators create every workspace), SSO providers, a static admin token for scripts, public /metrics.
8. Images and billing - the release to deploy and the registry to pull from (yours, after mirroring), Stripe for self-serve paid plans.
9. Review + create. A cluster-mode deployment takes about ten minutes; the first Ollama start pulls about 20 GB of weights in the background.

The deployment's Outputs tab lists webUrl, platformUrl, adminLoginUrl, apiUrl, the hostnames and static IP for your DNS records, the asuid verification id and the Azure OpenAI endpoint.

First sign-in

1. Open adminLoginUrl (<webUrl>/platform/admin/login) and sign in with the operator username and password from Basics. The console shows the deployment's health, users, workspaces, plans, providers and the support desk.
2. Providers: the Azure OpenAI account created by the template is connected and the cloud tiers (llm-small, llm-mid, llm-frontier) already execute on it. Add OpenAI, Anthropic, Bedrock, Vertex, vLLM or any OpenAI-compatible endpoint here, or let workspaces bring their own keys.
3. Open webUrl, Sign up (or create the workspace from the console when self-serve signup is off), create an API key and route a request - platform quickstart with OSR_API_URL=<apiUrl>.
4. osr login --url <webUrl> signs the command line in to your deployment.

Custom domain (two runs)

1. First run with your hostnames entered and Issue TLS certificates off: the apps bind the hostnames without TLS.
2. Create the DNS records from the outputs: CNAME <web hostname> -> webHostname, CNAME <api hostname> -> apiHostname (an apex domain gets A -> staticIp), and TXT asuid.<hostname> -> domainVerificationId for each hostname. If the template created the zone, delegate the domain to dnsNameServers instead - the records are already there.
3. Run the deployment again (Redeploy on the deployment, same values) with Issue TLS certificates on. Azure issues free managed certificates and the hostnames answer over HTTPS.

Upgrade, backup, remove

• Upgrade: redeploy with a newer OpenSmartRoute release (image tag). The database schema migrates itself at start-up; state lives in PostgreSQL and on the data share, never in the containers.
• Backup: the osr-db-backup job writes a nightly pg_dump to the osr-backups share (14 days). Managed state adds point-in-time restore on the server. Restore: platform/README.md "Containers and Azure".
• Remove: delete the resource group. Nothing is left outside it.

Sizing and cost (Azure list prices, rough)

Model usage (Azure OpenAI tokens) is billed by Azure OpenAI at its own prices; the dashboard's Savings page shows what routing saved against always calling the frontier model.

Azure with azd

From a checkout, the same template with images built from source:

azd auth login
azd env new prod
azd env set OSR_PLATFORM_ADMIN_USERNAME admin
azd env set OSR_PLATFORM_ADMIN_PASSWORD '<at least 8 characters>'
azd up

Every parameter of the Marketplace wizard is an azd env set OSR_... variable (infra/main.parameters.json); platform/README.md documents them, the custom-domain flow (scripts/domains.ps1), SSO (scripts/sso.ps1) and the pipelines that keep the publisher's deployment current.

Your own servers (Docker Compose)

Runs on any host with Docker Engine 24+ (Linux, or Docker Desktop for a workstation): on-premises, air-gapped, or a VM in any cloud. Minimum 4 vCPU / 8 GiB / 40 GB disk for the platform; add what your models need if you run them on the same host.

1. Get the stack

git clone the project repository && cd OpenSmartRoute
cp platform/.env.example platform/.env

Or download only platform/docker-compose.yml and platform/.env.example - the stack runs from the published images.

2. Configure

Edit platform/.env: the passwords and the session secret (all of them - the defaults are placeholders), the public URLs of the web app and the API as your users will reach them, the release images, and the e-mail transport. Everything is documented in the file; the same variables are explained in platform/README.md.

3. Start

docker compose -f platform/docker-compose.yml --env-file platform/.env pull
docker compose -f platform/docker-compose.yml --env-file platform/.env up -d
curl -fsS http://localhost:8080/readyz          # every check ok once PostgreSQL is up and the schema is created

Web http://localhost:3000, operator console http://localhost:3000/platform/admin/login (the operator username and password from .env), API http://localhost:8080/api/v1/info. Without OSR_API_IMAGE / OSR_WEB_IMAGE the stack builds the images from the checkout (up -d --build).

4. Connect models

The stack has no cloud account. Either connect providers in the console under Providers (any OpenAI-compatible endpoint: OpenAI, Azure OpenAI, Anthropic through a gateway, vLLM, Ollama, ...) or start the on-prem model server next to it:

docker compose -f platform/docker-compose.yml -f deploy/compose/providers.ollama.yaml --env-file platform/.env up -d

deploy/compose/providers.ollama.yaml runs Ollama with the curated open weights (Qwen3, Gemma 3, Qwen2.5-Coder, Qwen3-VL, an embedding model) and maps the private / specialist targets to it; deploy/compose/ollama.gpu.yaml adds an NVIDIA GPU. OSR_PLATFORM_PROVIDERS points the API at a providers file for anything else.

5. Put it behind TLS

Terminate TLS in your reverse proxy (nginx, Caddy, Traefik, an ingress) in front of ports 3000 (web) and 8080 (API), set OSR_PLATFORM_WEB_URL / OSR_PLATFORM_BASE_URL to the public https:// origins, and remove the host port mappings you do not want exposed. Several API replicas share PostgreSQL / Redis / Kafka: docker compose up -d --scale api=3 behind the proxy, exactly what the Azure template does.

Operate

• Backups: docker compose -f platform/docker-compose.yml exec postgres pg_dump -U osr osr | gzip > osr-$(date +%F).sql.gz (and osr_telemetry); restore with psql into a fresh volume.
• Upgrade: set the new tag in .env, pull, up -d. The schema migrates at start-up.
• Air-gapped: on a connected machine docker pull both images (and postgres:16-alpine, redis:7-alpine, apache/kafka:3.9.0), docker save | gzip, move, docker load; or mirror them into your registry (az acr import --source crosrua2c6onyidguw.azurecr.io/osr-platform-api:1.2.0 ...) and set OSR_API_IMAGE / OSR_WEB_IMAGE (or the Marketplace wizard's Image registry).
• Health: GET /readyz on the API (database, Redis, Kafka, mail transport, routing probe), /platform/admin/health in the console, GET /metrics for Prometheus.

Publishing the Marketplace offer (for the publisher)

The package Partner Center takes is built by the Release pipeline (artifact marketplace) and, locally, by

python -X utf8 scripts/marketplace_package.py check                       # wizard <-> template consistency
python -X utf8 scripts/marketplace_package.py build --ttk <arm-ttk dir>   # mainTemplate.json + createUiDefinition.json -> dist/marketplace/*.zip

marketplace/azure/mainTemplate.bicep wraps infra/resources.bicep for a resource-group deployment with the release images; marketplace/azure/createUiDefinition.json is the wizard. The Infra pipeline builds the package on every change; release.py check keeps the image tag in step with the release. The offer is published from Partner Center: an Azure Application offer of type Solution template, one plan per supported topology if you want separate listings (a single plan with the wizard's choices is enough), the zip uploaded under Technical configuration, listing texts and screenshots from docs/BRAND.md and the sales kit, then Review and publish - Partner Center runs the same arm-ttk checks and a test deployment before the preview goes live. Requirements the publisher keeps true: the two image repositories allow anonymous pull (the publishing deployment sets OSR_PUBLIC_REGISTRY=true, which provisions the registry as Standard with anonymous pull; customer deployments keep the default Basic registry), and every release runs the Release pipeline so osr-platform-api:<version> / osr-platform-web:<version> exist before the plan's image tag is bumped.