# Install the platform The **hosted platform** at [opensmartroute.ai](https://opensmartroute.ai) needs no installation - sign up, create a key, follow the [platform quickstart](https://opensmartroute.ai/docs/QUICKSTART_PLATFORM.md). This page is for running the **whole platform yourself**: the routing API with its OpenAI-compatible endpoint, the web app (public site, dashboard, operator console), the routing SLM service and the state behind them. The same images and the same `OSR_PLATFORM_*` settings run everywhere; only the way they are started differs. | Where | How | For | |---|---|---| | Azure, from the portal | [Azure Marketplace](#azure-marketplace) - a wizard, one resource group, no tooling | Teams that want the platform in their own subscription without a checkout | | Azure, from a checkout | [`azd up`](#azure-with-azd) - the same template, built from source | Contributors, custom builds, the publisher's own deployment | | Your own servers or any cloud VM | [Docker Compose](#your-own-servers-docker-compose) - PostgreSQL, Redis, Kafka and the platform as containers | On-premises, air-gapped, other clouds | | Kubernetes | [`osr serve` Helm chart](https://opensmartroute.ai/docs/deploy.md) for the router alone; the platform on Kubernetes runs the same containers as Compose | Sidecar / control-plane routing without the hosted-platform surfaces | Just the Python library or the `osr` command line: [README - Install](https://opensmartroute.ai/docs/readme.md#install). ## What you get Every install path deploys the same product: - `osr-platform-api` - the FastAPI platform: `POST /api/v1/route`, the OpenAI-compatible `/v1` proxy, MCP server, accounts, keys, workspaces, tenants, governance, marketplace, billing, notifications, support tickets and the operator API. In cluster mode it also runs once per domain (docs, rankings, marketplace, providers, onboarding, accounts, billing, admin, mcp, openai, routing) behind the API gateway, plus the routing SLM service. - `osr-platform-web` - the Next.js web app: the public site and documentation, `/platform` (sign-in, dashboard, playground, publish flow) and `/platform/admin` (the operator console). - State: PostgreSQL (always), Redis and Kafka (cluster mode), a data volume for the audit chain and the routing SLM. - Model providers: Azure OpenAI deployments on Azure; anything OpenAI-compatible everywhere (connected in the console under *Providers*, or by workspaces with their own keys under *Model providers*); optionally a self-hosted Ollama server for the private / specialist targets. Release images: `crosrua2c6onyidguw.azurecr.io/osr-platform-api:1.2.0` and `crosrua2c6onyidguw.azurecr.io/osr-platform-web:1.2.0` (anonymous pull; `:latest` follows the newest release, `:1.2` the newest patch). The web image is built without a baked hostname: it reads `OSR_PUBLIC_WEB_URL` at run time and the browser uses the origin it is talking to, so one image serves any domain. ## Azure Marketplace The offer **OpenSmartRoute platform** is an Azure Application (solution template): it deploys into a resource group of your subscription, you own every resource, nothing is shared with the publisher and there is no software charge on the Azure bill - you pay the Azure resources it creates. ### Deploy 1. In the Azure portal open *Marketplace*, search for **OpenSmartRoute**, choose the plan and *Create*. 2. **Basics** - subscription, a new or existing resource group, the region (Azure OpenAI and Container Apps must be offered there; the default `gpt-4.1` family is), a deployment name, the first **operator username and password** (the console sign-in), the edition (Enterprise: tenants, audit, statistics, persisted learners; Community: the lightest footprint). 3. **Size and state** - *Cluster mode* (Redis + Kafka, the API split into domain services scaled on demand; off = one API replica with PostgreSQL only, right for a trial) and *Managed state* (Azure Database for PostgreSQL with point-in-time restore, Azure Cache for Redis, Event Hubs - recommended for production; off = state containers on Azure Files, cheapest). 4. **Hostnames** - leave empty to use the Container Apps hostnames, or enter your web and API hostnames (and let the template create the Azure DNS zone). TLS certificates are issued in a second run once DNS resolves (below). 5. **E-mail** - Azure Communication Services or SMTP for verification, reset, invitation, alert and support-ticket mail. Without a transport every message waits in the console's *Mail* page; nothing is lost. 6. **Models and learning** - the self-hosted model server (Ollama, CPU or serverless GPU), the autopilot (retrain the routing model from your own feedback) and the marketplace catalogue refresh. 7. **Sign-in and access** - password sign-in, self-serve signups per IP (0 = operators create every workspace), SSO providers, a static admin token for scripts, public `/metrics`. 8. **Images and billing** - the release to deploy and the registry to pull from (yours, after mirroring), Stripe for self-serve paid plans. 9. *Review + create*. A cluster-mode deployment takes about ten minutes; the first Ollama start pulls about 20 GB of weights in the background. The deployment's **Outputs** tab lists `webUrl`, `platformUrl`, `adminLoginUrl`, `apiUrl`, the hostnames and static IP for your DNS records, the `asuid` verification id and the Azure OpenAI endpoint. ### First sign-in 1. Open `adminLoginUrl` (`/platform/admin/login`) and sign in with the operator username and password from Basics. The console shows the deployment's health, users, workspaces, plans, providers and the support desk. 2. *Providers*: the Azure OpenAI account created by the template is connected and the cloud tiers (`llm-small`, `llm-mid`, `llm-frontier`) already execute on it. Add OpenAI, Anthropic, Bedrock, Vertex, vLLM or any OpenAI-compatible endpoint here, or let workspaces bring their own keys. 3. Open `webUrl`, *Sign up* (or create the workspace from the console when self-serve signup is off), create an API key and route a request - [platform quickstart](https://opensmartroute.ai/docs/QUICKSTART_PLATFORM.md) with `OSR_API_URL=`. 4. `osr login --url ` signs the command line in to your deployment. ### Custom domain (two runs) 1. First run with your hostnames entered and *Issue TLS certificates* off: the apps bind the hostnames without TLS. 2. Create the DNS records from the outputs: `CNAME -> webHostname`, `CNAME -> apiHostname` (an apex domain gets `A -> staticIp`), and `TXT asuid. -> domainVerificationId` for each hostname. If the template created the zone, delegate the domain to `dnsNameServers` instead - the records are already there. 3. Run the deployment again (*Redeploy* on the deployment, same values) with *Issue TLS certificates* on. Azure issues free managed certificates and the hostnames answer over HTTPS. ### Upgrade, backup, remove - **Upgrade**: redeploy with a newer *OpenSmartRoute release* (image tag). The database schema migrates itself at start-up; state lives in PostgreSQL and on the data share, never in the containers. When an upgrade changes the `osr-postgres` container itself, the new replica waits for the previous one to shut down before it opens the database (Container Apps runs both for a moment); expect the API to pause for up to a minute. - **Backup**: the `osr-db-backup` job writes a nightly `pg_dump` to the `osr-backups` share (14 days). Managed state adds point-in-time restore on the server. Restore: platform/README.md (`platform/README.md`) "Containers and Azure". - **Remove**: delete the resource group. Nothing is left outside it. ### Sizing and cost (Azure list prices, rough) | Configuration | Monthly Azure cost | Suits | |---|---|---| | Community, cluster mode off, Azure Files state | ~40-70 USD (Container Apps consumption, storage, Log Analytics) + model usage | Trial, a team | | Enterprise, cluster mode, Azure Files state | ~150-250 USD | Production without SLA requirements | | Enterprise, cluster mode, managed state (B1ms PostgreSQL, C0 Redis, 1 TU Event Hubs) | ~250-400 USD | Production with backups, SLAs, HA options | | + Ollama on serverless GPU (T4) | pay per second of GPU while replicas run | Private / specialist models without cloud keys | Model usage (Azure OpenAI tokens) is billed by Azure OpenAI at its own prices; the dashboard's *Savings* page shows what routing saved against always calling the frontier model. ## Azure with azd From a checkout, the same template with images built from source: ```bash azd auth login azd env new prod azd env set OSR_PLATFORM_ADMIN_USERNAME admin azd env set OSR_PLATFORM_ADMIN_PASSWORD '' azd up ``` Every parameter of the Marketplace wizard is an `azd env set OSR_...` variable (infra/main.parameters.json (`infra/main.parameters.json`)); platform/README.md (`platform/README.md`) documents them, the custom-domain flow (`scripts/domains.ps1`), SSO (`scripts/sso.ps1`) and the pipelines that keep the publisher's deployment current. ## Your own servers (Docker Compose) Runs on any host with Docker Engine 24+ (Linux, or Docker Desktop for a workstation): on-premises, air-gapped, or a VM in any cloud. Minimum 4 vCPU / 8 GiB / 40 GB disk for the platform; add what your models need if you run them on the same host. ### 1. Get the stack ```bash git clone the project repository && cd OpenSmartRoute cp platform/.env.example platform/.env ``` Or download only platform/docker-compose.yml (`platform/docker-compose.yml`) and platform/.env.example (`platform/.env.example`) - the stack runs from the published images. ### 2. Configure Edit `platform/.env`: the passwords and the session secret (all of them - the defaults are placeholders), the public URLs of the web app and the API as your users will reach them, the release images, and the e-mail transport. Everything is documented in the file; the same variables are explained in platform/README.md (`platform/README.md`). ### 3. Start ```bash docker compose -f platform/docker-compose.yml --env-file platform/.env pull docker compose -f platform/docker-compose.yml --env-file platform/.env up -d curl -fsS http://localhost:8080/readyz # every check ok once PostgreSQL is up and the schema is created ``` Web `http://localhost:3000`, operator console `http://localhost:3000/platform/admin/login` (the operator username and password from `.env`), API `http://localhost:8080/api/v1/info`. Without `OSR_API_IMAGE` / `OSR_WEB_IMAGE` the stack builds the images from the checkout (`up -d --build`). ### 4. Connect models The stack has no cloud account. Either connect providers in the console under *Providers* (any OpenAI-compatible endpoint: OpenAI, Azure OpenAI, Anthropic through a gateway, vLLM, Ollama, ...) or start the on-prem model server next to it: ```bash docker compose -f platform/docker-compose.yml -f deploy/compose/providers.ollama.yaml --env-file platform/.env up -d ``` deploy/compose/providers.ollama.yaml (`deploy/compose/providers.ollama.yaml`) runs Ollama with the curated open weights (Qwen3, Gemma 3, Qwen2.5-Coder, Qwen3-VL, an embedding model) and maps the private / specialist targets to it; deploy/compose/ollama.gpu.yaml (`deploy/compose/ollama.gpu.yaml`) adds an NVIDIA GPU. `OSR_PLATFORM_PROVIDERS` points the API at a providers file for anything else. ### 5. Put it behind TLS Terminate TLS in your reverse proxy (nginx, Caddy, Traefik, an ingress) in front of ports 3000 (web) and 8080 (API), set `OSR_PLATFORM_WEB_URL` / `OSR_PLATFORM_BASE_URL` to the public `https://` origins, and remove the host port mappings you do not want exposed. Several API replicas share PostgreSQL / Redis / Kafka: `docker compose up -d --scale api=3` behind the proxy, exactly what the Azure template does. ### Operate - **Backups**: `docker compose -f platform/docker-compose.yml exec postgres pg_dump -U osr osr | gzip > osr-$(date +%F).sql.gz` (and `osr_telemetry`); restore with `psql` into a fresh volume. - **Upgrade**: set the new tag in `.env`, `pull`, `up -d`. The schema migrates at start-up. - **Air-gapped**: on a connected machine `docker pull` both images (and `postgres:16-alpine`, `redis:7-alpine`, `apache/kafka:3.9.0`), `docker save | gzip`, move, `docker load`; or mirror them into your registry (`az acr import --source crosrua2c6onyidguw.azurecr.io/osr-platform-api:1.2.0 ...`) and set `OSR_API_IMAGE` / `OSR_WEB_IMAGE` (or the Marketplace wizard's *Image registry*). - **Health**: `GET /readyz` on the API (database, Redis, Kafka, mail transport, routing probe), `/platform/admin/health` in the console, `GET /metrics` for Prometheus. ## Publishing the Marketplace offer (for the publisher) The package Partner Center takes is built by the *Release* pipeline (artifact `marketplace`) and, locally, by ```bash python -X utf8 scripts/marketplace_package.py check # wizard <-> template consistency python -X utf8 scripts/marketplace_package.py build --ttk # mainTemplate.json + createUiDefinition.json -> dist/marketplace/*.zip ``` marketplace/azure/mainTemplate.bicep (`marketplace/azure/mainTemplate.bicep`) wraps infra/resources.bicep (`infra/resources.bicep`) for a resource-group deployment with the release images; marketplace/azure/createUiDefinition.json (`marketplace/azure/createUiDefinition.json`) is the wizard. The `Infra` pipeline builds the package on every change; `release.py check` keeps the image tag in step with the release. The offer is published from [Partner Center](https://partner.microsoft.com/dashboard/marketplace-offers): an **Azure Application** offer of type **Solution template**, one plan per supported topology if you want separate listings (a single plan with the wizard's choices is enough), the zip uploaded under *Technical configuration*, listing texts and screenshots from docs/BRAND.md (`docs/BRAND.md`) and the sales kit, then *Review and publish* - Partner Center runs the same arm-ttk checks and a test deployment before the preview goes live. Requirements the publisher keeps true: the two image repositories allow anonymous pull (the publishing deployment sets `OSR_PUBLIC_REGISTRY=true`, which provisions the registry as Standard with anonymous pull; customer deployments keep the default Basic registry), and every release runs the *Release* pipeline so `osr-platform-api:` / `osr-platform-web:` exist before the plan's image tag is bumped.