<!-- OpenSmartRoute: Deploy with Docker and Helm. https://opensmartroute.ai/docs/deploy -->
# Reference deployments

OpenSmartRoute runs in three shapes. All three use the same `Router`; only where it lives differs.

| Shape | When | How |
|---|---|---|
| **Library** | one Python service, in-process decisions (~0.3 ms / route) | `pip install opensmartroute` and call `Router.route()`; see [docs/SDK.md](https://opensmartroute.ai/docs/SDK.md) |
| **Sidecar** | polyglot services, per-pod learner state | run the image next to your app: `docker run -p 8000:8000 -v ./examples:/config crosrua2c6onyidguw.azurecr.io/opensmartroute:1.0.0` and call `POST /route` |
| **Control plane** | many callers, shared catalogue, shared learning | the Helm chart in helm/opensmartroute (`deploy/helm/opensmartroute`) behind a `Service`/`Ingress`; state in Redis/SQL via `RouterBuilder.with_state_store` |

## Image

Dockerfile (`deploy/Dockerfile`) builds a non-root, read-only-rootfs image with the `yaml` and `server` extras:

```sh
docker build -t opensmartroute:1.0.0 -f deploy/Dockerfile .
docker run --rm -p 8000:8000 -v "$PWD/examples:/config:ro" opensmartroute:1.0.0
curl -s localhost:8000/route -H 'content-type: application/json' -d '{"text":"summarise this contract"}'
```

Configuration is by environment (`OSR_TARGETS`, `OSR_RULES`, `OSR_MODELS`, `OSR_SLM`, `OSR_STATE`,
`OSR_HOST`, `OSR_PORT`) so the same image serves both sidecar and control-plane roles;
entrypoint.sh (`deploy/entrypoint.sh`) translates them into `osr serve` arguments. `OSR_SLM` points at a
routing SLM file from `osr slm train`; it joins the strategy ensemble and is reported as the `slm`
strategy in every decision trace.

The OpenAI-compatible proxy (`/v1/chat/completions`) executes LLM targets when the container knows an
upstream: set `OSR_LLM_BASE_URL` (OpenAI, Azure OpenAI, vLLM, Ollama, OpenRouter, LiteLLM ...) and
`OSR_LLM_API_KEY` (or `OSR_LLM_API_KEY_ENV` naming another variable / `<NAME>_FILE` mount). Each LLM
target's `metadata.model` is the upstream model name; `OSR_LLM_MODEL` is the default for targets
without one. Without a base URL the proxy still routes but answers 502 for LLM targets.

Access control: the image starts open (every endpoint reachable) so it can act as a sidecar on a
private network. To require a token set `OSR_SERVER_AUTH_TOKENS=<t1>,<t2>` (or mount the list and
point `OSR_SERVER_AUTH_TOKENS_FILE` at it) and, for a hard guarantee, `OSR_SERVER_REQUIRE_AUTH=1` so
the container refuses to start without one. Mint tokens with `osr token generate`; clients send
`Authorization: Bearer <t>` or `X-API-Key: <t>`, and `osr login --url https://<host> --token <t>`
stores one for the CLI (`osr whoami` then shows `edition self-hosted`). `/healthz`, `/readyz`,
`/metrics`, `/whoami` and the OpenAPI documents stay open for probes and discovery.

Self-operation: `OSR_AUTOPILOT=1` (with `OSR_SLM` and a writable `OSR_STATE`) runs the SLM's
improvement loop inside the container - every `OSR_SLM_AUTOPILOT_INTERVAL_S` seconds (default one
hour) and early when the drift monitor sees the served success rate drop, it gathers evidence
(feedback joined to prompts, cached datasets, seeds), trains a challenger, and hot-swaps it into the
ensemble when it beats the champion on the holdout. The promoted model is written to
`$OSR_STATE/autopilot/slm.json` (or `OSR_CACHE_DIR`), which the entrypoint prefers over `OSR_SLM` on
the next start. `OSR_AUTOPILOT_ARGS` passes extra flags (`--offline`, `--source NAME`,
`--tier TIER=TARGET`, `--search QUERY`); `OSR_CATALOGUE` names a catalogue JSON to refresh. `GET /stats`
shows the `autopilot` status and `POST /autopilot/cycle` schedules a cycle now.

Observability: every request is traced (spans and events per stage, see
[docs/OBSERVABILITY.md](https://opensmartroute.ai/docs/OBSERVABILITY.md)). `GET /healthz` is the liveness probe and `GET /readyz`
the readiness probe (503 until the router can score). `GET /metrics` serves the Prometheus exposition
(`osr_route_decisions_total{target}`, `osr_route_latency_ms_bucket`, `osr_route_policy_rejections_total`,
`osr_route_errors_total`, `osr_outcome_*`, `osr_decision_cache_*`, a `targets` gauge, plus per-stage
`osr_span_duration_ms`), `GET /trace/{request_id}` the full trace of one
request (each response carries `X-OSR-Trace-Id`; send `traceparent` to join your own trace) and
`GET /events?name=route.*` the recent buffer. Tune with `OSR_OBSERVABILITY_MEMORY_EVENTS` (buffer size, `0`
off), `OSR_OBSERVABILITY_LOG_EVENTS=true` (JSON lines on stdout), `OSR_OBSERVABILITY_EVENTS_FILE`
(JSONL under the state volume), `OSR_OBSERVABILITY_OTEL=true` (OpenTelemetry, needs the `otel` extra and
the usual `OTEL_EXPORTER_*` variables) and `OSR_OBSERVABILITY_SAMPLE_RATE` under heavy load. The hosted
platform image (platform/README.md (`platform/README.md`)) reads the same variables and adds the
workspace-scoped `GET /api/v1/trace/{request_id}`, `GET /api/v1/events` and the public `GET /api/v1/status`.

## Helm

```sh
helm install router deploy/helm/opensmartroute \
  --set-file config.targets=examples/targets.yaml \
  --set-file config.rules=examples/rules.yaml
```

Key values (see values.yaml (`deploy/helm/opensmartroute/values.yaml`) for all):

- `config.targets` / `config.rules` / `config.models` -> rendered into a ConfigMap at `/config`;
  the pod restarts on change (config checksum annotation).
- `autopilot.enabled` / `autopilot.intervalSeconds` / `autopilot.args` -> self-operation as above
  (needs `config.slm` and `persistence.enabled`). The autopilot promotes models per pod: with
  `replicaCount` above 1 each replica trains and serves its own champion, so run one replica (or
  one autopilot pod behind a shared `StateStore`) when the loop is on.
- `auth.tokens` / `auth.existingSecret` / `auth.required` -> access tokens for the HTTP API (a
  chart-managed Secret, or your own with a `tokens` key), and whether the pod may start without one.
- `persistence.enabled` -> keep bandit posteriors and the feedback log on a PVC (single replica or
  RWX class). For several replicas leave it off and use the Redis/SQL `StateStore`.
- `autoscaling`, `podDisruptionBudget`, `networkPolicy`, `ingress` -> standard knobs, all off or
  conservative by default.
- Security: `runAsNonRoot`, `readOnlyRootFilesystem`, all capabilities dropped, `automountServiceAccountToken: false`.

Secrets (MCP manifest signing key `OSR_MCP_KEY`, encrypted-state key) are injected with `env`/`envFrom`
`secretKeyRef`s - never in `values.yaml`.

Endpoints: `GET /healthz`, `GET /whoami`, `GET /targets`, `GET /stats`, `POST /route`, `POST /feedback`,
`GET /v1/models`, `POST /v1/chat/completions` (OpenAI-compatible proxy, `model: "auto"`).