Observability and traceability

Nothing happens that you cannot replay.

A routing decision is never a label alone. OpenSmartRoute keeps, for every request, what it saw, which targets policy let through, how they ranked and why one won - then the execution, its cost and the outcome you reported. Engineers read it as a span waterfall; managers read it as a monthly report; auditors read it as a chain.

Watch a live trace Read the guide

A traced request, abridgedtext

$ osr trace 9f1c...  # X-OSR-Request-Id of any answer
route  84 ms  target=coding-agent confidence=0.87
├─ signals        3 ms  task=coding complexity=0.71 pii=false
├─ policy         1 ms  rejected llm-small: complexity 0.71 > max 0.45
├─ rank           6 ms  coding-agent 0.91 · llm-frontier 0.78 · llm-mid 0.61
└─ execute       74 ms  tokens=1,204 cost=$0.0048 provider=azure
outcome  success=true quality=0.9  (reported 41 s later)

What every trace carries

Five stages, in the order the router took them.

The trace view tells the decision as a story a non-engineer can follow, and keeps the raw spans, attributes and timings underneath for the one who needs them.

1What the request looks like
2Who was allowed to answer
3How the candidates ranked
4Why this target
5Plan and execution

In every answer

POST /api/v1/route returns the decision with these fields; the OpenAI-compatible endpoints carry the same in an opensmartroute metadata block and the X-OSR-Request-Id / X-OSR-Trace-Id headers.

target
confidence
signals
policy_rejections
ranked
cost_usd
latency_ms
trace_id

GET /api/v1/trace/{request_id} returns the spans and events of one request with its activity row, the feedback reported for it and its audit records - from the buffer, or from the store once evicted. A tenant only ever sees its own requests.

The route endpoint reference

Where to look

Four places, one set of facts.

Each surface reads the same spans, events and usage records; nothing is summarised from a different source than the trace itself.

Activity log
Every metered request: endpoint, key, target, provider, upstream model, tokens, cost, latency, success, and the outcome you reported. Open a row and the trace unfolds beside it.
Open activity
Events explorer
Guard blocks and redactions, no-route cases, fallbacks, breaker transitions, tenant rejections - filtered by name, level and time, from the last hour to seven days.
Open events
Audit trail and savings ledger
Decisions and outcomes in a hash-chained trail, exportable as CSV or JSONL; the savings ledger prices every executed request against the baseline you choose.
Open savings
Health and status
Per-provider health with when it was checked, breaker state, latency percentiles, alert conditions with the page that fixes each - and a public status page with the readiness checks.
Open status

Your tools, not ours

OpenTelemetry, Prometheus, or nothing to install.

The hosted platform is its own observability: it persists every span and event, keeps HTTP counters and evaluates alert rules in-process. When you already run Grafana, Datadog or Azure Monitor, one sink sends the same spans there.

Each router span becomes an OTel span, each event a span event and an osr.events counter, durations an osr.span_duration_ms histogram
Prometheus metrics from osr serve at /metrics: events, spans, decisions, executions, cost, learner outcomes, breaker transitionsGuide
Application Insights export on the platform image; the governance report shows whether it is onControls
osr trace and osr events read the same data from the command line, signed in with osr login

OpenTelemetrypython

from opensmartroute import configure_tracing
from opensmartroute.adapters.optional import OpenTelemetrySink

configure_tracing(OpenTelemetrySink())   # or OSR_OBSERVABILITY_OTEL=true
# spans: osr.route, osr.plan, osr.execute ...  attributes: osr.target,
# osr.confidence, osr.request_id, osr.trace_id  (joined to traceparent)

Close the loop

Outcomes make the next decision better.

A trace is also where learning starts. Report whether the answer was good - from the API, the dashboard, the extension or the desktop app - and the learners move the ranking; the routing model is retrained on your own traffic by the autopilot.

POST /api/v1/feedback with success, quality, cost and latency; the activity row shows the outcome next to the decisionActivity
Per-target outcome statistics and leaderboards on the learning pageLearning
Rankings by task from reported outcomes, public and per workspaceRankings
The impact page: what routing changed for this workspace, in its own numbersImpact

How the learners use feedback

Route one request. Read the whole story.

The free plan keeps the full trace for every decision. Send a request from the playground, open its activity row and follow the five stages.

Create a free account Open the playground

Free plan, no card. Fifteen thousand decisions a month with the full trace.

Nothing happens that you cannot replay.

$ osr trace 9f1c... # X-OSR-Request-Id of any answer route 84 ms target=coding-agent confidence=0.87 ├─ signals 3 ms task=coding complexity=0.71 pii=false ├─ policy 1 ms rejected llm-small: complexity 0.71 > max 0.45 ├─ rank 6 ms coding-agent 0.91 · llm-frontier 0.78 · llm-mid 0.61 └─ execute 74 ms tokens=1,204 cost=$0.0048 provider=azure outcome success=true quality=0.9 (reported 41 s later)

OpenTelemetry, Prometheus, or nothing to install.

Each router span becomes an OTel span, each event a span event and an osr.events counter, durations an osr.span_duration_ms histogram

Prometheus metrics from osr serve at /metrics: events, spans, decisions, executions, cost, learner outcomes, breaker transitionsGuide

Application Insights export on the platform image; the governance report shows whether it is onControls

osr trace and osr events read the same data from the command line, signed in with osr login

OpenTelemetrypython

from opensmartroute import configure_tracing
from opensmartroute.adapters.optional import OpenTelemetrySink

configure_tracing(OpenTelemetrySink())   # or OSR_OBSERVABILITY_OTEL=true
# spans: osr.route, osr.plan, osr.execute ...  attributes: osr.target,
# osr.confidence, osr.request_id, osr.trace_id  (joined to traceparent)

Outcomes make the next decision better.

POST /api/v1/feedback with success, quality, cost and latency; the activity row shows the outcome next to the decisionActivity

Per-target outcome statistics and leaderboards on the learning pageLearning

Rankings by task from reported outcomes, public and per workspaceRankings

The impact page: what routing changed for this workspace, in its own numbersImpact

How the learners use feedback

Nothing happens that you cannot replay.

Five stages, in the order the router took them.

Four places, one set of facts.

Activity log

Events explorer

Audit trail and savings ledger

Health and status

OpenTelemetry, Prometheus, or nothing to install.

Outcomes make the next decision better.

Route one request. Read the whole story.

Nothing happens that you cannot replay.

Five stages, in the order the router took them.

Four places, one set of facts.

Activity log

Events explorer

Audit trail and savings ledger

Health and status

OpenTelemetry, Prometheus, or nothing to install.

Outcomes make the next decision better.

Route one request. Read the whole story.