Platform guide

The hosted platform runs OpenSmartRoute as a service. A workspace has an API key and a plan; each request to the API is routed to a target (model, agent, skill, persona or tool) and answered with the decision, its trace and, on request, the target's output. An OpenAI-compatible endpoint accepts existing chat-completion clients.

This page covers calling the platform: authentication, routing, execution, the OpenAI-compatible endpoint, feedback, plans, organizations, tenants, governance (workspace policy and budgets), observability (probes, metrics, per-request traces and the event stream), caching, the dashboard and the MCP server. The marketplace - shared agents, skills, personas, prompts and stack templates - has its own page. To run the router on your own infrastructure see Deploy with Docker and Helm; to use it as a Python library see the user guide and the SDK guide. Every endpoint below is listed in the REST API reference (/docs/api) and in /openapi.json.

1. Authentication

1. Sign up at /platform/signup with Microsoft, Google, GitHub, GitLab, LinkedIn (the providers the deployment enables - GET /api/v1/info lists them under sso; the LinkedIn app the growth engine posts with doubles as the sign-in provider unless OSR_PLATFORM_LINKEDIN_SSO=false) or an email address - with a password if you want to sign in by email later (OSR_PLATFORM_PASSWORD_LOGIN, on by default). Signup creates a personal workspace on the free plan and issues an API key. With Google configured the site also shows Google's One Tap prompt ("Sign in to ... with google.com") to visitors who are not signed in; picking an account posts the ID token to POST /api/v1/auth/{provider}/credential {credential, next} (Google only for now), which has Google validate it and answers with the same browser session as the redirect callback - no page change, a first sign-up still shows the key once. The Google OAuth client must list the site's origin (OSR_PLATFORM_WEB_URL) under Authorized JavaScript origins or Google refuses to show the prompt.
2. The key is shown once. Store it in a secret manager. Keys are created, renamed, rotated and revoked in the dashboard (/platform/dashboard/keys) or with POST /api/v1/keys, PATCH /api/v1/keys/{key_id} {name}, POST /api/v1/keys/{key_id}/rotate (a new secret for the same key, shown once; the old one stops working immediately) and DELETE /api/v1/keys/{key_id} - all admin role, or the lead of the team a key was minted for (team on POST /api/v1/keys, Teams). A key does everything its workspace can unless you limit it: scopes (read - every GET; route - decisions without execution: /route, /estimate, MCP ranking tools; execute - anything that can spend money: execute: true, /v1/*, the MCP ask tool; feedback; manage - every other write) and expires_in_days (1-3650) on POST /api/v1/keys or PATCH /api/v1/keys/{key_id} {scopes}; GET /api/v1/keys/scopes lists them. A scoped key that steps outside its scopes is refused with 403 and X-Key-Scope: <missing scope> before anything is metered; an expired key answers 401 API key expired until it is revoked (rotation keeps the expiry). limits narrows what a key may spend and reach: daily_budget_usd / monthly_budget_usd cap the key's own spend (checked like the workspace and tenant budgets - 429 with the X-Budget-* headers once reached; routing without execution still works), requests_per_minute caps its rate below the plan's (429 with Retry-After and X-Key-Limit: requests_per_minute), tokens_per_minute caps the tokens it may send and receive in a sliding minute (charged with the request's bound - the prompt plus the output allowance - before any provider is called, so a burst is refused rather than billed: 429 with Retry-After and X-Key-Limit: tokens_per_minute), allowed_ips pins the key to client addresses or CIDR blocks (at most 32; the address is the one past the trusted proxies, never a header the client wrote; a call from elsewhere answers 403 with X-Key-Limit: allowed_ips) and allow_targets lists the target ids the key may route to (the router never leaves the list, a pinned model outside it answers 403 with X-Key-Limit: allow_targets, and a caller's own allow_targets can only narrow it further). required_headers names headers every request with the key must carry (a project tag, a ticket id, a cost centre - at most ten; a request without them answers 428 with X-Key-Limit: required_headers), and mcp_tools lists the connected MCP servers' tools the key may reach (server:tool or server:*; section 12). PATCH /api/v1/keys/{key_id} {limits} replaces them ({} clears); GET /api/v1/keys/scopes lists the limit keys; every change is audited (key.limited). Give keys that leave your hands - CI, a partner, a demo - the smallest scopes, an expiry and a budget.
3. Send the key with every metered request in either header, and - when you want the row attributed to a cost centre without a tenant - the project tag:

Authorization: Bearer osr_...
X-API-Key: osr_...
X-OSR-Project: checkout-service

X-OSR-Project (letters, digits, ., _, -; up to 64) is recorded on the usage row as project and broken out in GET /api/v1/usage as by_project - cheaper than a tenant when all that is wanted is a line in the report. The Google GenAI SDKs' x-goog-api-key header is accepted as the key too (section 4).

Public endpoints (/api/v1/info, /api/v1/models, /api/v1/llms, /api/v1/rankings, /api/v1/catalogue, /api/v1/estimate, /api/v1/stats/public, /api/v1/registry) need no key.

Email and password

POST /api/v1/auth/password/login {email, password} returns the same browser session as a single sign-on callback (session_token, user, account, plan); the dashboard stores it and sends it as a bearer token. Passwords are at least 8 characters (no composition rules) and are stored as salted scrypt hashes. A signed-in user sets or changes theirs with POST /api/v1/auth/password {password, current_password} - the current one is required once set and every other browser is signed out - and removes it with DELETE /api/v1/auth/password as long as a provider identity remains. Sign-in attempts are limited per address and per email; has_password on the user object says which users have one. /platform/dashboard/account has the form.

Two-factor authentication. A person with a password can add an authenticator app (TOTP, RFC 6238: six digits every 30 s, one step of drift accepted) under Account › Two-factor authentication: POST /api/v1/auth/mfa/setup {current_password} returns the secret and the otpauth:// URI, POST /api/v1/auth/mfa/enable {code} confirms it and returns eight single-use recovery codes (shown once; other browsers are signed out), POST /api/v1/auth/mfa/recovery-codes {code} replaces them and DELETE /api/v1/auth/mfa {current_password, code} switches the factor off - each change is e-mailed as a security notice (two_factor_changed). From then on POST /api/v1/auth/password/login answers mfa_required with a five-minute signed challenge instead of a session, and POST /api/v1/auth/mfa/verify {challenge, code} with the authenticator code (or an unused recovery code) issues it; the sign-in page shows the code step. A password-reset link proves the inbox, not the second factor, so POST /api/v1/auth/password/reset saves the password and answers the same challenge (password_changed: true). Identity providers bring their own second factor and are not asked again; API keys are the workspace, not a person, and never are. mfa_enabled, mfa_enabled_at and recovery_codes_left are on the user object; an operator resets a locked-out person with PATCH /api/v1/admin/users/{id} {"mfa_reset": true} (sessions end, the person is told). History: user.mfa_enabled, user.mfa_disabled, user.mfa_recovery_used, user.mfa_recovery_codes_replaced, user.login_mfa_failed, user.mfa_reset.

Account lifecycle: confirmation, recovery, deletion

Anyone can create an account with any email address. What happens around it:

Signup, confirmation, password recovery, invitation links and this checklist form the onboarding domain, which a deployment can run as its own service (see Services).

Every one of these events - plus sign-ins, failed sign-ins, role changes, removals and operator actions

• is written to the account audit log. Workspace admins read their workspace's entries with GET /api/v1/workspace/audit; operators see everything at /platform/admin/audit (GET /api/v1/admin/audit-log) and per user on /platform/admin/users/<id>, where they can also issue a reset link, re-send or force the email confirmation, disable, sign out everywhere or delete the user.

Sign-in locations and alerts

Every browser session and every account event records the address it came from (ip). An address alone tells a person little - 203.0.113.9 does not say whether that was their own phone on holiday or somebody else - so the platform can turn it into a place (location: country, region, city and a label such as Lisbon, Portugal) from one of two sources the operator chooses; without either, nothing is resolved and the address alone is shown:

Both settings are also under Runtime configuration in the console, so a deployment can switch them on without a redeploy. Choose a service whose terms fit your use: sending a visitor's address to a third party is a processing of personal data, and your privacy notice should name it.

Sign-in alerts. A sign-in from a device the account has not been seen on in ninety days, or - once places are resolved - from a country none of its recent sessions came from, is announced to the person by e-mail (new_sign_in, a security message that cannot be switched off) and in their notifications, with the device, the method, the place and the address; the account history records user.new_device or user.new_location. A person who wants to hear about every sign-in chooses Every sign-in under Sign-in alerts on /platform/dashboard/account (PUT /api/v1/me/login-alerts {mode}, new or always; login_alerts on the user object). The very first session of an account is not announced - the welcome covers it.

What operators see. The user page shows each session's place next to its address and a Sign-in places card: where the account was signed in from in the last ninety days, with the addresses, the devices, how many sign-ins and the first and last one - a traveller shows one device in several places, a takeover a new place on a new device. The deployment audit log carries the place of every event, filters by country (country=DE, -- for events without a place), searches it with q and exports it in the CSV.

Forgetting. Addresses and places are kept with the row for OSR_PLATFORM_NETWORK_DATA_RETENTION_DAYS (365; 0 = as long as the event) and then removed by the retention sweep - the event stays, the address and the place go. Forget addresses on the user page (DELETE /api/v1/admin/users/{id}/network-data) does the same for one person at once, for a privacy request; it is audited as user.network_data_cleared with the operator's own address and place, like every operator action.

Operators too. A console sign-in records its address and place on the operator's session (shown under Operators), operator.login, operator.login_failed, operator.login_mfa_failed and operator.logout are in the audit log with the place, and every operator action carries the address and place of the console session it came from. A console sign-in from a browser or a country the operator has not used in ninety days is written as operator.new_device / operator.new_location and filed in the console's bell for every operator to see.

Email: transports, templates, welcome, newsletter and reports

Every message the platform sends is composed once as a branded HTML card (privacy and terms links in the footer) with a plain-text alternative: the welcome message (greeting, a four-step checklist, dashboard and quick-start links - sent by signup, or by the first sign-in of a person who arrived another way: a shared or an organization's identity provider, an operator-created user, a password-reset link; never twice), address confirmation (sent by signup; the first sign-in sends it too when the address is still unconfirmed and no link ever went out), password reset, invitation, account deleted, password changed, the operator test message, usage reports, alerts, support-ticket mail, product news and newsletter issues. Each is written to the outbox first and then delivered by whichever transport is configured (the first configured one in this order wins).

Every message follows the same design so a reader knows what they are holding before they read a word: the wordmark, a coloured band and a small label naming the kind of message (Security in red, Alert in amber, Account in ink, Getting started and Report in blue, Release notes and Product news in green, Newsletter in violet, Support and Feedback in teal), one heading, short sentences that read at an eighth-grade level (tests/test_email_readability.py fails a template that does not), one clear action, a plain-text alternative, and a footer that says why the person is receiving it and where to change that. A message that could worry someone - a new sign-in, failed attempts, a change to the account - always says what happened, whether anything is lost, what to do if it was not them, and that a reply reaches a person.

Delivery guarantees. Sending is at-least-once with bounded retries. Every attempt is recorded on the outbox row (attempts, error, next_attempt_at); a failure is retried on a back-off ladder (30 s, 2 min, 10 min, 30 min, 2 h - six attempts in all) by a sweeper thread in the API, one leader per Redis lease in a cluster. A row whose sending process disappeared mid-send is retried after five minutes. Once the ladder is exhausted the row stays in the outbox as failed for an operator to resend by hand (POST /api/v1/admin/mail/{id}/resend always tries once more). ACS accepts a message with 202 and delivers asynchronously; the platform polls the operation status briefly and records a terminal Failed (unverified sender, suppressed recipient) as an error instead of a false "sent", keeping the operation id as provider_id. GET /api/v1/admin/mail returns a delivery block (transport ok, last error, rows retrying, sweeper state) next to summary.retrying and summary.abandoned; GET /readyz reports the same under mail (informational, never fails readiness).

The gallery: every message before a customer sees it. Operators manage what the platform says from /platform/admin/mail: GET /api/v1/admin/mail/templates lists every kind of message with its category, whether people can switch it off and how many went out; GET /api/v1/admin/mail/templates/{kind} renders it with sample data (subject, the HTML card, the plain-text alternative) and the page shows it as a customer would see it; POST /api/v1/admin/mail/templates/{kind}/send {to} sends that sample to the operator's own inbox (stored in the outbox under the kind with a [Sample] subject; audit mail.sample) to check the words and the design in a real mail client. Together with the outbox (read, resend, delete), the transport test, the suppression list, each user's switches (/platform/admin/users/<id>), the report schedule (/platform/admin/reports) and the campaign switches (PATCH /api/v1/admin/campaigns/{campaign} {enabled}, on /platform/admin/growth), every message the platform sends can be seen, tested, held back or re-sent from the console.

Bounces and the suppression list. An address the transport rejects for good (SMTP 550-554, a 5.1.x status, "user unknown", "mailbox unavailable", an ACS Suppressed/Bounced status) is not retried: the row is abandoned at once with the reason and the address goes on the suppression list (email_suppressions, reason bounce), so no later message - of any category, newsletter included - is composed for it; such a skip is counted as suppressed in delivery. Delivery reports keep the list honest after acceptance: with OSR_PLATFORM_MAIL_EVENTS_TOKEN set, POST /api/v1/email/events/acs?token=… receives Azure Communication Services' Event Grid events (subscribe the endpoint to EmailDeliveryReportReceived and EmailEngagementTrackingReportReceived; the handshake is answered automatically, a wrong token is 401, and the endpoint answers 404 without the setting) and stamps each outbox row with delivery_status / delivery_at (delivered, bounced, spam, quarantined, expired, failed); a bounce, quarantine or spam report also suppresses the address, a soft failure (expired) for thirty days only. Operators see and edit the list at /platform/admin/mail - GET /api/v1/admin/mail/suppressions (q, limit), POST /api/v1/admin/mail/suppressions (email, reason, detail - for complaints and requests made out of band) and DELETE /api/v1/admin/mail/suppressions/{email} to allow an address again; GET /api/v1/admin/mail carries the list size by reason under suppressions, whether delivery reports are on under delivery_reports, and the last thirty days of reported outcomes under delivery.delivery_30d. GET /metrics exposes the same as osr_mail_delivered_total, osr_mail_failed_total, osr_mail_suppressed_total, osr_mail_reports_total, osr_mail_retrying, osr_mail_suppression_list and osr_mail_delivery_30d{status=…}.

The sender's address. OSR_PLATFORM_MAIL_POSTAL_ADDRESS (one line, e.g. Acme Ltd, 1 Main St, Springfield) is printed in the footer of every message, HTML and text, as CAN-SPAM and most national rules expect of commercial mail; the console's mail page shows what is configured (postal_address in GET /api/v1/admin/mail).

What people receive, and how they say no. Mail falls into categories. Security (address confirmation, password changes, sign-in and deletion notices, an API key created or revoked - to every owner, naming who did it and the key's prefix) and account (welcome, invitations, payment problems, plan changes - the owners hear when the plan moves up or down, with the new limits) are always sent. Usage reports, alerts (quota, budget and error-rate alerts that reach the owners when a workspace has no alert channel), support (ticket confirmations and answers), product news and getting started (one reminder with the next step when a workspace has not routed a request a day after signing up - sent once per workspace, to signups of the last seven days only, by the accounts service on the digest cadence) are optional: every such message carries a Manage e-mail preferences link and an Unsubscribe link in the footer plus the List-Unsubscribe / List-Unsubscribe-Post headers (RFC 8058), so Gmail, Outlook and Apple Mail show their own unsubscribe button (Graph accepts only x- custom headers, so on the Microsoft 365 transport the footer links do the job). The links carry a signed token (HMAC of the user id, valid for six months, signed with OSR_PLATFORM_SESSION_SECRET), so they work without signing in: /platform/email/preferences?token=… shows the switches (GET/PUT /api/v1/email/preferences), and a mail client's one-click POST goes to POST /api/v1/email/unsubscribe?token=…&category=…. Signed in, the same switches - categories, the newsletter subscription and, for owners and admins, the workspace's report schedule - sit under Account › E-mail (GET/PUT /api/v1/me/email-preferences). A message to someone who switched its category off is skipped before it is composed and counted as skipped in GET /api/v1/admin/mail; every change is written to the audit log (user.email_preferences, user.email_unsubscribed).

Everyone's own clock. Scheduled mail is timed by the recipient's time zone, not the server's. The sign-up form and the newsletter form send the browser's IANA zone (timezone on POST /api/v1/signup and POST /api/v1/newsletter/subscribe); a signed-in person who never named one gets the browser's zone the first time they open Account › E-mail (and is told so), and the same card lets anyone pick another zone or clear it (timezone on PUT /api/v1/me/email-preferences and on the token variant; schedule in the answer names the zone, its current offset and the hours below; operators can set it with PATCH /api/v1/admin/users/{id}). Until a zone is known, UTC applies. What follows the clock: usage digests go out at OSR_PLATFORM_REPORTS_HOUR (default 8) o'clock local over the recipient's own completed day, week or month; the newsletter issue at OSR_PLATFORM_NEWSLETTER_HOUR (default 9) local and the evening issue at OSR_PLATFORM_NEWSLETTER_EVENING_HOUR (default 18) local; the getting-started reminder, the upgrade offer, release notes and a trial-ending notice that is not urgent only between 08

Newsletter. POST /api/v1/newsletter/subscribe (the form on /newsletter - the landing page that says what the daily issue contains and how the address is treated, with the latest posts - and in the site footer of every marketing page; off with OSR_PLATFORM_NEWSLETTER=false, which hides both forms) stores the address, sends a branded confirmation and always answers 202; POST /api/v1/newsletter/confirm completes the double opt-in and sends the welcome issue with its unsubscribe link. Every newsletter message carries a one-click unsubscribe (POST /api/v1/newsletter/unsubscribe, token as JSON from the web page or as ?token= from a mail client's List-Unsubscribe POST). A signed-in user with a verified address joins straight from the preferences card (no confirmation round trip); an unverified one gets the usual confirmation. Registered people read it by default (OSR_PLATFORM_NEWSLETTER_SIGNUP_DEFAULT, on): the signup form says so next to the terms, POST /api/v1/signup files the address as a pending reader (source signup, the browser's time zone) without a second confirmation mail, and the account's own proof of the address - the confirmation link (POST /api/v1/auth/verify), a password reset, an accepted invitation, an identity provider that vouches for it or an operator marking it verified - confirms the reader with a consent record naming that statement. The welcome mail says the newsletter comes with the account and where to turn it off; the switch on Account › E-mail shows after e-mail confirmation until then. An address that unsubscribed is never re-enrolled, a reader who confirmed through the form keeps their row, and GET /api/v1/info reports newsletter_signup_default so the site can word the form accordingly. The blog asks readers in. A visitor who is not signed in and reads on (a few seconds on /blog or a post, or a third of the page) sees a sheet at the foot of the page - Keep reading with a free account - with the deployment's identity providers (sso from GET /api/v1/info, one click), an e-mail field that lands pre-filled on /platform/signup, and the sign-in link, each carrying the page to come back to (next, same-origin paths only, checked again by the API's safe_next); the text says the newsletter comes with the account when newsletter_signup_default is on. The post is never held back: Not now keeps the sheet away for a week, a subscription from any newsletter form in that browser or a signed-in browser hides it for good, and it waits for the cookie banner's answer. After signing in the welcome page's button reads Back to the post / Continue where you left off instead of Go to the dashboard. Operators manage subscribers and campaigns at /platform/admin/newsletter: add an address by hand, export CSV, write a campaign in plain text (blank lines separate paragraphs, ## starts a heading), preview the rendered email, send a test to one address, then send once to every active subscriber. The daily issue follows each reader's clock: the issue of a calendar day is composed from the unmailed posts when the first reader's zone reaches OSR_PLATFORM_NEWSLETTER_HOUR and every later zone receives that issue at its own hour - one issue per reader per day, each post mailed once (/platform/admin/content shows the hour; its Send now delivers to everyone who has not had today's issue whatever their local time, force composes a fresh issue and repeats the day). Morning and evening. Daily readers get a second issue once their clock passes OSR_PLATFORM_NEWSLETTER_EVENING_HOUR (default 18, local): it carries the posts written since the morning issue ("N new posts since this morning", subject evening: ...), is composed the same way - once, when the first zone reaches the hour - and is skipped on a day with nothing new. -1, or an hour not after the morning one, sends the morning issue only; the hour is editable at /platform/admin/settings. Weekly readers are not affected. How much gets written. However many feeds deliver, the writer turns at most OSR_PLATFORM_CONTENT_POSTS_PER_DAY (default 6, 0 = no limit) entries into posts in any 24 hours, newest first and spread evenly over the day's polls, so the blog grows steadily and both issues have news; the entries left over wait for a later poll.

Daily or weekly, by topic, and the reader's own page. Every newsletter message links to /newsletter/preferences?token=… (the same signed token as the unsubscribe link; GET/PUT /api/v1/newsletter/preferences) where a reader picks frequency daily (default) or weekly, sets or clears the timezone, chooses the topics to receive (the blog categories models, llm, agents, research, ai; an empty list or all of them means everything), sees what is stored about them, and can erase the record outright (DELETE /api/v1/newsletter/preferences - the row is gone, not marked). A reader who follows some topics receives the day's issue cut to those posts, with a subject and opener written for what they get; a day with nothing in their topics sends them no e-mail (the issue is still noted as theirs, so it is never repeated). POST /api/v1/newsletter/subscribe accepts topics too - the form on a blog category page subscribes to that category - and the blog's RSS takes the same narrowing: /blog/feed.xml?category=agents or ?tag=… is a channel of its own. Weekly readers get one digest on OSR_PLATFORM_NEWSLETTER_WEEKLY_DAY (0 = Monday … 6 = Sunday, default 0) at the newsletter hour of their own zone, composed from the week's posts ("This week: …"); a day without posts sends nothing. Consent is recorded with the confirmation: the moment, the IP address, the browser and the exact statement the reader agreed to (consent on the subscriber in the operator listing), so a double opt-in can be shown later. GET /api/v1/admin/content/issues reports for each issue how many distinct readers opened it and how many clicked (engagement.opens, engagement.clickers; the single-issue endpoint adds the clicks per link). The counting uses a one-pixel image (GET /api/v1/newsletter/o/{issue}/{reader}/{sig}.gif) and a click redirect (GET /api/v1/newsletter/c/{issue}/{reader}/{sig}?u=…, 302 to the post, 404 on a bad signature) keyed by a hash of the subscriber id, kept for 180 days and never joined to anything else; OSR_PLATFORM_NEWSLETTER_TRACKING=false turns both off and the issues go out with direct links. Every copy also carries View this issue in your browser above the card: /newsletter/issues/{issue} (GET /api/v1/newsletter/issues/{issue}) shows the same posts the mail carried, as the blog lists them, for a client that mangles the HTML - a public page keyed by the random issue id, never by the reader, so it is not tracked and holds nothing about anyone.

What an issue - and every other message - reads like. Everything the platform writes to a person is held to plain language: sentences under about twenty words, everyday words, one idea per sentence, one action per message, and a line that says why the person is receiving it. Readability is measured (Flesch-Kincaid grade level, longest sentence, a short jargon list) and every release is gated so no customer-facing message reads above grade 9 - roughly an eighth grader on the first pass. The newsletter issue opens with how many posts there are and how long they take to read; each card carries the title, one line on what happened, up to three key points (highlights on the post: three or four concrete facts from the source - a number, a model name, a date, a limit - so a reader who stops there knows the news), a Why it matters line the writer produces (why on the post, also on GET /api/v1/blog/{slug}), the reading time, the source and the link; the issue ends by asking the reader what they want more of (replies reach the operators). The blog page shows the key points and Why it matters above the article; operators edit both (PATCH /api/v1/admin/content/posts/{post_id}, why and highlights). The writer's house skill asks for the same plain language, the key points, a ## Why it matters section and - when the source allows it - a ## What to do section in every post, and records the draft's readability grade next to the routing decision. Every post is a long-form article of the blog's own, not a rewritten excerpt: the writer reads the source page (up to OSR_PLATFORM_CONTENT_ARTICLE_SOURCE_CHARS characters, default 12,000), asks the model for the brief first - title, summary, key points, why it matters, an outline of six to nine sections, tags and keywords, one small JSON object - then writes the article from that outline in plain Markdown, aiming at OSR_PLATFORM_CONTENT_ARTICLE_WORDS words (default 2,000) and never under OSR_PLATFORM_CONTENT_ARTICLE_MIN_WORDS (default 1,500): a draft that stops short is continued once from where it ended, covering only the sections still missing. Every post ends its argument with a ## How OpenSmartRoute helps section, in front of What to do: what a team that routes its requests through the router gains from this particular story and what it would set up - written from a fixed list of product facts and the facts of the story, nothing else; on the site a card under that section takes the reader to the playground, the catalogue and the set-up guide. The brief plans it, the article pass writes it, and a post whose article skipped it gets the section in one more metered pass (passes shows product); a post that fell back to the excerpt or the brief's own text carries a short hand-written version angled by the story's tags (routing.product_section says model or template); posts written before the section existed receive it on the next poll, a few at a time, written by the model for each story - POST /api/v1/admin/content/posts/product-sections (Add product sections on the console's Posts tab) does up to a hundred at once, model: false uses the hand-written text only. Both passes are metered requests of the editorial workspace and both are graded for the learners; the post's routing carries them (passes) and the measured length (length: words, target, minimum, whether it came in short). When the article pass cannot produce at least 400 words the post keeps the brief's own text or the attributed excerpt and says so (reason). The three lengths are operator-editable at /platform/admin/settings and take effect on the next post. No picture, no post: with OSR_PLATFORM_CONTENT_REQUIRE_IMAGE (default true) a story becomes a post only when it has a real picture that answers as an image - the feed's own, or what the article page advertises (og:image, twitter:image, <link rel="image_src">, the JSON-LD image, the first photo inside <article>; logos and share banners never count) - otherwise the entry is marked no-image and left alone, and the issue only ever carries posts with a picture, so nobody opens a digest with a blank or broken image. POST /api/v1/admin/content/posts/images keeps probing older posts for a picture (Fetch missing images on the console's Posts tab). The platform keeps its own copy of every post's picture: when a post is written its image is downloaded (16 MB cap; sources that answer application/octet-stream are sniffed for JPEG, PNG, GIF and WebP), brought to web size - no wider than 1,600 pixels, encoded as WebP, transparency kept, animated GIFs left as they are - and stored; each poll copies up to 25 more of the older posts, newest first, and POST /api/v1/admin/content/posts/images/store copies up to 200 on demand (Copy pictures on the Posts tab). The site serves the copy at /blog/{slug}/image (GET /api/v1/blog/{slug}/image: a week of cache, ETag; the web app keeps each picture for a day so the API answers once per replica, not once per reader), so pages, the RSS enclosure and the newsletter never embed a third-party host and a source that later disappears does not leave a broken picture; until the copy exists the address redirects to the source, and a source that cannot be copied is tried again after a week. The post's image.source keeps the original address. Registering a feed (POST /api/v1/admin/content/feeds) looks at the URL first: a feed is named after its own title unless a name is given; a blog's home page yields the feed it advertises in its <head> (<link rel="alternate">) or at one of the conventional paths (/feed, /feed.xml, /rss.xml, /rss, /atom.xml, /index.xml); a page that is reachable but has no feed is refused with the reason; a URL that does not answer is registered with the error on its row and tried again on the next refresh. POST /api/v1/admin/content/feeds/preview is the same look without registering anything - the title, the site, the newest entries and whether the feed is already followed. POST /api/v1/admin/content/feeds/seed adds every curated default feed the deployment does not follow yet (Add the curated feeds on the Feeds tab); a release that adds curated feeds offers them to a running deployment once on its next start (a feed an operator deleted is never offered again). The curated list covers about seventy-five sources in four groups - the model vendors and platforms (OpenAI, Google, Hugging Face, Microsoft Foundry, NVIDIA, AWS, Databricks, Together, Replicate, Qwen, Mistral, Ollama, PyTorch, Cloudflare, Weights & Biases, Weaviate, Qdrant, Lightning AI, GitHub), the research labs (DeepMind, Google Research, Microsoft Research, Meta, Apple, Amazon Science, BAIR, EleutherAI, Ai2, MIT, arXiv cs.AI / cs.CL / cs.LG), practitioners and independent analysis (Simon Willison, Sebastian Raschka, Ethan Mollick, Lilian Weng, Hamel Husain, Eugene Yan, Andrej Karpathy, Chip Huyen, Jay Alammar, Phil Schmid, Sebastian Ruder, Interconnects, Latent Space, The Gradient, AI Snake Oil, SemiAnalysis, Last Week in AI, Towards Data Science, KDnuggets) and the press (Ars Technica, The Verge, Wired, MIT Technology Review, TechCrunch, The Decoder, MarkTechPost, InfoQ, SiliconANGLE, ZDNET, The Register, Unite.AI, Stratechery); every URL is verified to fetch and parse, and the topic filter keeps the mixed sources to posts about models and agents.

How a post credits the router. Under every article's byline the post page carries the decision that wrote it, in three tabs. Decision names the target and the model, the confidence, the skill layered on the prompt, the targets also considered or fallen back from, and the router's own reason. Waterfall draws one bar per pass

• the brief, the article, and the continuation when there was one - scaled to the slowest, with what each pass cost, how long it took and how many words it had produced by then, so a reader sees where the time and the money went. Evaluation shows the grade each pass was credited with - the same outcome the learners were taught, now kept on the post itself as grade rather than only on the learner - together with the finished text's Flesch-Kincaid reading level (readability_grade, 9 reads like a ninth-grade textbook) and its length against the minimum, which says plainly when a post came up short. A tab is only offered when the post carries the data behind it, so a post written before a measurement existed simply shows fewer. Every panel sits in the HTML whether or not it is the open tab, so a reader without JavaScript and a search engine still see the facts, and the tabs move with the arrow keys. A post that no model wrote says so in one line instead.

Lifecycle campaigns. Four messages the platform sends by itself (campaigns.py; the accounts service runs the sweep on the digest cadence, the Lifecycle campaigns card on /platform/admin/growth shows what is due, what went out and runs it now - GET /api/v1/admin/campaigns, POST /api/v1/admin/campaigns/run with an optional campaigns: [...] list). Each is one message per workspace, claimed in campaign_sends before it is composed, so replicas never double-send and nobody is asked twice:

The apps report in. The first batch of activity an app (browser-extension, desktop) records for a workspace sends Your browser extension / desktop app is connected (kind app_connected, account: always sent) to the person who connected it - the signed-in user, else the owners: what it does, that the workspace records counts only, the Apps page, the app's documentation, and what to do if they did not install it. Once per app and workspace; audit app.connected.

Every campaign has a switch: PATCH /api/v1/admin/campaigns/{campaign} {enabled: false} (the On column of the console's card) stops the sweep and Run now from sending it deployment-wide - the run reports disabled for it and release_news keeps recording the running version, so switching it back on never announces a backlog. On by default; audit campaign.disabled / campaign.enabled.

The release announcement does not wait for the digest cadence: the accounts service checks every OSR_PLATFORM_RELEASE_NEWS_INTERVAL_S (default 600 s, 0 = digest cadence and the console only) whether the running version differs from the last one announced, so the mail follows a deploy within minutes; the autonomy board lists it as the release-news job (GET /api/v1/admin/autonomy, run it with POST /api/v1/admin/autonomy/{job}/run and job=release-news).

Release notes everywhere else. The same changelog feeds GET /api/v1/releases (public: every released version newest first with date, sections, highlights and the changelog anchor, plus current - the version this deployment runs; ?limit=) and GET /api/v1/releases/{version}. Signed-in people see the releases they have not read yet on the dashboard overview (What's new, one card per visit until they press Got it): GET /api/v1/me/whats-new lists the releases newer than the user's acknowledged version up to the running one (a first-time reader sees the running release only, never history), POST /api/v1/me/whats-new/seen records the running version on the user, so the card stays away on every device. The footer's version badge asks the running API for platform_version after the page loads, so a cached page or a docs page built before the last deploy still names the version that is live, and links to the changelog.

Recipients are always people who signed up here. The platform never buys, rents, scrapes or guesses addresses and sends nothing to anyone who has not either created an account (transactional and opted-in product mail) or confirmed a newsletter subscription (the daily issue): that is what keeps deliverability - and the platform - on the right side of the law. Every run is audited (campaign.trial_ending, campaign.upgrade_offer, campaign.release_news, campaign.apps_reminder, campaigns.run from the console) and the messages appear in the outbox with their kind.

Scheduled reports. reports.py runs inside the API: every OSR_PLATFORM_REPORTS_INTERVAL_S (default hourly, 0 = on demand only) it checks, for each owner of each workspace, whether a daily (yesterday), weekly (Monday) or monthly (1st) window has completed on that owner's clock and it is past OSR_PLATFORM_REPORTS_HOUR there, and sends them the workspace's value report for that local window; the operators in OSR_PLATFORM_REPORT_RECIPIENTS get the deployment-wide digest for every period (UTC). The workspace report leads with what routing saved - the executed requests' provider cost against the baseline model's list price for the same tokens, the same formula as GET /api/v1/savings and the Impact page; a workspace that only asked for decisions sees N requests routed and how to get to the saving instead - then the figures each captioned against the previous window (requests routed, answered, provider cost, tokens), where the requests came from (browser extension, desktop app, CLI, API / SDK / playground - the X-OSR-App header), the models that answered, what the guard protected in the window (personal data redacted, requests blocked, answers withheld, boundary refusals, injection attempts; tracing off when there are no trace events), the workspace's activity (keys created, rotated or revoked, members joining and leaving, providers connected, policy and plan changes, support tickets) and a button to the Impact report. The subject carries the headline (weekly report for Acme · $142.60 saved). GET /api/v1/workspace/reports returns the same composition as preview (usage, previous, savings, guard, by_app, activity) for the caller's own zone (timezone, hour). Owners and admins pick the schedule (daily, weekly - the default -, monthly or off) with PUT /api/v1/workspace/reports or under Account › E-mail; each owner can also switch the Usage reports category off personally. A window is recorded per recipient in report_deliveries and per workspace in report_runs and never sent twice, so several API replicas stay idempotent. /platform/admin/reports shows the scheduler, past digests and sends one now (POST /api/v1/admin/reports/send, POST /api/v1/admin/reports/tick).

New registrations. Every time somebody registers - through POST /api/v1/signup or the first sign-in with an identity provider, which is where an account is created - the operators in OSR_PLATFORM_SIGNUP_RECIPIENTS get one message: who registered, the workspace, how they signed in, whether the provider vouched for the address, the acquisition source and campaign when the signup carried them, and a link to that person's page in the console. The list falls back to OSR_PLATFORM_SUPPORT_RECIPIENTS and then OSR_PLATFORM_REPORT_RECIPIENTS; with none of them set nothing is sent. A later sign-in by somebody who already has an account is not a registration and is quiet. OSR_PLATFORM_NOTIFY_SIGNUPS=false switches the message off for a deployment that would rather read the signup count in the daily digest and the console's activation funnel.

Support tickets. A user raises a concern under /platform/dashboard/support or with POST /api/v1/support/tickets (subject, body, category question / bug / billing / account / feature / security / other, priority low / normal / high / urgent, optional request_id of the routed request it is about). The answer - and a confirmation e-mail to the requester - carry the ticket reference, OSR-00042, which identifies the ticket everywhere (GET /api/v1/support/tickets/{key} takes the reference or the id). Operators are told at once: an e-mail to OSR_PLATFORM_SUPPORT_RECIPIENTS (falls back to OSR_PLATFORM_REPORT_RECIPIENTS) and an entry in the console inbox. The conversation lives on the ticket: POST /api/v1/support/tickets/{key}/messages adds the requester's reply (an answered or resolved ticket goes back to open), POST /api/v1/support/tickets/{key}/close ends it, and GET /api/v1/support/tickets/{key}/stream follows it live (server-sent events: the ticket with its conversation now and again on every change, a keep-alive every 15 s, a new connection every ten minutes - the ticket page uses it, so an answer appears without a refresh); GET /api/v1/support/tickets lists the workspace's tickets with counts per status (open = waiting for support, answered = waiting for the requester, resolved, closed). Every public answer from the support team and every status change is e-mailed to the requester with the reference in the subject and the message quoted, and appears in the workspace inbox (rule support); internal notes never leave the console. A workspace may have 25 tickets waiting at a time; every step is in the audit log (support.ticket_opened, support.ticket_answered, support.ticket_replied, support.ticket_updated, support.ticket_closed, support.note_added).

Product feedback. Every internal page has a Send feedback button in the dashboard header (and the page /platform/dashboard/feedback): a bug, a feature idea, an improvement, praise or anything else about the page the person is on. The browser attaches what an operator needs to reproduce it and the person cannot easily describe - the page and URL (query strings dropped), the browser and viewport, the platform version and plan, the last console errors and warnings and the last API calls with their status and X-Request-Id (never a request body, never a credential; anything that looks like a key or a bearer token is redacted in the browser and again on the API) - and up to three screenshots: captured from the screen (getDisplayMedia, the browser asks what to share), pasted or uploaded, at most 2 MB each, accepted by their bytes (PNG, JPEG, WebP, GIF). The item gets a reference, FB-00012, the person receives a confirmation e-mail that quotes it, and operators are told (e-mail to OSR_PLATFORM_SUPPORT_RECIPIENTS, console inbox). POST /api/v1/product-feedback (kind bug / idea / improvement / praise / other, title, body, page, url, optional request_id, diagnostics, attachments as data URLs) does the same from any client; GET /api/v1/product-feedback lists the workspace's items with counts per status, GET /api/v1/product-feedback/{key} returns one with its conversation, diagnostics and screenshot list, POST /api/v1/product-feedback/{key}/comments adds to it and GET /api/v1/product-feedback/{key}/attachments/{attachment_id} serves a screenshot (workspace-scoped, nosniff, sandboxed). Operators triage in the console (/platform/admin/feedback): status new -> reviewing -> planned -> in_progress -> shipped, or declined; an answer to the person; internal notes; assignment. Every status change and every public answer is e-mailed to the person with the reference in the subject and the note quoted (category feedback, switchable under Account › E-mail) and appears in the workspace inbox (rule feedback), so nobody has to come back and check; a shipped or declined item takes no more comments - new feedback that mentions the reference does. A workspace may have 50 open items at a time; every step is in the audit log (feedback.submitted, feedback.commented, feedback.answered, feedback.updated, feedback.note_added).

Membership and access notices. People are told about what happens to their account, not only the owners: whoever sent an invitation hears when it is accepted (member_joined), a member whose role changes or who is removed from a workspace receives a note saying who did it (member_role_changed, member_removed; a removed member's sessions on that workspace end and their personal workspace is untouched), and an operator who signs a person out everywhere, disables or re-enables their sign-in triggers a security / account notice to that person (sessions_revoked, account_disabled, account_enabled). These are account / security mail and always go out. The owners also hear when an API key is rotated (api_key_rotated: the old secret stopped working at once), as they do when one is created or revoked, and when what an active key may do changes (api_key_changed: the new scopes, budgets, rate or allowed targets in plain words, the secret untouched; a rename alone is not news). The owners hear when a member leaves by themselves (member_left: who, and that their keys and work stay), and every member - the owner who did it included - hears when an owner deletes the organization (workspace_deleted: the keys stopped working, a paid plan was cancelled, everyone was moved to their own personal workspace, and whom to contact if nobody meant it). The workspace inbox mirrors the roster and the keys for everyone in the dashboard: workspace.member_left, workspace.transferred (ownership handed over), workspace.key_rotated, workspace.key_scoped, workspace.key_limited beside the existing key_created, key_revoked, member_role and member_removed notices; and a request the router handed to a human queue is a workspace.handoff_pending notice (warning) until a person answers it (workspace.handoff_resolved), so a team sees a waiting hand-off without subscribing a channel. When no channel of the workspace listens for handoff.* events, the owners are also mailed (handoff_waiting, category alerts: the request's first lines, why the router chose a person, how many wait, the link to answer) - at most once an hour per person, so a busy queue never floods a mailbox; a channel subscribed to hand-off events keeps the mailbox quiet.

Sign-in notices. The account's owner hears about the things that decide who can get in, always (security):

Every browser session records the client address it was opened from (ip in GET /api/v1/auth/sessions, shown under the device on the account page and in the operator's session table) and, when the deployment resolves places, where that was (location; see Sign-in locations and alerts); older sessions have none. The account page's history (GET /api/v1/me/audit) names each event in plain words - signed in from a new device, signed in from a new country, sign-in method added - with the device, provider, place and address behind it.

Command line sign-in

Install the CLI (curl -LsSf https://opensmartroute.ai/install.sh | sh, or irm https://opensmartroute.ai/install.ps1 | iex on Windows) and run osr login. The CLI shows an eight-character code and opens /platform/cli/authorize; sign in there, check that the code and computer name match, choose the key name and approve. The platform mints a new workspace API key for that machine and the CLI stores it in ~/.config/opensmartroute/credentials.json. This is the OAuth 2.0 device authorization grant (RFC 8628) and works the same for the community and enterprise editions:

Only approve codes from a terminal you started yourself. Keys minted this way appear on /platform/dashboard/keys with the chosen name and can be revoked like any other. Pasting a key works too (osr login --token osr_live_..., or --with-token from stdin in CI); osr whoami shows the workspace, plan and edition behind the stored credential, osr token create|list|revoke manage keys and osr mcp --remote bridges an IDE to this workspace's MCP server without copying the key around.

2. Route a request

POST /api/v1/route returns a decision: the chosen target, the confidence, ranked alternatives and the signals extracted from the request. The target is not called unless execute is set. target.provider is the provider/model this workspace would run it on and target.model the reference vendor/model id behind it (both null for agents, skills, tools and humans), so a caller that executes elsewhere - LiteLLM, its own OpenAI client - can hand the chosen model straight to it and report the outcome with /api/v1/feedback.

curl -s "$OSR_URL/api/v1/route" \
  -H "Authorization: Bearer $OSR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"text": "Prove that sqrt(2) is irrational.", "top_k": 3}'

{
  "request_id": "7f3c...",
  "target": {"id": "llm-frontier", "kind": "llm", "name": "Frontier model", "provider": "azure/gpt-4.1", "model": "openai/gpt-4.1"},
  "confidence": 0.81,
  "alternatives": [{"id": "llm-mid", "kind": "llm", "utility": 0.62}],
  "signals": {"complexity": 0.74, "domains": ["math"], "reasoning_need": 0.9, "contains_pii": false, "...": "..."},
  "ranked": [{"id": "llm-frontier", "utility": 0.79, "quality_estimate": 0.91, "breakdown": {"...": "..."}}],
  "policy_rejections": {},
  "explanation": "llm-frontier: high reasoning need, math domain, quality weight dominates ...",
  "executable": true,
  "elapsed_ms": 1.9
}

Request fields:

Constraints (constraints object):

Your marketplace installs are in the pool. Every agent, skill, persona, tool, subagent or command your workspace installed from the marketplace is added to the routing pool of your requests: a plan picks an installed skill or persona for its slot when it fits better than the catalogue's, and an installed primary competes for the answer itself. An installed agent or tool whose manifest declares a public https endpoint (a2a, http or mcp) is bound to it and runs when it wins an executed request; one without a callable endpoint is left out of executed requests so the decision never lands on something that cannot run (skills and personas always can - a plan runs their instructions through its model). GET /api/v1/targets lists them with source: marketplace next to the catalogue's source: catalogue, executable says whether the platform can run each one (metadata.endpoint_unbound gives the reason when not); another workspace never sees them.

Quote before routing

POST /api/v1/estimate takes the same body and returns the price of each candidate without calling anything: targets[] with cost_usd, input_tokens, output_tokens, vendor and model, plus the router's recommended pick and the cheapest, best_quality and fastest candidates. output_tokens overrides the assumed reply length; monthly_requests adds a monthly projection (recommended_usd, cheapest_usd, dearest_usd, savings_usd). Without a key it is rate limited per client and ignores tenant rules; with a key it is metered as estimate and applies your plan and tenants. GET /api/v1/models/recommended returns the router's picks per use case from live quotes. Details, the SDK form and the /estimate page: Cost estimates and the MCP server.

3. Execute the chosen target

With "execute": true the platform runs the chosen target (or plan) with the configured providers and returns the output, the steps that ran and the metered cost:

curl -s "$OSR_URL/api/v1/route" -H "Authorization: Bearer $OSR_API_KEY" -H "Content-Type: application/json" \
  -d '{"text": "Summarise this contract clause for a non-lawyer: ...", "execute": true, "objective": {"cost": 0.4}}'

The response adds result with text, target_id, steps[] (role, target_id, ok, latency_ms, note - offered / called for the tools the plan gave the model, -N tokens for the compaction step), tokens, cost_usd, tokens_saved (estimated prompt tokens the compaction removed) and fallback_from (the target ids whose handler failed before another executable target answered - normally []). With execute: true the decision is made among the targets that can actually run here: models and tools without a handler are left out of the pool, while skills, personas and agents stay in because a plan runs them through its model slot. Without execute the decision is unrestricted and executable on each target tells you what would run.

Built-in tools

Three kinds of tool target execute without a chat model. The catalogue's calculator (tool-calculator, or any tool with metadata.builtin: calculator) is answered by the platform itself, deterministically and without a provider: arithmetic in symbols or words (What is 17% of 2,450?, 12 * 37 plus 5 percent, sqrt(2) ^ 2) and unit conversion for length, mass, volume, time, data, speed and temperature (Convert 5 miles to kilometers.). It costs no tokens and reports builtin/calculator as its provider (built-in tools are not listed among the upstream providers). The web search tool (tool-web-search, or metadata.builtin: web-search) searches the live web for a lookup, the latest news or a price - DuckDuckGo and Wikipedia without a key, a self-hosted SearXNG instance when SEARXNG_URL is set, Brave with BRAVE_API_KEY - and answers the top results as Markdown links with their snippets and the engines used (builtin/web-search, no tokens); the instruction is stripped from the query (Search the web for the latest EU AI Act news and cite sources searches for the latest EU AI Act news), hits whose title or snippet reads like an injected instruction are dropped, and a request with no result fails over to another executable target like any handler error. Image tools - a tool whose capabilities name the image action or modality, such as tool-image-gen - execute once the console maps them to a provider that serves POST /images/generations (OpenAI, Azure OpenAI, compatible gateways); the answer is the picture as Markdown (![prompt](url), or an inline data: URL when the provider returns base64), so chat clients and the playground render it, and each image counts as one completion token against the target's declared cost. A request that asks for a picture on a deployment without an image provider is routed to another executable target rather than failing with "has no handler".

Model providers

A target executes when it is mapped to a model on an OpenAI-compatible endpoint - OpenAI, Azure OpenAI, OpenRouter, Mistral, Groq, Together, Fireworks, DeepSeek, a local Ollama, vLLM or LiteLLM, anything that speaks /v1/chat/completions - or on Anthropic's own API: a provider whose kind is anthropic (or whose base URL is on anthropic.com) is spoken natively over the Messages API (x-api-key, anthropic-version, POST /v1/messages; tools, tool results, images, streaming and extended thinking translated both ways), so routing to Claude needs no OpenAI-compatible gateway in between. Two sources define the mapping and are merged (the console wins on the same provider name or target):

• the file the deployment mounts, OSR_PLATFORM_PROVIDERS (YAML/JSON path or inline JSON) with providers.<name> (base_url, api_key or api_key_env, optional api_key_header, extra_headers, timeout_s, max_retries, concurrency and queue_timeout_s - see Capacity below) and models.<target_id> (provider, model, optional system_prompt and request defaults). {"kind": "mock"} is a provider that needs no endpoint and no key: it answers deterministically in-process (the prompt echoed through reply, a template over {model}, {prompt} and the call count; JSON when response_format asks; a tool call when the prompt names a tool), streams word by word, embeds with a stable vector per text and fails on cue (fail_when - a substring of the prompt -, fail_every, latency_ms) - the provider for tests, demos and CI pipelines, metered like any other. At start-up the file is imported into the console additively: providers the store does not know yet and targets that are not mapped yet are added, rows an operator edited are left alone. A provider the file created and later stopped naming is disabled (its offers and mappings stay, and it is enabled again the day the file names it) - a deployment that drops a cloud provider takes its credential with it, and left enabled every call to it would fail with authentication failed. Providers an operator created are never touched.
• the operator console, /platform/admin/providers: add an endpoint from a preset, store the key inline or name the environment variable that holds it, Check it (GET <base_url>/models with its credentials records reachability, latency and the upstream model ids, which the mapping form then offers), map targets to upstream models, disable or delete. Every write re-binds the handlers in the API at once, and every other API replica re-reads the store within OSR_PLATFORM_PROVIDERS_RELOAD_S (30 s) - no restart, no redeploy. The providers owner also runs that check on its own every OSR_PLATFORM_PROVIDERS_CHECK_S (300 s; 0 = only on click) for every enabled provider - one replica per interval - so the health the public catalogue, the workspace Providers page and the alerts show is never older than that.

The platform's providers are what every workspace executes on by default: a fresh workspace needs nothing connected - its Providers page lists the included models under Included with the platform (per provider, with health and the target -> upstream model pairs), GET /api/v1/workspace/providers returns them as platform_providers, and the getting-started step Models ready is ticked. A mapping's request defaults travel with every call: the deployment's Ollama mappings carry reasoning_effort: none on every chat target except the reasoning-by-design ones (research-agent, gpt-oss), because a thinking model asked for a short answer otherwise spends the caller's whole max_tokens thinking and returns an empty message - a request that sends its own reasoning_effort overrides the default.

GET /api/v1/providers is the public, OpenRouter-style view of what a deployment executes on, rendered by the site at /providers: one row per executable target with provider, model, the recognised reference model, input_usd_per_1m / output_usd_per_1m (from the reference catalogue when the upstream model is known, else the target's declared cost), context window, tool and reasoning support, circuit-breaker health and seven-day traffic; plus each provider's host, kind, health and model count. Credentials never appear: the console shows only whether a key is set and its last four characters (api_key_set, api_key_hint). Keys stored through the console live in the platform database next to the organization SSO secrets; prefer api_key_env when the host injects secrets. Endpoints: GET|POST /api/v1/admin/providers, GET|PATCH|DELETE /api/v1/admin/providers/{id}, POST /api/v1/admin/providers/{id}/check, PUT|DELETE /api/v1/admin/providers/models/{target_id}. Targets without a mapping still route (decisions, plans, traces, quotes) and report executable: false; GET /api/v1/providers also counts the builtin tools that run without a provider.

Bring your own provider (workspace keys)

Every workspace (feature byok, on all plans of the default catalogue) connects its own endpoints and keys at /platform/dashboard/providers - Azure OpenAI / Azure AI Foundry, Amazon Bedrock (bedrock-runtime.<region>. amazonaws.com/openai/v1 with a Bedrock API key), Google Gemini (generativelanguage.googleapis.com/v1beta/openai) or Vertex AI (<region>-aiplatform.googleapis.com/v1/projects/<project>/locations/<region>/endpoints/openapi), OpenAI, Anthropic (spoken natively over its Messages API - no gateway needed), Mistral, Groq, xAI, Cohere, OpenRouter, an in-house Ollama, vLLM or LiteLLM gateway, or any other OpenAI-compatible URL - and maps catalogue targets to models on them (Azure: the deployment name; Bedrock: the model id). From then on every request of that workspace that executes - POST /api/v1/route with execute: true, /v1/chat/completions, the MCP ask tool - runs the chosen target on the workspace's endpoint with the workspace's credentials. The router, the trace, quotas, budgets and the learners are unchanged; only where the call goes differs, so the usage lands on the workspace's own cloud bill and prompts stay inside the data boundary it chose. Rules: a workspace mapping wins over the platform's for the same target; a target only the workspace mapped is executable for that workspace alone (GET /api/v1/targets and GET /v1/models are per workspace); unmapped targets keep the platform's providers, so a workspace can bring one Azure deployment and leave the rest to the platform. Disabling a provider withdraws its mappings; deleting it removes them. Responses name the answering endpoint: opensmartroute.provider on chat completions and result.provider on /api/v1/route (<provider>/<model>).

Credentials are stored AES-256-GCM encrypted under OSR_PLATFORM_SECRET_KEY (falling back to the session secret; set it explicitly on multi-replica deployments), bound to the workspace, never returned by the API (api_key_set, api_key_hint only) and never resolved from the API container's environment - a workspace cannot borrow the deployment's keys by leaving the field empty. A key may also be given by reference, so the secret itself never enters the platform database: keyvault://<vault>/<secret>[/<version>] (or the secret's https://<vault>.vault.azure.net/secrets/... URL) is read from Azure Key Vault with the API container's managed identity when the client is built (the identity needs the Key Vault Secrets User role; the value is cached for five minutes, so a rotation in the vault reaches the router without a redeploy), env://NAME reads a variable of the API container (the operator's store of last resort), and identity://<resource> needs no secret at all: the container's managed identity mints an access token for the resource (https://cognitiveservices.azure.com for Azure OpenAI and Azure AI Foundry, https://ai.azure.com) and sends it as the bearer credential - keyless, rotation-free; give the identity the Cognitive Services OpenAI User role on the resource (the console's Azure OpenAI / Foundry (managed identity) preset fills the reference in, and Hugging Face Inference Providers is a preset too). The platform's own file providers may use every form in api_key. A reference is not secret: the console shows it whole (api_key_hint, api_key_reference: true), and a vault that refuses the read is what Check reports. The endpoint must be a public host: base URLs on loopback, private, link-local or reserved addresses, bare hostnames and .internal / .local names are refused (400 base_url must be a public endpoint), at save time and again before Check, so a workspace cannot aim the platform's network position at its database, Redis, the model servers or the cloud metadata service; credentials embedded in the URL are refused too. OSR_PLATFORM_BYOK_ALLOW_PRIVATE=true lifts the host rule on a self-hosted deployment whose workspaces are trusted and whose providers live on the internal network (the development compose stack sets it). Members read the page; adding, changing, rotating (api_key) or removing (clear_api_key) a key, mapping and deleting need the admin role (API keys act as the workspace). Every change is in the workspace audit log (provider.created, provider.updated with key_rotated, provider.mapped, provider.unmapped, provider.deleted - never the key). Check calls the endpoint's /models with the stored key and offers the returned ids on the provider card and in the mapping dialog; the dashboard runs it right after connecting or editing a provider unless told not to. Endpoints: GET|POST /api/v1/workspace/providers, GET|PATCH|DELETE /api/v1/workspace/providers/{id}, POST /api/v1/workspace/providers/{id}/check, PUT|DELETE /api/v1/workspace/providers/models/{target_id} (provider_id, model, optional system_prompt and request defaults). Up to 20 providers per workspace; a change is in force on the next request on every API replica and service (the workspace's revision is compared per request, so a removed provider is never used again).

Self-hosted models

The deployment runs its own model servers next to the cloud providers: the model server (the OpenSmartRoute model host - Ollama and OpenAI APIs over llama.cpp, CPU or CUDA; the Ollama image remains a supported alternative, ollamaImage / OSR_OLLAMA_IMAGE picks) with a curated set of open weights on a shared file share (ollamaEnabled, on by default on Azure; the compose stacks run the same set, pulled by OSR_OLLAMA_MODELS). Every model is stored and mapped under its ollama.com tag; where that tag's blob is Ollama's own conversion that a standard llama.cpp cannot read (Qwen3.5, gpt-oss, GLM-4.7-Flash, GLM-OCR, TranslateGemma), ollamaModelSources names the stock GGUF on Hugging Face the model server pulls instead (qwen3.5:4b=hf.co/unsloth/Qwen3.5-4B-GGUF:Q4_K_M), so the whole set is hosted by the OpenSmartRoute model server without the Ollama image; ollamaModelImages stays for a per-model image exception. On CPU profiles every model runs as its own always-warm model server (one model pinned in memory per server), so a request never waits for another target's weights to be re-read from the share - on a shared CPU server that eviction churn, not the inference, is the latency. The set is the strongest open model per job - by the published benchmarks, not by name - that answers a short request in a few seconds on its hardware, so the on-prem and specialist targets of the catalogue execute without data leaving the deployment:

The frontier open weights no single node hosts - Kimi K2.7 Code, DeepSeek-V4-Flash, gpt-oss-120b - answer through Ollama Cloud (https://ollama.com/v1, the same OpenAI-compatible surface) when ollamaCloudApiKey (OSR_OLLAMA_CLOUD_API_KEY, an API key from ollama.com/settings/keys) is set: ollamaCloudTargets maps coding-agent, research-agent, gpt-oss, llm-mid and llm-frontier to them and replaces the local mapping of those targets; without the key they stay on the local models. Any other ollama.com model is one console mapping away (provider ollama-cloud).

ollamaWorkloadProfile picks the hardware. Consumption (the default) serves the small set: one always-warm 4 vCPU / 8 GiB app per model, each answering one generation at a time and scaling out to ollamaMaxReplicas (default 4, OSR_OLLAMA_MAX_REPLICAS) on bursts. A dedicated size (E4, E8, ... - E-series memory fits the 19 GB GLM-4.7-Flash) adds the ollamaLargeTargets models, one node per model: only models that earn a whole node are in that set - the mixture-of-experts coder and gpt-oss:20b, whose ~3B active parameters decode at CPU speed once the weights sit in RAM, and the visible-reasoning deepseek-r1:8b, whose answers are slow by design. A node costs the same whether it holds one model or four, and Ollama runs each loaded model in its own runner, so ollamaNodeGroups packs models that fit together onto one always-warm server per node with every model resident and preloaded (OLLAMA_MAX_LOADED_MODELS = the group size): a short OCR or embedding call runs beside a long reasoning generation instead of waiting for it, and each grouped model keeps its own provider name (ollama-<model>) so console mappings and health rows do not move. What limits a group is memory - the resident working sets must fit the node (about 22 GiB of an E4's 28; the per-model WorkingSetBytes metric is the figure to add up). The default keeps the two 22 GiB models (glm-4.7-flash, gpt-oss:20b) on nodes of their own and packs deepseek-r1:8b with translategemma:4b, glm-ocr and qwen3-embedding:4b; qwen3.5:4b, the busiest model, stays a Consumption replica that scales out on bursts - three nodes and one replica instead of four nodes and four replicas, every model as warm as before. OSR_OLLAMA_NODE_GROUPS (a JSON array of {"name": ..., "models": [...]}) overrides the grouping; a model named by no group keeps the per-model layout. A model server that drops out of the template when a model moves into a group is retired by the post-provision hook (OSR_OLLAMA_PRUNE=false keeps it). Dense 9-12B chat upgrades are deliberately not served on CPU (a dense model that size cannot answer a short request in seconds there); they return with a serverless GPU (Consumption-GPU-NC8as-T4, Consumption-GPU-NC24-A100; needs GPU quota in a region that offers it), where one shared server keeps several models resident and decodes 8 / 32 requests at once per replica (the T4 at 8K context with the KV cache in q8_0, the A100 the whole set at 16K), scaling out to ollamaMaxReplicas; the replicas share the model store and pull weights under a lock. ollamaTargets / ollamaLargeTargets change the mapping; the model tags are pulled at start-up and kept on the share, and GET /api/v1/providers shows which targets the servers answer for. Azure OpenAI is opt-in: azureOpenAiEnabled (OSR_AZURE_OPENAI_ENABLED, default false) creates the account and the openAiModels deployments and hands the azure provider every target the Ollama sets leave unmapped - which also gives image generation (tool-image-gen → gpt-image-2) and, as the fallback for transcribe-small, gpt-transcribe a home; off, those two wait for a console mapping (the self-hosted speech server covers transcription when voiceEnabled). Ollama serves text, chat, code, vision (image input) and embeddings.

Sizing a model server. The numbers behind those profiles are arithmetic, not folklore, and the SDK does it for you (opensmartroute.inference, osr capacity). A model in memory is its weights (parameters × bits per weight: a Q4_K_M file is about 4.85 bits, Q8_0 8.5, F16 16, gpt-oss's MXFP4 4.25) plus a KV cache the server reserves for OLLAMA_CONTEXT_LENGTH × OLLAMA_NUM_PARALLEL tokens (2 × layers × KV heads × head size × 2 bytes per token in f16; q8_0 halves it), plus 5-20 % of runtime buffers. On a CPU, generation is memory-bandwidth bound - every token re-reads the active weights, so a dense 8B Q4_K_M model (4.8 GB) decodes at about 4 tokens/s on a 20 GB/s node while a 30B-A3B mixture reads only its ~3B active parameters and runs like a 3B model once its 18 GB sits in RAM - and prompt evaluation is compute bound (about 2 FLOPs per active parameter per token). A cold load streams the weights from the model store, which on a network share is the 40-150 s that used to dominate answers. osr capacity show qwen3.5:4b glm-4.7-flash --memory 28Gi --cores 4 prints the footprint, whether it fits after the OS headroom, tokens/s and the warm and cold latency of a typical request; osr capacity plan <models...> --memory 28Gi [--hosts N] [--shrink] packs a set onto hosts largest first, shrinks a model that does not fit (coarser KV cache, shorter context, fewer slots) when asked, names what fits nowhere and prints the OLLAMA_MAX_LOADED_MODELS / OLLAMA_NUM_PARALLEL / OLLAMA_CONTEXT_LENGTH / OLLAMA_KV_CACHE_TYPE / OLLAMA_KEEP_ALIVE / OLLAMA_MAX_QUEUE lines that pin exactly that set (--strict exits 2 when a model is unplaced, for CI). With --url http://<ollama>:11434 the shapes come from the server's /api/show (exact layer and head counts, file size, trained context) instead of the tag heuristic, and osr capacity ps --url ... lists what it holds in memory right now, on CPU or GPU, resident or expiring. Defaults - 8K context, one slot, f16, 12 % headroom, 20 GB/s, 40 GFLOP/s per core, 0.3 GB/s cold reads - are OSR_INFERENCE_* settings. The test suite runs the curated set above through the planner against the 8 GiB Consumption app and the 14 / 28 GiB dedicated nodes, so a model that stops fitting its app fails CI before it fails a deployment.

Warm models, not cold loads. The measured latency of a self-hosted answer was never the inference - it was the cold load, so the platform now removes it twice over. Every model server loads its set into memory as soon as the weights are pulled (OSR_OLLAMA_PRELOAD=1 on the per-model CPU apps and the A100, where the whole set stays resident) and keeps it there (OLLAMA_KEEP_ALIVE=-1). Independently, the providers owner asks every self-hosted Ollama provider what it holds (/api/ps) on each health pass and, with OSR_PLATFORM_PROVIDERS_KEEP_WARM_MAX > 0 (Azure sets it to 1 on CPU profiles and to the resident count on a GPU), loads any mapped model that is not resident - but only when the provider maps at most that many distinct models, so a server with more models than memory is never made to evict them in turn. GET /api/v1/providers shows the result: providers[].loaded lists what each server holds and models[].warm says whether a target's upstream model answers from memory (null when the endpoint cannot say); the operator's Check records the same. osr capacity ps --url is the command-line view of the same fact. Voice is a second self-hosted server: voiceEnabled (OSR_VOICE_ENABLED, on by default) adds osr-voice, a Speaches container (faster-whisper speech to text, Kokoro text to speech, OpenAI-compatible /v1/audio/*, CPU, weights preloaded into its own share) and maps the transcribe-small / speak-small targets to it; voiceSttModel (OSR_VOICE_STT_MODEL, default Whisper large-v3-turbo - the accuracy of large-v3 with a 4-layer decoder, on 4 vCPU / 8 GiB; Systran/faster-whisper-small runs on 2 vCPU / 4 GiB) and voiceTtsModel pick the models - the compose stacks do the same with COMPOSE_PROFILES=llm,voice. With voice off, map speak-small to OpenAI or Azure OpenAI in the console (gpt-4o-mini-tts); the OpenAI Realtime API (a WebSocket session, not a request the router can score) and video generation always need a cloud provider called directly. The router decides which target answers either way.

The router ranks on what the deployment measured and bills, not on the catalogue's word. A CPU-hosted 4B model writes about five tokens a second: a one-line classification in ten seconds, an article in minutes. The reference catalogue declared it at 1.2 s and at five times the price of the small cloud model, so the router, the quotes and the latency SLO all read a fiction - production routed almost nothing to the six self-hosted servers while the platform's own writers paid the cloud model. Every routing process now calibrates its executable models every OSR_PLATFORM_CALIBRATION_INTERVAL_S seconds (300; 0 leaves the declared figures) from the last OSR_PLATFORM_CALIBRATION_DAYS (7) of answered requests: the median latency becomes the target's latency_ms, and when enough answers carry a token count and the time grows with it, a least-squares line splits it into a fixed part and a per-token part (latency_per_token_ms), so the router expects the length of each answer - the slow box keeps short requests and loses long drafts, and max_latency_ms rejects on the same expectation. The price becomes the reference card of the model the mapping names (the card the estimate and the usage pricing already bill) when the catalogue has one; a local model no card prices keeps its declared price, which is what the deployment bills. The declared figures stay under the target's metadata.declared, the pass records itself under metadata.calibrated (samples, fit, where the price came from), GET /api/v1/targets shows both, the rank trace carries the per-request estimate and GET /api/v1/learning -> calibration lists what the last pass set. Five answered requests are the least a target needs before its measurement replaces the catalogue.

Capacity: admission control and overflow

A model server has a fixed number of generations it can run at once - one on a CPU replica, 8 or 32 on a GPU - while a burst can bring thousands of chat completions in the same second. The platform therefore never queues requests on the model server. Every provider carries concurrency (the console field or the file key; 0 = unbounded, the default for cloud endpoints) and queue_timeout_s (file key, default 20 s): that many calls run on the endpoint at once, the next ones wait up to the timeout for a slot, and a call that gets none is re-routed by the fallback loop to the next executable target - a chat pinned to llm-onprem with the self-hosted server saturated answers from a cloud tier and says so in opensmartroute.fallback_from; osr.fallbacks: false or a candidate list that is entirely saturated returns 503 with Retry-After instead. The re-route is a routing decision like the first one, so every policy still applies: a request with constraints.data_boundary: on_prem (or a tenant / workspace policy that sets it) is never spilled to a cloud target, even when the caller listed cloud models as fallbacks - it waits for a slot and then answers 503. The Azure template sets the Ollama provider's concurrency to replicas x parallel slots automatically; the compose stacks set it next to OSR_OLLAMA_NUM_PARALLEL. The gates are per process and visible in GET /api/v1/admin/health (admission[]: concurrency, inflight, waiting, admitted, rejected, peak, avg_hold_ms) and GET /api/v1/admin/providers.

The API side holds OSR_PLATFORM_MAX_INFLIGHT requests per process (default 10 000): route handlers run on the worker-thread pool, which is widened to that number at start-up (the framework default is 40 threads, i.e. 40 concurrent completions per replica), and the HTTP server answers 503 beyond it rather than queueing sockets. In cluster mode the API and the openai service scale on in-flight requests (apiMaxReplicas), so ten thousand concurrent requests are spread over replicas, admitted to the model servers up to their capacity and spilled over to the cloud tiers for the rest - the routing decision, trace, budget and quota logic is identical on both paths.

Stopping without dropping a request. A replica that receives its stop signal (a deploy, a scale-in, a node move) does not close its listener at once: GET /readyz answers 503 with status: draining so the load balancer moves new requests to the other replicas, while this one keeps accepting for OSR_PLATFORM_DRAIN_S seconds (default 10 - the readiness probe's period times its failure threshold); then the listener closes and the requests still in flight - a long streamed completion included - get OSR_PLATFORM_GRACEFUL_SHUTDOWN_S seconds (default 120) to finish before the process stops. /healthz stays 200 throughout: the process is healthy, it is leaving. The Azure template keeps at least apiMinReplicas API replicas (default 2) in cluster mode so one can leave while the other answers, sets the container's termination grace period to cover both windows and probes readiness every 5 seconds. Set OSR_PLATFORM_DRAIN_S=0 on a laptop to stop at once.

Response headers

4. OpenAI-compatible endpoint

Point an OpenAI SDK at $OSR_URL/v1 and set model to auto. The router chooses the target per request, runs it and returns a standard chat.completion whose model field is the target that answered. Decision metadata is added in an opensmartroute object that other clients ignore.

from openai import OpenAI

client = OpenAI(base_url=f"{OSR_URL}/v1", api_key=OSR_API_KEY)
reply = client.chat.completions.create(
    model="auto",
    messages=[{"role": "user", "content": "Write a haiku about routing."}],
)
print(reply.model)  # e.g. "llm-small" - the target the router chose
print(reply.choices[0].message.content)

"opensmartroute": {
  "request_id": "7f3c...", "target": "llm-small", "kind": "llm", "confidence": 0.88,
  "plan": [], "alternatives": ["llm-mid"], "cost_usd": 0.00012, "latency_ms": 412.5,
  "fallback_from": [], "tokens_saved": 0
}

plan lists the persona, skill and tool targets the router composed with the model (tool slots are offered to the model as functions and run by the platform when the model calls them, unless the request brought its own tools); tokens_saved is the estimated prompt saving of the compaction step (whitespace, repeated lines and stale history removed before the model was called - only present when it saved anything).

Options outside the OpenAI schema go in the request body and are ignored by other providers:

Message content may be a list of parts: text parts form the request, image_url / input_audio / file parts reach the provider unchanged and make the router prefer multimodal targets (ModalitySignal).

GET /v1/models lists auto, osr/auto, its :cheap / :fast / :quality / :private / :agentic / :llm variants, every executable target and every catalogue model on offer by the token (OpenRouter's shape with the price this workspace pays; model: "openai/gpt-4.1" routes among its endpoints, "openai/gpt-4.1@azure" pins one - see Models by the token under section 6).

POST /v1/embeddings serves the OpenAI embeddings API on the same key: model: "auto" routes among the targets tagged embeddings (capabilities.tags: [embeddings], or metadata.embeddings: true) that a provider maps; a target id pins one. Requests are metered like chat completions and the opensmartroute metadata block names the target and its cost. Without an embeddings-capable target the endpoint answers 501.

Prompt registry

The instruction a request uses can live in the workspace instead of in the calling code. /api/v1/prompts keeps prompts by name with immutable numbered versions (POST /api/v1/prompts adds version 1 of a new name or the next version of an existing one), labels that deploy a version (production, staging; a label names one version, so PUT /api/v1/prompts/{name}/{version}/labels moves it - that is the release), {{variable}} placeholders with optional defaults ({{tone|calm}}; the variables a version uses are recorded with it) and a preview (POST /api/v1/prompts/{name}/render with version or label and variables) that renders without routing. Two kinds: a text prompt becomes the system instruction, in front of the request's own; a chat prompt ([{role, content}]) puts its system turns there and prepends its user / assistant turns to the history as examples. A request names one in osr.prompt (chat completions) or prompt (/api/v1/route): {"name": "support-reply", "label": "production", "variables": {"company": "Acme"}}; version pins a number, neither label nor version means the newest. A missing variable without a default is a 400 naming it; an unknown prompt or label a 404, before anything is routed or metered. The resolved name@version is on the response (X-OSR-Prompt) and in the request's context, so the trace and the request log show which wording acted; the guard and the data-protection pass see the rendered instruction as the model will. GET /api/v1/prompts lists names with newest version, version count and labels; GET /api/v1/prompts/{name} the versions with content; DELETE removes a version or a name; the workspace export carries every version. Up to 200 prompts of 100 versions each; writes take the manage scope and are audited (prompt.version_created, prompt.labels_changed, prompt.version_deleted, prompt.deleted). The usage row of a request that named a prompt keeps the tag under facts.prompt, and GET /api/v1/prompts/{name}/stats (?days=, 1-90, default 30) compares the versions on it: per version the requests, the answered share, tokens, cost, median latency and the outcomes reported on them (success rate, mean quality) - the figures a prompt change is judged by, next to each version on the dashboard. Dashboard: Prompts.

Conversations and memory

The desktop app, the browser extension's side panel and the playground share one chat path, so the platform gives all of them the same three things at once. Two osr options on POST /v1/chat/completions turn them on:

Every answer then carries a conversation object in its opensmartroute block: id, new, turns, topic_shift (the question shared no vocabulary or domain with the recent turns and was not a follow-up - a client offers Move to a new chat), resumed and idle_s (the conversation had been idle for hours), remembered (facts learned from this message, with their kind), forgot (memories removed on request), memories_used and memory (the switch). The conversation id is also the X-OSR-Conversation-Id response header.

Long-term memory is rules, not a model. Facts a person states about themselves are extracted from their own messages - my name is …, I work at …, I'm a data engineer, I'm based in …, we use Python and dbt, I prefer short answers, always answer in French, I'm working on …, remember that … - and stored per subject as name, role, employer, location, language, preference, uses, project or note (one value for name, location and language; at most 200 memories, the least used dropped first). Before a request the identity facts, the memories whose words the request mentions and the freshest notes (12 at most) are written into the system message as What you know about the person you are helping; forget that I work at Acme removes the matching memory. Nothing is inferred, summarised by a model or shared across subjects.

Whose memory. A personal workspace is one person: its memory and conversations belong to the workspace, whatever key or device is used. Inside an organization each signed-in user has their own; an API key has its own (key), because the platform cannot tell who holds a shared key. GET /api/v1/memory says which (subject: workspace, user or key).

Conversations and memories are deleted with the workspace. The dashboard's Memory page (/platform/dashboard/memory) lists both; the desktop app's Settings and the extension's settings page show the memory list with the same switch.

Knowledge graph and context engineering

Memory remembers what was said; the knowledge graph keeps facts - typed, sourced, weighted, expiring triples in one graph per workspace (the SDK's opensmartroute.knowledge, stored in the platform database). It fills itself from the traffic it serves: what a person states about themselves (person:<subject> - works_at, has_role, located_in, speaks, prefers, uses, works_on, noted, mentions), what their requests were about and where they were routed (asked_about, routed_to), what a live web search cited (cites, expiring) and what the workspace adds by hand. Before every routed chat the graph's context engineer chooses, ranks and trims what the model should know into a token budget (600 tokens by default, per subject) and accounts for every line; the answer's opensmartroute.knowledge block says how many tokens were placed, which sections, what was dropped and which facts were learned from the message. The personal section is placed by the graph only when conversation memory is off - otherwise memory already put those facts in front of the model and the graph adds what memory cannot: the workspace's shared facts that match the request. A person's facts are never shown to another subject of the same organization.

Self-managing. The process that owns the knowledge domain (the API, or the knowledge service in cluster mode) runs a maintenance pass every OSR_PLATFORM_KNOWLEDGE_MAINTAIN_S (an hour by default; one replica per interval): unconfirmed facts lose confidence and are dropped under the floor, expired citations go, and when TensorFlow is on the image a TransE link predictor (KnowledgeEmbedding) is refitted over each active workspace's shared facts and stored - GET /api/v1/knowledge/suggest answers from it (which domain is this target probably good at that nobody declared), falling back to frequency over peers without it.

The knowledge graph is deleted with the workspace. The dashboard's Memory page (/platform/dashboard/memory) shows it under Knowledge graph: the switch and the context budget, the facts the person may see in plain words with search and delete, Add a fact, Forget that ..., a preview of the context the graph would place for a message, the export in the four formats and - when the image has TensorFlow - Train now.

Responses API

POST /v1/responses serves the OpenAI Responses API - the surface the OpenAI Agents SDK and newer client code speak - over the same routed chat path and the same key: input (a string or a list of message, function_call and function_call_output items), instructions, flat tools, tool_choice, max_output_tokens, text.format and the osr routing options are translated to a chat completion; the answer is a response object (output items of type message / function_call, usage.input_tokens / output_tokens, the opensmartroute block) and stream: true emits the typed events (response.created, response.output_text.delta, response.function_call_arguments.delta, response.completed). Conversation state is kept per workspace: every response is remembered for OSR_PLATFORM_RESPONSES_STORE_TTL_S seconds (default one hour; store: false opts a response out, the answer's store says whether it was kept) and a follow-up turn may name it as previous_response_id - its turns come first, the new input after. The state lives on Redis when configured, else in the platform database (responses table), so every replica sees it and it survives a restart either way; a workspace can never resolve another workspace's id, and expired rows are dropped by the finance pass. An unknown or expired id is refused with 400 telling the caller to send the full input, never silently ignored. Only function tools are proxied: a built-in tool (web_search, file_search, code_interpreter, ...) is refused with 400 rather than dropped, so a completed response never implies a tool ran that did not. Metered like a chat completion.

Anthropic Messages API

POST /v1/messages serves the Anthropic Messages API - the surface of the Anthropic SDKs, Claude Code and the agent frameworks built on them - over the same routed chat path and the same key: point the client's base URL at the platform and send the API key as Authorization: Bearer or x-api-key. system (string or text blocks), messages with text / image (base64 or URL) / tool_use / tool_result blocks, tools with input_schema, tool_choice (auto / any / tool / none, disable_parallel_tool_use), max_tokens, stop_sequences, metadata.user_id and the osr routing options are translated to a chat completion; the answer is a message object (content blocks of type text / tool_use, stop_reason end_turn / max_tokens / tool_use / stop_sequence, usage.input_tokens / output_tokens, the opensmartroute block) and stream: true emits the Anthropic events (message_start, content_block_start, content_block_delta with text_delta / input_json_delta, content_block_stop, message_delta, message_stop). Errors use the {"type": "error", "error": {"type", "message"}} envelope. model is auto or a target id as everywhere else. Extended thinking is accepted and ignored; PDF document blocks and server-side tools are refused with 400. Metered like a chat completion. POST /v1/messages/count_tokens answers {"input_tokens": n} from the router's own token estimate - no provider call, nothing metered, the route scope suffices.

Google Gemini API

POST /v1beta/models/{model}:{action} serves the Google Gemini API - the surface of the Google GenAI SDKs and the Gemini CLI - over the routed chat path and the same key, action being generateContent, streamGenerateContent or countTokens: point the client's api_endpoint / base_url at the platform and send the key as x-goog-api-key (or Authorization: Bearer). contents with text, inlineData / fileData (images), functionCall and functionResponse parts, systemInstruction, tools with functionDeclarations (Gemini's upper-case schema types become JSON Schema), toolConfig (AUTO / ANY / NONE, allowedFunctionNames), generationConfig (temperature, maxOutputTokens, stopSequences, topP, responseMimeType: application/json) and an osr block of routing options translate to a chat completion; the answer is a GenerateContentResponse (candidates[0].content.parts of text / functionCall, finishReason STOP / MAX_TOKENS / SAFETY, usageMetadata, modelVersion, the opensmartroute block). :streamGenerateContent?alt=sse emits the same objects as data: frames, the last one carrying finishReason and usageMetadata; :countTokens answers {"totalTokens": n} from the router's estimate (nothing metered, the route scope suffices); GET /v1beta/models lists what the key may name. Errors use Google's {"error": {"code", "message", "status"}} envelope. model is auto or a target id (the SDKs' models/ prefix is accepted); the same routes answer under /v1. cachedContent and Google's built-in tools (search grounding, code execution) are refused with 400. Metered like a chat completion.

Rerank

POST /v1/rerank (query, documents as strings or {"text": ...} objects - up to 1 000 -, optional top_n, return_documents) answers in the Cohere / Jina shape: results[] ordered by relevance_score (0-1) with each document's original index and, when asked, its text; model is auto (the embeddings-capable target the router picks) or a target id. The query and the documents are embedded in one call and ranked by cosine similarity, so any embeddings provider gives you a reranker; one metered rerank row per call.

JSON repair

A request whose response_format asks for JSON (json_object or json_schema) gets its answer repaired when the model left it almost right: a Markdown fence or a sentence around the value, a trailing comma, a string or a bracket left open when max_tokens ran out. The smallest edit that makes the text parse is applied - values are never invented, a key cut before its value is dropped - and opensmartroute.json_repaired: true marks the answer. On a stream the missing suffix goes out as one more content delta before the finish chunk, so the concatenated deltas parse; a repair that would have to change bytes the client already holds is not attempted.

Moderation

POST /v1/moderations (input: a string or a list of strings / text parts) answers in the OpenAI moderation shape - results[].flagged, categories, category_scores - from the router's own guards rather than a hosted safety classifier: prompt_injection (the instruction-injection lexicon), router_steering (sentences that talk to the router about the request), confounder_gadget (token soup appended to inflate complexity), pii (reported under categories.pii; flag_pii: true also raises flagged), oversize and control_characters. The opensmartroute block of each result lists the reasons and the PII types found. No provider call, nothing metered; the route scope suffices. Use it to screen prompts before spending on a model.

Voice and images

The same key and the same router serve the OpenAI audio and image APIs, so client.audio.transcriptions.create, client.audio.speech.create and client.images.generate work unchanged with model: "auto":

A target id instead of auto pins one target (400 when it cannot do that job, 404 when unknown); without any capable executable target the endpoint answers 501 and says which tag to add. The voice and image targets execute once they are mapped to a provider that serves the corresponding OpenAI path - OpenAI, Azure OpenAI, the self-hosted Speaches server (voiceEnabled / the voice compose profile) for audio - and are metered under the endpoints transcriptions, speech and images; a transcription target may declare metadata.usd_per_minute so the cost follows the audio duration the provider reports. Uploads must be an audio container (flac, m4a, mp3, mp4, mpeg, mpga, oga, ogg, opus, wav, webm; 25 MB), the file name is reduced to its base name and the declared content type to an audio or video type; the audio bytes never enter the routed text, the request memory, the trace or the logs. Calls to the media providers never follow redirects, cap the answer at 64 MB, forward a fixed whitelist of options, and only relay an audio/* body as speech. osr.objective, osr.constraints, osr.tenant and osr.fallbacks apply to speech and images as on chat completions; a transcription (a form upload) carries no routing options, the workspace policy and the X-OSR-Tenant header still apply. Embeddings, transcription and speech targets are never chosen for a chat completion routed as auto; an image tool may be, in which case the chat answer is the picture as Markdown.

POST /v1/route and POST /v1/feedback are aliases of POST /api/v1/route and POST /api/v1/feedback (same body, same plan gates) for gateways that forward only the /v1 prefix.

TypeScript client

@opensmartroute/sdk (published on the npm registry with every release - npm install @opensmartroute/sdk) is a zero-dependency TypeScript client for the endpoints above - Node 20+, Deno, Bun, edge runtimes and browsers, fetch based. The same client talks to the hosted platform and to a self-hosted osr serve (apiPrefix: "", because the server mounts the routes at the root). Its version follows the platform release and its types are generated from that release's OpenAPI document; the full guide is the TypeScript SDK page.

import { OpenSmartRoute, RateLimitError } from "@opensmartroute/sdk";

const osr = new OpenSmartRoute({ baseUrl: "https://opensmartroute.ai", apiKey: process.env.OSR_API_KEY! });

const decision = await osr.route({ text: "Refund order #123", objective: { cost: 0.6 }, top_k: 3 });
await osr.feedback({ request_id: decision.request_id, target_id: decision.target.id, success: true, quality: 0.9 });

const reply = await osr.chat.completions.create({ model: "auto", messages: [{ role: "user", content: "Hi" }] });
console.log(reply.model, reply.opensmartroute.confidence);          // typed `OsrMetadata`

const stream = await osr.chat.completions.create({ messages: [{ role: "user", content: "Hi" }], stream: true });
for await (const chunk of stream) process.stdout.write(chunk.choices[0]?.delta.content ?? "");

const local = new OpenSmartRoute({ baseUrl: "http://localhost:8000", apiKey: "my-token", apiPrefix: "" });

Options: tenant (X-OSR-Tenant), app (X-OSR-App), headers, timeoutMs, a custom fetch; per call signal, requestId (X-Request-Id) and tenant. Non-2xx answers raise OpenSmartRouteError with status, detail, requestId, retryAfter and a stable code; AuthenticationError (401), PermissionDeniedError (403, upgradeRequired), NotFoundError (404) and RateLimitError (429, budget from the X-Budget-* headers) are subclasses. Request types (RouteIn, FeedbackIn, ChatCompletionIn, EmbeddingsIn) are generated from this API's OpenAPI document; RouteDecision, ChatCompletion, ChatCompletionChunk and EmbeddingsResponse type the answers. The package README documents the development and live-test workflow.

5. Feedback

POST /api/v1/feedback reports how a routed answer turned out. The learners (Bradley-Terry, IRT, LinUCB) update on the workspace's own traffic. Included in every plan of the default catalogue (feature feedback).

curl -s "$OSR_URL/api/v1/feedback" -H "Authorization: Bearer $OSR_API_KEY" -H "Content-Type: application/json" \
  -d '{"request_id": "7f3c...", "target_id": "llm-frontier", "success": true, "quality": 0.9, "latency_ms": 2100}'

Fields: request_id, target_id, success (required); quality (0..1), cost_usd, latency_ms, domains, complexity, preferred_over (the id of a target this one beat in a comparison), role (the plan slot the outcome grades - persona, skill, tool or llm - when it is about a companion of a composed answer rather than the target that answered). The request_id must be one the workspace itself routed (/route, /v1/chat/completions, /estimate, MCP): an unknown id, or another workspace's, is a 404, as is an unknown target_id - a catalogue target, one of the workspace's installed listings, or a published skill / persona a plan discovered for the request (section 11) are known.

Every report is kept against its request: GET /api/v1/activity summarises it per row (outcome), GET /api/v1/trace/{request_id} lists the reports (outcomes) next to the spans, and in the enterprise edition the audit chain records an outcome entry after the decision entry (section 10). The dashboard has the same form without curl: expand a row on /platform/dashboard/activity (or open its trace) and report good / poor, a quality score and, optionally, the target that would have done better - the latter is recorded as a pairwise preference for the Bradley-Terry learner.

The platform reports what it can see itself. Almost no client calls /feedback, so an execution's own assessment is kept the same way and needs nobody: when the answer was judged (a handler that rates its own output, or the implicit reading of the answer - a refusal, a truncation, an empty reply), and when a fallback settled a comparison (the target that answered a request another one had just failed beat it on that request), the outcome is written against the request and published on the feedback topic the SLM service trains from - for the answering target and, with their role, for every persona, skill and tool the plan composed the answer with, so the companions share the answer's judgement. An answer nothing was found wrong with is not an assessment and is not recorded.

The human queue

Routing to a person is a real execution, not an error. When a decision lands on a human target (escalation intent, high risk, low automated confidence - the defer strategy scores this), execute: true and the chat proxy queue a ticket instead of failing: the answer names the ticket, costs zero tokens, and the ticket keeps the request text (with PII restored - human targets declare pii_allowed) and the router's reason for deferring in a few plain sentences: what the request reads as (domain, difficulty, personal data, jailbreak risk), which targets policy ruled out and why, how the human target ranked against the runner-up, which strategies carried it, and which targets failed before the router re-routed there. The full decision trace stays one link away.

The dashboard's Human queue (/platform/dashboard/handoffs) shows how many tickets wait and for how long, lists each ticket with the request and the reasons side by side and a Full trace link into the activity log, and lets a teammate answer it - the answer goes back as the response and is recorded as the outcome with a quality in words (Solved ... Could not help much) - or cancel it, without curl.

Evaluation

Outcomes say how the router did on the traffic it got; an evaluation says how it would do on the prompts you choose, before you send them. POST /api/v1/evaluations takes a labelled dataset - the JSON-lines file osr eval reads, one object per row - and routes every row on the workspace's own pool: its policy, tenants (tenant in the body), allow / deny lists, API-key limits, installed listings and providers all apply, exactly as on /route. Nothing is executed and nothing is metered: a run calls no model, costs nothing, counts against no quota, and leaves no trace or event in the workspace's history (the rows are not traffic). The dataset is routed and discarded; only the report is kept - the newest 50 runs per workspace, readable at /platform/dashboard/evaluation.

curl -s "$OSR_URL/api/v1/evaluations" -H "Authorization: Bearer $OSR_API_KEY" -H "Content-Type: application/json" \
  -d '{"name": "support prompts, week 32", "rows": [
        {"text": "What is 12 * 37?", "expected": "tool-calculator"},
        {"text": "Summarise this clause in plain words: ...", "acceptable": ["llm-mid", "llm-frontier"]},
        {"text": "List customers with no orders this year.", "scores": {"skill-sql": 1.0, "llm-mid": 0.7}, "task_type": "sql"},
        {"text": "Translate good morning into French."}]}'

A row is text plus how it is judged: expected (the target id that should win), acceptable (several ids that all count), or per-target scores in 0..1 (RouterBench style; the best score is the label and the chosen target's score is its realised quality). context, history and task_type travel with the row. Rows without a label are fine - they still give the mix, the price and the confidence. objective ({"cost": 0.6}) and constraints are those of /route. The report carries:

A run routes at most OSR_PLATFORM_EVAL_MAX_ROWS rows (default 2000, 413 beyond it) and for at most two minutes (truncated: true with evaluated when it stopped early). GET /api/v1/evaluations/sample returns labelled rows made from the catalogue itself (ontology seed prompts scored by capability fit), so a workspace can see a report before it has a dataset; the page's Template button writes the row format. From the command line, osr eval rows.jsonl --remote (after osr login, or with --url / --token) sends the same file to the signed-in workspace instead of a local catalogue, prints the headline, the misroutes and the baselines, keeps the run on the Evaluation page and honours --min-accuracy for a CI gate; --baselines, --frontier, --limit, --preset and --json mean what they mean locally.

That gate is a few lines in any CI. For GitHub Actions the repository ships a composite action (.github/actions/osr-eval/action.yml - copy it into your own repository, or paste the step below): it installs the package, runs osr eval locally against a catalogue or remotely against the signed-in workspace, fails the job below min-accuracy and writes the accuracy to the job summary and to an accuracy output.

# .github/workflows/routing.yml - fail the build when a catalogue or rule change lowers routing accuracy
on: [pull_request]
jobs:
  routing:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with: { python-version: "3.12" }
      - name: Routing accuracy gate
        run: |
          pip install "opensmartroute[yaml]"
          osr -t targets.yaml -r rules.yaml eval evals/routing.jsonl --baselines --min-accuracy 0.85
      - name: The same rows on the hosted workspace   # optional: what production would decide
        env: { OSR_API_KEY: ${{ secrets.OSR_API_KEY }} }
        run: osr eval evals/routing.jsonl --remote --min-accuracy 0.85

Azure Pipelines, GitLab, Jenkins and CircleCI run the same two commands; exit status 2 is the failed gate.

POST /api/v1/evaluations/replay is the other direction: it re-routes the prompts the workspace kept under its log_payloads policy (section 9; 409 naming the policy when nothing is kept) through today's router - with everything the learners took from your outcomes, every catalogue and rule change since - and compares each decision with the target that answered then: changed / kept and the moves between targets, the kept token counts priced at the new target's list price against the provider cost metered then (cost; decision-only requests carry no tokens and are left out), and agreement with the outcomes people reported - good_kept (accepted answers the router still sends the same way), good_changed, bad_changed (rejected answers it now sends elsewhere) and bad_kept. days (1-90) and limit bound the window; a prompt masked by the redacted mode is replayed as kept, placeholders included.

Datasets from production. The prompts your users actually send are the dataset that matters, and the outcomes they reported are its labels. POST /api/v1/evaluations/datasets saves logged requests as a kept dataset - the Add to dataset action of the Activity page: name request_ids, or take a window over the prompts kept under log_payloads (days, limit, outcome good / bad / rated / any, target_id), or send explicit rows, or mix the three. A logged request becomes {text, expected, scores, context, task_type}: the kept prompt, the target that answered as expected when its outcome was good (with the reported quality as its score), the request id under context.source so the same request is never added twice. GET /api/v1/evaluations/datasets lists them (rows and labelled counts), GET .../datasets/{id} returns the rows (?format=jsonl downloads the file osr eval reads), POST .../datasets/{id}/rows adds to one, POST .../datasets/{id}/run evaluates it on today's pool and keeps the report (the body carries the run options - objective, constraints, tenant, baselines, frontier; the report names the dataset_id), DELETE removes it. A workspace keeps up to 50 datasets of up to 5 000 rows; all of it takes the route scope.

Scores from other frameworks. A dataset downloads in the shape Ragas, DeepEval and Langfuse read (?format=ragas|deepeval|langfuse; every row carries its row index and, for logged requests, the request_id in the metadata), and the scores those tools compute come back with POST /api/v1/evaluations/datasets/{id}/results: format names the shape - ragas (the result table, to_pandas().to_dict("records"): numeric columns are the metrics), deepeval (test results with metrics_data: name, score, success), langfuse (run items with their scores) or scores (plain {row | request_id | input, scores: {name: value}, passed} rows from TruLens, Arize or a script) - tool the framework for the record, threshold passes a row whose every metric reaches it when the tool gave no verdict. Rows join by index, request id or prompt text; the run is kept as an external evaluation (per-metric mean / min / max, the pass rate, the per-row scores; 409 when nothing joins), and a row that came from a logged request receives the mean score as an outcome with source eval:<tool> (feedback: false leaves that out), so the activity page, the trace and the learners see what the other tool thought.

The answer judge. The runs above grade which target was chosen; the judge grades the answer the customer received. With a judge block in the workspace policy (section 9: sample, target, threshold, endpoints, targets, max_per_day) a reproducible share of the executed answers is graded by the judge model after the answer has left, the grade becomes the request's outcome (feedback rows with source: judge:<model>, read by the learners, the rankings and the replay comparison like a hand-reported one), and every verdict keeps its reason. GET /api/v1/evaluations/judge (days, limit) returns the block in force, what was judged per answering target over the window (judged, graded, good, mean score, cost) and the latest verdicts; POST /api/v1/evaluations/judge/run {request_id} grades one kept request now (the Activity page's Judge this answer; execute scope, 409 naming log_payloads when the prompt and answer were not kept). Judging is metered as its own endpoint (judge) on the workspace's usage, never charged to the request it grades.

Every run is recorded in the workspace's audit trail as evaluation.run, travels with the data export (GET /api/v1/me/export -> workspace.evaluations) and is deleted with the workspace. A tenant-pinned API key cannot run evaluations (403) - they are the workspace's view of its whole pool. The TypeScript client has them as client.evaluations.run / replay / sample / list / get / delete.

Quality gates. An evaluation says how good the router is on your prompts today; a gate says whether it is still that good after the catalogue moved, a provider was swapped, a model was retired or somebody edited the workspace policy - before your own customers notice. Keep the dataset you signed off on as a gate (Keep as a quality gate on the Evaluation page, or POST /api/v1/evaluations/gates with the same rows, objective, constraints and tenant as a run) with the thresholds it must hold: min_accuracy (an absolute floor, 0..1) and / or max_drop (how many accuracy points a run may fall below the run that set the baseline - the first pass; default 0.05). The gate is routed at once - that first pass is the baseline - and again every OSR_PLATFORM_EVAL_GATES_INTERVAL_S (six hours; 0 = only on request and after a policy change) and whenever the workspace policy is replaced or removed. Each pass is a kept evaluation named Gate · …, so the full report is one click away. A pass under the floor is fail, one too far under the baseline regressed; the first failing pass files a warning in the workspace inbox (workspace.evaluation_gate_failed, linking the report) and the audit trail (evaluation.gate_failed), a scheduled pass that keeps failing stays quiet, and the pass that clears both thresholds again files evaluation_gate_recovered. A run on request or after a policy change always announces a failure. Nothing is executed or metered. A workspace keeps up to ten gates of up to 500 rows each, at least five of them labelled; creating and deleting a gate takes the admin role, running one the route scope.

Shadow routing. A dataset and a gate replay prompts; a shadow candidate rides the live traffic. Start one (Shadow routing on the Evaluation page, or POST /api/v1/shadow) with the policy you are considering - objective weights (quality, cost, latency, quality_floor) and / or the request-shaping constraint keys of the workspace policy (deny_targets, allow_targets, max_cost_per_1k, max_latency_ms, preferred_max_latency_ms, region, data_boundary, contains_pii) - and from then on every request the workspace routes is routed a second time under the candidate, inside the same pool and with the same tenant, key limits and installed listings. The second decision is never served, executed, metered or charged; what is kept is two target ids and the cost, quality estimate and latency each would have had (no text). GET /api/v1/shadow sums it up: agreement between the two, the spend delta, the quality and latency deltas, where the candidate moves traffic from and to, the requests it would have refused, and a plain-words verdict once a few dozen requests are in. When the comparison convinces you, POST /api/v1/shadow/promote writes the candidate into the workspace policy - its constraints are merged, its weights become the policy's objective, the default for every request that sets none (an explicit objective on a request still wins) - which is a policy change, so every quality gate the workspace keeps runs again on it: a promotion that regresses a kept dataset shows up in the inbox like any other bad edit. One candidate runs at a time; it ends when promoted, stopped (DELETE /api/v1/shadow) or after max_decisions requests (20,000 by default), and earlier candidates stay readable (GET /api/v1/shadow/{candidate_id}). Starting, stopping and promoting take the admin role; a tenant-pinned key is refused.

Observed usage: the browser extension's ledger

The browser extension (platform/extension, product page /extension) sees calls a team makes outside the router - on ChatGPT, Claude, Gemini, Copilot, Perplexity and the OpenAI, Anthropic, Google and OpenRouter consoles. With "Share observed usage" turned on it reports the facts of each call and never the text:

curl -s "$OSR_URL/api/v1/usage/observed" -H "Authorization: Bearer $OSR_API_KEY" -H "X-OSR-App: browser-extension" \
  -H "Content-Type: application/json" \
  -d '{"items": [{"ts": 1760000000, "source": "chat", "site": "chatgpt.com", "vendor": "openai", "model": "gpt-5",
                 "prompt_tokens": 1284, "completion_tokens": 310, "task_type": "summarise", "complexity": 0.3}]}'

Observations are kept for 90 days and deleted with the workspace. The report compares the same request shapes on both sides - it is a price difference, not a quality claim - and the dashboard shows it on /platform/dashboard/spend.

Insights: the figures read back as findings

The dashboard shows numbers; the insights endpoint says what they mean. It reads the workspace's own metered rows - the window against the one before it, per target and per day - and returns findings, each with the evidence behind the sentence and the page that acts on it: cost against the previous window with the targets that moved it, a day whose cost sits far above the average, failures and their rate, speed against the previous window, one target carrying almost all the traffic, decisions without execution, savings against the baseline and spend the extension saw outside the router. No model is involved: every sentence is computed, so asking twice gives the same answer. The dashboard overview carries it as Ask about your usage.

Bill analysis: the invoice or usage export you already have

A team that has not routed yet still has a bill. Every provider, gateway and tracing tool exports usage as a table - a date, a model, request and token counts, often the amount charged - and every provider issues a PDF invoice. Post either and the platform recognises the columns - or, for an invoice, the vendor, the billing period and every line that names a model with an amount - aggregates it per day, provider and model and prices every line twice: what the bill says (or, when the export carries no amounts, the model's catalogue list price) and what the router's recommended target would have charged for requests of the same token shape. Nothing in an export is text - dates, names, quantities and amounts are read, every other column or word (addresses, tax ids, people) is ignored and never stored.

The layouts recognised (source_format) - the tables from their header row, an invoice from its text:

Cloud bills are roll-ups, so their hourly rows are never counted as requests; a line without a request count is quoted as requests of about 2 000 input tokens. An invoice dates every line at the start of its billing period and the projection per month uses the whole period. A scanned invoice (a picture with no text layer) and a password-protected PDF are refused with what to do instead, and an old .xls should be saved as .xlsx or CSV.

curl -s "$OSR_URL/api/v1/usage/imports/upload" -H "Authorization: Bearer $OSR_API_KEY" \
  -F "file=@openai-invoice-2026-08.pdf" -F "name=August, OpenAI"
curl -s "$OSR_URL/api/v1/usage/imports/upload" -H "Authorization: Bearer $OSR_API_KEY" \
  -F "file=@openai-usage-2026-08.csv" -F "name=August, OpenAI (usage export)"
curl -s "$OSR_URL/api/v1/usage/imports?dry_run=true" -H "Authorization: Bearer $OSR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"name": "one line", "provider": "anthropic",
       "text": "date,model,input_tokens,output_tokens,cost\n2026-08-01,claude-sonnet-4,120000,30000,0.81"}'

The routed side is the estimator's quote under the catalogue and rules of today, so every figure is labelled an estimate; lines whose model the router cannot quote are carried over at what they cost today and the notes say how much of the bill was quoted. Analyses are the customer's own data: kept until they delete them, and deleted with the workspace.

Client activity: what the extension and the desktop app did

Most of what the browser extension and the desktop app do happens on the person's own machine and is never a routed request - the pre-send card, a redaction, a compaction, a writing fix, the command bar opening, an answer pasted back. Each app keeps a small queue of those moments and posts it to the workspace a few minutes later, so the dashboard's Apps page (/platform/dashboard/apps) shows how the apps are used, what they caught and what they saved. Every event is a kind, a hostname, counts and figures - never text:

curl -s "$OSR_URL/api/v1/usage/activity" -H "Authorization: Bearer $OSR_API_KEY" -H "X-OSR-App: desktop" \
  -H "Content-Type: application/json" \
  -d '{"items": [{"ts": 1760000000, "app": "desktop", "kind": "answer.received", "target_id": "gpt-4.1-mini",
                 "request_id": "6f1c...", "latency_ms": 2300, "version": "1.5.0"},
                {"ts": 1760000010, "app": "browser-extension", "kind": "pii.redacted", "site": "chatgpt.com", "n": 2}]}'

What the plan decides: Free keeps events 7 days and records up to 5 000 a day, Pay as you go and Pro 30 days and 50 000 a day, Team 90 days and 500 000 a day, Enterprise 365 days with no daily cap; a workspace's retention_days override (custom agreements) wins over the tier. The retention sweep purges each workspace at its own cutoff. The kinds the first-party clients send are listed in Browser extension and Desktop app; both apps have a Log my activity to the workspace switch (on by default, since there is no text in the events) and administrators can pin it through the extension's managed storage.

Every request the extension routes through the platform also carries X-OSR-App: browser-extension. Any client may name itself that way (osr-cli, langchain, your own service name; free text is reduced to a short slug): the value is stored on each metered row and GET /api/v1/usage returns by_app - requests, tokens and cost per app, null for calls that sent no header - which the Usage page shows as By app.

6. Plans and quotas

Quotas count metered requests (a decision, a chat completion or embedding through the proxy, an estimate, a feedback event) per calendar month (UTC); the counter resets on the first. GET /api/v1/info (plans) returns the values for the deployment you are calling (OSR_PLATFORM_PLAN_OVERRIDES patches them per deployment), GET /api/v1/me returns your plan and GET /api/v1/quota where you stand:

{"plan": "pro", "period": "month", "used": 241300, "limit": 300000, "remaining": 58700, "percent": 80.4,
 "state": "warning", "warn_at": 0.8, "resets_at": 1767225600, "today": 9120, "requests_per_minute": 300,
 "overage": {"allowed": true, "usd_per_100k": 10.0, "requests": 0, "estimated_usd": 0.0}}

state is ok, warning (80 % and above), overage (beyond the quota on a plan with an overage rate and a live subscription - requests keep flowing and are invoiced) or exhausted (beyond the quota on any other plan - metered calls answer 429). The dashboard shows the same bar on the overview, the governance page and Plan and billing; the quota alert rule fires at 80 % and escalates at 100 % (workspace channels, or the owners by e-mail when the workspace has no e-mail channel - outbox kind quota_notice; the budget and wallet rules reach the owners the same way, as budget_notice and credits_notice). Upgrades are made in the dashboard (/platform/dashboard/billing).

• 429 rate limit exceeded - per-minute window; wait Retry-After seconds.
• 429 monthly quota exhausted - X-Quota-Limit, X-Quota-Used, X-Quota-Reset and X-Upgrade: true.
• 403 with X-Upgrade: true - the feature is not in your plan.
• 403 ... requires the enterprise edition with X-Edition: enterprise - the deployment runs the community image, which does not expose tenants, statistics or audit regardless of plan.
• 403 this API key lacks the '<scope>' scope with X-Key-Scope - the key was created with scopes.

Every feature a plan sells is live on the workspace the moment the plan applies - there is no separate activation. Plan and billing (/platform/dashboard/billing) lists them under What the plan includes, each one a link to the page that uses it (providers, activity, integrations, tenants, health, audit trail, workspace SSO); a feature the plan lacks names the cheapest plan that has it with an Upgrade button. The sidebar marks pages the plan does not include with a lock, and the page itself explains the gap.

Support tiers. Plan.support is community (free: documentation, the assistant, the issue tracker), email (pro), priority (team) or dedicated (enterprise, with an SLA). Tickets opened from a workspace on priority or dedicated support start at high priority at least - a normal or low request is raised, a higher one is kept - and the operator notice and inbox line name the plan and the tier (Acme (team plan, priority support)).

Service levels. Each tier states what it promises and the platform measures against it, so the pricing page, the portal and the API read one source (plans.py): support_response_hours is the first-response target of the support tier (community none, e-mail 48 h, priority 8 h, dedicated 4 h) and sla_uptime_pct the monthly availability target the plan carries (99.9 % on enterprise; 0 elsewhere). The target is a contractual figure: the hosted platform is one stack per region, the service credits that apply when a month misses it are defined in the enterprise agreement, and the number the platform reports is its own measurement - not an external monitor. GET /api/v1/sla reports the calendar month so far from the workspace's own records: availability - the share of its metered requests the platform answered without a 5xx (a refused request - quota, policy, bad input - or a provider error is not a platform failure) against target_pct; support - the tickets opened this month, how many were answered, the median and slowest first response in hours, how many were answered within target_hours and how many are past it (breached, answered late or still waiting). met is true, false or null (nothing to measure yet); committed says whether the plan promises anything. Plan and billing shows the same figures under Service level on every plan, labelled committed or no commitment on this plan.

Talk to sales. Plans sold through sales (contact_sales, the enterprise plan) show Talk to sales on the pricing page and in the portal instead of a checkout; the button opens the form on /contact (?plan= preselects the plan). POST /api/v1/sales/inquiries (email, name, company, plan, interest hosted / self-hosted / both / other, volume, message; no key needed, five per address and day, a filled website field marks a bot) records the inquiry with a reference (OSR-S-00007), e-mails the requester a confirmation (outbox kind sales_received), the support recipients a notice (sales_operator) and the console inbox a line; a signed-in caller's workspace is attached. A second request from the same address within a day returns the open inquiry. Operators work the queue under /platform/admin/sales (GET /api/v1/admin/sales/inquiries?status=&q=, GET|PATCH /api/v1/admin/sales/inquiries/{key} - status new / contacted / qualified / won / lost, assigned_to, internal note; audit events sales.inquiry_opened, sales.inquiry_updated) and close the loop on the workspace page: plan, custom quotas, retention and region.

Custom quotas and retention. Enterprise agreements override a workspace's numbers without a new plan: an operator sends limits with PATCH /api/v1/admin/accounts/{id} - any of requests_per_month, requests_per_minute, max_keys, max_tenants, max_members and retention_days (whole numbers; the object replaces the previous overrides, {} returns the workspace to the plan as sold; audit event account.limits_changed). GET /api/v1/me then reports the effective plan with "custom": true and the overridden values, GET /api/v1/quota enforces the custom quota, the key, seat and tenant limits apply as the plan's own would, and Plan and billing shows a custom quotas badge. retention_days gives the workspace its own request-row retention: the retention sweep purges its rows at that age and leaves them out of the deployment-wide purge (OSR_PLATFORM_RETENTION_DAYS, which must be on for the sweep to run); GET /api/v1/governance reports the effective retention_days. The operator console edits the same fields under Custom quotas and retention on the workspace page (/platform/admin/workspaces/<id>); GET /api/v1/admin/retention lists the workspaces with their own retention under usage.overrides.

Billing: subscriptions, trials, overage, invoices and cancellation

Paid plans are flat tiers sold as one Stripe subscription per workspace, monthly or yearly. Self-serve billing is on when the deployment sets all three of OSR_PLATFORM_STRIPE_SECRET (the sk_live_... key), OSR_PLATFORM_STRIPE_WEBHOOK_SECRET (the whsec_... of the endpoint below) and OSR_PLATFORM_STRIPE_PRICES (JSON plan -> price id); billing: true in GET /api/v1/info tells the web app. Without them every billing endpoint answers 501 and operators set plans by hand (PATCH /api/v1/admin/accounts/{account_id}). The Stripe calls are plain HTTPS from the API, no SDK.

OSR_PLATFORM_STRIPE_PRICES keys: "pro" is the monthly recurring price, "pro:year" the annual one (sold when present - the pricing page shows the switch), "pro:overage" a metered price on a Stripe Billing Meter whose event name is OSR_PLATFORM_STRIPE_METER_EVENT (default osr_overage_requests, unit = requests), and "pro:overage:year" the same metered price with a yearly interval - Stripe only accepts items of one interval per subscription, so a plan with overage is sold per year only when both "pro:year" and "pro:overage:year" exist (a yearly subscriber's overage is then invoiced with the annual renewal). Each metered line lives on its own Stripe product - OpenSmartRoute Pro - extra requests with the unit label request, ... - metered usage - because Checkout and the invoice list every line by product name: a buyer must read extra requests, $0.0001 per request, not OpenSmartRoute Pro twice. A setup run on a deployment created before this moves an overage price left on the plan product to its own (lookup key transferred; set the printed price ids again). osr-platform-stripe-setup --webhook <api base url> (ships with the platform API; run it inside the API container) creates all of them from the plan catalogue (products, monthly / yearly / metered prices, the meter, the default Billing Portal configuration the Manage subscription button opens - payment method, billing details, invoices, cancel at period end, no plan changes

• and the webhook endpoint with the seven events below), idempotently - run it again after changing a price and the lookup keys move to the new prices - and prints the JSON, its base64 form for the deployment configuration and the webhook signing secret; the key comes from STRIPE_SECRET_KEY in the environment (start with a test-mode key and --dry-run):

{"pro": "price_...", "pro:year": "price_...", "pro:overage": "price_...", "pro:overage:year": "price_...",
 "team": "price_...", "team:year": "price_...", "team:overage": "price_...", "team:overage:year": "price_...",
 "enterprise": "price_...", "enterprise:overage": "price_..."}

To also invoice what a workspace spent by the token on the same invoice as its plan, run the setup with --usage-meter-event (bare, or with your own event name): it creates a second Billing Meter whose unit is one cent and a metered price per plan (the "pro:usage" / "pro:usage:year" keys), and prints OSR_PLATFORM_STRIPE_USAGE_METER_EVENT. With that variable set, a checkout and a plan change carry the usage item next to the overage one, and the same sync loop reports the month's charge_usd in cents (usage_charge_reports, one billing.usage_reported audit entry per increment). Without it the charges are still priced, stored and shown on Usage and charges - they simply never reach a Stripe invoice.

Tax. Checkouts ask Stripe Tax to compute VAT / GST / sales tax from the buyer's billing address and to collect a tax id, which is what the pricing page and the terms promise. Stripe only does that once the account's head office address and tax registrations are set (Stripe -> Settings -> Tax); until then Stripe refuses a session that asks for tax. The platform does not let that stop a purchase: it creates the session again without the tax fields, remembers the refusal (Billing.tax_problem) and raises the deployment alert stripe_tax_unconfigured (console Health and Finance, GET /api/v1/admin/health -> alerts) reading Stripe's tax settings (Billing.tax_status, cached an hour) - critical once a checkout had to drop tax, a warning before. Buyers then pay the list price with no tax line; fix the Stripe account, and the next checkout computes tax again.

Activation. A Stripe account that has not finished activation (business details, bank account, card payments turned on) refuses every paid Checkout Session - yet it accepts a free trial ($0 today), so a buyer could type a card into a trial that would be cancelled at its end without a word. The API reads the account from Stripe (Billing.account_status, Stripe's account object, cached an hour) and raises the critical deployment alert stripe_account_inactive while charges are disabled or card payments are not active, naming what is missing; while it stands, no Checkout Session is created at all - a plan with or without a trial, credits or a paid listing - and the buyer gets 503 Payments are not open on this deployment yet - the operators have been alerted. Contact sales to buy in the meantime. with Retry-After on both POST /billing/checkout and POST /billing/credits/checkout, never Stripe's developer sentence. Finish the activation under Stripe -> Settings -> Business and Settings -> Payment methods; checkouts go through and the alert clears within the hour.

A checkout adds the plan's overage item next to the recurring one; a plan change swaps both. Every OSR_PLATFORM_OVERAGE_SYNC_S seconds (default hourly, 0 = never) the API reports each billed workspace's requests beyond its quota this month as meter events (one increment per sync, de-duplicated by identifier and a compare-and-set row in overage_reports, so replicas never double-report; each report is a billing.overage_reported audit entry). The first checkout of a workspace starts the plan's trial_days (90 on every subscription plan) without a payment method: the plan applies at once and the workspace returns to free when the trial ends unless it subscribes (see Free trials, coupons and the launch offer); "coupon": "" pays now instead. A plan whose trial_needs_card is true carries its trial into Stripe Checkout instead (card collected, the subscription cancels if none is on file when the trial ends). A workspace that had a subscription or a trial before gets no second trial.

Point the Stripe webhook endpoint at POST /api/v1/billing/webhook with these events enabled: checkout.session.completed, customer.subscription.created, customer.subscription.updated, customer.subscription.deleted, invoice.paid, invoice.payment_failed, charge.refunded (the last one credits a refund back to the wallet; osr-platform-stripe-setup registers all seven). The workspace's billing_state (GET /api/v1/me -> account.billing_state, also stripe_subscription and billing_period_end) follows the Stripe subscription status:

invoice.payment_failed moves the workspace to past_due, writes a billing.payment_failed entry to the workspace's audit trail (GET /api/v1/workspace/audit) and sends the owners a dunning message (outbox kind billing_payment_failed, "Update payment method" linking to the hosted invoice); invoice.paid moves it back to active and, when the workspace was past due, tells the same owners the invoice is settled (outbox kind payment_received; inbox notice workspace.payment_received). The dashboard (/platform/dashboard/billing) shows the state badge, a warning banner while past due, the renewal or end date, Manage subscription (portal), Cancel at period end and the invoice table; members see the state and invoices but only owners and admins get the buttons.

Usage pricing, credits and provider finance

Every metered request carries two prices: the provider cost the router optimised (cost_usd, what the executed model cost upstream) and the customer charge (charge_usd, what the workspace pays), plus the token split (prompt_tokens, completion_tokens, cached_tokens), the provider_id and upstream_model that executed and the price_version the charge was computed under - so a margin report months later reproduces it. Successful /api/v1/route executions and /v1/chat/completions answers return the charge in the body (result.charge_usd, and charge_usd inside the opensmartroute object of a chat answer) and in the X-OSR-Charge-USD header.

The charge comes from a pricing engine with two books per model. The provider price book holds rates per unit (token_in, token_out, token_cached in USD per million; call, image, second per unit): catalogue rows are the list prices of the model catalogue (Seed from catalogue in the console, and rewritten by every catalogue refresh of the rankings loop so a vendor's price change reaches the books on its own - never overwriting the others), manual and contract rows of a provider beat the list price. Pricing rules turn cost into charge: markup_pct ({"pct": 20}), markup_fixed ({"usd_per_million": 0.5}), per_million ({"usd_per_million": 3.0, "unit": "token_out"} sets the customer rate outright), tier ({"tiers": [{"upto_tokens": n, "markup_pct": p}, ...]} against the month's token volume), promo ({"pct_off": 10}), min_margin ({"pct": 5} - the reseller's floor: after every markup and promotion, no unit is charged under the provider rate plus that percent, so a tariff typed below a contract rate or a 100 % giveaway can never sell below cost), fx ({"rate": 0.92}) and min_charge ({"cents": 1}). Rules have a scope - account (one workspace), plan or deployment - and the most specific wins, then priority; without any markup rule OSR_PLATFORM_DEFAULT_MARKUP_PCT applies (default 0: customers pay provider cost). Every change to a book or rule freezes a new price version (GET /api/v1/admin/finance/versions) and writes an audit entry with before, after and the reason the console requires.

Three semantics the engine guarantees on the metered row. A charge that prices to zero on purpose (a 100 % promo, a zero per_million tariff) stays zero - only a request that neither the books nor the executing handler could price falls back to the reported provider cost. Tiers read the workspace's token volume of the month before the request being priced (tier boundaries are crossed by the first request past them). The pipeline is USD-only: an fx rule may scale the charge but POST /api/v1/admin/finance/rules refuses one that names another currency (400), and a foreign-currency quote that reaches the engine anyway leaves the row unpriced (charge_usd null) rather than mislabelled. Unpriced rows - also the ones a pricing failure left behind - are retried by the finance pass (every OSR_PLATFORM_FINANCE_SYNC_S seconds, default 5 minutes, and on POST /api/v1/admin/finance/wallets/sync), each written exactly once; the finance block of the analytics reports how many remain (unpriced), the console shows them as a warning, and revenue and margin exclude them until they are priced.

A workspace pays for usage in one of three billing modes (billing_mode on the account, set by an operator with PUT /api/v1/admin/finance/accounts/{account_id}/billing-mode): subscription (default - the plan's flat fee plus overage; usage charges reach the same Stripe invoice through a dollar Billing Meter when OSR_PLATFORM_STRIPE_USAGE_METER_EVENT names it, one increment per sync with compare-and-set bookkeeping in usage_charge_reports), wallet (prepaid credits, below) or invoice (net terms, recorded for the statement). Credits are bought through a Stripe Checkout in payment mode (POST /api/v1/billing/credits/checkout {amount_cents}, whole dollars between OSR_PLATFORM_CREDITS_MIN_CENTS and OSR_PLATFORM_CREDITS_MAX_CENTS, tax collected by Stripe Tax); the webhook's checkout.session.completed with metadata[credits_cents] writes a purchase entry keyed by the event id, charge.refunded a refund entry keyed by the refund id. The ledger is append-only - corrections are adjustment / promo entries with a reason (POST /api/v1/admin/finance/accounts/{account_id}/ledger) - and the balance is its sum.

Admission and settlement. Before a call that can incur provider cost runs, an upper bound of what it can cost is held on both sides of the ledger: every target the router may pick is priced by the finance engine at the prompt's tokens plus the output allowance (max_tokens when the caller sets one, else OSR_PLATFORM_SPEND_BOUND_OUTPUT_TOKENS, 4096) under the workspace's tariff, and the dearest quote times OSR_PLATFORM_SPEND_BOUND_ATTEMPTS (fallback attempts that may each bill, default 1) is the bound - provider cost for the budgets, customer charge for the wallet. A surface that cannot describe its request falls back to the workspace's mean charge over its last hundred spending requests (at least OSR_PLATFORM_SPEND_RESERVE_MIN_USD), which is an estimate, not a cap, and is labelled as such. Budgets (daily_budget_usd / monthly_budget_usd of the workspace, a tenant or a key) are a strict cap on provider cost: a request is refused with 429 when recorded cost + the provider-cost holds of every request still in flight + its own bound reach the limit, X-Budget-Used includes the holds and GET /api/v1/policy shows them as held_usd. A wallet workspace is admitted only while balance - charges not yet settled - customer-charge holds in flight - bound does not go below zero (a bound that exactly meets the remaining credits is admitted; 402 otherwise, X-Wallet-Balance and X-Wallet-Available in cents) - so a one-cent wallet never admits a two-dollar first call. The hold is written, the checks are repeated against every hold now committed (two replicas admitting at the same instant both see each other and the later one steps back), it is renewed every half TTL while the request is still running, and it is released when the row is recorded; a hold left by a dead process stops being renewed and expires after OSR_PLATFORM_SPEND_RESERVE_TTL_S seconds. Every metered row also persists the plan it was sold under and the month's token volume before it, and a row the database refused when the request finished (an outage after the provider was paid) is written to usage-journal.jsonl on the data volume and replayed by the next finance pass with its original timestamp - a paid-for request is never invisible for longer than the outage. Repricing (reprice) prices such rows under the tariff in force at their own timestamp with those persisted inputs; only a row that tariff still cannot price (a broken rule closed after the fact) is priced under the current tariff, and the pass reports it as approximate rather than changing a charge silently. Settlement is cumulative and idempotent: each finance pass debits every wallet workspace everything charged through today minus everything ever debited as one usage entry whose idempotency key carries the running total - a sub-cent day accumulates, a month-end remainder is taken on the first pass of the next month, a row priced late is owed on the next pass, and two replicas settling the same state write one entry. The wallet alert warns at OSR_PLATFORM_WALLET_LOW_PCT of the last top-up.

The operator side lives under /api/v1/admin/finance: prices (list, add, end, prices/seed), rules (list, add, end), versions, quote (price a hypothetical request), alerts (the money alerts below, as the console's Margin watch shows them), wallets and wallets/sync (run the finance pass now: expire dead holds, reprice, settle), per-workspace ledger and billing-mode, providers (prepaid balance, credit limit, low-balance line, contract terms), providers/{id}/reconcile?period_start&period_end (expected spend from the metered rows against the invoices of the period, with the models behind it) and provider-invoices (record, then move to disputed / paid - settlement is recording paid). A prepaid provider whose projected balance crosses its low-balance line raises a deployment alert (provider_balance). Two more alerts guard the reseller's margin on the console health page and in the alert stream: margin - a model whose priced requests of the last 24 hours were charged below provider cost (critical) or under OSR_PLATFORM_MARGIN_ALERT_PCT gross margin (warning, default 5 %) - and above_list_price - one per provider, counting the enabled offers whose storefront rate per million tokens exceeds the vendor's catalogue list price (the worst one named), so the platform no longer beats buying direct there. Reads are open to every operator; writes need the finance role (price books, rules, ledger, billing mode, invoices, settlement) or provider_manager (provider accounts and pricing, the provider endpoints themselves - create, edit, delete, check, target mappings - and the offers listed on them) - superadmin passes everything. GET /api/v1/admin/analytics carries a finance block: usage revenue, provider cost, gross margin, the wallets and the breakdowns by day, provider, model, workspace and plan.

Models by the token: offers, prices, credits and provider preferences

Besides routing with auto, the platform sells named models by the token, the way OpenRouter does. You pick a model from the storefront, call it under its catalogue id, and pay the price shown per million tokens; the platform chooses which provider endpoint serves the call unless you pin one.

Three ids appear throughout this section:

An offer is an ordinary routing target: auto competes it with the rest of the catalogue, plans may use it, the learners grade it. Nothing is added to targets.yaml to sell a model.

Find a model and its price

The storefront at /models?view=offers lists every model on offer with two prices: the vendor's list price and the price your workspace pays, per million input and output tokens (cached input when the model supports it). Signed in, you see your own plan's prices; signed out, the deployment's public tariff. Each row opens the endpoints behind it with their health and last week's traffic.

The same data over the API:

GET /v1/models returns each model on offer with pricing.prompt / completion / input_cache_read in USD per token as strings, context_length, architecture, top_provider and supported_parameters - the fields OpenRouter clients read - plus an osr block with the endpoints (provider, upstream id, executable, healthy, region, data policy) and the vendor list_price. Tools written against OpenRouter's listing work unchanged.

The price you see before the call is the price recorded after it: usage is metered under the catalogue id even when the provider reports its own name (gpt-4.1 on Azure is openai/gpt-4.1), and the customer price comes from the pricing engine in the section above - a contract row of the provider in the price book beats the list price, then rules per deployment, plan or account apply.

Call a model by name

POST /v1/chat/completions (and the Responses and Messages dialects over it) accepts a catalogue id or an offer id in model, alongside auto and target ids. A models list may mix all three.

curl -s https://api.opensmartroute.ai/v1/chat/completions \
  -H "Authorization: Bearer $OSR_API_KEY" -H "Content-Type: application/json" \
  -d '{"model": "openai/gpt-4.1", "messages": [{"role": "user", "content": "Summarise this thread."}]}'

The response says which endpoint answered: model and the X-OSR-Target header carry the offer id (openai/gpt-4.1@azure), X-OSR-Charge-USD and charge_usd in the response's opensmartroute object carry what the call cost you, and the usage row records provider_id and upstream_model.

Key, workspace and tenant allow_targets / deny_targets may name a catalogue id; it stands for every endpoint of that model.

Steer the choice: provider preferences

Preferences travel in the request body under osr.provider. OpenRouter's top-level provider object is accepted as an alias and is never forwarded upstream.

{
  "model": "openai/gpt-4.1",
  "messages": [{"role": "user", "content": "..."}],
  "provider": {"order": ["azure", "openai"], "data_collection": "deny", "sort": "price"}
}

Preferences that exclude every endpoint of the named model are refused with 400 rather than silently routed elsewhere. An unknown key is a 400 too.

A workspace sets defaults for all of its requests with the policy (PUT /api/v1/policy, key provider_preferences, the same object, or the Providers block of the policy dialog); a request's own preferences are layered on top.

Pay as you go

The payg plan has no monthly fee and no request quota. Usage is paid from prepaid credits (billing_mode: wallet); when they run out, requests stop with 402. The abuse controls are the plan's requests_per_minute and the per-key spend limits below.

• Switch to it: POST /api/v1/billing/checkout {"plan": "payg"} applies the plan at once - no Stripe Checkout, no payment method - and puts the workspace on the wallet. It answers 409 while a subscription is live; cancel that first. Moving to a subscription plan later takes the workspace off the wallet again.
• Buy credits: the billing page offers fixed packs and a custom amount; POST /api/v1/billing/credits/checkout is the API. GET /api/v1/billing/wallet reports the packs, the purchase fee and the balance.
• Watch the balance from a client: every priced chat answer of a wallet workspace carries X-Wallet-Balance, the credits (in cents) one more request would be admitted against, so an integration can top up before the 402 arrives.

The fee is a second line item in Stripe Checkout and is booked on the ledger as a zero-sum pair of fee entries - collected, not spendable - so the balance carries only the credits and revenue analytics separate the fee from the spread.

Auto top-up. A wallet workspace can have credits bought for it before the balance reaches zero:

1. Buy credits once with save_payment_method: true (the billing page: Keep this card for auto top-up). The Checkout keeps the card on the workspace's Stripe customer for off-session charges; the webhook stores its label (visa 4242), never the number. Passing auto_topup: {amount_cents, threshold_cents} on the same purchase switches auto top-up on as soon as it completes.
2. Set or change the rule with PUT /api/v1/billing/wallet/auto-topup {amount_cents, threshold_cents} - buy amount_cents whenever the available balance falls to threshold_cents or below (whole dollars within the purchase bounds, the threshold below the amount; 409 without a saved card). DELETE turns it off; ?forget_card=true also detaches the card at Stripe. Owners and admins only.
3. The finance pass (the same one that settles usage, several times a day; POST /api/v1/admin/finance/wallets/sync runs it now) charges the saved card off-session and credits the wallet under the PaymentIntent id, so the payment_intent.succeeded webhook for the same intent is a no-op. One attempt per cooldown, whatever the outcome: a declined card is recorded as auto_topup.error, raises the wallet:auto_topup alert on the billing page and in the workspace alerts, and is not retried until the cooldown passes or the rule is saved again.

GET /api/v1/billing/wallet reports the rule under auto_topup (enabled, amount_cents, threshold_cents, payment_method, last_attempt_at, error, available - the deployment has Stripe). Every step is in the audit log: billing.payment_method_saved, billing.auto_topup_changed, ledger.auto_topup, billing.auto_topup_failed.

Referrals: give credits, get credits. Every workspace has a referral link, <site>/r/<code>, on Refer a team in the dashboard (GET /api/v1/referrals: the code, the link, the terms, the referrals so far and what they earned; POST /api/v1/referrals/invite e-mails the link to one address with a line from the sender, twenty a day). The site remembers the code the way it remembers the first touch and hands it to the signup (attribution.ref on POST /api/v1/signup and on a sign-in with a provider that creates the account); the latest referral link a browser opened is the one that counts. A referral is signed up when the new workspace exists, qualified when its owner has confirmed the address and the workspace has made its first successful request, and rewarded when the finance pass has credited both wallets - OSR_PLATFORM_REFERRAL_REWARD_CENTS to the referrer and OSR_PLATFORM_REFERRAL_WELCOME_CENTS to the new workspace (both 500 by default), as promo ledger entries that name the referral. A referrer earns at most OSR_PLATFORM_REFERRAL_MAX_REWARDS rewards (20; 0 = no cap) - a referral past the cap is capped and the invited side still gets its welcome credit. Self-referrals (the same person or address, a member of the inviting workspace) are not recorded and a workspace is referred once. GET /api/v1/referrals/lookup?code= tells a visitor on a link whether it is live and what they are offered, naming the inviter by initials only; the inviter sees the invited workspaces by initials. OSR_PLATFORM_REFERRAL_ENABLED=false switches the programme off (links keep working as plain sign-up links); all four are run-time settings. Operators read the programme on GET /api/v1/admin/referrals and run the qualification pass with POST /api/v1/admin/referrals/sweep. Events: referral.signed_up, referral.rewarded, referral.capped, referral.invited.

Money that moves is announced. Whenever credits land on the wallet - a purchase at checkout or an automatic top-up - the owners receive Credits added (outbox kind credits_added: the amount, the balance now, the source and any purchase fee) and the workspace inbox gets workspace.credits_added; a Stripe refund sends Payment refunded (credits_refunded, the balance after it) and files workspace.credits_refunded. Each goes out once per ledger entry, whichever of the finance pass or the webhook wrote it. Both are account mail and always go out.

Per-key spend limits

limits.spend_limit_cents (per calendar month) and limits.spend_limit_daily_cents (per UTC day) on POST /api/v1/keys cap what one key may charge the workspace: charge recorded so far, plus the charge side of its requests in flight, plus the bound of the request being admitted. A request that would pass the cap is refused with 429, X-Key-Limit: spend_limit_cents and the X-Budget-* headers in dollars. Budgets (daily_budget_usd) remain the cap on provider cost.

Operators: list, price and pause offers

The console page Offers (/platform/admin/offers) does everything in this section without curl.

1. Connect a provider under Providers and press Check. The check calls the provider's /models and reports, under offers, which upstream ids the reference catalogue recognises.
2. List models. One click creates an offer for every recognised id. Upstream ids that resolve to the same card are one offer (gpt-4.1 and gpt-4.1-2025-04-14 both mean openai/gpt-4.1). Azure and Bedrock do not return deployments from /models: name them in the models field.
3. State the data policy the provider publishes - trains, retains_days, logs_prompts, statement_url, stated_at. This is what data_collection: deny reads; an empty policy never excludes the endpoint.
4. Set the price on the Finance page: a contract row for the provider in the price book and the mark-up rules. Without either, the vendor's list price stands.
5. Pause or remove an offer from the same page when a provider is unreliable or a contract ends. Paused offers leave the storefront and GET /v1/models at once.

The same over the API:

OSR_PLATFORM_OFFERS_AUTO_LIST=true lists the recognised models on every provider check without the button.

An Ollama server offers only the models it is mapped to run. Several Ollama servers often share one model store, so each one's /api/tags lists every model on the disk while only the mapped one is resident and kept warm; an offer for any other model would send a chat into a cold load of tens of gigabytes and time out. List models therefore creates offers only for the upstream ids the server's target mappings name (the rest are returned under unserved), and every sync pauses an enabled offer whose server is not mapped to run the model, with the reason in the offer's notes. A provider is recognised as an Ollama server by its kind, by port 11434 or by a name starting with ollama-. Providers without mappings are not restricted.

A provider you cannot pay is unhealthy, not expensive. When a prepaid provider's projected balance is exhausted (see Usage pricing, credits and provider finance), every target it executes - offers and hand-written mappings alike - leaves the pool auto routes over, a request pinning one of them answers 503 with Retry-After, the storefront and GET /v1/models mark the endpoint as not executable, and the provider_balance alert names the provider on the console health page and in the alert stream. A workspace's own provider (BYOK) is never affected: it pays for its own calls.

Free trials, coupons and the launch offer

A coupon grants a paid plan for a number of days without payment - no Stripe, no payment method. Redeeming one puts the workspace on the plan at once and records account.trial_end (and coupon_code) on GET /api/v1/me; when the trial ends without a subscription having taken over, the workspace returns to free (on its next request and by a sweep every OSR_PLATFORM_TRIAL_SWEEP_S seconds, default 15 minutes; audit entry billing.trial_ended). The owners hear at each step, whether or not the deployment has Stripe: Your (plan) trial has started the moment it begins (outbox kind trial_started: until when, that no card was taken, what happens at the end), Your (plan) trial ends on (date) three days before (trial_ending, the lifecycle campaign) and Your (plan) trial has ended when the sweep moves the workspace back (trial_ended); the workspace inbox carries the same (workspace.trial_started, workspace.trial_moved when the trial moves to another plan, workspace.trial_ended) and each is a plan.changed event (reason trial, trial_moved, trial_expired) for the subscribed channels. While a coupon trial runs the quota is the plan's, overage is not billed (there is no subscription), and the workspace may move to any other plan the coupon covers with the same end date. A Stripe subscription that completes during the trial clears the deadline. Coupons work on deployments without Stripe.

The plans' own free trials ride the same machinery. Every subscription plan whose trial_days is set and whose trial_needs_card is false (the shipped catalogue: Pro, Team and Enterprise, 90 days each) is carried by the built-in promotion FREE-TRIAL, which the API keeps in step with the catalogue at start and whenever an operator changes a plan (PATCH /api/v1/admin/plans/{name}): it covers exactly those plans, advertises the longest of their trials and grants each plan its own number of days. The first POST /api/v1/billing/checkout of a workspace therefore starts the trial at once, without Stripe and without a card - on a deployment that has no Stripe at all, and on one whose Stripe account cannot charge yet. One trial per workspace: a workspace that had any trial or subscription before is refused (409) and pays instead. GET /api/v1/info shows the promotion with builtin: true; the running trial reports coupon: null; the operator console lists the row as created by plans and cannot type over its terms for long. Set every plan's trial_days to 0 and the promotion switches itself off.

The launch offer is an operator's coupon on top, created at start-up from the environment and applied automatically (the newest live promotion wins a checkout):

An auto-apply coupon is the promotion: GET /api/v1/info advertises it as promotion (code, trial_days, plans, valid_until, description, builtin), the pricing page and the plan cards say "N-day free trial", and POST /api/v1/billing/checkout applies it to every eligible workspace instead of starting a Stripe Checkout - nobody has to type the code. Eligible = the coupon covers the plan, is enabled, within its deadline and cap, the workspace has no live subscription and has not redeemed this code before (for the built-in trial: has never had a trial). The answer then carries coupon, builtin, trial_days, trial_end and a url back to /platform/dashboard/billing?redeemed=<plan>. Send "coupon": "" to skip the trial and the promotion and pay (the Subscribe now button on the trial card), or "coupon": "<code>" to use a specific one.

The dashboard's Plan and billing page shows the running trial (days left, end date, Subscribe now when Stripe is configured), marks the plans the promotion covers, and has a Have a coupon code? form; the operator console manages codes at /platform/admin/coupons.

7. Organizations

Signup creates a personal workspace. An organization is a second kind of workspace with its own members, roles, API keys, tenants, plan and usage; one user can belong to several and switches between them in the dashboard (/platform/dashboard/workspace, POST /api/v1/workspaces/{account_id}/switch). Creating one (POST /api/v1/workspaces with name and an optional slug) makes you its owner and issues its first API key. A personal workspace that already carries a paid plan, keys and history can instead become an organization in place: POST /api/v1/workspace/convert with name (and an optional slug) changes only its kind - plan, subscription, keys, tenants and usage stay on the same row, the browser session stays on it, and the user receives a fresh free personal workspace alongside (audit workspace.converted). The Members page of a personal workspace offers exactly this, so the seats a team or enterprise plan sells become usable without moving anything.

Roles are member < admin < owner:

API keys belong to the workspace and act with its full rights; the role model applies to signed-in browser sessions. Members are invited by email (POST /api/v1/workspace/invites): the invitee gets a message with the link when the deployment has a mail server, and the platform returns the same link once to the inviter so it can be shared directly. POST /api/v1/workspace/invites/{id}/resend sends a pending invitation again with a fresh link and a full validity period (the previous link stops working; audit member.invite_resent); invitations that expired without being accepted are purged by the retention sweep. Seats and tenants are limited per plan (max_members, max_tenants in GET /api/v1/info). An organization always keeps at least one owner. Transferring ownership is one call: POST /api/v1/workspace/transfer with the new owner's user_id and the role the current owner keeps (keep_role: admin - the default -, member, owner to share ownership, or leave to hand over and go), owner only, audited as workspace.transferred, both people e-mailed; the crown button on Members does the same. There is no per-workspace "suspend" for a person: remove the member (their sessions on the workspace end at once, the invitation can be sent again later), or an operator disables the whole user account.

Organization SSO (enterprise) lets people sign in with the company identity provider: PUT /api/v1/workspace/sso with kind (github, google, microsoft, gitlab, linkedin or oidc with an issuer), client_id, client_secret, the allowed email domains and the default_role given to anyone who signs in from one of those domains. The workspace needs a slug first; the sign-in page shows the provider as org-<slug>. A role_map ({"claim": "groups", "map": {"osr-admins": "admin"}, "require_match": false}) derives the workspace role from the provider's group claim instead: the mapping is re-applied at every sign-in (a user added to or removed from a mapped group changes role on the next login, recorded as member.role_sso in the workspace audit; the last owner is never demoted), unmatched users get default_role, and require_match: true refuses anyone outside the mapped groups. Entra groups / roles / wids, Okta groups, Keycloak realm_roles and Cognito cognito:groups are read from the id token and userinfo; ask the provider for the claim with the connection's scopes (groups).

SCIM provisioning (organizations) is the push side of the same identity provider: single sign-on lets a person in when they sign in, SCIM 2.0 (RFC 7643 / 7644) lets Entra ID, Okta, OneLogin, JumpCloud and the like create the account before the first sign-in and remove it the moment HR closes it. An owner mints the provisioning token on Workspace › Single sign-on (Provisioning (SCIM); POST /api/v1/workspace/scim/token, shown once, valid a year, one live token per organization - minting again replaces it, DELETE revokes it) and pastes it with the base URL <api>/scim/v2 into the provider's SCIM connector. The connector then calls:

Errors come back in SCIM's own shape (urn:ietf:params:scim:api:messages:2.0:Error with status, detail, scimType). Every change is in the workspace audit trail with identity provider (SCIM) as the actor - member.provisioned, member.deprovisioned, member.role_scim, user.scim_updated - and a deprovisioning files the same member removed notice in the workspace inbox as a removal by hand. GET /api/v1/workspace/scim (admin role) shows whether provisioning is on, the token's age and use and how many people came through it.

Not offered, by decision rather than omission: SAML (every supported identity provider - Entra, Okta, Google, Keycloak, Cognito - also speaks OIDC, which is what the oidc connection kind uses; SAML-only providers can front the platform through an OIDC bridge) and the OpenAI Batch API (asynchronous batch files - the router prices and routes each request live; send them concurrently).

Teams

A large customer does not want one admin minting every key. A team groups some of an organization's members - a product squad, an agency's client desk, the data-science group - and gives that group its own place in the workspace:

• Tenant binding. A team may be bound to one of the workspace's tenants. Keys minted for the team are pinned to that tenant, so its constraints, budgets and usage roll-up apply, and the team's spend shows on the tenant. GET /api/v1/workspace/teams reports each bound team's requests, tokens and cost this month and today. A tenant bound to a team cannot be deleted until the team is unbound (409).
• Leads. A team member with the team role lead manages the roster (adds and removes members, not other leads) and the team's keys - mints them (POST /api/v1/keys with team: "<slug>"; the key carries team and is pinned to the team's tenant), renames, rotates and revokes them - without holding the workspace admin role. Only admins make leads. This is the delegation that lets a customer run its own people: the admin creates the team, binds it and names a lead; the lead does the rest. A team with keys cannot be re-bound to another tenant (409; revoke its keys first); deleting a team detaches its active keys (they keep working as the tenant, managed by admins from then on - keys_detached in the answer).
• Attribution. GET /api/v1/workspace/members lists every person's teams, GET /api/v1/me the caller's own (teams), and the audit trail records team.created, team.updated, team.deleted, team.member_added and team.member_removed (also notification events).

Leaving or being removed from the workspace drops every team membership; deleting the organization drops its teams. Personal workspaces have no teams (400). The Members page carries the Teams card (create, bind, roster, leads) and shows each member's teams; the Tenants page names the teams that run under each tenant; the key dialog on API keys offers the teams the caller may mint for. Operators see the teams and their rosters on the workspace's console page (teams on GET /api/v1/admin/accounts/{account_id}), and the data export (GET /api/v1/me/export) carries them.

Regions

The hosted platform runs as independent stacks per region - an EU stack and a US stack - each with its own database, so a workspace's keys, usage and traces never leave the region it was created in. A stack declares its region with OSR_PLATFORM_REGION (eu, us; unset = a single-region deployment that enforces nothing). Every workspace created on a stack - at signup, POST /api/v1/workspaces or by an operator - is pinned to that region (region on the account in GET /api/v1/me and GET /api/v1/info reports the stack's region and regions_enforced), and API keys are minted where the workspace lives.

Using a key on the other stack answers 403 with code: "region_mismatch", region: "<r>" and detail: "this workspace is pinned to region '<r>'; use the <r> endpoint"; browser sessions get the same answer, which is how the dashboard tells you your workspace lives in the other region. Every authenticated response carries X-OSR-Region with the answering stack's region, so a client can check it is talking to the right one. Operators are not pinned, and public endpoints work on every stack.

Deploying two stacks = two deployments of the same template with OSR_PLATFORM_REGION set to eu / us and DNS such as eu.api.<domain> / us.api.<domain> in front of each. Moving a workspace is deliberate: an operator changes its region (PATCH /api/v1/admin/accounts/{account_id} with {"region": "us"}, audited as account.region_changed; "" unpins it), and the data itself is exported from one stack and imported on the other by hand (GET /api/v1/me/export). This is not the routing constraint of the same name: constraints.region on a request (or a tenant's region) chooses targets deployed in a region; the workspace region chooses the stack that accepts the key.

8. Tenants

Tenants (enterprise) apply a fixed set of hard constraints per customer, team or region without the caller repeating them. Create a tenant, then send its slug in X-OSR-Tenant (or osr.tenant).

curl -s -X PUT "$OSR_URL/api/v1/tenants" -H "Authorization: Bearer $OSR_API_KEY" -H "Content-Type: application/json" \
  -d '{"slug": "eu-health", "config": {"region": "eu", "data_boundary": "private", "max_cost_per_1k": 0.01, "deny_targets": ["llm-frontier"]}}'

curl -s "$OSR_URL/api/v1/route" -H "Authorization: Bearer $OSR_API_KEY" -H "X-OSR-Tenant: eu-health" \
  -H "Content-Type: application/json" -d '{"text": "Anonymise this discharge summary ..."}'

Tenant config keys: region, data_boundary, max_cost_per_1k, max_latency_ms, preferred_max_latency_ms, deny_targets, allow_targets, contains_pii, daily_budget_usd, monthly_budget_usd, log_payloads, marketplace_discovery, data_protection, output_guard, allowed_ips. Explicit request constraints win over the tenant defaults; the workspace policy (next section) applies on top of both. Executed spend is attributed to the tenant, so GET /api/v1/governance shows spend and budgets per tenant, and GET /api/v1/tenants returns each tenant with usage (month and today: requests, tokens, cost_usd), its budgets with what is used and remaining, and keys - the number of API keys pinned to it. GET /api/v1/usage adds by_tenant (requests, tokens and cost per tenant slug; null is the workspace's own traffic) and ?tenant=<slug> narrows the daily series and the target and endpoint breakdowns to one tenant; GET /api/v1/audit?tenant=<slug> keeps only that tenant's hash-chained records, so a per-customer audit extract is one export away. Audit reads are bounded: each chain file is read from its tail and the scan stops once limit matching records are in hand, and next_before in the answer is the cursor for the following page (GET /api/v1/audit?limit=100&before=<ts>). The Activity, Usage and Audit trail pages offer the tenant filter.

Tenant-pinned keys are the isolation boundary for a customer or team you hand a key to: POST /api/v1/keys with {"name": "acme", "tenant": "acme"} mints a key whose every request runs as that tenant - the header is not needed, naming another tenant answers 403, and the key can never act as the whole workspace. Its requests count against the tenant's budgets and appear under the tenant in usage and audit. The boundary holds on every surface: the key routes, executes and reports outcomes as its tenant; it reads only its tenant's rows on /activity, /trace/{id}, /events, /usage, /usage/report, /savings, /audit, /handoffs, /keys, /tenants, /me and /governance (another tenant's request id answers 404, ?tenant=<other> answers 403); every workspace-management call - keys, policy, tenants, providers, members, channels, billing, listings and the other reads of the workspace - answers 403 with X-Key-Tenant: <slug>. A tenant with active pinned keys cannot be deleted (409); revoke them first. The API keys page offers the tenant on creation and shows the scope of every key.

Fair share (enterprise, OSR_PLATFORM_FAIR_SHARE=true) keeps one workspace or tenant from crowding out the others on a shared deployment: the SDK's dominant-resource-fairness middleware watches requests, tokens and cost per workspace and per tenant over a sliding window (OSR_PLATFORM_FAIR_SHARE_WINDOW_S, 300 s), steers whoever takes more than OSR_PLATFORM_FAIR_SHARE_SOFT (1.5) x its share to cheaper targets and caps premium targets above OSR_PLATFORM_FAIR_SHARE_HARD (3) x - the trace shows fairshare.throttled / fairshare.capped events. GET /api/v1/governance -> fair_share reports the window, the counters and the dominant shares of the caller's own workspace and tenants only; controls.fair_share says whether it is on. Operators see every tenant across workspaces on /platform/admin/tenants and can edit a tenant's configuration or delete it there (PUT|DELETE /api/v1/admin/accounts/{account_id}/tenants/{slug}).

9. Governance: workspace policy, budgets and controls

A workspace policy applies to every request of the workspace, whichever key or tenant sent it. It takes the same keys as a tenant plus budgets, and needs the admin role:

curl -s -X PUT "$OSR_URL/api/v1/policy" -H "Authorization: Bearer $OSR_API_KEY" -H "Content-Type: application/json" \
  -d '{"config": {"data_boundary": "private", "deny_targets": ["llm-frontier"], "monthly_budget_usd": 250}}'

GET /api/v1/policy returns the policy, the accepted keys and the current budget lines; DELETE removes it. Constraints merge deterministically: a boundary or region fills in when the request has none, cost and latency caps take the stricter value, deny lists are merged and allow lists intersected. Unknown keys, unknown target ids and an id present in both allow_targets and deny_targets are rejected with 400.

Budgets count executed spend (execute: true, /v1/chat/completions, MCP ask) - quotes and route-only calls are free. daily_budget_usd resets at midnight UTC, monthly_budget_usd on the first of the month. Once a workspace or tenant budget is exhausted, routed calls return 429 ... budget ... exhausted with Retry-After and the X-Budget-* headers until the period rolls over or the limit is raised.

Request logging is off by default: the platform keeps the decision facts of every request (signals, ranking, policy rejections, execution steps, tokens, cost) but no prompt or answer text. A workspace that wants to read them next to each trace opts in with log_payloads in the policy (or per tenant): redacted stores the prompt and the answer with PII masked first (opensmartroute.security.Redactor: e-mails, phone numbers, card and ID numbers become <EMAIL_1>-style placeholders) and credential shapes masked as well (API keys, cloud and repository tokens, private keys become <SECRET_1>), full stores them as sent. Each text is cut to OSR_PLATFORM_REQUEST_LOG_MAX_CHARS characters (truncated marks the row). The stored payload appears as payload on GET /api/v1/trace/{request_id} and on the trace drawer of the Activity page, GET /api/v1/activity marks the rows that have one (logged) and reports the mode in force (logging), the data export carries every row (request_logs), the rows follow the usage retention, and DELETE /api/v1/activity/logs (admin) erases them all while the activity rows, traces and outcomes stay. The governance posture flags full logging as a warning and redacted as an informational finding.

Data protection decides what happens to personal and health data before the router or any provider sees the text - the policy key data_protection (per workspace, or per tenant in the tenant's config, which wins):

{"config": {"data_protection": {"mode": "mask", "entities": ["pii", "phi"], "restore": true}}}

entities chooses what counts: the groups pii (e-mail, phone, card, national ids, passport), phi (medical record number, NPI, DEA, Medicare id, insurance member id, ICD-10 code, date of birth) and financial (card, IBAN), or single names; the default is everything. Every response carries X-OSR-Data-Protection: <mode> when something was found, and the usage row's facts.pii records the mode, the entity counts and whether the text was masked (GET /api/v1/activity, the trace drawer) - the values themselves are never stored. Under mask the stored request log (log_payloads) keeps the masked text. The same facts object carries cache: hit, fallback_from (the targets that failed before the answer came from another one) and upstream_request_id - the provider's own id of the call (x-request-id at OpenAI, request-id at Anthropic, apim-request-id at Azure), what a support ticket with the provider needs.

Output guard decides what happens to an answer before the customer sees it - the policy key output_guard (per workspace, or per tenant, which wins): {"mode": "redact", "pii": true, "secrets": true, "injection_threshold": 0.7}. Every executed answer, streamed or not, is screened for leaked secrets (cloud and API keys, tokens, private keys), personal data and instruction injection. flag reports the finding and leaves the text; redact replaces secrets and personal data by placeholders (<SECRET_1>, <EMAIL_1>) - a streamed answer is held until it was screened, then sent; block refuses an answer that carries a secret or an injection with 400 and redacts personal data otherwise; off (default) leaves answers alone. pii: false keeps personal data in place (a support answer may quote an address) while secrets still go. The finding - kinds and counts, never the values - is on the answer (the output_guard field of the response's opensmartroute block on chat completions, result.output_guard on /api/v1/route), in the X-OSR-Output-Guard header (redact:secrets=1,pii=2) and on the usage row as facts.output_guard. Dashboard: Governance › Controls, next to the data protection.

Allowed addresses fence every routed request of the workspace: allowed_ips in the policy (addresses or CIDR blocks, at most 32) refuses a request from anywhere else with 403 and X-Policy-Limit: allowed_ips before anything is routed, judged on the client address past the trusted proxies - never a header the caller wrote. A tenant's own allowed_ips replaces the workspace's for its requests. Management calls (the policy itself, keys, billing) are not fenced, so a locked-out address can always be fixed; allowed_ips on one key (above) fences that key everywhere.

Marketplace discovery decides which published skills and personas a plan may use without an install (section 11): marketplace_discovery is verified (the deployment default unless the operator changed it - only listings an operator marked verified), all (every published listing) or off (the workspace's installs only). Discovered companions fill plan slots and never answer on their own; every decision names the listings it used under marketplace, each with installed: true|false.

Default objective. objective in the policy ({"quality": 1, "cost": 0.6, "latency": 0.1}, any of quality, cost, latency, quality_floor) is the objective of every request that sets none; a request's own objective still wins. It is what promoting a shadow candidate writes (the Evaluation page, section above).

Response cache. response_cache_ttl_s (seconds, up to seven days) answers an identical chat request the workspace already paid for from the store instead of a provider: the same model name, messages, options and tenant hash to one key, the stored answer comes back with zero tokens and cost on its usage row, a cache entry ({"hit": true, "stored_at": ...}) in the answer's routing block and an X-OSR-Cache: hit | miss | bypass header. A caller shortens the life of one answer with X-OSR-Cache-TTL: 60 (never lengthens it past the policy), skips the lookup with X-OSR-Cache: bypass, keeps an answer out of the store with X-OSR-Cache: no-store, or sets osr.cache: false in the body. GET /api/v1/cache reports hits, misses and what the hits saved; DELETE /api/v1/cache (admin) empties the workspace's entries. The store is Redis when the deployment has one, else the database.

response_cache_mode: "similar" adds near-duplicate matching: every entry also keeps a context key (the request with its last user message blanked - model, earlier turns, tools, sampling) and a signature of that message (hashed character shingles of its normalised text, never the text), and a request that misses the exact key is compared with the entries of the same context and served from the closest one whose similarity reaches response_cache_similarity (0.5-1, default 0.85). That catches the rewordings that make up most repeats - case, punctuation, a typo, a word - without an embedding call per request; the answer's cache entry says "match": "similar" with the score, which X-OSR-Cache-Similarity carries too. It is surface similarity, not meaning: two questions that differ in one decisive word can score high, so keep the threshold where your traffic makes it safe and leave exact for workspaces where that cannot be tolerated. Both keys are in the policy dialog.

Session affinity. session_affinity: true keeps a conversation on the target that served it: a request that carries a session (context.session_id, or the hosted osr.conversation_id) routes to the target of the previous turn while that target stays executable, allowed by the policy and not behind an open breaker; an explicit model in the request still wins. Agent loops stop flipping models mid-task and prompt caches on the provider side keep their prefix.

Aliases. aliases maps the model names your clients already send to what the router understands - a target id, a catalogue model, an offer (vendor/model@provider) or a preset (osr/auto:cheap): {"gpt-4o": "auto", "claude-3-5-sonnet": "llm-frontier"}. Resolution is one step and happens before everything else on /v1/chat/completions (model and models), so a code base migrates to the router without a search-and-replace of model names.

Answer judge. judge turns on the LLM judge for a share of the executed answers: {"sample": 0.1, "target": "llm-frontier", "threshold": 6, "endpoints": ["chat"], "targets": ["llm-small"], "max_per_day": 500}. The request id decides the sample (reproducibly), the judge model grades the answer against the prompt (0-10 with a reason), the grade lands in the feedback table as quality with source: judge:<model>

• exactly where a hand-reported outcome lands, so the learners, the rankings and the replay comparison use it - and the verdict with its reason stays on GET /api/v1/evaluations/judge. Judging is a metered call of its own (endpoint: judge) and runs on a thread after the answer has left; max_per_day bounds it. Section 5 has the endpoints.

GET /api/v1/governance is the compliance view of the workspace in one payload:

GET /api/v1/governance/report (admin role) is the evidence bundle behind that view: the governance payload plus the members and their roles, the SSO connection (client id and role map, never the secret), every active key with its scopes, limits, expiry and last use, the platform's identity (brand, versions, region) and who generated the report when; ?format=markdown downloads it as a Markdown document for an audit file. Every generation is recorded in the workspace's audit log (governance.report).

The dashboard renders the same data at /platform/dashboard/governance - the posture card links each finding to the page that fixes it, the guard-activity table links into the events page filtered on the event, the key-hygiene card into the keys page, and "Compliance report" downloads the Markdown evidence - with a policy editor for admins.

10. Observability and caching

The platform observes itself: every span, event, counter and alert below is produced, stored and served by the platform - there is no external metrics system, tracing backend or alerting tool to run. Every deployment exposes liveness and readiness probes and the counters without a key:

Responses carry X-Request-Id (send your own to correlate with your logs) and Server-Timing; the API writes one JSON access-log line per request (osr.platform.access) with the route template, status, latency and account - never the prompt. GET /api/v1/stats (enterprise) adds http, controls, cache and observability snapshots; /platform/dashboard/health shows them live.

Time series and alerts

The overview shows the last 24 hours (traffic, latency) and the alerts; /platform/dashboard/health adds the deployment series and the same alerts, refreshed with the page.

Notifications: alert delivery

Alerts are evaluated by the API itself - no external scheduler. Every OSR_PLATFORM_ALERTS_INTERVAL_S seconds (default 60; 0 turns the loop off and leaves Evaluate now to operators) one replica - the one holding the osr:alerts:leader lease in Redis, or the only one without Redis - re-evaluates the workspace rules for every workspace that was active in the last 15 minutes or has a channel or an open alert, plus the deployment rules. Each condition becomes a notification episode: opened when the rule starts firing, refreshed while it stays on (an escalation from warning to critical is delivered again), resolved when it clears. Episodes land in an inbox with unread counts per severity, are published to the alerts Kafka topic in cluster mode, and are delivered to the workspace's channels:

A channel filters by minimum severity (info, warning, critical) and optionally by rule names, can be disabled, records its last delivery and last error, and has Send test. Failed webhook deliveries are retried on the following evaluation rounds up to three attempts. Up to 20 channels per workspace; creating or changing them takes the admin role. Operators have the same inbox and channels for the deployment scope (readiness, 5xx rate, p95, circuit breakers, drift, autopilot, SLM age, tracing) at /platform/admin/notifications.

Account notices. The same inbox also carries what happened to your own account, so a security message does not depend on a mailbox that may be filtered, forwarded or shared. Each one arrives as an entry with state: "notice" (it never fires or resolves - it is simply read), a rule of security.*, and a link to the page that acts on it; the Account filter on /platform/dashboard/notifications shows only these. Every one of them is sent by e-mail as well, under the security category that cannot be switched off.

A notice belongs to one person: other members of the organization never see it, it follows its owner from one workspace to the next, and it is not in the operator console's deployment inbox. Notices are not delivered to channels either - a workspace Slack room is not the place for one person's sign-in.

Workspace notices. The same mechanism records what happened to the workspace, under workspace.* rules and scope: "workspace", so the people who live in the dashboard see it even though only the owners are mailed. Every member of the workspace sees these; they stay in the workspace they belong to.

Platform events. Besides alerts, a webhook, slack or teams channel can subscribe to what happens in the workspace: events on the channel is a list of event names or patterns ("*", "key.*"), validated against event_types in the channel listing. The event types are request.completed / request.failed (one per metered request - high volume), outcome.reported, handoff.pending / handoff.resolved / handoff.cancelled, key.created / key.revoked / key.rotated / key.scoped / key.limited, member.invited / member.joined / member.left / member.removed / member.role, team.created / team.updated / team.deleted / team.member_added / team.member_removed, plan.changed, billing.invoice_paid / billing.payment_failed (as Stripe confirms the subscription invoice), account.deleted, support.opened / support.answered / support.replied / support.closed, governance.report and workspace.reports. Every event is something the platform recorded - it is queued in the outbox on the request path (one insert) and delivered by the next evaluation round with the same envelope and X-OSR-Event / X-OSR-Delivery / X-OSR-Signature headers as alerts; GET /api/v1/notifications/deliveries shows the attempts. The same publications reach the Kafka topics below when a broker is configured, so an external consumer sees the deployment-wide stream and a workspace only its own events.

Shared state and replicas

The platform keeps its state in PostgreSQL: OSR_PLATFORM_DATABASE_URL is required, and every API replica and domain service connects to the same database. The data volume only holds what is local to a process (learner state, the audit chain, the SLM bundle, the autopilot cache). The hash-chained audit trail has one chain per process: the API appends to audit.jsonl, each domain service to audit.<service>.jsonl (audit.routing.jsonl, audit.openai.jsonl, audit.mcp.jsonl), because a hash chain has exactly one writer; /api/v1/audit, the trace view and the governance status verify every chain on the volume and merge the records (chains counts them, problem names the broken file). On a clustered deployment (Redis set, several replicas of one role) each replica writes its own file, audit.<role>.<hostname>.jsonl, because a file lock on a network share (Azure Files mounted nobrl, NFS) is not honoured across hosts and two replicas appending to one file interleave and break the chain; the readers merge all of them and the go-live checklist (Audit trail) shows whether every chain verifies. On a single-host deployment two processes of one role share a chain file safely: every append takes an exclusive file lock and continues from the last line on disk, so a second writer extends the chain instead of forking it. A chain that no longer verifies (a line removed, a record edited, a fork written before 1.8.1) is a critical deployment alert (audit_chain_broken) and every workspace's compliance posture shows it; a superadmin resolves it from the console's Health page (POST /api/v1/admin/audit/rotate with a reason): the broken file is kept beside the new one as <file>.<stamp>.archived, never edited, and the new chain opens with a chain.rotated record naming the archive, the problem, the reason and the operator - the rotation is itself on the record and in the operator audit log. Spans and events are persisted by every process that routes or executes (routing, openai, mcp) and read by the API, so a chat completion served by the OpenAI service is traceable at /api/v1/trace/{request_id} like a /route decision. A deployment that runs several replicas moves the remaining shared parts to services that are themselves containers - no managed cloud database, cache or broker is required, and the same images run on a laptop, a Kubernetes cluster or Azure Container Apps:

GET /api/v1/admin/overview -> storage says which mode a deployment runs ({database, clustered, redis, events, topics}; clustered is true once Redis shares the rate limits, topics is the full list above as the broker sees it) and the operator console shows it on /platform/admin. The public GET /api/v1/info never names where state lives: that is an operator fact. Event payloads carry the same fields as the activity log and never the prompt text. The topic list is one constant (cluster.TOPICS): the Azure template creates exactly these event hubs for a managed broker (Event Hubs does not auto-create topics), a self-hosted broker creates them on first use, and a test fails when code publishes on a topic that is not in the list.

An authenticated broker - Event Hubs, Confluent Cloud, any SASL or TLS endpoint - is reached with OSR_PLATFORM_KAFKA_SECURITY_PROTOCOL (SASL_SSL, SASL_PLAINTEXT, SSL), OSR_PLATFORM_KAFKA_SASL_MECHANISM (PLAIN, SCRAM-SHA-256, SCRAM-SHA-512) and the OSR_PLATFORM_KAFKA_SASL_USERNAME / OSR_PLATFORM_KAFKA_SASL_PASSWORD pair; for Event Hubs that is the literal user $ConnectionString and the namespace connection string. Such a gateway hangs up on a connection left idle for a few minutes - and so does the network in front of a self-hosted broker (the internal ingress of Azure Container Apps after four) - while the driver would hold one for nine, so the clients, whatever the protocol, release theirs first after OSR_PLATFORM_KAFKA_CONNECTIONS_MAX_IDLE_MS (default 180000, three minutes; 0 leaves the question to the driver). Without it a quiet topic finds the socket already gone on its next read: no event is lost - the producer retries and the consumer rebuilds - but the log fills with Connection lost: socket disconnected and ConnectionResetError: [Errno 104] Connection reset by peer under Error receiving network data, once per idle connection, every few minutes in every service.

Both shared services degrade rather than stop the API. When Redis is unreachable the rate limiter answers from its per-replica window and stays off Redis for a five-second cool-down before trying again (/readyz redis shows ok, latency_ms or the error); the shared learners keep learning in memory - a refresh that cannot read Redis leaves the replica's state untouched instead of blanking it, and the next successful save catches up. A request never waits for Kafka: emit only enqueues, one sender thread per process hands the events to the producer (queue bounded at 10 000 events; beyond that the newest are dropped and counted as events.overflow). Events are acknowledged by every in-sync replica (acks=all), retried with back-off and dropped - counted and reported as events.dropped / queued / last_error - only when the broker stays unreachable. The SLM service consumes training and feedback with manual offset commits after each ingested batch (at-least-once: a record is never committed before it was processed) and reports the consumer group, commits, errors and per-topic lag under consumer in GET /model. The self-hosted Compose stack starts PostgreSQL with the platform; the Azure template provisions it as a container app with internal TCP ingress next to the API and web app.

A deployment that predates 1.1 kept its state in a platform.sqlite3 file on the data volume. Import it once with osr-platform-import-sqlite /data/platform.sqlite3 --database-url postgresql://... (ships with the platform API; run it inside the API container - safe to re-run; rows already in PostgreSQL are left alone), then remove the file.

In that stack the routing SLM is its own service too: the API publishes each routed prompt to the training topic (opt-in, OSR_PLATFORM_TRAINING_EVENTS) together with the decision it took and the ranked propensities of the top candidates (target_id, ranked); the SLM service remembers prompts, decisions and workspace in autopilot/requests.jsonl on the shared volume - a restart or a deploy loses nothing - pairs prompts with the outcomes reported through POST /api/v1/feedback (feedback rows) and distils the ensemble's ranking on every routed prompt that has no outcome yet (decision rows, newest OSR_SLM_DECISION_ROWS_MAX), trains a challenger on a schedule, keeps it only when it beats the champion on a held-out split, and writes the promotion to the shared volume from which every API replica reloads it (a timer every OSR_PLATFORM_SLM_RELOAD_S seconds in every process that routes, idle replicas included). Synthetic seed rows are added only while real rows are scarce (OSR_SLM_SYNTHETIC_PER_TEMPLATE rows per prompt template - 8 in the shipped stacks, about 500 rows and a 100-row holdout, so the gate can tell two architectures apart before real traffic arrives); GET /model reports prompts_remembered, decisions_remembered, outcomes and the cycle's sources (feedback, decisions, datasets, synthetic). The trainer builds every challenger with the query encoder OSR_SLM_ENCODER names (attention in the shipped stacks - a transformer block over the token sequence on top of the hashed features, about five milliseconds per prediction; hashed for the bag-of-features encoder alone): when it differs from the champion's the challenger is a fresh model with the configured architecture and the gate decides, so the served model changes encoder only where it wins the holdout; GET /model shows encoder. OSR_SLM_EMBEDDER adds a real embedding model to every challenger as frozen dense features (openai:<model>@<base_url>, any OpenAI-compatible /embeddings): the Azure stack points it at the embedding model the deployment already serves for embed-small (its model server's /v1, nothing leaves the environment; azd env set OSR_SLM_EMBEDDER none keeps the hashed and attention features only, any other value names an endpoint of your own), Compose takes it from the same variable (openai:qwen3-embedding:4b@http://ollama:11434/v1 with the llm profile). The SLM then learns its head on what a sentence transformer knows a prompt means, while the hashed features keep the surface cues; the promoted file names the embedder, so every API replica rebuilds the same client on reload (OSR_SLM_EMBEDDER_TIMEOUT_S 5 s in the API, 120 s for the trainer's batches) and a request the endpoint cannot embed in time is scored by the other strategies alone. Both start from the warm-start model the API image ships at /app/models/slm.json (OSR_PLATFORM_SLM) - the attention model bundled inside the SDK package, trained on the example catalogue, its labelled dataset and follow-up conversations and measured on a held-out seed - so a fresh deployment routes with a learned strategy from the first request and the first promotion has a champion to beat. Operators see the model in service, the evidence ingested, every cycle and a prompt probe on /platform/admin/slm (GET /api/v1/admin/slm, /slm/reports, POST /slm/cycle, /slm/predict).

The labelled floor. The holdout a promotion is judged on is self-agreement: the router's own decisions on the deployment's traffic. On a deployment whose traffic is one kind of job (production's was its own blog and SEO writers) the served model promoted itself twenty times on holdouts of 0.92-0.97 and routed 23 of the 70 hand-labelled reference rows acceptably - and, heard in the ensemble, pulled it from 66 to 39 of 70. The SDK package now ships that labelled set next to the bundled model, and every promotion is held to it (OSR_SLM_LABELLED_FLOOR, on): a challenger that routes fewer of the applicable labelled rows acceptably than the champion, by more than OSR_SLM_LABELLED_TOLERANCE (5 %) of them, is refused whatever its holdout says, and the report carries both scores under labelled. At start the service holds the champion it loads from the volume to the same floor against the packaged model and adopts the packaged model when the champion fell below it (the replaced model stays as slm.prev.json; GET /model -> floor says what happened). Rows count only when the deployment's catalogue holds their targets; with fewer than OSR_SLM_LABELLED_MIN_ROWS (20) such rows the floor is off, so a catalogue of your own is judged by its own evidence alone.

Rehearsal. A floor alone would leave the loop stuck: a challenger fitted on one workspace's traffic learns that traffic perfectly (31 holdout wins, 0 losses) and forgets the reference set (44 of 70), so the floor refuses it - every cycle, for ever, and nothing is ever promoted. The applicable labelled rows therefore also train every challenger, replicated until they are about OSR_SLM_LABELLED_REHEARSAL (0.25) of its training rows (0 = off); the holdout stays the deployment's own rows. The same challenger then keeps 69-70 of the 70 reference rows while it learns the traffic, and is promoted; the report's sources count the rows under labelled.

The same cycle trains the model's slot models: the outcomes credited to a plan's persona, skill and tool slots (role on the feedback topic - the platform's own assessment of every composed answer carries one per companion, POST /api/v1/feedback takes one) become one small encoder per role, saved in the same file and promoted with the primary, so the ensemble's SLM also votes on which persona or skill a request wants - a marketplace listing that gathered a record included. A role trains from OSR_SLM_SLOT_MIN_ROWS (20) rows on; each cycle report lists the roles under slots (rows, accuracy, skill, or why the role was skipped) and GET /api/v1/learning shows the served slots under slm.slots.

Workspaces do not all route alike, so the service also trains per-workspace models: once a workspace has OSR_PLATFORM_SLM_WORKSPACE_MIN_ROWS (100; 0 = off) rows of its own - outcomes joined to prompts through the training topic plus its routed prompts' decision rows, attributed by account_id - each shared cycle also trains a challenger for it, starting from the shared champion and seeing only that workspace's rows. It is promoted to autopilot/slm.<account_id>.json under the same gates (a ten-row holdout at least, a significant win over the model it replaces - the shared champion the first time), and the API replicas pick the file up with the shared one and serve it for that workspace's requests only; everyone else keeps the shared model. Each of these models is a multi-megabyte bundle every replica holds in memory and every cycle trains in series, so a deployment keeps at most OSR_PLATFORM_SLM_WORKSPACE_MAX_MODELS (20; 0 = no cap) of them - the workspaces with the most evidence. GET /api/v1/learning lists them under workspace_slms, the service's /model under workspaces, and POST /predict takes an account_id to rank with a workspace's model.

Services

The API is one image that can run as one process or as several: the documentation JSON (docs), the public model catalogue and rankings (rankings), the marketplace (marketplace), provider management (providers), signup and account links (onboarding), identity and workspaces (accounts: sign-in, sessions, organizations, keys, tenants, policy, governance, usage), Stripe billing (billing), the operator console API (admin), the MCP endpoint (mcp), the OpenAI-compatible proxy (openai), the routing core with its trace, event and learning reads (routing) and the routing SLM (slm) each have their own entry point and port. /api/v1/info, /api/v1/status and /api/v1/estimate always stay with the API. The API remains the single public origin and forwards a domain's paths to its service when OSR_PLATFORM_<NAME>_URL is set (streamed completions are relayed as they arrive; a service that stays silent longer than OSR_PLATFORM_PROXY_TIMEOUT_S, default 300 s - a cold self-hosted model loading its weights - answers 503); the answer carries X-OSR-Service naming the process that served it. The topology is an operator fact: the public GET /api/v1/info does not list it; operators see the map and live health per service on /platform/admin/services (GET /api/v1/admin/services). The Compose stack and the Azure template deploy every service as its own container next to the gateway.

Traces and events

The router records every request as a tree of spans (request, route, plan, execute) with events inside them (route.signals, route.policy, route.rank, route.fallback, guard.*, execute.step, learn.outcome, cache.hit, ...). The newest ones stay in an in-memory buffer (1000 by default, OSR_OBSERVABILITY_MEMORY_EVENTS) and every one of them is also written to the platform database by the platform's own telemetry store (OSR_PLATFORM_TELEMETRY_STORE, on by default) - off the request path, from a background writer, so tracing never slows routing. Events and traces therefore survive restarts, can be read by time range and are purged after OSR_PLATFORM_TELEMETRY_RETENTION_DAYS (14). They are readable per workspace:

POST /api/v1/route responses include trace_id and the X-OSR-Trace-Id header when the request was traced, and GET /api/v1/activity marks rows whose spans are kept (in memory or in the store) with traced: true and summarises the feedback reported for each request in outcome (POST /api/v1/feedback is stored per request, so the loop from decision to result is visible per row). A caller's W3C traceparent header is honoured: the request's spans join the caller's trace (X-OSR-Trace-Id returns that trace id), and a well-formed client X-Request-Id (8-64 characters of A-Za-z0-9._:-, not yet used by the workspace) becomes the decision's request_id - what /feedback and /trace/{request_id} take. The TypeScript SDK, the browser extension and the desktop app send both on every call by default (trace: false on the SDK client turns it off), so every request of every client is one traceable row; the extension's side panel and the desktop's answer line link answered by to the request's trace on the dashboard. Attributes hold ids, numbers and short labels; the request text is represented only by its digest and length. The dashboard shows the history at /platform/dashboard/events (window of 1 hour to 7 days, search by request or trace id), opens a trace from any activity row (with a How it was routed story: what the request looked like, who was allowed to answer, how the candidates ranked with their expected quality, price and latency, why the winner beat the runner-up strategy by strategy with each vote's rationale, the plan slots filled or skipped, the decision and the execution steps with tokens and cost) and adds a Trace tab to the playground. GET /api/v1/trace/{request_id} also returns usage: tokens (prompt / completion), the metered cost, the target's list price and what the same tokens would have cost on the savings baseline (the dearest model the workspace itself ran in the last 30 days, or the catalogue's dearest when it ran none - the same rule as GET /api/v1/savings; usage.baseline.source says which), so every request shows what it saved.

The deployment-wide figures the site shows (GET /api/v1/stats/public: requests, tokens, cost_usd, success_rate, top_targets and the savings block, plus internal for the deployment's own jobs) are read from a lifetime ledger (platform_counters, one row per target and scope) that every metered request adds to and nothing subtracts from: the retention sweep, a workspace's own retention_days and a deleted workspace remove request rows, never what has already been counted. since is the oldest counted request. The ledger seeds itself from the request rows a deployment still has the first time it is read, so an upgrade starts from today's totals rather than from zero. Only the windows (requests_24h, tokens_24h, the latency figures) and accounts_routing follow the rows that exist.

Public catalogue reads (/api/v1/info, /api/v1/models*, /api/v1/rankings, /api/v1/llms*, /api/v1/catalogue, /api/v1/stats/public) return an ETag and Cache-Control: public, max-age=60, stale-while-revalidate=60; send If-None-Match to get 304 when nothing changed. Operators tune this and the decision cache with:

Request-log export to your own storage

The usage rows and the kept payloads stay queryable through the dashboard and the API; a team that runs its own data lake or SIEM wants the same rows where its other logs are, continuously, without polling GET /api/v1/activity. A log export does that: POST /api/v1/workspace/log-export names a destination and the platform writes every new row there in batches of up to 2 000, on the destination's own interval_s (60 s and up), with a cursor so nothing is sent twice and nothing is skipped when a batch fails (the error is recorded, the cursor stays, the next pass retries). Destinations: azure_blob - a container URL (https://<account>.blob.core.windows.net/<container>) with a container SAS as credential (or the API's managed identity when it is empty), one JSONL blob per batch under <prefix>/YYYY/MM/DD/<time>-<first id>-<last id>.jsonl - and webhook - a POST of the JSONL batch (application/x-ndjson, Authorization: Bearer <credential>, X-OSR-Export-Batch) to an HTTPS URL: Splunk HEC behind a shim, Datadog Logs intake, a Kafka REST proxy, Logstash, a function. Every line is one request with its metering fields (account_id, key, member, tenant, project, endpoint, target, provider and upstream model, tokens, cost and charge, latency, status); include_payloads: true adds prompt and response from the kept payloads (redacted when the policy says so, payload_redacted marks it); start_from exports from a past timestamp (default: from now on). The credential is stored encrypted like a provider key and never returned (credential_set only). GET lists the destinations with cursor, batch and row counts and the last error; PATCH renames, re-points, rotates or clears the credential (clear_credential), changes the interval, pauses (enabled: false); POST .../{id}/run ships the next batch now and reports what went; DELETE removes it. Feature byok (Pro and up), up to five destinations; writes take the manage scope and are audited (log_export.created, log_export.deleted).

Traces for your observability tool. Two more kinds ship the same rows shaped as traces, with the outcomes reported so far and the answer judge's grade as scores: langfuse - url is the Langfuse host (https://cloud.langfuse.com, a regional or self-hosted one), credential is <public key>:<secret key>; one POST /api/public/ingestion per batch with a trace-create per request (input / output when payloads are included, user, tags, metadata), a generation-create (model, token counts, cost, latency, level) and a score-create per outcome - and otlp - url is an OTLP/HTTP traces endpoint (the standard v1/traces path: Arize Phoenix at app.phoenix.arize.com, Arize AX at otlp.arize.com, a collector), credential a bearer token or header=value;header=value pairs (Arize: space_id=...;api_key=...); spans carry OpenInference attributes (openinference.span.kind CHAIN for the routing decision and LLM for the generation, llm.model_name, llm.token_count.prompt|completion|total, input.value / output.value, session.id, user.id, osr.* fields and osr.score.<name>), which Arize Phoenix, Arize AX and TruLens read natively and every OpenTelemetry backend stores as attributes. Trace and span ids derive from the request id, so a client that forwards X-Request-Id into its own tracing can join the two. Agent engineering places this next to the other evaluation tools.

Monitoring: what is always on, and Application Insights on Azure

Every deployment monitors itself without another tool: the telemetry store above, GET /metrics (Prometheus text exposition: HTTP counters and latency per route, routing decisions, outcomes, cache), GET /readyz (database, catalogue, router, audit path, plus Redis / event bus / mail health), GET /api/v1/status, the deployment alerts and the operator console's Health page (GET /api/v1/admin/health). On Azure the platform additionally exports to Application Insights - the Azure template and the Marketplace offer create a workspace-based resource next to the Log Analytics workspace and hand every platform container its connection string:

What arrives, all under the request's operation id so the end-to-end view lines up: requests (every HTTP request a process answered - route template, status, latency, workspace id as the authenticated user, the caller's user agent and the page it came from as origin and path, never a query string; a refused request carries the reason the platform gave, the message of a 400 or the fields a 422 named), dependencies (every router span: route, plan, execute, provider calls, with duration and outcome - and every SQL statement the request ran, type SQL, named by verb and table, a failed one carrying the error's type and message), customEvents (the routing events with their attributes - ids, numbers and short labels, never prompt text), traces and exceptions. Each process reports as its container app (cloud_RoleName = osr-api, osr-routing, ...). What a replica does to check itself - the readiness probe routes a sentence through the engine on every check - is not exported. Delivery is batched off the request path with bounded retries; the console's Health page shows the export (monitoring: endpoint, sent, dropped, SQL statements tracked, last error) and nothing about it can fail a request. Nothing is exported from a host that is not Azure unless you say on.

The browser is in the same picture. The web app reports to the same resource (cloud_RoleName = osr-web / osr-platform): one pageViews row per navigation named by its route shape (ids and tokens reduced to *), the Core Web Vitals (web.LCP, web.INP, web.CLS, web.FCP, web.TTFB) as customMetrics, uncaught errors and rejections as exceptions - and a page that failed to render as one too, filed by the error boundary that caught it (boundary = root, marketing, docs, auth, dashboard, console or global) with the reference the person was shown on the error page (digest, the same one the server log carries, else the page's operation id), so "quote the reference to support" finds the exception - every API call as a dependencies row of type Ajax, a dead address as a page.not_found event with the path and the page that linked to it, and the moments of the user flow as customEvents - sign_up, login, begin_checkout, playground.answered, outcome.reported, evaluation.run, evaluation.replayed, the marked calls to action - with the workspace as the user and one session id per tab. Correlation is by request id: every call carries a browser-minted X-Request-Id and a W3C traceparent; the API answers under that id (or its routing decision's id, which the response header carries back), files its request under it with the browser's span as parent, and its router spans and SQL statements under the same operation - so the end-to-end view reads page view -> call -> request -> route / execute / SQL, and a slow page can be followed to the statement that made it slow. The web server is in it too: every call a server-rendered page makes to the API (cloud_RoleName the web app, client_Type Server, the replica as the instance) is a dependencies row of type Http under its own request id - memoised per path, so one row per refresh rather than per visitor - and a call that got no answer or a 5xx is also one line in the container log ([upstream] GET /api/v1/services -> TimeoutError ... request_id=..., at most one per path per half minute), so a public page that rendered without its data is visible from the web tier whether or not a browser was involved. Nothing a person typed travels: URLs are reduced to the allow-listed query keys, the only ids are request, session and workspace. The marketing pages' Google tag is separate (section 14, consent-gated); the Application Insights copy is the deployment's own operations telemetry and runs on the product surfaces too.

Configuration preflight

At start-up every process checks its own configuration for the mistakes an install makes before the first request