Skip to content
OpenSmartRoute
Documentation
Python SDK

API reference

Every public module, class and function with its signature, generated from the source.

Python SDK

Every module of the opensmartroute package and every public name it exports, generated from the source. The first line of each docstring is the summary; help() on a name gives the full contract. Narrative documentation: User guide (usage), SDK guide (stability, extension points), Architecture, Mathematics, Enterprise builder, Security model, Research.

Stability: names re-exported from opensmartroute (the top-level package) are frozen and follow SemVer. Sub-module names are public but may change in a minor release with a CHANGELOG entry.

Modules

opensmartroute

OpenSmartRoute — an open, intelligent route to the right decision, solution, or destination.

NameKindSummary
APIErrorre-export of opensmartroute.errors.APIErrorThe platform or osr serve answered a call with an error status; status, detail and the.
AsyncRouterre-export of opensmartroute.aio.AsyncRouterawait-able wrapper around a Router or EnterpriseRouter: route / learn / execute / run off the event loop.
AuthenticationErrorre-export of opensmartroute.errors.AuthenticationErrorThe CLI has no valid access token for the platform / server, or a sign-in was denied or timed out.
BanditStrategyre-export of opensmartroute.strategies.bandit.BanditStrategyThompson-sampling Beta bandit per (context, target); context = dominant domain (+ plan role).
Capabilitiesre-export of opensmartroute.core.types.CapabilitiesDeclarative description of what a target is good at.
CapabilityStrategyre-export of opensmartroute.strategies.capability.CapabilityStrategyDeclarative fit: domain / action overlap, complexity band, language, modality and quality prior.
Cascadere-export of opensmartroute.strategies.cascade.CascadeExecute ranked targets in planner order (cheapest / MDP / POMDP), stopping when the quality gate passes.
CascadePlannerre-export of opensmartroute.strategies.cascade.CascadePlannerFinite-horizon MDP over an ordered cascade with a stop action after each step.
ComponentRegistryre-export of opensmartroute.sdk.ComponentRegistryBlueprints for every routing component, with decorators that register into it.
ConfigurationErrorre-export of opensmartroute.errors.ConfigurationErrorInvalid catalogue, rules, SKILL.md, settings or a missing optional dependency.
ContextEngineerre-export of opensmartroute.knowledge.context.ContextEngineerBuilds the engineered context (module docstring). budget_tokens / max_facts / max_web_hits.
ContextMiddlewarere-export of opensmartroute.knowledge.context.ContextMiddlewareRun the :class:ContextEngineer in front of every routed request and teach the graph from the traffic.
DeferStrategyre-export of opensmartroute.strategies.defer.DeferStrategyLearning-to-defer: scores human targets by risk, PII, escalation intent, frustration and model uncertainty.
EffortStrategyre-export of opensmartroute.strategies.defer.EffortStrategyMatch a target's reasoning effort_level to signals.reasoning_need; penalise over- and under-thinking.
Eventre-export of opensmartroute.observability.EventOne captured span or event: flat, JSON-friendly, never carries request text.
EventSinkre-export of opensmartroute.observability.EventSinkReceiver port: override :meth:emit; live bridges may also implement :meth:span_start / :meth:span_end.
ExecutionErrorre-export of opensmartroute.errors.ExecutionErrorA target handler failed while executing a plan.
ExecutionResultre-export of opensmartroute.execution.ExecutionResultWhat happened when a decision was executed.
ExecutionStepre-export of opensmartroute.execution.ExecutionStepOne executed plan slot: role, target, latency and whether it succeeded. stage orders the steps.
FeedbackStorere-export of opensmartroute.feedback.FeedbackStoreAppend-only :class:Outcome log (in memory or JSONL file) with per-target statistics.
KnowledgeGraphre-export of opensmartroute.knowledge.graph.KnowledgeGraphFacts indexed three ways, upserted on their key, persisted as JSONL (see the module docstring).
KnowledgeStrategyre-export of opensmartroute.knowledge.context.KnowledgeStrategyScores candidates by what the graph knows worked on this kind of task: the performed_on facts of each.
LLMJudgeStrategyre-export of opensmartroute.strategies.llm_judge.LLMJudgeStrategyLLM-as-router with optional score calibration.
NoRouteErrorre-export of opensmartroute.errors.NoRouteErrorNo target satisfied the hard constraints.
Objectivere-export of opensmartroute.core.types.ObjectiveWhat the caller wants to optimise. Weights are relative.
OpenSmartRoutere-export of opensmartroute.client.OpenSmartRouteA client for the platform (https://opensmartroute.ai or your own deployment) and for osr serve.
OpenSmartRouteErrorre-export of opensmartroute.errors.OpenSmartRouteErrorBase class for all SDK errors.
Outcomere-export of opensmartroute.core.types.OutcomeFeedback about how a routed request actually went.
PlanSlotre-export of opensmartroute.core.types.PlanSlotOne filled slot of a multi-target plan (persona -> skill -> model).
Policyre-export of opensmartroute.policy.PolicyOrdered chain of :data:PolicyRule; returns the first rejection reason or None.
ProgressRouterre-export of opensmartroute.strategies.progress.ProgressRouterRoute each step of a task with trajectory context.
RankedTargetre-export of opensmartroute.core.types.RankedTargetA scored candidate: utility, ensemble quality estimate and the per-strategy breakdown.
RateLimitErrorre-export of opensmartroute.errors.RateLimitError429: the rate limit, the monthly quota or a budget is exhausted; retry_after says when to try again.
RemoteDecisionre-export of opensmartroute.client.RemoteDecisionWhat route() returns: the decision as the server sent it (raw) with the fields people read first.
RequestConstraintsre-export of opensmartroute.core.types.RequestConstraintsHard constraints on the request (never traded off).
RouteDecisionre-export of opensmartroute.core.types.RouteDecisionThe answer to route(): chosen target, confidence, alternatives, optional plan, trace and propensities.
RoutePlanre-export of opensmartroute.core.types.RoutePlanA composed route (MasRouter-style): several targets working together, executed in stages.
RouteRequestre-export of opensmartroute.core.types.RouteRequestThe customer need.
RouteTargetre-export of opensmartroute.core.types.RouteTargetA routable destination: an LLM, agent, skill, persona, tool, workflow or human.
RouteTracere-export of opensmartroute.core.types.RouteTraceEverything needed to explain a decision.
Routerre-export of opensmartroute.router.RouterSignals -> policy -> strategies -> ensemble -> decision (+ optional plan).
RouterSLMre-export of opensmartroute.learning.slm.RouterSLMSmall routing model: dual encoder + target catalogue snapshot + calibration, in one JSON file.
Rulere-export of opensmartroute.strategies.rules.RuleIf all when conditions match, boost prefer targets and penalise avoid.
RulesStrategyre-export of opensmartroute.strategies.rules.RulesStrategyApplies declarative :class:Rule preferences (prefer / avoid / pin) when their when conditions match.
SLMStrategyre-export of opensmartroute.learning.slm.SLMStrategyEnsemble member backed by a :class:RouterSLM: scores are its probabilities, it keeps learning online.
SecurityErrorre-export of opensmartroute.errors.SecurityErrorRequest rejected by an input guard (prompt injection, oversize, etc.).
SelfImproverre-export of opensmartroute.learning.self_improve.SelfImproverClosed loop that keeps a :class:RouterSLM current with the model market and its own traffic.
Settingsre-export of opensmartroute.settings.SettingsAll tunables, grouped by consumer. Immutable; derive variants with :meth:replace.
Signalsre-export of opensmartroute.core.types.SignalsCheap deterministic features extracted from a request.
SimilarityStrategyre-export of opensmartroute.strategies.similarity.SimilarityStrategyEmbed the request and each target's examples / description; score by best and top-k mean similarity.
Spanre-export of opensmartroute.observability.SpanAn open unit of work; a context manager that records duration, status and nested events.
StateStoreErrorre-export of opensmartroute.errors.StateStoreErrorA learner-state store failed to load, save or migrate.
Strategyre-export of opensmartroute.strategies.base.StrategyScores each candidate target in [0, 1] and explains why.
StrategyScorere-export of opensmartroute.core.types.StrategyScoreOne strategy's opinion about one target.
TargetConstraintsre-export of opensmartroute.core.types.TargetConstraintsWhere / for whom a target may be used. Checked by the policy layer.
TargetKindre-export of opensmartroute.core.types.TargetKindKinds of things a request can be routed to.
TargetRegistryre-export of opensmartroute.core.registry.TargetRegistryIn-memory catalogue of :class:RouteTarget by id: add / upsert / remove, filtered listing, (de)serialisation.
TargetUnavailableErrorre-export of opensmartroute.errors.TargetUnavailableErrorA remote target or catalogue source could not be reached.
TaskTableStrategyre-export of opensmartroute.strategies.task_table.TaskTableStrategyStatic task_type -> target -> quality table with family and prior fallbacks; learns from outcomes.
Tracerre-export of opensmartroute.observability.TracerOpens spans, records events and fans them out to sinks; sample_rate < 1 traces a share of requests.
ValidationErrorre-export of opensmartroute.errors.ValidationErrorA request, outcome or target failed validation.
__version__constant
agentre-export of opensmartroute.sdk.agent@agent(id, ...): shorthand for an agent target.
aroutere-export of opensmartroute.sdk.arouteawait aroute(text): the async form of route().
arunre-export of opensmartroute.sdk.arunawait arun(text): the async form of run(); coroutine handlers are awaited.
componentsre-export of opensmartroute.sdk.components: The process-wide registry that the top-level decorators (opensmartroute.strategy ...) bind to.
configurere-export of opensmartroute.settings.configureInstall process-wide settings. configure() with no arguments re-reads the environment;.
configure_tracingre-export of opensmartroute.observability.configure_tracingInstall sinks on the process-wide tracer. With no sinks, use the ones named by settings.
feedbackre-export of opensmartroute.sdk.feedbackfeedback(decision, success=, quality=): learn from a decision in one call.
get_settingsre-export of opensmartroute.settings.get_settingsThe process-wide :class:Settings (environment overlay applied once, lazily).
get_tracerre-export of opensmartroute.observability.get_tracerThe process-wide tracer (disabled until :func:configure_tracing adds sinks).
guardre-export of opensmartroute.sdk.guard@guard: fn(request) -> reason | RouteRequest | None runs before routing.
handlerre-export of opensmartroute.sdk.handler@handler(*target_ids): bind code to targets declared in a catalogue.
humanre-export of opensmartroute.sdk.human@human(id, ...): shorthand for a human target.
llmre-export of opensmartroute.sdk.llm@llm(id, ...): shorthand for an llm target.
load_rulesre-export of opensmartroute.config.load_rulesLoad a rules file (top-level list or rules: key) into a :class:RulesStrategy.
load_targetsre-export of opensmartroute.config.load_targetsLoad a catalogue file (top-level list or targets: key) into a :class:TargetRegistry.
middlewarere-export of opensmartroute.sdk.middleware@middleware: register a Middleware class or fn(request, next_route).
on_decisionre-export of opensmartroute.sdk.on_decision@on_decision: fn(request, decision) after every route.
on_errorre-export of opensmartroute.sdk.on_error@on_error: fn(request, error) when a route fails.
on_outcomere-export of opensmartroute.sdk.on_outcome@on_outcome: fn(outcome) for every outcome learned.
personare-export of opensmartroute.sdk.persona@persona(id, ...): shorthand for a slot-only persona target.
policy_rulere-export of opensmartroute.sdk.policy_rule@policy_rule: register fn(target, request, signals) -> reason | None.
routere-export of opensmartroute.sdk.routeroute(text): one request through the process-wide registry's router.
rulere-export of opensmartroute.sdk.rulerule(name, prefer=, avoid=, pin=, **when) or @rule(...) over a predicate.
runre-export of opensmartroute.sdk.runrun(text): route + execute + learn through the process-wide registry.
signalre-export of opensmartroute.sdk.signal@signal: register a SignalExtractor class or fn(request, signals) -> mapping | value.
skillre-export of opensmartroute.sdk.skill@skill(id, ...): shorthand for a skill target.
strategyre-export of opensmartroute.sdk.strategy@strategy(weight=, name=): register a Strategy class or fn(request, signals, candidates).
targetre-export of opensmartroute.sdk.target@target(id, kind, ...): the decorated callable becomes a RouteTarget handler.
telemetryre-export of opensmartroute.sdk.telemetry@telemetry: register a Telemetry sink class or factory.
toolre-export of opensmartroute.sdk.tool@tool(id, ...): shorthand for a tool target.

opensmartroute.__main__

python -m opensmartroute ... is the osr command (IDE MCP configs, containers without the script on PATH).

No public names.

opensmartroute.adapters

Adapters connect OpenSmartRoute to real providers and infrastructure.

NameKindSummary
AgentHarnessre-export of opensmartroute.adapters.harness.AgentHarnessStructural interface of an agent runtime: run(task, context=, history=) -> HarnessResult.
AnthropicClientre-export of opensmartroute.adapters.anthropic_client.AnthropicClientThe Messages API behind the :class:OpenAICompatClient surface: chat / stream_chat take.
CallableHarnessre-export of opensmartroute.adapters.harness.CallableHarnessWrap an in-process agent: fn(task, context, history) -> str | dict | HarnessResult.
ChatResultre-export of opensmartroute.adapters.openai_compat.ChatResultResult of a chat completion: text, model, token counts, latency, tool calls, finish reason and raw response.
HTTPHarnessre-export of opensmartroute.adapters.harness.HTTPHarnessPOST the task as JSON to an agent endpoint.
HarnessResultre-export of opensmartroute.adapters.harness.HarnessResultWhat an agent harness returns: text, success, token / cost / latency usage and optional self-graded quality.
HttpMCPClientre-export of opensmartroute.adapters.mcp.HttpMCPClientStdlib MCP client for the Streamable HTTP transport (POST <url> per JSON-RPC message).
InMemoryQueuere-export of opensmartroute.adapters.handlers.InMemoryQueueReference :class:Queue: FIFO in memory, resolves into :class:~opensmartroute.Outcome.
Listingre-export of opensmartroute.adapters.marketplace.ListingA marketplace search hit (what GET /api/v1/registry returns per item).
MarketplaceClientre-export of opensmartroute.adapters.marketplace.MarketplaceClientSearch a marketplace and load its listings as routing targets.
MarketplaceSyncre-export of opensmartroute.adapters.marketplace.MarketplaceSyncKeep a registry in step with the marketplace on a schedule (the osr serve side of the daily refresh).
MockClientre-export of opensmartroute.adapters.mock_client.MockClientDeterministic in-process provider; see the module docstring for what it answers.
ModelCardre-export of opensmartroute.adapters.catalogue.ModelCardWhat the catalogue knows about one model: identity, price, limits, evidence of quality, provenance.
ModelCataloguere-export of opensmartroute.adapters.catalogue.ModelCatalogueMerged, persisted model cards from every source; the SLM's view of the target universe.
OpenAICompatClientre-export of opensmartroute.adapters.openai_compat.OpenAICompatClientStdlib-only client for the OpenAI chat / embeddings API (OpenAI, Azure, vLLM, Ollama, LiteLLM) with retries.
OpenTelemetrySinkre-export of opensmartroute.adapters.optional.OpenTelemetrySinkLive bridge from the tracer to OpenTelemetry: every OpenSmartRoute span becomes an OTel span.
OpenTelemetryTelemetryre-export of opensmartroute.adapters.optional.OpenTelemetryTelemetryEmits one span per decision and counters/histograms via the OTel API.
PendingResultre-export of opensmartroute.adapters.handlers.PendingResultImmediate answer of a queued target: the request was accepted and will be answered later.
Queuere-export of opensmartroute.adapters.handlers.QueueStructural interface of an asynchronous queue target (ticketing, human tier, workflow run).
QueuedItemre-export of opensmartroute.adapters.handlers.QueuedItemOne request waiting in (or resolved from) a queue.
SearchHitre-export of opensmartroute.adapters.websearch.SearchHitOne search result: where it came from, what it says, and when it was seen.
SemanticRouterImportre-export of opensmartroute.adapters.semantic_router.SemanticRouterImportResult of importing a vLLM semantic-router config: registry, rules, default model, categories, warnings.
ServerCardre-export of opensmartroute.adapters.mcp_servers.ServerCardDescription of an MCP server (tools, tags, auth, latency, cost, region, data boundary) for recommendation.
ServerRecommendationre-export of opensmartroute.adapters.mcp_servers.ServerRecommendationA ranked server from recommend_servers with its fused score, rationale and matched tool names.
StdioMCPClientre-export of opensmartroute.adapters.mcp.StdioMCPClientTiny JSON-RPC-over-stdio MCP client (newline-delimited). Thread-safe, blocking.
SubprocessHarnessre-export of opensmartroute.adapters.harness.SubprocessHarnessRun a CLI agent: task on stdin, answer on stdout, exit code 0 = success.
SyncReportre-export of opensmartroute.adapters.marketplace.SyncReportWhat :meth:MarketplaceClient.sync did to a registry.
WebKnowledgere-export of opensmartroute.adapters.websearch.WebKnowledgeFan a query out to search providers, de-duplicate by URL and cache the hits as JSON.
a2a_handlerre-export of opensmartroute.adapters.a2a.a2a_handlerReturn a handler that sends the request text to an A2A agent and returns its text.
agent_from_cardre-export of opensmartroute.adapters.a2a.agent_from_cardBuild an agent RouteTarget from an A2A agent card (name, skills, tags, input modes).
attach_chat_handlersre-export of opensmartroute.adapters.openai_compat.attach_chat_handlersGive every LLM target a :func:chat_handler on client; returns the ids that got one.
brave_searchre-export of opensmartroute.adapters.websearch.brave_searchBrave Search API web results; the key is read from BRAVE_API_KEY (or BRAVE_API_KEY_FILE).
card_to_targetre-export of opensmartroute.adapters.catalogue.card_to_targetA :class:RouteTarget for a model card; risky third-party descriptions are replaced by the name.
chat_handlerre-export of opensmartroute.adapters.openai_compat.chat_handlerAdapter for RouteTarget.handler: turns a RouteRequest into a chat call.
connect_mcpre-export of opensmartroute.adapters.mcp.connect_mcpSpawn a stdio MCP server, list its tools (and optionally resources and prompts) and return.
connect_mcp_httpre-export of opensmartroute.adapters.mcp.connect_mcp_httpConnect to a Streamable-HTTP MCP server, import its tools (and optionally resources and prompts) as.
duckduckgo_searchre-export of opensmartroute.adapters.websearch.duckduckgo_searchDuckDuckGo instant-answer API (abstract + related topics). Keyless; shallow but good for definitions.
embed_handlerre-export of opensmartroute.adapters.openai_compat.embed_handlerAdapter for an embeddings-capable RouteTarget.handler: embeds context["embeddings_input"] (or the text).
embedderre-export of opensmartroute.adapters.openai_compat.embedderAdapter for SimilarityStrategy(embedder=...).
enrich_descriptionre-export of opensmartroute.adapters.mcp.enrich_descriptionReturn (routing_description, examples, capabilities) for a tool.
fetch_agent_cardre-export of opensmartroute.adapters.a2a.fetch_agent_cardDownload an agent card over HTTPS (plain HTTP is refused); raises TargetUnavailableError on failure.
fetch_bytesre-export of opensmartroute.adapters.websearch.fetch_bytesGET url over https with a timeout and a body cap; transport errors become TargetUnavailableError.
fetch_feedre-export of opensmartroute.adapters.websearch.fetch_feedFetch an RSS / Atom feed over https and return its articles (see :func:parse_feed).
fetch_huggingface_modelsre-export of opensmartroute.adapters.catalogue.fetch_huggingface_modelsModel cards from the Hugging Face Hub search (downloads, likes, tags, model-index benchmarks).
fetch_jsonre-export of opensmartroute.adapters.websearch.fetch_jsonGET a JSON document (see :func:fetch_bytes); malformed bodies raise TargetUnavailableError.
fetch_leaderboard_qualityre-export of opensmartroute.adapters.catalogue.fetch_leaderboard_quality{hub model id: {benchmark: accuracy}} from the Open LLM Leaderboard table (official rows, unflagged).
fetch_openrouter_modelsre-export of opensmartroute.adapters.catalogue.fetch_openrouter_modelsModel cards from OpenRouter's public listing (prices per token, context, modalities, tool support).
fetch_page_textre-export of opensmartroute.adapters.websearch.fetch_page_textFetch a page and return {url, title, text, risk}; risk is the injection/gadget risk of the text.
harness_handlerre-export of opensmartroute.adapters.harness.harness_handlerAdapter for RouteTarget.handler: a RouteRequest becomes a harness task.
html_to_textre-export of opensmartroute.adapters.websearch.html_to_text(title, text) of an HTML document with scripts/styles removed and whitespace collapsed.
http_handlerre-export of opensmartroute.adapters.handlers.http_handlerRouteTarget.handler that POSTs the request to url and returns a :class:HarnessResult.
huggingface_searchre-export of opensmartroute.adapters.websearch.huggingface_searchSearch the Hugging Face Hub (what = models or datasets), ranked by downloads. No key needed.
judge_fnre-export of opensmartroute.adapters.openai_compat.judge_fnAdapter for LLMJudgeStrategy(llm=...): prompt in, completion text out.
langgraph_conditionre-export of opensmartroute.adapters.frameworks.langgraph_conditionEdge selector for add_conditional_edges: routes on target id (by='target') or kind.
langgraph_nodere-export of opensmartroute.adapters.frameworks.langgraph_nodeReturn a LangGraph-compatible node state -> dict (partial state update).
load_marketplace_targetsre-export of opensmartroute.adapters.marketplace.load_marketplace_targetsShortcut: MarketplaceClient(base_url, api_key=api_key).targets(q, kind=kind, tag=tag, limit=limit).
load_personasre-export of opensmartroute.adapters.personas.load_personasLoad personas from a directory of markdown files or a JSON/JSONL/CSV catalogue.
load_semantic_router_configre-export of opensmartroute.adapters.semantic_router.load_semantic_router_configConvert a vLLM semantic-router model_config / categories document into targets and rules.
load_skillre-export of opensmartroute.adapters.skills.load_skillLoad one skill directory (must contain SKILL.md).
load_skillsre-export of opensmartroute.adapters.skills.load_skillsLoad every */SKILL.md under root (one level deep, sorted by name).
maf_router_executorre-export of opensmartroute.adapters.frameworks.maf_router_executorAgent Framework style: executor(message, ctx) -> target id | response; handoffs maps.
manifest_from_targetsre-export of opensmartroute.adapters.mcp.manifest_from_targetsReverse: dump MCP-shaped tool dicts (for signing / publishing a catalogue).
mcp_server_handlerre-export of opensmartroute.adapters.mcp.mcp_server_handlerRouteTarget.handler for a target that stands for a whole remote MCP server (an mcp endpoint in an.
mcp_tool_handlerre-export of opensmartroute.adapters.handlers.mcp_tool_handlerRouteTarget.handler that invokes one MCP tool via call(name, arguments).
model_keyre-export of opensmartroute.adapters.catalogue.model_keyVendor-agnostic key for matching model names across sources: lowercase, no vendor prefix, no punctuation.
openai_tool_handlerre-export of opensmartroute.adapters.frameworks.openai_tool_handlerCallable behind openai_tool_spec: (text, objective?, kinds?) -> RouteDecision.to_dict().
openai_tool_specre-export of opensmartroute.adapters.frameworks.openai_tool_specOpenAI function-calling tool definition that lets a model ask the router for a target.
parse_feedre-export of opensmartroute.adapters.websearch.parse_feedArticles of an RSS 2.0 / RDF / Atom document as kind="article" hits, newest first.
persona_from_markdownre-export of opensmartroute.adapters.personas.persona_from_markdownParse a *.agent.md / *.chatmode.md / front-matter markdown file into a persona target.
persona_targetre-export of opensmartroute.adapters.personas.persona_targetBuild a persona RouteTarget whose instructions is the system prompt (non-primary by default).
personas_from_recordsre-export of opensmartroute.adapters.personas.personas_from_recordsPersona targets from JSON / CSV-style records (name + prompt/system keys); others skipped.
prompts_from_mcpre-export of opensmartroute.adapters.mcp.prompts_from_mcpConvert prompts/list output into instructions-only skill targets (primary=False).
quality_from_benchmarksre-export of opensmartroute.adapters.catalogue.quality_from_benchmarksMean of normalised benchmark scores (percentages are divided by 100); None when there are none.
quality_from_popularityre-export of opensmartroute.adapters.catalogue.quality_from_popularityWeak prior in [0.35, 0.75] from log-scaled downloads and likes (popularity is not quality; it is a hint).
queue_handlerre-export of opensmartroute.adapters.handlers.queue_handlerRouteTarget.handler that enqueues the request and returns a :class:PendingResult.
recommend_serversre-export of opensmartroute.adapters.mcp_servers.recommend_serversRank MCP servers for a task; constraint violations are excluded, not down-weighted.
resources_from_mcpre-export of opensmartroute.adapters.mcp.resources_from_mcpConvert resources/list output into read-only tool targets whose handler reads the resource.
route_and_executere-export of opensmartroute.adapters.frameworks.route_and_executeRoute text and optionally execute the plan; returns (decision, ExecutionResult | None).
searxng_searchre-export of opensmartroute.adapters.websearch.searxng_searchA self-hosted SearXNG instance (SEARXNG_URL or base_url; its JSON API must be enabled): real web.
sentence_transformers_embedderre-export of opensmartroute.adapters.optional.sentence_transformers_embedderSemantic embedder for SimilarityStrategy. Requires opensmartroute[embeddings].
sign_manifestre-export of opensmartroute.adapters.mcp.sign_manifestWrap a tool list in a signed manifest. algorithm = hmac-sha256 (key = shared.
skill_from_markdownre-export of opensmartroute.adapters.skills.skill_from_markdownParse one SKILL.md (Agent-Skills front matter + body) into a skill RouteTarget; strict validation.
skills_from_cardre-export of opensmartroute.adapters.a2a.skills_from_cardOne non-primary skill target per skill declared on an A2A agent card (<agent>/<skill>).
tools_from_manifestre-export of opensmartroute.adapters.mcp.tools_from_manifestImport tools from a (signed) manifest; kw goes to :func:tools_from_mcp.
tools_from_mcpre-export of opensmartroute.adapters.mcp.tools_from_mcpConvert tools/list output into targets. call(name, arguments) becomes the handler.
verify_manifestre-export of opensmartroute.adapters.mcp.verify_manifestConstant-time verification. key = shared secret (HMAC) or 32-byte Ed25519 public key.
wikipedia_searchre-export of opensmartroute.adapters.websearch.wikipedia_searchWikipedia article search (MediaWiki API, no key): title, first snippet and the article URL. Encyclopaedic,.

opensmartroute.adapters.a2a

Import A2A (Agent-to-Agent protocol) Agent Cards as TargetKind.AGENT targets.

NameKindSummary
TERMINAL_TASK_STATESconstant: A2A task states after which no more work happens (tasks/get polling stops).
a2a_handlerfunction (url: str, token: str | None=None, timeout_s: float=60.0, method: str='message/send', poll_interval_s: float=1.0)Return a handler that sends the request text to an A2A agent and returns its text.
agent_from_cardfunction (card: dict[str, Any], *, call: Callable[[str, RouteRequest], Any] | None=None, cost_per_1k_tokens: float=0.0, latency_ms: float=2000.0, quality_prior: float=0.65, languages: list[str] | None=None)Build an agent RouteTarget from an A2A agent card (name, skills, tags, input modes).
fetch_agent_cardfunction (base_url: str, timeout_s: float=10.0, path: str='/.well-known/agent.json')Download an agent card over HTTPS (plain HTTP is refused); raises TargetUnavailableError on failure.
skills_from_cardfunction (card: dict[str, Any], agent_id: str | None=None)One non-primary skill target per skill declared on an A2A agent card (<agent>/<skill>).

opensmartroute.adapters.anthropic_client

Native Anthropic Messages API client - route TO Claude, not only accept Claude-dialect requests.

NameKindSummary
ANTHROPIC_VERSIONconstant: The API version header every request carries (the newest stable revision the translation targets).
CACHE_MAX_BREAKPOINTSconstant: Breakpoints the Messages API accepts per request.
CACHE_MIN_CHARSconstant: cache breakpoint is worth placing on it; tools and the conversation prefix are marked whenever they exist.
DEFAULT_MAX_TOKENSconstant: The Messages API requires max_tokens; used when neither the caller nor the proxy set one.
THINKING_BUDGETSconstant: Extended-thinking budgets per reasoning_effort value.
AnthropicClientclassThe Messages API behind the :class:OpenAICompatClient surface: chat / stream_chat take.
cache_breakpointsfunction (body: dict[str, Any])Place Anthropic prompt-cache breakpoints on a Messages body the caller sent without any: the last tool.
to_messages_requestfunction (messages: list[dict[str, Any]], model: str, *, temperature: float=0.0, max_tokens: int | None=None, **kw: Any)A chat-completions call as a Messages API body (see the module docstring for the mapping).

opensmartroute.adapters.catalogue

Live model catalogue: collect model cards (price, context, modalities, benchmarks) from public sources.

NameKindSummary
LEADERBOARD_DATASETconstantOpen LLM Leaderboard v2 results table on the Hub.
OPENROUTER_MODELSconstantpublic model + pricing listing (no key).
ModelCardclassWhat the catalogue knows about one model: identity, price, limits, evidence of quality, provenance.
ModelCatalogueclassMerged, persisted model cards from every source; the SLM's view of the target universe.
card_to_targetfunction (card: ModelCard, *, default_latency_ms: float=1500.0, settings: Settings | None=None)A :class:RouteTarget for a model card; risky third-party descriptions are replaced by the name.
fetch_huggingface_modelsfunction (query: str='', *, limit: int=50, pipeline: str='text-generation', timeout_s: float | None=None, settings: Settings | None=None)Model cards from the Hugging Face Hub search (downloads, likes, tags, model-index benchmarks).
fetch_leaderboard_qualityfunction (*, limit: int=5000, dataset: str=LEADERBOARD_DATASET, timeout_s: float | None=None, settings: Settings | None=None){hub model id: {benchmark: accuracy}} from the Open LLM Leaderboard table (official rows, unflagged).
fetch_openrouter_modelsfunction (*, url: str=OPENROUTER_MODELS, timeout_s: float | None=None, settings: Settings | None=None)Model cards from OpenRouter's public listing (prices per token, context, modalities, tool support).
model_keyfunction (model_id: str)Vendor-agnostic key for matching model names across sources: lowercase, no vendor prefix, no punctuation.
quality_from_benchmarksfunction (benchmarks: dict[str, float])Mean of normalised benchmark scores (percentages are divided by 100); None when there are none.
quality_from_popularityfunction (downloads: int, likes: int)Weak prior in [0.35, 0.75] from log-scaled downloads and likes (popularity is not quality; it is a hint).

opensmartroute.adapters.frameworks

Drop-in nodes for agent frameworks.

NameKindSummary
langgraph_conditionfunction (by: str='target', default: str='__end__')Edge selector for add_conditional_edges: routes on target id (by='target') or kind.
langgraph_nodefunction (router: Router, *, execute: bool=False, plan: bool=False, context_key: str='route_context', on_no_route: str='__no_route__')Return a LangGraph-compatible node state -> dict (partial state update).
last_user_textfunction (state: dict[str, Any])(last user utterance, prior history as role/content dicts).
maf_router_executorfunction (router: Router, *, execute: bool=False)Agent Framework style: executor(message, ctx) -> target id | response; handoffs maps.
openai_tool_handlerfunction (router: Router)Callable behind openai_tool_spec: (text, objective?, kinds?) -> RouteDecision.to_dict().
openai_tool_specfunction (name: str='route_request')OpenAI function-calling tool definition that lets a model ask the router for a target.
route_and_executefunction (router: Router, text: str, *, history: list[dict[str, str]] | None=None, context: dict[str, Any] | None=None, objective: Objective | None=None, execute: bool=False, plan: bool=False, learn: bool=True)Route text and optionally execute the plan; returns (decision, ExecutionResult | None).

opensmartroute.adapters.handlers

Executors for non-LLM targets: HTTP endpoints, MCP tools and asynchronous queues.

NameKindSummary
InMemoryQueueclassReference :class:Queue: FIFO in memory, resolves into :class:~opensmartroute.Outcome.
PendingResultclassImmediate answer of a queued target: the request was accepted and will be answered later.
QueueclassStructural interface of an asynchronous queue target (ticketing, human tier, workflow run).
QueuedItemclassOne request waiting in (or resolved from) a queue.
http_handlerfunction (url: str, *, api_key: str | None=None, api_key_env: str | None=None, timeout_s: float=300.0, headers: Mapping[str, str] | None=None, payload: Callable[[str, Mapping[str, Any], list[dict[str, str]]], dict[str, Any]] | None=None, body: Mapping[str, Any] | None=None)RouteTarget.handler that POSTs the request to url and returns a :class:HarnessResult.
mcp_tool_handlerfunction (call: Callable[[str, dict[str, Any]], Any], tool: str, input_schema: dict[str, Any] | None=None, arguments: Callable[[RouteRequest], dict[str, Any]] | None=None)RouteTarget.handler that invokes one MCP tool via call(name, arguments).
queue_handlerfunction (queue: Queue, target_id: str | None=None)RouteTarget.handler that enqueues the request and returns a :class:PendingResult.

opensmartroute.adapters.harness

Agent-harness adapters: route to a runtime, not just a model.

NameKindSummary
AgentHarnessclassStructural interface of an agent runtime: run(task, context=, history=) -> HarnessResult.
CallableHarnessclassWrap an in-process agent: fn(task, context, history) -> str | dict | HarnessResult.
HTTPHarnessclassPOST the task as JSON to an agent endpoint.
HarnessResultclassWhat an agent harness returns: text, success, token / cost / latency usage and optional self-graded quality.
SubprocessHarnessclassRun a CLI agent: task on stdin, answer on stdout, exit code 0 = success.
coerce_resultfunction (out: Any)Normalise str | dict | HarnessResult | object-with-text into a HarnessResult.
harness_handlerfunction (harness: AgentHarness)Adapter for RouteTarget.handler: a RouteRequest becomes a harness task.

opensmartroute.adapters.marketplace

Pull skills, agents, tools and templates from an OpenSmartRoute marketplace into a router.

NameKindSummary
DEFAULT_MARKETPLACEconstantthe hosted marketplace API.
LISTING_KINDSconstant: Listing kinds the marketplace catalogue knows (search filter values).
ROUTABLE_KINDSconstant: Listing kinds whose manifest is an Open Capability Manifest the router can load as a target.
ListingclassA marketplace search hit (what GET /api/v1/registry returns per item).
MarketplaceClientclassSearch a marketplace and load its listings as routing targets.
MarketplaceSyncclassKeep a registry in step with the marketplace on a schedule (the osr serve side of the daily refresh).
SyncReportclassWhat :meth:MarketplaceClient.sync did to a registry.
load_marketplace_targetsfunction (base_url: str=DEFAULT_MARKETPLACE, q: str='', *, kind: str | None=None, tag: str | None=None, limit: int=24, api_key: str | None=None)Shortcut: MarketplaceClient(base_url, api_key=api_key).targets(q, kind=kind, tag=tag, limit=limit).

opensmartroute.adapters.mcp

Import MCP (Model Context Protocol) tools as TargetKind.TOOL targets.

NameKindSummary
HttpMCPClientclassStdlib MCP client for the Streamable HTTP transport (POST <url> per JSON-RPC message).
MCPToolclassA tool as listed by an MCP server: name, description, JSON input schema and annotations.
StdioMCPClientclassTiny JSON-RPC-over-stdio MCP client (newline-delimited). Thread-safe, blocking.
canonical_jsonfunction (obj: Any)Deterministic JSON encoding (sorted keys, no whitespace) used for manifest digests and signatures.
connect_mcpfunction (command: list[str], server: str='', *, resources: bool=False, prompts: bool=False, **kw: Any)Spawn a stdio MCP server, list its tools (and optionally resources and prompts) and return.
connect_mcp_httpfunction (url: str, api_key: str | None=None, server: str='', *, resources: bool=False, prompts: bool=False, timeout_s: float=60.0, **kw: Any)Connect to a Streamable-HTTP MCP server, import its tools (and optionally resources and prompts) as.
enrich_descriptionfunction (name: str, description: str, input_schema: dict[str, Any] | None, server: str='', enricher: Callable[[str], list[str]] | None=None)Return (routing_description, examples, capabilities) for a tool.
manifest_digestfunction (tools: list[dict[str, Any]])SHA-256 hex digest of a tool list in canonical JSON.
manifest_from_targetsfunction (targets: Iterable[RouteTarget])Reverse: dump MCP-shaped tool dicts (for signing / publishing a catalogue).
prompts_from_mcpfunction (payload: Any, server: str='', get: Callable[[str, dict[str, Any]], Any] | None=None, *, quality_prior: float=0.6)Convert prompts/list output into instructions-only skill targets (primary=False).
resources_from_mcpfunction (payload: Any, server: str='', read: Callable[[str], Any] | None=None, *, latency_ms: float=200.0, quality_prior: float=0.6)Convert resources/list output into read-only tool targets whose handler reads the resource.
sign_manifestfunction (tools: list[dict[str, Any]], key: bytes, *, server: str='', algorithm: str='hmac-sha256', key_id: str='')Wrap a tool list in a signed manifest. algorithm = hmac-sha256 (key = shared.
tools_from_manifestfunction (manifest: dict[str, Any], key: bytes | None, *, require_signature: bool=True, max_age_s: float | None=None, **kw: Any)Import tools from a (signed) manifest; kw goes to :func:tools_from_mcp.
tools_from_mcpfunction (payload: Any, server: str='', call: Callable[[str, dict[str, Any]], Any] | None=None, enricher: Callable[[str], list[str]] | None=None, cost_per_call_usd: float=0.0, latency_ms: float=300.0, quality_prior: float=0.6, max_description_risk: float=0.6, guard: InputGuard | None=None)Convert tools/list output into targets. call(name, arguments) becomes the handler.
verify_manifestfunction (manifest: dict[str, Any], key: bytes, max_age_s: float | None=None)Constant-time verification. key = shared secret (HMAC) or 32-byte Ed25519 public key.

opensmartroute.adapters.mcp_servers

MCP server recommendation (MCP-Zero 2506.01056; ToolRet 2603.06467).

NameKindSummary
ServerCardclassDescription of an MCP server (tools, tags, auth, latency, cost, region, data boundary) for recommendation.
ServerRecommendationclassA ranked server from recommend_servers with its fused score, rationale and matched tool names.
recommend_serversfunction (request: RouteRequest | str, servers: Iterable[ServerCard | dict[str, Any]], k: int=3, constraints: RequestConstraints | None=None, allowed_auth: Iterable[str] | None=None, min_score: float=0.0)Rank MCP servers for a task; constraint violations are excluded, not down-weighted.

opensmartroute.adapters.mock_client

A provider that answers deterministically without a network: the mock provider kind.

NameKindSummary
MOCK_SCHEMEconstanta base URL that selects this client.
MockClientclassDeterministic in-process provider; see the module docstring for what it answers.
is_mockfunction (kind: str | None, base_url: str | None)True when a provider spec selects the mock client (kind: mock or a mock:// base URL).

opensmartroute.adapters.openai_compat

OpenAI-compatible HTTP client (stdlib only).

NameKindSummary
PASSTHROUGH_FIELDSconstantChat-completion fields the proxy passes through to the upstream untouched (request.context["openai"]).
UPSTREAM_REQUEST_ID_HEADERSconstant: Response headers providers use for their own request id, in the order they are looked up.
ChatResultclassResult of a chat completion: text, model, token counts, latency, tool calls, finish reason and raw response.
OpenAICompatClientclassStdlib-only client for the OpenAI chat / embeddings API (OpenAI, Azure, vLLM, Ollama, LiteLLM) with retries.
attach_chat_handlersfunction (targets: Iterable[Any], client: OpenAICompatClient, *, model_key: str='model', default_model: str | None=None, overwrite: bool=False)Give every LLM target a :func:chat_handler on client; returns the ids that got one.
chat_handlerfunction (client: OpenAICompatClient, model: str, system_prompt: str | None=None, **defaults: Any)Adapter for RouteTarget.handler: turns a RouteRequest into a chat call.
embed_handlerfunction (client: OpenAICompatClient, model: str)Adapter for an embeddings-capable RouteTarget.handler: embeds context["embeddings_input"] (or the text).
embedderfunction (client: OpenAICompatClient, model: str, batch: int=64)Adapter for SimilarityStrategy(embedder=...).
judge_fnfunction (client: OpenAICompatClient, model: str, max_tokens: int=400)Adapter for LLMJudgeStrategy(llm=...): prompt in, completion text out.
upstream_request_idfunction (headers: Any)The provider's request id from a response's headers (:data:UPSTREAM_REQUEST_ID_HEADERS), or None.

opensmartroute.adapters.optional

Optional adapters that need extra dependencies. Everything is lazily imported so the.

NameKindSummary
OpenTelemetrySinkclassLive bridge from the tracer to OpenTelemetry: every OpenSmartRoute span becomes an OTel span.
OpenTelemetryTelemetryclassEmits one span per decision and counters/histograms via the OTel API.
sentence_transformers_embedderfunction (model_name: str='sentence-transformers/all-MiniLM-L6-v2', device: str | None=None, normalize: bool=True)Semantic embedder for SimilarityStrategy. Requires opensmartroute[embeddings].

opensmartroute.adapters.personas

Import persona catalogues as TargetKind.PERSONA targets.

NameKindSummary
load_personasfunction (path: str | Path, **kw: Any)Load personas from a directory of markdown files or a JSON/JSONL/CSV catalogue.
persona_from_markdownfunction (text: str, *, path: Path | None=None, **kw: Any)Parse a *.agent.md / *.chatmode.md / front-matter markdown file into a persona target.
persona_targetfunction (name: str, prompt: str, description: str='', *, id_prefix: str='persona:', domains: list[str] | None=None, tags: list[str] | None=None, languages: list[str] | None=None, primary: bool=False, quality_prior: float=0.6, source: str='', extra: dict[str, Any] | None=None)Build a persona RouteTarget whose instructions is the system prompt (non-primary by default).
personas_from_recordsfunction (records: Iterable[dict[str, Any]], **kw: Any)Persona targets from JSON / CSV-style records (name + prompt/system keys); others skipped.

opensmartroute.adapters.semantic_router

Import a vLLM semantic-router configuration (vllm-project/semantic-router).

NameKindSummary
DEFAULT_DOMAIN_MAPconstantsemantic-router's MMLU-style categories -> OpenSmartRoute ontology domains.
SemanticRouterImportclassResult of importing a vLLM semantic-router config: registry, rules, default model, categories, warnings.
load_semantic_router_configfunction (source: str | Path | dict[str, Any], domain_map: dict[str, list[str]] | None=None, rule_weight: float=0.8)Convert a vLLM semantic-router model_config / categories document into targets and rules.

opensmartroute.adapters.skills

Load Agent-Skills SKILL.md packages as TargetKind.SKILL targets.

NameKindSummary
load_skillfunction (skill_dir: str | Path, **kw: Any)Load one skill directory (must contain SKILL.md).
load_skillsfunction (root: str | Path, **kw: Any)Load every */SKILL.md under root (one level deep, sorted by name).
parse_frontmatterfunction (text: str)Split --- frontmatter from the body. Returns (frontmatter, body).
skill_from_markdownfunction (text: str, *, path: Path | None=None, cost_per_1k_tokens: float=0.0)Parse one SKILL.md (Agent-Skills front matter + body) into a skill RouteTarget; strict validation.

opensmartroute.adapters.websearch

Web knowledge for the self-improving router: stdlib HTTP fetch, search providers, page text.

NameKindSummary
BRAVE_APIconstantBrave web search (needs BRAVE_API_KEY).
DATASETS_SERVERconstantHugging Face datasets-server REST root (/rows).
DDG_APIconstantDuckDuckGo instant-answer JSON endpoint (no key).
HF_APIconstantHugging Face Hub REST root (models, datasets, search).
SEARXNG_URL_ENVconstantbase URL of a self-hosted SearXNG instance (JSON API); registers the provider.
WIKIPEDIA_APIconstantMediaWiki search API (no key; lang = edition).
SearchHitclassOne search result: where it came from, what it says, and when it was seen.
SearchProviderconstant(query, limit) -> hits.
WebKnowledgeclassFan a query out to search providers, de-duplicate by URL and cache the hits as JSON.
brave_searchfunction (query: str, limit: int=10, *, api_key_env: str='BRAVE_API_KEY', timeout_s: float | None=None, settings: Settings | None=None)Brave Search API web results; the key is read from BRAVE_API_KEY (or BRAVE_API_KEY_FILE).
duckduckgo_searchfunction (query: str, limit: int=10, *, timeout_s: float | None=None, settings: Settings | None=None)DuckDuckGo instant-answer API (abstract + related topics). Keyless; shallow but good for definitions.
fetch_bytesfunction (url: str, *, timeout_s: float | None=None, max_bytes: int | None=None, headers: Mapping[str, str] | None=None, settings: Settings | None=None)GET url over https with a timeout and a body cap; transport errors become TargetUnavailableError.
fetch_feedfunction (url: str, *, max_entries: int | None=None, settings: Settings | None=None)Fetch an RSS / Atom feed over https and return its articles (see :func:parse_feed).
fetch_jsonfunction (url: str, *, timeout_s: float | None=None, headers: Mapping[str, str] | None=None, settings: Settings | None=None)GET a JSON document (see :func:fetch_bytes); malformed bodies raise TargetUnavailableError.
fetch_page_textfunction (url: str, *, max_chars: int=20000, timeout_s: float | None=None, settings: Settings | None=None)Fetch a page and return {url, title, text, risk}; risk is the injection/gadget risk of the text.
html_to_textfunction (html: str)(title, text) of an HTML document with scripts/styles removed and whitespace collapsed.
huggingface_searchfunction (query: str, limit: int=10, *, what: str='models', pipeline_tag: str | None='text-generation', timeout_s: float | None=None, settings: Settings | None=None)Search the Hugging Face Hub (what = models or datasets), ranked by downloads. No key needed.
parse_feedfunction (data: bytes | str, *, source: str='feed', max_entries: int=50)Articles of an RSS 2.0 / RDF / Atom document as kind="article" hits, newest first.
searxng_searchfunction (query: str, limit: int=10, *, base_url: str | None=None, timeout_s: float | None=None, settings: Settings | None=None)A self-hosted SearXNG instance (SEARXNG_URL or base_url; its JSON API must be enabled): real web.
wikipedia_searchfunction (query: str, limit: int=10, *, lang: str='en', timeout_s: float | None=None, settings: Settings | None=None)Wikipedia article search (MediaWiki API, no key): title, first snippet and the article URL. Encyclopaedic,.

opensmartroute.agents

osr run <agent>: launch a coding agent or AI client with the router as its model endpoint.

NameKindSummary
AGENTSconstantthe coding agents osr run knows how to launch, in menu order.
AgentclassOne tool the router can sit behind, and how it is told where to go.
agent_environmentfunction (agent: Agent, base_url: str, key: str, model: str='auto')The variables to set for agent (without launching anything - what --print-env shows).
find_agentfunction (agent_id: str)Return the agent with this id; ConfigurationError names the known ids when there is none.
launchfunction (agent: Agent, base_url: str, key: str, extra_args: list[str], *, model: str='auto', environ: dict[str, str] | None=None, execvpe: Any=None)Replace this process with the agent's command, the router wired in through the environment.

opensmartroute.aio

Async façade. Routing itself is CPU-bound and sub-millisecond, so we run it in.

NameKindSummary
AsyncRouterclassawait-able wrapper around a Router or EnterpriseRouter: route / learn / execute / run off the event loop.

opensmartroute.anthropic

Anthropic Messages API (POST /v1/messages) on top of the chat-completions proxy.

NameKindSummary
ERROR_TYPESconstant: HTTP status -> Anthropic error type.
MESSAGE_ROLESconstant: Messages role values (Anthropic has no system role - system is a top-level field).
STOP_REASONSconstant: finish_reason of a chat completion -> Messages stop_reason.
MessageStreamTranslatorclassPush-based translation of chat.completion.chunk SSE bytes into Messages API streaming events.
chat_to_messagefunction (completion: dict[str, Any], request: dict[str, Any] | None=None)Shape a finished chat.completion as an Anthropic message object.
count_tokensfunction (body: dict[str, Any], estimator: Any=None)POST /v1/messages/count_tokens: {"input_tokens": n} for system + messages + tools.
error_bodyfunction (status: int, message: str, *, kind: str | None=None)The {"type": "error", "error": {"type", "message"}} envelope of the Messages API.
message_eventsfunction (events: Iterable[bytes], request: dict[str, Any] | None=None)Translate a chat.completion.chunk SSE byte stream into Messages API streaming events.
messages_to_chatfunction (body: dict[str, Any], *, default_model: str='auto')Translate an Anthropic Messages request body into a chat.completion request body.

opensmartroute.branding

OpenSmartRoute naming conventions: one place for every brand-bound identifier.

NameKindSummary
API_KEY_ENVconstantOSR_API_KEY - access token the CLI sends (overrides the credentials file).
API_URL_ENVconstantOSR_API_URL - platform / server URL the CLI talks to.
BRANDconstantproduct name, one word.
CLIconstantconsole script name.
CONFIG_DIR_ENVconstantOSR_CONFIG_DIR - overrides the per-user config folder.
CONFIG_DIR_NAMEconstantper-user config folder name (~/.config/opensmartroute, %APPDATA%\opensmartroute).
ENTRY_POINT_GROUPconstantimportlib.metadata entry-point group for plugins.
ENV_PREFIXconstantOSR_ - every settings environment variable starts with this.
ERROR_CODE_PREFIXconstantOSR_ - machine-readable error codes (OSR_NO_ROUTE).
INSTALL_SCRIPT_PS1constantWindows installer: irm .../install.ps1 | iex.
INSTALL_SCRIPT_SHconstantLinux / macOS installer: curl -fsSL .../install.sh | sh.
LOCAL_TOKEN_PREFIXconstantself-hosted osr serve access tokens (osr_local_...).
MARKETPLACE_URLconstanthosted marketplace API (osr marketplace, MarketplaceClient).
METADATA_PREFIXconstantosr- - SKILL.md / persona front-matter keys (osr-domains).
PACKAGEconstantPython package / distribution / logger root.
REPOSITORYconstantpublic home of the release files (installers, bundles); the source repository is private.
SHORT_NAMEconstantshort form used for env-var and error-code prefixes.
SKILLS_DIRconstant: Default Agent-Skills root (*/SKILL.md); the location Claude Code discovers project skills in.
STATE_DIRconstantdefault learner-state directory.
TOKEN_PREFIXconstanthosted-platform API keys (osr_live_...).
WEBSITEconstantpublic website; also the default hosted-platform URL of osr login.
env_keyfunction (*parts: str)env_key("routing", "softmax_temperature") -> "OSR_ROUTING_SOFTMAX_TEMPERATURE".
error_codefunction (kind: str)error_code("no_route") -> "OSR_NO_ROUTE".
loggerfunction (component: str | None=None)logger() -> "opensmartroute"; logger("enterprise") -> "opensmartroute.enterprise".
metadata_keyfunction (field: str)metadata_key("quality_prior") -> "osr-quality-prior" (SKILL.md / persona frontmatter).
platform_urlfunction (environ: Mapping[str, str] | None=None)The platform / server URL the CLI talks to: OSR_API_URL or the hosted platform (:data:WEBSITE).
user_agentfunction ()"opensmartroute/<version>" for outbound HTTP clients.
versionfunction ()The package version (opensmartroute.__version__), resolved at call time.

opensmartroute.cli

osr command-line interface.

NameKindSummary
STARTER_COMMANDSconstantwork without --targets (starter catalogue).
build_routerfunction (targets: str | None, rules: str | None=None, state: str | None=None, models: str | None=None, skills: str | None=None, slm: str | None=None, remember_requests: int=0)Assemble the CLI's router from targets / rules / state / models / SKILL.md / SLM paths (every command).
EXTRASconstantOptional extras -> the modules they install; osr doctor reports which are present without importing them.
doctor_reportfunction (targets: str | None=None, rules: str | None=None, profile: str='default')What osr doctor prints: interpreter, package, extras, bundled model, config dir, sign-in, catalogue.
mainfunction (argv: list[str] | None=None)Entry point of the osr console script; returns the process exit status.

opensmartroute.client

Client for the hosted platform and a self-hosted osr serve: route, quote, report, chat - standard library only.

NameKindSummary
AUTO_MODELconstantthe model name that lets the router choose on the OpenAI-compatible endpoint.
PLATFORM_API_PREFIXconstantwhere the REST routes live on the platform; osr serve has them at the root.
PRIVATE_HOST_SUFFIXESconstant: Host-name suffixes that mean "inside the network": a key may travel to them over plain http.
RETRY_STATUSESconstantanswers that are safe to retry (nothing was served).
SERVER_API_PREFIXconstantthe prefix an osr_local_... token selects.
HttpResponseconstant: What the transport returns: status, response headers, raw body (or an iterator of lines when streaming).
OpenSmartRouteclassA client for the platform (https://opensmartroute.ai or your own deployment) and for osr serve.
RemoteDecisionclassWhat route() returns: the decision as the server sent it (raw) with the fields people read first.
Transportconstant: iterator of raw lines when stream is true. Tests inject a fake; the default is urllib.
is_private_hostfunction (host: str)True for loopback and private addresses and for names that only resolve inside a network (no dot,.

opensmartroute.compress

Prompt compaction: fewer tokens into the model without changing what it is asked.

NameKindSummary
TOKENS_SAVED_KEYconstantcontext key carrying the estimated saving (ExecutionResult.tokens_saved, proxy).
compact_textfunction (text: str, *, dedupe_lines: bool=True, max_blank_lines: int=1, min_dedupe_chars: int=24)Whitespace- and repetition-compacted text; fenced code blocks are returned untouched.
compress_requestfunction (request: RouteRequest, settings: Settings | None=None)(request, tokens_saved): text, history and context["system"] compacted per CompressionSettings.
trim_historyfunction (history: list[dict[str, Any]], budget_tokens: int, *, keep_last_turns: int=2)history with the oldest non-system turns dropped until it fits budget_tokens (0 = unchanged).

opensmartroute.config

Configuration loading: targets and rules from JSON or YAML.

NameKindSummary
load_documentfunction (path: str | Path)Read a JSON or YAML file (YAML needs the yaml extra); raises ConfigurationError on any problem.
load_rulesfunction (path: str | Path)Load a rules file (top-level list or rules: key) into a :class:RulesStrategy.
load_targetsfunction (path: str | Path)Load a catalogue file (top-level list or targets: key) into a :class:TargetRegistry.

opensmartroute.core

NameKindSummary
PLAN_MODESre-export of opensmartroute.core.types.PLAN_MODES: merges their outputs in slot order, and calls the tools of one round concurrently.
Capabilitiesre-export of opensmartroute.core.types.CapabilitiesDeclarative description of what a target is good at.
Objectivere-export of opensmartroute.core.types.ObjectiveWhat the caller wants to optimise. Weights are relative.
Outcomere-export of opensmartroute.core.types.OutcomeFeedback about how a routed request actually went.
PlanSlotre-export of opensmartroute.core.types.PlanSlotOne filled slot of a multi-target plan (persona -> skill -> model).
RankedTargetre-export of opensmartroute.core.types.RankedTargetA scored candidate: utility, ensemble quality estimate and the per-strategy breakdown.
RequestConstraintsre-export of opensmartroute.core.types.RequestConstraintsHard constraints on the request (never traded off).
RouteDecisionre-export of opensmartroute.core.types.RouteDecisionThe answer to route(): chosen target, confidence, alternatives, optional plan, trace and propensities.
RoutePlanre-export of opensmartroute.core.types.RoutePlanA composed route (MasRouter-style): several targets working together, executed in stages.
RouteRequestre-export of opensmartroute.core.types.RouteRequestThe customer need.
RouteTargetre-export of opensmartroute.core.types.RouteTargetA routable destination: an LLM, agent, skill, persona, tool, workflow or human.
RouteTracere-export of opensmartroute.core.types.RouteTraceEverything needed to explain a decision.
Signalsre-export of opensmartroute.core.types.SignalsCheap deterministic features extracted from a request.
StrategyScorere-export of opensmartroute.core.types.StrategyScoreOne strategy's opinion about one target.
TargetConstraintsre-export of opensmartroute.core.types.TargetConstraintsWhere / for whom a target may be used. Checked by the policy layer.
TargetKindre-export of opensmartroute.core.types.TargetKindKinds of things a request can be routed to.
TargetRegistryre-export of opensmartroute.core.registry.TargetRegistryIn-memory catalogue of :class:RouteTarget by id: add / upsert / remove, filtered listing, (de)serialisation.

opensmartroute.core.registry

Target registry: the catalogue of everything a request may be routed to.

NameKindSummary
TargetRegistryclassIn-memory catalogue of :class:RouteTarget by id: add / upsert / remove, filtered listing, (de)serialisation.

opensmartroute.core.types

Core data model for OpenSmartRoute.

NameKindSummary
TargetKindclassKinds of things a request can be routed to.
CapabilitiesclassDeclarative description of what a target is good at.
TargetConstraintsclassWhere / for whom a target may be used. Checked by the policy layer.
RouteTargetclassA routable destination: an LLM, agent, skill, persona, tool, workflow or human.
EFFORT_LEVELSconstantNamed reasoning-effort levels -> numeric effort in [0, 1] (RouteTarget.effort).
ObjectiveclassWhat the caller wants to optimise. Weights are relative.
RequestConstraintsclassHard constraints on the request (never traded off).
RouteRequestclassThe customer need.
SignalsclassCheap deterministic features extracted from a request.
StrategyScoreclassOne strategy's opinion about one target.
RankedTargetclassA scored candidate: utility, ensemble quality estimate and the per-strategy breakdown.
RouteTraceclassEverything needed to explain a decision.
PlanSlotclassOne filled slot of a multi-target plan (persona -> skill -> model).
PLAN_MODESconstant: merges their outputs in slot order, and calls the tools of one round concurrently.
RoutePlanclassA composed route (MasRouter-style): several targets working together, executed in stages.
RouteDecisionclassThe answer to route(): chosen target, confidence, alternatives, optional plan, trace and propensities.
OutcomeclassFeedback about how a routed request actually went.

opensmartroute.credentials

Credentials for the osr CLI: where the access token lives and how it is obtained.

NameKindSummary
CREDENTIALS_FILEconstantfile name inside :func:config_dir.
DEFAULT_PROFILEconstantprofile used when --profile is not given.
DEVICE_CODE_PATHconstantRFC 8628 device-authorization endpoint of the platform.
DEVICE_TOKEN_PATHconstantnoqa: S105 # nosec B105 - RFC 8628 token endpoint (a URL path).
ME_PATHconstantplatform: who am I (workspace, plan, edition).
TOKEN_BYTESconstantentropy of :func:generate_token.
WHOAMI_PATHconstantself-hosted osr serve: is this token accepted.
CredentialclassOne saved sign-in: where (url), what (token) and what the platform said about it.
CredentialStoreclassProfiles in <config dir>/credentials.json (owner-only permissions on POSIX).
PlatformClientclassMinimal JSON client for the platform / server API used by the CLI (stdlib only, injectable transport).
Transportconstant: (method, url, headers, body, timeout) -> (status, json); tests inject a fake instead of urllib.
apply_identityfunction (cred: Credential, info: Mapping[str, Any])Copy what :func:whoami learned (kind, workspace, plan, edition) onto cred.
config_dirfunction (environ: Mapping[str, str] | None=None)Per-user configuration directory (OSR_CONFIG_DIR > %APPDATA% > $XDG_CONFIG_HOME > ~/.config).
device_loginfunction (url: str | None=None, *, client_name: str | None=None, open_browser: bool=True, out: Callable[[str], None]=print, transport: Transport | None=None, sleep: Callable[[float], None]=time.sleep, timeout_s: float | None=None)Sign in to the hosted platform with the device authorization grant and return the credential.
generate_tokenfunction (prefix: str=LOCAL_TOKEN_PREFIX)A fresh random access token (osr_local_<43 chars>) for a self-hosted osr serve.
redactfunction (token: str, keep: int=6)osr_live_abc123... - the prefix plus a few characters, never the whole token.
token_kindfunction (token: str)"platform" for osr_live_ keys, "server" for osr_local_ tokens, else "unknown".
whoamifunction (cred: Credential, *, transport: Transport | None=None, timeout: float=30.0)Validate cred against its server and describe the identity behind it.

opensmartroute.data

Data files shipped inside the package.

No public names.

opensmartroute.discovery

Tool discovery beyond text similarity.

NameKindSummary
CachePreservingSelectorclassKeep the serialized tool list prefix-stable across turns of a session.
SchemaAwareStrategyclassScores tools by how many of their input_schema parameters the request can fill (see schema_match).
SkillGraphclassDependency / conflict / composition graph over skill targets.
extract_entitiesfunction (text: str)Typed entity mentions found in text (kind -> values).
schema_matchfunction (request: RouteRequest | str, target: RouteTarget)Return (coverage, filled, missing) for the target's input schema.

opensmartroute.enterprise

Enterprise integration layer: ports (hexagonal architecture), middleware and telemetry.

NameKindSummary
AuditSinkclassTamper-evident audit trail port (hash-chained).
CacheMiddlewareclassLRU decision cache keyed on (text, constraints, objective, route options). TTL in seconds.
EnterpriseRouterclassRouter + middleware chain + telemetry + auto-learning + audit. Thread-safe façade.
FileAuditSinkclassAppend-only JSONL, each line {prev, hash, record} with.
FileStateStoreclassOne JSON file per key under root; keys are hashed so they can't traverse paths.
InMemoryStateStoreclassThread-safe dict-backed :class:StateStore; values are deep-copied through JSON on read and write.
LiveContextre-export of opensmartroute.enterprise.grounding.LiveContextWhat one grounding retrieved: the query, the hits (as dicts), when, how long, and the signal that asked.
LoggingTelemetryclassStructured JSON logs; never logs raw request text (only a hash + length).
MetricsTelemetryclassIn-process counters, outcome tallies and a latency histogram.
MiddlewareclassChain-of-responsibility hook around routing: __call__(request, next_) -> RouteDecision.
RouteFnconstantthe "next" callable a middleware wraps.
RouterBuilderclassFluent builder that validates configuration and wires all enterprise pieces.
SavingsEntryre-export of opensmartroute.enterprise.savings.SavingsEntryOne routed request in the ledger.
SavingsLedgerre-export of opensmartroute.enterprise.savings.SavingsLedgerTelemetry sink that keeps a per-request baseline-vs-routed cost ledger (see module docs).
SavingsReportre-export of opensmartroute.enterprise.savings.SavingsReportAggregate savings and the quality they were bought at.
StateStoreclassKey/value persistence port for learner state, caches, breaker state.
TelemetryclassObserver port with no-op defaults. Implement for OpenTelemetry, Prometheus, Datadog….
TenantMiddlewareclassEnforces that a tenant is present and applies per-tenant defaults/limits.
TimeoutMiddlewareclassSoft deadline: raise if routing itself exceeded budget_ms (should never happen.
WebGroundingMiddlewarere-export of opensmartroute.enterprise.grounding.WebGroundingMiddlewareSearch the web for requests about current facts and put the hits in front of the model (module docstring).
hardenfunction (router: Router, *, guard: bool=False, redact_pii: bool=False, learned_guard: bool=False, output_guard: str | Any='off', health: bool=False, latency_slo_ms: float | None=None, audit: str | Path | AuditSink | None=None, audit_outcomes: bool=False, origin_policy: Any=False, limits: Any=False, web_grounding: Any=False, knowledge: Any=False, state_store: StateStore | str | None=None, save_every: int=1, middleware: list[Middleware] | None=None, telemetry: list[Telemetry] | None=None)Wrap an already built :class:Router in the production controls (what osr serve --guard ... does).
search_queryre-export of opensmartroute.enterprise.grounding.search_queryThe search query for a request: its first sentence or line, whitespace-normalised, filler words dropped.

opensmartroute.enterprise.grounding

Live web context in front of the model: search the web for requests about now and hand the hits to the target.

NameKindSummary
GROUNDING_HEADERconstant: Opens the block appended to the system prompt; {when} is the retrieval time in UTC.
LiveContextclassWhat one grounding retrieved: the query, the hits (as dicts), when, how long, and the signal that asked.
WebGroundingMiddlewareclassSearch the web for requests about current facts and put the hits in front of the model (module docstring).
search_queryfunction (text: str, max_chars: int=200)The search query for a request: its first sentence or line, whitespace-normalised, filler words dropped.

opensmartroute.enterprise.ops

Operational controls: shadow / A-B routing, tenant fairness and queue-aware latency.

NameKindSummary
SPRTclassWald SPRT for two Bernoulli success rates: H0 p=p0 vs H1 p=p0+delta.
ABTestclassTraffic split + SPRT comparison of candidate vs control outcomes.
FairShareMiddlewareclassDominant Resource Fairness across tenants over a sliding window.
InflightTrackerclassPer-target in-flight counters + arrival/service statistics for queueing estimates.
QueueAwareStrategyclassScores targets by current time-to-first-token = catalogue latency + queueing wait vs.
ShadowMiddlewareclassRun candidate beside production. mode='shadow': log only. mode='ab': serve.
TenantUsageclassSliding-window ledger of one tenant's requests, cost and tokens for :class:FairShareMiddleware.

opensmartroute.enterprise.savings

Savings ledger - the always-on savings report that backs the ROI story and the dashboard.

NameKindSummary
SavingsEntryclassOne routed request in the ledger.
SavingsLedgerclassTelemetry sink that keeps a per-request baseline-vs-routed cost ledger (see module docs).
SavingsReportclassAggregate savings and the quality they were bought at.

opensmartroute.enterprise.stores

Production state-store backends and wrappers.

NameKindSummary
BatchedStateStoreclassWrite-behind buffer. Reads are served from the pending map first.
EncryptedStateStoreclassAES-256-GCM envelope encryption. key is 32 raw bytes, or read (base64/hex/raw).
Migrationconstantstate-document transform applied by VersionedStateStore.
NamespacedStateStoreclassPrefix every key with <namespace>/ so several routers or tenants can share one backing store.
RedisStateStoreclassStateStore on any redis-py-compatible client (get / set / delete), JSON values, optional TTL.
SQLStateStoreclassDB-API 2.0 key/value store (PostgreSQL, SQLite, MySQL).
VersionedStateStoreclassSchema-versioned envelope with forward migrations.
open_state_storefunction (url: str, *, encrypt_key_env: str | None=None)Build a store from a URL (osr serve --state-store, OSR_SERVER_STATE_STORE).

opensmartroute.errors

Exception hierarchy. Every error raised by the SDK derives from :class:OpenSmartRouteError.

NameKindSummary
OpenSmartRouteErrorclassBase class for all SDK errors.
ConfigurationErrorclassInvalid catalogue, rules, SKILL.md, settings or a missing optional dependency.
ValidationErrorclassA request, outcome or target failed validation.
NoRouteErrorclassNo target satisfied the hard constraints.
TargetUnavailableErrorclassA remote target or catalogue source could not be reached.
ExecutionErrorclassA target handler failed while executing a plan.
SecurityErrorclassRequest rejected by an input guard (prompt injection, oversize, etc.).
StateStoreErrorclassA learner-state store failed to load, save or migrate.
AuthenticationErrorclassThe CLI has no valid access token for the platform / server, or a sign-in was denied or timed out.
APIErrorclassThe platform or osr serve answered a call with an error status; status, detail and the.
RateLimitErrorclass429: the rate limit, the monthly quota or a budget is exhausted; retry_after says when to try again.
OpenSmartRouteDeprecationWarningclassEmitted by :func:deprecated; filter with warnings.simplefilter on this class.
deprecatedfunction (name: str, *, since: str, removal: str, replacement: str | None=None, stacklevel: int=3)Announce a deprecation according to the stability policy in SDK.md.

opensmartroute.estimate

Token, cost and latency estimates before a request is sent anywhere.

NameKindSummary
DEFAULT_OUTPUT_TOKENSconstant: Output length assumed when neither the caller nor the signals say (a short answer).
MESSAGE_OVERHEAD_TOKENSconstant: Tokens a chat API adds per message for role / separators (OpenAI-style framing).
PriceHookconstant: (usd_per_1k_input, usd_per_1k_output) for a target, or None to fall back to its declared cost.
RequestEstimateclassA quote for one request across every candidate, with named picks.
TargetEstimateclassThe quote for one candidate target.
build_token_counterfunction (spec: str)The counter a tokenizer spec names (EstimateSettings.tokenizer): tiktoken:<encoding> counts with.
estimatefunction (router: Router, request: RouteRequest | str, *, output_tokens: int | None=None, kinds: list[str] | None=None, prices: PriceHook | None=None, quality_tolerance: float=0.1, decision: RouteDecision | None=None)Quote request against every candidate the router would consider - nothing is executed.
estimate_messages_tokensfunction (messages: list[dict[str, Any]])Token estimate for an OpenAI-style message list (content plus per-message framing).
estimate_tokensfunction (text: str)Token count of text: the installed :data:token_counter when there is one (installed by.
heuristic_tokensfunction (text: str)The dependency-free tokenizer approximation (what :func:estimate_tokens uses unless a counter is installed).
set_token_counterfunction (counter: Callable[[str], int] | None)Install (or, with None, remove) the process-wide token counter behind :func:estimate_tokens; the.
target_pricesfunction (target: RouteTarget)(usd per 1k input tokens, usd per 1k output tokens) from the target's declared cost.
token_counterconstant: :class:~opensmartroute.learning.neural.TokenCountModel trained on billed counts (None = the heuristic).

opensmartroute.eval

RouterBench-style evaluation harness.

NameKindSummary
AuditReportre-export of opensmartroute.eval.audit.AuditReportAggregate of a shadow replay; to_markdown renders the Routing Audit deliverable.
AuditRowre-export of opensmartroute.eval.audit.AuditRowOne logged request: what happened, and what the router would have done.
DatasetCollectorre-export of opensmartroute.eval.collect.DatasetCollectorCache-backed corpus builder: collect sources, add feedback / synthetic rows, dedupe, split.
DatasetSourcere-export of opensmartroute.eval.collect.DatasetSourceOne Hub dataset split to collect: repo id, config, split, the preset that parses it, an optional model map.
EvalResultclassAggregate routing metrics over a dataset: accuracy, cost, latency, confidence, calibration, coverage.
EvalRowclassOne labelled prompt: expected / acceptable targets or per-target quality scores (RouterBench style).
area_under_frontierfunction (points: list[dict[str, float]])Trapezoidal area under accuracy(cost) — analogous to RouterBench's AIQ.
calibration_reportfunction (router: Router, rows: list[EvalRow], objective: Objective | None=None)Confidence calibration of the router on labelled rows: ECE, Brier, reliability bins and.
collect_datasetre-export of opensmartroute.eval.collect.collect_datasetDownload source from the Hub and parse it into rows.
cost_quality_frontierfunction (router: Router, rows: list[EvalRow], cost_weights: list[float] | None=None)Sweep the cost weight to trace the accuracy-vs-cost curve (RouterBench Fig. 1 style).
evaluatefunction (router: Router, rows: list[EvalRow], objective: Objective | None=None)Route every row and score accuracy, realised quality, cost, latency, ECE / Brier and conformal coverage.
fetch_hf_rowsre-export of opensmartroute.eval.collect.fetch_hf_rowsOne page (max 100) of records from the datasets-server /rows endpoint, flattened to {column: value}.
load_audit_logre-export of opensmartroute.eval.audit.load_audit_logRead a JSONL traffic log (text or prompt/messages per line).
load_datasetfunction (path: str | Path)Read a JSONL evaluation dataset (text or prompt plus the optional EvalRow keys); a line that is.
model_qualityre-export of opensmartroute.eval.collect.model_qualityData-derived quality prior per model in [0, 1]: pairwise rows (two scored models) are fitted with.
routing_auditre-export of opensmartroute.eval.audit.routing_auditReplay rows through router in shadow mode and aggregate an :class:AuditReport.
rows_from_feedbackre-export of opensmartroute.eval.collect.rows_from_feedbackRows from :class:Outcome records. texts maps request_id to the prompt (outcomes carry no text);.
summarize_auditre-export of opensmartroute.eval.audit.summarize_auditAggregate :class:AuditRow records into an :class:AuditReport.
synthetic_followup_rowsre-export of opensmartroute.eval.collect.synthetic_followup_rowsConversation rows built from labelled rows.
synthetic_rowsre-export of opensmartroute.eval.collect.synthetic_rowsOntology seed prompts (templates and, with natural, the everyday phrasings - see.
synthetic_slot_rowsre-export of opensmartroute.eval.collect.synthetic_slot_rowsOntology seed prompts scored for a plan slot: every role target of the catalogue (personas, skills,.

opensmartroute.eval.agentic

tau-bench-style agentic task evaluation: does per-step routing beat the best single agent?.

NameKindSummary
AgentTaskclassA multi-step task with the measured per-step success probability and latency of each agent.
load_agentic_tasksfunction (path: str | Path)Read tasks from JSONL: {"task_id", "steps", "success", "latency_ms", "cost_usd"?, "domain"?}.
synthetic_agentic_tasksfunction (n: int=200, *, seed: int=0, hard_share: float=0.3)Three agents (fast / balanced / strong) and n tasks of 3-5 steps whose steps are mostly.
task_routing_frontierfunction (registry: TargetRegistry, tasks: Sequence[AgentTask], *, retries: int=1, accuracy_tolerance: float=0.02, latency_ratio_target: float=0.9, objective: Objective | None=None, router_factory: Callable[[TargetRegistry], Router] | None=None, seed: int=0)Replay tasks step by step through :class:ProgressRouter (each agent's handler draws the.

opensmartroute.eval.audit

Routing Audit - shadow-mode replay of logged LLM traffic to quantify what the router would change.

NameKindSummary
DEFAULT_COMPLETION_TOKENSconstant: Default assumed output length when a log row carries no completion_tokens.
AuditReportclassAggregate of a shadow replay; to_markdown renders the Routing Audit deliverable.
AuditRowclassOne logged request: what happened, and what the router would have done.
load_audit_logfunction (path: str | Path)Read a JSONL traffic log (text or prompt/messages per line).
routing_auditfunction (router: Router, rows: list[dict[str, Any]], *, baseline: str | None=None, monthly_requests: int | None=None, plan: bool=False, keep_rows: bool=True)Replay rows through router in shadow mode and aggregate an :class:AuditReport.
summarize_auditfunction (rows: list[AuditRow], *, monthly_requests: int | None=None, keep_rows: bool=True)Aggregate :class:AuditRow records into an :class:AuditReport.

opensmartroute.eval.baselines

Baselines every router must beat, plus oracle ceilings and the sampling noise floor.

NameKindSummary
Policyconstanta baseline: (row, targets) -> chosen target id.
PolicyResultclassMetrics of one baseline policy on a dataset (same fields as the router's EvalResult headline).
baseline_suitefunction (rows: list[EvalRow], targets: Sequence[RouteTarget], train_rows: list[EvalRow] | None=None, quality_floor: float=0.7, seed: int=0)Run every baseline and both oracles; train_rows fit the task table.
best_prior_policyfunction (row: EvalRow, targets: Sequence[RouteTarget])Always the target with the highest declared quality_prior.
cheapest_policyfunction (row: EvalRow, targets: Sequence[RouteTarget])Always the lowest unit cost (ties broken by latency).
compare_policiesfunction (router_hits: Sequence[bool], baselines: Sequence[PolicyResult], *, seed: int=0, n_boot: int=2000, level: float=0.95)The router against each baseline on the same rows: accuracy delta with a paired percentile-bootstrap.
evaluate_policyfunction (name: str, policy: Policy, rows: list[EvalRow], targets: Sequence[RouteTarget])Run a baseline :data:Policy over the rows and aggregate accuracy, quality, cost and latency.
fit_task_tablefunction (rows: list[EvalRow], targets: Sequence[RouteTarget])task_type -> target with the highest mean quality on the training rows.
held_out_splitfunction (rows: Sequence[EvalRow], seed: int=0, train_fraction: float=0.5)A seeded shuffle split into (train, test): the task table (and anything else fitted) learns from.
most_expensive_policyfunction (row: EvalRow, targets: Sequence[RouteTarget])Always the highest unit cost - the "just use the frontier model" baseline.
multi_sample_oraclefunction (rows: list[EvalRow], targets: Sequence[RouteTarget])Mean quality of the oracle that picks by the mean over all samples per target.
noise_floorfunction (rows: list[EvalRow], targets: Sequence[RouteTarget], seed: int=0)How much accuracy is lost to sampling noise alone.
oracle_policyfunction (quality_floor: float | None=None)Per-row best target. With quality_floor -> cheapest target reaching the floor.
random_policyfunction (seed: int=0)Uniformly random target (seeded).
static_task_table_policyfunction (table: dict[str, str])Look the row's task_type up in a table from :func:fit_task_table ("*" = default).

opensmartroute.eval.collect

Collect routing datasets from the Hugging Face Hub, your own feedback log and synthetic seeds.

NameKindSummary
ARENA_TIERSconstanteverything else in those datasets is a 7B-14B chat model -> "small".
DATASETS_SERVERre-export of opensmartroute.adapters.websearch.DATASETS_SERVERHugging Face datasets-server REST root (/rows).
DEFAULT_SOURCESconstantpublic ones.
FOLLOWUP_PROMPTSconstant: task-neutral: "translate it" or "summarise that" name a task of their own and belong to a different target.
HISTORY_FILEconstantthe self-improver's report log; lives in the cache dir but is not a dataset.
KNOWN_SOURCESconstantpairwise human / judge preference battles - the winner is the label.
PAIRWISEconstantpreset name for battle datasets: prompt / conversation, model_a, model_b, winner.
PARSERSconstantcollector presets that are not wide / nested tables.
REWARD_BENCHconstantpreset name for RewardBench: prompt, chosen_model, rejected_model, subset.
ROUTELLM_GOOD_ENOUGHconstantRouteLLM's threshold: a weak-model score >= 4 means the cheap model was good enough.
ROUTELLM_GPT4constantpreset name for RouteLLM's gpt4_dataset: prompt + GPT-4-judged Mixtral score 1-5.
ROUTELLM_STRONGconstantthe strong model of RouteLLM's gpt4_dataset (its answers are the reference).
ROUTELLM_WEAKconstantthe weak model whose answer GPT-4 scores 1-5 against the reference.
ULTRAFEEDBACKconstantpreset name for UltraFeedback: instruction + N completions with a model and score.
DatasetCollectorclassCache-backed corpus builder: collect sources, add feedback / synthetic rows, dedupe, split.
DatasetSourceclassOne Hub dataset split to collect: repo id, config, split, the preset that parses it, an optional model map.
Parserconstantrecord -> row (or None to skip).
collect_datasetfunction (source: DatasetSource, *, timeout_s: float | None=None, settings: Settings | None=None)Download source from the Hub and parse it into rows.
fetch_hf_rowsfunction (dataset: str, *, config: str='default', split: str='train', offset: int=0, length: int=100, timeout_s: float | None=None, settings: Settings | None=None)One page (max 100) of records from the datasets-server /rows endpoint, flattened to {column: value}.
from_pairwise_rowfunction (rec: dict[str, Any], model_map: dict[str, str] | None=None, source: DatasetSource | None=None)A battle record -> row with scores 1 / 0 for winner / loser (0.5 each on a tie). Understands the.
from_reward_bench_rowfunction (rec: dict[str, Any], source: DatasetSource | None=None)A RewardBench record (prompt, chosen_model, rejected_model, subset) -> a pairwise row.
from_routellm_gpt4_rowfunction (rec: dict[str, Any], source: DatasetSource | None=None)A routellm/gpt4_dataset record (prompt + mixtral_score 1-5) -> the strong model scores 1.0 and.
from_ultrafeedback_rowfunction (rec: dict[str, Any], source: DatasetSource | None=None)An UltraFeedback record (instruction + completions[{model, overall_score}]) -> a scored row with.
iter_hf_rowsfunction (source: DatasetSource, *, offset: int=0, timeout_s: float | None=None, settings: Settings | None=None)Page through source from raw record offset until source.limit records or the split is exhausted.
model_qualityfunction (rows: Iterable[EvalRow], *, min_n: int=20, epochs: int=5, seed: int=0)Data-derived quality prior per model in [0, 1]: pairwise rows (two scored models) are fitted with.
rows_from_feedbackfunction (outcomes: Iterable[Outcome] | FeedbackStore, texts: dict[str, str] | None=None, *, min_quality: float=0.5, role: str | None=None)Rows from :class:Outcome records. texts maps request_id to the prompt (outcomes carry no text);.
rows_from_recordsfunction (records: Iterable[dict[str, Any]], source: DatasetSource)Parse raw records with the source's preset (text_key is tried first when set).
synthetic_followup_rowsfunction (rows: Sequence[EvalRow], *, per_row: int=1, switches: int=0, seed: int=0, assistant: str='(answer)')Conversation rows built from labelled rows.
synthetic_rowsfunction (targets: Sequence[RouteTarget], *, per_template: int=2, seed: int=0, router: Any | None=None, natural: bool | int=True, natural_phrases: Mapping[str, Sequence[str]] | None=None)Ontology seed prompts (templates and, with natural, the everyday phrasings - see.
synthetic_slot_rowsfunction (targets: Sequence[RouteTarget], role: str, *, per_template: int=2, seed: int=0)Ontology seed prompts scored for a plan slot: every role target of the catalogue (personas, skills,.
tier_model_mapfunction (tiers: Mapping[str, str], models: Iterable[str]=())model_map for a :class:DatasetSource: every known battle model (plus models) -> the target id.
tier_offunction (model: str)Tier of a battle-dataset model name: the :data:ARENA_TIERS entry, else the name rules, else the parameter.

opensmartroute.eval.criteria

Offline realisations of the release exit criteria.

NameKindSummary
CriterionResultclassOne measured exit criterion.
bootstrap_cifunction (values: Sequence[float], *, n_boot: int=1000, level: float=0.95, seed: int=0)Percentile bootstrap confidence interval for the mean of values.
cold_start_ratiofunction (n_outcomes: int=200, *, domain: str='legal', n_eval: int=200, seed: int=0, target_ratio: float=0.9, generalist_quality: float=0.6, explore_rate: float=0.1, max_requests: int=20000)Accuracy on domain prompts of a target declared with only id / kind / cost.
conformal_coveragefunction (alpha: float=0.1, *, n_cal: int=1000, n_test: int=1000, label_noise: float=0.5, seed: int=0, tolerance: float=0.02)Fit :class:ConformalCalibrator on the real router's propensities over labelled prompts and.
domain_expert_registryfunction (domains: Sequence[str]=_DOMAINS, *, strip: str | None=None, generalist: bool=False)One hand-configured expert per domain. strip=<domain> replaces that expert with a.
effort_token_savingsfunction (registry: TargetRegistry, rows: Sequence[EvalRow], *, quality_tolerance: float=0.02, token_ratio_target: float=0.7, objective: Objective | None=None, router_factory: Callable[[TargetRegistry], Router] | None=None)Route rows that carry per-target scores and tokens and compare the tokens spent.
knapsack_never_exceeds_capfunction (steps: int=1000000, *, window: int=1000, drift_every: int=100000, seed: int=0)Drive :class:MultiKnapsackBandit (on_capped="abstain") for steps pulls with.
match_at_1function (router: Router | EnterpriseRouter, prompts: Sequence[tuple[str, str]])Fraction of (text, expected_target_id) pairs the router gets right at rank 1.
match_at_1_at_scalefunction (small: int=50, large: int=5000, *, n_prompts: int=200, seed: int=0, max_drop: float=0.05, narrow_above: int=32, narrow_to: int=24)Match@1 on a small catalogue versus a large one with retrieve-then-rank narrowing.
multi_round_vs_best_singlefunction (registry: TargetRegistry, rows: Sequence[EvalRow], *, threshold: float=0.8, max_rounds: int=3, failures_before_switch: int=1, objective: Objective | None=None, cost_ratio_target: float=0.6, router_factory: Callable[[TargetRegistry], Router] | None=None)Run :class:MultiRoundExecutor over rows that carry per-target scores (RouterBench.
ope_within_live_cifunction (n_log: int=2000, *, seed: int=0, logging_objective: Objective | None=None, target_objective: Objective | None=None, logging_temperature: float=0.3)Log decisions from a cost-seeking router (actions sampled from its propensities), estimate.
run_allfunction (*, quick: bool=True, seed: int=0)Every offline criterion. quick shrinks the expensive simulations (10^5 steps, 1 000 tools,.
synthetic_effort_rowsfunction (n: int=400, *, seed: int=0)One reasoning model exposed as two effort siblings (reasoner@fast with effort="low".
synthetic_scored_rowsfunction (n: int=300, *, seed: int=0)A three-tier catalogue (small / medium / large) and RouterBench-style rows with a quality.
synthetic_tool_cataloguefunction (n: int, *, seed: int=0)n distinct tools (verb x object x qualifier, up to 5 000 unique combinations) and one.

opensmartroute.eval.datasets

Adapters from public routing benchmarks to :class:EvalRow.

NameKindSummary
PRESETSconstantNamed layouts accepted by load_benchmark / osr eval --preset.
NestedPresetclassLayout of a "nested" benchmark: a per-model mapping under container_keys holding score / cost / samples.
WidePresetclassColumn layout of a "wide" benchmark: one row per prompt, <model><suffix> columns for scores / costs.
from_nested_rowfunction (row: dict[str, Any], preset: NestedPreset, model_map: dict[str, str] | None=None)Convert one nested-format record to an :class:EvalRow, keeping repeated samples when present.
from_wide_rowfunction (row: dict[str, Any], preset: WidePreset, model_map: dict[str, str] | None=None)Convert one wide-format record to an :class:EvalRow (None when it has no prompt or scores).
load_benchmarkfunction (path: str | Path, preset: str | WidePreset | NestedPreset='routerbench', model_map: dict[str, str] | None=None, limit: int | None=None)Load a benchmark file (.jsonl / .json / .csv) into :class:EvalRow objects.
models_infunction (rows: Iterable[EvalRow])Distinct target ids that carry scores in the rows, in first-seen order.

opensmartroute.eval.frontier

Three-objective frontier and ablations.

NameKindSummary
ablation_reportfunction (router: Router, rows: list[EvalRow], objective: Objective | None=None)Leave-one-out over strategies (and extractors, when explicitly set on the router).
frontier3function (router: Router, rows: list[EvalRow], cost_weights: list[float] | None=None, latency_weights: list[float] | None=None)Sweep cost / latency objective weights and return the Pareto-optimal (quality, cost, latency) points.
hypervolumefunction (points: list[dict[str, float]], ref: tuple[float, float, float] | None=None)Dominated hypervolume of the Pareto points in normalised (quality, cost, latency) space.

opensmartroute.eval.headroom

Routing headroom: when does routing pay, and how much catalogue does it need?.

NameKindSummary
DiversityReportclassHow differently the targets score rows: pairwise disagreement, winner entropy and winner share.
HeadroomReportclassOracle vs best-single quality on a dataset, with the label-noise floor that says whether the gap is real.
learnability_by_difficultyfunction (rows: Sequence[EvalRow], targets: Sequence[RouteTarget], buckets: int=4)Routing gain (oracle - best single) per difficulty bucket. Difficulty of a row is.
min_cataloguefunction (rows: Sequence[EvalRow], targets: Sequence[RouteTarget], fraction: float=0.95)Greedy forward selection: the smallest ordered subset whose oracle reaches fraction of the.
routing_headroomfunction (rows: Sequence[EvalRow], targets: Sequence[RouteTarget], seed: int=0)Oracle minus best-single quality with the label-noise floor for context.
scaling_curvefunction (rows: Sequence[EvalRow], targets: Sequence[RouteTarget], sizes: Sequence[int] | None=None, *, trials: int=20, seed: int=0)Mean oracle / best-single quality and headroom on random subsets of each size.
target_diversityfunction (rows: Sequence[EvalRow], targets: Sequence[RouteTarget])How different the targets are on this data: pairwise score disagreement and winner spread.

opensmartroute.eval.ope

Off-policy evaluation from logged routing decisions.

NameKindSummary
LoggedDecisionclassOne logged routing event for off-policy evaluation: text, action taken, its propensity and observed reward.
OPEResultclassIPS, self-normalised IPS and doubly-robust value estimates with effective sample size and clip count.
ips_confidence_intervalfunction (logs: list[LoggedDecision], weights: list[float], z: float=1.96)Normal-approximation CI for the IPS estimate given per-sample weights.
mean_reward_modelfunction (logs: list[LoggedDecision])Simplest DR reward model: mean logged reward per action, global mean for unseen actions.
off_policy_evaluatefunction (router: Router, logs: list[LoggedDecision], objective: Objective | None=None, max_weight: float=20.0, reward_model: Callable[[LoggedDecision, str], float] | None=None, deterministic: bool=False)Estimate the value of router on logged traffic.
target_propensitiesfunction (router: Router, lg: LoggedDecision, objective: Objective | None=None)The evaluated router's P(target | x) for a logged context (empty when it has no route).

opensmartroute.eval.robustness

Robustness and fairness checks for a router.

NameKindSummary
coresetfunction (rows: list[EvalRow], k: int, embedder: Callable[[list[str]], list[list[float]]] | None=None, seed: int=0)Greedy k-center (farthest-first) subset of rows for a diverse evaluation set.
diversityfunction (rows: list[EvalRow], embedder: Callable[[list[str]], list[list[float]]] | None=None, sample: int=300)Mean pairwise cosine distance of the row texts (sampled), 0 = all identical.
paraphrase_robustnessfunction (router: Router, rows: list[EvalRow], n: int=4, objective: Objective | None=None, seed: int=0)Share of rule-based paraphrases that route to the same target as the original (robustness).
paraphrasesfunction (text: str, n: int=4, seed: int=0)n deterministic surface-level paraphrases of text (never returns the original).
profile_swap_fairnessfunction (router: Router, rows: list[EvalRow], profiles: Sequence[dict[str, Any]] | None=None, objective: Objective | None=None)Decision agreement across user profiles. dependence = fraction of rows whose.
repeat_flip_ratefunction (router: Router, rows: list[EvalRow], k: int=5, objective: Objective | None=None)Route each row k times; flip_rate = share of rows whose decision is not identical every time.

opensmartroute.execution

Plan-aware execution: turn a :class:RouteDecision into a real answer.

NameKindSummary
ON_DELTA_KEYconstantHandler keyword that receives streamed text deltas (also mirrored into request.context).
PRELUDE_ROLESconstantRoles executed before the primary target, in this order.
SSE_DONEconstantthe terminal event of an OpenAI chat stream.
SSE_KEEPALIVEconstantan SSE comment: keeps proxies and clients on the line while a model is silent.
TOOL_ROLEconstantPlan role whose targets are offered to the model as callable functions.
ChatStreamclassRelay one chat completion as OpenAI chat.completion.chunk events while the handler is still generating.
ExecutionResultclassWhat happened when a decision was executed.
ExecutionStepclassOne executed plan slot: role, target, latency and whether it succeeded. stage orders the steps.
aexecuteasync function (decision: RouteDecision, request: RouteRequest, learn: LearnFn | None=None, *, min_slot_confidence: float=0.0, task_id: str | None=None, **kw: Any)Async twin of :func:execute; awaits coroutine handlers.
completion_chunksfunction (completion: dict[str, Any], chunk_chars: int | None=None)Serve a finished OpenAI chat.completion as chat.completion.chunk events.
cost_tokensfunction (result: ExecutionResult)(total tokens, USD) of an execution: the handler's token count and the sum of every outcome's cost.
executefunction (decision: RouteDecision, request: RouteRequest, learn: LearnFn | None=None, *, min_slot_confidence: float=0.0, task_id: str | None=None, preferred_over: str | None=None, **kw: Any)Run the plan (persona -> skill -> primary) and record outcomes via learn.
execute_with_fallbackfunction (decision: RouteDecision, request: RouteRequest, candidates: list[RouteTarget] | None=None, fallbacks: bool=True, learn: LearnFn | None=None, *, router: Any=None, committed: Callable[[], bool] | None=None, on_attempt: Callable[[RouteDecision], None] | None=None, max_fallbacks: int | None=None, settings: Settings | None=None, **handler_kw: Any)Run the decision; when its handler fails, re-route among the remaining executable targets and retry.
repair_completionfunction (completion: dict[str, Any])Repair the assistant text of a finished chat completion in place when it is almost JSON.
sse_eventfunction (payload: dict[str, Any])One server-sent event carrying payload as JSON (data: {...}\n\n).
tool_namefunction (target: RouteTarget)The function name a tool target is offered under (OpenAI: ^[A-Za-z0-9_-]{1,64}$).
tool_specfunction (target: RouteTarget)OpenAI function-calling spec of a tool target: its MCP input_schema when it has one, else a single input.

opensmartroute.feedback

Feedback store: append-only outcome log that closes the learning loop.

NameKindSummary
FeedbackStoreclassAppend-only :class:Outcome log (in memory or JSONL file) with per-target statistics.

opensmartroute.gemini

Google Gemini API dialect: generateContent / streamGenerateContent over the routed chat path.

NameKindSummary
FINISH_REASONSconstant: Gemini finishReason for an OpenAI finish_reason.
STATUS_NAMESconstant: gRPC-style status names Google error envelopes carry, per HTTP status.
GenerateStreamTranslatorclassPush-based translation of chat.completion.chunk SSE bytes into streamGenerateContent?alt=sse.
chat_to_generatefunction (completion: dict[str, Any], request: dict[str, Any] | None=None)Shape a finished chat.completion as a GenerateContentResponse.
count_tokensfunction (body: dict[str, Any], model: str, estimator: Any=None):countTokens - {"totalTokens": n} from the router's own token estimate (no provider call).
decode_inlinefunction (inline: dict[str, Any])The bytes of an inlineData part (helper for callers that store or inspect uploads).
error_bodyfunction (status: int, message: str)Google's {"error": {"code", "message", "status"}} envelope.
generate_eventsfunction (events: Iterable[bytes], request: dict[str, Any] | None=None, model: str='auto')Translate a chat.completion.chunk SSE byte stream into streamGenerateContent SSE frames.
generate_to_chatfunction (body: dict[str, Any], model: str, *, default_model: str='auto')Translate a generateContent request into a chat.completion request body.

opensmartroute.inference

Run open-weight models on the CPU and RAM you have: footprint arithmetic, host planning, Ollama control.

NameKindSummary
DEFAULT_URLre-export of opensmartroute.inference.ollama.DEFAULT_URLwhere ollama serve listens by default.
GIBre-export of opensmartroute.inference.footprint.GIBbytes in a GiB (container memory limits are written as 28Gi).
KNOWN_TAGSre-export of opensmartroute.inference.footprint.KNOWN_TAGSOnly a fallback for planning without a server; /api/show gives the exact figures.
KV_FALLBACKSre-export of opensmartroute.inference.planner.KV_FALLBACKSThe order the planner tries KV precisions when a model does not fit at the requested one.
KV_TYPE_BYTESre-export of opensmartroute.inference.footprint.KV_TYPE_BYTESBytes per KV cache element by OLLAMA_KV_CACHE_TYPE (the quantised types carry block scales).
QUANT_BITSre-export of opensmartroute.inference.footprint.QUANT_BITSBits per weight of GGUF file types: measured file size x 8 / parameter count (llama.cpp quantize output).
Footprintre-export of opensmartroute.inference.footprint.FootprintMemory one loaded model takes at a given context, parallelism and KV precision.
Hostre-export of opensmartroute.inference.footprint.HostA machine (or container) that serves models: its memory, cores and, when known, bandwidth and store speed.
ModelSpecre-export of opensmartroute.inference.footprint.ModelSpecAn open-weight model as the planner sees it: size, quantisation and the attention shape behind its KV cache.
OllamaClientre-export of opensmartroute.inference.ollama.OllamaClientControl-plane client for one Ollama server: tags, show, ps, load, unload, exact :class:ModelSpec rows.
Placementre-export of opensmartroute.inference.planner.PlacementThe models one host keeps resident, what they use and the server environment that pins them.
Planre-export of opensmartroute.inference.planner.PlanPlacements per host plus the models that fit nowhere (with the reason).
ResidentModelre-export of opensmartroute.inference.ollama.ResidentModelOne row of /api/ps: a model currently loaded, where it sits and when it expires.
Throughputre-export of opensmartroute.inference.footprint.ThroughputTokens per second a host gives a model, and how long a cold load takes.
api_rootre-export of opensmartroute.inference.ollama.api_roothttp://host:11434/v1 or http://host:11434/ -> http://host:11434 (validated http/https).
footprintre-export of opensmartroute.inference.footprint.footprintMemory spec needs when loaded with context tokens per slot in parallel slots (defaults: settings).
format_bytesre-export of opensmartroute.inference.footprint.format_bytesHuman size in binary units (2.4 GiB), the way ollama ps and container limits read.
kv_type_bytesre-export of opensmartroute.inference.footprint.kv_type_bytesBytes per KV cache element for an OLLAMA_KV_CACHE_TYPE value.
ollama_envre-export of opensmartroute.inference.planner.ollama_envOLLAMA_* variables that make a server hold exactly these models with these slots.
parse_memoryre-export of opensmartroute.inference.footprint.parse_memory"28Gi" / "8 GiB" / "16GB" / "512Mi" / a number of bytes -> bytes (Kubernetes and SI units).
planre-export of opensmartroute.inference.planner.planFirst-fit-decreasing packing of models onto hosts at the requested context and parallelism.
quant_bitsre-export of opensmartroute.inference.footprint.quant_bitsBits per weight of a GGUF quantisation name (q4_k_m, Q8_0, F16, MXFP4); unknown Qn -> n.5.
recommendre-export of opensmartroute.inference.planner.recommendThe largest configuration of spec that fits budget bytes, or None when even the weights do not.
renderre-export of opensmartroute.inference.planner.renderA plain-text report of a plan: per host the resident models, memory used / free and the OLLAMA_* lines.
throughputre-export of opensmartroute.inference.footprint.throughputBandwidth-bound decode, compute-bound prefill and cold-load time of spec on host.

opensmartroute.inference.footprint

What an open-weight model needs from a CPU host: memory for weights and KV cache, and the tokens per second.

NameKindSummary
GIBconstantbytes in a GiB (container memory limits are written as 28Gi).
KNOWN_TAGSconstantOnly a fallback for planning without a server; /api/show gives the exact figures.
KV_TYPE_BYTESconstantBytes per KV cache element by OLLAMA_KV_CACHE_TYPE (the quantised types carry block scales).
QUANT_BITSconstantBits per weight of GGUF file types: measured file size x 8 / parameter count (llama.cpp quantize output).
FootprintclassMemory one loaded model takes at a given context, parallelism and KV precision.
HostclassA machine (or container) that serves models: its memory, cores and, when known, bandwidth and store speed.
ModelSpecclassAn open-weight model as the planner sees it: size, quantisation and the attention shape behind its KV cache.
ThroughputclassTokens per second a host gives a model, and how long a cold load takes.
footprintfunction (spec: ModelSpec, *, context: int | None=None, parallel: int | None=None, kv_type: str | None=None, settings: Settings | None=None)Memory spec needs when loaded with context tokens per slot in parallel slots (defaults: settings).
format_bytesfunction (n: int | float)Human size in binary units (2.4 GiB), the way ollama ps and container limits read.
kv_type_bytesfunction (kv_type: str)Bytes per KV cache element for an OLLAMA_KV_CACHE_TYPE value.
parse_memoryfunction (text: str | int | float)"28Gi" / "8 GiB" / "16GB" / "512Mi" / a number of bytes -> bytes (Kubernetes and SI units).
quant_bitsfunction (quant: str)Bits per weight of a GGUF quantisation name (q4_k_m, Q8_0, F16, MXFP4); unknown Qn -> n.5.
throughputfunction (spec: ModelSpec, host: Host, *, context_tokens: int=0, kv_type: str | None=None, settings: Settings | None=None)Bandwidth-bound decode, compute-bound prefill and cold-load time of spec on host.

opensmartroute.inference.ollama

Talk to an Ollama server's control API: which models it has, their exact shape, which are resident, load and unload.

NameKindSummary
DEFAULT_URLconstantwhere ollama serve listens by default.
OllamaClientclassControl-plane client for one Ollama server: tags, show, ps, load, unload, exact :class:ModelSpec rows.
ResidentModelclassOne row of /api/ps: a model currently loaded, where it sits and when it expires.
api_rootfunction (base_url: str)http://host:11434/v1 or http://host:11434/ -> http://host:11434 (validated http/https).

opensmartroute.inference.planner

Place models on hosts and write the server settings that make them fit and stay warm.

NameKindSummary
KV_FALLBACKSconstantThe order the planner tries KV precisions when a model does not fit at the requested one.
PlacementclassThe models one host keeps resident, what they use and the server environment that pins them.
PlanclassPlacements per host plus the models that fit nowhere (with the reason).
ollama_envfunction (footprints: Sequence[Footprint], *, keep_alive: str | None=None, settings: Settings | None=None)OLLAMA_* variables that make a server hold exactly these models with these slots.
planfunction (models: Iterable[ModelSpec], hosts: Sequence[Host], *, context: int | None=None, parallel: int | None=None, kv_type: str | None=None, shrink: bool=False, settings: Settings | None=None)First-fit-decreasing packing of models onto hosts at the requested context and parallelism.
recommendfunction (spec: ModelSpec, budget: int, *, context: int | None=None, parallel: int | None=None, kv_type: str | None=None, settings: Settings | None=None)The largest configuration of spec that fits budget bytes, or None when even the weights do not.
renderfunction (p: Plan, *, settings: Settings | None=None)A plain-text report of a plan: per host the resident models, memory used / free and the OLLAMA_* lines.

opensmartroute.jsonrepair

Repair the JSON a model almost produced.

NameKindSummary
completion_tailfunction (streamed: str)For a stream whose content so far is streamed: the suffix that completes it into valid JSON when the.
repair_jsonfunction (text: str)(repaired, changed) - repaired parses with :func:json.loads when any repair was possible; when the.
wants_jsonfunction (response_format: Any)True when an OpenAI response_format asks for JSON (json_object or json_schema).

opensmartroute.knowledge

Knowledge management and context engineering: a knowledge graph the router learns into and reads from.

NameKindSummary
CONTEXT_HEADERre-export of opensmartroute.knowledge.context.CONTEXT_HEADER: Opens the engineered block; the sections follow as ## <title> with one line per fact.
FACT_RULESre-export of opensmartroute.knowledge.ingest.FACT_RULES: (cue words match case-insensitively; proper-noun objects are then trimmed to their title-cased words).
GRAPH_FORMATre-export of opensmartroute.knowledge.graph.GRAPH_FORMATon-disk format of :meth:KnowledgeGraph.save.
PERSONAL_PREDICATESre-export of opensmartroute.knowledge.graph.PERSONAL_PREDICATES: Predicates whose subject is a person; they are placed under About the user and never exported to a shared graph.
PREDICATESre-export of opensmartroute.knowledge.graph.PREDICATES: The predicates the platform and the SDK ingest with, and what each connects (subject -> object).
ContextEngineerre-export of opensmartroute.knowledge.context.ContextEngineerBuilds the engineered context (module docstring). budget_tokens / max_facts / max_web_hits.
ContextLinere-export of opensmartroute.knowledge.context.ContextLineOne admitted or dropped line of context: its section, text, score, token cost and provenance.
ContextMiddlewarere-export of opensmartroute.knowledge.context.ContextMiddlewareRun the :class:ContextEngineer in front of every routed request and teach the graph from the traffic.
EngineeredContextre-export of opensmartroute.knowledge.context.EngineeredContextThe block placed in front of a request and the accounting behind it.
KnowledgeGraphre-export of opensmartroute.knowledge.graph.KnowledgeGraphFacts indexed three ways, upserted on their key, persisted as JSONL (see the module docstring).
KnowledgeStrategyre-export of opensmartroute.knowledge.context.KnowledgeStrategyScores candidates by what the graph knows worked on this kind of task: the performed_on facts of each.
Triplere-export of opensmartroute.knowledge.graph.TripleOne fact with its provenance.
entity_idre-export of opensmartroute.knowledge.graph.entity_idThe graph's spelling of a typed entity: kind:name with the name trimmed and single-spaced.
extract_factsre-export of opensmartroute.knowledge.ingest.extract_facts(predicate, object) pairs a person stated about themselves in text (:data:FACT_RULES plus the name.
ingest_decisionre-export of opensmartroute.knowledge.ingest.ingest_decisionA routing decision as facts: the task type and domains of the request point at the chosen target.
ingest_outcomere-export of opensmartroute.knowledge.ingest.ingest_outcomeAn outcome as evidence: target performed_on task with the running mean quality and the count in the.
ingest_targetsre-export of opensmartroute.knowledge.ingest.ingest_targetsThe catalogue as facts: kind, domains, actions, tags, provider / upstream model, price.
ingest_turnre-export of opensmartroute.knowledge.ingest.ingest_turnWhat a person's message states about them (:func:extract_facts) and names (:func:mentions), as facts of.
ingest_web_contextre-export of opensmartroute.knowledge.ingest.ingest_web_contextThe hits of a :class:~opensmartroute.enterprise.LiveContext (its to_dict()) as topic cites url facts.
mentionsre-export of opensmartroute.knowledge.ingest.mentionsEntities a message names: URLs, known technologies and capitalised phrases that do not start a sentence.
tokensre-export of opensmartroute.knowledge.graph.tokensLower-cased content words of text (stop words dropped) - what :meth:KnowledgeGraph.search matches on.

opensmartroute.knowledge.context

Context engineering: what the model should know for this request, chosen and trimmed to a token budget.

NameKindSummary
CONTEXT_HEADERconstant: Opens the engineered block; the sections follow as ## <title> with one line per fact.
ContextEngineerclassBuilds the engineered context (module docstring). budget_tokens / max_facts / max_web_hits.
ContextLineclassOne admitted or dropped line of context: its section, text, score, token cost and provenance.
ContextMiddlewareclassRun the :class:ContextEngineer in front of every routed request and teach the graph from the traffic.
EngineeredContextclassThe block placed in front of a request and the accounting behind it.
KnowledgeStrategyclassScores candidates by what the graph knows worked on this kind of task: the performed_on facts of each.

opensmartroute.knowledge.graph

The knowledge graph: typed facts with provenance, in memory, in one JSONL file, exportable to the open formats.

NameKindSummary
GRAPH_FORMATconstanton-disk format of :meth:KnowledgeGraph.save.
PERSONAL_PREDICATESconstant: Predicates whose subject is a person; they are placed under About the user and never exported to a shared graph.
PREDICATESconstant: The predicates the platform and the SDK ingest with, and what each connects (subject -> object).
KnowledgeGraphclassFacts indexed three ways, upserted on their key, persisted as JSONL (see the module docstring).
TripleclassOne fact with its provenance.
entity_idfunction (kind: str, name: str)The graph's spelling of a typed entity: kind:name with the name trimmed and single-spaced.
tokensfunction (text: str)Lower-cased content words of text (stop words dropped) - what :meth:KnowledgeGraph.search matches on.

opensmartroute.knowledge.ingest

Turning what the router sees into facts: catalogue, conversations, decisions, outcomes and live web hits.

NameKindSummary
FACT_RULESconstant: (cue words match case-insensitively; proper-noun objects are then trimmed to their title-cased words).
extract_factsfunction (text: str)(predicate, object) pairs a person stated about themselves in text (:data:FACT_RULES plus the name.
ingest_decisionfunction (graph: KnowledgeGraph, request: RouteRequest, decision: RouteDecision, *, subject: str | None=None)A routing decision as facts: the task type and domains of the request point at the chosen target.
ingest_outcomefunction (graph: KnowledgeGraph, outcome: Outcome, *, task_type: str | None=None, domains: Sequence[str]=(), ttl_s: float | None=None)An outcome as evidence: target performed_on task with the running mean quality and the count in the.
ingest_targetsfunction (graph: KnowledgeGraph, targets: Iterable[RouteTarget])The catalogue as facts: kind, domains, actions, tags, provider / upstream model, price.
ingest_turnfunction (graph: KnowledgeGraph, subject: str, text: str, *, conversation_id: str | None=None, confidence: float=0.9)What a person's message states about them (:func:extract_facts) and names (:func:mentions), as facts of.
ingest_web_contextfunction (graph: KnowledgeGraph, web: Mapping[str, Any], *, subject: str | None=None, ttl_s: float | None=None)The hits of a :class:~opensmartroute.enterprise.LiveContext (its to_dict()) as topic cites url facts.
mentionsfunction (text: str, *, limit: int=8)Entities a message names: URLs, known technologies and capitalised phrases that do not start a sentence.

opensmartroute.learning

Auto-learning strategies built on :mod:opensmartroute.math.

NameKindSummary
DEFAULT_DOMAINSconstantDomain one-hot block of the LinUCB context vector (ontology domains + the "general" fallback).
AttentionEncoderre-export of opensmartroute.learning.attention.AttentionEncoderOne self-attention block with attention pooling; encodes text into a dim-vector (unnormalised).
AutoLearnerclassSingle entry point for closing the loop.
Autopilotre-export of opensmartroute.learning.autopilot.AutopilotRuns a :class:SelfImprover on a schedule and on drift, inside a live process.
ContrastiveRouterre-export of opensmartroute.learning.contrastive.ContrastiveRouterDual-encoder router trained with a contrastive or a distillation objective.
ContrastiveStrategyre-export of opensmartroute.learning.contrastive.ContrastiveStrategyScores candidates with a trained :class:ContrastiveRouter.
DriftMonitorre-export of opensmartroute.learning.autopilot.DriftMonitorPage-Hinkley over outcomes: alarms when the served success rate (or quality) drops for real.
EmbeddingFeaturizerre-export of opensmartroute.learning.embed.EmbeddingFeaturizerHashed features plus a frozen dense embedding, indexed above the hashed space (dim + i).
ExampleMinerre-export of opensmartroute.learning.coldstart.ExampleMinerPromote prompts a target handled well into that target's examples.
HandoffPolicyre-export of opensmartroute.learning.handoff.HandoffPolicyPermanent handoff of a task to fallback_target once the failure risk is too high.
HistoryTargetModelre-export of opensmartroute.learning.multiturn.HistoryTargetModelLogistic model over h * e_t with a shared weight vector and per-target biases.
HistoryTargetStrategyre-export of opensmartroute.learning.multiturn.HistoryTargetStrategyScores each target by the learned success probability given the conversation so far.
IRTStrategyclass2PL Item Response Theory: target ability vs (domain, complexity-bucket) item difficulty, learned online.
ImprovementReportre-export of opensmartroute.learning.self_improve.ImprovementReportWhat one cycle did: evidence gathered, catalogue changes, champion vs challenger, and the verdict.
InputDriftMonitorre-export of opensmartroute.learning.slm.InputDriftMonitorLabel-free drift detection on what the SLM sees: Page-Hinkley over its top probability and over the share.
KnowledgeEmbeddingre-export of opensmartroute.learning.kge.KnowledgeEmbeddingTransE entity / relation embeddings over a graph's triples (module docstring).
LabelledCheckre-export of opensmartroute.learning.slm.LabelledCheckAn SLM's score on the hand-labelled routing set (:meth:RouterSLM.labelled_check).
LinUCBStrategyclassContextual bandit (LinUCB) on the signal vector; keeps the last context per request for the update.
MarkovStrategyclassPrefers targets that do well on the predicted next conversation state too.
MixtureCureModelre-export of opensmartroute.learning.handoff.MixtureCureModelWeibull mixture-cure model on cumulative risk with censoring.
NeuralRoutingHeadre-export of opensmartroute.learning.neural.NeuralRoutingHeadDense classifier over hashed prompt features that picks a target - and a learned prompt representation.
PolicyGradientStrategyre-export of opensmartroute.learning.policy_gradient.PolicyGradientStrategyREINFORCE-trained softmax routing policy over hashed request features.
PreferenceStrategyclassBradley-Terry strengths per domain, fed by Outcome.preferred_over pairwise comparisons.
ReactionLearnerre-export of opensmartroute.learning.implicit.ReactionLearnerTurns the next turn of a session into feedback on the previous one.
ReadingWindowre-export of opensmartroute.learning.window.ReadingWindowHow a token budget is spent on a prompt longer than it: head and tail shares of the budget go to the.
RegretReportre-export of opensmartroute.learning.policy_gradient.RegretReportDecision regret versus prediction error on a scored dataset.
ReplayMemoryre-export of opensmartroute.learning.neural.ReplayMemoryBounded experience replay for the routing head: (text, history, target, reward, ts) per observed outcome,.
RequestMemoryre-export of opensmartroute.learning.coldstart.RequestMemoryBounded LRU of request texts keyed by request id - and, when the router tells it, the decision it took.
ResponseAssessmentre-export of opensmartroute.learning.implicit.ResponseAssessmentWhat the answer itself says about its quality: a score, the signals that fired and the outcome source.
RouterSLMre-export of opensmartroute.learning.slm.RouterSLMSmall routing model: dual encoder + target catalogue snapshot + calibration, in one JSON file.
SLMReportre-export of opensmartroute.learning.slm.SLMReportHow an SLM did on a set of rows: accuracy, realised quality, cost, calibration and the training loss.
SLMStrategyre-export of opensmartroute.learning.slm.SLMStrategyEnsemble member backed by a :class:RouterSLM: scores are its probabilities, it keeps learning online.
SelfImproverre-export of opensmartroute.learning.self_improve.SelfImproverClosed loop that keeps a :class:RouterSLM current with the model market and its own traffic.
SimilarityFallbackre-export of opensmartroute.learning.coldstart.SimilarityFallbackFor targets with no observations yet, rank by request<->description similarity.
SkillAffinityre-export of opensmartroute.learning.personal.SkillAffinityProfile-conditioned skill relevance: Beta posterior per (profile bucket, skill).
TaskCreditre-export of opensmartroute.learning.credit.TaskCreditBuffers per-step outcomes of a task and redistributes the final reward (uniform / discounted / last / blend).
TaskPinsre-export of opensmartroute.learning.credit.TaskPinsAdmission-time pinning: task_id -> target_id while the target keeps succeeding.
TokenCountModelre-export of opensmartroute.learning.neural.TokenCountModelLearns billed token counts from text: tokens ~ heuristic * exp(f(stats)) with f a small Keras MLP.
UserAdaptiveStrategyre-export of opensmartroute.learning.personal.UserAdaptiveStrategyPer-user Beta posteriors per target, shrunk toward similar users and the global posterior.
acceptable_setre-export of opensmartroute.learning.contrastive.acceptable_setTargets that are 'fine' for a row: expected + acceptable when labelled, otherwise every.
assess_responsere-export of opensmartroute.learning.implicit.assess_responseEstimate the quality of response to request from the answer alone; None when the response.
configured_encoderre-export of opensmartroute.learning.slm.configured_encoderThe query encoder :class:RouterSLM builds from settings (OSR_SLM_ENCODER / OSR_SLM_EMBEDDER), in.
decision_regretre-export of opensmartroute.learning.policy_gradient.decision_regretMean decision regret of choose against the per-row oracle, next to the prediction error of.
decision_rewardre-export of opensmartroute.learning.policy_gradient.decision_rewardScalar decision reward of an outcome under objective (quality minus normalised cost/latency).
dequantize_rowsre-export of opensmartroute.learning.quant.dequantize_rowsInverse of :func:quantize_rows: key -> list of floats (q * scale per row).
distill_routerre-export of opensmartroute.learning.slm.distill_routerCompress the full router into an SLM: route every text, take the ensemble's ranked utilities as soft.
encoder_namere-export of opensmartroute.learning.slm.encoder_nameThe query encoder a model was trained with: hashed or attention, plus +embed(<name>) when a.
history_vectorre-export of opensmartroute.learning.multiturn.history_vectorRecency-weighted joint embedding of the last turns messages and the current text.
is_followupre-export of opensmartroute.learning.slm.is_followupWhether text only continues a conversation: at most max_words words (SLMSettings.context_max_words;.
keras_embedderre-export of opensmartroute.learning.neural.keras_embedderEmbedder from a saved :class:NeuralRoutingHead (keras:<path> in OSR_SLM_EMBEDDER).
load_embedderre-export of opensmartroute.learning.embed.load_embedderBuild the embedder a model file names (see the module docstring for the schemes). Raises.
merge_learnersfunction (local: Iterable[Strategy], remote: Iterable[Strategy])Federated merge: fold the evidence of remote strategies into the same-named local.
nearest_targetsre-export of opensmartroute.learning.coldstart.nearest_targetsThe k most similar existing targets (cosine + same-kind and domain-overlap bonuses) for warm starts.
negative_onlyre-export of opensmartroute.learning.contrastive.negative_onlyThe one target a row says was bad (no labels, exactly one score, below threshold), else None.
outcome_countsre-export of opensmartroute.learning.coldstart.outcome_countsNumber of recorded outcomes per target id.
paired_testre-export of opensmartroute.learning.self_improve.paired_test(wins, losses, p) of the challenger over the champion on the same rows: the discordant pairs and the.
profile_bucketre-export of opensmartroute.learning.personal.profile_bucketCoarse profile bucket used to pool users with the same declared attributes.
profile_vectorre-export of opensmartroute.learning.personal.profile_vectorHashed one-hot encoding of key=value pairs (lists expand to one pair per element).
quantization_reportre-export of opensmartroute.learning.quant.quantization_reportHow far the quantised block is from rows: max / RMS absolute error, the error relative to the RMS.
quantize_rowsre-export of opensmartroute.learning.quant.quantize_rowsQuantise rows (key -> vector, all of one length) to bits per weight with one scale per row.
routing_textre-export of opensmartroute.learning.slm.routing_textThe text the SLM encodes for a message inside a conversation.
rows_with_contextre-export of opensmartroute.learning.slm.rows_with_contextCopies of rows whose text is :func:routing_text of the row and its history (rows without a.
select_windowre-export of opensmartroute.learning.window.select_windowThe window view of :func:read: tokens cut to at most cap.
signal_vectorfunction (signals: Signals, domains: list[str] | None=None)Fixed-length numeric context for contextual bandits (dim = 8 + len(domains)).
soft_labelsre-export of opensmartroute.learning.contrastive.soft_labelsZooter soft labels: softmax(score / temperature) over the scored targets.
target_documentre-export of opensmartroute.learning.coldstart.target_documentText used to embed a target: name, description, domains, actions, tags and up to 12 examples.
target_embeddingre-export of opensmartroute.learning.coldstart.target_embeddingUnit-norm centroid of the target document and its examples (hashing embedder by default).
tensorflow_availablere-export of opensmartroute.learning.neural.tensorflow_availableTrue when TensorFlow (and Keras 3) import; never raises.
text_statsre-export of opensmartroute.learning.neural.text_statsFixed-length, dependency-free description of text (see :data:TEXT_STATS): logs of the raw sizes and the.
warm_startre-export of opensmartroute.learning.coldstart.warm_startSeed every learner with shrunk knowledge from the new target's nearest neighbours.
warm_start_from_matrixre-export of opensmartroute.learning.coldstart.warm_start_from_matrixOffline full-information reward-matrix warm start (OrcaRouter 2605.30736).

opensmartroute.learning.attention

A pure-Python transformer block for the routing SLM's query encoder.

NameKindSummary
AttentionContextclassEverything :meth:AttentionEncoder.backward needs from one forward pass (lists in pure Python, arrays.
AttentionEncoderclassOne self-attention block with attention pooling; encodes text into a dim-vector (unnormalised).

opensmartroute.learning.autopilot

Self-operation: the routing SLM runs its own improvement loop inside the live process.

NameKindSummary
AutopilotclassRuns a :class:SelfImprover on a schedule and on drift, inside a live process.
DriftMonitorclassPage-Hinkley over outcomes: alarms when the served success rate (or quality) drops for real.

opensmartroute.learning.coldstart

Cold start for new targets and self-improving target descriptions.

NameKindSummary
ExampleMinerclassPromote prompts a target handled well into that target's examples.
RequestMemoryclassBounded LRU of request texts keyed by request id - and, when the router tells it, the decision it took.
SimilarityFallbackclassFor targets with no observations yet, rank by request<->description similarity.
nearest_targetsfunction (new: RouteTarget, pool: Iterable[RouteTarget], k: int=3, embedder: Embedder | None=None)The k most similar existing targets (cosine + same-kind and domain-overlap bonuses) for warm starts.
outcome_countsfunction (feedback: FeedbackStore)Number of recorded outcomes per target id.
target_documentfunction (t: RouteTarget)Text used to embed a target: name, description, domains, actions, tags and up to 12 examples.
target_embeddingfunction (t: RouteTarget, embedder: Embedder | None=None)Unit-norm centroid of the target document and its examples (hashing embedder by default).
warm_startfunction (new: RouteTarget, registry: TargetRegistry, strategies: list[Strategy], k: int=3, shrink: float=0.5, embedder: Embedder | None=None)Seed every learner with shrunk knowledge from the new target's nearest neighbours.

opensmartroute.learning.contrastive

Contrastive and reward-distilled router training (RouterDC, NeurIPS 2024; Zooter 2311.08692).

NameKindSummary
ContrastiveRouterclassDual-encoder router trained with a contrastive or a distillation objective.
ContrastiveStrategyclassScores candidates with a trained :class:ContrastiveRouter.
acceptable_setfunction (row: EvalRow, slack: float=0.05)Targets that are 'fine' for a row: expected + acceptable when labelled, otherwise every.
negative_onlyfunction (row: EvalRow, threshold: float=0.5)The one target a row says was bad (no labels, exactly one score, below threshold), else None.
soft_labelsfunction (scores: dict[str, float], temperature: float=0.1)Zooter soft labels: softmax(score / temperature) over the scored targets.

opensmartroute.learning.credit

Delayed, task-level credit assignment for agentic trajectories.

NameKindSummary
TaskCreditclassBuffers per-step outcomes of a task and redistributes the final reward (uniform / discounted / last / blend).
TaskPinsclassAdmission-time pinning: task_id -> target_id while the target keeps succeeding.

opensmartroute.learning.embed

Pretrained transformer embeddings as frozen features for the routing SLM.

NameKindSummary
EmbeddingFeaturizerclassHashed features plus a frozen dense embedding, indexed above the hashed space (dim + i).
load_embedderfunction (name: str, *, api_key_env: str='OSR_LLM_API_KEY', timeout_s: float=20.0, dims: int=0)Build the embedder a model file names (see the module docstring for the schemes). Raises.
openai_embedderfunction (spec: str, *, api_key_env: str='OSR_LLM_API_KEY', timeout_s: float=20.0, batch: int=64, dims: int=0)Embedder over an OpenAI-compatible /embeddings endpoint. spec is <model>[@<base_url>]; without.

opensmartroute.learning.handoff

Permanent-handoff policy from censored teacher signals (TACIT-Switch 2608.27911).

NameKindSummary
HandoffPolicyclassPermanent handoff of a task to fallback_target once the failure risk is too high.
MixtureCureModelclassWeibull mixture-cure model on cumulative risk with censoring.
TrajectoryclassPer-task state tracked by :class:HandoffPolicy: cumulative risk, steps, failure and hand-off flags.

opensmartroute.learning.implicit

Implicit feedback: outcomes inferred from the answer itself and from what the user does next.

NameKindSummary
NEGATIVE_REACTIONSconstantA complaining next turn: the previous answer was wrong, unhelpful or off-target.
POSITIVE_REACTIONSconstantA satisfied next turn.
ReactionLearnerclassTurns the next turn of a session into feedback on the previous one.
ResponseAssessmentclassWhat the answer itself says about its quality: a score, the signals that fired and the outcome source.
assess_responsefunction (request: RouteRequest, response: Any, target: RouteTarget | None=None, *, signals: Signals | None=None, latency_ms: float | None=None, settings: Settings | None=None)Estimate the quality of response to request from the answer alone; None when the response.

opensmartroute.learning.kge

A learned link predictor for the knowledge graph (optional extra tensorflow to train; numpy to use).

NameKindSummary
KnowledgeEmbeddingclassTransE entity / relation embeddings over a graph's triples (module docstring).

opensmartroute.learning.multiturn

Multi-turn routing with history-target joint embeddings (MTRouter 2604.23530).

NameKindSummary
HistoryTargetModelclassLogistic model over h * e_t with a shared weight vector and per-target biases.
HistoryTargetStrategyclassScores each target by the learned success probability given the conversation so far.
history_vectorfunction (request: RouteRequest, embedder: Embedder, turns: int=6, decay: float=0.7)Recency-weighted joint embedding of the last turns messages and the current text.

opensmartroute.learning.neural

TensorFlow / Keras models that sharpen two things the router does on every request (optional extra tensorflow).

NameKindSummary
NeuralRoutingHeadclassDense classifier over hashed prompt features that picks a target - and a learned prompt representation.
ReplayMemoryclassBounded experience replay for the routing head: (text, history, target, reward, ts) per observed outcome,.
TEXT_STATSconstant: Names of the :func:text_stats features, in order (the model's input layer).
TokenCountModelclassLearns billed token counts from text: tokens ~ heuristic * exp(f(stats)) with f a small Keras MLP.
keras_embedderfunction (path: str | Path)Embedder from a saved :class:NeuralRoutingHead (keras:<path> in OSR_SLM_EMBEDDER).
tensorflow_availablefunction ()True when TensorFlow (and Keras 3) import; never raises.
text_statsfunction (text: str)Fixed-length, dependency-free description of text (see :data:TEXT_STATS): logs of the raw sizes and the.

opensmartroute.learning.personal

Few-shot personalisation (GMTRouter 2511.08590; SkillFeed 2608.28241).

NameKindSummary
SkillAffinityclassProfile-conditioned skill relevance: Beta posterior per (profile bucket, skill).
UserAdaptiveStrategyclassPer-user Beta posteriors per target, shrunk toward similar users and the global posterior.
profile_bucketfunction (profile: dict[str, Any], keys: tuple[str, ...]=('tier', 'expertise', 'role', 'language'))Coarse profile bucket used to pool users with the same declared attributes.
profile_vectorfunction (profile: dict[str, Any], dim: int=_PROFILE_DIM)Hashed one-hot encoding of key=value pairs (lists expand to one pair per element).

opensmartroute.learning.policy_gradient

End-to-end policy-gradient routing (Router-R1 2506.09033; RLCascadeRouter 2608.15817).

NameKindSummary
PolicyGradientStrategyclassREINFORCE-trained softmax routing policy over hashed request features.
RegretReportclassDecision regret versus prediction error on a scored dataset.
decision_regretfunction (rows: Sequence[EvalRow], targets: Sequence[RouteTarget], choose: Callable[[EvalRow, Sequence[RouteTarget]], str], predict: Callable[[EvalRow, RouteTarget], float] | None=None)Mean decision regret of choose against the per-row oracle, next to the prediction error of.
decision_rewardfunction (outcome: Outcome, target: RouteTarget | None, objective: Objective, *, cost_scale: float=0.01, latency_scale: float=2000.0)Scalar decision reward of an outcome under objective (quality minus normalised cost/latency).

opensmartroute.learning.quant

Weight quantisation for the routing SLM's model file.

NameKindSummary
QUANT_BITSconstantsupported weight widths; 0 in settings means "floats, no quantisation".
dequantize_rowsfunction (blob: Mapping[str, Any])Inverse of :func:quantize_rows: key -> list of floats (q * scale per row).
is_quantizedfunction (blob: Any)True for a mapping written by :func:quantize_rows.
quantization_reportfunction (rows: Mapping[str, Sequence[float]], blob: Mapping[str, Any])How far the quantised block is from rows: max / RMS absolute error, the error relative to the RMS.
quantize_rowsfunction (rows: Mapping[str, Sequence[float]], bits: int=8)Quantise rows (key -> vector, all of one length) to bits per weight with one scale per row.

opensmartroute.learning.self_improve

Self-improvement loop: refresh the catalogue, gather evidence, train a challenger, promote it only if better.

NameKindSummary
ImprovementReportclassWhat one cycle did: evidence gathered, catalogue changes, champion vs challenger, and the verdict.
SelfImproverclassClosed loop that keeps a :class:RouterSLM current with the model market and its own traffic.
paired_testfunction (champion: Sequence[bool], challenger: Sequence[bool])(wins, losses, p) of the challenger over the champion on the same rows: the discordant pairs and the.

opensmartroute.learning.slm

The OpenSmartRoute routing SLM: a small, self-contained model that picks the target for a prompt.

NameKindSummary
BUNDLEDconstantmodel path that names the SLM shipped inside the package (RouterSLM.bundled()).
CONTEXT_SEPARATORconstant: follows it under its own ctx: namespace (:data:~opensmartroute.signals.models.CONTEXT_MARK).
FOLLOWUP_REconstant: does not match and routes on its own words - the demonstratives alone ("this", "it") are not a cue.
SLM_FORMATconstanton-disk format version of RouterSLM.state() (2 = quantised weight blocks; 1 = float rows).
SLM_FORMATSconstantformats :meth:RouterSLM.load reads.
InputDriftMonitorclassLabel-free drift detection on what the SLM sees: Page-Hinkley over its top probability and over the share.
LabelledCheckclassAn SLM's score on the hand-labelled routing set (:meth:RouterSLM.labelled_check).
RouterSLMclassSmall routing model: dual encoder + target catalogue snapshot + calibration, in one JSON file.
SLMReportclassHow an SLM did on a set of rows: accuracy, realised quality, cost, calibration and the training loss.
SLMStrategyclassEnsemble member backed by a :class:RouterSLM: scores are its probabilities, it keeps learning online.
configured_encoderfunction (settings: Settings | None=None, champion: RouterSLM | None=None)The query encoder :class:RouterSLM builds from settings (OSR_SLM_ENCODER / OSR_SLM_EMBEDDER), in.
distill_routerfunction (router: Router, texts: Iterable[str], *, targets: Sequence[RouteTarget] | None=None, settings: Settings | None=None, seed: int=0)Compress the full router into an SLM: route every text, take the ensemble's ranked utilities as soft.
encoder_namefunction (slm: RouterSLM)The query encoder a model was trained with: hashed or attention, plus +embed(<name>) when a.
is_followupfunction (text: str, *, max_words: int | None=None, max_chars: int | None=None, settings: Settings | None=None)Whether text only continues a conversation: at most max_words words (SLMSettings.context_max_words;.
routing_textfunction (text: str, history: Sequence[Mapping[str, Any]] | None=None, *, turns: int | None=None, chars: int | None=None, max_text_chars: int | None=None, settings: Settings | None=None)The text the SLM encodes for a message inside a conversation.
rows_with_contextfunction (rows: Iterable[EvalRow], settings: Settings | None=None)Copies of rows whose text is :func:routing_text of the row and its history (rows without a.

opensmartroute.learning.window

The reading window of the routing SLM over a long prompt.

NameKindSummary
ReadingWindowclassHow a token budget is spent on a prompt longer than it: head and tail shares of the budget go to the.
instruction_cuesfunction ()Words a request opens with: the first word of every task template of the ontology (less a few that are not.
readfunction (tokens: Sequence[str], cap: int, window: ReadingWindow | None=None)(window, ask) of tokens under a budget of cap: the window is at most cap tokens - the salient.
segment_saliencefunction (tokens: Sequence[str], cues: frozenset[str] | None=None, *, counts: Mapping[str, int] | None=None, novelty: float=0.0)Instruction salience of one segment: cue words (an opening cue counts double) per square root of the.
select_asksfunction (tokens: Sequence[str], cap: int, window: ReadingWindow | None=None)The ask view of :func:read: the sentences of a prompt over cap that ask for something (empty within it).
select_windowfunction (tokens: Sequence[str], cap: int, window: ReadingWindow | None=None)The window view of :func:read: tokens cut to at most cap.

opensmartroute.math

Mathematical toolkit behind OpenSmartRoute's decisions.

NameKindSummary
EWMAre-export of opensmartroute.math.estimators.EWMAExponentially weighted moving mean and variance with a z-score helper.
Banditre-export of opensmartroute.math.bandits.BanditCommon interface. context is optional; non-contextual learners ignore it.
BradleyTerryre-export of opensmartroute.math.preference.BradleyTerryOnline Bradley-Terry: per-context target strengths from pairwise wins, with L2 and forgetting.
ConformalCalibratorre-export of opensmartroute.math.calibration.ConformalCalibratorSplit conformal prediction over routing candidates.
CostAwareBanditre-export of opensmartroute.math.bandits.CostAwareBanditLagrangian budget wrapper (C2MAB-V flavour).
DelayedFeedbackre-export of opensmartroute.math.bandits.DelayedFeedbackDelayed-feedback wrapper (Joulani, György & Szepesvári, 2013).
DirichletProbere-export of opensmartroute.math.dirichlet.DirichletProbeLinear Dirichlet head over a fixed target list.
Elore-export of opensmartroute.math.preference.EloElo rating — a fixed-step Bradley–Terry with K-factor; handy for fast warm-up.
EnergyModelre-export of opensmartroute.math.energy.EnergyModelPer-target linear energy model wh = e0 + e_in * prompt + e_out * output fit by ridge.
EpsilonGreedyre-export of opensmartroute.math.bandits.EpsilonGreedyExplore uniformly with probability epsilon, otherwise exploit the empirical means.
HardwareProfilere-export of opensmartroute.math.energy.HardwareProfileStatic device characterisation used before measurements exist.
IRTModelre-export of opensmartroute.math.irt.IRTModelOnline 2PL IRT: target ability, item difficulty / discrimination via SGD, with forgetting and merge support.
IsotonicCalibratorre-export of opensmartroute.math.calibration.IsotonicCalibratorMonotone non-decreasing map score -> P(correct), fitted with PAV.
LinUCBre-export of opensmartroute.math.bandits.LinUCBDisjoint LinUCB (Li et al., WWW 2010) with Sherman–Morrison updates and optional.
MarkovChainre-export of opensmartroute.math.markov.MarkovChainDirichlet-smoothed transition counts. decay < 1 multiplies a row's counts by decay.
MultiKnapsackBanditre-export of opensmartroute.math.bandits.MultiKnapsackBanditBandits with several knapsack constraints (Badanidiyuru et al., 2013) with the.
PageHinkleyre-export of opensmartroute.math.estimators.PageHinkleyDetects a decrease in the monitored mean (e.g. quality dropping).
RoutingMDPre-export of opensmartroute.math.markov.RoutingMDPFinite-horizon / discounted MDP over conversation states and route actions.
TemperatureScalerre-export of opensmartroute.math.calibration.TemperatureScalerFits τ\tau for confidence = softmax(u/\tau)[argmax] by 1-D golden-section.
ThompsonBetare-export of opensmartroute.math.bandits.ThompsonBetaBeta–Bernoulli Thompson sampling.
UCB1re-export of opensmartroute.math.bandits.UCB1UCB1: μ^a+c2ln⁡tna\hat\mu_a + c\sqrt{\frac{2\ln t}{n_a}}; untried arms get +inf.
Welfordre-export of opensmartroute.math.estimators.WelfordNumerically stable running mean / variance (Welford's algorithm).
WindowDriftre-export of opensmartroute.math.estimators.WindowDriftADWIN-lite: compare the first and second half of a sliding window with a.
bayesian_averagere-export of opensmartroute.math.estimators.bayesian_averageShrink a small-sample mean toward a prior: (n·m + k·μ₀)/(n + k).
brier_scorere-export of opensmartroute.math.calibration.brier_scoreMean squared error between confidence and the 0/1 correctness label.
conformal_quantilere-export of opensmartroute.math.calibration.conformal_quantileFinite-sample corrected ⌈(n+1)(1−α)⌉/n\lceil (n+1)(1-\alpha)\rceil / n empirical quantile.
digammare-export of opensmartroute.math.dirichlet.digammaDigamma ψ(x)\psi(x) for x > 0 via recurrence to x >= 6 and the asymptotic series.
dominatesre-export of opensmartroute.math.decision.dominatesa dominates b if it is ≥ quality, ≤ cost, ≤ latency and strictly better in one.
entropyre-export of opensmartroute.math.estimators.entropyShannon entropy (nats) of a probability vector.
erlang_cre-export of opensmartroute.math.decision.erlang_cP(an arriving request must wait) for M/M/c. Returns 1.0 if unstable.
expected_calibration_errorre-export of opensmartroute.math.estimators.expected_calibration_errorECE: how well does router confidence predict routing correctness?.
expected_waitre-export of opensmartroute.math.decision.expected_waitMean time in queue (Wq) for M/M/c, in the same time unit as the rates.
ginire-export of opensmartroute.math.estimators.giniGini impurity 1 - sum(p^2); 0 = certain.
hardware_profilere-export of opensmartroute.math.energy.hardware_profileBuilt-in profile by name (a100-80gb, h100-sxm, l4, rtx-4090, cpu-16c, npu-edge).
item_keyre-export of opensmartroute.math.irt.item_keyDefault item id: domain × difficulty bucket, e.g. legal/3.
kingman_waitre-export of opensmartroute.math.decision.kingman_waitKingman's G/G/1 approximation.
littles_lawre-export of opensmartroute.math.decision.littles_lawL = λ·W — average number of in-flight requests.
normalized_entropyre-export of opensmartroute.math.estimators.normalized_entropy0 = certain, 1 = uniform. Useful as an 'ask the LLM judge' trigger.
pareto_frontre-export of opensmartroute.math.decision.pareto_frontKeys of the non-dominated (quality, cost, latency) points.
reliability_diagramre-export of opensmartroute.math.calibration.reliability_diagramPer-bin (mean confidence, empirical accuracy, count) — plot or print it.
servers_for_slare-export of opensmartroute.math.decision.servers_for_slaSmallest c such that E[Wq] ≤ max_wait and P(wait) ≤ max_p_wait.
sigmoidre-export of opensmartroute.math.irt.sigmoidOverflow-safe logistic function.
softmaxre-export of opensmartroute.math.estimators.softmaxNumerically stable softmax; lower temperature sharpens the distribution.
topsisre-export of opensmartroute.math.decision.topsisTOPSIS closeness coefficient in [0,1]; quality is a benefit, cost/latency are costs.
weighted_sumre-export of opensmartroute.math.decision.weighted_sumScalarise w_q * quality - w_c * norm(cost) - w_l * norm(latency) with min-max normalised cost / latency.
wilson_intervalre-export of opensmartroute.math.estimators.wilson_intervalWilson score interval for a binomial proportion (robust at small n).

opensmartroute.math.bandits

Multi-armed and contextual bandits for online routing decisions.

NameKindSummary
BanditclassCommon interface. context is optional; non-contextual learners ignore it.
ThompsonBetaclassBeta–Bernoulli Thompson sampling.
UCB1classUCB1: μ^a+c2ln⁡tna\hat\mu_a + c\sqrt{\frac{2\ln t}{n_a}}; untried arms get +inf.
LinUCBclassDisjoint LinUCB (Li et al., WWW 2010) with Sherman–Morrison updates and optional.
EpsilonGreedyclassExplore uniformly with probability epsilon, otherwise exploit the empirical means.
CostAwareBanditclassLagrangian budget wrapper (C2MAB-V flavour).
MultiKnapsackBanditclassBandits with several knapsack constraints (Badanidiyuru et al., 2013) with the.
DelayedFeedbackclassDelayed-feedback wrapper (Joulani, György & Szepesvári, 2013).

opensmartroute.math.calibration

Calibration and distribution-free risk control for routing confidence.

NameKindSummary
ConformalCalibratorclassSplit conformal prediction over routing candidates.
IsotonicCalibratorclassMonotone non-decreasing map score -> P(correct), fitted with PAV.
TemperatureScalerclassFits τ\tau for confidence = softmax(u/\tau)[argmax] by 1-D golden-section.
brier_scorefunction (confidences: list[float], hits: list[bool])Mean squared error between confidence and the 0/1 correctness label.
conformal_quantilefunction (scores: list[float], alpha: float)Finite-sample corrected ⌈(n+1)(1−α)⌉/n\lceil (n+1)(1-\alpha)\rceil / n empirical quantile.
reliability_diagramfunction (confidences: list[float], hits: list[bool], bins: int=10)Per-bin (mean confidence, empirical accuracy, count) — plot or print it.

opensmartroute.math.decision

Multi-objective decision helpers and queueing theory for real-time routing.

NameKindSummary
dominatesfunction (a: Point, b: Point)a dominates b if it is ≥ quality, ≤ cost, ≤ latency and strictly better in one.
pareto_frontfunction (points: dict[str, Point])Keys of the non-dominated (quality, cost, latency) points.
weighted_sumfunction (points: dict[str, Point], w_q: float, w_c: float, w_l: float)Scalarise w_q * quality - w_c * norm(cost) - w_l * norm(latency) with min-max normalised cost / latency.
topsisfunction (points: dict[str, Point], weights: Sequence[float]=(0.6, 0.25, 0.15))TOPSIS closeness coefficient in [0,1]; quality is a benefit, cost/latency are costs.
erlang_cfunction (arrival_rate: float, service_rate: float, servers: int)P(an arriving request must wait) for M/M/c. Returns 1.0 if unstable.
expected_waitfunction (arrival_rate: float, service_rate: float, servers: int)Mean time in queue (Wq) for M/M/c, in the same time unit as the rates.
servers_for_slafunction (arrival_rate: float, service_rate: float, max_wait: float, max_p_wait: float=0.2)Smallest c such that E[Wq] ≤ max_wait and P(wait) ≤ max_p_wait.
littles_lawfunction (arrival_rate: float, mean_time_in_system: float)L = λ·W — average number of in-flight requests.
kingman_waitfunction (utilization: float, ca2: float, cs2: float, mean_service: float)Kingman's G/G/1 approximation.

opensmartroute.math.dirichlet

Dirichlet probe over host hidden states (ProbeDirichlet, RouterXBench 2602.11877).

NameKindSummary
DirichletProbeclassLinear Dirichlet head over a fixed target list.
digammafunction (x: float)Digamma ψ(x)\psi(x) for x > 0 via recurrence to x >= 6 and the asymptotic series.

opensmartroute.math.energy

Hardware-aware energy characterisation (HW-Router 2608.14575; 2608.28044).

NameKindSummary
EnergyModelclassPer-target linear energy model wh = e0 + e_in * prompt + e_out * output fit by ridge.
HardwareProfileclassStatic device characterisation used before measurements exist.
hardware_profilefunction (name: str)Built-in profile by name (a100-80gb, h100-sxm, l4, rtx-4090, cpu-16c, npu-edge).

opensmartroute.math.estimators

Streaming estimators, drift detection and calibration.

NameKindSummary
EWMAclassExponentially weighted moving mean and variance with a z-score helper.
WelfordclassNumerically stable running mean / variance (Welford's algorithm).
PageHinkleyclassDetects a decrease in the monitored mean (e.g. quality dropping).
WindowDriftclassADWIN-lite: compare the first and second half of a sliding window with a.
wilson_intervalfunction (successes: float, n: int, z: float=1.96)Wilson score interval for a binomial proportion (robust at small n).
bayesian_averagefunction (mean: float, n: int, prior_mean: float, prior_n: float=5.0)Shrink a small-sample mean toward a prior: (n·m + k·μ₀)/(n + k).
softmaxfunction (xs: list[float], temperature: float=1.0)Numerically stable softmax; lower temperature sharpens the distribution.
entropyfunction (ps: list[float])Shannon entropy (nats) of a probability vector.
normalized_entropyfunction (ps: list[float])0 = certain, 1 = uniform. Useful as an 'ask the LLM judge' trigger.
ginifunction (ps: list[float])Gini impurity 1 - sum(p^2); 0 = certain.
expected_calibration_errorfunction (confidences: list[float], hits: list[bool], bins: int=10)ECE: how well does router confidence predict routing correctness?.

opensmartroute.math.irt

Item Response Theory for routing (IRT-Router, ACL 2025).

NameKindSummary
sigmoidfunction (z: float)Overflow-safe logistic function.
IRTModelclassOnline 2PL IRT: target ability, item difficulty / discrimination via SGD, with forgetting and merge support.
item_keyfunction (domain: str, complexity: float, buckets: int=4)Default item id: domain × difficulty bucket, e.g. legal/3.

opensmartroute.math.markov

Markov chains and MDPs for conversational / multi-step routing.

NameKindSummary
MarkovChainclassDirichlet-smoothed transition counts. decay < 1 multiplies a row's counts by decay.
RoutingMDPclassFinite-horizon / discounted MDP over conversation states and route actions.

opensmartroute.math.preference

Bradley–Terry pairwise preference model (RouteLLM, Prompt-to-Leaderboard).

NameKindSummary
BradleyTerryclassOnline Bradley-Terry: per-context target strengths from pairwise wins, with L2 and forgetting.
EloclassElo rating — a fixed-step Bradley–Terry with K-factor; handy for fast warm-up.

opensmartroute.mcp_server

Model Context Protocol server: the router as a set of tools for any IDE or agent.

NameKindSummary
PROTOCOL_VERSIONconstant: Protocol revision this server speaks by default.
SESSION_TTL_Sconstant: How long an Mcp-Session-Id issued by :class:SessionIds stays valid.
SUPPORTED_PROTOCOL_VERSIONSconstant: Revisions accepted from clients (the client's choice is echoed when it is one of these).
MCPServerclassServe a :class:Router (or an enterprise router wrapping one) over MCP.
RemoteMCPclassForward JSON-RPC messages to an HTTP MCP endpoint (POST /mcp) - the bridge's counterpart of.
SessionIdsclassSigned Mcp-Session-Id values for a Streamable HTTP server: <subject>.<expiry>.<nonce>.<mac>, checkable.
ToolErrorclassRaised inside a tool: reported to the client as a tool result with isError (not a protocol error).
bridge_stdiofunction (url: str, api_key: str | None=None)Expose a remote HTTP MCP endpoint as a local stdio server (for clients that only spawn processes).
catalogue_fingerprintfunction (router: Any, tool_names: Iterable[str]=())A short digest of what tools/list / resources/list would show right now - the enabled targets of the.
decision_dictfunction (d: RouteDecision)RouteDecision.to_dict() plus signals and the ranked candidates - what IDE clients want to show.
log_messagefunction (level: str, data: Any, logger: str | None=None, *, minimum: str='info')Send a notifications/message (MCP logging) at level when the transport can push and level is at.
origin_allowedfunction (origin: str | None, allowed: Iterable[str]=())The Streamable HTTP DNS-rebinding guard: a browser Origin must be loopback or one of allowed.
report_progressfunction (progress: float, total: float | None=None, message: str | None=None)Send a notifications/progress for the tool call being served - a no-op when the client gave no.
serve_stdiofunction (server: Any, stdin: IO[bytes] | None=None, stdout: IO[str] | None=None)Run server (anything with handle_json) over newline-delimited JSON on stdin/stdout until EOF.
sse_framefunction (message: Any, event_id: str | None=None)One message event of a server-sent-events reply (id: first when the stream is resumable).

opensmartroute.media

Speech and image adapters for the OpenAI-compatible proxy (osr serve and the hosted platform).

NameKindSummary
AUDIO_EXTENSIONSconstant: File extensions /v1/audio/transcriptions accepts (what speech-to-text providers decode).
AUDIO_TYPESconstant: response_format -> media type of the audio /v1/audio/speech returns.
MAX_AUDIO_BYTESconstant: Largest audio upload accepted by /v1/audio/transcriptions (OpenAI's own limit).
MAX_SPEECH_CHARSconstant: Longest text /v1/audio/speech reads aloud in one call (OpenAI's own limit).
MAX_UPSTREAM_BYTESconstant: Largest body accepted back from a media provider (audio, image JSON, transcript) before the call is failed.
SPEECH_TAGconstant: Capability tag or metadata key that makes a target a /v1/audio/speech target.
TRANSCRIPTION_TAGconstant: Capability tag (capabilities.tags) or metadata key that makes a target a /v1/audio/transcriptions target.
AudioInputclassThe uploaded audio of a transcription request, carried in request.context["audio"] (the bytes stay.
audio_extensionfunction (filename: str)The lower-cased extension of an upload when it is one speech-to-text providers decode, else None.
image_handlerfunction (client: OpenAICompatClient, model: str, **defaults: Any)RouteTarget.handler for text-to-image targets mapped to an OpenAI-compatible provider.
images_responsefunction (decision: RouteDecision, result: Any, request: RouteRequest | None, fallback_from: list[str] | None, response_format: str)The OpenAI images object (created, data[].url or b64_json, revised_prompt) from an.
is_image_targetfunction (target: RouteTarget)A tool whose capabilities say it produces images (actions: [image] or an image modality).
media_onlyfunction (target: RouteTarget)A target that answers a non-chat endpoint only (embeddings, transcription, speech) and must never be.
post_bytesfunction (url: str, data: bytes, headers: dict[str, str], timeout_s: float, *, what: str, model: str)POST to a provider and return (body, content type).
speech_capablefunction (target: RouteTarget)True when a target serves /v1/audio/speech.
speech_handlerfunction (client: OpenAICompatClient, model: str, **defaults: Any)RouteTarget.handler for text-to-speech targets: POST {base_url}/audio/speech -> audio bytes.
speech_resultfunction (decision: RouteDecision, result: Any)(audio bytes, media type) of a speech handler result.
transcription_capablefunction (target: RouteTarget)True when a target serves /v1/audio/transcriptions.
transcription_handlerfunction (client: OpenAICompatClient, model: str, **defaults: Any)RouteTarget.handler for speech-to-text targets: POST {base_url}/audio/transcriptions (multipart).
transcription_responsefunction (decision: RouteDecision, result: Any, request: RouteRequest | None, fallback_from: list[str] | None)The OpenAI transcription object (text, plus language / duration / segments when the.

opensmartroute.migrate

Import another gateway's configuration as a routing catalogue.

NameKindSummary
LITELLM_PROVIDERSconstant: URL and credential variable each one resolves to when the config names none.
MigrationclassWhat the import produced: the catalogue, the providers map, the aliases and the report.
migrate_filefunction (path: str | Path, out_dir: str | Path | None=None)Read a LiteLLM config and, when out_dir is given, write the router's files.
migrate_litellmfunction (doc: Any)Translate a parsed LiteLLM config.yaml into a :class:Migration.
target_idfunction (name: str)A catalogue id from a deployment name: lower case, / and spaces become -.
write_migrationfunction (m: Migration, out_dir: str | Path)Write targets, providers and policy (aliases) files under out_dir; returns the paths written.

opensmartroute.observability

Tracing and observability: every step of routing, execution and learning as a structured event.

NameKindSummary
EVENT_NAMESconstantRouter.route.
METRIC_PREFIXconstantPrometheus metric name prefix (osr_).
SPAN_NAMESconstant: Every span name the SDK opens (an :class:Event with kind="span").
TRACEPARENT_KEYconstantrequest.context key (and HTTP header) carrying an inbound W3C trace context.
TRACE_ID_HEADERconstantresponse header osr serve sets to the trace id of the request.
EventclassOne captured span or event: flat, JSON-friendly, never carries request text.
EventSinkclassReceiver port: override :meth:emit; live bridges may also implement :meth:span_start / :meth:span_end.
FileSinkclassAppend-only JSONL file, one event per line.
LoggingSinkclassOne JSON line per event on the opensmartroute.events logger (level follows the event).
MemorySinkclassThread-safe ring buffer of the newest max_events events; queryable by request, trace, name or level.
MetricsSinkclassCounters and latency percentiles derived from events; :meth:prometheus renders the exposition text.
SpanclassAn open unit of work; a context manager that records duration, status and nested events.
TracerclassOpens spans, records events and fans them out to sinks; sample_rate < 1 traces a share of requests.
configure_tracingfunction (*sinks: EventSink, sample_rate: float | None=None, settings: Settings | None=None)Install sinks on the process-wide tracer. With no sinks, use the ones named by settings.
current_tracerfunction ()The tracer of the innermost open span, else the one bound with :func:use_tracer, else the global one.
get_tracerfunction ()The process-wide tracer (disabled until :func:configure_tracing adds sinks).
render_tracefunction (events: list[dict[str, Any]], *, width: int=100)Render the events of one request (as MemorySink.trace / GET /trace/{id} return them) as a text tree:.
text_digestfunction (text: str)Loggable stand-in for request text: {"text_sha256": <16 hex>, "text_len": n}.
untracedclasswith untraced(): records nothing for the spans and events opened inside - whichever tracer they use.
use_tracerclasswith use_tracer(t): makes t the tracer that :func:current_tracer returns in this context.

opensmartroute.ocm

Open Capability Manifest (OCM) - a vendor-neutral description of any routable capability.

NameKindSummary
OCM_KINDSconstant: Manifest kinds (identical to :class:~opensmartroute.TargetKind values).
OCM_PROTOCOLSconstant: Endpoint protocols a manifest may declare; http and a2a can be bound to executors directly.
OCM_VERSIONconstant: Current manifest version (the ocm field).
bind_endpointfunction (target: RouteTarget, region: str | None=None, **kw: Any)Attach an executor for the selected endpoint (http -> :func:~opensmartroute.adapters.http_handler,.
capability_from_targetfunction (target: RouteTarget)Reverse mapping: publish a target as an OCM v1 manifest (dict; dump as YAML or JSON).
dump_capabilityfunction (doc: dict[str, Any])Serialise a manifest to YAML when PyYAML is available, else to pretty JSON (both are valid OCM).
is_capabilityfunction (doc: Any)True when doc looks like an OCM manifest (has an ocm version field).
load_capabilitiesfunction (paths: Iterable[str | Path] | str | Path, *, strict: bool=True)Load manifests from files and/or directories (**/capability.{yaml,yml,json}).
load_capabilityfunction (path: str | Path, *, strict: bool=True)Load one capability.yaml / .json manifest as a target.
select_endpointfunction (target: RouteTarget, region: str | None=None)The manifest endpoint to use: a region-matching one first, then the first region-less one.
target_from_capabilityfunction (doc: dict[str, Any], *, strict: bool=True)Convert a manifest into a :class:~opensmartroute.RouteTarget (validating first when strict).
validate_capabilityfunction (doc: Any)Validate a manifest; returns a list of human-readable problems (empty = valid).

opensmartroute.policy

Policy layer: hard constraints that are never traded off against utility.

NameKindSummary
DataBoundaryRuleclassThe target's boundary must be at least as strict as the request's (public < private < on_prem).
JailbreakRuleclassRisky prompts may only reach humans or targets tagged as safe.
LanguageRuleclassThe detected language must be declared by the target, unless it declares the wildcard language.
PolicyclassOrdered chain of :data:PolicyRule; returns the first rejection reason or None.
PolicyRuleconstantA hard-constraint check: return a rejection reason, or None to let the target through.
allow_listfunction (target: RouteTarget, request: RouteRequest, signals: Signals)When the request names allow_targets, reject everything else.
allowed_kindsfunction (target: RouteTarget, request: RouteRequest, signals: Signals)Reject kinds outside the request's allowed_kinds.
context_windowfunction (target: RouteTarget, request: RouteRequest, signals: Signals)Reject when the estimated input exceeds the target's context_window.
cost_budgetfunction (target: RouteTarget, request: RouteRequest, signals: Signals)Reject targets whose unit cost exceeds the request's max_cost_per_1k.
default_rulesfunction (settings: PolicySettings | None=None)The built-in chain, in evaluation order (cheap identity checks first).
deny_listfunction (target: RouteTarget, request: RouteRequest, signals: Signals)Reject targets listed in the request's deny_targets.
enabledfunction (target: RouteTarget, request: RouteRequest, signals: Signals)Reject disabled targets.
input_tokensfunction (target: RouteTarget, request: RouteRequest, signals: Signals)Reject when the estimated input exceeds the target's max_tokens_in.
latency_slofunction (target: RouteTarget, request: RouteRequest, signals: Signals)Reject targets whose expected latency for this request's answer length exceeds max_latency_ms.
modalitiesfunction (target: RouteTarget, request: RouteRequest, signals: Signals)Every modality detected in the request must be supported by the target.
piifunction (target: RouteTarget, request: RouteRequest, signals: Signals)Requests flagged as containing PII may only reach targets with pii_allowed.
regionfunction (target: RouteTarget, request: RouteRequest, signals: Signals)The request's region must be one the target serves (targets with no regions serve all).
rule_namefunction (rule: PolicyRule)Display name of a rule: its name attribute, else __name__, else the class name.
tenantfunction (target: RouteTarget, request: RouteRequest, signals: Signals)The request's tenant must be permitted by targets that restrict tenants; a tenant permission covers its.
toolsfunction (target: RouteTarget, request: RouteRequest, signals: Signals)When tools are required (constraint or context['tools']), LLM targets must supports_tools.

opensmartroute.realtime

Real-time operational controls: health, circuit breaking, rate & budget limits.

NameKindSummary
LIMIT_KEYSconstant: metadata keys of a :class:RouteTarget that :meth:HealthRegistry.apply_limits reads.
BUDGET_WINDOWSconstant: Budget window names accepted under metadata.budget.per -> seconds.
BreakerStateclassCircuit-breaker states: closed (healthy), open (tripped), half_open (probing recovery).
CircuitBreakerclassClassic three-state breaker with a sliding failure window.
TokenBucketclassRate limiter: rate tokens/sec, burst up to capacity. Thread-safe.
BudgetclassRolling spend cap (e.g. USD per hour) per target or tenant.
LatencyWindowclassRolling window of the last size latencies with nearest-rank percentiles (p50 / p90 / p99).
TargetHealthclassLive health of one target: breaker, latency EWMA + percentiles, success counts, optional rate limit / budget.
HealthRegistryclassTracks health per target id. Shared by policy + strategy + executor.
HealthPolicyclassPolicy that additionally rejects targets whose breaker is open, whose rate.
HealthStrategyclassSoft signal: observed reliability x latency-SLO fit from live health data.

opensmartroute.responses

OpenAI Responses API (POST /v1/responses) on top of the chat-completions proxy.

NameKindSummary
BUILTIN_TOOL_TYPESconstant: Responses tools[].type values the chat path cannot execute (refused, never dropped).
INPUT_ROLESconstant: Responses input item roles that map onto chat messages.
ResponseStoreclassConversation state behind previous_response_id: the chat messages of each remembered response (its.
ResponseStreamTranslatorclassPush-based translation of chat.completion.chunk SSE bytes into Responses API streaming events.
chat_to_responsefunction (completion: dict[str, Any], request: dict[str, Any] | None=None, *, store: ResponseStore | None=None, scope: str | None=None, messages: list[dict[str, Any]] | None=None)Shape a finished chat.completion as a Responses API response object. With store and the chat.
response_eventsfunction (events: Iterable[bytes], request: dict[str, Any] | None=None)Translate a chat.completion.chunk SSE byte stream into Responses API streaming events.
responses_to_chatfunction (body: dict[str, Any], *, default_model: str='auto', store: ResponseStore | None=None, scope: str | None=None)Translate a Responses API request body into a chat.completion request body.

opensmartroute.retrieval

Retrieval-based candidate narrowing for very large target pools (ToolRet / Skill-RAG).

NameKindSummary
BM25IndexclassIn-memory BM25 (Okapi) index over target documents.
DenseIndexclassCosine index. Default = sparse signed-hashing features with an inverted index (pure.
NarrowingStatsclassCounters kept by the retrieval middleware: calls, how often it narrowed, last pool and kept sizes.
RetrievalResultclassNarrowed candidate ids with fused scores and how many came from the lexical / dense legs.
RetrieverclassHybrid lexical + dense narrowing that tracks a registry (or an explicit pool).
execute_tool_targetfunction (registry: TargetRegistry, allow: Callable[[RouteTarget, dict[str, Any]], bool] | None=None, target_id: str='execute_tool')A tool that runs a tool by id (with an optional allow-policy hook).
narrow_signals_hintfunction (signals: Signals)Extra lexical hints from signals (domains / actions) appended to the retrieval query.
rrffunction (rankings: list[list[tuple[str, float]]], k: int=60, weights: list[float] | None=None)Reciprocal Rank Fusion over several ranked lists of (id, score).
select_skill_setfunction (request: RouteRequest | str, pool: list[RouteTarget], k: int=3, embedder: Embedder | None=None, redundancy: float=0.7, min_gain: float=0.02, relevance: dict[str, float] | None=None)Greedy facility-location selection of complementary targets.
target_textfunction (t: RouteTarget)Lexical document for a target: id, name, description, capabilities and up to 8 examples.
tokenizefunction (text: str)Lower-case alphanumeric tokens.
tool_search_targetfunction (retriever: Retriever, registry: TargetRegistry, kinds: tuple[str, ...]=('tool', 'skill'), k: int=8, target_id: str='tool_search')A tool that finds tools: returns compact descriptors for the k best matches.

opensmartroute.router

The Router: signals -> policy -> strategies -> ensemble -> decision (+plan).

NameKindSummary
RouterclassSignals -> policy -> strategies -> ensemble -> decision (+ optional plan).
NoRouteErrorre-export of opensmartroute.errors.NoRouteErrorNo target satisfied the hard constraints.
DEFAULT_WEIGHTSconstant: Library-default ensemble weights (Settings().weights); kept for backwards compatibility.

opensmartroute.sdk

OpenSmartRoute SDK: decorator-driven registration of routing components.

NameKindSummary
RULE_CONDITIONSconstant: Every key a rule's when mapping understands (:class:~opensmartroute.strategies.rules.Rule).
ComponentKindclassThe component families a :class:ComponentRegistry can hold.
ComponentRegistryclassBlueprints for every routing component, with decorators that register into it.
FunctionGuardclassAdapts fn(request) -> None | str | False | RouteRequest into a middleware that runs before routing.
FunctionMiddlewareclassAdapts fn(request, next_) -> RouteDecision.
FunctionRuleclassA :class:~opensmartroute.strategies.rules.Rule whose when conditions may be completed by a.
FunctionSignalclassAdapts fn(request, signals) -> None | {field: value} | value.
FunctionStrategyclassAdapts fn(request, signals, candidates) -> {target_id: score | StrategyScore}.
FunctionTelemetryclassA telemetry sink made of plain callables: any of on_decision(request, decision),.
RegistrationclassA registered blueprint. factory() returns a fresh component instance.
agentconstant@agent(id, ...): shorthand for an agent target.
arouteconstantawait aroute(text): the async form of route().
arunconstantawait arun(text): the async form of run(); coroutine handlers are awaited.
componentsconstant: The process-wide registry that the top-level decorators (opensmartroute.strategy ...) bind to.
feedbackconstantfeedback(decision, success=, quality=): learn from a decision in one call.
guardconstant@guard: fn(request) -> reason | RouteRequest | None runs before routing.
handlerconstant@handler(*target_ids): bind code to targets declared in a catalogue.
humanconstant@human(id, ...): shorthand for a human target.
llmconstant@llm(id, ...): shorthand for an llm target.
middlewareconstant@middleware: register a Middleware class or fn(request, next_route).
on_decisionconstant@on_decision: fn(request, decision) after every route.
on_errorconstant@on_error: fn(request, error) when a route fails.
on_outcomeconstant@on_outcome: fn(outcome) for every outcome learned.
personaconstant@persona(id, ...): shorthand for a slot-only persona target.
policy_ruleconstant@policy_rule: register fn(target, request, signals) -> reason | None.
routeconstantroute(text): one request through the process-wide registry's router.
ruleconstantrule(name, prefer=, avoid=, pin=, **when) or @rule(...) over a predicate.
runconstantrun(text): route + execute + learn through the process-wide registry.
signalconstant@signal: register a SignalExtractor class or fn(request, signals) -> mapping | value.
skillconstant@skill(id, ...): shorthand for a skill target.
strategyconstant@strategy(weight=, name=): register a Strategy class or fn(request, signals, candidates).
targetconstant@target(id, kind, ...): the decorated callable becomes a RouteTarget handler.
telemetryconstant@telemetry: register a Telemetry sink class or factory.
toolconstant@tool(id, ...): shorthand for a tool target.

opensmartroute.security

Security controls for the routing control plane.

NameKindSummary
GadgetDetectorre-export of opensmartroute.security.gadget.GadgetDetectorLearned tail classifier. score(text) in [0, 1]; split(text) finds the gadget tail.
GuardMiddlewareclassApplies :class:InputGuard (+ optional redaction) before routing.
GuardReportclassOutcome of :meth:InputGuard.inspect: pass / fail, reasons, gadget suspicion score and the cleaned text.
InputGuardclassValidates and normalises request text. Cheap: O(len(text)).
ModerationResultre-export of opensmartroute.security.moderation.ModerationResultOne input's verdict: flagged plus per-category booleans, scores and the guard's reasons.
OriginPolicyre-export of opensmartroute.security.provenance.OriginPolicyDecides which tool arguments must be user-originated and verifies them.
OriginRulere-export of opensmartroute.security.provenance.OriginRulePer-target override. sensitive=None = infer from parameter names.
OriginViolationre-export of opensmartroute.security.provenance.OriginViolationRaised by :class:OriginPolicy when a sensitive argument of a state-changing tool comes from untrusted text.
OutputGuardre-export of opensmartroute.security.output.OutputGuardScreen a handler's answer for secrets, PII and instruction injection (see the module docstring).
OutputReportre-export of opensmartroute.security.output.OutputReportFindings of :meth:OutputGuard.inspect: the (possibly rewritten) text plus what was found.
RedactorclassReplace PII with typed placeholders, e.g. <EMAIL_1>. Reversible per-request.
ResourceLimitExceededre-export of opensmartroute.security.limits.ResourceLimitExceededA task went over one of its :class:ResourceLimits (steps, tool calls, depth, tokens, cost, wall time).
ResourceLimitMiddlewarere-export of opensmartroute.security.limits.ResourceLimitMiddlewareCharge one step (+ the request's token estimate) per route() call.
ResourceLimiterre-export of opensmartroute.security.limits.ResourceLimiterThread-safe per-task accounting against :class:ResourceLimits.
ResourceLimitsre-export of opensmartroute.security.limits.ResourceLimitsPer-task ceilings that stop runaway agents; enforced by :class:ResourceLimiter.
SafetyCasere-export of opensmartroute.security.safety.SafetyCaseOne red-team scenario: an adversarial request, its clean baseline and the routing invariants to check.
apply_limitsre-export of opensmartroute.security.limits.apply_limitsWrap handlers of the given kinds. Returns the number wrapped.
apply_origin_policyre-export of opensmartroute.security.provenance.apply_origin_policyWrap every state-changing target in registry. Returns the number wrapped.
description_riskre-export of opensmartroute.security.injection.description_riskRisk of a tool/agent description: max of injection lexicon and learned gadget score.
expand_entitiesfunction (names: list[str] | tuple[str, ...])Entity names with the groups of :data:~opensmartroute.signals.PII_GROUPS expanded; unknown names raise.
guard_handlerre-export of opensmartroute.security.provenance.guard_handlerReturn a copy of target whose handler enforces policy before invoking the tool.
injection_riskre-export of opensmartroute.security.injection.injection_riskShortcut for inspect_injection(text).score.
inspect_injectionre-export of opensmartroute.security.injection.inspect_injectionScore instruction-injection likelihood with the matched lexicon entries for the trace.
limit_handlerre-export of opensmartroute.security.limits.limit_handlerCopy of target whose handler charges one tool call (and nesting depth) per invocation.
load_secretfunction (name: str, *, file_env_suffix: str='_FILE', default: str | None=None)12-factor secret loading: NAME env var, else the file at NAME_FILE (k8s/Docker secrets).
mark_untrustedre-export of opensmartroute.security.provenance.mark_untrustedRecord content that entered the request from a non-user source.
moderatere-export of opensmartroute.security.moderation.moderateThe OpenAI moderation object for one string or a list of strings / {"type": "text", "text": ...}.
moderate_textre-export of opensmartroute.security.moderation.moderate_textClassify one text with the input guard, the injection lexicon and the PII redactor.
run_safety_suitere-export of opensmartroute.security.safety.run_safety_suiteRun every case through route and return pass/fail details grouped by category.
sanitize_for_promptfunction (text: str, max_len: int=4000)Make user text safe to embed in an LLM-judge prompt.
shannon_entropyfunction (s: str)Character-level Shannon entropy in bits (high values flag encoded / random payloads).
strip_steeringre-export of opensmartroute.security.injection.strip_steeringRemove sentences that instruct the router (self-declared complexity, "use the best.
synthesize_gadget_corpusre-export of opensmartroute.security.gadget.synthesize_gadget_corpusLabelled (text, label) rows: clean prompts + prompts with an appended gadget.

opensmartroute.security.gadget

Learned confounder-gadget detector (Rerouting LLM Routers, Shafran et al. 2025).

NameKindSummary
CLEANconstantclass labels used by the detector and the synthetic corpus.
GADGETconstantclass labels used by the detector and the synthetic corpus.
GadgetDetectorclassLearned tail classifier. score(text) in [0, 1]; split(text) finds the gadget tail.
synthesize_gadget_corpusfunction (per_template: int=2, n_gadgets: int=400, seed: int=0)Labelled (text, label) rows: clean prompts + prompts with an appended gadget.
synthesize_gadgetsfunction (n: int, seed: int=0)Adversarial suffixes drawn from the gadget families described in the paper.
tail_featuresfunction (tokens: Sequence[str])Hand-crafted statistics of a token window (all in [0, 1]) – complements hashed n-grams.

opensmartroute.security.injection

Instruction-injection detection for text that is not the user's request.

NameKindSummary
InjectionReportclassInstruction-injection score in [0, 1] plus the lexicon hits that produced it.
description_riskfunction (text: str, gadget_score: float | None=None)Risk of a tool/agent description: max of injection lexicon and learned gadget score.
injection_riskfunction (text: str)Shortcut for inspect_injection(text).score.
inspect_injectionfunction (text: str)Score instruction-injection likelihood with the matched lexicon entries for the trace.

opensmartroute.security.limits

Resource-amplification limits ("Beyond Max Tokens").

NameKindSummary
ResourceLimitExceededclassA task went over one of its :class:ResourceLimits (steps, tool calls, depth, tokens, cost, wall time).
ResourceLimitMiddlewareclassCharge one step (+ the request's token estimate) per route() call.
ResourceLimiterclassThread-safe per-task accounting against :class:ResourceLimits.
ResourceLimitsclassPer-task ceilings that stop runaway agents; enforced by :class:ResourceLimiter.
TaskUsageclassRunning consumption of one task (steps, tool calls, depth, tokens, cost, timestamps).
apply_limitsfunction (registry: TargetRegistry, limiter: ResourceLimiter, kinds: tuple[str, ...]=('tool', 'agent', 'workflow'))Wrap handlers of the given kinds. Returns the number wrapped.
limit_handlerfunction (target: RouteTarget, limiter: ResourceLimiter)Copy of target whose handler charges one tool call (and nesting depth) per invocation.
task_id_offunction (request: RouteRequest, key: str='task_id')context[key] when present, otherwise the request id (each request is its own task).

opensmartroute.security.moderation

POST /v1/moderations over the router's own guards.

NameKindSummary
MODERATION_CATEGORIESconstant: Category names of a moderation result (each has a boolean and a score in [0, 1]).
MODERATION_MODELconstant: model reported by :func:moderate.
ModerationResultclassOne input's verdict: flagged plus per-category booleans, scores and the guard's reasons.
moderatefunction (inputs: str | Iterable[Any], *, guard: Any | None=None, redactor: Any | None=None, flag_pii: bool=False, model: str=MODERATION_MODEL)The OpenAI moderation object for one string or a list of strings / {"type": "text", "text": ...}.
moderate_textfunction (text: str, *, guard: Any | None=None, redactor: Any | None=None, flag_pii: bool=False)Classify one text with the input guard, the injection lexicon and the PII redactor.

opensmartroute.security.output

Response-side guard: what a target answered is screened before it leaves the router.

NameKindSummary
OUTPUT_GUARD_MODESconstant: Guard modes, weakest first.
SECRET_PATTERNSconstant: (name, pattern) of credential shapes an answer must not carry verbatim.
STREAM_HOLD_CHARSconstant: whole by :data:_OPEN_BLOCKS until its end arrives).
STREAM_MAX_PENDING_CHARSconstant: first-output delay stay bounded - a finding longer than this may reach the consumer in part.
STREAM_MAX_RETAINED_CHARSconstant: it the oldest text is dropped and :attr:StreamScreen.truncated is set.
OutputGuardclassScreen a handler's answer for secrets, PII and instruction injection (see the module docstring).
OutputReportclassFindings of :meth:OutputGuard.inspect: the (possibly rewritten) text plus what was found.
StreamScreenclassPre-emission filter for one streamed answer: call it with every delta in place of the consumer's.

opensmartroute.security.provenance

Origin (provenance) policy for tool parameters – ROPE-style control-flow integrity.

NameKindSummary
OriginFindingclassWhere one tool argument's value came from (user, context, untrusted content or unknown).
OriginPolicyclassDecides which tool arguments must be user-originated and verifies them.
OriginReportclassProvenance check of a tool call: all findings plus the sensitive parameters with a disallowed origin.
OriginRuleclassPer-target override. sensitive=None = infer from parameter names.
OriginViolationclassRaised by :class:OriginPolicy when a sensitive argument of a state-changing tool comes from untrusted text.
SENSITIVE_PARAM_HINTSconstantParameter-name fragments treated as sensitive when a target has no explicit OriginRule.
STATE_CHANGING_ACTIONSconstantVerbs in a tool's id / description / actions that mark it as state-changing (side effects).
apply_origin_policyfunction (registry: TargetRegistry, policy: OriginPolicy)Wrap every state-changing target in registry. Returns the number wrapped.
guard_handlerfunction (target: RouteTarget, policy: OriginPolicy, arguments_key: str='tool_arguments')Return a copy of target whose handler enforces policy before invoking the tool.
mark_untrustedfunction (request: RouteRequest, text: str, source: str='retrieved')Record content that entered the request from a non-user source.
untrusted_textsfunction (request: RouteRequest)(source, text) pairs recorded by :func:mark_untrusted on this request.
user_textsfunction (request: RouteRequest)Everything the user actually typed: current text, prior user turns and the pre-guard original text.

opensmartroute.security.safety

Safety-routing regression suite ("When Safety Routing Breaks").

NameKindSummary
SafetyCaseclassOne red-team scenario: an adversarial request, its clean baseline and the routing invariants to check.
SafetyResultclassVerdict for one :class:SafetyCase: chosen vs baseline target and the invariants that failed.
default_casesfunction (seed: int=0)Deployment-agnostic red-team cases; extend with catalogue-specific ones.
run_safety_suitefunction (route: RouteCall, cases: Sequence[SafetyCase] | None=None, *, seed: int=0)Run every case through route and return pass/fail details grouped by category.

opensmartroute.server

Optional FastAPI server exposing the router over HTTP.

NameKindSummary
APP_HEADERconstant: Request header naming the calling application (OpenRouter-style attribution); echoed in metadata.
ATTACHMENT_TYPESconstant: Content-part types of a multi-part user message that are attachments (kept in context["attachments"]).
AUTO_SLUGconstant: Slug prefix of the router's own pseudo-models on the OpenAI-compatible proxy (osr/auto).
AUTO_VARIANTSconstant: osr/auto:<variant> presets: objective weights, hard constraints and allowed kinds.
DEFAULT_PROXY_ALIASESconstant: Model names that mean "let the router choose" on the OpenAI-compatible proxy.
EMBEDDINGS_TAGconstant: Capability tag / metadata key marking a target whose handler answers /v1/embeddings.
EXECUTE_PATHSconstant: ServerSettings.readonly_executes says otherwise. /mcp counts only for tools/call.
OPEN_PATHSconstant: Paths that never require a token when access control is on (probes, metrics, token check, OpenAPI pages).
REQUEST_ID_HEADERconstant: Response headers carrying the decision (target id, request id) beside the OpenAI-shaped body.
TARGET_HEADERconstant: Response headers carrying the decision (target id, request id) beside the OpenAI-shaped body.
UNTRACED_PATHSconstant: Paths that never open an http.request span (probes and the observability endpoints themselves).
WRITE_PATHSconstant: Paths a read-only token (<token>:ro) may not call: they change learner state or trigger work.
AccessTokenclassOne configured access token: the secret, whether it is read-only and an optional expiry (unix seconds).
bearer_tokenfunction (headers: Any)The access token of a request: X-API-Key first, else Authorization: Bearer <token>.
create_appfunction (router: Any, proxy_aliases: frozenset[str]=DEFAULT_PROXY_ALIASES, autopilot: Any=None, auth_tokens: Iterable[str]=(), marketplace_sync: Any=None)FastAPI app: /route, /feedback, /targets, /stats, /healthz and the OpenAI-compatible /v1.
embeddings_capablefunction (target: RouteTarget)True when a target serves /v1/embeddings: tagged :data:EMBEDDINGS_TAG or metadata["embeddings"] set.
embeddings_responsefunction (decision: RouteDecision, result: Any, request: RouteRequest | None=None, fallback_from: list[str] | None=None)Shape an embeddings handler result ({"embeddings": [[...]], "model": ..., "prompt_tokens": n}) as an.
find_tokenfunction (token: str | None, tokens: Iterable[str])The configured :class:AccessToken the presented token matches (expired entries included), else None.
proxy_requestfunction (body: dict[str, Any], aliases: frozenset[str]=frozenset({'auto'}), app: str | None=None)Turn an OpenAI chat-completion body into a :class:RouteRequest.
proxy_responsefunction (decision: RouteDecision, result: Any, request: RouteRequest | None=None, fallback_from: list[str] | None=None)Shape an execution result as an OpenAI chat.completion object (model = chosen target id).
rerank_responsefunction (embeddings: dict[str, Any], documents: list[str], *, top_n: int | None=None, return_documents: bool=False)Shape an embeddings list object whose first vector is the query and the rest the documents as a.
resolve_modelfunction (model: str, aliases: frozenset[str]=DEFAULT_PROXY_ALIASES)Split a proxy model into (pinned target id | None, route options).
token_acceptedfunction (token: str | None, tokens: Iterable[str])Constant-time membership test of token in the configured access tokens (scoped entries match too).

opensmartroute.settings

Typed, environment-overridable settings: the only home for OpenSmartRoute's tunable constants.

NameKindSummary
BanditSettingsclassThompson-sampling strategy (:class:~opensmartroute.BanditStrategy).
CapabilitySettingsclassCapability-fit strategy (:class:~opensmartroute.CapabilityStrategy).
ClientSettingsclassThe platform / osr serve client (:class:opensmartroute.client.OpenSmartRoute): timeouts and retries.
CompressionSettingsclassPrompt compaction before an LLM / agent handler runs (:mod:opensmartroute.compress).
EstimateSettingsclassToken counting behind :func:opensmartroute.estimate.estimate_tokens (signals, admission bound, quotes).
ImplicitSettingsclassImplicit feedback (:mod:opensmartroute.learning.implicit): outcomes inferred from the response itself and.
InferenceSettingsclassSizing self-hosted model servers (:mod:opensmartroute.inference): what a model needs, what a host gives.
KnowledgeSettingsclassKnowledge graph and context engineering (:mod:opensmartroute.knowledge).
LearningSettingsclassClosing the loop (:class:~opensmartroute.learning.AutoLearner) against a shared state store.
MarketplaceSettingsclassFollowing the marketplace (:class:~opensmartroute.adapters.marketplace.MarketplaceSync): osr serve keeps.
ObservabilitySettingsclassTracing and event capture (:mod:opensmartroute.observability); read by Tracer.from_settings.
PolicySettingsclassHard-constraint thresholds (read by :class:~opensmartroute.Policy).
RoutingSettingsclassEnsemble, confidence and plan composition (read by :class:~opensmartroute.Router).
RulesSettingsclassDeclarative rules strategy (:class:~opensmartroute.RulesStrategy).
SLMSettingsclassRouting SLM, dataset collection and the self-improvement loop (:mod:opensmartroute.learning.slm).
ServerSettingsclassosr serve (:mod:opensmartroute.server): access control of the self-hosted HTTP API.
SettingsclassAll tunables, grouped by consumer. Immutable; derive variants with :meth:replace.
WeightSettingsclassEnsemble weight per strategy name (looked up as weights[strategy.name]).
configurefunction (settings: Settings | None=None, **groups: Any)Install process-wide settings. configure() with no arguments re-reads the environment;.
get_settingsfunction ()The process-wide :class:Settings (environment overlay applied once, lazily).
resolvefunction (settings: Settings | None)settings if given, else the process-wide settings (component constructors use this).

opensmartroute.signals

Signal extraction: cheap, deterministic features computed from the request.

NameKindSummary
ACTION_LEXICONconstantAction label -> trigger phrases; the keys are the ontology action names (summarize, translate, ... escalate).
DEFAULT_EXTRACTORSconstanta caller that knows the task type corrects the lexicon before anything reads the actions.
DEFAULT_ONTOLOGYre-export of opensmartroute.signals.ontology.DEFAULT_ONTOLOGYthe built-in TASK_TYPES ontology used by TaskTypeSignal.
DOMAIN_LEXICONconstantDomain label -> trigger phrases; the keys are the ontology domain names targets and skills may declare.
EVENT_LEXICONre-export of opensmartroute.signals.events.EVENT_LEXICONEvent name -> (domains, actions). Keys are matched exactly, then by subject.* prefix.
EVENT_SUBJECTSre-export of opensmartroute.signals.events.EVENT_SUBJECTSGeneric fallbacks when the exact event / prefix is unknown: subject -> domain, verb -> action.
EVENT_VERBSre-export of opensmartroute.signals.events.EVENT_VERBSevent verb -> action when the exact event / prefix is unknown.
FRESHNESS_CUESconstant: Phrases that make an answer depend on the state of the world right now (weighted; see :func:freshness_need).
STALE_CUESconstant: Phrases that pin a request to the past or to timeless knowledge; they pull the freshness need down.
TASK_PRIORSre-export of opensmartroute.signals.models.TASK_PRIORSname: (difficulty, reasoning_need, expected_output_tokens).
TASK_TYPESre-export of opensmartroute.signals.ontology.TASK_TYPES---------------------------------------------------------------- transformation.
ComplexitySignalclassHybrid-LLM-style difficulty estimate in [0, 1].
DomainActionSignalclassArch-Router-style domain / action detection via lexicon matching.
DraftResponseSignalre-export of opensmartroute.signals.uncertainty.DraftResponseSignalQuery-response mixed representation: run a cheap drafter and expose draft features.
EventInfore-export of opensmartroute.signals.events.EventInfoParsed event: raw name, subject, verb and the domains / actions it implies.
EventSignalre-export of opensmartroute.signals.events.EventSignalDomains / actions from context["event"] and context["intent"] (event-driven requests).
EventTriggerre-export of opensmartroute.signals.uncertainty.EventTriggerEvent-triggered invocation: evaluate uncertainty features against rules and return the.
FreshnessSignalclasssignals.freshness_need - whether the request is about the world now (:func:freshness_need). A high.
HashedClassifierre-export of opensmartroute.signals.models.HashedClassifierMultinomial logistic regression over hashed features (sparse weights per class).
HashedFeaturizerre-export of opensmartroute.signals.models.HashedFeaturizerHashing-trick sparse features: words, bigrams, character n-grams and a few numeric text statistics. A text.
HashedRegressorre-export of opensmartroute.signals.models.HashedRegressorSquared-loss linear regressor on hashed features; output clipped to [lo, hi].
HistorySignalclassConversation-state features from request.history (RCRouter-style):.
LanguageSignalclasslanguage from script / stop-word hints (LANGUAGE_HINTS); defaults to en.
LearnedDifficultySignalclassBlend the heuristic complexity with a trained regressor (Hybrid-LLM style difficulty).
LengthSignalclasstoken_estimate from text + history length (about 4 characters per token).
ModalitySignalclassmodalities from context keys (images / image_url, audio, files) on top of text.
OutputLengthSignalclassExpected output tokens (drives cost estimates and effort/max_tokens choices).
ProfileSignalclassUser-profile features (request.profile): tier, expertise, preferences.
ReasoningNeedSignalclassHow much a request benefits from extended thinking (ThinkSwitcher / Sketch-of-Thought).
SensitivitySignalclassPII and prompt-injection / jailbreak heuristics (vLLM-semantic-router-style).
SignalExtractorclassFills in part of a :class:Signals object.
SignalModelBundlere-export of opensmartroute.signals.models.SignalModelBundleAll learned signal models together, persisted as one JSON document.
TaskHintSignalclasscontext["task_type"] from the caller (a form that knows it collects a translation, the synthetic corpus.
TaskOntologyre-export of opensmartroute.signals.ontology.TaskOntologyLookup helpers over :data:TASK_TYPES.
TaskTypere-export of opensmartroute.signals.ontology.TaskTypeOne leaf of the task ontology: family, description, the actions it maps to and template phrasings.
TaskTypeSignalclassOntology task type. Uses the learned classifier when one is loaded, else maps the.
TriggerRulere-export of opensmartroute.signals.uncertainty.TriggerRulefeature >= threshold (or <= when below=True) fires action.
UncertaintyGatere-export of opensmartroute.signals.uncertainty.UncertaintyGateA cascade quality gate built from response uncertainty.
VerbalisedDifficultySignalre-export of opensmartroute.signals.uncertainty.VerbalisedDifficultySignalBlend a verbalised difficulty into complexity and reasoning_need.
WorkflowSignalre-export of opensmartroute.signals.events.WorkflowSignalDetect a workflow trigger (context["workflow"] or "run the workflow|process|...").
apply_task_hintfunction (signals: Signals, task_type: str, ontology: TaskOntology=DEFAULT_ONTOLOGY)Trust a known task type over the lexicon: when task_type names an ontology type, its declared actions.
extract_signalsfunction (request: RouteRequest, extractors: list[SignalExtractor] | None=None)Run the extractor chain (DEFAULT_EXTRACTORS unless given) and return the populated :class:Signals.
freshness_needfunction (text: str, *, now: float | None=None)How much an answer to text depends on facts that change - 0 for timeless or historical asks, towards 1.
learned_extractorsfunction (bundle: SignalModelBundle)Extractor pipeline with the trained :class:SignalModelBundle plugged in.
load_training_rowsre-export of opensmartroute.signals.models.load_training_rowsRead JSONL training rows. Accepts eval-dataset rows too (prompt/text,.
parse_difficultyre-export of opensmartroute.signals.uncertainty.parse_difficultyMap a verbalised difficulty (number, "7/10", "hard") to [0, 1]; None if unparseable.
parse_eventre-export of opensmartroute.signals.events.parse_eventMap an event name (subject.verb, subject:verb or subject_verb) onto domains / actions.
response_uncertaintyre-export of opensmartroute.signals.uncertainty.response_uncertaintyPost-hoc uncertainty features for a set of sampled answers.
semantic_entropyre-export of opensmartroute.signals.uncertainty.semantic_entropyEntropy (nats) over meaning clusters of sampled answers; 0 when every sample agrees.
synthesize_datasetre-export of opensmartroute.signals.models.synthesize_datasetGenerate labelled prompts from ontology templates x topics with light noise.

opensmartroute.signals.corpus

Natural-language prompts per task type: how people actually ask, not how the ontology templates phrase it.

NameKindSummary
NATURAL_PROMPTSconstant: Phrasings per ontology task type (TASK_TYPES names). Every entry is labelled with its key.

opensmartroute.signals.events

Event- and workflow-driven signals for agentic inputs.

NameKindSummary
EVENT_LEXICONconstantEvent name -> (domains, actions). Keys are matched exactly, then by subject.* prefix.
EVENT_SUBJECTSconstantGeneric fallbacks when the exact event / prefix is unknown: subject -> domain, verb -> action.
EVENT_VERBSconstantevent verb -> action when the exact event / prefix is unknown.
EventInfoclassParsed event: raw name, subject, verb and the domains / actions it implies.
EventSignalclassDomains / actions from context["event"] and context["intent"] (event-driven requests).
WorkflowSignalclassDetect a workflow trigger (context["workflow"] or "run the workflow|process|...").
parse_eventfunction (name: str)Map an event name (subject.verb, subject:verb or subject_verb) onto domains / actions.

opensmartroute.signals.models

Learned signal models: hashed n-gram linear models with no dependencies.

NameKindSummary
TASK_PRIORSconstantname: (difficulty, reasoning_need, expected_output_tokens).
HashedClassifierclassMultinomial logistic regression over hashed features (sparse weights per class).
HashedFeaturizerclassHashing-trick sparse features: words, bigrams, character n-grams and a few numeric text statistics. A text.
HashedRegressorclassSquared-loss linear regressor on hashed features; output clipped to [lo, hi].
SignalModelBundleclassAll learned signal models together, persisted as one JSON document.
load_training_rowsfunction (path: str | Path)Read JSONL training rows. Accepts eval-dataset rows too (prompt/text,.
synthesize_datasetfunction (ontology: TaskOntology=DEFAULT_ONTOLOGY, topics: Sequence[str] | None=None, per_template: int=6, seed: int=0, natural: bool | int=True, natural_phrases: Mapping[str, Sequence[str]] | None=None)Generate labelled prompts from ontology templates x topics with light noise.

opensmartroute.signals.ontology

Task ontology: families -> types -> subtypes, with an orthogonal domain axis.

NameKindSummary
DEFAULT_ONTOLOGYconstantthe built-in TASK_TYPES ontology used by TaskTypeSignal.
TASK_TYPESconstant---------------------------------------------------------------- transformation.
TaskOntologyclassLookup helpers over :data:TASK_TYPES.
TaskTypeclassOne leaf of the task ontology: family, description, the actions it maps to and template phrasings.

opensmartroute.signals.uncertainty

Uncertainty signals that come from outside the query text.

NameKindSummary
HEDGESconstantHedging phrases counted in a draft response (uncertainty evidence).
SELF_CORRECTIONSconstantSelf-correction markers counted in a draft response.
DraftResponseSignalclassQuery-response mixed representation: run a cheap drafter and expose draft features.
EventTriggerclassEvent-triggered invocation: evaluate uncertainty features against rules and return the.
TriggerRuleclassfeature >= threshold (or <= when below=True) fires action.
UncertaintyGateclassA cascade quality gate built from response uncertainty.
VerbalisedDifficultySignalclassBlend a verbalised difficulty into complexity and reasoning_need.
draft_featuresfunction (text: str, draft: str, expected_tokens: int=0)Features of a cheap draft answer relative to the query: hedging, refusal, self-correction,.
parse_difficultyfunction (value: Any)Map a verbalised difficulty (number, "7/10", "hard") to [0, 1]; None if unparseable.
response_uncertaintyfunction (samples: list[str], embedder: Embedder | None=None, threshold: float=0.85, p_true: float | None=None)Post-hoc uncertainty features for a set of sampled answers.
semantic_clustersfunction (samples: Iterable[str], embedder: Embedder | None=None, threshold: float=0.85)Greedy single-link clustering of answers into meaning classes; returns a cluster id per sample.
semantic_entropyfunction (samples: Iterable[str], embedder: Embedder | None=None, threshold: float=0.85)Entropy (nats) over meaning clusters of sampled answers; 0 when every sample agrees.

opensmartroute.stack

Declarative stacks: one document that describes a whole routing setup.

NameKindSummary
REGISTRY_SCHEMEconstant: Scheme of marketplace references inside imports (registry://<slug>[@<version>]).
STACK_VERSIONconstant: Current stack document version (the osr field).
Resolverconstant: Resolver for registry:// imports: takes the reference (without the scheme) and returns the stack document.
StackclassA resolved stack: every import merged, ready to build a router.
StackChangeclassOne line of a plan: add / change / remove of a target, rule, setting or objective weight.
StackPlanclassWhat applying desired would change compared with current (osr stack plan).
dump_stackfunction (doc: dict[str, Any], fmt: str | None=None)Serialise a stack document: fmt = "yaml" | "json"; default YAML when PyYAML is installed.
is_stackfunction (doc: Any)True when doc looks like a stack document (kind: stack or an osr version with stack sections).
load_stackfunction (source: str | Path | dict[str, Any], *, resolver: Resolver | None=None, settings: Settings | None=None, _seen: frozenset[str]=frozenset())Load and resolve a stack (file path, registry:// reference or inline mapping), imports first.
plan_stackfunction (desired: Stack, current: Stack | None=None)Diff desired against current (None = nothing deployed: everything is an add).
registry_resolverfunction (base_url: str)Resolver for registry://slug[@version] imports against a marketplace.
starter_stackfunction (name: str='starter', description: str='')A minimal, valid stack document to start from (what the marketplace publish wizard pre-fills).
validate_stackfunction (doc: Any)Validate a stack document; returns human-readable problems (empty list = valid). Imports are not resolved.

opensmartroute.strategies

NameKindSummary
DEFAULT_RULESre-export of opensmartroute.strategies.protocol.DEFAULT_RULESBuilt-in risk / budget ladder: handoff > debate > aggregate > cascade > single.
AggregateResultre-export of opensmartroute.strategies.aggregate.AggregateResultOutcome of :class:MixtureOfAgents: the final response, participants, winner and agreement ratio.
AnnotatorPoolre-export of opensmartroute.strategies.human.AnnotatorPoolSkill estimation from agreement (Dawid-Skene EM) and quorum selection. Labels usually arrive from.
AnnotatorSkillre-export of opensmartroute.strategies.human.AnnotatorSkillPer-domain Beta accuracy estimate, cost and latency of one human annotator.
AuctionResultre-export of opensmartroute.strategies.auction.AuctionResultWinner, second-price payment and the per-bidder surplus / corrected claims of one auction.
AuctionStrategyre-export of opensmartroute.strategies.auction.AuctionStrategyError-aware reverse auction: bias-corrected bids, highest surplus wins, second-price payment.
BanditStrategyre-export of opensmartroute.strategies.bandit.BanditStrategyThompson-sampling Beta bandit per (context, target); context = dominant domain (+ plan role).
BeliefTrackerre-export of opensmartroute.strategies.cascade.BeliefTrackerAutoMix-style POMDP belief over the hidden state "current answer is correct".
BudgetVariantre-export of opensmartroute.strategies.elastic.BudgetVariantOne budget of an elastic model. quality and cost_scale are relative to the parent.
CacheEntryre-export of opensmartroute.strategies.semantic_cache.CacheEntryA cached response keyed by the embedding of its prompt, with source target, quality, timestamp and hits.
CacheHitre-export of opensmartroute.strategies.semantic_cache.CacheHitThe matched :class:CacheEntry and its similarity to the query.
CapabilityStrategyre-export of opensmartroute.strategies.capability.CapabilityStrategyDeclarative fit: domain / action overlap, complexity band, language, modality and quality prior.
Cascadere-export of opensmartroute.strategies.cascade.CascadeExecute ranked targets in planner order (cheapest / MDP / POMDP), stopping when the quality gate passes.
CascadePlannerre-export of opensmartroute.strategies.cascade.CascadePlannerFinite-horizon MDP over an ordered cascade with a stop action after each step.
CascadeResultre-export of opensmartroute.strategies.cascade.CascadeResultFinal response of a cascade with every step, the planned order and the planner's expected value.
CascadeStepre-export of opensmartroute.strategies.cascade.CascadeStepOne executed rung of a cascade: gate quality, latency, cost, acceptance and POMDP belief.
DeferStrategyre-export of opensmartroute.strategies.defer.DeferStrategyLearning-to-defer: scores human targets by risk, PII, escalation intent, frustration and model uncertainty.
EdgeCloudStrategyre-export of opensmartroute.strategies.edge.EdgeCloudStrategyEdge vs cloud tier choice: learned edge competence per complexity bucket vs upload / decode penalties.
EffortStrategyre-export of opensmartroute.strategies.defer.EffortStrategyMatch a target's reasoning effort_level to signals.reasoning_need; penalise over- and under-thinking.
EscalationDecisionre-export of opensmartroute.strategies.escalation.EscalationDecisionVerdict after a streamed chunk: continue / escalate / done with the competence estimate and reason.
EscalationResultre-export of opensmartroute.strategies.modality.EscalationResultOutcome of :class:ModalityEscalation: which target answered, whether it escalated, confidence, cost.
HiddenStateStrategyre-export of opensmartroute.strategies.probe.HiddenStateStrategyDirichlet probe over a dense request representation; confidence drops with epistemic uncertainty.
HumanRoutingStrategyre-export of opensmartroute.strategies.human.HumanRoutingStrategyScores HUMAN candidates by estimated accuracy in the request's domain.
ImportanceGatere-export of opensmartroute.strategies.memory.ImportanceGateHashed logistic gate: P(item will be used later | text, hint).
LLMJudgeStrategyre-export of opensmartroute.strategies.llm_judge.LLMJudgeStrategyLLM-as-router with optional score calibration.
MemoryItemre-export of opensmartroute.strategies.memory.MemoryItemOne stored memory: text, size, turn written, learned importance gate, tier and usage counters.
MemoryRouterre-export of opensmartroute.strategies.memory.MemoryRouterRoutes memory writes to tiers under budgets and recalls the most valuable items per token.
MemoryTierre-export of opensmartroute.strategies.memory.MemoryTierA storage tier: token capacity, read / write cost per token, latency and the minimum item value it accepts.
MixtureOfAgentsre-export of opensmartroute.strategies.aggregate.MixtureOfAgentsRoute-or-aggregate switch: below a confidence threshold call the top-k alternatives and aggregate.
ModalityEscalationre-export of opensmartroute.strategies.modality.ModalityEscalationTry the text-only target on the surrogate first; escalate to the multimodal target on low confidence.
ModalityStrategyre-export of opensmartroute.strategies.modality.ModalityStrategyCoverage of the request's modalities by each candidate, with surrogate discounts.
MultiRoundExecutorre-export of opensmartroute.strategies.progress.MultiRoundExecutorRouter-R1 style: keep routing/executing until the judge accepts or rounds run out.
ProgressRouterre-export of opensmartroute.strategies.progress.ProgressRouterRoute each step of a task with trajectory context.
ProtocolChoicere-export of opensmartroute.strategies.protocol.ProtocolChoiceThe protocol picked for a request, the risk that drove it, the reason and the matching rule.
ProtocolPolicyre-export of opensmartroute.strategies.protocol.ProtocolPolicyOrdered rule table from (risk, budget, task type) to a protocol, with per-protocol ledgers.
ProtocolRulere-export of opensmartroute.strategies.protocol.ProtocolRuleFirst matching rule wins. None bounds are open.
QuorumPlanre-export of opensmartroute.strategies.human.QuorumPlanA chosen set of annotators with the quorum's majority accuracy, total cost and latency.
RecallResultre-export of opensmartroute.strategies.memory.RecallResultItems recalled for a turn with their total tokens, read cost and latency.
Roundre-export of opensmartroute.strategies.progress.RoundOne route -> execute -> judge iteration of :class:MultiRoundExecutor.
RoundResultre-export of opensmartroute.strategies.progress.RoundResultFinal response of a multi-round run with every round and the shared task id.
Rulere-export of opensmartroute.strategies.rules.RuleIf all when conditions match, boost prefer targets and penalise avoid.
RulesStrategyre-export of opensmartroute.strategies.rules.RulesStrategyApplies declarative :class:Rule preferences (prefer / avoid / pin) when their when conditions match.
SelfEscalationre-export of opensmartroute.strategies.escalation.SelfEscalationStreaming competence monitor with Bayesian optimal stopping.
SemanticCachere-export of opensmartroute.strategies.semantic_cache.SemanticCacheEmbedding-keyed LRU cache with TTL and a similarity threshold. Safe to share across threads:.
SemanticCacheStrategyre-export of opensmartroute.strategies.semantic_cache.SemanticCacheStrategyScores the cache target by calibrated hit quality; leaves real targets to the other strategies.
SessionAffinityStrategyre-export of opensmartroute.strategies.session.SessionAffinityStrategyPrefer the target already serving context["session_id"] unless the intent shifted or it failed.
SessionStatere-export of opensmartroute.strategies.session.SessionStateWhat the strategy remembers about one session.
SimilarityStrategyre-export of opensmartroute.strategies.similarity.SimilarityStrategyEmbed the request and each target's examples / description; score by best and top-k mean similarity.
SpeculativeCascadere-export of opensmartroute.strategies.speculative.SpeculativeCascadeTwo-target cascade that overlaps the draft and the strong call when it pays.
SpeculativeResultre-export of opensmartroute.strategies.speculative.SpeculativeResultOutcome of a speculative run: chosen mode, whether the draft was accepted, latency and cost.
SpreadStrategyre-export of opensmartroute.strategies.spread.SpreadStrategySpread requests across same-family targets in proportion to metadata[weight_key].
Strategyre-export of opensmartroute.strategies.base.StrategyScores each candidate target in [0, 1] and explains why.
StreamResultre-export of opensmartroute.strategies.escalation.StreamResultText consumed by :func:wrap_stream, whether it escalated and the final decision.
TaskProgressre-export of opensmartroute.strategies.progress.TaskProgressWhere a multi-step task stands: step index, budget spent, failures in a row, last target, history.
TaskTableStrategyre-export of opensmartroute.strategies.task_table.TaskTableStrategyStatic task_type -> target -> quality table with family and prior fallbacks; learns from outcomes.
TokenBudgetStrategyre-export of opensmartroute.strategies.elastic.TokenBudgetStrategyScore budgeted siblings by fit between the budget and the tokens the answer needs.
decide_effortre-export of opensmartroute.strategies.defer.decide_effortPick the effort level whose numeric value is closest to the reasoning need.
default_bidre-export of opensmartroute.strategies.auction.default_bidCatalogue-derived bid: quality prior as the claim, unit cost per 1k tokens x tokens as the price.
default_strategiesfunction (seed: int | None=None, settings: Settings | None=None, state_dir: str | Path | None=None)The zero-configuration ensemble: capability fit + example similarity + Thompson bandit.
expand_elasticre-export of opensmartroute.strategies.elastic.expand_elasticCreate one sibling target per budget variant, sharing the parent's family, handler and metadata.
expected_mode_costsre-export of opensmartroute.strategies.speculative.expected_mode_costsExpected weighted cost (objective.cost x USD + objective.latency x seconds) per mode.
failure_riskre-export of opensmartroute.strategies.protocol.failure_riskRisk in [0, 1] that a single call fails: low confidence, high complexity and reasoning need,.
hashing_embedderre-export of opensmartroute.strategies.similarity.hashing_embedderWord + bigram hashing embedder. Deterministic, zero deps, decent for routing.
majority_votere-export of opensmartroute.strategies.aggregate.majority_voteLargest meaning cluster wins; returns (answer, winner_target_id, agreement).
quorum_accuracyre-export of opensmartroute.strategies.human.quorum_accuracyP(weighted majority is correct) for independent annotators with the given accuracies.
request_modalitiesre-export of opensmartroute.strategies.modality.request_modalitiesModalities a request carries and which of them have a text surrogate in context.
wrap_streamre-export of opensmartroute.strategies.escalation.wrap_streamConsume chunks until the monitor escalates or the stream ends.

opensmartroute.strategies.aggregate

Routing / aggregation switch (Mixture-of-Agents, Wang et al. 2024; JiSi 2601.01330).

NameKindSummary
AggregateResultclassOutcome of :class:MixtureOfAgents: the final response, participants, winner and agreement ratio.
Aggregatorconstant(request, [(target_id, answer)]) -> answer.
MixtureOfAgentsclassRoute-or-aggregate switch: below a confidence threshold call the top-k alternatives and aggregate.
ParticipantclassOne target's contribution to an aggregate: response, cost, latency, error and whether it agreed.
majority_votefunction (request: RouteRequest, answers: list[tuple[str, Any]], embedder: Embedder | None=None, threshold: float=0.85)Largest meaning cluster wins; returns (answer, winner_target_id, agreement).

opensmartroute.strategies.auction

Error-aware reverse auction across providers (EA-RAM 2608.12719).

NameKindSummary
AuctionResultclassWinner, second-price payment and the per-bidder surplus / corrected claims of one auction.
AuctionStrategyclassError-aware reverse auction: bias-corrected bids, highest surplus wins, second-price payment.
BidFnconstant-> (claimed P(success), price).
BidderRecordclassCalibration ledger of one bidder: signed claim bias and a Beta record of realised success.
default_bidfunction (target: RouteTarget, request: RouteRequest, signals: Signals)Catalogue-derived bid: quality prior as the claim, unit cost per 1k tokens x tokens as the price.

opensmartroute.strategies.bandit

Online-learning strategy: contextual Thompson-sampling bandit.

NameKindSummary
BanditStrategyclassThompson-sampling Beta bandit per (context, target); context = dominant domain (+ plan role).

opensmartroute.strategies.base

Strategy interface.

NameKindSummary
StrategyclassScores each candidate target in [0, 1] and explains why.
role_offunction (signals: Signals)Plan-slot role (persona / skill / llm) the router is currently filling, or.
memory_keyfunction (request_id: str, role: str | None)Key for per-request strategy memory: the same request is scored once per plan role, and.

opensmartroute.strategies.capability

Capability-fit strategy: match request signals to a target's declared capabilities.

NameKindSummary
WEB_TAGSconstant: Capability tags that say a target can reach the live web (grounded search models, browsing agents, search tools).
GENERAL_DOMAINconstant: The extractor's fallback domain; a target that declares it serves any domain and is never "off-domain".
CapabilityStrategyclassDeclarative fit: domain / action overlap, complexity band, language, modality and quality prior.

opensmartroute.strategies.cascade

Cascade execution (FrugalGPT / Router-R1 multi-round / AutoMix POMDP).

NameKindSummary
CascadeStepclassOne executed rung of a cascade: gate quality, latency, cost, acceptance and POMDP belief.
CascadeResultclassFinal response of a cascade with every step, the planned order and the planner's expected value.
CascadePlannerclassFinite-horizon MDP over an ordered cascade with a stop action after each step.
BeliefTrackerclassAutoMix-style POMDP belief over the hidden state "current answer is correct".
CascadeclassExecute ranked targets in planner order (cheapest / MDP / POMDP), stopping when the quality gate passes.

opensmartroute.strategies.defer

Defer-to-human and effort ("think or not") strategies.

NameKindSummary
DeferStrategyclassLearning-to-defer: scores human targets by risk, PII, escalation intent, frustration and model uncertainty.
EffortStrategyclassMatch a target's reasoning effort_level to signals.reasoning_need; penalise over- and under-thinking.
decide_effortfunction (signals: Signals, levels: tuple[str, ...]=('none', 'low', 'medium', 'high'))Pick the effort level whose numeric value is closest to the reasoning need.

opensmartroute.strategies.edge

Edge-cloud token-aware routing (Pro-Router 2608.28726; RelayLLM 2601.05167).

NameKindSummary
EdgeCloudStrategyclassEdge vs cloud tier choice: learned edge competence per complexity bucket vs upload / decode penalties.

opensmartroute.strategies.elastic

Token-budget-aware routing to elastic / many-in-one models (Nemotron Elastic 2511.16664; Star 2605.07182).

NameKindSummary
BudgetVariantclassOne budget of an elastic model. quality and cost_scale are relative to the parent.
TokenBudgetStrategyclassScore budgeted siblings by fit between the budget and the tokens the answer needs.
expand_elasticfunction (parent: RouteTarget, variants: list[BudgetVariant | dict[str, Any]])Create one sibling target per budget variant, sharing the parent's family, handler and metadata.

opensmartroute.strategies.escalation

Bayesian self-escalation during generation (2608.24087).

NameKindSummary
REFUSALSconstantRefusal phrases that lower the streaming competence posterior.
EscalationDecisionclassVerdict after a streamed chunk: continue / escalate / done with the competence estimate and reason.
SelfEscalationclassStreaming competence monitor with Bayesian optimal stopping.
StreamResultclassText consumed by :func:wrap_stream, whether it escalated and the final decision.
wrap_streamfunction (chunks: Iterable[str], monitor: SelfEscalation | None=None)Consume chunks until the monitor escalates or the stream ends.

opensmartroute.strategies.human

Routing among human annotators and experts (QUORUM 2608.27974; Dawid-Skene 1979).

NameKindSummary
AnnotatorPoolclassSkill estimation from agreement (Dawid-Skene EM) and quorum selection. Labels usually arrive from.
AnnotatorSkillclassPer-domain Beta accuracy estimate, cost and latency of one human annotator.
HumanRoutingStrategyclassScores HUMAN candidates by estimated accuracy in the request's domain.
QuorumPlanclassA chosen set of annotators with the quorum's majority accuracy, total cost and latency.
quorum_accuracyfunction (accuracies: Sequence[float])P(weighted majority is correct) for independent annotators with the given accuracies.

opensmartroute.strategies.llm_judge

Generative routing: let an LLM act as the router (Router-R1 / LLM-as-judge).

NameKindSummary
PROMPTconstantPrompt template for the judge; formatted with complexity, domains, actions, the catalogue and the request.
LLMJudgeStrategyclassLLM-as-router with optional score calibration.

opensmartroute.strategies.memory

Memory-tier routing for agents (BudgetMem 2602.06025; Gated-Memory Routing 2609.00237).

NameKindSummary
ImportanceGateclassHashed logistic gate: P(item will be used later | text, hint).
MemoryItemclassOne stored memory: text, size, turn written, learned importance gate, tier and usage counters.
MemoryRouterclassRoutes memory writes to tiers under budgets and recalls the most valuable items per token.
MemoryTierclassA storage tier: token capacity, read / write cost per token, latency and the minimum item value it accepts.
RecallResultclassItems recalled for a turn with their total tokens, read cost and latency.

opensmartroute.strategies.modality

Multimodal routing and modality escalation (LatentRouter 2605.11301; modality escalation.

NameKindSummary
EscalationResultclassOutcome of :class:ModalityEscalation: which target answered, whether it escalated, confidence, cost.
ModalityEscalationclassTry the text-only target on the surrogate first; escalate to the multimodal target on low confidence.
ModalityStrategyclassCoverage of the request's modalities by each candidate, with surrogate discounts.
request_modalitiesfunction (request: RouteRequest)Modalities a request carries and which of them have a text surrogate in context.

opensmartroute.strategies.probe

Hidden-state routing with a Dirichlet probe (ProbeDirichlet, RouterXBench 2602.11877).

NameKindSummary
HiddenStateStrategyclassDirichlet probe over a dense request representation; confidence drops with epistemic uncertainty.
StateFnconstantrequest -> dense feature / hidden-state vector.

opensmartroute.strategies.progress

Agentic trajectories: per-step routing and multi-round execution.

NameKindSummary
MultiRoundExecutorclassRouter-R1 style: keep routing/executing until the judge accepts or rounds run out.
ProgressRouterclassRoute each step of a task with trajectory context.
RoundclassOne route -> execute -> judge iteration of :class:MultiRoundExecutor.
RoundResultclassFinal response of a multi-round run with every round and the shared task id.
TaskProgressclassWhere a multi-step task stands: step index, budget spent, failures in a row, last target, history.

opensmartroute.strategies.protocol

Collaboration-protocol selection (2608.14927).

NameKindSummary
DEFAULT_RULESconstantBuilt-in risk / budget ladder: handoff > debate > aggregate > cascade > single.
PROTOCOLSconstantin escalation order.
Protocolconstantexecution protocols.
ProtocolChoiceclassThe protocol picked for a request, the risk that drove it, the reason and the matching rule.
ProtocolPolicyclassOrdered rule table from (risk, budget, task type) to a protocol, with per-protocol ledgers.
ProtocolRuleclassFirst matching rule wins. None bounds are open.
failure_riskfunction (decision: RouteDecision | None, signals: Signals | None=None, uncertainty: float | None=None)Risk in [0, 1] that a single call fails: low confidence, high complexity and reasoning need,.

opensmartroute.strategies.rules

Declarative rule-based routing (Arch-Router-style domain/action preferences).

NameKindSummary
RuleclassIf all when conditions match, boost prefer targets and penalise avoid.
RulesStrategyclassApplies declarative :class:Rule preferences (prefer / avoid / pin) when their when conditions match.

opensmartroute.strategies.semantic_cache

Semantic caching as a routing target (GPTCache; vLLM semantic router 2603.04444).

NameKindSummary
CacheEntryclassA cached response keyed by the embedding of its prompt, with source target, quality, timestamp and hits.
CacheHitclassThe matched :class:CacheEntry and its similarity to the query.
SemanticCacheclassEmbedding-keyed LRU cache with TTL and a similarity threshold. Safe to share across threads:.
SemanticCacheStrategyclassScores the cache target by calibrated hit quality; leaves real targets to the other strategies.

opensmartroute.strategies.session

Session affinity: keep a conversation with the target that is already serving it.

NameKindSummary
SessionAffinityStrategyclassPrefer the target already serving context["session_id"] unless the intent shifted or it failed.
SessionStateclassWhat the strategy remembers about one session.

opensmartroute.strategies.similarity

Similarity-based routing (UniRoute / GraphRouter flavour).

NameKindSummary
hashing_embedderfunction (dim: int=512)Word + bigram hashing embedder. Deterministic, zero deps, decent for routing.
cosinefunction (a: list[float], b: list[float])Cosine similarity; zero vectors are treated as unit norm (no division by zero).
SimilarityStrategyclassEmbed the request and each target's examples / description; score by best and top-k mean similarity.

opensmartroute.strategies.speculative

Speculative (draft-based) cascades (speculative cascades, Narasimhan et al. 2024; Differential.

NameKindSummary
SpeculativeCascadeclassTwo-target cascade that overlaps the draft and the strong call when it pays.
SpeculativeResultclassOutcome of a speculative run: chosen mode, whether the draft was accepted, latency and cost.
expected_mode_costsfunction (p_accept: float, draft: RouteTarget, strong: RouteTarget, objective: Objective, *, tokens: int=500, cancellable: bool=False, verify_ms: float=0.0)Expected weighted cost (objective.cost x USD + objective.latency x seconds) per mode.

opensmartroute.strategies.spread

Weighted spread across interchangeable deployments (LiteLLM "simple-shuffle", OpenRouter provider weights).

NameKindSummary
SpreadStrategyclassSpread requests across same-family targets in proportion to metadata[weight_key].

opensmartroute.strategies.task_table

Static task table strategy (SCX Router, 2609.02292).

NameKindSummary
TaskTableStrategyclassStatic task_type -> target -> quality table with family and prior fallbacks; learns from outcomes.

Something wrong or out of date on this page? Tell us - the page is named for you, so a one-line note is enough.

Published benchmark runs

The published run on MT-Bench human, PPE human and WebDev Arena against the declarative router and every baseline.

Deploy with Docker and Helm

The osr serve container image, entrypoint environment, Helm chart and health checks.

On this page

Modulesopensmartrouteopensmartroute.__main__opensmartroute.adaptersopensmartroute.adapters.a2aopensmartroute.adapters.anthropic_clientopensmartroute.adapters.catalogueopensmartroute.adapters.frameworksopensmartroute.adapters.handlersopensmartroute.adapters.harnessopensmartroute.adapters.marketplaceopensmartroute.adapters.mcpopensmartroute.adapters.mcp_serversopensmartroute.adapters.mock_clientopensmartroute.adapters.openai_compatopensmartroute.adapters.optionalopensmartroute.adapters.personasopensmartroute.adapters.semantic_routeropensmartroute.adapters.skillsopensmartroute.adapters.websearchopensmartroute.agentsopensmartroute.aioopensmartroute.anthropicopensmartroute.brandingopensmartroute.cliopensmartroute.clientopensmartroute.compressopensmartroute.configopensmartroute.coreopensmartroute.core.registryopensmartroute.core.typesopensmartroute.credentialsopensmartroute.dataopensmartroute.discoveryopensmartroute.enterpriseopensmartroute.enterprise.groundingopensmartroute.enterprise.opsopensmartroute.enterprise.savingsopensmartroute.enterprise.storesopensmartroute.errorsopensmartroute.estimateopensmartroute.evalopensmartroute.eval.agenticopensmartroute.eval.auditopensmartroute.eval.baselinesopensmartroute.eval.collectopensmartroute.eval.criteriaopensmartroute.eval.datasetsopensmartroute.eval.frontieropensmartroute.eval.headroomopensmartroute.eval.opeopensmartroute.eval.robustnessopensmartroute.executionopensmartroute.feedbackopensmartroute.geminiopensmartroute.inferenceopensmartroute.inference.footprintopensmartroute.inference.ollamaopensmartroute.inference.planneropensmartroute.jsonrepairopensmartroute.knowledgeopensmartroute.knowledge.contextopensmartroute.knowledge.graphopensmartroute.knowledge.ingestopensmartroute.learningopensmartroute.learning.attentionopensmartroute.learning.autopilotopensmartroute.learning.coldstartopensmartroute.learning.contrastiveopensmartroute.learning.creditopensmartroute.learning.embedopensmartroute.learning.handoffopensmartroute.learning.implicitopensmartroute.learning.kgeopensmartroute.learning.multiturnopensmartroute.learning.neuralopensmartroute.learning.personalopensmartroute.learning.policy_gradientopensmartroute.learning.quantopensmartroute.learning.self_improveopensmartroute.learning.slmopensmartroute.learning.windowopensmartroute.mathopensmartroute.math.banditsopensmartroute.math.calibrationopensmartroute.math.decisionopensmartroute.math.dirichletopensmartroute.math.energyopensmartroute.math.estimatorsopensmartroute.math.irtopensmartroute.math.markovopensmartroute.math.preferenceopensmartroute.mcp_serveropensmartroute.mediaopensmartroute.migrateopensmartroute.observabilityopensmartroute.ocmopensmartroute.policyopensmartroute.realtimeopensmartroute.responsesopensmartroute.retrievalopensmartroute.routeropensmartroute.sdkopensmartroute.securityopensmartroute.security.gadgetopensmartroute.security.injectionopensmartroute.security.limitsopensmartroute.security.moderationopensmartroute.security.outputopensmartroute.security.provenanceopensmartroute.security.safetyopensmartroute.serveropensmartroute.settingsopensmartroute.signalsopensmartroute.signals.corpusopensmartroute.signals.eventsopensmartroute.signals.modelsopensmartroute.signals.ontologyopensmartroute.signals.uncertaintyopensmartroute.stackopensmartroute.strategiesopensmartroute.strategies.aggregateopensmartroute.strategies.auctionopensmartroute.strategies.banditopensmartroute.strategies.baseopensmartroute.strategies.capabilityopensmartroute.strategies.cascadeopensmartroute.strategies.deferopensmartroute.strategies.edgeopensmartroute.strategies.elasticopensmartroute.strategies.escalationopensmartroute.strategies.humanopensmartroute.strategies.llm_judgeopensmartroute.strategies.memoryopensmartroute.strategies.modalityopensmartroute.strategies.probeopensmartroute.strategies.progressopensmartroute.strategies.protocolopensmartroute.strategies.rulesopensmartroute.strategies.semantic_cacheopensmartroute.strategies.sessionopensmartroute.strategies.similarityopensmartroute.strategies.speculativeopensmartroute.strategies.spreadopensmartroute.strategies.task_table