Skip to content
OpenSmartRoute

API reference

Every public module, class and function with its signature, generated from the source.

docs/REFERENCE.md

Every module under src/opensmartroute and every public name it exports, generated from the source by python scripts/api_reference.py (checked by tests/test_docs.py; do not edit by hand). The first line of each docstring is the summary; open the module for the full contract. Narrative documentation: GUIDE.md (usage), SDK.md (stability, extension points), ARCHITECTURE.md, MATH.md, ENTERPRISE.md, SECURITY.md, RESEARCH.md.

Stability: names re-exported from opensmartroute (the top-level package) are frozen by tests/public_api.json and follow SemVer. Sub-module names are public but may change in a minor release with a CHANGELOG entry.

Modules#

opensmartroute#

Source: src/opensmartroute/init.py

OpenSmartRoute — an open, intelligent route to the right decision, solution, or destination.

NameKindSummary
AsyncRouterre-export of opensmartroute.aio.AsyncRouterawait-able wrapper around a :class:Router: route / learn / execute / run off the event loop.
BanditStrategyre-export of opensmartroute.strategies.bandit.BanditStrategyThompson-sampling Beta bandit per (context, target); context = dominant domain (+ plan role).
Capabilitiesre-export of opensmartroute.core.types.CapabilitiesDeclarative description of what a target is good at.
CapabilityStrategyre-export of opensmartroute.strategies.capability.CapabilityStrategyDeclarative fit: domain / action overlap, complexity band, language, modality and quality prior.
Cascadere-export of opensmartroute.strategies.cascade.CascadeExecute ranked targets in planner order (cheapest / MDP / POMDP), stopping when the quality gate passes.
CascadePlannerre-export of opensmartroute.strategies.cascade.CascadePlannerFinite-horizon MDP over an ordered cascade with a stop action after each step.
ComponentRegistryre-export of opensmartroute.sdk.ComponentRegistryBlueprints for every routing component, with decorators that register into it.
ConfigurationErrorre-export of opensmartroute.errors.ConfigurationErrorInvalid catalogue, rules, SKILL.md, settings or a missing optional dependency.
DeferStrategyre-export of opensmartroute.strategies.defer.DeferStrategyLearning-to-defer: scores human targets by risk, PII, escalation intent, frustration and model uncertainty.
EffortStrategyre-export of opensmartroute.strategies.defer.EffortStrategyMatch a target's reasoning effort_level to signals.reasoning_need; penalise over- and under-thinking.
Eventre-export of opensmartroute.observability.EventOne captured span or event: flat, JSON-friendly, never carries request text.
EventSinkre-export of opensmartroute.observability.EventSinkReceiver port: override :meth:emit; live bridges may also implement :meth:span_start / :meth:span_end.
ExecutionErrorre-export of opensmartroute.errors.ExecutionErrorA target handler failed while executing a plan.
ExecutionResultre-export of opensmartroute.execution.ExecutionResultWhat happened when a decision was executed.
ExecutionStepre-export of opensmartroute.execution.ExecutionStepOne executed plan slot: role, target, latency and whether it succeeded.
FeedbackStorere-export of opensmartroute.feedback.FeedbackStoreAppend-only :class:Outcome log (in memory or JSONL file) with per-target statistics.
LLMJudgeStrategyre-export of opensmartroute.strategies.llm_judge.LLMJudgeStrategyLLM-as-router with optional score calibration.
NoRouteErrorre-export of opensmartroute.errors.NoRouteErrorNo target satisfied the hard constraints.
Objectivere-export of opensmartroute.core.types.ObjectiveWhat the caller wants to optimise. Weights are relative.
OpenSmartRouteErrorre-export of opensmartroute.errors.OpenSmartRouteErrorBase class for all SDK errors.
Outcomere-export of opensmartroute.core.types.OutcomeFeedback about how a routed request actually went.
PlanSlotre-export of opensmartroute.core.types.PlanSlotOne filled slot of a multi-target plan (persona -> skill -> model).
Policyre-export of opensmartroute.policy.PolicyOrdered chain of :data:PolicyRule; returns the first rejection reason or None.
ProgressRouterre-export of opensmartroute.strategies.progress.ProgressRouterRoute each step of a task with trajectory context.
RankedTargetre-export of opensmartroute.core.types.RankedTargetA scored candidate: utility, ensemble quality estimate and the per-strategy breakdown.
RequestConstraintsre-export of opensmartroute.core.types.RequestConstraintsHard constraints on the request (never traded off).
RouteDecisionre-export of opensmartroute.core.types.RouteDecisionThe answer to route(): chosen target, confidence, alternatives, optional plan, trace and propensities.
RoutePlanre-export of opensmartroute.core.types.RoutePlanA composed route (MasRouter-style): several targets working together.
RouteRequestre-export of opensmartroute.core.types.RouteRequestThe customer need.
RouteTargetre-export of opensmartroute.core.types.RouteTargetA routable destination: an LLM, agent, skill, persona, tool, workflow or human.
RouteTracere-export of opensmartroute.core.types.RouteTraceEverything needed to explain a decision.
Routerre-export of opensmartroute.router.RouterSignals -> policy -> strategies -> ensemble -> decision (+ optional plan).
RouterSLMre-export of opensmartroute.learning.slm.RouterSLMSmall routing model: dual encoder + target catalogue snapshot + calibration, in one JSON file.
Rulere-export of opensmartroute.strategies.rules.RuleIf all when conditions match, boost prefer targets and penalise avoid.
RulesStrategyre-export of opensmartroute.strategies.rules.RulesStrategyApplies declarative :class:Rule preferences (prefer / avoid / pin) when their when conditions match.
SLMStrategyre-export of opensmartroute.learning.slm.SLMStrategyEnsemble member backed by a :class:RouterSLM; scores are its probabilities, and it keeps learning online.
SecurityErrorre-export of opensmartroute.errors.SecurityErrorRequest rejected by an input guard (prompt injection, oversize, etc.).
SelfImproverre-export of opensmartroute.learning.self_improve.SelfImproverClosed loop that keeps a :class:RouterSLM current with the model market and its own traffic.
Settingsre-export of opensmartroute.settings.SettingsAll tunables, grouped by consumer. Immutable; derive variants with :meth:replace.
Signalsre-export of opensmartroute.core.types.SignalsCheap deterministic features extracted from a request.
SimilarityStrategyre-export of opensmartroute.strategies.similarity.SimilarityStrategyEmbed the request and each target's examples / description; score by best and top-k mean similarity.
Spanre-export of opensmartroute.observability.SpanAn open unit of work; a context manager that records duration, status and nested events.
StateStoreErrorre-export of opensmartroute.errors.StateStoreErrorA learner-state store failed to load, save or migrate.
Strategyre-export of opensmartroute.strategies.base.StrategyScores each candidate target in [0, 1] and explains why.
StrategyScorere-export of opensmartroute.core.types.StrategyScoreOne strategy's opinion about one target.
TargetConstraintsre-export of opensmartroute.core.types.TargetConstraintsWhere / for whom a target may be used. Checked by the policy layer.
TargetKindre-export of opensmartroute.core.types.TargetKindKinds of things a request can be routed to.
TargetRegistryre-export of opensmartroute.core.registry.TargetRegistryIn-memory catalogue of :class:RouteTarget by id: add / upsert / remove, filtered listing, (de)serialisation.
TargetUnavailableErrorre-export of opensmartroute.errors.TargetUnavailableErrorA remote target or catalogue source could not be reached.
TaskTableStrategyre-export of opensmartroute.strategies.task_table.TaskTableStrategyStatic task_type -> target -> quality table with family and prior fallbacks; learns from outcomes.
Tracerre-export of opensmartroute.observability.TracerOpens spans, records events and fans them out to sinks; sample_rate < 1 traces a share of requests.
ValidationErrorre-export of opensmartroute.errors.ValidationErrorA request, outcome or target failed validation.
__version__constant
agentre-export of opensmartroute.sdk.agent@agent(id, ...): shorthand for an agent target.
componentsre-export of opensmartroute.sdk.components: The process-wide registry that the top-level decorators (opensmartroute.strategy ...) bind to.
configurere-export of opensmartroute.settings.configureInstall process-wide settings. configure() with no arguments re-reads the environment;.
configure_tracingre-export of opensmartroute.observability.configure_tracingInstall sinks on the process-wide tracer. With no sinks, use the ones named by settings.
get_settingsre-export of opensmartroute.settings.get_settingsThe process-wide :class:Settings (environment overlay applied once, lazily).
get_tracerre-export of opensmartroute.observability.get_tracerThe process-wide tracer (disabled until :func:configure_tracing adds sinks).
load_rulesre-export of opensmartroute.config.load_rulesLoad a rules file (top-level list or rules: key) into a :class:RulesStrategy.
load_targetsre-export of opensmartroute.config.load_targetsLoad a catalogue file (top-level list or targets: key) into a :class:TargetRegistry.
middlewarere-export of opensmartroute.sdk.middleware@middleware: register a Middleware class or fn(request, next_route).
policy_rulere-export of opensmartroute.sdk.policy_rule@policy_rule: register fn(target, request, signals) -> reason | None.
signalre-export of opensmartroute.sdk.signal@signal: register a SignalExtractor class or fn(request, signals) -> mapping.
skillre-export of opensmartroute.sdk.skill@skill(id, ...): shorthand for a skill target.
strategyre-export of opensmartroute.sdk.strategy@strategy(weight=, name=): register a Strategy class or fn(request, signals, candidates).
targetre-export of opensmartroute.sdk.target@target(id, kind, ...): the decorated callable becomes a RouteTarget handler.
telemetryre-export of opensmartroute.sdk.telemetry@telemetry: register a Telemetry sink class or factory.
toolre-export of opensmartroute.sdk.tool@tool(id, ...): shorthand for a tool target.

opensmartroute.adapters#

Source: src/opensmartroute/adapters/init.py

Adapters connect OpenSmartRoute to real providers and infrastructure.

NameKindSummary
AgentHarnessre-export of opensmartroute.adapters.harness.AgentHarnessStructural interface of an agent runtime: run(task, context=, history=) -> HarnessResult.
CallableHarnessre-export of opensmartroute.adapters.harness.CallableHarnessWrap an in-process agent: fn(task, context, history) -> str | dict | HarnessResult.
ChatResultre-export of opensmartroute.adapters.openai_compat.ChatResultResult of a chat completion: text, model, token counts, latency and the raw response.
HTTPHarnessre-export of opensmartroute.adapters.harness.HTTPHarnessPOST the task as JSON to an agent endpoint.
HarnessResultre-export of opensmartroute.adapters.harness.HarnessResultWhat an agent harness returns: text, success, token / cost / latency usage and optional self-graded quality.
InMemoryQueuere-export of opensmartroute.adapters.handlers.InMemoryQueueReference :class:Queue: FIFO in memory, resolves into :class:~opensmartroute.Outcome.
ModelCardre-export of opensmartroute.adapters.catalogue.ModelCardWhat the catalogue knows about one model: identity, price, limits, evidence of quality, provenance.
ModelCataloguere-export of opensmartroute.adapters.catalogue.ModelCatalogueMerged, persisted model cards from every source; the SLM's view of the target universe.
OpenAICompatClientre-export of opensmartroute.adapters.openai_compat.OpenAICompatClientStdlib-only client for the OpenAI chat / embeddings API (OpenAI, Azure, vLLM, Ollama, LiteLLM) with retries.
OpenTelemetrySinkre-export of opensmartroute.adapters.optional.OpenTelemetrySinkLive bridge from the tracer to OpenTelemetry: every OpenSmartRoute span becomes an OTel span.
OpenTelemetryTelemetryre-export of opensmartroute.adapters.optional.OpenTelemetryTelemetryEmits one span per decision and counters/histograms via the OTel API.
PendingResultre-export of opensmartroute.adapters.handlers.PendingResultImmediate answer of a queued target: the request was accepted and will be answered later.
Queuere-export of opensmartroute.adapters.handlers.QueueStructural interface of an asynchronous queue target (ticketing, human tier, workflow run).
QueuedItemre-export of opensmartroute.adapters.handlers.QueuedItemOne request waiting in (or resolved from) a queue.
SearchHitre-export of opensmartroute.adapters.websearch.SearchHitOne search result: where it came from, what it says, and when it was seen.
SemanticRouterImportre-export of opensmartroute.adapters.semantic_router.SemanticRouterImportResult of importing a vLLM semantic-router config: registry, rules, default model, categories, warnings.
ServerCardre-export of opensmartroute.adapters.mcp_servers.ServerCardDescription of an MCP server (tools, tags, auth, latency, cost, region, data boundary) for recommendation.
ServerRecommendationre-export of opensmartroute.adapters.mcp_servers.ServerRecommendationA ranked server from recommend_servers with its fused score, rationale and matched tool names.
StdioMCPClientre-export of opensmartroute.adapters.mcp.StdioMCPClientTiny JSON-RPC-over-stdio MCP client (newline-delimited). Thread-safe, blocking.
SubprocessHarnessre-export of opensmartroute.adapters.harness.SubprocessHarnessRun a CLI agent: task on stdin, answer on stdout, exit code 0 = success.
WebKnowledgere-export of opensmartroute.adapters.websearch.WebKnowledgeFan a query out to search providers, de-duplicate by URL and cache the hits as JSON.
a2a_handlerre-export of opensmartroute.adapters.a2a.a2a_handlerReturn a handler that sends the request text to an A2A agent and returns its text.
agent_from_cardre-export of opensmartroute.adapters.a2a.agent_from_cardBuild an agent RouteTarget from an A2A agent card (name, skills, tags, input modes).
brave_searchre-export of opensmartroute.adapters.websearch.brave_searchBrave Search API web results; the key is read from BRAVE_API_KEY (or BRAVE_API_KEY_FILE).
card_to_targetre-export of opensmartroute.adapters.catalogue.card_to_targetA :class:RouteTarget for a model card; risky third-party descriptions are replaced by the name.
attach_chat_handlersre-export of opensmartroute.adapters.openai_compat.attach_chat_handlersGive every LLM target a :func:chat_handler on client; returns the ids that got one.
chat_handlerre-export of opensmartroute.adapters.openai_compat.chat_handlerAdapter for RouteTarget.handler: turns a RouteRequest into a chat call.
connect_mcpre-export of opensmartroute.adapters.mcp.connect_mcpSpawn a stdio MCP server, list its tools and return (client, targets).
duckduckgo_searchre-export of opensmartroute.adapters.websearch.duckduckgo_searchDuckDuckGo instant-answer API (abstract + related topics). Keyless; shallow but good for definitions.
embedderre-export of opensmartroute.adapters.openai_compat.embedderAdapter for SimilarityStrategy(embedder=...).
enrich_descriptionre-export of opensmartroute.adapters.mcp.enrich_descriptionReturn (routing_description, examples, capabilities) for a tool.
fetch_agent_cardre-export of opensmartroute.adapters.a2a.fetch_agent_cardDownload an agent card over HTTPS (plain HTTP is refused); raises TargetUnavailableError on failure.
fetch_bytesre-export of opensmartroute.adapters.websearch.fetch_bytesGET url over https with a timeout and a body cap; transport errors become TargetUnavailableError.
fetch_huggingface_modelsre-export of opensmartroute.adapters.catalogue.fetch_huggingface_modelsModel cards from the Hugging Face Hub search (downloads, likes, tags, model-index benchmarks).
fetch_jsonre-export of opensmartroute.adapters.websearch.fetch_jsonGET a JSON document (see :func:fetch_bytes); malformed bodies raise TargetUnavailableError.
fetch_leaderboard_qualityre-export of opensmartroute.adapters.catalogue.fetch_leaderboard_quality{hub model id: {benchmark: accuracy}} from the Open LLM Leaderboard table (official rows, unflagged).
fetch_openrouter_modelsre-export of opensmartroute.adapters.catalogue.fetch_openrouter_modelsModel cards from OpenRouter's public listing (prices per token, context, modalities, tool support).
fetch_page_textre-export of opensmartroute.adapters.websearch.fetch_page_textFetch a page and return {url, title, text, risk}; risk is the injection/gadget risk of the text.
harness_handlerre-export of opensmartroute.adapters.harness.harness_handlerAdapter for RouteTarget.handler: a RouteRequest becomes a harness task.
html_to_textre-export of opensmartroute.adapters.websearch.html_to_text(title, text) of an HTML document with scripts/styles removed and whitespace collapsed.
http_handlerre-export of opensmartroute.adapters.handlers.http_handlerRouteTarget.handler that POSTs the request to url and returns a :class:HarnessResult.
huggingface_searchre-export of opensmartroute.adapters.websearch.huggingface_searchSearch the Hugging Face Hub (what = models or datasets), ranked by downloads. No key needed.
judge_fnre-export of opensmartroute.adapters.openai_compat.judge_fnAdapter for LLMJudgeStrategy(llm=...): prompt in, completion text out.
langgraph_conditionre-export of opensmartroute.adapters.frameworks.langgraph_conditionEdge selector for add_conditional_edges: routes on target id (by='target') or kind.
langgraph_nodere-export of opensmartroute.adapters.frameworks.langgraph_nodeReturn a LangGraph-compatible node state -> dict (partial state update).
load_personasre-export of opensmartroute.adapters.personas.load_personasLoad personas from a directory of markdown files or a JSON/JSONL/CSV catalogue.
load_semantic_router_configre-export of opensmartroute.adapters.semantic_router.load_semantic_router_configConvert a vLLM semantic-router model_config / categories document into targets and rules.
load_skillre-export of opensmartroute.adapters.skills.load_skillLoad one skill directory (must contain SKILL.md).
load_skillsre-export of opensmartroute.adapters.skills.load_skillsLoad every */SKILL.md under root (one level deep, sorted by name).
maf_router_executorre-export of opensmartroute.adapters.frameworks.maf_router_executorAgent Framework style: executor(message, ctx) -> target id | response; handoffs maps.
manifest_from_targetsre-export of opensmartroute.adapters.mcp.manifest_from_targetsReverse: dump MCP-shaped tool dicts (for signing / publishing a catalogue).
model_keyre-export of opensmartroute.adapters.catalogue.model_keyVendor-agnostic key for matching model names across sources: lowercase, no vendor prefix, no punctuation.
mcp_tool_handlerre-export of opensmartroute.adapters.handlers.mcp_tool_handlerRouteTarget.handler that invokes one MCP tool via call(name, arguments).
openai_tool_handlerre-export of opensmartroute.adapters.frameworks.openai_tool_handlerCallable behind openai_tool_spec: (text, objective?, kinds?) -> RouteDecision.to_dict().
openai_tool_specre-export of opensmartroute.adapters.frameworks.openai_tool_specOpenAI function-calling tool definition that lets a model ask the router for a target.
persona_from_markdownre-export of opensmartroute.adapters.personas.persona_from_markdownParse a *.agent.md / *.chatmode.md / front-matter markdown file into a persona target.
persona_targetre-export of opensmartroute.adapters.personas.persona_targetBuild a persona RouteTarget whose instructions is the system prompt (non-primary by default).
personas_from_recordsre-export of opensmartroute.adapters.personas.personas_from_recordsPersona targets from JSON / CSV-style records (name + prompt/system keys); others skipped.
quality_from_benchmarksre-export of opensmartroute.adapters.catalogue.quality_from_benchmarksMean of normalised benchmark scores (percentages are divided by 100); None when there are none.
quality_from_popularityre-export of opensmartroute.adapters.catalogue.quality_from_popularityWeak prior in [0.35, 0.75] from log-scaled downloads and likes (popularity is not quality; it is a hint).
queue_handlerre-export of opensmartroute.adapters.handlers.queue_handlerRouteTarget.handler that enqueues the request and returns a :class:PendingResult.
recommend_serversre-export of opensmartroute.adapters.mcp_servers.recommend_serversRank MCP servers for a task; constraint violations are excluded, not down-weighted.
route_and_executere-export of opensmartroute.adapters.frameworks.route_and_executeRoute text and optionally execute the plan; returns (decision, ExecutionResult | None).
sentence_transformers_embedderre-export of opensmartroute.adapters.optional.sentence_transformers_embedderSemantic embedder for SimilarityStrategy. Requires opensmartroute[embeddings].
sign_manifestre-export of opensmartroute.adapters.mcp.sign_manifestWrap a tool list in a signed manifest. algorithm = hmac-sha256 (key = shared.
skill_from_markdownre-export of opensmartroute.adapters.skills.skill_from_markdownParse one SKILL.md (Agent-Skills front matter + body) into a skill RouteTarget; strict validation.
skills_from_cardre-export of opensmartroute.adapters.a2a.skills_from_cardOne non-primary skill target per skill declared on an A2A agent card (<agent>/<skill>).
tools_from_manifestre-export of opensmartroute.adapters.mcp.tools_from_manifestImport tools from a (signed) manifest; kw goes to :func:tools_from_mcp.
tools_from_mcpre-export of opensmartroute.adapters.mcp.tools_from_mcpConvert tools/list output into targets. call(name, arguments) becomes the handler.
verify_manifestre-export of opensmartroute.adapters.mcp.verify_manifestConstant-time verification. key = shared secret (HMAC) or 32-byte Ed25519 public key.

opensmartroute.adapters.a2a#

Source: src/opensmartroute/adapters/a2a.py

Import A2A (Agent-to-Agent protocol) Agent Cards as TargetKind.AGENT targets.

NameKindSummary
a2a_handlerfunction (url: str, token: str | None=None, timeout_s: float=60.0, method: str='message/send')Return a handler that sends the request text to an A2A agent and returns its text.
agent_from_cardfunction (card: dict[str, Any], *, call: Callable[[str, RouteRequest], Any] | None=None, cost_per_1k_tokens: float=0.0, latency_ms: float=2000.0, quality_prior: float=0.65, languages: list[str] | None=None)Build an agent RouteTarget from an A2A agent card (name, skills, tags, input modes).
fetch_agent_cardfunction (base_url: str, timeout_s: float=10.0, path: str='/.well-known/agent.json')Download an agent card over HTTPS (plain HTTP is refused); raises TargetUnavailableError on failure.
skills_from_cardfunction (card: dict[str, Any], agent_id: str | None=None)One non-primary skill target per skill declared on an A2A agent card (<agent>/<skill>).

opensmartroute.adapters.catalogue#

Source: src/opensmartroute/adapters/catalogue.py

Live model catalogue: collect model cards (price, context, modalities, benchmarks) from public sources.

NameKindSummary
LEADERBOARD_DATASETconstantOpen LLM Leaderboard v2 results table on the Hub.
OPENROUTER_MODELSconstantpublic model + pricing listing (no key).
ModelCardclassWhat the catalogue knows about one model: identity, price, limits, evidence of quality, provenance.
ModelCatalogueclassMerged, persisted model cards from every source; the SLM's view of the target universe.
card_to_targetfunction (card: ModelCard, *, default_latency_ms: float=1500.0, settings: Settings | None=None)A :class:RouteTarget for a model card; risky third-party descriptions are replaced by the name.
fetch_huggingface_modelsfunction (query: str='', *, limit: int=50, pipeline: str='text-generation', timeout_s: float | None=None, settings: Settings | None=None)Model cards from the Hugging Face Hub search (downloads, likes, tags, model-index benchmarks).
fetch_leaderboard_qualityfunction (*, limit: int=5000, dataset: str=LEADERBOARD_DATASET, timeout_s: float | None=None, settings: Settings | None=None){hub model id: {benchmark: accuracy}} from the Open LLM Leaderboard table (official rows, unflagged).
fetch_openrouter_modelsfunction (*, url: str=OPENROUTER_MODELS, timeout_s: float | None=None, settings: Settings | None=None)Model cards from OpenRouter's public listing (prices per token, context, modalities, tool support).
model_keyfunction (model_id: str)Vendor-agnostic key for matching model names across sources: lowercase, no vendor prefix, no punctuation.
quality_from_benchmarksfunction (benchmarks: dict[str, float])Mean of normalised benchmark scores (percentages are divided by 100); None when there are none.
quality_from_popularityfunction (downloads: int, likes: int)Weak prior in [0.35, 0.75] from log-scaled downloads and likes (popularity is not quality; it is a hint).

opensmartroute.adapters.frameworks#

Source: src/opensmartroute/adapters/frameworks.py

Drop-in nodes for agent frameworks.

NameKindSummary
langgraph_conditionfunction (by: str='target', default: str='__end__')Edge selector for add_conditional_edges: routes on target id (by='target') or kind.
langgraph_nodefunction (router: Router, *, execute: bool=False, plan: bool=False, context_key: str='route_context', on_no_route: str='__no_route__')Return a LangGraph-compatible node state -> dict (partial state update).
last_user_textfunction (state: dict[str, Any])(last user utterance, prior history as role/content dicts).
maf_router_executorfunction (router: Router, *, execute: bool=False)Agent Framework style: executor(message, ctx) -> target id | response; handoffs maps.
openai_tool_handlerfunction (router: Router)Callable behind openai_tool_spec: (text, objective?, kinds?) -> RouteDecision.to_dict().
openai_tool_specfunction (name: str='route_request')OpenAI function-calling tool definition that lets a model ask the router for a target.
route_and_executefunction (router: Router, text: str, *, history: list[dict[str, str]] | None=None, context: dict[str, Any] | None=None, objective: Objective | None=None, execute: bool=False, plan: bool=False, learn: bool=True)Route text and optionally execute the plan; returns (decision, ExecutionResult | None).

opensmartroute.adapters.handlers#

Source: src/opensmartroute/adapters/handlers.py

Executors for non-LLM targets: HTTP endpoints, MCP tools and asynchronous queues.

NameKindSummary
InMemoryQueueclassReference :class:Queue: FIFO in memory, resolves into :class:~opensmartroute.Outcome.
PendingResultclassImmediate answer of a queued target: the request was accepted and will be answered later.
QueueclassStructural interface of an asynchronous queue target (ticketing, human tier, workflow run).
QueuedItemclassOne request waiting in (or resolved from) a queue.
http_handlerfunction (url: str, *, api_key: str | None=None, api_key_env: str | None=None, timeout_s: float=300.0, headers: Mapping[str, str] | None=None, payload: Callable[[str, Mapping[str, Any], list[dict[str, str]]], dict[str, Any]] | None=None, body: Mapping[str, Any] | None=None)RouteTarget.handler that POSTs the request to url and returns a :class:HarnessResult.
mcp_tool_handlerfunction (call: Callable[[str, dict[str, Any]], Any], tool: str, input_schema: dict[str, Any] | None=None, arguments: Callable[[RouteRequest], dict[str, Any]] | None=None)RouteTarget.handler that invokes one MCP tool via call(name, arguments).
queue_handlerfunction (queue: Queue, target_id: str | None=None)RouteTarget.handler that enqueues the request and returns a :class:PendingResult.

opensmartroute.adapters.harness#

Source: src/opensmartroute/adapters/harness.py

Agent-harness adapters: route to a runtime, not just a model.

NameKindSummary
AgentHarnessclassStructural interface of an agent runtime: run(task, context=, history=) -> HarnessResult.
CallableHarnessclassWrap an in-process agent: fn(task, context, history) -> str | dict | HarnessResult.
HTTPHarnessclassPOST the task as JSON to an agent endpoint.
HarnessResultclassWhat an agent harness returns: text, success, token / cost / latency usage and optional self-graded quality.
SubprocessHarnessclassRun a CLI agent: task on stdin, answer on stdout, exit code 0 = success.
coerce_resultfunction (out: Any)Normalise str | dict | HarnessResult | object-with-text into a HarnessResult.
harness_handlerfunction (harness: AgentHarness)Adapter for RouteTarget.handler: a RouteRequest becomes a harness task.

opensmartroute.adapters.mcp#

Source: src/opensmartroute/adapters/mcp.py

Import MCP (Model Context Protocol) tools as TargetKind.TOOL targets.

NameKindSummary
MCPToolclassA tool as listed by an MCP server: name, description, JSON input schema and annotations.
StdioMCPClientclassTiny JSON-RPC-over-stdio MCP client (newline-delimited). Thread-safe, blocking.
canonical_jsonfunction (obj: Any)Deterministic JSON encoding (sorted keys, no whitespace) used for manifest digests and signatures.
connect_mcpfunction (command: list[str], server: str='', **kw: Any)Spawn a stdio MCP server, list its tools and return (client, targets).
enrich_descriptionfunction (name: str, description: str, input_schema: dict[str, Any] | None, server: str='', enricher: Callable[[str], list[str]] | None=None)Return (routing_description, examples, capabilities) for a tool.
manifest_digestfunction (tools: list[dict[str, Any]])SHA-256 hex digest of a tool list in canonical JSON.
manifest_from_targetsfunction (targets: Iterable[RouteTarget])Reverse: dump MCP-shaped tool dicts (for signing / publishing a catalogue).
sign_manifestfunction (tools: list[dict[str, Any]], key: bytes, *, server: str='', algorithm: str='hmac-sha256', key_id: str='')Wrap a tool list in a signed manifest. algorithm = hmac-sha256 (key = shared.
tools_from_manifestfunction (manifest: dict[str, Any], key: bytes | None, *, require_signature: bool=True, max_age_s: float | None=None, **kw: Any)Import tools from a (signed) manifest; kw goes to :func:tools_from_mcp.
tools_from_mcpfunction (payload: Any, server: str='', call: Callable[[str, dict[str, Any]], Any] | None=None, enricher: Callable[[str], list[str]] | None=None, cost_per_call_usd: float=0.0, latency_ms: float=300.0, quality_prior: float=0.6, max_description_risk: float=0.6, guard: InputGuard | None=None)Convert tools/list output into targets. call(name, arguments) becomes the handler.
verify_manifestfunction (manifest: dict[str, Any], key: bytes, max_age_s: float | None=None)Constant-time verification. key = shared secret (HMAC) or 32-byte Ed25519 public key.

opensmartroute.adapters.mcp_servers#

Source: src/opensmartroute/adapters/mcp_servers.py

MCP server recommendation (MCP-Zero 2506.01056; ToolRet 2603.06467).

NameKindSummary
ServerCardclassDescription of an MCP server (tools, tags, auth, latency, cost, region, data boundary) for recommendation.
ServerRecommendationclassA ranked server from recommend_servers with its fused score, rationale and matched tool names.
recommend_serversfunction (request: RouteRequest | str, servers: Iterable[ServerCard | dict[str, Any]], k: int=3, constraints: RequestConstraints | None=None, allowed_auth: Iterable[str] | None=None, min_score: float=0.0)Rank MCP servers for a task; constraint violations are excluded, not down-weighted.

opensmartroute.adapters.openai_compat#

Source: src/opensmartroute/adapters/openai_compat.py

OpenAI-compatible HTTP client (stdlib only).

NameKindSummary
ChatResultclassResult of a chat completion: text, model, token counts, latency and the raw response.
OpenAICompatClientclassStdlib-only client for the OpenAI chat / embeddings API (OpenAI, Azure, vLLM, Ollama, LiteLLM) with retries.
attach_chat_handlersfunction (targets: Iterable[Any], client: OpenAICompatClient, *, model_key: str='model', default_model: str | None=None, overwrite: bool=False)Give every LLM target a :func:chat_handler on client; returns the ids that got one.
chat_handlerfunction (client: OpenAICompatClient, model: str, system_prompt: str | None=None, **defaults: Any)Adapter for RouteTarget.handler: turns a RouteRequest into a chat call.
embedderfunction (client: OpenAICompatClient, model: str, batch: int=64)Adapter for SimilarityStrategy(embedder=...).
judge_fnfunction (client: OpenAICompatClient, model: str, max_tokens: int=400)Adapter for LLMJudgeStrategy(llm=...): prompt in, completion text out.

opensmartroute.adapters.optional#

Source: src/opensmartroute/adapters/optional.py

Optional adapters that need extra dependencies. Everything is lazily imported so the.

NameKindSummary
OpenTelemetrySinkclassLive bridge from the tracer to OpenTelemetry: every OpenSmartRoute span becomes an OTel span.
OpenTelemetryTelemetryclassEmits one span per decision and counters/histograms via the OTel API.
sentence_transformers_embedderfunction (model_name: str='sentence-transformers/all-MiniLM-L6-v2', device: str | None=None, normalize: bool=True)Semantic embedder for SimilarityStrategy. Requires opensmartroute[embeddings].

opensmartroute.adapters.personas#

Source: src/opensmartroute/adapters/personas.py

Import persona catalogues as TargetKind.PERSONA targets.

NameKindSummary
load_personasfunction (path: str | Path, **kw: Any)Load personas from a directory of markdown files or a JSON/JSONL/CSV catalogue.
persona_from_markdownfunction (text: str, *, path: Path | None=None, **kw: Any)Parse a *.agent.md / *.chatmode.md / front-matter markdown file into a persona target.
persona_targetfunction (name: str, prompt: str, description: str='', *, id_prefix: str='persona:', domains: list[str] | None=None, tags: list[str] | None=None, languages: list[str] | None=None, primary: bool=False, quality_prior: float=0.6, source: str='', extra: dict[str, Any] | None=None)Build a persona RouteTarget whose instructions is the system prompt (non-primary by default).
personas_from_recordsfunction (records: Iterable[dict[str, Any]], **kw: Any)Persona targets from JSON / CSV-style records (name + prompt/system keys); others skipped.

opensmartroute.adapters.semantic_router#

Source: src/opensmartroute/adapters/semantic_router.py

Import a vLLM semantic-router configuration (vllm-project/semantic-router).

NameKindSummary
DEFAULT_DOMAIN_MAPconstantsemantic-router's MMLU-style categories -> OpenSmartRoute ontology domains.
SemanticRouterImportclassResult of importing a vLLM semantic-router config: registry, rules, default model, categories, warnings.
load_semantic_router_configfunction (source: str | Path | dict[str, Any], domain_map: dict[str, list[str]] | None=None, rule_weight: float=0.8)Convert a vLLM semantic-router model_config / categories document into targets and rules.

opensmartroute.adapters.skills#

Source: src/opensmartroute/adapters/skills.py

Load Agent-Skills SKILL.md packages as TargetKind.SKILL targets.

NameKindSummary
load_skillfunction (skill_dir: str | Path, **kw: Any)Load one skill directory (must contain SKILL.md).
load_skillsfunction (root: str | Path, **kw: Any)Load every */SKILL.md under root (one level deep, sorted by name).
parse_frontmatterfunction (text: str)Split --- frontmatter from the body. Returns (frontmatter, body).
skill_from_markdownfunction (text: str, *, path: Path | None=None, cost_per_1k_tokens: float=0.0)Parse one SKILL.md (Agent-Skills front matter + body) into a skill RouteTarget; strict validation.

opensmartroute.adapters.websearch#

Source: src/opensmartroute/adapters/websearch.py

Web knowledge for the self-improving router: stdlib HTTP fetch, search providers, page text.

NameKindSummary
BRAVE_APIconstantBrave web search (needs BRAVE_API_KEY).
DATASETS_SERVERconstantHugging Face datasets-server REST root (/rows).
DDG_APIconstantDuckDuckGo instant-answer JSON endpoint (no key).
HF_APIconstantHugging Face Hub REST root (models, datasets, search).
SearchHitclassOne search result: where it came from, what it says, and when it was seen.
SearchProviderconstant(query, limit) -> hits.
WebKnowledgeclassFan a query out to search providers, de-duplicate by URL and cache the hits as JSON.
brave_searchfunction (query: str, limit: int=10, *, api_key_env: str='BRAVE_API_KEY', timeout_s: float | None=None, settings: Settings | None=None)Brave Search API web results; the key is read from BRAVE_API_KEY (or BRAVE_API_KEY_FILE).
duckduckgo_searchfunction (query: str, limit: int=10, *, timeout_s: float | None=None, settings: Settings | None=None)DuckDuckGo instant-answer API (abstract + related topics). Keyless; shallow but good for definitions.
fetch_bytesfunction (url: str, *, timeout_s: float | None=None, max_bytes: int | None=None, headers: Mapping[str, str] | None=None, settings: Settings | None=None)GET url over https with a timeout and a body cap; transport errors become TargetUnavailableError.
fetch_jsonfunction (url: str, *, timeout_s: float | None=None, headers: Mapping[str, str] | None=None, settings: Settings | None=None)GET a JSON document (see :func:fetch_bytes); malformed bodies raise TargetUnavailableError.
fetch_page_textfunction (url: str, *, max_chars: int=20000, timeout_s: float | None=None, settings: Settings | None=None)Fetch a page and return {url, title, text, risk}; risk is the injection/gadget risk of the text.
html_to_textfunction (html: str)(title, text) of an HTML document with scripts/styles removed and whitespace collapsed.
huggingface_searchfunction (query: str, limit: int=10, *, what: str='models', pipeline_tag: str | None='text-generation', timeout_s: float | None=None, settings: Settings | None=None)Search the Hugging Face Hub (what = models or datasets), ranked by downloads. No key needed.

opensmartroute.aio#

Source: src/opensmartroute/aio.py

Async façade. Routing itself is CPU-bound and sub-millisecond, so we run it in.

NameKindSummary
AsyncRouterclassawait-able wrapper around a :class:Router: route / learn / execute / run off the event loop.

opensmartroute.branding#

Source: src/opensmartroute/branding.py

OpenSmartRoute naming conventions: one place for every brand-bound identifier.

NameKindSummary
API_KEY_ENVconstantOSR_API_KEY - access token the CLI sends (overrides the credentials file).
API_URL_ENVconstantOSR_API_URL - platform / server URL the CLI talks to.
BRANDconstantproduct name, one word.
CLIconstantconsole script name.
CONFIG_DIR_ENVconstantOSR_CONFIG_DIR - overrides the per-user config folder.
CONFIG_DIR_NAMEconstantper-user config folder name (~/.config/opensmartroute, %APPDATA%\opensmartroute).
ENTRY_POINT_GROUPconstantimportlib.metadata entry-point group for plugins.
ENV_PREFIXconstantOSR_ - every settings environment variable starts with this.
ERROR_CODE_PREFIXconstantOSR_ - machine-readable error codes (OSR_NO_ROUTE).
INSTALL_SCRIPT_PS1constantWindows installer: irm .../install.ps1 | iex.
INSTALL_SCRIPT_SHconstantLinux / macOS installer: curl -fsSL .../install.sh | sh.
LOCAL_TOKEN_PREFIXconstantself-hosted osr serve access tokens (osr_local_...).
METADATA_PREFIXconstantosr- - SKILL.md / persona front-matter keys (osr-domains).
PACKAGEconstantPython package / distribution / logger root.
REPOSITORYconstantsource repository (installer fallback URLs).
SHORT_NAMEconstantshort form used for env-var and error-code prefixes.
SKILLS_DIRconstant: Default Agent-Skills root (*/SKILL.md); the location Claude Code discovers project skills in.
STATE_DIRconstantdefault learner-state directory.
TOKEN_PREFIXconstanthosted-platform API keys (osr_live_...).
WEBSITEconstantpublic website; also the default hosted-platform URL of osr login.
env_keyfunction (*parts: str)env_key("routing", "softmax_temperature") -> "OSR_ROUTING_SOFTMAX_TEMPERATURE".
error_codefunction (kind: str)error_code("no_route") -> "OSR_NO_ROUTE".
loggerfunction (component: str | None=None)logger() -> "opensmartroute"; logger("enterprise") -> "opensmartroute.enterprise".
metadata_keyfunction (field: str)metadata_key("quality_prior") -> "osr-quality-prior" (SKILL.md / persona frontmatter).
platform_urlfunction (environ: Mapping[str, str] | None=None)The platform / server URL the CLI talks to: OSR_API_URL or the hosted platform (:data:WEBSITE).
user_agentfunction ()"opensmartroute/<version>" for outbound HTTP clients.
versionfunction ()The package version (opensmartroute.__version__), resolved at call time.

opensmartroute.cli#

Source: src/opensmartroute/cli.py

osr command-line interface.

NameKindSummary
build_routerfunction (targets: str, rules: str | None=None, state: str | None=None, models: str | None=None, skills: str | None=None, slm: str | None=None, remember_requests: int=0)Assemble the CLI's router from targets / rules / state / models / SKILL.md / SLM paths (every command).
mainfunction (argv: list[str] | None=None)Entry point of the osr console script; returns the process exit status.

opensmartroute.config#

Source: src/opensmartroute/config.py

Configuration loading: targets and rules from JSON or YAML.

NameKindSummary
load_documentfunction (path: str | Path)Read a JSON or YAML file (YAML needs the yaml extra); raises ConfigurationError on any problem.
load_rulesfunction (path: str | Path)Load a rules file (top-level list or rules: key) into a :class:RulesStrategy.
load_targetsfunction (path: str | Path)Load a catalogue file (top-level list or targets: key) into a :class:TargetRegistry.

opensmartroute.core#

Source: src/opensmartroute/core/init.py

NameKindSummary
Capabilitiesre-export of opensmartroute.core.types.CapabilitiesDeclarative description of what a target is good at.
Objectivere-export of opensmartroute.core.types.ObjectiveWhat the caller wants to optimise. Weights are relative.
Outcomere-export of opensmartroute.core.types.OutcomeFeedback about how a routed request actually went.
PlanSlotre-export of opensmartroute.core.types.PlanSlotOne filled slot of a multi-target plan (persona -> skill -> model).
RankedTargetre-export of opensmartroute.core.types.RankedTargetA scored candidate: utility, ensemble quality estimate and the per-strategy breakdown.
RequestConstraintsre-export of opensmartroute.core.types.RequestConstraintsHard constraints on the request (never traded off).
RouteDecisionre-export of opensmartroute.core.types.RouteDecisionThe answer to route(): chosen target, confidence, alternatives, optional plan, trace and propensities.
RoutePlanre-export of opensmartroute.core.types.RoutePlanA composed route (MasRouter-style): several targets working together.
RouteRequestre-export of opensmartroute.core.types.RouteRequestThe customer need.
RouteTargetre-export of opensmartroute.core.types.RouteTargetA routable destination: an LLM, agent, skill, persona, tool, workflow or human.
RouteTracere-export of opensmartroute.core.types.RouteTraceEverything needed to explain a decision.
Signalsre-export of opensmartroute.core.types.SignalsCheap deterministic features extracted from a request.
StrategyScorere-export of opensmartroute.core.types.StrategyScoreOne strategy's opinion about one target.
TargetConstraintsre-export of opensmartroute.core.types.TargetConstraintsWhere / for whom a target may be used. Checked by the policy layer.
TargetKindre-export of opensmartroute.core.types.TargetKindKinds of things a request can be routed to.
TargetRegistryre-export of opensmartroute.core.registry.TargetRegistryIn-memory catalogue of :class:RouteTarget by id: add / upsert / remove, filtered listing, (de)serialisation.

opensmartroute.core.registry#

Source: src/opensmartroute/core/registry.py

Target registry: the catalogue of everything a request may be routed to.

NameKindSummary
TargetRegistryclassIn-memory catalogue of :class:RouteTarget by id: add / upsert / remove, filtered listing, (de)serialisation.

opensmartroute.core.types#

Source: src/opensmartroute/core/types.py

Core data model for OpenSmartRoute.

NameKindSummary
TargetKindclassKinds of things a request can be routed to.
CapabilitiesclassDeclarative description of what a target is good at.
TargetConstraintsclassWhere / for whom a target may be used. Checked by the policy layer.
RouteTargetclassA routable destination: an LLM, agent, skill, persona, tool, workflow or human.
EFFORT_LEVELSconstantNamed reasoning-effort levels -> numeric effort in [0, 1] (RouteTarget.effort).
ObjectiveclassWhat the caller wants to optimise. Weights are relative.
RequestConstraintsclassHard constraints on the request (never traded off).
RouteRequestclassThe customer need.
SignalsclassCheap deterministic features extracted from a request.
StrategyScoreclassOne strategy's opinion about one target.
RankedTargetclassA scored candidate: utility, ensemble quality estimate and the per-strategy breakdown.
RouteTraceclassEverything needed to explain a decision.
PlanSlotclassOne filled slot of a multi-target plan (persona -> skill -> model).
RoutePlanclassA composed route (MasRouter-style): several targets working together.
RouteDecisionclassThe answer to route(): chosen target, confidence, alternatives, optional plan, trace and propensities.
OutcomeclassFeedback about how a routed request actually went.

opensmartroute.credentials#

Source: src/opensmartroute/credentials.py

Credentials for the osr CLI: where the access token lives and how it is obtained.

NameKindSummary
CREDENTIALS_FILEconstantfile name inside :func:config_dir.
DEFAULT_PROFILEconstantprofile used when --profile is not given.
DEVICE_CODE_PATHconstantRFC 8628 device-authorization endpoint of the platform.
DEVICE_TOKEN_PATHconstantnoqa: S105 # nosec B105 - RFC 8628 token endpoint (a URL path).
ME_PATHconstantplatform: who am I (workspace, plan, edition).
TOKEN_BYTESconstantentropy of :func:generate_token.
WHOAMI_PATHconstantself-hosted osr serve: is this token accepted.
CredentialclassOne saved sign-in: where (url), what (token) and what the platform said about it.
CredentialStoreclassProfiles in <config dir>/credentials.json (owner-only permissions on POSIX).
PlatformClientclassMinimal JSON client for the platform / server API used by the CLI (stdlib only, injectable transport).
Transportconstant: (method, url, headers, body, timeout) -> (status, json); tests inject a fake instead of urllib.
apply_identityfunction (cred: Credential, info: Mapping[str, Any])Copy what :func:whoami learned (kind, workspace, plan, edition) onto cred.
config_dirfunction (environ: Mapping[str, str] | None=None)Per-user configuration directory (OSR_CONFIG_DIR > %APPDATA% > $XDG_CONFIG_HOME > ~/.config).
device_loginfunction (url: str | None=None, *, client_name: str | None=None, open_browser: bool=True, out: Callable[[str], None]=print, transport: Transport | None=None, sleep: Callable[[float], None]=time.sleep, timeout_s: float | None=None)Sign in to the hosted platform with the device authorization grant and return the credential.
generate_tokenfunction (prefix: str=LOCAL_TOKEN_PREFIX)A fresh random access token (osr_local_<43 chars>) for a self-hosted osr serve.
redactfunction (token: str, keep: int=6)osr_live_abc123... - the prefix plus a few characters, never the whole token.
token_kindfunction (token: str)"platform" for osr_live_ keys, "server" for osr_local_ tokens, else "unknown".
whoamifunction (cred: Credential, *, transport: Transport | None=None, timeout: float=30.0)Validate cred against its server and describe the identity behind it.

opensmartroute.discovery#

Source: src/opensmartroute/discovery.py

Tool discovery beyond text similarity.

NameKindSummary
CachePreservingSelectorclassKeep the serialized tool list prefix-stable across turns of a session.
SchemaAwareStrategyclassScores tools by how many of their input_schema parameters the request can fill (see schema_match).
SkillGraphclassDependency / conflict / composition graph over skill targets.
extract_entitiesfunction (text: str)Typed entity mentions found in text (kind -> values).
schema_matchfunction (request: RouteRequest | str, target: RouteTarget)Return (coverage, filled, missing) for the target's input schema.

opensmartroute.enterprise#

Source: src/opensmartroute/enterprise/init.py

Enterprise integration layer: ports (hexagonal architecture), middleware and telemetry.

NameKindSummary
AuditSinkclassTamper-evident audit trail port (hash-chained).
CacheMiddlewareclassLRU decision cache keyed on (text, constraints, objective, route options). TTL in seconds.
EnterpriseRouterclassRouter + middleware chain + telemetry + auto-learning + audit. Thread-safe façade.
FileAuditSinkclassAppend-only JSONL, each line {prev, hash, record} with.
FileStateStoreclassOne JSON file per key under root; keys are hashed so they can't traverse paths.
InMemoryStateStoreclassThread-safe dict-backed :class:StateStore; values are deep-copied through JSON on read and write.
LoggingTelemetryclassStructured JSON logs; never logs raw request text (only a hash + length).
MetricsTelemetryclassIn-process counters, outcome tallies and a latency histogram.
MiddlewareclassChain-of-responsibility hook around routing: __call__(request, next_) -> RouteDecision.
RouteFnconstantthe "next" callable a middleware wraps.
RouterBuilderclassFluent builder that validates configuration and wires all enterprise pieces.
SavingsEntryre-export of opensmartroute.enterprise.savings.SavingsEntryOne routed request in the ledger.
SavingsLedgerre-export of opensmartroute.enterprise.savings.SavingsLedgerTelemetry sink that keeps a per-request baseline-vs-routed cost ledger (see module docs).
SavingsReportre-export of opensmartroute.enterprise.savings.SavingsReportAggregate savings and the quality they were bought at.
StateStoreclassKey/value persistence port for learner state, caches, breaker state.
TelemetryclassObserver port with no-op defaults. Implement for OpenTelemetry, Prometheus, Datadog….
TenantMiddlewareclassEnforces that a tenant is present and applies per-tenant defaults/limits.
TimeoutMiddlewareclassSoft deadline: raise if routing itself exceeded budget_ms (should never happen.

opensmartroute.enterprise.ops#

Source: src/opensmartroute/enterprise/ops.py

Operational controls: shadow / A-B routing, tenant fairness and queue-aware latency.

NameKindSummary
SPRTclassWald SPRT for two Bernoulli success rates: H0 p=p0 vs H1 p=p0+delta.
ABTestclassTraffic split + SPRT comparison of candidate vs control outcomes.
FairShareMiddlewareclassDominant Resource Fairness across tenants over a sliding window.
InflightTrackerclassPer-target in-flight counters + arrival/service statistics for queueing estimates.
QueueAwareStrategyclassScores targets by current time-to-first-token = catalogue latency + queueing wait vs.
ShadowMiddlewareclassRun candidate beside production. mode='shadow': log only. mode='ab': serve.
TenantUsageclassSliding-window ledger of one tenant's requests, cost and tokens for :class:FairShareMiddleware.

opensmartroute.enterprise.savings#

Source: src/opensmartroute/enterprise/savings.py

Savings ledger - the always-on savings report that backs the ROI story and the dashboard.

NameKindSummary
SavingsEntryclassOne routed request in the ledger.
SavingsLedgerclassTelemetry sink that keeps a per-request baseline-vs-routed cost ledger (see module docs).
SavingsReportclassAggregate savings and the quality they were bought at.

opensmartroute.enterprise.stores#

Source: src/opensmartroute/enterprise/stores.py

Production state-store backends and wrappers.

NameKindSummary
BatchedStateStoreclassWrite-behind buffer. Reads are served from the pending map first.
EncryptedStateStoreclassAES-256-GCM envelope encryption. key is 32 raw bytes, or read (base64/hex/raw).
Migrationconstantstate-document transform applied by VersionedStateStore.
NamespacedStateStoreclassPrefix every key with <namespace>/ so several routers or tenants can share one backing store.
RedisStateStoreclassStateStore on any redis-py-compatible client (get / set / delete), JSON values, optional TTL.
SQLStateStoreclassDB-API 2.0 key/value store (PostgreSQL, SQLite, MySQL).
VersionedStateStoreclassSchema-versioned envelope with forward migrations.

opensmartroute.errors#

Source: src/opensmartroute/errors.py

Exception hierarchy. Every error raised by the SDK derives from :class:OpenSmartRouteError.

NameKindSummary
OpenSmartRouteErrorclassBase class for all SDK errors.
ConfigurationErrorclassInvalid catalogue, rules, SKILL.md, settings or a missing optional dependency.
ValidationErrorclassA request, outcome or target failed validation.
NoRouteErrorclassNo target satisfied the hard constraints.
TargetUnavailableErrorclassA remote target or catalogue source could not be reached.
ExecutionErrorclassA target handler failed while executing a plan.
SecurityErrorclassRequest rejected by an input guard (prompt injection, oversize, etc.).
StateStoreErrorclassA learner-state store failed to load, save or migrate.
AuthenticationErrorclassThe CLI has no valid access token for the platform / server, or a sign-in was denied or timed out.
OpenSmartRouteDeprecationWarningclassEmitted by :func:deprecated; filter with warnings.simplefilter on this class.
deprecatedfunction (name: str, *, since: str, removal: str, replacement: str | None=None, stacklevel: int=3)Announce a deprecation according to the policy in CONTRIBUTING.md.

opensmartroute.estimate#

Source: src/opensmartroute/estimate.py

Token, cost and latency estimates before a request is sent anywhere.

NameKindSummary
DEFAULT_OUTPUT_TOKENSconstant: Output length assumed when neither the caller nor the signals say (a short answer).
MESSAGE_OVERHEAD_TOKENSconstant: Tokens a chat API adds per message for role / separators (OpenAI-style framing).
PriceHookconstant: (usd_per_1k_input, usd_per_1k_output) for a target, or None to fall back to its declared cost.
RequestEstimateclassA quote for one request across every candidate, with named picks.
TargetEstimateclassThe quote for one candidate target.
estimatefunction (router: Router, request: RouteRequest | str, *, output_tokens: int | None=None, kinds: list[str] | None=None, prices: PriceHook | None=None, quality_tolerance: float=0.1, decision: RouteDecision | None=None)Quote request against every candidate the router would consider - nothing is executed.
estimate_messages_tokensfunction (messages: list[dict[str, Any]])Token estimate for an OpenAI-style message list (content plus per-message framing).
estimate_tokensfunction (text: str)Approximate the tokenizer count of text without any tokenizer library.
target_pricesfunction (target: RouteTarget)(usd per 1k input tokens, usd per 1k output tokens) from the target's declared cost.

opensmartroute.eval#

Source: src/opensmartroute/eval/init.py

RouterBench-style evaluation harness.

NameKindSummary
AuditReportre-export of opensmartroute.eval.audit.AuditReportAggregate of a shadow replay; to_markdown renders the Routing Audit deliverable.
AuditRowre-export of opensmartroute.eval.audit.AuditRowOne logged request: what happened, and what the router would have done.
DatasetCollectorre-export of opensmartroute.eval.collect.DatasetCollectorCache-backed corpus builder: collect sources, add feedback / synthetic rows, dedupe, split.
DatasetSourcere-export of opensmartroute.eval.collect.DatasetSourceOne Hub dataset split to collect: repo id, config, split, the preset that parses it, an optional model map.
EvalResultclassAggregate routing metrics over a dataset: accuracy, cost, latency, confidence, calibration, coverage.
EvalRowclassOne labelled prompt: expected / acceptable targets or per-target quality scores (RouterBench style).
area_under_frontierfunction (points: list[dict[str, float]])Trapezoidal area under accuracy(cost) — analogous to RouterBench's AIQ.
calibration_reportfunction (router: Router, rows: list[EvalRow], objective: Objective | None=None)Confidence calibration of the router on labelled rows: ECE, Brier, reliability bins and.
collect_datasetre-export of opensmartroute.eval.collect.collect_datasetDownload source from the Hub and parse it into rows.
cost_quality_frontierfunction (router: Router, rows: list[EvalRow], cost_weights: list[float] | None=None)Sweep the cost weight to trace the accuracy-vs-cost curve (RouterBench Fig. 1 style).
evaluatefunction (router: Router, rows: list[EvalRow], objective: Objective | None=None)Route every row and score accuracy, realised quality, cost, latency, ECE / Brier and conformal coverage.
fetch_hf_rowsre-export of opensmartroute.eval.collect.fetch_hf_rowsOne page (max 100) of records from the datasets-server /rows endpoint, flattened to {column: value}.
load_audit_logre-export of opensmartroute.eval.audit.load_audit_logRead a JSONL traffic log (text or prompt/messages per line).
load_datasetfunction (path: str | Path)Read a JSONL evaluation dataset (text or prompt plus the optional EvalRow keys); a line that is.
model_qualityre-export of opensmartroute.eval.collect.model_qualityData-derived quality prior per model in [0, 1]: pairwise rows (two scored models) are fitted with.
routing_auditre-export of opensmartroute.eval.audit.routing_auditReplay rows through router in shadow mode and aggregate an :class:AuditReport.
rows_from_feedbackre-export of opensmartroute.eval.collect.rows_from_feedbackRows from :class:Outcome records. texts maps request_id to the prompt (outcomes carry no text);.
summarize_auditre-export of opensmartroute.eval.audit.summarize_auditAggregate :class:AuditRow records into an :class:AuditReport.
synthetic_rowsre-export of opensmartroute.eval.collect.synthetic_rowsOntology seed prompts labelled with the best capability-fit target (cold-start supervision).

opensmartroute.eval.agentic#

Source: src/opensmartroute/eval/agentic.py

tau-bench-style agentic task evaluation: does per-step routing beat the best single agent?.

NameKindSummary
AgentTaskclassA multi-step task with the measured per-step success probability and latency of each agent.
load_agentic_tasksfunction (path: str | Path)Read tasks from JSONL: {"task_id", "steps", "success", "latency_ms", "cost_usd"?, "domain"?}.
synthetic_agentic_tasksfunction (n: int=200, *, seed: int=0, hard_share: float=0.3)Three agents (fast / balanced / strong) and n tasks of 3-5 steps whose steps are mostly.
task_routing_frontierfunction (registry: TargetRegistry, tasks: Sequence[AgentTask], *, retries: int=1, accuracy_tolerance: float=0.02, latency_ratio_target: float=0.9, objective: Objective | None=None, router_factory: Callable[[TargetRegistry], Router] | None=None, seed: int=0)Replay tasks step by step through :class:ProgressRouter (each agent's handler draws the.

opensmartroute.eval.audit#

Source: src/opensmartroute/eval/audit.py

Routing Audit - shadow-mode replay of logged LLM traffic to quantify what the router would change.

NameKindSummary
DEFAULT_COMPLETION_TOKENSconstant: Default assumed output length when a log row carries no completion_tokens.
AuditReportclassAggregate of a shadow replay; to_markdown renders the Routing Audit deliverable.
AuditRowclassOne logged request: what happened, and what the router would have done.
load_audit_logfunction (path: str | Path)Read a JSONL traffic log (text or prompt/messages per line).
routing_auditfunction (router: Router, rows: list[dict[str, Any]], *, baseline: str | None=None, monthly_requests: int | None=None, plan: bool=False, keep_rows: bool=True)Replay rows through router in shadow mode and aggregate an :class:AuditReport.
summarize_auditfunction (rows: list[AuditRow], *, monthly_requests: int | None=None, keep_rows: bool=True)Aggregate :class:AuditRow records into an :class:AuditReport.

opensmartroute.eval.baselines#

Source: src/opensmartroute/eval/baselines.py

Baselines every router must beat, plus oracle ceilings and the sampling noise floor.

NameKindSummary
Policyconstanta baseline: (row, targets) -> chosen target id.
PolicyResultclassMetrics of one baseline policy on a dataset (same fields as the router's EvalResult headline).
baseline_suitefunction (rows: list[EvalRow], targets: Sequence[RouteTarget], train_rows: list[EvalRow] | None=None, quality_floor: float=0.7, seed: int=0)Run every baseline and both oracles; train_rows (default: rows) fit the task table.
best_prior_policyfunction (row: EvalRow, targets: Sequence[RouteTarget])Always the target with the highest declared quality_prior.
cheapest_policyfunction (row: EvalRow, targets: Sequence[RouteTarget])Always the lowest unit cost (ties broken by latency).
evaluate_policyfunction (name: str, policy: Policy, rows: list[EvalRow], targets: Sequence[RouteTarget])Run a baseline :data:Policy over the rows and aggregate accuracy, quality, cost and latency.
fit_task_tablefunction (rows: list[EvalRow], targets: Sequence[RouteTarget])task_type -> target with the highest mean quality on the training rows.
most_expensive_policyfunction (row: EvalRow, targets: Sequence[RouteTarget])Always the highest unit cost - the "just use the frontier model" baseline.
multi_sample_oraclefunction (rows: list[EvalRow], targets: Sequence[RouteTarget])Mean quality of the oracle that picks by the mean over all samples per target.
noise_floorfunction (rows: list[EvalRow], targets: Sequence[RouteTarget], seed: int=0)How much accuracy is lost to sampling noise alone.
oracle_policyfunction (quality_floor: float | None=None)Per-row best target. With quality_floor -> cheapest target reaching the floor.
random_policyfunction (seed: int=0)Uniformly random target (seeded).
static_task_table_policyfunction (table: dict[str, str])Look the row's task_type up in a table from :func:fit_task_table ("*" = default).

opensmartroute.eval.collect#

Source: src/opensmartroute/eval/collect.py

Collect routing datasets from the Hugging Face Hub, your own feedback log and synthetic seeds.

NameKindSummary
ARENA_TIERSconstanteverything else in those datasets is a 7B-14B chat model -> "small".
DATASETS_SERVERre-export of opensmartroute.adapters.websearch.DATASETS_SERVERHugging Face datasets-server REST root (/rows).
DEFAULT_SOURCESconstantpublic ones.
HISTORY_FILEconstantthe self-improver's report log; lives in the cache dir but is not a dataset.
KNOWN_SOURCESconstantpairwise human / judge preference battles - the winner is the label.
PAIRWISEconstantpreset name for battle datasets: prompt / conversation, model_a, model_b, winner.
PARSERSconstantcollector presets that are not wide / nested tables.
REWARD_BENCHconstantpreset name for RewardBench: prompt, chosen_model, rejected_model, subset.
ROUTELLM_GOOD_ENOUGHconstantRouteLLM's threshold: a weak-model score >= 4 means the cheap model was good enough.
ROUTELLM_GPT4constantpreset name for RouteLLM's gpt4_dataset: prompt + GPT-4-judged Mixtral score 1-5.
ROUTELLM_STRONGconstantthe strong model of RouteLLM's gpt4_dataset (its answers are the reference).
ROUTELLM_WEAKconstantthe weak model whose answer GPT-4 scores 1-5 against the reference.
ULTRAFEEDBACKconstantpreset name for UltraFeedback: instruction + N completions with a model and score.
DatasetCollectorclassCache-backed corpus builder: collect sources, add feedback / synthetic rows, dedupe, split.
DatasetSourceclassOne Hub dataset split to collect: repo id, config, split, the preset that parses it, an optional model map.
Parserconstantrecord -> row (or None to skip).
collect_datasetfunction (source: DatasetSource, *, timeout_s: float | None=None, settings: Settings | None=None)Download source from the Hub and parse it into rows.
fetch_hf_rowsfunction (dataset: str, *, config: str='default', split: str='train', offset: int=0, length: int=100, timeout_s: float | None=None, settings: Settings | None=None)One page (max 100) of records from the datasets-server /rows endpoint, flattened to {column: value}.
from_pairwise_rowfunction (rec: dict[str, Any], model_map: dict[str, str] | None=None, source: DatasetSource | None=None)A battle record -> row with scores 1 / 0 for winner / loser (0.5 each on a tie). Understands the.
from_reward_bench_rowfunction (rec: dict[str, Any], source: DatasetSource | None=None)A RewardBench record (prompt, chosen_model, rejected_model, subset) -> a pairwise row.
from_routellm_gpt4_rowfunction (rec: dict[str, Any], source: DatasetSource | None=None)A routellm/gpt4_dataset record (prompt + mixtral_score 1-5) -> the strong model scores 1.0 and.
from_ultrafeedback_rowfunction (rec: dict[str, Any], source: DatasetSource | None=None)An UltraFeedback record (instruction + completions[{model, overall_score}]) -> a scored row with.
iter_hf_rowsfunction (source: DatasetSource, *, offset: int=0, timeout_s: float | None=None, settings: Settings | None=None)Page through source from raw record offset until source.limit records or the split is exhausted.
model_qualityfunction (rows: Iterable[EvalRow], *, min_n: int=20, epochs: int=5, seed: int=0)Data-derived quality prior per model in [0, 1]: pairwise rows (two scored models) are fitted with.
rows_from_feedbackfunction (outcomes: Iterable[Outcome] | FeedbackStore, texts: dict[str, str] | None=None, *, min_quality: float=0.5)Rows from :class:Outcome records. texts maps request_id to the prompt (outcomes carry no text);.
rows_from_recordsfunction (records: Iterable[dict[str, Any]], source: DatasetSource)Parse raw records with the source's preset (text_key is tried first when set).
synthetic_rowsfunction (targets: Sequence[RouteTarget], *, per_template: int=2, seed: int=0)Ontology seed prompts labelled with the best capability-fit target (cold-start supervision).
tier_model_mapfunction (tiers: Mapping[str, str], models: Iterable[str]=())model_map for a :class:DatasetSource: every known battle model (plus models) -> the target id.
tier_offunction (model: str)Tier of a battle-dataset model name: the :data:ARENA_TIERS entry, else the name rules, else the parameter.

opensmartroute.eval.criteria#

Source: src/opensmartroute/eval/criteria.py

Offline realisations of the ROADMAP exit criteria.

NameKindSummary
CriterionResultclassOne measured exit criterion.
bootstrap_cifunction (values: Sequence[float], *, n_boot: int=1000, level: float=0.95, seed: int=0)Percentile bootstrap confidence interval for the mean of values.
cold_start_ratiofunction (n_outcomes: int=200, *, domain: str='legal', n_eval: int=200, seed: int=0, target_ratio: float=0.9, generalist_quality: float=0.6, explore_rate: float=0.1, max_requests: int=20000)Accuracy on domain prompts of a target declared with only id / kind / cost.
conformal_coveragefunction (alpha: float=0.1, *, n_cal: int=1000, n_test: int=1000, label_noise: float=0.5, seed: int=0, tolerance: float=0.02)Fit :class:ConformalCalibrator on the real router's propensities over labelled prompts and.
domain_expert_registryfunction (domains: Sequence[str]=_DOMAINS, *, strip: str | None=None, generalist: bool=False)One hand-configured expert per domain. strip=<domain> replaces that expert with a.
effort_token_savingsfunction (registry: TargetRegistry, rows: Sequence[EvalRow], *, quality_tolerance: float=0.02, token_ratio_target: float=0.7, objective: Objective | None=None, router_factory: Callable[[TargetRegistry], Router] | None=None)Route rows that carry per-target scores and tokens and compare the tokens spent.
knapsack_never_exceeds_capfunction (steps: int=1000000, *, window: int=1000, drift_every: int=100000, seed: int=0)Drive :class:MultiKnapsackBandit (on_capped="abstain") for steps pulls with.
match_at_1function (router: Router | EnterpriseRouter, prompts: Sequence[tuple[str, str]])Fraction of (text, expected_target_id) pairs the router gets right at rank 1.
match_at_1_at_scalefunction (small: int=50, large: int=5000, *, n_prompts: int=200, seed: int=0, max_drop: float=0.05, narrow_above: int=32, narrow_to: int=24)Match@1 on a small catalogue versus a large one with retrieve-then-rank narrowing.
multi_round_vs_best_singlefunction (registry: TargetRegistry, rows: Sequence[EvalRow], *, threshold: float=0.8, max_rounds: int=3, failures_before_switch: int=1, objective: Objective | None=None, cost_ratio_target: float=0.6, router_factory: Callable[[TargetRegistry], Router] | None=None)Run :class:MultiRoundExecutor over rows that carry per-target scores (RouterBench.
ope_within_live_cifunction (n_log: int=2000, *, seed: int=0, logging_objective: Objective | None=None, target_objective: Objective | None=None, logging_temperature: float=0.3)Log decisions from a cost-seeking router (actions sampled from its propensities), estimate.
run_allfunction (*, quick: bool=True, seed: int=0)Every offline criterion. quick shrinks the expensive simulations (10^5 steps, 1 000 tools,.
synthetic_effort_rowsfunction (n: int=400, *, seed: int=0)One reasoning model exposed as two effort siblings (reasoner@fast with effort="low".
synthetic_scored_rowsfunction (n: int=300, *, seed: int=0)A three-tier catalogue (small / medium / large) and RouterBench-style rows with a quality.
synthetic_tool_cataloguefunction (n: int, *, seed: int=0)n distinct tools (verb x object x qualifier, up to 5 000 unique combinations) and one.

opensmartroute.eval.datasets#

Source: src/opensmartroute/eval/datasets.py

Adapters from public routing benchmarks to :class:EvalRow.

NameKindSummary
PRESETSconstantNamed layouts accepted by load_benchmark / osr eval --preset.
NestedPresetclassLayout of a "nested" benchmark: a per-model mapping under container_keys holding score / cost / samples.
WidePresetclassColumn layout of a "wide" benchmark: one row per prompt, <model><suffix> columns for scores / costs.
from_nested_rowfunction (row: dict[str, Any], preset: NestedPreset, model_map: dict[str, str] | None=None)Convert one nested-format record to an :class:EvalRow, keeping repeated samples when present.
from_wide_rowfunction (row: dict[str, Any], preset: WidePreset, model_map: dict[str, str] | None=None)Convert one wide-format record to an :class:EvalRow (None when it has no prompt or scores).
load_benchmarkfunction (path: str | Path, preset: str | WidePreset | NestedPreset='routerbench', model_map: dict[str, str] | None=None, limit: int | None=None)Load a benchmark file (.jsonl / .json / .csv) into :class:EvalRow objects.
models_infunction (rows: Iterable[EvalRow])Distinct target ids that carry scores in the rows, in first-seen order.

opensmartroute.eval.frontier#

Source: src/opensmartroute/eval/frontier.py

Three-objective frontier and ablations.

NameKindSummary
ablation_reportfunction (router: Router, rows: list[EvalRow], objective: Objective | None=None)Leave-one-out over strategies (and extractors, when explicitly set on the router).
frontier3function (router: Router, rows: list[EvalRow], cost_weights: list[float] | None=None, latency_weights: list[float] | None=None)Sweep cost / latency objective weights and return the Pareto-optimal (quality, cost, latency) points.
hypervolumefunction (points: list[dict[str, float]], ref: tuple[float, float, float] | None=None)Dominated hypervolume of the Pareto points in normalised (quality, cost, latency) space.

opensmartroute.eval.headroom#

Source: src/opensmartroute/eval/headroom.py

Routing headroom: when does routing pay, and how much catalogue does it need?.

NameKindSummary
DiversityReportclassHow differently the targets score rows: pairwise disagreement, winner entropy and winner share.
HeadroomReportclassOracle vs best-single quality on a dataset, with the label-noise floor that says whether the gap is real.
learnability_by_difficultyfunction (rows: Sequence[EvalRow], targets: Sequence[RouteTarget], buckets: int=4)Routing gain (oracle - best single) per difficulty bucket. Difficulty of a row is.
min_cataloguefunction (rows: Sequence[EvalRow], targets: Sequence[RouteTarget], fraction: float=0.95)Greedy forward selection: the smallest ordered subset whose oracle reaches fraction of the.
routing_headroomfunction (rows: Sequence[EvalRow], targets: Sequence[RouteTarget], seed: int=0)Oracle minus best-single quality with the label-noise floor for context.
scaling_curvefunction (rows: Sequence[EvalRow], targets: Sequence[RouteTarget], sizes: Sequence[int] | None=None, *, trials: int=20, seed: int=0)Mean oracle / best-single quality and headroom on random subsets of each size.
target_diversityfunction (rows: Sequence[EvalRow], targets: Sequence[RouteTarget])How different the targets are on this data: pairwise score disagreement and winner spread.

opensmartroute.eval.ope#

Source: src/opensmartroute/eval/ope.py

Off-policy evaluation from logged routing decisions.

NameKindSummary
LoggedDecisionclassOne logged routing event for off-policy evaluation: text, action taken, its propensity and observed reward.
OPEResultclassIPS, self-normalised IPS and doubly-robust value estimates with effective sample size and clip count.
ips_confidence_intervalfunction (logs: list[LoggedDecision], weights: list[float], z: float=1.96)Normal-approximation CI for the IPS estimate given per-sample weights.
mean_reward_modelfunction (logs: list[LoggedDecision])Simplest DR reward model: mean logged reward per action, global mean for unseen actions.
off_policy_evaluatefunction (router: Router, logs: list[LoggedDecision], objective: Objective | None=None, max_weight: float=20.0, reward_model: Callable[[LoggedDecision, str], float] | None=None, deterministic: bool=False)Estimate the value of router on logged traffic.
target_propensitiesfunction (router: Router, lg: LoggedDecision, objective: Objective | None=None)The evaluated router's P(target | x) for a logged context (empty when it has no route).

opensmartroute.eval.robustness#

Source: src/opensmartroute/eval/robustness.py

Robustness and fairness checks for a router.

NameKindSummary
coresetfunction (rows: list[EvalRow], k: int, embedder: Callable[[list[str]], list[list[float]]] | None=None, seed: int=0)Greedy k-center (farthest-first) subset of rows for a diverse evaluation set.
diversityfunction (rows: list[EvalRow], embedder: Callable[[list[str]], list[list[float]]] | None=None, sample: int=300)Mean pairwise cosine distance of the row texts (sampled), 0 = all identical.
paraphrase_robustnessfunction (router: Router, rows: list[EvalRow], n: int=4, objective: Objective | None=None, seed: int=0)Share of rule-based paraphrases that route to the same target as the original (robustness).
paraphrasesfunction (text: str, n: int=4, seed: int=0)n deterministic surface-level paraphrases of text (never returns the original).
profile_swap_fairnessfunction (router: Router, rows: list[EvalRow], profiles: Sequence[dict[str, Any]] | None=None, objective: Objective | None=None)Decision agreement across user profiles. dependence = fraction of rows whose.
repeat_flip_ratefunction (router: Router, rows: list[EvalRow], k: int=5, objective: Objective | None=None)Route each row k times; flip_rate = share of rows whose decision is not identical every time.

opensmartroute.execution#

Source: src/opensmartroute/execution.py

Plan-aware execution: turn a :class:RouteDecision into a real answer.

NameKindSummary
PRELUDE_ROLESconstantRoles executed before the primary target, in this order.
ExecutionResultclassWhat happened when a decision was executed.
ExecutionStepclassOne executed plan slot: role, target, latency and whether it succeeded.
aexecuteasync function (decision: RouteDecision, request: RouteRequest, learn: LearnFn | None=None, *, min_slot_confidence: float=0.0, task_id: str | None=None, **kw: Any)Async twin of :func:execute; awaits coroutine handlers.
executefunction (decision: RouteDecision, request: RouteRequest, learn: LearnFn | None=None, *, min_slot_confidence: float=0.0, task_id: str | None=None, **kw: Any)Run the plan (persona -> skill -> primary) and record outcomes via learn.

opensmartroute.feedback#

Source: src/opensmartroute/feedback/init.py

Feedback store: append-only outcome log that closes the learning loop.

NameKindSummary
FeedbackStoreclassAppend-only :class:Outcome log (in memory or JSONL file) with per-target statistics.

opensmartroute.learning#

Source: src/opensmartroute/learning/init.py

Auto-learning strategies built on :mod:opensmartroute.math.

NameKindSummary
DEFAULT_DOMAINSconstantDomain one-hot block of the LinUCB context vector (ontology domains + the "general" fallback).
AttentionEncoderre-export of opensmartroute.learning.attention.AttentionEncoderOne self-attention block with attention pooling; encodes text into a dim-vector (unnormalised).
AutoLearnerclassSingle entry point for closing the loop.
Autopilotre-export of opensmartroute.learning.autopilot.AutopilotRuns a :class:SelfImprover on a schedule and on drift, inside a live process.
ContrastiveRouterre-export of opensmartroute.learning.contrastive.ContrastiveRouterDual-encoder router trained with a contrastive or a distillation objective.
ContrastiveStrategyre-export of opensmartroute.learning.contrastive.ContrastiveStrategyScores candidates with a trained :class:ContrastiveRouter.
DriftMonitorre-export of opensmartroute.learning.autopilot.DriftMonitorPage-Hinkley over outcomes: alarms when the served success rate (or quality) drops for real.
EmbeddingFeaturizerre-export of opensmartroute.learning.embed.EmbeddingFeaturizerHashed features plus a frozen dense embedding, indexed above the hashed space (dim + i).
ExampleMinerre-export of opensmartroute.learning.coldstart.ExampleMinerPromote prompts a target handled well into that target's examples.
HandoffPolicyre-export of opensmartroute.learning.handoff.HandoffPolicyPermanent handoff of a task to fallback_target once the failure risk is too high.
HistoryTargetModelre-export of opensmartroute.learning.multiturn.HistoryTargetModelLogistic model over h * e_t with a shared weight vector and per-target biases.
HistoryTargetStrategyre-export of opensmartroute.learning.multiturn.HistoryTargetStrategyScores each target by the learned success probability given the conversation so far.
IRTStrategyclass2PL Item Response Theory: target ability vs (domain, complexity-bucket) item difficulty, learned online.
ImprovementReportre-export of opensmartroute.learning.self_improve.ImprovementReportWhat one cycle did: evidence gathered, catalogue changes, champion vs challenger, and the verdict.
LinUCBStrategyclassContextual bandit (LinUCB) on the signal vector; keeps the last context per request for the update.
MarkovStrategyclassPrefers targets that do well on the predicted next conversation state too.
MixtureCureModelre-export of opensmartroute.learning.handoff.MixtureCureModelWeibull mixture-cure model on cumulative risk with censoring.
PolicyGradientStrategyre-export of opensmartroute.learning.policy_gradient.PolicyGradientStrategyREINFORCE-trained softmax routing policy over hashed request features.
PreferenceStrategyclassBradley-Terry strengths per domain, fed by Outcome.preferred_over pairwise comparisons.
RegretReportre-export of opensmartroute.learning.policy_gradient.RegretReportDecision regret versus prediction error on a scored dataset.
RequestMemoryre-export of opensmartroute.learning.coldstart.RequestMemoryBounded LRU of request texts keyed by request id.
RouterSLMre-export of opensmartroute.learning.slm.RouterSLMSmall routing model: dual encoder + target catalogue snapshot + calibration, in one JSON file.
SLMReportre-export of opensmartroute.learning.slm.SLMReportHow an SLM did on a set of rows: accuracy, realised quality, cost, calibration and the training loss.
SLMStrategyre-export of opensmartroute.learning.slm.SLMStrategyEnsemble member backed by a :class:RouterSLM; scores are its probabilities, and it keeps learning online.
SelfImproverre-export of opensmartroute.learning.self_improve.SelfImproverClosed loop that keeps a :class:RouterSLM current with the model market and its own traffic.
SimilarityFallbackre-export of opensmartroute.learning.coldstart.SimilarityFallbackFor targets with no observations yet, rank by request<->description similarity.
SkillAffinityre-export of opensmartroute.learning.personal.SkillAffinityProfile-conditioned skill relevance: Beta posterior per (profile bucket, skill).
TaskCreditre-export of opensmartroute.learning.credit.TaskCreditBuffers per-step outcomes of a task and redistributes the final reward (uniform / discounted / last / blend).
TaskPinsre-export of opensmartroute.learning.credit.TaskPinsAdmission-time pinning: task_id -> target_id while the target keeps succeeding.
UserAdaptiveStrategyre-export of opensmartroute.learning.personal.UserAdaptiveStrategyPer-user Beta posteriors per target, shrunk toward similar users and the global posterior.
acceptable_setre-export of opensmartroute.learning.contrastive.acceptable_setTargets that are 'fine' for a row: expected + acceptable when labelled, otherwise every.
decision_regretre-export of opensmartroute.learning.policy_gradient.decision_regretMean decision regret of choose against the per-row oracle, next to the prediction error of.
decision_rewardre-export of opensmartroute.learning.policy_gradient.decision_rewardScalar decision reward of an outcome under objective (quality minus normalised cost/latency).
distill_routerre-export of opensmartroute.learning.slm.distill_routerCompress the full router into an SLM: route every text, take the ensemble's ranked utilities as soft.
history_vectorre-export of opensmartroute.learning.multiturn.history_vectorRecency-weighted joint embedding of the last turns messages and the current text.
load_embedderre-export of opensmartroute.learning.embed.load_embedderBuild the embedder a model file names: sentence-transformers/... (or any Hugging Face id) via.
merge_learnersfunction (local: Iterable[Strategy], remote: Iterable[Strategy])Federated merge: fold the evidence of remote strategies into the same-named local.
nearest_targetsre-export of opensmartroute.learning.coldstart.nearest_targetsThe k most similar existing targets (cosine + same-kind and domain-overlap bonuses) for warm starts.
outcome_countsre-export of opensmartroute.learning.coldstart.outcome_countsNumber of recorded outcomes per target id.
profile_bucketre-export of opensmartroute.learning.personal.profile_bucketCoarse profile bucket used to pool users with the same declared attributes.
profile_vectorre-export of opensmartroute.learning.personal.profile_vectorHashed one-hot encoding of key=value pairs (lists expand to one pair per element).
signal_vectorfunction (signals: Signals, domains: list[str] | None=None)Fixed-length numeric context for contextual bandits (dim = 8 + len(domains)).
soft_labelsre-export of opensmartroute.learning.contrastive.soft_labelsZooter soft labels: softmax(score / temperature) over the scored targets.
target_documentre-export of opensmartroute.learning.coldstart.target_documentText used to embed a target: name, description, domains, actions, tags and up to 12 examples.
target_embeddingre-export of opensmartroute.learning.coldstart.target_embeddingUnit-norm centroid of the target document and its examples (hashing embedder by default).
warm_startre-export of opensmartroute.learning.coldstart.warm_startSeed every learner with shrunk knowledge from the new target's nearest neighbours.
warm_start_from_matrixre-export of opensmartroute.learning.coldstart.warm_start_from_matrixOffline full-information reward-matrix warm start (OrcaRouter 2605.30736).

opensmartroute.learning.attention#

Source: src/opensmartroute/learning/attention.py

A pure-Python transformer block for the routing SLM's query encoder.

NameKindSummary
AttentionContextclassEverything :meth:AttentionEncoder.backward needs from one forward pass.
AttentionEncoderclassOne self-attention block with attention pooling; encodes text into a dim-vector (unnormalised).

opensmartroute.learning.autopilot#

Source: src/opensmartroute/learning/autopilot.py

Self-operation: the routing SLM runs its own improvement loop inside the live process.

NameKindSummary
AutopilotclassRuns a :class:SelfImprover on a schedule and on drift, inside a live process.
DriftMonitorclassPage-Hinkley over outcomes: alarms when the served success rate (or quality) drops for real.

opensmartroute.learning.coldstart#

Source: src/opensmartroute/learning/coldstart.py

Cold start for new targets and self-improving target descriptions.

NameKindSummary
ExampleMinerclassPromote prompts a target handled well into that target's examples.
RequestMemoryclassBounded LRU of request texts keyed by request id.
SimilarityFallbackclassFor targets with no observations yet, rank by request<->description similarity.
nearest_targetsfunction (new: RouteTarget, pool: Iterable[RouteTarget], k: int=3, embedder: Embedder | None=None)The k most similar existing targets (cosine + same-kind and domain-overlap bonuses) for warm starts.
outcome_countsfunction (feedback: FeedbackStore)Number of recorded outcomes per target id.
target_documentfunction (t: RouteTarget)Text used to embed a target: name, description, domains, actions, tags and up to 12 examples.
target_embeddingfunction (t: RouteTarget, embedder: Embedder | None=None)Unit-norm centroid of the target document and its examples (hashing embedder by default).
warm_startfunction (new: RouteTarget, registry: TargetRegistry, strategies: list[Strategy], k: int=3, shrink: float=0.5, embedder: Embedder | None=None)Seed every learner with shrunk knowledge from the new target's nearest neighbours.

opensmartroute.learning.contrastive#

Source: src/opensmartroute/learning/contrastive.py

Contrastive and reward-distilled router training (RouterDC, NeurIPS 2024; Zooter 2311.08692).

NameKindSummary
ContrastiveRouterclassDual-encoder router trained with a contrastive or a distillation objective.
ContrastiveStrategyclassScores candidates with a trained :class:ContrastiveRouter.
acceptable_setfunction (row: EvalRow, slack: float=0.05)Targets that are 'fine' for a row: expected + acceptable when labelled, otherwise every.
soft_labelsfunction (scores: dict[str, float], temperature: float=0.1)Zooter soft labels: softmax(score / temperature) over the scored targets.

opensmartroute.learning.credit#

Source: src/opensmartroute/learning/credit.py

Delayed, task-level credit assignment for agentic trajectories.

NameKindSummary
TaskCreditclassBuffers per-step outcomes of a task and redistributes the final reward (uniform / discounted / last / blend).
TaskPinsclassAdmission-time pinning: task_id -> target_id while the target keeps succeeding.

opensmartroute.learning.embed#

Source: src/opensmartroute/learning/embed.py

Pretrained transformer embeddings as frozen features for the routing SLM.

NameKindSummary
EmbeddingFeaturizerclassHashed features plus a frozen dense embedding, indexed above the hashed space (dim + i).
load_embedderfunction (name: str)Build the embedder a model file names: sentence-transformers/... (or any Hugging Face id) via.

opensmartroute.learning.handoff#

Source: src/opensmartroute/learning/handoff.py

Permanent-handoff policy from censored teacher signals (TACIT-Switch 2608.27911).

NameKindSummary
HandoffPolicyclassPermanent handoff of a task to fallback_target once the failure risk is too high.
MixtureCureModelclassWeibull mixture-cure model on cumulative risk with censoring.
TrajectoryclassPer-task state tracked by :class:HandoffPolicy: cumulative risk, steps, failure and hand-off flags.

opensmartroute.learning.multiturn#

Source: src/opensmartroute/learning/multiturn.py

Multi-turn routing with history-target joint embeddings (MTRouter 2604.23530).

NameKindSummary
HistoryTargetModelclassLogistic model over h * e_t with a shared weight vector and per-target biases.
HistoryTargetStrategyclassScores each target by the learned success probability given the conversation so far.
history_vectorfunction (request: RouteRequest, embedder: Embedder, turns: int=6, decay: float=0.7)Recency-weighted joint embedding of the last turns messages and the current text.

opensmartroute.learning.personal#

Source: src/opensmartroute/learning/personal.py

Few-shot personalisation (GMTRouter 2511.08590; SkillFeed 2608.28241).

NameKindSummary
SkillAffinityclassProfile-conditioned skill relevance: Beta posterior per (profile bucket, skill).
UserAdaptiveStrategyclassPer-user Beta posteriors per target, shrunk toward similar users and the global posterior.
profile_bucketfunction (profile: dict[str, Any], keys: tuple[str, ...]=('tier', 'expertise', 'role', 'language'))Coarse profile bucket used to pool users with the same declared attributes.
profile_vectorfunction (profile: dict[str, Any], dim: int=_PROFILE_DIM)Hashed one-hot encoding of key=value pairs (lists expand to one pair per element).

opensmartroute.learning.policy_gradient#

Source: src/opensmartroute/learning/policy_gradient.py

End-to-end policy-gradient routing (Router-R1 2506.09033; RLCascadeRouter 2608.15817).

NameKindSummary
PolicyGradientStrategyclassREINFORCE-trained softmax routing policy over hashed request features.
RegretReportclassDecision regret versus prediction error on a scored dataset.
decision_regretfunction (rows: Sequence[EvalRow], targets: Sequence[RouteTarget], choose: Callable[[EvalRow, Sequence[RouteTarget]], str], predict: Callable[[EvalRow, RouteTarget], float] | None=None)Mean decision regret of choose against the per-row oracle, next to the prediction error of.
decision_rewardfunction (outcome: Outcome, target: RouteTarget | None, objective: Objective, *, cost_scale: float=0.01, latency_scale: float=2000.0)Scalar decision reward of an outcome under objective (quality minus normalised cost/latency).

opensmartroute.learning.self_improve#

Source: src/opensmartroute/learning/self_improve.py

Self-improvement loop: refresh the catalogue, gather evidence, train a challenger, promote it only if better.

NameKindSummary
ImprovementReportclassWhat one cycle did: evidence gathered, catalogue changes, champion vs challenger, and the verdict.
SelfImproverclassClosed loop that keeps a :class:RouterSLM current with the model market and its own traffic.

opensmartroute.learning.slm#

Source: src/opensmartroute/learning/slm.py

The OpenSmartRoute routing SLM: a small, self-contained model that picks the target for a prompt.

NameKindSummary
SLM_FORMATconstanton-disk format version of RouterSLM.state().
RouterSLMclassSmall routing model: dual encoder + target catalogue snapshot + calibration, in one JSON file.
SLMReportclassHow an SLM did on a set of rows: accuracy, realised quality, cost, calibration and the training loss.
SLMStrategyclassEnsemble member backed by a :class:RouterSLM; scores are its probabilities, and it keeps learning online.
distill_routerfunction (router: Router, texts: Iterable[str], *, targets: Sequence[RouteTarget] | None=None, settings: Settings | None=None, seed: int=0)Compress the full router into an SLM: route every text, take the ensemble's ranked utilities as soft.

opensmartroute.math#

Source: src/opensmartroute/math/init.py

Mathematical toolkit behind OpenSmartRoute's decisions.

NameKindSummary
EWMAre-export of opensmartroute.math.estimators.EWMAExponentially weighted moving mean and variance with a z-score helper.
Banditre-export of opensmartroute.math.bandits.BanditCommon interface. context is optional; non-contextual learners ignore it.
BradleyTerryre-export of opensmartroute.math.preference.BradleyTerryOnline Bradley-Terry: per-context target strengths from pairwise wins, with L2 and forgetting.
ConformalCalibratorre-export of opensmartroute.math.calibration.ConformalCalibratorSplit conformal prediction over routing candidates.
CostAwareBanditre-export of opensmartroute.math.bandits.CostAwareBanditLagrangian budget wrapper (C2MAB-V flavour).
DelayedFeedbackre-export of opensmartroute.math.bandits.DelayedFeedbackDelayed-feedback wrapper (Joulani, György & Szepesvári, 2013).
DirichletProbere-export of opensmartroute.math.dirichlet.DirichletProbeLinear Dirichlet head over a fixed target list.
Elore-export of opensmartroute.math.preference.EloElo rating — a fixed-step Bradley–Terry with K-factor; handy for fast warm-up.
EnergyModelre-export of opensmartroute.math.energy.EnergyModelPer-target linear energy model wh = e0 + e_in * prompt + e_out * output fit by ridge.
EpsilonGreedyre-export of opensmartroute.math.bandits.EpsilonGreedyExplore uniformly with probability epsilon, otherwise exploit the empirical means.
HardwareProfilere-export of opensmartroute.math.energy.HardwareProfileStatic device characterisation used before measurements exist.
IRTModelre-export of opensmartroute.math.irt.IRTModelOnline 2PL IRT: target ability, item difficulty / discrimination via SGD, with forgetting and merge support.
IsotonicCalibratorre-export of opensmartroute.math.calibration.IsotonicCalibratorMonotone non-decreasing map score -> P(correct), fitted with PAV.
LinUCBre-export of opensmartroute.math.bandits.LinUCBDisjoint LinUCB (Li et al., WWW 2010) with Sherman–Morrison updates and optional.
MarkovChainre-export of opensmartroute.math.markov.MarkovChainDirichlet-smoothed transition counts. decay < 1 multiplies a row's counts by decay.
MultiKnapsackBanditre-export of opensmartroute.math.bandits.MultiKnapsackBanditBandits with several knapsack constraints (Badanidiyuru et al., 2013) with the.
PageHinkleyre-export of opensmartroute.math.estimators.PageHinkleyDetects a decrease in the monitored mean (e.g. quality dropping).
RoutingMDPre-export of opensmartroute.math.markov.RoutingMDPFinite-horizon / discounted MDP over conversation states and route actions.
TemperatureScalerre-export of opensmartroute.math.calibration.TemperatureScalerFits :math:\tau for confidence = softmax(u/\tau)[argmax] by 1-D golden-section.
ThompsonBetare-export of opensmartroute.math.bandits.ThompsonBetaBeta–Bernoulli Thompson sampling.
UCB1re-export of opensmartroute.math.bandits.UCB1UCB1: :math:\hat\mu_a + c\sqrt{\frac{2\ln t}{n_a}}; untried arms get +inf.
Welfordre-export of opensmartroute.math.estimators.WelfordNumerically stable running mean / variance (Welford's algorithm).
WindowDriftre-export of opensmartroute.math.estimators.WindowDriftADWIN-lite: compare the first and second half of a sliding window with a.
bayesian_averagere-export of opensmartroute.math.estimators.bayesian_averageShrink a small-sample mean toward a prior: (n·m + k·μ₀)/(n + k).
brier_scorere-export of opensmartroute.math.calibration.brier_scoreMean squared error between confidence and the 0/1 correctness label.
conformal_quantilere-export of opensmartroute.math.calibration.conformal_quantileFinite-sample corrected :math:\lceil (n+1)(1-\alpha)\rceil / n empirical quantile.
digammare-export of opensmartroute.math.dirichlet.digammaDigamma :math:\psi(x) for x > 0 via recurrence to x >= 6 and the asymptotic series.
dominatesre-export of opensmartroute.math.decision.dominatesa dominates b if it is ≥ quality, ≤ cost, ≤ latency and strictly better in one.
entropyre-export of opensmartroute.math.estimators.entropyShannon entropy (nats) of a probability vector.
erlang_cre-export of opensmartroute.math.decision.erlang_cP(an arriving request must wait) for M/M/c. Returns 1.0 if unstable.
expected_calibration_errorre-export of opensmartroute.math.estimators.expected_calibration_errorECE: how well does router confidence predict routing correctness?.
expected_waitre-export of opensmartroute.math.decision.expected_waitMean time in queue (Wq) for M/M/c, in the same time unit as the rates.
ginire-export of opensmartroute.math.estimators.giniGini impurity 1 - sum(p^2); 0 = certain.
hardware_profilere-export of opensmartroute.math.energy.hardware_profileBuilt-in profile by name (a100-80gb, h100-sxm, l4, rtx-4090, cpu-16c, npu-edge).
item_keyre-export of opensmartroute.math.irt.item_keyDefault item id: domain × difficulty bucket, e.g. legal/3.
kingman_waitre-export of opensmartroute.math.decision.kingman_waitKingman's G/G/1 approximation.
littles_lawre-export of opensmartroute.math.decision.littles_lawL = λ·W — average number of in-flight requests.
normalized_entropyre-export of opensmartroute.math.estimators.normalized_entropy0 = certain, 1 = uniform. Useful as an 'ask the LLM judge' trigger.
pareto_frontre-export of opensmartroute.math.decision.pareto_frontKeys of the non-dominated (quality, cost, latency) points.
reliability_diagramre-export of opensmartroute.math.calibration.reliability_diagramPer-bin (mean confidence, empirical accuracy, count) — plot or print it.
servers_for_slare-export of opensmartroute.math.decision.servers_for_slaSmallest c such that E[Wq] ≤ max_wait and P(wait) ≤ max_p_wait.
sigmoidre-export of opensmartroute.math.irt.sigmoidOverflow-safe logistic function.
softmaxre-export of opensmartroute.math.estimators.softmaxNumerically stable softmax; lower temperature sharpens the distribution.
topsisre-export of opensmartroute.math.decision.topsisTOPSIS closeness coefficient in [0,1]; quality is a benefit, cost/latency are costs.
weighted_sumre-export of opensmartroute.math.decision.weighted_sumScalarise w_q * quality - w_c * norm(cost) - w_l * norm(latency) with min-max normalised cost / latency.
wilson_intervalre-export of opensmartroute.math.estimators.wilson_intervalWilson score interval for a binomial proportion (robust at small n).

opensmartroute.math.bandits#

Source: src/opensmartroute/math/bandits.py

Multi-armed and contextual bandits for online routing decisions.

NameKindSummary
BanditclassCommon interface. context is optional; non-contextual learners ignore it.
ThompsonBetaclassBeta–Bernoulli Thompson sampling.
UCB1classUCB1: :math:\hat\mu_a + c\sqrt{\frac{2\ln t}{n_a}}; untried arms get +inf.
LinUCBclassDisjoint LinUCB (Li et al., WWW 2010) with Sherman–Morrison updates and optional.
EpsilonGreedyclassExplore uniformly with probability epsilon, otherwise exploit the empirical means.
CostAwareBanditclassLagrangian budget wrapper (C2MAB-V flavour).
MultiKnapsackBanditclassBandits with several knapsack constraints (Badanidiyuru et al., 2013) with the.
DelayedFeedbackclassDelayed-feedback wrapper (Joulani, György & Szepesvári, 2013).

opensmartroute.math.calibration#

Source: src/opensmartroute/math/calibration.py

Calibration and distribution-free risk control for routing confidence.

NameKindSummary
ConformalCalibratorclassSplit conformal prediction over routing candidates.
IsotonicCalibratorclassMonotone non-decreasing map score -> P(correct), fitted with PAV.
TemperatureScalerclassFits :math:\tau for confidence = softmax(u/\tau)[argmax] by 1-D golden-section.
brier_scorefunction (confidences: list[float], hits: list[bool])Mean squared error between confidence and the 0/1 correctness label.
conformal_quantilefunction (scores: list[float], alpha: float)Finite-sample corrected :math:\lceil (n+1)(1-\alpha)\rceil / n empirical quantile.
reliability_diagramfunction (confidences: list[float], hits: list[bool], bins: int=10)Per-bin (mean confidence, empirical accuracy, count) — plot or print it.

opensmartroute.math.decision#

Source: src/opensmartroute/math/decision.py

Multi-objective decision helpers and queueing theory for real-time routing.

NameKindSummary
dominatesfunction (a: Point, b: Point)a dominates b if it is ≥ quality, ≤ cost, ≤ latency and strictly better in one.
pareto_frontfunction (points: dict[str, Point])Keys of the non-dominated (quality, cost, latency) points.
weighted_sumfunction (points: dict[str, Point], w_q: float, w_c: float, w_l: float)Scalarise w_q * quality - w_c * norm(cost) - w_l * norm(latency) with min-max normalised cost / latency.
topsisfunction (points: dict[str, Point], weights: Sequence[float]=(0.6, 0.25, 0.15))TOPSIS closeness coefficient in [0,1]; quality is a benefit, cost/latency are costs.
erlang_cfunction (arrival_rate: float, service_rate: float, servers: int)P(an arriving request must wait) for M/M/c. Returns 1.0 if unstable.
expected_waitfunction (arrival_rate: float, service_rate: float, servers: int)Mean time in queue (Wq) for M/M/c, in the same time unit as the rates.
servers_for_slafunction (arrival_rate: float, service_rate: float, max_wait: float, max_p_wait: float=0.2)Smallest c such that E[Wq] ≤ max_wait and P(wait) ≤ max_p_wait.
littles_lawfunction (arrival_rate: float, mean_time_in_system: float)L = λ·W — average number of in-flight requests.
kingman_waitfunction (utilization: float, ca2: float, cs2: float, mean_service: float)Kingman's G/G/1 approximation.

opensmartroute.math.dirichlet#

Source: src/opensmartroute/math/dirichlet.py

Dirichlet probe over host hidden states (ProbeDirichlet, RouterXBench 2602.11877).

NameKindSummary
DirichletProbeclassLinear Dirichlet head over a fixed target list.
digammafunction (x: float)Digamma :math:\psi(x) for x > 0 via recurrence to x >= 6 and the asymptotic series.

opensmartroute.math.energy#

Source: src/opensmartroute/math/energy.py

Hardware-aware energy characterisation (HW-Router 2608.14575; 2608.28044).

NameKindSummary
EnergyModelclassPer-target linear energy model wh = e0 + e_in * prompt + e_out * output fit by ridge.
HardwareProfileclassStatic device characterisation used before measurements exist.
hardware_profilefunction (name: str)Built-in profile by name (a100-80gb, h100-sxm, l4, rtx-4090, cpu-16c, npu-edge).

opensmartroute.math.estimators#

Source: src/opensmartroute/math/estimators.py

Streaming estimators, drift detection and calibration.

NameKindSummary
EWMAclassExponentially weighted moving mean and variance with a z-score helper.
WelfordclassNumerically stable running mean / variance (Welford's algorithm).
PageHinkleyclassDetects a decrease in the monitored mean (e.g. quality dropping).
WindowDriftclassADWIN-lite: compare the first and second half of a sliding window with a.
wilson_intervalfunction (successes: float, n: int, z: float=1.96)Wilson score interval for a binomial proportion (robust at small n).
bayesian_averagefunction (mean: float, n: int, prior_mean: float, prior_n: float=5.0)Shrink a small-sample mean toward a prior: (n·m + k·μ₀)/(n + k).
softmaxfunction (xs: list[float], temperature: float=1.0)Numerically stable softmax; lower temperature sharpens the distribution.
entropyfunction (ps: list[float])Shannon entropy (nats) of a probability vector.
normalized_entropyfunction (ps: list[float])0 = certain, 1 = uniform. Useful as an 'ask the LLM judge' trigger.
ginifunction (ps: list[float])Gini impurity 1 - sum(p^2); 0 = certain.
expected_calibration_errorfunction (confidences: list[float], hits: list[bool], bins: int=10)ECE: how well does router confidence predict routing correctness?.

opensmartroute.math.irt#

Source: src/opensmartroute/math/irt.py

Item Response Theory for routing (IRT-Router, ACL 2025).

NameKindSummary
sigmoidfunction (z: float)Overflow-safe logistic function.
IRTModelclassOnline 2PL IRT: target ability, item difficulty / discrimination via SGD, with forgetting and merge support.
item_keyfunction (domain: str, complexity: float, buckets: int=4)Default item id: domain × difficulty bucket, e.g. legal/3.

opensmartroute.math.markov#

Source: src/opensmartroute/math/markov.py

Markov chains and MDPs for conversational / multi-step routing.

NameKindSummary
MarkovChainclassDirichlet-smoothed transition counts. decay < 1 multiplies a row's counts by decay.
RoutingMDPclassFinite-horizon / discounted MDP over conversation states and route actions.

opensmartroute.math.preference#

Source: src/opensmartroute/math/preference.py

Bradley–Terry pairwise preference model (RouteLLM, Prompt-to-Leaderboard).

NameKindSummary
BradleyTerryclassOnline Bradley-Terry: per-context target strengths from pairwise wins, with L2 and forgetting.
EloclassElo rating — a fixed-step Bradley–Terry with K-factor; handy for fast warm-up.

opensmartroute.mcp_server#

Source: src/opensmartroute/mcp_server.py

Model Context Protocol server: the router as a set of tools for any IDE or agent.

NameKindSummary
PROTOCOL_VERSIONconstant: Protocol revision this server speaks by default.
SUPPORTED_PROTOCOL_VERSIONSconstant: Revisions accepted from clients (the client's choice is echoed when it is one of these).
MCPServerclassServe a :class:Router (or an enterprise router wrapping one) over MCP.
RemoteMCPclassForward JSON-RPC messages to an HTTP MCP endpoint (POST /mcp) - the bridge's counterpart of.
ToolErrorclassRaised inside a tool: reported to the client as a tool result with isError (not a protocol error).
bridge_stdiofunction (url: str, api_key: str | None=None)Expose a remote HTTP MCP endpoint as a local stdio server (for clients that only spawn processes).
decision_dictfunction (d: RouteDecision)RouteDecision.to_dict() plus signals and the ranked candidates - what IDE clients want to show.
serve_stdiofunction (server: Any, stdin: IO[bytes] | None=None, stdout: IO[str] | None=None)Run server (anything with handle_json) over newline-delimited JSON on stdin/stdout until EOF.

opensmartroute.observability#

Source: src/opensmartroute/observability.py

Tracing and observability: every step of routing, execution and learning as a structured event.

NameKindSummary
EVENT_NAMESconstantRouter.route.
METRIC_PREFIXconstantPrometheus metric name prefix (osr_).
SPAN_NAMESconstant: Every span name the SDK opens (an :class:Event with kind="span").
TRACEPARENT_KEYconstantrequest.context key (and HTTP header) carrying an inbound W3C trace context.
TRACE_ID_HEADERconstantresponse header osr serve sets to the trace id of the request.
EventclassOne captured span or event: flat, JSON-friendly, never carries request text.
EventSinkclassReceiver port: override :meth:emit; live bridges may also implement :meth:span_start / :meth:span_end.
FileSinkclassAppend-only JSONL file, one event per line.
LoggingSinkclassOne JSON line per event on the opensmartroute.events logger (level follows the event).
MemorySinkclassThread-safe ring buffer of the newest max_events events; queryable by request, trace, name or level.
MetricsSinkclassCounters and latency percentiles derived from events; :meth:prometheus renders the exposition text.
SpanclassAn open unit of work; a context manager that records duration, status and nested events.
TracerclassOpens spans, records events and fans them out to sinks; sample_rate < 1 traces a share of requests.
configure_tracingfunction (*sinks: EventSink, sample_rate: float | None=None, settings: Settings | None=None)Install sinks on the process-wide tracer. With no sinks, use the ones named by settings.
current_tracerfunction ()The tracer of the innermost open span, else the one bound with :func:use_tracer, else the global one.
get_tracerfunction ()The process-wide tracer (disabled until :func:configure_tracing adds sinks).
text_digestfunction (text: str)Loggable stand-in for request text: {"text_sha256": <16 hex>, "text_len": n}.
use_tracerclasswith use_tracer(t): makes t the tracer that :func:current_tracer returns in this context.

opensmartroute.ocm#

Source: src/opensmartroute/ocm.py

Open Capability Manifest (OCM) - a vendor-neutral description of any routable capability.

NameKindSummary
OCM_KINDSconstant: Manifest kinds (identical to :class:~opensmartroute.TargetKind values).
OCM_PROTOCOLSconstant: Endpoint protocols a manifest may declare; http and a2a can be bound to executors directly.
OCM_VERSIONconstant: Current manifest version (the ocm field).
bind_endpointfunction (target: RouteTarget, region: str | None=None, **kw: Any)Attach an executor for the selected endpoint (http -> :func:~opensmartroute.adapters.http_handler,.
capability_from_targetfunction (target: RouteTarget)Reverse mapping: publish a target as an OCM v1 manifest (dict; dump as YAML or JSON).
dump_capabilityfunction (doc: dict[str, Any])Serialise a manifest to YAML when PyYAML is available, else to pretty JSON (both are valid OCM).
is_capabilityfunction (doc: Any)True when doc looks like an OCM manifest (has an ocm version field).
load_capabilitiesfunction (paths: Iterable[str | Path] | str | Path, *, strict: bool=True)Load manifests from files and/or directories (**/capability.{yaml,yml,json}).
load_capabilityfunction (path: str | Path, *, strict: bool=True)Load one capability.yaml / .json manifest as a target.
select_endpointfunction (target: RouteTarget, region: str | None=None)The manifest endpoint to use: a region-matching one first, then the first region-less one.
target_from_capabilityfunction (doc: dict[str, Any], *, strict: bool=True)Convert a manifest into a :class:~opensmartroute.RouteTarget (validating first when strict).
validate_capabilityfunction (doc: Any)Validate a manifest; returns a list of human-readable problems (empty = valid).

opensmartroute.policy#

Source: src/opensmartroute/policy/init.py

Policy layer: hard constraints that are never traded off against utility.

NameKindSummary
DataBoundaryRuleclassThe target's boundary must be at least as strict as the request's (public < private < on_prem).
JailbreakRuleclassRisky prompts may only reach humans or targets tagged as safe.
LanguageRuleclassThe detected language must be declared by the target, unless it declares the wildcard language.
PolicyclassOrdered chain of :data:PolicyRule; returns the first rejection reason or None.
PolicyRuleconstantA hard-constraint check: return a rejection reason, or None to let the target through.
allow_listfunction (target: RouteTarget, request: RouteRequest, signals: Signals)When the request names allow_targets, reject everything else.
allowed_kindsfunction (target: RouteTarget, request: RouteRequest, signals: Signals)Reject kinds outside the request's allowed_kinds.
context_windowfunction (target: RouteTarget, request: RouteRequest, signals: Signals)Reject when the estimated input exceeds the target's context_window.
cost_budgetfunction (target: RouteTarget, request: RouteRequest, signals: Signals)Reject targets whose unit cost exceeds the request's max_cost_per_1k.
default_rulesfunction (settings: PolicySettings | None=None)The built-in chain, in evaluation order (cheap identity checks first).
deny_listfunction (target: RouteTarget, request: RouteRequest, signals: Signals)Reject targets listed in the request's deny_targets.
enabledfunction (target: RouteTarget, request: RouteRequest, signals: Signals)Reject disabled targets.
input_tokensfunction (target: RouteTarget, request: RouteRequest, signals: Signals)Reject when the estimated input exceeds the target's max_tokens_in.
latency_slofunction (target: RouteTarget, request: RouteRequest, signals: Signals)Reject targets whose declared latency exceeds the request's max_latency_ms.
modalitiesfunction (target: RouteTarget, request: RouteRequest, signals: Signals)Every modality detected in the request must be supported by the target.
piifunction (target: RouteTarget, request: RouteRequest, signals: Signals)Requests flagged as containing PII may only reach targets with pii_allowed.
regionfunction (target: RouteTarget, request: RouteRequest, signals: Signals)The request's region must be one the target serves (targets with no regions serve all).
rule_namefunction (rule: PolicyRule)Display name of a rule: its name attribute, else __name__, else the class name.
tenantfunction (target: RouteTarget, request: RouteRequest, signals: Signals)The request's tenant must be permitted by targets that restrict tenants.
toolsfunction (target: RouteTarget, request: RouteRequest, signals: Signals)When tools are required (constraint or context['tools']), LLM targets must supports_tools.

opensmartroute.realtime#

Source: src/opensmartroute/realtime/init.py

Real-time operational controls: health, circuit breaking, rate & budget limits.

NameKindSummary
BreakerStateclassCircuit-breaker states: closed (healthy), open (tripped), half_open (probing recovery).
CircuitBreakerclassClassic three-state breaker with a sliding failure window.
TokenBucketclassRate limiter: rate tokens/sec, burst up to capacity. Thread-safe.
BudgetclassRolling spend cap (e.g. USD per hour) per target or tenant.
LatencyWindowclassRolling window of the last size latencies with nearest-rank percentiles (p50 / p90 / p99).
TargetHealthclassLive health of one target: breaker, latency EWMA + percentiles, success counts, optional rate limit / budget.
HealthRegistryclassTracks health per target id. Shared by policy + strategy + executor.
HealthPolicyclassPolicy that additionally rejects targets whose breaker is open, whose rate.
HealthStrategyclassSoft signal: observed reliability x latency-SLO fit from live health data.

opensmartroute.retrieval#

Source: src/opensmartroute/retrieval.py

Retrieval-based candidate narrowing for very large target pools (ToolRet / Skill-RAG).

NameKindSummary
BM25IndexclassIn-memory BM25 (Okapi) index over target documents.
DenseIndexclassCosine index. Default = sparse signed-hashing features with an inverted index (pure.
NarrowingStatsclassCounters kept by the retrieval middleware: calls, how often it narrowed, last pool and kept sizes.
RetrievalResultclassNarrowed candidate ids with fused scores and how many came from the lexical / dense legs.
RetrieverclassHybrid lexical + dense narrowing that tracks a registry (or an explicit pool).
execute_tool_targetfunction (registry: TargetRegistry, allow: Callable[[RouteTarget, dict[str, Any]], bool] | None=None, target_id: str='execute_tool')A tool that runs a tool by id (with an optional allow-policy hook).
narrow_signals_hintfunction (signals: Signals)Extra lexical hints from signals (domains / actions) appended to the retrieval query.
rrffunction (rankings: list[list[tuple[str, float]]], k: int=60, weights: list[float] | None=None)Reciprocal Rank Fusion over several ranked lists of (id, score).
select_skill_setfunction (request: RouteRequest | str, pool: list[RouteTarget], k: int=3, embedder: Embedder | None=None, redundancy: float=0.7, min_gain: float=0.02, relevance: dict[str, float] | None=None)Greedy facility-location selection of complementary targets.
target_textfunction (t: RouteTarget)Lexical document for a target: id, name, description, capabilities and up to 8 examples.
tokenizefunction (text: str)Lower-case alphanumeric tokens.
tool_search_targetfunction (retriever: Retriever, registry: TargetRegistry, kinds: tuple[str, ...]=('tool', 'skill'), k: int=8, target_id: str='tool_search')A tool that finds tools: returns compact descriptors for the k best matches.

opensmartroute.router#

Source: src/opensmartroute/router.py

The Router: signals -> policy -> strategies -> ensemble -> decision (+plan).

NameKindSummary
RouterclassSignals -> policy -> strategies -> ensemble -> decision (+ optional plan).
NoRouteErrorre-export of opensmartroute.errors.NoRouteErrorNo target satisfied the hard constraints.
DEFAULT_WEIGHTSconstant: Library-default ensemble weights (Settings().weights); kept for backwards compatibility.

opensmartroute.sdk#

Source: src/opensmartroute/sdk.py

OpenSmartRoute SDK: decorator-driven registration of routing components.

NameKindSummary
ComponentKindclassThe component families a :class:ComponentRegistry can hold.
ComponentRegistryclassBlueprints for every routing component, with decorators that register into it.
FunctionMiddlewareclassAdapts fn(request, next_) -> RouteDecision.
FunctionSignalclassAdapts fn(request, signals) -> None | {field: value}.
FunctionStrategyclassAdapts fn(request, signals, candidates) -> {target_id: score | StrategyScore}.
RegistrationclassA registered blueprint. factory() returns a fresh component instance.
agentconstant@agent(id, ...): shorthand for an agent target.
componentsconstant: The process-wide registry that the top-level decorators (opensmartroute.strategy ...) bind to.
middlewareconstant@middleware: register a Middleware class or fn(request, next_route).
policy_ruleconstant@policy_rule: register fn(target, request, signals) -> reason | None.
signalconstant@signal: register a SignalExtractor class or fn(request, signals) -> mapping.
skillconstant@skill(id, ...): shorthand for a skill target.
strategyconstant@strategy(weight=, name=): register a Strategy class or fn(request, signals, candidates).
targetconstant@target(id, kind, ...): the decorated callable becomes a RouteTarget handler.
telemetryconstant@telemetry: register a Telemetry sink class or factory.
toolconstant@tool(id, ...): shorthand for a tool target.

opensmartroute.security#

Source: src/opensmartroute/security/init.py

Security controls for the routing control plane.

NameKindSummary
GadgetDetectorre-export of opensmartroute.security.gadget.GadgetDetectorLearned tail classifier. score(text) in [0, 1]; split(text) finds the gadget tail.
GuardMiddlewareclassApplies :class:InputGuard (+ optional redaction) before routing.
GuardReportclassOutcome of :meth:InputGuard.inspect: pass / fail, reasons, gadget suspicion score and the cleaned text.
InputGuardclassValidates and normalises request text. Cheap: O(len(text)).
OriginPolicyre-export of opensmartroute.security.provenance.OriginPolicyDecides which tool arguments must be user-originated and verifies them.
OriginRulere-export of opensmartroute.security.provenance.OriginRulePer-target override. sensitive=None = infer from parameter names.
OriginViolationre-export of opensmartroute.security.provenance.OriginViolationRaised by :class:OriginPolicy when a sensitive argument of a state-changing tool comes from untrusted text.
RedactorclassReplace PII with typed placeholders, e.g. <EMAIL_1>. Reversible per-request.
ResourceLimitExceededre-export of opensmartroute.security.limits.ResourceLimitExceededA task went over one of its :class:ResourceLimits (steps, tool calls, depth, tokens, cost, wall time).
ResourceLimitMiddlewarere-export of opensmartroute.security.limits.ResourceLimitMiddlewareCharge one step (+ the request's token estimate) per route() call.
ResourceLimiterre-export of opensmartroute.security.limits.ResourceLimiterThread-safe per-task accounting against :class:ResourceLimits.
ResourceLimitsre-export of opensmartroute.security.limits.ResourceLimitsPer-task ceilings that stop runaway agents; enforced by :class:ResourceLimiter.
SafetyCasere-export of opensmartroute.security.safety.SafetyCaseOne red-team scenario: an adversarial request, its clean baseline and the routing invariants to check.
apply_limitsre-export of opensmartroute.security.limits.apply_limitsWrap handlers of the given kinds. Returns the number wrapped.
apply_origin_policyre-export of opensmartroute.security.provenance.apply_origin_policyWrap every state-changing target in registry. Returns the number wrapped.
description_riskre-export of opensmartroute.security.injection.description_riskRisk of a tool/agent description: max of injection lexicon and learned gadget score.
guard_handlerre-export of opensmartroute.security.provenance.guard_handlerReturn a copy of target whose handler enforces policy before invoking the tool.
injection_riskre-export of opensmartroute.security.injection.injection_riskShortcut for inspect_injection(text).score.
inspect_injectionre-export of opensmartroute.security.injection.inspect_injectionScore instruction-injection likelihood with the matched lexicon entries for the trace.
limit_handlerre-export of opensmartroute.security.limits.limit_handlerCopy of target whose handler charges one tool call (and nesting depth) per invocation.
load_secretfunction (name: str, *, file_env_suffix: str='_FILE', default: str | None=None)12-factor secret loading: NAME env var, else the file at NAME_FILE (k8s/Docker secrets).
mark_untrustedre-export of opensmartroute.security.provenance.mark_untrustedRecord content that entered the request from a non-user source.
run_safety_suitere-export of opensmartroute.security.safety.run_safety_suiteRun every case through route and return pass/fail details grouped by category.
sanitize_for_promptfunction (text: str, max_len: int=4000)Make user text safe to embed in an LLM-judge prompt.
shannon_entropyfunction (s: str)Character-level Shannon entropy in bits (high values flag encoded / random payloads).
strip_steeringre-export of opensmartroute.security.injection.strip_steeringRemove sentences that instruct the router (self-declared complexity, "use the best.
synthesize_gadget_corpusre-export of opensmartroute.security.gadget.synthesize_gadget_corpusLabelled (text, label) rows: clean prompts + prompts with an appended gadget.

opensmartroute.security.gadget#

Source: src/opensmartroute/security/gadget.py

Learned confounder-gadget detector (Rerouting LLM Routers, Shafran et al. 2025).

NameKindSummary
CLEANconstantclass labels used by the detector and the synthetic corpus.
GADGETconstantclass labels used by the detector and the synthetic corpus.
GadgetDetectorclassLearned tail classifier. score(text) in [0, 1]; split(text) finds the gadget tail.
synthesize_gadget_corpusfunction (per_template: int=2, n_gadgets: int=260, seed: int=0)Labelled (text, label) rows: clean prompts + prompts with an appended gadget.
synthesize_gadgetsfunction (n: int, seed: int=0)Adversarial suffixes drawn from the gadget families described in the paper.
tail_featuresfunction (tokens: Sequence[str])Hand-crafted statistics of a token window (all in [0, 1]) – complements hashed n-grams.

opensmartroute.security.injection#

Source: src/opensmartroute/security/injection.py

Instruction-injection detection for text that is not the user's request.

NameKindSummary
InjectionReportclassInstruction-injection score in [0, 1] plus the lexicon hits that produced it.
description_riskfunction (text: str, gadget_score: float | None=None)Risk of a tool/agent description: max of injection lexicon and learned gadget score.
injection_riskfunction (text: str)Shortcut for inspect_injection(text).score.
inspect_injectionfunction (text: str)Score instruction-injection likelihood with the matched lexicon entries for the trace.

opensmartroute.security.limits#

Source: src/opensmartroute/security/limits.py

Resource-amplification limits ("Beyond Max Tokens").

NameKindSummary
ResourceLimitExceededclassA task went over one of its :class:ResourceLimits (steps, tool calls, depth, tokens, cost, wall time).
ResourceLimitMiddlewareclassCharge one step (+ the request's token estimate) per route() call.
ResourceLimiterclassThread-safe per-task accounting against :class:ResourceLimits.
ResourceLimitsclassPer-task ceilings that stop runaway agents; enforced by :class:ResourceLimiter.
TaskUsageclassRunning consumption of one task (steps, tool calls, depth, tokens, cost, timestamps).
apply_limitsfunction (registry: TargetRegistry, limiter: ResourceLimiter, kinds: tuple[str, ...]=('tool', 'agent', 'workflow'))Wrap handlers of the given kinds. Returns the number wrapped.
limit_handlerfunction (target: RouteTarget, limiter: ResourceLimiter)Copy of target whose handler charges one tool call (and nesting depth) per invocation.
task_id_offunction (request: RouteRequest, key: str='task_id')context[key] when present, otherwise the request id (each request is its own task).

opensmartroute.security.provenance#

Source: src/opensmartroute/security/provenance.py

Origin (provenance) policy for tool parameters – ROPE-style control-flow integrity.

NameKindSummary
OriginFindingclassWhere one tool argument's value came from (user, context, untrusted content or unknown).
OriginPolicyclassDecides which tool arguments must be user-originated and verifies them.
OriginReportclassProvenance check of a tool call: all findings plus the sensitive parameters with a disallowed origin.
OriginRuleclassPer-target override. sensitive=None = infer from parameter names.
OriginViolationclassRaised by :class:OriginPolicy when a sensitive argument of a state-changing tool comes from untrusted text.
SENSITIVE_PARAM_HINTSconstantParameter-name fragments treated as sensitive when a target has no explicit OriginRule.
STATE_CHANGING_ACTIONSconstantVerbs in a tool's id / description / actions that mark it as state-changing (side effects).
apply_origin_policyfunction (registry: TargetRegistry, policy: OriginPolicy)Wrap every state-changing target in registry. Returns the number wrapped.
guard_handlerfunction (target: RouteTarget, policy: OriginPolicy, arguments_key: str='tool_arguments')Return a copy of target whose handler enforces policy before invoking the tool.
mark_untrustedfunction (request: RouteRequest, text: str, source: str='retrieved')Record content that entered the request from a non-user source.
untrusted_textsfunction (request: RouteRequest)(source, text) pairs recorded by :func:mark_untrusted on this request.
user_textsfunction (request: RouteRequest)Everything the user actually typed: current text, prior user turns and the pre-guard original text.

opensmartroute.security.safety#

Source: src/opensmartroute/security/safety.py

Safety-routing regression suite ("When Safety Routing Breaks").

NameKindSummary
SafetyCaseclassOne red-team scenario: an adversarial request, its clean baseline and the routing invariants to check.
SafetyResultclassVerdict for one :class:SafetyCase: chosen vs baseline target and the invariants that failed.
default_casesfunction (seed: int=0)Deployment-agnostic red-team cases; extend with catalogue-specific ones.
run_safety_suitefunction (route: RouteCall, cases: Sequence[SafetyCase] | None=None, *, seed: int=0)Run every case through route and return pass/fail details grouped by category.

opensmartroute.server#

Source: src/opensmartroute/server.py

Optional FastAPI server exposing the router over HTTP.

NameKindSummary
AUTO_SLUGconstant: Slug prefix of the router's own pseudo-models on the OpenAI-compatible proxy (osr/auto).
DEFAULT_PROXY_ALIASESconstant: Model names that mean "let the router choose" on the OpenAI-compatible proxy.
AUTO_VARIANTSconstant: osr/auto:<variant> presets: objective weights, hard constraints and allowed kinds.
APP_HEADERconstant: Request header naming the calling application (OpenRouter-style attribution); echoed in metadata.
TARGET_HEADERconstant: Response headers carrying the decision (target id, request id) beside the OpenAI-shaped body.
REQUEST_ID_HEADERconstant: Response headers carrying the decision (target id, request id) beside the OpenAI-shaped body.
OPEN_PATHSconstant: Paths that never require a token when access control is on (probes, metrics, token check, OpenAPI pages).
UNTRACED_PATHSconstant: Paths that never open an http.request span (probes and the observability endpoints themselves).
bearer_tokenfunction (headers: Any)The access token of a request: X-API-Key first, else Authorization: Bearer <token>.
token_acceptedfunction (token: str | None, tokens: Iterable[str])Constant-time membership test of token in the configured access tokens.
create_appfunction (router: Any, proxy_aliases: frozenset[str]=DEFAULT_PROXY_ALIASES, autopilot: Any=None, auth_tokens: Iterable[str]=())FastAPI app: /route, /feedback, /targets, /stats, /healthz and the OpenAI-compatible /v1.
resolve_modelfunction (model: str, aliases: frozenset[str]=DEFAULT_PROXY_ALIASES)Split a proxy model into (pinned target id | None, route options).
proxy_requestfunction (body: dict[str, Any], aliases: frozenset[str]=frozenset({'auto'}), app: str | None=None)Turn an OpenAI chat-completion body into a :class:RouteRequest.
proxy_responsefunction (decision: RouteDecision, result: Any, request: RouteRequest | None=None)Shape an execution result as an OpenAI chat.completion object (model = chosen target id).

opensmartroute.settings#

Source: src/opensmartroute/settings.py

Typed, environment-overridable settings: the only home for OpenSmartRoute's tunable constants.

NameKindSummary
BanditSettingsclassThompson-sampling strategy (:class:~opensmartroute.BanditStrategy).
CapabilitySettingsclassCapability-fit strategy (:class:~opensmartroute.CapabilityStrategy).
ObservabilitySettingsclassTracing and event capture (:mod:opensmartroute.observability); read by Tracer.from_settings.
PolicySettingsclassHard-constraint thresholds (read by :class:~opensmartroute.Policy).
RoutingSettingsclassEnsemble, confidence and plan composition (read by :class:~opensmartroute.Router).
RulesSettingsclassDeclarative rules strategy (:class:~opensmartroute.RulesStrategy).
SLMSettingsclassRouting SLM, dataset collection and the self-improvement loop (:mod:opensmartroute.learning.slm).
ServerSettingsclassosr serve (:mod:opensmartroute.server): access control of the self-hosted HTTP API.
SettingsclassAll tunables, grouped by consumer. Immutable; derive variants with :meth:replace.
WeightSettingsclassEnsemble weight per strategy name (looked up as weights[strategy.name]).
configurefunction (settings: Settings | None=None, **groups: Any)Install process-wide settings. configure() with no arguments re-reads the environment;.
get_settingsfunction ()The process-wide :class:Settings (environment overlay applied once, lazily).
resolvefunction (settings: Settings | None)settings if given, else the process-wide settings (component constructors use this).

opensmartroute.signals#

Source: src/opensmartroute/signals/init.py

Signal extraction: cheap, deterministic features computed from the request.

NameKindSummary
ACTION_LEXICONconstantAction label -> trigger phrases; the keys are the ontology action names (summarize, translate, ... escalate).
DEFAULT_EXTRACTORSconstantafter domain detection (uses domain bonus).
DEFAULT_ONTOLOGYre-export of opensmartroute.signals.ontology.DEFAULT_ONTOLOGYthe built-in TASK_TYPES ontology used by TaskTypeSignal.
DOMAIN_LEXICONconstantDomain label -> trigger phrases; the keys are the ontology domain names targets and skills may declare.
EVENT_LEXICONre-export of opensmartroute.signals.events.EVENT_LEXICONEvent name -> (domains, actions). Keys are matched exactly, then by subject.* prefix.
EVENT_SUBJECTSre-export of opensmartroute.signals.events.EVENT_SUBJECTSGeneric fallbacks when the exact event / prefix is unknown: subject -> domain, verb -> action.
EVENT_VERBSre-export of opensmartroute.signals.events.EVENT_VERBSevent verb -> action when the exact event / prefix is unknown.
TASK_PRIORSre-export of opensmartroute.signals.models.TASK_PRIORSname: (difficulty, reasoning_need, expected_output_tokens).
TASK_TYPESre-export of opensmartroute.signals.ontology.TASK_TYPES---------------------------------------------------------------- transformation.
ComplexitySignalclassHybrid-LLM-style difficulty estimate in [0, 1].
DomainActionSignalclassArch-Router-style domain / action detection via lexicon matching.
DraftResponseSignalre-export of opensmartroute.signals.uncertainty.DraftResponseSignalQuery-response mixed representation: run a cheap drafter and expose draft features.
EventInfore-export of opensmartroute.signals.events.EventInfoParsed event: raw name, subject, verb and the domains / actions it implies.
EventSignalre-export of opensmartroute.signals.events.EventSignalDomains / actions from context["event"] and context["intent"] (event-driven requests).
EventTriggerre-export of opensmartroute.signals.uncertainty.EventTriggerEvent-triggered invocation: evaluate uncertainty features against rules and return the.
HashedClassifierre-export of opensmartroute.signals.models.HashedClassifierMultinomial logistic regression over hashed features (sparse weights per class).
HashedFeaturizerre-export of opensmartroute.signals.models.HashedFeaturizerHashing-trick sparse features: words, bigrams, character n-grams and a few numeric text statistics.
HashedRegressorre-export of opensmartroute.signals.models.HashedRegressorSquared-loss linear regressor on hashed features; output clipped to [lo, hi].
HistorySignalclassConversation-state features from request.history (RCRouter-style):.
LanguageSignalclasslanguage from script / stop-word hints (LANGUAGE_HINTS); defaults to en.
LearnedDifficultySignalclassBlend the heuristic complexity with a trained regressor (Hybrid-LLM style difficulty).
LengthSignalclasstoken_estimate from text + history length (about 4 characters per token).
ModalitySignalclassmodalities from context keys (images / image_url, audio, files) on top of text.
OutputLengthSignalclassExpected output tokens (drives cost estimates and effort/max_tokens choices).
ProfileSignalclassUser-profile features (request.profile): tier, expertise, preferences.
ReasoningNeedSignalclassHow much a request benefits from extended thinking (ThinkSwitcher / Sketch-of-Thought).
SensitivitySignalclassPII and prompt-injection / jailbreak heuristics (vLLM-semantic-router-style).
SignalExtractorclassFills in part of a :class:Signals object.
SignalModelBundlere-export of opensmartroute.signals.models.SignalModelBundleAll learned signal models together, persisted as one JSON document.
TaskOntologyre-export of opensmartroute.signals.ontology.TaskOntologyLookup helpers over :data:TASK_TYPES.
TaskTypere-export of opensmartroute.signals.ontology.TaskTypeOne leaf of the task ontology: family, description, the actions it maps to and template phrasings.
TaskTypeSignalclassOntology task type. Uses the learned classifier when one is loaded, else maps the.
TriggerRulere-export of opensmartroute.signals.uncertainty.TriggerRulefeature >= threshold (or <= when below=True) fires action.
UncertaintyGatere-export of opensmartroute.signals.uncertainty.UncertaintyGateA cascade quality gate built from response uncertainty.
VerbalisedDifficultySignalre-export of opensmartroute.signals.uncertainty.VerbalisedDifficultySignalBlend a verbalised difficulty into complexity and reasoning_need.
WorkflowSignalre-export of opensmartroute.signals.events.WorkflowSignalDetect a workflow trigger (context["workflow"] or "run the workflow|process|...").
extract_signalsfunction (request: RouteRequest, extractors: list[SignalExtractor] | None=None)Run the extractor chain (DEFAULT_EXTRACTORS unless given) and return the populated :class:Signals.
learned_extractorsfunction (bundle: SignalModelBundle)Extractor pipeline with the trained :class:SignalModelBundle plugged in.
load_training_rowsre-export of opensmartroute.signals.models.load_training_rowsRead JSONL training rows. Accepts eval-dataset rows too (prompt/text,.
parse_difficultyre-export of opensmartroute.signals.uncertainty.parse_difficultyMap a verbalised difficulty (number, "7/10", "hard") to [0, 1]; None if unparseable.
parse_eventre-export of opensmartroute.signals.events.parse_eventMap an event name (subject.verb, subject:verb or subject_verb) onto domains / actions.
response_uncertaintyre-export of opensmartroute.signals.uncertainty.response_uncertaintyPost-hoc uncertainty features for a set of sampled answers.
semantic_entropyre-export of opensmartroute.signals.uncertainty.semantic_entropyEntropy (nats) over meaning clusters of sampled answers; 0 when every sample agrees.
synthesize_datasetre-export of opensmartroute.signals.models.synthesize_datasetGenerate labelled prompts from ontology templates x topics with light noise.

opensmartroute.signals.events#

Source: src/opensmartroute/signals/events.py

Event- and workflow-driven signals for agentic inputs.

NameKindSummary
EVENT_LEXICONconstantEvent name -> (domains, actions). Keys are matched exactly, then by subject.* prefix.
EVENT_SUBJECTSconstantGeneric fallbacks when the exact event / prefix is unknown: subject -> domain, verb -> action.
EVENT_VERBSconstantevent verb -> action when the exact event / prefix is unknown.
EventInfoclassParsed event: raw name, subject, verb and the domains / actions it implies.
EventSignalclassDomains / actions from context["event"] and context["intent"] (event-driven requests).
WorkflowSignalclassDetect a workflow trigger (context["workflow"] or "run the workflow|process|...").
parse_eventfunction (name: str)Map an event name (subject.verb, subject:verb or subject_verb) onto domains / actions.

opensmartroute.signals.models#

Source: src/opensmartroute/signals/models.py

Learned signal models: hashed n-gram linear models with no dependencies.

NameKindSummary
TASK_PRIORSconstantname: (difficulty, reasoning_need, expected_output_tokens).
HashedClassifierclassMultinomial logistic regression over hashed features (sparse weights per class).
HashedFeaturizerclassHashing-trick sparse features: words, bigrams, character n-grams and a few numeric text statistics.
HashedRegressorclassSquared-loss linear regressor on hashed features; output clipped to [lo, hi].
SignalModelBundleclassAll learned signal models together, persisted as one JSON document.
load_training_rowsfunction (path: str | Path)Read JSONL training rows. Accepts eval-dataset rows too (prompt/text,.
synthesize_datasetfunction (ontology: TaskOntology=DEFAULT_ONTOLOGY, topics: Sequence[str] | None=None, per_template: int=6, seed: int=0)Generate labelled prompts from ontology templates x topics with light noise.

opensmartroute.signals.ontology#

Source: src/opensmartroute/signals/ontology.py

Task ontology: families -> types -> subtypes, with an orthogonal domain axis.

NameKindSummary
DEFAULT_ONTOLOGYconstantthe built-in TASK_TYPES ontology used by TaskTypeSignal.
TASK_TYPESconstant---------------------------------------------------------------- transformation.
TaskOntologyclassLookup helpers over :data:TASK_TYPES.
TaskTypeclassOne leaf of the task ontology: family, description, the actions it maps to and template phrasings.

opensmartroute.signals.uncertainty#

Source: src/opensmartroute/signals/uncertainty.py

Uncertainty signals that come from outside the query text.

NameKindSummary
HEDGESconstantHedging phrases counted in a draft response (uncertainty evidence).
SELF_CORRECTIONSconstantSelf-correction markers counted in a draft response.
DraftResponseSignalclassQuery-response mixed representation: run a cheap drafter and expose draft features.
EventTriggerclassEvent-triggered invocation: evaluate uncertainty features against rules and return the.
TriggerRuleclassfeature >= threshold (or <= when below=True) fires action.
UncertaintyGateclassA cascade quality gate built from response uncertainty.
VerbalisedDifficultySignalclassBlend a verbalised difficulty into complexity and reasoning_need.
draft_featuresfunction (text: str, draft: str, expected_tokens: int=0)Features of a cheap draft answer relative to the query: hedging, refusal, self-correction,.
parse_difficultyfunction (value: Any)Map a verbalised difficulty (number, "7/10", "hard") to [0, 1]; None if unparseable.
response_uncertaintyfunction (samples: list[str], embedder: Embedder | None=None, threshold: float=0.85, p_true: float | None=None)Post-hoc uncertainty features for a set of sampled answers.
semantic_clustersfunction (samples: Iterable[str], embedder: Embedder | None=None, threshold: float=0.85)Greedy single-link clustering of answers into meaning classes; returns a cluster id per sample.
semantic_entropyfunction (samples: Iterable[str], embedder: Embedder | None=None, threshold: float=0.85)Entropy (nats) over meaning clusters of sampled answers; 0 when every sample agrees.

opensmartroute.stack#

Source: src/opensmartroute/stack.py

Declarative stacks: one document that describes a whole routing setup.

NameKindSummary
REGISTRY_SCHEMEconstant: Scheme of marketplace references inside imports (registry://<slug>[@<version>]).
STACK_VERSIONconstant: Current stack document version (the osr field).
Resolverconstant: Resolver for registry:// imports: takes the reference (without the scheme) and returns the stack document.
StackclassA resolved stack: every import merged, ready to build a router.
StackChangeclassOne line of a plan: add / change / remove of a target, rule, setting or objective weight.
StackPlanclassWhat applying desired would change compared with current (osr stack plan).
dump_stackfunction (doc: dict[str, Any], fmt: str | None=None)Serialise a stack document: fmt = "yaml" | "json"; default YAML when PyYAML is installed.
is_stackfunction (doc: Any)True when doc looks like a stack document (kind: stack or an osr version with stack sections).
load_stackfunction (source: str | Path | dict[str, Any], *, resolver: Resolver | None=None, settings: Settings | None=None, _seen: frozenset[str]=frozenset())Load and resolve a stack (file path, registry:// reference or inline mapping), imports first.
plan_stackfunction (desired: Stack, current: Stack | None=None)Diff desired against current (None = nothing deployed: everything is an add).
starter_stackfunction (name: str='starter', description: str='')A minimal, valid stack document to start from (what the marketplace publish wizard pre-fills).
validate_stackfunction (doc: Any)Validate a stack document; returns human-readable problems (empty list = valid). Imports are not resolved.

opensmartroute.strategies#

Source: src/opensmartroute/strategies/init.py

NameKindSummary
DEFAULT_RULESre-export of opensmartroute.strategies.protocol.DEFAULT_RULESBuilt-in risk / budget ladder: handoff > debate > aggregate > cascade > single.
AggregateResultre-export of opensmartroute.strategies.aggregate.AggregateResultOutcome of :class:MixtureOfAgents: the final response, participants, winner and agreement ratio.
AnnotatorPoolre-export of opensmartroute.strategies.human.AnnotatorPoolSkill estimation from agreement (Dawid-Skene EM) and quorum selection. Labels usually arrive from.
AnnotatorSkillre-export of opensmartroute.strategies.human.AnnotatorSkillPer-domain Beta accuracy estimate, cost and latency of one human annotator.
AuctionResultre-export of opensmartroute.strategies.auction.AuctionResultWinner, second-price payment and the per-bidder surplus / corrected claims of one auction.
AuctionStrategyre-export of opensmartroute.strategies.auction.AuctionStrategyError-aware reverse auction: bias-corrected bids, highest surplus wins, second-price payment.
BanditStrategyre-export of opensmartroute.strategies.bandit.BanditStrategyThompson-sampling Beta bandit per (context, target); context = dominant domain (+ plan role).
BeliefTrackerre-export of opensmartroute.strategies.cascade.BeliefTrackerAutoMix-style POMDP belief over the hidden state "current answer is correct".
BudgetVariantre-export of opensmartroute.strategies.elastic.BudgetVariantOne budget of an elastic model. quality and cost_scale are relative to the parent.
CacheEntryre-export of opensmartroute.strategies.semantic_cache.CacheEntryA cached response keyed by the embedding of its prompt, with source target, quality, timestamp and hits.
CacheHitre-export of opensmartroute.strategies.semantic_cache.CacheHitThe matched :class:CacheEntry and its similarity to the query.
CapabilityStrategyre-export of opensmartroute.strategies.capability.CapabilityStrategyDeclarative fit: domain / action overlap, complexity band, language, modality and quality prior.
Cascadere-export of opensmartroute.strategies.cascade.CascadeExecute ranked targets in planner order (cheapest / MDP / POMDP), stopping when the quality gate passes.
CascadePlannerre-export of opensmartroute.strategies.cascade.CascadePlannerFinite-horizon MDP over an ordered cascade with a stop action after each step.
CascadeResultre-export of opensmartroute.strategies.cascade.CascadeResultFinal response of a cascade with every step, the planned order and the planner's expected value.
CascadeStepre-export of opensmartroute.strategies.cascade.CascadeStepOne executed rung of a cascade: gate quality, latency, cost, acceptance and POMDP belief.
DeferStrategyre-export of opensmartroute.strategies.defer.DeferStrategyLearning-to-defer: scores human targets by risk, PII, escalation intent, frustration and model uncertainty.
EdgeCloudStrategyre-export of opensmartroute.strategies.edge.EdgeCloudStrategyEdge vs cloud tier choice: learned edge competence per complexity bucket vs upload / decode penalties.
EffortStrategyre-export of opensmartroute.strategies.defer.EffortStrategyMatch a target's reasoning effort_level to signals.reasoning_need; penalise over- and under-thinking.
EscalationDecisionre-export of opensmartroute.strategies.escalation.EscalationDecisionVerdict after a streamed chunk: continue / escalate / done with the competence estimate and reason.
EscalationResultre-export of opensmartroute.strategies.modality.EscalationResultOutcome of :class:ModalityEscalation: which target answered, whether it escalated, confidence, cost.
HiddenStateStrategyre-export of opensmartroute.strategies.probe.HiddenStateStrategyDirichlet probe over a dense request representation; confidence drops with epistemic uncertainty.
HumanRoutingStrategyre-export of opensmartroute.strategies.human.HumanRoutingStrategyScores HUMAN candidates by estimated accuracy in the request's domain.
ImportanceGatere-export of opensmartroute.strategies.memory.ImportanceGateHashed logistic gate: P(item will be used later | text, hint).
LLMJudgeStrategyre-export of opensmartroute.strategies.llm_judge.LLMJudgeStrategyLLM-as-router with optional score calibration.
MemoryItemre-export of opensmartroute.strategies.memory.MemoryItemOne stored memory: text, size, turn written, learned importance gate, tier and usage counters.
MemoryRouterre-export of opensmartroute.strategies.memory.MemoryRouterRoutes memory writes to tiers under budgets and recalls the most valuable items per token.
MemoryTierre-export of opensmartroute.strategies.memory.MemoryTierA storage tier: token capacity, read / write cost per token, latency and the minimum item value it accepts.
MixtureOfAgentsre-export of opensmartroute.strategies.aggregate.MixtureOfAgentsRoute-or-aggregate switch: below a confidence threshold call the top-k alternatives and aggregate.
ModalityEscalationre-export of opensmartroute.strategies.modality.ModalityEscalationTry the text-only target on the surrogate first; escalate to the multimodal target on low confidence.
ModalityStrategyre-export of opensmartroute.strategies.modality.ModalityStrategyCoverage of the request's modalities by each candidate, with surrogate discounts.
MultiRoundExecutorre-export of opensmartroute.strategies.progress.MultiRoundExecutorRouter-R1 style: keep routing/executing until the judge accepts or rounds run out.
ProgressRouterre-export of opensmartroute.strategies.progress.ProgressRouterRoute each step of a task with trajectory context.
ProtocolChoicere-export of opensmartroute.strategies.protocol.ProtocolChoiceThe protocol picked for a request, the risk that drove it, the reason and the matching rule.
ProtocolPolicyre-export of opensmartroute.strategies.protocol.ProtocolPolicyOrdered rule table from (risk, budget, task type) to a protocol, with per-protocol ledgers.
ProtocolRulere-export of opensmartroute.strategies.protocol.ProtocolRuleFirst matching rule wins. None bounds are open.
QuorumPlanre-export of opensmartroute.strategies.human.QuorumPlanA chosen set of annotators with the quorum's majority accuracy, total cost and latency.
RecallResultre-export of opensmartroute.strategies.memory.RecallResultItems recalled for a turn with their total tokens, read cost and latency.
Roundre-export of opensmartroute.strategies.progress.RoundOne route -> execute -> judge iteration of :class:MultiRoundExecutor.
RoundResultre-export of opensmartroute.strategies.progress.RoundResultFinal response of a multi-round run with every round and the shared task id.
Rulere-export of opensmartroute.strategies.rules.RuleIf all when conditions match, boost prefer targets and penalise avoid.
RulesStrategyre-export of opensmartroute.strategies.rules.RulesStrategyApplies declarative :class:Rule preferences (prefer / avoid / pin) when their when conditions match.
SelfEscalationre-export of opensmartroute.strategies.escalation.SelfEscalationStreaming competence monitor with Bayesian optimal stopping.
SemanticCachere-export of opensmartroute.strategies.semantic_cache.SemanticCacheEmbedding-keyed LRU cache with TTL and a similarity threshold. Safe to share across threads:.
SemanticCacheStrategyre-export of opensmartroute.strategies.semantic_cache.SemanticCacheStrategyScores the cache target by calibrated hit quality; leaves real targets to the other strategies.
SessionAffinityStrategyre-export of opensmartroute.strategies.session.SessionAffinityStrategyPrefer the target already serving context["session_id"] unless the intent shifted or it failed.
SessionStatere-export of opensmartroute.strategies.session.SessionStateWhat the strategy remembers about one session.
SimilarityStrategyre-export of opensmartroute.strategies.similarity.SimilarityStrategyEmbed the request and each target's examples / description; score by best and top-k mean similarity.
SpeculativeCascadere-export of opensmartroute.strategies.speculative.SpeculativeCascadeTwo-target cascade that overlaps the draft and the strong call when it pays.
SpeculativeResultre-export of opensmartroute.strategies.speculative.SpeculativeResultOutcome of a speculative run: chosen mode, whether the draft was accepted, latency and cost.
Strategyre-export of opensmartroute.strategies.base.StrategyScores each candidate target in [0, 1] and explains why.
StreamResultre-export of opensmartroute.strategies.escalation.StreamResultText consumed by :func:wrap_stream, whether it escalated and the final decision.
TaskProgressre-export of opensmartroute.strategies.progress.TaskProgressWhere a multi-step task stands: step index, budget spent, failures in a row, last target, history.
TaskTableStrategyre-export of opensmartroute.strategies.task_table.TaskTableStrategyStatic task_type -> target -> quality table with family and prior fallbacks; learns from outcomes.
TokenBudgetStrategyre-export of opensmartroute.strategies.elastic.TokenBudgetStrategyScore budgeted siblings by fit between the budget and the tokens the answer needs.
decide_effortre-export of opensmartroute.strategies.defer.decide_effortPick the effort level whose numeric value is closest to the reasoning need.
default_bidre-export of opensmartroute.strategies.auction.default_bidCatalogue-derived bid: quality prior as the claim, unit cost per 1k tokens x tokens as the price.
default_strategiesfunction (seed: int | None=None, settings: Settings | None=None, state_dir: str | Path | None=None)The zero-configuration ensemble: capability fit + example similarity + Thompson bandit.
expand_elasticre-export of opensmartroute.strategies.elastic.expand_elasticCreate one sibling target per budget variant, sharing the parent's family, handler and metadata.
expected_mode_costsre-export of opensmartroute.strategies.speculative.expected_mode_costsExpected weighted cost (objective.cost x USD + objective.latency x seconds) per mode.
failure_riskre-export of opensmartroute.strategies.protocol.failure_riskRisk in [0, 1] that a single call fails: low confidence, high complexity and reasoning need,.
hashing_embedderre-export of opensmartroute.strategies.similarity.hashing_embedderWord + bigram hashing embedder. Deterministic, zero deps, decent for routing.
majority_votere-export of opensmartroute.strategies.aggregate.majority_voteLargest meaning cluster wins; returns (answer, winner_target_id, agreement).
quorum_accuracyre-export of opensmartroute.strategies.human.quorum_accuracyP(weighted majority is correct) for independent annotators with the given accuracies.
request_modalitiesre-export of opensmartroute.strategies.modality.request_modalitiesModalities a request carries and which of them have a text surrogate in context.
wrap_streamre-export of opensmartroute.strategies.escalation.wrap_streamConsume chunks until the monitor escalates or the stream ends.

opensmartroute.strategies.aggregate#

Source: src/opensmartroute/strategies/aggregate.py

Routing / aggregation switch (Mixture-of-Agents, Wang et al. 2024; JiSi 2601.01330).

NameKindSummary
AggregateResultclassOutcome of :class:MixtureOfAgents: the final response, participants, winner and agreement ratio.
Aggregatorconstant(request, [(target_id, answer)]) -> answer.
MixtureOfAgentsclassRoute-or-aggregate switch: below a confidence threshold call the top-k alternatives and aggregate.
ParticipantclassOne target's contribution to an aggregate: response, cost, latency, error and whether it agreed.
majority_votefunction (request: RouteRequest, answers: list[tuple[str, Any]], embedder: Embedder | None=None, threshold: float=0.85)Largest meaning cluster wins; returns (answer, winner_target_id, agreement).

opensmartroute.strategies.auction#

Source: src/opensmartroute/strategies/auction.py

Error-aware reverse auction across providers (EA-RAM 2608.12719).

NameKindSummary
AuctionResultclassWinner, second-price payment and the per-bidder surplus / corrected claims of one auction.
AuctionStrategyclassError-aware reverse auction: bias-corrected bids, highest surplus wins, second-price payment.
BidFnconstant-> (claimed P(success), price).
BidderRecordclassCalibration ledger of one bidder: signed claim bias and a Beta record of realised success.
default_bidfunction (target: RouteTarget, request: RouteRequest, signals: Signals)Catalogue-derived bid: quality prior as the claim, unit cost per 1k tokens x tokens as the price.

opensmartroute.strategies.bandit#

Source: src/opensmartroute/strategies/bandit.py

Online-learning strategy: contextual Thompson-sampling bandit.

NameKindSummary
BanditStrategyclassThompson-sampling Beta bandit per (context, target); context = dominant domain (+ plan role).

opensmartroute.strategies.base#

Source: src/opensmartroute/strategies/base.py

Strategy interface.

NameKindSummary
StrategyclassScores each candidate target in [0, 1] and explains why.
role_offunction (signals: Signals)Plan-slot role (persona / skill / llm) the router is currently filling, or.
memory_keyfunction (request_id: str, role: str | None)Key for per-request strategy memory: the same request is scored once per plan role, and.

opensmartroute.strategies.capability#

Source: src/opensmartroute/strategies/capability.py

Capability-fit strategy: match request signals to a target's declared capabilities.

NameKindSummary
CapabilityStrategyclassDeclarative fit: domain / action overlap, complexity band, language, modality and quality prior.

opensmartroute.strategies.cascade#

Source: src/opensmartroute/strategies/cascade.py

Cascade execution (FrugalGPT / Router-R1 multi-round / AutoMix POMDP).

NameKindSummary
CascadeStepclassOne executed rung of a cascade: gate quality, latency, cost, acceptance and POMDP belief.
CascadeResultclassFinal response of a cascade with every step, the planned order and the planner's expected value.
CascadePlannerclassFinite-horizon MDP over an ordered cascade with a stop action after each step.
BeliefTrackerclassAutoMix-style POMDP belief over the hidden state "current answer is correct".
CascadeclassExecute ranked targets in planner order (cheapest / MDP / POMDP), stopping when the quality gate passes.

opensmartroute.strategies.defer#

Source: src/opensmartroute/strategies/defer.py

Defer-to-human and effort ("think or not") strategies.

NameKindSummary
DeferStrategyclassLearning-to-defer: scores human targets by risk, PII, escalation intent, frustration and model uncertainty.
EffortStrategyclassMatch a target's reasoning effort_level to signals.reasoning_need; penalise over- and under-thinking.
decide_effortfunction (signals: Signals, levels: tuple[str, ...]=('none', 'low', 'medium', 'high'))Pick the effort level whose numeric value is closest to the reasoning need.

opensmartroute.strategies.edge#

Source: src/opensmartroute/strategies/edge.py

Edge-cloud token-aware routing (Pro-Router 2608.28726; RelayLLM 2601.05167).

NameKindSummary
EdgeCloudStrategyclassEdge vs cloud tier choice: learned edge competence per complexity bucket vs upload / decode penalties.

opensmartroute.strategies.elastic#

Source: src/opensmartroute/strategies/elastic.py

Token-budget-aware routing to elastic / many-in-one models (Nemotron Elastic 2511.16664; Star 2605.07182).

NameKindSummary
BudgetVariantclassOne budget of an elastic model. quality and cost_scale are relative to the parent.
TokenBudgetStrategyclassScore budgeted siblings by fit between the budget and the tokens the answer needs.
expand_elasticfunction (parent: RouteTarget, variants: list[BudgetVariant | dict[str, Any]])Create one sibling target per budget variant, sharing the parent's family, handler and metadata.

opensmartroute.strategies.escalation#

Source: src/opensmartroute/strategies/escalation.py

Bayesian self-escalation during generation (2608.24087).

NameKindSummary
REFUSALSconstantRefusal phrases that lower the streaming competence posterior.
EscalationDecisionclassVerdict after a streamed chunk: continue / escalate / done with the competence estimate and reason.
SelfEscalationclassStreaming competence monitor with Bayesian optimal stopping.
StreamResultclassText consumed by :func:wrap_stream, whether it escalated and the final decision.
wrap_streamfunction (chunks: Iterable[str], monitor: SelfEscalation | None=None)Consume chunks until the monitor escalates or the stream ends.

opensmartroute.strategies.human#

Source: src/opensmartroute/strategies/human.py

Routing among human annotators and experts (QUORUM 2608.27974; Dawid-Skene 1979).

NameKindSummary
AnnotatorPoolclassSkill estimation from agreement (Dawid-Skene EM) and quorum selection. Labels usually arrive from.
AnnotatorSkillclassPer-domain Beta accuracy estimate, cost and latency of one human annotator.
HumanRoutingStrategyclassScores HUMAN candidates by estimated accuracy in the request's domain.
QuorumPlanclassA chosen set of annotators with the quorum's majority accuracy, total cost and latency.
quorum_accuracyfunction (accuracies: Sequence[float])P(weighted majority is correct) for independent annotators with the given accuracies.

opensmartroute.strategies.llm_judge#

Source: src/opensmartroute/strategies/llm_judge.py

Generative routing: let an LLM act as the router (Router-R1 / LLM-as-judge).

NameKindSummary
PROMPTconstantPrompt template for the judge; formatted with complexity, domains, actions, the catalogue and the request.
LLMJudgeStrategyclassLLM-as-router with optional score calibration.

opensmartroute.strategies.memory#

Source: src/opensmartroute/strategies/memory.py

Memory-tier routing for agents (BudgetMem 2602.06025; Gated-Memory Routing 2609.00237).

NameKindSummary
ImportanceGateclassHashed logistic gate: P(item will be used later | text, hint).
MemoryItemclassOne stored memory: text, size, turn written, learned importance gate, tier and usage counters.
MemoryRouterclassRoutes memory writes to tiers under budgets and recalls the most valuable items per token.
MemoryTierclassA storage tier: token capacity, read / write cost per token, latency and the minimum item value it accepts.
RecallResultclassItems recalled for a turn with their total tokens, read cost and latency.

opensmartroute.strategies.modality#

Source: src/opensmartroute/strategies/modality.py

Multimodal routing and modality escalation (LatentRouter 2605.11301; modality escalation.

NameKindSummary
EscalationResultclassOutcome of :class:ModalityEscalation: which target answered, whether it escalated, confidence, cost.
ModalityEscalationclassTry the text-only target on the surrogate first; escalate to the multimodal target on low confidence.
ModalityStrategyclassCoverage of the request's modalities by each candidate, with surrogate discounts.
request_modalitiesfunction (request: RouteRequest)Modalities a request carries and which of them have a text surrogate in context.

opensmartroute.strategies.probe#

Source: src/opensmartroute/strategies/probe.py

Hidden-state routing with a Dirichlet probe (ProbeDirichlet, RouterXBench 2602.11877).

NameKindSummary
HiddenStateStrategyclassDirichlet probe over a dense request representation; confidence drops with epistemic uncertainty.
StateFnconstantrequest -> dense feature / hidden-state vector.

opensmartroute.strategies.progress#

Source: src/opensmartroute/strategies/progress.py

Agentic trajectories: per-step routing and multi-round execution.

NameKindSummary
MultiRoundExecutorclassRouter-R1 style: keep routing/executing until the judge accepts or rounds run out.
ProgressRouterclassRoute each step of a task with trajectory context.
RoundclassOne route -> execute -> judge iteration of :class:MultiRoundExecutor.
RoundResultclassFinal response of a multi-round run with every round and the shared task id.
TaskProgressclassWhere a multi-step task stands: step index, budget spent, failures in a row, last target, history.

opensmartroute.strategies.protocol#

Source: src/opensmartroute/strategies/protocol.py

Collaboration-protocol selection (2608.14927).

NameKindSummary
DEFAULT_RULESconstantBuilt-in risk / budget ladder: handoff > debate > aggregate > cascade > single.
PROTOCOLSconstantin escalation order.
Protocolconstantexecution protocols.
ProtocolChoiceclassThe protocol picked for a request, the risk that drove it, the reason and the matching rule.
ProtocolPolicyclassOrdered rule table from (risk, budget, task type) to a protocol, with per-protocol ledgers.
ProtocolRuleclassFirst matching rule wins. None bounds are open.
failure_riskfunction (decision: RouteDecision | None, signals: Signals | None=None, uncertainty: float | None=None)Risk in [0, 1] that a single call fails: low confidence, high complexity and reasoning need,.

opensmartroute.strategies.rules#

Source: src/opensmartroute/strategies/rules.py

Declarative rule-based routing (Arch-Router-style domain/action preferences).

NameKindSummary
RuleclassIf all when conditions match, boost prefer targets and penalise avoid.
RulesStrategyclassApplies declarative :class:Rule preferences (prefer / avoid / pin) when their when conditions match.

opensmartroute.strategies.semantic_cache#

Source: src/opensmartroute/strategies/semantic_cache.py

Semantic caching as a routing target (GPTCache; vLLM semantic router 2603.04444).

NameKindSummary
CacheEntryclassA cached response keyed by the embedding of its prompt, with source target, quality, timestamp and hits.
CacheHitclassThe matched :class:CacheEntry and its similarity to the query.
SemanticCacheclassEmbedding-keyed LRU cache with TTL and a similarity threshold. Safe to share across threads:.
SemanticCacheStrategyclassScores the cache target by calibrated hit quality; leaves real targets to the other strategies.

opensmartroute.strategies.session#

Source: src/opensmartroute/strategies/session.py

Session affinity: keep a conversation with the target that is already serving it.

NameKindSummary
SessionAffinityStrategyclassPrefer the target already serving context["session_id"] unless the intent shifted or it failed.
SessionStateclassWhat the strategy remembers about one session.

opensmartroute.strategies.similarity#

Source: src/opensmartroute/strategies/similarity.py

Similarity-based routing (UniRoute / GraphRouter flavour).

NameKindSummary
hashing_embedderfunction (dim: int=512)Word + bigram hashing embedder. Deterministic, zero deps, decent for routing.
cosinefunction (a: list[float], b: list[float])Cosine similarity; zero vectors are treated as unit norm (no division by zero).
SimilarityStrategyclassEmbed the request and each target's examples / description; score by best and top-k mean similarity.

opensmartroute.strategies.speculative#

Source: src/opensmartroute/strategies/speculative.py

Speculative (draft-based) cascades (speculative cascades, Narasimhan et al. 2024; Differential.

NameKindSummary
SpeculativeCascadeclassTwo-target cascade that overlaps the draft and the strong call when it pays.
SpeculativeResultclassOutcome of a speculative run: chosen mode, whether the draft was accepted, latency and cost.
expected_mode_costsfunction (p_accept: float, draft: RouteTarget, strong: RouteTarget, objective: Objective, *, tokens: int=500, cancellable: bool=False, verify_ms: float=0.0)Expected weighted cost (objective.cost x USD + objective.latency x seconds) per mode.

opensmartroute.strategies.task_table#

Source: src/opensmartroute/strategies/task_table.py

Static task table strategy (SCX Router, 2609.02292).

NameKindSummary
TaskTableStrategyclassStatic task_type -> target -> quality table with family and prior fallbacks; learns from outcomes.