--- name: osr-routing-catalogue description: Define and tune an OpenSmartRoute routing catalogue. Add a new target to targets.yaml (LLM, agent, skill, persona, tool, human) with capabilities, domains, a complexity band, cost, latency and a quality prior. Write a rules.yaml rule that prefers, avoids or pins targets, for example pin PII requests to an on-prem model. Set hard constraints (region, data boundary, tenant, max cost) and objective weights per request. Build plans and close the learn loop with Outcomes. Debug why a request routed to a given target with the trace. Use when editing targets or rules YAML or explaining a routing decision. license: Apache-2.0 compatibility: OpenSmartRoute >= 0.4, Python >= 3.10 metadata: author: opensmartroute osr-domains: "coding general" osr-tags: "opensmartroute routing catalogue" osr-quality-prior: "0.85" osr-primary: "false" --- # OpenSmartRoute routing catalogue Routing is `signals -> policy -> strategies -> ensemble -> decision (+plan)`. The catalogue is data: add a target and it is routable immediately, no retraining. ## Target schema (`targets.yaml` / `.json`, or `TargetRegistry.from_dicts`) Top level is `targets: [...]` (a bare list also works). Keys are exactly the `RouteTarget` fields - unknown keys raise `ConfigurationError`. ```yaml targets: - id: llm-small # required, unique; used in rules, outcomes, plans kind: llm # llm | agent | skill | persona | tool | workflow | human | destination name: Small fast model description: Cheap, fast generalist for simple chat, classification and extraction. capabilities: domains: [general, customer_support] # match signals.domains actions: [qa, classify, extract, summarize] languages: [en, es] # default [en]; "*" = any modalities: [text] # default [text] min_complexity: 0.0 # request complexity band this target fits max_complexity: 0.45 context_window: 128000 # optional; policy rejects oversize inputs supports_tools: true tags: [] constraints: regions: [us, eu] # empty = anywhere data_boundary: public # public | private | on_prem pii_allowed: true max_tokens_in: null tenants: [] # empty = every tenant cost: { usd_per_1k_tokens: 0.0002 } # also wh_per_1k_tokens, gco2_per_1k_tokens latency_ms: 300 quality_prior: 0.55 # 0..1 belief before any outcomes examples: ["Hi, how are you today?"] # anchor the similarity strategy instructions: "" # system prompt fragment, disclosed only when selected primary: true # false = slot-only (persona/skill layered on a model) family: "" # siblings share breaker/budget state effort: "" # reasoning-effort variant label enabled: true metadata: {} # free-form; keys read by optional strategies below ``` `metadata` keys that optional strategies read (all opt-in, none required): - `tier: edge | cloud` - `strategies.EdgeCloudStrategy` (edge decode time vs SLO, cloud upload penalty, cloud excluded for private / on_prem requests). - `input_schema: {properties: {...}, required: [...]}` on `kind: tool` - `discovery.SchemaAwareStrategy` scores how many required parameters the request text can fill (MCP imports set it automatically). - `requires`, `conflicts`, `composes` (lists of skill ids) on `kind: skill` - `discovery.SkillGraph`; SKILL.md packages declare them as `osr-requires` / `osr-conflicts` / `osr-composes` frontmatter. - Elastic models: do not hand-write `@` siblings; call `strategies.expand_elastic(parent, [BudgetVariant(tokens, quality=, cost_scale=)])` and add the result; it sets `metadata.token_budget`, which `strategies.TokenBudgetStrategy` scores against the tokens the answer needs. Guidance: - Give every target 2-5 `examples` phrased like real requests; similarity is the cheapest strong signal. - Set `min_complexity`/`max_complexity` bands so the capability strategy can penalise overkill and too-hard routes; overlap bands slightly. - `quality_prior` is a prior, not a score - bandits/IRT move it with `Outcome`s. - Personas and skills are usually `primary: false`; they fill plan slots. - Python handlers: `registry.get("id").handler = fn` where `fn(request, **kw) -> response`. ## Rules (`rules.yaml`, Arch-Router style) ```yaml rules: - name: pii-stays-onprem when: { contains_pii: true } # domains, actions, language, min/max_complexity, prefer: [llm-onprem, human-escalation] # contains_pii, needs_tools, context: {key: value} avoid: [llm-frontier] weight: 1.0 pin: false # true = restrict candidates to `prefer` ``` Rules push scores; they never override policy. There is no `boost` key - use `weight`. ## Hard constraints and objectives (per request) ```python from opensmartroute import RouteRequest, RequestConstraints, Objective req = RouteRequest("Summarise this contract", constraints=RequestConstraints(region="eu", data_boundary="private", max_cost_per_1k=0.01, max_latency_ms=3000, allowed_kinds=["llm"], deny_targets=[], tenant="acme", require_tools=False), objective=Objective(quality=1.0, cost=0.3, latency=0.05, quality_floor=0.6, energy=0.0, carbon=0.0)) ``` Constraints are filtered by `Policy` (rejections appear in `decision.trace.policy_rejections`); objective weights shape the utility `quality - cost*norm(cost) - latency*norm(latency)`. Request `context` keys that strategies read: `user_id` (`learning.UserAdaptiveStrategy`), `last_target` and `last_failed` (`learning.HistoryTargetStrategy` incumbent bonus / penalty), `difficulty` (`signals.VerbalisedDifficultySignal`), `draft` (`signals.DraftResponseSignal`), `session_id` (`MarkovStrategy` transitions). Pass prior turns in `RouteRequest.history=[{"role": ..., "content": ...}]` so multi-turn strategies see them. ## Route, plan, learn ```python from opensmartroute import Router, Outcome, load_targets, load_rules router = Router(load_targets("targets.yaml"), strategies=[load_rules("rules.yaml"), *default_strategies()]) d = router.route(req, plan=True) # d.target, d.confidence, d.alternatives, d.plan.slots, d.trace print(d.trace.explain()) # per-strategy scores for every candidate router.learn(Outcome(d.request_id, d.target.id, success=True, quality=0.9, cost_usd=0.002, latency_ms=1200, domains=d.trace.signals.domains, complexity=d.trace.signals.complexity)) res = router.run(req) # route + execute plan; res.text, res.outcomes, res.system_prompt ``` CLI equivalents: `osr -t targets.yaml -r rules.yaml route "text" [--plan] [--json]`, `osr -t targets.yaml targets`, `osr -t targets.yaml stats`. ## Debugging a surprising route 1. `d.trace.explain()` - check `policy: N -> M candidates` and the rejection reasons first. 2. Compare `capability` (domain/action/complexity fit) vs `similarity` (examples) columns; fix the catalogue (examples, complexity band) before touching weights. 3. Weights and thresholds live in `Settings` (`osr settings`); override with `OSR_WEIGHTS_` or `Router(weights={...})` only after the catalogue is right.