Mathematical Foundations

Every routing decision in OpenSmartRoute is the output of explicit, inspectable formulas. This document lists them, the research they come from, and where they live in the code.

1. The routing problem

Given a request $x$ with signal vector $\phi(x)$ and candidate set $\mathcal{T}$ (after policy filtering), choose

$$t^\* = \arg\max_{t \in \mathcal{T}} \; U(x,t), \qquad U(x,t) = w_q\,\hat q(x,t) - w_c\,\tilde c_t - w_l\,\tilde \ell_t$$

subject to $\hat q(x,t) \ge q_{\min}$ (quality floor, Hybrid-LLM). Costs and latencies are log-min-max normalised so one outlier (a $0.50/1k human queue) doesn't flatten the rest:

$$\tilde c_t = \frac{\log(1 + s\,c_t) - \min_k \log(1 + s\,c_k)}{\max_k \log(1 + s\,c_k) - \min_k \log(1 + s\,c_k)}$$

Code: router.py::_norm, Router._rank.

2. Ensemble quality estimate

Each strategy $k$ returns a score $s_k(t) \in [0,1]$ and a self-confidence $\kappa_k(t)$. The ensemble is a confidence-weighted mean:

$$\hat q(x,t) = \frac{\sum_k w_k\,\kappa_k(t)\,s_k(t)}{\sum_k w_k\,\kappa_k(t)}$$

Learners report $\kappa = \min(1, n/n_0)$ so they are silent until they have data — this is what lets rules and declared capabilities dominate at cold start and learned models take over as evidence accumulates.

3. Decision confidence

$$\text{conf} = \frac{e^{U_1/\tau}}{\sum_j e^{U_j/\tau}}, \quad \tau = 0.1$$

Low confidence (or high normalised entropy $H/\log|\mathcal T|$) triggers the LLM-judge escalation (Router-R1 pattern). Calibration is measured with ECE (math.estimators.expected_calibration_error).

4. Online learning (auto-learn)

4.1 Thompson sampling (Beta–Bernoulli) — math.bandits.ThompsonBeta

$\theta_t \sim \mathrm{Beta}(\alpha_t,\beta_t)$; on reward $r\in[0,1]$: $\alpha_t \mathrel{+}= r$, $\beta_t \mathrel{+}= 1-r$. Optional forgetting $\alpha \leftarrow \alpha_0 + \gamma(\alpha-\alpha_0)$ for non-stationarity.

4.2 UCB1 — math.bandits.UCB1

$\hat\mu_t + c\sqrt{2\ln n / n_t}$ (Auer, Cesa-Bianchi & Fischer 2002).

4.3 LinUCB (contextual) — math.bandits.LinUCB, learning.LinUCBStrategy

Per arm $A_t = I + \sum x x^\top$, $b_t = \sum r x$, $\hat\theta_t = A_t^{-1} b_t$, score $x^\top \hat\theta_t + \alpha\sqrt{x^\top A_t^{-1} x}$ (Li et al., WWW 2010). Context $x$ = learning.signal_vector (complexity, length, PII, jailbreak, tools, language, multimodal, domain one-hots). This is the MixLLM formulation.

4.4 Cost-aware bandit (Lagrangian) — math.bandits.CostAwareBandit

Score $= \text{inner}(t) - \lambda c_t$ with dual ascent $\lambda \leftarrow \max(0, \lambda + \eta(\bar c - B))$ so average spend tracks budget $B$ (C2MAB-V, Dai et al. 2024).

4.5 Item Response Theory (2PL) — math.irt.IRTModel, learning.IRTStrategy

$$P(\text{success}\mid t,i) = \sigma\big(a_i(\theta_t - b_i)\big)$$

Target ability $\theta_t$, item (domain × difficulty bucket) difficulty $b_i$, discrimination $a_i$; fitted by online SGA on Bernoulli log-likelihood with L2. Interpretable and cold-start friendly (IRT-Router, Song et al., ACL 2025).

4.6 Bradley–Terry / Elo — math.preference, learning.PreferenceStrategy

$$P(i \succ j) = \sigma(\xi_i - \xi_j)$$

Learned per domain → a prompt-specific leaderboard (Prompt-to-Leaderboard, Frick et al. 2025; RouteLLM preference data). Selection probability via Luce: $P(t) = e^{\xi_t}/\sum_k e^{\xi_k}$.

4.7 Markov chain & MDP — math.markov, learning.MarkovStrategy

Conversation state $s$ = dominant domain. Transition matrix with Dirichlet smoothing

$$P_{ij} = \frac{n_{ij} + \alpha}{\sum_k n_{ik} + \alpha|S|}$$

$k$-step prediction $\pi P^k$, first-passage probability to escalate, stationary distribution by power iteration, next-state entropy. Routing MDP (value iteration): $V(s) = \max_a\Big[R(s,a) + \gamma\sum_{s'}P(s'|s)V(s')\Big]$MarkovStrategy scores $(1-\beta)\,Q(s,a) + \beta\sum_{s'}P(s'|s)\,Q(s',a)$ — prefer targets that are also good for where the conversation is going.

4.8 Drift detection — math.estimators

• Page–Hinkley: $m_T=\sum_{t\le T}(x_t-\bar x_t-\delta)$, alarm if $\max_t m_t - m_T > \lambda$.
• ADWIN-lite: split window, Hoeffding bound $\epsilon=\sqrt{\tfrac{1}{2m}\ln\tfrac{4}{\delta}}$. AutoLearner flags drifted targets; HealthStrategy/HealthPolicy can demote them.

4.9 Uncertainty on success rates

Wilson interval lower bound is used as the reliability estimate so a target with 2/2 successes is not considered 100 % reliable.

4.10 The routing SLM — learning.contrastive, learning.attention, learning.slm

Dual encoder: query $q = \operatorname{norm}\big(W\,\phi(x) + \operatorname{Attn}(x)\big)$ over hashed word / bigram / character-gram features $\phi(x)$ (and an optional one-block transformer with sinusoidal positions and attention pooling), one learned embedding $e_t$ and bias $b_t$ per target, logit $s_t = q^\top e_t/\tau + b_t$. Losses are softmax cross-entropies over the row's candidate set: multi-positive InfoNCE over the acceptable set $P$ (RouterDC), $\mathcal L = -\log \sum_{p\in P} e^{s_p} / \sum_{c} e^{s_c}$, or Zooter distillation towards $y = \operatorname{softmax}(r/T)$ of per-target scores $r$. Training is SGD with inverse-time decay $\eta_e = \eta_0/(1+e\,\lambda)$, L2 shrinkage, early stopping on a validation split, and two regularisers on the inputs: every row is also presented as augment_views word-dropout copies (each word kept with probability $1-p$, drawn once per fit) and, optionally, labels are smoothed $y \leftarrow (1-\epsilon)\,y + \epsilon/|C|$. Reported probabilities are temperature-scaled on the validation rows, $p = \operatorname{softmax}(s/T^\*)$ with $T^\*$ minimising the NLL of the top pick being acceptable, and SLMStrategy reads its margin and confidence from those calibrated probabilities. Prompts are read up to max_tokens whitespace tokens through a reading window (learning.window): head and tail shares of the budget, the rest for the most instruction-salient sentences of the middle - salience $= (\text{cues} / \sqrt{n} + [\text{asks}]) \cdot ((1-\nu) + \nu\, r)$ with $r$ the share of the sentence's words seen at most twice in the prompt and $\nu$ the novelty weight - placed first; and a long prompt's query is $q = \operatorname{norm}(\hat q_{\text{window}} + a\,\hat q_{\text{ask}})$, the unit vectors of the window view and of the asking sentences alone at weight $a$. The attention block has its own window under the same policy. The numpy path runs the same steps as dense products and draws the same random numbers, so both backends produce the same model file (§ Speed in User guide).

5. Multi-objective decisions — math.decision

• Pareto front over (quality↑, cost↓, latency↓).
• TOPSIS closeness $C = d^-/(d^+ + d^-)$ to the ideal point.
• Weighted sum (the default scalarisation).

6. Real-time capacity — math.decision

• Erlang-C: $P(\text{wait}) = \dfrac{\frac{A^c}{c!}\frac{1}{1-\rho}}{\sum_{k<c}\frac{A^k}{k!} + \frac{A^c}{c!}\frac{1}{1-\rho}}$, $A=\lambda/\mu$, $\rho=A/c$.
• Mean wait $W_q = P(\text{wait})/(c\mu - \lambda)$; servers_for_sla sizes a human/agent pool.
• Little's law $L=\lambda W$.
• Kingman G/G/1: $W_q \approx \frac{\rho}{1-\rho}\cdot\frac{c_a^2+c_s^2}{2}\cdot\tau$ for bursty LLM endpoints.

7. Resilience state machines — realtime

Circuit breaker (closed → open after $k$ failures in window $W$ → half-open after $T_r$ → closed on probe success), token-bucket rate limit, rolling-window budget.

8. Similarity — strategies.similarity

Hashing-trick embeddings (word + bigram, signed hashing, L2-normalised), cosine similarity; score $= 0.6\,\tfrac{1+\max}{2} + 0.4\,\tfrac{1+\text{mean}_{k}}{2}$ (UniRoute-style example-based target representation). Swap in sentence-transformers for semantic quality.

9. Adversarial robustness — security.InputGuard

Confounder-gadget score compares head vs tail of the prompt on: alphabetic-token ratio, function-word share, Shannon entropy jump, all-unique tail. Based on the attack in "Rerouting LLM Routers" (Shafran et al., 2025). Signals are computed on the cleaned head.

10. Evidential routing — math.dirichlet.DirichletProbe, strategies.HiddenStateStrategy

A linear head over a feature vector $x$ (host hidden state, or any embedding) outputs concentrations $\alpha_k = 1 + \mathrm{softplus}(w_k^\top x + b_k)$ of a Dirichlet over the $K$ targets, $S=\sum_k \alpha_k$. Prediction $P(\text{best}=k)=\alpha_k/S$; epistemic uncertainty$u = K/S$ (all evidence near zero gives $u\to 1$). Training minimises the expected cross-entropy under the Dirichlet plus a KL term that removes evidence from wrong classes (Sensoy et al., 2018):

$$\mathcal L=\sum_k y_k\big(\psi(S)-\psi(\alpha_k)\big)+\lambda\,\mathrm{KL}\!\big(\mathrm{Dir}(\tilde\alpha)\,\|\,\mathrm{Dir}(\mathbf 1)\big),\qquad \tilde\alpha = y + (1-y)\odot\alpha,$$

by SGD with $\psi$ (digamma) and $\psi'$ (trigamma) in pure Python. HiddenStateStrategy uses $1-u$ as its confidence so an unfamiliar state hands the decision to the other strategies.

11. Response-side uncertainty — signals.uncertainty

• Semantic entropy (Kuhn et al., 2023): cluster $n$ sampled answers by meaning (cosine over hashed or real embeddings, threshold $\tau$); with cluster mass $p_c$, $H=-\sum_c p_c\log p_c$ (nats; response_uncertainty normalises by $\log n$). Agreement $= \max_c p_c$.
• P(True) (Kadavath et al., 2022): an optional judge returns $P(\text{answer is correct})$.
• response_uncertainty $u = 0.5\,\tilde H + 0.3\,(1-\text{agreement}) + 0.2\,\min(1,\text{hedging})$, and with a judge $u \leftarrow 0.6\,u + 0.4\,(1-P(\text{True}))$; UncertaintyGate returns $1-u$ as a cascade quality gate. EventTrigger fires named actions on threshold crossings (2607.13048).

12. Optimal stopping during generation — strategies.escalation.SelfEscalation

Competence $\theta\sim\mathrm{Beta}(a,b)$, prior centred on the target's quality_prior. Each streamed chunk adds pseudo-counts: hedges $+0.8$, refusals $+1.5$, self-corrections $+0.6$, repetition $+1.0$, length overrun $+0.5$ to $b$; clean progress $+0.4$ to $a$. After each chunk

$$\text{escalate iff}\quad \theta<\theta_0\ \wedge\ c_{\text{esc}} + c_{\text{tok}}(t + T) \;<\; (1-\mathbb E[\theta])\,c_{\text{fail}} + c_{\text{tok}}\,(T-t),$$

where $t$ is tokens decoded, $T$ the expected answer length: restarting elsewhere pays the escalation overhead and regenerates the answer, continuing risks a failed answer (2608.24087).

13. Mixture-cure hand-off — learning.handoff

A task fails once its cumulative teacher risk $R_t=\sum_{s\le t} r_s$ crosses a latent threshold; a fraction $\pi$ of tasks is cured and never fails; tasks that end without failing are censored.

$$P(\text{fail by }R)=(1-\pi)\big(1-e^{-(R/\lambda)^k}\big),\qquad P(\text{uncured}\mid\text{alive at }R)=\frac{(1-\pi)S_W(R)}{\pi+(1-\pi)S_W(R)}.$$

MixtureCureModel.fit() maximises the censored likelihood (density for failures, survival for censored) over a grid in $(\pi,\lambda,k)$ with one refinement. HandoffPolicy hands over permanently when $P(\text{uncured}\mid R)\ge\tau$ or the probability of failing within the next $h$ steps at the trajectory's mean step risk exceeds $\tau_h$ (TACIT-Switch 2608.27911).

14. Error-aware reverse auction — strategies.auction.AuctionStrategy

Bidder $i$ claims success $\hat p_i$ at price $c_i$. Its claim is corrected by the observed bias $\beta_i=\overline{\hat p_i - r_i}$ and realised rate $\bar r_i$ with shrinkage $w_i=n_i/(n_i+\kappa)$: $\tilde p_i=(1-\tfrac{w_i}{2})(\hat p_i-w_i\beta_i)+\tfrac{w_i}{2}\bar r_i$. Surplus $s_i=\tilde p_i V - c_i$; the highest surplus wins and pays the second price $\max\!\big(c_w,\ \tilde p_w V - s_{(2)}\big)$ so over-claiming is not rewarded (EA-RAM 2608.12719).

15. Energy and carbon — math.energy

Per target, ridge regression on measured samples gives $\mathrm{Wh}=e_0+e_{\text{in}}\,n_{\text{prompt}}+e_{\text{out}}\,n_{\text{output}}$ (normal equations, $3\times3$ solve); $\mathrm{gCO_2}=\mathrm{Wh}\cdot g/1000$ with the grid factor $g$. Without a meter, HardwareProfile estimates $\mathrm{Wh}=\mathrm{TDP}\cdot\mathrm{PUE}\cdot (t_{\text{prefill}}+t_{\text{decode}}+t_{\text{fixed}})/3600$ from the accelerator's throughput (2608.28044; HW-Router).

16. Personalisation by shrinkage — learning.personal.UserAdaptiveStrategy

$\hat p_{u,t}=w_u\,\bar\theta_{u,t}+(1-w_u)\big[w_n\,\bar\theta_{N(u),t}+(1-w_n)\,\bar\theta_{\cdot,t}\big]$, $w=\tfrac{n}{n+\kappa}$: a user's own Beta posterior, the similarity-weighted posterior of the $k$ nearest users (cosine over hashed profile vectors) and the global posterior. New users are routed by their neighbours and the crowd and take over after $\kappa$ outcomes (GMTRouter 2511.08590).

17. Multi-turn joint embeddings — learning.multiturn.HistoryTargetStrategy

History vector $h=\mathrm{normalise}\big(\sum_k \lambda^{k}\,\phi(m_{n-k})\big)$ over the last $K$ role-tagged messages and the current text ($\phi$ = hashed word/bigram embedding, $\lambda<1$ recency decay); target vector $e_t$ = catalogue embedding. Joint feature $z=\sqrt{d}\,(h\odot e_t)$ and

$$\hat p(\text{success}\mid h,t)=\sigma\big(w^\top z+b_t\big),$$

a bilinear form with a diagonal weight matrix shared by every target; only $b_t$ is target-specific, so an unseen target is scored through $e_t$ alone (inductive). Online logistic SGD on the reward with L2 shrinkage; $b_t$ is initialised at $\mathrm{logit}(\text{quality\_prior})$. Confidence $0.15+0.85\,n_t/(n_t+\kappa)$. Incumbent continuity adds $\pm$stay_bonus (MTRouter 2604.23530).

References

See Research for the full bibliography.