Mathematical Foundations

Every routing decision in OpenSmartRoute is the output of explicit, inspectable formulas. This document lists them, the research they come from, and where they live in the code.

1. The routing problem#

Given a request $x$ with signal vector $\phi(x)$ and candidate set $\mathcal{T}$ (after policy filtering), choose

$$ t^* = \arg\max_{t \in \mathcal{T}} ; U(x,t), \qquad U(x,t) = w_q,\hat q(x,t) - w_c,\tilde c_t - w_l,\tilde \ell_t $$

subject to $\hat q(x,t) \ge q_{\min}$ (quality floor, Hybrid-LLM). Costs and latencies are log-min-max normalised so one outlier (a $0.50/1k human queue) doesn't flatten the rest:

$$ \tilde c_t = \frac{\log(1 + s,c_t) - \min_k \log(1 + s,c_k)}{\max_k \log(1 + s,c_k) - \min_k \log(1 + s,c_k)} $$

Code: router.py::_norm, Router._rank.

2. Ensemble quality estimate#

Each strategy $k$ returns a score $s_k(t) \in [0,1]$ and a self-confidence $\kappa_k(t)$. The ensemble is a confidence-weighted mean:

$$ \hat q(x,t) = \frac{\sum_k w_k,\kappa_k(t),s_k(t)}{\sum_k w_k,\kappa_k(t)} $$

Learners report $\kappa = \min(1, n/n_0)$ so they are silent until they have data — this is what lets rules and declared capabilities dominate at cold start and learned models take over as evidence accumulates.

3. Decision confidence#

$$ \text{conf} = \frac{e^{U_1/\tau}}{\sum_j e^{U_j/\tau}}, \quad \tau = 0.1 $$

Low confidence (or high normalised entropy $H/\log|\mathcal T|$) triggers the LLM-judge escalation (Router-R1 pattern). Calibration is measured with ECE (math.estimators.expected_calibration_error).

4. Online learning (auto-learn)#

4.1 Thompson sampling (Beta–Bernoulli) — math.bandits.ThompsonBeta#

$\theta_t \sim \mathrm{Beta}(\alpha_t,\beta_t)$; on reward $r\in[0,1]$: $\alpha_t \mathrel{+}= r$, $\beta_t \mathrel{+}= 1-r$. Optional forgetting $\alpha \leftarrow \alpha_0 + \gamma(\alpha-\alpha_0)$ for non-stationarity.

4.2 UCB1 — math.bandits.UCB1#

$\hat\mu_t + c\sqrt{2\ln n / n_t}$ (Auer, Cesa-Bianchi & Fischer 2002).

4.3 LinUCB (contextual) — math.bandits.LinUCB, learning.LinUCBStrategy#

Per arm $A_t = I + \sum x x^\top$, $b_t = \sum r x$, $\hat\theta_t = A_t^{-1} b_t$, score $x^\top \hat\theta_t + \alpha\sqrt{x^\top A_t^{-1} x}$ (Li et al., WWW 2010). Context $x$ = learning.signal_vector (complexity, length, PII, jailbreak, tools, language, multimodal, domain one-hots). This is the MixLLM formulation.

4.4 Cost-aware bandit (Lagrangian) — math.bandits.CostAwareBandit#

Score $= \text{inner}(t) - \lambda c_t$ with dual ascent $\lambda \leftarrow \max(0, \lambda + \eta(\bar c - B))$ so average spend tracks budget $B$ (C2MAB-V, Dai et al. 2024).

4.5 Item Response Theory (2PL) — math.irt.IRTModel, learning.IRTStrategy#

$$P(\text{success}\mid t,i) = \sigma\big(a_i(\theta_t - b_i)\big)$$ Target ability $\theta_t$, item (domain × difficulty bucket) difficulty $b_i$, discrimination $a_i$; fitted by online SGA on Bernoulli log-likelihood with L2. Interpretable and cold-start friendly (IRT-Router, Song et al., ACL 2025).

4.6 Bradley–Terry / Elo — math.preference, learning.PreferenceStrategy#

$$P(i \succ j) = \sigma(\xi_i - \xi_j)$$ Learned per domain → a prompt-specific leaderboard (Prompt-to-Leaderboard, Frick et al. 2025; RouteLLM preference data). Selection probability via Luce: $P(t) = e^{\xi_t}/\sum_k e^{\xi_k}$.

4.7 Markov chain & MDP — math.markov, learning.MarkovStrategy#

Conversation state $s$ = dominant domain. Transition matrix with Dirichlet smoothing $$P_{ij} = \frac{n_{ij} + \alpha}{\sum_k n_{ik} + \alpha|S|}$$ $k$-step prediction $\pi P^k$, first-passage probability to escalate, stationary distribution by power iteration, next-state entropy. Routing MDP (value iteration): $$V(s) = \max_a\Big[R(s,a) + \gamma\sum_{s'}P(s'|s)V(s')\Big]$$ MarkovStrategy scores $(1-\beta),Q(s,a) + \beta\sum_{s'}P(s'|s),Q(s',a)$ — prefer targets that are also good for where the conversation is going.

4.8 Drift detection — math.estimators#

• Page–Hinkley: $m_T=\sum_{t\le T}(x_t-\bar x_t-\delta)$, alarm if $\max_t m_t - m_T > \lambda$.
• ADWIN-lite: split window, Hoeffding bound $\epsilon=\sqrt{\tfrac{1}{2m}\ln\tfrac{4}{\delta}}$. AutoLearner flags drifted targets; HealthStrategy/HealthPolicy can demote them.

4.9 Uncertainty on success rates#

Wilson interval lower bound is used as the reliability estimate so a target with 2/2 successes is not considered 100 % reliable.

5. Multi-objective decisions — math.decision#

• Pareto front over (quality↑, cost↓, latency↓).
• TOPSIS closeness $C = d^-/(d^+ + d^-)$ to the ideal point.
• Weighted sum (the default scalarisation).

6. Real-time capacity — math.decision#

• Erlang-C: $P(\text{wait}) = \dfrac{\frac{A^c}{c!}\frac{1}{1-\rho}}{\sum_{k<c}\frac{A^k}{k!} + \frac{A^c}{c!}\frac{1}{1-\rho}}$, $A=\lambda/\mu$, $\rho=A/c$.
• Mean wait $W_q = P(\text{wait})/(c\mu - \lambda)$; servers_for_sla sizes a human/agent pool.
• Little's law $L=\lambda W$.
• Kingman G/G/1: $W_q \approx \frac{\rho}{1-\rho}\cdot\frac{c_a^2+c_s^2}{2}\cdot\tau$ for bursty LLM endpoints.

7. Resilience state machines — realtime#

Circuit breaker (closed → open after $k$ failures in window $W$ → half-open after $T_r$ → closed on probe success), token-bucket rate limit, rolling-window budget.

8. Similarity — strategies.similarity#

Hashing-trick embeddings (word + bigram, signed hashing, L2-normalised), cosine similarity; score $= 0.6,\tfrac{1+\max}{2} + 0.4,\tfrac{1+\text{mean}_{k}}{2}$ (UniRoute-style example-based target representation). Swap in sentence-transformers for semantic quality.

9. Adversarial robustness — security.InputGuard#

Confounder-gadget score compares head vs tail of the prompt on: alphabetic-token ratio, function-word share, Shannon entropy jump, all-unique tail. Based on the attack in "Rerouting LLM Routers" (Shafran et al., 2025). Signals are computed on the cleaned head.

10. Evidential routing — math.dirichlet.DirichletProbe, strategies.HiddenStateStrategy#

A linear head over a feature vector $x$ (host hidden state, or any embedding) outputs concentrations $\alpha_k = 1 + \mathrm{softplus}(w_k^\top x + b_k)$ of a Dirichlet over the $K$ targets, $S=\sum_k \alpha_k$. Prediction $P(\text{best}=k)=\alpha_k/S$; epistemic uncertainty $u = K/S$ (all evidence near zero gives $u\to 1$). Training minimises the expected cross-entropy under the Dirichlet plus a KL term that removes evidence from wrong classes (Sensoy et al., 2018): $$\mathcal L=\sum_k y_k\big(\psi(S)-\psi(\alpha_k)\big)+\lambda,\mathrm{KL}!\big(\mathrm{Dir}(\tilde\alpha),|,\mathrm{Dir}(\mathbf 1)\big),\qquad \tilde\alpha = y + (1-y)\odot\alpha,$$ by SGD with $\psi$ (digamma) and $\psi'$ (trigamma) in pure Python. HiddenStateStrategy uses $1-u$ as its confidence so an unfamiliar state hands the decision to the other strategies.

11. Response-side uncertainty — signals.uncertainty#

• Semantic entropy (Kuhn et al., 2023): cluster $n$ sampled answers by meaning (cosine over hashed or real embeddings, threshold $\tau$); with cluster mass $p_c$, $H=-\sum_c p_c\log p_c$ (nats; response_uncertainty normalises by $\log n$). Agreement $= \max_c p_c$.
• P(True) (Kadavath et al., 2022): an optional judge returns $P(\text{answer is correct})$.
• response_uncertainty $u = 0.5,\tilde H + 0.3,(1-\text{agreement}) + 0.2,\min(1,\text{hedging})$, and with a judge $u \leftarrow 0.6,u + 0.4,(1-P(\text{True}))$; UncertaintyGate returns $1-u$ as a cascade quality gate. EventTrigger fires named actions on threshold crossings (2607.13048).

12. Optimal stopping during generation — strategies.escalation.SelfEscalation#

Competence $\theta\sim\mathrm{Beta}(a,b)$, prior centred on the target's quality_prior. Each streamed chunk adds pseudo-counts: hedges $+0.8$, refusals $+1.5$, self-corrections $+0.6$, repetition $+1.0$, length overrun $+0.5$ to $b$; clean progress $+0.4$ to $a$. After each chunk $$\text{escalate iff}\quad \theta<\theta_0\ \wedge\ c_{\text{esc}} + c_{\text{tok}}(t + T) ;<; (1-\mathbb E[\theta]),c_{\text{fail}} + c_{\text{tok}},(T-t),$$ where $t$ is tokens decoded, $T$ the expected answer length: restarting elsewhere pays the escalation overhead and regenerates the answer, continuing risks a failed answer (2608.24087).

13. Mixture-cure hand-off — learning.handoff#

A task fails once its cumulative teacher risk $R_t=\sum_{s\le t} r_s$ crosses a latent threshold; a fraction $\pi$ of tasks is cured and never fails; tasks that end without failing are censored. $$P(\text{fail by }R)=(1-\pi)\big(1-e^{-(R/\lambda)^k}\big),\qquad P(\text{uncured}\mid\text{alive at }R)=\frac{(1-\pi)S_W(R)}{\pi+(1-\pi)S_W(R)}.$$ MixtureCureModel.fit() maximises the censored likelihood (density for failures, survival for censored) over a grid in $(\pi,\lambda,k)$ with one refinement. HandoffPolicy hands over permanently when $P(\text{uncured}\mid R)\ge\tau$ or the probability of failing within the next $h$ steps at the trajectory's mean step risk exceeds $\tau_h$ (TACIT-Switch 2608.27911).

14. Error-aware reverse auction — strategies.auction.AuctionStrategy#

Bidder $i$ claims success $\hat p_i$ at price $c_i$. Its claim is corrected by the observed bias $\beta_i=\overline{\hat p_i - r_i}$ and realised rate $\bar r_i$ with shrinkage $w_i=n_i/(n_i+\kappa)$: $\tilde p_i=(1-\tfrac{w_i}{2})(\hat p_i-w_i\beta_i)+\tfrac{w_i}{2}\bar r_i$. Surplus $s_i=\tilde p_i V - c_i$; the highest surplus wins and pays the second price $\max!\big(c_w,\ \tilde p_w V - s_{(2)}\big)$ so over-claiming is not rewarded (EA-RAM 2608.12719).

15. Energy and carbon — math.energy#

Per target, ridge regression on measured samples gives $\mathrm{Wh}=e_0+e_{\text{in}},n_{\text{prompt}}+e_{\text{out}},n_{\text{output}}$ (normal equations, $3\times3$ solve); $\mathrm{gCO_2}=\mathrm{Wh}\cdot g/1000$ with the grid factor $g$. Without a meter, HardwareProfile estimates $\mathrm{Wh}=\mathrm{TDP}\cdot\mathrm{PUE}\cdot (t_{\text{prefill}}+t_{\text{decode}}+t_{\text{fixed}})/3600$ from the accelerator's throughput (2608.28044; HW-Router).

16. Personalisation by shrinkage — learning.personal.UserAdaptiveStrategy#

$\hat p_{u,t}=w_u,\bar\theta_{u,t}+(1-w_u)\big[w_n,\bar\theta_{N(u),t}+(1-w_n),\bar\theta_{\cdot,t}\big]$, $w=\tfrac{n}{n+\kappa}$: a user's own Beta posterior, the similarity-weighted posterior of the $k$ nearest users (cosine over hashed profile vectors) and the global posterior. New users are routed by their neighbours and the crowd and take over after $\kappa$ outcomes (GMTRouter 2511.08590).

17. Multi-turn joint embeddings — learning.multiturn.HistoryTargetStrategy#

History vector $h=\mathrm{normalise}\big(\sum_k \lambda^{k},\phi(m_{n-k})\big)$ over the last $K$ role-tagged messages and the current text ($\phi$ = hashed word/bigram embedding, $\lambda<1$ recency decay); target vector $e_t$ = catalogue embedding. Joint feature $z=\sqrt{d},(h\odot e_t)$ and

$$\hat p(\text{success}\mid h,t)=\sigma\big(w^\top z+b_t\big),$$

a bilinear form with a diagonal weight matrix shared by every target; only $b_t$ is target-specific, so an unseen target is scored through $e_t$ alone (inductive). Online logistic SGD on the reward with L2 shrinkage; $b_t$ is initialised at $\mathrm{logit}(\text{quality_prior})$. Confidence $0.15+0.85,n_t/(n_t+\kappa)$. Incumbent continuity adds $\pm$stay_bonus (MTRouter 2604.23530).

References#

See RESEARCH.md for the full bibliography.