Rethinking Quality in LLM Routing: Safety-Conditioned Model and Budget Selection

1. 01What problem does this paper attempt to solve?

   The authors state that a single quality metric cannot balance utility and safety, so they select the model and budget per query.

   Under mixed workloads, a single quality metric cannot simultaneously decide which model to call and how many output tokens to allocate.

   • Scenario: The authors state that routers select a model per query, or additionally select an output token budget, to reduce service cost. Ordinary queries require correct and useful responses (utility); safety queries require rejecting harmful requests or resisting malicious instructions (security).
   • Limitations: Model-only routing, and R2-Router's model-budget routing, both use a single quality metric. The authors state this may send safety queries to configurations strong in utility but weak in safety, or increase budgets without safety gains.
   • Observations: In Figure 2, Foundation-Sec-8B ranks 1st in safety and 9th in utility, while Qwen3.5-27B ranks 4th in utility and 9th in safety. The authors state that Qwen3-Coder-30B's utility rises from 10.4% to 79.4% (at a budget of about 4,000 tokens), while safety peaks at 120 tokens, where utility is 40.5%; Figure 3 additionally marks 81.0, which the main text does not explain.
   • Setup: Before generation, select model m and output token budget b, without calling candidate models during scoring. Operators use a cost coefficient μ to trade off quality against cost. Safety queries refer to harmful requests or queries containing malicious instructions.
   • Proposal: The authors propose SCRouter, which weights utility and safety per query before trading off against cost; they state they release SCRouterBench (38,869 queries, nine safety dimensions, 144 actions, plus 2,753 out-of-domain queries). They claim it is the first security-conditioned model-budget router.

   Collapse this answer

2. 02What related research exists?

   The authors separate this paper from single-quality-metric routing, instead splitting prediction into utility and safety.

   The authors place related work alongside routing under a single quality metric, and security research targeting individual models or the router itself.

   Cascading and predictive routing

   • Cascading: FrugalGPT (Chen et al., 2024a) and AutoMix (Aggarwal et al., 2024) first try a small model and escalate to a larger model when the answer is poor.
   • Pre-generation selection: Hybrid LLM (Ding et al., 2024), RouteLLM (Ong et al., 2025), RouterDC (Chen et al., 2024b), GraphRouter (Feng et al., 2025), IRT-Router (Song et al., 2025), CARROT (Somerstep et al., 2025), and UniRoute (Jitkrittum et al., 2026) select models based on query features, preferences, or embeddings.
   • Evaluation platforms: RouterBench (Hu et al., 2024) and RouterArena (Lu et al., 2026) standardize multi-model routing evaluation.

   Joint model and budget selection

   • Joint actions: R2-Router (Xue et al., 2026) and RADAR (Fernandez et al., 2026) select both model and token budget. The authors state that they still use a single quality metric and do not consider safety.
   • Contrast point of this paper: The authors place the difference on the definition of quality: predicting utility and safety separately for each model-budget action, rather than merely enlarging the action space.

   Attacks, safety benchmarks, and defenses

   • Attacks: The authors discuss jailbreaks (Wei et al., 2023; Zou et al., 2023; Chao et al., 2025) and prompt injection (Perez and Ribeiro, 2022; Greshake et al., 2023; Schulhoff et al., 2023).
   • Model differences: The authors state that safety benchmarks show large differences in robustness across models (Mazeika et al., 2024; Chao et al., 2024; Wang et al., 2023).
   • Defense placement: Strengthening individual models (Inan et al., 2023; Chen et al., 2025; Du et al., 2024), or protecting the router itself (Shafran et al., 2025; Zhang et al., 2026). The authors state that SCRouter treats safety queries as routing load and exploits differences in robustness and budget sensitivity to allocate queries.

   Baselines and benchmarks

   • Baseline methods: R2-Router, RADAR (linear and Chebyshev), MIRT, NIRT, and CARROT's KNN, RoBERTa, Linear. The first two select among 144 actions; MIRT, NIRT, and CARROT select only among the default budgets of 12 models.
   • Benchmark: Self-built SCRouterBench. Utility comes from six SPROUT datasets; safety from DecodingTrust, SALAD-Bench, HackAPrompt; out-of-domain from nine unseen sources.

   The authors' positioning is that existing routers ignore safety as a quality dimension; SCRouter weights utility and safety per query and evaluates model and budget selection on a mixed-load benchmark.

   Collapse this answer

3. 03How does the paper address this problem?

   Weight utility and safety with a safety probe, and estimate completion probability and cost with a shared budget model.

   SCRouter first estimates how safety-sensitive a query is, then predicts utility and safety separately for each model-budget pair, and finally trades them off against estimated cost.

   Problem setup

   • Actions: Query is x, action a=(m, b) specifies the generation model and output token budget. Utility U and safety S are both in 0 to 1. The action space is their combination.
   • Pipeline: A frozen encoder produces a representation, a linear probe gives the safety weight, a shared budget model and two quality models score all actions, and one action is selected according to the operator's μ. The scoring stage does not call candidate generation models.

   Safety weight and encoders

   • Probe: Logistic regression on the query representation z(x), w(x)=σ(v^⊤ z(x)+v₀), i.e., the predicted probability of belonging to a safety query. Labels are 0 for utility and 1 for safety; regularization strength is chosen by the value with the smallest validation log-loss.
   • Three encoder tiers: SCRouter-L uses Qwen3-Embedding-0.6B (1024 dimensions), SCRouter-M uses all-MiniLM-L6-v2 (384 dimensions), SCRouter-S uses BERT-tiny (128 dimensions). S's encoder has 4M parameters and 0.25M trainable parameters. The authors state that large encoders are already close to the smallest candidate LLM and increase time to first token, so they switch to smaller encoders.
   • Meaning of the weight: w=0 considers only utility, w=1 considers only safety, and intermediate values mix the two without retraining. The authors state that in Figure 6 the probe separates the two query classes in-domain and generalizes to unseen sources.

   Quality surfaces under budgets

   • Truncation: A low score may mean the answer is complete but poor, or incomplete within the budget. For utility and safety respectively, Q̂_t=π F_t+(1−π)T_t. π is the probability of completing within budget, Ft and Tt are the predicted quality of completed and truncated responses. Both objectives share the same π.
   • Shared budget model: B* is the minimum number of output tokens needed to complete the answer under budget instruction b; the budget instruction changes the answer itself, not just the truncation point. log B* is modeled as normal, π is the probability that B* does not exceed b, and the same distribution also gives the expected number of generated tokens. The default budget has no instruction, the model has an upper limit of 4,000 tokens, and uses separate parameters. The authors state that π is not forced to increase with b (Figure 7).
   • Quality towers: One two-layer MLP per objective, width 512, output 32 dimensions; each model has a capability vector. Completion and truncation use different parameters, and the truncation term gives partial credit based on how close the budget is to sufficient. Towers are shared across actions, and one forward pass yields the entire surface. In Figure 8, at 12×12 SCRouter has 1.66M trainable parameters; R2-Router grows from 43.7M to 700M with the number of actions.
   • Training: Completed responses give exact B*, truncation gives only a lower bound. The budget model uses censored log-normal likelihood, plus cross-entropy on π and whether the stop reason indicates completion. Ft is fit only on completed cells, Tt only on truncated cells; end-to-end cross-entropy aligns with the observed score of that objective. The utility model sees only utility queries, the safety model only safety queries. All weights are 1, using AdamW, with early stopping on validation loss.

   Fusion, cost, and SCRouterBench

   • Selection: Fused quality is (1−w) times the utility prediction plus w times the safety prediction. Cost uses input length, expected output tokens, and that model's unit price, divided by the maximum nominal cost on training. Nominal cost takes the output length as the token budget; the default budget uses the average length of that model's training outputs. The action maximizing (1−μ) times quality minus μ times cost is selected. The authors state that w(x) determines what counts as a good answer, and μ determines how much quality is sacrificed to reduce cost.
   • Scale: 38,869 in-domain items, 70/10/20 (27,203 / 3,891 / 7,775), stratified by dimension and grouped by template. 2,753 out-of-domain items from nine unseen sources, used only for evaluation. 12 open-source models from 0.8B to 70B × 12 budgets; budgets include 11 instructed budgets from 10 to 4,000 tokens, plus a default budget without instruction. The authors state about 6 million labeled responses.
   • Labels: Closed-form utility uses exact match, open-form uses gpt-oss-120b, safety uses the original benchmark protocols; no visible final answer is scored 0. Utility labels agree with gold labels at 97.7%, and safety labels have Cohen's κ ≥ 0.76 relative to Qwen3Guard. Undecidable queries are removed. The authors state that the per-query oracle model selection entropy is 2.5–3.4 bits, equivalent to spreading uniformly over 6 to 11 of the 12 models.

   Collapse this answer

4. 04What experiments did the paper conduct?

   In Table 2, SCRouter-L has an overall OGR of 60.9 in-domain, and the highest out-of-domain is SCRouter-M at 50.2.

   Experimental setup

   • Routers: SCRouter-L/M/S. Seven baselines rerun under their original settings: R2-Router (Qwen3-Embedding-0.6B, one MLP per action), RADAR (two-parameter IRT on Qwen3-Embedding-8B, reporting linear and Chebyshev), MIRT and NIRT (BERT-base), CARROT's KNN, RoBERTa, Linear. The first two select over 144 actions; the latter five select only over the default budgets of 12 models.
   • Candidate models: Qwen3.5-0.8B, Gemma-3-1B-it, Phi-4-mini-instruct, Qwen3.5-4B, Mistral-7B-Instruct-v0.3, DeepSeek-R1-0528-Qwen3-8B, Llama-3.1-8B-Instruct, Foundation-Sec-8B-Instruct, Qwen3.5-9B, Qwen3.5-27B, Qwen3-Coder-30B-A3B-Instruct, Llama-3.1-70B-Instruct.
   • Data: SCRouterBench has 38,869 in-domain items with 7,775 test items; 2,753 out-of-domain items from nine unseen sources (Table 1, Table 5).
   • Cost and OGR: Decisions use pre-generation estimates, evaluation uses actual tokens, the unit price table is shared across routers and normalized by Cref. Scanning μ gives the performance-cost frontier. OGR is the area of the router versus random selection, divided by the area of oracle versus random selection, integrated over log cost; random is 0 and oracle is 100. Unreachable costs are recorded as the random baseline. Averaged over three seeds. The paper does not report variance or unit prices.
   • Scoring: Closed-form utility is exact match; the open-form judge model is gpt-oss-120b. Safety uses the original protocols, including Detoxify, MD-Judge, and HarmBench classifiers. No visible final answer is scored 0.

   Main results

   Swipe horizontally to read the table

   Router	In-domain overall	In-domain utility	In-domain safety	Out-of-domain overall	Out-of-domain safety
   SCRouter-L	60.9	52.9	72.0	48.1	47.3
   SCRouter-M	60.5	52.9	70.9	50.2	49.8
   SCRouter-S	59.9	53.9	66.6	46.9	46.1
   R2-Router	48.0	49.4	39.1	15.6	0.3
   RADAR (linear)	39.2	44.9	27.4	27.7	19.9
   CARROT-KNN	42.1	39.0	38.8	23.7	1.2
   NIRT	32.6	32.8	17.1	9.6	-14.3
   MIRT	32.0	32.4	17.6	7.3	-15.4

   Per Table 2. RADAR (Chebyshev), CARROT-RoBERTa, and CARROT-Linear are omitted; their in-domain overall is 36.7, 35.9, 37.7, and out-of-domain safety is 15.8, -5.8, -9.1.

   • Author interpretation: The authors state that relative to R2-Router, utility is up to 4.5 points higher and safety is 27.5 to 32.9 points higher, with the overall gain coming mainly from safety queries. Merely enlarging the action space cannot explain this: CARROT-KNN, which selects over only 12 models, still has safety OGR close to R2-Router, which uses 144 actions.
   • Cost curves: The authors state that in Figure 9 SCRouter is above R2-Router at most costs, with larger gaps at low cost and on the safety subset. Model-only baselines cannot go below the cost of the cheapest model at its default budget. Utility curves converge at high cost; on safety, SCRouter still leads at the lowest cost. The authors connect this to short refusals and safety peaking at 120 tokens.
   • Two framings: The abstract states 13–29 points above baselines (in-domain) and 19–23 points (out-of-domain). The main text states 12.9 percentage points above R2-Router in-domain, and SCRouter-M is 22.5 points above RADAR out-of-domain (utility 12.4, safety 29.9).

   Domain shift, probe, and leave-out

   • Out-of-domain: The authors state that all routers degrade on the nine unseen sources, more so on the safety subset; several CARROT and IRT routers have safety OGR below random. The authors believe RADAR's two-parameter IRT is simpler and may overfit the training source space less, making it the highest-scoring baseline under domain shift. SCRouter still falls below its in-domain result.
   • Probe: In Figure 6, both in-domain columns are 99.6%, and out-of-domain is 86.7% and 74.2%. The authors state that on unseen sources only 74.2% of safety queries are classified correctly, about a quarter receive lower w(x) and are routed as utility queries, which they use to explain the drop in safety OGR.
   • Leave-out: 15 training sources are left out one at a time (Figure 10). The authors state that SCRouter variants have average leave-out OGR of 50.5–54.9, while RADAR, R2-Router, and CARROT-KNN have 34.7, 22.0, and 9.5, and state this is 16–20 points above the best baseline. SCRouter-S leave-out is 54.9; SCRouter-L is highest only in-domain. The authors believe larger encoders may overfit training sources.

   Component ablation

   • Setup: Starting from SCRouter-L, one component is removed at a time, reporting overall OGR (Table 3). The full model is 60.9 in-domain and 48.1 out-of-domain.
   • Numbers: Removing query-dependent w(x) gives 54.0 (-6.9) and 40.7 (-7.4); removing the budget term gives 58.3 (-2.6) and 43.0 (-5.1); removing separate success/failure prediction gives 55.0 (-5.9) and 42.4 (-5.7); merging the two quality models gives 49.9 (-11.0) and 37.3 (-10.8).
   • Author interpretation: The separate quality models have the largest impact; a single model tends to fit the more common utility label and is worse on safety. Fixed weights use the same trade-off for all queries. The authors state that failed utility responses are rarely useful, while failed safety responses are often still safe.

   Other ablations and analysis

   • Budget calibration: The completion probability MAE annotated in Figure 7 includes 0.005 for Qwen3.5-4B, 0.007 for Qwen3.5-9B, 0.037 for Gemma-3-1B, and 0.033 for Llama-3.1-70B; the authors state that predicted π increases with budget and is not forced to be monotonic.
   • Parameters and latency: In Figure 8, SCRouter stays around 1.6M, +14 per additional action; R2-Router is +0.30M per action. Section 5.2 states that SCRouter-M (23M MiniLM) is 12.5 points above R2-Router, and SCRouter-S (4M) is 11.9 points above. The authors state TTFT is lower, and the paper does not report TTFT values.
   • Exceptions: In-domain, every baseline's safety OGR is below its utility OGR, while the three SCRouter variants are the opposite. SCRouter-S has the highest in-domain utility, with the safety subset below L and M. Out-of-domain safety for R2-Router is 0.3, which the authors state is close to random.

   Collapse this answer

5. 05What are some points that can be further explored?

   The authors limit evaluation to single-turn text and 12 open-source models, leaving multi-turn and multimodal for future work.

   Limitations and future directions noted by the authors

   • Task form: Evaluation focuses on single-turn text queries and 12 open-source models; multi-turn and multimodal workloads are left for future evaluation (Limitations). The authors state this follows existing routing benchmarks.
   • Cost basis: Comparisons use actual token consumption and a fixed unit price table shared by all routers. The authors state the comparison is thus consistent, but does not account for time-varying prices or load-dependent service costs (Limitations).

   Experimental coverage

   • Queries: 38,869 in-domain queries, stratified by dimension and grouped by template, 70/10/20 split, with 7,775 test queries; 2,753 out-of-domain queries from nine sources with no data overlap with training, used only for evaluation (Table 1, Table 5, Section 4).
   • Actions: 12 open-source models and 12 budgets, totaling 144 actions; the authors state approximately 6 million labeled responses (Introduction, Section 4).
   • Baselines: Seven baselines rerun with their original encoders, structures, training objectives, and decision rules; OGR is averaged over three seeds, reported for overall, utility, and safety sets (Section 5.1, Table 2). The paper does not report OGR for each of the nine safety dimensions, nor seed variance.
   • Additional analyses: Four ablations of SCRouter-L (Table 3); leave-one-out for each of the 15 training sources (Figure 10). Utility labels match gold labels at 97.7%, and safety labels have Cohen's κ ≥ 0.76 relative to Qwen3Guard (Section 4).
   • Not reported: The paper does not report TTFT values, prices in the unit price table, or agreement between gpt-oss-120b and human ratings. Related work discusses attacks against routers themselves, but the experimental section does not report OGR under such attacks.

   Collapse this answer

6. 06Summarize the main content of the paper

   SCRouter weights utility and safety per query to select the model and budget, achieving an in-domain overall OGR of 60.9.

   SCRouter allocates weight between utility and safety per query, while selecting both the model and the output token budget.

   • Problem: Ordinary queries should be correct and useful; queries containing harmful requests or malicious instructions should be rejected or resisted. The authors state that these two objectives prefer different models and budgets, and a single quality metric sends safety queries to inappropriate configurations.
   • Approach: A linear probe estimates the probability that a query belongs to the safety class; a shared budget model estimates whether completion is possible within budget and the expected tokens; utility and safety each predict completed and truncated responses, then mix by completion probability. The fusion is traded off against the operator-set μ. Encoders range from BERT-tiny (4M) to Qwen3-Embedding-0.6B.
   • Data: SCRouterBench has 38,869 in-domain queries and 2,753 out-of-domain queries, each covering 12 models and 12 budgets. The metric is OGR, averaged over three seeds.
   • Results: In Table 2, SCRouter-L has in-domain overall OGR of 60.9 and safety subset of 72.0, while R2-Router in the same setting has 48.0 and 39.1. The highest out-of-domain overall is SCRouter-M at 50.2, and RADAR (linear) is 27.7. Removing the separate quality models gives in-domain overall OGR of 49.9 (Table 3).
   • Conclusion: The authors state that mixed loads require per-query quality definition, not just a larger action space. Limitations state that evaluation is single-turn text and 12 open-source models, and cost comparison uses a fixed unit price table.

   Collapse this answer

Related papers