--- title: "Generalising LLM Routing using Past Performance Retrieval: A" description: "The authors propose CONTEXTUAL ROUTER, a meta-evaluation approach. 1 reference." authors: ["Clovis Varangot-Reille", "Christophe Bouvard", "Antoine Gourru"] year: 2026 venue: "Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 4: Student Research Workshop)" source: "https://aclanthology.org/2026.eacl-srw.22/" doi: "10.18653/v1/2026.eacl-srw.22" licence: "CC BY 4.0 (ACL Anthology)" licence_url: "https://creativecommons.org/licenses/by/4.0/" summary_licence: "Elamur Cosmic License" canonical: "https://research.elamur.ai/free/en/papers/reille-2026-generalising-llm-routing-past-performance/" --- [Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/reille-2026-generalising-llm-routing-past-performance/) [home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/13/) / Generalising LLM Routing using Past Performance Retrieval: A Few-Shot Router is Sufficient # Generalising LLM Routing using Past Performance Retrieval: A Few-Shot Router is Sufficient **Authors:** Clovis Varangot-Reille, Christophe Bouvard and Antoine Gourru · **Year:** 2026 · **Venue:** [Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 4: Student Research Workshop)](https://md.research.elamur.ai/free/en/venues/eacl/) ([EACL 2026](https://md.research.elamur.ai/free/en/venues/eacl/2026/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/) **Links:** [Publisher's page](https://aclanthology.org/2026.eacl-srw.22/) · [PDF](https://aclanthology.org/2026.eacl-srw.22.pdf) · [DOI 10.18653/v1/2026.eacl-srw.22](https://doi.org/10.18653/v1/2026.eacl-srw.22) · [OpenAlex](https://openalex.org/W7140101710) **Citations:** 1 citations (OpenAlex, 2026-10-14) ## Summary The paper asks whether LLM routing that adapts to new candidate models without retraining really needs complex architectures such as graph neural networks or learned model representations. The authors propose CONTEXTUAL ROUTER, a meta-evaluation approach. A new query is embedded and matched by cosine similarity against a small history of queries with known per-model scores. The similarities and neighbour scores are then combined through a light attention-style regressor to estimate each candidate's performance. The final model choice maximises estimated performance minus a cost term weighted by a tunable penalty λ. Because the input size depends on k rather than on the number of models, a new LLM can join by being scored on the history set. The method was tested on SPROUT, RouterBench, LiveBench, BigGenBench and EmbedLLM against UMR. It reached quality–cost trade-offs comparable to other generalisable routers. However, plain k-NN averaging did as well or better. The takeaway is that good retrieval over a small performance log is enough for practical, low-resource routing. ## Key points - Five routing benchmarks were used: SPROUT, RouterBench, LiveBench, BigGenBench and EmbedLLM. The main baselines were UMR and K-NN@40, and k = 40 was the default neighbourhood size. - Changing the encoder made no significant difference. The authors compared potion-multilingual-128M (static, 256d), snowflake-arctic-embed-m-v2.0 (768d) and OpenAI text-embedding-3-large (3072d). The static model came close to the best, which was Snowflake's. - In a low-data test on the unseen FusionBench, retrieval-based routers did as well with 1% of the history (n=53) as with 10% (n=535). This held for both original and newly added candidates. UMR, in contrast, degraded at 1%. - To simulate realistic queries, 3000 samples each from RouterBench, EmbedLLM and Sprout were paraphrased with gpt-4o-mini. The resulting changes were small, with ΔPeak between −0.3 and 2.0 and ΔDTO_w between −3.0 and 1.1. - The main practical cost is that a collection of queries, each paired with recorded scores for the candidate LLMs, must be kept available during inference, though it only needs to be very small. *Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.* ## Institutions - [Laboratoire Hubert Curien](https://md.research.elamur.ai/free/en/labs/laboratoire-hubert-curien/) — Research institute, Saint-Etienne, France · 3 works · 364 citations - [Université Jean Monnet](https://md.research.elamur.ai/free/en/universities/universite-jean-monnet/) — University, Saint-Etienne, France · 2 works · 14 citations 📚 Shelves (5): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 18,614 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 17,965 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 17,450 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 17,296 works · [🗄️ Information Systems](https://md.research.elamur.ai/free/en/shelves/computer-science/information-systems/) · 1,730 works 🧭 Topics (2): [🧠 Language models & pretraining](https://md.research.elamur.ai/free/en/topics/language-models/) · 3,894 works · [🔍 Information retrieval & search](https://md.research.elamur.ai/free/en/topics/information-retrieval/) · 552 works ## Cite Cite as: Clovis Varangot-Reille, Christophe Bouvard and Antoine Gourru. 2026. Generalising LLM Routing using Past Performance Retrieval: A Few-Shot Router is Sufficient. Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 4: Student Research Workshop). The paper itself, not this page. BibTeX from the library's record of the work. ```bibtex @inproceedings{varangotreille2026generalising, title = {{Generalising LLM Routing using Past Performance Retrieval: A Few-Shot Router is Sufficient}}, author = {Clovis Varangot-Reille, Christophe Bouvard and Antoine Gourru}, booktitle = {Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 4: Student Research Workshop)}, year = {2026}, doi = {10.18653/v1/2026.eacl-srw.22}, url = {https://doi.org/10.18653/v1/2026.eacl-srw.22} } ``` - [← previous GenAIus at SemEval-2026 Task 8: Beyond Retrieval with Relevance-Aware RAG for Faithful Multi-Turn Generation](https://md.research.elamur.ai/free/en/papers/yeniterzi-2026-genaius-semeval-2026-task-8/) - [next → Generative Personality Simulation via Theory-Informed Structured Interview](https://md.research.elamur.ai/free/en/papers/wang-2026-generative-personality-simulation-theory-informed/) - [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/13/) ## References 1 references, 0 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet. 1. Sumit Kumar Dam, Choong Seon Hong, Yu Qiao, and Chaoning Zhang. 2024. A complete survey on LLM-based AI chatbots. Preprint, arXiv:2406.16937. — [arXiv:2406.16937](https://arxiv.org/abs/2406.16937) [find](https://scholar.google.com/scholar?q=Sumit+Kumar+Dam%2C+Choong+Seon+Hong%2C+Yu+Qiao%2C+and+Chaoning+Zhang%2E+2024%2E+A+complete+survey+on+LLM%2Dbased+AI+chatbots%2E+Preprint%2C+arXiv%3A2406%2E16937%2E+%E2%80%94+%5BarXiv%3A2406%2E16937%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2406%2E16937%29) *Updated 31 October 2026*