---
title: "RouteLLM: Learning to Route LLMs with Preference Data (2024)"
description: "To evaluate the cost/quality balance, the paper introduces the metrics PGR, APGR and CPT. 36 references, cited by 100 papers here."
authors: ["Isaac Ong", "Amjad Almahairi", "Vincent Wu", "Wei-Lin Chiang", "Tianhao Wu", "Joseph E. Gonzalez", "M Waleed Kadous", "Ion Stoica"]
year: 2024
venue: "arXiv:2406.18665"
source: "https://arxiv.org/abs/2406.18665"
doi: "10.48550/arxiv.2406.18665"
arxiv: "2406.18665"
licence: "arXiv — non-exclusive license to distribute"
licence_url: "http://arxiv.org/licenses/nonexclusive-distrib/1.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/ong-2024-routellm-learning-route-llms-preference/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/ong-2024-routellm-learning-route-llms-preference/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2024](https://md.research.elamur.ai/free/en/papers/2024/page/4/) / RouteLLM: Learning to Route LLMs with Preference Data

# RouteLLM: Learning to Route LLMs with Preference Data

**Authors:** [Isaac Ong](https://md.research.elamur.ai/free/en/authors/isaac-ong/), [Amjad Almahairi](https://md.research.elamur.ai/free/en/authors/amjad-almahairi/), Vincent Wu, [Wei-Lin Chiang](https://md.research.elamur.ai/free/en/authors/wei-lin-chiang/), [Tianhao Wu](https://md.research.elamur.ai/free/en/authors/tianhao-wu/), [Joseph E. Gonzalez](https://md.research.elamur.ai/free/en/authors/joseph-e-gonzalez/), M Waleed Kadous and [Ion Stoica](https://md.research.elamur.ai/free/en/authors/ion-gabriel-stoica/) · **Year:** 2024 · **Venue:** arXiv:2406.18665 · **Licence:** [arXiv — non-exclusive license to distribute](http://arxiv.org/licenses/nonexclusive-distrib/1.0/)

**Links:** [Publisher's page](https://arxiv.org/abs/2406.18665) · [PDF](https://arxiv.org/pdf/2406.18665) · [DOI 10.48550/arxiv.2406.18665](https://doi.org/10.48550/arxiv.2406.18665) · [arXiv 2406.18665](https://arxiv.org/abs/2406.18665) · [OpenAlex](https://openalex.org/W4400141368)

**Citations:** 9 citations (OpenAlex, 2026-10-14) · [100](https://md.research.elamur.ai/free/en/papers/ong-2024-routellm-learning-route-llms-preference/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
Deploying LLMs forces a trade-off: frontier models such as GPT-4 give the best answers but are costly, while cheaper models like Mixtral-8x7B struggle with harder requests. The authors treat this as binary routing. A lightweight router examines each incoming query and sends it to exactly one model, either the strong or the weak one, so it avoids the extra latency of cascades and ensembles. The router learns from human preference judgments collected in Chatbot Arena, and the training data is enlarged through augmentation. It predicts how likely the strong model is to win on a given query and compares that probability with a tunable threshold α. To evaluate the cost/quality balance, the paper introduces the metrics PGR, APGR and CPT. On decontaminated MT Bench, MMLU and GSM8K, the routers cut costs by more than half while keeping response quality nearly intact. They also keep working when paired with LLMs that were absent from training. The framework is released as open source, which offers a practical way to reduce inference spending.

## Key points
- The router is built from two parts: a win-prediction model P_θ(win_s|q), fit by maximum likelihood on win/tie/loss labels, and a cost threshold α that converts this probability into a routing decision.
- The metrics work as follows:
  - **PGR** measures how much of the weak-to-strong performance gap the router recovers.
  - **APGR** averages PGR across 10 cost levels.
  - **CPT(x%)** gives the smallest share of strong-model calls needed to reach a target PGR; in one example, CPT(50%) ≈ 37% of GPT-4 calls.
- Augmentation (D_arena + D_judge) raises performance for every router architecture tested. Evaluation sets were cleaned by removing items whose text-embedding-3-small cosine similarity to training data was 0.95 or higher.
- The matrix factorization and causal LLM routers beat the commercial routers Unify AI and Martian on MT Bench, matching their quality with up to 40% fewer GPT-4 calls.
- The cost analysis assumes roughly $24.7 per 1M tokens for GPT-4 (gpt-4-1106 pricing) versus $0.24 for Mixtral 8x7B. Chatbot Arena models are grouped into tiers 0–9 to form the strong and weak classes.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

📚 Shelves (5): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 18,614 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 17,965 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 17,450 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 17,296 works · [👁️ Computer Vision and Pattern Recognition](https://md.research.elamur.ai/free/en/shelves/computer-science/computer-vision-and-pattern-recognition/) · 3,486 works

🧭 Topics (1): [🧠 Language models & pretraining](https://md.research.elamur.ai/free/en/topics/language-models/) · 3,894 works

## Cite

Cite as: Isaac Ong, Amjad Almahairi, Vincent Wu, Wei-Lin Chiang, Tianhao Wu, Joseph E. Gonzalez, M Waleed Kadous and Ion Stoica. 2024. RouteLLM: Learning to Route LLMs with Preference Data. arXiv:2406.18665.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@misc{ong2024routellm,
  title = {{RouteLLM: Learning to Route LLMs with Preference Data}},
  author = {Isaac Ong, Amjad Almahairi, Vincent Wu, Wei-Lin Chiang, Tianhao Wu, Joseph E. Gonzalez, M Waleed Kadous and Ion Stoica},
  year = {2024},
  doi = {10.48550/arxiv.2406.18665},
  eprint = {2406.18665},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2406.18665}
}
```

- [← previous jina-embeddings-v3: Multilingual Embeddings With Task LoRA](https://md.research.elamur.ai/free/en/papers/sturua-2024-jina-embeddings-v3-multilingual-embeddings/)
- [next → CharacterEval: A Chinese Benchmark for Role-Playing Conversational Agent Evaluation](https://md.research.elamur.ai/free/en/papers/tu-2024-charactereval-chinese-benchmark-role-playing/)
- [↑ back to papers of 2024](https://md.research.elamur.ai/free/en/papers/2024/page/4/)

## References

36 references, 21 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. [Gpt-4 technical report](https://md.research.elamur.ai/free/en/papers/openai-2023-gpt-4-technical-report/). arXiv preprint arXiv:2303.08774, 2023.
2. Pranjal Aggarwal, Aman Madaan, Ankit Anand, Srividya Pranavi Potharaju, Swaroop Mishra, Pei Zhou, Aditya Gupta, Dheeraj Rajagopal, Karthik Kappaganthu, Yiming Yang, Shyam Upadhyay, Manaal Faruqui, and Mausam. Automix: Automatically mixing language models, 2024. URL https://arxiv.org/abs/2310.12963. — [arXiv:2310.12963](https://arxiv.org/abs/2310.12963) [↗](https://arxiv.org/abs/2310.12963)
3. AI@Meta. Llama 3.1 model card, 2024a. URL [https://github.com/meta-llama/llama-models/blob/main/models/llama3_1/MODEL_CARD.md](https://github.com/meta-llama/llama-models/blob/main/models/llama3_1/MODEL_CARD.md). Accessed: 2024-09-29. [↗](https://github.com/meta-llama/llama-models/blob/main/models/llama3_1/MODEL_CARD.md)
4. AI@Meta. Introducing meta llama 3: The most capable openly available llm to date, 2024b. URL [https://ai.meta.com/blog/meta-llama-3/](https://ai.meta.com/blog/meta-llama-3/). Accessed: 2024-05-21. [↗](https://ai.meta.com/blog/meta-llama-3/)
5. Anthropic. "introducing the next generation of claude", 2024. URL [https://www.anthropic.com/news/claude-3-family](https://www.anthropic.com/news/claude-3-family). Accessed: 2024-05-22. [↗](https://www.anthropic.com/news/claude-3-family)
6. Jinze Bai, Shuai Bai, Yunfei Chu, Zeyu Cui, Kai Dang, Xiaodong Deng, Yang Fan, Wenbin Ge, Yu Han, Fei Huang, Binyuan Hui, Luo Ji, Mei Li, Junyang Lin, Runji Lin, Dayiheng Liu, Gao Liu, Chengqiang Lu, Keming Lu, Jianxin Ma, Rui Men, Xingzhang Ren, Xuancheng Ren, Chuanqi Tan, Sinan Tan, Jianhong Tu, P… — [Qwen Technical Report](https://md.research.elamur.ai/free/en/papers/bai-2023-qwen-technical-report/)
7. Ralph Allan Bradley and Milton E Terry. Rank analysis of incomplete block designs: I. the method of paired comparisons. Biometrika, 39(3/4):324–345, 1952. [find](https://scholar.google.com/scholar?q=Ralph+Allan+Bradley+and+Milton+E+Terry%2E+Rank+analysis+of+incomplete+block+designs%3A+I%2E+the+method+of+paired+comparisons%2E+Biometrika%2C+39%283%2F4%29%3A324%E2%80%93345%2C+1952%2E)
8. Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. [Language models are few-shot learners](https://md.research.elamur.ai/free/en/papers/brown-2020-language-models-few-shot-learners/). Advances in neural information processing systems, 33:1877–1901, 2020.
9. Sébastien Bubeck, Varun Chandrasekaran, Ronen Eldan, Johannes Gehrke, Eric Horvitz, Ece Kamar, Peter Lee, Yin Tat Lee, Yuanzhi Li, Scott Lundberg, et al. [Sparks of artificial general intelligence: Early experiments with gpt-4](https://md.research.elamur.ai/free/en/papers/bubeck-2023-sparks-artificial-general-intelligence-early/). arXiv preprint arXiv:2303.12712, 2023.
10. Lingjiao Chen, Matei Zaharia, and James Zou. [Frugalgpt: How to use large language models while reducing cost and improving performance](https://md.research.elamur.ai/free/en/papers/chen-2023-frugalgpt-use-large-language-models/). arXiv preprint arXiv:2305.05176, 2023.
11. Wei-Lin Chiang, Lianmin Zheng, Ying Sheng, Anastasios Nikolas Angelopoulos, Tianle Li, Dacheng Li, Hao Zhang, Banghua Zhu, Michael Jordan, Joseph E. Gonzalez, and Ion Stoica. [Chatbot arena: An open platform for evaluating llms by human preference](https://md.research.elamur.ai/free/en/papers/chiang-2024-chatbot-arena-open-platform-evaluating/), 2024.
12. Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, et al. [Training verifiers to solve math word problems](https://md.research.elamur.ai/free/en/papers/cobbe-2021-training-verifiers-solve-math-word/). arXiv preprint arXiv:2110.14168, 2021.
13. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. [Bert: Pre-training of deep bidirectional transformers for language understanding](https://md.research.elamur.ai/free/en/papers/devlin-2019-bert-pre-training-deep-bidirectional/). arXiv preprint arXiv:1810.04805, 2018.
14. Dujian Ding, Ankur Mallick, Chi Wang, Robert Sim, Subhabrata Mukherjee, Victor Rühle, Laks V. S. Lakshmanan, and Ahmed Hassan Awadallah. [Hybrid LLM: Cost-efficient and quality-aware query routing](https://md.research.elamur.ai/free/en/papers/ding-2024-hybrid-llm-cost-efficient-quality/). In The Twelfth International Conference on Learning Representations, 2024. URL [https://openreview.net/f…](https://openreview.net/f…) [↗](https://openreview.net/f…)
15. Yann Dubois, Chen Xuechen Li, Rohan Taori, Tianyi Zhang, Ishaan Gulrajani, Jimmy Ba, Carlos Guestrin, Percy S Liang, and Tatsunori B Hashimoto. [Alpacafarm: A simulation framework for methods that learn from human feedback](https://md.research.elamur.ai/free/en/papers/dubois-2023-alpacafarm-simulation-framework-methods-that/). Advances in Neural Information Processing Systems, 36, 2024.
16. Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. [Measuring massive multitask language understanding](https://md.research.elamur.ai/free/en/papers/hendrycks-2020-measuring-massive-multitask-language-understanding/). In International Conference on Learning Representations, 2020.
17. Qitian Jason Hu, Jacob Bieker, Xiuyu Li, Nan Jiang, Benjamin Keigwin, Gaurav Ranganath, Kurt Keutzer, and Shriyash Kaustubh Upadhyay. Routerbench: A benchmark for multi-llm routing system, 2024. URL https://arxiv.org/abs/2403.12031. — [arXiv:2403.12031](https://arxiv.org/abs/2403.12031) [↗](https://arxiv.org/abs/2403.12031)
18. Albert Q Jiang, Alexandre Sablayrolles, Antoine Roux, Arthur Mensch, Blanche Savary, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Emma Bou Hanna, Florian Bressand, et al. Mixtral of experts. arXiv preprint arXiv:2401.04088, 2024. [find](https://scholar.google.com/scholar?q=Albert+Q+Jiang%2C+Alexandre+Sablayrolles%2C+Antoine+Roux%2C+Arthur+Mensch%2C+Blanche+Savary%2C+Chris+Bamford%2C+Devendra+Singh+Chaplot%2C+Diego+de+las+Casas%2C+Emma+Bou+Hanna%2C+Florian+Bressand%2C+et+al%2E+Mixtral+of+expe)
19. Dongfu Jiang, Xiang Ren, and Bill Yuchen Lin. [Llm-blender: Ensembling large language models with pairwise ranking and generative fusion](https://md.research.elamur.ai/free/en/papers/jiang-2023-llm-blender-ensembling-large-language/). arXiv preprint arXiv:2306.02561, 2023.
20. Diederik P. Kingma and Jimmy Ba. [Adam: A method for stochastic optimization](https://md.research.elamur.ai/free/en/papers/kingma-2014-adam-method-stochastic-optimization/), 2017. URL https://arxiv.org/abs/1412.6980. [↗](https://arxiv.org/abs/1412.6980)
21. Yehuda Koren, Robert Bell, and Chris Volinsky. Matrix factorization techniques for recommender systems. Computer, 42(8):30–37, 2009. [find](https://scholar.google.com/scholar?q=Yehuda+Koren%2C+Robert+Bell%2C+and+Chris+Volinsky%2E+Matrix+factorization+techniques+for+recommender+systems%2E+Computer%2C+42%288%29%3A30%E2%80%9337%2C+2009%2E)
22. Keming Lu, Hongyi Yuan, Runji Lin, Junyang Lin, Zheng Yuan, Chang Zhou, and Jingren Zhou. [Routing to the expert: Efficient reward-guided ensemble of large language models](https://md.research.elamur.ai/free/en/papers/lu-2023-routing-expert-efficient-reward-guided/), 2023. URL https://arxiv.org/abs/2311.08692. [↗](https://arxiv.org/abs/2311.08692)
23. Martian. Martian router, 2024. URL [https://withmartian.com/](https://withmartian.com/). Accessed: 2024-06-30. [↗](https://withmartian.com/)
24. OpenAI. [Gpt-4 technical report](https://md.research.elamur.ai/free/en/papers/openai-2023-gpt-4-technical-report/). arXiv preprint arXiv:2303.08774, 2023.
25. OpenAI. Openai pricing, 2024. URL [https://openai.com/api/pricing/](https://openai.com/api/pricing/). Accessed: 2024-06-30. [↗](https://openai.com/api/pricing/)
26. Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. [Training language models to follow instructions with human feedback](https://md.research.elamur.ai/free/en/papers/ouyang-2022-training-language-models-follow-instructions/). Advances in neural information processing systems, 35:27730– 27744, 2022.
27. Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. Language models are unsupervised multitask learners. OpenAI blog, 1(8):9, 2019. [find](https://scholar.google.com/scholar?q=Alec+Radford%2C+Jeffrey+Wu%2C+Rewon+Child%2C+David+Luan%2C+Dario+Amodei%2C+Ilya+Sutskever%2C+et+al%2E+Language+models+are+unsupervised+multitask+learners%2E+OpenAI+blog%2C+1%288%29%3A9%2C+2019%2E)
28. Together.AI. Together.ai pricing, 2024. URL [https://www.together.ai/pricing](https://www.together.ai/pricing). Accessed: 2024-06-30. [↗](https://www.together.ai/pricing)
29. Andreas Töscher, Michael Jahrer, and Robert M Bell. The bigchaos solution to the netflix grand prize. Netflix prize documentation, pp. 1–52, 2009. [find](https://scholar.google.com/scholar?q=Andreas+T%C3%B6scher%2C+Michael+Jahrer%2C+and+Robert+M+Bell%2E+The+bigchaos+solution+to+the+netflix+grand+prize%2E+Netflix+prize+documentation%2C+pp%2E+1%E2%80%9352%2C+2009%2E)
30. Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. [Llama 2: Open foundation and fine-tuned chat models](https://md.research.elamur.ai/free/en/papers/touvron-2023-llama-2-open-foundation-fine/). arXiv preprint arXiv:2307.09288, 2023.
31. UnifyAI. Unifyai, 2024. URL [https://unify.ai](https://unify.ai). Accessed: 2024-06-30. [↗](https://unify.ai)
32. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. [Attention is all you need](https://md.research.elamur.ai/free/en/papers/vaswani-2017-attention-all-need/). Advances in neural information processing systems, 30, 2017.
33. Jason Wei, Maarten Bosma, Vincent Y Zhao, Kelvin Guu, Adams Wei Yu, Brian Lester, Nan Du, Andrew M Dai, and Quoc V Le. [Finetuned language models are zero-shot learners](https://md.research.elamur.ai/free/en/papers/wei-2021-finetuned-language-models-zero-shot/). arXiv preprint arXiv:2109.01652, 2021.
34. Weizhe Yuan, Graham Neubig, and Pengfei Liu. [Bartscore: Evaluating generated text as text generation](https://md.research.elamur.ai/free/en/papers/yuan-2021-bartscore-evaluating-generated-text-text/). Advances in Neural Information Processing Systems, 34:27263–27277, 2021.
35. Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric Xing, Hao Zhang, Joseph E. Gonzalez, and Ion Stoica. [Judging LLM-as-a-judge with MT-bench and chatbot arena](https://md.research.elamur.ai/free/en/papers/zheng-2023-judging-llm-judge-mt-bench/). In Thirty-seventh Conference on Neural Information Processing Syste…
36. Banghua Zhu, Evan Frick, Tianhao Wu, Hanlin Zhu, and Jiantao Jiao. Starling-7b: Improving llm helpfulness & harmlessness with rlaif, November 2023. [find](https://scholar.google.com/scholar?q=Banghua+Zhu%2C+Evan+Frick%2C+Tianhao+Wu%2C+Hanlin+Zhu%2C+and+Jiantao+Jiao%2E+Starling%2D7b%3A+Improving+llm+helpfulness+%26+harmlessness+with+rlaif%2C+November+2023%2E)

*Updated 2 November 2026*