---
title: "Hunyuan-A13B Technical Report — paper by Team et al. (2026)"
description: "Serving frontier LLMs is costly because dense models use every weight for every token. 51 references, cited by 1 paper here."
authors: ["Tencent Hunyuan Team", "Ao Liu", "Botong Zhou", "Can Xu", "Chayse Zhou", "ChenChen Zhang", "Chengcheng Xu", "Chenhao Wang", "Decheng Wu", "Dengpeng Wu", "Dian Jiao", "Dong Du", "Dong Wang", "Feng Zhang", "Fengzong Lian", "Guanghui Xu", "Guanwei Zhang", "Hai Wang", "Haipeng Luo", "Han Hu", "Huilin Xu", "Jiajia Wu", "Jianchen Zhu", "Jianfeng Yan", "Jiaqi Zhu", "Jihong Zhang", "Jinbao Xue", "Jun Xia", "Junqiang Zheng", "Kai Liu", "Kai Zhang", "Kai Zheng", "Kejiao Li", "Keyao Wang", "Lan Jiang", "Lixin Liu", "Lulu Wu", "Mengyuan Huang", "Peijie Yu", "Peiqi Wang", "Qian Wang", "Qianbiao Xiang", "Qibin Liu", "Qingfeng Sun", "Richard Guo", "Ruobing Xie", "Saiyong Yang", "Shaohua Chen", "Shihui Hu", "Shuai Li", "Shuaipeng Li", "Shuang Chen", "Suncong Zheng", "Tao Yang", "Tian Zhang", "Tinghao Yu", "Weidong Han", "Weijie Liu", "Weijin Zhou", "Weikang Wang", "Wesleye Chen", "Xiao Feng", "Xiaoqin Ren", "Xingwu Sun", "Xiong Kuang", "Xuemeng Huang", "Xun Cao", "Yanfeng Chen", "Yang Du", "Zhen Yang", "Yangyu Tao", "Yaping Deng", "Yi Shen", "Yigeng Hong", "Yiqi Chen"]
year: 2026
venue: "arXiv:2609.27284"
source: "https://arxiv.org/abs/2609.27284"
doi: "10.48550/arxiv.2609.27284"
arxiv: "2609.27284"
licence: "CC BY 4.0 (arXiv)"
licence_url: "http://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/team-2026-hunyuan-hunyuan-a13b-technical-report/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/team-2026-hunyuan-hunyuan-a13b-technical-report/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/13/) / Hunyuan-A13B Technical Report

# Hunyuan-A13B Technical Report

**Authors:** [Tencent Hunyuan Team](https://md.research.elamur.ai/free/en/authors/tencent-hunyuan-team/), [Ao Liu](https://md.research.elamur.ai/free/en/authors/ao-liu/), [Botong Zhou](https://md.research.elamur.ai/free/en/authors/botong-zhou/), [Can Xu](https://md.research.elamur.ai/free/en/authors/can-xu/), [Chayse Zhou](https://md.research.elamur.ai/free/en/authors/chayse-zhou/), [ChenChen Zhang](https://md.research.elamur.ai/free/en/authors/chenchen-zhang/), [Chengcheng Xu](https://md.research.elamur.ai/free/en/authors/chengcheng-xu/), [Chenhao Wang](https://md.research.elamur.ai/free/en/authors/chenhao-wang/), [Decheng Wu](https://md.research.elamur.ai/free/en/authors/decheng-wu/), [Dengpeng Wu](https://md.research.elamur.ai/free/en/authors/dengpeng-wu/), [Dian Jiao](https://md.research.elamur.ai/free/en/authors/dian-jiao/), [Dong Du](https://md.research.elamur.ai/free/en/authors/dong-du/), [Dong Wang](https://md.research.elamur.ai/free/en/authors/dong-wang/), [Feng Zhang](https://md.research.elamur.ai/free/en/authors/feng-zhang/), Fengzong Lian, Guanghui Xu, Guanwei Zhang, [Hai Wang](https://md.research.elamur.ai/free/en/authors/hai-wang/), Haipeng Luo, [Han Hu](https://md.research.elamur.ai/free/en/authors/han-hu/), Huilin Xu, Jiajia Wu, Jianchen Zhu, Jianfeng Yan, [Jiaqi Zhu](https://md.research.elamur.ai/free/en/authors/jiaqi-zhu/), Jihong Zhang, Jinbao Xue, Jun Xia, Junqiang Zheng, [Kai Liu](https://md.research.elamur.ai/free/en/authors/kai-liu/), [Kai Zhang](https://md.research.elamur.ai/free/en/authors/kai-zhang/), [Kai Zheng](https://md.research.elamur.ai/free/en/authors/kai-zheng/), Kejiao Li, Keyao Wang, Lan Jiang, Lixin Liu, Lulu Wu, Mengyuan Huang, Peijie Yu, Peiqi Wang, [Qian Wang](https://md.research.elamur.ai/free/en/authors/qian-wang/), Qianbiao Xiang, Qibin Liu, [Qingfeng Sun](https://md.research.elamur.ai/free/en/authors/qingfeng-sun/), Richard Guo, [Ruobing Xie](https://md.research.elamur.ai/free/en/authors/ruobing-xie/), Saiyong Yang, Shaohua Chen, Shihui Hu, Shuai Li, [Shuaipeng Li](https://md.research.elamur.ai/free/en/authors/shuaipeng-li/), Shuang Chen, Suncong Zheng, [Tao Yang](https://md.research.elamur.ai/free/en/authors/tao-yang/), Tian Zhang, Tinghao Yu, Weidong Han, [Weijie Liu](https://md.research.elamur.ai/free/en/authors/weijie-liu/), Weijin Zhou, [Weikang Wang](https://md.research.elamur.ai/free/en/authors/weikang-wang/), Wesleye Chen, Xiao Feng, Xiaoqin Ren, [Xingwu Sun](https://md.research.elamur.ai/free/en/authors/xingwu-sun/), Xiong Kuang, Xuemeng Huang, Xun Cao, Yanfeng Chen, Yang Du, [Zhen Yang](https://md.research.elamur.ai/free/en/authors/zhen-yang/), Yangyu Tao, Yaping Deng, Yi Shen, Yigeng Hong and Yiqi Chen · **Year:** 2026 · **Venue:** arXiv:2609.27284 · **Licence:** [CC BY 4.0 (arXiv)](http://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://arxiv.org/abs/2609.27284) · [PDF](https://arxiv.org/pdf/2609.27284) · [DOI 10.48550/arxiv.2609.27284](https://doi.org/10.48550/arxiv.2609.27284) · [arXiv 2609.27284](https://arxiv.org/abs/2609.27284) · [OpenAlex](https://openalex.org/W7214196674)

**Citations:** 0 citations (OpenAlex, 2026-10-14) · 1 in our library, of the 149,439 papers whose bibliographies we read

## Summary
Serving frontier LLMs is costly because dense models use every weight for every token. Tencent's Hunyuan team addresses this with Hunyuan-A13B, an openly released sparse Mixture-of-Experts model. It has 80B parameters in total, but only about 13B are active per token: one always-on shared expert plus 8 of 64 fine-grained routed experts. Pretraining used more than 20T tokens prepared with the Hunyuan-TurboS curation pipeline, which was upgraded specifically for STEM material. A brief annealing phase followed, then context extension up to 256K. Post-training ran in four steps: reasoning-focused SFT and RL, then all-scenario SFT and RL. Users can choose between a quick short-CoT mode and a slower long-CoT mode. The authors report that reasoning, coding and agent results approach those of much larger models. Long-context scores are strong, with RULER QA holding at 73.9 for 64K–128K inputs. The model also achieves high throughput on vLLM, SGLang and TensorRT-LLM. Its significance lies in delivering competitive quality at a modest inference cost.

## Key points
- **Architecture:** 32 layers, GQA (32 attention heads, 8 KV heads), SwiGLU, 128K vocabulary (same tokenizer as Hunyuan-Large), hidden size 4096, expert FFN size 3072. Scaling experiments showed that one shared expert helps and adding more brings little gain.
- **Data and schedule:** 250B tokens of filtered STEM data. The learning rate peaked at 3×10⁻⁴ and followed a cosine decay to 3×10⁻⁵ over 13.5T tokens with a 4096 context. Annealing covered 300B tokens down to 8×10⁻⁶ at an 8192 context. NTK-aware RoPE scaling used alpha 50 for 32K and 1000 for 256K.
- **Long-context results:**
  - PenguinScrolls: 87.7.
  - LongBench-v2: 55.0, second only to Gemini 2.5 Pro (60.9).
  - FRAMES: 81.1, behind DeepSeek R1 (85.7) and Qwen3-A22B (84.0).
  - RULER QA average: 76.7.
- **Efficiency:** With 2048 input and 14,336 output tokens, throughput is 190.84 tok/s at batch 1 and 1981.99 tok/s at batch 32. Supported quantization formats include Weight Only INT8, W8A8 and KV Cache FP8.
- **Caveats for the library:** The extracted text omits the middle sections, so math, coding and agent benchmark numbers are not captured here. The document header date (2025-06-27) also conflicts with the arXiv metadata (2026).

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

📚 Shelves (5): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 18,614 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 17,965 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 17,450 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 17,296 works · [👁️ Computer Vision and Pattern Recognition](https://md.research.elamur.ai/free/en/shelves/computer-science/computer-vision-and-pattern-recognition/) · 3,486 works

## Cite

Cite as: Tencent Hunyuan Team, Ao Liu, Botong Zhou, Can Xu, Chayse Zhou, ChenChen Zhang, Chengcheng Xu, Chenhao Wang, Decheng Wu, Dengpeng Wu, Dian Jiao, Dong Du, Dong Wang, Feng Zhang, Fengzong Lian, Guanghui Xu, Guanwei Zhang, Hai Wang, Haipeng Luo, Han Hu, Huilin Xu, Jiajia Wu, Jianchen Zhu, Jianfeng Yan, Jiaqi Zhu, Jihong Zhang, Jinbao Xue, Jun Xia, Junqiang Zheng, Kai Liu, Kai Zhang, Kai Zheng, Kejiao Li, Keyao Wang, Lan Jiang, Lixin Liu, Lulu Wu, Mengyuan Huang, Peijie Yu, Peiqi Wang, Qian Wang, Qianbiao Xiang, Qibin Liu, Qingfeng Sun, Richard Guo, Ruobing Xie, Saiyong Yang, Shaohua Chen, Shihui Hu, Shuai Li, Shuaipeng Li, Shuang Chen, Suncong Zheng, Tao Yang, Tian Zhang, Tinghao Yu, Weidong Han, Weijie Liu, Weijin Zhou, Weikang Wang, Wesleye Chen, Xiao Feng, Xiaoqin Ren, Xingwu Sun, Xiong Kuang, Xuemeng Huang, Xun Cao, Yanfeng Chen, Yang Du, Zhen Yang, Yangyu Tao, Yaping Deng, Yi Shen, Yigeng Hong and Yiqi Chen. 2026. Hunyuan-A13B Technical Report. arXiv:2609.27284.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@misc{team2026hunyuan,
  title = {{Hunyuan-A13B Technical Report}},
  author = {Tencent Hunyuan Team, Ao Liu, Botong Zhou, Can Xu, Chayse Zhou, ChenChen Zhang, Chengcheng Xu, Chenhao Wang, Decheng Wu, Dengpeng Wu, Dian Jiao, Dong Du, Dong Wang, Feng Zhang, Fengzong Lian, Guanghui Xu, Guanwei Zhang, Hai Wang, Haipeng Luo, Han Hu, Huilin Xu, Jiajia Wu, Jianchen Zhu, Jianfeng Yan, Jiaqi Zhu, Jihong Zhang, Jinbao Xue, Jun Xia, Junqiang Zheng, Kai Liu, Kai Zhang, Kai Zheng, Kejiao Li, Keyao Wang, Lan Jiang, Lixin Liu, Lulu Wu, Mengyuan Huang, Peijie Yu, Peiqi Wang, Qian Wang, Qianbiao Xiang, Qibin Liu, Qingfeng Sun, Richard Guo, Ruobing Xie, Saiyong Yang, Shaohua Chen, Shihui Hu, Shuai Li, Shuaipeng Li, Shuang Chen, Suncong Zheng, Tao Yang, Tian Zhang, Tinghao Yu, Weidong Han, Weijie Liu, Weijin Zhou, Weikang Wang, Wesleye Chen, Xiao Feng, Xiaoqin Ren, Xingwu Sun, Xiong Kuang, Xuemeng Huang, Xun Cao, Yanfeng Chen, Yang Du, Zhen Yang, Yangyu Tao, Yaping Deng, Yi Shen, Yigeng Hong and Yiqi Chen},
  year = {2026},
  doi = {10.48550/arxiv.2609.27284},
  eprint = {2609.27284},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2609.27284}
}
```

- [← previous Humor Generation – Text-based Humor Generation (English)](https://md.research.elamur.ai/free/en/papers/parthiban-2026-humor-generation-text-based-humor/)
- [next → Hurdles of Automatic Metric for Speech Translation Evaluation](https://md.research.elamur.ai/free/en/papers/zarzu-2026-hurdles-automatic-metric-speech-translation/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/13/)

## References

51 references, 29 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. AIME. Aime problems and solutions. — [GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpoints](https://md.research.elamur.ai/free/en/papers/ainslie-2023-gqa-training-generalized-multi-query/) [https://artofproblemsolving.com/wiki/index.php/AIME](https://artofproblemsolving.com/wiki/index.php/AIME) Probl ems and Solutions. Joshua Ainslie, James Lee-Thorp, Michiel de Jong, Yury Zemlyanskiy, Federico Lebrón, and Sumit Sanghai. GQA: Training generalized multi-query transformer models from multi-head checkpoint… [↗](https://artofproblemsolving.com/wiki/index.php/AIME)
2. Jacob Austin, Augustus Odena, Maxwell Nye, Maarten Bosma, Henryk Michalewski, David Dohan, Ellen Jiang, Carrie Cai, Michael Terry, Quoc Le, et al. [Program synthesis with large language models](https://md.research.elamur.ai/free/en/papers/austin-2021-program-synthesis-large-language-models/). arXiv preprint arXiv:2108.07732, 2021.
3. Yushi Bai, Shangqing Tu, Jiajie Zhang, Hao Peng, Xiaozhi Wang, Xin Lv, Shulin Cao, Jiazheng Xu, Lei Hou, Yuxiao Dong, et al. [Longbench v2: Towards deeper understanding and reasoning on realistic long-context multitasks](https://md.research.elamur.ai/free/en/papers/bai-2024-longbench-v2-deeper-understanding/). arXiv preprint arXiv:2412.15204, 2024.
4. Federico Cassano, John Gouwar, Daniel Nguyen, Sydney Nguyen, Luna Phipps-Costin, Donald Pinckney, Ming-Ho Yee, Yangtian Zi, Carolyn Jane Anderson, Molly Q Feldman, et al. [Multipl-e: A scalable and extensible approach to benchmarking neural code generation](https://md.research.elamur.ai/free/en/papers/cassano-2022-multipl-e-scalable-extensible-approach/). arXiv preprint arXiv:2208.08227, 2022.
5. Linzheng Chai, Shukai Liu, Jian Yang, Yuwei Yin, Ke Jin, Jiaheng Liu, Tao Sun, Ge Zhang, Changyu Ren, Hongcheng Guo, et al. Mceval: Massively multilingual code evaluation. arXiv preprint arXiv:2406.07436, 2024. — [arXiv:2406.07436](https://arxiv.org/abs/2406.07436) [find](https://scholar.google.com/scholar?q=Linzheng+Chai%2C+Shukai+Liu%2C+Jian+Yang%2C+Yuwei+Yin%2C+Ke+Jin%2C+Jiaheng+Liu%2C+Tao+Sun%2C+Ge+Zhang%2C+Changyu+Ren%2C+Hongcheng+Guo%2C+et+al%2E+Mceval%3A+Massively+multilingual+code+evaluation%2E+arXiv+preprint+arXiv%3A2406%2E07)
6. Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde De Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, et al. [Evaluating large language models trained on code](https://md.research.elamur.ai/free/en/papers/chen-2021-evaluating-large-language-models-trained/). arXiv preprint arXiv:2107.03374, 2021.
7. Yao Cheng, Jianfeng Chen, Jie Chen, Li Chen, Liyu Chen, Wentao Chen, Zhengyu Chen, Shijie Geng, Aoyan Li, Bo Li, et al. Fullstack bench: Evaluating llms as full stack coders. arXiv e-prints, pp. arXiv–2412, 2024. [find](https://scholar.google.com/scholar?q=Yao+Cheng%2C+Jianfeng+Chen%2C+Jie+Chen%2C+Li+Chen%2C+Liyu+Chen%2C+Wentao+Chen%2C+Zhengyu+Chen%2C+Shijie+Geng%2C+Aoyan+Li%2C+Bo+Li%2C+et+al%2E+Fullstack+bench%3A+Evaluating+llms+as+full+stack+coders%2E+arXiv+e%2Dprints%2C+pp%2E+arXiv)
8. Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, et al. [Training verifiers to solve math word problems](https://md.research.elamur.ai/free/en/papers/cobbe-2021-training-verifiers-solve-math-word/). arXiv preprint arXiv:2110.14168, 2021.
9. Damai Dai, Chengqi Deng, Chenggang Zhao, RX Xu, Huazuo Gao, Deli Chen, Jiashi Li, Wangding Zeng, Xingkai Yu, Y Wu, et al. DeepSeekMoE: Towards ultimate expert specialization in mixture-of-experts language models. arXiv preprint arXiv:2401.06066, 2024. [find](https://scholar.google.com/scholar?q=Damai+Dai%2C+Chengqi+Deng%2C+Chenggang+Zhao%2C+RX+Xu%2C+Huazuo+Gao%2C+Deli+Chen%2C+Jiashi+Li%2C+Wangding+Zeng%2C+Xingkai+Yu%2C+Y+Wu%2C+et+al%2E+DeepSeekMoE%3A+Towards+ultimate+expert+specialization+in+mixture%2Dof%2Dexperts+lang)
10. Google DeepMind. Gemini 2.5, 2025. URL [https://blog.google/technology/google-deepmind/gemini](https://blog.google/technology/google-deepmind/gemini) -model-thinking-updates-march-2025/. [↗](https://blog.google/technology/google-deepmind/gemini)
11. Xinrun Du, Yifan Yao, Kaijing Ma, Bingli Wang, Tianyu Zheng, King Zhu, Minghao Liu, Yiming Liang, Xiaolong Jin, Zhenlin Wei, et al. [Supergpqa: Scaling llm evaluation across 285 graduate disciplines](https://md.research.elamur.ai/free/en/papers/team-2025-supergpqa-scaling-llm-evaluation-across/). arXiv preprint arXiv:2502.14739, 2025.
12. Dheeru Dua, Yizhong Wang, Pradeep Dasigi, Gabriel Stanovsky, Sameer Singh, and Matt Gardner. [Drop: A reading comprehension benchmark requiring discrete reasoning over paragraphs](https://md.research.elamur.ai/free/en/papers/dua-2019-drop-reading-comprehension-benchmark-requiring/). arXiv preprint arXiv:1903.00161, 2019.
13. Aryo Pradipta Gema, Joshua Ong Jun Leang, Giwon Hong, Alessio Devoto, Alberto Carlo Maria Mancino, Rohit Saxena, Xuanli He, Yu Zhao, Xiaotang Du, Mohammad Reza Ghasemi Madani, et al. Are we done with mmlu? arXiv preprint arXiv:2406.04127, 2024. [find](https://scholar.google.com/scholar?q=Aryo+Pradipta+Gema%2C+Joshua+Ong+Jun+Leang%2C+Giwon+Hong%2C+Alessio+Devoto%2C+Alberto+Carlo+Maria+Mancino%2C+Rohit+Saxena%2C+Xuanli+He%2C+Yu+Zhao%2C+Xiaotang+Du%2C+Mohammad+Reza+Ghasemi+Madani%2C+et+al%2E+Are+we+done+with+)
14. Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. [DeepSeek-R1: Incentivizing reasoning capability in LLMs via reinforcement learning](https://md.research.elamur.ai/free/en/papers/deepseek-ai-2025-deepseek-r1-incentivizing-reasoning-capability/). arXiv preprint arXiv:2501.12948, 2025.
15. Chaoqun He, Renjie Luo, Yuzhuo Bai, Shengding Hu, Zhen Leng Thai, Junhao Shen, Jinyi Hu, Xu Han, Yujie Huang, Yuxiang Zhang, et al. [Olympiadbench: A challenging benchmark for promoting agi with olympiad-level bilingual multimodal scientific problems](https://md.research.elamur.ai/free/en/papers/he-2024-olympiadbench-challenging-benchmark-promoting-agi/). arXiv preprint arXiv:2402.14008, 2024.
16. Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. [Measuring mathematical problem solving with the math dataset](https://md.research.elamur.ai/free/en/papers/hendrycks-2021-measuring-mathematical-problem-solving-math/). arXiv preprint arXiv:2103.03874, 2021.
17. Cheng-Ping Hsieh, Simeng Sun, Samuel Kriman, Shantanu Acharya, Dima Rekesh, Fei Jia, Yang Zhang, and Boris Ginsburg. Ruler: What’s the real context size of your long-context language models? arXiv preprint arXiv:2404.[06654, 2024](https://md.research.elamur.ai/free/en/papers/hsieh-2024-ruler-s-real-context-size/).
18. Tencent Hunyuan. PenguinScrolls: A user-aligned fine-grained benchmark for long-context language model evaluation, 2024. URL [https://github.com/Penguin-Scrolls/PenguinScrolls](https://github.com/Penguin-Scrolls/PenguinScrolls). [↗](https://github.com/Penguin-Scrolls/PenguinScrolls)
19. Naman Jain, King Han, Alex Gu, Wen-Ding Li, Fanjia Yan, Tianjun Zhang, Sida Wang, Armando Solar-Lezama, Koushik Sen, and Ion Stoica. [Livecodebench: Holistic and contamination free evaluation of large language models for code](https://md.research.elamur.ai/free/en/papers/jain-2024-livecodebench-holistic-contamination-free-evaluation/). arXiv preprint arXiv:2403.07974, 2024.
20. Charlie Cheng-Jie Ji, Huanzhi Mao, Fanjia Yan, Shishir G. Patil, Tianjun Zhang, Ion Stoica, and Joseph E. Gonzalez. Gorilla bfcl v3. [https://gorilla.cs.berkeley.edu/leaderboard.html](https://gorilla.cs.berkeley.edu/leaderboard.html). Accessed: 2025-01-17. [↗](https://gorilla.cs.berkeley.edu/leaderboard.html)
21. Satyapriya Krishna, Kalpesh Krishna, Anhad Mohananey, Steven Schwarcz, Adam Stambler, Shyam Upadhyay, and Manaal Faruqui. Fact, fetch, and reason: A unified evaluation of retrieval-augmented generation, 2024. URL https://arxiv.org/abs/2409.12941. [↗](https://arxiv.org/abs/2409.12941)
22. Hunter Lightman, Vineet Kosaraju, Yuri Burda, Harrison Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever, and Karl Cobbe. Let’s verify step by step. In The Twelfth International Conference on Learning Representations, 2023. — [Let's Verify Step by Step](https://md.research.elamur.ai/free/en/papers/lightman-2023-let-s-verify-step-step/)
23. Bill Yuchen Lin, Ronan Le Bras, Kyle Richardson, Ashish Sabharwal, Radha Poovendran, Peter Clark, and Yejin Choi. [Zebralogic: On the scaling limits of llms for logical reasoning](https://md.research.elamur.ai/free/en/papers/lin-2025-zebralogic-scaling-limits-llms-logical/). arXiv preprint arXiv:2502.01100, 2025a.
24. Bill Yuchen Lin, Ronan Le Bras, Kyle Richardson, Ashish Sabharwal, Radha Poovendran, Peter Clark, and Yejin Choi. [Zebralogic: On the scaling limits of llms for logical reasoning](https://md.research.elamur.ai/free/en/papers/lin-2025-zebralogic-scaling-limits-llms-logical/), 2025b. URL https://arxiv.org/abs/2502.01100. [↗](https://arxiv.org/abs/2502.01100)
25. Ao Liu, Botong Zhou, Can Xu, Chayse Zhou, ChenChen Zhang, Chengcheng Xu, Chenhao Wang, Decheng Wu, Dengpeng Wu, Dian Jiao, et al. [Hunyuan-turbos: Advancing large language models through mamba-transformer synergy and adaptive chain-of-thought](https://md.research.elamur.ai/free/en/papers/team-2025-hunyuan-hunyuan-turbos-advancing-large/). arXiv preprint arXiv:2505.15431, 2025.
26. Jiawei Liu, Chunqiu Steven Xia, Yuyao Wang, and Lingming Zhang. [Is your code generated by chatgpt really correct? rigorous evaluation of large language models for code generation](https://md.research.elamur.ai/free/en/papers/liu-2023-your-code-generated-chatgpt-really/). Advances in Neural Information Processing Systems, 36:21558–21572, 2023.
27. Michael Luo, Sijun Tan, Justin Wong, Xiaoxiang Shi, William Y. Tang, Manan Roongta, Colin Cai, Jeffrey Luo, Li Erran Li, Raluca Ada Popa, and Ion Stoica. Deepscaler: Surpassing o1-preview with a 1.5b model by scaling rl. [https://pretty-radio-b75.notion.site/DeepScaleR-Surpassing-O1-Preview](https://pretty-radio-b75.notion.site/DeepScaleR-Surpassing-O1-Preview) -with-a-1… [↗](https://pretty-radio-b75.notion.site/DeepScaleR-Surpassing-O1-Preview)
28. Dakota Mahan, Duy Van Phung, Rafael Rafailov, Chase Blagden, Nathan Lile, Louis Castricato, Jan-Philipp Fränken, Chelsea Finn, and Alon Albalak. Generative reward models. arXiv preprint arXiv:2410.12832, 2024. [find](https://scholar.google.com/scholar?q=Dakota+Mahan%2C+Duy+Van+Phung%2C+Rafael+Rafailov%2C+Chase+Blagden%2C+Nathan+Lile%2C+Louis+Castricato%2C+Jan%2DPhilipp+Fra%CC%88nken%2C+Chelsea+Finn%2C+and+Alon+Albalak%2E+Generative+reward+models%2E+arXiv+preprint+arXiv%3A2410%2E12)
29. OpenAI. Introducing openai o1, 2024. URL [https://openai.com/o1/](https://openai.com/o1/). [↗](https://openai.com/o1/)
30. OpenAI. Hello GPT-4o, 2024. URL [https://openai.com/index/hello-gpt-4o/](https://openai.com/index/hello-gpt-4o/). [↗](https://openai.com/index/hello-gpt-4o/)
31. OpenAI. Introducing openai o3 and o4-mini, 2025. URL [https://openai.com/index/introducing-o3-a](https://openai.com/index/introducing-o3-a) nd-o4-mini/. [↗](https://openai.com/index/introducing-o3-a)
32. Bowen Peng and Jeffrey Quesnelle. Ntk-aware scaled rope allows llama models to have extended (8k+) context size without any fine-tuning and minimal perplexity degradation, 2023. [find](https://scholar.google.com/scholar?q=Bowen+Peng+and+Jeffrey+Quesnelle%2E+Ntk%2Daware+scaled+rope+allows+llama+models+to+have+extended+%288k%2B%29+context+size+without+any+fine%2Dtuning+and+minimal+perplexity+degradation%2C+2023%2E)
33. Yanzhao Qin, Tao Zhang, Yanjun Shen, Wenjing Luo, Haoze Sun, Yan Zhang, Yujing Qiao, Weipeng Chen, Zenan Zhou, Wentao Zhang, et al. Sysbench: Can large language models follow system messages? arXiv preprint arXiv:2408.10943, 2024. — [arXiv:2408.10943](https://arxiv.org/abs/2408.10943) [find](https://scholar.google.com/scholar?q=Yanzhao+Qin%2C+Tao+Zhang%2C+Yanjun+Shen%2C+Wenjing+Luo%2C+Haoze+Sun%2C+Yan+Zhang%2C+Yujing+Qiao%2C+Weipeng+Chen%2C+Zenan+Zhou%2C+Wentao+Zhang%2C+et+al%2E+Sysbench%3A+Can+large+language+models+follow+system+messages%3F+arXiv+pr)
34. David Rein, Betty Li Hou, Asa Cooper Stickland, Jackson Petty, Richard Yuanzhe Pang, Julien Dirani, Julian Michael, and Samuel R Bowman. [Gpqa: A graduate-level google-proof q&a benchmark](https://md.research.elamur.ai/free/en/papers/rein-2023-gpqa-graduate-level-google-proof/). In First Conference on Language Modeling, 2024.
35. Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Y Wu, et al. [Deepseekmath: Pushing the limits of mathematical reasoning in open language models](https://md.research.elamur.ai/free/en/papers/shao-2024-deepseekmath-pushing-limits-mathematical-reasoning/). arXiv preprint arXiv:2402.03300, 2024.
36. Noam Shazeer. GLU variants improve transformer. arXiv preprint arXiv:2002.05202, 2020. [find](https://scholar.google.com/scholar?q=Noam+Shazeer%2E+GLU+variants+improve+transformer%2E+arXiv+preprint+arXiv%3A2002%2E05202%2C+2020%2E)
37. Xingwu Sun, Yanfeng Chen, Yiqing Huang, Ruobing Xie, Jiaqi Zhu, Kai Zhang, Shuaipeng Li, Zhen Yang, Jonny Han, Xiaobo Shu, et al. [Hunyuan-large: An open-source moe model with 52 billion activated parameters by tencent](https://md.research.elamur.ai/free/en/papers/sun-2024-hunyuan-large-open-source/). arXiv preprint arXiv:2411.02265, 2024.
38. Mirac Suzgun, Nathan Scales, Nathanael Schärli, Sebastian Gehrmann, Yi Tay, Hyung Won Chung, Aakanksha Chowdhery, Quoc V Le, Ed H Chi, Denny Zhou, et al. [Challenging big-bench tasks and whether chain-of-thought can solve them](https://md.research.elamur.ai/free/en/papers/suzgun-2023-challenging-big-bench-tasks-whether/). arXiv preprint arXiv:2210.09261, 2022.
39. Yubo Wang, Xueguang Ma, Ge Zhang, Yuansheng Ni, Abhranil Chandra, Shiguang Guo, Weiming Ren, Aaran Arulraj, Xuan He, Ziyan Jiang, et al. [MMLU-Pro: A more robust and challenging multi-task language understanding benchmark](https://md.research.elamur.ai/free/en/papers/wang-2024-mmlu-pro-more-robust-challenging/). In Proceedings of NeurIPS, 2024.
40. Tianwen Wei, Jian Luan, Wei Liu, Shuang Dong, and Bin Wang. [CMATH: Can your language model pass chinese elementary school math test?](https://md.research.elamur.ai/free/en/papers/wei-2023-cmath-language-model-pass/) arXiv preprint arXiv:2306.16636, 2023.
41. Yuxiang Wei, Zhe Wang, Jiawei Liu, Yifeng Ding, and Lingming Zhang. Magicoder: Empowering code generation with OSS-instruct. In Proceedings of the 41st International Conference on Machine Learning, volume 235 of Proceedings of Machine Learning Research, pp. 52632–52657. PMLR, 21–27 Jul 2024. URL htt… [find](https://scholar.google.com/scholar?q=Yuxiang+Wei%2C+Zhe+Wang%2C+Jiawei+Liu%2C+Yifeng+Ding%2C+and+Lingming+Zhang%2E+Magicoder%3A+Empowering+code+generation+with+OSS%2Dinstruct%2E+In+Proceedings+of+the+41st+International+Conference+on+Machine+Learning%2C+vo)
42. Wenyuan Xu, Xiaochen Zuo, Chao Xin, Yu Yue, Lin Yan, and Yonghui Wu. A unified pairwise framework for rlhf: Bridging generative reward modeling and policy optimization, 2025. URL https://arxiv.org/abs/2504.04950. — [arXiv:2504.04950](https://arxiv.org/abs/2504.04950) [↗](https://arxiv.org/abs/2504.04950)
43. An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, Chujie Zheng, Dayiheng Liu, Fan Zhou, Fei Huang, Feng Hu, Hao Ge, Haoran Wei, Huan Lin, Jialong Tang, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jing Zhou, Jin… — [Qwen3 Technical Report](https://md.research.elamur.ai/free/en/papers/yang-2025-qwen3-technical-report/)
44. An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, et al. [Qwen3 technical report](https://md.research.elamur.ai/free/en/papers/yang-2025-qwen3-technical-report/). arXiv preprint arXiv:2505.09388, 2025b.
45. An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, Huan Lin, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jingren Zhou, Junyang Lin, Kai Dang, Keming Lu, Keqin Bao, Kexin Yang, Le Yu, Mei Li, Mingfeng Xue, Pei… [find](https://scholar.google.com/scholar?q=An+Yang%2C+Baosong+Yang%2C+Beichen+Zhang%2C+Binyuan+Hui%2C+Bo+Zheng%2C+Bowen+Yu%2C+Chengyuan+Li%2C+Dayiheng+Liu%2C+Fei+Huang%2C+Haoran+Wei%2C+Huan+Lin%2C+Jian+Yang%2C+Jianhong+Tu%2C+Jianwei+Zhang%2C+Jianxin+Yang%2C+Jiaxi+Yang%2C+Jin)
46. Shunyu Yao, Noah Shinn, Pedram Razavi, and Karthik Narasimhan. τ-bench: A benchmark for tool-agent-user interaction in real-world domains. In The Thirteenth International Conference on Learning Representations, 2025. [find](https://scholar.google.com/scholar?q=Shunyu+Yao%2C+Noah+Shinn%2C+Pedram+Razavi%2C+and+Karthik+Narasimhan%2E+%CF%84%2Dbench%3A+A+benchmark+for+tool%2Dagent%2Duser+interaction+in+real%2Dworld+domains%2E+In+The+Thirteenth+International+Conference+on+Learning+Repres)
47. Peijie Yu, Yifan Yang, Jinjian Li, Zelong Zhang, Haorui Wang, Xiao Feng, and Feng Zhang. c 3 -bench: The things real disturbing llm based agent in multi-tasking. arXiv preprint arXiv:2505.18746, 2025. URL https://arxiv.org/abs/2505.18746. — [arXiv:2505.18746](https://arxiv.org/abs/2505.18746) [↗](https://arxiv.org/abs/2505.18746)
48. Qiying Yu, Zheng Zhang, Ruofei Zhu, Yufeng Yuan, Xiaochen Zuo, Yu Yue, Tiantian Fan, Gaohong Liu, Lingjun Liu, Xin Liu, et al. [Dapo: An open-source llm reinforcement learning system at scale](https://md.research.elamur.ai/free/en/papers/yu-2025-dapo-open-source-llm-reinforcement/), 2025. URL https://arxiv.org/abs/2503.14476. [↗](https://arxiv.org/abs/2503.14476)
49. Lunjun Zhang, Arian Hosseini, Hritik Bansal, Mehran Kazemi, Aviral Kumar, and Rishabh Agarwal. [Generative verifiers: Reward modeling as next-token prediction](https://md.research.elamur.ai/free/en/papers/zhang-2024-generative-verifiers-reward-modeling-as/). arXiv preprint arXiv:2408.15240, 2024.
50. Lucen Zhong, Zhengxiao Du, Xiaohan Zhang, Haiyi Hu, and Jie Tang. Complexfuncbench: Exploring multi-step and constrained function calling under long-context scenario. arXiv preprint arXiv:2501.10132, 2025. — [arXiv:2501.10132](https://arxiv.org/abs/2501.10132) [find](https://scholar.google.com/scholar?q=Lucen+Zhong%2C+Zhengxiao+Du%2C+Xiaohan+Zhang%2C+Haiyi+Hu%2C+and+Jie+Tang%2E+Complexfuncbench%3A+Exploring+multi%2Dstep+and+constrained+function+calling+under+long%2Dcontext+scenario%2E+arXiv+preprint+arXiv%3A2501%2E10132%2C+)
51. Jeffrey Zhou, Tianjian Lu, Swaroop Mishra, Siddhartha Brahma, Sujoy Basu, Yi Luan, Denny Zhou, and Le Hou. [Instruction-following evaluation for large language models](https://md.research.elamur.ai/free/en/papers/zhou-2023-instruction-following-evaluation-large-language/). arXiv preprint arXiv:2311.07911, 2023.

*Updated 2 November 2026*