---
title: "GLM-4.5: Agentic, Reasoning, and Coding (ARC) — Zeng 2025"
description: "Open-source LLMs have trailed proprietary systems such as o3 and Claude Sonnet 4 when a single model must combine tool use, multi-step reasoning, and…"
authors: ["Aohan Zeng", "Xin Lv", "Qinkai Zheng", "Zhenyu Hou", "Bin Chen", "Chengxing Xie", "Cunxiang Wang", "Da Yin", "Hao Zeng", "Jiajie Zhang", "Kedong Wang", "Lucen Zhong", "Mingdao Liu", "Rui Lu", "Shulin Cao", "Xiaohan Zhang", "Xuancheng Huang", "Yao Wei", "Yean Cheng", "Yifan An", "Yilin Niu", "Yuanhao Wen", "Yushi Bai", "Zhengxiao Du", "Zihan Wang", "Zilin Zhu", "Bohan Zhang", "Bosi Wen", "Bowen Wu", "Bowen Xu", "Can Huang", "Casey Zhao", "Changpeng Cai", "Chao Yu", "Chen Li", "Chendi Ge", "Chenghua Huang", "Chenhui Zhang", "Chenxi Xu", "Chenzheng Zhu", "Chuang Li", "Congfeng Yin", "Daoyan Lin", "Dayong Yang", "Dazhi Jiang", "Ding Ai", "Erle Zhu", "Fei Wang", "Gengzheng Pan", "Guo Wang", "Hailong Sun", "Haitao Li", "Haiyang Li", "Haiyi Hu", "Hanyu Zhang", "Hao Peng", "Hao Tai", "Haoke Zhang", "Haoran Wang", "Haoyu Yang", "He Liu", "He Zhao", "Hongwei Liu", "Hongxi Yan", "Huan Liu", "Huilong Chen", "Ji Li", "Jiajing Zhao", "Jiamin Ren", "Jian Jiao", "Jiani Zhao", "Jianyang Yan", "Jiaqi Wang", "Jiayi Gui", "Jiayue Zhao", "Jie Liu", "Jijie Li", "Jing Li", "Jing Lu", "Jingsen Wang", "Jingwei Yuan", "Jingxuan Li", "Jingzhao Du", "Jinhua Du", "Jinxin Liu", "Junkai Zhi", "Junli Gao", "Ke Wang", "Lekang Yang", "Liang Xu", "Lin Fan", "Lindong Wu", "Lintao Ding", "Lu Wang", "Man Zhang", "Minghao Li", "Minghuan Xu", "Mingming Zhao", "Mingshu Zhai", "Pengfan Du", "Qian Dong", "Shangde Lei", "Shangqing Tu", "Shangtong Yang", "Shaoyou Lu", "Shijie Li", "Shuang Li", "Shuang-Li", "Shuxun Yang", "Sibo Yi", "Tianshu Yu", "Wei Tian", "Weihan Wang", "Wenbo Yu", "Weng Lam Tam", "Wenjie Liang", "Wentao Liu", "Xiao Wang", "Xiaohan Jia", "Xiaotao Gu", "Xiaoying Ling", "Xin Wang", "Xing Fan", "Xingru Pan", "Xinyuan Zhang", "Xinze Zhang", "Xiuqing Fu", "Xunkai Zhang", "Yabo Xu", "Yandong Wu", "Yida Lu", "Yidong Wang", "Yilin Zhou", "Yiming Pan", "Ying Zhang", "Yingli Wang", "Yingru Li", "Yinpei Su", "Yipeng Geng", "Yitong Zhu", "Yongkun Yang", "Yuhang Li", "Yuhao Wu", "Yujiang Li", "Yunan Liu", "Yunqing Wang", "Yuntao Li", "Yuxuan Zhang", "Zezhen Liu", "Zhen Yang", "Zhengda Zhou", "Zhongpei Qiao", "Zhuoer Feng", "Zhuorui Liu", "Zichen Zhang", "Zihan Wang", "Zijun Yao", "Zikang Wang", "Ziqiang Liu", "Ziwei Chai", "Zixuan Li", "Zuodong Zhao", "Wenguang Chen", "Jidong Zhai", "Bin Xu", "Minlie Huang", "Hongning Wang", "Juanzi Li", "Yuxiao Dong", "Jie Tang"]
year: 2025
venue: "arXiv:2508.06471"
source: "https://arxiv.org/abs/2508.06471"
doi: "10.48550/arxiv.2508.06471"
arxiv: "2508.06471"
licence: "CC BY 4.0 (arXiv)"
licence_url: "http://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/glm-2025-4-glm-4-5-agentic/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/glm-2025-4-glm-4-5-agentic/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2025](https://md.research.elamur.ai/free/en/papers/2025/page/2/) / GLM-4.5: Agentic, Reasoning, and Coding (ARC) Foundation Models

# GLM-4.5: Agentic, Reasoning, and Coding (ARC) Foundation Models

**Authors:** [Aohan Zeng](https://md.research.elamur.ai/free/en/authors/aohan-zeng/), [Xin Lv](https://md.research.elamur.ai/free/en/authors/xin-lv/), [Qinkai Zheng](https://md.research.elamur.ai/free/en/authors/qinkai-zheng/), [Zhenyu Hou](https://md.research.elamur.ai/free/en/authors/zhenyu-hou/), [Bin Chen](https://md.research.elamur.ai/free/en/authors/bin-chen/), [Chengxing Xie](https://md.research.elamur.ai/free/en/authors/chengxing-xie/), [Cunxiang Wang](https://md.research.elamur.ai/free/en/authors/cunxiang-wang/), [Da Yin](https://md.research.elamur.ai/free/en/authors/da-yin/), Hao Zeng, [Jiajie Zhang](https://md.research.elamur.ai/free/en/authors/jiajie-zhang/), Kedong Wang, Lucen Zhong et al. (170 authors) · **Year:** 2025 · **Venue:** arXiv:2508.06471 · **Licence:** [CC BY 4.0 (arXiv)](http://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://arxiv.org/abs/2508.06471) · [PDF](https://arxiv.org/pdf/2508.06471) · [DOI 10.48550/arxiv.2508.06471](https://doi.org/10.48550/arxiv.2508.06471) · [arXiv 2508.06471](https://arxiv.org/abs/2508.06471) · [OpenAlex](https://openalex.org/W4415194046)

**Citations:** 6 citations (openalex-snapshot · 2026-10-14) · [151](#cited-by) in our library, of the 140673 papers whose bibliographies we read

## Summary
Open-source LLMs have trailed proprietary systems such as o3 and Claude Sonnet 4 when a single model must combine tool use, multi-step reasoning, and real-world software engineering. The GLM-4.5 Team (Zhipu AI & Tsinghua University) addresses this gap with GLM-4.5, a Mixture-of-Experts model with 355B total and 32B active parameters, and a lighter GLM-4.5-Air (106B total, 12B active). Both models can either reason step by step or answer directly. The design favors depth over width and adds extra attention heads, QK-Norm, and an MTP layer for speculative decoding. Training covers roughly 23T tokens. It begins with quality-bucketed general data and then shifts toward code and science. Mid-training follows, using repository-level code, synthetic reasoning traces, and long-context and agent data, while the context window grows from 4K to 128K. Post-training combines expert model iteration with reinforcement learning. Across 12 ARC benchmarks, GLM-4.5 ranks 3rd overall and 2nd on agentic tasks, despite having far fewer parameters than DeepSeek-R1 or Kimi K2. The open weights and evaluation toolkit give researchers a strong base for agent and coding work.

## Key points
- Headline scores for GLM-4.5: TAU-Bench 70.1%, BFCL v3 77.8%, BrowseComp 26.4%, AIME 24 91.0%, GPQA 79.1%, SWE-bench Verified 64.2%, and Terminal-Bench 37.5%.
- Architecture: GLM-4.5 has 89 MoE layers, 160 experts with 8 active per token, and 96 attention heads with a 5120 hidden dimension. More heads did not lower training loss but did lift reasoning benchmarks such as MMLU and BBH.
- Data recipe:
  - 15T-token general corpus, followed by a 7T code- and reasoning-focused corpus.
  - Mid-training with about 500B tokens of repo-level code, 500B of synthetic reasoning data, and 100B of long-context and agent data.
  - Web deduplication via SemDedup, with multilingual text drawn partly from Fineweb-2.
- In a blind human evaluation on 100 hard translation cases (0–3 scale), GLM-4.5 scored 1.71, against 0.65 or less for Qwen-MT-plus, Qwen-MT-turbo, and Seed-X.
- Weights are released at huggingface.co/zai-org/GLM-4.5, along with the glm-simple-evals toolkit for reproducing the benchmark results.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

Fields: [Natural language processing](https://md.research.elamur.ai/free/en/shelves/nlp/)

## Cited by

Cited by

151 papers of our library cite it (of the 140673 whose bibliographies we read); 3 of them have a page here, 148 not yet.

- [DeepSeek-V3.2: Pushing the Frontier of Open Large Language Models](https://md.research.elamur.ai/free/en/papers/deepseek-2025-ai-deepseek-v3-2-pushing/)
- [Olmo 3](https://md.research.elamur.ai/free/en/papers/ettinger-2025-olmo-3/)
- [Beyond Literal Mapping: Benchmarking and Improving Non-Literal Translation Evaluation](https://md.research.elamur.ai/free/en/papers/tian-2026-beyond-literal-mapping-benchmarking-improving/)

## Cite

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@misc{zeng2025glm,
  title = {{GLM-4.5: Agentic, Reasoning, and Coding (ARC) Foundation Models}},
  author = {Zeng, Aohan and Lv, Xin and Zheng, Qinkai and Hou, Zhenyu and Chen, Bin and Xie, Chengxing and Wang, Cunxiang and Yin, Da and Zeng, Hao and Zhang, Jiajie and Wang, Kedong and Zhong, Lucen and Liu, Mingdao and Lu, Rui and Cao, Shulin and Zhang, Xiaohan and Huang, Xuancheng and Wei, Yao and Cheng, Yean and An, Yifan and Niu, Yilin and Wen, Yuanhao and Bai, Yushi and Du, Zhengxiao and Wang, Zihan and Zhu, Zilin and Zhang, Bohan and Wen, Bosi and Wu, Bowen and Xu, Bowen and Huang, Can and Zhao, Casey and Cai, Changpeng and Yu, Chao and Li, Chen and Ge, Chendi and Huang, Chenghua and Zhang, Chenhui and Xu, Chenxi and Zhu, Chenzheng and Li, Chuang and Yin, Congfeng and Lin, Daoyan and Yang, Dayong and Jiang, Dazhi and Ai, Ding and Zhu, Erle and Wang, Fei and Pan, Gengzheng and Wang, Guo and Sun, Hailong and Li, Haitao and Li, Haiyang and Hu, Haiyi and Zhang, Hanyu and Peng, Hao and Tai, Hao and Zhang, Haoke and Wang, Haoran and Yang, Haoyu and Liu, He and Zhao, He and Liu, Hongwei and Yan, Hongxi and Liu, Huan and Chen, Huilong and Li, Ji and Zhao, Jiajing and Ren, Jiamin and Jiao, Jian and Zhao, Jiani and Yan, Jianyang and Wang, Jiaqi and Gui, Jiayi and Zhao, Jiayue and Liu, Jie and Li, Jijie and Li, Jing and Lu, Jing and Wang, Jingsen and Yuan, Jingwei and Li, Jingxuan and Du, Jingzhao and Du, Jinhua and Liu, Jinxin and Zhi, Junkai and Gao, Junli and Wang, Ke and Yang, Lekang and Xu, Liang and Fan, Lin and Wu, Lindong and Ding, Lintao and Wang, Lu and Zhang, Man and Li, Minghao and Xu, Minghuan and Zhao, Mingming and Zhai, Mingshu and Du, Pengfan and Dong, Qian and Lei, Shangde and Tu, Shangqing and Yang, Shangtong and Lu, Shaoyou and Li, Shijie and Li, Shuang and Shuang-Li and Yang, Shuxun and Yi, Sibo and Yu, Tianshu and Tian, Wei and Wang, Weihan and Yu, Wenbo and Tam, Weng Lam and Liang, Wenjie and Liu, Wentao and Wang, Xiao and Jia, Xiaohan and Gu, Xiaotao and Ling, Xiaoying and Wang, Xin and Fan, Xing and Pan, Xingru and Zhang, Xinyuan and Zhang, Xinze and Fu, Xiuqing and Zhang, Xunkai and Xu, Yabo and Wu, Yandong and Lu, Yida and Wang, Yidong and Zhou, Yilin and Pan, Yiming and Zhang, Ying and Wang, Yingli and Li, Yingru and Su, Yinpei and Geng, Yipeng and Zhu, Yitong and Yang, Yongkun and Li, Yuhang and Wu, Yuhao and Li, Yujiang and Liu, Yunan and Wang, Yunqing and Li, Yuntao and Zhang, Yuxuan and Liu, Zezhen and Yang, Zhen and Zhou, Zhengda and Qiao, Zhongpei and Feng, Zhuoer and Liu, Zhuorui and Zhang, Zichen and Wang, Zihan and Yao, Zijun and Wang, Zikang and Liu, Ziqiang and Chai, Ziwei and Li, Zixuan and Zhao, Zuodong and Chen, Wenguang and Zhai, Jidong and Xu, Bin and Huang, Minlie and Wang, Hongning and Li, Juanzi and Dong, Yuxiao and Tang, Jie},
  year = {2025},
  doi = {10.48550/arxiv.2508.06471},
  eprint = {2508.06471},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2508.06471}
}
```

- [← previous Generating Text from Uniform Meaning Representation](https://md.research.elamur.ai/free/en/papers/markle-2025-generating-text-uniform-meaning-representation/)
- [next → Global MMLU: Understanding and Addressing Cultural and Linguistic Biases in Multilingual Evaluation](https://md.research.elamur.ai/free/en/papers/singh-2025-global-mmlu-understanding-addressing-cultural/)
- [↑ back to papers of 2025](https://md.research.elamur.ai/free/en/papers/2025/page/2/)

## References

52 references, 27 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. A. Abbas, K. Tirumala, D. Simig, S. Ganguli, and A. S. Morcos. Semdedup: Data-efficient learning at web-scale through semantic deduplication. arXiv preprint arXiv:2303.09540, 2023. [find](https://scholar.google.com/scholar?q=A%2E+Abbas%2C+K%2E+Tirumala%2C+D%2E+Simig%2C+S%2E+Ganguli%2C+and+A%2E+S%2E+Morcos%2E+Semdedup%3A+Data%2Defficient+learning+at+web%2Dscale+through+semantic+deduplication%2E+arXiv+preprint+arXiv%3A2303%2E09540%2C+2023%2E) [our card](https://md.research.elamur.ai/free/en/papers/abbas-2023-semdedup-data-efficient-learning-web/)
2. C. An, Z. Xie, X. Li, L. Li, J. Zhang, S. Gong, M. Zhong, J. Xu, X. Qiu, M. Wang, and L. Kong. Polaris: A post-training recipe for scaling reinforcement learning on advanced reasoning models, 2025. [find](https://scholar.google.com/scholar?q=C%2E+An%2C+Z%2E+Xie%2C+X%2E+Li%2C+L%2E+Li%2C+J%2E+Zhang%2C+S%2E+Gong%2C+M%2E+Zhong%2C+J%2E+Xu%2C+X%2E+Qiu%2C+M%2E+Wang%2C+and+L%2E+Kong%2E+Polaris%3A+A+post%2Dtraining+recipe+for+scaling+reinforcement+learning+on+advanced+reasoning+models%2C+2025%2E)
3. Y. Bai, X. Lv, J. Zhang, H. Lyu, J. Tang, Z. Huang, Z. Du, X. Liu, A. Zeng, L. Hou, et al. Longbench: A bilingual, multitask benchmark for long context understanding. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 3119–3137,… [find](https://scholar.google.com/scholar?q=Y%2E+Bai%2C+X%2E+Lv%2C+J%2E+Zhang%2C+H%2E+Lyu%2C+J%2E+Tang%2C+Z%2E+Huang%2C+Z%2E+Du%2C+X%2E+Liu%2C+A%2E+Zeng%2C+L%2E+Hou%2C+et+al%2E+Longbench%3A+A+bilingual%2C+multitask+benchmark+for+long+context+understanding%2E+In+Proceedings+of+the+62nd+Annual) [our card](https://md.research.elamur.ai/free/en/papers/bai-2023-longbench-bilingual-multitask-benchmark/)
4. Y. Bai, S. Tu, J. Zhang, H. Peng, X. Wang, X. Lv, S. Cao, J. Xu, L. Hou, Y. Dong, J. Tang, and J. Li. LongBench v2: Towards deeper understanding and reasoning on realistic long-context multitasks. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: L… [find](https://scholar.google.com/scholar?q=Y%2E+Bai%2C+S%2E+Tu%2C+J%2E+Zhang%2C+H%2E+Peng%2C+X%2E+Wang%2C+X%2E+Lv%2C+S%2E+Cao%2C+J%2E+Xu%2C+L%2E+Hou%2C+Y%2E+Dong%2C+J%2E+Tang%2C+and+J%2E+Li%2E+LongBench+v2%3A+Towards+deeper+understanding+and+reasoning+on+realistic+long%2Dcontext+multitasks%2E+In+) [our card](https://md.research.elamur.ai/free/en/papers/bai-2024-longbench-v2-deeper-understanding/)
5. M. Bavarian, H. Jun, N. Tezak, J. Schulman, C. McLeavey, J. Tworek, and M. Chen. Efficient training of language models to fill in the middle, 2022. [find](https://scholar.google.com/scholar?q=M%2E+Bavarian%2C+H%2E+Jun%2C+N%2E+Tezak%2C+J%2E+Schulman%2C+C%2E+McLeavey%2C+J%2E+Tworek%2C+and+M%2E+Chen%2E+Efficient+training+of+language+models+to+fill+in+the+middle%2C+2022%2E) [our card](https://md.research.elamur.ai/free/en/papers/bavarian-2022-efficient-training-language-models-fill/)
6. T. Brown, B. Mann, N. Ryder, M. Subbiah, J. D. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell, et al. Language models are few-shot learners. Advances in neural information processing systems, 33:1877–1901, 2020. [find](https://scholar.google.com/scholar?q=T%2E+Brown%2C+B%2E+Mann%2C+N%2E+Ryder%2C+M%2E+Subbiah%2C+J%2E+D%2E+Kaplan%2C+P%2E+Dhariwal%2C+A%2E+Neelakantan%2C+P%2E+Shyam%2C+G%2E+Sastry%2C+A%2E+Askell%2C+et+al%2E+Language+models+are+few%2Dshot+learners%2E+Advances+in+neural+information+process)
7. A. Chen, A. Li, B. Gong, B. Jiang, B. Fei, B. Yang, B. Shan, C. Yu, C. Wang, C. Zhu, et al. Minimax-m1: Scaling test-time compute efficiently with lightning attention. arXiv preprint arXiv:2506.13585, 2025. [find](https://scholar.google.com/scholar?q=A%2E+Chen%2C+A%2E+Li%2C+B%2E+Gong%2C+B%2E+Jiang%2C+B%2E+Fei%2C+B%2E+Yang%2C+B%2E+Shan%2C+C%2E+Yu%2C+C%2E+Wang%2C+C%2E+Zhu%2C+et+al%2E+Minimax%2Dm1%3A+Scaling+test%2Dtime+compute+efficiently+with+lightning+attention%2E+arXiv+preprint+arXiv%3A2506%2E13585%2C)
8. M. Chen, J. Tworek, H. Jun, Q. Yuan, H. P. D. O. Pinto, J. Kaplan, H. Edwards, Y. Burda, N. Joseph, G. Brockman, et al. Evaluating large language models trained on code. arXiv preprint arXiv:2107.03374, 2021. [find](https://scholar.google.com/scholar?q=M%2E+Chen%2C+J%2E+Tworek%2C+H%2E+Jun%2C+Q%2E+Yuan%2C+H%2E+P%2E+D%2E+O%2E+Pinto%2C+J%2E+Kaplan%2C+H%2E+Edwards%2C+Y%2E+Burda%2C+N%2E+Joseph%2C+G%2E+Brockman%2C+et+al%2E+Evaluating+large+language+models+trained+on+code%2E+arXiv+preprint+arXiv%3A2107%2E0337) [our card](https://md.research.elamur.ai/free/en/papers/chen-2021-evaluating-large-language-models-trained/)
9. S. Cheng, Y. Bao, Q. Cao, L. Huang, L. Kang, Z. Liu, Y. Lu, W. Zhu, Z. Huang, T. Li, et al. Seed-x: Building strong multilingual translation llm with 7b parameters. arXiv preprint arXiv:2507.13618, 2025. — [arXiv:2507.13618](https://arxiv.org/abs/2507.13618) [find](https://scholar.google.com/scholar?q=S%2E+Cheng%2C+Y%2E+Bao%2C+Q%2E+Cao%2C+L%2E+Huang%2C+L%2E+Kang%2C+Z%2E+Liu%2C+Y%2E+Lu%2C+W%2E+Zhu%2C+Z%2E+Huang%2C+T%2E+Li%2C+et+al%2E+Seed%2Dx%3A+Building+strong+multilingual+translation+llm+with+7b+parameters%2E+arXiv+preprint+arXiv%3A2507%2E13618%2C+20)
10. K. Deshpande, V. Sirdeshmukh, J. B. Mols, L. Jin, E.-Y. Hernandez-Cardona, D. Lee, J. Kritz, W. E. Primack, S. Yue, and C. Xing. Multichallenge: A realistic multi-turn conversation evaluation benchmark challenging to frontier llms. In Findings of the Association for Computational Linguistics: ACL 20… [find](https://scholar.google.com/scholar?q=K%2E+Deshpande%2C+V%2E+Sirdeshmukh%2C+J%2E+B%2E+Mols%2C+L%2E+Jin%2C+E%2E%2DY%2E+Hernandez%2DCardona%2C+D%2E+Lee%2C+J%2E+Kritz%2C+W%2E+E%2E+Primack%2C+S%2E+Yue%2C+and+C%2E+Xing%2E+Multichallenge%3A+A+realistic+multi%2Dturn+conversation+evaluation+benchmar)
11. H. Ding, Z. Wang, G. Paolini, V. Kumar, A. Deoras, D. Roth, and S. Soatto. Fewer truncations improve language modeling. In Proceedings of the 41st International Conference on Machine Learning, pages 11030–11048, 2024. [find](https://scholar.google.com/scholar?q=H%2E+Ding%2C+Z%2E+Wang%2C+G%2E+Paolini%2C+V%2E+Kumar%2C+A%2E+Deoras%2C+D%2E+Roth%2C+and+S%2E+Soatto%2E+Fewer+truncations+improve+language+modeling%2E+In+Proceedings+of+the+41st+International+Conference+on+Machine+Learning%2C+pages+1)
12. F. Gloeckle, B. Y. Idrissi, B. Rozière, D. Lopez-Paz, and G. Synnaeve. Better & faster large language models via multi-token prediction. arXiv preprint arXiv:2404.19737, 2024. [find](https://scholar.google.com/scholar?q=F%2E+Gloeckle%2C+B%2E+Y%2E+Idrissi%2C+B%2E+Rozi%C3%A8re%2C+D%2E+Lopez%2DPaz%2C+and+G%2E+Synnaeve%2E+Better+%26+faster+large+language+models+via+multi%2Dtoken+prediction%2E+arXiv+preprint+arXiv%3A2404%2E19737%2C+2024%2E)
13. D. Guo, D. Yang, H. Zhang, J. Song, R. Zhang, R. Xu, Q. Zhu, S. Ma, P. Wang, X. Bi, et al. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning. arXiv preprint arXiv:2501.12948, 2025. [find](https://scholar.google.com/scholar?q=D%2E+Guo%2C+D%2E+Yang%2C+H%2E+Zhang%2C+J%2E+Song%2C+R%2E+Zhang%2C+R%2E+Xu%2C+Q%2E+Zhu%2C+S%2E+Ma%2C+P%2E+Wang%2C+X%2E+Bi%2C+et+al%2E+Deepseek%2Dr1%3A+Incentivizing+reasoning+capability+in+llms+via+reinforcement+learning%2E+arXiv+preprint+arXiv%3A2501) [our card](https://md.research.elamur.ai/free/en/papers/deepseek-2025-ai-deepseek-r1-incentivizing-reasoning-capability/)
14. D. Hendrycks, C. Burns, S. Kadavath, A. Arora, S. Basart, E. Tang, D. Song, and J. Steinhardt. Measuring mathematical problem solving with the math dataset. In Thirty-fifth Conference on Neural Information Processing Systems Datasets and Benchmarks Track (Round 2). [find](https://scholar.google.com/scholar?q=D%2E+Hendrycks%2C+C%2E+Burns%2C+S%2E+Kadavath%2C+A%2E+Arora%2C+S%2E+Basart%2C+E%2E+Tang%2C+D%2E+Song%2C+and+J%2E+Steinhardt%2E+Measuring+mathematical+problem+solving+with+the+math+dataset%2E+In+Thirty%2Dfifth+Conference+on+Neural+Inform) [our card](https://md.research.elamur.ai/free/en/papers/hendrycks-2021-measuring-mathematical-problem-solving-math/)
15. A. Henry, P. R. Dachapally, S. Pawar, and Y. Chen. Query-key normalization for transformers, 2020. [find](https://scholar.google.com/scholar?q=A%2E+Henry%2C+P%2E+R%2E+Dachapally%2C+S%2E+Pawar%2C+and+Y%2E+Chen%2E+Query%2Dkey+normalization+for+transformers%2C+2020%2E)
16. C.-P. Hsieh, S. Sun, S. Kriman, S. Acharya, D. Rekesh, F. Jia, and B. Ginsburg. Ruler: What’s the real context size of your long-context language models? In First Conference on Language Modeling. [find](https://scholar.google.com/scholar?q=C%2E%2DP%2E+Hsieh%2C+S%2E+Sun%2C+S%2E+Kriman%2C+S%2E+Acharya%2C+D%2E+Rekesh%2C+F%2E+Jia%2C+and+B%2E+Ginsburg%2E+Ruler%3A+What%E2%80%99s+the+real+context+size+of+your+long%2Dcontext+language+models%3F+In+First+Conference+on+Language+Modeling%2E) [our card](https://md.research.elamur.ai/free/en/papers/hsieh-2024-ruler-s-real-context-size/)
17. S. Hu, Y. Tu, X. Han, G. Cui, C. He, W. Zhao, X. Long, Z. Zheng, Y. Fang, Y. Huang, et al. Minicpm: Unveiling the potential of small language models with scalable training strategies. In First Conference on Language Modeling. [find](https://scholar.google.com/scholar?q=S%2E+Hu%2C+Y%2E+Tu%2C+X%2E+Han%2C+G%2E+Cui%2C+C%2E+He%2C+W%2E+Zhao%2C+X%2E+Long%2C+Z%2E+Zheng%2C+Y%2E+Fang%2C+Y%2E+Huang%2C+et+al%2E+Minicpm%3A+Unveiling+the+potential+of+small+language+models+with+scalable+training+strategies%2E+In+First+Confere) [our card](https://md.research.elamur.ai/free/en/papers/hu-2024-minicpm-unveiling-potential-small-language/)
18. A. Jaech, A. Kalai, A. Lerer, A. Richardson, A. El-Kishky, A. Low, A. Helyar, A. Madry, A. Beutel, A. Carney, et al. Openai o1 system card. arXiv preprint arXiv:2412.16720, 2024. [find](https://scholar.google.com/scholar?q=A%2E+Jaech%2C+A%2E+Kalai%2C+A%2E+Lerer%2C+A%2E+Richardson%2C+A%2E+El%2DKishky%2C+A%2E+Low%2C+A%2E+Helyar%2C+A%2E+Madry%2C+A%2E+Beutel%2C+A%2E+Carney%2C+et+al%2E+Openai+o1+system+card%2E+arXiv+preprint+arXiv%3A2412%2E16720%2C+2024%2E) [our card](https://md.research.elamur.ai/free/en/papers/openai-2024-openai-o1-system-card/)
19. N. Jain, K. Han, A. Gu, W.-D. Li, F. Yan, T. Zhang, S. Wang, A. Solar-Lezama, K. Sen, and I. Stoica. Livecodebench: Holistic and contamination free evaluation of large language models for code. In The Thirteenth International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=N%2E+Jain%2C+K%2E+Han%2C+A%2E+Gu%2C+W%2E%2DD%2E+Li%2C+F%2E+Yan%2C+T%2E+Zhang%2C+S%2E+Wang%2C+A%2E+Solar%2DLezama%2C+K%2E+Sen%2C+and+I%2E+Stoica%2E+Livecodebench%3A+Holistic+and+contamination+free+evaluation+of+large+language+models+for+code%2E+In+The) [our card](https://md.research.elamur.ai/free/en/papers/jain-2024-livecodebench-holistic-contamination-free-evaluation/)
20. C. E. Jimenez, J. Yang, A. Wettig, S. Yao, K. Pei, O. Press, and K. Narasimhan. Swe-bench: Can language models resolve real-world github issues? arXiv preprint arXiv:2310.06770, 2023. [find](https://scholar.google.com/scholar?q=C%2E+E%2E+Jimenez%2C+J%2E+Yang%2C+A%2E+Wettig%2C+S%2E+Yao%2C+K%2E+Pei%2C+O%2E+Press%2C+and+K%2E+Narasimhan%2E+Swe%2Dbench%3A+Can+language+models+resolve+real%2Dworld+github+issues%3F+arXiv+preprint+arXiv%3A2310%2E06770%2C+2023%2E) [our card](https://md.research.elamur.ai/free/en/papers/jimenez-2024-swe-bench-can-language-models/)
21. K. Jordan, Y. Jin, V. Boza, Y. Jiacheng, F. Cecista, L. Newhouse, and J. Bernstein. Muon: An optimizer for hidden layers in neural networks, 2024. URL [https://kellerjordan.github.io/posts/muon](https://kellerjordan.github.io/posts/muon), 6. [↗](https://kellerjordan.github.io/posts/muon)
22. A. Joulin, E. Grave, P. Bojanowski, and T. Mikolov. Bag of tricks for efficient text classification. In Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 2, Short Papers, pages 427–431. Association for Computational Linguistics, April… [find](https://scholar.google.com/scholar?q=A%2E+Joulin%2C+E%2E+Grave%2C+P%2E+Bojanowski%2C+and+T%2E+Mikolov%2E+Bag+of+tricks+for+efficient+text+classification%2E+In+Proceedings+of+the+15th+Conference+of+the+European+Chapter+of+the+Association+for+Computational+)
23. A. Liu, B. Feng, B. Xue, B. Wang, B. Wu, C. Lu, C. Zhao, C. Deng, C. Zhang, C. Ruan, et al. Deepseek-v3 technical report. arXiv preprint arXiv:2412.19437, 2024. [find](https://scholar.google.com/scholar?q=A%2E+Liu%2C+B%2E+Feng%2C+B%2E+Xue%2C+B%2E+Wang%2C+B%2E+Wu%2C+C%2E+Lu%2C+C%2E+Zhao%2C+C%2E+Deng%2C+C%2E+Zhang%2C+C%2E+Ruan%2C+et+al%2E+Deepseek%2Dv3+technical+report%2E+arXiv+preprint+arXiv%3A2412%2E19437%2C+2024%2E)
24. J. Liu, J. Su, X. Yao, Z. Jiang, G. Lai, Y. Du, Y. Qin, W. Xu, E. Lu, J. Yan, et al. Muon is scalable for llm training. arXiv preprint arXiv:2502.16982, 2025. [find](https://scholar.google.com/scholar?q=J%2E+Liu%2C+J%2E+Su%2C+X%2E+Yao%2C+Z%2E+Jiang%2C+G%2E+Lai%2C+Y%2E+Du%2C+Y%2E+Qin%2C+W%2E+Xu%2C+E%2E+Lu%2C+J%2E+Yan%2C+et+al%2E+Muon+is+scalable+for+llm+training%2E+arXiv+preprint+arXiv%3A2502%2E16982%2C+2025%2E)
25. M. Luo, S. Tan, J. Wong, X. Shi, W. Y. Tang, M. Roongta, C. Cai, J. Luo, L. E. Li, R. A. Popa, and I. Stoica. Deepscaler: Surpassing o1-preview with a 1.5b model by scaling rl. [https://pretty-radio-b75.notion.site/DeepScaleR-Surpassing-O1-Preview-with-a-1-5B-Model-by-Scaling-RL-19681902c1468005bed8c…](https://pretty-radio-b75.notion.site/DeepScaleR-Surpassing-O1-Preview-with-a-1-5B-Model-by-Scaling-RL-19681902c1468005bed8c…) [↗](https://pretty-radio-b75.notion.site/DeepScaleR-Surpassing-O1-Preview-with-a-1-5B-Model-by-Scaling-RL-19681902c1468005bed8c…)
26. S. G. Patil, H. Mao, C. Cheng-Jie Ji, F. Yan, V. Suresh, I. Stoica, and J. E. Gonzalez. The berkeley function calling leaderboard (bfcl): From tool use to agentic evaluation of large language models. In Forty-second International Conference on Machine Learning, 2025. [find](https://scholar.google.com/scholar?q=S%2E+G%2E+Patil%2C+H%2E+Mao%2C+C%2E+Cheng%2DJie+Ji%2C+F%2E+Yan%2C+V%2E+Suresh%2C+I%2E+Stoica%2C+and+J%2E+E%2E+Gonzalez%2E+The+berkeley+function+calling+leaderboard+%28bfcl%29%3A+From+tool+use+to+agentic+evaluation+of+large+language+models%2E+)
27. G. Penedo, H. Kydlíček, V. Sabolčec, B. Messmer, N. Foroutan, A. H. Kargaran, C. Raffel, M. Jaggi, L. Von Werra, and T. Wolf. Fineweb2: One pipeline to scale them all–adapting pre-training data processing to every language. arXiv preprint arXiv:2506.20920, 2025. [find](https://scholar.google.com/scholar?q=G%2E+Penedo%2C+H%2E+Kydl%C3%ADc%CC%8Cek%2C+V%2E+Sabolc%CC%8Cec%2C+B%2E+Messmer%2C+N%2E+Foroutan%2C+A%2E+H%2E+Kargaran%2C+C%2E+Raffel%2C+M%2E+Jaggi%2C+L%2E+Von+Werra%2C+and+T%2E+Wolf%2E+Fineweb2%3A+One+pipeline+to+scale+them+all%E2%80%93adapting+pre%2Dtraining+data+proc)
28. L. Phan, A. Gatti, Z. Han, N. Li, J. Hu, H. Zhang, C. B. C. Zhang, M. Shaaban, J. Ling, S. Shi, et al. Humanity’s last exam. arXiv preprint arXiv:2501.14249, 2025. [find](https://scholar.google.com/scholar?q=L%2E+Phan%2C+A%2E+Gatti%2C+Z%2E+Han%2C+N%2E+Li%2C+J%2E+Hu%2C+H%2E+Zhang%2C+C%2E+B%2E+C%2E+Zhang%2C+M%2E+Shaaban%2C+J%2E+Ling%2C+S%2E+Shi%2C+et+al%2E+Humanity%E2%80%99s+last+exam%2E+arXiv+preprint+arXiv%3A2501%2E14249%2C+2025%2E) [our card](https://md.research.elamur.ai/free/en/papers/phan-2025-humanity-s-last-exam/)
29. Y. Qin, T. Zhang, Y. Shen, W. Luo, Y. Zhang, Y. Qiao, Z. Zhou, W. Zhang, B. CUI, et al. Sysbench: Can llms follow system message? In The Thirteenth International Conference on Learning Representations, 2024. [find](https://scholar.google.com/scholar?q=Y%2E+Qin%2C+T%2E+Zhang%2C+Y%2E+Shen%2C+W%2E+Luo%2C+Y%2E+Zhang%2C+Y%2E+Qiao%2C+Z%2E+Zhou%2C+W%2E+Zhang%2C+B%2E+CUI%2C+et+al%2E+Sysbench%3A+Can+llms+follow+system+message%3F+In+The+Thirteenth+International+Conference+on+Learning+Representations)
30. D. Rein, B. L. Hou, A. C. Stickland, J. Petty, R. Y. Pang, J. Dirani, J. Michael, and S. R. Bowman. Gpqa: A graduate-level google-proof q&a benchmark. In First Conference on Language Modeling, 2024. [find](https://scholar.google.com/scholar?q=D%2E+Rein%2C+B%2E+L%2E+Hou%2C+A%2E+C%2E+Stickland%2C+J%2E+Petty%2C+R%2E+Y%2E+Pang%2C+J%2E+Dirani%2C+J%2E+Michael%2C+and+S%2E+R%2E+Bowman%2E+Gpqa%3A+A+graduate%2Dlevel+google%2Dproof+q%26a+benchmark%2E+In+First+Conference+on+Language+Modeling%2C+2024%2E) [our card](https://md.research.elamur.ai/free/en/papers/rein-2023-gpqa-graduate-level-google-proof/)
31. Z. Shao, P. Wang, Q. Zhu, R. Xu, J. Song, X. Bi, H. Zhang, M. Zhang, Y. Li, Y. Wu, et al. Deepseekmath: Pushing the limits of mathematical reasoning in open language models. arXiv preprint arXiv:2402.03300, 2024. [find](https://scholar.google.com/scholar?q=Z%2E+Shao%2C+P%2E+Wang%2C+Q%2E+Zhu%2C+R%2E+Xu%2C+J%2E+Song%2C+X%2E+Bi%2C+H%2E+Zhang%2C+M%2E+Zhang%2C+Y%2E+Li%2C+Y%2E+Wu%2C+et+al%2E+Deepseekmath%3A+Pushing+the+limits+of+mathematical+reasoning+in+open+language+models%2E+arXiv+preprint+arXiv%3A2402%2E) [our card](https://md.research.elamur.ai/free/en/papers/shao-2024-deepseekmath-pushing-limits-mathematical-reasoning/)
32. D. Su, K. Kong, Y. Lin, J. Jennings, B. Norick, M. Kliegl, M. Patwary, M. Shoeybi, and B. Catanzaro. Nemotron-cc: Transforming common crawl into a refined long-horizon pretraining dataset. arXiv preprint arXiv:2412.02595, 2024. [find](https://scholar.google.com/scholar?q=D%2E+Su%2C+K%2E+Kong%2C+Y%2E+Lin%2C+J%2E+Jennings%2C+B%2E+Norick%2C+M%2E+Kliegl%2C+M%2E+Patwary%2C+M%2E+Shoeybi%2C+and+B%2E+Catanzaro%2E+Nemotron%2Dcc%3A+Transforming+common+crawl+into+a+refined+long%2Dhorizon+pretraining+dataset%2E+arXiv+prepr)
33. G. Team, R. Anil, S. Borgeaud, J.-B. Alayrac, J. Yu, R. Soricut, J. Schalkwyk, A. M. Dai, A. Hauth, K. Millican, et al. Gemini: a family of highly capable multimodal models. arXiv preprint arXiv:2312.11805, 2023. [find](https://scholar.google.com/scholar?q=G%2E+Team%2C+R%2E+Anil%2C+S%2E+Borgeaud%2C+J%2E%2DB%2E+Alayrac%2C+J%2E+Yu%2C+R%2E+Soricut%2C+J%2E+Schalkwyk%2C+A%2E+M%2E+Dai%2C+A%2E+Hauth%2C+K%2E+Millican%2C+et+al%2E+Gemini%3A+a+family+of+highly+capable+multimodal+models%2E+arXiv+preprint+arXiv%3A2312%2E) [our card](https://md.research.elamur.ai/free/en/papers/gemini-2023-team-gemini-family-highly-capable-multimodal/)
34. K. Team, Y. Bai, Y. Bao, G. Chen, J. Chen, N. Chen, R. Chen, Y. Chen, Y. Chen, Y. Chen, et al. Kimi k2: Open agentic intelligence. arXiv preprint arXiv:2507.20534, 2025. [find](https://scholar.google.com/scholar?q=K%2E+Team%2C+Y%2E+Bai%2C+Y%2E+Bao%2C+G%2E+Chen%2C+J%2E+Chen%2C+N%2E+Chen%2C+R%2E+Chen%2C+Y%2E+Chen%2C+Y%2E+Chen%2C+Y%2E+Chen%2C+et+al%2E+Kimi+k2%3A+Open+agentic+intelligence%2E+arXiv+preprint+arXiv%3A2507%2E20534%2C+2025%2E) [our card](https://md.research.elamur.ai/free/en/papers/kimi-2025-team-kimi-k2-open-agentic/)
35. T. T.-B. Team. Terminal-bench: A benchmark for ai agents in terminal environments, Apr 2025. [find](https://scholar.google.com/scholar?q=T%2E+T%2E%2DB%2E+Team%2E+Terminal%2Dbench%3A+A+benchmark+for+ai+agents+in+terminal+environments%2C+Apr+2025%2E)
36. M. Tian, L. Gao, S. Zhang, X. Chen, C. Fan, X. Guo, R. Haas, P. Ji, K. Krongchon, Y. Li, et al. Scicode: A research coding benchmark curated by scientists. Advances in Neural Information Processing Systems, 37:30624–30650, 2024. [find](https://scholar.google.com/scholar?q=M%2E+Tian%2C+L%2E+Gao%2C+S%2E+Zhang%2C+X%2E+Chen%2C+C%2E+Fan%2C+X%2E+Guo%2C+R%2E+Haas%2C+P%2E+Ji%2C+K%2E+Krongchon%2C+Y%2E+Li%2C+et+al%2E+Scicode%3A+A+research+coding+benchmark+curated+by+scientists%2E+Advances+in+Neural+Information+Processing+Sy)
37. H. Touvron, T. Lavril, G. Izacard, X. Martinet, M.-A. Lachaux, T. Lacroix, B. Rozière, N. Goyal, E. Hambro, F. Azhar, et al. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971, 2023. [find](https://scholar.google.com/scholar?q=H%2E+Touvron%2C+T%2E+Lavril%2C+G%2E+Izacard%2C+X%2E+Martinet%2C+M%2E%2DA%2E+Lachaux%2C+T%2E+Lacroix%2C+B%2E+Rozi%C3%A8re%2C+N%2E+Goyal%2C+E%2E+Hambro%2C+F%2E+Azhar%2C+et+al%2E+Llama%3A+Open+and+efficient+foundation+language+models%2E+arXiv+preprint+arXiv%3A)
38. K. Vodrahalli, S. Ontanon, N. Tripuraneni, K. Xu, S. Jain, R. Shivanna, J. Hui, N. Dikkala, M. Kazemi, B. Fatemi, R. Anil, E. Dyer, S. Shakeri, R. Vij, H. Mehta, V. Ramasesh, Q. Le, E. Chi, Y. Lu, O. Firat, A. Lazaridou, J.-B. Lespiau, N. Attaluri, and K. Olszewska. Michelangelo: Long context evalua… [find](https://scholar.google.com/scholar?q=K%2E+Vodrahalli%2C+S%2E+Ontanon%2C+N%2E+Tripuraneni%2C+K%2E+Xu%2C+S%2E+Jain%2C+R%2E+Shivanna%2C+J%2E+Hui%2C+N%2E+Dikkala%2C+M%2E+Kazemi%2C+B%2E+Fatemi%2C+R%2E+Anil%2C+E%2E+Dyer%2C+S%2E+Shakeri%2C+R%2E+Vij%2C+H%2E+Mehta%2C+V%2E+Ramasesh%2C+Q%2E+Le%2C+E%2E+Chi%2C+Y%2E+Lu%2C+O%2E+)
39. F. Wan, W. Shen, S. Liao, Y. Shi, C. Li, Z. Yang, J. Zhang, F. Huang, J. Zhou, and M. Yan. Qwenlong-l1: Towards long-context large reasoning models with reinforcement learning. arXiv preprint arXiv:2505.17667, 2025. [find](https://scholar.google.com/scholar?q=F%2E+Wan%2C+W%2E+Shen%2C+S%2E+Liao%2C+Y%2E+Shi%2C+C%2E+Li%2C+Z%2E+Yang%2C+J%2E+Zhang%2C+F%2E+Huang%2C+J%2E+Zhou%2C+and+M%2E+Yan%2E+Qwenlong%2Dl1%3A+Towards+long%2Dcontext+large+reasoning+models+with+reinforcement+learning%2E+arXiv+preprint+arXiv%3A25)
40. L. Wang, H. Gao, C. Zhao, X. Sun, and D. Dai. Auxiliary-loss-free load balancing strategy for mixture-of-experts. arXiv preprint arXiv:2408.15664, 2024. — [arXiv:2408.15664](https://arxiv.org/abs/2408.15664) [find](https://scholar.google.com/scholar?q=L%2E+Wang%2C+H%2E+Gao%2C+C%2E+Zhao%2C+X%2E+Sun%2C+and+D%2E+Dai%2E+Auxiliary%2Dloss%2Dfree+load+balancing+strategy+for+mixture%2Dof%2Dexperts%2E+arXiv+preprint+arXiv%3A2408%2E15664%2C+2024%2E+%E2%80%94+%5BarXiv%3A2408%2E15664%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2408)
41. S. Wang, L. Yu, C. Gao, C. Zheng, S. Liu, R. Lu, K. Dang, X. Chen, J. Yang, Z. Zhang, et al. Beyond the 80/20 rule: High-entropy minority tokens drive effective reinforcement learning for llm reasoning. arXiv preprint arXiv:2506.01939, 2025. [find](https://scholar.google.com/scholar?q=S%2E+Wang%2C+L%2E+Yu%2C+C%2E+Gao%2C+C%2E+Zheng%2C+S%2E+Liu%2C+R%2E+Lu%2C+K%2E+Dang%2C+X%2E+Chen%2C+J%2E+Yang%2C+Z%2E+Zhang%2C+et+al%2E+Beyond+the+80%2F20+rule%3A+High%2Dentropy+minority+tokens+drive+effective+reinforcement+learning+for+llm+reasonin) [our card](https://md.research.elamur.ai/free/en/papers/wang-2025-80-20-rule-high-entropy/)
42. X. Wang, B. Li, Y. Song, F. F. Xu, X. Tang, M. Zhuge, J. Pan, Y. Song, B. Li, J. Singh, H. H. Tran, F. Li, R. Ma, M. Zheng, B. Qian, Y. Shao, N. Muennighoff, Y. Zhang, B. Hui, J. Lin, R. Brennan, H. Peng, H. Ji, and G. Neubig. Openhands: An open platform for AI software developers as generalist agen… [find](https://scholar.google.com/scholar?q=X%2E+Wang%2C+B%2E+Li%2C+Y%2E+Song%2C+F%2E+F%2E+Xu%2C+X%2E+Tang%2C+M%2E+Zhuge%2C+J%2E+Pan%2C+Y%2E+Song%2C+B%2E+Li%2C+J%2E+Singh%2C+H%2E+H%2E+Tran%2C+F%2E+Li%2C+R%2E+Ma%2C+M%2E+Zheng%2C+B%2E+Qian%2C+Y%2E+Shao%2C+N%2E+Muennighoff%2C+Y%2E+Zhang%2C+B%2E+Hui%2C+J%2E+Lin%2C+R%2E+Brennan%2C+H%2E+P) [our card](https://md.research.elamur.ai/free/en/papers/wang-2024-openhands-open-platform-ai/)
43. Y. Wang, X. Ma, G. Zhang, Y. Ni, A. Chandra, S. Guo, W. Ren, A. Arulraj, X. He, Z. Jiang, et al. Mmlu-pro: A more robust and challenging multi-task language understanding benchmark. Advances in Neural Information Processing Systems, 37:95266–95290, 2024. [find](https://scholar.google.com/scholar?q=Y%2E+Wang%2C+X%2E+Ma%2C+G%2E+Zhang%2C+Y%2E+Ni%2C+A%2E+Chandra%2C+S%2E+Guo%2C+W%2E+Ren%2C+A%2E+Arulraj%2C+X%2E+He%2C+Z%2E+Jiang%2C+et+al%2E+Mmlu%2Dpro%3A+A+more+robust+and+challenging+multi%2Dtask+language+understanding+benchmark%2E+Advances+in+Neural) [our card](https://md.research.elamur.ai/free/en/papers/wang-2024-mmlu-pro-more-robust-challenging/)
44. J. Wei, N. Karina, H. W. Chung, Y. J. Jiao, S. Papay, A. Glaese, J. Schulman, and W. Fedus. Measuring short-form factuality in large language models, 2024. [find](https://scholar.google.com/scholar?q=J%2E+Wei%2C+N%2E+Karina%2C+H%2E+W%2E+Chung%2C+Y%2E+J%2E+Jiao%2C+S%2E+Papay%2C+A%2E+Glaese%2C+J%2E+Schulman%2C+and+W%2E+Fedus%2E+Measuring+short%2Dform+factuality+in+large+language+models%2C+2024%2E) [our card](https://md.research.elamur.ai/free/en/papers/wei-2024-measuring-short-form-factuality-large/)
45. J. Wei, Z. Sun, S. Papay, S. McKinney, J. Han, I. Fulford, H. W. Chung, A. T. Passos, W. Fedus, and A. Glaese. Browsecomp: A simple yet challenging benchmark for browsing agents. arXiv preprint arXiv:2504.12516, 2025. [find](https://scholar.google.com/scholar?q=J%2E+Wei%2C+Z%2E+Sun%2C+S%2E+Papay%2C+S%2E+McKinney%2C+J%2E+Han%2C+I%2E+Fulford%2C+H%2E+W%2E+Chung%2C+A%2E+T%2E+Passos%2C+W%2E+Fedus%2C+and+A%2E+Glaese%2E+Browsecomp%3A+A+simple+yet+challenging+benchmark+for+browsing+agents%2E+arXiv+preprint+arXiv%3A)
46. Z. Xi, Y. Ding, W. Chen, B. Hong, H. Guo, J. Wang, D. Yang, C. Liao, X. Guo, W. He, et al. Agentgym: Evolving large language model-based agents across diverse environments. arXiv preprint arXiv:2406.04151, 2024. [find](https://scholar.google.com/scholar?q=Z%2E+Xi%2C+Y%2E+Ding%2C+W%2E+Chen%2C+B%2E+Hong%2C+H%2E+Guo%2C+J%2E+Wang%2C+D%2E+Yang%2C+C%2E+Liao%2C+X%2E+Guo%2C+W%2E+He%2C+et+al%2E+Agentgym%3A+Evolving+large+language+model%2Dbased+agents+across+diverse+environments%2E+arXiv+preprint+arXiv%3A2406%2E0)
47. A. Yang, A. Li, B. Yang, B. Zhang, B. Hui, B. Zheng, B. Yu, C. Gao, C. Huang, C. Lv, et al. Qwen3 technical report. arXiv preprint arXiv:2505.09388, 2025. [find](https://scholar.google.com/scholar?q=A%2E+Yang%2C+A%2E+Li%2C+B%2E+Yang%2C+B%2E+Zhang%2C+B%2E+Hui%2C+B%2E+Zheng%2C+B%2E+Yu%2C+C%2E+Gao%2C+C%2E+Huang%2C+C%2E+Lv%2C+et+al%2E+Qwen3+technical+report%2E+arXiv+preprint+arXiv%3A2505%2E09388%2C+2025%2E) [our card](https://md.research.elamur.ai/free/en/papers/yang-2025-qwen3-technical-report/)
48. S. Yao, N. Shinn, P. Razavi, and K. Narasimhan. tau-bench: A benchmark for tool-agent-user interaction in real-world domains. arXiv preprint arXiv:2406.12045, 2024. [find](https://scholar.google.com/scholar?q=S%2E+Yao%2C+N%2E+Shinn%2C+P%2E+Razavi%2C+and+K%2E+Narasimhan%2E+tau%2Dbench%3A+A+benchmark+for+tool%2Dagent%2Duser+interaction+in+real%2Dworld+domains%2E+arXiv+preprint+arXiv%3A2406%2E12045%2C+2024%2E) [our card](https://md.research.elamur.ai/free/en/papers/yao-2024-tau-bench-benchmark-tool/)
49. Q. Yu, Z. Zhang, R. Zhu, Y. Yuan, X. Zuo, Y. Yue, W. Dai, T. Fan, G. Liu, L. Liu, et al. Dapo: An open-source llm reinforcement learning system at scale. arXiv preprint arXiv:2503.14476, 2025. [find](https://scholar.google.com/scholar?q=Q%2E+Yu%2C+Z%2E+Zhang%2C+R%2E+Zhu%2C+Y%2E+Yuan%2C+X%2E+Zuo%2C+Y%2E+Yue%2C+W%2E+Dai%2C+T%2E+Fan%2C+G%2E+Liu%2C+L%2E+Liu%2C+et+al%2E+Dapo%3A+An+open%2Dsource+llm+reinforcement+learning+system+at+scale%2E+arXiv+preprint+arXiv%3A2503%2E14476%2C+2025%2E) [our card](https://md.research.elamur.ai/free/en/papers/yu-2025-dapo-open-source-llm-reinforcement/)
50. A. Zeng, X. Liu, Z. Du, Z. Wang, H. Lai, M. Ding, Z. Yang, Y. Xu, W. Zheng, X. Xia, et al. Glm-130b: An open bilingual pre-trained model. In The Eleventh International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=A%2E+Zeng%2C+X%2E+Liu%2C+Z%2E+Du%2C+Z%2E+Wang%2C+H%2E+Lai%2C+M%2E+Ding%2C+Z%2E+Yang%2C+Y%2E+Xu%2C+W%2E+Zheng%2C+X%2E+Xia%2C+et+al%2E+Glm%2D130b%3A+An+open+bilingual+pre%2Dtrained+model%2E+In+The+Eleventh+International+Conference+on+Learning+Represent) [our card](https://md.research.elamur.ai/free/en/papers/zeng-2022-glm-130b-open-bilingual-pre/)
51. Z. Zhang, L. Lei, L. Wu, R. Sun, Y. Huang, C. Long, X. Liu, X. Lei, J. Tang, and M. Huang. Safetybench: Evaluating the safety of large language models with multiple choice questions. arXiv preprint arXiv:2309.07045, 2023. [find](https://scholar.google.com/scholar?q=Z%2E+Zhang%2C+L%2E+Lei%2C+L%2E+Wu%2C+R%2E+Sun%2C+Y%2E+Huang%2C+C%2E+Long%2C+X%2E+Liu%2C+X%2E+Lei%2C+J%2E+Tang%2C+and+M%2E+Huang%2E+Safetybench%3A+Evaluating+the+safety+of+large+language+models+with+multiple+choice+questions%2E+arXiv+preprint+ar) [our card](https://md.research.elamur.ai/free/en/papers/zhang-2023-safetybench-evaluating-safety-large/)
52. J. Zhou, T. Lu, S. Mishra, S. Brahma, S. Basu, Y. Luan, D. Zhou, and L. Hou. Instruction-following evaluation for large language models. arXiv preprint arXiv:2311.07911, 2023. [find](https://scholar.google.com/scholar?q=J%2E+Zhou%2C+T%2E+Lu%2C+S%2E+Mishra%2C+S%2E+Brahma%2C+S%2E+Basu%2C+Y%2E+Luan%2C+D%2E+Zhou%2C+and+L%2E+Hou%2E+Instruction%2Dfollowing+evaluation+for+large+language+models%2E+arXiv+preprint+arXiv%3A2311%2E07911%2C+2023%2E) [our card](https://md.research.elamur.ai/free/en/papers/zhou-2023-instruction-following-evaluation-large-language/)