---
title: "EfficientTool — paper by Mu et al. (2026) — summary and ideas"
description: "Small and medium-sized enterprises (SMEs) struggle to align proprietary LLM agents. 51 references."
authors: ["Yuanqi Mu", "Bingfeng Pi", "Defei Xia", "Lei Zuo", "Yongqi Zhang"]
year: 2026
venue: "Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 6: Industry Track)"
source: "https://aclanthology.org/2026.acl-industry.56/"
doi: "10.18653/v1/2026.acl-industry.56"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/mu-2026-efficienttool-cost-effective-aligning-framework/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/mu-2026-efficienttool-cost-effective-aligning-framework/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/28/) / EfficientTool: A Cost-Effective Aligning Framework for Tool-Conditioned Agents in SME Scenarios

# EfficientTool: A Cost-Effective Aligning Framework for Tool-Conditioned Agents in SME Scenarios

**Authors:** Yuanqi Mu, Bingfeng Pi, Defei Xia, Lei Zuo and Yongqi Zhang · **Year:** 2026 · **Venue:** [Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 6: Industry Track)](https://md.research.elamur.ai/free/en/venues/acl/) ([ACL 2026](https://md.research.elamur.ai/free/en/venues/acl/2026/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2026.acl-industry.56/) · [PDF](https://aclanthology.org/2026.acl-industry.56.pdf) · [DOI 10.18653/v1/2026.acl-industry.56](https://doi.org/10.18653/v1/2026.acl-industry.56) · [OpenAlex](https://openalex.org/W7166822659)

**Citations:** 0 citations (OpenAlex, 2026-10-14)

## Summary
Small and medium-sized enterprises (SMEs) struggle to align proprietary LLM agents. They have little curated data, tight GPU budgets, fragmented tool ecosystems, and cannot afford separate reward models or pair-wise preference datasets. EfficientTool tackles this with a closed loop of three pipelines:

- **Trajectory Collection Pipeline:** a teacher model expands hand-written seed queries. A filter with rule-based, teacher-model and human layers then cleans the generated trajectories, followed by deduplication and replay.
- **Model Training Pipeline:** LoRA-based SFT is followed by Kahneman-Tversky Optimization (KTO). KTO learns from individually labeled good or bad trajectories rather than paired comparisons.
- **Interaction and Evaluation Pipeline:** automatic, LLM-judge and expert checks decide whether training moves to the next stage or a new iteration starts.

Tool retrieval is also improved through intent refinement and multi-path vector search. The authors report successful alignment on private and public benchmarks without losing general tool-calling ability, along with much lower compute cost. The framework is deployed in a production customer-service agent. The work matters as a practical recipe for affordable agent adaptation outside large labs.

## Key points
- **Models and setup:** Qwen3-32B is the base model, DeepSeek-R1-Distill-Llama-70B the teacher, and Qwen3-Embedding-0.6B handles retrieval. Training uses LLaMA-Factory on 8× A100 (40GB) GPUs. Qwen3-Max produces the first 20% of trajectories, after which the lightly SFT-tuned model takes over.
- **Labeling cost:** sampling positive cases for manual review according to model performance reportedly cuts human labeling effort by about 81%.
- **Evaluation:** the automatic level uses the 4T metric (TCR, TFR, TIR, TPS). An LLM judge scores five quality dimensions. Ten senior architects give accept/reject verdicts decided by majority vote.
- **Resource comparison:** the most economical setting reported uses about 35 GB per GPU and 6.6 h on 4k entries; the authors attribute these savings to SFT+KTO with LoRA. Extrapolated alternatives such as GRPO or full SFT need up to 120 GB and 110 h.
- **Deployment (DataSteward):** the agent integrates 700+ APIs and 100 domain datasets. It fully resolves 39.6% of inquiries and partly addresses 27.7%, while 32.6% are escalated to humans. Earlier human-only support reached 21.6% proactive satisfaction.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [The Hong Kong University of Science and Technology (Guangzhou)](https://md.research.elamur.ai/free/en/universities/the-hong-kong-university-of-science-and-technology-guangzhou/) — University, Guangzhou, China · 53 works · 1,367 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works

## Cite

Cite as: Yuanqi Mu, Bingfeng Pi, Defei Xia, Lei Zuo and Yongqi Zhang. 2026. EfficientTool: A Cost-Effective Aligning Framework for Tool-Conditioned Agents in SME Scenarios. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 6: Industry Track).

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{mu2026efficienttool,
  title = {{EfficientTool: A Cost-Effective Aligning Framework for Tool-Conditioned Agents in SME Scenarios}},
  author = {Yuanqi Mu, Bingfeng Pi, Defei Xia, Lei Zuo and Yongqi Zhang},
  booktitle = {Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 6: Industry Track)},
  year = {2026},
  doi = {10.18653/v1/2026.acl-industry.56},
  url = {https://doi.org/10.18653/v1/2026.acl-industry.56}
}
```

- [← previous Efficient universal generation in a fragment of Optimality Theory](https://md.research.elamur.ai/free/en/papers/siewert-2026-efficient-universal-generation-fragment-optimality/)
- [next → EgoMemory: Memory-Augmented Personalized Retrieval for Long-Context Egocentric Video](https://md.research.elamur.ai/free/en/papers/tang-2026-egomemory-memory-augmented-personalized-retrieval/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/28/)

## References

51 references, 22 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Emre Can Acikgoz, Jeremiah Greer, Akul Datta, Ze Yang, William Zeng, Oussama Elachqar, Emmanouil Koukoumidis, Dilek Hakkani-Tur, and Gokhan Tur. 2025. [Can a single model master both multi-turn conversations and tool use? coalm: A unified conversational agentic language model](https://md.research.elamur.ai/free/en/papers/acikgoz-2025-can-single-model-master-both/). In Proceedings of the 6…
2. Afra Amini, Tim Vieira, and Ryan Cotterell. 2024. Direct preference optimization with an offset. In Findings of the Association for Computational Linguistics ACL 2024, pages 9954–9972. [find](https://scholar.google.com/scholar?q=Afra+Amini%2C+Tim+Vieira%2C+and+Ryan+Cotterell%2E+2024%2E+Direct+preference+optimization+with+an+offset%2E+In+Findings+of+the+Association+for+Computational+Linguistics+ACL+2024%2C+pages+9954%E2%80%939972%2E)
3. Xiaoyin Chen and Sam Wiseman. 2023. Bm25 query augmentation learned end-to-end. arXiv preprint arXiv:2305.14087. — [arXiv:2305.14087](https://arxiv.org/abs/2305.14087) [find](https://scholar.google.com/scholar?q=Xiaoyin+Chen+and+Sam+Wiseman%2E+2023%2E+Bm25+query+augmentation+learned+end%2Dto%2Dend%2E+arXiv+preprint+arXiv%3A2305%2E14087%2E+%E2%80%94+%5BarXiv%3A2305%2E14087%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2305%2E14087%29)
4. Wei-Lin Chiang, Zhuohan Li, Zi Lin, Ying Sheng, Zhanghao Wu, Hao Zhang, Lianmin Zheng, Siyuan Zhuang, Yonghao Zhuang, Joseph E. Gonzalez, Ion Stoica, and Eric P. Xing. 2023. Vicuna: An open-source chatbot impressing gpt-4 with 90 [https://lmsys.org/blog/2023-03-30-vicuna/](https://lmsys.org/blog/2023-03-30-vicuna/). LM-SYS Org. [↗](https://lmsys.org/blog/2023-03-30-vicuna/)
5. Gordon V Cormack, Charles LA Clarke, and Stefan Buettcher. 2009. [Reciprocal rank fusion outperforms condorcet and individual rank learning methods](https://md.research.elamur.ai/free/en/papers/cormack-2009-reciprocal-rank-fusion-outperforms/). In Proceedings of the 32nd international ACM SIGIR conference on Research and development in information retrieval, pages 758–759.
6. DeepSeek-AI. 2025. [Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning](https://md.research.elamur.ai/free/en/papers/deepseek-ai-2025-deepseek-r1-incentivizing-reasoning-capability/). Preprint, arXiv:2501.12948.
7. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. [Bert: Pre-training of deep bidirectional transformers for language understanding](https://md.research.elamur.ai/free/en/papers/devlin-2019-bert-pre-training-deep-bidirectional/). In Proceedings of the 2019 conference of the North American chapter of the association for computational linguistics: human language technologies,…
8. Yuyang Ding, Chi Zhang, Juntao Li, Haibin Lin, Xin Liu, and Min Zhang. 2025. Fapo: Flawed-aware policy optimization for efficient and reliable reasoning. arXiv preprint arXiv:2510.22543. — [arXiv:2510.22543](https://arxiv.org/abs/2510.22543) [find](https://scholar.google.com/scholar?q=Yuyang+Ding%2C+Chi+Zhang%2C+Juntao+Li%2C+Haibin+Lin%2C+Xin+Liu%2C+and+Min+Zhang%2E+2025%2E+Fapo%3A+Flawed%2Daware+policy+optimization+for+efficient+and+reliable+reasoning%2E+arXiv+preprint+arXiv%3A2510%2E22543%2E+%E2%80%94+%5BarXiv%3A2510)
9. Kawin Ethayarajh, Winnie Xu, Niklas Muennighoff, Dan Jurafsky, and Douwe Kiela. 2024. Kto: Model alignment as prospect theoretic optimization. arXiv preprint arXiv:2402.01306. [find](https://scholar.google.com/scholar?q=Kawin+Ethayarajh%2C+Winnie+Xu%2C+Niklas+Muennighoff%2C+Dan+Jurafsky%2C+and+Douwe+Kiela%2E+2024%2E+Kto%3A+Model+alignment+as+prospect+theoretic+optimization%2E+arXiv+preprint+arXiv%3A2402%2E01306%2E)
10. Shiqing Fan, Xichen Ding, Liang Zhang, and Linjian Mo. 2025. Mcptoolbench++: A large scale ai agent model context protocol mcp tool use benchmark. arXiv preprint arXiv:2508.07575. — [arXiv:2508.07575](https://arxiv.org/abs/2508.07575) [find](https://scholar.google.com/scholar?q=Shiqing+Fan%2C+Xichen+Ding%2C+Liang+Zhang%2C+and+Linjian+Mo%2E+2025%2E+Mcptoolbench%2B%2B%3A+A+large+scale+ai+agent+model+context+protocol+mcp+tool+use+benchmark%2E+arXiv+preprint+arXiv%3A2508%2E07575%2E+%E2%80%94+%5BarXiv%3A2508%2E07575%5D)
11. Yuchuan Fu, Xiaohan Yuan, and Dongxia Wang. 2025. Ras-eval: A comprehensive benchmark for security evaluation of llm agents in real-world environments. arXiv preprint arXiv:2506.15253. — [arXiv:2506.15253](https://arxiv.org/abs/2506.15253) [find](https://scholar.google.com/scholar?q=Yuchuan+Fu%2C+Xiaohan+Yuan%2C+and+Dongxia+Wang%2E+2025%2E+Ras%2Deval%3A+A+comprehensive+benchmark+for+security+evaluation+of+llm+agents+in+real%2Dworld+environments%2E+arXiv+preprint+arXiv%3A2506%2E15253%2E+%E2%80%94+%5BarXiv%3A2506%2E1)
12. Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, and 1 others. 2024. [The llama 3 herd of models](https://md.research.elamur.ai/free/en/papers/grattafiori-2024-llama-3-herd-models/). arXiv preprint arXiv:2407.21783.
13. Aaron Harlap, Deepak Narayanan, Amar Phanishayee, Vivek Seshadri, Nikhil Devanur, Greg Ganger, and Phil Gibbons. 2018. Pipedream: Fast and efficient pipeline parallel dnn training. arXiv preprint arXiv:1806.03377. [find](https://scholar.google.com/scholar?q=Aaron+Harlap%2C+Deepak+Narayanan%2C+Amar+Phanishayee%2C+Vivek+Seshadri%2C+Nikhil+Devanur%2C+Greg+Ganger%2C+and+Phil+Gibbons%2E+2018%2E+Pipedream%3A+Fast+and+efficient+pipeline+parallel+dnn+training%2E+arXiv+preprint+arXi)
14. Jiwoo Hong, Noah Lee, and James Thorne. 2024. Orpo: Monolithic preference optimization without reference model. In 2024 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Jiwoo+Hong%2C+Noah+Lee%2C+and+James+Thorne%2E+2024%2E+Orpo%3A+Monolithic+preference+optimization+without+reference+model%2E+In+2024+Conference+on+Empirical+Methods+in+Natural+Language+Processing%2E+Association+for+)
15. Neil Houlsby, Andrei Giurgiu, Stanislaw Jastrzebski, Bruna Morrone, Quentin De Laroussilhe, Andrea Gesmundo, Mona Attariyan, and Sylvain Gelly. 2019. [find](https://scholar.google.com/scholar?q=Neil+Houlsby%2C+Andrei+Giurgiu%2C+Stanislaw+Jastrzebski%2C+Bruna+Morrone%2C+Quentin+De+Laroussilhe%2C+Andrea+Gesmundo%2C+Mona+Attariyan%2C+and+Sylvain+Gelly%2E+2019%2E)
16. Parameter-efficient transfer learning for nlp. In International conference on machine learning, pages 2790–2799. PMLR. [find](https://scholar.google.com/scholar?q=Parameter%2Defficient+transfer+learning+for+nlp%2E+In+International+conference+on+machine+learning%2C+pages+2790%E2%80%932799%2E+PMLR%2E)
17. Pin-Lun Hsu, Yun Dai, Vignesh Kothapalli, Qingquan Song, Shao Tang, Siyu Zhu, Steven Shimizu, Shivam Sahni, Haowen Ning, Yanning Chen, and Zhipeng Wang. 2025. Liger-kernel: Efficient triton kernels for LLM training. In Championing Open-source DEvelopment in ML Workshop @ ICML25. [find](https://scholar.google.com/scholar?q=Pin%2DLun+Hsu%2C+Yun+Dai%2C+Vignesh+Kothapalli%2C+Qingquan+Song%2C+Shao+Tang%2C+Siyu+Zhu%2C+Steven+Shimizu%2C+Shivam+Sahni%2C+Haowen+Ning%2C+Yanning+Chen%2C+and+Zhipeng+Wang%2E+2025%2E+Liger%2Dkernel%3A+Efficient+triton+kernels+fo)
18. Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, Weizhu Chen, and 1 others. 2022. [Lora: Low-rank adaptation of large language models](https://md.research.elamur.ai/free/en/papers/hu-2021-lora-low-rank-adaptation-large/). ICLR, 1(2):3.
19. Nikhil Kandpal and Colin Raffel. 2025. Position: The most expensive part of an llm should be its training data. arXiv preprint arXiv:2504.12427. [find](https://scholar.google.com/scholar?q=Nikhil+Kandpal+and+Colin+Raffel%2E+2025%2E+Position%3A+The+most+expensive+part+of+an+llm+should+be+its+training+data%2E+arXiv+preprint+arXiv%3A2504%2E12427%2E)
20. Aixin Liu, Aoxue Mei, Bangcai Lin, Bing Xue, Bingxuan Wang, Bingzheng Xu, Bochao Wu, Bowei Zhang, Chaofan Lin, Chen Dong, and 1 others. 2025. [Deepseek-v3.2: Pushing the frontier of open large language models](https://md.research.elamur.ai/free/en/papers/deepseek-ai-2025-deepseek-v3-2-pushing/). arXiv preprint arXiv:2512.02556.
21. Weiwen Liu, Xu Huang, Xingshan Zeng, Xinlong Hao, Shuai Yu, Dexun Li, Shuai Wang, Weinan Gan, Zhengying Liu, Yuanqing Yu, Zezhong Wang, Yuxian Wang, Wu Ning, Yutai Hou, Bin Wang, Chuhan Wu, Xinzhi Wang, Yong Liu, Yasheng Wang, and 8 others. 2024. Toolace: Winning the points of llm function calling.… [find](https://scholar.google.com/scholar?q=Weiwen+Liu%2C+Xu+Huang%2C+Xingshan+Zeng%2C+Xinlong+Hao%2C+Shuai+Yu%2C+Dexun+Li%2C+Shuai+Wang%2C+Weinan+Gan%2C+Zhengying+Liu%2C+Yuanqing+Yu%2C+Zezhong+Wang%2C+Yuxian+Wang%2C+Wu+Ning%2C+Yutai+Hou%2C+Bin+Wang%2C+Chuhan+Wu%2C+Xinzhi+Wan)
22. Yun Luo, Zhen Yang, Fandong Meng, Yafu Li, Jie Zhou, and Yue Zhang. 2025. An empirical study of catastrophic forgetting in large language models during continual fine-tuning. IEEE Transactions on Audio, Speech and Language Processing. [find](https://scholar.google.com/scholar?q=Yun+Luo%2C+Zhen+Yang%2C+Fandong+Meng%2C+Yafu+Li%2C+Jie+Zhou%2C+and+Yue+Zhang%2E+2025%2E+An+empirical+study+of+catastrophic+forgetting+in+large+language+models+during+continual+fine%2Dtuning%2E+IEEE+Transactions+on+Audi)
23. Shiho Matta, Yin Jou Huang, Fei Cheng, Hirokazu Kiyomaru, and Yugo Murawaki. 2025. Optimizing cost-efficiency with llm-generated training data for conversational semantic frame analysis. In Proceedings of the 9th Joint SIGHUM Workshop on Computational Linguistics for Cultural Heritage, Social Scienc… [find](https://scholar.google.com/scholar?q=Shiho+Matta%2C+Yin+Jou+Huang%2C+Fei+Cheng%2C+Hirokazu+Kiyomaru%2C+and+Yugo+Murawaki%2E+2025%2E+Optimizing+cost%2Defficiency+with+llm%2Dgenerated+training+data+for+conversational+semantic+frame+analysis%2E+In+Proceeding)
24. Yu Meng, Mengzhou Xia, and Danqi Chen. 2024. Simpo: Simple preference optimization with a reference-free reward. Advances in Neural Information Processing Systems, 37:124198–124235. [find](https://scholar.google.com/scholar?q=Yu+Meng%2C+Mengzhou+Xia%2C+and+Danqi+Chen%2E+2024%2E+Simpo%3A+Simple+preference+optimization+with+a+reference%2Dfree+reward%2E+Advances+in+Neural+Information+Processing+Systems%2C+37%3A124198%E2%80%93124235%2E)
25. NVIDIA. 2025. [Nemotron 3 Nano: Open, efficient mixture-of-experts hybrid Mamba-Transformer model for Agentic reasoning](https://md.research.elamur.ai/free/en/papers/blakeman-2025-nemotron-3-nano-open-efficient/). Technical report.
26. Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, and 1 others. 2022. [Training language models to follow instructions with human feedback](https://md.research.elamur.ai/free/en/papers/ouyang-2022-training-language-models-follow-instructions/). Advances in neural information processing systems, 35:27730–27744.
27. Shishir G Patil, Huanzhi Mao, Fanjia Yan, Charlie Cheng-Jie Ji, Vishnu Suresh, Ion Stoica, and Joseph E. Gonzalez. 2025. The berkeley function calling leaderboard (BFCL): From tool use to agentic evaluation of large language models. In Forty-second International Conference on Machine Learning. [find](https://scholar.google.com/scholar?q=Shishir+G+Patil%2C+Huanzhi+Mao%2C+Fanjia+Yan%2C+Charlie+Cheng%2DJie+Ji%2C+Vishnu+Suresh%2C+Ion+Stoica%2C+and+Joseph+E%2E+Gonzalez%2E+2025%2E+The+berkeley+function+calling+leaderboard+%28BFCL%29%3A+From+tool+use+to+agentic+eval)
28. Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D Manning, Stefano Ermon, and Chelsea Finn. 2023. [Direct preference optimization: Your language model is secretly a reward model](https://md.research.elamur.ai/free/en/papers/rafailov-2023-direct-preference-optimization-language-model/). Advances in neural information processing systems, 36:53728–53741.
29. Samyam Rajbhandari, Jeff Rasley, Olatunji Ruwase, and Yuxiong He. 2020. Zero: Memory optimizations toward training trillion parameter models. In SC20: International Conference for High Performance Computing, Networking, Storage and Analysis, pages 1– 16. IEEE. [find](https://scholar.google.com/scholar?q=Samyam+Rajbhandari%2C+Jeff+Rasley%2C+Olatunji+Ruwase%2C+and+Yuxiong+He%2E+2020%2E+Zero%3A+Memory+optimizations+toward+training+trillion+parameter+models%2E+In+SC20%3A+International+Conference+for+High+Performance+Com)
30. Clemens Rawert, Marc Klingen, and Maximilian De-ichmann. 2023. Langfuse — open-source llm engineering platform. Software available from [https://langfuse.com](https://langfuse.com). [↗](https://langfuse.com)
31. Ville Satopaa, Jeannie Albrecht, David Irwin, and Barath Raghavan. 2011. Finding a" kneedle" in a haystack: Detecting knee points in system behavior. In 2011 31st international conference on distributed computing systems workshops, pages 166–171. IEEE. [find](https://scholar.google.com/scholar?q=Ville+Satopaa%2C+Jeannie+Albrecht%2C+David+Irwin%2C+and+Barath+Raghavan%2E+2011%2E+Finding+a%22+kneedle%22+in+a+haystack%3A+Detecting+knee+points+in+system+behavior%2E+In+2011+31st+international+conference+on+distribut)
32. John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. 2017. Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347. [find](https://scholar.google.com/scholar?q=John+Schulman%2C+Filip+Wolski%2C+Prafulla+Dhariwal%2C+Alec+Radford%2C+and+Oleg+Klimov%2E+2017%2E+Proximal+policy+optimization+algorithms%2E+arXiv+preprint+arXiv%3A1707%2E06347%2E)
33. Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, and 1 others. 2024. [Deepseekmath: Pushing the limits of mathematical reasoning in open language models](https://md.research.elamur.ai/free/en/papers/shao-2024-deepseekmath-pushing-limits-mathematical-reasoning/). arXiv preprint arXiv:2402.03300.
34. Guangming Sheng, Chi Zhang, Zilingfeng Ye, Xibin Wu, Wang Zhang, Ru Zhang, Yanghua Peng, Haibin Lin, and Chuan Wu. 2024. [Hybridflow: A flexible and efficient rlhf framework](https://md.research.elamur.ai/free/en/papers/sheng-2024-hybridflow-flexible-efficient-rlhf-framework/). arXiv preprint arXiv: 2409.19256.
35. Kimi Team, Yifan Bai, Yiping Bao, Guanduo Chen, Jiahao Chen, Ningxin Chen, Ruijue Chen, Yanru Chen, Yuankun Chen, Yutian Chen, Zhuofu Chen, Jialei Cui, Hao Ding, Mengnan Dong, Angang Du, Chenzhuang Du, Dikang Du, Yulun Du, Yu Fan, and 150 others. 2025a. [Kimi k2: Open agentic intelligence](https://md.research.elamur.ai/free/en/papers/kimi-team-2025-kimi-k2-open-agentic/). Preprint,…
36. V Team, Wenyi Hong, Wenmeng Yu, Xiaotao Gu, Guo Wang, Guobing Gan, Haomiao Tang, Jiale Cheng, Ji Qi, Junhui Ji, Lihang Pan, Shuaiqi Duan, Weihan Wang, Yan Wang, Yean Cheng, Zehai He, Zhe Su, Zhen Yang, Ziyang Pan, and 69 others. 2025b. [Glm-4.5v and glm-4.1v-thinking: Towards versatile multimodal rea…](https://md.research.elamur.ai/free/en/papers/team-2025-v-glm-4-5v-glm/)
37. Leandro von Werra, Younes Belkada, Lewis Tunstall, Edward Beeching, Tristan Thrush, Nathan Lambert, Shengyi Huang, Kashif Rasul, and Quentin Gallouédec. 2020. TRL: Transformers Reinforcement Learning. [find](https://scholar.google.com/scholar?q=Leandro+von+Werra%2C+Younes+Belkada%2C+Lewis+Tunstall%2C+Edward+Beeching%2C+Tristan+Thrush%2C+Nathan+Lambert%2C+Shengyi+Huang%2C+Kashif+Rasul%2C+and+Quentin+Gallou%C3%A9dec%2E+2020%2E+TRL%3A+Transformers+Reinforcement+Learning%2E)
38. Xingyao Wang, Boxuan Li, Yufan Song, Frank F Xu, Xiangru Tang, Mingchen Zhuge, Jiayi Pan, Yueqi Song, Bowen Li, Jaskirat Singh, and 1 others. 2024. [Openhands: An open platform for ai software developers as generalist agents](https://md.research.elamur.ai/free/en/papers/wang-2024-openhands-open-platform-ai/). arXiv preprint arXiv:2407.16741.
39. Yubo Wang, Xueguang Ma, Ping Nie, Huaye Zeng, Zhiheng Lyu, Yuxuan Zhang, Benjamin Schneider, Yi Lu, Xiang Yue, and Wenhu Chen. 2025. Scholarcopilot: Training large language models for academic writing with accurate citations. CoRR. [find](https://scholar.google.com/scholar?q=Yubo+Wang%2C+Xueguang+Ma%2C+Ping+Nie%2C+Huaye+Zeng%2C+Zhiheng+Lyu%2C+Yuxuan+Zhang%2C+Benjamin+Schneider%2C+Yi+Lu%2C+Xiang+Yue%2C+and+Wenhu+Chen%2E+2025%2E+Scholarcopilot%3A+Training+large+language+models+for+academic+writing)
40. Defei Xia, Bingfeng Pi, Shenbin Zhang, Song Hua, Yunfei Wei, and Lei Zuo. 2026. Jenius agent: Towards experience-driven accuracy optimization in real-world scenarios. arXiv preprint arXiv:2601.01857. — [arXiv:2601.01857](https://arxiv.org/abs/2601.01857) [find](https://scholar.google.com/scholar?q=Defei+Xia%2C+Bingfeng+Pi%2C+Shenbin+Zhang%2C+Song+Hua%2C+Yunfei+Wei%2C+and+Lei+Zuo%2E+2026%2E+Jenius+agent%3A+Towards+experience%2Ddriven+accuracy+optimization+in+real%2Dworld+scenarios%2E+arXiv+preprint+arXiv%3A2601%2E01857%2E+)
41. Weikai Xu, Chengrui Huang, Shen Gao, and Shuo Shang. 2025. Llm-based agents for tool learning: A survey. Data Science and Engineering, pages 1– 31. [find](https://scholar.google.com/scholar?q=Weikai+Xu%2C+Chengrui+Huang%2C+Shen+Gao%2C+and+Shuo+Shang%2E+2025%2E+Llm%2Dbased+agents+for+tool+learning%3A+A+survey%2E+Data+Science+and+Engineering%2C+pages+1%E2%80%93+31%2E)
42. An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, and 1 others. 2025. [Qwen3 technical report](https://md.research.elamur.ai/free/en/papers/yang-2025-qwen3-technical-report/). arXiv preprint arXiv:2505.09388.
43. An Yang, Baosong Yang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Zhou, Chengpeng Li, Chengyuan Li, Dayiheng Liu, Fei Huang, Guanting Dong, Haoran Wei, Huan Lin, Jialong Tang, Jialin Wang, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Ma, and 40 others. 2024. Qwen2 technical report. arXiv preprint arXi… [find](https://scholar.google.com/scholar?q=An+Yang%2C+Baosong+Yang%2C+Binyuan+Hui%2C+Bo+Zheng%2C+Bowen+Yu%2C+Chang+Zhou%2C+Chengpeng+Li%2C+Chengyuan+Li%2C+Dayiheng+Liu%2C+Fei+Huang%2C+Guanting+Dong%2C+Haoran+Wei%2C+Huan+Lin%2C+Jialong+Tang%2C+Jialin+Wang%2C+Jian+Yang%2C+Jian)
44. Longhui Yu, Weisen Jiang, Han Shi, Jincheng Yu, Zhengying Liu, Yu Zhang, James T Kwok, Zhenguo Li, Adrian Weller, and Weiyang Liu. 2023. [Metamath: Bootstrap your own mathematical questions for large language models](https://md.research.elamur.ai/free/en/papers/yu-2023-metamath-bootstrap-your-own-mathematical/). arXiv preprint arXiv:2309.12284.
45. Qiying Yu, Zheng Zhang, Ruofei Zhu, Yufeng Yuan, Xiaochen Zuo, Yu Yue, Tiantian Fan, Gaohong Liu, Lingjun Liu, Xin Liu, and 1 others. 2025. [Dapo: An open-source llm reinforcement learning system at scale](https://md.research.elamur.ai/free/en/papers/yu-2025-dapo-open-source-llm-reinforcement/). CoRR.
46. Jianguo Zhang, Tian Lan, Ming Zhu, Zuxin Liu, Thai Quoc Hoang, Shirley Kokane, Weiran Yao, Juntao Tan, Akshara Prabhakar, Haolin Chen, and 1 others. 2025a. [xlam: A family of large action models to empower ai agent systems](https://md.research.elamur.ai/free/en/papers/zhang-2025-xlam-family-large-action-models/). In Proceedings of the 2025 Conference of the Nations of the Americas Chapter…
47. Shaokun Zhang, Yi Dong, Jieyu Zhang, Jan Kautz, Bryan Catanzaro, Andrew Tao, Qingyun Wu, Zhiding Yu, and Guilin Liu. 2025b. Nemotron-research-tool-n1: Tool-using language models with reinforced reasoning. arXiv preprint arXiv:2505.00024. — [arXiv:2505.00024](https://arxiv.org/abs/2505.00024) [find](https://scholar.google.com/scholar?q=Shaokun+Zhang%2C+Yi+Dong%2C+Jieyu+Zhang%2C+Jan+Kautz%2C+Bryan+Catanzaro%2C+Andrew+Tao%2C+Qingyun+Wu%2C+Zhiding+Yu%2C+and+Guilin+Liu%2E+2025b%2E+Nemotron%2Dresearch%2Dtool%2Dn1%3A+Tool%2Dusing+language+models+with+reinforced+reason)
48. Yanzhao Zhang, Mingxin Li, Dingkun Long, Xin Zhang, Huan Lin, Baosong Yang, Pengjun Xie, An Yang, Dayiheng Liu, Junyang Lin, and 1 others. 2025c. [Qwen3 embedding: Advancing text embedding and reranking through foundation models](https://md.research.elamur.ai/free/en/papers/zhang-2025-qwen3-embedding-advancing-text/). arXiv preprint arXiv:2506.05176.
49. Yuzhong Zhao, Yue Liu, Junpeng Liu, Jingye Chen, Xun Wu, Yaru Hao, Tengchao Lv, Shaohan Huang, Lei Cui, Qixiang Ye, and 1 others. 2025. Geometric-mean policy optimization. arXiv preprint arXiv:2507.20673. — [arXiv:2507.20673](https://arxiv.org/abs/2507.20673) [find](https://scholar.google.com/scholar?q=Yuzhong+Zhao%2C+Yue+Liu%2C+Junpeng+Liu%2C+Jingye+Chen%2C+Xun+Wu%2C+Yaru+Hao%2C+Tengchao+Lv%2C+Shaohan+Huang%2C+Lei+Cui%2C+Qixiang+Ye%2C+and+1+others%2E+2025%2E+Geometric%2Dmean+policy+optimization%2E+arXiv+preprint+arXiv%3A2507%2E20)
50. Chujie Zheng, Shixuan Liu, Mingze Li, Xiong-Hui Chen, Bowen Yu, Chang Gao, Kai Dang, Yuqiong Liu, Rui Men, An Yang, and 1 others. 2025. [Group sequence policy optimization](https://md.research.elamur.ai/free/en/papers/zheng-2025-group-sequence-policy-optimization/). arXiv preprint arXiv:2507.18071.
51. Yaowei Zheng, Richong Zhang, Junhao Zhang, Yanhan Ye, Zheyan Luo, Zhangchi Feng, and Yongqiang Ma. 2024. [Llamafactory: Unified efficient fine-tuning of 100+ language models](https://md.research.elamur.ai/free/en/papers/zheng-2024-llamafactory-unified-efficient-fine-tuning/). In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 3: System Demonstrations),…

*Updated 1 November 2026*