---
title: "Meeseeks — paper by Wang et al. (2026) — summary and ideas"
description: "Large language models often fail to satisfy every requirement in prompts that contain many constraints. 56 references, cited by 1 paper here."
authors: ["Jiaming Wang", "Yunke Zhao", "Peng Ding", "Jun Kuang", "Yibin Shen", "Zhe Tang", "Yilin Jin", "ZongYu Wang", "Xiaoyu Li", "Xuezhi Cao"]
year: 2026
venue: "Findings of the Association for Computational Linguistics: ACL 2026"
source: "https://aclanthology.org/2026.findings-acl.725/"
doi: "10.18653/v1/2026.findings-acl.725"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/wang-2026-meeseeks-feedback-driven-iterative-self/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/wang-2026-meeseeks-feedback-driven-iterative-self/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/15/) / Meeseeks: A Feedback-Driven, Iterative Self-Correction Benchmark evaluating LLMs' Instruction Following Capability

# Meeseeks: A Feedback-Driven, Iterative Self-Correction Benchmark evaluating LLMs' Instruction Following Capability

**Authors:** [Jiaming Wang](https://md.research.elamur.ai/free/en/authors/jiaming-wang/), Yunke Zhao, [Peng Ding](https://md.research.elamur.ai/free/en/authors/peng-ding/), Jun Kuang, Yibin Shen, [Zhe Tang](https://md.research.elamur.ai/free/en/authors/zhe-tang/), [Yilin Jin](https://md.research.elamur.ai/free/en/authors/yilin-jin/), ZongYu Wang, [Xiaoyu Li](https://md.research.elamur.ai/free/en/authors/xiaoyu-li/) and [Xuezhi Cao](https://md.research.elamur.ai/free/en/authors/xuezhi-cao/) · **Year:** 2026 · **Venue:** [Findings of the Association for Computational Linguistics: ACL 2026](https://md.research.elamur.ai/free/en/venues/findings/) ([Findings of the Association for Computational Linguistics 2026](https://md.research.elamur.ai/free/en/venues/findings/2026/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2026.findings-acl.725/) · [PDF](https://aclanthology.org/2026.findings-acl.725.pdf) · [DOI 10.18653/v1/2026.findings-acl.725](https://doi.org/10.18653/v1/2026.findings-acl.725) · [OpenAlex](https://openalex.org/W7166838090)

**Citations:** 0 citations (OpenAlex, 2026-10-14) · [1](https://md.research.elamur.ai/free/en/papers/wang-2026-meeseeks-feedback-driven-iterative-self/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
Large language models often fail to satisfy every requirement in prompts that contain many constraints. Meeseeks is a benchmark that measures how far models can improve when they get repeated, targeted feedback. It works in cycles. The tested model answers a prompt, and each constraint is checked automatically. The model then receives explanations of which requirements it missed and why, and it revises its answer over up to 20 rounds.

The dataset has over 700 curated Chinese and English instances. Each carries some of 32 capability tags, and the instances fall under three broad categories: understanding what the user intends, checking content at a fine-grained level, and verifying the structure of the output. To keep repeated judging cheap and reliable, the authors have an LLM evaluator write code that extracts the relevant parts of a response. Python rule scripts then verify the constraints, so the evaluator never has to rewrite the entire answer.

Across 17 commercial and open-source models, results vary widely. Even after 20 correction rounds, no model fully solves the benchmark. This exposes a clear gap between first-attempt performance and each model's correction ceiling.

## Key points
- Resource: over 700 instances in Chinese and English with 32 capability tags, open-sourced at github.com/ADoublLEN/Meeseeks.
- The code-guided, rule-augmented LLM-based evaluation raises judging accuracy from 78.7% to 98.4% compared with the regenerate-and-clean approach.
- Across 17 models and 20 feedback rounds, final accuracy stays at or below about 90–91%.
- Word-count constraints trigger overcorrection that swings in both directions, and models handle small counts better. Many models also misread meta-tasks that ask them to edit an embedded prompt.
- Reasoning models generally beat non-reasoning ones and gain more from feedback. Only o3-mini (high) used feedback to fix Chinese tonal-pattern (pingze) tasks.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

📚 Shelves (5): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works · [👁️ Computer Vision and Pattern Recognition](https://md.research.elamur.ai/free/en/shelves/computer-science/computer-vision-and-pattern-recognition/) · 2,869 works

🧭 Topics (2): [🧠 Language models & pretraining](https://md.research.elamur.ai/free/en/topics/language-models/) · 3,320 works · [📏 Evaluation & benchmarks](https://md.research.elamur.ai/free/en/topics/evaluation-benchmarks/) · 1,220 works

## Cite

Cite as: Jiaming Wang, Yunke Zhao, Peng Ding, Jun Kuang, Yibin Shen, Zhe Tang, Yilin Jin, ZongYu Wang, Xiaoyu Li and Xuezhi Cao. 2026. Meeseeks: A Feedback-Driven, Iterative Self-Correction Benchmark evaluating LLMs' Instruction Following Capability. Findings of the Association for Computational Linguistics: ACL 2026.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{wang2026meeseeks,
  title = {{Meeseeks: A Feedback-Driven, Iterative Self-Correction Benchmark evaluating LLMs' Instruction Following Capability}},
  author = {Jiaming Wang, Yunke Zhao, Peng Ding, Jun Kuang, Yibin Shen, Zhe Tang, Yilin Jin, ZongYu Wang, Xiaoyu Li and Xuezhi Cao},
  booktitle = {Findings of the Association for Computational Linguistics: ACL 2026},
  year = {2026},
  doi = {10.18653/v1/2026.findings-acl.725},
  url = {https://doi.org/10.18653/v1/2026.findings-acl.725}
}
```

- [← previous MedMind AI at #SMM4H-HeaRD 2026: Data Extraction and Generation Using Prompt Engineering and Structured Outputs (Tasks 1–6)](https://md.research.elamur.ai/free/en/papers/pradhan-2026-medmind-ai-smm4h-heard-2026/)
- [next → Mem2ActBench: A Benchmark for Evaluating Long-Term Memory Utilization in Task-Oriented Autonomous Agents](https://md.research.elamur.ai/free/en/papers/shen-2026-mem2actbench-benchmark-evaluating-long-term/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/15/)

## References

56 references, 21 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. anthropic. 2025. Claude sonnet 4. Visited date: 2025-09-15. [find](https://scholar.google.com/scholar?q=anthropic%2E+2025%2E+Claude+sonnet+4%2E+Visited+date%3A+2025%2D09%2D15%2E)
2. Abhinav Arun, Ashish Dhiman, Mehul Soni, and Yibei Hu. 2023. Numerical reasoning for financial reports. Preprint, arXiv:2312.14870. — [arXiv:2312.14870](https://arxiv.org/abs/2312.14870) [find](https://scholar.google.com/scholar?q=Abhinav+Arun%2C+Ashish+Dhiman%2C+Mehul+Soni%2C+and+Yibei+Hu%2E+2023%2E+Numerical+reasoning+for+financial+reports%2E+Preprint%2C+arXiv%3A2312%2E14870%2E+%E2%80%94+%5BarXiv%3A2312%2E14870%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2312%2E14870%29)
3. Yejin Bang, Ziwei Ji, Alan Schelten, Anthony Hartshorn, Tara Fowler, Cheng Zhang, Nicola Cancedda, and Pascale Fung. 2025. [Hallulens: Llm hallucination benchmark](https://md.research.elamur.ai/free/en/papers/bang-2025-hallulens-llm-hallucination-benchmark/). Preprint, arXiv:2504.17550.
4. Bytedance. 2025. Doubao-ai. Visited date: 2025-09-15. [find](https://scholar.google.com/scholar?q=Bytedance%2E+2025%2E+Doubao%2Dai%2E+Visited+date%3A+2025%2D09%2D15%2E)
5. Kedi Chen, Qin Chen, Jie Zhou, Yishen He, and Liang He. 2024. Diahalu: A dialogue-level hallucination evaluation benchmark for large language models. Preprint, arXiv:2403.00896. [find](https://scholar.google.com/scholar?q=Kedi+Chen%2C+Qin+Chen%2C+Jie+Zhou%2C+Yishen+He%2C+and+Liang+He%2E+2024%2E+Diahalu%3A+A+dialogue%2Dlevel+hallucination+evaluation+benchmark+for+large+language+models%2E+Preprint%2C+arXiv%3A2403%2E00896%2E)
6. DeepSeek-AI. 2024. Deepseek-v3 technical report. Preprint, arXiv:2412.19437. [find](https://scholar.google.com/scholar?q=DeepSeek%2DAI%2E+2024%2E+Deepseek%2Dv3+technical+report%2E+Preprint%2C+arXiv%3A2412%2E19437%2E)
7. Deep Ganguli, Amanda Askell, Nicholas Schiefer, Thomas I. Liao, Kamilė Lukošiūtė, Anna Chen, Anna Goldie, Azalia Mirhoseini, Catherine Olsson, Danny Hernandez, Dawn Drain, Dustin Li, Eli Tran-Johnson, Ethan Perez, Jackson Kernion, Jamie Kerr, Jared Mueller, Joshua Landau, Kamal Ndousse, and 30 ot… — [The Capacity for Moral Self-Correction in Large Language Models](https://md.research.elamur.ai/free/en/papers/ganguli-2023-capacity-moral-self-correction-large/)
8. Google DeepMind. 2025. Gemini flash. Visited date: 2025-09-15. [find](https://scholar.google.com/scholar?q=Google+DeepMind%2E+2025%2E+Gemini+flash%2E+Visited+date%3A+2025%2D09%2D15%2E)
9. Zishan Gu, Changchang Yin, Fenglin Liu, and Ping Zhang. 2024. Medvh: Towards systematic evaluation of hallucination for large vision language models in the medical context. Preprint, arXiv:2407.02730. — [arXiv:2407.02730](https://arxiv.org/abs/2407.02730) [find](https://scholar.google.com/scholar?q=Zishan+Gu%2C+Changchang+Yin%2C+Fenglin+Liu%2C+and+Ping+Zhang%2E+2024%2E+Medvh%3A+Towards+systematic+evaluation+of+hallucination+for+large+vision+language+models+in+the+medical+context%2E+Preprint%2C+arXiv%3A2407%2E02730%2E)
10. Yijie Hao, Haofei Yu, and Jiaxuan You. 2025. Beyond facts: Evaluating intent hallucination in large language models. Preprint, arXiv:2506.06539. [find](https://scholar.google.com/scholar?q=Yijie+Hao%2C+Haofei+Yu%2C+and+Jiaxuan+You%2E+2025%2E+Beyond+facts%3A+Evaluating+intent+hallucination+in+large+language+models%2E+Preprint%2C+arXiv%3A2506%2E06539%2E)
11. Qianyu He, Jie Zeng, Wenhao Huang, Lina Chen, Jin Xiao, Qianxi He, Xunzhe Zhou, Lida Chen, Xintao Wang, Yuncheng Huang, Haoning Ye, Zihan Li, Shisong Chen, Yikai Zhang, Zhouhong Gu, Jiaqing Liang, and Yanghua Xiao. 2024a. Can large language models understand real-world complex instructions? Preprint… — [arXiv:2309.09150](https://arxiv.org/abs/2309.09150) [find](https://scholar.google.com/scholar?q=Qianyu+He%2C+Jie+Zeng%2C+Wenhao+Huang%2C+Lina+Chen%2C+Jin+Xiao%2C+Qianxi+He%2C+Xunzhe+Zhou%2C+Lida+Chen%2C+Xintao+Wang%2C+Yuncheng+Huang%2C+Haoning+Ye%2C+Zihan+Li%2C+Shisong+Chen%2C+Yikai+Zhang%2C+Zhouhong+Gu%2C+Jiaqing+Liang%2C+and)
12. Yun He, Di Jin, Chaoqi Wang, Chloe Bi, Karishma Mandyam, Hejia Zhang, Chen Zhu, Ning Li, Tengyu Xu, Hongjiang Lv, Shruti Bhosale, Chenguang Zhu, Karthik Abinav Sankararaman, Eryk Helenowski, Melanie Kambadur, Aditya Tayade, Hao Ma, Han Fang, and Sinong Wang. 2024b. Multi-if: Benchmarking llms on mul… — [arXiv:2410.15553](https://arxiv.org/abs/2410.15553) [find](https://scholar.google.com/scholar?q=Yun+He%2C+Di+Jin%2C+Chaoqi+Wang%2C+Chloe+Bi%2C+Karishma+Mandyam%2C+Hejia+Zhang%2C+Chen+Zhu%2C+Ning+Li%2C+Tengyu+Xu%2C+Hongjiang+Lv%2C+Shruti+Bhosale%2C+Chenguang+Zhu%2C+Karthik+Abinav+Sankararaman%2C+Eryk+Helenowski%2C+Melanie+K)
13. Tatsuya Hiraoka and Naoaki Okazaki. 2024. Knowledge of pretrained language models on surface information of tokens. Preprint, arXiv:2402.09808. — [arXiv:2402.09808](https://arxiv.org/abs/2402.09808) [find](https://scholar.google.com/scholar?q=Tatsuya+Hiraoka+and+Naoaki+Okazaki%2E+2024%2E+Knowledge+of+pretrained+language+models+on+surface+information+of+tokens%2E+Preprint%2C+arXiv%3A2402%2E09808%2E+%E2%80%94+%5BarXiv%3A2402%2E09808%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2402%2E09808%29)
14. Bairu Hou, Qibin Chen, Jianyu Wang, Guoli Yin, Chong Wang, Nan Du, Ruoming Pang, Shiyu Chang, and Tao Lei. 2025. Instruction-following pruning for large language models. Preprint, arXiv:2501.02086. — [arXiv:2501.02086](https://arxiv.org/abs/2501.02086) [find](https://scholar.google.com/scholar?q=Bairu+Hou%2C+Qibin+Chen%2C+Jianyu+Wang%2C+Guoli+Yin%2C+Chong+Wang%2C+Nan+Du%2C+Ruoming+Pang%2C+Shiyu+Chang%2C+and+Tao+Lei%2E+2025%2E+Instruction%2Dfollowing+pruning+for+large+language+models%2E+Preprint%2C+arXiv%3A2501%2E02086%2E+%E2%80%94+)
15. Jie Huang, Xinyun Chen, Swaroop Mishra, Huaixiu Steven Zheng, Adams Wei Yu, Xinying Song, and Denny Zhou. 2024. [Large language models cannot self-correct reasoning yet](https://md.research.elamur.ai/free/en/papers/huang-2023-large-language-models-cannot-self/). Preprint, arXiv:2310.01798.
16. Lei Huang, Weijiang Yu, Weitao Ma, Weihong Zhong, Zhangyin Feng, Haotian Wang, Qianglong Chen, Weihua Peng, Xiaocheng Feng, Bing Qin, and Ting Liu. 2025. [A survey on hallucination in large language models: Principles, taxonomy, challenges, and open questions](https://md.research.elamur.ai/free/en/papers/huang-2023-survey-hallucination-large-language-models/). ACM Transactions on Information Systems,…
17. Yuxin Jiang, Yufei Wang, Xingshan Zeng, Wanjun Zhong, Liangyou Li, Fei Mi, Lifeng Shang, Xin Jiang, Qun Liu, and Wei Wang. 2024. [Followbench: A multi-level fine-grained constraints following benchmark for large language models](https://md.research.elamur.ai/free/en/papers/jiang-2024-followbench-multi-level-fine-grained/). Preprint, arXiv:2310.20410.
18. Guy Kaplan, Matanel Oren, Yuval Reif, and Roy Schwartz. 2025. From tokens to words: On the inner lexicon of llms. Preprint, arXiv:2410.05864. — [arXiv:2410.05864](https://arxiv.org/abs/2410.05864) [find](https://scholar.google.com/scholar?q=Guy+Kaplan%2C+Matanel+Oren%2C+Yuval+Reif%2C+and+Roy+Schwartz%2E+2025%2E+From+tokens+to+words%3A+On+the+inner+lexicon+of+llms%2E+Preprint%2C+arXiv%3A2410%2E05864%2E+%E2%80%94+%5BarXiv%3A2410%2E05864%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2410%2E05864%29)
19. Alex Kim, Maximilian Muhn, and Valeri Nikolaev. 2025. Financial statement analysis with large language models. Preprint, arXiv:2407.17866. — [arXiv:2407.17866](https://arxiv.org/abs/2407.17866) [find](https://scholar.google.com/scholar?q=Alex+Kim%2C+Maximilian+Muhn%2C+and+Valeri+Nikolaev%2E+2025%2E+Financial+statement+analysis+with+large+language+models%2E+Preprint%2C+arXiv%3A2407%2E17866%2E+%E2%80%94+%5BarXiv%3A2407%2E17866%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2407%2E17866%29)
20. Aviral Kumar, Vincent Zhuang, Rishabh Agarwal, Yi Su, John D Co-Reyes, Avi Singh, Kate Baumli, Shariq Iqbal, Colton Bishop, Rebecca Roelofs, Lei M Zhang, Kay McKinney, Disha Shrivastava, Cosmin Paduraru, George Tucker, Doina Precup, Feryal Behbahani, and Aleksandra Faust. 2024. [Training language mod…](https://md.research.elamur.ai/free/en/papers/kumar-2024-training-language-models-self-correct/)
21. Dong-Ho Lee, Jay Pujara, Mohit Sewak, Ryen W. White, and Sujay Kumar Jauhar. 2023. Making large language models better data creators. Preprint, arXiv:2310.20111. [find](https://scholar.google.com/scholar?q=Dong%2DHo+Lee%2C+Jay+Pujara%2C+Mohit+Sewak%2C+Ryen+W%2E+White%2C+and+Sujay+Kumar+Jauhar%2E+2023%2E+Making+large+language+models+better+data+creators%2E+Preprint%2C+arXiv%3A2310%2E20111%2E)
22. Jinnan Li, Jinzhe Li, Yue Wang, Yi Chang, and Yuan Wu. 2025a. Structflowbench: A structured flow benchmark for multi-turn instruction following. Preprint, arXiv:2502.14494. [find](https://scholar.google.com/scholar?q=Jinnan+Li%2C+Jinzhe+Li%2C+Yue+Wang%2C+Yi+Chang%2C+and+Yuan+Wu%2E+2025a%2E+Structflowbench%3A+A+structured+flow+benchmark+for+multi%2Dturn+instruction+following%2E+Preprint%2C+arXiv%3A2502%2E14494%2E)
23. Junyi Li, Xiaoxue Cheng, Wayne Xin Zhao, Jian-Yun Nie, and Ji-Rong Wen. 2023a. [Halueval: A large-scale hallucination evaluation benchmark for large language models](https://md.research.elamur.ai/free/en/papers/li-2023-halueval-large-scale-hallucination-evaluation/). Preprint, arXiv:2305.11747.
24. Xiaomin Li, Zhou Yu, Zhiwei Zhang, Xupeng Chen, Ziji Zhang, Yingying Zhuang, Narayanan Sadagopan, and Anurag Beniwal. 2025b. When thinking fails: The pitfalls of reasoning for instruction-following in llms. Preprint, arXiv:2505.11423. 14756 12 — [arXiv:2505.11423](https://arxiv.org/abs/2505.11423) [find](https://scholar.google.com/scholar?q=Xiaomin+Li%2C+Zhou+Yu%2C+Zhiwei+Zhang%2C+Xupeng+Chen%2C+Ziji+Zhang%2C+Yingying+Zhuang%2C+Narayanan+Sadagopan%2C+and+Anurag+Beniwal%2E+2025b%2E+When+thinking+fails%3A+The+pitfalls+of+reasoning+for+instruction%2Dfollowing+in)
25. Yunxiang Li, Zihan Li, Kai Zhang, Ruilong Dan, Steve Jiang, and You Zhang. 2023b. [Chatdoctor: A medical chat model fine-tuned on a large language model meta-ai (llama) using medical domain knowledge](https://md.research.elamur.ai/free/en/papers/li-2023-chatdoctor-medical-chat-model-fine/). Preprint, arXiv:2303.14070.
26. Dancheng Liu, Amir Nassereldine, Ziming Yang, Chenhui Xu, Yuting Hu, Jiajie Li, Utkarsh Kumar, Changjae Lee, Ruiyang Qin, Yiyu Shi, and Jinjun Xiong. 2024a. Large language models have intrinsic self-correction ability. Preprint, arXiv:2406.15673. — [arXiv:2406.15673](https://arxiv.org/abs/2406.15673) [find](https://scholar.google.com/scholar?q=Dancheng+Liu%2C+Amir+Nassereldine%2C+Ziming+Yang%2C+Chenhui+Xu%2C+Yuting+Hu%2C+Jiajie+Li%2C+Utkarsh+Kumar%2C+Changjae+Lee%2C+Ruiyang+Qin%2C+Yiyu+Shi%2C+and+Jinjun+Xiong%2E+2024a%2E+Large+language+models+have+intrinsic+self%2Dc)
27. Guangliang Liu, Haitao Mao, Bochuan Cao, Zhiyu Xue, Xitong Zhang, Rongrong Wang, Jiliang Tang, and Kristen Johnson. 2024b. On the intrinsic self-correction capability of llms: Uncertainty and latent concept. Preprint, arXiv:2406.02378. — [arXiv:2406.02378](https://arxiv.org/abs/2406.02378) [find](https://scholar.google.com/scholar?q=Guangliang+Liu%2C+Haitao+Mao%2C+Bochuan+Cao%2C+Zhiyu+Xue%2C+Xitong+Zhang%2C+Rongrong+Wang%2C+Jiliang+Tang%2C+and+Kristen+Johnson%2E+2024b%2E+On+the+intrinsic+self%2Dcorrection+capability+of+llms%3A+Uncertainty+and+latent+c)
28. Shervin Minaee, Tomas Mikolov, Narjes Nikzad, Meysam Chenaghlu, Richard Socher, Xavier Amatriain, and Jianfeng Gao. 2025. [Large language models: A survey](https://md.research.elamur.ai/free/en/papers/minaee-2024-large-language-models-survey/). Preprint, arXiv:2402.06196.
29. OpenAI. 2025a. Gpt-4.1. Visited date: 2025-09-15. [find](https://scholar.google.com/scholar?q=OpenAI%2E+2025a%2E+Gpt%2D4%2E1%2E+Visited+date%3A+2025%2D09%2D15%2E)
30. OpenAI. 2025b. o3-mini. Visited date: 2025-09-15. [find](https://scholar.google.com/scholar?q=OpenAI%2E+2025b%2E+o3%2Dmini%2E+Visited+date%3A+2025%2D09%2D15%2E)
31. Long Ouyang, Jeff Wu, Xu Jiang, Diogo Almeida, Carroll L. Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, John Schulman, Jacob Hilton, Fraser Kelton, Luke Miller, Maddie Simens, Amanda Askell, Peter Welinder, Paul Christiano, Jan Leike, and Ryan Lowe. 2022. [Train…](https://md.research.elamur.ai/free/en/papers/ouyang-2022-training-language-models-follow-instructions/)
32. Keivalya Pandya and Mehfuza Holia. 2023. Automating customer service using langchain: Building custom open-source gpt chatbot for organizations. Preprint, arXiv:2310.05421. — [arXiv:2310.05421](https://arxiv.org/abs/2310.05421) [find](https://scholar.google.com/scholar?q=Keivalya+Pandya+and+Mehfuza+Holia%2E+2023%2E+Automating+customer+service+using+langchain%3A+Building+custom+open%2Dsource+gpt+chatbot+for+organizations%2E+Preprint%2C+arXiv%3A2310%2E05421%2E+%E2%80%94+%5BarXiv%3A2310%2E05421%5D%28https%3A)
33. Cheng Qian, Chi Han, Yi R. Fung, Yujia Qin, Zhiyuan Liu, and Heng Ji. 2024. [Creator: Tool creation for disentangling abstract and concrete reasoning of large language models](https://md.research.elamur.ai/free/en/papers/qian-2023-creator-tool-creation-disentangling-abstract/). Preprint, arXiv:2305.14318.
34. Yiwei Qin, Kaiqiang Song, Yebowen Hu, Wenlin Yao, Sangwoo Cho, Xiaoyang Wang, Xuansheng Wu, Fei Liu, Pengfei Liu, and Dong Yu. 2024. [Infobench: Evaluating instruction following ability in large language models](https://md.research.elamur.ai/free/en/papers/qin-2024-infobench-evaluating-instruction-following-ability/). Preprint, arXiv:2401.03601.
35. Han Qiu, Jiaxing Huang, Peng Gao, Qin Qi, Xiaoqin Zhang, Ling Shao, and Shijian Lu. 2024a. Longhalqa: Long-context hallucination evaluation for multimodal large language models. Preprint, arXiv:2410.09962. — [arXiv:2410.09962](https://arxiv.org/abs/2410.09962) [find](https://scholar.google.com/scholar?q=Han+Qiu%2C+Jiaxing+Huang%2C+Peng+Gao%2C+Qin+Qi%2C+Xiaoqin+Zhang%2C+Ling+Shao%2C+and+Shijian+Lu%2E+2024a%2E+Longhalqa%3A+Long%2Dcontext+hallucination+evaluation+for+multimodal+large+language+models%2E+Preprint%2C+arXiv%3A2410%2E0)
36. Yang, Jiaxi Yang, Jingren Zhou, and 25 others. 2025. Qwen2.5 technical report. Preprint, arXiv:2412.15115. [find](https://scholar.google.com/scholar?q=Yang%2C+Jiaxi+Yang%2C+Jingren+Zhou%2C+and+25+others%2E+2025%2E+Qwen2%2E5+technical+report%2E+Preprint%2C+arXiv%3A2412%2E15115%2E)
37. Alessandro Stolfo, Vidhisha Balachandran, Safoora Yousefi, Eric Horvitz, and Besmira Nushi. 2025. Improving instruction-following in language models through activation steering. Preprint, arXiv:2410.12877. — [arXiv:2410.12877](https://arxiv.org/abs/2410.12877) [find](https://scholar.google.com/scholar?q=Alessandro+Stolfo%2C+Vidhisha+Balachandran%2C+Safoora+Yousefi%2C+Eric+Horvitz%2C+and+Besmira+Nushi%2E+2025%2E+Improving+instruction%2Dfollowing+in+language+models+through+activation+steering%2E+Preprint%2C+arXiv%3A2410%2E1)
38. Yuchong Sun, Che Liu, Kun Zhou, Jinwen Huang, Ruihua Song, Wayne Xin Zhao, Fuzheng Zhang, Di Zhang, and Kun Gai. 2024. Parrot: Enhancing multi-turn instruction following for large language models. Preprint, arXiv:2310.07301. 5 Team, Aohan Zeng, Xin Lv, Qinkai Zheng, Zhenyu Hou, Bin Chen, Chengxing X… [find](https://scholar.google.com/scholar?q=Yuchong+Sun%2C+Che+Liu%2C+Kun+Zhou%2C+Jinwen+Huang%2C+Ruihua+Song%2C+Wayne+Xin+Zhao%2C+Fuzheng+Zhang%2C+Di+Zhang%2C+and+Kun+Gai%2E+2024%2E+Parrot%3A+Enhancing+multi%2Dturn+instruction+following+for+large+language+models%2E+Pre)
39. Kimi Team, Yifan Bai, Yiping Bao, Guanduo Chen, Jiahao Chen, Ningxin Chen, Ruijue Chen, Yanru Chen, Yuankun Chen, Yutian Chen, Zhuofu Chen, Jialei Cui, Hao Ding, Mengnan Dong, Angang Du, Chenzhuang Du, Dikang Du, Yulun Du, Yu Fan, and 150 others. 2025b. [Kimi k2: Open agentic intelligence](https://md.research.elamur.ai/free/en/papers/kimi-team-2025-kimi-k2-open-agentic/). Preprint,…
40. Tim Vieira, Ben LeBrun, Mario Giulianelli, Juan Luis Gastaldi, Brian DuSell, John Terilla, Timothy J. O’Donnell, and Ryan Cotterell. 2025. From language models over tokens to language models over characters. Preprint, arXiv:2412.03719. — [arXiv:2412.03719](https://arxiv.org/abs/2412.03719) [find](https://scholar.google.com/scholar?q=Tim+Vieira%2C+Ben+LeBrun%2C+Mario+Giulianelli%2C+Juan+Luis+Gastaldi%2C+Brian+DuSell%2C+John+Terilla%2C+Timothy+J%2E+O%E2%80%99Donnell%2C+and+Ryan+Cotterell%2E+2025%2E+From+language+models+over+tokens+to+language+models+over+char)
41. Boshi Wang, Sewon Min, Xiang Deng, Jiaming Shen, You Wu, Luke Zettlemoyer, and Huan Sun. 2023. [Towards understanding chain-of-thought prompting: An empirical study of what matters](https://md.research.elamur.ai/free/en/papers/wang-2023-understanding-chain-thought-prompting-empirical/). Preprint, arXiv:2212.10001.
42. Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed Chi, Quoc Le, and Denny Zhou. 2022. [Chain-of-thought prompting elicits reasoning in large language models](https://md.research.elamur.ai/free/en/papers/wei-2022-chain-thought-prompting-elicits-reasoning/). Preprint, arXiv:2201.11903.
43. Bosi Wen, Pei Ke, Xiaotao Gu, Lindong Wu, Hao Huang, Jinfeng Zhou, Wenchuang Li, Binxin Hu, Wendy Gao, Jiaxin Xu, Yiming Liu, Jie Tang, Hongning Wang, and Minlie Huang. 2024. Benchmarking complex instruction-following with multiple constraints composition. Preprint, arXiv:2407.03978. — [arXiv:2407.03978](https://arxiv.org/abs/2407.03978) [find](https://scholar.google.com/scholar?q=Bosi+Wen%2C+Pei+Ke%2C+Xiaotao+Gu%2C+Lindong+Wu%2C+Hao+Huang%2C+Jinfeng+Zhou%2C+Wenchuang+Li%2C+Binxin+Hu%2C+Wendy+Gao%2C+Jiaxin+Xu%2C+Yiming+Liu%2C+Jie+Tang%2C+Hongning+Wang%2C+and+Minlie+Huang%2E+2024%2E+Benchmarking+complex+inst)
44. J. Qiu, K. Lam, G. Li, and 1 others. 2024b. Llm-based agentic systems in medicine and healthcare. Nature Machine Intelligence, 6(12):1418–1420. [find](https://scholar.google.com/scholar?q=J%2E+Qiu%2C+K%2E+Lam%2C+G%2E+Li%2C+and+1+others%2E+2024b%2E+Llm%2Dbased+agentic+systems+in+medicine+and+healthcare%2E+Nature+Machine+Intelligence%2C+6%2812%29%3A1418%E2%80%931420%2E)
45. Jochen Wulf and Juerg Meierhofer. 2024. Exploring the potential of large language models for automation in technical customer service. Preprint, arXiv:2405.09161. — [arXiv:2405.09161](https://arxiv.org/abs/2405.09161) [find](https://scholar.google.com/scholar?q=Jochen+Wulf+and+Juerg+Meierhofer%2E+2024%2E+Exploring+the+potential+of+large+language+models+for+automation+in+technical+customer+service%2E+Preprint%2C+arXiv%3A2405%2E09161%2E+%E2%80%94+%5BarXiv%3A2405%2E09161%5D%28https%3A%2F%2Farxiv%2Eor)
46. Qwen, :, An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, Huan Lin, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Bei Yan, Jie Zhang, Zheng Yuan, Shiguang Shan, and Xilin Chen. 2024. Evaluating the quality of hallucination ben… — [arXiv:2406.17115](https://arxiv.org/abs/2406.17115) [find](https://scholar.google.com/scholar?q=Qwen%2C+%3A%2C+An+Yang%2C+Baosong+Yang%2C+Beichen+Zhang%2C+Binyuan+Hui%2C+Bo+Zheng%2C+Bowen+Yu%2C+Chengyuan+Li%2C+Dayiheng+Liu%2C+Fei+Huang%2C+Haoran+Wei%2C+Huan+Lin%2C+Jian+Yang%2C+Jianhong+Tu%2C+Jianwei+Zhang%2C+Jianxin+Bei+Yan%2C+Jie)
47. An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, Chujie Zheng, Dayiheng Liu, Fan Zhou, Fei Huang, Feng Hu, Hao Ge, Haoran Wei, Huan Lin, Jialong Tang, and 41 others. 2025. [Qwen3 technical report](https://md.research.elamur.ai/free/en/papers/yang-2025-qwen3-technical-report/). Preprint, arXiv:2505.09388.
48. Shunyu Yao, Howard Chen, Austin W. Hanjie, Runzhe Yang, and Karthik Narasimhan. 2023. [Collie: Systematic construction of constrained text generation tasks](https://md.research.elamur.ai/free/en/papers/yao-2023-collie-systematic-construction-constrained-text/). Preprint, arXiv:2307.08689.
49. Zihan Yu, Liang He, Zhen Wu, Xinyu Dai, and Jiajun Chen. 2023. Towards better chain-of-thought prompting strategies: A survey. Preprint, arXiv:2310.04959. — [arXiv:2310.04959](https://arxiv.org/abs/2310.04959) [find](https://scholar.google.com/scholar?q=Zihan+Yu%2C+Liang+He%2C+Zhen+Wu%2C+Xinyu+Dai%2C+and+Jiajun+Chen%2E+2023%2E+Towards+better+chain%2Dof%2Dthought+prompting+strategies%3A+A+survey%2E+Preprint%2C+arXiv%3A2310%2E04959%2E+%E2%80%94+%5BarXiv%3A2310%2E04959%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F23)
50. Zhiyuan Zeng, Jiatong Yu, Tianyu Gao, Yu Meng, Tanya Goyal, and Danqi Chen. 2024. Evaluating large language models at evaluating instruction following. Preprint, arXiv:2310.07641. [find](https://scholar.google.com/scholar?q=Zhiyuan+Zeng%2C+Jiatong+Yu%2C+Tianyu+Gao%2C+Yu+Meng%2C+Tanya+Goyal%2C+and+Danqi+Chen%2E+2024%2E+Evaluating+large+language+models+at+evaluating+instruction+following%2E+Preprint%2C+arXiv%3A2310%2E07641%2E)
51. Shengyu Zhang, Linfeng Dong, Xiaoya Li, Sen Zhang, Xiaofei Sun, Shuhe Wang, Jiwei Li, Runyi Hu, Tianwei Zhang, Fei Wu, and Guoyin Wang. 2024. [Instruction tuning for large language models: A survey](https://md.research.elamur.ai/free/en/papers/zhang-2023-instruction-tuning-large-language-models/). Preprint, arXiv:2308.10792.
52. Zhuosheng Zhang, Aston Zhang, Mu Li, and Alex Smola. 2022. [Automatic chain of thought prompting in large language models](https://md.research.elamur.ai/free/en/papers/zhang-2022-automatic-chain-thought-prompting-large/). Preprint, arXiv:2210.03493.
53. Chenzhuo Zhao, Xinda Wang, Yue Huang, Junting Lu, and Ziqian Liu. 2025a. Tase: Token awareness and structured evaluation for multilingual language models. Preprint, arXiv:2508.05468. — [arXiv:2508.05468](https://arxiv.org/abs/2508.05468) [find](https://scholar.google.com/scholar?q=Chenzhuo+Zhao%2C+Xinda+Wang%2C+Yue+Huang%2C+Junting+Lu%2C+and+Ziqian+Liu%2E+2025a%2E+Tase%3A+Token+awareness+and+structured+evaluation+for+multilingual+language+models%2E+Preprint%2C+arXiv%3A2508%2E05468%2E+%E2%80%94+%5BarXiv%3A2508%2E054)
54. Wayne Xin Zhao, Kun Zhou, Junyi Li, Tianyi Tang, Xiaolei Wang, Yupeng Hou, Yingqian Min, Beichen Zhang, Junjie Zhang, Zican Dong, Yifan Du, Chen Yang, Yushuo Chen, Zhipeng Chen, Jinhao Jiang, Ruiyang Ren, Yifan Li, Xinyu Tang, Zikang Liu, and 3 others. 2025b. [A survey of large language models](https://md.research.elamur.ai/free/en/papers/zhao-2023-survey-large-language-models/). Prepr…
55. Jeffrey Zhou, Tianjian Lu, Swaroop Mishra, Siddhartha Brahma, Sujoy Basu, Yi Luan, Denny Zhou, and Le Hou. 2023. [Instruction-following evaluation for large language models](https://md.research.elamur.ai/free/en/papers/zhou-2023-instruction-following-evaluation-large-language/). Preprint, arXiv:2311.07911.
56. Zhiying Zhu, Yiming Yang, and Zhiqing Sun. 2024. Halueval-wild: Evaluating hallucinations of language models in the wild. Preprint, arXiv:2403.04307. — [arXiv:2403.04307](https://arxiv.org/abs/2403.04307) [find](https://scholar.google.com/scholar?q=Zhiying+Zhu%2C+Yiming+Yang%2C+and+Zhiqing+Sun%2E+2024%2E+Halueval%2Dwild%3A+Evaluating+hallucinations+of+language+models+in+the+wild%2E+Preprint%2C+arXiv%3A2403%2E04307%2E+%E2%80%94+%5BarXiv%3A2403%2E04307%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2403%2E04)

*Updated 1 November 2026*