---
title: "RoadMapper — paper by Liu et al. (2026) — summary and ideas"
description: "The paper introduces roadmap generation as a new task. 38 references. Among them: 0."
authors: ["Jiacheng Liu", "Zichen Tang", "Zhongjun Yang", "Xinyi Hu", "Xueyuan Lin", "Linwei Jia", "Ruofei Bai", "Rongjin Li", "Shiyao Peng", "Haocheng Gao", "Haihong E"]
year: 2026
venue: "Findings of the Association for Computational Linguistics: ACL 2026"
source: "https://aclanthology.org/2026.findings-acl.378/"
doi: "10.18653/v1/2026.findings-acl.378"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/liu-2026-roadmapper-multi-agent-system-roadmap/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/liu-2026-roadmapper-multi-agent-system-roadmap/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/41/) / RoadMapper: A Multi-Agent System for Roadmap Generation of Solving Complex Research Problems

# RoadMapper: A Multi-Agent System for Roadmap Generation of Solving Complex Research Problems

**Authors:** [Jiacheng Liu](https://md.research.elamur.ai/free/en/authors/jiacheng-liu/), [Zichen Tang](https://md.research.elamur.ai/free/en/authors/zichen-tang/), [Zhongjun Yang](https://md.research.elamur.ai/free/en/authors/zhongjun-yang/), Xinyi Hu, [Xueyuan Lin](https://md.research.elamur.ai/free/en/authors/xueyuan-lin/), Linwei Jia, Ruofei Bai, [Rongjin Li](https://md.research.elamur.ai/free/en/authors/rongjin-li/), Shiyao Peng, Haocheng Gao and [Haihong E](https://md.research.elamur.ai/free/en/authors/haihong-e/) · **Year:** 2026 · **Venue:** [Findings of the Association for Computational Linguistics: ACL 2026](https://md.research.elamur.ai/free/en/venues/findings/) ([Findings of the Association for Computational Linguistics 2026](https://md.research.elamur.ai/free/en/venues/findings/2026/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2026.findings-acl.378/) · [PDF](https://aclanthology.org/2026.findings-acl.378.pdf) · [DOI 10.18653/v1/2026.findings-acl.378](https://doi.org/10.18653/v1/2026.findings-acl.378) · [OpenAlex](https://openalex.org/W7166881806)

**Citations:** 0 citations (OpenAlex, 2026-10-14)

## Summary
The paper introduces roadmap generation as a new task. Given a complex research problem, an LLM must produce a hierarchical, tree-shaped plan of subtasks that guides a researcher step by step. The plan is encoded as Markdown headings with numbered indices.

To study this task, the authors built the RoadMap benchmark from graduate dissertations published since 2018 and collected from ProQuest and CNKI. Gemini 2.5 Flash extracted the core problems and skill points, and domain experts then crafted reference roadmaps. Several verification passes checked relevance, formatting and duplication.

Testing LLMs on the benchmark revealed three weaknesses: missing domain knowledge, poor task decomposition, and incoherent ordering of steps. To address them, the authors propose RoadMapper, a six-agent pipeline. One agent drafts a roadmap and another enriches it with retrieved skill points. Two critics review granularity and logic separately, a reviser applies their suggestions, and a DPO-tuned evaluator decides when to stop iterating.

On four newly proposed metrics, RoadMapper raises average performance by more than 8%. It also cuts design time by about 84% compared with human experts, which points to practical value for research planning and education.

## Key points
- **RoadMap benchmark:**
  - Golden-Roadmap contains 1,705 expert roadmaps (500 EN / 1,205 CN).
  - Skill-Repo contains 8,436 skill points.
  - Coverage spans 10 research fields and 5 research types.
  - An average roadmap has 21.60 key steps and a depth of 3.22.
- **Agents:** Init, Knowledge, Granularity Critique, Logic Critique, Revise and Evaluate. The Evaluate agent is trained with DPO, using flawed expert drafts kept from benchmark construction as preference data.
- **Metrics and gains:** The metrics are DegreeScore, DepthScore, StepScore and LogicScore. RoadMapper improves StepScore by 8.24, the structure metrics by 7.04 and LogicScore by 7.79.
- **Efficiency:**
  - Early stopping averages 1.64 iterations with Llama 3.3 70B and 1.51 with GPT-4o mini.
  - Compared with ReConcile, RoadMapper uses 36.5% less time and 45.8% fewer tokens.
- **Human evaluation:**
  - Seven experts compared roadmaps pairwise across five dimensions and preferred RoadMapper in 86% of cases.
  - GPT-4o mini as a judge matched the experts' verdicts 93% of the time.
  - The data is released under CC BY 4.0 and the code under Apache 2.0.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎯 Decision Sciences](https://md.research.elamur.ai/free/en/shelves/decision-sciences/) · 209 works · [🎯 Management Science and Operations Research](https://md.research.elamur.ai/free/en/shelves/decision-sciences/management-science-and-operations-research/) · 108 works

## Cite

Cite as: Jiacheng Liu, Zichen Tang, Zhongjun Yang, Xinyi Hu, Xueyuan Lin, Linwei Jia, Ruofei Bai, Rongjin Li, Shiyao Peng, Haocheng Gao and Haihong E. 2026. RoadMapper: A Multi-Agent System for Roadmap Generation of Solving Complex Research Problems. Findings of the Association for Computational Linguistics: ACL 2026.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{liu2026roadmapper,
  title = {{RoadMapper: A Multi-Agent System for Roadmap Generation of Solving Complex Research Problems}},
  author = {Jiacheng Liu, Zichen Tang, Zhongjun Yang, Xinyi Hu, Xueyuan Lin, Linwei Jia, Ruofei Bai, Rongjin Li, Shiyao Peng, Haocheng Gao and Haihong E},
  booktitle = {Findings of the Association for Computational Linguistics: ACL 2026},
  year = {2026},
  doi = {10.18653/v1/2026.findings-acl.378},
  url = {https://doi.org/10.18653/v1/2026.findings-acl.378}
}
```

- [← previous RLShield: Dynamic Jailbreak Detection for LLMs via Reinforced Adaptive Learning](https://md.research.elamur.ai/free/en/papers/tong-2026-rlshield-dynamic-jailbreak-detection-llms/)
- [next → RoBSA: RoPE-based Blockwise Sparse Multi-head Latent Attention](https://md.research.elamur.ai/free/en/papers/shi-2026-robsa-rope-based-blockwise-sparse/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/41/)

## References

38 references, 16 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Alfonso Amayuelas, Xianjun Yang, Antonis Antoniades, Wenyue Hua, Liangming Pan, and William Yang Wang. 2024. MultiAgent collaboration attack: Investigating adversarial attacks in large language model collaborations via debate. In Findings of the Association for Computational Linguistics: EMNLP 2024,… [find](https://scholar.google.com/scholar?q=Alfonso+Amayuelas%2C+Xianjun+Yang%2C+Antonis+Antoniades%2C+Wenyue+Hua%2C+Liangming+Pan%2C+and+William+Yang+Wang%2E+2024%2E+MultiAgent+collaboration+attack%3A+Investigating+adversarial+attacks+in+large+language+model+)
2. Anthropic. 2024. Claude 3 Family. [find](https://scholar.google.com/scholar?q=Anthropic%2E+2024%2E+Claude+3+Family%2E)
3. Jacob Austin, Augustus Odena, Maxwell Nye, Maarten Bosma, Henryk Michalewski, David Dohan, Ellen Jiang, Carrie Cai, Michael Terry, Quoc Le, and Charles Sutton. 2021. [Program synthesis with large language models](https://md.research.elamur.ai/free/en/papers/austin-2021-program-synthesis-large-language-models/). Preprint, arXiv:2108.07732.
4. Dhaivat J. Bhatt, Seyed Ahmad Abdollahpouri Hosseini, Federico Fancellu, and Afsaneh Fazly. 2024. End-to-end parsing of procedural text into flow graphs. In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024), page… [find](https://scholar.google.com/scholar?q=Dhaivat+J%2E+Bhatt%2C+Seyed+Ahmad+Abdollahpouri+Hosseini%2C+Federico+Fancellu%2C+and+Afsaneh+Fazly%2E+2024%2E+End%2Dto%2Dend+parsing+of+procedural+text+into+flow+graphs%2E+In+Proceedings+of+the+2024+Joint+International)
5. Antoine Bosselut, Hannah Rashkin, Maarten Sap, Chaitanya Malaviya, Asli Celikyilmaz, and Yejin Choi. 2019. [COMET: Commonsense transformers for automatic knowledge graph construction](https://md.research.elamur.ai/free/en/papers/bosselut-2019-comet-commonsense-transformers-automatic-knowledge/). In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 4762–4779, Florence…
6. Philip E Burian, Lynda Rogerson, and Francis R Maffei III. 2010. The research roadmap: A primer to the approach and process. Contemporary Issues in Education Research, 3(8):43–58. \#\# Acknowledgments Justin Chen, Swarnadeep Saha, and Mohit Bansal. 2024. ReConcile: Round-table conference improves reas… [find](https://scholar.google.com/scholar?q=Philip+E+Burian%2C+Lynda+Rogerson%2C+and+Francis+R+Maffei+III%2E+2010%2E+The+research+roadmap%3A+A+primer+to+the+approach+and+process%2E+Contemporary+Issues+in+Education+Research%2C+3%288%29%3A43%E2%80%9358%2E+%5C%23%5C%23+Acknowledgments)
7. This work is supported by the National Natural Science Foundation of China (Grant Nos. 62473271, 62176026), the Fundamental Research Funds for the Beijing University of Posts and Telecommunications (Grant No. 2025[AI4S03), and the BUPT Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo…](https://md.research.elamur.ai/free/en/papers/cobbe-2021-training-verifiers-solve-math-word/)
8. DeepSeek-AI, Aixin Liu, Aoxue Mei, Bangcai Lin, Bing Xue, Bingxuan Wang, Bingzheng Xu, Bochao Wu, Bowei Zhang, Chaofan Lin, Chen Dong, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenhao Xu, Chong Ruan, Damai Dai, Daya Guo, Dejian Yang, and 245 others. 2025. [Deepseek-v3.2: Pushing the frontier of open…](https://md.research.elamur.ai/free/en/papers/deepseek-ai-2025-deepseek-v3-2-pushing/)
9. Zheye Deng, Chunkit Chan, Weiqi Wang, Yuxi Sun, Wei Fan, Tianshi Zheng, Yauwai Yim, and Yangqiu Song. 2024. Text-tuple-table: Towards information integration in text-to-table generation via global tuple extraction. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Proces… [find](https://scholar.google.com/scholar?q=Zheye+Deng%2C+Chunkit+Chan%2C+Weiqi+Wang%2C+Yuxi+Sun%2C+Wei+Fan%2C+Tianshi+Zheng%2C+Yauwai+Yim%2C+and+Yangqiu+Song%2E+2024%2E+Text%2Dtuple%2Dtable%3A+Towards+information+integration+in+text%2Dto%2Dtable+generation+via+global+tup)
10. Björn Engelmann, Christin Katharina Kreutz, Fabian Haak, and Philipp Schaer. 2024. ARTS: Assessing readability & text simplicity. In Findings of the Association for Computational Linguistics: EMNLP 2024, pages 14925–14942, Miami, Florida, USA. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Bj%C3%B6rn+Engelmann%2C+Christin+Katharina+Kreutz%2C+Fabian+Haak%2C+and+Philipp+Schaer%2E+2024%2E+ARTS%3A+Assessing+readability+%26+text+simplicity%2E+In+Findings+of+the+Association+for+Computational+Linguistics%3A+EMNLP+20)
11. Gemini Team. 2025. Gemini 3.0: A new era of intelligence. Technical report, Google DeepMind. [find](https://scholar.google.com/scholar?q=Gemini+Team%2E+2025%2E+Gemini+3%2E0%3A+A+new+era+of+intelligence%2E+Technical+report%2C+Google+DeepMind%2E)
12. Taicheng Guo, Xiuying Chen, Yaqi Wang, Ruidi Chang, Shichao Pei, Nitesh V. Chawla, Olaf Wiest, and Xiangliang Zhang. 2024. [Large language model based multi-agents: A survey of progress and challenges](https://md.research.elamur.ai/free/en/papers/guo-2024-large-language-model-based/). In Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence, IJCAI…
13. Fabian Haak and Philipp Schaer. 2025. Pairwise comparison for bias identification and quantification. Preprint, arXiv:2512.14565. — [arXiv:2512.14565](https://arxiv.org/abs/2512.14565) [find](https://scholar.google.com/scholar?q=Fabian+Haak+and+Philipp+Schaer%2E+2025%2E+Pairwise+comparison+for+bias+identification+and+quantification%2E+Preprint%2C+arXiv%3A2512%2E14565%2E+%E2%80%94+%5BarXiv%3A2512%2E14565%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2512%2E14565%29)
14. Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. 2021. [Measuring mathematical problem solving with the math dataset](https://md.research.elamur.ai/free/en/papers/hendrycks-2021-measuring-mathematical-problem-solving-math/). In Proceedings of the Neural Information Processing Systems Track on Datasets and Benchmarks, volume 1.
15. Mengting Hu, Honglei Guo, Shiwan Zhao, Hang Gao, and Zhong Su. 2021. Efficient mind-map generation via sequence-to-graph and reinforced graph refinement. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 8130–8141, Online and Punta Cana, Dominican Repub… [find](https://scholar.google.com/scholar?q=Mengting+Hu%2C+Honglei+Guo%2C+Shiwan+Zhao%2C+Hang+Gao%2C+and+Zhong+Su%2E+2021%2E+Efficient+mind%2Dmap+generation+via+sequence%2Dto%2Dgraph+and+reinforced+graph+refinement%2E+In+Proceedings+of+the+2021+Conference+on+Empir)
16. Lei Huang, Weijiang Yu, Weitao Ma, Weihong Zhong, Zhangyin Feng, Haotian Wang, Qianglong Chen, Weihua Peng, Xiaocheng Feng, Bing Qin, and Ting Liu. 2025. [A survey on hallucination in large language models: Principles, taxonomy, challenges, and open questions](https://md.research.elamur.ai/free/en/papers/huang-2023-survey-hallucination-large-language-models/). ACM Trans. Inf. Syst., 43(2).
17. Parag Jain, Andreea Marzoca, and Francesco Piccinno. 2024. STRUCTSUM generation for faster text comprehension. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 7876–7896, Bangkok, Thailand. Association for Computational Linguis… [find](https://scholar.google.com/scholar?q=Parag+Jain%2C+Andreea+Marzoca%2C+and+Francesco+Piccinno%2E+2024%2E+STRUCTSUM+generation+for+faster+text+comprehension%2E+In+Proceedings+of+the+62nd+Annual+Meeting+of+the+Association+for+Computational+Linguistic)
18. Bingxuan Li, Yiwei Wang, Jiuxiang Gu, Kai-Wei Chang, and Nanyun Peng. 2025a. [METAL: A multi-agent framework for chart generation with test-time scaling](https://md.research.elamur.ai/free/en/papers/li-2025-metal-multi-agent-framework-chart/). In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 30054–30069, Vienna, Aust…
19. Tong Li, Zhihao Wang, Liangying Shao, Xuling Zheng, Xiaoli Wang, and Jinsong Su. 2023. A sequence-to-sequence&set model for text-to-table generation. In Findings of the Association for Computational Linguistics: ACL 2023, pages 5358–5370, Toronto, Canada. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Tong+Li%2C+Zhihao+Wang%2C+Liangying+Shao%2C+Xuling+Zheng%2C+Xiaoli+Wang%2C+and+Jinsong+Su%2E+2023%2E+A+sequence%2Dto%2Dsequence%26set+model+for+text%2Dto%2Dtable+generation%2E+In+Findings+of+the+Association+for+Computational+L)
20. Zhuoqun Li, Haiyang Yu, Xuanang Chen, Hongyu Lin, Yaojie Lu, Fei Huang, Xianpei Han, Yongbin Li, and Le Sun. 2025b. [DeepSolution: Boosting complex engineering solution design via tree-based exploration and bi-point thinking](https://md.research.elamur.ai/free/en/papers/li-2025-deepsolution-boosting-complex-engineering-solution/). In Proceedings of the 63rd Annual Meeting of the Association for Computatio…
21. Zijun Liu, Yanzhe Zhang, Peng Li, Yang Liu, and Diyi Yang. 2024. A dynamic llm-powered agent network for task-oriented agent collaboration. Preprint, arXiv:2310.02170. [find](https://scholar.google.com/scholar?q=Zijun+Liu%2C+Yanzhe+Zhang%2C+Peng+Li%2C+Yang+Liu%2C+and+Diyi+Yang%2E+2024%2E+A+dynamic+llm%2Dpowered+agent+network+for+task%2Doriented+agent+collaboration%2E+Preprint%2C+arXiv%3A2310%2E02170%2E)
22. Meng Lu, Brandon Ho, Dennis Ren, and Xuan Wang. 2024. TriageAgent: Towards better multi-agents collaborations for large language model-based clinical triage. In Findings of the Association for Computational Linguistics: EMNLP 2024, pages 5747–5764, Miami, Florida, USA. Association for Computational… [find](https://scholar.google.com/scholar?q=Meng+Lu%2C+Brandon+Ho%2C+Dennis+Ren%2C+and+Xuan+Wang%2E+2024%2E+TriageAgent%3A+Towards+better+multi%2Dagents+collaborations+for+large+language+model%2Dbased+clinical+triage%2E+In+Findings+of+the+Association+for+Computa)
23. Yaojie Lu, Qing Liu, Dai Dai, Xinyan Xiao, Hongyu Lin, Xianpei Han, Le Sun, and Hua Wu. 2022. Unified structure generation for universal information extraction. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 5755–5772, Dublin… [find](https://scholar.google.com/scholar?q=Yaojie+Lu%2C+Qing+Liu%2C+Dai+Dai%2C+Xinyan+Xiao%2C+Hongyu+Lin%2C+Xianpei+Han%2C+Le+Sun%2C+and+Hua+Wu%2E+2022%2E+Unified+structure+generation+for+universal+information+extraction%2E+In+Proceedings+of+the+60th+Annual+Meeti)
24. MISTRALAI. 2025. Mistral Small 3.2: 24B Multimodal LLM with Enhanced Instruction Following. [find](https://scholar.google.com/scholar?q=MISTRALAI%2E+2025%2E+Mistral+Small+3%2E2%3A+24B+Multimodal+LLM+with+Enhanced+Instruction+Following%2E)
25. OpenAI, :, Sandhini Agarwal, Lama Ahmad, Jason Ai, Sam Altman, Andy Applebaum, Edwin Arbus, Rahul K. Arora, Yu Bai, Bowen Baker, Haiming Bao, Boaz Barak, Ally Bennett, Tyler Bertao, Nivedita Brett, Eugene Brevdo, Greg Brockman, Sebastien Bubeck, and 108 others. 2025. [gpt-oss-120b gpt-oss-20b model c…](https://md.research.elamur.ai/free/en/papers/agarwal-2025-gpt-oss-120b-gpt-oss/)
26. OpenAI, :, Aaron Hurst, Adam Lerer, Adam P. Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Ostrow, Akila Welihinda, Alan Hayes, Alec Radford, Aleksander M adry, Alex Baker-Whitcomb, Alex Beutel, Alex Borzunov, Alex Carney, Alex Chow, Alex Kirillov, and 401 others. 2024. [Gpt-4o system card](https://md.research.elamur.ai/free/en/papers/openai-2024-gpt-4o-system-card/).…
27. Chanjun Park and Hyeonwoo Kim. 2025. Understanding LLM development through longitudinal study: Insights from the open Ko-LLM leaderboard. In Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume… [find](https://scholar.google.com/scholar?q=Chanjun+Park+and+Hyeonwoo+Kim%2E+2025%2E+Understanding+LLM+development+through+longitudinal+study%3A+Insights+from+the+open+Ko%2DLLM+leaderboard%2E+In+Proceedings+of+the+2025+Conference+of+the+Nations+of+the+Am)
28. Haopeng Ren, Yushi Zeng, Yi Cai, Bihan Zhou, and Zetao Lian. 2023. Constructing procedural graphs with multiple dependency relations: A new dataset and baseline. In Findings of the Association for Computational Linguistics: ACL 2023, pages 8474–8486, Toronto, Canada. Association for Computational Li… [find](https://scholar.google.com/scholar?q=Haopeng+Ren%2C+Yushi+Zeng%2C+Yi+Cai%2C+Bihan+Zhou%2C+and+Zetao+Lian%2E+2023%2E+Constructing+procedural+graphs+with+multiple+dependency+relations%3A+A+new+dataset+and+baseline%2E+In+Findings+of+the+Association+for+Com)
29. Taylor Sorensen, Jared Moore, Jillian Fisher, Mitchell L Gordon, Niloofar Mireshghallah, Christopher Michael Rytting, Andre Ye, Liwei Jiang, Ximing Lu, Nouha Dziri, Tim Althoff, and Yejin Choi. 2024. Position: [A roadmap to pluralistic alignment](https://md.research.elamur.ai/free/en/papers/sorensen-2024-roadmap-pluralistic-alignment/). In Proceedings of the 41st International Conference on…
30. Haochen Tan, Zhijiang Guo, Zhan Shi, Lu Xu, Zhili Liu, Yunlong Feng, Xiaoguang Li, Yasheng Wang, Lifeng Shang, Qun Liu, and Linqi Song. 2024. [ProxyQA: An alternative framework for evaluating long-form text generation with large language models](https://md.research.elamur.ai/free/en/papers/tan-2024-proxyqa-alternative-framework-evaluating-long/). In Proceedings of the 62nd Annual Meeting of the Associ…
31. Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, brian ichter, Fei Xia, Ed Chi, Quoc V Le, and Denny Zhou. 2022. [Chain-of-thought prompting elicits reasoning in large language models](https://md.research.elamur.ai/free/en/papers/wei-2022-chain-thought-prompting-elicits-reasoning/). In Advances in Neural Information Processing Systems, volume 35, pages 24824–24837. Curran Associates, Inc.
32. Colin White, Samuel Dooley, Manley Roberts, Arka Pal, Benjamin Feuer, Siddhartha Jain, Ravid Shwartz-Ziv, Neel Jain, Khalid Saifullah, Sreemanti Dey, Shubh-Agrawal, Sandeep Sandha, Siddartha Naidu, Chinmay Hegde, Yann LeCun, Tom Goldstein, Willie Neiswanger, and Micah Goldblum. 2025. Livebench: A ch… [find](https://scholar.google.com/scholar?q=Colin+White%2C+Samuel+Dooley%2C+Manley+Roberts%2C+Arka+Pal%2C+Benjamin+Feuer%2C+Siddhartha+Jain%2C+Ravid+Shwartz%2DZiv%2C+Neel+Jain%2C+Khalid+Saifullah%2C+Sreemanti+Dey%2C+Shubh%2DAgrawal%2C+Sandeep+Sandha%2C+Siddartha+Naidu%2C+Ch)
33. Derong Xu, Wei Chen, Wenjun Peng, Chao Zhang, Tong Xu, Xiangyu Zhao, Xian Wu, Yefeng Zheng, Yang Wang, and Enhong Chen. 2024a. [Large language models for generative information extraction: a survey](https://md.research.elamur.ai/free/en/papers/xu-2023-large-language-models-generative-information/). Front. Comput. Sci., 18(6).
34. Zelai Xu, Chao Yu, Fei Fang, Yu Wang, and Yi Wu. 2024b. Language agents with reinforcement learning for strategic play in the werewolf game. In Proceedings of the 41st International Conference on Machine Learning, volume 235 of Proceedings of Machine Learning Research, pages 55434–55464. PMLR. [find](https://scholar.google.com/scholar?q=Zelai+Xu%2C+Chao+Yu%2C+Fei+Fang%2C+Yu+Wang%2C+and+Yi+Wu%2E+2024b%2E+Language+agents+with+reinforcement+learning+for+strategic+play+in+the+werewolf+game%2E+In+Proceedings+of+the+41st+International+Conference+on+Mach)
35. Zikang Zhang, Wangjie You, Tianci Wu, Xinrui Wang, Juntao Li, and Min Zhang. 2025. A survey of generative information extraction. In Proceedings of the 31st International Conference on Computational Linguistics, pages 4840–4870, Abu Dhabi, UAE. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Zikang+Zhang%2C+Wangjie+You%2C+Tianci+Wu%2C+Xinrui+Wang%2C+Juntao+Li%2C+and+Min+Zhang%2E+2025%2E+A+survey+of+generative+information+extraction%2E+In+Proceedings+of+the+31st+International+Conference+on+Computational+L)
36. Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric P. Xing, Hao Zhang, Joseph E. Gonzalez, and Ion Stoica. 2023. [Judging llm-as-a-judge with mt-bench and chatbot arena](https://md.research.elamur.ai/free/en/papers/zheng-2023-judging-llm-judge-mt-bench/). Preprint, arXiv:2306.05685.
37. Bin Wang, Chao Xu, Xiaomeng Zhao, Linke Ouyang, Fan Wu, Zhiyuan Zhao, Rui Xu, Kaiwen Liu, Yuan Qu, Fukai Shang, Bo Zhang, Liqun Wei, Zhihao Sui, Wei Li, Botian Shi, Yu Qiao, Dahua Lin, and Conghui He. 2024a. Mineru: An open-source solution for precise document content extraction. Preprint, arXiv:240… [find](https://scholar.google.com/scholar?q=Bin+Wang%2C+Chao+Xu%2C+Xiaomeng+Zhao%2C+Linke+Ouyang%2C+Fan+Wu%2C+Zhiyuan+Zhao%2C+Rui+Xu%2C+Kaiwen+Liu%2C+Yuan+Qu%2C+Fukai+Shang%2C+Bo+Zhang%2C+Liqun+Wei%2C+Zhihao+Sui%2C+Wei+Li%2C+Botian+Shi%2C+Yu+Qiao%2C+Dahua+Lin%2C+and+Conghui+He%2E)
38. Minzheng Wang, Longze Chen, Fu Cheng, Shengyi Liao, Xinghua Zhang, Bingli Wu, Haiyang Yu, Nan Xu, Lei Zhang, Run Luo, Yunshui Li, Min Yang, Fei Huang, and Yongbin Li. 2024b. Leave no document [find](https://scholar.google.com/scholar?q=Minzheng+Wang%2C+Longze+Chen%2C+Fu+Cheng%2C+Shengyi+Liao%2C+Xinghua+Zhang%2C+Bingli+Wu%2C+Haiyang+Yu%2C+Nan+Xu%2C+Lei+Zhang%2C+Run+Luo%2C+Yunshui+Li%2C+Min+Yang%2C+Fei+Huang%2C+and+Yongbin+Li%2E+2024b%2E+Leave+no+document)

*Updated 1 November 2026*