---
title: "Can LLMs Identify Critical Limitations within Scientific Research?"
description: "The authors first build a taxonomy of limitation types for AI research, spanning four areas: the methods… 39 references, cited by 7 papers here."
authors: ["Zhijian Xu", "Yilun Zhao", "Manasi Patwardhan", "Lovekesh Vig", "Arman Cohan"]
year: 2025
venue: "Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)"
source: "https://aclanthology.org/2025.acl-long.1009/"
doi: "10.18653/v1/2025.acl-long.1009"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/xu-2025-can-llms-identify-critical-limitations/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/xu-2025-can-llms-identify-critical-limitations/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2025](https://md.research.elamur.ai/free/en/papers/2025/page/7/) / Can LLMs Identify Critical Limitations within Scientific Research? A Systematic Evaluation on AI Research Papers

# Can LLMs Identify Critical Limitations within Scientific Research? A Systematic Evaluation on AI Research Papers

**Authors:** [Zhijian Xu](https://md.research.elamur.ai/free/en/authors/zhijian-xu/), [Yilun Zhao](https://md.research.elamur.ai/free/en/authors/yilun-zhao/), [Manasi Patwardhan](https://md.research.elamur.ai/free/en/authors/manasi-patwardhan/), [Lovekesh Vig](https://md.research.elamur.ai/free/en/authors/lovekesh-vig/) and [Arman Cohan](https://md.research.elamur.ai/free/en/authors/arman-cohan/) · **Year:** 2025 · **Venue:** [Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)](https://md.research.elamur.ai/free/en/venues/acl/) ([ACL 2025](https://md.research.elamur.ai/free/en/venues/acl/2025/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2025.acl-long.1009/) · [PDF](https://aclanthology.org/2025.acl-long.1009.pdf) · [DOI 10.18653/v1/2025.acl-long.1009](https://doi.org/10.18653/v1/2025.acl-long.1009) · [OpenAlex](https://openalex.org/W4412944959)

**Citations:** 4 citations (OpenAlex, 2026-10-14) · [7](https://md.research.elamur.ai/free/en/papers/xu-2025-can-llms-identify-critical-limitations/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
Peer review is increasingly strained as publication volume grows. Existing benchmarks for LLM-generated reviews mostly judge overall review quality and pay little attention to whether models can find a paper's real weaknesses. This work isolates that skill. The authors first build a taxonomy of limitation types for AI research, spanning four areas: the methods used, how experiments are designed, how results are analyzed, and how prior literature is reviewed. They then release LimitGen, a benchmark with two parts:

- **LimitGen-Syn** takes strong papers and deliberately edits them to plant specific flaws, such as removed baselines, weak metrics or missing experimental details.
- **LimitGen-Human** pairs papers with limitations that reviewers actually raised on ICLR 2025 submissions.

Given a paper and a target aspect, a model must describe the paper's limitations. The authors also add a retrieval step in which the LLM queries the Semantic Scholar API for related work and uses it as grounding. LLMs and agent systems still fall well short of producing concrete, actionable critiques. Retrieval consistently improves their output, which points toward retrieval-grounded tools for early feedback that complement human reviewers.

## Key points
- The taxonomy covers four aspects (Methodology, Experimental Design, Result Analysis, Literature Review), each split into finer subtypes such as low data quality or an inappropriate method.
- LimitGen-Human draws on ICLR 2025 reviews. This choice reduces training-data contamination, and ICLR reviews tend to be thorough because they are public and go through an extended rebuttal phase.
- RAG via Semantic Scholar helped the systems with stronger reasoning most, notably GPT-4o and the multi-agent MARG.
- A user study tested transfer to biomedical sciences and computer networks. Experts perturbed papers published after May 15, 2024, producing 32 examples, and the RAG pipeline improved GPT-4o and Llama-3.3-70B there as well.
- Stated limitations: the benchmark is text-only (no figures), uses only basic retrieval, covers a narrow time window and relies on automated evaluation. The dataset is released under CC BY 4.0 (yale-nlp/LimitGen).

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

📚 Shelves (7): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works · [🎯 Decision Sciences](https://md.research.elamur.ai/free/en/shelves/decision-sciences/) · 209 works · [🎯 Statistics, Probability and Uncertainty](https://md.research.elamur.ai/free/en/shelves/decision-sciences/statistics-probability-and-uncertainty/) · 61 works · [🎭 History and Philosophy of Science](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/history-and-philosophy-of-science/) · 53 works

🧭 Topics (2): [🧠 Language models & pretraining](https://md.research.elamur.ai/free/en/topics/language-models/) · 3,320 works · [📏 Evaluation & benchmarks](https://md.research.elamur.ai/free/en/topics/evaluation-benchmarks/) · 1,220 works

## Cite

Cite as: Zhijian Xu, Yilun Zhao, Manasi Patwardhan, Lovekesh Vig and Arman Cohan. 2025. Can LLMs Identify Critical Limitations within Scientific Research? A Systematic Evaluation on AI Research Papers. Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{xu2025can,
  title = {{Can LLMs Identify Critical Limitations within Scientific Research? A Systematic Evaluation on AI Research Papers}},
  author = {Zhijian Xu, Yilun Zhao, Manasi Patwardhan, Lovekesh Vig and Arman Cohan},
  booktitle = {Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2025},
  doi = {10.18653/v1/2025.acl-long.1009},
  url = {https://doi.org/10.18653/v1/2025.acl-long.1009}
}
```

- [← previous Can LLM Watermarks Robustly Prevent Unauthorized Knowledge Distillation?](https://md.research.elamur.ai/free/en/papers/pan-2025-can-llm-watermarks-robustly-prevent/)
- [next → Can Multimodal Large Language Models Understand Spatial Relations?](https://md.research.elamur.ai/free/en/papers/liu-2025-can-multimodal-large-language-models/)
- [↑ back to papers of 2025](https://md.research.elamur.ai/free/en/papers/2025/page/7/)

## References

39 references, 16 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Shubham Agarwal, Issam H Laradji, Laurent Charlin, and Christopher Pal. 2024. Litllm: A toolkit for scientific literature review. arXiv preprint arXiv:2402.01788. [find](https://scholar.google.com/scholar?q=Shubham+Agarwal%2C+Issam+H+Laradji%2C+Laurent+Charlin%2C+and+Christopher+Pal%2E+2024%2E+Litllm%3A+A+toolkit+for+scientific+literature+review%2E+arXiv+preprint+arXiv%3A2402%2E01788%2E)
2. AI@Meta. 2024. [The llama 3 herd of models](https://md.research.elamur.ai/free/en/papers/grattafiori-2024-llama-3-herd-models/).
3. Anirudh Ajith, Mengzhou Xia, Alexis Chevalier, Tanya Goyal, Danqi Chen, and Tianyu Gao. 2024. Lit-Search: A retrieval benchmark for scientific literature search. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 15068–15083, Miami, Florida, USA. Associa… [find](https://scholar.google.com/scholar?q=Anirudh+Ajith%2C+Mengzhou+Xia%2C+Alexis+Chevalier%2C+Tanya+Goyal%2C+Danqi+Chen%2C+and+Tianyu+Gao%2E+2024%2E+Lit%2DSearch%3A+A+retrieval+benchmark+for+scientific+literature+search%2E+In+Proceedings+of+the+2024+Conference+)
4. Akari Asai, Jacqueline He, Rulin Shao, Weijia Shi, Amanpreet Singh, Joseph Chee Chang, Kyle Lo, Luca Soldaini, Sergey Feldman, Mike D’arcy, et al. 2024. Openscholar: Synthesizing scientific literature with retrieval-augmented lms. arXiv preprint arXiv:2411.14199. [find](https://scholar.google.com/scholar?q=Akari+Asai%2C+Jacqueline+He%2C+Rulin+Shao%2C+Weijia+Shi%2C+Amanpreet+Singh%2C+Joseph+Chee+Chang%2C+Kyle+Lo%2C+Luca+Soldaini%2C+Sergey+Feldman%2C+Mike+D%E2%80%99arcy%2C+et+al%2E+2024%2E+Openscholar%3A+Synthesizing+scientific+literature)
5. Eric Chamoun, Michael Schlichtkrull, and Andreas Vlachos. 2024. Automated focused feedback generation for scientific writing assistance. In Findings of the Association for Computational Linguistics: ACL 2024, pages 9742–9763, Bangkok, Thailand. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Eric+Chamoun%2C+Michael+Schlichtkrull%2C+and+Andreas+Vlachos%2E+2024%2E+Automated+focused+feedback+generation+for+scientific+writing+assistance%2E+In+Findings+of+the+Association+for+Computational+Linguistics%3A+A)
6. Haonan Chen, Zhicheng Dou, Kelong Mao, Jiongnan Liu, and Ziliang Zhao. 2024. Generalizing conversational dense retrieval via LLM-cognition data augmentation. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 2700–2718, Bangkok,… [find](https://scholar.google.com/scholar?q=Haonan+Chen%2C+Zhicheng+Dou%2C+Kelong+Mao%2C+Jiongnan+Liu%2C+and+Ziliang+Zhao%2E+2024%2E+Generalizing+conversational+dense+retrieval+via+LLM%2Dcognition+data+augmentation%2E+In+Proceedings+of+the+62nd+Annual+Meeting+)
7. Mike D’Arcy, Tom Hope, Larry Birnbaum, and Doug Downey. 2024. Marg: Multi-agent review generation for scientific papers. arXiv preprint arXiv:2401.04259. [find](https://scholar.google.com/scholar?q=Mike+D%E2%80%99Arcy%2C+Tom+Hope%2C+Larry+Birnbaum%2C+and+Doug+Downey%2E+2024%2E+Marg%3A+Multi%2Dagent+review+generation+for+scientific+papers%2E+arXiv+preprint+arXiv%3A2401%2E04259%2E)
8. Jiangshu Du, Yibo Wang, Wenting Zhao, Zhongfen Deng, Shuaiqi Liu, Renze Lou, Henry Peng Zou, Pranav Narayanan Venkit, Nan Zhang, Mukund Srinath, Haoran Ranran Zhang, Vipul Gupta, Yinghui Li, Tao Li, Fei Wang, Qin Liu, Tianlin Liu, Pengzhi Gao, Congying Xia, Chen Xing, Cheng Jiayang, Zhaowei Wang, Yi… — [LLMs Assist NLP Researchers: Critique Paper (Meta-)Reviewing](https://md.research.elamur.ai/free/en/papers/du-2024-llms-assist-nlp-researchers-critique/)
9. Zhaolin Gao, Kianté Brantley, and Thorsten Joachims. 2024. [Reviewer2: Optimizing review generation through prompt generation](https://md.research.elamur.ai/free/en/papers/gao-2024-reviewer2-optimizing-review-generation-through/). arXiv preprint arXiv:2402.10886.
10. Hangfeng He, Hongming Zhang, and Dan Roth. 2022. Rethinking with retrieval: Faithful large language model inference. arXiv preprint arXiv:2301.00303. [find](https://scholar.google.com/scholar?q=Hangfeng+He%2C+Hongming+Zhang%2C+and+Dan+Roth%2E+2022%2E+Rethinking+with+retrieval%3A+Faithful+large+language+model+inference%2E+arXiv+preprint+arXiv%3A2301%2E00303%2E)
11. Qian Huang, Jian Vora, Percy Liang, and Jure Leskovec. 2024. [Mlagentbench: Evaluating language agents on machine learning experimentation](https://md.research.elamur.ai/free/en/papers/huang-2023-mlagentbench-evaluating-language-agents/). In Forty-first International Conference on Machine Learning.
12. Gautier Izacard, Patrick Lewis, Maria Lomeli, Lucas Hosseini, Fabio Petroni, Timo Schick, Jane Dwivedi-Yu, Armand Joulin, Sebastian Riedel, and Edouard Grave. 2023. [Atlas: Few-shot learning with retrieval augmented language models](https://md.research.elamur.ai/free/en/papers/izacard-2022-atlas-few-shot-learning-retrieval/). Journal of Machine Learning Research, 24(251):1–43.
13. Zhengbao Jiang, Frank Xu, Luyu Gao, Zhiqing Sun, Qian Liu, Jane Dwivedi-Yu, Yiming Yang, Jamie Callan, and Graham Neubig. 2023. [Active retrieval augmented generation](https://md.research.elamur.ai/free/en/papers/jiang-2023-active-retrieval-augmented-generation/). In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 7969–7992, Singapore. Association fo…
14. Jungo Kasai, Keisuke Sakaguchi, Ronan Le Bras, Akari Asai, Xinyan Yu, Dragomir Radev, Noah A Smith, Yejin Choi, Kentaro Inui, et al. 2024. [Realtime qa: what’s the answer right now?](https://md.research.elamur.ai/free/en/papers/kasai-2022-realtime-qa-s-answer-right/) Advances in Neural Information Processing Systems, 36.
15. Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, et al. 2020. [Retrieval-augmented generation for knowledge-intensive nlp tasks](https://md.research.elamur.ai/free/en/papers/lewis-2020-retrieval-augmented-generation-knowledge-intensive/). Advances in Neural Information Processing Systems, 33:9459–9474.
16. Weixin Liang, Yuhui Zhang, Hancheng Cao, Binglu Wang, Daisy Yi Ding, Xinyu Yang, Kailas Vodrahalli, Siyu He, Daniel Scott Smith, Yian Yin, et al. 2024. Can large language models provide useful feedback on research papers? a large-scale empirical analysis. NEJM AI, 1(8):AIoa2400196. [find](https://scholar.google.com/scholar?q=Weixin+Liang%2C+Yuhui+Zhang%2C+Hancheng+Cao%2C+Binglu+Wang%2C+Daisy+Yi+Ding%2C+Xinyu+Yang%2C+Kailas+Vodrahalli%2C+Siyu+He%2C+Daniel+Scott+Smith%2C+Yian+Yin%2C+et+al%2E+2024%2E+Can+large+language+models+provide+useful+feedbac)
17. Ryan Liu and Nihar B Shah. 2023. Reviewergpt? an exploratory study on using large language models for paper reviewing. arXiv preprint arXiv:2306.00622. [find](https://scholar.google.com/scholar?q=Ryan+Liu+and+Nihar+B+Shah%2E+2023%2E+Reviewergpt%3F+an+exploratory+study+on+using+large+language+models+for+paper+reviewing%2E+arXiv+preprint+arXiv%3A2306%2E00622%2E)
18. Kyle Lo, Lucy Lu Wang, Mark Neumann, Rodney Kinney, and Daniel Weld. 2020. [S2ORC: The semantic scholar open research corpus](https://md.research.elamur.ai/free/en/papers/lo-2019-s2orc-semantic-scholar-open-research/). In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 4969–4983, Online. Association for Computational Linguistics.
19. Renze Lou, Hanzi Xu, Sijia Wang, Jiangshu Du, Ryo Kamoi, Xiaoxin Lu, Jian Xie, Yuxuan Sun, Yusen Zhang, Jihyun Janice Ahn, et al. 2024. Aaar-1.0: Assessing ai’s potential to assist research. arXiv preprint arXiv:2410.22394. — [arXiv:2410.22394](https://arxiv.org/abs/2410.22394) [find](https://scholar.google.com/scholar?q=Renze+Lou%2C+Hanzi+Xu%2C+Sijia+Wang%2C+Jiangshu+Du%2C+Ryo+Kamoi%2C+Xiaoxin+Lu%2C+Jian+Xie%2C+Yuxuan+Sun%2C+Yusen+Zhang%2C+Jihyun+Janice+Ahn%2C+et+al%2E+2024%2E+Aaar%2D1%2E0%3A+Assessing+ai%E2%80%99s+potential+to+assist+research%2E+arXiv+pre)
20. Chris Lu, Cong Lu, Robert Tjarko Lange, Jakob Foerster, Jeff Clune, and David Ha. 2024. [The ai scientist: Towards fully automated open-ended scientific discovery](https://md.research.elamur.ai/free/en/papers/lu-2024-ai-scientist-fully-automated/). arXiv preprint arXiv:2408.06292.
21. Alex Mallen, Akari Asai, Victor Zhong, Rajarshi Das, Daniel Khashabi, and Hannaneh Hajishirzi. 2023. When not to trust language models: Investigating effectiveness of parametric and non-parametric memories. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (V… [find](https://scholar.google.com/scholar?q=Alex+Mallen%2C+Akari+Asai%2C+Victor+Zhong%2C+Rajarshi+Das%2C+Daniel+Khashabi%2C+and+Hannaneh+Hajishirzi%2E+2023%2E+When+not+to+trust+language+models%3A+Investigating+effectiveness+of+parametric+and+non%2Dparametric+mem)
22. Abhika Mishra, Akari Asai, Vidhisha Balachandran, Yizhong Wang, Graham Neubig, Yulia Tsvetkov, and Hannaneh Hajishirzi. 2024. [Fine-grained hallucination detection and editing for language models](https://md.research.elamur.ai/free/en/papers/mishra-2024-fine-grained-hallucination-detection-editing/). In First Conference on Language Modeling.
23. OpenAI. 2024. Hello gpt-4o. [find](https://scholar.google.com/scholar?q=OpenAI%2E+2024%2E+Hello+gpt%2D4o%2E)
24. Ori Press, Andreas Hochlehnert, Ameya Prabhu, Vishaal Udandarao, Ofir Press, and Matthias Bethge. 2024. CiteME: Can language models accurately cite scientific claims? In The Thirty-eight Conference on Neural Information Processing Systems Datasets and Benchmarks Track. [find](https://scholar.google.com/scholar?q=Ori+Press%2C+Andreas+Hochlehnert%2C+Ameya+Prabhu%2C+Vishaal+Udandarao%2C+Ofir+Press%2C+and+Matthias+Bethge%2E+2024%2E+CiteME%3A+Can+language+models+accurately+cite+scientific+claims%3F+In+The+Thirty%2Deight+Conference+on)
25. Zhihong Shao, Yeyun Gong, Yelong Shen, Minlie Huang, Nan Duan, and Weizhu Chen. 2023. [Enhancing retrieval-augmented large language models with iterative retrieval-generation synergy](https://md.research.elamur.ai/free/en/papers/shao-2023-enhancing-retrieval-augmented-large-language/). In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 9248–9274, Singapore. Association for…
26. Weijia Shi, Sewon Min, Michihiro Yasunaga, Minjoon Seo, Richard James, Mike Lewis, Luke Zettlemoyer, and Wen-tau Yih. 2024. [REPLUG: Retrieval-augmented black-box language models](https://md.research.elamur.ai/free/en/papers/shi-2023-replug-retrieval-augmented-black-box/). In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Hum…
27. Kurt Shuster, Spencer Poff, Moya Chen, Douwe Kiela, and Jason Weston. 2021. [Retrieval augmentation reduces hallucination in conversation](https://md.research.elamur.ai/free/en/papers/shuster-2021-retrieval-augmentation-reduces-hallucination-conversation/). In Findings of the Association for Computational Linguistics: EMNLP 2021, pages 3784–3803, Punta Cana, Dominican Republic. Association for Computational Linguisti…
28. Chenglei Si, Diyi Yang, and Tatsunori Hashimoto. 2024. [Can llms generate novel research ideas? a large-scale human study with 100+ nlp researchers](https://md.research.elamur.ai/free/en/papers/si-2024-llms-generate-novel-research/). arXiv preprint arXiv:2409.04109.
29. Michael D Skarlinski, Sam Cox, Jon M Laurent, James D Braza, Michaela Hinks, Michael J Hammerling, Manvitha Ponnapati, Samuel G Rodriques, and Andrew D White. 2024. Language agents achieve superhuman synthesis of scientific knowledge. arXiv preprint arXiv:2409.13740. [find](https://scholar.google.com/scholar?q=Michael+D+Skarlinski%2C+Sam+Cox%2C+Jon+M+Laurent%2C+James+D+Braza%2C+Michaela+Hinks%2C+Michael+J+Hammerling%2C+Manvitha+Ponnapati%2C+Samuel+G+Rodriques%2C+and+Andrew+D+White%2E+2024%2E+Language+agents+achieve+superhuman+)
30. Xi Ye, Ruoxi Sun, Sercan Arik, and Tomas Pfister. 2024. Effective large language model adaptation for improved grounding and citation generation. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume… [find](https://scholar.google.com/scholar?q=Xi+Ye%2C+Ruoxi+Sun%2C+Sercan+Arik%2C+and+Tomas+Pfister%2E+2024%2E+Effective+large+language+model+adaptation+for+improved+grounding+and+citation+generation%2E+In+Proceedings+of+the+2024+Conference+of+the+North+Ame)
31. Jianxiang Yu, Zichen Ding, Jiaqi Tan, Kangyang Luo, Zhenmin Weng, Chenghua Gong, Long Zeng, Ren-Jing Cui, Chengcheng Han, Qiushi Sun, et al. 2024. Automated peer reviewing in paper sea: Standardization, evaluation, and analysis. In Findings of the Association for Computational Linguistics: EMNLP 202… [find](https://scholar.google.com/scholar?q=Jianxiang+Yu%2C+Zichen+Ding%2C+Jiaqi+Tan%2C+Kangyang+Luo%2C+Zhenmin+Weng%2C+Chenghua+Gong%2C+Long+Zeng%2C+Ren%2DJing+Cui%2C+Chengcheng+Han%2C+Qiushi+Sun%2C+et+al%2E+2024%2E+Automated+peer+reviewing+in+paper+sea%3A+Standardizatio)
32. Ruiyang Zhou, Lu Chen, and Kai Yu. 2024a. [Is llm a reliable reviewer? a comprehensive evaluation of llm on automatic paper reviewing tasks](https://md.research.elamur.ai/free/en/papers/zhou-2024-is-llm-reliable-reviewer-comprehensive/). In International Conference on Language Resources and Evaluation.
33. Yangqiaoyu Zhou, Haokun Liu, Tejes Srivastava, Hongyuan Mei, and Chenhao Tan. 2024b. Hypothesis generation with large language models. [find](https://scholar.google.com/scholar?q=Yangqiaoyu+Zhou%2C+Haokun+Liu%2C+Tejes+Srivastava%2C+Hongyuan+Mei%2C+and+Chenhao+Tan%2E+2024b%2E+Hypothesis+generation+with+large+language+models%2E)
34. Cheng Tan, Dongxin Lyu, Siyuan Li, Zhangyang Gao, Jingxuan Wei, Siqi Ma, Zicheng Liu, and Stan Z Li. 2024. Peer review as a multi-turn and long-context dialogue with role-based interactions. arXiv preprint arXiv:2406.05688. — [arXiv:2406.05688](https://arxiv.org/abs/2406.05688) [find](https://scholar.google.com/scholar?q=Cheng+Tan%2C+Dongxin+Lyu%2C+Siyuan+Li%2C+Zhangyang+Gao%2C+Jingxuan+Wei%2C+Siqi+Ma%2C+Zicheng+Liu%2C+and+Stan+Z+Li%2E+2024%2E+Peer+review+as+a+multi%2Dturn+and+long%2Dcontext+dialogue+with+role%2Dbased+interactions%2E+arXiv+pre)
35. Minyang Tian, Luyu Gao, Shizhuo Dylan Zhang, Xinan Chen, Cunwei Fan, Xuefei Guo, Roland Haas, Pan Ji, Kittithat Krongchon, Yao Li, et al. 2024. Scicode: A research coding benchmark curated by scientists. arXiv preprint arXiv:2407.13168. [find](https://scholar.google.com/scholar?q=Minyang+Tian%2C+Luyu+Gao%2C+Shizhuo+Dylan+Zhang%2C+Xinan+Chen%2C+Cunwei+Fan%2C+Xuefei+Guo%2C+Roland+Haas%2C+Pan+Ji%2C+Kittithat+Krongchon%2C+Yao+Li%2C+et+al%2E+2024%2E+Scicode%3A+A+research+coding+benchmark+curated+by+scientis)
36. Keith Tyser, Ben Segev, Gaston Longhitano, Xin-Yu Zhang, Zachary Meeks, Jason Lee, Uday Garg, Nicholas Belsten, Avi Shporer, Madeleine Udell, et al. 2024. Ai-driven review systems: Evaluating llms in scalable and bias-aware academic reviews. arXiv preprint arXiv:2408.10365. [find](https://scholar.google.com/scholar?q=Keith+Tyser%2C+Ben+Segev%2C+Gaston+Longhitano%2C+Xin%2DYu+Zhang%2C+Zachary+Meeks%2C+Jason+Lee%2C+Uday+Garg%2C+Nicholas+Belsten%2C+Avi+Shporer%2C+Madeleine+Udell%2C+et+al%2E+2024%2E+Ai%2Ddriven+review+systems%3A+Evaluating+llms+in+)
37. Qingyun Wang, Doug Downey, Heng Ji, and Tom Hope. 2024. Scimon: Scientific inspiration machines optimized for novelty. [find](https://scholar.google.com/scholar?q=Qingyun+Wang%2C+Doug+Downey%2C+Heng+Ji%2C+and+Tom+Hope%2E+2024%2E+Scimon%3A+Scientific+inspiration+machines+optimized+for+novelty%2E)
38. Fangyuan Xu, Kyle Lo, Luca Soldaini, Bailey Kuehl, Eunsol Choi, and David Wadden. 2024. KIWI: A dataset of knowledge-intensive writing instructions for answering research questions. In Findings of the Association for Computational Linguistics: ACL 2024, pages 12969–12990, Bangkok, Thailand. Associat… [find](https://scholar.google.com/scholar?q=Fangyuan+Xu%2C+Kyle+Lo%2C+Luca+Soldaini%2C+Bailey+Kuehl%2C+Eunsol+Choi%2C+and+David+Wadden%2E+2024%2E+KIWI%3A+A+dataset+of+knowledge%2Dintensive+writing+instructions+for+answering+research+questions%2E+In+Findings+of+the)
39. An Yang, Baosong Yang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Zhou, Chengpeng Li, Chengyuan Li, Dayiheng Liu, Fei Huang, et al. 2024. Qwen2 technical report. arXiv preprint arXiv:2407.10671. [find](https://scholar.google.com/scholar?q=An+Yang%2C+Baosong+Yang%2C+Binyuan+Hui%2C+Bo+Zheng%2C+Bowen+Yu%2C+Chang+Zhou%2C+Chengpeng+Li%2C+Chengyuan+Li%2C+Dayiheng+Liu%2C+Fei+Huang%2C+et+al%2E+2024%2E+Qwen2+technical+report%2E+arXiv+preprint+arXiv%3A2407%2E10671%2E)

*Updated 31 October 2026*