---
title: "NovBench — paper by Wu et al. (2026) — summary and ideas"
description: "The benchmark offers a controlled, aspect-specific testbed for AI-assisted reviewing. 80 references, cited by 1 paper here."
authors: ["Wenqing Wu", "Yi Zhao", "Yuzhuo Wang", "Siyou Li", "Juexi Shao", "Yunfei Long", "Chengzhi Zhang"]
year: 2026
venue: "Findings of the Association for Computational Linguistics: ACL 2026"
source: "https://aclanthology.org/2026.findings-acl.1607/"
doi: "10.18653/v1/2026.findings-acl.1607"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/wu-2026-novbench-evaluating-large-language-models/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/wu-2026-novbench-evaluating-large-language-models/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/4/) / NovBench: Evaluating Large Language Models on Academic Paper Novelty Assessment

# NovBench: Evaluating Large Language Models on Academic Paper Novelty Assessment

**Authors:** Wenqing Wu, [Yi Zhao](https://md.research.elamur.ai/free/en/authors/yi-zhao/), Yuzhuo Wang, Siyou Li, Juexi Shao, [Yunfei Long](https://md.research.elamur.ai/free/en/authors/yunfei-long/) and Chengzhi Zhang · **Year:** 2026 · **Venue:** [Findings of the Association for Computational Linguistics: ACL 2026](https://md.research.elamur.ai/free/en/venues/findings/) ([Findings of the Association for Computational Linguistics 2026](https://md.research.elamur.ai/free/en/venues/findings/2026/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2026.findings-acl.1607/) · [PDF](https://aclanthology.org/2026.findings-acl.1607.pdf) · [DOI 10.18653/v1/2026.findings-acl.1607](https://doi.org/10.18653/v1/2026.findings-acl.1607) · [OpenAlex](https://openalex.org/W7166877060)

**Citations:** 3 citations (OpenAlex, 2026-10-14) · [1](https://md.research.elamur.ai/free/en/papers/wu-2026-novbench-evaluating-large-language-models/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
Novelty is a central criterion in peer review, and rising submission volumes are straining reviewers. LLMs can already draft review comments, but no benchmark existed to test whether they can judge novelty specifically. Most prior evaluation either scored whole reviews or relied on lexical metrics such as ROUGE/BLEU or opaque LLM-as-judge setups. The authors built NovBench, which pairs papers from EMNLP 2023 with expert reviews. For each paper, it contains the novelty claims taken from the introduction and the reviewers' novelty-related remarks, grouped by sentiment. They also propose an interpretable four-dimensional scoring scheme for generated novelty assessments. General-purpose LLMs (e.g., GPT-5, Gemini-2.5-flash) and peer-review-tuned models were tested with several prompting strategies. The results show that current models grasp scientific novelty only weakly. Specialized fine-tuned models often fail to follow instructions, which sharply lowers their scores. The benchmark offers a controlled, aspect-specific testbed for AI-assisted reviewing. The findings suggest that fine-tuning should target both novelty understanding and instruction adherence.

## Key points
- **Data:** NovBench contains 1,684 paper–review pairs from EMNLP 2023, sourced from NLPeer (papers parsed in ITG format) and OpenReview.
- **Construction pipeline:**
  - Extraction prompts were first validated on a manually annotated COLING 2020 subset of 87 papers.
  - GPT-5 with in-context learning extracted novelty descriptions from introductions.
  - GPT-4o-mini extracted novelty comments from reviews, after being compared against other models on an aspect-identification resource.
  - GPT-4o removed duplicate comments and grouped them by sentiment polarity.
- **Evaluation framework:** Generated novelty assessments are scored on four dimensions: Relevance, Correctness, Coverage, and Clarity.
- **Additional analyses:**
  - Performance stays stable across model generations, publication years, and input perturbations, which argues against memorization or temporal leakage.
  - Models do better on resource papers than on methodological ones.
  - When reviewers disagree, model outputs align more closely with high-confidence reviews.
- **Limitations and availability:**
  - Only introductions are used as input, and the venues are limited to NLP.
  - Reviews come mostly from accepted papers, which may introduce selection bias.
  - All resources are released at github.com/njust-winchy/llm4novelty.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Queen Mary University of London](https://md.research.elamur.ai/free/en/universities/queen-mary-university-of-london/) — University, London, United Kingdom · 57 works · 3,600 citations
- [Nanjing University of Science and Technology](https://md.research.elamur.ai/free/en/universities/nanjing-university-of-science-and-technology/) — University, Nanjing, China · 19 works · 1,590 citations
- [Anhui University](https://md.research.elamur.ai/free/en/universities/anhui-university/) — University, Hefei, China · 11 works · 7 citations

📚 Shelves (5): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 18,614 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 17,965 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 17,450 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 17,296 works · [🎭 History and Philosophy of Science](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/history-and-philosophy-of-science/) · 62 works

🧭 Topics (1): [🧠 Language models & pretraining](https://md.research.elamur.ai/free/en/topics/language-models/) · 3,894 works

## Cite

Cite as: Wenqing Wu, Yi Zhao, Yuzhuo Wang, Siyou Li, Juexi Shao, Yunfei Long and Chengzhi Zhang. 2026. NovBench: Evaluating Large Language Models on Academic Paper Novelty Assessment. Findings of the Association for Computational Linguistics: ACL 2026.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{wu2026novbench,
  title = {{NovBench: Evaluating Large Language Models on Academic Paper Novelty Assessment}},
  author = {Wenqing Wu, Yi Zhao, Yuzhuo Wang, Siyou Li, Juexi Shao, Yunfei Long and Chengzhi Zhang},
  booktitle = {Findings of the Association for Computational Linguistics: ACL 2026},
  year = {2026},
  doi = {10.18653/v1/2026.findings-acl.1607},
  url = {https://doi.org/10.18653/v1/2026.findings-acl.1607}
}
```

- [← previous Multimodal Item Scoring for Natural Language Recommendation via Gaussian Process Regression with LLM Relevance Judgments](https://md.research.elamur.ai/free/en/papers/liu-2026-multimodal-item-scoring-natural-language/)
- [next → OctoBench: Benchmarking Scaffold-Aware Instruction Following in Repository-Grounded Agentic Coding](https://md.research.elamur.ai/free/en/papers/ding-2026-octobench-benchmarking-scaffold-aware-instruction/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/4/)

## References

80 references, 31 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Marah Abdin, Jyoti Aneja, Harkirat Behl, Sébastien Bubeck, Ronen Eldan, Suriya Gunasekar, Michael Harrison, Russell J. Hewett, Mojan Javaheripi, Piero Kauffmann, James R. Lee, Yin Tat Lee, Yuanzhi Li, Weishung Liu, Caio C. T. Mendes, Anh Nguyen, Eric Price, Gustavo de Rosa, Olli Saarikivi, and 8 oth… — [Phi-4 Technical Report](https://md.research.elamur.ai/free/en/papers/abdin-2024-phi-4-technical-report/)
2. David Adam. 2025. The peer-review crisis: how to fix an overloaded system. Nature, 644(8075):24–27. [find](https://scholar.google.com/scholar?q=David+Adam%2E+2025%2E+The+peer%2Dreview+crisis%3A+how+to+fix+an+overloaded+system%2E+Nature%2C+644%288075%29%3A24%E2%80%9327%2E)
3. Osama Mohammed Afzal, Preslav Nakov, Tom Hope, and Iryna Gurevych. 2026. Beyond “not novel enough”: Enriching scholarly critique with LLM-assisted feedback. In Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers), pages 2… [find](https://scholar.google.com/scholar?q=Osama+Mohammed+Afzal%2C+Preslav+Nakov%2C+Tom+Hope%2C+and+Iryna+Gurevych%2E+2026%2E+Beyond+%E2%80%9Cnot+novel+enough%E2%80%9D%3A+Enriching+scholarly+critique+with+LLM%2Dassisted+feedback%2E+In+Proceedings+of+the+19th+Conference+of+th)
4. Bruce Alberts, Brooks Hanson, and Katrina L. Kelner. 2008. Reviewing peer review. Science, 321(5885):15–15. [find](https://scholar.google.com/scholar?q=Bruce+Alberts%2C+Brooks+Hanson%2C+and+Katrina+L%2E+Kelner%2E+2008%2E+Reviewing+peer+review%2E+Science%2C+321%285885%29%3A15%E2%80%9315%2E)
5. Tong Bao, Mir Tafseer Nayeem, Davood Rafiei, and Chengzhi Zhang. 2025. SurveyGen: Quality-aware scientific survey generation with large language models. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pages 2712–2736, Suzhou, China. Association for Computat… [find](https://scholar.google.com/scholar?q=Tong+Bao%2C+Mir+Tafseer+Nayeem%2C+Davood+Rafiei%2C+and+Chengzhi+Zhang%2E+2025%2E+SurveyGen%3A+Quality%2Daware+scientific+survey+generation+with+large+language+models%2E+In+Proceedings+of+the+2025+Conference+on+Empiri)
6. Rewina Bedemariam, Natalie Perez, Sreyoshi Bhaduri, Satya Kapoor, Alex Gil, Elizabeth Conjar, Ikkei Itoku, David Theil, Aman Chadha, and Naumaan Nayyar. 2025. Potential and perils of large language models as judges of unstructured textual data. Preprint, arXiv:2501.08167. — [arXiv:2501.08167](https://arxiv.org/abs/2501.08167) [find](https://scholar.google.com/scholar?q=Rewina+Bedemariam%2C+Natalie+Perez%2C+Sreyoshi+Bhaduri%2C+Satya+Kapoor%2C+Alex+Gil%2C+Elizabeth+Conjar%2C+Ikkei+Itoku%2C+David+Theil%2C+Aman+Chadha%2C+and+Naumaan+Nayyar%2E+2025%2E+Potential+and+perils+of+large+language+mo)
7. Yuan Chang, Ziyue Li, Hengyuan Zhang, Yuanbo Kong, Yanru Wu, Hayden Kwok-Hay So, Zhijiang Guo, Liya Zhu, and Ngai Wong. 2025. [TreeReview: A dynamic tree of questions framework for deep and efficient LLM-based scientific peer review](https://md.research.elamur.ai/free/en/papers/chang-2025-treereview-dynamic-tree-questions-framework/). In Proceedings of the 2025 Conference on Empirical Methods in Natur…
8. Junyi Chen, Shihao Bai, Zaijun Wang, Siyu Wu, Chuheng Du, Hailong Yang, Ruihao Gong, Shengzhong Liu, Fan Wu, and Guihai Chen. 2025. Pre 3 : Enabling deterministic pushdown automata for faster structured LLM generation. In Proceedings of the 63rd Annual Meeting of the Association for Computational Li… [find](https://scholar.google.com/scholar?q=Junyi+Chen%2C+Shihao+Bai%2C+Zaijun+Wang%2C+Siyu+Wu%2C+Chuheng+Du%2C+Hailong+Yang%2C+Ruihao+Gong%2C+Shengzhong+Liu%2C+Fan+Wu%2C+and+Guihai+Chen%2E+2025%2E+Pre+3+%3A+Enabling+deterministic+pushdown+automata+for+faster+structur)
9. Barak A Cohen. 2017. Point of view: How should novelty be valued in science? eLife, 6:e28699. [find](https://scholar.google.com/scholar?q=Barak+A+Cohen%2E+2017%2E+Point+of+view%3A+How+should+novelty+be+valued+in+science%3F+eLife%2C+6%3Ae28699%2E)
10. Italo Luis da Silva, Hanqi Yan, Lin Gui, and Yulan He. 2025. GraphMind: Interactive novelty assessment system for accelerating scientific discovery. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pages 286–294, Suzhou, China. Associa… [find](https://scholar.google.com/scholar?q=Italo+Luis+da+Silva%2C+Hanqi+Yan%2C+Lin+Gui%2C+and+Yulan+He%2E+2025%2E+GraphMind%3A+Interactive+novelty+assessment+system+for+accelerating+scientific+discovery%2E+In+Proceedings+of+the+2025+Conference+on+Empirical+)
11. Mike D’Arcy, Tom Hope, Larry Birnbaum, and Doug Downey. 2024. [Marg: Multi-agent review generation for scientific papers](https://md.research.elamur.ai/free/en/papers/arcy-2024-marg-multi-agent-review/). Preprint, arXiv:2401.04259.
12. DeepSeek-AI, Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, Xiaokang Zhang, Xingkai Yu, Yu Wu, Z. F. Wu, Zhibin Gou, Zhihong Shao, Zhuoshu Li, Ziyi Gao, and 181 others. 2025. [Deepseek-r1: Incentivizing reasoning capability in ll…](https://md.research.elamur.ai/free/en/papers/deepseek-ai-2025-deepseek-r1-incentivizing-reasoning-capability/)
13. Jiangshu Du, Yibo Wang, Wenting Zhao, Zhongfen Deng, Shuaiqi Liu, Renze Lou, Henry Peng Zou, Pranav Narayanan Venkit, Nan Zhang, Mukund Srinath, Haoran Ranran Zhang, Vipul Gupta, Yinghui Li, Tao Li, Fei Wang, Qin Liu, Tianlin Liu, Pengzhi Gao, Congying Xia, and 21 others. 2024. [LLMs assist NLP resea…](https://md.research.elamur.ai/free/en/papers/du-2024-llms-assist-nlp-researchers-critique/)
14. Nils Dycke, Ilia Kuznetsov, and Iryna Gurevych. 2023. [NLPeer: A unified resource for the computational study of peer review](https://md.research.elamur.ai/free/en/papers/dycke-2023-nlpeer-unified-resource-computational-study/). In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 5049– 5073, Toronto, Canada. Association for Computat…
15. Joseph L Fleiss. 1971. Measuring nominal scale agreement among many raters. Psychol. Bull., 76(5):378– 382. [find](https://scholar.google.com/scholar?q=Joseph+L+Fleiss%2E+1971%2E+Measuring+nominal+scale+agreement+among+many+raters%2E+Psychol%2E+Bull%2E%2C+76%285%29%3A378%E2%80%93+382%2E)
16. Zhaolin Gao, Kianté Brantley, and Thorsten Joachims. 2024. [Reviewer2: Optimizing review generation through prompt generation](https://md.research.elamur.ai/free/en/papers/gao-2024-reviewer2-optimizing-review-generation-through/). Preprint, arXiv:2402.10886.
17. Google Gemini Team. 2025. [Gemini 2.5: Pushing the frontier with advanced reasoning, multimodality, long context, and next generation agentic capabilities](https://md.research.elamur.ai/free/en/papers/comanici-2025-gemini-2-5-pushing-frontier/). Preprint, arXiv:2507.06261.
18. Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, Amy Yang, Angela Fan, Anirudh Goyal, Anthony Hartshorn, Aobo Yang, Archi Mitra, Archie Sravankumar, Artem Korenev, Arthur Hinsvark, and 542 o… — [The Llama 3 Herd of Models](https://md.research.elamur.ai/free/en/papers/grattafiori-2024-llama-3-herd-models/)
19. Shengzhi Huang, Yong Huang, Yinpeng Liu, Zhuoran Luo, and Wei Lu. 2025a. Are large language models qualified reviewers in originality evaluation? Information Processing & Management, 62(3):103973. [find](https://scholar.google.com/scholar?q=Shengzhi+Huang%2C+Yong+Huang%2C+Yinpeng+Liu%2C+Zhuoran+Luo%2C+and+Wei+Lu%2E+2025a%2E+Are+large+language+models+qualified+reviewers+in+originality+evaluation%3F+Information+Processing+%26+Management%2C+62%283%29%3A103973%2E)
20. Xuanjing Huang, Shihan Dou, and Zhangyue Yin. 2025b. The dual-edged sword: artificial intelligence’s evolving role in academic peer review. Sci. China Inf. Sci., 68(11). [find](https://scholar.google.com/scholar?q=Xuanjing+Huang%2C+Shihan+Dou%2C+and+Zhangyue+Yin%2E+2025b%2E+The+dual%2Dedged+sword%3A+artificial+intelligence%E2%80%99s+evolving+role+in+academic+peer+review%2E+Sci%2E+China+Inf%2E+Sci%2E%2C+68%2811%29%2E)
21. Maximilian Idahl and Zahra Ahmadi. 2025. OpenReviewer: A specialized large language model for generating critical scientific paper reviews. In Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Syst… [find](https://scholar.google.com/scholar?q=Maximilian+Idahl+and+Zahra+Ahmadi%2E+2025%2E+OpenReviewer%3A+A+specialized+large+language+model+for+generating+critical+scientific+paper+reviews%2E+In+Proceedings+of+the+2025+Conference+of+the+Nations+of+the+)
22. Daeseong Jeon, Junyoup Lee, Joon Mo Ahn, and Changyong Lee. 2023. Measuring the novelty of scientific publications: A fasttext and local outlier factor approach. Journal of Informetrics, 17(4):101450. [find](https://scholar.google.com/scholar?q=Daeseong+Jeon%2C+Junyoup+Lee%2C+Joon+Mo+Ahn%2C+and+Changyong+Lee%2E+2023%2E+Measuring+the+novelty+of+scientific+publications%3A+A+fasttext+and+local+outlier+factor+approach%2E+Journal+of+Informetrics%2C+17%284%29%3A101450%2E)
23. Albert Q. Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, Lélio Renard Lavaud, Marie-Anne Lachaux, Pierre Stock, Teven Le Scao, Thibaut Lavril, Thomas Wang, Timothée Lacroix,… [find](https://scholar.google.com/scholar?q=Albert+Q%2E+Jiang%2C+Alexandre+Sablayrolles%2C+Arthur+Mensch%2C+Chris+Bamford%2C+Devendra+Singh+Chaplot%2C+Diego+de+las+Casas%2C+Florian+Bressand%2C+Gianna+Lengyel%2C+Guillaume+Lample%2C+Lucile+Saulnier%2C+L%C3%A9lio+Renard+Lav)
24. Yiqiao Jin, Qinlin Zhao, Yiyang Wang, Hao Chen, Kaijie Zhu, Yijia Xiao, and Jindong Wang. 2024. [AgentReview: Exploring peer review dynamics with LLM agents](https://md.research.elamur.ai/free/en/papers/jin-2024-agentreview-exploring-peer-review/). In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 1208–1226, Miami, Florida, USA. Association fo…
25. Dongyeop Kang, Waleed Ammar, Bhavana Dalvi, Madeleine van Zuylen, Sebastian Kohlmeier, Eduard Hovy, and Roy Schwartz. 2018. [A dataset of peer reviews (PeerRead): Collection, insights and NLP applications](https://md.research.elamur.ai/free/en/papers/kang-2018-dataset-peer-reviews-peerread-collection/). In Proceedings of the 2018 Conference of the North American Chapter of the Association for Comp…
26. Thomas King, Simon Butcher, and Lukasz Zalewski. 2017. Apocrita - High Performance Computing Cluster for Queen Mary University of London. [find](https://scholar.google.com/scholar?q=Thomas+King%2C+Simon+Butcher%2C+and+Lukasz+Zalewski%2E+2017%2E+Apocrita+%2D+High+Performance+Computing+Cluster+for+Queen+Mary+University+of+London%2E)
27. Sandeep Kumar, Tirthankar Ghosal, Vinayak Goyal, and Asif Ekbal. 2025. [Can large language models unlock novel scientific research ideas?](https://md.research.elamur.ai/free/en/papers/kumar-2025-can-large-language-models-unlock/) In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pages 33551–33575, Suzhou, China. Association for Computational Linguist…
28. Ilia Kuznetsov, Osama Mohammed Afzal, Koen Dercksen, Nils Dycke, Alexander Goldberg, Tom Hope, Dirk Hovy, Jonathan K. Kummerfeld, Anne Lauscher, Kevin Leyton-Brown, Sheng Lu, Mausam, Margot Mieskes, Aurélie Névéol, Danish Pruthi, Lizhen Qu, Roy Schwartz, Noah A. Smith, Thamar Solorio, and 5 others.… [find](https://scholar.google.com/scholar?q=Ilia+Kuznetsov%2C+Osama+Mohammed+Afzal%2C+Koen+Dercksen%2C+Nils+Dycke%2C+Alexander+Goldberg%2C+Tom+Hope%2C+Dirk+Hovy%2C+Jonathan+K%2E+Kummerfeld%2C+Anne+Lauscher%2C+Kevin+Leyton%2DBrown%2C+Sheng+Lu%2C+Mausam%2C+Margot+Mieskes%2C+A)
29. Ilia Kuznetsov, Jan Buchmann, Max Eichler, and Iryna Gurevych. 2022. [Revise and resubmit: An intertextual model of text-based collaboration in peer review](https://md.research.elamur.ai/free/en/papers/kuznetsov-2022-revise-resubmit-intertextual-model-text/). Computational Linguistics, 48(4):949–986.
30. Anne Lauscher, Goran Glavaš, and Simone Paolo Ponzetto. 2018. An argument-annotated corpus of scientific publications. In Proceedings of the 5th Workshop on Argument Mining, pages 40–46, Brussels, Belgium. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Anne+Lauscher%2C+Goran+Glava%C5%A1%2C+and+Simone+Paolo+Ponzetto%2E+2018%2E+An+argument%2Dannotated+corpus+of+scientific+publications%2E+In+Proceedings+of+the+5th+Workshop+on+Argument+Mining%2C+pages+40%E2%80%9346%2C+Brussels%2C+Bel)
31. Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, Sebastian Riedel, and Douwe Kiela. 2020. [Retrieval-augmented generation for knowledge-intensive nlp tasks](https://md.research.elamur.ai/free/en/papers/lewis-2020-retrieval-augmented-generation-knowledge-intensive/). In Proceedings of the 34th Internation…
32. Jiyi Li, Ayaka Sato, Kazuya Shimura, and Fumiyo Fukumoto. 2020. Multi-task peer-review score prediction. In Proceedings of the First Workshop on Scholarly Document Processing, pages 121–126, Online. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Jiyi+Li%2C+Ayaka+Sato%2C+Kazuya+Shimura%2C+and+Fumiyo+Fukumoto%2E+2020%2E+Multi%2Dtask+peer%2Dreview+score+prediction%2E+In+Proceedings+of+the+First+Workshop+on+Scholarly+Document+Processing%2C+pages+121%E2%80%93126%2C+Online%2E+A)
33. Xiangci Li, Gully Burns, and Nanyun Peng. 2021. Scientific discourse tagging for evidence extraction. In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume, pages 2550–2562, Online. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Xiangci+Li%2C+Gully+Burns%2C+and+Nanyun+Peng%2E+2021%2E+Scientific+discourse+tagging+for+evidence+extraction%2E+In+Proceedings+of+the+16th+Conference+of+the+European+Chapter+of+the+Association+for+Computational)
34. Yuanfan Li, Zhaohan Zhang, Chengzhengxu Li, Chao Shen, and Xiaoming Liu. 2025. Iron sharpens iron: Defending against attacks in machine-generated text detection with adversarial training. Preprint, arXiv:2502.12734. [find](https://scholar.google.com/scholar?q=Yuanfan+Li%2C+Zhaohan+Zhang%2C+Chengzhengxu+Li%2C+Chao+Shen%2C+and+Xiaoming+Liu%2E+2025%2E+Iron+sharpens+iron%3A+Defending+against+attacks+in+machine%2Dgenerated+text+detection+with+adversarial+training%2E+Preprint%2C+ar)
35. Weixin Liang, Yuhui Zhang, Hancheng Cao, Binglu Wang, Daisy Yi Ding, Xinyu Yang, Kailas Vodrahalli, Siyu He, Daniel Scott Smith, Yian Yin, Daniel A. McFarland, and James Zou. 2024. [Can large language models provide useful feedback on research papers? a large-scale empirical analysis](https://md.research.elamur.ai/free/en/papers/liang-2023-large-language-models-provide/). NEJM AI, 1(8):A…
36. Chin-Yew Lin. 2004. [ROUGE: A package for automatic evaluation of summaries](https://md.research.elamur.ai/free/en/papers/lin-2004-rouge-package-automatic-evaluation-summaries/). In Text Summarization Branches Out, pages 74–81, Barcelona, Spain. Association for Computational Linguistics.
37. Ethan Lin, Zhiyuan Peng, and Yi Fang. 2025. Evaluating and enhancing large language models for novelty assessment in scholarly publications. In Proceedings of the 1st Workshop on AI and Scientific Discovery: Directions and Opportunities, pages 46–57, Albuquerque, New Mexico, USA. Association for Com… [find](https://scholar.google.com/scholar?q=Ethan+Lin%2C+Zhiyuan+Peng%2C+and+Yi+Fang%2E+2025%2E+Evaluating+and+enhancing+large+language+models+for+novelty+assessment+in+scholarly+publications%2E+In+Proceedings+of+the+1st+Workshop+on+AI+and+Scientific+Dis)
38. Jialiang Lin, Jiaxin Song, Zhangping Zhou, Yidong Chen, and Xiaodong Shi. 2023. Automated scholarly paper review: Concepts, technologies, and challenges. Information Fusion, 98:101830. [find](https://scholar.google.com/scholar?q=Jialiang+Lin%2C+Jiaxin+Song%2C+Zhangping+Zhou%2C+Yidong+Chen%2C+and+Xiaodong+Shi%2E+2023%2E+Automated+scholarly+paper+review%3A+Concepts%2C+technologies%2C+and+challenges%2E+Information+Fusion%2C+98%3A101830%2E)
39. Xiaoming Liu, Zhaohan Zhang, Yichen Wang, Hang Pu, Yu Lan, and Chao Shen. 2023. Coco: Coherence-enhanced machine-generated text detection under low resource with contrastive learning. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 16167–16188. [find](https://scholar.google.com/scholar?q=Xiaoming+Liu%2C+Zhaohan+Zhang%2C+Yichen+Wang%2C+Hang+Pu%2C+Yu+Lan%2C+and+Chao+Shen%2E+2023%2E+Coco%3A+Coherence%2Denhanced+machine%2Dgenerated+text+detection+under+low+resource+with+contrastive+learning%2E+In+Proceedings+o)
40. Yan Liu, Zonglin Yang, Soujanya Poria, Thanh-Son Nguyen, and Erik Cambria. 2025a. Harnessing large language models for scientific novelty detection. Preprint, arXiv:2505.24615. — [arXiv:2505.24615](https://arxiv.org/abs/2505.24615) [find](https://scholar.google.com/scholar?q=Yan+Liu%2C+Zonglin+Yang%2C+Soujanya+Poria%2C+Thanh%2DSon+Nguyen%2C+and+Erik+Cambria%2E+2025a%2E+Harnessing+large+language+models+for+scientific+novelty+detection%2E+Preprint%2C+arXiv%3A2505%2E24615%2E+%E2%80%94+%5BarXiv%3A2505%2E24615%5D%28ht)
41. Yan Liu, Zonglin Yang, Soujanya Poria, Thanh-Son Nguyen, and Erik Cambria. 2025b. Harnessing large language models for scientific novelty detection. Preprint, arXiv:2505.24615. — [arXiv:2505.24615](https://arxiv.org/abs/2505.24615) [find](https://scholar.google.com/scholar?q=Yan+Liu%2C+Zonglin+Yang%2C+Soujanya+Poria%2C+Thanh%2DSon+Nguyen%2C+and+Erik+Cambria%2E+2025b%2E+Harnessing+large+language+models+for+scientific+novelty+detection%2E+Preprint%2C+arXiv%3A2505%2E24615%2E+%E2%80%94+%5BarXiv%3A2505%2E24615%5D%28ht)
42. Annie Louis and Ani Nenkova. 2013. [Automatically assessing machine summary content without a gold standard](https://md.research.elamur.ai/free/en/papers/louis-2013-automatically-assessing-machine-summary-content/). Computational Linguistics, 39(2):267– 300.
43. Sheng Lu, Ilia Kuznetsov, and Iryna Gurevych. 2025. Identifying aspects in peer reviews. In Findings of the Association for Computational Linguistics: EMNLP 2025, pages 6145–6167, Suzhou, China. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Sheng+Lu%2C+Ilia+Kuznetsov%2C+and+Iryna+Gurevych%2E+2025%2E+Identifying+aspects+in+peer+reviews%2E+In+Findings+of+the+Association+for+Computational+Linguistics%3A+EMNLP+2025%2C+pages+6145%E2%80%936167%2C+Suzhou%2C+China%2E+Assoc)
44. Kuniko Matsumoto, Sotaro Shibayama, Byeongwoo Kang, and Masatsura Igami. 2021. Introducing a novelty indicator for scientific research: validating the knowledge-based combinatorial approach. Scientometrics, 126(8):6891–6915. [find](https://scholar.google.com/scholar?q=Kuniko+Matsumoto%2C+Sotaro+Shibayama%2C+Byeongwoo+Kang%2C+and+Masatsura+Igami%2E+2021%2E+Introducing+a+novelty+indicator+for+scientific+research%3A+validating+the+knowledge%2Dbased+combinatorial+approach%2E+Scientome)
45. OpenAI, :, Sandhini Agarwal, Lama Ahmad, Jason Ai, Sam Altman, Andy Applebaum, Edwin Arbus, Rahul K. Arora, Yu Bai, Bowen Baker, Haiming Bao, Boaz Barak, Ally Bennett, Tyler Bertao, Nivedita Brett, Eugene Brevdo, Greg Brockman, Sebastien Bubeck, and 108 others. 2025. [gpt-oss-120b & gpt-oss-20b model…](https://md.research.elamur.ai/free/en/papers/agarwal-2025-gpt-oss-120b-gpt-oss/)
46. OpenAI, :, Aaron Hurst, Adam Lerer, Adam P. Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Ostrow, Akila Welihinda, Alan Hayes, Alec Radford, Aleksander M adry, Alex Baker-Whitcomb, Alex Beutel, Alex Borzunov, Alex Carney, Alex Chow, Alex Kirillov, and 401 others. 2024. [Gpt-4o system card](https://md.research.elamur.ai/free/en/papers/openai-2024-gpt-4o-system-card/).…
47. OpenAI. 2025. Gpt-5 system card. [https://cdn.openai.com/gpt-5-system-card.pdf](https://cdn.openai.com/gpt-5-system-card.pdf). [↗](https://cdn.openai.com/gpt-5-system-card.pdf)
48. Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. [Bleu: a method for automatic evaluation of machine translation](https://md.research.elamur.ai/free/en/papers/papineni-2002-bleu-method-automatic-evaluation-machine/). In Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, pages 311–318, Philadelphia, Pennsylvania, USA. Association for Computational…
49. Qwen, :, An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, Huan Lin, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jingren Zhou, and 25 others. 2025. Qwen2.5 technical report. Preprint, arXiv:2412.15115. [find](https://scholar.google.com/scholar?q=Qwen%2C+%3A%2C+An+Yang%2C+Baosong+Yang%2C+Beichen+Zhang%2C+Binyuan+Hui%2C+Bo+Zheng%2C+Bowen+Yu%2C+Chengyuan+Li%2C+Dayiheng+Liu%2C+Fei+Huang%2C+Haoran+Wei%2C+Huan+Lin%2C+Jian+Yang%2C+Jianhong+Tu%2C+Jianwei+Zhang%2C+Jianxin+Yang%2C+Jiaxi+)
50. Nils Reimers and Iryna Gurevych. 2019. [Sentence-BERT: Sentence embeddings using Siamese BERT-networks](https://md.research.elamur.ai/free/en/papers/reimers-2019-sentence-bert-sentence-embeddings-siamese/). In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 3982–3992,…
51. Victor Sanh, Lysandre Debut, Julien Chaumond, and Thomas Wolf. 2020. [Distilbert, a distilled version of bert: smaller, faster, cheaper and lighter](https://md.research.elamur.ai/free/en/papers/sanh-2019-distilbert-distilled-version-bert-smaller/). Preprint, arXiv:1910.01108.
52. Simra Shahid, Marissa Radensky, Raymond Fok, Pao Siangliulue, Daniel S Weld, and Tom Hope. 2025. Literature-grounded novelty assessment of scientific ideas. In Proceedings of the Fifth Workshop on Scholarly Document Processing (SDP 2025), pages 96– 113, Vienna, Austria. Association for Computational… [find](https://scholar.google.com/scholar?q=Simra+Shahid%2C+Marissa+Radensky%2C+Raymond+Fok%2C+Pao+Siangliulue%2C+Daniel+S+Weld%2C+and+Tom+Hope%2E+2025%2E+Literature%2Dgrounded+novelty+assessment+of+scientific+ideas%2E+In+Proceedings+of+the+Fifth+Workshop+on+Sch)
53. Sotaro Shibayama, Zhao Wu, Deyun Yin, and Kazuki Yokota. 2025. State of the art of novelty indicators. Technical report, SSRN Electronic Journal. Available at SSRN: [https://ssrn.com/abstract=5379973](https://ssrn.com/abstract=5379973). [↗](https://ssrn.com/abstract=5379973)
54. Haoyang Su, Renqi Chen, Shixiang Tang, Zhenfei Yin, Xinzhe Zheng, Jinzhe Li, Biqing Qi, Qi Wu, Hui Li, Wanli Ouyang, Philip Torr, Bowen Zhou, and Nanqing Dong. 2025. [Many heads are better than one: Improved scientific idea generation by a LLM-based multi-agent system](https://md.research.elamur.ai/free/en/papers/su-2025-many-heads-are-better-than/). In Proceedings of the 63rd Annu…
55. Hongming Tan, Shaoxiong Zhan, Fengwei Jia, Hai-Tao Zheng, and Wai Kin (Victor) Chan. 2026. A hierarchical framework for measuring scientific paper innovation via large language models. Information Sciences, 728:122787. [find](https://scholar.google.com/scholar?q=Hongming+Tan%2C+Shaoxiong+Zhan%2C+Fengwei+Jia%2C+Hai%2DTao+Zheng%2C+and+Wai+Kin+%28Victor%29+Chan%2E+2026%2E+A+hierarchical+framework+for+measuring+scientific+paper+innovation+via+large+language+models%2E+Information+Sci)
56. Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, Dan Bikel, Lukas Blecher, Cristian Canton Ferrer, Moya Chen, Guillem Cucurull, David Esiobu, Jude Fernandes, Jeremy Fu, Wenyin Fu, and 49 others… — [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://md.research.elamur.ai/free/en/papers/touvron-2023-llama-2-open-foundation-fine/)
57. Brian Uzzi, Satyam Mukherjee, Michael Stringer, and Ben Jones. 2013. Atypical combinations and scientific impact. Science, 342(6157):468–472. [find](https://scholar.google.com/scholar?q=Brian+Uzzi%2C+Satyam+Mukherjee%2C+Michael+Stringer%2C+and+Ben+Jones%2E+2013%2E+Atypical+combinations+and+scientific+impact%2E+Science%2C+342%286157%29%3A468%E2%80%93472%2E)
58. Reinhilde Veugelers and Jian Wang. 2019. Scientific novelty and technological impact. Research Policy, 48(6):1362–1372. [find](https://scholar.google.com/scholar?q=Reinhilde+Veugelers+and+Jian+Wang%2E+2019%2E+Scientific+novelty+and+technological+impact%2E+Research+Policy%2C+48%286%29%3A1362%E2%80%931372%2E)
59. Qingyun Wang, Qi Zeng, Lifu Huang, Kevin Knight, Heng Ji, and Nazneen Fatema Rajani. 2020. ReviewRobot: Explainable paper review generation based on knowledge synthesis. In Proceedings of the 13th International Conference on Natural Language Generation, pages 384–397, Dublin, Ireland. Association fo… [find](https://scholar.google.com/scholar?q=Qingyun+Wang%2C+Qi+Zeng%2C+Lifu+Huang%2C+Kevin+Knight%2C+Heng+Ji%2C+and+Nazneen+Fatema+Rajani%2E+2020%2E+ReviewRobot%3A+Explainable+paper+review+generation+based+on+knowledge+synthesis%2E+In+Proceedings+of+the+13th+Int)
60. Yidong Wang, Qi Guo, Wenjin Yao, Hongbo Zhang, Xin Zhang, Zhen Wu, Meishan Zhang, Xinyu Dai, Min Zhang, Qingsong Wen, Wei Ye, Shikun Zhang, and Yue Zhang. 2024. [Autosurvey: Large language models can automatically write surveys](https://md.research.elamur.ai/free/en/papers/wang-2024-autosurvey-large-language-models-automatically/). In The Thirty-eighth Annual Conference on Neural Information Processing…
61. Yixuan Weng, Minjun Zhu, Guangsheng Bao, Hongbo Zhang, Jindong Wang, Yue Zhang, and Linyi Yang. 2025. [Cycleresearcher: Improving automated research via automated review](https://md.research.elamur.ai/free/en/papers/weng-2024-cycleresearcher-improving-automated-research/). In The Thirteenth International Conference on Learning Representations.
62. Dustin Wright, Jiaxin Pei, David Jurgens, and Isabelle Augenstein. 2022. Modeling information change in science communication with semantically matched paraphrases. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 1783–1807, Abu Dhabi, United Arab Emir… [find](https://scholar.google.com/scholar?q=Dustin+Wright%2C+Jiaxin+Pei%2C+David+Jurgens%2C+and+Isabelle+Augenstein%2E+2022%2E+Modeling+information+change+in+science+communication+with+semantically+matched+paraphrases%2E+In+Proceedings+of+the+2022+Conferen)
63. Wenqing Wu, Chengzhi Zhang, Tong Bao, and Yi Zhao. 2025a. Sc4anm: Identifying optimal section combinations for automated novelty prediction in academic papers. Expert Systems with Applications, 273:126778. [find](https://scholar.google.com/scholar?q=Wenqing+Wu%2C+Chengzhi+Zhang%2C+Tong+Bao%2C+and+Yi+Zhao%2E+2025a%2E+Sc4anm%3A+Identifying+optimal+section+combinations+for+automated+novelty+prediction+in+academic+papers%2E+Expert+Systems+with+Applications%2C+273%3A12)
64. Wenqing Wu, Chengzhi Zhang, and Yi Zhao. 2025b. Automated novelty evaluation of academic paper: A collaborative approach integrating human and large language model knowledge. Journal of the Association for Information Science and Technology, 76(11):1452–1469. [find](https://scholar.google.com/scholar?q=Wenqing+Wu%2C+Chengzhi+Zhang%2C+and+Yi+Zhao%2E+2025b%2E+Automated+novelty+evaluation+of+academic+paper%3A+A+collaborative+approach+integrating+human+and+large+language+model+knowledge%2E+Journal+of+the+Associatio)
65. Zhijian Xu, Yilun Zhao, Manasi Patwardhan, Lovekesh Vig, and Arman Cohan. 2025. [Can LLMs identify critical limitations within scientific research? a systematic evaluation on AI research papers](https://md.research.elamur.ai/free/en/papers/xu-2025-can-llms-identify-critical-limitations/). In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long…
66. An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, Chujie Zheng, Dayiheng Liu, Fan Zhou, Fei Huang, Feng Hu, Hao Ge, Haoran Wei, Huan Lin, Jialong Tang, and 41 others. 2025. [Qwen3 technical report](https://md.research.elamur.ai/free/en/papers/yang-2025-qwen3-technical-report/). arXiv preprint arXiv:2505.09388.
67. Heng Yu, Junfeng Kang, Rui Li, Qi Liu, Liyang He, Zhenya Huang, Shuanghong Shen, and Junyu Lu. 2025. CA-GAR: Context-aware alignment of LLM generation for document retrieval. In Findings of the Association for Computational Linguistics: ACL 2025, pages 5836–5849, Vienna, Austria. Association for Com… [find](https://scholar.google.com/scholar?q=Heng+Yu%2C+Junfeng+Kang%2C+Rui+Li%2C+Qi+Liu%2C+Liyang+He%2C+Zhenya+Huang%2C+Shuanghong+Shen%2C+and+Junyu+Lu%2E+2025%2E+CA%2DGAR%3A+Context%2Daware+alignment+of+LLM+generation+for+document+retrieval%2E+In+Findings+of+the+Associ)
68. Jianxiang Yu, Zichen Ding, Jiaqi Tan, Kangyang Luo, Zhenmin Weng, Chenghua Gong, Long Zeng, Ren-Jing Cui, Chengcheng Han, Qiushi Sun, Zhiyong Wu, Yunshi Lan, and Xiang Li. 2024a. Automated peer reviewing in paper SEA: Standardization, evaluation, and analysis. In Findings of the Association for Comp… [find](https://scholar.google.com/scholar?q=Jianxiang+Yu%2C+Zichen+Ding%2C+Jiaqi+Tan%2C+Kangyang+Luo%2C+Zhenmin+Weng%2C+Chenghua+Gong%2C+Long+Zeng%2C+Ren%2DJing+Cui%2C+Chengcheng+Han%2C+Qiushi+Sun%2C+Zhiyong+Wu%2C+Yunshi+Lan%2C+and+Xiang+Li%2E+2024a%2E+Automated+peer+review)
69. Jianxiang Yu, Zichen Ding, Jiaqi Tan, Kangyang Luo, Zhenmin Weng, Chenghua Gong, Long Zeng, Ren-Jing Cui, Chengcheng Han, Qiushi Sun, Zhiyong Wu, Yunshi Lan, and Xiang Li. 2024b. Automated peer reviewing in paper SEA: Standardization, evaluation, and analysis. In Findings of the Association for Comp… [find](https://scholar.google.com/scholar?q=Jianxiang+Yu%2C+Zichen+Ding%2C+Jiaqi+Tan%2C+Kangyang+Luo%2C+Zhenmin+Weng%2C+Chenghua+Gong%2C+Long+Zeng%2C+Ren%2DJing+Cui%2C+Chengcheng+Han%2C+Qiushi+Sun%2C+Zhiyong+Wu%2C+Yunshi+Lan%2C+and+Xiang+Li%2E+2024b%2E+Automated+peer+review)
70. Weizhe Yuan and Pengfei Liu. 2022. Kid-review: Knowledge-guided scientific review generation with oracle pre-training. Proceedings of the AAAI Conference on Artificial Intelligence, 36(10):11639–11647. [find](https://scholar.google.com/scholar?q=Weizhe+Yuan+and+Pengfei+Liu%2E+2022%2E+Kid%2Dreview%3A+Knowledge%2Dguided+scientific+review+generation+with+oracle+pre%2Dtraining%2E+Proceedings+of+the+AAAI+Conference+on+Artificial+Intelligence%2C+36%2810%29%3A11639%E2%80%9311647)
71. Weizhe Yuan, Pengfei Liu, and Graham Neubig. 2022. [Can we automate scientific reviewing?](https://md.research.elamur.ai/free/en/papers/yuan-2021-automate-scientific-reviewing/) Journal of Artificial Intelligence Research, 75:171–212.
72. Hongxin Zhang, Yanzhe Zhang, Ruiyi Zhang, and Diyi Yang. 2022. Robustness of demonstration-based learning under limited data scenario. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 1769–1782, Abu Dhabi, United Arab Emirates. Association for Computat… [find](https://scholar.google.com/scholar?q=Hongxin+Zhang%2C+Yanzhe+Zhang%2C+Ruiyi+Zhang%2C+and+Diyi+Yang%2E+2022%2E+Robustness+of+demonstration%2Dbased+learning+under+limited+data+scenario%2E+In+Proceedings+of+the+2022+Conference+on+Empirical+Methods+in+Nat)
73. Tianyi Zhang*, Varsha Kishore*, Felix Wu*, Kilian Q. Weinberger, and Yoav Artzi. 2020. Bertscore: Evaluating text generation with bert. In International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Tianyi+Zhang%2A%2C+Varsha+Kishore%2A%2C+Felix+Wu%2A%2C+Kilian+Q%2E+Weinberger%2C+and+Yoav+Artzi%2E+2020%2E+Bertscore%3A+Evaluating+text+generation+with+bert%2E+In+International+Conference+on+Learning+Representations%2E)
74. Zhaohan Zhang, Chengzhengxu Li, Xiaoming Liu, Chao Shen, Ziquan Liu, and Ioannis Patras. 2026. [Confidence should be calibrated more than one turn deep](https://md.research.elamur.ai/free/en/papers/zhang-2026-confidence-should-be-calibrated-more/). arXiv preprint arXiv:2604.05397.
75. Zhaohan Zhang, Ziquan Liu, and Ioannis Patras. 2025. Grace: A generative approach to better confidence elicitation in large language models. arXiv preprint arXiv:2509.09438. — [arXiv:2509.09438](https://arxiv.org/abs/2509.09438) [find](https://scholar.google.com/scholar?q=Zhaohan+Zhang%2C+Ziquan+Liu%2C+and+Ioannis+Patras%2E+2025%2E+Grace%3A+A+generative+approach+to+better+confidence+elicitation+in+large+language+models%2E+arXiv+preprint+arXiv%3A2509%2E09438%2E+%E2%80%94+%5BarXiv%3A2509%2E09438%5D%28https)
76. Yi Zhao and Chengzhi Zhang. 2025. A review on the novelty measurements of academic papers. Scientometrics, 130(2):727–753. [find](https://scholar.google.com/scholar?q=Yi+Zhao+and+Chengzhi+Zhang%2E+2025%2E+A+review+on+the+novelty+measurements+of+academic+papers%2E+Scientometrics%2C+130%282%29%3A727%E2%80%93753%2E)
77. Ruiyang Zhou, Lu Chen, and Kai Yu. 2024. [Is LLM a reliable reviewer? a comprehensive evaluation of LLM on automatic paper reviewing tasks](https://md.research.elamur.ai/free/en/papers/zhou-2024-is-llm-reliable-reviewer-comprehensive/). In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024), pages 9340– 9351, T…
78. Minjun Zhu, Yixuan Weng, Linyi Yang, and Yue Zhang. 2025a. DeepReview: Improving LLM-based paper review with human-like deep thinking process. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 29330–29355, Vienna, Austria. Assoc… [find](https://scholar.google.com/scholar?q=Minjun+Zhu%2C+Yixuan+Weng%2C+Linyi+Yang%2C+and+Yue+Zhang%2E+2025a%2E+DeepReview%3A+Improving+LLM%2Dbased+paper+review+with+human%2Dlike+deep+thinking+process%2E+In+Proceedings+of+the+63rd+Annual+Meeting+of+the+Associat)
79. Minjun Zhu, Yixuan Weng, Linyi Yang, and Yue Zhang. 2025b. DeepReview: Improving LLM-based paper review with human-like deep thinking process. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 29330–29355, Vienna, Austria. Assoc… [find](https://scholar.google.com/scholar?q=Minjun+Zhu%2C+Yixuan+Weng%2C+Linyi+Yang%2C+and+Yue+Zhang%2E+2025b%2E+DeepReview%3A+Improving+LLM%2Dbased+paper+review+with+human%2Dlike+deep+thinking+process%2E+In+Proceedings+of+the+63rd+Annual+Meeting+of+the+Associat)
80. Zhenzhen Zhuang, Jiandong Chen, Hongfeng Xu, Yuwen Jiang, and Jialiang Lin. 2025. Large language models for automated scholarly paper review: A survey. Information Fusion, 124:103332. [find](https://scholar.google.com/scholar?q=Zhenzhen+Zhuang%2C+Jiandong+Chen%2C+Hongfeng+Xu%2C+Yuwen+Jiang%2C+and+Jialiang+Lin%2E+2025%2E+Large+language+models+for+automated+scholarly+paper+review%3A+A+survey%2E+Information+Fusion%2C+124%3A103332%2E)

*Updated 2 November 2026*