--- title: "The AI Review Lottery — paper by Latona et al. (2024)" description: "Large language models may now be helping reviewers write their peer reviews, and many fear this… 0.14 points. Cited by 17 papers here." authors: ["Giuseppe Russo Latona", "Manoel Horta Ribeiro", "Tim R. Davidson", "Veniamin Veselovsky", "Robert West"] year: 2024 venue: "arXiv:2405.02150" source: "https://arxiv.org/abs/2405.02150" doi: "10.48550/arxiv.2405.02150" arxiv: "2405.02150" licence: "arXiv non-exclusive distribution license" licence_url: "http://arxiv.org/licenses/nonexclusive-distrib/1.0/" summary_licence: "Elamur Cosmic License" canonical: "https://research.elamur.ai/free/en/papers/latona-2024-ai-review-lottery-widespread/" --- [Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/latona-2024-ai-review-lottery-widespread/) [home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2024](https://md.research.elamur.ai/free/en/papers/2024/page/10/) / The AI Review Lottery: Widespread AI-Assisted Peer Reviews Boost Paper Scores and Acceptance Rates # The AI Review Lottery: Widespread AI-Assisted Peer Reviews Boost Paper Scores and Acceptance Rates **Authors:** Giuseppe Russo Latona, [Manoel Horta Ribeiro](https://md.research.elamur.ai/free/en/authors/manoel-horta-ribeiro/), Tim R. Davidson, [Veniamin Veselovsky](https://md.research.elamur.ai/free/en/authors/veniamin-veselovsky/) and [Robert West](https://md.research.elamur.ai/free/en/authors/robert-west/) · **Year:** 2024 · **Venue:** arXiv:2405.02150 · **Licence:** [arXiv non-exclusive distribution license](http://arxiv.org/licenses/nonexclusive-distrib/1.0/) **Links:** [Publisher's page](https://arxiv.org/abs/2405.02150) · [PDF](https://arxiv.org/pdf/2405.02150) · [DOI 10.48550/arxiv.2405.02150](https://doi.org/10.48550/arxiv.2405.02150) · [arXiv 2405.02150](https://arxiv.org/abs/2405.02150) · [OpenAlex](https://openalex.org/W4396784687) **Citations:** 12 citations (OpenAlex, 2026-10-14) · [17](https://md.research.elamur.ai/free/en/papers/latona-2024-ai-review-lottery-widespread/cited-by/) in our library, of the 149,439 papers whose bibliographies we read ## Summary Large language models may now be helping reviewers write their peer reviews, and many fear this could distort scientific evaluation. This paper asks how common such AI-assisted reviewing actually is and whether it changes outcomes. The authors collected all ICLR 2024 submissions and reviews through the OpenReview API. They flagged reviews as likely AI-assisted using the commercial GPTZero detector, and they estimated the detector's false-positive rate from pre-ChatGPT years (2018–2023). They then ran two quasi-experimental analyses. The first compares AI-assisted and human reviews written for the same paper. The second matches papers with similar topics, review counts and human scores, where only one paper in each pair got an AI-assisted review. According to the results, roughly one in six reviews involved LLM help. These reviews tended to give more generous scores and raised acceptance chances, especially for borderline papers. The authors argue that this measurable influence supports concerns about the reliability of and trust in peer review. Code and annotated data are released publicly. ## Key points - **Data:** 7,404 ICLR 2024 submissions and 28,028 reviews from OpenReview. A review was labelled AI-assisted when GPTZero gave it a human probability below 0.5, and the results were robust to this cutoff. - **Prevalence (Study 1):** At least 15.8% of reviews (4,428) were AI-assisted. This is a lower bound, because it is corrected for false positives but not for false negatives. Nearly half of the submissions (49.4%) received at least one flagged review. - **Scores (Study 2):** The analysis covered 3,357 papers and 9,666 review combinations. When an AI-assisted review and a human review of the same paper disagreed, the AI-assisted one was higher in 53.4% of cases (p = 0.002). That corresponds to a +14.4% relative difference, or about 0.14 points higher on average. - **Acceptance (Study 3):** In matched pairs, receiving an AI-assisted review raised acceptance by 3.1 percentage points overall. For borderline papers the increase was 4.9 percentage points (p = 0.024), which amounts to about 31.1% higher odds of acceptance. - **Resource:** Code and data are available at github.com/epfl-dlab/AIReviewLottery. *Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.* ## Institutions - [École Polytechnique Fédérale de Lausanne](https://md.research.elamur.ai/free/en/universities/ecole-polytechnique-federale-de-lausanne/) — University, Lausanne, Switzerland · 56 works · 8,189 citations 📚 Shelves (6): [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 17,959 works · [🤝 Social Sciences](https://md.research.elamur.ai/free/en/shelves/social-sciences/) · 2,009 works · [🤝 Safety Research](https://md.research.elamur.ai/free/en/shelves/social-sciences/safety-research/) · 665 works · [🎯 Decision Sciences](https://md.research.elamur.ai/free/en/shelves/decision-sciences/) · 306 works · [🎯 Information Systems and Management](https://md.research.elamur.ai/free/en/shelves/decision-sciences/information-systems-and-management/) · 96 works · [🎭 History and Philosophy of Science](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/history-and-philosophy-of-science/) · 83 works ## Cite Cite as: Giuseppe Russo Latona, Manoel Horta Ribeiro, Tim R. Davidson, Veniamin Veselovsky and Robert West. 2024. The AI Review Lottery: Widespread AI-Assisted Peer Reviews Boost Paper Scores and Acceptance Rates. arXiv:2405.02150. The paper itself, not this page. BibTeX from the library's record of the work. ```bibtex @misc{latona2024ai, title = {{The AI Review Lottery: Widespread AI-Assisted Peer Reviews Boost Paper Scores and Acceptance Rates}}, author = {Giuseppe Russo Latona, Manoel Horta Ribeiro, Tim R. Davidson, Veniamin Veselovsky and Robert West}, year = {2024}, doi = {10.48550/arxiv.2405.02150}, eprint = {2405.02150}, archivePrefix = {arXiv}, url = {https://doi.org/10.48550/arxiv.2405.02150} } ``` - [← previous NovelQA: Benchmarking Question Answering on Documents Exceeding 200K Tokens](https://md.research.elamur.ai/free/en/papers/wang-2024-novelqa-benchmarking-question-answering-documents/) - [next → TICKing All the Boxes: Generated Checklists Improve LLM Evaluation and Generation](https://md.research.elamur.ai/free/en/papers/cook-2024-ticking-all-boxes-generated-checklists/) - [↑ back to papers of 2024](https://md.research.elamur.ai/free/en/papers/2024/page/10/) *Updated 3 November 2026*