---
title: "Evaluation & benchmarks — papers, page 3 — Elamur Research"
description: "Evaluation & benchmarks: 684 papers of the Elamur Research library · page 3 of 8 · 126–225, the most cited first."
canonical: "https://research.elamur.ai/free/en/topics/evaluation-benchmarks/page/3/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/topics/evaluation-benchmarks/page/3/)

[home](https://md.research.elamur.ai/free/en/) / [topics](https://md.research.elamur.ai/free/en/topics/) / [Evaluation & benchmarks](https://md.research.elamur.ai/free/en/topics/evaluation-benchmarks/) / page 3

# 📏 Evaluation & benchmarks

684 papers · page 3 of 8 · 126–225 · the most cited first · [all topics](https://md.research.elamur.ai/free/en/topics/)

How systems are measured: benchmarks, metrics, human judgements and what the numbers mean.

## Papers (684)

[← prev](https://md.research.elamur.ai/free/en/topics/evaluation-benchmarks/page/2/) [1](https://md.research.elamur.ai/free/en/topics/evaluation-benchmarks/) [2](https://md.research.elamur.ai/free/en/topics/evaluation-benchmarks/page/2/) **3** [4](https://md.research.elamur.ai/free/en/topics/evaluation-benchmarks/page/4/) … [8](https://md.research.elamur.ai/free/en/topics/evaluation-benchmarks/page/8/) [next →](https://md.research.elamur.ai/free/en/topics/evaluation-benchmarks/page/4/)

126. **[Are We on the Right Way for Evaluating Large Vision-Language Models?](https://md.research.elamur.ai/free/en/papers/chen-2024-right-way-evaluating-large-vision/) [Conference on Neural Information Processing Systems (NeurIPS)](https://md.research.elamur.ai/free/en/venues/neural-information-processing-systems/)**Lin Chen, Jinsong Li, Xiaoyi Dong, et al.*2024 · 263 citations*
127. [**Results of WMT22 Metrics Shared Task: Stop Using BLEU – Neural Metrics Are Better and More Robust**Markus Freitag, Ricardo Rei, Nitika Mathur, et al.*2022 · 263 citations*](https://md.research.elamur.ai/free/en/papers/freitag-2022-results-wmt22-metrics-shared-task/)
128. [**BEIR: A Heterogenous Benchmark for Zero-shot Evaluation of Information Retrieval Models NeurIPS 2021 Datasets and Benchmarks**Nandan Thakur, Nils Reimers, Andreas Rücklé, et al.*2021 · 262 citations*](https://md.research.elamur.ai/free/en/papers/thakur-2021-beir-heterogenous-benchmark-zero/)
129. **[Findings of the VarDial Evaluation Campaign 2017](https://md.research.elamur.ai/free/en/papers/zampieri-2017-findings-vardial-evaluation-campaign/) [Workshop on NLP for Similar Languages, Varieties and Dialects (VarDial)](https://md.research.elamur.ai/free/en/venues/vardial/)**Marcos Zampieri, Shervin Malmasi, Nikola Ljubešić, et al.*2017 · 261 citations*
130. **[The Belebele Benchmark: a Parallel Reading Comprehension Dataset in 122 Language Variants](https://md.research.elamur.ai/free/en/papers/bandarkar-2023-belebele-benchmark-parallel-reading-comprehension/) [Annual Meeting of the Association for Computational Linguistics (ACL)](https://md.research.elamur.ai/free/en/venues/acl/)**Lucas Bandarkar, Davis Liang, Benjamin Muller, et al.*2023 · 259 citations*
131. **[Automatic Machine Translation Evaluation in Many Languages via Zero-Shot Paraphrasing](https://md.research.elamur.ai/free/en/papers/thompson-2020-automatic-machine-translation-evaluation-many/) [Conference on Empirical Methods in Natural Language Processing (EMNLP)](https://md.research.elamur.ai/free/en/venues/emnlp/)**Brian Thompson, Matt Post*2020 · 259 citations*
132. **[How Can We Accelerate Progress Towards Human-like Linguistic Generalization?](https://md.research.elamur.ai/free/en/papers/linzen-2020-how-can-we-accelerate-progress/) [Annual Meeting of the Association for Computational Linguistics (ACL)](https://md.research.elamur.ai/free/en/venues/acl/)**Tal Linzen*2020 · 258 citations*
133. **[Re-evaluating Evaluation in Text Summarization](https://md.research.elamur.ai/free/en/papers/bhandari-2020-re-evaluating-evaluation-text-summarization/) [Conference on Empirical Methods in Natural Language Processing (EMNLP)](https://md.research.elamur.ai/free/en/venues/emnlp/)**Manik Bhandari, Pranav Narayan Gour, Atabak Ashfaq, et al.*2020 · 258 citations*
134. [**MiniF2F: a cross-system benchmark for formal Olympiad-level mathematics arXiv:2109.00110**Kunhao Zheng, Jesse Michael Han, Stanislas Polu*2021 · 255 citations*](https://md.research.elamur.ai/free/en/papers/zheng-2021-minif2f-cross-system-benchmark-formal/)
135. **[BLEU is Not Suitable for the Evaluation of Text Simplification](https://md.research.elamur.ai/free/en/papers/sulem-2018-bleu-is-not-suitable-evaluation/) [Conference on Empirical Methods in Natural Language Processing (EMNLP)](https://md.research.elamur.ai/free/en/venues/emnlp/)**Elior Sulem, Omri Abend, Ari Rappoport*2018 · 255 citations*
136. [**LexGLUE: A Benchmark Dataset for Legal Language Understanding in English arXiv:2110.00976**Ilias Chalkidis; Abhik Jana; Dirk Hartung; Michael Bommarito; Ion Androutsopoulos; Daniel Martin Katz;, Nikolaos Aletras*2021 · 254 citations*](https://md.research.elamur.ai/free/en/papers/chalkidis-2021-lexglue-benchmark-dataset-legal-language/)
137. [**Evaluating Discourse Phenomena in Neural Machine Translation arXiv:1711.00513**Rachel Bawden, Rico Sennrich, Alexandra Birch, et al.*2017 · 252 citations*](https://md.research.elamur.ai/free/en/papers/bawden-2017-evaluating-discourse-phenomena-neural-machine/)
138. [**Syntactic Features for Evaluation of Machine Translation**Ding Liu, Daniel Gildea*2005 · 251 citations*](https://md.research.elamur.ai/free/en/papers/liu-2005-syntactic-features-evaluation-machine-translation/)
139. **[Overview of the EVALITA 2018 Hate Speech Detection Task](https://md.research.elamur.ai/free/en/papers/bosco-2018-overview-evalita-2018-hate-speech/) [Evaluation Campaign of Natural Language Processing and Speech Tools for Italian. Final Workshop](https://md.research.elamur.ai/free/en/venues/evalita/)**Cristina Bosco, Felice Dell'Orletta, Fabio Poletto, et al.*2018 · 248 citations*
140. **[Automatic Evaluation of Information Ordering: Kendall's Tau](https://md.research.elamur.ai/free/en/papers/lapata-2006-automatic-evaluation-information-ordering-kendall/) [Computational Linguistics](https://md.research.elamur.ai/free/en/venues/cl/)**Mirella Lapata*2006 · 246 citations*
141. **[NLP Evaluation in trouble: On the Need to Measure LLM Data Contamination for each Benchmark](https://md.research.elamur.ai/free/en/papers/sainz-2023-nlp-evaluation-trouble-need-measure/) [Findings of the Association for Computational Linguistics](https://md.research.elamur.ai/free/en/venues/findings/)**Oscar Sainz, Jon Campos, Iker García-Ferrero, et al.*2023 · 245 citations*
142. **[Overview of the SPMRL 2013 Shared Task: A Cross-Framework Evaluation of Parsing Morphologically Rich Languages](https://md.research.elamur.ai/free/en/papers/seddah-2013-overview-spmrl-2013-shared-task/) [Workshop on Statistical Parsing of Morphologically-Rich Languages](https://md.research.elamur.ai/free/en/venues/workshop-on-statistical-parsing-of-morphologically-rich-languages/)**Djamé Seddah, Reut Tsarfaty, Sandra Kübler, et al.*2013 · 244 citations*
143. [**CLUE: A Chinese Language Understanding Evaluation Benchmark arXiv:2004.05986**Liang Xu, Hai Hu, Xuanwei Zhang, et al.*2020 · 243 citations*](https://md.research.elamur.ai/free/en/papers/xu-2020-clue-chinese-language-understanding/)
144. **[Comparing Automatic and Human Evaluation of Local Explanations for Text Classification](https://md.research.elamur.ai/free/en/papers/nguyen-2018-comparing-automatic-human-evaluation-local/) [Conference of the North American Chapter of the Association for Computational Linguistics (NAACL)](https://md.research.elamur.ai/free/en/venues/naacl/)**Dong Nguyen*2018 · 240 citations*
145. **[MTOP: A Comprehensive Multilingual Task-Oriented Semantic Parsing Benchmark](https://md.research.elamur.ai/free/en/papers/li-2021-mtop-comprehensive-multilingual-task-oriented/) [Conference of the European Chapter of the Association for Computational Linguistics (EACL)](https://md.research.elamur.ai/free/en/venues/eacl/)**Haoran Li, Abhinav Arora, Shuohui Chen, et al.*2021 · 239 citations*
146. **[On LLMs-Driven Synthetic Data Generation, Curation, and Evaluation: A Survey](https://md.research.elamur.ai/free/en/papers/long-2024-llms-driven-synthetic-data-generation/) [Findings of the Association for Computational Linguistics](https://md.research.elamur.ai/free/en/venues/findings/)**Lin Long, Rui Wang, Ruixuan Xiao, et al.*2024 · 236 citations*
147. **[Evaluating Open-Domain Question Answering in the Era of Large Language Models](https://md.research.elamur.ai/free/en/papers/kamalloo-2023-evaluating-open-domain-question-answering/) [Annual Meeting of the Association for Computational Linguistics (ACL)](https://md.research.elamur.ai/free/en/venues/acl/)**Ehsan Kamalloo, Nouha Dziri, Charles Clarke, et al.*2023 · 233 citations*
148. **[Evaluating Automated and Manual Acquisition of Anaphora Resolution Strategies](https://md.research.elamur.ai/free/en/papers/aone-1995-evaluating-automated-manual-acquisition-anaphora/) [Annual Meeting of the Association for Computational Linguistics (ACL)](https://md.research.elamur.ai/free/en/venues/acl/)**Chinatsu Aone, Scott William Bennett*1995 · 233 citations*
149. **[The ARPA MT Evaluation Methodologies: Evolution, Lessons, and Future Approaches](https://md.research.elamur.ai/free/en/papers/white-1994-arpa-mt-evaluation-methodologies-evolution/) [Conference of the Association for Machine Translation in the Americas (AMTA)](https://md.research.elamur.ai/free/en/venues/amta/)**John S. White, Theresa A. O'Connell, Francis E. O'Mara*1994 · 229 citations*
150. [**XSTest: A Test Suite for Identifying Exaggerated Safety Behaviours in Large Language Models arXiv:2308.01263**Paul Rottger, Hannah Rose Kirk, Bertie Vidgen, et al.*2023 · 227 citations*](https://md.research.elamur.ai/free/en/papers/rottger-2023-xstest-test-suite-identifying-exaggerated/)
151. **[Findings of the Third Shared Task on Multimodal Machine Translation](https://md.research.elamur.ai/free/en/papers/barrault-2018-findings-third-shared-task-multimodal/) [Conference on Machine Translation: Shared Task Papers](https://md.research.elamur.ai/free/en/venues/conference-on-machine-translation-shared-task-papers/)**Loïc Barrault, Fethi Bougares, Lucia Specia, et al.*2018 · 227 citations*
152. [**Evaluating Layers of Representation in Neural Machine Translation on Part-of-Speech and Semantic Tagging Tasks arXiv:1801.07772 · IJCNLP 8 (2017), volume 1, 1-10**Yonatan Belinkov, Lluis Marquez, Hassan Sajjad, et al.*2017 · 225 citations*](https://md.research.elamur.ai/free/en/papers/belinkov-2018-evaluating-layers-representation-neural-machine/)
153. [**Overview of the TREC 2019 deep learning track arXiv:2003.07820**Nick Craswell, Bhaskar Mitra, Emine Yilmaz, et al.*2020 · 224 citations*](https://md.research.elamur.ai/free/en/papers/craswell-2020-overview-trec-2019-deep-learning/)
154. **[BLEU might be Guilty but References are not Innocent](https://md.research.elamur.ai/free/en/papers/freitag-2020-bleu-might-be-guilty-but/) [Conference on Empirical Methods in Natural Language Processing (EMNLP)](https://md.research.elamur.ai/free/en/venues/emnlp/)**Markus Freitag, David Grangier, Isaac Caswell*2020 · 223 citations*
155. **[Unsupervised Evaluation of Interactive Dialog with DialoGPT](https://md.research.elamur.ai/free/en/papers/mehri-2020-unsupervised-evaluation-interactive-dialog-dialogpt/) [Annual Meeting of the Special Interest Group on Discourse and Dialogue (SIGDIAL)](https://md.research.elamur.ai/free/en/venues/sigdial/)**Shikib Mehri, Maxine Eskenazi*2020 · 223 citations*
156. **[GLUECoS: An Evaluation Benchmark for Code-Switched NLP](https://md.research.elamur.ai/free/en/papers/khanuja-2020-gluecos-evaluation-benchmark-code-switched/) [Annual Meeting of the Association for Computational Linguistics (ACL)](https://md.research.elamur.ai/free/en/venues/acl/)**Simran Khanuja, Sandipan Dandapat, Anirudh Srinivasan, et al.*2020 · 222 citations*
157. **[Beyond BLEU: Training Neural Machine Translation with Semantic Similarity](https://md.research.elamur.ai/free/en/papers/wieting-2019-beyond-bleu-training-neural-machine/) [Annual Meeting of the Association for Computational Linguistics (ACL)](https://md.research.elamur.ai/free/en/venues/acl/)**John Wieting, Taylor Berg-Kirkpatrick, Kevin Gimpel, et al.*2019 · 221 citations*
158. **[Break It Down: A Question Understanding Benchmark](https://md.research.elamur.ai/free/en/papers/wolfson-2020-break-it-down-question-understanding/) [Transactions of the Association for Computational Linguistics (TACL)](https://md.research.elamur.ai/free/en/venues/tacl/)**Tomer Wolfson, Mor Geva, Ankit Gupta, et al.*2020 · 218 citations*
159. **[Automatically Evaluating Text Coherence Using Discourse Relations](https://md.research.elamur.ai/free/en/papers/lin-2011-automatically-evaluating-text-coherence-discourse/) [Annual Meeting of the Association for Computational Linguistics (ACL)](https://md.research.elamur.ai/free/en/venues/acl/)**Ziheng Lin, Hwee Tou Ng, Min-Yen Kan*2011 · 218 citations*
160. **[YiSi - a Unified Semantic MT Quality Evaluation and Estimation Metric for Languages with Different Levels of Available Resources](https://md.research.elamur.ai/free/en/papers/lo-2019-yisi-unified-semantic-mt-quality/) [Conference on Machine Translation (WMT)](https://md.research.elamur.ai/free/en/venues/wmt/)**Chi-kiu Lo*2019 · 217 citations*
161. **[Better Summarization Evaluation with Word Embeddings for ROUGE](https://md.research.elamur.ai/free/en/papers/ng-2015-better-summarization-evaluation-word-embeddings/) [Conference on Empirical Methods in Natural Language Processing (EMNLP)](https://md.research.elamur.ai/free/en/venues/emnlp/)**Jun-Ping Ng, Viktoria Abrecht*2015 · 217 citations*
162. [**Evaluating the Logical Reasoning Ability of ChatGPT and GPT-4 arXiv:2304.03439**Hanmeng Liu, Ruoxi Ning, Zhiyang Teng, et al.*2023 · 215 citations*](https://md.research.elamur.ai/free/en/papers/liu-2023-evaluating-logical-reasoning-ability-chatgpt/)
163. **[Data-Driven Sentence Simplification: Survey and Benchmark](https://md.research.elamur.ai/free/en/papers/manchego-2020-data-driven-sentence-simplification-survey/) [Computational Linguistics](https://md.research.elamur.ai/free/en/venues/cl/)**Fernando Alva-Manchego, Carolina Scarton, Lucia Specia*2020 · 213 citations*
164. **[Results of the WMT17 Metrics Shared Task](https://md.research.elamur.ai/free/en/papers/bojar-2017-results-wmt17-metrics-shared-task/) [Conference on Machine Translation (WMT)](https://md.research.elamur.ai/free/en/venues/wmt/)**Ondřej Bojar, Yvette Graham, Amir Kamran*2017 · 212 citations*
165. [**GAIA: a benchmark for General AI Assistants arXiv:2311.12983**Gregoire Mialon, Clementine Fourrier, Craig Swift, et al.*2023 · 211 citations*](https://md.research.elamur.ai/free/en/papers/mialon-2023-gaia-benchmark-general-ai-assistants/)
166. **[Answers Unite! Unsupervised Metrics for Reinforced Summarization Models](https://md.research.elamur.ai/free/en/papers/scialom-2019-answers-unite-unsupervised-metrics-reinforced/) [Conference on Empirical Methods in Natural Language Processing (EMNLP)](https://md.research.elamur.ai/free/en/venues/emnlp/)**Thomas Scialom, Sylvain Lamprier, Benjamin Piwowarski, et al.*2019 · 211 citations*
167. [**Which Coreference Evaluation Metric Do You Trust? A Proposal for a Link-based Entity Aware Metric**Nafise Sadat Moosavi, Michael Strube*2016 · 210 citations*](https://md.research.elamur.ai/free/en/papers/moosavi-2016-which-coreference-evaluation-metric-do/)
168. **[PlanBench: An Extensible Benchmark for Evaluating Large Language Models on Planning and Reasoning about Change](https://md.research.elamur.ai/free/en/papers/valmeekam-2022-planbench-extensible-benchmark-evaluating-large/) [Conference on Neural Information Processing Systems (NeurIPS)](https://md.research.elamur.ai/free/en/venues/neural-information-processing-systems/)**Karthik Valmeekam, Matthew Marquez, Alberto Olmo, et al.*2022 · 209 citations*
169. **[Re-evaluating Automatic Summarization with BLEU and 192 Shades of ROUGE](https://md.research.elamur.ai/free/en/papers/graham-2015-re-evaluating-automatic-summarization-bleu/) [Conference on Empirical Methods in Natural Language Processing (EMNLP)](https://md.research.elamur.ai/free/en/venues/emnlp/)**Yvette Graham*2015 · 209 citations*
170. **[(Meta-) Evaluation of Machine Translation](https://md.research.elamur.ai/free/en/papers/burch-2007-meta-evaluation-machine-translation/) [Conference on Machine Translation (WMT)](https://md.research.elamur.ai/free/en/venues/wmt/)**Chris Callison-Burch, Cameron Fordyce, Philipp Koehn, et al.*2007 · 209 citations*
171. **[Findings of the 2012 Workshop on Statistical Machine Translation](https://md.research.elamur.ai/free/en/papers/burch-2012-findings-2012-workshop-statistical-machine/) [Conference on Machine Translation (WMT)](https://md.research.elamur.ai/free/en/venues/wmt/)**Chris Callison-Burch, Philipp Koehn, Christof Monz, et al.*2012 · 204 citations*
172. **[Evaluating Factuality in Generation with Dependency-level Entailment](https://md.research.elamur.ai/free/en/papers/goyal-2020-evaluating-factuality-generation-dependency-level/) [Findings of the Association for Computational Linguistics](https://md.research.elamur.ai/free/en/venues/findings/)**Tanya Goyal, Greg Durrett*2020 · 203 citations*
173. **[A Systematic Study and Comprehensive Evaluation of ChatGPT on Benchmark Datasets](https://md.research.elamur.ai/free/en/papers/laskar-2023-systematic-study-comprehensive-evaluation-chatgpt/) [Findings of the Association for Computational Linguistics](https://md.research.elamur.ai/free/en/venues/findings/)**Md Tahmid Rahman Laskar, M Saiful Bari, Mizanur Rahman, et al.*2023 · 200 citations*
174. **[Results of the WMT20 Metrics Shared Task](https://md.research.elamur.ai/free/en/papers/mathur-2020-results-wmt20-metrics-shared-task/) [Conference on Machine Translation (WMT)](https://md.research.elamur.ai/free/en/venues/wmt/)**Nitika Mathur, Johnny Wei, Markus Freitag, et al.*2020 · 195 citations*
175. **[A Benchmark Dataset for Automatic Detection of Claims and Evidence in the Context of Controversial Topics](https://md.research.elamur.ai/free/en/papers/aharoni-2014-benchmark-dataset-automatic-detection-claims/) [Workshop on Argumentation Mining](https://md.research.elamur.ai/free/en/venues/workshop-on-argumentation-mining/)**Ehud Aharoni, Anatoly Polnarov, Tamar Lavee, et al.*2014 · 195 citations*
176. **[Report on the 11th IWSLT evaluation campaign](https://md.research.elamur.ai/free/en/papers/cettolo-2014-report-11th-iwslt-evaluation-campaign/) [International Conference on Spoken Language Translation (IWSLT)](https://md.research.elamur.ai/free/en/venues/iwslt/)**Mauro Cettolo, Jan Niehues, Sebastian Stüker, et al.*2014 · 195 citations*
177. [**PandaLM: An Automatic Evaluation Benchmark for LLM Instruction Tuning Optimization arXiv:2306.05087**Yidong Wang, Zhuohao Yu, Zhengran Zeng, et al.*2023 · 192 citations*](https://md.research.elamur.ai/free/en/papers/wang-2023-pandalm-automatic-evaluation-benchmark-llm/)
178. [**From Crowdsourced Data to High-Quality Benchmarks: Arena-Hard and BenchBuilder Pipeline arXiv:2406.11939**Tianle Li, Wei-Lin Chiang, Evan Frick, et al.*2024 · 191 citations*](https://md.research.elamur.ai/free/en/papers/li-2024-crowdsourced-data-high-quality-benchmarks/)
179. **[GEMBA-MQM: Detecting Translation Quality Error Spans with GPT-4](https://md.research.elamur.ai/free/en/papers/kocmi-2023-gemba-mqm-detecting-translation-quality/) [Conference on Machine Translation (WMT)](https://md.research.elamur.ai/free/en/venues/wmt/)**Tom Kocmi, Christian Federmann*2023 · 191 citations*
180. **[Collecting Diverse Natural Language Inference Problems for Sentence Representation Evaluation](https://md.research.elamur.ai/free/en/papers/poliak-2018-collecting-diverse-natural-language-inference/) [Conference on Empirical Methods in Natural Language Processing (EMNLP)](https://md.research.elamur.ai/free/en/venues/emnlp/)**Adam Poliak, Aparajita Haldar, Rachel Rudinger, et al.*2018 · 190 citations*
181. **[Findings of the 2021 Conference on Machine Translation (WMT21)](https://md.research.elamur.ai/free/en/papers/akhbardeh-2021-findings-2021-conference-machine-translation/) [Conference on Machine Translation (WMT)](https://md.research.elamur.ai/free/en/venues/wmt/)**Farhad Akhbardeh, Arkady Arkhangorodsky, Magdalena Biesialska, et al.*2021 · 189 citations*
182. **[EVALITA 2020: Overview of the 7th Evaluation Campaign of Natural Language Processing and Speech Tools for Italian](https://md.research.elamur.ai/free/en/papers/basile-2020-evalita-2020-overview-7th-evaluation/) [Evaluation Campaign of Natural Language Processing and Speech Tools for Italian. Final Workshop](https://md.research.elamur.ai/free/en/venues/evalita/)**Valerio Basile, Danilo Croce, Maria Di Maro, et al.*2020 · 188 citations*
183. [**We Need to Consider Disagreement in Evaluation Proceedings of the 1st Workshop on Benchmarking: Past, Present and Future**Valerio Basile, Michael Fell, Tommaso Fornaciari, et al.*2021 · 185 citations*](https://md.research.elamur.ai/free/en/papers/basile-2021-we-need-consider-disagreement-evaluation/)
184. **[GeoQA: A Geometric Question Answering Benchmark Towards Multimodal Numerical Reasoning](https://md.research.elamur.ai/free/en/papers/chen-2021-geoqa-geometric-question-answering-benchmark/) [Findings of the Association for Computational Linguistics](https://md.research.elamur.ai/free/en/venues/findings/)**Jiaqi Chen, Jianheng Tang, Jinghui Qin, et al.*2021 · 183 citations*
185. **[Overview of the MEDIQA 2019 Shared Task on Textual Inference, Question Entailment and Question Answering](https://md.research.elamur.ai/free/en/papers/abacha-2019-overview-mediqa-2019-shared-task/) [BioNLP Shared Task Workshop](https://md.research.elamur.ai/free/en/venues/bionlp-workshop-and-shared-task/)**Asma Ben Abacha, Chaitanya Shivade, Dina Demner-Fushman*2019 · 179 citations*
186. [**Evaluating Large Language Models: A Comprehensive Survey arXiv:2310.19736**Zishan Guo, Renren Jin, Chuang Liu, et al.*2023 · 178 citations*](https://md.research.elamur.ai/free/en/papers/guo-2023-evaluating-large-language-models-comprehensive/)
187. **[Findings of the WMT24 General Machine Translation Shared Task: The LLM Era Is Here but MT Is Not Solved Yet](https://md.research.elamur.ai/free/en/papers/kocmi-2024-findings-wmt24-general-machine-translation/) [Conference on Machine Translation (WMT)](https://md.research.elamur.ai/free/en/venues/wmt/)**Tom Kocmi, Eleftherios Avramidis, Rachel Bawden, et al.*2024 · 177 citations*
188. **[Findings of the IWSLT 2022 Evaluation Campaign](https://md.research.elamur.ai/free/en/papers/anastasopoulos-2022-findings-iwslt-2022-evaluation-campaign/) [International Conference on Spoken Language Translation (IWSLT)](https://md.research.elamur.ai/free/en/venues/iwslt/)**Antonios Anastasopoulos, Loïc Barrault, Luisa Bentivogli, et al.*2022 · 177 citations*
189. **[LILA: A Unified Benchmark for Mathematical Reasoning](https://md.research.elamur.ai/free/en/papers/mishra-2022-lila-unified-benchmark-mathematical-reasoning/) [Conference on Empirical Methods in Natural Language Processing (EMNLP)](https://md.research.elamur.ai/free/en/venues/emnlp/)**Swaroop Mishra, Matthew Finlayson, Pan Lu, et al.*2022 · 177 citations*
190. **[Results of the WMT16 Metrics Shared Task](https://md.research.elamur.ai/free/en/papers/bojar-2016-results-wmt16-metrics-shared-task/) [Conference on Machine Translation: Shared Task Papers](https://md.research.elamur.ai/free/en/venues/conference-on-machine-translation-shared-task-papers/)**Ondřej Bojar, Yvette Graham, Amir Kamran, et al.*2016 · 177 citations*
191. **[Results of the WMT14 Metrics Shared Task](https://md.research.elamur.ai/free/en/papers/machacek-2014-results-wmt14-metrics-shared-task/) [Conference on Machine Translation (WMT)](https://md.research.elamur.ai/free/en/venues/wmt/)**Matouš Macháček, Ondřej Bojar*2014 · 176 citations*
192. **[Towards Leaving No Indic Language Behind: Building Monolingual Corpora, Benchmark and Models for Indic Languages](https://md.research.elamur.ai/free/en/papers/doddapaneni-2023-leaving-no-indic-language-behind/) [Annual Meeting of the Association for Computational Linguistics (ACL)](https://md.research.elamur.ai/free/en/venues/acl/)**Sumanth Doddapaneni, Rahul Aralikatte, Gowtham Ramesh, et al.*2023 · 174 citations*
193. **[Findings of the WMT 2019 Shared Tasks on Quality Estimation](https://md.research.elamur.ai/free/en/papers/fonseca-2019-findings-wmt-2019-shared-tasks/) [Conference on Machine Translation (WMT)](https://md.research.elamur.ai/free/en/venues/wmt/)**Erick Fonseca, Lisa Yankovskaya, André F. T. Martins, et al.*2019 · 173 citations*
194. **[CDER: Efficient MT Evaluation Using Block Movements](https://md.research.elamur.ai/free/en/papers/leusch-2006-cder-efficient-mt-evaluation-block/) [Conference of the European Chapter of the Association for Computational Linguistics (EACL)](https://md.research.elamur.ai/free/en/venues/eacl/)**Gregor Leusch, Nicola Ueffing, Hermann Ney*2006 · 172 citations*
195. **[MT-Bench-101: A Fine-Grained Benchmark for Evaluating Large Language Models in Multi-Turn Dialogues](https://md.research.elamur.ai/free/en/papers/bai-2024-mt-bench-101-fine-grained/) [Annual Meeting of the Association for Computational Linguistics (ACL)](https://md.research.elamur.ai/free/en/venues/acl/)**Ge Bai, Jie Liu, Xingyuan Bu, et al.*2024 · 171 citations*
196. **[Leak, Cheat, Repeat: Data Contamination and Evaluation Malpractices in Closed-Source LLMs](https://md.research.elamur.ai/free/en/papers/balloccu-2024-leak-cheat-repeat-data-contamination/) [Conference of the European Chapter of the Association for Computational Linguistics (EACL)](https://md.research.elamur.ai/free/en/venues/eacl/)**Simone Balloccu, Patrícia Schmidtová, Mateusz Lango, et al.*2024 · 170 citations*
197. **[Findings of the Shared Task on Hope Speech Detection for Equality, Diversity, and Inclusion](https://md.research.elamur.ai/free/en/papers/chakravarthi-2021-findings-shared-task-hope-speech/) [Workshop on Language Technology for Equality, Diversity, Inclusion](https://md.research.elamur.ai/free/en/venues/ltedi/)**Bharathi Raja Chakravarthi, Vigneshwaran Muralidaran*2021 · 170 citations*
198. **[Results of the WMT21 Metrics Shared Task: Evaluating Metrics with Expert-based Human Evaluations on TED and News Domain](https://md.research.elamur.ai/free/en/papers/freitag-2021-results-wmt21-metrics-shared-task/) [Conference on Machine Translation (WMT)](https://md.research.elamur.ai/free/en/venues/wmt/)**Markus Freitag, Ricardo Rei, Nitika Mathur, et al.*2021 · 169 citations*
199. [**Is Reinforcement Learning (Not) for Natural Language Processing: Benchmarks, Baselines, and Building Blocks for Natural Language Policy Optimization arXiv:2210.01241**Rajkumar Ramamurthy, Prithviraj Ammanabrolu, Kiante Brantley, et al.*2022 · 168 citations*](https://md.research.elamur.ai/free/en/papers/ramamurthy-2022-reinforcement-learning-natural-language-processing/)
200. **[Results of the WMT18 Metrics Shared Task: Both characters and embeddings achieve good performance](https://md.research.elamur.ai/free/en/papers/ma-2018-results-wmt18-metrics-shared-task/) [Conference on Machine Translation: Shared Task Papers](https://md.research.elamur.ai/free/en/venues/conference-on-machine-translation-shared-task-papers/)**Qingsong Ma, Ondřej Bojar, Yvette Graham*2018 · 165 citations*
201. [**Overview of the DSL Shared Task 2015 LT4VarDial 2015**Marcos Zampieri, Liling Tan, Nikola Ljubešić, et al.*2015 · 164 citations*](https://md.research.elamur.ai/free/en/papers/zampieri-2015-overview-dsl-shared-task/)
202. **[On Evaluation of Adversarial Perturbations for Sequence-to-Sequence Models](https://md.research.elamur.ai/free/en/papers/michel-2019-evaluation-adversarial-perturbations-sequence-sequence/) [Conference of the North American Chapter of the Association for Computational Linguistics (NAACL)](https://md.research.elamur.ai/free/en/venues/naacl/)**Paul Michel, Xian Li, Graham Neubig, et al.*2019 · 162 citations*
203. **[Chart-to-Text: A Large-Scale Benchmark for Chart Summarization](https://md.research.elamur.ai/free/en/papers/kantharaj-2022-chart-text-large-scale-benchmark/) [Annual Meeting of the Association for Computational Linguistics (ACL)](https://md.research.elamur.ai/free/en/venues/acl/)**Shankar Kantharaj, Rixie Tiffany Leong, Xiang Lin, et al.*2022 · 161 citations*
204. **[Findings of the WMT 2020 Shared Task on Quality Estimation](https://md.research.elamur.ai/free/en/papers/specia-2020-findings-wmt-2020-shared-task-2020-wmt-1-79/) [Conference on Machine Translation (WMT)](https://md.research.elamur.ai/free/en/venues/wmt/)**Lucia Specia, Frédéric Blain, Marina Fomicheva, et al.*2020 · 160 citations*
205. [**Omni-MATH: A Universal Olympiad Level Mathematic Benchmark For Large Language Models arXiv:2410.07985**Bofei Gao, Feifan Song, Zhe Yang, et al.*2024 · 159 citations*](https://md.research.elamur.ai/free/en/papers/gao-2024-omni-math-universal-olympiad-level/)
206. **[Large Language Models Effectively Leverage Document-level Context for Literary Translation, but Critical Errors Persist](https://md.research.elamur.ai/free/en/papers/karpinska-2023-large-language-models-effectively-leverage/) [Conference on Machine Translation (WMT)](https://md.research.elamur.ai/free/en/venues/wmt/)**Marzena Karpinska, Mohit Iyyer*2023 · 156 citations*
207. **[The Devil Is in the Errors: Leveraging Large Language Models for Fine-grained Machine Translation Evaluation](https://md.research.elamur.ai/free/en/papers/fernandes-2023-devil-is-errors-leveraging-large/) [Conference on Machine Translation (WMT)](https://md.research.elamur.ai/free/en/venues/wmt/)**Patrick Fernandes, Daniel Deutsch, Mara Finkelstein, et al.*2023 · 156 citations*
208. **[The (Un)Suitability of Automatic Evaluation Metrics for Text Simplification](https://md.research.elamur.ai/free/en/papers/manchego-2021-un-suitability-automatic-evaluation-metrics/) [Computational Linguistics](https://md.research.elamur.ai/free/en/venues/cl/)**Fernando Alva-Manchego, Carolina Scarton, Lucia Specia*2021 · 156 citations*
209. **[Question Answering as an Automatic Evaluation Metric for News Article Summarization](https://md.research.elamur.ai/free/en/papers/eyal-2019-question-answering-as-automatic-evaluation/) [Conference of the North American Chapter of the Association for Computational Linguistics (NAACL)](https://md.research.elamur.ai/free/en/venues/naacl/)**Matan Eyal, Tal Baumel, Michael Elhadad*2019 · 156 citations*
210. **[Accurate Evaluation of Segment-level Machine Translation Metrics](https://md.research.elamur.ai/free/en/papers/graham-2015-accurate-evaluation-segment-level-machine/) [Conference of the North American Chapter of the Association for Computational Linguistics (NAACL)](https://md.research.elamur.ai/free/en/venues/naacl/)**Yvette Graham, Timothy Baldwin, Nitika Mathur*2015 · 156 citations*
211. **[Results of WMT23 Metrics Shared Task: Metrics Might Be Guilty but References Are Not Innocent](https://md.research.elamur.ai/free/en/papers/freitag-2023-results-wmt23-metrics-shared-task/) [Conference on Machine Translation (WMT)](https://md.research.elamur.ai/free/en/venues/wmt/)**Markus Freitag, Nitika Mathur, Chi-kiu Lo, et al.*2023 · 155 citations*
212. **[Evaluating Style Transfer for Text](https://md.research.elamur.ai/free/en/papers/mir-2019-evaluating-style-transfer-text/) [Conference of the North American Chapter of the Association for Computational Linguistics (NAACL)](https://md.research.elamur.ai/free/en/venues/naacl/)**Remi Mir, Bjarke Felbo, Nick Obradovich, et al.*2019 · 155 citations*
213. **[LLM-based NLG Evaluation: Current Status and Challenges](https://md.research.elamur.ai/free/en/papers/gao-2025-llm-based-nlg-evaluation-current/) [Computational Linguistics](https://md.research.elamur.ai/free/en/venues/cl/)**Mingqi Gao, Xinyu Hu, Xunjian Yin, et al.*2025 · 152 citations*
214. **[FINDINGS OF THE IWSLT 2023 EVALUATION CAMPAIGN](https://md.research.elamur.ai/free/en/papers/agarwal-2023-findings-iwslt-2023-evaluation-campaign/) [International Conference on Spoken Language Translation (IWSLT)](https://md.research.elamur.ai/free/en/venues/iwslt/)**Milind Agarwal, Sweta Agrawal, Antonios Anastasopoulos, et al.*2023 · 152 citations*
215. [**A Perspective on Word Sense Disambiguation Methods and Their Evaluation Tagging Text with Lexical Semantics: Why, What, and How?**Philip Resnik, David Yarowsky*1997 · 151 citations*](https://md.research.elamur.ai/free/en/papers/resnik-1997-perspective-word-sense-disambiguation-methods/)
216. **[Statistical Power and Translationese in Machine Translation Evaluation](https://md.research.elamur.ai/free/en/papers/graham-2020-statistical-power-translationese-machine-translation/) [Conference on Empirical Methods in Natural Language Processing (EMNLP)](https://md.research.elamur.ai/free/en/venues/emnlp/)**Yvette Graham, Barry Haddow, Philipp Koehn*2020 · 150 citations*
217. **[Overview of the IWSLT 2017 Evaluation Campaign](https://md.research.elamur.ai/free/en/papers/cettolo-2017-overview-iwslt-2017-evaluation-campaign/) [International Conference on Spoken Language Translation (IWSLT)](https://md.research.elamur.ai/free/en/venues/iwslt/)**Mauro Cettolo, Marcello Federico, Luisa Bentivogli, et al.*2017 · 150 citations*
218. **[Towards Question-Answering as an Automatic Metric for Evaluating the Content Quality of a Summary](https://md.research.elamur.ai/free/en/papers/deutsch-2021-question-answering-as-automatic-metric/) [Transactions of the Association for Computational Linguistics (TACL)](https://md.research.elamur.ai/free/en/venues/tacl/)**Daniel Deutsch, Tania Bedrax-Weiss, Dan Roth*2021 · 148 citations*
219. **[BLEnD: A Benchmark for LLMs on Everyday Knowledge in Diverse Cultures and Languages](https://md.research.elamur.ai/free/en/papers/myung-2024-blend-benchmark-llms-everyday-knowledge/) [Conference on Neural Information Processing Systems (NeurIPS)](https://md.research.elamur.ai/free/en/venues/neural-information-processing-systems/)**Junho Myung, Nayeon Lee, Yi Zhou, et al.*2024 · 147 citations*
220. **[How Grammatical is Character-level Neural Machine Translation? Assessing MT Quality with Contrastive Translation Pairs](https://md.research.elamur.ai/free/en/papers/sennrich-2017-how-grammatical-is-character-level/) [Conference of the European Chapter of the Association for Computational Linguistics (EACL)](https://md.research.elamur.ai/free/en/venues/eacl/)**Rico Sennrich*2017 · 147 citations*
221. **[Evaluating Machine Translation Output with Automatic Sentence Segmentation](https://md.research.elamur.ai/free/en/papers/matusov-2005-evaluating-machine-translation-output-automatic/) [International Conference on Spoken Language Translation (IWSLT)](https://md.research.elamur.ai/free/en/venues/iwslt/)**Evgeny Matusov, Gregor Leusch, Oliver Bender, et al.*2005 · 147 citations*
222. **[Evaluating Correctness and Faithfulness of Instruction-Following Models for Question Answering](https://md.research.elamur.ai/free/en/papers/adlakha-2024-evaluating-correctness-faithfulness-instruction-following/) [Transactions of the Association for Computational Linguistics (TACL)](https://md.research.elamur.ai/free/en/venues/tacl/)**Vaibhav Adlakha, Parishad BehnamGhader, Xing Han Lu, et al.*2024 · 145 citations*
223. **[Self-Evaluation Guided Beam Search for Reasoning](https://md.research.elamur.ai/free/en/papers/xie-2023-self-evaluation-guided-beam-search/) [Conference on Neural Information Processing Systems (NeurIPS)](https://md.research.elamur.ai/free/en/venues/neural-information-processing-systems/)**Yuxi Xie, Kenji Kawaguchi, Yiran Zhao, et al.*2023 · 145 citations*
224. **[AfriSenti: A Twitter Sentiment Analysis Benchmark for African Languages](https://md.research.elamur.ai/free/en/papers/muhammad-2023-afrisenti-twitter-sentiment-analysis-benchmark/) [Conference on Empirical Methods in Natural Language Processing (EMNLP)](https://md.research.elamur.ai/free/en/venues/emnlp/)**Shamsuddeen Hassan Muhammad, Idris Abdulmumin, Abinew Ali Ayele, et al.*2023 · 143 citations*
225. **[ORANGE: a Method for Evaluating Automatic Evaluation Metrics for Machine Translation](https://md.research.elamur.ai/free/en/papers/lin-2004-orange-method-evaluating-automatic-evaluation/) [International Conference on Computational Linguistics (COLING)](https://md.research.elamur.ai/free/en/venues/coling/)**Chin-Yew Lin, Franz Josef Och*2004 · 143 citations*

[← prev](https://md.research.elamur.ai/free/en/topics/evaluation-benchmarks/page/2/) [1](https://md.research.elamur.ai/free/en/topics/evaluation-benchmarks/) [2](https://md.research.elamur.ai/free/en/topics/evaluation-benchmarks/page/2/) **3** [4](https://md.research.elamur.ai/free/en/topics/evaluation-benchmarks/page/4/) … [8](https://md.research.elamur.ai/free/en/topics/evaluation-benchmarks/page/8/) [next →](https://md.research.elamur.ai/free/en/topics/evaluation-benchmarks/page/4/)