---
title: "Evaluation & benchmarks — papers, page 4 — Elamur Research"
description: "Evaluation & benchmarks: 684 papers of the Elamur Research library · page 4 of 8 · 226–325, the most cited first."
canonical: "https://research.elamur.ai/free/en/topics/evaluation-benchmarks/page/4/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/topics/evaluation-benchmarks/page/4/)

[home](https://md.research.elamur.ai/free/en/) / [topics](https://md.research.elamur.ai/free/en/topics/) / [Evaluation & benchmarks](https://md.research.elamur.ai/free/en/topics/evaluation-benchmarks/) / page 4

# 📏 Evaluation & benchmarks

684 papers · page 4 of 8 · 226–325 · the most cited first · [all topics](https://md.research.elamur.ai/free/en/topics/)

How systems are measured: benchmarks, metrics, human judgements and what the numbers mean.

## Papers (684)

[← prev](https://md.research.elamur.ai/free/en/topics/evaluation-benchmarks/page/3/) [1](https://md.research.elamur.ai/free/en/topics/evaluation-benchmarks/) [2](https://md.research.elamur.ai/free/en/topics/evaluation-benchmarks/page/2/) [3](https://md.research.elamur.ai/free/en/topics/evaluation-benchmarks/page/3/) **4** [5](https://md.research.elamur.ai/free/en/topics/evaluation-benchmarks/page/5/) … [8](https://md.research.elamur.ai/free/en/topics/evaluation-benchmarks/page/8/) [next →](https://md.research.elamur.ai/free/en/topics/evaluation-benchmarks/page/5/)

226. **[Are Large Language Models Really Good Logical Reasoners? A Comprehensive Evaluation and Beyond](https://md.research.elamur.ai/free/en/papers/xu-2023-large-language-models-really-good/) [IEEE Transactions on Knowledge and Data Engineering](https://md.research.elamur.ai/free/en/venues/ieee-transactions-on-knowledge-and-data-engineering/)**Fangzhi Xu, Qika Lin, Jiawei Han, et al.*2023 · 141 citations*
227. **[RUSE: Regressor Using Sentence Embeddings for Automatic Machine Translation Evaluation](https://md.research.elamur.ai/free/en/papers/shimanaka-2018-ruse-regressor-sentence-embeddings-automatic/) [Conference on Machine Translation: Shared Task Papers](https://md.research.elamur.ai/free/en/venues/conference-on-machine-translation-shared-task-papers/)**Hiroki Shimanaka, Tomoyuki Kajiwara, Mamoru Komachi*2018 · 141 citations*
228. **[A Token-level Reference-free Hallucination Detection Benchmark for Free-form Text Generation](https://md.research.elamur.ai/free/en/papers/liu-2022-token-level-reference-free-hallucination/) [Annual Meeting of the Association for Computational Linguistics (ACL)](https://md.research.elamur.ai/free/en/venues/acl/)**Tianyu Liu, Yizhe Zhang, Chris Brockett, et al.*2022 · 140 citations*
229. **[TURINGBENCH: A Benchmark Environment for Turing Test in the Age of Neural Text Generation](https://md.research.elamur.ai/free/en/papers/uchendu-2021-turingbench-benchmark-environment-turing-test/) [Findings of the Association for Computational Linguistics](https://md.research.elamur.ai/free/en/venues/findings/)**Adaku Uchendu, Zeyu Ma, Thai Le, et al.*2021 · 140 citations*
230. **[On The Evaluation of Machine Translation Systems Trained With Back-Translation](https://md.research.elamur.ai/free/en/papers/edunov-2020-evaluation-machine-translation-systems-trained/) [Annual Meeting of the Association for Computational Linguistics (ACL)](https://md.research.elamur.ai/free/en/venues/acl/)**Sergey Edunov, Myle Ott, Marc'Aurelio Ranzato, et al.*2020 · 140 citations*
231. **[FaithDial: A Faithful Benchmark for Information-Seeking Dialogue](https://md.research.elamur.ai/free/en/papers/dziri-2022-faithdial-faithful-benchmark-information-seeking/) [Transactions of the Association for Computational Linguistics (TACL)](https://md.research.elamur.ai/free/en/venues/tacl/)**Nouha Dziri, Ehsan Kamalloo, Sivan Milton, et al.*2022 · 138 citations*
232. **[Investigating Data Contamination in Modern Benchmarks for Large Language Models](https://md.research.elamur.ai/free/en/papers/deng-2024-investigating-data-contamination-modern-benchmarks/) [Conference of the North American Chapter of the Association for Computational Linguistics (NAACL)](https://md.research.elamur.ai/free/en/venues/naacl/)**Chunyuan Deng, Yilun Zhao, Xiangru Tang, et al.*2024 · 137 citations*
233. [**LLMs instead of Human Judges? A Large Scale Empirical Study across 20 NLP Evaluation Tasks arXiv:2406.18403**Anna Bavaresco, Raffaella Bernardi, Leonardo Bertolazzi, et al.*2024 · 137 citations*](https://md.research.elamur.ai/free/en/papers/bavaresco-2024-llms-instead-human-judges-large/)
234. **[Findings of the 2009 Workshop on Statistical Machine Translation](https://md.research.elamur.ai/free/en/papers/burch-2009-findings-2009-workshop-statistical-machine/) [Conference on Machine Translation (WMT)](https://md.research.elamur.ai/free/en/venues/wmt/)**Chris Callison-Burch, Philipp Koehn, Christof Monz, et al.*2009 · 137 citations*
235. **[Findings of the 2011 Workshop on Statistical Machine Translation](https://md.research.elamur.ai/free/en/papers/burch-2011-findings-2011-workshop-statistical-machine/) [Conference on Machine Translation (WMT)](https://md.research.elamur.ai/free/en/venues/wmt/)**Chris Callison-Burch, Philipp Koehn, Christof Monz, et al.*2011 · 136 citations*
236. **[Error Analysis Prompting Enables Human-Like Translation Evaluation in Large Language Models](https://md.research.elamur.ai/free/en/papers/lu-2024-error-analysis-prompting-enables-human/) [Findings of the Association for Computational Linguistics](https://md.research.elamur.ai/free/en/venues/findings/)**Qingyu Lu, Baopu Qiu, Liang Ding, et al.*2024 · 135 citations*
237. [**Semantic Structural Evaluation for Text Simplification**Elior Sulem, Omri Abend, Ari Rappoport*2018 · 135 citations*](https://md.research.elamur.ai/free/en/papers/sulem-2018-semantic-structural-evaluation-text-simplification/)
238. **[Political Compass or Spinning Arrow? Towards More Meaningful Evaluations for Values and Opinions in Large Language Models](https://md.research.elamur.ai/free/en/papers/rottger-2024-political-compass-or-spinning-arrow/) [Annual Meeting of the Association for Computational Linguistics (ACL)](https://md.research.elamur.ai/free/en/venues/acl/)**Paul Röttger, Valentin Hofmann, Valentina Pyatkin, et al.*2024 · 134 citations*
239. [**Recent Advances in Natural Language Inference: A Survey of Benchmarks, Resources, and Approaches arXiv:1904.01172**Shane Storks, Qiaozi Gao, Joyce Y. Chai*2019 · 134 citations*](https://md.research.elamur.ai/free/en/papers/storks-2019-recent-advances-natural-language-inference/)
240. **[Findings of the WMT 2018 Shared Task on Quality Estimation](https://md.research.elamur.ai/free/en/papers/specia-2018-findings-wmt-2018-shared-task/) [Conference on Machine Translation: Shared Task Papers](https://md.research.elamur.ai/free/en/venues/conference-on-machine-translation-shared-task-papers/)**Lucia Specia, Frédéric Blain, Varvara Logacheva, et al.*2018 · 134 citations*
241. **[Findings of the Second Shared Task on Multimodal Machine Translation and Multilingual Image Description](https://md.research.elamur.ai/free/en/papers/elliott-2017-findings-second-shared-task-multimodal/) [Conference on Machine Translation (WMT)](https://md.research.elamur.ai/free/en/venues/wmt/)**Desmond Elliott, Stella Frank, Loïc Barrault, et al.*2017 · 134 citations*
242. **[Learning Compact Metrics for MT](https://md.research.elamur.ai/free/en/papers/pu-2021-learning-compact-metrics-mt/) [Conference on Empirical Methods in Natural Language Processing (EMNLP)](https://md.research.elamur.ai/free/en/venues/emnlp/)**Amy Pu, Hyung Won Chung, Ankur Parikh, et al.*2021 · 130 citations*
243. [**An Analysis of Natural Language Inference Benchmarks through the Lens of Negation**Md Mosharaf Hossain, Venelin Kovatchev, Pranoy Dutta, et al.*2020 · 130 citations*](https://md.research.elamur.ai/free/en/papers/hossain-2020-analysis-natural-language-inference-benchmarks/)
244. **[Findings of the WMT 2023 Shared Task on Quality Estimation](https://md.research.elamur.ai/free/en/papers/blain-2023-findings-wmt-2023-shared-task/) [Conference on Machine Translation (WMT)](https://md.research.elamur.ai/free/en/venues/wmt/)**Frederic Blain, Chrysoula Zerva, Ricardo Rei, et al.*2023 · 129 citations*
245. **[ZeroSCROLLS: A Zero-Shot Benchmark for Long Text Understanding](https://md.research.elamur.ai/free/en/papers/shaham-2023-zeroscrolls-zero-shot-benchmark-long/) [Findings of the Association for Computational Linguistics](https://md.research.elamur.ai/free/en/venues/findings/)**Uri Shaham, Maor Ivgi, Avia Efrat, et al.*2023 · 129 citations*
246. **[The Text Anonymization Benchmark (TAB): A Dedicated Corpus and Evaluation Framework for Text Anonymization](https://md.research.elamur.ai/free/en/papers/pilan-2022-text-anonymization-benchmark-tab-dedicated/) [Computational Linguistics](https://md.research.elamur.ai/free/en/venues/cl/)**Ildikó Pilán, Pierre Lison, Lilja Øvrelid, et al.*2022 · 129 citations*
247. **[GO FIGURE: A Meta Evaluation of Factuality in Summarization](https://md.research.elamur.ai/free/en/papers/gabriel-2021-go-figure-meta-evaluation-factuality/) [Findings of the Association for Computational Linguistics](https://md.research.elamur.ai/free/en/venues/findings/)**Saadia Gabriel, Asli Celikyilmaz, Rahul Jha, et al.*2021 · 129 citations*
248. **[Swiss-Judgment-Prediction: A Multilingual Legal Judgment Prediction Benchmark](https://md.research.elamur.ai/free/en/papers/niklaus-2021-swiss-judgment-prediction-multilingual-legal/) [Natural Legal Language Processing Workshop](https://md.research.elamur.ai/free/en/venues/nllp/)**Joel Niklaus, Ilias Chalkidis, Matthias Stürmer*2021 · 129 citations*
249. [**Task-oriented Evaluation of Syntactic Parsers and Their Representations**Yusuke Miyao, Rune Sætre, Kenji Sagae, et al.*2008 · 129 citations*](https://md.research.elamur.ai/free/en/papers/miyao-2008-task-oriented-evaluation-syntactic-parsers/)
250. **[Prompting and Evaluating Large Language Models for Proactive Dialogues: Clarification, Target-guided, and Non-collaboration](https://md.research.elamur.ai/free/en/papers/deng-2023-prompting-evaluating-large-language-models/) [Findings of the Association for Computational Linguistics](https://md.research.elamur.ai/free/en/venues/findings/)**Yang Deng, Lizi Liao, Liang Chen, et al.*2023 · 128 citations*
251. [**A Comprehensive Assessment of Dialog Evaluation Metrics The First Workshop on Evaluations and Assessments of Neural Conversation Systems**Yi-Ting Yeh, Maxine Eskenazi, Shikib Mehri*2021 · 128 citations*](https://md.research.elamur.ai/free/en/papers/yeh-2021-comprehensive-assessment-dialog-evaluation-metrics/)
252. [**A comprehensive evaluation of ChatGPT's zero-shot Text-to-SQL capability arXiv:2303.13547**Aiwei Liu, Xuming Hu, Lijie Wen, et al.*2023 · 127 citations*](https://md.research.elamur.ai/free/en/papers/liu-2023-comprehensive-evaluation-chatgpt-s-zero/)
253. **[A Corpus-Based Evaluation of Centering and Pronoun Resolution](https://md.research.elamur.ai/free/en/papers/tetreault-2001-corpus-based-evaluation-centering-pronoun/) [Computational Linguistics](https://md.research.elamur.ai/free/en/venues/cl/)**Joel R. Tetreault*2001 · 125 citations*
254. [**τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domains arXiv:2406.12045**Shunyu Yao, Noah Shinn, Pedram Razavi, et al.*2024 · 123 citations*](https://md.research.elamur.ai/free/en/papers/yao-2024-tau-bench-benchmark-tool/)
255. **[LEXTREME: A Multi-Lingual and Multi-Task Benchmark for the Legal Domain](https://md.research.elamur.ai/free/en/papers/niklaus-2023-lextreme-multi-lingual-multi-task/) [Findings of the Association for Computational Linguistics](https://md.research.elamur.ai/free/en/venues/findings/)**Joel Niklaus, Veton Matoshi, Pooja Rani, et al.*2023 · 123 citations*
256. **[A Large-Scale Test Set for the Evaluation of Context-Aware Pronoun Translation in Neural Machine Translation](https://md.research.elamur.ai/free/en/papers/muller-2018-large-scale-test-set-evaluation/) [Conference on Machine Translation: Research Papers](https://md.research.elamur.ai/free/en/venues/conference-on-machine-translation-research-papers/)**Mathias Müller, Annette Rios, Elena Voita, et al.*2018 · 123 citations*
257. [**EQUATE: A Benchmark Evaluation Framework for Quantitative Reasoning in Natural Language Inference**Abhilasha Ravichander, Aakanksha Naik, Carolyn Rose, et al.*2019 · 122 citations*](https://md.research.elamur.ai/free/en/papers/ravichander-2019-equate-benchmark-evaluation-framework-quantitative/)
258. [**Generative Judge for Evaluating Alignment arXiv:2310.05470**Junlong Li, Shichao Sun, Weizhe Yuan, et al.*2023 · 121 citations*](https://md.research.elamur.ai/free/en/papers/li-2023-generative-judge-evaluating-alignment/)
259. **[The Moral Integrity Corpus: A Benchmark for Ethical Dialogue Systems](https://md.research.elamur.ai/free/en/papers/ziems-2022-moral-integrity-corpus-benchmark-ethical/) [Annual Meeting of the Association for Computational Linguistics (ACL)](https://md.research.elamur.ai/free/en/venues/acl/)**Caleb Ziems, Jane A. Yu, Yi-Chia Wang, et al.*2022 · 121 citations*
260. **[Automated evaluation of written discourse coherence using GPT-4](https://md.research.elamur.ai/free/en/papers/naismith-2023-automated-evaluation-written-discourse-coherence/) [Workshop on Innovative Use of NLP for Building Educational Applications (BEA)](https://md.research.elamur.ai/free/en/venues/bea/)**Ben Naismith, Phoebe Mulcaire, Jill Burstein*2023 · 120 citations*
261. **[Overview of the WANLP 2022 Shared Task on Propaganda Detection in Arabic](https://md.research.elamur.ai/free/en/papers/alam-2022-overview-wanlp-2022-shared-task/) [Arabic Natural Language Processing Workshop (WANLP)](https://md.research.elamur.ai/free/en/venues/wanlp/)**Firoj Alam, Hamdy Mubarak, Wajdi Zaghouani, et al.*2022 · 119 citations*
262. **[X-Fact: A New Benchmark Dataset for Multilingual Fact Checking](https://md.research.elamur.ai/free/en/papers/gupta-2021-x-fact-new-benchmark-dataset/) [Annual Meeting of the Association for Computational Linguistics (ACL)](https://md.research.elamur.ai/free/en/venues/acl/)**Ashim Gupta, Vivek Srikumar*2021 · 119 citations*
263. **[MetricX-24: The Google Submission to the WMT 2024 Metrics Shared Task](https://md.research.elamur.ai/free/en/papers/juraska-2024-metricx-24-google-submission-wmt/) [Conference on Machine Translation (WMT)](https://md.research.elamur.ai/free/en/venues/wmt/)**Juraj Juraska, Daniel Deutsch, Mara Finkelstein, et al.*2024 · 118 citations*
264. **[MetricX-23: The Google Submission to the WMT 2023 Metrics Shared Task](https://md.research.elamur.ai/free/en/papers/juraska-2023-metricx-23-google-submission-wmt/) [Conference on Machine Translation (WMT)](https://md.research.elamur.ai/free/en/venues/wmt/)**Juraj Juraska, Mara Finkelstein, Daniel Deutsch, et al.*2023 · 118 citations*
265. **[Findings of the AmericasNLP 2021 Shared Task on Open Machine Translation for Indigenous Languages of the Americas](https://md.research.elamur.ai/free/en/papers/mager-2021-findings-americasnlp-2021-shared-task/) [Workshop on NLP for Indigenous Languages of the Americas (AmericasNLP)](https://md.research.elamur.ai/free/en/venues/americasnlp/)**Manuel Mager, Arturo Oncevay, Abteen Ebrahimi, et al.*2021 · 118 citations*
266. **[Findings of the 2010 Joint Workshop on Statistical Machine Translation and Metrics for Machine Translation](https://md.research.elamur.ai/free/en/papers/burch-2010-findings-2010-joint-workshop-statistical/) [Conference on Machine Translation (WMT)](https://md.research.elamur.ai/free/en/venues/wmt/)**Chris Callison-Burch, Philipp Koehn, Christof Monz, et al.*2010 · 118 citations*
267. **[Constructing Corpora for the Development and Evaluation of Paraphrase Systems](https://md.research.elamur.ai/free/en/papers/cohn-2008-constructing-corpora-development-evaluation-paraphrase/) [Computational Linguistics](https://md.research.elamur.ai/free/en/venues/cl/)**Trevor Cohn, Chris Callison-Burch, Mirella Lapata*2008 · 118 citations*
268. **[How Much Do Language Models Copy From Their Training Data? Evaluating Linguistic Novelty in Text Generation Using RAVEN](https://md.research.elamur.ai/free/en/papers/mccoy-2023-how-much-do-language-models/) [Transactions of the Association for Computational Linguistics (TACL)](https://md.research.elamur.ai/free/en/venues/tacl/)**R. Thomas McCoy, Paul Smolensky, Tal Linzen, et al.*2023 · 116 citations*
269. [**Revisiting the Gold Standard: Grounding Summarization Evaluation with Robust Human Evaluation arXiv:2212.07981**Yixin Liu, Alexander R. Fabbri, Pengfei Liu, et al.*2022 · 116 citations*](https://md.research.elamur.ai/free/en/papers/liu-2022-revisiting-gold-standard-grounding-summarization/)
270. **[Intrinsic Bias Metrics Do Not Correlate with Application Bias](https://md.research.elamur.ai/free/en/papers/tarrant-2021-intrinsic-bias-metrics-do-not/) [Annual Meeting of the Association for Computational Linguistics (ACL)](https://md.research.elamur.ai/free/en/venues/acl/)**Seraphina Goldfarb-Tarrant, Rebecca Marchant, Ricardo Muñoz Sánchez, et al.*2021 · 116 citations*
271. **[A Statistical Analysis of Summarization Evaluation Metrics Using Resampling Methods](https://md.research.elamur.ai/free/en/papers/deutsch-2021-statistical-analysis-summarization-evaluation-metrics/) [Transactions of the Association for Computational Linguistics (TACL)](https://md.research.elamur.ai/free/en/venues/tacl/)**Daniel Deutsch, Rotem Dror, Dan Roth*2021 · 115 citations*
272. **[Fluency, Adequacy, or HTER? Exploring Different Human Judgments with a Tunable MT Metric](https://md.research.elamur.ai/free/en/papers/snover-2009-fluency-adequacy-or-hter-exploring/) [Conference on Machine Translation (WMT)](https://md.research.elamur.ai/free/en/venues/wmt/)**Matthew Snover, Nitin Madnani, Bonnie Dorr, et al.*2009 · 115 citations*
273. **[SALAD-Bench: A Hierarchical and Comprehensive Safety Benchmark for Large Language Models](https://md.research.elamur.ai/free/en/papers/li-2024-salad-bench-hierarchical-comprehensive-safety/) [Findings of the Association for Computational Linguistics](https://md.research.elamur.ai/free/en/venues/findings/)**Lijun Li, Bowen Dong, Ruohui Wang, et al.*2024 · 114 citations*
274. **[Fast, Cheap, and Creative: Evaluating Translation Quality Using Amazon's Mechanical Turk](https://md.research.elamur.ai/free/en/papers/burch-2009-fast-cheap-creative-evaluating-translation/) [Conference on Empirical Methods in Natural Language Processing (EMNLP)](https://md.research.elamur.ai/free/en/venues/emnlp/)**Chris Callison-Burch*2009 · 114 citations*
275. [**CRUXEval: A Benchmark for Code Reasoning, Understanding and Execution arXiv:2401.03065**Alex Gu, Baptiste Rozière, Hugh Leather, et al.*2024 · 113 citations*](https://md.research.elamur.ai/free/en/papers/gu-2024-cruxeval-benchmark-code-reasoning-understanding/)
276. **[Exploring the Use of Large Language Models for Reference-Free Text Quality Evaluation: An Empirical Study](https://md.research.elamur.ai/free/en/papers/chen-2023-exploring-use-large-language-models/) [Findings of the Association for Computational Linguistics](https://md.research.elamur.ai/free/en/venues/findings/)**Yi Chen, Rui Wang, Haiyun Jiang, et al.*2023 · 113 citations*
277. **[LongEval: Guidelines for Human Evaluation of Faithfulness in Long-form Summarization](https://md.research.elamur.ai/free/en/papers/krishna-2023-longeval-guidelines-human-evaluation-faithfulness/) [Conference of the European Chapter of the Association for Computational Linguistics (EACL)](https://md.research.elamur.ai/free/en/venues/eacl/)**Kalpesh Krishna, Erin Bransom, Bailey Kuehl, et al.*2023 · 113 citations*
278. **[A Critical Evaluation of Evaluations for Long-form Question Answering](https://md.research.elamur.ai/free/en/papers/xu-2023-critical-evaluation-evaluations-long-form/) [Annual Meeting of the Association for Computational Linguistics (ACL)](https://md.research.elamur.ai/free/en/venues/acl/)**Fangyuan Xu, Yixiao Song, Mohit Iyyer, et al.*2023 · 112 citations*
279. **[Overview of the WANLP 2021 Shared Task on Sarcasm and Sentiment Detection in Arabic](https://md.research.elamur.ai/free/en/papers/farha-2021-overview-wanlp-2021-shared-task/) [Arabic Natural Language Processing Workshop (WANLP)](https://md.research.elamur.ai/free/en/venues/wanlp/)**Ibrahim Abu Farha, Wajdi Zaghouani, Walid Magdy*2021 · 112 citations*
280. **[XL-WiC: A Multilingual Benchmark for Evaluating Semantic Contextualization](https://md.research.elamur.ai/free/en/papers/raganato-2020-xl-wic-multilingual-benchmark-evaluating/) [Conference on Empirical Methods in Natural Language Processing (EMNLP)](https://md.research.elamur.ai/free/en/venues/emnlp/)**Alessandro Raganato, Tommaso Pasini, Jose Camacho-Collados, et al.*2020 · 111 citations*
281. [**Corpus Annotation for Parser Evaluation**John Carroll, Guido Minnen, Ted Briscoe*1999 · 111 citations*](https://md.research.elamur.ai/free/en/papers/carroll-1999-corpus-annotation-parser-evaluation/)
282. **[RAID: A Shared Benchmark for Robust Evaluation of Machine-Generated Text Detectors](https://md.research.elamur.ai/free/en/papers/dugan-2024-raid-shared-benchmark-robust-evaluation/) [Annual Meeting of the Association for Computational Linguistics (ACL)](https://md.research.elamur.ai/free/en/venues/acl/)**Liam Dugan, Alyssa Hwang, Filip Trhlík, et al.*2024 · 110 citations*
283. [**Evaluating the Performance of Large Language Models on GAOKAO Benchmark arXiv:2305.12474**Xiaotian Zhang, Chunyang Li, Yi Zong, et al.*2023 · 109 citations*](https://md.research.elamur.ai/free/en/papers/zhang-2023-evaluating-performance-large-language/)
284. **[CharacterEval: A Chinese Benchmark for Role-Playing Conversational Agent Evaluation](https://md.research.elamur.ai/free/en/papers/tu-2024-charactereval-chinese-benchmark-role-playing/) [Annual Meeting of the Association for Computational Linguistics (ACL)](https://md.research.elamur.ai/free/en/venues/acl/)**Quan Tu, Shilong Fan, Zihang Tian, et al.*2024 · 108 citations*
285. **[GSM-Plus: A Comprehensive Benchmark for Evaluating the Robustness of LLMs as Mathematical Problem Solvers](https://md.research.elamur.ai/free/en/papers/li-2024-gsm-plus-comprehensive-benchmark-evaluating/) [Annual Meeting of the Association for Computational Linguistics (ACL)](https://md.research.elamur.ai/free/en/venues/acl/)**Qintong Li, Leyang Cui, Xueliang Zhao, et al.*2024 · 107 citations*
286. **[Do Androids Laugh at Electric Sheep? Humor “Understanding” Benchmarks from The New Yorker Caption Contest](https://md.research.elamur.ai/free/en/papers/hessel-2023-do-androids-laugh-electric-sheep/) [Annual Meeting of the Association for Computational Linguistics (ACL)](https://md.research.elamur.ai/free/en/venues/acl/)**Jack Hessel, Ana Marasovic, Jena D. Hwang, et al.*2023 · 107 citations*
287. **[Evaluating Explanations: How Much Do Explanations from the Teacher Aid Students?](https://md.research.elamur.ai/free/en/papers/pruthi-2022-evaluating-explanations-how-much-do/) [Transactions of the Association for Computational Linguistics (TACL)](https://md.research.elamur.ai/free/en/venues/tacl/)**Danish Pruthi, Rachit Bansal, Bhuwan Dhingra, et al.*2022 · 107 citations*
288. **[APE at Scale and Its Implications on MT Evaluation Biases](https://md.research.elamur.ai/free/en/papers/freitag-2019-ape-scale-its-implications-mt/) [Conference on Machine Translation (WMT)](https://md.research.elamur.ai/free/en/venues/wmt/)**Markus Freitag, Isaac Caswell, Scott Roy*2019 · 107 citations*
289. **[Are LLMs Breaking MT Metrics? Results of the WMT24 Metrics Shared Task](https://md.research.elamur.ai/free/en/papers/freitag-2024-are-llms-breaking-mt-metrics/) [Conference on Machine Translation (WMT)](https://md.research.elamur.ai/free/en/venues/wmt/)**Markus Freitag, Nitika Mathur, Daniel Deutsch, et al.*2024 · 106 citations*
290. [**NTREX-128 – News Test References for MT Evaluation of 128 Languages Proceedings of the First Workshop on Scaling Up Multilingual Evaluation**Christian Federmann, Tom Kocmi, Ying Xin*2022 · 104 citations*](https://md.research.elamur.ai/free/en/papers/federmann-2022-ntrex-128-news-test-references/)
291. **[LENS: A Learnable Evaluation Metric for Text Simplification](https://md.research.elamur.ai/free/en/papers/maddela-2023-lens-learnable-evaluation-metric-text/) [Annual Meeting of the Association for Computational Linguistics (ACL)](https://md.research.elamur.ai/free/en/venues/acl/)**Mounica Maddela, Yao Dou, David Heineman, et al.*2023 · 103 citations*
292. **[Evaluating the Evaluation of Diversity in Natural Language Generation](https://md.research.elamur.ai/free/en/papers/tevet-2021-evaluating-evaluation-diversity-natural-language/) [Conference of the European Chapter of the Association for Computational Linguistics (EACL)](https://md.research.elamur.ai/free/en/venues/eacl/)**Guy Tevet, Jonathan Berant*2021 · 103 citations*
293. [**Findings of the Shared Task on Offensive Language Identification in Tamil, Malayalam, and Kannada Proceedings of the First Workshop on Speech and Language Technologies for Dravidian Languages**Bharathi Raja Chakravarthi, Ruba Priyadharshini, Navya Jose, et al.*2021 · 103 citations*](https://md.research.elamur.ai/free/en/papers/chakravarthi-2021-findings-shared-task-offensive-language/)
294. **[On the Safety of Conversational Models: Taxonomy, Dataset, and Benchmark](https://md.research.elamur.ai/free/en/papers/sun-2022-safety-conversational-models-taxonomy-dataset/) [Findings of the Association for Computational Linguistics](https://md.research.elamur.ai/free/en/venues/findings/)**Hao Sun, Guangxuan Xu, Jiawen Deng, et al.*2022 · 102 citations*
295. **[Findings of the WMT 2019 Shared Task on Automatic Post-Editing](https://md.research.elamur.ai/free/en/papers/chatterjee-2019-findings-wmt-2019-shared-task/) [Conference on Machine Translation (WMT)](https://md.research.elamur.ai/free/en/venues/wmt/)**Rajen Chatterjee, Christian Federmann, Matteo Negri, et al.*2019 · 102 citations*
296. [**MMTEB: Massive Multilingual Text Embedding Benchmark arXiv:2502.13595**Kenneth Enevoldsen, Isaac Chung, Imene Kerboua, et al.*2025 · 101 citations*](https://md.research.elamur.ai/free/en/papers/enevoldsen-2025-mmteb-massive-multilingual-text-embedding/)
297. [**Rethinking Benchmark and Contamination for Language Models with Rephrased Samples arXiv:2311.04850**Shuo Yang, Wei-Lin Chiang, Lianmin Zheng, et al.*2023 · 100 citations*](https://md.research.elamur.ai/free/en/papers/yang-2023-rethinking-benchmark-contamination-language-models/)
298. **[On the Intrinsic and Extrinsic Fairness Evaluation Metrics for Contextualized Language Representations](https://md.research.elamur.ai/free/en/papers/cao-2022-intrinsic-extrinsic-fairness-evaluation-metrics/) [Annual Meeting of the Association for Computational Linguistics (ACL)](https://md.research.elamur.ai/free/en/venues/acl/)**Yang Trista Cao, Yada Pruksachatkun, Kai-Wei Chang, et al.*2022 · 100 citations*
299. **[Tomayto, Tomahto. Beyond Token-level Answer Equivalence for Question Answering Evaluation](https://md.research.elamur.ai/free/en/papers/bulian-2022-tomayto-tomahto-beyond-token-level/) [Conference on Empirical Methods in Natural Language Processing (EMNLP)](https://md.research.elamur.ai/free/en/venues/emnlp/)**Jannis Bulian, Christian Buck, Wojciech Gajewski, et al.*2022 · 100 citations*
300. **[UniTE: Unified Translation Evaluation](https://md.research.elamur.ai/free/en/papers/wan-2022-unite-unified-translation-evaluation/) [Annual Meeting of the Association for Computational Linguistics (ACL)](https://md.research.elamur.ai/free/en/venues/acl/)**Yu Wan, Dayiheng Liu, Baosong Yang, et al.*2022 · 100 citations*
301. **[Gender in Danger? Evaluating Speech Translation Technology on the MuST-SHE Corpus](https://md.research.elamur.ai/free/en/papers/bentivogli-2020-gender-danger-evaluating-speech-translation/) [Annual Meeting of the Association for Computational Linguistics (ACL)](https://md.research.elamur.ai/free/en/venues/acl/)**Luisa Bentivogli, Beatrice Savoldi, Matteo Negri, et al.*2020 · 100 citations*
302. **[Evaluating Attribution in Dialogue Systems: The BEGIN Benchmark](https://md.research.elamur.ai/free/en/papers/dziri-2022-evaluating-attribution-dialogue-systems-begin/) [Transactions of the Association for Computational Linguistics (TACL)](https://md.research.elamur.ai/free/en/venues/tacl/)**Nouha Dziri, Hannah Rashkin, Tal Linzen, et al.*2022 · 99 citations*
303. **[Unbabel's Participation in the WMT20 Metrics Shared Task](https://md.research.elamur.ai/free/en/papers/rei-2020-unbabel-s-participation-wmt20-metrics/) [Conference on Machine Translation (WMT)](https://md.research.elamur.ai/free/en/venues/wmt/)**Ricardo Rei, Craig Stewart, Ana C Farinha, et al.*2020 · 98 citations*
304. **[DialFact: A Benchmark for Fact-Checking in Dialogue](https://md.research.elamur.ai/free/en/papers/gupta-2022-dialfact-benchmark-fact-checking-dialogue/) [Annual Meeting of the Association for Computational Linguistics (ACL)](https://md.research.elamur.ai/free/en/venues/acl/)**Prakhar Gupta, Chien-Sheng Wu, Wenhao Liu, et al.*2022 · 97 citations*
305. **[Cross-Linguistic Syntactic Evaluation of Word Prediction Models](https://md.research.elamur.ai/free/en/papers/mueller-2020-cross-linguistic-syntactic-evaluation-word/) [Annual Meeting of the Association for Computational Linguistics (ACL)](https://md.research.elamur.ai/free/en/venues/acl/)**Aaron Mueller, Garrett Nicolai, Panayiota Petrou-Zeniou, et al.*2020 · 97 citations*
306. **[Evaluation of machine translation and its evaluation](https://md.research.elamur.ai/free/en/papers/turian-2003-evaluation-machine-translation-its-evaluation/) [Machine Translation Summit (MT Summit)](https://md.research.elamur.ai/free/en/venues/mtsummit/)**Joseph P. Turian, Luke Shen, I. Dan Melamed*2003 · 97 citations*
307. **[LinCE: A Centralized Benchmark for Linguistic Code-switching Evaluation](https://md.research.elamur.ai/free/en/papers/aguilar-2020-lince-centralized-benchmark-linguistic-code/) [Language Resources and Evaluation Conference (LREC)](https://md.research.elamur.ai/free/en/venues/lrec/)**Gustavo Aguilar, Sudipta Kar, Thamar Solorio*2020 · 96 citations*
308. **[Overview of the EVALITA 2018 Task on Irony Detection in Italian Tweets (IronITA)](https://md.research.elamur.ai/free/en/papers/cignarella-2018-overview-evalita-2018-task-irony/) [Evaluation Campaign of Natural Language Processing and Speech Tools for Italian. Final Workshop](https://md.research.elamur.ai/free/en/venues/evalita/)**Alessandra Teresa Cignarella, Simona Frenda, Valerio Basile, et al.*2018 · 95 citations*
309. [**Using Test Suites in Evaluation of Machine Translation Systems**Margaret King, Kirsten Falkedal*1990 · 95 citations*](https://md.research.elamur.ai/free/en/papers/king-1990-test-suites-evaluation-machine-translation/)
310. [**SPARTQA: A Textual Question Answering Benchmark for Spatial Reasoning**Roshanak Mirzaee, Hossein Rajaby Faghihi, Qiang Ning, et al.*2021 · 93 citations*](https://md.research.elamur.ai/free/en/papers/mirzaee-2021-spartqa-textual-question-answering-benchmark/)
311. **[Findings of the WMT 2022 Shared Task on Quality Estimation](https://md.research.elamur.ai/free/en/papers/zerva-2022-findings-wmt-2022-shared-task/) [Conference on Machine Translation (WMT)](https://md.research.elamur.ai/free/en/venues/wmt/)**Chrysoula Zerva, Frédéric Blain, Ricardo Rei, et al.*2022 · 91 citations*
312. **[Evaluation Metrics in the Era of GPT-4: Reliably Evaluating Large Language Models on Sequence to Sequence Tasks](https://md.research.elamur.ai/free/en/papers/sottana-2023-evaluation-metrics-era-gpt-4/) [Conference on Empirical Methods in Natural Language Processing (EMNLP)](https://md.research.elamur.ai/free/en/venues/emnlp/)**Andrea Sottana, Bin Liang, Kai Zou, et al.*2023 · 89 citations*
313. [**A Pairwise Event Coreference Model, Feature Impact and Evaluation for Event Coreference Resolution Proceedings of the Workshop on Events in Emerging Text Types**Zheng Chen, Heng Ji, Robert Haralick*2009 · 88 citations*](https://md.research.elamur.ai/free/en/papers/chen-2009-pairwise-event-coreference-model-feature/)
314. [**Multidimensional quality metrics: a flexible system for assessing translation quality**Arle Richard Lommel, Aljoscha Burchardt, Hans Uszkoreit*2013 · 87 citations*](https://md.research.elamur.ai/free/en/papers/lommel-2013-multidimensional-quality-metrics-flexible-system/)
315. **[Evaluation Metrics For End-to-End Coreference Resolution Systems](https://md.research.elamur.ai/free/en/papers/cai-2010-evaluation-metrics-end-end-coreference/) [SIGDIAL Conference](https://md.research.elamur.ai/free/en/venues/sigdial-conference/)**Jie Cai, Michael Strube*2010 · 87 citations*
316. [**Error Analysis of Statistical Machine Translation Output**David Vilar, Jia Xu, Luis Fernando D'Haro, et al.*2006 · 87 citations*](https://md.research.elamur.ai/free/en/papers/vilar-2006-error-analysis-statistical-machine-translation/)
317. **[Validation of an Automatic Metric for the Accuracy of Pronoun Translation (APT)](https://md.research.elamur.ai/free/en/papers/werlen-2017-validation-automatic-metric-accuracy-pronoun/) [Workshop on Discourse in Machine Translation](https://md.research.elamur.ai/free/en/venues/workshop-on-discourse-in-machine-translation/)**Lesly Miculicich Werlen, Andrei Popescu-Belis*2017 · 86 citations*
318. [**Unbounded Dependency Recovery for Parser Evaluation**Laura Rimell, Stephen Clark, Mark Steedman*2009 · 86 citations*](https://md.research.elamur.ai/free/en/papers/rimell-2009-unbounded-dependency-recovery-parser-evaluation/)
319. **[FollowBench: A Multi-level Fine-grained Constraints Following Benchmark for Large Language Models](https://md.research.elamur.ai/free/en/papers/jiang-2024-followbench-multi-level-fine-grained/) [Annual Meeting of the Association for Computational Linguistics (ACL)](https://md.research.elamur.ai/free/en/venues/acl/)**Yuxin Jiang, Yufei Wang, Xingshan Zeng, et al.*2024 · 85 citations*
320. **[BlonDe: An Automatic Evaluation Metric for Document-level Machine Translation](https://md.research.elamur.ai/free/en/papers/jiang-2022-blonde-automatic-evaluation-metric-document/) [Conference of the North American Chapter of the Association for Computational Linguistics (NAACL)](https://md.research.elamur.ai/free/en/venues/naacl/)**Yuchen Jiang, Tianyu Liu, Shuming Ma, et al.*2022 · 85 citations*
321. **[Are References Really Needed? Unbabel-IST 2021 Submission for the Metrics Shared Task](https://md.research.elamur.ai/free/en/papers/rei-2021-are-references-really-needed-unbabel/) [Conference on Machine Translation (WMT)](https://md.research.elamur.ai/free/en/venues/wmt/)**Ricardo Rei, Ana C Farinha, Chrysoula Zerva, et al.*2021 · 85 citations*
322. [**Evaluation of Dependency Parsers on Unbounded Dependencies**Joakim Nivre, Laura Rimell, Ryan McDonald, et al.*2010 · 85 citations*](https://md.research.elamur.ai/free/en/papers/nivre-2010-evaluation-dependency-parsers-unbounded-dependencies/)
323. **[Evaluating Mathematical Reasoning Beyond Accuracy](https://md.research.elamur.ai/free/en/papers/xia-2024-evaluating-mathematical-reasoning-accuracy/) [Proceedings of the AAAI Conference on Artificial Intelligence](https://md.research.elamur.ai/free/en/venues/proceedings-of-the-aaai-conference-on-artificial-intelligence/)**Shijie Xia, Xuefeng Li, Yixin Liu, et al.*2024 · 84 citations*
324. **[Error Span Annotation: A Balanced Approach for Human Evaluation of Machine Translation](https://md.research.elamur.ai/free/en/papers/kocmi-2024-error-span-annotation-balanced-approach/) [Conference on Machine Translation (WMT)](https://md.research.elamur.ai/free/en/venues/wmt/)**Tom Kocmi, Vilém Zouhar, Eleftherios Avramidis, et al.*2024 · 83 citations*
325. **[ORCA: A Challenging Benchmark for Arabic Language Understanding](https://md.research.elamur.ai/free/en/papers/elmadany-2023-orca-challenging-benchmark-arabic-language/) [Findings of the Association for Computational Linguistics](https://md.research.elamur.ai/free/en/venues/findings/)**AbdelRahim Elmadany, ElMoatez Billah Nagoudi, Muhammad Abdul-Mageed*2023 · 83 citations*

[← prev](https://md.research.elamur.ai/free/en/topics/evaluation-benchmarks/page/3/) [1](https://md.research.elamur.ai/free/en/topics/evaluation-benchmarks/) [2](https://md.research.elamur.ai/free/en/topics/evaluation-benchmarks/page/2/) [3](https://md.research.elamur.ai/free/en/topics/evaluation-benchmarks/page/3/) **4** [5](https://md.research.elamur.ai/free/en/topics/evaluation-benchmarks/page/5/) … [8](https://md.research.elamur.ai/free/en/topics/evaluation-benchmarks/page/8/) [next →](https://md.research.elamur.ai/free/en/topics/evaluation-benchmarks/page/5/)