---
title: "Evaluation & benchmarks — papers, page 2 — Elamur Research"
description: "Evaluation & benchmarks: 684 papers of the Elamur Research library · page 2 of 8 · 26–125, the most cited first."
canonical: "https://research.elamur.ai/free/en/topics/evaluation-benchmarks/page/2/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/topics/evaluation-benchmarks/page/2/)

[home](https://md.research.elamur.ai/free/en/) / [topics](https://md.research.elamur.ai/free/en/topics/) / [Evaluation & benchmarks](https://md.research.elamur.ai/free/en/topics/evaluation-benchmarks/) / page 2

# 📏 Evaluation & benchmarks

684 papers · page 2 of 8 · 26–125 · the most cited first · [all topics](https://md.research.elamur.ai/free/en/topics/)

How systems are measured: benchmarks, metrics, human judgements and what the numbers mean.

## Papers (684)

[← prev](https://md.research.elamur.ai/free/en/topics/evaluation-benchmarks/) [1](https://md.research.elamur.ai/free/en/topics/evaluation-benchmarks/) **2** [3](https://md.research.elamur.ai/free/en/topics/evaluation-benchmarks/page/3/) … [8](https://md.research.elamur.ai/free/en/topics/evaluation-benchmarks/page/8/) [next →](https://md.research.elamur.ai/free/en/topics/evaluation-benchmarks/page/3/)

26. **[One Billion Word Benchmark for Measuring Progress in Statistical Language Modeling](https://md.research.elamur.ai/free/en/papers/chelba-2013-one-billion-word-benchmark-measuring/) [Interspeech](https://md.research.elamur.ai/free/en/venues/interspeech-usb/)**Ciprian Chelba, Tomas Mikolov, Mike Schuster, et al.*2013 · 949 citations*
27. **[MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGI](https://md.research.elamur.ai/free/en/papers/yue-2023-mmmu-massive-multi-discipline-multimodal/) [IEEE Computer Society Conference on Computer Vision and Pattern Recognition](https://md.research.elamur.ai/free/en/venues/ieee-computer-society-conference-on-computer-vision-and-pattern-recognition/)**Xiang Yue, Yuansheng Ni, Kai Zhang, et al.*2023 · 948 citations*
28. [**DROP: A Reading Comprehension Benchmark Requiring Discrete Reasoning Over Paragraphs**Dheeru Dua, Yizhong Wang, Pradeep Dasigi, et al.*2019 · 943 citations*](https://md.research.elamur.ai/free/en/papers/dua-2019-drop-reading-comprehension-benchmark-requiring/)
29. **[Automatic Evaluation of Machine Translation Quality Using Longest Common Subsequence and Skip-Bigram Statistics](https://md.research.elamur.ai/free/en/papers/lin-2004-automatic-evaluation-machine-translation-quality/) [Annual Meeting of the Association for Computational Linguistics (ACL)](https://md.research.elamur.ai/free/en/venues/acl/)**Chin-Yew Lin, Franz Josef Och*2004 · 940 citations*
30. **[Findings of the 2016 Conference on Machine Translation](https://md.research.elamur.ai/free/en/papers/bojar-2016-findings-2016-conference-machine-translation/) [Conference on Machine Translation: Shared Task Papers](https://md.research.elamur.ai/free/en/venues/conference-on-machine-translation-shared-task-papers/)**Ondřej Bojar, Rajen Chatterjee, Christian Federmann, et al.*2016 · 938 citations*
31. **[Findings of the 2014 Workshop on Statistical Machine Translation](https://md.research.elamur.ai/free/en/papers/bojar-2014-findings-2014-workshop-statistical-machine/) [Conference on Machine Translation (WMT)](https://md.research.elamur.ai/free/en/venues/wmt/)**Ondřej Bojar, Christian Buck, Christian Federmann, et al.*2014 · 929 citations*
32. **[Collecting Highly Parallel Data for Paraphrase Evaluation](https://md.research.elamur.ai/free/en/papers/chen-2011-collecting-highly-parallel-data-paraphrase/) [Annual Meeting of the Association for Computational Linguistics (ACL)](https://md.research.elamur.ai/free/en/venues/acl/)**David Chen, William Dolan*2011 · 929 citations*
33. [**A Multitask, Multilingual, Multimodal Evaluation of ChatGPT on Reasoning, Hallucination, and Interactivity arXiv:2302.04023**Yejin Bang, Samuel Cahyawijaya, Nayeon Lee, et al.*2023 · 876 citations*](https://md.research.elamur.ai/free/en/papers/bang-2023-multitask-multilingual-multimodal-evaluation-chatgpt/)
34. [**On Coreference Resolution Performance Metrics**Xiaoqiang Luo*2005 · 855 citations*](https://md.research.elamur.ai/free/en/papers/luo-2005-coreference-resolution-performance-metrics/)
35. **[Findings of the 2019 Conference on Machine Translation (WMT19)](https://md.research.elamur.ai/free/en/papers/barrault-2019-findings-2019-conference-machine-translation/) [Conference on Machine Translation (WMT)](https://md.research.elamur.ai/free/en/venues/wmt/)**Loïc Barrault, Ondřej Bojar, Marta R. Costa-jussà, et al.*2019 · 841 citations*
36. **[Automatic Evaluation of Topic Coherence](https://md.research.elamur.ai/free/en/papers/newman-2010-automatic-evaluation-topic-coherence/) [Conference of the North American Chapter of the Association for Computational Linguistics (NAACL)](https://md.research.elamur.ai/free/en/venues/naacl/)**David Newman, Jey Han Lau, Karl Grieser, et al.*2010 · 825 citations*
37. **[ERASER: A Benchmark to Evaluate Rationalized NLP Models](https://md.research.elamur.ai/free/en/papers/deyoung-2020-eraser-benchmark-evaluate-rationalized-nlp/) [Annual Meeting of the Association for Computational Linguistics (ACL)](https://md.research.elamur.ai/free/en/venues/acl/)**Jay DeYoung, Sarthak Jain, Nazneen Fatema Rajani, et al.*2020 · 821 citations*
38. **[Findings of the 2018 Conference on Machine Translation (WMT18)](https://md.research.elamur.ai/free/en/papers/bojar-2018-findings-2018-conference-machine-translation/) [Conference on Machine Translation: Shared Task Papers](https://md.research.elamur.ai/free/en/venues/conference-on-machine-translation-shared-task-papers/)**Ondřej Bojar, Christian Federmann, Mark Fishel, et al.*2018 · 750 citations*
39. **[ChartQA: A Benchmark for Question Answering about Charts with Visual and Logical Reasoning](https://md.research.elamur.ai/free/en/papers/masry-2022-chartqa-benchmark-question-answering-about/) [Findings of the Association for Computational Linguistics](https://md.research.elamur.ai/free/en/venues/findings/)**Ahmed Masry, Do Xuan Long, Jia Qing Tan, et al.*2022 · 745 citations*
40. **[Constructing A Multi-hop QA Dataset for Comprehensive Evaluation of Reasoning Steps](https://md.research.elamur.ai/free/en/papers/ho-2020-constructing-multi-hop-qa-dataset/) [International Conference on Computational Linguistics (COLING)](https://md.research.elamur.ai/free/en/venues/coling/)**Xanh Ho, Anh-Khoa Duong Nguyen, Saku Sugawara, et al.*2020 · 735 citations*
41. **[MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmark](https://md.research.elamur.ai/free/en/papers/wang-2024-mmlu-pro-more-robust-challenging/) [Conference on Neural Information Processing Systems (NeurIPS)](https://md.research.elamur.ai/free/en/venues/neural-information-processing-systems/)**Yubo Wang, Xueguang Ma, Ge Zhang, et al.*2024 · 731 citations*
42. **[IndicNLPSuite: Monolingual Corpora, Evaluation Benchmarks and Pre-trained Multilingual Language Models for Indian Languages](https://md.research.elamur.ai/free/en/papers/kakwani-2020-indicnlpsuite-monolingual-corpora-evaluation-benchmarks/) [Findings of the Association for Computational Linguistics](https://md.research.elamur.ai/free/en/venues/findings/)**Divyanshu Kakwani, Anoop Kunchukuttan, Satish Golla, et al.*2020 · 730 citations*
43. **[ALFRED: A Benchmark for Interpreting Grounded Instructions for Everyday Tasks](https://md.research.elamur.ai/free/en/papers/shridhar-2019-alfred-benchmark-interpreting-grounded-instructions/) [IEEE Computer Society Conference on Computer Vision and Pattern Recognition](https://md.research.elamur.ai/free/en/venues/ieee-computer-society-conference-on-computer-vision-and-pattern-recognition/)**Mohit Shridhar; Jesse Thomason; Daniel Gordon; Yonatan Bisk; Winson Han; Roozbeh Mottaghi; Luke Zettlemoyer;, Dieter Fox*2019 · 726 citations*
44. **[TyDi QA: A Benchmark for Information-Seeking Question Answering in Typologically Diverse Languages](https://md.research.elamur.ai/free/en/papers/clark-2020-tydi-qa-benchmark-information-seeking/) [Transactions of the Association for Computational Linguistics (TACL)](https://md.research.elamur.ai/free/en/venues/tacl/)**Jonathan H. Clark, Eunsol Choi, Michael Collins, et al.*2020 · 692 citations*
45. **[Findings of the 2017 Conference on Machine Translation (WMT17)](https://md.research.elamur.ai/free/en/papers/bojar-2017-findings-2017-conference-machine-translation/) [Conference on Machine Translation (WMT)](https://md.research.elamur.ai/free/en/venues/wmt/)**Ondřej Bojar, Rajen Chatterjee, Christian Federmann, et al.*2017 · 689 citations*
46. **[Why We Need New Evaluation Metrics for NLG](https://md.research.elamur.ai/free/en/papers/novikova-2017-why-we-need-new-evaluation/) [Conference on Empirical Methods in Natural Language Processing (EMNLP)](https://md.research.elamur.ai/free/en/venues/emnlp/)**Jekaterina Novikova, Ondřej Dušek, Amanda Cercas Curry, et al.*2017 · 688 citations*
47. **[Evaluation methods for unsupervised word embeddings](https://md.research.elamur.ai/free/en/papers/schnabel-2015-evaluation-methods-unsupervised-word-embeddings/) [Conference on Empirical Methods in Natural Language Processing (EMNLP)](https://md.research.elamur.ai/free/en/venues/emnlp/)**Tobias Schnabel, Igor Labutov, David Mimno, et al.*2015 · 674 citations*
48. **[KILT: a Benchmark for Knowledge Intensive Language Tasks](https://md.research.elamur.ai/free/en/papers/petroni-2020-kilt-benchmark-knowledge-intensive-language/) [Conference of the North American Chapter of the Association for Computational Linguistics (NAACL)](https://md.research.elamur.ai/free/en/venues/naacl/)**Fabio Petroni, Aleksandra Piktus, Angela Fan, et al.*2020 · 665 citations*
49. [**Experts, Errors, and Context: A Large-Scale Study of Human Evaluation for Machine Translation**Markus Freitag, George Foster, David Grangier, et al.*2021 · 660 citations*](https://md.research.elamur.ai/free/en/papers/freitag-2021-experts-errors-context-large-scale/)
50. **[HaluEval: A Large-Scale Hallucination Evaluation Benchmark for Large Language Models](https://md.research.elamur.ai/free/en/papers/li-2023-halueval-large-scale-hallucination-evaluation/) [Conference on Empirical Methods in Natural Language Processing (EMNLP)](https://md.research.elamur.ai/free/en/venues/emnlp/)**Junyi Li, Xiaoxue Cheng, Wayne Xin Zhao, et al.*2023 · 647 citations*
51. **[Is Your Code Generated by ChatGPT Really Correct? Rigorous Evaluation of Large Language Models for Code Generation](https://md.research.elamur.ai/free/en/papers/liu-2023-your-code-generated-chatgpt-really/) [Conference on Neural Information Processing Systems (NeurIPS)](https://md.research.elamur.ai/free/en/venues/neural-information-processing-systems/)**Jiawei Liu, Chunqiu Steven Xia, Yuyao Wang, et al.*2023 · 632 citations*
52. **[A Critique and Improvement of an Evaluation Metric for Text Segmentation](https://md.research.elamur.ai/free/en/papers/pevzner-2002-critique-improvement-evaluation-metric-text/) [Computational Linguistics](https://md.research.elamur.ai/free/en/venues/cl/)**Lev Pevzner, Marti A. Hearst*2002 · 595 citations*
53. **[Evaluating Gender Bias in Machine Translation](https://md.research.elamur.ai/free/en/papers/stanovsky-2019-evaluating-gender-bias-machine-translation/) [Annual Meeting of the Association for Computational Linguistics (ACL)](https://md.research.elamur.ai/free/en/venues/acl/)**Gabriel Stanovsky, Noah A. Smith, Luke Zettlemoyer*2019 · 589 citations*
54. [**Instruction-Following Evaluation for Large Language Models arXiv:2311.07911**Jeffrey Zhou, Tianjian Lu, Swaroop Mishra, et al.*2023 · 586 citations*](https://md.research.elamur.ai/free/en/papers/zhou-2023-instruction-following-evaluation-large-language/)
55. [**Dear Sir or Madam, May I Introduce the GYAFC Dataset: Corpus, Benchmarks and Metrics for Formality Style Transfer**Sudha Rao, Joel Tetreault*2018 · 582 citations*](https://md.research.elamur.ai/free/en/papers/rao-2018-dear-sir-or-madam-may/)
56. **[AGIEval: A Human-Centric Benchmark for Evaluating Foundation Models](https://md.research.elamur.ai/free/en/papers/zhong-2024-agieval-human-centric-benchmark-evaluating/) [Findings of the Association for Computational Linguistics](https://md.research.elamur.ai/free/en/venues/findings/)**Wanjun Zhong, Ruixiang Cui, Yiduo Guo, et al.*2024 · 564 citations*
57. [**How Close is ChatGPT to Human Experts? Comparison Corpus, Evaluation, and Detection arXiv:2301.07597**Biyang Guo, Xin Zhang, Ziyuan Wang, et al.*2023 · 553 citations*](https://md.research.elamur.ai/free/en/papers/guo-2023-close-chatgpt-human-experts-comparison/)
58. [**XTREME: A Massively Multilingual Multi-task Benchmark for Evaluating Cross-lingual Generalization arXiv:2003.11080**Junjie Hu, Sebastian Ruder, Aditya Siddhant, et al.*2020 · 552 citations*](https://md.research.elamur.ai/free/en/papers/hu-2020-xtreme-massively-multilingual-multi-task/)
59. **[Edinburgh System Description for the 2005 IWSLT Speech Translation Evaluation](https://md.research.elamur.ai/free/en/papers/koehn-2005-edinburgh-system-description-2005-iwslt/) [International Conference on Spoken Language Translation (IWSLT)](https://md.research.elamur.ai/free/en/venues/iwslt/)**Philipp Koehn, Amittai Axelrod, Alexandra Birch Mayne, et al.*2005 · 545 citations*
60. [**A Structured Review of the Validity of BLEU**Ehud Reiter*2018 · 541 citations*](https://md.research.elamur.ai/free/en/papers/reiter-2018-structured-review-validity-bleu/)
61. **[Meteor 1.3: Automatic Metric for Reliable Optimization and Evaluation of Machine Translation Systems](https://md.research.elamur.ai/free/en/papers/denkowski-2011-meteor-1-3-automatic-metric/) [Conference on Machine Translation (WMT)](https://md.research.elamur.ai/free/en/venues/wmt/)**Michael Denkowski, Alon Lavie*2011 · 541 citations*
62. [**Automatic Annotation and Evaluation of Error Types for Grammatical Error Correction**Christopher Bryant, Mariano Felice, Ted Briscoe*2017 · 534 citations*](https://md.research.elamur.ai/free/en/papers/bryant-2017-automatic-annotation-evaluation-error-types/)
63. **[MME: A Comprehensive Evaluation Benchmark for Multimodal Large Language Models](https://md.research.elamur.ai/free/en/papers/fu-2023-mme-comprehensive-evaluation-benchmark-multimodal/) [Conference on Neural Information Processing Systems (NeurIPS)](https://md.research.elamur.ai/free/en/venues/neural-information-processing-systems/)**Chaoyou Fu, Peixian Chen, Yunhang Shen, et al.*2023 · 529 citations*
64. [**Evaluating Models' Local Decision Boundaries via Contrast Sets**Matt Gardner, Yoav Artzi, Victoria Basmov, et al.*2020 · 522 citations*](https://md.research.elamur.ai/free/en/papers/gardner-2020-evaluating-models-local-decision-boundaries/)
65. [**LongBench: A Bilingual, Multitask Benchmark for Long Context Understanding arXiv:2308.14508**Yushi Bai, Xin Lv, Jiajie Zhang, et al.*2023 · 506 citations*](https://md.research.elamur.ai/free/en/papers/bai-2023-longbench-bilingual-multitask-benchmark/)
66. **[A-OKVQA: A Benchmark for Visual Question Answering using World Knowledge](https://md.research.elamur.ai/free/en/papers/schwenk-2022-okvqa-benchmark-visual-question-answering/) [Lecture notes in computer science](https://md.research.elamur.ai/free/en/venues/lecture-notes-in-computer-science/)**Dustin Schwenk, Apoorv Khandelwal, Christopher Clark, et al.*2022 · 490 citations*
67. **[BBQ: A hand-built bias benchmark for question answering](https://md.research.elamur.ai/free/en/papers/parrish-2022-bbq-hand-built-bias-benchmark/) [Findings of the Association for Computational Linguistics](https://md.research.elamur.ai/free/en/venues/findings/)**Alicia Parrish, Angelica Chen, Nikita Nangia, et al.*2022 · 488 citations*
68. **[Evaluating Large Language Models in Theory of Mind Tasks](https://md.research.elamur.ai/free/en/papers/kosinski-2023-evaluating-large-language-models-theory/) [Proceedings of the National Academy of Sciences](https://md.research.elamur.ai/free/en/venues/proceedings-of-the-national-academy-of-sciences/)**Michal Kosinski*2023 · 467 citations*
69. **[Findings of the 2015 Workshop on Statistical Machine Translation](https://md.research.elamur.ai/free/en/papers/bojar-2015-findings-2015-workshop-statistical-machine/) [Conference on Machine Translation (WMT)](https://md.research.elamur.ai/free/en/venues/wmt/)**Ondřej Bojar, Rajen Chatterjee, Christian Federmann, et al.*2015 · 462 citations*
70. [**Text-to-SQL Empowered by Large Language Models: A Benchmark Evaluation arXiv:2308.15363**Dawei Gao, Haibin Wang, Yaliang Li, et al.*2023 · 461 citations*](https://md.research.elamur.ai/free/en/papers/gao-2023-text-sql-empowered-large-language/)
71. **[Automatic Evaluation of Translation Quality for Distant Language Pairs](https://md.research.elamur.ai/free/en/papers/isozaki-2010-automatic-evaluation-translation-quality-distant/) [Conference on Empirical Methods in Natural Language Processing (EMNLP)](https://md.research.elamur.ai/free/en/venues/emnlp/)**Hideki Isozaki, Tsutomu Hirao, Kevin Duh, et al.*2010 · 461 citations*
72. **[OlympiadBench: A Challenging Benchmark for Promoting AGI with Olympiad-Level Bilingual Multimodal Scientific Problems](https://md.research.elamur.ai/free/en/papers/he-2024-olympiadbench-challenging-benchmark-promoting-agi/) [Annual Meeting of the Association for Computational Linguistics (ACL)](https://md.research.elamur.ai/free/en/venues/acl/)**Chaoqun He, Renjie Luo, Yuzhuo Bai, et al.*2024 · 448 citations*
73. **[Has Machine Translation Achieved Human Parity? A Case for Document-level Evaluation](https://md.research.elamur.ai/free/en/papers/laubli-2018-has-machine-translation-achieved-human/) [Conference on Empirical Methods in Natural Language Processing (EMNLP)](https://md.research.elamur.ai/free/en/venues/emnlp/)**Samuel Läubli, Rico Sennrich, Martin Volk*2018 · 443 citations*
74. **[Findings of the 2013 Workshop on Statistical Machine Translation](https://md.research.elamur.ai/free/en/papers/bojar-2013-findings-2013-workshop-statistical-machine/) [Conference on Machine Translation (WMT)](https://md.research.elamur.ai/free/en/venues/wmt/)**Ondřej Bojar, Christian Buck, Chris Callison-Burch, et al.*2013 · 443 citations*
75. **[MVBench: A Comprehensive Multi-modal Video Understanding Benchmark](https://md.research.elamur.ai/free/en/papers/li-2023-mvbench-comprehensive-multi-modal-video/) [IEEE Computer Society Conference on Computer Vision and Pattern Recognition](https://md.research.elamur.ai/free/en/venues/ieee-computer-society-conference-on-computer-vision-and-pattern-recognition/)**Kunchang Li, Yali Wang, Yinan He, et al.*2023 · 442 citations*
76. **[Neural versus Phrase-Based Machine Translation Quality: a Case Study](https://md.research.elamur.ai/free/en/papers/bentivogli-2016-neural-versus-phrase-based-machine/) [Conference on Empirical Methods in Natural Language Processing (EMNLP)](https://md.research.elamur.ai/free/en/venues/emnlp/)**Luisa Bentivogli, Arianna Bisazza, Mauro Cettolo, et al.*2016 · 437 citations*
77. **[Manual and Automatic Evaluation of Machine Translation between European Languages](https://md.research.elamur.ai/free/en/papers/koehn-2006-manual-automatic-evaluation-machine-translation/) [Conference on Machine Translation (WMT)](https://md.research.elamur.ai/free/en/venues/wmt/)**Philipp Koehn, Christof Monz*2006 · 434 citations*
78. **[COMET-22: Unbabel-IST 2022 Submission for the Metrics Shared Task](https://md.research.elamur.ai/free/en/papers/rei-2022-comet-22-unbabel-ist-2022/) [Conference on Machine Translation (WMT)](https://md.research.elamur.ai/free/en/venues/wmt/)**Ricardo Rei, José G. C. de Souza, Duarte Alves, et al.*2022 · 433 citations*
79. **[How we BLESSed distributional semantic evaluation](https://md.research.elamur.ai/free/en/papers/baroni-2011-how-we-blessed-distributional-semantic/) [GEMS Workshop on GEometrical Models of Natural Language Semantics](https://md.research.elamur.ai/free/en/venues/gems-workshop-on-geometrical-models-of-natural-language-semantics/)**Marco Baroni, Alessandro Lenci*2011 · 429 citations*
80. **[Understanding Factuality in Abstractive Summarization with FRANK: A Benchmark for Factuality Metrics](https://md.research.elamur.ai/free/en/papers/pagnoni-2021-understanding-factuality-abstractive-summarization-frank/) [Conference of the North American Chapter of the Association for Computational Linguistics (NAACL)](https://md.research.elamur.ai/free/en/venues/naacl/)**Artidoro Pagnoni, Vidhisha Balachandran, Yulia Tsvetkov*2021 · 418 citations*
81. **[Lattice-based Minimum Error Rate Training for Statistical Machine Translation](https://md.research.elamur.ai/free/en/papers/macherey-2008-lattice-based-minimum-error-rate/) [Conference on Empirical Methods in Natural Language Processing (EMNLP)](https://md.research.elamur.ai/free/en/venues/emnlp/)**Wolfgang Macherey, Franz Och, Ignacio Thayer, et al.*2008 · 410 citations*
82. **[Video-MME: The First-Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analysis](https://md.research.elamur.ai/free/en/papers/fu-2024-video-mme-first-ever-comprehensive/) [IEEE Computer Society Conference on Computer Vision and Pattern Recognition](https://md.research.elamur.ai/free/en/venues/ieee-computer-society-conference-on-computer-vision-and-pattern-recognition/)**Chaoyou Fu, Yuhan Dai, Yongdong Luo, et al.*2024 · 404 citations*
83. **[Findings of the 2020 Conference on Machine Translation (WMT20)](https://md.research.elamur.ai/free/en/papers/barrault-2020-findings-2020-conference-machine-translation/) [Conference on Machine Translation (WMT)](https://md.research.elamur.ai/free/en/venues/wmt/)**Loïc Barrault, Magdalena Biesialska, Ondřej Bojar, et al.*2020 · 401 citations*
84. **[QMSum: A New Benchmark for Query-based Multi-domain Meeting Summarization](https://md.research.elamur.ai/free/en/papers/zhong-2021-qmsum-new-benchmark-query-based/) [Conference of the North American Chapter of the Association for Computational Linguistics (NAACL)](https://md.research.elamur.ai/free/en/venues/naacl/)**Ming Zhong, Da Yin, Tao Yu, et al.*2021 · 400 citations*
85. [**Long Range Arena: A Benchmark for Efficient Transformers arXiv:2011.04006**Yi Tay, Mostafa Dehghani, Samira Abnar, et al.*2020 · 399 citations*](https://md.research.elamur.ai/free/en/papers/tay-2020-long-range-arena-benchmark-efficient/)
86. **[TAT-QA: A Question Answering Benchmark on a Hybrid of Tabular and Textual Content in Finance](https://md.research.elamur.ai/free/en/papers/zhu-2021-tat-qa-question-answering-benchmark/) [Annual Meeting of the Association for Computational Linguistics (ACL)](https://md.research.elamur.ai/free/en/venues/acl/)**Fengbin Zhu, Wenqiang Lei, Youcheng Huang, et al.*2021 · 397 citations*
87. **[Cheap and Fast – But is it Good? Evaluating Non-Expert Annotations for Natural Language Tasks](https://md.research.elamur.ai/free/en/papers/snow-2008-cheap-fast-but-is-it/) [Conference on Empirical Methods in Natural Language Processing (EMNLP)](https://md.research.elamur.ai/free/en/venues/emnlp/)**Rion Snow, Brendan O'Connor, Daniel Jurafsky, et al.*2008 · 392 citations*
88. **[METEOR: An Automatic Metric for MT Evaluation with High Levels of Correlation with Human Judgments](https://md.research.elamur.ai/free/en/papers/lavie-2007-meteor-automatic-metric-mt-evaluation/) [Conference on Machine Translation (WMT)](https://md.research.elamur.ai/free/en/venues/wmt/)**Alon Lavie, Abhaya Agarwal*2007 · 389 citations*
89. [**Tangled up in BLEU: Reevaluating the Evaluation of Automatic Machine Translation Evaluation Metrics**Nitika Mathur, Timothy Baldwin, Trevor Cohn*2020 · 386 citations*](https://md.research.elamur.ai/free/en/papers/mathur-2020-tangled-up-bleu-reevaluating-evaluation/)
90. [**LegalBench: A Collaboratively Built Benchmark for Measuring Legal Reasoning in Large Language Models arXiv:2308.11462**Neel Guha; Julian Nyarko; Daniel E. Ho; Christopher Re; Adam Chilton; Aditya Narayana; Alex Chohlas-Wood; Austin Peters; Brandon Waldon; Daniel N. Rockmore; Diego Zambrano; Dmitry Talisman; Enam Hoque; Faiz Surani; Frank Fagan; Galit Sarfaty; Gregory M. Dickinson; Haggai Porat; Jason Hegland; Jessica Wu; Joe Nudell; Joel Niklaus; John Nay; Jonathan H. Choi; Kevin Tobia; Margaret Hagan; Megan Ma; Michael Livermore; Nikon Rasumov-Rahe; Nils Holzenberger; Noam Kolt; Peter Henderson; Sean Rehaag; Sharad Goel; Shang Gao; Spencer Williams; Sunny Gandhi; Tom Zur; Varun Iyer;, Zehua Li*2023 · 381 citations*](https://md.research.elamur.ai/free/en/papers/guha-2023-legalbench-collaboratively-built-benchmark-measuring/)
91. **[An Evaluation Tool for Machine Translation: Fast Evaluation for MT Research](https://md.research.elamur.ai/free/en/papers/niessen-2000-evaluation-tool-machine-translation-fast/) [Language Resources and Evaluation Conference (LREC)](https://md.research.elamur.ai/free/en/venues/lrec/)**Sonja Nießen, Franz Josef Och, Gregor Leusch, et al.*2000 · 368 citations*
92. **[Latent Hatred: A Benchmark for Understanding Implicit Hate Speech](https://md.research.elamur.ai/free/en/papers/elsherief-2021-latent-hatred-benchmark-understanding-implicit/) [Conference on Empirical Methods in Natural Language Processing (EMNLP)](https://md.research.elamur.ai/free/en/venues/emnlp/)**Mai ElSherief, Caleb Ziems, David Muchlinski, et al.*2021 · 365 citations*
93. **[The FLORES Evaluation Datasets for Low-Resource Machine Translation: Nepali–English and Sinhala–English](https://md.research.elamur.ai/free/en/papers/guzman-2019-flores-evaluation-datasets-low-resource/) [Conference on Empirical Methods in Natural Language Processing (EMNLP)](https://md.research.elamur.ai/free/en/venues/emnlp/)**Francisco Guzmán, Peng-Jen Chen, Myle Ott, et al.*2019 · 363 citations*
94. [**How Much Reading Does Reading Comprehension Require? A Critical Investigation of Popular Benchmarks**Divyansh Kaushik, Zachary C. Lipton*2018 · 361 citations*](https://md.research.elamur.ai/free/en/papers/kaushik-2018-how-much-reading-does-reading/)
95. **[XGLUE: A New Benchmark Dataset for Cross-lingual Pre-training, Understanding and Generation](https://md.research.elamur.ai/free/en/papers/liang-2020-xglue-new-benchmark-dataset-cross/) [Conference on Empirical Methods in Natural Language Processing (EMNLP)](https://md.research.elamur.ai/free/en/venues/emnlp/)**Yaobo Liang, Nan Duan, Yeyun Gong, et al.*2020 · 356 citations*
96. **[Improving Text-to-SQL Evaluation Methodology](https://md.research.elamur.ai/free/en/papers/dollak-2018-improving-text-sql-evaluation-methodology/) [Annual Meeting of the Association for Computational Linguistics (ACL)](https://md.research.elamur.ai/free/en/venues/acl/)**Catherine Finegan-Dollak, Jonathan K. Kummerfeld, Li Zhang, et al.*2018 · 350 citations*
97. **[Problems With Evaluation of Word Embeddings Using Word Similarity Tasks](https://md.research.elamur.ai/free/en/papers/faruqui-2016-problems-evaluation-word-embeddings-word/) [Workshop on Evaluating Vector Space Representations for NLP](https://md.research.elamur.ai/free/en/venues/workshop-on-evaluating-vector-space-representations-for-nlp/)**Manaal Faruqui, Yulia Tsvetkov, Pushpendre Rastogi, et al.*2016 · 334 citations*
98. **[xCOMET: Transparent Machine Translation Evaluation through Fine-grained Error Detection](https://md.research.elamur.ai/free/en/papers/guerreiro-2024-xcomet-transparent-machine-translation-evaluation/) [Transactions of the Association for Computational Linguistics (TACL)](https://md.research.elamur.ai/free/en/venues/tacl/)**Nuno M. Guerreiro, Ricardo Rei, Daan van Stigt, et al.*2024 · 333 citations*
99. [**An Evaluation Exercise for Word Alignment Proceedings of the HLT-NAACL 2003 Workshop on Building and Using Parallel Texts: Data Driven Machine Translation and Beyond**Rada Mihalcea, Ted Pedersen*2003 · 326 citations*](https://md.research.elamur.ai/free/en/papers/mihalcea-2003-evaluation-exercise-word-alignment/)
100. [**Algorithms for Scoring Coreference Chains**Amit Bagga, Breck Baldwin*1998 · 325 citations*](https://md.research.elamur.ai/free/en/papers/bagga-1998-algorithms-scoring-coreference-chains/)
101. **[Paraphrasing for Automatic Evaluation](https://md.research.elamur.ai/free/en/papers/kauchak-2006-paraphrasing-automatic-evaluation/) [Conference of the North American Chapter of the Association for Computational Linguistics (NAACL)](https://md.research.elamur.ai/free/en/venues/naacl/)**David Kauchak, Regina Barzilay*2006 · 323 citations*
102. **[Attaining the Unattainable? Reassessing Claims of Human Parity in Neural Machine Translation](https://md.research.elamur.ai/free/en/papers/toral-2018-attaining-unattainable-reassessing-claims-human/) [Conference on Machine Translation: Research Papers](https://md.research.elamur.ai/free/en/venues/conference-on-machine-translation-research-papers/)**Antonio Toral, Sheila Castilho, Ke Hu, et al.*2018 · 321 citations*
103. **[On Some Pitfalls in Automatic Evaluation and Significance Testing for MT](https://md.research.elamur.ai/free/en/papers/riezler-2005-some-pitfalls-automatic-evaluation-significance/) [ACL Workshop on Intrinsic and Extrinsic Evaluation Measures for Machine Translation and/or Summarization](https://md.research.elamur.ai/free/en/venues/acl-workshop-on-intrinsic-and-extrinsic-evaluation-measures-for-machine/)**Stefan Riezler, John T. Maxwell*2005 · 318 citations*
104. **[Overview of the Evalita 2018 Task on Automatic Misogyny Identification (AMI)](https://md.research.elamur.ai/free/en/papers/fersini-2018-overview-evalita-2018-task-automatic/) [Evaluation Campaign of Natural Language Processing and Speech Tools for Italian. Final Workshop](https://md.research.elamur.ai/free/en/venues/evalita/)**Elisabetta Fersini, Debora Nozza, Paolo Rosso*2018 · 315 citations*
105. **[Evaluating Explainable AI: Which Algorithmic Explanations Help Users Predict Model Behavior?](https://md.research.elamur.ai/free/en/papers/hase-2020-evaluating-explainable-ai-which-algorithmic/) [Annual Meeting of the Association for Computational Linguistics (ACL)](https://md.research.elamur.ai/free/en/venues/acl/)**Peter Hase, Mohit Bansal*2020 · 309 citations*
106. [**Discovering Language Model Behaviors with Model-Written Evaluations arXiv:2212.09251**Ethan Perez, Sam Ringer, Kamilė Lukošiūtė, et al.*2022 · 307 citations*](https://md.research.elamur.ai/free/en/papers/perez-2022-discovering-language-model-behaviors-model/)
107. **[An Empirical Evaluation of Knowledge Sources and Learning Algorithms for Word Sense Disambiguation](https://md.research.elamur.ai/free/en/papers/lee-2002-empirical-evaluation-knowledge-sources-learning/) [Conference on Empirical Methods in Natural Language Processing (EMNLP)](https://md.research.elamur.ai/free/en/venues/emnlp/)**Yoong Keok Lee, Hwee Tou Ng*2002 · 303 citations*
108. **[Improving Translation Quality by Discarding Most of the Phrasetable](https://md.research.elamur.ai/free/en/papers/johnson-2007-improving-translation-quality-discarding-most/) [Conference on Empirical Methods in Natural Language Processing (EMNLP)](https://md.research.elamur.ai/free/en/venues/emnlp/)**Howard Johnson, Joel Martin, George Foster, et al.*2007 · 300 citations*
109. **[BanglaBERT: Language Model Pretraining and Benchmarks for Low-Resource Language Understanding Evaluation in Bangla](https://md.research.elamur.ai/free/en/papers/bhattacharjee-2022-banglabert-language-model-pretraining-benchmarks/) [Findings of the Association for Computational Linguistics](https://md.research.elamur.ai/free/en/venues/findings/)**Abhik Bhattacharjee, Tahmid Hasan, Wasi Ahmad, et al.*2022 · 296 citations*
110. **[Re-evaluating the Role of Bleu in Machine Translation Research](https://md.research.elamur.ai/free/en/papers/burch-2006-re-evaluating-role-bleu-machine/) [Conference of the European Chapter of the Association for Computational Linguistics (EACL)](https://md.research.elamur.ai/free/en/venues/eacl/)**Chris Callison-Burch, Miles Osborne, Philipp Koehn*2006 · 295 citations*
111. **[Evaluating Verifiability in Generative Search Engines](https://md.research.elamur.ai/free/en/papers/liu-2023-evaluating-verifiability-generative-search-engines/) [Findings of the Association for Computational Linguistics](https://md.research.elamur.ai/free/en/venues/findings/)**Nelson Liu, Tianyi Zhang, Percy Liang*2023 · 294 citations*
112. **[A Benchmark Dataset for Learning to Intervene in Online Hate Speech](https://md.research.elamur.ai/free/en/papers/qian-2019-benchmark-dataset-learning-intervene-online/) [Conference on Empirical Methods in Natural Language Processing (EMNLP)](https://md.research.elamur.ai/free/en/venues/emnlp/)**Jing Qian, Anna Bethke, Yinyin Liu, et al.*2019 · 292 citations*
113. **[Diverse Few-Shot Text Classification with Multiple Metrics](https://md.research.elamur.ai/free/en/papers/yu-2018-diverse-few-shot-text-classification/) [Conference of the North American Chapter of the Association for Computational Linguistics (NAACL)](https://md.research.elamur.ai/free/en/venues/naacl/)**Mo Yu, Xiaoxiao Guo, Jinfeng Yi, et al.*2018 · 291 citations*
114. **[USR: An Unsupervised and Reference Free Evaluation Metric for Dialog Generation](https://md.research.elamur.ai/free/en/papers/mehri-2020-usr-unsupervised-reference-free-evaluation/) [Annual Meeting of the Association for Computational Linguistics (ACL)](https://md.research.elamur.ai/free/en/venues/acl/)**Shikib Mehri, Maxine Eskenazi*2020 · 284 citations*
115. **[Results of the WMT19 Metrics Shared Task: Segment-Level and Strong MT Systems Pose Big Challenges](https://md.research.elamur.ai/free/en/papers/ma-2019-results-wmt19-metrics-shared-task/) [Conference on Machine Translation (WMT)](https://md.research.elamur.ai/free/en/venues/wmt/)**Qingsong Ma, Johnny Wei, Ondřej Bojar, et al.*2019 · 284 citations*
116. **[API-Bank: A Comprehensive Benchmark for Tool-Augmented LLMs](https://md.research.elamur.ai/free/en/papers/li-2023-api-bank-comprehensive-benchmark-tool/) [Conference on Empirical Methods in Natural Language Processing (EMNLP)](https://md.research.elamur.ai/free/en/venues/emnlp/)**Minghao Li, Yingxiu Zhao, Bowen Yu, et al.*2023 · 282 citations*
117. **[A Systematic Comparison of Smoothing Techniques for Sentence-Level BLEU](https://md.research.elamur.ai/free/en/papers/chen-2014-systematic-comparison-smoothing-techniques-sentence/) [Conference on Machine Translation (WMT)](https://md.research.elamur.ai/free/en/venues/wmt/)**Boxing Chen, Colin Cherry*2014 · 282 citations*
118. **[JailbreakBench: An Open Robustness Benchmark for Jailbreaking Large Language Models](https://md.research.elamur.ai/free/en/papers/chao-2024-jailbreakbench-open-robustness-benchmark-jailbreaking/) [Conference on Neural Information Processing Systems (NeurIPS)](https://md.research.elamur.ai/free/en/venues/neural-information-processing-systems/)**Patrick Chao, Edoardo Debenedetti, Alexander Robey, et al.*2024 · 273 citations*
119. **[WikiLingua: A New Benchmark Dataset for Cross-Lingual Abstractive Summarization](https://md.research.elamur.ai/free/en/papers/ladhak-2020-wikilingua-new-benchmark-dataset-cross/) [Findings of the Association for Computational Linguistics](https://md.research.elamur.ai/free/en/venues/findings/)**Faisal Ladhak, Esin Durmus, Claire Cardie, et al.*2020 · 273 citations*
120. **[Re-examining Machine Translation Metrics for Paraphrase Identification](https://md.research.elamur.ai/free/en/papers/madnani-2012-re-examining-machine-translation-metrics/) [Conference of the North American Chapter of the Association for Computational Linguistics (NAACL)](https://md.research.elamur.ai/free/en/venues/naacl/)**Nitin Madnani, Joel Tetreault, Martin Chodorow*2012 · 272 citations*
121. [**A Challenge Set Approach to Evaluating Machine Translation**Pierre Isabelle, Colin Cherry, George Foster*2017 · 269 citations*](https://md.research.elamur.ai/free/en/papers/isabelle-2017-challenge-set-approach-evaluating-machine/)
122. [**Prometheus: Inducing Fine-grained Evaluation Capability in Language Models arXiv:2310.08491**Seungone Kim, Jamin Shin, Yejin Cho, et al.*2023 · 267 citations*](https://md.research.elamur.ai/free/en/papers/kim-2023-prometheus-inducing-fine-grained-evaluation/)
123. **[Handling Divergent Reference Texts when Evaluating Table-to-Text Generation](https://md.research.elamur.ai/free/en/papers/dhingra-2019-handling-divergent-reference-texts-when/) [Annual Meeting of the Association for Computational Linguistics (ACL)](https://md.research.elamur.ai/free/en/venues/acl/)**Bhuwan Dhingra, Manaal Faruqui, Ankur Parikh, et al.*2019 · 267 citations*
124. **[IndoNLU: Benchmark and Resources for Evaluating Indonesian Natural Language Understanding](https://md.research.elamur.ai/free/en/papers/wilie-2020-indonlu-benchmark-resources-evaluating-indonesian/) [Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics (AACL)](https://md.research.elamur.ai/free/en/venues/aacl/)**Bryan Wilie, Karissa Vincentio, Genta Indra Winata, et al.*2020 · 265 citations*
125. **[Evaluating the Underlying Gender Bias in Contextualized Word Embeddings](https://md.research.elamur.ai/free/en/papers/basta-2019-evaluating-underlying-gender-bias-contextualized/) [Workshop on Gender Bias in Natural Language Processing](https://md.research.elamur.ai/free/en/venues/gebnlp/)**Christine Basta, Marta R. Costa-jussà, Noe Casas*2019 · 265 citations*

[← prev](https://md.research.elamur.ai/free/en/topics/evaluation-benchmarks/) [1](https://md.research.elamur.ai/free/en/topics/evaluation-benchmarks/) **2** [3](https://md.research.elamur.ai/free/en/topics/evaluation-benchmarks/page/3/) … [8](https://md.research.elamur.ai/free/en/topics/evaluation-benchmarks/page/8/) [next →](https://md.research.elamur.ai/free/en/topics/evaluation-benchmarks/page/3/)