---
title: "Evaluation & benchmarks — papers, page 7 — Elamur Research"
description: "Evaluation & benchmarks: 684 papers of the Elamur Research library · page 7 of 8 · 526–625, the most cited first."
canonical: "https://research.elamur.ai/free/en/topics/evaluation-benchmarks/page/7/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/topics/evaluation-benchmarks/page/7/)

[home](https://md.research.elamur.ai/free/en/) / [topics](https://md.research.elamur.ai/free/en/topics/) / [Evaluation & benchmarks](https://md.research.elamur.ai/free/en/topics/evaluation-benchmarks/) / page 7

# 📏 Evaluation & benchmarks

684 papers · page 7 of 8 · 526–625 · the most cited first · [all topics](https://md.research.elamur.ai/free/en/topics/)

How systems are measured: benchmarks, metrics, human judgements and what the numbers mean.

## Papers (684)

[← prev](https://md.research.elamur.ai/free/en/topics/evaluation-benchmarks/page/6/) [1](https://md.research.elamur.ai/free/en/topics/evaluation-benchmarks/) … [6](https://md.research.elamur.ai/free/en/topics/evaluation-benchmarks/page/6/) **7** [8](https://md.research.elamur.ai/free/en/topics/evaluation-benchmarks/page/8/) [next →](https://md.research.elamur.ai/free/en/topics/evaluation-benchmarks/page/8/)

526. **[Ara-HOPE: Human-Centric Post-Editing Evaluation for Dialectal Arabic to Modern Standard Arabic Translation](https://md.research.elamur.ai/free/en/papers/alabdullah-2026-ara-hope-human-centric-post/) [Workshop on NLP for Similar Languages, Varieties and Dialects (VarDial)](https://md.research.elamur.ai/free/en/venues/vardial/)**Abdullah Alabdullah, Lifeng Han, Chenghua Lin*2026 · 2 citations*
527. **[Culture-Aware Machine Translation in Large Language Models: Benchmarking and Investigation](https://md.research.elamur.ai/free/en/papers/yuan-2026-culture-aware-machine-translation-large/) [Annual Meeting of the Association for Computational Linguistics (ACL)](https://md.research.elamur.ai/free/en/venues/acl/)**Zekun Yuan, Yangfan Ye, Xiaocheng Feng, et al.*2026 · 2 citations*
528. **[MTQE.en-he: Machine Translation Quality Estimation for English-Hebrew](https://md.research.elamur.ai/free/en/papers/rosenbaum-2026-mtqe-en-he-machine-translation/) [Workshop on Language Models for Low-Resource Languages](https://md.research.elamur.ai/free/en/venues/loreslm/)**Andy Rosenbaum, Assaf Siani, Ilan Kernerman*2026 · 2 citations*
529. **[PHEB: An European Portuguese High School-Level LLM Benchmark](https://md.research.elamur.ai/free/en/papers/tavares-2026-pheb-european-portuguese-high-school/) [Language Resources and Evaluation Conference (LREC)](https://md.research.elamur.ai/free/en/venues/lrec/)**Diogo C. Tavares, Rafael Ferreira, Afonso Simplício, et al.*2026 · 2 citations*
530. **[Remedy-R: Generative Reasoning for Machine Translation Evaluation without Error Annotations](https://md.research.elamur.ai/free/en/papers/tan-2026-remedy-r-generative-reasoning-machine/) [Findings of the Association for Computational Linguistics](https://md.research.elamur.ai/free/en/venues/findings/)**Shaomu Tan, Ryosuke Mitani, Ritvik Choudhary, et al.*2026 · 2 citations*
531. **[UrBLiMP: A Benchmark for Evaluating the Linguistic Competence of Large Language Models in Urdu](https://md.research.elamur.ai/free/en/papers/adeeba-2026-urblimp-benchmark-evaluating-linguistic-competence/) [Findings of the Association for Computational Linguistics](https://md.research.elamur.ai/free/en/venues/findings/)**Farah Adeeba, Brian Dillon, Hassan Sajjad, et al.*2026 · 2 citations*
532. **[When Flores Bloomz Wrong: Cross-Direction Contamination in Machine Translation Evaluation](https://md.research.elamur.ai/free/en/papers/tan-2026-when-flores-bloomz-wrong-cross/) [Conference of the European Chapter of the Association for Computational Linguistics (EACL)](https://md.research.elamur.ai/free/en/venues/eacl/)**David Tan, Pinzhen Chen, Josef van Genabith, et al.*2026 · 2 citations*
533. **[Who Watches the Watchmen? Humans Disagree With Translation Metrics on Unseen Domains](https://md.research.elamur.ai/free/en/papers/schmidt-2026-who-watches-watchmen-humans-disagree/) [Findings of the Association for Computational Linguistics](https://md.research.elamur.ai/free/en/venues/findings/)**Finn Schmidt, Jan Philip Wahle, Terry Ruas, et al.*2026 · 2 citations*
534. [**AmaSQuAD: A Benchmark for Amharic Extractive Question Answering arXiv:2502.02047**Nebiyou Daniel Hailemariam, Blessed Guda, Tsegazeab Tefferi*2025 · 2 citations*](https://md.research.elamur.ai/free/en/papers/hailemariam-2025-amasquad-benchmark-amharic-extractive-question/)
535. [**UI-Level Evaluation of ALLaM 34B: Measuring an Arabic-Centric LLM via HUMAIN Chat arXiv:2508.17378**Omer Nacar*2025 · 2 citations*](https://md.research.elamur.ai/free/en/papers/nacar-2025-ui-level-evaluation-allam-34b/)
536. **[Evaluating the Quality of a Corpus Annotation Scheme Using Pretrained Language Models](https://md.research.elamur.ai/free/en/papers/akkurt-2024-evaluating-quality-corpus-annotation-scheme/) [Language Resources and Evaluation Conference (LREC)](https://md.research.elamur.ai/free/en/venues/lrec/)**Furkan Akkurt, Onur Gungor, Büşra Marşan, et al.*2024 · 2 citations*
537. [**Evaluation of Semantic Search and its Role in Retrieved-Augmented-Generation (RAG) for Arabic Language arXiv:2403.18350**Ali Mahboub, Muhy Eddin Za'ter, Bashar Al-Rfooh, et al.*2024 · 2 citations*](https://md.research.elamur.ai/free/en/papers/mahboub-2024-evaluation-semantic-search-role-retrieved/)
538. [**On the Evaluation of Terminology Translation Errors in NMT and PB-SMT in the Legal Domain: a Study on the Translation of Arabic Legal Documents into English and French Proceedings of the Workshop on Computational Terminology in NLP and Translation Studies (ConTeNTS) Incorporating the 16th Workshop on Building and Using Comparable Corpora (BUCC)**Khadija Ait ElFqih, Johanna Monti*2023 · 2 citations*](https://md.research.elamur.ai/free/en/papers/elfqih-2023-evaluation-terminology-translation-errors-nmt/)
539. [**Evaluating Universal Dependency Parser Recovery of Predicate Argument Structure via CompChain Analysis**Sagar Indurkhya, Beracah Yankama, Robert C. Berwick*2021 · 2 citations*](https://md.research.elamur.ai/free/en/papers/indurkhya-2021-evaluating-universal-dependency-parser-recovery/)
540. [**Parser Evaluation Using Derivation Trees: A Complement to evalb**Seth Kulick, Ann Bies, Justin Mott, et al.*2014 · 2 citations*](https://md.research.elamur.ai/free/en/papers/kulick-2014-parser-evaluation-derivation-trees-complement/)
541. [**Ranking Translations using Error Analysis and Quality Estimation**Mark Fishel*2013 · 2 citations*](https://md.research.elamur.ai/free/en/papers/fishel-2013-ranking-translations-error-analysis-quality/)
542. **[A Bilingual Bimodal Benchmark for Arabic-English NLP across Grammatical Correction, Essay Scoring, Morphological Tagging, and Speech Recognition](https://md.research.elamur.ai/free/en/papers/alhafni-2026-bilingual-bimodal-benchmark-arabic-english/) [Language Resources and Evaluation Conference (LREC)](https://md.research.elamur.ai/free/en/venues/lrec/)**Bashar Alhafni, Injy Hamed, Fadhl Eryani, et al.*2026 · 1 citation*
543. [**AHA-Memes: A Fine-Grained Multimodal Benchmark for Understanding Hate in Arabic Memes arXiv:2607.27393**Mohamed Bayan Kmainasi, Ali Ezzat Shahroor, Abul Hasnat, et al.*2026 · 1 citation*](https://md.research.elamur.ai/free/en/papers/kmainasi-2026-aha-memes-fine-grained-multimodal/)
544. **[Anchoring the Judge: Curriculum-Based Adaptation and Reference-Anchored MQM for LLM-Based Machine Translation of an Unseen Low-Resource Language - A Case of Nupe](https://md.research.elamur.ai/free/en/papers/umar-2026-anchoring-judge-curriculum-based-adaptation/) [Workshop on Language Models for Low-Resource Languages](https://md.research.elamur.ai/free/en/venues/loreslm/)**Umar Baba Umar, Sulaimon Adebayo Bashir, Abdulmalik Danlami Mohammed*2026 · 1 citation*
545. [**ArabicDialectSafety: A Dialect-Aware Benchmark for Arabic Content Safety Classification arXiv:2608.01291 · EMNLP 2026 Findings**Wajdi Zaghouani, Md. Rafiul Biswas, Kholoud Khalil Aldous, et al.*2026 · 1 citation*](https://md.research.elamur.ai/free/en/papers/zaghouani-2026-arabicdialectsafety-dialect-aware-benchmark-arabic/)
546. **[Biases in Translation: Assessing Opinion Distortion in Machine Translated Texts](https://md.research.elamur.ai/free/en/papers/shafiabadi-2026-biases-translation-assessing-opinion-distortion/) [Language Resources and Evaluation Conference (LREC)](https://md.research.elamur.ai/free/en/venues/lrec/)**Nazanin Shafiabadi, François Yvon*2026 · 1 citation*
547. **[BIS Reasoning 1.0: The First Large-Scale Japanese Benchmark for Belief-Inconsistent Syllogistic Reasoning](https://md.research.elamur.ai/free/en/papers/nguyen-2026-bis-reasoning-1-0-first/) [Language Resources and Evaluation Conference (LREC)](https://md.research.elamur.ai/free/en/venues/lrec/)**Ha Thanh Nguyen, Hideyuki Tachibana, Chaoran Liu, et al.*2026 · 1 citation*
548. **[EPOP: A Benchmark Corpus for Assessing NLP Models on Structured Information Extraction in Plant Health](https://md.research.elamur.ai/free/en/papers/nedellec-2026-epop-benchmark-corpus-assessing-nlp/) [Language Resources and Evaluation Conference (LREC)](https://md.research.elamur.ai/free/en/venues/lrec/)**Claire Nedellec, Marine Courtin, Xinzhi Yao, et al.*2026 · 1 citation*
549. **[Evaluating Machine Translation Datasets for Low-Web Data Languages: A Gendered Lens](https://md.research.elamur.ai/free/en/papers/nigatu-2026-evaluating-machine-translation-datasets-low/) [Findings of the Association for Computational Linguistics](https://md.research.elamur.ai/free/en/venues/findings/)**Hellina Hailu Nigatu, Bethelhem Yemane Mamo, Bontu Fufa Balcha, et al.*2026 · 1 citation*
550. [**Evaluation of Adversarial Robustness in Arabic Language Models arXiv:2607.25814**Anwar Alajmi, Ayed Salman, Imtiaz Ahmad*2026 · 1 citation*](https://md.research.elamur.ai/free/en/papers/alajmi-2026-evaluation-adversarial-robustness-arabic-language/)
551. **[Gretino: A Greek and Latin Dataset to Benchmark Retrieval Systems in Classical Languages](https://md.research.elamur.ai/free/en/papers/toyin-2026-gretino-greek-latin-dataset-benchmark/) [Language Resources and Evaluation Conference (LREC)](https://md.research.elamur.ai/free/en/venues/lrec/)**Hawau Olamide Toyin, Federico Iezzi, Elia Scapini, et al.*2026 · 1 citation*
552. **[Hurdles of Automatic Metric for Speech Translation Evaluation](https://md.research.elamur.ai/free/en/papers/zarzu-2026-hurdles-automatic-metric-speech-translation/) [International Conference on Spoken Language Translation (IWSLT)](https://md.research.elamur.ai/free/en/venues/iwslt/)**Victor Eugen Zarzu, Vilem Zouhar*2026 · 1 citation*
553. **[Is This Idea Novel? An Automated Benchmark for Judgment of Research Ideas](https://md.research.elamur.ai/free/en/papers/schopf-2026-is-this-idea-novel-automated/) [Language Resources and Evaluation Conference (LREC)](https://md.research.elamur.ai/free/en/venues/lrec/)**Tim Schopf, Michael Färber*2026 · 1 citation*
554. **[Multi-modal, Multi-task, Multi-criteria Automatic Evaluation with Vision Language Models](https://md.research.elamur.ai/free/en/papers/oi-2026-multi-modal-multi-task-multi/) [Language Resources and Evaluation Conference (LREC)](https://md.research.elamur.ai/free/en/venues/lrec/)**Masanari Oi, Masahiro Kaneko, Naoaki Okazaki, et al.*2026 · 1 citation*
555. **[MUSCAT: MUltilingual, SCientific ConversATion Benchmark](https://md.research.elamur.ai/free/en/papers/sinhamahapatra-2026-muscat-multilingual-scientific-conversation-benchmark/) [Language Resources and Evaluation Conference (LREC)](https://md.research.elamur.ai/free/en/venues/lrec/)**Supriti Sinhamahapatra, Thai-Binh Nguyen, Yiğit Oğuz, et al.*2026 · 1 citation*
556. **[PEAR: Pairwise Evaluation for Automatic Relative Scoring in Machine Translation](https://md.research.elamur.ai/free/en/papers/proietti-2026-pear-pairwise-evaluation-automatic-relative/) [Annual Meeting of the Association for Computational Linguistics (ACL)](https://md.research.elamur.ai/free/en/venues/acl/)**Lorenzo Proietti, Roman Grundkiewicz, Matt Post*2026 · 1 citation*
557. **[Recovered in Translation: Efficient Pipeline for Automated Translation of Benchmarks and Datasets](https://md.research.elamur.ai/free/en/papers/yukhymenko-2026-recovered-translation-efficient-pipeline-automated/) [Findings of the Association for Computational Linguistics](https://md.research.elamur.ai/free/en/venues/findings/)**Hanna Yukhymenko, Anton Alexandrov, Martin Vechev*2026 · 1 citation*
558. **[Revisiting Metric Reliability for Fine-grained Evaluation of Machine Translation and Summarization in Indian Languages](https://md.research.elamur.ai/free/en/papers/yari-2026-revisiting-metric-reliability-fine-grained/) [Annual Meeting of the Association for Computational Linguistics (ACL)](https://md.research.elamur.ai/free/en/venues/acl/)**Amir Hossein Yari, Kalmit Kulkarni, Ahmad Raza Khan, et al.*2026 · 1 citation*
559. **[SALT-31: A Machine Translation Benchmark Dataset for 31 Ugandan Languages](https://md.research.elamur.ai/free/en/papers/nsumba-2026-salt-31-machine-translation-benchmark/) [Workshop on African Natural Language Processing](https://md.research.elamur.ai/free/en/venues/africanlp/)**Solomon Nsumba, Benjamin Akera, Evelyn Nafula Ouma, et al.*2026 · 1 citation*
560. **[Tie-Calibrated COMETKiwi for Speech Translation Quality Estimation: IWSLT2026 Metrics Track](https://md.research.elamur.ai/free/en/papers/shah-2026-tie-calibrated-cometkiwi-speech-translation/) [International Conference on Spoken Language Translation (IWSLT)](https://md.research.elamur.ai/free/en/venues/iwslt/)**Mubashir Hussain Shah, Aymen Fatima, Kiho Choi, et al.*2026 · 1 citation*
561. **[Towards a Diagnostic and Predictive Evaluation Methodology for Sequence Labeling Tasks](https://md.research.elamur.ai/free/en/papers/mellado-2026-diagnostic-predictive-evaluation-methodology-sequence/) [Language Resources and Evaluation Conference (LREC)](https://md.research.elamur.ai/free/en/venues/lrec/)**Elena Alvarez-Mellado, Julio Gonzalo*2026 · 1 citation*
562. **[Uncertainty Quantification for Evaluating Gender Bias in Machine Translation](https://md.research.elamur.ai/free/en/papers/staliunaite-2026-uncertainty-quantification-evaluating-gender-bias/) [Findings of the Association for Computational Linguistics](https://md.research.elamur.ai/free/en/venues/findings/)**Ieva Staliunaite, Julius Cheng, Andreas Vlachos*2026 · 1 citation*
563. [**Word Alignment-Based Evaluation of Uniform Meaning Representations**Daniel Zeman, Federica Gamba*2026 · 1 citation*](https://md.research.elamur.ai/free/en/papers/zeman-2026-word-alignment-based-evaluation-uniform/)
564. **[XQ-MEval: A Dataset with Cross-lingual Parallel Quality for Benchmarking Translation Metrics](https://md.research.elamur.ai/free/en/papers/liu-2026-xq-meval-dataset-cross-lingual/) [Findings of the Association for Computational Linguistics](https://md.research.elamur.ai/free/en/venues/findings/)**Jingxuan Liu, Zhi Qu, Jin Tei, et al.*2026 · 1 citation*
565. [**A Comparative Benchmark of a Moroccan Darija Toxicity Detection Model (Typica.ai) and Major LLM-Based Moderation APIs (OpenAI, Mistral, Anthropic) arXiv:2505.04640**Hicham Assoudi*2025 · 1 citation*](https://md.research.elamur.ai/free/en/papers/assoudi-2025-comparative-benchmark-moroccan-darija-toxicity/)
566. [**Intertextual Parallel Detection in Biblical Hebrew: A Transformer-Based Benchmark**David M. Smiley*2025 · 1 citation*](https://md.research.elamur.ai/free/en/papers/smiley-2025-intertextual-parallel-detection-biblical-hebrew/)
567. [**A New Benchmark for Evaluating Automatic Speech Recognition in the Arabic Call Domain arXiv:2403.04280**Qusai Abo Obaidah, Muhy Eddin Za'ter, Adnan Jaljuli, et al.*2024 · 1 citation*](https://md.research.elamur.ai/free/en/papers/obaidah-2024-new-benchmark-evaluating-automatic-speech/)
568. [**Allen Institute for AI @ SIGTYP 2024 Shared Task on Word Embedding Evaluation for Ancient and Historical Languages**Lester James V. Miranda*2024 · 1 citation*](https://md.research.elamur.ai/free/en/papers/miranda-2024-allen-institute-ai-sigtyp-2024/)
569. [**Noor-Ghateh: A Benchmark Dataset for Evaluating Arabic Word Segmenters in Hadith Domain arXiv:2307.09630**Huda AlShuhayeb, Behrouz Minaei-Bidgoli, Mohammad E. Shenassa, et al.*2023 · 1 citation*](https://md.research.elamur.ai/free/en/papers/alshuhayeb-2023-noor-ghateh-benchmark-dataset-evaluating/)
570. **[OSMAN — A Novel Arabic Readability Metric](https://md.research.elamur.ai/free/en/papers/haj-2016-osman-textemdash-novel-arabic-readability/) [Language Resources and Evaluation Conference (LREC)](https://md.research.elamur.ai/free/en/venues/lrec/)**Mahmoud El-Haj, Paul Rayson*2016 · 1 citation*
571. **[A Benchmark Dataset and Comparative Evaluation of Phonemized and Romanized Urdu for Text-to-Speech](https://md.research.elamur.ai/free/en/papers/shahid-2026-benchmark-dataset-comparative-evaluation-phonemized/) [Language Resources and Evaluation Conference (LREC)](https://md.research.elamur.ai/free/en/venues/lrec/)**M Kaab Bin Shahid, Muhammed Izharuddin*2026*
572. **[A Comparative Evaluation of Open-Source Models for Russian-Kazakh Translation](https://md.research.elamur.ai/free/en/papers/shanshin-2026-comparative-evaluation-open-source-models/) [Workshop on Technologies for Machine Translation of Low-Resource Languages (LoResMT)](https://md.research.elamur.ai/free/en/venues/loresmt/)**Gleb Shanshin*2026*
573. **[A Fine-Grained Linguistic Evaluation of Low-Resource Luxembourgish–English MT](https://md.research.elamur.ai/free/en/papers/rehlinger-2026-fine-grained-linguistic-evaluation-low/) [Workshop on Technologies for Machine Translation of Low-Resource Languages (LoResMT)](https://md.research.elamur.ai/free/en/venues/loresmt/)**Nils Rehlinger*2026*
574. **[A Practical Evaluation Method for Long-Form Simultaneous Speech-to-Speech Translation](https://md.research.elamur.ai/free/en/papers/xue-2026-practical-evaluation-method-long-form/) [International Conference on Spoken Language Translation (IWSLT)](https://md.research.elamur.ai/free/en/venues/iwslt/)**Yulin Xue, Siqi Ouyang, Lei Li*2026*
575. [**Almieyar: A Culturally Grounded Benchmark for Multi-Dialect Arabic Speech Recognition arXiv:2609.35564**Omid Ghahroodi, Anas Madkoor, Dima Faris Al Saudi, et al.*2026*](https://md.research.elamur.ai/free/en/papers/ghahroodi-2026-almieyar-culturally-grounded-benchmark-multi/)
576. [**AmharicStoryQA: A Multicultural Story Question Answering Benchmark in Amharic arXiv:2602.02774**Israel Abebe Azime, Abenezer Kebede Angamo, Hana Mekonen Tamiru, et al.*2026*](https://md.research.elamur.ai/free/en/papers/azime-2026-amharicstoryqa-multicultural-story-question-answering/)
577. **[Ancient Greek to Modern Greek Machine Translation: A Novel Benchmark and Fine-Tuning Experiments on LLMs and NMT Models](https://md.research.elamur.ai/free/en/papers/mavromatis-2026-ancient-greek-modern-greek-machine/) [Language Resources and Evaluation Conference (LREC)](https://md.research.elamur.ai/free/en/venues/lrec/)**Spyridon Mavromatis, Sokratis Sofianopoulos, Prokopis Prokopidis, et al.*2026*
578. **[Arabic ChartSumm: An English-to-Arabic Benchmark for Metadata-to-Text Summarization](https://md.research.elamur.ai/free/en/papers/elchafei-2026-arabic-chartsumm-english-arabic-benchmark/) [Language Resources and Evaluation Conference (LREC)](https://md.research.elamur.ai/free/en/venues/lrec/)**Passant Elchafei, Amany Fashwan*2026*
579. [**Are Arabic Benchmarks Reliable? QIMMA's Quality-First Approach to LLM Evaluation arXiv:2604.03395**Leen AlQadi, Ahmed Alzubaidi, Mohammed Alyafeai, et al.*2026*](https://md.research.elamur.ai/free/en/papers/alqadi-2026-arabic-benchmarks-reliable-qimma-s/)
580. **[Are Language Models Borrowing-Blind? A Multilingual Evaluation of Loanword Identification across 10 Languages](https://md.research.elamur.ai/free/en/papers/silva-2026-are-language-models-borrowing-blind/) [Language Resources and Evaluation Conference (LREC)](https://md.research.elamur.ai/free/en/venues/lrec/)**Merilin Sousa Silva, Sina Ahmadi*2026*
581. [**ASCAT: An Arabic Scientific Corpus and Benchmark for Advanced Translation Evaluation arXiv:2604.00015**Serry Sibaee, Khloud Al Jallad, Zineb Yousfi, et al.*2026*](https://md.research.elamur.ai/free/en/papers/sibaee-2026-ascat-arabic-scientific-corpus-benchmark/)
582. **[ASCAT: Arabic Scientific Benchmark for Advanced Translation Evaluation](https://md.research.elamur.ai/free/en/papers/sibaee-2026-ascat-arabic-scientific-benchmark-advanced/) [Workshop on Open-Source Arabic Corpora and Processing Tools (OSACT) with Shared Tasks on Arabic LLMs Hallucination and Dialect to MSA Machine Translation @ LREC-COLING](https://md.research.elamur.ai/free/en/venues/osact/)**Serry Sibaee, Khloud Al Jallad, Zineb Yousfi, et al.*2026*
583. **[Benchmark Data Contamination in Underrepresented Languages: A Comprehensive Analysis Using Brazilian Data](https://md.research.elamur.ai/free/en/papers/vilar-2026-benchmark-data-contamination-underrepresented-languages/) [Language Resources and Evaluation Conference (LREC)](https://md.research.elamur.ai/free/en/venues/lrec/)**Iriedson Souto Maior de Moraes Vilar, David Candeia Maia, João Brunet, et al.*2026*
584. [**Benchmarking Arabic — Russian Machine Translation: A Comparison of Fine-tuned NMT and Few-shot LLMs under Rich Morphology and Low Lexical Overlap arXiv:2609.29559**Mullosharaf K. Arabov*2026*](https://md.research.elamur.ai/free/en/papers/arabov-2026-benchmarking-arabic-russian-machine-translation/)
585. [**Beyond BLEU: Linguistic Invisibility and Interactional Repair Sequence in End-to-End Sign Language Translation Proceedings of the LREC 2026 12th Workshop on the Representation and Processing of Sign Languages: Language in Motion**Zirui Wang, Mayumi Bono*2026*](https://md.research.elamur.ai/free/en/papers/wang-2026-beyond-bleu-linguistic-invisibility-interactional/)
586. [**Beyond Fluency: A Rubric-Based Benchmark for Evaluating Saudi Dialect and Cultural Competence in Large Language Models arXiv:2608.29990**Ghassan Al-Sumaidaee, Sajjad Abdoli, Ahmed Rashad, et al.*2026*](https://md.research.elamur.ai/free/en/papers/al-sumaidaee-2026-fluency-rubric-based-benchmark-evaluating/)
587. [**Beyond Word Boundaries: A Hebrew Coreference Benchmark and an Evaluation Protocol for Morphologically Complex Text**Refael Shaked Greenfeld, Reut Tsarfaty*2026*](https://md.research.elamur.ai/free/en/papers/greenfeld-2026-beyond-word-boundaries-hebrew-coreference/)
588. **[Breaking the Benchmark: Revealing LLM Bias via Minimal Contextual Augmentation](https://md.research.elamur.ai/free/en/papers/miandoab-2026-breaking-benchmark-revealing-llm-bias/) [Language Resources and Evaluation Conference (LREC)](https://md.research.elamur.ai/free/en/venues/lrec/)**Kaveh Eskandari Miandoab, Mahammed Kamruzzaman, Arshia Gharooni, et al.*2026*
589. **[Can Video LLMs See Through Illusions? Video-Illusion QA Benchmark Dataset](https://md.research.elamur.ai/free/en/papers/ohira-2026-can-video-llms-see-through/) [Language Resources and Evaluation Conference (LREC)](https://md.research.elamur.ai/free/en/venues/lrec/)**Souto Ohira, Tosho Hirasawa, Mamoru Komachi*2026*
590. **[Chinese Coreference Corpus from Verb-Driven Events and Its Application for LLM Evaluation (动词驱动事件的共指关系中文语料库构建及大模型评测)](https://md.research.elamur.ai/free/en/papers/wang-2026-chinese-coreference-corpus/) [Journal of Chinese Information Processing](https://md.research.elamur.ai/free/en/venues/journal-of-chinese-information-processing/)**Haoze Wang (王皓泽), Xin Xin (辛欣)*2026*
591. **[ChineseDevBench: A Chinese Developmental Benchmark for Language Development](https://md.research.elamur.ai/free/en/papers/wang-2026-chinesedevbench-chinese-developmental-benchmark-language/) [Workshop on Cognitive Modeling and Computational Linguistics (CMCL)](https://md.research.elamur.ai/free/en/venues/cmcl/)**Shaonan Wang, Yiwen Wu, Na Li, et al.*2026*
592. **[DAMETA: An LLM Benchmark for Danish Metaphor Interpretation with Systematically Varied Distractors](https://md.research.elamur.ai/free/en/papers/schneidermann-2026-dameta-llm-benchmark-danish-metaphor/) [Language Resources and Evaluation Conference (LREC)](https://md.research.elamur.ai/free/en/venues/lrec/)**Nina Skovgaard Schneidermann, Sanni Nimb, Nathalie Carmen Hau Norman, et al.*2026*
593. **[Detecting Latin in Historical Books with Large Language Models: A Multimodal Benchmark](https://md.research.elamur.ai/free/en/papers/wu-2026-detecting-latin-historical-books-large/) [Conference of the European Chapter of the Association for Computational Linguistics (EACL)](https://md.research.elamur.ai/free/en/venues/eacl/)**Yu Wu, Ke Shu, Jonas Fischer, et al.*2026*
594. **[Diagnosing Translated Benchmarks: An Automated Quality Assurance Study of the EU20 Benchmark Suite](https://md.research.elamur.ai/free/en/papers/thellmann-2026-diagnosing-translated-benchmarks-automated-quality/) [Language Resources and Evaluation Conference (LREC)](https://md.research.elamur.ai/free/en/venues/lrec/)**Klaudia Thellmann, Bernhard Stadler, Michael Färber*2026*
595. **[Discourse Structure Variation for Large Language Model Evaluation in Text Simplification (大模型文本简化的篇章结构变化分析与评测)](https://md.research.elamur.ai/free/en/papers/zhu-2026-discourse-structure-variation/) [Journal of Chinese Information Processing](https://md.research.elamur.ai/free/en/venues/journal-of-chinese-information-processing/)**Junhui Zhu (朱君辉), Erhong Yang (杨尔弘), Haonan Zhu (朱浩楠), et al.*2026*
596. **[Dynamic Meta-Metrics: Source-Sentence Conditioned Weighting for MT Evaluation](https://md.research.elamur.ai/free/en/papers/zhang-2026-dynamic-meta-metrics-source-sentence/) [Annual Meeting of the Association for Computational Linguistics (ACL)](https://md.research.elamur.ai/free/en/venues/acl/)**Luke Zhang, Justin Vasselli, Aditya Khan, et al.*2026*
597. **[E-CONAN (Entailment, CONtradition And Neutral) Benchmarks: Arabic Textual Entailment and Natural Inference Datasets](https://md.research.elamur.ai/free/en/papers/jallad-2026-e-conan-entailment-contradition-neutral-benchmarks/) [IEEE Access](https://md.research.elamur.ai/free/en/venues/ieee-access/)**Khloud AL Jallad, Nada Ghneim, Ghaida Rebdawi*2026*
598. **[EduBench: A Portuguese Benchmark for Open-Ended Discursive Question Answering](https://md.research.elamur.ai/free/en/papers/paiola-2026-edubench-portuguese-benchmark-open-ended/) [Language Resources and Evaluation Conference (LREC)](https://md.research.elamur.ai/free/en/venues/lrec/)**Pedro Henrique Paiola, Luís Gabriel Damiati Mendes, Bruno de Oliveira Monchelato, et al.*2026*
599. **[EIFFEL: a novel benchmark to measure bias of English heavy training on French idiomatic expressions](https://md.research.elamur.ai/free/en/papers/noel-2026-eiffel-novel-benchmark-measure-bias/) [Annual Meeting of the Association for Computational Linguistics (ACL)](https://md.research.elamur.ai/free/en/venues/acl/)**Charlotte Noel, Nicholas Asher, Olivier Gouvert, et al.*2026*
600. **[Emotion Transcription in Conversation: A Benchmark for Capturing Subtle and Complex Emotional States through Natural Language](https://md.research.elamur.ai/free/en/papers/tanaka-2026-emotion-transcription-conversation-benchmark-capturing/) [Language Resources and Evaluation Conference (LREC)](https://md.research.elamur.ai/free/en/venues/lrec/)**Yoshiki Tanaka, Ryuichi Uehara, Koji Inoue, et al.*2026*
601. **[eSciBench: An Extensible Scientific PDF Extraction Benchmark](https://md.research.elamur.ai/free/en/papers/taillon-2026-escibench-extensible-scientific-pdf-extraction/) [Language Resources and Evaluation Conference (LREC)](https://md.research.elamur.ai/free/en/venues/lrec/)**Noah Tremblay Taillon, Phillippe Langlais*2026*
602. **[Evaluating Discriminability of Vision-Language Models](https://md.research.elamur.ai/free/en/papers/muraoka-2026-evaluating-discriminability-vision-language-models/) [Language Resources and Evaluation Conference (LREC)](https://md.research.elamur.ai/free/en/venues/lrec/)**Masayasu Muraoka, Naoaki Okazaki*2026*
603. **[Evaluating Frontier LLM Translation Capability for Lakota](https://md.research.elamur.ai/free/en/papers/robertson-2026-evaluating-frontier-llm-translation-capability/) [Workshop on NLP for Indigenous Languages of the Americas (AmericasNLP)](https://md.research.elamur.ai/free/en/venues/americasnlp/)**Lance Robertson*2026*
604. **[Evaluating Hierarchical Aggregation and LLM-Based Matching for Synset Selection in Ancient Greek](https://md.research.elamur.ai/free/en/papers/villa-2026-evaluating-hierarchical-aggregation-llm-based/) [Workshop on Language Technologies for Historical and Ancient Languages @ LREC](https://md.research.elamur.ai/free/en/venues/lt4hala/)**Luca Brigada Villa, Marco Passarotti, Chiara Zanchi, et al.*2026*
605. **[Evaluating Humanities Theory Alignment in Large Language Models: Incremental Prompting and Statistical Assessment](https://md.research.elamur.ai/free/en/papers/pichler-2026-evaluating-humanities-theory-alignment-large/) [Workshop on Computational Linguistics for Cultural Heritage, Social Sciences, Humanities and Literature (LaTeCH-CLfL)](https://md.research.elamur.ai/free/en/venues/latechclfl/)**Axel Pichler, Janis Pagel*2026*
606. **[Evaluating Latin and Ancient Greek Sentence Alignment through Parallel Sentence Mining](https://md.research.elamur.ai/free/en/papers/reichbauer-2026-evaluating-latin-ancient-greek-sentence/) [Workshop on Natural Language Processing for Digital Humanities](https://md.research.elamur.ai/free/en/venues/nlp4dh/)**Sebastian Reichbauer, Shu Okabe, Alexander Fraser*2026*
607. **[Evaluating LLM Style Transfer Through Readability-Based Age Assessments](https://md.research.elamur.ai/free/en/papers/maro-2026-evaluating-llm-style-transfer-through/) [International Workshop on Spoken Dialogue System Technology](https://md.research.elamur.ai/free/en/venues/iwsds/)**Maria Di Maro, Antonio Origlia, Leonilda Bilo, et al.*2026*
608. **[Evaluating Multimodal Large Language Model Narrative Interpretation through the Lens of Appraisal Theory](https://md.research.elamur.ai/free/en/papers/teotia-2026-evaluating-multimodal-large-language-model/) [Language Resources and Evaluation Conference (LREC)](https://md.research.elamur.ai/free/en/venues/lrec/)**Jayant Teotia, Xiaowei Wang, Xulang Zhang, et al.*2026*
609. **[Evaluating Native-Speaker Preferences on Machine Translation and Post-Edits for Five African Languages](https://md.research.elamur.ai/free/en/papers/oirghi-2026-evaluating-native-speaker-preferences-machine/) [Workshop on African Natural Language Processing](https://md.research.elamur.ai/free/en/venues/africanlp/)**Hiba El Oirghi, Tajuddeen Gwadabe, Marine Carpuat*2026*
610. **[Evaluating Small Language Models for English-to-Portuguese Translation: Impact of Model Scale and Quantization](https://md.research.elamur.ai/free/en/papers/tamiosso-2026-evaluating-small-language-models-english/) [International Conference on Computational Processing of Portuguese](https://md.research.elamur.ai/free/en/venues/propor/)**Gustavo Lopes Tamiosso, Rafael Oleques Nunes, Dennis Giovani Balreira*2026*
611. **[Evaluating the Adaptability of Large Language Models to Linguistic Variation](https://md.research.elamur.ai/free/en/papers/xu-2026-evaluating-adaptability-large-language-models/) [Language Resources and Evaluation Conference (LREC)](https://md.research.elamur.ai/free/en/venues/lrec/)**Ziyan Xu, Marina Seghier, Alice Millour, et al.*2026*
612. **[Evaluating the Effect of Question Wording Variations on Answer Consistency in Large Language Models](https://md.research.elamur.ai/free/en/papers/takayama-2026-evaluating-effect-question-wording-variations/) [Language Resources and Evaluation Conference (LREC)](https://md.research.elamur.ai/free/en/venues/lrec/)**Junya Takayama, Masaya Ohagi, Tomoya Mizumoto, et al.*2026*
613. **[Evaluating the Impact of Source Diversity for RAG in Historical Research](https://md.research.elamur.ai/free/en/papers/mahadeshwar-2026-evaluating-impact-source-diversity-rag/) [Language Resources and Evaluation Conference (LREC)](https://md.research.elamur.ai/free/en/venues/lrec/)**Ruhi Mahadeshwar, Andreas van Cranenburgh, Tommaso Caselli, et al.*2026*
614. **[Evaluating the Retrieval Component in a Retrieval-Augmented Summarization System for Patient Records in French](https://md.research.elamur.ai/free/en/papers/naguib-2026-evaluating-retrieval-component-retrieval-augmented/) [Clinical Natural Language Processing Workshop](https://md.research.elamur.ai/free/en/venues/clinicalnlp/)**Marco Naguib, Christel Gérardin, Victor Beaucoté, et al.*2026*
615. **[Evaluation Drift in LLM Personality Induction: Are We Moving the Goalpost?](https://md.research.elamur.ai/free/en/papers/rajput-2026-evaluation-drift-llm-personality-induction/) [Language Resources and Evaluation Conference (LREC)](https://md.research.elamur.ai/free/en/venues/lrec/)**Prateek Kumar Rajput, Yewei Song, Iyiola Emmanuel Olatunji, et al.*2026*
616. **[Evaluation of Document-Level Text Simplification in Japanese](https://md.research.elamur.ai/free/en/papers/yamashita-2026-evaluation-document-level-text-simplification/) [Language Resources and Evaluation Conference (LREC)](https://md.research.elamur.ai/free/en/venues/lrec/)**Iori Yamashita, Hikari Tanaka, Hajime Kiyama, et al.*2026*
617. **[Evaluation of Multilingual Text Simplification for the Mental Health Domain: Exploring Small Language Models](https://md.research.elamur.ai/free/en/papers/pelloni-2026-evaluation-multilingual-text-simplification-mental/) [BioNLP](https://md.research.elamur.ai/free/en/venues/bionlp/)**Olga Pelloni, Sandra Just, Lars Bongo*2026*
618. **[Evaluation of Paralinguistic-Aware Spoken Dialogue Systems using Next-Utterance Classification](https://md.research.elamur.ai/free/en/papers/miyazawa-2026-evaluation-paralinguistic-aware-spoken-dialogue/) [Annual Meeting of the Special Interest Group on Discourse and Dialogue (SIGDIAL)](https://md.research.elamur.ai/free/en/venues/sigdial/)**Kouki Miyazawa, Yoshinao Sato*2026*
619. **[Extending Czech Aspect-Based Sentiment Analysis with Opinion Terms: Dataset and LLM Benchmarks](https://md.research.elamur.ai/free/en/papers/smid-2026-extending-czech-aspect-based-sentiment/) [Language Resources and Evaluation Conference (LREC)](https://md.research.elamur.ai/free/en/venues/lrec/)**Jakub Šmíd, Pavel Priban, Pavel Kral*2026*
620. [**Extractive Summarization for Arabic Documents Using SAraBERT with a Semantic Siamese Similarity Evaluation Metric arXiv:2608.20964**Sami Shames El Deen, Mariette Awad*2026*](https://md.research.elamur.ai/free/en/papers/deen-2026-extractive-summarization-arabic-documents-using/)
621. **[Frame2KG: A Benchmark and Evaluation Toolkit for Interpretable Frame-to-Graph Generation](https://md.research.elamur.ai/free/en/papers/watson-2026-frame2kg-benchmark-evaluation-toolkit-interpretable/) [Language Resources and Evaluation Conference (LREC)](https://md.research.elamur.ai/free/en/venues/lrec/)**Lewis N. Watson, Carl Strathearn, Kenny Mitchell, et al.*2026*
622. **[From Rosetta to Match-Up: A Paired Corpus of Linguistic Puzzles with Human and LLM Benchmarks](https://md.research.elamur.ai/free/en/papers/majmudar-2026-rosetta-match-up-paired-corpus/) [Language Resources and Evaluation Conference (LREC)](https://md.research.elamur.ai/free/en/venues/lrec/)**Neh Majmudar, Anne Huang, Jinfan Frank Hu, et al.*2026*
623. **[GreekCommonGen: A Benchmark for Evaluating Generative Commonsense Reasoning in Greek](https://md.research.elamur.ai/free/en/papers/stamopoulos-2026-greekcommongen-benchmark-evaluating-generative-commonsense/) [SIGUL Joint Workshop with ELE, EURALI, and DCLRL: Towards Inclusivity and Equality: Language Resources and Technologies for Under-Resourced and Endangered Languages](https://md.research.elamur.ai/free/en/venues/sigul/)**Aristotelis Stamopoulos, Dimitrios Galanis*2026*
624. [**Harm or Humor: A Multimodal, Multilingual Benchmark for Overt and Covert Harmful Humor arXiv:2603.17759**Ahmed Sharshar, Hosam Elgendy, Saad El Dine Ahmed, et al.*2026*](https://md.research.elamur.ai/free/en/papers/sharshar-2026-harm-humor-multimodal-multilingual-benchmark/)
625. **[HERMeS: Human Evaluation & Ranking of MultiplE Systems](https://md.research.elamur.ai/free/en/papers/vanhorn-2026-hermes-human-evaluation-ranking-multiple/) [Annual Conference of the European Association for Machine Translation (EAMT)](https://md.research.elamur.ai/free/en/venues/eamt/)**Rex Vanhorn*2026*

[← prev](https://md.research.elamur.ai/free/en/topics/evaluation-benchmarks/page/6/) [1](https://md.research.elamur.ai/free/en/topics/evaluation-benchmarks/) … [6](https://md.research.elamur.ai/free/en/topics/evaluation-benchmarks/page/6/) **7** [8](https://md.research.elamur.ai/free/en/topics/evaluation-benchmarks/page/8/) [next →](https://md.research.elamur.ai/free/en/topics/evaluation-benchmarks/page/8/)