---
title: "Workshop on Evaluation and Comparison of NLP Systems 2021"
description: "Workshop on Evaluation and Comparison of NLP Systems 2021 — accepted papers: 1 paper in the Elamur Research library (Punta Cana…"
canonical: "https://research.elamur.ai/free/en/venues/eval4nlp/2021/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/venues/eval4nlp/2021/)

[home](https://md.research.elamur.ai/free/en/) / [venues](https://md.research.elamur.ai/free/en/venues/) / [Workshop on Evaluation and Comparison…](https://md.research.elamur.ai/free/en/venues/eval4nlp/) / 2021

# Workshop on Evaluation and Comparison of NLP Systems 2021 — accepted papers

Workshop on Evaluation and Comparison of NLP Systems 2021 is the 2021 edition of the Workshop on Evaluation and Comparison of NLP Systems, held in Punta Cana, Dominican Republic, November 2021 (publisher: Association for Computational Linguistics). The ACL Anthology lists 24 papers of it; our library holds 1 with our summaries. Below: the most cited papers, every accepted paper, and ours with their topics, authors and institutions.

[**24** papers in the proceedings](#all) · [**1** work in our library](https://md.research.elamur.ai/free/en/venues/eval4nlp/2021/#works) · [**9** citations of these works](https://md.research.elamur.ai/free/en/venues/eval4nlp/2021/#works)**Punta Cana, Dominican Republic** where it was held**November 2021** when

## Most cited papers

By their OpenAlex citations; a paper with our summary links to its page here, the others to the original.

1. **[HinGE: A Dataset for Generation and Evaluation of Code-Mixed Hinglish Text ↗](https://aclanthology.org/2021.eval4nlp-1.20/)**Vivek Srivastava, Mayank Singh*35 citations*
2. **[The Eval4NLP Shared Task on Explainable Quality Estimation: Overview and Results ↗](https://aclanthology.org/2021.eval4nlp-1.17/)**Marina Fomicheva, Piyawat Lertvittayakumjorn, Wei Zhao et al.*20 citations*
3. **[Statistically Significant Detection of Semantic Shifts using Contextual Word Embeddings ↗](https://aclanthology.org/2021.eval4nlp-1.11/)**Yang Liu, Alan Medlar, Dorota Glowacka*18 citations*
4. **[Developing a Benchmark for Reducing Data Bias in Authorship Attribution ↗](https://aclanthology.org/2021.eval4nlp-1.18/)**Benjamin Murauer, Günther Specht*10 citations*
5. **[StoryDB: Broad Multi-language Narrative Dataset ↗](https://aclanthology.org/2021.eval4nlp-1.4/)**Alexey Tikhonov, Igor Samenko, Ivan P. Yamshchikov*8 citations*
6. **[SeqScore: Addressing Barriers to Reproducible Named Entity Recognition Evaluation ↗](https://aclanthology.org/2021.eval4nlp-1.5/)**Chester Palen-Michel, Nolan Holley, Constantine Lignos*7 citations*
7. **[Validating Label Consistency in NER Data Annotation ↗](https://aclanthology.org/2021.eval4nlp-1.2/)**Qingkai Zeng, Mengxia Yu, Wenhao Yu et al.*7 citations*
8. **[Error Identification for Machine Translation with Metric Embedding and Attention ↗](https://aclanthology.org/2021.eval4nlp-1.15/)**Raphael Rubino, Atsushi Fujita, Benjamin Marie*6 citations*
9. **[IST-Unbabel 2021 Submission for the Explainable Quality Estimation Shared Task ↗](https://aclanthology.org/2021.eval4nlp-1.14/)**Marcos Treviso, Nuno M. Guerreiro, Ricardo Rei et al.*6 citations*
10. **[How Emotionally Stable is ALBERT? Testing Robustness with Stochastic Weight Averaging on a Sentiment Analysis Task ↗](https://aclanthology.org/2021.eval4nlp-1.3/)**Urja Khurana, Eric Nalisnick, Antske Fokkens*5 citations*

## All accepted papers (24)

The most cited first; 1 with our summary link to their page here, the others to the original.

1. **[HinGE: A Dataset for Generation and Evaluation of Code-Mixed Hinglish Text ↗](https://aclanthology.org/2021.eval4nlp-1.20/)**Vivek Srivastava, Mayank Singh*35 citations*
2. **[The Eval4NLP Shared Task on Explainable Quality Estimation: Overview and Results ↗](https://aclanthology.org/2021.eval4nlp-1.17/)**Marina Fomicheva, Piyawat Lertvittayakumjorn, Wei Zhao et al.*20 citations*
3. **[Statistically Significant Detection of Semantic Shifts using Contextual Word Embeddings ↗](https://aclanthology.org/2021.eval4nlp-1.11/)**Yang Liu, Alan Medlar, Dorota Glowacka*18 citations*
4. **[Developing a Benchmark for Reducing Data Bias in Authorship Attribution ↗](https://aclanthology.org/2021.eval4nlp-1.18/)**Benjamin Murauer, Günther Specht*10 citations*
5. **[StoryDB: Broad Multi-language Narrative Dataset ↗](https://aclanthology.org/2021.eval4nlp-1.4/)**Alexey Tikhonov, Igor Samenko, Ivan P. Yamshchikov*8 citations*
6. **[SeqScore: Addressing Barriers to Reproducible Named Entity Recognition Evaluation ↗](https://aclanthology.org/2021.eval4nlp-1.5/)**Chester Palen-Michel, Nolan Holley, Constantine Lignos*7 citations*
7. **[Validating Label Consistency in NER Data Annotation ↗](https://aclanthology.org/2021.eval4nlp-1.2/)**Qingkai Zeng, Mengxia Yu, Wenhao Yu et al.*7 citations*
8. **[Error Identification for Machine Translation with Metric Embedding and Attention ↗](https://aclanthology.org/2021.eval4nlp-1.15/)**Raphael Rubino, Atsushi Fujita, Benjamin Marie*6 citations*
9. **[IST-Unbabel 2021 Submission for the Explainable Quality Estimation Shared Task ↗](https://aclanthology.org/2021.eval4nlp-1.14/)**Marcos Treviso, Nuno M. Guerreiro, Ricardo Rei et al.*6 citations*
10. **[How Emotionally Stable is ALBERT? Testing Robustness with Stochastic Weight Averaging on a Sentiment Analysis Task ↗](https://aclanthology.org/2021.eval4nlp-1.3/)**Urja Khurana, Eric Nalisnick, Antske Fokkens*5 citations*
11. **[Writing Style Author Embedding Evaluation ↗](https://aclanthology.org/2021.eval4nlp-1.9/)**Enzo Terreau, Antoine Gourru, Julien Velcin*5 citations*
12. **[Explaining Errors in Machine Translation with Absolute Gradient Ensembles ↗](https://aclanthology.org/2021.eval4nlp-1.23/)**Melda Eksi, Erik Gelbing, Jonathan Stieber et al.*3 citations*
13. **[MIPE: A Metric Independent Pipeline for Effective Code-Mixed NLG Evaluation ↗](https://aclanthology.org/2021.eval4nlp-1.13/)**Ayush Garg, Sammed Kagi, Vivek Srivastava et al.*3 citations*
14. **[Reference-Free Word- and Sentence-Level Translation Evaluation with Token-Matching Metrics ↗](https://aclanthology.org/2021.eval4nlp-1.16/)**Christoph Wolfgang Leiter*3 citations*
15. **[Trainable Ranking Models to Evaluate the Semantic Accuracy of Data-to-Text Neural Generator ↗](https://aclanthology.org/2021.eval4nlp-1.6/)**Nicolas Garneau, Luc Lamontagne*3 citations*
16. **[ESTIME: Estimation of Summary-to-Text Inconsistency by Mismatched Embeddings ↗](https://aclanthology.org/2021.eval4nlp-1.10/)**Oleg Vasilyev, John Bohannon*2 citations*
17. **[Testing Cross-Database Semantic Parsers With Canonical Utterances ↗](https://aclanthology.org/2021.eval4nlp-1.8/)**Heather Lent, Semih Yavuz, Tao Yu et al.*2 citations*
18. **[What is SemEval evaluating? A Systematic Analysis of Evaluation Campaigns in NLP ↗](https://aclanthology.org/2021.eval4nlp-1.21/)**Oskar Wysocki, Malina Florea, Dónal Landers et al.*2 citations*
19. **[Differential Evaluation: a Qualitative Analysis of Natural Language Processing System Behavior Based Upon Data Resistance to Processing ↗](https://aclanthology.org/2021.eval4nlp-1.1/)**Lucie Gianola, Hicham El Boukkouri, Cyril Grouin et al.*1 citation*
20. **[Error-Sensitive Evaluation for Ordinal Target Variables ↗](https://aclanthology.org/2021.eval4nlp-1.19/)**David Chen, Maury Courtland, Adam Faulkner et al.*1 citation*
21. **[Explainable Quality Estimation: CUNI Eval4NLP Submission ↗](https://aclanthology.org/2021.eval4nlp-1.24/)**Peter Polák, Muskaan Singh, Ondřej Bojar*1 citation*
22. **[Referenceless Parsing-Based Evaluation of AMR-to-English Generation](https://md.research.elamur.ai/free/en/papers/manning-2021-referenceless-parsing-based-evaluation-amr/)**Emma Manning, Nathan Schneider*1 citation · our summary*
23. **[The UMD Submission to the Explainable MT Quality Estimation Shared Task: Combining Explanation Models with Sequence Labeling ↗](https://aclanthology.org/2021.eval4nlp-1.22/)**Tasnim Kabir, Marine Carpuat*1 citation*
24. **[Evaluation of Unsupervised Automatic Readability Assessors Using Rank Correlations ↗](https://aclanthology.org/2021.eval4nlp-1.7/)**Yo Ehara

## Papers with our summary in our library (1)

The most cited first (OpenAlex citations plus our library's).

1. [**Referenceless Parsing-Based Evaluation of AMR-to-English Generation**Emma Manning, Nathan Schneider*2021 · 9 citations*](https://md.research.elamur.ai/free/en/papers/manning-2021-referenceless-parsing-based-evaluation-amr/)

## Top topics

Topics of its works in our library, with the number of works on each.

[Evaluation & benchmarks (1)](https://md.research.elamur.ai/free/en/topics/evaluation-benchmarks/) · [Semantic parsing & meaning representation (1)](https://md.research.elamur.ai/free/en/topics/semantic-parsing/) · [English (1)](https://md.research.elamur.ai/free/en/topics/english/)

## Fields

[Arts and Humanities (1)](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · [Computer Science (1)](https://md.research.elamur.ai/free/en/shelves/computer-science/)

## Top authors

Authors with a page who published here most often: their works here, and the citations of those works.

[🧑‍🔬**Emma Manning**1 work here · 9 citations](https://md.research.elamur.ai/free/en/authors/emma-manning/) · [🧑‍🔬**Nathan Schneider**1 work here · 9 citations](https://md.research.elamur.ai/free/en/authors/nathan-schneider/)

## Top institutions

Institutions on the bylines of its works here, by the number of works.

[🎓**Georgetown University** Washington · United States 1 work here](https://md.research.elamur.ai/free/en/universities/georgetown-university/)

Place and dates from the event's site or the ACL Anthology; works and citations from OpenAlex and our library.

- [← previous Workshop on Evaluation and Comparison of NLP Systems 2020](https://md.research.elamur.ai/free/en/venues/eval4nlp/2020/)
- [↑ back to Workshop on Evaluation and Comparison of NLP Systems](https://md.research.elamur.ai/free/en/venues/eval4nlp/)