--- title: "Workshop on Evaluation and Comparison of NLP Systems 2020" description: "Workshop on Evaluation and Comparison of NLP Systems 2020 — accepted papers: 3 papers in the Elamur Research library (Online), with…" canonical: "https://research.elamur.ai/free/en/venues/eval4nlp/2020/" --- [Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/venues/eval4nlp/2020/) [home](https://md.research.elamur.ai/free/en/) / [venues](https://md.research.elamur.ai/free/en/venues/) / [Workshop on Evaluation and Comparison…](https://md.research.elamur.ai/free/en/venues/eval4nlp/) / 2020 # Workshop on Evaluation and Comparison of NLP Systems 2020 — accepted papers Workshop on Evaluation and Comparison of NLP Systems 2020 is the 2020 edition of the Workshop on Evaluation and Comparison of NLP Systems, held in Online, November 2020 (publisher: Association for Computational Linguistics). The ACL Anthology lists 17 papers of it; our library holds 3 with our summaries. Below: the most cited papers, every accepted paper, and ours with their topics, authors and institutions. [**17** papers in the proceedings](#all) · [**3** works in our library](https://md.research.elamur.ai/free/en/venues/eval4nlp/2020/#works) · [**238** citations of these works](https://md.research.elamur.ai/free/en/venues/eval4nlp/2020/#works)**Online** where it was held**November 2020** when ## Most cited papers By their OpenAlex citations; a paper with our summary links to its page here, the others to the original. 1. **[Probabilistic Extension of Precision, Recall, and F1 Score for More Thorough Evaluation of Classification Models](https://aclanthology.org/2020.eval4nlp-1.9/)**Reda Yacouby, Dustin Axman*778 citations* 2. **[Fill in the BLANC: Human-free quality estimation of document summaries](https://md.research.elamur.ai/free/en/papers/vasilyev-2020-fill-blanc-human-free-quality/)**Oleg Vasilyev, Vedant Dharnidharka, John Bohannon*77 citations · our summary* 3. **[ViLBERTScore: Evaluating Image Caption Using Vision-and-Language BERT](https://aclanthology.org/2020.eval4nlp-1.4/)**Hwanhee Lee, Seunghyun Yoon, Franck Dernoncourt et al.*43 citations* 4. **[A survey on Recognizing Textual Entailment as an NLP Evaluation](https://md.research.elamur.ai/free/en/papers/poliak-2020-survey-recognizing-textual-entailment-as/)**Adam Poliak*41 citations · our summary* 5. **[Are Some Words Worth More than Others?](https://aclanthology.org/2020.eval4nlp-1.13/)**Shiran Dudy, Steven Bedrick*31 citations* 6. **[On Aligning OpenIE Extractions with Knowledge Bases: A Case Study](https://aclanthology.org/2020.eval4nlp-1.14/)**Kiril Gashteovski, Rainer Gemulla, Bhushan Kotnis et al.*22 citations* 7. **[Item Response Theory for Efficient Human Evaluation of Chatbots](https://aclanthology.org/2020.eval4nlp-1.3/)**João Sedoc, Lyle Ungar*21 citations* 8. **[Truth or Error? Towards systematic analysis of factual errors in abstractive summaries](https://aclanthology.org/2020.eval4nlp-1.1/)**Klaus-Michael Lux, Maya Sappelli, Martha Larson*14 citations* 9. **[Best Practices for Crowd-based Evaluation of German Summarization: Comparing Crowd, Expert and Automatic Evaluation](https://aclanthology.org/2020.eval4nlp-1.16/)**Neslihan Iskender, Tim Polzehl, Sebastian Möller*9 citations* 10. **[Grammaticality and Language Modelling](https://aclanthology.org/2020.eval4nlp-1.11/)**Jingcheng Niu, Gerald Penn*5 citations* ## All accepted papers (17) The most cited first; 3 with our summary link to their page here, the others to the original. 1. **[Probabilistic Extension of Precision, Recall, and F1 Score for More Thorough Evaluation of Classification Models](https://aclanthology.org/2020.eval4nlp-1.9/)**Reda Yacouby, Dustin Axman*778 citations* 2. **[Fill in the BLANC: Human-free quality estimation of document summaries](https://md.research.elamur.ai/free/en/papers/vasilyev-2020-fill-blanc-human-free-quality/)**Oleg Vasilyev, Vedant Dharnidharka, John Bohannon*77 citations · our summary* 3. **[ViLBERTScore: Evaluating Image Caption Using Vision-and-Language BERT](https://aclanthology.org/2020.eval4nlp-1.4/)**Hwanhee Lee, Seunghyun Yoon, Franck Dernoncourt et al.*43 citations* 4. **[A survey on Recognizing Textual Entailment as an NLP Evaluation](https://md.research.elamur.ai/free/en/papers/poliak-2020-survey-recognizing-textual-entailment-as/)**Adam Poliak*41 citations · our summary* 5. **[Are Some Words Worth More than Others?](https://aclanthology.org/2020.eval4nlp-1.13/)**Shiran Dudy, Steven Bedrick*31 citations* 6. **[On Aligning OpenIE Extractions with Knowledge Bases: A Case Study](https://aclanthology.org/2020.eval4nlp-1.14/)**Kiril Gashteovski, Rainer Gemulla, Bhushan Kotnis et al.*22 citations* 7. **[Item Response Theory for Efficient Human Evaluation of Chatbots](https://aclanthology.org/2020.eval4nlp-1.3/)**João Sedoc, Lyle Ungar*21 citations* 8. **[Truth or Error? Towards systematic analysis of factual errors in abstractive summaries](https://aclanthology.org/2020.eval4nlp-1.1/)**Klaus-Michael Lux, Maya Sappelli, Martha Larson*14 citations* 9. **[Best Practices for Crowd-based Evaluation of German Summarization: Comparing Crowd, Expert and Automatic Evaluation](https://aclanthology.org/2020.eval4nlp-1.16/)**Neslihan Iskender, Tim Polzehl, Sebastian Möller*9 citations* 10. **[Grammaticality and Language Modelling](https://aclanthology.org/2020.eval4nlp-1.11/)**Jingcheng Niu, Gerald Penn*5 citations* 11. **[BLEU Neighbors: A Reference-less Approach to Automatic Evaluation](https://aclanthology.org/2020.eval4nlp-1.5/)**Kawin Ethayarajh, Dorsa Sadigh*4 citations* 12. **[Evaluating Word Embeddings on Low-Resource Languages](https://md.research.elamur.ai/free/en/papers/stringham-2020-evaluating-word-embeddings-low-resource/)**Nathan Stringham, Mike Izbicki*4 citations · our summary* 13. **[Improving Text Generation Evaluation with Batch Centering and Tempered Word Mover Distance](https://aclanthology.org/2020.eval4nlp-1.6/)**Xi Chen, Nan Ding, Tomer Levinboim et al.*4 citations* 14. **[ClusterDataSplit: Exploring Challenging Clustering-Based Data Splits for Model Performance Evaluation](https://aclanthology.org/2020.eval4nlp-1.15/)**Hanna Wecker, Annemarie Friedrich, Heike Adel*3 citations* 15. **[One of these words is not like the other: a reproduction of outlier identification using non-contextual word representations](https://aclanthology.org/2020.eval4nlp-1.12/)**Jesper Brink Andersen, Mikkel Bak Bertelsen, Mikkel Hørby Schou et al.*3 citations* 16. **[Artemis: A Novel Annotation Methodology for Indicative Single Document Summarization](https://aclanthology.org/2020.eval4nlp-1.8/)**Rahul Jha, Keping Bi, Yang Li et al.*1 citation* 17. **[On the Evaluation of Machine Translation n-best Lists](https://aclanthology.org/2020.eval4nlp-1.7/)**Jacob Bremerman, Huda Khayrallah, Douglas Oard et al. ## Papers with our summary in our library (3) The most cited first (OpenAlex citations plus our library's). 1. **[Fill in the BLANC: Human-free quality estimation of document summaries](https://md.research.elamur.ai/free/en/papers/vasilyev-2020-fill-blanc-human-free-quality/) [Workshop on Evaluation and Comparison of NLP Systems](https://md.research.elamur.ai/free/en/venues/eval4nlp/)**Oleg Vasilyev, Vedant Dharnidharka, John Bohannon*2020 · 154 citations* 2. **[A survey on Recognizing Textual Entailment as an NLP Evaluation](https://md.research.elamur.ai/free/en/papers/poliak-2020-survey-recognizing-textual-entailment-as/) [Workshop on Evaluation and Comparison of NLP Systems](https://md.research.elamur.ai/free/en/venues/eval4nlp/)**Adam Poliak*2020 · 76 citations* 3. [**Evaluating Word Embeddings on Low-Resource Languages**Nathan Stringham, Mike Izbicki*2020 · 8 citations*](https://md.research.elamur.ai/free/en/papers/stringham-2020-evaluating-word-embeddings-low-resource/) ## Top topics Topics of its works in our library, with the number of works on each. [Evaluation & benchmarks (2)](https://md.research.elamur.ai/free/en/topics/evaluation-benchmarks/) · [Low-resource languages (1)](https://md.research.elamur.ai/free/en/topics/low-resource-languages/) · [Reasoning (1)](https://md.research.elamur.ai/free/en/topics/reasoning/) · [Hebrew & Aramaic (1)](https://md.research.elamur.ai/free/en/topics/hebrew-aramaic/) · [Biblical Hebrew (1)](https://md.research.elamur.ai/free/en/topics/biblical-hebrew/) ## Fields [Arts and Humanities (3)](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · [Computer Science (3)](https://md.research.elamur.ai/free/en/shelves/computer-science/) ## Top authors Authors with a page who published here most often: their works here, and the citations of those works. [🧑‍🔬**Adam Poliak**1 work here · 76 citations](https://md.research.elamur.ai/free/en/authors/adam-poliak/) ## Top institutions Institutions on the bylines of its works here, by the number of works. [🎓**Pomona College** Claremont · United States 1 work here](https://md.research.elamur.ai/free/en/universities/pomona-college/) · [🎓**Columbia University** New York · United States 1 work here](https://md.research.elamur.ai/free/en/universities/columbia-university/) · [🎓**Barnard College** New York · United States 1 work here](https://md.research.elamur.ai/free/en/universities/barnard-college/) Place and dates from the event's site or the ACL Anthology; works and citations from OpenAlex and our library. - [next → Workshop on Evaluation and Comparison of NLP Systems 2021](https://md.research.elamur.ai/free/en/venues/eval4nlp/2021/) - [↑ back to Workshop on Evaluation and Comparison of NLP Systems](https://md.research.elamur.ai/free/en/venues/eval4nlp/)