--- title: "HumEval 2021 — accepted papers — Elamur Research" description: "HumEval 2021 — accepted papers: 1 paper in the Elamur Research library (Online), with our summaries, ranked by citations." canonical: "https://research.elamur.ai/free/en/venues/humeval/2021/" --- [Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/venues/humeval/2021/) [home](https://md.research.elamur.ai/free/en/) / [venues](https://md.research.elamur.ai/free/en/venues/) / [HumEval](https://md.research.elamur.ai/free/en/venues/humeval/) / 2021 # HumEval 2021 — accepted papers HumEval 2021 is the 2021 edition of the Workshop on Human Evaluation of NLP Systems (HumEval), held in Online, April 2021 (publisher: Association for Computational Linguistics). The ACL Anthology lists 15 papers of it; our library holds 1 with our summaries. Below: the most cited papers, every accepted paper, and ours with their topics, authors and institutions. 📍 Online · 🗓 April 2021 [**15** papers in the proceedings](#all) · [**1** work in our library](https://md.research.elamur.ai/free/en/venues/humeval/2021/#works) · [**88** citations of these works](https://md.research.elamur.ai/free/en/venues/humeval/2021/#works) ## All accepted papers (15) The most cited first; 1 with our summary link to their page here, the others to the original. 1. **[A Case Study of Efficacy and Challenges in Practical Human-in-Loop Evaluation of NLP Systems Using Checklist ↗](https://aclanthology.org/2021.humeval-1.14/)**Shaily Bhatt, Rahul Jain, Sandipan Dandapat et al. 2. **[A Preliminary Study on Evaluating Consultation Notes With Post-Editing ↗](https://aclanthology.org/2021.humeval-1.7/)**Francesco Moramarco, Alex Papadopoulos Korfiatis, Aleksandar Savkov et al. 3. **[A View From the Crowd: Evaluation Challenges for Time-Offset Interaction Applications ↗](https://aclanthology.org/2021.humeval-1.9/)**Alberto Chierici, Nizar Habash 4. **[Detecting Post-Edited References and Their Effect on Human Evaluation ↗](https://aclanthology.org/2021.humeval-1.13/)**Věra Kloudová, Ondřej Bojar, Martin Popel 5. **[Eliciting Explicit Knowledge From Domain Experts in Direct Intrinsic Evaluation of Word Embeddings for Specialized Domains ↗](https://aclanthology.org/2021.humeval-1.12/)**Goya van Boven, Jelke Bloem 6. **[Estimating Subjective Crowd-Evaluations as an Additional Objective to Improve Natural Language Generation ↗](https://aclanthology.org/2021.humeval-1.2/)**Jakob Nyberg, Maike Paetzel, Ramesh Manuvinakurike 7. **[Interrater Disagreement Resolution: A Systematic Procedure to Reach Consensus in Annotation Tasks ↗](https://aclanthology.org/2021.humeval-1.15/)**Yvette Oortwijn, Thijs Ossenkoppele, Arianna Betti 8. **[Is This Translation Error Critical?: Classification-Based Human and Automatic Machine Translation Evaluation Focusing on Critical Errors ↗](https://aclanthology.org/2021.humeval-1.5/)**Katsuhito Sudoh, Kosuke Takahashi, Satoshi Nakamura 9. **[It's Commonsense, isn't it? Demystifying Human Evaluations in Commonsense-Enhanced NLG Systems ↗](https://aclanthology.org/2021.humeval-1.1/)**Miruna-Adriana Clinciu, Dimitra Gkatzia, Saad Mahamood 10. **[On User Interfaces for Large-Scale Document-Level Human Evaluation of Machine Translation Outputs ↗](https://aclanthology.org/2021.humeval-1.11/)**Roman Grundkiewicz, Marcin Junczys-Dowmunt, Christian Federmann et al. 11. **[Reliability of Human Evaluation for Text Summarization: Lessons Learned and Challenges Ahead ↗](https://aclanthology.org/2021.humeval-1.10/)**Neslihan Iskender, Tim Polzehl, Sebastian Möller 12. **[The Great Misalignment Problem in Human Evaluation of NLP Methods ↗](https://aclanthology.org/2021.humeval-1.8/)**Mika Hämäläinen, Khalid Alnajjar 13. **[Towards Document-Level Human MT Evaluation: On the Issues of Annotator Agreement, Effort and Misevaluation ↗](https://aclanthology.org/2021.humeval-1.4/)**Sheila Castilho 14. **[Towards Objectively Evaluating the Quality of Generated Medical Summaries ↗](https://aclanthology.org/2021.humeval-1.6/)**Francesco Moramarco, Damir Juric, Aleksandar Savkov et al. 15. **[Trading Off Diversity and Quality in Natural Language Generation](https://md.research.elamur.ai/free/en/papers/zhang-2021-trading-off-diversity-quality-natural/)**Hugh Zhang, Daniel Duckworth, Daphne Ippolito et al.*our summary* ## Papers with our summary in our library (1) The most cited first (OpenAlex citations plus our library's). 1. **[Trading Off Diversity and Quality in Natural Language Generation](https://md.research.elamur.ai/free/en/papers/zhang-2021-trading-off-diversity-quality-natural/) [Workshop on Human Evaluation of NLP Systems (HumEval)](https://md.research.elamur.ai/free/en/venues/humeval/)**Hugh Zhang, Daniel Duckworth, Daphne Ippolito, et al.*2021 · 88 citations* ## Top topics Topics of its works in our library, with the number of works on each. [Text generation (1)](https://md.research.elamur.ai/free/en/topics/text-generation/) ## Fields [Arts and Humanities (1)](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · [Computer Science (1)](https://md.research.elamur.ai/free/en/shelves/computer-science/) ## Top authors Authors with a page who published here most often: their works here, and the citations of those works. [🧑‍🔬**Arvind Neelakantan**1 work here · 88 citations](https://md.research.elamur.ai/free/en/authors/arvind-neelakantan/) · [🧑‍🔬**Daniel Duckworth**1 work here · 88 citations](https://md.research.elamur.ai/free/en/authors/daniel-duckworth/) · [🧑‍🔬**Daphne Ippolito**1 work here · 88 citations](https://md.research.elamur.ai/free/en/authors/daphne-ippolito/) · [🧑‍🔬**Hugh Zhang**1 work here · 88 citations](https://md.research.elamur.ai/free/en/authors/hugh-zhang/) ## Top institutions Institutions on the bylines of its works here, by the number of works. [🏢**Google** Mountain View · United States 1 work here](https://md.research.elamur.ai/free/en/companies/google/) · [🏢**OpenAI** San Francisco · United States 1 work here](https://md.research.elamur.ai/free/en/companies/openai/) · [🎓**University of Pennsylvania** Philadelphia · United States 1 work here](https://md.research.elamur.ai/free/en/universities/university-of-pennsylvania/) · [🎓**Stanford University** Stanford · United States 1 work here](https://md.research.elamur.ai/free/en/universities/stanford-university/) Place and dates from the event's site or the ACL Anthology; works and citations from OpenAlex and our library. - [next → HumEval 2022](https://md.research.elamur.ai/free/en/venues/humeval/2022/) - [↑ back to HumEval](https://md.research.elamur.ai/free/en/venues/humeval/)