---
title: "Markus Freitag — all works — Elamur Research"
description: "All 41 works of Markus Freitag in the Elamur Research library, the most cited first: our summaries where we have them, else the original."
canonical: "https://research.elamur.ai/free/en/authors/markus-freitag/works/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/authors/markus-freitag/works/)

[home](https://md.research.elamur.ai/free/en/) / [authors](https://md.research.elamur.ai/free/en/authors/) / [Markus Freitag](https://md.research.elamur.ai/free/en/authors/markus-freitag/) / all works

# Markus Freitag — all works

41 works · [← Markus Freitag](https://md.research.elamur.ai/free/en/authors/markus-freitag/)

1. [**Experts, Errors, and Context: A Large-Scale Study of Human Evaluation for Machine Translation**Markus Freitag, George Foster, David Grangier, et al.*Machine translation · 661 citations*](https://md.research.elamur.ai/free/en/papers/freitag-2021-experts-errors-context-large-scale/)
2. [**Language Models are Multilingual Chain-of-Thought Reasoners arXiv:2210.03057**Freda Shi, Mirac Suzgun, Markus Freitag, et al.*2022 · 494 citations*](https://md.research.elamur.ai/free/en/papers/shi-2022-language-models-multilingual-chain-thought/)
3. **[Beam Search Strategies for Neural Machine Translation](https://md.research.elamur.ai/free/en/papers/freitag-2017-beam-search-strategies-neural-machine/) [Workshop on Neural Machine Translation](https://md.research.elamur.ai/free/en/venues/workshop-on-neural-machine-translation/)**Markus Freitag, Yaser Al-Onaizan*2017 · 386 citations*
4. [**Results of WMT22 Metrics Shared Task: Stop Using BLEU – Neural Metrics Are Better and More Robust**Markus Freitag, Ricardo Rei, Nitika Mathur, et al.*Machine translation · 265 citations*](https://md.research.elamur.ai/free/en/papers/freitag-2022-results-wmt22-metrics-shared-task/)
5. **[BLEU might be Guilty but References are not Innocent](https://md.research.elamur.ai/free/en/papers/freitag-2020-bleu-might-be-guilty-but/) [Conference on Empirical Methods in Natural Language Processing (EMNLP)](https://md.research.elamur.ai/free/en/venues/emnlp/)**Markus Freitag, David Grangier, Isaac Caswell*2020 · 225 citations*
6. **[Results of the WMT20 Metrics Shared Task](https://md.research.elamur.ai/free/en/papers/mathur-2020-results-wmt20-metrics-shared-task/) [Conference on Machine Translation (WMT)](https://md.research.elamur.ai/free/en/venues/wmt/)**Nitika Mathur, Johnny Wei, Markus Freitag, et al.*2020 · 195 citations*
7. **[Findings of the 2021 Conference on Machine Translation (WMT21)](https://md.research.elamur.ai/free/en/papers/akhbardeh-2021-findings-2021-conference-machine-translation/) [Conference on Machine Translation (WMT)](https://md.research.elamur.ai/free/en/venues/wmt/)**Farhad Akhbardeh, Arkady Arkhangorodsky, Magdalena Biesialska, et al.*2021 · 189 citations*
8. **[Findings of the WMT24 General Machine Translation Shared Task: The LLM Era Is Here but MT Is Not Solved Yet](https://md.research.elamur.ai/free/en/papers/kocmi-2024-findings-wmt24-general-machine-translation/) [Conference on Machine Translation (WMT)](https://md.research.elamur.ai/free/en/venues/wmt/)**Tom Kocmi, Eleftherios Avramidis, Rachel Bawden, et al.*2024 · 178 citations*
9. **[Results of the WMT21 Metrics Shared Task: Evaluating Metrics with Expert-based Human Evaluations on TED and News Domain](https://md.research.elamur.ai/free/en/papers/freitag-2021-results-wmt21-metrics-shared-task/) [Conference on Machine Translation (WMT)](https://md.research.elamur.ai/free/en/venues/wmt/)**Markus Freitag, Ricardo Rei, Nitika Mathur, et al.*2021 · 169 citations*
10. **[The Devil Is in the Errors: Leveraging Large Language Models for Fine-grained Machine Translation Evaluation](https://md.research.elamur.ai/free/en/papers/fernandes-2023-devil-is-errors-leveraging-large/) [Conference on Machine Translation (WMT)](https://md.research.elamur.ai/free/en/venues/wmt/)**Patrick Fernandes, Daniel Deutsch, Mara Finkelstein, et al.*2023 · 158 citations*
11. **[Results of WMT23 Metrics Shared Task: Metrics Might Be Guilty but References Are Not Innocent](https://md.research.elamur.ai/free/en/papers/freitag-2023-results-wmt23-metrics-shared-task/) [Conference on Machine Translation (WMT)](https://md.research.elamur.ai/free/en/venues/wmt/)**Markus Freitag, Nitika Mathur, Chi-kiu Lo, et al.*2023 · 155 citations*
12. **[INSTRUCTSCORE: Towards Explainable Text Generation Evaluation with Automatic Feedback](https://md.research.elamur.ai/free/en/papers/xu-2023-instructscore-explainable-text-generation-evaluation/) [Conference on Empirical Methods in Natural Language Processing (EMNLP)](https://md.research.elamur.ai/free/en/venues/emnlp/)**Wenda Xu, Danqing Wang, Liangming Pan, et al.*2023 · 123 citations*
13. **[High Quality Rather than High Model Probability: Minimum Bayes Risk Decoding with Neural Metrics](https://md.research.elamur.ai/free/en/papers/freitag-2022-high-quality-rather-than-high/) [Transactions of the Association for Computational Linguistics (TACL)](https://md.research.elamur.ai/free/en/venues/tacl/)**Markus Freitag, David Grangier, Qijun Tan, et al.*2022 · 121 citations*
14. **[MetricX-24: The Google Submission to the WMT 2024 Metrics Shared Task](https://md.research.elamur.ai/free/en/papers/juraska-2024-metricx-24-google-submission-wmt/) [Conference on Machine Translation (WMT)](https://md.research.elamur.ai/free/en/venues/wmt/)**Juraj Juraska, Daniel Deutsch, Mara Finkelstein, et al.*2024 · 118 citations*
15. **[MetricX-23: The Google Submission to the WMT 2023 Metrics Shared Task](https://md.research.elamur.ai/free/en/papers/juraska-2023-metricx-23-google-submission-wmt/) [Conference on Machine Translation (WMT)](https://md.research.elamur.ai/free/en/venues/wmt/)**Juraj Juraska, Mara Finkelstein, Daniel Deutsch, et al.*2023 · 118 citations*
16. **[APE at Scale and Its Implications on MT Evaluation Biases](https://md.research.elamur.ai/free/en/papers/freitag-2019-ape-scale-its-implications-mt/) [Conference on Machine Translation (WMT)](https://md.research.elamur.ai/free/en/venues/wmt/)**Markus Freitag, Isaac Caswell, Scott Roy*2019 · 108 citations*
17. **[Are LLMs Breaking MT Metrics? Results of the WMT24 Metrics Shared Task](https://md.research.elamur.ai/free/en/papers/freitag-2024-are-llms-breaking-mt-metrics/) [Conference on Machine Translation (WMT)](https://md.research.elamur.ai/free/en/venues/wmt/)**Markus Freitag, Nitika Mathur, Daniel Deutsch, et al.*2024 · 107 citations*
18. **[Ties Matter: Meta-Evaluating Modern Metrics with Pairwise Accuracy and Tie Calibration](https://md.research.elamur.ai/free/en/papers/deutsch-2023-ties-matter-meta-evaluating-modern/) [Conference on Empirical Methods in Natural Language Processing (EMNLP)](https://md.research.elamur.ai/free/en/venues/emnlp/)**Daniel Deutsch, George Foster, Markus Freitag*2023 · 76 citations*
19. **[Findings of the WMT25 General Machine Translation Shared Task: Time to Stop Evaluating on Easy Test Sets](https://md.research.elamur.ai/free/en/papers/kocmi-2025-findings-wmt25-general-machine-translation/) [Conference on Machine Translation (WMT)](https://md.research.elamur.ai/free/en/venues/wmt/)**Tom Kocmi, Ekaterina Artemova, Eleftherios Avramidis, et al.*2025 · 73 citations*
20. **[Learning to Evaluate Translation Beyond English: BLEURT Submissions to the WMT Metrics 2020 Shared Task](https://md.research.elamur.ai/free/en/papers/sellam-2020-learning-evaluate-translation-beyond-english/) [Conference on Machine Translation (WMT)](https://md.research.elamur.ai/free/en/venues/wmt/)**Thibault Sellam, Amy Pu, Hyung Won Chung, et al.*2020 · 55 citations*
21. **[Translating Step-by-Step: Decomposing the Translation Process for Improved Translation Quality of Long-Form Texts](https://md.research.elamur.ai/free/en/papers/briakou-2024-translating-step-step-decomposing-translation/) [Conference on Machine Translation (WMT)](https://md.research.elamur.ai/free/en/venues/wmt/)**Eleftheria Briakou, Jiaming Luo, Colin Cherry, et al.*2024 · 46 citations*
22. **[Findings of the WMT25 Shared Task on Automated Translation Evaluation Systems: Linguistic Diversity is Challenging and References Still Help ↗](https://aclanthology.org/2025.wmt-1.24/) Proceedings of the Tenth Conference on Machine Translation · no summary yet**Alon Lavie et al.*2025 · 39 citations*
23. **[Findings of the Quality Estimation Shared Task at WMT 2024: Are LLMs Closing the Gap in QE? ↗](https://aclanthology.org/2024.wmt-1.3/) Proceedings of the Ninth Conference on Machine Translation · no summary yet**Chrysoula Zerva et al.*2024 · 34 citations*
24. [**MBR and QE Finetuning: Training-time Distillation of the Best and Most Expensive Decoding Methods arXiv:2309.10966**Mara Finkelstein; Subhajit Naskar; Mehdi Mirzazadeh; Apurva Shah;, Markus Freitag*2023 · 32 citations*](https://md.research.elamur.ai/free/en/papers/finkelstein-2023-mbr-qe-finetuning-training-time/)
25. **[Training and Meta-Evaluating Machine Translation Evaluation Metrics at the Paragraph Level ↗](https://aclanthology.org/2023.wmt-1.96/) Proceedings of the Eighth Conference on Machine Translation · no summary yet**Daniel Deutsch et al.*2023 · 25 citations*
26. **[Human-Paraphrased References Improve Neural Machine Translation ↗](https://aclanthology.org/2020.wmt-1.140/) Proceedings of the Fifth Conference on Machine Translation · no summary yet**Markus Freitag et al.*2020 · 24 citations*
27. **[Discriminative Reordering Extensions for Hierarchical Phrase-Based Machine Translation ↗](https://aclanthology.org/2012.eamt-1.66/) Proceedings of the 16th Annual Conference of the European Association for Machine Translation · no summary yet**Matthias Huck et al.*2012 · 23 citations*
28. [**KoBE: Knowledge-Based Machine Translation Evaluation**Zorik Gekhman, Roee Aharoni, Genady Beryozkin, et al.*Machine translation · 20 citations*](https://md.research.elamur.ai/free/en/papers/gekhman-2020-kobe-knowledge-based-machine-translation/)
29. **[MetricX-25 and GemSpanEval: Google Translate Submissions to the WMT25 Evaluation Shared Task ↗](https://aclanthology.org/2025.wmt-1.70/) Proceedings of the Tenth Conference on Machine Translation · no summary yet**Juraj Juraska et al.*2025 · 11 citations*
30. **[Assessing Reference-Free Peer Evaluation for Machine Translation ↗](https://aclanthology.org/2021.naacl-main.91/) Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies · no summary yet**Sweta Agrawal et al.*2021 · 11 citations*
31. **[On the Implications of Verbose LLM Outputs: A Case Study in Translation Evaluation ↗](https://arxiv.org/abs/2410.00863) arXiv:2410.00863 · no summary yet**Eleftheria Briakou et al.*2024 · 9 citations*
32. **[Quality-Aware Translation Models: Efficient Generation and Quality Estimation in a Single Model ↗](https://aclanthology.org/2024.acl-long.836/) Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · no summary yet**Christian Tomani et al.*2024 · 9 citations*
33. **[Better punctuation prediction with hierarchical phrase-based translation ↗](https://aclanthology.org/2014.iwslt-papers.17/) Proceedings of the 11th International Workshop on Spoken Language Translation: Papers · no summary yet**Stephan Peitz, Markus Freitag, Hermann Ney*2014 · 6 citations*
34. **[Overestimation in LLM Evaluation: A Controlled Large-Scale Study on Data Contamination's Impact on Machine Translation ↗](https://arxiv.org/abs/2501.18771) arXiv:2501.18771 · no summary yet**Muhammed Yusuf Kocyigit et al.*2025 · 5 citations*
35. **[Mind the Gap... or Not? How Translation Errors and Evaluation Details Skew Multilingual Results](https://md.research.elamur.ai/free/en/papers/peter-2026-mind-gap-or-not-how/) [Workshop on Generation, Evaluation and Metrics (GEM)](https://md.research.elamur.ai/free/en/venues/gem/)**Jan-Thorsten Peter, David Vilar, Tobias Domhan, et al.*2026 · 4 citations*
36. **[Generating Difficult-to-Translate Texts](https://md.research.elamur.ai/free/en/papers/zouhar-2026-generating-difficult-translate-texts/) [Workshop on Multilingual Multicultural Evaluation](https://md.research.elamur.ai/free/en/venues/mme/)**Vilém Zouhar, Wenda Xu, Parker Riley, et al.*2026 · 3 citations*
37. **[Feeding Two Birds or Favoring One? Adequacy-Fluency Tradeoffs in Evaluation and Meta-Evaluation of Machine Translation ↗](https://arxiv.org/abs/2509.20287) arXiv:2509.20287 · no summary yet**Behzad Shayegh et al.*2025 · 2 citations*
38. **[Feeding Two Birds or Favoring One? Adequacy–Fluency Tradeoffs in Evaluation and Meta-Evaluation of Machine Translation ↗](https://aclanthology.org/2025.wmt-1.16/) Proceedings of the Tenth Conference on Machine Translation · no summary yet**Behzad Shayegh et al.*2025 · 2 citations*
39. **[Penalizing Length: Uncovering Systematic Bias in Quality Estimation Metrics ↗](https://arxiv.org/abs/2510.22028) arXiv:2510.22028 · no summary yet**Yilin Zhang et al.*2025 · 2 citations*
40. **[Learning from others' mistakes: Finetuning machine translation models with span-level error annotations ↗](https://arxiv.org/abs/2410.16509) arXiv:2410.16509 · no summary yet**Lily H. Zhang et al.*2024 · 1 citation*
41. **[MQM Re-Annotation: A Technique for Collaborative Evaluation of Machine Translation](https://md.research.elamur.ai/free/en/papers/riley-2026-mqm-re-annotation-technique-collaborative/) [Annual Meeting of the Association for Computational Linguistics (ACL)](https://md.research.elamur.ai/free/en/venues/acl/)**Parker Riley, Daniel Deutsch, Mara Finkelstein, et al.*2026*