---
title: "Markus Freitag — papers and cards"
description: "Markus Freitag: 3 paper cards in the Research library (2020–2022), with what each paper is about, the ideas Mova took from it, and the co-authors."
canonical: "https://research.elamur.ai/free/en/authors/markus-freitag/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/authors/markus-freitag/)

[home](https://md.research.elamur.ai/free/en/) / [authors](https://md.research.elamur.ai/free/en/authors/) / Markus Freitag

# Markus Freitag

3 paper cards in the Research library · 2020–2022

Markus Freitag wrote 3 of the papers Mova reads, in 2020–2022, most of them on [Machine translation](https://md.research.elamur.ai/free/en/topics/machine-translation/); often with [George Foster](https://md.research.elamur.ai/free/en/authors/george-foster/), [Wolfgang Macherey](https://md.research.elamur.ai/free/en/authors/wolfgang-macherey/), [Alon Lavie](https://md.research.elamur.ai/free/en/authors/alon-lavie/).

[OpenAlex](https://openalex.org/A5068438716) · [**h-index 19**](https://openalex.org/authors/A5068438716) · [1635 citations](https://openalex.org/authors/A5068438716) · [93 works](https://openalex.org/works?filter=author.id:A5068438716) — openalex-snapshot, 2026-10-14

The profile is the library's match of the name; two people with one name are not always told apart.

Elsewhere: [Google Scholar](https://scholar.google.com/citations?view_op=search_authors&mauthors=Markus+Freitag) · [Semantic Scholar](https://www.semanticscholar.org/search?q=Markus+Freitag) · [dblp](https://dblp.org/search?q=Markus+Freitag) · [ACL Anthology](https://aclanthology.org/search/?q=Markus+Freitag) · [ORCID](https://orcid.org/orcid-search/search?searchQuery=Markus+Freitag)

Topics: [Machine translation](https://md.research.elamur.ai/free/en/topics/machine-translation/) 3

## Papers

1. **[Results of WMT22 Metrics Shared Task: Stop Using BLEU – Neural Metrics Are Better and More Robust](https://md.research.elamur.ai/free/en/papers/freitag-2022-results-wmt22-metrics-shared-task/)** *(2022)* The shared task scored submitted MT metrics on WMT22 system outputs for three pairs (English to German, English to Russian, Chinese to English) across four domains: news, social, e-commerce and chat.
2. **[Experts, Errors, and Context: A Large-Scale Study of Human Evaluation for Machine Translation](https://md.research.elamur.ai/free/en/papers/freitag-2021-experts-errors-context-large-scale/)** *(2021)* The paper replaces the usual one-score human judgment with explicit error annotation. Professional translators, working with the full document, marked errors in the outputs of the ten top systems of WMT 2020 (human references included) for English-to-German and Chinese-to-English, using the MQM framework. The errors…
3. **[KoBE: Knowledge-Based Machine Translation Evaluation](https://md.research.elamur.ai/free/en/papers/gekhman-2020-kobe-knowledge-based-machine-translation/)** *(2020)* KoBE is a reference-free metric. Each source sentence and each candidate translation is linked to a large multilingual knowledge base, using entity linking. The score is the recall of source entities that are linked to the same knowledge-base entry in the candidate, clipped by the number of times each entity occurs…

## Co-authors (16)

- [George Foster](https://md.research.elamur.ai/free/en/authors/george-foster/) ×2
- [Wolfgang Macherey](https://md.research.elamur.ai/free/en/authors/wolfgang-macherey/) ×2
- [Alon Lavie](https://md.research.elamur.ai/free/en/authors/alon-lavie/)
- André F. T. Martins
- [Chi-kiu Lo](https://md.research.elamur.ai/free/en/authors/chi-kiu-lo/)
- [Craig Stewart](https://md.research.elamur.ai/free/en/authors/craig-stewart/)
- David Grangier
- [Eleftherios Avramidis](https://md.research.elamur.ai/free/en/authors/eleftherios-avramidis/)
- Genady Beryozkin
- [Nitika Mathur](https://md.research.elamur.ai/free/en/authors/nitika-mathur/)
- Qijun Tan
- [Ricardo Rei](https://md.research.elamur.ai/free/en/authors/ricardo-rei/)
- [Roee Aharoni](https://md.research.elamur.ai/free/en/authors/roee-aharoni/)
- Tom Kocmi
- Viresh Ratnakar
- Zorik Gekhman

## More papers in the library (21)

Pages of the Research library: our summary, the ideas, references and who cites each.

- [MQM Re-Annotation: A Technique for Collaborative Evaluation of Machine Translation](https://md.research.elamur.ai/free/en/papers/riley-2026-mqm-re-annotation-technique-collaborative/) — Parker Riley, Daniel Deutsch, Mara Finkelstein, et al. · 2026
- [Mind the Gap... or Not? How Translation Errors and Evaluation Details Skew Multilingual Results](https://md.research.elamur.ai/free/en/papers/peter-2026-mind-gap-or-not-how/) — Jan-Thorsten Peter, David Vilar, Tobias Domhan, et al. · 2026
- [Findings of the WMT25 General Machine Translation Shared Task: Time to Stop Evaluating on Easy Test Sets](https://md.research.elamur.ai/free/en/papers/kocmi-2025-findings-wmt25-general-machine-translation/) — Tom Kocmi, Ekaterina Artemova, Eleftherios Avramidis, et al. · 2025
- [Are LLMs Breaking MT Metrics? Results of the WMT24 Metrics Shared Task](https://md.research.elamur.ai/free/en/papers/freitag-2024-are-llms-breaking-mt-metrics/) — Markus Freitag, Nitika Mathur, Daniel Deutsch, et al. · 2024
- [Findings of the WMT24 General Machine Translation Shared Task: The LLM Era Is Here but MT Is Not Solved Yet](https://md.research.elamur.ai/free/en/papers/kocmi-2024-findings-wmt24-general-machine-translation/) — Tom Kocmi, Eleftherios Avramidis, Rachel Bawden, et al. · 2024
- [MetricX-24: The Google Submission to the WMT 2024 Metrics Shared Task](https://md.research.elamur.ai/free/en/papers/juraska-2024-metricx-24-google-submission-wmt/) — Juraj Juraska, Daniel Deutsch, Mara Finkelstein, et al. · 2024
- [Translating Step-by-Step: Decomposing the Translation Process for Improved Translation Quality of Long-Form Texts](https://md.research.elamur.ai/free/en/papers/briakou-2024-translating-step-step-decomposing-translation/) — Eleftheria Briakou, Jiaming Luo, Colin Cherry, et al. · 2024
- [INSTRUCTSCORE: Towards Explainable Text Generation Evaluation with Automatic Feedback](https://md.research.elamur.ai/free/en/papers/xu-2023-instructscore-explainable-text-generation-evaluation/) — Wenda Xu, Danqing Wang, Liangming Pan, et al. · 2023
- [MBR and QE Finetuning: Training-time Distillation of the Best and Most Expensive Decoding Methods](https://md.research.elamur.ai/free/en/papers/finkelstein-2023-mbr-qe-finetuning-training-time/) — Mara Finkelstein; Subhajit Naskar; Mehdi Mirzazadeh; Apurva Shah;, Markus Freitag · 2023
- [MetricX-23: The Google Submission to the WMT 2023 Metrics Shared Task](https://md.research.elamur.ai/free/en/papers/juraska-2023-metricx-23-google-submission-wmt/) — Juraj Juraska, Mara Finkelstein, Daniel Deutsch, et al. · 2023
- [Results of WMT23 Metrics Shared Task: Metrics Might Be Guilty but References Are Not Innocent](https://md.research.elamur.ai/free/en/papers/freitag-2023-results-wmt23-metrics-shared-task/) — Markus Freitag, Nitika Mathur, Chi-kiu Lo, et al. · 2023
- [The Devil Is in the Errors: Leveraging Large Language Models for Fine-grained Machine Translation Evaluation](https://md.research.elamur.ai/free/en/papers/fernandes-2023-devil-is-errors-leveraging-large/) — Patrick Fernandes, Daniel Deutsch, Mara Finkelstein, et al. · 2023
- [High Quality Rather than High Model Probability: Minimum Bayes Risk Decoding with Neural Metrics](https://md.research.elamur.ai/free/en/papers/freitag-2022-high-quality-rather-than-high/) — Markus Freitag, David Grangier, Qijun Tan, et al. · 2022
- [Language Models are Multilingual Chain-of-Thought Reasoners](https://md.research.elamur.ai/free/en/papers/shi-2022-language-models-multilingual-chain-thought/) — Freda Shi, Mirac Suzgun, Markus Freitag, et al. · 2022
- [Findings of the 2021 Conference on Machine Translation (WMT21)](https://md.research.elamur.ai/free/en/papers/akhbardeh-2021-findings-2021-conference-machine-translation/) — Farhad Akhbardeh, Arkady Arkhangorodsky, Magdalena Biesialska, et al. · 2021
- [Results of the WMT21 Metrics Shared Task: Evaluating Metrics with Expert-based Human Evaluations on TED and News Domain](https://md.research.elamur.ai/free/en/papers/freitag-2021-results-wmt21-metrics-shared-task/) — Markus Freitag, Ricardo Rei, Nitika Mathur, et al. · 2021
- [BLEU might be Guilty but References are not Innocent](https://md.research.elamur.ai/free/en/papers/freitag-2020-bleu-might-be-guilty-but/) — Markus Freitag, David Grangier, Isaac Caswell · 2020
- [Learning to Evaluate Translation Beyond English: BLEURT Submissions to the WMT Metrics 2020 Shared Task](https://md.research.elamur.ai/free/en/papers/sellam-2020-learning-evaluate-translation-beyond-english/) — Thibault Sellam, Amy Pu, Hyung Won Chung, et al. · 2020
- [Results of the WMT20 Metrics Shared Task](https://md.research.elamur.ai/free/en/papers/mathur-2020-results-wmt20-metrics-shared-task/) — Nitika Mathur, Johnny Wei, Markus Freitag, et al. · 2020
- [APE at Scale and Its Implications on MT Evaluation Biases](https://md.research.elamur.ai/free/en/papers/freitag-2019-ape-scale-its-implications-mt/) — Markus Freitag, Isaac Caswell, Scott Roy · 2019
- [Beam Search Strategies for Neural Machine Translation](https://md.research.elamur.ai/free/en/papers/freitag-2017-beam-search-strategies-neural-machine/) — Markus Freitag, Yaser Al-Onaizan · 2017