--- title: "AI Post-Editing in Production — Nunziatini 2026" description: "The paper asks how language service providers should use LLMs in real localization pipelines. 0.11 chrF. 3 references." authors: ["Mara Nunziatini", "Mercedes Speroni"] year: 2026 venue: "Proceedings of the 26th Annual Conference of the European Association for Machine Translation (Volume 2)" source: "https://aclanthology.org/2026.eamt-2.24/" licence: "CC BY-ND 4.0 (EAMT)" licence_url: "https://creativecommons.org/licenses/by-nd/4.0/" summary_licence: "Elamur Cosmic License" canonical: "https://research.elamur.ai/free/en/papers/nunziatini-2026-ai-post-editing-production-71/" --- [Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/nunziatini-2026-ai-post-editing-production-71/) [home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/4/) / AI Post-Editing in Production: A 71,262-Segment Evaluation Across Five Domains, Ten Languages and Five Systems # AI Post-Editing in Production: A 71,262-Segment Evaluation Across Five Domains, Ten Languages and Five Systems **Authors:** Mara Nunziatini and Mercedes Speroni · **Year:** 2026 · **Venue:** Proceedings of the 26th Annual Conference of the European Association for Machine Translation (Volume 2) · **Licence:** [CC BY-ND 4.0 (EAMT)](https://creativecommons.org/licenses/by-nd/4.0/) **Links:** [Publisher's page](https://aclanthology.org/2026.eamt-2.24/) · [PDF](https://aclanthology.org/2026.eamt-2.24.pdf) ## Summary The paper asks how language service providers should use LLMs in real localization pipelines. One option is AI post-editing (AIPE), where an LLM refines an existing translation. The other is direct LLM translation (LLMT). The authors compare two AIPE setups with LLMT, Google Translate and DeepL on English source content in ten target languages and five domains. The AIPE setups are Generic MT + AIPE, which edits Google Translate output, and Production AIPE, which edits a mix of TM fuzzy matches and generic or custom MT. Both AIPE and LLMT draw on RAG-retrieved style guides and human-reviewed Translation Pairs. Automatic metrics cover 71,262 production segments. Professional translators also annotated errors with DQF-MQM and ranked outputs for a stratified subset. Both AIPE variants beat all baselines. Production AIPE scores best on automatic metrics, but human judges prefer Generic MT + AIPE. LLMT is cheaper but does not reach AIPE quality. Pre-translation quality also matters: mixed inputs that include fuzzy matches produce more severe errors. This gives practical, production-scale guidance for deployment choices. ## Key points - **Data:** 71,262 segments (553,518 words) in 10 locales (fr-FR, es-ES, de-DE, zh-CN, ja-JP, pt-BR, ko-KR, it-IT, ru-RU, tr-TR) and 5 domains. Human evaluation covered 6,618 segments, judged blindly by 60 translators. - **Automatic metrics:** Production AIPE led on every measure (HTER 0.11, chrF 82.96, BLEU 70.02, COMET 0.92). Google Translate, LLMT and DeepL reached only about 0.20–0.22 HTER. - **Human ranking:** Generic MT + AIPE was ranked best in 57% of annotations, compared with 38% for DeepL. It also had the lowest major/critical error share (13.7%), against 21.7% for Production AIPE and 25.0% for DeepL. - **Fuzzy matches:** Fuzzy TM matches made up 19% of Production AIPE inputs but 33% of its severe errors. The authors therefore recommend a single, consistent pre-translation source. - **Domain and locale effects:** Medical Devices content showed the highest concentration of severe errors. Locales also differed in severity, not just in error volume; for example, ko-KR had few errors overall but a high severe share. This supports using severity-weighted evaluation. *Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.* Fields: [Linguistics & languages](https://md.research.elamur.ai/free/en/shelves/linguistics/) · [Machine translation](https://md.research.elamur.ai/free/en/shelves/machine-translation/) Topics: [🔁 Machine translation](https://md.research.elamur.ai/free/en/topics/machine-translation/) ## Cite Cite as: Mara Nunziatini and Mercedes Speroni. 2026. AI Post-Editing in Production: A 71,262-Segment Evaluation Across Five Domains, Ten Languages and Five Systems. Proceedings of the 26th Annual Conference of the European Association for Machine Translation (Volume 2). The paper itself, not this page. BibTeX from the library's record of the work. ```bibtex @inproceedings{nunziatini2026ai, title = {{AI Post-Editing in Production: A 71,262-Segment Evaluation Across Five Domains, Ten Languages and Five Systems}}, author = {Mara Nunziatini and Mercedes Speroni}, booktitle = {Proceedings of the 26th Annual Conference of the European Association for Machine Translation (Volume 2)}, year = {2026}, url = {https://aclanthology.org/2026.eamt-2.24/} } ``` - [← previous AI Corpus Linguist: More than a Year of Experience](https://md.research.elamur.ai/free/en/papers/milicka-2026-ai-corpus-linguist-more-than/) - [next → AI Safety Lost in Translation: Evaluating the Effectiveness of English-Italian Cross-Lingual LLM Safety Alignment](https://md.research.elamur.ai/free/en/papers/wu-2026-ai-safety-lost-translation-evaluating/) - [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/4/) ## References 3 references, 1 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet. 1. Jiao, Wenxiang, Wenxuan Wang, Jen-tse Huang, Xing Wang, Shuming Shi, and Zhaopeng Tu. 2023. Is ChatGPT a good translator? Yes with GPT-4 as the engine. arXiv preprint arXiv:2301.08745. [our card](https://md.research.elamur.ai/free/en/papers/jiao-2023-chatgpt-good-translator-yes-gpt/) 2. Koponen, Maarit. 2016. Is machine translation post-editing worth the effort? A survey of research into post-editing and effort. The Journal of Specialised Translation, 25:131–148. [find](https://scholar.google.com/scholar?q=Koponen%2C+Maarit%2E+2016%2E+Is+machine+translation+post%2Dediting+worth+the+effort%3F+A+survey+of+research+into+post%2Dediting+and+effort%2E+The+Journal+of+Specialised+Translation%2C+25%3A131%E2%80%93148%2E) 3. Lommel, Arle, Aljoscha Burchardt, and Hans Uszkoreit. 2014. Multidimensional quality metrics (MQM): A framework for declaring and describing translation quality metrics. Tradumàtica, 12:455– 463. [find](https://scholar.google.com/scholar?q=Lommel%2C+Arle%2C+Aljoscha+Burchardt%2C+and+Hans+Uszkoreit%2E+2014%2E+Multidimensional+quality+metrics+%28MQM%29%3A+A+framework+for+declaring+and+describing+translation+quality+metrics%2E+Traduma%CC%80tica%2C+12%3A455%E2%80%93+463%2E)