---
title: "Diversity and Homogenisation in Generative — Shterionov 2026"
description: "Large language models are now widely used for translation, but little is known about whether their output narrows the variety of language."
authors: ["Dimitar Shterionov", "Noa van Helleman", "Eva Vanmassenhove"]
year: 2026
venue: "Proceedings of the 26th Annual Conference of the European Association for Machine Translation (Volume 1)"
source: "https://aclanthology.org/2026.eamt-1.6/"
licence: "CC BY-ND 4.0 (EAMT)"
licence_url: "https://creativecommons.org/licenses/by-nd/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/shterionov-2026-diversity-homogenisation-generative-ai-translation/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/shterionov-2026-diversity-homogenisation-generative-ai-translation/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/3/) / Diversity and Homogenisation in Generative AI Translation: A Comparative Study of English-Dutch Translation Across Domains

# Diversity and Homogenisation in Generative AI Translation: A Comparative Study of English-Dutch Translation Across Domains

**Authors:** [Dimitar Shterionov](https://md.research.elamur.ai/free/en/authors/dimitar-shterionov/), Noa van Helleman, [Eva Vanmassenhove](https://md.research.elamur.ai/free/en/authors/eva-vanmassenhove/) · **Year:** 2026 · **Venue:** Proceedings of the 26th Annual Conference of the European Association for Machine Translation (Volume 1) · **Licence:** [CC BY-ND 4.0 (EAMT)](https://creativecommons.org/licenses/by-nd/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2026.eamt-1.6/) · [PDF](https://aclanthology.org/2026.eamt-1.6.pdf)

## Summary
Large language models are now widely used for translation, but little is known about whether their output narrows the variety of language. The study analyses Dutch renderings of English source texts in news, literary prose and poetry, produced by mBART, Jamba-1.5-large, GPT-4o and DeepSeek R1, a set of four LLMs capable of handling multiple languages. It compares these translations with human ones in three steps:

- **Quality:** each system is scored with standard statistical and neural MT metrics.
- **Diversity:** lexical and grammatical richness is measured.
- **Homogeneity:** per-domain logistic regression classifiers and k-means clustering on TF-IDF features test how easily the sources can be told apart.

GPT-4o obtains the best quality scores in every domain, but the diversity measures show a different pattern. In news and poetry, human translations are richer and clearly separable from machine output. In literature, the AI translations match or even exceed human diversity. The three recent LLMs are difficult to distinguish from one another despite their different architectures, which suggests cross-model convergence. The authors argue that conventional quality metrics overlook stylistic variation. They offer their approach as a framework for studying homogenisation in AI translation.

## Key points
- The setup covers the EN→NL pair, four models (mBART, Jamba-1.5-large, GPT-4o, DeepSeek R1) and three domains (News, Literature, Poetry). Test sets and code are released at github.com/dimitarsh1/LLM_LexRichness.
- Homogeneity is tested with LR classifiers that predict a text's origin (human or one of the four models) and with k-means using 5 clusters per domain, both built on TF-IDF representations.
- Jamba (Mamba-based), GPT-4o (dense transformer) and DeepSeek (MoE) are consistently harder to tell apart from each other than from the older mBART. The authors attribute this to shared training paradigms and data.
- In poetry, more than 90% of all texts, human and machine, fall into a single cluster. News and Literature clusters are somewhat more spread out.
- The authors list several limitations: a small, non-benchmark dataset, a single fixed prompt and temperature setting, a Poetry set built differently from the other domains, and simple features. Planned future work includes contextual embeddings such as BERTje, more language pairs and human evaluation.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

Fields: [Linguistics & languages](https://md.research.elamur.ai/free/en/shelves/linguistics/) · [Machine translation](https://md.research.elamur.ai/free/en/shelves/machine-translation/)

## Cite

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{shterionov2026diversity,
  title = {{Diversity and Homogenisation in Generative AI Translation: A Comparative Study of English-Dutch Translation Across Domains}},
  author = {Shterionov, Dimitar and Helleman, Noa van and Vanmassenhove, Eva},
  booktitle = {Proceedings of the 26th Annual Conference of the European Association for Machine Translation (Volume 1)},
  year = {2026},
  url = {https://aclanthology.org/2026.eamt-1.6/}
}
```

- [← previous Disentangling Annotator Skill from Verifier Strictness in Cross-Verified Dialogue Annotation](https://md.research.elamur.ai/free/en/papers/tao-2026-disentangling-annotator-skill-verifier-strictness/)
- [next → Do Diacritics Matter? Evaluating the Impact of Arabic Diacritics on Tokenization and LLM Benchmarks](https://md.research.elamur.ai/free/en/papers/aclbulk-2026-inoue-do-diacritics-matter-evaluating-impact/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/3/)

## References

57 references, 13 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Alemohammad, Sina, Josue Casco-Rodriguez, Lorenzo Luzi, Ahmed Imtiaz Humayun, Hossein Babaei, Daniel LeJeune, Ali Siahkoohi, and Richard G Baraniuk. 2023. Self-consuming generative models go mad. arXiv preprint arXiv:2307.01850, 4:14. — [arXiv:2307.01850](https://arxiv.org/abs/2307.01850) [find](https://scholar.google.com/scholar?q=Alemohammad%2C+Sina%2C+Josue+Casco%2DRodriguez%2C+Lorenzo+Luzi%2C+Ahmed+Imtiaz+Humayun%2C+Hossein+Babaei%2C+Daniel+LeJeune%2C+Ali+Siahkoohi%2C+and+Richard+G+Baraniuk%2E+2023%2E+Self%2Dconsuming+generative+models+go+mad%2E+arXi)
2. Aranberri, Nora and Jose A Pascual. 2025. Propagating machine translation traits to predict potential impact on the target language. Natural Language Processing, 31(6):1450–1469. [find](https://scholar.google.com/scholar?q=Aranberri%2C+Nora+and+Jose+A+Pascual%2E+2025%2E+Propagating+machine+translation+traits+to+predict+potential+impact+on+the+target+language%2E+Natural+Language+Processing%2C+31%286%29%3A1450%E2%80%931469%2E)
3. Arenas, Ana Guerberof and Antonio Toral. 2022. Creativity in translation: machine translation as a constraint for literary texts. CoRR, abs/2204.05655. — [arXiv:2204.05655](https://arxiv.org/abs/2204.05655) [find](https://scholar.google.com/scholar?q=Arenas%2C+Ana+Guerberof+and+Antonio+Toral%2E+2022%2E+Creativity+in+translation%3A+machine+translation+as+a+constraint+for+literary+texts%2E+CoRR%2C+abs%2F2204%2E05655%2E+%E2%80%94+%5BarXiv%3A2204%2E05655%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2204%2E)
4. Besacier, Laurent and Lane Schwartz. 2015. Automated translation of a literary work: A pilot study. In Feldman, Anna, Anna Kazantseva, Stan Szpakowicz, and Corina Koolen, editors, Proceedings of the Fourth Workshop on Computational Linguistics for Literature, pages 114–122, Denver, Colorado, USA, Ju… [find](https://scholar.google.com/scholar?q=Besacier%2C+Laurent+and+Lane+Schwartz%2E+2015%2E+Automated+translation+of+a+literary+work%3A+A+pilot+study%2E+In+Feldman%2C+Anna%2C+Anna+Kazantseva%2C+Stan+Szpakowicz%2C+and+Corina+Koolen%2C+editors%2C+Proceedings+of+the+F)
5. Edvard Tijan, Tihomir Katulic, Predrag Pale, Tihana Galinac Grbac, Nikola Filip Fijan, Adrian Boukalov, Dragan Cisic, and Vera Gradisnik, editors, 43rd International Convention on Information, Communication and Electronic Technology, MIPRO 2020, Opatija, Croatia, September 28 - October 2, 2020, page… [find](https://scholar.google.com/scholar?q=Edvard+Tijan%2C+Tihomir+Katulic%2C+Predrag+Pale%2C+Tihana+Galinac+Grbac%2C+Nikola+Filip+Fijan%2C+Adrian+Boukalov%2C+Dragan+Cisic%2C+and+Vera+Gradisnik%2C+editors%2C+43rd+International+Convention+on+Information%2C+Communi)
6. Bird, Steven, Ewan Klein, and Edward Loper. 2009. Natural language processing with Python: analyzing text with the natural language toolkit. " O’Reilly Media, Inc.". [find](https://scholar.google.com/scholar?q=Bird%2C+Steven%2C+Ewan+Klein%2C+and+Edward+Loper%2E+2009%2E+Natural+language+processing+with+Python%3A+analyzing+text+with+the+natural+language+toolkit%2E+%22+O%E2%80%99Reilly+Media%2C+Inc%2E%22%2E)
7. Fonteyne, Margot, Arda Tezcan, and Lieve Macken. 2020. Literary machine translation under the magnifying glass: Assessing the quality of an NMT-translated detective novel on document level. In Calzolari, Nicoletta, Frédéric Béchet, Philippe Blache, Khalid Choukri, Christopher Cieri, Thierry Declerck… [find](https://scholar.google.com/scholar?q=Fonteyne%2C+Margot%2C+Arda+Tezcan%2C+and+Lieve+Macken%2E+2020%2E+Literary+machine+translation+under+the+magnifying+glass%3A+Assessing+the+quality+of+an+NMT%2Dtranslated+detective+novel+on+document+level%2E+In+Calzola)
8. Brglez, Mojca and Špela Vintar. 2022. Lexical diversity in statistical and neural machine translation. Information, 13(2). [find](https://scholar.google.com/scholar?q=Brglez%2C+Mojca+and+%C5%A0pela+Vintar%2E+2022%2E+Lexical+diversity+in+statistical+and+neural+machine+translation%2E+Information%2C+13%282%29%2E)
9. Castilho, Sheila and Natália Resende. 2022. Post-editese in literary translations. Information, 13(2):66. [find](https://scholar.google.com/scholar?q=Castilho%2C+Sheila+and+Nat%C3%A1lia+Resende%2E+2022%2E+Post%2Deditese+in+literary+translations%2E+Information%2C+13%282%29%3A66%2E)
10. Chakrabarty, Tuhin, Arkadiy Saakyan, and Smaranda Muresan. 2021. Don’t go far off: An empirical study on neural poetry translation. In Moens, Marie-Francine, Xuanjing Huang, Lucia Specia, and Scott Wen-tau Yih, editors, Proceedings of the 2021 Conference on Empirical Methods in Natural Language Proc… [find](https://scholar.google.com/scholar?q=Chakrabarty%2C+Tuhin%2C+Arkadiy+Saakyan%2C+and+Smaranda+Muresan%2E+2021%2E+Don%E2%80%99t+go+far+off%3A+An+empirical+study+on+neural+poetry+translation%2E+In+Moens%2C+Marie%2DFrancine%2C+Xuanjing+Huang%2C+Lucia+Specia%2C+and+Scott+We)
11. Deutsch, Daniel, Eleftheria Briakou, Isaac Caswell, Mara Finkelstein, Rebecca Galor, Juraj Juraska, Geza Kovacs, Alison Lui, Ricardo Rei, Jason Riesa, Shruti Rijhwani, Parker Riley, Elizabeth Salesky, Firas Trabelsi, Stephanie Winkler, Biao Zhang, and Markus Freitag. 2025. Wmt24++: Expanding the lan… [find](https://scholar.google.com/scholar?q=Deutsch%2C+Daniel%2C+Eleftheria+Briakou%2C+Isaac+Caswell%2C+Mara+Finkelstein%2C+Rebecca+Galor%2C+Juraj+Juraska%2C+Geza+Kovacs%2C+Alison+Lui%2C+Ricardo+Rei%2C+Jason+Riesa%2C+Shruti+Rijhwani%2C+Parker+Riley%2C+Elizabeth+Salesky%2C)
12. Du, Shuxiang, Ana Guerberof Arenas, Antonio Toral, Kyo Gerrits, and Josep Marco Borillo. 2025. Optimising ChatGPT for creativity in literary translation: A case study from English into Dutch, Chinese, Catalan and Spanish. In Bouillon, Pierrette, Johanna Gerlach, Sabrina Girletti, Lise Volkart, Rapha… [find](https://scholar.google.com/scholar?q=Du%2C+Shuxiang%2C+Ana+Guerberof+Arenas%2C+Antonio+Toral%2C+Kyo+Gerrits%2C+and+Josep+Marco+Borillo%2E+2025%2E+Optimising+ChatGPT+for+creativity+in+literary+translation%3A+A+case+study+from+English+into+Dutch%2C+Chinese%2C)
13. Dunder, Ivan, Sanja Seljan, and Marko Pavlovski. 2020. Automatic machine translation of poetry and a low-resource language pair. In Koricic, Marko, Karolj Skala, Zeljka Car, Marina Cicin-Sain, Vlado Sruk, Dejan Skvorc, Slobodan Ribaric, Bojan Jerbic, Stjepan Gros, Boris Vrdoljak, Mladen Mauher, Gain… [find](https://scholar.google.com/scholar?q=Dunder%2C+Ivan%2C+Sanja+Seljan%2C+and+Marko+Pavlovski%2E+2020%2E+Automatic+machine+translation+of+poetry+and+a+low%2Dresource+language+pair%2E+In+Koricic%2C+Marko%2C+Karolj+Skala%2C+Zeljka+Car%2C+Marina+Cicin%2DSain%2C+Vlado+S)
14. Geng, Mingmeng, Caixi Chen, Yanru Wu, Yao Wan, Pan Zhou, and Dongping Chen. 2025. The impact of large language models in academia: from writing to speaking. In Findings of the Association for Computational Linguistics: ACL 2025, pages 19303– 19319. [find](https://scholar.google.com/scholar?q=Geng%2C+Mingmeng%2C+Caixi+Chen%2C+Yanru+Wu%2C+Yao+Wan%2C+Pan+Zhou%2C+and+Dongping+Chen%2E+2025%2E+The+impact+of+large+language+models+in+academia%3A+from+writing+to+speaking%2E+In+Findings+of+the+Association+for+Computat)
15. Ghazvininejad, Marjan, Yejin Choi, and Kevin Knight. 2018. Neural poetry translation. In Walker, Marilyn A., Heng Ji, and Amanda Stent, editors, Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT,… [find](https://scholar.google.com/scholar?q=Ghazvininejad%2C+Marjan%2C+Yejin+Choi%2C+and+Kevin+Knight%2E+2018%2E+Neural+poetry+translation%2E+In+Walker%2C+Marilyn+A%2E%2C+Heng+Ji%2C+and+Amanda+Stent%2C+editors%2C+Proceedings+of+the+2018+Conference+of+the+North+America)
16. Guerberof-Arenas, Ana and Antonio Toral. 2020. The impact of post-editing and machine translation on creativity and reading experience. Translation Spaces, 9(2):255–282. [find](https://scholar.google.com/scholar?q=Guerberof%2DArenas%2C+Ana+and+Antonio+Toral%2E+2020%2E+The+impact+of+post%2Dediting+and+machine+translation+on+creativity+and+reading+experience%2E+Translation+Spaces%2C+9%282%29%3A255%E2%80%93282%2E)
17. Guo, Yanzhu, Guokan Shang, and Chloé Clavel. 2025. Benchmarking linguistic diversity of large language models. Transactions of the Association for Computational Linguistics, 13:1507–1526, 11. [find](https://scholar.google.com/scholar?q=Guo%2C+Yanzhu%2C+Guokan+Shang%2C+and+Chlo%C3%A9+Clavel%2E+2025%2E+Benchmarking+linguistic+diversity+of+large+language+models%2E+Transactions+of+the+Association+for+Computational+Linguistics%2C+13%3A1507%E2%80%931526%2C+11%2E)
18. Hassan, Hany, Anthony Aue, C. Chen, Vishal Chowdhary, J. Clark, C. Federmann, Xuedong Huang, Marcin Junczys-Dowmunt, W. Lewis, M. Li, Shujie Liu, T. Liu, Renqian Luo, Arul Menezes, Tao Qin, F. Seide, Xu Tan, Fei Tian, Lijun Wu, Shuangzhi Wu, Yingce Xia, Dongdong Zhang, Zhirui Zhang, and M. Zhou. 201… [find](https://scholar.google.com/scholar?q=Hassan%2C+Hany%2C+Anthony+Aue%2C+C%2E+Chen%2C+Vishal+Chowdhary%2C+J%2E+Clark%2C+C%2E+Federmann%2C+Xuedong+Huang%2C+Marcin+Junczys%2DDowmunt%2C+W%2E+Lewis%2C+M%2E+Li%2C+Shujie+Liu%2C+T%2E+Liu%2C+Renqian+Luo%2C+Arul+Menezes%2C+Tao+Qin%2C+F%2E+Seide%2C+) [our card](https://md.research.elamur.ai/free/en/papers/hassan-2018-achieving-human-parity-automatic-chinese/)
19. Humblé, Philippe. 2020. Machine translation and poetry. the case of english and portuguese. Ilha do Desterro, 72:41–56, 08. [find](https://scholar.google.com/scholar?q=Humbl%C3%A9%2C+Philippe%2E+2020%2E+Machine+translation+and+poetry%2E+the+case+of+english+and+portuguese%2E+Ilha+do+Desterro%2C+72%3A41%E2%80%9356%2C+08%2E)
20. Jamba Team, Barak Lenz, Alan Arazi, Amir Bergman, Avshalom Manevich, Barak Peleg, Ben Aviram, Chen Almagor, Clara Fridman, Dan Padnos, Daniel Gissin, Daniel Jannai, Dor Muhlgay, Dor Zimberg, Edden M Gerber, Elad Dolev, Eran Krakovsky, Erez Safahi, Erez Schwartz, Gal Cohen, Gal Shachaf, Haim Rozenblu… [find](https://scholar.google.com/scholar?q=Jamba+Team%2C+Barak+Lenz%2C+Alan+Arazi%2C+Amir+Bergman%2C+Avshalom+Manevich%2C+Barak+Peleg%2C+Ben+Aviram%2C+Chen+Almagor%2C+Clara+Fridman%2C+Dan+Padnos%2C+Daniel+Gissin%2C+Daniel+Jannai%2C+Dor+Muhlgay%2C+Dor+Zimberg%2C+Edden+M+G)
21. Jiang, Liwei, Yuanjun Chai, Margaret Li, Mickel Liu, Raymond Fok, Nouha Dziri, Yulia Tsvetkov, Maarten Sap, Alon Albalak, and Yejin Choi. 2025. Artificial hivemind: The open-ended homogeneity of language models (and beyond). arXiv preprint arXiv:2510.22954. — [arXiv:2510.22954](https://arxiv.org/abs/2510.22954) [find](https://scholar.google.com/scholar?q=Jiang%2C+Liwei%2C+Yuanjun+Chai%2C+Margaret+Li%2C+Mickel+Liu%2C+Raymond+Fok%2C+Nouha+Dziri%2C+Yulia+Tsvetkov%2C+Maarten+Sap%2C+Alon+Albalak%2C+and+Yejin+Choi%2E+2025%2E+Artificial+hivemind%3A+The+open%2Dended+homogeneity+of+langu)
22. Karpinska, Marzena and Mohit Iyyer. 2023. Large language models effectively leverage document-level context for literary translation, but critical errors persist. In Koehn, Philipp, Barry Haddow, Tom Kocmi, and Christof Monz, editors, Proceedings of the Eighth Conference on Machine Translation, page… [find](https://scholar.google.com/scholar?q=Karpinska%2C+Marzena+and+Mohit+Iyyer%2E+2023%2E+Large+language+models+effectively+leverage+document%2Dlevel+context+for+literary+translation%2C+but+critical+errors+persist%2E+In+Koehn%2C+Philipp%2C+Barry+Haddow%2C+Tom+) [our card](https://md.research.elamur.ai/free/en/papers/karpinska-2023-large-language-models-effectively-leverage/)
23. Kobak, Dmitry, Rita González-Márquez, Emőke-Ágnes Horvát, and Jan Lause. 2024. Delving into chatgpt usage in academic writing through excess vocabulary. arXiv preprint arXiv:2406.07016. — [arXiv:2406.07016](https://arxiv.org/abs/2406.07016) [find](https://scholar.google.com/scholar?q=Kobak%2C+Dmitry%2C+Rita+Gonz%C3%A1lez%2DM%C3%A1rquez%2C+Emo%CC%8Bke%2D%C3%81gnes+Horv%C3%A1t%2C+and+Jan+Lause%2E+2024%2E+Delving+into+chatgpt+usage+in+academic+writing+through+excess+vocabulary%2E+arXiv+preprint+arXiv%3A2406%2E07016%2E+%E2%80%94+%5BarXiv%3A2406)
24. Kocmi, Tom, Eleftherios Avramidis, Rachel Bawden, Ondřej Bojar, Anton Dvorkovich, Christian Federmann, Mark Fishel, Markus Freitag, Thamme Gowda, Roman Grundkiewicz, Barry Haddow, Marzena Karpinska, Philipp Koehn, Benjamin Marie, Christof Monz, Kenton Murray, Masaaki Nagata, Martin Popel, Maja Popo… [find](https://scholar.google.com/scholar?q=Kocmi%2C+Tom%2C+Eleftherios+Avramidis%2C+Rachel+Bawden%2C+Ondr%CC%8Cej+Bojar%2C+Anton+Dvorkovich%2C+Christian+Federmann%2C+Mark+Fishel%2C+Markus+Freitag%2C+Thamme+Gowda%2C+Roman+Grundkiewicz%2C+Barry+Haddow%2C+Marzena+Karpinska%2C+) [our card](https://md.research.elamur.ai/free/en/papers/kocmi-2024-findings-wmt24-general-machine-translation/)
25. Kocmi, Tom, Ekaterina Artemova, Eleftherios Avramidis, Rachel Bawden, Ondřej Bojar, Konstantin Dranch, Anton Dvorkovich, Sergey Dukanov, Mark Fishel, Markus Freitag, Thamme Gowda, Roman Grundkiewicz, Barry Haddow, Marzena Karpinska, Philipp Koehn, Howard Lakougna, Jessica Lundin, Christof Monz, Ken… [find](https://scholar.google.com/scholar?q=Kocmi%2C+Tom%2C+Ekaterina+Artemova%2C+Eleftherios+Avramidis%2C+Rachel+Bawden%2C+Ondr%CC%8Cej+Bojar%2C+Konstantin+Dranch%2C+Anton+Dvorkovich%2C+Sergey+Dukanov%2C+Mark+Fishel%2C+Markus+Freitag%2C+Thamme+Gowda%2C+Roman+Grundkiewicz%2C) [our card](https://md.research.elamur.ai/free/en/papers/kocmi-2025-findings-wmt25-general-machine-translation/)
26. Liang, Weixin, Yaohui Zhang, Mihai Codreanu, Jiayu Wang, Hancheng Cao, and James Zou. 2025. The widespread adoption of large language model-assisted writing across society. Patterns, 6(12). [find](https://scholar.google.com/scholar?q=Liang%2C+Weixin%2C+Yaohui+Zhang%2C+Mihai+Codreanu%2C+Jiayu+Wang%2C+Hancheng+Cao%2C+and+James+Zou%2E+2025%2E+The+widespread+adoption+of+large+language+model%2Dassisted+writing+across+society%2E+Patterns%2C+6%2812%29%2E)
27. Luo, Jiaming, Colin Cherry, and George Foster. 2024. To diverge or not to diverge: A morphosyntactic perspective on machine translation vs human translation. Transactions of the Association for Computational Linguistics, 12:355–371. [find](https://scholar.google.com/scholar?q=Luo%2C+Jiaming%2C+Colin+Cherry%2C+and+George+Foster%2E+2024%2E+To+diverge+or+not+to+diverge%3A+A+morphosyntactic+perspective+on+machine+translation+vs+human+translation%2E+Transactions+of+the+Association+for+Comput)
28. Macken, Lieve, Orphée De Clercq, and Hans Paulussen. 2011. Dutch parallel corpus: a balanced copyright-cleared parallel corpus. META, 56(2):374–390. [find](https://scholar.google.com/scholar?q=Macken%2C+Lieve%2C+Orph%C3%A9e+De+Clercq%2C+and+Hans+Paulussen%2E+2011%2E+Dutch+parallel+corpus%3A+a+balanced+copyright%2Dcleared+parallel+corpus%2E+META%2C+56%282%29%3A374%E2%80%93390%2E)
29. Macken, Lieve. 2024. Machine translation meets large language models: Evaluating ChatGPT’s ability to automatically post-edit literary texts. In Vanroy, Bram, Marie-Aude Lefer, Lieve Macken, and Paola Ruffo, editors, Proceedings of the 1st Workshop on Creative-text Translation and Technology, pages… [find](https://scholar.google.com/scholar?q=Macken%2C+Lieve%2E+2024%2E+Machine+translation+meets+large+language+models%3A+Evaluating+ChatGPT%E2%80%99s+ability+to+automatically+post%2Dedit+literary+texts%2E+In+Vanroy%2C+Bram%2C+Marie%2DAude+Lefer%2C+Lieve+Macken%2C+and+Paola)
30. Papineni, Kishore, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. BLEU: A Method for Automatic Evaluation of Machine Translation. In Proceedings of the 40th Annual Meeting on Association for Computational Linguistics (ACL 2002), pages 311–318, Philadelphia, Pennsylvania, USA, July. Association for… [find](https://scholar.google.com/scholar?q=Papineni%2C+Kishore%2C+Salim+Roukos%2C+Todd+Ward%2C+and+Wei%2DJing+Zhu%2E+2002%2E+BLEU%3A+A+Method+for+Automatic+Evaluation+of+Machine+Translation%2E+In+Proceedings+of+the+40th+Annual+Meeting+on+Association+for+Computa) [our card](https://md.research.elamur.ai/free/en/papers/papineni-2002-bleu-method-automatic-evaluation-machine/)
31. Pedregosa, Fabian, Gaël Varoquaux, Alexandre Gramfort, Vincent Michel, Bertrand Thirion, Olivier Grisel, Mathieu Blondel, Peter Prettenhofer, Ron Weiss, Vincent Dubourg, et al. 2011. Scikit-learn: Machine learning in python. Journal of machine learning research, 12(Oct):2825–2830. [find](https://scholar.google.com/scholar?q=Pedregosa%2C+Fabian%2C+Ga%C3%ABl+Varoquaux%2C+Alexandre+Gramfort%2C+Vincent+Michel%2C+Bertrand+Thirion%2C+Olivier+Grisel%2C+Mathieu+Blondel%2C+Peter+Prettenhofer%2C+Ron+Weiss%2C+Vincent+Dubourg%2C+et+al%2E+2011%2E+Scikit%2Dlearn%3A+Mac)
32. Popović, Maja. 2015. chrF: character n-gram f-score for automatic MT evaluation. In Proceedings of the Tenth Workshop on Statistical Machine Translation (WMT@EMNLP 2015), pages 392–395, Lisbon, Portugal. [find](https://scholar.google.com/scholar?q=Popovic%CC%81%2C+Maja%2E+2015%2E+chrF%3A+character+n%2Dgram+f%2Dscore+for+automatic+MT+evaluation%2E+In+Proceedings+of+the+Tenth+Workshop+on+Statistical+Machine+Translation+%28WMT%40EMNLP+2015%29%2C+pages+392%E2%80%93395%2C+Lisbon%2C+Portu) [our card](https://md.research.elamur.ai/free/en/papers/popovic-2015-chrf-character-n-gram-f/)
33. Post, Matt. 2018. A call for clarity in reporting BLEU scores. In Proceedings of the Third Conference on Machine Translation: Research Papers, pages 186– 191, Belgium, Brussels, October. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Post%2C+Matt%2E+2018%2E+A+call+for+clarity+in+reporting+BLEU+scores%2E+In+Proceedings+of+the+Third+Conference+on+Machine+Translation%3A+Research+Papers%2C+pages+186%E2%80%93+191%2C+Belgium%2C+Brussels%2C+October%2E+Association+f) [our card](https://md.research.elamur.ai/free/en/papers/post-2018-call-clarity-reporting-bleu-scores/)
34. Rei, Ricardo, Craig Stewart, Ana C Farinha, and Alon Lavie. 2020. Comet: A neural framework for mt evaluation. [find](https://scholar.google.com/scholar?q=Rei%2C+Ricardo%2C+Craig+Stewart%2C+Ana+C+Farinha%2C+and+Alon+Lavie%2E+2020%2E+Comet%3A+A+neural+framework+for+mt+evaluation%2E) [our card](https://md.research.elamur.ai/free/en/papers/rei-2020-comet-neural-framework-mt-evaluation/)
35. Schneider, Johannes, Christian Meske, and Pauline Kuss. 2024. Foundation models: a new paradigm for artificial intelligence. Business & Information Systems Engineering, pages 1–11. [find](https://scholar.google.com/scholar?q=Schneider%2C+Johannes%2C+Christian+Meske%2C+and+Pauline+Kuss%2E+2024%2E+Foundation+models%3A+a+new+paradigm+for+artificial+intelligence%2E+Business+%26+Information+Systems+Engineering%2C+pages+1%E2%80%9311%2E)
36. Seljan, Sanja, Ivan Dunder, and Marko Pavlovski. 2020. Human quality evaluation of machine-translated poetry. In Koricic, Marko, Karolj Skala, Zeljka Car, Marina Cicin-Sain, Vlado Sruk, Dejan Skvorc, Slobodan Ribaric, Bojan Jerbic, Stjepan Gros, Boris Vrdoljak, Mladen Mauher, Edvard Tijan, Tihomir K… [find](https://scholar.google.com/scholar?q=Seljan%2C+Sanja%2C+Ivan+Dunder%2C+and+Marko+Pavlovski%2E+2020%2E+Human+quality+evaluation+of+machine%2Dtranslated+poetry%2E+In+Koricic%2C+Marko%2C+Karolj+Skala%2C+Zeljka+Car%2C+Marina+Cicin%2DSain%2C+Vlado+Sruk%2C+Dejan+Skvorc%2C+)
37. Sellam, Thibault, Dipanjan Das, and Ankur Parikh. 2020. BLEURT: Learning robust metrics for text generation. In Jurafsky, Dan, Joyce Chai, Natalie Schluter, and Joel Tetreault, editors, Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 7881–7892, Online,… [find](https://scholar.google.com/scholar?q=Sellam%2C+Thibault%2C+Dipanjan+Das%2C+and+Ankur+Parikh%2E+2020%2E+BLEURT%3A+Learning+robust+metrics+for+text+generation%2E+In+Jurafsky%2C+Dan%2C+Joyce+Chai%2C+Natalie+Schluter%2C+and+Joel+Tetreault%2C+editors%2C+Proceedings+of)
38. Shumailov, Ilia, Zakhar Shumaylov, Yiren Zhao, Yarin Gal, Nicolas Papernot, and Ross J. Anderson. 2023. The curse of recursion: Training on generated data makes models forget. arXiv preprint arXiv:2307.07003. [find](https://scholar.google.com/scholar?q=Shumailov%2C+Ilia%2C+Zakhar+Shumaylov%2C+Yiren+Zhao%2C+Yarin+Gal%2C+Nicolas+Papernot%2C+and+Ross+J%2E+Anderson%2E+2023%2E+The+curse+of+recursion%3A+Training+on+generated+data+makes+models+forget%2E+arXiv+preprint+arXiv%3A230) [our card](https://md.research.elamur.ai/free/en/papers/shumailov-2023-curse-recursion-training-generated-data/)
39. Símonarson, Haukur Barri, Vésteinn Snæbjarnarson, Pétur Orri Ragnarson, Haukur Jónsson, and Vilhjalmur Thorsteinsson. 2021. [find](https://scholar.google.com/scholar?q=S%C3%ADmonarson%2C+Haukur+Barri%2C+V%C3%A9steinn+Sn%C3%A6bjarnarson%2C+P%C3%A9tur+Orri+Ragnarson%2C+Haukur+J%C3%B3nsson%2C+and+Vilhjalmur+Thorsteinsson%2E+2021%2E)
40. Miðeind’s WMT 2021 submission. In Barrault, Loic, Ondrej Bojar, Fethi Bougares, Rajen Chatterjee, Marta R. Costa-jussa, Christian Federmann, Mark Fishel, Alexander Fraser, Markus Freitag, Yvette Graham, Roman Grundkiewicz, Paco Guzman, Barry Haddow, Matthias Huck, Antonio Jimeno Yepes, Philipp Koehn… [find](https://scholar.google.com/scholar?q=Mi%C3%B0eind%E2%80%99s+WMT+2021+submission%2E+In+Barrault%2C+Loic%2C+Ondrej+Bojar%2C+Fethi+Bougares%2C+Rajen+Chatterjee%2C+Marta+R%2E+Costa%2Djussa%2C+Christian+Federmann%2C+Mark+Fishel%2C+Alexander+Fraser%2C+Markus+Freitag%2C+Yvette+Graha)
41. Snover, Matthew, Bonnie Dorr, Richard Schwartz, Linnea Micciulla, and John Makhoul. 2006. A study of translation edit rate with targeted human annotation. In Proceedings of the 7th Conference of the Association for Machine Translation of the Americas (AMTA 2006). Visions for the Future of Machine Tr… [find](https://scholar.google.com/scholar?q=Snover%2C+Matthew%2C+Bonnie+Dorr%2C+Richard+Schwartz%2C+Linnea+Micciulla%2C+and+John+Makhoul%2E+2006%2E+A+study+of+translation+edit+rate+with+targeted+human+annotation%2E+In+Proceedings+of+the+7th+Conference+of+the+A) [our card](https://md.research.elamur.ai/free/en/papers/snover-2006-study-translation-edit-rate-targeted/)
42. Sourati, Zhivar, Farzan Karimi-Malekabadi, Meltem Ozcan, Colin McDaniel, Alireza Ziabari, Jackson Trager, Ala Tak, Meng Chen, Fred Morstatter, and Morteza Dehghani. 2025. The shrinking landscape of linguistic diversity in the age of large language models. arXiv preprint arXiv:2502.11266. — [arXiv:2502.11266](https://arxiv.org/abs/2502.11266) [find](https://scholar.google.com/scholar?q=Sourati%2C+Zhivar%2C+Farzan+Karimi%2DMalekabadi%2C+Meltem+Ozcan%2C+Colin+McDaniel%2C+Alireza+Ziabari%2C+Jackson+Trager%2C+Ala+Tak%2C+Meng+Chen%2C+Fred+Morstatter%2C+and+Morteza+Dehghani%2E+2025%2E+The+shrinking+landscape+of+li)
43. Thompson, Brian, Mehak Dhaliwal, Peter Frisch, Tobias Domhan, and Marcello Federico. 2024. A shocking amount of the web is machine translated: Insights from multi-way parallelism. In Findings of the Association for Computational Linguistics: ACL 2024, pages 1763–1775. [find](https://scholar.google.com/scholar?q=Thompson%2C+Brian%2C+Mehak+Dhaliwal%2C+Peter+Frisch%2C+Tobias+Domhan%2C+and+Marcello+Federico%2E+2024%2E+A+shocking+amount+of+the+web+is+machine+translated%3A+Insights+from+multi%2Dway+parallelism%2E+In+Findings+of+the+A)
44. Toral, Antonio and Andy Way. 2015. Machine-assisted translation of literary text: A case study. Translation Spaces, 4:240–267, 12. [find](https://scholar.google.com/scholar?q=Toral%2C+Antonio+and+Andy+Way%2E+2015%2E+Machine%2Dassisted+translation+of+literary+text%3A+A+case+study%2E+Translation+Spaces%2C+4%3A240%E2%80%93267%2C+12%2E)
45. Toral, Antonio and Andy Way. 2018. What level of quality can neural machine translation attain on literary text? CoRR, abs/1801.04962. — [arXiv:1801.04962](https://arxiv.org/abs/1801.04962) [find](https://scholar.google.com/scholar?q=Toral%2C+Antonio+and+Andy+Way%2E+2018%2E+What+level+of+quality+can+neural+machine+translation+attain+on+literary+text%3F+CoRR%2C+abs%2F1801%2E04962%2E+%E2%80%94+%5BarXiv%3A1801%2E04962%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F1801%2E04962%29)
46. Toral, Antonio, Sheila Castilho, Ke Hu, and Andy Way. 2018. Attaining the unattainable? reassessing claims of human parity in neural machine translation. In Proceedings of the Third Conference on Machine Translation: Research Papers, WMT 2018, Belgium, Brussels, October 31 - November 1, 2018, pages… [find](https://scholar.google.com/scholar?q=Toral%2C+Antonio%2C+Sheila+Castilho%2C+Ke+Hu%2C+and+Andy+Way%2E+2018%2E+Attaining+the+unattainable%3F+reassessing+claims+of+human+parity+in+neural+machine+translation%2E+In+Proceedings+of+the+Third+Conference+on+Mach) [our card](https://md.research.elamur.ai/free/en/papers/toral-2018-attaining-unattainable-reassessing-claims-human/)
47. Toral, Antonio. 2019. Post-editese: an exacerbated translationese. In Forcada, Mikel, Andy Way, Barry Haddow, and Rico Sennrich, editors, Proceedings of Machine Translation Summit XVII: Research Track, pages 273–281, Dublin, Ireland, August. European Association for Machine Translation. [find](https://scholar.google.com/scholar?q=Toral%2C+Antonio%2E+2019%2E+Post%2Deditese%3A+an+exacerbated+translationese%2E+In+Forcada%2C+Mikel%2C+Andy+Way%2C+Barry+Haddow%2C+and+Rico+Sennrich%2C+editors%2C+Proceedings+of+Machine+Translation+Summit+XVII%3A+Research+Track)
48. Vanmassenhove, Eva, Dimitar Shterionov, and Andy Way. 2019. Lost in translation: Loss and decay of linguistic richness in machine translation. In Proceedings of Machine Translation Summit XVII Volume 1: Research Track, pages 222–232, Dublin, Ireland, August. European Association for Machine Translat… [find](https://scholar.google.com/scholar?q=Vanmassenhove%2C+Eva%2C+Dimitar+Shterionov%2C+and+Andy+Way%2E+2019%2E+Lost+in+translation%3A+Loss+and+decay+of+linguistic+richness+in+machine+translation%2E+In+Proceedings+of+Machine+Translation+Summit+XVII+Volume+) [our card](https://md.research.elamur.ai/free/en/papers/vanmassenhove-2019-lost-translation-loss-decay-linguistic/)
49. Vanmassenhove, Eva, Dimitar Shterionov, and Matthew Gwilliam. 2021. Machine translationese: Effects of algorithmic bias on linguistic complexity in machine translation. In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume, pages… [find](https://scholar.google.com/scholar?q=Vanmassenhove%2C+Eva%2C+Dimitar+Shterionov%2C+and+Matthew+Gwilliam%2E+2021%2E+Machine+translationese%3A+Effects+of+algorithmic+bias+on+linguistic+complexity+in+machine+translation%2E+In+Proceedings+of+the+16th+Conf)
50. Vanmassenhove, Eva. 2025. Losing our tail–again: On (un) natural selection and multilingual large language models. arXiv preprint arXiv:2507.03933. — [arXiv:2507.03933](https://arxiv.org/abs/2507.03933) [find](https://scholar.google.com/scholar?q=Vanmassenhove%2C+Eva%2E+2025%2E+Losing+our+tail%E2%80%93again%3A+On+%28un%29+natural+selection+and+multilingual+large+language+models%2E+arXiv+preprint+arXiv%3A2507%2E03933%2E+%E2%80%94+%5BarXiv%3A2507%2E03933%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2507%2E0393)
51. Vujović, Dušan. 2024. Generative ai: Riding the new general purpose technology storm. Ekonomika pre-duzeća, 72(1-2):125–136. [find](https://scholar.google.com/scholar?q=Vujovic%CC%81%2C+Du%C5%A1an%2E+2024%2E+Generative+ai%3A+Riding+the+new+general+purpose+technology+storm%2E+Ekonomika+pre%2Dduzec%CC%81a%2C+72%281%2D2%29%3A125%E2%80%93136%2E)
52. Wang, Huashu and Ying Wu. 2026. Technologies are reshaping translation. AI Through LLMs: Transforming Translation Practice and Teaching, page 3. [find](https://scholar.google.com/scholar?q=Wang%2C+Huashu+and+Ying+Wu%2E+2026%2E+Technologies+are+reshaping+translation%2E+AI+Through+LLMs%3A+Transforming+Translation+Practice+and+Teaching%2C+page+3%2E)
53. Wang, Shanshan, Derek Wong, Jingming Yao, and Lidia Chao. 2024. What is the best way for Chat-GPT to translate poetry? In Ku, Lun-Wei, Andre Martins, and Vivek Srikumar, editors, Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 14… [find](https://scholar.google.com/scholar?q=Wang%2C+Shanshan%2C+Derek+Wong%2C+Jingming+Yao%2C+and+Lidia+Chao%2E+2024%2E+What+is+the+best+way+for+Chat%2DGPT+to+translate+poetry%3F+In+Ku%2C+Lun%2DWei%2C+Andre+Martins%2C+and+Vivek+Srikumar%2C+editors%2C+Proceedings+of+the+62)
54. Wolf, Thomas, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pier-ric Cistac, Tim Rault, Rémi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Mariama Drame, Quentin Lh… [find](https://scholar.google.com/scholar?q=Wolf%2C+Thomas%2C+Lysandre+Debut%2C+Victor+Sanh%2C+Julien+Chaumond%2C+Clement+Delangue%2C+Anthony+Moi%2C+Pier%2Dric+Cistac%2C+Tim+Rault%2C+R%C3%A9mi+Louf%2C+Morgan+Funtowicz%2C+Joe+Davison%2C+Sam+Shleifer%2C+Patrick+von+Platen%2C+Clara) [our card](https://md.research.elamur.ai/free/en/papers/wolf-2020-transformers-state-art-natural-language/)
55. Wu, Yangjian and Gang Hu. 2023. Exploring prompt engineering with GPT language models for document-level machine translation: Insights and findings. In Koehn, Philipp, Barry Haddow, Tom Kocmi, and Christof Monz, editors, Proceedings of the Eighth Conference on Machine Translation, pages 166–169, Sin… [find](https://scholar.google.com/scholar?q=Wu%2C+Yangjian+and+Gang+Hu%2E+2023%2E+Exploring+prompt+engineering+with+GPT+language+models+for+document%2Dlevel+machine+translation%3A+Insights+and+findings%2E+In+Koehn%2C+Philipp%2C+Barry+Haddow%2C+Tom+Kocmi%2C+and+Chr)
56. Yakura, Hiromu, Ezequiel Lopez-Lopez, Levin Brinkmann, Ignacio Serna, Prateek Gupta, Ivan Soraperra, and Iyad Rahwan. 2024. Empirical evidence of large language model’s influence on human spoken communication. arXiv preprint arXiv:2409.01754. — [arXiv:2409.01754](https://arxiv.org/abs/2409.01754) [find](https://scholar.google.com/scholar?q=Yakura%2C+Hiromu%2C+Ezequiel+Lopez%2DLopez%2C+Levin+Brinkmann%2C+Ignacio+Serna%2C+Prateek+Gupta%2C+Ivan+Soraperra%2C+and+Iyad+Rahwan%2E+2024%2E+Empirical+evidence+of+large+language+model%E2%80%99s+influence+on+human+spoken+commu)
57. Zhang, Ran, Wei Zhao, and Steffen Eger. 2025. How good are LLMs for literary translation, really? literary translation evaluation with humans and LLMs. In Chiruzzo, Luis, Alan Ritter, and Lu Wang, editors, Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association f… [find](https://scholar.google.com/scholar?q=Zhang%2C+Ran%2C+Wei+Zhao%2C+and+Steffen+Eger%2E+2025%2E+How+good+are+LLMs+for+literary+translation%2C+really%3F+literary+translation+evaluation+with+humans+and+LLMs%2E+In+Chiruzzo%2C+Luis%2C+Alan+Ritter%2C+and+Lu+Wang%2C+edi)