---
title: "Lexical and Syntactic Diversity — Volkart 2026"
description: "This study asks whether recent advances in machine translation have reduced two known stylistic weaknesses of MT: limited lexical variety and close…"
authors: ["Lise Volkart", "Pierrette Bouillon"]
year: 2026
venue: "Proceedings of the First Workshop on Style in GenAI-Translated Content (StyGenAI)"
source: "https://aclanthology.org/2026.stygenai-1.8/"
licence: "CC BY-ND 4.0 (stated in the PDF)"
licence_url: "https://creativecommons.org/licenses/by-nd/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/papers/volkart-2026-lexical-syntactic-diversity-still-lost/"
---
[Elamur Research](https://md.research.elamur.ai/) · [this page in HTML](https://research.elamur.ai/free/papers/volkart-2026-lexical-syntactic-diversity-still-lost/)

[home](https://md.research.elamur.ai/) / [papers](https://md.research.elamur.ai/free/papers/) / [2026](https://md.research.elamur.ai/free/papers/2026/page/3/) / Lexical and Syntactic Diversity: Still Lost in Machine Translation?

# Lexical and Syntactic Diversity: Still Lost in Machine Translation?

**Authors:** [Lise Volkart](https://md.research.elamur.ai/free/authors/lise-volkart/), [Pierrette Bouillon](https://md.research.elamur.ai/free/authors/pierrette-bouillon/) · **Year:** 2026 · **Venue:** Proceedings of the First Workshop on Style in GenAI-Translated Content (StyGenAI) · **Licence:** [CC BY-ND 4.0 (stated in the PDF)](https://creativecommons.org/licenses/by-nd/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2026.stygenai-1.8/) · [PDF](https://aclanthology.org/2026.stygenai-1.8.pdf) · [OpenAlex](https://openalex.org/W7165092958)

**Citations:** 0 citations (openalex-snapshot · 2026-10-14)

## Summary

This study asks whether recent advances in machine translation have reduced two known stylistic weaknesses of MT: limited lexical variety and close imitation of source-sentence syntax. The authors reused two professional English→French corpora (C1 and C2) whose source texts had been translated with DeepL's NMT in February 2024. In February 2026 they translated the same sources again with two DeepL models: the classic NMT model and the LLM-based "next-gen" model. Lexical diversity was measured with word translation entropy (HTra). Syntactic closeness to the source was measured with four metrics from the ASTrED library.

NMT output from 2024 and 2026 was essentially identical on both dimensions. The LLM-based model showed a small rise in HTra, mostly because it occasionally added extra translation variants, but lexical overgeneralisation remained. Syntactic similarity to the source barely changed, and all MT versions stayed much closer to one another than to human translation. The findings suggest MT style has plateaued, so earlier results still apply and professional post-editing remains valuable.

## Key points

- **Corpora:** C1 comes from the European Investment Bank (1,852 segments, about 47k tokens per MT version). C2 comes from a Swiss sports organization (2,280 segments, about 49–53k tokens). The original human translations and post-edited MT were collected in 2021–2022.
- **Compared outputs:** NMT24, NMT26 and LLM-MT26, all produced with DeepL.
- **Lexical method:** HTra was computed for content lemmas occurring at least 3 times in the source. Word alignment used awesome-align, and tagging and lemmatisation used spaCy transformer models.
- **Syntactic method:** four ASTrED-library metrics were used: SACr, ASTrED, Label POS and Label DEP.
- **Statistics:** significance was tested with the Wilcoxon signed-rank test.
  - LLM-MT26 vs. NMT24 HTra gains were significant for all POS combined, nouns and verbs, but effect sizes were small (−0.09 to −0.17).
  - Syntactic differences in C2 were also significant but had negligible effect sizes (0.08–0.09).

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

Fields: [Machine translation](https://md.research.elamur.ai/free/shelves/machine-translation/) · [Natural language processing](https://md.research.elamur.ai/free/shelves/nlp/)

Topics: [Machine translation](https://md.research.elamur.ai/free/topics/machine-translation/)

## Cite

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{volkart2026lexical,
  title = {{Lexical and Syntactic Diversity: Still Lost in Machine Translation?}},
  author = {Volkart, Lise and Bouillon, Pierrette},
  booktitle = {Proceedings of the First Workshop on Style in GenAI-Translated Content (StyGenAI)},
  year = {2026},
  url = {https://aclanthology.org/2026.stygenai-1.8/}
}
```

- [← previous Leveraging Digitized Newspapers to Collect Summarization Data in Low-Resource Languages](https://md.research.elamur.ai/free/papers/aclbulk-2026-dahan-leveraging-digitized-newspapers-collect-summarization/)
- [next → LIUM Submission for IWSLT 2026 Low-resource Speech Translation Track](https://md.research.elamur.ai/free/papers/mohammadamini-2026-lium-submission-iwslt-2026-low/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/papers/2026/page/3/)

## References

17 references, 2 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Ataman, Duygu, Alexandra Birch, Nizar Habash, Marcello Federico, Philipp Koehn, and Kyunghyun Cho. 2025. Machine Translation in the Era of Large Language Models:A Survey of Historical and Emerging Problems. Information. https://doi.org/10.3390/info16090723. [↗](https://doi.org/10.3390/info16090723) [our card](https://md.research.elamur.ai/free/papers/minaee-2024-large-language-models-survey/)
2. Bangalore, Srinivas, Bergljot Behrens, Michael Carl, Maheshwar Ghankot, Arndt Heilmann, Jean Nitzke, Moritz Schaeffer, and Annegret Sturm. 2016. Syntactic variance and priming effects in translation. In New directions in empirical translation process research, pages 211–238. Springer. [↗](https://doi.org/10.1007/978-3-319-20358-4_10)
3. Carl, Michael, Moritz Schaeffer, and Srinivas Bangalore. 2016. The CRITT translation process research database. In New directions in empirical translation process research, pages 13– 54. Springer. https://doi.org/10.1007/978-3-319-20358-4_2. [↗](https://doi.org/10.1007/978-3-319-20358-4_2)
4. Dou, Zi-Yi and Graham Neubig. 2021. Word alignment by fine-tuning embeddings on parallel corpora. In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume, pages 2112– 2128. ACL. https://doi.org/10.18653/v1/2021.eacl-main.181. [↗](https://doi.org/10.18653/v1/2021.eacl-main.181)
5. Hendy, Amr, Mohamed Abdelrehim, Amr Sharaf, Vikas Raunak, Mohamed Gabr, Hitokazu Matsushita, Young Jin Kim, Mohamed Afify, and Hany Hassan Awadalla. 2023. How good are GPT models at machine translation? a comprehensive evaluation. Preprint : arXiv:2302.09210. https://doi.org/10.48550/arXiv.2302.09210. [↗](https://doi.org/10.48550/arxiv.2302.09210)
6. Lazarov, Todor. 2026. LLMs and AI in the Language Industry: An Overview. In 2026 International Conference on Artificial Intelligence, Computer, Data Sciences and Applications (ACDSA), pages 1–6. https://doi.org/10.1109/ACDSA67686.2026.11468231. [↗](https://doi.org/10.1109/acdsa67686.2026.11468231)
7. Li, Yafu, Ronghao Zhang, Zhilin Wang, Huajian Zhang, Leyang Cui, Yongjing Yin, Tong Xiao, and Yue Zhang. 2025. Lost in Literalism: NMT24 NMT26 LLM-MT26 0,17 ⋄ 0,17 ⋆ 0,17 0,16 0,13 ⋄ 0,13 0,13 0,13 0,20 ⋄ 0,20 ⋆ 0,19 0,20 0,57 ⋄ 0,57 ⋄ 0,57 ⋄ 0,57 How Supervised Training Shapes Translationese in LLMs. In Che, Wanxiang, Joyce Nabende, Ekaterina Shutova, and Mohammad Taher Pilehvar, editors, Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 12875–12894, Vienna, Austria, July. Association for Computational Linguistics. https://doi.org/10.18653/v1/2025.acl-long.630. [↗](https://doi.org/10.18653/v1/2025.acl-long.630)
8. Martikainen, Hanna and Natalie Kübler. 2016. Ergonomie cognitive de la post-édition de traduction automatique : enjeux pour la qualité des traductions. ILCEA, (27), November. https://doi.org/10.4000/ilcea.3863. [↗](https://doi.org/10.4000/ilcea.3863)
9. Raunak, Vikas, Arul Menezes, Matt Post, and Hany Hassan Awadalla. 2023. Do GPTs Produce Less Literal Translations? pages 1041–1050, Toronto, Canada, July. https://doi.org/10.18653/v1/2023.acl-short.90. [↗](https://doi.org/10.18653/v1/2023.acl-short.90)
10. Shaitarova, Anastassia, Anne Göhring, and Martin Volk. 2023. Machine vs. Human: Exploring Syntax and Lexicon in German Translations, with a Spotlight on Anglicisms. In The 24th Nordic Conference on Computational Linguistics. https://aclanthology.org/2023. nodalida-1.22/. [↗](https://doi.org/10.5167/uzh-233906)
11. Tezcan, Arda, Joke Daems, and Lieve Macken. 2019. When a ’sport’ is a person and other issues for NMT of novels. In Machine Translation Summit XVII, pages 40–49. European Association for Machine Translation. https://aclanthology.org/W19-7306/. [↗](https://aclanthology.org/W19-7306/)
12. Toral, Antonio. 2019. Post-editese: an Exacerbated Translationese. In Proceedings of MT Summit XVII, volume 1, pages 273 – 281, Dublin, Ireland. https://aclanthology.org/W19-6627/. [↗](https://aclanthology.org/W19-6627/)
13. Vanmassenhove, Eva, Dimitar Shterionov, and Andy Way. 2019. Lost in Translation: Loss and Decay of Linguistic Richness in Machine Translation. In Proceedings of MT Summit XVII, volume 1, pages 222 – 232, Dublin, Ireland. https://aclanthology.org/W19-6622/. [↗](https://aclanthology.org/W19-6622/) [our card](https://md.research.elamur.ai/free/papers/vanmassenhove-2019-lost-translation-loss-decay-linguistic/)
14. Vanmassenhove, Eva, Dimitar Shterionov, and Matthew Gwilliam. 2021. Machine Translationese: Effects of Algorithmic Bias on Linguistic Complexity in Machine Translation. In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume, pages 2203–2213, Online. ACL. https://aclanthology.org/2021. eacl-main.188. [↗](https://aclanthology.org/2021)
15. Vanroy, Bram, Orphée De Clercq, Arda Tezcan, Joke Daems, and Lieve Macken. 2021. Metrics of syntactic equivalence to assess translation difficulty. In Explorations in empirical translation process research, pages 259–294. Springer. https://doi.org/10.1007/978-3-030-69777-8_10. [↗](https://doi.org/10.1007/978-3-030-69777-8_10)
16. Volkart, Lise and Pierrette Bouillon. 2024. Post-editors as Gatekeepers of Lexical and Syntactic Diversity: Comparative Analysis of Human Translation and Post-editing in Professional Settings. In Proceedings of the 25th Annual Conference of the European Association for Machine Translation (Volume 1), Sheffield. https://aclanthology.org/2024.eamt-1.33/. Volkart, Lise. forthcoming. Traduction automatique neuronale et post-édition : vers une uniformisation des textes traduits ? Étude sur la langue post-éditée en contexte professionnel. Ph.D. thesis, University of Geneva. [↗](https://aclanthology.org/2024.eamt-1.33/)
17. Webster, Rebecca, Margot Fonteyne, Arda Tezcan, Lieve Macken, and Joke Daems. 2020. Gutenberg Goes Neural: Comparing Features of Dutch Human Translations with Raw Neural Machine Translation Outputs in a Corpus of English Literary Classics. Informatics, 7(3). https://doi.org/10.3390/informatics7030032. [↗](https://doi.org/10.3390/informatics7030032)