---
title: "Losing our Tail, Again: (Un)Natural Selection & Multilingual LLMs"
description: "It presents no new dataset or experiment. 67 references. Among them: 0."
authors: ["Eva Vanmassenhove"]
year: 2026
venue: "Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)"
source: "https://aclanthology.org/2026.acl-long.532/"
doi: "10.18653/v1/2026.acl-long.532"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/vanmassenhove-2026-losing-our-tail-again-un/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/vanmassenhove-2026-losing-our-tail-again-un/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/40/) / Losing our Tail, Again: (Un)Natural Selection & Multilingual LLMs

# Losing our Tail, Again: (Un)Natural Selection & Multilingual LLMs

**Authors:** [Eva Vanmassenhove](https://md.research.elamur.ai/free/en/authors/eva-vanmassenhove/) · **Year:** 2026 · **Venue:** [Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)](https://md.research.elamur.ai/free/en/venues/acl/) ([ACL 2026](https://md.research.elamur.ai/free/en/venues/acl/2026/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2026.acl-long.532/) · [PDF](https://aclanthology.org/2026.acl-long.532.pdf) · [DOI 10.18653/v1/2026.acl-long.532](https://doi.org/10.18653/v1/2026.acl-long.532) · [OpenAlex](https://openalex.org/W7166836467)

**Citations:** 0 citations (OpenAlex, 2026-10-14)

## Summary
This position paper asks whether multilingual LLMs, trained to predict the next token, are wearing away the long statistical "tails" of language: rare words, uncommon constructions, minority languages and culturally specific variation. It presents no new dataset or experiment. Instead, the author argues by contrasting human-driven language change with technology-driven selection, drawing on iterated learning, studies of lexical growth and competition, and recent model-collapse research in Computer Vision, NLP and Machine Translation. Translation technology is the main case study. The core idea is that human cognitive limits push languages toward structure and compositionality while vocabularies stay rich. LLMs, by contrast, drift toward the most probable forms, especially when machine-generated text re-enters training corpora in self-consuming loops. The main claim is that, under current data and training practices, LLM-driven selection shrinks diversity instead of transforming it, and it spreads averaged, dominant biases. For NLP practitioners, the paper urges training and evaluating systems for expressive multilingual richness rather than only fluency or accuracy.

## Key points
- **Model collapse** is defined as synthetic output feeding back into training data, which distorts the distribution and leaves low-probability linguistic phenomena underrepresented.
- **Human versus LLM selection:** human transmission produces structure through inductive biases toward simplicity and compressibility (Kirby et al., Smith et al.). LLMs instead converge on statistically likely forms and cannot critically reflect on or correct themselves.
- **Context figures cited:**
  - About 6,000 languages exist (estimates range from 3,000 to 10,000), and nearly half are endangered.
  - According to Michel et al. (2011), the English lexicon expanded by more than 70% over 50 years, gaining about 8,500 words annually.
- **Lexical competition example:** "X-ray" displaced "Roentgenogram" and "radiogram", showing that word survival depends on more than frequency.
- **Stated limitations:** the critique targets current generative AI rather than all possible architectures. The author acknowledges a selection bias in the literature reviewed and notes that some researchers may see "flattening" as gained efficiency.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Tilburg University](https://md.research.elamur.ai/free/en/universities/tilburg-university/) — University, Tilburg, Netherlands · 51 works · 6,798 citations

📚 Shelves (6): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 18,614 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 17,965 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 17,450 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 17,296 works · [🤝 Social Sciences](https://md.research.elamur.ai/free/en/shelves/social-sciences/) · 1,954 works · [🤝 Safety Research](https://md.research.elamur.ai/free/en/shelves/social-sciences/safety-research/) · 626 works

🧭 Topics (2): [🧠 Language models & pretraining](https://md.research.elamur.ai/free/en/topics/language-models/) · 3,894 works · [🌍 Multilingual & cross-lingual NLP](https://md.research.elamur.ai/free/en/topics/multilingual-nlp/) · 882 works

## Cite

Cite as: Eva Vanmassenhove. 2026. Losing our Tail, Again: (Un)Natural Selection & Multilingual LLMs. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{vanmassenhove2026losing,
  title = {{Losing our Tail, Again: (Un)Natural Selection \& Multilingual LLMs}},
  author = {Eva Vanmassenhove},
  booktitle = {Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2026},
  doi = {10.18653/v1/2026.acl-long.532},
  url = {https://doi.org/10.18653/v1/2026.acl-long.532}
}
```

- [← previous LoResMT 2026 Shared Task System Description](https://md.research.elamur.ai/free/en/papers/panov-2026-loresmt-2026-shared-task-system/)
- [next → Losses that Cook: Topological Optimal Transport for Structured Recipe Generation](https://md.research.elamur.ai/free/en/papers/ottoborgo-2026-losses-that-cook-topological-optimal/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/40/)

## References

67 references, 18 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Sina Alemohammad, Josue Casco-Rodriguez, Lorenzo Luzi, Ahmed Imtiaz Humayun, Hossein Babaei, Daniel LeJeune, Ali Siahkoohi, and Richard G Baraniuk. 2023. Self-consuming generative models go mad. arXiv preprint arXiv:2307.01850, 4:14. — [arXiv:2307.01850](https://arxiv.org/abs/2307.01850) [find](https://scholar.google.com/scholar?q=Sina+Alemohammad%2C+Josue+Casco%2DRodriguez%2C+Lorenzo+Luzi%2C+Ahmed+Imtiaz+Humayun%2C+Hossein+Babaei%2C+Daniel+LeJeune%2C+Ali+Siahkoohi%2C+and+Richard+G+Baraniuk%2E+2023%2E+Self%2Dconsuming+generative+models+go+mad%2E+arXiv)
2. Eduardo G Altmann, Janet B Pierrehumbert, and Adilson E Motter. 2011. Niche as a determinant of word fate in online groups. PloS one, 6(5):e19009. [find](https://scholar.google.com/scholar?q=Eduardo+G+Altmann%2C+Janet+B+Pierrehumbert%2C+and+Adilson+E+Motter%2E+2011%2E+Niche+as+a+determinant+of+word+fate+in+online+groups%2E+PloS+one%2C+6%285%29%3Ae19009%2E)
3. Yuntao Bai, Saurav Kadavath, Sandipan Kundu, Amanda Askell, Jackson Kernion, Andy Jones, Anna Chen, Anna Goldie, Azalia Mirhoseini, Cameron McKinnon, et al. 2022. [Constitutional ai: Harmlessness from ai feedback](https://md.research.elamur.ai/free/en/papers/bai-2022-constitutional-ai-harmlessness-ai-feedback/). arXiv preprint arXiv:2212.08073.
4. Satanjeev Banerjee and Alon Lavie. 2005. [Meteor: An automatic metric for mt evaluation with improved correlation with human judgments](https://md.research.elamur.ai/free/en/papers/banerjee-2005-meteor-automatic-metric-mt-evaluation/). In Proceedings of the acl workshop on intrinsic and extrinsic evaluation measures for machine translation and/or summarization, pages 65–72.
5. Emily M Bender, Timnit Gebru, Angelina McMillan-Major, and Shmargaret Shmitchell. 2021. On the dangers of stochastic parrots: Can language models be too big? In Proceedings of the 2021 ACM conference on fairness, accountability, and transparency, pages 610–623. [find](https://scholar.google.com/scholar?q=Emily+M+Bender%2C+Timnit+Gebru%2C+Angelina+McMillan%2DMajor%2C+and+Shmargaret+Shmitchell%2E+2021%2E+On+the+dangers+of+stochastic+parrots%3A+Can+language+models+be+too+big%3F+In+Proceedings+of+the+2021+ACM+conference+)
6. Sebastian Bernhardsson, Luis Enrique Correa da Rocha, and Petter Minnhagen. 2009. The meta book and size-dependent properties of written language. New Journal of Physics, 11(12):123015. [find](https://scholar.google.com/scholar?q=Sebastian+Bernhardsson%2C+Luis+Enrique+Correa+da+Rocha%2C+and+Petter+Minnhagen%2E+2009%2E+The+meta+book+and+size%2Ddependent+properties+of+written+language%2E+New+Journal+of+Physics%2C+11%2812%29%3A123015%2E)
7. Alexander Bick, Adam Blandin, and David J Deming. 2026. The rapid adoption of generative ai. Management Science. [find](https://scholar.google.com/scholar?q=Alexander+Bick%2C+Adam+Blandin%2C+and+David+J+Deming%2E+2026%2E+The+rapid+adoption+of+generative+ai%2E+Management+Science%2E)
8. Henry Brighton, Kenny Smith, and Simon Kirby. 2005. Language as an evolutionary system. Physics of Life Reviews, 2(3):177–226. [find](https://scholar.google.com/scholar?q=Henry+Brighton%2C+Kenny+Smith%2C+and+Simon+Kirby%2E+2005%2E+Language+as+an+evolutionary+system%2E+Physics+of+Life+Reviews%2C+2%283%29%3A177%E2%80%93226%2E)
9. Lindell Bromham, Russell Dinnage, Hedvig Skirgård, Andrew Ritchie, Marcel Cardillo, Felicity Meakins, Simon Greenhill, and Xia Hua. 2022. Global predictors of language endangerment and the future of linguistic diversity. Nature ecology & evolution, 6(2):163–173. [find](https://scholar.google.com/scholar?q=Lindell+Bromham%2C+Russell+Dinnage%2C+Hedvig+Skirg%C3%A5rd%2C+Andrew+Ritchie%2C+Marcel+Cardillo%2C+Felicity+Meakins%2C+Simon+Greenhill%2C+and+Xia+Hua%2E+2022%2E+Global+predictors+of+language+endangerment+and+the+future+of+l)
10. Yang Trista Cao, Anna Sotnikova, Jieyu Zhao, Linda X Zou, Rachel Rudinger, and Hal Daume III. 2023. Multilingual large language models leak human stereotypes across language boundaries. arXiv preprint arXiv:2312.07141. [find](https://scholar.google.com/scholar?q=Yang+Trista+Cao%2C+Anna+Sotnikova%2C+Jieyu+Zhao%2C+Linda+X+Zou%2C+Rachel+Rudinger%2C+and+Hal+Daume+III%2E+2023%2E+Multilingual+large+language+models+leak+human+stereotypes+across+language+boundaries%2E+arXiv+preprint)
11. Sheila Castilho, Zoe Fitzsimmons, Claire Holton, and Aoife Mc Donagh. 2025. Synthetic fluency: Hallucinations, confabulations, and the creation of irish words in llm-generated translations. Proceedings of Machine Translation Summit XVII: Research Track. [find](https://scholar.google.com/scholar?q=Sheila+Castilho%2C+Zoe+Fitzsimmons%2C+Claire+Holton%2C+and+Aoife+Mc+Donagh%2E+2025%2E+Synthetic+fluency%3A+Hallucinations%2C+confabulations%2C+and+the+creation+of+irish+words+in+llm%2Dgenerated+translations%2E+Proceeding)
12. Noam Chomsky. 1980. Rules and representations. Behavioral and brain sciences, 3(1):1–15. [find](https://scholar.google.com/scholar?q=Noam+Chomsky%2E+1980%2E+Rules+and+representations%2E+Behavioral+and+brain+sciences%2C+3%281%29%3A1%E2%80%9315%2E)
13. David Crystal. 2002. Language death. Cambridge university press. [find](https://scholar.google.com/scholar?q=David+Crystal%2E+2002%2E+Language+death%2E+Cambridge+university+press%2E)
14. Charles Darwin. 1871. The Descent of Man, and Selection in Relation to Sex, volume 1. John Murray, London. [find](https://scholar.google.com/scholar?q=Charles+Darwin%2E+1871%2E+The+Descent+of+Man%2C+and+Selection+in+Relation+to+Sex%2C+volume+1%2E+John+Murray%2C+London%2E)
15. Marci S DeCaro, Robin D Thomas, and Sian L Beilock. 2008. Individual differences in category learning: Sometimes less working memory capacity is better than more. Cognition, 107(1):284–294. [find](https://scholar.google.com/scholar?q=Marci+S+DeCaro%2C+Robin+D+Thomas%2C+and+Sian+L+Beilock%2E+2008%2E+Individual+differences+in+category+learning%3A+Sometimes+less+working+memory+capacity+is+better+than+more%2E+Cognition%2C+107%281%29%3A284%E2%80%93294%2E)
16. Kristian Dokic, Barbara Pisker, and Bojan Radisic. 2025. Mirroring cultural dominance: Disclosing large language models social values, attitudes and stereotypes. Societies, 15(5):142. [find](https://scholar.google.com/scholar?q=Kristian+Dokic%2C+Barbara+Pisker%2C+and+Bojan+Radisic%2E+2025%2E+Mirroring+cultural+dominance%3A+Disclosing+large+language+models+social+values%2C+attitudes+and+stereotypes%2E+Societies%2C+15%285%29%3A142%2E)
17. Nouha Dziri, Ximing Lu, Melanie Sclar, Xiang Lorraine Li, Liwei Jiang, Bill Yuchen Lin, Sean Welleck, Peter West, Chandra Bhagavatula, Ronan Le Bras, Jena D. Hwang, Soumya Sanyal, Xiang Ren, Allyson Ettinger, Zaïd Harchaoui, and Yejin Choi. 2023. [Faith and fate: Limits of transformers on composition…](https://md.research.elamur.ai/free/en/papers/dziri-2023-faith-fate-limits-transformers-compositionality/)
18. Daniel L Everett. 2017. How language began: The story of humanity’s greatest invention. Liveright Publishing. [find](https://scholar.google.com/scholar?q=Daniel+L+Everett%2E+2017%2E+How+language+began%3A+The+story+of+humanity%E2%80%99s+greatest+invention%2E+Liveright+Publishing%2E)
19. Martin Gerlach and Eduardo G Altmann. 2013. Stochastic model for the vocabulary growth in natural languages. Physical Review X, 3(2):021006. [find](https://scholar.google.com/scholar?q=Martin+Gerlach+and+Eduardo+G+Altmann%2E+2013%2E+Stochastic+model+for+the+vocabulary+growth+in+natural+languages%2E+Physical+Review+X%2C+3%282%29%3A021006%2E)
20. Gerd Gigerenzer and Daniel G Goldstein. 1996. Reasoning the fast and frugal way: models of bounded rationality. Psychological review, 103(4):650. [find](https://scholar.google.com/scholar?q=Gerd+Gigerenzer+and+Daniel+G+Goldstein%2E+1996%2E+Reasoning+the+fast+and+frugal+way%3A+models+of+bounded+rationality%2E+Psychological+review%2C+103%284%29%3A650%2E)
21. Alison Gopnik. 2023. Large language models as cultural technologies: Imitation and innovation in children and models. https://doi.org/10.48448/ 8qf0-j172. Keynote lecture at ACL 2023, Toronto, July 12, 2023. [find](https://scholar.google.com/scholar?q=Alison+Gopnik%2E+2023%2E+Large+language+models+as+cultural+technologies%3A+Imitation+and+innovation+in+children+and+models%2E+https%3A%2F%2Fdoi%2Eorg%2F10%2E48448%2F+8qf0%2Dj172%2E+Keynote+lecture+at+ACL+2023%2C+Toronto%2C+July+12)
22. Yanzhu Guo, Guokan Shang, and Chloé Clavel. 2024. [Benchmarking linguistic diversity of large language models](https://md.research.elamur.ai/free/en/papers/guo-2025-benchmarking-linguistic-diversity-large-language/). arXiv preprint arXiv:2412.10271.
23. Liwei Jiang, Yuanjun Chai, Margaret Li, Mickel Liu, Raymond Fok, Nouha Dziri, Yulia Tsvetkov, Maarten Sap, Alon Albalak, and Yejin Choi. 2025. Artificial hivemind: The open-ended homogeneity of language models (and beyond). arXiv preprint arXiv:2510.22954. — [arXiv:2510.22954](https://arxiv.org/abs/2510.22954) [find](https://scholar.google.com/scholar?q=Liwei+Jiang%2C+Yuanjun+Chai%2C+Margaret+Li%2C+Mickel+Liu%2C+Raymond+Fok%2C+Nouha+Dziri%2C+Yulia+Tsvetkov%2C+Maarten+Sap%2C+Alon+Albalak%2C+and+Yejin+Choi%2E+2025%2E+Artificial+hivemind%3A+The+open%2Dended+homogeneity+of+langua)
24. Pratik Joshi, Sebastin Santy, Amar Budhiraja, Kalika Bali, and Monojit Choudhury. 2020. [The state and fate of linguistic diversity and inclusion in the nlp world](https://md.research.elamur.ai/free/en/papers/joshi-2020-state-fate-linguistic-diversity-inclusion/). In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 6282–6293.
25. Alina Karakanta, Mayra Nas, and Aletta G. Dorst. 2025. Metaphors in literary machine translation: Close but no cigar? Proceedings of Machine Translation Summit XVII: Research Track. [find](https://scholar.google.com/scholar?q=Alina+Karakanta%2C+Mayra+Nas%2C+and+Aletta+G%2E+Dorst%2E+2025%2E+Metaphors+in+literary+machine+translation%3A+Close+but+no+cigar%3F+Proceedings+of+Machine+Translation+Summit+XVII%3A+Research+Track%2E)
26. Simon Kirby, Tom Griffiths, and Kenny Smith. 2014. Iterated learning and the evolution of language. Current opinion in neurobiology, 28:108–114. [find](https://scholar.google.com/scholar?q=Simon+Kirby%2C+Tom+Griffiths%2C+and+Kenny+Smith%2E+2014%2E+Iterated+learning+and+the+evolution+of+language%2E+Current+opinion+in+neurobiology%2C+28%3A108%E2%80%93114%2E)
27. Simon Kirby, Monica Tamariz, Hannah Cornish, and Kenny Smith. 2015. Compression and communication in the cultural evolution of linguistic structure. Cognition, 141:87–102. [find](https://scholar.google.com/scholar?q=Simon+Kirby%2C+Monica+Tamariz%2C+Hannah+Cornish%2C+and+Kenny+Smith%2E+2015%2E+Compression+and+communication+in+the+cultural+evolution+of+linguistic+structure%2E+Cognition%2C+141%3A87%E2%80%93102%2E)
28. Robert Kirk, Ishita Mediratta, Christoforos Nalmpantis, Jelena Luketina, Eric Hambro, Edward Grefenstette, and Roberta Raileanu. 2023. [Understanding the effects of rlhf on llm generalisation and diversity](https://md.research.elamur.ai/free/en/papers/kirk-2023-understanding-effects-rlhf-llm-generalisation/). In NeurIPS 2023 Workshop on Instruction Tuning and Instruction Following.
29. Dmitry Kobak, Rita González-Márquez, Emőke-Ágnes Horvát, and Jan Lause. 2024. Delving into chatgpt usage in academic writing through excess vocabulary. arXiv preprint arXiv:2406.07016. [find](https://scholar.google.com/scholar?q=Dmitry+Kobak%2C+Rita+Gonz%C3%A1lez%2DM%C3%A1rquez%2C+Emo%CC%8Bke%2D%C3%81gnes+Horv%C3%A1t%2C+and+Jan+Lause%2E+2024%2E+Delving+into+chatgpt+usage+in+academic+writing+through+excess+vocabulary%2E+arXiv+preprint+arXiv%3A2406%2E07016%2E)
30. András Kornai. 2013. Digital language death. PloS one, 8(10):e77056. [find](https://scholar.google.com/scholar?q=Andr%C3%A1s+Kornai%2E+2013%2E+Digital+language+death%2E+PloS+one%2C+8%2810%29%3Ae77056%2E)
31. Tom Kouwenhoven, Max Peeperkorn, Roy de Kleijn, and Tessa Verhoef. 2025a. Shaping shared languages: Human and large language models’ inductive biases in emergent communication. arXiv preprint arXiv:2503.04395. — [arXiv:2503.04395](https://arxiv.org/abs/2503.04395) [find](https://scholar.google.com/scholar?q=Tom+Kouwenhoven%2C+Max+Peeperkorn%2C+Roy+de+Kleijn%2C+and+Tessa+Verhoef%2E+2025a%2E+Shaping+shared+languages%3A+Human+and+large+language+models%E2%80%99+inductive+biases+in+emergent+communication%2E+arXiv+preprint+arXiv%3A25)
32. Tom Kouwenhoven, Max Peeperkorn, and Tessa Verhoef. 2025b. Searching for structure: Investigating emergent communication with large language models. In Proceedings of the 31st International Conference on Computational Linguistics, pages 9977– 9991, Abu Dhabi, UAE. Association for Computational Lingu… [find](https://scholar.google.com/scholar?q=Tom+Kouwenhoven%2C+Max+Peeperkorn%2C+and+Tessa+Verhoef%2E+2025b%2E+Searching+for+structure%3A+Investigating+emergent+communication+with+large+language+models%2E+In+Proceedings+of+the+31st+International+Conference)
33. Svenja Kranich. 2014. Translations as a locus of language contact. In Translation: A multidisciplinary approach, pages 96–115. Springer. [find](https://scholar.google.com/scholar?q=Svenja+Kranich%2E+2014%2E+Translations+as+a+locus+of+language+contact%2E+In+Translation%3A+A+multidisciplinary+approach%2C+pages+96%E2%80%93115%2E+Springer%2E)
34. Lionbridge. 2020. Localizing long-tail languages. [https://www.lionbridge.com/blog/](https://www.lionbridge.com/blog/) translation-localization/localizing-long-tail-languages/. Accessed: 17 April 2026. [↗](https://www.lionbridge.com/blog/)
35. Jiaming Luo, Colin Cherry, and George Foster. 2024. [To diverge or not to diverge: A morphosyntactic perspective on machine translation vs human translation](https://md.research.elamur.ai/free/en/papers/luo-2024-diverge-or-not-diverge-morphosyntactic/). Transactions of the Association for Computational Linguistics, 12:355–371.
36. Gary Lupyan and Rick Dale. 2010. Language structure is partly determined by social structure. PloS one, 5(1):e8559. [find](https://scholar.google.com/scholar?q=Gary+Lupyan+and+Rick+Dale%2E+2010%2E+Language+structure+is+partly+determined+by+social+structure%2E+PloS+one%2C+5%281%29%3Ae8559%2E)
37. Gonzalo Martínez, Lauren Watson, Pedro Reviriego, José Alberto Hernández, Marc Juarez, and Rik Sarkar. 2023. Towards understanding the interplay of generative artificial intelligence and the internet. In International Workshop on Epistemic Uncertainty in Artificial Intelligence, pages 59–73. Springe… [find](https://scholar.google.com/scholar?q=Gonzalo+Mart%C3%ADnez%2C+Lauren+Watson%2C+Pedro+Reviriego%2C+Jos%C3%A9+Alberto+Hern%C3%A1ndez%2C+Marc+Juarez%2C+and+Rik+Sarkar%2E+2023%2E+Towards+understanding+the+interplay+of+generative+artificial+intelligence+and+the+internet%2E)
38. R Thomas McCoy, Shunyu Yao, Dan Friedman, Matthew Hardy, and Thomas L Griffiths. 2023. Embers of autoregression: Understanding large language models through the problem they are trained to solve. arXiv preprint arXiv:2309.13638. [find](https://scholar.google.com/scholar?q=R+Thomas+McCoy%2C+Shunyu+Yao%2C+Dan+Friedman%2C+Matthew+Hardy%2C+and+Thomas+L+Griffiths%2E+2023%2E+Embers+of+autoregression%3A+Understanding+large+language+models+through+the+problem+they+are+trained+to+solve%2E+arXi)
39. Jean-Baptiste Michel, Yuan Kui Shen, Aviva Presser Aiden, Adrian Veres, Matthew K Gray, Google Books Team, Joseph P Pickett, Dale Hoiberg, Dan Clancy, Peter Norvig, et al. 2011. Quantitative analysis of culture using millions of digitized books. science, 331(6014):176–182. [find](https://scholar.google.com/scholar?q=Jean%2DBaptiste+Michel%2C+Yuan+Kui+Shen%2C+Aviva+Presser+Aiden%2C+Adrian+Veres%2C+Matthew+K+Gray%2C+Google+Books+Team%2C+Joseph+P+Pickett%2C+Dale+Hoiberg%2C+Dan+Clancy%2C+Peter+Norvig%2C+et+al%2E+2011%2E+Quantitative+analysis+)
40. Steven Mithen. 2024. The language puzzle: How we talked our way out of the stone age. Profile Books. [find](https://scholar.google.com/scholar?q=Steven+Mithen%2E+2024%2E+The+language+puzzle%3A+How+we+talked+our+way+out+of+the+stone+age%2E+Profile+Books%2E)
41. Niklas Muennighoff, Thomas Wang, Lintang Sutawika, Adam Roberts, Stella Biderman, Teven Le Scao, M Saiful Bari, Sheng Shen, Zheng-Xin Yong, Hailey Schoelkopf, et al. 2022. [Crosslingual generalization through multitask finetuning](https://md.research.elamur.ai/free/en/papers/muennighoff-2022-crosslingual-generalization-through-multitask-finetuning/). arXiv preprint arXiv:2211.01786.
42. Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. 2022. [Training language models to follow instructions with human feedback](https://md.research.elamur.ai/free/en/papers/ouyang-2022-training-language-models-follow-instructions/). Advances in neural information processing systems, 35:27730–27744.
43. Mark Pagel, Mark Beaumont, Andrew Meade, Annemarie Verkerk, and Andreea Calude. 2019. Dominant words rise to the top by positive frequency-dependent selection. Proceedings of the National Academy of Sciences, 116(15):7397–7402. [find](https://scholar.google.com/scholar?q=Mark+Pagel%2C+Mark+Beaumont%2C+Andrew+Meade%2C+Annemarie+Verkerk%2C+and+Andreea+Calude%2E+2019%2E+Dominant+words+rise+to+the+top+by+positive+frequency%2Ddependent+selection%2E+Proceedings+of+the+National+Academy+of+S)
44. Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. [Bleu: a method for automatic evaluation of machine translation](https://md.research.elamur.ai/free/en/papers/papineni-2002-bleu-method-automatic-evaluation-machine/). In Proceedings of the 40th annual meeting of the Association for Computational Linguistics, pages 311–318.
45. Alexander M Petersen, Joel Tenenbaum, Shlomo Havlin, and H Eugene Stanley. 2012. Statistical laws governing fluctuations in word use from word birth to word death. Scientific reports, 2(1):313. [find](https://scholar.google.com/scholar?q=Alexander+M+Petersen%2C+Joel+Tenenbaum%2C+Shlomo+Havlin%2C+and+H+Eugene+Stanley%2E+2012%2E+Statistical+laws+governing+fluctuations+in+word+use+from+word+birth+to+word+death%2E+Scientific+reports%2C+2%281%29%3A313%2E)
46. Steven T Piantadosi, Harry Tily, and Edward Gibson. 2011. Word lengths are optimized for efficient communication. Proceedings of the National Academy of Sciences, 108(9):3526–3529. [find](https://scholar.google.com/scholar?q=Steven+T+Piantadosi%2C+Harry+Tily%2C+and+Edward+Gibson%2E+2011%2E+Word+lengths+are+optimized+for+efficient+communication%2E+Proceedings+of+the+National+Academy+of+Sciences%2C+108%289%29%3A3526%E2%80%933529%2E)
47. Plato. 2002. Phaedrus. Oxford World Classics. Oxford University Press, Oxford. [find](https://scholar.google.com/scholar?q=Plato%2E+2002%2E+Phaedrus%2E+Oxford+World+Classics%2E+Oxford+University+Press%2C+Oxford%2E)
48. Florencia Reali, Nick Chater, and Morten H Christiansen. 2018. Simpler grammar, larger vocabulary: How population size affects language. Proceedings of the Royal Society B: Biological Sciences, 285(1871):20172586. [find](https://scholar.google.com/scholar?q=Florencia+Reali%2C+Nick+Chater%2C+and+Morten+H+Christiansen%2E+2018%2E+Simpler+grammar%2C+larger+vocabulary%3A+How+population+size+affects+language%2E+Proceedings+of+the+Royal+Society+B%3A+Biological+Sciences%2C+285%2818)
49. Ricardo Rei, Craig Stewart, Ana C Farinha, and Alon Lavie. 2020. [COMET: A neural framework for MT evaluation](https://md.research.elamur.ai/free/en/papers/rei-2020-comet-neural-framework-mt-evaluation/). In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 2685–2702, Online. Association for Computational Linguistics.
50. Yi Ren, Shangmin Guo, Linlu Qiu, Bailin Wang, and Danica J Sutherland. 2024a. Bias amplification in language model evolution: An iterated learning perspective. arXiv preprint arXiv:2404.04286. — [arXiv:2404.04286](https://arxiv.org/abs/2404.04286) [find](https://scholar.google.com/scholar?q=Yi+Ren%2C+Shangmin+Guo%2C+Linlu+Qiu%2C+Bailin+Wang%2C+and+Danica+J+Sutherland%2E+2024a%2E+Bias+amplification+in+language+model+evolution%3A+An+iterated+learning+perspective%2E+arXiv+preprint+arXiv%3A2404%2E04286%2E+%E2%80%94+%5BarXi)
51. Yi Ren, Shangmin Guo, Linlu Qiu, Bailin Wang, and Danica J Sutherland. 2024b. Bias amplification in language model evolution: An iterated learning perspective. arXiv preprint arXiv:2404.04286. — [arXiv:2404.04286](https://arxiv.org/abs/2404.04286) [find](https://scholar.google.com/scholar?q=Yi+Ren%2C+Shangmin+Guo%2C+Linlu+Qiu%2C+Bailin+Wang%2C+and+Danica+J+Sutherland%2E+2024b%2E+Bias+amplification+in+language+model+evolution%3A+An+iterated+learning+perspective%2E+arXiv+preprint+arXiv%3A2404%2E04286%2E+%E2%80%94+%5BarXi)
52. Pedro Reviriego, Javier Conde, Elena Merino-Gómez, Gonzalo Martínez, and José Alberto Hernández. 2024. Playing with words: Comparing the vocabulary and lexical diversity of chatgpt and humans. Machine Learning with Applications, 18:100602. [find](https://scholar.google.com/scholar?q=Pedro+Reviriego%2C+Javier+Conde%2C+Elena+Merino%2DG%C3%B3mez%2C+Gonzalo+Mart%C3%ADnez%2C+and+Jos%C3%A9+Alberto+Hern%C3%A1ndez%2E+2024%2E+Playing+with+words%3A+Comparing+the+vocabulary+and+lexical+diversity+of+chatgpt+and+humans%2E+Machine)
53. Yu Sasaki. 2017. Publishing nations: Technology acquisition and language standardization for european ethnic groups. The Journal of Economic History, 77(4):1007–1047. [find](https://scholar.google.com/scholar?q=Yu+Sasaki%2E+2017%2E+Publishing+nations%3A+Technology+acquisition+and+language+standardization+for+european+ethnic+groups%2E+The+Journal+of+Economic+History%2C+77%284%29%3A1007%E2%80%931047%2E)
54. Rylan Schaeffer, Joshua Kazdan, Alvan Caleb Aru-landu, and Sanmi Koyejo. 2025. Position: Model collapse does not mean what you think. arXiv preprint arXiv:2503.03150. — [arXiv:2503.03150](https://arxiv.org/abs/2503.03150) [find](https://scholar.google.com/scholar?q=Rylan+Schaeffer%2C+Joshua+Kazdan%2C+Alvan+Caleb+Aru%2Dlandu%2C+and+Sanmi+Koyejo%2E+2025%2E+Position%3A+Model+collapse+does+not+mean+what+you+think%2E+arXiv+preprint+arXiv%3A2503%2E03150%2E+%E2%80%94+%5BarXiv%3A2503%2E03150%5D%28https%3A%2F%2Farxi)
55. Ilia Shumailov, Zakhar Shumaylov, Yiren Zhao, Yarin Gal, Nicolas Papernot, and Ross Anderson. 2023. [The curse of recursion: Training on generated data makes models forget](https://md.research.elamur.ai/free/en/papers/shumailov-2023-curse-recursion-training-generated-data/). arXiv preprint arXiv:2305.17493.
56. Kenny Smith, Henry Brighton, and Simon Kirby. 2003a. Complex systems in language evolution: the cultural emergence of compositional structure. Advances in complex systems, 6(04):537–558. [find](https://scholar.google.com/scholar?q=Kenny+Smith%2C+Henry+Brighton%2C+and+Simon+Kirby%2E+2003a%2E+Complex+systems+in+language+evolution%3A+the+cultural+emergence+of+compositional+structure%2E+Advances+in+complex+systems%2C+6%2804%29%3A537%E2%80%93558%2E)
57. Kenny Smith, Simon Kirby, and Henry Brighton. 2003b. Iterated learning: A framework for the emergence of language. Artificial life, 9(4):371–386. [find](https://scholar.google.com/scholar?q=Kenny+Smith%2C+Simon+Kirby%2C+and+Henry+Brighton%2E+2003b%2E+Iterated+learning%3A+A+framework+for+the+emergence+of+language%2E+Artificial+life%2C+9%284%29%3A371%E2%80%93386%2E)
58. Linting Xue, Noah Constant, Adam Roberts, Mihir Kale, Rami Al-Rfou, Aditya Siddhant, Aditya Barua, and Colin Raffel. 2021. [mt5: A massively multilingual pre-trained text-to-text transformer](https://md.research.elamur.ai/free/en/papers/xue-2021-mt5-massively-multilingual-pre-trained/). In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Ling…
59. Yongjing Yin, Lian Fu, Yafu Li, and Yue Zhang. 2024. On compositional generalization of transformer-based neural machine translation. Information Fusion, 111:102491. [find](https://scholar.google.com/scholar?q=Yongjing+Yin%2C+Lian+Fu%2C+Yafu+Li%2C+and+Yue+Zhang%2E+2024%2E+On+compositional+generalization+of+transformer%2Dbased+neural+machine+translation%2E+Information+Fusion%2C+111%3A102491%2E)
60. Longfei Yun, Chenyang An, Zilong Wang, Letian Peng, and Jingbo Shang. 2025. The price of format: Diversity collapse in llms. [find](https://scholar.google.com/scholar?q=Longfei+Yun%2C+Chenyang+An%2C+Zilong+Wang%2C+Letian+Peng%2C+and+Jingbo+Shang%2E+2025%2E+The+price+of+format%3A+Diversity+collapse+in+llms%2E)
61. Xiang Zhou, Yichen Jiang, and Mohit Bansal. 2023. Data factors for better compositional generalization. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 14549–14566. [find](https://scholar.google.com/scholar?q=Xiang+Zhou%2C+Yichen+Jiang%2C+and+Mohit+Bansal%2E+2023%2E+Data+factors+for+better+compositional+generalization%2E+In+Proceedings+of+the+2023+Conference+on+Empirical+Methods+in+Natural+Language+Processing%2C+pages)
62. Matthew Snover, Bonnie Dorr, Richard Schwartz, Linnea Micciulla, and John Makhoul. 2006. [A study of translation edit rate with targeted human annotation](https://md.research.elamur.ai/free/en/papers/snover-2006-study-translation-edit-rate-targeted/). In Proceedings of the 7th Conference of the Association for Machine Translation in the Americas: Technical Papers, pages 223–231.
63. Brian Thompson, Mehak Dhaliwal, Peter Frisch, Tobias Domhan, and Marcello Federico. 2024. A shocking amount of the web is machine translated: Insights from multi-way parallelism. In Findings of the Association for Computational Linguistics: ACL 2024, pages 1763–1775, Bangkok, Thailand. Association f… [find](https://scholar.google.com/scholar?q=Brian+Thompson%2C+Mehak+Dhaliwal%2C+Peter+Frisch%2C+Tobias+Domhan%2C+and+Marcello+Federico%2E+2024%2E+A+shocking+amount+of+the+web+is+machine+translated%3A+Insights+from+multi%2Dway+parallelism%2E+In+Findings+of+the+As)
64. Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. 2023. [Llama 2: Open foundation and fine-tuned chat models](https://md.research.elamur.ai/free/en/papers/touvron-2023-llama-2-open-foundation-fine/). arXiv preprint arXiv:2307.09288.
65. Eva Vanmassenhove, Dimitar Shterionov, and Matthew Gwilliam. 2021. [Machine translationese: Effects of algorithmic bias on linguistic complexity in machine translation](https://md.research.elamur.ai/free/en/papers/vanmassenhove-2021-machine-translationese-effects-algorithmic-bias/). In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume, pages 2…
66. Eva Vanmassenhove, Dimitar Shterionov, and Andy Way. 2019. [Lost in translation: Loss and decay of linguistic richness in machine translation](https://md.research.elamur.ai/free/en/papers/vanmassenhove-2019-lost-translation-loss-decay-linguistic/). In Proceedings of Machine Translation Summit XVII: Research Track, pages 222–232, Dublin, Ireland. European Association for Machine Translation.
67. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. [Attention is all you need](https://md.research.elamur.ai/free/en/papers/vaswani-2017-attention-all-need/). Advances in neural information processing systems, 30.

*Updated 1 November 2026*