---
title: "Neutral evolution and turnover over centuries of English word"
description: "The proposed partial-sampling Neutral model (PNM) adds a second layer. 48 references. Among them: 0."
authors: ["Damian Ruck", "R. Alexander Bentley", "Alberto Acerbi", "Philip Garnett", "Daniel J. Hruschka"]
year: 2017
venue: "arXiv:1703.10698"
source: "https://arxiv.org/abs/1703.10698"
doi: "10.48550/arxiv.1703.10698"
arxiv: "1703.10698"
licence: "arXiv — non-exclusive license to distribute"
licence_url: "http://arxiv.org/licenses/nonexclusive-distrib/1.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/ruck-2017-neutral-evolution-turnover-over-centuries/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/ruck-2017-neutral-evolution-turnover-over-centuries/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2017](https://md.research.elamur.ai/free/en/papers/2017/page/5/) / Neutral evolution and turnover over centuries of English word popularity

# Neutral evolution and turnover over centuries of English word popularity

**Authors:** Damian Ruck, R. Alexander Bentley, Alberto Acerbi, Philip Garnett and Daniel J. Hruschka · **Year:** 2017 · **Venue:** arXiv:1703.10698 · **Licence:** [arXiv — non-exclusive license to distribute](http://arxiv.org/licenses/nonexclusive-distrib/1.0/)

**Links:** [Publisher's page](https://arxiv.org/abs/1703.10698) · [PDF](https://arxiv.org/pdf/1703.10698) · [DOI 10.48550/arxiv.1703.10698](https://doi.org/10.48550/arxiv.1703.10698) · [arXiv 1703.10698](https://arxiv.org/abs/1703.10698) · [OpenAlex](https://openalex.org/W2604480082)

**Citations:** 0 citations (OpenAlex, 2026-10-14)

## Summary
The paper asks whether unbiased random copying can explain the long-term statistics of English word use. The authors use the Google Ngram English 2012 1-grams, covering books from 1700 to 2000. They test two neutral models against three properties of the data: the Zipf-like frequency distribution, Heaps law (vocabulary grows more slowly than corpus size), and turnover among the most frequent words. In the standard full-sampling Neutral model (FNM), writers copy words from the previous year's books and occasionally invent new ones. The FNM reproduces Zipf's law but makes vocabulary grow linearly with corpus size. The proposed partial-sampling Neutral model (PNM) adds a second layer. A small canon of fixed yearly size evolves neutrally, and an exponentially growing body of books samples from the canon accumulated since 1700. This two-layer structure matches the frequency distribution, a Heaps exponent near 0.5, and slowing turnover. The result matters because it suggests that concentrated attention on a small, highly visible subset, with no word-level fitness, can account for the observed statistics.

## Key points
- Data: English 1-grams from Google Ngram (2012 set). Symbols, numbers, and tokens with three identical consecutive consonants were removed. Counts were normalized by the yearly count of "the". Turnover was analysed only from 1800 onward because of OCR errors confusing "s" and "f".
- Measured Zipf exponent for 1700–1800 is α = 1.75 ± 0.12, estimated by maximum likelihood.
- Heaps exponent over 100 runs:
  - PNM gives β ≈ 0.52 ± 0.006, inside the range reported for several Ngram sub-corpora.
  - FNM gives β ≈ 1 ± 0.002, which does not match the data.
- Model settings were found with a genetic algorithm search:
  - innovation rate µ = 0.003;
  - growth exponent 0.021, taking the sample from 3000 to 1.5 million words over 301 steps;
  - the canon adds 10,000 words per year.
- The Python code for both models is published on GitHub (dr2g08).

## Interesting ideas
- The paper uses a no-bias copying model as a null hypothesis, so that systematic deviations from its predicted turnover curve (z_y ∝ y^0.86) can be read as evidence of conformity or anti-conformity.
- A model is accepted only if it reproduces several static and dynamic statistics at the same time, because one matching statistic (Zipf) cannot separate competing models.
- Yearly counts are normalized by the frequency of the most common word, which gives a stable reference across corpora of very different sizes.
- Cheap character-level heuristics clean OCR-era noise: dropping tokens with triple repeated consonants and distrusting text where "s" and "f" are confused.
- A small high-visibility "canon" sampled by a much larger population explains sublinear vocabulary growth, a "small core feeding a large periphery" design.

## For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

- **I1** (learn): Build a random-choice null baseline for each evaluation, so that a gain from a repair rule or teacher tier is reported against what chance-level behaviour would already produce.
- **I2** (learn): Score parsers and silver data on several independent distributional checks at once (label distributions, attachment-length profiles, per-genre stability) rather than on LAS alone.
- **I3** (tokens): Store lemma frequency priors relative to a high-frequency anchor word, so priors from small and large corpora can be merged comparably.
- **I4** (tokens): Add explainable noise filters like these in the tokenizer when ingesting digitized old fables and tales, flagging suspicious forms instead of tagging them blindly.
- **I5** (world): Model compiled knowledge as a compact, curated core (frequent verb classes, ontology nodes) that most texts draw on, with a long tail handled by fallback lookup, and track turnover in the core as the knowledge base grows.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [University of York](https://md.research.elamur.ai/free/en/universities/university-of-york/) — University, York, United Kingdom · 13 works · 6,512 citations
- [Arizona State University](https://md.research.elamur.ai/free/en/universities/arizona-state-university/) — University, Tempe, United States · 13 works · 1,572 citations
- [University of Houston](https://md.research.elamur.ai/free/en/universities/university-of-houston/) — University, Houston, United States · 7 works · 1,105 citations
- [University of Bristol](https://md.research.elamur.ai/free/en/universities/university-of-bristol/) — University, Bristol, United Kingdom · 7 works · 658 citations
- [Eindhoven University of Technology](https://md.research.elamur.ai/free/en/universities/eindhoven-university-of-technology/) — University, Eindhoven, Netherlands · 4 works · 525 citations

📚 Shelves (6): [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 9,296 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 9,206 works · [🤝 Social Sciences](https://md.research.elamur.ai/free/en/shelves/social-sciences/) · 744 works · [🧬 Biochemistry, Genetics and Molecular Biology](https://md.research.elamur.ai/free/en/shelves/biochemistry-genetics-and-molecular-biology/) · 116 works · [🤝 Cultural Studies](https://md.research.elamur.ai/free/en/shelves/social-sciences/cultural-studies/) · 10 works · [🧬 Genetics](https://md.research.elamur.ai/free/en/shelves/biochemistry-genetics-and-molecular-biology/genetics/) · 2 works

🧭 Topics (1): [English](https://md.research.elamur.ai/free/en/topics/english/) · 1,373 works

## Cite

Cite as: Damian Ruck, R. Alexander Bentley, Alberto Acerbi, Philip Garnett and Daniel J. Hruschka. 2017. Neutral evolution and turnover over centuries of English word popularity. arXiv:1703.10698.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@misc{ruck2017neutral,
  title = {{Neutral evolution and turnover over centuries of English word popularity}},
  author = {Damian Ruck, R. Alexander Bentley, Alberto Acerbi, Philip Garnett and Daniel J. Hruschka},
  year = {2017},
  doi = {10.48550/arxiv.1703.10698},
  eprint = {1703.10698},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.1703.10698}
}
```

- [← previous Modern English grammar: Morphology and syntax (a review)](https://md.research.elamur.ai/free/en/papers/malyuga-2017-modern-english-grammar-morphology-syntax/)
- [next → New perspectives on cohesion and coherence: Implications for translation](https://md.research.elamur.ai/free/en/papers/kunz-2017-new-perspectives-cohesion-coherence/)
- [↑ back to papers of 2017](https://md.research.elamur.ai/free/en/papers/2017/page/5/)

## References

48 references, 1 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Acerbi, A, Lampos V, Garnett P, Bentley RA (2013). The expression of emotions in 20th century books. PLoS ONE 8(3): e59030. [↗](https://doi.org/10.1371/journal.pone.0059030)
2. Acerbi A, Bentley RA (2014). Biases in cultural transmission shape the turnover of popular traits. Evolution & Human Behavior 35: 228–236. [↗](https://doi.org/10.1016/j.evolhumbehav.2014.02.003)
3. Altmann EG, Pierrehumbert JB, Motter AE (2011). Niche as a determinant of word fate in online groups. PLoS ONE 6(5): e19009. [↗](https://doi.org/10.1371/journal.pone.0019009)
4. Batty M (2006). Rank clocks. Nature 444: 592–596. [find](https://scholar.google.com/scholar?q=Batty+M+%282006%29%2E+Rank+clocks%2E+Nature+444%3A+592%E2%80%93596%2E)
5. Barucca P, Rocchi J, Marinari E, Parisi G, Ricci-Tersenghi F (2015). Cross-correlations of American baby names. PNAS 112: 7943–7947. [↗](https://doi.org/10.1073/pnas.1507143112)
6. Bentley RA, Acerbi A, Lampos V, Ormerod P (2014). Books average previous decade of economic misery. PLoS ONE 9(1): e83147. [↗](https://doi.org/10.1371/journal.pone.0083147)
7. Bentley RA, Caiado C, Ormerod P (2014). Effects of memory on spatial heterogeneity in neutrally transmitted culture. Evolution & Human Behavior 35: 257–263. [↗](https://doi.org/10.1016/j.evolhumbehav.2014.02.001)
8. Bentley RA,Garnett P, O’Brien MJ, Brock WA (2012). Word diffusion and climate science. PLoS ONE 7(11): e47966. [↗](https://doi.org/10.1371/journal.pone.0047966)
9. Bentley RA, Ormerod P, Batty M (2011). Evolving social influence in large populations. Behavioral Ecology & Sociobiology 65: 537–546. [↗](https://doi.org/10.1007/s00265-010-1102-1)
10. Bentley RA, Shennan SJ, Ormerod P (2011). Population-level neutral model already explains linguistic patterns. Proceedings B 278: 1770–1772. [↗](https://doi.org/10.1098/rspb.2010.2581)
11. Bentley RA, Hahn MW, Shennan SJ (2004). Random drift and culture change. Proceedings B 271: 1443–1450. [↗](https://doi.org/10.1098/rspb.2004.2746)
12. Bentley RA, Lipo CP, Herzog HA, Hahn MW (2007). Regular rates of popular culture change reflect random copying. Evolution & Human Behavior 28: 151–158. [↗](https://doi.org/10.1016/j.evolhumbehav.2006.10.002)
13. Bentley RA (2008). Random drift versus selection in academic vocabulary. PLoS ONE 3(8): e3057. [find](https://scholar.google.com/scholar?q=Bentley+RA+%282008%29%2E+Random+drift+versus+selection+in+academic+vocabulary%2E+PLoS+ONE+3%288%29%3A+e3057%2E)
14. Christiansen MH, Chater N (2008). Language as shaped by the brain. Behavioral & Brain Sciences 31: 489–509. [↗](https://doi.org/10.1017/s0140525x08004998)
15. Clauset A, Shalizi CR, Newman MEJ (2007). Power-law distributions in empirical data. SIAM Review 51: 661–703. [find](https://scholar.google.com/scholar?q=Clauset+A%2C+Shalizi+CR%2C+Newman+MEJ+%282007%29%2E+Power%2Dlaw+distributions+in+empirical+data%2E+SIAM+Review+51%3A+661%E2%80%93703%2E)
16. Dehaene S, Mehler J (1992). Cross-linguistic regularities in the frequency of number words. Cognition 43: 1–29. [↗](https://doi.org/10.1016/0010-0277(92)90030-l)
17. Eriksson K, Jansson F, Sjöstrand, J (2010). Bentley’s conjecture on popularity toplist turnover under random copying. Ramanujan Journal 23: 371–396. [↗](https://doi.org/10.1007/s11139-010-9280-z)
18. Evans TS (2007). Exact solutions for network rewiring models. European Physical Journal B 56: 65–69. [↗](https://doi.org/10.1140/epjb/e2007-00084-8)
19. Evans, TS, Giometto, A (2011). Turnover rate of popularity charts in neutral models. arXiv: 11054044v1. [find](https://scholar.google.com/scholar?q=Evans%2C+TS%2C+Giometto%2C+A+%282011%29%2E+Turnover+rate+of+popularity+charts+in+neutral+models%2E+arXiv%3A+11054044v1%2E)
20. Ferrer i Cancho R, Riordan O, Bollobás B (2005). The consequences of Zipf’s law for syntax and symbolic reference. Proceedings B 2005; 272: 561–565. [↗](https://doi.org/10.1098/rspb.2004.2957)
21. Gabaix X (2009). Power laws in economics and finance. Annual Review of Economics 1: 255-293. [↗](https://doi.org/10.1146/annurev.economics.050708.142940)
22. Gao J, Hu J, Mao X, Perc M (2012). Culturomics meets random fractal theory: Insights into long-range correlations of social and natural phenomena over the past two centuries. J. R Soc Interface 9: 1956–1964. [↗](https://doi.org/10.1098/rsif.2011.0846)
23. Ghoshal G, Barabási A-L (2011). Ranking stability and super-stable nodes in complex networks. Nature Communications 2: 394. [↗](https://doi.org/10.1038/ncomms1396)
24. Gleeson JP, Cellai D, Onnela J-P, Porter MA, Reed-Tsochas F (2014). A simple generative model of collective online behavior. PNAS 111: 10411–10415. [↗](https://doi.org/10.1073/pnas.1313895111)
25. Hahn MW, Bentley RA (2003). Drift as a mechanism for cultural change: an example from baby names. Proceedings B 270: S1–S4. [↗](https://doi.org/10.1098/rsbl.2003.0045)
26. Hruschka DJ, Christiansen MH, Blythe RA, Croft W, Heggarty P, Mufwene SS, Pierrehumbert JB, Poplack S (2009). Building social cognitive models of language change. Trends in Cognitive Sciences 13: 464–469. [↗](https://doi.org/10.1016/j.tics.2009.08.008)
27. Hughes JM, Foti NJ, Krakauer DC, Rockmore DN (2012). Quantitative patterns of stylistic influence in the evolution of literature. PNAS 109: 7682–7686. [↗](https://doi.org/10.1073/pnas.1115407109)
28. Kandler A, Shennan S (2013). A non-equilibrium neutral model for analysing cultural change. J. Theoretical Biology 330: 18–25. [↗](https://doi.org/10.1016/j.jtbi.2013.03.006)
29. Laherrère J, Sornette D (1998). Stretched exponential distributions in nature and economy: ‘fat tails’ with characteristic scales. European Physical Journal B 2: 525–539. [↗](https://doi.org/10.1007/s100510050276)
30. Lanfear R, Kokko H, Eyre-Walker A (2014). Population size and the rate of evolution. Trends in Ecology & Evolution 29: 33–41. [↗](https://doi.org/10.5167/uzh-177114)
31. Li W (1992). Random texts exhibit Zipf’s-law-like word frequency distribution. IEEE Trans Inf Theory 38: 1842–1845. [↗](https://doi.org/10.1109/18.165464)
32. Lieberman E, Hauert C, Nowak MA (2005). Evolutionary dynamics on graphs. Nature 433: 312–316. [find](https://scholar.google.com/scholar?q=Lieberman+E%2C+Hauert+C%2C+Nowak+MA+%282005%29%2E+Evolutionary+dynamics+on+graphs%2E+Nature+433%3A+312%E2%80%93316%2E)
33. Lieberman E, Michel J-P, Jackson J, Tang T, Nowak MA (2007). Quantifying the evolutionary dynamics of language. Nature 449: 713–716. [find](https://scholar.google.com/scholar?q=Lieberman+E%2C+Michel+J%2DP%2C+Jackson+J%2C+Tang+T%2C+Nowak+MA+%282007%29%2E+Quantifying+the+evolutionary+dynamics+of+language%2E+Nature+449%3A+713%E2%80%93716%2E)
34. Lin Y, Michel JB, Aiden EL, Orwant J, Brockman W, Petrov S (2012). Syntactic annotations for the google books ngram corpus. In: Proceedings of the ACL 2012 System Demonstrations. Association for Computational Linguistics, pp.169–174. [find](https://scholar.google.com/scholar?q=Lin+Y%2C+Michel+JB%2C+Aiden+EL%2C+Orwant+J%2C+Brockman+W%2C+Petrov+S+%282012%29%2E+Syntactic+annotations+for+the+google+books+ngram+corpus%2E+In%3A+Proceedings+of+the+ACL+2012+System+Demonstrations%2E+Association+for+Compu)
35. Lü L, Zhang Z-K, Zhou T (2010). Zipf’s Law leads to Heaps’ Law: Analyzing their relation in finite-size systems. PLoS ONE 5(12): e14139. [↗](https://doi.org/10.1371/journal.pone.0014139)
36. Michel JB, Shen YK, Aiden AP, Veres A, Gray MK, Pickett JP, Hoiberg D, Clancy D, Norvig P, Orwant J, Pinker S, Nowak MA, Aiden EL (2011). Quantitative analysis of culture using millions of digitized books. Science 331:176–182. [↗](https://doi.org/10.1126/science.1199644)
37. Neiman FD (1995). Stylistic variation in evolutionary perspective. American Antiquity 60: 7–36. [find](https://scholar.google.com/scholar?q=Neiman+FD+%281995%29%2E+Stylistic+variation+in+evolutionary+perspective%2E+American+Antiquity+60%3A+7%E2%80%9336%2E)
38. Pagel M, Atkinson QD, Meade A (2007). Frequency of word-use predicts rates of lexical evolution throughout Indo-European history. Nature 449: 717–721. [↗](https://doi.org/10.1038/nature06176)
39. Pan RK, Petersen AM, Pammolli F, Fortunato S (2016). The memory of science: inflation, myopia, and the knowledge network. arXiv: 160705606v1. [find](https://scholar.google.com/scholar?q=Pan+RK%2C+Petersen+AM%2C+Pammolli+F%2C+Fortunato+S+%282016%29%2E+The+memory+of+science%3A+inflation%2C+myopia%2C+and+the+knowledge+network%2E+arXiv%3A+160705606v1%2E)
40. Perc M (2012). [Evolution of the most common English words and phrases over the centuries](https://md.research.elamur.ai/free/en/papers/perc-2012-evolution-most-common-english-words/). J R Soc Interface 9: 3323–3328. [↗](https://doi.org/10.1098/rsif.2012.0491)
41. Petersen AM, Tenenbaum J, Havlin S, Stanley HE, Perc M (2012). Languages cool as they expand: Allometric scaling and the decreasing need for new words. Scientific Reports 2: 943. [↗](https://doi.org/10.1038/srep00943)
42. Petersen AM, Tenenbaum J, Havlin S, Stanley HE (2012). Statistical laws governing fluctuations in word use from Word Birth to Word Death. Scientific Reports 2: 313. [↗](https://doi.org/10.1038/srep00313)
43. Piantadosi ST, Tily H, Gibson E (2011). Word lengths are optimized for efficient communication. PNAS 108: 3526–3529. [↗](https://doi.org/10.1073/pnas.1012551108)
44. Reali F, Griffiths TL (2010). Words as alleles: connecting language evolution with Bayesian learners to models of genetic drift. Proceedings B 277: 429–436. [↗](https://doi.org/10.1098/rspb.2009.1513)
45. Sigurd B, Eeg-Olofsson M, van de Weijer J (2004). Word length, sentence length and frequency–Zipf revisited. Studia Linguistica 58: 37–52. [↗](https://doi.org/10.1111/j.0039-3193.2004.00109.x)
46. Strimling P, Sjöstrand J, Eriksson K, Enquist M (2009). Accumulation of cultural traits. Theoretical Population Biology 76: 77–83. [find](https://scholar.google.com/scholar?q=Strimling+P%2C+Sjo%CC%88strand+J%2C+Eriksson+K%2C+Enquist+M+%282009%29%2E+Accumulation+of+cultural+traits%2E+Theoretical+Population+Biology+76%3A+77%E2%80%9383%2E)
47. Williams JR, Lessard PR, Desu S, Clark E, Bagrow JP, Danforth CM, Dodds PS (2015). Zipf’s law holds for phrases, not words. Scientific Reports 5: 12209. [↗](https://doi.org/10.1038/srep12209)
48. Zipf GK (1949). Human Behavior and the Principle of Least Effort. Cambridge, MA: Addison Wesley. [find](https://scholar.google.com/scholar?q=Zipf+GK+%281949%29%2E+Human+Behavior+and+the+Principle+of+Least+Effort%2E+Cambridge%2C+MA%3A+Addison+Wesley%2E)

*Updated 24 October 2026*