---
title: "Is language evolution grinding to a halt? The scaling of lexical — paper by Pechenick et al. (2015)"
description: "The authors re-examine that claim on the Google Books 2012 English Fiction corpus for 1800–2000. Cited by 2 papers here."
authors: ["Eitan Adam Pechenick", "Christopher M. Danforth", "Peter Sheridan Dodds"]
year: 2015
venue: "arXiv:1503.03512"
source: "https://arxiv.org/abs/1503.03512"
doi: "10.48550/arxiv.1503.03512"
arxiv: "1503.03512"
licence: "arXiv — non-exclusive license to distribute"
licence_url: "http://arxiv.org/licenses/nonexclusive-distrib/1.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/pechenick-2015-language-evolution-grinding-halt-scaling/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/pechenick-2015-language-evolution-grinding-halt-scaling/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2015](https://md.research.elamur.ai/free/en/papers/2015/page/3/) / Is language evolution grinding to a halt? The scaling of lexical turbulence in English fiction suggests it is not

# Is language evolution grinding to a halt? The scaling of lexical turbulence in English fiction suggests it is not

**Authors:** Eitan Adam Pechenick, [Christopher M. Danforth](https://md.research.elamur.ai/free/en/authors/christopher-m-danforth/) and [Peter Sheridan Dodds](https://md.research.elamur.ai/free/en/authors/peter-sheridan-dodds/) · **Year:** 2015 · **Venue:** arXiv:1503.03512 · **Publisher:** [Elsevier BV](https://md.research.elamur.ai/free/en/publishers/elsevier-bv/) · **Licence:** [arXiv — non-exclusive license to distribute](http://arxiv.org/licenses/nonexclusive-distrib/1.0/)

**Links:** [Publisher's page](https://arxiv.org/abs/1503.03512) · [PDF](https://arxiv.org/pdf/1503.03512) · [DOI 10.48550/arxiv.1503.03512](https://doi.org/10.48550/arxiv.1503.03512) · [arXiv 1503.03512](https://arxiv.org/abs/1503.03512) · [OpenAlex](https://openalex.org/W2606133927)

**Citations:** 20 citations (OpenAlex, 2026-10-14) · [2](https://md.research.elamur.ai/free/en/papers/pechenick-2015-language-evolution-grinding-halt-scaling/cited-by/) in our library, of the 141,840 papers whose bibliographies we read

## Summary
The paper asks whether the English lexicon is still renewing itself. Earlier work by Petersen et al. reported that word births fell sharply and word deaths rose in the late 20th century. The authors re-examine that claim on the Google Books 2012 English Fiction corpus for 1800–2000. They chose this subset because their earlier study found the other Google Books sets heavily skewed toward scientific text.

They rebuilt the median-based birth/death criterion and moved the end year of the analysis window. This showed that the apparent rise in deaths comes mostly from where the arbitrary "present" is placed. Birth rates do fall, but true word death turns out to be very rare. As a sturdier alternative, they count how many words cross fixed relative-frequency thresholds between consecutive decades. They also use Jensen-Shannon divergence word shifts to show which words drive the change.

The overall Zipf distribution stays stable, yet beneath it there is constant "lexical turbulence." The lexicon keeps turning over, so language evolution is not stopping.

## Key points
- Data: Google Books 2012 English Fiction 1-grams, 1800–2000. Each book is counted once, so the counts say nothing about how popular a book was or how often it was read. Corpus volume grows roughly exponentially, with dips around the American Civil War and both World Wars.
- Petersen et al.'s rule treats a word as born or dead when its frequency crosses 0.05× its own median frequency. The authors identify three weaknesses:
  - the threshold depends on the word and on the chosen year range;
  - rare words have a median of zero;
  - Google keeps only words seen at least 200 times, so a zero count does not mean the word was absent.
- When they rerun the analysis with end years from 1860 to 2000 in 20-year steps, birth rates decline roughly exponentially. Death rates, however, mainly reflect the cutoff year.
- Word flux across frequency thresholds grows superlinearly with word rank. Its scaling break lines up with the break in Zipf's law.
- JSD word shifts reveal both content and corpus problems:
  - Example: the 1960s→1970s shift at threshold 10⁻⁵ has a total JSD of 0.0031 bits.
  - Many prominent words point to scholarly criticism mixed into the fiction corpus, such as author names and "vols".
  - The authors recommend separating criticism from fiction in a future revision of the corpus.

## Interesting ideas
- Any event defined relative to the edge of a time window should be rerun with the edge shifted; if the result moves with the cutoff, it is an artifact rather than a finding.
- Instead of making binary birth/death calls, count items crossing fixed frequency bands in both directions, which avoids per-item thresholds that are poorly defined.
- Split a divergence between two distributions into ranked per-item contributions (a word shift), so that a single number becomes a readable explanation.
- Inspect the items that dominate a measured change to discover what a corpus really contains, for example literary criticism hidden inside "fiction."
- Treat frequency rank like a temperature: the most frequent words are nearly frozen, while rarer words change more and more.

## For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

- **I1** (learn): Evaluation could add a sliding-cutoff or shifted-split control, so that apparent gains or losses are checked for being produced by how the data was split.
- **I2** (tokens): Lexicon and lemma coverage could be tracked per frequency band across text sources, showing where unknown-word handling and morphology fallbacks are under the most pressure.
- **I3** (learn): Disagreements among witness parsers, or between silver and gold label distributions, could be broken down into ranked per-relation or per-lemma contributions, so repair rules target the items that matter most.
- **I4** (tree): Before training gparse or the perceptron parser on a treebank or silver corpus, word-shift style audits could flag genre contamination that would bias the model.
- **I5** (tokens): Closed-class and very frequent words can safely get hand-built, stable rules, while effort for robustness and updates goes to the volatile rare tail.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [University of Vermont](https://md.research.elamur.ai/free/en/universities/university-of-vermont/) — University, Burlington, United States · 4 works · 3,733 citations

📚 Shelves (5): [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 9,296 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 9,206 works · [🤝 Social Sciences](https://md.research.elamur.ai/free/en/shelves/social-sciences/) · 744 works · [📚 Literature and Literary Theory](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/literature-and-literary-theory/) · 340 works · [🤝 Cultural Studies](https://md.research.elamur.ai/free/en/shelves/social-sciences/cultural-studies/) · 10 works

🧭 Topics (2): [English](https://md.research.elamur.ai/free/en/topics/english/) · 1,373 works · [⏳ Historical linguistics & language change](https://md.research.elamur.ai/free/en/topics/language-change/) · 159 works

## Cite

Cite as: Eitan Adam Pechenick, Christopher M. Danforth and Peter Sheridan Dodds. 2015. Is language evolution grinding to a halt? The scaling of lexical turbulence in English fiction suggests it is not. arXiv:1503.03512.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@misc{pechenick2015is,
  title = {{Is language evolution grinding to a halt? The scaling of lexical turbulence in English fiction suggests it is not}},
  author = {Eitan Adam Pechenick, Christopher M. Danforth and Peter Sheridan Dodds},
  year = {2015},
  doi = {10.48550/arxiv.1503.03512},
  eprint = {1503.03512},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.1503.03512}
}
```

- [← previous Recovering discourse relations: Varying influence of discourse adverbials](https://md.research.elamur.ai/free/en/papers/rohde-2015-recovering-discourse-relations-varying-influence/)
- [next → Using Word Semantics To Assist English as a Second Language Learners](https://md.research.elamur.ai/free/en/papers/azab-2015-word-semantics-assist-english-as/)
- [↑ back to papers of 2015](https://md.research.elamur.ai/free/en/papers/2015/page/3/)

*Updated 24 October 2026*