---
title: "A Cross-Linguistic Pressure for Uniform Information Density in"
description: "Languages differ in word order but share statistical tendencies, which are often explained by… 72 references, cited by 15 papers here."
authors: ["Thomas Hikaru Clark", "Clara Meister", "Tiago Pimentel", "Michael Hahn", "Ryan Cotterell", "Richard Futrell", "Roger Levy"]
year: 2023
venue: "Transactions of the Association for Computational Linguistics"
source: "https://aclanthology.org/2023.tacl-1.59/"
doi: "10.1162/tacl_a_00589"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/clark-2023-cross-linguistic-pressure-uniform-information/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/clark-2023-cross-linguistic-pressure-uniform-information/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2023](https://md.research.elamur.ai/free/en/papers/2023/page/12/) / A Cross-Linguistic Pressure for Uniform Information Density in Word Order

# A Cross-Linguistic Pressure for Uniform Information Density in Word Order

**Authors:** [Thomas Hikaru Clark](https://md.research.elamur.ai/free/en/authors/thomas-hikaru-clark/), [Clara Meister](https://md.research.elamur.ai/free/en/authors/clara-meister/), [Tiago Pimentel](https://md.research.elamur.ai/free/en/authors/tiago-pimentel/), [Michael Hahn](https://md.research.elamur.ai/free/en/authors/michael-hahn/), [Ryan Cotterell](https://md.research.elamur.ai/free/en/authors/ryan-cotterell/), [Richard Futrell](https://md.research.elamur.ai/free/en/authors/richard-futrell/) and [Roger Levy](https://md.research.elamur.ai/free/en/authors/roger-levy/) · **Year:** 2023 · **Venue:** [Transactions of the Association for Computational Linguistics](https://md.research.elamur.ai/free/en/venues/tacl/) ([Transactions of the Association for Computational Linguistics (TACL) 2023](https://md.research.elamur.ai/free/en/venues/tacl/2023/)) · **Publisher:** [The MIT Press](https://md.research.elamur.ai/free/en/publishers/the-mit-press/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2023.tacl-1.59/) · [PDF](https://aclanthology.org/2023.tacl-1.59.pdf) · [DOI 10.1162/tacl\_a\_00589](https://doi.org/10.1162/tacl_a_00589) · [OpenAlex](https://openalex.org/W4385859687)

**Citations:** 9 citations (OpenAlex, 2026-10-14) · [15](https://md.research.elamur.ai/free/en/papers/clark-2023-cross-linguistic-pressure-uniform-information/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
Languages differ in word order but share statistical tendencies, which are often explained by functional pressures such as dependency length minimization. This paper asks whether uniform information density (UID) also shapes word order. UID is the idea that communication works best when information is spread evenly, without sharp peaks. The authors took dependency-parsed sentences from 10 typologically diverse languages, using UD relations with function words treated as heads. They rearranged each sentence under several counterfactual grammars, building on the procedure of Hahn et al. (2020). Transformer language models trained on the real and counterfactual corpora supplied per-word surprisal, from which three uniformity scores were derived. In SVO languages, attested orders were consistently more even than their reversed versions. Only an implausible variant that sorts words by frequency reliably beat real orders, and it does so at a large cost in expressivity. Verb-before-object grammars were generally smoother than object-before-verb ones. The findings point to a UID pressure acting through both grammar and speakers' choices, and they offer a possible account of why SVO is so widespread.

## Key points
- **UID measures:** Uniformity is scored three ways: within-sentence surprisal variance (UID_v), average squared word-to-word surprisal change (UID_lv), and a power mean with k > 1 (UID_p). Lower values mean greater uniformity.
- **Counterfactual grammars:** The tested variants are REVERSE, EFFICIENT-VO and EFFICIENT-OV, MIN-DL-LOC and MIN-DL-OPT, RANDOM1–RANDOM5, and SORT-FREQ.
- **VO vs. OV:** EFFICIENT-VO had lower surprisal variance than EFFICIENT-OV in 9 of 10 languages.
- **SOV languages:** The preference for REAL over REVERSE did not hold consistently for Turkish, Hindi and Farsi. The authors attribute this to basic word order and speaker choices pulling in opposite directions.
- **Robustness checks:**
  - The main corpus is Wiki40b, and a subset of languages was rerun on CC100.
  - Analyses restricted to document-initial sentences largely kept the REAL \< REVERSE variance pattern.
  - Code is available at github.com/thomashikaru/word-order-uid.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [University of Cambridge](https://md.research.elamur.ai/free/en/universities/university-of-cambridge/) — University, Cambridge, United Kingdom · 256 works · 123,373 citations
- [Massachusetts Institute of Technology](https://md.research.elamur.ai/free/en/universities/massachusetts-institute-of-technology/) — University, Cambridge, United States · 243 works · 76,506 citations
- [Saarland University](https://md.research.elamur.ai/free/en/universities/saarland-university/) — University, Saarbrücken, Germany · 149 works · 18,353 citations
- [ETH Zurich](https://md.research.elamur.ai/free/en/universities/eth-zurich/) — University, Zurich, Switzerland · 84 works · 9,181 citations
- [University of California, Irvine](https://md.research.elamur.ai/free/en/universities/university-of-california-irvine/) — University, Irvine, United States · 43 works · 36,565 citations

📚 Shelves (6): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 18,614 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 17,965 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 17,450 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 17,296 works · [💭 Psychology](https://md.research.elamur.ai/free/en/shelves/psychology/) · 1,649 works · [💭 Developmental and Educational Psychology](https://md.research.elamur.ai/free/en/shelves/psychology/developmental-and-educational-psychology/) · 523 works

## Cite

Cite as: Thomas Hikaru Clark, Clara Meister, Tiago Pimentel, Michael Hahn, Ryan Cotterell, Richard Futrell and Roger Levy. 2023. A Cross-Linguistic Pressure for Uniform Information Density in Word Order. Transactions of the Association for Computational Linguistics.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@article{clark2023cross,
  title = {{A Cross-Linguistic Pressure for Uniform Information Density in Word Order}},
  author = {Thomas Hikaru Clark, Clara Meister, Tiago Pimentel, Michael Hahn, Ryan Cotterell, Richard Futrell and Roger Levy},
  journal = {Transactions of the Association for Computational Linguistics},
  year = {2023},
  doi = {10.1162/tacl\_a\_00589},
  url = {https://doi.org/10.1162/tacl_a_00589}
}
```

- [← previous Two Failures of Self-Consistency in the Multi-Step Reasoning of LLMs](https://md.research.elamur.ai/free/en/papers/chen-2023-two-failures-self-consistency-multi/)
- [next → Arabic Dialect Identification under Scrutiny: Limitations of Single-label Classification](https://md.research.elamur.ai/free/en/papers/keleg-2023-arabic-dialect-identification-under-scrutiny/)
- [↑ back to papers of 2023](https://md.research.elamur.ai/free/en/papers/2023/page/12/)

## References

72 references, 13 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Matthew Aylett and Alice Turk. 2004. The smooth signal redundancy hypothesis: A functional explanation for relationships between redundancy, prosodic prominence, and duration in spontaneous speech. Language and Speech, 47(1):31–56. https://doi.org/10 .1177/00238309040470010201, PubMed: 15298329 [find](https://scholar.google.com/scholar?q=Matthew+Aylett+and+Alice+Turk%2E+2004%2E+The+smooth+signal+redundancy+hypothesis%3A+A+functional+explanation+for+relationships+between+redundancy%2C+prosodic+prominence%2C+and+duration+in+spontaneous+speech%2E+La)
2. Brian Bartek, Richard L. Lewis, Shravan Vasishth, and Mason R. Smith. 2011. In search of online locality effects in sentence comprehension. Journal of Experimental Psychology: Learning, Memory, and Cognition, 37(5):1178–1198. https://doi.org/10.1037/a0024194, PubMed: 21707210 — [doi:10.1037/a0024194](https://doi.org/10.1037/a0024194) [↗](https://doi.org/10.1037/a0024194)
3. Jelke Bloem. 2016. Testing the processing hypothesis of word order variation using a probabilistic language model. In Proceedings of the Workshop on Computational Linguistics for Linguistic Complexity, pages 174–185, Osaka, Japan. [find](https://scholar.google.com/scholar?q=Jelke+Bloem%2E+2016%2E+Testing+the+processing+hypothesis+of+word+order+variation+using+a+probabilistic+language+model%2E+In+Proceedings+of+the+Workshop+on+Computational+Linguistics+for+Linguistic+Complexity)
4. The COLING 2016 Organizing Committee. [find](https://scholar.google.com/scholar?q=The+COLING+2016+Organizing+Committee%2E)
5. Thomas Hikaru Clark, Ethan Gotlieb Wilcox, Edward Gibson, and Roger P. Levy. 2022. Evidence for availability effects on speaker choice in the Russian comparative alternation. In Proceedings of the 44th Annual Meeting of the Cognitive Science Society. [find](https://scholar.google.com/scholar?q=Thomas+Hikaru+Clark%2C+Ethan+Gotlieb+Wilcox%2C+Edward+Gibson%2C+and+Roger+P%2E+Levy%2E+2022%2E+Evidence+for+availability+effects+on+speaker+choice+in+the+Russian+comparative+alternation%2E+In+Proceedings+of+the+44t)
6. Michael Xavier Collins. 2014. Information density and dependency length as complementary cognitive models. Journal of Psycholinguistic Research, 43(5):651–681. https://doi.org/10.1007/s10936-013-9273-3, PubMed: 24077911 — [doi:10.1007/s10936-013-9273-3](https://doi.org/10.1007/s10936-013-9273-3) [↗](https://doi.org/10.1007/s10936-013-9273-3)
7. Alexis Conneau, Kartikay Khandelwal, Naman Goyal, Vishrav Chaudhary, Guillaume Wenzek, Francisco Guzmán, Edouard Grave, Myle Ott, Luke Zettlemoyer, and Veselin Stoyanov. 2020. [Unsupervised cross-lingual representation learning at scale](https://md.research.elamur.ai/free/en/papers/conneau-2020-unsupervised-cross-lingual-representation-learning/). In Proceedings of the 58th Annual Meeting of the Association f…
8. Matthew S. Dryer. 2013. Order of subject, object and verb (v2020.3). In Matthew S. Dryer and Martin Haspelmath, editors, The World Atlas of Language Structures Online. [find](https://scholar.google.com/scholar?q=Matthew+S%2E+Dryer%2E+2013%2E+Order+of+subject%2C+object+and+verb+%28v2020%2E3%29%2E+In+Matthew+S%2E+Dryer+and+Martin+Haspelmath%2C+editors%2C+The+World+Atlas+of+Language+Structures+Online%2E)
9. Li Du, Lucas Torroba Hennigen, Tiago Pimentel, Clara Meister, Jason Eisner, and Ryan Cotterell. 2022. A measure-theoretic characterization of tight language models. arXiv preprint arXiv: 2212.10502v1. https://doi.org/10.48550/arXiv.2212.10502 [↗](https://doi.org/10.48550/arXiv.2212.10502)
10. Nick Ellis. 2002. Frequency effects in language processing. Studies in Second Language Acquisition, 24(2):143–188. https://doi.org/10.1017/S0272263102002024 [↗](https://doi.org/10.1017/S0272263102002024)
11. August Fenk and Gertrud Fenk. 1980. Konstanz im Kurzzeitgedächtnis—Konstanz im sprachlichen Informationsfluß. Zeitschrift für ex-perimentelle und angewandte Psychologie, 27(3):400–414. [find](https://scholar.google.com/scholar?q=August+Fenk+and+Gertrud+Fenk%2E+1980%2E+Konstanz+im+Kurzzeitgeda%CC%88chtnis%E2%80%94Konstanz+im+sprachlichen+Informationsflu%C3%9F%2E+Zeitschrift+fu%CC%88r+ex%2Dperimentelle+und+angewandte+Psychologie%2C+27%283%29%3A400%E2%80%93414%2E)
12. Ramon Ferrer-i-Cancho. 2004. Euclidean distance between syntactically linked words. Physical Review E, 70(5 Pt 2):056135. https://doi.org/10.1103/PhysRevE.70.056135, PubMed: 15600720 — [doi:10.1103/physreve.70.056135](https://doi.org/10.1103/physreve.70.056135) [↗](https://doi.org/10.1103/PhysRevE.70.056135)
13. Ramon Ferrer-i-Cancho, Carlos Gómez-Rodrı́guez, and Juan Luis Esteban. 2018. Are crossing dependencies really scarce? Physica A: Statistical Mechanics and its Applications, 493:311–329. https://doi.org/10.1016/j.physa.2017.10.048 [↗](https://doi.org/10.1016/j.physa.2017.10.048)
14. A. Frank and T. F. Jaeger. 2008. Speaking rationally: Uniform information density as an optimal strategy for language production. In Proceedings of the Cognitive Science Society. [find](https://scholar.google.com/scholar?q=A%2E+Frank+and+T%2E+F%2E+Jaeger%2E+2008%2E+Speaking+rationally%3A+Uniform+information+density+as+an+optimal+strategy+for+language+production%2E+In+Proceedings+of+the+Cognitive+Science+Society%2E)
15. Richard Futrell, Tina Hickey, Aldrin Lee, Eunice Lim, Elena Luchkina, and Edward Gibson. 2015a. Cross-linguistic gestures reflect typological universals: A subject-initial, verb-final bias in speakers of diverse languages. Cognition, 136:215–221. https://doi.org/10.1016/j .cognition.2014.11.022, Pub… — [doi:10.1016/j](https://doi.org/10.1016/j) [find](https://scholar.google.com/scholar?q=Richard+Futrell%2C+Tina+Hickey%2C+Aldrin+Lee%2C+Eunice+Lim%2C+Elena+Luchkina%2C+and+Edward+Gibson%2E+2015a%2E+Cross%2Dlinguistic+gestures+reflect+typological+universals%3A+A+subject%2Dinitial%2C+verb%2Dfinal+bias+in+speakers)
16. Richard Futrell and Roger Levy. 2017. Noisy-context surprisal as a human sentence processing cost model. In Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 1, Long Papers, pages 688–698, Valencia, Spain. Association for Computationa… [find](https://scholar.google.com/scholar?q=Richard+Futrell+and+Roger+Levy%2E+2017%2E+Noisy%2Dcontext+surprisal+as+a+human+sentence+processing+cost+model%2E+In+Proceedings+of+the+15th+Conference+of+the+European+Chapter+of+the+Association+for+Computatio)
17. Richard Futrell, Roger P. Levy, and Edward Gibson. 2020. Dependency locality as an explanatory principle for word order. Language, 96(2):371–413. https://doi.org/10.1353/lan.2020.0024 [↗](https://doi.org/10.1353/lan.2020.0024)
18. Richard Futrell, Kyle Mahowald, and Edward Gibson. 2015b. Large-scale evidence of dependency length minimization in 37 languages. Proceedings of the National Academy of Sciences, U.S.A., 112(33):10336–10341. https://doi.org/10.1073/pnas.1502134112, PubMed: 26240370 — [doi:10.1073/pnas.1502134112](https://doi.org/10.1073/pnas.1502134112) [↗](https://doi.org/10.1073/pnas.1502134112)
19. Richard Futrell, Peng Qian, Edward Gibson, Evelina Fedorenko, and Idan Blank. 2019. Syntactic dependencies correspond to word pairs with high mutual information. In Proceedings of the Fifth International Conference on Dependency Linguistics (Depling, SyntaxFest 2019), pages 3–13, Paris, France. Asso… [find](https://scholar.google.com/scholar?q=Richard+Futrell%2C+Peng+Qian%2C+Edward+Gibson%2C+Evelina+Fedorenko%2C+and+Idan+Blank%2E+2019%2E+Syntactic+dependencies+correspond+to+word+pairs+with+high+mutual+information%2E+In+Proceedings+of+the+Fifth+Internatio)
20. Georg von der Gabelentz. 1901. Die Sprachwissenschaft, ihre Aufgaben, Methoden, und bisherigen Ergebnisse, 2nd edition. C. H. Tauchnitz, Leipzig. [find](https://scholar.google.com/scholar?q=Georg+von+der+Gabelentz%2E+1901%2E+Die+Sprachwissenschaft%2C+ihre+Aufgaben%2C+Methoden%2C+und+bisherigen+Ergebnisse%2C+2nd+edition%2E+C%2E+H%2E+Tauchnitz%2C+Leipzig%2E)
21. Dmitriy Genzel and Eugene Charniak. 2002. [Entropy rate constancy in text](https://md.research.elamur.ai/free/en/papers/genzel-2002-entropy-rate-constancy-text/). In Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, pages 199–206, Philadelphia, Pennsylvania, USA. Association for Computational Linguistics. https://doi.org/10.3115/1073083.1073117 [↗](https://doi.org/10.3115/1073083.1073117)
22. Kim Gerdes, Bruno Guillaume, Sylvain Kahane, and Guy Perrier. 2018. [SUD or surface-syntactic universal dependencies: An annotation scheme near-isomorphic to UD](https://md.research.elamur.ai/free/en/papers/gerdes-2018-sud-or-surface-syntactic-universal/). In Proceedings of the Second Workshop on Universal Dependencies, UDW@EMNLP 2018, Brussels, Belgium, November 1, 2018, pages 66–74, Philadel…
23. Edward Gibson. 1998. Linguistic complexity: Locality of syntactic dependencies. Cognition, 68(1):1–76. https://doi.org/10.1016 /S0010-0277(98)00034-1, PubMed: 9775516 [find](https://scholar.google.com/scholar?q=Edward+Gibson%2E+1998%2E+Linguistic+complexity%3A+Locality+of+syntactic+dependencies%2E+Cognition%2C+68%281%29%3A1%E2%80%9376%2E+https%3A%2F%2Fdoi%2Eorg%2F10%2E1016+%2FS0010%2D0277%2898%2900034%2D1%2C+PubMed%3A+9775516)
24. Edward Gibson. 2000. The dependency locality theory: A distance-based theory of linguistic complexity. In Image, Language, Brain: Papers from the First Mind Articulation Project Symposium, pages 95–126, Cambridge, MA. MIT Press. [find](https://scholar.google.com/scholar?q=Edward+Gibson%2E+2000%2E+The+dependency+locality+theory%3A+A+distance%2Dbased+theory+of+linguistic+complexity%2E+In+Image%2C+Language%2C+Brain%3A+Papers+from+the+First+Mind+Articulation+Project+Symposium%2C+pages+95%E2%80%9312)
25. Edward Gibson, Richard Futrell, Steven P. Piantadosi, Isabelle Dautriche, Kyle Mahowald, Leon Bergen, and Roger Levy. 2019. How efficiency shapes human language. Trends in Cognitive Sciences, 23(5):389–407. https://doi.org/10.1016/j.tics.2019.02.003, PubMed: 31006626 — [doi:10.1016/j.tics.2019.02.003](https://doi.org/10.1016/j.tics.2019.02.003) [↗](https://doi.org/10.1016/j.tics.2019.02.003)
26. Edward Gibson, Steven T. Piantadosi, Kimberly Brink, Leon Bergen, Eunice Lim, and Rebecca Saxe. 2013. A noisy-channel account of cross-linguistic word-order variation. Psychological Science, 24(7):1079–1088. https://doi.org/10.1177/0956797612463705, PubMed: 23649563 — [doi:10.1177/0956797612463705](https://doi.org/10.1177/0956797612463705) [↗](https://doi.org/10.1177/0956797612463705)
27. Daniel Gildea and T. Florian Jaeger. 2015. Human languages order information efficiently. arXiv preprint arXiv:1510.02823. https://doi.org/10.48550/arXiv.1510.02823 — [doi:10.48550/arxiv.1510.02823](https://doi.org/10.48550/arxiv.1510.02823) [↗](https://doi.org/10.48550/arXiv.1510.02823)
28. Daniel Gildea and David Temperley. 2007. Optimizing grammars for minimum dependency length. In Proceedings of the 45th Annual Meeting of the Association of Computational Linguistics, pages 184–191, Prague, Czech Republic. Association for Computational Linguistics. different languages represent event… [find](https://scholar.google.com/scholar?q=Daniel+Gildea+and+David+Temperley%2E+2007%2E+Optimizing+grammars+for+minimum+dependency+length%2E+In+Proceedings+of+the+45th+Annual+Meeting+of+the+Association+of+Computational+Linguistics%2C+pages+184%E2%80%93191%2C+Pr)
29. Joseph H. Greenberg. 1963. Some Universals of Grammar with Particular Reference to the Order of Meaningful Elements. MIT Press, Cambridge, MA. [find](https://scholar.google.com/scholar?q=Joseph+H%2E+Greenberg%2E+1963%2E+Some+Universals+of+Grammar+with+Particular+Reference+to+the+Order+of+Meaningful+Elements%2E+MIT+Press%2C+Cambridge%2C+MA%2E)
30. Daniel Grodner and Edward Gibson. 2005. Consequences of the serial nature of linguistic input for sentential complexity. Cognitive Science, 29(2):261–290. https://doi.org/10.1207/s15516709cog0000 7, PubMed: 21702774 [↗](https://doi.org/10.1207/s15516709cog0000)
31. Mandy Guo, Zihang Dai, Denny Vrandecic, and Rami Al-Rfou. 2020. Wiki-40b: Multilingual language model dataset. In Proceedings of the 12th Language Resources and Evaluation Conference, pages 2440–2452, Marseille, France. European Language Resources Association. [find](https://scholar.google.com/scholar?q=Mandy+Guo%2C+Zihang+Dai%2C+Denny+Vrandecic%2C+and+Rami+Al%2DRfou%2E+2020%2E+Wiki%2D40b%3A+Multilingual+language+model+dataset%2E+In+Proceedings+of+the+12th+Language+Resources+and+Evaluation+Conference%2C+pages+2440%E2%80%932452%2C)
32. Michael Hahn, Dan Jurafsky, and Richard Futrell. 2020. Universals of word order reflect optimization of grammars for efficient communication. Proceedings of the National Academy of Sciences, U.S.A., 117(5):2347–2353. https://doi.org/10.1073/pnas.1910923117, PubMed: 31964811 — [doi:10.1073/pnas.1910923117](https://doi.org/10.1073/pnas.1910923117) [↗](https://doi.org/10.1073/pnas.1910923117)
33. Michael Hahn and Yang Xu. 2022. Crosslinguistic word order variation reflects evolutionary pressures of dependency and information locality. Proceedings of the National Academy of Sciences, U.S.A., 119(24):e2122604119. https://doi.org/10.1073/pnas.2122604119, PubMed: 35675428 — [doi:10.1073/pnas.2122604119](https://doi.org/10.1073/pnas.2122604119) [↗](https://doi.org/10.1073/pnas.2122604119)
34. John Hale. 2001. [A probabilistic Earley parser as a psycholinguistic model](https://md.research.elamur.ai/free/en/papers/hale-2001-probabilistic-earley-parser-as-psycholinguistic/). In Second Meeting of the North American Chapter of the Association for Computational Linguistics, pages 1–8. https://doi.org/10.3115/1073336.1073357 [↗](https://doi.org/10.3115/1073336.1073357)
35. Daniel Gildea and David Temperley. 2010. Do grammars minimize dependency length? Cognitive Science, 34(2):286–310. https://doi.org/10.1111/j.1551-6709.2009 .01073.x, PubMed: 21564213 — [doi:10.1111/j.1551-6709.2009](https://doi.org/10.1111/j.1551-6709.2009) [↗](https://doi.org/10.1111/j.1551-6709.2009)
36. Martin Haspelmath. 2008. Parametric versus functional explanations of syntactic universals. In T. Biberauer, editor, The Limits of Syntactic Variation, pages 75–107. John Benjamins, Amsterdam. https://doi.org/10.1075/la.132.04has Susan Goldin-Meadow, Wing Chee So, Asli Özyürek, and Carolyn Myland… [↗](https://doi.org/10.1075/la.132.04has)
37. John A. Hawkins. 1994. A Performance Theory of Order and Constituency. Cambridge University Press, Cambridge. [find](https://scholar.google.com/scholar?q=John+A%2E+Hawkins%2E+1994%2E+A+Performance+Theory+of+Order+and+Constituency%2E+Cambridge+University+Press%2C+Cambridge%2E)
38. John A. Hawkins. 2004. Efficiency and Complexity in Grammars. Oxford University Press, Oxford. https://doi.org/10.1093/acprof :oso/9780199252695.001.0001 — [doi:10.1093/acprof](https://doi.org/10.1093/acprof) [↗](https://doi.org/10.1093/acprof)
39. John A. Hawkins. 2014. Cross-linguistic Variation and Efficiency. Oxford University Press, Oxford. https://doi.org/10.1093/acprof :oso/9780199664993.001.0001 — [doi:10.1093/acprof](https://doi.org/10.1093/acprof) [↗](https://doi.org/10.1093/acprof)
40. Jacob L. Hoover, Morgan Sonderegger, Steven T. Piantadosi, and Timothy J. O’Donnell. 2022. The plausibility of sampling as an algorithmic theory of sentence processing. PsyArXiv preprint PsyArXiv:qjnpv. https://doi.org/10.31234/osf.io/qjnpv T. Florian Jaeger. 2010. Redundancy and reduction: Speaker… — [doi:10.31234/osf.io/qjnpv](https://doi.org/10.31234/osf.io/qjnpv) [↗](https://doi.org/10.31234/osf.io/qjnpv)
41. T. Florian Jaeger and Harry Tily. 2011. On language ‘utility’: Processing complexity and communicative efficiency. Wiley Interdisciplinary Reviews: Cognitive Science, 2(3):323–335. https://doi.org/10.1002/wcs.126, PubMed: 26302080 — [doi:10.1002/wcs.126](https://doi.org/10.1002/wcs.126) [↗](https://doi.org/10.1002/wcs.126)
42. Diederik P. Kingma and Jimmy Ba. 2017. [Adam: A method for stochastic optimization](https://md.research.elamur.ai/free/en/papers/kingma-2014-adam-method-stochastic-optimization/). arXiv preprint arXiv:1412.6980. https://doi.org/10.48550/arXiv.1412.6980 [↗](https://doi.org/10.48550/arXiv.1412.6980)
43. Moshe Koppel and Noam Ordan. 2011. [Translationese and its dialects](https://md.research.elamur.ai/free/en/papers/koppel-2011-translationese-its-dialects/). In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, pages 1318–1326, Portland, Oregon, USA. Association for Computational Linguistics.
44. Marco Kuhlmann. 2010. Projective Dependency Structures, Lecture Notes in Computer Science, Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-14568-1 3 — [doi:10.1007/978-3-642-14568-1](https://doi.org/10.1007/978-3-642-14568-1) [↗](https://doi.org/10.1007/978-3-642-14568-1)
45. Roger Levy. 2008. Expectation-based syntactic comprehension. Cognition, 106(3):1126–1177. https://doi.org/10.1016/j.cognition .2007.05.006, PubMed: 17662975 — [doi:10.1016/j.cognition](https://doi.org/10.1016/j.cognition) [↗](https://doi.org/10.1016/j.cognition)
46. Roger Levy. 2018. Communicative efficiency, uniform information density, and the rational speech act theory. In Proceedings for the 40th Annual Meeting of the Cognitive Science Society, pages 684–689. https://doi.org/10 .31234/osf.io/4cgxh Roger Levy and T. Florian Jaeger. 2006. Speakers optimize in… [find](https://scholar.google.com/scholar?q=Roger+Levy%2E+2018%2E+Communicative+efficiency%2C+uniform+information+density%2C+and+the+rational+speech+act+theory%2E+In+Proceedings+for+the+40th+Annual+Meeting+of+the+Cognitive+Science+Society%2C+pages+684%E2%80%93689%2E)
47. Roger Levy and T. Florian Jaeger. 2007. Speakers optimize information density through syntactic reduction. Advances in Neural Information Processing Systems, 19:849–856. [find](https://scholar.google.com/scholar?q=Roger+Levy+and+T%2E+Florian+Jaeger%2E+2007%2E+Speakers+optimize+information+density+through+syntactic+reduction%2E+Advances+in+Neural+Information+Processing+Systems%2C+19%3A849%E2%80%93856%2E)
48. Haitao Liu. 2008. Dependency distance as a metric of language comprehension difficulty. Journal of Cognitive Science, 9(2):159–191. https://doi.org/10.17791/jcs.2008 .9.2.159 — [doi:10.17791/jcs.2008](https://doi.org/10.17791/jcs.2008) [↗](https://doi.org/10.17791/jcs.2008)
49. Steven G. Luke and Kiel Christianson. 2016. Limits on lexical prediction during reading. Cognitive Psychology, 88:22–60. https://doi.org/10.1016/j.cogpsych.2016 .06.002 — [doi:10.1016/j.cogpsych.2016](https://doi.org/10.1016/j.cogpsych.2016) [↗](https://doi.org/10.1016/j.cogpsych.2016)
50. Kyle Mahowald, Evgeniia Diachek, Edward Gibson, Evelina Fedorenko, and Richard Futrell. 2022. Experimentally measuring the redundancy of grammatical cues in transitive clauses. arXiv preprint arXiv:2201.12911. https://doi.org/10.48550/arXiv.2201 .12911 — [doi:10.48550/arxiv.2201](https://doi.org/10.48550/arxiv.2201) [↗](https://doi.org/10.48550/arXiv.2201)
51. Marie-Catherine de Marneffe, Christopher D. Manning, Joakim Nivre, and Daniel Zeman. 2021. [Universal Dependencies](https://md.research.elamur.ai/free/en/papers/marneffe-2021-universal-dependencies-2/). Computational Linguistics, 47(2):255–308. https://doi.org/10.1162/coli_a_00402 [↗](https://doi.org/10.1162/coli_a_00402)
52. Luke Maurits, Dan Navarro, and Amy Perfors. 2010. Why are some word orders more common than others? A Uniform Information Density account. In Advances in Neural Information Processing Systems, pages 1585–1593. [find](https://scholar.google.com/scholar?q=Luke+Maurits%2C+Dan+Navarro%2C+and+Amy+Perfors%2E+2010%2E+Why+are+some+word+orders+more+common+than+others%3F+A+Uniform+Information+Density+account%2E+In+Advances+in+Neural+Information+Processing+Systems%2C+pages+1)
53. Clara Meister, Tiago Pimentel, Patrick Haller, Lena Jäger, Ryan Cotterell, and Roger Levy. 2021. [Revisiting the Uniform Information Density hypothesis](https://md.research.elamur.ai/free/en/papers/meister-2021-revisiting-uniform-information-density-hypothesis/). In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 963–980, Online and Punta Cana, Dominican Republic…
54. Association for Computational Linguistics. https://doi.org/10.18653/v1/2021 .emnlp-main.74 — [doi:10.18653/v1/2021](https://doi.org/10.18653/v1/2021) [↗](https://doi.org/10.18653/v1/2021)
55. Myle Ott, Sergey Edunov, Alexei Baevski, Angela Fan, Sam Gross, Nathan Ng, David Grangier, and Michael Auli. 2019. [fairseq: A fast, extensible toolkit for sequence modeling](https://md.research.elamur.ai/free/en/papers/ott-2019-fairseq-fast-extensible-toolkit-sequence/). In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics (Demonstr…
56. Steven T. Piantadosi, Harry Tily, and Edward Gibson. 2011. Word lengths are optimized for efficient communication. Proceedings of the National Academy of Sciences, U.S.A., 108(9):3526–3529. https://doi.org/10.1073/pnas.1012551108, PubMed: 21278332 [↗](https://doi.org/10.1073/pnas.1012551108)
57. Tiago Pimentel, Ryan Cotterell, and Brian Roark. 2021a. Disambiguatory signals are stronger in word-initial positions. In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume, pages 31–41, Online. Association for Computational Lingu… [find](https://scholar.google.com/scholar?q=Tiago+Pimentel%2C+Ryan+Cotterell%2C+and+Brian+Roark%2E+2021a%2E+Disambiguatory+signals+are+stronger+in+word%2Dinitial+positions%2E+In+Proceedings+of+the+16th+Conference+of+the+European+Chapter+of+the+Association+)
58. Tiago Pimentel, Clara Meister, Elizabeth Salesky, Simone Teufel, Damián Blasi, and Ryan Cotterell. 2021b. A surprisal–duration trade-off across and within the world’s languages. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 949–962, Online and Punt… [find](https://scholar.google.com/scholar?q=Tiago+Pimentel%2C+Clara+Meister%2C+Elizabeth+Salesky%2C+Simone+Teufel%2C+Damia%CC%81n+Blasi%2C+and+Ryan+Cotterell%2E+2021b%2E+A+surprisal%E2%80%93duration+trade%2Doff+across+and+within+the+world%E2%80%99s+languages%2E+In+Proceedings+of+the)
59. Tiago Pimentel, Irene Nikkarinen, Kyle Mahowald, Ryan Cotterell, and Damián Blasi. 2021c. How (non-)optimal is the lexicon? In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 4426–4438, Online. Ass… [find](https://scholar.google.com/scholar?q=Tiago+Pimentel%2C+Irene+Nikkarinen%2C+Kyle+Mahowald%2C+Ryan+Cotterell%2C+and+Damia%CC%81n+Blasi%2E+2021c%2E+How+%28non%2D%29optimal+is+the+lexicon%3F+In+Proceedings+of+the+2021+Conference+of+the+North+American+Chapter+of+the+)
60. Shauli Ravfogel, Yoav Goldberg, and Tal Linzen. 2019. [Studying the inductive biases of RNNs with synthetic variations of natural languages](https://md.research.elamur.ai/free/en/papers/ravfogel-2019-studying-inductive-biases-rnns-synthetic/). In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Lon…
61. Jan Rijkhoff. 1986. Word order universals revisited: The principle of head proximity. Belgian Journal of Linguistics, 1:95–125. https://doi.org/10.1075/bjl.1.05rij — [doi:10.1075/bjl.1.05rij](https://doi.org/10.1075/bjl.1.05rij) [↗](https://doi.org/10.1075/bjl.1.05rij)
62. Jan Rijkhoff. 1990. Explaining word order in the noun phrase. Linguistics, 28(1):5–42. https://doi.org/10.1515/ling.1990.28.1.5 — [doi:10.1515/ling.1990.28.1.5](https://doi.org/10.1515/ling.1990.28.1.5) [↗](https://doi.org/10.1515/ling.1990.28.1.5)
63. Rico Sennrich, Barry Haddow, and Alexandra Birch. 2016. [Neural machine translation of rare words with subword units](https://md.research.elamur.ai/free/en/papers/sennrich-2016-neural-machine-translation-rare-words/). In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1715–1725, Berlin, Germany. Association for Computational Lin…
64. Cory Shain, Clara Meister, Tiago Pimentel, Ryan Cotterell, and Roger P. Levy. 2022. Large-scale evidence for logarithmic effects of word predictability on reading time. PsyArXiv preprint PsyArXiv:4hyna. https://doi.org/10.31234/osf.io/4hyna Claude E. Shannon. 1948. A mathematical theory of communic… — [doi:10.31234/osf.io/4hyna](https://doi.org/10.31234/osf.io/4hyna) [↗](https://doi.org/10.31234/osf.io/4hyna)
65. Milan Straka and Jana Straková. 2017. [Tokenizing, POS tagging, lemmatizing and parsing UD 2.0 with UDPipe](https://md.research.elamur.ai/free/en/papers/straka-2017-tokenizing-pos-tagging-lemmatizing-parsing/). In Proceedings of the CoNLL 2017 Shared Task: Multilingual Parsing from Raw Text to Universal Dependencies, pages 88–99, Vancouver, Canada. Association for Computational Linguistics. https://d…
66. David Temperley and Daniel Gildea. 2018. Minimizing syntactic dependency lengths: Typological/cognitive universal? Annual Review of Linguistics, 4:1–15. https://doi.org/10 .1146/annurev-linguistics-011817 [find](https://scholar.google.com/scholar?q=David+Temperley+and+Daniel+Gildea%2E+2018%2E+Minimizing+syntactic+dependency+lengths%3A+Typological%2Fcognitive+universal%3F+Annual+Review+of+Linguistics%2C+4%3A1%E2%80%9315%2E+https%3A%2F%2Fdoi%2Eorg%2F10+%2E1146%2Fannurev%2Dlinguistics%2D01)
67. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. [Attention is all you need](https://md.research.elamur.ai/free/en/papers/vaswani-2017-attention-all-need/). In Advances in Neural Information Processing Systems, volume 30. Curran Associates, Inc.
68. Jennifer C. White and Ryan Cotterell. 2021. Examining the inductive bias of neural language models with artificial languages. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume… [find](https://scholar.google.com/scholar?q=Jennifer+C%2E+White+and+Ryan+Cotterell%2E+2021%2E+Examining+the+inductive+bias+of+neural+language+models+with+artificial+languages%2E+In+Proceedings+of+the+59th+Annual+Meeting+of+the+Association+for+Computati)
69. Himanshu Yadav, Samar Husain, and Richard Futrell. 2021. Do dependency lengths explain constraints on crossing dependencies? Linguistics Vanguard, 7(s3). https://doi.org/10 .1515/lingvan-2019-0070 [find](https://scholar.google.com/scholar?q=Himanshu+Yadav%2C+Samar+Husain%2C+and+Richard+Futrell%2E+2021%2E+Do+dependency+lengths+explain+constraints+on+crossing+dependencies%3F+Linguistics+Vanguard%2C+7%28s3%29%2E+https%3A%2F%2Fdoi%2Eorg%2F10+%2E1515%2Flingvan%2D2019%2D0070)
70. Meilin Zhan and Roger Levy. 2018. Comparing theories of speaker choice using a model of classifier production in Mandarin Chinese. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers)… [find](https://scholar.google.com/scholar?q=Meilin+Zhan+and+Roger+Levy%2E+2018%2E+Comparing+theories+of+speaker+choice+using+a+model+of+classifier+production+in+Mandarin+Chinese%2E+In+Proceedings+of+the+2018+Conference+of+the+North+American+Chapter+o)
71. George Kingsley Zipf. 1935. The Psycho-biology of Language: An Introduction to Dynamic Philology. Houghton-Mifflin, Boston. [find](https://scholar.google.com/scholar?q=George+Kingsley+Zipf%2E+1935%2E+The+Psycho%2Dbiology+of+Language%3A+An+Introduction+to+Dynamic+Philology%2E+Houghton%2DMifflin%2C+Boston%2E)
72. Ran Zmigrod, Tim Vieira, and Ryan Cotterell. 2020. Please mind the root: Decoding arborescences for dependency parsing. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 4809–4819, Online. Association for Computational Linguistics. [https://doi.o…](https://doi.o…) [↗](https://doi.o…)

*Updated 2 November 2026*