--- title: "Phonotactic Complexity and Its Trade-offs — Pimentel 2020" description: "The authors propose an information-theoretic measure: bits per phoneme. 1 reference, cited by 23 papers here." authors: ["Tiago Pimentel", "Brian Roark", "Ryan Cotterell"] year: 2020 venue: "Transactions of the Association for Computational Linguistics" source: "https://aclanthology.org/2020.tacl-1.1/" doi: "10.1162/tacl_a_00296" licence: "CC BY 4.0 (ACL Anthology)" licence_url: "https://creativecommons.org/licenses/by/4.0/" summary_licence: "Elamur Cosmic License" canonical: "https://research.elamur.ai/free/en/papers/pimentel-2020-phonotactic-complexity-its-trade-offs/" --- [Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/pimentel-2020-phonotactic-complexity-its-trade-offs/) [home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2020](https://md.research.elamur.ai/free/en/papers/2020/page/6/) / Phonotactic Complexity and Its Trade-offs # Phonotactic Complexity and Its Trade-offs **Authors:** [Tiago Pimentel](https://md.research.elamur.ai/free/en/authors/tiago-pimentel/), [Brian Roark](https://md.research.elamur.ai/free/en/authors/brian-roark/) and [Ryan Cotterell](https://md.research.elamur.ai/free/en/authors/ryan-cotterell/) · **Year:** 2020 · **Venue:** [Transactions of the Association for Computational Linguistics](https://md.research.elamur.ai/free/en/venues/tacl/) ([Transactions of the Association for Computational Linguistics (TACL) 2020](https://md.research.elamur.ai/free/en/venues/tacl/2020/)) · **Publisher:** [The MIT Press](https://md.research.elamur.ai/free/en/publishers/the-mit-press/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/) **Links:** [Publisher's page](https://aclanthology.org/2020.tacl-1.1/) · [PDF](https://aclanthology.org/2020.tacl-1.1.pdf) · [DOI 10.1162/tacl\_a\_00296](https://doi.org/10.1162/tacl_a_00296) · [OpenAlex](https://openalex.org/W3001772156) **Citations:** 60 citations (OpenAlex, 2026-10-14) · [23](https://md.research.elamur.ai/free/en/papers/pimentel-2020-phonotactic-complexity-its-trade-offs/cited-by/) in our library, of the 149,439 papers whose bibliographies we read ## Summary The paper asks how to compare phonotactic complexity across languages in a way that is more fine-grained than counting inventory units. Counts of vowels, consonants or syllables cannot see longer-range dependencies such as vowel harmony. The authors propose an information-theoretic measure: bits per phoneme. For each language, they train a model over words written as phoneme strings and score held-out word types by their negative log-probability per segment. Because this cross-entropy is an upper bound on the true entropy, it can be compared across languages. They use concept-aligned lists of 1016 basic words in 106 languages and compare unigram, trigram and LSTM models. The LSTM score correlates strongly and negatively with average word length (−0.74). This trade-off appears both across and within language families, while inventory-size measures show only weak links to word length. The work supports the compensation and equal-capacity debate with a reproducible, quantitative tool. The authors stay neutral on what causes the trade-off. ## Key points - **Main correlation:** The LSTM-based bits per phoneme correlates with mean word length at about −0.74 (Spearman). Phoneme, vowel and consonant counts correlate much more weakly. Inventory sizes correlate weakly, and unigram scores give about −0.22. - **Language-level effect:** A permutation test built 10⁴ shuffled "fake languages". None matched the real correlation, which supports a genuine language-level effect (p \< 10⁻⁴). - **Constraints lower entropy:** Artificially removing phonological constraints made the languages less predictable. - Removing vowel harmony raised LSTM complexity by roughly 0.62 bits per phoneme, about 16% more entropy. - Removing final-obstruent devoicing gave a small but significant rise of about 0.034 bits. - **Within families:** In the five families with at least 4 languages (Dravidian, Indo-European, Nakh-Daghestanian, Turkic, Uralic), the LSTM measure correlates with word length, with ρ between −0.662 and −1.0 (p \< 0.1). Vowel inventory size is significant only within Turkic. - **Feature embeddings:** Phoneme-feature embeddings, shared or independent, barely change results compared with plain lookup embeddings. Lookup embeddings reach about 2.98 bits per phoneme, versus 3.27 for trigram and 4.48 for unigram models. Code is available at github.com/tpimentelms/phonotactic-complexity. *Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.* ## Institutions - [Google](https://md.research.elamur.ai/free/en/companies/google/) — Company, Mountain View, United States · 492 works · 440,462 citations - [University of Cambridge](https://md.research.elamur.ai/free/en/universities/university-of-cambridge/) — University, Cambridge, United Kingdom · 256 works · 123,373 citations 📚 Shelves (6): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 18,614 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 17,965 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 17,450 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 17,296 works · [💭 Psychology](https://md.research.elamur.ai/free/en/shelves/psychology/) · 1,649 works · [💭 Experimental and Cognitive Psychology](https://md.research.elamur.ai/free/en/shelves/psychology/experimental-and-cognitive-psychology/) · 930 works ## Cite Cite as: Tiago Pimentel, Brian Roark and Ryan Cotterell. 2020. Phonotactic Complexity and Its Trade-offs. Transactions of the Association for Computational Linguistics. The paper itself, not this page. BibTeX from the library's record of the work. ```bibtex @article{pimentel2020phonotactic, title = {{Phonotactic Complexity and Its Trade-offs}}, author = {Tiago Pimentel, Brian Roark and Ryan Cotterell}, journal = {Transactions of the Association for Computational Linguistics}, year = {2020}, doi = {10.1162/tacl\_a\_00296}, url = {https://doi.org/10.1162/tacl_a_00296} } ``` - [← previous Deconstructing information structure](https://md.research.elamur.ai/free/en/papers/kratzer-2020-deconstructing-information-structure/) - [next → Will I Sound Like Me? Improving Persona Consistency in Dialogues through Pragmatic Self-Consciousness](https://md.research.elamur.ai/free/en/papers/kim-2020-will-i-sound-like-me/) - [↑ back to papers of 2020](https://md.research.elamur.ai/free/en/papers/2020/page/6/) ## References 1 references, 1 of them with a card of ours. ↗ leads to the work. 1. Peter F. Brown, Vincent J. Della Pietra, Robert L. Mercer, Stephen A. Della Pietra, and Jennifer C. Lai. 1992. [An estimate of an upper bound for the entropy of English](https://md.research.elamur.ai/free/en/papers/brown-1992-estimate-upper-bound-entropy-english/). Computational Linguistics, 18(1):31–40. *Table 8: Average cross-entropy across all languages and the correlation between complexity… *Updated 30 October 2026*