---
title: "Rethinking the Relationship between the Power Law and Hierarchical Structures — Nakaishi 2026"
description: "Across many corpora, the statistical dependence between linguistic units (phones, characters, words) weakens with distance following a… 65 references."
authors: ["Kai Nakaishi", "Ryo Yoshida", "Kohei Kajikawa", "Koji Hukuahima", "Yohei Oseki"]
year: 2026
venue: "Transactions of the Association for Computational Linguistics"
source: "https://aclanthology.org/2026.tacl-1.87/"
doi: "10.1162/tacl.a.785"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/nakaishi-2026-rethinking-relationship-between-power-law/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/nakaishi-2026-rethinking-relationship-between-power-law/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/9/) / Rethinking the Relationship between the Power Law and Hierarchical Structures

# Rethinking the Relationship between the Power Law and Hierarchical Structures

**Authors:** Kai Nakaishi, [Ryo Yoshida](https://md.research.elamur.ai/free/en/authors/ryo-yoshida/), Kohei Kajikawa, Koji Hukuahima and [Yohei Oseki](https://md.research.elamur.ai/free/en/authors/yohei-oseki/) · **Year:** 2026 · **Venue:** Transactions of the Association for Computational Linguistics · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2026.tacl-1.87/) · [PDF](https://aclanthology.org/2026.tacl-1.87.pdf) · [DOI 10.1162/tacl.a.785](https://doi.org/10.1162/tacl.a.785) · [OpenAlex](https://openalex.org/W4416097743)

**Citations:** 0 citations (OpenAlex, 2026-10-14)

## Summary

Across many corpora, the statistical dependence between linguistic units (phones, characters, words) weakens with distance following a power law rather than an exponential curve. Since Lin and Tegmark (2017), this pattern has often been taken as a sign of hidden hierarchical structure, and the same reasoning has been applied to child speech, birdsong and chimpanzee action sequences. Their derivation depends on two conditions. First, correlation must fall off exponentially with distance inside the tree. Second, trees must be balanced, so that surface distance grows exponentially with tree distance.

Nakaishi et al. test these conditions directly on syntactic parse trees from English (BLLIP, WikiText) and Japanese (NPCMJ). They measure mutual information along tree paths and deviations from PCFG behaviour. They also compute the same statistics for a PCFG fitted to BLLIP. Neither condition holds for real syntax. The fitted PCFG also produces trees too small to reach its theoretical asymptotic regime. The hierarchy-based explanation of power-law correlations therefore needs rethinking, possibly at the discourse level or through critical phenomena.

## Key points

- The authors pose three questions: whether correlation decays exponentially in syntactic trees, whether sequential distance grows exponentially with structural distance, and whether real trees deviate from PCFGs. Real syntax contradicts the assumed answer to every one. Mutual information within trees itself follows a power law with structural distance. Strong branching bias makes sequential distance grow more slowly than linearly. Natural trees differ significantly from PCFG-generated ones.
- The results are consistent across the BLLIP treebank under several annotation schemes, the WikiText corpus and the Japanese treebank NPCMJ.
- In the maximum-likelihood PCFG estimated from BLLIP, asymptotic behaviour does not appear within single sequences. This casts doubt on the argument for other sequences of sentence-like length, such as child speech and animal behaviour. It also limits how far PCFG-based synthetic data can explain how large language models learn natural syntax.
- Future work needs large discourse-annotated corpora to estimate quantities such as I_POS, I_tag and J beyond the sentence level. An alternative account is critical phenomena, which Nakaishi et al. (2024) observed in LLM text as the temperature parameter varies.
- The paper appeared in TACL, vol. 14, pp. 1917–1934 (2026), DOI 10.1162/TACL.a.785.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

Fields: [Natural language processing](https://md.research.elamur.ai/free/en/shelves/nlp/)

## Institutions

- [The University of Tokyo](https://md.research.elamur.ai/free/en/universities/the-university-of-tokyo/) — University, Tokyo, Japan · 31 works · 7,442 citations
- [Georgetown University](https://md.research.elamur.ai/free/en/universities/georgetown-university/) — University, Washington, United States · 62 works · 4,229 citations
- [National Institute of Informatics](https://md.research.elamur.ai/free/en/labs/national-institute-of-informatics/) — Research institute, Tokyo, Japan · 12 works · 2,482 citations
- [RIKEN](https://md.research.elamur.ai/free/en/labs/riken/) — Research institute, Wako, Japan · 6 works · 207 citations
- [National Institute for Japanese Language and Linguistics](https://md.research.elamur.ai/free/en/labs/national-institute-for-japanese-language-and-linguistics/) — Research institute, Tokyo, Japan · 4 works · 2 citations

## Cite

Cite as: Kai Nakaishi, Ryo Yoshida, Kohei Kajikawa, Koji Hukuahima and Yohei Oseki. 2026. Rethinking the Relationship between the Power Law and Hierarchical Structures. Transactions of the Association for Computational Linguistics.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@article{nakaishi2026rethinking,
  title = {{Rethinking the Relationship between the Power Law and Hierarchical Structures}},
  author = {Kai Nakaishi, Ryo Yoshida, Kohei Kajikawa, Koji Hukuahima and Yohei Oseki},
  journal = {Transactions of the Association for Computational Linguistics},
  year = {2026},
  doi = {10.1162/tacl.a.785},
  url = {https://doi.org/10.1162/tacl.a.785}
}
```

- [← previous Rethinking the Idiomaticity Decomposability Hypothesis: Evidence from Distributional Learning](https://md.research.elamur.ai/free/en/papers/mi-2026-rethinking-idiomaticity-decomposability-hypothesis-evidence/)
- [next → Retrieval-augmented Chain-of-thought Prompting Method for Chinese Frame Semantic Parsing](https://md.research.elamur.ai/free/en/papers/li-2026-retrieval-augmented-chain/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/9/)

## References

65 references, 9 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Hirotugu Akaike. 1974. A new look at the statistical model identification. IEEE Transactions on Automatic Control, 19(6):716–723. [find](https://scholar.google.com/scholar?q=Hirotugu+Akaike%2E+1974%2E+A+new+look+at+the+statistical+model+identification%2E+IEEE+Transactions+on+Automatic+Control%2C+19%286%29%3A716%E2%80%93723%2E)
2. In Advances in Neural Information Processing Systems 37. [find](https://scholar.google.com/scholar?q=In+Advances+in+Neural+Information+Processing+Systems+37%2E)
3. Anne Chao and Tsung-Jen Shen. 2003. Non-parametric estimation of shannon’s index of diversity when there are unseen species in sample. Environmental and ecological statistics, 10(4):429–443. [find](https://scholar.google.com/scholar?q=Anne+Chao+and+Tsung%2DJen+Shen%2E+2003%2E+Non%2Dparametric+estimation+of+shannon%E2%80%99s+index+of+diversity+when+there+are+unseen+species+in+sample%2E+Environmental+and+ecological+statistics%2C+10%284%29%3A429%E2%80%93443%2E)
4. Eugene Charniak. 1997. Statistical techniques for natural language parsing. The AI Magazine, 18:33–44. [find](https://scholar.google.com/scholar?q=Eugene+Charniak%2E+1997%2E+Statistical+techniques+for+natural+language+parsing%2E+The+AI+Magazine%2C+18%3A33%E2%80%9344%2E)
5. Eugene Charniak, Don Blaheta, Niyu Ge, Keith Hall, John Hale, and Mark Johnson. 2000. BLLIP 1987-89 WSJCorpus Release 1 LDC2000T43. Linguistic Data Consortium. [find](https://scholar.google.com/scholar?q=Eugene+Charniak%2C+Don+Blaheta%2C+Niyu+Ge%2C+Keith+Hall%2C+John+Hale%2C+and+Mark+Johnson%2E+2000%2E+BLLIP+1987%2D89+WSJCorpus+Release+1+LDC2000T43%2E+Linguistic+Data+Consortium%2E)
6. Noam Chomsky. 1956. Three models for the description of language. IRE Transactions on information theory, 2(3):113–124. [find](https://scholar.google.com/scholar?q=Noam+Chomsky%2E+1956%2E+Three+models+for+the+description+of+language%2E+IRE+Transactions+on+information+theory%2C+2%283%29%3A113%E2%80%93124%2E)
7. Zeyuan Allen-Zhu and Yuanzhi Li. 2023. Physics of language models: Part 1, learning hierarchical language structures. Available at SSRN. [find](https://scholar.google.com/scholar?q=Zeyuan+Allen%2DZhu+and+Yuanzhi+Li%2E+2023%2E+Physics+of+language+models%3A+Part+1%2C+learning+hierarchical+language+structures%2E+Available+at+SSRN%2E)
8. Michael Collins. 2003. Head-driven statistical models for natural language parsing. Computational Linguistics, 29(4):589–637. [our card](https://md.research.elamur.ai/free/en/papers/collins-2003-head-driven-statistical-models-natural/)
9. Eduardo G. Altmann, Giampaolo Cristadoro, and Mirko Degli Esposti. 2012. On the origin of long-range correlations in texts. Proceedings of the National Academy of Sciences, 109(29):11582–11587. [find](https://scholar.google.com/scholar?q=Eduardo+G%2E+Altmann%2C+Giampaolo+Cristadoro%2C+and+Mirko+Degli+Esposti%2E+2012%2E+On+the+origin+of+long%2Drange+correlations+in+texts%2E+Proceedings+of+the+National+Academy+of+Sciences%2C+109%2829%29%3A11582%E2%80%9311587%2E)
10. Matthew S. Dryer. 1992. The greenbergian word order correlations. Language, 68(1):81–138. [find](https://scholar.google.com/scholar?q=Matthew+S%2E+Dryer%2E+1992%2E+The+greenbergian+word+order+correlations%2E+Language%2C+68%281%29%3A81%E2%80%93138%2E)
11. Enrique Alvarez-Lacalle, Beate Dorow, Jean-Pierre Eckmann, and Elisha Moses. 2006. Hierarchical structures induce long-range dynamical correlations in written texts. Proceedings of the National Academy of Sciences, 103(21):7956– 7961. [find](https://scholar.google.com/scholar?q=Enrique+Alvarez%2DLacalle%2C+Beate+Dorow%2C+Jean%2DPierre+Eckmann%2C+and+Elisha+Moses%2E+2006%2E+Hierarchical+structures+induce+long%2Drange+dynamical+correlations+in+written+texts%2E+Proceedings+of+the+National+Academ)
12. Aryaman Arora, Clara Meister, and Ryan Cotterell. 2022. Estimating the entropy of linguistic distributions. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 175–195, Dublin, Ireland. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Aryaman+Arora%2C+Clara+Meister%2C+and+Ryan+Cotterell%2E+2022%2E+Estimating+the+entropy+of+linguistic+distributions%2E+In+Proceedings+of+the+60th+Annual+Meeting+of+the+Association+for+Computational+Linguistics+%28)
13. Steven Bird and Edward Loper. 2004. NLTK: The natural language toolkit. In Proceedings of the ACL Interactive Poster and Demonstration Sessions, pages 214–217, Barcelona, Spain. Association for Computational Linguistics. [our card](https://md.research.elamur.ai/free/en/papers/bird-2006-nltk-natural-language-toolkit/)
14. Francesco Cagnetta and Matthieu Wyart. 2024. Towards a theory of how the structure of language is acquired by deep neural networks. [find](https://scholar.google.com/scholar?q=Francesco+Cagnetta+and+Matthieu+Wyart%2E+2024%2E+Towards+a+theory+of+how+the+structure+of+language+is+acquired+by+deep+neural+networks%2E)
15. Werner Ebeling and Alexander Neiman. 1995. Long-range correlations between letters and sentences in texts. Physica A, 215(3):233–241. [find](https://scholar.google.com/scholar?q=Werner+Ebeling+and+Alexander+Neiman%2E+1995%2E+Long%2Drange+correlations+between+letters+and+sentences+in+texts%2E+Physica+A%2C+215%283%29%3A233%E2%80%93241%2E)
16. Werner Ebeling and Thorsten Pöschel. 1994. Entropy and Long-Range correlations in literary english. Europhysics Letters, 26(4):241–246. [find](https://scholar.google.com/scholar?q=Werner+Ebeling+and+Thorsten+P%C3%B6schel%2E+1994%2E+Entropy+and+Long%2DRange+correlations+in+literary+english%2E+Europhysics+Letters%2C+26%284%29%3A241%E2%80%93246%2E)
17. Kevin Ellis, Armando Solar-Lezama, and Josh Tenenbaum. 2015. Unsupervised learning by program synthesis. In Advances in Neural Information Processing Systems 28. [find](https://scholar.google.com/scholar?q=Kevin+Ellis%2C+Armando+Solar%2DLezama%2C+and+Josh+Tenenbaum%2E+2015%2E+Unsupervised+learning+by+program+synthesis%2E+In+Advances+in+Neural+Information+Processing+Systems+28%2E)
18. Édouard Gilbert and Darrell Conklin. 2007. A probabilistic context-free grammar for melodic reduction? In Proceedings of the International Workshop on Artificial Intelligence and Music, 20th International Joint Conference on Artificial Intelligence. [find](https://scholar.google.com/scholar?q=%C3%89douard+Gilbert+and+Darrell+Conklin%2E+2007%2E+A+probabilistic+context%2Dfree+grammar+for+melodic+reduction%3F+In+Proceedings+of+the+International+Workshop+on+Artificial+Intelligence+and+Music%2C+20th+Internati)
19. Peter Grassberger. 2003. Entropy estimates from insufficient samplings. arXiv:physics/0307138v2. — [arXiv:physics/0307138](https://arxiv.org/abs/physics/0307138) [find](https://scholar.google.com/scholar?q=Peter+Grassberger%2E+2003%2E+Entropy+estimates+from+insufficient+samplings%2E+arXiv%3Aphysics%2F0307138v2%2E+%E2%80%94+%5BarXiv%3Aphysics%2F0307138%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2Fphysics%2F0307138%29)
20. Daniel Harlow, Stephen H. Shenker, Douglas Stanford, and Leonard Susskind. 2012. Tree-like structure of eternal inflation: A solvable model. Physical Review D, 85(6):063516. [find](https://scholar.google.com/scholar?q=Daniel+Harlow%2C+Stephen+H%2E+Shenker%2C+Douglas+Stanford%2C+and+Leonard+Susskind%2E+2012%2E+Tree%2Dlike+structure+of+eternal+inflation%3A+A+solvable+model%2E+Physical+Review+D%2C+85%286%29%3A063516%2E)
21. John A. Hawkins. 1990. A parsing theory of word order universals. Linguistic inquiry, 21(2):223– 261. [find](https://scholar.google.com/scholar?q=John+A%2E+Hawkins%2E+1990%2E+A+parsing+theory+of+word+order+universals%2E+Linguistic+inquiry%2C+21%282%29%3A223%E2%80%93+261%2E)
22. Harold Stanley Heaps. 1978. Information Retrieval: Computational and Theoretical Aspects. Academic Press, Inc. [find](https://scholar.google.com/scholar?q=Harold+Stanley+Heaps%2E+1978%2E+Information+Retrieval%3A+Computational+and+Theoretical+Aspects%2E+Academic+Press%2C+Inc%2E)
23. Matthew Honnibal, Ines Montani, Sofie Van Lan-deghem, and Adriane Boyd. 2020. spaCy: Industrial-strength Natural Language Processing in Python. [find](https://scholar.google.com/scholar?q=Matthew+Honnibal%2C+Ines+Montani%2C+Sofie+Van+Lan%2Ddeghem%2C+and+Adriane+Boyd%2E+2020%2E+spaCy%3A+Industrial%2Dstrength+Natural+Language+Processing+in+Python%2E)
24. Daniel G. Horvitz and Donovan J. Thompson. 1952. A generalization of sampling without replacement from a finite universe. Journal of the American statistical Association, 47(260):663– 685. [find](https://scholar.google.com/scholar?q=Daniel+G%2E+Horvitz+and+Donovan+J%2E+Thompson%2E+1952%2E+A+generalization+of+sampling+without+replacement+from+a+finite+universe%2E+Journal+of+the+American+statistical+Association%2C+47%28260%29%3A663%E2%80%93+685%2E)
25. Elliot Howard-Spink, Misato Hayashi, Tetsuro Matsuzawa, Daniel Schofield, Thibaud Gruber, and Dora Biro. 2024. Nonadjacent dependencies and sequential structure of chimpanzee action during a natural tool-use task. PeerJ, 12:e18484. [find](https://scholar.google.com/scholar?q=Elliot+Howard%2DSpink%2C+Misato+Hayashi%2C+Tetsuro+Matsuzawa%2C+Daniel+Schofield%2C+Thibaud+Gruber%2C+and+Dora+Biro%2E+2024%2E+Nonadjacent+dependencies+and+sequential+structure+of+chimpanzee+action+during+a+natural+t)
26. Filip Jelinek, John D. Lafferty, and Robert L. Mercer. 1992. Basic methods of probabilistic context free grammars. In Speech Recognition and Understanding. [find](https://scholar.google.com/scholar?q=Filip+Jelinek%2C+John+D%2E+Lafferty%2C+and+Robert+L%2E+Mercer%2E+1992%2E+Basic+methods+of+probabilistic+context+free+grammars%2E+In+Speech+Recognition+and+Understanding%2E)
27. Mark Johnson, Thomas L. Griffiths, and Sharon Goldwater. 2006. Adaptor grammars: A framework for specifying compositional nonparametric bayesian models. In Advances in Neural Information Processing Systems 19. [find](https://scholar.google.com/scholar?q=Mark+Johnson%2C+Thomas+L%2E+Griffiths%2C+and+Sharon+Goldwater%2E+2006%2E+Adaptor+grammars%3A+A+framework+for+specifying+compositional+nonparametric+bayesian+models%2E+In+Advances+in+Neural+Information+Processing+Sy)
28. Nikita Kitaev, Steven Cao, and Dan Klein. 2019. Multilingual constituency parsing with self-attention and pre-training. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 3499–3505, Florence, Italy. Association for Computational Linguistics. [our card](https://md.research.elamur.ai/free/en/papers/kitaev-2019-multilingual-constituency-parsing-self-attention/)
29. Nikita Kitaev and Dan Klein. 2018. Constituency parsing with a self-attentive encoder. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 2676–2686, Melbourne, Australia. Association for Computational Linguistics. [our card](https://md.research.elamur.ai/free/en/papers/kitaev-2018-constituency-parsing-self-attentive-encoder/)
30. Bjarne Knudsen and Jotun Hein. 1999. RNA secondary structure prediction using stochastic context-free grammars and evolutionary history. Bioinformatics, 15(6):446–454. [find](https://scholar.google.com/scholar?q=Bjarne+Knudsen+and+Jotun+Hein%2E+1999%2E+RNA+secondary+structure+prediction+using+stochastic+context%2Dfree+grammars+and+evolutionary+history%2E+Bioinformatics%2C+15%286%29%3A446%E2%80%93454%2E)
31. Wentian Li. 1989a. Mutual information functions of natural language texts. Technical Report 89-10-008, Santa Fe Institute. Accessed 2026-03-02. [find](https://scholar.google.com/scholar?q=Wentian+Li%2E+1989a%2E+Mutual+information+functions+of+natural+language+texts%2E+Technical+Report+89%2D10%2D008%2C+Santa+Fe+Institute%2E+Accessed+2026%2D03%2D02%2E)
32. Wentian Li. 1989b. Spatial 1/f spectra in open dynamical systems. Europhysics Letters, 10(5):395. [find](https://scholar.google.com/scholar?q=Wentian+Li%2E+1989b%2E+Spatial+1%2Ff+spectra+in+open+dynamical+systems%2E+Europhysics+Letters%2C+10%285%29%3A395%2E)
33. Wentian Li. 1990. Mutual information functions versus correlation functions. Journal of statistical physics, 60:823–837. [find](https://scholar.google.com/scholar?q=Wentian+Li%2E+1990%2E+Mutual+information+functions+versus+correlation+functions%2E+Journal+of+statistical+physics%2C+60%3A823%E2%80%93837%2E)
34. Wentian Li. 1991. Expansion-modification systems: a model for spatial 1/f spectra. Physical Review A, 43(10):5240. [find](https://scholar.google.com/scholar?q=Wentian+Li%2E+1991%2E+Expansion%2Dmodification+systems%3A+a+model+for+spatial+1%2Ff+spectra%2E+Physical+Review+A%2C+43%2810%29%3A5240%2E)
35. Henry W. Lin and Max Tegmark. 2017. Critical behavior in physics and probabilistic formal languages. Entropy, 19(7):299. [find](https://scholar.google.com/scholar?q=Henry+W%2E+Lin+and+Max+Tegmark%2E+2017%2E+Critical+behavior+in+physics+and+probabilistic+formal+languages%2E+Entropy%2C+19%287%29%3A299%2E)
36. William G. Madow. 1948. On the limiting distributions of estimates based on samples from finite universes. The Annals of Mathematical Statistics, pages 535–545. [find](https://scholar.google.com/scholar?q=William+G%2E+Madow%2E+1948%2E+On+the+limiting+distributions+of+estimates+based+on+samples+from+finite+universes%2E+The+Annals+of+Mathematical+Statistics%2C+pages+535%E2%80%93545%2E)
37. William C. Mann and Sandra A. Thompson. 1988. Rhetorical Structure Theory: Toward a functional theory of text organization. Text - Interdisciplinary Journal for the Study of Discourse, 8(3):243–281. [our card](https://md.research.elamur.ai/free/en/papers/mann-1988-rhetorical-structure-theory-functional/)
38. Mitchell P. Marcus, Beatrice Santorini, and Mary Ann Marcinkiewicz. 1993. Building a large annotated corpus of English: The Penn Treebank. Computational Linguistics, 19(2):313–330. [our card](https://md.research.elamur.ai/free/en/papers/marcus-1993-building-large-annotated-corpus-english/)
39. Marie-Catherine de Marneffe, Christopher D. Manning, Joakim Nivre, and Daniel Zeman. 2021. Universal Dependencies. Computational Linguistics, 47(2):255–308. [our card](https://md.research.elamur.ai/free/en/papers/marneffe-2021-universal-dependencies-2/)
40. Stephen Merity, Caiming Xiong, James Bradbury, and Richard Socher. 2017. Pointer sentinel mixture models. In International Conference on Learning Representations. [our card](https://md.research.elamur.ai/free/en/papers/merity-2016-pointer-sentinel-mixture-models/)
41. Nikolay Mikhaylovskiy and Ilya Churilov. 2023. Autocorrelations decay in texts and applicability limits of language models. arXiv:2305.06615v1. — [arXiv:2305.06615](https://arxiv.org/abs/2305.06615) [find](https://scholar.google.com/scholar?q=Nikolay+Mikhaylovskiy+and+Ilya+Churilov%2E+2023%2E+Autocorrelations+decay+in+texts+and+applicability+limits+of+language+models%2E+arXiv%3A2305%2E06615v1%2E+%E2%80%94+%5BarXiv%3A2305%2E06615%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2305%2E06615%29)
42. George Miller. 1955. Note on the bias of information estimates. Information theory in psychology: Problems and methods. [find](https://scholar.google.com/scholar?q=George+Miller%2E+1955%2E+Note+on+the+bias+of+information+estimates%2E+Information+theory+in+psychology%3A+Problems+and+methods%2E)
43. Kai Nakaishi and Koji Hukushima. 2024. Statistical properties of probabilistic context-sensitive grammars. Physical Review Research, 6(3):033216. [find](https://scholar.google.com/scholar?q=Kai+Nakaishi+and+Koji+Hukushima%2E+2024%2E+Statistical+properties+of+probabilistic+context%2Dsensitive+grammars%2E+Physical+Review+Research%2C+6%283%29%3A033216%2E)
44. Kai Nakaishi, Yoshihiko Nishikawa, and Koji Hukushima. 2024. Critical phase transition in large language models. arXiv:2406.05335v2. — [arXiv:2406.05335](https://arxiv.org/abs/2406.05335) [find](https://scholar.google.com/scholar?q=Kai+Nakaishi%2C+Yoshihiko+Nishikawa%2C+and+Koji+Hukushima%2E+2024%2E+Critical+phase+transition+in+large+language+models%2E+arXiv%3A2406%2E05335v2%2E+%E2%80%94+%5BarXiv%3A2406%2E05335%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2406%2E05335%29)
45. National Institute for Japanese Language and Linguistics. 2016. Ninjal parsed corpus of modern japanese. Version 1.0. Available at [https://npcmj.ninjal.ac.jp/interfaces/](https://npcmj.ninjal.ac.jp/interfaces/) (accessed 2025-10-29). Licensed under a Creative Commons Attribution 4.0 International License (https://creativecommons. org/lice… [↗](https://npcmj.ninjal.ac.jp/interfaces/)
46. Ilya Nemenman, Fariel Shafee, and William Bialek. 2001. Entropy and inference, revisited. Advances in Neural Information Processing Systems 14. [find](https://scholar.google.com/scholar?q=Ilya+Nemenman%2C+Fariel+Shafee%2C+and+William+Bialek%2E+2001%2E+Entropy+and+inference%2C+revisited%2E+Advances+in+Neural+Information+Processing+Systems+14%2E)
47. Matthew Newville, Till Stensitzki, Daniel B. Allen, and Antonino Ingargiola. 2014. Lmfit: Non-linear least-square minimization and curve-fitting for python. [find](https://scholar.google.com/scholar?q=Matthew+Newville%2C+Till+Stensitzki%2C+Daniel+B%2E+Allen%2C+and+Antonino+Ingargiola%2E+2014%2E+Lmfit%3A+Non%2Dlinear+least%2Dsquare+minimization+and+curve%2Dfitting+for+python%2E)
48. Timothy J. O’Donnell, Joshua B. Tenenbaum, and Noah D. Goodman. 2009. Fragment grammars: Exploring computation and reuse in language. Technical Report MIT-CSAIL-TR-2009-013, Massachusetts Institute of Technology. Accessed 2026-03-02. [find](https://scholar.google.com/scholar?q=Timothy+J%2E+O%E2%80%99Donnell%2C+Joshua+B%2E+Tenenbaum%2C+and+Noah+D%2E+Goodman%2E+2009%2E+Fragment+grammars%3A+Exploring+computation+and+reuse+in+language%2E+Technical+Report+MIT%2DCSAIL%2DTR%2D2009%2D013%2C+Massachusetts+Institute+of)
49. Rashmi Prasad, Bonnie Webber, and Aravind Joshi. 2014. Reflections on the Penn Discourse TreeBank, comparable corpora, and complementary annotation. Computational Linguistics, 40(4):921–950. [our card](https://md.research.elamur.ai/free/en/papers/prasad-2014-reflections-penn-discourse-treebank-comparable/)
50. Tim Sainburg and Timothy Q. Gentner. 2021. Toward a computational neuroethology of vocal communication: from bioacoustics to neurophysiology, emerging tools and future directions. Frontiers in Behavioral Neuroscience, 15:811737. [find](https://scholar.google.com/scholar?q=Tim+Sainburg+and+Timothy+Q%2E+Gentner%2E+2021%2E+Toward+a+computational+neuroethology+of+vocal+communication%3A+from+bioacoustics+to+neurophysiology%2C+emerging+tools+and+future+directions%2E+Frontiers+in+Behavio)
51. Tim Sainburg, Anna Mai, and Timothy Q. Gentner. 2022. Long-range sequential dependencies precede complex syntactic production in language acquisition. Proceedings of the Royal Society B, 289(1970):20212657. [find](https://scholar.google.com/scholar?q=Tim+Sainburg%2C+Anna+Mai%2C+and+Timothy+Q%2E+Gentner%2E+2022%2E+Long%2Drange+sequential+dependencies+precede+complex+syntactic+production+in+language+acquisition%2E+Proceedings+of+the+Royal+Society+B%2C+289%281970%29%3A202)
52. Tim Sainburg, Brad Theilman, Marvin Thielk, and Timothy Q. Gentner. 2019. Parallels in the sequential organization of birdsong and human speech. Nature Communications, 10(1):3636. [find](https://scholar.google.com/scholar?q=Tim+Sainburg%2C+Brad+Theilman%2C+Marvin+Thielk%2C+and+Timothy+Q%2E+Gentner%2E+2019%2E+Parallels+in+the+sequential+organization+of+birdsong+and+human+speech%2E+Nature+Communications%2C+10%281%29%3A3636%2E)
53. Gideon Schwarz. 1978. Estimating the dimension of a model. The annals of statistics, pages 461– 464. [find](https://scholar.google.com/scholar?q=Gideon+Schwarz%2E+1978%2E+Estimating+the+dimension+of+a+model%2E+The+annals+of+statistics%2C+pages+461%E2%80%93+464%2E)
54. Huitao Shen. 2019. Mutual information scaling and expressive power of sequence models. arXiv:1905.04271v1. — [arXiv:1905.04271](https://arxiv.org/abs/1905.04271) [find](https://scholar.google.com/scholar?q=Huitao+Shen%2E+2019%2E+Mutual+information+scaling+and+expressive+power+of+sequence+models%2E+arXiv%3A1905%2E04271v1%2E+%E2%80%94+%5BarXiv%3A1905%2E04271%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F1905%2E04271%29)
55. Harry E. Stanley. 1971. Introduction to Phase Transitions and Critical Phenomena. Oxford University Press. [find](https://scholar.google.com/scholar?q=Harry+E%2E+Stanley%2E+1971%2E+Introduction+to+Phase+Transitions+and+Critical+Phenomena%2E+Oxford+University+Press%2E)
56. Shuntaro Takahashi and Kumiko Tanaka-Ishii. 2017. Do neural nets learn statistical laws behind natural language? PloS one, 12(12):e0189326. [find](https://scholar.google.com/scholar?q=Shuntaro+Takahashi+and+Kumiko+Tanaka%2DIshii%2E+2017%2E+Do+neural+nets+learn+statistical+laws+behind+natural+language%3F+PloS+one%2C+12%2812%29%3Ae0189326%2E)
57. Shuntaro Takahashi and Kumiko Tanaka-Ishii. 2019. Evaluating computational language models with scaling properties of natural language. Computational Linguistics, 45(3):481–513. [find](https://scholar.google.com/scholar?q=Shuntaro+Takahashi+and+Kumiko+Tanaka%2DIshii%2E+2019%2E+Evaluating+computational+language+models+with+scaling+properties+of+natural+language%2E+Computational+Linguistics%2C+45%283%29%3A481%E2%80%93513%2E)
58. Kumiko Tanaka-Ishii and Armin Bunde. 2016. Long-range memory in literary texts: On the universal clustering of the rare words. PLoS One, 11(11):e0164658. [find](https://scholar.google.com/scholar?q=Kumiko+Tanaka%2DIshii+and+Armin+Bunde%2E+2016%2E+Long%2Drange+memory+in+literary+texts%3A+On+the+universal+clustering+of+the+rare+words%2E+PLoS+One%2C+11%2811%29%3Ae0164658%2E)
59. Pablo Tano, Sergio Romano, Mariano Sigman, Alejo Salles, and Santiago Figueira. 2020. Towards a more flexible language of thought: Bayesian grammar updates after each concept exposure. Physical Review E, 101(4):042128. [find](https://scholar.google.com/scholar?q=Pablo+Tano%2C+Sergio+Romano%2C+Mariano+Sigman%2C+Alejo+Salles%2C+and+Santiago+Figueira%2E+2020%2E+Towards+a+more+flexible+language+of+thought%3A+Bayesian+grammar+updates+after+each+concept+exposure%2E+Physical+Review)
60. David H. Wolpert and David R. Wolf. 1995. Estimating functions of probability distributions from a finite set of samples. Physical Review E, 52(6):6841. [find](https://scholar.google.com/scholar?q=David+H%2E+Wolpert+and+David+R%2E+Wolf%2E+1995%2E+Estimating+functions+of+probability+distributions+from+a+finite+set+of+samples%2E+Physical+Review+E%2C+52%286%29%3A6841%2E)
61. Peter Worth and Susan Stepney. 2005. Growing music: Musical interpretations of l-systems. In Applications of Evolutionary Computing, pages 545–550. [find](https://scholar.google.com/scholar?q=Peter+Worth+and+Susan+Stepney%2E+2005%2E+Growing+music%3A+Musical+interpretations+of+l%2Dsystems%2E+In+Applications+of+Evolutionary+Computing%2C+pages+545%E2%80%93550%2E)
62. Peter Worth and Susan Stepney. 2021. Recursive bayesian networks: Generalising and unifying probabilistic context-free grammars and dynamic bayesian networks. In Advances in Neural Information Processing Systems 34, pages 4370–4383. [find](https://scholar.google.com/scholar?q=Peter+Worth+and+Susan+Stepney%2E+2021%2E+Recursive+bayesian+networks%3A+Generalising+and+unifying+probabilistic+context%2Dfree+grammars+and+dynamic+bayesian+networks%2E+In+Advances+in+Neural+Information+Process)
63. Mason Youngblood. 2024. Language-like efficiency and structure in house finch song. Proceedings of the Royal Society B, 291(2020):20240250. [find](https://scholar.google.com/scholar?q=Mason+Youngblood%2E+2024%2E+Language%2Dlike+efficiency+and+structure+in+house+finch+song%2E+Proceedings+of+the+Royal+Society+B%2C+291%282020%29%3A20240250%2E)
64. WikiText and the seven categories for NPCMJ are shown in Tables 2 and 3, respectively. [find](https://scholar.google.com/scholar?q=WikiText+and+the+seven+categories+for+NPCMJ+are+shown+in+Tables+2+and+3%2C+respectively%2E)
65. Samuel Zahl. 1977. Jackknifing an index of diversity. Ecology, 58(4):907–913. [find](https://scholar.google.com/scholar?q=Samuel+Zahl%2E+1977%2E+Jackknifing+an+index+of+diversity%2E+Ecology%2C+58%284%29%3A907%E2%80%93913%2E)