---
title: "Clozing the Gap: Exploring Why Language Model Surprisal Outperforms Cloze Surprisal — Nair 2026"
description: "The paper asks why next-word probabilities from language models predict reading times better than probabilities… 59 references, cited by 1 paper here."
authors: ["Sathvik Nair", "Byung-Doh Oh"]
year: 2026
venue: "Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)"
source: "https://aclanthology.org/2026.acl-long.1534/"
doi: "10.18653/v1/2026.acl-long.1534"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/nair-2026-clozing-gap-exploring-why-language/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/nair-2026-clozing-gap-exploring-why-language/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/2/) / Clozing the Gap: Exploring Why Language Model Surprisal Outperforms Cloze Surprisal

# Clozing the Gap: Exploring Why Language Model Surprisal Outperforms Cloze Surprisal

**Authors:** Sathvik Nair and [Byung-Doh Oh](https://md.research.elamur.ai/free/en/authors/byung-doh-oh/) · **Year:** 2026 · **Venue:** Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2026.acl-long.1534/) · [PDF](https://aclanthology.org/2026.acl-long.1534.pdf) · [DOI 10.18653/v1/2026.acl-long.1534](https://doi.org/10.18653/v1/2026.acl-long.1534) · [OpenAlex](https://openalex.org/W7166846609)

**Citations:** 0 citations (OpenAlex, 2026-10-14) · 1 in our library, of the 140,884 papers whose bibliographies we read

## Summary
The paper asks why next-word probabilities from language models predict reading times better than probabilities estimated from human cloze responses. This matters because the choice of predictability measure can change scientific conclusions about prediction in comprehension.

In Experiment 1, the authors use four English reading-time datasets that have matching cloze norms. They compare regression fits using cloze surprisal, GPT2 surprisal, or both. They also test several ways of smoothing and transforming the cloze estimates. The comparison confirms that LM surprisal is the stronger predictor, at least for eye-tracking measures.

In Experiment 2, they deliberately manipulate GPT2's probabilities to test three explanations for this advantage:
- **Resolution:** cloze studies have few responses per word, so their estimates are coarse.
- **Semantics:** LMs can separate words with similar meanings.
- **Frequency:** LMs can assign sensible probabilities to rare words.

All three explanations receive support. The authors argue that cloze studies need finer resolution, for example through timed or maze-like variants. They also call for experiments testing whether human prediction actually makes the fine distinctions that LMs make.

## Key points
- **Datasets:** BK21 SPR has 216 sentence triplets with about 90 cloze responses each. Provo ET has 55 paragraphs (2,746 words) with about 40 responses per word and 84 readers. UCL SPR/ET has 361 sentences, and cloze data from de Varda et al. (2024) covers a 205-sentence subset.
- **Cloze smoothing and transforms:** cloze probabilities were add-one smoothed with a factor V ranging from 50 to 2000. They were then entered as raw probability, surprisal, or several power transforms of surprisal.
- **Language model:** surprisal came from GPT2 (small), chosen because it is widely used in psycholinguistic modeling.
- **Statistical evaluation:** models were compared by cross-validated per-observation log likelihood, with Bonferroni-corrected significance tests.
- **Scope and code:** the study is limited to English, since no cross-linguistic cloze datasets aligned to reading times are known. The code is released at github.com/sathvikn/cloze-surprisal.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

Fields: [Natural language processing](https://md.research.elamur.ai/free/en/shelves/nlp/)

Topics: [🧠 Language models & pretraining](https://md.research.elamur.ai/free/en/topics/language-models/) · [🧒 Psycholinguistics & language acquisition](https://md.research.elamur.ai/free/en/topics/psycholinguistics/)

## Institutions

- [Nanyang Technological University](https://md.research.elamur.ai/free/en/universities/nanyang-technological-university/) — University, Singapore, Singapore · 56 works · 22,783 citations
- [University of Maryland, College Park](https://md.research.elamur.ai/free/en/universities/university-of-maryland-college-park/) — University, College Park, United States · 65 works · 21,800 citations

## Cite

Cite as: Sathvik Nair and Byung-Doh Oh. 2026. Clozing the Gap: Exploring Why Language Model Surprisal Outperforms Cloze Surprisal. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{nair2026clozing,
  title = {{Clozing the Gap: Exploring Why Language Model Surprisal Outperforms Cloze Surprisal}},
  author = {Sathvik Nair and Byung-Doh Oh},
  booktitle = {Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2026},
  doi = {10.18653/v1/2026.acl-long.1534},
  url = {https://doi.org/10.18653/v1/2026.acl-long.1534}
}
```

- [← previous CLASE: A Hybrid Method for Chinese Legalese Stylistic Evaluation](https://md.research.elamur.ai/free/en/papers/ma-2026-clase-hybrid-method-chinese-legalese/)
- [next → Cohesion-6K: An Arabic Dataset for Analyzing Social Cohesion and Conflict in Online Discourse](https://md.research.elamur.ai/free/en/papers/athba-2026-cohesion-6k-arabic-dataset-analyzing/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/2/)

## References

59 references, 5 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Yana Arkhipova, Alessandro Lopopolo, Shravan Vasishth, and Milena Rabovsky. 2025. When meaning matters most: Rethinking cloze probability in N400 research. bioRxiv. [find](https://scholar.google.com/scholar?q=Yana+Arkhipova%2C+Alessandro+Lopopolo%2C+Shravan+Vasishth%2C+and+Milena+Rabovsky%2E+2025%2E+When+meaning+matters+most%3A+Rethinking+cloze+probability+in+N400+research%2E+bioRxiv%2E)
2. Douglas Bates, Martin Mächler, Ben Bolker, and Steve Walker. 2015. Fitting linear mixed-effects models using lme4. Journal of Statistical Software, 67(1):1– 48. [our card](https://md.research.elamur.ai/free/en/papers/bates-2015-fitting-linear-mixed-effects-models/)
3. Stella Biderman, Hailey Schoelkopf, Quentin Gregory Anthony, Herbie Bradley, Kyle O’Brien, Eric Hallahan, Mohammad Aflah Khan, Shivanshu Purohit, USVSN Sai Prashanth, Edward Raff, Aviya Skowron, Lintang Sutawika, and Oskar van der Wal. 2023. Pythia: A suite for analyzing large language models across… [find](https://scholar.google.com/scholar?q=Stella+Biderman%2C+Hailey+Schoelkopf%2C+Quentin+Gregory+Anthony%2C+Herbie+Bradley%2C+Kyle+O%E2%80%99Brien%2C+Eric+Hallahan%2C+Mohammad+Aflah+Khan%2C+Shivanshu+Purohit%2C+USVSN+Sai+Prashanth%2C+Edward+Raff%2C+Aviya+Skowron%2C+Linta)
4. Trevor Brothers and Gina R. Kuperberg. 2021. Word predictability effects are linear, not logarithmic: Implications for probabilistic models of sentence comprehension. Journal of Memory and Language, 116:104174. [find](https://scholar.google.com/scholar?q=Trevor+Brothers+and+Gina+R%2E+Kuperberg%2E+2021%2E+Word+predictability+effects+are+linear%2C+not+logarithmic%3A+Implications+for+probabilistic+models+of+sentence+comprehension%2E+Journal+of+Memory+and+Language%2C+1)
5. Trevor Brothers, Emily Morgan, Anthony Yacovone, and Gina Kuperberg. 2023. Multiple predictions during language comprehension: Friends, foes, or indifferent companions? Cognition, 241:105602. [find](https://scholar.google.com/scholar?q=Trevor+Brothers%2C+Emily+Morgan%2C+Anthony+Yacovone%2C+and+Gina+Kuperberg%2E+2023%2E+Multiple+predictions+during+language+comprehension%3A+Friends%2C+foes%2C+or+indifferent+companions%3F+Cognition%2C+241%3A105602%2E)
6. Ryan Buggy, Stephanie Cho, Cory Shain, and Adrian Staub. 2026. Predictability effects in natural reading are logarithmic, not linear: Evidence from an eye-movement replication of Brothers and Kuperberg (2021). PsyArXiv, Kt3sw_v1. [find](https://scholar.google.com/scholar?q=Ryan+Buggy%2C+Stephanie+Cho%2C+Cory+Shain%2C+and+Adrian+Staub%2E+2026%2E+Predictability+effects+in+natural+reading+are+logarithmic%2C+not+linear%3A+Evidence+from+an+eye%2Dmovement+replication+of+Brothers+and+Kuperber)
7. Andy Clark. 2013. Whatever next? Predictive brains, situated agents, and the future of cognitive science. Behavioral and Brain Sciences, 36(3):181–204. [find](https://scholar.google.com/scholar?q=Andy+Clark%2E+2013%2E+Whatever+next%3F+Predictive+brains%2C+situated+agents%2C+and+the+future+of+cognitive+science%2E+Behavioral+and+Brain+Sciences%2C+36%283%29%3A181%E2%80%93204%2E)
8. Andrea Gregor de Varda, Marco Marelli, and Simona Amenta. 2024. Cloze probability, predictability ratings, and computational estimates for 205 English sentences, aligned with existing EEG and reading time data. Behavior Research Methods, 56:5190– 5213. [find](https://scholar.google.com/scholar?q=Andrea+Gregor+de+Varda%2C+Marco+Marelli%2C+and+Simona+Amenta%2E+2024%2E+Cloze+probability%2C+predictability+ratings%2C+and+computational+estimates+for+205+English+sentences%2C+aligned+with+existing+EEG+and+reading+)
9. Susan F. Ehrlich and Keith Rayner. 1981. Contextual effects on word perception and eye movements during reading. Journal of Verbal Learning and Verbal Behavior, 20(6):641–655. [find](https://scholar.google.com/scholar?q=Susan+F%2E+Ehrlich+and+Keith+Rayner%2E+1981%2E+Contextual+effects+on+word+perception+and+eye+movements+during+reading%2E+Journal+of+Verbal+Learning+and+Verbal+Behavior%2C+20%286%29%3A641%E2%80%93655%2E)
10. Ute Essen and Volker Steinbiss. 1992. Cooccurrence smoothing for stochastic language modeling. In Proceedings of IEEE International Conference on Acoustics, Speech, and Signal Processing, volume 1, pages 161–164. [find](https://scholar.google.com/scholar?q=Ute+Essen+and+Volker+Steinbiss%2E+1992%2E+Cooccurrence+smoothing+for+stochastic+language+modeling%2E+In+Proceedings+of+IEEE+International+Conference+on+Acoustics%2C+Speech%2C+and+Signal+Processing%2C+volume+1%2C+pa)
11. Kara D. Federmeier. 2022. Connecting and considering: Electrophysiology provides insights into comprehension. Psychophysiology, 59(1):e13940. [find](https://scholar.google.com/scholar?q=Kara+D%2E+Federmeier%2E+2022%2E+Connecting+and+considering%3A+Electrophysiology+provides+insights+into+comprehension%2E+Psychophysiology%2C+59%281%29%3Ae13940%2E)
12. Kara D. Federmeier and Marta Kutas. 1999. A rose by any other name: Long-term memory structure and sentence processing. Journal of Memory and Language, 41(4):469–495. [find](https://scholar.google.com/scholar?q=Kara+D%2E+Federmeier+and+Marta+Kutas%2E+1999%2E+A+rose+by+any+other+name%3A+Long%2Dterm+memory+structure+and+sentence+processing%2E+Journal+of+Memory+and+Language%2C+41%284%29%3A469%E2%80%93495%2E)
13. Stefan L. Frank, Irene Fernandez Monsalve, Robin L. Thompson, and Gabriella Vigliocco. 2013. Reading time data for evaluating broad-coverage models of English sentence processing. Behavior Research Methods, 45(4):1182–1190. [find](https://scholar.google.com/scholar?q=Stefan+L%2E+Frank%2C+Irene+Fernandez+Monsalve%2C+Robin+L%2E+Thompson%2C+and+Gabriella+Vigliocco%2E+2013%2E+Reading+time+data+for+evaluating+broad%2Dcoverage+models+of+English+sentence+processing%2E+Behavior+Research+Me)
14. Mario Giulianelli, Luca Malagutti, Juan Luis Gastaldi, Brian DuSell, Tim Vieira, and Ryan Cotterell. 2024a. On the proper treatment of tokenization in psycholinguistics. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 18556–18572. [our card](https://md.research.elamur.ai/free/en/papers/giulianelli-2024-proper-treatment-tokenization-psycholinguistics/)
15. Mario Giulianelli, Andreas Opedal, and Ryan Cotterell. 2024b. Generalized measures of anticipation and responsivity in online language processing. In Findings of the Association for Computational Linguistics: EMNLP 2024, pages 11648–11669. [find](https://scholar.google.com/scholar?q=Mario+Giulianelli%2C+Andreas+Opedal%2C+and+Ryan+Cotterell%2E+2024b%2E+Generalized+measures+of+anticipation+and+responsivity+in+online+language+processing%2E+In+Findings+of+the+Association+for+Computational+Ling)
16. Mario Giulianelli, Sarenne Wallbridge, Ryan Cotterell, and Raquel Fernández. 2026. Incremental alternative sampling as a lens into the temporal and representational resolution of linguistic prediction. Journal of Memory and Language, 148:104715. [find](https://scholar.google.com/scholar?q=Mario+Giulianelli%2C+Sarenne+Wallbridge%2C+Ryan+Cotterell%2C+and+Raquel+Fern%C3%A1ndez%2E+2026%2E+Incremental+alternative+sampling+as+a+lens+into+the+temporal+and+representational+resolution+of+linguistic+prediction)
17. Aaron Gokaslan and Vanya Cohen. 2019. OpenWeb-Text Corpus. [find](https://scholar.google.com/scholar?q=Aaron+Gokaslan+and+Vanya+Cohen%2E+2019%2E+OpenWeb%2DText+Corpus%2E)
18. Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, Amy Yang, Angela Fan, Anirudh Goyal, Anthony Hartshorn, Aobo Yang, Archi Mitra, Archie Sravankumar, Artem Korenev, Arthur Hinsvark, and 542 o… [find](https://scholar.google.com/scholar?q=Aaron+Grattafiori%2C+Abhimanyu+Dubey%2C+Abhinav+Jauhri%2C+Abhinav+Pandey%2C+Abhishek+Kadian%2C+Ahmad+Al%2DDahle%2C+Aiesha+Letman%2C+Akhil+Mathur%2C+Alan+Schelten%2C+Alex+Vaughan%2C+Amy+Yang%2C+Angela+Fan%2C+Anirudh+Goyal%2C+Anth)
19. Yiding Hao, Simon Mendelsohn, Rachel Sterneck, Randi Martinez, and Robert Frank. 2020. Probabilistic predictions of people perusing: Evaluating metrics of language model performance for psycholinguistic modeling. In Proceedings of the Workshop on Cognitive Modeling and Computational Linguistics, pag… [find](https://scholar.google.com/scholar?q=Yiding+Hao%2C+Simon+Mendelsohn%2C+Rachel+Sterneck%2C+Randi+Martinez%2C+and+Robert+Frank%2E+2020%2E+Probabilistic+predictions+of+people+perusing%3A+Evaluating+metrics+of+language+model+performance+for+psycholinguist)
20. Kenneth Heafield, Ivan Pouzyrevsky, Jonathan H. Clark, and Philipp Koehn. 2013. Scalable modified Kneser-Ney language model estimation. In Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics, pages 690–696. [our card](https://md.research.elamur.ai/free/en/papers/heafield-2013-scalable-modified-kneser-ney-language/)
21. Markus J. Hofmann, Steffen Remus, Chris Biemann, Ralph Radach, and Lars Kuchinke. 2022. Language models explain word reading times better than empirical predictability. Frontiers in Artificial Intelligence, 4:730570. [find](https://scholar.google.com/scholar?q=Markus+J%2E+Hofmann%2C+Steffen+Remus%2C+Chris+Biemann%2C+Ralph+Radach%2C+and+Lars+Kuchinke%2E+2022%2E+Language+models+explain+word+reading+times+better+than+empirical+predictability%2E+Frontiers+in+Artificial+Intelli)
22. Jacob Louis Hoover, Morgan Sonderegger, Steven T. Piantadosi, and Timothy J. O’Donnell. 2023. The plausibility of sampling as an algorithmic theory of sentence processing. Open Mind, 7:350–391. [find](https://scholar.google.com/scholar?q=Jacob+Louis+Hoover%2C+Morgan+Sonderegger%2C+Steven+T%2E+Piantadosi%2C+and+Timothy+J%2E+O%E2%80%99Donnell%2E+2023%2E+The+plausibility+of+sampling+as+an+algorithmic+theory+of+sentence+processing%2E+Open+Mind%2C+7%3A350%E2%80%93391%2E)
23. Kuan-Jung Huang, Suhas Arehalli, Mari Kugemoto, Christian Muxica, Grusha Prasad, Brian Dillon, and Tal Linzen. 2024. Large-scale benchmark yields no evidence that language model surprisal explains syntactic disambiguation difficulty. Journal of Memory and Language, 137:104510. [find](https://scholar.google.com/scholar?q=Kuan%2DJung+Huang%2C+Suhas+Arehalli%2C+Mari+Kugemoto%2C+Christian+Muxica%2C+Grusha+Prasad%2C+Brian+Dillon%2C+and+Tal+Linzen%2E+2024%2E+Large%2Dscale+benchmark+yields+no+evidence+that+language+model+surprisal+explains+syn)
24. Lawrence Hubert and Phipps Arabie. 1985. Comparing partitions. Journal of Classification, 2:193–218. [find](https://scholar.google.com/scholar?q=Lawrence+Hubert+and+Phipps+Arabie%2E+1985%2E+Comparing+partitions%2E+Journal+of+Classification%2C+2%3A193%E2%80%93218%2E)
25. Cassandra L. Jacobs, Loïc Grobol, and Alvin Tsang. 2024. Large-scale cloze evaluation reveals that token prediction tasks are neither lexically nor semantically aligned. arXiv preprint, arXiv:2410.12057v2. — [arXiv:2410.12057](https://arxiv.org/abs/2410.12057) [find](https://scholar.google.com/scholar?q=Cassandra+L%2E+Jacobs%2C+Lo%C3%AFc+Grobol%2C+and+Alvin+Tsang%2E+2024%2E+Large%2Dscale+cloze+evaluation+reveals+that+token+prediction+tasks+are+neither+lexically+nor+semantically+aligned%2E+arXiv+preprint%2C+arXiv%3A2410%2E120)
26. Cassandra L. Jacobs, Ryan J. Hubbard, Loïc Grobol, and Kara D. Federmeier. 2025. Uncovering patterns of semantic predictability in sentence processing. Journal of Memory and Language, 144:104653. [find](https://scholar.google.com/scholar?q=Cassandra+L%2E+Jacobs%2C+Ryan+J%2E+Hubbard%2C+Lo%C3%AFc+Grobol%2C+and+Kara+D%2E+Federmeier%2E+2025%2E+Uncovering+patterns+of+semantic+predictability+in+sentence+processing%2E+Journal+of+Memory+and+Language%2C+144%3A104653%2E)
27. Reinhold Kliegl, Ellen Grabner, Martin Rolfs, and Ralf Engbert. 2004. Length, frequency, and predictability effects of words on eye movements in reading. European Journal of Cognitive Psychology, 16(1-2):262–284. [find](https://scholar.google.com/scholar?q=Reinhold+Kliegl%2C+Ellen+Grabner%2C+Martin+Rolfs%2C+and+Ralf+Engbert%2E+2004%2E+Length%2C+frequency%2C+and+predictability+effects+of+words+on+eye+movements+in+reading%2E+European+Journal+of+Cognitive+Psychology%2C+16%281)
28. Tatsuki Kuribayashi, Yohei Oseki, Souhaib Ben Taieb, Kentaro Inui, and Timothy Baldwin. 2025. Large language models are human-like internally. Transactions of the Association for Computational Linguistics, 13:1743–1766. [find](https://scholar.google.com/scholar?q=Tatsuki+Kuribayashi%2C+Yohei+Oseki%2C+Souhaib+Ben+Taieb%2C+Kentaro+Inui%2C+and+Timothy+Baldwin%2E+2025%2E+Large+language+models+are+human%2Dlike+internally%2E+Transactions+of+the+Association+for+Computational+Linguis)
29. Marta Kutas and Steven A. Hillyard. 1984. Brain potentials during reading reflect word expectancy and semantic association. Nature, 307(5947):161–163. [find](https://scholar.google.com/scholar?q=Marta+Kutas+and+Steven+A%2E+Hillyard%2E+1984%2E+Brain+potentials+during+reading+reflect+word+expectancy+and+semantic+association%2E+Nature%2C+307%285947%29%3A161%E2%80%93163%2E)
30. Steven G. Luke and Kiel Christianson. 2018. The Provo Corpus: A large eye-tracking corpus with predictability norms. Behavior Research Methods, 50(2):826– 833. [find](https://scholar.google.com/scholar?q=Steven+G%2E+Luke+and+Kiel+Christianson%2E+2018%2E+The+Provo+Corpus%3A+A+large+eye%2Dtracking+corpus+with+predictability+norms%2E+Behavior+Research+Methods%2C+50%282%29%3A826%E2%80%93+833%2E)
31. Clara Meister, Mario Giulianelli, and Tiago Pimentel. 2024. Towards a similarity-adjusted surprisal theory. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 16485–16498. [find](https://scholar.google.com/scholar?q=Clara+Meister%2C+Mario+Giulianelli%2C+and+Tiago+Pimentel%2E+2024%2E+Towards+a+similarity%2Dadjusted+surprisal+theory%2E+In+Proceedings+of+the+2024+Conference+on+Empirical+Methods+in+Natural+Language+Processing%2C+p)
32. Clara Meister, Tiago Pimentel, Patrick Haller, Lena Jäger, Ryan Cotterell, and Roger Levy. 2021. Revisiting the Uniform Information Density hypothesis. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 963–980. [find](https://scholar.google.com/scholar?q=Clara+Meister%2C+Tiago+Pimentel%2C+Patrick+Haller%2C+Lena+J%C3%A4ger%2C+Ryan+Cotterell%2C+and+Roger+Levy%2E+2021%2E+Revisiting+the+Uniform+Information+Density+hypothesis%2E+In+Proceedings+of+the+2021+Conference+on+Empiric)
33. James A. Michaelov, Seana Coulson, and Benjamin K. Bergen. 2023. So cloze yet so far: N400 amplitude is better predicted by distributional information than human predictability judgements. IEEE Transactions on Cognitive and Developmental Systems, 15(3):1033–1042. [find](https://scholar.google.com/scholar?q=James+A%2E+Michaelov%2C+Seana+Coulson%2C+and+Benjamin+K%2E+Bergen%2E+2023%2E+So+cloze+yet+so+far%3A+N400+amplitude+is+better+predicted+by+distributional+information+than+human+predictability+judgements%2E+IEEE+Transa)
34. Sathvik Nair and Colin Phillips. 2026. Across the levels of analysis: Explaining predictive processing in humans requires more than machine-estimated probabilities. arXiv preprint, arXiv:2604.09466. — [arXiv:2604.09466](https://arxiv.org/abs/2604.09466) [find](https://scholar.google.com/scholar?q=Sathvik+Nair+and+Colin+Phillips%2E+2026%2E+Across+the+levels+of+analysis%3A+Explaining+predictive+processing+in+humans+requires+more+than+machine%2Destimated+probabilities%2E+arXiv+preprint%2C+arXiv%3A2604%2E09466%2E+%E2%80%94)
35. Sathvik Nair and Philip Resnik. 2023. Words, subwords, and morphemes: What really matters in the surprisal-reading time relationship? In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 11251–11260. [find](https://scholar.google.com/scholar?q=Sathvik+Nair+and+Philip+Resnik%2E+2023%2E+Words%2C+subwords%2C+and+morphemes%3A+What+really+matters+in+the+surprisal%2Dreading+time+relationship%3F+In+Findings+of+the+Association+for+Computational+Linguistics%3A+EMNL)
36. Samer Nour Eddine, Trevor Brothers, Lin Wang, Michael Spratling, and Gina R. Kuperberg. 2024. A predictive coding model of the N400. Cognition, 246:105755. [find](https://scholar.google.com/scholar?q=Samer+Nour+Eddine%2C+Trevor+Brothers%2C+Lin+Wang%2C+Michael+Spratling%2C+and+Gina+R%2E+Kuperberg%2E+2024%2E+A+predictive+coding+model+of+the+N400%2E+Cognition%2C+246%3A105755%2E)
37. Byung-Doh Oh and Tal Linzen. 2025. To model human linguistic prediction, make LLMs less superhuman. arXiv preprint, arXiv:2510.05141v1. — [arXiv:2510.05141](https://arxiv.org/abs/2510.05141) [find](https://scholar.google.com/scholar?q=Byung%2DDoh+Oh+and+Tal+Linzen%2E+2025%2E+To+model+human+linguistic+prediction%2C+make+LLMs+less+superhuman%2E+arXiv+preprint%2C+arXiv%3A2510%2E05141v1%2E+%E2%80%94+%5BarXiv%3A2510%2E05141%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2510%2E05141%29)
38. Byung-Doh Oh and William Schuler. 2023. Why does surprisal from larger Transformer-based language models provide a poorer fit to human reading times? Transactions of the Association for Computational Linguistics, 11:336–350. [our card](https://md.research.elamur.ai/free/en/papers/oh-2023-why-does-surprisal-larger-transformer/)
39. Byung-Doh Oh and William Schuler. 2024. Leading whitespaces of language models’ subword vocabulary pose a confound for calculating word probabilities. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 3464–3472. [find](https://scholar.google.com/scholar?q=Byung%2DDoh+Oh+and+William+Schuler%2E+2024%2E+Leading+whitespaces+of+language+models%E2%80%99+subword+vocabulary+pose+a+confound+for+calculating+word+probabilities%2E+In+Proceedings+of+the+2024+Conference+on+Empirica)
40. Byung-Doh Oh and William Schuler. 2025. Dissociable frequency effects attenuate as large language model surprisal predictors improve. Journal of Memory and Language, 143:104645. [find](https://scholar.google.com/scholar?q=Byung%2DDoh+Oh+and+William+Schuler%2E+2025%2E+Dissociable+frequency+effects+attenuate+as+large+language+model+surprisal+predictors+improve%2E+Journal+of+Memory+and+Language%2C+143%3A104645%2E)
41. Chiebuka Ohams, Sathvik Nair, Shohini Bhattasali, and Philip Resnik. 2026. A predictive coding model for online sentence processing. Journal of Memory and Language, 146:104705. [find](https://scholar.google.com/scholar?q=Chiebuka+Ohams%2C+Sathvik+Nair%2C+Shohini+Bhattasali%2C+and+Philip+Resnik%2E+2026%2E+A+predictive+coding+model+for+online+sentence+processing%2E+Journal+of+Memory+and+Language%2C+146%3A104705%2E)
42. Tiago Pimentel and Clara Meister. 2024. How to compute the probability of a word. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 18358–18375. [find](https://scholar.google.com/scholar?q=Tiago+Pimentel+and+Clara+Meister%2E+2024%2E+How+to+compute+the+probability+of+a+word%2E+In+Proceedings+of+the+2024+Conference+on+Empirical+Methods+in+Natural+Language+Processing%2C+pages+18358%E2%80%9318375%2E)
43. Alec Radford, Jeff Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. 2019. Language models are unsupervised multitask learners. OpenAI Technical Report. [find](https://scholar.google.com/scholar?q=Alec+Radford%2C+Jeff+Wu%2C+Rewon+Child%2C+David+Luan%2C+Dario+Amodei%2C+and+Ilya+Sutskever%2E+2019%2E+Language+models+are+unsupervised+multitask+learners%2E+OpenAI+Technical+Report%2E)
44. Cory Shain. 2024. Word frequency and predictability dissociate in naturalistic reading. Open Mind, 8:177– 201. [find](https://scholar.google.com/scholar?q=Cory+Shain%2E+2024%2E+Word+frequency+and+predictability+dissociate+in+naturalistic+reading%2E+Open+Mind%2C+8%3A177%E2%80%93+201%2E)
45. Cory Shain, Clara Meister, Tiago Pimentel, Ryan Cotterell, and Roger Levy. 2024. Large-scale evidence for logarithmic effects of word predictability on reading time. Proceedings of the National Academy of Sciences, 121(10):e2307876121. [find](https://scholar.google.com/scholar?q=Cory+Shain%2C+Clara+Meister%2C+Tiago+Pimentel%2C+Ryan+Cotterell%2C+and+Roger+Levy%2E+2024%2E+Large%2Dscale+evidence+for+logarithmic+effects+of+word+predictability+on+reading+time%2E+Proceedings+of+the+National+Academ)
46. Buck Shlegeris, Fabien Roger, Lawrence Chan, and Euan McLean. 2024. Language models are better than humans at next-token prediction. Transactions on Machine Learning Research. [find](https://scholar.google.com/scholar?q=Buck+Shlegeris%2C+Fabien+Roger%2C+Lawrence+Chan%2C+and+Euan+McLean%2E+2024%2E+Language+models+are+better+than+humans+at+next%2Dtoken+prediction%2E+Transactions+on+Machine+Learning+Research%2E)
47. Sophie Slaats and Andrea E. Martin. 2025. What’s surprising about surprisal. Computational Brain & Behavior, 8:233–248. [find](https://scholar.google.com/scholar?q=Sophie+Slaats+and+Andrea+E%2E+Martin%2E+2025%2E+What%E2%80%99s+surprising+about+surprisal%2E+Computational+Brain+%26+Behavior%2C+8%3A233%E2%80%93248%2E)
48. Nathaniel J. Smith and Roger Levy. 2011. Cloze but no cigar: The complex relationship between cloze, corpus, and subjective probabilities in language processing. In Proceedings of the Annual Meeting of the Cognitive Science Society, volume 33, pages 1637– 1642. [find](https://scholar.google.com/scholar?q=Nathaniel+J%2E+Smith+and+Roger+Levy%2E+2011%2E+Cloze+but+no+cigar%3A+The+complex+relationship+between+cloze%2C+corpus%2C+and+subjective+probabilities+in+language+processing%2E+In+Proceedings+of+the+Annual+Meeting+o)
49. Nathaniel J. Smith and Roger Levy. 2013. The effect of word predictability on reading time is logarithmic. Cognition, 128:302–319. [find](https://scholar.google.com/scholar?q=Nathaniel+J%2E+Smith+and+Roger+Levy%2E+2013%2E+The+effect+of+word+predictability+on+reading+time+is+logarithmic%2E+Cognition%2C+128%3A302%E2%80%93319%2E)
50. Robyn Speer. 2022. rspeer/wordfreq: v3.0. [find](https://scholar.google.com/scholar?q=Robyn+Speer%2E+2022%2E+rspeer%2Fwordfreq%3A+v3%2E0%2E)
51. Miloš Stanojević, Jonathan R. Brennan, Donald Dunagan, Mark Steedman, and John T. Hale. 2023. Modeling structure-building in the brain with CCG parsing and large language models. Cognitive Science, 47(7):e13312. [find](https://scholar.google.com/scholar?q=Milo%C5%A1+Stanojevic%CC%81%2C+Jonathan+R%2E+Brennan%2C+Donald+Dunagan%2C+Mark+Steedman%2C+and+John+T%2E+Hale%2E+2023%2E+Modeling+structure%2Dbuilding+in+the+brain+with+CCG+parsing+and+large+language+models%2E+Cognitive+Science%2C+4)
52. Adrian Staub. 2025. Predictability in language comprehension: Prospects and problems for surprisal. Annual Review of Linguistics, 11:17–34. [find](https://scholar.google.com/scholar?q=Adrian+Staub%2E+2025%2E+Predictability+in+language+comprehension%3A+Prospects+and+problems+for+surprisal%2E+Annual+Review+of+Linguistics%2C+11%3A17%E2%80%9334%2E)
53. Adrian Staub, Margaret Grant, Lori Astheimer, and Andrew Cohen. 2015. The influence of cloze probability and item constraint on cloze task response time. Journal of Memory and Language, 82:1–17. [find](https://scholar.google.com/scholar?q=Adrian+Staub%2C+Margaret+Grant%2C+Lori+Astheimer%2C+and+Andrew+Cohen%2E+2015%2E+The+influence+of+cloze+probability+and+item+constraint+on+cloze+task+response+time%2E+Journal+of+Memory+and+Language%2C+82%3A1%E2%80%9317%2E)
54. Jakub M. Szewczyk and Kara D. Federmeier. 2022. Context-based facilitation of semantic access follows both logarithmic and linear functions of stimulus probability. Journal of Memory and Language, 123:104311. [find](https://scholar.google.com/scholar?q=Jakub+M%2E+Szewczyk+and+Kara+D%2E+Federmeier%2E+2022%2E+Context%2Dbased+facilitation+of+semantic+access+follows+both+logarithmic+and+linear+functions+of+stimulus+probability%2E+Journal+of+Memory+and+Language%2C+123)
55. Wilson L. Taylor. 1953. “Cloze procedure”: A new tool for measuring readability. Journalism Quarterly, 30(4):415–433. [find](https://scholar.google.com/scholar?q=Wilson+L%2E+Taylor%2E+1953%2E+%E2%80%9CCloze+procedure%E2%80%9D%3A+A+new+tool+for+measuring+readability%2E+Journalism+Quarterly%2C+30%284%29%3A415%E2%80%93433%2E)
56. William Timkey, Kuan-Jung Huang, Byung-Doh Oh, Grusha Prasad, Suhas Arehalli, Tal Linzen, and Brian Dillon. 2025. Eye movements reveal a dissociation between prediction and structural processing in language comprehension. PsyArXiv, eq2ra_v1. [find](https://scholar.google.com/scholar?q=William+Timkey%2C+Kuan%2DJung+Huang%2C+Byung%2DDoh+Oh%2C+Grusha+Prasad%2C+Suhas+Arehalli%2C+Tal+Linzen%2C+and+Brian+Dillon%2E+2025%2E+Eye+movements+reveal+a+dissociation+between+prediction+and+structural+processing+in+la)
57. Shravan Vasishth. 2006. On the proper treatment of spillover in real-time reading studies: Consequences for psycholinguistic theories. In Proceedings of the International Conference on Linguistic Evidence, pages 96–100. [find](https://scholar.google.com/scholar?q=Shravan+Vasishth%2E+2006%2E+On+the+proper+treatment+of+spillover+in+real%2Dtime+reading+studies%3A+Consequences+for+psycholinguistic+theories%2E+In+Proceedings+of+the+International+Conference+on+Linguistic+Evid)
58. Ethan Gotlieb Wilcox, Tiago Pimentel, Clara Meister, Ryan Cotterell, and Roger P. Levy. 2023. Testing the predictions of surprisal theory in 11 languages. Transactions of the Association for Computational Linguistics, 11:1451–1470. [our card](https://md.research.elamur.ai/free/en/papers/wilcox-2023-testing-predictions-surprisal-theory-11/)
59. Weijie Xu, Jason Chon, Tianran Liu, and Richard Futrell. 2023. The linearity of the effect of surprisal on reading times across languages. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 15711–15721. [find](https://scholar.google.com/scholar?q=Weijie+Xu%2C+Jason+Chon%2C+Tianran+Liu%2C+and+Richard+Futrell%2E+2023%2E+The+linearity+of+the+effect+of+surprisal+on+reading+times+across+languages%2E+In+Findings+of+the+Association+for+Computational+Linguistics%3A)