---
title: "RNNs as psycholinguistic subjects: Syntactic state and grammatical dependency — Futrell 2018"
description: "They build controlled sets of sentences and use per-word surprisal as a stand-in for human reading time, analysing… 58 references, cited by 39 papers here."
authors: ["Richard Futrell", "Ethan Wilcox", "Takashi Morita", "Roger Levy"]
year: 2018
venue: "arXiv:1809.01329"
source: "https://arxiv.org/abs/1809.01329"
doi: "10.48550/arxiv.1809.01329"
arxiv: "1809.01329"
licence: "arXiv — non-exclusive licence to distribute"
licence_url: "http://arxiv.org/licenses/nonexclusive-distrib/1.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/futrell-2018-rnns-psycholinguistic-subjects-syntactic-state/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/futrell-2018-rnns-psycholinguistic-subjects-syntactic-state/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2018](https://md.research.elamur.ai/free/en/papers/2018/page/3/) / RNNs as psycholinguistic subjects: Syntactic state and grammatical dependency

# RNNs as psycholinguistic subjects: Syntactic state and grammatical dependency

**Authors:** [Richard Futrell](https://md.research.elamur.ai/free/en/authors/richard-futrell/), [Ethan Wilcox](https://md.research.elamur.ai/free/en/authors/ethan-gotlieb-wilcox/), [Takashi Morita](https://md.research.elamur.ai/free/en/authors/takashi-morita/) and [Roger Levy](https://md.research.elamur.ai/free/en/authors/roger-levy/) · **Year:** 2018 · **Venue:** arXiv:1809.01329 · **Licence:** [arXiv — non-exclusive licence to distribute](http://arxiv.org/licenses/nonexclusive-distrib/1.0/)

**Links:** [Publisher's page](https://arxiv.org/abs/1809.01329) · [PDF](https://arxiv.org/pdf/1809.01329) · [DOI 10.48550/arxiv.1809.01329](https://doi.org/10.48550/arxiv.1809.01329) · [arXiv 1809.01329](https://arxiv.org/abs/1809.01329) · [OpenAlex](https://openalex.org/W2891399254)

**Citations:** 36 citations (OpenAlex, 2026-10-14) · [39](https://md.research.elamur.ai/free/en/papers/futrell-2018-rnns-psycholinguistic-subjects-syntactic-state/cited-by/) in our library, of the 140,884 papers whose bibliographies we read

## Summary
The paper asks which aspects of grammar LSTM language models pick up when trained only to predict the next word. The authors treat the networks the way psycholinguists treat human participants. They build controlled sets of sentences and use per-word surprisal as a stand-in for human reading time, analysing it with linear mixed-effects regression. All analyses were preregistered.

Three models are tested: two public English LSTMs (JRNN and GRNN) and a new character-level Japanese LSTM (JPRNN). The study probes two kinds of knowledge:
- **Syntactic state:** garden-path sentences, center-embedding, and expectations for obligatory upcoming material.
- **Grammatical dependency:** reflexive pronoun binding and licensing of negative polarity items (NPIs).

The models clearly track incremental syntactic structure and show human-like garden-path effects. They also maintain expectations for required continuations, although these weaken as the intervening material grows longer or more complex. However, none of the models learns the correct structural conditions for reflexives or NPIs, in English or Japanese. The authors conclude that syntax can emerge from string input alone but does not reliably generalize across constructions.

## Key points
- **Models tested:**
  - JRNN (Jozefowicz et al. 2016): trained on the One Billion Word Benchmark, with 2 layers of 8196 units and CNN character inputs.
  - GRNN (Gulordava et al. 2018): trained on 90M tokens of English Wikipedia, with 2 layers of 650 units.
  - JPRNN: character-based, 650 units, trained on 800k of 900k Japanese Wikipedia paragraphs (the rest held out for validation), reaching validation perplexity 12.67.
- **Garden paths:** In 29 Main Verb/Reduced Relative items, surprisal at the disambiguating verb is highest in the condition that pairs a reduced relative clause with an ambiguous verb form (interaction p \< 0.001 in both English models).
- **Partial verb-form knowledge:** An unambiguous participle lowers surprisal but does not remove the reduced-RC penalty. The models apparently treat the participle as only a noisy cue to a relative clause.
- **Japanese NPI shika:** The model learns that shika predicts a later negated verb. It fails, however, to restrict this expectation to the structurally correct clause and wrongly carries it over into the matrix clause.
- **Resources:** Scripts, materials, and results are available at github.com/Futrell/rnn_psycholinguistic_subjects.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

Fields: [Other](https://md.research.elamur.ai/free/en/shelves/other/)

Topics: [🧒 Psycholinguistics & language acquisition](https://md.research.elamur.ai/free/en/topics/psycholinguistics/)

## Institutions

- [Massachusetts Institute of Technology](https://md.research.elamur.ai/free/en/universities/massachusetts-institute-of-technology/) — University, Cambridge, United States · 125 works · 60,031 citations
- [Harvard University](https://md.research.elamur.ai/free/en/universities/harvard-university/) — University, Cambridge, United States · 64 works · 35,670 citations
- [University of California, Irvine](https://md.research.elamur.ai/free/en/universities/university-of-california-irvine/) — University, Irvine, United States · 22 works · 35,120 citations
- [Kyoto University](https://md.research.elamur.ai/free/en/universities/kyoto-university/) — University, Kyoto, Japan · 21 works · 7,437 citations

## Cite

Cite as: Richard Futrell, Ethan Wilcox, Takashi Morita and Roger Levy. 2018. RNNs as psycholinguistic subjects: Syntactic state and grammatical dependency. arXiv:1809.01329.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@misc{futrell2018rnns,
  title = {{RNNs as psycholinguistic subjects: Syntactic state and grammatical dependency}},
  author = {Richard Futrell, Ethan Wilcox, Takashi Morita and Roger Levy},
  year = {2018},
  doi = {10.48550/arxiv.1809.01329},
  eprint = {1809.01329},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.1809.01329}
}
```

- [← previous A Dependency Perspective on RST Discourse Parsing and Evaluation](https://md.research.elamur.ai/free/en/papers/morey-2018-dependency-perspective-rst-discourse-parsing/)
- [next → Stance Detection in Fake News A Combined Feature Representation](https://md.research.elamur.ai/free/en/papers/ghanem-2018-stance-detection-fake-news-combined/)
- [↑ back to papers of 2018](https://md.research.elamur.ai/free/en/papers/2018/page/3/)

## References

58 references, 8 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Hiroshi Aoyagi and Toru Ishii. 1994. On agreement-inducing vs. non-agreement-inducing NPIs. In Proceedings of the North East Linguistic Society 24, pages 1 – 15. GLSA Publications. [find](https://scholar.google.com/scholar?q=Hiroshi+Aoyagi+and+Toru+Ishii%2E+1994%2E+On+agreement%2Dinducing+vs%2E+non%2Dagreement%2Dinducing+NPIs%2E+In+Proceedings+of+the+North+East+Linguistic+Society+24%2C+pages+1+%E2%80%93+15%2E+GLSA+Publications%2E)
2. R.H. Baayen, D.J. Davidson, and D.M. Bates. 2008. Mixed-effects modeling with crossed random effects for subjects and items. Journal of Memory and Language, 59(4):390–412. [find](https://scholar.google.com/scholar?q=R%2EH%2E+Baayen%2C+D%2EJ%2E+Davidson%2C+and+D%2EM%2E+Bates%2E+2008%2E+Mixed%2Deffects+modeling+with+crossed+random+effects+for+subjects+and+items%2E+Journal+of+Memory+and+Language%2C+59%284%29%3A390%E2%80%93412%2E)
3. Dale J Barr, Roger Levy, Christoph Scheepers, and Harry J Tily. 2013. Random effects structure for confirmatory hypothesis testing: Keep it maximal. Journal of Memory and Language, 68(3):255–278. [find](https://scholar.google.com/scholar?q=Dale+J+Barr%2C+Roger+Levy%2C+Christoph+Scheepers%2C+and+Harry+J+Tily%2E+2013%2E+Random+effects+structure+for+confirmatory+hypothesis+testing%3A+Keep+it+maximal%2E+Journal+of+Memory+and+Language%2C+68%283%29%3A255%E2%80%93278%2E)
4. Yonatan Belinkov, Lluı́s Màrquez, Hassan Sajjad, Nadir Durrani, Fahim Dalvi, and James Glass. 2018. Evaluating layers of representation in neural machine translation on part-of-speech and semantic tagging tasks. arXiv preprint arXiv:1801.07772. [our card](https://md.research.elamur.ai/free/en/papers/belinkov-2018-evaluating-layers-representation-neural-machine/)
5. Jean-Philippe Bernardy and Shalom Lappin. 2017. Using deep neural networks to learn syntactic agreement. Linguistic Issues in Language Technology, 15:1–15. [find](https://scholar.google.com/scholar?q=Jean%2DPhilippe+Bernardy+and+Shalom+Lappin%2E+2017%2E+Using+deep+neural+networks+to+learn+syntactic+agreement%2E+Linguistic+Issues+in+Language+Technology%2C+15%3A1%E2%80%9315%2E)
6. Thomas G. Bever. 1970. The cognitive basis for linguistic structures. In J. R. Hayes, editor, Cognition and the Development of Language. Wiley, New York. [find](https://scholar.google.com/scholar?q=Thomas+G%2E+Bever%2E+1970%2E+The+cognitive+basis+for+linguistic+structures%2E+In+J%2E+R%2E+Hayes%2C+editor%2C+Cognition+and+the+Development+of+Language%2E+Wiley%2C+New+York%2E)
7. Terra Blevins, Omer Levy, and Luke Zettlemoyer. 2018. Deep RNNs encode soft hierarchical syntax. arXiv preprint arXiv:1805.04218. [find](https://scholar.google.com/scholar?q=Terra+Blevins%2C+Omer+Levy%2C+and+Luke+Zettlemoyer%2E+2018%2E+Deep+RNNs+encode+soft+hierarchical+syntax%2E+arXiv+preprint+arXiv%3A1805%2E04218%2E)
8. Kathryn Bock and Carol A Miller. 1991. Broken agreement. Cognitive Psychology, 23(1):45–93. [find](https://scholar.google.com/scholar?q=Kathryn+Bock+and+Carol+A+Miller%2E+1991%2E+Broken+agreement%2E+Cognitive+Psychology%2C+23%281%29%3A45%E2%80%9393%2E)
9. Bureau of Labor Statistics. 2017. Labor force statistics from the current population survey. [find](https://scholar.google.com/scholar?q=Bureau+of+Labor+Statistics%2E+2017%2E+Labor+force+statistics+from+the+current+population+survey%2E)
10. Aylin Caliskan, Joanna J Bryson, and Arvind Narayanan. 2017. Semantics derived automatically from language corpora contain human-like biases. Science, 356(6334):183–186. [find](https://scholar.google.com/scholar?q=Aylin+Caliskan%2C+Joanna+J+Bryson%2C+and+Arvind+Narayanan%2E+2017%2E+Semantics+derived+automatically+from+language+corpora+contain+human%2Dlike+biases%2E+Science%2C+356%286334%29%3A183%E2%80%93186%2E)
11. Ciprian Chelba, Tomas Mikolov, Mike Schuster, Qi Ge, Thorsten Brants, Phillipp Koehn, and Tony Robinson. 2013. One billion word benchmark for measuring progress in statistical language modeling. arXiv preprint arXiv:1312.3005. [our card](https://md.research.elamur.ai/free/en/papers/chelba-2013-one-billion-word-benchmark-measuring/)
12. N. Chomsky. 1981. Lectures on Government and Binding. Foris Publications, Dordrecht, The Netherlands. [find](https://scholar.google.com/scholar?q=N%2E+Chomsky%2E+1981%2E+Lectures+on+Government+and+Binding%2E+Foris+Publications%2C+Dordrecht%2C+The+Netherlands%2E)
13. Morten H. Christiansen and Nick Chater. 1999. Toward a connectionist model of recursion in human linguistic performance. Cognitive Science, 23(2):157–205. [find](https://scholar.google.com/scholar?q=Morten+H%2E+Christiansen+and+Nick+Chater%2E+1999%2E+Toward+a+connectionist+model+of+recursion+in+human+linguistic+performance%2E+Cognitive+Science%2C+23%282%29%3A157%E2%80%93205%2E)
14. Charles Clifton Jr., Matthew J. Traxler, Mohamed Taha Mohamed, Rihana S. Williams, Robin K. Morris, and Keith Rayner. 2003. The use of thematic role information in parsing: Syntactic processing autonomy revisited. Journal of Memory and Language, 49:317–334. [find](https://scholar.google.com/scholar?q=Charles+Clifton+Jr%2E%2C+Matthew+J%2E+Traxler%2C+Mohamed+Taha+Mohamed%2C+Rihana+S%2E+Williams%2C+Robin+K%2E+Morris%2C+and+Keith+Rayner%2E+2003%2E+The+use+of+thematic+role+information+in+parsing%3A+Syntactic+processing+autono)
15. Brian Dillon, Alan Mishler, Shayne Sloggett, and Colin Phillips. 2013. Contrasting intrusion profiles for agreement and anaphora: Experimental and modeling evidence. Journal of Memory and Language, 69(2):85–103. [find](https://scholar.google.com/scholar?q=Brian+Dillon%2C+Alan+Mishler%2C+Shayne+Sloggett%2C+and+Colin+Phillips%2E+2013%2E+Contrasting+intrusion+profiles+for+agreement+and+anaphora%3A+Experimental+and+modeling+evidence%2E+Journal+of+Memory+and+Language%2C+69)
16. J.L. Elman. 1990. Finding structure in time. Cognitive Science, 14(2):179–211. [find](https://scholar.google.com/scholar?q=J%2EL%2E+Elman%2E+1990%2E+Finding+structure+in+time%2E+Cognitive+Science%2C+14%282%29%3A179%E2%80%93211%2E)
17. Emile Enguehard, Yoav Goldberg, and Tal Linzen. 2017. Exploring the syntactic abilities of RNNs with multi-task learning. arXiv preprint arXiv:1706.03542. [find](https://scholar.google.com/scholar?q=Emile+Enguehard%2C+Yoav+Goldberg%2C+and+Tal+Linzen%2E+2017%2E+Exploring+the+syntactic+abilities+of+RNNs+with+multi%2Dtask+learning%2E+arXiv+preprint+arXiv%3A1706%2E03542%2E)
18. Allyson Ettinger, Sudha Rao, Hal Daumé III, and Emily M Bender. 2017. Towards linguistically generalizable NLP systems: A workshop and shared task. In Proceedings of the First Workshop on Building Linguistically Generalizable NLP Systems, pages 1–10. [find](https://scholar.google.com/scholar?q=Allyson+Ettinger%2C+Sudha+Rao%2C+Hal+Daume%CC%81+III%2C+and+Emily+M+Bender%2E+2017%2E+Towards+linguistically+generalizable+NLP+systems%3A+A+workshop+and+shared+task%2E+In+Proceedings+of+the+First+Workshop+on+Building+Li)
19. Fernanda Ferreira and Charles Clifton. 1986. The independence of syntactic processing. Journal of Memory and Language, 25(3):348–368. [find](https://scholar.google.com/scholar?q=Fernanda+Ferreira+and+Charles+Clifton%2E+1986%2E+The+independence+of+syntactic+processing%2E+Journal+of+Memory+and+Language%2C+25%283%29%3A348%E2%80%93368%2E)
20. Stefan L. Frank and Rens Bod. 2011. Insensitivity of the human sentence-processing system to hierarchical structure. Psychological Science, 22(6):829– 834. [find](https://scholar.google.com/scholar?q=Stefan+L%2E+Frank+and+Rens+Bod%2E+2011%2E+Insensitivity+of+the+human+sentence%2Dprocessing+system+to+hierarchical+structure%2E+Psychological+Science%2C+22%286%29%3A829%E2%80%93+834%2E)
21. Stefan L. Frank, Thijs Trompenaars, Richard L. Lewis, and Shravan Vasishth. 2016. Cross-linguistic differences in processing double-embedded relative clauses: Working-memory constraints or language statistics? Cognitive Science, 40:554–578. [find](https://scholar.google.com/scholar?q=Stefan+L%2E+Frank%2C+Thijs+Trompenaars%2C+Richard+L%2E+Lewis%2C+and+Shravan+Vasishth%2E+2016%2E+Cross%2Dlinguistic+differences+in+processing+double%2Dembedded+relative+clauses%3A+Working%2Dmemory+constraints+or+language+st)
22. Richard Futrell and Roger Levy. 2017. Noisy-context surprisal as a human sentence processing cost model. In Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 1, Long Papers, pages 688–698, Valencia, Spain. [find](https://scholar.google.com/scholar?q=Richard+Futrell+and+Roger+Levy%2E+2017%2E+Noisy%2Dcontext+surprisal+as+a+human+sentence+processing+cost+model%2E+In+Proceedings+of+the+15th+Conference+of+the+European+Chapter+of+the+Association+for+Computatio)
23. Edward Gibson and James Thomas. 1999. Memory limitations and structural forgetting: The perception of complex ungrammatical sentences as grammatical. Language and Cognitive Processes, 14(3):225– 248. [find](https://scholar.google.com/scholar?q=Edward+Gibson+and+James+Thomas%2E+1999%2E+Memory+limitations+and+structural+forgetting%3A+The+perception+of+complex+ungrammatical+sentences+as+grammatical%2E+Language+and+Cognitive+Processes%2C+14%283%29%3A225%E2%80%93+248%2E)
24. Yoav Goldberg. 2017. Neural network methods for natural language processing. Synthesis Lectures on Human Language Technologies, 10(1):1–309. [find](https://scholar.google.com/scholar?q=Yoav+Goldberg%2E+2017%2E+Neural+network+methods+for+natural+language+processing%2E+Synthesis+Lectures+on+Human+Language+Technologies%2C+10%281%29%3A1%E2%80%93309%2E)
25. K. Gulordava, P. Bojanowski, E. Grave, T. Linzen, and M. Baroni. 2018. Colorless green recurrent networks dream hierarchically. In Proceedings of NAACL. [our card](https://md.research.elamur.ai/free/en/papers/gulordava-2018-colorless-green-recurrent-networks-dream/)
26. John T. Hale. 2001. A probabilistic Earley parser as a psycholinguistic model. In Proceedings of the Second Meeting of the North American Chapter of the Association for Computational Linguistics and Language Technologies, pages 1–8. [our card](https://md.research.elamur.ai/free/en/papers/hale-2001-probabilistic-earley-parser-as-psycholinguistic/)
27. Rafal Jozefowicz, Oriol Vinyals, Mike Schuster, Noam Shazeer, and Yonghui Wu. 2016. Exploring the limits of language modeling. arXiv, 1602.02410. [find](https://scholar.google.com/scholar?q=Rafal+Jozefowicz%2C+Oriol+Vinyals%2C+Mike+Schuster%2C+Noam+Shazeer%2C+and+Yonghui+Wu%2E+2016%2E+Exploring+the+limits+of+language+modeling%2E+arXiv%2C+1602%2E02410%2E)
28. Ákos Kádár, Grzegorz Chrupała, and Afra Alishahi. 2017. Representation of linguistic form and function in recurrent neural networks. Computational Linguistics, 43(4):761–780. [find](https://scholar.google.com/scholar?q=A%CC%81kos+Ka%CC%81da%CC%81r%2C+Grzegorz+Chrupa%C5%82a%2C+and+Afra+Alishahi%2E+2017%2E+Representation+of+linguistic+form+and+function+in+recurrent+neural+networks%2E+Computational+Linguistics%2C+43%284%29%3A761%E2%80%93780%2E)
29. Kiyoko Kataoka. 2006. Nihongo hitei-bun no koozoo: kakimaze-bun to hitei-koou hyoogen. Kuroshio, Tokyo. [find](https://scholar.google.com/scholar?q=Kiyoko+Kataoka%2E+2006%2E+Nihongo+hitei%2Dbun+no+koozoo%3A+kakimaze%2Dbun+to+hitei%2Dkoou+hyoogen%2E+Kuroshio%2C+Tokyo%2E)
30. Yoon Kim, Yacine Jernite, David Sontag, and Alexander M. Rush. 2016. Character-aware neural language models. In Proceedings of the Thirtieth AAAI Conference on Artificial Intelligence, AAAI’16, pages 2741–2749. AAAI Press. [our card](https://md.research.elamur.ai/free/en/papers/kim-2015-character-aware-neural-language-models/)
31. Brenden M Lake and Marco Baroni. 2017. Still not systematic after all these years: On the compositional skills of sequence-to-sequence recurrent networks. arXiv preprint arXiv:1711.00350. [find](https://scholar.google.com/scholar?q=Brenden+M+Lake+and+Marco+Baroni%2E+2017%2E+Still+not+systematic+after+all+these+years%3A+On+the+compositional+skills+of+sequence%2Dto%2Dsequence+recurrent+networks%2E+arXiv+preprint+arXiv%3A1711%2E00350%2E)
32. Ellen Lau, Clare Stroud, Silke Plesch, and Colin Phillips. 2006. The role of structural prediction in rapid syntactic analysis. Brain & Language, 98:74– 88. [find](https://scholar.google.com/scholar?q=Ellen+Lau%2C+Clare+Stroud%2C+Silke+Plesch%2C+and+Colin+Phillips%2E+2006%2E+The+role+of+structural+prediction+in+rapid+syntactic+analysis%2E+Brain+%26+Language%2C+98%3A74%E2%80%93+88%2E)
33. Roger Levy. 2008. Expectation-based syntactic comprehension. Cognition, 106(3):1126–1177. [find](https://scholar.google.com/scholar?q=Roger+Levy%2E+2008%2E+Expectation%2Dbased+syntactic+comprehension%2E+Cognition%2C+106%283%29%3A1126%E2%80%931177%2E)
34. Roger Levy, Evelina Fedorenko, Mara Breen, and Ted Gibson. 2012. The processing of extraposed structures in English. Cognition, 122(1):12–36. [find](https://scholar.google.com/scholar?q=Roger+Levy%2C+Evelina+Fedorenko%2C+Mara+Breen%2C+and+Ted+Gibson%2E+2012%2E+The+processing+of+extraposed+structures+in+English%2E+Cognition%2C+122%281%29%3A12%E2%80%9336%2E)
35. Tal Linzen, Emmanuel Dupoux, and Yoav Goldberg. 2016. Assessing the ability of LSTMs to learn syntax-sensitive dependencies. Transactions of the Association for Computational Linguistics, 4:521– 535. [our card](https://md.research.elamur.ai/free/en/papers/linzen-2016-assessing-ability-lstms-learn-syntax/)
36. Maryellen C. MacDonald and Morten H. Christiansen. 2002. Reassessing working memory: Comment on Just and Carpenter (1992) and Waters and Caplan (1996). Psychological Review, 109(1):35–54. [find](https://scholar.google.com/scholar?q=Maryellen+C%2E+MacDonald+and+Morten+H%2E+Christiansen%2E+2002%2E+Reassessing+working+memory%3A+Comment+on+Just+and+Carpenter+%281992%29+and+Waters+and+Caplan+%281996%29%2E+Psychological+Review%2C+109%281%29%3A35%E2%80%9354%2E)
37. Maryellen C. MacDonald, Neal J. Pearlmutter, and Mark S. Seidenberg. 1994. The lexical nature of syntactic ambiguity resolution. Psychological Review, 101(4):676. [find](https://scholar.google.com/scholar?q=Maryellen+C%2E+MacDonald%2C+Neal+J%2E+Pearlmutter%2C+and+Mark+S%2E+Seidenberg%2E+1994%2E+The+lexical+nature+of+syntactic+ambiguity+resolution%2E+Psychological+Review%2C+101%284%29%3A676%2E)
38. Michael EJ Masson and Geoffrey R Loftus. 2003. Using confidence intervals for graphically based data interpretation. Canadian Journal of Experimental Psychology/Revue canadienne de psychologie expérimentale, 57(3):203. [find](https://scholar.google.com/scholar?q=Michael+EJ+Masson+and+Geoffrey+R+Loftus%2E+2003%2E+Using+confidence+intervals+for+graphically+based+data+interpretation%2E+Canadian+Journal+of+Experimental+Psychology%2FRevue+canadienne+de+psychologie+expe%CC%81ri)
39. G.A. Miller and N. Chomsky. 1963. Finitary models of language users. Handbook of Mathematical Psychology, 2:419–491. [find](https://scholar.google.com/scholar?q=G%2EA%2E+Miller+and+N%2E+Chomsky%2E+1963%2E+Finitary+models+of+language+users%2E+Handbook+of+Mathematical+Psychology%2C+2%3A419%E2%80%93491%2E)
40. Masatake Muraki. 1978. The sika nai construction and predicate restructuring. In John Hinds and Irwin Howard, editors, Problems in Japanese syntax and semantics, pages 155 – 177. Kaitakusha, Tokyo. [find](https://scholar.google.com/scholar?q=Masatake+Muraki%2E+1978%2E+The+sika+nai+construction+and+predicate+restructuring%2E+In+John+Hinds+and+Irwin+Howard%2C+editors%2C+Problems+in+Japanese+syntax+and+semantics%2C+pages+155+%E2%80%93+177%2E+Kaitakusha%2C+Tokyo%2E)
41. Carl Pollard and Ivan A. Sag. 1994. Head-Driven Phrase Structure Grammar. Center for the Study of Language and Information, Stanford, CA. [find](https://scholar.google.com/scholar?q=Carl+Pollard+and+Ivan+A%2E+Sag%2E+1994%2E+Head%2DDriven+Phrase+Structure+Grammar%2E+Center+for+the+Study+of+Language+and+Information%2C+Stanford%2C+CA%2E)
42. Tanya Reinhart. 1981. Definite NP anaphora and c-command domains. Linguistic Inquiry, 12(4):605– 635. [find](https://scholar.google.com/scholar?q=Tanya+Reinhart%2E+1981%2E+Definite+NP+anaphora+and+c%2Dcommand+domains%2E+Linguistic+Inquiry%2C+12%284%29%3A605%E2%80%93+635%2E)
43. Rachel Rudinger, Jason Naradowsky, Brian Leonard, and Benjamin Van Durme. 2018. Gender bias in coreference resolution. arXiv preprint arXiv:1804.09301. [our card](https://md.research.elamur.ai/free/en/papers/rudinger-2018-gender-bias-coreference-resolution/)
44. Marten van Schijndel and Tal Linzen. 2018. Modeling garden path effects without explicit hierarchical syntax. In Proceedings of the 40th Annual Meeting of the Cognitive Science Society. [find](https://scholar.google.com/scholar?q=Marten+van+Schijndel+and+Tal+Linzen%2E+2018%2E+Modeling+garden+path+effects+without+explicit+hierarchical+syntax%2E+In+Proceedings+of+the+40th+Annual+Meeting+of+the+Cognitive+Science+Society%2E)
45. Xing Shi, Inkit Padhi, and Kevin Knight. 2016. Does string-based neural MT learn source syntax? In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 1526– 1534. [our card](https://md.research.elamur.ai/free/en/papers/shi-2016-does-string-based-neural-mt/)
46. Nathaniel J. Smith and Roger Levy. 2013. The effect of word predictability on reading time is logarithmic. Cognition, 128(3):302–319. [find](https://scholar.google.com/scholar?q=Nathaniel+J%2E+Smith+and+Roger+Levy%2E+2013%2E+The+effect+of+word+predictability+on+reading+time+is+logarithmic%2E+Cognition%2C+128%283%29%3A302%E2%80%93319%2E)
47. Adrian Staub and Charles Clifton. 2006. Syntactic prediction in language comprehension: Evidence from either . . . or. Journal of Experimental Psychology: Learning, Memory, & Cognition, 32(2):425–436. [find](https://scholar.google.com/scholar?q=Adrian+Staub+and+Charles+Clifton%2E+2006%2E+Syntactic+prediction+in+language+comprehension%3A+Evidence+from+either+%2E+%2E+%2E+or%2E+Journal+of+Experimental+Psychology%3A+Learning%2C+Memory%2C+%26+Cognition%2C+32%282%29%3A425%E2%80%93436%2E)
48. Patrick Sturt. 2003. The time-course of the application of binding constraints in reference resolution. Journal of Memory and Language, 48:542–562. [find](https://scholar.google.com/scholar?q=Patrick+Sturt%2E+2003%2E+The+time%2Dcourse+of+the+application+of+binding+constraints+in+reference+resolution%2E+Journal+of+Memory+and+Language%2C+48%3A542%E2%80%93562%2E)
49. Ilya Sutskever, Oriol Vinyals, and Quoc V Le. 2014. Sequence to sequence learning with neural networks. In Advances in Neural Information Processing Systems, pages 3104–3112. [find](https://scholar.google.com/scholar?q=Ilya+Sutskever%2C+Oriol+Vinyals%2C+and+Quoc+V+Le%2E+2014%2E+Sequence+to+sequence+learning+with+neural+networks%2E+In+Advances+in+Neural+Information+Processing+Systems%2C+pages+3104%E2%80%933112%2E)
50. Hidekazu Tanaka. 1997. Invisible movement in sika-nai and the linear crossing constraint. Journal of East Asian Linguistics, 6(2):143–188. [find](https://scholar.google.com/scholar?q=Hidekazu+Tanaka%2E+1997%2E+Invisible+movement+in+sika%2Dnai+and+the+linear+crossing+constraint%2E+Journal+of+East+Asian+Linguistics%2C+6%282%29%3A143%E2%80%93188%2E)
51. John C. Trueswell, Michael K. Tanenhaus, and S. Garnsey. 1994. Semantic influences on parsing: Use of thematic role information in syntactic ambiguity resolution. Journal of Memory and Language, 33:285– 318. [find](https://scholar.google.com/scholar?q=John+C%2E+Trueswell%2C+Michael+K%2E+Tanenhaus%2C+and+S%2E+Garnsey%2E+1994%2E+Semantic+influences+on+parsing%3A+Use+of+thematic+role+information+in+syntactic+ambiguity+resolution%2E+Journal+of+Memory+and+Language%2C+33%3A28)
52. Shravan Vasishth, Sven Brüssow, Richard L Lewis, and Heiner Drenhaus. 2008. Processing polarity: How the ungrammatical intrudes on the grammatical. Cognitive Science, 32(4):685–712. [find](https://scholar.google.com/scholar?q=Shravan+Vasishth%2C+Sven+Bru%CC%88ssow%2C+Richard+L+Lewis%2C+and+Heiner+Drenhaus%2E+2008%2E+Processing+polarity%3A+How+the+ungrammatical+intrudes+on+the+grammatical%2E+Cognitive+Science%2C+32%284%29%3A685%E2%80%93712%2E)
53. Shravan Vasishth, Katja Suckow, Richard L Lewis, and Sabine Kern. 2010. Short-term forgetting in sentence comprehension: Crosslinguistic evidence from verb-final structures. Language and Cognitive Processes, 25(4):533–567. [find](https://scholar.google.com/scholar?q=Shravan+Vasishth%2C+Katja+Suckow%2C+Richard+L+Lewis%2C+and+Sabine+Kern%2E+2010%2E+Short%2Dterm+forgetting+in+sentence+comprehension%3A+Crosslinguistic+evidence+from+verb%2Dfinal+structures%2E+Language+and+Cognitive+Pro)
54. Noah Weber, Leena Shekhar, and Niranjan Balasubramanian. 2018. The fine line between linguistic generalization and failure in seq2seq-attention models. In Workshop on New Forms of Generalization in Deep Learning and NLP (NAACL 2018). [find](https://scholar.google.com/scholar?q=Noah+Weber%2C+Leena+Shekhar%2C+and+Niranjan+Balasubramanian%2E+2018%2E+The+fine+line+between+linguistic+generalization+and+failure+in+seq2seq%2Dattention+models%2E+In+Workshop+on+New+Forms+of+Generalization+in+De)
55. Ethan G. Wilcox, Roger P. Levy, Takashi Morita, and Richard Futrell. 2018. What do RNNs learn about filler–gap dependencies? In Proceedings of BlackboxNLP 2018. [find](https://scholar.google.com/scholar?q=Ethan+G%2E+Wilcox%2C+Roger+P%2E+Levy%2C+Takashi+Morita%2C+and+Richard+Futrell%2E+2018%2E+What+do+RNNs+learn+about+filler%E2%80%93gap+dependencies%3F+In+Proceedings+of+BlackboxNLP+2018%2E)
56. Adina Williams, Andrew Drozdov, and Samuel R Bowman. 2018. Do latent tree learning models identify meaningful structure in sentences? Transactions of the Association for Computational Linguistics, 6:253–267. [find](https://scholar.google.com/scholar?q=Adina+Williams%2C+Andrew+Drozdov%2C+and+Samuel+R+Bowman%2E+2018%2E+Do+latent+tree+learning+models+identify+meaningful+structure+in+sentences%3F+Transactions+of+the+Association+for+Computational+Linguistics%2C+6%3A2)
57. Ming Xiang, Brian Dillon, and Colin Phillips. 2009. Illusory licensing effects across dependency types: ERP evidence. Brain & Language, 108(1):40–55. [find](https://scholar.google.com/scholar?q=Ming+Xiang%2C+Brian+Dillon%2C+and+Colin+Phillips%2E+2009%2E+Illusory+licensing+effects+across+dependency+types%3A+ERP+evidence%2E+Brain+%26+Language%2C+108%281%29%3A40%E2%80%9355%2E)
58. Victor H. Yngve. 1960. A model and an hypothesis for language structure. Proceedings of the American Philosophical Society, 104(5):444–466. [find](https://scholar.google.com/scholar?q=Victor+H%2E+Yngve%2E+1960%2E+A+model+and+an+hypothesis+for+language+structure%2E+Proceedings+of+the+American+Philosophical+Society%2C+104%285%29%3A444%E2%80%93466%2E)