---
title: "A Transparent Model of Syntactic and Semantic Cue-based Retrieval — Yue 2026"
description: "Yue and Hale propose a small, explicitly parameterized alternative. 30 references."
authors: ["Shisen Yue", "John T. Hale"]
year: 2026
venue: "Proceedings of the Society for Computation in Linguistics 2026"
source: "https://aclanthology.org/2026.scil-main.38/"
doi: "10.18653/v1/2026.scil-main.38"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/yue-2026-transparent-model-syntactic-semantic-cue/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/yue-2026-transparent-model-syntactic-semantic-cue/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/4/) / A Transparent Model of Syntactic and Semantic Cue-based Retrieval

# A Transparent Model of Syntactic and Semantic Cue-based Retrieval

**Authors:** [Shisen Yue](https://md.research.elamur.ai/free/en/authors/shisen-yue/) and [John T. Hale](https://md.research.elamur.ai/free/en/authors/john-hale/) · **Year:** 2026 · **Venue:** Proceedings of the Society for Computation in Linguistics 2026 · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2026.scil-main.38/) · [PDF](https://aclanthology.org/2026.scil-main.38.pdf) · [DOI 10.18653/v1/2026.scil-main.38](https://doi.org/10.18653/v1/2026.scil-main.38) · [OpenAlex](https://openalex.org/W7166816337)

**Citations:** 0 citations (OpenAlex, 2026-10-14)

## Summary
When readers link a verb to its subject, they struggle more if another noun in the sentence resembles the true subject. Cue-based retrieval theory explains this interference. Recent work has instead used Transformer attention heads to model it, but those signals are hard to tie to specific linguistic operations.

Yue and Hale propose a small, explicitly parameterized alternative. Each dependency arc is treated as a retrieval event, focusing on nsubj. Two separate components score candidate targets in the preceding context:
- **Grammar-bilinear model:** a bilinear syntactic scorer over CCG supertags, trained on the English Web Treebank with Universal Dependencies relations.
- **Asymw2v:** an asymmetric semantic scorer built on dependency-based embeddings, with distinct cue and target spaces.

A linear mix of the two reproduces the graded interference across the 2 × 2 conditions of Van Dyke (2007). The combined model also beats GPT-2 attention heads claimed to specialize in subjecthood, both in retrieval accuracy and in fit to human difficulty. The work suggests that syntactic and semantic interference can be separated and inspected directly, without relying on opaque neural components.

## Key points
- **Syntactic input:** a cue's context is the concatenation of one-hot CCG supertags for the current word and its two predecessors. A trainable matrix W of size 3V × 3V scores candidates, and a softmax over all preceding words normalizes the scores.
- **Syntactic training data:** supertags come from ccg-tools, and gold nsubj targets come from UD annotations of the English Web Treebank.
- **Asymw2v:** adapts Levy and Goldberg (2014) dependency embeddings, pairing words with relation-labeled contexts such as coach/nsubj. It learns separate input (cue) and output (role-bound target) matrices to capture selectional preferences.
- **GPT-2-small comparison:** head4_3, proposed by Ryu and Lewis (2021) as specialized for nsubj, reacts to syntactic interference but barely to semantic interference. Head3_6 shows interference-like patterns, yet its retrieval accuracy is near zero, and it mostly attends to the auxiliary before the verb.
- **Evaluation materials:** Van Dyke (2007) items crossing syntactic similarity with semantic plausibility, plus materials from Tan et al. (2017).

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

Fields: [Linguistics & languages](https://md.research.elamur.ai/free/en/shelves/linguistics/)

## Institutions

- [Johns Hopkins University](https://md.research.elamur.ai/free/en/universities/johns-hopkins-university/) — University, Baltimore, United States · 151 works · 84,111 citations

## Cite

Cite as: Shisen Yue and John T. Hale. 2026. A Transparent Model of Syntactic and Semantic Cue-based Retrieval. Proceedings of the Society for Computation in Linguistics 2026.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{yue2026transparent,
  title = {{A Transparent Model of Syntactic and Semantic Cue-based Retrieval}},
  author = {Shisen Yue and John T. Hale},
  booktitle = {Proceedings of the Society for Computation in Linguistics 2026},
  year = {2026},
  doi = {10.18653/v1/2026.scil-main.38},
  url = {https://doi.org/10.18653/v1/2026.scil-main.38}
}
```

- [← previous A Teacher-Student Approach to Creating Verified Synthetic Clarification and Correction Dialogues for TableQA Tasks](https://md.research.elamur.ai/free/en/papers/poelitz-2026-teacher-student-approach-creating-verified/)
- [next → A Zipfian Analysis of Visual Token Distributions for AI-Generated Images](https://md.research.elamur.ai/free/en/papers/shin-2026-zipfian-analysis-visual-token-distributions/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/4/)

## References

30 references, 4 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Suhas Arehalli, Brian Dillon, and Tal Linzen. 2022. Syntactic surprisal from neural models predicts, but underestimates, human processing difficulty from syntactic ambiguities. In Proceedings of the 26th Conference on Computational Natural Language Learning (CoNLL), pages 301–313, Abu Dhabi, United… [find](https://scholar.google.com/scholar?q=Suhas+Arehalli%2C+Brian+Dillon%2C+and+Tal+Linzen%2E+2022%2E+Syntactic+surprisal+from+neural+models+predicts%2C+but+underestimates%2C+human+processing+difficulty+from+syntactic+ambiguities%2E+In+Proceedings+of+the+2)
2. Srinivas Bangalore and Aravind K. Joshi. 1999. Supertagging: An approach to almost parsing. Computational Linguistics, 25(2):237–265. [our card](https://md.research.elamur.ai/free/en/papers/bangalore-1999-supertagging-approach-almost-parsing/)
3. Charlotte Caucheteux, Alexandre Gramfort, and Jean-Rémi King. 2022. Deep language algorithms predict semantic comprehension from brain activity. Scientific Reports, 12(1):16327. [find](https://scholar.google.com/scholar?q=Charlotte+Caucheteux%2C+Alexandre+Gramfort%2C+and+Jean%2DR%C3%A9mi+King%2E+2022%2E+Deep+language+algorithms+predict+semantic+comprehension+from+brain+activity%2E+Scientific+Reports%2C+12%281%29%3A16327%2E)
4. Omer Levy and Yoav Goldberg. 2014. Dependency-based word embeddings. In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 302–308, Baltimore, Maryland. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Omer+Levy+and+Yoav+Goldberg%2E+2014%2E+Dependency%2Dbased+word+embeddings%2E+In+Proceedings+of+the+52nd+Annual+Meeting+of+the+Association+for+Computational+Linguistics+%28Volume+2%3A+Short+Papers%29%2C+pages+302%E2%80%93308%2C)
5. Richard L Lewis and Shravan Vasishth. 2005. An activation-based model of sentence processing as skilled memory retrieval. Cognitive science, 29(3):375–419. [find](https://scholar.google.com/scholar?q=Richard+L+Lewis+and+Shravan+Vasishth%2E+2005%2E+An+activation%2Dbased+model+of+sentence+processing+as+skilled+memory+retrieval%2E+Cognitive+science%2C+29%283%29%3A375%E2%80%93419%2E)
6. Richard L. Lewis, Shravan Vasishth, and Julie A. Van Dyke. 2006. Computational principles of working memory in sentence comprehension. Trends in Cognitive Sciences, 10(10):447–454. [find](https://scholar.google.com/scholar?q=Richard+L%2E+Lewis%2C+Shravan+Vasishth%2C+and+Julie+A%2E+Van+Dyke%2E+2006%2E+Computational+principles+of+working+memory+in+sentence+comprehension%2E+Trends+in+Cognitive+Sciences%2C+10%2810%29%3A447%E2%80%93454%2E)
7. Bofang Li, Tao Liu, Zhe Zhao, Buzhou Tang, Aleksandr Drozd, Anna Rogers, and Xiaoyong Du. 2017. Investigating different syntactic context types and context representations for learning word embeddings. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 2… [find](https://scholar.google.com/scholar?q=Bofang+Li%2C+Tao+Liu%2C+Zhe+Zhao%2C+Buzhou+Tang%2C+Aleksandr+Drozd%2C+Anna+Rogers%2C+and+Xiaoyong+Du%2E+2017%2E+Investigating+different+syntactic+context+types+and+context+representations+for+learning+word+embeddings)
8. Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean. 2013. Efficient estimation of word representations in vector space. arXiv preprint arXiv:1301.3781. [our card](https://md.research.elamur.ai/free/en/papers/mikolov-2013-efficient-estimation-word-representations-vector/)
9. Klaus Oberauer and Stephan Lewandowsky. 2008. Forgetting in immediate serial recall: Decay, temporal distinctiveness, or interference? Psychological review, 115(3):544. [find](https://scholar.google.com/scholar?q=Klaus+Oberauer+and+Stephan+Lewandowsky%2E+2008%2E+Forgetting+in+immediate+serial+recall%3A+Decay%2C+temporal+distinctiveness%2C+or+interference%3F+Psychological+review%2C+115%283%29%3A544%2E)
10. Natalia Silveira, Timothy Dozat, Marie-Catherine de Marneffe, Samuel Bowman, Miriam Connor, John Bauer, and Christopher D. Manning. 2014. A gold standard dependency corpus for English. In Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC-2014). [find](https://scholar.google.com/scholar?q=Natalia+Silveira%2C+Timothy+Dozat%2C+Marie%2DCatherine+de+Marneffe%2C+Samuel+Bowman%2C+Miriam+Connor%2C+John+Bauer%2C+and+Christopher+D%2E+Manning%2E+2014%2E+A+gold+standard+dependency+corpus+for+English%2E+In+Proceedings+)
11. Miloš Stanojević, Jonathan R Brennan, Donald Dunagan, Mark Steedman, and John T Hale. 2023. Modeling structure-building in the brain with ccg parsing and large language models. Cognitive science, 47(7):e13312. [find](https://scholar.google.com/scholar?q=Milo%C5%A1+Stanojevic%CC%81%2C+Jonathan+R+Brennan%2C+Donald+Dunagan%2C+Mark+Steedman%2C+and+John+T+Hale%2E+2023%2E+Modeling+structure%2Dbuilding+in+the+brain+with+ccg+parsing+and+large+language+models%2E+Cognitive+science%2C+47%28)
12. Byung-Doh Oh and William Schuler. 2022. Entropy-and distance-based predictors from GPT-2 attention patterns predict reading times over and above GPT-2 surprisal. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 9324–9334, Abu Dhabi, United Arab Emirate… [find](https://scholar.google.com/scholar?q=Byung%2DDoh+Oh+and+William+Schuler%2E+2022%2E+Entropy%2Dand+distance%2Dbased+predictors+from+GPT%2D2+attention+patterns+predict+reading+times+over+and+above+GPT%2D2+surprisal%2E+In+Proceedings+of+the+2022+Conference+)
13. Mark Steedman. 2001. The syntactic process. MIT press. [find](https://scholar.google.com/scholar?q=Mark+Steedman%2E+2001%2E+The+syntactic+process%2E+MIT+press%2E)
14. Byung-Doh Oh and William Schuler. 2023a. Transformer-based language model surprisal predicts human reading times best with about two billion training tokens. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 1915–1921, Singapore. Association for Computational Linguistic… [find](https://scholar.google.com/scholar?q=Byung%2DDoh+Oh+and+William+Schuler%2E+2023a%2E+Transformer%2Dbased+language+model+surprisal+predicts+human+reading+times+best+with+about+two+billion+training+tokens%2E+In+Findings+of+the+Association+for+Computa)
15. Julius Steuer, Marius Mosbach, and Dietrich Klakow. 2023. Large GPT-like models are bad babies: A closer look at the relationship between linguistic competence and psycholinguistic measures. In Proceedings of the BabyLM Challenge at the 27th Conference on Computational Natural Language Learning, pag… [find](https://scholar.google.com/scholar?q=Julius+Steuer%2C+Marius+Mosbach%2C+and+Dietrich+Klakow%2E+2023%2E+Large+GPT%2Dlike+models+are+bad+babies%3A+A+closer+look+at+the+relationship+between+linguistic+competence+and+psycholinguistic+measures%2E+In+Procee)
16. Byung-Doh Oh and William Schuler. 2023b. Why does surprisal from larger transformer-based language models provide a poorer fit to human reading times? Transactions of the Association for Computational Linguistics, 11:336–350. [our card](https://md.research.elamur.ai/free/en/papers/oh-2023-why-does-surprisal-larger-transformer/)
17. Yingying Tan, Randi C Martin, and Julie A Van Dyke. 2017. Semantic and syntactic interference in sentence comprehension: A comparison of working memory models. Frontiers in psychology, 8:198. [find](https://scholar.google.com/scholar?q=Yingying+Tan%2C+Randi+C+Martin%2C+and+Julie+A+Van+Dyke%2E+2017%2E+Semantic+and+syntactic+interference+in+sentence+comprehension%3A+A+comparison+of+working+memory+models%2E+Frontiers+in+psychology%2C+8%3A198%2E)
18. Dan Parker, Michael Shvartsman, and Julie A. Van Dyke. 2017. The cue-based retrieval theory of sentence comprehension: New findings and new challenges. In Linda Escobar, Vicenç Torrens, and Teresa Parodi, editors, Language Processing and Disorders, pages 121–144. Cambridge Scholars Publishing, Newca… [find](https://scholar.google.com/scholar?q=Dan+Parker%2C+Michael+Shvartsman%2C+and+Julie+A%2E+Van+Dyke%2E+2017%2E+The+cue%2Dbased+retrieval+theory+of+sentence+comprehension%3A+New+findings+and+new+challenges%2E+In+Linda+Escobar%2C+Vicen%C3%A7+Torrens%2C+and+Teresa+Par)
19. Soo Hyun Ryu and Richard Lewis. 2021. Accounting for agreement phenomena in sentence comprehension with transformer language models: Effects of similarity-based interference on surprisal and attention. In Proceedings of the Workshop on Cognitive Modeling and Computational Linguistics, pages 61– 71,… [find](https://scholar.google.com/scholar?q=Soo+Hyun+Ryu+and+Richard+Lewis%2E+2021%2E+Accounting+for+agreement+phenomena+in+sentence+comprehension+with+transformer+language+models%3A+Effects+of+similarity%2Dbased+interference+on+surprisal+and+attention)
20. Soo Hyun Ryu and Richard L Lewis. 2025. Memory for prediction: A transformer-based theory of sentence processing. Journal of Memory and Language, 145:104670. [find](https://scholar.google.com/scholar?q=Soo+Hyun+Ryu+and+Richard+L+Lewis%2E+2025%2E+Memory+for+prediction%3A+A+transformer%2Dbased+theory+of+sentence+processing%2E+Journal+of+Memory+and+Language%2C+145%3A104670%2E)
21. Ferdinand de Saussure. 1916. Rapports syntagmatiques et rapports associatifs. Cours de linguistique générale. Payot, Paris, France, pages 170–175. [find](https://scholar.google.com/scholar?q=Ferdinand+de+Saussure%2E+1916%2E+Rapports+syntagmatiques+et+rapports+associatifs%2E+Cours+de+linguistique+g%C3%A9n%C3%A9rale%2E+Payot%2C+Paris%2C+France%2C+pages+170%E2%80%93175%2E)
22. Cory Shain. 2024. Word frequency and predictability dissociate in naturalistic reading. Open Mind, 8:177– 201. [find](https://scholar.google.com/scholar?q=Cory+Shain%2E+2024%2E+Word+frequency+and+predictability+dissociate+in+naturalistic+reading%2E+Open+Mind%2C+8%3A177%E2%80%93+201%2E)
23. Mark Steedman and Jason Baldridge. 2011. Combinatory categorial grammar. Non-transformational syntax: Formal and explicit models of grammar, pages 181–224. [find](https://scholar.google.com/scholar?q=Mark+Steedman+and+Jason+Baldridge%2E+2011%2E+Combinatory+categorial+grammar%2E+Non%2Dtransformational+syntax%3A+Formal+and+explicit+models+of+grammar%2C+pages+181%E2%80%93224%2E)
24. William Timkey and Tal Linzen. 2023. A language model with limited memory capacity captures interference in human sentence processing. arXiv preprint arXiv:2310.16142. [find](https://scholar.google.com/scholar?q=William+Timkey+and+Tal+Linzen%2E+2023%2E+A+language+model+with+limited+memory+capacity+captures+interference+in+human+sentence+processing%2E+arXiv+preprint+arXiv%3A2310%2E16142%2E)
25. Julie A Van Dyke. 2007. Interference effects from grammatically unavailable constituents during sentence processing. Journal of Experimental Psychology: Learning, Memory, and Cognition, 33(2):407. [find](https://scholar.google.com/scholar?q=Julie+A+Van+Dyke%2E+2007%2E+Interference+effects+from+grammatically+unavailable+constituents+during+sentence+processing%2E+Journal+of+Experimental+Psychology%3A+Learning%2C+Memory%2C+and+Cognition%2C+33%282%29%3A407%2E)
26. Julie A Van Dyke and Richard L Lewis. 2003. Distinguishing effects of structure and decay on attachment and repair: A cue-based parsing account of recovery from misanalyzed ambiguities. Journal of Memory and Language, 49(3):285–316. [find](https://scholar.google.com/scholar?q=Julie+A+Van+Dyke+and+Richard+L+Lewis%2E+2003%2E+Distinguishing+effects+of+structure+and+decay+on+attachment+and+repair%3A+A+cue%2Dbased+parsing+account+of+recovery+from+misanalyzed+ambiguities%2E+Journal+of+Mem)
27. Julie A. Van Dyke and Brian McElree. 2006. Retrieval interference in sentence comprehension. Journal of Memory and Language, 55(2):157–166. [find](https://scholar.google.com/scholar?q=Julie+A%2E+Van+Dyke+and+Brian+McElree%2E+2006%2E+Retrieval+interference+in+sentence+comprehension%2E+Journal+of+Memory+and+Language%2C+55%282%29%3A157%E2%80%93166%2E)
28. Julie A. Van Dyke and Brian McElree. 2011. Cue-dependent interference in comprehension. Journal of Memory and Language, 65(3):247–263. [find](https://scholar.google.com/scholar?q=Julie+A%2E+Van+Dyke+and+Brian+McElree%2E+2011%2E+Cue%2Ddependent+interference+in+comprehension%2E+Journal+of+Memory+and+Language%2C+65%283%29%3A247%E2%80%93263%2E)
29. Shravan Vasishth and Felix Engelmann. 2021. Sentence comprehension as a cognitive process: A computational approach. Cambridge University Press. [find](https://scholar.google.com/scholar?q=Shravan+Vasishth+and+Felix+Engelmann%2E+2021%2E+Sentence+comprehension+as+a+cognitive+process%3A+A+computational+approach%2E+Cambridge+University+Press%2E)
30. Elena Voita, David Talbot, Fedor Moiseev, Rico Sennrich, and Ivan Titov. 2019. Analyzing multi-head self-attention: Specialized heads do the heavy lifting, the rest can be pruned. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 5797–5808, Florence, I… [our card](https://md.research.elamur.ai/free/en/papers/voita-2019-analyzing-multi-head-self-attention/)