---
title: "Language models show human-like content effects on reasoning"
description: "Ideally, abstract reasoning should not depend on what a problem is about. 120 references, cited by 112 papers here."
authors: ["Ishita Dasgupta", "Andrew K. Lampinen", "Stephanie C. Y. Chan", "Hannah R. Sheahan", "Antonia Creswell", "Dharshan Kumaran", "James L. McClelland", "Felix Hill"]
year: 2022
venue: "arXiv:2207.07051"
source: "https://arxiv.org/abs/2207.07051"
doi: "10.48550/arxiv.2207.07051"
arxiv: "2207.07051"
licence: "arXiv — non-exclusive licence to distribute"
licence_url: "http://arxiv.org/licenses/nonexclusive-distrib/1.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/dasgupta-2022-language-models-show-human-like/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/dasgupta-2022-language-models-show-human-like/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2022](https://md.research.elamur.ai/free/en/papers/2022/page/3/) / Language models show human-like content effects on reasoning tasks

# Language models show human-like content effects on reasoning tasks

**Authors:** [Ishita Dasgupta](https://md.research.elamur.ai/free/en/authors/ishita-dasgupta/), [Andrew K. Lampinen](https://md.research.elamur.ai/free/en/authors/andrew-lampinen/), [Stephanie C. Y. Chan](https://md.research.elamur.ai/free/en/authors/stephanie-chan/), Hannah R. Sheahan, [Antonia Creswell](https://md.research.elamur.ai/free/en/authors/antonia-creswell/), Dharshan Kumaran, [James L. McClelland](https://md.research.elamur.ai/free/en/authors/james-l-mcclelland/) and [Felix Hill](https://md.research.elamur.ai/free/en/authors/felix-hill/) · **Year:** 2022 · **Venue:** arXiv:2207.07051 · **Licence:** [arXiv — non-exclusive licence to distribute](http://arxiv.org/licenses/nonexclusive-distrib/1.0/)

**Links:** [Publisher's page](https://arxiv.org/abs/2207.07051) · [PDF](https://arxiv.org/pdf/2207.07051) · [DOI 10.48550/arxiv.2207.07051](https://doi.org/10.48550/arxiv.2207.07051) · [arXiv 2207.07051](https://arxiv.org/abs/2207.07051) · [OpenAlex](https://openalex.org/W4285594979)

**Citations:** 50 citations (OpenAlex, 2026-10-14) · [112](https://md.research.elamur.ai/free/en/papers/dasgupta-2022-language-models-show-human-like/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary

Ideally, abstract reasoning should not depend on what a problem is about. Yet people reason better when a problem's meaning agrees with the logically correct answer. Psychologists call these "content effects." This paper asks whether large language models, whose expectations come from human-written text, blend meaning and logic in the same way.

To avoid training-data contamination, the authors built fresh stimuli for three tasks:
- a forced-choice Natural Language Inference task about comparisons
- judging whether syllogisms are valid
- the Wason selection task

Each task has belief-consistent or realistic versions, belief-violating or arbitrary versions, and versions using nonsense words. Several LMs were scored by answer likelihood. Online human participants solved the same problems with matching instructions.

Both groups were more accurate when content supported the right inference. The effect was modest on NLI and stronger on syllogisms and Wason. Model answer distributions also tracked human response times. The results suggest such biases can emerge from learning on naturalistic data, which matters for cognitive theory and for interpreting LM performance.

## Key points

- Models tested were Chinchilla (70B), PaLM 2-M and PaLM 2-L, Flan-PaLM 2, and GPT-3.5-turbo-instruct. Content effects were similar across base and instruction-tuned models.
- Syllogism and Wason answers were scored with the DC-PMI correction (Holtzman et al.). NLI used plain maximum likelihood, because DC-PMI would cancel the bias being measured. The direction of the effects mostly held under alternative scoring.
- The human data were collected in 2023 from native English speakers in the UK. The first sample had 625 people. A second sample of 360 people focused on Wason only and offered a £0.50 accuracy bonus because Wason accuracy was low.
- Nonsense-word controls (e.g., "vuffs," "feps") remove semantic content entirely. The analyses used mixed-effects logistic regression with random effects for items and models.
- For library citation: Andrew K. Lampinen and Ishita Dasgupta are equal-contribution first authors (Google DeepMind). Lampinen is listed first in arXiv:2207.07051.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 18,614 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 17,965 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 17,450 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 17,296 works

🧭 Topics (2): [🧠 Language models & pretraining](https://md.research.elamur.ai/free/en/topics/language-models/) · 3,894 works · [🧮 Reasoning](https://md.research.elamur.ai/free/en/topics/reasoning/) · 1,685 works

## Cite

Cite as: Ishita Dasgupta, Andrew K. Lampinen, Stephanie C. Y. Chan, Hannah R. Sheahan, Antonia Creswell, Dharshan Kumaran, James L. McClelland and Felix Hill. 2022. Language models show human-like content effects on reasoning tasks. arXiv:2207.07051.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@misc{dasgupta2022language,
  title = {{Language models show human-like content effects on reasoning tasks}},
  author = {Ishita Dasgupta, Andrew K. Lampinen, Stephanie C. Y. Chan, Hannah R. Sheahan, Antonia Creswell, Dharshan Kumaran, James L. McClelland and Felix Hill},
  year = {2022},
  doi = {10.48550/arxiv.2207.07051},
  eprint = {2207.07051},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2207.07051}
}
```

- [← previous Chart-to-Text: A Large-Scale Benchmark for Chart Summarization](https://md.research.elamur.ai/free/en/papers/kantharaj-2022-chart-text-large-scale-benchmark/)
- [next → Reasoning with Language Model Prompting: A Survey](https://md.research.elamur.ai/free/en/papers/qiao-2022-reasoning-language-model-prompting-survey/)
- [↑ back to papers of 2022](https://md.research.elamur.ai/free/en/papers/2022/page/3/)

## References

120 references, 25 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. P. C. Wason, Reasoning about a rule. Quarterly Journal of Experimental Psychology 20, 273 - 281 (1968). [find](https://scholar.google.com/scholar?q=P%2E+C%2E+Wason%2C+Reasoning+about+a+rule%2E+Quarterly+Journal+of+Experimental+Psychology+20%2C+273+%2D+281+%281968%29%2E)
2. A. Newell, Physical symbol systems. Cognitive science 4, 135–183 (1980). [find](https://scholar.google.com/scholar?q=A%2E+Newell%2C+Physical+symbol+systems%2E+Cognitive+science+4%2C+135%E2%80%93183+%281980%29%2E)
3. J. A. Fodor, Z. W. Pylyshyn, Connectionism and cognitive architecture: A critical analysis. Cognition 28, 3–71 (1988). [find](https://scholar.google.com/scholar?q=J%2E+A%2E+Fodor%2C+Z%2E+W%2E+Pylyshyn%2C+Connectionism+and+cognitive+architecture%3A+A+critical+analysis%2E+Cognition+28%2C+3%E2%80%9371+%281988%29%2E)
4. G. Marcus, [The next decade in ai: four steps towards robust artificial intelligence](https://md.research.elamur.ai/free/en/papers/marcus-2020-next-decade-ai-four/). arXiv preprint arXiv:2002.06177 (2020).
5. M. Mitchell, [Abstraction and analogy-making in artificial intelligence](https://md.research.elamur.ai/free/en/papers/mitchell-2021-abstraction-analogy-making-artificial-intelligence/). Annals of the New York Academy of Sciences 1505, 79–101 (2021).
6. J. Russin, R. C. O’Reilly, Y. Bengio, Deep learning needs a prefrontal cortex. Work Bridging AI Cogn Sci 107, 603–616 (2020). [find](https://scholar.google.com/scholar?q=J%2E+Russin%2C+R%2E+C%2E+O%E2%80%99Reilly%2C+Y%2E+Bengio%2C+Deep+learning+needs+a+prefrontal+cortex%2E+Work+Bridging+AI+Cogn+Sci+107%2C+603%E2%80%93616+%282020%29%2E)
7. T. Brown, B. Mann, N. Ryder, M. Subbiah, J. D. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell, et al., [Language models are few-shot learners](https://md.research.elamur.ai/free/en/papers/brown-2020-language-models-few-shot-learners/). Advances in neural information processing systems 33, 1877–1901 (2020).
8. D. Ganguli, D. Hernandez, L. Lovitt, N. DasSarma, T. Henighan, A. Jones, N. Joseph, J. Kernion, B. Mann, A. Askell, et al., [Predictability and surprise in large generative models](https://md.research.elamur.ai/free/en/papers/ganguli-2022-predictability-surprise-large-generative-models/). arXiv preprint arXiv:2202.07785 (2022).
9. M. Nye, A. J. Andreassen, G. Gur-Ari, H. Michalewski, J. Austin, D. Bieber, D. Dohan, A. Lewkowycz, M. Bosma, D. Luan, et al., [Show your work: Scratchpads for intermediate computation with language models](https://md.research.elamur.ai/free/en/papers/al-2021-show-your-work-scratchpads-intermediate/). arXiv preprint arXiv:2112.00114 (2021).
10. T. Kojima, S. S. Gu, M. Reid, Y. Matsuo, Y. Iwasawa, [Large language models are zero-shot reasoners](https://md.research.elamur.ai/free/en/papers/kojima-2022-large-language-models-zero-shot/). arXiv preprint arXiv:2205.11916 (2022).
11. J. Wei, Y. Tay, R. Bommasani, C. Raffel, B. Zoph, S. Borgeaud, D. Yogatama, M. Bosma, D. Zhou, D. Metzler, et al., [Emergent abilities of large language models](https://md.research.elamur.ai/free/en/papers/wei-2022-emergent-abilities-large-language-models/). arXiv preprint arXiv:2206.07682 (2022).
12. R. Schaeffer, B. Miranda, S. Koyejo, [Are emergent abilities of large language models a mirage?](https://md.research.elamur.ai/free/en/papers/schaeffer-2023-emergent-abilities-large-language-models/) arXiv preprint arXiv:2304.15004 (2023).
13. J. W. Rae, S. Borgeaud, T. Cai, K. Millican, J. Hoffmann, F. Song, J. Aslanides, S. Henderson, R. Ring, S. Young, et al., Scaling language models: Methods, analysis & insights from training gopher. arXiv preprint arXiv:2112.11446 (2021). [find](https://scholar.google.com/scholar?q=J%2E+W%2E+Rae%2C+S%2E+Borgeaud%2C+T%2E+Cai%2C+K%2E+Millican%2C+J%2E+Hoffmann%2C+F%2E+Song%2C+J%2E+Aslanides%2C+S%2E+Henderson%2C+R%2E+Ring%2C+S%2E+Young%2C+et+al%2E%2C+Scaling+language+models%3A+Methods%2C+analysis+%26+insights+from+training+gopher%2E+ar)
14. Y. Razeghi, R. L. Logan IV, M. Gardner, S. Singh, Impact of pretraining term frequencies on few-shot reasoning. arXiv preprint arXiv:2202.07206 (2022). — [arXiv:2202.07206](https://arxiv.org/abs/2202.07206) [find](https://scholar.google.com/scholar?q=Y%2E+Razeghi%2C+R%2E+L%2E+Logan+IV%2C+M%2E+Gardner%2C+S%2E+Singh%2C+Impact+of+pretraining+term+frequencies+on+few%2Dshot+reasoning%2E+arXiv+preprint+arXiv%3A2202%2E07206+%282022%29%2E+%E2%80%94+%5BarXiv%3A2202%2E07206%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2202%2E)
15. A. Patel, S. Bhattamishra, N. Goyal, [Are nlp models really able to solve simple math word problems?](https://md.research.elamur.ai/free/en/papers/patel-2021-are-nlp-models-really-able/) arXiv preprint arXiv:2103.07191 (2021).
16. K. Valmeekam, A. Olmo, S. Sreedharan, S. Kambhampati, Large language models still can’t plan (a benchmark for llms on planning and reasoning about change) (2022). [find](https://scholar.google.com/scholar?q=K%2E+Valmeekam%2C+A%2E+Olmo%2C+S%2E+Sreedharan%2C+S%2E+Kambhampati%2C+Large+language+models+still+can%E2%80%99t+plan+%28a+benchmark+for+llms+on+planning+and+reasoning+about+change%29+%282022%29%2E)
17. G. Gigerenzer, W. Gaissmaier, Heuristic decision making. Annual review of psychology 62, 451–482 (2011). [find](https://scholar.google.com/scholar?q=G%2E+Gigerenzer%2C+W%2E+Gaissmaier%2C+Heuristic+decision+making%2E+Annual+review+of+psychology+62%2C+451%E2%80%93482+%282011%29%2E)
18. D. Kahneman, S. P. Slovic, P. Slovic, A. Tversky, Judgment under uncertainty: Heuristics and biases (Cambridge university press, 1982). [find](https://scholar.google.com/scholar?q=D%2E+Kahneman%2C+S%2E+P%2E+Slovic%2C+P%2E+Slovic%2C+A%2E+Tversky%2C+Judgment+under+uncertainty%3A+Heuristics+and+biases+%28Cambridge+university+press%2C+1982%29%2E)
19. G. Marcus, Kluge: The haphazard evolution of the human mind (Houghton Mifflin Harcourt, 2009). [find](https://scholar.google.com/scholar?q=G%2E+Marcus%2C+Kluge%3A+The+haphazard+evolution+of+the+human+mind+%28Houghton+Mifflin+Harcourt%2C+2009%29%2E)
20. J. Evans, J. L. Barston, P. Pollard, On the conflict between logic and belief in syllogistic reasoning. Memory & cognition 11, 295–306 (1983). [find](https://scholar.google.com/scholar?q=J%2E+Evans%2C+J%2E+L%2E+Barston%2C+P%2E+Pollard%2C+On+the+conflict+between+logic+and+belief+in+syllogistic+reasoning%2E+Memory+%26+cognition+11%2C+295%E2%80%93306+%281983%29%2E)
21. J. S. B. Evans, T. S. Perry, Belief bias in children’s reasoning. Cahiers de Psychologie Cognitive/Current Psychology of Cognition (1995). [find](https://scholar.google.com/scholar?q=J%2E+S%2E+B%2E+Evans%2C+T%2E+S%2E+Perry%2C+Belief+bias+in+children%E2%80%99s+reasoning%2E+Cahiers+de+Psychologie+Cognitive%2FCurrent+Psychology+of+Cognition+%281995%29%2E)
22. K. C. Klauer, J. Musch, B. Naumer, On belief bias in syllogistic reasoning. Psychological review 107, 852 (2000). [find](https://scholar.google.com/scholar?q=K%2E+C%2E+Klauer%2C+J%2E+Musch%2C+B%2E+Naumer%2C+On+belief+bias+in+syllogistic+reasoning%2E+Psychological+review+107%2C+852+%282000%29%2E)
23. P. N. Johnson-Laird, Deductive reasoning. Annual review of psychology 50, 109–135 (1999). [find](https://scholar.google.com/scholar?q=P%2E+N%2E+Johnson%2DLaird%2C+Deductive+reasoning%2E+Annual+review+of+psychology+50%2C+109%E2%80%93135+%281999%29%2E)
24. P. W. Cheng, K. J. Holyoak, Pragmatic reasoning schemas. Cognitive psychology 17, 391–416 (1985). [find](https://scholar.google.com/scholar?q=P%2E+W%2E+Cheng%2C+K%2E+J%2E+Holyoak%2C+Pragmatic+reasoning+schemas%2E+Cognitive+psychology+17%2C+391%E2%80%93416+%281985%29%2E)
25. L. Cosmides, The logic of social exchange: Has natural selection shaped how humans reason? studies with the wason selection task. Cognition 31, 187–276 (1989). [find](https://scholar.google.com/scholar?q=L%2E+Cosmides%2C+The+logic+of+social+exchange%3A+Has+natural+selection+shaped+how+humans+reason%3F+studies+with+the+wason+selection+task%2E+Cognition+31%2C+187%E2%80%93276+%281989%29%2E)
26. L. Cosmides, J. Tooby, Cognitive adaptations for social exchange. The adapted mind: Evolutionary psychology and the generation of culture 163, 163–228 (1992). [find](https://scholar.google.com/scholar?q=L%2E+Cosmides%2C+J%2E+Tooby%2C+Cognitive+adaptations+for+social+exchange%2E+The+adapted+mind%3A+Evolutionary+psychology+and+the+generation+of+culture+163%2C+163%E2%80%93228+%281992%29%2E)
27. A. L. Cohen, S. Sidlowski, A. Staub, Beliefs and bayesian reasoning. Psychonomic Bulletin & Review 24, 972–978 (2017). [find](https://scholar.google.com/scholar?q=A%2E+L%2E+Cohen%2C+S%2E+Sidlowski%2C+A%2E+Staub%2C+Beliefs+and+bayesian+reasoning%2E+Psychonomic+Bulletin+%26+Review+24%2C+972%E2%80%93978+%282017%29%2E)
28. I. Dasgupta, E. Schulz, J. B. Tenenbaum, S. J. Gershman, A theory of learning to infer. Psychological review 127, 412 (2020). [find](https://scholar.google.com/scholar?q=I%2E+Dasgupta%2C+E%2E+Schulz%2C+J%2E+B%2E+Tenenbaum%2C+S%2E+J%2E+Gershman%2C+A+theory+of+learning+to+infer%2E+Psychological+review+127%2C+412+%282020%29%2E)
29. D. J. Benjamin, Errors in probabilistic reasoning and judgment biases, Tech. rep., National Bureau of Economic Research (2018). [find](https://scholar.google.com/scholar?q=D%2E+J%2E+Benjamin%2C+Errors+in+probabilistic+reasoning+and+judgment+biases%2C+Tech%2E+rep%2E%2C+National+Bureau+of+Economic+Research+%282018%29%2E)
30. C. R. Peterson, Z. Ulehla, Uncertainty, inference difficulty, and probability learning. Journal of Experimental Psychology 67, 523–530 (1964). [find](https://scholar.google.com/scholar?q=C%2E+R%2E+Peterson%2C+Z%2E+Ulehla%2C+Uncertainty%2C+inference+difficulty%2C+and+probability+learning%2E+Journal+of+Experimental+Psychology+67%2C+523%E2%80%93530+%281964%29%2E)
31. P. N. Johnson-Laird, P. Legrenzi, M. S. Legrenzi, Reasoning and a sense of reality. British journal of Psychology 63, 395–400 (1972). [find](https://scholar.google.com/scholar?q=P%2E+N%2E+Johnson%2DLaird%2C+P%2E+Legrenzi%2C+M%2E+S%2E+Legrenzi%2C+Reasoning+and+a+sense+of+reality%2E+British+journal+of+Psychology+63%2C+395%E2%80%93400+%281972%29%2E)
32. P. C. Wason, P. N. Johnson-Laird, Psychology of reasoning: Structure and content, vol. 86 (Harvard University Press, 1972). [find](https://scholar.google.com/scholar?q=P%2E+C%2E+Wason%2C+P%2E+N%2E+Johnson%2DLaird%2C+Psychology+of+reasoning%3A+Structure+and+content%2C+vol%2E+86+%28Harvard+University+Press%2C+1972%29%2E)
33. J. S. B. Evans, Bias in human reasoning: Causes and consequences. (Lawrence Erlbaum Associates, Inc, 1989). [find](https://scholar.google.com/scholar?q=J%2E+S%2E+B%2E+Evans%2C+Bias+in+human+reasoning%3A+Causes+and+consequences%2E+%28Lawrence+Erlbaum+Associates%2C+Inc%2C+1989%29%2E)
34. G. F. Marcus, The algebraic mind: Integrating connectionism and cognitive science (MIT press, 2003). [find](https://scholar.google.com/scholar?q=G%2E+F%2E+Marcus%2C+The+algebraic+mind%3A+Integrating+connectionism+and+cognitive+science+%28MIT+press%2C+2003%29%2E)
35. J. L. McClelland, M. M. Botvinick, D. C. Noelle, D. C. Plaut, T. T. Rogers, M. S. Seidenberg, L. B. Smith, Letting structure emerge: connectionist and dynamical systems approaches to cognition. Trends in cognitive sciences 14, 348–356 (2010). [find](https://scholar.google.com/scholar?q=J%2E+L%2E+McClelland%2C+M%2E+M%2E+Botvinick%2C+D%2E+C%2E+Noelle%2C+D%2E+C%2E+Plaut%2C+T%2E+T%2E+Rogers%2C+M%2E+S%2E+Seidenberg%2C+L%2E+B%2E+Smith%2C+Letting+structure+emerge%3A+connectionist+and+dynamical+systems+approaches+to+cognition%2E+Trends)
36. A. Santoro, A. Lampinen, K. Mathewson, T. Lillicrap, D. Raposo, Symbolic behaviour in artificial intelligence. arXiv preprint arXiv:2102.03406 (2021). — [arXiv:2102.03406](https://arxiv.org/abs/2102.03406) [find](https://scholar.google.com/scholar?q=A%2E+Santoro%2C+A%2E+Lampinen%2C+K%2E+Mathewson%2C+T%2E+Lillicrap%2C+D%2E+Raposo%2C+Symbolic+behaviour+in+artificial+intelligence%2E+arXiv+preprint+arXiv%3A2102%2E03406+%282021%29%2E+%E2%80%94+%5BarXiv%3A2102%2E03406%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2102%2E0)
37. R. Bommasani, D. A. Hudson, E. Adeli, R. Altman, S. Arora, S. von Arx, M. S. Bernstein, J. Bohg, A. Bosselut, E. Brunskill, et al., [On the opportunities and risks of foundation models](https://md.research.elamur.ai/free/en/papers/bommasani-2021-opportunities-risks-foundation-models/). arXiv preprint arXiv:2108.07258 (2021).
38. S. Bhatia, R. Richie, W. Zou, Distributed semantic representations for modeling human judgment. Current Opinion in Behavioral Sciences 29, 31–36 (2019). [find](https://scholar.google.com/scholar?q=S%2E+Bhatia%2C+R%2E+Richie%2C+W%2E+Zou%2C+Distributed+semantic+representations+for+modeling+human+judgment%2E+Current+Opinion+in+Behavioral+Sciences+29%2C+31%E2%80%9336+%282019%29%2E)
39. S. Bhatia, R. Richie, Transformer networks of human conceptual knowledge. Psychological Review (2021). [find](https://scholar.google.com/scholar?q=S%2E+Bhatia%2C+R%2E+Richie%2C+Transformer+networks+of+human+conceptual+knowledge%2E+Psychological+Review+%282021%29%2E)
40. T. H. Trinh, Q. V. Le, A simple method for commonsense reasoning. arXiv preprint arXiv:1806.02847 (2018). [find](https://scholar.google.com/scholar?q=T%2E+H%2E+Trinh%2C+Q%2E+V%2E+Le%2C+A+simple+method+for+commonsense+reasoning%2E+arXiv+preprint+arXiv%3A1806%2E02847+%282018%29%2E)
41. F. Petroni, T. Rocktäschel, S. Riedel, P. Lewis, A. Bakhtin, Y. Wu, A. Miller, Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) (2019), pp. 2463–2473. [find](https://scholar.google.com/scholar?q=F%2E+Petroni%2C+T%2E+Rockta%CC%88schel%2C+S%2E+Riedel%2C+P%2E+Lewis%2C+A%2E+Bakhtin%2C+Y%2E+Wu%2C+A%2E+Miller%2C+Proceedings+of+the+2019+Conference+on+Empirical+Methods+in+Natural+Language+Processing+and+the+9th+International+Joint+C)
42. L. Z. Liu, Y. Wang, J. Kasai, H. Hajishirzi, N. A. Smith, [Probing across time: What does roberta know and when?](https://md.research.elamur.ai/free/en/papers/liu-2021-probing-across-time-what-does/) arXiv preprint arXiv:2104.07885 (2021).
43. Z. Jiang, J. Araki, H. Ding, G. Neubig, How can we know when language models know? on the calibration of language models for question answering. Transactions of the Association for Computational Linguistics 9, 962–977 (2021). [find](https://scholar.google.com/scholar?q=Z%2E+Jiang%2C+J%2E+Araki%2C+H%2E+Ding%2C+G%2E+Neubig%2C+How+can+we+know+when+language+models+know%3F+on+the+calibration+of+language+models+for+question+answering%2E+Transactions+of+the+Association+for+Computational+Lingu)
44. A. Søgaard, Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing (2021), pp. 8240–8245. [find](https://scholar.google.com/scholar?q=A%2E+S%C3%B8gaard%2C+Proceedings+of+the+2021+Conference+on+Empirical+Methods+in+Natural+Language+Processing+%282021%29%2C+pp%2E+8240%E2%80%938245%2E)
45. J. Hoffmann, S. Borgeaud, A. Mensch, E. Buchatskaya, T. Cai, E. Rutherford, D. d. L. Casas, L. A. Hendricks, J. Welbl, A. Clark, et al., [Training compute-optimal large language models](https://md.research.elamur.ai/free/en/papers/hoffmann-2022-training-compute-optimal-large-language/). arXiv preprint arXiv:2203.15556 (2022).
46. Gpt-3.5, [https://platform.openai.com/docs/models/gpt-3-5](https://platform.openai.com/docs/models/gpt-3-5). trieved September 19th, 2023. [↗](https://platform.openai.com/docs/models/gpt-3-5)
47. R. Anil, A. M. Dai, O. Firat, M. Johnson, D. Lepikhin, A. Passos, S. Shakeri, E. Taropa, P. Bailey, Z. Chen, et al., [Palm 2 technical report](https://md.research.elamur.ai/free/en/papers/anil-2023-palm-2-technical-report/). arXiv preprint arXiv:2305.10403 (2023).
48. B. MacCartney, C. D. Manning, Proceedings of the ACL-PASCAL Workshop on Textual Entailment and Paraphrasing (2007), pp. 193–200. [find](https://scholar.google.com/scholar?q=B%2E+MacCartney%2C+C%2E+D%2E+Manning%2C+Proceedings+of+the+ACL%2DPASCAL+Workshop+on+Textual+Entailment+and+Paraphrasing+%282007%29%2C+pp%2E+193%E2%80%93200%2E)
49. S. R. Bowman, G. Angeli, C. Potts, C. D. Manning, [A large annotated corpus for learning natural language inference](https://md.research.elamur.ai/free/en/papers/bowman-2015-large-annotated-corpus-learning-natural/). arXiv preprint arXiv:1508.05326 (2015).
50. I. Dasgupta, D. Guo, A. Stuhlmüller, S. J. Gershman, N. D. Goodman, Evaluating compositionality in sentence embeddings. arXiv preprint arXiv:1802.04302 (2018). — [arXiv:1802.04302](https://arxiv.org/abs/1802.04302) [find](https://scholar.google.com/scholar?q=I%2E+Dasgupta%2C+D%2E+Guo%2C+A%2E+Stuhlmu%CC%88ller%2C+S%2E+J%2E+Gershman%2C+N%2E+D%2E+Goodman%2C+Evaluating+compositionality+in+sentence+embeddings%2E+arXiv+preprint+arXiv%3A1802%2E04302+%282018%29%2E+%E2%80%94+%5BarXiv%3A1802%2E04302%5D%28https%3A%2F%2Farxiv%2Eorg%2F)
51. R. Smith, The Stanford Encyclopedia of Philosophy, E. N. Zalta, ed. (Metaphysics Research Lab, Stanford University, 2020), Fall 2020 edn. [find](https://scholar.google.com/scholar?q=R%2E+Smith%2C+The+Stanford+Encyclopedia+of+Philosophy%2C+E%2E+N%2E+Zalta%2C+ed%2E+%28Metaphysics+Research+Lab%2C+Stanford+University%2C+2020%29%2C+Fall+2020+edn%2E)
52. M. H. Tessler, J. B. Tenenbaum, N. D. Goodman, Logic, probability, and pragmatics in syllogistic reasoning. Topics in Cognitive Science (2022). Re- [find](https://scholar.google.com/scholar?q=M%2E+H%2E+Tessler%2C+J%2E+B%2E+Tenenbaum%2C+N%2E+D%2E+Goodman%2C+Logic%2C+probability%2C+and+pragmatics+in+syllogistic+reasoning%2E+Topics+in+Cognitive+Science+%282022%29%2E+Re%2D)
53. C. Dube, C. M. Rotello, E. Heit, Assessing the belief bias effect with rocs: it’s a response bias effect. Psychological review 117, 831 (2010). [find](https://scholar.google.com/scholar?q=C%2E+Dube%2C+C%2E+M%2E+Rotello%2C+E%2E+Heit%2C+Assessing+the+belief+bias+effect+with+rocs%3A+it%E2%80%99s+a+response+bias+effect%2E+Psychological+review+117%2C+831+%282010%29%2E)
54. D. Trippas, M. F. Verde, S. J. Handley, Using forced choice to test belief bias in syllogistic reasoning. Cognition 133, 586–600 (2014). [find](https://scholar.google.com/scholar?q=D%2E+Trippas%2C+M%2E+F%2E+Verde%2C+S%2E+J%2E+Handley%2C+Using+forced+choice+to+test+belief+bias+in+syllogistic+reasoning%2E+Cognition+133%2C+586%E2%80%93600+%282014%29%2E)
55. M. H. Tessler, Understanding belief bias by measuring prior beliefs for a bayesian model of syllogistic reasoning. Proceedings of ESSLLI pp. 225–237 (2015). [find](https://scholar.google.com/scholar?q=M%2E+H%2E+Tessler%2C+Understanding+belief+bias+by+measuring+prior+beliefs+for+a+bayesian+model+of+syllogistic+reasoning%2E+Proceedings+of+ESSLLI+pp%2E+225%E2%80%93237+%282015%29%2E)
56. P. C. Wason, D. Shapiro, Natural and contrived experience in a reasoning problem. Quarterly journal of experimental psychology 23, 63–71 (1971). [find](https://scholar.google.com/scholar?q=P%2E+C%2E+Wason%2C+D%2E+Shapiro%2C+Natural+and+contrived+experience+in+a+reasoning+problem%2E+Quarterly+journal+of+experimental+psychology+23%2C+63%E2%80%9371+%281971%29%2E)
57. M. Inglis, A. Simpson, Mathematicians and the selection task. International Group for the Psychology of Mathematics Education (2004). [find](https://scholar.google.com/scholar?q=M%2E+Inglis%2C+A%2E+Simpson%2C+Mathematicians+and+the+selection+task%2E+International+Group+for+the+Psychology+of+Mathematics+Education+%282004%29%2E)
58. C. Cresswell, C. P. Speelman, Does mathematics training lead to better logical thinking and reasoning? a cross-sectional assessment from students to professors. PLOS ONE 15, 1-21 (2020). [find](https://scholar.google.com/scholar?q=C%2E+Cresswell%2C+C%2E+P%2E+Speelman%2C+Does+mathematics+training+lead+to+better+logical+thinking+and+reasoning%3F+a+cross%2Dsectional+assessment+from+students+to+professors%2E+PLOS+ONE+15%2C+1%2D21+%282020%29%2E)
59. J. S. B. Evans, J. Curtis-Holmes, Rapid responding increases belief bias: Evidence for the dual-process theory of reasoning. Thinking & Reasoning 11, 382–389 (2005). [find](https://scholar.google.com/scholar?q=J%2E+S%2E+B%2E+Evans%2C+J%2E+Curtis%2DHolmes%2C+Rapid+responding+increases+belief+bias%3A+Evidence+for+the+dual%2Dprocess+theory+of+reasoning%2E+Thinking+%26+Reasoning+11%2C+382%E2%80%93389+%282005%29%2E)
60. J. S. B. Evans, S. Newstead, J. Allen, P. Pollard, Debiasing by instruction: The case of belief bias. European Journal of Cognitive Psychology 6, 263–285 (1994). [find](https://scholar.google.com/scholar?q=J%2E+S%2E+B%2E+Evans%2C+S%2E+Newstead%2C+J%2E+Allen%2C+P%2E+Pollard%2C+Debiasing+by+instruction%3A+The+case+of+belief+bias%2E+European+Journal+of+Cognitive+Psychology+6%2C+263%E2%80%93285+%281994%29%2E)
61. W. A. Wickelgren, Speed-accuracy tradeoff and information processing dynamics. Acta psychologica 41, 67–85 (1977). [find](https://scholar.google.com/scholar?q=W%2E+A%2E+Wickelgren%2C+Speed%2Daccuracy+tradeoff+and+information+processing+dynamics%2E+Acta+psychologica+41%2C+67%E2%80%9385+%281977%29%2E)
62. S. L. Wise, X. Kong, Response time effort: A new measure of examinee motivation in computer-based tests. Applied Measurement in Education 18, 163–183 (2005). [find](https://scholar.google.com/scholar?q=S%2E+L%2E+Wise%2C+X%2E+Kong%2C+Response+time+effort%3A+A+new+measure+of+examinee+motivation+in+computer%2Dbased+tests%2E+Applied+Measurement+in+Education+18%2C+163%E2%80%93183+%282005%29%2E)
63. S. Kadavath, T. Conerly, A. Askell, T. Henighan, D. Drain, E. Perez, N. Schiefer, Z. H. Dodds, N. DasSarma, E. Tran-Johnson, S. Johnston, S. El-Showk, A. Jones, N. Elhage, T. Hume, A. Chen, Y. Bai, S. Bowman, S. Fort, D. Ganguli, D. Hernandez, J. Jacobson, J. Kernion, S. Kravec, L. Lovitt, K. Ndouss… [find](https://scholar.google.com/scholar?q=S%2E+Kadavath%2C+T%2E+Conerly%2C+A%2E+Askell%2C+T%2E+Henighan%2C+D%2E+Drain%2C+E%2E+Perez%2C+N%2E+Schiefer%2C+Z%2E+H%2E+Dodds%2C+N%2E+DasSarma%2C+E%2E+Tran%2DJohnson%2C+S%2E+Johnston%2C+S%2E+El%2DShowk%2C+A%2E+Jones%2C+N%2E+Elhage%2C+T%2E+Hume%2C+A%2E+Chen%2C+Y%2E+Bai%2C+S%2E)
64. A. Goodkind, K. Bicknell, Proceedings of the 8th workshop on cognitive modeling and computational linguistics (CMCL 2018) (2018), pp. 10–18. [find](https://scholar.google.com/scholar?q=A%2E+Goodkind%2C+K%2E+Bicknell%2C+Proceedings+of+the+8th+workshop+on+cognitive+modeling+and+computational+linguistics+%28CMCL+2018%29+%282018%29%2C+pp%2E+10%E2%80%9318%2E)
65. R. Futrell, E. Wilcox, T. Morita, P. Qian, M. Ballesteros, R. Levy, Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) (Association for Computational Linguistics, Minneapolis… [find](https://scholar.google.com/scholar?q=R%2E+Futrell%2C+E%2E+Wilcox%2C+T%2E+Morita%2C+P%2E+Qian%2C+M%2E+Ballesteros%2C+R%2E+Levy%2C+Proceedings+of+the+2019+Conference+of+the+North+American+Chapter+of+the+Association+for+Computational+Linguistics%3A+Human+Language+Te)
66. M. Binz, E. Schulz, Using cognitive psychology to understand gpt-3 (2022). [find](https://scholar.google.com/scholar?q=M%2E+Binz%2C+E%2E+Schulz%2C+Using+cognitive+psychology+to+understand+gpt%2D3+%282022%29%2E)
67. J. Wei, X. Wang, D. Schuurmans, M. Bosma, E. Chi, Q. Le, D. Zhou, Chain of thought prompting elicits reasoning in large language models. arXiv preprint arXiv:2201.[11903 (2022)](https://md.research.elamur.ai/free/en/papers/wei-2022-chain-thought-prompting-elicits-reasoning/).
68. D. Khashabi, C. Baral, Y. Choi, H. Hajishirzi, Findings of the Association for Computational Linguistics: ACL 2022 (2022), pp. 589–612. [find](https://scholar.google.com/scholar?q=D%2E+Khashabi%2C+C%2E+Baral%2C+Y%2E+Choi%2C+H%2E+Hajishirzi%2C+Findings+of+the+Association+for+Computational+Linguistics%3A+ACL+2022+%282022%29%2C+pp%2E+589%E2%80%93612%2E)
69. A. K. Lampinen, I. Dasgupta, S. C. Chan, K. Matthewson, M. H. Tessler, A. Creswell, J. L. McClelland, J. X. Wang, F. Hill, [Can language models learn from explanations in context?](https://md.research.elamur.ai/free/en/papers/lampinen-2022-can-language-models-learn-explanations/) arXiv preprint arXiv:2204.02329 (2022).
70. X. Wang, J. Wei, D. Schuurmans, Q. Le, E. Chi, D. Zhou, [Self-consistency improves chain of thought reasoning in language models](https://md.research.elamur.ai/free/en/papers/wang-2022-self-consistency-improves-chain-thought/). arXiv preprint arXiv:2203.11171 (2022).
71. H. Mercier, D. Sperber, The Enigma of Reason (Harvard University Press, 2017). [find](https://scholar.google.com/scholar?q=H%2E+Mercier%2C+D%2E+Sperber%2C+The+Enigma+of+Reason+%28Harvard+University+Press%2C+2017%29%2E)
72. J. S. B. Evans, Heuristic and analytic processes in reasoning. British Journal of Psychology 75, 451–468 (1984). [find](https://scholar.google.com/scholar?q=J%2E+S%2E+B%2E+Evans%2C+Heuristic+and+analytic+processes+in+reasoning%2E+British+Journal+of+Psychology+75%2C+451%E2%80%93468+%281984%29%2E)
73. J. S. B. Evans, In two minds: dual-process accounts of reasoning. Trends in cognitive sciences 7, 454–459 (2003). [find](https://scholar.google.com/scholar?q=J%2E+S%2E+B%2E+Evans%2C+In+two+minds%3A+dual%2Dprocess+accounts+of+reasoning%2E+Trends+in+cognitive+sciences+7%2C+454%E2%80%93459+%282003%29%2E)
74. M. Oaksford, N. Chater, Optimal data selection: Revision, review, and reevaluation. Psychonomic Bulletin & Review 10, 289–318 (2003). [find](https://scholar.google.com/scholar?q=M%2E+Oaksford%2C+N%2E+Chater%2C+Optimal+data+selection%3A+Revision%2C+review%2C+and+reevaluation%2E+Psychonomic+Bulletin+%26+Review+10%2C+289%E2%80%93318+%282003%29%2E)
75. D. Kahneman, Thinking, fast and slow (Macmillan, 2011). [find](https://scholar.google.com/scholar?q=D%2E+Kahneman%2C+Thinking%2C+fast+and+slow+%28Macmillan%2C+2011%29%2E)
76. J. S. B. Evans, D. E. Over, Rationality and reasoning (Psychology Press, 2013). [find](https://scholar.google.com/scholar?q=J%2E+S%2E+B%2E+Evans%2C+D%2E+E%2E+Over%2C+Rationality+and+reasoning+%28Psychology+Press%2C+2013%29%2E)
77. Y. Bengio, The consciousness prior. arXiv preprint arXiv:1709.08568 (2017). — [arXiv:1709.08568](https://arxiv.org/abs/1709.08568) [find](https://scholar.google.com/scholar?q=Y%2E+Bengio%2C+The+consciousness+prior%2E+arXiv+preprint+arXiv%3A1709%2E08568+%282017%29%2E+%E2%80%94+%5BarXiv%3A1709%2E08568%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F1709%2E08568%29)
78. M. Nye, M. Tessler, J. Tenenbaum, B. M. Lake, Improving coherence and consistency in neural sequence models with dual-system, neuro-symbolic reasoning. Advances in Neural Information Processing Systems 34, 25192–25204 (2021). [find](https://scholar.google.com/scholar?q=M%2E+Nye%2C+M%2E+Tessler%2C+J%2E+Tenenbaum%2C+B%2E+M%2E+Lake%2C+Improving+coherence+and+consistency+in+neural+sequence+models+with+dual%2Dsystem%2C+neuro%2Dsymbolic+reasoning%2E+Advances+in+Neural+Information+Processing+System)
79. J. D. Cohen, K. Dunbar, J. L. McClelland, On the control of automatic processes: a parallel distributed processing account of the stroop effect. Psychological review 97, 332 (1990). [find](https://scholar.google.com/scholar?q=J%2E+D%2E+Cohen%2C+K%2E+Dunbar%2C+J%2E+L%2E+McClelland%2C+On+the+control+of+automatic+processes%3A+a+parallel+distributed+processing+account+of+the+stroop+effect%2E+Psychological+review+97%2C+332+%281990%29%2E)
80. M. M. Botvinick, J. D. Cohen, The computational and neural basis of cognitive control: charted territory and new frontiers. Cognitive science 38, 1249–1285 (2014). [find](https://scholar.google.com/scholar?q=M%2E+M%2E+Botvinick%2C+J%2E+D%2E+Cohen%2C+The+computational+and+neural+basis+of+cognitive+control%3A+charted+territory+and+new+frontiers%2E+Cognitive+science+38%2C+1249%E2%80%931285+%282014%29%2E)
81. J. Duncan, M. Assem, S. Shashidhara, Integrated intelligence from distributed brain activity. Trends in Cognitive Sciences 24, 838–852 (2020). [find](https://scholar.google.com/scholar?q=J%2E+Duncan%2C+M%2E+Assem%2C+S%2E+Shashidhara%2C+Integrated+intelligence+from+distributed+brain+activity%2E+Trends+in+Cognitive+Sciences+24%2C+838%E2%80%93852+%282020%29%2E)
82. Y. Li, J. L. McClelland, A weighted constraint satisfaction approach to human goal-directed decision making. PLOS Computational Biology 18, e1009553 (2022). [find](https://scholar.google.com/scholar?q=Y%2E+Li%2C+J%2E+L%2E+McClelland%2C+A+weighted+constraint+satisfaction+approach+to+human+goal%2Ddirected+decision+making%2E+PLOS+Computational+Biology+18%2C+e1009553+%282022%29%2E)
83. D. L. Yamins, H. Hong, C. F. Cadieu, E. A. Solomon, D. Seibert, J. J. DiCarlo, Performance-optimized hierarchical models predict neural responses in higher visual cortex. Proceedings of the national academy of sciences 111, 8619–8624 (2014). [find](https://scholar.google.com/scholar?q=D%2E+L%2E+Yamins%2C+H%2E+Hong%2C+C%2E+F%2E+Cadieu%2C+E%2E+A%2E+Solomon%2C+D%2E+Seibert%2C+J%2E+J%2E+DiCarlo%2C+Performance%2Doptimized+hierarchical+models+predict+neural+responses+in+higher+visual+cortex%2E+Proceedings+of+the+national+a)
84. D. L. Yamins, J. J. DiCarlo, Using goal-driven deep learning models to understand sensory cortex. Nature neuroscience 19, 356–365 (2016). [find](https://scholar.google.com/scholar?q=D%2E+L%2E+Yamins%2C+J%2E+J%2E+DiCarlo%2C+Using+goal%2Ddriven+deep+learning+models+to+understand+sensory+cortex%2E+Nature+neuroscience+19%2C+356%E2%80%93365+%282016%29%2E)
85. R. Cao, D. Yamins, Explanatory models in neuroscience: Part 1–taking mechanistic abstraction seriously. arXiv preprint arXiv:2104.01490 (2021). — [arXiv:2104.01490](https://arxiv.org/abs/2104.01490) [find](https://scholar.google.com/scholar?q=R%2E+Cao%2C+D%2E+Yamins%2C+Explanatory+models+in+neuroscience%3A+Part+1%E2%80%93taking+mechanistic+abstraction+seriously%2E+arXiv+preprint+arXiv%3A2104%2E01490+%282021%29%2E+%E2%80%94+%5BarXiv%3A2104%2E01490%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2104%2E01490%29)
86. R. Cao, D. Yamins, Explanatory models in neuroscience: Part 2–constraint-based intelligibility. arXiv preprint arXiv:2104.01489 (2021). — [arXiv:2104.01489](https://arxiv.org/abs/2104.01489) [find](https://scholar.google.com/scholar?q=R%2E+Cao%2C+D%2E+Yamins%2C+Explanatory+models+in+neuroscience%3A+Part+2%E2%80%93constraint%2Dbased+intelligibility%2E+arXiv+preprint+arXiv%3A2104%2E01489+%282021%29%2E+%E2%80%94+%5BarXiv%3A2104%2E01489%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2104%2E01489%29)
87. M. Schrimpf, I. A. Blank, G. Tuckute, C. Kauf, E. A. Hosseini, N. Kanwisher, J. B. Tenenbaum, E. Fedorenko, The neural architecture of language: Integrative modeling converges on predictive processing. Proceedings of the National Academy of Sciences 118 (2021). [find](https://scholar.google.com/scholar?q=M%2E+Schrimpf%2C+I%2E+A%2E+Blank%2C+G%2E+Tuckute%2C+C%2E+Kauf%2C+E%2E+A%2E+Hosseini%2C+N%2E+Kanwisher%2C+J%2E+B%2E+Tenenbaum%2C+E%2E+Fedorenko%2C+The+neural+architecture+of+language%3A+Integrative+modeling+converges+on+predictive+processing)
88. S. Kumar, T. R. Sumers, T. Yamakoshi, A. Goldstein, U. Hasson, K. A. Norman, T. L. Griffiths, R. D. Hawkins, S. A. Nastase, Reconstructing the cascade of language processing in the brain using the internal computations of a transformer-based language model. bioRxiv (2022). [find](https://scholar.google.com/scholar?q=S%2E+Kumar%2C+T%2E+R%2E+Sumers%2C+T%2E+Yamakoshi%2C+A%2E+Goldstein%2C+U%2E+Hasson%2C+K%2E+A%2E+Norman%2C+T%2E+L%2E+Griffiths%2C+R%2E+D%2E+Hawkins%2C+S%2E+A%2E+Nastase%2C+Reconstructing+the+cascade+of+language+processing+in+the+brain+using+the+int)
89. A. Goldstein, Z. Zada, E. Buchnik, M. Schain, A. Price, B. Aubrey, S. A. Nastase, A. Feder, D. Emanuel, A. Cohen, et al., Shared computational principles for language processing in humans and deep language models. Nature neuroscience 25, 369–380 (2022). [find](https://scholar.google.com/scholar?q=A%2E+Goldstein%2C+Z%2E+Zada%2C+E%2E+Buchnik%2C+M%2E+Schain%2C+A%2E+Price%2C+B%2E+Aubrey%2C+S%2E+A%2E+Nastase%2C+A%2E+Feder%2C+D%2E+Emanuel%2C+A%2E+Cohen%2C+et+al%2E%2C+Shared+computational+principles+for+language+processing+in+humans+and+deep+lan)
90. E. G. Wilcox, J. Gauthier, J. Hu, P. Qian, R. Levy, [On the predictive power of neural language models for human real-time comprehension behavior](https://md.research.elamur.ai/free/en/papers/wilcox-2020-predictive-power-neural-language-models/). arXiv:2006.01912 (2020).
91. F. Lieder, T. L. Griffiths, Resource-rational analysis: Understanding human cognition as the optimal use of limited computational resources. Behavioral and brain sciences 43 (2020). [find](https://scholar.google.com/scholar?q=F%2E+Lieder%2C+T%2E+L%2E+Griffiths%2C+Resource%2Drational+analysis%3A+Understanding+human+cognition+as+the+optimal+use+of+limited+computational+resources%2E+Behavioral+and+brain+sciences+43+%282020%29%2E)
92. S. J. Gershman, E. J. Horvitz, J. B. Tenenbaum, Computational rationality: A converging paradigm for intelligence in brains, minds, and machines. Science 349, 273–278 (2015). [find](https://scholar.google.com/scholar?q=S%2E+J%2E+Gershman%2C+E%2E+J%2E+Horvitz%2C+J%2E+B%2E+Tenenbaum%2C+Computational+rationality%3A+A+converging+paradigm+for+intelligence+in+brains%2C+minds%2C+and+machines%2E+Science+349%2C+273%E2%80%93278+%282015%29%2E)
93. H. A. Simon, Utility and probability (Springer, 1990), pp. 15–18. [find](https://scholar.google.com/scholar?q=H%2E+A%2E+Simon%2C+Utility+and+probability+%28Springer%2C+1990%29%2C+pp%2E+15%E2%80%9318%2E)
94. A. Webson, E. Pavlick, [Do prompt-based models really understand the meaning of their prompts?](https://md.research.elamur.ai/free/en/papers/webson-2022-do-prompt-based-models-really/) arXiv preprint arXiv:2109.01247 (2021).
95. S. Mishra, D. Khashabi, C. Baral, H. Hajishirzi, Cross-task generalization via natural language crowdsourcing instructions. arXiv preprint arXiv:2104.08773 (2021). [find](https://scholar.google.com/scholar?q=S%2E+Mishra%2C+D%2E+Khashabi%2C+C%2E+Baral%2C+H%2E+Hajishirzi%2C+Cross%2Dtask+generalization+via+natural+language+crowdsourcing+instructions%2E+arXiv+preprint+arXiv%3A2104%2E08773+%282021%29%2E)
96. J. L. McClelland, F. Hill, M. Rudolph, J. Baldridge, H. Schütze, Placing language in an integrated understanding system: Next steps toward human-level performance in neural language models. Proceedings of the National Academy of Sciences 117, 25966–25974 (2020). [find](https://scholar.google.com/scholar?q=J%2E+L%2E+McClelland%2C+F%2E+Hill%2C+M%2E+Rudolph%2C+J%2E+Baldridge%2C+H%2E+Schu%CC%88tze%2C+Placing+language+in+an+integrated+understanding+system%3A+Next+steps+toward+human%2Dlevel+performance+in+neural+language+models%2E+Proceedin)
97. M. W. Alibali, R. Boncoddo, A. B. Hostetter, Gesture in reasoning: An embodied perspective. The Routledge handbook of embodied cognition p. 150 (2014). [find](https://scholar.google.com/scholar?q=M%2E+W%2E+Alibali%2C+R%2E+Boncoddo%2C+A%2E+B%2E+Hostetter%2C+Gesture+in+reasoning%3A+An+embodied+perspective%2E+The+Routledge+handbook+of+embodied+cognition+p%2E+150+%282014%29%2E)
98. M. J. Nathan, K. E. Schenck, R. Vinsonhaler, J. E. Michaelis, M. I. Swart, C. Walkington, Embodied geometric reasoning: Dynamic gestures during intuition, insight, and proof. Journal of Educational Psychology (2020). arXiv preprint [find](https://scholar.google.com/scholar?q=M%2E+J%2E+Nathan%2C+K%2E+E%2E+Schenck%2C+R%2E+Vinsonhaler%2C+J%2E+E%2E+Michaelis%2C+M%2E+I%2E+Swart%2C+C%2E+Walkington%2C+Embodied+geometric+reasoning%3A+Dynamic+gestures+during+intuition%2C+insight%2C+and+proof%2E+Journal+of+Educational+Ps)
99. H. H. Clark, Using language (Cambridge university press, 1996). [find](https://scholar.google.com/scholar?q=H%2E+H%2E+Clark%2C+Using+language+%28Cambridge+university+press%2C+1996%29%2E)
100. D. Schlangen, Norm participation grounds language. arXiv preprint arXiv:2206.02885 (2022). — [arXiv:2206.02885](https://arxiv.org/abs/2206.02885) [find](https://scholar.google.com/scholar?q=D%2E+Schlangen%2C+Norm+participation+grounds+language%2E+arXiv+preprint+arXiv%3A2206%2E02885+%282022%29%2E+%E2%80%94+%5BarXiv%3A2206%2E02885%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2206%2E02885%29)
101. A. K. Luria, Towards the problem of the historical nature of psychological processes. International Journal of Psychology 6, 259–272 (1971). [find](https://scholar.google.com/scholar?q=A%2E+K%2E+Luria%2C+Towards+the+problem+of+the+historical+nature+of+psychological+processes%2E+International+Journal+of+Psychology+6%2C+259%E2%80%93272+%281971%29%2E)
102. D. R. Lehman, R. E. Nisbett, A longitudinal study of the effects of undergraduate training on reasoning. Developmental Psychology 26, 952 (1990). [find](https://scholar.google.com/scholar?q=D%2E+R%2E+Lehman%2C+R%2E+E%2E+Nisbett%2C+A+longitudinal+study+of+the+effects+of+undergraduate+training+on+reasoning%2E+Developmental+Psychology+26%2C+952+%281990%29%2E)
103. N. Attridge, A. Aberdein, M. Inglis, Does studying logic improve logical reasoning? (2016). [find](https://scholar.google.com/scholar?q=N%2E+Attridge%2C+A%2E+Aberdein%2C+M%2E+Inglis%2C+Does+studying+logic+improve+logical+reasoning%3F+%282016%29%2E)
104. A. J. Nam, J. L. McClelland, What underlies rapid learning and systematic generalization in humans. arXiv preprint arXiv:2107.06994 (2021). — [arXiv:2107.06994](https://arxiv.org/abs/2107.06994) [find](https://scholar.google.com/scholar?q=A%2E+J%2E+Nam%2C+J%2E+L%2E+McClelland%2C+What+underlies+rapid+learning+and+systematic+generalization+in+humans%2E+arXiv+preprint+arXiv%3A2107%2E06994+%282021%29%2E+%E2%80%94+%5BarXiv%3A2107%2E06994%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2107%2E06994%29)
105. P. Clark, O. Tafjord, K. Richardson, [Transformers as soft reasoners over language](https://md.research.elamur.ai/free/en/papers/clark-2020-transformers-soft-reasoners-over-language/). arXiv preprint arXiv:2002.05867 (2020).
106. Y. Wu, M. N. Rabe, W. Li, J. Ba, R. B. Grosse, C. Szegedy, International Conference on Machine Learning (PMLR, 2021), pp. 11251–11262. [find](https://scholar.google.com/scholar?q=Y%2E+Wu%2C+M%2E+N%2E+Rabe%2C+W%2E+Li%2C+J%2E+Ba%2C+R%2E+B%2E+Grosse%2C+C%2E+Szegedy%2C+International+Conference+on+Machine+Learning+%28PMLR%2C+2021%29%2C+pp%2E+11251%E2%80%9311262%2E)
107. T. Schick, S. Udupa, H. Schütze, [Self-Diagnosis and Self-Debiasing: A Proposal for Reducing Corpus-Based Bias in NLP](https://md.research.elamur.ai/free/en/papers/schick-2021-self-diagnosis-self-debiasing-proposal/). Transactions of the Association for Computational Linguistics 9, 1408-1424 (2021).
108. K. Cobbe, V. Kosaraju, M. Bavarian, J. Hilton, R. Nakano, C. Hesse, J. Schulman, [Training verifiers to solve math word problems](https://md.research.elamur.ai/free/en/papers/cobbe-2021-training-verifiers-solve-math-word/). arXiv preprint arXiv:2110.14168 (2021).
109. W. Saunders, C. Yeh, J. Wu, S. Bills, L. Ouyang, J. Ward, J. Leike, [Self-critiquing models for assisting human evaluators](https://md.research.elamur.ai/free/en/papers/saunders-2022-self-critiquing-models-assisting-human/). arXiv preprint arXiv:2206.05802 (2022).
110. E. Zelikman, Y. Wu, N. D. Goodman, [Star: Bootstrapping reasoning with reasoning](https://md.research.elamur.ai/free/en/papers/zelikman-2022-star-bootstrapping-reasoning-reasoning/). arXiv preprint arXiv:2203.14465 (2022).
111. J. X. Wang, Meta-learning in natural and artificial intelligence. Current Opinion in Behavioral Sciences 38, 90–95 (2021). [find](https://scholar.google.com/scholar?q=J%2E+X%2E+Wang%2C+Meta%2Dlearning+in+natural+and+artificial+intelligence%2E+Current+Opinion+in+Behavioral+Sciences+38%2C+90%E2%80%9395+%282021%29%2E)
112. S. Chan, A. Santoro, A. Lampinen, J. Wang, A. Singh, P. Richemond, J. McClelland, F. Hill, Data distributional properties drive emergent in-context learning in transformers. Advances in Neural Information Processing Systems 35, 18878–18891 (2022). [find](https://scholar.google.com/scholar?q=S%2E+Chan%2C+A%2E+Santoro%2C+A%2E+Lampinen%2C+J%2E+Wang%2C+A%2E+Singh%2C+P%2E+Richemond%2C+J%2E+McClelland%2C+F%2E+Hill%2C+Data+distributional+properties+drive+emergent+in%2Dcontext+learning+in+transformers%2E+Advances+in+Neural+Informa)
113. B. Prystawski, N. D. Goodman, Why think step-by-step? reasoning emerges from the locality of experience. arXiv preprint arXiv:2304.03843 (2023). [find](https://scholar.google.com/scholar?q=B%2E+Prystawski%2C+N%2E+D%2E+Goodman%2C+Why+think+step%2Dby%2Dstep%3F+reasoning+emerges+from+the+locality+of+experience%2E+arXiv+preprint+arXiv%3A2304%2E03843+%282023%29%2E)
114. A. Emami, A. Trischler, K. Suleman, J. C. K. Cheung, An analysis of dataset overlap on winograd-style tasks. arXiv preprint arXiv:2011.04767 (2020). [find](https://scholar.google.com/scholar?q=A%2E+Emami%2C+A%2E+Trischler%2C+K%2E+Suleman%2C+J%2E+C%2E+K%2E+Cheung%2C+An+analysis+of+dataset+overlap+on+winograd%2Dstyle+tasks%2E+arXiv+preprint+arXiv%3A2011%2E04767+%282020%29%2E)
115. J. Dodge, M. Sap, A. Marasović, W. Agnew, G. Ilharco, D. Groeneveld, M. Mitchell, M. Gardner, Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing (Association for Computational Linguistics, Online and Punta Cana, Dominican Republic, 2021), pp. 1286–1305. [find](https://scholar.google.com/scholar?q=J%2E+Dodge%2C+M%2E+Sap%2C+A%2E+Marasovic%CC%81%2C+W%2E+Agnew%2C+G%2E+Ilharco%2C+D%2E+Groeneveld%2C+M%2E+Mitchell%2C+M%2E+Gardner%2C+Proceedings+of+the+2021+Conference+on+Empirical+Methods+in+Natural+Language+Processing+%28Association+for+C)
116. Y. Tay, M. Dehghani, V. Q. Tran, X. Garcia, J. Wei, X. Wang, H. W. Chung, D. Bahri, T. Schuster, S. Zheng, et al., The Eleventh International Conference on Learning Representations (2022). [find](https://scholar.google.com/scholar?q=Y%2E+Tay%2C+M%2E+Dehghani%2C+V%2E+Q%2E+Tran%2C+X%2E+Garcia%2C+J%2E+Wei%2C+X%2E+Wang%2C+H%2E+W%2E+Chung%2C+D%2E+Bahri%2C+T%2E+Schuster%2C+S%2E+Zheng%2C+et+al%2E%2C+The+Eleventh+International+Conference+on+Learning+Representations+%282022%29%2E)
117. A. Holtzman, P. West, V. Shwartz, Y. Choi, L. Zettlemoyer, Surface form competition: Why the highest probability answer isn’t always right. arXiv preprint arXiv:2104.[08315 (2021)](https://md.research.elamur.ai/free/en/papers/holtzman-2021-surface-form-competition-why-highest/).
118. A. Holtzman, J. Buys, L. Du, M. Forbes, Y. Choi, International Conference on Learning Representations (2019). [find](https://scholar.google.com/scholar?q=A%2E+Holtzman%2C+J%2E+Buys%2C+L%2E+Du%2C+M%2E+Forbes%2C+Y%2E+Choi%2C+International+Conference+on+Learning+Representations+%282019%29%2E)
119. A. Hosseini, S. Reddy, D. Bahdanau, R. D. Hjelm, A. Sordoni, A. Courville, Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Association for Computational Linguistics, Online, 2021), pp. 1301–1312. [find](https://scholar.google.com/scholar?q=A%2E+Hosseini%2C+S%2E+Reddy%2C+D%2E+Bahdanau%2C+R%2E+D%2E+Hjelm%2C+A%2E+Sordoni%2C+A%2E+Courville%2C+Proceedings+of+the+2021+Conference+of+the+North+American+Chapter+of+the+Association+for+Computational+Linguistics%3A+Human+Lang)
120. J. S. B. Evans, J. Clibbens, B. Rood, The role of implicit and explicit negation in conditional reasoning bias. Journal of Memory and Language 35, 392–409 (1996). Acknowledgements: We thank Michiel Bakker, Michael Henry Tessler, Matt Botvinick, and Adam Santoro for helpful comments and suggestions.… [find](https://scholar.google.com/scholar?q=J%2E+S%2E+B%2E+Evans%2C+J%2E+Clibbens%2C+B%2E+Rood%2C+The+role+of+implicit+and+explicit+negation+in+conditional+reasoning+bias%2E+Journal+of+Memory+and+Language+35%2C+392%E2%80%93409+%281996%29%2E+Acknowledgements%3A+We+thank+Michiel+Ba)

*Updated 2 November 2026*