---
title: "ToMChallenges — paper by Ma et al. (2023)"
description: "Drawing on developmental psychology, they propose two criteria a valid test should meet: mentalizing… 30 references, cited by 13 papers here."
authors: ["Xiaomeng Ma", "Lingyu Gao", "Qihui Xu"]
year: 2023
venue: "arXiv:2305.15068"
source: "https://arxiv.org/abs/2305.15068"
doi: "10.48550/arxiv.2305.15068"
arxiv: "2305.15068"
licence: "CC BY 4.0 (arXiv)"
licence_url: "http://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/ma-2023-tomchallenges-principle-guided-dataset-diverse/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/ma-2023-tomchallenges-principle-guided-dataset-diverse/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2023](https://md.research.elamur.ai/free/en/papers/2023/page/13/) / ToMChallenges: A Principle-Guided Dataset and Diverse Evaluation Tasks for Exploring Theory of Mind

# ToMChallenges: A Principle-Guided Dataset and Diverse Evaluation Tasks for Exploring Theory of Mind

**Authors:** [Xiaomeng Ma](https://md.research.elamur.ai/free/en/authors/xiaomeng-ma/), [Lingyu Gao](https://md.research.elamur.ai/free/en/authors/lingyu-gao/) and Qihui Xu · **Year:** 2023 · **Venue:** [arXiv:2305.15068](https://md.research.elamur.ai/free/en/venues/conll/) ([CoNLL 2023](https://md.research.elamur.ai/free/en/venues/conll/2023/)) · **Licence:** [CC BY 4.0 (arXiv)](http://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://arxiv.org/abs/2305.15068) · [PDF](https://arxiv.org/pdf/2305.15068) · [DOI 10.48550/arxiv.2305.15068](https://doi.org/10.48550/arxiv.2305.15068) · [arXiv 2305.15068](https://arxiv.org/abs/2305.15068) · [OpenAlex](https://openalex.org/W4389519164)

**Citations:** 4 citations (OpenAlex, 2026-10-14) · [13](https://md.research.elamur.ai/free/en/papers/ma-2023-tomchallenges-principle-guided-dataset-diverse/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary

Studies disagree on whether large language models have Theory of Mind (ToM). The authors argue that much of this disagreement comes from inconsistent and sometimes invalid testing. Earlier work used a few examples in a single prompt format, or used tasks that tap empathy or social attribution rather than ToM. Drawing on developmental psychology, they propose two criteria a valid test should meet: mentalizing, meaning that mental states are inferred from physical events, and false-belief understanding, meaning that one's own belief is kept separate from another person's.

They introduce ToMChallenges, a dataset built from templates of the classic Sally-Anne and Smarties false-belief tests. Each story is probed with six question types and presented in six prompt formats. Because free-form answers are costly to grade by hand, they also built a GPT-4-based auto-grader, which proved highly accurate.

They tested davinci, turbo and gpt-4. Accuracy shifted with the prompt format, question type and story, so none of the models handles ToM tasks robustly. gpt-4 was the strongest and most stable. The work shows that ToM claims about LLMs should only be made after checking many prompts and tasks.

## Key points

- The dataset contains 30 template-generated story variants of the Sally-Anne and Smarties tests. Questions cover reality, belief, and first- and second-order belief. Data and code are released at github.com/xiaomeng-ma/ToMChallenges.
- The six evaluation formats are multiple choice, fill-in-the-blank, true/false, chain-of-thought true/false, Q&A and text completion.
- Performance was highly uneven:
  - On Sally-Anne, davinci scored 0.5 on most formats.
  - On Smarties, turbo fell to 0.34–0.46 on the true/false, chain-of-thought true/false and Q&A formats.
  - gpt-4 ranged roughly from 0.84 to 1.0 across formats.
- Error analysis of the Q&A and completion outputs found three error types:
  - genuine ToM failures, such as davinci's 136 on Sally-Anne;
  - overly conservative refusals to infer a belief, such as turbo's 114 on Smarties;
  - hallucinations, meaning details not supported by the story.
- A stated limitation is that no human baseline was collected for comparison.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Toyota Technological Institute at Chicago](https://md.research.elamur.ai/free/en/universities/toyota-technological-institute-at-chicago/) — University, Chicago, United States · 35 works · 13,377 citations
- [The Graduate Center, CUNY](https://md.research.elamur.ai/free/en/universities/the-graduate-center-cuny/) — University, New York, United States · 13 works · 207 citations

Also on the byline: [Basque Center on Cognition, Brain and Language](https://ror.org/01a28zg77).

📚 Shelves (7): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 18,614 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 17,965 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 17,450 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 17,296 works · [👁️ Computer Vision and Pattern Recognition](https://md.research.elamur.ai/free/en/shelves/computer-science/computer-vision-and-pattern-recognition/) · 3,486 works · [💭 Psychology](https://md.research.elamur.ai/free/en/shelves/psychology/) · 1,649 works · [💭 Developmental and Educational Psychology](https://md.research.elamur.ai/free/en/shelves/psychology/developmental-and-educational-psychology/) · 523 works

## Cite

Cite as: Xiaomeng Ma, Lingyu Gao and Qihui Xu. 2023. ToMChallenges: A Principle-Guided Dataset and Diverse Evaluation Tasks for Exploring Theory of Mind. arXiv:2305.15068.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@misc{ma2023tomchallenges,
  title = {{ToMChallenges: A Principle-Guided Dataset and Diverse Evaluation Tasks for Exploring Theory of Mind}},
  author = {Xiaomeng Ma, Lingyu Gao and Qihui Xu},
  year = {2023},
  doi = {10.48550/arxiv.2305.15068},
  eprint = {2305.15068},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2305.15068}
}
```

- [← previous TADA : Task Agnostic Dialect Adapters for English](https://md.research.elamur.ai/free/en/papers/held-2023-tada-task-agnostic-dialect-adapters/)
- [next → ÚFAL CorPipe at CRAC 2023: Larger Context Improves Multilingual Coreference Resolution](https://md.research.elamur.ai/free/en/papers/straka-2023-fal-corpipe-crac-2023-larger/)
- [↑ back to papers of 2023](https://md.research.elamur.ai/free/en/papers/2023/page/13/)

## References

30 references, 5 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Jaan Aru, Aqeel Labash, Oriol Corcoll, and Raul Vicente. 2023. Mind the gap: Challenges of deep learning approaches to theory of mind. Artificial Intelligence Review, pages 1–16. [find](https://scholar.google.com/scholar?q=Jaan+Aru%2C+Aqeel+Labash%2C+Oriol+Corcoll%2C+and+Raul+Vicente%2E+2023%2E+Mind+the+gap%3A+Challenges+of+deep+learning+approaches+to+theory+of+mind%2E+Artificial+Intelligence+Review%2C+pages+1%E2%80%9316%2E)
2. Simon Baron-Cohen, Alan M Leslie, and Uta Frith. 1985. Does the autistic child have a “theory of mind”? Cognition, 21(1):37–46. [find](https://scholar.google.com/scholar?q=Simon+Baron%2DCohen%2C+Alan+M+Leslie%2C+and+Uta+Frith%2E+1985%2E+Does+the+autistic+child+have+a+%E2%80%9Ctheory+of+mind%E2%80%9D%3F+Cognition%2C+21%281%29%3A37%E2%80%9346%2E)
3. Marcel Brass, Perrine Ruby, and Stephanie Spengler. 2009. Inhibition of imitative behaviour and social cognition. Philosophical Transactions of the Royal Society B: Biological Sciences, 364(1528):2359– 2367. [find](https://scholar.google.com/scholar?q=Marcel+Brass%2C+Perrine+Ruby%2C+and+Stephanie+Spengler%2E+2009%2E+Inhibition+of+imitative+behaviour+and+social+cognition%2E+Philosophical+Transactions+of+the+Royal+Society+B%3A+Biological+Sciences%2C+364%281528%29%3A2359)
4. Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter,… — [Language Models are Few-Shot Learners](https://md.research.elamur.ai/free/en/papers/brown-2020-language-models-few-shot-learners/)
5. Sébastien Bubeck, Varun Chandrasekaran, Ronen Eldan, Johannes Gehrke, Eric Horvitz, Ece Kamar, Peter Lee, Yin Tat Lee, Yuanzhi Li, Scott Lundberg, et al. 2023. [Sparks of artificial general intelligence: Early experiments with gpt-4](https://md.research.elamur.ai/free/en/papers/bubeck-2023-sparks-artificial-general-intelligence-early/). arXiv preprint arXiv:2303.12712.
6. Stephanie M Carlson, Melissa A Koenig, and Madeline B Harms. 2013. Theory of mind. Wiley Interdisciplinary Reviews: Cognitive Science, 4(4):391–402. [find](https://scholar.google.com/scholar?q=Stephanie+M+Carlson%2C+Melissa+A+Koenig%2C+and+Madeline+B+Harms%2E+2013%2E+Theory+of+mind%2E+Wiley+Interdisciplinary+Reviews%3A+Cognitive+Science%2C+4%284%29%3A391%E2%80%93402%2E)
7. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. [BERT: Pre-training of deep bidirectional transformers for language understanding](https://md.research.elamur.ai/free/en/papers/devlin-2019-bert-pre-training-deep-bidirectional/). In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies,…
8. Vittorio Gallese and Corrado Sinigaglia. 2011. What is so special about embodied simulation? Trends in cognitive sciences, 15(11):512–519. [find](https://scholar.google.com/scholar?q=Vittorio+Gallese+and+Corrado+Sinigaglia%2E+2011%2E+What+is+so+special+about+embodied+simulation%3F+Trends+in+cognitive+sciences%2C+15%2811%29%3A512%E2%80%93519%2E)
9. Alison Gopnik and Janet W Astington. 1988. Children’s understanding of representational change and its relation to the understanding of false belief and the appearance-reality distinction. Child development, pages 26–37. [find](https://scholar.google.com/scholar?q=Alison+Gopnik+and+Janet+W+Astington%2E+1988%2E+Children%E2%80%99s+understanding+of+representational+change+and+its+relation+to+the+understanding+of+false+belief+and+the+appearance%2Dreality+distinction%2E+Child+devel)
10. Francesca Happé, Jennifer L Cook, and Geoffrey Bird. 2017. The structure of social cognition: In (ter) dependence of sociocognitive processes. Annual review of psychology, 68:243–267. [find](https://scholar.google.com/scholar?q=Francesca+Happ%C3%A9%2C+Jennifer+L+Cook%2C+and+Geoffrey+Bird%2E+2017%2E+The+structure+of+social+cognition%3A+In+%28ter%29+dependence+of+sociocognitive+processes%2E+Annual+review+of+psychology%2C+68%3A243%E2%80%93267%2E)
11. Fritz Heider and Marianne Simmel. 1944. An experimental study of apparent behavior. The American journal of psychology, 57(2):243–259. [find](https://scholar.google.com/scholar?q=Fritz+Heider+and+Marianne+Simmel%2E+1944%2E+An+experimental+study+of+apparent+behavior%2E+The+American+journal+of+psychology%2C+57%282%29%3A243%E2%80%93259%2E)
12. Philipp Kanske, Anne Böckler, Fynn-Mathis Trautwein, and Tania Singer. 2015. Dissecting the social brain: Introducing the empatom to reveal distinct neural networks and brain–behavior relations for empathy and theory of mind. NeuroImage, 122:6–19. [find](https://scholar.google.com/scholar?q=Philipp+Kanske%2C+Anne+B%C3%B6ckler%2C+Fynn%2DMathis+Trautwein%2C+and+Tania+Singer%2E+2015%2E+Dissecting+the+social+brain%3A+Introducing+the+empatom+to+reveal+distinct+neural+networks+and+brain%E2%80%93behavior+relations+for+em)
13. Michal Kosinski. 2023. [Theory of mind may have spontaneously emerged in large language models](https://md.research.elamur.ai/free/en/papers/kosinski-2023-evaluating-large-language-models-theory/). CoRR, abs/2302.02083.
14. Matthew Le, Y-Lan Boureau, and Maximilian Nickel. 2019. Revisiting the evaluation of theory of mind through question answering. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-I… [find](https://scholar.google.com/scholar?q=Matthew+Le%2C+Y%2DLan+Boureau%2C+and+Maximilian+Nickel%2E+2019%2E+Revisiting+the+evaluation+of+theory+of+mind+through+question+answering%2E+In+Proceedings+of+the+2019+Conference+on+Empirical+Methods+in+Natural+La)
15. Ester Navarro. 2022. What is theory of mind? a psychometric study of theory of mind and intelligence. Cognitive Psychology, 136:101495. [find](https://scholar.google.com/scholar?q=Ester+Navarro%2E+2022%2E+What+is+theory+of+mind%3F+a+psychometric+study+of+theory+of+mind+and+intelligence%2E+Cognitive+Psychology%2C+136%3A101495%2E)
16. Aida Nematzadeh, Kaylee Burns, Erin Grant, Alison Gopnik, and Tom Griffiths. 2018. Evaluating theory of mind in question answering. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 2392–2400, Brussels, Belgium. Association for Computational Linguistics… [find](https://scholar.google.com/scholar?q=Aida+Nematzadeh%2C+Kaylee+Burns%2C+Erin+Grant%2C+Alison+Gopnik%2C+and+Tom+Griffiths%2E+2018%2E+Evaluating+theory+of+mind+in+question+answering%2E+In+Proceedings+of+the+2018+Conference+on+Empirical+Methods+in+Natura)
17. OpenAI. 2022. Openai api. [https://api.openai.com/](https://api.openai.com/). [↗](https://api.openai.com/)
18. François Quesque and Yves Rossetti. 2020. What do theory-of-mind tasks actually measure? theory and practice. Perspectives on Psychological Science, 15(2):384–396. [find](https://scholar.google.com/scholar?q=Fran%C3%A7ois+Quesque+and+Yves+Rossetti%2E+2020%2E+What+do+theory%2Dof%2Dmind+tasks+actually+measure%3F+theory+and+practice%2E+Perspectives+on+Psychological+Science%2C+15%282%29%3A384%E2%80%93396%2E)
19. Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. 2020. [Exploring the limits of transfer learning with a unified text-to-text transformer](https://md.research.elamur.ai/free/en/papers/raffel-2019-exploring-limits-transfer-learning-unified/). J. Mach. Learn. Res., 21:140:1–140:67.
20. Maarten Sap, Ronan Le Bras, Daniel Fried, and Yejin trivial. 2022. Neural theory-of-mind? on the limits of social intelligence in large LMs. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 3762–3780, Abu Dhabi, United Arab Emirates. Association for Co… [find](https://scholar.google.com/scholar?q=Maarten+Sap%2C+Ronan+Le+Bras%2C+Daniel+Fried%2C+and+Yejin+trivial%2E+2022%2E+Neural+theory%2Dof%2Dmind%3F+on+the+limits+of+social+intelligence+in+large+LMs%2E+In+Proceedings+of+the+2022+Conference+on+Empirical+Methods+)
21. Maarten Sap, Hannah Rashkin, Derek Chen, Ronan Le Bras, and Yejin Choi. 2019. Social IQa: Commonsense reasoning about social interactions. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processi… [find](https://scholar.google.com/scholar?q=Maarten+Sap%2C+Hannah+Rashkin%2C+Derek+Chen%2C+Ronan+Le+Bras%2C+and+Yejin+Choi%2E+2019%2E+Social+IQa%3A+Commonsense+reasoning+about+social+interactions%2E+In+Proceedings+of+the+2019+Conference+on+Empirical+Methods+in)
22. Matthias Schurz, Joaquim Radua, Matthias G Tholen, Lara Maliske, Daniel S Margulies, Rogier B Mars, Jerome Sallet, and Philipp Kanske. 2021. Toward a hierarchical model of social cognition: A neuroimaging meta-analysis and integrative review of empathy and theory of mind. Psychological Bulletin, 147… [find](https://scholar.google.com/scholar?q=Matthias+Schurz%2C+Joaquim+Radua%2C+Matthias+G+Tholen%2C+Lara+Maliske%2C+Daniel+S+Margulies%2C+Rogier+B+Mars%2C+Jerome+Sallet%2C+and+Philipp+Kanske%2E+2021%2E+Toward+a+hierarchical+model+of+social+cognition%3A+A+neuroima)
23. Natalie Shapira, Mosh Levy, Seyed Hossein Alavi, Xuhui Zhou, Yejin Choi, Yoav Goldberg, Maarten Sap, and Vered Shwartz. 2023. Clever hans or neural theory of mind? stress testing social reasoning in large language models. arXiv preprint arXiv:2305.14763. [find](https://scholar.google.com/scholar?q=Natalie+Shapira%2C+Mosh+Levy%2C+Seyed+Hossein+Alavi%2C+Xuhui+Zhou%2C+Yejin+Choi%2C+Yoav+Goldberg%2C+Maarten+Sap%2C+and+Vered+Shwartz%2E+2023%2E+Clever+hans+or+neural+theory+of+mind%3F+stress+testing+social+reasoning+in+l)
24. Damien Sileo and Antoine Lernould. 2023. Mindgames: Targeting theory of mind in large language models with dynamic epistemic modal logic. arXiv preprint arXiv:2305.03353. [find](https://scholar.google.com/scholar?q=Damien+Sileo+and+Antoine+Lernould%2E+2023%2E+Mindgames%3A+Targeting+theory+of+mind+in+large+language+models+with+dynamic+epistemic+modal+logic%2E+arXiv+preprint+arXiv%3A2305%2E03353%2E)
25. Tomer Ullman. 2023a. Large language models fail on trivial alterations to theory-of-mind tasks. arXiv preprint arXiv:2302.08399. [find](https://scholar.google.com/scholar?q=Tomer+Ullman%2E+2023a%2E+Large+language+models+fail+on+trivial+alterations+to+theory%2Dof%2Dmind+tasks%2E+arXiv+preprint+arXiv%3A2302%2E08399%2E)
26. Tomer D. Ullman. 2023b. Large language models fail on trivial alterations to theory-of-mind tasks. CoRR, abs/2302.08399. [find](https://scholar.google.com/scholar?q=Tomer+D%2E+Ullman%2E+2023b%2E+Large+language+models+fail+on+trivial+alterations+to+theory%2Dof%2Dmind+tasks%2E+CoRR%2C+abs%2F2302%2E08399%2E)
27. Henry M Wellman, David Cross, and Julanne Watson. 2001. Meta-analysis of theory-of-mind development: The truth about false belief. Child development, 72(3):655–684. [find](https://scholar.google.com/scholar?q=Henry+M+Wellman%2C+David+Cross%2C+and+Julanne+Watson%2E+2001%2E+Meta%2Danalysis+of+theory%2Dof%2Dmind+development%3A+The+truth+about+false+belief%2E+Child+development%2C+72%283%29%3A655%E2%80%93684%2E)
28. Heinz Wimmer and Josef Perner. 1983. Beliefs about beliefs: Representation and constraining function of wrong beliefs in young children’s understanding of deception. Cognition, 13(1):103–128. [find](https://scholar.google.com/scholar?q=Heinz+Wimmer+and+Josef+Perner%2E+1983%2E+Beliefs+about+beliefs%3A+Representation+and+constraining+function+of+wrong+beliefs+in+young+children%E2%80%99s+understanding+of+deception%2E+Cognition%2C+13%281%29%3A103%E2%80%93128%2E)
29. Jincenzi Wu, Zhuang Chen, Jiawen Deng, Sahand Sabour, and Minlie Huang. 2023. Coke: A cognitive knowledge graph for machine theory of mind. arXiv preprint arXiv:2305.05390. [find](https://scholar.google.com/scholar?q=Jincenzi+Wu%2C+Zhuang+Chen%2C+Jiawen+Deng%2C+Sahand+Sabour%2C+and+Minlie+Huang%2E+2023%2E+Coke%3A+A+cognitive+knowledge+graph+for+machine+theory+of+mind%2E+arXiv+preprint+arXiv%3A2305%2E05390%2E)
30. Jamil Zaki and Kevin N Ochsner. 2012. The neuroscience of empathy: progress, pitfalls and promise. Nature neuroscience, 15(5):675–680. [find](https://scholar.google.com/scholar?q=Jamil+Zaki+and+Kevin+N+Ochsner%2E+2012%2E+The+neuroscience+of+empathy%3A+progress%2C+pitfalls+and+promise%2E+Nature+neuroscience%2C+15%285%29%3A675%E2%80%93680%2E)

*Updated 30 October 2026*