--- title: "CogToM — paper by Tong et al. (2026) — summary and ideas" description: "The authors evaluated 22 LLMs of different sizes, families and release dates. 2 references." authors: ["Haibo Tong", "Zeyang Yue", "Feifei Zhao", "Erliang Lin", "Lu Jia", "Ruolin Chen", "Yinqian Sun", "Qian Zhang", "Yi Zeng"] year: 2026 venue: "Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)" source: "https://aclanthology.org/2026.acl-long.1448/" doi: "10.18653/v1/2026.acl-long.1448" licence: "CC BY 4.0 (ACL Anthology)" licence_url: "https://creativecommons.org/licenses/by/4.0/" summary_licence: "Elamur Cosmic License" canonical: "https://research.elamur.ai/free/en/papers/tong-2026-cogtom-comprehensive-theory-mind-benchmark/" --- [Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/tong-2026-cogtom-comprehensive-theory-mind-benchmark/) [home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/25/) / CogToM: A Comprehensive Theory of Mind Benchmark inspired by Human Cognition for Large Language Models # CogToM: A Comprehensive Theory of Mind Benchmark inspired by Human Cognition for Large Language Models **Authors:** Haibo Tong, Zeyang Yue, Feifei Zhao, Erliang Lin, Lu Jia, Ruolin Chen, Yinqian Sun, [Qian Zhang](https://md.research.elamur.ai/free/en/authors/qian-zhang/) and [Yi Zeng](https://md.research.elamur.ai/free/en/authors/yi-zeng/) · **Year:** 2026 · **Venue:** [Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)](https://md.research.elamur.ai/free/en/venues/acl/) ([ACL 2026](https://md.research.elamur.ai/free/en/venues/acl/2026/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/) **Links:** [Publisher's page](https://aclanthology.org/2026.acl-long.1448/) · [PDF](https://aclanthology.org/2026.acl-long.1448.pdf) · [DOI 10.18653/v1/2026.acl-long.1448](https://doi.org/10.18653/v1/2026.acl-long.1448) · [OpenAlex](https://openalex.org/W7166865324) **Citations:** 0 citations (OpenAlex, 2026-10-14) ## Summary Benchmarks that test Theory of Mind (ToM) in LLMs mostly rely on false-belief stories. This narrow focus covers only a small part of what developmental psychology studies. CogToM addresses the gap by converting 46 established and newly designed psychological paradigms into scene-based multiple-choice items in Chinese and English. Each item is linked to the ATOMS-based taxonomy of ToM abilities. The data were seeded by experts, expanded with GPT-5.1 and then checked through several rounds of human review and double annotation, with 49 people involved in total. The authors evaluated 22 LLMs of different sizes, families and release dates. Scores varied widely, which shows the benchmark can separate models well. Perception-related tasks were the hardest for every model, even though people find them trivial. Models also often did better on skills that children acquire late than on those acquired early. The authors interpret this as a form of Moravec's Paradox, suggesting that LLM "ToM" comes from linguistic pattern matching rather than grounded, human-like development. ## Key points - **Scale and structure:** CogToM contains 8,513 bilingual (Chinese-English) entries across 46 tasks, with 54–130 data groups per task. The tasks map onto 7 capability categories and 36 sub-capabilities. Four paradigms are entirely new, including Expanding Tasks [Synesthetic Fallacy] and a Spatial Construction Task. - **Construction:** A six-stage pipeline produced the data, and each entry passed at least five rounds of human supervision. GPT-5.1 generated the expansions, and 42 graduate students in Philosophy and AI performed the annotation. - **Models tested:** 22 LLMs were evaluated, including GPT-5.1 and Qwen3-Max. Chain-of-thought prompting improved Belief tasks and, for frontier models, Percept tasks. Even so, Percept remained the lowest-scoring dimension for all models. - **Human-model mismatch:** On tasks where human inter-annotator agreement exceeded 90%, average model accuracy dropped to 30–80%. On Percept tasks, humans agreed 100% while models scored below 30%. Most of these hard cases come from the newly introduced paradigms. - **Developmental inversion:** GPT-5.1 reached 62% on the early-childhood See-Know Task but 96% on the later-acquired TEC: Hidden Emotions task. Qwen-7B-Chat climbed from 5% to 71% along a sequence of tasks ordered by the age at which children master them, the reverse of the human pattern. - **Availability:** Code and data are released at github.com/Beijing-AISI/CogToM. *Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.* 📚 Shelves (7): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works · [👁️ Computer Vision and Pattern Recognition](https://md.research.elamur.ai/free/en/shelves/computer-science/computer-vision-and-pattern-recognition/) · 2,869 works · [💭 Psychology](https://md.research.elamur.ai/free/en/shelves/psychology/) · 1,552 works · [💭 Developmental and Educational Psychology](https://md.research.elamur.ai/free/en/shelves/psychology/developmental-and-educational-psychology/) · 499 works 🧭 Topics (2): [🧠 Language models & pretraining](https://md.research.elamur.ai/free/en/topics/language-models/) · 3,320 works · [📏 Evaluation & benchmarks](https://md.research.elamur.ai/free/en/topics/evaluation-benchmarks/) · 1,220 works ## Cite Cite as: Haibo Tong, Zeyang Yue, Feifei Zhao, Erliang Lin, Lu Jia, Ruolin Chen, Yinqian Sun, Qian Zhang and Yi Zeng. 2026. CogToM: A Comprehensive Theory of Mind Benchmark inspired by Human Cognition for Large Language Models. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). The paper itself, not this page. BibTeX from the library's record of the work. ```bibtex @inproceedings{tong2026cogtom, title = {{CogToM: A Comprehensive Theory of Mind Benchmark inspired by Human Cognition for Large Language Models}}, author = {Haibo Tong, Zeyang Yue, Feifei Zhao, Erliang Lin, Lu Jia, Ruolin Chen, Yinqian Sun, Qian Zhang and Yi Zeng}, booktitle = {Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)}, year = {2026}, doi = {10.18653/v1/2026.acl-long.1448}, url = {https://doi.org/10.18653/v1/2026.acl-long.1448} } ``` - [← previous Cognitive Effects and Biases in Large Language Models](https://md.research.elamur.ai/free/en/papers/schedl-2026-cognitive-effects-biases-large-language/) - [next → Cohere Labs Community at FoodBench-QA 2026: The Cake Makes the Ingredients](https://md.research.elamur.ai/free/en/papers/ranjan-2026-cohere-labs-community-foodbench-qa/) - [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/25/) ## References 2 references, 0 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet. 1. Despite its comprehensive scope, several limitations of our work should be acknowledged. [find](https://scholar.google.com/scholar?q=Despite+its+comprehensive+scope%2C+several+limitations+of+our+work+should+be+acknowledged%2E) 2. Frances Abell, Frances Happe, and Uta Frith. 2000. Do triangles play tricks? attribution of mental states to animated shapes in normal and abnormal development. Cognitive Development, 15(1):1–16. [find](https://scholar.google.com/scholar?q=Frances+Abell%2C+Frances+Happe%2C+and+Uta+Frith%2E+2000%2E+Do+triangles+play+tricks%3F+attribution+of+mental+states+to+animated+shapes+in+normal+and+abnormal+development%2E+Cognitive+Development%2C+15%281%29%3A1%E2%80%9316%2E) *Updated 1 November 2026*