---
title: "The Aftermath of DrawEduMath — paper by Lucy et al. (2026)"
description: "This paper revisits DrawEduMath, a QA benchmark built from real K-12 students' handwritten and hand-drawn math answers… 56 references."
authors: ["Li Lucy", "Albert Zhang", "Nathan Anderson", "Ryan Knight", "Kyle Lo"]
year: 2026
venue: "Proceedings of the 21st Workshop on Innovative Use of NLP for Building Educational Applications (BEA 2026)"
source: "https://aclanthology.org/2026.bea-1.5/"
doi: "10.18653/v1/2026.bea-1.5"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/lucy-2026-aftermath-drawedumath-vision-language-models/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/lucy-2026-aftermath-drawedumath-vision-language-models/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/45/) / The Aftermath of DrawEduMath: Vision Language Models Underperform with Struggling Students and Misdiagnose Errors

# The Aftermath of DrawEduMath: Vision Language Models Underperform with Struggling Students and Misdiagnose Errors

**Authors:** [Li Lucy](https://md.research.elamur.ai/free/en/authors/li-lucy/), Albert Zhang, Nathan Anderson, Ryan Knight and [Kyle Lo](https://md.research.elamur.ai/free/en/authors/kyle-lo/) · **Year:** 2026 · **Venue:** [Proceedings of the 21st Workshop on Innovative Use of NLP for Building Educational Applications (BEA 2026)](https://md.research.elamur.ai/free/en/venues/bea/) ([BEA 2026](https://md.research.elamur.ai/free/en/venues/bea/2026/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2026.bea-1.5/) · [PDF](https://aclanthology.org/2026.bea-1.5.pdf) · [DOI 10.18653/v1/2026.bea-1.5](https://doi.org/10.18653/v1/2026.bea-1.5) · [OpenAlex](https://openalex.org/W7166886316)

**Citations:** 0 citations (OpenAlex, 2026-10-14)

## Summary
This paper revisits DrawEduMath, a QA benchmark built from real K-12 students' handwritten and hand-drawn math answers, one year after its release. It asks whether vision-language models (VLMs) serve all students equally well. The authors evaluated 11 VLMs from 2025 on the benchmark's 2,030 teacher-annotated images. They scored answers with a refreshed LLM-judge ensemble and split student work into erroneous and error-free responses, using teachers' error descriptions to make that split.

Two patterns appear across every model:
- **F1:** VLMs describe flawed student work less accurately than correct work.
- **F2:** Questions about judging correctness and errors remain the hardest category.

Five follow-up analyses tested explanations for these patterns:
- The F1 gap persists after controlling for the specific problem and after reducing image noise.
- Some wrong answers on flawed work resemble the gold answers for correct work, suggesting models assume the math is right.
- Supplying gold textual descriptions improves error assessment, but not to the level of other question types.

The authors conclude that benchmarks for educational AI should be disaggregated by student proficiency. Otherwise, model abilities may be overstated and achievement gaps could widen.

## Key points
- **Models evaluated:** 11 VLMs in total: GPT-4.1, GPT-4.5 Preview, o4-mini and GPT-5 from OpenAI; Claude Sonnet 3.7/4/4.5 from Anthropic; three Gemini variants (2.0 Flash, 2.5 Pro and 2.5 Pro Preview) from Google; and Llama 4 Scout from Meta.
- **Data:** images come from ASSISTments and cover 188 problems, with about 12.64 student responses per problem. An estimated 85% of the images come from Title I schools.
- **Scoring:** the original Mixtral 8x22B judge was replaced by a majority vote of Claude Sonnet 4.5, Gemini 2.5 Pro and GPT-4o. The new judge correlates with human ratings at Spearman ρ = 0.808, versus 0.801 for the original.
- **Error effect:** an OLS regression with problem fixed effects finds a positive effect of student correctness on content-description accuracy, ranging from 0.0585 (GPT-5) to 0.1013 (Llama 4 Scout), with all p \< 1e-12. Error labels were assigned by GPT-5-mini and validated against a manual sample (F1 = 0.984, κ = 0.970).
- **Question types:** content description makes up 79.2% of questions, image creation/medium 12.3%, and correctness & errors 8.5%. On binary correctness questions, some models perform barely above chance, and their error-related behavior is idiosyncratic: some over-report errors, others miss them.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [University of Washington](https://md.research.elamur.ai/free/en/universities/university-of-washington/) — University, Seattle, United States · 353 works · 200,043 citations
- [Worcester Polytechnic Institute](https://md.research.elamur.ai/free/en/universities/worcester-polytechnic-institute/) — University, Worcester, United States · 8 works · 380 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [👁️ Computer Vision and Pattern Recognition](https://md.research.elamur.ai/free/en/shelves/computer-science/computer-vision-and-pattern-recognition/) · 2,869 works · [🗄️ Information Systems](https://md.research.elamur.ai/free/en/shelves/computer-science/information-systems/) · 1,437 works

🧭 Topics (2): [🧠 Language models & pretraining](https://md.research.elamur.ai/free/en/topics/language-models/) · 3,320 works · [🖼️ Vision & language](https://md.research.elamur.ai/free/en/topics/multimodal/) · 657 works

## Cite

Cite as: Li Lucy, Albert Zhang, Nathan Anderson, Ryan Knight and Kyle Lo. 2026. The Aftermath of DrawEduMath: Vision Language Models Underperform with Struggling Students and Misdiagnose Errors. Proceedings of the 21st Workshop on Innovative Use of NLP for Building Educational Applications (BEA 2026).

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{lucy2026aftermath,
  title = {{The Aftermath of DrawEduMath: Vision Language Models Underperform with Struggling Students and Misdiagnose Errors}},
  author = {Li Lucy, Albert Zhang, Nathan Anderson, Ryan Knight and Kyle Lo},
  booktitle = {Proceedings of the 21st Workshop on Innovative Use of NLP for Building Educational Applications (BEA 2026)},
  year = {2026},
  doi = {10.18653/v1/2026.bea-1.5},
  url = {https://doi.org/10.18653/v1/2026.bea-1.5}
}
```

- [← previous The African Language Tax: Quantifying the Cost, Latency, and Context Penalty of Tokenizing African Languages in Frontier LLMs](https://md.research.elamur.ai/free/en/papers/somide-2026-african-language-tax-quantifying-cost/)
- [next → The Algorithmic Unconscious: Structural Mechanisms and Implicit Biases in Large Language Models](https://md.research.elamur.ai/free/en/papers/boisnard-2026-algorithmic-unconscious-structural-mechanisms-implicit/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/45/)

## References

56 references, 5 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Shaden Alshammari, Kevin Wen, Abrar Zainal, Mark Hamilton, Navid Safaei, Sultan Albarakati, William T. Freeman, and Antonio Torralba. 2026. MathNet: A global multimodal benchmark for mathematical reasoning and retrieval. In The Fourteenth International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Shaden+Alshammari%2C+Kevin+Wen%2C+Abrar+Zainal%2C+Mark+Hamilton%2C+Navid+Safaei%2C+Sultan+Albarakati%2C+William+T%2E+Freeman%2C+and+Antonio+Torralba%2E+2026%2E+MathNet%3A+A+global+multimodal+benchmark+for+mathematical+reas)
2. Anwoy Chatterjee, H S V N S Kowndinya Renduchintala, Sumit Bhatia, and Tanmoy Chakraborty. 2024. POSIX: A prompt sensitivity index for large language models. In Findings of the Association for Computational Linguistics: EMNLP 2024, pages 14550– 14565, Miami, Florida, USA. Association for Computation… [find](https://scholar.google.com/scholar?q=Anwoy+Chatterjee%2C+H+S+V+N+S+Kowndinya+Renduchintala%2C+Sumit+Bhatia%2C+and+Tanmoy+Chakraborty%2E+2024%2E+POSIX%3A+A+prompt+sensitivity+index+for+large+language+models%2E+In+Findings+of+the+Association+for+Computa)
3. Lidia Altamura, Cristina Vargas, and Ladislao Salmerón. 2025. Do new forms of reading pay off? A meta-analysis on the relationship between leisure digital reading habits and text comprehension. Review of Educational Research, 95(1):53–88. [find](https://scholar.google.com/scholar?q=Lidia+Altamura%2C+Cristina+Vargas%2C+and+Ladislao+Salmer%C3%B3n%2E+2025%2E+Do+new+forms+of+reading+pay+off%3F+A+meta%2Danalysis+on+the+relationship+between+leisure+digital+reading+habits+and+text+comprehension%2E+Review)
4. Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman. 2021. [Training verifiers to solve math word problems](https://md.research.elamur.ai/free/en/papers/cobbe-2021-training-verifiers-solve-math-word/). Preprint, arXiv:2110.14168.
5. Lisa Anthony, Jie Yang, and Kenneth R. Koedinger. 2007. Benefits of handwritten input for students learning algebra equation solving. In Proceedings of the 2007 Conference on Artificial Intelligence in Education: Building Technology Rich Learning Contexts That Work, page 521–523, NLD. IOS Press. [find](https://scholar.google.com/scholar?q=Lisa+Anthony%2C+Jie+Yang%2C+and+Kenneth+R%2E+Koedinger%2E+2007%2E+Benefits+of+handwritten+input+for+students+learning+algebra+equation+solving%2E+In+Proceedings+of+the+2007+Conference+on+Artificial+Intelligence+i)
6. Josh Cowls, Andreas Tsamados, Mariarosaria Taddeo, and Luciano Floridi. 2021. A definition, benchmark and database of AI for social good initiatives. Nature Machine Intelligence, 3(2):111–115. [find](https://scholar.google.com/scholar?q=Josh+Cowls%2C+Andreas+Tsamados%2C+Mariarosaria+Taddeo%2C+and+Luciano+Floridi%2E+2021%2E+A+definition%2C+benchmark+and+database+of+AI+for+social+good+initiatives%2E+Nature+Machine+Intelligence%2C+3%282%29%3A111%E2%80%93115%2E)
7. Sami Baral, Li Lucy, Ryan Knight, Alice Ng, Luca Soldaini, Neil Heffernan, and Kyle Lo. 2024. Drawedumath: An expert-annotated dataset of students’ math images. In The 4th Workshop on Mathematical Reasoning and AI at NeurIPS’24. [find](https://scholar.google.com/scholar?q=Sami+Baral%2C+Li+Lucy%2C+Ryan+Knight%2C+Alice+Ng%2C+Luca+Soldaini%2C+Neil+Heffernan%2C+and+Kyle+Lo%2E+2024%2E+Drawedumath%3A+An+expert%2Dannotated+dataset+of+students%E2%80%99+math+images%2E+In+The+4th+Workshop+on+Mathematical+Rea)
8. Nico Daheim, Jakub Macina, Manu Kapur, Iryna Gurevych, and Mrinmaya Sachan. 2024. Stepwise verification and remediation of student reasoning errors with large language model tutors. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 8386–8411, Miami, Flo… [find](https://scholar.google.com/scholar?q=Nico+Daheim%2C+Jakub+Macina%2C+Manu+Kapur%2C+Iryna+Gurevych%2C+and+Mrinmaya+Sachan%2E+2024%2E+Stepwise+verification+and+remediation+of+student+reasoning+errors+with+large+language+model+tutors%2E+In+Proceedings+of+)
9. Sami Baral, Li Lucy, Ryan Knight, Alice Ng, Luca Soldaini, Neil Heffernan, and Kyle Lo. 2025. DrawEduMath: Evaluating vision language models with expert-annotated students’ hand-drawn math images. In Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Com… [find](https://scholar.google.com/scholar?q=Sami+Baral%2C+Li+Lucy%2C+Ryan+Knight%2C+Alice+Ng%2C+Luca+Soldaini%2C+Neil+Heffernan%2C+and+Kyle+Lo%2E+2025%2E+DrawEduMath%3A+Evaluating+vision+language+models+with+expert%2Dannotated+students%E2%80%99+hand%2Ddrawn+math+images%2E+In+)
10. Dorottya Demszky, Jing Liu, Heather C. Hill, Shyamoli Sanghi, and Ariel Chung. 2025. Automated feedback improves teachers’ questioning quality in brick-and-mortar classrooms: Opportunities for further enhancement. Computers & Education, 227:105183. [find](https://scholar.google.com/scholar?q=Dorottya+Demszky%2C+Jing+Liu%2C+Heather+C%2E+Hill%2C+Shyamoli+Sanghi%2C+and+Ariel+Chung%2E+2025%2E+Automated+feedback+improves+teachers%E2%80%99+questioning+quality+in+brick%2Dand%2Dmortar+classrooms%3A+Opportunities+for+further)
11. Mingyu Feng, Linlin Li, Chunwei Huang, Natalie Brezack, and Kim Luttgen. 2025. Empowering teachers with technology: A national study on a formative assessment platform. In International Conference on Artificial Intelligence in Education, pages 119–126. Springer. [find](https://scholar.google.com/scholar?q=Mingyu+Feng%2C+Linlin+Li%2C+Chunwei+Huang%2C+Natalie+Brezack%2C+and+Kim+Luttgen%2E+2025%2E+Empowering+teachers+with+technology%3A+A+national+study+on+a+formative+assessment+platform%2E+In+International+Conference+on+)
12. Hamsa Bastani, Osbert Bastani, Alp Sungu, Haosen Ge, Özge Kabakcı, and Rei Mariman. 2025. Generative AI without guardrails can harm learning: Evidence from high school mathematics. Proceedings of the National Academy of Sciences, 122(26):e2422633122. [find](https://scholar.google.com/scholar?q=Hamsa+Bastani%2C+Osbert+Bastani%2C+Alp+Sungu%2C+Haosen+Ge%2C+%C3%96zge+Kabakc%C4%B1%2C+and+Rei+Mariman%2E+2025%2E+Generative+AI+without+guardrails+can+harm+learning%3A+Evidence+from+high+school+mathematics%2E+Proceedings+of+the+)
13. Esther Gan, Yiran Zhao, Liying Cheng, Mao Yancan, Anirudh Goyal, Kenji Kawaguchi, Min-Yen Kan, and Michael Shieh. 2024. Reasoning robustness of LLMs to adversarial typographical errors. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 10449–10459, Miam… [find](https://scholar.google.com/scholar?q=Esther+Gan%2C+Yiran+Zhao%2C+Liying+Cheng%2C+Mao+Yancan%2C+Anirudh+Goyal%2C+Kenji+Kawaguchi%2C+Min%2DYen+Kan%2C+and+Michael+Shieh%2E+2024%2E+Reasoning+robustness+of+LLMs+to+adversarial+typographical+errors%2E+In+Proceedings)
14. Su Lin Blodgett and Michael Madaio. 2021. Risks of AI foundation models in education. arXiv preprint arXiv:2110.10024. — [arXiv:2110.10024](https://arxiv.org/abs/2110.10024) [find](https://scholar.google.com/scholar?q=Su+Lin+Blodgett+and+Michael+Madaio%2E+2021%2E+Risks+of+AI+foundation+models+in+education%2E+arXiv+preprint+arXiv%3A2110%2E10024%2E+%E2%80%94+%5BarXiv%3A2110%2E10024%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2110%2E10024%29)
15. Raffaella Borasi. 1994. Capitalizing on errors as “springboards for inquiry”: A teaching experiment. Journal for Research in Mathematics Education, 25(2):166–208. [find](https://scholar.google.com/scholar?q=Raffaella+Borasi%2E+1994%2E+Capitalizing+on+errors+as+%E2%80%9Cspringboards+for+inquiry%E2%80%9D%3A+A+teaching+experiment%2E+Journal+for+Research+in+Mathematics+Education%2C+25%282%29%3A166%E2%80%93208%2E)
16. Anthony Botelho, Sami Baral, John A. Erickson, Priyanka Benachamardi, and Neil T. Heffernan. 2023. Leveraging natural language processing to support automated assessment and feedback for student open responses in mathematics. Journal of Computer Assisted Learning, 39(3):823–840. [find](https://scholar.google.com/scholar?q=Anthony+Botelho%2C+Sami+Baral%2C+John+A%2E+Erickson%2C+Priyanka+Benachamardi%2C+and+Neil+T%2E+Heffernan%2E+2023%2E+Leveraging+natural+language+processing+to+support+automated+assessment+and+feedback+for+student+open+)
17. Google DeepMind. 2025. Gemini 3 Pro model card. [find](https://scholar.google.com/scholar?q=Google+DeepMind%2E+2025%2E+Gemini+3+Pro+model+card%2E)
18. Dorit Hadar Shoval. 2025. Artificial intelligence in higher education: Bridging or widening the gap for diverse student populations? Education Sciences, 15(5). [find](https://scholar.google.com/scholar?q=Dorit+Hadar+Shoval%2E+2025%2E+Artificial+intelligence+in+higher+education%3A+Bridging+or+widening+the+gap+for+diverse+student+populations%3F+Education+Sciences%2C+15%285%29%2E)
19. Emma Harvey, Allison Koenecke, and Rene F Kizilcec. 2024. Towards an educator-centered method for measuring bias in large language model-based chatbot tutors. In AI for Education: Bridging Innovation and Responsibility at the 38th AAAI Annual Conference on AI. [find](https://scholar.google.com/scholar?q=Emma+Harvey%2C+Allison+Koenecke%2C+and+Rene+F+Kizilcec%2E+2024%2E+Towards+an+educator%2Dcentered+method+for+measuring+bias+in+large+language+model%2Dbased+chatbot+tutors%2E+In+AI+for+Education%3A+Bridging+Innovation+)
20. Stefan Küchemann, Karina E Avila, Yavuz Dinc, Chiara Hortmann, Natalia Revenga, Verena Ruf, Niklas Stausberg, Steffen Steinert, Frank Fischer, Martin Fischer, and 1 others. 2025. On opportunities and challenges of large multimodal foundation models in education. npj Science of Learning, 10(1):11. [find](https://scholar.google.com/scholar?q=Stefan+K%C3%BCchemann%2C+Karina+E+Avila%2C+Yavuz+Dinc%2C+Chiara+Hortmann%2C+Natalia+Revenga%2C+Verena+Ruf%2C+Niklas+Stausberg%2C+Steffen+Steinert%2C+Frank+Fischer%2C+Martin+Fischer%2C+and+1+others%2E+2025%2E+On+opportunities+and+)
21. Emma Harvey, Allison Koenecke, and Rene F. Kizilcec. 2025. "Don’t forget the teachers": Towards an educator-centered understanding of harms from large language models in education. In Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems, CHI ’25, New York, NY, USA. Associatio… [find](https://scholar.google.com/scholar?q=Emma+Harvey%2C+Allison+Koenecke%2C+and+Rene+F%2E+Kizilcec%2E+2025%2E+%22Don%E2%80%99t+forget+the+teachers%22%3A+Towards+an+educator%2Dcentered+understanding+of+harms+from+large+language+models+in+education%2E+In+Proceedings+of+t)
22. Ehsan Latif, Zirak Khan, and Xiaoming Zhai. 2025. SketchMind: A multi-agent cognitive framework for assessing student-drawn scientific sketches. arXiv preprint arXiv:2507.22904. — [arXiv:2507.22904](https://arxiv.org/abs/2507.22904) [find](https://scholar.google.com/scholar?q=Ehsan+Latif%2C+Zirak+Khan%2C+and+Xiaoming+Zhai%2E+2025%2E+SketchMind%3A+A+multi%2Dagent+cognitive+framework+for+assessing+student%2Ddrawn+scientific+sketches%2E+arXiv+preprint+arXiv%3A2507%2E22904%2E+%E2%80%94+%5BarXiv%3A2507%2E22904%5D%28h)
23. Hao-Ping (Hank) Lee, Advait Sarkar, Lev Tankelevitch, Ian Drosos, Sean Rintel, Richard Banks, and Nicholas Wilson. 2025a. The impact of generative ai on critical thinking: Self-reported reductions in cognitive effort and confidence effects from a survey of knowledge workers. In Proceedings of the 20… [find](https://scholar.google.com/scholar?q=Hao%2DPing+%28Hank%29+Lee%2C+Advait+Sarkar%2C+Lev+Tankelevitch%2C+Ian+Drosos%2C+Sean+Rintel%2C+Richard+Banks%2C+and+Nicholas+Wilson%2E+2025a%2E+The+impact+of+generative+ai+on+critical+thinking%3A+Self%2Dreported+reductions+in+)
24. Neil T Heffernan and Cristina Lindquist Heffernan. 2014. The ASSISTments ecosystem: Building a platform that brings scientists and teachers together for minimally invasive research on human learning and teaching. International Journal of Artificial Intelligence in Education, 24(4):470–497. [find](https://scholar.google.com/scholar?q=Neil+T+Heffernan+and+Cristina+Lindquist+Heffernan%2E+2014%2E+The+ASSISTments+ecosystem%3A+Building+a+platform+that+brings+scientists+and+teachers+together+for+minimally+invasive+research+on+human+learning+a)
25. Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. 2021. [Measuring mathematical problem solving with the MATH dataset](https://md.research.elamur.ai/free/en/papers/hendrycks-2021-measuring-mathematical-problem-solving-math/). In Thirty-fifth Conference on Neural Information Processing Systems Datasets and Benchmarks Track (Round 2).
26. Jimin Lee, Steven-Shine Chen, and Paul Pu Liang. 2025b. Interactive Sketchpad: A multimodal tutoring system for collaborative, visual problem-solving. In Proceedings of the Extended Abstracts of the CHI Conference on Human Factors in Computing Systems, CHI EA ’25, New York, NY, USA. Association for… [find](https://scholar.google.com/scholar?q=Jimin+Lee%2C+Steven%2DShine+Chen%2C+and+Paul+Pu+Liang%2E+2025b%2E+Interactive+Sketchpad%3A+A+multimodal+tutoring+system+for+collaborative%2C+visual+problem%2Dsolving%2E+In+Proceedings+of+the+Extended+Abstracts+of+the+C)
27. Kenneth Holstein and Shayan Doroudi. 2021. Equity and artificial intelligence in education: Will "AIEd" amplify or alleviate inequities in education? Preprint, arXiv:2104.12920. — [arXiv:2104.12920](https://arxiv.org/abs/2104.12920) [find](https://scholar.google.com/scholar?q=Kenneth+Holstein+and+Shayan+Doroudi%2E+2021%2E+Equity+and+artificial+intelligence+in+education%3A+Will+%22AIEd%22+amplify+or+alleviate+inequities+in+education%3F+Preprint%2C+arXiv%3A2104%2E12920%2E+%E2%80%94+%5BarXiv%3A2104%2E12920%5D%28h)
28. Shayne Longpre, Gregory Yauney, Emily Reif, Katherine Lee, Adam Roberts, Barret Zoph, Denny Zhou, Jason Wei, Kevin Robinson, David Mimno, and Daphne Ippolito. 2024. A pretrainer’s guide to training data: Measuring the effects of data age, domain coverage, quality, & toxicity. In Proceedings of the 2… [find](https://scholar.google.com/scholar?q=Shayne+Longpre%2C+Gregory+Yauney%2C+Emily+Reif%2C+Katherine+Lee%2C+Adam+Roberts%2C+Barret+Zoph%2C+Denny+Zhou%2C+Jason+Wei%2C+Kevin+Robinson%2C+David+Mimno%2C+and+Daphne+Ippolito%2E+2024%2E+A+pretrainer%E2%80%99s+guide+to+training+da)
29. Hyoungwook Jin, Yoonsu Kim, Dongyun Jung, Seungju Kim, Kiyoon Choi, Jinho Son, and Juho Kim. 2025. Investigating large language models in diagnosing students’ cognitive skills in math problem-solving. arXiv preprint arXiv:2504.00843. — [arXiv:2504.00843](https://arxiv.org/abs/2504.00843) [find](https://scholar.google.com/scholar?q=Hyoungwook+Jin%2C+Yoonsu+Kim%2C+Dongyun+Jung%2C+Seungju+Kim%2C+Kiyoon+Choi%2C+Jinho+Son%2C+and+Juho+Kim%2E+2025%2E+Investigating+large+language+models+in+diagnosing+students%E2%80%99+cognitive+skills+in+math+problem%2Dsolving%2E)
30. Rogers Kaliisa, Kamila Misiejuk, Sonsoles López-Pernas, and Mohammed Saqr. 2026. How does artificial intelligence compare to human feedback? a meta-analysis of performance, feedback perception, and learning dispositions. Educational Psychology, 46(1):80–111. [find](https://scholar.google.com/scholar?q=Rogers+Kaliisa%2C+Kamila+Misiejuk%2C+Sonsoles+L%C3%B3pez%2DPernas%2C+and+Mohammed+Saqr%2E+2026%2E+How+does+artificial+intelligence+compare+to+human+feedback%3F+a+meta%2Danalysis+of+performance%2C+feedback+perception%2C+and+le)
31. Pan Lu, Hritik Bansal, Tony Xia, Jiacheng Liu, Chunyuan Li, Hannaneh Hajishirzi, Hao Cheng, Kai-Wei Chang, Michel Galley, and Jianfeng Gao. 2024. [MathVista: Evaluating mathematical reasoning of foundation models in visual contexts](https://md.research.elamur.ai/free/en/papers/lu-2023-mathvista-evaluating-mathematical-reasoning-foundation/). In The Twelfth International Conference on Learning Representations.
32. Blanka Klimova and Marcel Pikhart. 2025. Exploring the effects of artificial intelligence on student and academic well-being in higher education: A mini-review. Frontiers in Psychology, 16:1498132. [find](https://scholar.google.com/scholar?q=Blanka+Klimova+and+Marcel+Pikhart%2E+2025%2E+Exploring+the+effects+of+artificial+intelligence+on+student+and+academic+well%2Dbeing+in+higher+education%3A+A+mini%2Dreview%2E+Frontiers+in+Psychology%2C+16%3A1498132%2E)
33. Rabeeh Karimi Mahabadi, Sanjeev Satheesh, Shrimai Prabhumoye, Mostofa Patwary, Mohammad Shoeybi, and Bryan Catanzaro. 2025. Nemotron-CC-Math: A 133 billion-token-scale high quality math pretraining dataset. Preprint, arXiv:2508.15096. [find](https://scholar.google.com/scholar?q=Rabeeh+Karimi+Mahabadi%2C+Sanjeev+Satheesh%2C+Shrimai+Prabhumoye%2C+Mostofa+Patwary%2C+Mohammad+Shoeybi%2C+and+Bryan+Catanzaro%2E+2025%2E+Nemotron%2DCC%2DMath%3A+A+133+billion%2Dtoken%2Dscale+high+quality+math+pretraining+da)
34. Ekaterina Kochmar, Kaushal Maurya, Kseniia Petukhova, KV Aditya Srivatsa, Anaïs Tack, and Justin Vasselli. 2025. Findings of the BEA 2025 shared task on pedagogical ability assessment of AI-powered tutors. In Proceedings of the 20th Workshop on Innovative Use of NLP for Building Educational Applicat… [find](https://scholar.google.com/scholar?q=Ekaterina+Kochmar%2C+Kaushal+Maurya%2C+Kseniia+Petukhova%2C+KV+Aditya+Srivatsa%2C+Ana%C3%AFs+Tack%2C+and+Justin+Vasselli%2E+2025%2E+Findings+of+the+BEA+2025+shared+task+on+pedagogical+ability+assessment+of+AI%2Dpowered+tu)
35. Janet Metcalfe. 2017. Learning from errors. Annual Review of Psychology, 68:465–489. [find](https://scholar.google.com/scholar?q=Janet+Metcalfe%2E+2017%2E+Learning+from+errors%2E+Annual+Review+of+Psychology%2C+68%3A465%E2%80%93489%2E)
36. Masato Mita, Keisuke Sakaguchi, Masato Hagiwara, Tomoya Mizumoto, Jun Suzuki, and Kentaro Inui. 2024. Towards automated document revision: Grammatical error correction, fluency edits, and beyond. In Proceedings of the 19th Workshop on Innovative Use of NLP for Building Educational Applications Tianq… [find](https://scholar.google.com/scholar?q=Masato+Mita%2C+Keisuke+Sakaguchi%2C+Masato+Hagiwara%2C+Tomoya+Mizumoto%2C+Jun+Suzuki%2C+and+Kentaro+Inui%2E+2024%2E+Towards+automated+document+revision%3A+Grammatical+error+correction%2C+fluency+edits%2C+and+beyond%2E+In+P)
37. (BEA 2024), pages 251–265, Mexico City, Mexico. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=%28BEA+2024%29%2C+pages+251%E2%80%93265%2C+Mexico+City%2C+Mexico%2E+Association+for+Computational+Linguistics%2E)
38. Alexis Ross and Jacob Andreas. 2025. Learning to make MISTAKEs: Modeling incorrect student thinking and key errors. Preprint, arXiv:2510.11502. — [arXiv:2510.11502](https://arxiv.org/abs/2510.11502) [find](https://scholar.google.com/scholar?q=Alexis+Ross+and+Jacob+Andreas%2E+2025%2E+Learning+to+make+MISTAKEs%3A+Modeling+incorrect+student+thinking+and+key+errors%2E+Preprint%2C+arXiv%3A2510%2E11502%2E+%E2%80%94+%5BarXiv%3A2510%2E11502%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2510%2E11502%29)
39. Pam A. Mueller and Daniel M. Oppenheimer. 2014. The pen is mightier than the keyboard: Advantages of longhand over laptop note taking. Psychological Science, 25(6):1159–1168. PMID: 24760141. [find](https://scholar.google.com/scholar?q=Pam+A%2E+Mueller+and+Daniel+M%2E+Oppenheimer%2E+2014%2E+The+pen+is+mightier+than+the+keyboard%3A+Advantages+of+longhand+over+laptop+note+taking%2E+Psychological+Science%2C+25%286%29%3A1159%E2%80%931168%2E+PMID%3A+24760141%2E)
40. Nils-Jonathan Schaller, Yuning Ding, Andrea Horbach, Jennifer Meyer, and Thorben Jansen. 2024. Fairness in automated essay scoring: A comparative analysis of algorithms on German learner essays from secondary education. In Proceedings of the 19th Workshop on Innovative Use of NLP for Building Educat… [find](https://scholar.google.com/scholar?q=Nils%2DJonathan+Schaller%2C+Yuning+Ding%2C+Andrea+Horbach%2C+Jennifer+Meyer%2C+and+Thorben+Jansen%2E+2024%2E+Fairness+in+automated+essay+scoring%3A+A+comparative+analysis+of+algorithms+on+German+learner+essays+from+s)
41. Oikantik Nath, Hanani Bathina, Mohammed Safi Ur Rahman Khan, and Mitesh M Khapra. 2025. Can vision-language models evaluate handwritten math? In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 14784–14814, Vienna, Austria. Associ… [find](https://scholar.google.com/scholar?q=Oikantik+Nath%2C+Hanani+Bathina%2C+Mohammed+Safi+Ur+Rahman+Khan%2C+and+Mitesh+M+Khapra%2E+2025%2E+Can+vision%2Dlanguage+models+evaluate+handwritten+math%3F+In+Proceedings+of+the+63rd+Annual+Meeting+of+the+Associati)
42. Judy Hanwen Shen and Alex Tamkin. 2026. How AI impacts skill formation. arXiv preprint arXiv:2601.20245. — [arXiv:2601.20245](https://arxiv.org/abs/2601.20245) [find](https://scholar.google.com/scholar?q=Judy+Hanwen+Shen+and+Alex+Tamkin%2E+2026%2E+How+AI+impacts+skill+formation%2E+arXiv+preprint+arXiv%3A2601%2E20245%2E+%E2%80%94+%5BarXiv%3A2601%2E20245%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2601%2E20245%29)
43. Thu Phuong Nguyen, Duc M. Nguyen, Hyotaek Jeon, Hyunwook Lee, Hyunmin Song, Sungahn Ko, and Taehwan Kim. 2025. VEHME: A vision-language model for evaluating handwritten mathematics expressions. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pages 31793–318… [find](https://scholar.google.com/scholar?q=Thu+Phuong+Nguyen%2C+Duc+M%2E+Nguyen%2C+Hyotaek+Jeon%2C+Hyunwook+Lee%2C+Hyunmin+Song%2C+Sungahn+Ko%2C+and+Taehwan+Kim%2E+2025%2E+VEHME%3A+A+vision%2Dlanguage+model+for+evaluating+handwritten+mathematics+expressions%2E+In+Pro)
44. John P Smith III, Andrea A. diSessa, and Jeremy Roschelle. 1994. Misconceptions reconceived: A constructivist analysis of knowledge in transition. Journal of the Learning Sciences, 3(2):115–163. [find](https://scholar.google.com/scholar?q=John+P+Smith+III%2C+Andrea+A%2E+diSessa%2C+and+Jeremy+Roschelle%2E+1994%2E+Misconceptions+reconceived%3A+A+constructivist+analysis+of+knowledge+in+transition%2E+Journal+of+the+Learning+Sciences%2C+3%282%29%3A115%E2%80%93163%2E)
45. Neha Srikanth, Rupak Sarkar, Heran Mane, Elizabeth Aparicio, Quynh Nguyen, Rachel Rudinger, and Jordan Boyd-Graber. 2024. [Pregnant questions: The importance of pragmatic awareness in maternal health question answering](https://md.research.elamur.ai/free/en/papers/srikanth-2024-pregnant-questions-importance-pragmatic-awareness/). In Proceedings of the 2024 Conference of the North American Chapter of the Associ…
46. Behnam Parsaeifard, Martin Hlosta, and Per Bergamin. 2025. Automated grading of students’ handwritten graphs: A comparison of meta-learning and vision-large language models. arXiv preprint arXiv:2507.03056. — [arXiv:2507.03056](https://arxiv.org/abs/2507.03056) [find](https://scholar.google.com/scholar?q=Behnam+Parsaeifard%2C+Martin+Hlosta%2C+and+Per+Bergamin%2E+2025%2E+Automated+grading+of+students%E2%80%99+handwritten+graphs%3A+A+comparison+of+meta%2Dlearning+and+vision%2Dlarge+language+models%2E+arXiv+preprint+arXiv%3A2507%2E)
47. Keiran Paster, Marco Dos Santos, Zhangir Azerbayev, and Jimmy Ba. 2024. [OpenWebMath: An open dataset of high-quality mathematical web text](https://md.research.elamur.ai/free/en/papers/paster-2023-openwebmath-open-dataset-high-quality/). In The Twelfth International Conference on Learning Representations.
48. Kv Aditya Srivatsa, Kaushal Kumar Maurya, and Ekaterina Kochmar. 2025. LLMs cannot spot math errors, even when allowed to peek into the solution. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pages 10914–10928, Suzhou, China. Association for Computational… [find](https://scholar.google.com/scholar?q=Kv+Aditya+Srivatsa%2C+Kaushal+Kumar+Maurya%2C+and+Ekaterina+Kochmar%2E+2025%2E+LLMs+cannot+spot+math+errors%2C+even+when+allowed+to+peek+into+the+solution%2E+In+Proceedings+of+the+2025+Conference+on+Empirical+Met)
49. Hendrik Radatz. 1979. Error analysis in mathematics education. Journal for Research in mathematics Education, 10(3):163–172. [find](https://scholar.google.com/scholar?q=Hendrik+Radatz%2E+1979%2E+Error+analysis+in+mathematics+education%2E+Journal+for+Research+in+mathematics+Education%2C+10%283%29%3A163%E2%80%93172%2E)
50. Inioluwa Deborah Raji, I Elizabeth Kumar, Aaron Horowitz, and Andrew Selbst. 2022. The fallacy of AI functionality. In Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency, pages 959–972. [find](https://scholar.google.com/scholar?q=Inioluwa+Deborah+Raji%2C+I+Elizabeth+Kumar%2C+Aaron+Horowitz%2C+and+Andrew+Selbst%2E+2022%2E+The+fallacy+of+AI+functionality%2E+In+Proceedings+of+the+2022+ACM+Conference+on+Fairness%2C+Accountability%2C+and+Transpare)
51. Maja Stahl, Leon Biermann, Andreas Nehring, and Henning Wachsmuth. 2024. Exploring LLM prompting strategies for joint essay scoring and feedback generation. In Proceedings of the 19th Workshop on Innovative Use of NLP for Building Educational Applications (BEA 2024), pages 283–298, Mexico City, Mexi… [find](https://scholar.google.com/scholar?q=Maja+Stahl%2C+Leon+Biermann%2C+Andreas+Nehring%2C+and+Henning+Wachsmuth%2E+2024%2E+Exploring+LLM+prompting+strategies+for+joint+essay+scoring+and+feedback+generation%2E+In+Proceedings+of+the+19th+Workshop+on+Inno)
52. Inioluwa Deborah Raji, Andrew Smart, Rebecca N. White, Margaret Mitchell, Timnit Gebru, Ben Hutchinson, Jamila Smith-Loud, Daniel Theron, and Parker Barnes. 2020. Closing the AI accountability gap: defining an end-to-end framework for internal algorithmic auditing. In Proceedings of the 2020 Confere… [find](https://scholar.google.com/scholar?q=Inioluwa+Deborah+Raji%2C+Andrew+Smart%2C+Rebecca+N%2E+White%2C+Margaret+Mitchell%2C+Timnit+Gebru%2C+Ben+Hutchinson%2C+Jamila+Smith%2DLoud%2C+Daniel+Theron%2C+and+Parker+Barnes%2E+2020%2E+Closing+the+AI+accountability+gap%3A+de)
53. Keita Umejima, Takuya Ibaraki, Takahiro Yamazaki, and Kuniyoshi L Sakai. 2021. Paper notebooks vs. mobile devices: Brain activation differences during memory retrieval. Frontiers in Behavioral Neuroscience, 15:634158. [find](https://scholar.google.com/scholar?q=Keita+Umejima%2C+Takuya+Ibaraki%2C+Takahiro+Yamazaki%2C+and+Kuniyoshi+L+Sakai%2E+2021%2E+Paper+notebooks+vs%2E+mobile+devices%3A+Brain+activation+differences+during+memory+retrieval%2E+Frontiers+in+Behavioral+Neurosc)
54. Angelina Wang, Sayash Kapoor, Solon Barocas, and Arvind Narayanan. 2024. Against predictive optimization: On the legitimacy of decision-making algorithms that optimize predictive accuracy. ACM J. Responsib. Comput., 1(1). [find](https://scholar.google.com/scholar?q=Angelina+Wang%2C+Sayash+Kapoor%2C+Solon+Barocas%2C+and+Arvind+Narayanan%2E+2024%2E+Against+predictive+optimization%3A+On+the+legitimacy+of+decision%2Dmaking+algorithms+that+optimize+predictive+accuracy%2E+ACM+J%2E+Resp)
55. Bethany Rittle-Johnson, Rebecca Adler, Kelley Durkin, L Burleigh, Jules King, and Scott Crossley. 2025. Detecting math misconceptions: An AI benchmark dataset. In Proceedings of the Artificial Intelligence in Measurement and Education Conference (AIME-Con): Works in Progress, pages 20–24, Wyndham Gr… [find](https://scholar.google.com/scholar?q=Bethany+Rittle%2DJohnson%2C+Rebecca+Adler%2C+Kelley+Durkin%2C+L+Burleigh%2C+Jules+King%2C+and+Scott+Crossley%2E+2025%2E+Detecting+math+misconceptions%3A+An+AI+benchmark+dataset%2E+In+Proceedings+of+the+Artificial+Intelli)
56. Ryan Yixiang Wang, Matthew Finlayson, Luca Soldaini, Swabha Swayamdipta, and Robin Jia. 2025. Teaching models to understand (but not generate) high-risk data. In Second Conference on Language Modeling. [find](https://scholar.google.com/scholar?q=Ryan+Yixiang+Wang%2C+Matthew+Finlayson%2C+Luca+Soldaini%2C+Swabha+Swayamdipta%2C+and+Robin+Jia%2E+2025%2E+Teaching+models+to+understand+%28but+not+generate%29+high%2Drisk+data%2E+In+Second+Conference+on+Language+Modeling)

*Updated 1 November 2026*