---
title: "Automatic Generation of Socratic Subquestions for Teaching"
description: "The paper investigates whether large language models can automatically produce Socratic-style… 59 references, cited by 34 papers here."
authors: ["Kumar Shridhar", "Jakub Macina", "Mennatallah El-Assady", "Tanmay Sinha", "Manu Kapur", "Mrinmaya Sachan"]
year: 2022
venue: "Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing"
source: "https://aclanthology.org/2022.emnlp-main.277/"
doi: "10.18653/v1/2022.emnlp-main.277"
arxiv: "2211.12835"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/shridhar-2022-automatic-generation-socratic-subquestions-teaching/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/shridhar-2022-automatic-generation-socratic-subquestions-teaching/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2022](https://md.research.elamur.ai/free/en/papers/2022/page/5/) / Automatic Generation of Socratic Subquestions for Teaching Math Word Problems

# Automatic Generation of Socratic Subquestions for Teaching Math Word Problems

**Authors:** [Kumar Shridhar](https://md.research.elamur.ai/free/en/authors/kumar-shridhar/), [Jakub Macina](https://md.research.elamur.ai/free/en/authors/jakub-macina/), [Mennatallah El-Assady](https://md.research.elamur.ai/free/en/authors/mennatallah-el-assady/), [Tanmay Sinha](https://md.research.elamur.ai/free/en/authors/tanmay-sinha/), [Manu Kapur](https://md.research.elamur.ai/free/en/authors/manu-kapur/) and [Mrinmaya Sachan](https://md.research.elamur.ai/free/en/authors/mrinmaya-sachan/) · **Year:** 2022 · **Venue:** [Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing](https://md.research.elamur.ai/free/en/venues/emnlp/) ([EMNLP 2022](https://md.research.elamur.ai/free/en/venues/emnlp/2022/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2022.emnlp-main.277/) · [PDF](https://aclanthology.org/2022.emnlp-main.277.pdf) · [DOI 10.18653/v1/2022.emnlp-main.277](https://doi.org/10.18653/v1/2022.emnlp-main.277) · [arXiv 2211.12835](https://arxiv.org/abs/2211.12835) · [OpenAlex](https://openalex.org/W4385573161)

**Citations:** 36 citations (OpenAlex, 2026-10-14) · [34](https://md.research.elamur.ai/free/en/papers/shridhar-2022-automatic-generation-socratic-subquestions-teaching/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
The paper investigates whether large language models can automatically produce Socratic-style guidance for math word problems (MWPs). In this approach, each solution step becomes a sub-question that leads a learner, or a solver model, toward the answer. The authors first show that the order, granularity and content of sub-questions all affect how well a QA model answers. From this they derive two target properties: questions should be *focused* on critical content and *goal-driven* toward the final answer.

Using the GSM8K dataset, they train a transformer encoder-decoder question generator. It is conditioned on a content planner that predicts the operations needed, and it is fine-tuned with reinforcement learning using rewards for fluency, granularity and answerability. In both automatic and human evaluation, these constrained models produce better question sequences, and adding the generated questions as context improves an MWP solver.

A small user study found no overall gain for human learners. Sub-questions appeared to help on easier problems but hurt on harder ones. This suggests that educational deployment needs higher question quality and adaptation to individual learners.

## Key points
- Data: GSM8K multi-step MWPs. Each solution step s_i is mapped to one Socratic sub-question q_i.
- Method: a content planner (equations/operators) is combined with RL fine-tuning. The reward is R = R_fl + R_g + R_ans (fluency, granularity, answerability).
- Ablations show that permuting, re-granularizing or altering sub-questions lowers QA model accuracy.
- User study:
  - First-attempt success was 58.4% for the no-questions group and 66.0% for the questions group.
  - Second-attempt success was 35.8% vs. 31.0%, which is not significant (p = 0.659, Cohen's d = 0.157).
  - The questions group spent significantly more time on the second attempt.
- Code: github.com/eth-nlped/scaffolding-generation. The authors flag the user study as underpowered.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [ETH Zurich](https://md.research.elamur.ai/free/en/universities/eth-zurich/) — University, Zurich, Switzerland · 97 works · 9,612 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 19,503 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 18,805 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 17,959 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 17,789 works

🧭 Topics (1): [🧮 Reasoning](https://md.research.elamur.ai/free/en/topics/reasoning/) · 1,948 works

## Cite

Cite as: Kumar Shridhar, Jakub Macina, Mennatallah El-Assady, Tanmay Sinha, Manu Kapur and Mrinmaya Sachan. 2022. Automatic Generation of Socratic Subquestions for Teaching Math Word Problems. Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{shridhar2022automatic,
  title = {{Automatic Generation of Socratic Subquestions for Teaching Math Word Problems}},
  author = {Kumar Shridhar, Jakub Macina, Mennatallah El-Assady, Tanmay Sinha, Manu Kapur and Mrinmaya Sachan},
  booktitle = {Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing},
  year = {2022},
  doi = {10.18653/v1/2022.emnlp-main.277},
  eprint = {2211.12835},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.18653/v1/2022.emnlp-main.277}
}
```

- [← previous The SIGMORPHON 2022 Shared Task on Morpheme Segmentation](https://md.research.elamur.ai/free/en/papers/batsuren-2022-morpheme-segmentation/)
- [next → POLITICS: Pretraining with Same-story Article Comparison for Ideology Prediction and Stance Detection](https://md.research.elamur.ai/free/en/papers/liu-2022-politics-pretraining-same-story-article/)
- [↑ back to papers of 2022](https://md.research.elamur.ai/free/en/papers/2022/page/5/)

## References

59 references, 26 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Julia Anghileri. 2006. Scaffolding practices that enhance mathematics learning. Journal of Mathematics Teacher Education, 9(1):33–52. [find](https://scholar.google.com/scholar?q=Julia+Anghileri%2E+2006%2E+Scaffolding+practices+that+enhance+mathematics+learning%2E+Journal+of+Mathematics+Teacher+Education%2C+9%281%29%3A33%E2%80%9352%2E)
2. Melissa D Boston and Amber G Candela. 2018. The instructional quality assessment as a tool for reflecting on instructional practice. ZDM, 50(3):427–444. [find](https://scholar.google.com/scholar?q=Melissa+D+Boston+and+Amber+G+Candela%2E+2018%2E+The+instructional+quality+assessment+as+a+tool+for+reflecting+on+instructional+practice%2E+ZDM%2C+50%283%29%3A427%E2%80%93444%2E)
3. Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. 2020. [Language models are few-shot learners](https://md.research.elamur.ai/free/en/papers/brown-2020-language-models-few-shot-learners/). Advances in neural information processing systems, 33:1877–1901.
4. Fredrik Carlsson, Joey Öhman, Fangyu Liu, Severine Verlinden, Joakim Nivre, and Magnus Sahlgren. 2022. Fine-grained controllable text generation using non-residual prompting. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 683… [find](https://scholar.google.com/scholar?q=Fredrik+Carlsson%2C+Joey+%C3%96hman%2C+Fangyu+Liu%2C+Severine+Verlinden%2C+Joakim+Nivre%2C+and+Magnus+Sahlgren%2E+2022%2E+Fine%2Dgrained+controllable+text+generation+using+non%2Dresidual+prompting%2E+In+Proceedings+of+the+60t)
5. Yu Chen, Lingfei Wu, and Mohammed J. Zaki. 2019. Reinforcement learning based graph-to-sequence model for natural question generation. CoRR, abs/1908.04942. — [arXiv:1908.04942](https://arxiv.org/abs/1908.04942) [find](https://scholar.google.com/scholar?q=Yu+Chen%2C+Lingfei+Wu%2C+and+Mohammed+J%2E+Zaki%2E+2019%2E+Reinforcement+learning+based+graph%2Dto%2Dsequence+model+for+natural+question+generation%2E+CoRR%2C+abs%2F1908%2E04942%2E+%E2%80%94+%5BarXiv%3A1908%2E04942%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F)
6. Eunsol Choi, He He, Mohit Iyyer, Mark Yatskar, Wen-tau Yih, Yejin Choi, Percy Liang, and Luke Zettlemoyer. 2018. [QuAC: Question answering in context](https://md.research.elamur.ai/free/en/papers/choi-2018-quac-question-answering-context/). In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 2174–2184, Brussels, Belgium. Association for Computa…
7. Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung, Charles Sutton, Sebastian Gehrmann, et al. 2022. [Palm: Scaling language modeling with pathways](https://md.research.elamur.ai/free/en/papers/chowdhery-2022-palm-scaling-language-modeling-pathways/). arXiv preprint arXiv:2204.02311.
8. Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman. 2021. [Training verifiers to solve math word problems](https://md.research.elamur.ai/free/en/papers/cobbe-2021-training-verifiers-solve-math-word/). arXiv preprint arXiv:2110.14168.
9. Bhuwan Dhingra, Manzil Zaheer, Vidhisha Balachandran, Graham Neubig, Ruslan Salakhutdinov, and William W. Cohen. 2020. Differentiable reasoning over a virtual knowledge base. CoRR, abs/2002.10640. [find](https://scholar.google.com/scholar?q=Bhuwan+Dhingra%2C+Manzil+Zaheer%2C+Vidhisha+Balachandran%2C+Graham+Neubig%2C+Ruslan+Salakhutdinov%2C+and+William+W%2E+Cohen%2E+2020%2E+Differentiable+reasoning+over+a+virtual+knowledge+base%2E+CoRR%2C+abs%2F2002%2E10640%2E)
10. Zhihao Fan, Zhongyu Wei, Piji Li, Yanyan Lan, and Xuanjing Huang. 2018. A question type driven framework to diversify visual question generation. In IJCAI, pages 4048–4054. [find](https://scholar.google.com/scholar?q=Zhihao+Fan%2C+Zhongyu+Wei%2C+Piji+Li%2C+Yanyan+Lan%2C+and+Xuanjing+Huang%2E+2018%2E+A+question+type+driven+framework+to+diversify+visual+question+generation%2E+In+IJCAI%2C+pages+4048%E2%80%934054%2E)
11. Heng Gong, Wei Bi, Xiaocheng Feng, Bing Qin, Xiaojiang Liu, and Ting Liu. 2020. Enhancing content planning for table-to-text generation with data understanding and verification. In Findings of the Association for Computational Linguistics: EMNLP 2020, pages 2905–2914, Online. Association for Computa… [find](https://scholar.google.com/scholar?q=Heng+Gong%2C+Wei+Bi%2C+Xiaocheng+Feng%2C+Bing+Qin%2C+Xiaojiang+Liu%2C+and+Ting+Liu%2E+2020%2E+Enhancing+content+planning+for+table%2Dto%2Dtext+generation+with+data+understanding+and+verification%2E+In+Findings+of+the+Ass)
12. Pengcheng He, Xiaodong Liu, Jianfeng Gao, and Weizhu Chen. 2020. [Deberta: Decoding-enhanced bert with disentangled attention](https://md.research.elamur.ai/free/en/papers/he-2020-deberta-decoding-enhanced-bert-disentangled/). arXiv preprint arXiv:2006.03654.
13. Mohammad Javad Hosseini, Hannaneh Hajishirzi, Oren Etzioni, and Nate Kushman. 2014. [Learning to solve arithmetic word problems with verb categorization](https://md.research.elamur.ai/free/en/papers/hosseini-2014-learning-solve-arithmetic-word-problems/). In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 523–533.
14. Zhiting Hu, Zichao Yang, Xiaodan Liang, Ruslan Salakhutdinov, and Eric P. Xing. 2017. [Toward controlled generation of text](https://md.research.elamur.ai/free/en/papers/hu-2017-toward-controlled-generation-text/). In Proceedings of the 34th International Conference on Machine Learning - Volume 70, ICML’17, page 1587–1596. JMLR.org.
15. Qingbao Huang, Mingyi Fu, Linzhang Mo, Yi Cai, Jingyun Xu, Pijian Li, Qing Li, and Ho-fung Leung. 2021. Entity guided question generation with contextual structure and sequence information capturing. Proceedings of the AAAI Conference on Artificial Intelligence, 35:13064–13072. [find](https://scholar.google.com/scholar?q=Qingbao+Huang%2C+Mingyi+Fu%2C+Linzhang+Mo%2C+Yi+Cai%2C+Jingyun+Xu%2C+Pijian+Li%2C+Qing+Li%2C+and+Ho%2Dfung+Leung%2E+2021%2E+Entity+guided+question+generation+with+contextual+structure+and+sequence+information+capturing%2E+)
16. Nam Ju Kim, Brian R Belland, and Andrew E Walker. 2018. Effectiveness of computer-based scaffolding in the context of problem-based learning for stem education: Bayesian meta-analysis. Educational Psychology Review, 30(2):397–429. [find](https://scholar.google.com/scholar?q=Nam+Ju+Kim%2C+Brian+R+Belland%2C+and+Andrew+E+Walker%2E+2018%2E+Effectiveness+of+computer%2Dbased+scaffolding+in+the+context+of+problem%2Dbased+learning+for+stem+education%3A+Bayesian+meta%2Danalysis%2E+Educational+Psy)
17. Tassilo Klein and Moin Nabi. 2019. Learning to answer by learning to ask: Getting the best of gpt-2 and bert worlds. arXiv preprint arXiv:1911.02365. — [arXiv:1911.02365](https://arxiv.org/abs/1911.02365) [find](https://scholar.google.com/scholar?q=Tassilo+Klein+and+Moin+Nabi%2E+2019%2E+Learning+to+answer+by+learning+to+ask%3A+Getting+the+best+of+gpt%2D2+and+bert+worlds%2E+arXiv+preprint+arXiv%3A1911%2E02365%2E+%E2%80%94+%5BarXiv%3A1911%2E02365%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F1911%2E02)
18. Wei-Jen Ko, Te-yuan Chen, Yiyan Huang, Greg Durrett, and Junyi Jessy Li. 2020. [Inquisitive question generation for high level text comprehension](https://md.research.elamur.ai/free/en/papers/ko-2020-inquisitive-question-generation-high-level/). In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 6544–6555, Online. Association for Computational…
19. Takeshi Kojima, Shixiang Shane Gu, Machel Reid, Yutaka Matsuo, and Yusuke Iwasawa. 2022. [Large language models are zero-shot reasoners](https://md.research.elamur.ai/free/en/papers/kojima-2022-large-language-models-zero-shot/). arXiv preprint arXiv:2205.11916.
20. Nate Kushman, Yoav Artzi, Luke Zettlemoyer, and Regina Barzilay. 2014. Learning to automatically solve algebra word problems. In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 271–281. [find](https://scholar.google.com/scholar?q=Nate+Kushman%2C+Yoav+Artzi%2C+Luke+Zettlemoyer%2C+and+Regina+Barzilay%2E+2014%2E+Learning+to+automatically+solve+algebra+word+problems%2E+In+Proceedings+of+the+52nd+Annual+Meeting+of+the+Association+for+Computati)
21. Peter J Liu, Mohammad Saleh, Etienne Pot, Ben Goodrich, Ryan Sepassi, Lukasz Kaiser, and Noam Shazeer. 2018. [Generating wikipedia by summarizing long sequences](https://md.research.elamur.ai/free/en/papers/liu-2018-generating-wikipedia-summarizing-long-sequences/). arXiv preprint arXiv:1801.10198.
22. Ðord̄e Miladinović, Kumar Shridhar, Kushal Jain, Max B Paulus, Joachim M Buhmann, and Carl Allen. 2022. Learning to drop out: An adversarial approach to training sequence vaes. arXiv preprint arXiv:2209.12590. — [arXiv:2209.12590](https://arxiv.org/abs/2209.12590) [find](https://scholar.google.com/scholar?q=%C3%90ord%CC%84e+Miladinovic%CC%81%2C+Kumar+Shridhar%2C+Kushal+Jain%2C+Max+B+Paulus%2C+Joachim+M+Buhmann%2C+and+Carl+Allen%2E+2022%2E+Learning+to+drop+out%3A+An+adversarial+approach+to+training+sequence+vaes%2E+arXiv+preprint+arXiv%3A2)
23. Liangming Pan, Wenhu Chen, Wenhan Xiong, Min-Yen Kan, and William Yang Wang. 2021. Unsupervised multi-hop question answering by question generation. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, page… [find](https://scholar.google.com/scholar?q=Liangming+Pan%2C+Wenhu+Chen%2C+Wenhan+Xiong%2C+Min%2DYen+Kan%2C+and+William+Yang+Wang%2E+2021%2E+Unsupervised+multi%2Dhop+question+answering+by+question+generation%2E+In+Proceedings+of+the+2021+Conference+of+the+North+)
24. Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. [Bleu: a method for automatic evaluation of machine translation](https://md.research.elamur.ai/free/en/papers/papineni-2002-bleu-method-automatic-evaluation-machine/). In Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, pages 311–318, Philadelphia, Pennsylvania, USA. Association for Computational…
25. Arkil Patel, Satwik Bhattamishra, and Navin Goyal. 2021. [Are NLP models really able to solve simple math word problems?](https://md.research.elamur.ai/free/en/papers/patel-2021-are-nlp-models-really-able/) In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 2080–2094, Online. Associat…
26. Ethan Perez, Patrick Lewis, Wen-tau Yih, Kyunghyun Cho, and Douwe Kiela. 2020. [Unsupervised question decomposition for question answering](https://md.research.elamur.ai/free/en/papers/perez-2020-unsupervised-question-decomposition-question-answering/). In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 8864–8880, Online. Association for Computational Linguis…
27. Matt Post. 2018. [A call for clarity in reporting BLEU scores](https://md.research.elamur.ai/free/en/papers/post-2018-call-clarity-reporting-bleu-scores/). In Proceedings of the Third Conference on Machine Translation: Research Papers, pages 186–191, Belgium, Brussels. Association for Computational Linguistics.
28. Shrimai Prabhumoye, Yulia Tsvetkov, Ruslan Salakhutdinov, and Alan W Black. 2018. [Style transfer through back-translation](https://md.research.elamur.ai/free/en/papers/prabhumoye-2018-style-transfer-through-back-translation/). In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 866–876, Melbourne, Australia. Association for Computat…
29. Ratish Puduppully and Mirella Lapata. 2021. [Data-to-text generation with macro planning](https://md.research.elamur.ai/free/en/papers/puduppully-2021-data-text-generation-macro-planning/). Transactions of the Association for Computational Linguistics, 9:510– 527.
30. Raul Puri, Ryan Spring, Mohammad Shoeybi, Mostofa Patwary, and Bryan Catanzaro. 2020. [Training question answering models from synthetic data](https://md.research.elamur.ai/free/en/papers/puri-2020-training-question-answering-models-synthetic/). In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 5811–5826, Online. Association for Computational Ling…
31. Chris Quintana, Brian J. Reiser, Elizabeth A. Davis, Joseph Krajcik, Eric Fretz, Ravit Golan Duncan, Eleni Kyza, Daniel Edelson, and Elliot Soloway. 2004. A scaffolding design framework for software to support science inquiry. Journal of the Learning Sciences, 13(3):337–386. [find](https://scholar.google.com/scholar?q=Chris+Quintana%2C+Brian+J%2E+Reiser%2C+Elizabeth+A%2E+Davis%2C+Joseph+Krajcik%2C+Eric+Fretz%2C+Ravit+Golan+Duncan%2C+Eleni+Kyza%2C+Daniel+Edelson%2C+and+Elliot+Soloway%2E+2004%2E+A+scaffolding+design+framework+for+software+t)
32. Alec Radford, Jeff Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. 2019. Language models are unsupervised multitask learners. [find](https://scholar.google.com/scholar?q=Alec+Radford%2C+Jeff+Wu%2C+Rewon+Child%2C+David+Luan%2C+Dario+Amodei%2C+and+Ilya+Sutskever%2E+2019%2E+Language+models+are+unsupervised+multitask+learners%2E)
33. Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. 2020. [Exploring the limits of transfer learning with a unified text-to-text transformer](https://md.research.elamur.ai/free/en/papers/raffel-2019-exploring-limits-transfer-learning-unified/). Journal of Machine Learning Research, 21(140):1–67.
34. Pranav Rajpurkar, Jian Zhang, Konstantin Lopyrev, and Percy Liang. 2016. [SQuAD: 100,000+ questions for machine comprehension of text](https://md.research.elamur.ai/free/en/papers/rajpurkar-2016-squad-100-000-questions-machine/). In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 2383–2392, Austin, Texas. Association for Computational Linguistics.
35. Sudha Rao and Hal Daumé III. 2018. Learning to ask good questions: Ranking clarification questions using neural expected value of perfect information. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 2737–2746, Melbourne, Austr… [find](https://scholar.google.com/scholar?q=Sudha+Rao+and+Hal+Daum%C3%A9+III%2E+2018%2E+Learning+to+ask+good+questions%3A+Ranking+clarification+questions+using+neural+expected+value+of+perfect+information%2E+In+Proceedings+of+the+56th+Annual+Meeting+of+the+)
36. Siva Reddy, Danqi Chen, and Christopher D Manning. 2019. [Coqa: A conversational question answering challenge](https://md.research.elamur.ai/free/en/papers/reddy-2019-coqa-conversational-question-answering-challenge/). Transactions of the Association for Computational Linguistics, 7:249–266.
37. Brian J. Reiser. 2004. Scaffolding Complex Learning: The Mechanisms of Structuring and Problematizing Student Work. Journal of the Learning Sciences, 13(3):273–304. Publisher: Routledge _eprint: https://doi.org/10.1207/s15327809jls1303_2. — [doi:10.1207/s15327809jls1303_2](https://doi.org/10.1207/s15327809jls1303_2) [↗](https://doi.org/10.1207/s15327809jls1303_2)
38. Subhro Roy, Tim Vieira, and Dan Roth. 2015. Reasoning about quantities in natural language. Transactions of the Association for Computational Linguistics, 3:1–13. [find](https://scholar.google.com/scholar?q=Subhro+Roy%2C+Tim+Vieira%2C+and+Dan+Roth%2E+2015%2E+Reasoning+about+quantities+in+natural+language%2E+Transactions+of+the+Association+for+Computational+Linguistics%2C+3%3A1%E2%80%9313%2E)
39. Mrinmaya Sachan, Kumar Dubey, and Eric Xing. 2017. From textbooks to knowledge: A case study in harvesting axiomatic knowledge from textbooks to solve geometry problems. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 773–784. [find](https://scholar.google.com/scholar?q=Mrinmaya+Sachan%2C+Kumar+Dubey%2C+and+Eric+Xing%2E+2017%2E+From+textbooks+to+knowledge%3A+A+case+study+in+harvesting+axiomatic+knowledge+from+textbooks+to+solve+geometry+problems%2E+In+Proceedings+of+the+2017+Con)
40. Mrinmaya Sachan, Kumar Avinava Dubey, Tom M Mitchell, Dan Roth, and Eric P Xing. 2018. Learning pipelines with limited data and domain knowledge: A study in parsing physics problems. Advances in Neural Information Processing Systems, 31. [find](https://scholar.google.com/scholar?q=Mrinmaya+Sachan%2C+Kumar+Avinava+Dubey%2C+Tom+M+Mitchell%2C+Dan+Roth%2C+and+Eric+P+Xing%2E+2018%2E+Learning+pipelines+with+limited+data+and+domain+knowledge%3A+A+study+in+parsing+physics+problems%2E+Advances+in+Neura)
41. Mrinmaya Sachan and Eric Xing. 2017. Learning to solve geometry problems from natural language demonstrations in textbooks. In Proceedings of the 6th Joint Conference on Lexical and Computational Semantics (* SEM 2017), pages 251–261. [find](https://scholar.google.com/scholar?q=Mrinmaya+Sachan+and+Eric+Xing%2E+2017%2E+Learning+to+solve+geometry+problems+from+natural+language+demonstrations+in+textbooks%2E+In+Proceedings+of+the+6th+Joint+Conference+on+Lexical+and+Computational+Sema)
42. Minjoon Seo, Hannaneh Hajishirzi, Ali Farhadi, Oren Etzioni, and Clint Malcolm. 2015. [Solving geometry problems: Combining text and diagram interpretation](https://md.research.elamur.ai/free/en/papers/seo-2015-solving-geometry-problems-combining-text/). In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 1466– 1476, Lisbon, Portugal. Association for C…
43. Jianhao Shen, Yichun Yin, Lin Li, Lifeng Shang, Xin Jiang, Ming Zhang, and Qun Liu. 2021. Generate & rank: A multi-task framework for math word problems. In Findings of the Association for Computational Linguistics: EMNLP 2021, pages 2269–2279, Punta Cana, Dominican Republic. Association for Computa… [find](https://scholar.google.com/scholar?q=Jianhao+Shen%2C+Yichun+Yin%2C+Lin+Li%2C+Lifeng+Shang%2C+Xin+Jiang%2C+Ming+Zhang%2C+and+Qun+Liu%2E+2021%2E+Generate+%26+rank%3A+A+multi%2Dtask+framework+for+math+word+problems%2E+In+Findings+of+the+Association+for+Computation)
44. Vered Shwartz, Peter West, Ronan Le Bras, Chandra Bhagavatula, and Yejin Choi. 2020. Unsupervised commonsense question answering with self-talk. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 4615– 4629, Online. Association for Computational… [find](https://scholar.google.com/scholar?q=Vered+Shwartz%2C+Peter+West%2C+Ronan+Le+Bras%2C+Chandra+Bhagavatula%2C+and+Yejin+Choi%2E+2020%2E+Unsupervised+commonsense+question+answering+with+self%2Dtalk%2E+In+Proceedings+of+the+2020+Conference+on+Empirical+Meth)
45. Aarohi Srivastava, Abhinav Rastogi, Abhishek Rao, Abu Awal Md Shoeb, Abubakar Abid, Adam Fisch, Adam R Brown, Adam Santoro, Aditya Gupta, Adrià Garriga-Alonso, et al. 2022. [Beyond the imitation game: Quantifying and extrapolating the capabilities of language models](https://md.research.elamur.ai/free/en/papers/srivastava-2022-imitation-game-quantifying-extrapolating-capabilities/). arXiv preprint arXiv:2206.04615.
46. Katherine Stasaski and Marti A. Hearst. 2017. Multiple choice question generation utilizing an ontology. In Proceedings of the 12th Workshop on Innovative Use of NLP for Building Educational Applications, pages 303– 312, Copenhagen, Denmark. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Katherine+Stasaski+and+Marti+A%2E+Hearst%2E+2017%2E+Multiple+choice+question+generation+utilizing+an+ontology%2E+In+Proceedings+of+the+12th+Workshop+on+Innovative+Use+of+NLP+for+Building+Educational+Applicati)
47. Alessandro Stolfo, Zhijing Jin, Kumar Shridhar, Bernhard Schölkopf, and Mrinmaya Sachan. 2022. [A causal framework to quantify the robustness of mathematical reasoning with language models](https://md.research.elamur.ai/free/en/papers/stolfo-2023-causal-framework-quantify-robustness-mathematical/). arXiv preprint arXiv:2210.12023.
48. Yixuan Su, David Vandyke, Sihui Wang, Yimai Fang, and Nigel Collier. 2021. Plan-then-generate: Controlled data-to-text generation via planning. In Findings of the Association for Computational Linguistics: EMNLP 2021, pages 895–909, Punta Cana, Dominican Republic. Association for Computational Lingu… [find](https://scholar.google.com/scholar?q=Yixuan+Su%2C+David+Vandyke%2C+Sihui+Wang%2C+Yimai+Fang%2C+and+Nigel+Collier%2E+2021%2E+Plan%2Dthen%2Dgenerate%3A+Controlled+data%2Dto%2Dtext+generation+via+planning%2E+In+Findings+of+the+Association+for+Computational+Linguis)
49. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. [Attention is all you need](https://md.research.elamur.ai/free/en/papers/vaswani-2017-attention-all-need/). Advances in neural information processing systems, 30.
50. Ruonan Wang, Yuxi Qian, Fangxiang Feng, Xiaojie Wang, and Huixing Jiang. 2022. Co-VQA : Answering by interactive sub question sequence. In Findings of the Association for Computational Linguistics: ACL 2022, pages 2396–2408, Dublin, Ireland. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Ruonan+Wang%2C+Yuxi+Qian%2C+Fangxiang+Feng%2C+Xiaojie+Wang%2C+and+Huixing+Jiang%2E+2022%2E+Co%2DVQA+%3A+Answering+by+interactive+sub+question+sequence%2E+In+Findings+of+the+Association+for+Computational+Linguistics%3A+AC)
51. Zichao Wang, Andrew S Lan, Weili Nie, Andrew E Waters, Phillip J Grimaldi, and Richard G Baraniuk. 2018. Qg-net: a data-driven question generation model for educational content. In Proceedings of the Fifth Annual ACM Conference on Learning at Scale, pages 1–10. [find](https://scholar.google.com/scholar?q=Zichao+Wang%2C+Andrew+S+Lan%2C+Weili+Nie%2C+Andrew+E+Waters%2C+Phillip+J+Grimaldi%2C+and+Richard+G+Baraniuk%2E+2018%2E+Qg%2Dnet%3A+a+data%2Ddriven+question+generation+model+for+educational+content%2E+In+Proceedings+of+the+)
52. Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Ed H. Chi, Quoc Le, and Denny Zhou. 2022. [Chain of thought prompting elicits reasoning in large language models](https://md.research.elamur.ai/free/en/papers/wei-2022-chain-thought-prompting-elicits-reasoning/). CoRR, abs/2201.11903.
53. Ronald J. Williams. 1992. Simple statistical gradient-following algorithms for connectionist reinforcement learning. Mach. Learn., 8(3–4):229–256. [find](https://scholar.google.com/scholar?q=Ronald+J%2E+Williams%2E+1992%2E+Simple+statistical+gradient%2Dfollowing+algorithms+for+connectionist+reinforcement+learning%2E+Mach%2E+Learn%2E%2C+8%283%E2%80%934%29%3A229%E2%80%93256%2E)
54. Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Remi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Mariama Drame, Quentin Lhoe… — [Transformers: State-of-the-Art Natural Language Processing](https://md.research.elamur.ai/free/en/papers/wolf-2020-transformers-state-art-natural-language/)
55. David Wood, Jerome S Bruner, and Gail Ross. 1976. The role of tutoring in problem solving. Child Psychology & Psychiatry & Allied Disciplines. [find](https://scholar.google.com/scholar?q=David+Wood%2C+Jerome+S+Bruner%2C+and+Gail+Ross%2E+1976%2E+The+role+of+tutoring+in+problem+solving%2E+Child+Psychology+%26+Psychiatry+%26+Allied+Disciplines%2E)
56. Terry Wood. 1994. Patterns of interaction and the culture of mathematics classrooms. In Cultural perspectives on the mathematics classroom, pages 149–168. Springer. [find](https://scholar.google.com/scholar?q=Terry+Wood%2E+1994%2E+Patterns+of+interaction+and+the+culture+of+mathematics+classrooms%2E+In+Cultural+perspectives+on+the+mathematics+classroom%2C+pages+149%E2%80%93168%2E+Springer%2E)
57. Zhipeng Xie and Shichao Sun. 2019. A goal-driven tree-structured neural model for math word problems. In IJCAI, pages 5299–5305. [find](https://scholar.google.com/scholar?q=Zhipeng+Xie+and+Shichao+Sun%2E+2019%2E+A+goal%2Ddriven+tree%2Dstructured+neural+model+for+math+word+problems%2E+In+IJCAI%2C+pages+5299%E2%80%935305%2E)
58. Jipeng Zhang, Lei Wang, Roy Ka-Wei Lee, Yi Bin, Yan Wang, Jie Shao, and Ee-Peng Lim. 2020. Graph-to-tree learning for solving math word problems. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Jipeng+Zhang%2C+Lei+Wang%2C+Roy+Ka%2DWei+Lee%2C+Yi+Bin%2C+Yan+Wang%2C+Jie+Shao%2C+and+Ee%2DPeng+Lim%2E+2020%2E+Graph%2Dto%2Dtree+learning+for+solving+math+word+problems%2E+Association+for+Computational+Linguistics%2E)
59. Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q Weinberger, and Yoav Artzi. 2019. Bertscore: Evaluating text generation with bert. arXiv preprint arXiv:1904.09675. [find](https://scholar.google.com/scholar?q=Tianyi+Zhang%2C+Varsha+Kishore%2C+Felix+Wu%2C+Kilian+Q+Weinberger%2C+and+Yoav+Artzi%2E+2019%2E+Bertscore%3A+Evaluating+text+generation+with+bert%2E+arXiv+preprint+arXiv%3A1904%2E09675%2E)

*Updated 3 November 2026*