---
title: "An Incomplete Loop — paper by Liu et al. (2024)"
description: "The authors treat these routes as deductive, inductive, and abductive reasoning, and test whether skill… 43 references, cited by 9 papers here."
authors: ["Emmy Liu", "Graham Neubig", "Jacob Andreas"]
year: 2024
venue: "arXiv:2404.03028"
source: "https://arxiv.org/abs/2404.03028"
doi: "10.48550/arxiv.2404.03028"
arxiv: "2404.03028"
licence: "CC BY-SA 4.0 (arXiv)"
licence_url: "http://creativecommons.org/licenses/by-sa/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/liu-2024-incomplete-loop-deductive-inductive-abductive/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/liu-2024-incomplete-loop-deductive-inductive-abductive/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2024](https://md.research.elamur.ai/free/en/papers/2024/page/9/) / An Incomplete Loop: Deductive, Inductive, and Abductive Reasoning in Language Models

# An Incomplete Loop: Deductive, Inductive, and Abductive Reasoning in Language Models

**Authors:** [Emmy Liu](https://md.research.elamur.ai/free/en/authors/emmy-liu/), [Graham Neubig](https://md.research.elamur.ai/free/en/authors/graham-neubig/) and [Jacob Andreas](https://md.research.elamur.ai/free/en/authors/jacob-andreas/) · **Year:** 2024 · **Venue:** arXiv:2404.03028 · **Licence:** [CC BY-SA 4.0 (arXiv)](http://creativecommons.org/licenses/by-sa/4.0/)

**Links:** [Publisher's page](https://arxiv.org/abs/2404.03028) · [PDF](https://arxiv.org/pdf/2404.03028) · [DOI 10.48550/arxiv.2404.03028](https://doi.org/10.48550/arxiv.2404.03028) · [arXiv 2404.03028](https://arxiv.org/abs/2404.03028) · [OpenAlex](https://openalex.org/W4394007083)

**Citations:** 3 citations (OpenAlex, 2026-10-14) · [9](https://md.research.elamur.ai/free/en/papers/liu-2024-incomplete-loop-deductive-inductive-abductive/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
Language models can acquire a new task in three ways: from an explicit instruction, from a few examples, or by first writing their own task description from examples and then following it. The authors treat these routes as deductive, inductive, and abductive reasoning, and test whether skill in one predicts skill in the others. Four gpt and llama chat models are evaluated on linear function learning, an artificial colour language, and English–Kalamang translation. For the abductive route, a model proposes five hypotheses, which are reranked by verbalized confidence, by LM likelihood, or, for functions, by an external validator that executes the rule. Self-generated instructions beat plain few-shot prompting and zero-shot chain-of-thought on the synthetic tasks. On low-resource Kalamang, however, wrong hypotheses undermine them. The three abilities turn out to be largely dissociated: a model may learn from examples yet fail to state the rule, or state a useful rule yet not benefit from it. Following a correct supplied instruction, by contrast, is reliably strong. Prompting methods that look similar can therefore engage quite different mechanisms.

## Key points
- Models tested: gpt-3.5-turbo, gpt-4-turbo (gpt-4-1106-preview), llama-2-7b-chat and llama-2-70b-chat. Code, data and generated hypotheses are released on GitHub (nightingal3, "rule induction" repo).
- Domains and data:
  - 40 linear functions (e.g. f(x) = 20x − 13).
  - A nonce-word colour language (e.g. "lug" → blue).
  - Kalamang vocabulary and typological grammar features (e.g. SV order), evaluated with chrF in both directions (ek, ke).
- Hypothesis scoring options (n = 5 hypotheses per task):
  - verbalized confidence;
  - P(data), the summed log-probabilities of the in-context examples, computed by a separate LM;
  - P(answer), which counts only the answer tokens;
  - a parsing-based external validator, used only for functions.
- Correlation between correct induced hypotheses and few-shot success is generally weak. This was measured with point-biserial correlation and FDR (Benjamini–Hochberg) correction. The only exception is a small positive link for gpt models on Kalamang vocabulary.
- The authors argue that pipelines which select among hypotheses are better described as abduction than induction. They suggest future work on hypothesis verification and correction, and on consistency-enforcing joint training.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works

🧭 Topics (2): [🧠 Language models & pretraining](https://md.research.elamur.ai/free/en/topics/language-models/) · 3,320 works · [🧮 Reasoning](https://md.research.elamur.ai/free/en/topics/reasoning/) · 1,426 works

## Cite

Cite as: Emmy Liu, Graham Neubig and Jacob Andreas. 2024. An Incomplete Loop: Deductive, Inductive, and Abductive Reasoning in Language Models. arXiv:2404.03028.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@misc{liu2024incomplete,
  title = {{An Incomplete Loop: Deductive, Inductive, and Abductive Reasoning in Language Models}},
  author = {Emmy Liu, Graham Neubig and Jacob Andreas},
  year = {2024},
  doi = {10.48550/arxiv.2404.03028},
  eprint = {2404.03028},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2404.03028}
}
```

- [← previous Amharic LLaMA and LLaVA: Multimodal LLMs for Low Resource Languages](https://md.research.elamur.ai/free/en/papers/andersland-2024-amharic-llama-llava-multimodal-llms/)
- [next → Base of RoPE Bounds Context Length](https://md.research.elamur.ai/free/en/papers/men-2024-base-rope-bounds-context/)
- [↑ back to papers of 2024](https://md.research.elamur.ai/free/en/papers/2024/page/9/)

## References

43 references, 14 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Ekin Akyürek, Dale Schuurmans, Jacob Andreas, Tengyu Ma, and Denny Zhou. [What learning algorithm is in-context learning? investigations with linear models](https://md.research.elamur.ai/free/en/papers/akyurek-2022-learning-algorithm-context-learning-investigations/). In Proceedings of the Eleventh International Conference on Learning Representations (ICLR), 2023.
2. Jacob Andreas, Dan Klein, and Sergey Levine. Learning with latent language. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), pp. 2166–2179, New Orleans, Louisiana, 2018. Associat… [find](https://scholar.google.com/scholar?q=Jacob+Andreas%2C+Dan+Klein%2C+and+Sergey+Levine%2E+Learning+with+latent+language%2E+In+Proceedings+of+the+2018+Conference+of+the+North+American+Chapter+of+the+Association+for+Computational+Linguistics%3A+Human+)
3. Yoav Benjamini and Yosef Hochberg. Controlling the false discovery rate: a practical and powerful approach to multiple testing. Journal of the Royal Statistical Society. Series B (Methodological), 57(1):289–300, 1995. [find](https://scholar.google.com/scholar?q=Yoav+Benjamini+and+Yosef+Hochberg%2E+Controlling+the+false+discovery+rate%3A+a+practical+and+powerful+approach+to+multiple+testing%2E+Journal+of+the+Royal+Statistical+Society%2E+Series+B+%28Methodological%29%2C+57%28)
4. Chandra Bhagavatula, Ronan Le Bras, Chaitanya Malaviya, Keisuke Sakaguchi, Ari Holtzman, Hannah Rashkin, Doug Downey, Wen-tau Yih, and Yejin Choi. [Abductive commonsense reasoning](https://md.research.elamur.ai/free/en/papers/bhagavatula-2019-abductive-commonsense-reasoning/). In 8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020.
5. OpenReview.net, 2020. URL [https://openreview.net/forum?id=Byg1v1HKDB](https://openreview.net/forum?id=Byg1v1HKDB). [↗](https://openreview.net/forum?id=Byg1v1HKDB)
6. Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter,… — [Language Models are Few-Shot Learners](https://md.research.elamur.ai/free/en/papers/brown-2020-language-models-few-shot-learners/)
7. Igor Douven. Abduction. In Edward N. Zalta (ed.), The Stanford Encyclopedia of Philosophy. Metaphysics Research Lab, Stanford University, Summer 2021 edition, 2021. [find](https://scholar.google.com/scholar?q=Igor+Douven%2E+Abduction%2E+In+Edward+N%2E+Zalta+%28ed%2E%29%2C+The+Stanford+Encyclopedia+of+Philosophy%2E+Metaphysics+Research+Lab%2C+Stanford+University%2C+Summer+2021+edition%2C+2021%2E)
8. Matthew S. Dryer and Martin Haspelmath (eds.). WALS Online (v2020.3). Zenodo, 2013. doi: 10.5281/zenodo.7385533. URL https://doi.org/10.5281/zenodo.7385533. — [doi:10.5281/zenodo.7385533](https://doi.org/10.5281/zenodo.7385533) [↗](https://doi.org/10.5281/zenodo.7385533)
9. Kevin Ellis, Lionel Wong, Maxwell Nye, Mathias Sablé-Meyer, Luc Cary, Lore Anaya Pozo, Luke Hewitt, Armando Solar-Lezama, and Joshua B. Tenenbaum. Dreamcoder: Growing generalizable, interpretable knowledge with wake–sleep bayesian program learning. Philosophical Transactions of the Royal Society A:… — [doi:10.1098/rsta.2022.0050](https://doi.org/10.1098/rsta.2022.0050) [find](https://scholar.google.com/scholar?q=Kevin+Ellis%2C+Lionel+Wong%2C+Maxwell+Nye%2C+Mathias+Sable%CC%81%2DMeyer%2C+Luc+Cary%2C+Lore+Anaya+Pozo%2C+Luke+Hewitt%2C+Armando+Solar%2DLezama%2C+and+Joshua+B%2E+Tenenbaum%2E+Dreamcoder%3A+Growing+generalizable%2C+interpretable+kno)
10. H. Frankfurt. Peirce’s notion of abduction. Journal of Philosophy, 55:593–596, 1958. [find](https://scholar.google.com/scholar?q=H%2E+Frankfurt%2E+Peirce%E2%80%99s+notion+of+abduction%2E+Journal+of+Philosophy%2C+55%3A593%E2%80%93596%2C+1958%2E)
11. Shivam Garg, Dimitris Tsipras, Percy Liang, and Gregory Valiant. What can transformers learn in-context? a case study of simple function classes. In Advances in Neural Information Processing Systems 35 (NeurIPS 2022), 2022. [find](https://scholar.google.com/scholar?q=Shivam+Garg%2C+Dimitris+Tsipras%2C+Percy+Liang%2C+and+Gregory+Valiant%2E+What+can+transformers+learn+in%2Dcontext%3F+a+case+study+of+simple+function+classes%2E+In+Advances+in+Neural+Information+Processing+Systems+3)
12. James Hawthorne. Inductive Logic. In Edward N. Zalta (ed.), The Stanford Encyclopedia of Philosophy. Metaphysics Research Lab, Stanford University, Spring 2021 edition, 2021. [find](https://scholar.google.com/scholar?q=James+Hawthorne%2E+Inductive+Logic%2E+In+Edward+N%2E+Zalta+%28ed%2E%29%2C+The+Stanford+Encyclopedia+of+Philosophy%2E+Metaphysics+Research+Lab%2C+Stanford+University%2C+Spring+2021+edition%2C+2021%2E)
13. Or Honovich, Uri Shaham, Samuel R. Bowman, and Omer Levy. Instruction induction: From few examples to natural language task descriptions. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (ACL), pp. 1935–1952, 2023. [find](https://scholar.google.com/scholar?q=Or+Honovich%2C+Uri+Shaham%2C+Samuel+R%2E+Bowman%2C+and+Omer+Levy%2E+Instruction+induction%3A+From+few+examples+to+natural+language+task+descriptions%2E+In+Proceedings+of+the+61st+Annual+Meeting+of+the+Association+f)
14. Takeshi Kojima, Shixiang Shane Gu, Machel Reid, Yutaka Matsuo, and Yusuke Iwasawa. [Large language models are zero-shot reasoners](https://md.research.elamur.ai/free/en/papers/kojima-2022-large-language-models-zero-shot/). Advances in neural information processing systems, 35:22199–22213, 2022.
15. Brenden M. Lake and Marco Baroni. Generalization without systematicity: On the compositional skills of sequence-to-sequence recurrent networks. In Jennifer G. Dy and Andreas Krause (eds.), Proceedings of the 35th International Conference on Machine Learning, ICML 2018, Stockholmsmässan, Stockholm,… [find](https://scholar.google.com/scholar?q=Brenden+M%2E+Lake+and+Marco+Baroni%2E+Generalization+without+systematicity%3A+On+the+compositional+skills+of+sequence%2Dto%2Dsequence+recurrent+networks%2E+In+Jennifer+G%2E+Dy+and+Andreas+Krause+%28eds%2E%29%2C+Proceedings)
16. PMLR, 2018. URL [http://proceedings.mlr.press/v80/lake18a.html](http://proceedings.mlr.press/v80/lake18a.html). [↗](http://proceedings.mlr.press/v80/lake18a.html)
17. Brenden M. Lake, Tal Linzen, and Marco Baroni. Human few-shot learning of compositional instructions. In Annual Meeting of the Cognitive Science Society, 2019. URL [https://api.semanticscholar.org/CorpusID:58006558](https://api.semanticscholar.org/CorpusID:58006558). [↗](https://api.semanticscholar.org/CorpusID:58006558)
18. Xian Li, Ping Yu, Chunting Zhou, Timo Schick, Omer Levy, Luke Zettlemoyer, Jason Weston, and Mike Lewis. [Self-alignment with instruction backtranslation](https://md.research.elamur.ai/free/en/papers/li-2023-self-alignment-instruction-backtranslation/). In Proceedings of the Twelfth International Conference on Learning Representations (ICLR), 2024.
19. Peter Lipton. Inference to the Best Explanation. Routledge, 2001. [find](https://scholar.google.com/scholar?q=Peter+Lipton%2E+Inference+to+the+Best+Explanation%2E+Routledge%2C+2001%2E)
20. OpenAI, Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, Red Avila, Igor Babuschkin, Suchir Balaji, Valerie Balcom, Paul Baltescu, Haiming Bao, Mohammad Bavarian, Jeff Belgum, Irwan Bello, Ja… — [GPT-4 Technical Report](https://md.research.elamur.ai/free/en/papers/openai-2023-gpt-4-technical-report/)
21. Karl Pearson. Notes on regression and inheritance in the case of two parents. Proceedings of the Royal Society of London, 58:240–242, 1895. [find](https://scholar.google.com/scholar?q=Karl+Pearson%2E+Notes+on+regression+and+inheritance+in+the+case+of+two+parents%2E+Proceedings+of+the+Royal+Society+of+London%2C+58%3A240%E2%80%93242%2C+1895%2E)
22. Charles Sanders Peirce. Collected Papers of Charles Sanders Peirce, Volume 5, volume 5. Harvard University Press, 1965. URL [http://www.hup.harvard.edu/catalog.php?isbn=](http://www.hup.harvard.edu/catalog.php?isbn=) 9780674138001. [↗](http://www.hup.harvard.edu/catalog.php?isbn=)
23. Maja Popović. [chrF: character n-gram F-score for automatic MT evaluation](https://md.research.elamur.ai/free/en/papers/popovic-2015-chrf-character-n-gram-f/). In Proceedings of the Tenth Workshop on Statistical Machine Translation, pp. 392–395, Lisbon, Portugal, 2015. Association for Computational Linguistics. doi: 10.18653/v1/W15-3049. URL https://aclanthology.org/W15-3049. [↗](https://aclanthology.org/W15-3049)
24. Linlu Qiu, Liwei Jiang, Ximing Lu, Melanie Sclar, Valentina Pyatkin, Chandra Bhagavatula, Bailin Wang, Yoon Kim, Yejin Choi, Nouha Dziri, and Xiang Ren. Phenomenal yet puzzling: Testing inductive reasoning capabilities of language models with hypothesis refinement. In Proceedings of the Twelfth Inte… [find](https://scholar.google.com/scholar?q=Linlu+Qiu%2C+Liwei+Jiang%2C+Ximing+Lu%2C+Melanie+Sclar%2C+Valentina+Pyatkin%2C+Chandra+Bhagavatula%2C+Bailin+Wang%2C+Yoon+Kim%2C+Yejin+Choi%2C+Nouha+Dziri%2C+and+Xiang+Ren%2E+Phenomenal+yet+puzzling%3A+Testing+inductive+reas)
25. Victor Sanh, Albert Webson, Colin Raffel, Stephen H. Bach, Lintang Sutawika, Zaid Alyafeai, Antoine Chaffin, Arnaud Stiegler, Teven Le Scao, Arun Raja, et al. [Multitask prompted training enables zero-shot task generalization](https://md.research.elamur.ai/free/en/papers/sanh-2021-multitask-prompted-training-enables-zero/). In Proceedings of the Tenth International Conference on Learning Represent…
26. Stewart Shapiro and Teresa Kouri Kissel. Classical Logic. In Edward N. Zalta and Uri Nodelman (eds.), The Stanford Encyclopedia of Philosophy. Metaphysics Research Lab, Stanford University, Spring 2024 edition, 2024. [find](https://scholar.google.com/scholar?q=Stewart+Shapiro+and+Teresa+Kouri+Kissel%2E+Classical+Logic%2E+In+Edward+N%2E+Zalta+and+Uri+Nodelman+%28eds%2E%29%2C+The+Stanford+Encyclopedia+of+Philosophy%2E+Metaphysics+Research+Lab%2C+Stanford+University%2C+Spring+202)
27. Koustuv Sinha, Shagun Sodhani, Jin Dong, Joelle Pineau, and William L. Hamilton. CLUTRR: A diagnostic benchmark for inductive reasoning from text. In Kentaro Inui, Jing Jiang, Vincent Ng, and Xiaojun Wan (eds.), Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing a… [find](https://scholar.google.com/scholar?q=Koustuv+Sinha%2C+Shagun+Sodhani%2C+Jin+Dong%2C+Joelle+Pineau%2C+and+William+L%2E+Hamilton%2E+CLUTRR%3A+A+diagnostic+benchmark+for+inductive+reasoning+from+text%2E+In+Kentaro+Inui%2C+Jing+Jiang%2C+Vincent+Ng%2C+and+Xiaojun+)
28. Hedvig Skirgård, Hannah J. Haynie, Damián E. Blasi, Harald Hammarström, Jeremy Collins, Jay J. Latarche, Jakob Lesage, Tobias Weber, Alena Witzlack-Makarevich, Sam Passmore, Angela Chira, Luke Maurits, Russell Dinnage, Michael Dunn, Ger Reesink, Ruth Singer, Claire Bowern, Patience Epps, Jane Hil… — [doi:10.1126/sciadv.adg6175](https://doi.org/10.1126/sciadv.adg6175) [find](https://scholar.google.com/scholar?q=Hedvig+Skirga%CC%8Ard%2C+Hannah+J%2E+Haynie%2C+Damia%CC%81n+E%2E+Blasi%2C+Harald+Hammarstro%CC%88m%2C+Jeremy+Collins%2C+Jay+J%2E+Latarche%2C+Jakob+Lesage%2C+Tobias+Weber%2C+Alena+Witzlack%2DMakarevich%2C+Sam+Passmore%2C+Angela+Chira%2C+Luke+Maur)
29. Charles Spearman. The proof and measurement of association between two things. American Journal of Psychology, 15:72–101, 1904. [find](https://scholar.google.com/scholar?q=Charles+Spearman%2E+The+proof+and+measurement+of+association+between+two+things%2E+American+Journal+of+Psychology%2C+15%3A72%E2%80%93101%2C+1904%2E)
30. Garrett Tanzer, Mirac Suzgun, Eline Visser, Dan Jurafsky, and Luke Melas-Kyriazi. A benchmark for learning to translate a new language from one grammar book, 2024. [find](https://scholar.google.com/scholar?q=Garrett+Tanzer%2C+Mirac+Suzgun%2C+Eline+Visser%2C+Dan+Jurafsky%2C+and+Luke+Melas%2DKyriazi%2E+A+benchmark+for+learning+to+translate+a+new+language+from+one+grammar+book%2C+2024%2E)
31. Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, Dan Bikel, Lukas Blecher, Cristian Canton Ferrer, Moya Chen, Guillem Cucurull, David Esiobu, Jude Fernandes, Jeremy Fu, Wenyin Fu, Brian Fuller,… — [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://md.research.elamur.ai/free/en/papers/touvron-2023-llama-2-open-foundation-fine/)
32. Eline Visser. Kalamang dictionary. Dictionaria, (13):1–2737, 2020. URL [https://dictionaria.clld.org/contributions/kalamang](https://dictionaria.clld.org/contributions/kalamang). [↗](https://dictionaria.clld.org/contributions/kalamang)
33. Cunxiang Wang, Shuailong Liang, Yue Zhang, Xiaonan Li, and Tian Gao. [Does it make sense? and why? a pilot study for sense making and explanation](https://md.research.elamur.ai/free/en/papers/wang-2019-does-it-make-sense-why/). In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pp. 4020–4026, Florence, Italy, July 2019. Association for Com…
34. Ruocheng Wang, Eric Zelikman, Gabriel Poesia, Yewen Pu, Nick Haber, and Noah D. Goodman. Hypothesis search: Inductive reasoning with language models. In Proceedings of the Twelfth International Conference on Learning Representations (ICLR), 2024. [find](https://scholar.google.com/scholar?q=Ruocheng+Wang%2C+Eric+Zelikman%2C+Gabriel+Poesia%2C+Yewen+Pu%2C+Nick+Haber%2C+and+Noah+D%2E+Goodman%2E+Hypothesis+search%3A+Inductive+reasoning+with+language+models%2E+In+Proceedings+of+the+Twelfth+International+Confer)
35. Jason Wei, Maarten Bosma, Vincent Y. Zhao, Kelvin Guu, Adams Wei Yu, Brian Lester, Nan Du, Andrew M. Dai, and Quoc V. Le. [Finetuned language models are zero-shot learners](https://md.research.elamur.ai/free/en/papers/wei-2021-finetuned-language-models-zero-shot/). In Proceedings of the Tenth International Conference on Learning Representations (ICLR), 2022a. URL https://arxiv.org/abs/2109.0… [↗](https://arxiv.org/abs/2109.0…)
36. Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed Chi, Quoc Le, and Denny Zhou. [Chain-of-thought prompting elicits reasoning in large language models](https://md.research.elamur.ai/free/en/papers/wei-2022-chain-thought-prompting-elicits-reasoning/). In Proceedings of the 36th Conference on Neural Information Processing Systems (NeurIPS), 2022b.
37. Zonglin Yang, Li Dong, Xinya Du, Hao Cheng, Erik Cambria, Xiaodong Liu, Jianfeng Gao, and Furu Wei. [Language models as inductive reasoners](https://md.research.elamur.ai/free/en/papers/yang-2024-language-models-as-inductive-reasoners/). In Yvette Graham and Matthew Purver (eds.), Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics (Volume…
38. Hongming Zhang, Xinran Zhao, and Yangqiu Song. WinoWhy: A deep diagnosis of essential commonsense knowledge for answering Winograd schema challenge. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp. 5736–5745, Online, 2020. Association for Computational… [find](https://scholar.google.com/scholar?q=Hongming+Zhang%2C+Xinran+Zhao%2C+and+Yangqiu+Song%2E+WinoWhy%3A+A+deep+diagnosis+of+essential+commonsense+knowledge+for+answering+Winograd+schema+challenge%2E+In+Proceedings+of+the+58th+Annual+Meeting+of+the+As)
39. Wenting Zhao, Justin Chiu, Claire Cardie, and Alexander Rush. Abductive commonsense reasoning exploiting mutually exclusive explanations. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 14883– 14896, Toronto, Canada, July 2023.… [find](https://scholar.google.com/scholar?q=Wenting+Zhao%2C+Justin+Chiu%2C+Claire+Cardie%2C+and+Alexander+Rush%2E+Abductive+commonsense+reasoning+exploiting+mutually+exclusive+explanations%2E+In+Proceedings+of+the+61st+Annual+Meeting+of+the+Association+f)
40. Ruiqi Zhong, Charlie Snell, Dan Klein, and Jacob Steinhardt. Describing differences between text distributions with natural language. In Kamalika Chaudhuri, Stefanie Jegelka, Le Song, Csaba Szepesvári, Gang Niu, and Sivan Sabato (eds.), International Conference on Machine Learning, ICML 2022, 17-23… [find](https://scholar.google.com/scholar?q=Ruiqi+Zhong%2C+Charlie+Snell%2C+Dan+Klein%2C+and+Jacob+Steinhardt%2E+Describing+differences+between+text+distributions+with+natural+language%2E+In+Kamalika+Chaudhuri%2C+Stefanie+Jegelka%2C+Le+Song%2C+Csaba+Szepesva%CC%81r)
41. PMLR, 2022. URL [https://proceedings.mlr.press/v162/zhong22a.html](https://proceedings.mlr.press/v162/zhong22a.html). [↗](https://proceedings.mlr.press/v162/zhong22a.html)
42. Ruiqi Zhong, Peter Zhang, Steve Li, Jinwoo Ahn, Dan Klein, and Jacob Steinhardt. [Goal driven discovery of distributional differences via language descriptions](https://md.research.elamur.ai/free/en/papers/zhong-2023-goal-driven-discovery-distributional-differences/). In Advances in Neural Information Processing Systems 36 (NeurIPS 2023), 2023.
43. Zhaocheng Zhu, Yuan Xue, Xinyun Chen, Denny Zhou, Jian Tang, Dale Schuurmans, and Hanjun Dai. Large language models can learn rules, 2023. [find](https://scholar.google.com/scholar?q=Zhaocheng+Zhu%2C+Yuan+Xue%2C+Xinyun+Chen%2C+Denny+Zhou%2C+Jian+Tang%2C+Dale+Schuurmans%2C+and+Hanjun+Dai%2E+Large+language+models+can+learn+rules%2C+2023%2E)

*Updated 30 October 2026*