---
title: "Reasoning — papers — Elamur Research"
description: "Reasoning: 508 papers of the Elamur Research library, the most cited first. Among them: Chain-of-Thought Prompting Elicits Reasoning in…"
canonical: "https://research.elamur.ai/free/en/topics/reasoning/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/topics/reasoning/)

[home](https://md.research.elamur.ai/free/en/) / [topics](https://md.research.elamur.ai/free/en/topics/) / Reasoning

# 🧮 Reasoning

508 papers · page 1 of 6 · 1–25 · the most cited first · [all topics](https://md.research.elamur.ai/free/en/topics/)

Logic, inference and step-by-step problem solving: entailment, commonsense, mathematics.

💡 Explained: [Knowledge representation and reasoning](https://md.research.elamur.ai/free/en/concepts/knowledge-representation/)

A survey · about 20 minutes to read · every paper named here has its own page in this library

**In this article**

1. [What it is](#a-what-it-is)
2. [History and milestones](#a-history-and-milestones)
3. [Main problems and tasks](#a-main-problems-and-tasks)
4. [Methods and schools](#a-methods-and-schools)
5. [Datasets, resources and benchmarks](#a-datasets-resources-and-benchmarks)
6. [Landmark papers in this library](#a-landmark-papers-in-this-library)
7. [People](#a-people)
8. [Institutions and places](#a-institutions-and-places)
9. [Open questions](#a-open-questions)
10. [Neighbouring topics](#a-neighbouring-topics)
11. [Key works in this library](#a-key-works-in-this-library)

Reasoning, in the sense used on this page, is the ability of a system to reach a conclusion that is not written in its input by combining what it is given with rules, background knowledge or intermediate steps of its own. A reader who learns that a glass fell from a table and concludes that it probably broke, a student who turns a word problem into two subtractions and a division, and a program that checks whether one sentence follows from another are all doing something of this kind. In natural language processing the topic covers several families of work that grew up separately and later merged: recognising textual entailment, commonsense inference, multi-step question answering, mathematical problem solving, logical and formal deduction, and, since about 2022, the study of how large language models can be made to "think" before they answer.

The papers here run from a classic of cognitive science in the early 1980s to reports on reasoning-oriented language models released in 2025 and 2026, but the great majority were written after 2018. The topic therefore reads as a short, intense history. It moves from symbolic systems, through crowdsourced datasets and neural classifiers, to prompting, verifiers and reinforcement learning. A recurring question: when a model is right, did it reason or find a shortcut?

## What it is

### Kinds of reasoning

Philosophy and psychology distinguish several kinds of inference, and the field has kept the vocabulary. **Deductive** reasoning draws conclusions that must be true if the premises are true. **Inductive** reasoning generalises from examples to a rule. **Abductive** reasoning picks the most plausible explanation for an observation. Researchers also speak of **analogical**, **causal**, **spatial**, **temporal** and **numerical** reasoning, and of **commonsense** reasoning, which relies on unstated knowledge about how the physical and social world usually works. The survey [Towards Reasoning in Large Language Models: A Survey](https://md.research.elamur.ai/free/en/papers/huang-2023-reasoning-large-language-models-survey/) lays out these distinctions and separates formal reasoning, carried out in a logic with explicit rules, from informal reasoning in everyday language, which is where most of the work here sits.

### Core notions

A few ideas recur across the topic.

- **Entailment.** Given a premise and a hypothesis, does the first support the second, contradict it, or neither? This three-way decision, called recognising textual entailment (RTE) or natural language inference (NLI), became the standard test of whether a system understands sentence meaning.
- **Multi-step or multi-hop inference.** Many questions can only be answered by chaining facts: finding a bridge entity in one document and using it to look up another, or computing an intermediate quantity before the final one.
- **Rationales and chains of thought.** A rationale is a written justification for an answer. When a model writes out intermediate steps before answering, the steps are called a chain of thought.
- **Verification.** Checking whether a candidate answer, or each step leading to it, is correct, either with a learned judge or with an exact tool such as a calculator, a program interpreter or a proof assistant.
- **Shortcuts and artefacts.** Statistical regularities in a dataset that let a model guess the right label without performing the intended inference.

### How it differs from neighbours

Reasoning overlaps heavily with [Question answering](https://md.research.elamur.ai/free/en/topics/question-answering/), but the emphasis differs: question answering asks whether a system can find and return information, while reasoning work asks whether it can combine information it was given or already holds. It also overlaps with [Language models & pretraining](https://md.research.elamur.ai/free/en/topics/language-models/), since most recent reasoning research studies large pretrained models, and with [Evaluation & benchmarks](https://md.research.elamur.ai/free/en/topics/evaluation-benchmarks/), because much of the topic consists of building tests that resist shortcuts. Pragmatic inference, the reasoning a listener performs to recover what a speaker meant rather than what was said, is treated under [Implicature](https://md.research.elamur.ai/free/en/topics/implicature/) and [Speech act theory](https://md.research.elamur.ai/free/en/topics/speech-act-theory/), though some of its foundational papers appear here too.

## History and milestones

Landmark papers on this topic: the 12 most cited, in time The 12 most cited papers on this topic span 1981–2025; the dot grows with the citations.

**The numbers**

| Paper | Citations |
| --- | --- |
| [1981 — Categorization and Representation of Physics Problems by Experts and Novices](https://md.research.elamur.ai/free/en/papers/chi-1981-categorization-representation-physics-problems-experts/) | 5,250 |
| [2019 — GQA: A New Dataset for Real-World Visual Reasoning and Compositional Question Answering](https://md.research.elamur.ai/free/en/papers/hudson-2019-gqa-new-dataset-real-world/) | 2,006 |
| [2021 — Training Verifiers to Solve Math Word Problems](https://md.research.elamur.ai/free/en/papers/cobbe-2021-training-verifiers-solve-math-word/) | 3,680 |
| [2021 — Measuring Mathematical Problem Solving With the MATH Dataset](https://md.research.elamur.ai/free/en/papers/hendrycks-2021-measuring-mathematical-problem-solving-math/) | 2,320 |
| [2021 — Exploiting Cloze-Questions for Few-Shot Text Classification and Natural Language Inference](https://md.research.elamur.ai/free/en/papers/schick-2021-exploiting-cloze-questions-few-shot/) | 1,803 |
| [2022 — Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://md.research.elamur.ai/free/en/papers/wei-2022-chain-thought-prompting-elicits-reasoning/) | 11,308 |
| [2022 — Large language models are zero-shot reasoners](https://md.research.elamur.ai/free/en/papers/kojima-2022-large-language-models-zero-shot/) | 3,795 |
| [2022 — Self-Consistency Improves Chain of Thought Reasoning in Language Models](https://md.research.elamur.ai/free/en/papers/wang-2022-self-consistency-improves-chain-thought/) | 3,041 |
| [2022 — ReAct: Synergizing Reasoning and Acting in Language Models](https://md.research.elamur.ai/free/en/papers/yao-2022-react-synergizing-reasoning-acting-language/) | 2,497 |
| [2024 — DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models](https://md.research.elamur.ai/free/en/papers/shao-2024-deepseekmath-pushing-limits-mathematical-reasoning/) | 2,013 |
| [2025 — DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning](https://md.research.elamur.ai/free/en/papers/deepseek-ai-2025-deepseek-r1-incentivizing-reasoning-capability/) | 4,593 |
| [2025 — Qwen3 Technical Report](https://md.research.elamur.ai/free/en/papers/yang-2025-qwen3-technical-report/) | 3,276 |

### Cognitive roots and symbolic inference

The oldest work in this library comes from cognitive science rather than computing. [Categorization and Representation of Physics Problems by Experts and Novices](https://md.research.elamur.ai/free/en/papers/chi-1981-categorization-representation-physics-problems-experts/) showed that beginners sort mechanics problems by what they look like, while experts sort them by the physical principle that solves them.

Through the 1990s and 2000s, inference over language was mostly symbolic. Sentences were translated into logical forms and handed to theorem provers or model builders, an approach represented here by [Recognising Textual Entailment with Logical Inference](https://md.research.elamur.ai/free/en/papers/bos-2005-recognising-textual-entailment-logical-inference/). The PASCAL Recognising Textual Entailment challenges, which began in the mid-2000s, gave the field a shared task and a common definition, and a decade of methods was reviewed in [A Survey of Paraphrasing and Textual Entailment Methods](https://md.research.elamur.ai/free/en/papers/androutsopoulos-2009-survey-paraphrasing-textual-entailment-methods/). Alignment-based and tree-edit approaches such as [Tree Edit Models for Recognizing Textual Entailments, Paraphrases…](https://md.research.elamur.ai/free/en/papers/heilman-2010-tree-edit-models-recognizing-textual/) offered a statistical middle ground between full logic and word overlap.

### Shared tasks and the first large datasets (2012–2016)

In the early 2010s several shared tasks tied reasoning to concrete evaluation. [SemEval-2012 Task 7: Choice of Plausible Alternatives](https://md.research.elamur.ai/free/en/papers/gordon-2012-semeval-2012-task-7-choice/) introduced COPA, a small test of causal commonsense. [SemEval-2013 Task 7](https://md.research.elamur.ai/free/en/papers/dzikovska-2013-semeval-2013-task-7-joint/) folded entailment into the grading of student answers. [SemEval-2014 Task 1](https://md.research.elamur.ai/free/en/papers/marelli-2014-semeval-2014-task-1-evaluation/) released SICK, a set of sentence pairs designed to probe compositional meaning. In 2015 two developments changed the scale of the field. The Stanford Natural Language Inference corpus (SNLI) provided several hundred thousand crowdsourced sentence pairs, enough to train neural networks end to end, and the bAbI suite from Facebook AI Research, described in [Towards AI-Complete Question Answering: A Set of Prerequisite Toy Tasks](https://md.research.elamur.ai/free/en/papers/weston-2015-towards-ai-complete-question-answering/), offered twenty small synthetic tasks, each isolating one skill so that failure points to a specific gap.

### The neural NLI era and the discovery of shortcuts (2015–2020)

Neural models for entailment arrived quickly. [Reasoning about Entailment with Neural Attention](https://md.research.elamur.ai/free/en/papers/rocktaschel-2015-reasoning-about-entailment-neural-attention/) showed that an LSTM with attention could beat feature-engineered classifiers on SNLI; [Learning Natural Language Inference with LSTM](https://md.research.elamur.ai/free/en/papers/wang-2016-learning-natural-language-inference-lstm/), [Natural Language Inference by Tree-Based Convolution and Heuristic Matching](https://md.research.elamur.ai/free/en/papers/mou-2016-natural-language-inference-tree-based/) and [A Decomposable Attention Model for Natural Language Inference](https://md.research.elamur.ai/free/en/papers/parikh-2016-decomposable-attention-model-natural-language/) followed within months. NLI was then turned into a pretraining signal: [Supervised Learning of Universal Sentence Representations from Natural Language Inference Data](https://md.research.elamur.ai/free/en/papers/conneau-2017-supervised-learning-universal-sentence-representations/) used it to train general sentence encoders.

Accuracy rose fast, and so did suspicion. [Breaking NLI Systems with Sentences that Require Simple Lexical Inferences](https://md.research.elamur.ai/free/en/papers/glockner-2018-breaking-nli-systems-sentences-that/) showed that swapping one word for a synonym or antonym was enough to defeat strong models, and [Right for the Wrong Reasons: Diagnosing Syntactic Heuristics in Natural Language Inference](https://md.research.elamur.ai/free/en/papers/mccoy-2019-right-wrong-reasons-diagnosing-syntactic/) showed that models trained on MultiNLI mostly relied on word overlap. In parallel, vision researchers built diagnostic datasets for relational reasoning such as [CLEVR](https://md.research.elamur.ai/free/en/papers/johnson-2016-clevr-diagnostic-dataset-compositional-language/), and architectures such as the [Relation Network](https://md.research.elamur.ai/free/en/papers/santoro-2017-simple-neural-network-module-relational/) and [FiLM](https://md.research.elamur.ai/free/en/papers/perez-2017-film-visual-reasoning-general-conditioning/) to solve them.

Between 2018 and 2020 the field built dozens of benchmarks aimed at specific skills: commonsense ([PIQA](https://md.research.elamur.ai/free/en/papers/bisk-2019-piqa-reasoning-about-physical-commonsense/), [SocialIQA](https://md.research.elamur.ai/free/en/papers/sap-2019-socialiqa-commonsense-reasoning-about-social/), [Cosmos QA](https://md.research.elamur.ai/free/en/papers/huang-2019-cosmos-qa-machine-reading-comprehension/), [Abductive Commonsense Reasoning](https://md.research.elamur.ai/free/en/papers/bhagavatula-2019-abductive-commonsense-reasoning/)), discrete numerical reasoning ([DROP](https://md.research.elamur.ai/free/en/papers/dua-2019-drop-reading-comprehension-benchmark-requiring/)), logic ([LogiQA](https://md.research.elamur.ai/free/en/papers/liu-2020-logiqa-challenge-dataset-machine-reading/), [ReClor](https://md.research.elamur.ai/free/en/papers/yu-2020-reclor-reading-comprehension-dataset-requiring/)) and multi-hop retrieval ([2WikiMultiHopQA](https://md.research.elamur.ai/free/en/papers/ho-2020-constructing-multi-hop-qa-dataset/)). Knowledge was also made explicit: [COMET](https://md.research.elamur.ai/free/en/papers/bosselut-2019-comet-commonsense-transformers-automatic-knowledge/) trained a transformer to generate commonsense knowledge-graph entries.

### Prompting and the chain-of-thought turn (2021–2023)

The decisive change came with very large language models. [Training Verifiers to Solve Math Word Problems](https://md.research.elamur.ai/free/en/papers/cobbe-2021-training-verifiers-solve-math-word/) released GSM8K and showed that ranking sampled solutions with a trained judge could substitute for a much larger model. [Measuring Mathematical Problem Solving With the MATH Dataset](https://md.research.elamur.ai/free/en/papers/hendrycks-2021-measuring-mathematical-problem-solving-math/) supplied a much harder test. Then, in early 2022, [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://md.research.elamur.ai/free/en/papers/wei-2022-chain-thought-prompting-elicits-reasoning/) showed that a few worked examples in the prompt, each with its reasoning spelled out, unlocked multi-step problem solving in models of roughly a hundred billion parameters and more. Variants followed within months. Work published in 2023, the busiest year of the topic, both extended these methods and questioned whether the written chains reflect what models actually compute.

Papers on this topic by year 508 dated papers from 1981 to 2026; the busiest year is 2023 with 87, and 89% of them appeared in 2018 or later.

**The numbers**

| Year | Papers |
| --- | --- |
| 2026 | 50 |
| 2025 | 47 |
| 2024 | 56 |
| 2023 | 87 |
| 2022 | 58 |
| 2021 | 38 |
| 2020 | 49 |
| 2019 | 42 |
| 2018 | 25 |
| 2017 | 10 |
| 2016 | 8 |
| 2015 | 7 |
| 2014 | 7 |
| 2013 | 4 |
| 2012 | 3 |
| 2011 | 2 |
| 2010 | 5 |
| 2009 | 2 |
| 2008 | 1 |
| 2007 | 2 |
| 2006 | 1 |
| 2005 | 1 |
| 2003 | 1 |
| 1994 | 1 |
| 1981 | 1 |

### Training for reasoning (2024 onwards)

From 2024 the emphasis moved from prompting to training. Step-level reward models, large synthetic math corpora and reinforcement learning with automatically checkable rewards became central. OpenAI described its o1 models, which are trained to work through an extended internal argument before replying, in the [OpenAI o1 System Card](https://md.research.elamur.ai/free/en/papers/openai-2024-openai-o1-system-card/), and [DeepSeek-R1](https://md.research.elamur.ai/free/en/papers/deepseek-ai-2025-deepseek-r1-incentivizing-reasoning-capability/) showed openly that such behaviour can emerge from reinforcement learning on a base model with rule-based rewards. Model families such as [Qwen3](https://md.research.elamur.ai/free/en/papers/yang-2025-qwen3-technical-report/), [Gemini 2.5](https://md.research.elamur.ai/free/en/papers/comanici-2025-gemini-2-5-pushing-frontier/) and [gpt-oss](https://md.research.elamur.ai/free/en/papers/agarwal-2025-gpt-oss-120b-gpt-oss/) now offer adjustable "thinking" as a standard feature, and new research asks how to make such reasoning shorter, cheaper and more reliable.

## Main problems and tasks

### Natural language inference

NLI asks for one of three labels for a sentence pair. It is the most studied single task here and has spawned many variants: inference in dialogue ([Dialogue Natural Language Inference](https://md.research.elamur.ai/free/en/papers/welleck-2019-dialogue-natural-language-inference/)), in clinical notes ([Lessons from Natural Language Inference in the Clinical Domain](https://md.research.elamur.ai/free/en/papers/romanov-2018-lessons-natural-language-inference-clinical/)), over whole contracts ([ContractNLI](https://md.research.elamur.ai/free/en/papers/koreeda-2021-contractnli-dataset-document-level-natural/)), and for pragmatic phenomena ([Are Natural Language Inference Models IMPPRESsive?](https://md.research.elamur.ai/free/en/papers/jeretic-2020-are-natural-language-inference-models/)). NLI is also used as a tool inside other systems, for example to check whether a generated summary is supported by its source ([Ranking Generated Summaries by Correctness](https://md.research.elamur.ai/free/en/papers/falke-2019-ranking-generated-summaries-correctness-interesting/)) or whether evidence supports a claim ([UKP-Athene](https://md.research.elamur.ai/free/en/papers/hanselowski-2018-ukp-athene-multi-sentence-textual/)).

### Commonsense reasoning

Commonsense tasks test knowledge that people rarely write down because everyone shares it: that a towel absorbs water, that people feel embarrassed after a public mistake, that a broken alarm explains a late arrival. Benchmarks are usually multiple choice, and the challenge is to build distractors that cannot be ruled out by surface statistics. Numerical commonsense is a sub-area of its own, probed in [Birds have four legs?! NumerSense](https://md.research.elamur.ai/free/en/papers/lin-2020-birds-have-four-legs-numersense/) and [Ordinal Common-sense Inference](https://md.research.elamur.ai/free/en/papers/zhang-2017-ordinal-common-sense-inference/).

### Multi-hop and reading-based reasoning

Reading comprehension benchmarks became reasoning benchmarks once they required more than locating a span. Exam questions in [RACE](https://md.research.elamur.ai/free/en/papers/lai-2017-race-large-scale-reading-comprehension/), coreference chains in [Quoref](https://md.research.elamur.ai/free/en/papers/dasigi-2019-quoref-reading-comprehension-dataset-questions/), effects of described situations in [Reasoning Over Paragraph Effects in Situations](https://md.research.elamur.ai/free/en/papers/lin-2019-reasoning-over-paragraph-effects-situations/) and composed questions in [MuSiQue](https://md.research.elamur.ai/free/en/papers/trivedi-2022-musique-multihop-questions-single-hop/) all push in this direction. A cautionary result, [Compositional Questions Do Not Necessitate Multi-hop Reasoning](https://md.research.elamur.ai/free/en/papers/min-2019-compositional-questions-do-not-necessitate/), showed that many supposedly multi-hop questions can be answered from one paragraph.

### Mathematical and quantitative reasoning

Math word problems are attractive because the answer can be checked exactly. The line runs from operation-program datasets such as [MathQA](https://md.research.elamur.ai/free/en/papers/amini-2019-mathqa-interpretable-math-word-problem/) and systematic tests such as [Analysing Mathematical Reasoning Abilities of Neural Models](https://md.research.elamur.ai/free/en/papers/saxton-2019-analysing-mathematical-reasoning-abilities-neural/), through robustness checks like [SVAMP](https://md.research.elamur.ai/free/en/papers/patel-2021-are-nlp-models-really-able/) and [GSM-Symbolic](https://md.research.elamur.ai/free/en/papers/mirzadeh-2024-gsm-symbolic-understanding-limitations-mathematical/), to domain settings such as financial reports in [FinQA](https://md.research.elamur.ai/free/en/papers/chen-2021-finqa-dataset-numerical-reasoning-over/) and [ConvFinQA](https://md.research.elamur.ai/free/en/papers/chen-2022-convfinqa-exploring-chain-numerical-reasoning/).

### Formal theorem proving

At the formal end, a model must produce a proof that a proof assistant accepts. The [Lean 4](https://md.research.elamur.ai/free/en/papers/moura-2021-lean-4-theorem-prover-programming/) system, the [MiniF2F](https://md.research.elamur.ai/free/en/papers/zheng-2021-minif2f-cross-system-benchmark-formal/) benchmark and work such as [Generative Language Modeling for Automated Theorem Proving](https://md.research.elamur.ai/free/en/papers/polu-2020-generative-language-modeling-automated-theorem/), [HyperTree Proof Search](https://md.research.elamur.ai/free/en/papers/lample-2022-hypertree-proof-search-neural-theorem/), [LeanDojo](https://md.research.elamur.ai/free/en/papers/yang-2023-leandojo-theorem-proving-retrieval-augmented/) and the DeepSeek-Prover series ([V1.5](https://md.research.elamur.ai/free/en/papers/xin-2024-deepseek-prover-v1-5-harnessing/), [V2](https://md.research.elamur.ai/free/en/papers/ren-2025-deepseek-prover-v2-advancing-formal/)) connect language models with exact verification.

### Logical, legal and visual reasoning

Deductive reasoning in natural language is tested by [FOLIO](https://md.research.elamur.ai/free/en/papers/han-2022-folio-natural-language-reasoning-first/) and by analyses such as [Language Models Are Greedy Reasoners](https://md.research.elamur.ai/free/en/papers/saparov-2022-language-models-greedy-reasoners-systematic/). Legal reasoning, including statute application, appears in [LegalBench](https://md.research.elamur.ai/free/en/papers/guha-2023-legalbench-collaboratively-built-benchmark-measuring/) and [Can GPT-3 Perform Statutory Reasoning?](https://md.research.elamur.ai/free/en/papers/blair-stanek-2023-gpt-3-perform-statutory-reasoning/). Visual and multimodal reasoning, from images and charts to diagrams in math problems, is covered by [GQA](https://md.research.elamur.ai/free/en/papers/hudson-2019-gqa-new-dataset-real-world/), [NLVR2](https://md.research.elamur.ai/free/en/papers/suhr-2019-corpus-reasoning-natural-language-grounded/), [ChartQA](https://md.research.elamur.ai/free/en/papers/masry-2022-chartqa-benchmark-question-answering-about/) and [MathVista](https://md.research.elamur.ai/free/en/papers/lu-2023-mathvista-evaluating-mathematical-reasoning-foundation/). Planning, a close relative, is tested by [PlanBench](https://md.research.elamur.ai/free/en/papers/valmeekam-2022-planbench-extensible-benchmark-evaluating-large/).

## Methods and schools

### Symbolic and neuro-symbolic approaches

The oldest school translates language into a formal representation and lets a logic engine do the inference. It is precise but brittle. The approach has returned in hybrid form: [Logic-LM](https://md.research.elamur.ai/free/en/papers/pan-2023-logic-lm-empowering-large-language/) and [LINC](https://md.research.elamur.ai/free/en/papers/olausson-2023-linc-neurosymbolic-approach-logical-reasoning/) let a language model translate a problem into first-order logic and then call a solver; [Inter-GPS](https://md.research.elamur.ai/free/en/papers/lu-2021-inter-gps-interpretable-geometry-problem/) does the same for geometry; and [Solving olympiad geometry without human demonstrations](https://md.research.elamur.ai/free/en/papers/trinh-2024-solving-olympiad-geometry-without-human/) pairs a neural model that proposes auxiliary constructions with a symbolic deduction engine.

### Neural classifiers and architectural inductive bias

The second school trained neural networks directly on labelled examples. For NLI this meant attention and alignment between sentences; for visual and relational tasks it meant modules built to compare objects pairwise. These models scored well on in-distribution data, but stress tests repeatedly showed reliance on annotation artefacts, and methods such as [Unlearn Dataset Bias in Natural Language Inference by Fitting the Residual](https://md.research.elamur.ai/free/en/papers/he-2019-unlearn-dataset-bias-natural-language/) tried to train the bias out.

### Prompting large language models

The third school, dominant since 2022, keeps the model frozen and changes only the input. Chain-of-thought prompting adds worked steps to examples; [Large language models are zero-shot reasoners](https://md.research.elamur.ai/free/en/papers/kojima-2022-large-language-models-zero-shot/) removed the examples altogether; [Least-to-most prompting](https://md.research.elamur.ai/free/en/papers/al-2022-least-most-prompting-enables-complex/) splits a problem into easier subproblems; [Complexity-Based Prompting](https://md.research.elamur.ai/free/en/papers/fu-2022-complexity-based-prompting-multi-step/) picks demonstrations with more steps; [Maieutic Prompting](https://md.research.elamur.ai/free/en/papers/jung-2022-maieutic-prompting-logically-consistent-reasoning/) builds trees of explanations and enforces consistency among them. Program-aided variants let the model write code and leave the arithmetic to an interpreter, as in [ViperGPT](https://md.research.elamur.ai/free/en/papers/suris-2023-vipergpt-visual-inference-python-execution/), [MathPrompter](https://md.research.elamur.ai/free/en/papers/imani-2023-mathprompter-mathematical-reasoning-large-language/), [Fact-Checking Complex Claims with Program-Guided Reasoning](https://md.research.elamur.ai/free/en/papers/pan-2023-fact-checking-complex-claims-program/) and [ToRA](https://md.research.elamur.ai/free/en/papers/gou-2023-tora-tool-integrated-reasoning-agent/). Agentic variants such as [ReAct](https://md.research.elamur.ai/free/en/papers/yao-2022-react-synergizing-reasoning-acting-language/), [Chameleon](https://md.research.elamur.ai/free/en/papers/lu-2023-chameleon-plug-play-compositional-reasoning/) and [Search-o1](https://md.research.elamur.ai/free/en/papers/li-2025-search-o1-agentic-search-enhanced/) alternate thinking with calls to search engines and other tools.

### Sampling, voting, search and debate

A single generated chain is fragile. Sampling many and aggregating them is one remedy ([Self-Consistency](https://md.research.elamur.ai/free/en/papers/wang-2022-self-consistency-improves-chain-thought/)); searching over partial chains with the model's own scores is another ([Self-Evaluation Guided Beam Search for Reasoning](https://md.research.elamur.ai/free/en/papers/xie-2023-self-evaluation-guided-beam-search/)). Several model instances can also argue toward a shared answer, as in [Improving Factuality and Reasoning in Language Models through Multiagent Debate](https://md.research.elamur.ai/free/en/papers/du-2023-improving-factuality-reasoning-language-models/) and [ReConcile](https://md.research.elamur.ai/free/en/papers/chen-2024-reconcile-round-table-conference-improves/). Whether a model can repair its own mistakes without outside feedback is contested: [Large Language Models Cannot Self-Correct Reasoning Yet](https://md.research.elamur.ai/free/en/papers/huang-2023-large-language-models-cannot-self/) argued that it often cannot.

### Verifiers and reward models

A central debate concerns what to reward. Outcome supervision checks only the final answer; process supervision checks each step. [Solving math word problems with process- and outcome-based feedback](https://md.research.elamur.ai/free/en/papers/uesato-2022-solving-math-word-problems-process/) found the two roughly comparable on grade-school problems, while [Let's Verify Step by Step](https://md.research.elamur.ai/free/en/papers/lightman-2023-let-s-verify-step-step/) found a clear advantage for step-level feedback on harder ones. Later work tried to obtain step labels automatically ([Improve Mathematical Reasoning in Language Models by Automated Process Supervision](https://md.research.elamur.ai/free/en/papers/luo-2024-improve-mathematical-reasoning-language-models/)), to measure how well reward models find the first wrong step ([ProcessBench](https://md.research.elamur.ai/free/en/papers/zheng-2025-processbench-identifying-process-errors-mathematical/)), and to record [practical pitfalls in building them](https://md.research.elamur.ai/free/en/papers/zhang-2025-lessons-developing-process-reward-models/).

### Self-training, distillation and reinforcement learning

The fourth school changes the weights. [STaR](https://md.research.elamur.ai/free/en/papers/zelikman-2022-star-bootstrapping-reasoning-reasoning/) and [Large Language Models Can Self-Improve](https://md.research.elamur.ai/free/en/papers/huang-2023-large-language-models-can-self/) fine-tune a model on its own filtered rationales; [Beyond Human Data](https://md.research.elamur.ai/free/en/papers/singh-2023-human-data-scaling-self-training/) scales the idea with an external correctness check. Distillation transfers reasoning from large models to small ones, as in [Large Language Models Are Reasoning Teachers](https://md.research.elamur.ai/free/en/papers/ho-2023-large-language-models-are-reasoning/), [Distilling Reasoning Capabilities into Smaller Language Models](https://md.research.elamur.ai/free/en/papers/shridhar-2023-distilling-reasoning-capabilities-into-smaller/), [Specializing Smaller Language Models towards Multi-Step Reasoning](https://md.research.elamur.ai/free/en/papers/fu-2023-specializing-smaller-language-models-towards/), [SCOTT](https://md.research.elamur.ai/free/en/papers/wang-2023-scott-self-consistent-chain-thought/) and the [Orca](https://md.research.elamur.ai/free/en/papers/mukherjee-2023-orca-progressive-learning-complex-explanation/) models. Data augmentation for math, such as [MetaMath](https://md.research.elamur.ai/free/en/papers/yu-2023-metamath-bootstrap-your-own-mathematical/) and [WizardMath](https://md.research.elamur.ai/free/en/papers/luo-2023-wizardmath-empowering-mathematical-reasoning-large/), produces large synthetic training sets.

Reinforcement learning with verifiable rewards is the newest strand. [DeepSeekMath](https://md.research.elamur.ai/free/en/papers/shao-2024-deepseekmath-pushing-limits-mathematical-reasoning/) introduced GRPO, a critic-free variant of PPO; [ReFT](https://md.research.elamur.ai/free/en/papers/trung-2024-reft-reasoning-reinforced-fine-tuning/) and [Iterative Reasoning Preference Optimization](https://md.research.elamur.ai/free/en/papers/pang-2024-iterative-reasoning-preference-optimization/) explored related recipes; DeepSeek-R1 scaled the idea. The debate that followed asks what RL actually adds. [Does Reinforcement Learning Really Incentivize Reasoning Capacity in LLMs Beyond the Base Model?](https://md.research.elamur.ai/free/en/papers/yue-2025-reinforcement-learning-really-incentivize-reasoning/) argues that it mostly sharpens abilities already present; [LIMO](https://md.research.elamur.ai/free/en/papers/ye-2025-limo-less-more-reasoning/) suggests a small set of carefully chosen examples can elicit much of the same behaviour; [Beyond the 80/20 Rule](https://md.research.elamur.ai/free/en/papers/wang-2025-80-20-rule-high-entropy/) finds that a minority of uncertain tokens carries most of the learning signal; and [SimpleRL-Zoo](https://md.research.elamur.ai/free/en/papers/zeng-2025-simplerl-zoo-investigating-taming-zero/) studies RL from base models across families.

### Faithfulness and interpretability

A separate line asks whether written reasoning explains the answer at all. [Language Models Don't Always Say What They Think](https://md.research.elamur.ai/free/en/papers/turpin-2023-language-models-don-t-always/) showed that hidden cues in a prompt can steer answers without being mentioned in the explanation; [Measuring Faithfulness in Chain-of-Thought Reasoning](https://md.research.elamur.ai/free/en/papers/lanham-2023-measuring-faithfulness-chain-thought-reasoning/) tested how much answers depend on the stated steps; [Faithful Chain-of-Thought Reasoning](https://md.research.elamur.ai/free/en/papers/lyu-2023-faithful-chain-thought-reasoning/) proposed making the chain executable so that it must determine the result. Mechanistic studies such as [How does GPT-2 compute greater-than?](https://md.research.elamur.ai/free/en/papers/hanna-2023-gpt-2-compute-greater-than/) look inside the network instead.

## Datasets, resources and benchmarks

The topic is unusually rich in benchmarks, many of which became standard far beyond their original papers. The table groups the most prominent ones linked from this library.

| Area | Benchmarks and resources |
| --- | --- |
| Entailment | SNLI-based work, [SICK](https://md.research.elamur.ai/free/en/papers/marelli-2014-semeval-2014-task-1-evaluation/), [e-SNLI](https://md.research.elamur.ai/free/en/papers/camburu-2018-e-snli-natural-language-inference/), [HANS](https://md.research.elamur.ai/free/en/papers/mccoy-2019-right-wrong-reasons-diagnosing-syntactic/), [WANLI](https://md.research.elamur.ai/free/en/papers/liu-2022-wanli-worker-ai-collaboration-natural/), [diverse NLI recasts](https://md.research.elamur.ai/free/en/papers/poliak-2018-collecting-diverse-natural-language-inference/) |
| Commonsense | [COPA](https://md.research.elamur.ai/free/en/papers/gordon-2012-semeval-2012-task-7-choice/), [XCOPA](https://md.research.elamur.ai/free/en/papers/ponti-2020-xcopa-multilingual-dataset-causal-commonsense/), [PIQA](https://md.research.elamur.ai/free/en/papers/bisk-2019-piqa-reasoning-about-physical-commonsense/), [SocialIQA](https://md.research.elamur.ai/free/en/papers/sap-2019-socialiqa-commonsense-reasoning-about-social/), [Cosmos QA](https://md.research.elamur.ai/free/en/papers/huang-2019-cosmos-qa-machine-reading-comprehension/), [ReCoRD](https://md.research.elamur.ai/free/en/papers/zhang-2018-record-bridging-gap-between-human/), [CoS-E](https://md.research.elamur.ai/free/en/papers/rajani-2019-explain-yourself-leveraging-language-models/), [COMET-ATOMIC 2020](https://md.research.elamur.ai/free/en/papers/hwang-2020-comet-atomic-2020-symbolic-neural/) |
| Reading and multi-hop | [RACE](https://md.research.elamur.ai/free/en/papers/lai-2017-race-large-scale-reading-comprehension/), [DROP](https://md.research.elamur.ai/free/en/papers/dua-2019-drop-reading-comprehension-benchmark-requiring/), [2WikiMultiHopQA](https://md.research.elamur.ai/free/en/papers/ho-2020-constructing-multi-hop-qa-dataset/), [MuSiQue](https://md.research.elamur.ai/free/en/papers/trivedi-2022-musique-multihop-questions-single-hop/), [MuTual](https://md.research.elamur.ai/free/en/papers/cui-2020-mutual-dataset-multi-turn-dialogue/) |
| Logic | [LogiQA](https://md.research.elamur.ai/free/en/papers/liu-2020-logiqa-challenge-dataset-machine-reading/), [ReClor](https://md.research.elamur.ai/free/en/papers/yu-2020-reclor-reading-comprehension-dataset-requiring/), [FOLIO](https://md.research.elamur.ai/free/en/papers/han-2022-folio-natural-language-reasoning-first/), [Argument Reasoning Comprehension](https://md.research.elamur.ai/free/en/papers/habernal-2018-argument-reasoning-comprehension-task-identification/) |
| Mathematics | [GSM8K](https://md.research.elamur.ai/free/en/papers/cobbe-2021-training-verifiers-solve-math-word/), [MATH](https://md.research.elamur.ai/free/en/papers/hendrycks-2021-measuring-mathematical-problem-solving-math/), [SVAMP](https://md.research.elamur.ai/free/en/papers/patel-2021-are-nlp-models-really-able/), [MGSM](https://md.research.elamur.ai/free/en/papers/shi-2022-language-models-multilingual-chain-thought/), [LILA](https://md.research.elamur.ai/free/en/papers/mishra-2022-lila-unified-benchmark-mathematical-reasoning/), [PRM800K](https://md.research.elamur.ai/free/en/papers/lightman-2023-let-s-verify-step-step/), [MiniF2F](https://md.research.elamur.ai/free/en/papers/zheng-2021-minif2f-cross-system-benchmark-formal/) |
| General and expert | [BIG-Bench Hard](https://md.research.elamur.ai/free/en/papers/suzgun-2023-challenging-big-bench-tasks-whether/), [MMLU-Pro](https://md.research.elamur.ai/free/en/papers/wang-2024-mmlu-pro-more-robust-challenging/), [SciBench](https://md.research.elamur.ai/free/en/papers/wang-2023-scibench-evaluating-college-level-scientific/), [LongBench v2](https://md.research.elamur.ai/free/en/papers/bai-2024-longbench-v2-deeper-understanding/), [LegalBench](https://md.research.elamur.ai/free/en/papers/guha-2023-legalbench-collaboratively-built-benchmark-measuring/) |
| Multimodal | [CLEVR](https://md.research.elamur.ai/free/en/papers/johnson-2016-clevr-diagnostic-dataset-compositional-language/), [GQA](https://md.research.elamur.ai/free/en/papers/hudson-2019-gqa-new-dataset-real-world/), [NLVR2](https://md.research.elamur.ai/free/en/papers/suhr-2019-corpus-reasoning-natural-language-grounded/), [ChartQA](https://md.research.elamur.ai/free/en/papers/masry-2022-chartqa-benchmark-question-answering-about/), [MathVista](https://md.research.elamur.ai/free/en/papers/lu-2023-mathvista-evaluating-mathematical-reasoning-foundation/), [MATH-Vision](https://md.research.elamur.ai/free/en/papers/wang-2024-measuring-multimodal-mathematical-reasoning-math/), [MMMU](https://md.research.elamur.ai/free/en/papers/yue-2023-mmmu-massive-multi-discipline-multimodal/), [MME](https://md.research.elamur.ai/free/en/papers/fu-2023-mme-comprehensive-evaluation-benchmark-multimodal/) |

Most of these sets were built because an earlier one had been solved too easily, often through shortcuts, and saturation keeps arriving faster; newer sets therefore move toward competition mathematics, expert exams and formal proofs.

## Landmark papers in this library

Citations of the papers on this topic, by when they were published Papers published in 2020–2024 gather 60% of the 134,395 citations of the papers on this topic.

**The numbers**

| Published | Papers and their citations |
| --- | --- |
| 2025–2029 | 97 papers, 12,851 citations |
| 2020–2024 | 288 papers, 80,695 citations |
| 2015–2019 | 92 papers, 30,000 citations |
| 2010–2014 | 21 papers, 4,542 citations |
| 2005–2009 | 7 papers, 985 citations |
| 2000–2004 | 1 papers, 28 citations |
| 1990–1994 | 1 papers, 44 citations |
| 1980–1984 | 1 papers, 5,250 citations |

### Chain-of-thought prompting (2022)

[Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://md.research.elamur.ai/free/en/papers/wei-2022-chain-thought-prompting-elicits-reasoning/), from Google, is the most cited work on the topic. By adding hand-written intermediate steps to a handful of examples in the prompt, the authors raised PaLM's grade-school math accuracy several-fold without any fine-tuning, and showed that the benefit only appears once models are very large. Almost every later prompting method, and the idea of training models to think aloud, starts here.

### Experts and novices in physics (1981)

[Categorization and Representation of Physics Problems by Experts and Novices](https://md.research.elamur.ai/free/en/papers/chi-1981-categorization-representation-physics-problems-experts/) is the oldest landmark and comes from cognitive psychology. Its finding, that experts group problems by principle while novices group them by surface features, remains a reference point for anyone designing problem representations or diagnosing why models fail on rephrased problems.

### DeepSeek-R1 (2025)

[DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning](https://md.research.elamur.ai/free/en/papers/deepseek-ai-2025-deepseek-r1-incentivizing-reasoning-capability/) trained a model with reinforcement learning and simple rule-based rewards, starting directly from a pretrained base model. Long reasoning, self-checking and backtracking emerged without human-written traces, and the released weights and distilled small models made reasoning-model research broadly accessible.

### Zero-shot chain of thought (2022)

[Large language models are zero-shot reasoners](https://md.research.elamur.ai/free/en/papers/kojima-2022-large-language-models-zero-shot/) found that one generic instruction to reason step by step, placed before the answer, produced large gains on arithmetic and logic tasks without any examples. The result became the default baseline for reasoning evaluation and a building block for automatic demonstration methods.

### Verifiers and GSM8K (2021)

[Training Verifiers to Solve Math Word Problems](https://md.research.elamur.ai/free/en/papers/cobbe-2021-training-verifiers-solve-math-word/) gave the field its most used math benchmark and the generate-then-rank recipe. Its observation that a learned judge over many samples beats simply scaling the generator foreshadowed reward models and test-time compute.

### Self-consistency (2022)

The paper [Self-Consistency Improves Chain of Thought Reasoning in Language Models](https://md.research.elamur.ai/free/en/papers/wang-2022-self-consistency-improves-chain-thought/) replaced a single greedy chain with many sampled chains and a vote over their final answers. The idea is training-free, works with any model, and remains the simplest strong way to trade compute for accuracy.

### ReAct (2022)

[ReAct: Synergizing Reasoning and Acting in Language Models](https://md.research.elamur.ai/free/en/papers/yao-2022-react-synergizing-reasoning-acting-language/) let a model alternate between written thoughts and actions in an environment or a search API. It reduced invented facts on knowledge-heavy questions and set the pattern for tool-using agents.

### MATH (2021)

[Measuring Mathematical Problem Solving With the MATH Dataset](https://md.research.elamur.ai/free/en/papers/hendrycks-2021-measuring-mathematical-problem-solving-math/) collected competition problems with full worked solutions across several subjects and difficulty levels. Early models scored in single digits, and the authors argued that scale alone would not close the gap; the benchmark became the main yardstick for mathematical reasoning until frontier models approached its ceiling.

### DeepSeekMath and GRPO (2024)

[DeepSeekMath](https://md.research.elamur.ai/free/en/papers/shao-2024-deepseekmath-pushing-limits-mathematical-reasoning/) built a large math corpus from filtered web pages and introduced Group Relative Policy Optimization, which estimates advantages by comparing sampled answers to the same question instead of training a value network. GRPO became the workhorse algorithm of open reasoning-model training.

### Let's Verify Step by Step (2023)

[Let's Verify Step by Step](https://md.research.elamur.ai/free/en/papers/lightman-2023-let-s-verify-step-step/) compared reward models trained on final answers with reward models trained on human judgements of each step, and found the step-level version clearly better on difficult math. Its released step labels made process reward models a research area of their own.

### Pragmatic reasoning in reference games (2012)

[Predicting Pragmatic Reasoning in Language Games](https://md.research.elamur.ai/free/en/papers/frank-2012-predicting-pragmatic-reasoning-language-games/) modelled a listener as a Bayesian reasoner about an informative speaker and predicted human choices in simple reference games with no free parameters. It is a founding paper of the Rational Speech Acts framework and a reminder that reasoning about language includes reasoning about other minds; [Reasoning about Pragmatics with Neural Listeners and Speakers](https://md.research.elamur.ai/free/en/papers/andreas-2016-reasoning-about-pragmatics-neural-listeners/) carried the idea into neural models.

## People

Authors with the most papers on this topic Yejin Choi leads with 16 papers; the 12 authors shown wrote 102 of the papers on this topic between them, counting each paper once per author.

**The numbers**

| Author | Papers here · their citations |
| --- | --- |
| [Yejin Choi](https://md.research.elamur.ai/free/en/authors/yejin-choi/) | 16 papers · 6,589 citations |
| [Ronan Le Bras](https://md.research.elamur.ai/free/en/authors/ronan-le-bras/) | 10 papers · 3,652 citations |
| [Sean Welleck](https://md.research.elamur.ai/free/en/authors/sean-welleck/) | 10 papers · 2,199 citations |
| [Mohit Bansal](https://md.research.elamur.ai/free/en/authors/mohit-bansal/) | 10 papers · 1,159 citations |
| [Benjamin Van Durme](https://md.research.elamur.ai/free/en/authors/benjamin-van-durme/) | 9 papers · 1,460 citations |
| [Chandra Bhagavatula](https://md.research.elamur.ai/free/en/authors/chandra-bhagavatula/) | 8 papers · 2,231 citations |
| [Pan Lu](https://md.research.elamur.ai/free/en/authors/pan-lu/) | 7 papers · 1,533 citations |
| [Adam Poliak](https://md.research.elamur.ai/free/en/authors/adam-poliak/) | 7 papers · 705 citations |
| [Rachel Rudinger](https://md.research.elamur.ai/free/en/authors/rachel-rudinger/) | 7 papers · 670 citations |
| [Zhihong Shao](https://md.research.elamur.ai/free/en/authors/zhihong-shao/) | 6 papers · 3,061 citations |
| [Ashish Sabharwal](https://md.research.elamur.ai/free/en/authors/ashish-sabharwal/) | 6 papers · 1,939 citations |
| [Matt Gardner](https://md.research.elamur.ai/free/en/authors/matt-gardner/) | 6 papers · 1,866 citations |

- **[Yejin Choi](https://md.research.elamur.ai/free/en/authors/yejin-choi/)** (now at Stanford University) is the most prolific author on this topic. Her group shaped commonsense reasoning through COMET, PIQA, Cosmos QA, SocialIQA and Social Bias Frames, combining large-scale data collection with generative models of everyday inference.
- **[Ronan Le Bras](https://md.research.elamur.ai/free/en/authors/ronan-le-bras/)** co-authored much of the same commonsense line, including PIQA and Cosmos QA, and later analyses of what transformers can compose, such as [Faith and Fate](https://md.research.elamur.ai/free/en/papers/dziri-2023-faith-fate-limits-transformers-compositionality/).
- **[Chandra Bhagavatula](https://md.research.elamur.ai/free/en/authors/chandra-bhagavatula/)** led [Abductive Commonsense Reasoning](https://md.research.elamur.ai/free/en/papers/bhagavatula-2019-abductive-commonsense-reasoning/), which framed the choice of the most plausible explanation as a language task, and contributed to symbolic knowledge distillation.
- **[Maarten Sap](https://md.research.elamur.ai/free/en/authors/maarten-sap/)** worked on social commonsense, from SocialIQA and Social Bias Frames to the dialogue corpus [SODA](https://md.research.elamur.ai/free/en/papers/kim-2023-soda-million-scale-dialogue-distillation/).
- **[Sean Welleck](https://md.research.elamur.ai/free/en/authors/sean-welleck/)** (Carnegie Mellon University) spans dialogue NLI, compositionality limits and open mathematical models such as [Llemma](https://md.research.elamur.ai/free/en/papers/azerbayev-2023-llemma-open-language-model-mathematics/).
- **[Benjamin Van Durme](https://md.research.elamur.ai/free/en/authors/benjamin-van-durme/)** (Johns Hopkins University and Microsoft) contributed to inference datasets including ReCoRD, ordinal common-sense inference and recast NLI collections, and to the study of social bias in elicited inferences.
- **[Ashish Sabharwal](https://md.research.elamur.ai/free/en/authors/ashish-sabharwal/)** and **[Tushar Khot](https://md.research.elamur.ai/free/en/authors/tushar-khot/)** (Allen Institute for AI) co-authored MuSiQue, interleaved retrieval with chain-of-thought, complexity-based prompting and the specialisation of small models for multi-step reasoning.
- **[Matt Gardner](https://md.research.elamur.ai/free/en/authors/matt-gardner/)** led a cluster of reading-comprehension benchmarks that demand discrete reasoning, including DROP, Quoref and ROPES, and showed with colleagues that many multi-hop questions can be shortcut.
- **[Denny Zhou](https://md.research.elamur.ai/free/en/authors/denny-zhou/)** and **[Jason Wei](https://md.research.elamur.ai/free/en/authors/jason-wei/)**, with **[Quoc Le](https://md.research.elamur.ai/free/en/authors/quoc-v-le/)** and **[Xuezhi Wang](https://md.research.elamur.ai/free/en/authors/xuezhi-wang/)**, form the Google group behind chain-of-thought prompting, self-consistency, multilingual chain of thought and BIG-Bench Hard. Quoc Le also co-authored the AlphaGeometry paper.
- **[Pan Lu](https://md.research.elamur.ai/free/en/authors/pan-lu/)** is the lead author of MathVista and Chameleon and co-authored the LILA benchmark, linking mathematical reasoning with vision and tool use.
- **[Yuhuai Wu](https://md.research.elamur.ai/free/en/authors/yuhuai-wu/)** contributed to Minerva, STaR, [Draft, Sketch, and Prove](https://md.research.elamur.ai/free/en/papers/jiang-2022-draft-sketch-prove-guiding-formal/) and AlphaGeometry, bridging informal and formal mathematical reasoning.
- **[Zhihong Shao](https://md.research.elamur.ai/free/en/authors/zhihong-shao/)**, **[Qihao Zhu](https://md.research.elamur.ai/free/en/authors/qihao-zhu/)** and **[Junxiao Song](https://md.research.elamur.ai/free/en/authors/junxiao-song/)** are among the DeepSeek authors behind DeepSeekMath, the DeepSeek-Prover models and DeepSeek-R1.
- **[Iryna Gurevych](https://md.research.elamur.ai/free/en/authors/iryna-gurevych/)** (Technische Universität Darmstadt) brought entailment into fact verification and summary evaluation, and argument reasoning into the field through the implicit-warrant task.
- **[William Yang Wang](https://md.research.elamur.ai/free/en/authors/william-yang-wang/)** (University of California, Santa Barbara) co-authored FinQA, ConvFinQA and Logic-LM.

## Institutions and places

Where the authors on this topic work now Stanford University is the current home of authors of 55 papers on this topic; counted are the authors with a page here, by their latest institution.

**The numbers**

| Institution | Papers |
| --- | --- |
| [Stanford University](https://md.research.elamur.ai/free/en/universities/stanford-university/) | 55 papers |
| [Carnegie Mellon University](https://md.research.elamur.ai/free/en/universities/carnegie-mellon-university/) | 44 papers |
| [Google](https://md.research.elamur.ai/free/en/companies/google/) | 38 papers |
| [Allen Institute for Artificial Intelligence](https://md.research.elamur.ai/free/en/labs/allen-institute-for-artificial-intelligence/) | 33 papers |
| [Shanghai Jiao Tong University](https://md.research.elamur.ai/free/en/universities/shanghai-jiao-tong-university/) | 28 papers |
| [Microsoft](https://md.research.elamur.ai/free/en/companies/microsoft/) | 26 papers |
| [Tsinghua University](https://md.research.elamur.ai/free/en/universities/tsinghua-university/) | 24 papers |
| [University of Washington](https://md.research.elamur.ai/free/en/universities/university-of-washington/) | 24 papers |
| [Chinese Academy of Sciences](https://md.research.elamur.ai/free/en/government/chinese-academy-of-sciences/) | 23 papers |
| [Google DeepMind](https://md.research.elamur.ai/free/en/companies/google-deepmind/) | 23 papers |

The institutional map follows the history of the field. [Stanford University](https://md.research.elamur.ai/free/en/universities/stanford-university/) is the current home of the largest group of authors here, partly because several leading commonsense researchers have moved there. The [Allen Institute for Artificial Intelligence](https://md.research.elamur.ai/free/en/labs/allen-institute-for-artificial-intelligence/), together with the [University of Washington](https://md.research.elamur.ai/free/en/universities/university-of-washington/) and [Carnegie Mellon University](https://md.research.elamur.ai/free/en/universities/carnegie-mellon-university/), was the centre of the commonsense and reading-comprehension benchmarks of 2018 to 2022. Industrial labs dominate the prompting and training era: [Google](https://md.research.elamur.ai/free/en/companies/google/) and [Google DeepMind](https://md.research.elamur.ai/free/en/companies/google-deepmind/) for chain of thought, self-consistency, Minerva and AlphaGeometry, and [Microsoft](https://md.research.elamur.ai/free/en/companies/microsoft/) for Orca and multimodal work.

Countries of the authors on this topic United States comes first: authors working there now wrote 303 papers on this topic.

**The numbers**

| Country | Papers |
| --- | --- |
| [United States](https://md.research.elamur.ai/free/en/countries/united-states/) | 303 papers |
| [China](https://md.research.elamur.ai/free/en/countries/china/) | 178 papers |
| [United Kingdom](https://md.research.elamur.ai/free/en/countries/united-kingdom/) | 75 papers |
| [Germany](https://md.research.elamur.ai/free/en/countries/germany/) | 40 papers |
| [Canada](https://md.research.elamur.ai/free/en/countries/canada/) | 37 papers |
| Hong Kong | 32 papers |
| [Singapore](https://md.research.elamur.ai/free/en/countries/singapore/) | 26 papers |
| [Japan](https://md.research.elamur.ai/free/en/countries/japan/) | 24 papers |
| [Switzerland](https://md.research.elamur.ai/free/en/countries/switzerland/) | 22 papers |
| [Israel](https://md.research.elamur.ai/free/en/countries/israel/) | 19 papers |

The United States leads, with China a strong second and rising fastest in recent years. Chinese contributions come both from universities, such as [Shanghai Jiao Tong University](https://md.research.elamur.ai/free/en/universities/shanghai-jiao-tong-university/), [Tsinghua University](https://md.research.elamur.ai/free/en/universities/tsinghua-university/), [Peking University](https://md.research.elamur.ai/free/en/universities/peking-university/), [Zhejiang University](https://md.research.elamur.ai/free/en/universities/zhejiang-university/), [Beihang University](https://md.research.elamur.ai/free/en/universities/beihang-university/) and the [Chinese Academy of Sciences](https://md.research.elamur.ai/free/en/government/chinese-academy-of-sciences/), and from companies such as [Alibaba Group](https://md.research.elamur.ai/free/en/companies/alibaba-group/), home of the Qwen models. Other universities with a visible presence include [Massachusetts Institute of Technology](https://md.research.elamur.ai/free/en/universities/massachusetts-institute-of-technology/), [University of California, Berkeley](https://md.research.elamur.ai/free/en/universities/university-of-california-berkeley/), [Johns Hopkins University](https://md.research.elamur.ai/free/en/universities/johns-hopkins-university/), [New York University](https://md.research.elamur.ai/free/en/universities/new-york-university/) and [Princeton University](https://md.research.elamur.ai/free/en/universities/princeton-university/).

Where the papers on this topic were published Annual Meeting of the Association for Computational Linguistics (ACL) carries the most, 74 of 508.

**The numbers**

| Venue | Papers |
| --- | --- |
| [Annual Meeting of the Association for Computational Linguistics (ACL)](https://md.research.elamur.ai/free/en/venues/acl/) | 74 papers |
| [Conference on Empirical Methods in Natural Language Processing (EMNLP)](https://md.research.elamur.ai/free/en/venues/emnlp/) | 54 papers |
| [Findings of the Association for Computational Linguistics](https://md.research.elamur.ai/free/en/venues/findings/) | 43 papers |
| [Language Resources and Evaluation Conference (LREC)](https://md.research.elamur.ai/free/en/venues/lrec/) | 31 papers |
| [Conference on Neural Information Processing Systems (NeurIPS)](https://md.research.elamur.ai/free/en/venues/neural-information-processing-systems/) | 22 papers |
| [Conference of the North American Chapter of the Association for Computational Linguistics (NAACL)](https://md.research.elamur.ai/free/en/venues/naacl/) | 18 papers |
| [Journal of Chinese Information Processing (中文信息学报)](https://md.research.elamur.ai/free/en/venues/journal-of-chinese-information-processing/) | 9 papers |
| [Conference of the European Chapter of the Association for Computational Linguistics (EACL)](https://md.research.elamur.ai/free/en/venues/eacl/) | 8 papers |
| [Proceedings of the AAAI Conference on Artificial Intelligence](https://md.research.elamur.ai/free/en/venues/proceedings-of-the-aaai-conference-on-artificial-intelligence/) | 8 papers |
| [International Conference on Computational Linguistics (COLING)](https://md.research.elamur.ai/free/en/venues/coling/) | 7 papers |

Peer-reviewed work appeared mostly at ACL, EMNLP, NAACL, COLING, LREC and their Findings volumes, while many of the most influential recent papers circulated first as preprints and technical reports.

## Open questions

- **Is it reasoning?** Strong scores can coexist with brittleness. [Faith and Fate](https://md.research.elamur.ai/free/en/papers/dziri-2023-faith-fate-limits-transformers-compositionality/) argues that transformers often reduce multi-step composition to pattern matching, [GSM-Symbolic](https://md.research.elamur.ai/free/en/papers/mirzadeh-2024-gsm-symbolic-understanding-limitations-mathematical/) shows sensitivity to changes in names and numbers, and [Impact of Pretraining Term Frequencies on Few-Shot Numerical Reasoning](https://md.research.elamur.ai/free/en/papers/razeghi-2022-impact-pretraining-term-frequencies-few/) links arithmetic accuracy to how often numbers occur in training data.
- **Faithfulness of explanations.** Written chains can be persuasive and wrong at the same time, and a correct answer does not certify a correct chain ([The Unreliability of Explanations in Few-shot Prompting](https://md.research.elamur.ai/free/en/papers/ye-2022-unreliability-explanations-few-shot-prompting/)). How to score reasoning steps directly, as [ROSCOE](https://md.research.elamur.ai/free/en/papers/golovneva-2022-roscoe-suite-metrics-scoring-step/) attempts, is unsettled.
- **When does thinking help?** [To CoT or not to CoT?](https://md.research.elamur.ai/free/en/papers/sprague-2024-cot-cot-chain-thought-helps/) finds the gains concentrated in math and symbolic tasks, and [On Second Thought, Let's Not Think Step by Step!](https://md.research.elamur.ai/free/en/papers/shaikh-2023-second-thought-let-s-not/) shows that reasoning prompts can increase harmful outputs in some settings.
- **Cost and overthinking.** Long chains are expensive and often redundant. [Do NOT Think That Much for 2+3=?](https://md.research.elamur.ai/free/en/papers/chen-2024-think-that-much-2-3/), [Token-Budget-Aware LLM Reasoning](https://md.research.elamur.ai/free/en/papers/han-2024-token-budget-aware-llm-reasoning/) and the survey [Stop Overthinking](https://md.research.elamur.ai/free/en/papers/sui-2025-stop-overthinking-survey-efficient-reasoning/) look for ways to spend effort only where needed; longer inputs also degrade reasoning ([Same Task, More Tokens](https://md.research.elamur.ai/free/en/papers/levy-2024-same-task-more-tokens-impact/)).
- **What reinforcement learning adds.** Whether RL creates new capabilities or only makes existing ones more reliable is under active debate.
- **Beyond English and beyond benchmarks.** Multilingual reasoning sets such as MGSM and XCOPA remain small, and benchmark saturation outpaces the creation of trustworthy new tests.
- **Safety of capable reasoners.** System cards for reasoning models report new risks, such as models that pursue goals in agentic tests in ways their operators did not intend, which ties the topic to [Bias, fairness & safety](https://md.research.elamur.ai/free/en/topics/bias-fairness-safety/).

## Neighbouring topics

Neighbouring topics: papers shared Language models & pretraining shares the most: 134 of the 508 papers here (26%) stand on both.

**The numbers**

| Topic | Papers shared |
| --- | --- |
| [Language models & pretraining](https://md.research.elamur.ai/free/en/topics/language-models/) | 134 shared · 26% |
| [Evaluation & benchmarks](https://md.research.elamur.ai/free/en/topics/evaluation-benchmarks/) | 44 shared · 9% |
| [Question answering](https://md.research.elamur.ai/free/en/topics/question-answering/) | 37 shared · 7% |
| [English](https://md.research.elamur.ai/free/en/topics/english/) | 36 shared · 7% |
| [Vision & language](https://md.research.elamur.ai/free/en/topics/multimodal/) | 23 shared · 5% |
| [Knowledge graphs & ontologies](https://md.research.elamur.ai/free/en/topics/knowledge-graphs/) | 14 shared · 3% |
| [Semitic languages](https://md.research.elamur.ai/free/en/topics/semitic-languages/) | 14 shared · 3% |
| [Multilingual & cross-lingual NLP](https://md.research.elamur.ai/free/en/topics/multilingual-nlp/) | 12 shared · 2% |
| [Bias, fairness & safety](https://md.research.elamur.ai/free/en/topics/bias-fairness-safety/) | 9 shared · 2% |
| [Sentiment & emotion](https://md.research.elamur.ai/free/en/topics/sentiment-emotion/) | 9 shared · 2% |

The closest neighbour by far is [Language models & pretraining](https://md.research.elamur.ai/free/en/topics/language-models/): most recent reasoning work is about what pretrained models can do and how to train them further. [Evaluation & benchmarks](https://md.research.elamur.ai/free/en/topics/evaluation-benchmarks/) shares the many test sets described above, and [Question answering](https://md.research.elamur.ai/free/en/topics/question-answering/) shares the multi-hop and reading-comprehension work. Visual reasoning connects to [Vision & language](https://md.research.elamur.ai/free/en/topics/multimodal/), and work on retrieving evidence during reasoning connects to [Information retrieval & search](https://md.research.elamur.ai/free/en/topics/information-retrieval/) and [Knowledge graphs & ontologies](https://md.research.elamur.ai/free/en/topics/knowledge-graphs/).

Smaller overlaps are telling. Legal reasoning benchmarks link the topic to [Legal AI](https://md.research.elamur.ai/free/en/topics/legal-ai/); dialogue inference and MuTual link it to [Dialogue systems](https://md.research.elamur.ai/free/en/topics/dialogue-systems/); sarcasm detection framed as inference links it to [Sentiment & emotion](https://md.research.elamur.ai/free/en/topics/sentiment-emotion/); and multilingual chain-of-thought and XCOPA link it to [Multilingual & cross-lingual NLP](https://md.research.elamur.ai/free/en/topics/multilingual-nlp/) and [Machine translation](https://md.research.elamur.ai/free/en/topics/machine-translation/). Pragmatic inference links it to [Implicature](https://md.research.elamur.ai/free/en/topics/implicature/), reasoning over entity mentions to [Coreference resolution](https://md.research.elamur.ai/free/en/topics/coreference-resolution/), cheaper inference to [Efficient models & compression](https://md.research.elamur.ai/free/en/topics/efficient-models/), and analyses of internal computation to [Interpretability](https://md.research.elamur.ai/free/en/topics/interpretability/). Program-aided methods connect with [Code & programming languages](https://md.research.elamur.ai/free/en/topics/code-and-programming/), and logical forms with [Semantic parsing & meaning representation](https://md.research.elamur.ai/free/en/topics/semantic-parsing/).

The shelves this topic stands on Artificial Intelligence: 487 of the 508 papers here.

**The numbers**

| Shelf | Papers |
| --- | --- |
| [Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) | 487 papers |
| [Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) | 467 papers |
| [Computer Vision and Pattern Recognition](https://md.research.elamur.ai/free/en/shelves/computer-science/computer-vision-and-pattern-recognition/) | 180 papers |
| [Computational Theory and Mathematics](https://md.research.elamur.ai/free/en/shelves/computer-science/computational-theory-and-mathematics/) | 47 papers |
| [Applied Mathematics](https://md.research.elamur.ai/free/en/shelves/mathematics/applied-mathematics/) | 15 papers |
| [Information Systems](https://md.research.elamur.ai/free/en/shelves/computer-science/information-systems/) | 15 papers |
| [Safety Research](https://md.research.elamur.ai/free/en/shelves/social-sciences/safety-research/) | 9 papers |
| [Developmental and Educational Psychology](https://md.research.elamur.ai/free/en/shelves/psychology/developmental-and-educational-psychology/) | 8 papers |
| [Experimental and Cognitive Psychology](https://md.research.elamur.ai/free/en/shelves/psychology/experimental-and-cognitive-psychology/) | 8 papers |
| [Political Science and International Relations](https://md.research.elamur.ai/free/en/shelves/social-sciences/political-science-and-international-relations/) | 7 papers |
| [Health Informatics](https://md.research.elamur.ai/free/en/shelves/medicine/health-informatics/) | 5 papers |
| [Sociology and Political Science](https://md.research.elamur.ai/free/en/shelves/social-sciences/sociology-and-political-science/) | 5 papers |

On the shelves of the library, the topic stands mostly on [Linguistics & languages](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/), [Natural language processing](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) and [Mathematics & statistics](https://md.research.elamur.ai/free/en/shelves/mathematics/), with further papers on [AI & machine learning](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/), [Machine translation](https://md.research.elamur.ai/free/en/topics/machine-translation/) and [Law](https://md.research.elamur.ai/free/en/shelves/social-sciences/law/).

## Key works in this library

- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://md.research.elamur.ai/free/en/papers/wei-2022-chain-thought-prompting-elicits-reasoning/) — worked steps in the prompt unlock multi-step reasoning at scale.
- [Large language models are zero-shot reasoners](https://md.research.elamur.ai/free/en/papers/kojima-2022-large-language-models-zero-shot/) — one generic trigger phrase replaces hand-written examples.
- [Self-Consistency Improves Chain of Thought Reasoning in Language Models](https://md.research.elamur.ai/free/en/papers/wang-2022-self-consistency-improves-chain-thought/) — vote over many sampled chains.
- [Least-to-most prompting enables complex reasoning in large language models](https://md.research.elamur.ai/free/en/papers/al-2022-least-most-prompting-enables-complex/) — decompose, then solve in order.
- [ReAct: Synergizing Reasoning and Acting in Language Models](https://md.research.elamur.ai/free/en/papers/yao-2022-react-synergizing-reasoning-acting-language/) — thoughts interleaved with tool actions.
- [Training Verifiers to Solve Math Word Problems](https://md.research.elamur.ai/free/en/papers/cobbe-2021-training-verifiers-solve-math-word/) — GSM8K and generate-then-rank.
- [Measuring Mathematical Problem Solving With the MATH Dataset](https://md.research.elamur.ai/free/en/papers/hendrycks-2021-measuring-mathematical-problem-solving-math/) — competition math as a benchmark.
- [Let's Verify Step by Step](https://md.research.elamur.ai/free/en/papers/lightman-2023-let-s-verify-step-step/) — step-level reward models beat answer-level ones.
- [STaR: Bootstrapping Reasoning With Reasoning](https://md.research.elamur.ai/free/en/papers/zelikman-2022-star-bootstrapping-reasoning-reasoning/) — learning from one's own correct rationales.
- [Solving Quantitative Reasoning Problems with Language Models](https://md.research.elamur.ai/free/en/papers/lewkowycz-2022-solving-quantitative-reasoning-problems-language/) — Minerva, a model further trained on technical text.
- [DeepSeekMath](https://md.research.elamur.ai/free/en/papers/shao-2024-deepseekmath-pushing-limits-mathematical-reasoning/) — web-mined math data and GRPO.
- [DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning](https://md.research.elamur.ai/free/en/papers/deepseek-ai-2025-deepseek-r1-incentivizing-reasoning-capability/) — reasoning from reinforcement learning with checkable rewards.
- [Solving olympiad geometry without human demonstrations](https://md.research.elamur.ai/free/en/papers/trinh-2024-solving-olympiad-geometry-without-human/) — AlphaGeometry, neural proposals plus symbolic deduction.
- [Challenging BIG-Bench Tasks and Whether Chain-of-Thought Can Solve Them](https://md.research.elamur.ai/free/en/papers/suzgun-2023-challenging-big-bench-tasks-whether/) — the BIG-Bench Hard suite.
- [Language Models Don't Always Say What They Think: Unfaithful Explanations in Chain-of-Thought Prompting](https://md.research.elamur.ai/free/en/papers/turpin-2023-language-models-don-t-always/) — explanations that hide the real cause.
- [Right for the Wrong Reasons: Diagnosing Syntactic Heuristics in Natural Language Inference](https://md.research.elamur.ai/free/en/papers/mccoy-2019-right-wrong-reasons-diagnosing-syntactic/) — HANS and shortcut learning.
- [Reasoning about Entailment with Neural Attention](https://md.research.elamur.ai/free/en/papers/rocktaschel-2015-reasoning-about-entailment-neural-attention/) — attention-based neural NLI.
- [A Decomposable Attention Model for Natural Language Inference](https://md.research.elamur.ai/free/en/papers/parikh-2016-decomposable-attention-model-natural-language/) — compact alignment-based NLI.
- [e-SNLI: Natural Language Inference with Natural Language Explanations](https://md.research.elamur.ai/free/en/papers/camburu-2018-e-snli-natural-language-inference/) — human explanations for every inference pair.
- [COMET: Commonsense Transformers for Automatic Knowledge Graph Construction](https://md.research.elamur.ai/free/en/papers/bosselut-2019-comet-commonsense-transformers-automatic-knowledge/) — generating commonsense knowledge.
- [PIQA: Reasoning about Physical Commonsense in Natural Language](https://md.research.elamur.ai/free/en/papers/bisk-2019-piqa-reasoning-about-physical-commonsense/) — everyday physical knowledge.
- [DROP: A Reading Comprehension Benchmark Requiring Discrete Reasoning Over Paragraphs](https://md.research.elamur.ai/free/en/papers/dua-2019-drop-reading-comprehension-benchmark-requiring/) — counting, sorting and arithmetic over text.
- [GQA: A New Dataset for Real-World Visual Reasoning and Compositional Question Answering](https://md.research.elamur.ai/free/en/papers/hudson-2019-gqa-new-dataset-real-world/) — compositional questions about real photographs.
- [Towards AI-Complete Question Answering: A Set of Prerequisite Toy Tasks](https://md.research.elamur.ai/free/en/papers/weston-2015-towards-ai-complete-question-answering/) — bAbI, reasoning skills as unit tests.
- [Predicting Pragmatic Reasoning in Language Games](https://md.research.elamur.ai/free/en/papers/frank-2012-predicting-pragmatic-reasoning-language-games/) — Bayesian listeners and informative speakers.

## Papers (508)

**1** [2](https://md.research.elamur.ai/free/en/topics/reasoning/page/2/) … [6](https://md.research.elamur.ai/free/en/topics/reasoning/page/6/) [next →](https://md.research.elamur.ai/free/en/topics/reasoning/page/2/)

1. [**Chain-of-Thought Prompting Elicits Reasoning in Large Language Models arXiv:2201.11903**Jason Wei, Xuezhi Wang, Dale Schuurmans, et al.*2022 · 11,308 citations*](https://md.research.elamur.ai/free/en/papers/wei-2022-chain-thought-prompting-elicits-reasoning/)
2. [**Categorization and Representation of Physics Problems by Experts and Novices**Michelene T. H. Chi, Paul J. Feltovich, Robert Glaser*1981 · 5,250 citations*](https://md.research.elamur.ai/free/en/papers/chi-1981-categorization-representation-physics-problems-experts/)
3. **[DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning](https://md.research.elamur.ai/free/en/papers/deepseek-ai-2025-deepseek-r1-incentivizing-reasoning-capability/) [Nature](https://md.research.elamur.ai/free/en/venues/nature/)**DeepSeek-AI, Daya Guo, Dejian Yang, et al.*2025 · 4,593 citations*
4. **[Large language models are zero-shot reasoners](https://md.research.elamur.ai/free/en/papers/kojima-2022-large-language-models-zero-shot/) [Conference on Neural Information Processing Systems (NeurIPS)](https://md.research.elamur.ai/free/en/venues/neural-information-processing-systems/)**Takeshi Kojima, Shixiang Shane Gu, Machel Reid, et al.*2022 · 3,795 citations*
5. [**Training Verifiers to Solve Math Word Problems arXiv:2110.14168**Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, et al.*2021 · 3,680 citations*](https://md.research.elamur.ai/free/en/papers/cobbe-2021-training-verifiers-solve-math-word/)
6. [**Qwen3 Technical Report arXiv:2505.09388**An Yang, Anfeng Li, Baosong Yang, et al.*2025 · 3,276 citations*](https://md.research.elamur.ai/free/en/papers/yang-2025-qwen3-technical-report/)
7. [**Self-Consistency Improves Chain of Thought Reasoning in Language Models arXiv:2203.11171**Xuezhi Wang, Jason Wei, Dale Schuurmans, et al.*2022 · 3,041 citations*](https://md.research.elamur.ai/free/en/papers/wang-2022-self-consistency-improves-chain-thought/)
8. [**ReAct: Synergizing Reasoning and Acting in Language Models arXiv:2210.03629**Shunyu Yao, Jeffrey Zhao, Dian Yu, et al.*2022 · 2,497 citations*](https://md.research.elamur.ai/free/en/papers/yao-2022-react-synergizing-reasoning-acting-language/)
9. [**Measuring Mathematical Problem Solving With the MATH Dataset arXiv:2103.03874**Dan Hendrycks, Collin Burns, Saurav Kadavath, et al.*2021 · 2,320 citations*](https://md.research.elamur.ai/free/en/papers/hendrycks-2021-measuring-mathematical-problem-solving-math/)
10. [**DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models arXiv:2402.03300**Zhihong Shao, Peiyi Wang, Qihao Zhu, et al.*2024 · 2,013 citations*](https://md.research.elamur.ai/free/en/papers/shao-2024-deepseekmath-pushing-limits-mathematical-reasoning/)
11. **[GQA: A New Dataset for Real-World Visual Reasoning and Compositional Question Answering](https://md.research.elamur.ai/free/en/papers/hudson-2019-gqa-new-dataset-real-world/) [IEEE Computer Society Conference on Computer Vision and Pattern Recognition](https://md.research.elamur.ai/free/en/venues/ieee-computer-society-conference-on-computer-vision-and-pattern-recognition/)**Drew A. Hudson, Christopher D. Manning*2019 · 2,006 citations*
12. **[Exploiting Cloze-Questions for Few-Shot Text Classification and Natural Language Inference](https://md.research.elamur.ai/free/en/papers/schick-2021-exploiting-cloze-questions-few-shot/) [Conference of the European Chapter of the Association for Computational Linguistics (EACL)](https://md.research.elamur.ai/free/en/venues/eacl/)**Timo Schick, Hinrich Schütze*2021 · 1,803 citations*
13. **[Reasoning With Neural Tensor Networks for Knowledge Base Completion](https://md.research.elamur.ai/free/en/papers/socher-2013-reasoning-neural-tensor-networks-knowledge/) [Conference on Neural Information Processing Systems (NeurIPS)](https://md.research.elamur.ai/free/en/venues/neural-information-processing-systems/)**Richard Socher, Danqi Chen, Christopher D. Manning, et al.*2013 · 1,798 citations*
14. **[CommonsenseQA: A Question Answering Challenge Targeting Commonsense Knowledge](https://md.research.elamur.ai/free/en/papers/talmor-2019-commonsenseqa-question-answering-challenge-targeting/) [Conference of the North American Chapter of the Association for Computational Linguistics (NAACL)](https://md.research.elamur.ai/free/en/venues/naacl/)**Alon Talmor, Jonathan Herzig, Nicholas Lourie, et al.*2019 · 1,639 citations*
15. [**Right for the Wrong Reasons: Diagnosing Syntactic Heuristics in Natural Language Inference**R. Thomas McCoy, Ellie Pavlick, Tal Linzen*2019 · 1,635 citations*](https://md.research.elamur.ai/free/en/papers/mccoy-2019-right-wrong-reasons-diagnosing-syntactic/)
16. [**RACE: Large-scale ReAding Comprehension Dataset From Examinations**Guokun Lai, Qizhe Xie, Hanxiao Liu, et al.*2017 · 1,629 citations*](https://md.research.elamur.ai/free/en/papers/lai-2017-race-large-scale-reading-comprehension/)
17. **[A Decomposable Attention Model for Natural Language Inference](https://md.research.elamur.ai/free/en/papers/parikh-2016-decomposable-attention-model-natural-language/) [Conference on Empirical Methods in Natural Language Processing (EMNLP)](https://md.research.elamur.ai/free/en/venues/emnlp/)**Ankur Parikh, Oscar Täckström, Dipanjan Das, et al.*2016 · 1,601 citations*
18. **[COMET: Commonsense Transformers for Automatic Knowledge Graph Construction](https://md.research.elamur.ai/free/en/papers/bosselut-2019-comet-commonsense-transformers-automatic-knowledge/) [Annual Meeting of the Association for Computational Linguistics (ACL)](https://md.research.elamur.ai/free/en/venues/acl/)**Antoine Bosselut, Hannah Rashkin, Maarten Sap, et al.*2019 · 1,287 citations*
19. **[PIQA: Reasoning about Physical Commonsense in Natural Language](https://md.research.elamur.ai/free/en/papers/bisk-2019-piqa-reasoning-about-physical-commonsense/) [Proceedings of the AAAI Conference on Artificial Intelligence](https://md.research.elamur.ai/free/en/venues/proceedings-of-the-aaai-conference-on-artificial-intelligence/)**Yonatan Bisk, Rowan Zellers, Ronan Le Bras, et al.*2019 · 1,220 citations*
20. [**Least-to-most prompting enables complex reasoning in large language models***2022 · 1,148 citations*](https://md.research.elamur.ai/free/en/papers/al-2022-least-most-prompting-enables-complex/)
21. **[Challenging BIG-Bench Tasks and Whether Chain-of-Thought Can Solve Them](https://md.research.elamur.ai/free/en/papers/suzgun-2023-challenging-big-bench-tasks-whether/) [Findings of the Association for Computational Linguistics](https://md.research.elamur.ai/free/en/venues/findings/)**Mirac Suzgun, Nathan Scales, Nathanael Schärli, et al.*2023 · 1,096 citations*
22. [**OpenAI o1 System Card arXiv:2412.16720**OpenAI, Aaron Jaech, Adam Kalai, et al.*2024 · 1,065 citations*](https://md.research.elamur.ai/free/en/papers/openai-2024-openai-o1-system-card/)
23. **[Predicting Pragmatic Reasoning in Language Games](https://md.research.elamur.ai/free/en/papers/frank-2012-predicting-pragmatic-reasoning-language-games/) [Science](https://md.research.elamur.ai/free/en/venues/science/)**Michael C. Frank, Noah D. Goodman*2012 · 1,007 citations*
24. **[Solving Quantitative Reasoning Problems with Language Models](https://md.research.elamur.ai/free/en/papers/lewkowycz-2022-solving-quantitative-reasoning-problems-language/) [Conference on Neural Information Processing Systems (NeurIPS)](https://md.research.elamur.ai/free/en/venues/neural-information-processing-systems/)**Aitor Lewkowycz, Anders Andreassen, David Dohan, et al.*2022 · 975 citations*
25. **[MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGI](https://md.research.elamur.ai/free/en/papers/yue-2023-mmmu-massive-multi-discipline-multimodal/) [IEEE Computer Society Conference on Computer Vision and Pattern Recognition](https://md.research.elamur.ai/free/en/venues/ieee-computer-society-conference-on-computer-vision-and-pattern-recognition/)**Xiang Yue, Yuansheng Ni, Kai Zhang, et al.*2023 · 948 citations*

**1** [2](https://md.research.elamur.ai/free/en/topics/reasoning/page/2/) … [6](https://md.research.elamur.ai/free/en/topics/reasoning/page/6/) [next →](https://md.research.elamur.ai/free/en/topics/reasoning/page/2/)

Updated 24 October 2026

- [← previous topic Question answering](https://md.research.elamur.ai/free/en/topics/question-answering/)
- [next topic → Semantic parsing & meaning representation](https://md.research.elamur.ai/free/en/topics/semantic-parsing/)
- [↑ back to topics](https://md.research.elamur.ai/free/en/topics/)