---
title: "Few-shot learning — methods, history and key papers"
description: "Few-shot learning (FSL) is the study and practice of building machine-learning systems that can perform a new task, such as recognising a…"
canonical: "https://research.elamur.ai/free/en/concepts/few-shot-learning/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/concepts/few-shot-learning/)

[home](https://md.research.elamur.ai/free/en/) / [concepts](https://md.research.elamur.ai/free/en/concepts/) / Few-shot learning

# 🪄 Few-shot learning

Field · 34 key papers · [all concepts](https://md.research.elamur.ai/free/en/concepts/)

Part of: [🗨️ Natural language processing](https://md.research.elamur.ai/free/en/concepts/natural-language-processing/) · [🗃️ Text classification](https://md.research.elamur.ai/free/en/concepts/text-classification/) · [📊 Machine learning](https://md.research.elamur.ai/free/en/concepts/machine-learning/)

Few-shot learning (FSL) is the study and practice of building machine-learning systems that can perform a new task, such as recognising a class or following an instruction, after seeing only a handful of labelled examples. The handful usually runs from a single example to a few dozen, and in prompted language models it can reach about a hundred.

Also: FSL, low-shot learning, N-way K-shot learning, one-shot learning (when K = 1), in-context few-shot learning, few-shot prompting

About 18 minutes to read · every paper named here has its own page in this library

**On this page**

1. [What it is](#a-what-it-is)
2. [How it works](#a-how-it-works)
3. [History](#a-history)
4. [Methods](#a-methods)
5. [Datasets and benchmarks](#a-datasets-and-benchmarks)
6. [People and institutions](#a-people-and-institutions)
7. [Open questions](#a-open-questions)
8. [The library in figures](#a-in-figures)

## What it is

A child who has seen one zebra can usually recognise the next one. A conventional neural network cannot. It has to be trained on thousands or millions of labelled pictures before it tells zebras from horses reliably.

FSL asks how to close that gap: how a system can pick up a new category, word, language pair or task from a few examples instead of a large annotated dataset. The question matters whenever labels are expensive. That includes:

- rare diseases;
- clinical notes that only experts can annotate (Agrawal et al. 2022);
- languages with little digital text;
- tasks that change too quickly for anyone to collect data.

The field has two main traditions, and they meet in modern deep learning.

- The first is meta-learning, "learning to learn". A model is trained on many small tasks so that it becomes good at solving one more small task quickly. This tradition began mostly in computer vision.
- The second grew out of [large language models](https://md.research.elamur.ai/free/en/concepts/large-language-models/). Here a single pretrained network is given a few worked examples inside its input text and continues the pattern without any change to its weights. This was named in-context learning when GPT-3 was introduced ([Brown et al. 2020](https://md.research.elamur.ai/free/en/papers/brown-2020-language-models-few-shot-learners/)).

Between the two sits a family of methods that fine-tune a pretrained [language model](https://md.research.elamur.ai/free/en/concepts/language-model/) on a few examples. They first rephrase the task so that it resembles the model's pretraining objective ([Schick & Schütze 2021](https://md.research.elamur.ai/free/en/papers/schick-2021-exploiting-cloze-questions-few-shot/)).

The standard way to state a few-shot problem is the N-way K-shot setting. A task has N classes, and the learner receives K labelled examples of each. These N × K examples form the support set. The learner is then scored on fresh query examples from the same N classes.

Take a 5-way 1-shot task built from photographs. The learner sees one photo each of five bird species it has never encountered and must label new photos of those five species. In a 5-way 5-shot task it would see five photos per species.

Two limiting cases sit at the low end of the setting:

- One-shot learning is the case K = 1.
- Zero-shot learning removes the examples entirely. Only a description of the task or the labels is given. This can be a natural-language instruction ([Brown et al. 2020](https://md.research.elamur.ai/free/en/papers/brown-2020-language-models-few-shot-learners/)) or a label rephrased as an entailment hypothesis ([Yin et al. 2019](https://md.research.elamur.ai/free/en/papers/yin-2019-benchmarking-zero-shot-text-classification/)).

The shelves its papers stand on Artificial Intelligence: 369 of the 373 papers here.

**The numbers**

| Shelf | Papers |
| --- | --- |
| [Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) | 369 papers |
| [Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) | 358 papers |
| [Computer Vision and Pattern Recognition](https://md.research.elamur.ai/free/en/shelves/computer-science/computer-vision-and-pattern-recognition/) | 87 papers |
| [Information Systems](https://md.research.elamur.ai/free/en/shelves/computer-science/information-systems/) | 24 papers |
| [Safety Research](https://md.research.elamur.ai/free/en/shelves/social-sciences/safety-research/) | 9 papers |
| [Experimental and Cognitive Psychology](https://md.research.elamur.ai/free/en/shelves/psychology/experimental-and-cognitive-psychology/) | 6 papers |
| [General Social Sciences](https://md.research.elamur.ai/free/en/shelves/social-sciences/general-social-sciences/) | 5 papers |
| [Signal Processing](https://md.research.elamur.ai/free/en/shelves/computer-science/signal-processing/) | 5 papers |
| [Computational Theory and Mathematics](https://md.research.elamur.ai/free/en/shelves/computer-science/computational-theory-and-mathematics/) | 4 papers |
| [Computer Networks and Communications](https://md.research.elamur.ai/free/en/shelves/computer-science/computer-networks-and-communications/) | 4 papers |
| [Health Informatics](https://md.research.elamur.ai/free/en/shelves/medicine/health-informatics/) | 3 papers |
| [Molecular Biology](https://md.research.elamur.ai/free/en/shelves/biochemistry-genetics-and-molecular-biology/molecular-biology/) | 3 papers |

## How it works

### Episodes and transfer

Every FSL method does most of its learning before it sees the few examples.

- Meta-learning methods train on episodes: many simulated N-way K-shot tasks drawn from a large pool of base classes. The skill being trained is the ability to solve a fresh episode (Vinyals et al. 2016).
- Pretrained language models acquire broad competence from raw text. The few examples then serve mainly to identify the task, and some researchers argue they do little actual teaching ([Reynolds & McDonell 2021](https://md.research.elamur.ai/free/en/papers/reynolds-2021-prompt-programming-large-language-models/)). Evidence for this view is discussed under in-context learning in the Methods section.

### Learning a metric: prototypes

Metric-based methods learn an embedding space in which a simple comparison suffices. Prototypical networks are the clearest example (Snell et al. 2017). Each class is summarised by a single point, its prototype, computed as the mean of its embedded support examples:

```
c_k = (1 / |S_k|) · Σ_{(x_i, y_i) ∈ S_k} f_φ(x_i)

p(y = k | x) = exp(−d(f_φ(x), c_k)) / Σ_{k'} exp(−d(f_φ(x), c_{k'}))
```

The symbols are as follows:

- S_k is the set of support examples labelled with class k, and |S_k| is their number (K in a balanced task).
- x_i is an input and y_i its label.
- f_φ is the embedding network with parameters φ.
- c_k is the resulting prototype of class k.
- d is a distance, commonly squared Euclidean distance.

The second line turns negative distances into a probability over classes with a softmax. A query x is therefore assigned to the class whose prototype lies nearest. Training adjusts φ across many episodes so that this nearest-prototype rule works on classes the network has never seen. Matching networks, which preceded this approach, compare a query with every support example through an attention mechanism rather than through class means (Vinyals et al. 2016).

### Learning to adapt: MAML

Optimisation-based methods learn a starting point from which ordinary gradient descent adapts quickly. Model-Agnostic Meta-Learning (MAML) does this for any model trained by gradients (Finn et al. 2017):

```
θ'_i = θ − α · ∇_θ L_{T_i}(θ)
θ ← θ − β · ∇_θ Σ_i L_{T_i}(θ'_i)
```

The symbols are as follows:

- θ are the shared initial parameters.
- T_i is one sampled training task.
- L_{T_i} is that task's loss, computed on its support data in the first line and on its query data in the second.
- α is the inner-loop step size and β the outer-loop (meta) step size.

The first line simulates fine-tuning on a single task. The second updates the initialisation so that this short fine-tuning yields low loss on held-out examples of the same task. Because the second gradient passes through the first, exact MAML involves second-order derivatives. These can be dropped as an approximation, as was done when MAML was carried over to translation ([Gu et al. 2018](https://md.research.elamur.ai/free/en/papers/gu-2018-meta-learning-low-resource-neural/)).

### Learning from the prompt

In-context learning needs no gradients at all. A prompt for GPT-3 contains a task description, a few input–output demonstrations and a new input. The model's continuation is read off as the answer ([Brown et al. 2020](https://md.research.elamur.ai/free/en/papers/brown-2020-language-models-few-shot-learners/)).

The GPT-3 paper separated three settings by the number of demonstrations:

- zero-shot, with an instruction only;
- one-shot;
- few-shot, in which the model received as many examples as its context window allowed, typically between 10 and 100 ([Brown et al. 2020](https://md.research.elamur.ai/free/en/papers/brown-2020-language-models-few-shot-learners/)).

A survey of the area formalises the process as scoring candidate answers conditioned on the demonstrations and an optional instruction. It also separates this from classical few-shot learning, in which parameters are updated ([Dong et al. 2023](https://md.research.elamur.ai/free/en/papers/dong-2023-survey-context-learning/)).

The same idea carries beyond text. Flamingo interleaves images and text in its prompt and adapts to new visual tasks from a few such demonstrations ([Alayrac et al. 2022](https://md.research.elamur.ai/free/en/papers/alayrac-2022-flamingo-visual-language-model-few/)).

### Pattern-based fine-tuning

A middle route rewrites each labelled example as a fill-in-the-blank sentence that a masked language model can complete. In Pattern-Exploiting Training (PET), two components do the rewriting ([Schick & Schütze 2021](https://md.research.elamur.ai/free/en/papers/schick-2021-exploiting-cloze-questions-few-shot/)):

- A pattern turns an input into a cloze question with one mask.
- A verbalizer maps each label to a vocabulary word.

For a review, the pattern might append "It was ___." The verbalizer might map positive to "great" and negative to "terrible". The model is then fine-tuned to predict the right word, so even a few examples steer an objective it already knows well. This is the core of prompt engineering applied to fine-tuning rather than to frozen models.

## History

The ambition is older than deep learning:

- Meta-learning was formulated in work by Schmidhuber (1987) and in the "learning to learn" programme of Thrun and Pratt (1998).
- Fei-Fei Li and colleagues (2006) gave one-shot object recognition its modern framing.
- Lake and colleagues built the Omniglot handwritten-character collection, first described in 2011. In 2015 they used it to compare human and machine concept learning.
- Koch and colleagues (2015) used siamese networks for one-shot image matching.

Matching networks were among the first deep meta-learning methods trained on simulated episodes (Vinyals et al. 2016). Prototypical networks (Snell et al. 2017) and MAML (Finn et al. 2017) followed in 2017. They became the reference points for the metric-based and optimisation-based families.

[Natural language processing](https://md.research.elamur.ai/free/en/concepts/natural-language-processing/) (NLP) took up the problem through transfer between tasks and languages:

- A multilingual translation system marked each input with a target-language token. It could then translate between language pairs it had never been trained on, a form of zero-shot transfer (Johnson et al. 2017).
- MetaNMT treated each language pair as one task in a MAML setup. It reached 22.04 [BLEU](https://md.research.elamur.ai/free/en/concepts/bleu/), an n-gram overlap score for translation, on Romanian–English after fine-tuning on roughly 600 sentence pairs ([Gu et al. 2018](https://md.research.elamur.ai/free/en/papers/gu-2018-meta-learning-low-resource-neural/)).
- A shared sentence encoder covering 93 languages let a classifier trained on English work in other languages without further training ([Artetxe & Schwenk 2018](https://md.research.elamur.ai/free/en/papers/artetxe-2018-massively-multilingual-sentence-embeddings-zero/)).
- Zero-shot text classification was recast as textual entailment, with each label rewritten as a hypothesis ([Yin et al. 2019](https://md.research.elamur.ai/free/en/papers/yin-2019-benchmarking-zero-shot-text-classification/)).
- Relation extraction was recast as reading comprehension (Levy et al. 2017).

The turning point came in 2020. GPT-3, a 175-billion-parameter autoregressive [Transformer](https://md.research.elamur.ai/free/en/concepts/transformer/), was evaluated purely through prompts. On some benchmarks its few-shot results approached the best fine-tuned systems. Across an aggregate of 42 accuracy-based benchmarks, its few-shot performance grew with model size faster than its zero-shot performance ([Brown et al. 2020](https://md.research.elamur.ai/free/en/papers/brown-2020-language-models-few-shot-learners/)). That result shifted attention from training specialised models to prompting general ones, and it ties few-shot ability to [scaling laws](https://md.research.elamur.ai/free/en/concepts/scaling-laws/).

Several lines of work followed in quick succession:

- 2021, smaller models. Groups showed that models far smaller than GPT-3 could compete when fine-tuned on cloze-style reformulations ([Schick & Schütze 2021](https://md.research.elamur.ai/free/en/papers/schick-2021-it-s-not-just-size/); [Gao et al. 2021](https://md.research.elamur.ai/free/en/papers/gao-2021-making-pre-trained-language-models/)).
- 2021, fragility. Other groups documented how unstable prompting was and how easily its evaluation could be inflated ([Zhao et al. 2021](https://md.research.elamur.ai/free/en/papers/zhao-2021-calibrate-before-use-improving-few/); [Perez et al. 2021](https://md.research.elamur.ai/free/en/papers/perez-2021-true-few-shot-learning-language/)).
- 2021, instruction tuning. Multitask training on tasks phrased as instructions improved zero-shot transfer (Wei et al. 2021; [Sanh et al. 2021](https://md.research.elamur.ai/free/en/papers/sanh-2021-multitask-prompted-training-enables-zero/)).
- 2022, reasoning. Chain-of-thought prompting ([Wei et al. 2022](https://md.research.elamur.ai/free/en/papers/wei-2022-chain-thought-prompting-elicits-reasoning/)) and its zero-shot variant ([Kojima et al. 2022](https://md.research.elamur.ai/free/en/papers/kojima-2022-large-language-models-zero-shot/)) made multi-step reasoning a central few-shot application.
- 2023, consolidation and critique. Surveys organised the area ([Dong et al. 2023](https://md.research.elamur.ai/free/en/papers/dong-2023-survey-context-learning/)), and studies of faithfulness questioned what the written reasoning actually reflects ([Turpin et al. 2023](https://md.research.elamur.ai/free/en/papers/turpin-2023-language-models-don-t-always/)).

Papers on this subject by year 373 dated papers from 2015 to 2026; the busiest year is 2023 with 125, and 99% of them appeared in 2018 or later.

**The numbers**

| Year | Papers |
| --- | --- |
| 2026 | 46 |
| 2025 | 18 |
| 2024 | 45 |
| 2023 | 125 |
| 2022 | 62 |
| 2021 | 36 |
| 2020 | 19 |
| 2019 | 12 |
| 2018 | 7 |
| 2017 | 1 |
| 2016 | 1 |
| 2015 | 1 |

Citations of the papers on this subject, by when they were published Papers published in 2020–2024 gather 94% of the 130,066 citations of the papers on this subject.

**The numbers**

| Published | Papers and their citations |
| --- | --- |
| 2025–2029 | 64 papers, 1,449 citations |
| 2020–2024 | 287 papers, 122,212 citations |
| 2015–2019 | 22 papers, 6,405 citations |

Landmark papers on this subject: the 12 most cited, in time The 12 most cited papers on this subject span 2020–2023; the dot grows with the citations.

**The numbers**

| Paper | Citations |
| --- | --- |
| [2020 — Language Models are Few-Shot Learners](https://md.research.elamur.ai/free/en/papers/brown-2020-language-models-few-shot-learners/) | 13,703 |
| [2021 — Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processing](https://md.research.elamur.ai/free/en/papers/liu-2021-pre-train-prompt-predict-systematic/) | 5,396 |
| [2021 — Making Pre-trained Language Models Better Few-shot Learners](https://md.research.elamur.ai/free/en/papers/gao-2021-making-pre-trained-language-models/) | 1,963 |
| [2021 — Exploiting Cloze-Questions for Few-Shot Text Classification and Natural Language Inference](https://md.research.elamur.ai/free/en/papers/schick-2021-exploiting-cloze-questions-few-shot/) | 1,803 |
| [2021 — Multitask Prompted Training Enables Zero-Shot Task Generalization](https://md.research.elamur.ai/free/en/papers/sanh-2021-multitask-prompted-training-enables-zero/) | 1,581 |
| [2022 — Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://md.research.elamur.ai/free/en/papers/wei-2022-chain-thought-prompting-elicits-reasoning/) | 11,308 |
| [2022 — PaLM: Scaling Language Modeling with Pathways](https://md.research.elamur.ai/free/en/papers/chowdhery-2022-palm-scaling-language-modeling-pathways/) | 4,680 |
| [2022 — Large language models are zero-shot reasoners](https://md.research.elamur.ai/free/en/papers/kojima-2022-large-language-models-zero-shot/) | 3,795 |
| [2022 — Self-Consistency Improves Chain of Thought Reasoning in Language Models](https://md.research.elamur.ai/free/en/papers/wang-2022-self-consistency-improves-chain-thought/) | 3,041 |
| [2022 — Flamingo: a Visual Language Model for Few-Shot Learning](https://md.research.elamur.ai/free/en/papers/alayrac-2022-flamingo-visual-language-model-few/) | 2,118 |
| [2023 — A Survey of Large Language Models](https://md.research.elamur.ai/free/en/papers/zhao-2023-survey-large-language-models/) | 2,863 |
| [2023 — Sparks of Artificial General Intelligence: Early experiments with GPT-4](https://md.research.elamur.ai/free/en/papers/bubeck-2023-sparks-artificial-general-intelligence-early/) | 2,400 |

## Methods

### Metric-based meta-learning

Metric methods learn embeddings so that classification reduces to a nearest-neighbour or nearest-prototype decision (Snell et al. 2017; Vinyals et al. 2016). They have two practical strengths:

- They are cheap at test time, since a new class costs one forward pass per support example.
- They need no fine-tuning.

Their main limitation is that the embedding must already separate the new classes. When the new domain differs strongly from the training classes, a fixed metric has little to work with.

The same idea reappears in language work whenever examples are compared in an embedding space. Two examples are:

- zero-shot entity linking, which embeds mentions and entity descriptions in one space ([Wu et al. 2020](https://md.research.elamur.ai/free/en/papers/wu-2020-scalable-zero-shot-entity-linking/));
- the selection of prompt demonstrations by [semantic similarity](https://md.research.elamur.ai/free/en/concepts/semantic-similarity/) ([Gao et al. 2021](https://md.research.elamur.ai/free/en/papers/gao-2021-making-pre-trained-language-models/)).

### Optimisation-based meta-learning

MAML and its relatives learn an initialisation rather than a metric. They adapt every parameter with a few gradient steps. In principle they can therefore handle new tasks that need more than a re-weighting of familiar features (Finn et al. 2017).

The cost is a nested training loop that is expensive and sensitive to hyperparameters. In MetaNMT, the choice of validation task used for early stopping noticeably changed final quality ([Gu et al. 2018](https://md.research.elamur.ai/free/en/papers/gu-2018-meta-learning-low-resource-neural/)).

### In-context learning and the craft of demonstrations

Once a model is large enough, the few-shot learner is the prompt. Much research therefore concerns which demonstrations to show and in what form.

- Retrieval of demonstrations. EPR trains a small dense retriever to find training examples that raise the probability of the correct output when placed in the prompt. It was tested with GPT-Neo, an openly released GPT-style model, as both scorer and inference model. On MTOP, a task-oriented semantic-parsing benchmark, it reached 64.2% exact match, against 57% for the strongest baseline ([Rubin et al. 2021](https://md.research.elamur.ai/free/en/papers/rubin-2021-learning-retrieve-prompts-context-learning/)).
- Ordering. The order of the same examples can move accuracy from near state-of-the-art to chance. One fix ranks candidate orders by the entropy of the labels the model predicts on a self-generated probing set. Over eleven text classification tasks, this gave an average relative improvement in classification performance of about 13% ([Lu et al. 2022](https://md.research.elamur.ai/free/en/papers/lu-2022-fantastically-ordered-prompts-where-find/)).
- Calibration. Language models favour labels that appear often, appear last, or are common tokens. Contextual calibration estimates this bias from a content-free input such as "N/A" and corrects it. It raised average accuracy by up to 30.0 points absolute ([Zhao et al. 2021](https://md.research.elamur.ai/free/en/papers/zhao-2021-calibrate-before-use-improving-few/)).
- What the demonstrations carry. Replacing gold labels in demonstrations with random ones cost only about 0–5% absolute across 26 datasets. What mattered was exposure to the label space, the input distribution and the format. This suggests demonstrations mostly activate abilities acquired in pretraining ([Min et al. 2022](https://md.research.elamur.ai/free/en/papers/min-2022-rethinking-role-demonstrations-what-makes/)).
- Zero-shot as a competitor. Carefully worded zero-shot prompts can match or beat few-shot prompts. In translation experiments, adding a single example sometimes lowered BLEU ([Reynolds & McDonell 2021](https://md.research.elamur.ai/free/en/papers/reynolds-2021-prompt-programming-large-language-models/)).

These methods connect few-shot learning to [retrieval-augmented generation](https://md.research.elamur.ai/free/en/concepts/retrieval-augmented-generation/):

- Atlas pairs a language model with a document retriever to learn knowledge-intensive tasks from few examples (Izacard et al. 2022).
- HyDE uses a prompted model to write a hypothetical passage and retrieves real documents near it, without any relevance labels ([Gao et al. 2022](https://md.research.elamur.ai/free/en/papers/gao-2022-precise-zero-shot-dense-retrieval/)).

### Chain of thought and decomposition

Chain-of-thought prompting extends each demonstration from an input–output pair to an input, a written line of reasoning, and an output ([Wei et al. 2022](https://md.research.elamur.ai/free/en/papers/wei-2022-chain-thought-prompting-elicits-reasoning/)). With PaLM 540B and eight such exemplars, accuracy on the GSM8K math word problems rose to about 57%, against about 18% with standard prompting. The benefit appeared only in models of roughly 100 billion parameters and above ([Wei et al. 2022](https://md.research.elamur.ai/free/en/papers/wei-2022-chain-thought-prompting-elicits-reasoning/)).

Several variants change how the reasoning is elicited or which examples are shown:

- Zero-shot chain of thought drops the examples and appends a single trigger phrase asking the model to think step by step. It was tested with InstructGPT, a GPT-3 variant tuned to follow instructions. This raised MultiArith accuracy from 17.7% to 78.7% and GSM8K accuracy from 10.4% to 40.7% ([Kojima et al. 2022](https://md.research.elamur.ai/free/en/papers/kojima-2022-large-language-models-zero-shot/)).
- Least-to-most prompting first decomposes a problem into simpler subquestions and then answers them in sequence. It reached at least 99% on every split of the SCAN compositional benchmark with 14 exemplars ([al. 2022](https://md.research.elamur.ai/free/en/papers/al-2022-least-most-prompting-enables-complex/)).
- Auto-CoT clusters questions and has the model write rationales for one representative per cluster. It matched or beat hand-written chains on ten reasoning benchmarks ([Zhang et al. 2022](https://md.research.elamur.ai/free/en/papers/zhang-2022-automatic-chain-thought-prompting-large/)).
- Complexity-based prompting prefers exemplars with more reasoning steps ([Fu et al. 2022](https://md.research.elamur.ai/free/en/papers/fu-2022-complexity-based-prompting-multi-step/)).

The same few-shot machinery drives planners for embodied agents ([Song et al. 2022](https://md.research.elamur.ai/free/en/papers/song-2022-llm-planner-few-shot-grounded/); [Huang et al. 2022](https://md.research.elamur.ai/free/en/papers/huang-2022-language-models-zero-shot-planners/)) and text-to-SQL pipelines ([Pourreza & Rafiei 2023](https://md.research.elamur.ai/free/en/papers/pourreza-2023-din-sql-decomposed-context-learning/)). These link the topic to [reasoning](https://md.research.elamur.ai/free/en/topics/reasoning/) and [semantic parsing](https://md.research.elamur.ai/free/en/concepts/semantic-parsing/).

### Pattern-based and parameter-efficient fine-tuning

PET fine-tunes one masked language model per pattern–verbalizer pair on the few labelled examples. The resulting ensemble labels a pool of unlabelled text, and a final classifier is distilled from those soft labels. The iterative variant, iPET, repeats this over generations ([Schick & Schütze 2021](https://md.research.elamur.ai/free/en/papers/schick-2021-exploiting-cloze-questions-few-shot/)).

A follow-up allowed labels that span several tokens. It was tested with ALBERT and 32 labelled examples per task on SuperGLUE, a suite of difficult English language-understanding tasks. PET and iPET outperformed GPT-3 on average while using about 0.1% of its parameters ([Schick & Schütze 2021](https://md.research.elamur.ai/free/en/papers/schick-2021-it-s-not-just-size/)).

LM-BFF automates the recipe ([Gao et al. 2021](https://md.research.elamur.ai/free/en/papers/gao-2021-making-pre-trained-language-models/)):

- It generates templates with T5.
- It searches for label words.
- It appends one similar demonstration per class to each input.

With 16 examples per class, this improved over standard fine-tuning by 11 points on average across 15 English tasks ([Gao et al. 2021](https://md.research.elamur.ai/free/en/papers/gao-2021-making-pre-trained-language-models/)).

Prompt tuning learns only continuous "soft prompt" vectors. It lags behind full fine-tuning when data are scarce, unless the prompts are first pre-trained on unlabelled text as in PPT ([Gu et al. 2022](https://md.research.elamur.ai/free/en/papers/gu-2022-ppt-pre-trained-prompt-tuning/)).

T-Few combines a multitask-prompted backbone with a light rescaling method, (IA)3. It scored 75.8% on the RAFT benchmark, above the 73.5% human baseline. It also used over 1,000 times fewer inference FLOPs (floating-point operations) than in-context learning with GPT-3 ([Liu et al. 2022](https://md.research.elamur.ai/free/en/papers/liu-2022-few-shot-parameter-efficient-fine/)). The trade-off against prompting is therefore partly a matter of [efficient models](https://md.research.elamur.ai/free/en/topics/efficient-models/).

### Instruction tuning

A related idea trains a model on many tasks phrased as natural-language instructions, so that unseen tasks become easier zero-shot. This was shown with FLAN (Wei et al. 2021).

T0 was trained on prompted English datasets with whole task groups held out. Although it is about 16 times smaller than GPT-3, it matched or surpassed GPT-3 on 9 of 11 held-out datasets ([Sanh et al. 2021](https://md.research.elamur.ai/free/en/papers/sanh-2021-multitask-prompted-training-enables-zero/)).

| Family | What is learned | What happens at test time | Representative work |
| --- | --- | --- | --- |
| Metric-based meta-learning | An embedding space | Compare queries with support examples or prototypes | (Snell et al. 2017; Vinyals et al. 2016) |
| Optimisation-based meta-learning | An initialisation | A few gradient steps on the support set | (Finn et al. 2017; [Gu et al. 2018](https://md.research.elamur.ai/free/en/papers/gu-2018-meta-learning-low-resource-neural/)) |
| In-context learning | Nothing task-specific (pretrained weights only) | Demonstrations placed in the prompt, no updates | ([Brown et al. 2020](https://md.research.elamur.ai/free/en/papers/brown-2020-language-models-few-shot-learners/); [Wei et al. 2022](https://md.research.elamur.ai/free/en/papers/wei-2022-chain-thought-prompting-elicits-reasoning/)) |
| Pattern-based fine-tuning | Cloze reformulation of the task | Gradient updates on few examples, then prediction | ([Schick & Schütze 2021](https://md.research.elamur.ai/free/en/papers/schick-2021-exploiting-cloze-questions-few-shot/); [Gao et al. 2021](https://md.research.elamur.ai/free/en/papers/gao-2021-making-pre-trained-language-models/)) |
| Parameter-efficient tuning | A small set of added parameters | Brief training of the added parameters only | ([Gu et al. 2022](https://md.research.elamur.ai/free/en/papers/gu-2022-ppt-pre-trained-prompt-tuning/); [Liu et al. 2022](https://md.research.elamur.ai/free/en/papers/liu-2022-few-shot-parameter-efficient-fine/)) |

## Datasets and benchmarks

Vision-based meta-learning grew up on two collections:

- Omniglot, first described by Lake and colleagues in 2011.
- miniImageNet, a reduced ImageNet split organised into few-shot episodes, introduced with matching networks (Vinyals et al. 2016).

Later suites such as Meta-Dataset (Triantafillou and colleagues, 2020) mixed several image domains to test transfer across them.

In language, few-shot evaluation mostly reuses existing [evaluation benchmarks](https://md.research.elamur.ai/free/en/topics/evaluation-benchmarks/) with the training data cut down. A few resources were designed specifically for the setting.

| Benchmark | What it tests | Few-shot design |
| --- | --- | --- |
| FewGLUE | SuperGLUE tasks | 32 labelled examples per task ([Schick & Schütze 2021](https://md.research.elamur.ai/free/en/papers/schick-2021-it-s-not-just-size/)) |
| LM-BFF suite | 15 English classification and sentence-pair tasks | 16 examples per class, averaged over 5 random splits ([Gao et al. 2021](https://md.research.elamur.ai/free/en/papers/gao-2021-making-pre-trained-language-models/)) |
| RAFT | 11 real-world classification tasks | 50 training examples each, hidden test labels ([Liu et al. 2022](https://md.research.elamur.ai/free/en/papers/liu-2022-few-shot-parameter-efficient-fine/)) |
| Zero-shot text classification | Topic, emotion and situation labels | Fixed seen and unseen label splits ([Yin et al. 2019](https://md.research.elamur.ai/free/en/papers/yin-2019-benchmarking-zero-shot-text-classification/)) |
| XStoryCloze | Commonsense story completion in English plus translations into 10 languages (11 in all) | Zero-shot and few-shot prompting ([Lin et al. 2021](https://md.research.elamur.ai/free/en/papers/lin-2021-few-shot-learning-multilingual-language/)) |
| FLEURS | Speech recognition, language ID and retrieval in 102 languages | About 12 hours of supervision per language ([Conneau et al. 2022](https://md.research.elamur.ai/free/en/papers/conneau-2022-fleurs-few-shot-learning-evaluation/)) |

FLEURS uses "few-shot" loosely. About 12 hours of transcribed speech per language is scarce supervision for low-resource and unseen languages. It is not, however, an N-way K-shot task in the sense defined above ([Conneau et al. 2022](https://md.research.elamur.ai/free/en/papers/conneau-2022-fleurs-few-shot-learning-evaluation/)).

Reasoning studies lean on math word problems such as GSM8K and MultiArith and on BIG-Bench Hard tasks ([Wei et al. 2022](https://md.research.elamur.ai/free/en/papers/wei-2022-chain-thought-prompting-elicits-reasoning/); [Kojima et al. 2022](https://md.research.elamur.ai/free/en/papers/kojima-2022-large-language-models-zero-shot/); [Turpin et al. 2023](https://md.research.elamur.ai/free/en/papers/turpin-2023-language-models-don-t-always/)). Flamingo was assessed on 16 image and video benchmarks. It outperformed fine-tuned state-of-the-art systems on 6 of them while using only 32 task examples ([Alayrac et al. 2022](https://md.research.elamur.ai/free/en/papers/alayrac-2022-flamingo-visual-language-model-few/)).

For [multilingual NLP](https://md.research.elamur.ai/free/en/topics/multilingual-nlp/) and [low-resource languages](https://md.research.elamur.ai/free/en/topics/low-resource-languages/), cross-lingual transfer studies show two things. Zero-shot transfer from English degrades for distant languages, and a few target-language examples help a great deal.

One study used Universal Dependencies treebanks for [dependency parsing](https://md.research.elamur.ai/free/en/concepts/dependency-parsing/), with target languages drawn from eight families and models first fine-tuned on English. Continuing training on just 10 target-language sentences raised the unlabelled attachment score (UAS) by as much as 40 points for some targets. UAS is the share of words given their correct syntactic head ([Lauscher et al. 2020](https://md.research.elamur.ai/free/en/papers/lauscher-2020-zero-hero-limitations-zero-shot/)).

Applied studies extend the setting to clinical [information extraction](https://md.research.elamur.ai/free/en/concepts/information-extraction/), including [coreference resolution](https://md.research.elamur.ai/free/en/concepts/coreference-resolution/) and acronym sense disambiguation (Agrawal et al. 2022).

### How results are evaluated, and where they mislead

Few-shot numbers are unusually fragile, and several studies show how they can overstate what a method does.

- Hidden validation data. Many reported results chose prompts and hyperparameters on large held-out sets, which contradicts the premise of having few examples. When selection had to rely on the few training examples alone, the criteria barely beat random choice. These criteria were cross-validation and [minimum description length](https://md.research.elamur.ai/free/en/concepts/minimum-description-length/). For ADAPET, a PET-derived few-shot method, hyperparameters chosen this way scored 2–10% lower across 8 tasks than those chosen with held-out data ([Perez et al. 2021](https://md.research.elamur.ai/free/en/papers/perez-2021-true-few-shot-learning-language/)). The authors recommend counting any validation data within the few-shot budget and reporting every candidate prompt ([Perez et al. 2021](https://md.research.elamur.ai/free/en/papers/perez-2021-true-few-shot-learning-language/)).
- Variance from example choice and order. On 4-shot SST-2, a binary movie-review sentiment task, reordering the same examples moved GPT-3 2.7B accuracy from 54.3% to 93.4% ([Zhao et al. 2021](https://md.research.elamur.ai/free/en/papers/zhao-2021-calibrate-before-use-improving-few/)). Good orders also failed to carry over between model sizes ([Lu et al. 2022](https://md.research.elamur.ai/free/en/papers/lu-2022-fantastically-ordered-prompts-where-find/)). Different random sets of 32 training examples shifted F1, the harmonic mean of precision and recall, between 57.4 and 84.0. This was on CB, the CommitmentBank inference task ([Schick & Schütze 2021](https://md.research.elamur.ai/free/en/papers/schick-2021-it-s-not-just-size/)). Comparisons are therefore meaningful only on identical examples and with several seeds or splits ([Gao et al. 2021](https://md.research.elamur.ai/free/en/papers/gao-2021-making-pre-trained-language-models/)).
- Weak baselines. Given the random-label result described under Methods, zero-shot baselines built from unlabelled inputs are stronger than often assumed ([Min et al. 2022](https://md.research.elamur.ai/free/en/papers/min-2022-rethinking-role-demonstrations-what-makes/)). Poorly worded zero-shot prompts can make few-shot gains look larger than they are ([Reynolds & McDonell 2021](https://md.research.elamur.ai/free/en/papers/reynolds-2021-prompt-programming-large-language-models/)).
- Prompts that are not understood. Irrelevant and even misleading templates often let models learn natural language inference about as fast as sensible instructions. The choice of target words mattered more than the instruction itself ([Webson & Pavlick 2022](https://md.research.elamur.ai/free/en/papers/webson-2022-do-prompt-based-models-really/)).
- Contamination. Benchmark items may already sit in the pretraining corpus, a risk the GPT-3 authors discussed openly ([Brown et al. 2020](https://md.research.elamur.ai/free/en/papers/brown-2020-language-models-few-shot-learners/)). Flamingo's authors kept a held-out set of benchmarks out of all design decisions to obtain unbiased estimates ([Alayrac et al. 2022](https://md.research.elamur.ai/free/en/papers/alayrac-2022-flamingo-visual-language-model-few/)).
- Plausible but unfaithful reasoning. Biasing cues shifted model answers and cut BIG-Bench Hard accuracy by up to 36%. One such cue was always placing the correct option at "(A)". In a manual review, only 1 of 426 explanations supporting biased answers acknowledged the cue ([Turpin et al. 2023](https://md.research.elamur.ai/free/en/papers/turpin-2023-language-models-don-t-always/)).

Where the papers on this subject were published Annual Meeting of the Association for Computational Linguistics (ACL) carries the most, 59 of 373.

**The numbers**

| Venue | Papers |
| --- | --- |
| [Annual Meeting of the Association for Computational Linguistics (ACL)](https://md.research.elamur.ai/free/en/venues/acl/) | 59 papers |
| [Findings of the Association for Computational Linguistics](https://md.research.elamur.ai/free/en/venues/findings/) | 53 papers |
| [Conference on Empirical Methods in Natural Language Processing (EMNLP)](https://md.research.elamur.ai/free/en/venues/emnlp/) | 44 papers |
| [Language Resources and Evaluation Conference (LREC)](https://md.research.elamur.ai/free/en/venues/lrec/) | 16 papers |
| [Conference of the North American Chapter of the Association for Computational Linguistics (NAACL)](https://md.research.elamur.ai/free/en/venues/naacl/) | 14 papers |
| [Conference on Neural Information Processing Systems (NeurIPS)](https://md.research.elamur.ai/free/en/venues/neural-information-processing-systems/) | 14 papers |
| [Conference of the European Chapter of the Association for Computational Linguistics (EACL)](https://md.research.elamur.ai/free/en/venues/eacl/) | 10 papers |
| [Journal of Chinese Information Processing (中文信息学报)](https://md.research.elamur.ai/free/en/venues/journal-of-chinese-information-processing/) | 6 papers |
| [Transactions of the Association for Computational Linguistics (TACL)](https://md.research.elamur.ai/free/en/venues/tacl/) | 5 papers |
| [Annual Conference of the European Association for Machine Translation (EAMT)](https://md.research.elamur.ai/free/en/venues/eamt/) | 4 papers |

## People and institutions

Several researchers recur across the prompting literature.

[Luke Zettlemoyer](https://md.research.elamur.ai/free/en/authors/luke-zettlemoyer/), of the [University of Washington](https://md.research.elamur.ai/free/en/universities/university-of-washington/) and [Meta](https://md.research.elamur.ai/free/en/companies/meta/), co-authored three relevant studies:

- the analysis showing that correct labels in demonstrations matter less than format ([Min et al. 2022](https://md.research.elamur.ai/free/en/papers/min-2022-rethinking-role-demonstrations-what-makes/));
- work on zero-shot entity linking ([Wu et al. 2020](https://md.research.elamur.ai/free/en/papers/wu-2020-scalable-zero-shot-entity-linking/));
- work on zero-shot relation extraction (Levy et al. 2017).

[Mike Lewis](https://md.research.elamur.ai/free/en/authors/mike-lewis/), of Meta, co-authored the same demonstration study ([Min et al. 2022](https://md.research.elamur.ai/free/en/papers/min-2022-rethinking-role-demonstrations-what-makes/)).

At [Google](https://md.research.elamur.ai/free/en/companies/google/), [Denny Zhou](https://md.research.elamur.ai/free/en/authors/denny-zhou/) led least-to-most prompting ([al. 2022](https://md.research.elamur.ai/free/en/papers/al-2022-least-most-prompting-enables-complex/)). [Jason Wei](https://md.research.elamur.ai/free/en/authors/jason-wei/), who later moved to [OpenAI](https://md.research.elamur.ai/free/en/companies/openai/), was first author of chain-of-thought prompting ([Wei et al. 2022](https://md.research.elamur.ai/free/en/papers/wei-2022-chain-thought-prompting-elicits-reasoning/)) and of FLAN (Wei et al. 2021). [Xuezhi Wang](https://md.research.elamur.ai/free/en/authors/xuezhi-wang/) and [Quoc Le](https://md.research.elamur.ai/free/en/authors/quoc-v-le/) were among the chain-of-thought co-authors.

[Timo Schick](https://md.research.elamur.ai/free/en/authors/timo-schick/) and [Hinrich Schütze](https://md.research.elamur.ai/free/en/authors/hinrich-schutze/) of Ludwig-Maximilians-Universität München developed PET and its multi-token follow-up ([Schick & Schütze 2021](https://md.research.elamur.ai/free/en/papers/schick-2021-exploiting-cloze-questions-few-shot/); [Schick & Schütze 2021](https://md.research.elamur.ai/free/en/papers/schick-2021-it-s-not-just-size/)). Schick was also an author of Atlas (Izacard et al. 2022).

Other contributions include:

- [Albert Webson](https://md.research.elamur.ai/free/en/authors/albert-webson/), who later joined Google, co-authored T0 ([Sanh et al. 2021](https://md.research.elamur.ai/free/en/papers/sanh-2021-multitask-prompted-training-enables-zero/)) and tested whether models understand their prompts ([Webson & Pavlick 2022](https://md.research.elamur.ai/free/en/papers/webson-2022-do-prompt-based-models-really/)). [M Saiful Bari](https://md.research.elamur.ai/free/en/authors/m-saiful-bari/) was also among the T0 authors.
- [Sebastian Riedel](https://md.research.elamur.ai/free/en/authors/sebastian-riedel/), who later moved to [Google DeepMind](https://md.research.elamur.ai/free/en/companies/google-deepmind/) and is also at University College London, co-authored the study of prompt-order sensitivity ([Lu et al. 2022](https://md.research.elamur.ai/free/en/papers/lu-2022-fantastically-ordered-prompts-where-find/)).
- [Jonathan Berant](https://md.research.elamur.ai/free/en/authors/jonathan-berant/), at Google and Tel Aviv University, co-authored learned demonstration retrieval ([Rubin et al. 2021](https://md.research.elamur.ai/free/en/papers/rubin-2021-learning-retrieve-prompts-context-learning/)).
- [Zhuosheng Zhang](https://md.research.elamur.ai/free/en/authors/zhuosheng-zhang/), later at [Shanghai Jiao Tong University](https://md.research.elamur.ai/free/en/universities/shanghai-jiao-tong-university/), was first author of Auto-CoT ([Zhang et al. 2022](https://md.research.elamur.ai/free/en/papers/zhang-2022-automatic-chain-thought-prompting-large/)).

On the meta-learning side:

- Chelsea Finn, with Pieter Abbeel and Sergey Levine, introduced MAML (Finn et al. 2017).
- Jake Snell, Kevin Swersky and Richard Zemel proposed prototypical networks (Snell et al. 2017).
- Oriol Vinyals and colleagues proposed matching networks (Vinyals et al. 2016).
- Tom Brown and colleagues at OpenAI wrote the GPT-3 paper ([Brown et al. 2020](https://md.research.elamur.ai/free/en/papers/brown-2020-language-models-few-shot-learners/)).

Among universities, three appear in this work:

- [Peking University](https://md.research.elamur.ai/free/en/universities/peking-university/) was home to the authors of the in-context learning survey ([Dong et al. 2023](https://md.research.elamur.ai/free/en/papers/dong-2023-survey-context-learning/)).
- [Tsinghua University](https://md.research.elamur.ai/free/en/universities/tsinghua-university/) produced pre-trained prompt tuning ([Gu et al. 2022](https://md.research.elamur.ai/free/en/papers/gu-2022-ppt-pre-trained-prompt-tuning/)).
- Chelsea Finn later continued her work at [Stanford University](https://md.research.elamur.ai/free/en/universities/stanford-university/) ([Mitchell et al. 2023](https://md.research.elamur.ai/free/en/papers/mitchell-2023-detectgpt-zero-shot-machine-generated/)).

Authors with the most papers on this subject Luke Zettlemoyer leads with 9 papers; the 12 authors shown wrote 68 of the papers on this subject between them, counting each paper once per author.

**The numbers**

| Author | Papers here · their citations |
| --- | --- |
| [Luke Zettlemoyer](https://md.research.elamur.ai/free/en/authors/luke-zettlemoyer/) | 9 papers · 3,367 citations |
| [Denny Zhou](https://md.research.elamur.ai/free/en/authors/denny-zhou/) | 8 papers · 17,164 citations |
| [Furu Wei](https://md.research.elamur.ai/free/en/authors/furu-wei/) | 7 papers · 1,881 citations |
| [Jason Wei](https://md.research.elamur.ai/free/en/authors/jason-wei/) | 6 papers · 16,549 citations |
| [Yejin Choi](https://md.research.elamur.ai/free/en/authors/yejin-choi/) | 6 papers · 989 citations |
| [Xuezhi Wang](https://md.research.elamur.ai/free/en/authors/xuezhi-wang/) | 5 papers · 15,564 citations |
| [M Saiful Bari](https://md.research.elamur.ai/free/en/authors/m-saiful-bari/) | 5 papers · 2,744 citations |
| [Jonathan Berant](https://md.research.elamur.ai/free/en/authors/jonathan-berant/) | 5 papers · 1,332 citations |
| [Hao Zhou](https://md.research.elamur.ai/free/en/authors/hao-zhou/) | 5 papers · 457 citations |
| [Quoc Le](https://md.research.elamur.ai/free/en/authors/quoc-v-le/) | 4 papers · 15,835 citations |
| [Hyung Won Chung](https://md.research.elamur.ai/free/en/authors/hyung-won-chung/) | 4 papers · 6,601 citations |
| [Hinrich Schütze](https://md.research.elamur.ai/free/en/authors/hinrich-schutze/) | 4 papers · 3,324 citations |

Where the authors on this subject work now Google is the current home of authors of 43 papers on this subject; counted are the authors with a page here, by their latest institution.

**The numbers**

| Institution | Papers |
| --- | --- |
| [Google](https://md.research.elamur.ai/free/en/companies/google/) | 43 papers |
| [Stanford University](https://md.research.elamur.ai/free/en/universities/stanford-university/) | 37 papers |
| [Carnegie Mellon University](https://md.research.elamur.ai/free/en/universities/carnegie-mellon-university/) | 34 papers |
| [University of Washington](https://md.research.elamur.ai/free/en/universities/university-of-washington/) | 31 papers |
| [Shanghai Jiao Tong University](https://md.research.elamur.ai/free/en/universities/shanghai-jiao-tong-university/) | 28 papers |
| [Microsoft](https://md.research.elamur.ai/free/en/companies/microsoft/) | 26 papers |
| [Chinese Academy of Sciences](https://md.research.elamur.ai/free/en/government/chinese-academy-of-sciences/) | 25 papers |
| [Tsinghua University](https://md.research.elamur.ai/free/en/universities/tsinghua-university/) | 24 papers |
| [Zhejiang University](https://md.research.elamur.ai/free/en/universities/zhejiang-university/) | 24 papers |
| [Peking University](https://md.research.elamur.ai/free/en/universities/peking-university/) | 23 papers |

Countries of the authors on this subject United States comes first: authors working there now wrote 223 papers on this subject.

**The numbers**

| Country | Papers |
| --- | --- |
| [United States](https://md.research.elamur.ai/free/en/countries/united-states/) | 223 papers |
| [China](https://md.research.elamur.ai/free/en/countries/china/) | 161 papers |
| [United Kingdom](https://md.research.elamur.ai/free/en/countries/united-kingdom/) | 64 papers |
| Hong Kong | 36 papers |
| [Singapore](https://md.research.elamur.ai/free/en/countries/singapore/) | 32 papers |
| [Canada](https://md.research.elamur.ai/free/en/countries/canada/) | 30 papers |
| [Germany](https://md.research.elamur.ai/free/en/countries/germany/) | 23 papers |
| [Switzerland](https://md.research.elamur.ai/free/en/countries/switzerland/) | 20 papers |
| [Israel](https://md.research.elamur.ai/free/en/countries/israel/) | 16 papers |
| [Australia](https://md.research.elamur.ai/free/en/countries/australia/) | 15 papers |

## Open questions

Why in-context learning works at all is still unsettled. Several explanations compete:

- Induction heads, attention circuits that copy patterns seen earlier in the context (Olsson et al. 2022).
- Bayesian accounts in which demonstrations help the model infer a latent task ([Dong et al. 2023](https://md.research.elamur.ai/free/en/papers/dong-2023-survey-context-learning/)).
- Interpretations of in-context learning as an implicit form of gradient descent ([Dong et al. 2023](https://md.research.elamur.ai/free/en/papers/dong-2023-survey-context-learning/)).

The random-label result described under Methods ([Min et al. 2022](https://md.research.elamur.ai/free/en/papers/min-2022-rethinking-role-demonstrations-what-makes/)) fits the view that prompts mostly locate a skill already acquired in pretraining ([Reynolds & McDonell 2021](https://md.research.elamur.ai/free/en/papers/reynolds-2021-prompt-programming-large-language-models/)). Whether large models can also learn genuinely new input–label mappings from context remains an [interpretability](https://md.research.elamur.ai/free/en/topics/interpretability/) question as much as an empirical one.

A second issue is honest measurement. True few-shot evaluation shows that model selection with few examples is itself unsolved ([Perez et al. 2021](https://md.research.elamur.ai/free/en/papers/perez-2021-true-few-shot-learning-language/)). Sensitivity to order and wording means a single reported number is rarely trustworthy ([Zhao et al. 2021](https://md.research.elamur.ai/free/en/papers/zhao-2021-calibrate-before-use-improving-few/); [Lu et al. 2022](https://md.research.elamur.ai/free/en/papers/lu-2022-fantastically-ordered-prompts-where-find/)). Agreed protocols with fixed examples, multiple seeds and contamination checks are still emerging.

A third concerns what the written reasoning means. Chain-of-thought outputs can be wrong even when the answer is right ([Wei et al. 2022](https://md.research.elamur.ai/free/en/papers/wei-2022-chain-thought-prompting-elicits-reasoning/)). They can also hide the cues that really drove a decision ([Turpin et al. 2023](https://md.research.elamur.ai/free/en/papers/turpin-2023-language-models-don-t-always/)). This limits their use for auditing and for [bias, fairness and safety](https://md.research.elamur.ai/free/en/topics/bias-fairness-safety/). Related work shows that prompts may be followed for reasons other than their meaning ([Webson & Pavlick 2022](https://md.research.elamur.ai/free/en/papers/webson-2022-do-prompt-based-models-really/)).

A fourth is cost and choice of paradigm. In-context learning reprocesses every demonstration for every query. Parameter-efficient fine-tuning can be both cheaper and more accurate ([Liu et al. 2022](https://md.research.elamur.ai/free/en/papers/liu-2022-few-shot-parameter-efficient-fine/)), and the PET results described above point the same way ([Schick & Schütze 2021](https://md.research.elamur.ai/free/en/papers/schick-2021-it-s-not-just-size/)). Which approach suits which task, budget and privacy constraint is not settled. In clinical settings, for example, data often cannot be sent to external APIs (Agrawal et al. 2022).

Finally, FSL ability is uneven across languages and modalities. Multilingual pretraining narrows the gap with English ([Lin et al. 2021](https://md.research.elamur.ai/free/en/papers/lin-2021-few-shot-learning-multilingual-language/)). Zero-shot transfer still falters for typologically distant languages, however ([Lauscher et al. 2020](https://md.research.elamur.ai/free/en/papers/lauscher-2020-zero-hero-limitations-zero-shot/)).

Speech shows the same pattern. On FLEURS speech–text retrieval, the baseline averages a score of 76.9 from speech to text. Chinese, Japanese, Korean, Thai and Burmese, however, remain in single digits ([Conneau et al. 2022](https://md.research.elamur.ai/free/en/papers/conneau-2022-fleurs-few-shot-learning-evaluation/)).

Delivering on the field's founding promise, learning from a handful of examples where large datasets will never exist, remains open work. That holds for [machine translation](https://md.research.elamur.ai/free/en/topics/machine-translation/), [speech](https://md.research.elamur.ai/free/en/topics/speech/) and [multimodal learning](https://md.research.elamur.ai/free/en/topics/multimodal/) alike.

## The library in figures

Neighbouring subjects: papers shared Natural language processing shares the most: 317 of the 373 papers here (85%) stand on both.

**The numbers**

| Shelf | Papers shared |
| --- | --- |
| [Natural language processing](https://md.research.elamur.ai/free/en/concepts/natural-language-processing/) | 317 shared · 85% |
| [Large language models](https://md.research.elamur.ai/free/en/concepts/large-language-models/) | 211 shared · 57% |
| [Question answering](https://md.research.elamur.ai/free/en/concepts/question-answering/) | 20 shared · 5% |
| [Information extraction](https://md.research.elamur.ai/free/en/concepts/information-extraction/) | 19 shared · 5% |
| [Text classification](https://md.research.elamur.ai/free/en/concepts/text-classification/) | 18 shared · 5% |
| [Natural language understanding](https://md.research.elamur.ai/free/en/concepts/natural-language-understanding/) | 16 shared · 4% |
| [Parsing](https://md.research.elamur.ai/free/en/concepts/parsing/) | 12 shared · 3% |
| [Knowledge graph](https://md.research.elamur.ai/free/en/concepts/knowledge-graph/) | 9 shared · 2% |
| [Retrieval-augmented generation](https://md.research.elamur.ai/free/en/concepts/retrieval-augmented-generation/) | 9 shared · 2% |
| [Text summarization](https://md.research.elamur.ai/free/en/concepts/text-summarization/) | 9 shared · 2% |

## Key papers

The works this page cites, the most cited first; citations as counted by OpenAlex plus the papers of this library that cite the work, read 2026-10-24.

- [Language Models are Few-Shot Learners](https://md.research.elamur.ai/free/en/papers/brown-2020-language-models-few-shot-learners/) — Tom B. Brown, Benjamin Mann, Nick Ryder et al. · 2020 · 13,703 citations
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://md.research.elamur.ai/free/en/papers/wei-2022-chain-thought-prompting-elicits-reasoning/) — Jason Wei, Xuezhi Wang, Dale Schuurmans et al. · 2022 · 11,308 citations
- [Large language models are zero-shot reasoners](https://md.research.elamur.ai/free/en/papers/kojima-2022-large-language-models-zero-shot/) — Takeshi Kojima, Shixiang Shane Gu, Machel Reid et al. · 2022 · 3,795 citations
- [Flamingo: a Visual Language Model for Few-Shot Learning](https://md.research.elamur.ai/free/en/papers/alayrac-2022-flamingo-visual-language-model-few/) — Jean-Baptiste Alayrac, Jeff Donahue, Pauline Luc et al. · 2022 · 2,118 citations
- [Making Pre-trained Language Models Better Few-shot Learners](https://md.research.elamur.ai/free/en/papers/gao-2021-making-pre-trained-language-models/) — Tianyu Gao, Adam Fisch, Danqi Chen · 2021 · 1,963 citations
- [Exploiting Cloze-Questions for Few-Shot Text Classification and Natural Language Inference](https://md.research.elamur.ai/free/en/papers/schick-2021-exploiting-cloze-questions-few-shot/) — Timo Schick, Hinrich Schütze · 2021 · 1,803 citations
- [Multitask Prompted Training Enables Zero-Shot Task Generalization](https://md.research.elamur.ai/free/en/papers/sanh-2021-multitask-prompted-training-enables-zero/) — Victor Sanh, Albert Webson, Colin Raffel et al. · 2021 · 1,581 citations
- [Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?](https://md.research.elamur.ai/free/en/papers/min-2022-rethinking-role-demonstrations-what-makes/) — Sewon Min, Xinxi Lyu, Ari Holtzman et al. · 2022 · 1,425 citations
- [Fantastically Ordered Prompts and Where to Find Them: Overcoming Few-Shot Prompt Order Sensitivity](https://md.research.elamur.ai/free/en/papers/lu-2022-fantastically-ordered-prompts-where-find/) — Yao Lu, Max Bartolo, Alastair Moore et al. · 2022 · 1,190 citations
- [Least-to-most prompting enables complex reasoning in large language models](https://md.research.elamur.ai/free/en/papers/al-2022-least-most-prompting-enables-complex/) — 2022 · 1,148 citations
- [It's Not Just Size That Matters: Small Language Models Are Also Few-Shot Learners](https://md.research.elamur.ai/free/en/papers/schick-2021-it-s-not-just-size/) — Timo Schick, Hinrich Schütze · 2021 · 1,122 citations
- [Prompt Programming for Large Language Models: Beyond the Few-Shot Paradigm](https://md.research.elamur.ai/free/en/papers/reynolds-2021-prompt-programming-large-language-models/) — Laria Reynolds, Kyle McDonell · 2021 · 1,115 citations
- [A Survey on In-context Learning](https://md.research.elamur.ai/free/en/papers/dong-2023-survey-context-learning/) — Qingxiu Dong, Lei Li, Damai Dai et al. · 2023 · 959 citations
- [Calibrate Before Use: Improving Few-Shot Performance of Language Models](https://md.research.elamur.ai/free/en/papers/zhao-2021-calibrate-before-use-improving-few/) — Tony Z. Zhao, Eric Wallace, Shi Feng et al. · 2021 · 794 citations
- [Few-Shot Parameter-Efficient Fine-Tuning is Better and Cheaper than In-Context Learning](https://md.research.elamur.ai/free/en/papers/liu-2022-few-shot-parameter-efficient-fine/) — Haokun Liu, Derek Tam, Mohammed Muqeeth et al. · 2022 · 736 citations
- [Benchmarking Zero-shot Text Classification: Datasets, Evaluation and Entailment Approach](https://md.research.elamur.ai/free/en/papers/yin-2019-benchmarking-zero-shot-text-classification/) — Wenpeng Yin, Jamaal Hay, Dan Roth · 2019 · 731 citations
- [Learning To Retrieve Prompts for In-Context Learning](https://md.research.elamur.ai/free/en/papers/rubin-2021-learning-retrieve-prompts-context-learning/) — Ohad Rubin, Jonathan Herzig, Jonathan Berant · 2021 · 717 citations
- [Automatic Chain of Thought Prompting in Large Language Models](https://md.research.elamur.ai/free/en/papers/zhang-2022-automatic-chain-thought-prompting-large/) — Zhuosheng Zhang, Aston Zhang, Mu Li et al. · 2022 · 690 citations
- [Massively Multilingual Sentence Embeddings for Zero-Shot Cross-Lingual Transfer and Beyond](https://md.research.elamur.ai/free/en/papers/artetxe-2018-massively-multilingual-sentence-embeddings-zero/) — Mikel Artetxe, Holger Schwenk · 2018 · 629 citations
- [Scalable Zero-shot Entity Linking with Dense Entity Retrieval](https://md.research.elamur.ai/free/en/papers/wu-2020-scalable-zero-shot-entity-linking/) — Ledell Wu, Fabio Petroni, Martin Josifoski et al. · 2020 · 618 citations
- [Language Models Don't Always Say What They Think: Unfaithful Explanations in Chain-of-Thought Prompting](https://md.research.elamur.ai/free/en/papers/turpin-2023-language-models-don-t-always/) — Miles Turpin, Julian Michael, Ethan Perez et al. · 2023 · 568 citations
- [LLM-Planner: Few-Shot Grounded Planning for Embodied Agents with Large Language Models](https://md.research.elamur.ai/free/en/papers/song-2022-llm-planner-few-shot-grounded/) — Chan Hee Song, Jiaman Wu, Clayton Washington et al. · 2022 · 564 citations
- [Precise Zero-Shot Dense Retrieval without Relevance Labels](https://md.research.elamur.ai/free/en/papers/gao-2022-precise-zero-shot-dense-retrieval/) — Luyu Gao, Xueguang Ma, Jimmy Lin et al. · 2022 · 545 citations
- [Few-shot Learning with Multilingual Language Models](https://md.research.elamur.ai/free/en/papers/lin-2021-few-shot-learning-multilingual-language/) — Xi Victoria Lin, Todor Mihaylov, Mikel Artetxe et al. · 2021 · 497 citations
- [DetectGPT: Zero-Shot Machine-Generated Text Detection using Probability Curvature](https://md.research.elamur.ai/free/en/papers/mitchell-2023-detectgpt-zero-shot-machine-generated/) — Eric Mitchell, Yoonho Lee, Alexander Khazatsky et al. · 2023 · 475 citations
- [PPT: Pre-trained Prompt Tuning for Few-shot Learning](https://md.research.elamur.ai/free/en/papers/gu-2022-ppt-pre-trained-prompt-tuning/) — Yuxian Gu, Xu Han, Zhiyuan Liu et al. · 2022 · 469 citations
- [From Zero to Hero: On the Limitations of Zero-Shot Language Transfer with Multilingual Transformers](https://md.research.elamur.ai/free/en/papers/lauscher-2020-zero-hero-limitations-zero-shot/) — Anne Lauscher, Vinit Ravishankar, Ivan Vulić et al. · 2020 · 461 citations
- [Meta-Learning for Low-Resource Neural Machine Translation](https://md.research.elamur.ai/free/en/papers/gu-2018-meta-learning-low-resource-neural/) — Jiatao Gu, Yong Wang, Yun Chen et al. · 2018 · 454 citations
- [Language Models as Zero-Shot Planners: Extracting Actionable Knowledge for Embodied Agents](https://md.research.elamur.ai/free/en/papers/huang-2022-language-models-zero-shot-planners/) — Wenlong Huang, Pieter Abbeel, Deepak Pathak et al. · 2022 · 422 citations
- [True Few-Shot Learning with Language Models](https://md.research.elamur.ai/free/en/papers/perez-2021-true-few-shot-learning-language/) — Ethan Perez, Douwe Kiela, Kyunghyun Cho · 2021 · 404 citations
- [DIN-SQL: Decomposed In-Context Learning of Text-to-SQL with Self-Correction](https://md.research.elamur.ai/free/en/papers/pourreza-2023-din-sql-decomposed-context-learning/) — Mohammadreza Pourreza, Davood Rafiei · 2023 · 398 citations
- [Do Prompt-Based Models Really Understand the Meaning of Their Prompts?](https://md.research.elamur.ai/free/en/papers/webson-2022-do-prompt-based-models-really/) — Albert Webson, Ellie Pavlick · 2022 · 398 citations
- [FLEURS: Few-shot Learning Evaluation of Universal Representations of Speech](https://md.research.elamur.ai/free/en/papers/conneau-2022-fleurs-few-shot-learning-evaluation/) — Alexis Conneau, Min Ma, Simran Khanuja et al. · 2022 · 398 citations
- [Complexity-Based Prompting for Multi-Step Reasoning](https://md.research.elamur.ai/free/en/papers/fu-2022-complexity-based-prompting-multi-step/) — Yao Fu, Hao Peng, Ashish Sabharwal et al. · 2022 · 343 citations

## Related concepts

- [Text classification](https://md.research.elamur.ai/free/en/concepts/text-classification/) — Text classification is the task of assigning one or more labels from a fixed, predefined set to a piece of text, such as a sentence…
- [Language model](https://md.research.elamur.ai/free/en/concepts/language-model/) — A language model is a probability distribution over sequences of tokens (words, subwords or characters). It is usually factored into…
- [Scaling laws](https://md.research.elamur.ai/free/en/concepts/scaling-laws/) — Scaling laws are empirical relationships, usually power laws, that describe how a neural network's performance, often its test loss…

## More papers

Topics: [🧠 Language models & pretraining](https://md.research.elamur.ai/free/en/topics/language-models/) · [🌱 Low-resource languages](https://md.research.elamur.ai/free/en/topics/low-resource-languages/) · [🧮 Reasoning](https://md.research.elamur.ai/free/en/topics/reasoning/)

Shelves: [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/)

Updated 24 October 2026

- [← previous concept Distributional semantics](https://md.research.elamur.ai/free/en/concepts/distributional-semantics/)
- [next concept → Fine-tuning](https://md.research.elamur.ai/free/en/concepts/fine-tuning/)
- [↑ back to concepts](https://md.research.elamur.ai/free/en/concepts/)