---
title: "Large Language Models Are Latent Variable Models: Explaining"
description: "In-context learning in LLMs depends heavily on which few-shot demonstrations are chosen. 53 references, cited by 85 papers here."
authors: ["Xinyi Wang", "Wanrong Zhu", "Michael Saxon", "Mark Steyvers", "William Yang Wang"]
year: 2023
venue: "arXiv:2301.11916"
source: "https://arxiv.org/abs/2301.11916"
doi: "10.48550/arxiv.2301.11916"
arxiv: "2301.11916"
licence: "arXiv — non-exclusive license to distribute"
licence_url: "http://arxiv.org/licenses/nonexclusive-distrib/1.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/wang-2023-large-language-models-latent-variable/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/wang-2023-large-language-models-latent-variable/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2023](https://md.research.elamur.ai/free/en/papers/2023/page/6/) / Large Language Models Are Latent Variable Models: Explaining and Finding Good Demonstrations for In-Context Learning

# Large Language Models Are Latent Variable Models: Explaining and Finding Good Demonstrations for In-Context Learning

**Authors:** [Xinyi Wang](https://md.research.elamur.ai/free/en/authors/xinyi-wang/), [Wanrong Zhu](https://md.research.elamur.ai/free/en/authors/wanrong-zhu/), [Michael Saxon](https://md.research.elamur.ai/free/en/authors/michael-saxon/), Mark Steyvers and [William Yang Wang](https://md.research.elamur.ai/free/en/authors/william-yang-wang/) · **Year:** 2023 · **Venue:** [arXiv:2301.11916](https://md.research.elamur.ai/free/en/venues/neural-information-processing-systems/) ([NeurIPS 2023](https://md.research.elamur.ai/free/en/venues/neural-information-processing-systems/2023/)) · **Licence:** [arXiv — non-exclusive license to distribute](http://arxiv.org/licenses/nonexclusive-distrib/1.0/)

**Links:** [Publisher's page](https://arxiv.org/abs/2301.11916) · [PDF](https://arxiv.org/pdf/2301.11916) · [DOI 10.48550/arxiv.2301.11916](https://doi.org/10.48550/arxiv.2301.11916) · [arXiv 2301.11916](https://arxiv.org/abs/2301.11916) · [OpenAlex](https://openalex.org/W7133183456)

**Citations:** 31 citations (OpenAlex, 2026-10-14) · [85](https://md.research.elamur.ai/free/en/papers/wang-2023-large-language-models-latent-variable/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
In-context learning in LLMs depends heavily on which few-shot demonstrations are chosen. Existing theories of why it works have mostly been tested on synthetic data and toy Transformers. This paper treats real-world LLMs as implicit latent variable models, similar to topic models. In this view, the prompt lets the model infer a hidden concept variable that carries task and format information. The authors model data generation with a three-variable causal graph linking input, label and concept, in either causal direction. They prove that a finite set of well-chosen demonstrations can bring the in-context predictor to the Bayes optimal one. Their practical method has two phases. First, a small LM (under 1B parameters) learns concept tokens for each task. Second, the annotated examples most likely to predict those tokens are kept as demonstrations. These demonstrations transfer directly to larger GPT models, up to 175B parameters. Averaged over eight GPT models and eight text classification datasets, the method clearly beats baselines, and it also helps on GSM8K. This gives a cheap, model-transferable way to pick prompts and empirical support for the latent-concept view.

## Key points
- Evaluation covers eight GPT models, eight real-world text classification datasets and the GSM8K math word problem set. Baselines include random and similarity-based demonstration selection.
- Concept tokens are learned on a small LM and reused for selection. Lengthening the concept token sequence from 10 to 20 lowered performance, likely because extra tokens add noise.
- Selected demonstrations were mostly insensitive to ordering, with EmoC as the high-variance exception. This suggests task information lives in the demonstrations' content rather than their order.
- Chosen examples tend to be longer and harder. For SST2 they sometimes contain contrast ("but"), for GSM8K they show more deductive steps, and their labels are usually class-balanced.
- Published at NeurIPS 2023. Code is at github.com/WANGXinyiLinda/concept-based-demonstration-selection.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works

🧭 Topics (1): [🧠 Language models & pretraining](https://md.research.elamur.ai/free/en/topics/language-models/) · 3,320 works

💡 Concepts (1): [💡 Topic modeling](https://md.research.elamur.ai/free/en/concepts/topic-modeling/) · 47 works

## Cite

Cite as: Xinyi Wang, Wanrong Zhu, Michael Saxon, Mark Steyvers and William Yang Wang. 2023. Large Language Models Are Latent Variable Models: Explaining and Finding Good Demonstrations for In-Context Learning. arXiv:2301.11916.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@misc{wang2023large,
  title = {{Large Language Models Are Latent Variable Models: Explaining and Finding Good Demonstrations for In-Context Learning}},
  author = {Xinyi Wang, Wanrong Zhu, Michael Saxon, Mark Steyvers and William Yang Wang},
  year = {2023},
  doi = {10.48550/arxiv.2301.11916},
  eprint = {2301.11916},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2301.11916}
}
```

- [← previous How to Train Your Dragon: Diverse Augmentation Towards Generalizable Dense Retrieval](https://md.research.elamur.ai/free/en/papers/lin-2023-how-train-your-dragon-diverse/)
- [next → Large Language Models Understand and Can be Enhanced by Emotional Stimuli](https://md.research.elamur.ai/free/en/papers/li-2023-large-language-models-understand-enhanced/)
- [↑ back to papers of 2023](https://md.research.elamur.ai/free/en/papers/2023/page/6/)

## References

53 references, 30 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. E. Akyürek, D. Schuurmans, J. Andreas, T. Ma, and D. Zhou. [What learning algorithm is in-context learning? investigations with linear models](https://md.research.elamur.ai/free/en/papers/akyurek-2022-learning-algorithm-context-learning-investigations/). arXiv preprint arXiv:2211.15661, 2022.
2. H. Bansal, K. Gopalakrishnan, S. Dingliwal, S. Bodapati, K. Kirchhoff, and D. Roth. Rethinking the role of scale for in-context learning: An interpretability-based case study at 66 billion scale. arXiv preprint arXiv:2212.09095, 2022. [find](https://scholar.google.com/scholar?q=H%2E+Bansal%2C+K%2E+Gopalakrishnan%2C+S%2E+Dingliwal%2C+S%2E+Bodapati%2C+K%2E+Kirchhoff%2C+and+D%2E+Roth%2E+Rethinking+the+role+of+scale+for+in%2Dcontext+learning%3A+An+interpretability%2Dbased+case+study+at+66+billion+scale%2E+arXi)
3. D. M. Blei, A. Y. Ng, and M. I. Jordan. [Latent dirichlet allocation](https://md.research.elamur.ai/free/en/papers/blei-2003-latent-dirichlet-allocation/). J. Mach. Learn. Res., 3 (null):993–1022, mar 2003. ISSN 1532-4435.
4. T. Brown, B. Mann, N. Ryder, M. Subbiah, J. D. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell, et al. [Language models are few-shot learners](https://md.research.elamur.ai/free/en/papers/brown-2020-language-models-few-shot-learners/). Advances in neural information processing systems, 33:1877–1901, 2020.
5. S. C. Chan, A. Santoro, A. K. Lampinen, J. X. Wang, A. Singh, P. H. Richemond, J. McClelland, and F. Hill. Data distributional properties drive emergent few-shot learning in transformers. arXiv preprint arXiv:2205.05055, 2022. [find](https://scholar.google.com/scholar?q=S%2E+C%2E+Chan%2C+A%2E+Santoro%2C+A%2E+K%2E+Lampinen%2C+J%2E+X%2E+Wang%2C+A%2E+Singh%2C+P%2E+H%2E+Richemond%2C+J%2E+McClelland%2C+and+F%2E+Hill%2E+Data+distributional+properties+drive+emergent+few%2Dshot+learning+in+transformers%2E+arXiv+prepri)
6. A. Chatterjee, K. N. Narahari, M. Joshi, and P. Agrawal. [Semeval-2019 task 3: Emocontext contextual emotion detection in text](https://md.research.elamur.ai/free/en/papers/chatterjee-2019-semeval-2019-task-3-emocontext/). In Proceedings of the 13th International Workshop on Semantic Evaluation, pages 39–48, Minneapolis, Minnesota, USA, 2019. Association for Computational Linguistics. doi: 10.…
7. J. Chen, L. Chen, and T. Zhou. It takes one to tango but more make trouble? in-context training with different number of demonstrations. arXiv preprint arXiv:2303.08119, 2023. — [arXiv:2303.08119](https://arxiv.org/abs/2303.08119) [find](https://scholar.google.com/scholar?q=J%2E+Chen%2C+L%2E+Chen%2C+and+T%2E+Zhou%2E+It+takes+one+to+tango+but+more+make+trouble%3F+in%2Dcontext+training+with+different+number+of+demonstrations%2E+arXiv+preprint+arXiv%3A2303%2E08119%2C+2023%2E+%E2%80%94+%5BarXiv%3A2303%2E08119%5D%28htt)
8. W. Chen, X. Ma, X. Wang, and W. W. Cohen. Program of thoughts prompting: Disentangling computation from reasoning for numerical reasoning tasks. arXiv preprint arXiv:2211.12588, 2022. [find](https://scholar.google.com/scholar?q=W%2E+Chen%2C+X%2E+Ma%2C+X%2E+Wang%2C+and+W%2E+W%2E+Cohen%2E+Program+of+thoughts+prompting%3A+Disentangling+computation+from+reasoning+for+numerical+reasoning+tasks%2E+arXiv+preprint+arXiv%3A2211%2E12588%2C+2022%2E)
9. K. Cobbe, V. Kosaraju, M. Bavarian, M. Chen, H. Jun, L. Kaiser, M. Plappert, J. Tworek, J. Hilton, R. Nakano, et al. [Training verifiers to solve math word problems](https://md.research.elamur.ai/free/en/papers/cobbe-2021-training-verifiers-solve-math-word/). arXiv preprint arXiv:2110.14168, 2021.
10. D. Dai, Y. Sun, L. Dong, Y. Hao, Z. Sui, and F. Wei. Why can gpt learn in-context? language models secretly perform gradient descent as meta optimizers. arXiv preprint arXiv:2212.[10559, 2022](https://md.research.elamur.ai/free/en/papers/dai-2023-why-can-gpt-learn-context/).
11. L. Devroye, L. Györfi, and G. Lugosi. A probabilistic theory of pattern recognition. In Stochastic Modelling and Applied Probability, 1996. [find](https://scholar.google.com/scholar?q=L%2E+Devroye%2C+L%2E+Gy%C3%B6rfi%2C+and+G%2E+Lugosi%2E+A+probabilistic+theory+of+pattern+recognition%2E+In+Stochastic+Modelling+and+Applied+Probability%2C+1996%2E)
12. M. Hahn and N. Goyal. A theory of emergent in-context learning as implicit structure induction. arXiv preprint arXiv:2303.07971, 2023. — [arXiv:2303.07971](https://arxiv.org/abs/2303.07971) [find](https://scholar.google.com/scholar?q=M%2E+Hahn+and+N%2E+Goyal%2E+A+theory+of+emergent+in%2Dcontext+learning+as+implicit+structure+induction%2E+arXiv+preprint+arXiv%3A2303%2E07971%2C+2023%2E+%E2%80%94+%5BarXiv%3A2303%2E07971%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2303%2E07971%29)
13. X. Han, D. Simig, T. Mihaylov, Y. Tsvetkov, A. Celikyilmaz, and T. Wang. Understanding in-context learning via supportive pretraining data. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 12660–12673, 2023. [find](https://scholar.google.com/scholar?q=X%2E+Han%2C+D%2E+Simig%2C+T%2E+Mihaylov%2C+Y%2E+Tsvetkov%2C+A%2E+Celikyilmaz%2C+and+T%2E+Wang%2E+Understanding+in%2Dcontext+learning+via+supportive+pretraining+data%2E+In+Proceedings+of+the+61st+Annual+Meeting+of+the+Association)
14. H. Jiang. A latent space theory for emergent abilities in large language models. arXiv preprint arXiv:2304.09960, 2023. — [arXiv:2304.09960](https://arxiv.org/abs/2304.09960) [find](https://scholar.google.com/scholar?q=H%2E+Jiang%2E+A+latent+space+theory+for+emergent+abilities+in+large+language+models%2E+arXiv+preprint+arXiv%3A2304%2E09960%2C+2023%2E+%E2%80%94+%5BarXiv%3A2304%2E09960%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2304%2E09960%29)
15. B. LeBrun, A. Sordoni, and T. J. O’Donnell. Evaluating distributional distortion in neural language modeling. In International Conference on Learning Representations, 2022. [find](https://scholar.google.com/scholar?q=B%2E+LeBrun%2C+A%2E+Sordoni%2C+and+T%2E+J%2E+O%E2%80%99Donnell%2E+Evaluating+distributional+distortion+in+neural+language+modeling%2E+In+International+Conference+on+Learning+Representations%2C+2022%2E)
16. J. Lehmann, R. Isele, M. Jakob, A. Jentzsch, D. Kontokostas, P. N. Mendes, S. Hellmann, M. Morsey, P. Van Kleef, S. Auer, et al. Dbpedia–a large-scale, multilingual knowledge base extracted from wikipedia. Semantic web, 6(2):167–195, 2015. — [DBpedia – A Large-scale, Multilingual Knowledge Base Extracted from Wikipedia](https://md.research.elamur.ai/free/en/papers/lehmann-2015-dbpedia-large-scale-multilingual-knowledge/)
17. B. Lester, R. Al-Rfou, and N. Constant. [The power of scale for parameter-efficient prompt tuning](https://md.research.elamur.ai/free/en/papers/lester-2021-power-scale-parameter-efficient-prompt/). In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 3045–3059, 2021.
18. Y. Li, M. E. Ildiz, D. Papailiopoulos, and S. Oymak. Transformers as algorithms: Generalization and implicit model selection in in-context learning. arXiv preprint arXiv:2301.07067, 2023. — [arXiv:2301.07067](https://arxiv.org/abs/2301.07067) [find](https://scholar.google.com/scholar?q=Y%2E+Li%2C+M%2E+E%2E+Ildiz%2C+D%2E+Papailiopoulos%2C+and+S%2E+Oymak%2E+Transformers+as+algorithms%3A+Generalization+and+implicit+model+selection+in+in%2Dcontext+learning%2E+arXiv+preprint+arXiv%3A2301%2E07067%2C+2023%2E+%E2%80%94+%5BarXiv%3A230)
19. J. Liu, D. Shen, Y. Zhang, B. Dolan, L. Carin, and W. Chen. [What makes good in-context examples for GPT-3?](https://md.research.elamur.ai/free/en/papers/liu-2022-what-makes-good-context-examples/) In Proceedings of Deep Learning Inside Out (DeeLIO 2022): The 3rd Workshop on Knowledge Extraction and Integration for Deep Learning Architectures, pages 100–114, Dublin, Ireland and Online, Ma…
20. Y. Lu, M. Bartolo, A. Moore, S. Riedel, and P. Stenetorp. [Fantastically ordered prompts and where to find them: Overcoming few-shot prompt order sensitivity](https://md.research.elamur.ai/free/en/papers/lu-2022-fantastically-ordered-prompts-where-find/). In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 8086–8098, 2022.
21. P. Malo, A. Sinha, P. Korhonen, J. Wallenius, and P. Takala. Good debt or bad debt: Detecting semantic orientations in economic texts. Journal of the Association for Information Science and Technology, 65, 2014. [find](https://scholar.google.com/scholar?q=P%2E+Malo%2C+A%2E+Sinha%2C+P%2E+Korhonen%2C+J%2E+Wallenius%2C+and+P%2E+Takala%2E+Good+debt+or+bad+debt%3A+Detecting+semantic+orientations+in+economic+texts%2E+Journal+of+the+Association+for+Information+Science+and+Technology)
22. Y. Miao, L. Yu, and P. Blunsom. Neural variational inference for text processing. In M. F. Balcan and K. Q. Weinberger, editors, Proceedings of The 33rd International Conference on Machine Learning, volume 48 of Proceedings of Machine Learning Research, pages 1727–1736, New York, New York, USA, 20–2… [find](https://scholar.google.com/scholar?q=Y%2E+Miao%2C+L%2E+Yu%2C+and+P%2E+Blunsom%2E+Neural+variational+inference+for+text+processing%2E+In+M%2E+F%2E+Balcan+and+K%2E+Q%2E+Weinberger%2C+editors%2C+Proceedings+of+The+33rd+International+Conference+on+Machine+Learning%2C+v)
23. Y. Miao, E. Grefenstette, and P. Blunsom. Discovering discrete latent topics with neural variational inference. In International conference on machine learning, pages 2410–2419. PMLR, 2017. [find](https://scholar.google.com/scholar?q=Y%2E+Miao%2C+E%2E+Grefenstette%2C+and+P%2E+Blunsom%2E+Discovering+discrete+latent+topics+with+neural+variational+inference%2E+In+International+conference+on+machine+learning%2C+pages+2410%E2%80%932419%2E+PMLR%2C+2017%2E)
24. S. Min, M. Lewis, H. Hajishirzi, and L. Zettlemoyer. [Noisy channel language model prompting for few-shot text classification](https://md.research.elamur.ai/free/en/papers/min-2021-noisy-channel-language-model-prompting/). In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 5316–5330, 2022.
25. S. Min, M. Lewis, L. Zettlemoyer, and H. Hajishirzi. MetaICL: Learning to learn in context. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 2791–2809, Seattle, United States, July 2022. Associati… [find](https://scholar.google.com/scholar?q=S%2E+Min%2C+M%2E+Lewis%2C+L%2E+Zettlemoyer%2C+and+H%2E+Hajishirzi%2E+MetaICL%3A+Learning+to+learn+in+context%2E+In+Proceedings+of+the+2022+Conference+of+the+North+American+Chapter+of+the+Association+for+Computational+Lin)
26. S. Min, X. Lyu, A. Holtzman, M. Artetxe, M. Lewis, H. Hajishirzi, and L. Zettlemoyer. [Rethinking the role of demonstrations: What makes in-context learning work?](https://md.research.elamur.ai/free/en/papers/min-2022-rethinking-role-demonstrations-what-makes/) In EMNLP, 2022.
27. I. Mollas, Z. Chrysopoulou, S. Karlos, and G. Tsoumakas. Ethos: an online hate speech detection dataset, 2020. [find](https://scholar.google.com/scholar?q=I%2E+Mollas%2C+Z%2E+Chrysopoulou%2C+S%2E+Karlos%2C+and+G%2E+Tsoumakas%2E+Ethos%3A+an+online+hate+speech+detection+dataset%2C+2020%2E)
28. L. Ouyang, J. Wu, X. Jiang, D. Almeida, C. L. Wainwright, P. Mishkin, C. Zhang, S. Agarwal, K. Slama, A. Ray, et al. [Training language models to follow instructions with human feedback](https://md.research.elamur.ai/free/en/papers/ouyang-2022-training-language-models-follow-instructions/). arXiv preprint arXiv:2203.02155, 2022.
29. E. Perez, D. Kiela, and K. Cho. [True few-shot learning with language models](https://md.research.elamur.ai/free/en/papers/perez-2021-true-few-shot-learning-language/). In A. Beygelzimer, Y. Dauphin, P. Liang, and J. W. Vaughan, editors, Advances in Neural Information Processing Systems, 2021. URL [https://openreview.net/forum?id=ShnM-rRh4T](https://openreview.net/forum?id=ShnM-rRh4T). [↗](https://openreview.net/forum?id=ShnM-rRh4T)
30. A. Radford, J. Wu, R. Child, D. Luan, D. Amodei, and I. Sutskever. Language models are unsupervised multitask learners. 2019. [find](https://scholar.google.com/scholar?q=A%2E+Radford%2C+J%2E+Wu%2C+R%2E+Child%2C+D%2E+Luan%2C+D%2E+Amodei%2C+and+I%2E+Sutskever%2E+Language+models+are+unsupervised+multitask+learners%2E+2019%2E)
31. N. Reimers and I. Gurevych. [Sentence-bert: Sentence embeddings using siamese bert-networks](https://md.research.elamur.ai/free/en/papers/reimers-2019-sentence-bert-sentence-embeddings-siamese/). In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, 11 2019. URL https://arxiv.org/abs/1908.10084. [↗](https://arxiv.org/abs/1908.10084)
32. O. Rubin, J. Herzig, and J. Berant. [Learning to retrieve prompts for in-context learning](https://md.research.elamur.ai/free/en/papers/rubin-2021-learning-retrieve-prompts-context-learning/). arXiv preprint arXiv:2112.08633, 2021.
33. E. Saravia, H.-C. T. Liu, Y.-H. Huang, J. Wu, and Y.-S. Chen. [CARER: Contextualized affect representations for emotion recognition](https://md.research.elamur.ai/free/en/papers/saravia-2018-carer-contextualized-affect-representations-emotion/). In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 3687–3697, Brussels, Belgium, Oct.-Nov. 2018. Association for Computati…
34. N. Saunshi, S. Malladi, and S. Arora. A mathematical exploration of why language models help solve downstream tasks. In International Conference on Learning Representations, 2021. URL [https://openreview.net/forum?id=vVjIW3sEc1s](https://openreview.net/forum?id=vVjIW3sEc1s). [↗](https://openreview.net/forum?id=vVjIW3sEc1s)
35. R. Socher, A. Perelygin, J. Wu, J. Chuang, C. D. Manning, A. Ng, and C. Potts. [Recursive deep models for semantic compositionality over a sentiment treebank](https://md.research.elamur.ai/free/en/papers/socher-2013-recursive-deep-models-semantic-compositionality/). In Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, pages 1631– 1642, Seattle, Washington, USA, Oct. 20…
36. N. Stiennon, L. Ouyang, J. Wu, D. Ziegler, R. Lowe, C. Voss, A. Radford, D. Amodei, and P. F. Christiano. Learning to summarize with human feedback. Advances in Neural Information Processing Systems, 33:3008–3021, 2020. [find](https://scholar.google.com/scholar?q=N%2E+Stiennon%2C+L%2E+Ouyang%2C+J%2E+Wu%2C+D%2E+Ziegler%2C+R%2E+Lowe%2C+C%2E+Voss%2C+A%2E+Radford%2C+D%2E+Amodei%2C+and+P%2E+F%2E+Christiano%2E+Learning+to+summarize+with+human+feedback%2E+Advances+in+Neural+Information+Processing+Systems%2C+)
37. H. Su, J. Kasai, C. H. Wu, W. Shi, T. Wang, J. Xin, R. Zhang, M. Ostendorf, L. Zettlemoyer, N. A. Smith, et al. Selective annotation makes language models better few-shot learners. arXiv preprint arXiv:2209.01975, 2022. [find](https://scholar.google.com/scholar?q=H%2E+Su%2C+J%2E+Kasai%2C+C%2E+H%2E+Wu%2C+W%2E+Shi%2C+T%2E+Wang%2C+J%2E+Xin%2C+R%2E+Zhang%2C+M%2E+Ostendorf%2C+L%2E+Zettlemoyer%2C+N%2E+A%2E+Smith%2C+et+al%2E+Selective+annotation+makes+language+models+better+few%2Dshot+learners%2E+arXiv+preprint+arXi)
38. H. Touvron, T. Lavril, G. Izacard, X. Martinet, M.-A. Lachaux, T. Lacroix, B. Rozière, N. Goyal, E. Hambro, F. Azhar, et al. [Llama: Open and efficient foundation language models](https://md.research.elamur.ai/free/en/papers/touvron-2023-llama-open-efficient-foundation-language/). arXiv preprint arXiv:2302.13971, 2023.
39. H. Touvron, L. Martin, K. Stone, P. Albert, A. Almahairi, Y. Babaei, N. Bashlykov, S. Batra, P. Bhargava, S. Bhosale, D. Bikel, L. Blecher, C. C. Ferrer, M. Chen, G. Cucurull, D. Esiobu, J. Fernandes, J. Fu, W. Fu, B. Fuller, C. Gao, V. Goswami, N. Goyal, A. Hartshorn, S. Hosseini, R. Hou, H. Inan,… — [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://md.research.elamur.ai/free/en/papers/touvron-2023-llama-2-open-foundation-fine/)
40. L. van der Maaten and G. Hinton. Visualizing data using t-sne. Journal of Machine Learning Research, 9(86):2579–2605, 2008. URL [http://jmlr.org/papers/v9/vandermaaten08a](http://jmlr.org/papers/v9/vandermaaten08a). html. [↗](http://jmlr.org/papers/v9/vandermaaten08a)
41. A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin. [Attention is all you need](https://md.research.elamur.ai/free/en/papers/vaswani-2017-attention-all-need/). Advances in neural information processing systems, 30, 2017.
42. J. von Oswald, E. Niklasson, E. Randazzo, J. Sacramento, A. Mordvintsev, A. Zhmoginov, and M. Vladymyrov. [Transformers learn in-context by gradient descent](https://md.research.elamur.ai/free/en/papers/oswald-2022-transformers-learn-context-gradient-descent/). arXiv preprint arXiv:2212.07677, 2022.
43. A. Wang, A. Singh, J. Michael, F. Hill, O. Levy, and S. R. Bowman. [Glue: A multi-task benchmark and analysis platform for natural language understanding](https://md.research.elamur.ai/free/en/papers/wang-2018-glue-multi-task-benchmark-analysis/). EMNLP 2018, page 353, 2018.
44. B. Wang and A. Komatsuzaki. GPT-J-6B: A 6 Billion Parameter Autoregressive Language Model. [https://github.com/kingoflolz/mesh-transformer-jax](https://github.com/kingoflolz/mesh-transformer-jax), May 2021. [↗](https://github.com/kingoflolz/mesh-transformer-jax)
45. X. Wang, J. Wei, D. Schuurmans, Q. Le, E. Chi, and D. Zhou. [Self-consistency improves chain of thought reasoning in language models](https://md.research.elamur.ai/free/en/papers/wang-2022-self-consistency-improves-chain-thought/). arXiv preprint arXiv:2203.11171, 2022.
46. A. Warstadt, A. Singh, and S. R. Bowman. Neural network acceptability judgments. arXiv preprint arXiv:1805.12471, 2018. [find](https://scholar.google.com/scholar?q=A%2E+Warstadt%2C+A%2E+Singh%2C+and+S%2E+R%2E+Bowman%2E+Neural+network+acceptability+judgments%2E+arXiv+preprint+arXiv%3A1805%2E12471%2C+2018%2E)
47. C. Wei, S. M. Xie, and T. Ma. Why do pretrained language models help in downstream tasks? an analysis of head and prompt tuning. Neural Information Processing Systems (NeurIPS), 2021. [find](https://scholar.google.com/scholar?q=C%2E+Wei%2C+S%2E+M%2E+Xie%2C+and+T%2E+Ma%2E+Why+do+pretrained+language+models+help+in+downstream+tasks%3F+an+analysis+of+head+and+prompt+tuning%2E+Neural+Information+Processing+Systems+%28NeurIPS%29%2C+2021%2E)
48. J. Wei, X. Wang, D. Schuurmans, M. Bosma, E. Chi, Q. Le, and D. Zhou. Chain of thought prompting elicits reasoning in large language models. arXiv preprint arXiv:2201.[11903, 2022](https://md.research.elamur.ai/free/en/papers/wei-2022-chain-thought-prompting-elicits-reasoning/).
49. T. Wolf, L. Debut, V. Sanh, J. Chaumond, C. Delangue, A. Moi, P. Cistac, T. Rault, R. Louf, M. Funtowicz, et al. Huggingface’s [transformers: State-of-the-art natural language processing](https://md.research.elamur.ai/free/en/papers/wolf-2020-transformers-state-art-natural-language/). arXiv preprint arXiv:1910.03771, 2019.
50. S. M. Xie, A. Raghunathan, P. Liang, and T. Ma. [An explanation of in-context learning as implicit bayesian inference](https://md.research.elamur.ai/free/en/papers/xie-2021-explanation-context-learning-implicit-bayesian/). In International Conference on Learning Representations, 2022. URL [https://openreview.net/forum?id=RdJVFCHjUMI](https://openreview.net/forum?id=RdJVFCHjUMI). [↗](https://openreview.net/forum?id=RdJVFCHjUMI)
51. S. Zhang, S. Roller, N. Goyal, M. Artetxe, M. Chen, S. Chen, C. Dewan, M. Diab, X. Li, X. V. Lin, et al. [Opt: Open pre-trained transformer language models](https://md.research.elamur.ai/free/en/papers/zhang-2022-opt-open-pre-trained-transformer/). arXiv preprint arXiv:2205.01068, 2022.
52. X. Zhang, J. Zhao, and Y. LeCun. [Character-level convolutional networks for text classification](https://md.research.elamur.ai/free/en/papers/zhang-2015-character-level-convolutional-networks-text/). Advances in neural information processing systems, 28, 2015.
53. D. Zhou, N. Schärli, L. Hou, J. Wei, N. Scales, X. Wang, D. Schuurmans, O. Bousquet, Q. Le, and E. Chi. [Least-to-most prompting enables complex reasoning in large language models](https://md.research.elamur.ai/free/en/papers/al-2022-least-most-prompting-enables-complex/). arXiv preprint arXiv:2205.10625, 2022.

*Updated 29 October 2026*