---
title: "Is In-Context Learning in Large Language Models Bayesian? A"
description: "From the theory they build practical diagnostics and test statistics. 48 references, cited by 9 papers here."
authors: ["Fabian Falck", "Ziyu Wang", "Chris Holmes"]
year: 2024
venue: "arXiv:2406.00793"
source: "https://arxiv.org/abs/2406.00793"
doi: "10.48550/arxiv.2406.00793"
arxiv: "2406.00793"
licence: "CC BY 4.0 (arXiv)"
licence_url: "http://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/falck-2024-context-learning-large-language-models/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/falck-2024-context-learning-large-language-models/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2024](https://md.research.elamur.ai/free/en/papers/2024/page/13/) / Is In-Context Learning in Large Language Models Bayesian? A Martingale Perspective

# Is In-Context Learning in Large Language Models Bayesian? A Martingale Perspective

**Authors:** Fabian Falck, Ziyu Wang and Chris Holmes · **Year:** 2024 · **Venue:** arXiv:2406.00793 · **Licence:** [CC BY 4.0 (arXiv)](http://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://arxiv.org/abs/2406.00793) · [PDF](https://arxiv.org/pdf/2406.00793) · [DOI 10.48550/arxiv.2406.00793](https://doi.org/10.48550/arxiv.2406.00793) · [arXiv 2406.00793](https://arxiv.org/abs/2406.00793) · [OpenAlex](https://openalex.org/W4399399219)

**Citations:** 2 citations (OpenAlex, 2026-10-14) · [9](https://md.research.elamur.ai/free/en/papers/falck-2024-context-learning-large-language-models/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
The paper asks whether in-context learning (ICL) in large language models behaves like Bayesian inference when the prompt contains exchangeable data, a view many earlier works have assumed. The authors approach this through the martingale property. A Bayesian learner must satisfy it: if the model imputes missing samples from itself and then predicts, the averaged prediction should equal the prediction made from the observed data alone. They argue that this property is needed for order-independent, unambiguous predictions and for splitting uncertainty into epistemic and aleatoric parts. From the theory they build practical diagnostics and test statistics. They also compare how fast LLM uncertainty shrinks with more data against a Bayesian reference. The tests run on deliberately simple synthetic tasks with known data-generating processes. Across Llama-2, Mistral, GPT-3, GPT-3.5 and GPT-4, they find violations of the martingale property and non-Bayesian uncertainty scaling. The authors therefore reject the claim that ICL is Bayesian and warn against relying on LLM ICL in safety-critical, exchangeable-data settings such as clinical trial imputation.

## Key points
- The motivating example is an interim analysis of a two-arm RCT (100 patients, 50 per arm), in which an LLM imputes the remaining patients. If the martingale property fails, the result can depend on which patient is imputed first.
- When a model shifts its own predictions on average by querying itself, the authors call this "introspective hallucination." The violations show up as drift over longer sampled paths.
- The martingale property approximately holds for all LLMs only in a Bernoulli setting with θ = 0.5 and a short horizon (m = n/2). Epistemic uncertainty scaling (statistic T3) is examined in that setting and compared with a reference Bayesian model and α-fractional Bayesian posteriors (α = 0.5 and α = 2).
- Llama-2-7B and GPT-3.5 shrink uncertainty more slowly than a Bayesian model, and GPT-3.5 is over-confident when samples are few. GPT-3-170B and Mistral-7B come closer to Bayesian scaling and also adhere best to the martingale property.
- The paper was published at ICML 2024 (University of Oxford). The authors name the restriction to small synthetic datasets as the main limitation and point to tabular-data prediction and synthetic data generation as uses for the diagnostics.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [University of Oxford](https://md.research.elamur.ai/free/en/universities/university-of-oxford/) — University, Oxford, United Kingdom · 138 works · 161,448 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 19,503 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 18,805 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 17,959 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 17,789 works

🧭 Topics (1): [🧠 Language models & pretraining](https://md.research.elamur.ai/free/en/topics/language-models/) · 4,145 works

## Cite

Cite as: Fabian Falck, Ziyu Wang and Chris Holmes. 2024. Is In-Context Learning in Large Language Models Bayesian? A Martingale Perspective. arXiv:2406.00793.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@misc{falck2024is,
  title = {{Is In-Context Learning in Large Language Models Bayesian? A Martingale Perspective}},
  author = {Fabian Falck, Ziyu Wang and Chris Holmes},
  year = {2024},
  doi = {10.48550/arxiv.2406.00793},
  eprint = {2406.00793},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2406.00793}
}
```

- [← previous GAOKAO-MM: A Chinese Human-Level Benchmark for Multimodal Models Evaluation](https://md.research.elamur.ai/free/en/papers/zong-2024-gaokao-mm-chinese-human/)
- [next → Is Your Paper Being Reviewed by an LLM? Investigating AI Text Detectability in Peer Review](https://md.research.elamur.ai/free/en/papers/yu-2024-paper-being-reviewed-llm/)
- [↑ back to papers of 2024](https://md.research.elamur.ai/free/en/papers/2024/page/13/)

## References

48 references, 16 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Akyürek, E., Schuurmans, D., Andreas, J., Ma, T., and Zhou, D. [What learning algorithm is in-context learning? investigations with linear models](https://md.research.elamur.ai/free/en/papers/akyurek-2022-learning-algorithm-context-learning-investigations/). arXiv preprint arXiv:2211.15661, 2022.
2. Hahn, M. and Goyal, N. A theory of emergent in-context learning as implicit structure induction. arXiv preprint arXiv:2303.07971, 2023. — [arXiv:2303.07971](https://arxiv.org/abs/2303.07971) [find](https://scholar.google.com/scholar?q=Hahn%2C+M%2E+and+Goyal%2C+N%2E+A+theory+of+emergent+in%2Dcontext+learning+as+implicit+structure+induction%2E+arXiv+preprint+arXiv%3A2303%2E07971%2C+2023%2E+%E2%80%94+%5BarXiv%3A2303%2E07971%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2303%2E07971%29)
3. Bai, Y., Chen, F., Wang, H., Xiong, C., and Mei, S. Transformers as statisticians: Provable in-context learning with in-context algorithm selection. arXiv preprint arXiv:2306.04637, 2023. — [arXiv:2306.04637](https://arxiv.org/abs/2306.04637) [find](https://scholar.google.com/scholar?q=Bai%2C+Y%2E%2C+Chen%2C+F%2E%2C+Wang%2C+H%2E%2C+Xiong%2C+C%2E%2C+and+Mei%2C+S%2E+Transformers+as+statisticians%3A+Provable+in%2Dcontext+learning+with+in%2Dcontext+algorithm+selection%2E+arXiv+preprint+arXiv%3A2306%2E04637%2C+2023%2E+%E2%80%94+%5BarXiv%3A230)
4. Hämäläinen, P., Tavast, M., and Kunnari, A. Evaluating large language models in generating synthetic hci research data: a case study. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems, pp. 1–19, 2023. [find](https://scholar.google.com/scholar?q=Ha%CC%88ma%CC%88la%CC%88inen%2C+P%2E%2C+Tavast%2C+M%2E%2C+and+Kunnari%2C+A%2E+Evaluating+large+language+models+in+generating+synthetic+hci+research+data%3A+a+case+study%2E+In+Proceedings+of+the+2023+CHI+Conference+on+Human+Factors+in+C)
5. Berti, P., Pratelli, L., and Rigo, P. Limit theorems for a class of identically distributed random variables. The Annals of Probability, 32(3), July 2004. ISSN 0091-1798. doi: 10.1214/009117904000000676. — [doi:10.1214/009117904000000676](https://doi.org/10.1214/009117904000000676) [find](https://scholar.google.com/scholar?q=Berti%2C+P%2E%2C+Pratelli%2C+L%2E%2C+and+Rigo%2C+P%2E+Limit+theorems+for+a+class+of+identically+distributed+random+variables%2E+The+Annals+of+Probability%2C+32%283%29%2C+July+2004%2E+ISSN+0091%2D1798%2E+doi%3A+10%2E1214%2F0091179040000006)
6. Bhattacharya, A., Pati, D., and Yang, Y. Bayesian fractional posteriors. Annals of Statistics, 47(1):39–66, 2019. [find](https://scholar.google.com/scholar?q=Bhattacharya%2C+A%2E%2C+Pati%2C+D%2E%2C+and+Yang%2C+Y%2E+Bayesian+fractional+posteriors%2E+Annals+of+Statistics%2C+47%281%29%3A39%E2%80%9366%2C+2019%2E)
7. Harris, C. R., Millman, K. J., van der Walt, S. J., Gommers, R., Virtanen, P., Cournapeau, D., Wieser, E., Taylor, J., Berg, S., Smith, N. J., et al. [Array programming with NumPy](https://md.research.elamur.ai/free/en/papers/harris-2020-array-programming-numpy/). Nature, 585(7825):357–362, 2020.
8. Biewald, L. Experiment tracking with weights and biases, 2020. URL [https://www.wandb.com/](https://www.wandb.com/). Software available from wandb.com. [↗](https://www.wandb.com/)
9. Hunter, J. D. Matplotlib: A 2D graphics environment. Computing in Science & Engineering, 9(3):90–95, 2007. doi: 10.1109/MCSE.2007.55. — [doi:10.1109/mcse.2007.55](https://doi.org/10.1109/mcse.2007.55) [find](https://scholar.google.com/scholar?q=Hunter%2C+J%2E+D%2E+Matplotlib%3A+A+2D+graphics+environment%2E+Computing+in+Science+%26+Engineering%2C+9%283%29%3A90%E2%80%9395%2C+2007%2E+doi%3A+10%2E1109%2FMCSE%2E2007%2E55%2E+%E2%80%94+%5Bdoi%3A10%2E1109%2Fmcse%2E2007%2E55%5D%28https%3A%2F%2Fdoi%2Eorg%2F10%2E1109%2Fmcse%2E2007%2E55%29)
10. Borisov, V., Seßler, K., Leemann, T., Pawelczyk, M., and Kasneci, G. Language models are realistic tabular data generators. arXiv preprint arXiv:2210.06280, 2022. — [arXiv:2210.06280](https://arxiv.org/abs/2210.06280) [find](https://scholar.google.com/scholar?q=Borisov%2C+V%2E%2C+Se%C3%9Fler%2C+K%2E%2C+Leemann%2C+T%2E%2C+Pawelczyk%2C+M%2E%2C+and+Kasneci%2C+G%2E+Language+models+are+realistic+tabular+data+generators%2E+arXiv+preprint+arXiv%3A2210%2E06280%2C+2022%2E+%E2%80%94+%5BarXiv%3A2210%2E06280%5D%28https%3A%2F%2Farxiv%2Eor)
11. Huszár, F. Implicit bayesian inference in large language models. [https://www.inference.vc/implicit-bayesian-inference-in-sequence-models/](https://www.inference.vc/implicit-bayesian-inference-in-sequence-models/), 2022. [↗](https://www.inference.vc/implicit-bayesian-inference-in-sequence-models/)
12. Brown, T., Mann, B., Ryder, N., Subbiah, M., Kaplan, J. D., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. [Language models are few-shot learners](https://md.research.elamur.ai/free/en/papers/brown-2020-language-models-few-shot-learners/). Advances in neural information processing systems, 33: 1877–1901, 2020.
13. Jiang, A. Q., Sablayrolles, A., Mensch, A., Bamford, C., Chaplot, D. S., Casas, D. d. l., Bressand, F., Lengyel, G., Lample, G., Saulnier, L., et al. Mistral 7b. arXiv preprint arXiv:2310.06825, 2023. [find](https://scholar.google.com/scholar?q=Jiang%2C+A%2E+Q%2E%2C+Sablayrolles%2C+A%2E%2C+Mensch%2C+A%2E%2C+Bamford%2C+C%2E%2C+Chaplot%2C+D%2E+S%2E%2C+Casas%2C+D%2E+d%2E+l%2E%2C+Bressand%2C+F%2E%2C+Lengyel%2C+G%2E%2C+Lample%2C+G%2E%2C+Saulnier%2C+L%2E%2C+et+al%2E+Mistral+7b%2E+arXiv+preprint+arXiv%3A2310%2E06825%2C+2023%2E)
14. Jiang, H. A latent space theory for emergent abilities in large language models. arXiv preprint arXiv:2304.09960, 2023. — [arXiv:2304.09960](https://arxiv.org/abs/2304.09960) [find](https://scholar.google.com/scholar?q=Jiang%2C+H%2E+A+latent+space+theory+for+emergent+abilities+in+large+language+models%2E+arXiv+preprint+arXiv%3A2304%2E09960%2C+2023%2E+%E2%80%94+%5BarXiv%3A2304%2E09960%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2304%2E09960%29)
15. De Finetti, B. Funzione caratteristica di un fenomeno aleato-rio. In Atti del Congresso Internazionale dei Matematici: Jin, M., Wang, S., Ma, L., Chu, Z., Zhang, J. Y., Shi, X., Chen, P.-Y., Liang, Y., Li, Y.-F., Pan, S., et al. Time-llm: Time series forecasting by reprogramming large language model… — [Time-LLM: Time Series Forecasting by Reprogramming Large Language Models](https://md.research.elamur.ai/free/en/papers/jin-2023-time-llm-time-series-forecasting/)
16. Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Prettenhofer, P., Weiss, R., Dubourg, V., Vanderplas, J., Passos, A., Cournapeau, D., Brucher, M., Perrot, M., and Duchesnay, E. Scikit-learn: Machine learning in Python. Journal of Machine Learning Researc… [find](https://scholar.google.com/scholar?q=Pedregosa%2C+F%2E%2C+Varoquaux%2C+G%2E%2C+Gramfort%2C+A%2E%2C+Michel%2C+V%2E%2C+Thirion%2C+B%2E%2C+Grisel%2C+O%2E%2C+Blondel%2C+M%2E%2C+Prettenhofer%2C+P%2E%2C+Weiss%2C+R%2E%2C+Dubourg%2C+V%2E%2C+Vanderplas%2C+J%2E%2C+Passos%2C+A%2E%2C+Cournapeau%2C+D%2E%2C+Brucher%2C+M%2E%2C+Perrot%2C)
17. Kalai, A. T. and Vempala, S. S. Calibrated language models must hallucinate. arXiv preprint arXiv:2311.14648, 2023. — [arXiv:2311.14648](https://arxiv.org/abs/2311.14648) [find](https://scholar.google.com/scholar?q=Kalai%2C+A%2E+T%2E+and+Vempala%2C+S%2E+S%2E+Calibrated+language+models+must+hallucinate%2E+arXiv+preprint+arXiv%3A2311%2E14648%2C+2023%2E+%E2%80%94+%5BarXiv%3A2311%2E14648%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2311%2E14648%29)
18. Raventós, A., Paul, M., Chen, F., and Ganguli, S. Pretraining task diversity and the emergence of non-bayesian in-context learning for regression. arXiv preprint arXiv:2306.15063, 2023. — [arXiv:2306.15063](https://arxiv.org/abs/2306.15063) [find](https://scholar.google.com/scholar?q=Ravento%CC%81s%2C+A%2E%2C+Paul%2C+M%2E%2C+Chen%2C+F%2E%2C+and+Ganguli%2C+S%2E+Pretraining+task+diversity+and+the+emergence+of+non%2Dbayesian+in%2Dcontext+learning+for+regression%2E+arXiv+preprint+arXiv%3A2306%2E15063%2C+2023%2E+%E2%80%94+%5BarXiv%3A2306)
19. Kallenberg, O. Foundations of modern probability, volume 2. Springer, 1997. [find](https://scholar.google.com/scholar?q=Kallenberg%2C+O%2E+Foundations+of+modern+probability%2C+volume+2%2E+Springer%2C+1997%2E)
20. Li, Z., Zhu, H., Lu, Z., and Yin, M. [Synthetic data generation with large language models for text classification: Potential and limitations](https://md.research.elamur.ai/free/en/papers/li-2023-synthetic-data-generation-large-language/). arXiv preprint arXiv:2310.07849, 2023.
21. Shumailov, I., Shumaylov, Z., Zhao, Y., Gal, Y., Papernot, N., and Anderson, R. Model dementia: Generated data makes models forget. arXiv e-prints, pp. arXiv–2305, 2023. [find](https://scholar.google.com/scholar?q=Shumailov%2C+I%2E%2C+Shumaylov%2C+Z%2E%2C+Zhao%2C+Y%2E%2C+Gal%2C+Y%2E%2C+Papernot%2C+N%2E%2C+and+Anderson%2C+R%2E+Model+dementia%3A+Generated+data+makes+models+forget%2E+arXiv+e%2Dprints%2C+pp%2E+arXiv%E2%80%932305%2C+2023%2E)
22. Lu, Y., Bartolo, M., Moore, A., Riedel, S., and Stenetorp, P. [Fantastically ordered prompts and where to find them: Overcoming few-shot prompt order sensitivity](https://md.research.elamur.ai/free/en/papers/lu-2022-fantastically-ordered-prompts-where-find/). arXiv preprint arXiv:2104.08786, 2021.
23. Singh, A. K., Chan, S. C., Moskovitz, T., Grant, E., Saxe, A. M., and Hill, F. The transient nature of emergent in-context learning in transformers. arXiv preprint arXiv:2311.08360, 2023. — [arXiv:2311.08360](https://arxiv.org/abs/2311.08360) [find](https://scholar.google.com/scholar?q=Singh%2C+A%2E+K%2E%2C+Chan%2C+S%2E+C%2E%2C+Moskovitz%2C+T%2E%2C+Grant%2C+E%2E%2C+Saxe%2C+A%2E+M%2E%2C+and+Hill%2C+F%2E+The+transient+nature+of+emergent+in%2Dcontext+learning+in+transformers%2E+arXiv+preprint+arXiv%3A2311%2E08360%2C+2023%2E+%E2%80%94+%5BarXiv%3A231)
24. Manikandan, H., Jiang, Y., and Kolter, J. Z. Language models are weak learners, June 2023. URL http://arxiv.org/abs/2306.14101. arXiv:2306.14101 [cs]. — [arXiv:2306.14101](https://arxiv.org/abs/2306.14101) [↗](http://arxiv.org/abs/2306.14101)
25. Tang, R., Han, X., Jiang, X., and Hu, X. Does synthetic data generation of llms help clinical text mining? arXiv preprint arXiv:2303.04360, 2023. — [arXiv:2303.04360](https://arxiv.org/abs/2303.04360) [find](https://scholar.google.com/scholar?q=Tang%2C+R%2E%2C+Han%2C+X%2E%2C+Jiang%2C+X%2E%2C+and+Hu%2C+X%2E+Does+synthetic+data+generation+of+llms+help+clinical+text+mining%3F+arXiv+preprint+arXiv%3A2303%2E04360%2C+2023%2E+%E2%80%94+%5BarXiv%3A2303%2E04360%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2303%2E04360%29)
26. Mei, Y., Song, S., Fang, C., Yang, H., Fang, J., and Long, J. Capturing semantics for imputation with pre-trained language models. In 2021 IEEE 37th International Conference on Data Engineering (ICDE), pp. 61–72. [find](https://scholar.google.com/scholar?q=Mei%2C+Y%2E%2C+Song%2C+S%2E%2C+Fang%2C+C%2E%2C+Yang%2C+H%2E%2C+Fang%2C+J%2E%2C+and+Long%2C+J%2E+Capturing+semantics+for+imputation+with+pre%2Dtrained+language+models%2E+In+2021+IEEE+37th+International+Conference+on+Data+Engineering+%28ICDE%29)
27. Touvron, H., Martin, L., Stone, K., Albert, P., Almahairi, A., Babaei, Y., Bashlykov, N., Batra, S., Bhargava, P., Bhosale, S., et al. [Llama 2: Open foundation and fine-tuned chat models](https://md.research.elamur.ai/free/en/papers/touvron-2023-llama-2-open-foundation-fine/). arXiv preprint arXiv:2307.09288, 2023.
28. Min, S., Lewis, M., Zettlemoyer, L., and Hajishirzi, H. Metaicl: Learning to learn in context. arXiv preprint arXiv:2110.15943, 2021. tqdm contributors. Imageio. [https://github.com/tqdm/tqdm](https://github.com/tqdm/tqdm), 2022. [↗](https://github.com/tqdm/tqdm)
29. OpenAI. [Gpt-4 technical report](https://md.research.elamur.ai/free/en/papers/openai-2023-gpt-4-technical-report/), 2023.
30. Van der Vaart, A. W. Asymptotic statistics, volume 3. Cambridge university press, 2000. [find](https://scholar.google.com/scholar?q=Van+der+Vaart%2C+A%2E+W%2E+Asymptotic+statistics%2C+volume+3%2E+Cambridge+university+press%2C+2000%2E)
31. Ouyang, L., Wu, J., Jiang, X., Almeida, D., Wainwright, C., Mishkin, P., Zhang, C., Agarwal, S., Slama, K., Ray, A., et al. [Training language models to follow instructions with human feedback](https://md.research.elamur.ai/free/en/papers/ouyang-2022-training-language-models-follow-instructions/). Advances in neural information processing systems, 35:27730–27744, 2022.
32. Van Rossum, G. The Python Library Reference, release 3.8.2. Python Software Foundation, 2020. [find](https://scholar.google.com/scholar?q=Van+Rossum%2C+G%2E+The+Python+Library+Reference%2C+release+3%2E8%2E2%2E+Python+Software+Foundation%2C+2020%2E)
33. Veselovsky, V., Ribeiro, M. H., Arora, A., Josifoski, M., Anderson, A., and West, R. Generating faithful synthetic data with large language models: A case study in computational social science. arXiv preprint arXiv:2305.15041, 2023. — [arXiv:2305.15041](https://arxiv.org/abs/2305.15041) [find](https://scholar.google.com/scholar?q=Veselovsky%2C+V%2E%2C+Ribeiro%2C+M%2E+H%2E%2C+Arora%2C+A%2E%2C+Josifoski%2C+M%2E%2C+Anderson%2C+A%2E%2C+and+West%2C+R%2E+Generating+faithful+synthetic+data+with+large+language+models%3A+A+case+study+in+computational+social+science%2E+arXiv+)
34. Panwar, M., Ahuja, K., and Goyal, N. In-context learning through the bayesian prism. In The Twelfth International Conference on Learning Representations, 2024. URL [https://openreview.net/forum?](https://openreview.net/forum?) id=HX5ujdsSon. [↗](https://openreview.net/forum?)
35. Wang, X., Zhu, W., Saxon, M., Steyvers, M., and Wang, W. Y. [Large language models are latent variable models: Explaining and finding good demonstrations for in-context learning](https://md.research.elamur.ai/free/en/papers/wang-2023-large-language-models-latent-variable/). In Thirty-seventh Conference on Neural Information Processing Systems, 2023.
36. Paszke, A., Gross, S., Massa, F., Lerer, A., Bradbury, J., Chanan, G., Killeen, T., Lin, Z., Gimelshein, N., Antiga, L., Desmaison, A., Kopf, A., Yang, E., DeVito, Z., Raison, M., Tejani, A., Chilamkurthy, S., Steiner, B., Fang, L., Bai, J., and Chintala, S. PyTorch: An imperative style, high-perfor… — [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://md.research.elamur.ai/free/en/papers/paszke-2019-pytorch-imperative-style-high-performance/)
37. Wei, J., Wang, X., Schuurmans, D., Bosma, M., Xia, F., Chi, E., Le, Q. V., Zhou, D., et al. [Chain-of-thought prompting elicits reasoning in large language models](https://md.research.elamur.ai/free/en/papers/wei-2022-chain-thought-prompting-elicits-reasoning/). Advances in Neural Information Processing Systems, 35: 24824–24837, 2022.
38. Wes McKinney. [Data Structures for Statistical Computing in Python](https://md.research.elamur.ai/free/en/papers/mckinney-2010-data-structures-statistical-computing-python/). In Stéfan van der Walt and Jarrod Millman (eds.), Proceedings of the 9th Python in Science Conference, pp. 56 – 61, 2010. doi: 10.25080/Majora-92bf1922-00a.
39. Wolf, T., Debut, L., Sanh, V., Chaumond, J., Delangue, C., Moi, A., Cistac, P., Rault, T., Louf, R., Funtowicz, M., Davison, J., Shleifer, S., von Platen, P., Ma, C., Jernite, Y., Plu, J., Xu, C., Scao, T. L., Gugger, S., Drame, M., Lhoest, Q., and Rush, A. M. Transformers: State-ofthe-art natural l… [find](https://scholar.google.com/scholar?q=Wolf%2C+T%2E%2C+Debut%2C+L%2E%2C+Sanh%2C+V%2E%2C+Chaumond%2C+J%2E%2C+Delangue%2C+C%2E%2C+Moi%2C+A%2E%2C+Cistac%2C+P%2E%2C+Rault%2C+T%2E%2C+Louf%2C+R%2E%2C+Funtowicz%2C+M%2E%2C+Davison%2C+J%2E%2C+Shleifer%2C+S%2E%2C+von+Platen%2C+P%2E%2C+Ma%2C+C%2E%2C+Jernite%2C+Y%2E%2C+Plu%2C+J%2E%2C+Xu%2C+C%2E%2C+Sca)
40. Xiao, Y., Liang, P. P., Bhatt, U., Neiswanger, W., Salakhutdinov, R., and Morency, L.-P. Uncertainty quantification with pre-trained language models: A large-scale empirical analysis. arXiv preprint arXiv:2210.04714, 2022. [find](https://scholar.google.com/scholar?q=Xiao%2C+Y%2E%2C+Liang%2C+P%2E+P%2E%2C+Bhatt%2C+U%2E%2C+Neiswanger%2C+W%2E%2C+Salakhutdinov%2C+R%2E%2C+and+Morency%2C+L%2E%2DP%2E+Uncertainty+quantification+with+pre%2Dtrained+language+models%3A+A+large%2Dscale+empirical+analysis%2E+arXiv+preprint+a)
41. Xie, S. M., Raghunathan, A., Liang, P., and Ma, T. [An explanation of in-context learning as implicit bayesian inference](https://md.research.elamur.ai/free/en/papers/xie-2021-explanation-context-learning-implicit-bayesian/). arXiv preprint arXiv:2111.02080, 2021.
42. Yan, J., Zheng, B., Xu, H., Zhu, Y., Chen, D., Sun, J., Wu, J., and Chen, J. Making pre-trained language models great on tabular prediction. In International Conference on Learning Representations, 2024. [find](https://scholar.google.com/scholar?q=Yan%2C+J%2E%2C+Zheng%2C+B%2E%2C+Xu%2C+H%2E%2C+Zhu%2C+Y%2E%2C+Chen%2C+D%2E%2C+Sun%2C+J%2E%2C+Wu%2C+J%2E%2C+and+Chen%2C+J%2E+Making+pre%2Dtrained+language+models+great+on+tabular+prediction%2E+In+International+Conference+on+Learning+Representations%2C+20)
43. Ye, N., Yang, H., Siah, A., and Namkoong, H. Pre-training and in-context learning IS bayesian inference a la de finetti. In ICLR 2024 Workshop on Mathematical and Empirical Understanding of Foundation Models, 2024. URL [https://openreview.net/forum?](https://openreview.net/forum?) id=ttupfosvgx. [↗](https://openreview.net/forum?)
44. Zellner, A. Optimal information processing and bayes’s theorem. The American Statistician, 42(4):278–280, 1988. [find](https://scholar.google.com/scholar?q=Zellner%2C+A%2E+Optimal+information+processing+and+bayes%E2%80%99s+theorem%2E+The+American+Statistician%2C+42%284%29%3A278%E2%80%93280%2C+1988%2E)
45. Zhang, L., McCoy, R. T., Sumers, T. R., Zhu, J.-Q., and Griffiths, T. L. Deep de finetti: Recovering topic distributions from large language models. arXiv preprint arXiv:2312.14226, 2023a. — [arXiv:2312.14226](https://arxiv.org/abs/2312.14226) [find](https://scholar.google.com/scholar?q=Zhang%2C+L%2E%2C+McCoy%2C+R%2E+T%2E%2C+Sumers%2C+T%2E+R%2E%2C+Zhu%2C+J%2E%2DQ%2E%2C+and+Griffiths%2C+T%2E+L%2E+Deep+de+finetti%3A+Recovering+topic+distributions+from+large+language+models%2E+arXiv+preprint+arXiv%3A2312%2E14226%2C+2023a%2E+%E2%80%94+%5BarXiv%3A23)
46. Zhang, Y., Zhang, F., Yang, Z., and Wang, Z. What and how does in-context learning learn? bayesian model averaging, parameterization, and generalization. arXiv preprint arXiv:2305.19420, 2023b. — [arXiv:2305.19420](https://arxiv.org/abs/2305.19420) [find](https://scholar.google.com/scholar?q=Zhang%2C+Y%2E%2C+Zhang%2C+F%2E%2C+Yang%2C+Z%2E%2C+and+Wang%2C+Z%2E+What+and+how+does+in%2Dcontext+learning+learn%3F+bayesian+model+averaging%2C+parameterization%2C+and+generalization%2E+arXiv+preprint+arXiv%3A2305%2E19420%2C+2023b%2E+%E2%80%94+%5BarX)
47. Zhao, W. X., Zhou, K., Li, J., Tang, T., Wang, X., Hou, Y., Min, Y., Zhang, B., Zhang, J., Dong, Z., et al. [A survey of large language models](https://md.research.elamur.ai/free/en/papers/zhao-2023-survey-large-language-models/). arXiv preprint arXiv:2303.18223, 2023.
48. Zhao, Z., Wallace, E., Feng, S., Klein, D., and Singh, S. [Calibrate before use: Improving few-shot performance of language models](https://md.research.elamur.ai/free/en/papers/zhao-2021-calibrate-before-use-improving-few/). In International Conference on Machine Learning, pp. 12697–12706. PMLR, 2021.

*Updated 3 November 2026*