---
title: "A Gentle Introduction to Conformal Prediction and"
description: "Modern machine learning models that operate as a \"black box\" are increasingly deployed in… 132 references, cited by 52 papers here."
authors: ["Anastasios N. Angelopoulos", "Stephen Bates"]
year: 2021
venue: "arXiv:2107.07511"
source: "https://arxiv.org/abs/2107.07511"
doi: "10.48550/arxiv.2107.07511"
arxiv: "2107.07511"
licence: "arXiv — non-exclusive licence to distribute"
licence_url: "http://arxiv.org/licenses/nonexclusive-distrib/1.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/angelopoulos-2021-gentle-introduction-conformal-prediction-distribution/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/angelopoulos-2021-gentle-introduction-conformal-prediction-distribution/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2021](https://md.research.elamur.ai/free/en/papers/2021/page/5/) / A Gentle Introduction to Conformal Prediction and Distribution-Free Uncertainty Quantification

# A Gentle Introduction to Conformal Prediction and Distribution-Free Uncertainty Quantification

**Authors:** [Anastasios N. Angelopoulos](https://md.research.elamur.ai/free/en/authors/anastasios-n-angelopoulos/) and Stephen Bates · **Year:** 2021 · **Venue:** arXiv:2107.07511 · **Licence:** [arXiv — non-exclusive licence to distribute](http://arxiv.org/licenses/nonexclusive-distrib/1.0/)

**Links:** [Publisher's page](https://arxiv.org/abs/2107.07511) · [PDF](https://arxiv.org/pdf/2107.07511) · [DOI 10.48550/arxiv.2107.07511](https://doi.org/10.48550/arxiv.2107.07511) · [arXiv 2107.07511](https://arxiv.org/abs/2107.07511) · [OpenAlex](https://openalex.org/W3180462067)

**Citations:** 27 citations (OpenAlex, 2026-10-14) · [52](https://md.research.elamur.ai/free/en/papers/angelopoulos-2021-gentle-introduction-conformal-prediction-distribution/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary

Modern machine learning models that operate as a "black box" are increasingly deployed in high-stakes domains such as medicine. This calls for a reliable assessment of how confident a model is. The authors write a tutorial-style overview of conformal prediction. This approach turns any heuristic uncertainty measure of an already trained model into a prediction set or interval with a rigorous statistical guarantee. All it requires is a small held-out calibration set: a conformal score is computed on it, and its adjusted quantile is taken. The guarantee does not rely on assumptions about the data distribution or the quality of the model. It only requires the calibration and test examples to be exchangeable (in particular, i.i.d.). The tutorial combines theory with examples for classification, regression, segmentation, time series, outlier detection, and distribution shift. Each example comes with Python code and a Jupyter notebook on real data. The work matters because it makes rigorous uncertainty quantification accessible to practitioners in NLP, computer vision, and other fields.

## Key points

- **Coverage guarantee:** the prediction set contains the correct answer with probability at least 1 − α. This probability is bounded above by 1 − α + 1/(n+1). For example, with α = 0.1 the coverage is about 90%. The main variant of the method the authors consider is split conformal prediction.
- **Simplicity:** calibration takes only a few lines of code and about 500 calibration examples. In the basic classification example on Imagenet, the score equals one minus the softmax probability of the correct class.
- **Extensions:** Adaptive Prediction Sets, conformalized quantile regression, conformal risk control, class- and group-conditional coverage, covariate shift, distribution drift, full conformal, CV+ and Jackknife+, as well as the Learn then Test approach for controlling general risks.
- **Worked examples:** multilabel classification, tumor segmentation, weather forecasting under temporal shift, detecting toxic comments as outliers, selective classification.
- **Applications and resources:** The Washington Post used the method to estimate not-yet-counted votes in the 2020 US elections. The authors also mention the scikit-learn-compatible MAPIE library and the Awesome Conformal Prediction repository.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

📚 Shelves (2): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 18,614 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 17,965 works

## Cite

Cite as: Anastasios N. Angelopoulos and Stephen Bates. 2021. A Gentle Introduction to Conformal Prediction and Distribution-Free Uncertainty Quantification. arXiv:2107.07511.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@misc{angelopoulos2021gentle,
  title = {{A Gentle Introduction to Conformal Prediction and Distribution-Free Uncertainty Quantification}},
  author = {Anastasios N. Angelopoulos and Stephen Bates},
  year = {2021},
  doi = {10.48550/arxiv.2107.07511},
  eprint = {2107.07511},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2107.07511}
}
```

- [← previous Findings of the WMT 2021 Shared Task on Quality Estimation](https://md.research.elamur.ai/free/en/papers/specia-2021-findings-wmt-2021-shared-task/)
- [next → AraGPT2: Pre-Trained Transformer for Arabic Language Generation](https://md.research.elamur.ai/free/en/papers/antoun-2021-aragpt2-pre-trained-transformer-arabic/)
- [↑ back to papers of 2021](https://md.research.elamur.ai/free/en/papers/2021/page/5/)

## References

132 references, 2 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. V. Vovk, A. Gammerman, and G. Shafer, Algorithmic Learning in a Random World. Springer, 2005. [find](https://scholar.google.com/scholar?q=V%2E+Vovk%2C+A%2E+Gammerman%2C+and+G%2E+Shafer%2C+Algorithmic+Learning+in+a+Random+World%2E+Springer%2C+2005%2E)
2. H. Papadopoulos, K. Proedrou, V. Vovk, and A. Gammerman, “Inductive confidence machines for regression,” in Machine Learning: European Conference on Machine Learning, 2002, pp. 345–356. [find](https://scholar.google.com/scholar?q=H%2E+Papadopoulos%2C+K%2E+Proedrou%2C+V%2E+Vovk%2C+and+A%2E+Gammerman%2C+%E2%80%9CInductive+confidence+machines+for+regression%2C%E2%80%9D+in+Machine+Learning%3A+European+Conference+on+Machine+Learning%2C+2002%2C+pp%2E+345%E2%80%93356%2E)
3. J. Lei and L. Wasserman, “Distribution-free prediction bands for non-parametric regression,” Journal of the Royal Statistical Society: Series B: Statistical Methodology, pp. 71–96, 2014. [find](https://scholar.google.com/scholar?q=J%2E+Lei+and+L%2E+Wasserman%2C+%E2%80%9CDistribution%2Dfree+prediction+bands+for+non%2Dparametric+regression%2C%E2%80%9D+Journal+of+the+Royal+Statistical+Society%3A+Series+B%3A+Statistical+Methodology%2C+pp%2E+71%E2%80%9396%2C+2014%2E)
4. A. N. Angelopoulos, S. Bates, J. Malik, and M. I. Jordan, “Uncertainty sets for image classifiers using conformal prediction,” in International Conference on Learning Representations, 2021. [find](https://scholar.google.com/scholar?q=A%2E+N%2E+Angelopoulos%2C+S%2E+Bates%2C+J%2E+Malik%2C+and+M%2E+I%2E+Jordan%2C+%E2%80%9CUncertainty+sets+for+image+classifiers+using+conformal+prediction%2C%E2%80%9D+in+International+Conference+on+Learning+Representations%2C+2021%2E)
5. V. Vovk, A. Gammerman, and C. Saunders, “Machine-learning applications of algorithmic randomness,” in International Conference on Machine Learning, 1999, pp. 444–453. [find](https://scholar.google.com/scholar?q=V%2E+Vovk%2C+A%2E+Gammerman%2C+and+C%2E+Saunders%2C+%E2%80%9CMachine%2Dlearning+applications+of+algorithmic+randomness%2C%E2%80%9D+in+International+Conference+on+Machine+Learning%2C+1999%2C+pp%2E+444%E2%80%93453%2E)
6. M. Sadinle, J. Lei, and L. Wasserman, “Least ambiguous set-valued classifiers with bounded error levels,” Journal of the American Statistical Association, vol. 114, pp. 223–234, 2019. [find](https://scholar.google.com/scholar?q=M%2E+Sadinle%2C+J%2E+Lei%2C+and+L%2E+Wasserman%2C+%E2%80%9CLeast+ambiguous+set%2Dvalued+classifiers+with+bounded+error+levels%2C%E2%80%9D+Journal+of+the+American+Statistical+Association%2C+vol%2E+114%2C+pp%2E+223%E2%80%93234%2C+2019%2E)
7. Y. Romano, M. Sesia, and E. J. Candès, “Classification with valid and adaptive coverage,” arXiv:2006.02544, 2020. — [arXiv:2006.02544](https://arxiv.org/abs/2006.02544) [find](https://scholar.google.com/scholar?q=Y%2E+Romano%2C+M%2E+Sesia%2C+and+E%2E+J%2E+Cande%CC%80s%2C+%E2%80%9CClassification+with+valid+and+adaptive+coverage%2C%E2%80%9D+arXiv%3A2006%2E02544%2C+2020%2E+%E2%80%94+%5BarXiv%3A2006%2E02544%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2006%2E02544%29)
8. Y. Romano, E. Patterson, and E. Candès, “Conformalized quantile regression,” in Advances in Neural Information Processing Systems, vol. 32, 2019, pp. 3543–3553. [find](https://scholar.google.com/scholar?q=Y%2E+Romano%2C+E%2E+Patterson%2C+and+E%2E+Cande%CC%80s%2C+%E2%80%9CConformalized+quantile+regression%2C%E2%80%9D+in+Advances+in+Neural+Information+Processing+Systems%2C+vol%2E+32%2C+2019%2C+pp%2E+3543%E2%80%933553%2E)
9. R. Koenker and G. Bassett Jr, “Regression quantiles,” Econometrica: Journal of the Econometric Society, vol. 46, no. 1, pp. 33–50, 1978. [find](https://scholar.google.com/scholar?q=R%2E+Koenker+and+G%2E+Bassett+Jr%2C+%E2%80%9CRegression+quantiles%2C%E2%80%9D+Econometrica%3A+Journal+of+the+Econometric+Society%2C+vol%2E+46%2C+no%2E+1%2C+pp%2E+33%E2%80%9350%2C+1978%2E)
10. A. N. Angelopoulos, A. P. Kohli, S. Bates, M. I. Jordan, J. Malik, T. Alshaabi, S. Upadhyayula, and Y. Romano, “Image-to-image regression with distribution-free uncertainty quantification and applications in imaging,” arXiv preprint arXiv:2202.05265, 2022. — [arXiv:2202.05265](https://arxiv.org/abs/2202.05265) [find](https://scholar.google.com/scholar?q=A%2E+N%2E+Angelopoulos%2C+A%2E+P%2E+Kohli%2C+S%2E+Bates%2C+M%2E+I%2E+Jordan%2C+J%2E+Malik%2C+T%2E+Alshaabi%2C+S%2E+Upadhyayula%2C+and+Y%2E+Romano%2C+%E2%80%9CImage%2Dto%2Dimage+regression+with+distribution%2Dfree+uncertainty+quantification+and+applicat)
11. P. Hoff, “Bayes-optimal prediction with frequentist coverage control,” arXiv:2105.14045, 2021. — [arXiv:2105.14045](https://arxiv.org/abs/2105.14045) [find](https://scholar.google.com/scholar?q=P%2E+Hoff%2C+%E2%80%9CBayes%2Doptimal+prediction+with+frequentist+coverage+control%2C%E2%80%9D+arXiv%3A2105%2E14045%2C+2021%2E+%E2%80%94+%5BarXiv%3A2105%2E14045%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2105%2E14045%29)
12. L. Wasserman, “Frasian inference,” Statistical Science, vol. 26, no. 3, pp. 322–325, 2011. [find](https://scholar.google.com/scholar?q=L%2E+Wasserman%2C+%E2%80%9CFrasian+inference%2C%E2%80%9D+Statistical+Science%2C+vol%2E+26%2C+no%2E+3%2C+pp%2E+322%E2%80%93325%2C+2011%2E)
13. T. Melluish, C. Saunders, I. Nouretdinov, and V. Vovk, “Comparing the bayes and typicalness frameworks,” in European Conference on Machine Learning, Springer, 2001, pp. 360–371. [find](https://scholar.google.com/scholar?q=T%2E+Melluish%2C+C%2E+Saunders%2C+I%2E+Nouretdinov%2C+and+V%2E+Vovk%2C+%E2%80%9CComparing+the+bayes+and+typicalness+frameworks%2C%E2%80%9D+in+European+Conference+on+Machine+Learning%2C+Springer%2C+2001%2C+pp%2E+360%E2%80%93371%2E)
14. V. Vovk, “Conditional validity of inductive conformal predictors,” in Proceedings of the Asian Conference on Machine Learning, vol. 25, 2012, pp. 475–490. [find](https://scholar.google.com/scholar?q=V%2E+Vovk%2C+%E2%80%9CConditional+validity+of+inductive+conformal+predictors%2C%E2%80%9D+in+Proceedings+of+the+Asian+Conference+on+Machine+Learning%2C+vol%2E+25%2C+2012%2C+pp%2E+475%E2%80%93490%2E)
15. M. Cauchois, S. Gupta, and J. Duchi, “Knowing what you know: Valid and validated confidence sets in multiclass and multilabel prediction,” arXiv:2004.10181, 2020. — [arXiv:2004.10181](https://arxiv.org/abs/2004.10181) [find](https://scholar.google.com/scholar?q=M%2E+Cauchois%2C+S%2E+Gupta%2C+and+J%2E+Duchi%2C+%E2%80%9CKnowing+what+you+know%3A+Valid+and+validated+confidence+sets+in+multiclass+and+multilabel+prediction%2C%E2%80%9D+arXiv%3A2004%2E10181%2C+2020%2E+%E2%80%94+%5BarXiv%3A2004%2E10181%5D%28https%3A%2F%2Farxiv%2Eor)
16. S. Feldman, S. Bates, and Y. Romano, “Improving conditional coverage via orthogonal quantile regression,” in Advances in Neural Information Processing Systems, 2021. [find](https://scholar.google.com/scholar?q=S%2E+Feldman%2C+S%2E+Bates%2C+and+Y%2E+Romano%2C+%E2%80%9CImproving+conditional+coverage+via+orthogonal+quantile+regression%2C%E2%80%9D+in+Advances+in+Neural+Information+Processing+Systems%2C+2021%2E)
17. A. N. Angelopoulos, S. Bates, A. Fisch, L. Lei, and T. Schuster, “Conformal risk control,” arXiv preprint arXiv:2208.02814, 2022. — [arXiv:2208.02814](https://arxiv.org/abs/2208.02814) [find](https://scholar.google.com/scholar?q=A%2E+N%2E+Angelopoulos%2C+S%2E+Bates%2C+A%2E+Fisch%2C+L%2E+Lei%2C+and+T%2E+Schuster%2C+%E2%80%9CConformal+risk+control%2C%E2%80%9D+arXiv+preprint+arXiv%3A2208%2E02814%2C+2022%2E+%E2%80%94+%5BarXiv%3A2208%2E02814%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2208%2E02814%29)
18. A. N. Angelopoulos, S. Bates, E. J. Candès, M. I. Jordan, and L. Lei, “Learn then test: Calibrating predictive algorithms to achieve risk control,” arXiv:2110.01052, 2021. — [arXiv:2110.01052](https://arxiv.org/abs/2110.01052) [find](https://scholar.google.com/scholar?q=A%2E+N%2E+Angelopoulos%2C+S%2E+Bates%2C+E%2E+J%2E+Cande%CC%80s%2C+M%2E+I%2E+Jordan%2C+and+L%2E+Lei%2C+%E2%80%9CLearn+then+test%3A+Calibrating+predictive+algorithms+to+achieve+risk+control%2C%E2%80%9D+arXiv%3A2110%2E01052%2C+2021%2E+%E2%80%94+%5BarXiv%3A2110%2E01052%5D%28https%3A)
19. M. A. Pimentel, D. A. Clifton, L. Clifton, and L. Tarassenko, “A review of novelty detection,” Signal Processing, vol. 99, pp. 215–249, 2014. [find](https://scholar.google.com/scholar?q=M%2E+A%2E+Pimentel%2C+D%2E+A%2E+Clifton%2C+L%2E+Clifton%2C+and+L%2E+Tarassenko%2C+%E2%80%9CA+review+of+novelty+detection%2C%E2%80%9D+Signal+Processing%2C+vol%2E+99%2C+pp%2E+215%E2%80%93249%2C+2014%2E)
20. R. A. Fisher, “Design of experiments,” British Medical Journal, vol. 1, no. 3923, p. 554, 1936. [find](https://scholar.google.com/scholar?q=R%2E+A%2E+Fisher%2C+%E2%80%9CDesign+of+experiments%2C%E2%80%9D+British+Medical+Journal%2C+vol%2E+1%2C+no%2E+3923%2C+p%2E+554%2C+1936%2E)
21. E. J. Pitman, “Significance tests which may be applied to samples from any populations,” Supplement to the Journal of the Royal Statistical Society, vol. 4, no. 1, pp. 119–130, 1937. [find](https://scholar.google.com/scholar?q=E%2E+J%2E+Pitman%2C+%E2%80%9CSignificance+tests+which+may+be+applied+to+samples+from+any+populations%2C%E2%80%9D+Supplement+to+the+Journal+of+the+Royal+Statistical+Society%2C+vol%2E+4%2C+no%2E+1%2C+pp%2E+119%E2%80%93130%2C+1937%2E)
22. V. Vovk, I. Nouretdinov, and A. Gammerman, “Testing exchangeability on-line,” in Proceedings of the 20th International Conference on Machine Learning (ICML-03), 2003, pp. 768–775. [find](https://scholar.google.com/scholar?q=V%2E+Vovk%2C+I%2E+Nouretdinov%2C+and+A%2E+Gammerman%2C+%E2%80%9CTesting+exchangeability+on%2Dline%2C%E2%80%9D+in+Proceedings+of+the+20th+International+Conference+on+Machine+Learning+%28ICML%2D03%29%2C+2003%2C+pp%2E+768%E2%80%93775%2E)
23. L. Guan and R. Tibshirani, “Prediction and outlier detection in classification problems,” arXiv:1905.04396, 2019. — [arXiv:1905.04396](https://arxiv.org/abs/1905.04396) [find](https://scholar.google.com/scholar?q=L%2E+Guan+and+R%2E+Tibshirani%2C+%E2%80%9CPrediction+and+outlier+detection+in+classification+problems%2C%E2%80%9D+arXiv%3A1905%2E04396%2C+2019%2E+%E2%80%94+%5BarXiv%3A1905%2E04396%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F1905%2E04396%29)
24. S. Bates, E. Candès, L. Lei, Y. Romano, and M. Sesia, “Testing for outliers with conformal p-values,” arXiv:2104.08279, 2021. — [arXiv:2104.08279](https://arxiv.org/abs/2104.08279) [find](https://scholar.google.com/scholar?q=S%2E+Bates%2C+E%2E+Cande%CC%80s%2C+L%2E+Lei%2C+Y%2E+Romano%2C+and+M%2E+Sesia%2C+%E2%80%9CTesting+for+outliers+with+conformal+p%2Dvalues%2C%E2%80%9D+arXiv%3A2104%2E08279%2C+2021%2E+%E2%80%94+%5BarXiv%3A2104%2E08279%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2104%2E08279%29)
25. R. J. Tibshirani, R. Foygel Barber, E. Candes, and A. Ramdas, “Conformal prediction under covariate shift,” in Advances in Neural Information Processing Systems 32, 2019, pp. 2530–2540. [find](https://scholar.google.com/scholar?q=R%2E+J%2E+Tibshirani%2C+R%2E+Foygel+Barber%2C+E%2E+Candes%2C+and+A%2E+Ramdas%2C+%E2%80%9CConformal+prediction+under+covariate+shift%2C%E2%80%9D+in+Advances+in+Neural+Information+Processing+Systems+32%2C+2019%2C+pp%2E+2530%E2%80%932540%2E)
26. R. F. Barber, E. J. Candes, A. Ramdas, and R. J. Tibshirani, “Conformal prediction beyond exchangeability,” arXiv:2202.13415, 2022. — [arXiv:2202.13415](https://arxiv.org/abs/2202.13415) [find](https://scholar.google.com/scholar?q=R%2E+F%2E+Barber%2C+E%2E+J%2E+Candes%2C+A%2E+Ramdas%2C+and+R%2E+J%2E+Tibshirani%2C+%E2%80%9CConformal+prediction+beyond+exchangeability%2C%E2%80%9D+arXiv%3A2202%2E13415%2C+2022%2E+%E2%80%94+%5BarXiv%3A2202%2E13415%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2202%2E13415%29)
27. L. Guan, “Conformal prediction with localization,” arXiv:1908.08558, 2020. — [arXiv:1908.08558](https://arxiv.org/abs/1908.08558) [find](https://scholar.google.com/scholar?q=L%2E+Guan%2C+%E2%80%9CConformal+prediction+with+localization%2C%E2%80%9D+arXiv%3A1908%2E08558%2C+2020%2E+%E2%80%94+%5BarXiv%3A1908%2E08558%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F1908%2E08558%29)
28. T.-Y. Lin, M. Maire, S. Belongie, J. Hays, P. Perona, D. Ramanan, P. Dollár, and C. L. Zitnick, “[Microsoft coco: Common objects in context](https://md.research.elamur.ai/free/en/papers/lin-2014-microsoft-coco-common-objects-context/),” in European conference on computer vision, Springer, 2014, pp. 740–755.
29. A. Malinin, N. Band, G. Chesnokov, Y. Gal, M. J. Gales, A. Noskov, A. Ploskonosov, L. Prokhorenkova, I. Provilkov, V. Raina, et al., “Shifts: A dataset of real distributional shift across multiple large-scale tasks,” arXiv preprint arXiv:2107.07455, 2021. — [arXiv:2107.07455](https://arxiv.org/abs/2107.07455) [find](https://scholar.google.com/scholar?q=A%2E+Malinin%2C+N%2E+Band%2C+G%2E+Chesnokov%2C+Y%2E+Gal%2C+M%2E+J%2E+Gales%2C+A%2E+Noskov%2C+A%2E+Ploskonosov%2C+L%2E+Prokhorenkova%2C+I%2E+Provilkov%2C+V%2E+Raina%2C+et+al%2E%2C+%E2%80%9CShifts%3A+A+dataset+of+real+distributional+shift+across+multiple+lar)
30. A. V. Dorogush, V. Ershov, and A. Gulin, “Catboost: Gradient boosting with categorical features support,” arXiv preprint arXiv:1810.11363, 2018. [find](https://scholar.google.com/scholar?q=A%2E+V%2E+Dorogush%2C+V%2E+Ershov%2C+and+A%2E+Gulin%2C+%E2%80%9CCatboost%3A+Gradient+boosting+with+categorical+features+support%2C%E2%80%9D+arXiv+preprint+arXiv%3A1810%2E11363%2C+2018%2E)
31. I. Gibbs and E. Candès, “Adaptive conformal inference under distribution shift,” arXiv:2106.00170, 2021. — [arXiv:2106.00170](https://arxiv.org/abs/2106.00170) [find](https://scholar.google.com/scholar?q=I%2E+Gibbs+and+E%2E+Cande%CC%80s%2C+%E2%80%9CAdaptive+conformal+inference+under+distribution+shift%2C%E2%80%9D+arXiv%3A2106%2E00170%2C+2021%2E+%E2%80%94+%5BarXiv%3A2106%2E00170%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2106%2E00170%29)
32. M. Zaffran, O. Féron, Y. Goude, J. Josse, and A. Dieuleveut, “Adaptive conformal predictions for time series,” in International Conference on Machine Learning, PMLR, 2022, pp. 25 834–25 866. [find](https://scholar.google.com/scholar?q=M%2E+Zaffran%2C+O%2E+Fe%CC%81ron%2C+Y%2E+Goude%2C+J%2E+Josse%2C+and+A%2E+Dieuleveut%2C+%E2%80%9CAdaptive+conformal+predictions+for+time+series%2C%E2%80%9D+in+International+Conference+on+Machine+Learning%2C+PMLR%2C+2022%2C+pp%2E+25+834%E2%80%9325+866%2E)
33. I. Gibbs and E. Candès, “Conformal inference for online prediction with arbitrary distribution shifts,” arXiv preprint arXiv:2208.08401, 2022. — [arXiv:2208.08401](https://arxiv.org/abs/2208.08401) [find](https://scholar.google.com/scholar?q=I%2E+Gibbs+and+E%2E+Cande%CC%80s%2C+%E2%80%9CConformal+inference+for+online+prediction+with+arbitrary+distribution+shifts%2C%E2%80%9D+arXiv+preprint+arXiv%3A2208%2E08401%2C+2022%2E+%E2%80%94+%5BarXiv%3A2208%2E08401%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2208%2E08401%29)
34. C. Xu and Y. Xie, “Conformal prediction interval for dynamic time-series,” in International Conference on Machine Learning, PMLR, 2021, pp. 11 559–11 569. [find](https://scholar.google.com/scholar?q=C%2E+Xu+and+Y%2E+Xie%2C+%E2%80%9CConformal+prediction+interval+for+dynamic+time%2Dseries%2C%E2%80%9D+in+International+Conference+on+Machine+Learning%2C+PMLR%2C+2021%2C+pp%2E+11+559%E2%80%9311+569%2E)
35. L. Hanu and Unitary team, Detoxify, Github. [https://github.com/unitaryai/detoxify](https://github.com/unitaryai/detoxify), 2020. [↗](https://github.com/unitaryai/detoxify)
36. J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, “[Bert: Pre-training of deep bidirectional transformers for language understanding](https://md.research.elamur.ai/free/en/papers/devlin-2019-bert-pre-training-deep-bidirectional/),” arXiv preprint arXiv:1810.04805, 2018.
37. P. W. Koh, S. Sagawa, H. Marklund, S. M. Xie, M. Zhang, A. Balsubramani, W. Hu, M. Yasunaga, R. L. Phillips, I. Gao, et al., “Wilds: A benchmark of in-the-wild distribution shifts,” in International Conference on Machine Learning, PMLR, 2021, pp. 5637–5664. [find](https://scholar.google.com/scholar?q=P%2E+W%2E+Koh%2C+S%2E+Sagawa%2C+H%2E+Marklund%2C+S%2E+M%2E+Xie%2C+M%2E+Zhang%2C+A%2E+Balsubramani%2C+W%2E+Hu%2C+M%2E+Yasunaga%2C+R%2E+L%2E+Phillips%2C+I%2E+Gao%2C+et+al%2E%2C+%E2%80%9CWilds%3A+A+benchmark+of+in%2Dthe%2Dwild+distribution+shifts%2C%E2%80%9D+in+International+C)
38. G. Shafer and V. Vovk, “A tutorial on conformal prediction,” Journal of Machine Learning Research, vol. 9, no. Mar, pp. 371–421, 2008. [find](https://scholar.google.com/scholar?q=G%2E+Shafer+and+V%2E+Vovk%2C+%E2%80%9CA+tutorial+on+conformal+prediction%2C%E2%80%9D+Journal+of+Machine+Learning+Research%2C+vol%2E+9%2C+no%2E+Mar%2C+pp%2E+371%E2%80%93421%2C+2008%2E)
39. E. Ndiaye and I. Takeuchi, “Computing full conformal prediction set with approximate homotopy,” in Advances in Neural Information Processing Systems, 2019. [find](https://scholar.google.com/scholar?q=E%2E+Ndiaye+and+I%2E+Takeuchi%2C+%E2%80%9CComputing+full+conformal+prediction+set+with+approximate+homotopy%2C%E2%80%9D+in+Advances+in+Neural+Information+Processing+Systems%2C+2019%2E)
40. E. Ndiaye and I. Takeuchi, “Root-finding approaches for computing conformal prediction set,” Machine Learning, 2022. [find](https://scholar.google.com/scholar?q=E%2E+Ndiaye+and+I%2E+Takeuchi%2C+%E2%80%9CRoot%2Dfinding+approaches+for+computing+conformal+prediction+set%2C%E2%80%9D+Machine+Learning%2C+2022%2E)
41. V. Vovk, “Cross-conformal predictors,” Annals of Mathematics and Artificial Intelligence, vol. 74, no. 1-2, pp. 9–28, 2015. [find](https://scholar.google.com/scholar?q=V%2E+Vovk%2C+%E2%80%9CCross%2Dconformal+predictors%2C%E2%80%9D+Annals+of+Mathematics+and+Artificial+Intelligence%2C+vol%2E+74%2C+no%2E+1%2D2%2C+pp%2E+9%E2%80%9328%2C+2015%2E)
42. R. F. Barber, E. J. Candes, A. Ramdas, and R. J. Tibshirani, “Predictive inference with the jackknife+,” The Annals of Statistics, vol. 49, no. 1, pp. 486–507, 2021. [find](https://scholar.google.com/scholar?q=R%2E+F%2E+Barber%2C+E%2E+J%2E+Candes%2C+A%2E+Ramdas%2C+and+R%2E+J%2E+Tibshirani%2C+%E2%80%9CPredictive+inference+with+the+jackknife%2B%2C%E2%80%9D+The+Annals+of+Statistics%2C+vol%2E+49%2C+no%2E+1%2C+pp%2E+486%E2%80%93507%2C+2021%2E)
43. E. Chung and J. P. Romano, “Exact and asymptotically robust permutation tests,” The Annals of Statistics, vol. 41, no. 2, pp. 484–507, 2013. [find](https://scholar.google.com/scholar?q=E%2E+Chung+and+J%2E+P%2E+Romano%2C+%E2%80%9CExact+and+asymptotically+robust+permutation+tests%2C%E2%80%9D+The+Annals+of+Statistics%2C+vol%2E+41%2C+no%2E+2%2C+pp%2E+484%E2%80%93507%2C+2013%2E)
44. H. B. Mann and D. R. Whitney, “On a test of whether one of two random variables is stochastically larger than the other,” The Annals of Mathematical Statistics, pp. 50–60, 1947. [find](https://scholar.google.com/scholar?q=H%2E+B%2E+Mann+and+D%2E+R%2E+Whitney%2C+%E2%80%9COn+a+test+of+whether+one+of+two+random+variables+is+stochastically+larger+than+the+other%2C%E2%80%9D+The+Annals+of+Mathematical+Statistics%2C+pp%2E+50%E2%80%9360%2C+1947%2E)
45. E. L. Lehmann, “The power of rank tests,” The Annals of Mathematical Statistics, pp. 23–43, 1953. [find](https://scholar.google.com/scholar?q=E%2E+L%2E+Lehmann%2C+%E2%80%9CThe+power+of+rank+tests%2C%E2%80%9D+The+Annals+of+Mathematical+Statistics%2C+pp%2E+23%E2%80%9343%2C+1953%2E)
46. Z. Sidak, P. K. Sen, and J. Hajek, Theory of rank tests. Elsevier, 1999. [find](https://scholar.google.com/scholar?q=Z%2E+Sidak%2C+P%2E+K%2E+Sen%2C+and+J%2E+Hajek%2C+Theory+of+rank+tests%2E+Elsevier%2C+1999%2E)
47. B. Efron and R. J. Tibshirani, An introduction to the bootstrap. CRC press, 1994. [find](https://scholar.google.com/scholar?q=B%2E+Efron+and+R%2E+J%2E+Tibshirani%2C+An+introduction+to+the+bootstrap%2E+CRC+press%2C+1994%2E)
48. S. Chatterjee and P. Qiu, “Distribution-free cumulative sum control charts using bootstrap-based control limits,” The Annals of Applied Statistics, vol. 3, no. 1, pp. 349–369, 2009. [find](https://scholar.google.com/scholar?q=S%2E+Chatterjee+and+P%2E+Qiu%2C+%E2%80%9CDistribution%2Dfree+cumulative+sum+control+charts+using+bootstrap%2Dbased+control+limits%2C%E2%80%9D+The+Annals+of+Applied+Statistics%2C+vol%2E+3%2C+no%2E+1%2C+pp%2E+349%E2%80%93369%2C+2009%2E)
49. G. T. Fechner, Kollektivmasslehre. Engelmann, 1897. [find](https://scholar.google.com/scholar?q=G%2E+T%2E+Fechner%2C+Kollektivmasslehre%2E+Engelmann%2C+1897%2E)
50. R. von Mises, “Grundlagen der wahrscheinlichkeitsrechnung,” Mathematische Zeitschrift, vol. 5, no. 1, pp. 52–99, 1919. [find](https://scholar.google.com/scholar?q=R%2E+von+Mises%2C+%E2%80%9CGrundlagen+der+wahrscheinlichkeitsrechnung%2C%E2%80%9D+Mathematische+Zeitschrift%2C+vol%2E+5%2C+no%2E+1%2C+pp%2E+52%E2%80%9399%2C+1919%2E)
51. A. Wald, “Die widerspruchfreiheit des kollectivbegriffes der wahrscheinlichkeitsrechnung,” Ergebnisse Eines Mathematischen Kolloquiums, vol. 8, no. 38-72, p. 37, 1937. [find](https://scholar.google.com/scholar?q=A%2E+Wald%2C+%E2%80%9CDie+widerspruchfreiheit+des+kollectivbegriffes+der+wahrscheinlichkeitsrechnung%2C%E2%80%9D+Ergebnisse+Eines+Mathematischen+Kolloquiums%2C+vol%2E+8%2C+no%2E+38%2D72%2C+p%2E+37%2C+1937%2E)
52. A. Church, “On the concept of a random sequence,” Bulletin of the American Mathematical Society, vol. 46, no. 2, pp. 130–135, 1940. [find](https://scholar.google.com/scholar?q=A%2E+Church%2C+%E2%80%9COn+the+concept+of+a+random+sequence%2C%E2%80%9D+Bulletin+of+the+American+Mathematical+Society%2C+vol%2E+46%2C+no%2E+2%2C+pp%2E+130%E2%80%93135%2C+1940%2E)
53. J. Ville, “Etude critique de la notion de collectif,” Bull. Amer. Math. Soc, vol. 45, no. 11, p. 824, 1939. [find](https://scholar.google.com/scholar?q=J%2E+Ville%2C+%E2%80%9CEtude+critique+de+la+notion+de+collectif%2C%E2%80%9D+Bull%2E+Amer%2E+Math%2E+Soc%2C+vol%2E+45%2C+no%2E+11%2C+p%2E+824%2C+1939%2E)
54. G. Shafer and V. Vovk, “The sources of Kolmogorov’s Grundbegriffe,” Statistical Science, vol. 21, no. 1, pp. 70–98, 2006. [find](https://scholar.google.com/scholar?q=G%2E+Shafer+and+V%2E+Vovk%2C+%E2%80%9CThe+sources+of+Kolmogorov%E2%80%99s+Grundbegriffe%2C%E2%80%9D+Statistical+Science%2C+vol%2E+21%2C+no%2E+1%2C+pp%2E+70%E2%80%9398%2C+2006%2E)
55. V. Vovk, “Kolmogorov’s complexity conception of probability,” Synthese Library, pp. 51–70, 2001. [find](https://scholar.google.com/scholar?q=V%2E+Vovk%2C+%E2%80%9CKolmogorov%E2%80%99s+complexity+conception+of+probability%2C%E2%80%9D+Synthese+Library%2C+pp%2E+51%E2%80%9370%2C+2001%2E)
56. C. P. Porter, “Kolmogorov on the role of randomness in probability theory,” Mathematical Structures in Computer Science, vol. 24, no. 3, 2014. [find](https://scholar.google.com/scholar?q=C%2E+P%2E+Porter%2C+%E2%80%9CKolmogorov+on+the+role+of+randomness+in+probability+theory%2C%E2%80%9D+Mathematical+Structures+in+Computer+Science%2C+vol%2E+24%2C+no%2E+3%2C+2014%2E)
57. A. N. Kolmogorov, “Three approaches to the quantitative definition of information,” Problems of Information Transmission, vol. 1, no. 1, pp. 1–7, 1965. [find](https://scholar.google.com/scholar?q=A%2E+N%2E+Kolmogorov%2C+%E2%80%9CThree+approaches+to+the+quantitative+definition+of+information%2C%E2%80%9D+Problems+of+Information+Transmission%2C+vol%2E+1%2C+no%2E+1%2C+pp%2E+1%E2%80%937%2C+1965%2E)
58. A. Kolmogorov, “Logical basis for information theory and probability theory,” IEEE Transactions on Information Theory, vol. 14, no. 5, pp. 662–664, 1968. [find](https://scholar.google.com/scholar?q=A%2E+Kolmogorov%2C+%E2%80%9CLogical+basis+for+information+theory+and+probability+theory%2C%E2%80%9D+IEEE+Transactions+on+Information+Theory%2C+vol%2E+14%2C+no%2E+5%2C+pp%2E+662%E2%80%93664%2C+1968%2E)
59. A. N. Kolmogorov, “Combinatorial foundations of information theory and the calculus of probabilities,” Russian Mathematical Surveys, vol. 38, no. 4, pp. 29–40, 1983. [find](https://scholar.google.com/scholar?q=A%2E+N%2E+Kolmogorov%2C+%E2%80%9CCombinatorial+foundations+of+information+theory+and+the+calculus+of+probabilities%2C%E2%80%9D+Russian+Mathematical+Surveys%2C+vol%2E+38%2C+no%2E+4%2C+pp%2E+29%E2%80%9340%2C+1983%2E)
60. V. G. Vovk, “On the concept of the Bernoulli property,” Russian Mathematical Surveys, vol. 41, no. 1, p. 247, 1986. [find](https://scholar.google.com/scholar?q=V%2E+G%2E+Vovk%2C+%E2%80%9COn+the+concept+of+the+Bernoulli+property%2C%E2%80%9D+Russian+Mathematical+Surveys%2C+vol%2E+41%2C+no%2E+1%2C+p%2E+247%2C+1986%2E)
61. V. Vovk, “Testing randomness online,” Statistical Science, vol. 36, no. 4, pp. 595–611, 2021. [find](https://scholar.google.com/scholar?q=V%2E+Vovk%2C+%E2%80%9CTesting+randomness+online%2C%E2%80%9D+Statistical+Science%2C+vol%2E+36%2C+no%2E+4%2C+pp%2E+595%E2%80%93611%2C+2021%2E)
62. F. Mota, S. Aaronson, L. Antunes, and A. Souto, “Sophistication as randomness deficiency,” in International Workshop on Descriptional Complexity of Formal Systems, Springer, 2013, pp. 172– 181. [find](https://scholar.google.com/scholar?q=F%2E+Mota%2C+S%2E+Aaronson%2C+L%2E+Antunes%2C+and+A%2E+Souto%2C+%E2%80%9CSophistication+as+randomness+deficiency%2C%E2%80%9D+in+International+Workshop+on+Descriptional+Complexity+of+Formal+Systems%2C+Springer%2C+2013%2C+pp%2E+172%E2%80%93+181%2E)
63. S. S. Wilks, “Determination of sample sizes for setting tolerance limits,” Annals of Mathematical Statistics, vol. 12, no. 1, pp. 91–96, 1941. [find](https://scholar.google.com/scholar?q=S%2E+S%2E+Wilks%2C+%E2%80%9CDetermination+of+sample+sizes+for+setting+tolerance+limits%2C%E2%80%9D+Annals+of+Mathematical+Statistics%2C+vol%2E+12%2C+no%2E+1%2C+pp%2E+91%E2%80%9396%2C+1941%2E)
64. ——, “Statistical prediction with special reference to the problem of tolerance limits,” Annals of Mathematical Statistics, vol. 13, no. 4, pp. 400–409, 1942. [find](https://scholar.google.com/scholar?q=%E2%80%94%E2%80%94%2C+%E2%80%9CStatistical+prediction+with+special+reference+to+the+problem+of+tolerance+limits%2C%E2%80%9D+Annals+of+Mathematical+Statistics%2C+vol%2E+13%2C+no%2E+4%2C+pp%2E+400%E2%80%93409%2C+1942%2E)
65. A. Wald, “An extension of Wilks’ method for setting tolerance limits,” Annals of Mathematical Statistics, vol. 14, no. 1, pp. 45–55, 1943. [find](https://scholar.google.com/scholar?q=A%2E+Wald%2C+%E2%80%9CAn+extension+of+Wilks%E2%80%99+method+for+setting+tolerance+limits%2C%E2%80%9D+Annals+of+Mathematical+Statistics%2C+vol%2E+14%2C+no%2E+1%2C+pp%2E+45%E2%80%9355%2C+1943%2E)
66. J. W. Tukey, “Non-parametric estimation II. Statistically equivalent blocks and tolerance regions–the continuous case,” Annals of Mathematical Statistics, vol. 18, no. 4, pp. 529–539, 1947. [find](https://scholar.google.com/scholar?q=J%2E+W%2E+Tukey%2C+%E2%80%9CNon%2Dparametric+estimation+II%2E+Statistically+equivalent+blocks+and+tolerance+regions%E2%80%93the+continuous+case%2C%E2%80%9D+Annals+of+Mathematical+Statistics%2C+vol%2E+18%2C+no%2E+4%2C+pp%2E+529%E2%80%93539%2C+1947%2E)
67. P. Diaconis and D. Freedman, “Finite exchangeable sequences,” The Annals of Probability, pp. 745– 764, 1980. [find](https://scholar.google.com/scholar?q=P%2E+Diaconis+and+D%2E+Freedman%2C+%E2%80%9CFinite+exchangeable+sequences%2C%E2%80%9D+The+Annals+of+Probability%2C+pp%2E+745%E2%80%93+764%2C+1980%2E)
68. D. J. Aldous, “Exchangeability and related topics,” in École d’Été de Probabilités de Saint-Flour XIII—1983, 1985, pp. 1–198. [find](https://scholar.google.com/scholar?q=D%2E+J%2E+Aldous%2C+%E2%80%9CExchangeability+and+related+topics%2C%E2%80%9D+in+E%CC%81cole+d%E2%80%99E%CC%81te%CC%81+de+Probabilite%CC%81s+de+Saint%2DFlour+XIII%E2%80%941983%2C+1985%2C+pp%2E+1%E2%80%93198%2E)
69. B. De Finetti, “Funzione caratteristica di un fenomeno aleatorio,” in Atti del Congresso Internazionale dei Matematici: Bologna del 3 al 10 de Settembre di 1928, 1929, pp. 179–190. [find](https://scholar.google.com/scholar?q=B%2E+De+Finetti%2C+%E2%80%9CFunzione+caratteristica+di+un+fenomeno+aleatorio%2C%E2%80%9D+in+Atti+del+Congresso+Internazionale+dei+Matematici%3A+Bologna+del+3+al+10+de+Settembre+di+1928%2C+1929%2C+pp%2E+179%E2%80%93190%2E)
70. D. A. Freedman, “Bernard Friedman’s urn,” The Annals of Mathematical Statistics, pp. 956–970, 1965. [find](https://scholar.google.com/scholar?q=D%2E+A%2E+Freedman%2C+%E2%80%9CBernard+Friedman%E2%80%99s+urn%2C%E2%80%9D+The+Annals+of+Mathematical+Statistics%2C+pp%2E+956%E2%80%93970%2C+1965%2E)
71. E. Hewitt and L. J. Savage, “Symmetric measures on Cartesian products,” Transactions of the American Mathematical Society, vol. 80, no. 2, pp. 470–501, 1955. [find](https://scholar.google.com/scholar?q=E%2E+Hewitt+and+L%2E+J%2E+Savage%2C+%E2%80%9CSymmetric+measures+on+Cartesian+products%2C%E2%80%9D+Transactions+of+the+American+Mathematical+Society%2C+vol%2E+80%2C+no%2E+2%2C+pp%2E+470%E2%80%93501%2C+1955%2E)
72. J. F. Kingman, “Uses of exchangeability,” The Annals of Probability, vol. 6, no. 2, pp. 183–197, 1978. [find](https://scholar.google.com/scholar?q=J%2E+F%2E+Kingman%2C+%E2%80%9CUses+of+exchangeability%2C%E2%80%9D+The+Annals+of+Probability%2C+vol%2E+6%2C+no%2E+2%2C+pp%2E+183%E2%80%93197%2C+1978%2E)
73. E. Dobriban, Topics in Modern Statistical Learning (STAT 991, UPenn, 2022 Spring), Dec. 2022. [find](https://scholar.google.com/scholar?q=E%2E+Dobriban%2C+Topics+in+Modern+Statistical+Learning+%28STAT+991%2C+UPenn%2C+2022+Spring%29%2C+Dec%2E+2022%2E)
74. A. Gammerman, V. Vovk, and V. Vapnik, “Learning by transduction,” Proceedings of the Fourteenth Conference on Uncertainty in Artificial Intelligence, vol. 14, pp. 148–155, 1998. [find](https://scholar.google.com/scholar?q=A%2E+Gammerman%2C+V%2E+Vovk%2C+and+V%2E+Vapnik%2C+%E2%80%9CLearning+by+transduction%2C%E2%80%9D+Proceedings+of+the+Fourteenth+Conference+on+Uncertainty+in+Artificial+Intelligence%2C+vol%2E+14%2C+pp%2E+148%E2%80%93155%2C+1998%2E)
75. C. Saunders, A. Gammerman, and V. Vovk, “Transduction with confidence and credibility,” 1999. [find](https://scholar.google.com/scholar?q=C%2E+Saunders%2C+A%2E+Gammerman%2C+and+V%2E+Vovk%2C+%E2%80%9CTransduction+with+confidence+and+credibility%2C%E2%80%9D+1999%2E)
76. V. Vovk, “On-line confidence machines are well-calibrated,” in The 43rd Annual IEEE Symposium on Foundations of Computer Science, IEEE, 2002, pp. 187–196. [find](https://scholar.google.com/scholar?q=V%2E+Vovk%2C+%E2%80%9COn%2Dline+confidence+machines+are+well%2Dcalibrated%2C%E2%80%9D+in+The+43rd+Annual+IEEE+Symposium+on+Foundations+of+Computer+Science%2C+IEEE%2C+2002%2C+pp%2E+187%E2%80%93196%2E)
77. V. Vovk, G. Shafer, and I. Nouretdinov, “Self-calibrating probability forecasting.,” in Neural Information Processing Systems, 2003, pp. 1133–1140. [find](https://scholar.google.com/scholar?q=V%2E+Vovk%2C+G%2E+Shafer%2C+and+I%2E+Nouretdinov%2C+%E2%80%9CSelf%2Dcalibrating+probability+forecasting%2E%2C%E2%80%9D+in+Neural+Information+Processing+Systems%2C+2003%2C+pp%2E+1133%E2%80%931140%2E)
78. V. Vovk and I. Petej, “Venn-Abers predictors,” arXiv:1211.0025, 2012. — [arXiv:1211.0025](https://arxiv.org/abs/1211.0025) [find](https://scholar.google.com/scholar?q=V%2E+Vovk+and+I%2E+Petej%2C+%E2%80%9CVenn%2DAbers+predictors%2C%E2%80%9D+arXiv%3A1211%2E0025%2C+2012%2E+%E2%80%94+%5BarXiv%3A1211%2E0025%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F1211%2E0025%29)
79. V. Vovk, J. Shen, V. Manokhin, and M.-g. Xie, “Nonparametric predictive distributions based on conformal prediction,” Machine Learning, pp. 1–30, 2017. [find](https://scholar.google.com/scholar?q=V%2E+Vovk%2C+J%2E+Shen%2C+V%2E+Manokhin%2C+and+M%2E%2Dg%2E+Xie%2C+%E2%80%9CNonparametric+predictive+distributions+based+on+conformal+prediction%2C%E2%80%9D+Machine+Learning%2C+pp%2E+1%E2%80%9330%2C+2017%2E)
80. J. Lei, J. Robins, and L. Wasserman, “Efficient nonparametric conformal prediction regions,” arXiv:1111.1418, 2011. — [arXiv:1111.1418](https://arxiv.org/abs/1111.1418) [find](https://scholar.google.com/scholar?q=J%2E+Lei%2C+J%2E+Robins%2C+and+L%2E+Wasserman%2C+%E2%80%9CEfficient+nonparametric+conformal+prediction+regions%2C%E2%80%9D+arXiv%3A1111%2E1418%2C+2011%2E+%E2%80%94+%5BarXiv%3A1111%2E1418%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F1111%2E1418%29)
81. ——, “Distribution-free prediction sets,” Journal of the American Statistical Association, vol. 108, no. 501, pp. 278–287, 2013. [find](https://scholar.google.com/scholar?q=%E2%80%94%E2%80%94%2C+%E2%80%9CDistribution%2Dfree+prediction+sets%2C%E2%80%9D+Journal+of+the+American+Statistical+Association%2C+vol%2E+108%2C+no%2E+501%2C+pp%2E+278%E2%80%93287%2C+2013%2E)
82. B. Póczos, A. Singh, A. Rinaldo, and L. Wasserman, “Distribution-free distribution regression,” in Artificial Intelligence and Statistics, PMLR, 2013, pp. 507–515. [find](https://scholar.google.com/scholar?q=B%2E+Po%CC%81czos%2C+A%2E+Singh%2C+A%2E+Rinaldo%2C+and+L%2E+Wasserman%2C+%E2%80%9CDistribution%2Dfree+distribution+regression%2C%E2%80%9D+in+Artificial+Intelligence+and+Statistics%2C+PMLR%2C+2013%2C+pp%2E+507%E2%80%93515%2E)
83. J. Lei, M. G’Sell, A. Rinaldo, R. J. Tibshirani, and L. Wasserman, “Distribution-free predictive inference for regression,” Journal of the American Statistical Association, vol. 113, no. 523, pp. 1094– 1111, 2018. [find](https://scholar.google.com/scholar?q=J%2E+Lei%2C+M%2E+G%E2%80%99Sell%2C+A%2E+Rinaldo%2C+R%2E+J%2E+Tibshirani%2C+and+L%2E+Wasserman%2C+%E2%80%9CDistribution%2Dfree+predictive+inference+for+regression%2C%E2%80%9D+Journal+of+the+American+Statistical+Association%2C+vol%2E+113%2C+no%2E+523%2C+pp%2E+1094)
84. J. Lei, A. Rinaldo, and L. Wasserman, “A conformal prediction approach to explore functional data,” Annals of Mathematics and Artificial Intelligence, vol. 74, pp. 29–43, 2015. [find](https://scholar.google.com/scholar?q=J%2E+Lei%2C+A%2E+Rinaldo%2C+and+L%2E+Wasserman%2C+%E2%80%9CA+conformal+prediction+approach+to+explore+functional+data%2C%E2%80%9D+Annals+of+Mathematics+and+Artificial+Intelligence%2C+vol%2E+74%2C+pp%2E+29%E2%80%9343%2C+2015%2E)
85. J. Lei, “Fast exact conformalization of the lasso using piecewise linear homotopy,” Biometrika, vol. 106, no. 4, pp. 749–764, 2019. [find](https://scholar.google.com/scholar?q=J%2E+Lei%2C+%E2%80%9CFast+exact+conformalization+of+the+lasso+using+piecewise+linear+homotopy%2C%E2%80%9D+Biometrika%2C+vol%2E+106%2C+no%2E+4%2C+pp%2E+749%E2%80%93764%2C+2019%2E)
86. C. Gupta, A. K. Kuchibhotla, and A. Ramdas, “Nested conformal prediction and quantile out-of-bag ensemble methods,” Pattern Recognition, p. 108 496, 2021. [find](https://scholar.google.com/scholar?q=C%2E+Gupta%2C+A%2E+K%2E+Kuchibhotla%2C+and+A%2E+Ramdas%2C+%E2%80%9CNested+conformal+prediction+and+quantile+out%2Dof%2Dbag+ensemble+methods%2C%E2%80%9D+Pattern+Recognition%2C+p%2E+108+496%2C+2021%2E)
87. R. Foygel Barber, E. J. Candes, A. Ramdas, and R. J. Tibshirani, “The limits of distribution-free conditional predictive inference,” Information and Inference: A Journal of the IMA, vol. 10, no. 2, pp. 455–482, 2021. [find](https://scholar.google.com/scholar?q=R%2E+Foygel+Barber%2C+E%2E+J%2E+Candes%2C+A%2E+Ramdas%2C+and+R%2E+J%2E+Tibshirani%2C+%E2%80%9CThe+limits+of+distribution%2Dfree+conditional+predictive+inference%2C%E2%80%9D+Information+and+Inference%3A+A+Journal+of+the+IMA%2C+vol%2E+10%2C+no%2E+2%2C+pp)
88. Y. Lee and R. F. Barber, “Distribution-free inference for regression: Discrete, continuous, and in between,” arXiv:2105.14075, 2021. — [arXiv:2105.14075](https://arxiv.org/abs/2105.14075) [find](https://scholar.google.com/scholar?q=Y%2E+Lee+and+R%2E+F%2E+Barber%2C+%E2%80%9CDistribution%2Dfree+inference+for+regression%3A+Discrete%2C+continuous%2C+and+in+between%2C%E2%80%9D+arXiv%3A2105%2E14075%2C+2021%2E+%E2%80%94+%5BarXiv%3A2105%2E14075%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2105%2E14075%29)
89. R. Izbicki, G. Shimizu, and R. Stern, “Flexible distribution-free conditional predictive bands using density estimators,” in Proceedings of Machine Learning Research, vol. 108, PMLR, 2020, pp. 3068– 3077. [find](https://scholar.google.com/scholar?q=R%2E+Izbicki%2C+G%2E+Shimizu%2C+and+R%2E+Stern%2C+%E2%80%9CFlexible+distribution%2Dfree+conditional+predictive+bands+using+density+estimators%2C%E2%80%9D+in+Proceedings+of+Machine+Learning+Research%2C+vol%2E+108%2C+PMLR%2C+2020%2C+pp%2E+3068%E2%80%93+3)
90. J. Lei, “Classification with confidence,” Biometrika, vol. 101, no. 4, pp. 755–769, Oct. 2014. [find](https://scholar.google.com/scholar?q=J%2E+Lei%2C+%E2%80%9CClassification+with+confidence%2C%E2%80%9D+Biometrika%2C+vol%2E+101%2C+no%2E+4%2C+pp%2E+755%E2%80%93769%2C+Oct%2E+2014%2E)
91. Y. Hechtlinger, B. Poczos, and L. Wasserman, “Cautious deep learning,” arXiv:1805.09460, 2018. — [arXiv:1805.09460](https://arxiv.org/abs/1805.09460) [find](https://scholar.google.com/scholar?q=Y%2E+Hechtlinger%2C+B%2E+Poczos%2C+and+L%2E+Wasserman%2C+%E2%80%9CCautious+deep+learning%2C%E2%80%9D+arXiv%3A1805%2E09460%2C+2018%2E+%E2%80%94+%5BarXiv%3A1805%2E09460%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F1805%2E09460%29)
92. D. Stutz, K. D. Dvijotham, A. T. Cemgil, and A. Doucet, “Learning optimal conformal classifiers,” in International Conference on Learning Representations, 2022. [find](https://scholar.google.com/scholar?q=D%2E+Stutz%2C+K%2E+D%2E+Dvijotham%2C+A%2E+T%2E+Cemgil%2C+and+A%2E+Doucet%2C+%E2%80%9CLearning+optimal+conformal+classifiers%2C%E2%80%9D+in+International+Conference+on+Learning+Representations%2C+2022%2E)
93. M. Cauchois, S. Gupta, A. Ali, and J. C. Duchi, “Robust validation: Confident predictions even when distributions shift,” arXiv:2008.04267, 2020. — [arXiv:2008.04267](https://arxiv.org/abs/2008.04267) [find](https://scholar.google.com/scholar?q=M%2E+Cauchois%2C+S%2E+Gupta%2C+A%2E+Ali%2C+and+J%2E+C%2E+Duchi%2C+%E2%80%9CRobust+validation%3A+Confident+predictions+even+when+distributions+shift%2C%E2%80%9D+arXiv%3A2008%2E04267%2C+2020%2E+%E2%80%94+%5BarXiv%3A2008%2E04267%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2008%2E04267%29)
94. L. Lei and E. J. Candès, “Conformal inference of counterfactuals and individual treatment effects,” arXiv:2006.06138, 2020. — [arXiv:2006.06138](https://arxiv.org/abs/2006.06138) [find](https://scholar.google.com/scholar?q=L%2E+Lei+and+E%2E+J%2E+Cande%CC%80s%2C+%E2%80%9CConformal+inference+of+counterfactuals+and+individual+treatment+effects%2C%E2%80%9D+arXiv%3A2006%2E06138%2C+2020%2E+%E2%80%94+%5BarXiv%3A2006%2E06138%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2006%2E06138%29)
95. M. Yin, C. Shi, Y. Wang, and D. M. Blei, “Conformal sensitivity analysis for individual treatment effects,” arXiv:2112.03493, 2021. [find](https://scholar.google.com/scholar?q=M%2E+Yin%2C+C%2E+Shi%2C+Y%2E+Wang%2C+and+D%2E+M%2E+Blei%2C+%E2%80%9CConformal+sensitivity+analysis+for+individual+treatment+effects%2C%E2%80%9D+arXiv%3A2112%2E03493%2C+2021%2E)
96. V. Chernozhukov, K. Wüthrich, and Y. Zhu, “An exact and robust conformal inference method for counterfactual and synthetic controls,” Journal of the American Statistical Association, pp. 1–16, 2021. [find](https://scholar.google.com/scholar?q=V%2E+Chernozhukov%2C+K%2E+Wu%CC%88thrich%2C+and+Y%2E+Zhu%2C+%E2%80%9CAn+exact+and+robust+conformal+inference+method+for+counterfactual+and+synthetic+controls%2C%E2%80%9D+Journal+of+the+American+Statistical+Association%2C+pp%2E+1%E2%80%9316%2C+2021%2E)
97. E. J. Candès, L. Lei, and Z. Ren, “Conformalized survival analysis,” arXiv:2103.09763, 2021. — [arXiv:2103.09763](https://arxiv.org/abs/2103.09763) [find](https://scholar.google.com/scholar?q=E%2E+J%2E+Cande%CC%80s%2C+L%2E+Lei%2C+and+Z%2E+Ren%2C+%E2%80%9CConformalized+survival+analysis%2C%E2%80%9D+arXiv%3A2103%2E09763%2C+2021%2E+%E2%80%94+%5BarXiv%3A2103%2E09763%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2103%2E09763%29)
98. A. N. Angelopoulos, S. Bates, T. Zrnic, and M. I. Jordan, “Private prediction sets,” arXiv:2102.06202, 2021. — [arXiv:2102.06202](https://arxiv.org/abs/2102.06202) [find](https://scholar.google.com/scholar?q=A%2E+N%2E+Angelopoulos%2C+S%2E+Bates%2C+T%2E+Zrnic%2C+and+M%2E+I%2E+Jordan%2C+%E2%80%9CPrivate+prediction+sets%2C%E2%80%9D+arXiv%3A2102%2E06202%2C+2021%2E+%E2%80%94+%5BarXiv%3A2102%2E06202%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2102%2E06202%29)
99. V. Chernozhukov, K. Wüthrich, and Z. Yinchu, “Exact and robust conformal inference methods for predictive machine learning with dependent data,” in Conference On Learning Theory, PMLR, 2018, pp. 732–749. [find](https://scholar.google.com/scholar?q=V%2E+Chernozhukov%2C+K%2E+Wu%CC%88thrich%2C+and+Z%2E+Yinchu%2C+%E2%80%9CExact+and+robust+conformal+inference+methods+for+predictive+machine+learning+with+dependent+data%2C%E2%80%9D+in+Conference+On+Learning+Theory%2C+PMLR%2C+2018%2C+pp%2E+732%E2%80%93)
100. R. Dunn, L. Wasserman, and A. Ramdas, “Distribution-free prediction sets with random effects,” arXiv:1809.07441, 2018. — [arXiv:1809.07441](https://arxiv.org/abs/1809.07441) [find](https://scholar.google.com/scholar?q=R%2E+Dunn%2C+L%2E+Wasserman%2C+and+A%2E+Ramdas%2C+%E2%80%9CDistribution%2Dfree+prediction+sets+with+random+effects%2C%E2%80%9D+arXiv%3A1809%2E07441%2C+2018%2E+%E2%80%94+%5BarXiv%3A1809%2E07441%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F1809%2E07441%29)
101. R. I. Oliveira, P. Orenstein, T. Ramos, and J. V. Romano, “Split conformal prediction for dependent data,” arXiv:2203.15885, 2022. — [arXiv:2203.15885](https://arxiv.org/abs/2203.15885) [find](https://scholar.google.com/scholar?q=R%2E+I%2E+Oliveira%2C+P%2E+Orenstein%2C+T%2E+Ramos%2C+and+J%2E+V%2E+Romano%2C+%E2%80%9CSplit+conformal+prediction+for+dependent+data%2C%E2%80%9D+arXiv%3A2203%2E15885%2C+2022%2E+%E2%80%94+%5BarXiv%3A2203%2E15885%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2203%2E15885%29)
102. O. Bastani, V. Gupta, C. Jung, G. Noarov, R. Ramalingam, and A. Roth, “Practical adversarial multivalid conformal prediction,” in Advances in Neural Information Processing Systems, A. H. Oh, A. Agarwal, D. Belgrave, and K. Cho, Eds., 2022. [find](https://scholar.google.com/scholar?q=O%2E+Bastani%2C+V%2E+Gupta%2C+C%2E+Jung%2C+G%2E+Noarov%2C+R%2E+Ramalingam%2C+and+A%2E+Roth%2C+%E2%80%9CPractical+adversarial+multivalid+conformal+prediction%2C%E2%80%9D+in+Advances+in+Neural+Information+Processing+Systems%2C+A%2E+H%2E+Oh%2C+A%2E+Agarwa)
103. C. Jung, G. Noarov, R. Ramalingam, and A. Roth, “Batch multivalid conformal prediction,” arXiv preprint arXiv:2209.15145, 2022. — [arXiv:2209.15145](https://arxiv.org/abs/2209.15145) [find](https://scholar.google.com/scholar?q=C%2E+Jung%2C+G%2E+Noarov%2C+R%2E+Ramalingam%2C+and+A%2E+Roth%2C+%E2%80%9CBatch+multivalid+conformal+prediction%2C%E2%80%9D+arXiv+preprint+arXiv%3A2209%2E15145%2C+2022%2E+%E2%80%94+%5BarXiv%3A2209%2E15145%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2209%2E15145%29)
104. C. Gupta and A. Ramdas, “Distribution-free calibration guarantees for histogram binning without sample splitting,” in International Conference on Machine Learning, vol. 139, 2021, pp. 3942–3952. [find](https://scholar.google.com/scholar?q=C%2E+Gupta+and+A%2E+Ramdas%2C+%E2%80%9CDistribution%2Dfree+calibration+guarantees+for+histogram+binning+without+sample+splitting%2C%E2%80%9D+in+International+Conference+on+Machine+Learning%2C+vol%2E+139%2C+2021%2C+pp%2E+3942%E2%80%933952%2E)
105. S. Park, S. Li, O. Bastani, and I. Lee, “PAC confidence predictions for deep neural network classifiers,” in International Conference on Learning Representations, 2021. [find](https://scholar.google.com/scholar?q=S%2E+Park%2C+S%2E+Li%2C+O%2E+Bastani%2C+and+I%2E+Lee%2C+%E2%80%9CPAC+confidence+predictions+for+deep+neural+network+classifiers%2C%E2%80%9D+in+International+Conference+on+Learning+Representations%2C+2021%2E)
106. D. Volkhonskiy, E. Burnaev, I. Nouretdinov, A. Gammerman, and V. Vovk, “Inductive conformal martingales for change-point detection,” in Conformal and Probabilistic Prediction and Applications, PMLR, 2017, pp. 132–153. [find](https://scholar.google.com/scholar?q=D%2E+Volkhonskiy%2C+E%2E+Burnaev%2C+I%2E+Nouretdinov%2C+A%2E+Gammerman%2C+and+V%2E+Vovk%2C+%E2%80%9CInductive+conformal+martingales+for+change%2Dpoint+detection%2C%E2%80%9D+in+Conformal+and+Probabilistic+Prediction+and+Applications%2C+PMLR%2C+2)
107. X. Hu and J. Lei, “A distribution-free test of covariate shift using conformal prediction,” arXiv:2010.07147, 2020. — [arXiv:2010.07147](https://arxiv.org/abs/2010.07147) [find](https://scholar.google.com/scholar?q=X%2E+Hu+and+J%2E+Lei%2C+%E2%80%9CA+distribution%2Dfree+test+of+covariate+shift+using+conformal+prediction%2C%E2%80%9D+arXiv%3A2010%2E07147%2C+2020%2E+%E2%80%94+%5BarXiv%3A2010%2E07147%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2010%2E07147%29)
108. A. Podkopaev and A. Ramdas, “Tracking the risk of a deployed model and detecting harmful distribution shifts,” arXiv:2110.06177, 2021. — [arXiv:2110.06177](https://arxiv.org/abs/2110.06177) [find](https://scholar.google.com/scholar?q=A%2E+Podkopaev+and+A%2E+Ramdas%2C+%E2%80%9CTracking+the+risk+of+a+deployed+model+and+detecting+harmful+distribution+shifts%2C%E2%80%9D+arXiv%3A2110%2E06177%2C+2021%2E+%E2%80%94+%5BarXiv%3A2110%2E06177%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2110%2E06177%29)
109. P. Chaudhuri, “Global nonparametric estimation of conditional quantile functions and their derivatives,” Journal of Multivariate Analysis, vol. 39, no. 2, pp. 246–269, 1991. [find](https://scholar.google.com/scholar?q=P%2E+Chaudhuri%2C+%E2%80%9CGlobal+nonparametric+estimation+of+conditional+quantile+functions+and+their+derivatives%2C%E2%80%9D+Journal+of+Multivariate+Analysis%2C+vol%2E+39%2C+no%2E+2%2C+pp%2E+246%E2%80%93269%2C+1991%2E)
110. I. Steinwart and A. Christmann, “Estimating conditional quantiles with the help of the pinball loss,” Bernoulli, vol. 17, no. 1, pp. 211–225, 2011. [find](https://scholar.google.com/scholar?q=I%2E+Steinwart+and+A%2E+Christmann%2C+%E2%80%9CEstimating+conditional+quantiles+with+the+help+of+the+pinball+loss%2C%E2%80%9D+Bernoulli%2C+vol%2E+17%2C+no%2E+1%2C+pp%2E+211%E2%80%93225%2C+2011%2E)
111. I. Takeuchi, Q. V. Le, T. D. Sears, and A. J. Smola, “Nonparametric quantile estimation,” Journal of Machine Learning Research, vol. 7, pp. 1231–1264, 2006. [find](https://scholar.google.com/scholar?q=I%2E+Takeuchi%2C+Q%2E+V%2E+Le%2C+T%2E+D%2E+Sears%2C+and+A%2E+J%2E+Smola%2C+%E2%80%9CNonparametric+quantile+estimation%2C%E2%80%9D+Journal+of+Machine+Learning+Research%2C+vol%2E+7%2C+pp%2E+1231%E2%80%931264%2C+2006%2E)
112. K. Q. Zhou, S. L. Portnoy, et al., “Direct use of regression quantiles to construct confidence sets in linear models,” The Annals of Statistics, vol. 24, no. 1, pp. 287–306, 1996. [find](https://scholar.google.com/scholar?q=K%2E+Q%2E+Zhou%2C+S%2E+L%2E+Portnoy%2C+et+al%2E%2C+%E2%80%9CDirect+use+of+regression+quantiles+to+construct+confidence+sets+in+linear+models%2C%E2%80%9D+The+Annals+of+Statistics%2C+vol%2E+24%2C+no%2E+1%2C+pp%2E+287%E2%80%93306%2C+1996%2E)
113. K. Q. Zhou and S. L. Portnoy, “Statistical inference on heteroscedastic models based on regression quantiles,” Journal of Nonparametric Statistics, vol. 9, no. 3, pp. 239–260, 1998. [find](https://scholar.google.com/scholar?q=K%2E+Q%2E+Zhou+and+S%2E+L%2E+Portnoy%2C+%E2%80%9CStatistical+inference+on+heteroscedastic+models+based+on+regression+quantiles%2C%E2%80%9D+Journal+of+Nonparametric+Statistics%2C+vol%2E+9%2C+no%2E+3%2C+pp%2E+239%E2%80%93260%2C+1998%2E)
114. R. Koenker, Quantile Regression. Cambridge University Press, 2005. [find](https://scholar.google.com/scholar?q=R%2E+Koenker%2C+Quantile+Regression%2E+Cambridge+University+Press%2C+2005%2E)
115. R. Koenker, V. Chernozhukov, X. He, and L. Peng, “Handbook of quantile regression,” 2018. [find](https://scholar.google.com/scholar?q=R%2E+Koenker%2C+V%2E+Chernozhukov%2C+X%2E+He%2C+and+L%2E+Peng%2C+%E2%80%9CHandbook+of+quantile+regression%2C%E2%80%9D+2018%2E)
116. R. Koenker, “Additive models for quantile regression: Model selection and confidence bandaids,” Brazilian Journal of Probability and Statistics, vol. 25, no. 3, pp. 239–262, 2011. [find](https://scholar.google.com/scholar?q=R%2E+Koenker%2C+%E2%80%9CAdditive+models+for+quantile+regression%3A+Model+selection+and+confidence+bandaids%2C%E2%80%9D+Brazilian+Journal+of+Probability+and+Statistics%2C+vol%2E+25%2C+no%2E+3%2C+pp%2E+239%E2%80%93262%2C+2011%2E)
117. A. Fisch, T. Schuster, T. S. Jaakkola, and R. Barzilay, “Efficient conformal prediction via cascaded inference with expanded admission,” in International Conference on Learning Representations, 2021. [find](https://scholar.google.com/scholar?q=A%2E+Fisch%2C+T%2E+Schuster%2C+T%2E+S%2E+Jaakkola%2C+and+R%2E+Barzilay%2C+%E2%80%9CEfficient+conformal+prediction+via+cascaded+inference+with+expanded+admission%2C%E2%80%9D+in+International+Conference+on+Learning+Representations%2C+2021%2E)
118. T. Schuster, A. Fisch, T. Jaakkola, and R. Barzilay, “Consistent accelerated inference via confident adaptive transformers,” Empirical Methods in Natural Language Processing, 2021. [find](https://scholar.google.com/scholar?q=T%2E+Schuster%2C+A%2E+Fisch%2C+T%2E+Jaakkola%2C+and+R%2E+Barzilay%2C+%E2%80%9CConsistent+accelerated+inference+via+confident+adaptive+transformers%2C%E2%80%9D+Empirical+Methods+in+Natural+Language+Processing%2C+2021%2E)
119. A. Fisch, T. Schuster, T. Jaakkola, and D. Barzilay, “Few-shot conformal prediction with auxiliary tasks,” in International Conference on Machine Learning, vol. 139, 2021, pp. 3329–3339. [find](https://scholar.google.com/scholar?q=A%2E+Fisch%2C+T%2E+Schuster%2C+T%2E+Jaakkola%2C+and+D%2E+Barzilay%2C+%E2%80%9CFew%2Dshot+conformal+prediction+with+auxiliary+tasks%2C%E2%80%9D+in+International+Conference+on+Machine+Learning%2C+vol%2E+139%2C+2021%2C+pp%2E+3329%E2%80%933339%2E)
120. U. Johansson, H. Boström, T. Löfström, and H. Linusson, “Regression conformal prediction with random forests,” Machine learning, vol. 97, no. 1, pp. 155–176, 2014. [find](https://scholar.google.com/scholar?q=U%2E+Johansson%2C+H%2E+Bostro%CC%88m%2C+T%2E+Lo%CC%88fstro%CC%88m%2C+and+H%2E+Linusson%2C+%E2%80%9CRegression+conformal+prediction+with+random+forests%2C%E2%80%9D+Machine+learning%2C+vol%2E+97%2C+no%2E+1%2C+pp%2E+155%E2%80%93176%2C+2014%2E)
121. H. Linusson, U. Norinder, H. Boström, U. Johansson, and T. Löfström, “On the calibration of aggregated conformal predictors,” in Conformal and probabilistic prediction and applications, PMLR, 2017, pp. 154–173. [find](https://scholar.google.com/scholar?q=H%2E+Linusson%2C+U%2E+Norinder%2C+H%2E+Bostro%CC%88m%2C+U%2E+Johansson%2C+and+T%2E+Lo%CC%88fstro%CC%88m%2C+%E2%80%9COn+the+calibration+of+aggregated+conformal+predictors%2C%E2%80%9D+in+Conformal+and+probabilistic+prediction+and+applications%2C+PMLR%2C+2017%2C)
122. H. Boström, H. Linusson, T. Löfström, and U. Johansson, “Accelerating difficulty estimation for conformal regression forests,” Annals of Mathematics and Artificial Intelligence, vol. 81, no. 1, pp. 125– 144, 2017. [find](https://scholar.google.com/scholar?q=H%2E+Bostro%CC%88m%2C+H%2E+Linusson%2C+T%2E+Lo%CC%88fstro%CC%88m%2C+and+U%2E+Johansson%2C+%E2%80%9CAccelerating+difficulty+estimation+for+conformal+regression+forests%2C%E2%80%9D+Annals+of+Mathematics+and+Artificial+Intelligence%2C+vol%2E+81%2C+no%2E+1%2C+pp%2E)
123. J. Cherian and L. Bronner, “How the Washington Post estimates outstanding votes for the 2020 presidential election,” Washington Post, 2021, [https://s3.us-east-1.amazonaws.com/elex-models-prod/2020-general/write-up/election](https://s3.us-east-1.amazonaws.com/elex-models-prod/2020-general/write-up/election) model writeup.pdf. [↗](https://s3.us-east-1.amazonaws.com/elex-models-prod/2020-general/write-up/election)
124. C. Lu and J. Kalpathy-Cramer, “Distribution-free federated learning with conformal predictions,” arXiv:2110.07661, 2021. — [arXiv:2110.07661](https://arxiv.org/abs/2110.07661) [find](https://scholar.google.com/scholar?q=C%2E+Lu+and+J%2E+Kalpathy%2DCramer%2C+%E2%80%9CDistribution%2Dfree+federated+learning+with+conformal+predictions%2C%E2%80%9D+arXiv%3A2110%2E07661%2C+2021%2E+%E2%80%94+%5BarXiv%3A2110%2E07661%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2110%2E07661%29)
125. C. Lu, A. Lemay, K. Chang, K. Hoebel, and J. Kalpathy-Cramer, “Fair conformal predictors for applications in medical imaging,” arXiv:2109.04392, 2021. — [arXiv:2109.04392](https://arxiv.org/abs/2109.04392) [find](https://scholar.google.com/scholar?q=C%2E+Lu%2C+A%2E+Lemay%2C+K%2E+Chang%2C+K%2E+Hoebel%2C+and+J%2E+Kalpathy%2DCramer%2C+%E2%80%9CFair+conformal+predictors+for+applications+in+medical+imaging%2C%E2%80%9D+arXiv%3A2109%2E04392%2C+2021%2E+%E2%80%94+%5BarXiv%3A2109%2E04392%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2109%2E0)
126. Y. Romano, R. F. Barber, C. Sabatti, and E. Candès, “With malice toward none: Assessing uncertainty via equalized coverage,” Harvard Data Science Review, vol. 2, no. 2, Apr. 30, 2020. [find](https://scholar.google.com/scholar?q=Y%2E+Romano%2C+R%2E+F%2E+Barber%2C+C%2E+Sabatti%2C+and+E%2E+Cande%CC%80s%2C+%E2%80%9CWith+malice+toward+none%3A+Assessing+uncertainty+via+equalized+coverage%2C%E2%80%9D+Harvard+Data+Science+Review%2C+vol%2E+2%2C+no%2E+2%2C+Apr%2E+30%2C+2020%2E)
127. A. K. Kuchibhotla and R. A. Berk, “Nested conformal prediction sets for classification with applications to probation data,” arXiv:2104.09358, 2021. — [arXiv:2104.09358](https://arxiv.org/abs/2104.09358) [find](https://scholar.google.com/scholar?q=A%2E+K%2E+Kuchibhotla+and+R%2E+A%2E+Berk%2C+%E2%80%9CNested+conformal+prediction+sets+for+classification+with+applications+to+probation+data%2C%E2%80%9D+arXiv%3A2104%2E09358%2C+2021%2E+%E2%80%94+%5BarXiv%3A2104%2E09358%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2104%2E093)
128. L. Lindemann, M. Cleaveland, G. Shim, and G. J. Pappas, “Safe planning in dynamic environments using conformal prediction,” arXiv preprint arXiv:2210.10254, 2022. — [arXiv:2210.10254](https://arxiv.org/abs/2210.10254) [find](https://scholar.google.com/scholar?q=L%2E+Lindemann%2C+M%2E+Cleaveland%2C+G%2E+Shim%2C+and+G%2E+J%2E+Pappas%2C+%E2%80%9CSafe+planning+in+dynamic+environments+using+conformal+prediction%2C%E2%80%9D+arXiv+preprint+arXiv%3A2210%2E10254%2C+2022%2E+%E2%80%94+%5BarXiv%3A2210%2E10254%5D%28https%3A%2F%2Farxiv%2Eor)
129. A. Dixit, L. Lindemann, M. Cleaveland, S. Wei, G. J. Pappas, and J. W. Burdick, “Adaptive conformal prediction for motion planning among dynamic agents,” arXiv preprint arXiv:2212.00278, 2022. — [arXiv:2212.00278](https://arxiv.org/abs/2212.00278) [find](https://scholar.google.com/scholar?q=A%2E+Dixit%2C+L%2E+Lindemann%2C+M%2E+Cleaveland%2C+S%2E+Wei%2C+G%2E+J%2E+Pappas%2C+and+J%2E+W%2E+Burdick%2C+%E2%80%9CAdaptive+conformal+prediction+for+motion+planning+among+dynamic+agents%2C%E2%80%9D+arXiv+preprint+arXiv%3A2212%2E00278%2C+2022%2E+%E2%80%94+%5BarXi)
130. V. Manokhin, Awesome Conformal Prediction, version v1.0.0, Apr. 2022. [find](https://scholar.google.com/scholar?q=V%2E+Manokhin%2C+Awesome+Conformal+Prediction%2C+version+v1%2E0%2E0%2C+Apr%2E+2022%2E)
131. S. Bates, A. Angelopoulos, L. Lei, J. Malik, and M. Jordan, “Distribution-free, risk-controlling prediction sets,” Journal of the Association for Computing Machinery, vol. 68, no. 6, Sep. 2021. [find](https://scholar.google.com/scholar?q=S%2E+Bates%2C+A%2E+Angelopoulos%2C+L%2E+Lei%2C+J%2E+Malik%2C+and+M%2E+Jordan%2C+%E2%80%9CDistribution%2Dfree%2C+risk%2Dcontrolling+prediction+sets%2C%E2%80%9D+Journal+of+the+Association+for+Computing+Machinery%2C+vol%2E+68%2C+no%2E+6%2C+Sep%2E+2021%2E)
132. F. Bretz, W. Maurer, W. Brannath, and M. Posch, “A graphical approach to sequentially rejective multiple test procedures,” Statistics in Medicine, vol. 28, no. 4, pp. 586–604, 2009. Figure 19: Object detection with simultaneous distribution-free guarantees on the expected intersection-over-union, re… [find](https://scholar.google.com/scholar?q=F%2E+Bretz%2C+W%2E+Maurer%2C+W%2E+Brannath%2C+and+M%2E+Posch%2C+%E2%80%9CA+graphical+approach+to+sequentially+rejective+multiple+test+procedures%2C%E2%80%9D+Statistics+in+Medicine%2C+vol%2E+28%2C+no%2E+4%2C+pp%2E+586%E2%80%93604%2C+2009%2E+Figure+19%3A+Object+)

*Updated 2 November 2026*