---
title: "Benchmarking Neural Network Robustness to Common"
description: "They release two test-only benchmarks built from the ImageNet validation set. 63 references, cited by 143 papers here."
authors: ["Dan Hendrycks", "Thomas Dietterich"]
year: 2019
venue: "arXiv:1903.12261"
source: "https://arxiv.org/abs/1903.12261"
doi: "10.48550/arxiv.1903.12261"
arxiv: "1903.12261"
licence: "arXiv — non-exclusive license to distribute"
licence_url: "http://arxiv.org/licenses/nonexclusive-distrib/1.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/hendrycks-2019-benchmarking-neural-network-robustness-common/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/hendrycks-2019-benchmarking-neural-network-robustness-common/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2019](https://md.research.elamur.ai/free/en/papers/2019/page/2/) / Benchmarking Neural Network Robustness to Common Corruptions and Perturbations

# Benchmarking Neural Network Robustness to Common Corruptions and Perturbations

**Authors:** [Dan Hendrycks](https://md.research.elamur.ai/free/en/authors/dan-hendrycks/) and [Thomas Dietterich](https://md.research.elamur.ai/free/en/authors/thomas-g-dietterich/) · **Year:** 2019 · **Venue:** arXiv:1903.12261 · **Licence:** [arXiv — non-exclusive license to distribute](http://arxiv.org/licenses/nonexclusive-distrib/1.0/)

**Links:** [Publisher's page](https://arxiv.org/abs/1903.12261) · [PDF](https://arxiv.org/pdf/1903.12261) · [DOI 10.48550/arxiv.1903.12261](https://doi.org/10.48550/arxiv.1903.12261) · [arXiv 1903.12261](https://arxiv.org/abs/1903.12261) · [OpenAlex](https://openalex.org/W2905631704)

**Citations:** 291 citations (OpenAlex, 2026-10-14) · [143](https://md.research.elamur.ai/free/en/papers/hendrycks-2019-benchmarking-neural-network-robustness-common/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
Most robustness research in computer vision focuses on worst-case adversarial attacks. Deployed classifiers, however, mostly face everyday degradations such as sensor noise, blur, bad weather and compression. Hendrycks and Dietterich formalize two average-case notions of robustness. Corruption robustness is accuracy on degraded inputs. Perturbation robustness is how stable predictions stay under small, non-adversarial changes. They release two test-only benchmarks built from the ImageNet validation set. ImageNet-C applies algorithmically generated corruptions at graded intensities. ImageNet-P supplies sequences of gradually altered images to measure how often predictions change. Across architectures from AlexNet to ResNet, higher clean accuracy brings almost no gain in relative corruption robustness, which leaves substantial room for progress. The authors find architectures and techniques that improve both robustness types without sacrificing accuracy. They also show that Adversarial Logit Pairing, an adversarial defense that has been circumvented, still helps noticeably against common perturbations. Intuitive remedies such as denoising, 10-crop inference or compact models do not help. The benchmarks standardize evaluation and discourage cherry-picked results, supporting reliability in safety-critical systems.

## Key points
- ImageNet-C contains 15 corruption types in four groups (noise, blur, weather, digital), each at 5 severity levels, giving 75 corruptions. Smaller editions include CIFAR-10-C, Tiny ImageNet-C and ImageNet 64×64-C, plus an Inception-sized version. Code and data are at github.com/hendrycks/robustness.
- Robustness is reported as mCE (mean corruption error) for ImageNet-C and as flip-rate-based metrics such as mFR for ImageNet-P. Models must be trained on clean data, never on the benchmark corruptions.
- Non-local means denoising made results worse, raising mCE from 76.7% to 82.1%, because it removed fine detail even from noise-free images.
- Smaller models did not help. CondenseNet (C = G = 4) reached 26.3% error and 80.8% mCE. The more heavily pruned CondenseNet (C = G = 8) reached 28.9% error and 84.6% mCE. Both proved more vulnerable to corruptions than bigger networks with comparable clean accuracy.
- An extra experiment on subtype robustness used 25 broad categories from ImageNet-22K. Classifiers recognized unseen subtypes of known categories clearly worse than seen ones.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [University of California, Berkeley](https://md.research.elamur.ai/free/en/universities/university-of-california-berkeley/) — University, Berkeley, United States · 209 works · 244,782 citations
- [Oregon State University](https://md.research.elamur.ai/free/en/universities/oregon-state-university/) — University, Corvallis, United States · 11 works · 1,422 citations

📚 Shelves (5): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 18,614 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 17,965 works · [👁️ Computer Vision and Pattern Recognition](https://md.research.elamur.ai/free/en/shelves/computer-science/computer-vision-and-pattern-recognition/) · 3,486 works · [🌾 Agricultural and Biological Sciences](https://md.research.elamur.ai/free/en/shelves/agricultural-and-biological-sciences/) · 8 works · [🌾 Plant Science](https://md.research.elamur.ai/free/en/shelves/agricultural-and-biological-sciences/plant-science/) · 5 works

## Cite

Cite as: Dan Hendrycks and Thomas Dietterich. 2019. Benchmarking Neural Network Robustness to Common Corruptions and Perturbations. arXiv:1903.12261.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@misc{hendrycks2019benchmarking,
  title = {{Benchmarking Neural Network Robustness to Common Corruptions and Perturbations}},
  author = {Dan Hendrycks and Thomas Dietterich},
  year = {2019},
  doi = {10.48550/arxiv.1903.12261},
  eprint = {1903.12261},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.1903.12261}
}
```

- [← previous Are Sixteen Heads Really Better than One?](https://md.research.elamur.ai/free/en/papers/michel-2019-sixteen-heads-really-better-than/)
- [next → A general framework for information extraction using dynamic span graphs](https://md.research.elamur.ai/free/en/papers/luan-2019-general-framework-information-extraction-dynamic/)
- [↑ back to papers of 2019](https://md.research.elamur.ai/free/en/papers/2019/page/2/)

## References

63 references, 5 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Ossama Abdel-Hamid, Abdel rahman Mohamed, Hui Jiang, and Gerald Penn. Applying convolutional neural networks concepts to hybrid nn-hmm model for speech recognition. ICASSP, 2013. [find](https://scholar.google.com/scholar?q=Ossama+Abdel%2DHamid%2C+Abdel+rahman+Mohamed%2C+Hui+Jiang%2C+and+Gerald+Penn%2E+Applying+convolutional+neural+networks+concepts+to+hybrid+nn%2Dhmm+model+for+speech+recognition%2E+ICASSP%2C+2013%2E)
2. Aharon Azulay and Yair Weiss. Why do deep convolutional networks generalize so poorly to small image transformations? arXiv preprint, 2018. [find](https://scholar.google.com/scholar?q=Aharon+Azulay+and+Yair+Weiss%2E+Why+do+deep+convolutional+networks+generalize+so+poorly+to+small+image+transformations%3F+arXiv+preprint%2C+2018%2E)
3. Osbert Bastani, Yani Ioannou, Leonidas Lampropoulos, Dimitrios Vytiniotis, Aditya Nori, and Antonio Criminisi. Measuring neural net robustness with constraints. In NIPS. 2016. [find](https://scholar.google.com/scholar?q=Osbert+Bastani%2C+Yani+Ioannou%2C+Leonidas+Lampropoulos%2C+Dimitrios+Vytiniotis%2C+Aditya+Nori%2C+and+Antonio+Criminisi%2E+Measuring+neural+net+robustness+with+constraints%2E+In+NIPS%2E+2016%2E)
4. Antoni Buades and Bartomeu Coll. A non-local algorithm for image denoising. In CVPR, 2005. [find](https://scholar.google.com/scholar?q=Antoni+Buades+and+Bartomeu+Coll%2E+A+non%2Dlocal+algorithm+for+image+denoising%2E+In+CVPR%2C+2005%2E)
5. Nicholas Carlini and David Wagner. Defensive distillation is not robust to adversarial examples, 2016. [find](https://scholar.google.com/scholar?q=Nicholas+Carlini+and+David+Wagner%2E+Defensive+distillation+is+not+robust+to+adversarial+examples%2C+2016%2E)
6. Nicholas Carlini and David Wagner. Adversarial examples are not easily detected: Bypassing ten detection methods, 2017. [find](https://scholar.google.com/scholar?q=Nicholas+Carlini+and+David+Wagner%2E+Adversarial+examples+are+not+easily+detected%3A+Bypassing+ten+detection+methods%2C+2017%2E)
7. Nicholas Carlini, Guy Katz, Clark Barrett, and David L. Dill. Ground-truth adversarial examples, 2017. [find](https://scholar.google.com/scholar?q=Nicholas+Carlini%2C+Guy+Katz%2C+Clark+Barrett%2C+and+David+L%2E+Dill%2E+Ground%2Dtruth+adversarial+examples%2C+2017%2E)
8. Jia Deng, Wei Dong, Richard Socher, Li jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. CVPR, 2009. [find](https://scholar.google.com/scholar?q=Jia+Deng%2C+Wei+Dong%2C+Richard+Socher%2C+Li+jia+Li%2C+Kai+Li%2C+and+Li+Fei%2DFei%2E+Imagenet%3A+A+large%2Dscale+hierarchical+image+database%2E+CVPR%2C+2009%2E)
9. Samuel Dodge and Lina Karam. Quality resilient deep neural networks, 2017a. [find](https://scholar.google.com/scholar?q=Samuel+Dodge+and+Lina+Karam%2E+Quality+resilient+deep+neural+networks%2C+2017a%2E)
10. Samuel Dodge and Lina Karam. A study and comparison of human and deep learning recognition performance under visual distortions, 2017b. [find](https://scholar.google.com/scholar?q=Samuel+Dodge+and+Lina+Karam%2E+A+study+and+comparison+of+human+and+deep+learning+recognition+performance+under+visual+distortions%2C+2017b%2E)
11. David Donoho and Iain Johnstone. Ideal spatial adaptation by wavelet shrinkage. Biometrika, 1993. [find](https://scholar.google.com/scholar?q=David+Donoho+and+Iain+Johnstone%2E+Ideal+spatial+adaptation+by+wavelet+shrinkage%2E+Biometrika%2C+1993%2E)
12. Logan Engstrom, Andrew Ilyas, and Anish Athalye. Evaluating and understanding the robustness of adversarial logit pairing. arXiv preprint, 2018. [find](https://scholar.google.com/scholar?q=Logan+Engstrom%2C+Andrew+Ilyas%2C+and+Anish+Athalye%2E+Evaluating+and+understanding+the+robustness+of+adversarial+logit+pairing%2E+arXiv+preprint%2C+2018%2E)
13. Ivan Evtimov, Kevin Eykholt, Earlence Fernandes, Tadayoshi Kohno, Bo Li, Atul Prakash, Amir Rahmati, and Dawn Song. Robust physical-world attacks on deep learning models, 2017. [find](https://scholar.google.com/scholar?q=Ivan+Evtimov%2C+Kevin+Eykholt%2C+Earlence+Fernandes%2C+Tadayoshi+Kohno%2C+Bo+Li%2C+Atul+Prakash%2C+Amir+Rahmati%2C+and+Dawn+Song%2E+Robust+physical%2Dworld+attacks+on+deep+learning+models%2C+2017%2E)
14. Ruth Fong and Andrea Vedaldi. Interpretable explanations of black boxes by meaningful perturbation. ICCV, 2017. [find](https://scholar.google.com/scholar?q=Ruth+Fong+and+Andrea+Vedaldi%2E+Interpretable+explanations+of+black+boxes+by+meaningful+perturbation%2E+ICCV%2C+2017%2E)
15. Leon Gatys, Alexander Ecker, and Matthias Bethge. Image style transfer using convolutional neural networks. CVPR, 2016. [find](https://scholar.google.com/scholar?q=Leon+Gatys%2C+Alexander+Ecker%2C+and+Matthias+Bethge%2E+Image+style+transfer+using+convolutional+neural+networks%2E+CVPR%2C+2016%2E)
16. Robert Geirhos, David H. J. Janssen, Heiko H. Schütt, Jonas Rauber, Matthias Bethge, and Felix A. Wichmann. Comparing deep neural networks against humans: object recognition when the signal gets weaker, 2017. [find](https://scholar.google.com/scholar?q=Robert+Geirhos%2C+David+H%2E+J%2E+Janssen%2C+Heiko+H%2E+Sch%C3%BCtt%2C+Jonas+Rauber%2C+Matthias+Bethge%2C+and+Felix+A%2E+Wichmann%2E+Comparing+deep+neural+networks+against+humans%3A+object+recognition+when+the+signal+gets+weake)
17. Robert Geirhos, Patricia Rubisch, Claudio Michaelis, Matthias Bethge, Felix A Wichmann, and Wieland Brendel. Imagenet-trained cnns are biased towards texture; increasing shape bias improves accuracy and robustness. ICLR, 2019. [find](https://scholar.google.com/scholar?q=Robert+Geirhos%2C+Patricia+Rubisch%2C+Claudio+Michaelis%2C+Matthias+Bethge%2C+Felix+A+Wichmann%2C+and+Wieland+Brendel%2E+Imagenet%2Dtrained+cnns+are+biased+towards+texture%3B+increasing+shape+bias+improves+accuracy+a)
18. Justin Gilmer, Ryan P. Adams, Ian Goodfellow, David Andersen, and George E. Dahl. Motivating the rules of the game for adversarial example research. arXiv preprint, 2018a. [find](https://scholar.google.com/scholar?q=Justin+Gilmer%2C+Ryan+P%2E+Adams%2C+Ian+Goodfellow%2C+David+Andersen%2C+and+George+E%2E+Dahl%2E+Motivating+the+rules+of+the+game+for+adversarial+example+research%2E+arXiv+preprint%2C+2018a%2E)
19. Justin Gilmer, Luke Metz, Fartash Faghri, Samuel S. Schoenholz, Maithra Raghu, Martin Wattenberg, and Ian Goodfellow. Adversarial spheres. ICLR Workshop, 2018b. [find](https://scholar.google.com/scholar?q=Justin+Gilmer%2C+Luke+Metz%2C+Fartash+Faghri%2C+Samuel+S%2E+Schoenholz%2C+Maithra+Raghu%2C+Martin+Wattenberg%2C+and+Ian+Goodfellow%2E+Adversarial+spheres%2E+ICLR+Workshop%2C+2018b%2E)
20. Chuan Guo, Geoff Pleiss, Yu Sun, and Kilian Q. Weinberger. On calibration of modern neural networks. International Conference on Machine Learning, 2017. [find](https://scholar.google.com/scholar?q=Chuan+Guo%2C+Geoff+Pleiss%2C+Yu+Sun%2C+and+Kilian+Q%2E+Weinberger%2E+On+calibration+of+modern+neural+networks%2E+International+Conference+on+Machine+Learning%2C+2017%2E)
21. Mark Harvilla and Richard Stern. Histogram-based subband powerwarping and spectral averaging for robust speech recognition under matched and multistyle training, 2012. [find](https://scholar.google.com/scholar?q=Mark+Harvilla+and+Richard+Stern%2E+Histogram%2Dbased+subband+powerwarping+and+spectral+averaging+for+robust+speech+recognition+under+matched+and+multistyle+training%2C+2012%2E)
22. Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. [Deep residual learning for image recognition](https://md.research.elamur.ai/free/en/papers/he-2015-deep-residual-learning-image-recognition/). CVPR, 2015.
23. Dan Hendrycks and Kevin Gimpel. Early methods for detecting adversarial images, 2017a. [find](https://scholar.google.com/scholar?q=Dan+Hendrycks+and+Kevin+Gimpel%2E+Early+methods+for+detecting+adversarial+images%2C+2017a%2E)
24. Dan Hendrycks and Kevin Gimpel. A baseline for detecting misclassified and out-of-distribution examples in neural networks. In ICLR, 2017b. [find](https://scholar.google.com/scholar?q=Dan+Hendrycks+and+Kevin+Gimpel%2E+A+baseline+for+detecting+misclassified+and+out%2Dof%2Ddistribution+examples+in+neural+networks%2E+In+ICLR%2C+2017b%2E)
25. Dan Hendrycks, Mantas Mazeika, Duncan Wilson, and Kevin Gimpel. Using trusted data to train deep networks on labels corrupted by severe noise. NIPS, 2018. [find](https://scholar.google.com/scholar?q=Dan+Hendrycks%2C+Mantas+Mazeika%2C+Duncan+Wilson%2C+and+Kevin+Gimpel%2E+Using+trusted+data+to+train+deep+networks+on+labels+corrupted+by+severe+noise%2E+NIPS%2C+2018%2E)
26. Dan Hendrycks, Mantas Mazeika, and Thomas Dietterich. Deep anomaly detection with outlier exposure. ICLR, 2019. [find](https://scholar.google.com/scholar?q=Dan+Hendrycks%2C+Mantas+Mazeika%2C+and+Thomas+Dietterich%2E+Deep+anomaly+detection+with+outlier+exposure%2E+ICLR%2C+2019%2E)
27. Hans-Günter Hirsch. Aurora-5 experimental framework for the performance evaluation of speech recognition in case of a hands-free speech input in noisy environments, 2007. [find](https://scholar.google.com/scholar?q=Hans%2DG%C3%BCnter+Hirsch%2E+Aurora%2D5+experimental+framework+for+the+performance+evaluation+of+speech+recognition+in+case+of+a+hands%2Dfree+speech+input+in+noisy+environments%2C+2007%2E)
28. Hans-Günter Hirsch and David Pearce. The Aurora experimental framework for the performance evaluation of speech recognition systems under noisy conditions. ISCA ITRW ASR2000, 2000. [find](https://scholar.google.com/scholar?q=Hans%2DG%C3%BCnter+Hirsch+and+David+Pearce%2E+The+Aurora+experimental+framework+for+the+performance+evaluation+of+speech+recognition+systems+under+noisy+conditions%2E+ISCA+ITRW+ASR2000%2C+2000%2E)
29. Hossein Hosseini, Baicen Xiao, and Radha Poovendran. Google’s cloud vision api is not robust to noise, 2017. [find](https://scholar.google.com/scholar?q=Hossein+Hosseini%2C+Baicen+Xiao%2C+and+Radha+Poovendran%2E+Google%E2%80%99s+cloud+vision+api+is+not+robust+to+noise%2C+2017%2E)
30. Gao Huang, Shichen Liu, Laurens van der Maaten, and Kilian Q Weinberger. Condensenet: An efficient DenseNet using learned group convolutions. arXiv preprint, 2017a. [find](https://scholar.google.com/scholar?q=Gao+Huang%2C+Shichen+Liu%2C+Laurens+van+der+Maaten%2C+and+Kilian+Q+Weinberger%2E+Condensenet%3A+An+efficient+DenseNet+using+learned+group+convolutions%2E+arXiv+preprint%2C+2017a%2E)
31. Gao Huang, Zhuang Liu, Laurens van der Maaten, and Kilian Q Weinberger. Densely connected convolutional networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017b. [find](https://scholar.google.com/scholar?q=Gao+Huang%2C+Zhuang+Liu%2C+Laurens+van+der+Maaten%2C+and+Kilian+Q+Weinberger%2E+Densely+connected+convolutional+networks%2E+In+Proceedings+of+the+IEEE+Conference+on+Computer+Vision+and+Pattern+Recognition%2C+2017)
32. Gao Huang, Danlu Chen, Tianhong Li, Felix Wu, Laurens van der Maaten, and Kilian Q. Weinberger. Multi-scale dense networks for resource efficient image classification. ICLR, 2018. [find](https://scholar.google.com/scholar?q=Gao+Huang%2C+Danlu+Chen%2C+Tianhong+Li%2C+Felix+Wu%2C+Laurens+van+der+Maaten%2C+and+Kilian+Q%2E+Weinberger%2E+Multi%2Dscale+dense+networks+for+resource+efficient+image+classification%2E+ICLR%2C+2018%2E)
33. Sergey Ioffe and Christian Szegedy. Batch normalization: Accelerating deep network training by reducing internal covariate shift. JMLR, 2015. [find](https://scholar.google.com/scholar?q=Sergey+Ioffe+and+Christian+Szegedy%2E+Batch+normalization%3A+Accelerating+deep+network+training+by+reducing+internal+covariate+shift%2E+JMLR%2C+2015%2E)
34. Harini Kannan, Alexey Kurakin, and Ian Goodfellow. Adversarial logit pairing. NIPS, 2018. [find](https://scholar.google.com/scholar?q=Harini+Kannan%2C+Alexey+Kurakin%2C+and+Ian+Goodfellow%2E+Adversarial+logit+pairing%2E+NIPS%2C+2018%2E)
35. Tsung-Wei Ke, Michael Maire, and Stella X. Yu. Multigrid neural architectures, 2017. [find](https://scholar.google.com/scholar?q=Tsung%2DWei+Ke%2C+Michael+Maire%2C+and+Stella+X%2E+Yu%2E+Multigrid+neural+architectures%2C+2017%2E)
36. Chanwoo Kim and Richard M. Stern. Power-normalized cepstral coefficients (PNCC) for robust speech recognition. IEEE/ACM Trans. Audio, Speech and Lang. Proc., 24(7):1315–1329, July 2016. ISSN 2329-9290. [find](https://scholar.google.com/scholar?q=Chanwoo+Kim+and+Richard+M%2E+Stern%2E+Power%2Dnormalized+cepstral+coefficients+%28PNCC%29+for+robust+speech+recognition%2E+IEEE%2FACM+Trans%2E+Audio%2C+Speech+and+Lang%2E+Proc%2E%2C+24%287%29%3A1315%E2%80%931329%2C+July+2016%2E+ISSN+2329%2D9290)
37. Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. [Imagenet classification with deep convolutional neural networks](https://md.research.elamur.ai/free/en/papers/krizhevsky-2012-imagenet-classification-deep-convolutional/). NIPS, 2012.
38. Ravi Kumar and Sergei Vassilvitskii. Generalized distances between rankings, 2010. [find](https://scholar.google.com/scholar?q=Ravi+Kumar+and+Sergei+Vassilvitskii%2E+Generalized+distances+between+rankings%2C+2010%2E)
39. Alexey Kurakin, Ian Goodfellow, and Samy Bengio. Adversarial machine learning at scale. ICLR, 2017. [find](https://scholar.google.com/scholar?q=Alexey+Kurakin%2C+Ian+Goodfellow%2C+and+Samy+Bengio%2E+Adversarial+machine+learning+at+scale%2E+ICLR%2C+2017%2E)
40. Jinyu Li, Li Deng, Yifan Gong, and Reinhold Haeb-Umbach. An overview of noise-robust automatic speech recognition. 2014. [find](https://scholar.google.com/scholar?q=Jinyu+Li%2C+Li+Deng%2C+Yifan+Gong%2C+and+Reinhold+Haeb%2DUmbach%2E+An+overview+of+noise%2Drobust+automatic+speech+recognition%2E+2014%2E)
41. Fu-Hua Liu, Richard M. Stern, Xuedong Huang, and Alex Acero. Efficient cepstral normalization for robust speech recognition. In Proc. of DARPA Speech and Natural Language Workshop, 1993. [find](https://scholar.google.com/scholar?q=Fu%2DHua+Liu%2C+Richard+M%2E+Stern%2C+Xuedong+Huang%2C+and+Alex+Acero%2E+Efficient+cepstral+normalization+for+robust+speech+recognition%2E+In+Proc%2E+of+DARPA+Speech+and+Natural+Language+Workshop%2C+1993%2E)
42. Si Liu, Risheek Garrepalli, Thomas Dietterich, Alan Fern, and Dan Hendrycks. Open category detection with PAC guarantees. In ICML, 2018. [find](https://scholar.google.com/scholar?q=Si+Liu%2C+Risheek+Garrepalli%2C+Thomas+Dietterich%2C+Alan+Fern%2C+and+Dan+Hendrycks%2E+Open+category+detection+with+PAC+guarantees%2E+In+ICML%2C+2018%2E)
43. Jiajun Lu, Hussein Sibai, Evan Fabry, and David Forsyth. Standard detectors aren’t (currently) fooled by physical adversarial stop signs, 2017. [find](https://scholar.google.com/scholar?q=Jiajun+Lu%2C+Hussein+Sibai%2C+Evan+Fabry%2C+and+David+Forsyth%2E+Standard+detectors+aren%E2%80%99t+%28currently%29+fooled+by+physical+adversarial+stop+signs%2C+2017%2E)
44. Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. [Towards deep learning models resistant to adversarial attacks](https://md.research.elamur.ai/free/en/papers/madry-2017-towards-deep-learning-models-resistant/). ICLR, 2018.
45. Jan Hendrik Metzen, Tim Genewein, Volker Fischer, and Bastian Bischoff. On detecting adversarial perturbations, 2017. [find](https://scholar.google.com/scholar?q=Jan+Hendrik+Metzen%2C+Tim+Genewein%2C+Volker+Fischer%2C+and+Bastian+Bischoff%2E+On+detecting+adversarial+perturbations%2C+2017%2E)
46. Vikramjit Mitra, Horacio Franco, Richard Stern, Julien Van Hout, Luciana Ferrer, Martin Graciarena, Wen Wang, Dimitra Vergyri, Abeer Alwan, and John H.L. Hansen. Robust features in deep learning based speech recognition, 2017. [find](https://scholar.google.com/scholar?q=Vikramjit+Mitra%2C+Horacio+Franco%2C+Richard+Stern%2C+Julien+Van+Hout%2C+Luciana+Ferrer%2C+Martin+Graciarena%2C+Wen+Wang%2C+Dimitra+Vergyri%2C+Abeer+Alwan%2C+and+John+H%2EL%2E+Hansen%2E+Robust+features+in+deep+learning+based)
47. Chris Olah, Alexander Mordvintsev, and Ludwig Schubert. Feature visualization. Distill, 2017. [find](https://scholar.google.com/scholar?q=Chris+Olah%2C+Alexander+Mordvintsev%2C+and+Ludwig+Schubert%2E+Feature+visualization%2E+Distill%2C+2017%2E)
48. Nicolas Papernot, Patrick McDaniel, Xi Wu, Somesh Jha, and Ananthram Swami. Distillation as a defense to adversarial perturbations against deep neural networks, 2017. [find](https://scholar.google.com/scholar?q=Nicolas+Papernot%2C+Patrick+McDaniel%2C+Xi+Wu%2C+Somesh+Jha%2C+and+Ananthram+Swami%2E+Distillation+as+a+defense+to+adversarial+perturbations+against+deep+neural+networks%2C+2017%2E)
49. Stephen M. Pizer, E. Philip Amburn, John D. Austin, Robert Cromartie, Ari Geselowitz, Trey Greer, Bart Ter Haar Romeny, and John B. Zimmerman. Adaptive histogram equalization and its variations. Computer Vision, Graphics, and Image Processing, 1987. [find](https://scholar.google.com/scholar?q=Stephen+M%2E+Pizer%2C+E%2E+Philip+Amburn%2C+John+D%2E+Austin%2C+Robert+Cromartie%2C+Ari+Geselowitz%2C+Trey+Greer%2C+Bart+Ter+Haar+Romeny%2C+and+John+B%2E+Zimmerman%2E+Adaptive+histogram+equalization+and+its+variations%2E+Compu)
50. Jonas Rauber, Wieland Brendel, and Matthias Bethge. Foolbox v0.8.0: A python toolbox to benchmark the robustness of machine learning models, 2017. [find](https://scholar.google.com/scholar?q=Jonas+Rauber%2C+Wieland+Brendel%2C+and+Matthias+Bethge%2E+Foolbox+v0%2E8%2E0%3A+A+python+toolbox+to+benchmark+the+robustness+of+machine+learning+models%2C+2017%2E)
51. Benjamin Recht, Rebecca Roelofs, Ludwig Schmidt, and Vaishaal Shankar. Do cifar-10 classifiers generalize to cifar-10? arXiv preprint, 2018. [find](https://scholar.google.com/scholar?q=Benjamin+Recht%2C+Rebecca+Roelofs%2C+Ludwig+Schmidt%2C+and+Vaishaal+Shankar%2E+Do+cifar%2D10+classifiers+generalize+to+cifar%2D10%3F+arXiv+preprint%2C+2018%2E)
52. Ludwig Schmidt, Shibani Santurkar, Dimitris Tsipras, Kunal Talwar, and Aleksander Madry. Adversarially robust generalization requires more data. arXiv preprint, 2018. [find](https://scholar.google.com/scholar?q=Ludwig+Schmidt%2C+Shibani+Santurkar%2C+Dimitris+Tsipras%2C+Kunal+Talwar%2C+and+Aleksander+Madry%2E+Adversarially+robust+generalization+requires+more+data%2E+arXiv+preprint%2C+2018%2E)
53. Lukas Schott, Jonas Rauber, Matthias Bethge, and Wieland Brendel. [Towards deep learning models resistant to adversarial attacks](https://md.research.elamur.ai/free/en/papers/madry-2017-towards-deep-learning-models-resistant/). arXiv preprint, 2018.
54. Jacob Steinhardt, Pang Wei Koh, and Percy Liang. Certified defenses for data poisoning attacks. NIPS, 2017. [find](https://scholar.google.com/scholar?q=Jacob+Steinhardt%2C+Pang+Wei+Koh%2C+and+Percy+Liang%2E+Certified+defenses+for+data+poisoning+attacks%2E+NIPS%2C+2017%2E)
55. Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, and Rob Fergus. [Intriguing properties of neural networks](https://md.research.elamur.ai/free/en/papers/szegedy-2013-intriguing-properties-neural-networks/), 2014.
56. Dogancan Temel and Ghassan AlRegib. Traffic signs in the wild: Highlights from the ieee video and image processing cup 2017 student competition. [find](https://scholar.google.com/scholar?q=Dogancan+Temel+and+Ghassan+AlRegib%2E+Traffic+signs+in+the+wild%3A+Highlights+from+the+ieee+video+and+image+processing+cup+2017+student+competition%2E)
57. IEEE Signal Processing Magazine, 2018. [find](https://scholar.google.com/scholar?q=IEEE+Signal+Processing+Magazine%2C+2018%2E)
58. Dogancan Temel, Gukyeong Kwon, Mohit Prabhushankar, and Ghassan AlRegib. Cure-tsr: Challenging unreal and real environments for traffic sign recognition. NIPS Workshop, 2017. [find](https://scholar.google.com/scholar?q=Dogancan+Temel%2C+Gukyeong+Kwon%2C+Mohit+Prabhushankar%2C+and+Ghassan+AlRegib%2E+Cure%2Dtsr%3A+Challenging+unreal+and+real+environments+for+traffic+sign+recognition%2E+NIPS+Workshop%2C+2017%2E)
59. Dogancan Temel, Jinsol Lee, and Ghassan AlRegib. Cure-or: Challenging unreal and real environments for object recognition. ICMLA, 2018. [find](https://scholar.google.com/scholar?q=Dogancan+Temel%2C+Jinsol+Lee%2C+and+Ghassan+AlRegib%2E+Cure%2Dor%3A+Challenging+unreal+and+real+environments+for+object+recognition%2E+ICMLA%2C+2018%2E)
60. Ángel de la Torre, Antonio Peinado, José Segura, José Pérez-Córdoba, Ma Carmen Benítez, and Antonio Rubio. Histogram equalization of speech representation for robust speech recognition. IEEE Signal Processing Society, 2005. [find](https://scholar.google.com/scholar?q=%C3%81ngel+de+la+Torre%2C+Antonio+Peinado%2C+Jos%C3%A9+Segura%2C+Jos%C3%A9+P%C3%A9rez%2DC%C3%B3rdoba%2C+Ma+Carmen+Ben%C3%ADtez%2C+and+Antonio+Rubio%2E+Histogram+equalization+of+speech+representation+for+robust+speech+recognition%2E+IEEE+Signal+Pr)
61. Igor Vasiljevic, Ayan Chakrabarti, and Gregory Shakhnarovich. Examining the impact of blur on recognition by convolutional networks, 2016. [find](https://scholar.google.com/scholar?q=Igor+Vasiljevic%2C+Ayan+Chakrabarti%2C+and+Gregory+Shakhnarovich%2E+Examining+the+impact+of+blur+on+recognition+by+convolutional+networks%2C+2016%2E)
62. Saining Xie, Ross Girshick, Piotr Dollár, Zhuowen Tu, and Kaiming He. Aggregated residual transformations for deep neural networks. CVPR, 2016. [find](https://scholar.google.com/scholar?q=Saining+Xie%2C+Ross+Girshick%2C+Piotr+Doll%C3%A1r%2C+Zhuowen+Tu%2C+and+Kaiming+He%2E+Aggregated+residual+transformations+for+deep+neural+networks%2E+CVPR%2C+2016%2E)
63. Stephan Zheng, Yang Song, Thomas Leung, and Ian Goodfellow. Improving the robustness of deep neural networks via stability training, 2016. [find](https://scholar.google.com/scholar?q=Stephan+Zheng%2C+Yang+Song%2C+Thomas+Leung%2C+and+Ian+Goodfellow%2E+Improving+the+robustness+of+deep+neural+networks+via+stability+training%2C+2016%2E)

*Updated 2 November 2026*