---
title: "ImageNet Classification with Deep Convolutional Neural Networks"
description: "The authors worked on image classification from the ImageNet LSVRC-2010 contest. 26 references, cited by 2,047 papers here."
authors: ["Alex Krizhevsky", "Ilya Sutskever", "Geoffrey E. Hinton"]
year: 2012
venue: "NeurIPS 2012"
source: "https://proceedings.neurips.cc/paper/2012/hash/c399862d3b9d6b76c8436e924a68c45b-Abstract.html"
licence: "All rights reserved — free to read at the source"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/krizhevsky-2012-imagenet-classification-deep-convolutional/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/krizhevsky-2012-imagenet-classification-deep-convolutional/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2012](https://md.research.elamur.ai/free/en/papers/2012/) / ImageNet Classification with Deep Convolutional Neural Networks

# ImageNet Classification with Deep Convolutional Neural Networks

**Authors:** Alex Krizhevsky, [Ilya Sutskever](https://md.research.elamur.ai/free/en/authors/ilya-sutskever/) and [Geoffrey E. Hinton](https://md.research.elamur.ai/free/en/authors/geoffrey-e-hinton/) · **Year:** 2012 · **Venue:** NeurIPS 2012 · **Licence:** All rights reserved — free to read at the source

**Links:** [Publisher's page](https://proceedings.neurips.cc/paper/2012/hash/c399862d3b9d6b76c8436e924a68c45b-Abstract.html) · [PDF](https://proceedings.neurips.cc/paper_files/paper/2012/file/c399862d3b9d6b76c8436e924a68c45b-Paper.pdf) · [OpenAlex](https://openalex.org/W3147600416)

**Citations:** 754 citations (OpenAlex, 2026-10-14) · [2047](https://md.research.elamur.ai/free/en/papers/krizhevsky-2012-imagenet-classification-deep-convolutional/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
The authors worked on image classification from the ImageNet LSVRC-2010 contest. The training set contained about 1.2 million photographs in 1000 classes. They built a deep convolutional network (CNN) with five convolutional and three fully connected layers. The network has approximately 60 million parameters. To speed up training, the authors used the non-saturating ReLU activation. The model was split across two GTX 580 graphics cards. For this, they wrote their own optimized GPU implementation of convolutions (cuda-convnet) and released it publicly. To combat overfitting, they applied data augmentation and dropout in the fully connected layers. They also added local response normalization and overlapping pooling. On the 2010 test set, the top-1 error was 37.5% and the top-5 error was 17.0%, considerably better than the previous state of the art. One variant of the model won ILSVRC-2012 by a large margin. The work showed that a deep CNN trained purely with supervised learning outperforms methods based on hand-crafted features, provided there is plenty of labeled data and GPU computation. It was this work that launched the rapid spread of deep learning in computer vision.

## Key points
- **Architecture:** 8 learned layers, approximately 60 million parameters and 650 thousand neurons, with a 1000-way softmax at the output. Training took 5–6 days on two GTX 580 GPUs with 3 GB of memory each.
- **ReLU vs. tanh:** on CIFAR-10, a four-layer network with ReLU reached 25% training error approximately six times faster.
- **Contribution of individual techniques:**
  - response normalization reduces the top-1 and top-5 errors by 1.4 and 1.2 percentage points;
  - the two-GPU scheme reduces them by 1.7 and 1.2 percentage points;
  - removing any of the middle convolutional layers lowers top-1 accuracy by approximately 2%.
- **ILSVRC-2012:** an ensemble of seven CNNs achieved 15.3% top-5 error on the test set, while second place (SIFT + FVs) had 26.2%. Two networks in the ensemble were first pre-trained on ImageNet Fall 2011 (15 million images, 22 thousand categories).
- **ImageNet Fall 2009** (10,184 categories, 8.9 million images): the top-1 and top-5 errors were 67.4% and 40.9%. The best previously published results were 78.1% and 60.9%.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

📚 Shelves (3): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [👁️ Computer Vision and Pattern Recognition](https://md.research.elamur.ai/free/en/shelves/computer-science/computer-vision-and-pattern-recognition/) · 2,869 works

💡 Concepts (3): [💡 Natural language processing](https://md.research.elamur.ai/free/en/concepts/natural-language-processing/) · 131 works · [💡 Machine learning](https://md.research.elamur.ai/free/en/concepts/machine-learning/) · 51 works · [💡 Artificial intelligence](https://md.research.elamur.ai/free/en/concepts/artificial-intelligence/) · 49 works

## Cite

Cite as: Alex Krizhevsky, Ilya Sutskever and Geoffrey E. Hinton. 2012. ImageNet Classification with Deep Convolutional Neural Networks. NeurIPS 2012.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{krizhevsky2012imagenet,
  title = {{ImageNet Classification with Deep Convolutional Neural Networks}},
  author = {Alex Krizhevsky, Ilya Sutskever and Geoffrey E. Hinton},
  booktitle = {NeurIPS 2012},
  year = {2012},
  url = {https://proceedings.neurips.cc/paper/2012/hash/c399862d3b9d6b76c8436e924a68c45b-Abstract.html}
}
```

- [← previous ADADELTA: An Adaptive Learning Rate Method](https://md.research.elamur.ai/free/en/papers/zeiler-2012-adadelta-adaptive-learning-rate-method/)
- [next → Information structure in discourse: Towards an integrated formal theory of pragmatics](https://md.research.elamur.ai/free/en/papers/roberts-2012-information-structure-discourse-towards-integrated/)
- [↑ back to papers of 2012](https://md.research.elamur.ai/free/en/papers/2012/)

## References

26 references, 0 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. R.M. Bell and Y. Koren. Lessons from the netflix prize challenge. ACM SIGKDD Explorations Newsletter, 9(2):75–79, 2007. [find](https://scholar.google.com/scholar?q=R%2EM%2E+Bell+and+Y%2E+Koren%2E+Lessons+from+the+netflix+prize+challenge%2E+ACM+SIGKDD+Explorations+Newsletter%2C+9%282%29%3A75%E2%80%9379%2C+2007%2E)
2. A. Berg, J. Deng, and L. Fei-Fei. Large scale visual recognition challenge 2010. www.image-net.org/challenges. 2010. [find](https://scholar.google.com/scholar?q=A%2E+Berg%2C+J%2E+Deng%2C+and+L%2E+Fei%2DFei%2E+Large+scale+visual+recognition+challenge+2010%2E+www%2Eimage%2Dnet%2Eorg%2Fchallenges%2E+2010%2E)
3. L. Breiman. Random forests. Machine learning, 45(1):5–32, 2001. [find](https://scholar.google.com/scholar?q=L%2E+Breiman%2E+Random+forests%2E+Machine+learning%2C+45%281%29%3A5%E2%80%9332%2C+2001%2E)
4. D. Cireşan, U. Meier, and J. Schmidhuber. Multi-column deep neural networks for image classification. Arxiv preprint arXiv:1202.2745, 2012. [find](https://scholar.google.com/scholar?q=D%2E+Cires%CC%A7an%2C+U%2E+Meier%2C+and+J%2E+Schmidhuber%2E+Multi%2Dcolumn+deep+neural+networks+for+image+classification%2E+Arxiv+preprint+arXiv%3A1202%2E2745%2C+2012%2E)
5. D.C. Cireşan, U. Meier, J. Masci, L.M. Gambardella, and J. Schmidhuber. High-performance neural networks for visual object classification. Arxiv preprint arXiv:1102.0183, 2011. — [arXiv:1102.0183](https://arxiv.org/abs/1102.0183) [find](https://scholar.google.com/scholar?q=D%2EC%2E+Cires%CC%A7an%2C+U%2E+Meier%2C+J%2E+Masci%2C+L%2EM%2E+Gambardella%2C+and+J%2E+Schmidhuber%2E+High%2Dperformance+neural+networks+for+visual+object+classification%2E+Arxiv+preprint+arXiv%3A1102%2E0183%2C+2011%2E+%E2%80%94+%5BarXiv%3A1102%2E0183%5D%28ht)
6. J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei. ImageNet: A Large-Scale Hierarchical Image Database. In CVPR09, 2009. [find](https://scholar.google.com/scholar?q=J%2E+Deng%2C+W%2E+Dong%2C+R%2E+Socher%2C+L%2E%2DJ%2E+Li%2C+K%2E+Li%2C+and+L%2E+Fei%2DFei%2E+ImageNet%3A+A+Large%2DScale+Hierarchical+Image+Database%2E+In+CVPR09%2C+2009%2E)
7. J. Deng, A. Berg, S. Satheesh, H. Su, A. Khosla, and L. Fei-Fei. ILSVRC-2012, 2012. URL [http://www.image-net.org/challenges/LSVRC/2012/](http://www.image-net.org/challenges/LSVRC/2012/). [↗](http://www.image-net.org/challenges/LSVRC/2012/)
8. L. Fei-Fei, R. Fergus, and P. Perona. Learning generative visual models from few training examples: An incremental bayesian approach tested on 101 object categories. Computer Vision and Image Understanding, 106(1):59–70, 2007. [find](https://scholar.google.com/scholar?q=L%2E+Fei%2DFei%2C+R%2E+Fergus%2C+and+P%2E+Perona%2E+Learning+generative+visual+models+from+few+training+examples%3A+An+incremental+bayesian+approach+tested+on+101+object+categories%2E+Computer+Vision+and+Image+Understa)
9. G. Griffin, A. Holub, and P. Perona. Caltech-256 object category dataset. Technical Report 7694, California Institute of Technology, 2007. URL [http://authors.library.caltech.edu/7694](http://authors.library.caltech.edu/7694). [↗](http://authors.library.caltech.edu/7694)
10. G.E. Hinton, N. Srivastava, A. Krizhevsky, I. Sutskever, and R.R. Salakhutdinov. Improving neural networks by preventing co-adaptation of feature detectors. arXiv preprint arXiv:1207.0580, 2012. [find](https://scholar.google.com/scholar?q=G%2EE%2E+Hinton%2C+N%2E+Srivastava%2C+A%2E+Krizhevsky%2C+I%2E+Sutskever%2C+and+R%2ER%2E+Salakhutdinov%2E+Improving+neural+networks+by+preventing+co%2Dadaptation+of+feature+detectors%2E+arXiv+preprint+arXiv%3A1207%2E0580%2C+2012%2E)
11. K. Jarrett, K. Kavukcuoglu, M. A. Ranzato, and Y. LeCun. What is the best multi-stage architecture for object recognition? In International Conference on Computer Vision, pages 2146–2153. IEEE, 2009. [find](https://scholar.google.com/scholar?q=K%2E+Jarrett%2C+K%2E+Kavukcuoglu%2C+M%2E+A%2E+Ranzato%2C+and+Y%2E+LeCun%2E+What+is+the+best+multi%2Dstage+architecture+for+object+recognition%3F+In+International+Conference+on+Computer+Vision%2C+pages+2146%E2%80%932153%2E+IEEE%2C+2009%2E)
12. A. Krizhevsky. Learning multiple layers of features from tiny images. Master’s thesis, Department of Computer Science, University of Toronto, 2009. [find](https://scholar.google.com/scholar?q=A%2E+Krizhevsky%2E+Learning+multiple+layers+of+features+from+tiny+images%2E+Master%E2%80%99s+thesis%2C+Department+of+Computer+Science%2C+University+of+Toronto%2C+2009%2E)
13. A. Krizhevsky. Convolutional deep belief networks on cifar-10. Unpublished manuscript, 2010. [find](https://scholar.google.com/scholar?q=A%2E+Krizhevsky%2E+Convolutional+deep+belief+networks+on+cifar%2D10%2E+Unpublished+manuscript%2C+2010%2E)
14. A. Krizhevsky and G.E. Hinton. Using very deep autoencoders for content-based image retrieval. In ESANN, 2011. [find](https://scholar.google.com/scholar?q=A%2E+Krizhevsky+and+G%2EE%2E+Hinton%2E+Using+very+deep+autoencoders+for+content%2Dbased+image+retrieval%2E+In+ESANN%2C+2011%2E)
15. Y. Le Cun, B. Boser, J.S. Denker, D. Henderson, R.E. Howard, W. Hubbard, L.D. Jackel, et al. Handwritten digit recognition with a back-propagation network. In Advances in neural information processing systems, 1990. [find](https://scholar.google.com/scholar?q=Y%2E+Le+Cun%2C+B%2E+Boser%2C+J%2ES%2E+Denker%2C+D%2E+Henderson%2C+R%2EE%2E+Howard%2C+W%2E+Hubbard%2C+L%2ED%2E+Jackel%2C+et+al%2E+Handwritten+digit+recognition+with+a+back%2Dpropagation+network%2E+In+Advances+in+neural+information+processing)
16. Y. LeCun, F.J. Huang, and L. Bottou. Learning methods for generic object recognition with invariance to pose and lighting. In Computer Vision and Pattern Recognition, 2004. CVPR 2004. Proceedings of the 2004 IEEE Computer Society Conference on, volume 2, pages II–97. IEEE, 2004. [find](https://scholar.google.com/scholar?q=Y%2E+LeCun%2C+F%2EJ%2E+Huang%2C+and+L%2E+Bottou%2E+Learning+methods+for+generic+object+recognition+with+invariance+to+pose+and+lighting%2E+In+Computer+Vision+and+Pattern+Recognition%2C+2004%2E+CVPR+2004%2E+Proceedings+of+t)
17. Y. LeCun, K. Kavukcuoglu, and C. Farabet. Convolutional networks and applications in vision. In Circuits and Systems (ISCAS), Proceedings of 2010 IEEE International Symposium on, pages 253–256. IEEE, 2010. [find](https://scholar.google.com/scholar?q=Y%2E+LeCun%2C+K%2E+Kavukcuoglu%2C+and+C%2E+Farabet%2E+Convolutional+networks+and+applications+in+vision%2E+In+Circuits+and+Systems+%28ISCAS%29%2C+Proceedings+of+2010+IEEE+International+Symposium+on%2C+pages+253%E2%80%93256%2E+IEEE%2C+)
18. H. Lee, R. Grosse, R. Ranganath, and A.Y. Ng. Convolutional deep belief networks for scalable unsupervised learning of hierarchical representations. In Proceedings of the 26th Annual International Conference on Machine Learning, pages 609–616. ACM, 2009. [find](https://scholar.google.com/scholar?q=H%2E+Lee%2C+R%2E+Grosse%2C+R%2E+Ranganath%2C+and+A%2EY%2E+Ng%2E+Convolutional+deep+belief+networks+for+scalable+unsupervised+learning+of+hierarchical+representations%2E+In+Proceedings+of+the+26th+Annual+International+Con)
19. T. Mensink, J. Verbeek, F. Perronnin, and G. Csurka. Metric Learning for Large Scale Image Classification: Generalizing to New Classes at Near-Zero Cost. In ECCV - European Conference on Computer Vision, Florence, Italy, October 2012. [find](https://scholar.google.com/scholar?q=T%2E+Mensink%2C+J%2E+Verbeek%2C+F%2E+Perronnin%2C+and+G%2E+Csurka%2E+Metric+Learning+for+Large+Scale+Image+Classification%3A+Generalizing+to+New+Classes+at+Near%2DZero+Cost%2E+In+ECCV+%2D+European+Conference+on+Computer+Visi)
20. V. Nair and G. E. Hinton. Rectified linear units improve restricted boltzmann machines. In Proc. 27th International Conference on Machine Learning, 2010. [find](https://scholar.google.com/scholar?q=V%2E+Nair+and+G%2E+E%2E+Hinton%2E+Rectified+linear+units+improve+restricted+boltzmann+machines%2E+In+Proc%2E+27th+International+Conference+on+Machine+Learning%2C+2010%2E)
21. N. Pinto, D.D. Cox, and J.J. DiCarlo. Why is real-world visual object recognition hard? PLoS computational biology, 4(1):e27, 2008. [find](https://scholar.google.com/scholar?q=N%2E+Pinto%2C+D%2ED%2E+Cox%2C+and+J%2EJ%2E+DiCarlo%2E+Why+is+real%2Dworld+visual+object+recognition+hard%3F+PLoS+computational+biology%2C+4%281%29%3Ae27%2C+2008%2E)
22. N. Pinto, D. Doukhan, J.J. DiCarlo, and D.D. Cox. A high-throughput screening approach to discovering good forms of biologically inspired visual representation. PLoS computational biology, 5(11):e1000579, 2009. [find](https://scholar.google.com/scholar?q=N%2E+Pinto%2C+D%2E+Doukhan%2C+J%2EJ%2E+DiCarlo%2C+and+D%2ED%2E+Cox%2E+A+high%2Dthroughput+screening+approach+to+discovering+good+forms+of+biologically+inspired+visual+representation%2E+PLoS+computational+biology%2C+5%2811%29%3Ae1000)
23. B.C. Russell, A. Torralba, K.P. Murphy, and W.T. Freeman. Labelme: a database and web-based tool for image annotation. International journal of computer vision, 77(1):157–173, 2008. [find](https://scholar.google.com/scholar?q=B%2EC%2E+Russell%2C+A%2E+Torralba%2C+K%2EP%2E+Murphy%2C+and+W%2ET%2E+Freeman%2E+Labelme%3A+a+database+and+web%2Dbased+tool+for+image+annotation%2E+International+journal+of+computer+vision%2C+77%281%29%3A157%E2%80%93173%2C+2008%2E)
24. J. Sánchez and F. Perronnin. High-dimensional signature compression for large-scale image classification. In Computer Vision and Pattern Recognition (CVPR), 2011 IEEE Conference on, pages 1665–1672. IEEE, 2011. [find](https://scholar.google.com/scholar?q=J%2E+S%C3%A1nchez+and+F%2E+Perronnin%2E+High%2Ddimensional+signature+compression+for+large%2Dscale+image+classification%2E+In+Computer+Vision+and+Pattern+Recognition+%28CVPR%29%2C+2011+IEEE+Conference+on%2C+pages+1665%E2%80%931672%2E+I)
25. P.Y. Simard, D. Steinkraus, and J.C. Platt. Best practices for convolutional neural networks applied to visual document analysis. In Proceedings of the Seventh International Conference on Document Analysis and Recognition, volume 2, pages 958–962, 2003. [find](https://scholar.google.com/scholar?q=P%2EY%2E+Simard%2C+D%2E+Steinkraus%2C+and+J%2EC%2E+Platt%2E+Best+practices+for+convolutional+neural+networks+applied+to+visual+document+analysis%2E+In+Proceedings+of+the+Seventh+International+Conference+on+Document+Ana)
26. S.C. Turaga, J.F. Murray, V. Jain, F. Roth, M. Helmstaedter, K. Briggman, W. Denk, and H.S. Seung. Convolutional networks can learn to generate affinity graphs for image segmentation. Neural Computation, 22(2):511–538, 2010. [find](https://scholar.google.com/scholar?q=S%2EC%2E+Turaga%2C+J%2EF%2E+Murray%2C+V%2E+Jain%2C+F%2E+Roth%2C+M%2E+Helmstaedter%2C+K%2E+Briggman%2C+W%2E+Denk%2C+and+H%2ES%2E+Seung%2E+Convolutional+networks+can+learn+to+generate+affinity+graphs+for+image+segmentation%2E+Neural+Computati)

*Updated 29 October 2026*