---
title: "Dataset Distillation — paper by Wang et al. (2018)"
description: "Large training sets require many thousands of update steps. 54% accuracy. 54 references, cited by 76 papers here."
authors: ["Tongzhou Wang", "Jun-Yan Zhu", "Antonio Torralba", "Alexei A. Efros"]
year: 2018
venue: "arXiv:1811.10959"
source: "https://arxiv.org/abs/1811.10959"
doi: "10.48550/arxiv.1811.10959"
arxiv: "1811.10959"
licence: "arXiv — non-exclusive license to distribute"
licence_url: "http://arxiv.org/licenses/nonexclusive-distrib/1.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/wang-2018-dataset-distillation/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/wang-2018-dataset-distillation/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2018](https://md.research.elamur.ai/free/en/papers/2018/page/5/) / Dataset Distillation

# Dataset Distillation

**Authors:** [Tongzhou Wang](https://md.research.elamur.ai/free/en/authors/tongzhou-wang/), Jun-Yan Zhu, [Antonio Torralba](https://md.research.elamur.ai/free/en/authors/antonio-b-torralba/) and Alexei A. Efros · **Year:** 2018 · **Venue:** arXiv:1811.10959 · **Licence:** [arXiv — non-exclusive license to distribute](http://arxiv.org/licenses/nonexclusive-distrib/1.0/)

**Links:** [Publisher's page](https://arxiv.org/abs/1811.10959) · [PDF](https://arxiv.org/pdf/1811.10959) · [DOI 10.48550/arxiv.1811.10959](https://doi.org/10.48550/arxiv.1811.10959) · [arXiv 1811.10959](https://arxiv.org/abs/1811.10959) · [OpenAlex](https://openalex.org/W2902287157)

**Citations:** 21 citations (OpenAlex, 2026-10-14) · [76](https://md.research.elamur.ai/free/en/papers/wang-2018-dataset-distillation/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
Large training sets require many thousands of update steps. This paper asks whether their knowledge can instead be packed into a handful of synthetic examples. Unlike model distillation, which shrinks the network, dataset distillation keeps the architecture fixed and learns a tiny artificial training set that need not resemble real data. The authors express the network weights after one or more gradient descent steps as a differentiable function of the synthetic inputs and a learned step size. They then optimize the pixels directly, so that the updated model performs well on the full original data.

They first handle a known starting point. They then average the objective over a distribution of initializations, extend the approach to multiple steps and epochs, and derive a lower bound on distilled set size for a linear model. Tests cover classification, fine-tuning of pre-trained networks and data poisoning, and the method generally beats real-image baselines. The work matters for studying how compressible datasets are and for rapidly "loading" knowledge into a network.

## Key points
- On MNIST, 10 distilled images (one per class) train LeNet from a fixed initialization to 94% test accuracy, versus 99% with all 60,000 images. For random initializations, 100 images reach about 80%.
- On CIFAR10, 100 distilled images give 54% accuracy from a fixed initialization, compared with 80% with full training.
- 100 images capturing the SVHN-to-MNIST domain gap raise a pre-trained SVHN network from 52% to 85% MNIST accuracy.
- In the poisoning setting, 300 synthetic images applied in a single step drop accuracy on CIFAR10 "plane" from 82% to 7%.
- The evaluation uses MNIST, CIFAR10, PASCAL-VOC and CUB-200, with four baselines:
  - random real images
  - selected real images
  - k-means centroids
  - per-class mean images

  Training uses Adam, and a typical run takes 1–4 hours on Titan Xp/V100 GPUs.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [University of California, Berkeley](https://md.research.elamur.ai/free/en/universities/university-of-california-berkeley/) — University, Berkeley, United States · 209 works · 244,782 citations
- [Massachusetts Institute of Technology](https://md.research.elamur.ai/free/en/universities/massachusetts-institute-of-technology/) — University, Cambridge, United States · 243 works · 76,506 citations
- [MIT Computer Science and Artificial Intelligence Laboratory](https://md.research.elamur.ai/free/en/labs/mit-computer-science-and-artificial-intelligence-laboratory/) — Research institute, Cambridge, United States · 76 works · 32,207 citations

📚 Shelves (3): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 18,614 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 17,965 works · [👁️ Computer Vision and Pattern Recognition](https://md.research.elamur.ai/free/en/shelves/computer-science/computer-vision-and-pattern-recognition/) · 3,486 works

🧭 Topics (1): [🪶 Efficient models & compression](https://md.research.elamur.ai/free/en/topics/efficient-models/) · 297 works

## Cite

Cite as: Tongzhou Wang, Jun-Yan Zhu, Antonio Torralba and Alexei A. Efros. 2018. Dataset Distillation. arXiv:1811.10959.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@misc{wang2018dataset,
  title = {{Dataset Distillation}},
  author = {Tongzhou Wang, Jun-Yan Zhu, Antonio Torralba and Alexei A. Efros},
  year = {2018},
  doi = {10.48550/arxiv.1811.10959},
  eprint = {1811.10959},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.1811.10959}
}
```

- [← previous Pragmatically Informative Image Captioning with Character-Level Inference](https://md.research.elamur.ai/free/en/papers/gordon-2018-pragmatically-informative-image-captioning-character/)
- [next → Improving Implicit Discourse Relation Classification by Modeling Inter-dependencies of Discourse Units in a Paragraph](https://md.research.elamur.ai/free/en/papers/dai-2018-improving-implicit-discourse-relation-classification/)
- [↑ back to papers of 2018](https://md.research.elamur.ai/free/en/papers/2018/page/5/)

## References

54 references, 7 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Anelia Angelova, Yaser Abu-Mostafam, and Pietro Perona. Pruning training sets for learning of object categories. In CVPR, 2005. 3 [find](https://scholar.google.com/scholar?q=Anelia+Angelova%2C+Yaser+Abu%2DMostafam%2C+and+Pietro+Perona%2E+Pruning+training+sets+for+learning+of+object+categories%2E+In+CVPR%2C+2005%2E+3)
2. Jimmy Ba and Rich Caruana. Do deep nets really need to be deep? In NIPS, 2014. 2 [find](https://scholar.google.com/scholar?q=Jimmy+Ba+and+Rich+Caruana%2E+Do+deep+nets+really+need+to+be+deep%3F+In+NIPS%2C+2014%2E+2)
3. Olivier Bachem, Mario Lucic, and Andreas Krause. Practical coreset constructions for machine learning. arXiv preprint arXiv:1703.06476, 2017. 3 — [arXiv:1703.06476](https://arxiv.org/abs/1703.06476) [find](https://scholar.google.com/scholar?q=Olivier+Bachem%2C+Mario+Lucic%2C+and+Andreas+Krause%2E+Practical+coreset+constructions+for+machine+learning%2E+arXiv+preprint+arXiv%3A1703%2E06476%2C+2017%2E+3+%E2%80%94+%5BarXiv%3A1703%2E06476%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F1703%2E06476%29)
4. David Bau, Bolei Zhou, Aditya Khosla, Aude Oliva, and Antonio Torralba. Network dissection: Quantifying interpretability of deep visual representations. In CVPR, 2017. 3 [find](https://scholar.google.com/scholar?q=David+Bau%2C+Bolei+Zhou%2C+Aditya+Khosla%2C+Aude+Oliva%2C+and+Antonio+Torralba%2E+Network+dissection%3A+Quantifying+interpretability+of+deep+visual+representations%2E+In+CVPR%2C+2017%2E+3)
5. Yoshua Bengio. Gradient-based optimization of hyperparameters. Neural computation, 12(8):1889–1900, 2000. 3 [find](https://scholar.google.com/scholar?q=Yoshua+Bengio%2E+Gradient%2Dbased+optimization+of+hyperparameters%2E+Neural+computation%2C+12%288%29%3A1889%E2%80%931900%2C+2000%2E+3)
6. Battista Biggio, Blaine Nelson, and Pavel Laskov. Poisoning attacks against support vector machines. In ICML, 2012. 6 [find](https://scholar.google.com/scholar?q=Battista+Biggio%2C+Blaine+Nelson%2C+and+Pavel+Laskov%2E+Poisoning+attacks+against+support+vector+machines%2E+In+ICML%2C+2012%2E+6)
7. David A Cohn, Zoubin Ghahramani, and Michael I Jordan. Active learning with statistical models. Journal of artificial intelligence research, 4:129–145, 1996. 3 [find](https://scholar.google.com/scholar?q=David+A+Cohn%2C+Zoubin+Ghahramani%2C+and+Michael+I+Jordan%2E+Active+learning+with+statistical+models%2E+Journal+of+artificial+intelligence+research%2C+4%3A129%E2%80%93145%2C+1996%2E+3)
8. Hal Daume III. [Frustratingly easy domain adaptation](https://md.research.elamur.ai/free/en/papers/iii-2007-frustratingly-easy-domain-adaptation/). In ACL, 2007. 6
9. Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In CVPR, 2009. 6, 9 [find](https://scholar.google.com/scholar?q=Jia+Deng%2C+Wei+Dong%2C+Richard+Socher%2C+Li%2DJia+Li%2C+Kai+Li%2C+and+Li+Fei%2DFei%2E+Imagenet%3A+A+large%2Dscale+hierarchical+image+database%2E+In+CVPR%2C+2009%2E+6%2C+9)
10. Justin Domke. Generic methods for optimization-based modeling. In Artificial Intelligence and Statistics, pp. 318–326, 2012. 3, 5 [find](https://scholar.google.com/scholar?q=Justin+Domke%2E+Generic+methods+for+optimization%2Dbased+modeling%2E+In+Artificial+Intelligence+and+Statistics%2C+pp%2E+318%E2%80%93326%2C+2012%2E+3%2C+5)
11. Mark Everingham, Luc Van Gool, Christopher KI Williams, John Winn, and Andrew Zisserman. The pascal visual object classes (voc) challenge. IJCV, 88(2):303–338, 2010. 9 [find](https://scholar.google.com/scholar?q=Mark+Everingham%2C+Luc+Van+Gool%2C+Christopher+KI+Williams%2C+John+Winn%2C+and+Andrew+Zisserman%2E+The+pascal+visual+object+classes+%28voc%29+challenge%2E+IJCV%2C+88%282%29%3A303%E2%80%93338%2C+2010%2E+9)
12. Pedro F Felzenszwalb, Ross B Girshick, David McAllester, and Deva Ramanan. Object detection with discriminatively trained part-based models. PAMI, 32(9):1627–1645, 2010. 3 [find](https://scholar.google.com/scholar?q=Pedro+F+Felzenszwalb%2C+Ross+B+Girshick%2C+David+McAllester%2C+and+Deva+Ramanan%2E+Object+detection+with+discriminatively+trained+part%2Dbased+models%2E+PAMI%2C+32%289%29%3A1627%E2%80%931645%2C+2010%2E+3)
13. Xavier Glorot and Yoshua Bengio. Understanding the difficulty of training deep feedforward neural networks. In International conference on artificial intelligence and statistics, 2010. 6, 7 [find](https://scholar.google.com/scholar?q=Xavier+Glorot+and+Yoshua+Bengio%2E+Understanding+the+difficulty+of+training+deep+feedforward+neural+networks%2E+In+International+conference+on+artificial+intelligence+and+statistics%2C+2010%2E+6%2C+7)
14. Sally A Goldman and Michael J Kearns. On the complexity of teaching. Journal of Computer and System Sciences, 50(1):20–31, 1995. 3 [find](https://scholar.google.com/scholar?q=Sally+A+Goldman+and+Michael+J+Kearns%2E+On+the+complexity+of+teaching%2E+Journal+of+Computer+and+System+Sciences%2C+50%281%29%3A20%E2%80%9331%2C+1995%2E+3)
15. Sariel Har-Peled and Akash Kushal. Smaller coresets for k-median and k-means clustering. Discrete & Computational Geometry, 37(1):3–19, 2007. 3 [find](https://scholar.google.com/scholar?q=Sariel+Har%2DPeled+and+Akash+Kushal%2E+Smaller+coresets+for+k%2Dmedian+and+k%2Dmeans+clustering%2E+Discrete+%26+Computational+Geometry%2C+37%281%29%3A3%E2%80%9319%2C+2007%2E+3)
16. Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Delving deep into rectifiers: Surpassing human-level performance on imagenet classification. In ICCV, 2015. 6 [find](https://scholar.google.com/scholar?q=Kaiming+He%2C+Xiangyu+Zhang%2C+Shaoqing+Ren%2C+and+Jian+Sun%2E+Delving+deep+into+rectifiers%3A+Surpassing+human%2Dlevel+performance+on+imagenet+classification%2E+In+ICCV%2C+2015%2E+6)
17. Marti A. Hearst, Susan T Dumais, Edgar Osuna, John Platt, and Bernhard Scholkopf. Support vector machines. IEEE Intelligent Systems and their applications, 13(4):18–28, 1998. 3 [find](https://scholar.google.com/scholar?q=Marti+A%2E+Hearst%2C+Susan+T+Dumais%2C+Edgar+Osuna%2C+John+Platt%2C+and+Bernhard+Scholkopf%2E+Support+vector+machines%2E+IEEE+Intelligent+Systems+and+their+applications%2C+13%284%29%3A18%E2%80%9328%2C+1998%2E+3)
18. Geoffrey Hinton, Oriol Vinyals, and Jeffrey Dean. [Distilling the knowledge in a neural network](https://md.research.elamur.ai/free/en/papers/hinton-2015-distilling-knowledge-neural-network/). In NIPS Deep Learning and Representation Learning Workshop, 2015. 1, 2
19. Andrew G Howard, Menglong Zhu, Bo Chen, Dmitry Kalenichenko, Weijun Wang, Tobias Weyand, Marco Andreetto, and Hartwig Adam. [Mobilenets: Efficient convolutional neural networks for mobile vision applications](https://md.research.elamur.ai/free/en/papers/howard-2017-mobilenets-efficient-convolutional-neural-networks/). In CVPR, 2017. 2
20. Jonathan J. Hull. A database for handwritten text recognition research. PAMI, 16(5):550–554, 1994. 9 [find](https://scholar.google.com/scholar?q=Jonathan+J%2E+Hull%2E+A+database+for+handwritten+text+recognition+research%2E+PAMI%2C+16%285%29%3A550%E2%80%93554%2C+1994%2E+9)
21. Diederik Kingma and Jimmy Ba. [Adam: A method for stochastic optimization](https://md.research.elamur.ai/free/en/papers/kingma-2014-adam-method-stochastic-optimization/). In ICLR, 2015. 14
22. Pang Wei Koh and Percy Liang. [Understanding black-box predictions via influence functions](https://md.research.elamur.ai/free/en/papers/koh-2017-understanding-black-box-predictions-influence/). In ICML, 2017. 3, 6
23. Alex Krizhevsky. cuda-convnet: High-performance c++/cuda implementation of convolutional neural networks. [https://github.com/akrizhevsky/cuda-convnet2](https://github.com/akrizhevsky/cuda-convnet2), 2012. 7 [↗](https://github.com/akrizhevsky/cuda-convnet2)
24. Alex Krizhevsky and Geoffrey Hinton. Learning multiple layers of features from tiny images. Technical report, Citeseer, 2009. 7 [find](https://scholar.google.com/scholar?q=Alex+Krizhevsky+and+Geoffrey+Hinton%2E+Learning+multiple+layers+of+features+from+tiny+images%2E+Technical+report%2C+Citeseer%2C+2009%2E+7)
25. Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. [Imagenet classification with deep convolutional neural networks](https://md.research.elamur.ai/free/en/papers/krizhevsky-2012-imagenet-classification-deep-convolutional/). In NIPS, 2012. 6, 9
26. Agata Lapedriza, Hamed Pirsiavash, Zoya Bylinskii, and Antonio Torralba. Are all training examples equally valuable? arXiv preprint arXiv:1311.6510, 2013. 3 — [arXiv:1311.6510](https://arxiv.org/abs/1311.6510) [find](https://scholar.google.com/scholar?q=Agata+Lapedriza%2C+Hamed+Pirsiavash%2C+Zoya+Bylinskii%2C+and+Antonio+Torralba%2E+Are+all+training+examples+equally+valuable%3F+arXiv+preprint+arXiv%3A1311%2E6510%2C+2013%2E+3+%E2%80%94+%5BarXiv%3A1311%2E6510%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F1)
27. Yann LeCun. The mnist database of handwritten digits. [http://yann.lecun](http://yann.lecun). com/exdb/mnist/, 1998. 7 [↗](http://yann.lecun)
28. Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner. Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(11):2278–2324, 1998. 1, 7 [find](https://scholar.google.com/scholar?q=Yann+LeCun%2C+L%C3%A9on+Bottou%2C+Yoshua+Bengio%2C+and+Patrick+Haffner%2E+Gradient%2Dbased+learning+applied+to+document+recognition%2E+Proceedings+of+the+IEEE%2C+86%2811%29%3A2278%E2%80%932324%2C+1998%2E+1%2C+7)
29. Bo Li, Yining Wang, Aarti Singh, and Yevgeniy Vorobeychik. Data poisoning attacks on factorization-based collaborative filtering. In NIPS, 2016. 6 [find](https://scholar.google.com/scholar?q=Bo+Li%2C+Yining+Wang%2C+Aarti+Singh%2C+and+Yevgeniy+Vorobeychik%2E+Data+poisoning+attacks+on+factorization%2Dbased+collaborative+filtering%2E+In+NIPS%2C+2016%2E+6)
30. Raphael Gontijo Lopes, Stefano Fenu, and Thad Starner. Data-free knowledge distillation for deep neural networks. arXiv preprint arXiv:1710.07535, 2017. 3 — [arXiv:1710.07535](https://arxiv.org/abs/1710.07535) [find](https://scholar.google.com/scholar?q=Raphael+Gontijo+Lopes%2C+Stefano+Fenu%2C+and+Thad+Starner%2E+Data%2Dfree+knowledge+distillation+for+deep+neural+networks%2E+arXiv+preprint+arXiv%3A1710%2E07535%2C+2017%2E+3+%E2%80%94+%5BarXiv%3A1710%2E07535%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F17)
31. Dougal Maclaurin, David Duvenaud, and Ryan Adams. Gradient-based hyperparameter optimization through reversible learning. In ICML, 2015. 3, 5 [find](https://scholar.google.com/scholar?q=Dougal+Maclaurin%2C+David+Duvenaud%2C+and+Ryan+Adams%2E+Gradient%2Dbased+hyperparameter+optimization+through+reversible+learning%2E+In+ICML%2C+2015%2E+3%2C+5)
32. Aravindh Mahendran and Andrea Vedaldi. Understanding deep image representations by inverting them. In CVPR, 2015. 3 [find](https://scholar.google.com/scholar?q=Aravindh+Mahendran+and+Andrea+Vedaldi%2E+Understanding+deep+image+representations+by+inverting+them%2E+In+CVPR%2C+2015%2E+3)
33. Saeid Motiian, Quinn Jones, Seyed Iranmanesh, and Gianfranco Doretto. Few-shot adversarial domain adaptation. In NIPS, 2017. 9 [find](https://scholar.google.com/scholar?q=Saeid+Motiian%2C+Quinn+Jones%2C+Seyed+Iranmanesh%2C+and+Gianfranco+Doretto%2E+Few%2Dshot+adversarial+domain+adaptation%2E+In+NIPS%2C+2017%2E+9)
34. Luis Muñoz-González, Battista Biggio, Ambra Demontis, Andrea Paudice, Vasin Wongrassamee, Emil C Lupu, and Fabio Roli. Towards poisoning of deep learning algorithms with back-gradient optimization. In Proceedings of the 10th ACM Workshop on Artificial Intelligence and Security, pp. 27–38, 2017. 6 [find](https://scholar.google.com/scholar?q=Luis+Mu%C3%B1oz%2DGonz%C3%A1lez%2C+Battista+Biggio%2C+Ambra+Demontis%2C+Andrea+Paudice%2C+Vasin+Wongrassamee%2C+Emil+C+Lupu%2C+and+Fabio+Roli%2E+Towards+poisoning+of+deep+learning+algorithms+with+back%2Dgradient+optimization%2E+In)
35. Yuval Netzer, Tao Wang, Adam Coates, Alessandro Bissacco, Bo Wu, and Andrew Y Ng. Reading digits in natural images with unsupervised feature learning. In NIPS workshop, 2011. 9 [find](https://scholar.google.com/scholar?q=Yuval+Netzer%2C+Tao+Wang%2C+Adam+Coates%2C+Alessandro+Bissacco%2C+Bo+Wu%2C+and+Andrew+Y+Ng%2E+Reading+digits+in+natural+images+with+unsupervised+feature+learning%2E+In+NIPS+workshop%2C+2011%2E+9)
36. J Arturo Olvera-López, J Ariel Carrasco-Ochoa, J Francisco Martínez-Trinidad, and Josef Kittler. A review of instance selection methods. Artificial Intelligence Review, 34(2):133–143, 2010. 3 [find](https://scholar.google.com/scholar?q=J+Arturo+Olvera%2DL%C3%B3pez%2C+J+Ariel+Carrasco%2DOchoa%2C+J+Francisco+Mart%C3%ADnez%2DTrinidad%2C+and+Josef+Kittler%2E+A+review+of+instance+selection+methods%2E+Artificial+Intelligence+Review%2C+34%282%29%3A133%E2%80%93143%2C+2010%2E+3)
37. Adam Paszke, Sam Gross, Soumith Chintala, Gregory Chanan, Edward Yang, Zachary DeVito, Zeming Lin, Alban Desmaison, Luca Antiga, and Adam Lerer. Automatic differentiation in pytorch. In ICLR Workshop, 2017. 5 [find](https://scholar.google.com/scholar?q=Adam+Paszke%2C+Sam+Gross%2C+Soumith+Chintala%2C+Gregory+Chanan%2C+Edward+Yang%2C+Zachary+DeVito%2C+Zeming+Lin%2C+Alban+Desmaison%2C+Luca+Antiga%2C+and+Adam+Lerer%2E+Automatic+differentiation+in+pytorch%2E+In+ICLR+Workshop%2C)
38. Barak A Pearlmutter. Fast exact multiplication by the hessian. Neural computation, 6(1):147–160, 1994. 5 [find](https://scholar.google.com/scholar?q=Barak+A+Pearlmutter%2E+Fast+exact+multiplication+by+the+hessian%2E+Neural+computation%2C+6%281%29%3A147%E2%80%93160%2C+1994%2E+5)
39. Fabian Pedregosa. Hyperparameter optimization with approximate gradient. In ICML, 2016. 3 [find](https://scholar.google.com/scholar?q=Fabian+Pedregosa%2E+Hyperparameter+optimization+with+approximate+gradient%2E+In+ICML%2C+2016%2E+3)
40. Jean Ponce, Tamara L Berg, Mark Everingham, David A Forsyth, Martial Hebert, Svetlana Lazebnik, Marcin Marszalek, Cordelia Schmid, Bryan C Russell, Antonio Torralba, et al. Dataset issues in object recognition. In Toward category-level object recognition, pp. 29–48. 2006. 3 [find](https://scholar.google.com/scholar?q=Jean+Ponce%2C+Tamara+L+Berg%2C+Mark+Everingham%2C+David+A+Forsyth%2C+Martial+Hebert%2C+Svetlana+Lazebnik%2C+Marcin+Marszalek%2C+Cordelia+Schmid%2C+Bryan+C+Russell%2C+Antonio+Torralba%2C+et+al%2E+Dataset+issues+in+object+re)
41. Ilija Radosavovic, Piotr Dollár, Ross Girshick, Georgia Gkioxari, and Kaiming He. Data distillation: Towards omni-supervised learning. In CVPR, 2018. 2 [find](https://scholar.google.com/scholar?q=Ilija+Radosavovic%2C+Piotr+Doll%C3%A1r%2C+Ross+Girshick%2C+Georgia+Gkioxari%2C+and+Kaiming+He%2E+Data+distillation%3A+Towards+omni%2Dsupervised+learning%2E+In+CVPR%2C+2018%2E+2)
42. Adriana Romero, Nicolas Ballas, Samira Ebrahimi Kahou, Antoine Chassang, Carlo Gatta, and Yoshua Bengio. Fitnets: Hints for thin deep nets. In ICLR, 2015. 2 [find](https://scholar.google.com/scholar?q=Adriana+Romero%2C+Nicolas+Ballas%2C+Samira+Ebrahimi+Kahou%2C+Antoine+Chassang%2C+Carlo+Gatta%2C+and+Yoshua+Bengio%2E+Fitnets%3A+Hints+for+thin+deep+nets%2E+In+ICLR%2C+2015%2E+2)
43. Frank Rosenblatt. The perceptron, a perceiving and recognizing automaton Project Para. Cornell Aeronautical Laboratory, 1957. 3 [find](https://scholar.google.com/scholar?q=Frank+Rosenblatt%2E+The+perceptron%2C+a+perceiving+and+recognizing+automaton+Project+Para%2E+Cornell+Aeronautical+Laboratory%2C+1957%2E+3)
44. Kate Saenko, Brian Kulis, Mario Fritz, and Trevor Darrell. Adapting visual category models to new domains. In ECCV, 2010. 6 [find](https://scholar.google.com/scholar?q=Kate+Saenko%2C+Brian+Kulis%2C+Mario+Fritz%2C+and+Trevor+Darrell%2E+Adapting+visual+category+models+to+new+domains%2E+In+ECCV%2C+2010%2E+6)
45. Ozan Sener and Silvio Savarese. Active learning for convolutional neural networks: A core-set approach. In ICLR, 2018. 3 [find](https://scholar.google.com/scholar?q=Ozan+Sener+and+Silvio+Savarese%2E+Active+learning+for+convolutional+neural+networks%3A+A+core%2Dset+approach%2E+In+ICLR%2C+2018%2E+3)
46. Ayumi Shinohara and Satoru Miyano. Teachability in computational learning. New Generation Computing, 8(4): 337–347, 1991. 3 [find](https://scholar.google.com/scholar?q=Ayumi+Shinohara+and+Satoru+Miyano%2E+Teachability+in+computational+learning%2E+New+Generation+Computing%2C+8%284%29%3A+337%E2%80%93347%2C+1991%2E+3)
47. Simon Tong and Daphne Koller. Support vector machine active learning with applications to text classification. JMLR, 2(Nov):45–66, 2001. 3 [find](https://scholar.google.com/scholar?q=Simon+Tong+and+Daphne+Koller%2E+Support+vector+machine+active+learning+with+applications+to+text+classification%2E+JMLR%2C+2%28Nov%29%3A45%E2%80%9366%2C+2001%2E+3)
48. Antonio Torralba and Alexei A Efros. Unbiased look at dataset bias. In CVPR, pp. 1521–1528. IEEE, 2011. 3, 6 [find](https://scholar.google.com/scholar?q=Antonio+Torralba+and+Alexei+A+Efros%2E+Unbiased+look+at+dataset+bias%2E+In+CVPR%2C+pp%2E+1521%E2%80%931528%2E+IEEE%2C+2011%2E+3%2C+6)
49. Ivor W Tsang, James T Kwok, and Pak-Ming Cheung. Core vector machines: Fast svm training on very large data sets. JMLR, 6(Apr):363–392, 2005. 3 [find](https://scholar.google.com/scholar?q=Ivor+W+Tsang%2C+James+T+Kwok%2C+and+Pak%2DMing+Cheung%2E+Core+vector+machines%3A+Fast+svm+training+on+very+large+data+sets%2E+JMLR%2C+6%28Apr%29%3A363%E2%80%93392%2C+2005%2E+3)
50. C. Wah, S. Branson, P. Welinder, P. Perona, and S. Belongie. The Caltech-UCSD Birds-200-2011 Dataset. Technical Report CNS-TR-2011-001, California Institute of Technology, 2011. 9 [find](https://scholar.google.com/scholar?q=C%2E+Wah%2C+S%2E+Branson%2C+P%2E+Welinder%2C+P%2E+Perona%2C+and+S%2E+Belongie%2E+The+Caltech%2DUCSD+Birds%2D200%2D2011+Dataset%2E+Technical+Report+CNS%2DTR%2D2011%2D001%2C+California+Institute+of+Technology%2C+2011%2E+9)
51. Matthew D Zeiler and Rob Fergus. [Visualizing and understanding convolutional networks](https://md.research.elamur.ai/free/en/papers/zeiler-2013-visualizing-understanding-convolutional-networks/). In ECCV, 2014. 3
52. Bolei Zhou, Aditya Khosla, Agata Lapedriza, Aude Oliva, and Antonio Torralba. Object detectors emerge in deep scene cnns. In ICLR, 2015. 3 [find](https://scholar.google.com/scholar?q=Bolei+Zhou%2C+Aditya+Khosla%2C+Agata+Lapedriza%2C+Aude+Oliva%2C+and+Antonio+Torralba%2E+Object+detectors+emerge+in+deep+scene+cnns%2E+In+ICLR%2C+2015%2E+3)
53. Xiaojin Zhu. Machine teaching for bayesian learners in the exponential family. In NIPS, 2013. 3 [find](https://scholar.google.com/scholar?q=Xiaojin+Zhu%2E+Machine+teaching+for+bayesian+learners+in+the+exponential+family%2E+In+NIPS%2C+2013%2E+3)
54. Xiaojin Zhu. Machine teaching: An inverse problem to machine learning and an approach toward optimal education. In AAAI, 2015. 3 [find](https://scholar.google.com/scholar?q=Xiaojin+Zhu%2E+Machine+teaching%3A+An+inverse+problem+to+machine+learning+and+an+approach+toward+optimal+education%2E+In+AAAI%2C+2015%2E+3)

*Updated 2 November 2026*