--- title: "Learning multiple visual domains with — Rebuffi 2017" description: "The paper asks whether one deep network can serve many visually unrelated image domains, such as handwritten characters, traffic signs and flower…" authors: ["Sylvestre-Alvise Rebuffi", "Hakan Bilen", "Andrea Vedaldi"] year: 2017 venue: "arXiv:1705.08045" source: "https://arxiv.org/abs/1705.08045" doi: "10.48550/arxiv.1705.08045" arxiv: "1705.08045" licence: "arXiv — non-exclusive license to distribute" licence_url: "http://arxiv.org/licenses/nonexclusive-distrib/1.0/" summary_licence: "Elamur Cosmic License" canonical: "https://research.elamur.ai/free/en/papers/rebuffi-2017-learning-multiple-visual-domains-residual/" --- [Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/rebuffi-2017-learning-multiple-visual-domains-residual/) [home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2017](https://md.research.elamur.ai/free/en/papers/2017/page/2/) / Learning multiple visual domains with residual adapters # Learning multiple visual domains with residual adapters **Authors:** Sylvestre-Alvise Rebuffi, Hakan Bilen, [Andrea Vedaldi](https://md.research.elamur.ai/free/en/authors/andrea-vedaldi/) · **Year:** 2017 · **Venue:** arXiv:1705.08045 · **Licence:** [arXiv — non-exclusive license to distribute](http://arxiv.org/licenses/nonexclusive-distrib/1.0/) **Links:** [Publisher's page](https://arxiv.org/abs/1705.08045) · [PDF](https://arxiv.org/pdf/1705.08045) · [DOI 10.48550/arxiv.1705.08045](https://doi.org/10.48550/arxiv.1705.08045) · [arXiv 1705.08045](https://arxiv.org/abs/1705.08045) · [OpenAlex](https://openalex.org/W2616957565) **Citations:** 514 citations (openalex-snapshot · 2026-10-14) · [156](#cited-by) in our library, of the 140673 papers whose bibliographies we read ## Summary The paper asks whether one deep network can serve many visually unrelated image domains, such as handwritten characters, traffic signs and flower species, without keeping a separate full model for each. The authors start from ResNet and insert small residual adapter modules. These are lightweight domain-specific layers placed beside large shared filters, and they hold less than 10% of the parameters. They show that schemes in which one network predicts another network's weights reduce to this kind of adapter layer. Training proceeds as follows: - The network and adapters are first pretrained on ImageNet. - The shared weights are then frozen, and only the adapters are trained for each new domain, so earlier domains are not forgotten. - Optionally, all ten tasks are trained jointly end-to-end. To evaluate the approach, they introduce the Visual Decathlon Challenge, a ten-dataset benchmark. Its score rewards models that do well on every domain rather than just on average. With weight decay tuned per domain, adapters match or beat full fine-tuning while using about twice the capacity of a single ResNet. This makes compact multi-domain models practical. ## Key points - **The Visual Decathlon benchmark** covers ten datasets: Aircraft, CIFAR-100, Daimler Pedestrians, Describable Textures (DTD), German Traffic Signs, ILSVRC 2012 (ImageNet), VGG-Flowers, OmniGlot, SVHN and UCF101 Dynamic Images. The maximum score is 10,000 points. Ten separately fine-tuned networks score 2,500. - **Tuned residual adapters score 2643 points at 2× the size of one ResNet.** With a single shared weight decay, average accuracy is still high (73.88%), but the score falls to 2118 because DTD and Flowers perform poorly. - **Weight decay should depend on dataset size.** Higher decay (0.002 or 0.005, versus the default 0.0005) helps small datasets like Flowers, while larger datasets prefer the lower value. - **Controls perform worse.** Fine-tuning only the last two convolutional layers reaches 64.7% mean accuracy, versus 73.9% for adapters. Adapters without the residual connection reach 70.3% and converge more slowly. - **Domain labels are not required at test time.** A small ResNet predicts the domain with 99.8% accuracy, causing only a marginal drop. A larger adapter model built on ResNet28 (12× capacity) beats fine-tuning by about 600 points and exceeds the prior CIFAR-100 state of the art (81.2%) by 3.5 points. *Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.* Fields: [Other](https://md.research.elamur.ai/free/en/shelves/other/) ## Cited by Cited by 156 papers of our library cite it (of the 140673 whose bibliographies we read); 13 of them have a page here, 143 not yet. - [On the Cross-lingual Transferability of Monolingual Representations](https://md.research.elamur.ai/free/en/papers/artetxe-2020-cross-lingual-transferability-monolingual-representations/) - [Simple, Scalable Adaptation for Neural Machine Translation](https://md.research.elamur.ai/free/en/papers/bapna-2019-simple-scalable-adaptation-neural-machine/) - [StyleRemix: Interpretable Authorship Obfuscation via Distillation and Perturbation of Style Elements](https://md.research.elamur.ai/free/en/papers/fisher-2024-styleremix-interpretable-authorship-obfuscation-distillation/) - [Few-Shot Parameter-Efficient Fine-Tuning is Better and Cheaper than In-Context Learning](https://md.research.elamur.ai/free/en/papers/liu-2022-few-shot-parameter-efficient-fine/) - [Noisy Channel Language Model Prompting for Few-Shot Text Classification](https://md.research.elamur.ai/free/en/papers/min-2021-noisy-channel-language-model-prompting/) - [AdapterFusion: Non-Destructive Task Composition for Transfer Learning](https://md.research.elamur.ai/free/en/papers/pfeiffer-2020-adapterfusion-non-destructive-task-composition/) - [AdapterHub: A Framework for Adapting Transformers](https://md.research.elamur.ai/free/en/papers/pfeiffer-2020-adapterhub-framework-adapting-transformers/) - [MAD-X: An Adapter-Based Framework for Multi-Task Cross-Lingual Transfer](https://md.research.elamur.ai/free/en/papers/pfeiffer-2020-mad-x-adapter-based-framework/) - [UNKs Everywhere: Adapting Multilingual Language Models to New Scripts](https://md.research.elamur.ai/free/en/papers/pfeiffer-2021-unks-everywhere-adapting-multilingual-language/) - [Lifting the Curse of Multilinguality by Pre-training Modular Transformers](https://md.research.elamur.ai/free/en/papers/pfeiffer-2022-lifting-curse-multilinguality-pre-training/) - [A Parameter-Efficient Learning Approach to Arabic Dialect Identification with Pre-Trained General-Purpose Speech Model](https://md.research.elamur.ai/free/en/papers/radhakrishnan-2023-parameter-efficient-learning-approach-arabic/) - [On Negative Interference in Multilingual Models: Findings and A Meta-Learning Treatment](https://md.research.elamur.ai/free/en/papers/wang-2020-negative-interference-multilingual-models-findings/) - [AdaLoRA: Adaptive Budget Allocation for Parameter-Efficient Fine-Tuning](https://md.research.elamur.ai/free/en/papers/zhang-2023-adalora-adaptive-budget-allocation-parameter/) ## Cite The paper itself, not this page. BibTeX from the library's record of the work. ```bibtex @misc{rebuffi2017learning, title = {{Learning multiple visual domains with residual adapters}}, author = {Rebuffi, Sylvestre-Alvise and Bilen, Hakan and Vedaldi, Andrea}, year = {2017}, doi = {10.48550/arxiv.1705.08045}, eprint = {1705.08045}, archivePrefix = {arXiv}, url = {https://doi.org/10.48550/arxiv.1705.08045} } ``` - [← previous Learning Joint Multilingual Sentence Representations with Neural Machine Translation](https://md.research.elamur.ai/free/en/papers/schwenk-2017-learning-joint-multilingual-sentence-representations/) - [next → Learning to Ask: Neural Question Generation for Reading Comprehension](https://md.research.elamur.ai/free/en/papers/du-2017-learning-ask-neural-question-generation/) - [↑ back to papers of 2017](https://md.research.elamur.ai/free/en/papers/2017/page/2/) ## References 42 references, 2 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet. 1. A. Argyriou, T. Evgeniou, and M Pontil. Multi-task feature learning. In Proc. NIPS, volume 19, page 41. MIT; 1998, 2007. [find](https://scholar.google.com/scholar?q=A%2E+Argyriou%2C+T%2E+Evgeniou%2C+and+M+Pontil%2E+Multi%2Dtask+feature+learning%2E+In+Proc%2E+NIPS%2C+volume+19%2C+page+41%2E+MIT%3B+1998%2C+2007%2E) 2. L. Bertinetto, J. F. Henriques, J. Valmadre, P. Torr, and A. Vedaldi. Learning feed-forward one-shot learners. In Proc. NIPS, pages 523–531, 2016. [find](https://scholar.google.com/scholar?q=L%2E+Bertinetto%2C+J%2E+F%2E+Henriques%2C+J%2E+Valmadre%2C+P%2E+Torr%2C+and+A%2E+Vedaldi%2E+Learning+feed%2Dforward+one%2Dshot+learners%2E+In+Proc%2E+NIPS%2C+pages+523%E2%80%93531%2C+2016%2E) 3. H. Bilen, B. Fernando, E. Gavves, A. Vedaldi, and S. Gould. Dynamic image networks for action recognition. In Proc. CVPR, 2016. [find](https://scholar.google.com/scholar?q=H%2E+Bilen%2C+B%2E+Fernando%2C+E%2E+Gavves%2C+A%2E+Vedaldi%2C+and+S%2E+Gould%2E+Dynamic+image+networks+for+action+recognition%2E+In+Proc%2E+CVPR%2C+2016%2E) 4. H. Bilen and A. Vedaldi. Integrated perception with recurrent multi-task neural networks. In Proc. NIPS, 2016. [find](https://scholar.google.com/scholar?q=H%2E+Bilen+and+A%2E+Vedaldi%2E+Integrated+perception+with+recurrent+multi%2Dtask+neural+networks%2E+In+Proc%2E+NIPS%2C+2016%2E) 5. H. Bilen and A. Vedaldi. Universal representations: The missing link between faces, text, planktons, and cat breeds. arXiv preprint arXiv:1701.07275, 2017. — [arXiv:1701.07275](https://arxiv.org/abs/1701.07275) [find](https://scholar.google.com/scholar?q=H%2E+Bilen+and+A%2E+Vedaldi%2E+Universal+representations%3A+The+missing+link+between+faces%2C+text%2C+planktons%2C+and+cat+breeds%2E+arXiv+preprint+arXiv%3A1701%2E07275%2C+2017%2E+%E2%80%94+%5BarXiv%3A1701%2E07275%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F1) 6. R. Caruana. Multitask learning. Machine Learning, 28, 1997. [find](https://scholar.google.com/scholar?q=R%2E+Caruana%2E+Multitask+learning%2E+Machine+Learning%2C+28%2C+1997%2E) 7. M. Cimpoi, S. Maji, I. Kokkinos, S. Mohamed, and A. Vedaldi. Describing textures in the wild. In Proc. CVPR, 2014. [find](https://scholar.google.com/scholar?q=M%2E+Cimpoi%2C+S%2E+Maji%2C+I%2E+Kokkinos%2C+S%2E+Mohamed%2C+and+A%2E+Vedaldi%2E+Describing+textures+in+the+wild%2E+In+Proc%2E+CVPR%2C+2014%2E) 8. R. Collobert and J. Weston. A unified architecture for natural language processing: Deep neural networks with multitask learning. In icml, pages 160–167. ACM, 2008. [find](https://scholar.google.com/scholar?q=R%2E+Collobert+and+J%2E+Weston%2E+A+unified+architecture+for+natural+language+processing%3A+Deep+neural+networks+with+multitask+learning%2E+In+icml%2C+pages+160%E2%80%93167%2E+ACM%2C+2008%2E) 9. H. Daumé III. Frustratingly easy domain adaptation. ACL 2007, page 256, 2007. [find](https://scholar.google.com/scholar?q=H%2E+Daum%C3%A9+III%2E+Frustratingly+easy+domain+adaptation%2E+ACL+2007%2C+page+256%2C+2007%2E) 10. T. Evgeniou and M. Pontil. Regularized multi–task learning. In SIGKDD, pages 109–117. ACM, 2004. [find](https://scholar.google.com/scholar?q=T%2E+Evgeniou+and+M%2E+Pontil%2E+Regularized+multi%E2%80%93task+learning%2E+In+SIGKDD%2C+pages+109%E2%80%93117%2E+ACM%2C+2004%2E) 11. R. M. French. Catastrophic forgetting in connectionist networks. Trends in cognitive sciences, 3(4):128–135, 1999. [find](https://scholar.google.com/scholar?q=R%2E+M%2E+French%2E+Catastrophic+forgetting+in+connectionist+networks%2E+Trends+in+cognitive+sciences%2C+3%284%29%3A128%E2%80%93135%2C+1999%2E) 12. Y. Ganin and V. Lempitsky. Unsupervised domain adaptation by backpropagation. Proc. ICML, 2015. [find](https://scholar.google.com/scholar?q=Y%2E+Ganin+and+V%2E+Lempitsky%2E+Unsupervised+domain+adaptation+by+backpropagation%2E+Proc%2E+ICML%2C+2015%2E) 13. K. He, X. Zhang, S. Ren, and J. Sun. Identity mappings in deep residual networks. In Proc. ECCV, pages 630–645. Springer, 2016. [find](https://scholar.google.com/scholar?q=K%2E+He%2C+X%2E+Zhang%2C+S%2E+Ren%2C+and+J%2E+Sun%2E+Identity+mappings+in+deep+residual+networks%2E+In+Proc%2E+ECCV%2C+pages+630%E2%80%93645%2E+Springer%2C+2016%2E) 14. J. T. Huang, J. Li, D. Yu, L. Deng, and Y. Gong. Cross-language knowledge transfer using multilingual deep neural network with shared hidden layers. In ICASSP, pages 7304–7308, 2013. [find](https://scholar.google.com/scholar?q=J%2E+T%2E+Huang%2C+J%2E+Li%2C+D%2E+Yu%2C+L%2E+Deng%2C+and+Y%2E+Gong%2E+Cross%2Dlanguage+knowledge+transfer+using+multilingual+deep+neural+network+with+shared+hidden+layers%2E+In+ICASSP%2C+pages+7304%E2%80%937308%2C+2013%2E) 15. S. Ioffe and C. Szegedy. Batch normalization: Accelerating deep network training by reducing internal covariate shift. CoRR, 2015. [find](https://scholar.google.com/scholar?q=S%2E+Ioffe+and+C%2E+Szegedy%2E+Batch+normalization%3A+Accelerating+deep+network+training+by+reducing+internal+covariate+shift%2E+CoRR%2C+2015%2E) 16. X. Jia, B. De Brabandere, T. Tuytelaars, and L. Gool. Dynamic filter networks. In Proc. NIPS, pages 667–675, 2016. [find](https://scholar.google.com/scholar?q=X%2E+Jia%2C+B%2E+De+Brabandere%2C+T%2E+Tuytelaars%2C+and+L%2E+Gool%2E+Dynamic+filter+networks%2E+In+Proc%2E+NIPS%2C+pages+667%E2%80%93675%2C+2016%2E) 17. J. Kirkpatrick, E. Pascanu, N. Rabinowitz, J. Veness, G. Desjardins, A. A. Rusu, K. Milan, J. Quan, T. Ramalho, A. Grabska-Barwinska, et al. Overcoming catastrophic forgetting in neural networks. National Academy of Sciences, 2017. [find](https://scholar.google.com/scholar?q=J%2E+Kirkpatrick%2C+E%2E+Pascanu%2C+N%2E+Rabinowitz%2C+J%2E+Veness%2C+G%2E+Desjardins%2C+A%2E+A%2E+Rusu%2C+K%2E+Milan%2C+J%2E+Quan%2C+T%2E+Ramalho%2C+A%2E+Grabska%2DBarwinska%2C+et+al%2E+Overcoming+catastrophic+forgetting+in+neural+networks%2E+Nati) 18. I. Kokkinos. Ubernet: Training auniversal’convolutional neural network for low-, mid-, and high-level vision using diverse datasets and limited memory. Proc. CVPR, 2017. [find](https://scholar.google.com/scholar?q=I%2E+Kokkinos%2E+Ubernet%3A+Training+auniversal%E2%80%99convolutional+neural+network+for+low%2D%2C+mid%2D%2C+and+high%2Dlevel+vision+using+diverse+datasets+and+limited+memory%2E+Proc%2E+CVPR%2C+2017%2E) 19. A. Krizhevsky and G. Hinton. Learning multiple layers of features from tiny images. 2009. [find](https://scholar.google.com/scholar?q=A%2E+Krizhevsky+and+G%2E+Hinton%2E+Learning+multiple+layers+of+features+from+tiny+images%2E+2009%2E) 20. B. M. Lake, R. Salakhutdinov, and J. B. Tenenbaum. Human-level concept learning through probabilistic program induction. Science, 350(6266):1332–1338, 2015. [find](https://scholar.google.com/scholar?q=B%2E+M%2E+Lake%2C+R%2E+Salakhutdinov%2C+and+J%2E+B%2E+Tenenbaum%2E+Human%2Dlevel+concept+learning+through+probabilistic+program+induction%2E+Science%2C+350%286266%29%3A1332%E2%80%931338%2C+2015%2E) 21. Z. Li and D. Hoiem. Learning without forgetting. In Proc. ECCV, pages 614–629, 2016. [find](https://scholar.google.com/scholar?q=Z%2E+Li+and+D%2E+Hoiem%2E+Learning+without+forgetting%2E+In+Proc%2E+ECCV%2C+pages+614%E2%80%93629%2C+2016%2E) 22. X. Liu, J. Gao, X. He, L. Deng, K. Duh, and Y. Wang. Representation learning using multi-task deep neural networks for semantic classification and information retrieval. In HLT-NAACL, pages 912–921, 2015. [find](https://scholar.google.com/scholar?q=X%2E+Liu%2C+J%2E+Gao%2C+X%2E+He%2C+L%2E+Deng%2C+K%2E+Duh%2C+and+Y%2E+Wang%2E+Representation+learning+using+multi%2Dtask+deep+neural+networks+for+semantic+classification+and+information+retrieval%2E+In+HLT%2DNAACL%2C+pages+912%E2%80%93921%2C+2) [our card](https://md.research.elamur.ai/free/en/papers/liu-2015-representation-learning-multi-task-deep/) 23. M. Long, H. Zhu, J. Wang, and M. I. Jordan. Unsupervised Domain Adaptation with Residual Transfer Networks. In Proc. NIPS, pages 136–144, 2016. [find](https://scholar.google.com/scholar?q=M%2E+Long%2C+H%2E+Zhu%2C+J%2E+Wang%2C+and+M%2E+I%2E+Jordan%2E+Unsupervised+Domain+Adaptation+with+Residual+Transfer+Networks%2E+In+Proc%2E+NIPS%2C+pages+136%E2%80%93144%2C+2016%2E) 24. S. Maji, J. Kannala, E. Rahtu, M. Blaschko, and A. Vedaldi. Fine-grained visual classification of aircraft. Technical report, 2013. [find](https://scholar.google.com/scholar?q=S%2E+Maji%2C+J%2E+Kannala%2C+E%2E+Rahtu%2C+M%2E+Blaschko%2C+and+A%2E+Vedaldi%2E+Fine%2Dgrained+visual+classification+of+aircraft%2E+Technical+report%2C+2013%2E) 25. T. Mitchell. Never-ending learning. Technical report, DTIC Document, 2010. [find](https://scholar.google.com/scholar?q=T%2E+Mitchell%2E+Never%2Dending+learning%2E+Technical+report%2C+DTIC+Document%2C+2010%2E) 26. S. Munder and D. M. Gavrila. An experimental study on pedestrian classification. PAMI, 28(11):1863–1868, 2006. [find](https://scholar.google.com/scholar?q=S%2E+Munder+and+D%2E+M%2E+Gavrila%2E+An+experimental+study+on+pedestrian+classification%2E+PAMI%2C+28%2811%29%3A1863%E2%80%931868%2C+2006%2E) 27. Y. Netzer, T. Wang, A. Coates, A. Bissacco, B. Wu, and A. Y. Ng. Reading digits in natural images with unsupervised feature learning. In NIPS Workshop on Deep Learning and Unsupervised Feature Learning, 2011. [find](https://scholar.google.com/scholar?q=Y%2E+Netzer%2C+T%2E+Wang%2C+A%2E+Coates%2C+A%2E+Bissacco%2C+B%2E+Wu%2C+and+A%2E+Y%2E+Ng%2E+Reading+digits+in+natural+images+with+unsupervised+feature+learning%2E+In+NIPS+Workshop+on+Deep+Learning+and+Unsupervised+Feature+Learnin) 28. M-E. Nilsback and A. Zisserman. Automated flower classification over a large number of classes. In ICCVGIP, Dec 2008. [find](https://scholar.google.com/scholar?q=M%2DE%2E+Nilsback+and+A%2E+Zisserman%2E+Automated+flower+classification+over+a+large+number+of+classes%2E+In+ICCVGIP%2C+Dec+2008%2E) 29. Ali Sharif Razavian, Hossein Azizpour, Josephine Sullivan, and Stefan Carlsson. CNN features off-the-shelf: an astounding baseline for recognition. In CVPR DeepVision Workshop, 2014. [find](https://scholar.google.com/scholar?q=Ali+Sharif+Razavian%2C+Hossein+Azizpour%2C+Josephine+Sullivan%2C+and+Stefan+Carlsson%2E+CNN+features+off%2Dthe%2Dshelf%3A+an+astounding+baseline+for+recognition%2E+In+CVPR+DeepVision+Workshop%2C+2014%2E) 30. S. A. Rebuffi, A. Kolesnikov, G. Sperl, and C. H. Lampert. icarl: Incremental classifier and representation learning. In Proc. CVPR, 2017. [find](https://scholar.google.com/scholar?q=S%2E+A%2E+Rebuffi%2C+A%2E+Kolesnikov%2C+G%2E+Sperl%2C+and+C%2E+H%2E+Lampert%2E+icarl%3A+Incremental+classifier+and+representation+learning%2E+In+Proc%2E+CVPR%2C+2017%2E) 31. Amir Rosenfeld and John K Tsotsos. Incremental learning through deep adaptation. arXiv preprint arXiv:1705.04228, 2017. — [arXiv:1705.04228](https://arxiv.org/abs/1705.04228) [find](https://scholar.google.com/scholar?q=Amir+Rosenfeld+and+John+K+Tsotsos%2E+Incremental+learning+through+deep+adaptation%2E+arXiv+preprint+arXiv%3A1705%2E04228%2C+2017%2E+%E2%80%94+%5BarXiv%3A1705%2E04228%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F1705%2E04228%29) 32. O. Russakovsky, J. Deng, H. Su, J. Krause, S. Satheesh, S. Ma, Z. Huang, A. Karpathy, A. Khosla, M. Bernstein, A. C. Berg, and K. Fei-Fei. Imagenet large scale visual recognition challenge, 2014. [find](https://scholar.google.com/scholar?q=O%2E+Russakovsky%2C+J%2E+Deng%2C+H%2E+Su%2C+J%2E+Krause%2C+S%2E+Satheesh%2C+S%2E+Ma%2C+Z%2E+Huang%2C+A%2E+Karpathy%2C+A%2E+Khosla%2C+M%2E+Bernstein%2C+A%2E+C%2E+Berg%2C+and+K%2E+Fei%2DFei%2E+Imagenet+large+scale+visual+recognition+challenge%2C+2014%2E) [our card](https://md.research.elamur.ai/free/en/papers/russakovsky-2014-imagenet-large-scale-visual-recognition/) 33. A. A. Rusu, N. C. Rabinowitz, G. Desjardins, H. Soyer, J. Kirkpatrick, K. Kavukcuoglu, R. Pascanu, and R. Hadsell. Progressive neural networks. arXiv preprint arXiv:1606.04671, 2016. [find](https://scholar.google.com/scholar?q=A%2E+A%2E+Rusu%2C+N%2E+C%2E+Rabinowitz%2C+G%2E+Desjardins%2C+H%2E+Soyer%2C+J%2E+Kirkpatrick%2C+K%2E+Kavukcuoglu%2C+R%2E+Pascanu%2C+and+R%2E+Hadsell%2E+Progressive+neural+networks%2E+arXiv+preprint+arXiv%3A1606%2E04671%2C+2016%2E) 34. J. Schmidhuber. Learning to control fast-weight memories: An alternative to dynamic recurrent networks. Neural Computation, 4(1):131–139, 1992. [find](https://scholar.google.com/scholar?q=J%2E+Schmidhuber%2E+Learning+to+control+fast%2Dweight+memories%3A+An+alternative+to+dynamic+recurrent+networks%2E+Neural+Computation%2C+4%281%29%3A131%E2%80%93139%2C+1992%2E) 35. K. Soomro, A. R. Zamir, and M. Shah. Ucf101: A dataset of 101 human actions classes from videos in the wild. arXiv preprint arXiv:1212.0402, 2012. — [arXiv:1212.0402](https://arxiv.org/abs/1212.0402) [find](https://scholar.google.com/scholar?q=K%2E+Soomro%2C+A%2E+R%2E+Zamir%2C+and+M%2E+Shah%2E+Ucf101%3A+A+dataset+of+101+human+actions+classes+from+videos+in+the+wild%2E+arXiv+preprint+arXiv%3A1212%2E0402%2C+2012%2E+%E2%80%94+%5BarXiv%3A1212%2E0402%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F1212%2E0402%29) 36. J. Stallkamp, M. Schlipsing, J. Salmen, and C. Igel. Man vs. computer: Benchmarking machine learning algorithms for traffic sign recognition. Neural Networks, 32(0):323–332, 2012. [find](https://scholar.google.com/scholar?q=J%2E+Stallkamp%2C+M%2E+Schlipsing%2C+J%2E+Salmen%2C+and+C%2E+Igel%2E+Man+vs%2E+computer%3A+Benchmarking+machine+learning+algorithms+for+traffic+sign+recognition%2E+Neural+Networks%2C+32%280%29%3A323%E2%80%93332%2C+2012%2E) 37. A. V. Terekhov, G. Montone, and J. K. O’Regan. Knowledge transfer in deep block-modular neural networks. In Biomimetic and Biohybrid Systems, pages 268–279, 2015. [find](https://scholar.google.com/scholar?q=A%2E+V%2E+Terekhov%2C+G%2E+Montone%2C+and+J%2E+K%2E+O%E2%80%99Regan%2E+Knowledge+transfer+in+deep+block%2Dmodular+neural+networks%2E+In+Biomimetic+and+Biohybrid+Systems%2C+pages+268%E2%80%93279%2C+2015%2E) 38. S. Thrun. Lifelong learning algorithms. In Learning to learn, pages 181–209. Springer, 1998. [find](https://scholar.google.com/scholar?q=S%2E+Thrun%2E+Lifelong+learning+algorithms%2E+In+Learning+to+learn%2C+pages+181%E2%80%93209%2E+Springer%2C+1998%2E) 39. E. Tzeng, J. Hoffman, T. Darrell, and K. Saenko. Simultaneous deep transfer across domains and tasks. In Proc. CVPR, pages 4068–4076, 2015. [find](https://scholar.google.com/scholar?q=E%2E+Tzeng%2C+J%2E+Hoffman%2C+T%2E+Darrell%2C+and+K%2E+Saenko%2E+Simultaneous+deep+transfer+across+domains+and+tasks%2E+In+Proc%2E+CVPR%2C+pages+4068%E2%80%934076%2C+2015%2E) 40. Sergey Zagoruyko and Nikos Komodakis. arXiv:1605.07146, 2016. — [arXiv:1605.07146](https://arxiv.org/abs/1605.07146) [find](https://scholar.google.com/scholar?q=Sergey+Zagoruyko+and+Nikos+Komodakis%2E+arXiv%3A1605%2E07146%2C+2016%2E+%E2%80%94+%5BarXiv%3A1605%2E07146%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F1605%2E07146%29) 41. T. Zhang, B. Ghanem, S. Liu, and N. Ahuja. Robust visual tracking via structured multi-task sparse learning. IJCV, 101(2):367–383, 2013. [find](https://scholar.google.com/scholar?q=T%2E+Zhang%2C+B%2E+Ghanem%2C+S%2E+Liu%2C+and+N%2E+Ahuja%2E+Robust+visual+tracking+via+structured+multi%2Dtask+sparse+learning%2E+IJCV%2C+101%282%29%3A367%E2%80%93383%2C+2013%2E) 42. Z. Zhang, P. Luo, C. C. Loy, and X. Tang. Facial landmark detection by deep multi-task learning. In Proc. ECCV, 2014. Wide residual networks. arXiv preprint [find](https://scholar.google.com/scholar?q=Z%2E+Zhang%2C+P%2E+Luo%2C+C%2E+C%2E+Loy%2C+and+X%2E+Tang%2E+Facial+landmark+detection+by+deep+multi%2Dtask+learning%2E+In+Proc%2E+ECCV%2C+2014%2E+Wide+residual+networks%2E+arXiv+preprint)