--- title: "ViLT: Vision-and-Language Transformer Without — Kim 2021" description: "Most vision-and-language pre-training (VLP) systems turn images into inputs with heavy convolutional machinery, usually an object detector such as Faster…" authors: ["Wonjae Kim", "Bokyung Son", "Ildoo Kim"] year: 2021 venue: "arXiv:2102.03334" source: "https://arxiv.org/abs/2102.03334" doi: "10.48550/arxiv.2102.03334" arxiv: "2102.03334" licence: "arXiv — non-exclusive license to distribute" licence_url: "http://arxiv.org/licenses/nonexclusive-distrib/1.0/" summary_licence: "Elamur Cosmic License" canonical: "https://research.elamur.ai/free/en/papers/kim-2021-vilt-vision-language-transformer-convolution/" --- [Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/kim-2021-vilt-vision-language-transformer-convolution/) [home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2021](https://md.research.elamur.ai/free/en/papers/2021/page/5/) / ViLT: Vision-and-Language Transformer Without Convolution or Region Supervision # ViLT: Vision-and-Language Transformer Without Convolution or Region Supervision **Authors:** Wonjae Kim, Bokyung Son, Ildoo Kim · **Year:** 2021 · **Venue:** arXiv:2102.03334 · **Licence:** [arXiv — non-exclusive license to distribute](http://arxiv.org/licenses/nonexclusive-distrib/1.0/) **Links:** [Publisher's page](https://arxiv.org/abs/2102.03334) · [PDF](https://arxiv.org/pdf/2102.03334) · [DOI 10.48550/arxiv.2102.03334](https://doi.org/10.48550/arxiv.2102.03334) · [arXiv 2102.03334](https://arxiv.org/abs/2102.03334) · [OpenAlex](https://openalex.org/W3126792443) **Citations:** 520 citations (openalex-snapshot · 2026-10-14) · [242](#cited-by) in our library, of the 140884 papers whose bibliographies we read ## Summary Most vision-and-language pre-training (VLP) systems turn images into inputs with heavy convolutional machinery, usually an object detector such as Faster R-CNN or a ResNet backbone. The authors argue this has two costs. At inference time, extracting visual features takes far more compute than the cross-modal reasoning itself. The model's capacity is also capped by the detector and its fixed set of visual classes. They propose ViLT, a single-stream transformer that embeds images much as it embeds text: images are cut into patches, and each patch is linearly projected, with no CNN and no region supervision. Pre-training uses image-text matching and masked language modeling, plus a word-patch alignment signal. The authors also show for the first time that whole word masking and image augmentation help VLP. ViLT-B/32 matches or beats heavier models on NLVR2 and Flickr30k retrieval while running up to tens of times faster. The authors argue that VLP research should invest in modality interaction rather than ever larger visual encoders. ## Key points - **Visual embedding cost:** ViLT's patch projection takes about 0.4 ms. By comparison, region-feature pipelines like UNITER need roughly 75 ms for a ResNet-101 backbone plus around 810 ms for detector operations (RPN, RoI Align, NMS, RoI heads). - **Accuracy (NLVR2 test-P / Flickr30k TR R@1 / IR R@1):** - ViLT-B/32: 76.1 / 83.5 / 64.4 - UNITER-Base: 75.8 / 85.9 / 72.5 - Pixel-BERT-R50: 72.4 / 75.7 / 53.4 - **Input size:** ViLT works on 384×640 images and produces at most 240 visual tokens. Region-based models use 800×1,333 inputs. - **Shallow fusion fails:** A dot-product fusion of CLIP embeddings reached only about 51% dev accuracy on NLVR2, which is close to chance. This supports the need for deep cross-modal interaction. - **Masked patch prediction:** Predicting the mean RGB value of masked patches (MPP) did not improve downstream results. Code and weights are released at github.com/dandelin/vilt. *Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.* Fields: [Other](https://md.research.elamur.ai/free/en/shelves/other/) ## Cited by Cited by 242 papers of our library cite it (of the 140884 whose bibliographies we read); 16 of them have a page here, 226 not yet. - [Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyond](https://md.research.elamur.ai/free/en/papers/bai-2023-qwen-vl-versatile-vision-language/) - [On the Opportunities and Risks of Foundation Models](https://md.research.elamur.ai/free/en/papers/bommasani-2021-opportunities-risks-foundation-models/) - [Bridging the Gap between Synthetic and Authentic Images for Multimodal Machine Translation](https://md.research.elamur.ai/free/en/papers/guo-2023-bridging-gap-between-synthetic-authentic/) - [LayoutLMv3: Pre-training for Document AI with Unified Text and Image Masking](https://md.research.elamur.ai/free/en/papers/huang-2022-layoutlmv3-pre-training-document-ai/) - [OCR-free Document Understanding Transformer](https://md.research.elamur.ai/free/en/papers/kim-2021-ocr-free-document-understanding-transformer/) - [BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation](https://md.research.elamur.ai/free/en/papers/li-2022-blip-bootstrapping-language-image-pre/) - [Multi-modal Stance Detection: New Datasets and Model](https://md.research.elamur.ai/free/en/papers/liang-2024-multi-modal-stance-detection-new/) - [Video-LLaVA: Learning United Visual Representation by Alignment Before Projection](https://md.research.elamur.ai/free/en/papers/lin-2023-video-llava-learning-united-visual/) - [Chameleon: Plug-and-Play Compositional Reasoning with Large Language Models](https://md.research.elamur.ai/free/en/papers/lu-2023-chameleon-plug-play-compositional-reasoning/) - [Text and Code Embeddings by Contrastive Pre-Training](https://md.research.elamur.ai/free/en/papers/neelakantan-2022-text-code-embeddings-contrastive-pre/) - [SimVLM: Simple Visual Language Model Pretraining with Weak Supervision](https://md.research.elamur.ai/free/en/papers/wang-2021-simvlm-simple-visual-language-model/) - [The Dawn of LMMs: Preliminary Explorations with GPT-4V(ision)](https://md.research.elamur.ai/free/en/papers/yang-2023-dawn-lmms-preliminary-explorations-gpt/) - [视觉语言预训练综述](https://md.research.elamur.ai/free/en/papers/yin-2022-vision-language-pre/) - [CoCa: Contrastive Captioners are Image-Text Foundation Models](https://md.research.elamur.ai/free/en/papers/yu-2022-coca-contrastive-captioners-image-text/) - [MM-Vet: Evaluating Large Multimodal Models for Integrated Capabilities](https://md.research.elamur.ai/free/en/papers/yu-2023-mm-vet-evaluating-large-multimodal/) - [Multimodal Chain-of-Thought Reasoning in Language Models](https://md.research.elamur.ai/free/en/papers/zhang-2023-multimodal-chain-thought-reasoning-language/) ## Cite The paper itself, not this page. BibTeX from the library's record of the work. ```bibtex @misc{kim2021vilt, title = {{ViLT: Vision-and-Language Transformer Without Convolution or Region Supervision}}, author = {Kim, Wonjae and Son, Bokyung and Kim, Ildoo}, year = {2021}, doi = {10.48550/arxiv.2102.03334}, eprint = {2102.03334}, archivePrefix = {arXiv}, url = {https://doi.org/10.48550/arxiv.2102.03334} } ``` - [← previous Validation of Universal Dependencies by regeneration](https://md.research.elamur.ai/free/en/papers/ud-2021-lapalme-validation-universal-dependencies-regeneration/) - [next → VoxPopuli: A Large-Scale Multilingual Speech Corpus for Representation Learning, Semi-Supervised Learning and Interpretation](https://md.research.elamur.ai/free/en/papers/wang-2021-voxpopuli-large-scale-multilingual-speech/) - [↑ back to papers of 2021](https://md.research.elamur.ai/free/en/papers/2021/page/5/) ## References 60 references, 21 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet. 1. Anderson, P., He, X., Buehler, C., Teney, D., Johnson, M., Gould, S., and Zhang, L. Bottom-up and top-down attention for image captioning and visual question answering. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 6077–6086, 2018. [find](https://scholar.google.com/scholar?q=Anderson%2C+P%2E%2C+He%2C+X%2E%2C+Buehler%2C+C%2E%2C+Teney%2C+D%2E%2C+Johnson%2C+M%2E%2C+Gould%2C+S%2E%2C+and+Zhang%2C+L%2E+Bottom%2Dup+and+top%2Ddown+attention+for+image+captioning+and+visual+question+answering%2E+In+Proceedings+of+the+IEEE+conf) 2. Asano, Y., Rupprecht, C., and Vedaldi, A. Self-labelling via simultaneous clustering and representation learning. In International Conference on Learning Representations, 2019. [find](https://scholar.google.com/scholar?q=Asano%2C+Y%2E%2C+Rupprecht%2C+C%2E%2C+and+Vedaldi%2C+A%2E+Self%2Dlabelling+via+simultaneous+clustering+and+representation+learning%2E+In+International+Conference+on+Learning+Representations%2C+2019%2E) 3. Berman, M., Jégou, H., Vedaldi, A., Kokkinos, I., and Douze, M. Multigrain: a unified image embedding for classes and instances. arXiv preprint arXiv:1902.05509, 2019. — [arXiv:1902.05509](https://arxiv.org/abs/1902.05509) [find](https://scholar.google.com/scholar?q=Berman%2C+M%2E%2C+J%C3%A9gou%2C+H%2E%2C+Vedaldi%2C+A%2E%2C+Kokkinos%2C+I%2E%2C+and+Douze%2C+M%2E+Multigrain%3A+a+unified+image+embedding+for+classes+and+instances%2E+arXiv+preprint+arXiv%3A1902%2E05509%2C+2019%2E+%E2%80%94+%5BarXiv%3A1902%2E05509%5D%28https%3A%2F%2Farx) 4. Bugliarello, E., Cotterell, R., Okazaki, N., and Elliott, D. Multimodal pretraining unmasked: Unifying the vision and language berts. arXiv preprint arXiv:2011.15124, 2020. — [arXiv:2011.15124](https://arxiv.org/abs/2011.15124) [find](https://scholar.google.com/scholar?q=Bugliarello%2C+E%2E%2C+Cotterell%2C+R%2E%2C+Okazaki%2C+N%2E%2C+and+Elliott%2C+D%2E+Multimodal+pretraining+unmasked%3A+Unifying+the+vision+and+language+berts%2E+arXiv+preprint+arXiv%3A2011%2E15124%2C+2020%2E+%E2%80%94+%5BarXiv%3A2011%2E15124%5D%28https%3A) 5. Caron, M., Bojanowski, P., Joulin, A., and Douze, M. Deep clustering for unsupervised learning of visual features. In Proceedings of the European Conference on Computer Vision (ECCV), pp. 132–149, 2018. [find](https://scholar.google.com/scholar?q=Caron%2C+M%2E%2C+Bojanowski%2C+P%2E%2C+Joulin%2C+A%2E%2C+and+Douze%2C+M%2E+Deep+clustering+for+unsupervised+learning+of+visual+features%2E+In+Proceedings+of+the+European+Conference+on+Computer+Vision+%28ECCV%29%2C+pp%2E+132%E2%80%93149%2C+201) 6. Caron, M., Bojanowski, P., Mairal, J., and Joulin, A. Unsupervised pre-training of image features on non-curated data. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 2959–2968, 2019. [find](https://scholar.google.com/scholar?q=Caron%2C+M%2E%2C+Bojanowski%2C+P%2E%2C+Mairal%2C+J%2E%2C+and+Joulin%2C+A%2E+Unsupervised+pre%2Dtraining+of+image+features+on+non%2Dcurated+data%2E+In+Proceedings+of+the+IEEE%2FCVF+International+Conference+on+Computer+Vision%2C+pp%2E+2) 7. Caron, M., Misra, I., Mairal, J., Goyal, P., Bojanowski, P., and Joulin, A. Unsupervised learning of visual features by contrasting cluster assignments. arXiv preprint arXiv:2006.09882, 2020. — [arXiv:2006.09882](https://arxiv.org/abs/2006.09882) [find](https://scholar.google.com/scholar?q=Caron%2C+M%2E%2C+Misra%2C+I%2E%2C+Mairal%2C+J%2E%2C+Goyal%2C+P%2E%2C+Bojanowski%2C+P%2E%2C+and+Joulin%2C+A%2E+Unsupervised+learning+of+visual+features+by+contrasting+cluster+assignments%2E+arXiv+preprint+arXiv%3A2006%2E09882%2C+2020%2E+%E2%80%94+%5BarXiv) 8. Chen, T., Kornblith, S., Norouzi, M., and Hinton, G. A simple framework for contrastive learning of visual representations. In International conference on machine learning, pp. 1597–1607. PMLR, 2020a. [find](https://scholar.google.com/scholar?q=Chen%2C+T%2E%2C+Kornblith%2C+S%2E%2C+Norouzi%2C+M%2E%2C+and+Hinton%2C+G%2E+A+simple+framework+for+contrastive+learning+of+visual+representations%2E+In+International+conference+on+machine+learning%2C+pp%2E+1597%E2%80%931607%2E+PMLR%2C+2020a%2E) [our card](https://md.research.elamur.ai/free/en/papers/chen-2020-simple-framework-contrastive-learning-visual/) 9. Chen, X., Fan, H., Girshick, R., and He, K. Improved baselines with momentum contrastive learning. arXiv preprint arXiv:2003.04297, 2020b. — [arXiv:2003.04297](https://arxiv.org/abs/2003.04297) [find](https://scholar.google.com/scholar?q=Chen%2C+X%2E%2C+Fan%2C+H%2E%2C+Girshick%2C+R%2E%2C+and+He%2C+K%2E+Improved+baselines+with+momentum+contrastive+learning%2E+arXiv+preprint+arXiv%3A2003%2E04297%2C+2020b%2E+%E2%80%94+%5BarXiv%3A2003%2E04297%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2003%2E04297%29) 10. Chen, Y.-C., Li, L., Yu, L., Kholy, A. E., Ahmed, F., Gan, Z., Cheng, Y., and Liu, J. Uniter: Learning universal image-text representations. arXiv preprint arXiv:1909.11740, 2019. [find](https://scholar.google.com/scholar?q=Chen%2C+Y%2E%2DC%2E%2C+Li%2C+L%2E%2C+Yu%2C+L%2E%2C+Kholy%2C+A%2E+E%2E%2C+Ahmed%2C+F%2E%2C+Gan%2C+Z%2E%2C+Cheng%2C+Y%2E%2C+and+Liu%2C+J%2E+Uniter%3A+Learning+universal+image%2Dtext+representations%2E+arXiv+preprint+arXiv%3A1909%2E11740%2C+2019%2E) [our card](https://md.research.elamur.ai/free/en/papers/chen-2019-uniter-universal-image-text-representation/) 11. Cho, J., Lu, J., Schwenk, D., Hajishirzi, H., and Kembhavi, A. X-lxmert: Paint, caption and answer questions with multi-modal transformers. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 8785–8805, 2020. [find](https://scholar.google.com/scholar?q=Cho%2C+J%2E%2C+Lu%2C+J%2E%2C+Schwenk%2C+D%2E%2C+Hajishirzi%2C+H%2E%2C+and+Kembhavi%2C+A%2E+X%2Dlxmert%3A+Paint%2C+caption+and+answer+questions+with+multi%2Dmodal+transformers%2E+In+Proceedings+of+the+2020+Conference+on+Empirical+Methods+i) 12. Cubuk, E. D., Zoph, B., Shlens, J., and Le, Q. V. Randaugment: Practical automated data augmentation with a reduced search space. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 702–703, 2020. [find](https://scholar.google.com/scholar?q=Cubuk%2C+E%2E+D%2E%2C+Zoph%2C+B%2E%2C+Shlens%2C+J%2E%2C+and+Le%2C+Q%2E+V%2E+Randaugment%3A+Practical+automated+data+augmentation+with+a+reduced+search+space%2E+In+Proceedings+of+the+IEEE%2FCVF+Conference+on+Computer+Vision+and+Patte) 13. Cui, Y., Che, W., Liu, T., Qin, B., Yang, Z., Wang, S., and Hu, G. Pre-training with whole word masking for chinese bert. arXiv preprint arXiv:1906.08101, 2019. [find](https://scholar.google.com/scholar?q=Cui%2C+Y%2E%2C+Che%2C+W%2E%2C+Liu%2C+T%2E%2C+Qin%2C+B%2E%2C+Yang%2C+Z%2E%2C+Wang%2C+S%2E%2C+and+Hu%2C+G%2E+Pre%2Dtraining+with+whole+word+masking+for+chinese+bert%2E+arXiv+preprint+arXiv%3A1906%2E08101%2C+2019%2E) [our card](https://md.research.elamur.ai/free/en/papers/cui-2019-pre-training-whole-word-masking/) 14. Devlin, J., Chang, M.-W., Lee, K., and Toutanova, K. Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and… [find](https://scholar.google.com/scholar?q=Devlin%2C+J%2E%2C+Chang%2C+M%2E%2DW%2E%2C+Lee%2C+K%2E%2C+and+Toutanova%2C+K%2E+Bert%3A+Pre%2Dtraining+of+deep+bidirectional+transformers+for+language+understanding%2E+In+Proceedings+of+the+2019+Conference+of+the+North+American+Chapt) 15. Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., et al. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929, 2020. [find](https://scholar.google.com/scholar?q=Dosovitskiy%2C+A%2E%2C+Beyer%2C+L%2E%2C+Kolesnikov%2C+A%2E%2C+Weissenborn%2C+D%2E%2C+Zhai%2C+X%2E%2C+Unterthiner%2C+T%2E%2C+Dehghani%2C+M%2E%2C+Minderer%2C+M%2E%2C+Heigold%2C+G%2E%2C+Gelly%2C+S%2E%2C+et+al%2E+An+image+is+worth+16x16+words%3A+Transformers+for+image) 16. Faghri, F., Fleet, D. J., Kiros, J. R., and Fidler, S. Vse++: Improving visual-semantic embeddings with hard negatives. arXiv preprint arXiv:1707.05612, 2017. — [arXiv:1707.05612](https://arxiv.org/abs/1707.05612) [find](https://scholar.google.com/scholar?q=Faghri%2C+F%2E%2C+Fleet%2C+D%2E+J%2E%2C+Kiros%2C+J%2E+R%2E%2C+and+Fidler%2C+S%2E+Vse%2B%2B%3A+Improving+visual%2Dsemantic+embeddings+with+hard+negatives%2E+arXiv+preprint+arXiv%3A1707%2E05612%2C+2017%2E+%E2%80%94+%5BarXiv%3A1707%2E05612%5D%28https%3A%2F%2Farxiv%2Eorg%2Fab) 17. Gan, Z., Chen, Y.-C., Li, L., Zhu, C., Cheng, Y., and Liu, J. Large-scale adversarial training for vision-and-language representation learning. arXiv preprint arXiv:2006.06195, 2020. — [arXiv:2006.06195](https://arxiv.org/abs/2006.06195) [find](https://scholar.google.com/scholar?q=Gan%2C+Z%2E%2C+Chen%2C+Y%2E%2DC%2E%2C+Li%2C+L%2E%2C+Zhu%2C+C%2E%2C+Cheng%2C+Y%2E%2C+and+Liu%2C+J%2E+Large%2Dscale+adversarial+training+for+vision%2Dand%2Dlanguage+representation+learning%2E+arXiv+preprint+arXiv%3A2006%2E06195%2C+2020%2E+%E2%80%94+%5BarXiv%3A2006%2E061) 18. Goyal, Y., Khot, T., Summers-Stay, D., Batra, D., and Parikh, D. Making the v in vqa matter: Elevating the role of image understanding in visual question answering. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 6904–6913, 2017. [find](https://scholar.google.com/scholar?q=Goyal%2C+Y%2E%2C+Khot%2C+T%2E%2C+Summers%2DStay%2C+D%2E%2C+Batra%2C+D%2E%2C+and+Parikh%2C+D%2E+Making+the+v+in+vqa+matter%3A+Elevating+the+role+of+image+understanding+in+visual+question+answering%2E+In+Proceedings+of+the+IEEE+Conferen) [our card](https://md.research.elamur.ai/free/en/papers/goyal-2016-making-v-vqa-matter-elevating/) 19. He, K., Zhang, X., Ren, S., and Sun, J. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 770–778, 2016. [find](https://scholar.google.com/scholar?q=He%2C+K%2E%2C+Zhang%2C+X%2E%2C+Ren%2C+S%2E%2C+and+Sun%2C+J%2E+Deep+residual+learning+for+image+recognition%2E+In+Proceedings+of+the+IEEE+conference+on+computer+vision+and+pattern+recognition%2C+pp%2E+770%E2%80%93778%2C+2016%2E) 20. He, K., Gkioxari, G., Dollár, P., and Girshick, R. Mask rcnn. In Proceedings of the IEEE international conference on computer vision, pp. 2961–2969, 2017. [find](https://scholar.google.com/scholar?q=He%2C+K%2E%2C+Gkioxari%2C+G%2E%2C+Doll%C3%A1r%2C+P%2E%2C+and+Girshick%2C+R%2E+Mask+rcnn%2E+In+Proceedings+of+the+IEEE+international+conference+on+computer+vision%2C+pp%2E+2961%E2%80%932969%2C+2017%2E) 21. He, K., Fan, H., Wu, Y., Xie, S., and Girshick, R. Momentum contrast for unsupervised visual representation learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 9729– 9738, 2020. [find](https://scholar.google.com/scholar?q=He%2C+K%2E%2C+Fan%2C+H%2E%2C+Wu%2C+Y%2E%2C+Xie%2C+S%2E%2C+and+Girshick%2C+R%2E+Momentum+contrast+for+unsupervised+visual+representation+learning%2E+In+Proceedings+of+the+IEEE%2FCVF+Conference+on+Computer+Vision+and+Pattern+Recogniti) [our card](https://md.research.elamur.ai/free/en/papers/he-2019-momentum-contrast-unsupervised-visual-representation/) 22. Hoffer, E., Ben-Nun, T., Hubara, I., Giladi, N., Hoefler, T., and Soudry, D. Augment your batch: Improving generalization through instance repetition. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 8129–8138, 2020. [find](https://scholar.google.com/scholar?q=Hoffer%2C+E%2E%2C+Ben%2DNun%2C+T%2E%2C+Hubara%2C+I%2E%2C+Giladi%2C+N%2E%2C+Hoefler%2C+T%2E%2C+and+Soudry%2C+D%2E+Augment+your+batch%3A+Improving+generalization+through+instance+repetition%2E+In+Proceedings+of+the+IEEE%2FCVF+Conference+on+Comp) 23. Huang, Z., Zeng, Z., Liu, B., Fu, D., and Fu, J. Pixel-bert: Aligning image pixels with text by deep multi-modal transformers. arXiv preprint arXiv:2004.00849, 2020. — [arXiv:2004.00849](https://arxiv.org/abs/2004.00849) [find](https://scholar.google.com/scholar?q=Huang%2C+Z%2E%2C+Zeng%2C+Z%2E%2C+Liu%2C+B%2E%2C+Fu%2C+D%2E%2C+and+Fu%2C+J%2E+Pixel%2Dbert%3A+Aligning+image+pixels+with+text+by+deep+multi%2Dmodal+transformers%2E+arXiv+preprint+arXiv%3A2004%2E00849%2C+2020%2E+%E2%80%94+%5BarXiv%3A2004%2E00849%5D%28https%3A%2F%2Farxiv) 24. Jiang, H., Misra, I., Rohrbach, M., Learned-Miller, E., and Chen, X. In defense of grid features for visual question answering. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 10267– 10276, 2020. [find](https://scholar.google.com/scholar?q=Jiang%2C+H%2E%2C+Misra%2C+I%2E%2C+Rohrbach%2C+M%2E%2C+Learned%2DMiller%2C+E%2E%2C+and+Chen%2C+X%2E+In+defense+of+grid+features+for+visual+question+answering%2E+In+Proceedings+of+the+IEEE%2FCVF+Conference+on+Computer+Vision+and+Pattern) 25. Jiang, Y., Natarajan, V., Chen, X., Rohrbach, M., Batra, D., and Parikh, D. Pythia v0. 1: the winning entry to the vqa challenge 2018. arXiv preprint arXiv:1807.09956, 2018. — [arXiv:1807.09956](https://arxiv.org/abs/1807.09956) [find](https://scholar.google.com/scholar?q=Jiang%2C+Y%2E%2C+Natarajan%2C+V%2E%2C+Chen%2C+X%2E%2C+Rohrbach%2C+M%2E%2C+Batra%2C+D%2E%2C+and+Parikh%2C+D%2E+Pythia+v0%2E+1%3A+the+winning+entry+to+the+vqa+challenge+2018%2E+arXiv+preprint+arXiv%3A1807%2E09956%2C+2018%2E+%E2%80%94+%5BarXiv%3A1807%2E09956%5D%28https) 26. Karpathy, A. and Fei-Fei, L. Deep visual-semantic alignments for generating image descriptions. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 3128–3137, 2015. [find](https://scholar.google.com/scholar?q=Karpathy%2C+A%2E+and+Fei%2DFei%2C+L%2E+Deep+visual%2Dsemantic+alignments+for+generating+image+descriptions%2E+In+Proceedings+of+the+IEEE+conference+on+computer+vision+and+pattern+recognition%2C+pp%2E+3128%E2%80%933137%2C+2015%2E) [our card](https://md.research.elamur.ai/free/en/papers/karpathy-2014-deep-visual-semantic-alignments-generating/) 27. Krishna, R., Zhu, Y., Groth, O., Johnson, J., Hata, K., Kravitz, J., Chen, S., Kalantidis, Y., Li, L.-J., Shamma, D. A., et al. Visual genome: Connecting language and vision using crowdsourced dense image annotations. International journal of computer vision, 123(1):32–73, 2017. [find](https://scholar.google.com/scholar?q=Krishna%2C+R%2E%2C+Zhu%2C+Y%2E%2C+Groth%2C+O%2E%2C+Johnson%2C+J%2E%2C+Hata%2C+K%2E%2C+Kravitz%2C+J%2E%2C+Chen%2C+S%2E%2C+Kalantidis%2C+Y%2E%2C+Li%2C+L%2E%2DJ%2E%2C+Shamma%2C+D%2E+A%2E%2C+et+al%2E+Visual+genome%3A+Connecting+language+and+vision+using+crowdsourced+dense+i) [our card](https://md.research.elamur.ai/free/en/papers/krishna-2016-visual-genome-connecting-language-vision/) 28. Krizhevsky, A., Sutskever, I., and Hinton, G. E. Imagenet classification with deep convolutional neural networks. In NIPS, 2012. [find](https://scholar.google.com/scholar?q=Krizhevsky%2C+A%2E%2C+Sutskever%2C+I%2E%2C+and+Hinton%2C+G%2E+E%2E+Imagenet+classification+with+deep+convolutional+neural+networks%2E+In+NIPS%2C+2012%2E) 29. Kuznetsova, A., Rom, H., Alldrin, N., Uijlings, J., Krasin, I., Pont-Tuset, J., Kamali, S., Popov, S., Malloci, M., Kolesnikov, A., et al. The open images dataset v4. International Journal of Computer Vision, pp. 1–26, 2020. [find](https://scholar.google.com/scholar?q=Kuznetsova%2C+A%2E%2C+Rom%2C+H%2E%2C+Alldrin%2C+N%2E%2C+Uijlings%2C+J%2E%2C+Krasin%2C+I%2E%2C+Pont%2DTuset%2C+J%2E%2C+Kamali%2C+S%2E%2C+Popov%2C+S%2E%2C+Malloci%2C+M%2E%2C+Kolesnikov%2C+A%2E%2C+et+al%2E+The+open+images+dataset+v4%2E+International+Journal+of+Computer) 30. Lee, K.-H., Chen, X., Hua, G., Hu, H., and He, X. Stacked cross attention for image-text matching. In Proceedings of the European Conference on Computer Vision (ECCV), pp. 201–216, 2018. [find](https://scholar.google.com/scholar?q=Lee%2C+K%2E%2DH%2E%2C+Chen%2C+X%2E%2C+Hua%2C+G%2E%2C+Hu%2C+H%2E%2C+and+He%2C+X%2E+Stacked+cross+attention+for+image%2Dtext+matching%2E+In+Proceedings+of+the+European+Conference+on+Computer+Vision+%28ECCV%29%2C+pp%2E+201%E2%80%93216%2C+2018%2E) 31. Li, G., Duan, N., Fang, Y., Gong, M., Jiang, D., and Zhou, M. Unicoder-vl: A universal encoder for vision and language by cross-modal pre-training. In AAAI, pp. 11336– 11344, 2020a. [find](https://scholar.google.com/scholar?q=Li%2C+G%2E%2C+Duan%2C+N%2E%2C+Fang%2C+Y%2E%2C+Gong%2C+M%2E%2C+Jiang%2C+D%2E%2C+and+Zhou%2C+M%2E+Unicoder%2Dvl%3A+A+universal+encoder+for+vision+and+language+by+cross%2Dmodal+pre%2Dtraining%2E+In+AAAI%2C+pp%2E+11336%E2%80%93+11344%2C+2020a%2E) 32. Li, L. H., Yatskar, M., Yin, D., Hsieh, C.-J., and Chang, K.-W. Visualbert: A simple and performant baseline for vision and language. arXiv preprint arXiv:1908.03557, 2019. [find](https://scholar.google.com/scholar?q=Li%2C+L%2E+H%2E%2C+Yatskar%2C+M%2E%2C+Yin%2C+D%2E%2C+Hsieh%2C+C%2E%2DJ%2E%2C+and+Chang%2C+K%2E%2DW%2E+Visualbert%3A+A+simple+and+performant+baseline+for+vision+and+language%2E+arXiv+preprint+arXiv%3A1908%2E03557%2C+2019%2E) [our card](https://md.research.elamur.ai/free/en/papers/li-2019-visualbert-simple-performant-baseline-vision/) 33. Li, X., Yin, X., Li, C., Zhang, P., Hu, X., Zhang, L., Wang, L., Hu, H., Dong, L., Wei, F., et al. Oscar: Object-semantics aligned pre-training for vision-language tasks. In European Conference on Computer Vision, pp. 121– 137. Springer, 2020b. [find](https://scholar.google.com/scholar?q=Li%2C+X%2E%2C+Yin%2C+X%2E%2C+Li%2C+C%2E%2C+Zhang%2C+P%2E%2C+Hu%2C+X%2E%2C+Zhang%2C+L%2E%2C+Wang%2C+L%2E%2C+Hu%2C+H%2E%2C+Dong%2C+L%2E%2C+Wei%2C+F%2E%2C+et+al%2E+Oscar%3A+Object%2Dsemantics+aligned+pre%2Dtraining+for+vision%2Dlanguage+tasks%2E+In+European+Conference+on+Com) [our card](https://md.research.elamur.ai/free/en/papers/li-2020-oscar-object-semantics-aligned-pre/) 34. Lin, T.-Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P., and Zitnick, C. L. Microsoft coco: Common objects in context. In European conference on computer vision, pp. 740–755. Springer, 2014. [find](https://scholar.google.com/scholar?q=Lin%2C+T%2E%2DY%2E%2C+Maire%2C+M%2E%2C+Belongie%2C+S%2E%2C+Hays%2C+J%2E%2C+Perona%2C+P%2E%2C+Ramanan%2C+D%2E%2C+Doll%C3%A1r%2C+P%2E%2C+and+Zitnick%2C+C%2E+L%2E+Microsoft+coco%3A+Common+objects+in+context%2E+In+European+conference+on+computer+vision%2C+pp%2E+740%E2%80%93755) [our card](https://md.research.elamur.ai/free/en/papers/lin-2014-microsoft-coco-common-objects-context/) 35. Loshchilov, I. and Hutter, F. Decoupled weight decay regularization. In International Conference on Learning Representations, 2018. [find](https://scholar.google.com/scholar?q=Loshchilov%2C+I%2E+and+Hutter%2C+F%2E+Decoupled+weight+decay+regularization%2E+In+International+Conference+on+Learning+Representations%2C+2018%2E) 36. Lu, J., Batra, D., Parikh, D., and Lee, S. Vilbert: Pretraining task-agnostic visiolinguistic representations for vision-and-language tasks. In Advances in Neural Information Processing Systems, pp. 13–23, 2019. [find](https://scholar.google.com/scholar?q=Lu%2C+J%2E%2C+Batra%2C+D%2E%2C+Parikh%2C+D%2E%2C+and+Lee%2C+S%2E+Vilbert%3A+Pretraining+task%2Dagnostic+visiolinguistic+representations+for+vision%2Dand%2Dlanguage+tasks%2E+In+Advances+in+Neural+Information+Processing+Systems%2C+pp%2E+1) [our card](https://md.research.elamur.ai/free/en/papers/lu-2019-vilbert-pretraining-task-agnostic-visiolinguistic/) 37. Lu, J., Goswami, V., Rohrbach, M., Parikh, D., and Lee, S. 12-in-1: Multi-task vision and language representation learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 10437– 10446, 2020. [find](https://scholar.google.com/scholar?q=Lu%2C+J%2E%2C+Goswami%2C+V%2E%2C+Rohrbach%2C+M%2E%2C+Parikh%2C+D%2E%2C+and+Lee%2C+S%2E+12%2Din%2D1%3A+Multi%2Dtask+vision+and+language+representation+learning%2E+In+Proceedings+of+the+IEEE%2FCVF+Conference+on+Computer+Vision+and+Pattern+Rec) 38. Nguyen, D.-K., Goswami, V., and Chen, X. Revisiting modulated convolutions for visual counting and beyond. arXiv preprint arXiv:2004.11883, 2020. — [arXiv:2004.11883](https://arxiv.org/abs/2004.11883) [find](https://scholar.google.com/scholar?q=Nguyen%2C+D%2E%2DK%2E%2C+Goswami%2C+V%2E%2C+and+Chen%2C+X%2E+Revisiting+modulated+convolutions+for+visual+counting+and+beyond%2E+arXiv+preprint+arXiv%3A2004%2E11883%2C+2020%2E+%E2%80%94+%5BarXiv%3A2004%2E11883%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2004%2E11883%29) 39. Ordonez, V., Kulkarni, G., and Berg, T. Im2text: Describing images using 1 million captioned photographs. Advances in neural information processing systems, 24:1143–1151, 2011. [find](https://scholar.google.com/scholar?q=Ordonez%2C+V%2E%2C+Kulkarni%2C+G%2E%2C+and+Berg%2C+T%2E+Im2text%3A+Describing+images+using+1+million+captioned+photographs%2E+Advances+in+neural+information+processing+systems%2C+24%3A1143%E2%80%931151%2C+2011%2E) 40. Perez, E., Strub, F., De Vries, H., Dumoulin, V., and Courville, A. Film: Visual reasoning with a general conditioning layer. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 32, 2018. [find](https://scholar.google.com/scholar?q=Perez%2C+E%2E%2C+Strub%2C+F%2E%2C+De+Vries%2C+H%2E%2C+Dumoulin%2C+V%2E%2C+and+Courville%2C+A%2E+Film%3A+Visual+reasoning+with+a+general+conditioning+layer%2E+In+Proceedings+of+the+AAAI+Conference+on+Artificial+Intelligence%2C+volume+3) [our card](https://md.research.elamur.ai/free/en/papers/perez-2017-film-visual-reasoning-general-conditioning/) 41. Plummer, B. A., Wang, L., Cervantes, C. M., Caicedo, J. C., Hockenmaier, J., and Lazebnik, S. Flickr30k entities: Collecting region-to-phrase correspondences for richer image-to-sentence models. In Proceedings of the IEEE international conference on computer vision, pp. 2641– 2649, 2015. [find](https://scholar.google.com/scholar?q=Plummer%2C+B%2E+A%2E%2C+Wang%2C+L%2E%2C+Cervantes%2C+C%2E+M%2E%2C+Caicedo%2C+J%2E+C%2E%2C+Hockenmaier%2C+J%2E%2C+and+Lazebnik%2C+S%2E+Flickr30k+entities%3A+Collecting+region%2Dto%2Dphrase+correspondences+for+richer+image%2Dto%2Dsentence+models%2E+In+Pr) 42. Qi, D., Su, L., Song, J., Cui, E., Bharti, T., and Sacheti, A. Imagebert: Cross-modal pre-training with large-scale weak-supervised image-text data. arXiv preprint arXiv:2001.07966, 2020. — [arXiv:2001.07966](https://arxiv.org/abs/2001.07966) [find](https://scholar.google.com/scholar?q=Qi%2C+D%2E%2C+Su%2C+L%2E%2C+Song%2C+J%2E%2C+Cui%2C+E%2E%2C+Bharti%2C+T%2E%2C+and+Sacheti%2C+A%2E+Imagebert%3A+Cross%2Dmodal+pre%2Dtraining+with+large%2Dscale+weak%2Dsupervised+image%2Dtext+data%2E+arXiv+preprint+arXiv%3A2001%2E07966%2C+2020%2E+%E2%80%94+%5BarXiv%3A200) 43. Radford, A., Sutskever, I., Kim, J., Krueger, G., and Agarwal, S. Learning transferable visual models from natural language supervision, 2021. [find](https://scholar.google.com/scholar?q=Radford%2C+A%2E%2C+Sutskever%2C+I%2E%2C+Kim%2C+J%2E%2C+Krueger%2C+G%2E%2C+and+Agarwal%2C+S%2E+Learning+transferable+visual+models+from+natural+language+supervision%2C+2021%2E) [our card](https://md.research.elamur.ai/free/en/papers/radford-2021-learning-transferable-visual-models-natural/) 44. Ren, S., He, K., Girshick, R., and Sun, J. Faster r-cnn: Towards real-time object detection with region proposal networks. IEEE transactions on pattern analysis and machine intelligence, 39(6):1137–1149, 2016. [find](https://scholar.google.com/scholar?q=Ren%2C+S%2E%2C+He%2C+K%2E%2C+Girshick%2C+R%2E%2C+and+Sun%2C+J%2E+Faster+r%2Dcnn%3A+Towards+real%2Dtime+object+detection+with+region+proposal+networks%2E+IEEE+transactions+on+pattern+analysis+and+machine+intelligence%2C+39%286%29%3A1137%E2%80%9311) [our card](https://md.research.elamur.ai/free/en/papers/ren-2015-faster-r-cnn-towards-real/) 45. Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., et al. Imagenet large scale visual recognition challenge. International journal of computer vision, 115(3): 211–252, 2015. [find](https://scholar.google.com/scholar?q=Russakovsky%2C+O%2E%2C+Deng%2C+J%2E%2C+Su%2C+H%2E%2C+Krause%2C+J%2E%2C+Satheesh%2C+S%2E%2C+Ma%2C+S%2E%2C+Huang%2C+Z%2E%2C+Karpathy%2C+A%2E%2C+Khosla%2C+A%2E%2C+Bernstein%2C+M%2E%2C+et+al%2E+Imagenet+large+scale+visual+recognition+challenge%2E+International+journal) [our card](https://md.research.elamur.ai/free/en/papers/russakovsky-2014-imagenet-large-scale-visual-recognition/) 46. Sharma, P., Ding, N., Goodman, S., and Soricut, R. Conceptual captions: A cleaned, hypernymed, image alt-text dataset for automatic image captioning. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 2556– 2565, 2018. [find](https://scholar.google.com/scholar?q=Sharma%2C+P%2E%2C+Ding%2C+N%2E%2C+Goodman%2C+S%2E%2C+and+Soricut%2C+R%2E+Conceptual+captions%3A+A+cleaned%2C+hypernymed%2C+image+alt%2Dtext+dataset+for+automatic+image+captioning%2E+In+Proceedings+of+the+56th+Annual+Meeting+of+the+A) [our card](https://md.research.elamur.ai/free/en/papers/sharma-2018-conceptual-captions-cleaned-hypernymed-image/) 47. Shorten, C. and Khoshgoftaar, T. M. A survey on image data augmentation for deep learning. Journal of Big Data, 6(1):1–48, 2019. [find](https://scholar.google.com/scholar?q=Shorten%2C+C%2E+and+Khoshgoftaar%2C+T%2E+M%2E+A+survey+on+image+data+augmentation+for+deep+learning%2E+Journal+of+Big+Data%2C+6%281%29%3A1%E2%80%9348%2C+2019%2E) 48. Su, W., Zhu, X., Cao, Y., Li, B., Lu, L., Wei, F., and Dai, J. Vl-bert: Pre-training of generic visual-linguistic representations. arXiv preprint arXiv:1908.08530, 2019. [find](https://scholar.google.com/scholar?q=Su%2C+W%2E%2C+Zhu%2C+X%2E%2C+Cao%2C+Y%2E%2C+Li%2C+B%2E%2C+Lu%2C+L%2E%2C+Wei%2C+F%2E%2C+and+Dai%2C+J%2E+Vl%2Dbert%3A+Pre%2Dtraining+of+generic+visual%2Dlinguistic+representations%2E+arXiv+preprint+arXiv%3A1908%2E08530%2C+2019%2E) [our card](https://md.research.elamur.ai/free/en/papers/su-2019-vl-bert-pre-training-generic/) 49. Suhr, A., Zhou, S., Zhang, A., Zhang, I., Bai, H., and Artzi, Y. A corpus for reasoning about natural language grounded in photographs. arXiv preprint arXiv:1811.00491, 2018. [find](https://scholar.google.com/scholar?q=Suhr%2C+A%2E%2C+Zhou%2C+S%2E%2C+Zhang%2C+A%2E%2C+Zhang%2C+I%2E%2C+Bai%2C+H%2E%2C+and+Artzi%2C+Y%2E+A+corpus+for+reasoning+about+natural+language+grounded+in+photographs%2E+arXiv+preprint+arXiv%3A1811%2E00491%2C+2018%2E) [our card](https://md.research.elamur.ai/free/en/papers/suhr-2019-corpus-reasoning-natural-language-grounded/) 50. Tan, H. and Bansal, M. Lxmert: Learning cross-modality encoder representations from transformers. arXiv preprint arXiv:1908.07490, 2019. [find](https://scholar.google.com/scholar?q=Tan%2C+H%2E+and+Bansal%2C+M%2E+Lxmert%3A+Learning+cross%2Dmodality+encoder+representations+from+transformers%2E+arXiv+preprint+arXiv%3A1908%2E07490%2C+2019%2E) [our card](https://md.research.elamur.ai/free/en/papers/tan-2019-lxmert-learning-cross-modality-encoder/) 51. Touvron, H., Cord, M., Douze, M., Massa, F., Sablayrolles, A., and Jégou, H. Training data-efficient image transformers & distillation through attention. arXiv preprint arXiv:2012.12877, 2020. [find](https://scholar.google.com/scholar?q=Touvron%2C+H%2E%2C+Cord%2C+M%2E%2C+Douze%2C+M%2E%2C+Massa%2C+F%2E%2C+Sablayrolles%2C+A%2E%2C+and+J%C3%A9gou%2C+H%2E+Training+data%2Defficient+image+transformers+%26+distillation+through+attention%2E+arXiv+preprint+arXiv%3A2012%2E12877%2C+2020%2E) [our card](https://md.research.elamur.ai/free/en/papers/touvron-2020-training-data-efficient-image-transformers/) 52. Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, Ł., and Polosukhin, I. Attention is all you need. Advances in neural information processing systems, 30:5998–6008, 2017. [find](https://scholar.google.com/scholar?q=Vaswani%2C+A%2E%2C+Shazeer%2C+N%2E%2C+Parmar%2C+N%2E%2C+Uszkoreit%2C+J%2E%2C+Jones%2C+L%2E%2C+Gomez%2C+A%2E+N%2E%2C+Kaiser%2C+%C5%81%2E%2C+and+Polosukhin%2C+I%2E+Attention+is+all+you+need%2E+Advances+in+neural+information+processing+systems%2C+30%3A5998%E2%80%936008%2C) 53. Xie, S., Girshick, R., Dollár, P., Tu, Z., and He, K. Aggregated residual transformations for deep neural networks. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 1492–1500, 2017. [find](https://scholar.google.com/scholar?q=Xie%2C+S%2E%2C+Girshick%2C+R%2E%2C+Doll%C3%A1r%2C+P%2E%2C+Tu%2C+Z%2E%2C+and+He%2C+K%2E+Aggregated+residual+transformations+for+deep+neural+networks%2E+In+Proceedings+of+the+IEEE+conference+on+computer+vision+and+pattern+recognition%2C+pp) 54. Xie, Y., Wang, X., Wang, R., and Zha, H. A fast proximal point method for computing exact wasserstein distance. In Uncertainty in Artificial Intelligence, pp. 433–453. PMLR, 2020. [find](https://scholar.google.com/scholar?q=Xie%2C+Y%2E%2C+Wang%2C+X%2E%2C+Wang%2C+R%2E%2C+and+Zha%2C+H%2E+A+fast+proximal+point+method+for+computing+exact+wasserstein+distance%2E+In+Uncertainty+in+Artificial+Intelligence%2C+pp%2E+433%E2%80%93453%2E+PMLR%2C+2020%2E) 55. Yu, F., Tang, J., Yin, W., Sun, Y., Tian, H., Wu, H., and Wang, H. Ernie-vil: Knowledge enhanced vision-language representations through scene graph. arXiv preprint arXiv:2006.16934, 2020. [find](https://scholar.google.com/scholar?q=Yu%2C+F%2E%2C+Tang%2C+J%2E%2C+Yin%2C+W%2E%2C+Sun%2C+Y%2E%2C+Tian%2C+H%2E%2C+Wu%2C+H%2E%2C+and+Wang%2C+H%2E+Ernie%2Dvil%3A+Knowledge+enhanced+vision%2Dlanguage+representations+through+scene+graph%2E+arXiv+preprint+arXiv%3A2006%2E16934%2C+2020%2E) 56. Yu, Z., Yu, J., Cui, Y., Tao, D., and Tian, Q. Deep modular co-attention networks for visual question answering. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 6281–6290, 2019. [find](https://scholar.google.com/scholar?q=Yu%2C+Z%2E%2C+Yu%2C+J%2E%2C+Cui%2C+Y%2E%2C+Tao%2C+D%2E%2C+and+Tian%2C+Q%2E+Deep+modular+co%2Dattention+networks+for+visual+question+answering%2E+In+Proceedings+of+the+IEEE%2FCVF+Conference+on+Computer+Vision+and+Pattern+Recognition%2C+p) 57. Yun, S., Han, D., Oh, S. J., Chun, S., Choe, J., and Yoo, Y. Cutmix: Regularization strategy to train strong classifiers with localizable features. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 6023–6032, 2019. [find](https://scholar.google.com/scholar?q=Yun%2C+S%2E%2C+Han%2C+D%2E%2C+Oh%2C+S%2E+J%2E%2C+Chun%2C+S%2E%2C+Choe%2C+J%2E%2C+and+Yoo%2C+Y%2E+Cutmix%3A+Regularization+strategy+to+train+strong+classifiers+with+localizable+features%2E+In+Proceedings+of+the+IEEE%2FCVF+International+Confere) 58. Zhang, H., Cisse, M., Dauphin, Y. N., and Lopez-Paz, D. mixup: Beyond empirical risk minimization. arXiv preprint arXiv:1710.09412, 2017. [find](https://scholar.google.com/scholar?q=Zhang%2C+H%2E%2C+Cisse%2C+M%2E%2C+Dauphin%2C+Y%2E+N%2E%2C+and+Lopez%2DPaz%2C+D%2E+mixup%3A+Beyond+empirical+risk+minimization%2E+arXiv+preprint+arXiv%3A1710%2E09412%2C+2017%2E) [our card](https://md.research.elamur.ai/free/en/papers/zhang-2017-mixup-empirical-risk-minimization/) 59. Zhang, P., Li, X., Hu, X., Yang, J., Zhang, L., Wang, L., Choi, Y., and Gao, J. Vinvl: Making visual representations matter in vision-language models. arXiv preprint arXiv:2101.00529, 2021. — [arXiv:2101.00529](https://arxiv.org/abs/2101.00529) [find](https://scholar.google.com/scholar?q=Zhang%2C+P%2E%2C+Li%2C+X%2E%2C+Hu%2C+X%2E%2C+Yang%2C+J%2E%2C+Zhang%2C+L%2E%2C+Wang%2C+L%2E%2C+Choi%2C+Y%2E%2C+and+Gao%2C+J%2E+Vinvl%3A+Making+visual+representations+matter+in+vision%2Dlanguage+models%2E+arXiv+preprint+arXiv%3A2101%2E00529%2C+2021%2E+%E2%80%94+%5BarXiv%3A2) 60. Zhou, L., Palangi, H., Zhang, L., Hu, H., Corso, J. J., and Gao, J. Unified vision-language pre-training for image captioning and vqa. In AAAI, pp. 13041–13049, 2020. [find](https://scholar.google.com/scholar?q=Zhou%2C+L%2E%2C+Palangi%2C+H%2E%2C+Zhang%2C+L%2E%2C+Hu%2C+H%2E%2C+Corso%2C+J%2E+J%2E%2C+and+Gao%2C+J%2E+Unified+vision%2Dlanguage+pre%2Dtraining+for+image+captioning+and+vqa%2E+In+AAAI%2C+pp%2E+13041%E2%80%9313049%2C+2020%2E)