---
title: "Vision-Language Models for Vision Tasks: A Survey (2023)"
description: "This survey, by researchers at Nanyang Technological University, addresses two drawbacks of… 205 references, cited by 92 papers here."
authors: ["Jingyi Zhang", "Jiaxing Huang", "Sheng Jin", "Shijian Lu"]
year: 2023
venue: "arXiv:2304.00685"
source: "https://arxiv.org/abs/2304.00685"
doi: "10.48550/arxiv.2304.00685"
arxiv: "2304.00685"
licence: "arXiv — non-exclusive license to distribute"
licence_url: "http://arxiv.org/licenses/nonexclusive-distrib/1.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/zhang-2023-vision-language-models-vision-tasks/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/zhang-2023-vision-language-models-vision-tasks/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2023](https://md.research.elamur.ai/free/en/papers/2023/) / Vision-Language Models for Vision Tasks: A Survey

# Vision-Language Models for Vision Tasks: A Survey

**Authors:** [Jingyi Zhang](https://md.research.elamur.ai/free/en/authors/jingyi-zhang/), [Jiaxing Huang](https://md.research.elamur.ai/free/en/authors/jiaxing-huang/), Sheng Jin and [Shijian Lu](https://md.research.elamur.ai/free/en/authors/shijian-lu/) · **Year:** 2023 · **Venue:** [arXiv:2304.00685](https://md.research.elamur.ai/free/en/venues/ieee-transactions-on-pattern-analysis-and-machine-intelligence/) ([IEEE Transactions on Pattern Analysis and Machine Intelligence 2023](https://md.research.elamur.ai/free/en/venues/ieee-transactions-on-pattern-analysis-and-machine-intelligence/2023/)) · **Publisher:** [IEEE Computer Society](https://md.research.elamur.ai/free/en/publishers/ieee-computer-society/) · **Licence:** [arXiv — non-exclusive license to distribute](http://arxiv.org/licenses/nonexclusive-distrib/1.0/)

**Links:** [Publisher's page](https://arxiv.org/abs/2304.00685) · [PDF](https://arxiv.org/pdf/2304.00685) · [DOI 10.48550/arxiv.2304.00685](https://doi.org/10.48550/arxiv.2304.00685) · [arXiv 2304.00685](https://arxiv.org/abs/2304.00685) · [OpenAlex](https://openalex.org/W4392172801)

**Citations:** 955 citations (OpenAlex, 2026-10-14) · [92](https://md.research.elamur.ai/free/en/papers/zhang-2023-vision-language-models-vision-tasks/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
This survey, by researchers at Nanyang Technological University, addresses two drawbacks of mainstream visual recognition: dependence on costly crowd-labelled data and the habit of training a separate network for every task. It follows the shift toward vision-language models (VLMs), which learn links between images and text from huge web-crawled image-text pairs. A single such model can then handle many recognition tasks zero-shot, without any fine-tuning. The authors trace five stages of training paradigms, ending with VLM pre-training plus zero-shot prediction, pioneered by CLIP in 2021. They then cover common architectures, pre-training objectives and frameworks, and the datasets used for pre-training and evaluation. Existing work is sorted into three families: pre-training methods, transfer learning, and knowledge distillation into detection and segmentation models. The survey benchmarks the reviewed methods on public datasets and closes with open problems. It is a structured entry point to a field whose publication count has grown exponentially since CLIP.

## Key points
- CLIP uses an image-text contrastive objective, which brings matching image and caption embeddings together and separates mismatched ones. It achieved strong zero-shot results across 36 visual recognition tasks.
- The paper describes transfer approaches, including prompt tuning and visual adaptation, and distillation approaches for object detection and semantic segmentation.
- The evaluation datasets span several task types:
  - **Classification:** ImageNet-1k, SVHN, IIIT5k, and Rendered SST2 (built from Stanford Sentiment Treebank sentences).
  - **Action recognition:** UCF101, Kinetics700, RareAct.
  - **Segmentation:** ADE20k (150 classes), PASCAL VOC 2012, Cityscapes.
  - **Detection:** MS COCO, LVIS (1203 categories), ODinW.
  - **Retrieval:** Flickr30k, COCO Caption.
- ODinW combines 35 public detection datasets covering 314 concepts, with about 132k training and 20k test images.
- A companion repository is maintained at github.com/jingyi0000/VLM survey.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Nanyang Technological University](https://md.research.elamur.ai/free/en/universities/nanyang-technological-university/) — University, Singapore, Singapore · 184 works · 27,484 citations

📚 Shelves (3): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 18,614 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 17,965 works · [👁️ Computer Vision and Pattern Recognition](https://md.research.elamur.ai/free/en/shelves/computer-science/computer-vision-and-pattern-recognition/) · 3,486 works

🧭 Topics (2): [🧠 Language models & pretraining](https://md.research.elamur.ai/free/en/topics/language-models/) · 3,894 works · [🖼️ Vision & language](https://md.research.elamur.ai/free/en/topics/multimodal/) · 811 works

## Cite

Cite as: Jingyi Zhang, Jiaxing Huang, Sheng Jin and Shijian Lu. 2023. Vision-Language Models for Vision Tasks: A Survey. arXiv:2304.00685.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@misc{zhang2023vision,
  title = {{Vision-Language Models for Vision Tasks: A Survey}},
  author = {Jingyi Zhang, Jiaxing Huang, Sheng Jin and Shijian Lu},
  year = {2023},
  doi = {10.48550/arxiv.2304.00685},
  eprint = {2304.00685},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2304.00685}
}
```

- [← previous A Survey on Multimodal Large Language Models](https://md.research.elamur.ai/free/en/papers/yin-2023-survey-multimodal-large-language-models/)
- [next → GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpoints](https://md.research.elamur.ai/free/en/papers/ainslie-2023-gqa-training-generalized-multi-query/)
- [↑ back to papers of 2023](https://md.research.elamur.ai/free/en/papers/2023/)

## References

205 references, 34 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. A. Geiger et al. Are we ready for autonomous driving? the kitti vision benchmark suite. In CVPR, pp. 3354–3361. IEEE, 2012. [find](https://scholar.google.com/scholar?q=A%2E+Geiger+et+al%2E+Are+we+ready+for+autonomous+driving%3F+the+kitti+vision+benchmark+suite%2E+In+CVPR%2C+pp%2E+3354%E2%80%933361%2E+IEEE%2C+2012%2E)
2. G. Cheng et al. Remote sensing image scene classification meets deep learning: Challenges, methods, benchmarks, and opportunities. JSTARS, 13:3735–3756, 2020. [find](https://scholar.google.com/scholar?q=G%2E+Cheng+et+al%2E+Remote+sensing+image+scene+classification+meets+deep+learning%3A+Challenges%2C+methods%2C+benchmarks%2C+and+opportunities%2E+JSTARS%2C+13%3A3735%E2%80%933756%2C+2020%2E)
3. H. A. Pierson and M. S. Gashler. Deep learning in robotics: a review of recent research. Advanced Robotics, 31(16):821–835, 2017. [find](https://scholar.google.com/scholar?q=H%2E+A%2E+Pierson+and+M%2E+S%2E+Gashler%2E+Deep+learning+in+robotics%3A+a+review+of+recent+research%2E+Advanced+Robotics%2C+31%2816%29%3A821%E2%80%93835%2C+2017%2E)
4. A. Krizhevsky et al. Imagenet classification with deep convolutional neural networks. Communications of the ACM, 60(6):84–90, 2017. [find](https://scholar.google.com/scholar?q=A%2E+Krizhevsky+et+al%2E+Imagenet+classification+with+deep+convolutional+neural+networks%2E+Communications+of+the+ACM%2C+60%286%29%3A84%E2%80%9390%2C+2017%2E)
5. K. Simonyan and A. Zisserman. [Very deep convolutional networks for large-scale image recognition](https://md.research.elamur.ai/free/en/papers/simonyan-2014-very-deep-convolutional-networks-large/). arXiv preprint arXiv:1409.1556, 2014.
6. K. He et al. [Deep residual learning for image recognition](https://md.research.elamur.ai/free/en/papers/he-2015-deep-residual-learning-image-recognition/). In CVPR, pp. 770–778, 2016.
7. L. E. Peterson. K-nearest neighbor. Scholarpedia, 4(2):1883, 2009. [find](https://scholar.google.com/scholar?q=L%2E+E%2E+Peterson%2E+K%2Dnearest+neighbor%2E+Scholarpedia%2C+4%282%29%3A1883%2C+2009%2E)
8. C. Cortes and V. Vapnik. [Support-vector networks](https://md.research.elamur.ai/free/en/papers/cortes-1995-support-vector-networks/). Machine learning, 20:273–297, 1995.
9. A. Mathur and G. M. Foody. Multiclass and binary svm classification: Implications for training and classification users. IEEE Geoscience and remote sensing letters, 5(2):241–245, 2008. [find](https://scholar.google.com/scholar?q=A%2E+Mathur+and+G%2E+M%2E+Foody%2E+Multiclass+and+binary+svm+classification%3A+Implications+for+training+and+classification+users%2E+IEEE+Geoscience+and+remote+sensing+letters%2C+5%282%29%3A241%E2%80%93245%2C+2008%2E)
10. A. Radford et al. [Learning transferable visual models from natural language supervision](https://md.research.elamur.ai/free/en/papers/radford-2021-learning-transferable-visual-models-natural/). In ICML, pp. 8748–8763. PMLR, 2021.
11. R. Girshick. Fast r-cnn. In ICCV, pp. 1440–1448, 2015. [find](https://scholar.google.com/scholar?q=R%2E+Girshick%2E+Fast+r%2Dcnn%2E+In+ICCV%2C+pp%2E+1440%E2%80%931448%2C+2015%2E)
12. K. He et al. [Momentum contrast for unsupervised visual representation learning](https://md.research.elamur.ai/free/en/papers/he-2019-momentum-contrast-unsupervised-visual-representation/). In CVPR, pp. 9729–9738, 2020.
13. T. Chen et al. [A simple framework for contrastive learning of visual representations](https://md.research.elamur.ai/free/en/papers/chen-2020-simple-framework-contrastive-learning-visual/). In ICML, pp. 1597–1607. PMLR, 2020.
14. J. Devlin et al. [Bert: Pre-training of deep bidirectional transformers for language understanding](https://md.research.elamur.ai/free/en/papers/devlin-2019-bert-pre-training-deep-bidirectional/). arXiv preprint arXiv:1810.04805, 2018.
15. A. Radford et al. Improving language understanding by generative pre-training. 2018. [find](https://scholar.google.com/scholar?q=A%2E+Radford+et+al%2E+Improving+language+understanding+by+generative+pre%2Dtraining%2E+2018%2E)
16. A. Radford et al. Language models are unsupervised multitask learners. OpenAI blog, 1(8):9, 2019. [find](https://scholar.google.com/scholar?q=A%2E+Radford+et+al%2E+Language+models+are+unsupervised+multitask+learners%2E+OpenAI+blog%2C+1%288%29%3A9%2C+2019%2E)
17. C. Jia et al. [Scaling up visual and vision-language representation learning with noisy text supervision](https://md.research.elamur.ai/free/en/papers/jia-2021-scaling-up-visual-vision-language/). In ICML, pp. 4904–4916. PMLR, 2021.
18. L. Yao et al. Filip: Fine-grained interactive language-image pre-training. In ICLR, 2021. [find](https://scholar.google.com/scholar?q=L%2E+Yao+et+al%2E+Filip%3A+Fine%2Dgrained+interactive+language%2Dimage+pre%2Dtraining%2E+In+ICLR%2C+2021%2E)
19. J. Yu et al. [Coca: Contrastive captioners are image-text foundation models](https://md.research.elamur.ai/free/en/papers/yu-2022-coca-contrastive-captioners-image-text/). arXiv preprint arXiv:2205.01917, 2022.
20. C. Schuhmann et al. [Laion-5b: An open large-scale dataset for training next generation image-text models](https://md.research.elamur.ai/free/en/papers/schuhmann-2022-laion-5b-open-large-scale/). arXiv preprint arXiv:2210.08402, 2022.
21. C. Schuhmann et al. [Laion-400m: Open dataset of clip-filtered 400 million image-text pairs](https://md.research.elamur.ai/free/en/papers/schuhmann-2021-laion-400m-open-dataset-clip/). arXiv preprint arXiv:2111.02114, 2021.
22. L. Bossard et al. Food-101–mining discriminative components with random forests. In ECCV, pp. 446–461. Springer, 2014. [find](https://scholar.google.com/scholar?q=L%2E+Bossard+et+al%2E+Food%2D101%E2%80%93mining+discriminative+components+with+random+forests%2E+In+ECCV%2C+pp%2E+446%E2%80%93461%2E+Springer%2C+2014%2E)
23. A. Krizhevsky et al. Learning multiple layers of features from tiny images. 2009. [find](https://scholar.google.com/scholar?q=A%2E+Krizhevsky+et+al%2E+Learning+multiple+layers+of+features+from+tiny+images%2E+2009%2E)
24. J. Xiao et al. Sun database: Large-scale scene recognition from abbey to zoo. In 2010 IEEE computer society conference on computer vision and pattern recognition, pp. 3485–3492. IEEE, 2010. [find](https://scholar.google.com/scholar?q=J%2E+Xiao+et+al%2E+Sun+database%3A+Large%2Dscale+scene+recognition+from+abbey+to+zoo%2E+In+2010+IEEE+computer+society+conference+on+computer+vision+and+pattern+recognition%2C+pp%2E+3485%E2%80%933492%2E+IEEE%2C+2010%2E)
25. J. Krause et al. Collecting a large-scale dataset of fine-grained cars. 2013. [find](https://scholar.google.com/scholar?q=J%2E+Krause+et+al%2E+Collecting+a+large%2Dscale+dataset+of+fine%2Dgrained+cars%2E+2013%2E)
26. O. M. Parkhi et al. Cats and dogs. In CVPR, pp. 3498–3505. IEEE, 2012. [find](https://scholar.google.com/scholar?q=O%2E+M%2E+Parkhi+et+al%2E+Cats+and+dogs%2E+In+CVPR%2C+pp%2E+3498%E2%80%933505%2E+IEEE%2C+2012%2E)
27. D. Kiela et al. [The hateful memes challenge: Detecting hate speech in multimodal memes](https://md.research.elamur.ai/free/en/papers/kiela-2020-hateful-memes-challenge-detecting-hate/). NeurIPS, 33:2611–2624, 2020.
28. A. Miech et al. Rareact: A video dataset of unusual interactions. arXiv preprint arXiv:2008.01018, 2020. [find](https://scholar.google.com/scholar?q=A%2E+Miech+et+al%2E+Rareact%3A+A+video+dataset+of+unusual+interactions%2E+arXiv+preprint+arXiv%3A2008%2E01018%2C+2020%2E)
29. K. Soomro et al. Ucf101: A dataset of 101 human actions classes from videos in the wild. arXiv preprint arXiv:1212.0402, 2012. — [arXiv:1212.0402](https://arxiv.org/abs/1212.0402) [find](https://scholar.google.com/scholar?q=K%2E+Soomro+et+al%2E+Ucf101%3A+A+dataset+of+101+human+actions+classes+from+videos+in+the+wild%2E+arXiv+preprint+arXiv%3A1212%2E0402%2C+2012%2E+%E2%80%94+%5BarXiv%3A1212%2E0402%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F1212%2E0402%29)
30. J. Carreira et al. A short note on the kinetics-700 human action dataset. arXiv preprint arXiv:1907.06987, 2019. [find](https://scholar.google.com/scholar?q=J%2E+Carreira+et+al%2E+A+short+note+on+the+kinetics%2D700+human+action+dataset%2E+arXiv+preprint+arXiv%3A1907%2E06987%2C+2019%2E)
31. K. Zhou et al. Learning to prompt for vision-language models. IJCV, 130(9):2337–2348, 2022. [find](https://scholar.google.com/scholar?q=K%2E+Zhou+et+al%2E+Learning+to+prompt+for+vision%2Dlanguage+models%2E+IJCV%2C+130%289%29%3A2337%E2%80%932348%2C+2022%2E)
32. K. Zhou et al. Conditional prompt learning for vision-language models. In CVPR, pp. 16816–16825, 2022. [find](https://scholar.google.com/scholar?q=K%2E+Zhou+et+al%2E+Conditional+prompt+learning+for+vision%2Dlanguage+models%2E+In+CVPR%2C+pp%2E+16816%E2%80%9316825%2C+2022%2E)
33. P. Gao et al. Clip-adapter: Better vision-language models with feature adapters. arXiv preprint arXiv:2110.04544, 2021. [find](https://scholar.google.com/scholar?q=P%2E+Gao+et+al%2E+Clip%2Dadapter%3A+Better+vision%2Dlanguage+models+with+feature+adapters%2E+arXiv+preprint+arXiv%3A2110%2E04544%2C+2021%2E)
34. R. Zhang et al. Tip-adapter: Training-free clip-adapter for better vision-language modeling. arXiv preprint arXiv:2111.03930, 2021. [find](https://scholar.google.com/scholar?q=R%2E+Zhang+et+al%2E+Tip%2Dadapter%3A+Training%2Dfree+clip%2Dadapter+for+better+vision%2Dlanguage+modeling%2E+arXiv+preprint+arXiv%3A2111%2E03930%2C+2021%2E)
35. J. Ding et al. Decoupling zero-shot semantic segmentation. In CVPR, pp. 11583–11592, 2022. [find](https://scholar.google.com/scholar?q=J%2E+Ding+et+al%2E+Decoupling+zero%2Dshot+semantic+segmentation%2E+In+CVPR%2C+pp%2E+11583%E2%80%9311592%2C+2022%2E)
36. X. Gu et al. Open-vocabulary object detection via vision and language knowledge distillation. In ICLR, 2021. [find](https://scholar.google.com/scholar?q=X%2E+Gu+et+al%2E+Open%2Dvocabulary+object+detection+via+vision+and+language+knowledge+distillation%2E+In+ICLR%2C+2021%2E)
37. Y. Du et al. Learning to prompt for open-vocabulary object detection with vision-language model. In CVPR, pp. 14084– 14093, 2022. [find](https://scholar.google.com/scholar?q=Y%2E+Du+et+al%2E+Learning+to+prompt+for+open%2Dvocabulary+object+detection+with+vision%2Dlanguage+model%2E+In+CVPR%2C+pp%2E+14084%E2%80%93+14093%2C+2022%2E)
38. D. G. Lowe. Distinctive image features from scale-invariant keypoints. IJCV, 60:91–110, 2004. [find](https://scholar.google.com/scholar?q=D%2E+G%2E+Lowe%2E+Distinctive+image+features+from+scale%2Dinvariant+keypoints%2E+IJCV%2C+60%3A91%E2%80%93110%2C+2004%2E)
39. L. Breiman. Random forests. Machine learning, 45:5–32, 2001. [find](https://scholar.google.com/scholar?q=L%2E+Breiman%2E+Random+forests%2E+Machine+learning%2C+45%3A5%E2%80%9332%2C+2001%2E)
40. J. Deng et al. Imagenet: A large-scale hierarchical image database. In CVPR, pp. 248–255. Ieee, 2009. [find](https://scholar.google.com/scholar?q=J%2E+Deng+et+al%2E+Imagenet%3A+A+large%2Dscale+hierarchical+image+database%2E+In+CVPR%2C+pp%2E+248%E2%80%93255%2E+Ieee%2C+2009%2E)
41. K. He et al. [Masked autoencoders are scalable vision learners](https://md.research.elamur.ai/free/en/papers/he-2021-masked-autoencoders-scalable-vision-learners/). In CVPR, pp. 16000–16009, 2022.
42. A. Singh et al. Flava: A foundational language and vision alignment model. In CVPR, pp. 15638–15650, 2022. [find](https://scholar.google.com/scholar?q=A%2E+Singh+et+al%2E+Flava%3A+A+foundational+language+and+vision+alignment+model%2E+In+CVPR%2C+pp%2E+15638%E2%80%9315650%2C+2022%2E)
43. M. Tschannen et al. Image-and-language understanding from pixels only. arXiv preprint arXiv:2212.08045, 2022. — [arXiv:2212.08045](https://arxiv.org/abs/2212.08045) [find](https://scholar.google.com/scholar?q=M%2E+Tschannen+et+al%2E+Image%2Dand%2Dlanguage+understanding+from+pixels+only%2E+arXiv+preprint+arXiv%3A2212%2E08045%2C+2022%2E+%E2%80%94+%5BarXiv%3A2212%2E08045%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2212%2E08045%29)
44. J. Jang et al. Unifying vision-language representation space with single-tower transformer. In AAAI, volume 37, pp. 980–988, 2023. [find](https://scholar.google.com/scholar?q=J%2E+Jang+et+al%2E+Unifying+vision%2Dlanguage+representation+space+with+single%2Dtower+transformer%2E+In+AAAI%2C+volume+37%2C+pp%2E+980%E2%80%93988%2C+2023%2E)
45. L. Yao et al. Detclip: Dictionary-enriched visual-concept paralleled pre-training for open-world detection. In NeurIPS. [find](https://scholar.google.com/scholar?q=L%2E+Yao+et+al%2E+Detclip%3A+Dictionary%2Denriched+visual%2Dconcept+paralleled+pre%2Dtraining+for+open%2Dworld+detection%2E+In+NeurIPS%2E)
46. H. Luo et al. Segclip: Patch aggregation with learnable centers for open-vocabulary semantic segmentation. arXiv preprint arXiv:2211.14813, 2022. — [arXiv:2211.14813](https://arxiv.org/abs/2211.14813) [find](https://scholar.google.com/scholar?q=H%2E+Luo+et+al%2E+Segclip%3A+Patch+aggregation+with+learnable+centers+for+open%2Dvocabulary+semantic+segmentation%2E+arXiv+preprint+arXiv%3A2211%2E14813%2C+2022%2E+%E2%80%94+%5BarXiv%3A2211%2E14813%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2211%2E14813%29)
47. S. Antol et al. Vqa: Visual question answering. In ICCV, pp. 2425–2433, 2015. [find](https://scholar.google.com/scholar?q=S%2E+Antol+et+al%2E+Vqa%3A+Visual+question+answering%2E+In+ICCV%2C+pp%2E+2425%E2%80%932433%2C+2015%2E)
48. A. Suhr et al. [A corpus for reasoning about natural language grounded in photographs](https://md.research.elamur.ai/free/en/papers/suhr-2019-corpus-reasoning-natural-language-grounded/). arXiv preprint arXiv:1811.00491, 2018.
49. A. Karpathy et al. Deep fragment embeddings for bidirectional image sentence mapping. NeurIPS, 27, 2014. [find](https://scholar.google.com/scholar?q=A%2E+Karpathy+et+al%2E+Deep+fragment+embeddings+for+bidirectional+image+sentence+mapping%2E+NeurIPS%2C+27%2C+2014%2E)
50. F. Li et al. Vision-language intelligence: Tasks, representation learning, and large models. arXiv preprint arXiv:2203.01922, 2022. — [arXiv:2203.01922](https://arxiv.org/abs/2203.01922) [find](https://scholar.google.com/scholar?q=F%2E+Li+et+al%2E+Vision%2Dlanguage+intelligence%3A+Tasks%2C+representation+learning%2C+and+large+models%2E+arXiv+preprint+arXiv%3A2203%2E01922%2C+2022%2E+%E2%80%94+%5BarXiv%3A2203%2E01922%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2203%2E01922%29)
51. Y. Du et al. A survey of vision-language pre-trained models. arXiv preprint arXiv:2202.10936, 2022. — [arXiv:2202.10936](https://arxiv.org/abs/2202.10936) [find](https://scholar.google.com/scholar?q=Y%2E+Du+et+al%2E+A+survey+of+vision%2Dlanguage+pre%2Dtrained+models%2E+arXiv+preprint+arXiv%3A2202%2E10936%2C+2022%2E+%E2%80%94+%5BarXiv%3A2202%2E10936%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2202%2E10936%29)
52. F.-L. Chen et al. Vlp: A survey on vision-language pre-training. Machine Intelligence Research, 20(1):38–56, 2023. [find](https://scholar.google.com/scholar?q=F%2E%2DL%2E+Chen+et+al%2E+Vlp%3A+A+survey+on+vision%2Dlanguage+pre%2Dtraining%2E+Machine+Intelligence+Research%2C+20%281%29%3A38%E2%80%9356%2C+2023%2E)
53. P. Xu et al. Multimodal learning with transformers: A survey. arXiv preprint arXiv:2206.06488, 2022. — [arXiv:2206.06488](https://arxiv.org/abs/2206.06488) [find](https://scholar.google.com/scholar?q=P%2E+Xu+et+al%2E+Multimodal+learning+with+transformers%3A+A+survey%2E+arXiv+preprint+arXiv%3A2206%2E06488%2C+2022%2E+%E2%80%94+%5BarXiv%3A2206%2E06488%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2206%2E06488%29)
54. X. Wang et al. Large-scale multi-modal pre-trained models: A comprehensive survey. arXiv preprint arXiv:2302.10035, 2023. — [arXiv:2302.10035](https://arxiv.org/abs/2302.10035) [find](https://scholar.google.com/scholar?q=X%2E+Wang+et+al%2E+Large%2Dscale+multi%2Dmodal+pre%2Dtrained+models%3A+A+comprehensive+survey%2E+arXiv+preprint+arXiv%3A2302%2E10035%2C+2023%2E+%E2%80%94+%5BarXiv%3A2302%2E10035%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2302%2E10035%29)
55. S. Ren et al. [Faster r-cnn: Towards real-time object detection with region proposal networks](https://md.research.elamur.ai/free/en/papers/ren-2015-faster-r-cnn-towards-real/). NeurIPS, 28, 2015.
56. L.-C. Chen et al. Deeplab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs. TPAMI, 40(4):834–848, 2017. [find](https://scholar.google.com/scholar?q=L%2E%2DC%2E+Chen+et+al%2E+Deeplab%3A+Semantic+image+segmentation+with+deep+convolutional+nets%2C+atrous+convolution%2C+and+fully+connected+crfs%2E+TPAMI%2C+40%284%29%3A834%E2%80%93848%2C+2017%2E)
57. A. Dosovitskiy et al. [An image is worth 16x16 words: Transformers for image recognition at scale](https://md.research.elamur.ai/free/en/papers/dosovitskiy-2020-image-worth-16x16-words-transformers/). arXiv preprint arXiv:2010.11929, 2020.
58. A. Vaswani et al. [Attention is all you need](https://md.research.elamur.ai/free/en/papers/vaswani-2017-attention-all-need/). NeurIPS, 30, 2017.
59. M. Tan and Q. Le. [Efficientnet: Rethinking model scaling for convolutional neural networks](https://md.research.elamur.ai/free/en/papers/tan-2019-efficientnet-rethinking-model-scaling-convolutional/). In ICML, pp. 6105–6114. PMLR, 2019.
60. T. He et al. Bag of tricks for image classification with convolutional neural networks. In CVPR, pp. 558–567, 2019. [find](https://scholar.google.com/scholar?q=T%2E+He+et+al%2E+Bag+of+tricks+for+image+classification+with+convolutional+neural+networks%2E+In+CVPR%2C+pp%2E+558%E2%80%93567%2C+2019%2E)
61. R. Zhang. Making convolutional networks shift-invariant again. In ICML, pp. 7324–7334. PMLR, 2019. [find](https://scholar.google.com/scholar?q=R%2E+Zhang%2E+Making+convolutional+networks+shift%2Dinvariant+again%2E+In+ICML%2C+pp%2E+7324%E2%80%937334%2E+PMLR%2C+2019%2E)
62. N. Carion et al. [End-to-end object detection with transformers](https://md.research.elamur.ai/free/en/papers/carion-2020-end-end-object-detection-transformers/). In ECCV, pp. 213–229. Springer, 2020.
63. E. Xie et al. Segformer: Simple and efficient design for semantic segmentation with transformers. NeurIPS, 34:12077–12090, 2021. [find](https://scholar.google.com/scholar?q=E%2E+Xie+et+al%2E+Segformer%3A+Simple+and+efficient+design+for+semantic+segmentation+with+transformers%2E+NeurIPS%2C+34%3A12077%E2%80%9312090%2C+2021%2E)
64. N. Mu et al. Slip: Self-supervision meets language-image pre-training. In ECCV, pp. 529–544. Springer, 2022. [find](https://scholar.google.com/scholar?q=N%2E+Mu+et+al%2E+Slip%3A+Self%2Dsupervision+meets+language%2Dimage+pre%2Dtraining%2E+In+ECCV%2C+pp%2E+529%E2%80%93544%2E+Springer%2C+2022%2E)
65. J. Yang et al. Unified contrastive learning in image-text-label space. In CVPR, pp. 19163–19173, 2022. [find](https://scholar.google.com/scholar?q=J%2E+Yang+et+al%2E+Unified+contrastive+learning+in+image%2Dtext%2Dlabel+space%2E+In+CVPR%2C+pp%2E+19163%E2%80%9319173%2C+2022%2E)
66. K. He et al. [Masked autoencoders are scalable vision learners](https://md.research.elamur.ai/free/en/papers/he-2021-masked-autoencoders-scalable-vision-learners/). arXiv preprint arXiv:2111.06377, 2021.
67. L. H. Li et al. Grounded language-image pre-training. In CVPR, pp. 10965–10975, 2022. [find](https://scholar.google.com/scholar?q=L%2E+H%2E+Li+et+al%2E+Grounded+language%2Dimage+pre%2Dtraining%2E+In+CVPR%2C+pp%2E+10965%E2%80%9310975%2C+2022%2E)
68. A. v. d. Oord et al. [Representation learning with contrastive predictive coding](https://md.research.elamur.ai/free/en/papers/oord-2018-representation-learning-contrastive-predictive-coding/). arXiv preprint arXiv:1807.03748, 2018.
69. P. Khosla et al. Supervised contrastive learning. NeurIPS, 33:18661–18673, 2020. [find](https://scholar.google.com/scholar?q=P%2E+Khosla+et+al%2E+Supervised+contrastive+learning%2E+NeurIPS%2C+33%3A18661%E2%80%9318673%2C+2020%2E)
70. H. Bao et al. Beit: Bert pre-training of image transformers. arXiv preprint arXiv:2106.08254, 2021. — [arXiv:2106.08254](https://arxiv.org/abs/2106.08254) [find](https://scholar.google.com/scholar?q=H%2E+Bao+et+al%2E+Beit%3A+Bert+pre%2Dtraining+of+image+transformers%2E+arXiv+preprint+arXiv%3A2106%2E08254%2C+2021%2E+%E2%80%94+%5BarXiv%3A2106%2E08254%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2106%2E08254%29)
71. Z.-Y. Dou et al. Coarse-to-fine vision-language pre-training with fusion in the backbone. In NeurIPS. [find](https://scholar.google.com/scholar?q=Z%2E%2DY%2E+Dou+et+al%2E+Coarse%2Dto%2Dfine+vision%2Dlanguage+pre%2Dtraining+with+fusion+in+the+backbone%2E+In+NeurIPS%2E)
72. H. Bao et al. Vlmo: Unified vision-language pre-training with mixture-of-modality-experts. arXiv preprint arXiv:2111.02358, 2021. [find](https://scholar.google.com/scholar?q=H%2E+Bao+et+al%2E+Vlmo%3A+Unified+vision%2Dlanguage+pre%2Dtraining+with+mixture%2Dof%2Dmodality%2Dexperts%2E+arXiv+preprint+arXiv%3A2111%2E02358%2C+2021%2E)
73. V. Ordonez et al. Im2text: Describing images using 1 million captioned photographs. NeurIPS, 24, 2011. [find](https://scholar.google.com/scholar?q=V%2E+Ordonez+et+al%2E+Im2text%3A+Describing+images+using+1+million+captioned+photographs%2E+NeurIPS%2C+24%2C+2011%2E)
74. X. Chen et al. [Microsoft coco captions: Data collection and evaluation server](https://md.research.elamur.ai/free/en/papers/chen-2015-microsoft-coco-captions-data-collection/). arXiv preprint arXiv:1504.00325, 2015.
75. B. Thomee et al. [Yfcc100m: The new data in multimedia research](https://md.research.elamur.ai/free/en/papers/thomee-2015-yfcc100m-new-data-multimedia-research/). Communications of the ACM, 59(2):64–73, 2016.
76. R. Krishna et al. [Visual genome: Connecting language and vision using crowdsourced dense image annotations](https://md.research.elamur.ai/free/en/papers/krishna-2016-visual-genome-connecting-language-vision/). IJCV, 123(1):32– 73, 2017.
77. P. Sharma et al. [Conceptual captions: A cleaned, hypernymed, image alt-text dataset for automatic image captioning](https://md.research.elamur.ai/free/en/papers/sharma-2018-conceptual-captions-cleaned-hypernymed-image/). In ACL, pp. 2556–2565, 2018.
78. J. Pont-Tuset et al. Connecting vision and language with localized narratives. In ECCV, pp. 647–664. Springer, 2020. [find](https://scholar.google.com/scholar?q=J%2E+Pont%2DTuset+et+al%2E+Connecting+vision+and+language+with+localized+narratives%2E+In+ECCV%2C+pp%2E+647%E2%80%93664%2E+Springer%2C+2020%2E)
79. S. Changpinyo et al. Conceptual 12m: Pushing web-scale image-text pre-training to recognize long-tail visual concepts. In CVPR, pp. 3558–3568, 2021. [find](https://scholar.google.com/scholar?q=S%2E+Changpinyo+et+al%2E+Conceptual+12m%3A+Pushing+web%2Dscale+image%2Dtext+pre%2Dtraining+to+recognize+long%2Dtail+visual+concepts%2E+In+CVPR%2C+pp%2E+3558%E2%80%933568%2C+2021%2E)
80. K. Srinivasan et al. [Wit: Wikipedia-based image text dataset for multimodal multilingual machine learning](https://md.research.elamur.ai/free/en/papers/srinivasan-2021-wit-wikipedia-based-image-text/). In ACM SIGIR, pp. 2443–2449, 2021.
81. K. Desai et al. Redcaps: Web-curated image-text data created by the people, for the people. arXiv preprint arXiv:2111.11431, 2021. — [arXiv:2111.11431](https://arxiv.org/abs/2111.11431) [find](https://scholar.google.com/scholar?q=K%2E+Desai+et+al%2E+Redcaps%3A+Web%2Dcurated+image%2Dtext+data+created+by+the+people%2C+for+the+people%2E+arXiv+preprint+arXiv%3A2111%2E11431%2C+2021%2E+%E2%80%94+%5BarXiv%3A2111%2E11431%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2111%2E11431%29)
82. J. Gu et al. Wukong: 100 million large-scale chinese cross-modal pre-training dataset and a foundation framework. arXiv preprint arXiv:2202.06767, 2022. — [arXiv:2202.06767](https://arxiv.org/abs/2202.06767) [find](https://scholar.google.com/scholar?q=J%2E+Gu+et+al%2E+Wukong%3A+100+million+large%2Dscale+chinese+cross%2Dmodal+pre%2Dtraining+dataset+and+a+foundation+framework%2E+arXiv+preprint+arXiv%3A2202%2E06767%2C+2022%2E+%E2%80%94+%5BarXiv%3A2202%2E06767%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2202)
83. X. Chen et al. [Pali: A jointly-scaled multilingual language-image model](https://md.research.elamur.ai/free/en/papers/chen-2022-pali-jointly-scaled-multilingual-language/). arXiv preprint arXiv:2209.06794, 2022.
84. J. Lee et al. Uniclip: Unified framework for contrastive language-image pre-training. In NeurIPS. [find](https://scholar.google.com/scholar?q=J%2E+Lee+et+al%2E+Uniclip%3A+Unified+framework+for+contrastive+language%2Dimage+pre%2Dtraining%2E+In+NeurIPS%2E)
85. S. Shao et al. Objects365: A large-scale, high-quality dataset for object detection. In ICCV, pp. 8430–8439, 2019. [find](https://scholar.google.com/scholar?q=S%2E+Shao+et+al%2E+Objects365%3A+A+large%2Dscale%2C+high%2Dquality+dataset+for+object+detection%2E+In+ICCV%2C+pp%2E+8430%E2%80%938439%2C+2019%2E)
86. A. Kamath et al. Mdetr-modulated detection for end-to-end multi-modal understanding. In ICCV, pp. 1780–1790, 2021. [find](https://scholar.google.com/scholar?q=A%2E+Kamath+et+al%2E+Mdetr%2Dmodulated+detection+for+end%2Dto%2Dend+multi%2Dmodal+understanding%2E+In+ICCV%2C+pp%2E+1780%E2%80%931790%2C+2021%2E)
87. M. Cao et al. Image-text retrieval: A survey on recent research and development. arXiv preprint arXiv:2203.14713, 2022. — [arXiv:2203.14713](https://arxiv.org/abs/2203.14713) [find](https://scholar.google.com/scholar?q=M%2E+Cao+et+al%2E+Image%2Dtext+retrieval%3A+A+survey+on+recent+research+and+development%2E+arXiv+preprint+arXiv%3A2203%2E14713%2C+2022%2E+%E2%80%94+%5BarXiv%3A2203%2E14713%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2203%2E14713%29)
88. Y. LeCun et al. Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(11):2278–2324, 1998. [find](https://scholar.google.com/scholar?q=Y%2E+LeCun+et+al%2E+Gradient%2Dbased+learning+applied+to+document+recognition%2E+Proceedings+of+the+IEEE%2C+86%2811%29%3A2278%E2%80%932324%2C+1998%2E)
89. L. Fei-Fei et al. Learning generative visual models from few training examples: An incremental bayesian approach tested on 101 object categories. In CVPR workshop, pp. 178–178. IEEE, 2004. [find](https://scholar.google.com/scholar?q=L%2E+Fei%2DFei+et+al%2E+Learning+generative+visual+models+from+few+training+examples%3A+An+incremental+bayesian+approach+tested+on+101+object+categories%2E+In+CVPR+workshop%2C+pp%2E+178%E2%80%93178%2E+IEEE%2C+2004%2E)
90. M. Everingham et al. The pascal visual object classes (voc) challenge. IJCV, 88(2):303–338, 2010. [find](https://scholar.google.com/scholar?q=M%2E+Everingham+et+al%2E+The+pascal+visual+object+classes+%28voc%29+challenge%2E+IJCV%2C+88%282%29%3A303%E2%80%93338%2C+2010%2E)
91. M.-E. Nilsback and A. Zisserman. Automated flower classification over a large number of classes. In ICVGIP, pp. 722–729. IEEE, 2008. [find](https://scholar.google.com/scholar?q=M%2E%2DE%2E+Nilsback+and+A%2E+Zisserman%2E+Automated+flower+classification+over+a+large+number+of+classes%2E+In+ICVGIP%2C+pp%2E+722%E2%80%93729%2E+IEEE%2C+2008%2E)
92. Y. Netzer et al. Reading digits in natural images with unsupervised feature learning. 2011. [find](https://scholar.google.com/scholar?q=Y%2E+Netzer+et+al%2E+Reading+digits+in+natural+images+with+unsupervised+feature+learning%2E+2011%2E)
93. A. Coates et al. An analysis of single-layer networks in unsupervised feature learning. In AISTATS, pp. 215–223. JMLR Workshop and Conference Proceedings, 2011. [find](https://scholar.google.com/scholar?q=A%2E+Coates+et+al%2E+An+analysis+of+single%2Dlayer+networks+in+unsupervised+feature+learning%2E+In+AISTATS%2C+pp%2E+215%E2%80%93223%2E+JMLR+Workshop+and+Conference+Proceedings%2C+2011%2E)
94. J. Stallkamp et al. The german traffic sign recognition benchmark: a multi-class classification competition. In IJCNN, pp. 1453–1460. IEEE, 2011. [find](https://scholar.google.com/scholar?q=J%2E+Stallkamp+et+al%2E+The+german+traffic+sign+recognition+benchmark%3A+a+multi%2Dclass+classification+competition%2E+In+IJCNN%2C+pp%2E+1453%E2%80%931460%2E+IEEE%2C+2011%2E)
95. A. Mishra et al. Scene text recognition using higher order language priors. In BMVC-British machine vision conference. BMVA, 2012. [find](https://scholar.google.com/scholar?q=A%2E+Mishra+et+al%2E+Scene+text+recognition+using+higher+order+language+priors%2E+In+BMVC%2DBritish+machine+vision+conference%2E+BMVA%2C+2012%2E)
96. S. Maji et al. Fine-grained visual classification of aircraft. arXiv preprint arXiv:1306.5151, 2013. — [arXiv:1306.5151](https://arxiv.org/abs/1306.5151) [find](https://scholar.google.com/scholar?q=S%2E+Maji+et+al%2E+Fine%2Dgrained+visual+classification+of+aircraft%2E+arXiv+preprint+arXiv%3A1306%2E5151%2C+2013%2E+%E2%80%94+%5BarXiv%3A1306%2E5151%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F1306%2E5151%29)
97. I. J. Goodfellow et al. Challenges in representation learning: A report on three machine learning contests. In ICONIP, pp. 117– 124. Springer, 2013. [find](https://scholar.google.com/scholar?q=I%2E+J%2E+Goodfellow+et+al%2E+Challenges+in+representation+learning%3A+A+report+on+three+machine+learning+contests%2E+In+ICONIP%2C+pp%2E+117%E2%80%93+124%2E+Springer%2C+2013%2E)
98. R. Socher et al. [Recursive deep models for semantic compositionality over a sentiment treebank](https://md.research.elamur.ai/free/en/papers/socher-2013-recursive-deep-models-semantic-compositionality/). In EMNLP, pp. 1631–1642, 2013.
99. M. Cimpoi et al. Describing textures in the wild. In CVPR, pp. 3606–3613, 2014. [find](https://scholar.google.com/scholar?q=M%2E+Cimpoi+et+al%2E+Describing+textures+in+the+wild%2E+In+CVPR%2C+pp%2E+3606%E2%80%933613%2C+2014%2E)
100. T. Berg et al. Birdsnap: Large-scale fine-grained visual categorization of birds. In CVPR, pp. 2011–2018, 2014. [find](https://scholar.google.com/scholar?q=T%2E+Berg+et+al%2E+Birdsnap%3A+Large%2Dscale+fine%2Dgrained+visual+categorization+of+birds%2E+In+CVPR%2C+pp%2E+2011%E2%80%932018%2C+2014%2E)
101. G. Cheng et al. Remote sensing image scene classification: Benchmark and state of the art. Proceedings of the IEEE, 105(10):1865– 1883, 2017. [find](https://scholar.google.com/scholar?q=G%2E+Cheng+et+al%2E+Remote+sensing+image+scene+classification%3A+Benchmark+and+state+of+the+art%2E+Proceedings+of+the+IEEE%2C+105%2810%29%3A1865%E2%80%93+1883%2C+2017%2E)
102. J. Johnson et al. [Clevr: A diagnostic dataset for compositional language and elementary visual reasoning](https://md.research.elamur.ai/free/en/papers/johnson-2016-clevr-diagnostic-dataset-compositional-language/). In CVPR, pp. 2901– 2910, 2017.
103. B. S. Veeling et al. Rotation equivariant cnns for digital pathology. In International Conference on Medical image computing and computer-assisted intervention, pp. 210–218. Springer, 2018. [find](https://scholar.google.com/scholar?q=B%2E+S%2E+Veeling+et+al%2E+Rotation+equivariant+cnns+for+digital+pathology%2E+In+International+Conference+on+Medical+image+computing+and+computer%2Dassisted+intervention%2C+pp%2E+210%E2%80%93218%2E+Springer%2C+2018%2E)
104. P. Helber et al. Eurosat: A novel dataset and deep learning benchmark for land use and land cover classification. JSTARS, 12(7):2217–2226, 2019. [find](https://scholar.google.com/scholar?q=P%2E+Helber+et+al%2E+Eurosat%3A+A+novel+dataset+and+deep+learning+benchmark+for+land+use+and+land+cover+classification%2E+JSTARS%2C+12%287%29%3A2217%E2%80%932226%2C+2019%2E)
105. P. Young et al. From image descriptions to visual denotations: New similarity metrics for semantic inference over event descriptions. TACL, 2:67–78, 2014. [find](https://scholar.google.com/scholar?q=P%2E+Young+et+al%2E+From+image+descriptions+to+visual+denotations%3A+New+similarity+metrics+for+semantic+inference+over+event+descriptions%2E+TACL%2C+2%3A67%E2%80%9378%2C+2014%2E)
106. T.-Y. Lin et al. [Microsoft coco: Common objects in context](https://md.research.elamur.ai/free/en/papers/lin-2014-microsoft-coco-common-objects-context/). In ECCV, pp. 740–755. Springer, 2014.
107. A. Gupta et al. Lvis: A dataset for large vocabulary instance segmentation. In CVPR, pp. 5356–5364, 2019. [find](https://scholar.google.com/scholar?q=A%2E+Gupta+et+al%2E+Lvis%3A+A+dataset+for+large+vocabulary+instance+segmentation%2E+In+CVPR%2C+pp%2E+5356%E2%80%935364%2C+2019%2E)
108. C. Li et al. Elevater: A benchmark and toolkit for evaluating language-augmented visual models. arXiv preprint arXiv:2204.08790, 2022. — [arXiv:2204.08790](https://arxiv.org/abs/2204.08790) [find](https://scholar.google.com/scholar?q=C%2E+Li+et+al%2E+Elevater%3A+A+benchmark+and+toolkit+for+evaluating+language%2Daugmented+visual+models%2E+arXiv+preprint+arXiv%3A2204%2E08790%2C+2022%2E+%E2%80%94+%5BarXiv%3A2204%2E08790%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2204%2E08790%29)
109. R. Mottaghi et al. The role of context for object detection and semantic segmentation in the wild. In CVPR, pp. 891–898, 2014. [find](https://scholar.google.com/scholar?q=R%2E+Mottaghi+et+al%2E+The+role+of+context+for+object+detection+and+semantic+segmentation+in+the+wild%2E+In+CVPR%2C+pp%2E+891%E2%80%93898%2C+2014%2E)
110. M. Cordts et al. The cityscapes dataset for semantic urban scene understanding. In CVPR, pp. 3213–3223, 2016. [find](https://scholar.google.com/scholar?q=M%2E+Cordts+et+al%2E+The+cityscapes+dataset+for+semantic+urban+scene+understanding%2E+In+CVPR%2C+pp%2E+3213%E2%80%933223%2C+2016%2E)
111. B. Zhou et al. Scene parsing through ade20k dataset. In CVPR, pp. 633–641, 2017. [find](https://scholar.google.com/scholar?q=B%2E+Zhou+et+al%2E+Scene+parsing+through+ade20k+dataset%2E+In+CVPR%2C+pp%2E+633%E2%80%93641%2C+2017%2E)
112. B. Wu et al. Data efficient language-supervised zero-shot recognition with optimal transport distillation. In ICLR, 2021. [find](https://scholar.google.com/scholar?q=B%2E+Wu+et+al%2E+Data+efficient+language%2Dsupervised+zero%2Dshot+recognition+with+optimal+transport+distillation%2E+In+ICLR%2C+2021%2E)
113. Y. Li et al. Supervision exists everywhere: A data efficient contrastive language-image pre-training paradigm. In ICLR, 2021. [find](https://scholar.google.com/scholar?q=Y%2E+Li+et+al%2E+Supervision+exists+everywhere%3A+A+data+efficient+contrastive+language%2Dimage+pre%2Dtraining+paradigm%2E+In+ICLR%2C+2021%2E)
114. Q. Cui et al. Contrastive vision-language pre-training with limited resources. In ECCV, pp. 236–253. Springer, 2022. [find](https://scholar.google.com/scholar?q=Q%2E+Cui+et+al%2E+Contrastive+vision%2Dlanguage+pre%2Dtraining+with+limited+resources%2E+In+ECCV%2C+pp%2E+236%E2%80%93253%2E+Springer%2C+2022%2E)
115. L. Yuan et al. Florence: A new foundation model for computer vision. arXiv preprint arXiv:2111.11432, 2021. — [arXiv:2111.11432](https://arxiv.org/abs/2111.11432) [find](https://scholar.google.com/scholar?q=L%2E+Yuan+et+al%2E+Florence%3A+A+new+foundation+model+for+computer+vision%2E+arXiv+preprint+arXiv%3A2111%2E11432%2C+2021%2E+%E2%80%94+%5BarXiv%3A2111%2E11432%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2111%2E11432%29)
116. Y. Gao et al. Pyramidclip: Hierarchical feature alignment for vision-language model pretraining. arXiv preprint arXiv:2204.14095, 2022. — [arXiv:2204.14095](https://arxiv.org/abs/2204.14095) [find](https://scholar.google.com/scholar?q=Y%2E+Gao+et+al%2E+Pyramidclip%3A+Hierarchical+feature+alignment+for+vision%2Dlanguage+model+pretraining%2E+arXiv+preprint+arXiv%3A2204%2E14095%2C+2022%2E+%E2%80%94+%5BarXiv%3A2204%2E14095%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2204%2E14095%29)
117. A. Yang et al. Chinese clip: Contrastive vision-language pretraining in chinese. arXiv preprint arXiv:2211.01335, 2022. — [arXiv:2211.01335](https://arxiv.org/abs/2211.01335) [find](https://scholar.google.com/scholar?q=A%2E+Yang+et+al%2E+Chinese+clip%3A+Contrastive+vision%2Dlanguage+pretraining+in+chinese%2E+arXiv+preprint+arXiv%3A2211%2E01335%2C+2022%2E+%E2%80%94+%5BarXiv%3A2211%2E01335%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2211%2E01335%29)
118. X. Zhai et al. Lit: Zero-shot transfer with locked-image text tuning. In CVPR, pp. 18123–18133, 2022. [find](https://scholar.google.com/scholar?q=X%2E+Zhai+et+al%2E+Lit%3A+Zero%2Dshot+transfer+with+locked%2Dimage+text+tuning%2E+In+CVPR%2C+pp%2E+18123%E2%80%9318133%2C+2022%2E)
119. Z. Chen et al. Altclip: Altering the language encoder in clip for extended language capabilities. arXiv preprint arXiv:2211.06679, 2022. [find](https://scholar.google.com/scholar?q=Z%2E+Chen+et+al%2E+Altclip%3A+Altering+the+language+encoder+in+clip+for+extended+language+capabilities%2E+arXiv+preprint+arXiv%3A2211%2E06679%2C+2022%2E)
120. B. Ko and G. Gu. Large-scale bilingual language-image contrastive learning. arXiv preprint arXiv:2203.14463, 2022. — [arXiv:2203.14463](https://arxiv.org/abs/2203.14463) [find](https://scholar.google.com/scholar?q=B%2E+Ko+and+G%2E+Gu%2E+Large%2Dscale+bilingual+language%2Dimage+contrastive+learning%2E+arXiv+preprint+arXiv%3A2203%2E14463%2C+2022%2E+%E2%80%94+%5BarXiv%3A2203%2E14463%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2203%2E14463%29)
121. J. Zhou et al. Non-contrastive learning meets language-image pre-training. arXiv preprint arXiv:2210.09304, 2022. — [arXiv:2210.09304](https://arxiv.org/abs/2210.09304) [find](https://scholar.google.com/scholar?q=J%2E+Zhou+et+al%2E+Non%2Dcontrastive+learning+meets+language%2Dimage+pre%2Dtraining%2E+arXiv+preprint+arXiv%3A2210%2E09304%2C+2022%2E+%E2%80%94+%5BarXiv%3A2210%2E09304%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2210%2E09304%29)
122. S. Shen et al. K-lite: Learning transferable visual models with external knowledge. arXiv preprint arXiv:2204.09222, 2022. [find](https://scholar.google.com/scholar?q=S%2E+Shen+et+al%2E+K%2Dlite%3A+Learning+transferable+visual+models+with+external+knowledge%2E+arXiv+preprint+arXiv%3A2204%2E09222%2C+2022%2E)
123. R. Huang et al. Nlip: Noise-robust language-image pre-training. arXiv preprint arXiv:2212.07086, 2022. — [arXiv:2212.07086](https://arxiv.org/abs/2212.07086) [find](https://scholar.google.com/scholar?q=R%2E+Huang+et+al%2E+Nlip%3A+Noise%2Drobust+language%2Dimage+pre%2Dtraining%2E+arXiv+preprint+arXiv%3A2212%2E07086%2C+2022%2E+%E2%80%94+%5BarXiv%3A2212%2E07086%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2212%2E07086%29)
124. S. Geng et al. Hiclip: Contrastive language-image pretraining with hierarchy-aware attention. arXiv preprint arXiv:2303.02995, 2023. — [arXiv:2303.02995](https://arxiv.org/abs/2303.02995) [find](https://scholar.google.com/scholar?q=S%2E+Geng+et+al%2E+Hiclip%3A+Contrastive+language%2Dimage+pretraining+with+hierarchy%2Daware+attention%2E+arXiv+preprint+arXiv%3A2303%2E02995%2C+2023%2E+%E2%80%94+%5BarXiv%3A2303%2E02995%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2303%2E02995%29)
125. C.-W. Xie et al. Ra-clip: Retrieval augmented contrastive language-image pre-training. In CVPR, pp. 19265–19274, 2023. [find](https://scholar.google.com/scholar?q=C%2E%2DW%2E+Xie+et+al%2E+Ra%2Dclip%3A+Retrieval+augmented+contrastive+language%2Dimage+pre%2Dtraining%2E+In+CVPR%2C+pp%2E+19265%E2%80%9319274%2C+2023%2E)
126. L. Fan et al. Improving clip training with language rewrites. arXiv preprint arXiv:2305.20088, 2023. — [arXiv:2305.20088](https://arxiv.org/abs/2305.20088) [find](https://scholar.google.com/scholar?q=L%2E+Fan+et+al%2E+Improving+clip+training+with+language+rewrites%2E+arXiv+preprint+arXiv%3A2305%2E20088%2C+2023%2E+%E2%80%94+%5BarXiv%3A2305%2E20088%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2305%2E20088%29)
127. K. Yang et al. Alip: Adaptive language-image pre-training with synthetic caption. In ICCV, pp. 2922–2931, 2023. [find](https://scholar.google.com/scholar?q=K%2E+Yang+et+al%2E+Alip%3A+Adaptive+language%2Dimage+pre%2Dtraining+with+synthetic+caption%2E+In+ICCV%2C+pp%2E+2922%E2%80%932931%2C+2023%2E)
128. X. Deng et al. Growclip: Data-aware automatic model growing for large-scale contrastive language-image pre-training. In ICCV, pp. 22178–22189, 2023. [find](https://scholar.google.com/scholar?q=X%2E+Deng+et+al%2E+Growclip%3A+Data%2Daware+automatic+model+growing+for+large%2Dscale+contrastive+language%2Dimage+pre%2Dtraining%2E+In+ICCV%2C+pp%2E+22178%E2%80%9322189%2C+2023%2E)
129. J. Xu et al. Groupvit: Semantic segmentation emerges from text supervision. In CVPR, pp. 18134–18144, 2022. [find](https://scholar.google.com/scholar?q=J%2E+Xu+et+al%2E+Groupvit%3A+Semantic+segmentation+emerges+from+text+supervision%2E+In+CVPR%2C+pp%2E+18134%E2%80%9318144%2C+2022%2E)
130. K. Ranasinghe et al. Perceptual grouping in vision-language models. arXiv preprint arXiv:2210.09996, 2022. — [arXiv:2210.09996](https://arxiv.org/abs/2210.09996) [find](https://scholar.google.com/scholar?q=K%2E+Ranasinghe+et+al%2E+Perceptual+grouping+in+vision%2Dlanguage+models%2E+arXiv+preprint+arXiv%3A2210%2E09996%2C+2022%2E+%E2%80%94+%5BarXiv%3A2210%2E09996%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2210%2E09996%29)
131. Y. Zhong et al. Regionclip: Region-based language-image pre-training. In CVPR, pp. 16793–16803, 2022. [find](https://scholar.google.com/scholar?q=Y%2E+Zhong+et+al%2E+Regionclip%3A+Region%2Dbased+language%2Dimage+pre%2Dtraining%2E+In+CVPR%2C+pp%2E+16793%E2%80%9316803%2C+2022%2E)
132. C. Ma et al. Understanding and mitigating overfitting in prompt tuning for vision-language models. arXiv preprint arXiv:2211.02219, 2022. — [arXiv:2211.02219](https://arxiv.org/abs/2211.02219) [find](https://scholar.google.com/scholar?q=C%2E+Ma+et+al%2E+Understanding+and+mitigating+overfitting+in+prompt+tuning+for+vision%2Dlanguage+models%2E+arXiv+preprint+arXiv%3A2211%2E02219%2C+2022%2E+%E2%80%94+%5BarXiv%3A2211%2E02219%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2211%2E02219%29)
133. A. Bulat and G. Tzimiropoulos. Language-aware soft prompting for vision & language foundation models. arXiv preprint arXiv:2210.01115, 2022. — [arXiv:2210.01115](https://arxiv.org/abs/2210.01115) [find](https://scholar.google.com/scholar?q=A%2E+Bulat+and+G%2E+Tzimiropoulos%2E+Language%2Daware+soft+prompting+for+vision+%26+language+foundation+models%2E+arXiv+preprint+arXiv%3A2210%2E01115%2C+2022%2E+%E2%80%94+%5BarXiv%3A2210%2E01115%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2210%2E01115%29)
134. Y. Lu et al. Prompt distribution learning. In CVPR, pp. 5206–5215, 2022. [find](https://scholar.google.com/scholar?q=Y%2E+Lu+et+al%2E+Prompt+distribution+learning%2E+In+CVPR%2C+pp%2E+5206%E2%80%935215%2C+2022%2E)
135. M. M. Derakhshani et al. Variational prompt tuning improves generalization of vision-language models. arXiv preprint arXiv:2210.02390, 2022. — [arXiv:2210.02390](https://arxiv.org/abs/2210.02390) [find](https://scholar.google.com/scholar?q=M%2E+M%2E+Derakhshani+et+al%2E+Variational+prompt+tuning+improves+generalization+of+vision%2Dlanguage+models%2E+arXiv+preprint+arXiv%3A2210%2E02390%2C+2022%2E+%E2%80%94+%5BarXiv%3A2210%2E02390%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2210%2E02390%29)
136. B. Zhu et al. Prompt-aligned gradient for prompt tuning. arXiv preprint arXiv:2205.14865, 2022. [find](https://scholar.google.com/scholar?q=B%2E+Zhu+et+al%2E+Prompt%2Daligned+gradient+for+prompt+tuning%2E+arXiv+preprint+arXiv%3A2205%2E14865%2C+2022%2E)
137. X. He et al. Cpl: Counterfactual prompt learning for vision and language models. arXiv preprint arXiv:2210.10362, 2022. [find](https://scholar.google.com/scholar?q=X%2E+He+et+al%2E+Cpl%3A+Counterfactual+prompt+learning+for+vision+and+language+models%2E+arXiv+preprint+arXiv%3A2210%2E10362%2C+2022%2E)
138. G. Chen et al. Prompt learning with optimal transport for vision-language models. arXiv preprint arXiv:2210.01253, 2022. — [arXiv:2210.01253](https://arxiv.org/abs/2210.01253) [find](https://scholar.google.com/scholar?q=G%2E+Chen+et+al%2E+Prompt+learning+with+optimal+transport+for+vision%2Dlanguage+models%2E+arXiv+preprint+arXiv%3A2210%2E01253%2C+2022%2E+%E2%80%94+%5BarXiv%3A2210%2E01253%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2210%2E01253%29)
139. X. Sun et al. Dualcoop: Fast adaptation to multi-label recognition with limited annotations. In NeurIPS. [find](https://scholar.google.com/scholar?q=X%2E+Sun+et+al%2E+Dualcoop%3A+Fast+adaptation+to+multi%2Dlabel+recognition+with+limited+annotations%2E+In+NeurIPS%2E)
140. Z. Guo et al. Texts as images in prompt tuning for multi-label image recognition. arXiv preprint arXiv:2211.12739, 2022. — [arXiv:2211.12739](https://arxiv.org/abs/2211.12739) [find](https://scholar.google.com/scholar?q=Z%2E+Guo+et+al%2E+Texts+as+images+in+prompt+tuning+for+multi%2Dlabel+image+recognition%2E+arXiv+preprint+arXiv%3A2211%2E12739%2C+2022%2E+%E2%80%94+%5BarXiv%3A2211%2E12739%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2211%2E12739%29)
141. K. Ding et al. Prompt tuning with soft context sharing for vision-language models. arXiv preprint arXiv:2208.13474, 2022. — [arXiv:2208.13474](https://arxiv.org/abs/2208.13474) [find](https://scholar.google.com/scholar?q=K%2E+Ding+et+al%2E+Prompt+tuning+with+soft+context+sharing+for+vision%2Dlanguage+models%2E+arXiv+preprint+arXiv%3A2208%2E13474%2C+2022%2E+%E2%80%94+%5BarXiv%3A2208%2E13474%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2208%2E13474%29)
142. Y. Rao et al. Denseclip: Language-guided dense prediction with context-aware prompting. In CVPR, pp. 18082–18091, 2022. [find](https://scholar.google.com/scholar?q=Y%2E+Rao+et+al%2E+Denseclip%3A+Language%2Dguided+dense+prediction+with+context%2Daware+prompting%2E+In+CVPR%2C+pp%2E+18082%E2%80%9318091%2C+2022%2E)
143. T. Huang et al. Unsupervised prompt learning for vision-language models. arXiv preprint arXiv:2204.03649, 2022. — [arXiv:2204.03649](https://arxiv.org/abs/2204.03649) [find](https://scholar.google.com/scholar?q=T%2E+Huang+et+al%2E+Unsupervised+prompt+learning+for+vision%2Dlanguage+models%2E+arXiv+preprint+arXiv%3A2204%2E03649%2C+2022%2E+%E2%80%94+%5BarXiv%3A2204%2E03649%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2204%2E03649%29)
144. M. Shu et al. Test-time prompt tuning for zero-shot generalization in vision-language models. In NeurIPS. [find](https://scholar.google.com/scholar?q=M%2E+Shu+et+al%2E+Test%2Dtime+prompt+tuning+for+zero%2Dshot+generalization+in+vision%2Dlanguage+models%2E+In+NeurIPS%2E)
145. H. Yao et al. Visual-language prompt tuning with knowledge-guided context optimization. In CVPR, pp. 6757–6767, 2023. [find](https://scholar.google.com/scholar?q=H%2E+Yao+et+al%2E+Visual%2Dlanguage+prompt+tuning+with+knowledge%2Dguided+context+optimization%2E+In+CVPR%2C+pp%2E+6757%E2%80%936767%2C+2023%2E)
146. T.-Y. Wu et al. Protect: Prompt tuning for hierarchical consistency. arXiv preprint arXiv:2306.02240, 2023. — [arXiv:2306.02240](https://arxiv.org/abs/2306.02240) [find](https://scholar.google.com/scholar?q=T%2E%2DY%2E+Wu+et+al%2E+Protect%3A+Prompt+tuning+for+hierarchical+consistency%2E+arXiv+preprint+arXiv%3A2306%2E02240%2C+2023%2E+%E2%80%94+%5BarXiv%3A2306%2E02240%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2306%2E02240%29)
147. H. Bahng et al. Exploring visual prompts for adapting large-scale models. arXiv preprint arXiv:2203.17274, 1(3):4, 2022. — [arXiv:2203.17274](https://arxiv.org/abs/2203.17274) [find](https://scholar.google.com/scholar?q=H%2E+Bahng+et+al%2E+Exploring+visual+prompts+for+adapting+large%2Dscale+models%2E+arXiv+preprint+arXiv%3A2203%2E17274%2C+1%283%29%3A4%2C+2022%2E+%E2%80%94+%5BarXiv%3A2203%2E17274%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2203%2E17274%29)
148. J. Rong et al. Retrieval-enhanced visual prompt learning for few-shot classification. arXiv preprint arXiv:2306.02243, 2023. — [arXiv:2306.02243](https://arxiv.org/abs/2306.02243) [find](https://scholar.google.com/scholar?q=J%2E+Rong+et+al%2E+Retrieval%2Denhanced+visual+prompt+learning+for+few%2Dshot+classification%2E+arXiv+preprint+arXiv%3A2306%2E02243%2C+2023%2E+%E2%80%94+%5BarXiv%3A2306%2E02243%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2306%2E02243%29)
149. Y. Zang et al. Unified vision and language prompt learning. arXiv preprint arXiv:2210.07225, 2022. — [arXiv:2210.07225](https://arxiv.org/abs/2210.07225) [find](https://scholar.google.com/scholar?q=Y%2E+Zang+et+al%2E+Unified+vision+and+language+prompt+learning%2E+arXiv+preprint+arXiv%3A2210%2E07225%2C+2022%2E+%E2%80%94+%5BarXiv%3A2210%2E07225%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2210%2E07225%29)
150. S. Shen et al. Multitask vision-language prompt tuning. arXiv preprint arXiv:2211.11720, 2022. [find](https://scholar.google.com/scholar?q=S%2E+Shen+et+al%2E+Multitask+vision%2Dlanguage+prompt+tuning%2E+arXiv+preprint+arXiv%3A2211%2E11720%2C+2022%2E)
151. M. U. Khattak et al. Maple: Multi-modal prompt learning. arXiv preprint arXiv:2210.03117, 2022. [find](https://scholar.google.com/scholar?q=M%2E+U%2E+Khattak+et+al%2E+Maple%3A+Multi%2Dmodal+prompt+learning%2E+arXiv+preprint+arXiv%3A2210%2E03117%2C+2022%2E)
152. Y. Xing et al. Class-aware visual prompt tuning for vision-language pre-trained model. arXiv preprint arXiv:2208.08340, 2022. — [arXiv:2208.08340](https://arxiv.org/abs/2208.08340) [find](https://scholar.google.com/scholar?q=Y%2E+Xing+et+al%2E+Class%2Daware+visual+prompt+tuning+for+vision%2Dlanguage+pre%2Dtrained+model%2E+arXiv+preprint+arXiv%3A2208%2E08340%2C+2022%2E+%E2%80%94+%5BarXiv%3A2208%2E08340%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2208%2E08340%29)
153. O. Pantazis et al. Svl-adapter: Self-supervised adapter for vision-language pretrained models. arXiv preprint arXiv:2210.03794, 2022. — [arXiv:2210.03794](https://arxiv.org/abs/2210.03794) [find](https://scholar.google.com/scholar?q=O%2E+Pantazis+et+al%2E+Svl%2Dadapter%3A+Self%2Dsupervised+adapter+for+vision%2Dlanguage+pretrained+models%2E+arXiv+preprint+arXiv%3A2210%2E03794%2C+2022%2E+%E2%80%94+%5BarXiv%3A2210%2E03794%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2210%2E03794%29)
154. V. Udandarao et al. Sus-x: Training-free name-only transfer of vision-language models. arXiv preprint arXiv:2211.16198, 2022. — [arXiv:2211.16198](https://arxiv.org/abs/2211.16198) [find](https://scholar.google.com/scholar?q=V%2E+Udandarao+et+al%2E+Sus%2Dx%3A+Training%2Dfree+name%2Donly+transfer+of+vision%2Dlanguage+models%2E+arXiv+preprint+arXiv%3A2211%2E16198%2C+2022%2E+%E2%80%94+%5BarXiv%3A2211%2E16198%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2211%2E16198%29)
155. J. Kahana et al. Improving zero-shot models with label distribution priors. arXiv preprint arXiv:2212.00784, 2022. — [arXiv:2212.00784](https://arxiv.org/abs/2212.00784) [find](https://scholar.google.com/scholar?q=J%2E+Kahana+et+al%2E+Improving+zero%2Dshot+models+with+label+distribution+priors%2E+arXiv+preprint+arXiv%3A2212%2E00784%2C+2022%2E+%E2%80%94+%5BarXiv%3A2212%2E00784%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2212%2E00784%29)
156. F. Peng et al. Sgva-clip: Semantic-guided visual adapting of vision-language models for few-shot image classification. arXiv preprint arXiv:2211.16191, 2022. — [arXiv:2211.16191](https://arxiv.org/abs/2211.16191) [find](https://scholar.google.com/scholar?q=F%2E+Peng+et+al%2E+Sgva%2Dclip%3A+Semantic%2Dguided+visual+adapting+of+vision%2Dlanguage+models+for+few%2Dshot+image+classification%2E+arXiv+preprint+arXiv%3A2211%2E16191%2C+2022%2E+%E2%80%94+%5BarXiv%3A2211%2E16191%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs)
157. R. Zhang et al. Vt-clip: Enhancing vision-language models with visual-guided texts. arXiv preprint arXiv:2112.02399, 2021. — [arXiv:2112.02399](https://arxiv.org/abs/2112.02399) [find](https://scholar.google.com/scholar?q=R%2E+Zhang+et+al%2E+Vt%2Dclip%3A+Enhancing+vision%2Dlanguage+models+with+visual%2Dguided+texts%2E+arXiv+preprint+arXiv%3A2112%2E02399%2C+2021%2E+%E2%80%94+%5BarXiv%3A2112%2E02399%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2112%2E02399%29)
158. Z. Guo et al. Calip: Zero-shot enhancement of clip with parameter-free attention. arXiv preprint arXiv:2209.14169, 2022. — [arXiv:2209.14169](https://arxiv.org/abs/2209.14169) [find](https://scholar.google.com/scholar?q=Z%2E+Guo+et+al%2E+Calip%3A+Zero%2Dshot+enhancement+of+clip+with+parameter%2Dfree+attention%2E+arXiv+preprint+arXiv%3A2209%2E14169%2C+2022%2E+%E2%80%94+%5BarXiv%3A2209%2E14169%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2209%2E14169%29)
159. T. Yu et al. Task residual for tuning vision-language models. In CVPR, pp. 10899–10909, 2023. [find](https://scholar.google.com/scholar?q=T%2E+Yu+et+al%2E+Task+residual+for+tuning+vision%2Dlanguage+models%2E+In+CVPR%2C+pp%2E+10899%E2%80%9310909%2C+2023%2E)
160. S. Pratt et al. What does a platypus look like? generating customized prompts for zero-shot image classification. arXiv preprint arXiv:2209.03320, 2022. [find](https://scholar.google.com/scholar?q=S%2E+Pratt+et+al%2E+What+does+a+platypus+look+like%3F+generating+customized+prompts+for+zero%2Dshot+image+classification%2E+arXiv+preprint+arXiv%3A2209%2E03320%2C+2022%2E)
161. S. Menon and C. Vondrick. Visual classification via description from large language models. arXiv preprint arXiv:2210.07183, 2022. — [arXiv:2210.07183](https://arxiv.org/abs/2210.07183) [find](https://scholar.google.com/scholar?q=S%2E+Menon+and+C%2E+Vondrick%2E+Visual+classification+via+description+from+large+language+models%2E+arXiv+preprint+arXiv%3A2210%2E07183%2C+2022%2E+%E2%80%94+%5BarXiv%3A2210%2E07183%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2210%2E07183%29)
162. M. Wortsman et al. Robust fine-tuning of zero-shot models. In CVPR, pp. 7959–7971, 2022. [find](https://scholar.google.com/scholar?q=M%2E+Wortsman+et+al%2E+Robust+fine%2Dtuning+of+zero%2Dshot+models%2E+In+CVPR%2C+pp%2E+7959%E2%80%937971%2C+2022%2E)
163. C. Zhou et al. Extract free dense labels from clip. In ECCV, pp. 696–712. Springer, 2022. [find](https://scholar.google.com/scholar?q=C%2E+Zhou+et+al%2E+Extract+free+dense+labels+from+clip%2E+In+ECCV%2C+pp%2E+696%E2%80%93712%2E+Springer%2C+2022%2E)
164. J. Li et al. Masked unsupervised self-training for zero-shot image classification. arXiv preprint arXiv:2206.02967, 2022. — [arXiv:2206.02967](https://arxiv.org/abs/2206.02967) [find](https://scholar.google.com/scholar?q=J%2E+Li+et+al%2E+Masked+unsupervised+self%2Dtraining+for+zero%2Dshot+image+classification%2E+arXiv+preprint+arXiv%3A2206%2E02967%2C+2022%2E+%E2%80%94+%5BarXiv%3A2206%2E02967%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2206%2E02967%29)
165. P. Liu et al. [Pre-train, prompt, and predict: A systematic survey of prompting methods in natural language processing](https://md.research.elamur.ai/free/en/papers/liu-2021-pre-train-prompt-predict-systematic/). ACM Computing Surveys, 55(9):1–35, 2023.
166. M. Jia et al. Visual prompt tuning. arXiv preprint arXiv:2203.12119, 2022. — [arXiv:2203.12119](https://arxiv.org/abs/2203.12119) [find](https://scholar.google.com/scholar?q=M%2E+Jia+et+al%2E+Visual+prompt+tuning%2E+arXiv+preprint+arXiv%3A2203%2E12119%2C+2022%2E+%E2%80%94+%5BarXiv%3A2203%2E12119%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2203%2E12119%29)
167. N. Houlsby et al. Parameter-efficient transfer learning for nlp. In ICML, pp. 2790–2799. PMLR, 2019. [find](https://scholar.google.com/scholar?q=N%2E+Houlsby+et+al%2E+Parameter%2Defficient+transfer+learning+for+nlp%2E+In+ICML%2C+pp%2E+2790%E2%80%932799%2E+PMLR%2C+2019%2E)
168. R. Zhang et al. Pointclip: Point cloud understanding by clip. In CVPR, pp. 8552–8562, 2022. [find](https://scholar.google.com/scholar?q=R%2E+Zhang+et+al%2E+Pointclip%3A+Point+cloud+understanding+by+clip%2E+In+CVPR%2C+pp%2E+8552%E2%80%938562%2C+2022%2E)
169. T. Huang et al. Clip2point: Transfer clip to point cloud classification with image-depth pre-training. In ICCV, pp. 22157–22167, 2023. [find](https://scholar.google.com/scholar?q=T%2E+Huang+et+al%2E+Clip2point%3A+Transfer+clip+to+point+cloud+classification+with+image%2Ddepth+pre%2Dtraining%2E+In+ICCV%2C+pp%2E+22157%E2%80%9322167%2C+2023%2E)
170. M. Xu et al. Side adapter network for open-vocabulary semantic segmentation. In CVPR, pp. 2945–2954, 2023. [find](https://scholar.google.com/scholar?q=M%2E+Xu+et+al%2E+Side+adapter+network+for+open%2Dvocabulary+semantic+segmentation%2E+In+CVPR%2C+pp%2E+2945%E2%80%932954%2C+2023%2E)
171. G. Chen et al. Tem-adapter: Adapting image-text pretraining for video question answer. In ICCV, pp. 13945–13955, 2023. [find](https://scholar.google.com/scholar?q=G%2E+Chen+et+al%2E+Tem%2Dadapter%3A+Adapting+image%2Dtext+pretraining+for+video+question+answer%2E+In+ICCV%2C+pp%2E+13945%E2%80%9313955%2C+2023%2E)
172. T. Brown et al. [Language models are few-shot learners](https://md.research.elamur.ai/free/en/papers/brown-2020-language-models-few-shot-learners/). NeurIPS, 33:1877–1901, 2020.
173. Y. Zang et al. Open-vocabulary detr with conditional matching. arXiv preprint arXiv:2203.11876, 2022. — [arXiv:2203.11876](https://arxiv.org/abs/2203.11876) [find](https://scholar.google.com/scholar?q=Y%2E+Zang+et+al%2E+Open%2Dvocabulary+detr+with+conditional+matching%2E+arXiv+preprint+arXiv%3A2203%2E11876%2C+2022%2E+%E2%80%94+%5BarXiv%3A2203%2E11876%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2203%2E11876%29)
174. Z. Zhou et al. Zegclip: Towards adapting clip for zero-shot semantic segmentation. arXiv preprint arXiv:2212.03588, 2022. [find](https://scholar.google.com/scholar?q=Z%2E+Zhou+et+al%2E+Zegclip%3A+Towards+adapting+clip+for+zero%2Dshot+semantic+segmentation%2E+arXiv+preprint+arXiv%3A2212%2E03588%2C+2022%2E)
175. T. Lüddecke and A. Ecker. Image segmentation using text and image prompts. In CVPR, pp. 7086–7096, 2022. [find](https://scholar.google.com/scholar?q=T%2E+Lu%CC%88ddecke+and+A%2E+Ecker%2E+Image+segmentation+using+text+and+image+prompts%2E+In+CVPR%2C+pp%2E+7086%E2%80%937096%2C+2022%2E)
176. B. Li et al. Language-driven semantic segmentation. In ICLR, 2021. [find](https://scholar.google.com/scholar?q=B%2E+Li+et+al%2E+Language%2Ddriven+semantic+segmentation%2E+In+ICLR%2C+2021%2E)
177. N. Zabari and Y. Hoshen. Semantic segmentation in-the-wild without seeing any segmentation examples. arXiv preprint arXiv:2112.03185, 2021. — [arXiv:2112.03185](https://arxiv.org/abs/2112.03185) [find](https://scholar.google.com/scholar?q=N%2E+Zabari+and+Y%2E+Hoshen%2E+Semantic+segmentation+in%2Dthe%2Dwild+without+seeing+any+segmentation+examples%2E+arXiv+preprint+arXiv%3A2112%2E03185%2C+2021%2E+%E2%80%94+%5BarXiv%3A2112%2E03185%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2112%2E03185%29)
178. C. Ma et al. Open-vocabulary semantic segmentation with frozen vision-language models. arXiv preprint arXiv:2210.15138, 2022. — [arXiv:2210.15138](https://arxiv.org/abs/2210.15138) [find](https://scholar.google.com/scholar?q=C%2E+Ma+et+al%2E+Open%2Dvocabulary+semantic+segmentation+with+frozen+vision%2Dlanguage+models%2E+arXiv+preprint+arXiv%3A2210%2E15138%2C+2022%2E+%E2%80%94+%5BarXiv%3A2210%2E15138%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2210%2E15138%29)
179. F. Liang et al. Open-vocabulary semantic segmentation with mask-adapted clip. arXiv preprint arXiv:2210.04150, 2022. [find](https://scholar.google.com/scholar?q=F%2E+Liang+et+al%2E+Open%2Dvocabulary+semantic+segmentation+with+mask%2Dadapted+clip%2E+arXiv+preprint+arXiv%3A2210%2E04150%2C+2022%2E)
180. M. Xu et al. A simple baseline for open-vocabulary semantic segmentation with pre-trained vision-language model. In ECCV, pp. 736–753. Springer, 2022. [find](https://scholar.google.com/scholar?q=M%2E+Xu+et+al%2E+A+simple+baseline+for+open%2Dvocabulary+semantic+segmentation+with+pre%2Dtrained+vision%2Dlanguage+model%2E+In+ECCV%2C+pp%2E+736%E2%80%93753%2E+Springer%2C+2022%2E)
181. G. Ghiasi et al. Scaling open-vocabulary image segmentation with image-level labels. In ECCV, pp. 540–557. Springer, 2022. [find](https://scholar.google.com/scholar?q=G%2E+Ghiasi+et+al%2E+Scaling+open%2Dvocabulary+image+segmentation+with+image%2Dlevel+labels%2E+In+ECCV%2C+pp%2E+540%E2%80%93557%2E+Springer%2C+2022%2E)
182. G. Shin et al. Reco: Retrieve and co-segment for zero-shot transfer. In NeurIPS. [find](https://scholar.google.com/scholar?q=G%2E+Shin+et+al%2E+Reco%3A+Retrieve+and+co%2Dsegment+for+zero%2Dshot+transfer%2E+In+NeurIPS%2E)
183. J. Xie et al. Clims: Cross language image matching for weakly supervised semantic segmentation. In CVPR, pp. 4483–4492, 2022. [find](https://scholar.google.com/scholar?q=J%2E+Xie+et+al%2E+Clims%3A+Cross+language+image+matching+for+weakly+supervised+semantic+segmentation%2E+In+CVPR%2C+pp%2E+4483%E2%80%934492%2C+2022%2E)
184. Y. Lin et al. Clip is also an efficient segmenter: A text-driven approach for weakly supervised semantic segmentation. arXiv preprint arXiv:2212.09506, 2022. [find](https://scholar.google.com/scholar?q=Y%2E+Lin+et+al%2E+Clip+is+also+an+efficient+segmenter%3A+A+text%2Ddriven+approach+for+weakly+supervised+semantic+segmentation%2E+arXiv+preprint+arXiv%3A2212%2E09506%2C+2022%2E)
185. J. Qin et al. Freeseg: Unified, universal and open-vocabulary image segmentation. In CVPR, pp. 19446–19455, 2023. [find](https://scholar.google.com/scholar?q=J%2E+Qin+et+al%2E+Freeseg%3A+Unified%2C+universal+and+open%2Dvocabulary+image+segmentation%2E+In+CVPR%2C+pp%2E+19446%E2%80%9319455%2C+2023%2E)
186. Z. Ma et al. Open-vocabulary one-stage detection with hierarchical visual-language knowledge distillation. In CVPR, pp. 14074– 14083, 2022. [find](https://scholar.google.com/scholar?q=Z%2E+Ma+et+al%2E+Open%2Dvocabulary+one%2Dstage+detection+with+hierarchical+visual%2Dlanguage+knowledge+distillation%2E+In+CVPR%2C+pp%2E+14074%E2%80%93+14083%2C+2022%2E)
187. H. A. Rasheed et al. Bridging the gap between object and image-level representations for open-vocabulary detection. In NeurIPS. [find](https://scholar.google.com/scholar?q=H%2E+A%2E+Rasheed+et+al%2E+Bridging+the+gap+between+object+and+image%2Dlevel+representations+for+open%2Dvocabulary+detection%2E+In+NeurIPS%2E)
188. C. Feng et al. Promptdet: Towards open-vocabulary detection using uncurated images. In ECCV, pp. 701–717. Springer, 2022. [find](https://scholar.google.com/scholar?q=C%2E+Feng+et+al%2E+Promptdet%3A+Towards+open%2Dvocabulary+detection+using+uncurated+images%2E+In+ECCV%2C+pp%2E+701%E2%80%93717%2E+Springer%2C+2022%2E)
189. M. Gao et al. Open vocabulary object detection with pseudo bounding-box labels. In ECCV, pp. 266–282. Springer, 2022. [find](https://scholar.google.com/scholar?q=M%2E+Gao+et+al%2E+Open+vocabulary+object+detection+with+pseudo+bounding%2Dbox+labels%2E+In+ECCV%2C+pp%2E+266%E2%80%93282%2E+Springer%2C+2022%2E)
190. T. Wang and N. Li. Learning to detect and segment for open vocabulary object detection. arXiv preprint arXiv:2212.12130, 2022. — [arXiv:2212.12130](https://arxiv.org/abs/2212.12130) [find](https://scholar.google.com/scholar?q=T%2E+Wang+and+N%2E+Li%2E+Learning+to+detect+and+segment+for+open+vocabulary+object+detection%2E+arXiv+preprint+arXiv%3A2212%2E12130%2C+2022%2E+%E2%80%94+%5BarXiv%3A2212%2E12130%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2212%2E12130%29)
191. C. Lin et al. Learning object-language alignments for open-vocabulary object detection. arXiv preprint arXiv:2211.14843, 2022. — [arXiv:2211.14843](https://arxiv.org/abs/2211.14843) [find](https://scholar.google.com/scholar?q=C%2E+Lin+et+al%2E+Learning+object%2Dlanguage+alignments+for+open%2Dvocabulary+object+detection%2E+arXiv+preprint+arXiv%3A2211%2E14843%2C+2022%2E+%E2%80%94+%5BarXiv%3A2211%2E14843%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2211%2E14843%29)
192. W. Kuo et al. F-vlm: Open-vocabulary object detection upon frozen vision and language models. arXiv preprint arXiv:2209.15639, 2022. — [arXiv:2209.15639](https://arxiv.org/abs/2209.15639) [find](https://scholar.google.com/scholar?q=W%2E+Kuo+et+al%2E+F%2Dvlm%3A+Open%2Dvocabulary+object+detection+upon+frozen+vision+and+language+models%2E+arXiv+preprint+arXiv%3A2209%2E15639%2C+2022%2E+%E2%80%94+%5BarXiv%3A2209%2E15639%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2209%2E15639%29)
193. X. Zhou et al. Detecting twenty-thousand classes using image-level supervision. In ECCV, pp. 350–368. Springer, 2022. [find](https://scholar.google.com/scholar?q=X%2E+Zhou+et+al%2E+Detecting+twenty%2Dthousand+classes+using+image%2Dlevel+supervision%2E+In+ECCV%2C+pp%2E+350%E2%80%93368%2E+Springer%2C+2022%2E)
194. D. Huynh et al. Open-vocabulary instance segmentation via robust cross-modal pseudo-labeling. In CVPR, pp. 7020–7031, 2022. [find](https://scholar.google.com/scholar?q=D%2E+Huynh+et+al%2E+Open%2Dvocabulary+instance+segmentation+via+robust+cross%2Dmodal+pseudo%2Dlabeling%2E+In+CVPR%2C+pp%2E+7020%E2%80%937031%2C+2022%2E)
195. M. Minderer et al. Simple open-vocabulary object detection with vision transformers. arXiv preprint arXiv:2205.06230, 2022. — [arXiv:2205.06230](https://arxiv.org/abs/2205.06230) [find](https://scholar.google.com/scholar?q=M%2E+Minderer+et+al%2E+Simple+open%2Dvocabulary+object+detection+with+vision+transformers%2E+arXiv+preprint+arXiv%3A2205%2E06230%2C+2022%2E+%E2%80%94+%5BarXiv%3A2205%2E06230%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2205%2E06230%29)
196. S. Zhao et al. Exploiting unlabeled data with vision and language models for object detection. In ECCV, pp. 159–175. Springer, 2022. [find](https://scholar.google.com/scholar?q=S%2E+Zhao+et+al%2E+Exploiting+unlabeled+data+with+vision+and+language+models+for+object+detection%2E+In+ECCV%2C+pp%2E+159%E2%80%93175%2E+Springer%2C+2022%2E)
197. Y. Long et al. P3ovd: Fine-grained visual-text prompt-driven self-training for open-vocabulary object detection. arXiv preprint arXiv:2211.00849, 2022. — [arXiv:2211.00849](https://arxiv.org/abs/2211.00849) [find](https://scholar.google.com/scholar?q=Y%2E+Long+et+al%2E+P3ovd%3A+Fine%2Dgrained+visual%2Dtext+prompt%2Ddriven+self%2Dtraining+for+open%2Dvocabulary+object+detection%2E+arXiv+preprint+arXiv%3A2211%2E00849%2C+2022%2E+%E2%80%94+%5BarXiv%3A2211%2E00849%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2211%2E)
198. J. Xie and S. Zheng. Zsd-yolo: Zero-shot yolo detection using vision-language knowledgedistillation. arXiv preprint arXiv:2109.12066, 2021. — [arXiv:2109.12066](https://arxiv.org/abs/2109.12066) [find](https://scholar.google.com/scholar?q=J%2E+Xie+and+S%2E+Zheng%2E+Zsd%2Dyolo%3A+Zero%2Dshot+yolo+detection+using+vision%2Dlanguage+knowledgedistillation%2E+arXiv+preprint+arXiv%3A2109%2E12066%2C+2021%2E+%E2%80%94+%5BarXiv%3A2109%2E12066%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2109%2E12066%29)
199. D. Kim et al. Region-aware pretraining for open-vocabulary object detection with vision transformers. In CVPR, pp. 11144– 11154, 2023. [find](https://scholar.google.com/scholar?q=D%2E+Kim+et+al%2E+Region%2Daware+pretraining+for+open%2Dvocabulary+object+detection+with+vision+transformers%2E+In+CVPR%2C+pp%2E+11144%E2%80%93+11154%2C+2023%2E)
200. S. Wu et al. Aligning bag of regions for open-vocabulary object detection. In CVPR, pp. 15254–15264, 2023. [find](https://scholar.google.com/scholar?q=S%2E+Wu+et+al%2E+Aligning+bag+of+regions+for+open%2Dvocabulary+object+detection%2E+In+CVPR%2C+pp%2E+15254%E2%80%9315264%2C+2023%2E)
201. L. Wang et al. Object-aware distillation pyramid for open-vocabulary object detection. In CVPR, pp. 11186–11196, 2023. [find](https://scholar.google.com/scholar?q=L%2E+Wang+et+al%2E+Object%2Daware+distillation+pyramid+for+open%2Dvocabulary+object+detection%2E+In+CVPR%2C+pp%2E+11186%E2%80%9311196%2C+2023%2E)
202. M. Cherti et al. [Reproducible scaling laws for contrastive language-image learning](https://md.research.elamur.ai/free/en/papers/cherti-2022-reproducible-scaling-laws-contrastive-language/). In CVPR, pp. 2818–2829, 2023.
203. Y. Xian et al. Semantic projection network for zero-and few-label semantic segmentation. In CVPR, pp. 8256–8265, 2019. [find](https://scholar.google.com/scholar?q=Y%2E+Xian+et+al%2E+Semantic+projection+network+for+zero%2Dand+few%2Dlabel+semantic+segmentation%2E+In+CVPR%2C+pp%2E+8256%E2%80%938265%2C+2019%2E)
204. X. Zhai et al. Scaling vision transformers. In CVPR, pp. 12104– 12113, 2022. [find](https://scholar.google.com/scholar?q=X%2E+Zhai+et+al%2E+Scaling+vision+transformers%2E+In+CVPR%2C+pp%2E+12104%E2%80%93+12113%2C+2022%2E)
205. C. Raffel et al. [Exploring the limits of transfer learning with a unified text-to-text transformer](https://md.research.elamur.ai/free/en/papers/raffel-2019-exploring-limits-transfer-learning-unified/). J. Mach. Learn. Res., 21(140):1–67, 2020. A PPENDIX A. S TATISTICS ON V ISUAL R ECOGNITION VLM P UBLICATIONS

*Updated 2 November 2026*