---
title: "Zero-Shot Multimodal Retrieval with Multi-Scale Contextual"
description: "The authors propose Multi-Score, a training-free, two-stage pipeline. 64 references. Among them: 1."
authors: ["Sourajit Saha", "Tejas Gokhale"]
year: 2026
venue: "Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)"
source: "https://aclanthology.org/2026.acl-long.930/"
doi: "10.18653/v1/2026.acl-long.930"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/saha-2026-zero-shot-multimodal-retrieval-multi/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/saha-2026-zero-shot-multimodal-retrieval-multi/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/19/) / Zero-Shot Multimodal Retrieval with Multi-Scale Contextual Representations

# Zero-Shot Multimodal Retrieval with Multi-Scale Contextual Representations

**Authors:** Sourajit Saha and Tejas Gokhale · **Year:** 2026 · **Venue:** [Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)](https://md.research.elamur.ai/free/en/venues/acl/) ([ACL 2026](https://md.research.elamur.ai/free/en/venues/acl/2026/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2026.acl-long.930/) · [PDF](https://aclanthology.org/2026.acl-long.930.pdf) · [DOI 10.18653/v1/2026.acl-long.930](https://doi.org/10.18653/v1/2026.acl-long.930) · [OpenAlex](https://openalex.org/W7166807435)

**Citations:** 1 citations (OpenAlex, 2026-10-14)

## Summary
The paper addresses multimodal information retrieval (MMIR), where queries and database items may be text, images, video, or audio. Supervised retrievers need annotation and retraining as databases change, and they often transfer poorly across modalities. The authors propose Multi-Score, a training-free, two-stage pipeline.

In Stage 1, all inputs are converted to text with Qwen-family captioning and transcription models and then embedded with Qwen3-MRL Matryoshka embeddings. A new algorithm, Pyramid Rank, starts with short embedding prefixes and uses a Cauchy-Schwarz-based upper bound on similarity to drop weak candidates early. It moves to longer embeddings only when needed to isolate the top-K.

In Stage 2, the remaining candidates are re-ranked with two signals:
- **Bidirectional-CoT Embedding Score:** chain-of-thought prompting runs in both query-to-candidate and candidate-to-query directions, and an \<emb> token's hidden state serves as the embedding.
- **Question Answering Relevance Score:** an MLLM checks how well each candidate answers questions generated from the query.

Multi-Score reaches state-of-the-art zero-shot results and often beats supervised systems. This matters for evolving, mixed-modality collections where retraining is costly.

## Key points
- Evaluated on 12 MMIR tasks spanning 32 datasets, plus a joint setting that merges all of them into a 5.7M-item database. It outperforms supervised methods on 23 datasets.
- Stage 1 uses Qwen3-VL-8B, Qwen2-Audio-7B and Qwen2.5-Omni-7B to convert inputs to text, then Qwen3-MRL embeddings at 6 nested levels (32 up to 1024 dimensions).
- The authors claim retrieval about 6x faster than the previous SOTA. Per-query online cost is 0.87 s, offline captioning of the 5.7M database takes 20 GPU-hours, and storage is about 16.7 KB per video versus 700–1400 KB for the comparison methods.
- With a Qwen3-VL-8B (MRL) backbone, R@1 rises from 74.3 (embeddings alone) and 77.1 (standard LLaVA-7B re-ranking) to 82.7 with Multi-Score. Ablations indicate the gains are largely independent of backbone family, size and prompt wording.
- Limitation: Qwen3-MRL is text-only, so non-text data must be captioned first, which may lose information and adds preprocessing cost.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [University of Maryland, Baltimore County](https://md.research.elamur.ai/free/en/universities/university-of-maryland-baltimore-county/) — University, Baltimore, United States · 14 works · 915 citations

📚 Shelves (3): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [👁️ Computer Vision and Pattern Recognition](https://md.research.elamur.ai/free/en/shelves/computer-science/computer-vision-and-pattern-recognition/) · 2,869 works · [🗄️ Information Systems](https://md.research.elamur.ai/free/en/shelves/computer-science/information-systems/) · 1,437 works

🧭 Topics (2): [🖼️ Vision & language](https://md.research.elamur.ai/free/en/topics/multimodal/) · 657 works · [🔍 Information retrieval & search](https://md.research.elamur.ai/free/en/topics/information-retrieval/) · 473 works

## Cite

Cite as: Sourajit Saha and Tejas Gokhale. 2026. Zero-Shot Multimodal Retrieval with Multi-Scale Contextual Representations. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{saha2026zero,
  title = {{Zero-Shot Multimodal Retrieval with Multi-Scale Contextual Representations}},
  author = {Sourajit Saha and Tejas Gokhale},
  booktitle = {Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2026},
  doi = {10.18653/v1/2026.acl-long.930},
  url = {https://doi.org/10.18653/v1/2026.acl-long.930}
}
```

- [← previous Your LLM Agent Can Leak Your Data: Data Exfiltration via Backdoored Tool Use](https://md.research.elamur.ai/free/en/papers/zhang-2026-your-llm-agent-can-leak/)
- [next → zhangpeng at SemEval-2026 Task 10: PsyCoMark - Psycholinguistic Conspiracy Marker Extraction and Detection](https://md.research.elamur.ai/free/en/papers/peng-2026-zhangpeng-semeval-2026-task-10/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/19/)

## References

64 references, 14 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Edward H Adelson, Charles H Anderson, James R Bergen, Peter J Burt, and Joan M Ogden. 1984. Pyramid methods in image processing. RCA engineer, 29(6):33–41. [find](https://scholar.google.com/scholar?q=Edward+H+Adelson%2C+Charles+H+Anderson%2C+James+R+Bergen%2C+Peter+J+Burt%2C+and+Joan+M+Ogden%2E+1984%2E+Pyramid+methods+in+image+processing%2E+RCA+engineer%2C+29%286%29%3A33%E2%80%9341%2E)
2. Elad Amrani, Rami Ben-Ari, Daniel Rotman, and Alex Bronstein. 2021. Noise estimation using density estimation for self-supervised multimodal learning. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pages 6644–6652. [find](https://scholar.google.com/scholar?q=Elad+Amrani%2C+Rami+Ben%2DAri%2C+Daniel+Rotman%2C+and+Alex+Bronstein%2E+2021%2E+Noise+estimation+using+density+estimation+for+self%2Dsupervised+multimodal+learning%2E+In+Proceedings+of+the+AAAI+Conference+on+Artifici)
3. Max Bain, Arsha Nagrani, Gül Varol, and Andrew Zisserman. 2021. Frozen in time: A joint video and image encoder for end-to-end retrieval. In Proceedings of the IEEE/CVF international conference on computer vision, pages 1728–1738. [find](https://scholar.google.com/scholar?q=Max+Bain%2C+Arsha+Nagrani%2C+G%C3%BCl+Varol%2C+and+Andrew+Zisserman%2E+2021%2E+Frozen+in+time%3A+A+joint+video+and+image+encoder+for+end%2Dto%2Dend+retrieval%2E+In+Proceedings+of+the+IEEE%2FCVF+international+conference+on+com)
4. Harry G Barrow and Jay M Tenenbaum. 1981. Computational vision. Proceedings of the IEEE, 69(5):572– 595. [find](https://scholar.google.com/scholar?q=Harry+G+Barrow+and+Jay+M+Tenenbaum%2E+1981%2E+Computational+vision%2E+Proceedings+of+the+IEEE%2C+69%285%29%3A572%E2%80%93+595%2E)
5. Joao Carreira and Andrew Zisserman. 2017. Quo vadis, action recognition? a new model and the kinetics dataset. In proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 6299–6308. [find](https://scholar.google.com/scholar?q=Joao+Carreira+and+Andrew+Zisserman%2E+2017%2E+Quo+vadis%2C+action+recognition%3F+a+new+model+and+the+kinetics+dataset%2E+In+proceedings+of+the+IEEE+Conference+on+Computer+Vision+and+Pattern+Recognition%2C+pages+6)
6. David Chen and William B Dolan. 2011. [Collecting highly parallel data for paraphrase evaluation](https://md.research.elamur.ai/free/en/papers/chen-2011-collecting-highly-parallel-data-paraphrase/). In Proceedings of the 49th annual meeting of the association for computational linguistics: human language technologies, pages 190–200.
7. Xing Cheng, Hezheng Lin, Xiangyu Wu, Fan Yang, and Dong Shen. 2021. Improving video-text retrieval by multi-stream corpus alignment and dual softmax loss. arXiv preprint arXiv:2109.04290. — [arXiv:2109.04290](https://arxiv.org/abs/2109.04290) [find](https://scholar.google.com/scholar?q=Xing+Cheng%2C+Hezheng+Lin%2C+Xiangyu+Wu%2C+Fan+Yang%2C+and+Dong+Shen%2E+2021%2E+Improving+video%2Dtext+retrieval+by+multi%2Dstream+corpus+alignment+and+dual+softmax+loss%2E+arXiv+preprint+arXiv%3A2109%2E04290%2E+%E2%80%94+%5BarXiv%3A210)
8. Yunfei Chu, Jin Xu, Qian Yang, Haojie Wei, Xipin Wei, Zhifang Guo, Yichong Leng, Yuanjun Lv, Jinzheng He, Junyang Lin, Chang Zhou, and Jingren Zhou. 2024. [Qwen2-audio technical report](https://md.research.elamur.ai/free/en/papers/chu-2024-qwen2-audio-technical-report/). arXiv preprint arXiv:2407.10759.
9. Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. 2009. Imagenet: A large-scale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition, pages 248–255. Ieee. [find](https://scholar.google.com/scholar?q=Jia+Deng%2C+Wei+Dong%2C+Richard+Socher%2C+Li%2DJia+Li%2C+Kai+Li%2C+and+Li+Fei%2DFei%2E+2009%2E+Imagenet%3A+A+large%2Dscale+hierarchical+image+database%2E+In+2009+IEEE+conference+on+computer+vision+and+pattern+recognition%2C+pa)
10. Konstantinos Drossos, Samuel Lipping, and Tuomas Virtanen. 2020. Clotho: An audio captioning dataset. In ICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 736–740. IEEE. [find](https://scholar.google.com/scholar?q=Konstantinos+Drossos%2C+Samuel+Lipping%2C+and+Tuomas+Virtanen%2E+2020%2E+Clotho%3A+An+audio+captioning+dataset%2E+In+ICASSP+2020%2D2020+IEEE+International+Conference+on+Acoustics%2C+Speech+and+Signal+Processing+%28ICAS)
11. Christoph Feichtenhofer. 2020. X3d: Expanding architectures for efficient video recognition. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 203–213. [find](https://scholar.google.com/scholar?q=Christoph+Feichtenhofer%2E+2020%2E+X3d%3A+Expanding+architectures+for+efficient+video+recognition%2E+In+Proceedings+of+the+IEEE%2FCVF+conference+on+computer+vision+and+pattern+recognition%2C+pages+203%E2%80%93213%2E)
12. Christoph Feichtenhofer, Haoqi Fan, Jitendra Malik, and Kaiming He. 2019. Slowfast networks for video recognition. In Proceedings of the IEEE/CVF international conference on computer vision, pages 6202–6211. [find](https://scholar.google.com/scholar?q=Christoph+Feichtenhofer%2C+Haoqi+Fan%2C+Jitendra+Malik%2C+and+Kaiming+He%2E+2019%2E+Slowfast+networks+for+video+recognition%2E+In+Proceedings+of+the+IEEE%2FCVF+international+conference+on+computer+vision%2C+pages+620)
13. Stephanie Fu, Netanel Tamir, Shobhita Sundaram, Lucy Chai, Richard Zhang, Tali Dekel, and Phillip Isola. 2023. Dreamsim: Learning new dimensions of human visual similarity using synthetic data. Advances in Neural Information Processing Systems, 36:50742– 50768. [find](https://scholar.google.com/scholar?q=Stephanie+Fu%2C+Netanel+Tamir%2C+Shobhita+Sundaram%2C+Lucy+Chai%2C+Richard+Zhang%2C+Tali+Dekel%2C+and+Phillip+Isola%2E+2023%2E+Dreamsim%3A+Learning+new+dimensions+of+human+visual+similarity+using+synthetic+data%2E+Advanc)
14. Yuying Ge, Yixiao Ge, Xihui Liu, Dian Li, Ying Shan, Xiaohu Qie, and Ping Luo. 2022. Bridging video-text retrieval with multiple choice questions. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 16167–16176. [find](https://scholar.google.com/scholar?q=Yuying+Ge%2C+Yixiao+Ge%2C+Xihui+Liu%2C+Dian+Li%2C+Ying+Shan%2C+Xiaohu+Qie%2C+and+Ping+Luo%2E+2022%2E+Bridging+video%2Dtext+retrieval+with+multiple+choice+questions%2E+In+Proceedings+of+the+IEEE%2FCVF+conference+on+computer)
15. Satya Krishna Gorti, Noël Vouitsis, Junwei Ma, Keyvan Golestan, Maksims Volkovs, Animesh Garg, and Guangwei Yu. 2022. X-pool: Cross-modal language-video attention for text-video retrieval. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 5006–5015. [find](https://scholar.google.com/scholar?q=Satya+Krishna+Gorti%2C+No%C3%ABl+Vouitsis%2C+Junwei+Ma%2C+Keyvan+Golestan%2C+Maksims+Volkovs%2C+Animesh+Garg%2C+and+Guangwei+Yu%2E+2022%2E+X%2Dpool%3A+Cross%2Dmodal+language%2Dvideo+attention+for+text%2Dvideo+retrieval%2E+In+Proceedi)
16. Huakai Lai, Guoxin Xiong, Huayu Mai, Xiang Liu, and Tianzhu Zhang. 2025. Rethinking noisy video-text retrieval via relation-aware alignment. In Proceedings of the Computer Vision and Pattern Recognition Conference, pages 9231–9241. [find](https://scholar.google.com/scholar?q=Huakai+Lai%2C+Guoxin+Xiong%2C+Huayu+Mai%2C+Xiang+Liu%2C+and+Tianzhu+Zhang%2E+2025%2E+Rethinking+noisy+video%2Dtext+retrieval+via+relation%2Daware+alignment%2E+In+Proceedings+of+the+Computer+Vision+and+Pattern+Recogniti)
17. Andrey Guzhov, Federico Raue, Jörn Hees, and Andreas Dengel. 2022. Audioclip: Extending clip to image, text and audio. In ICASSP 2022-2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 976–980. IEEE. [find](https://scholar.google.com/scholar?q=Andrey+Guzhov%2C+Federico+Raue%2C+J%C3%B6rn+Hees%2C+and+Andreas+Dengel%2E+2022%2E+Audioclip%3A+Extending+clip+to+image%2C+text+and+audio%2E+In+ICASSP+2022%2D2022+IEEE+International+Conference+on+Acoustics%2C+Speech+and+Signal)
18. Svetlana Lazebnik, Cordelia Schmid, and Jean Ponce. 2006. Beyond bags of features: Spatial pyramid matching for recognizing natural scene categories. In 2006 IEEE computer society conference on computer vision and pattern recognition (CVPR’06), volume 2, pages 2169–2178. IEEE. [find](https://scholar.google.com/scholar?q=Svetlana+Lazebnik%2C+Cordelia+Schmid%2C+and+Jean+Ponce%2E+2006%2E+Beyond+bags+of+features%3A+Spatial+pyramid+matching+for+recognizing+natural+scene+categories%2E+In+2006+IEEE+computer+society+conference+on+comput)
19. Efstathios Hadjidemetriou, Michael D Grossberg, and Shree K Nayar. 2004. Multiresolution histograms and their use for recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 26(7):831–847. [find](https://scholar.google.com/scholar?q=Efstathios+Hadjidemetriou%2C+Michael+D+Grossberg%2C+and+Shree+K+Nayar%2E+2004%2E+Multiresolution+histograms+and+their+use+for+recognition%2E+IEEE+Transactions+on+Pattern+Analysis+and+Machine+Intelligence%2C+26%287%29)
20. Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi. 2023. [Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models](https://md.research.elamur.ai/free/en/papers/li-2023-blip-2-bootstrapping-language-image/). In International conference on machine learning, pages 19730–19742. PMLR.
21. Godfrey Harold Hardy, John Edensor Littlewood, and George Pólya. 1952. Inequalities. Cambridge university press. [find](https://scholar.google.com/scholar?q=Godfrey+Harold+Hardy%2C+John+Edensor+Littlewood%2C+and+George+P%C3%B3lya%2E+1952%2E+Inequalities%2E+Cambridge+university+press%2E)
22. Junnan Li, Dongxu Li, Caiming Xiong, and Steven Hoi. 2022. [Blip: Bootstrapping language-image pre-training for unified vision-language understanding and generation](https://md.research.elamur.ai/free/en/papers/li-2022-blip-bootstrapping-language-image-pre/). In International conference on machine learning, pages 12888–12900. PMLR.
23. Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. [Deep residual learning for image recognition](https://md.research.elamur.ai/free/en/papers/he-2015-deep-residual-learning-image-recognition/). In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR).
24. Lisa Anne Hendricks, Oliver Wang, Eli Shechtman, Josef Sivic, Trevor Darrell, and Bryan Russell. 2017. Localizing moments in video with natural language. In Proceedings of the IEEE international conference on computer vision, pages 5803–5812. [find](https://scholar.google.com/scholar?q=Lisa+Anne+Hendricks%2C+Oliver+Wang%2C+Eli+Shechtman%2C+Josef+Sivic%2C+Trevor+Darrell%2C+and+Bryan+Russell%2E+2017%2E+Localizing+moments+in+video+with+natural+language%2E+In+Proceedings+of+the+IEEE+international+confe)
25. Ting Jiang, Shaohan Huang, Zhongzhi Luan, Deqing Wang, and Fuzhen Zhuang. 2024a. [Scaling sentence embeddings with large language models](https://md.research.elamur.ai/free/en/papers/jiang-2024-scaling-sentence-embeddings-large-language/). In Findings of the Association for Computational Linguistics: EMNLP 2024, pages 3182–3196.
26. Ting Jiang, Minghui Song, Zihan Zhang, Haizhen Huang, Weiwei Deng, Feng Sun, Qi Zhang, Deqing Wang, and Fuzhen Zhuang. 2024b. E5-v: Universal embeddings with multimodal large language models. arXiv preprint arXiv:2407.12580. — [arXiv:2407.12580](https://arxiv.org/abs/2407.12580) [find](https://scholar.google.com/scholar?q=Ting+Jiang%2C+Minghui+Song%2C+Zihan+Zhang%2C+Haizhen+Huang%2C+Weiwei+Deng%2C+Feng+Sun%2C+Qi+Zhang%2C+Deqing+Wang%2C+and+Fuzhen+Zhuang%2E+2024b%2E+E5%2Dv%3A+Universal+embeddings+with+multimodal+large+language+models%2E+arXiv+pr)
27. Yongqi Li, Wenjie Wang, Leigang Qu, Liqiang Nie, Wenjie Li, and Tat-Seng Chua. 2024. Generative cross-modal retrieval: Memorizing images in multimodal language models for retrieval and beyond. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long… [find](https://scholar.google.com/scholar?q=Yongqi+Li%2C+Wenjie+Wang%2C+Leigang+Qu%2C+Liqiang+Nie%2C+Wenjie+Li%2C+and+Tat%2DSeng+Chua%2E+2024%2E+Generative+cross%2Dmodal+retrieval%3A+Memorizing+images+in+multimodal+language+models+for+retrieval+and+beyond%2E+In+Proc)
28. Zengrong Lin, Zheng Wang, Tianwen Qian, Pan Mu, Sixian Chan, and Cong Bai. 2025. Neighborretr: Balancing hub centrality in cross-modal retrieval. In Proceedings of the Computer Vision and Pattern Recognition Conference, pages 9263–9273. [find](https://scholar.google.com/scholar?q=Zengrong+Lin%2C+Zheng+Wang%2C+Tianwen+Qian%2C+Pan+Mu%2C+Sixian+Chan%2C+and+Cong+Bai%2E+2025%2E+Neighborretr%3A+Balancing+hub+centrality+in+cross%2Dmodal+retrieval%2E+In+Proceedings+of+the+Computer+Vision+and+Pattern+Reco)
29. Yikun Liu, Yajie Zhang, Jiayin Cai, Xiaolong Jiang, Yao Hu, Jiangchao Yao, Yanfeng Wang, and Weidi Xie. 2025. Lamra: Large multimodal model as your advanced retrieval assistant. In Proceedings of the Computer Vision and Pattern Recognition Conference, pages 4015–4025. [find](https://scholar.google.com/scholar?q=Yikun+Liu%2C+Yajie+Zhang%2C+Jiayin+Cai%2C+Xiaolong+Jiang%2C+Yao+Hu%2C+Jiangchao+Yao%2C+Yanfeng+Wang%2C+and+Weidi+Xie%2E+2025%2E+Lamra%3A+Large+multimodal+model+as+your+advanced+retrieval+assistant%2E+In+Proceedings+of+the+)
30. Chris Dongjoo Kim, Byeongchang Kim, Hyunmin Lee, and Gunhee Kim. 2019. [Audiocaps: Generating captions for audios in the wild](https://md.research.elamur.ai/free/en/papers/kim-2019-audiocaps-generating-captions-audios-wild/). In Proceedings of NAACL-HLT, pages 119–132.
31. Huaishao Luo, Lei Ji, Ming Zhong, Yang Chen, Wen Lei, Nan Duan, and Tianrui Li. 2022. Clip4clip: An empirical study of clip for end to end video clip retrieval and captioning. Neurocomputing, 508:293– 304. [find](https://scholar.google.com/scholar?q=Huaishao+Luo%2C+Lei+Ji%2C+Ming+Zhong%2C+Yang+Chen%2C+Wen+Lei%2C+Nan+Duan%2C+and+Tianrui+Li%2E+2022%2E+Clip4clip%3A+An+empirical+study+of+clip+for+end+to+end+video+clip+retrieval+and+captioning%2E+Neurocomputing%2C+508%3A293%E2%80%93)
32. Sungyeon Kim, Xinliang Zhu, Xiaofan Lin, Muhammet Bastan, Douglas Gray, and Suha Kwak. 2025. Genius: A generative framework for universal multimodal search. In Proceedings of the Computer Vision and Pattern Recognition Conference, pages 19659– 19669. [find](https://scholar.google.com/scholar?q=Sungyeon+Kim%2C+Xinliang+Zhu%2C+Xiaofan+Lin%2C+Muhammet+Bastan%2C+Douglas+Gray%2C+and+Suha+Kwak%2E+2025%2E+Genius%3A+A+generative+framework+for+universal+multimodal+search%2E+In+Proceedings+of+the+Computer+Vision+and+P)
33. Man Luo, Zhiyuan Fang, Tejas Gokhale, Yezhou Yang, and Chitta Baral. 2023. End-to-end knowledge retrieval with multi-modal queries. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 8573–8589. [find](https://scholar.google.com/scholar?q=Man+Luo%2C+Zhiyuan+Fang%2C+Tejas+Gokhale%2C+Yezhou+Yang%2C+and+Chitta+Baral%2E+2023%2E+End%2Dto%2Dend+knowledge+retrieval+with+multi%2Dmodal+queries%2E+In+Proceedings+of+the+61st+Annual+Meeting+of+the+Association+for+Com)
34. Aditya Kusupati, Gantavya Bhatt, Aniket Rege, Matthew Wallingford, Aditya Sinha, Vivek Ramanujan, William Howard-Snyder, Kaifeng Chen, Sham Kakade, Prateek Jain, and 1 others. 2022. Matryoshka representation learning. Advances in Neural Information Processing Systems, 35:30233–30249. [find](https://scholar.google.com/scholar?q=Aditya+Kusupati%2C+Gantavya+Bhatt%2C+Aniket+Rege%2C+Matthew+Wallingford%2C+Aditya+Sinha%2C+Vivek+Ramanujan%2C+William+Howard%2DSnyder%2C+Kaifeng+Chen%2C+Sham+Kakade%2C+Prateek+Jain%2C+and+1+others%2E+2022%2E+Matryoshka+represe)
35. David Marr. 1982. Vision: A computational investigation into the human representation and processing of visual information. [find](https://scholar.google.com/scholar?q=David+Marr%2E+1982%2E+Vision%3A+A+computational+investigation+into+the+human+representation+and+processing+of+visual+information%2E)
36. Arsha Nagrani, Paul Hongsuck Seo, Bryan Seybold, Anja Hauth, Santiago Manen, Chen Sun, and Cordelia Schmid. 2022. Learning audio-video modalities from image captions. In European Conference on Computer Vision, pages 407–426. Springer. similarity. In Proceedings of the IEEE/CVF Conference on Computer… [find](https://scholar.google.com/scholar?q=Arsha+Nagrani%2C+Paul+Hongsuck+Seo%2C+Bryan+Seybold%2C+Anja+Hauth%2C+Santiago+Manen%2C+Chen+Sun%2C+and+Cordelia+Schmid%2E+2022%2E+Learning+audio%2Dvideo+modalities+from+image+captions%2E+In+European+Conference+on+Compute)
37. Andreea-Maria Oncescu, A Koepke, João F Henriques, Zeynep Akata, and Samuel Albanie. 2021. Audio retrieval with natural language queries. In Proc. [find](https://scholar.google.com/scholar?q=Andreea%2DMaria+Oncescu%2C+A+Koepke%2C+Jo%C3%A3o+F+Henriques%2C+Zeynep+Akata%2C+and+Samuel+Albanie%2E+2021%2E+Audio+retrieval+with+natural+language+queries%2E+In+Proc%2E)
38. Interspeech 2021, pages 2411–2415. [find](https://scholar.google.com/scholar?q=Interspeech+2021%2C+pages+2411%E2%80%932415%2E)
39. Jinpeng Wang, Yixiao Ge, Guanyu Cai, Rui Yan, Xudong Lin, Ying Shan, Xiaohu Qie, and Mike Zheng Shou. 2022. Object-aware video-language pre-training for retrieval. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 3313–3322. [find](https://scholar.google.com/scholar?q=Jinpeng+Wang%2C+Yixiao+Ge%2C+Guanyu+Cai%2C+Rui+Yan%2C+Xudong+Lin%2C+Ying+Shan%2C+Xiaohu+Qie%2C+and+Mike+Zheng+Shou%2E+2022%2E+Object%2Daware+video%2Dlanguage+pre%2Dtraining+for+retrieval%2E+In+Proceedings+of+the+IEEE%2FCVF+confe)
40. Bryan A Plummer, Liwei Wang, Chris M Cervantes, Juan C Caicedo, Julia Hockenmaier, and Svetlana Lazebnik. 2015. Flickr30k entities: Collecting region-to-phrase correspondences for richer image-to-sentence models. In Proceedings of the IEEE international conference on computer vision, pages 2641–2649… [find](https://scholar.google.com/scholar?q=Bryan+A+Plummer%2C+Liwei+Wang%2C+Chris+M+Cervantes%2C+Juan+C+Caicedo%2C+Julia+Hockenmaier%2C+and+Svetlana+Lazebnik%2E+2015%2E+Flickr30k+entities%3A+Collecting+region%2Dto%2Dphrase+correspondences+for+richer+image%2Dto%2Dsent)
41. Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, and 1 others. 2021. [Learning transferable visual models from natural language supervision](https://md.research.elamur.ai/free/en/papers/radford-2021-learning-transferable-visual-models-natural/). In International conference on machine learning, pages 8748–8…
42. Arun Reddy, Alexander Martin, Eugene Yang, Andrew Yates, Kate Sanders, Kenton Murray, Reno Kriz, Celso M de Melo, Benjamin Van Durme, and Rama Chellappa. 2025. Video-colbert: Contextualized late interaction for text-to-video retrieval. In Proceedings of the Computer Vision and Pattern Recognition Co… [find](https://scholar.google.com/scholar?q=Arun+Reddy%2C+Alexander+Martin%2C+Eugene+Yang%2C+Andrew+Yates%2C+Kate+Sanders%2C+Kenton+Murray%2C+Reno+Kriz%2C+Celso+M+de+Melo%2C+Benjamin+Van+Durme%2C+and+Rama+Chellappa%2E+2025%2E+Video%2Dcolbert%3A+Contextualized+late+inter)
43. Anna Rohrbach, Marcus Rohrbach, Niket Tandon, and Bernt Schiele. 2015. A dataset for movie description. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 3202–3212. [find](https://scholar.google.com/scholar?q=Anna+Rohrbach%2C+Marcus+Rohrbach%2C+Niket+Tandon%2C+and+Bernt+Schiele%2E+2015%2E+A+dataset+for+movie+description%2E+In+Proceedings+of+the+IEEE+conference+on+computer+vision+and+pattern+recognition%2C+pages+3202%E2%80%93321)
44. Hermann Amandus Schwarz. 1890. Ueber ein die flächen kleinsten flächeninhalts betreffendes problem der variationsrechnung: Festschrift zum siebzigsten geburtstage des herrn karl weierstrass. In Gesammelte Mathematische Abhandlungen: Erster Band, pages 223–269. Springer. [find](https://scholar.google.com/scholar?q=Hermann+Amandus+Schwarz%2E+1890%2E+Ueber+ein+die+fl%C3%A4chen+kleinsten+fl%C3%A4cheninhalts+betreffendes+problem+der+variationsrechnung%3A+Festschrift+zum+siebzigsten+geburtstage+des+herrn+karl+weierstrass%2E+In+Gesamm)
45. Peng Wang, Shuai Bai, Sinan Tan, Shijie Wang, Zhihao Fan, Jinze Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Yang Fan, Kai Dang, Mengfei Du, Xuancheng Ren, Rui Men, Dayiheng Liu, Chang Zhou, Jingren Zhou, and Junyang Lin. 2024. [Qwen2-vl: Enhancing vision-language model’s perception of the w…](https://md.research.elamur.ai/free/en/papers/wang-2024-qwen2-vl-enhancing-vision/)
46. Xiaohan Wang, Linchao Zhu, and Yi Yang. 2021. T2vlad: global-local sequence alignment for text-video retrieval. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 5079–5088. [find](https://scholar.google.com/scholar?q=Xiaohan+Wang%2C+Linchao+Zhu%2C+and+Yi+Yang%2E+2021%2E+T2vlad%3A+global%2Dlocal+sequence+alignment+for+text%2Dvideo+retrieval%2E+In+Proceedings+of+the+IEEE%2FCVF+conference+on+computer+vision+and+pattern+recognition%2C+pa)
47. Cong Wei, Yang Chen, Haonan Chen, Hexiang Hu, Ge Zhang, Jie Fu, Alan Ritter, and Wenhu Chen. 2024. Uniir: Training and benchmarking universal multimodal information retrievers. In European Conference on Computer Vision, pages 387–404. Springer. [find](https://scholar.google.com/scholar?q=Cong+Wei%2C+Yang+Chen%2C+Haonan+Chen%2C+Hexiang+Hu%2C+Ge+Zhang%2C+Jie+Fu%2C+Alan+Ritter%2C+and+Wenhu+Chen%2E+2024%2E+Uniir%3A+Training+and+benchmarking+universal+multimodal+information+retrievers%2E+In+European+Conference+)
48. Bin Xiao, Haiping Wu, Weijian Xu, Xiyang Dai, Houdong Hu, Yumao Lu, Michael Zeng, Ce Liu, and Lu Yuan. 2024. Florence-2: Advancing a unified representation for a variety of vision tasks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 4818–4829. [find](https://scholar.google.com/scholar?q=Bin+Xiao%2C+Haiping+Wu%2C+Weijian+Xu%2C+Xiyang+Dai%2C+Houdong+Hu%2C+Yumao+Lu%2C+Michael+Zeng%2C+Ce+Liu%2C+and+Lu+Yuan%2E+2024%2E+Florence%2D2%3A+Advancing+a+unified+representation+for+a+variety+of+vision+tasks%2E+In+Proceeding)
49. Saining Xie, Chen Sun, Jonathan Huang, Zhuowen Tu, and Kevin Murphy. 2018. Rethinking spatiotemporal feature learning: Speed-accuracy trade-offs in video classification. In Proceedings of the European conference on computer vision (ECCV), pages 305–321. [find](https://scholar.google.com/scholar?q=Saining+Xie%2C+Chen+Sun%2C+Jonathan+Huang%2C+Zhuowen+Tu%2C+and+Kevin+Murphy%2E+2018%2E+Rethinking+spatiotemporal+feature+learning%3A+Speed%2Daccuracy+trade%2Doffs+in+video+classification%2E+In+Proceedings+of+the+European)
50. Nina Shvetsova, Brian Chen, Andrew Rouditchenko, Samuel Thomas, Brian Kingsbury, Rogerio S Feris, David Harwath, James Glass, and Hilde Kuehne. 2022. Everything at once-multi-modal fusion transformer for video retrieval. In Proceedings of the ieee/cvf conference on computer vision and pattern recogn… [find](https://scholar.google.com/scholar?q=Nina+Shvetsova%2C+Brian+Chen%2C+Andrew+Rouditchenko%2C+Samuel+Thomas%2C+Brian+Kingsbury%2C+Rogerio+S+Feris%2C+David+Harwath%2C+James+Glass%2C+and+Hilde+Kuehne%2E+2022%2E+Everything+at+once%2Dmulti%2Dmodal+fusion+transformer+)
51. Hu Xu, Gargi Ghosh, Po-Yao Huang, Dmytro Okhonko, Armen Aghajanyan, Florian Metze, Luke Zettlemoyer, and Christoph Feichtenhofer. 2021. Videoclip: Contrastive pre-training for zero-shot video-text understanding. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processin… [find](https://scholar.google.com/scholar?q=Hu+Xu%2C+Gargi+Ghosh%2C+Po%2DYao+Huang%2C+Dmytro+Okhonko%2C+Armen+Aghajanyan%2C+Florian+Metze%2C+Luke+Zettlemoyer%2C+and+Christoph+Feichtenhofer%2E+2021%2E+Videoclip%3A+Contrastive+pre%2Dtraining+for+zero%2Dshot+video%2Dtext+und)
52. Quan Sun, Jinsheng Wang, Qiying Yu, Yufeng Cui, Fan Zhang, Xiaosong Zhang, and Xinlong Wang. 2024. Eva-clip-18b: Scaling clip to 18 billion parameters. arXiv preprint arXiv:2402.04252. — [arXiv:2402.04252](https://arxiv.org/abs/2402.04252) [find](https://scholar.google.com/scholar?q=Quan+Sun%2C+Jinsheng+Wang%2C+Qiying+Yu%2C+Yufeng+Cui%2C+Fan+Zhang%2C+Xiaosong+Zhang%2C+and+Xinlong+Wang%2E+2024%2E+Eva%2Dclip%2D18b%3A+Scaling+clip+to+18+billion+parameters%2E+arXiv+preprint+arXiv%3A2402%2E04252%2E+%E2%80%94+%5BarXiv%3A2402%2E0)
53. Jin Xu, Zhifang Guo, Jinzheng He, Hangrui Hu, Ting He, Shuai Bai, Keqin Chen, Jialin Wang, Yang Fan, Kai Dang, and 1 others. 2025a. Qwen2. 5-omni technical report. arXiv preprint arXiv:2503.20215. — [Qwen2.5-Omni Technical Report](https://md.research.elamur.ai/free/en/papers/xu-2025-qwen2-5-omni-technical/)
54. Qwen Team. 2025a. Qwen2.5-vl. [find](https://scholar.google.com/scholar?q=Qwen+Team%2E+2025a%2E+Qwen2%2E5%2Dvl%2E)
55. Qwen Team. 2025b. [Qwen3 technical report](https://md.research.elamur.ai/free/en/papers/yang-2025-qwen3-technical-report/). Preprint, arXiv:2505.09388.
56. Jin Xu, Zhifang Guo, Hangrui Hu, Yunfei Chu, Xiong Wang, Jinzheng He, Yuxuan Wang, Xian Shi, Ting He, and Xinfa Zhu. 2025b. [Qwen3-omni technical report](https://md.research.elamur.ai/free/en/papers/xu-2025-qwen3-omni-technical-report/). arXiv preprint arXiv:2509.17765.
57. Sagar Vaze, Nicolas Carion, and Ishan Misra. 2023. Genecis: A benchmark for general conditional image Jun Xu, Tao Mei, Ting Yao, and Yong Rui. 2016. Msr-vtt: A large video description dataset for bridging video and language. In Proceedings of the IEEE conference on computer vision and pattern recogn… [find](https://scholar.google.com/scholar?q=Sagar+Vaze%2C+Nicolas+Carion%2C+and+Ishan+Misra%2E+2023%2E+Genecis%3A+A+benchmark+for+general+conditional+image+Jun+Xu%2C+Tao+Mei%2C+Ting+Yao%2C+and+Yong+Rui%2E+2016%2E+Msr%2Dvtt%3A+A+large+video+description+dataset+for+brid)
58. Yang Yuan. 2023. On the power of foundation models. In International conference on machine learning, pages 40519–40530. PMLR. [find](https://scholar.google.com/scholar?q=Yang+Yuan%2E+2023%2E+On+the+power+of+foundation+models%2E+In+International+conference+on+machine+learning%2C+pages+40519%E2%80%9340530%2E+PMLR%2E)
59. Xiaohua Zhai, Basil Mustafa, Alexander Kolesnikov, and Lucas Beyer. 2023. [Sigmoid loss for language image pre-training](https://md.research.elamur.ai/free/en/papers/zhai-2023-sigmoid-loss-language-image-pre/). In Proceedings of the IEEE/CVF international conference on computer vision, pages 11975–11986.
60. Beichen Zhang, Pan Zhang, Xiaoyi Dong, Yuhang Zang, and Jiaqi Wang. 2024a. Long-clip: Unlocking the long-text capability of clip. In European conference on computer vision, pages 310–325. Springer. [find](https://scholar.google.com/scholar?q=Beichen+Zhang%2C+Pan+Zhang%2C+Xiaoyi+Dong%2C+Yuhang+Zang%2C+and+Jiaqi+Wang%2E+2024a%2E+Long%2Dclip%3A+Unlocking+the+long%2Dtext+capability+of+clip%2E+In+European+conference+on+computer+vision%2C+pages+310%E2%80%93325%2E+Springer%2E)
61. Biao Zhang, Lixin Chen, Tong Liu, and Bo Zheng. 2025a. Smec: Rethinking matryoshka representation learning for retrieval embedding compression. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pages 26220–26233. [find](https://scholar.google.com/scholar?q=Biao+Zhang%2C+Lixin+Chen%2C+Tong+Liu%2C+and+Bo+Zheng%2E+2025a%2E+Smec%3A+Rethinking+matryoshka+representation+learning+for+retrieval+embedding+compression%2E+In+Proceedings+of+the+2025+Conference+on+Empirical+Metho)
62. Kai Zhang, Yi Luan, Hexiang Hu, Kenton Lee, Siyuan Qiao, Wenhu Chen, Yu Su, and Ming-Wei Chang. 2024b. Magiclens: Self-supervised image retrieval with open-ended instructions. In International Conference on Machine Learning, pages 59403–59420. PMLR. [find](https://scholar.google.com/scholar?q=Kai+Zhang%2C+Yi+Luan%2C+Hexiang+Hu%2C+Kenton+Lee%2C+Siyuan+Qiao%2C+Wenhu+Chen%2C+Yu+Su%2C+and+Ming%2DWei+Chang%2E+2024b%2E+Magiclens%3A+Self%2Dsupervised+image+retrieval+with+open%2Dended+instructions%2E+In+International+Confere)
63. Yanzhao Zhang, Mingxin Li, Dingkun Long, Xin Zhang, Huan Lin, Baosong Yang, Pengjun Xie, An Yang, Dayiheng Liu, Junyang Lin, and 1 others. 2025b. [Qwen3 embedding: Advancing text embedding and reranking through foundation models](https://md.research.elamur.ai/free/en/papers/zhang-2025-qwen3-embedding-advancing-text/). arXiv preprint arXiv:2506.05176.
64. Yidan Zhang, Ting Zhang, Dong Chen, Yujing Wang, Qi Chen, Xing Xie, Hao Sun, Weiwei Deng, Qi Zhang, Fan Yang, and 1 others. 2024c. Irgen: Generative modeling for image retrieval. In European Conference on Computer Vision, pages 21–41. Springer. [find](https://scholar.google.com/scholar?q=Yidan+Zhang%2C+Ting+Zhang%2C+Dong+Chen%2C+Yujing+Wang%2C+Qi+Chen%2C+Xing+Xie%2C+Hao+Sun%2C+Weiwei+Deng%2C+Qi+Zhang%2C+Fan+Yang%2C+and+1+others%2E+2024c%2E+Irgen%3A+Generative+modeling+for+image+retrieval%2E+In+European+Conferenc)

*Updated 31 October 2026*