--- title: "Unifying Multimodal Retrieval via Document Screenshot" description: "The authors propose Document Screenshot Embedding (DSE), which skips extraction and treats a… 47 references, cited by 34 papers here." authors: ["Xueguang Ma", "Sheng-Chieh Lin", "Minghan Li", "Wenhu Chen", "Jimmy Lin"] year: 2024 venue: "Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing" source: "https://aclanthology.org/2024.emnlp-main.373/" doi: "10.18653/v1/2024.emnlp-main.373" licence: "CC BY 4.0 (ACL Anthology)" licence_url: "https://creativecommons.org/licenses/by/4.0/" summary_licence: "Elamur Cosmic License" canonical: "https://research.elamur.ai/free/en/papers/ma-2024-unifying-multimodal-retrieval-document-screenshot/" --- [Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/ma-2024-unifying-multimodal-retrieval-document-screenshot/) [home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2024](https://md.research.elamur.ai/free/en/papers/2024/page/5/) / Unifying Multimodal Retrieval via Document Screenshot Embedding # Unifying Multimodal Retrieval via Document Screenshot Embedding **Authors:** [Xueguang Ma](https://md.research.elamur.ai/free/en/authors/xueguang-ma/), [Sheng-Chieh Lin](https://md.research.elamur.ai/free/en/authors/sheng-chieh-lin/), [Minghan Li](https://md.research.elamur.ai/free/en/authors/minghan-li/), [Wenhu Chen](https://md.research.elamur.ai/free/en/authors/wenhu-chen/) and [Jimmy Lin](https://md.research.elamur.ai/free/en/authors/jimmy-lin/) · **Year:** 2024 · **Venue:** [Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing](https://md.research.elamur.ai/free/en/venues/emnlp/) ([EMNLP 2024](https://md.research.elamur.ai/free/en/venues/emnlp/2024/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/) **Links:** [Publisher's page](https://aclanthology.org/2024.emnlp-main.373/) · [PDF](https://aclanthology.org/2024.emnlp-main.373.pdf) · [DOI 10.18653/v1/2024.emnlp-main.373](https://doi.org/10.18653/v1/2024.emnlp-main.373) · [OpenAlex](https://openalex.org/W4404782721) **Citations:** 27 citations (OpenAlex, 2026-10-14) · [34](https://md.research.elamur.ai/free/en/papers/ma-2024-unifying-multimodal-retrieval-document-screenshot/cited-by/) in our library, of the 149,439 papers whose bibliographies we read ## Summary Conventional retrieval pipelines depend on format-specific parsing such as HTML extractors and OCR. These steps are laborious and error-prone, and they discard layout and visual cues. The authors propose Document Screenshot Embedding (DSE), which skips extraction and treats a screenshot of each document as its only input. DSE is a bi-encoder built on the large vision-language model Phi-3-vision. Screenshots are split into multiple sub-images plus a global view, so fine-grained text remains readable. Text queries are encoded by the same underlying language model. The authors found that a plain CLIP vision encoder could not capture dense text well enough. For evaluation, they built Wiki-SS, a corpus of 1.3M Wikipedia page screenshots, paired with Natural Questions queries. They also recast SlideVQA as open-domain retrieval over roughly 50K slides. DSE clearly beats BM25 on Wiki-SS, rivals parsing-based dense text retrievers, and surpasses OCR-based methods on slides. The work points toward extraction-free, visually grounded retrieval-augmented generation (V-RAG). ## Key points - On NQ questions over Wiki-SS, DSE improves top-1 retrieval accuracy by 17 points over BM25. - On slide retrieval from SlideVQA, DSE beats OCR-based text retrievers, including BM25 and dense text models, by more than 15 points in nDCG@10. - Each screenshot is cropped into C_x × C_y sub-images of 336×336 pixels plus one global image, each producing 576 patches. Groups of four patch representations are then merged before entering the language model. - Error analysis covered 50 slide cases where the Phi-3 text retriever failed but DSE succeeded. Of these, 22 traced back to OCR errors and 28 needed visual context. A separate check found that 7 of 50 "irrelevant" Wiki-SS hits actually contained the answer in tables or captions. - Checkpoints, code, and the Wiki-SS collection are released via tevatron.ai. *Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.* ## Institutions - [University of Waterloo](https://md.research.elamur.ai/free/en/universities/university-of-waterloo/) — University, Waterloo, Canada · 67 works · 13,592 citations 📚 Shelves (6): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works · [👁️ Computer Vision and Pattern Recognition](https://md.research.elamur.ai/free/en/shelves/computer-science/computer-vision-and-pattern-recognition/) · 2,869 works · [🗄️ Information Systems](https://md.research.elamur.ai/free/en/shelves/computer-science/information-systems/) · 1,437 works 🧭 Topics (2): [🖼️ Vision & language](https://md.research.elamur.ai/free/en/topics/multimodal/) · 657 works · [🔍 Information retrieval & search](https://md.research.elamur.ai/free/en/topics/information-retrieval/) · 473 works ## Cite Cite as: Xueguang Ma, Sheng-Chieh Lin, Minghan Li, Wenhu Chen and Jimmy Lin. 2024. Unifying Multimodal Retrieval via Document Screenshot Embedding. Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing. The paper itself, not this page. BibTeX from the library's record of the work. ```bibtex @inproceedings{ma2024unifying, title = {{Unifying Multimodal Retrieval via Document Screenshot Embedding}}, author = {Xueguang Ma, Sheng-Chieh Lin, Minghan Li, Wenhu Chen and Jimmy Lin}, booktitle = {Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing}, year = {2024}, doi = {10.18653/v1/2024.emnlp-main.373}, url = {https://doi.org/10.18653/v1/2024.emnlp-main.373} } ``` - [← previous One-Shot Learning as Instruction Data Prospector for Large Language Models](https://md.research.elamur.ai/free/en/papers/li-2024-one-shot-learning-as-instruction/) - [next → A Family of Pretrained Transformer Language Models for Russian](https://md.research.elamur.ai/free/en/papers/zmitrovich-2024-family-pretrained-transformer-language-models/) - [↑ back to papers of 2024](https://md.research.elamur.ai/free/en/papers/2024/page/5/) ## References 47 references, 33 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet. 1. Marah Abdin, Sam Ade Jacobs, Ammar Ahmad Awan, Jyoti Aneja, Ahmed Awadallah, Hany Awadalla, Nguyen Bach, Amit Bahree, Arash Bakhtiari, Jianmin Bao, Harkirat Behl, Alon Benhaim, Misha Bilenko, Johan Bjorck, Sébastien Bubeck, Qin Cai, Martin Cai, Caio César Teodoro Mendes, Weizhu Chen, Vishrav Chaudha… — [Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone](https://md.research.elamur.ai/free/en/papers/abdin-2024-phi-3-technical-report-highly/) 2. Akari Asai, Timo Schick, Patrick Lewis, Xilun Chen, Gautier Izacard, Sebastian Riedel, Hannaneh Hajishirzi, and Wen-tau Yih. 2023. [Task-aware retrieval with instructions](https://md.research.elamur.ai/free/en/papers/asai-2023-task-aware-retrieval-instructions/). In Findings of the Association for Computational Linguistics: ACL 2023, pages 3650– 3675, Toronto, Canada. Association for Comput… 3. Payal Bajaj, Daniel Campos, Nick Craswell, Li Deng, Jianfeng Gao, Xiaodong Liu, Rangan Majumder, Andrew McNamara, Bhaskar Mitra, Tri Nguyen, Mir Rosenberg, Xia Song, Alina Stoica, Saurabh Tiwary, and Tong Wang. 2018. [MS MARCO: A human generated machine reading comprehension dataset](https://md.research.elamur.ai/free/en/papers/bajaj-2016-ms-marco-human-generated/). arXiv:1611.09268… 4. Xingyu Chen, Zihan Zhao, Lu Chen, JiaBao Ji, Danyang Zhang, Ao Luo, Yuxuan Xiong, and Kai Yu. 2021. WebSRC: A dataset for web-based structural reading comprehension. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 4173–4185, Online and Punta Cana, Dom… [find](https://scholar.google.com/scholar?q=Xingyu+Chen%2C+Zihan+Zhao%2C+Lu+Chen%2C+JiaBao+Ji%2C+Danyang+Zhang%2C+Ao+Luo%2C+Yuxuan+Xiong%2C+and+Kai+Yu%2E+2021%2E+WebSRC%3A+A+dataset+for+web%2Dbased+structural+reading+comprehension%2E+In+Proceedings+of+the+2021+Confere) 5. Tri Dao. 2024. [FlashAttention-2: Faster attention with better parallelism and work partitioning](https://md.research.elamur.ai/free/en/papers/dao-2023-flashattention-2-faster-attention-better/). In The Twelfth International Conference on Learning Representations. 6. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. [BERT: Pre-training of deep bidirectional transformers for language understanding](https://md.research.elamur.ai/free/en/papers/devlin-2019-bert-pre-training-deep-bidirectional/). In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies,… 7. Matthijs Douze, Alexandr Guzhva, Chengqi Deng, Jeff Johnson, Gergely Szilvasy, Pierre-Emmanuel Mazaré, Maria Lomeli, Lucas Hosseini, and Hervé Jégou. 2024. [The Faiss library](https://md.research.elamur.ai/free/en/papers/douze-2024-faiss-library/). arXiv:2401.08281. 8. Luyu Gao and Jamie Callan. 2022. [Unsupervised corpus aware language model pre-training for dense passage retrieval](https://md.research.elamur.ai/free/en/papers/gao-2022-unsupervised-corpus-aware-language-model/). In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 2843–2853, Dublin, Ireland. Association for Computational Ling… 9. Luyu Gao, Xueguang Ma, Jimmy Lin, and Jamie Callan. 2023. Tevatron: An efficient and flexible toolkit for neural retrieval. In Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR ’23, page 3120–3124, New York, NY, USA. Association fo… [find](https://scholar.google.com/scholar?q=Luyu+Gao%2C+Xueguang+Ma%2C+Jimmy+Lin%2C+and+Jamie+Callan%2E+2023%2E+Tevatron%3A+An+efficient+and+flexible+toolkit+for+neural+retrieval%2E+In+Proceedings+of+the+46th+International+ACM+SIGIR+Conference+on+Research+an) 10. Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2022. [LoRA: Low-rank adaptation of large language models](https://md.research.elamur.ai/free/en/papers/hu-2021-lora-low-rank-adaptation-large/). In International Conference on Learning Representations. 11. Yupan Huang, Tengchao Lv, Lei Cui, Yutong Lu, and Furu Wei. 2022. [LayoutLMv3: Pre-training for document AI with unified text and image masking](https://md.research.elamur.ai/free/en/papers/huang-2022-layoutlmv3-pre-training-document-ai/). In Proceedings of the 30th ACM International Conference on Multimedia, MM ’22, page 4083–4091, New York, NY, USA. Association for Computing Machinery. 12. Gautier Izacard, Mathilde Caron, Lucas Hosseini, Sebastian Riedel, Piotr Bojanowski, Armand Joulin, and Edouard Grave. 2021. [Unsupervised dense information retrieval with contrastive learning](https://md.research.elamur.ai/free/en/papers/izacard-2021-unsupervised-dense-information-retrieval-contrastive/). arXiv:2112.09118. 13. Gautier Izacard, Patrick Lewis, Maria Lomeli, Lucas Hosseini, Fabio Petroni, Timo Schick, Jane Dwivedi-Yu, Armand Joulin, Sebastian Riedel, and Edouard Grave. 2024. [Atlas: few-shot learning with retrieval augmented language models](https://md.research.elamur.ai/free/en/papers/izacard-2022-atlas-few-shot-learning-retrieval/). J. Mach. Learn. Res., 24(1). 14. Mandar Joshi, Eunsol Choi, Daniel Weld, and Luke Zettlemoyer. 2017. [TriviaQA: A large scale distantly supervised challenge dataset for reading comprehension](https://md.research.elamur.ai/free/en/papers/joshi-2017-triviaqa-large-scale-distantly-supervised/). In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1601–1611, Vancouver… 15. Vladimir Karpukhin, Barlas Oguz, Sewon Min, Patrick Lewis, Ledell Wu, Sergey Edunov, Danqi Chen, and Wen-tau Yih. 2020. [Dense passage retrieval for open-domain question answering](https://md.research.elamur.ai/free/en/papers/karpukhin-2020-dense-passage-retrieval-open-domain/). In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 6769–6781, Onli… 16. Andreas Koukounas, Georgios Mastrapas, Michael Günther, Bo Wang, Scott Martens, Isabelle Mohr, Saba Sturua, Mohammad Kalim Akram, Joan Fontanals Martínez, Saahil Ognawala, Susana Guzman, Maximilian Werk, Nan Wang, and Han Xiao. 2024. Jina CLIP: Your CLIP model is also your text retriever. arXiv:2405… — [arXiv:2405.20204](https://arxiv.org/abs/2405.20204) [find](https://scholar.google.com/scholar?q=Andreas+Koukounas%2C+Georgios+Mastrapas%2C+Michael+G%C3%BCnther%2C+Bo+Wang%2C+Scott+Martens%2C+Isabelle+Mohr%2C+Saba+Sturua%2C+Mohammad+Kalim+Akram%2C+Joan+Fontanals+Mart%C3%ADnez%2C+Saahil+Ognawala%2C+Susana+Guzman%2C+Maximilian+We) 17. Tom Kwiatkowski, Jennimaria Palomaki, Olivia Redfield, Michael Collins, Ankur Parikh, Chris Alberti, Danielle Epstein, Illia Polosukhin, Jacob Devlin, Kenton Lee, Kristina Toutanova, Llion Jones, Matthew Kelcey, Ming-Wei Chang, Andrew M. Dai, Jakob Uszkoreit, Quoc Le, and Slav Petrov. 2019. [Natural…](https://md.research.elamur.ai/free/en/papers/kwiatkowski-2019-natural-questions-benchmark-question-answering/) 18. Hugo Laurençon, Léo Tronchon, Matthieu Cord, and Victor Sanh. 2024. [What matters when building vision-language models?](https://md.research.elamur.ai/free/en/papers/lauren-ccon-2024-matters-building-vision-language-models/) arXiv:2405.02246. 19. Chankyu Lee, Rajarshi Roy, Mengyao Xu, Jonathan Raiman, Mohammad Shoeybi, Bryan Catanzaro, and Wei Ping. 2024. [NV-Embed: Improved techniques for training llms as generalist embedding models](https://md.research.elamur.ai/free/en/papers/lee-2024-nv-embed-improved-techniques-training/). arXiv:2405.17428. 20. Sheng-Chieh Lin, Akari Asai, Minghan Li, Barlas Oguz, Jimmy Lin, Yashar Mehdad, Wen-tau Yih, and Xilun Chen. 2023. [How to train your dragon: Diverse augmentation towards generalizable dense retrieval](https://md.research.elamur.ai/free/en/papers/lin-2023-how-train-your-dragon-diverse/). In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 6385–6400, Singapor… 21. Sheng-Chieh Lin, Jheng-Hong Yang, and Jimmy Lin. 2021. In-batch negatives for knowledge distillation with tightly-coupled teachers for dense retrieval. In Proceedings of the 6th Workshop on Representation Learning for NLP (RepL4NLP-2021), pages 163–173, Online. Association for Computational Linguist… [find](https://scholar.google.com/scholar?q=Sheng%2DChieh+Lin%2C+Jheng%2DHong+Yang%2C+and+Jimmy+Lin%2E+2021%2E+In%2Dbatch+negatives+for+knowledge+distillation+with+tightly%2Dcoupled+teachers+for+dense+retrieval%2E+In+Proceedings+of+the+6th+Workshop+on+Representa) 22. Haotian Liu, Chunyuan Li, Yuheng Li, Bo Li, Yuanhan Zhang, Sheng Shen, and Yong Jae Lee. 2024a. LLaVA-NeXT: Improved reasoning, OCR, and world knowledge. [https://llava-vl.github.io/blog/2024-01-30-llava-next/](https://llava-vl.github.io/blog/2024-01-30-llava-next/). [↗](https://llava-vl.github.io/blog/2024-01-30-llava-next/) 23. Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. 2023. [Visual instruction tuning](https://md.research.elamur.ai/free/en/papers/liu-2023-visual-instruction-tuning/). In Thirty-seventh Conference on Neural Information Processing Systems. 24. Yuliang Liu, Zhang Li, Biao Yang, Chunyuan Li, Xucheng Yin, Cheng lin Liu, Lianwen Jin, and Xiang Bai. 2024b. [On the hidden mystery of OCR in large multimodal models](https://md.research.elamur.ai/free/en/papers/liu-2023-ocrbench-hidden-mystery-ocr-large/). arXiv:2305.07895. 25. Xueguang Ma, Kai Sun, Ronak Pradeep, Minghan Li, and Jimmy Lin. 2022. Another look at DPR: Reproduction of training and replication of retrieval. In Advances in Information Retrieval, pages 613–626, Cham. Springer International Publishing. [find](https://scholar.google.com/scholar?q=Xueguang+Ma%2C+Kai+Sun%2C+Ronak+Pradeep%2C+Minghan+Li%2C+and+Jimmy+Lin%2E+2022%2E+Another+look+at+DPR%3A+Reproduction+of+training+and+replication+of+retrieval%2E+In+Advances+in+Information+Retrieval%2C+pages+613%E2%80%93626%2C+C) 26. Xueguang Ma, Liang Wang, Nan Yang, Furu Wei, and Jimmy Lin. 2024. [Fine-tuning LLaMA for multi-stage text retrieval](https://md.research.elamur.ai/free/en/papers/ma-2023-fine-tuning-llama-multi-stage/). In Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR ’24, page 2421–2425, New York, NY, USA. Association for Comput… 27. Minesh Mathew, Viraj Bagal, Rubèn Tito, Dimosthenis Karatzas, Ernest Valveny, and C.V. Jawahar. 2022. InfographicVQA. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), pages 1697–1706. [find](https://scholar.google.com/scholar?q=Minesh+Mathew%2C+Viraj+Bagal%2C+Rub%C3%A8n+Tito%2C+Dimosthenis+Karatzas%2C+Ernest+Valveny%2C+and+C%2EV%2E+Jawahar%2E+2022%2E+InfographicVQA%2E+In+Proceedings+of+the+IEEE%2FCVF+Winter+Conference+on+Applications+of+Computer+Visio) 28. Minesh Mathew, Dimosthenis Karatzas, and C. V. Jawahar. 2021. [DocVQA: A dataset for vqa on document images](https://md.research.elamur.ai/free/en/papers/mathew-2020-docvqa-dataset-vqa-document-images/). In 2021 IEEE Winter Conference on Applications of Computer Vision (WACV), pages 2199–2208. 29. Niklas Muennighoff, Hongjin Su, Liang Wang, Nan Yang, Furu Wei, Tao Yu, Amanpreet Singh, and Douwe Kiela. 2024. [Generative representational instruction tuning](https://md.research.elamur.ai/free/en/papers/muennighoff-2024-generative-representational-instruction-tuning/). arXiv:2402.09906. 30. Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. 2021. [Learning transferable visual models from natural language supervision](https://md.research.elamur.ai/free/en/papers/radford-2021-learning-transferable-visual-models-natural/). arXiv:2103.00020. 31. Jeff Rasley, Samyam Rajbhandari, Olatunji Ruwase, and Yuxiong He. 2020. DeepSpeed: System optimizations enable training deep learning models with over 100 billion parameters. In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, KDD ’20, page 3505–3506,… [find](https://scholar.google.com/scholar?q=Jeff+Rasley%2C+Samyam+Rajbhandari%2C+Olatunji+Ruwase%2C+and+Yuxiong+He%2E+2020%2E+DeepSpeed%3A+System+optimizations+enable+training+deep+learning+models+with+over+100+billion+parameters%2E+In+Proceedings+of+the+26t) 32. Ruiyang Ren, Yingqi Qu, Jing Liu, Wayne Xin Zhao, QiaoQiao She, Hua Wu, Haifeng Wang, and Ji-Rong Wen. 2021. [RocketQAv2: A joint training method for dense passage retrieval and passage re-ranking](https://md.research.elamur.ai/free/en/papers/ren-2021-rocketqav2-joint-training-method-dense/). In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 2825–2… 33. Stephen E. Robertson and Hugo Zaragoza. 2009. The probabilistic relevance framework: BM25 and beyond. Found. Trends Inf. Retr., 3:333–389. [find](https://scholar.google.com/scholar?q=Stephen+E%2E+Robertson+and+Hugo+Zaragoza%2E+2009%2E+The+probabilistic+relevance+framework%3A+BM25+and+beyond%2E+Found%2E+Trends+Inf%2E+Retr%2E%2C+3%3A333%E2%80%93389%2E) 34. Hongjin Su, Weijia Shi, Jungo Kasai, Yizhong Wang, Yushi Hu, Mari Ostendorf, Wen-tau Yih, Noah A. Smith, Luke Zettlemoyer, and Tao Yu. 2023. [One embedder, any task: Instruction-finetuned text embeddings](https://md.research.elamur.ai/free/en/papers/su-2022-one-embedder-any-task-instruction/). In Findings of the Association for Computational Linguistics: ACL 2023, pages 1102–1121, Toronto… 35. Ryota Tanaka, Kyosuke Nishida, Kosuke Nishida, Taku Hasegawa, Itsumi Saito, and Kuniko Saito. 2023. SlideVQA: A dataset for document visual question answering on multiple images. Proceedings of the AAAI Conference on Artificial Intelligence, 37(11):13636–13645. [find](https://scholar.google.com/scholar?q=Ryota+Tanaka%2C+Kyosuke+Nishida%2C+Kosuke+Nishida%2C+Taku+Hasegawa%2C+Itsumi+Saito%2C+and+Kuniko+Saito%2E+2023%2E+SlideVQA%3A+A+dataset+for+document+visual+question+answering+on+multiple+images%2E+Proceedings+of+the+AA) 36. Ryota Tanaka, Kyosuke Nishida, and Sen Yoshida. 2021. VisualMRC: Machine reading comprehension on document images. Proceedings of the AAAI Conference on Artificial Intelligence, 35(15):13878–13888. [find](https://scholar.google.com/scholar?q=Ryota+Tanaka%2C+Kyosuke+Nishida%2C+and+Sen+Yoshida%2E+2021%2E+VisualMRC%3A+Machine+reading+comprehension+on+document+images%2E+Proceedings+of+the+AAAI+Conference+on+Artificial+Intelligence%2C+35%2815%29%3A13878%E2%80%9313888%2E) 37. Nandan Thakur, Nils Reimers, Andreas Rücklé, Abhishek Srivastava, and Iryna Gurevych. 2021. BEIR: A heterogeneous benchmark for zero-shot evaluation of information retrieval models. In Thirty-fifth Conference on Neural Information Processing Systems Datasets and Benchmarks Track (Round 2). [find](https://scholar.google.com/scholar?q=Nandan+Thakur%2C+Nils+Reimers%2C+Andreas+R%C3%BCckl%C3%A9%2C+Abhishek+Srivastava%2C+and+Iryna+Gurevych%2E+2021%2E+BEIR%3A+A+heterogeneous+benchmark+for+zero%2Dshot+evaluation+of+information+retrieval+models%2E+In+Thirty%2Dfifth+Co) 38. Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, Aurelien Rodriguez, Armand Joulin, Edouard Grave, and Guillaume Lample. 2023. [LLaMA: Open and efficient foundation language models](https://md.research.elamur.ai/free/en/papers/touvron-2023-llama-open-efficient-foundation-language/). arXiv:23… 39. Liang Wang, Nan Yang, Xiaolong Huang, Binxing Jiao, Linjun Yang, Daxin Jiang, Rangan Majumder, and Furu Wei. 2022. [Text embeddings by weakly-supervised contrastive pre-training](https://md.research.elamur.ai/free/en/papers/wang-2022-text-embeddings-weakly-supervised-contrastive/). arXiv:2212.03533. 40. Liang Wang, Nan Yang, Xiaolong Huang, Binxing Jiao, Linjun Yang, Daxin Jiang, Rangan Majumder, and Furu Wei. 2023. [SimLM: Pre-training with representation bottleneck for dense passage retrieval](https://md.research.elamur.ai/free/en/papers/wang-2023-simlm-pre-training-representation-bottleneck/). In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Lon… 41. Liang Wang, Nan Yang, Xiaolong Huang, Linjun Yang, Rangan Majumder, and Furu Wei. 2024. [Improving text embeddings with large language models](https://md.research.elamur.ai/free/en/papers/wang-2024-improving-text-embeddings-large-language/). In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 11897–11916, Bangkok, Thailand. Asso… 42. Cong Wei, Yang Chen, Haonan Chen, Hexiang Hu, Ge Zhang, Jie Fu, Alan Ritter, and Wenhu Chen. 2023. UniIR: Training and benchmarking universal multimodal information retrievers. arXiv:2311.17136. — [arXiv:2311.17136](https://arxiv.org/abs/2311.17136) [find](https://scholar.google.com/scholar?q=Cong+Wei%2C+Yang+Chen%2C+Haonan+Chen%2C+Hexiang+Hu%2C+Ge+Zhang%2C+Jie+Fu%2C+Alan+Ritter%2C+and+Wenhu+Chen%2E+2023%2E+UniIR%3A+Training+and+benchmarking+universal+multimodal+information+retrievers%2E+arXiv%3A2311%2E17136%2E+%E2%80%94+%5Bar) 43. Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed H. Chi, Quoc V. Le, and Denny Zhou. 2022. [Chain of thought prompting elicits reasoning in large language models](https://md.research.elamur.ai/free/en/papers/wei-2022-chain-thought-prompting-elicits-reasoning/). In Advances in Neural Information Processing Systems. 44. Shitao Xiao, Zheng Liu, Peitian Zhang, and Niklas Muennighoff. 2023. [C-Pack: Packaged resources to advance general Chinese embedding](https://md.research.elamur.ai/free/en/papers/xiao-2023-c-pack-packed-resources/). arXiv: 2309.07597. 45. Lee Xiong, Chenyan Xiong, Ye Li, Kwok-Fung Tang, Jialin Liu, Paul N. Bennett, Junaid Ahmed, and Arnold Overwijk. 2021. [Approximate nearest neighbor negative contrastive learning for dense text retrieval](https://md.research.elamur.ai/free/en/papers/xiong-2020-approximate-nearest-neighbor-negative-contrastive/). In International Conference on Learning Representations. 46. Yiheng Xu, Minghao Li, Lei Cui, Shaohan Huang, Furu Wei, and Ming Zhou. 2020. [LayoutLM: Pre-training of text and layout for document image understanding](https://md.research.elamur.ai/free/en/papers/xu-2019-layoutlm-pre-training-text-layout/). In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, KDD ’20, page 1192–1200, New York, NY, USA. A… 47. Jheng-Hong Yang, Carlos Lassance, Rafael Sampaio De Rezende, Krishna Srinivasan, Miriam Redi, Stéphane Clinchant, and Jimmy Lin. 2023. Atomic: An image/text retrieval test collection to support multimedia content creation. In Proceedings of the 46th International ACM SIGIR Conference on Research and… [find](https://scholar.google.com/scholar?q=Jheng%2DHong+Yang%2C+Carlos+Lassance%2C+Rafael+Sampaio+De+Rezende%2C+Krishna+Srinivasan%2C+Miriam+Redi%2C+St%C3%A9phane+Clinchant%2C+and+Jimmy+Lin%2E+2023%2E+Atomic%3A+An+image%2Ftext+retrieval+test+collection+to+support+multim) *Updated 31 October 2026*