---
title: "Benchmarking Vision Language Models for Cultural Understanding"
description: "The authors present CulturalVQA, an open-ended visual question-answering benchmark for cultural… 41 references, cited by 51 papers here."
authors: ["Shravan Nayak", "Kanishk Jain", "Rabiul Awal", "Siva Reddy", "Sjoerd van Steenkiste", "Lisa Anne Hendricks", "Karolina Stanczak", "Aishwarya Agrawal"]
year: 2024
venue: "Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing"
source: "https://aclanthology.org/2024.emnlp-main.329/"
doi: "10.18653/v1/2024.emnlp-main.329"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/nayak-2024-benchmarking-vision-language-models-cultural/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/nayak-2024-benchmarking-vision-language-models-cultural/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2024](https://md.research.elamur.ai/free/en/papers/2024/page/4/) / Benchmarking Vision Language Models for Cultural Understanding

# Benchmarking Vision Language Models for Cultural Understanding

**Authors:** Shravan Nayak, Kanishk Jain, Rabiul Awal, [Siva Reddy](https://md.research.elamur.ai/free/en/authors/siva-reddy/), Sjoerd van Steenkiste, [Lisa Anne Hendricks](https://md.research.elamur.ai/free/en/authors/lisa-anne-hendricks/), [Karolina Stanczak](https://md.research.elamur.ai/free/en/authors/karolina-stanczak/) and [Aishwarya Agrawal](https://md.research.elamur.ai/free/en/authors/aishwarya-agrawal/) · **Year:** 2024 · **Venue:** [Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing](https://md.research.elamur.ai/free/en/venues/emnlp/) ([EMNLP 2024](https://md.research.elamur.ai/free/en/venues/emnlp/2024/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2024.emnlp-main.329/) · [PDF](https://aclanthology.org/2024.emnlp-main.329.pdf) · [DOI 10.18653/v1/2024.emnlp-main.329](https://doi.org/10.18653/v1/2024.emnlp-main.329) · [OpenAlex](https://openalex.org/W4404783219)

**Citations:** 34 citations (OpenAlex, 2026-10-14) · [51](https://md.research.elamur.ai/free/en/papers/nayak-2024-benchmarking-vision-language-models-cultural/cited-by/) in our library, of the 141,840 papers whose bibliographies we read

## Summary
Vision Language Models (VLMs) are usually tested on generic scene understanding, such as spotting objects, attributes and actions. Their grasp of culture is rarely tested. Earlier resources like MaRVL, GD-VCR and CVQA either focus on visual reasoning, draw only on movie scenes, or score models with multiple-choice or True/False formats. The authors present CulturalVQA, an open-ended visual question-answering benchmark for cultural knowledge across 11 countries on five continents.

They start from cultural commonsense assertions in the text-only CANDLE dataset and automatically retrieve images that match them. Annotators from each culture then write questions and reference answers. The questions cover five facets: clothing, food, drink, rituals and traditions.

Tests of open and proprietary models, including GPT-4o and Gemini, show three uneven patterns:
- **By region:** models handle North American content far better than African content.
- **By model type:** open-source systems trail closed ones by a wide margin.
- **By facet:** food and drink questions are the hardest.

The benchmark shows where cultural knowledge in VLMs is missing and offers a way to track progress toward more globally inclusive models.

## Key points
- **Dataset size:** 2,378 questions on 2,328 unique images, each with 1–5 reference answers (7,206 answers in total), from 11 countries.
- **Focus on intangible culture:** traditions and rituals make up 44.1% of the questions, versus about 13% in the concurrent CVQA. Answers are open-ended and scored with the LAVE metric, which was checked against human ratings.
- **Regional gap:** GPT-4o, the top model, reaches 67–72% accuracy on North American concepts but only 43–56% on African ones.
- **Closed vs. open models:** on Ethiopia, the weakest country, the best closed-source model leads the best open-source one by 29.78 points.
- **Limitations:** the data are English-only, images come from English-language websites, and countries stand in for cultural groups. Annotators were paid $10–15 per hour.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Google](https://md.research.elamur.ai/free/en/companies/google/) — Company, Mountain View, United States · 353 works · 364,806 citations
- [Université de Montréal](https://md.research.elamur.ai/free/en/universities/universite-de-montreal/) — University, Montreal, Canada · 47 works · 73,845 citations
- [Google DeepMind](https://md.research.elamur.ai/free/en/companies/google-deepmind/) — Company, London, United Kingdom · 35 works · 75,886 citations
- [McGill University](https://md.research.elamur.ai/free/en/universities/mcgill-university/) — University, Montreal, Canada · 51 works · 10,476 citations
- [Mila - Quebec Artificial Intelligence Institute](https://md.research.elamur.ai/free/en/labs/mila-quebec-artificial-intelligence-institute/) — Research institute, Montreal, Canada · 31 works · 13,061 citations

📚 Shelves (5): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 9,085 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 8,888 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 9,296 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 9,206 works · [👁️ Computer Vision and Pattern Recognition](https://md.research.elamur.ai/free/en/shelves/computer-science/computer-vision-and-pattern-recognition/) · 1,282 works

🧭 Topics (1): [🖼️ Vision & language](https://md.research.elamur.ai/free/en/topics/multimodal/) · 269 works

## Cite

Cite as: Shravan Nayak, Kanishk Jain, Rabiul Awal, Siva Reddy, Sjoerd van Steenkiste, Lisa Anne Hendricks, Karolina Stanczak and Aishwarya Agrawal. 2024. Benchmarking Vision Language Models for Cultural Understanding. Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{nayak2024benchmarking,
  title = {{Benchmarking Vision Language Models for Cultural Understanding}},
  author = {Shravan Nayak, Kanishk Jain, Rabiul Awal, Siva Reddy, Sjoerd {van Steenkiste}, Lisa Anne Hendricks, Karolina Stanczak and Aishwarya Agrawal},
  booktitle = {Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing},
  year = {2024},
  doi = {10.18653/v1/2024.emnlp-main.329},
  url = {https://doi.org/10.18653/v1/2024.emnlp-main.329}
}
```

- [← previous ADaPT: As-Needed Decomposition and Planning with Language Models](https://md.research.elamur.ai/free/en/papers/prasad-2024-adapt-as-needed-decomposition-planning/)
- [next → BUFFET: Benchmarking Large Language Models for Few-shot Cross-lingual Transfer](https://md.research.elamur.ai/free/en/papers/asai-2024-buffet-benchmarking-large-language-models/)
- [↑ back to papers of 2024](https://md.research.elamur.ai/free/en/papers/2024/page/4/)

## References

41 references, 15 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Muhammad Farid Adilazuarda, Sagnik Mukherjee, Pradhyumna Lavania, Siddhant Singh, Ashutosh Dwivedi, Alham Fikri Aji, Jacki O’Neill, Ashutosh Modi, and Monojit Choudhury. 2024. [Towards measuring and modeling “culture” in LLMs: A survey](https://md.research.elamur.ai/free/en/papers/adilazuarda-2024-measuring-modeling-culture-llms-survey/). Preprint, arXiv:2403.15412.
2. Anthropic. 2024. Claude 3.5 sonnet. [https://www.anthropic.com/news/claude-3-5-sonnet](https://www.anthropic.com/news/claude-3-5-sonnet). [↗](https://www.anthropic.com/news/claude-3-5-sonnet)
3. Yujin Baek, ChaeHun Park, Jaeseok Kim, Yu-Jung Heo, Du-Seong Chang, and Jaegul Choo. 2024. Evaluating visual and cultural interpretation: The k-viscuit benchmark with human-vlm collaboration. Preprint, arXiv:2406.16469. [find](https://scholar.google.com/scholar?q=Yujin+Baek%2C+ChaeHun+Park%2C+Jaeseok+Kim%2C+Yu%2DJung+Heo%2C+Du%2DSeong+Chang%2C+and+Jaegul+Choo%2E+2024%2E+Evaluating+visual+and+cultural+interpretation%3A+The+k%2Dviscuit+benchmark+with+human%2Dvlm+collaboration%2E+Preprint)
4. Emily M. Bender and Batya Friedman. 2018. Data statements for natural language processing: Toward mitigating system bias and enabling better science. Transactions of the Association for Computational Linguistics, 6:587–604. [find](https://scholar.google.com/scholar?q=Emily+M%2E+Bender+and+Batya+Friedman%2E+2018%2E+Data+statements+for+natural+language+processing%3A+Toward+mitigating+system+bias+and+enabling+better+science%2E+Transactions+of+the+Association+for+Computational+)
5. Lucas Beyer, Andreas Steiner, André Susano Pinto, Alexander Kolesnikov, Xiao Wang, Daniel Salz, Maxim Neumann, Ibrahim Alabdulmohsin, Michael Tschannen, Emanuele Bugliarello, Thomas Unterthiner, Daniel Keysers, Skanda Koppula, Fangyu Liu, Adam Grycner, Alexey Gritsenko, Neil Houlsby, Manoj Kumar, Ke… [find](https://scholar.google.com/scholar?q=Lucas+Beyer%2C+Andreas+Steiner%2C+Andr%C3%A9+Susano+Pinto%2C+Alexander+Kolesnikov%2C+Xiao+Wang%2C+Daniel+Salz%2C+Maxim+Neumann%2C+Ibrahim+Alabdulmohsin%2C+Michael+Tschannen%2C+Emanuele+Bugliarello%2C+Thomas+Unterthiner%2C+Danie)
6. Mehar Bhatia, Sahithya Ravi, Aditya Chinchure, Eunjeong Hwang, and Vered Shwartz. 2024. From local concepts to universals: Evaluating the multicultural understanding of vision-language models. Preprint, arXiv:2407.00263. [find](https://scholar.google.com/scholar?q=Mehar+Bhatia%2C+Sahithya+Ravi%2C+Aditya+Chinchure%2C+Eunjeong+Hwang%2C+and+Vered+Shwartz%2E+2024%2E+From+local+concepts+to+universals%3A+Evaluating+the+multicultural+understanding+of+vision%2Dlanguage+models%2E+Preprin)
7. Emanuele Bugliarello, Fangyu Liu, Jonas Pfeiffer, Siva Reddy, Desmond Elliott, Edoardo Maria Ponti, and Ivan Vulić. 2022. IGLUE: A benchmark for transfer learning across modalities, tasks, and languages. In Proceedings of the 39th International Conference on Machine Learning, volume 162 of Proceedi… [find](https://scholar.google.com/scholar?q=Emanuele+Bugliarello%2C+Fangyu+Liu%2C+Jonas+Pfeiffer%2C+Siva+Reddy%2C+Desmond+Elliott%2C+Edoardo+Maria+Ponti%2C+and+Ivan+Vulic%CC%81%2E+2022%2E+IGLUE%3A+A+benchmark+for+transfer+learning+across+modalities%2C+tasks%2C+and+langua)
8. Olena Burda-Lassen, Aman Chadha, Shashank Goswami, and Vinija Jain. 2024. How culturally aware are vision-language models? Preprint, arXiv:2405.17475. — [arXiv:2405.17475](https://arxiv.org/abs/2405.17475) [find](https://scholar.google.com/scholar?q=Olena+Burda%2DLassen%2C+Aman+Chadha%2C+Shashank+Goswami%2C+and+Vinija+Jain%2E+2024%2E+How+culturally+aware+are+vision%2Dlanguage+models%3F+Preprint%2C+arXiv%3A2405%2E17475%2E+%E2%80%94+%5BarXiv%3A2405%2E17475%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2405%2E1)
9. Soravit Changpinyo, Linting Xue, Michal Yarom, Ashish Thapliyal, Idan Szpektor, Julien Amelot, Xi Chen, and Radu Soricut. 2023. MaXM: Towards multilingual visual question answering. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 2667– 2682, Singapore. Association for… [find](https://scholar.google.com/scholar?q=Soravit+Changpinyo%2C+Linting+Xue%2C+Michal+Yarom%2C+Ashish+Thapliyal%2C+Idan+Szpektor%2C+Julien+Amelot%2C+Xi+Chen%2C+and+Radu+Soricut%2E+2023%2E+MaXM%3A+Towards+multilingual+visual+question+answering%2E+In+Findings+of+the)
10. Zhe Chen, Weiyun Wang, Hao Tian, Shenglong Ye, Zhangwei Gao, Erfei Cui, Wenwen Tong, Kongzhi Hu, Jiapeng Luo, Zheng Ma, et al. 2024. [How far are we to GPT-4V? Closing the gap to commercial multimodal models with open-source suites](https://md.research.elamur.ai/free/en/papers/chen-2024-far-gpt-4v-closing/). arXiv preprint arXiv:2404.16821.
11. Hyung Won Chung, Le Hou, Shayne Longpre, Barret Zoph, Yi Tay, William Fedus, Yunxuan Li, Xuezhi Wang, Mostafa Dehghani, Siddhartha Brahma, Albert Webson, Shixiang Shane Gu, Zhuyun Dai, Mirac Suzgun, Xinyun Chen, Aakanksha Chowdhery, Alex Castro-Ros, Marie Pellat, Kevin Robinson, Dasha Valter, Sharan… [find](https://scholar.google.com/scholar?q=Hyung+Won+Chung%2C+Le+Hou%2C+Shayne+Longpre%2C+Barret+Zoph%2C+Yi+Tay%2C+William+Fedus%2C+Yunxuan+Li%2C+Xuezhi+Wang%2C+Mostafa+Dehghani%2C+Siddhartha+Brahma%2C+Albert+Webson%2C+Shixiang+Shane+Gu%2C+Zhuyun+Dai%2C+Mirac+Suzgun%2C+X)
12. Wenliang Dai, Junnan Li, Dongxu Li, Anthony Meng Huat Tiong, Junqi Zhao, Weisheng Wang, Boyang Li, Pascale N Fung, and Steven Hoi. 2024. [InstructBLIP: Towards general-purpose vision-language models with instruction tuning](https://md.research.elamur.ai/free/en/papers/dai-2023-instructblip-towards-general-purpose-vision/). Advances in Neural Information Processing Systems, 36.
13. William Gaviria Rojas, Sudnya Diamos, Keertan Kini, David Kanter, Vijay Janapa Reddi, and Cody Coleman. 2022. The dollar street dataset: Images representing the geographic and socioeconomic diversity of the world. In Advances in Neural Information Processing Systems, volume 35, pages 12979–12990. Cu… [find](https://scholar.google.com/scholar?q=William+Gaviria+Rojas%2C+Sudnya+Diamos%2C+Keertan+Kini%2C+David+Kanter%2C+Vijay+Janapa+Reddi%2C+and+Cody+Coleman%2E+2022%2E+The+dollar+street+dataset%3A+Images+representing+the+geographic+and+socioeconomic+diversity+)
14. Gregor Geigle, Abhay Jain, Radu Timofte, and Goran Glavaš. 2023. mblip: Efficient bootstrapping of multilingual vision-llms. arXiv, abs/2307.06930. [find](https://scholar.google.com/scholar?q=Gregor+Geigle%2C+Abhay+Jain%2C+Radu+Timofte%2C+and+Goran+Glava%C5%A1%2E+2023%2E+mblip%3A+Efficient+bootstrapping+of+multilingual+vision%2Dllms%2E+arXiv%2C+abs%2F2307%2E06930%2E)
15. Gemini Team et al. 2023. [Gemini: A Family of Highly Capable Multimodal Models](https://md.research.elamur.ai/free/en/papers/gemini-team-2023-gemini-family-highly-capable-multimodal/). arXiv e-prints, arXiv:2312.11805.
16. Google. 2017. Google lens api. [https://serpapi.com/google-lens-api](https://serpapi.com/google-lens-api). Accessed: 2017-10-4. [↗](https://serpapi.com/google-lens-api)
17. Christian Haerpfer, Ronald Inglehart, Alejandro Moreno, Christian Welzel, Kseniya Kizilova, Marta Lagos, Juan Diez-Medrano, Pippa Norris, Eduard Ponarin, and Bi Puranen. 2022. World Values Survey: Round seven - country-pooled datafile version 3.0. Madrid, Spain & Vienna, Austria: JD Systems Institut… [find](https://scholar.google.com/scholar?q=Christian+Haerpfer%2C+Ronald+Inglehart%2C+Alejandro+Moreno%2C+Christian+Welzel%2C+Kseniya+Kizilova%2C+Marta+Lagos%2C+Juan+Diez%2DMedrano%2C+Pippa+Norris%2C+Eduard+Ponarin%2C+and+Bi+Puranen%2E+2022%2E+World+Values+Survey%3A+Rou)
18. Daniel Hershcovich, Stella Frank, Heather Lent, Miryam de Lhoneux, Mostafa Abdou, Stephanie Brandl, Emanuele Bugliarello, Laura Cabello Piqueras, Ilias Chalkidis, Ruixiang Cui, Constanza Fierro, Katerina Margatina, Phillip Rust, and Anders Søgaard. 2022. [Challenges and strategies in cross-cultural N…](https://md.research.elamur.ai/free/en/papers/hershcovich-2022-challenges-strategies-cross-cultural-nlp/)
19. Jack Hessel, Ari Holtzman, Maxwell Forbes, Ronan Le Bras, and Yejin Choi. 2021. [CLIPScore: A reference-free evaluation metric for image captioning](https://md.research.elamur.ai/free/en/papers/hessel-2021-clipscore-reference-free-evaluation-metric/). In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 7514–7528, Online and Punta Cana, Dominican Republic. A…
20. Geert Hofstede, Gert Jan Hofstede, and Michael Minkov. 2010. Cultures and organizations: software of the mind: intercultural cooperation and its importance for survival, 3rd edition. McGraw-Hill, New York; London. [find](https://scholar.google.com/scholar?q=Geert+Hofstede%2C+Gert+Jan+Hofstede%2C+and+Michael+Minkov%2E+2010%2E+Cultures+and+organizations%3A+software+of+the+mind%3A+intercultural+cooperation+and+its+importance+for+survival%2C+3rd+edition%2E+McGraw%2DHill%2C+New+)
21. Hugo Laurençon, Léo Tronchon, Matthieu Cord, and Victor Sanh. 2024. [What matters when building vision-language models?](https://md.research.elamur.ai/free/en/papers/lauren-ccon-2024-matters-building-vision-language-models/) arXiv preprint arXiv:2405.02246.
22. Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi. 2023. [BLIP-2: Bootstrapping language-image pre-training with frozen image encoders and large language models](https://md.research.elamur.ai/free/en/papers/li-2023-blip-2-bootstrapping-language-image/). In International Conference on Machine Learning, pages 19730– 19742.
23. Wenyan Li, Xinyu Zhang, Jiaang Li, Qiwei Peng, Raphael Tang, Li Zhou, Weijia Zhang, Guimin Hu, Yifei Yuan, Anders Søgaard, Daniel Hershcovich, and Desmond Elliott. 2024. Foodieqa: A multimodal dataset for fine-grained understanding of chinese food culture. Preprint, arXiv:2406.11030. [find](https://scholar.google.com/scholar?q=Wenyan+Li%2C+Xinyu+Zhang%2C+Jiaang+Li%2C+Qiwei+Peng%2C+Raphael+Tang%2C+Li+Zhou%2C+Weijia+Zhang%2C+Guimin+Hu%2C+Yifei+Yuan%2C+Anders+S%C3%B8gaard%2C+Daniel+Hershcovich%2C+and+Desmond+Elliott%2E+2024%2E+Foodieqa%3A+A+multimodal+dataset)
24. Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C. Lawrence Zitnick. 2014. [Microsoft COCO: Common objects in context](https://md.research.elamur.ai/free/en/papers/lin-2014-microsoft-coco-common-objects-context/). In Computer Vision – ECCV 2014, pages 740–755, Cham. Springer International Publishing.
25. Fangyu Liu, Emanuele Bugliarello, Edoardo Maria Ponti, Siva Reddy, Nigel Collier, and Desmond Elliott. 2021. Visually grounded reasoning across languages and cultures. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 10467–10485, Online and Punta Cana,… [find](https://scholar.google.com/scholar?q=Fangyu+Liu%2C+Emanuele+Bugliarello%2C+Edoardo+Maria+Ponti%2C+Siva+Reddy%2C+Nigel+Collier%2C+and+Desmond+Elliott%2E+2021%2E+Visually+grounded+reasoning+across+languages+and+cultures%2E+In+Proceedings+of+the+2021+Confe)
26. Haotian Liu, Chunyuan Li, Yuheng Li, Bo Li, Yuanhan Zhang, Sheng Shen, and Yong Jae Lee. 2024. LLaVA-NeXT: Improved reasoning, OCR, and world knowledge. [find](https://scholar.google.com/scholar?q=Haotian+Liu%2C+Chunyuan+Li%2C+Yuheng+Li%2C+Bo+Li%2C+Yuanhan+Zhang%2C+Sheng+Shen%2C+and+Yong+Jae+Lee%2E+2024%2E+LLaVA%2DNeXT%3A+Improved+reasoning%2C+OCR%2C+and+world+knowledge%2E)
27. Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. 2023. [Visual instruction tuning](https://md.research.elamur.ai/free/en/papers/liu-2023-visual-instruction-tuning/). In Advances in Neural Information Processing Systems, volume 36, pages 34892– 34916. Curran Associates, Inc.
28. Yujie Lu, Dongfu Jiang, Wenhu Chen, William Wang, Yejin Choi, and Bill Yuchen Lin. 2024. WildVision Arena: Benchmarking multimodal LLMs in the wild. [find](https://scholar.google.com/scholar?q=Yujie+Lu%2C+Dongfu+Jiang%2C+Wenhu+Chen%2C+William+Wang%2C+Yejin+Choi%2C+and+Bill+Yuchen+Lin%2E+2024%2E+WildVision+Arena%3A+Benchmarking+multimodal+LLMs+in+the+wild%2E)
29. Oscar Mañas, Benno Krojer, and Aishwarya Agrawal. 2024. Improving automatic VQA evaluation using large language models. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 4171–4179. AAAI. [find](https://scholar.google.com/scholar?q=Oscar+Ma%C3%B1as%2C+Benno+Krojer%2C+and+Aishwarya+Agrawal%2E+2024%2E+Improving+automatic+VQA+evaluation+using+large+language+models%2E+In+Proceedings+of+the+AAAI+Conference+on+Artificial+Intelligence%2C+volume+38%2C+pag)
30. Niklas Muennighoff, Thomas Wang, Lintang Sutawika, Adam Roberts, Stella Biderman, Teven Le Scao, M Saiful Bari, Sheng Shen, Zheng-Xin Yong, Hailey Schoelkopf, Xiangru Tang, Dragomir Radev, Alham Fikri Aji, Khalid Almubarak, Samuel Albanie, Zaid Alyafeai, Albert Webson, Edward Raff, and Colin Raffel.… — [Crosslingual Generalization through Multitask Finetuning](https://md.research.elamur.ai/free/en/papers/muennighoff-2022-crosslingual-generalization-through-multitask-finetuning/)
31. Tuan-Phong Nguyen, Simon Razniewski, Aparna Varde, and Gerhard Weikum. 2023. Extracting cultural commonsense knowledge at scale. In Proceedings of the ACM Web Conference, page 1907–1917. [find](https://scholar.google.com/scholar?q=Tuan%2DPhong+Nguyen%2C+Simon+Razniewski%2C+Aparna+Varde%2C+and+Gerhard+Weikum%2E+2023%2E+Extracting+cultural+commonsense+knowledge+at+scale%2E+In+Proceedings+of+the+ACM+Web+Conference%2C+page+1907%E2%80%931917%2E)
32. OpenAI. 2023. Gpt-4v. Retrieved from [https://cdn.openai.com/papers/GPTV_System_Card.pdf](https://cdn.openai.com/papers/GPTV_System_Card.pdf). [↗](https://cdn.openai.com/papers/GPTV_System_Card.pdf)
33. Zhiliang Peng, Wenhui Wang, Li Dong, Yaru Hao, Shaohan Huang, Shuming Ma, and Furu Wei. 2023. [Kosmos-2: Grounding multimodal large language models to the world](https://md.research.elamur.ai/free/en/papers/peng-2023-kosmos-2-grounding-multimodal-large/). arXiv preprint arXiv:2306.14824.
34. Angéline Pouget, Lucas Beyer, Emanuele Bugliarello, Xiao Wang, Andreas Peter Steiner, Xiaohua Zhai, and Ibrahim Alabdulmohsin. 2024. No filter: Cultural and socioeconomic diversity in contrastive vision-language models. arXiv preprint arXiv:22405.13777. [find](https://scholar.google.com/scholar?q=Ang%C3%A9line+Pouget%2C+Lucas+Beyer%2C+Emanuele+Bugliarello%2C+Xiao+Wang%2C+Andreas+Peter+Steiner%2C+Xiaohua+Zhai%2C+and+Ibrahim+Alabdulmohsin%2E+2024%2E+No+filter%3A+Cultural+and+socioeconomic+diversity+in+contrastive+visi)
35. Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. 2021. [Learning transferable visual models from natural language supervision](https://md.research.elamur.ai/free/en/papers/radford-2021-learning-transferable-visual-models-natural/). In Proceedings of the 38th Internatio…
36. Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. 2020. [Exploring the limits of transfer learning with a unified text-to-text transformer](https://md.research.elamur.ai/free/en/papers/raffel-2019-exploring-limits-transfer-learning-unified/). J. Mach. Learn. Res., 21(1).
37. David Romero, Chenyang Lyu, Haryo Akbarianto Wibowo, Teresa Lynn, Injy Hamed, Aditya Nanda Kishore, Aishik Mandal, Alina Dragonetti, Artem Abzaliev, Atnafu Lambebo Tonja, Bontu Fufa Balcha, Chenxi Whitehouse, Christian Salamea, Dan John Velasco, David Ifeoluwa Adelani, David Le Meur, Emilio Villa-Cu… [find](https://scholar.google.com/scholar?q=David+Romero%2C+Chenyang+Lyu%2C+Haryo+Akbarianto+Wibowo%2C+Teresa+Lynn%2C+Injy+Hamed%2C+Aditya+Nanda+Kishore%2C+Aishik+Mandal%2C+Alina+Dragonetti%2C+Artem+Abzaliev%2C+Atnafu+Lambebo+Tonja%2C+Bontu+Fufa+Balcha%2C+Chenxi+Whi)
38. Ashish V. Thapliyal, Jordi Pont Tuset, Xi Chen, and Radu Soricut. 2022. [Crossmodal-3600: A massively multilingual multimodal evaluation dataset](https://md.research.elamur.ai/free/en/papers/thapliyal-2022-crossmodal-3600-massively-multilingual-multimodal/). In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 715–729, Abu Dhabi, United Arab Emirates. Association for…
39. T. Weyand, A. Araujo, B. Cao, and J. Sim. 2020. Google landmarks dataset v2 – a large-scale benchmark for instance-level recognition and retrieval. In 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 2572–2581, Los Alamitos, CA, USA. IEEE Computer Society. [find](https://scholar.google.com/scholar?q=T%2E+Weyand%2C+A%2E+Araujo%2C+B%2E+Cao%2C+and+J%2E+Sim%2E+2020%2E+Google+landmarks+dataset+v2+%E2%80%93+a+large%2Dscale+benchmark+for+instance%2Dlevel+recognition+and+retrieval%2E+In+2020+IEEE%2FCVF+Conference+on+Computer+Vision+and+P)
40. Da Yin, Liunian Harold Li, Ziniu Hu, Nanyun Peng, and Kai-Wei Chang. 2021. Broaden the vision: Geo-diverse visual commonsense reasoning. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 2115–2129, Online and Punta Cana, Dominican Republic. Association… [find](https://scholar.google.com/scholar?q=Da+Yin%2C+Liunian+Harold+Li%2C+Ziniu+Hu%2C+Nanyun+Peng%2C+and+Kai%2DWei+Chang%2E+2021%2E+Broaden+the+vision%3A+Geo%2Ddiverse+visual+commonsense+reasoning%2E+In+Proceedings+of+the+2021+Conference+on+Empirical+Methods+in+N)
41. Wanrong Zhu, Jack Hessel, Anas Awadalla, Samir Yitzhak Gadre, Jesse Dodge, Alex Fang, Youngjae Yu, Ludwig Schmidt, William Yang Wang, and Yejin Choi. 2023. Multimodal C4: An open, billion-scale corpus of images interleaved with text. arXiv preprint arXiv:2304.06939. — [arXiv:2304.06939](https://arxiv.org/abs/2304.06939) [find](https://scholar.google.com/scholar?q=Wanrong+Zhu%2C+Jack+Hessel%2C+Anas+Awadalla%2C+Samir+Yitzhak+Gadre%2C+Jesse+Dodge%2C+Alex+Fang%2C+Youngjae+Yu%2C+Ludwig+Schmidt%2C+William+Yang+Wang%2C+and+Yejin+Choi%2E+2023%2E+Multimodal+C4%3A+An+open%2C+billion%2Dscale+corpus)

*Updated 24 October 2026*