---
title: "Kosmos-2: Grounding Multimodal Large Language Models to the World — Peng 2023"
description: "Multimodal LLMs can read images and reply in free text, but they cannot point at specific objects or accept a… 33 references, cited by 143 papers here."
authors: ["Zhiliang Peng", "Wenhui Wang", "Li Dong", "Yaru Hao", "Shaohan Huang", "Shuming Ma", "Furu Wei"]
year: 2023
venue: "arXiv:2306.14824"
source: "https://arxiv.org/abs/2306.14824"
doi: "10.48550/arxiv.2306.14824"
arxiv: "2306.14824"
licence: "arXiv — non-exclusive license to distribute"
licence_url: "http://arxiv.org/licenses/nonexclusive-distrib/1.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/peng-2023-kosmos-2-grounding-multimodal-large/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/peng-2023-kosmos-2-grounding-multimodal-large/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2023](https://md.research.elamur.ai/free/en/papers/2023/page/3/) / Kosmos-2: Grounding Multimodal Large Language Models to the World

# Kosmos-2: Grounding Multimodal Large Language Models to the World

**Authors:** Zhiliang Peng, [Wenhui Wang](https://md.research.elamur.ai/free/en/authors/wenhui-wang/), [Li Dong](https://md.research.elamur.ai/free/en/authors/li-dong/), [Yaru Hao](https://md.research.elamur.ai/free/en/authors/yaru-hao/), [Shaohan Huang](https://md.research.elamur.ai/free/en/authors/shaohan-huang/), [Shuming Ma](https://md.research.elamur.ai/free/en/authors/shuming-ma/) and [Furu Wei](https://md.research.elamur.ai/free/en/authors/furu-wei/) · **Year:** 2023 · **Venue:** arXiv:2306.14824 · **Licence:** [arXiv — non-exclusive license to distribute](http://arxiv.org/licenses/nonexclusive-distrib/1.0/)

**Links:** [Publisher's page](https://arxiv.org/abs/2306.14824) · [PDF](https://arxiv.org/pdf/2306.14824) · [DOI 10.48550/arxiv.2306.14824](https://doi.org/10.48550/arxiv.2306.14824) · [arXiv 2306.14824](https://arxiv.org/abs/2306.14824) · [OpenAlex](https://openalex.org/W4382490555)

**Citations:** 129 citations (OpenAlex, 2026-10-14) · [143](https://md.research.elamur.ai/free/en/papers/peng-2023-kosmos-2-grounding-multimodal-large/cited-by/) in our library, of the 140,884 papers whose bibliographies we read

## Summary
Multimodal LLMs can read images and reply in free text, but they cannot point at specific objects or accept a region the user marks. Kosmos-2, from Microsoft Research, adds grounding and referring on top of Kosmos-1. It keeps the same Transformer causal architecture and next-token objective. Bounding boxes are quantized into discrete location tokens: a P×P grid of bins is added to the vocabulary, and each box is written as a Markdown-style link attached to its text span.

For training data, the authors built GRIT, a web-scale corpus of grounded image-text pairs drawn from subsets of LAION-2B and COYO-700M. spaCy finds noun chunks, GLIP localizes them, and dependency parses extend the chunks into longer referring expressions.

Trained on GRIT plus the Kosmos-1 corpora, the model performs phrase grounding, referring expression comprehension and generation, and grounded captioning and VQA. It stays close to Kosmos-1 on captioning, VQA and text-only tasks. This makes region-level links between language and images a built-in skill of a general MLLM. Code, models and a GRIT subset are public.

## Key points
- GRIT holds about 91M images, 115M text spans and 137M bounding boxes. Abstract nouns are filtered out, boxes are pruned with non-maximum suppression, and only GLIP detections with confidence above 0.65 are kept.
- A box is encoded with special tokens as `<box><loc_top-left><loc_bottom-right></box>`, and the linked phrase is wrapped in `<p>…</p>`. Several boxes for one phrase are separated by `<delim>`, and a `<grounding>` token requests grounded output.
- On referring expression generation (RefCOCOg), zero-shot Kosmos-2 reaches 60.3 CIDEr, beating the finetuned SLR model (59.2). With 4-shot prompting it rises to 62.3.
- In zero-shot evaluation without instruction tuning, Flickr30k captioning reaches 66.7 CIDEr (Kosmos-1: 65.2), while VQAv2 accuracy dips slightly to 45.6 (Kosmos-1: 46.7).
- On eight language benchmarks, results are mostly on par with Kosmos-1. BoolQ improves (62.0 vs 56.4) and CB drops (30.4 vs 44.6).

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Microsoft Research (United Kingdom)](https://md.research.elamur.ai/free/en/companies/microsoft-research-united-kingdom/) — Company, Cambridge, United Kingdom · 4 works · 1,645 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 7,349 works · [👁️ Computer Vision and Pattern Recognition](https://md.research.elamur.ai/free/en/shelves/computer-science/computer-vision-and-pattern-recognition/) · 1,029 works · [⚙️ Engineering](https://md.research.elamur.ai/free/en/shelves/engineering/) · 44 works · [⚙️ Control and Systems Engineering](https://md.research.elamur.ai/free/en/shelves/engineering/control-and-systems-engineering/) · 20 works

🧭 Topics (2): [🧠 Language models & pretraining](https://md.research.elamur.ai/free/en/topics/language-models/) · 1,079 works · [🖼️ Vision & language](https://md.research.elamur.ai/free/en/topics/multimodal/) · 222 works

## Appears in

Agglomerations: [🏙️ Cambridge](https://md.research.elamur.ai/free/en/agglomerations/cambridge/)

## Cite

Cite as: Zhiliang Peng, Wenhui Wang, Li Dong, Yaru Hao, Shaohan Huang, Shuming Ma and Furu Wei. 2023. Kosmos-2: Grounding Multimodal Large Language Models to the World. arXiv:2306.14824.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@misc{peng2023kosmos,
  title = {{Kosmos-2: Grounding Multimodal Large Language Models to the World}},
  author = {Zhiliang Peng, Wenhui Wang, Li Dong, Yaru Hao, Shaohan Huang, Shuming Ma and Furu Wei},
  year = {2023},
  doi = {10.48550/arxiv.2306.14824},
  eprint = {2306.14824},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2306.14824}
}
```

- [← previous ChatIE: Zero-Shot Information Extraction via Chatting with ChatGPT](https://md.research.elamur.ai/free/en/papers/wei-2023-chatie-zero-shot-information-extraction/)
- [next → Having Beer after Prayer? Measuring Cultural Bias in Large Language Models](https://md.research.elamur.ai/free/en/papers/naous-2023-having-beer-after-prayer-measuring/)
- [↑ back to papers of 2023](https://md.research.elamur.ai/free/en/papers/2023/page/3/)

## References

33 references, 16 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Jean-Baptiste Alayrac, Jeff Donahue, Pauline Luc, Antoine Miech, Iain Barr, Yana Hasson, Karel Lenc, Arthur Mensch, Katherine Millican, Malcolm Reynolds, Roman Ring, Eliza Rutherford, Serkan Cabi, Tengda Han, Zhitao Gong, Sina Samangooei, Marianne Monteiro, Jacob Menick, Sebastian Borgeaud, Andrew B… [our card](https://md.research.elamur.ai/free/en/papers/alayrac-2022-flamingo-visual-language-model-few/)
2. Armen Aghajanyan, Bernie Huang, Candace Ross, Vladimir Karpukhin, Hu Xu, Naman Goyal, Dmytro Okhonko, Mandar Joshi, Gargi Ghosh, Mike Lewis, and Luke Zettlemoyer. CM3: A causal masked multimodal model of the Internet. ArXiv, abs/2201.07520, 2022. — [arXiv:2201.07520](https://arxiv.org/abs/2201.07520) [find](https://scholar.google.com/scholar?q=Armen+Aghajanyan%2C+Bernie+Huang%2C+Candace+Ross%2C+Vladimir+Karpukhin%2C+Hu+Xu%2C+Naman+Goyal%2C+Dmytro+Okhonko%2C+Mandar+Joshi%2C+Gargi+Ghosh%2C+Mike+Lewis%2C+and+Luke+Zettlemoyer%2E+CM3%3A+A+causal+masked+multimodal+model)
3. Nitzan Bitton-Guetta, Yonatan Bitton, Jack Hessel, Ludwig Schmidt, Yuval Elovici, Gabriel Stanovsky, and Roy Schwartz. Breaking common sense: WHOOPS! a vision-and-language benchmark of synthetic and compositional images. ArXiv, abs/2303.07274, 2023. [https://www.microsoft.com/ai/responsible-ai](https://www.microsoft.com/ai/responsible-ai) Winogr… — [arXiv:2303.07274](https://arxiv.org/abs/2303.07274) [↗](https://www.microsoft.com/ai/responsible-ai)
4. Minwoo Byeon, Beomhee Park, Haecheon Kim, Sungjun Lee, Woonhyuk Baek, and Saehoon Kim. Coyo-700m: Image-text pair dataset, 2022. [find](https://scholar.google.com/scholar?q=Minwoo+Byeon%2C+Beomhee+Park%2C+Haecheon+Kim%2C+Sungjun+Lee%2C+Woonhyuk+Baek%2C+and+Saehoon+Kim%2E+Coyo%2D700m%3A+Image%2Dtext+pair+dataset%2C+2022%2E)
5. Yen-Chun Chen, Linjie Li, Licheng Yu, Ahmed El Kholy, Faisal Ahmed, Zhe Gan, Yu Cheng, and Jingjing Liu. Uniter: Universal image-text representation learning. In European Conference on Computer Vision, 2019. [our card](https://md.research.elamur.ai/free/en/papers/chen-2019-uniter-universal-image-text-representation/)
6. Ting Chen, Saurabh Saxena, Lala Li, David J. Fleet, and Geo rey E. Hinton. Pix2seq: A language modeling framework for object detection. ArXiv, abs/2109.10852, 2021. — [arXiv:2109.10852](https://arxiv.org/abs/2109.10852) [find](https://scholar.google.com/scholar?q=Ting+Chen%2C+Saurabh+Saxena%2C+Lala+Li%2C+David+J%2E+Fleet%2C+and+Geo+rey+E%2E+Hinton%2E+Pix2seq%3A+A+language+modeling+framework+for+object+detection%2E+ArXiv%2C+abs%2F2109%2E10852%2C+2021%2E+%E2%80%94+%5BarXiv%3A2109%2E10852%5D%28https%3A%2F%2Farxiv%2E)
7. Zi-Yi Dou, Aishwarya Kamath, Zhe Gan, Pengchuan Zhang, Jianfeng Wang, Linjie Li, Zicheng Liu, Ce Liu, Yann LeCun, Nanyun Peng, Jianfeng Gao, and Lijuan Wang. Coarse-to-fine vision-language pre-training with fusion in the backbone. ArXiv, abs/2206.07643, 2022. — [arXiv:2206.07643](https://arxiv.org/abs/2206.07643) [find](https://scholar.google.com/scholar?q=Zi%2DYi+Dou%2C+Aishwarya+Kamath%2C+Zhe+Gan%2C+Pengchuan+Zhang%2C+Jianfeng+Wang%2C+Linjie+Li%2C+Zicheng+Liu%2C+Ce+Liu%2C+Yann+LeCun%2C+Nanyun+Peng%2C+Jianfeng+Gao%2C+and+Lijuan+Wang%2E+Coarse%2Dto%2Dfine+vision%2Dlanguage+pre%2Dtrainin)
8. Danny Driess, F. Xia, Mehdi S. M. Sajjadi, Corey Lynch, Aakanksha Chowdhery, Brian Ichter, Ayzaan Wahid, Jonathan Tompson, Quan Ho Vuong, Tianhe Yu, Wenlong Huang, Yevgen Chebotar, Pierre Sermanet, Daniel Duckworth, Sergey Levine, Vincent Vanhoucke, Karol Hausman, Marc Toussaint, Klaus Greff, Andy Z… [find](https://scholar.google.com/scholar?q=Danny+Driess%2C+F%2E+Xia%2C+Mehdi+S%2E+M%2E+Sajjadi%2C+Corey+Lynch%2C+Aakanksha+Chowdhery%2C+Brian+Ichter%2C+Ayzaan+Wahid%2C+Jonathan+Tompson%2C+Quan+Ho+Vuong%2C+Tianhe+Yu%2C+Wenlong+Huang%2C+Yevgen+Chebotar%2C+Pierre+Sermanet%2C+Da)
9. Shaohan Huang, Li Dong, Wenhui Wang, Yaru Hao, Saksham Singhal, Shuming Ma, Tengchao Lv, Lei Cui, Owais Khan Mohammed, Qiang Liu, Kriti Aggarwal, Zewen Chi, Johan Bjorck, Vishrav Chaudhary, Subhojit Som, Xia Song, and Furu Wei. Language is not all you need: Aligning perception with language models.… [our card](https://md.research.elamur.ai/free/en/papers/huang-2023-language-all-need-aligning-perception/)
10. Matthew Honnibal, Ines Montani, Sofie Van Landeghem, and Adriane Boyd. spaCy: Industrial-strength Natural Language Processing in Python. 2020. [find](https://scholar.google.com/scholar?q=Matthew+Honnibal%2C+Ines+Montani%2C+Sofie+Van+Landeghem%2C+and+Adriane+Boyd%2E+spaCy%3A+Industrial%2Dstrength+Natural+Language+Processing+in+Python%2E+2020%2E)
11. Yaru Hao, Haoyu Song, Li Dong, Shaohan Huang, Zewen Chi, Wenhui Wang, Shuming Ma, and Furu Wei. Language models are general-purpose interfaces. ArXiv, abs/2206.06336, 2022. — [arXiv:2206.06336](https://arxiv.org/abs/2206.06336) [find](https://scholar.google.com/scholar?q=Yaru+Hao%2C+Haoyu+Song%2C+Li+Dong%2C+Shaohan+Huang%2C+Zewen+Chi%2C+Wenhui+Wang%2C+Shuming+Ma%2C+and+Furu+Wei%2E+Language+models+are+general%2Dpurpose+interfaces%2E+ArXiv%2C+abs%2F2206%2E06336%2C+2022%2E+%E2%80%94+%5BarXiv%3A2206%2E06336%5D%28https%3A)
12. Or Honovich, Thomas Scialom, Omer Levy, and Timo Schick. Unnatural instructions: Tuning language models with (almost) no human labor, 2022. [our card](https://md.research.elamur.ai/free/en/papers/honovich-2023-unnatural-instructions-tuning-language-models/)
13. Woojeong Jin, Yu Cheng, Yelong Shen, Weizhu Chen, and Xiang Ren. A good prompt is worth millions of parameters: Low-resource prompt-based learning for vision-language models. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 276… [find](https://scholar.google.com/scholar?q=Woojeong+Jin%2C+Yu+Cheng%2C+Yelong+Shen%2C+Weizhu+Chen%2C+and+Xiang+Ren%2E+A+good+prompt+is+worth+millions+of+parameters%3A+Low%2Dresource+prompt%2Dbased+learning+for+vision%2Dlanguage+models%2E+In+Proceedings+of+the+60t)
14. Woojeong Jin, Subhabrata Mukherjee, Yu Cheng, Yelong Shen, Weizhu Chen, Ahmed Hassan Awadallah, Damien Jose, and Xiang Ren. Grill: Grounded vision-language pre-training via aligning text and image regions. ArXiv, abs/2305.14676, 2023. — [arXiv:2305.14676](https://arxiv.org/abs/2305.14676) [find](https://scholar.google.com/scholar?q=Woojeong+Jin%2C+Subhabrata+Mukherjee%2C+Yu+Cheng%2C+Yelong+Shen%2C+Weizhu+Chen%2C+Ahmed+Hassan+Awadallah%2C+Damien+Jose%2C+and+Xiang+Ren%2E+Grill%3A+Grounded+vision%2Dlanguage+pre%2Dtraining+via+aligning+text+and+image+reg)
15. Aishwarya Kamath, Mannat Singh, Yann LeCun, Ishan Misra, Gabriel Synnaeve, and Nicolas Carion. Mdetr - modulated detection for end-to-end multi-modal understanding. 2021 IEEE/CVF International Conference on Computer Vision (ICCV), pages 1760– 1770, 2021. [find](https://scholar.google.com/scholar?q=Aishwarya+Kamath%2C+Mannat+Singh%2C+Yann+LeCun%2C+Ishan+Misra%2C+Gabriel+Synnaeve%2C+and+Nicolas+Carion%2E+Mdetr+%2D+modulated+detection+for+end%2Dto%2Dend+multi%2Dmodal+understanding%2E+2021+IEEE%2FCVF+International+Confere)
16. Ranjay Krishna, Yuke Zhu, Oliver Groth, Justin Johnson, Kenji Hata, Joshua Kravitz, Stephanie Chen, Yannis Kalantidis, Li-Jia Li, David A. Shamma, Michael S. Bernstein, and Li Fei-Fei. Visual genome: Connecting language and vision using crowdsourced dense image annotations. International Journal of… [our card](https://md.research.elamur.ai/free/en/papers/krishna-2016-visual-genome-connecting-language-vision/)
17. Shayne Longpre, Le Hou, Tu Vu, Albert Webson, Hyung Won Chung, Yi Tay, Denny Zhou, Quoc V Le, Barret Zoph, Jason Wei, et al. The flan collection: Designing data and methods for effective instruction tuning. arXiv preprint arXiv:2301.13688, 2023. [our card](https://md.research.elamur.ai/free/en/papers/longpre-2023-flan-collection-designing-data-methods/)
18. Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi. BLIP-2: Bootstrapping language-image pre-training with frozen image encoders and large language models. ArXiv, abs/2301.12597, 2023. [our card](https://md.research.elamur.ai/free/en/papers/li-2023-blip-2-bootstrapping-language-image/)
19. Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning. arXiv preprint arXiv:2304.08485, 2023. [our card](https://md.research.elamur.ai/free/en/papers/liu-2023-visual-instruction-tuning/)
20. Liunian Harold Li, Mark Yatskar, Da Yin, Cho-Jui Hsieh, and Kai-Wei Chang. Visualbert: A simple and performant baseline for vision and language. ArXiv, abs/1908.03557, 2019. [our card](https://md.research.elamur.ai/free/en/papers/li-2019-visualbert-simple-performant-baseline-vision/)
21. Liunian Harold Li*, Pengchuan Zhang*, Haotian Zhang*, Jianwei Yang, Chunyuan Li, Yiwu Zhong, Lijuan Wang, Lu Yuan, Lei Zhang, Jenq-Neng Hwang, Kai-Wei Chang, and Jianfeng Gao. Grounded language-image pre-training. In CVPR, 2022. [find](https://scholar.google.com/scholar?q=Liunian+Harold+Li%2A%2C+Pengchuan+Zhang%2A%2C+Haotian+Zhang%2A%2C+Jianwei+Yang%2C+Chunyuan+Li%2C+Yiwu+Zhong%2C+Lijuan+Wang%2C+Lu+Yuan%2C+Lei+Zhang%2C+Jenq%2DNeng+Hwang%2C+Kai%2DWei+Chang%2C+and+Jianfeng+Gao%2E+Grounded+language%2Dimage+)
22. Junhua Mao, Jonathan Huang, Alexander Toshev, Oana-Maria Camburu, Alan Loddon Yuille, and Kevin P. Murphy. Generation and comprehension of unambiguous object descriptions. 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 11–20, 2015. [our card](https://md.research.elamur.ai/free/en/papers/mao-2015-generation-comprehension-unambiguous-object-descriptions/)
23. Shuming Ma, Hongyu Wang, Shaohan Huang, Wenhui Wang, Zewen Chi, Li Dong, Alon Benhaim, Barun Patra, Vishrav Chaudhary, Xia Song, and Furu Wei. TorchScale: Transformers at scale. CoRR, abs/2211.13184, 2022. — [arXiv:2211.13184](https://arxiv.org/abs/2211.13184) [find](https://scholar.google.com/scholar?q=Shuming+Ma%2C+Hongyu+Wang%2C+Shaohan+Huang%2C+Wenhui+Wang%2C+Zewen+Chi%2C+Li+Dong%2C+Alon+Benhaim%2C+Barun+Patra%2C+Vishrav+Chaudhary%2C+Xia+Song%2C+and+Furu+Wei%2E+TorchScale%3A+Transformers+at+scale%2E+CoRR%2C+abs%2F2211%2E13184%2C+)
24. OpenAI. Gpt-4 technical report. 2023. [our card](https://md.research.elamur.ai/free/en/papers/openai-2023-gpt-4-technical-report/)
25. Bryan A. Plummer, Liwei Wang, Christopher M. Cervantes, Juan C. Caicedo, J. Hockenmaier, and Svetlana Lazebnik. Flickr30k entities: Collecting region-to-phrase correspondences for richer image-to-sentence models. International Journal of Computer Vision, 123:74–93, 2015. [find](https://scholar.google.com/scholar?q=Bryan+A%2E+Plummer%2C+Liwei+Wang%2C+Christopher+M%2E+Cervantes%2C+Juan+C%2E+Caicedo%2C+J%2E+Hockenmaier%2C+and+Svetlana+Lazebnik%2E+Flickr30k+entities%3A+Collecting+region%2Dto%2Dphrase+correspondences+for+richer+image%2Dto%2Dsent)
26. Christoph Schuhmann, Romain Beaumont, Richard Vencu, Cade Gordon, Ross Wightman, Mehdi Cherti, Theo Coombes, Aarush Katta, Clayton Mullis, Mitchell Wortsman, et al. Laion-5b: An open large-scale dataset for training next generation image-text models. arXiv preprint arXiv:2210.08402, 2022. [our card](https://md.research.elamur.ai/free/en/papers/schuhmann-2022-laion-5b-open-large-scale/)
27. Ramakrishna Vedantam, C Lawrence Zitnick, and Devi Parikh. Cider: Consensus-based image description evaluation. In CVPR, pages 4566–4575, 2015. [our card](https://md.research.elamur.ai/free/en/papers/vedantam-2014-cider-consensus-based-image-description/)
28. Wen Wang, Zhe Chen, Xiaokang Chen, Jiannan Wu, Xizhou Zhu, Gang Zeng, Ping Luo, Tong Lu, Jie Zhou, Y. Qiao, and Jifeng Dai. Visionllm: Large language model is also an open-ended decoder for vision-centric tasks. ArXiv, abs/2305.11175, 2023. — [arXiv:2305.11175](https://arxiv.org/abs/2305.11175) [find](https://scholar.google.com/scholar?q=Wen+Wang%2C+Zhe+Chen%2C+Xiaokang+Chen%2C+Jiannan+Wu%2C+Xizhou+Zhu%2C+Gang+Zeng%2C+Ping+Luo%2C+Tong+Lu%2C+Jie+Zhou%2C+Y%2E+Qiao%2C+and+Jifeng+Dai%2E+Visionllm%3A+Large+language+model+is+also+an+open%2Dended+decoder+for+vision%2Dcen)
29. Hongyu Wang, Shuming Ma, Shaohan Huang, Li Dong, Wenhui Wang, Zhiliang Peng, Yu Wu, Payal Bajaj, Saksham Singhal, Alon Benhaim, Barun Patra, Zhun Liu, Vishrav Chaudhary, Xia Song, and Furu Wei. Foundation transformers. CoRR, abs/2210.06423, 2022. — [arXiv:2210.06423](https://arxiv.org/abs/2210.06423) [find](https://scholar.google.com/scholar?q=Hongyu+Wang%2C+Shuming+Ma%2C+Shaohan+Huang%2C+Li+Dong%2C+Wenhui+Wang%2C+Zhiliang+Peng%2C+Yu+Wu%2C+Payal+Bajaj%2C+Saksham+Singhal%2C+Alon+Benhaim%2C+Barun+Patra%2C+Zhun+Liu%2C+Vishrav+Chaudhary%2C+Xia+Song%2C+and+Furu+Wei%2E+Founda)
30. Alex Wang, Yada Pruksachatkun, Nikita Nangia, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel R Bowman. SuperGLUE: A stickier benchmark for general-purpose language understanding systems. arXiv preprint arXiv:1905.00537, 2019. [our card](https://md.research.elamur.ai/free/en/papers/wang-2019-superglue-stickier-benchmark-general-purpose/)
31. Peng Wang, An Yang, Rui Men, Junyang Lin, Shuai Bai, Zhikang Li, Jianxin Ma, Chang Zhou, Jingren Zhou, and Hongxia Yang. Unifying architectures, tasks, and modalities through a simple sequence-to-sequence learning framework. In International Conference on Machine Learning, 2022. [our card](https://md.research.elamur.ai/free/en/papers/wang-2022-ofa-unifying-architectures-tasks-modalities/)
32. Licheng Yu, Patrick Poirson, Shan Yang, Alexander C. Berg, and Tamara L. Berg. Modeling context in referring expressions. ArXiv, abs/1608.00272, 2016. [our card](https://md.research.elamur.ai/free/en/papers/yu-2016-modeling-context-referring-expressions/)
33. Licheng Yu, Hao Tan, Mohit Bansal, and Tamara L. Berg. A joint speaker-listener-reinforcer model for referring expressions. In 2017 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017, Honolulu, HI, USA, July 21-26, 2017, pages 3521–3529. IEEE Computer Society, 2017. [find](https://scholar.google.com/scholar?q=Licheng+Yu%2C+Hao+Tan%2C+Mohit+Bansal%2C+and+Tamara+L%2E+Berg%2E+A+joint+speaker%2Dlistener%2Dreinforcer+model+for+referring+expressions%2E+In+2017+IEEE+Conference+on+Computer+Vision+and+Pattern+Recognition%2C+CVPR+201)

*Updated 23 October 2026*