---
title: "Common Objects Out of Context (COOCo): Investigating"
description: "The paper asks how much vision-language models (VLMs) draw on surrounding scene context when naming an object, and… 59 references."
authors: ["Filippo Merlo", "Ece Takmaz", "Wenkai Chen", "Albert Gatt"]
year: 2026
venue: "Transactions of the Association for Computational Linguistics"
source: "https://aclanthology.org/2026.tacl-1.52/"
doi: "10.1162/tacl.a.701"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/merlo-2026-common-objects-out-context-cooco/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/merlo-2026-common-objects-out-context-cooco/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/25/) / Common Objects Out of Context (COOCo): Investigating Multimodal Context and Semantic Scene Violations in Referential Communication

# Common Objects Out of Context (COOCo): Investigating Multimodal Context and Semantic Scene Violations in Referential Communication

**Authors:** Filippo Merlo, [Ece Takmaz](https://md.research.elamur.ai/free/en/authors/ece-takmaz/), Wenkai Chen and [Albert Gatt](https://md.research.elamur.ai/free/en/authors/albert-gatt/) · **Year:** 2026 · **Venue:** [Transactions of the Association for Computational Linguistics](https://md.research.elamur.ai/free/en/venues/tacl/) ([Transactions of the Association for Computational Linguistics (TACL) 2026](https://md.research.elamur.ai/free/en/venues/tacl/2026/)) · **Publisher:** [The MIT Press](https://md.research.elamur.ai/free/en/publishers/the-mit-press/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2026.tacl-1.52/) · [PDF](https://aclanthology.org/2026.tacl-1.52.pdf) · [DOI 10.1162/tacl.a.701](https://doi.org/10.1162/tacl.a.701) · [OpenAlex](https://openalex.org/W4416261365)

**Citations:** 0 citations (OpenAlex, 2026-10-14)

## Summary
The paper asks how much vision-language models (VLMs) draw on surrounding scene context when naming an object, and whether this depends on how well the object fits the scene. The authors built COOCo from COCO-Search18 photos. They used an inpainting pipeline to replace a target object with alternatives whose relatedness to the scene was graded as low, medium or high, plus same-category replacements as controls. Several VLMs were tested on object naming, with Gaussian noise applied either to the target or to the context. Models relied on context selectively. When the object fit the scene, context helped and made models robust to a degraded target. When the object clashed with the scene, context acted as a distractor, and hiding it improved performance. An attention analysis of LLaVA-OneVision showed that correct naming went with stronger mid-layer attention to the target. Attention also varied with semantic fit in a U-shaped way, resembling human eye-tracking findings. This matters for building and interpreting robust referring-expression systems.

## Key points
- COOCo has 18,395 images built from 2,241 retained COCO-Search18 images. These include 1,862 originals with object-removed versions, 5,480 low-relatedness images, 5,572 medium-relatedness images and 1,771 high-relatedness images.
- Scene types were assigned by a ViT finetuned on SUN-397, reduced to 29 labels.
- Replacement objects came from THINGSplus norms, filtered for typicality and real-world size. Their scene relatedness was scored by cosine similarity of ConceptNet Numberbatch embeddings.
- In LLaVA-OneVision-0.5B, correct categorisation was associated with higher attention to the target in layers 7–15. Quadratic fits of attention against semantic fit were convex, with the minimum near 0.764 on COOCo.
- The findings were replicated on the human-validated VISIONS dataset (1,136 images), evaluated with RefCLIPScore and accuracy. Moderate target noise (0.5) increased attention to the target, while full noise (1.0) pushed reliance onto context.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [University of Trento](https://md.research.elamur.ai/free/en/universities/university-of-trento/) — University, Trento, Italy · 85 works · 15,524 citations
- [Utrecht University](https://md.research.elamur.ai/free/en/universities/utrecht-university/) — University, Utrecht, Netherlands · 39 works · 1,603 citations

📚 Shelves (2): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [👁️ Computer Vision and Pattern Recognition](https://md.research.elamur.ai/free/en/shelves/computer-science/computer-vision-and-pattern-recognition/) · 2,869 works

🧭 Topics (1): [🖼️ Vision & language](https://md.research.elamur.ai/free/en/topics/multimodal/) · 657 works

## Cite

Cite as: Filippo Merlo, Ece Takmaz, Wenkai Chen and Albert Gatt. 2026. Common Objects Out of Context (COOCo): Investigating Multimodal Context and Semantic Scene Violations in Referential Communication. Transactions of the Association for Computational Linguistics.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@article{merlo2026common,
  title = {{Common Objects Out of Context (COOCo): Investigating Multimodal Context and Semantic Scene Violations in Referential Communication}},
  author = {Filippo Merlo, Ece Takmaz, Wenkai Chen and Albert Gatt},
  journal = {Transactions of the Association for Computational Linguistics},
  year = {2026},
  doi = {10.1162/tacl.a.701},
  url = {https://doi.org/10.1162/tacl.a.701}
}
```

- [← previous Combining Semantic Embeddings and Knowledge Graphs for Identifying Decision Patterns in Brazilian Judicial Decisions](https://md.research.elamur.ai/free/en/papers/silva-2026-combining-semantic-embeddings-knowledge-graphs/)
- [next → Common Sense vs. Morality: The Curious Case of Narrative Focus Bias in LLMs](https://md.research.elamur.ai/free/en/papers/purkayastha-2026-common-sense-vs-morality-curious/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/25/)

## References

59 references, 14 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Elena Allegretti, Giorgia D’Innocenzo, and Moreno I. Coco. 2025. The Visual Integration of Semantic and Spatial Information of Objects in Naturalistic Scenes (VISIONS) database: attentional, conceptual, and perceptual norms. Behavior Research Methods, 57(1):42. [find](https://scholar.google.com/scholar?q=Elena+Allegretti%2C+Giorgia+D%E2%80%99Innocenzo%2C+and+Moreno+I%2E+Coco%2E+2025%2E+The+Visual+Integration+of+Semantic+and+Spatial+Information+of+Objects+in+Naturalistic+Scenes+%28VISIONS%29+database%3A+attentional%2C+conceptua)
2. Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, Humen Zhong, Yuanzhi Zhu, Mingkun Yang, Zhaohai Li, Jianqiang Wan, Pengfei Wang, Wei Ding, Zheren Fu, Yiheng Xu, Jiabo Ye, Xi Zhang, Tianbao Xie, Zesen Cheng, Hang Zhang, Zhibo Yang, Ha… — [Qwen2.5-VL Technical Report](https://md.research.elamur.ai/free/en/papers/bai-2025-qwen2-5-vl-technical/)
3. Moshe Bar. 2004. Visual objects in context. Nature Reviews Neuroscience, 5(8):617–629. [find](https://scholar.google.com/scholar?q=Moshe+Bar%2E+2004%2E+Visual+objects+in+context%2E+Nature+Reviews+Neuroscience%2C+5%288%29%3A617%E2%80%93629%2E)
4. Irving Biederman, Robert J. Mezzanotte, and Jan C. Rabinowitz. 1982. Scene perception: Detecting and judging objects undergoing relational violations. Cognitive Psychology, 14(2):143–177. [find](https://scholar.google.com/scholar?q=Irving+Biederman%2C+Robert+J%2E+Mezzanotte%2C+and+Jan+C%2E+Rabinowitz%2E+1982%2E+Scene+perception%3A+Detecting+and+judging+objects+undergoing+relational+violations%2E+Cognitive+Psychology%2C+14%282%29%3A143%E2%80%93177%2E)
5. Sage E. P. Boettcher, Dejan Draschkow, Eric Di-enhart, and Melissa L.-H. Võ. 2018. Anchoring visual search in scenes: Assessing the role of anchor objects on eye movements during visual search. Journal of Vision, 18(13):11. [find](https://scholar.google.com/scholar?q=Sage+E%2E+P%2E+Boettcher%2C+Dejan+Draschkow%2C+Eric+Di%2Denhart%2C+and+Melissa+L%2E%2DH%2E+V%C3%B5%2E+2018%2E+Anchoring+visual+search+in+scenes%3A+Assessing+the+role+of+anchor+objects+on+eye+movements+during+visual+search%2E+Journa)
6. Philipp Bomatter, Mengmi Zhang, Dimitar Karev, Spandan Madan, Claire Tseng, and Gabriel Kreiman. 2021. When Pigs Fly: Contextual Reasoning in Synthetic and Natural Scenes. ArXiv:2104.02215 [cs]. — [arXiv:2104.02215](https://arxiv.org/abs/2104.02215) [find](https://scholar.google.com/scholar?q=Philipp+Bomatter%2C+Mengmi+Zhang%2C+Dimitar+Karev%2C+Spandan+Madan%2C+Claire+Tseng%2C+and+Gabriel+Kreiman%2E+2021%2E+When+Pigs+Fly%3A+Contextual+Reasoning+in+Synthetic+and+Natural+Scenes%2E+ArXiv%3A2104%2E02215+%5Bcs%5D%2E+%E2%80%94+%5Bar)
7. Yingshan Chang, Mridu Narang, Hisami Suzuki, Guihong Cao, Jianfeng Gao, and Yonatan Bisk. 2022. WebQA: Multihop and Multimodal QA. ArXiv:2109.00590 [cs]. — [arXiv:2109.00590](https://arxiv.org/abs/2109.00590) [find](https://scholar.google.com/scholar?q=Yingshan+Chang%2C+Mridu+Narang%2C+Hisami+Suzuki%2C+Guihong+Cao%2C+Jianfeng+Gao%2C+and+Yonatan+Bisk%2E+2022%2E+WebQA%3A+Multihop+and+Multimodal+QA%2E+ArXiv%3A2109%2E00590+%5Bcs%5D%2E+%E2%80%94+%5BarXiv%3A2109%2E00590%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F210)
8. Yupei Chen, Zhibo Yang, Seoyoung Ahn, Dimitris Samaras, Minh Hoai, and Gregory Zelinsky. 2021. COCO-Search18 fixation dataset for predicting goal-directed attention control. Scientific Reports, 11(1):8776. [find](https://scholar.google.com/scholar?q=Yupei+Chen%2C+Zhibo+Yang%2C+Seoyoung+Ahn%2C+Dimitris+Samaras%2C+Minh+Hoai%2C+and+Gregory+Zelinsky%2E+2021%2E+COCO%2DSearch18+fixation+dataset+for+predicting+goal%2Ddirected+attention+control%2E+Scientific+Reports%2C+11%281%29%3A)
9. Alasdair D F Clarke, Moreno I. Coco, and Frank Keller. 2013. The impact of attentional, linguistic, and visual features during object naming. Frontiers in Psychology, 4(DEC):1–12. [find](https://scholar.google.com/scholar?q=Alasdair+D+F+Clarke%2C+Moreno+I%2E+Coco%2C+and+Frank+Keller%2E+2013%2E+The+impact+of+attentional%2C+linguistic%2C+and+visual+features+during+object+naming%2E+Frontiers+in+Psychology%2C+4%28DEC%29%3A1%E2%80%9312%2E)
10. Moreno I. Coco, Frank Keller, and George L. Malcolm. 2016. Anticipation in real-world scenes: The role of visual context and visual memory. Cognitive Science, 40(8):1995–2024. [find](https://scholar.google.com/scholar?q=Moreno+I%2E+Coco%2C+Frank+Keller%2C+and+George+L%2E+Malcolm%2E+2016%2E+Anticipation+in+real%2Dworld+scenes%3A+The+role+of+visual+context+and+visual+memory%2E+Cognitive+Science%2C+40%288%29%3A1995%E2%80%932024%2E)
11. Claudia Damiano, Maarten Leemans, and Johan Wagemans. 2024. Exploring the Semantic-Inconsistency Effect in Scenes Using a Continuous Measure of Linguistic-Semantic Similarity. Psychological Science, 35(6):623–634. [find](https://scholar.google.com/scholar?q=Claudia+Damiano%2C+Maarten+Leemans%2C+and+Johan+Wagemans%2E+2024%2E+Exploring+the+Semantic%2DInconsistency+Effect+in+Scenes+Using+a+Continuous+Measure+of+Linguistic%2DSemantic+Similarity%2E+Psychological+Science%2C+3)
12. Matt Deitke, Christopher Clark, Sangho Lee, Rohun Tripathi, Yue Yang, Jae Sung Park, Mohammadreza Salehi, Niklas Muennighoff, Kyle Lo, Luca Soldaini, Jiasen Lu, Taira Anderson, Erin Bransom, Kiana Ehsani, Huong Ngo, Yen-Sung Chen, Ajay Patel, Mark Yatskar, Chris Callison-Burch, Andrew Head, Rose Hen… — [Molmo and PixMo: Open Weights and Open Data for State-of-the-Art Vision-Language Models](https://md.research.elamur.ai/free/en/papers/deitke-2024-molmo-pixmo-open-weights-open/)
13. Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby. 2021. [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://md.research.elamur.ai/free/en/papers/dosovitskiy-2020-image-worth-16x16-words-transformers/). ArXiv:201…
14. Haley C. Dresang, Michael Walsh Dickey, and Tessa C. Warren. 2019. Semantic memory for objects, actions, and events: A novel test of event-related conceptual semantic knowledge. Cognitive Neuropsychology, 36(7-8):313–335. [find](https://scholar.google.com/scholar?q=Haley+C%2E+Dresang%2C+Michael+Walsh+Dickey%2C+and+Tessa+C%2E+Warren%2E+2019%2E+Semantic+memory+for+objects%2C+actions%2C+and+events%3A+A+novel+test+of+event%2Drelated+conceptual+semantic+knowledge%2E+Cognitive+Neuropsychol)
15. John M. Henderson, Taylor R. Hayes, Candace E. Peacock, and Gwendolyn Rehrig. 2019. Meaning and Attentional Guidance in Scenes: A Review of the Meaning Map Approach. Vision, 3(2):19. [find](https://scholar.google.com/scholar?q=John+M%2E+Henderson%2C+Taylor+R%2E+Hayes%2C+Candace+E%2E+Peacock%2C+and+Gwendolyn+Rehrig%2E+2019%2E+Meaning+and+Attentional+Guidance+in+Scenes%3A+A+Review+of+the+Meaning+Map+Approach%2E+Vision%2C+3%282%29%3A19%2E)
16. Albert End and Matthias Gamer. 2017. Preferential processing of social features and their interplay with physical saliency in complex naturalistic scenes. Frontiers in Psychology, 8(March):418. [find](https://scholar.google.com/scholar?q=Albert+End+and+Matthias+Gamer%2E+2017%2E+Preferential+processing+of+social+features+and+their+interplay+with+physical+saliency+in+complex+naturalistic+scenes%2E+Frontiers+in+Psychology%2C+8%28March%29%3A418%2E)
17. Jack Hessel, Ari Holtzman, Maxwell Forbes, Ronan Le Bras, and Yejin Choi. 2022. [CLIPScore: A Reference-free Evaluation Metric for Image Captioning](https://md.research.elamur.ai/free/en/papers/hessel-2021-clipscore-reference-free-evaluation-metric/). ArXiv:2104.08718.
18. Giorgio Ganis and Marta Kutas. 2003. An electrophysiological study of scene effects on object identification. Cognitive Brain Research, 16(2):123–144. [find](https://scholar.google.com/scholar?q=Giorgio+Ganis+and+Marta+Kutas%2E+2003%2E+An+electrophysiological+study+of+scene+effects+on+object+identification%2E+Cognitive+Brain+Research%2C+16%282%29%3A123%E2%80%93144%2E)
19. Michal Golovanevsky, William Rudman, Vedant Palit, Carsten Eickhoff, and Ritambhara Singh. 2025. What Do VLMs NOTICE? A Mechanistic Interpretability Pipeline for Gaussian-Noise-free Text-Image Corruption and Evaluation. In Proceedings of the 2025 Conference of the Nations of the Americas Chapter of… [find](https://scholar.google.com/scholar?q=Michal+Golovanevsky%2C+William+Rudman%2C+Vedant+Palit%2C+Carsten+Eickhoff%2C+and+Ritambhara+Singh%2E+2025%2E+What+Do+VLMs+NOTICE%3F+A+Mechanistic+Interpretability+Pipeline+for+Gaussian%2DNoise%2Dfree+Text%2DImage+Corrupt)
20. Floor de Groot, Falk Huettig, and Christian N. L. Olivers. 2015. When Meaning Matters: The Temporal Dynamics of Semantic Influences on Visual Attention. Journal of Experimental Psychology: Human Perception and Performance. [find](https://scholar.google.com/scholar?q=Floor+de+Groot%2C+Falk+Huettig%2C+and+Christian+N%2E+L%2E+Olivers%2E+2015%2E+When+Meaning+Matters%3A+The+Temporal+Dynamics+of+Semantic+Influences+on+Visual+Attention%2E+Journal+of+Experimental+Psychology%3A+Human+Perce)
21. Floor de Groot, Falk Huettig, and Christian N.L. Olivers. 2017. Language-induced visual and semantic biases in visual search are subject to task requirements. Visual Cognition, 0(0):1–16. Publisher: Taylor & Francis. [find](https://scholar.google.com/scholar?q=Floor+de+Groot%2C+Falk+Huettig%2C+and+Christian+N%2EL%2E+Olivers%2E+2017%2E+Language%2Dinduced+visual+and+semantic+biases+in+visual+search+are+subject+to+task+requirements%2E+Visual+Cognition%2C+0%280%29%3A1%E2%80%9316%2E+Publisher%3A+T)
22. Taylor R. Hayes and John M. Henderson. 2021. Looking for semantic similarity: What a vector-space model of semantics can tell us about attention in real-world scenes. Psychological Science, 32(8):1262–1270. PMID: 34252325. [find](https://scholar.google.com/scholar?q=Taylor+R%2E+Hayes+and+John+M%2E+Henderson%2E+2021%2E+Looking+for+semantic+similarity%3A+What+a+vector%2Dspace+model+of+semantics+can+tell+us+about+attention+in+real%2Dworld+scenes%2E+Psychological+Science%2C+32%288%29%3A1262)
23. Martin N. Hebart, Adam H. Dickter, Alexis Kidder, Wan Y. Kwok, Anna Corriveau, Caitlin Van Wicklin, and Chris I. Baker. 2019. THINGS: A database of 1,854 object concepts Alex D. Hwang, Hsueh-Cheng Wang, and Marc Pomplun. 2011. Semantic guidance of eye movements in real-world scenes. Vision Research,… [find](https://scholar.google.com/scholar?q=Martin+N%2E+Hebart%2C+Adam+H%2E+Dickter%2C+Alexis+Kidder%2C+Wan+Y%2E+Kwok%2C+Anna+Corriveau%2C+Caitlin+Van+Wicklin%2C+and+Chris+I%2E+Baker%2E+2019%2E+THINGS%3A+A+database+of+1%2C854+object+concepts+Alex+D%2E+Hwang%2C+Hsueh%2DCheng+Wan)
24. Benoit Jacob, Skirmantas Kligys, Bo Chen, Menglong Zhu, Matthew Tang, Andrew Howard, Hartwig Adam, and Dmitry Kalenichenko. 2017. Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inference. ArXiv:1712.05877 [cs]. [find](https://scholar.google.com/scholar?q=Benoit+Jacob%2C+Skirmantas+Kligys%2C+Bo+Chen%2C+Menglong+Zhu%2C+Matthew+Tang%2C+Andrew+Howard%2C+Hartwig+Adam%2C+and+Dmitry+Kalenichenko%2E+2017%2E+Quantization+and+Training+of+Neural+Networks+for+Efficient+Integer%2DAri)
25. Simeon Junker and Sina Zarrieß. 2024. Resilience through scene context in visual referring expression generation. In Proceedings of the 17th International Natural Language Generation Conference, pages 344–357, Tokyo, Japan. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Simeon+Junker+and+Sina+Zarrie%C3%9F%2E+2024%2E+Resilience+through+scene+context+in+visual+referring+expression+generation%2E+In+Proceedings+of+the+17th+International+Natural+Language+Generation+Conference%2C+pages)
26. Simeon Junker and Sina Zarrieß. 2025. Scene-Gram: Conceptualizing and Describing Tangrams in Scene Context. arXiv preprint arXiv:2506.11631. — [arXiv:2506.11631](https://arxiv.org/abs/2506.11631) [find](https://scholar.google.com/scholar?q=Simeon+Junker+and+Sina+Zarrie%C3%9F%2E+2025%2E+Scene%2DGram%3A+Conceptualizing+and+Describing+Tangrams+in+Scene+Context%2E+arXiv+preprint+arXiv%3A2506%2E11631%2E+%E2%80%94+%5BarXiv%3A2506%2E11631%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2506%2E11631%29)
27. Sahar Kazemzadeh, Vicente Ordonez, Mark Matten, and Tamara Berg. 2014. [ReferItGame: Referring to Objects in Photographs of Natural Scenes](https://md.research.elamur.ai/free/en/papers/kazemzadeh-2014-referitgame-referring-objects-photographs-natural/). In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 787–798, Doha, Qatar. Association for Computational Ling…
28. Emiel Krahmer and Kees Van Deemter. 2012. [Computational Generation of Referring Expressions: A Survey](https://md.research.elamur.ai/free/en/papers/krahmer-2012-computational-generation-referring-expressions-survey/). Computational Linguistics, 38(1):173–218.
29. Ranjay Krishna, Yuke Zhu, Oliver Groth, Justin Johnson, Kenji Hata, Joshua Kravitz, Stephanie Chen, Yannis Kalantidis, Li-Jia Li, David A. Shamma, Michael S. Bernstein, and Fei-Fei Li. 2016. [Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotations](https://md.research.elamur.ai/free/en/papers/krishna-2016-visual-genome-connecting-language-vision/). ArXiv:1602.07332 [c…
30. Tim Lauer, Tim H. W. Cornelissen, Dejan Draschkow, Verena Willenbockel, and Melissa L.-H. Võ. 2018. The role of scene summary statistics in object recognition. Scientific Reports, 8(1):14666. [find](https://scholar.google.com/scholar?q=Tim+Lauer%2C+Tim+H%2E+W%2E+Cornelissen%2C+Dejan+Draschkow%2C+Verena+Willenbockel%2C+and+Melissa+L%2E%2DH%2E+V%C3%B5%2E+2018%2E+The+role+of+scene+summary+statistics+in+object+recognition%2E+Scientific+Reports%2C+8%281%29%3A14666%2E)
31. Tim Lauer, Filipp Schmidt, and Melissa L.-H. Võ. 2021. The role of contextual materials in object recognition. Scientific Reports, 11(1):21988. [find](https://scholar.google.com/scholar?q=Tim+Lauer%2C+Filipp+Schmidt%2C+and+Melissa+L%2E%2DH%2E+V%C3%B5%2E+2021%2E+The+role+of+contextual+materials+in+object+recognition%2E+Scientific+Reports%2C+11%281%29%3A21988%2E)
32. Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. 2021. [Learning Transferable Visual Models From Natural Language Supervision](https://md.research.elamur.ai/free/en/papers/radford-2021-learning-transferable-visual-models-natural/). ArXiv:2103.00020 [cs].
33. Simeon Schüz, Albert Gatt, and Sina Zarrieß. 2023. Rethinking symbolic and visual context in Referring Expression Generation. Frontiers in Artificial Intelligence, 6:1067125. [find](https://scholar.google.com/scholar?q=Simeon+Sch%C3%BCz%2C+Albert+Gatt%2C+and+Sina+Zarrie%C3%9F%2E+2023%2E+Rethinking+symbolic+and+visual+context+in+Referring+Expression+Generation%2E+Frontiers+in+Artificial+Intelligence%2C+6%3A1067125%2E)
34. Bo Li, Yuanhan Zhang, Dong Guo, Renrui Zhang, Feng Li, Hao Zhang, Kaichen Zhang, Peiyuan Zhang, Yanwei Li, Ziwei Liu, and Chunyuan Li. 2025. [LLaVA-onevision: Easy visual task transfer](https://md.research.elamur.ai/free/en/papers/li-2024-llava-onevision-easy-visual-task/). Transactions on Machine Learning Research.
35. Simeon Schüz and Sina Zarrieß. 2023. Keeping an Eye on Context: Attention Allocation over Input Partitions in Referring Expression Generation. In Proceedings of the Workshop on Multimodal, Multilingual Natural Language Generation and Multilingual WebNLG Challenge (MM-NLG 2023), pages 20–27, Prague,… [find](https://scholar.google.com/scholar?q=Simeon+Sch%C3%BCz+and+Sina+Zarrie%C3%9F%2E+2023%2E+Keeping+an+Eye+on+Context%3A+Attention+Allocation+over+Input+Partitions+in+Referring+Expression+Generation%2E+In+Proceedings+of+the+Workshop+on+Multimodal%2C+Multilingua)
36. Tsung-Yi Lin, Michael Maire, Serge Belongie, Lubomir Bourdev, Ross Girshick, James Hays, Pietro Perona, Deva Ramanan, C. Lawrence Zitnick, and Piotr Dollár. 2015. [Microsoft COCO: Common Objects in Context](https://md.research.elamur.ai/free/en/papers/lin-2014-microsoft-coco-common-objects-context/). ArXiv:1405.0312 [cs].
37. Yarden Shir, Naphtali Abudarham, and Liad Mu-drik. 2021. You won’t believe what this guy is doing with the potato: The ObjAct stimulus-set depicting human actions on congruent and incongruent objects. Behavior Research Methods, 53(5):1895–1909. [find](https://scholar.google.com/scholar?q=Yarden+Shir%2C+Naphtali+Abudarham%2C+and+Liad+Mu%2Ddrik%2E+2021%2E+You+won%E2%80%99t+believe+what+this+guy+is+doing+with+the+potato%3A+The+ObjAct+stimulus%2Dset+depicting+human+actions+on+congruent+and+incongruent+objects%2E)
38. Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. 2023. [Visual Instruction Tuning](https://md.research.elamur.ai/free/en/papers/liu-2023-visual-instruction-tuning/). ArXiv:2304.08485 [cs].
39. Robyn Speer, Joshua Chin, and Catherine Havasi. 2018. ConceptNet 5.5: An Open Multilingual Graph of General Knowledge. ArXiv:1612.03975 [cs]. [find](https://scholar.google.com/scholar?q=Robyn+Speer%2C+Joshua+Chin%2C+and+Catherine+Havasi%2E+2018%2E+ConceptNet+5%2E5%3A+An+Open+Multilingual+Graph+of+General+Knowledge%2E+ArXiv%3A1612%2E03975+%5Bcs%5D%2E)
40. Sabine Öhlschläger and Melissa L.-H. Võ. 2016. Scegram: An image database for semantic and syntactic inconsistencies in scenes. Behavior Research Methods, 49:1780–1791. [find](https://scholar.google.com/scholar?q=Sabine+%C3%96hlschl%C3%A4ger+and+Melissa+L%2E%2DH%2E+V%C3%B5%2E+2016%2E+Scegram%3A+An+image+database+for+semantic+and+syntactic+inconsistencies+in+scenes%2E+Behavior+Research+Methods%2C+49%3A1780%E2%80%931791%2E)
41. Aude Oliva and Antonio Torralba. 2007. The role of context in object recognition. Trends in Cognitive Sciences, 11(12):520–527. [find](https://scholar.google.com/scholar?q=Aude+Oliva+and+Antonio+Torralba%2E+2007%2E+The+role+of+context+in+object+recognition%2E+Trends+in+Cognitive+Sciences%2C+11%2812%29%3A520%E2%80%93527%2E)
42. Laura M. Stoinski, Jonas Perkuhn, and Martin N. Hebart. 2023. THINGSplus: New norms and metadata for the THINGS database of 1854 object concepts and 26,107 natural object images. Behavior Research Methods. [find](https://scholar.google.com/scholar?q=Laura+M%2E+Stoinski%2C+Jonas+Perkuhn%2C+and+Martin+N%2E+Hebart%2E+2023%2E+THINGSplus%3A+New+norms+and+metadata+for+the+THINGS+database+of+1854+object+concepts+and+26%2C107+natural+object+images%2E+Behavior+Research+Met)
43. Marius V. Peelen, Eva Berlot, and Floris P. De Lange. 2023. Predictive processing of scenes and objects. Nature Reviews Psychology, 3(1):13–26. [find](https://scholar.google.com/scholar?q=Marius+V%2E+Peelen%2C+Eva+Berlot%2C+and+Floris+P%2E+De+Lange%2E+2023%2E+Predictive+processing+of+scenes+and+objects%2E+Nature+Reviews+Psychology%2C+3%281%29%3A13%E2%80%9326%2E)
44. Roman Suvorov, Elizaveta Logacheva, Anton Mashikhin, Anastasia Remizova, Arsenii Ashukha, Aleksei Silvestrov, Naejin Kong, Harshith Goka, Kiwoong Park, and Victor Lempitsky. 2021. Resolution-robust Large Mask Inpainting with Fourier Convolutions. ArXiv:2109.07161 [cs]. — [arXiv:2109.07161](https://arxiv.org/abs/2109.07161) [find](https://scholar.google.com/scholar?q=Roman+Suvorov%2C+Elizaveta+Logacheva%2C+Anton+Mashikhin%2C+Anastasia+Remizova%2C+Arsenii+Ashukha%2C+Aleksei+Silvestrov%2C+Naejin+Kong%2C+Harshith+Goka%2C+Kiwoong+Park%2C+and+Victor+Lempitsky%2E+2021%2E+Resolution%2Drobust+La)
45. Zhiliang Peng, Wenhui Wang, Li Dong, Yaru Hao, Shaohan Huang, Shuming Ma, Qixiang Ye, and Furu Wei. 2024. Grounding multimodal large language models to the world. In The Twelfth International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Zhiliang+Peng%2C+Wenhui+Wang%2C+Li+Dong%2C+Yaru+Hao%2C+Shaohan+Huang%2C+Shuming+Ma%2C+Qixiang+Ye%2C+and+Furu+Wei%2E+2024%2E+Grounding+multimodal+large+language+models+to+the+world%2E+In+The+Twelfth+International+Conferen)
46. Antonio Torralba, Aude Oliva, Monica S. Castel-hano, and John M. Henderson. 2006. Contextual guidance of eye movements and attention in real-world scenes: The role of global features in object search. Psychological Review, 113(4):766–786. [find](https://scholar.google.com/scholar?q=Antonio+Torralba%2C+Aude+Oliva%2C+Monica+S%2E+Castel%2Dhano%2C+and+John+M%2E+Henderson%2E+2006%2E+Contextual+guidance+of+eye+movements+and+attention+in+real%2Dworld+scenes%3A+The+role+of+global+features+in+object+search%2E)
47. Jesse Vig and Yonatan Belinkov. 2019. [Analyzing the Structure of Attention in a Transformer Language Model](https://md.research.elamur.ai/free/en/papers/vig-2019-analyzing-structure-attention-transformer-language/). In Proceedings of the 2019 ACL Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP, pages 63–76, Florence, Italy. Association for Computational Linguistics.
48. Melissa Le-Hoa Võ. 2021. The meaning and structure of scenes. Vision Research, 181:10– 20. [find](https://scholar.google.com/scholar?q=Melissa+Le%2DHoa+V%C3%B5%2E+2021%2E+The+meaning+and+structure+of+scenes%2E+Vision+Research%2C+181%3A10%E2%80%93+20%2E)
49. Jianxiong Xiao, James Hays, Krista A. Ehinger, Aude Oliva, and Antonio Torralba. 2010. SUN database: Large-scale scene recognition from abbey to zoo. In 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pages 3485–3492, San Francisco, CA, USA. IEEE. [find](https://scholar.google.com/scholar?q=Jianxiong+Xiao%2C+James+Hays%2C+Krista+A%2E+Ehinger%2C+Aude+Oliva%2C+and+Antonio+Torralba%2E+2010%2E+SUN+database%3A+Large%2Dscale+scene+recognition+from+abbey+to+zoo%2E+In+2010+IEEE+Computer+Society+Conference+on+Comput)
50. Le Xue, Manli Shu, Anas Awadalla, Jun Wang, An Yan, Senthil Purushwalkam, Honglu Zhou, Viraj Prabhu, Yutong Dai, Michael S Ryoo, Shrikant Kendre, Jieyu Zhang, Can Qin, Shu Zhang, Chia-Chih Chen, Ning Yu, Juntao Tan, Tulika Manoj Awalgaonkar, Shelby Heinecke, Huan Wang, Yejin Choi, Ludwig Schmidt, Ze… [find](https://scholar.google.com/scholar?q=Le+Xue%2C+Manli+Shu%2C+Anas+Awadalla%2C+Jun+Wang%2C+An+Yan%2C+Senthil+Purushwalkam%2C+Honglu+Zhou%2C+Viraj+Prabhu%2C+Yutong+Dai%2C+Michael+S+Ryoo%2C+Shrikant+Kendre%2C+Jieyu+Zhang%2C+Can+Qin%2C+Shu+Zhang%2C+Chia%2DChih+Chen%2C+Ning+)
51. Tianze Yang, Tyson Jordan, Ninghao Liu, and Jin Sun. 2025. Common Inpainted Objects In-N-Out of Context. ArXiv:2506.00721 [cs]. — [arXiv:2506.00721](https://arxiv.org/abs/2506.00721) [find](https://scholar.google.com/scholar?q=Tianze+Yang%2C+Tyson+Jordan%2C+Ninghao+Liu%2C+and+Jin+Sun%2E+2025%2E+Common+Inpainted+Objects+In%2DN%2DOut+of+Context%2E+ArXiv%3A2506%2E00721+%5Bcs%5D%2E+%E2%80%94+%5BarXiv%3A2506%2E00721%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2506%2E00721%29)
52. Licheng Yu, Patrick Poirson, Shan Yang, Alexander C Berg, and Tamara L Berg. 2016. [Modeling Context in Referring Expressions](https://md.research.elamur.ai/free/en/papers/yu-2016-modeling-context-referring-expressions/). In ECCV. ArXiv: 1608.00272.
53. Woo-han Yun, Taewoo Kim, Jaeyeon Lee, Jaehong Kim, and Junmo Kim. 2021. Cut-and-Paste Dataset Generation for Balancing Domain Gaps in Object Instance Detection. IEEE Access, 9:14319–14329. ArXiv:1909.11972 [cs]. — [arXiv:1909.11972](https://arxiv.org/abs/1909.11972) [find](https://scholar.google.com/scholar?q=Woo%2Dhan+Yun%2C+Taewoo+Kim%2C+Jaeyeon+Lee%2C+Jaehong+Kim%2C+and+Junmo+Kim%2E+2021%2E+Cut%2Dand%2DPaste+Dataset+Generation+for+Balancing+Domain+Gaps+in+Object+Instance+Detection%2E+IEEE+Access%2C+9%3A14319%E2%80%9314329%2E+ArXiv%3A1909%2E)
54. Xiaohua Zhai, Basil Mustafa, Alexander Kolesnikov, and Lucas Beyer. 2023. [Sigmoid Loss for Language Image Pre-Training](https://md.research.elamur.ai/free/en/papers/zhai-2023-sigmoid-loss-language-image-pre/). ArXiv:2303.15343 [cs].
55. Jiarui Zhang, Mahyar Khayatkhoei, Prateek Chhikara, and Filip Ilievski. 2025a. MLLMs Know Where to Look: Training-free Perception of Small Visual Details with Multimodal LLMs. ArXiv:2502.17422 [cs]. — [arXiv:2502.17422](https://arxiv.org/abs/2502.17422) [find](https://scholar.google.com/scholar?q=Jiarui+Zhang%2C+Mahyar+Khayatkhoei%2C+Prateek+Chhikara%2C+and+Filip+Ilievski%2E+2025a%2E+MLLMs+Know+Where+to+Look%3A+Training%2Dfree+Perception+of+Small+Visual+Details+with+Multimodal+LLMs%2E+ArXiv%3A2502%2E17422+%5Bcs%5D%2E+%E2%80%94)
56. Zhi Zhang, Srishti Yadav, Fengze Han, and Ekaterina Shutova. 2025b. Cross-modal Information Flow in Multimodal Large Language Models. ArXiv:2411.18620 [cs]. — [arXiv:2411.18620](https://arxiv.org/abs/2411.18620) [find](https://scholar.google.com/scholar?q=Zhi+Zhang%2C+Srishti+Yadav%2C+Fengze+Han%2C+and+Ekaterina+Shutova%2E+2025b%2E+Cross%2Dmodal+Information+Flow+in+Multimodal+Large+Language+Models%2E+ArXiv%3A2411%2E18620+%5Bcs%5D%2E+%E2%80%94+%5BarXiv%3A2411%2E18620%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F)
57. Junhao Zhuang, Yanhong Zeng, Wenran Liu, Chun Yuan, and Kai Chen. 2024. A Task is Worth One Word: Learning with Task Prompts for High-Quality Versatile Image Inpainting. ArXiv:2312.03594 [cs]. — [arXiv:2312.03594](https://arxiv.org/abs/2312.03594) [find](https://scholar.google.com/scholar?q=Junhao+Zhuang%2C+Yanhong+Zeng%2C+Wenran+Liu%2C+Chun+Yuan%2C+and+Kai+Chen%2E+2024%2E+A+Task+is+Worth+One+Word%3A+Learning+with+Task+Prompts+for+High%2DQuality+Versatile+Image+Inpainting%2E+ArXiv%3A2312%2E03594+%5Bcs%5D%2E+%E2%80%94+%5BarXi)
58. Pelin Çelikkol, Jochen Laubrock, and David Schlangen. 2023. TF-IDF based Scene-Object Relations Correlate With Visual Attention. In 2023 Symposium on Eye Tracking Research and Applications, pages 1–6, Tubingen Germany. ACM. [find](https://scholar.google.com/scholar?q=Pelin+%C3%87elikkol%2C+Jochen+Laubrock%2C+and+David+Schlangen%2E+2023%2E+TF%2DIDF+based+Scene%2DObject+Relations+Correlate+With+Visual+Attention%2E+In+2023+Symposium+on+Eye+Tracking+Research+and+Applications%2C+pages+1%E2%80%936%2C)
59. Sabine Öhlschläger and Melissa Le-Hoa Võ. 2017. SCEGRAM: An image database for semantic and syntactic inconsistencies in scenes. Behavior Research Methods, 49(5):1780–1791. [find](https://scholar.google.com/scholar?q=Sabine+%C3%96hlschl%C3%A4ger+and+Melissa+Le%2DHoa+V%C3%B5%2E+2017%2E+SCEGRAM%3A+An+image+database+for+semantic+and+syntactic+inconsistencies+in+scenes%2E+Behavior+Research+Methods%2C+49%285%29%3A1780%E2%80%931791%2E)

*Updated 1 November 2026*