---
title: "MMCLIP — paper by Wu et al. (2026) — summary and ideas"
description: "Medical vision-and-language pretraining usually pairs contrastive learning with masked modelling. 72.43 AUC. 53 references."
authors: ["Biao Wu", "Yutong Xie", "Zeyu Zhang", "Vu Minh Hieu Phan", "Qi Chen", "Ling Chen", "Qi Wu"]
year: 2026
venue: "Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)"
source: "https://aclanthology.org/2026.acl-long.113/"
doi: "10.18653/v1/2026.acl-long.113"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/wu-2026-mmclip-cross-modal-attention-masked/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/wu-2026-mmclip-cross-modal-attention-masked/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/6/) / MMCLIP: Cross-Modal Attention Masked Modelling for Medical Language-Image Pre-Training

# MMCLIP: Cross-Modal Attention Masked Modelling for Medical Language-Image Pre-Training

**Authors:** [Biao Wu](https://md.research.elamur.ai/free/en/authors/biao-wu/), [Yutong Xie](https://md.research.elamur.ai/free/en/authors/yutong-xie/), [Zeyu Zhang](https://md.research.elamur.ai/free/en/authors/zeyu-zhang/), Vu Minh Hieu Phan, [Qi Chen](https://md.research.elamur.ai/free/en/authors/qi-chen/), [Ling Chen](https://md.research.elamur.ai/free/en/authors/ling-chen/) and [Qi Wu](https://md.research.elamur.ai/free/en/authors/qi-wu/) · **Year:** 2026 · **Venue:** [Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)](https://md.research.elamur.ai/free/en/venues/acl/) ([ACL 2026](https://md.research.elamur.ai/free/en/venues/acl/2026/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2026.acl-long.113/) · [PDF](https://aclanthology.org/2026.acl-long.113.pdf) · [DOI 10.18653/v1/2026.acl-long.113](https://doi.org/10.18653/v1/2026.acl-long.113) · [OpenAlex](https://openalex.org/W7166870991)

**Citations:** 3 citations (OpenAlex, 2026-10-14)

## Summary
Medical vision-and-language pretraining usually pairs contrastive learning with masked modelling. Random masking, however, tends to miss small pathological details, and most methods train on either paired image–report data or image-only data, not both. MMCLIP addresses this with two masking modules built on CLIP. AttMIM selects image patches to hide by combining three attention signals: image self-attention, image–report cross-attention, and attention to a fixed prompt listing common chest findings. Because the prompt does not need a report, unpaired images can also be used for training. EntMLM uses medical NER to find clinical entities in reports, masks them, and reconstructs them with help from image features. The masked-modelling objectives are warmed up before joint training with the contrastive loss. Pre-training uses MIMIC-CXR (paired) and PadChest (images only). The authors report state-of-the-art zero-shot and fine-tuning results on five classification benchmarks, beating MedKLIP, GLoRIA and CheXzero. This suggests that guided masking and the inclusion of unpaired data both improve medical representation learning.

## Key points
- **Architecture:** ViT image encoder (patch size 16) and 12-layer text encoder with BPE tokens, both initialised from clip-base. The disease prompt covers 14 chest conditions and devices, such as Atelectasis, Edema and Pneumothorax.
- **AttMIM settings:** each attention strategy first masks 70% of features. A random subset of the merged masks is then taken so that 75% of features end up masked.
- **Ablation on a new 2,200-pair validation set (built because the CheXpert validation set is small):**
  - CLIP baseline: 72.43 AUC.
  - Adding random MIM: +0.42 AUC.
  - A 15,000-iteration MIM warm-up: a further ~1.1 AUC.
  - Full AttMIM: 74.96 AUC.
- **EntMLM:** works best when 20% of NER-detected entities are masked, reaching 75.78 AUC and 35.02 F1. This beats both full masking and 50% random masking.
- **Fine-tuning:** with 1%, 10% and 100% of labels, adding PadChest image-only data further improves AUC. The code is released at github.com/AIGeeksGroup/MMCLIP.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Mohamed bin Zayed University of Artificial Intelligence](https://md.research.elamur.ai/free/en/universities/mohamed-bin-zayed-university-of-artificial-intelligence/) — University, Abu Dhabi, United Arab Emirates · 124 works · 5,676 citations
- [University of Technology Sydney](https://md.research.elamur.ai/free/en/universities/university-of-technology-sydney/) — University, Sydney, Australia · 18 works · 12,833 citations
- [Australian National University](https://md.research.elamur.ai/free/en/universities/australian-national-university/) — University, Canberra, Australia · 23 works · 6,522 citations
- [The University of Adelaide](https://md.research.elamur.ai/free/en/universities/the-university-of-adelaide/) — University, Adelaide, Australia · 6 works · 1,417 citations
- [Adelaide University](https://md.research.elamur.ai/free/en/universities/adelaide-university/) — University, Adelaide, Australia · 5 works · 3 citations

📚 Shelves (5): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [👁️ Computer Vision and Pattern Recognition](https://md.research.elamur.ai/free/en/shelves/computer-science/computer-vision-and-pattern-recognition/) · 2,869 works · [🩺 Medicine](https://md.research.elamur.ai/free/en/shelves/medicine/) · 241 works · [🩺 Health Informatics](https://md.research.elamur.ai/free/en/shelves/medicine/health-informatics/) · 184 works

## Cite

Cite as: Biao Wu, Yutong Xie, Zeyu Zhang, Vu Minh Hieu Phan, Qi Chen, Ling Chen and Qi Wu. 2026. MMCLIP: Cross-Modal Attention Masked Modelling for Medical Language-Image Pre-Training. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{wu2026mmclip,
  title = {{MMCLIP: Cross-Modal Attention Masked Modelling for Medical Language-Image Pre-Training}},
  author = {Biao Wu, Yutong Xie, Zeyu Zhang, Vu Minh Hieu Phan, Qi Chen, Ling Chen and Qi Wu},
  booktitle = {Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2026},
  doi = {10.18653/v1/2026.acl-long.113},
  url = {https://doi.org/10.18653/v1/2026.acl-long.113}
}
```

- [← previous Mitigating Catastrophic Forgetting in Target Language Adaptation of LLMs via Source-Shielded Updates](https://md.research.elamur.ai/free/en/papers/yamaguchi-2026-mitigating-catastrophic-forgetting-target-language/)
- [next → MT-OSC: Path for LLMs that Get Lost in Multi-Turn Conversation](https://md.research.elamur.ai/free/en/papers/singh-2026-mt-osc-path-llms-that/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/6/)

## References

53 references, 14 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. et al. Anouk Stein. 2018. Rsna pneumonia detection challenge. [find](https://scholar.google.com/scholar?q=et+al%2E+Anouk+Stein%2E+2018%2E+Rsna+pneumonia+detection+challenge%2E)
2. Muhammad Awais, Muzammal Naseer, Salman Khan, Rao Muhammad Anwer, Hisham Cholakkal, Mubarak Shah, Ming-Hsuan Yang, and Fahad Shahbaz Khan. 2023. Foundational models defining a new era in vision: A survey and outlook. arXiv preprint arXiv:2307.13721. — [arXiv:2307.13721](https://arxiv.org/abs/2307.13721) [find](https://scholar.google.com/scholar?q=Muhammad+Awais%2C+Muzammal+Naseer%2C+Salman+Khan%2C+Rao+Muhammad+Anwer%2C+Hisham+Cholakkal%2C+Mubarak+Shah%2C+Ming%2DHsuan+Yang%2C+and+Fahad+Shahbaz+Khan%2E+2023%2E+Foundational+models+defining+a+new+era+in+vision%3A+A+sur)
3. Shruthi Bannur, Stephanie Hyland, Qianchu Liu, Fernando Perez-Garcia, Maximilian Ilse, Daniel C Castro, Benedikt Boecking, Harshita Sharma, Kenza Bouzid, Anja Thieme, and 1 others. 2023. Learning to exploit temporal structure for biomedical vision-language processing. In Proceedings of the IEEE/CVF… [find](https://scholar.google.com/scholar?q=Shruthi+Bannur%2C+Stephanie+Hyland%2C+Qianchu+Liu%2C+Fernando+Perez%2DGarcia%2C+Maximilian+Ilse%2C+Daniel+C+Castro%2C+Benedikt+Boecking%2C+Harshita+Sharma%2C+Kenza+Bouzid%2C+Anja+Thieme%2C+and+1+others%2E+2023%2E+Learning+to+e)
4. Hangbo Bao, Li Dong, Songhao Piao, and Furu Wei. 2021. Beit: Bert pre-training of image transformers. arXiv preprint arXiv:2106.08254. — [arXiv:2106.08254](https://arxiv.org/abs/2106.08254) [find](https://scholar.google.com/scholar?q=Hangbo+Bao%2C+Li+Dong%2C+Songhao+Piao%2C+and+Furu+Wei%2E+2021%2E+Beit%3A+Bert+pre%2Dtraining+of+image+transformers%2E+arXiv+preprint+arXiv%3A2106%2E08254%2E+%E2%80%94+%5BarXiv%3A2106%2E08254%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2106%2E08254%29)
5. Aurelia Bustos, Antonio Pertusa, Jose-Maria Salinas, and Maria De La Iglesia-Vaya. 2020. Padchest: A large chest x-ray image dataset with multi-label annotated reports. Medical image analysis, 66:101797. [find](https://scholar.google.com/scholar?q=Aurelia+Bustos%2C+Antonio+Pertusa%2C+Jose%2DMaria+Salinas%2C+and+Maria+De+La+Iglesia%2DVaya%2E+2020%2E+Padchest%3A+A+large+chest+x%2Dray+image+dataset+with+multi%2Dlabel+annotated+reports%2E+Medical+image+analysis%2C+66%3A1017)
6. Cheng Chen, Aoxiao Zhong, Dufan Wu, Jie Luo, and Quanzheng Li. 2023. Contrastive masked image-text modeling for medical visual representation learning. In International Conference on Medical Image Computing and Computer-Assisted Intervention, pages 493–503. Springer. [find](https://scholar.google.com/scholar?q=Cheng+Chen%2C+Aoxiao+Zhong%2C+Dufan+Wu%2C+Jie+Luo%2C+and+Quanzheng+Li%2E+2023%2E+Contrastive+masked+image%2Dtext+modeling+for+medical+visual+representation+learning%2E+In+International+Conference+on+Medical+Image+Com)
7. Yen-Chun Chen, Linjie Li, Licheng Yu, Ahmed El Kholy, Faisal Ahmed, Zhe Gan, Yu Cheng, and Jingjing Liu. 2020. [Uniter: Universal image-text representation learning](https://md.research.elamur.ai/free/en/papers/chen-2019-uniter-universal-image-text-representation/). In European conference on computer vision, pages 104–120. Springer.
8. Zhihong Chen, Yuhao Du, Jinpeng Hu, Yang Liu, Guanbin Li, Xiang Wan, and Tsung-Hui Chang. 2022. Multi-modal masked autoencoders for medical vision-and-language pre-training. In International Conference on Medical Image Computing and Computer-Assisted Intervention, pages 679–689. Springer. [find](https://scholar.google.com/scholar?q=Zhihong+Chen%2C+Yuhao+Du%2C+Jinpeng+Hu%2C+Yang+Liu%2C+Guanbin+Li%2C+Xiang+Wan%2C+and+Tsung%2DHui+Chang%2E+2022%2E+Multi%2Dmodal+masked+autoencoders+for+medical+vision%2Dand%2Dlanguage+pre%2Dtraining%2E+In+International+Conferenc)
9. Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, and 1 others. 2020. [An image is worth 16x16 words: Transformers for image recognition at scale](https://md.research.elamur.ai/free/en/papers/dosovitskiy-2020-image-worth-16x16-words-transformers/). In International Conference on…
10. Kaiming He, Xinlei Chen, Saining Xie, Yanghao Li, Piotr Dollár, and Ross Girshick. 2022. [Masked autoencoders are scalable vision learners](https://md.research.elamur.ai/free/en/papers/he-2021-masked-autoencoders-scalable-vision-learners/). In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 16000–16009.
11. Shih-Cheng Huang, Liyue Shen, Matthew P Lungren, and Serena Yeung. 2021. Gloria: A multimodal global-local representation learning framework for label-efficient medical image recognition. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 3942–3951. [find](https://scholar.google.com/scholar?q=Shih%2DCheng+Huang%2C+Liyue+Shen%2C+Matthew+P+Lungren%2C+and+Serena+Yeung%2E+2021%2E+Gloria%3A+A+multimodal+global%2Dlocal+representation+learning+framework+for+label%2Defficient+medical+image+recognition%2E+In+Proceedin)
12. Zhi Huang, Federico Bianchi, Mert Yuksekgonul, Thomas Montine, and James Zou. 2023. Leveraging medical twitter to build a visual–language foundation model for pathology ai. bioRxiv, pages 2023–03. [find](https://scholar.google.com/scholar?q=Zhi+Huang%2C+Federico+Bianchi%2C+Mert+Yuksekgonul%2C+Thomas+Montine%2C+and+James+Zou%2E+2023%2E+Leveraging+medical+twitter+to+build+a+visual%E2%80%93language+foundation+model+for+pathology+ai%2E+bioRxiv%2C+pages+2023%E2%80%9303%2E)
13. Jeremy Irvin, Pranav Rajpurkar, Michael Ko, Yifan Yu, Silviana Ciurea-Ilcus, Chris Chute, Henrik Marklund, Behzad Haghgoo, Robyn Ball, Katie Shpanskaya, and 1 others. 2019. Chexpert: A large chest radiograph dataset with uncertainty labels and expert comparison. In Proceedings of the AAAI conference… [find](https://scholar.google.com/scholar?q=Jeremy+Irvin%2C+Pranav+Rajpurkar%2C+Michael+Ko%2C+Yifan+Yu%2C+Silviana+Ciurea%2DIlcus%2C+Chris+Chute%2C+Henrik+Marklund%2C+Behzad+Haghgoo%2C+Robyn+Ball%2C+Katie+Shpanskaya%2C+and+1+others%2E+2019%2E+Chexpert%3A+A+large+chest+rad)
14. Yiping Ji, Hemanth Saratchandran, Cameron Gordon, Zeyu Zhang, and Simon Lucey. 2024. Sine activated low-rank matrices for parameter efficient learning. arXiv preprint arXiv:2403.19243. — [arXiv:2403.19243](https://arxiv.org/abs/2403.19243) [find](https://scholar.google.com/scholar?q=Yiping+Ji%2C+Hemanth+Saratchandran%2C+Cameron+Gordon%2C+Zeyu+Zhang%2C+and+Simon+Lucey%2E+2024%2E+Sine+activated+low%2Drank+matrices+for+parameter+efficient+learning%2E+arXiv+preprint+arXiv%3A2403%2E19243%2E+%E2%80%94+%5BarXiv%3A2403%2E1)
15. Chao Jia, Yinfei Yang, Ye Xia, Yi-Ting Chen, Zarana Parekh, Hieu Pham, Quoc Le, Yun-Hsuan Sung, Zhen Li, and Tom Duerig. 2021. [Scaling up visual and vision-language representation learning with noisy text supervision](https://md.research.elamur.ai/free/en/papers/jia-2021-scaling-up-visual-vision-language/). In International conference on machine learning, pages 4904–4916. PMLR.
16. Alistair EW Johnson, Tom J Pollard, Seth J Berkowitz, Nathaniel R Greenbaum, Matthew P Lungren, Chih-ying Deng, Roger G Mark, and Steven Horng. 2019. Mimic-cxr, a de-identified publicly available database of chest radiographs with free-text reports. Scientific data, 6(1):317. [find](https://scholar.google.com/scholar?q=Alistair+EW+Johnson%2C+Tom+J+Pollard%2C+Seth+J+Berkowitz%2C+Nathaniel+R+Greenbaum%2C+Matthew+P+Lungren%2C+Chih%2Dying+Deng%2C+Roger+G+Mark%2C+and+Steven+Horng%2E+2019%2E+Mimic%2Dcxr%2C+a+de%2Didentified+publicly+available+data)
17. Ioannis Kakogeorgiou, Spyros Gidaris, Bill Psomas, Yannis Avrithis, Andrei Bursuc, Konstantinos Karantzalos, and Nikos Komodakis. 2022. What to hide from your students: Attention-guided masked image modeling. In European Conference on Computer Vision, pages 300–318. Springer. [find](https://scholar.google.com/scholar?q=Ioannis+Kakogeorgiou%2C+Spyros+Gidaris%2C+Bill+Psomas%2C+Yannis+Avrithis%2C+Andrei+Bursuc%2C+Konstantinos+Karantzalos%2C+and+Nikos+Komodakis%2E+2022%2E+What+to+hide+from+your+students%3A+Attention%2Dguided+masked+image+m)
18. Jacob Devlin Ming-Wei Chang Kenton and Lee Kristina Toutanova. 2019. [Bert: Pre-training of deep bidirectional transformers for language understanding](https://md.research.elamur.ai/free/en/papers/devlin-2019-bert-pre-training-deep-bidirectional/). In Proceedings of NAACL-HLT, pages 4171–4186.
19. Wonjae Kim, Bokyung Son, and Ildoo Kim. 2021. [Vilt: Vision-and-language transformer without convolution or region supervision](https://md.research.elamur.ai/free/en/papers/kim-2021-vilt-vision-language-transformer-convolution/). In International Conference on Machine Learning, pages 5583–5594. PMLR.
20. Gukyeong Kwon, Zhaowei Cai, Avinash Ravichandran, Erhan Bas, Rahul Bhotika, and Stefano Soatto. 2022. Masked vision and language modeling for multi-modal representation learning. arXiv preprint arXiv:2208.02131. — [arXiv:2208.02131](https://arxiv.org/abs/2208.02131) [find](https://scholar.google.com/scholar?q=Gukyeong+Kwon%2C+Zhaowei+Cai%2C+Avinash+Ravichandran%2C+Erhan+Bas%2C+Rahul+Bhotika%2C+and+Stefano+Soatto%2E+2022%2E+Masked+vision+and+language+modeling+for+multi%2Dmodal+representation+learning%2E+arXiv+preprint+arXiv%3A)
21. Junnan Li, Ramprasaath Selvaraju, Akhilesh Gotmare, Shafiq Joty, Caiming Xiong, and Steven Chu Hong Hoi. 2021. Align before fuse: Vision and language representation learning with momentum distillation. Advances in neural information processing systems, 34:9694–9705. [find](https://scholar.google.com/scholar?q=Junnan+Li%2C+Ramprasaath+Selvaraju%2C+Akhilesh+Gotmare%2C+Shafiq+Joty%2C+Caiming+Xiong%2C+and+Steven+Chu+Hong+Hoi%2E+2021%2E+Align+before+fuse%3A+Vision+and+language+representation+learning+with+momentum+distillation)
22. Liunian Harold Li, Mark Yatskar, Da Yin, Cho-Jui Hsieh, and Kai-Wei Chang. 2019. [Visualbert: A simple and performant baseline for vision and language](https://md.research.elamur.ai/free/en/papers/li-2019-visualbert-simple-performant-baseline-vision/). arXiv preprint arXiv:1908.03557.
23. Xiujun Li, Xi Yin, Chunyuan Li, Pengchuan Zhang, Xiaowei Hu, Lei Zhang, Lijuan Wang, Houdong Hu, Li Dong, Furu Wei, and 1 others. 2020. [Oscar: Object-semantics aligned pre-training for vision-language tasks](https://md.research.elamur.ai/free/en/papers/li-2020-oscar-object-semantics-aligned-pre/). In Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23– 28, 2020, Pr…
24. Yanghao Li, Haoqi Fan, Ronghang Hu, Christoph Feichtenhofer, and Kaiming He. 2023. Scaling language-image pre-training via masking. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 23390–23400. [find](https://scholar.google.com/scholar?q=Yanghao+Li%2C+Haoqi+Fan%2C+Ronghang+Hu%2C+Christoph+Feichtenhofer%2C+and+Kaiming+He%2E+2023%2E+Scaling+language%2Dimage+pre%2Dtraining+via+masking%2E+In+Proceedings+of+the+IEEE%2FCVF+Conference+on+Computer+Vision+and+Pat)
25. Weixiong Lin, Ziheng Zhao, Xiaoman Zhang, Chaoyi Wu, Ya Zhang, Yanfeng Wang, and Weidi Xie. 2023. Pmc-clip: Contrastive language-image pre-training using biomedical documents. arXiv preprint arXiv:2303.07240. — [arXiv:2303.07240](https://arxiv.org/abs/2303.07240) [find](https://scholar.google.com/scholar?q=Weixiong+Lin%2C+Ziheng+Zhao%2C+Xiaoman+Zhang%2C+Chaoyi+Wu%2C+Ya+Zhang%2C+Yanfeng+Wang%2C+and+Weidi+Xie%2E+2023%2E+Pmc%2Dclip%3A+Contrastive+language%2Dimage+pre%2Dtraining+using+biomedical+documents%2E+arXiv+preprint+arXiv%3A230)
26. Xiaoxuan Liu, Livia Faes, Aditya U Kale, Siegfried K Wagner, Dun Jack Fu, Alice Bruynseels, Thushika Mahendiran, Gabriella Moraes, Mohith Shamdas, Christoph Kern, and 1 others. 2019. A comparison of deep learning performance against health-care professionals in detecting diseases from medical imagin… [find](https://scholar.google.com/scholar?q=Xiaoxuan+Liu%2C+Livia+Faes%2C+Aditya+U+Kale%2C+Siegfried+K+Wagner%2C+Dun+Jack+Fu%2C+Alice+Bruynseels%2C+Thushika+Mahendiran%2C+Gabriella+Moraes%2C+Mohith+Shamdas%2C+Christoph+Kern%2C+and+1+others%2E+2019%2E+A+comparison+of+d)
27. Jiasen Lu, Dhruv Batra, Devi Parikh, and Stefan Lee. 2019. [Vilbert: Pretraining task-agnostic visiolinguistic representations for vision-and-language tasks](https://md.research.elamur.ai/free/en/papers/lu-2019-vilbert-pretraining-task-agnostic-visiolinguistic/). Advances in neural information processing systems, 32.
28. Norman Mu, Alexander Kirillov, David Wagner, and Saining Xie. 2022. Slip: Self-supervision meets language-image pre-training. In European Conference on Computer Vision, pages 529–544. Springer. [find](https://scholar.google.com/scholar?q=Norman+Mu%2C+Alexander+Kirillov%2C+David+Wagner%2C+and+Saining+Xie%2E+2022%2E+Slip%3A+Self%2Dsupervision+meets+language%2Dimage+pre%2Dtraining%2E+In+European+Conference+on+Computer+Vision%2C+pages+529%E2%80%93544%2E+Springer%2E)
29. Philip Müller, Georgios Kaissis, Congyu Zou, and Daniel Rueckert. 2022. Radiological reports improve pre-training for localized imaging tasks on chest x-rays. In International Conference on Medical Image Computing and Computer-Assisted Intervention, pages 647–657. Springer. [find](https://scholar.google.com/scholar?q=Philip+M%C3%BCller%2C+Georgios+Kaissis%2C+Congyu+Zou%2C+and+Daniel+Rueckert%2E+2022%2E+Radiological+reports+improve+pre%2Dtraining+for+localized+imaging+tasks+on+chest+x%2Drays%2E+In+International+Conference+on+Medical+Im)
30. American College of Radiology, American Association of Physicists in Medicine (AAPM), and 1 others. 2014. Society for imaging informatics in medicine (siim). Practice guideline for digital radiography (Resolution 42, adopted in 2007)[visionato il 4 dicembre 2012]. Disponibile su: www. siimweb. org. [find](https://scholar.google.com/scholar?q=American+College+of+Radiology%2C+American+Association+of+Physicists+in+Medicine+%28AAPM%29%2C+and+1+others%2E+2014%2E+Society+for+imaging+informatics+in+medicine+%28siim%29%2E+Practice+guideline+for+digital+radiography)
31. Maya Pavlova, Naomi Terhljan, Audrey G Chung, Andy Zhao, Siddharth Surana, Hossein Aboutalebi, Hayden Gunraj, Ali Sabri, Amer Alaref, and Alexander Wong. 2022. Covid-net cxr-2: An enhanced deep convolutional neural network design for detection of covid-19 cases from chest x-ray images. Frontiers in… [find](https://scholar.google.com/scholar?q=Maya+Pavlova%2C+Naomi+Terhljan%2C+Audrey+G+Chung%2C+Andy+Zhao%2C+Siddharth+Surana%2C+Hossein+Aboutalebi%2C+Hayden+Gunraj%2C+Ali+Sabri%2C+Amer+Alaref%2C+and+Alexander+Wong%2E+2022%2E+Covid%2Dnet+cxr%2D2%3A+An+enhanced+deep+convol)
32. Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, and 1 others. 2021. [Learning transferable visual models from natural language supervision](https://md.research.elamur.ai/free/en/papers/radford-2021-learning-transferable-visual-models-natural/). In International conference on machine learning, pages 8748–8…
33. Weijie Su, Xizhou Zhu, Yue Cao, Bin Li, Lewei Lu, Furu Wei, and Jifeng Dai. 2019. [Vl-bert: Pre-training of generic visual-linguistic representations](https://md.research.elamur.ai/free/en/papers/su-2019-vl-bert-pre-training-generic/). arXiv preprint arXiv:1908.08530.
34. Quan Sun, Yuxin Fang, Ledell Wu, Xinlong Wang, and Yue Cao. 2023. [Eva-clip: Improved training techniques for clip at scale](https://md.research.elamur.ai/free/en/papers/sun-2023-eva-clip-improved-training-techniques/). arXiv preprint arXiv:2303.15389.
35. Hao Tan and Mohit Bansal. 2019. [Lxmert: Learning cross-modality encoder representations from transformers](https://md.research.elamur.ai/free/en/papers/tan-2019-lxmert-learning-cross-modality-encoder/). arXiv preprint arXiv:1908.07490.
36. Ekin Tiu, Ellie Talius, Pujan Patel, Curtis P Langlotz, Andrew Y Ng, and Pranav Rajpurkar. 2022. Expert-level detection of pathologies from unannotated chest x-ray images via self-supervised learning. Nature Biomedical Engineering, 6(12):1399–1406. [find](https://scholar.google.com/scholar?q=Ekin+Tiu%2C+Ellie+Talius%2C+Pujan+Patel%2C+Curtis+P+Langlotz%2C+Andrew+Y+Ng%2C+and+Pranav+Rajpurkar%2E+2022%2E+Expert%2Dlevel+detection+of+pathologies+from+unannotated+chest+x%2Dray+images+via+self%2Dsupervised+learning%2E)
37. Haochen Wang, Kaiyou Song, Junsong Fan, Yuxi Wang, Jin Xie, and Zhaoxiang Zhang. 2023. Hard patches mining for masked image modeling. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10375–10385. [find](https://scholar.google.com/scholar?q=Haochen+Wang%2C+Kaiyou+Song%2C+Junsong+Fan%2C+Yuxi+Wang%2C+Jin+Xie%2C+and+Zhaoxiang+Zhang%2E+2023%2E+Hard+patches+mining+for+masked+image+modeling%2E+In+Proceedings+of+the+IEEE%2FCVF+Conference+on+Computer+Vision+and+P)
38. Xiaosong Wang, Yifan Peng, Le Lu, Zhiyong Lu, Mo-hammadhadi Bagheri, and Ronald M Summers. 2017. Chestx-ray8: Hospital-scale chest x-ray database and benchmarks on weakly-supervised classification and localization of common thorax diseases. In Proceedings of the IEEE conference on computer vision an… [find](https://scholar.google.com/scholar?q=Xiaosong+Wang%2C+Yifan+Peng%2C+Le+Lu%2C+Zhiyong+Lu%2C+Mo%2Dhammadhadi+Bagheri%2C+and+Ronald+M+Summers%2E+2017%2E+Chestx%2Dray8%3A+Hospital%2Dscale+chest+x%2Dray+database+and+benchmarks+on+weakly%2Dsupervised+classification+and)
39. Zifeng Wang, Zhenbang Wu, Dinesh Agarwal, and Jimeng Sun. 2022. Medclip: Contrastive learning from unpaired medical images and text. arXiv preprint arXiv:2210.10163. [find](https://scholar.google.com/scholar?q=Zifeng+Wang%2C+Zhenbang+Wu%2C+Dinesh+Agarwal%2C+and+Jimeng+Sun%2E+2022%2E+Medclip%3A+Contrastive+learning+from+unpaired+medical+images+and+text%2E+arXiv+preprint+arXiv%3A2210%2E10163%2E)
40. Biao Wu, Yutong Xie, Zeyu Zhang, Jinchao Ge, Kaspar Yaxley, Suzan Bahadir, Qi Wu, Yifan Liu, and Minh-Son To. 2023a. Bhsd: A 3d multi-class brain hemorrhage segmentation dataset. In International Workshop on Machine Learning in Medical Imaging, pages 147–156. Springer. [find](https://scholar.google.com/scholar?q=Biao+Wu%2C+Yutong+Xie%2C+Zeyu+Zhang%2C+Jinchao+Ge%2C+Kaspar+Yaxley%2C+Suzan+Bahadir%2C+Qi+Wu%2C+Yifan+Liu%2C+and+Minh%2DSon+To%2E+2023a%2E+Bhsd%3A+A+3d+multi%2Dclass+brain+hemorrhage+segmentation+dataset%2E+In+International+Work)
41. Chaoyi Wu, Xiaoman Zhang, Ya Zhang, Yanfeng Wang, and Weidi Xie. 2023b. Medklip: Medical knowledge enhanced language-image pre-training. medRxiv, pages 2023–01. [find](https://scholar.google.com/scholar?q=Chaoyi+Wu%2C+Xiaoman+Zhang%2C+Ya+Zhang%2C+Yanfeng+Wang%2C+and+Weidi+Xie%2E+2023b%2E+Medklip%3A+Medical+knowledge+enhanced+language%2Dimage+pre%2Dtraining%2E+medRxiv%2C+pages+2023%E2%80%9301%2E)
42. Yutong Xie, Lin Gu, Tatsuya Harada, Jianpeng Zhang, Yong Xia, and Qi Wu. 2023. Medim: Boost medical image representation via radiology report-guided masking. In International Conference on Medical Image Computing and Computer-Assisted Intervention, pages 13–23. Springer. [find](https://scholar.google.com/scholar?q=Yutong+Xie%2C+Lin+Gu%2C+Tatsuya+Harada%2C+Jianpeng+Zhang%2C+Yong+Xia%2C+and+Qi+Wu%2E+2023%2E+Medim%3A+Boost+medical+image+representation+via+radiology+report%2Dguided+masking%2E+In+International+Conference+on+Medical+Ima)
43. Yutong Xie, Jianpeng Zhang, Yong Xia, and Qi Wu. 2022a. Unimiss: Universal medical self-supervised learning via breaking dimensionality barrier. In European Conference on Computer Vision, pages 558– 575. Springer. [find](https://scholar.google.com/scholar?q=Yutong+Xie%2C+Jianpeng+Zhang%2C+Yong+Xia%2C+and+Qi+Wu%2E+2022a%2E+Unimiss%3A+Universal+medical+self%2Dsupervised+learning+via+breaking+dimensionality+barrier%2E+In+European+Conference+on+Computer+Vision%2C+pages+558%E2%80%93+5)
44. Hong-Yu Zhou, Xiaoyu Chen, Yinghao Zhang, Ruibang Luo, Liansheng Wang, and Yizhou Yu. 2022. Generalized radiograph representation learning via cross-supervision between images and free-text radiology reports. Nature Machine Intelligence, 4(1):32–40. [find](https://scholar.google.com/scholar?q=Hong%2DYu+Zhou%2C+Xiaoyu+Chen%2C+Yinghao+Zhang%2C+Ruibang+Luo%2C+Liansheng+Wang%2C+and+Yizhou+Yu%2E+2022%2E+Generalized+radiograph+representation+learning+via+cross%2Dsupervision+between+images+and+free%2Dtext+radiology+)
45. Jinghao Zhou, Chen Wei, Huiyu Wang, Wei Shen, Cihang Xie, Alan Yuille, and Tao Kong. 2021. ibot: Image bert pre-training with online tokenizer. arXiv preprint arXiv:2111.07832. — [arXiv:2111.07832](https://arxiv.org/abs/2111.07832) [find](https://scholar.google.com/scholar?q=Jinghao+Zhou%2C+Chen+Wei%2C+Huiyu+Wang%2C+Wei+Shen%2C+Cihang+Xie%2C+Alan+Yuille%2C+and+Tao+Kong%2E+2021%2E+ibot%3A+Image+bert+pre%2Dtraining+with+online+tokenizer%2E+arXiv+preprint+arXiv%3A2111%2E07832%2E+%E2%80%94+%5BarXiv%3A2111%2E07832%5D%28ht)
46. Zhenda Xie, Zheng Zhang, Yue Cao, Yutong Lin, Jianmin Bao, Zhuliang Yao, Qi Dai, and Han Hu. 2022b. Simmim: A simple framework for masked image modeling. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 9653–9663. [find](https://scholar.google.com/scholar?q=Zhenda+Xie%2C+Zheng+Zhang%2C+Yue+Cao%2C+Yutong+Lin%2C+Jianmin+Bao%2C+Zhuliang+Yao%2C+Qi+Dai%2C+and+Han+Hu%2E+2022b%2E+Simmim%3A+A+simple+framework+for+masked+image+modeling%2E+In+Proceedings+of+the+IEEE%2FCVF+Conference+on+C)
47. Lewei Yao, Runhui Huang, Lu Hou, Guansong Lu, Minzhe Niu, Hang Xu, Xiaodan Liang, Zhenguo Li, Xin Jiang, and Chunjing Xu. 2021. Filip: Fine-grained interactive language-image pre-training. arXiv preprint arXiv:2111.07783. — [arXiv:2111.07783](https://arxiv.org/abs/2111.07783) [find](https://scholar.google.com/scholar?q=Lewei+Yao%2C+Runhui+Huang%2C+Lu+Hou%2C+Guansong+Lu%2C+Minzhe+Niu%2C+Hang+Xu%2C+Xiaodan+Liang%2C+Zhenguo+Li%2C+Xin+Jiang%2C+and+Chunjing+Xu%2E+2021%2E+Filip%3A+Fine%2Dgrained+interactive+language%2Dimage+pre%2Dtraining%2E+arXiv+prepr)
48. Fei Yu, Jiji Tang, Weichong Yin, Yu Sun, Hao Tian, Hua Wu, and Haifeng Wang. 2021. Ernie-vil: Knowledge enhanced vision-language representations through scene graphs. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pages 3208– 3216. [find](https://scholar.google.com/scholar?q=Fei+Yu%2C+Jiji+Tang%2C+Weichong+Yin%2C+Yu+Sun%2C+Hao+Tian%2C+Hua+Wu%2C+and+Haifeng+Wang%2E+2021%2E+Ernie%2Dvil%3A+Knowledge+enhanced+vision%2Dlanguage+representations+through+scene+graphs%2E+In+Proceedings+of+the+AAAI+Confer)
49. Jiahui Yu, Zirui Wang, Vijay Vasudevan, Legg Yeung, Mojtaba Seyedhosseini, and Yonghui Wu. 2022. [Coca: Contrastive captioners are image-text foundation models](https://md.research.elamur.ai/free/en/papers/yu-2022-coca-contrastive-captioners-image-text/). arXiv preprint arXiv:2205.01917.
50. Yuhao Zhang, Hang Jiang, Yasuhide Miura, Christopher D Manning, and Curtis P Langlotz. 2022. Contrastive learning of medical visual representations from paired images and text. In Machine Learning for Healthcare Conference, pages 2–25. PMLR. [find](https://scholar.google.com/scholar?q=Yuhao+Zhang%2C+Hang+Jiang%2C+Yasuhide+Miura%2C+Christopher+D+Manning%2C+and+Curtis+P+Langlotz%2E+2022%2E+Contrastive+learning+of+medical+visual+representations+from+paired+images+and+text%2E+In+Machine+Learning+for)
51. Zeyu Zhang, Xuyin Qi, Mingxi Chen, Guangxi Li, Ryan Pham, Ayub Zuhair, Ella Berry, Zhibin Liao, Owen Siggs, Robert Mclaughlin, and 1 others. 2024. Jointvit: Modeling oxygen saturation levels with joint supervision on long-tailed octa. arXiv preprint arXiv:2404.11525. — [arXiv:2404.11525](https://arxiv.org/abs/2404.11525) [find](https://scholar.google.com/scholar?q=Zeyu+Zhang%2C+Xuyin+Qi%2C+Mingxi+Chen%2C+Guangxi+Li%2C+Ryan+Pham%2C+Ayub+Zuhair%2C+Ella+Berry%2C+Zhibin+Liao%2C+Owen+Siggs%2C+Robert+Mclaughlin%2C+and+1+others%2E+2024%2E+Jointvit%3A+Modeling+oxygen+saturation+levels+with+join)
52. Zeyu Zhang, Xuyin Qi, Bowen Zhang, Biao Wu, Hien Le, Bora Jeong, Minh-Son To, and Richard Hartley. 2023a. Segreg: Segmenting oars by registering mr images and ct annotations. arXiv preprint arXiv:2311.06956. — [arXiv:2311.06956](https://arxiv.org/abs/2311.06956) [find](https://scholar.google.com/scholar?q=Zeyu+Zhang%2C+Xuyin+Qi%2C+Bowen+Zhang%2C+Biao+Wu%2C+Hien+Le%2C+Bora+Jeong%2C+Minh%2DSon+To%2C+and+Richard+Hartley%2E+2023a%2E+Segreg%3A+Segmenting+oars+by+registering+mr+images+and+ct+annotations%2E+arXiv+preprint+arXiv%3A2311)
53. Zeyu Zhang, Bowen Zhang, Abhiram Hiwase, Christen Barras, Feng Chen, Biao Wu, Adam James Wells, Daniel Y Ellis, Benjamin Reddi, Andrew William Burgan, Minh-Son To, Ian Reid, and Richard Hartley. 2023b. Thin-thick adapter: Segmenting thin scans using thick annotations. OpenReview. [find](https://scholar.google.com/scholar?q=Zeyu+Zhang%2C+Bowen+Zhang%2C+Abhiram+Hiwase%2C+Christen+Barras%2C+Feng+Chen%2C+Biao+Wu%2C+Adam+James+Wells%2C+Daniel+Y+Ellis%2C+Benjamin+Reddi%2C+Andrew+William+Burgan%2C+Minh%2DSon+To%2C+Ian+Reid%2C+and+Richard+Hartley%2E+2023b)

*Updated 1 November 2026*