---
title: "Multilingual-To-Multimodal (M2M) — paper by Pasi (2026)"
description: "The author releases code, checkpoints and three machine-translated evaluation sets. 67 references."
authors: ["Piyush Singh Pasi"]
year: 2026
venue: "Findings of the Association for Computational Linguistics: EACL 2026"
source: "https://aclanthology.org/2026.findings-eacl.143/"
doi: "10.18653/v1/2026.findings-eacl.143"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/pasi-2026-multilingual-multimodal-m2m-unlocking-new/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/pasi-2026-multilingual-multimodal-m2m-unlocking-new/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/37/) / Multilingual-To-Multimodal (M2M): Unlocking New Languages with Monolingual Text

# Multilingual-To-Multimodal (M2M): Unlocking New Languages with Monolingual Text

**Authors:** Piyush Singh Pasi · **Year:** 2026 · **Venue:** [Findings of the Association for Computational Linguistics: EACL 2026](https://md.research.elamur.ai/free/en/venues/findings/) ([Findings of the Association for Computational Linguistics 2026](https://md.research.elamur.ai/free/en/venues/findings/2026/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2026.findings-eacl.143/) · [PDF](https://aclanthology.org/2026.findings-eacl.143.pdf) · [DOI 10.18653/v1/2026.findings-eacl.143](https://doi.org/10.18653/v1/2026.findings-eacl.143) · [OpenAlex](https://openalex.org/W7140133877)

**Citations:** 0 citations (OpenAlex, 2026-10-14)

## Summary
Image–text and audio–text models such as CLIP and CLAP work well in English but degrade in other languages, because paired multimodal data outside English is scarce. Existing fixes usually depend on machine translation or multilingual multimodal fine-tuning. M2M instead links a frozen multilingual text encoder to a frozen multimodal text space with a small stack of linear layers trained only on English captions. Since the same English sentence passes through both encoders, English serves as the bridge between them. Training minimizes MSE between projected and target embeddings, plus a term that keeps in-batch pairwise cosine similarities consistent. Other languages then transfer zero-shot, with no non-English image or audio pairs ever seen. On XTD text-to-image retrieval, the mapped model equals the English baseline and stays close to it across ten unseen languages. The approach also extends to audio–text retrieval and to text-to-image generation with FLUX. The author releases code, checkpoints and three machine-translated evaluation sets. The work shows that cheap latent-space alignment can recover much of the benefit of costly multilingual multimodal pretraining.

## Key points
- Main setup: Jina-CLIP-v1 as the multimodal model and M-MPNET as the multilingual encoder. The mapping has roughly 1–2M parameters, is trained on English captions from GCC, MSCOCO and VizWiz, and stays effective with about 1K sentences.
- XTD Text-to-Image Recall@10: 94.9% in English and 89.5% averaged over 11 languages, 10 of them unseen. In ablations, MSE plus the structure loss beat L1 and cosine-similarity objectives.
- Released datasets on HF:
  - AudioCaps Multilingual (160K samples) and Clotho Multilingual (172K samples), covering 33 languages.
  - MSCOCO Multilingual 30K (270K samples) in 9 extra languages, including ru, hi, fa and ko; Ukrainian is not among them.
  - Translations were made with IndicTrans2 and Aya-23-35B, checked with spBLEU and chrF++, and not human-verified.
- Text-to-image generation with FLUX×M-MPNET reaches an average Inception Score of 31.81 (35.9 in English), but FID is weaker: 43.4 versus 23.4 for stock FLUX. Outputs are sometimes less faithful to the prompt.
- Limitations: alignment works only at sentence level, so it does not yet suit MLLMs needing token-level signals. Joint cross-modal encoders were not tested.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

📚 Shelves (5): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works · [👁️ Computer Vision and Pattern Recognition](https://md.research.elamur.ai/free/en/shelves/computer-science/computer-vision-and-pattern-recognition/) · 2,869 works

🧭 Topics (2): [🌍 Multilingual & cross-lingual NLP](https://md.research.elamur.ai/free/en/topics/multilingual-nlp/) · 828 works · [🖼️ Vision & language](https://md.research.elamur.ai/free/en/topics/multimodal/) · 657 works

## Cite

Cite as: Piyush Singh Pasi. 2026. Multilingual-To-Multimodal (M2M): Unlocking New Languages with Monolingual Text. Findings of the Association for Computational Linguistics: EACL 2026.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{pasi2026multilingual,
  title = {{Multilingual-To-Multimodal (M2M): Unlocking New Languages with Monolingual Text}},
  author = {Piyush Singh Pasi},
  booktitle = {Findings of the Association for Computational Linguistics: EACL 2026},
  year = {2026},
  doi = {10.18653/v1/2026.findings-eacl.143},
  url = {https://doi.org/10.18653/v1/2026.findings-eacl.143}
}
```

- [← previous Multilingual Target-Stance Extraction](https://md.research.elamur.ai/free/en/papers/mines-2026-multilingual-target-stance-extraction/)
- [next → Multimodal Analysis of State-Funded News Coverage of the Israel–Hamas War on YouTube Shorts](https://md.research.elamur.ai/free/en/papers/miehling-2026-multimodal-analysis-state-funded-news/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/37/)

## References

67 references, 23 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Pranav Aggarwal and Ajinkya Kale. 2020. Towards zero-shot cross-lingual image retrieval. arXiv preprint arXiv:2012.05107. — [arXiv:2012.05107](https://arxiv.org/abs/2012.05107) [find](https://scholar.google.com/scholar?q=Pranav+Aggarwal+and+Ajinkya+Kale%2E+2020%2E+Towards+zero%2Dshot+cross%2Dlingual+image+retrieval%2E+arXiv+preprint+arXiv%3A2012%2E05107%2E+%E2%80%94+%5BarXiv%3A2012%2E05107%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2012%2E05107%29)
2. Nahid Alam, Karthik Reddy Kanjula, Surya Guthikonda, Timothy Chung, Bala Krishna S Vegesna, Abhipsha Das, Anthony Susevski, Ryan Sze-Yin Chan, SM Uddin, Shayekh Bin Islam, et al. 2024. Maya: An instruction finetuned multilingual multimodal model. arXiv preprint arXiv:2412.07112. — [arXiv:2412.07112](https://arxiv.org/abs/2412.07112) [find](https://scholar.google.com/scholar?q=Nahid+Alam%2C+Karthik+Reddy+Kanjula%2C+Surya+Guthikonda%2C+Timothy+Chung%2C+Bala+Krishna+S+Vegesna%2C+Abhipsha+Das%2C+Anthony+Susevski%2C+Ryan+Sze%2DYin+Chan%2C+SM+Uddin%2C+Shayekh+Bin+Islam%2C+et+al%2E+2024%2E+Maya%3A+An+instru)
3. Viraat Aryabumi, John Dang, Dwarak Talupuru, Saurabh Dash, David Cairuz, Hangyu Lin, Bharat Venkitesh, Madeline Smith, Kelly Marchisio, Sebastian Ruder, Acyr F. Locatelli, Julia Kreutzer, Nick Frosst, Phil Blunsom, Marzieh Fadaee, A. Ustun, and Sara Hooker. 2024. [Aya 23: Open weight releases to furt…](https://md.research.elamur.ai/free/en/papers/aryabumi-2024-aya-23-open-weight-releases/)
4. Loïc Barrault, Fethi Bougares, Lucia Specia, Chiraag Lala, Desmond Elliott, and Stella Frank. 2018. [Findings of the third shared task on multimodal machine translation](https://md.research.elamur.ai/free/en/papers/barrault-2018-findings-third-shared-task-multimodal/). In Proceedings of the Third Conference on Machine Translation: Shared Task Papers, pages 304–323.
5. Jeffrey P. Bigham, Chandrika Jayant, Hanjie Ji, Greg Little, Andrew Miller, Rob Miller, Robin Miller, Aubrey Tatarowicz, Brandyn Allen White, Samuel White, and Tom Yeh. 2010. Vizwiz: nearly real-time answers to visual questions. Proceedings of the 23nd annual ACM symposium on User interface software… [find](https://scholar.google.com/scholar?q=Jeffrey+P%2E+Bigham%2C+Chandrika+Jayant%2C+Hanjie+Ji%2C+Greg+Little%2C+Andrew+Miller%2C+Rob+Miller%2C+Robin+Miller%2C+Aubrey+Tatarowicz%2C+Brandyn+Allen+White%2C+Samuel+White%2C+and+Tom+Yeh%2E+2010%2E+Vizwiz%3A+nearly+real%2Dtime+)
6. Fredrik Carlsson, Philipp Eisen, Faton Rekathati, and Magnus Sahlgren. 2022. Cross-lingual and multilingual CLIP. In Proceedings of the Thirteenth Language Resources and Evaluation Conference, pages 6848–6854, Marseille, France. European Language Resources Association. [find](https://scholar.google.com/scholar?q=Fredrik+Carlsson%2C+Philipp+Eisen%2C+Faton+Rekathati%2C+and+Magnus+Sahlgren%2E+2022%2E+Cross%2Dlingual+and+multilingual+CLIP%2E+In+Proceedings+of+the+Thirteenth+Language+Resources+and+Evaluation+Conference%2C+pages+6)
7. Guanhua Chen, Lu Hou, Yun Chen, Wenliang Dai, Lifeng Shang, Xin Jiang, Qun Liu, Jia Pan, and Wenping Wang. 2023. mCLIP: Multilingual CLIP via cross-lingual transfer. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 13028– 13043… [find](https://scholar.google.com/scholar?q=Guanhua+Chen%2C+Lu+Hou%2C+Yun+Chen%2C+Wenliang+Dai%2C+Lifeng+Shang%2C+Xin+Jiang%2C+Qun+Liu%2C+Jia+Pan%2C+and+Wenping+Wang%2E+2023%2E+mCLIP%3A+Multilingual+CLIP+via+cross%2Dlingual+transfer%2E+In+Proceedings+of+the+61st+Annual+)
8. Zhongzhi Chen, Guangyi Liu, Bo-Wen Zhang, Fulong Ye, Qinghong Yang, and Ledell Yu Wu. 2022. Altclip: Altering the language encoder in clip for extended language capabilities. ArXiv, abs/2211.06679. [find](https://scholar.google.com/scholar?q=Zhongzhi+Chen%2C+Guangyi+Liu%2C+Bo%2DWen+Zhang%2C+Fulong+Ye%2C+Qinghong+Yang%2C+and+Ledell+Yu+Wu%2E+2022%2E+Altclip%3A+Altering+the+language+encoder+in+clip+for+extended+language+capabilities%2E+ArXiv%2C+abs%2F2211%2E06679%2E)
9. Marta R Costa-jussà, James Cross, Onur Çelebi, Maha Elbayad, Kenneth Heafield, Kevin Heffernan, Elahe Kalbassi, Janice Lam, Daniel Licht, Jean Maillard, et al. 2022. No language left behind: Scaling human-centered machine translation. arXiv preprint arXiv:2207.04672. [find](https://scholar.google.com/scholar?q=Marta+R+Costa%2Djuss%C3%A0%2C+James+Cross%2C+Onur+%C3%87elebi%2C+Maha+Elbayad%2C+Kenneth+Heafield%2C+Kevin+Heffernan%2C+Elahe+Kalbassi%2C+Janice+Lam%2C+Daniel+Licht%2C+Jean+Maillard%2C+et+al%2E+2022%2E+No+language+left+behind%3A+Scaling+h)
10. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. [BERT: Pre-training of deep bidirectional transformers for language understanding](https://md.research.elamur.ai/free/en/papers/devlin-2019-bert-pre-training-deep-bidirectional/). In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies,…
11. Ming Ding, Zhuoyi Yang, Wenyi Hong, Wendi Zheng, Chang Zhou, Da Yin, Junyang Lin, Xu Zou, Zhou Shao, Hongxia Yang, et al. 2021. [Cogview: Mastering text-to-image generation via transformers](https://md.research.elamur.ai/free/en/papers/ding-2021-cogview-mastering-text-image/). Advances in neural information processing systems, 34:19822–19835.
12. Konstantinos Drossos, Samuel Lipping, and Tuomas Virtanen. 2019. Clotho: an audio captioning dataset. [find](https://scholar.google.com/scholar?q=Konstantinos+Drossos%2C+Samuel+Lipping%2C+and+Tuomas+Virtanen%2E+2019%2E+Clotho%3A+an+audio+captioning+dataset%2E)
13. ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 736–740. [find](https://scholar.google.com/scholar?q=ICASSP+2020+%2D+2020+IEEE+International+Conference+on+Acoustics%2C+Speech+and+Signal+Processing+%28ICASSP%29%2C+pages+736%E2%80%93740%2E)
14. Benjamin Elizalde, Soham Deshmukh, Mahmoud Al Ismail, and Huaming Wang. 2023. Clap learning audio concepts from natural language supervision. [find](https://scholar.google.com/scholar?q=Benjamin+Elizalde%2C+Soham+Deshmukh%2C+Mahmoud+Al+Ismail%2C+and+Huaming+Wang%2E+2023%2E+Clap+learning+audio+concepts+from+natural+language+supervision%2E)
15. ICASSP 2023 - 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 1–5. [find](https://scholar.google.com/scholar?q=ICASSP+2023+%2D+2023+IEEE+International+Conference+on+Acoustics%2C+Speech+and+Signal+Processing+%28ICASSP%29%2C+pages+1%E2%80%935%2E)
16. Desmond Elliott, Stella Frank, Loïc Barrault, Fethi Bougares, and Lucia Specia. 2017. [Findings of the second shared task on multimodal machine translation and multilingual image description](https://md.research.elamur.ai/free/en/papers/elliott-2017-findings-second-shared-task-multimodal/). In Proceedings of the Second Conference on Machine Translation, Volume 2: Shared Task Papers, pages 215–233,…
17. Desmond Elliott, Stella Frank, Khalil Sima’an, and Lucia Specia. 2016. [Multi30k: Multilingual english-german image descriptions](https://md.research.elamur.ai/free/en/papers/elliott-2016-multi30k-multilingual-english-german-image/). In Proceedings of the 5th Workshop on Vision and Language, pages 70–74. Association for Computational Linguistics.
18. Fangxiaoyu Feng, Yinfei Yang, Daniel Matthew Cer, N. Arivazhagan, and Wei Wang. 2020. [Language-agnostic bert sentence embedding](https://md.research.elamur.ai/free/en/papers/feng-2020-language-agnostic-bert-sentence-embedding/). In Annual Meeting of the Association for Computational Linguistics.
19. Samir Yitzhak Gadre, Gabriel Ilharco, Alex Fang, Jonathan Hayase, Georgios Smyrnis, Thao Nguyen, Ryan Marten, Mitchell Wortsman, Dhruba Ghosh, Jieyu Zhang, Eyal Orgad, Rahim Entezari, Giannis Daras, Sarah Pratt, Vivek Ramanujan, Yonatan Bitton, Kalyani Marathe, Stephen Mussmann, Richard Vencu, Mehdi… [find](https://scholar.google.com/scholar?q=Samir+Yitzhak+Gadre%2C+Gabriel+Ilharco%2C+Alex+Fang%2C+Jonathan+Hayase%2C+Georgios+Smyrnis%2C+Thao+Nguyen%2C+Ryan+Marten%2C+Mitchell+Wortsman%2C+Dhruba+Ghosh%2C+Jieyu+Zhang%2C+Eyal+Orgad%2C+Rahim+Entezari%2C+Giannis+Daras%2C+S)
20. Jay P. Gala, Pranjal A. Chitale, AK Raghavan, Varun Gumma, Sumanth Doddapaneni, M. AswanthKumar, Janki Atul Nawale, Anupama Sujatha, Ratish Puduppully, Vivek Raghavan, Pratyush Kumar, Mitesh M. Khapra, Raj Dabre, and Anoop Kunchukuttan. 2023. [Indictrans2: Towards high-quality and accessible machine…](https://md.research.elamur.ai/free/en/papers/gala-2023-indictrans2-towards-high-quality-accessible/)
21. Trans. Mach. Learn. Res., 2023. [find](https://scholar.google.com/scholar?q=Trans%2E+Mach%2E+Learn%2E+Res%2E%2C+2023%2E)
22. John C Gower. 1975. Generalized procrustes analysis. Psychometrika, 40:33–51. [find](https://scholar.google.com/scholar?q=John+C+Gower%2E+1975%2E+Generalized+procrustes+analysis%2E+Psychometrika%2C+40%3A33%E2%80%9351%2E)
23. Kaiming He, X. Zhang, Shaoqing Ren, and Jian Sun. 2015. [Deep residual learning for image recognition](https://md.research.elamur.ai/free/en/papers/he-2015-deep-residual-learning-image-recognition/). 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 770–778.
24. Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. 2017. Gans trained by a two time-scale update rule converge to a local nash equilibrium. In Neural Information Processing Systems. [find](https://scholar.google.com/scholar?q=Martin+Heusel%2C+Hubert+Ramsauer%2C+Thomas+Unterthiner%2C+Bernhard+Nessler%2C+and+Sepp+Hochreiter%2E+2017%2E+Gans+trained+by+a+two+time%2Dscale+update+rule+converge+to+a+local+nash+equilibrium%2E+In+Neural+Informatio)
25. Weiquan Huang, Aoqi Wu, Yifan Yang, Xufang Luo, Yuqing Yang, Liang Hu, Qi Dai, Chunyu Wang, Xiyang Dai, Dongdong Chen, et al. 2024. Llm2clip: Powerful language model unlocks richer visual representation. arXiv preprint arXiv:2411.04997. — [arXiv:2411.04997](https://arxiv.org/abs/2411.04997) [find](https://scholar.google.com/scholar?q=Weiquan+Huang%2C+Aoqi+Wu%2C+Yifan+Yang%2C+Xufang+Luo%2C+Yuqing+Yang%2C+Liang+Hu%2C+Qi+Dai%2C+Chunyu+Wang%2C+Xiyang+Dai%2C+Dongdong+Chen%2C+et+al%2E+2024%2E+Llm2clip%3A+Powerful+language+model+unlocks+richer+visual+representati)
26. Aashi Jain, Mandy Guo, Krishna Srinivasan, Ting Chen, Sneha Kudugunta, Chao Jia, Yinfei Yang, and Jason Baldridge. 2021. Mural: Multimodal, multitask retrieval across languages. ArXiv, abs/2109.05125. — [arXiv:2109.05125](https://arxiv.org/abs/2109.05125) [find](https://scholar.google.com/scholar?q=Aashi+Jain%2C+Mandy+Guo%2C+Krishna+Srinivasan%2C+Ting+Chen%2C+Sneha+Kudugunta%2C+Chao+Jia%2C+Yinfei+Yang%2C+and+Jason+Baldridge%2E+2021%2E+Mural%3A+Multimodal%2C+multitask+retrieval+across+languages%2E+ArXiv%2C+abs%2F2109%2E05125%2E)
27. Chao Jia, Yinfei Yang, Ye Xia, Yi-Ting Chen, Zarana Parekh, Hieu Pham, Quoc Le, Yun-Hsuan Sung, Zhen Li, and Tom Duerig. 2021. [Scaling up visual and vision-language representation learning with noisy text supervision](https://md.research.elamur.ai/free/en/papers/jia-2021-scaling-up-visual-vision-language/). In International conference on machine learning, pages 4904–4916. PMLR.
28. Chris Dongjoo Kim, Byeongchang Kim, Hyunmin Lee, and Gunhee Kim. 2019. [AudioCaps: Generating captions for audios in the wild](https://md.research.elamur.ai/free/en/papers/kim-2019-audiocaps-generating-captions-audios-wild/). In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Pa…
29. Andreas Koukounas, Georgios Mastrapas, Michael Günther, Bo Wang, Scott Martens, Isabelle Mohr, Saba Sturua, Mohammad Kalim Akram, Joan Fontanals Mart’inez, Saahil Ognawala, Susana Guzman, Maximilian Werk, Nan Wang, and Han Xiao. 2024a. Jina clip: Your clip model is also your text retriever. ArXiv, a… — [arXiv:2405.20204](https://arxiv.org/abs/2405.20204) [find](https://scholar.google.com/scholar?q=Andreas+Koukounas%2C+Georgios+Mastrapas%2C+Michael+G%C3%BCnther%2C+Bo+Wang%2C+Scott+Martens%2C+Isabelle+Mohr%2C+Saba+Sturua%2C+Mohammad+Kalim+Akram%2C+Joan+Fontanals+Mart%E2%80%99inez%2C+Saahil+Ognawala%2C+Susana+Guzman%2C+Maximilian+W)
30. Andreas Koukounas, Georgios Mastrapas, Bo Wang, Mohammad Kalim Akram, Sedigheh Eslami, Michael Gunther, Isabelle Mohr, Saba Sturua, Scott Martens, Nan Wang, and Han Xiao. 2024b. jina-clip-v2: Multilingual multimodal embeddings for text and images. ArXiv, abs/2412.08802. — [arXiv:2412.08802](https://arxiv.org/abs/2412.08802) [find](https://scholar.google.com/scholar?q=Andreas+Koukounas%2C+Georgios+Mastrapas%2C+Bo+Wang%2C+Mohammad+Kalim+Akram%2C+Sedigheh+Eslami%2C+Michael+Gunther%2C+Isabelle+Mohr%2C+Saba+Sturua%2C+Scott+Martens%2C+Nan+Wang%2C+and+Han+Xiao%2E+2024b%2E+jina%2Dclip%2Dv2%3A+Multilin)
31. Black Forest Labs. 2024. Flux. [https://github.com/](https://github.com/) black-forest-labs/flux. Accessed: 2025-02-11. [↗](https://github.com/)
32. Zorah Lähner and Michael Moeller. 2024. On the direct alignment of latent spaces. In Proceedings of UniReps: the First Workshop on Unifying Representations in Neural Models, pages 158–169. PMLR. [find](https://scholar.google.com/scholar?q=Zorah+L%C3%A4hner+and+Michael+Moeller%2E+2024%2E+On+the+direct+alignment+of+latent+spaces%2E+In+Proceedings+of+UniReps%3A+the+First+Workshop+on+Unifying+Representations+in+Neural+Models%2C+pages+158%E2%80%93169%2E+PMLR%2E)
33. Yaoyiran Li, Ching-Yun Chang, Stephen Rawls, Ivan Vulić, and Anna Korhonen. 2023. Translation-enhanced multilingual text-to-image generation. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 9174–9193, Toronto, Canada. Associa… [find](https://scholar.google.com/scholar?q=Yaoyiran+Li%2C+Ching%2DYun+Chang%2C+Stephen+Rawls%2C+Ivan+Vulic%CC%81%2C+and+Anna+Korhonen%2E+2023%2E+Translation%2Denhanced+multilingual+text%2Dto%2Dimage+generation%2E+In+Proceedings+of+the+61st+Annual+Meeting+of+the+Associat)
34. Tsung-Yi Lin, Michael Maire, Serge J. Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C. Lawrence Zitnick. 2014. [Microsoft coco: Common objects in context](https://md.research.elamur.ai/free/en/papers/lin-2014-microsoft-coco-common-objects-context/). In European Conference on Computer Vision.
35. I Loshchilov. 2017. Decoupled weight decay regularization. arXiv preprint arXiv:1711.05101. [find](https://scholar.google.com/scholar?q=I+Loshchilov%2E+2017%2E+Decoupled+weight+decay+regularization%2E+arXiv+preprint+arXiv%3A1711%2E05101%2E)
36. Valentino Maiorca, Luca Moschella, Marco Fumero, Francesco Locatello, and Emanuele Rodolà. 2024a. Latent space translation via inverse relative projection. arXiv preprint arXiv:2406.15057. — [arXiv:2406.15057](https://arxiv.org/abs/2406.15057) [find](https://scholar.google.com/scholar?q=Valentino+Maiorca%2C+Luca+Moschella%2C+Marco+Fumero%2C+Francesco+Locatello%2C+and+Emanuele+Rodol%C3%A0%2E+2024a%2E+Latent+space+translation+via+inverse+relative+projection%2E+arXiv+preprint+arXiv%3A2406%2E15057%2E+%E2%80%94+%5BarXiv%3A24)
37. Valentino Maiorca, Luca Moschella, Antonio Norelli, Marco Fumero, Francesco Locatello, and Emanuele Rodolà. 2024b. Latent space translation via semantic alignment. Advances in Neural Information Processing Systems, 36. [find](https://scholar.google.com/scholar?q=Valentino+Maiorca%2C+Luca+Moschella%2C+Antonio+Norelli%2C+Marco+Fumero%2C+Francesco+Locatello%2C+and+Emanuele+Rodol%C3%A0%2E+2024b%2E+Latent+space+translation+via+semantic+alignment%2E+Advances+in+Neural+Information+Proce)
38. Xinhao Mei, Chutong Meng, Haohe Liu, Qiuqiang Kong, Tom Ko, Chengqi Zhao, Mark D. Plumbley, Yuexian Zou, and Wenwu Wang. 2023. Wavcaps: A chatgpt-assisted weakly-labelled audio captioning dataset for audio-language multimodal research. IEEE/ACM Transactions on Audio, Speech, and Language Processing,… [find](https://scholar.google.com/scholar?q=Xinhao+Mei%2C+Chutong+Meng%2C+Haohe+Liu%2C+Qiuqiang+Kong%2C+Tom+Ko%2C+Chengqi+Zhao%2C+Mark+D%2E+Plumbley%2C+Yuexian+Zou%2C+and+Wenwu+Wang%2E+2023%2E+Wavcaps%3A+A+chatgpt%2Dassisted+weakly%2Dlabelled+audio+captioning+dataset+for+)
39. Luca Moschella, Valentino Maiorca, Marco Fumero, Antonio Norelli, Francesco Locatello, and Emanuele Rodolà. 2022. Relative representations enable zero-shot latent space communication. ArXiv, abs/2209.15430. — [arXiv:2209.15430](https://arxiv.org/abs/2209.15430) [find](https://scholar.google.com/scholar?q=Luca+Moschella%2C+Valentino+Maiorca%2C+Marco+Fumero%2C+Antonio+Norelli%2C+Francesco+Locatello%2C+and+Emanuele+Rodol%C3%A0%2E+2022%2E+Relative+representations+enable+zero%2Dshot+latent+space+communication%2E+ArXiv%2C+abs%2F2209%2E)
40. Antonio Norelli, Marco Fumero, Valentino Maiorca, Luca Moschella, Emanuele Rodolà, and Francesco Locatello. 2022. Asif: Coupled data turns unimodal models to multimodal without training. ArXiv, abs/2210.01738. — [arXiv:2210.01738](https://arxiv.org/abs/2210.01738) [find](https://scholar.google.com/scholar?q=Antonio+Norelli%2C+Marco+Fumero%2C+Valentino+Maiorca%2C+Luca+Moschella%2C+Emanuele+Rodol%C3%A0%2C+and+Francesco+Locatello%2E+2022%2E+Asif%3A+Coupled+data+turns+unimodal+models+to+multimodal+without+training%2E+ArXiv%2C+abs%2F22)
41. Anton Obukhov, Maximilian Seitzer, Po-Wei Wu, Semen Zhydenko, Jonathan Kyl, and Elvis Yu-Jing Lin. 2020. High-fidelity performance metrics for generative models in pytorch. Version: 0.3.0, DOI: 10.5281/zenodo.4957738. — [doi:10.5281/zenodo.4957738](https://doi.org/10.5281/zenodo.4957738) [find](https://scholar.google.com/scholar?q=Anton+Obukhov%2C+Maximilian+Seitzer%2C+Po%2DWei+Wu%2C+Semen+Zhydenko%2C+Jonathan+Kyl%2C+and+Elvis+Yu%2DJing+Lin%2E+2020%2E+High%2Dfidelity+performance+metrics+for+generative+models+in+pytorch%2E+Version%3A+0%2E3%2E0%2C+DOI%3A+10%2E528)
42. Maja Popović. 2017. [chrF++: words helping character n-grams](https://md.research.elamur.ai/free/en/papers/popovic-2017-chrf-words-helping-character-n/). In Proceedings of the Second Conference on Machine Translation, pages 612–618, Copenhagen, Denmark. Association for Computational Linguistics.
43. Matt Post. 2018. [A call for clarity in reporting BLEU scores](https://md.research.elamur.ai/free/en/papers/post-2018-call-clarity-reporting-bleu-scores/). In Proceedings of the Third Conference on Machine Translation: Research Papers, pages 186– 191, Brussels, Belgium. Association for Computational Linguistics.
44. Alec Radford. 2018. Improving language understanding by generative pre-training. [find](https://scholar.google.com/scholar?q=Alec+Radford%2E+2018%2E+Improving+language+understanding+by+generative+pre%2Dtraining%2E)
45. Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. 2021. [Learning transferable visual models from natural language supervision](https://md.research.elamur.ai/free/en/papers/radford-2021-learning-transferable-visual-models-natural/). In International Conference on Machin…
46. Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu. 2020. [Exploring the limits of transfer learning with a unified text-to-text transformer](https://md.research.elamur.ai/free/en/papers/raffel-2019-exploring-limits-transfer-learning-unified/). Journal of machine learning research, 21(140):1–67.
47. Janarthanan Rajendran, Mitesh M. Khapra, Sarath Chandar, and Balaraman Ravindran. 2016. Bridge correlational neural networks for multilingual multimodal representation learning. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Huma… [find](https://scholar.google.com/scholar?q=Janarthanan+Rajendran%2C+Mitesh+M%2E+Khapra%2C+Sarath+Chandar%2C+and+Balaraman+Ravindran%2E+2016%2E+Bridge+correlational+neural+networks+for+multilingual+multimodal+representation+learning%2E+In+Proceedings+of+the+)
48. Aditya Ramesh, Mikhail Pavlov, Gabriel Goh, Scott Gray, Chelsea Voss, Alec Radford, Mark Chen, and Ilya Sutskever. 2021. [Zero-shot text-to-image generation](https://md.research.elamur.ai/free/en/papers/ramesh-2021-zero-shot-text-image-generation/). ArXiv, abs/2102.12092.
49. Nils Reimers and Iryna Gurevych. 2020. [Making monolingual sentence embeddings multilingual using knowledge distillation](https://md.research.elamur.ai/free/en/papers/reimers-2020-making-monolingual-sentence-embeddings-multilingual/). In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics.
50. Robin Rombach, A. Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. 2021. High-resolution image synthesis with latent diffusion models. 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 10674– 10685. [find](https://scholar.google.com/scholar?q=Robin+Rombach%2C+A%2E+Blattmann%2C+Dominik+Lorenz%2C+Patrick+Esser%2C+and+Bj%C3%B6rn+Ommer%2E+2021%2E+High%2Dresolution+image+synthesis+with+latent+diffusion+models%2E+2022+IEEE%2FCVF+Conference+on+Computer+Vision+and+Pattern)
51. Elan Rosenfeld, Preetum Nakkiran, Hadi Pouransari, Oncel Tuzel, and Fartash Faghri. 2022. Ape: Aligning pretrained encoders to quickly learn aligned multimodal representations. ArXiv, abs/2210.03927. — [arXiv:2210.03927](https://arxiv.org/abs/2210.03927) [find](https://scholar.google.com/scholar?q=Elan+Rosenfeld%2C+Preetum+Nakkiran%2C+Hadi+Pouransari%2C+Oncel+Tuzel%2C+and+Fartash+Faghri%2E+2022%2E+Ape%3A+Aligning+pretrained+encoders+to+quickly+learn+aligned+multimodal+representations%2E+ArXiv%2C+abs%2F2210%2E03927%2E+)
52. Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily L. Denton, Seyed Kamyar Seyed Ghasemipour, Burcu Karagol Ayan, Seyedeh Sara Mahdavi, Raphael Gontijo Lopes, Tim Salimans, Jonathan Ho, David J. Fleet, and Mohammad Norouzi. 2022. [Photorealistic text-to-image diffusion models wi…](https://md.research.elamur.ai/free/en/papers/saharia-2022-photorealistic-text-image-diffusion-models/)
53. Tim Salimans, Ian J. Goodfellow, Wojciech Zaremba, Vicki Cheung, Alec Radford, and Xi Chen. 2016. [Improved techniques for training gans](https://md.research.elamur.ai/free/en/papers/salimans-2016-improved-techniques-training-gans/). ArXiv, abs/1606.03498.
54. Piyush Sharma, Nan Ding, Sebastian Goodman, and Radu Soricut. 2018. [Conceptual captions: A cleaned, hypernymed, image alt-text dataset for automatic image captioning](https://md.research.elamur.ai/free/en/papers/sharma-2018-conceptual-captions-cleaned-hypernymed-image/). In Annual Meeting of the Association for Computational Linguistics.
55. Saba Sturua, Isabelle Mohr, Mohammad Kalim Akram, Michael Günther, Bo Wang, Markus Krimmel, Feng Wang, Georgios Mastrapas, Andreas Koukounas, Nan Wang, et al. 2024. jina-embeddings-v3: Multilingual embeddings with task lora. arXiv preprint arXiv:2409.10173. locked-image text tuning. 2022 IEEE/CVF Co… [find](https://scholar.google.com/scholar?q=Saba+Sturua%2C+Isabelle+Mohr%2C+Mohammad+Kalim+Akram%2C+Michael+G%C3%BCnther%2C+Bo+Wang%2C+Markus+Krimmel%2C+Feng+Wang%2C+Georgios+Mastrapas%2C+Andreas+Koukounas%2C+Nan+Wang%2C+et+al%2E+2024%2E+jina%2Dembeddings%2Dv3%3A+Multilingual+em)
56. Ashish V. Thapliyal, Jordi Pont Tuset, Xi Chen, and Radu Soricut. 2022. [Crossmodal-3600: A massively multilingual multimodal evaluation dataset](https://md.research.elamur.ai/free/en/papers/thapliyal-2022-crossmodal-3600-massively-multilingual-multimodal/). In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 715–729, Abu Dhabi, United Arab Emirates. Association for…
57. Yufan Zhou, Ruiyi Zhang, Changyou Chen, Chunyuan Li, Chris Tensmeyer, Tong Yu, Jiuxiang Gu, Jinhui Xu, and Tong Sun. 2022. Towards language-free training for text-to-image generation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 17907–17917. [find](https://scholar.google.com/scholar?q=Yufan+Zhou%2C+Ruiyi+Zhang%2C+Changyou+Chen%2C+Chunyuan+Li%2C+Chris+Tensmeyer%2C+Tong+Yu%2C+Jiuxiang+Gu%2C+Jinhui+Xu%2C+and+Tong+Sun%2E+2022%2E+Towards+language%2Dfree+training+for+text%2Dto%2Dimage+generation%2E+In+Proceedings+o)
58. Michael Tschannen, Alexey Gritsenko, Xiao Wang, Muhammad Ferjad Naeem, Ibrahim Alabdulmohsin, Nikhil Parthasarathy, Talfan Evans, Lucas Beyer, Ye Xia, Basil Mustafa, et al. 2025. Siglip 2: Multilingual vision-language encoders with improved semantic understanding, localization, and dense features. a… [find](https://scholar.google.com/scholar?q=Michael+Tschannen%2C+Alexey+Gritsenko%2C+Xiao+Wang%2C+Muhammad+Ferjad+Naeem%2C+Ibrahim+Alabdulmohsin%2C+Nikhil+Parthasarathy%2C+Talfan+Evans%2C+Lucas+Beyer%2C+Ye+Xia%2C+Basil+Mustafa%2C+et+al%2E+2025%2E+Siglip+2%3A+Multilingua)
59. Yusong Wu, K. Chen, Tianyu Zhang, Yuchen Hui, Taylor Berg-Kirkpatrick, and Shlomo Dubnov. 2022. Large-scale contrastive language-audio pretraining with feature fusion and keyword-to-caption augmentation. [find](https://scholar.google.com/scholar?q=Yusong+Wu%2C+K%2E+Chen%2C+Tianyu+Zhang%2C+Yuchen+Hui%2C+Taylor+Berg%2DKirkpatrick%2C+and+Shlomo+Dubnov%2E+2022%2E+Large%2Dscale+contrastive+language%2Daudio+pretraining+with+feature+fusion+and+keyword%2Dto%2Dcaption+augmentati)
60. ICASSP 2023 - 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 1–5. [find](https://scholar.google.com/scholar?q=ICASSP+2023+%2D+2023+IEEE+International+Conference+on+Acoustics%2C+Speech+and+Signal+Processing+%28ICASSP%29%2C+pages+1%E2%80%935%2E)
61. Zhiyong Yan, Heinrich Dinkel, Yongqing Wang, Jizhong Liu, Junbo Zhang, Yujun Wang, and Bin Wang. 2024. Bridging language gaps in audio-text retrieval. ArXiv, abs/2406.07012. — [arXiv:2406.07012](https://arxiv.org/abs/2406.07012) [find](https://scholar.google.com/scholar?q=Zhiyong+Yan%2C+Heinrich+Dinkel%2C+Yongqing+Wang%2C+Jizhong+Liu%2C+Junbo+Zhang%2C+Yujun+Wang%2C+and+Bin+Wang%2E+2024%2E+Bridging+language+gaps+in+audio%2Dtext+retrieval%2E+ArXiv%2C+abs%2F2406%2E07012%2E+%E2%80%94+%5BarXiv%3A2406%2E07012%5D%28https)
62. Chenglin Yang, Celong Liu, Xueqing Deng, Dongwon Kim, Xing Mei, Xiaohui Shen, and Liang-Chieh Chen. 2024. 1.58-bit flux. arXiv preprint arXiv:2412.18653. — [arXiv:2412.18653](https://arxiv.org/abs/2412.18653) [find](https://scholar.google.com/scholar?q=Chenglin+Yang%2C+Celong+Liu%2C+Xueqing+Deng%2C+Dongwon+Kim%2C+Xing+Mei%2C+Xiaohui+Shen%2C+and+Liang%2DChieh+Chen%2E+2024%2E+1%2E58%2Dbit+flux%2E+arXiv+preprint+arXiv%3A2412%2E18653%2E+%E2%80%94+%5BarXiv%3A2412%2E18653%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F241)
63. Fulong Ye, Guang Liu, Xinya Wu, and Ledell Wu. 2024. Altdiffusion: A multilingual text-to-image diffusion model. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 6648–6656. [find](https://scholar.google.com/scholar?q=Fulong+Ye%2C+Guang+Liu%2C+Xinya+Wu%2C+and+Ledell+Wu%2E+2024%2E+Altdiffusion%3A+A+multilingual+text%2Dto%2Dimage+diffusion+model%2E+In+Proceedings+of+the+AAAI+Conference+on+Artificial+Intelligence%2C+volume+38%2C+pages+6648)
64. Boogeo Yoon, Youhan Lee, and Woonhyuk Baek. 2022. Coyo-align. [https://github.com/kakaobrain/](https://github.com/kakaobrain/) coyo-align. [↗](https://github.com/kakaobrain/)
65. Yuya Yoshikawa, Yutaro Shigeto, and Akikazu Takeuchi. 2017. STAIR captions: Constructing a large-scale Japanese image caption dataset. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 417–421, Vancouver, Canada. Association fo… [find](https://scholar.google.com/scholar?q=Yuya+Yoshikawa%2C+Yutaro+Shigeto%2C+and+Akikazu+Takeuchi%2E+2017%2E+STAIR+captions%3A+Constructing+a+large%2Dscale+Japanese+image+caption+dataset%2E+In+Proceedings+of+the+55th+Annual+Meeting+of+the+Association+for+)
66. Xiaohua Zhai, Basil Mustafa, Alexander Kolesnikov, and Lucas Beyer. 2023. [Sigmoid loss for language image pre-training](https://md.research.elamur.ai/free/en/papers/zhai-2023-sigmoid-loss-language-image-pre/). In Proceedings of the IEEE/CVF international conference on computer vision, pages 11975–11986.
67. Xiaohua Zhai, Xiao Wang, Basil Mustafa, Andreas Steiner, Daniel Keysers, Alexander Kolesnikov, and Lucas Beyer. 2021. Lit: Zero-shot transfer with [find](https://scholar.google.com/scholar?q=Xiaohua+Zhai%2C+Xiao+Wang%2C+Basil+Mustafa%2C+Andreas+Steiner%2C+Daniel+Keysers%2C+Alexander+Kolesnikov%2C+and+Lucas+Beyer%2E+2021%2E+Lit%3A+Zero%2Dshot+transfer+with)

*Updated 1 November 2026*