---
title: "Towards Unified Multimodal Large Language Models: A survey"
description: "They describe four reasons to pursue unification: a shared model of reality, transfer across modalities, transfer across tasks… 174 references."
authors: ["Xu Ma", "Yitian Zhang", "Yun Fu"]
year: 2026
venue: "Findings of the Association for Computational Linguistics: ACL 2026"
source: "https://aclanthology.org/2026.findings-acl.1853/"
doi: "10.18653/v1/2026.findings-acl.1853"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/ma-2026-unified-multimodal-large-language-models/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/ma-2026-unified-multimodal-large-language-models/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/54/) / Towards Unified Multimodal Large Language Models: A survey

# Towards Unified Multimodal Large Language Models: A survey

**Authors:** Xu Ma, [Yitian Zhang](https://md.research.elamur.ai/free/en/authors/yitian-zhang/) and [Yun Fu](https://md.research.elamur.ai/free/en/authors/yun-fu/) · **Year:** 2026 · **Venue:** [Findings of the Association for Computational Linguistics: ACL 2026](https://md.research.elamur.ai/free/en/venues/findings/) ([Findings of the Association for Computational Linguistics 2026](https://md.research.elamur.ai/free/en/venues/findings/2026/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2026.findings-acl.1853/) · [PDF](https://aclanthology.org/2026.findings-acl.1853.pdf) · [DOI 10.18653/v1/2026.findings-acl.1853](https://doi.org/10.18653/v1/2026.findings-acl.1853) · [OpenAlex](https://openalex.org/W7166838875)

**Citations:** 0 citations (OpenAlex, 2026-10-14)

## Summary
Unified multimodal large language models (MLLMs) aim to both understand and generate text, images, audio and other modalities within one system. Research on them has grown quickly but in scattered directions, often without clearly stated motivations or trade-offs. This survey from Northeastern University brings that work together. It first covers the building blocks: tokenizers for text, vision, audio and other data types, and the autoregressive LLM objective. It then sorts recent unified models into a taxonomy based on architecture, visual representation, loss design, alignment and training strategy. Text–vision models get the most attention, since they dominate the field. The authors also take positions of their own. They favour semantic-level visual features, citing the Platonic Representation Hypothesis, and recommend simple, compute-aware designs. They describe four reasons to pursue unification: a shared model of reality, transfer across modalities, transfer across tasks within one modality, and emergent zero-shot abilities. They also note that strong evidence of cross-modal benefit is still limited. The survey serves as a roadmap for researchers building general multimodal systems.

## Key points
- The core resource is a taxonomy table (Table 1) of unified models listed by release date. Each entry records modalities, supported tasks, key features and design choices, and examples include SEED, EMU/EMU2/EMU3, Chameleon, Transfusion, Show-o, MIO, Janus and OmniGen2.
- The tokenizer overview covers several modalities:
  - text: BPE, WordPiece, Unigram, SentencePiece, ByT5, BLT;
  - vision: VQ-style discrete codebooks, CLIP-like continuous features, Q-Former;
  - audio: EnCodec, SoundStream, HuBERT, WavTokenizer, and Whisper-large-v3 as used in Qwen2.5-Omni;
  - other data: point clouds, graphs, proteins, time series.
- Visual representations are split into pixel-level (VAE-based) and semantic-level (CLIP-based). The authors argue for the semantic-level option.
- The authors find reproducible evidence of mutual cross-modal benefit still scarce, and attribute this to limits in model capacity, data balance and compute. MetaMorph is cited as evidence that a shared visual encoder helps both understanding and generation.
- The authors acknowledge limitations: closed systems such as Gemini and ChatGPT cannot be analysed in depth, and training data details are often undisclosed. The PDF header gives the title as "...: More than A Survey".

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Northeastern University](https://md.research.elamur.ai/free/en/universities/northeastern-university/) — University, Boston, United States · 68 works · 7,316 citations

📚 Shelves (2): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 18,614 works · [👁️ Computer Vision and Pattern Recognition](https://md.research.elamur.ai/free/en/shelves/computer-science/computer-vision-and-pattern-recognition/) · 3,486 works

🧭 Topics (2): [🧠 Language models & pretraining](https://md.research.elamur.ai/free/en/topics/language-models/) · 3,894 works · [🖼️ Vision & language](https://md.research.elamur.ai/free/en/topics/multimodal/) · 811 works

## Cite

Cite as: Xu Ma, Yitian Zhang and Yun Fu. 2026. Towards Unified Multimodal Large Language Models: A survey. Findings of the Association for Computational Linguistics: ACL 2026.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{ma2026unified,
  title = {{Towards Unified Multimodal Large Language Models: A survey}},
  author = {Xu Ma, Yitian Zhang and Yun Fu},
  booktitle = {Findings of the Association for Computational Linguistics: ACL 2026},
  year = {2026},
  doi = {10.18653/v1/2026.findings-acl.1853},
  url = {https://doi.org/10.18653/v1/2026.findings-acl.1853}
}
```

- [← previous Towards Unified Factuality Evaluation for Biomedical QA and Summarization: Aligning Metrics with Clinical Use-Cases](https://md.research.elamur.ai/free/en/papers/roy-2026-unified-factuality-evaluation-biomedical-qa/)
- [next → Toxic dialogues: triggers and interaction patterns on Brazilian Reddit](https://md.research.elamur.ai/free/en/papers/piorino-2026-dialogos-toxicos-gatilhos-e-padroes/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/54/)

## References

174 references, 63 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Gemini 2.5 flash image (nano banana) — google ai studio. [https://aistudio.google.com/models/](https://aistudio.google.com/models/) gemini-2-5-flash-image. Accessed: 2025-10-05. [↗](https://aistudio.google.com/models/)
2. Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, and 1 others. 2023. [Gpt-4 technical report](https://md.research.elamur.ai/free/en/papers/openai-2023-gpt-4-technical-report/). arXiv preprint arXiv:2303.08774.
3. Niket Agarwal, Arslan Ali, Maciej Bala, Yogesh Balaji, Erik Barker, Tiffany Cai, Prithvijit Chattopadhyay, Yongxin Chen, Yin Cui, Yifan Ding, and 1 others. 2025. Cosmos world foundation model platform for physical ai. arXiv preprint arXiv:2501.03575. — [arXiv:2501.03575](https://arxiv.org/abs/2501.03575) [find](https://scholar.google.com/scholar?q=Niket+Agarwal%2C+Arslan+Ali%2C+Maciej+Bala%2C+Yogesh+Balaji%2C+Erik+Barker%2C+Tiffany+Cai%2C+Prithvijit+Chattopadhyay%2C+Yongxin+Chen%2C+Yin+Cui%2C+Yifan+Ding%2C+and+1+others%2E+2025%2E+Cosmos+world+foundation+model+platform)
4. Walid Ahmad, Elana Simon, Seyone Chithrananda, Gabriel Grand, and Bharath Ramsundar. 2022. Chemberta-2: Towards chemical foundation models. arXiv preprint arXiv:2209.01712. — [arXiv:2209.01712](https://arxiv.org/abs/2209.01712) [find](https://scholar.google.com/scholar?q=Walid+Ahmad%2C+Elana+Simon%2C+Seyone+Chithrananda%2C+Gabriel+Grand%2C+and+Bharath+Ramsundar%2E+2022%2E+Chemberta%2D2%3A+Towards+chemical+foundation+models%2E+arXiv+preprint+arXiv%3A2209%2E01712%2E+%E2%80%94+%5BarXiv%3A2209%2E01712%5D%28https%3A)
5. Inclusion AI, Biao Gong, Cheng Zou, Chuanyang Zheng, Chunluan Zhou, Canxiang Yan, Chunxiang Jin, Chunjie Shen, Dandan Zheng, Fudong Wang, and 1 others. 2025. Ming-omni: A unified multimodal model for perception and generation. arXiv preprint arXiv:2506.09344. — [arXiv:2506.09344](https://arxiv.org/abs/2506.09344) [find](https://scholar.google.com/scholar?q=Inclusion+AI%2C+Biao+Gong%2C+Cheng+Zou%2C+Chuanyang+Zheng%2C+Chunluan+Zhou%2C+Canxiang+Yan%2C+Chunxiang+Jin%2C+Chunjie+Shen%2C+Dandan+Zheng%2C+Fudong+Wang%2C+and+1+others%2E+2025%2E+Ming%2Domni%3A+A+unified+multimodal+model+for+)
6. Jean-Baptiste Alayrac, Jeff Donahue, Pauline Luc, Antoine Miech, Iain Barr, Yana Hasson, Karel Lenc, Arthur Mensch, Katherine Millican, Malcolm Reynolds, and 1 others. 2022. [Flamingo: a visual language model for few-shot learning](https://md.research.elamur.ai/free/en/papers/alayrac-2022-flamingo-visual-language-model-few/). Advances in neural information processing systems, 35:23716– 23736.
7. Mido Assran, Adrien Bardes, David Fan, Quentin Garrido, Russell Howes, Matthew Muckley, Ammar Rizvi, Claire Roberts, Koustuv Sinha, Artem Zholus, and 1 others. 2025. V-jepa 2: Self-supervised video models enable understanding, prediction and planning. arXiv preprint arXiv:2506.09985. — [arXiv:2506.09985](https://arxiv.org/abs/2506.09985) [find](https://scholar.google.com/scholar?q=Mido+Assran%2C+Adrien+Bardes%2C+David+Fan%2C+Quentin+Garrido%2C+Russell+Howes%2C+Matthew+Muckley%2C+Ammar+Rizvi%2C+Claire+Roberts%2C+Koustuv+Sinha%2C+Artem+Zholus%2C+and+1+others%2E+2025%2E+V%2Djepa+2%3A+Self%2Dsupervised+video+mo)
8. Roman Bachmann, Jesse Allardice, David Mizrahi, Enrico Fini, Oğuzhan Fatih Kar, Elmira Amirloo, Alaaeldin El-Nouby, Amir Zamir, and Afshin Dehghan. 2025. Flextok: Resampling images into 1d token sequences of flexible length. In Forty-second International Conference on Machine Learning. [find](https://scholar.google.com/scholar?q=Roman+Bachmann%2C+Jesse+Allardice%2C+David+Mizrahi%2C+Enrico+Fini%2C+Og%CC%86uzhan+Fatih+Kar%2C+Elmira+Amirloo%2C+Alaaeldin+El%2DNouby%2C+Amir+Zamir%2C+and+Afshin+Dehghan%2E+2025%2E+Flextok%3A+Resampling+images+into+1d+token+sequ)
9. Alexei Baevski, Yuhao Zhou, Abdelrahman Mohamed, and Michael Auli. 2020. [wav2vec 2.0: A framework for self-supervised learning of speech representations](https://md.research.elamur.ai/free/en/papers/baevski-2020-wav2vec-2-0-framework-self/). Advances in neural information processing systems, 33:12449–12460.
10. Jinze Bai, Shuai Bai, Shusheng Yang, Shijie Wang, Sinan Tan, Peng Wang, Junyang Lin, Chang Zhou, and Jingren Zhou. 2023. [Qwen-vl: A versatile vision-language model for understanding, localization, text reading, and beyond](https://md.research.elamur.ai/free/en/papers/bai-2023-qwen-vl-versatile-vision-language/). Preprint, arXiv:2308.12966.
11. Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, and 1 others. 2025. Qwen2. 5-vl technical report. arXiv preprint arXiv:2502.13923. — [Qwen2.5-VL Technical Report](https://md.research.elamur.ai/free/en/papers/bai-2025-qwen2-5-vl-technical/)
12. Zechen Bai, Jianxiong Gao, Ziteng Gao, Pichao Wang, Zheng Zhang, Tong He, and Mike Zheng Shou. 2024. Factorized visual tokenization and generation. arXiv preprint arXiv:2411.16681. — [arXiv:2411.16681](https://arxiv.org/abs/2411.16681) [find](https://scholar.google.com/scholar?q=Zechen+Bai%2C+Jianxiong+Gao%2C+Ziteng+Gao%2C+Pichao+Wang%2C+Zheng+Zhang%2C+Tong+He%2C+and+Mike+Zheng+Shou%2E+2024%2E+Factorized+visual+tokenization+and+generation%2E+arXiv+preprint+arXiv%3A2411%2E16681%2E+%E2%80%94+%5BarXiv%3A2411%2E16681)
13. Yonatan Bisk, Rowan Zellers, Jianfeng Gao, Yejin Choi, and 1 others. 2020. [Piqa: Reasoning about physical commonsense in natural language](https://md.research.elamur.ai/free/en/papers/bisk-2019-piqa-reasoning-about-physical-commonsense/). In Proceedings of the AAAI conference on artificial intelligence, volume 34, pages 7432–7439.
14. Daniel Bolya, Po-Yao Huang, Peize Sun, Jang Hyun Cho, Andrea Madotto, Chen Wei, Tengyu Ma, Jiale Zhi, Jathushan Rajasegaran, Hanoona Rasheed, and 1 others. 2025. Perception encoder: The best visual embeddings are not at the output of the network. arXiv preprint arXiv:2504.13181. — [arXiv:2504.13181](https://arxiv.org/abs/2504.13181) [find](https://scholar.google.com/scholar?q=Daniel+Bolya%2C+Po%2DYao+Huang%2C+Peize+Sun%2C+Jang+Hyun+Cho%2C+Andrea+Madotto%2C+Chen+Wei%2C+Tengyu+Ma%2C+Jiale+Zhi%2C+Jathushan+Rajasegaran%2C+Hanoona+Rasheed%2C+and+1+others%2E+2025%2E+Perception+encoder%3A+The+best+visual+em)
15. Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, and 1 others. 2020. [Language models are few-shot learners](https://md.research.elamur.ai/free/en/papers/brown-2020-language-models-few-shot-learners/). Advances in neural information processing systems, 33:1877–1901.
16. Siyu Cao, Hangting Chen, Peng Chen, Yiji Cheng, Yutao Cui, Xinchi Deng, Ying Dong, Kipper Gong, Tianpeng Gu, Xiusen Gu, and 1 others. 2025. [Hunyuanimage 3.0 technical report](https://md.research.elamur.ai/free/en/papers/team-2025-hunyuan-hunyuanimage-3-0-technical/). arXiv preprint arXiv:2509.23951.
17. Joao Carreira and Andrew Zisserman. 2017. Quo vadis, action recognition? a new model and the kinetics dataset. In proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 6299–6308. [find](https://scholar.google.com/scholar?q=Joao+Carreira+and+Andrew+Zisserman%2E+2017%2E+Quo+vadis%2C+action+recognition%3F+a+new+model+and+the+kinetics+dataset%2E+In+proceedings+of+the+IEEE+Conference+on+Computer+Vision+and+Pattern+Recognition%2C+pages+6)
18. Fengjiao Chen, Minhao Jing, Weitao Lu, Yan Feng, Xiaoyu Li, and Xuezhi Cao. 2025a. Unihetero: Could generation enhance understanding for vision-language-model at large data scale? arXiv preprint arXiv:2512.23512. — [arXiv:2512.23512](https://arxiv.org/abs/2512.23512) [find](https://scholar.google.com/scholar?q=Fengjiao+Chen%2C+Minhao+Jing%2C+Weitao+Lu%2C+Yan+Feng%2C+Xiaoyu+Li%2C+and+Xuezhi+Cao%2E+2025a%2E+Unihetero%3A+Could+generation+enhance+understanding+for+vision%2Dlanguage%2Dmodel+at+large+data+scale%3F+arXiv+preprint+arXiv)
19. Jiuhai Chen, Zhiyang Xu, Xichen Pan, Yushi Hu, Can Qin, Tom Goldstein, Lifu Huang, Tianyi Zhou, Saining Xie, Silvio Savarese, and 1 others. 2025b. Blip3-o: A family of fully open unified multimodal models-architecture, training and dataset. arXiv preprint arXiv:2505.09568. [find](https://scholar.google.com/scholar?q=Jiuhai+Chen%2C+Zhiyang+Xu%2C+Xichen+Pan%2C+Yushi+Hu%2C+Can+Qin%2C+Tom+Goldstein%2C+Lifu+Huang%2C+Tianyi+Zhou%2C+Saining+Xie%2C+Silvio+Savarese%2C+and+1+others%2E+2025b%2E+Blip3%2Do%3A+A+family+of+fully+open+unified+multimodal+mo)
20. Ke Chen, Xingjian Du, Bilei Zhu, Zejun Ma, Taylor Berg-Kirkpatrick, and Shlomo Dubnov. 2022. Hts-at: A hierarchical token-semantic audio transformer for sound classification and detection. In IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP. [find](https://scholar.google.com/scholar?q=Ke+Chen%2C+Xingjian+Du%2C+Bilei+Zhu%2C+Zejun+Ma%2C+Taylor+Berg%2DKirkpatrick%2C+and+Shlomo+Dubnov%2E+2022%2E+Hts%2Dat%3A+A+hierarchical+token%2Dsemantic+audio+transformer+for+sound+classification+and+detection%2E+In+IEEE+Int)
21. Mark Chen, Alec Radford, Rewon Child, Jeffrey Wu, Heewoo Jun, David Luan, and Ilya Sutskever. 2020. Generative pretraining from pixels. In International conference on machine learning, pages 1691–1703. PMLR. [find](https://scholar.google.com/scholar?q=Mark+Chen%2C+Alec+Radford%2C+Rewon+Child%2C+Jeffrey+Wu%2C+Heewoo+Jun%2C+David+Luan%2C+and+Ilya+Sutskever%2E+2020%2E+Generative+pretraining+from+pixels%2E+In+International+conference+on+machine+learning%2C+pages+1691%E2%80%931703)
22. Wenxi Chen, Yuzhe Liang, Ziyang Ma, Zhisheng Zheng, and Xie Chen. 2024. Eat: Self-supervised pre-training with efficient audio transformer. arXiv preprint arXiv:2401.03497. — [arXiv:2401.03497](https://arxiv.org/abs/2401.03497) [find](https://scholar.google.com/scholar?q=Wenxi+Chen%2C+Yuzhe+Liang%2C+Ziyang+Ma%2C+Zhisheng+Zheng%2C+and+Xie+Chen%2E+2024%2E+Eat%3A+Self%2Dsupervised+pre%2Dtraining+with+efficient+audio+transformer%2E+arXiv+preprint+arXiv%3A2401%2E03497%2E+%E2%80%94+%5BarXiv%3A2401%2E03497%5D%28https%3A)
23. Xiaokang Chen, Zhiyu Wu, Xingchao Liu, Zizheng Pan, Wen Liu, Zhenda Xie, Xingkai Yu, and Chong Ruan. 2025c. [Janus-pro: Unified multimodal understanding and generation with data and model scaling](https://md.research.elamur.ai/free/en/papers/chen-2025-janus-pro-unified-multimodal/). arXiv preprint arXiv:2501.17811.
24. Jonathan H Clark, Dan Garrette, Iulia Turc, and John Wieting. 2022. [Canine: Pre-training an efficient tokenization-free encoder for language representation](https://md.research.elamur.ai/free/en/papers/clark-2022-canine-pre-training-efficient-tokenization/). Transactions of the Association for Computational Linguistics, 10:73–91.
25. Alexis Conneau, Kartikay Khandelwal, Naman Goyal, Vishrav Chaudhary, Guillaume Wenzek, Francisco Guzmán, Edouard Grave, Myle Ott, Luke Zettlemoyer, and Veselin Stoyanov. 2019. [Unsupervised cross-lingual representation learning at scale](https://md.research.elamur.ai/free/en/papers/conneau-2020-unsupervised-cross-lingual-representation-learning/). arXiv preprint arXiv:1911.02116.
26. Wenliang Dai, Junnan Li, Dongxu Li, Anthony Tiong, Junqi Zhao, Weisheng Wang, Boyang Li, Pascale N Fung, and Steven Hoi. 2023. [Instructblip: Towards general-purpose vision-language models with instruction tuning](https://md.research.elamur.ai/free/en/papers/dai-2023-instructblip-towards-general-purpose-vision/). Advances in neural information processing systems, 36:49250–49267.
27. Alexandre Défossez, Jade Copet, Gabriel Synnaeve, and Yossi Adi. 2022. [High fidelity neural audio compression](https://md.research.elamur.ai/free/en/papers/defossez-2022-high-fidelity-neural-audio-compression/). arXiv preprint arXiv:2210.13438.
28. Chaorui Deng, Deyao Zhu, Kunchang Li, Chenhui Gou, Feng Li, Zeyu Wang, Shu Zhong, Weihao Yu, Xiaonan Nie, Ziang Song, and 1 others. 2025. Emerging properties in unified multimodal pretraining. arXiv preprint arXiv:2505.14683. — [arXiv:2505.14683](https://arxiv.org/abs/2505.14683) [find](https://scholar.google.com/scholar?q=Chaorui+Deng%2C+Deyao+Zhu%2C+Kunchang+Li%2C+Chenhui+Gou%2C+Feng+Li%2C+Zeyu+Wang%2C+Shu+Zhong%2C+Weihao+Yu%2C+Xiaonan+Nie%2C+Ziang+Song%2C+and+1+others%2E+2025%2E+Emerging+properties+in+unified+multimodal+pretraining%2E+arXiv+p)
29. Patrick Esser, Sumith Kulal, Andreas Blattmann, Rahim Entezari, Jonas Müller, Harry Saini, Yam Levi, Dominik Lorenz, Axel Sauer, Frederic Boesel, and 1 others. 2024. Scaling rectified flow transformers for high-resolution image synthesis. In Forty-first international conference on machine learning. [find](https://scholar.google.com/scholar?q=Patrick+Esser%2C+Sumith+Kulal%2C+Andreas+Blattmann%2C+Rahim+Entezari%2C+Jonas+M%C3%BCller%2C+Harry+Saini%2C+Yam+Levi%2C+Dominik+Lorenz%2C+Axel+Sauer%2C+Frederic+Boesel%2C+and+1+others%2E+2024%2E+Scaling+rectified+flow+transformer)
30. Patrick Esser, Robin Rombach, and Bjorn Ommer. 2021. Taming transformers for high-resolution image synthesis. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 12873–12883. [find](https://scholar.google.com/scholar?q=Patrick+Esser%2C+Robin+Rombach%2C+and+Bjorn+Ommer%2E+2021%2E+Taming+transformers+for+high%2Dresolution+image+synthesis%2E+In+Proceedings+of+the+IEEE%2FCVF+conference+on+computer+vision+and+pattern+recognition%2C+page)
31. Lijie Fan, Luming Tang, Siyang Qin, Tianhong Li, Xuan Yang, Siyuan Qiao, Andreas Steiner, Chen Sun, Yuanzhen Li, Tao Zhu, and 1 others. 2025. Unified autoregressive visual generation and understanding with continuous tokens. arXiv preprint arXiv:2503.13436. — [arXiv:2503.13436](https://arxiv.org/abs/2503.13436) [find](https://scholar.google.com/scholar?q=Lijie+Fan%2C+Luming+Tang%2C+Siyang+Qin%2C+Tianhong+Li%2C+Xuan+Yang%2C+Siyuan+Qiao%2C+Andreas+Steiner%2C+Chen+Sun%2C+Yuanzhen+Li%2C+Tao+Zhu%2C+and+1+others%2E+2025%2E+Unified+autoregressive+visual+generation+and+understanding)
32. Yuying Ge, Sijie Zhao, Ziyun Zeng, Yixiao Ge, Chen Li, Xintao Wang, and Ying Shan. 2023. Making llama see and draw with seed tokenizer. arXiv preprint arXiv:2310.01218. — [arXiv:2310.01218](https://arxiv.org/abs/2310.01218) [find](https://scholar.google.com/scholar?q=Yuying+Ge%2C+Sijie+Zhao%2C+Ziyun+Zeng%2C+Yixiao+Ge%2C+Chen+Li%2C+Xintao+Wang%2C+and+Ying+Shan%2E+2023%2E+Making+llama+see+and+draw+with+seed+tokenizer%2E+arXiv+preprint+arXiv%3A2310%2E01218%2E+%E2%80%94+%5BarXiv%3A2310%2E01218%5D%28https%3A%2F%2Far)
33. Yuying Ge, Sijie Zhao, Jinguo Zhu, Yixiao Ge, Kun Yi, Lin Song, Chen Li, Xiaohan Ding, and Ying Shan. 2024. Seed-x: Multimodal models with unified multi-granularity comprehension and generation. arXiv preprint arXiv:2404.14396. — [arXiv:2404.14396](https://arxiv.org/abs/2404.14396) [find](https://scholar.google.com/scholar?q=Yuying+Ge%2C+Sijie+Zhao%2C+Jinguo+Zhu%2C+Yixiao+Ge%2C+Kun+Yi%2C+Lin+Song%2C+Chen+Li%2C+Xiaohan+Ding%2C+and+Ying+Shan%2E+2024%2E+Seed%2Dx%3A+Multimodal+models+with+unified+multi%2Dgranularity+comprehension+and+generation%2E+arXiv)
34. Zigang Geng, Yibing Wang, Yeyao Ma, Chen Li, Yongming Rao, Shuyang Gu, Zhao Zhong, Qinglin Lu, Han Hu, Xiaosong Zhang, and 1 others. 2025. X-omni: Reinforcement learning makes discrete autoregressive image generative models great again. arXiv preprint arXiv:2507.22058. — [arXiv:2507.22058](https://arxiv.org/abs/2507.22058) [find](https://scholar.google.com/scholar?q=Zigang+Geng%2C+Yibing+Wang%2C+Yeyao+Ma%2C+Chen+Li%2C+Yongming+Rao%2C+Shuyang+Gu%2C+Zhao+Zhong%2C+Qinglin+Lu%2C+Han+Hu%2C+Xiaosong+Zhang%2C+and+1+others%2E+2025%2E+X%2Domni%3A+Reinforcement+learning+makes+discrete+autoregressive+)
35. Dhruba Ghosh, Hannaneh Hajishirzi, and Ludwig Schmidt. 2023. Geneval: An object-focused framework for evaluating text-to-image alignment. Advances in Neural Information Processing Systems, 36:52132–52152. [find](https://scholar.google.com/scholar?q=Dhruba+Ghosh%2C+Hannaneh+Hajishirzi%2C+and+Ludwig+Schmidt%2E+2023%2E+Geneval%3A+An+object%2Dfocused+framework+for+evaluating+text%2Dto%2Dimage+alignment%2E+Advances+in+Neural+Information+Processing+Systems%2C+36%3A52132%E2%80%9352)
36. Yash Goyal, Tejas Khot, Douglas Summers-Stay, Dhruv Batra, and Devi Parikh. 2017. [Making the v in vqa matter: Elevating the role of image understanding in visual question answering](https://md.research.elamur.ai/free/en/papers/goyal-2016-making-v-vqa-matter-elevating/). In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 6904–6913.
37. Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, and 1 others. 2024. [The llama 3 herd of models](https://md.research.elamur.ai/free/en/papers/grattafiori-2024-llama-3-herd-models/). arXiv preprint arXiv:2407.21783.
38. Jian Han, Jinlai Liu, Yi Jiang, Bin Yan, Yuqi Zhang, Zehuan Yuan, Bingyue Peng, and Xiaobing Liu. 2025. Infinity: Scaling bitwise autoregressive modeling for high-resolution image synthesis. In Proceedings of the Computer Vision and Pattern Recognition Conference, pages 15733–15744. [find](https://scholar.google.com/scholar?q=Jian+Han%2C+Jinlai+Liu%2C+Yi+Jiang%2C+Bin+Yan%2C+Yuqi+Zhang%2C+Zehuan+Yuan%2C+Bingyue+Peng%2C+and+Xiaobing+Liu%2E+2025%2E+Infinity%3A+Scaling+bitwise+autoregressive+modeling+for+high%2Dresolution+image+synthesis%2E+In+Procee)
39. Philippe Hansen-Estruch, David Yan, Ching-Yao Chung, Orr Zohar, Jialiang Wang, Tingbo Hou, Tao Xu, Sriram Vishwanath, Peter Vajda, and Xinlei Chen. 2025. Learnings from scaling visual tokenizers for reconstruction and generation. arXiv preprint arXiv:2501.09755. — [arXiv:2501.09755](https://arxiv.org/abs/2501.09755) [find](https://scholar.google.com/scholar?q=Philippe+Hansen%2DEstruch%2C+David+Yan%2C+Ching%2DYao+Chung%2C+Orr+Zohar%2C+Jialiang+Wang%2C+Tingbo+Hou%2C+Tao+Xu%2C+Sriram+Vishwanath%2C+Peter+Vajda%2C+and+Xinlei+Chen%2E+2025%2E+Learnings+from+scaling+visual+tokenizers+for+r)
40. Jitai Hao, Hao Liu, Xinyan Xiao, Qiang Huang, and Jun Yu. 2025. Uni-x: Mitigating modality conflict with a two-end-separated architecture for unified multimodal models. arXiv preprint arXiv:2509.24365. — [arXiv:2509.24365](https://arxiv.org/abs/2509.24365) [find](https://scholar.google.com/scholar?q=Jitai+Hao%2C+Hao+Liu%2C+Xinyan+Xiao%2C+Qiang+Huang%2C+and+Jun+Yu%2E+2025%2E+Uni%2Dx%3A+Mitigating+modality+conflict+with+a+two%2Dend%2Dseparated+architecture+for+unified+multimodal+models%2E+arXiv+preprint+arXiv%3A2509%2E24365)
41. Xin He, Longhui Wei, Jianbo Ouyang, Lingxi Xie, and Qi Tian. 2025. Emma: Efficient multimodal understanding, generation, and editing with a unified architecture. arXiv preprint arXiv:2512.04810. — [arXiv:2512.04810](https://arxiv.org/abs/2512.04810) [find](https://scholar.google.com/scholar?q=Xin+He%2C+Longhui+Wei%2C+Jianbo+Ouyang%2C+Lingxi+Xie%2C+and+Qi+Tian%2E+2025%2E+Emma%3A+Efficient+multimodal+understanding%2C+generation%2C+and+editing+with+a+unified+architecture%2E+arXiv+preprint+arXiv%3A2512%2E04810%2E+%E2%80%94+%5Bar)
42. Xiwei Hu, Rui Wang, Yixiao Fang, Bin Fu, Pei Cheng, and Gang Yu. 2024. Ella: Equip diffusion models with llm for enhanced semantic alignment. arXiv preprint arXiv:2403.05135. — [arXiv:2403.05135](https://arxiv.org/abs/2403.05135) [find](https://scholar.google.com/scholar?q=Xiwei+Hu%2C+Rui+Wang%2C+Yixiao+Fang%2C+Bin+Fu%2C+Pei+Cheng%2C+and+Gang+Yu%2E+2024%2E+Ella%3A+Equip+diffusion+models+with+llm+for+enhanced+semantic+alignment%2E+arXiv+preprint+arXiv%3A2403%2E05135%2E+%E2%80%94+%5BarXiv%3A2403%2E05135%5D%28http)
43. Minyoung Huh, Brian Cheung, Tongzhou Wang, and Phillip Isola. 2024. [The platonic representation hypothesis](https://md.research.elamur.ai/free/en/papers/huh-2024-platonic-representation-hypothesis/). arXiv preprint arXiv:2405.07987.
44. Team Hunyuan3D, Bowen Zhang, Chunchao Guo, Haolin Liu, Hongyu Yan, Huiwen Shi, Jingwei Huang, Junlin Yu, Kunhong Li, Penghao Wang, and 1 others. 2025. Hunyuan3d-omni: A unified framework for controllable generation of 3d assets. arXiv preprint arXiv:2509.21245. — [arXiv:2509.21245](https://arxiv.org/abs/2509.21245) [find](https://scholar.google.com/scholar?q=Team+Hunyuan3D%2C+Bowen+Zhang%2C+Chunchao+Guo%2C+Haolin+Liu%2C+Hongyu+Yan%2C+Huiwen+Shi%2C+Jingwei+Huang%2C+Junlin+Yu%2C+Kunhong+Li%2C+Penghao+Wang%2C+and+1+others%2E+2025%2E+Hunyuan3d%2Domni%3A+A+unified+framework+for+controlla)
45. Shengpeng Ji, Ziyue Jiang, Wen Wang, Yifu Chen, Minghui Fang, Jialong Zuo, Qian Yang, Xize Cheng, Zehan Wang, Ruiqi Li, Ziang Zhang, Xiaoda Yang, Rongjie Huang, Yidi Jiang, Qian Chen, Siqi Zheng, and Zhou Zhao. 2025. Wavtokenizer: an efficient acoustic discrete codec tokenizer for audio language mod… [find](https://scholar.google.com/scholar?q=Shengpeng+Ji%2C+Ziyue+Jiang%2C+Wen+Wang%2C+Yifu+Chen%2C+Minghui+Fang%2C+Jialong+Zuo%2C+Qian+Yang%2C+Xize+Cheng%2C+Zehan+Wang%2C+Ruiqi+Li%2C+Ziang+Zhang%2C+Xiaoda+Yang%2C+Rongjie+Huang%2C+Yidi+Jiang%2C+Qian+Chen%2C+Siqi+Zheng%2C+and+)
46. Chao Jia, Yinfei Yang, Ye Xia, Yi-Ting Chen, Zarana Parekh, Hieu Pham, Quoc Le, Yun-Hsuan Sung, Zhen Li, and Tom Duerig. 2021. [Scaling up visual and vision-language representation learning with noisy text supervision](https://md.research.elamur.ai/free/en/papers/jia-2021-scaling-up-visual-vision-language/). In International conference on machine learning, pages 4904–4916. PMLR.
47. Jiyue Jiang, Zikang Wang, Yuheng Shan, Heyan Chai, Jiayi Li, Zixian Ma, Xinrui Zhang, and Yu Li. 2025a. Biological sequence with language model prompting: A survey. arXiv preprint arXiv:2503.04135. — [arXiv:2503.04135](https://arxiv.org/abs/2503.04135) [find](https://scholar.google.com/scholar?q=Jiyue+Jiang%2C+Zikang+Wang%2C+Yuheng+Shan%2C+Heyan+Chai%2C+Jiayi+Li%2C+Zixian+Ma%2C+Xinrui+Zhang%2C+and+Yu+Li%2E+2025a%2E+Biological+sequence+with+language+model+prompting%3A+A+survey%2E+arXiv+preprint+arXiv%3A2503%2E04135%2E+%E2%80%94+)
48. Shixin Jiang, Jiafeng Liang, Jiyuan Wang, Xuan Dong, Heng Chang, Weijiang Yu, Jinhua Du, Ming Liu, and Bing Qin. 2025b. From specific-mllms to omni-mllms: a survey on mllms aligned with multi-modalities. In Findings of the Association for Computational Linguistics: ACL 2025, pages 8617–8652. [find](https://scholar.google.com/scholar?q=Shixin+Jiang%2C+Jiafeng+Liang%2C+Jiyuan+Wang%2C+Xuan+Dong%2C+Heng+Chang%2C+Weijiang+Yu%2C+Jinhua+Du%2C+Ming+Liu%2C+and+Bing+Qin%2E+2025b%2E+From+specific%2Dmllms+to+omni%2Dmllms%3A+a+survey+on+mllms+aligned+with+multi%2Dmodaliti)
49. Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. 2020. [Measuring massive multitask language understanding](https://md.research.elamur.ai/free/en/papers/hendrycks-2020-measuring-massive-multitask-language-understanding/). arXiv preprint arXiv:2009.03300.
50. Adam Tauman Kalai, Ofir Nachum, Santosh S Vempala, and Edwin Zhang. 2025. Why language models hallucinate. arXiv preprint arXiv:2509.04664. — [arXiv:2509.04664](https://arxiv.org/abs/2509.04664) [find](https://scholar.google.com/scholar?q=Adam+Tauman+Kalai%2C+Ofir+Nachum%2C+Santosh+S+Vempala%2C+and+Edwin+Zhang%2E+2025%2E+Why+language+models+hallucinate%2E+arXiv+preprint+arXiv%3A2509%2E04664%2E+%E2%80%94+%5BarXiv%3A2509%2E04664%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2509%2E04664%29)
51. Jonathan Ho, Ajay Jain, and Pieter Abbeel. 2020. Denoising diffusion probabilistic models. Advances in neural information processing systems, 33:6840– 6851. [find](https://scholar.google.com/scholar?q=Jonathan+Ho%2C+Ajay+Jain%2C+and+Pieter+Abbeel%2E+2020%2E+Denoising+diffusion+probabilistic+models%2E+Advances+in+neural+information+processing+systems%2C+33%3A6840%E2%80%93+6851%2E)
52. Jinwoo Kim, Dat Nguyen, Seonwoo Min, Sungjun Cho, Moontae Lee, Honglak Lee, and Seunghoon Hong. 2022. Pure transformers are powerful graph learners. Advances in Neural Information Processing Systems, 35:14582–14595. [find](https://scholar.google.com/scholar?q=Jinwoo+Kim%2C+Dat+Nguyen%2C+Seonwoo+Min%2C+Sungjun+Cho%2C+Moontae+Lee%2C+Honglak+Lee%2C+and+Seunghoon+Hong%2E+2022%2E+Pure+transformers+are+powerful+graph+learners%2E+Advances+in+Neural+Information+Processing+Systems%2C+)
53. Jack Hong, Shilin Yan, Jiayin Cai, Xiaolong Jiang, Yao Hu, and Weidi Xie. 2025. Worldsense: Evaluating real-world omnimodal understanding for multimodal llms. arXiv preprint arXiv:2502.04326. — [arXiv:2502.04326](https://arxiv.org/abs/2502.04326) [find](https://scholar.google.com/scholar?q=Jack+Hong%2C+Shilin+Yan%2C+Jiayin+Cai%2C+Xiaolong+Jiang%2C+Yao+Hu%2C+and+Weidi+Xie%2E+2025%2E+Worldsense%3A+Evaluating+real%2Dworld+omnimodal+understanding+for+multimodal+llms%2E+arXiv+preprint+arXiv%3A2502%2E04326%2E+%E2%80%94+%5BarXiv)
54. Taku Kudo. 2018. [Subword regularization: Improving neural network translation models with multiple subword candidates](https://md.research.elamur.ai/free/en/papers/kudo-2018-subword-regularization-improving-neural-network/). arXiv preprint arXiv:1804.10959.
55. Wei-Ning Hsu, Benjamin Bolte, Yao-Hung Hubert Tsai, Kushal Lakhotia, Ruslan Salakhutdinov, and Abdelrahman Mohamed. 2021. [Hubert: Self-supervised speech representation learning by masked prediction of hidden units](https://md.research.elamur.ai/free/en/papers/hsu-2021-hubert-self-supervised-speech-representation/). IEEE/ACM transactions on audio, speech, and language processing, 29:3451–3460.
56. Taku Kudo and John Richardson. 2018. [Sentencepiece: A simple and language independent subword tokenizer and detokenizer for neural text processing](https://md.research.elamur.ai/free/en/papers/kudo-2018-sentencepiece-simple-language-independent-subword/). arXiv preprint arXiv:1808.06226.
57. Doyup Lee, Chiheon Kim, Saehoon Kim, Minsu Cho, and Wook-Shin Han. 2022. Autoregressive image generation using residual quantization. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 11523–11532. [find](https://scholar.google.com/scholar?q=Doyup+Lee%2C+Chiheon+Kim%2C+Saehoon+Kim%2C+Minsu+Cho%2C+and+Wook%2DShin+Han%2E+2022%2E+Autoregressive+image+generation+using+residual+quantization%2E+In+Proceedings+of+the+IEEE%2FCVF+conference+on+computer+vision+and+p)
58. Bohao Li, Rui Wang, Guangzhi Wang, Yuying Ge, Yixiao Ge, and Ying Shan. 2023a. [Seed-bench: Benchmarking multimodal llms with generative comprehension](https://md.research.elamur.ai/free/en/papers/li-2023-seed-bench-benchmarking-multimodal-llms/). arXiv preprint arXiv:2307.16125.
59. Feng Li, Renrui Zhang, Hao Zhang, Yuanhan Zhang, Bo Li, Wei Li, Zejun Ma, and Chunyuan Li. 2024a. [Llava-next-interleave: Tackling multi-image, video, and 3d in large multimodal models](https://md.research.elamur.ai/free/en/papers/li-2024-llava-next-interleave-tackling-multi/). arXiv preprint arXiv:2407.07895.
60. Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi. 2023b. [Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models](https://md.research.elamur.ai/free/en/papers/li-2023-blip-2-bootstrapping-language-image/). In International conference on machine learning, pages 19730–19742. PMLR.
61. Junnan Li, Dongxu Li, Caiming Xiong, and Steven Hoi. 2022. [Blip: Bootstrapping language-image pre-training for unified vision-language understanding and generation](https://md.research.elamur.ai/free/en/papers/li-2022-blip-bootstrapping-language-image-pre/). In International conference on machine learning, pages 12888–12900. PMLR.
62. Tianhong Li, Yonglong Tian, He Li, Mingyang Deng, and Kaiming He. 2024b. Autoregressive image generation without vector quantization. Advances in Neural Information Processing Systems, 37:56424– 56445. [find](https://scholar.google.com/scholar?q=Tianhong+Li%2C+Yonglong+Tian%2C+He+Li%2C+Mingyang+Deng%2C+and+Kaiming+He%2E+2024b%2E+Autoregressive+image+generation+without+vector+quantization%2E+Advances+in+Neural+Information+Processing+Systems%2C+37%3A56424%E2%80%93+56445)
63. Yanghao Li, Rui Qian, Bowen Pan, Haotian Zhang, Haoshuo Huang, Bowen Zhang, Jialing Tong, Haoxuan You, Xianzhi Du, Zhe Gan, and 1 others. 2025a. Manzano: A simple and scalable unified multimodal model with a hybrid vision tokenizer. arXiv preprint arXiv:2509.16197. — [arXiv:2509.16197](https://arxiv.org/abs/2509.16197) [find](https://scholar.google.com/scholar?q=Yanghao+Li%2C+Rui+Qian%2C+Bowen+Pan%2C+Haotian+Zhang%2C+Haoshuo+Huang%2C+Bowen+Zhang%2C+Jialing+Tong%2C+Haoxuan+You%2C+Xianzhi+Du%2C+Zhe+Gan%2C+and+1+others%2E+2025a%2E+Manzano%3A+A+simple+and+scalable+unified+multimodal+model)
64. Yi Li, Haonan Wang, Qixiang Zhang, Boyu Xiao, Chenchang Hu, Hualiang Wang, and Xiaomeng Li. 2025b. Unieval: Unified holistic evaluation for unified multimodal understanding and generation. arXiv preprint arXiv:2505.10483. — [arXiv:2505.10483](https://arxiv.org/abs/2505.10483) [find](https://scholar.google.com/scholar?q=Yi+Li%2C+Haonan+Wang%2C+Qixiang+Zhang%2C+Boyu+Xiao%2C+Chenchang+Hu%2C+Hualiang+Wang%2C+and+Xiaomeng+Li%2E+2025b%2E+Unieval%3A+Unified+holistic+evaluation+for+unified+multimodal+understanding+and+generation%2E+arXiv+prepr)
65. Chao Liao, Liyang Liu, Xun Wang, Zhengxiong Luo, Xinyu Zhang, Wenliang Zhao, Jie Wu, Liang Li, Zhi Tian, and Weilin Huang. 2025. Mogao: An omni foundation model for interleaved multi-modal generation. arXiv preprint arXiv:2505.05472. — [arXiv:2505.05472](https://arxiv.org/abs/2505.05472) [find](https://scholar.google.com/scholar?q=Chao+Liao%2C+Liyang+Liu%2C+Xun+Wang%2C+Zhengxiong+Luo%2C+Xinyu+Zhang%2C+Wenliang+Zhao%2C+Jie+Wu%2C+Liang+Li%2C+Zhi+Tian%2C+and+Weilin+Huang%2E+2025%2E+Mogao%3A+An+omni+foundation+model+for+interleaved+multi%2Dmodal+generation%2E)
66. Zhiqiu Lin, Deepak Pathak, Baiqi Li, Jiayao Li, Xide Xia, Graham Neubig, Pengchuan Zhang, and Deva Ramanan. 2024b. Evaluating text-to-visual generation with image-to-text generation. In European Conference on Computer Vision, pages 366–384. Springer. [find](https://scholar.google.com/scholar?q=Zhiqiu+Lin%2C+Deepak+Pathak%2C+Baiqi+Li%2C+Jiayao+Li%2C+Xide+Xia%2C+Graham+Neubig%2C+Pengchuan+Zhang%2C+and+Deva+Ramanan%2E+2024b%2E+Evaluating+text%2Dto%2Dvisual+generation+with+image%2Dto%2Dtext+generation%2E+In+European+Confe)
67. Yaron Lipman, Ricky TQ Chen, Heli Ben-Hamu, Maximilian Nickel, and Matt Le. 2022. [Flow matching for generative modeling](https://md.research.elamur.ai/free/en/papers/lipman-2022-flow-matching-generative-modeling/). arXiv preprint arXiv:2210.02747.
68. Dongyang Liu, Shitian Zhao, Le Zhuo, Weifeng Lin, Yi Xin, Xinyue Li, Qi Qin, Yu Qiao, Hongsheng Li, and Peng Gao. 2024a. Lumina-mgpt: Illuminate flexible photorealistic text-to-image generation with multimodal generative pretraining. arXiv preprint arXiv:2408.02657. — [arXiv:2408.02657](https://arxiv.org/abs/2408.02657) [find](https://scholar.google.com/scholar?q=Dongyang+Liu%2C+Shitian+Zhao%2C+Le+Zhuo%2C+Weifeng+Lin%2C+Yi+Xin%2C+Xinyue+Li%2C+Qi+Qin%2C+Yu+Qiao%2C+Hongsheng+Li%2C+and+Peng+Gao%2E+2024a%2E+Lumina%2Dmgpt%3A+Illuminate+flexible+photorealistic+text%2Dto%2Dimage+generation+with+m)
69. Haotian Liu, Chunyuan Li, Yuheng Li, and Yong Jae Lee. 2024b. [Improved baselines with visual instruction tuning](https://md.research.elamur.ai/free/en/papers/liu-2023-improved-baselines-visual-instruction-tuning/). In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 26296–26306.
70. Haotian Liu, Chunyuan Li, Yuheng Li, Bo Li, Yuanhan Zhang, Sheng Shen, and Yong Jae Lee. 2024c. Llava-next: Improved reasoning, ocr, and world knowledge. [find](https://scholar.google.com/scholar?q=Haotian+Liu%2C+Chunyuan+Li%2C+Yuheng+Li%2C+Bo+Li%2C+Yuanhan+Zhang%2C+Sheng+Shen%2C+and+Yong+Jae+Lee%2E+2024c%2E+Llava%2Dnext%3A+Improved+reasoning%2C+ocr%2C+and+world+knowledge%2E)
71. Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. 2023. [Visual instruction tuning](https://md.research.elamur.ai/free/en/papers/liu-2023-visual-instruction-tuning/). Advances in neural information processing systems, 36:34892– 34916.
72. Kai Liu, Jungang Li, Yuchong Sun, Shengqiong Wu, Jianzhang Gao, Daoan Zhang, Wei Zhang, Sheng Jin, Sicheng Yu, Geng Zhan, and 1 others. 2025a. Javisgpt: A unified multi-modal llm for sounding-video comprehension and generation. arXiv preprint arXiv:2512.22905. — [arXiv:2512.22905](https://arxiv.org/abs/2512.22905) [find](https://scholar.google.com/scholar?q=Kai+Liu%2C+Jungang+Li%2C+Yuchong+Sun%2C+Shengqiong+Wu%2C+Jianzhang+Gao%2C+Daoan+Zhang%2C+Wei+Zhang%2C+Sheng+Jin%2C+Sicheng+Yu%2C+Geng+Zhan%2C+and+1+others%2E+2025a%2E+Javisgpt%3A+A+unified+multi%2Dmodal+llm+for+sounding%2Dvideo+co)
73. Shiyu Liu, Yucheng Han, Peng Xing, Fukun Yin, Rui Wang, Wei Cheng, Jiaqi Liao, Yingming Wang, Honghao Fu, Chunrui Han, and 1 others. 2025b. [Step1x-edit: A practical framework for general image editing](https://md.research.elamur.ai/free/en/papers/liu-2025-step1x-edit-practical-framework/). arXiv preprint arXiv:2504.17761.
74. Xingchao Liu, Chengyue Gong, and Qiang Liu. 2022. [Flow straight and fast: Learning to generate and transfer data with rectified flow](https://md.research.elamur.ai/free/en/papers/liu-2022-flow-straight-fast-learning-generate/). arXiv preprint arXiv:2209.03003.
75. Xi Victoria Lin, Akshat Shrivastava, Liang Luo, Srinivasan Iyer, Mike Lewis, Gargi Ghosh, Luke Zettlemoyer, and Armen Aghajanyan. 2024a. Moma: Efficient early-fusion pre-training with mixture of modality-aware experts. arXiv preprint arXiv:2407.21770. [find](https://scholar.google.com/scholar?q=Xi+Victoria+Lin%2C+Akshat+Shrivastava%2C+Liang+Luo%2C+Srinivasan+Iyer%2C+Mike+Lewis%2C+Gargi+Ghosh%2C+Luke+Zettlemoyer%2C+and+Armen+Aghajanyan%2E+2024a%2E+Moma%3A+Efficient+early%2Dfusion+pre%2Dtraining+with+mixture+of+modal)
76. Yuan Liu, Haodong Duan, Yuanhan Zhang, Bo Li, Songyang Zhang, Wangbo Zhao, Yike Yuan, Jiaqi Wang, Conghui He, Ziwei Liu, and 1 others. 2024d. [Mmbench: Is your multi-modal model an all-around player?](https://md.research.elamur.ai/free/en/papers/liu-2023-mmbench-multi-modal-model-all/) In European conference on computer vision, pages 216–233. Springer.
77. Zeming Lin, Halil Akin, Roshan Rao, Brian Hie, Zhongkai Zhu, Wenting Lu, Allan dos Santos Costa, Maryam Fazel-Zarandi, Tom Sercu, Sal Candido, and 1 others. 2022. Language models of protein sequences at the scale of evolution enable accurate structure prediction. BioRxiv, 2022:500902. [find](https://scholar.google.com/scholar?q=Zeming+Lin%2C+Halil+Akin%2C+Roshan+Rao%2C+Brian+Hie%2C+Zhongkai+Zhu%2C+Wenting+Lu%2C+Allan+dos+Santos+Costa%2C+Maryam+Fazel%2DZarandi%2C+Tom+Sercu%2C+Sal+Candido%2C+and+1+others%2E+2022%2E+Language+models+of+protein+sequences+)
78. Zhiheng Liu, Weiming Ren, Haozhe Liu, Zijian Zhou, Shoufa Chen, Haonan Qiu, Xiaoke Huang, Zhao-chong An, Fanny Yang, Aditya Patel, and 1 others. 2025c. Tuna: Taming unified visual representations for native unified multimodal models. arXiv preprint arXiv:2512.02014. — [arXiv:2512.02014](https://arxiv.org/abs/2512.02014) [find](https://scholar.google.com/scholar?q=Zhiheng+Liu%2C+Weiming+Ren%2C+Haozhe+Liu%2C+Zijian+Zhou%2C+Shoufa+Chen%2C+Haonan+Qiu%2C+Xiaoke+Huang%2C+Zhao%2Dchong+An%2C+Fanny+Yang%2C+Aditya+Patel%2C+and+1+others%2E+2025c%2E+Tuna%3A+Taming+unified+visual+representations+for+)
79. Cheng Lu, Yuhao Zhou, Fan Bao, Jianfei Chen, Chongxuan Li, and Jun Zhu. 2022. Dpm-solver: A fast ode solver for diffusion probabilistic model sampling in around 10 steps. Advances in neural information processing systems, 35:5775–5787. [find](https://scholar.google.com/scholar?q=Cheng+Lu%2C+Yuhao+Zhou%2C+Fan+Bao%2C+Jianfei+Chen%2C+Chongxuan+Li%2C+and+Jun+Zhu%2E+2022%2E+Dpm%2Dsolver%3A+A+fast+ode+solver+for+diffusion+probabilistic+model+sampling+in+around+10+steps%2E+Advances+in+neural+informatio)
80. Jiasen Lu, Liangchen Song, Mingze Xu, Byeongjoo Ahn, Yanjun Wang, Chen Chen, Afshin Dehghan, and Yinfei Yang. 2025. Atoken: A unified tokenizer for vision. arXiv preprint arXiv:2509.14476. — [arXiv:2509.14476](https://arxiv.org/abs/2509.14476) [find](https://scholar.google.com/scholar?q=Jiasen+Lu%2C+Liangchen+Song%2C+Mingze+Xu%2C+Byeongjoo+Ahn%2C+Yanjun+Wang%2C+Chen+Chen%2C+Afshin+Dehghan%2C+and+Yinfei+Yang%2E+2025%2E+Atoken%3A+A+unified+tokenizer+for+vision%2E+arXiv+preprint+arXiv%3A2509%2E14476%2E+%E2%80%94+%5BarXiv%3A25)
81. Liuzhenghao Lv, Zongying Lin, Hao Li, Yuyang Liu, Jiaxi Cui, Calvin Yu-Chian Chen, Li Yuan, and Yonghong Tian. 2025. Prollama: A protein large language model for multi-task protein language processing. IEEE Transactions on Artificial Intelligence. [find](https://scholar.google.com/scholar?q=Liuzhenghao+Lv%2C+Zongying+Lin%2C+Hao+Li%2C+Yuyang+Liu%2C+Jiaxi+Cui%2C+Calvin+Yu%2DChian+Chen%2C+Li+Yuan%2C+and+Yonghong+Tian%2E+2025%2E+Prollama%3A+A+protein+large+language+model+for+multi%2Dtask+protein+language+processing)
82. Chuofan Ma, Yi Jiang, Junfeng Wu, Jihan Yang, Xin Yu, Zehuan Yuan, Bingyue Peng, and Xiaojuan Qi. 2025a. Unitok: A unified tokenizer for visual generation and understanding. arXiv preprint arXiv:2502.20321. — [arXiv:2502.20321](https://arxiv.org/abs/2502.20321) [find](https://scholar.google.com/scholar?q=Chuofan+Ma%2C+Yi+Jiang%2C+Junfeng+Wu%2C+Jihan+Yang%2C+Xin+Yu%2C+Zehuan+Yuan%2C+Bingyue+Peng%2C+and+Xiaojuan+Qi%2E+2025a%2E+Unitok%3A+A+unified+tokenizer+for+visual+generation+and+understanding%2E+arXiv+preprint+arXiv%3A2502%2E)
83. Xu Ma, Can Qin, Haoxuan You, Haoxi Ran, and Yun Fu. 2022. Rethinking network design and local geometry in point cloud: A simple residual mlp framework. arXiv preprint arXiv:2202.07123. — [arXiv:2202.07123](https://arxiv.org/abs/2202.07123) [find](https://scholar.google.com/scholar?q=Xu+Ma%2C+Can+Qin%2C+Haoxuan+You%2C+Haoxi+Ran%2C+and+Yun+Fu%2E+2022%2E+Rethinking+network+design+and+local+geometry+in+point+cloud%3A+A+simple+residual+mlp+framework%2E+arXiv+preprint+arXiv%3A2202%2E07123%2E+%E2%80%94+%5BarXiv%3A2202%2E0)
84. Xu Ma, Peize Sun, Haoyu Ma, Hao Tang, Chih-Yao Ma, Jialiang Wang, Kunpeng Li, Xiaoliang Dai, Yujun Shi, Xuan Ju, and 1 others. 2025b. Token-shuffle: Towards high-resolution image generation with autoregressive models. arXiv preprint arXiv:2504.17789. — [arXiv:2504.17789](https://arxiv.org/abs/2504.17789) [find](https://scholar.google.com/scholar?q=Xu+Ma%2C+Peize+Sun%2C+Haoyu+Ma%2C+Hao+Tang%2C+Chih%2DYao+Ma%2C+Jialiang+Wang%2C+Kunpeng+Li%2C+Xiaoliang+Dai%2C+Yujun+Shi%2C+Xuan+Ju%2C+and+1+others%2E+2025b%2E+Token%2Dshuffle%3A+Towards+high%2Dresolution+image+generation+with+autor)
85. Yiyang Ma, Xingchao Liu, Xiaokang Chen, Wen Liu, Chengyue Wu, Zhiyu Wu, Zizheng Pan, Zhenda Xie, Haowei Zhang, Xingkai Yu, and 1 others. 2025c. [Janusflow: Harmonizing autoregression and rectified flow for unified multimodal understanding and generation](https://md.research.elamur.ai/free/en/papers/ma-2024-janusflow-harmonizing-autoregression-rectified/). In Proceedings of the Computer Vision and Patt…
86. Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean. 2013. [Efficient estimation of word representations in vector space](https://md.research.elamur.ai/free/en/papers/mikolov-2013-efficient-estimation-word-representations-vector/). arXiv preprint arXiv:1301.3781.
87. Sicheng Mo, Thao Nguyen, Xun Huang, Siddharth Srinivasan Iyer, Yijun Li, Yuchen Liu, Abhishek Tandon, Eli Shechtman, Krishna Kumar Singh, Yong Jae Lee, and 1 others. 2025. X-fusion: Introducing new modality to frozen large language models. arXiv preprint arXiv:2504.20996. — [arXiv:2504.20996](https://arxiv.org/abs/2504.20996) [find](https://scholar.google.com/scholar?q=Sicheng+Mo%2C+Thao+Nguyen%2C+Xun+Huang%2C+Siddharth+Srinivasan+Iyer%2C+Yijun+Li%2C+Yuchen+Liu%2C+Abhishek+Tandon%2C+Eli+Shechtman%2C+Krishna+Kumar+Singh%2C+Yong+Jae+Lee%2C+and+1+others%2E+2025%2E+X%2Dfusion%3A+Introducing+new+mo)
88. Pooneh Mousavi, Gallil Maimon, Adel Moumen, Darius Petermann, Jiatong Shi, Haibin Wu, Haici Yang, Anastasia Kuznetsova, Artem Ploujnikov, Ricard Marxer, and 1 others. 2025. Discrete audio tokens: More than a survey! arXiv preprint arXiv:2506.10274. — [arXiv:2506.10274](https://arxiv.org/abs/2506.10274) [find](https://scholar.google.com/scholar?q=Pooneh+Mousavi%2C+Gallil+Maimon%2C+Adel+Moumen%2C+Darius+Petermann%2C+Jiatong+Shi%2C+Haibin+Wu%2C+Haici+Yang%2C+Anastasia+Kuznetsova%2C+Artem+Ploujnikov%2C+Ricard+Marxer%2C+and+1+others%2E+2025%2E+Discrete+audio+tokens%3A+More)
89. Yuwei Niu, Munan Ning, Mengren Zheng, Weiyang Jin, Bin Lin, Peng Jin, Jiaqi Liao, Chaoran Feng, Kunpeng Ning, Bin Zhu, and 1 others. 2025. Wise: A world knowledge-informed semantic evaluation for text-to-image generation. arXiv preprint arXiv:2503.07265. — [arXiv:2503.07265](https://arxiv.org/abs/2503.07265) [find](https://scholar.google.com/scholar?q=Yuwei+Niu%2C+Munan+Ning%2C+Mengren+Zheng%2C+Weiyang+Jin%2C+Bin+Lin%2C+Peng+Jin%2C+Jiaqi+Liao%2C+Chaoran+Feng%2C+Kunpeng+Ning%2C+Bin+Zhu%2C+and+1+others%2E+2025%2E+Wise%3A+A+world+knowledge%2Dinformed+semantic+evaluation+for+text)
90. Artidoro Pagnoni, Ramakanth Pasunuru, Pedro Rodriguez, John Nguyen, Benjamin Muller, Margaret Li, Chunting Zhou, Lili Yu, Jason E Weston, Luke Zettlemoyer, and 1 others. 2025. Byte latent transformer: Patches scale better than tokens. In Proceedings of the 63rd Annual Meeting of the Association for… [find](https://scholar.google.com/scholar?q=Artidoro+Pagnoni%2C+Ramakanth+Pasunuru%2C+Pedro+Rodriguez%2C+John+Nguyen%2C+Benjamin+Muller%2C+Margaret+Li%2C+Chunting+Zhou%2C+Lili+Yu%2C+Jason+E+Weston%2C+Luke+Zettlemoyer%2C+and+1+others%2E+2025%2E+Byte+latent+transformer%3A)
91. Xichen Pan, Satya Narayan Shukla, Aashu Singh, Zhuokai Zhao, Shlok Kumar Mishra, Jialiang Wang, Zhiyang Xu, Jiuhai Chen, Kunpeng Li, Felix Juefei-Xu, and 1 others. 2025. Transfer between modalities with metaqueries. arXiv preprint arXiv:2504.06256. — [arXiv:2504.06256](https://arxiv.org/abs/2504.06256) [find](https://scholar.google.com/scholar?q=Xichen+Pan%2C+Satya+Narayan+Shukla%2C+Aashu+Singh%2C+Zhuokai+Zhao%2C+Shlok+Kumar+Mishra%2C+Jialiang+Wang%2C+Zhiyang+Xu%2C+Jiuhai+Chen%2C+Kunpeng+Li%2C+Felix+Juefei%2DXu%2C+and+1+others%2E+2025%2E+Transfer+between+modalities+wi)
92. Yatian Pang, Eng Hock Francis Tay, Li Yuan, and Zhenghua Chen. 2023. Masked autoencoders for 3d point cloud self-supervised learning. World Scientific Annual Review of Artificial Intelligence, 1:2440001. [find](https://scholar.google.com/scholar?q=Yatian+Pang%2C+Eng+Hock+Francis+Tay%2C+Li+Yuan%2C+and+Zhenghua+Chen%2E+2023%2E+Masked+autoencoders+for+3d+point+cloud+self%2Dsupervised+learning%2E+World+Scientific+Annual+Review+of+Artificial+Intelligence%2C+1%3A24400)
93. Jeffrey Pennington, Richard Socher, and Christopher D Manning. 2014. [Glove: Global vectors for word representation](https://md.research.elamur.ai/free/en/papers/pennington-2014-glove-global-vectors-word-representation/). In Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP), pages 1532–1543.
94. Liao Qu, Huichao Zhang, Yiheng Liu, Xu Wang, Yi Jiang, Yiming Gao, Hu Ye, Daniel K Du, Zehuan Yuan, and Xinglong Wu. 2025. Tokenflow: Unified image tokenizer for multimodal understanding and generation. In Proceedings of the Computer Vision and Pattern Recognition Conference, pages 2545–2555. [find](https://scholar.google.com/scholar?q=Liao+Qu%2C+Huichao+Zhang%2C+Yiheng+Liu%2C+Xu+Wang%2C+Yi+Jiang%2C+Yiming+Gao%2C+Hu+Ye%2C+Daniel+K+Du%2C+Zehuan+Yuan%2C+and+Xinglong+Wu%2E+2025%2E+Tokenflow%3A+Unified+image+tokenizer+for+multimodal+understanding+and+generatio)
95. Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, and 1 others. 2021. [Learning transferable visual models from natural language supervision](https://md.research.elamur.ai/free/en/papers/radford-2021-learning-transferable-visual-models-natural/). In International conference on machine learning, pages 8748–8…
96. Alec Radford, Jong Wook Kim, Tao Xu, Greg Brockman, Christine McLeavey, and Ilya Sutskever. 2023. [Robust speech recognition via large-scale weak supervision](https://md.research.elamur.ai/free/en/papers/radford-2022-robust-speech-recognition-large-scale/). In International conference on machine learning, pages 28492–28518. PMLR.
97. Alec Radford, Karthik Narasimhan, Tim Salimans, Ilya Sutskever, and 1 others. 2018. Improving language understanding by generative pre-training. [find](https://scholar.google.com/scholar?q=Alec+Radford%2C+Karthik+Narasimhan%2C+Tim+Salimans%2C+Ilya+Sutskever%2C+and+1+others%2E+2018%2E+Improving+language+understanding+by+generative+pre%2Dtraining%2E)
98. Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, and 1 others. 2019. Language models are unsupervised multitask learners. OpenAI blog, 1(8):9. [find](https://scholar.google.com/scholar?q=Alec+Radford%2C+Jeffrey+Wu%2C+Rewon+Child%2C+David+Luan%2C+Dario+Amodei%2C+Ilya+Sutskever%2C+and+1+others%2E+2019%2E+Language+models+are+unsupervised+multitask+learners%2E+OpenAI+blog%2C+1%288%29%3A9%2E)
99. Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, and Mark Chen. 2022. [Hierarchical text-conditional image generation with clip latents](https://md.research.elamur.ai/free/en/papers/ramesh-2022-hierarchical-text-conditional-image-generation/). arXiv preprint arXiv:2204.06125, 1(2):3.
100. Aditya Ramesh, Mikhail Pavlov, Gabriel Goh, Scott Gray, Chelsea Voss, Alec Radford, Mark Chen, and Ilya Sutskever. 2021. [Zero-shot text-to-image generation](https://md.research.elamur.ai/free/en/papers/ramesh-2021-zero-shot-text-image-generation/). In International conference on machine learning, pages 8821–8831. Pmlr.
101. Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. 2022. [High-resolution image synthesis with latent diffusion models](https://md.research.elamur.ai/free/en/papers/rombach-2021-high-resolution-image-synthesis-latent/). In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10684–10695.
102. Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily L Denton, Kamyar Ghasemipour, Raphael Gontijo Lopes, Burcu Karagol Ayan, Tim Salimans, and 1 others. 2022. [Photorealistic text-to-image diffusion models with deep language understanding](https://md.research.elamur.ai/free/en/papers/saharia-2022-photorealistic-text-image-diffusion-models/). Advances in neural information processin…
103. Maarten Sap, Hannah Rashkin, Derek Chen, Ronan LeBras, and Yejin Choi. 2019. [Socialiqa: Commonsense reasoning about social interactions](https://md.research.elamur.ai/free/en/papers/sap-2019-socialiqa-commonsense-reasoning-about-social/). arXiv preprint arXiv:1904.09728.
104. Quan Sun, Yufeng Cui, Xiaosong Zhang, Fan Zhang, Qiying Yu, Yueze Wang, Yongming Rao, Jingjing Liu, Tiejun Huang, and Xinlong Wang. 2024b. Generative multimodal models are in-context learners. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 14398– 14409. [find](https://scholar.google.com/scholar?q=Quan+Sun%2C+Yufeng+Cui%2C+Xiaosong+Zhang%2C+Fan+Zhang%2C+Qiying+Yu%2C+Yueze+Wang%2C+Yongming+Rao%2C+Jingjing+Liu%2C+Tiejun+Huang%2C+and+Xinlong+Wang%2E+2024b%2E+Generative+multimodal+models+are+in%2Dcontext+learners%2E+In+Proc)
105. Quan Sun, Qiying Yu, Yufeng Cui, Fan Zhang, Xiaosong Zhang, Yueze Wang, Hongcheng Gao, Jingjing Liu, Tiejun Huang, and Xinlong Wang. 2023. Emu: Generative pretraining in multimodality. arXiv preprint arXiv:2307.05222. — [arXiv:2307.05222](https://arxiv.org/abs/2307.05222) [find](https://scholar.google.com/scholar?q=Quan+Sun%2C+Qiying+Yu%2C+Yufeng+Cui%2C+Fan+Zhang%2C+Xiaosong+Zhang%2C+Yueze+Wang%2C+Hongcheng+Gao%2C+Jingjing+Liu%2C+Tiejun+Huang%2C+and+Xinlong+Wang%2E+2023%2E+Emu%3A+Generative+pretraining+in+multimodality%2E+arXiv+preprint+)
106. Sabera Talukder, Yisong Yue, and Georgia Gkioxari. 2024. Totem: Tokenized time series embeddings for general time series analysis. arXiv preprint arXiv:2402.16412. — [arXiv:2402.16412](https://arxiv.org/abs/2402.16412) [find](https://scholar.google.com/scholar?q=Sabera+Talukder%2C+Yisong+Yue%2C+and+Georgia+Gkioxari%2E+2024%2E+Totem%3A+Tokenized+time+series+embeddings+for+general+time+series+analysis%2E+arXiv+preprint+arXiv%3A2402%2E16412%2E+%E2%80%94+%5BarXiv%3A2402%2E16412%5D%28https%3A%2F%2Farxiv%2Eo)
107. Rico Sennrich, Barry Haddow, and Alexandra Birch. 2015. [Neural machine translation of rare words with subword units](https://md.research.elamur.ai/free/en/papers/sennrich-2016-neural-machine-translation-rare-words/). arXiv preprint arXiv:1508.07909.
108. Chameleon Team. 2024. [Chameleon: Mixed-modal early-fusion foundation models](https://md.research.elamur.ai/free/en/papers/team-2024-chameleon-mixed-modal-early-fusion/). arXiv preprint arXiv:2405.09818.
109. Shelly Sheynin, Adam Polyak, Uriel Singer, Yuval Kirstain, Amit Zohar, Oron Ashual, Devi Parikh, and Yaniv Taigman. 2024. Emu edit: Precise image editing via recognition and generation tasks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 8871–8879. [find](https://scholar.google.com/scholar?q=Shelly+Sheynin%2C+Adam+Polyak%2C+Uriel+Singer%2C+Yuval+Kirstain%2C+Amit+Zohar%2C+Oron+Ashual%2C+Devi+Parikh%2C+and+Yaniv+Taigman%2E+2024%2E+Emu+edit%3A+Precise+image+editing+via+recognition+and+generation+tasks%2E+In+Proce)
110. Keyu Tian, Yi Jiang, Zehuan Yuan, Bingyue Peng, and Liwei Wang. 2024. Visual autoregressive modeling: Scalable image generation via next-scale prediction. Advances in neural information processing systems, 37:84839–84865. [find](https://scholar.google.com/scholar?q=Keyu+Tian%2C+Yi+Jiang%2C+Zehuan+Yuan%2C+Bingyue+Peng%2C+and+Liwei+Wang%2E+2024%2E+Visual+autoregressive+modeling%3A+Scalable+image+generation+via+next%2Dscale+prediction%2E+Advances+in+neural+information+processing+sys)
111. Weijia Shi, Xiaochuang Han, Chunting Zhou, Weixin Liang, Xi Victoria Lin, Luke Zettlemoyer, and Lili Yu. 2024. Lmfusion: Adapting pretrained language models for multimodal generation. arXiv preprint arXiv:2412.15188. [find](https://scholar.google.com/scholar?q=Weijia+Shi%2C+Xiaochuang+Han%2C+Chunting+Zhou%2C+Weixin+Liang%2C+Xi+Victoria+Lin%2C+Luke+Zettlemoyer%2C+and+Lili+Yu%2E+2024%2E+Lmfusion%3A+Adapting+pretrained+language+models+for+multimodal+generation%2E+arXiv+preprint+a)
112. Shengbang Tong, David Fan, Jiachen Zhu, Yunyang Xiong, Xinlei Chen, Koustuv Sinha, Michael Rabbat, Yann LeCun, Saining Xie, and Zhuang Liu. 2024. Metamorph: Multimodal understanding and generation via instruction tuning. arXiv preprint arXiv:2412.14164. [find](https://scholar.google.com/scholar?q=Shengbang+Tong%2C+David+Fan%2C+Jiachen+Zhu%2C+Yunyang+Xiong%2C+Xinlei+Chen%2C+Koustuv+Sinha%2C+Michael+Rabbat%2C+Yann+LeCun%2C+Saining+Xie%2C+and+Zhuang+Liu%2E+2024%2E+Metamorph%3A+Multimodal+understanding+and+generation+via)
113. Yang Shi, Yuhao Dong, Yue Ding, Yuran Wang, Xuanyu Zhu, Sheng Zhou, Wenting Liu, Haochen Tian, Rundong Wang, Huanqian Wang, and 1 others. 2025. Realunify: Do unified models truly benefit from unification? a comprehensive benchmark. arXiv preprint arXiv:2509.24897. — [arXiv:2509.24897](https://arxiv.org/abs/2509.24897) [find](https://scholar.google.com/scholar?q=Yang+Shi%2C+Yuhao+Dong%2C+Yue+Ding%2C+Yuran+Wang%2C+Xuanyu+Zhu%2C+Sheng+Zhou%2C+Wenting+Liu%2C+Haochen+Tian%2C+Rundong+Wang%2C+Huanqian+Wang%2C+and+1+others%2E+2025%2E+Realunify%3A+Do+unified+models+truly+benefit+from+unificat)
114. Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, and 1 others. 2023a. [Llama: Open and efficient foundation language models](https://md.research.elamur.ai/free/en/papers/touvron-2023-llama-open-efficient-foundation-language/). arXiv preprint arXiv:2302.13971.
115. Amanpreet Singh, Vivek Natarajan, Meet Shah, Yu Jiang, Xinlei Chen, Dhruv Batra, Devi Parikh, and Marcus Rohrbach. 2019. Towards vqa models that can read. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 8317–8326. [find](https://scholar.google.com/scholar?q=Amanpreet+Singh%2C+Vivek+Natarajan%2C+Meet+Shah%2C+Yu+Jiang%2C+Xinlei+Chen%2C+Dhruv+Batra%2C+Devi+Parikh%2C+and+Marcus+Rohrbach%2E+2019%2E+Towards+vqa+models+that+can+read%2E+In+Proceedings+of+the+IEEE%2FCVF+conference+on+)
116. Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, and 1 others. 2023b. [Llama 2: Open foundation and fine-tuned chat models](https://md.research.elamur.ai/free/en/papers/touvron-2023-llama-2-open-foundation-fine/). arXiv preprint arXiv:2307.09288.
117. Jascha Sohl-Dickstein, Eric Weiss, Niru Maheswaranathan, and Surya Ganguli. 2015. [Deep unsupervised learning using nonequilibrium thermodynamics](https://md.research.elamur.ai/free/en/papers/sohl-dickstein-2015-deep-unsupervised-learning-using-nonequilibrium/). In International conference on machine learning, pages 2256–2265. pmlr.
118. Michael Tschannen, Cian Eastwood, and Fabian Mentzer. 2024. Givt: Generative infinite-vocabulary transformers. In European Conference on Computer Vision, pages 292–309. Springer. [find](https://scholar.google.com/scholar?q=Michael+Tschannen%2C+Cian+Eastwood%2C+and+Fabian+Mentzer%2E+2024%2E+Givt%3A+Generative+infinite%2Dvocabulary+transformers%2E+In+European+Conference+on+Computer+Vision%2C+pages+292%E2%80%93309%2E+Springer%2E)
119. Jiaming Song, Chenlin Meng, and Stefano Ermon. 2020. [Denoising diffusion implicit models](https://md.research.elamur.ai/free/en/papers/song-2020-denoising-diffusion-implicit-models/). arXiv preprint arXiv:2010.02502.
120. Aaron Van Den Oord, Oriol Vinyals, and 1 others. 2017. Neural discrete representation learning. Advances in neural information processing systems, 30. [find](https://scholar.google.com/scholar?q=Aaron+Van+Den+Oord%2C+Oriol+Vinyals%2C+and+1+others%2E+2017%2E+Neural+discrete+representation+learning%2E+Advances+in+neural+information+processing+systems%2C+30%2E)
121. Peize Sun, Yi Jiang, Shoufa Chen, Shilong Zhang, Bingyue Peng, Ping Luo, and Zehuan Yuan. 2024a. Autoregressive model beats diffusion: Llama for scalable image generation. arXiv preprint arXiv:2406.06525. — [arXiv:2406.06525](https://arxiv.org/abs/2406.06525) [find](https://scholar.google.com/scholar?q=Peize+Sun%2C+Yi+Jiang%2C+Shoufa+Chen%2C+Shilong+Zhang%2C+Bingyue+Peng%2C+Ping+Luo%2C+and+Zehuan+Yuan%2E+2024a%2E+Autoregressive+model+beats+diffusion%3A+Llama+for+scalable+image+generation%2E+arXiv+preprint+arXiv%3A2406%2E06)
122. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. [Attention is all you need](https://md.research.elamur.ai/free/en/papers/vaswani-2017-attention-all-need/). Advances in neural information processing systems, 30.
123. Team Wan, Ang Wang, Baole Ai, Bin Wen, Chaojie Mao, Chen-Wei Xie, Di Chen, Feiwu Yu, Haiming Zhao, Jianxiao Yang, and 1 others. 2025. [Wan: Open and advanced large-scale video generative models](https://md.research.elamur.ai/free/en/papers/wan-2025-wan-open-advanced-large-scale/). arXiv preprint arXiv:2503.20314.
124. Xingkai Yu, Chong Ruan, and 1 others. 2025b. [Janus: Decoupling visual encoding for unified multimodal understanding and generation](https://md.research.elamur.ai/free/en/papers/wu-2024-janus-decoupling-visual-encoding/). In Proceedings of the Computer Vision and Pattern Recognition Conference, pages 12966–12977.
125. Bohan Wang, Zhongqi Yue, Fengda Zhang, Shuo Chen, Li’an Bi, Junzhe Zhang, Xue Song, Kennard Yanting Chan, Jiachun Pan, Weijia Wu, and 1 others. 2025a. Selftok: Discrete visual tokens of autoregression, by diffusion, and for reasoning. arXiv preprint arXiv:2505.07538. — [arXiv:2505.07538](https://arxiv.org/abs/2505.07538) [find](https://scholar.google.com/scholar?q=Bohan+Wang%2C+Zhongqi+Yue%2C+Fengda+Zhang%2C+Shuo+Chen%2C+Li%E2%80%99an+Bi%2C+Junzhe+Zhang%2C+Xue+Song%2C+Kennard+Yanting+Chan%2C+Jiachun+Pan%2C+Weijia+Wu%2C+and+1+others%2E+2025a%2E+Selftok%3A+Discrete+visual+tokens+of+autoregression)
126. Chenyuan Wu, Pengfei Zheng, Ruiran Yan, Shitao Xiao, Xin Luo, Yueze Wang, Wanli Li, Xiyan Jiang, Yexin Liu, Junjie Zhou, and 1 others. 2025c. Omnigen2: Exploration to advanced multimodal generation. arXiv preprint arXiv:2506.18871. — [arXiv:2506.18871](https://arxiv.org/abs/2506.18871) [find](https://scholar.google.com/scholar?q=Chenyuan+Wu%2C+Pengfei+Zheng%2C+Ruiran+Yan%2C+Shitao+Xiao%2C+Xin+Luo%2C+Yueze+Wang%2C+Wanli+Li%2C+Xiyan+Jiang%2C+Yexin+Liu%2C+Junjie+Zhou%2C+and+1+others%2E+2025c%2E+Omnigen2%3A+Exploration+to+advanced+multimodal+generation%2E+a)
127. Guo-Hua Wang, Shanshan Zhao, Xinjie Zhang, Liangfu Cao, Pengxin Zhan, Lunhao Duan, Shiyin Lu, Minghao Fu, Xiaohao Chen, Jianshan Zhao, and 1 others. 2025b. Ovis-u1 technical report. arXiv preprint arXiv:2506.23044. — [arXiv:2506.23044](https://arxiv.org/abs/2506.23044) [find](https://scholar.google.com/scholar?q=Guo%2DHua+Wang%2C+Shanshan+Zhao%2C+Xinjie+Zhang%2C+Liangfu+Cao%2C+Pengxin+Zhan%2C+Lunhao+Duan%2C+Shiyin+Lu%2C+Minghao+Fu%2C+Xiaohao+Chen%2C+Jianshan+Zhao%2C+and+1+others%2E+2025b%2E+Ovis%2Du1+technical+report%2E+arXiv+preprint+arX)
128. Junke Wang, Yi Jiang, Zehuan Yuan, Bingyue Peng, Zuxuan Wu, and Yu-Gang Jiang. 2024a. Omnitokenizer: A joint image-video tokenizer for visual generation. Advances in Neural Information Processing Systems, 37:28281–28295. [find](https://scholar.google.com/scholar?q=Junke+Wang%2C+Yi+Jiang%2C+Zehuan+Yuan%2C+Bingyue+Peng%2C+Zuxuan+Wu%2C+and+Yu%2DGang+Jiang%2E+2024a%2E+Omnitokenizer%3A+A+joint+image%2Dvideo+tokenizer+for+visual+generation%2E+Advances+in+Neural+Information+Processing+Syst)
129. Peiyu Wang, Yi Peng, Yimeng Gan, Liang Hu, Tianyidan Xie, Xiaokun Wang, Yichen Wei, Chuanxin Tang, Bo Zhu, Changshi Li, and 1 others. 2025c. Skywork unipic: Unified autoregressive modeling for visual understanding and generation. arXiv preprint arXiv:2508.03320. [find](https://scholar.google.com/scholar?q=Peiyu+Wang%2C+Yi+Peng%2C+Yimeng+Gan%2C+Liang+Hu%2C+Tianyidan+Xie%2C+Xiaokun+Wang%2C+Yichen+Wei%2C+Chuanxin+Tang%2C+Bo+Zhu%2C+Changshi+Li%2C+and+1+others%2E+2025c%2E+Skywork+unipic%3A+Unified+autoregressive+modeling+for+visual+)
130. Peng Wang, Shuai Bai, Sinan Tan, Shijie Wang, Zhihao Fan, Jinze Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, and 1 others. 2024b. [Qwen2-vl: Enhancing vision-language model’s perception of the world at any resolution](https://md.research.elamur.ai/free/en/papers/wang-2024-qwen2-vl-enhancing-vision/). arXiv preprint arXiv:2409.12191.
131. Xinlong Wang, Xiaosong Zhang, Zhengxiong Luo, Quan Sun, Yufeng Cui, Jinsheng Wang, Fan Zhang, Yueze Wang, Zhen Li, Qiying Yu, and 1 others. 2024c. Emu3: Next-token prediction is all you need. arXiv preprint arXiv:2409.18869. — [arXiv:2409.18869](https://arxiv.org/abs/2409.18869) [find](https://scholar.google.com/scholar?q=Xinlong+Wang%2C+Xiaosong+Zhang%2C+Zhengxiong+Luo%2C+Quan+Sun%2C+Yufeng+Cui%2C+Jinsheng+Wang%2C+Fan+Zhang%2C+Yueze+Wang%2C+Zhen+Li%2C+Qiying+Yu%2C+and+1+others%2E+2024c%2E+Emu3%3A+Next%2Dtoken+prediction+is+all+you+need%2E+arXiv+pr)
132. Zekun Wang, King Zhu, Chunpu Xu, Wangchunshu Zhou, Jiaheng Liu, Yibo Zhang, Jiashuo Wang, Ning Shi, Siyu Li, Yizhi Li, and 1 others. 2024d. Mio: A foundation model on multimodal tokens. arXiv preprint arXiv:2409.17692. [find](https://scholar.google.com/scholar?q=Zekun+Wang%2C+King+Zhu%2C+Chunpu+Xu%2C+Wangchunshu+Zhou%2C+Jiaheng+Liu%2C+Yibo+Zhang%2C+Jiashuo+Wang%2C+Ning+Shi%2C+Siyu+Li%2C+Yizhi+Li%2C+and+1+others%2E+2024d%2E+Mio%3A+A+foundation+model+on+multimodal+tokens%2E+arXiv+preprint)
133. Hongyang Wei, Baixin Xu, Hongbo Liu, Cyrus Wu, Jie Liu, Yi Peng, Peiyu Wang, Zexiang Liu, Jingwen He, Yidan Xietian, and 1 others. 2025. Skywork unipic 2.0: Building kontext model with online rl for unified multimodal model. arXiv preprint arXiv:2509.04548. — [arXiv:2509.04548](https://arxiv.org/abs/2509.04548) [find](https://scholar.google.com/scholar?q=Hongyang+Wei%2C+Baixin+Xu%2C+Hongbo+Liu%2C+Cyrus+Wu%2C+Jie+Liu%2C+Yi+Peng%2C+Peiyu+Wang%2C+Zexiang+Liu%2C+Jingwen+He%2C+Yidan+Xietian%2C+and+1+others%2E+2025%2E+Skywork+unipic+2%2E0%3A+Building+kontext+model+with+online+rl+for+u)
134. Chenfei Wu, Jiahao Li, Jingren Zhou, Junyang Lin, Kaiyuan Gao, Kun Yan, Sheng-ming Yin, Shuai Bai, Xiao Xu, Yilei Chen, and 1 others. 2025a. [Qwen-image technical report](https://md.research.elamur.ai/free/en/papers/wu-2025-qwen-image-technical-report/). arXiv preprint arXiv:2508.02324.
135. Chengyue Wu, Xiaokang Chen, Zhiyu Wu, Yiyang Ma, Xingchao Liu, Zizheng Pan, Wen Liu, Zhenda Xie, Junfeng Wu, Yi Jiang, Chuofan Ma, Yuliang Liu, Hengshuang Zhao, Zehuan Yuan, Song Bai, and Xiang Bai. 2024a. Liquid: Language models are scalable and unified multi-modal generators. arXiv preprint arXiv:… — [arXiv:2412.04332](https://arxiv.org/abs/2412.04332) [find](https://scholar.google.com/scholar?q=Chengyue+Wu%2C+Xiaokang+Chen%2C+Zhiyu+Wu%2C+Yiyang+Ma%2C+Xingchao+Liu%2C+Zizheng+Pan%2C+Wen+Liu%2C+Zhenda+Xie%2C+Junfeng+Wu%2C+Yi+Jiang%2C+Chuofan+Ma%2C+Yuliang+Liu%2C+Hengshuang+Zhao%2C+Zehuan+Yuan%2C+Song+Bai%2C+and+Xiang+Bai%2E+2)
136. Siyuan Wu, Yue Huang, Chujie Gao, Dongping Chen, Qihui Zhang, Yao Wan, Tianyi Zhou, Xiangliang Zhang, Jianfeng Gao, Chaowei Xiao, and 1 others. 2024b. Unigen: A unified framework for textual dataset generation using large language models. arXiv preprint arXiv:2406.18966. — [arXiv:2406.18966](https://arxiv.org/abs/2406.18966) [find](https://scholar.google.com/scholar?q=Siyuan+Wu%2C+Yue+Huang%2C+Chujie+Gao%2C+Dongping+Chen%2C+Qihui+Zhang%2C+Yao+Wan%2C+Tianyi+Zhou%2C+Xiangliang+Zhang%2C+Jianfeng+Gao%2C+Chaowei+Xiao%2C+and+1+others%2E+2024b%2E+Unigen%3A+A+unified+framework+for+textual+dataset+g)
137. Size Wu, Zhonghua Wu, Zerui Gong, Qingyi Tao, Sheng Jin, Qinyue Li, Wei Li, and Chen Change Loy. 2025d. Openuni: A simple baseline for unified multimodal understanding and generation. arXiv preprint arXiv:2505.23661. — [arXiv:2505.23661](https://arxiv.org/abs/2505.23661) [find](https://scholar.google.com/scholar?q=Size+Wu%2C+Zhonghua+Wu%2C+Zerui+Gong%2C+Qingyi+Tao%2C+Sheng+Jin%2C+Qinyue+Li%2C+Wei+Li%2C+and+Chen+Change+Loy%2E+2025d%2E+Openuni%3A+A+simple+baseline+for+unified+multimodal+understanding+and+generation%2E+arXiv+preprint+a)
138. Xiaoyang Wu, Li Jiang, Peng-Shuai Wang, Zhijian Liu, Xihui Liu, Yu Qiao, Wanli Ouyang, Tong He, and Hengshuang Zhao. 2024c. Point transformer v3: Simpler faster stronger. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 4840–4851. [find](https://scholar.google.com/scholar?q=Xiaoyang+Wu%2C+Li+Jiang%2C+Peng%2DShuai+Wang%2C+Zhijian+Liu%2C+Xihui+Liu%2C+Yu+Qiao%2C+Wanli+Ouyang%2C+Tong+He%2C+and+Hengshuang+Zhao%2E+2024c%2E+Point+transformer+v3%3A+Simpler+faster+stronger%2E+In+Proceedings+of+the+IEEE%2FCV)
139. Yecheng Wu, Zhuoyang Zhang, Junyu Chen, Haotian Tang, Dacheng Li, Yunhao Fang, Ligeng Zhu, Enze Xie, Hongxu Yin, Li Yi, and 1 others. 2024d. Vila-u: a unified foundation model integrating visual understanding and generation. arXiv preprint arXiv:2409.04429. — [arXiv:2409.04429](https://arxiv.org/abs/2409.04429) [find](https://scholar.google.com/scholar?q=Yecheng+Wu%2C+Zhuoyang+Zhang%2C+Junyu+Chen%2C+Haotian+Tang%2C+Dacheng+Li%2C+Yunhao+Fang%2C+Ligeng+Zhu%2C+Enze+Xie%2C+Hongxu+Yin%2C+Li+Yi%2C+and+1+others%2E+2024d%2E+Vila%2Du%3A+a+unified+foundation+model+integrating+visual+under)
140. Yonghui Wu, Mike Schuster, Zhifeng Chen, Quoc V Le, Mohammad Norouzi, Wolfgang Macherey, Maxim Krikun, Yuan Cao, Qin Gao, Klaus Macherey, and 1 others. 2016. Google’s neural machine translation system: Bridging the gap between human and machine translation. arXiv preprint arXiv:1609.08144. [find](https://scholar.google.com/scholar?q=Yonghui+Wu%2C+Mike+Schuster%2C+Zhifeng+Chen%2C+Quoc+V+Le%2C+Mohammad+Norouzi%2C+Wolfgang+Macherey%2C+Maxim+Krikun%2C+Yuan+Cao%2C+Qin+Gao%2C+Klaus+Macherey%2C+and+1+others%2E+2016%2E+Google%E2%80%99s+neural+machine+translation+system)
141. Yusong Wu, Ke Chen, Tianyu Zhang, Yuchen Hui, Taylor Berg-Kirkpatrick, and Shlomo Dubnov. 2023. Large-scale contrastive language-audio pretraining with feature fusion and keyword-to-caption augmentation. In ICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Processing (IC… [find](https://scholar.google.com/scholar?q=Yusong+Wu%2C+Ke+Chen%2C+Tianyu+Zhang%2C+Yuchen+Hui%2C+Taylor+Berg%2DKirkpatrick%2C+and+Shlomo+Dubnov%2E+2023%2E+Large%2Dscale+contrastive+language%2Daudio+pretraining+with+feature+fusion+and+keyword%2Dto%2Dcaption+augmentati)
142. Jinheng Xie, Weijia Mao, Zechen Bai, David Junhao Zhang, Weihao Wang, Kevin Qinghong Lin, Yuchao Gu, Zhijie Chen, Zhenheng Yang, and Mike Zheng Shou. 2024. Show-o: One single transformer to unify multimodal understanding and generation. arXiv preprint arXiv:2408.12528. — [arXiv:2408.12528](https://arxiv.org/abs/2408.12528) [find](https://scholar.google.com/scholar?q=Jinheng+Xie%2C+Weijia+Mao%2C+Zechen+Bai%2C+David+Junhao+Zhang%2C+Weihao+Wang%2C+Kevin+Qinghong+Lin%2C+Yuchao+Gu%2C+Zhijie+Chen%2C+Zhenheng+Yang%2C+and+Mike+Zheng+Shou%2E+2024%2E+Show%2Do%3A+One+single+transformer+to+unify+mult)
143. Jinheng Xie, Zhenheng Yang, and Mike Zheng Shou. 2025. Show-o2: Improved native unified multimodal models. arXiv preprint arXiv:2506.15564. — [arXiv:2506.15564](https://arxiv.org/abs/2506.15564) [find](https://scholar.google.com/scholar?q=Jinheng+Xie%2C+Zhenheng+Yang%2C+and+Mike+Zheng+Shou%2E+2025%2E+Show%2Do2%3A+Improved+native+unified+multimodal+models%2E+arXiv+preprint+arXiv%3A2506%2E15564%2E+%E2%80%94+%5BarXiv%3A2506%2E15564%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2506%2E15564%29)
144. Yi Xin, Juncheng Yan, Qi Qin, Zhen Li, Dongyang Liu, Shicheng Li, Victor Shea-Jay Huang, Yupeng Zhou, Renrui Zhang, Le Zhuo, and 1 others. 2025. Lumina-mgpt 2.0: Stand-alone autoregressive image modeling. arXiv preprint arXiv:2507.17801. — [arXiv:2507.17801](https://arxiv.org/abs/2507.17801) [find](https://scholar.google.com/scholar?q=Yi+Xin%2C+Juncheng+Yan%2C+Qi+Qin%2C+Zhen+Li%2C+Dongyang+Liu%2C+Shicheng+Li%2C+Victor+Shea%2DJay+Huang%2C+Yupeng+Zhou%2C+Renrui+Zhang%2C+Le+Zhuo%2C+and+1+others%2E+2025%2E+Lumina%2Dmgpt+2%2E0%3A+Stand%2Dalone+autoregressive+image+model)
145. Tianwei Xiong, Jun Hao Liew, Zilong Huang, Jiashi Feng, and Xihui Liu. 2025. Gigatok: Scaling visual tokenizers to 3 billion parameters for autoregressive image generation. arXiv preprint arXiv:2504.08736. — [arXiv:2504.08736](https://arxiv.org/abs/2504.08736) [find](https://scholar.google.com/scholar?q=Tianwei+Xiong%2C+Jun+Hao+Liew%2C+Zilong+Huang%2C+Jiashi+Feng%2C+and+Xihui+Liu%2E+2025%2E+Gigatok%3A+Scaling+visual+tokenizers+to+3+billion+parameters+for+autoregressive+image+generation%2E+arXiv+preprint+arXiv%3A2504%2E0)
146. Hu Xu, Saining Xie, Xiaoqing Ellen Tan, Po-Yao Huang, Russell Howes, Vasu Sharma, Shang-Wen Li, Gargi Ghosh, Luke Zettlemoyer, and Christoph Feichtenhofer. 2023. Demystifying clip data. arXiv preprint arXiv:2309.16671. [find](https://scholar.google.com/scholar?q=Hu+Xu%2C+Saining+Xie%2C+Xiaoqing+Ellen+Tan%2C+Po%2DYao+Huang%2C+Russell+Howes%2C+Vasu+Sharma%2C+Shang%2DWen+Li%2C+Gargi+Ghosh%2C+Luke+Zettlemoyer%2C+and+Christoph+Feichtenhofer%2E+2023%2E+Demystifying+clip+data%2E+arXiv+preprint)
147. Jin Xu, Zhifang Guo, Jinzheng He, Hangrui Hu, Ting He, Shuai Bai, Keqin Chen, Jialin Wang, Yang Fan, Kai Dang, and 1 others. 2025. Qwen2. 5-omni technical report. arXiv preprint arXiv:2503.20215. — [Qwen2.5-Omni Technical Report](https://md.research.elamur.ai/free/en/papers/xu-2025-qwen2-5-omni-technical/)
148. Yifan Xu, Xiaoshan Yang, Yaguang Song, and Changsheng Xu. 2024a. Libra: Building decoupled vision system on large language models. arXiv preprint arXiv:2405.10140. — [arXiv:2405.10140](https://arxiv.org/abs/2405.10140) [find](https://scholar.google.com/scholar?q=Yifan+Xu%2C+Xiaoshan+Yang%2C+Yaguang+Song%2C+and+Changsheng+Xu%2E+2024a%2E+Libra%3A+Building+decoupled+vision+system+on+large+language+models%2E+arXiv+preprint+arXiv%3A2405%2E10140%2E+%E2%80%94+%5BarXiv%3A2405%2E10140%5D%28https%3A%2F%2Farxiv%2Eo)
149. Ziwei Xu, Sanjay Jain, and Mohan Kankanhalli. 2024b. [Hallucination is inevitable: An innate limitation of large language models](https://md.research.elamur.ai/free/en/papers/xu-2024-hallucination-inevitable-innate-limitation-large/). arXiv preprint arXiv:2401.11817.
150. Linting Xue, Aditya Barua, Noah Constant, Rami Al-Rfou, Sharan Narang, Mihir Kale, Adam Roberts, and Colin Raffel. 2022. Byt5: Towards a token-free future with pre-trained byte-to-byte models. Transactions of the Association for Computational Linguistics, 10:291–306. [find](https://scholar.google.com/scholar?q=Linting+Xue%2C+Aditya+Barua%2C+Noah+Constant%2C+Rami+Al%2DRfou%2C+Sharan+Narang%2C+Mihir+Kale%2C+Adam+Roberts%2C+and+Colin+Raffel%2E+2022%2E+Byt5%3A+Towards+a+token%2Dfree+future+with+pre%2Dtrained+byte%2Dto%2Dbyte+models%2E+Transac)
151. Linting Xue, Noah Constant, Adam Roberts, Mihir Kale, Rami Al-Rfou, Aditya Siddhant, Aditya Barua, and Colin Raffel. 2020. [mt5: A massively multilingual pre-trained text-to-text transformer](https://md.research.elamur.ai/free/en/papers/xue-2021-mt5-massively-multilingual-pre-trained/). arXiv preprint arXiv:2010.11934.
152. Jianwei Yang, Reuben Tan, Qianhui Wu, Ruijie Zheng, Baolin Peng, Yongyuan Liang, Yu Gu, Mu Cai, Seonghyeon Ye, Joel Jang, and 1 others. 2025. Magma: A foundation model for multimodal ai agents. In Proceedings of the Computer Vision and Pattern Recognition Conference, pages 14203– 14214. [find](https://scholar.google.com/scholar?q=Jianwei+Yang%2C+Reuben+Tan%2C+Qianhui+Wu%2C+Ruijie+Zheng%2C+Baolin+Peng%2C+Yongyuan+Liang%2C+Yu+Gu%2C+Mu+Cai%2C+Seonghyeon+Ye%2C+Joel+Jang%2C+and+1+others%2E+2025%2E+Magma%3A+A+foundation+model+for+multimodal+ai+agents%2E+In+Pro)
153. Jiahui Yu, Yuanzhong Xu, Jing Yu Koh, Thang Luong, Gunjan Baid, Zirui Wang, Vijay Vasudevan, Alexander Ku, Yinfei Yang, Burcu Karagol Ayan, and 1 others. 2022. Scaling autoregressive models for content-rich text-to-image generation. arXiv preprint arXiv:2206.10789, 2(3):5. — [arXiv:2206.10789](https://arxiv.org/abs/2206.10789) [find](https://scholar.google.com/scholar?q=Jiahui+Yu%2C+Yuanzhong+Xu%2C+Jing+Yu+Koh%2C+Thang+Luong%2C+Gunjan+Baid%2C+Zirui+Wang%2C+Vijay+Vasudevan%2C+Alexander+Ku%2C+Yinfei+Yang%2C+Burcu+Karagol+Ayan%2C+and+1+others%2E+2022%2E+Scaling+autoregressive+models+for+conten)
154. Lijun Yu, Yong Cheng, Kihyuk Sohn, José Lezama, Han Zhang, Huiwen Chang, Alexander G Hauptmann, Ming-Hsuan Yang, Yuan Hao, Irfan Essa, and 1 others. 2023a. Magvit: Masked generative video transformer. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10459–1… [find](https://scholar.google.com/scholar?q=Lijun+Yu%2C+Yong+Cheng%2C+Kihyuk+Sohn%2C+Jos%C3%A9+Lezama%2C+Han+Zhang%2C+Huiwen+Chang%2C+Alexander+G+Hauptmann%2C+Ming%2DHsuan+Yang%2C+Yuan+Hao%2C+Irfan+Essa%2C+and+1+others%2E+2023a%2E+Magvit%3A+Masked+generative+video+transformer%2E)
155. Lijun Yu, José Lezama, Nitesh B Gundavarapu, Luca Versari, Kihyuk Sohn, David Minnen, Yong Cheng, Vighnesh Birodkar, Agrim Gupta, Xiuye Gu, and 1 others. 2023b. Language model beats diffusion– tokenizer is key to visual generation. arXiv preprint arXiv:2310.05737. — [arXiv:2310.05737](https://arxiv.org/abs/2310.05737) [find](https://scholar.google.com/scholar?q=Lijun+Yu%2C+Jos%C3%A9+Lezama%2C+Nitesh+B+Gundavarapu%2C+Luca+Versari%2C+Kihyuk+Sohn%2C+David+Minnen%2C+Yong+Cheng%2C+Vighnesh+Birodkar%2C+Agrim+Gupta%2C+Xiuye+Gu%2C+and+1+others%2E+2023b%2E+Language+model+beats+diffusion%E2%80%93+tokeniz)
156. Qihang Yu, Mark Weber, Xueqing Deng, Xiaohui Shen, Daniel Cremers, and Liang-Chieh Chen. 2024. An image is worth 32 tokens for reconstruction and generation. Advances in Neural Information Processing Systems, 37:128940–128966. [find](https://scholar.google.com/scholar?q=Qihang+Yu%2C+Mark+Weber%2C+Xueqing+Deng%2C+Xiaohui+Shen%2C+Daniel+Cremers%2C+and+Liang%2DChieh+Chen%2E+2024%2E+An+image+is+worth+32+tokens+for+reconstruction+and+generation%2E+Advances+in+Neural+Information+Processing+)
157. Weihao Yu, Zhengyuan Yang, Linjie Li, Jianfeng Wang, Kevin Lin, Zicheng Liu, Xinchao Wang, and Lijuan Wang. 2023c. [Mm-vet: Evaluating large multimodal models for integrated capabilities](https://md.research.elamur.ai/free/en/papers/yu-2023-mm-vet-evaluating-large-multimodal/). arXiv preprint arXiv:2308.02490.
158. Jingyang Yuan, Huazuo Gao, Damai Dai, Junyu Luo, Liang Zhao, Zhengyan Zhang, Zhenda Xie, YX Wei, Lean Wang, Zhiping Xiao, and 1 others. 2025. [Native sparse attention: Hardware-aligned and natively trainable sparse attention](https://md.research.elamur.ai/free/en/papers/yuan-2025-native-sparse-attention-hardware/). arXiv preprint arXiv:2502.11089.
159. Xiang Yue, Yuansheng Ni, Kai Zhang, Tianyu Zheng, Ruoqi Liu, Ge Zhang, Samuel Stevens, Dongfu Jiang, Weiming Ren, Yuxuan Sun, and 1 others. 2024. [Mmmu: A massive multi-discipline multimodal understanding and reasoning benchmark for expert agi](https://md.research.elamur.ai/free/en/papers/yue-2023-mmmu-massive-multi-discipline-multimodal/). In Proceedings of the IEEE/CVF Conference on Computer Vi…
160. Neil Zeghidour, Alejandro Luebs, Ahmed Omran, Jan Skoglund, and Marco Tagliasacchi. 2021. [Soundstream: An end-to-end neural audio codec](https://md.research.elamur.ai/free/en/papers/zeghidour-2021-soundstream-end-end-neural-audio/). IEEE/ACM Transactions on Audio, Speech, and Language Processing, 30:495–507.
161. Yang Ye, Xianyi He, Zongjian Li, Bin Lin, Shenghai Yuan, Zhiyuan Yan, Bohan Hou, and Li Yuan. 2025. Imgedit: A unified image editing dataset and benchmark. arXiv preprint arXiv:2505.20275. — [arXiv:2505.20275](https://arxiv.org/abs/2505.20275) [find](https://scholar.google.com/scholar?q=Yang+Ye%2C+Xianyi+He%2C+Zongjian+Li%2C+Bin+Lin%2C+Shenghai+Yuan%2C+Zhiyuan+Yan%2C+Bohan+Hou%2C+and+Li+Yuan%2E+2025%2E+Imgedit%3A+A+unified+image+editing+dataset+and+benchmark%2E+arXiv+preprint+arXiv%3A2505%2E20275%2E+%E2%80%94+%5BarXiv%3A25)
162. Rowan Zellers, Ari Holtzman, Yonatan Bisk, Ali Farhadi, and Yejin Choi. 2019. Hellaswag: Can a machine really finish your sentence? arXiv preprint arXiv:1905.07830. [find](https://scholar.google.com/scholar?q=Rowan+Zellers%2C+Ari+Holtzman%2C+Yonatan+Bisk%2C+Ali+Farhadi%2C+and+Yejin+Choi%2E+2019%2E+Hellaswag%3A+Can+a+machine+really+finish+your+sentence%3F+arXiv+preprint+arXiv%3A1905%2E07830%2E)
163. Chengxuan Ying, Tianle Cai, Shengjie Luo, Shuxin Zheng, Guolin Ke, Di He, Yanming Shen, and Tie-Yan Liu. 2021. Do transformers really perform badly for graph representation? Advances in neural information processing systems, 34:28877–28888. [find](https://scholar.google.com/scholar?q=Chengxuan+Ying%2C+Tianle+Cai%2C+Shengjie+Luo%2C+Shuxin+Zheng%2C+Guolin+Ke%2C+Di+He%2C+Yanming+Shen%2C+and+Tie%2DYan+Liu%2E+2021%2E+Do+transformers+really+perform+badly+for+graph+representation%3F+Advances+in+neural+informa)
164. Xiaohua Zhai, Basil Mustafa, Alexander Kolesnikov, and Lucas Beyer. 2023. [Sigmoid loss for language image pre-training](https://md.research.elamur.ai/free/en/papers/zhai-2023-sigmoid-loss-language-image-pre/). In Proceedings of the IEEE/CVF international conference on computer vision, pages 11975–11986.
165. Duzhen Zhang, Yahan Yu, Jiahua Dong, Chenxing Li, Dan Su, Chenhui Chu, and Dong Yu. 2024. [Mm-llms: Recent advances in multimodal large language models](https://md.research.elamur.ai/free/en/papers/zhang-2024-mm-llms-recent-advances-multimodal/). arXiv preprint arXiv:2401.13601.
166. Hong Zhang, Zhongjie Duan, Xingjun Wang, Yuze Zhao, Weiyi Lu, Zhipeng Di, Yixuan Xu, Yingda Chen, and Yu Zhang. 2025a. Nexus-gen: A unified model for image understanding, generation, and editing. arXiv preprint arXiv:2504.21356. — [arXiv:2504.21356](https://arxiv.org/abs/2504.21356) [find](https://scholar.google.com/scholar?q=Hong+Zhang%2C+Zhongjie+Duan%2C+Xingjun+Wang%2C+Yuze+Zhao%2C+Weiyi+Lu%2C+Zhipeng+Di%2C+Yixuan+Xu%2C+Yingda+Chen%2C+and+Yu+Zhang%2E+2025a%2E+Nexus%2Dgen%3A+A+unified+model+for+image+understanding%2C+generation%2C+and+editing%2E+arXi)
167. Pengchuan Zhang, Xiujun Li, Xiaowei Hu, Jianwei Yang, Lei Zhang, Lijuan Wang, Yejin Choi, and Jianfeng Gao. 2021. Vinvl: Revisiting visual representations in vision-language models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 5579–5588. [find](https://scholar.google.com/scholar?q=Pengchuan+Zhang%2C+Xiujun+Li%2C+Xiaowei+Hu%2C+Jianwei+Yang%2C+Lei+Zhang%2C+Lijuan+Wang%2C+Yejin+Choi%2C+and+Jianfeng+Gao%2E+2021%2E+Vinvl%3A+Revisiting+visual+representations+in+vision%2Dlanguage+models%2E+In+Proceedings+of+)
168. Xinjie Zhang, Jintao Guo, Shanshan Zhao, Minghao Fu, Lunhao Duan, Jiakui Hu, Yong Xien Chng, Guo-Hua Wang, Qing-Guo Chen, Zhao Xu, and 1 others. 2025b. Unified multimodal understanding and generation models: Advances, challenges, and opportunities. arXiv preprint arXiv:2505.02567. — [arXiv:2505.02567](https://arxiv.org/abs/2505.02567) [find](https://scholar.google.com/scholar?q=Xinjie+Zhang%2C+Jintao+Guo%2C+Shanshan+Zhao%2C+Minghao+Fu%2C+Lunhao+Duan%2C+Jiakui+Hu%2C+Yong+Xien+Chng%2C+Guo%2DHua+Wang%2C+Qing%2DGuo+Chen%2C+Zhao+Xu%2C+and+1+others%2E+2025b%2E+Unified+multimodal+understanding+and+generation+)
169. Yitian Zhang, Long Mai, Aniruddha Mahapatra, David Bourgin, Yicong Hong, Jonah Casebeer, Feng Liu, and Yun Fu. 2025c. Regen: Learning compact video embedding with (re-) generative decoder. arXiv preprint arXiv:2503.08665. — [arXiv:2503.08665](https://arxiv.org/abs/2503.08665) [find](https://scholar.google.com/scholar?q=Yitian+Zhang%2C+Long+Mai%2C+Aniruddha+Mahapatra%2C+David+Bourgin%2C+Yicong+Hong%2C+Jonah+Casebeer%2C+Feng+Liu%2C+and+Yun+Fu%2E+2025c%2E+Regen%3A+Learning+compact+video+embedding+with+%28re%2D%29+generative+decoder%2E+arXiv+prepr)
170. Chuyang Zhao, Yuxing Song, Wenhao Wang, Haocheng Feng, Errui Ding, Yifan Sun, Xinyan Xiao, and Jingdong Wang. 2024a. Monoformer: One transformer for both diffusion and autoregression. arXiv preprint arXiv:2409.16280. — [arXiv:2409.16280](https://arxiv.org/abs/2409.16280) [find](https://scholar.google.com/scholar?q=Chuyang+Zhao%2C+Yuxing+Song%2C+Wenhao+Wang%2C+Haocheng+Feng%2C+Errui+Ding%2C+Yifan+Sun%2C+Xinyan+Xiao%2C+and+Jingdong+Wang%2E+2024a%2E+Monoformer%3A+One+transformer+for+both+diffusion+and+autoregression%2E+arXiv+preprint+a)
171. Long Zhao, Sanghyun Woo, Ziyu Wan, Yandong Li, Han Zhang, Boqing Gong, Hartwig Adam, Xuhui Jia, and Ting Liu. 2024b. epsilon-vae: Denoising as visual decoding. arXiv preprint arXiv:2410.04081. — [arXiv:2410.04081](https://arxiv.org/abs/2410.04081) [find](https://scholar.google.com/scholar?q=Long+Zhao%2C+Sanghyun+Woo%2C+Ziyu+Wan%2C+Yandong+Li%2C+Han+Zhang%2C+Boqing+Gong%2C+Hartwig+Adam%2C+Xuhui+Jia%2C+and+Ting+Liu%2E+2024b%2E+epsilon%2Dvae%3A+Denoising+as+visual+decoding%2E+arXiv+preprint+arXiv%3A2410%2E04081%2E+%E2%80%94+%5BarXi)
172. Chunting Zhou, Lili Yu, Arun Babu, Kushal Tirumala, Michihiro Yasunaga, Leonid Shamis, Jacob Kahn, Xuezhe Ma, Luke Zettlemoyer, and Omer Levy. 2024. [Transfusion: Predict the next token and diffuse images with one multi-modal model](https://md.research.elamur.ai/free/en/papers/zhou-2024-transfusion-predict-next-token-diffuse/). arXiv preprint arXiv:2408.11039.
173. Bin Zhu, Bin Lin, Munan Ning, Yang Yan, Jiaxi Cui, HongFa Wang, Yatian Pang, Wenhao Jiang, Junwu Zhang, Zongwei Li, and 1 others. 2023a. Languagebind: Extending video-language pretraining to n-modality by language-based semantic alignment. arXiv preprint arXiv:2310.01852. — [arXiv:2310.01852](https://arxiv.org/abs/2310.01852) [find](https://scholar.google.com/scholar?q=Bin+Zhu%2C+Bin+Lin%2C+Munan+Ning%2C+Yang+Yan%2C+Jiaxi+Cui%2C+HongFa+Wang%2C+Yatian+Pang%2C+Wenhao+Jiang%2C+Junwu+Zhang%2C+Zongwei+Li%2C+and+1+others%2E+2023a%2E+Languagebind%3A+Extending+video%2Dlanguage+pretraining+to+n%2Dmodalit)
174. Deyao Zhu, Jun Chen, Xiaoqian Shen, Xiang Li, and Mohamed Elhoseiny. 2023b. [Minigpt-4: Enhancing vision-language understanding with advanced large language models](https://md.research.elamur.ai/free/en/papers/zhu-2023-minigpt-4-enhancing-vision-language/). arXiv preprint arXiv:2304.10592.

*Updated 2 November 2026*