---
title: "FactVerse — paper by Shan et al. (2026) — summary and ideas"
description: "The benchmark therefore offers a factuality-focused way to evaluate multimodal generators and points to concrete targets for… 54 references."
authors: ["Yubo Shan", "Kun Zhang", "Qiming Xu", "Liping Cao", "Yingying Cao", "Jian Zhang", "Yu Wang", "Jingyuan Li", "Yuanzhuo Wang"]
year: 2026
venue: "Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)"
source: "https://aclanthology.org/2026.acl-long.1323/"
doi: "10.18653/v1/2026.acl-long.1323"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/shan-2026-factverse-benchmark-factual-consistency-interleaved/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/shan-2026-factverse-benchmark-factual-consistency-interleaved/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/33/) / FactVerse: A Benchmark for Factual Consistency in Interleaved Image–Text Generation

# FactVerse: A Benchmark for Factual Consistency in Interleaved Image–Text Generation

**Authors:** Yubo Shan, [Kun Zhang](https://md.research.elamur.ai/free/en/authors/kun-zhang/), Qiming Xu, Liping Cao, Yingying Cao, [Jian Zhang](https://md.research.elamur.ai/free/en/authors/jian-zhang/), [Yu Wang](https://md.research.elamur.ai/free/en/authors/yu-wang/), [Jingyuan Li](https://md.research.elamur.ai/free/en/authors/jingyuan-li/) and [Yuanzhuo Wang](https://md.research.elamur.ai/free/en/authors/yuanzhuo-wang/) · **Year:** 2026 · **Venue:** [Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)](https://md.research.elamur.ai/free/en/venues/acl/) ([ACL 2026](https://md.research.elamur.ai/free/en/venues/acl/2026/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2026.acl-long.1323/) · [PDF](https://aclanthology.org/2026.acl-long.1323.pdf) · [DOI 10.18653/v1/2026.acl-long.1323](https://doi.org/10.18653/v1/2026.acl-long.1323) · [OpenAlex](https://openalex.org/W7166796142)

**Citations:** 0 citations (OpenAlex, 2026-10-14)

## Summary
Generative MLLMs that interleave images and text can produce content that looks authoritative but contains factual errors. This is especially risky in medicine, science education and news. Existing interleaved-generation benchmarks mostly reward relevance, coherence and visual quality, and they lack verifiable ground truth, so dangerous mistakes go unnoticed. For example, a Heimlich maneuver illustration with the hands on the chest can still score highly. FactVerse fills this gap with 3,000 human-verified English and Chinese instances. These are grouped into four categories (Entity, Procedural, Mechanism, Data), 24 subtasks and more than 50 domains. The authors pair the dataset with a three-part evaluation framework: a fine-tuned discriminator called FactJudge, VQA checks anchored to decomposed ground truth, and rule-based metrics. The framework agrees with human ratings much better than holistic GPT-4o judging. Testing 10 interleaved generation baselines shows persistent weaknesses in factual and cross-modal consistency. The benchmark therefore offers a factuality-focused way to evaluate multimodal generators and points to concrete targets for improving them.

## Key points
- **Dataset:** 3,000 human-verified instances in EN and ZH, with ground truth for verification. According to the authors, it is the only benchmark in their comparison that covers both factuality accuracy and multilingual support (vs. OpenLeaf, InterleavedBench, MMIE, OpenING, ISG-BENCH).
- **Evaluation framework:**
  - FactJudge, which outperformed both base Qwen3-VL-8B and GPT-4o when used as the judge.
  - Semantics-anchored VQA (SAS).
  - Rule-based metrics such as Entity Recall, CLIP similarity and structure checks (CC).
- **Agreement with humans:** The full framework reaches Kendall's τ 0.78 and Spearman ρ 0.85, compared with 0.61 / 0.67 for GPT-4o holistic scoring.
- **Ablation:** Removing FactJudge lowers τ to 0.42 (−0.36) and shifts model ranks by 4.2 positions on average. Dropping VQA reduces τ by 0.11.
- **Observed failure types:**
  - Cross-modal attribute misalignment (e.g., a fiddler crab drawn symmetric).
  - Weak spatial-structural reasoning.
  - Identity drift across procedural steps (e.g., CPR images).
  - Combined perception and numerical-generation errors in charts.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Tencent](https://md.research.elamur.ai/free/en/companies/tencent/) — Company, Shenzhen, China · 163 works · 21,032 citations
- [Chinese Academy of Sciences](https://md.research.elamur.ai/free/en/government/chinese-academy-of-sciences/) — Government body, Beijing, China · 246 works · 24,951 citations
- [Institute of Computing Technology](https://md.research.elamur.ai/free/en/labs/institute-of-computing-technology/) — Research institute, Beijing, China · 57 works · 3,782 citations
- [Zhengzhou University](https://md.research.elamur.ai/free/en/universities/zhengzhou-university/) — University, Zhengzhou, China · 4 works · 71 citations

Also on the byline: [Beijing Technology and Business University](https://ror.org/013e0zm98).

📚 Shelves (5): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 18,614 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 17,965 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 17,450 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 17,296 works · [👁️ Computer Vision and Pattern Recognition](https://md.research.elamur.ai/free/en/shelves/computer-science/computer-vision-and-pattern-recognition/) · 3,486 works

🧭 Topics (3): [📏 Evaluation & benchmarks](https://md.research.elamur.ai/free/en/topics/evaluation-benchmarks/) · 1,341 works · [🖼️ Vision & language](https://md.research.elamur.ai/free/en/topics/multimodal/) · 811 works · [✍️ Text generation](https://md.research.elamur.ai/free/en/topics/text-generation/) · 214 works

## Cite

Cite as: Yubo Shan, Kun Zhang, Qiming Xu, Liping Cao, Yingying Cao, Jian Zhang, Yu Wang, Jingyuan Li and Yuanzhuo Wang. 2026. FactVerse: A Benchmark for Factual Consistency in Interleaved Image–Text Generation. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{shan2026factverse,
  title = {{FactVerse: A Benchmark for Factual Consistency in Interleaved Image–Text Generation}},
  author = {Yubo Shan, Kun Zhang, Qiming Xu, Liping Cao, Yingying Cao, Jian Zhang, Yu Wang, Jingyuan Li and Yuanzhuo Wang},
  booktitle = {Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2026},
  doi = {10.18653/v1/2026.acl-long.1323},
  url = {https://doi.org/10.18653/v1/2026.acl-long.1323}
}
```

- [← previous FactSelfCheck: Fact-Level Black-Box Hallucination Detection for LLMs](https://md.research.elamur.ai/free/en/papers/sawczyn-2026-factselfcheck-fact-level-black-box/)
- [next → FADE: Probing the Limits of VLMs on fine-grained OCR](https://md.research.elamur.ai/free/en/papers/shah-2026-fade-probing-limits-vlms-fine/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/33/)

## References

54 references, 25 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Firoj Alam, Stefano Cresci, Tanmoy Chakraborty, Fabrizio Silvestri, Dimiter Dimitrov, Giovanni Da San Martino, Shaden Shaar, Hamed Firooz, and Preslav Nakov. 2022. A survey on multimodal disinformation detection. In Proceedings of the 29th international conference on computational linguistics, pages… [find](https://scholar.google.com/scholar?q=Firoj+Alam%2C+Stefano+Cresci%2C+Tanmoy+Chakraborty%2C+Fabrizio+Silvestri%2C+Dimiter+Dimitrov%2C+Giovanni+Da+San+Martino%2C+Shaden+Shaar%2C+Hamed+Firooz%2C+and+Preslav+Nakov%2E+2022%2E+A+survey+on+multimodal+disinformatio)
2. Jean-Baptiste Alayrac, Jeff Donahue, Pauline Luc, Antoine Miech, Iain Barr, and 1 others. 2022. [Flamingo: a visual language model for few-shot learning](https://md.research.elamur.ai/free/en/papers/alayrac-2022-flamingo-visual-language-model-few/). Advances in Neural Information Processing Systems, 35:23716–23736.
3. Jie An, Zhengyuan Yang, Linjie Li, Jianfeng Wang, Kevin Lin, Zicheng Liu, Lijuan Wang, and Jiebo Luo. 2023. Openleaf: Open-domain interleaved image-text generation and evaluation. arXiv preprint arXiv:2310.07749. — [arXiv:2310.07749](https://arxiv.org/abs/2310.07749) [find](https://scholar.google.com/scholar?q=Jie+An%2C+Zhengyuan+Yang%2C+Linjie+Li%2C+Jianfeng+Wang%2C+Kevin+Lin%2C+Zicheng+Liu%2C+Lijuan+Wang%2C+and+Jiebo+Luo%2E+2023%2E+Openleaf%3A+Open%2Ddomain+interleaved+image%2Dtext+generation+and+evaluation%2E+arXiv+preprint+arXiv)
4. Lorin W Anderson and David R Krathwohl. 2001. A taxonomy for learning, teaching, and assessing: A revision of Bloom’s taxonomy of educational objectives: complete edition. Addison Wesley Longman, Inc. [find](https://scholar.google.com/scholar?q=Lorin+W+Anderson+and+David+R+Krathwohl%2E+2001%2E+A+taxonomy+for+learning%2C+teaching%2C+and+assessing%3A+A+revision+of+Bloom%E2%80%99s+taxonomy+of+educational+objectives%3A+complete+edition%2E+Addison+Wesley+Longman%2C+Inc%2E)
5. Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, and 1 others. 2025. Qwen2. 5-vl technical report. arXiv preprint arXiv:2502.13923. — [Qwen2.5-VL Technical Report](https://md.research.elamur.ai/free/en/papers/bai-2025-qwen2-5-vl-technical/)
6. James Betker, Gabriel Goh, Li Jing, Tim Brooks, Jianfeng Wang, Linjie Li, Long Ouyang, Juntang Zhuang, Joyce Lee, Yufei Guo, and 1 others. 2023. Improving image generation with better captions. Computer Science. [https://cdn.openai](https://cdn.openai). com/papers/dall-e-3. pdf, 2(3):8. [↗](https://cdn.openai)
7. Black Forest Labs. 2024. Flux.1: Text-to-image generation model. [https://github.com/](https://github.com/) black-forest-labs/flux. Accessed: 2025-12-28. [↗](https://github.com/)
8. Dongping Chen, Ruoxi Chen, Shu Pu, Zhaoyi Liu, Yanru Wu, Caixi Chen, Benlin Liu, Yue Huang, Yao Wan, Pan Zhou, and 1 others. 2024a. Interleaved scene graphs for interleaved text-and-image generation assessment. arXiv preprint arXiv:2411.17188. [find](https://scholar.google.com/scholar?q=Dongping+Chen%2C+Ruoxi+Chen%2C+Shu+Pu%2C+Zhaoyi+Liu%2C+Yanru+Wu%2C+Caixi+Chen%2C+Benlin+Liu%2C+Yue+Huang%2C+Yao+Wan%2C+Pan+Zhou%2C+and+1+others%2E+2024a%2E+Interleaved+scene+graphs+for+interleaved+text%2Dand%2Dimage+generation+a)
9. Lin Chen, Jinsong Li, Xiaoyi Dong, Pan Zhang, Yuhang Zang, Zehui Chen, Haodong Duan, Jiaqi Wang, Yu Qiao, Dahua Lin, and 1 others. 2024b. [Are we on the right way for evaluating large vision-language models?](https://md.research.elamur.ai/free/en/papers/chen-2024-right-way-evaluating-large-vision/) Advances in Neural Information Processing Systems, 37:27056–27087.
10. Zhe Chen, Jiannan Wu, Wenhai Wang, Weijie Su, Guo Chen, Sen Xing, Muyan Zhong, Qinglong Zhang, Xizhou Zhu, Lewei Lu, and 1 others. 2024c. [Internvl: Scaling up vision foundation models and aligning for generic visual-linguistic tasks](https://md.research.elamur.ai/free/en/papers/chen-2023-internvl-scaling-up-vision/). In Proceedings of the IEEE/CVF conference on computer vision and p…
11. Ethan Chern, Jiadi Su, Yan Ma, and Pengfei Liu. 2024. Anole: An open, autoregressive, native large multimodal models for interleaved image-text generation. arXiv preprint arXiv:2407.06135. — [arXiv:2407.06135](https://arxiv.org/abs/2407.06135) [find](https://scholar.google.com/scholar?q=Ethan+Chern%2C+Jiadi+Su%2C+Yan+Ma%2C+and+Pengfei+Liu%2E+2024%2E+Anole%3A+An+open%2C+autoregressive%2C+native+large+multimodal+models+for+interleaved+image%2Dtext+generation%2E+arXiv+preprint+arXiv%3A2407%2E06135%2E+%E2%80%94+%5BarXiv%3A24)
12. Jaemin Cho, Yushi Hu, Roopal Garg, Peter Anderson, Ranjay Krishna, Jason Baldridge, Mohit Bansal, Jordi Pont-Tuset, and Su Wang. 2023. Davidsonian scene graph: Improving reliability in fine-grained evaluation for text-to-image generation. arXiv preprint arXiv:2310.18235. [find](https://scholar.google.com/scholar?q=Jaemin+Cho%2C+Yushi+Hu%2C+Roopal+Garg%2C+Peter+Anderson%2C+Ranjay+Krishna%2C+Jason+Baldridge%2C+Mohit+Bansal%2C+Jordi+Pont%2DTuset%2C+and+Su+Wang%2E+2023%2E+Davidsonian+scene+graph%3A+Improving+reliability+in+fine%2Dgrained+ev)
13. Gheorghe Comanici, Eric Bieber, Mike Schaekermann, Ice Pasupat, Noveen Sachdeva, Inderjit Dhillon, Marcel Blistein, Ori Ram, Dan Zhang, Evan Rosen, and 1 others. 2025. [Gemini 2.5: Pushing the frontier with advanced reasoning, multimodality, long context, and next generation agentic capabilities](https://md.research.elamur.ai/free/en/papers/comanici-2025-gemini-2-5-pushing-frontier/). arX…
14. Patrick Esser, Sumith Kulal, Andreas Blattmann, Rahim Entezari, Jonas Müller, Harry Saini, Yam Levi, Dominik Lorenz, Axel Sauer, Frederic Boesel, and 1 others. 2024. Scaling rectified flow transformers for high-resolution image synthesis. In Forty-first international conference on machine learning. [find](https://scholar.google.com/scholar?q=Patrick+Esser%2C+Sumith+Kulal%2C+Andreas+Blattmann%2C+Rahim+Entezari%2C+Jonas+M%C3%BCller%2C+Harry+Saini%2C+Yam+Levi%2C+Dominik+Lorenz%2C+Axel+Sauer%2C+Frederic+Boesel%2C+and+1+others%2E+2024%2E+Scaling+rectified+flow+transformer)
15. Chaoyou Fu, Peixian Chen, Yunhang Shen, Yulei Qin, Mengdan Zhang, Xu Lin, Jinrui Yang, Xiawu Zheng, Ke Li, Xing Sun, and 1 others. 2025. [Mme: A comprehensive evaluation benchmark for multimodal large language models](https://md.research.elamur.ai/free/en/papers/fu-2023-mme-comprehensive-evaluation-benchmark-multimodal/). In The Thirty-ninth Annual Conference on Neural Information Processing Systems Data…
16. Yuying Ge, Sijie Zhao, Jinguo Zhu, Yixiao Ge, Kun Yi, Lin Song, Chen Li, Xiaohan Ding, and Ying Shan. 2024. Seed-x: Multimodal models with unified multi-granularity comprehension and generation. arXiv preprint arXiv:2404.14396. — [arXiv:2404.14396](https://arxiv.org/abs/2404.14396) [find](https://scholar.google.com/scholar?q=Yuying+Ge%2C+Sijie+Zhao%2C+Jinguo+Zhu%2C+Yixiao+Ge%2C+Kun+Yi%2C+Lin+Song%2C+Chen+Li%2C+Xiaohan+Ding%2C+and+Ying+Shan%2E+2024%2E+Seed%2Dx%3A+Multimodal+models+with+unified+multi%2Dgranularity+comprehension+and+generation%2E+arXiv)
17. Dhruba Ghosh, Hannaneh Hajishirzi, and Ludwig Schmidt. 2023. Geneval: An object-focused framework for evaluating text-to-image alignment. Advances in Neural Information Processing Systems, 36:52132–52152. [find](https://scholar.google.com/scholar?q=Dhruba+Ghosh%2C+Hannaneh+Hajishirzi%2C+and+Ludwig+Schmidt%2E+2023%2E+Geneval%3A+An+object%2Dfocused+framework+for+evaluating+text%2Dto%2Dimage+alignment%2E+Advances+in+Neural+Information+Processing+Systems%2C+36%3A52132%E2%80%9352)
18. Jiawei Gu, Xuhui Jiang, Zhichao Shi, Hexiang Tan, Xuehao Zhai, Chengjin Xu, Wei Li, Yinghan Shen, Shengjie Ma, Honghao Liu, and 1 others. 2024. A survey on llm-as-a-judge. The Innovation. [find](https://scholar.google.com/scholar?q=Jiawei+Gu%2C+Xuhui+Jiang%2C+Zhichao+Shi%2C+Hexiang+Tan%2C+Xuehao+Zhai%2C+Chengjin+Xu%2C+Wei+Li%2C+Yinghan+Shen%2C+Shengjie+Ma%2C+Honghao+Liu%2C+and+1+others%2E+2024%2E+A+survey+on+llm%2Das%2Da%2Djudge%2E+The+Innovation%2E)
19. Arnav Gudibande, Eric Wallace, Charlie Snell, Xinyang Geng, Hao Liu, Pieter Abbeel, Sergey Levine, and Dawn Song. 2023. [The false promise of imitating proprietary llms](https://md.research.elamur.ai/free/en/papers/gudibande-2023-false-promise-imitating-proprietary-llms/). arXiv preprint arXiv:2305.15717.
20. Jack Hessel, Ari Holtzman, Maxwell Forbes, Ronan Le Bras, and Yejin Choi. 2021. [Clipscore: A reference-free evaluation metric for image captioning](https://md.research.elamur.ai/free/en/papers/hessel-2021-clipscore-reference-free-evaluation-metric/). In Proceedings of the 2021 conference on empirical methods in natural language processing, pages 7514– 7528.
21. Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. 2017. Gans trained by a two time-scale update rule converge to a local nash equilibrium. Advances in neural information processing systems, 30. [find](https://scholar.google.com/scholar?q=Martin+Heusel%2C+Hubert+Ramsauer%2C+Thomas+Unterthiner%2C+Bernhard+Nessler%2C+and+Sepp+Hochreiter%2E+2017%2E+Gans+trained+by+a+two+time%2Dscale+update+rule+converge+to+a+local+nash+equilibrium%2E+Advances+in+neural+i)
22. Yushi Hu, Benlin Liu, Jungo Kasai, Yizhong Wang, Mari Ostendorf, Ranjay Krishna, and Noah A Smith. 2023. [Tifa: Accurate and interpretable text-to-image faithfulness evaluation with question answering](https://md.research.elamur.ai/free/en/papers/hu-2023-tifa-accurate-interpretable-text-image/). In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 20406–20417.
23. Yue Huang, Lichao Sun, Haoran Wang, Siyuan Wu, Qihui Zhang, Yuan Li, Chujie Gao, Yixin Huang, Wenhan Lyu, Yixuan Zhang, and 1 others. 2024. [Trustllm: Trustworthiness in large language models](https://md.research.elamur.ai/free/en/papers/huang-2024-trustllm-trustworthiness-large-language-models/). arXiv preprint arXiv:2401.05561.
24. Aaron Hurst, Adam Lerer, Adam P Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Ostrow, Akila Welihinda, Alan Hayes, Alec Radford, and 1 others. 2024. [Gpt-4o system card](https://md.research.elamur.ai/free/en/papers/openai-2024-gpt-4o-system-card/). arXiv preprint arXiv:2410.21276.
25. Jing Yu Koh, Daniel Fried, and Russ R Salakhutdinov. 2023. Generating images with multimodal language models. Advances in Neural Information Processing Systems, 36:21487–21506. [find](https://scholar.google.com/scholar?q=Jing+Yu+Koh%2C+Daniel+Fried%2C+and+Russ+R+Salakhutdinov%2E+2023%2E+Generating+images+with+multimodal+language+models%2E+Advances+in+Neural+Information+Processing+Systems%2C+36%3A21487%E2%80%9321506%2E)
26. Bohao Li, Rui Wang, Guangzhi Wang, Yuying Ge, Yixiao Ge, and Ying Shan. 2023. [Seed-bench: Benchmarking multimodal llms with generative comprehension](https://md.research.elamur.ai/free/en/papers/li-2023-seed-bench-benchmarking-multimodal-llms/). arXiv preprint arXiv:2307.16125.
27. Chin-Yew Lin. 2004. [Rouge: A package for automatic evaluation of summaries](https://md.research.elamur.ai/free/en/papers/lin-2004-rouge-package-automatic-evaluation-summaries/). In Text summarization branches out, pages 74–81.
28. Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. 2023. [Visual instruction tuning](https://md.research.elamur.ai/free/en/papers/liu-2023-visual-instruction-tuning/). Advances in neural information processing systems, 36:34892– 34916.
29. Minqian Liu, Zhiyang Xu, Zihao Lin, Trevor Ashby, Joy Rimchala, Jiaxin Zhang, and Lifu Huang. 2024a. Holistic evaluation for interleaved text-and-image generation. arXiv preprint arXiv:2406.14643. [find](https://scholar.google.com/scholar?q=Minqian+Liu%2C+Zhiyang+Xu%2C+Zihao+Lin%2C+Trevor+Ashby%2C+Joy+Rimchala%2C+Jiaxin+Zhang%2C+and+Lifu+Huang%2E+2024a%2E+Holistic+evaluation+for+interleaved+text%2Dand%2Dimage+generation%2E+arXiv+preprint+arXiv%3A2406%2E14643%2E)
30. Yuan Liu, Haodong Duan, Yuanhan Zhang, Bo Li, Songyang Zhang, Wangbo Zhao, Yike Yuan, Jiaqi Wang, Conghui He, Ziwei Liu, and 1 others. 2024b. [Mmbench: Is your multi-modal model an all-around player?](https://md.research.elamur.ai/free/en/papers/liu-2023-mmbench-multi-modal-model-all/) In European conference on computer vision, pages 216–233. Springer.
31. Yujie Lu, Xianjun Yang, Xiujun Li, Xin Eric Wang, and William Yang Wang. 2023. Llmscore: Unveiling the power of large language models in text-to-image synthesis evaluation. Advances in neural information processing systems, 36:23075–23093. [find](https://scholar.google.com/scholar?q=Yujie+Lu%2C+Xianjun+Yang%2C+Xiujun+Li%2C+Xin+Eric+Wang%2C+and+William+Yang+Wang%2E+2023%2E+Llmscore%3A+Unveiling+the+power+of+large+language+models+in+text%2Dto%2Dimage+synthesis+evaluation%2E+Advances+in+neural+informat)
32. Ninareh Mehrabi, Palash Goyal, Apurv Verma, Jwala Dhamala, Varun Kumar, Qian Hu, Kai-Wei Chang, Richard Zemel, Aram Galstyan, and Rahul Gupta. 2023. Resolving ambiguities in text-to-image generative models. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (V… [find](https://scholar.google.com/scholar?q=Ninareh+Mehrabi%2C+Palash+Goyal%2C+Apurv+Verma%2C+Jwala+Dhamala%2C+Varun+Kumar%2C+Qian+Hu%2C+Kai%2DWei+Chang%2C+Richard+Zemel%2C+Aram+Galstyan%2C+and+Rahul+Gupta%2E+2023%2E+Resolving+ambiguities+in+text%2Dto%2Dimage+generative+m)
33. OpenAI. 2025. Gpt-5 system card. Technical report, OpenAI. [find](https://scholar.google.com/scholar?q=OpenAI%2E+2025%2E+Gpt%2D5+system+card%2E+Technical+report%2C+OpenAI%2E)
34. Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, and 1 others. 2022. [Training language models to follow instructions with human feedback](https://md.research.elamur.ai/free/en/papers/ouyang-2022-training-language-models-follow-instructions/). Advances in neural information processing systems, 35:27730–27744.
35. Arjun Panickssery, Samuel Bowman, and Shi Feng. 2024. Llm evaluators recognize and favor their own generations. Advances in Neural Information Processing Systems, 37:68772–68802. [find](https://scholar.google.com/scholar?q=Arjun+Panickssery%2C+Samuel+Bowman%2C+and+Shi+Feng%2E+2024%2E+Llm+evaluators+recognize+and+favor+their+own+generations%2E+Advances+in+Neural+Information+Processing+Systems%2C+37%3A68772%E2%80%9368802%2E)
36. Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. [Bleu: a method for automatic evaluation of machine translation](https://md.research.elamur.ai/free/en/papers/papineni-2002-bleu-method-automatic-evaluation-machine/). In Proceedings of the 40th annual meeting of the Association for Computational Linguistics, pages 311–318.
37. Alec Radford, Karthik Narasimhan, Tim Salimans, Ilya Sutskever, and 1 others. 2018. Improving language understanding by generative pre-training. [find](https://scholar.google.com/scholar?q=Alec+Radford%2C+Karthik+Narasimhan%2C+Tim+Salimans%2C+Ilya+Sutskever%2C+and+1+others%2E+2018%2E+Improving+language+understanding+by+generative+pre%2Dtraining%2E)
38. Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. 2022. [High-resolution image synthesis with latent diffusion models](https://md.research.elamur.ai/free/en/papers/rombach-2021-high-resolution-image-synthesis-latent/). In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10684–10695.
39. Tim Salimans, Ian Goodfellow, Wojciech Zaremba, Vicki Cheung, Alec Radford, and Xi Chen. 2016. [Improved techniques for training gans](https://md.research.elamur.ai/free/en/papers/salimans-2016-improved-techniques-training-gans/). Advances in neural information processing systems, 29.
40. S Shyam Sundar, Maria D Molina, and Eugene Cho. 2021. Seeing is believing: Is video modality more powerful in spreading fake news via online messaging apps? Journal of Computer-Mediated Communication, 26(6):301–319. [find](https://scholar.google.com/scholar?q=S+Shyam+Sundar%2C+Maria+D+Molina%2C+and+Eugene+Cho%2E+2021%2E+Seeing+is+believing%3A+Is+video+modality+more+powerful+in+spreading+fake+news+via+online+messaging+apps%3F+Journal+of+Computer%2DMediated+Communication%2C)
41. Gemini Team, Petko Georgiev, Ving Ian Lei, Ryan Burnell, Libin Bai, Anmol Gulati, Garrett Tanzer, Damien Vincent, Zhufeng Pan, Shibo Wang, and 1 others. 2024. [Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context](https://md.research.elamur.ai/free/en/papers/gemini-team-2024-gemini-1-5-unlocking-multimodal/). arXiv preprint arXiv:2403.05530.
42. Peng Wang, Shuai Bai, Sinan Tan, Shijie Wang, Zhihao Fan, Jinze Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, and 1 others. 2024a. [Qwen2-vl: Enhancing vision-language model’s perception of the world at any resolution](https://md.research.elamur.ai/free/en/papers/wang-2024-qwen2-vl-enhancing-vision/). arXiv preprint arXiv:2409.12191.
43. Xinlong Wang, Xiaosong Zhang, Zhengxiong Luo, Quan Sun, Yufeng Cui, Jinsheng Wang, Fan Zhang, Yueze Wang, Zhen Li, Qiying Yu, and 1 others. 2024b. Emu3: Next-token prediction is all you need. arXiv preprint arXiv:2409.18869. — [arXiv:2409.18869](https://arxiv.org/abs/2409.18869) [find](https://scholar.google.com/scholar?q=Xinlong+Wang%2C+Xiaosong+Zhang%2C+Zhengxiong+Luo%2C+Quan+Sun%2C+Yufeng+Cui%2C+Jinsheng+Wang%2C+Fan+Zhang%2C+Yueze+Wang%2C+Zhen+Li%2C+Qiying+Yu%2C+and+1+others%2E+2024b%2E+Emu3%3A+Next%2Dtoken+prediction+is+all+you+need%2E+arXiv+pr)
44. Shengqiong Wu, Hao Fei, Leigang Qu, Wei Ji, and Tat-Seng Chua. 2024a. [Next-gpt: Any-to-any multimodal llm](https://md.research.elamur.ai/free/en/papers/wu-2023-next-gpt-any-any-multimodal/). In Forty-first International Conference on Machine Learning.
45. Yecheng Wu, Zhuoyang Zhang, Junyu Chen, Haotian Tang, Dacheng Li, Yunhao Fang, Ligeng Zhu, Enze Xie, Hongxu Yin, Li Yi, and 1 others. 2024b. Vila-u: a unified foundation model integrating visual understanding and generation. arXiv preprint arXiv:2409.04429. — [arXiv:2409.04429](https://arxiv.org/abs/2409.04429) [find](https://scholar.google.com/scholar?q=Yecheng+Wu%2C+Zhuoyang+Zhang%2C+Junyu+Chen%2C+Haotian+Tang%2C+Dacheng+Li%2C+Yunhao+Fang%2C+Ligeng+Zhu%2C+Enze+Xie%2C+Hongxu+Yin%2C+Li+Yi%2C+and+1+others%2E+2024b%2E+Vila%2Du%3A+a+unified+foundation+model+integrating+visual+under)
46. Peng Xia, Siwei Han, Shi Qiu, Yiyang Zhou, Zhaoyang Wang, Wenhao Zheng, Zhaorun Chen, Chenhang Cui, Mingyu Ding, Linjie Li, and 1 others. 2024. Mmie: Massive multimodal interleaved comprehension benchmark for large vision-language models. arXiv preprint arXiv:2410.10139. — [arXiv:2410.10139](https://arxiv.org/abs/2410.10139) [find](https://scholar.google.com/scholar?q=Peng+Xia%2C+Siwei+Han%2C+Shi+Qiu%2C+Yiyang+Zhou%2C+Zhaoyang+Wang%2C+Wenhao+Zheng%2C+Zhaorun+Chen%2C+Chenhang+Cui%2C+Mingyu+Ding%2C+Linjie+Li%2C+and+1+others%2E+2024%2E+Mmie%3A+Massive+multimodal+interleaved+comprehension+bench)
47. Jinheng Xie, Weijia Mao, Zechen Bai, David Junhao Zhang, Weihao Wang, Kevin Qinghong Lin, Yuchao Gu, Zhijie Chen, Zhenheng Yang, and Mike Zheng Shou. 2024. Show-o: One single transformer to unify multimodal understanding and generation. arXiv preprint arXiv:2408.12528. — [arXiv:2408.12528](https://arxiv.org/abs/2408.12528) [find](https://scholar.google.com/scholar?q=Jinheng+Xie%2C+Weijia+Mao%2C+Zechen+Bai%2C+David+Junhao+Zhang%2C+Weihao+Wang%2C+Kevin+Qinghong+Lin%2C+Yuchao+Gu%2C+Zhijie+Chen%2C+Zhenheng+Yang%2C+and+Mike+Zheng+Shou%2E+2024%2E+Show%2Do%3A+One+single+transformer+to+unify+mult)
48. An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, and 1 others. 2025. [Qwen3 technical report](https://md.research.elamur.ai/free/en/papers/yang-2025-qwen3-technical-report/). arXiv preprint arXiv:2505.09388.
49. Kaizhi Zheng, Xuehai He, and Xin Eric Wang. 2023a. Minigpt-5: Interleaved vision-and-language generation via generative vokens. arXiv preprint arXiv:2310.02239. — [arXiv:2310.02239](https://arxiv.org/abs/2310.02239) [find](https://scholar.google.com/scholar?q=Kaizhi+Zheng%2C+Xuehai+He%2C+and+Xin+Eric+Wang%2E+2023a%2E+Minigpt%2D5%3A+Interleaved+vision%2Dand%2Dlanguage+generation+via+generative+vokens%2E+arXiv+preprint+arXiv%3A2310%2E02239%2E+%E2%80%94+%5BarXiv%3A2310%2E02239%5D%28https%3A%2F%2Farxiv%2Eorg%2F)
50. Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric Xing, and 1 others. 2023b. [Judging llm-as-a-judge with mt-bench and chatbot arena](https://md.research.elamur.ai/free/en/papers/zheng-2023-judging-llm-judge-mt-bench/). Advances in neural information processing systems, 36:46595–46623.
51. Chunting Zhou, Lili Yu, Arun Babu, Kushal Tirumala, Michihiro Yasunaga, Leonid Shamis, Jacob Kahn, Xuezhe Ma, Luke Zettlemoyer, and Omer Levy. 2024. [Transfusion: Predict the next token and diffuse images with one multi-modal model](https://md.research.elamur.ai/free/en/papers/zhou-2024-transfusion-predict-next-token-diffuse/). arXiv preprint arXiv:2408.11039.
52. Pengfei Zhou, Xiaopeng Peng, Jiajun Song, Chuanhao Li, Zhaopan Xu, Yue Yang, Ziyao Guo, Hao Zhang, Yuqi Lin, Yefei He, and 1 others. 2025. Opening: A comprehensive benchmark for judging open-ended interleaved image-text generation. In Proceedings of the Computer Vision and Pattern Recognition Confer… [find](https://scholar.google.com/scholar?q=Pengfei+Zhou%2C+Xiaopeng+Peng%2C+Jiajun+Song%2C+Chuanhao+Li%2C+Zhaopan+Xu%2C+Yue+Yang%2C+Ziyao+Guo%2C+Hao+Zhang%2C+Yuqi+Lin%2C+Yefei+He%2C+and+1+others%2E+2025%2E+Opening%3A+A+comprehensive+benchmark+for+judging+open%2Dended+int)
53. Wanrong Zhu, Jack Hessel, Anas Awadalla, Samir Yitzhak Gadre, Jesse Dodge, Alex Fang, Youngjae Yu, Ludwig Schmidt, William Yang Wang, and Yejin Choi. 2023. Multimodal c4: An open, billion-scale corpus of images interleaved with text. Advances in Neural Information Processing Systems, 36:8958–8974. [find](https://scholar.google.com/scholar?q=Wanrong+Zhu%2C+Jack+Hessel%2C+Anas+Awadalla%2C+Samir+Yitzhak+Gadre%2C+Jesse+Dodge%2C+Alex+Fang%2C+Youngjae+Yu%2C+Ludwig+Schmidt%2C+William+Yang+Wang%2C+and+Yejin+Choi%2E+2023%2E+Multimodal+c4%3A+An+open%2C+billion%2Dscale+corpus)
54. Xianwei Zhuang, Yuxin Xie, Yufan Deng, Liming Liang, Jinghan Ru, Yuguo Yin, and Yuexian Zou. 2025. Vargpt: Unified understanding and generation in a visual autoregressive multimodal large language model. arXiv preprint arXiv:2501.12327. — [arXiv:2501.12327](https://arxiv.org/abs/2501.12327) [find](https://scholar.google.com/scholar?q=Xianwei+Zhuang%2C+Yuxin+Xie%2C+Yufan+Deng%2C+Liming+Liang%2C+Jinghan+Ru%2C+Yuguo+Yin%2C+and+Yuexian+Zou%2E+2025%2E+Vargpt%3A+Unified+understanding+and+generation+in+a+visual+autoregressive+multimodal+large+language+mod)

*Updated 2 November 2026*