---
title: "Accelerating Training of Autoregressive Video Generation Models"
description: "The authors first test a simple shortcut: training on fewer frames and extending videos step by step at inference. 48 references."
authors: ["Yucheng Zhou", "Jianbing Shen"]
year: 2026
venue: "Findings of the Association for Computational Linguistics: ACL 2026"
source: "https://aclanthology.org/2026.findings-acl.1501/"
doi: "10.18653/v1/2026.findings-acl.1501"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/zhou-2026-accelerating-training-autoregressive-video-generation/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/zhou-2026-accelerating-training-autoregressive-video-generation/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/21/) / Accelerating Training of Autoregressive Video Generation Models via Local Optimization with Representation Continuity

# Accelerating Training of Autoregressive Video Generation Models via Local Optimization with Representation Continuity

**Authors:** [Yucheng Zhou](https://md.research.elamur.ai/free/en/authors/yucheng-zhou/) and [Jianbing Shen](https://md.research.elamur.ai/free/en/authors/jianbing-shen/) · **Year:** 2026 · **Venue:** [Findings of the Association for Computational Linguistics: ACL 2026](https://md.research.elamur.ai/free/en/venues/findings/) ([Findings of the Association for Computational Linguistics 2026](https://md.research.elamur.ai/free/en/venues/findings/2026/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2026.findings-acl.1501/) · [PDF](https://aclanthology.org/2026.findings-acl.1501.pdf) · [DOI 10.18653/v1/2026.findings-acl.1501](https://doi.org/10.18653/v1/2026.findings-acl.1501) · [OpenAlex](https://openalex.org/W7166840277)

**Citations:** 0 citations (OpenAlex, 2026-10-14)

## Summary
Autoregressive video generators predict long sequences of discrete VQ-VAE tokens, which makes training slow and costly. The authors first test a simple shortcut: training on fewer frames and extending videos step by step at inference. This trains faster, but inference becomes slower and errors compound, so frames lose coherence. They show this both theoretically and through PSNR and optical-flow measurements.

Their fix, Local Optimization (Local Opt.), computes the loss only on tokens inside randomly placed, overlapping windows of frame blocks, while earlier tokens act as context. Windows are sampled more often at the start of the video, because poor early frames degrade the later ones. A second component, Representation Continuity (ReCo), is motivated by Lipschitz continuity. It adds a loss term that discourages sudden jumps in hidden representations between steps.

Experiments cover class-conditional benchmarks (FFS, SKY, UCF, Taichi) and text-to-video generation (trained on Vimeo, evaluated zero-shot on MSR-VTT). The approach matches or beats the baselines at about half the training cost. Cheaper training of this kind is relevant to building unified multimodal LLMs.

## Key points
- **Fewer-Frames baseline is a poor shortcut.** It trains roughly 2.5× faster, but FVD collapses compared with the Baseline:
  - FFS: 229.32 vs 73.65
  - SKY: 292.41 vs 89.09
- **Training speed.** Local Opt. with ReCo trains about 2× faster than the Baseline, keeps baseline-level inference time, and matches the Baseline in PSNR-based consistency.
- **Class-conditional results (FVD, lower is better).** The 770M ReCo⋆ beats Baseline⋆ on all four datasets:

  | Dataset | ReCo⋆ | Baseline⋆ |
  |---|---|---|
  | FFS | 42.5 | 46.1 |
  | SKY | 58.8 | 62.7 |
  | UCF | 251.4 | 254.5 |
  | Taichi | 98.3 | 105.5 |

- **Compatibility with a stronger tokenizer.** Combined with the LARP tokenizer, ReCo♠ reaches FVD 46.2 on FFS and 56.1 on UCF, improving on LARP itself (62.6 and 57.0).
- **Text-to-video (Loong 7B, MSR-VTT zero-shot).**
  - At equal training cost, ReCo raises CLIP Score from 0.2903 to 0.3056 and lowers FVD from 274 to 212.
  - ReCo* gets 0.2911 CLIP and 267 FVD using only 50% of the training cost.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [University of Macau](https://md.research.elamur.ai/free/en/universities/university-of-macau/) — University, Macao, China · 43 works · 2,368 citations

📚 Shelves (2): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 18,614 works · [👁️ Computer Vision and Pattern Recognition](https://md.research.elamur.ai/free/en/shelves/computer-science/computer-vision-and-pattern-recognition/) · 3,486 works

## Cite

Cite as: Yucheng Zhou and Jianbing Shen. 2026. Accelerating Training of Autoregressive Video Generation Models via Local Optimization with Representation Continuity. Findings of the Association for Computational Linguistics: ACL 2026.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{zhou2026accelerating,
  title = {{Accelerating Training of Autoregressive Video Generation Models via Local Optimization with Representation Continuity}},
  author = {Yucheng Zhou and Jianbing Shen},
  booktitle = {Findings of the Association for Computational Linguistics: ACL 2026},
  year = {2026},
  doi = {10.18653/v1/2026.findings-acl.1501},
  url = {https://doi.org/10.18653/v1/2026.findings-acl.1501}
}
```

- [← previous Accelerating LLM Fine-Tuning via Embedding Knowledge Transfer](https://md.research.elamur.ai/free/en/papers/peng-2026-accelerating-llm-fine-tuning-embedding/)
- [next → Acceleration of Backpropagation in Linear Layers of Transformer Models Based on Gradient Structure](https://md.research.elamur.ai/free/en/papers/topchii-2026-acceleration-backpropagation-linear-layers-transformer/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/21/)

## References

48 references, 3 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Andreas Blattmann, Tim Dockhorn, Sumith Kulal, Daniel Mendelevitch, Maciej Kilian, Dominik Lorenz, Yam Levi, Zion English, Vikram Voleti, Adam Letts, Varun Jampani, and Robin Rombach. 2023. Stable video diffusion: Scaling latent video diffusion models to large datasets. CoRR, abs/2311.15127. — [arXiv:2311.15127](https://arxiv.org/abs/2311.15127) [find](https://scholar.google.com/scholar?q=Andreas+Blattmann%2C+Tim+Dockhorn%2C+Sumith+Kulal%2C+Daniel+Mendelevitch%2C+Maciej+Kilian%2C+Dominik+Lorenz%2C+Yam+Levi%2C+Zion+English%2C+Vikram+Voleti%2C+Adam+Letts%2C+Varun+Jampani%2C+and+Robin+Rombach%2E+2023%2E+Stable+vid)
2. Guanjie Chen, Xinyu Zhao, Yucheng Zhou, Tianlong Chen, and Yu Cheng. 2024. Accelerating vision diffusion transformers with skip branches. CoRR, abs/2411.17616. — [arXiv:2411.17616](https://arxiv.org/abs/2411.17616) [find](https://scholar.google.com/scholar?q=Guanjie+Chen%2C+Xinyu+Zhao%2C+Yucheng+Zhou%2C+Tianlong+Chen%2C+and+Yu+Cheng%2E+2024%2E+Accelerating+vision+diffusion+transformers+with+skip+branches%2E+CoRR%2C+abs%2F2411%2E17616%2E+%E2%80%94+%5BarXiv%3A2411%2E17616%5D%28https%3A%2F%2Farxiv%2Eorg%2Fa)
3. Guanjie Chen, Xinyu Zhao, Yucheng Zhou, Xiaoye Qu, Tianlong Chen, and Yu Cheng. 2025. Towards stabilized and efficient diffusion transformers through long-skip-connections with spectral constraints. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 17708–17718. [find](https://scholar.google.com/scholar?q=Guanjie+Chen%2C+Xinyu+Zhao%2C+Yucheng+Zhou%2C+Xiaoye+Qu%2C+Tianlong+Chen%2C+and+Yu+Cheng%2E+2025%2E+Towards+stabilized+and+efficient+diffusion+transformers+through+long%2Dskip%2Dconnections+with+spectral+constraints%2E+I)
4. Haoge Deng, Ting Pan, Haiwen Diao, Zhengxiong Luo, Yufeng Cui, Huchuan Lu, Shiguang Shan, Yonggang Qi, and Xinlong Wang. 2024. Autoregressive video generation without vector quantization. arXiv preprint arXiv:2412.14169. — [arXiv:2412.14169](https://arxiv.org/abs/2412.14169) [find](https://scholar.google.com/scholar?q=Haoge+Deng%2C+Ting+Pan%2C+Haiwen+Diao%2C+Zhengxiong+Luo%2C+Yufeng+Cui%2C+Huchuan+Lu%2C+Shiguang+Shan%2C+Yonggang+Qi%2C+and+Xinlong+Wang%2E+2024%2E+Autoregressive+video+generation+without+vector+quantization%2E+arXiv+prepri)
5. Gunnar Farnebäck. 2003. Two-frame motion estimation based on polynomial expansion. In Image Analysis, 13th Scandinavian Conference, SCIA 2003, Halmstad, Sweden, June 29 - July 2, 2003, Proceedings, volume 2749 of Lecture Notes in Computer Science, pages 363–370. Springer. [find](https://scholar.google.com/scholar?q=Gunnar+Farneb%C3%A4ck%2E+2003%2E+Two%2Dframe+motion+estimation+based+on+polynomial+expansion%2E+In+Image+Analysis%2C+13th+Scandinavian+Conference%2C+SCIA+2003%2C+Halmstad%2C+Sweden%2C+June+29+%2D+July+2%2C+2003%2C+Proceedings%2C+vo)
6. Yingqing He, Tianyu Yang, Yong Zhang, Ying Shan, and Qifeng Chen. 2022. Latent video diffusion models for high-fidelity long video generation. arXiv preprint arXiv:2211.13221. — [arXiv:2211.13221](https://arxiv.org/abs/2211.13221) [find](https://scholar.google.com/scholar?q=Yingqing+He%2C+Tianyu+Yang%2C+Yong+Zhang%2C+Ying+Shan%2C+and+Qifeng+Chen%2E+2022%2E+Latent+video+diffusion+models+for+high%2Dfidelity+long+video+generation%2E+arXiv+preprint+arXiv%3A2211%2E13221%2E+%E2%80%94+%5BarXiv%3A2211%2E13221%5D%28htt)
7. Wenyi Hong, Ming Ding, Wendi Zheng, Xinghan Liu, and Jie Tang. 2023. [Cogvideo: Large-scale pretraining for text-to-video generation via transformers](https://md.research.elamur.ai/free/en/papers/hong-2022-cogvideo-large-scale-pretraining/). In The Eleventh International Conference on Learning Representations, ICLR 2023, Kigali, Rwanda, May 1-5, 2023. OpenReview.net.
8. Tianhong Li, Yonglong Tian, He Li, Mingyang Deng, and Kaiming He. 2024a. Autoregressive image generation without vector quantization. CoRR, abs/2406.11838. [find](https://scholar.google.com/scholar?q=Tianhong+Li%2C+Yonglong+Tian%2C+He+Li%2C+Mingyang+Deng%2C+and+Kaiming+He%2E+2024a%2E+Autoregressive+image+generation+without+vector+quantization%2E+CoRR%2C+abs%2F2406%2E11838%2E)
9. Wenhao Li, Yichao Cao, Xiu Su, Xi Lin, Shan You, Mingkai Zheng, Yi Chen, and Chang Xu. 2024b. Training-free long video generation with chain of diffusion model experts. arXiv preprint arXiv:2408.13423. — [arXiv:2408.13423](https://arxiv.org/abs/2408.13423) [find](https://scholar.google.com/scholar?q=Wenhao+Li%2C+Yichao+Cao%2C+Xiu+Su%2C+Xi+Lin%2C+Shan+You%2C+Mingkai+Zheng%2C+Yi+Chen%2C+and+Chang+Xu%2E+2024b%2E+Training%2Dfree+long+video+generation+with+chain+of+diffusion+model+experts%2E+arXiv+preprint+arXiv%3A2408%2E13423)
10. Yizhuo Li, Yuying Ge, Yixiao Ge, Ping Luo, and Ying Shan. 2024c. Dicode: Diffusion-compressed deep tokens for autoregressive video generation with language models. CoRR, abs/2412.04446. [find](https://scholar.google.com/scholar?q=Yizhuo+Li%2C+Yuying+Ge%2C+Yixiao+Ge%2C+Ping+Luo%2C+and+Ying+Shan%2E+2024c%2E+Dicode%3A+Diffusion%2Dcompressed+deep+tokens+for+autoregressive+video+generation+with+language+models%2E+CoRR%2C+abs%2F2412%2E04446%2E)
11. Xin Ma, Yaohui Wang, Gengyun Jia, Xinyuan Chen, Ziwei Liu, Yuan-Fang Li, Cunjian Chen, and Yu Qiao. 2024. Latte: Latent diffusion transformer for video generation. CoRR, abs/2401.03048. — [arXiv:2401.03048](https://arxiv.org/abs/2401.03048) [find](https://scholar.google.com/scholar?q=Xin+Ma%2C+Yaohui+Wang%2C+Gengyun+Jia%2C+Xinyuan+Chen%2C+Ziwei+Liu%2C+Yuan%2DFang+Li%2C+Cunjian+Chen%2C+and+Yu+Qiao%2E+2024%2E+Latte%3A+Latent+diffusion+transformer+for+video+generation%2E+CoRR%2C+abs%2F2401%2E03048%2E+%E2%80%94+%5BarXiv%3A2401%2E)
12. Adam Polyak, Amit Zohar, Andrew Brown, Andros Tjandra, Animesh Sinha, Ann Lee, Apoorv Vyas, Bowen Shi, Chih-Yao Ma, Ching-Yao Chuang, David Yan, Dhruv Choudhary, Dingkang Wang, Geet Sethi, Guan Pang, Haoyu Ma, Ishan Misra, Ji Hou, Jialiang Wang, and 69 others. 2024. Movie gen: A cast of media founda… — [arXiv:2410.13720](https://arxiv.org/abs/2410.13720) [find](https://scholar.google.com/scholar?q=Adam+Polyak%2C+Amit+Zohar%2C+Andrew+Brown%2C+Andros+Tjandra%2C+Animesh+Sinha%2C+Ann+Lee%2C+Apoorv+Vyas%2C+Bowen+Shi%2C+Chih%2DYao+Ma%2C+Ching%2DYao+Chuang%2C+David+Yan%2C+Dhruv+Choudhary%2C+Dingkang+Wang%2C+Geet+Sethi%2C+Guan+Pang%2C+)
13. Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, and 1 others. 2019. Language models are unsupervised multitask learners. OpenAI blog, 1(8):9. [find](https://scholar.google.com/scholar?q=Alec+Radford%2C+Jeffrey+Wu%2C+Rewon+Child%2C+David+Luan%2C+Dario+Amodei%2C+Ilya+Sutskever%2C+and+1+others%2E+2019%2E+Language+models+are+unsupervised+multitask+learners%2E+OpenAI+blog%2C+1%288%29%3A9%2E)
14. Shuhuai Ren, Shuming Ma, Xu Sun, and Furu Wei. 2025. Next block prediction: Video generation via semi-autoregressive modeling. arXiv preprint arXiv:2502.07737. — [arXiv:2502.07737](https://arxiv.org/abs/2502.07737) [find](https://scholar.google.com/scholar?q=Shuhuai+Ren%2C+Shuming+Ma%2C+Xu+Sun%2C+and+Furu+Wei%2E+2025%2E+Next+block+prediction%3A+Video+generation+via+semi%2Dautoregressive+modeling%2E+arXiv+preprint+arXiv%3A2502%2E07737%2E+%E2%80%94+%5BarXiv%3A2502%2E07737%5D%28https%3A%2F%2Farxiv%2Eorg%2Fa)
15. Andreas Rössler, Davide Cozzolino, Luisa Verdoliva, Christian Riess, Justus Thies, and Matthias Nießner. 2018. Faceforensics: A large-scale video dataset for forgery detection in human faces. CoRR, abs/1803.09179. — [arXiv:1803.09179](https://arxiv.org/abs/1803.09179) [find](https://scholar.google.com/scholar?q=Andreas+R%C3%B6ssler%2C+Davide+Cozzolino%2C+Luisa+Verdoliva%2C+Christian+Riess%2C+Justus+Thies%2C+and+Matthias+Nie%C3%9Fner%2E+2018%2E+Faceforensics%3A+A+large%2Dscale+video+dataset+for+forgery+detection+in+human+faces%2E+CoRR%2C+ab)
16. Aliaksandr Siarohin, Stéphane Lathuilière, Sergey Tulyakov, Elisa Ricci, and Nicu Sebe. 2019. First order motion model for image animation. In Advances in Neural Information Processing Systems 32: Annual Conference on Neural Information Processing Systems 2019, NeurIPS 2019, December 8-14, 2019, Van… [find](https://scholar.google.com/scholar?q=Aliaksandr+Siarohin%2C+St%C3%A9phane+Lathuili%C3%A8re%2C+Sergey+Tulyakov%2C+Elisa+Ricci%2C+and+Nicu+Sebe%2E+2019%2E+First+order+motion+model+for+image+animation%2E+In+Advances+in+Neural+Information+Processing+Systems+32%3A+Ann)
17. Han Song, Yucheng Zhou, Jianbing Shen, and Yu Cheng. 2026. From broad exploration to stable synthesis: Entropy-guided optimization for autoregressive image generation. In The Fourteenth International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Han+Song%2C+Yucheng+Zhou%2C+Jianbing+Shen%2C+and+Yu+Cheng%2E+2026%2E+From+broad+exploration+to+stable+synthesis%3A+Entropy%2Dguided+optimization+for+autoregressive+image+generation%2E+In+The+Fourteenth+International+)
18. K Soomro. 2012. Ucf101: A dataset of 101 human actions classes from videos in the wild. arXiv preprint arXiv:1212.0402. — [arXiv:1212.0402](https://arxiv.org/abs/1212.0402) [find](https://scholar.google.com/scholar?q=K+Soomro%2E+2012%2E+Ucf101%3A+A+dataset+of+101+human+actions+classes+from+videos+in+the+wild%2E+arXiv+preprint+arXiv%3A1212%2E0402%2E+%E2%80%94+%5BarXiv%3A1212%2E0402%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F1212%2E0402%29)
19. Peize Sun, Yi Jiang, Shoufa Chen, Shilong Zhang, Bingyue Peng, Ping Luo, and Zehuan Yuan. 2024a. Autoregressive model beats diffusion: Llama for scalable image generation. CoRR, abs/2406.06525. — [arXiv:2406.06525](https://arxiv.org/abs/2406.06525) [find](https://scholar.google.com/scholar?q=Peize+Sun%2C+Yi+Jiang%2C+Shoufa+Chen%2C+Shilong+Zhang%2C+Bingyue+Peng%2C+Ping+Luo%2C+and+Zehuan+Yuan%2E+2024a%2E+Autoregressive+model+beats+diffusion%3A+Llama+for+scalable+image+generation%2E+CoRR%2C+abs%2F2406%2E06525%2E+%E2%80%94+%5BarX)
20. Yutao Sun, Hangbo Bao, Wenhui Wang, Zhiliang Peng, Li Dong, Shaohan Huang, Jianyong Wang, and Furu Wei. 2024b. Multimodal latent language modeling with next-token diffusion. CoRR, abs/2412.08635. — [arXiv:2412.08635](https://arxiv.org/abs/2412.08635) [find](https://scholar.google.com/scholar?q=Yutao+Sun%2C+Hangbo+Bao%2C+Wenhui+Wang%2C+Zhiliang+Peng%2C+Li+Dong%2C+Shaohan+Huang%2C+Jianyong+Wang%2C+and+Furu+Wei%2E+2024b%2E+Multimodal+latent+language+modeling+with+next%2Dtoken+diffusion%2E+CoRR%2C+abs%2F2412%2E08635%2E+%E2%80%94+%5Ba)
21. Chameleon Team. 2024. [Chameleon: modal early-fusion foundation models](https://md.research.elamur.ai/free/en/papers/team-2024-chameleon-mixed-modal-early-fusion/). abs/2405.09818.
22. Mixed-CoRR, Sergey Tulyakov, Ming-Yu Liu, Xiaodong Yang, and Jan Kautz. 2018. Mocogan: Decomposing motion and content for video generation. In 2018 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2018, Salt Lake City, UT, USA, June 18-22, 2018, pages 1526–1535. Computer Vision Found… [find](https://scholar.google.com/scholar?q=Mixed%2DCoRR%2C+Sergey+Tulyakov%2C+Ming%2DYu+Liu%2C+Xiaodong+Yang%2C+and+Jan+Kautz%2E+2018%2E+Mocogan%3A+Decomposing+motion+and+content+for+video+generation%2E+In+2018+IEEE+Conference+on+Computer+Vision+and+Pattern+Recog)
23. Thomas Unterthiner, Sjoerd van Steenkiste, Karol Kurach, Raphaël Marinier, Marcin Michalski, and Sylvain Gelly. 2018. Towards accurate generative models of video: A new metric & challenges. CoRR, abs/1812.01717. — [arXiv:1812.01717](https://arxiv.org/abs/1812.01717) [find](https://scholar.google.com/scholar?q=Thomas+Unterthiner%2C+Sjoerd+van+Steenkiste%2C+Karol+Kurach%2C+Rapha%C3%ABl+Marinier%2C+Marcin+Michalski%2C+and+Sylvain+Gelly%2E+2018%2E+Towards+accurate+generative+models+of+video%3A+A+new+metric+%26+challenges%2E+CoRR%2C+abs%2F)
24. Ruben Villegas, Mohammad Babaeizadeh, Pieter-Jan Kindermans, Hernan Moraldo, Han Zhang, Mohammad Taghi Saffar, Santiago Castro, Julius Kunze, and Dumitru Erhan. 2023. Phenaki: Variable length video generation from open domain textual descriptions. In The Eleventh International Conference on Learning… [find](https://scholar.google.com/scholar?q=Ruben+Villegas%2C+Mohammad+Babaeizadeh%2C+Pieter%2DJan+Kindermans%2C+Hernan+Moraldo%2C+Han+Zhang%2C+Mohammad+Taghi+Saffar%2C+Santiago+Castro%2C+Julius+Kunze%2C+and+Dumitru+Erhan%2E+2023%2E+Phenaki%3A+Variable+length+video+ge)
25. Chenglin Wang, Yucheng Zhou, Shawn Chen, Tao Wang, and Kai Zhang. 2026. [Ladr: Locality-aware dynamic rescue for efficient text-to-image generation with diffusion large language models](https://md.research.elamur.ai/free/en/papers/wang-2026-ladr-locality-aware-dynamic-rescue/). arXiv preprint arXiv:2603.13450.
26. Hanyu Wang, Saksham Suri, Yixuan Ren, Hao Chen, and Abhinav Shrivastava. 2025. LARP: Tokenizing videos with a learned autoregressive generative prior. In The Thirteenth International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Hanyu+Wang%2C+Saksham+Suri%2C+Yixuan+Ren%2C+Hao+Chen%2C+and+Abhinav+Shrivastava%2E+2025%2E+LARP%3A+Tokenizing+videos+with+a+learned+autoregressive+generative+prior%2E+In+The+Thirteenth+International+Conference+on+Lea)
27. Jiuniu Wang, Hangjie Yuan, Dayou Chen, Yingya Zhang, Xiang Wang, and Shiwei Zhang. 2023a. Modelscope text-to-video technical report. CoRR, abs/2308.06571. — [arXiv:2308.06571](https://arxiv.org/abs/2308.06571) [find](https://scholar.google.com/scholar?q=Jiuniu+Wang%2C+Hangjie+Yuan%2C+Dayou+Chen%2C+Yingya+Zhang%2C+Xiang+Wang%2C+and+Shiwei+Zhang%2E+2023a%2E+Modelscope+text%2Dto%2Dvideo+technical+report%2E+CoRR%2C+abs%2F2308%2E06571%2E+%E2%80%94+%5BarXiv%3A2308%2E06571%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F23)
28. Junke Wang, Yi Jiang, Zehuan Yuan, Bingyue Peng, Zuxuan Wu, and Yu-Gang Jiang. 2024a. Omnitokenizer: A joint image-video tokenizer for visual generation. CoRR, abs/2406.09399. — [arXiv:2406.09399](https://arxiv.org/abs/2406.09399) [find](https://scholar.google.com/scholar?q=Junke+Wang%2C+Yi+Jiang%2C+Zehuan+Yuan%2C+Bingyue+Peng%2C+Zuxuan+Wu%2C+and+Yu%2DGang+Jiang%2E+2024a%2E+Omnitokenizer%3A+A+joint+image%2Dvideo+tokenizer+for+visual+generation%2E+CoRR%2C+abs%2F2406%2E09399%2E+%E2%80%94+%5BarXiv%3A2406%2E09399%5D%28htt)
29. Wenjing Wang, Huan Yang, Zixi Tuo, Huiguo He, Junchen Zhu, Jianlong Fu, and Jiaying Liu. 2023b. Videofactory: Swap attention in spatiotemporal diffusions for text-to-video generation. CoRR, abs/2305.10874. — [arXiv:2305.10874](https://arxiv.org/abs/2305.10874) [find](https://scholar.google.com/scholar?q=Wenjing+Wang%2C+Huan+Yang%2C+Zixi+Tuo%2C+Huiguo+He%2C+Junchen+Zhu%2C+Jianlong+Fu%2C+and+Jiaying+Liu%2E+2023b%2E+Videofactory%3A+Swap+attention+in+spatiotemporal+diffusions+for+text%2Dto%2Dvideo+generation%2E+CoRR%2C+abs%2F2305%2E1)
30. Yuqing Wang, Tianwei Xiong, Daquan Zhou, Zhijie Lin, Yang Zhao, Bingyi Kang, Jiashi Feng, and Xihui Liu. 2024c. Loong: Generating minute-level long videos with autoregressive language models. arXiv preprint arXiv:2410.02757. — [arXiv:2410.02757](https://arxiv.org/abs/2410.02757) [find](https://scholar.google.com/scholar?q=Yuqing+Wang%2C+Tianwei+Xiong%2C+Daquan+Zhou%2C+Zhijie+Lin%2C+Yang+Zhao%2C+Bingyi+Kang%2C+Jiashi+Feng%2C+and+Xihui+Liu%2E+2024c%2E+Loong%3A+Generating+minute%2Dlevel+long+videos+with+autoregressive+language+models%2E+arXiv+pr)
31. Zhou Wang, Alan C. Bovik, Hamid R. Sheikh, and Eero P. Simoncelli. 2004. Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Process., 13(4):600–612. [find](https://scholar.google.com/scholar?q=Zhou+Wang%2C+Alan+C%2E+Bovik%2C+Hamid+R%2E+Sheikh%2C+and+Eero+P%2E+Simoncelli%2E+2004%2E+Image+quality+assessment%3A+from+error+visibility+to+structural+similarity%2E+IEEE+Trans%2E+Image+Process%2E%2C+13%284%29%3A600%E2%80%93612%2E)
32. Wei Xiong, Wenhan Luo, Lin Ma, Wei Liu, and Jiebo Luo. 2018. Learning to generate time-lapse videos using multi-stage dynamic generative adversarial networks. In 2018 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2018, Salt Lake City, UT, USA, June 18-22, 2018, pages 2364–2373. Co… [find](https://scholar.google.com/scholar?q=Wei+Xiong%2C+Wenhan+Luo%2C+Lin+Ma%2C+Wei+Liu%2C+and+Jiebo+Luo%2E+2018%2E+Learning+to+generate+time%2Dlapse+videos+using+multi%2Dstage+dynamic+generative+adversarial+networks%2E+In+2018+IEEE+Conference+on+Computer+Visio)
33. Wilson Yan, Yunzhi Zhang, Pieter Abbeel, and Aravind Srinivas. 2021. Videogpt: Video generation using vq-vae and transformers. arXiv preprint arXiv:2104.10157. — [arXiv:2104.10157](https://arxiv.org/abs/2104.10157) [find](https://scholar.google.com/scholar?q=Wilson+Yan%2C+Yunzhi+Zhang%2C+Pieter+Abbeel%2C+and+Aravind+Srinivas%2E+2021%2E+Videogpt%3A+Video+generation+using+vq%2Dvae+and+transformers%2E+arXiv+preprint+arXiv%3A2104%2E10157%2E+%E2%80%94+%5BarXiv%3A2104%2E10157%5D%28https%3A%2F%2Farxiv%2Eorg%2Fa)
34. Hongji Yang, Wencheng Han, Yucheng Zhou, and Jianbing Shen. 2025a. Dc-controlnet: Decoupling inter-and intra-element conditions in image generation with diffusion models. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 19065–19074. [find](https://scholar.google.com/scholar?q=Hongji+Yang%2C+Wencheng+Han%2C+Yucheng+Zhou%2C+and+Jianbing+Shen%2E+2025a%2E+Dc%2Dcontrolnet%3A+Decoupling+inter%2Dand+intra%2Delement+conditions+in+image+generation+with+diffusion+models%2E+In+Proceedings+of+the+IEEE%2FCV)
35. Hongji Yang, Yucheng Zhou, Wencheng Han, and Jianbing Shen. 2025b. Self-rewarding large vision-language models for optimizing prompts in text-to-image generation. In Findings of the Association for Computational Linguistics: ACL 2025, pages 7332– 7349. [find](https://scholar.google.com/scholar?q=Hongji+Yang%2C+Yucheng+Zhou%2C+Wencheng+Han%2C+and+Jianbing+Shen%2E+2025b%2E+Self%2Drewarding+large+vision%2Dlanguage+models+for+optimizing+prompts+in+text%2Dto%2Dimage+generation%2E+In+Findings+of+the+Association+for+Co)
36. Hongji Yang, Yucheng Zhou, Wencheng Han, Runzhou Tao, Zhongying Qiu, Jianfei Yang, and Jianbing Shen. 2025c. Hicogen: Hierarchical compositional text-to-image generation in diffusion models via reinforcement learning. arXiv preprint arXiv:2511.19965. — [arXiv:2511.19965](https://arxiv.org/abs/2511.19965) [find](https://scholar.google.com/scholar?q=Hongji+Yang%2C+Yucheng+Zhou%2C+Wencheng+Han%2C+Runzhou+Tao%2C+Zhongying+Qiu%2C+Jianfei+Yang%2C+and+Jianbing+Shen%2E+2025c%2E+Hicogen%3A+Hierarchical+compositional+text%2Dto%2Dimage+generation+in+diffusion+models+via+reinfo)
37. Sihyun Yu, Kihyuk Sohn, Subin Kim, and Jinwoo Shin. 2023. Video probabilistic diffusion models in projected latent space. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2023, Vancouver, BC, Canada, June 17-24, 2023, pages 18456–18466. IEEE. [find](https://scholar.google.com/scholar?q=Sihyun+Yu%2C+Kihyuk+Sohn%2C+Subin+Kim%2C+and+Jinwoo+Shin%2E+2023%2E+Video+probabilistic+diffusion+models+in+projected+latent+space%2E+In+IEEE%2FCVF+Conference+on+Computer+Vision+and+Pattern+Recognition%2C+CVPR+2023%2C+)
38. Xiang Wang, Shiwei Zhang, Han Zhang, Yu Liu, Yingya Zhang, Changxin Gao, and Nong Sang. 2023c. Vide-olcm: Video latent consistency model. CoRR, abs/2312.09109. — [arXiv:2312.09109](https://arxiv.org/abs/2312.09109) [find](https://scholar.google.com/scholar?q=Xiang+Wang%2C+Shiwei+Zhang%2C+Han+Zhang%2C+Yu+Liu%2C+Yingya+Zhang%2C+Changxin+Gao%2C+and+Nong+Sang%2E+2023c%2E+Vide%2Dolcm%3A+Video+latent+consistency+model%2E+CoRR%2C+abs%2F2312%2E09109%2E+%E2%80%94+%5BarXiv%3A2312%2E09109%5D%28https%3A%2F%2Farxiv%2Eorg%2Fa)
39. Sihyun Yu, Jihoon Tack, Sangwoo Mo, Hyunsu Kim, Junho Kim, Jung-Woo Ha, and Jinwoo Shin. 2022. Generating videos with dynamics-aware implicit generative adversarial networks. In The Tenth International Conference on Learning Representations, ICLR 2022, Virtual Event, April 25-29, 2022. Open-Review.n… [find](https://scholar.google.com/scholar?q=Sihyun+Yu%2C+Jihoon+Tack%2C+Sangwoo+Mo%2C+Hyunsu+Kim%2C+Junho+Kim%2C+Jung%2DWoo+Ha%2C+and+Jinwoo+Shin%2E+2022%2E+Generating+videos+with+dynamics%2Daware+implicit+generative+adversarial+networks%2E+In+The+Tenth+Internationa)
40. Xinlong Wang, Xiaosong Zhang, Zhengxiong Luo, Quan Sun, Yufeng Cui, Jinsheng Wang, Fan Zhang, Yueze Wang, Zhen Li, Qiying Yu, Yingli Zhao, Yulong Ao, Xuebin Min, Tao Li, Boya Wu, Bo Zhao, Bowen Zhang, Liangdong Wang, Guang Liu, and 6 others. 2024b. Emu3: Next-token prediction is all you need. CoRR,… — [arXiv:2409.18869](https://arxiv.org/abs/2409.18869) [find](https://scholar.google.com/scholar?q=Xinlong+Wang%2C+Xiaosong+Zhang%2C+Zhengxiong+Luo%2C+Quan+Sun%2C+Yufeng+Cui%2C+Jinsheng+Wang%2C+Fan+Zhang%2C+Yueze+Wang%2C+Zhen+Li%2C+Qiying+Yu%2C+Yingli+Zhao%2C+Yulong+Ao%2C+Xuebin+Min%2C+Tao+Li%2C+Boya+Wu%2C+Bo+Zhao%2C+Bowen+Zhang%2C)
41. Hangjie Yuan, Shiwei Zhang, Xiang Wang, Yujie Wei, Tao Feng, Yining Pan, Yingya Zhang, Ziwei Liu, Samuel Albanie, and Dong Ni. 2024. Instructvideo: Instructing video diffusion models with human feedback. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2024, Seattle, WA, USA,… [find](https://scholar.google.com/scholar?q=Hangjie+Yuan%2C+Shiwei+Zhang%2C+Xiang+Wang%2C+Yujie+Wei%2C+Tao+Feng%2C+Yining+Pan%2C+Yingya+Zhang%2C+Ziwei+Liu%2C+Samuel+Albanie%2C+and+Dong+Ni%2E+2024%2E+Instructvideo%3A+Instructing+video+diffusion+models+with+human+feedba)
42. David Junhao Zhang, Jay Zhangjie Wu, Jia-Wei Liu, Rui Zhao, Lingmin Ran, Yuchao Gu, Difei Gao, and Mike Zheng Shou. 2023. Show-1: Marrying pixel and latent diffusion models for text-to-video generation. CoRR, abs/2309.15818. [find](https://scholar.google.com/scholar?q=David+Junhao+Zhang%2C+Jay+Zhangjie+Wu%2C+Jia%2DWei+Liu%2C+Rui+Zhao%2C+Lingmin+Ran%2C+Yuchao+Gu%2C+Difei+Gao%2C+and+Mike+Zheng+Shou%2E+2023%2E+Show%2D1%3A+Marrying+pixel+and+latent+diffusion+models+for+text%2Dto%2Dvideo+generatio)
43. Zangwei Zheng, Xiangyu Peng, Tianji Yang, Chenhui Shen, Shenggui Li, Hongxin Liu, Yukun Zhou, Tianyi Li, and Yang You. 2024. Open-sora: Democratizing efficient video production for all. arXiv preprint arXiv:2412.20404. — [arXiv:2412.20404](https://arxiv.org/abs/2412.20404) [find](https://scholar.google.com/scholar?q=Zangwei+Zheng%2C+Xiangyu+Peng%2C+Tianji+Yang%2C+Chenhui+Shen%2C+Shenggui+Li%2C+Hongxin+Liu%2C+Yukun+Zhou%2C+Tianyi+Li%2C+and+Yang+You%2E+2024%2E+Open%2Dsora%3A+Democratizing+efficient+video+production+for+all%2E+arXiv+preprint)
44. Yucheng Zhou, Xiubo Geng, Tao Shen, Chongyang Tao, Guodong Long, Jian-Guang Lou, and Jianbing Shen. 2023. Thread of thought unraveling chaotic contexts. arXiv preprint arXiv:2311.08734. — [arXiv:2311.08734](https://arxiv.org/abs/2311.08734) [find](https://scholar.google.com/scholar?q=Yucheng+Zhou%2C+Xiubo+Geng%2C+Tao+Shen%2C+Chongyang+Tao%2C+Guodong+Long%2C+Jian%2DGuang+Lou%2C+and+Jianbing+Shen%2E+2023%2E+Thread+of+thought+unraveling+chaotic+contexts%2E+arXiv+preprint+arXiv%3A2311%2E08734%2E+%E2%80%94+%5BarXiv%3A2311%2E)
45. Yucheng Zhou, Hao Li, and Jianbing Shen. 2026. Condition errors refinement in autoregressive image generation with diffusion loss. In The Fourteenth International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Yucheng+Zhou%2C+Hao+Li%2C+and+Jianbing+Shen%2E+2026%2E+Condition+errors+refinement+in+autoregressive+image+generation+with+diffusion+loss%2E+In+The+Fourteenth+International+Conference+on+Learning+Representation)
46. Yucheng Zhou, Xiang Li, Qianning Wang, and Jianbing Shen. 2024a. Visual in-context learning for large vision-language models. In Findings of the Association for Computational Linguistics: ACL 2024, pages 15890–15902. [find](https://scholar.google.com/scholar?q=Yucheng+Zhou%2C+Xiang+Li%2C+Qianning+Wang%2C+and+Jianbing+Shen%2E+2024a%2E+Visual+in%2Dcontext+learning+for+large+vision%2Dlanguage+models%2E+In+Findings+of+the+Association+for+Computational+Linguistics%3A+ACL+2024%2C+pa)
47. Yucheng Zhou, Jianbing Shen, and Yu Cheng. 2025. Weak to strong generalization for large language models with multi-capabilities. In The Thirteenth International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Yucheng+Zhou%2C+Jianbing+Shen%2C+and+Yu+Cheng%2E+2025%2E+Weak+to+strong+generalization+for+large+language+models+with+multi%2Dcapabilities%2E+In+The+Thirteenth+International+Conference+on+Learning+Representations)
48. Yucheng Zhou, Jihai Zhang, Guanjie Chen, Jianbing Shen, and Yu Cheng. 2024b. Less is more: Vision representation compression for efficient video generation with large language models. [find](https://scholar.google.com/scholar?q=Yucheng+Zhou%2C+Jihai+Zhang%2C+Guanjie+Chen%2C+Jianbing+Shen%2C+and+Yu+Cheng%2E+2024b%2E+Less+is+more%3A+Vision+representation+compression+for+efficient+video+generation+with+large+language+models%2E)

*Updated 2 November 2026*