---
title: "Too Many Frames, Not All Useful — paper by Park et al. (2026)"
description: "The authors propose LVNet, a modular framework that needs no video-level training. 68.2% accuracy. 66 references, cited by 9 papers here."
authors: ["Jongwoo Park", "Kanchana Ranasinghe", "Kumara Kahatapitiya", "Wonjeong Ryu", "Donghyun Kim", "Michael S Ryoo"]
year: 2026
venue: "Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers)"
source: "https://aclanthology.org/2026.eacl-long.164/"
doi: "10.18653/v1/2026.eacl-long.164"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/park-2026-too-many-frames-not-all/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/park-2026-too-many-frames-not-all/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/) / Too Many Frames, Not All Useful: Efficient Strategies for Long-Form Video QA

# Too Many Frames, Not All Useful: Efficient Strategies for Long-Form Video QA

**Authors:** Jongwoo Park, Kanchana Ranasinghe, Kumara Kahatapitiya, Wonjeong Ryu, [Donghyun Kim](https://md.research.elamur.ai/free/en/authors/donghyun-kim/) and [Michael S Ryoo](https://md.research.elamur.ai/free/en/authors/michael-s-ryoo/) · **Year:** 2026 · **Venue:** [Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers)](https://md.research.elamur.ai/free/en/venues/eacl/) ([EACL 2026](https://md.research.elamur.ai/free/en/venues/eacl/2026/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2026.eacl-long.164/) · [PDF](https://aclanthology.org/2026.eacl-long.164.pdf) · [DOI 10.18653/v1/2026.eacl-long.164](https://doi.org/10.18653/v1/2026.eacl-long.164) · [OpenAlex](https://openalex.org/W7140114028)

**Citations:** 4 citations (OpenAlex, 2026-10-14) · [9](https://md.research.elamur.ai/free/en/papers/park-2026-too-many-frames-not-all/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
Long videos contain a great deal of redundant content, and answering a question about them usually depends on only a few moments. Many LLM-based pipelines for long-form video question answering (LVQA) still caption large numbers of uniformly sampled frames, which is costly and mostly wasted effort. The authors propose LVNet, a modular framework that needs no video-level training. Its core is a Hierarchical Keyframe Selector (HKS), which applies cheap processing to many frames and more expensive models to progressively smaller subsets:

- **Temporal Scene Clustering:** groups frames into scenes using ResNet-18 features.
- **Coarse Keyframe Detector:** matches frames against keywords that an LLM extracts from the question.
- **Fine Keyframe Detector:** refines the selection by prompting a VLM with templated frames.

A VLM then captions the remaining frames, and an LLM answers the question from those captions. LVNet reports state-of-the-art results among comparably configured systems on EgoSchema, NExT-QA, IntentQA and VideoMME. It also cuts LLM cost by more than ten times compared with prompting GPT-4o directly. This shows that careful frame selection can substitute for brute-force processing of every frame.

## Key points
- HKS reduces input of up to 1800 frames (about an hour of video) to a small set of keyframes, filtering out over 98% of frames. In a typical run, 900 frames become roughly 390 clustered frames, then 32 coarse keyframes, then 12 fine keyframes.
- On a 500-video EgoSchema subset with GPT-4o, LVNet reaches 68.2% accuracy using 12 captions. Under the same setup, VideoAgent and VideoTree reach 63.6%.
- In the ablation, accuracy rises as each HKS module is added: uniform sampling gives 62.6%, adding TSC gives 64.5%, adding CKD gives 65.8%, and adding FKD gives 68.2%.
- On VideoMME-Long, raising the frame budget from 12 to 24 improves accuracy from 50.1% to 53.9%. Swapping in stronger LLMs without retraining also helps: GPT-3.5 scores 61.0%, GPT-4 scores 65.4%, and GPT-4o scores 68.2%.
- The code is available at github.com/jongwoopark7978/LVNet.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Korea Advanced Institute of Science and Technology](https://md.research.elamur.ai/free/en/universities/korea-advanced-institute-of-science-and-technology/) — University, Daejeon, South Korea · 51 works · 3,167 citations
- [Stony Brook University](https://md.research.elamur.ai/free/en/universities/stony-brook-university/) — University, Stony Brook, United States · 43 works · 2,571 citations
- [Korea University](https://md.research.elamur.ai/free/en/universities/korea-university/) — University, Seoul, South Korea · 26 works · 9,490 citations

📚 Shelves (5): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works · [👁️ Computer Vision and Pattern Recognition](https://md.research.elamur.ai/free/en/shelves/computer-science/computer-vision-and-pattern-recognition/) · 2,869 works

🧭 Topics (1): [❓ Question answering](https://md.research.elamur.ai/free/en/topics/question-answering/) · 441 works

## Cite

Cite as: Jongwoo Park, Kanchana Ranasinghe, Kumara Kahatapitiya, Wonjeong Ryu, Donghyun Kim and Michael S Ryoo. 2026. Too Many Frames, Not All Useful: Efficient Strategies for Long-Form Video QA. Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{park2026too,
  title = {{Too Many Frames, Not All Useful: Efficient Strategies for Long-Form Video QA}},
  author = {Jongwoo Park, Kanchana Ranasinghe, Kumara Kahatapitiya, Wonjeong Ryu, Donghyun Kim and Michael S Ryoo},
  booktitle = {Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2026},
  doi = {10.18653/v1/2026.eacl-long.164},
  url = {https://doi.org/10.18653/v1/2026.eacl-long.164}
}
```

- [← previous TigerCoder: A Novel Suite of LLMs for Code Generation in Bangla](https://md.research.elamur.ai/free/en/papers/raihan-2026-tigercoder-novel-suite-llms-code/)
- [next → CriticLean: Critic-Guided Reinforcement Learning for Mathematical Formalization](https://md.research.elamur.ai/free/en/papers/peng-2026-criticlean-critic-guided-reinforcement-learning/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/)

## References

66 references, 17 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Jake K. Aggarwal and Michael S. Ryoo. 2011. Human activity analysis. ACM Computing Surveys (CSUR), 43:1 – 43. [find](https://scholar.google.com/scholar?q=Jake+K%2E+Aggarwal+and+Michael+S%2E+Ryoo%2E+2011%2E+Human+activity+analysis%2E+ACM+Computing+Surveys+%28CSUR%29%2C+43%3A1+%E2%80%93+43%2E)
2. Aishwarya Agrawal, Jiasen Lu, Stanislaw Antol, Margaret Mitchell, C. Lawrence Zitnick, Devi Parikh, and Dhruv Batra. 2015. Vqa: Visual question answering. International Journal of Computer Vision, 123:4 – 31. [find](https://scholar.google.com/scholar?q=Aishwarya+Agrawal%2C+Jiasen+Lu%2C+Stanislaw+Antol%2C+Margaret+Mitchell%2C+C%2E+Lawrence+Zitnick%2C+Devi+Parikh%2C+and+Dhruv+Batra%2E+2015%2E+Vqa%3A+Visual+question+answering%2E+International+Journal+of+Computer+Vision%2C+123)
3. James F Allen and George Ferguson. 1994. Actions and events in interval temporal logic. Journal of logic and computation, 4(5):531–579. [find](https://scholar.google.com/scholar?q=James+F+Allen+and+George+Ferguson%2E+1994%2E+Actions+and+events+in+interval+temporal+logic%2E+Journal+of+logic+and+computation%2C+4%285%29%3A531%E2%80%93579%2E)
4. Jinze Bai, Shuai Bai, Shusheng Yang, Shijie Wang, Sinan Tan, Peng Wang, Junyang Lin, Chang Zhou, and Jingren Zhou. 2023. [Qwen-vl: A versatile vision-language model for understanding, localization, text reading, and beyond](https://md.research.elamur.ai/free/en/papers/bai-2023-qwen-vl-versatile-vision-language/). Preprint, arXiv:2308.12966.
5. S. Buch, Cristobal Eyzaguirre, Adrien Gaidon, Jiajun Wu, Li Fei-Fei, and Juan Carlos Niebles. 2022. Revisiting the “video” in video-language understanding. 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 2907–2917. [find](https://scholar.google.com/scholar?q=S%2E+Buch%2C+Cristobal+Eyzaguirre%2C+Adrien+Gaidon%2C+Jiajun+Wu%2C+Li+Fei%2DFei%2C+and+Juan+Carlos+Niebles%2E+2022%2E+Revisiting+the+%E2%80%9Cvideo%E2%80%9D+in+video%2Dlanguage+understanding%2E+2022+IEEE%2FCVF+Conference+on+Computer+Vision+)
6. Yukang Chen, Fuzhao Xue, Dacheng Li, Qinghao Hu, Ligeng Zhu, Xiuyu Li, Yunhao Fang, Haotian Tang, Shang Yang, Zhijian Liu, Ethan He, Hongxu Yin, Pavlo Molchanov, Jan Kautz, Linxi Fan, Yuke Zhu, Yao Lu, and Song Han. 2024a. Longvila: Scaling long-context visual language models for long videos. Prepri… [find](https://scholar.google.com/scholar?q=Yukang+Chen%2C+Fuzhao+Xue%2C+Dacheng+Li%2C+Qinghao+Hu%2C+Ligeng+Zhu%2C+Xiuyu+Li%2C+Yunhao+Fang%2C+Haotian+Tang%2C+Shang+Yang%2C+Zhijian+Liu%2C+Ethan+He%2C+Hongxu+Yin%2C+Pavlo+Molchanov%2C+Jan+Kautz%2C+Linxi+Fan%2C+Yuke+Zhu%2C+Yao+Lu)
7. Zhe Chen, Weiyun Wang, Yue Cao, Yangzhou Liu, Zhangwei Gao, Erfei Cui, Jinguo Zhu, Shenglong Ye, Hao Tian, Zhaoyang Liu, et al. 2024b. [Expanding performance boundaries of open-source multimodal models with model, data, and test-time scaling](https://md.research.elamur.ai/free/en/papers/chen-2024-expanding-performance-boundaries-open/). arXiv preprint arXiv:2412.05271.
8. Zhe Chen, Weiyun Wang, Hao Tian, Shenglong Ye, Zhangwei Gao, Erfei Cui, Wenwen Tong, Kongzhi Hu, Jiapeng Luo, Zheng Ma, et al. 2024c. [How far are we to gpt-4v? closing the gap to commercial multimodal models with open-source suites](https://md.research.elamur.ai/free/en/papers/chen-2024-far-gpt-4v-closing/). Science China Information Sciences, 67(12):220101.
9. Zesen Cheng, Sicong Leng, Hang Zhang, Yifei Xin, Xin Li, Guanzheng Chen, Yongxin Zhu, Wenqi Zhang, Ziyang Luo, Deli Zhao, et al. 2024. Videollama 2: Advancing spatial-temporal modeling and audio understanding in video-llms. arXiv preprint arXiv:2406.07476. [find](https://scholar.google.com/scholar?q=Zesen+Cheng%2C+Sicong+Leng%2C+Hang+Zhang%2C+Yifei+Xin%2C+Xin+Li%2C+Guanzheng+Chen%2C+Yongxin+Zhu%2C+Wenqi+Zhang%2C+Ziyang+Luo%2C+Deli+Zhao%2C+et+al%2E+2024%2E+Videollama+2%3A+Advancing+spatial%2Dtemporal+modeling+and+audio+under)
10. Rohan Choudhury, Koichiro Niinuma, Kris M Kitani, and László A Jeni. 2023. Zero-shot video question answering with procedural programs. arXiv preprint arXiv:2312.00937. — [arXiv:2312.00937](https://arxiv.org/abs/2312.00937) [find](https://scholar.google.com/scholar?q=Rohan+Choudhury%2C+Koichiro+Niinuma%2C+Kris+M+Kitani%2C+and+L%C3%A1szl%C3%B3+A+Jeni%2E+2023%2E+Zero%2Dshot+video+question+answering+with+procedural+programs%2E+arXiv+preprint+arXiv%3A2312%2E00937%2E+%E2%80%94+%5BarXiv%3A2312%2E00937%5D%28https%3A%2F%2Far)
11. Wenliang Dai, Junnan Li, Dongxu Li, Anthony Meng Huat Tiong, Junqi Zhao, Weisheng Wang, Boyang Li, Pascale Fung, and Steven Hoi. 2023. [Instructblip: Towards general-purpose vision-language models with instruction tuning](https://md.research.elamur.ai/free/en/papers/dai-2023-instructblip-towards-general-purpose-vision/). arXiv preprint arXiv:2305.06500.
12. James Davis and Aaron Bobick. 1997. The representation and recognition of action using temporal templates. In Proceedings of the IEEE International Conference on Computer Vision, pages 2736–2744. [find](https://scholar.google.com/scholar?q=James+Davis+and+Aaron+Bobick%2E+1997%2E+The+representation+and+recognition+of+action+using+temporal+templates%2E+In+Proceedings+of+the+IEEE+International+Conference+on+Computer+Vision%2C+pages+2736%E2%80%932744%2E)
13. Yue Fan, Xiaojian Ma, Rujie Wu, Yuntao Du, Jiaqi Li, Zhi Gao, and Qing Li. 2024. Videoagent: A memory-augmented multimodal agent for video understanding. arXiv preprint arXiv:2403.11481. — [arXiv:2403.11481](https://arxiv.org/abs/2403.11481) [find](https://scholar.google.com/scholar?q=Yue+Fan%2C+Xiaojian+Ma%2C+Rujie+Wu%2C+Yuntao+Du%2C+Jiaqi+Li%2C+Zhi+Gao%2C+and+Qing+Li%2E+2024%2E+Videoagent%3A+A+memory%2Daugmented+multimodal+agent+for+video+understanding%2E+arXiv+preprint+arXiv%3A2403%2E11481%2E+%E2%80%94+%5BarXiv%3A2403)
14. Chaoyou Fu, Yuhan Dai, Yongdong Luo, Lei Li, Shuhuai Ren, Renrui Zhang, Zihan Wang, Chenyu Zhou, Yunhang Shen, Mengdan Zhang, et al. 2024. [find](https://scholar.google.com/scholar?q=Chaoyou+Fu%2C+Yuhan+Dai%2C+Yongdong+Luo%2C+Lei+Li%2C+Shuhuai+Ren%2C+Renrui+Zhang%2C+Zihan+Wang%2C+Chenyu+Zhou%2C+Yunhang+Shen%2C+Mengdan+Zhang%2C+et+al%2E+2024%2E)
15. [Video-mme: The first-ever comprehensive evaluation benchmark of multi-modal llms in video analysis](https://md.research.elamur.ai/free/en/papers/fu-2024-video-mme-first-ever-comprehensive/). arXiv preprint arXiv:2405.21075.
16. Tsu-Jui Fu, Linjie Li, Zhe Gan, Kevin Lin, William Yang Wang, Lijuan Wang, and Zicheng Liu. 2023. An empirical study of end-to-end video-language transformers with masked visual modeling. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 22898–22909. [find](https://scholar.google.com/scholar?q=Tsu%2DJui+Fu%2C+Linjie+Li%2C+Zhe+Gan%2C+Kevin+Lin%2C+William+Yang+Wang%2C+Lijuan+Wang%2C+and+Zicheng+Liu%2E+2023%2E+An+empirical+study+of+end%2Dto%2Dend+video%2Dlanguage+transformers+with+masked+visual+modeling%2E+In+Proceedin)
17. Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016a. [Deep residual learning for image recognition](https://md.research.elamur.ai/free/en/papers/he-2015-deep-residual-learning-image-recognition/). In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770– 778.
18. Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016b. [Deep residual learning for image recognition](https://md.research.elamur.ai/free/en/papers/he-2015-deep-residual-learning-image-recognition/). In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770– 778.
19. Somboon Hongeng, Ram Nevatia, and Francois Bre-mond. 2004. Video-based event recognition: activity representation and probabilistic recognition methods. Computer Vision and Image Understanding, 96(2):129–162. [find](https://scholar.google.com/scholar?q=Somboon+Hongeng%2C+Ram+Nevatia%2C+and+Francois+Bre%2Dmond%2E+2004%2E+Video%2Dbased+event+recognition%3A+activity+representation+and+probabilistic+recognition+methods%2E+Computer+Vision+and+Image+Understanding%2C+96%282%29%3A)
20. Pedram Hosseini, David A. Broniatowski, and Mona Diab. 2022. Knowledge-augmented language models for cause-effect relation classification. In Proceedings of the First Workshop on Commonsense Representation and Reasoning (CSRR 2022), pages 43–48, Dublin, Ireland. Association for Computational Linguis… [find](https://scholar.google.com/scholar?q=Pedram+Hosseini%2C+David+A%2E+Broniatowski%2C+and+Mona+Diab%2E+2022%2E+Knowledge%2Daugmented+language+models+for+cause%2Deffect+relation+classification%2E+In+Proceedings+of+the+First+Workshop+on+Commonsense+Represent)
21. Yuri A. Ivanov and Aaron F. Bobick. 2000. Recognition of visual activities and interactions by stochastic parsing. IEEE Transactions on Pattern Analysis and Machine Intelligence, 22(8):852–872. [find](https://scholar.google.com/scholar?q=Yuri+A%2E+Ivanov+and+Aaron+F%2E+Bobick%2E+2000%2E+Recognition+of+visual+activities+and+interactions+by+stochastic+parsing%2E+IEEE+Transactions+on+Pattern+Analysis+and+Machine+Intelligence%2C+22%288%29%3A852%E2%80%93872%2E)
22. Kumara Kahatapitiya, Kanchana Ranasinghe, Jongwoo Park, and Michael S Ryoo. 2024. Language repository for long video understanding. [find](https://scholar.google.com/scholar?q=Kumara+Kahatapitiya%2C+Kanchana+Ranasinghe%2C+Jongwoo+Park%2C+and+Michael+S+Ryoo%2E+2024%2E+Language+repository+for+long+video+understanding%2E)
23. Wonkyun Kim, Changin Choi, Wonseok Lee, and Wonjong Rhee. 2024. An image grid can be worth a video: Zero-shot video question answering using a vlm. arXiv preprint arXiv:2403.18406. — [arXiv:2403.18406](https://arxiv.org/abs/2403.18406) [find](https://scholar.google.com/scholar?q=Wonkyun+Kim%2C+Changin+Choi%2C+Wonseok+Lee%2C+and+Wonjong+Rhee%2E+2024%2E+An+image+grid+can+be+worth+a+video%3A+Zero%2Dshot+video+question+answering+using+a+vlm%2E+arXiv+preprint+arXiv%3A2403%2E18406%2E+%E2%80%94+%5BarXiv%3A2403%2E18406)
24. Jie Lei, Licheng Yu, Mohit Bansal, and Tamara Berg. 2018. [TVQA: Localized, compositional video question answering](https://md.research.elamur.ai/free/en/papers/lei-2018-tvqa-localized-compositional-video-question/). In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing (EMNLP).
25. Jie Lei, Licheng Yu, Tamara Berg, and Mohit Bansal. 2020. TVQA+: Spatio-temporal grounding for video question answering. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 8211–8225, Online. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Jie+Lei%2C+Licheng+Yu%2C+Tamara+Berg%2C+and+Mohit+Bansal%2E+2020%2E+TVQA%2B%3A+Spatio%2Dtemporal+grounding+for+video+question+answering%2E+In+Proceedings+of+the+58th+Annual+Meeting+of+the+Association+for+Computational+)
26. Bo Li, Yuanhan Zhang, Dong Guo, Renrui Zhang, Feng Li, Hao Zhang, Kaichen Zhang, Peiyuan Zhang, Yanwei Li, Ziwei Liu, and Chunyuan Li. 2024. [Llava-onevision: Easy visual task transfer](https://md.research.elamur.ai/free/en/papers/li-2024-llava-onevision-easy-visual-task/). Preprint, arXiv:2408.03326.
27. Michael S. Ryoo and Jake K. Aggarwal. 2006. Recognition of composite human activities through context-free grammar based representation. 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’06), 2:1709–1718. [find](https://scholar.google.com/scholar?q=Michael+S%2E+Ryoo+and+Jake+K%2E+Aggarwal%2E+2006%2E+Recognition+of+composite+human+activities+through+context%2Dfree+grammar+based+representation%2E+2006+IEEE+Computer+Society+Conference+on+Computer+Vision+and+Pa)
28. Jiangtong Li, Li Niu, and Liqing Zhang. 2022. From representation to reasoning: Towards both evidence and commonsense reasoning for video question-answering. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). [find](https://scholar.google.com/scholar?q=Jiangtong+Li%2C+Li+Niu%2C+and+Liqing+Zhang%2E+2022%2E+From+representation+to+reasoning%3A+Towards+both+evidence+and+commonsense+reasoning+for+video+question%2Danswering%2E+In+Proceedings+of+the+IEEE%2FCVF+Conference+)
29. Chuyi Shang, Amos You, Sanjay Subramanian, Trevor Darrell, and Roei Herzig. 2024. Traveler: A modular multi-lmm agent framework for video question-answering. arXiv preprint arXiv:2404.01476. [find](https://scholar.google.com/scholar?q=Chuyi+Shang%2C+Amos+You%2C+Sanjay+Subramanian%2C+Trevor+Darrell%2C+and+Roei+Herzig%2E+2024%2E+Traveler%3A+A+modular+multi%2Dlmm+agent+framework+for+video+question%2Danswering%2E+arXiv+preprint+arXiv%3A2404%2E01476%2E)
30. Jiapeng Li, Ping Wei, Wenjuan Han, and Lifeng Fan. 2023. Intentqa: Context-aware video intent reasoning. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 11963– 11974. [find](https://scholar.google.com/scholar?q=Jiapeng+Li%2C+Ping+Wei%2C+Wenjuan+Han%2C+and+Lifeng+Fan%2E+2023%2E+Intentqa%3A+Context%2Daware+video+intent+reasoning%2E+In+Proceedings+of+the+IEEE%2FCVF+International+Conference+on+Computer+Vision%2C+pages+11963%E2%80%93+11974%2E)
31. Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. 2023. [Visual instruction tuning](https://md.research.elamur.ai/free/en/papers/liu-2023-visual-instruction-tuning/). arXiv preprint arXiv:2304.08485.
32. Muhammad Maaz, Hanoona Abdul Rasheed, Salman H. Khan, and Fahad Shahbaz Khan. 2023. [Video-chatgpt: Towards detailed video understanding via large vision and language models](https://md.research.elamur.ai/free/en/papers/maaz-2023-video-chatgpt-towards-detailed-video/). In Annual Meeting of the Association for Computational Linguistics.
33. Karttikeya Mangalam, Raiymbek Akshulakov, and Jitendra Malik. 2023. Egoschema: A diagnostic benchmark for very long-form video language understanding. ArXiv, abs/2308.09126. [find](https://scholar.google.com/scholar?q=Karttikeya+Mangalam%2C+Raiymbek+Akshulakov%2C+and+Jitendra+Malik%2E+2023%2E+Egoschema%3A+A+diagnostic+benchmark+for+very+long%2Dform+video+language+understanding%2E+ArXiv%2C+abs%2F2308%2E09126%2E)
34. Juhong Min, Shyamal Buch, Arsha Nagrani, Minsu Cho, and Cordelia Schmid. 2024. Morevqa: Exploring modular reasoning models for video question answering. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 13235–13245. [find](https://scholar.google.com/scholar?q=Juhong+Min%2C+Shyamal+Buch%2C+Arsha+Nagrani%2C+Minsu+Cho%2C+and+Cordelia+Schmid%2E+2024%2E+Morevqa%3A+Exploring+modular+reasoning+models+for+video+question+answering%2E+In+Proceedings+of+the+IEEE%2FCVF+Conference+on+Co)
35. Liliane Momeni, Mathilde Caron, Arsha Nagrani, Andrew Zisserman, and Cordelia Schmid. 2023. Verbs in action: Improving verb understanding in video-language models. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 15579–15591. [find](https://scholar.google.com/scholar?q=Liliane+Momeni%2C+Mathilde+Caron%2C+Arsha+Nagrani%2C+Andrew+Zisserman%2C+and+Cordelia+Schmid%2E+2023%2E+Verbs+in+action%3A+Improving+verb+understanding+in+video%2Dlanguage+models%2E+In+Proceedings+of+the+IEEE%2FCVF+Inter)
36. Pinelopi Papalampidi, Skanda Koppula, Shreya Pathak, Justin Chiu, Joe Heyward, Viorica Patraucean, Jiajun Shen, Antoine Miech, Andrew Zisserman, and Aida Nematzdeh. 2023. A simple recipe for contrastively pre-training video-first encoders beyond 16 frames. arXiv preprint arXiv:2312.07395. [find](https://scholar.google.com/scholar?q=Pinelopi+Papalampidi%2C+Skanda+Koppula%2C+Shreya+Pathak%2C+Justin+Chiu%2C+Joe+Heyward%2C+Viorica+Patraucean%2C+Jiajun+Shen%2C+Antoine+Miech%2C+Andrew+Zisserman%2C+and+Aida+Nematzdeh%2E+2023%2E+A+simple+recipe+for+contrasti)
37. Yifan Shi, Yan Huang, David Minnen, Aaron Bobick, and Irfan Essa. 2004. Propagation networks for recognition of partially ordered sequential action. In CVPR. [find](https://scholar.google.com/scholar?q=Yifan+Shi%2C+Yan+Huang%2C+David+Minnen%2C+Aaron+Bobick%2C+and+Irfan+Essa%2E+2004%2E+Propagation+networks+for+recognition+of+partially+ordered+sequential+action%2E+In+CVPR%2E)
38. Makarand Tapaswi, Yukun Zhu, Rainer Stiefelhagen, Antonio Torralba, Raquel Urtasun, and Sanja Fidler. 2016. MovieQA: Understanding stories in movies through question-answering. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). [find](https://scholar.google.com/scholar?q=Makarand+Tapaswi%2C+Yukun+Zhu%2C+Rainer+Stiefelhagen%2C+Antonio+Torralba%2C+Raquel+Urtasun%2C+and+Sanja+Fidler%2E+2016%2E+MovieQA%3A+Understanding+stories+in+movies+through+question%2Danswering%2E+In+Proceedings+of+the+I)
39. Jiawei Wang, Liping Yuan, and Yuchen Zhang. 2024a. Tarsier: Recipes for training and evaluating large video description models. arXiv preprint arXiv:2407.00634. — [arXiv:2407.00634](https://arxiv.org/abs/2407.00634) [find](https://scholar.google.com/scholar?q=Jiawei+Wang%2C+Liping+Yuan%2C+and+Yuchen+Zhang%2E+2024a%2E+Tarsier%3A+Recipes+for+training+and+evaluating+large+video+description+models%2E+arXiv+preprint+arXiv%3A2407%2E00634%2E+%E2%80%94+%5BarXiv%3A2407%2E00634%5D%28https%3A%2F%2Farxiv%2Eorg%2F)
40. Peng Wang, Shuai Bai, Sinan Tan, Shijie Wang, Zhihao Fan, Jinze Bai, Ke-Yang Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Yang Fan, Kai Dang, Mengfei Du, Xuancheng Ren, Rui Men, Dayiheng Liu, Chang Zhou, Jingren Zhou, and Junyang Lin. 2024b. [Qwen2-vl: Enhancing vision-language model’s perception of th…](https://md.research.elamur.ai/free/en/papers/wang-2024-qwen2-vl-enhancing-vision/)
41. Peng Wang, Shuai Bai, Sinan Tan, Shijie Wang, Zhihao Fan, Jinze Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Yang Fan, Kai Dang, Mengfei Du, Xuancheng Ren, Rui Men, Dayiheng Liu, Chang Zhou, Jingren Zhou, and Junyang Lin. 2024c. [Qwen2-vl: Enhancing vision-language model’s perception of the…](https://md.research.elamur.ai/free/en/papers/wang-2024-qwen2-vl-enhancing-vision/)
42. Shijie Wang, Qi Zhao, Minh Quan Do, Nakul Agarwal, Kwonjoon Lee, and Chen Sun. 2023. Vamos: Versatile action models for video understanding. arXiv preprint arXiv:2311.13627. — [arXiv:2311.13627](https://arxiv.org/abs/2311.13627) [find](https://scholar.google.com/scholar?q=Shijie+Wang%2C+Qi+Zhao%2C+Minh+Quan+Do%2C+Nakul+Agarwal%2C+Kwonjoon+Lee%2C+and+Chen+Sun%2E+2023%2E+Vamos%3A+Versatile+action+models+for+video+understanding%2E+arXiv+preprint+arXiv%3A2311%2E13627%2E+%E2%80%94+%5BarXiv%3A2311%2E13627%5D%28https)
43. Xiaohan Wang, Yuhui Zhang, Orr Zohar, and Serena Yeung-Levy. 2024d. Videoagent: Long-form video understanding with large language model as agent. [find](https://scholar.google.com/scholar?q=Xiaohan+Wang%2C+Yuhui+Zhang%2C+Orr+Zohar%2C+and+Serena+Yeung%2DLevy%2E+2024d%2E+Videoagent%3A+Long%2Dform+video+understanding+with+large+language+model+as+agent%2E)
44. Kanchana Ranasinghe, Xiang Li, Kumara Kahatapitiya, and Michael Ryoo. 2024. Understanding long videos in one multimodal language model pass. [find](https://scholar.google.com/scholar?q=Kanchana+Ranasinghe%2C+Xiang+Li%2C+Kumara+Kahatapitiya%2C+and+Michael+Ryoo%2E+2024%2E+Understanding+long+videos+in+one+multimodal+language+model+pass%2E)
45. Yi Wang, Kunchang Li, Xinhao Li, Jiashuo Yu, Yinan He, Guo Chen, Baoqi Pei, Rongkun Zheng, Jilan Xu, Zun Wang, et al. 2024e. [Internvideo2: Scaling video foundation models for multimodal video understanding](https://md.research.elamur.ai/free/en/papers/wang-2024-internvideo2-scaling-foundation-models/). arXiv preprint arXiv:2403.15377.
46. Kanchana Ranasinghe, Brandon McKinzie, Sachin Ravi, Yinfei Yang, Alexander Toshev, and Jonathon Shlens. 2023. Perceptual grouping in contrastive vision-language models. In ICCV. [find](https://scholar.google.com/scholar?q=Kanchana+Ranasinghe%2C+Brandon+McKinzie%2C+Sachin+Ravi%2C+Yinfei+Yang%2C+Alexander+Toshev%2C+and+Jonathon+Shlens%2E+2023%2E+Perceptual+grouping+in+contrastive+vision%2Dlanguage+models%2E+In+ICCV%2E)
47. Ying Wang, Yanlai Yang, and Mengye Ren. 2024f. Lifelongmemory: Leveraging llms for answering queries in long-form egocentric videos. Preprint, arXiv:2312.05269. — [arXiv:2312.05269](https://arxiv.org/abs/2312.05269) [find](https://scholar.google.com/scholar?q=Ying+Wang%2C+Yanlai+Yang%2C+and+Mengye+Ren%2E+2024f%2E+Lifelongmemory%3A+Leveraging+llms+for+answering+queries+in+long%2Dform+egocentric+videos%2E+Preprint%2C+arXiv%3A2312%2E05269%2E+%E2%80%94+%5BarXiv%3A2312%2E05269%5D%28https%3A%2F%2Farxiv%2Eorg%2F)
48. Ziyang Wang, Shoubin Yu, Elias Stengel-Eskin, Jaehong Yoon, Feng Cheng, Gedas Bertasius, and Mohit Bansal. 2024g. Videotree: Adaptive tree-based video representation for llm reasoning on long videos. arXiv preprint arXiv:2405.19209. [find](https://scholar.google.com/scholar?q=Ziyang+Wang%2C+Shoubin+Yu%2C+Elias+Stengel%2DEskin%2C+Jaehong+Yoon%2C+Feng+Cheng%2C+Gedas+Bertasius%2C+and+Mohit+Bansal%2E+2024g%2E+Videotree%3A+Adaptive+tree%2Dbased+video+representation+for+llm+reasoning+on+long+videos%2E+)
49. Zhou Yu, Dejing Xu, Jun Yu, Ting Yu, Zhou Zhao, Yueting Zhuang, and Dacheng Tao. 2019b. [ActivityNet-QA: A dataset for understanding complex web videos via question answering](https://md.research.elamur.ai/free/en/papers/yu-2019-activitynet-qa-dataset-understanding-complex/). In Proceedings of the AAAI Conference on Artificial Intelligence (AAAI).
50. Junbin Xiao, Xindi Shang, Angela Yao, and Tat-Seng Chua. 2021. NExT-QA: Next phase of question-answering to explaining temporal actions. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). [find](https://scholar.google.com/scholar?q=Junbin+Xiao%2C+Xindi+Shang%2C+Angela+Yao%2C+and+Tat%2DSeng+Chua%2E+2021%2E+NExT%2DQA%3A+Next+phase+of+question%2Danswering+to+explaining+temporal+actions%2E+In+Proceedings+of+the+IEEE%2FCVF+Conference+on+Computer+Vision+an)
51. Kuo-Hao Zeng, Tseng-Hung Chen, Ching-Yao Chuang, Yuan-Hong Liao, Juan Carlos Niebles, and Min Sun. 2017. Leveraging video descriptions to learn video question answering. In AAAI Conference on Artificial Intelligence. [find](https://scholar.google.com/scholar?q=Kuo%2DHao+Zeng%2C+Tseng%2DHung+Chen%2C+Ching%2DYao+Chuang%2C+Yuan%2DHong+Liao%2C+Juan+Carlos+Niebles%2C+and+Min+Sun%2E+2017%2E+Leveraging+video+descriptions+to+learn+video+question+answering%2E+In+AAAI+Conference+on+Artifici)
52. Junbin Xiao, Angela Yao, Zhiyuan Liu, Yicong Li, Wei Ji, and Tat-Seng Chua. 2022a. Video as conditional graph hierarchy for multi-granular question answering. In Proceedings of the 36th AAAI Conference on Artificial Intelligence (AAAI), pages 2804–2812. [find](https://scholar.google.com/scholar?q=Junbin+Xiao%2C+Angela+Yao%2C+Zhiyuan+Liu%2C+Yicong+Li%2C+Wei+Ji%2C+and+Tat%2DSeng+Chua%2E+2022a%2E+Video+as+conditional+graph+hierarchy+for+multi%2Dgranular+question+answering%2E+In+Proceedings+of+the+36th+AAAI+Conferenc)
53. Xiangyu Zeng, Kunchang Li, Chenting Wang, Xinhao Li, Tianxiang Jiang, Ziang Yan, Songze Li, Yansong Shi, Zhengrong Yue, Yi Wang, et al. 2024. Timesuite: Improving mllms for long video understanding via grounded tuning. arXiv preprint arXiv:2410.19702. — [arXiv:2410.19702](https://arxiv.org/abs/2410.19702) [find](https://scholar.google.com/scholar?q=Xiangyu+Zeng%2C+Kunchang+Li%2C+Chenting+Wang%2C+Xinhao+Li%2C+Tianxiang+Jiang%2C+Ziang+Yan%2C+Songze+Li%2C+Yansong+Shi%2C+Zhengrong+Yue%2C+Yi+Wang%2C+et+al%2E+2024%2E+Timesuite%3A+Improving+mllms+for+long+video+understanding+vi)
54. Junbin Xiao, Pan Zhou, Tat-Seng Chua, and Shuicheng Yan. 2022b. Video graph transformer for video question answering. In European Conference on Computer Vision, pages 39–58. Springer. [find](https://scholar.google.com/scholar?q=Junbin+Xiao%2C+Pan+Zhou%2C+Tat%2DSeng+Chua%2C+and+Shuicheng+Yan%2E+2022b%2E+Video+graph+transformer+for+video+question+answering%2E+In+European+Conference+on+Computer+Vision%2C+pages+39%E2%80%9358%2E+Springer%2E)
55. Ce Zhang, Taixi Lu, Md Mohaiminul Islam, Ziyang Wang, Shoubin Yu, Mohit Bansal, and Gedas Bertasius. 2023. A simple llm framework for long-range video question-answering. arXiv preprint arXiv:2312.17235. [find](https://scholar.google.com/scholar?q=Ce+Zhang%2C+Taixi+Lu%2C+Md+Mohaiminul+Islam%2C+Ziyang+Wang%2C+Shoubin+Yu%2C+Mohit+Bansal%2C+and+Gedas+Bertasius%2E+2023%2E+A+simple+llm+framework+for+long%2Drange+video+question%2Danswering%2E+arXiv+preprint+arXiv%3A2312%2E172)
56. Dejing Xu, Zhou Zhao, Jun Xiao, Fei Wu, Hanwang Zhang, Xiangnan He, and Yueting Zhuang. 2017. Video question answering via gradually refined attention over appearance and motion. In ACM Multimedia. [find](https://scholar.google.com/scholar?q=Dejing+Xu%2C+Zhou+Zhao%2C+Jun+Xiao%2C+Fei+Wu%2C+Hanwang+Zhang%2C+Xiangnan+He%2C+and+Yueting+Zhuang%2E+2017%2E+Video+question+answering+via+gradually+refined+attention+over+appearance+and+motion%2E+In+ACM+Multimedia%2E)
57. Yuanhan Zhang, Jinming Wu, Wei Li, Bo Li, Zejun Ma, Ziwei Liu, and Chunyuan Li. 2024a. Video instruction tuning with synthetic data. Preprint, arXiv:2410.02713. [find](https://scholar.google.com/scholar?q=Yuanhan+Zhang%2C+Jinming+Wu%2C+Wei+Li%2C+Bo+Li%2C+Zejun+Ma%2C+Ziwei+Liu%2C+and+Chunyuan+Li%2E+2024a%2E+Video+instruction+tuning+with+synthetic+data%2E+Preprint%2C+arXiv%3A2410%2E02713%2E)
58. Antoine Yang, Antoine Miech, Josef Sivic, Ivan Laptev, and Cordelia Schmid. 2022. Zero-shot video question answering via frozen bidirectional language models. Advances in Neural Information Processing Systems, 35:124–141. [find](https://scholar.google.com/scholar?q=Antoine+Yang%2C+Antoine+Miech%2C+Josef+Sivic%2C+Ivan+Laptev%2C+and+Cordelia+Schmid%2E+2022%2E+Zero%2Dshot+video+question+answering+via+frozen+bidirectional+language+models%2E+Advances+in+Neural+Information+Processing)
59. Qinghao Ye, Haiyang Xu, Guohai Xu, Jiabo Ye, Ming Yan, Yiyang Zhou, Junyang Wang, Anwen Hu, Pengcheng Shi, Yaya Shi, et al. 2023. [mplug-owl: Modularization empowers large language models with multimodality](https://md.research.elamur.ai/free/en/papers/ye-2023-mplug-owl-modularization-empowers-large/). arXiv preprint arXiv:2304.14178.
60. Shoubin Yu, Jaemin Cho, Prateek Yadav, and Mohit Bansal. 2023. Self-chained image-language model for video localization and question answering. ArXiv, abs/2305.06988. [find](https://scholar.google.com/scholar?q=Shoubin+Yu%2C+Jaemin+Cho%2C+Prateek+Yadav%2C+and+Mohit+Bansal%2E+2023%2E+Self%2Dchained+image%2Dlanguage+model+for+video+localization+and+question+answering%2E+ArXiv%2C+abs%2F2305%2E06988%2E)
61. Yuanhan Zhang, Jinming Wu, Wei Li, Bo Li, Zejun Ma, Ziwei Liu, and Chunyuan Li. 2024b. Video instruction tuning with synthetic data. Preprint, arXiv:2410.02713. [find](https://scholar.google.com/scholar?q=Yuanhan+Zhang%2C+Jinming+Wu%2C+Wei+Li%2C+Bo+Li%2C+Zejun+Ma%2C+Ziwei+Liu%2C+and+Chunyuan+Li%2E+2024b%2E+Video+instruction+tuning+with+synthetic+data%2E+Preprint%2C+arXiv%3A2410%2E02713%2E)
62. Zhichen Zhao, Huimin Ma, and Shaodi You. 2017. Single image action recognition using semantic body part actions. In The IEEE International Conference on Computer Vision (ICCV). [find](https://scholar.google.com/scholar?q=Zhichen+Zhao%2C+Huimin+Ma%2C+and+Shaodi+You%2E+2017%2E+Single+image+action+recognition+using+semantic+body+part+actions%2E+In+The+IEEE+International+Conference+on+Computer+Vision+%28ICCV%29%2E)
63. Junjie Zhou, Yan Shu, Bo Zhao, Boya Wu, Shitao Xiao, Xi Yang, Yongping Xiong, Bo Zhang, Tiejun Huang, and Zheng Liu. 2024. Mlvu: A comprehensive benchmark for multi-task long video understanding. arXiv preprint arXiv:2406.04264. [find](https://scholar.google.com/scholar?q=Junjie+Zhou%2C+Yan+Shu%2C+Bo+Zhao%2C+Boya+Wu%2C+Shitao+Xiao%2C+Xi+Yang%2C+Yongping+Xiong%2C+Bo+Zhang%2C+Tiejun+Huang%2C+and+Zheng+Liu%2E+2024%2E+Mlvu%3A+A+comprehensive+benchmark+for+multi%2Dtask+long+video+understanding%2E+arXi)
64. Shoubin Yu, Jaemin Cho, Prateek Yadav, and Mohit Bansal. 2024a. Self-chained image-language model for video localization and question answering. Advances in Neural Information Processing Systems, 36. [find](https://scholar.google.com/scholar?q=Shoubin+Yu%2C+Jaemin+Cho%2C+Prateek+Yadav%2C+and+Mohit+Bansal%2E+2024a%2E+Self%2Dchained+image%2Dlanguage+model+for+video+localization+and+question+answering%2E+Advances+in+Neural+Information+Processing+Systems%2C+36%2E)
65. Sicheng Yu, Chengkai Jin, Huanyu Wang, Zhenghao Chen, Sheng Jin, Zhongrong Zuo, Xiaolei Xu, Zhenbang Sun, Bingni Zhang, Jiawei Wu, et al. 2024b. Frame-voyager: Learning to query frames for video large language models. arXiv preprint arXiv:2410.03226. — [arXiv:2410.03226](https://arxiv.org/abs/2410.03226) [find](https://scholar.google.com/scholar?q=Sicheng+Yu%2C+Chengkai+Jin%2C+Huanyu+Wang%2C+Zhenghao+Chen%2C+Sheng+Jin%2C+Zhongrong+Zuo%2C+Xiaolei+Xu%2C+Zhenbang+Sun%2C+Bingni+Zhang%2C+Jiawei+Wu%2C+et+al%2E+2024b%2E+Frame%2Dvoyager%3A+Learning+to+query+frames+for+video+large)
66. Zhou Yu, D. Xu, Jun Yu, Ting Yu, Zhou Zhao, Yueting Zhuang, and Dacheng Tao. 2019a. [Activitynet-qa: A dataset for understanding complex web videos via question answering](https://md.research.elamur.ai/free/en/papers/yu-2019-activitynet-qa-dataset-understanding-complex/). ArXiv, abs/1906.02467.

*Updated 31 October 2026*