---
title: "RATION — paper by Xu et al. (2026) — summary and ideas"
description: "Multimodal LLMs often fail on visually demanding questions because they spread visual attention badly. 80% accuracy. 45 references."
authors: ["Xingle Xu", "Fanheng Kong", "Dexian Cai", "Shi Feng", "Xiaocui Yang", "Daling Wang", "Yifei Zhang"]
year: 2026
venue: "Findings of the Association for Computational Linguistics: ACL 2026"
source: "https://aclanthology.org/2026.findings-acl.1238/"
doi: "10.18653/v1/2026.findings-acl.1238"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/xu-2026-ration-entropy-driven-task-adaptive/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/xu-2026-ration-entropy-driven-task-adaptive/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/46/) / RATION: Entropy-Driven Task-Adaptive Visual Attention Allocation Framework for Multimodal Reasoning

# RATION: Entropy-Driven Task-Adaptive Visual Attention Allocation Framework for Multimodal Reasoning

**Authors:** [Xingle Xu](https://md.research.elamur.ai/free/en/authors/xingle-xu/), [Fanheng Kong](https://md.research.elamur.ai/free/en/authors/fanheng-kong/), [Dexian Cai](https://md.research.elamur.ai/free/en/authors/dexian-cai/), [Shi Feng](https://md.research.elamur.ai/free/en/authors/shi-feng/), [Xiaocui Yang](https://md.research.elamur.ai/free/en/authors/xiaocui-yang/), [Daling Wang](https://md.research.elamur.ai/free/en/authors/daling-wang/) and [Yifei Zhang](https://md.research.elamur.ai/free/en/authors/yifei-zhang/) · **Year:** 2026 · **Venue:** [Findings of the Association for Computational Linguistics: ACL 2026](https://md.research.elamur.ai/free/en/venues/findings/) ([Findings of the Association for Computational Linguistics 2026](https://md.research.elamur.ai/free/en/venues/findings/2026/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2026.findings-acl.1238/) · [PDF](https://aclanthology.org/2026.findings-acl.1238.pdf) · [DOI 10.18653/v1/2026.findings-acl.1238](https://doi.org/10.18653/v1/2026.findings-acl.1238) · [OpenAlex](https://openalex.org/W7166820128)

**Citations:** 0 citations (OpenAlex, 2026-10-14)

## Summary
Multimodal LLMs often fail on visually demanding questions because they spread visual attention badly. In counting, attention can lock onto a few spots and miss objects. In OCR, it can drift over irrelevant regions instead of the text. The authors argue that earlier fixes, such as better alignment, chain-of-thought or extra inference compute, ignore how visual needs differ between tasks.

RATION is a training-free, inference-time framework that works in three stages:
- **Offline layer profiling:** the authors zero out visual key/value states from a given layer onward to find which decoder layers matter for each of four task types (recognition, counting, spatial, OCR).
- **Task routing:** for each input, a single forward pass compares layer-wise activation-change patterns against these profiles to guess the task and pick its key layers.
- **Entropy-driven allocation:** at those key layers, the entropy of visual attention guides whether attention is made more diffuse (counting) or more concentrated (other tasks).

Across 4 MLLMs and 7 benchmarks, RATION gives consistent accuracy gains. This suggests that matching internal attention to task needs is a cheap way to make multimodal reasoning more reliable.

## Key points
- **No training needed:** the method only modulates attention states at selected layers during inference. Code is released at github.com/betterfly123/RATION.
- **Routing accuracy:** without any extra supervision, task routing predicted the task type with over 80% accuracy in every category of the authors' diagnostic set.
- **Diagnostic finding:** counting benefits from diffuse attention, while recognition, spatial and OCR do better with focused attention. This motivates the task-dependent direction of the adjustment.
- **Case study on Qwen3-VL:** routing picked layers 16/18 as key for counting and 23/24 for OCR. The resulting heatmaps show wider object coverage for counting and tighter focus on text regions for OCR.
- **Scale tests:** Qwen3-VL at 2B, 4B, 8B and 32B improved in all four task categories, though gains shrink on some tasks for larger models.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Northeastern University](https://md.research.elamur.ai/free/en/universities/northeastern-university-cn/) — University, Shenyang, China · 53 works · 2,205 citations

📚 Shelves (2): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 18,614 works · [👁️ Computer Vision and Pattern Recognition](https://md.research.elamur.ai/free/en/shelves/computer-science/computer-vision-and-pattern-recognition/) · 3,486 works

🧭 Topics (2): [🧮 Reasoning](https://md.research.elamur.ai/free/en/topics/reasoning/) · 1,685 works · [🖼️ Vision & language](https://md.research.elamur.ai/free/en/topics/multimodal/) · 811 works

## Cite

Cite as: Xingle Xu, Fanheng Kong, Dexian Cai, Shi Feng, Xiaocui Yang, Daling Wang and Yifei Zhang. 2026. RATION: Entropy-Driven Task-Adaptive Visual Attention Allocation Framework for Multimodal Reasoning. Findings of the Association for Computational Linguistics: ACL 2026.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{xu2026ration,
  title = {{RATION: Entropy-Driven Task-Adaptive Visual Attention Allocation Framework for Multimodal Reasoning}},
  author = {Xingle Xu, Fanheng Kong, Dexian Cai, Shi Feng, Xiaocui Yang, Daling Wang and Yifei Zhang},
  booktitle = {Findings of the Association for Computational Linguistics: ACL 2026},
  year = {2026},
  doi = {10.18653/v1/2026.findings-acl.1238},
  url = {https://doi.org/10.18653/v1/2026.findings-acl.1238}
}
```

- [← previous Rating–Text Mismatch in Brazilian Portuguese Reviews: How Reliable Are Zero-Shot LLMs?](https://md.research.elamur.ai/free/en/papers/marreira-2026-rating-text-mismatch-brazilian-portuguese/)
- [next → Rational Communication Shapes Morphological Composition](https://md.research.elamur.ai/free/en/papers/yang-2026-rational-communication-shapes-morphological-composition/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/46/)

## References

45 references, 23 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. David Acuna, Ximing Lu, Jaehun Jung, Hyunwoo Kim, Amlan Kar, Sanja Fidler, and Yejin Choi. 2025. Socratic-MCTS: Test-time visual reasoning by asking the right questions. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pages 24158–24171, Suzhou, China. Assoc… [find](https://scholar.google.com/scholar?q=David+Acuna%2C+Ximing+Lu%2C+Jaehun+Jung%2C+Hyunwoo+Kim%2C+Amlan+Kar%2C+Sanja+Fidler%2C+and+Yejin+Choi%2E+2025%2E+Socratic%2DMCTS%3A+Test%2Dtime+visual+reasoning+by+asking+the+right+questions%2E+In+Proceedings+of+the+2025+Con)
2. Jinze Bai, Shuai Bai, Shusheng Yang, Shijie Wang, Sinan Tan, Peng Wang, Junyang Lin, Chang Zhou, and Jingren Zhou. 2024. [Qwen-VL: A versatile vision-language model for understanding, localization, text reading, and beyond](https://md.research.elamur.ai/free/en/papers/bai-2023-qwen-vl-versatile-vision-language/).
3. Shuai Bai, Yuxuan Cai, Ruizhe Chen, Keqin Chen, Xionghui Chen, Zesen Cheng, Lianghao Deng, Wei Ding, Chang Gao, Chunjiang Ge, Wenbin Ge, Zhifang Guo, Qidong Huang, Jie Huang, Fei Huang, Binyuan Hui, Shutong Jiang, Zhaohai Li, Mingsheng Li, and 45 others. 2025a. [Qwen3-vl technical report](https://md.research.elamur.ai/free/en/papers/bai-2025-qwen3-vl-technical-report/). Preprint, a…
4. Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, and 1 others. 2025b. Qwen2. 5-vl technical report. arXiv preprint arXiv:2502.13923. — [Qwen2.5-VL Technical Report](https://md.research.elamur.ai/free/en/papers/bai-2025-qwen2-5-vl-technical/)
5. Han Bao, Ryuichiro Hataya, and Ryo Karakida. 2024. Self-attention networks localize when qk-eigenspectrum concentrates. In Proceedings of the 41st International Conference on Machine Learning, pages 2903–2922. [find](https://scholar.google.com/scholar?q=Han+Bao%2C+Ryuichiro+Hataya%2C+and+Ryo+Karakida%2E+2024%2E+Self%2Dattention+networks+localize+when+qk%2Deigenspectrum+concentrates%2E+In+Proceedings+of+the+41st+International+Conference+on+Machine+Learning%2C+pages+2)
6. Joya Chen, Zhaoyang Lv, Shiwei Wu, Kevin Qinghong Lin, Chenan Song, Difei Gao, Jia-Wei Liu, Ziteng Gao, Dongxing Mao, and Mike Zheng Shou. 2024a. Videollm-online: Online video large language model for streaming video. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recogniti… [find](https://scholar.google.com/scholar?q=Joya+Chen%2C+Zhaoyang+Lv%2C+Shiwei+Wu%2C+Kevin+Qinghong+Lin%2C+Chenan+Song%2C+Difei+Gao%2C+Jia%2DWei+Liu%2C+Ziteng+Gao%2C+Dongxing+Mao%2C+and+Mike+Zheng+Shou%2E+2024a%2E+Videollm%2Donline%3A+Online+video+large+language+model+for)
7. Zhe Chen, Jiannan Wu, Wenhai Wang, Weijie Su, Guo Chen, Sen Xing, Muyan Zhong, Qinglong Zhang, Xizhou Zhu, Lewei Lu, and 1 others. 2024b. [Internvl: Scaling up vision foundation models and aligning for generic visual-linguistic tasks](https://md.research.elamur.ai/free/en/papers/chen-2023-internvl-scaling-up-vision/). In Proceedings of the IEEE/CVF conference on computer vision and p…
8. The work is supported by the National Natural Science Foundation of China (No. 62272092, Chaoyou Fu, Peixian Chen, Yunhang Shen, Yulei Qin, Mengdan Zhang, Xu Lin, Jinrui Yang, Xiawu Zheng, Ke Li, Xing Sun, and 1 others. 2025. [Mme: A comprehensive evaluation benchmark for multimodal large language mo…](https://md.research.elamur.ai/free/en/papers/fu-2023-mme-comprehensive-evaluation-benchmark-multimodal/)
9. Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, and 1 others. 2025. [Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning](https://md.research.elamur.ai/free/en/papers/deepseek-ai-2025-deepseek-r1-incentivizing-reasoning-capability/). arXiv preprint arXiv:2501.12948.
10. Danna Gurari, Qing Li, Abigale J Stangl, Anhong Guo, Chi Lin, Kristen Grauman, Jiebo Luo, and Jeffrey P Bigham. 2018. Vizwiz grand challenge: Answering visual questions from blind people. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 3608–3617. [find](https://scholar.google.com/scholar?q=Danna+Gurari%2C+Qing+Li%2C+Abigale+J+Stangl%2C+Anhong+Guo%2C+Chi+Lin%2C+Kristen+Grauman%2C+Jiebo+Luo%2C+and+Jeffrey+P+Bigham%2E+2018%2E+Vizwiz+grand+challenge%3A+Answering+visual+questions+from+blind+people%2E+In+Proceedin)
11. Mengdi Jia, Zekun Qi, Shaochen Zhang, Wenyao Zhang, Xinqiang Yu, Jiawei He, He Wang, and Li Yi. 2025. Omnispatial: Towards comprehensive spatial reasoning benchmark for vision language models. arXiv preprint arXiv:2506.03135. — [arXiv:2506.03135](https://arxiv.org/abs/2506.03135) [find](https://scholar.google.com/scholar?q=Mengdi+Jia%2C+Zekun+Qi%2C+Shaochen+Zhang%2C+Wenyao+Zhang%2C+Xinqiang+Yu%2C+Jiawei+He%2C+He+Wang%2C+and+Li+Yi%2E+2025%2E+Omnispatial%3A+Towards+comprehensive+spatial+reasoning+benchmark+for+vision+language+models%2E+arXiv+p)
12. Zixi Jia, Jiqiang Liu, Hexiao Li, Qinghua Liu, and Hongbin Gao. 2024. Dcot: Dual chain-of-thought prompting for large multimodal models. In The 16th Asian Conference on Machine Learning (Conference Track). [find](https://scholar.google.com/scholar?q=Zixi+Jia%2C+Jiqiang+Liu%2C+Hexiao+Li%2C+Qinghua+Liu%2C+and+Hongbin+Gao%2E+2024%2E+Dcot%3A+Dual+chain%2Dof%2Dthought+prompting+for+large+multimodal+models%2E+In+The+16th+Asian+Conference+on+Machine+Learning+%28Conference+Tr)
13. Fanheng Kong, Jingyuan Zhang, Yahui Liu, Hongzhi Zhang, Shi Feng, Xiaocui Yang, Daling Wang, Yu Tian, Fuzheng Zhang, Guorui Zhou, and 1 others. 2025a. Modality curation: Building universal embeddings for advanced multimodal information retrieval. arXiv preprint arXiv:2505.19650. — [arXiv:2505.19650](https://arxiv.org/abs/2505.19650) [find](https://scholar.google.com/scholar?q=Fanheng+Kong%2C+Jingyuan+Zhang%2C+Yahui+Liu%2C+Hongzhi+Zhang%2C+Shi+Feng%2C+Xiaocui+Yang%2C+Daling+Wang%2C+Yu+Tian%2C+Fuzheng+Zhang%2C+Guorui+Zhou%2C+and+1+others%2E+2025a%2E+Modality+curation%3A+Building+universal+embeddings+)
14. Fanheng Kong, Jingyuan Zhang, Hongzhi Zhang, Shi Feng, Daling Wang, Linhao Yu, Xingguang Ji, Yu Tian, Fuzheng Zhang, and 1 others. 2025b. Tuna: Comprehensive fine-grained temporal understanding evaluation on dense dynamic videos. In Proceedings of the 63rd Annual Meeting of the Association for Compu… [find](https://scholar.google.com/scholar?q=Fanheng+Kong%2C+Jingyuan+Zhang%2C+Hongzhi+Zhang%2C+Shi+Feng%2C+Daling+Wang%2C+Linhao+Yu%2C+Xingguang+Ji%2C+Yu+Tian%2C+Fuzheng+Zhang%2C+and+1+others%2E+2025b%2E+Tuna%3A+Comprehensive+fine%2Dgrained+temporal+understanding+evalua)
15. Bohao Li, Rui Wang, Guangzhi Wang, Yuying Ge, Yixiao Ge, and Ying Shan. 2023a. [Seed-bench: Benchmarking multimodal llms with generative comprehension](https://md.research.elamur.ai/free/en/papers/li-2023-seed-bench-benchmarking-multimodal-llms/). arXiv preprint arXiv:2307.16125.
16. Shengzhi Li, Rongyu Lin, and Shichao Pei. 2024a. Multi-modal preference alignment remedies degradation of visual instruction tuning on language models. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 14188–14200. [find](https://scholar.google.com/scholar?q=Shengzhi+Li%2C+Rongyu+Lin%2C+and+Shichao+Pei%2E+2024a%2E+Multi%2Dmodal+preference+alignment+remedies+degradation+of+visual+instruction+tuning+on+language+models%2E+In+Proceedings+of+the+62nd+Annual+Meeting+of+the)
17. Xinghang Li, Minghuan Liu, Hanbo Zhang, Cunjun Yu, Jie Xu, Hongtao Wu, Chilam Cheang, Ya Jing, Weinan Zhang, Huaping Liu, Hang Li, and Tao Kong. 2024b. Vision-language foundation models as effective robot imitators. In The Twelfth International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Xinghang+Li%2C+Minghuan+Liu%2C+Hanbo+Zhang%2C+Cunjun+Yu%2C+Jie+Xu%2C+Hongtao+Wu%2C+Chilam+Cheang%2C+Ya+Jing%2C+Weinan+Zhang%2C+Huaping+Liu%2C+Hang+Li%2C+and+Tao+Kong%2E+2024b%2E+Vision%2Dlanguage+foundation+models+as+effective+r)
18. Yifan Li, Yifan Du, Kun Zhou, Jinpeng Wang, Wayne Xin Zhao, and Ji-Rong Wen. 2023b. [Evaluating object hallucination in large vision-language models](https://md.research.elamur.ai/free/en/papers/li-2023-evaluating-object-hallucination-large-vision/). In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 292–305.
19. Ji Lin, Hongxu Yin, Wei Ping, Pavlo Molchanov, Mohammad Shoeybi, and Song Han. 2024. [Vila: On pre-training for visual language models](https://md.research.elamur.ai/free/en/papers/lin-2023-vila-pre-training-visual-language/). In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 26689–26699.
20. Zhihang Lin, Mingbao Lin, Luxi Lin, and Rongrong Ji. 2025. Boosting multimodal large language models with visual tokens withdrawal for rapid inference. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, pages 5334–5342. [find](https://scholar.google.com/scholar?q=Zhihang+Lin%2C+Mingbao+Lin%2C+Luxi+Lin%2C+and+Rongrong+Ji%2E+2025%2E+Boosting+multimodal+large+language+models+with+visual+tokens+withdrawal+for+rapid+inference%2E+In+Proceedings+of+the+AAAI+Conference+on+Artific)
21. Fuxiao Liu, Kevin Lin, Linjie Li, Jianfeng Wang, Yaser Yacoob, and Lijuan Wang. 2024. [Mitigating hallucination in large multi-modal models via robust instruction tuning](https://md.research.elamur.ai/free/en/papers/liu-2023-mitigating-hallucination-large-multi-modal/). In ICLR.
22. Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. 2023. [Visual instruction tuning](https://md.research.elamur.ai/free/en/papers/liu-2023-visual-instruction-tuning/). Advances in neural information processing systems, 36:34892– 34916.
23. Junteng Liu, Weihao Zeng, Xiwen Zhang, Yijun Wang, Zifei Shan, and Junxian He. 2025a. On the perception bottleneck of VLMs for chart understanding. In Findings of the Association for Computational Linguistics: EMNLP 2025, pages 10829–10841, Suzhou, China. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Junteng+Liu%2C+Weihao+Zeng%2C+Xiwen+Zhang%2C+Yijun+Wang%2C+Zifei+Shan%2C+and+Junxian+He%2E+2025a%2E+On+the+perception+bottleneck+of+VLMs+for+chart+understanding%2E+In+Findings+of+the+Association+for+Computational+Lin)
24. Yexiang Liu, Zekun Li, Zhi Fang, Nan Xu, Ran He, and Tieniu Tan. 2025b. Rethinking the role of prompting strategies in LLM test-time scaling: A perspective of probability theory. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages… [find](https://scholar.google.com/scholar?q=Yexiang+Liu%2C+Zekun+Li%2C+Zhi+Fang%2C+Nan+Xu%2C+Ran+He%2C+and+Tieniu+Tan%2E+2025b%2E+Rethinking+the+role+of+prompting+strategies+in+LLM+test%2Dtime+scaling%3A+A+perspective+of+probability+theory%2E+In+Proceedings+of+the)
25. Ahmed Masry, Xuan Long Do, Jia Qing Tan, Shafiq Joty, and Enamul Hoque. 2022. [Chartqa: A benchmark for question answering about charts with visual and logical reasoning](https://md.research.elamur.ai/free/en/papers/masry-2022-chartqa-benchmark-question-answering-about/). In Findings of the association for computational linguistics: ACL 2022, pages 2263– 2279.
26. Luca M Schulze Buschoff, Elif Akata, Matthias Bethge, and Eric Schulz. 2025. Visual cognition in multimodal large language models. Nature Machine Intelligence, 7(1):96–106. [find](https://scholar.google.com/scholar?q=Luca+M+Schulze+Buschoff%2C+Elif+Akata%2C+Matthias+Bethge%2C+and+Eric+Schulz%2E+2025%2E+Visual+cognition+in+multimodal+large+language+models%2E+Nature+Machine+Intelligence%2C+7%281%29%3A96%E2%80%93106%2E)
27. Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, and 1 others. 2024. [Deepseekmath: Pushing the limits of mathematical reasoning in open language models](https://md.research.elamur.ai/free/en/papers/shao-2024-deepseekmath-pushing-limits-mathematical-reasoning/). arXiv preprint arXiv:2402.03300.
28. Cheng Shi, Yizhou Yu, and Sibei Yang. 2025. Vision function layer in multimodal LLMs. In The Thirty-ninth Annual Conference on Neural Information Processing Systems. [find](https://scholar.google.com/scholar?q=Cheng+Shi%2C+Yizhou+Yu%2C+and+Sibei+Yang%2E+2025%2E+Vision+function+layer+in+multimodal+LLMs%2E+In+The+Thirty%2Dninth+Annual+Conference+on+Neural+Information+Processing+Systems%2E)
29. Shengbang Tong, Zhuang Liu, Yuexiang Zhai, Yi Ma, Yann LeCun, and Saining Xie. 2024. [Eyes wide shut? exploring the visual shortcomings of multimodal llms](https://md.research.elamur.ai/free/en/papers/tong-2024-eyes-wide-shut-exploring-visual/). In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 9568–9578.
30. Peidong Wang, Zhiming Ma, Xin Dai, Yongkang Liu, Shi Feng, Xiaocui Yang, Wenxing Hu, Zhihao Wang, Mingjun Pan, Li Yuan, and 1 others. 2026. [Safe-qaq: End-to-end slow-thinking audio-text fraud detection via reinforcement learning](https://md.research.elamur.ai/free/en/papers/wang-2026-safe-qaq-end-end-slow/). arXiv preprint arXiv:2601.01392.
31. Weiyun Wang, Zhangwei Gao, Lixin Gu, Hengjun Pu, Long Cui, Xingguang Wei, Zhaoyang Liu, Linglin Jing, Shenglong Ye, Jie Shao, and 1 others. 2025a. [Internvl3](https://md.research.elamur.ai/free/en/papers/wang-2025-internvl3-5-advancing-open/). 5: Advancing open-source multimodal models in versatility, reasoning, and efficiency. arXiv preprint arXiv:2508.18265.
32. Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc V Le, Ed H. Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. 2023. [Self-consistency improves chain of thought reasoning in language models](https://md.research.elamur.ai/free/en/papers/wang-2022-self-consistency-improves-chain-thought/). In The Eleventh International Conference on Learning Representations. in space: How multimodal large langu…
33. Shuo Yang, Caren Han, Siwen Luo, and Eduard Hovy. 2025b. Magic-vqa: Multimodal and grounded inference with commonsense knowledge for visual question answering. In Findings of the Association for Computational Linguistics: ACL 2025, pages 16967– 16986. [find](https://scholar.google.com/scholar?q=Shuo+Yang%2C+Caren+Han%2C+Siwen+Luo%2C+and+Eduard+Hovy%2E+2025b%2E+Magic%2Dvqa%3A+Multimodal+and+grounded+inference+with+commonsense+knowledge+for+visual+question+answering%2E+In+Findings+of+the+Association+for+Compu)
34. Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik R Narasimhan, and Yuan Cao. 2022. [React: Synergizing reasoning and acting in language models](https://md.research.elamur.ai/free/en/papers/yao-2022-react-synergizing-reasoning-acting-language/). In The eleventh international conference on learning representations.
35. Xiang Yue, Yuansheng Ni, Kai Zhang, Tianyu Zheng, Ruoqi Liu, Ge Zhang, Samuel Stevens, Dongfu Jiang, Weiming Ren, Yuxuan Sun, and 1 others. 2024. [Mmmu: A massive multi-discipline multimodal understanding and reasoning benchmark for expert agi](https://md.research.elamur.ai/free/en/papers/yue-2023-mmmu-massive-multi-discipline-multimodal/). In Proceedings of the IEEE/CVF Conference on Computer Vi…
36. Ziwei Wang, Weizhi Chen, Leyang Yang, Sheng Zhou, Shengchu Zhao, Hanbei Zhan, Jiongchao Jin, Liangcheng Li, Zirui Shao, and Jiajun Bu. 2025b. Mp-gui: Modality perception with mllms for gui understanding. In Proceedings of the Computer Vision and Pattern Recognition Conference, pages 29711– 29721. [find](https://scholar.google.com/scholar?q=Ziwei+Wang%2C+Weizhi+Chen%2C+Leyang+Yang%2C+Sheng+Zhou%2C+Shengchu+Zhao%2C+Hanbei+Zhan%2C+Jiongchao+Jin%2C+Liangcheng+Li%2C+Zirui+Shao%2C+and+Jiajun+Bu%2E+2025b%2E+Mp%2Dgui%3A+Modality+perception+with+mllms+for+gui+understandi)
37. Zhiyuan Zeng, Qinyuan Cheng, Zhangyue Yin, Yunhua Zhou, and Xipeng Qiu. 2025. [Revisiting the test-time scaling of o1-like models: Do they truly possess test-time scaling capabilities?](https://md.research.elamur.ai/free/en/papers/zeng-2025-revisiting-test-time-scaling-o1/) In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers),…
38. Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, and 1 others. 2022. [Chain-of-thought prompting elicits reasoning in large language models](https://md.research.elamur.ai/free/en/papers/wei-2022-chain-thought-prompting-elicits-reasoning/). Advances in neural information processing systems, 35:24824– 24837.
39. Kaichen Zhang, Bo Li, Peiyuan Zhang, Fanyi Pu, Joshua Adrian Cahyono, Kairui Hu, Shuai Liu, Yuanhan Zhang, Jingkang Yang, Chunyuan Li, and Ziwei Liu. 2024a. [Lmms-eval: Reality check on the evaluation of large multimodal models](https://md.research.elamur.ai/free/en/papers/zhang-2025-lmms-eval-reality-check-evaluation/). Preprint, arXiv:2407.12772.
40. Mingrui Wu, Xinyue Cai, Jiayi Ji, Jiale Li, Oucheng Huang, Gen Luo, Hao Fei, Guannan Jiang, Xiaoshuai Sun, and Rongrong Ji. 2024. Controlmllm: Training-free visual prompt learning for multimodal large language models. Advances in Neural Information Processing Systems, 37:45206–45234. [find](https://scholar.google.com/scholar?q=Mingrui+Wu%2C+Xinyue+Cai%2C+Jiayi+Ji%2C+Jiale+Li%2C+Oucheng+Huang%2C+Gen+Luo%2C+Hao+Fei%2C+Guannan+Jiang%2C+Xiaoshuai+Sun%2C+and+Rongrong+Ji%2E+2024%2E+Controlmllm%3A+Training%2Dfree+visual+prompt+learning+for+multimodal+large)
41. Tao Zhang, Jinyong Wen, Zhen Chen, Kun Ding, Shiming Xiang, and Chunhong Pan. 2025. UNIP: Rethinking pre-trained attention patterns for infrared semantic segmentation. In The Thirteenth International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Tao+Zhang%2C+Jinyong+Wen%2C+Zhen+Chen%2C+Kun+Ding%2C+Shiming+Xiang%2C+and+Chunhong+Pan%2E+2025%2E+UNIP%3A+Rethinking+pre%2Dtrained+attention+patterns+for+infrared+semantic+segmentation%2E+In+The+Thirteenth+International+)
42. Qiong Wu, Wenhao Lin, Yiyi Zhou, Weihao Ye, Zhanpeng Zeng, Xiaoshuai Sun, and Rongrong Ji. 2025. Accelerating multimodal large language models via dynamic visual-token exit and the empirical findings. In The Thirty-ninth Annual Conference on Neural Information Processing Systems. [find](https://scholar.google.com/scholar?q=Qiong+Wu%2C+Wenhao+Lin%2C+Yiyi+Zhou%2C+Weihao+Ye%2C+Zhanpeng+Zeng%2C+Xiaoshuai+Sun%2C+and+Rongrong+Ji%2E+2025%2E+Accelerating+multimodal+large+language+models+via+dynamic+visual%2Dtoken+exit+and+the+empirical+findings%2E)
43. Zhuosheng Zhang, Aston Zhang, Mu Li, Hai Zhao, George Karypis, and Alex Smola. 2024b. [Multimodal chain-of-thought reasoning in language models](https://md.research.elamur.ai/free/en/papers/zhang-2023-multimodal-chain-thought-reasoning-language/). Transactions on Machine Learning Research, 2024.
44. Zhiyang Xu, Chao Feng, Rulin Shao, Trevor Ashby, Ying Shen, Di Jin, Yu Cheng, Qifan Wang, and Lifu Huang. 2024. Vision-flan: Scaling human-labeled tasks in visual instruction tuning. In Findings of the Association for Computational Linguistics ACL 2024, pages 15271–15342. [find](https://scholar.google.com/scholar?q=Zhiyang+Xu%2C+Chao+Feng%2C+Rulin+Shao%2C+Trevor+Ashby%2C+Ying+Shen%2C+Di+Jin%2C+Yu+Cheng%2C+Qifan+Wang%2C+and+Lifu+Huang%2E+2024%2E+Vision%2Dflan%3A+Scaling+human%2Dlabeled+tasks+in+visual+instruction+tuning%2E+In+Findings+of+th)
45. Jihan Yang, Shusheng Yang, Anjali W Gupta, Rilyn Han, Li Fei-Fei, and Saining Xie. 2025a. Thinking [find](https://scholar.google.com/scholar?q=Jihan+Yang%2C+Shusheng+Yang%2C+Anjali+W+Gupta%2C+Rilyn+Han%2C+Li+Fei%2DFei%2C+and+Saining+Xie%2E+2025a%2E+Thinking)

*Updated 2 November 2026*