---
title: "VLN-MME — paper by Zhao et al. (2026) — summary and ideas"
description: "The authors present VLN-MME, a simulator-free, modular framework that separates Model, Agent and Task components. 54 references."
authors: ["Xunyi Zhao", "Gengze Zhou", "Qi Wu"]
year: 2026
venue: "Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)"
source: "https://aclanthology.org/2026.acl-long.1300/"
doi: "10.18653/v1/2026.acl-long.1300"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/zhao-2026-vln-mme-diagnosing-mllms-as/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/zhao-2026-vln-mme-diagnosing-mllms-as/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/55/) / VLN-MME: Diagnosing MLLMs as Language-guided Visual Navigation Agents

# VLN-MME: Diagnosing MLLMs as Language-guided Visual Navigation Agents

**Authors:** Xunyi Zhao, Gengze Zhou and [Qi Wu](https://md.research.elamur.ai/free/en/authors/qi-wu/) · **Year:** 2026 · **Venue:** [Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)](https://md.research.elamur.ai/free/en/venues/acl/) ([ACL 2026](https://md.research.elamur.ai/free/en/venues/acl/2026/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2026.acl-long.1300/) · [PDF](https://aclanthology.org/2026.acl-long.1300.pdf) · [DOI 10.18653/v1/2026.acl-long.1300](https://doi.org/10.18653/v1/2026.acl-long.1300) · [OpenAlex](https://openalex.org/W7166877632)

**Citations:** 0 citations (OpenAlex, 2026-10-14)

## Summary
Multimodal large language models (MLLMs) do well on static vision-language benchmarks. It is less clear how they perform as embodied agents that must reason about 3D space over many interaction steps. Current Vision-and-Language Navigation (VLN) evaluation depends on costly simulators such as Matterport3D or Habitat. Suites like NavBench also fix one agent design, so a model's own ability cannot be separated from its prompting strategy.

The authors present VLN-MME, a simulator-free, modular framework that separates Model, Agent and Task components. It comes with a compact benchmark sampled from R2R, REVERIE and ObjectNav. Testing MLLMs as zero-shot navigators, they find that Chain-of-Thought and self-reflection unexpectedly hurt performance. Models often recognise relevant objects but fail to act on them: they loop or do not stop at the goal. The authors call this a "perception-action gap." Case studies show that planning guidance and prompts with failure examples recover much of the lost performance.

The work offers diagnostic guidance for post-training MLLMs as embodied agents.

## Key points
- **Framework design:** Models, agents and tasks are registered through a factory pattern and coordinated by a central Runner. Agents differ in memory type (text-summarised action history vs. topological text maps) and in reasoning style (direct prediction, CoT, self-reflection).
- **Benchmark construction:** Episodes are drawn by stratified sampling from the validation unseen splits of R2R, REVERIE and ObjectNav. Sampling is balanced by Matterport3D scan, path-length bin, instruction choice and, for ObjectNav, target category.
- **Hard-negative case study:** The authors selected 25 trajectories on which all open-source models failed and used Qwen2.5VL-7B as the navigator. Adding a Qwen3VL-4B oracle raised SR from 0 to 52, OSR to 68 and SPL to 41.28.
- **Failure-aware few-shot prompting:** Gains were smaller than with the oracle. SR reached 12 / 16 / 16 with 1 / 2 / 3 failure examples, and SPL rose to 14.29 with three examples.
- **Main limitation and release:** The study diagnoses problems but proposes no algorithmic fix. Code, processed datasets and environment artifacts are released (github.com/billzhao1030/VLN-MME).

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

📚 Shelves (5): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 18,614 works · [👁️ Computer Vision and Pattern Recognition](https://md.research.elamur.ai/free/en/shelves/computer-science/computer-vision-and-pattern-recognition/) · 3,486 works · [⚙️ Engineering](https://md.research.elamur.ai/free/en/shelves/engineering/) · 188 works · [⚙️ Control and Systems Engineering](https://md.research.elamur.ai/free/en/shelves/engineering/control-and-systems-engineering/) · 62 works · [⚙️ Automotive Engineering](https://md.research.elamur.ai/free/en/shelves/engineering/automotive-engineering/) · 9 works

## Cite

Cite as: Xunyi Zhao, Gengze Zhou and Qi Wu. 2026. VLN-MME: Diagnosing MLLMs as Language-guided Visual Navigation Agents. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{zhao2026vln,
  title = {{VLN-MME: Diagnosing MLLMs as Language-guided Visual Navigation Agents}},
  author = {Xunyi Zhao, Gengze Zhou and Qi Wu},
  booktitle = {Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2026},
  doi = {10.18653/v1/2026.acl-long.1300},
  url = {https://doi.org/10.18653/v1/2026.acl-long.1300}
}
```

- [← previous VLCE: A Knowledge-Enhanced Framework for Image Description in Disaster Assessment](https://md.research.elamur.ai/free/en/papers/rahman-2026-vlce-knowledge-enhanced-framework-image/)
- [next → Vocabulary Shapes Cross-Lingual Variation of Word-Order Learnability in Language Models](https://md.research.elamur.ai/free/en/papers/martins-2026-vocabulary-shapes-cross-lingual-variation/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/55/)

## References

54 references, 18 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Peter Anderson, Qi Wu, Damien Teney, Jake Bruce, Mark Johnson, Niko Sünderhauf, Ian Reid, Stephen Gould, and Anton van den Hengel. 2018. [Vision-and-language navigation: Interpreting visually-grounded navigation instructions in real environments](https://md.research.elamur.ai/free/en/papers/anderson-2017-vision-language-navigation-interpreting-visually/). In Proceedings of the IEEE Conference on Computer Visi…
2. Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, and 1 others. 2025. Qwen2. 5-vl technical report. arXiv preprint arXiv:2502.13923. — [Qwen2.5-VL Technical Report](https://md.research.elamur.ai/free/en/papers/bai-2025-qwen2-5-vl-technical/)
3. Dhruv Batra, Aaron Gokaslan, Aniruddha Kembhavi, Oleksandr Maksymets, Roozbeh Mottaghi, Manolis Savva, Alexander Toshev, and Erik Wijmans. 2020. ObjectNav Revisited: On Evaluation of Embodied Agents Navigating to Objects. In arXiv:2006.13171. — [arXiv:2006.13171](https://arxiv.org/abs/2006.13171) [find](https://scholar.google.com/scholar?q=Dhruv+Batra%2C+Aaron+Gokaslan%2C+Aniruddha+Kembhavi%2C+Oleksandr+Maksymets%2C+Roozbeh+Mottaghi%2C+Manolis+Savva%2C+Alexander+Toshev%2C+and+Erik+Wijmans%2E+2020%2E+ObjectNav+Revisited%3A+On+Evaluation+of+Embodied+Agents+N)
4. Angel Chang, Angela Dai, Thomas Funkhouser, Maciej Halber, Matthias Niebner, Manolis Savva, Shuran Song, Andy Zeng, and Yinda Zhang. 2017. Matterport3d: Learning from rgb-d data in indoor environments. In 2017 International Conference on 3D Vision (3DV), pages 667–676. IEEE. [find](https://scholar.google.com/scholar?q=Angel+Chang%2C+Angela+Dai%2C+Thomas+Funkhouser%2C+Maciej+Halber%2C+Matthias+Niebner%2C+Manolis+Savva%2C+Shuran+Song%2C+Andy+Zeng%2C+and+Yinda+Zhang%2E+2017%2E+Matterport3d%3A+Learning+from+rgb%2Dd+data+in+indoor+environments)
5. Fu Chaoyou, Chen Peixian, Shen Yunhang, Qin Yulei, Zhang Mengdan, Lin Xu, Yang Jinrui, Zheng Xiawu, Li Ke, Sun Xing, and 1 others. 2023. [Mme: A comprehensive evaluation benchmark for multimodal large language models](https://md.research.elamur.ai/free/en/papers/fu-2023-mme-comprehensive-evaluation-benchmark-multimodal/). arXiv preprint arXiv:2306.13394, 3.
6. Jiaqi Chen, Bingqian Lin, Ran Xu, Zhenhua Chai, Xiaodan Liang, and Kwan-Yee K Wong. 2024. Mapgpt: Map-guided prompting for unified vision-and-language navigation. arXiv preprint arXiv:2401.07314. — [arXiv:2401.07314](https://arxiv.org/abs/2401.07314) [find](https://scholar.google.com/scholar?q=Jiaqi+Chen%2C+Bingqian+Lin%2C+Ran+Xu%2C+Zhenhua+Chai%2C+Xiaodan+Liang%2C+and+Kwan%2DYee+K+Wong%2E+2024%2E+Mapgpt%3A+Map%2Dguided+prompting+for+unified+vision%2Dand%2Dlanguage+navigation%2E+arXiv+preprint+arXiv%3A2401%2E07314%2E+%E2%80%94+%5Ba)
7. Shizhe Chen, Pierre-Louis Guhur, Cordelia Schmid, and Ivan Laptev. 2021. History aware multimodal transformer for vision-and-language navigation. Advances in Neural Information Processing Systems, 34:5834–5847. [find](https://scholar.google.com/scholar?q=Shizhe+Chen%2C+Pierre%2DLouis+Guhur%2C+Cordelia+Schmid%2C+and+Ivan+Laptev%2E+2021%2E+History+aware+multimodal+transformer+for+vision%2Dand%2Dlanguage+navigation%2E+Advances+in+Neural+Information+Processing+Systems%2C+34%3A)
8. Shizhe Chen, Pierre-Louis Guhur, Makarand Tapaswi, Cordelia Schmid, and Ivan Laptev. 2022. Think global, act local: Dual-scale graph transformer for vision-and-language navigation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 16537–16547. [find](https://scholar.google.com/scholar?q=Shizhe+Chen%2C+Pierre%2DLouis+Guhur%2C+Makarand+Tapaswi%2C+Cordelia+Schmid%2C+and+Ivan+Laptev%2E+2022%2E+Think+global%2C+act+local%3A+Dual%2Dscale+graph+transformer+for+vision%2Dand%2Dlanguage+navigation%2E+In+Proceedings+of+t)
9. An-Chieh Cheng, Yandong Ji, Zhaojing Yang, Zaitian Gongye, Xueyan Zou, Jan Kautz, Erdem Bıyık, Hongxu Yin, Sifei Liu, and Xiaolong Wang. 2024. Navila: Legged robot vision-language-action model for navigation. arXiv preprint arXiv:2412.04453. — [arXiv:2412.04453](https://arxiv.org/abs/2412.04453) [find](https://scholar.google.com/scholar?q=An%2DChieh+Cheng%2C+Yandong+Ji%2C+Zhaojing+Yang%2C+Zaitian+Gongye%2C+Xueyan+Zou%2C+Jan+Kautz%2C+Erdem+B%C4%B1y%C4%B1k%2C+Hongxu+Yin%2C+Sifei+Liu%2C+and+Xiaolong+Wang%2E+2024%2E+Navila%3A+Legged+robot+vision%2Dlanguage%2Daction+model+for+nav)
10. Zhili Cheng, Yuge Tu, Ran Li, Shiqi Dai, Jinyi Hu, Shengding Hu, Jiahao Li, Yang Shi, Tianyu Yu, Weize Chen, and 1 others. 2025. Embodiedeval: Evaluate multimodal llms as embodied agents. arXiv preprint arXiv:2501.11858. — [arXiv:2501.11858](https://arxiv.org/abs/2501.11858) [find](https://scholar.google.com/scholar?q=Zhili+Cheng%2C+Yuge+Tu%2C+Ran+Li%2C+Shiqi+Dai%2C+Jinyi+Hu%2C+Shengding+Hu%2C+Jiahao+Li%2C+Yang+Shi%2C+Tianyu+Yu%2C+Weize+Chen%2C+and+1+others%2E+2025%2E+Embodiedeval%3A+Evaluate+multimodal+llms+as+embodied+agents%2E+arXiv+prepri)
11. Erik Daxberger, Nina Wenzel, David Griffiths, Haiming Gang, Justin Lazarow, Gefen Kohavi, Kai Kang, Marcin Eichner, Yinfei Yang, Afshin Dehghan, and 1 others. 2025. Mm-spatial: Exploring 3d spatial understanding in multimodal llms. In Proceedings of the IEEE/CVF International Conference on Computer… [find](https://scholar.google.com/scholar?q=Erik+Daxberger%2C+Nina+Wenzel%2C+David+Griffiths%2C+Haiming+Gang%2C+Justin+Lazarow%2C+Gefen+Kohavi%2C+Kai+Kang%2C+Marcin+Eichner%2C+Yinfei+Yang%2C+Afshin+Dehghan%2C+and+1+others%2E+2025%2E+Mm%2Dspatial%3A+Exploring+3d+spatial+un)
12. Xiang Deng, Yu Gu, Boyuan Zheng, Shijie Chen, Sam Stevens, Boshi Wang, Huan Sun, and Yu Su. 2023. [Mind2web: Towards a generalist agent for the web](https://md.research.elamur.ai/free/en/papers/deng-2023-mind2web-towards-generalist-agent-web/). Advances in Neural Information Processing Systems, 36:28091–28114.
13. Xiangyu Dong, Haoran Zhao, Jiang Gao, Haozhou Li, Xiaoguang Ma, Yaoming Zhou, Fuhai Chen, and Juan Liu. 2025. Se-vln: A self-evolving vision-language navigation framework based on multimodal large language models. arXiv preprint arXiv:2507.13152. — [arXiv:2507.13152](https://arxiv.org/abs/2507.13152) [find](https://scholar.google.com/scholar?q=Xiangyu+Dong%2C+Haoran+Zhao%2C+Jiang+Gao%2C+Haozhou+Li%2C+Xiaoguang+Ma%2C+Yaoming+Zhou%2C+Fuhai+Chen%2C+and+Juan+Liu%2E+2025%2E+Se%2Dvln%3A+A+self%2Devolving+vision%2Dlanguage+navigation+framework+based+on+multimodal+large+lan)
14. Hao Fei, Yuan Zhou, Juncheng Li, Xiangtai Li, Qingshan Xu, Bobo Li, Shengqiong Wu, Yaoting Wang, Junbao Zhou, Jiahao Meng, and 1 others. 2025. On path to multimodal generalist: General-level and general-bench. In Forty-second International Conference on Machine Learning. [find](https://scholar.google.com/scholar?q=Hao+Fei%2C+Yuan+Zhou%2C+Juncheng+Li%2C+Xiangtai+Li%2C+Qingshan+Xu%2C+Bobo+Li%2C+Shengqiong+Wu%2C+Yaoting+Wang%2C+Junbao+Zhou%2C+Jiahao+Meng%2C+and+1+others%2E+2025%2E+On+path+to+multimodal+generalist%3A+General%2Dlevel+and+gener)
15. Yicong Hong, Qi Wu, Yuankai Qi, Cristian Rodriguez-Opazo, and Stephen Gould. 2021. A recurrent vision-and-language bert for navigation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 1643–1653. [find](https://scholar.google.com/scholar?q=Yicong+Hong%2C+Qi+Wu%2C+Yuankai+Qi%2C+Cristian+Rodriguez%2DOpazo%2C+and+Stephen+Gould%2E+2021%2E+A+recurrent+vision%2Dand%2Dlanguage+bert+for+navigation%2E+In+Proceedings+of+the+IEEE%2FCVF+Conference+on+Computer+Vision+and)
16. Wenbo Hu, Yining Hong, Yanjun Wang, Leison Gao, Zibu Wei, Xingcheng Yao, Nanyun Peng, Yonatan Bitton, Idan Szpektor, and Kai-Wei Chang. 2025. 3dllm-mem: Long-term spatial-temporal memory for embodied 3d large language model. arXiv preprint arXiv:2505.22657. — [arXiv:2505.22657](https://arxiv.org/abs/2505.22657) [find](https://scholar.google.com/scholar?q=Wenbo+Hu%2C+Yining+Hong%2C+Yanjun+Wang%2C+Leison+Gao%2C+Zibu+Wei%2C+Xingcheng+Yao%2C+Nanyun+Peng%2C+Yonatan+Bitton%2C+Idan+Szpektor%2C+and+Kai%2DWei+Chang%2E+2025%2E+3dllm%2Dmem%3A+Long%2Dterm+spatial%2Dtemporal+memory+for+embodied+)
17. Gabriel Ilharco, Vihan Jain, Alexander Ku, Eugene Ie, and Jason Baldridge. 2019. General evaluation for instruction conditioned navigation using dynamic time warping. arXiv preprint arXiv:1907.05446. — [arXiv:1907.05446](https://arxiv.org/abs/1907.05446) [find](https://scholar.google.com/scholar?q=Gabriel+Ilharco%2C+Vihan+Jain%2C+Alexander+Ku%2C+Eugene+Ie%2C+and+Jason+Baldridge%2E+2019%2E+General+evaluation+for+instruction+conditioned+navigation+using+dynamic+time+warping%2E+arXiv+preprint+arXiv%3A1907%2E05446%2E+)
18. Vihan Jain, Gabriel Magalhaes, Alexander Ku, Ashish Vaswani, Eugene Ie, and Jason Baldridge. 2019. Stay on the path: Instruction fidelity in vision-and-language navigation. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 1862–1872. [find](https://scholar.google.com/scholar?q=Vihan+Jain%2C+Gabriel+Magalhaes%2C+Alexander+Ku%2C+Ashish+Vaswani%2C+Eugene+Ie%2C+and+Jason+Baldridge%2E+2019%2E+Stay+on+the+path%3A+Instruction+fidelity+in+vision%2Dand%2Dlanguage+navigation%2E+In+Proceedings+of+the+57th+)
19. Jacob Krantz, Erik Wijmans, Arjun Majumdar, Dhruv Batra, and Stefan Lee. 2020. Beyond the nav-graph: Vision-and-language navigation in continuous environments. In European Conference on Computer Vision, pages 104–120. Springer. [find](https://scholar.google.com/scholar?q=Jacob+Krantz%2C+Erik+Wijmans%2C+Arjun+Majumdar%2C+Dhruv+Batra%2C+and+Stefan+Lee%2E+2020%2E+Beyond+the+nav%2Dgraph%3A+Vision%2Dand%2Dlanguage+navigation+in+continuous+environments%2E+In+European+Conference+on+Computer+Visio)
20. Alexander Ku, Peter Anderson, Roma Patel, Eugene Ie, and Jason Baldridge. 2020. Room-across-room: Multilingual vision-and-language navigation with dense spatiotemporal grounding. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 4392–4412. [find](https://scholar.google.com/scholar?q=Alexander+Ku%2C+Peter+Anderson%2C+Roma+Patel%2C+Eugene+Ie%2C+and+Jason+Baldridge%2E+2020%2E+Room%2Dacross%2Droom%3A+Multilingual+vision%2Dand%2Dlanguage+navigation+with+dense+spatiotemporal+grounding%2E+In+Proceedings+of+the)
21. Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph Gonzalez, Hao Zhang, and Ion Stoica. 2023. [Efficient memory management for large language model serving with pagedattention](https://md.research.elamur.ai/free/en/papers/kwon-2023-efficient-memory-management-large-language/). In Proceedings of the 29th symposium on operating systems principles, pages 611–626.
22. Bo Li, Yuanhan Zhang, Dong Guo, Renrui Zhang, Feng Li, Hao Zhang, Kaichen Zhang, Peiyuan Zhang, Yanwei Li, Ziwei Liu, and 1 others. 2024a. [Llava-onevision: Easy visual task transfer](https://md.research.elamur.ai/free/en/papers/li-2024-llava-onevision-easy-visual-task/). arXiv preprint arXiv:2408.03326.
23. Chengshu Li, Ruohan Zhang, Josiah Wong, Cem Gokmen, Sanjana Srivastava, Roberto Martín-Martín, Chen Wang, Gabrael Levine, Wensi Ai, Benjamin Martinez, and 1 others. 2024b. Behavior-1k: A human-centered, embodied ai benchmark with 1,000 everyday activities and realistic simulation. arXiv preprint arX… — [arXiv:2403.09227](https://arxiv.org/abs/2403.09227) [find](https://scholar.google.com/scholar?q=Chengshu+Li%2C+Ruohan+Zhang%2C+Josiah+Wong%2C+Cem+Gokmen%2C+Sanjana+Srivastava%2C+Roberto+Mart%C3%ADn%2DMart%C3%ADn%2C+Chen+Wang%2C+Gabrael+Levine%2C+Wensi+Ai%2C+Benjamin+Martinez%2C+and+1+others%2E+2024b%2E+Behavior%2D1k%3A+A+human%2Dcentere)
24. Yuan-Hong Liao, Rafid Mahmood, Sanja Fidler, and David Acuna. 2024. Reasoning paths with reference objects elicit quantitative spatial reasoning in large vision-language models. arXiv preprint arXiv:2409.09788. [find](https://scholar.google.com/scholar?q=Yuan%2DHong+Liao%2C+Rafid+Mahmood%2C+Sanja+Fidler%2C+and+David+Acuna%2E+2024%2E+Reasoning+paths+with+reference+objects+elicit+quantitative+spatial+reasoning+in+large+vision%2Dlanguage+models%2E+arXiv+preprint+arXiv%3A2)
25. Bingqian Lin, Yunshuang Nie, Ziming Wei, Jiaqi Chen, Shikui Ma, Jianhua Han, Hang Xu, Xiaojun Chang, and Xiaodan Liang. 2024. Navcot: Boosting llm-based vision-and-language navigation via learning disentangled reasoning. arXiv preprint arXiv:2403.07376. — [arXiv:2403.07376](https://arxiv.org/abs/2403.07376) [find](https://scholar.google.com/scholar?q=Bingqian+Lin%2C+Yunshuang+Nie%2C+Ziming+Wei%2C+Jiaqi+Chen%2C+Shikui+Ma%2C+Jianhua+Han%2C+Hang+Xu%2C+Xiaojun+Chang%2C+and+Xiaodan+Liang%2E+2024%2E+Navcot%3A+Boosting+llm%2Dbased+vision%2Dand%2Dlanguage+navigation+via+learning+dis)
26. Yuan Liu, Haodong Duan, Yuanhan Zhang, Bo Li, Songyang Zhang, Wangbo Zhao, Yike Yuan, Jiaqi Wang, Conghui He, Ziwei Liu, and 1 others. 2024. [Mmbench: Is your multi-modal model an all-around player?](https://md.research.elamur.ai/free/en/papers/liu-2023-mmbench-multi-modal-model-all/) In European conference on computer vision, pages 216–233. Springer.
27. Yuxing Long, Xiaoqi Li, Wenzhe Cai, and Hao Dong. 2023. Discuss before moving: Visual language navigation via multi-expert discussions. arXiv preprint arXiv:2309.11382. — [arXiv:2309.11382](https://arxiv.org/abs/2309.11382) [find](https://scholar.google.com/scholar?q=Yuxing+Long%2C+Xiaoqi+Li%2C+Wenzhe+Cai%2C+and+Hao+Dong%2E+2023%2E+Discuss+before+moving%3A+Visual+language+navigation+via+multi%2Dexpert+discussions%2E+arXiv+preprint+arXiv%3A2309%2E11382%2E+%E2%80%94+%5BarXiv%3A2309%2E11382%5D%28https%3A%2F%2Far)
28. Pan Lu, Hritik Bansal, Tony Xia, Jiacheng Liu, Chunyuan Li, Hannaneh Hajishirzi, Hao Cheng, Kai-Wei Chang, Michel Galley, and Jianfeng Gao. 2023. [Mathvista: Evaluating mathematical reasoning of foundation models in visual contexts](https://md.research.elamur.ai/free/en/papers/lu-2023-mathvista-evaluating-mathematical-reasoning-foundation/). arXiv preprint arXiv:2310.02255.
29. Bowen Pan, Rameswar Panda, SouYoung Jin, Rogerio Feris, Aude Oliva, Phillip Isola, and Yoon Kim. 2023. Langnav: Language as a perceptual representation for navigation. arXiv preprint arXiv:2310.07889. [find](https://scholar.google.com/scholar?q=Bowen+Pan%2C+Rameswar+Panda%2C+SouYoung+Jin%2C+Rogerio+Feris%2C+Aude+Oliva%2C+Phillip+Isola%2C+and+Yoon+Kim%2E+2023%2E+Langnav%3A+Language+as+a+perceptual+representation+for+navigation%2E+arXiv+preprint+arXiv%3A2310%2E07889%2E)
30. Yuankai Qi, Qi Wu, Peter Anderson, Xin Wang, William Yang Wang, Chunhua Shen, and Anton van den Hengel. 2020. Reverie: Remote embodied visual referring expression in real indoor environments. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 9982–9991. [find](https://scholar.google.com/scholar?q=Yuankai+Qi%2C+Qi+Wu%2C+Peter+Anderson%2C+Xin+Wang%2C+William+Yang+Wang%2C+Chunhua+Shen%2C+and+Anton+van+den+Hengel%2E+2020%2E+Reverie%3A+Remote+embodied+visual+referring+expression+in+real+indoor+environments%2E+In+Proce)
31. Yanyuan Qiao, Haodong Hong, Wenqi Lyu, Dong An, Siqi Zhang, Yutong Xie, Xinyu Wang, and Qi Wu. 2025. Navbench: Probing multimodal large language models for embodied navigation. arXiv preprint arXiv:2506.01031. — [arXiv:2506.01031](https://arxiv.org/abs/2506.01031) [find](https://scholar.google.com/scholar?q=Yanyuan+Qiao%2C+Haodong+Hong%2C+Wenqi+Lyu%2C+Dong+An%2C+Siqi+Zhang%2C+Yutong+Xie%2C+Xinyu+Wang%2C+and+Qi+Wu%2E+2025%2E+Navbench%3A+Probing+multimodal+large+language+models+for+embodied+navigation%2E+arXiv+preprint+arXiv%3A25)
32. Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi. 2023. [Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models](https://md.research.elamur.ai/free/en/papers/li-2023-blip-2-bootstrapping-language-image/). arXiv preprint arXiv:2301.12597.
33. Yanyuan Qiao, Yuankai Qi, Zheng Yu, Jing Liu, and Qi Wu. 2023. March in chat: Interactive prompting for remote embodied referring expression. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 15758–15767. [find](https://scholar.google.com/scholar?q=Yanyuan+Qiao%2C+Yuankai+Qi%2C+Zheng+Yu%2C+Jing+Liu%2C+and+Qi+Wu%2E+2023%2E+March+in+chat%3A+Interactive+prompting+for+remote+embodied+referring+expression%2E+In+Proceedings+of+the+IEEE%2FCVF+International+Conference+on)
34. Kunchang Li, Yali Wang, Yinan He, Yizhuo Li, Yi Wang, Yi Liu, Zun Wang, Jilan Xu, Guo Chen, Ping Luo, and 1 others. 2024c. [Mvbench: A comprehensive multi-modal video understanding benchmark](https://md.research.elamur.ai/free/en/papers/li-2023-mvbench-comprehensive-multi-modal-video/). In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 22195–22206.
35. Manolis Savva, Abhishek Kadian, Oleksandr Maksymets, Yili Zhao, Erik Wijmans, Bhavana Jain, Julian Straub, Jia Liu, Vladlen Koltun, Jitendra Malik, and 1 others. 2019. [Habitat: A platform for embodied ai research](https://md.research.elamur.ai/free/en/papers/savva-2019-habitat-platform-embodied-ai-research/). In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 9339…
36. Xijia Tao, Yihua Teng, Xinxing Su, Xinyu Fu, Jihao Wu, Chaofan Tao, Ziru Liu, Haoli Bai, Rui Liu, and Lingpeng Kong. 2025. Mmsearch-plus: A simple yet challenging benchmark for multimodal browsing agents. arXiv e-prints, pages arXiv–2508. [find](https://scholar.google.com/scholar?q=Xijia+Tao%2C+Yihua+Teng%2C+Xinxing+Su%2C+Xinyu+Fu%2C+Jihao+Wu%2C+Chaofan+Tao%2C+Ziru+Liu%2C+Haoli+Bai%2C+Rui+Liu%2C+and+Lingpeng+Kong%2E+2025%2E+Mmsearch%2Dplus%3A+A+simple+yet+challenging+benchmark+for+multimodal+browsing+age)
37. Jesse Thomason, Michael Murray, Maya Cakmak, and Luke Zettlemoyer. 2020. Vision-and-dialog navigation. In Conference on Robot Learning, pages 394–406. [find](https://scholar.google.com/scholar?q=Jesse+Thomason%2C+Michael+Murray%2C+Maya+Cakmak%2C+and+Luke+Zettlemoyer%2E+2020%2E+Vision%2Dand%2Ddialog+navigation%2E+In+Conference+on+Robot+Learning%2C+pages+394%E2%80%93406%2E)
38. Harsh Trivedi, Tushar Khot, Mareike Hartmann, Ruskin Manku, Vinty Dong, Edward Li, Shashank Gupta, Ashish Sabharwal, and Niranjan Balasubramanian. 2024. [Appworld: A controllable world of apps and people for benchmarking interactive coding agents](https://md.research.elamur.ai/free/en/papers/trivedi-2024-appworld-controllable-world-apps-people/). arXiv preprint arXiv:2407.18901.
39. Weixuan Wang, Dongge Han, Daniel Madrigal Diaz, Jin Xu, Victor Rühle, and Saravan Rajmohan. 2025. Odysseybench: Evaluating llm agents on long-horizon complex office application workflows. arXiv preprint arXiv:2508.09124. — [arXiv:2508.09124](https://arxiv.org/abs/2508.09124) [find](https://scholar.google.com/scholar?q=Weixuan+Wang%2C+Dongge+Han%2C+Daniel+Madrigal+Diaz%2C+Jin+Xu%2C+Victor+R%C3%BChle%2C+and+Saravan+Rajmohan%2E+2025%2E+Odysseybench%3A+Evaluating+llm+agents+on+long%2Dhorizon+complex+office+application+workflows%2E+arXiv+prepri)
40. Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Ed Chi, Quoc Le, and Denny Zhou. 2022. [Chain of thought prompting elicits reasoning in large language models](https://md.research.elamur.ai/free/en/papers/wei-2022-chain-thought-prompting-elicits-reasoning/). arXiv preprint arXiv:2201.11903.
41. Peiran Xu, Sudong Wang, Yao Zhu, Jianing Li, and Yunjian Zhang. 2025. Spatialbench: Benchmarking multimodal large language models for spatial cognition. arXiv preprint arXiv:2511.21471. — [arXiv:2511.21471](https://arxiv.org/abs/2511.21471) [find](https://scholar.google.com/scholar?q=Peiran+Xu%2C+Sudong+Wang%2C+Yao+Zhu%2C+Jianing+Li%2C+and+Yunjian+Zhang%2E+2025%2E+Spatialbench%3A+Benchmarking+multimodal+large+language+models+for+spatial+cognition%2E+arXiv+preprint+arXiv%3A2511%2E21471%2E+%E2%80%94+%5BarXiv%3A2511%2E)
42. An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, and 1 others. 2025a. [Qwen3 technical report](https://md.research.elamur.ai/free/en/papers/yang-2025-qwen3-technical-report/). arXiv preprint arXiv:2505.09388.
43. Jihan Yang, Shusheng Yang, Anjali W Gupta, Rilyn Han, Li Fei-Fei, and Saining Xie. 2025b. [Thinking in space: How multimodal large language models see, remember, and recall spaces](https://md.research.elamur.ai/free/en/papers/yang-2024-thinking-space-multimodal-large-language/). In Proceedings of the Computer Vision and Pattern Recognition Conference, pages 10632–10643.
44. Rui Yang, Hanyang Chen, Junyu Zhang, Mark Zhao, Cheng Qian, Kangrui Wang, Qineng Wang, Teja Venkat Koripella, Marziyeh Movahedi, Manling Li, and 1 others. 2025c. Embodiedbench: Comprehensive benchmarking multi-modal large language models for vision-driven embodied agents. arXiv preprint arXiv:2502.0… — [arXiv:2502.09560](https://arxiv.org/abs/2502.09560) [find](https://scholar.google.com/scholar?q=Rui+Yang%2C+Hanyang+Chen%2C+Junyu+Zhang%2C+Mark+Zhao%2C+Cheng+Qian%2C+Kangrui+Wang%2C+Qineng+Wang%2C+Teja+Venkat+Koripella%2C+Marziyeh+Movahedi%2C+Manling+Li%2C+and+1+others%2E+2025c%2E+Embodiedbench%3A+Comprehensive+benchmark)
45. Xiang Yue, Yuansheng Ni, Kai Zhang, Tianyu Zheng, Ruoqi Liu, Ge Zhang, Samuel Stevens, Dongfu Jiang, Weiming Ren, Yuxuan Sun, and 1 others. 2024. [Mmmu: A massive multi-discipline multimodal understanding and reasoning benchmark for expert agi](https://md.research.elamur.ai/free/en/papers/yue-2023-mmmu-massive-multi-discipline-multimodal/). In Proceedings of the IEEE/CVF Conference on Computer Vi…
46. Jiazhao Zhang, Kunyu Wang, Shaoan Wang, Minghan Li, Haoran Liu, Songlin Wei, Zhongyuan Wang, Zhizheng Zhang, and He Wang. 2024a. Uni-navid: A video-based vision-language-action model for unifying embodied navigation tasks. arXiv preprint arXiv:2412.06224. — [arXiv:2412.06224](https://arxiv.org/abs/2412.06224) [find](https://scholar.google.com/scholar?q=Jiazhao+Zhang%2C+Kunyu+Wang%2C+Shaoan+Wang%2C+Minghan+Li%2C+Haoran+Liu%2C+Songlin+Wei%2C+Zhongyuan+Wang%2C+Zhizheng+Zhang%2C+and+He+Wang%2E+2024a%2E+Uni%2Dnavid%3A+A+video%2Dbased+vision%2Dlanguage%2Daction+model+for+unifying+embo)
47. Jiazhao Zhang, Kunyu Wang, Rongtao Xu, Gengze Zhou, Yicong Hong, Xiaomeng Fang, Qi Wu, Zhizheng Zhang, and Wang He. 2024b. Navid: Video-based vlm plans the next step for vision-and-language navigation. arXiv preprint arXiv:2402.15852. — [arXiv:2402.15852](https://arxiv.org/abs/2402.15852) [find](https://scholar.google.com/scholar?q=Jiazhao+Zhang%2C+Kunyu+Wang%2C+Rongtao+Xu%2C+Gengze+Zhou%2C+Yicong+Hong%2C+Xiaomeng+Fang%2C+Qi+Wu%2C+Zhizheng+Zhang%2C+and+Wang+He%2E+2024b%2E+Navid%3A+Video%2Dbased+vlm+plans+the+next+step+for+vision%2Dand%2Dlanguage+navigation)
48. Duo Zheng, Shijia Huang, Lin Zhao, Yiwu Zhong, and Liwei Wang. 2023. Towards learning a generalist model for embodied navigation. arXiv preprint arXiv:2312.02010. — [arXiv:2312.02010](https://arxiv.org/abs/2312.02010) [find](https://scholar.google.com/scholar?q=Duo+Zheng%2C+Shijia+Huang%2C+Lin+Zhao%2C+Yiwu+Zhong%2C+and+Liwei+Wang%2E+2023%2E+Towards+learning+a+generalist+model+for+embodied+navigation%2E+arXiv+preprint+arXiv%3A2312%2E02010%2E+%E2%80%94+%5BarXiv%3A2312%2E02010%5D%28https%3A%2F%2Farxiv%2Eor)
49. Gengze Zhou, Yicong Hong, Zun Wang, Xin Eric Wang, and Qi Wu. 2025a. Navgpt-2: Unleashing navigational reasoning capability for large vision-language models. In European Conference on Computer Vision, pages 260–278. Springer. [find](https://scholar.google.com/scholar?q=Gengze+Zhou%2C+Yicong+Hong%2C+Zun+Wang%2C+Xin+Eric+Wang%2C+and+Qi+Wu%2E+2025a%2E+Navgpt%2D2%3A+Unleashing+navigational+reasoning+capability+for+large+vision%2Dlanguage+models%2E+In+European+Conference+on+Computer+Vision%2C)
50. Gengze Zhou, Yicong Hong, Zun Wang, Chongyang Zhao, Mohit Bansal, and Qi Wu. 2025b. Same: Learning generic language-guided visual navigation with state-adaptive mixture of experts. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 7794–7807. [find](https://scholar.google.com/scholar?q=Gengze+Zhou%2C+Yicong+Hong%2C+Zun+Wang%2C+Chongyang+Zhao%2C+Mohit+Bansal%2C+and+Qi+Wu%2E+2025b%2E+Same%3A+Learning+generic+language%2Dguided+visual+navigation+with+state%2Dadaptive+mixture+of+experts%2E+In+Proceedings+of+t)
51. Gengze Zhou, Yicong Hong, and Qi Wu. 2024. Navgpt: Explicit reasoning in vision-and-language navigation with large language models. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 7641–7649. [find](https://scholar.google.com/scholar?q=Gengze+Zhou%2C+Yicong+Hong%2C+and+Qi+Wu%2E+2024%2E+Navgpt%3A+Explicit+reasoning+in+vision%2Dand%2Dlanguage+navigation+with+large+language+models%2E+In+Proceedings+of+the+AAAI+Conference+on+Artificial+Intelligence%2C+vo)
52. Jinguo Zhu, Weiyun Wang, Zhe Chen, Zhaoyang Liu, Shenglong Ye, Lixin Gu, Hao Tian, Yuchen Duan, Weijie Su, Jie Shao, and 1 others. 2025. [Internvl3: Exploring advanced training and test-time recipes for open-source multimodal models](https://md.research.elamur.ai/free/en/papers/zhu-2025-internvl3-exploring-advanced-training/). arXiv preprint arXiv:2504.10479.
53. Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. 2022. [React: Synergizing reasoning and acting in language models](https://md.research.elamur.ai/free/en/papers/yao-2022-react-synergizing-reasoning-acting-language/). arXiv preprint arXiv:2210.03629.
54. Weihao Yu, Zhengyuan Yang, Lingfeng Ren, Linjie Li, Jianfeng Wang, Kevin Lin, Chung-Ching Lin, Zicheng Liu, Lijuan Wang, and Xinchao Wang. 2024. Mm-vet v2: A challenging benchmark to evaluate large multimodal models for integrated capabilities. arXiv preprint arXiv:2408.00765. — [arXiv:2408.00765](https://arxiv.org/abs/2408.00765) [find](https://scholar.google.com/scholar?q=Weihao+Yu%2C+Zhengyuan+Yang%2C+Lingfeng+Ren%2C+Linjie+Li%2C+Jianfeng+Wang%2C+Kevin+Lin%2C+Chung%2DChing+Lin%2C+Zicheng+Liu%2C+Lijuan+Wang%2C+and+Xinchao+Wang%2E+2024%2E+Mm%2Dvet+v2%3A+A+challenging+benchmark+to+evaluate+large+mu)

*Updated 2 November 2026*