---
title: "ChartAgent — paper by Kaur et al. (2026) — summary and ideas"
description: "Morgan AI Research) introduce ChartAgent, an agent that reasons directly within the… 17.31 points. 92 references, cited by 3 papers here."
authors: ["Rachneet Kaur", "Nishan Srishankar", "Zhen Zeng", "Sumitra Ganesh"]
year: 2026
venue: "Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)"
source: "https://aclanthology.org/2026.acl-long.843/"
doi: "10.18653/v1/2026.acl-long.843"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/kaur-2026-chartagent-multimodal-agent-visually-grounded/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/kaur-2026-chartagent-multimodal-agent-visually-grounded/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/3/) / ChartAgent: A Multimodal Agent for Visually Grounded Reasoning in Complex Chart Question Answering

# ChartAgent: A Multimodal Agent for Visually Grounded Reasoning in Complex Chart Question Answering

**Authors:** Rachneet Kaur, Nishan Srishankar, Zhen Zeng and [Sumitra Ganesh](https://md.research.elamur.ai/free/en/authors/sumitra-ganesh/) · **Year:** 2026 · **Venue:** [Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)](https://md.research.elamur.ai/free/en/venues/acl/) ([ACL 2026](https://md.research.elamur.ai/free/en/venues/acl/2026/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2026.acl-long.843/) · [PDF](https://aclanthology.org/2026.acl-long.843.pdf) · [DOI 10.18653/v1/2026.acl-long.843](https://doi.org/10.18653/v1/2026.acl-long.843) · [OpenAlex](https://openalex.org/W4414971708)

**Citations:** 2 citations (OpenAlex, 2026-10-14) · 3 in our library, of the 149,439 papers whose bibliographies we read

## Summary
Multimodal LLMs answer chart questions fairly well when values are printed on the chart. They struggle with unannotated charts, where the answer must be estimated from graphics such as bar heights or slice sizes. The authors (J.P. Morgan AI Research) introduce ChartAgent, an agent that reasons directly within the image rather than only in text.

In a ReAct-style loop of thought, action and observation, the base MLLM splits a question into mostly visual subtasks. It then calls tools from a purpose-built library to segment pie slices, isolate bars, find legend markers, locate axis ticks or interpolate values. Each tool also outputs a visualization the agent can inspect, so it can verify intermediate steps and retry with other parameters or tools.

On ChartBench and ChartX, which together cover 40+ chart types, ChartAgent sets a new state of the art. Its biggest gains are on unannotated charts and numeric questions. Because it plugs into different MLLMs, the method is useful wherever precise quantitative chart reading matters, such as finance or data journalism.

## Key points
- Outperforms more than 30 baselines by up to 16.07 absolute percentage points overall and 17.31 points on unannotated, numerically heavy queries.
- The tool library is modular and works at the level of chart components. Examples include `compute_segment_area()`, `get_marker_rgb()`, `get_bar()`, `compute_bar_height()` and `segment_and_mark()`.
- Evaluated on public benchmarks: ChartBench, ChartX and ChartQA-unannotated. Gains hold across chart types, complexity levels and different base MLLMs (plug-and-play).
- Failure analysis shows most errors come from perception/tool mistakes rather than planning. 3D and radar plots remain harder.
- Limitations:
  - Higher latency and cost from iterative tool loops.
  - Single-chart evaluation only.
  - Text-only in-context examples.
  - Hand-designed tools.
  - No standard way to score tool-level accuracy.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

📚 Shelves (5): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works · [👁️ Computer Vision and Pattern Recognition](https://md.research.elamur.ai/free/en/shelves/computer-science/computer-vision-and-pattern-recognition/) · 2,869 works

🧭 Topics (3): [🧮 Reasoning](https://md.research.elamur.ai/free/en/topics/reasoning/) · 1,426 works · [🖼️ Vision & language](https://md.research.elamur.ai/free/en/topics/multimodal/) · 657 works · [❓ Question answering](https://md.research.elamur.ai/free/en/topics/question-answering/) · 441 works

## Cite

Cite as: Rachneet Kaur, Nishan Srishankar, Zhen Zeng and Sumitra Ganesh. 2026. ChartAgent: A Multimodal Agent for Visually Grounded Reasoning in Complex Chart Question Answering. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{kaur2026chartagent,
  title = {{ChartAgent: A Multimodal Agent for Visually Grounded Reasoning in Complex Chart Question Answering}},
  author = {Rachneet Kaur, Nishan Srishankar, Zhen Zeng and Sumitra Ganesh},
  booktitle = {Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2026},
  doi = {10.18653/v1/2026.acl-long.843},
  url = {https://doi.org/10.18653/v1/2026.acl-long.843}
}
```

- [← previous Can LLMs Evaluate What They Cannot Annotate? Revisiting LLM Reliability in Hate Speech Detection](https://md.research.elamur.ai/free/en/papers/piot-2026-can-llms-evaluate-what-they/)
- [next → ChartVerse: Scaling Chart Reasoning via Reliable Programmatic Synthesis from Scratch](https://md.research.elamur.ai/free/en/papers/liu-2026-chartverse-scaling-chart-reasoning-reliable/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/3/)

## References

92 references, 34 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. AutoGPT. [https://github.com/](https://github.com/) Accessed: Significant-Gravitas/Auto-GPT. 2025-05-01. [↗](https://github.com/)
2. CrewAI. [https://github.com/crewAIInc/crewAI](https://github.com/crewAIInc/crewAI). Accessed: 2025-05-01. [↗](https://github.com/crewAIInc/crewAI)
3. EasyOCR. [https://github.com/JaidedAI/](https://github.com/JaidedAI/) EasyOCR. Accessed: 2025-05-01. [↗](https://github.com/JaidedAI/)
4. LangChain. [https://github.com/langchain-ai/](https://github.com/langchain-ai/) langchain. Accessed: 2025-05-01. [↗](https://github.com/langchain-ai/)
5. LangGraph. [https://github.com/langchain-ai/](https://github.com/langchain-ai/) langgraph. Accessed: 2025-05-01. [↗](https://github.com/langchain-ai/)
6. Pravesh Agrawal, Szymon Antoniak, Emma Bou Hanna, Baptiste Bout, Devendra Chaplot, Jessica Chudnovsky, Diogo Costa, Baudouin De Monicault, Saurabh Garg, Theophile Gervet, and 1 others. 2024. Pixtral 12b. arXiv preprint arXiv:2410.07073. [find](https://scholar.google.com/scholar?q=Pravesh+Agrawal%2C+Szymon+Antoniak%2C+Emma+Bou+Hanna%2C+Baptiste+Bout%2C+Devendra+Chaplot%2C+Jessica+Chudnovsky%2C+Diogo+Costa%2C+Baudouin+De+Monicault%2C+Saurabh+Garg%2C+Theophile+Gervet%2C+and+1+others%2E+2024%2E+Pixtral+1)
7. Jaided AI. 2020. Easyocr. [https://github.com/](https://github.com/) JaidedAI/EasyOCR. [↗](https://github.com/)
8. Anthropic. 2024a. The claude 3 model family: Opus, sonnet, haiku. [find](https://scholar.google.com/scholar?q=Anthropic%2E+2024a%2E+The+claude+3+model+family%3A+Opus%2C+sonnet%2C+haiku%2E)
9. Anthropic. 2024b. Model card addendum: Claude 3.5 haiku and upgraded claude 3.5 sonnet. Accessed: 2025-07-09. [find](https://scholar.google.com/scholar?q=Anthropic%2E+2024b%2E+Model+card+addendum%3A+Claude+3%2E5+haiku+and+upgraded+claude+3%2E5+sonnet%2E+Accessed%3A+2025%2D07%2D09%2E)
10. Anthropic. 2025. Claude 3.7 sonnet system card. Accessed: 2025-07-09. [find](https://scholar.google.com/scholar?q=Anthropic%2E+2025%2E+Claude+3%2E7+sonnet+system+card%2E+Accessed%3A+2025%2D07%2D09%2E)
11. Jinze Bai, Shuai Bai, Shusheng Yang, Shijie Wang, Sinan Tan, Peng Wang, Junyang Lin, Chang Zhou, and Jingren Zhou. 2023. [Qwen-vl: A versatile vision-language model for understanding, localization, text reading, and beyond](https://md.research.elamur.ai/free/en/papers/bai-2023-qwen-vl-versatile-vision-language/). arXiv preprint arXiv:2308.12966.
12. Jinyue Chen, Lingyu Kong, Haoran Wei, Chenglong Liu, Zheng Ge, Liang Zhao, Jianjian Sun, Chunrui Han, and Xiangyu Zhang. 2024. Onechart: Purify the chart structural extraction via one auxiliary token. In Proceedings of the 32nd ACM International Conference on Multimedia, pages 147–155. [find](https://scholar.google.com/scholar?q=Jinyue+Chen%2C+Lingyu+Kong%2C+Haoran+Wei%2C+Chenglong+Liu%2C+Zheng+Ge%2C+Liang+Zhao%2C+Jianjian+Sun%2C+Chunrui+Han%2C+and+Xiangyu+Zhang%2E+2024%2E+Onechart%3A+Purify+the+chart+structural+extraction+via+one+auxiliary+token%2E)
13. Jawad Chishtie, Iwona Anna Bielska, Aldo Barrera, Jean-Sebastien Marchand, Muhammad Imran, Syed Farhan Ali Tirmizi, Luke A Turcotte, Sarah Munce, John Shepherd, Arrani Senthinathan, and 1 others. 2022. Interactive visualization applications in population health and health services research: systemat… [find](https://scholar.google.com/scholar?q=Jawad+Chishtie%2C+Iwona+Anna+Bielska%2C+Aldo+Barrera%2C+Jean%2DSebastien+Marchand%2C+Muhammad+Imran%2C+Syed+Farhan+Ali+Tirmizi%2C+Luke+A+Turcotte%2C+Sarah+Munce%2C+John+Shepherd%2C+Arrani+Senthinathan%2C+and+1+others%2E+2022)
14. Wenliang Dai, Junnan Li, Dongxu Li, Anthony Meng Huat Tiong, Junqi Zhao, Weisheng Wang, Boyang Li, Pascale Fung, and Steven Hoi. 2023. [Instructblip: Towards general-purpose vision-language models with instruction tuning](https://md.research.elamur.ai/free/en/papers/dai-2023-instructblip-towards-general-purpose-vision/).
15. Preprint, 2024. [GPT4o-mini](https://md.research.elamur.ai/free/en/papers/dai-2023-instructblip-towards-general-purpose-vision/). [https://openai.com/index/](https://openai.com/index/) arXiv:2305.06500. gpt-4o-mini-advancing-cost-efficient-intelligence/. Team GLM, :, Aohan Zeng, Bin Xu, Bowen Wang, Accessed: 2025-05-01. Chenhui Zhang, Da Yin, Dan Zhang, Diego Ro-Marah Abdin, Sam Ade Jacobs, Ammar Ahmad Awan, jas, Guanyu Feng, H… [↗](https://openai.com/index/)
16. Tesseract OCR. [https://github.com/](https://github.com/) tesseract-ocr/tesseract. Accessed: 2025-05-01. [↗](https://github.com/)
17. Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, and 1 others. 2023. [Gpt-4 technical report](https://md.research.elamur.ai/free/en/papers/openai-2023-gpt-4-technical-report/). arXiv preprint arXiv:2303.08774.
18. Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, and 1 others. 2024. [The llama 3 herd of models](https://md.research.elamur.ai/free/en/papers/grattafiori-2024-llama-3-herd-models/). arXiv preprint arXiv:2407.21783.
19. Tanmay Gupta and Aniruddha Kembhavi. 2023. Visual programming: Compositional visual reasoning without training. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 14953–14962. [find](https://scholar.google.com/scholar?q=Tanmay+Gupta+and+Aniruddha+Kembhavi%2E+2023%2E+Visual+programming%3A+Compositional+visual+reasoning+without+training%2E+In+Proceedings+of+the+IEEE%2FCVF+Conference+on+Computer+Vision+and+Pattern+Recognition%2C+pa)
20. Yucheng Han, Chi Zhang, Xin Chen, Xu Yang, Zhibin Wang, Gang Yu, Bin Fu, and Hanwang Zhang. 2023. Chartllama: A multimodal llm for chart understanding and generation. Preprint, arXiv:2311.16483. [find](https://scholar.google.com/scholar?q=Yucheng+Han%2C+Chi+Zhang%2C+Xin+Chen%2C+Xu+Yang%2C+Zhibin+Wang%2C+Gang+Yu%2C+Bin+Fu%2C+and+Hanwang+Zhang%2E+2023%2E+Chartllama%3A+A+multimodal+llm+for+chart+understanding+and+generation%2E+Preprint%2C+arXiv%3A2311%2E16483%2E)
21. Hongliang He, Wenlin Yao, Kaixin Ma, Wenhao Yu, Yong Dai, Hongming Zhang, Zhenzhong Lan, and Dong Yu. 2024. [Webvoyager: Building an end-to-end web agent with large multimodal models](https://md.research.elamur.ai/free/en/papers/he-2024-webvoyager-building-end-end-web/). arXiv preprint arXiv:2401.13919.
22. Sirui Hong, Mingchen Zhuge, Jonathan Chen, Xiawu Zheng, Yuheng Cheng, Jinlin Wang, Ceyao Zhang, Zili Wang, Steven Ka Shing Yau, Zijuan Lin, and 1 others. 2024. [Metagpt: Meta programming for a multi-agent collaborative framework](https://md.research.elamur.ai/free/en/papers/hong-2023-metagpt-meta-programming-multi/). In The Twelfth International Conference on Learning Representations.
23. Wenyi Hong, Weihan Wang, Qingsong Lv, Jiazheng Xu, Wenmeng Yu, Junhui Ji, Yan Wang, Zihan Wang, Yuxiao Dong, Ming Ding, and Jie Tang. 2023. [Cogagent: A visual language model for gui agents](https://md.research.elamur.ai/free/en/papers/hong-2023-cogagent-visual-language-model/). Preprint, arXiv:2312.08914.
24. Chiori Hori, Motonari Kambara, Komei Sugiura, Kei Ota, Sameer Khurana, Siddarth Jain, Radu Corcodel, Devesh Jha, Diego Romeres, and Jonathan Le Roux. 2025. Interactive robot action replanning using multimodal llm trained from human demonstration videos. In ICASSP 2025-2025 IEEE International Confere… [find](https://scholar.google.com/scholar?q=Chiori+Hori%2C+Motonari+Kambara%2C+Komei+Sugiura%2C+Kei+Ota%2C+Sameer+Khurana%2C+Siddarth+Jain%2C+Radu+Corcodel%2C+Devesh+Jha%2C+Diego+Romeres%2C+and+Jonathan+Le+Roux%2E+2025%2E+Interactive+robot+action+replanning+using+mu)
25. Anwen Hu, Haiyang Xu, Jiabo Ye, Ming Yan, Liang Zhang, Bo Zhang, Chen Li, Ji Zhang, Qin Jin, Fei Huang, and 1 others. 2024a. [mplug-docowl 1.5: Unified structure learning for ocr-free document understanding](https://md.research.elamur.ai/free/en/papers/hu-2024-mplug-docowl-1-5-unified/). arXiv preprint arXiv:2403.12895.
26. Yushi Hu, Weijia Shi, Xingyu Fu, Dan Roth, Mari Ostendorf, Luke Zettlemoyer, Noah A Smith, and Ranjay Krishna. 2024b. Visual sketchpad: Sketching as a visual chain of thought for multimodal language models. In The Thirty-eighth Annual Conference on Neural Information Processing Systems. [find](https://scholar.google.com/scholar?q=Yushi+Hu%2C+Weijia+Shi%2C+Xingyu+Fu%2C+Dan+Roth%2C+Mari+Ostendorf%2C+Luke+Zettlemoyer%2C+Noah+A+Smith%2C+and+Ranjay+Krishna%2E+2024b%2E+Visual+sketchpad%3A+Sketching+as+a+visual+chain+of+thought+for+multimodal+language+m)
27. Muye Huang, Lai Han, Xinyu Zhang, Wenjun Wu, Jie Ma, Lingling Zhang, and Jun Liu. 2024. Evochart: A benchmark and a self-training approach towards real-world chart understanding. arXiv preprint arXiv:2409.01577. — [arXiv:2409.01577](https://arxiv.org/abs/2409.01577) [find](https://scholar.google.com/scholar?q=Muye+Huang%2C+Lai+Han%2C+Xinyu+Zhang%2C+Wenjun+Wu%2C+Jie+Ma%2C+Lingling+Zhang%2C+and+Jun+Liu%2E+2024%2E+Evochart%3A+A+benchmark+and+a+self%2Dtraining+approach+towards+real%2Dworld+chart+understanding%2E+arXiv+preprint+arXiv%3A)
28. Aaron Hurst, Adam Lerer, Adam P Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Ostrow, Akila Welihinda, Alan Hayes, Alec Radford, and 1 others. 2024. [Gpt-4o system card](https://md.research.elamur.ai/free/en/papers/openai-2024-gpt-4o-system-card/). arXiv preprint arXiv:2410.21276.
29. Mohammed Saidul Islam, Raian Rahman, Ahmed Masry, Md Tahmid Rahman Laskar, Mir Tafseer Nayeem, and Enamul Hoque. 2024. Are large vision language models up to the challenge of chart comprehension and reasoning. In Findings of the Association for Computational Linguistics: EMNLP 2024, pages 3334–3368,… [find](https://scholar.google.com/scholar?q=Mohammed+Saidul+Islam%2C+Raian+Rahman%2C+Ahmed+Masry%2C+Md+Tahmid+Rahman+Laskar%2C+Mir+Tafseer+Nayeem%2C+and+Enamul+Hoque%2E+2024%2E+Are+large+vision+language+models+up+to+the+challenge+of+chart+comprehension+and+r)
30. Nidhal Jegham, Marwan Abdelatti, and Abdeltawab Hendawi. 2025. Visual reasoning evaluation of grok, deepseek janus, gemini, qwen, mistral, and chatgpt. arXiv preprint arXiv:2502.16428. — [arXiv:2502.16428](https://arxiv.org/abs/2502.16428) [find](https://scholar.google.com/scholar?q=Nidhal+Jegham%2C+Marwan+Abdelatti%2C+and+Abdeltawab+Hendawi%2E+2025%2E+Visual+reasoning+evaluation+of+grok%2C+deepseek+janus%2C+gemini%2C+qwen%2C+mistral%2C+and+chatgpt%2E+arXiv+preprint+arXiv%3A2502%2E16428%2E+%E2%80%94+%5BarXiv%3A2502%2E1)
31. Carlos E Jimenez, John Yang, Alexander Wettig, Shunyu Yao, Kexin Pei, Ofir Press, and Karthik R Narasimhan. 2024. [Swe-bench: Can language models resolve real-world github issues?](https://md.research.elamur.ai/free/en/papers/jimenez-2024-swe-bench-can-language-models/) In The Twelfth International Conference on Learning Representations.
32. Kushal Kafle, Brian Price, Scott Cohen, and Christopher Kanan. 2018. Dvqa: Understanding data visualizations via question answering. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 5648–5656. [find](https://scholar.google.com/scholar?q=Kushal+Kafle%2C+Brian+Price%2C+Scott+Cohen%2C+and+Christopher+Kanan%2E+2018%2E+Dvqa%3A+Understanding+data+visualizations+via+question+answering%2E+In+Proceedings+of+the+IEEE+conference+on+computer+vision+and+patter)
33. Samira Ebrahimi Kahou, Vincent Michalski, Adam Atkinson, Ákos Kádár, Adam Trischler, and Yoshua Bengio. 2017. Figureqa: An annotated figure dataset for visual reasoning. arXiv preprint arXiv:1710.07300. [find](https://scholar.google.com/scholar?q=Samira+Ebrahimi+Kahou%2C+Vincent+Michalski%2C+Adam+Atkinson%2C+%C3%81kos+K%C3%A1d%C3%A1r%2C+Adam+Trischler%2C+and+Yoshua+Bengio%2E+2017%2E+Figureqa%3A+An+annotated+figure+dataset+for+visual+reasoning%2E+arXiv+preprint+arXiv%3A1710%2E0730)
34. Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer Whitehead, Alexander C Berg, Wan-Yen Lo, and 1 others. 2023. [Segment anything](https://md.research.elamur.ai/free/en/papers/kirillov-2023-segment-anything/). In Proceedings of the IEEE/CVF international conference on computer vision, pages 4015–4026.
35. Jing Yu Koh, Robert Lo, Lawrence Jang, Vikram Duvvur, Ming Lim, Po-Yu Huang, Graham Neubig, Shuyan Zhou, Russ Salakhutdinov, and Daniel Fried. 2024. [VisualWebArena: Evaluating multimodal agents on realistic visual web tasks](https://md.research.elamur.ai/free/en/papers/koh-2024-visualwebarena-evaluating-multimodal-agents-realistic/). In Proceedings of the 62nd Annual Meeting of the Association for Computatio…
36. Bo Li, Yuanhan Zhang, Dong Guo, Renrui Zhang, Feng Li, Hao Zhang, Kaichen Zhang, Peiyuan Zhang, Yanwei Li, Ziwei Liu, and Chunyuan Li. 2024. [Llava-onevision: Easy visual task transfer](https://md.research.elamur.ai/free/en/papers/li-2024-llava-onevision-easy-visual-task/). Preprint, arXiv:2408.03326.
37. F Li, H Zhang, P Sun, X Zou, S Liu, J Yang, C Li, L Zhang, and J Gao. Semantic-sam: Segment and recognize anything at any granularity. arxiv 2023. arXiv preprint arXiv:2307.04767. — [arXiv:2307.04767](https://arxiv.org/abs/2307.04767) [find](https://scholar.google.com/scholar?q=F+Li%2C+H+Zhang%2C+P+Sun%2C+X+Zou%2C+S+Liu%2C+J+Yang%2C+C+Li%2C+L+Zhang%2C+and+J+Gao%2E+Semantic%2Dsam%3A+Segment+and+recognize+anything+at+any+granularity%2E+arxiv+2023%2E+arXiv+preprint+arXiv%3A2307%2E04767%2E+%E2%80%94+%5BarXiv%3A2307%2E04767%5D)
38. Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi. 2023. [Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models](https://md.research.elamur.ai/free/en/papers/li-2023-blip-2-bootstrapping-language-image/). In International conference on machine learning, pages 19730–19742. PMLR.
39. Weifeng Lin, Xinyu Wei, Ruichuan An, Peng Gao, Bocheng Zou, Yulin Luo, Siyuan Huang, Shanghang Zhang, and Hongsheng Li. 2025. Draw-and-understand: Leveraging visual prompts to enable MLLMs to comprehend what you want. In The Thirteenth International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Weifeng+Lin%2C+Xinyu+Wei%2C+Ruichuan+An%2C+Peng+Gao%2C+Bocheng+Zou%2C+Yulin+Luo%2C+Siyuan+Huang%2C+Shanghang+Zhang%2C+and+Hongsheng+Li%2E+2025%2E+Draw%2Dand%2Dunderstand%3A+Leveraging+visual+prompts+to+enable+MLLMs+to+comprehe)
40. Fangyu Liu, Julian Eisenschlos, Francesco Piccinno, Syrine Krichene, Chenxi Pang, Kenton Lee, Mandar Joshi, Wenhu Chen, Nigel Collier, and Yasemin Altun. 2023a. [DePlot: One-shot visual language reasoning by plot-to-table translation](https://md.research.elamur.ai/free/en/papers/liu-2023-deplot-one-shot-visual-language/). In Findings of the Association for Computational Linguistics: ACL…
41. Fangyu Liu, Francesco Piccinno, Syrine Krichene, Chenxi Pang, Kenton Lee, Mandar Joshi, Yasemin Altun, Nigel Collier, and Julian Eisenschlos. 2023b. MatCha: Enhancing visual language pretraining with math reasoning and chart derendering. In Proceedings of the 61st Annual Meeting of the Association f… [find](https://scholar.google.com/scholar?q=Fangyu+Liu%2C+Francesco+Piccinno%2C+Syrine+Krichene%2C+Chenxi+Pang%2C+Kenton+Lee%2C+Mandar+Joshi%2C+Yasemin+Altun%2C+Nigel+Collier%2C+and+Julian+Eisenschlos%2E+2023b%2E+MatCha%3A+Enhancing+visual+language+pretraining+with+)
42. Fuxiao Liu, Xiaoyang Wang, Wenlin Yao, Jianshu Chen, Kaiqiang Song, Sangwoo Cho, Yaser Yacoob, and Dong Yu. 2024a. [Mmc: Advancing multimodal chart understanding with large-scale instruction tuning](https://md.research.elamur.ai/free/en/papers/liu-2024-mmc-advancing-multimodal-chart-understanding/). In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computation…
43. Haotian Liu, Chunyuan Li, Yuheng Li, and Yong Jae Lee. 2023c. [Improved baselines with visual instruction tuning](https://md.research.elamur.ai/free/en/papers/liu-2023-improved-baselines-visual-instruction-tuning/).
44. Haotian Liu, Chunyuan Li, Yuheng Li, Bo Li, Yuanhan Zhang, Sheng Shen, and Yong Jae Lee. 2024b. Llava-next: Improved reasoning, ocr, and world knowledge. [find](https://scholar.google.com/scholar?q=Haotian+Liu%2C+Chunyuan+Li%2C+Yuheng+Li%2C+Bo+Li%2C+Yuanhan+Zhang%2C+Sheng+Shen%2C+and+Yong+Jae+Lee%2E+2024b%2E+Llava%2Dnext%3A+Improved+reasoning%2C+ocr%2C+and+world+knowledge%2E)
45. Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. 2023d. [Visual instruction tuning](https://md.research.elamur.ai/free/en/papers/liu-2023-visual-instruction-tuning/). Advances in neural information processing systems, 36:34892– 34916.
46. Shilong Liu, Hao Cheng, Haotian Liu, Hao Zhang, Feng Li, Tianhe Ren, Xueyan Zou, Jianwei Yang, Hang Su, Jun Zhu, and 1 others. 2025. Llava-plus: Learning to use tools for creating multimodal agents. In European Conference on Computer Vision, pages 126–142. Springer. [find](https://scholar.google.com/scholar?q=Shilong+Liu%2C+Hao+Cheng%2C+Haotian+Liu%2C+Hao+Zhang%2C+Feng+Li%2C+Tianhe+Ren%2C+Xueyan+Zou%2C+Jianwei+Yang%2C+Hang+Su%2C+Jun+Zhu%2C+and+1+others%2E+2025%2E+Llava%2Dplus%3A+Learning+to+use+tools+for+creating+multimodal+agents%2E+I)
47. Junyu Luo, Zekun Li, Jinpeng Wang, and Chin-Yew Lin. 2021. Chartocr: Data extraction from charts images via a deep hybrid framework. In Proceedings of the IEEE/CVF winter conference on applications of computer vision, pages 1917–1925. [find](https://scholar.google.com/scholar?q=Junyu+Luo%2C+Zekun+Li%2C+Jinpeng+Wang%2C+and+Chin%2DYew+Lin%2E+2021%2E+Chartocr%3A+Data+extraction+from+charts+images+via+a+deep+hybrid+framework%2E+In+Proceedings+of+the+IEEE%2FCVF+winter+conference+on+applications+of)
48. Andrés Marafioti, Orr Zohar, Miquel Farré, Merve Noyan, Elie Bakouch, Pedro Cuenca, Cyril Zakka, Loubna Ben Allal, Anton Lozhkov, Nouamane Tazi, Vaibhav Srivastav, Joshua Lochner, Hugo Larcher, Mathieu Morlon, Lewis Tunstall, Leandro von Werra, and Thomas Wolf. 2025. Smolvlm: Redefining small and ef… [find](https://scholar.google.com/scholar?q=Andr%C3%A9s+Marafioti%2C+Orr+Zohar%2C+Miquel+Farr%C3%A9%2C+Merve+Noyan%2C+Elie+Bakouch%2C+Pedro+Cuenca%2C+Cyril+Zakka%2C+Loubna+Ben+Allal%2C+Anton+Lozhkov%2C+Nouamane+Tazi%2C+Vaibhav+Srivastav%2C+Joshua+Lochner%2C+Hugo+Larcher%2C+Mathie)
49. Ahmed Masry, Xuan Long Do, Jia Qing Tan, Shafiq Joty, and Enamul Hoque. 2022. [ChartQA: A benchmark for question answering about charts with visual and logical reasoning](https://md.research.elamur.ai/free/en/papers/masry-2022-chartqa-benchmark-question-answering-about/). In Findings of the Association for Computational Linguistics: ACL 2022, pages 2263– 2279, Dublin, Ireland. Association for Computa…
50. Ahmed Masry, Parsa Kavehzadeh, Xuan Long Do, Enamul Hoque, and Shafiq Joty. 2023. [UniChart: A universal vision-language pretrained model for chart comprehension and reasoning](https://md.research.elamur.ai/free/en/papers/masry-2023-unichart-universal-vision-language-pretrained/). In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 14662–14684, Singapore. Ass…
51. Ahmed Masry, Mehrad Shahmohammadi, Md Rizwan Parvez, Enamul Hoque, and Shafiq Joty. 2024. ChartInstruct: Instruction tuning for chart comprehension and reasoning. In Findings of the Association for Computational Linguistics: ACL 2024, pages 10387–10409, Bangkok, Thailand. Association for Computation… [find](https://scholar.google.com/scholar?q=Ahmed+Masry%2C+Mehrad+Shahmohammadi%2C+Md+Rizwan+Parvez%2C+Enamul+Hoque%2C+and+Shafiq+Joty%2E+2024%2E+ChartInstruct%3A+Instruction+tuning+for+chart+comprehension+and+reasoning%2E+In+Findings+of+the+Association+for+Co)
52. Ahmed Masry, Megh Thakkar, Aayush Bajaj, Aaryaman Kartha, Enamul Hoque, and Shafiq Joty. 2025. ChartGemma: Visual instruction-tuning for chart reasoning in the wild. In Proceedings of the 31st International Conference on Computational Linguistics: Industry Track, pages 625–643, Abu Dhabi, UAE. Assoc… [find](https://scholar.google.com/scholar?q=Ahmed+Masry%2C+Megh+Thakkar%2C+Aayush+Bajaj%2C+Aaryaman+Kartha%2C+Enamul+Hoque%2C+and+Shafiq+Joty%2E+2025%2E+ChartGemma%3A+Visual+instruction%2Dtuning+for+chart+reasoning+in+the+wild%2E+In+Proceedings+of+the+31st+Interna)
53. Nitesh Methani, Pritha Ganguly, Mitesh M Khapra, and Pratyush Kumar. 2020. Plotqa: Reasoning over scientific plots. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 1527–1536. [find](https://scholar.google.com/scholar?q=Nitesh+Methani%2C+Pritha+Ganguly%2C+Mitesh+M+Khapra%2C+and+Pratyush+Kumar%2E+2020%2E+Plotqa%3A+Reasoning+over+scientific+plots%2E+In+Proceedings+of+the+IEEE%2FCVF+Winter+Conference+on+Applications+of+Computer+Vision%2C)
54. MistralAI. 2025. Mistral small 3. [find](https://scholar.google.com/scholar?q=MistralAI%2E+2025%2E+Mistral+small+3%2E)
55. Srija Mukhopadhyay, Adnan Qidwai, Aparna Garimella, Pritika Ramu, Vivek Gupta, and Dan Roth. 2024. Unraveling the truth: Do vlms really understand charts? a deep dive into consistency and robustness. In Findings of the Association for Computational Linguistics: EMNLP 2024, pages 16696–16717. [find](https://scholar.google.com/scholar?q=Srija+Mukhopadhyay%2C+Adnan+Qidwai%2C+Aparna+Garimella%2C+Pritika+Ramu%2C+Vivek+Gupta%2C+and+Dan+Roth%2E+2024%2E+Unraveling+the+truth%3A+Do+vlms+really+understand+charts%3F+a+deep+dive+into+consistency+and+robustness%2E+)
56. Soroush Nasiriany, Fei Xia, Wenhao Yu, Ted Xiao, Jacky Liang, Ishita Dasgupta, Annie Xie, Danny Driess, Ayzaan Wahid, Zhuo Xu, and 1 others. 2024. Pivot: Iterative visual prompting elicits actionable knowledge for vlms. In Forty-first International Conference on Machine Learning. [find](https://scholar.google.com/scholar?q=Soroush+Nasiriany%2C+Fei+Xia%2C+Wenhao+Yu%2C+Ted+Xiao%2C+Jacky+Liang%2C+Ishita+Dasgupta%2C+Annie+Xie%2C+Danny+Driess%2C+Ayzaan+Wahid%2C+Zhuo+Xu%2C+and+1+others%2E+2024%2E+Pivot%3A+Iterative+visual+prompting+elicits+actionable+)
57. OpenAI. 2024. O1 system card. [https://cdn.openai.com/o1-system-card-20241205.pdf](https://cdn.openai.com/o1-system-card-20241205.pdf). Accessed: 2025-10-06. [↗](https://cdn.openai.com/o1-system-card-20241205.pdf)
58. OpenAI. 2025a. Gpt-5 system card. [https://cdn.openai.com/gpt-5-system-card.pdf](https://cdn.openai.com/gpt-5-system-card.pdf). Accessed: 2025-10-06. [↗](https://cdn.openai.com/gpt-5-system-card.pdf)
59. OpenAI. 2025b. Introducing gpt-4.1 in the api. Accessed: 2025-07-09. [find](https://scholar.google.com/scholar?q=OpenAI%2E+2025b%2E+Introducing+gpt%2D4%2E1+in+the+api%2E+Accessed%3A+2025%2D07%2D09%2E)
60. OpenAI. 2025c. Openai o3 and o4-mini system card. Accessed: 2025-07-09. [find](https://scholar.google.com/scholar?q=OpenAI%2E+2025c%2E+Openai+o3+and+o4%2Dmini+system+card%2E+Accessed%3A+2025%2D07%2D09%2E)
61. Yasaman Razeghi, Ishita Dasgupta, Fangyu Liu, Vinay Venkatesh Ramasesh, and Sameer Singh. 2024. Plot twist: Multimodal models don‘t comprehend simple chart details. In Findings of the Association for Computational Linguistics: EMNLP 2024, pages 5922–5937, Miami, Florida, USA. Association for Computa… [find](https://scholar.google.com/scholar?q=Yasaman+Razeghi%2C+Ishita+Dasgupta%2C+Fangyu+Liu%2C+Vinay+Venkatesh+Ramasesh%2C+and+Sameer+Singh%2E+2024%2E+Plot+twist%3A+Multimodal+models+don%E2%80%98t+comprehend+simple+chart+details%2E+In+Findings+of+the+Association+for+)
62. Timo Schick, Jane Dwivedi-Yu, Roberto Dessì, Roberta Raileanu, Maria Lomeli, Eric Hambro, Luke Zettlemoyer, Nicola Cancedda, and Thomas Scialom. 2023. [Toolformer: Language models can teach themselves to use tools](https://md.research.elamur.ai/free/en/papers/schick-2023-toolformer-language-models-can-teach/). Advances in Neural Information Processing Systems, 36:68539–68551.
63. Hao Shao, Shengju Qian, Han Xiao, Guanglu Song, Zhuofan Zong, Letian Wang, Yu Liu, and Hongsheng Li. 2024. Visual cot: Advancing multi-modal language models with a comprehensive dataset and benchmark for chain-of-thought reasoning. In The Thirty-eight Conference on Neural Information Processing Syst… [find](https://scholar.google.com/scholar?q=Hao+Shao%2C+Shengju+Qian%2C+Han+Xiao%2C+Guanglu+Song%2C+Zhuofan+Zong%2C+Letian+Wang%2C+Yu+Liu%2C+and+Hongsheng+Li%2E+2024%2E+Visual+cot%3A+Advancing+multi%2Dmodal+language+models+with+a+comprehensive+dataset+and+benchmark+)
64. Ray Smith. 2007. An overview of the tesseract ocr engine. In ICDAR ’07: Proceedings of the Ninth International Conference on Document Analysis and Recognition, pages 629–633, Washington, DC, USA. IEEE Computer Society. [find](https://scholar.google.com/scholar?q=Ray+Smith%2E+2007%2E+An+overview+of+the+tesseract+ocr+engine%2E+In+ICDAR+%E2%80%9907%3A+Proceedings+of+the+Ninth+International+Conference+on+Document+Analysis+and+Recognition%2C+pages+629%E2%80%93633%2C+Washington%2C+DC%2C+USA%2E+IEEE)
65. Archita Srivastava, Abhas Kumar, Rajesh Kumar, and Prabhakar Srinivasan. 2025. Enhancing financial vqa in vision language models using intermediate structured representations. arXiv preprint arXiv:2501.04675. — [arXiv:2501.04675](https://arxiv.org/abs/2501.04675) [find](https://scholar.google.com/scholar?q=Archita+Srivastava%2C+Abhas+Kumar%2C+Rajesh+Kumar%2C+and+Prabhakar+Srinivasan%2E+2025%2E+Enhancing+financial+vqa+in+vision+language+models+using+intermediate+structured+representations%2E+arXiv+preprint+arXiv%3A250)
66. Zhaochen Su, Peng Xia, Hangyu Guo, Zhenhua Liu, Yan Ma, Xiaoye Qu, Jiaqi Liu, Yanshu Li, Kaide Zeng, Zhengyuan Yang, and 1 others. 2025. Thinking with images for multimodal reasoning: Foundations, methods, and future frontiers. arXiv preprint arXiv:2506.23918. — [arXiv:2506.23918](https://arxiv.org/abs/2506.23918) [find](https://scholar.google.com/scholar?q=Zhaochen+Su%2C+Peng+Xia%2C+Hangyu+Guo%2C+Zhenhua+Liu%2C+Yan+Ma%2C+Xiaoye+Qu%2C+Jiaqi+Liu%2C+Yanshu+Li%2C+Kaide+Zeng%2C+Zhengyuan+Yang%2C+and+1+others%2E+2025%2E+Thinking+with+images+for+multimodal+reasoning%3A+Foundations%2C+met)
67. Dídac Surís, Sachit Menon, and Carl Vondrick. 2023. [Vipergpt: Visual inference via python execution for reasoning](https://md.research.elamur.ai/free/en/papers/suris-2023-vipergpt-visual-inference-python-execution/). In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 11888– 11898.
68. Gemini Team, Rohan Anil, Sebastian Borgeaud, Jean-Baptiste Alayrac, Jiahui Yu, Radu Soricut, Johan Schalkwyk, Andrew M Dai, Anja Hauth, Katie Millican, and 1 others. 2023. [Gemini: a family of highly capable multimodal models](https://md.research.elamur.ai/free/en/papers/gemini-team-2023-gemini-family-highly-capable-multimodal/). arXiv preprint arXiv:2312.11805.
69. Gemini Team, Petko Georgiev, Ving Ian Lei, Ryan Burnell, Libin Bai, Anmol Gulati, Garrett Tanzer, Damien Vincent, Zhufeng Pan, Shibo Wang, Soroosh Mariooryad, Yifan Ding, Xinyang Geng, Fred Alcober, Roy Frostig, Mark Omernick, Lexi Walker, Cosmin Paduraru, Christina Sorokin, and 1118 others. 2024a.… — [Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context](https://md.research.elamur.ai/free/en/papers/gemini-team-2024-gemini-1-5-unlocking-multimodal/)
70. Gemini Team, Petko Georgiev, Ving Ian Lei, Ryan Burnell, Libin Bai, Anmol Gulati, Garrett Tanzer, Damien Vincent, Zhufeng Pan, Shibo Wang, and 1 others. 2024b. [Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context](https://md.research.elamur.ai/free/en/papers/gemini-team-2024-gemini-1-5-unlocking-multimodal/). arXiv preprint arXiv:2403.05530.
71. Gaurav Verma, Rachneet Kaur, Nishan Srishankar, Zhen Zeng, Tucker Balch, and Manuela Veloso. 2025. Adaptagent: Adapting multimodal web agents with few-shot learning from human demonstrations. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long P… [find](https://scholar.google.com/scholar?q=Gaurav+Verma%2C+Rachneet+Kaur%2C+Nishan+Srishankar%2C+Zhen+Zeng%2C+Tucker+Balch%2C+and+Manuela+Veloso%2E+2025%2E+Adaptagent%3A+Adapting+multimodal+web+agents+with+few%2Dshot+learning+from+human+demonstrations%2E+In+Proce)
72. Peng Wang, Shuai Bai, Sinan Tan, Shijie Wang, Zhihao Fan, Jinze Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Yang Fan, Kai Dang, Mengfei Du, Xuancheng Ren, Rui Men, Dayiheng Liu, Chang Zhou, Jingren Zhou, and Junyang Lin. 2024a. [Qwen2-vl: Enhancing vision-language model’s perception of the…](https://md.research.elamur.ai/free/en/papers/wang-2024-qwen2-vl-enhancing-vision/)
73. Weihan Wang, Qingsong Lv, Wenmeng Yu, Wenyi Hong, Ji Qi, Yan Wang, Junhui Ji, Zhuoyi Yang, Lei Zhao, Xixuan Song, Jiazheng Xu, Bin Xu, Juanzi Li, Yuxiao Dong, Ming Ding, and Jie Tang. 2023. [Cogvlm: Visual expert for pretrained language models](https://md.research.elamur.ai/free/en/papers/wang-2023-cogvlm-visual-expert-pretrained/). Preprint, arXiv:2311.03079.
74. Xiaohan Wang, Yuhui Zhang, Orr Zohar, and Serena Yeung-Levy. 2024b. Videoagent: Long-form video understanding with large language model as agent. In European Conference on Computer Vision, pages 58–76. Springer. [find](https://scholar.google.com/scholar?q=Xiaohan+Wang%2C+Yuhui+Zhang%2C+Orr+Zohar%2C+and+Serena+Yeung%2DLevy%2E+2024b%2E+Videoagent%3A+Long%2Dform+video+understanding+with+large+language+model+as+agent%2E+In+European+Conference+on+Computer+Vision%2C+pages+58%E2%80%9376)
75. Zirui Wang, Mengzhou Xia, Luxi He, Howard Chen, Yitao Liu, Richard Zhu, Kaiqu Liang, Xindi Wu, Haotian Liu, Sadhika Malladi, and 1 others. 2024c. Charxiv: Charting gaps in realistic chart understanding in multimodal llms. In The Thirty-eight Conference on Neural Information Processing Systems Datase… [find](https://scholar.google.com/scholar?q=Zirui+Wang%2C+Mengzhou+Xia%2C+Luxi+He%2C+Howard+Chen%2C+Yitao+Liu%2C+Richard+Zhu%2C+Kaiqu+Liang%2C+Xindi+Wu%2C+Haotian+Liu%2C+Sadhika+Malladi%2C+and+1+others%2E+2024c%2E+Charxiv%3A+Charting+gaps+in+realistic+chart+understandin)
76. Webb Wright. 2024. Ai agents with more autonomy than chatbots are coming. some safety experts are worried. [find](https://scholar.google.com/scholar?q=Webb+Wright%2E+2024%2E+Ai+agents+with+more+autonomy+than+chatbots+are+coming%2E+some+safety+experts+are+worried%2E)
77. Chenfei Wu, Shengming Yin, Weizhen Qi, Xiaodong Wang, Zecheng Tang, and Nan Duan. 2023. [Visual chatgpt: Talking, drawing and editing with visual foundation models](https://md.research.elamur.ai/free/en/papers/wu-2023-visual-chatgpt-talking-drawing-editing/). arXiv preprint arXiv:2303.04671.
78. Qingyun Wu, Gagan Bansal, Jieyu Zhang, Yiran Wu, Beibin Li, Erkang Zhu, Li Jiang, Xiaoyun Zhang, Shaokun Zhang, Jiale Liu, Ahmed Hassan Awadallah, Ryen W White, Doug Burger, and Chi Wang. 2024a. Autogen: Enabling next-gen LLM applications via multi-agent conversations. In First Conference on Languag… [find](https://scholar.google.com/scholar?q=Qingyun+Wu%2C+Gagan+Bansal%2C+Jieyu+Zhang%2C+Yiran+Wu%2C+Beibin+Li%2C+Erkang+Zhu%2C+Li+Jiang%2C+Xiaoyun+Zhang%2C+Shaokun+Zhang%2C+Jiale+Liu%2C+Ahmed+Hassan+Awadallah%2C+Ryen+W+White%2C+Doug+Burger%2C+and+Chi+Wang%2E+2024a%2E+Autog)
79. Yifan Wu, Lutao Yan, Leixian Shen, Yunhai Wang, Nan Tang, and Yuyu Luo. 2024b. Chartinsights: Evaluating multimodal large language models for low-level chart question answering. In Findings of the Association for Computational Linguistics: EMNLP 2024, pages 12174–12200. [find](https://scholar.google.com/scholar?q=Yifan+Wu%2C+Lutao+Yan%2C+Leixian+Shen%2C+Yunhai+Wang%2C+Nan+Tang%2C+and+Yuyu+Luo%2E+2024b%2E+Chartinsights%3A+Evaluating+multimodal+large+language+models+for+low%2Dlevel+chart+question+answering%2E+In+Findings+of+the+Ass)
80. Zhiyu Wu, Xiaokang Chen, Zizheng Pan, Xingchao Liu, Wen Liu, Damai Dai, Huazuo Gao, Yiyang Ma, Chengyue Wu, Bingxuan Wang, Zhenda Xie, Yu Wu, Kai Hu, Jiawei Wang, Yaofeng Sun, Yukun Li, Yishi Piao, Kang Guan, Aixin Liu, and 8 others. 2024c. [Deepseek-vl2: Mixture-of-experts vision-language models for…](https://md.research.elamur.ai/free/en/papers/wu-2024-deepseek-vl2-mixture-experts/)
81. Renqiu Xia, Bo Zhang, Hancheng Ye, Xiangchao Yan, Qi Liu, Hongbin Zhou, Zijun Chen, Min Dou, Botian Shi, Junchi Yan, and 1 others. 2024. Chartx & chartvlm: A versatile benchmark and foundation model for complicated chart reasoning. arXiv preprint arXiv:2402.12185. — [arXiv:2402.12185](https://arxiv.org/abs/2402.12185) [find](https://scholar.google.com/scholar?q=Renqiu+Xia%2C+Bo+Zhang%2C+Hancheng+Ye%2C+Xiangchao+Yan%2C+Qi+Liu%2C+Hongbin+Zhou%2C+Zijun+Chen%2C+Min+Dou%2C+Botian+Shi%2C+Junchi+Yan%2C+and+1+others%2E+2024%2E+Chartx+%26+chartvlm%3A+A+versatile+benchmark+and+foundation+model+f)
82. Tianbao Xie, Danyang Zhang, Jixuan Chen, Xiaochuan Li, Siheng Zhao, Ruisheng Cao, Toh Jing Hua, Zhoujun Cheng, Dongchan Shin, Fangyu Lei, and 1 others. 2024. [Osworld: Benchmarking multimodal agents for open-ended tasks in real computer environments](https://md.research.elamur.ai/free/en/papers/xie-2024-osworld-benchmarking-multimodal-agents-open/). arXiv preprint arXiv:2404.07972.
83. Boyuan Zheng, Boyu Gou, Jihyung Kil, Huan Sun, and Yu Su. 2024. [Gpt-4v (ision) is a generalist web agent, if grounded](https://md.research.elamur.ai/free/en/papers/zheng-2024-gpt-4v-ision-generalist-web/). In Forty-first International Conference on Machine Learning.
84. Ziwei Zheng, Michael Yang, Jack Hong, Chenxiao Zhao, Guohai Xu, Le Yang, Chao Shen, and Xing Yu. 2025. Deepeyes: Incentivizing" thinking with images" via reinforcement learning. arXiv preprint arXiv:2505.14362. — [arXiv:2505.14362](https://arxiv.org/abs/2505.14362) [find](https://scholar.google.com/scholar?q=Ziwei+Zheng%2C+Michael+Yang%2C+Jack+Hong%2C+Chenxiao+Zhao%2C+Guohai+Xu%2C+Le+Yang%2C+Chao+Shen%2C+and+Xing+Yu%2E+2025%2E+Deepeyes%3A+Incentivizing%22+thinking+with+images%22+via+reinforcement+learning%2E+arXiv+preprint+arXiv%3A2)
85. Zhuo Zhi, Qiangqiang Wu, Wenbo Li, Yinchuan Li, Kun Shao, Kaiwen Zhou, and 1 others. 2025. Videoagent2: Enhancing the llm-based agent system for long-form video understanding by uncertainty-aware cot. arXiv preprint arXiv:2504.04471. — [arXiv:2504.04471](https://arxiv.org/abs/2504.04471) [find](https://scholar.google.com/scholar?q=Zhuo+Zhi%2C+Qiangqiang+Wu%2C+Wenbo+Li%2C+Yinchuan+Li%2C+Kun+Shao%2C+Kaiwen+Zhou%2C+and+1+others%2E+2025%2E+Videoagent2%3A+Enhancing+the+llm%2Dbased+agent+system+for+long%2Dform+video+understanding+by+uncertainty%2Daware+cot%2E)
86. Jinguo Zhu, Weiyun Wang, Zhe Chen, Zhaoyang Liu, Shenglong Ye, Lixin Gu, Hao Tian, Yuchen Duan, Weijie Su, Jie Shao, Zhangwei Gao, Erfei Cui, Xuehui Wang, Yue Cao, Yangzhou Liu, Xingguang Wei, Hongjie Zhang, Haomin Wang, Weiye Xu, and 32 others. 2025. [Internvl3: Exploring advanced training and test-…](https://md.research.elamur.ai/free/en/papers/zhu-2025-internvl3-exploring-advanced-training/)
87. Zhengzhuo Xu, Sinan Du, Yiyan Qi, Chengjin Xu, Chun Yuan, and Jian Guo. 2023. Chartbench: A benchmark for complex visual reasoning in charts. arXiv preprint arXiv:2312.15915. — [arXiv:2312.15915](https://arxiv.org/abs/2312.15915) [find](https://scholar.google.com/scholar?q=Zhengzhuo+Xu%2C+Sinan+Du%2C+Yiyan+Qi%2C+Chengjin+Xu%2C+Chun+Yuan%2C+and+Jian+Guo%2E+2023%2E+Chartbench%3A+A+benchmark+for+complex+visual+reasoning+in+charts%2E+arXiv+preprint+arXiv%3A2312%2E15915%2E+%E2%80%94+%5BarXiv%3A2312%2E15915%5D%28http)
88. Jianwei Yang, Hao Zhang, Feng Li, Xueyan Zou, Chunyuan Li, and Jianfeng Gao. 2023a. Set-of-mark prompting unleashes extraordinary visual grounding in gpt-4v. arXiv preprint arXiv:2310.11441. [find](https://scholar.google.com/scholar?q=Jianwei+Yang%2C+Hao+Zhang%2C+Feng+Li%2C+Xueyan+Zou%2C+Chunyuan+Li%2C+and+Jianfeng+Gao%2E+2023a%2E+Set%2Dof%2Dmark+prompting+unleashes+extraordinary+visual+grounding+in+gpt%2D4v%2E+arXiv+preprint+arXiv%3A2310%2E11441%2E)
89. Zhengyuan Yang, Linjie Li, Jianfeng Wang, Kevin Lin, Ehsan Azarnasab, Faisal Ahmed, Zicheng Liu, Ce Liu, Michael Zeng, and Lijuan Wang. 2023b. Mm-react: Prompting chatgpt for multimodal reasoning and action. arXiv preprint arXiv:2303.11381. [find](https://scholar.google.com/scholar?q=Zhengyuan+Yang%2C+Linjie+Li%2C+Jianfeng+Wang%2C+Kevin+Lin%2C+Ehsan+Azarnasab%2C+Faisal+Ahmed%2C+Zicheng+Liu%2C+Ce+Liu%2C+Michael+Zeng%2C+and+Lijuan+Wang%2E+2023b%2E+Mm%2Dreact%3A+Prompting+chatgpt+for+multimodal+reasoning+and+)
90. Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. 2023. [React: Synergizing reasoning and acting in language models](https://md.research.elamur.ai/free/en/papers/yao-2022-react-synergizing-reasoning-acting-language/). In International Conference on Learning Representations (ICLR).
91. Jiabo Ye, Haiyang Xu, Haowei Liu, Anwen Hu, Ming Yan, Qi Qian, Ji Zhang, Fei Huang, and Jingren Zhou. 2024. mplug-owl3: Towards long image-sequence understanding in multi-modal large language models. Preprint, arXiv:2408.04840. [find](https://scholar.google.com/scholar?q=Jiabo+Ye%2C+Haiyang+Xu%2C+Haowei+Liu%2C+Anwen+Hu%2C+Ming+Yan%2C+Qi+Qian%2C+Ji+Zhang%2C+Fei+Huang%2C+and+Jingren+Zhou%2E+2024%2E+mplug%2Dowl3%3A+Towards+long+image%2Dsequence+understanding+in+multi%2Dmodal+large+language+models%2E+)
92. Liang Zhang, Anwen Hu, Haiyang Xu, Ming Yan, Yichen Xu, Qin Jin, Ji Zhang, and Fei Huang. 2024. Tinychart: Efficient chart understanding with program-of-thoughts learning and visual token merging. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 1882–1… [find](https://scholar.google.com/scholar?q=Liang+Zhang%2C+Anwen+Hu%2C+Haiyang+Xu%2C+Ming+Yan%2C+Yichen+Xu%2C+Qin+Jin%2C+Ji+Zhang%2C+and+Fei+Huang%2E+2024%2E+Tinychart%3A+Efficient+chart+understanding+with+program%2Dof%2Dthoughts+learning+and+visual+token+merging%2E+In+)

*Updated 1 November 2026*