---
title: "Seeing Beyond Words — paper by Wu et al. (2026)"
description: "The authors present MatVQA, a benchmark of 1,672 multiple-choice questions built from recent arXiv materials… 28% accuracy. 34 references."
authors: ["Sifan Wu", "Huan Zhang", "Yizhan Li", "Farshid Effaty", "Hongyuan Mei", "Amirreza Ataei", "Bang Liu"]
year: 2026
venue: "Findings of the Association for Computational Linguistics: ACL 2026"
source: "https://aclanthology.org/2026.findings-acl.1181/"
doi: "10.18653/v1/2026.findings-acl.1181"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/wu-2026-seeing-beyond-words-matvqa-challenging/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/wu-2026-seeing-beyond-words-matvqa-challenging/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/49/) / Seeing Beyond Words: MatVQA for Challenging Visual-Scientific Reasoning in Materials Science

# Seeing Beyond Words: MatVQA for Challenging Visual-Scientific Reasoning in Materials Science

**Authors:** Sifan Wu, [Huan Zhang](https://md.research.elamur.ai/free/en/authors/huan-zhang/), Yizhan Li, Farshid Effaty, [Hongyuan Mei](https://md.research.elamur.ai/free/en/authors/hongyuan-mei/), Amirreza Ataei and [Bang Liu](https://md.research.elamur.ai/free/en/authors/bang-liu/) · **Year:** 2026 · **Venue:** [Findings of the Association for Computational Linguistics: ACL 2026](https://md.research.elamur.ai/free/en/venues/findings/) ([Findings of the Association for Computational Linguistics 2026](https://md.research.elamur.ai/free/en/venues/findings/2026/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2026.findings-acl.1181/) · [PDF](https://aclanthology.org/2026.findings-acl.1181.pdf) · [DOI 10.18653/v1/2026.findings-acl.1181](https://doi.org/10.18653/v1/2026.findings-acl.1181) · [OpenAlex](https://openalex.org/W4414581165)

**Citations:** 0 citations (OpenAlex, 2026-10-14)

## Summary
Existing materials-science QA benchmarks such as MaScQA and SciQA are mostly text-only. They do not test whether multimodal models can read experimental figures and reason about them at a research level. The authors present MatVQA, a benchmark of 1,672 multiple-choice questions built from recent arXiv materials papers by an automated pipeline called MArxivAgent. The questions cover four structure–property–performance (SPP) task types: quantitative, comparative, causal, and hypothetical variation.

A key step in MArxivAgent is a loop that removes shortcuts. An evaluator agent tries to answer each question from the text alone, then from the text plus the figure caption. If either attempt succeeds, a rewriter revises the question, and a consistency checker makes sure the scientific claim stays intact. The surviving questions can only be solved by close inspection of images such as microscopy or diffraction patterns.

Across 19 open- and closed-source MLLMs, this refinement cut accuracy sharply. Error analysis indicates that missing domain knowledge, not only weak perception, is the main obstacle. The resource matters because it offers a scalable, visually grounded way to evaluate MLLMs for materials discovery.

## Key points
- **Dataset composition:** 1,672 MCQs drawn from 731 unique images, 120 papers, and 32 research areas.
  - Causal SPP: 950 questions
  - Quantitative SPP: 354 questions
  - Hypothetical SPP: 256 questions
  - Comparative SPP: 112 questions
  - Domain experts reviewed half of all items.
- **Effect of shortcut removal:**
  - Removing language shortcuts lowered accuracy by about 12% on average.
  - Removing caption shortcuts led to an extra drop of about 19%.
  - Example: GPT-4o fell from 71.5% to 49.7%, and Claude-3.7-Sonnet from 75.9% to 52.5%.
- **Human baseline:** two materials-science experts averaged 28% accuracy on a random sample of 100 questions.
- **Error audit (40 o1 mistakes):**
  - Material knowledge misunderstanding: 47.5%
  - Visual perception errors: 30.0%
  - Faulty reasoning judgement: 22.5%
- **Availability:** the dataset is released on Hugging Face (trqcbf/matvqa_v2) under a research-only, non-commercial license.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

📚 Shelves (7): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 18,614 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 17,965 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 17,450 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 17,296 works · [👁️ Computer Vision and Pattern Recognition](https://md.research.elamur.ai/free/en/shelves/computer-science/computer-vision-and-pattern-recognition/) · 3,486 works · [🧱 Materials Science](https://md.research.elamur.ai/free/en/shelves/materials-science/) · 31 works · [🧱 Materials Chemistry](https://md.research.elamur.ai/free/en/shelves/materials-science/materials-chemistry/) · 31 works

🧭 Topics (1): [🧮 Reasoning](https://md.research.elamur.ai/free/en/topics/reasoning/) · 1,685 works

## Cite

Cite as: Sifan Wu, Huan Zhang, Yizhan Li, Farshid Effaty, Hongyuan Mei, Amirreza Ataei and Bang Liu. 2026. Seeing Beyond Words: MatVQA for Challenging Visual-Scientific Reasoning in Materials Science. Findings of the Association for Computational Linguistics: ACL 2026.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{wu2026seeing,
  title = {{Seeing Beyond Words: MatVQA for Challenging Visual-Scientific Reasoning in Materials Science}},
  author = {Sifan Wu, Huan Zhang, Yizhan Li, Farshid Effaty, Hongyuan Mei, Amirreza Ataei and Bang Liu},
  booktitle = {Findings of the Association for Computational Linguistics: ACL 2026},
  year = {2026},
  doi = {10.18653/v1/2026.findings-acl.1181},
  url = {https://doi.org/10.18653/v1/2026.findings-acl.1181}
}
```

- [← previous See2Refine: Vision-Language Feedback Improves LLM-Based eHMI Action Designers](https://md.research.elamur.ai/free/en/papers/xia-2026-see2refine-vision-language-feedback-improves/)
- [next → Seeing Isn't Believing: Mitigating Belief Inertia via Active Intervention in Embodied Agents](https://md.research.elamur.ai/free/en/papers/wang-2026-seeing-isn-t-believing-mitigating/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/49/)

## References

34 references, 5 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Google AI. Gemini 2.5: Our most intelligent ai model. [https://blog.google/technology/google-deepmind/](https://blog.google/technology/google-deepmind/) gemini-model-thinking-updates-march-2025/, Mar 2025. [↗](https://blog.google/technology/google-deepmind/)
2. Sören Auer, Dante AC Barone, Cassiano Bartz, Eduardo G Cortes, Mohamad Yaser Jaradeh, Oliver Karras, Manolis Koubarakis, Dmitry Mouromtsev, Dmitrii Pliukhin, Daniil Radyush, et al. The sciqa scientific question answering benchmark for scholarly knowledge. Scientific Reports, 13(1):7240, 2023. [find](https://scholar.google.com/scholar?q=S%C3%B6ren+Auer%2C+Dante+AC+Barone%2C+Cassiano+Bartz%2C+Eduardo+G+Cortes%2C+Mohamad+Yaser+Jaradeh%2C+Oliver+Karras%2C+Manolis+Koubarakis%2C+Dmitry+Mouromtsev%2C+Dmitrii+Pliukhin%2C+Daniil+Radyush%2C+et+al%2E+The+sciqa+scientifi)
3. Suhana Bedi, Yutong Liu, Lucy Orr-Ewing, Dev Dash, Sanmi Koyejo, Alison Callahan, Jason A Fries, Michael Wornow, Akshay Swaminathan, Lisa Soleymani Lehmann, et al. Testing and evaluation of health care applications of large language models: A systematic review. JAMA, 333(4):319, 2024. [find](https://scholar.google.com/scholar?q=Suhana+Bedi%2C+Yutong+Liu%2C+Lucy+Orr%2DEwing%2C+Dev+Dash%2C+Sanmi+Koyejo%2C+Alison+Callahan%2C+Jason+A+Fries%2C+Michael+Wornow%2C+Akshay+Swaminathan%2C+Lisa+Soleymani+Lehmann%2C+et+al%2E+Testing+and+evaluation+of+health+car)
4. Lenz Belzner, Thomas Gabor, and Martin Wirsing. Large language model assisted software engineering: prospects, challenges, and a case study. In International Conference on Bridging the Gap between AI and Reality, pp. 355–374. Springer, 2023. [find](https://scholar.google.com/scholar?q=Lenz+Belzner%2C+Thomas+Gabor%2C+and+Martin+Wirsing%2E+Large+language+model+assisted+software+engineering%3A+prospects%2C+challenges%2C+and+a+case+study%2E+In+International+Conference+on+Bridging+the+Gap+between+AI+)
5. Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. [Language models are few-shot learners](https://md.research.elamur.ai/free/en/papers/brown-2020-language-models-few-shot-learners/). Advances in neural information processing systems, 33:1877–1901, 2020.
6. James Burgess, Jeffrey J Nirschl, Laura Bravo-Sánchez, Alejandro Lozano, Sanket Rajan Gupte, Jesus G Galaz-Montoya, Yuhui Zhang, Yuchang Su, Disha Bhowmik, Zachary Coman, et al. Microvqa: A multimodal reasoning benchmark for microscopy-based scientific research. In Proceedings of the IEEE/CVF Confer… [find](https://scholar.google.com/scholar?q=James+Burgess%2C+Jeffrey+J+Nirschl%2C+Laura+Bravo%2DS%C3%A1nchez%2C+Alejandro+Lozano%2C+Sanket+Rajan+Gupte%2C+Jesus+G+Galaz%2DMontoya%2C+Yuhui+Zhang%2C+Yuchang+Su%2C+Disha+Bhowmik%2C+Zachary+Coman%2C+et+al%2E+Microvqa%3A+A+multimodal)
7. Wei Chow, Jiageng Mao, Boyi Li, Daniel Seita, Vitor Guizilini, and Yue Wang. Physbench: Benchmarking and enhancing vision-language models for physical world understanding, 2025. URL https://arxiv.org/abs/2501.16411. — [arXiv:2501.16411](https://arxiv.org/abs/2501.16411) [↗](https://arxiv.org/abs/2501.16411)
8. Elizabeth Dinella, Satish Chandra, and Petros Maniatis. Crqbench: A benchmark of code reasoning questions, 2024. URL https://arxiv.org/abs/2408. 08453. [↗](https://arxiv.org/abs/2408)
9. Alex Gu, Baptiste Rozière, Hugh Leather, Armando Solar-Lezama, Gabriel Synnaeve, and Sida I. Wang. [Cruxeval: A benchmark for code reasoning, understanding and execution](https://md.research.elamur.ai/free/en/papers/gu-2024-cruxeval-benchmark-code-reasoning-understanding/), 2024. URL https://arxiv.org/abs/2401.03065. [↗](https://arxiv.org/abs/2401.03065)
10. Kehan Guo, Bozhao Nan, Yujun Zhou, Taicheng Guo, Zhichun Guo, Mihir Surve, Zhenwen Liang, Nitesh V Chawla, Olaf Wiest, and Xiangliang Zhang. Can llms solve molecule puzzles? a multimodal benchmark for molecular structure elucidation. Advances in Neural Information Processing Systems, 37:134721–13474… [find](https://scholar.google.com/scholar?q=Kehan+Guo%2C+Bozhao+Nan%2C+Yujun+Zhou%2C+Taicheng+Guo%2C+Zhichun+Guo%2C+Mihir+Surve%2C+Zhenwen+Liang%2C+Nitesh+V+Chawla%2C+Olaf+Wiest%2C+and+Xiangliang+Zhang%2E+Can+llms+solve+molecule+puzzles%3F+a+multimodal+benchmark+for)
11. Yunzhuo Hao, Jiawei Gu, Huichen Will Wang, Linjie Li, Zhengyuan Yang, Lijuan Wang, and Yu Cheng. Can mllms reason in multimodality? emma: An enhanced multimodal reasoning benchmark, 2025. URL https://arxiv.org/abs/2501.05444. — [arXiv:2501.05444](https://arxiv.org/abs/2501.05444) [↗](https://arxiv.org/abs/2501.05444)
12. Noah Hollmann, Samuel Müller, Lennart Purucker, Arjun Krishnakumar, Max Körfer, Shi Bin Hoo, Robin Tibor Schirrmeister, and Frank Hutter. Accurate predictions on small data with a tabular foundation model. Nature, 637(8045):319–326, 2025. [find](https://scholar.google.com/scholar?q=Noah+Hollmann%2C+Samuel+M%C3%BCller%2C+Lennart+Purucker%2C+Arjun+Krishnakumar%2C+Max+K%C3%B6rfer%2C+Shi+Bin+Hoo%2C+Robin+Tibor+Schirrmeister%2C+and+Frank+Hutter%2E+Accurate+predictions+on+small+data+with+a+tabular+foundation+m)
13. Kevin Maik Jablonka, Qianxiang Ai, Alexander Al-Feghali, Shruti Badhwar, Joshua D Bocarsly, Andres M Bran, Stefan Bringuier, L Catherine Brinson, Kamal Choudhary, Defne Circi, et al. 14 examples of how llms can transform materials science and chemistry: a reflection on a large language model hackath… [find](https://scholar.google.com/scholar?q=Kevin+Maik+Jablonka%2C+Qianxiang+Ai%2C+Alexander+Al%2DFeghali%2C+Shruti+Badhwar%2C+Joshua+D+Bocarsly%2C+Andres+M+Bran%2C+Stefan+Bringuier%2C+L+Catherine+Brinson%2C+Kamal+Choudhary%2C+Defne+Circi%2C+et+al%2E+14+examples+of+ho)
14. Naman Jain, King Han, Alex Gu, Wen-Ding Li, Fanjia Yan, Tianjun Zhang, Sida Wang, Armando Solar-Lezama, Koushik Sen, and Ion Stoica. [Livecodebench: Holistic and contamination free evaluation of large language models for code](https://md.research.elamur.ai/free/en/papers/jain-2024-livecodebench-holistic-contamination-free-evaluation/). arXiv preprint arXiv:2403.07974, 2024.
15. Sung Jae Jung, Hajung Kim, and Kyoung Sang Jang. Llm based biological named entity recognition from scientific literature. In 2024 IEEE International Conference on Big Data and Smart Computing (Big-Comp), pp. 433–435. [find](https://scholar.google.com/scholar?q=Sung+Jae+Jung%2C+Hajung+Kim%2C+and+Kyoung+Sang+Jang%2E+Llm+based+biological+named+entity+recognition+from+scientific+literature%2E+In+2024+IEEE+International+Conference+on+Big+Data+and+Smart+Computing+%28Big%2DCo)
16. Ehsan Latif, Ramviyas Parasuraman, and Xiaoming Zhai. Physicsassistant: An llm-powered interactive learning robot for physics lab investigations. In 2024 33rd IEEE International Conference on Robot and Human Interactive Communication (ROMAN), pp. 864–871. [find](https://scholar.google.com/scholar?q=Ehsan+Latif%2C+Ramviyas+Parasuraman%2C+and+Xiaoming+Zhai%2E+Physicsassistant%3A+An+llm%2Dpowered+interactive+learning+robot+for+physics+lab+investigations%2E+In+2024+33rd+IEEE+International+Conference+on+Robot+an)
17. Yu Song, Santiago Miret, Huan Zhang, and Bang Liu. Honeybee: Progressive instruction finetuning of large language models for materials science. arXiv preprint arXiv:2310.08511, 2023. [find](https://scholar.google.com/scholar?q=Yu+Song%2C+Santiago+Miret%2C+Huan+Zhang%2C+and+Bang+Liu%2E+Honeybee%3A+Progressive+instruction+finetuning+of+large+language+models+for+materials+science%2E+arXiv+preprint+arXiv%3A2310%2E08511%2C+2023%2E)
18. Yizhan Li, Florence Cloutier, Sifan Wu, Ali Parviz, Boris Knyazev, Yan Zhang, Glen Berseth, and Bang Liu. Mˆ 4olgen: Multi-agent, multi-stage molecular generation under precise multi-property constraints. arXiv preprint arXiv:2601.10131, 2026. — [arXiv:2601.10131](https://arxiv.org/abs/2601.10131) [find](https://scholar.google.com/scholar?q=Yizhan+Li%2C+Florence+Cloutier%2C+Sifan+Wu%2C+Ali+Parviz%2C+Boris+Knyazev%2C+Yan+Zhang%2C+Glen+Berseth%2C+and+Bang+Liu%2E+M%CB%86+4olgen%3A+Multi%2Dagent%2C+multi%2Dstage+molecular+generation+under+precise+multi%2Dproperty+constrai)
19. Alon Talmor, Ori Yoran, Amnon Catav, Dan Lahav, Yizhong Wang, Akari Asai, Gabriel Ilharco, Hannaneh Hajishirzi, and Jonathan Berant. Multimodalqa: Complex question answering over text, tables and images, 2021. [find](https://scholar.google.com/scholar?q=Alon+Talmor%2C+Ori+Yoran%2C+Amnon+Catav%2C+Dan+Lahav%2C+Yizhong+Wang%2C+Akari+Asai%2C+Gabriel+Ilharco%2C+Hannaneh+Hajishirzi%2C+and+Jonathan+Berant%2E+Multimodalqa%3A+Complex+question+answering+over+text%2C+tables+and+imag)
20. Alejandro Lozano, Jeffrey Nirschl, James Burgess, Sanket Rajan Gupte, Yuhui Zhang, Alyssa Unell, and Serena Yeung-Levy. Micro-bench: A microscopy benchmark for vision-language understanding. In A. Globerson, L. Mackey, D. Belgrave, A. Fan, U. Paquet, J. Tomczak, and C. Zhang (eds.), Advances in Neur… [find](https://scholar.google.com/scholar?q=Alejandro+Lozano%2C+Jeffrey+Nirschl%2C+James+Burgess%2C+Sanket+Rajan+Gupte%2C+Yuhui+Zhang%2C+Alyssa+Unell%2C+and+Serena+Yeung%2DLevy%2E+Micro%2Dbench%3A+A+microscopy+benchmark+for+vision%2Dlanguage+understanding%2E+In+A%2E+Glo)
21. Hongchen Wang, Kangming Li, Scott Ramsay, Yao Fehlis, Edward Kim, and Jason Hattrick-Simpers. Evaluating the performance and robustness of llms in materials science q&a and property predictions. Digital Discovery, 4(6):1612–1624, 2025. [find](https://scholar.google.com/scholar?q=Hongchen+Wang%2C+Kangming+Li%2C+Scott+Ramsay%2C+Yao+Fehlis%2C+Edward+Kim%2C+and+Jason+Hattrick%2DSimpers%2E+Evaluating+the+performance+and+robustness+of+llms+in+materials+science+q%26a+and+property+predictions%2E+Digit)
22. Pan Lu, Hritik Bansal, Tony Xia, Jiacheng Liu, Chunyuan Li, Hannaneh Hajishirzi, Hao Cheng, Kai-Wei Chang, Michel Galley, and Jianfeng Gao. [Mathvista: Evaluating mathematical reasoning of foundation models in visual contexts](https://md.research.elamur.ai/free/en/papers/lu-2023-mathvista-evaluating-mathematical-reasoning-foundation/), 2024. URL https://arxiv.org/abs/2310.02255. [↗](https://arxiv.org/abs/2310.02255)
23. Angel Yanguas-Gil, Matthew T. Dearing, Jeffrey W. Elam, Jessica C. Jones, Sungjoon Kim, Adnan Mohammad, Chi Thang Nguyen, and Bratin Sengupta. Benchmarking large language models for materials synthesis: the case of atomic layer deposition, 2024. URL https://arxiv.org/abs/2412.10477. — [arXiv:2412.10477](https://arxiv.org/abs/2412.10477) [↗](https://arxiv.org/abs/2412.10477)
24. Rachel K Luu and Markus J Buehler. Bioinspiredllm: Conversational large language model for the mechanics of biological and bio-inspired materials. Advanced Science, 11(10):2306724, 2024. [find](https://scholar.google.com/scholar?q=Rachel+K+Luu+and+Markus+J+Buehler%2E+Bioinspiredllm%3A+Conversational+large+language+model+for+the+mechanics+of+biological+and+bio%2Dinspired+materials%2E+Advanced+Science%2C+11%2810%29%3A2306724%2C+2024%2E)
25. Xiang Yue, Tianyu Zheng, Yuansheng Ni, Yubo Wang, Kai Zhang, Shengbang Tong, Yuxuan Sun, Botao Yu, Ge Zhang, Huan Sun, et al. [Mmmu-pro: A more robust multi-discipline multimodal understanding benchmark](https://md.research.elamur.ai/free/en/papers/yue-2024-mmmu-pro-more-robust-multi/). pp. 15134–15186, 2025.
26. Daye Nam, Andrew Macvean, Vincent Hellendoorn, Bogdan Vasilescu, and Brad Myers. Using an llm to help with code understanding. In Proceedings of the IEEE/ACM 46th International Conference on Software Engineering, pp. 1–13, 2024. [find](https://scholar.google.com/scholar?q=Daye+Nam%2C+Andrew+Macvean%2C+Vincent+Hellendoorn%2C+Bogdan+Vasilescu%2C+and+Brad+Myers%2E+Using+an+llm+to+help+with+code+understanding%2E+In+Proceedings+of+the+IEEE%2FACM+46th+International+Conference+on+Software+)
27. Zabir Al Nazi and Wei Peng. Large language models in healthcare and medical domain: A review. In Informatics, volume 11, pp. 57. MDPI, 2024. [find](https://scholar.google.com/scholar?q=Zabir+Al+Nazi+and+Wei+Peng%2E+Large+language+models+in+healthcare+and+medical+domain%3A+A+review%2E+In+Informatics%2C+volume+11%2C+pp%2E+57%2E+MDPI%2C+2024%2E)
28. Dung Manh Nguyen, Thang Chau Phan, Nam Le Hai, Tien-Thong Doan, Nam V Nguyen, Quang Pham, and Nghi DQ Bui. Codemmlu: A multi-task benchmark for assessing code understanding & reasoning capabilities of codellms. In The Thirteenth International Conference on Learning Representations, 2025. xAI. Grok 4… [find](https://scholar.google.com/scholar?q=Dung+Manh+Nguyen%2C+Thang+Chau+Phan%2C+Nam+Le+Hai%2C+Tien%2DThong+Doan%2C+Nam+V+Nguyen%2C+Quang+Pham%2C+and+Nghi+DQ+Bui%2E+Codemmlu%3A+A+multi%2Dtask+benchmark+for+assessing+code+understanding+%26+reasoning+capabilities+of)
29. Mohd Zaki, NM Anoop Krishnan, et al. Mascqa: investigating materials science knowledge of large language models. Digital Discovery, 3(2):313–327, 2024. [find](https://scholar.google.com/scholar?q=Mohd+Zaki%2C+NM+Anoop+Krishnan%2C+et+al%2E+Mascqa%3A+investigating+materials+science+knowledge+of+large+language+models%2E+Digital+Discovery%2C+3%282%29%3A313%E2%80%93327%2C+2024%2E)
30. Di Zhang, Wei Liu, Qian Tan, Jingdan Chen, Hang Yan, Yuliang Yan, Jiatong Li, Weiran Huang, Xiangyu Yue, Wanli Ouyang, et al. Chemllm: A chemical large language model. arXiv preprint arXiv:2402.06852, 2024a. [find](https://scholar.google.com/scholar?q=Di+Zhang%2C+Wei+Liu%2C+Qian+Tan%2C+Jingdan+Chen%2C+Hang+Yan%2C+Yuliang+Yan%2C+Jiatong+Li%2C+Weiran+Huang%2C+Xiangyu+Yue%2C+Wanli+Ouyang%2C+et+al%2E+Chemllm%3A+A+chemical+large+language+model%2E+arXiv+preprint+arXiv%3A2402%2E06852%2C)
31. Huan Zhang, Yu Song, Ziyu Hou, Santiago Miret, and Bang Liu. Honeycomb: A flexible llm-based agent system for materials science. arXiv preprint arXiv:2409.00135, 2024b. [find](https://scholar.google.com/scholar?q=Huan+Zhang%2C+Yu+Song%2C+Ziyu+Hou%2C+Santiago+Miret%2C+and+Bang+Liu%2E+Honeycomb%3A+A+flexible+llm%2Dbased+agent+system+for+materials+science%2E+arXiv+preprint+arXiv%3A2409%2E00135%2C+2024b%2E)
32. Renrui Zhang, Dongzhi Jiang, Yichi Zhang, Haokun Lin, Ziyu Guo, Pengshuo Qiu, Aojun Zhou, Pan Lu, OpenAI. Gpt-4o mini: Advancing cost-efficient intelli-Kai-Wei Chang, Yu Qiao, et al. Mathverse: Does gence, 2024. URL [https://openai.com/index/](https://openai.com/index/) your multi-modal llm truly see the diagrams in vigpt-4o-mi… [↗](https://openai.com/index/)
33. Springer, 2024c. Nima PourNejatian, Anthony B Costa, Cheryl Martin, Chengke Zou, Xingang Guo, Rui Yang, Junyu Zhang, Mona G Flores, Ying Zhang, Tanja Magoc, et al. A Bin Hu, and Huan Zhang. Dynamath: A dynamic study of generative large language model for medical visual benchmark for evaluating mathe… [find](https://scholar.google.com/scholar?q=Springer%2C+2024c%2E+Nima+PourNejatian%2C+Anthony+B+Costa%2C+Cheryl+Martin%2C+Chengke+Zou%2C+Xingang+Guo%2C+Rui+Yang%2C+Junyu+Zhang%2C+Mona+G+Flores%2C+Ying+Zhang%2C+Tanja+Magoc%2C+et+al%2E+A+Bin+Hu%2C+and+Huan+Zhang%2E+Dynamath%3A+)
34. Andre Niyongabo Rubungo, Kangming Li, Jason Hattrick-Simpers, and Adji Bousso Dieng. Llm4matbench: benchmarking large language models for materials property prediction. arXiv preprint arXiv:2411.00177, 2024. — [arXiv:2411.00177](https://arxiv.org/abs/2411.00177) [find](https://scholar.google.com/scholar?q=Andre+Niyongabo+Rubungo%2C+Kangming+Li%2C+Jason+Hattrick%2DSimpers%2C+and+Adji+Bousso+Dieng%2E+Llm4matbench%3A+benchmarking+large+language+models+for+materials+property+prediction%2E+arXiv+preprint+arXiv%3A2411%2E00177)

*Updated 2 November 2026*