---
title: "SciVer — paper by Wang et al. (2025) — summary and ideas"
description: "SciVer is a benchmark that tests whether multimodal foundation models can judge if a scientific claim… 43 references, cited by 8 papers here."
authors: ["Chengye Wang", "Yifei Shen", "Zexi Kuang", "Arman Cohan", "Yilun Zhao"]
year: 2025
venue: "Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)"
source: "https://aclanthology.org/2025.acl-long.420/"
doi: "10.18653/v1/2025.acl-long.420"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/wang-2025-sciver-evaluating-foundation-models-multimodal/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/wang-2025-sciver-evaluating-foundation-models-multimodal/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2025](https://md.research.elamur.ai/free/en/papers/2025/page/8/) / SciVer: Evaluating Foundation Models for Multimodal Scientific Claim Verification

# SciVer: Evaluating Foundation Models for Multimodal Scientific Claim Verification

**Authors:** [Chengye Wang](https://md.research.elamur.ai/free/en/authors/chengye-wang/), [Yifei Shen](https://md.research.elamur.ai/free/en/authors/yifei-shen/), Zexi Kuang, [Arman Cohan](https://md.research.elamur.ai/free/en/authors/arman-cohan/) and [Yilun Zhao](https://md.research.elamur.ai/free/en/authors/yilun-zhao/) · **Year:** 2025 · **Venue:** [Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)](https://md.research.elamur.ai/free/en/venues/acl/) ([ACL 2025](https://md.research.elamur.ai/free/en/venues/acl/2025/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2025.acl-long.420/) · [PDF](https://aclanthology.org/2025.acl-long.420.pdf) · [DOI 10.18653/v1/2025.acl-long.420](https://doi.org/10.18653/v1/2025.acl-long.420) · [OpenAlex](https://openalex.org/W4412945493)

**Citations:** 3 citations (OpenAlex, 2026-10-14) · [8](https://md.research.elamur.ai/free/en/papers/wang-2025-sciver-evaluating-foundation-models-multimodal/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
SciVer is a benchmark that tests whether multimodal foundation models can judge if a scientific claim is entailed or refuted by the paper it comes from. Unlike earlier datasets, which ground claims in plain text, a single table or a single chart, SciVer asks models to search a paper's paragraphs, charts and table screenshots together. Domain experts wrote 3,000 claims drawn from 1,113 arXiv computer science papers. They grouped the claims into four reasoning types and marked the supporting evidence for each one. The authors tested 21 open and proprietary models. Top systems come close to human accuracy on simple lookups but lag well behind experts on harder reasoning. A manual error analysis found that the most common failure is missing the relevant evidence. Experiments with retrieval-augmented generation show that better retrieval, or having an LLM filter the evidence, improves verification accuracy. The resource gives a controlled way to measure how well models reason across modalities in real research papers.

## Key points
- **Data:** 3,000 expert-annotated examples from 1,113 CS papers, split into four subsets: direct, parallel, sequential and analytical reasoning. Each example carries gold supporting evidence, and the label set is {entailed, refuted}.
- **Model results:** 21 models were evaluated, including o4-mini, Gemini-2.5-Flash, GPT-4.1, Llama-3.2-Vision and Qwen2.5-VL. On analytical reasoning, GPT-4.1 scores 70.8% against 90.0% for human experts. Accuracy falls as the number of required evidence pieces grows.
- **Error analysis:** The main failure types are:
  - missing relevant evidence (32%)
  - misreading charts or tables (21%)
  - broken multi-step reasoning (17%)
  - over-reliance on text (12%)
  - domain misconceptions (10%)
- **RAG experiments:** Tables and charts were indexed as their captions plus GPT-4o-generated descriptions, and three retrievers were compared: BM25, Contriever and OpenAI text-embedding-3-large. OpenAI embeddings retrieved best, with 81.0 Recall@5. With oracle evidence, Qwen2.5-VL-72B rises from 70.2 to 75.3. The paper's text attributes the 75.3 figure to the OpenAI retriever, but its own Table 4 lists 72.9 for that setting.
- **Evidence filtering and limits:** Letting an LLM filter evidence raised GPT-4o-mini from 63.8 to 67.5 and Qwen2.5-VL-72B from 70.2 to 74.4. The benchmark covers only CS papers and does not include equations or experimental images. Data and code are released as chengyewang/SciVer and QDRhhhh/SciVer.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Yale University](https://md.research.elamur.ai/free/en/universities/yale-university/) — University, New Haven, United States · 95 works · 12,765 citations

📚 Shelves (7): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works · [👁️ Computer Vision and Pattern Recognition](https://md.research.elamur.ai/free/en/shelves/computer-science/computer-vision-and-pattern-recognition/) · 2,869 works · [🤝 Social Sciences](https://md.research.elamur.ai/free/en/shelves/social-sciences/) · 1,772 works · [🤝 Sociology and Political Science](https://md.research.elamur.ai/free/en/shelves/social-sciences/sociology-and-political-science/) · 577 works

🧭 Topics (2): [🧠 Language models & pretraining](https://md.research.elamur.ai/free/en/topics/language-models/) · 3,320 works · [🖼️ Vision & language](https://md.research.elamur.ai/free/en/topics/multimodal/) · 657 works

## Cite

Cite as: Chengye Wang, Yifei Shen, Zexi Kuang, Arman Cohan and Yilun Zhao. 2025. SciVer: Evaluating Foundation Models for Multimodal Scientific Claim Verification. Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{wang2025sciver,
  title = {{SciVer: Evaluating Foundation Models for Multimodal Scientific Claim Verification}},
  author = {Chengye Wang, Yifei Shen, Zexi Kuang, Arman Cohan and Yilun Zhao},
  booktitle = {Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2025},
  doi = {10.18653/v1/2025.acl-long.420},
  url = {https://doi.org/10.18653/v1/2025.acl-long.420}
}
```

- [← previous SciArena: An Open Evaluation Platform for Non-Verifiable Scientific Literature-Grounded Tasks](https://md.research.elamur.ai/free/en/papers/zhao-2025-sciarena-open-evaluation-platform-non/)
- [next → Second Language (Arabic) Acquisition of LLMs via Progressive Vocabulary Expansion](https://md.research.elamur.ai/free/en/papers/zhu-2025-second-language-arabic-acquisition-llms/)
- [↑ back to papers of 2025](https://md.research.elamur.ai/free/en/papers/2025/page/8/)

## References

43 references, 25 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Pravesh Agrawal, Szymon Antoniak, Emma Bou Hanna, Baptiste Bout, Devendra Chaplot, Jessica Chudnovsky, Diogo Costa, Baudouin De Monicault, Saurabh Garg, Theophile Gervet, Soham Ghosh, Amélie Héliou, Paul Jacob, Albert Q. Jiang, Kartik Khandelwal, Timothée Lacroix, Guillaume Lample, Diego Las Casas,… [find](https://scholar.google.com/scholar?q=Pravesh+Agrawal%2C+Szymon+Antoniak%2C+Emma+Bou+Hanna%2C+Baptiste+Bout%2C+Devendra+Chaplot%2C+Jessica+Chudnovsky%2C+Diogo+Costa%2C+Baudouin+De+Monicault%2C+Saurabh+Garg%2C+Theophile+Gervet%2C+Soham+Ghosh%2C+Am%C3%A9lie+H%C3%A9liou%2C+P)
2. Mubashara Akhtar, Nikesh Subedi, Vivek Gupta, Sahar Tahmasebi, Oana Cocarascu, and Elena Simperl. 2024. ChartCheck: Explainable fact-checking over real-world chart images. In Findings of the Association for Computational Linguistics: ACL 2024, pages 13921–13937, Bangkok, Thailand. Association for Co… [find](https://scholar.google.com/scholar?q=Mubashara+Akhtar%2C+Nikesh+Subedi%2C+Vivek+Gupta%2C+Sahar+Tahmasebi%2C+Oana+Cocarascu%2C+and+Elena+Simperl%2E+2024%2E+ChartCheck%3A+Explainable+fact%2Dchecking+over+real%2Dworld+chart+images%2E+In+Findings+of+the+Associati)
3. Rami Aly, Zhijiang Guo, Michael Sejr Schlichtkrull, James Thorne, Andreas Vlachos, Christos Christodoulopoulos, Oana Cocarascu, and Arpit Mittal. 2021. [The fact extraction and VERification over unstructured and structured information (FEVEROUS) shared task](https://md.research.elamur.ai/free/en/papers/aly-2021-fact-extraction-verification-over-unstructured/). In Proceedings of the Fourth Workshop on F…
4. Akari Asai, Jacqueline He, Rulin Shao, Weijia Shi, Amanpreet Singh, Joseph Chee Chang, Kyle Lo, Luca Soldaini, Sergey Feldman, Mike D’arcy, David Wadden, Matt Latzke, Minyang Tian, Pan Ji, Shengyan Liu, Hao Tong, Bohao Wu, Yanyu Xiong, Luke Zettlemoyer, Graham Neubig, Dan Weld, Doug Downey, Wen tau… [find](https://scholar.google.com/scholar?q=Akari+Asai%2C+Jacqueline+He%2C+Rulin+Shao%2C+Weijia+Shi%2C+Amanpreet+Singh%2C+Joseph+Chee+Chang%2C+Kyle+Lo%2C+Luca+Soldaini%2C+Sergey+Feldman%2C+Mike+D%E2%80%99arcy%2C+David+Wadden%2C+Matt+Latzke%2C+Minyang+Tian%2C+Pan+Ji%2C+Shengyan+Li)
5. Jinze Bai, Shuai Bai, Shusheng Yang, Shijie Wang, Sinan Tan, Peng Wang, Junyang Lin, Chang Zhou, and Jingren Zhou. 2023. [Qwen-vl: A versatile vision-language model for understanding, localization, text reading, and beyond](https://md.research.elamur.ai/free/en/papers/bai-2023-qwen-vl-versatile-vision-language/). arXiv preprint arXiv:2308.12966.
6. Wenhu Chen, Hongmin Wang, Jianshu Chen, Yunkai Zhang, Hong Wang, Shiyang Li, Xiyou Zhou, and William Yang Wang. 2020. [Tabfact: A large-scale dataset for table-based fact verification](https://md.research.elamur.ai/free/en/papers/chen-2019-tabfact-large-scale-dataset-table/). In International Conference on Learning Representations.
7. Zhe Chen, Weiyun Wang, Yue Cao, Yangzhou Liu, Zhangwei Gao, Erfei Cui, Jinguo Zhu, Shenglong Ye, Hao Tian, Zhaoyang Liu, et al. 2024a. [Expanding performance boundaries of open-source multimodal models with model, data, and test-time scaling](https://md.research.elamur.ai/free/en/papers/chen-2024-expanding-performance-boundaries-open/). arXiv preprint arXiv:2412.05271.
8. Zhe Chen, Weiyun Wang, Hao Tian, Shenglong Ye, Zhangwei Gao, Erfei Cui, Wenwen Tong, Kongzhi Hu, Jiapeng Luo, Zheng Ma, et al. 2024b. [How far are we to gpt-4v? closing the gap to commercial multimodal models with open-source suites](https://md.research.elamur.ai/free/en/papers/chen-2024-far-gpt-4v-closing/). arXiv preprint arXiv:2404.16821.
9. Zhe Chen, Jiannan Wu, Wenhai Wang, Weijie Su, Guo Chen, Sen Xing, Muyan Zhong, Qinglong Zhang, Xizhou Zhu, Lewei Lu, Bin Li, Ping Luo, Tong Lu, Yu Qiao, and Jifeng Dai. 2023. [Internvl: Scaling up vision foundation models and aligning for generic visual-linguistic tasks](https://md.research.elamur.ai/free/en/papers/chen-2023-internvl-scaling-up-vision/). arXiv preprint arXiv:2312.142…
10. Pradeep Dasigi, Kyle Lo, Iz Beltagy, Arman Cohan, Noah A. Smith, and Matt Gardner. 2021. [A dataset of information-seeking questions and answers anchored in research papers](https://md.research.elamur.ai/free/en/papers/dasigi-2021-dataset-information-seeking-questions-answers/). In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Lan…
11. Naihao Deng, Zhenjie Sun, Ruiqi He, Aman Sikka, Yulong Chen, Lin Ma, Yue Zhang, and Rada Mihalcea. 2024. Tables as texts or images: Evaluating the table reasoning ability of LLMs and MLLMs. In Findings of the Association for Computational Linguistics: ACL 2024, pages 407–426, Bangkok, Thailand. Asso… [find](https://scholar.google.com/scholar?q=Naihao+Deng%2C+Zhenjie+Sun%2C+Ruiqi+He%2C+Aman+Sikka%2C+Yulong+Chen%2C+Lin+Ma%2C+Yue+Zhang%2C+and+Rada+Mihalcea%2E+2024%2E+Tables+as+texts+or+images%3A+Evaluating+the+table+reasoning+ability+of+LLMs+and+MLLMs%2E+In+Finding)
12. Max Glockner, Ieva Staliūnaitė, James Thorne, Gisela Vallejo, Andreas Vlachos, and Iryna Gurevych. 2024. [AmbiFC: Fact-checking ambiguous claims with evidence](https://md.research.elamur.ai/free/en/papers/glockner-2024-ambifc-fact-checking-ambiguous-claims/). Transactions of the Association for Computational Linguistics, 12:1–18.
13. Google. 2024. [Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context](https://md.research.elamur.ai/free/en/papers/gemini-team-2024-gemini-1-5-unlocking-multimodal/).
14. Google. 2025. Gemini 2.5 flash. [find](https://scholar.google.com/scholar?q=Google%2E+2025%2E+Gemini+2%2E5+flash%2E)
15. Vivek Gupta, Maitrey Mehta, Pegah Nokhiz, and Vivek Srikumar. 2020. [INFOTABS: Inference on tables as semi-structured data](https://md.research.elamur.ai/free/en/papers/gupta-2020-infotabs-inference-tables-as-semi/). In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 2309–2324, Online. Association for Computational Linguistics.
16. Gautier Izacard, Mathilde Caron, Lucas Hosseini, Sebastian Riedel, Piotr Bojanowski, Armand Joulin, and Edouard Grave. 2021. [Unsupervised dense information retrieval with contrastive learning](https://md.research.elamur.ai/free/en/papers/izacard-2021-unsupervised-dense-information-retrieval-contrastive/).
17. Ryo Kamoi, Tanya Goyal, Juan Diego Rodriguez, and Greg Durrett. 2023. [WiCE: Real-world entailment for claims in Wikipedia](https://md.research.elamur.ai/free/en/papers/kamoi-2023-wice-real-world-entailment-claims/). In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 7561–7583, Singapore. Association for Computational Linguistics.
18. Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph E. Gonzalez, Hao Zhang, and Ion Stoica. 2023. [Efficient memory management for large language model serving with pagedattention](https://md.research.elamur.ai/free/en/papers/kwon-2023-efficient-memory-management-large-language/). In Proceedings of the ACM SIGOPS 29th Symposium on Operating Systems Principles.
19. Yoonjoo Lee, Kyungjae Lee, Sunghyun Park, Dasol Hwang, Jaehyeon Kim, Hong-In Lee, and Moontae Lee. 2023. QASA: Advanced question answering on scientific articles. In Proceedings of the 40th International Conference on Machine Learning, volume 202 of Proceedings of Machine Learning Research, pages 19… [find](https://scholar.google.com/scholar?q=Yoonjoo+Lee%2C+Kyungjae+Lee%2C+Sunghyun+Park%2C+Dasol+Hwang%2C+Jaehyeon+Kim%2C+Hong%2DIn+Lee%2C+and+Moontae+Lee%2E+2023%2E+QASA%3A+Advanced+question+answering+on+scientific+articles%2E+In+Proceedings+of+the+40th+Internatio)
20. Bo Li, Yuanhan Zhang, Dong Guo, Renrui Zhang, Feng Li, Hao Zhang, Kaichen Zhang, Peiyuan Zhang, Yanwei Li, Ziwei Liu, and Chunyuan Li. 2024a. [Llava-onevision: Easy visual task transfer](https://md.research.elamur.ai/free/en/papers/li-2024-llava-onevision-easy-visual-task/).
21. Chuhan Li, Ziyao Shangguan, Yilun Zhao, Deyuan Li, Yixin Liu, and Arman Cohan. 2024b. M3SciQA: A multi-modal multi-document scientific QA benchmark for evaluating foundation models. In Findings of the Association for Computational Linguistics: EMNLP 2024, pages 15419–15446, Miami, Florida, USA. Asso… [find](https://scholar.google.com/scholar?q=Chuhan+Li%2C+Ziyao+Shangguan%2C+Yilun+Zhao%2C+Deyuan+Li%2C+Yixin+Liu%2C+and+Arman+Cohan%2E+2024b%2E+M3SciQA%3A+A+multi%2Dmodal+multi%2Ddocument+scientific+QA+benchmark+for+evaluating+foundation+models%2E+In+Findings+of+the)
22. Lei Li, Yuqi Wang, Runxin Xu, Peiyi Wang, Xiachong Feng, Lingpeng Kong, and Qi Liu. 2024c. [Multimodal ArXiv: A dataset for improving scientific comprehension of large vision-language models](https://md.research.elamur.ai/free/en/papers/li-2024-multimodal-arxiv-dataset-improving-scientific/). In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Pa…
23. Zekun Li, Xianjun Yang, Kyuri Choi, Wanrong Zhu, Ryan Hsieh, HyeonJung Kim, Jin Hyuk Lim, Sungyoung Ji, Byungju Lee, Xifeng Yan, Linda Ruth Petzold, Stephen D. Wilson, Woosang Lim, and William Yang Wang. 2024d. [Mmsci: A dataset for graduate-level multi-discipline multimodal scientific understanding](https://md.research.elamur.ai/free/en/papers/li-2024-mmsci-dataset-graduate-level-multi/).
24. Xinyuan Lu, Liangming Pan, Qian Liu, Preslav Nakov, and Min-Yen Kan. 2023. SCITAB: A challenging benchmark for compositional reasoning and claim verification on scientific tables. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 7787–7813, Singapore. A… [find](https://scholar.google.com/scholar?q=Xinyuan+Lu%2C+Liangming+Pan%2C+Qian+Liu%2C+Preslav+Nakov%2C+and+Min%2DYen+Kan%2E+2023%2E+SCITAB%3A+A+challenging+benchmark+for+compositional+reasoning+and+claim+verification+on+scientific+tables%2E+In+Proceedings+of+th)
25. Meta. 2024. [The llama 3 herd of models](https://md.research.elamur.ai/free/en/papers/grattafiori-2024-llama-3-herd-models/).
26. Microsoft, :, Abdelrahman Abouelenin, Atabak Ashfaq, Adam Atkinson, Hany Awadalla, Nguyen Bach, Jianmin Bao, Alon Benhaim, Martin Cai, Vishrav Chaudhary, Congcong Chen, Dong Chen, Dongdong Chen, Junkun Chen, Weizhu Chen, Yen-Chun Chen, Yi ling Chen, Qi Dai, Xiyang Dai, Ruchao Fan, Mei Gao, Min Gao,… — [Phi-4-Mini Technical Report: Compact yet Powerful Multimodal Language Models via Mixture-of-LoRAs](https://md.research.elamur.ai/free/en/papers/microsoft-2025-phi-4-mini-technical-report/)
27. Microsoft. 2024. [Phi-3 technical report: A highly capable language model locally on your phone](https://md.research.elamur.ai/free/en/papers/abdin-2024-phi-3-technical-report-highly/).
28. Mistral AI. 2025. Mistral-small-3.1-24b-instruct-2503. [https://huggingface.co/mistralai/Mistral-Small-3](https://huggingface.co/mistralai/Mistral-Small-3). 1-24B-Instruct-2503. Apache 2.0 License. [↗](https://huggingface.co/mistralai/Mistral-Small-3)
29. OpenAI. 2024. Hello gpt-4o. [find](https://scholar.google.com/scholar?q=OpenAI%2E+2024%2E+Hello+gpt%2D4o%2E)
30. OpenAI. 2025a. Addendum to openai o3 and o4-mini system card: Openai o3 operator. [find](https://scholar.google.com/scholar?q=OpenAI%2E+2025a%2E+Addendum+to+openai+o3+and+o4%2Dmini+system+card%3A+Openai+o3+operator%2E)
31. OpenAI. 2025b. Introducing gpt-4.1 in the api. [find](https://scholar.google.com/scholar?q=OpenAI%2E+2025b%2E+Introducing+gpt%2D4%2E1+in+the+api%2E)
32. Aman Rangapur, Haoran Wang, Ling Jian, and Kai Shu. 2024. Fin-fact: A benchmark dataset for multimodal financial fact checking and explanation generation. [find](https://scholar.google.com/scholar?q=Aman+Rangapur%2C+Haoran+Wang%2C+Ling+Jian%2C+and+Kai+Shu%2E+2024%2E+Fin%2Dfact%3A+A+benchmark+dataset+for+multimodal+financial+fact+checking+and+explanation+generation%2E)
33. Michael D. Skarlinski, Sam Cox, Jon M. Laurent, James D. Braza, Michaela Hinks, Michael J. Hammerling, Manvitha Ponnapati, Samuel G. Rodriques, and Andrew D. White. 2024. Language agents achieve superhuman synthesis of scientific knowledge. [find](https://scholar.google.com/scholar?q=Michael+D%2E+Skarlinski%2C+Sam+Cox%2C+Jon+M%2E+Laurent%2C+James+D%2E+Braza%2C+Michaela+Hinks%2C+Michael+J%2E+Hammerling%2C+Manvitha+Ponnapati%2C+Samuel+G%2E+Rodriques%2C+and+Andrew+D%2E+White%2E+2024%2E+Language+agents+achieve+super)
34. James Thorne, Andreas Vlachos, Christos Christodoulopoulos, and Arpit Mittal. 2018. [FEVER: a large-scale dataset for fact extraction and VERification](https://md.research.elamur.ai/free/en/papers/thorne-2018-fever-large-scale-dataset-fact/). In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Vo…
35. Andreas Vlachos and Sebastian Riedel. 2014. [Fact checking: Task definition and dataset construction](https://md.research.elamur.ai/free/en/papers/vlachos-2014-fact-checking-task-definition-dataset/). In Proceedings of the ACL 2014 Workshop on Language Technologies and Computational Social Science, pages 18–22, Baltimore, MD, USA. Association for Computational Linguistics.
36. Yilun Zhao, Yitao Long, Tintin Jiang, Chengye Wang, Weiyuan Chen, Hongjun Liu, Xiangru Tang, Yiming Zhang, Chen Zhao, and Arman Cohan. 2024. FinD-Ver: Explainable claim verification over long and hybrid-content financial documents. In Proceedings of the 2024 Conference on Empirical Methods in Natura… [find](https://scholar.google.com/scholar?q=Yilun+Zhao%2C+Yitao+Long%2C+Tintin+Jiang%2C+Chengye+Wang%2C+Weiyuan+Chen%2C+Hongjun+Liu%2C+Xiangru+Tang%2C+Yiming+Zhang%2C+Chen+Zhao%2C+and+Arman+Cohan%2E+2024%2E+FinD%2DVer%3A+Explainable+claim+verification+over+long+and+hybr)
37. Mingyu Zheng, Xinwei Feng, Qingyi Si, Qiaoqiao She, Zheng Lin, Wenbin Jiang, and Weiping Wang. 2024. [Multimodal table understanding](https://md.research.elamur.ai/free/en/papers/zheng-2024-multimodal-table-understanding/). In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 9102–9124, Bangkok, Thailand. Association for…
38. David Wadden, Shanchuan Lin, Kyle Lo, Lucy Lu Wang, Madeleine van Zuylen, Arman Cohan, and Hannaneh Hajishirzi. 2020. [Fact or fiction: Verifying scientific claims](https://md.research.elamur.ai/free/en/papers/wadden-2020-fact-or-fiction-verifying-scientific/). In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 7534–7550, Online. Association…
39. David Wadden, Kyle Lo, Bailey Kuehl, Arman Cohan, Iz Beltagy, Lucy Lu Wang, and Hannaneh Hajishirzi. 2022. [SciFact-open: Towards open-domain scientific claim verification](https://md.research.elamur.ai/free/en/papers/wadden-2022-scifact-open-open-domain-scientific/). In Findings of the Association for Computational Linguistics: EMNLP 2022, pages 4719–4734, Abu Dhabi, United Arab Emirates. Asso…
40. Peng Wang, Shuai Bai, Sinan Tan, Shijie Wang, Zhihao Fan, Jinze Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Yang Fan, Kai Dang, Mengfei Du, Xuancheng Ren, Rui Men, Dayiheng Liu, Chang Zhou, Jingren Zhou, and Junyang Lin. 2024a. [Qwen2-vl: Enhancing vision-language model’s perception of the…](https://md.research.elamur.ai/free/en/papers/wang-2024-qwen2-vl-enhancing-vision/)
41. Zirui Wang, Mengzhou Xia, Luxi He, Howard Chen, Yitao Liu, Richard Zhu, Kaiqu Liang, Xindi Wu, Haotian Liu, Sadhika Malladi, Alexis Chevalier, Sanjeev Arora, and Danqi Chen. 2024b. Charxiv: Charting gaps in realistic chart understanding in multimodal llms. CoRR, abs/2406.18521. [find](https://scholar.google.com/scholar?q=Zirui+Wang%2C+Mengzhou+Xia%2C+Luxi+He%2C+Howard+Chen%2C+Yitao+Liu%2C+Richard+Zhu%2C+Kaiqu+Liang%2C+Xindi+Wu%2C+Haotian+Liu%2C+Sadhika+Malladi%2C+Alexis+Chevalier%2C+Sanjeev+Arora%2C+and+Danqi+Chen%2E+2024b%2E+Charxiv%3A+Charting+g)
42. Xiang Yue, Tianyu Zheng, Yuansheng Ni, Yubo Wang, Kai Zhang, Shengbang Tong, Yuxuan Sun, Botao Yu, Ge Zhang, Huan Sun, Yu Su, Wenhu Chen, and Graham Neubig. 2024. Mmmu-pro: A more robust multi-discipline multimodal understanding benchmark. [find](https://scholar.google.com/scholar?q=Xiang+Yue%2C+Tianyu+Zheng%2C+Yuansheng+Ni%2C+Yubo+Wang%2C+Kai+Zhang%2C+Shengbang+Tong%2C+Yuxuan+Sun%2C+Botao+Yu%2C+Ge+Zhang%2C+Huan+Sun%2C+Yu+Su%2C+Wenhu+Chen%2C+and+Graham+Neubig%2E+2024%2E+Mmmu%2Dpro%3A+A+more+robust+multi%2Ddiscipl)
43. Xiang Yue, Tianyu Zheng, Yuansheng Ni, Yubo Wang, Kai Zhang, Shengbang Tong, Yuxuan Sun, Botao Yu, Ge Zhang, Huan Sun, Yu Su, Wenhu Chen, and Graham Neubig. 2025. MMMU-pro: A more robust multi-discipline multimodal understanding benchmark. [find](https://scholar.google.com/scholar?q=Xiang+Yue%2C+Tianyu+Zheng%2C+Yuansheng+Ni%2C+Yubo+Wang%2C+Kai+Zhang%2C+Shengbang+Tong%2C+Yuxuan+Sun%2C+Botao+Yu%2C+Ge+Zhang%2C+Huan+Sun%2C+Yu+Su%2C+Wenhu+Chen%2C+and+Graham+Neubig%2E+2025%2E+MMMU%2Dpro%3A+A+more+robust+multi%2Ddiscipl)

*Updated 31 October 2026*