---
title: "Curriculum Learning based Hierarchical Scoring and Analysis"
description: "Evaluating question answering (QA) outputs from LLMs at scale is hard. 1.3 points. 103 references."
authors: ["Qiong Wu", "Tan Yue", "Jianxin Liang", "Zhen Li", "Kai He", "Shuai Zhao", "Dongyan Zhao"]
year: 2026
venue: "Findings of the Association for Computational Linguistics: ACL 2026"
source: "https://aclanthology.org/2026.findings-acl.332/"
doi: "10.18653/v1/2026.findings-acl.332"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/wu-2026-curriculum-learning-based-hierarchical-scoring/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/wu-2026-curriculum-learning-based-hierarchical-scoring/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/28/) / Curriculum Learning based Hierarchical Scoring and Analysis Framework for Question Answering Task Evaluation

# Curriculum Learning based Hierarchical Scoring and Analysis Framework for Question Answering Task Evaluation

**Authors:** [Qiong Wu](https://md.research.elamur.ai/free/en/authors/qiong-wu/), Tan Yue, Jianxin Liang, [Zhen Li](https://md.research.elamur.ai/free/en/authors/zhen-li/), [Kai He](https://md.research.elamur.ai/free/en/authors/kai-he/), [Shuai Zhao](https://md.research.elamur.ai/free/en/authors/shuai-zhao/) and [Dongyan Zhao](https://md.research.elamur.ai/free/en/authors/dongyan-zhao/) · **Year:** 2026 · **Venue:** [Findings of the Association for Computational Linguistics: ACL 2026](https://md.research.elamur.ai/free/en/venues/findings/) ([Findings of the Association for Computational Linguistics 2026](https://md.research.elamur.ai/free/en/venues/findings/2026/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2026.findings-acl.332/) · [PDF](https://aclanthology.org/2026.findings-acl.332.pdf) · [DOI 10.18653/v1/2026.findings-acl.332](https://doi.org/10.18653/v1/2026.findings-acl.332) · [OpenAlex](https://openalex.org/W7166867213)

**Citations:** 0 citations (OpenAlex, 2026-10-14)

## Summary
Evaluating question answering (QA) outputs from LLMs at scale is hard. Lexical metrics such as EM, F1 or BLEU miss semantic equivalence. LLM-as-a-Judge setups based on commercial APIs are slow and expensive and rarely explain why an answer failed. HiEval addresses both problems with two models trained in sequence, following a curriculum.

- **HiEval-QS** assigns one of three labels: Correct, Partial or Wrong. It is trained with a class-balanced focal loss whose weights are square-root smoothed, which keeps the rare Partial class from being neglected.
- **HiEval-EA** is built by further training HiEval-QS. It assigns faulty answers to one of five error categories. Experience replay keeps the original scoring skill intact, and a contrastive unlikelihood objective helps separate the error types.

The authors also built human-annotated training data sampled from 50 multimodal QA datasets and released two benchmarks. HiEval beats every baseline, GPT-5 included, on both scoring and error analysis. It runs roughly 25 times faster and has no API cost, which makes detailed QA evaluation practical for large workloads. It only works for questions that have reference answers.

## Key points
- **Data:** 25,996 quick-scoring and 10,545 error-analysis instances, drawn from 50 QA datasets spanning 8 task types, plus two benchmarks for evaluating evaluators.
- **Error types:** Reasoning Error, Image Misunderstanding, Overthinking, Unanswerable, and Other.
- **Main results:**
  - HiEval-QS reaches 93.8% scoring accuracy, against 93% for GPT-5.
  - HiEval-EA reaches 85.5% error-analysis accuracy, against 57.8% for GPT-5 and 43.8% for the best open-source model, Qwen3-VL-4B-Instruct.
  - Per-category accuracy rises to 99.05% on Overthinking and 98% on Unanswerable.
- **Ablations:**
  - Dropping experience replay causes a collapse: scoring falls to 55.2% and analysis to 41.3%.
  - Replacing the focal loss with cross-entropy costs about 1.3 points.
  - Removing contrastive unlikelihood costs about 1.5 points.
- **Limitations:** The framework does not handle open-ended tasks without reference answers. It names the main error type but does not explain the error or point to where it occurs. Code is available at github.com/wq-pku/HiEval.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

📚 Shelves (5): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 18,614 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 17,965 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 17,450 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 17,296 works · [🗄️ Information Systems](https://md.research.elamur.ai/free/en/shelves/computer-science/information-systems/) · 1,730 works

🧭 Topics (2): [📏 Evaluation & benchmarks](https://md.research.elamur.ai/free/en/topics/evaluation-benchmarks/) · 1,341 works · [❓ Question answering](https://md.research.elamur.ai/free/en/topics/question-answering/) · 487 works

## Cite

Cite as: Qiong Wu, Tan Yue, Jianxin Liang, Zhen Li, Kai He, Shuai Zhao and Dongyan Zhao. 2026. Curriculum Learning based Hierarchical Scoring and Analysis Framework for Question Answering Task Evaluation. Findings of the Association for Computational Linguistics: ACL 2026.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{wu2026curriculum,
  title = {{Curriculum Learning based Hierarchical Scoring and Analysis Framework for Question Answering Task Evaluation}},
  author = {Qiong Wu, Tan Yue, Jianxin Liang, Zhen Li, Kai He, Shuai Zhao and Dongyan Zhao},
  booktitle = {Findings of the Association for Computational Linguistics: ACL 2026},
  year = {2026},
  doi = {10.18653/v1/2026.findings-acl.332},
  url = {https://doi.org/10.18653/v1/2026.findings-acl.332}
}
```

- [← previous Curriculum Learning and Pseudo-Labeling Improve the Generalization of Multi-Label Arabic Dialect Identification Models](https://md.research.elamur.ai/free/en/papers/mekky-2026-curriculum-learning-pseudo-labeling-improve/)
- [next → CURUPIRA: Clever guard for harm and linguistic prompt mitigation in Brazilian Portuguese](https://md.research.elamur.ai/free/en/papers/sousa-2026-curupira-clever-guard-harm-linguistic/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/28/)

## References

103 references, 46 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, and 1 others. 2023. [Gpt-4 technical report](https://md.research.elamur.ai/free/en/papers/openai-2023-gpt-4-technical-report/). arXiv preprint arXiv:2303.08774.
2. AI Anthropic. 2024. The claude 3 model family: Opus, sonnet, haiku. Claude-3 Model Card, 1(1):4. [find](https://scholar.google.com/scholar?q=AI+Anthropic%2E+2024%2E+The+claude+3+model+family%3A+Opus%2C+sonnet%2C+haiku%2E+Claude%2D3+Model+Card%2C+1%281%29%3A4%2E)
3. Stanislaw Antol, Aishwarya Agrawal, Jiasen Lu, Margaret Mitchell, Dhruv Batra, C Lawrence Zitnick, and Devi Parikh. 2015. Vqa: Visual question answering. In 2015 IEEE International Conference on Computer Vision (ICCV), pages 2425–2433. IEEE. [find](https://scholar.google.com/scholar?q=Stanislaw+Antol%2C+Aishwarya+Agrawal%2C+Jiasen+Lu%2C+Margaret+Mitchell%2C+Dhruv+Batra%2C+C+Lawrence+Zitnick%2C+and+Devi+Parikh%2E+2015%2E+Vqa%3A+Visual+question+answering%2E+In+2015+IEEE+International+Conference+on+Compu)
4. Akari Asai and Eunsol Choi. 2021. Challenges in information-seeking qa: Unanswerable questions and paragraph retrieval. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Lon… [find](https://scholar.google.com/scholar?q=Akari+Asai+and+Eunsol+Choi%2E+2021%2E+Challenges+in+information%2Dseeking+qa%3A+Unanswerable+questions+and+paragraph+retrieval%2E+In+Proceedings+of+the+59th+Annual+Meeting+of+the+Association+for+Computational+L)
5. Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, and 1 others. 2025. Qwen2. 5-vl technical report. arXiv preprint arXiv:2502.13923. — [Qwen2.5-VL Technical Report](https://md.research.elamur.ai/free/en/papers/bai-2025-qwen2-5-vl-technical/)
6. Yoshua Bengio, Jérôme Louradour, Ronan Collobert, and Jason Weston. 2009. [Curriculum learning](https://md.research.elamur.ai/free/en/papers/bengio-2009-curriculum-learning/). In Proceedings of the 26th annual international conference on machine learning, pages 41–48.
7. Jeffrey P Bigham, Chandrika Jayant, Hanjie Ji, Greg Little, Andrew Miller, Robert C Miller, Robin Miller, Aubrey Tatarowicz, Brandyn White, Samual White, and 1 others. 2010. Vizwiz: nearly real-time answers to visual questions. In Proceedings of the 23nd annual ACM symposium on User interface softwa… [find](https://scholar.google.com/scholar?q=Jeffrey+P+Bigham%2C+Chandrika+Jayant%2C+Hanjie+Ji%2C+Greg+Little%2C+Andrew+Miller%2C+Robert+C+Miller%2C+Robin+Miller%2C+Aubrey+Tatarowicz%2C+Brandyn+White%2C+Samual+White%2C+and+1+others%2E+2010%2E+Vizwiz%3A+nearly+real%2Dtime+a)
8. Jannis Bulian, Christian Buck, Wojciech Gajewski, Benjamin Börschinger, and Tal Schuster. 2022. [Tomayto, tomahto. beyond token-level answer equivalence for question answering evaluation](https://md.research.elamur.ai/free/en/papers/bulian-2022-tomayto-tomahto-beyond-token-level/). In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 291–305.
9. Jie Cao and Jing Xiao. 2022. An augmented benchmark dataset for geometric question answering through dual parallel text encoding. In Proceedings of the 29th international conference on computational linguistics, pages 1511–1520. [find](https://scholar.google.com/scholar?q=Jie+Cao+and+Jing+Xiao%2E+2022%2E+An+augmented+benchmark+dataset+for+geometric+question+answering+through+dual+parallel+text+encoding%2E+In+Proceedings+of+the+29th+international+conference+on+computational+l)
10. Shuaichen Chang, David Palzer, Jialin Li, Eric Fosler-Lussier, and Ningchuan Xiao. 2022. Mapqa: A dataset for question answering on choropleth maps. arXiv preprint arXiv:2211.08545. — [arXiv:2211.08545](https://arxiv.org/abs/2211.08545) [find](https://scholar.google.com/scholar?q=Shuaichen+Chang%2C+David+Palzer%2C+Jialin+Li%2C+Eric+Fosler%2DLussier%2C+and+Ningchuan+Xiao%2E+2022%2E+Mapqa%3A+A+dataset+for+question+answering+on+choropleth+maps%2E+arXiv+preprint+arXiv%3A2211%2E08545%2E+%E2%80%94+%5BarXiv%3A2211%2E0854)
11. Jiaqi Chen, Tong Li, Jinghui Qin, Pan Lu, Liang Lin, Chongyu Chen, and Xiaodan Liang. [Unigeo: Unifying geometry logical reasoning via reformulating mathematical expression](https://md.research.elamur.ai/free/en/papers/chen-2022-unigeo-unifying-geometry-logical-reasoning/).
12. Jiefeng Chen, Jinsung Yoon, Sayna Ebrahimi, Sercan Arik, Tomas Pfister, and Somesh Jha. 2023a. Adaptation with self-evaluation to improve selective prediction in llms. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 5190– 5213. [find](https://scholar.google.com/scholar?q=Jiefeng+Chen%2C+Jinsung+Yoon%2C+Sayna+Ebrahimi%2C+Sercan+Arik%2C+Tomas+Pfister%2C+and+Somesh+Jha%2E+2023a%2E+Adaptation+with+self%2Devaluation+to+improve+selective+prediction+in+llms%2E+In+Findings+of+the+Association+f)
13. Lin Chen, Jinsong Li, Xiaoyi Dong, Pan Zhang, Yuhang Zang, Zehui Chen, Haodong Duan, Jiaqi Wang, Yu Qiao, Dahua Lin, and 1 others. 2024a. [Are we on the right way for evaluating large vision-language models?](https://md.research.elamur.ai/free/en/papers/chen-2024-right-way-evaluating-large-vision/) Advances in Neural Information Processing Systems, 37:27056–27087.
14. Qiguang Chen, Libo Qin, Jin Zhang, Zhi Chen, Xiao Xu, and Wanxiang Che. 2024b. M3cot: A novel benchmark for multi-domain multi-step multi-modal chain-of-thought. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 8199–8221. [find](https://scholar.google.com/scholar?q=Qiguang+Chen%2C+Libo+Qin%2C+Jin+Zhang%2C+Zhi+Chen%2C+Xiao+Xu%2C+and+Wanxiang+Che%2E+2024b%2E+M3cot%3A+A+novel+benchmark+for+multi%2Ddomain+multi%2Dstep+multi%2Dmodal+chain%2Dof%2Dthought%2E+In+Proceedings+of+the+62nd+Annual+Meet)
15. Wenhu Chen, Ming Yin, Max Ku, Pan Lu, Yixin Wan, Xueguang Ma, Jianyu Xu, Xinyi Wang, and Tony Xia. 2023b. [Theoremqa: A theorem-driven question answering dataset](https://md.research.elamur.ai/free/en/papers/chen-2023-theoremqa-theorem-driven-question-answering/). In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 7889–7901.
16. Zihui Cheng, Qiguang Chen, Jin Zhang, Hao Fei, Xiaocheng Feng, Wanxiang Che, Min Li, and Libo Qin. 2025. Comt: A novel benchmark for chain of multimodal thought on large vision-language models. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39. [find](https://scholar.google.com/scholar?q=Zihui+Cheng%2C+Qiguang+Chen%2C+Jin+Zhang%2C+Hao+Fei%2C+Xiaocheng+Feng%2C+Wanxiang+Che%2C+Min+Li%2C+and+Libo+Qin%2E+2025%2E+Comt%3A+A+novel+benchmark+for+chain+of+multimodal+thought+on+large+vision%2Dlanguage+models%2E+In+Pro)
17. Cheng-Han Chiang, Wei-Chih Chen, Chun-Yi Kuan, Chienchou Yang, and Hung-yi Lee. 2024. Large language model as an assignment evaluator: Insights, feedback, and challenges in a 1000+ student course. arXiv preprint arXiv:2407.05216. [find](https://scholar.google.com/scholar?q=Cheng%2DHan+Chiang%2C+Wei%2DChih+Chen%2C+Chun%2DYi+Kuan%2C+Chienchou+Yang%2C+and+Hung%2Dyi+Lee%2E+2024%2E+Large+language+model+as+an+assignment+evaluator%3A+Insights%2C+feedback%2C+and+challenges+in+a+1000%2B+student+course%2E+arX)
18. Mahesh Deshwal and Apoorva Chawla. 2024. Phudge: Phi-3 as scalable judge. arXiv preprint arXiv:2405.08029. — [arXiv:2405.08029](https://arxiv.org/abs/2405.08029) [find](https://scholar.google.com/scholar?q=Mahesh+Deshwal+and+Apoorva+Chawla%2E+2024%2E+Phudge%3A+Phi%2D3+as+scalable+judge%2E+arXiv+preprint+arXiv%3A2405%2E08029%2E+%E2%80%94+%5BarXiv%3A2405%2E08029%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2405%2E08029%29)
19. Yijiang River Dong, Tiancheng Hu, and Nigel Collier. 2024. Can llm be a personalized judge? arXiv preprint arXiv:2406.11657. [find](https://scholar.google.com/scholar?q=Yijiang+River+Dong%2C+Tiancheng+Hu%2C+and+Nigel+Collier%2E+2024%2E+Can+llm+be+a+personalized+judge%3F+arXiv+preprint+arXiv%3A2406%2E11657%2E)
20. Jessica López Espejel, El Hassane Ettifouri, Mahaman Sanoussi Yahaya Alassan, El Mehdi Chouham, and Walid Dahhane. 2023. Gpt-3.5, gpt-4, or bard? evaluating llms reasoning ability in zero-shot setting and performance boosting through prompts. Natural Language Processing Journal, 5:100032. [find](https://scholar.google.com/scholar?q=Jessica+L%C3%B3pez+Espejel%2C+El+Hassane+Ettifouri%2C+Mahaman+Sanoussi+Yahaya+Alassan%2C+El+Mehdi+Chouham%2C+and+Walid+Dahhane%2E+2023%2E+Gpt%2D3%2E5%2C+gpt%2D4%2C+or+bard%3F+evaluating+llms+reasoning+ability+in+zero%2Dshot+setting)
21. Chenrui Fan, Ming Li, Lichao Sun, and Tianyi Zhou. 2025. Missing premise exacerbates overthinking: Are reasoning models losing critical thinking skill? arXiv preprint arXiv:2504.06514. — [arXiv:2504.06514](https://arxiv.org/abs/2504.06514) [find](https://scholar.google.com/scholar?q=Chenrui+Fan%2C+Ming+Li%2C+Lichao+Sun%2C+and+Tianyi+Zhou%2E+2025%2E+Missing+premise+exacerbates+overthinking%3A+Are+reasoning+models+losing+critical+thinking+skill%3F+arXiv+preprint+arXiv%3A2504%2E06514%2E+%E2%80%94+%5BarXiv%3A2504%2E0)
22. Chaoyou Fu, Yi-Fan Zhang, Shukang Yin, Bo Li, Xinyu Fang, Sirui Zhao, Haodong Duan, Xing Sun, Ziwei Liu, Liang Wang, and 1 others. 2024. Mme-survey: A comprehensive survey on evaluation of multimodal llms. arXiv preprint arXiv:2411.15296. — [arXiv:2411.15296](https://arxiv.org/abs/2411.15296) [find](https://scholar.google.com/scholar?q=Chaoyou+Fu%2C+Yi%2DFan+Zhang%2C+Shukang+Yin%2C+Bo+Li%2C+Xinyu+Fang%2C+Sirui+Zhao%2C+Haodong+Duan%2C+Xing+Sun%2C+Ziwei+Liu%2C+Liang+Wang%2C+and+1+others%2E+2024%2E+Mme%2Dsurvey%3A+A+comprehensive+survey+on+evaluation+of+multimodal+)
23. Jinlan Fu, See-Kiong Ng, Zhengbao Jiang, and Pengfei Liu. 2023. Gptscore: Evaluate as you desire. In North American Chapter of the Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Jinlan+Fu%2C+See%2DKiong+Ng%2C+Zhengbao+Jiang%2C+and+Pengfei+Liu%2E+2023%2E+Gptscore%3A+Evaluate+as+you+desire%2E+In+North+American+Chapter+of+the+Association+for+Computational+Linguistics%2E)
24. Noa Garcia, Chentao Ye, Zihua Liu, Qingtao Hu, Mayu Otani, Chenhui Chu, Yuta Nakashima, and Teruko Mitamura. 2020. A dataset and baselines for visual question answering on art. In European Conference on Computer Vision, pages 92–108. [find](https://scholar.google.com/scholar?q=Noa+Garcia%2C+Chentao+Ye%2C+Zihua+Liu%2C+Qingtao+Hu%2C+Mayu+Otani%2C+Chenhui+Chu%2C+Yuta+Nakashima%2C+and+Teruko+Mitamura%2E+2020%2E+A+dataset+and+baselines+for+visual+question+answering+on+art%2E+In+European+Conference+)
25. Yash Goyal, Tejas Khot, Douglas Summers-Stay, Dhruv Batra, and Devi Parikh. 2017. [Making the v in vqa matter: Elevating the role of image understanding in visual question answering](https://md.research.elamur.ai/free/en/papers/goyal-2016-making-v-vqa-matter-elevating/). In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 6904–6913.
26. Xuehai He, Yichen Zhang, Luntian Mou, Eric Xing, and Pengtao Xie. 2020. Pathvqa: 30000+ questions for medical visual question answering. arXiv preprint arXiv:2003.10286. — [arXiv:2003.10286](https://arxiv.org/abs/2003.10286) [find](https://scholar.google.com/scholar?q=Xuehai+He%2C+Yichen+Zhang%2C+Luntian+Mou%2C+Eric+Xing%2C+and+Pengtao+Xie%2E+2020%2E+Pathvqa%3A+30000%2B+questions+for+medical+visual+question+answering%2E+arXiv+preprint+arXiv%3A2003%2E10286%2E+%E2%80%94+%5BarXiv%3A2003%2E10286%5D%28https%3A%2F%2Fa)
27. Hui Huang, Xingyuan Bu, Hongli Zhou, Yingqi Qu, Jing Liu, Muyun Yang, Bing Xu, and Tiejun Zhao. 2025. An empirical study of llm-as-a-judge for llm evaluation: Fine-tuned judge model is not a general substitute for gpt-4. In Findings of the Association for Computational Linguistics: ACL 2025, pages 5… [find](https://scholar.google.com/scholar?q=Hui+Huang%2C+Xingyuan+Bu%2C+Hongli+Zhou%2C+Yingqi+Qu%2C+Jing+Liu%2C+Muyun+Yang%2C+Bing+Xu%2C+and+Tiejun+Zhao%2E+2025%2E+An+empirical+study+of+llm%2Das%2Da%2Djudge+for+llm+evaluation%3A+Fine%2Dtuned+judge+model+is+not+a+general+s)
28. Kushal Kafle, Brian Price, Scott Cohen, and Christopher Kanan. 2018. Dvqa: Understanding data visualizations via question answering. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 5648–5656. [find](https://scholar.google.com/scholar?q=Kushal+Kafle%2C+Brian+Price%2C+Scott+Cohen%2C+and+Christopher+Kanan%2E+2018%2E+Dvqa%3A+Understanding+data+visualizations+via+question+answering%2E+In+Proceedings+of+the+IEEE+conference+on+computer+vision+and+patter)
29. Samira Ebrahimi Kahou, Vincent Michalski, Adam Atkinson, Ákos Kádár, Adam Trischler, and Yoshua Bengio. 2017. [Figureqa: An annotated figure dataset for visual reasoning](https://md.research.elamur.ai/free/en/papers/kahou-2017-figureqa-annotated-figure-dataset-visual/). arXiv preprint arXiv:1710.07300.
30. Adam Tauman Kalai, Ofir Nachum, Santosh S Vempala, and Edwin Zhang. 2025. Why language models hallucinate. arXiv preprint arXiv:2509.04664. — [arXiv:2509.04664](https://arxiv.org/abs/2509.04664) [find](https://scholar.google.com/scholar?q=Adam+Tauman+Kalai%2C+Ofir+Nachum%2C+Santosh+S+Vempala%2C+and+Edwin+Zhang%2E+2025%2E+Why+language+models+hallucinate%2E+arXiv+preprint+arXiv%3A2509%2E04664%2E+%E2%80%94+%5BarXiv%3A2509%2E04664%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2509%2E04664%29)
31. Ehsan Kamalloo, Nouha Dziri, Charles Clarke, and Davood Rafiei. 2023. [Evaluating open-domain question answering in the era of large language models](https://md.research.elamur.ai/free/en/papers/kamalloo-2023-evaluating-open-domain-question-answering/). In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 5591–5606.
32. Aniruddha Kembhavi, Mike Salvato, Eric Kolve, Minjoon Seo, Hannaneh Hajishirzi, and Ali Farhadi. 2016. [A diagram is worth a dozen images](https://md.research.elamur.ai/free/en/papers/kembhavi-2016-diagram-worth-dozen-images/). In European conference on computer vision, pages 235–251. Springer.
33. Aniruddha Kembhavi, Minjoon Seo, Dustin Schwenk, Jonghyun Choi, Ali Farhadi, and Hannaneh Hajishirzi. 2017. Are you smarter than a sixth grader? textbook question answering for multimodal machine comprehension. In Proceedings of the IEEE Conference on Computer Vision and Pattern recognition, pages 4… [find](https://scholar.google.com/scholar?q=Aniruddha+Kembhavi%2C+Minjoon+Seo%2C+Dustin+Schwenk%2C+Jonghyun+Choi%2C+Ali+Farhadi%2C+and+Hannaneh+Hajishirzi%2E+2017%2E+Are+you+smarter+than+a+sixth+grader%3F+textbook+question+answering+for+multimodal+machine+comp)
34. Seungone Kim, Jamin Shin, Yejin Cho, Joel Jang, Shayne Longpre, Hwaran Lee, Sangdoo Yun, Seongjin Shin, Sungdong Kim, James Thorne, and 1 others. 2024a. [Prometheus: Inducing fine-grained evaluation capability in language models](https://md.research.elamur.ai/free/en/papers/kim-2023-prometheus-inducing-fine-grained-evaluation/). In The Twelfth International Conference on Learning Representations.
35. Seungone Kim, Juyoung Suk, Shayne Longpre, Bill Yuchen Lin, Jamin Shin, Sean Welleck, Graham Neubig, Moontae Lee, Kyungjae Lee, and Minjoon Seo. 2024b. [Prometheus 2: An open source language model specialized in evaluating other language models](https://md.research.elamur.ai/free/en/papers/kim-2024-prometheus-2-open-source-language/). arXiv preprint arXiv:2405.01535.
36. Jason J Lau, Soumya Gayen, Asma Ben Abacha, and Dina Demner-Fushman. 2018. A dataset of clinically generated visual questions and answers about radiology images. Scientific data, 5(1):1–10. [find](https://scholar.google.com/scholar?q=Jason+J+Lau%2C+Soumya+Gayen%2C+Asma+Ben+Abacha%2C+and+Dina+Demner%2DFushman%2E+2018%2E+A+dataset+of+clinically+generated+visual+questions+and+answers+about+radiology+images%2E+Scientific+data%2C+5%281%29%3A1%E2%80%9310%2E)
37. Bohao Li, Rui Wang, Guangzhi Wang, Yuying Ge, Yixiao Ge, and Ying Shan. 2023a. [Seed-bench: Benchmarking multimodal llms with generative comprehension](https://md.research.elamur.ai/free/en/papers/li-2023-seed-bench-benchmarking-multimodal-llms/). arXiv preprint arXiv:2307.16125.
38. Haitao Li, Qian Dong, Junjie Chen, Huixue Su, Yujia Zhou, Qingyao Ai, Ziyi Ye, and Yiqun Liu. 2024a. [Llms-as-judges: A comprehensive survey on llm-based evaluation methods](https://md.research.elamur.ai/free/en/papers/li-2024-llms-judges-comprehensive-survey-llm/). arXiv preprint arXiv:2412.05579.
39. Yifan Li, Yifan Du, Kun Zhou, Jinpeng Wang, Wayne Xin Zhao, and Ji-Rong Wen. 2023b. [Evaluating object hallucination in large vision-language models](https://md.research.elamur.ai/free/en/papers/li-2023-evaluating-object-hallucination-large-vision/). arXiv preprint arXiv:2305.10355.
40. Zhuowan Li, Xingrui Wong, Elias Stengel-Eskin, Adam Kortylewski, Wufei Ma, Benjamin Van Durme, and Alan Yuille. 2023c. Super-clevr: A virtual benchmark to diagnose domain robustness in visual reasoning. In 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 14963– 14973… [find](https://scholar.google.com/scholar?q=Zhuowan+Li%2C+Xingrui+Wong%2C+Elias+Stengel%2DEskin%2C+Adam+Kortylewski%2C+Wufei+Ma%2C+Benjamin+Van+Durme%2C+and+Alan+Yuille%2E+2023c%2E+Super%2Dclevr%3A+A+virtual+benchmark+to+diagnose+domain+robustness+in+visual+reasonin)
41. Zongxia Li, Ishani Mondal, Huy Nghiem, Yijun Liang, and Jordan Lee Boyd-Graber. 2024b. Pedants: Cheap but effective and interpretable answer equivalence. In Findings of the Association for Computational Linguistics: EMNLP 2024, pages 9373–9398. [find](https://scholar.google.com/scholar?q=Zongxia+Li%2C+Ishani+Mondal%2C+Huy+Nghiem%2C+Yijun+Liang%2C+and+Jordan+Lee+Boyd%2DGraber%2E+2024b%2E+Pedants%3A+Cheap+but+effective+and+interpretable+answer+equivalence%2E+In+Findings+of+the+Association+for+Computation)
42. Chin-Yew Lin. 2004. [Rouge: A package for automatic evaluation of summaries](https://md.research.elamur.ai/free/en/papers/lin-2004-rouge-package-automatic-evaluation-summaries/). In Annual Meeting of the Association for Computational Linguistics.
43. Yen-Ting Lin and Yun-Nung Chen. 2023. [Llm-eval: Unified multi-dimensional automatic evaluation for open-domain conversations with large language models](https://md.research.elamur.ai/free/en/papers/lin-2023-llm-eval-unified-multi-dimensional/). arXiv preprint arXiv:2305.13711.
44. Adam Dahlgren Lindström and Savitha Sam Abraham. 2022. Clevr-math: A dataset for compositional language, visual and mathematical reasoning. arXiv preprint arXiv:2208.05358. — [arXiv:2208.05358](https://arxiv.org/abs/2208.05358) [find](https://scholar.google.com/scholar?q=Adam+Dahlgren+Lindstr%C3%B6m+and+Savitha+Sam+Abraham%2E+2022%2E+Clevr%2Dmath%3A+A+dataset+for+compositional+language%2C+visual+and+mathematical+reasoning%2E+arXiv+preprint+arXiv%3A2208%2E05358%2E+%E2%80%94+%5BarXiv%3A2208%2E05358%5D%28https%3A)
45. Bo Liu, Li-Ming Zhan, Li Xu, Lin Ma, Yan Yang, and Xiao-Ming Wu. 2021. Slake: A semantically-labeled knowledge-enhanced dataset for medical visual question answering. In 2021 IEEE 18th international symposium on biomedical imaging (ISBI), pages 1650– 1654. IEEE. [find](https://scholar.google.com/scholar?q=Bo+Liu%2C+Li%2DMing+Zhan%2C+Li+Xu%2C+Lin+Ma%2C+Yan+Yang%2C+and+Xiao%2DMing+Wu%2E+2021%2E+Slake%3A+A+semantically%2Dlabeled+knowledge%2Denhanced+dataset+for+medical+visual+question+answering%2E+In+2021+IEEE+18th+international+s)
46. Haotian Liu, Chunyuan Li, Yuheng Li, and Yong Jae Lee. 2024a. [Improved baselines with visual instruction tuning](https://md.research.elamur.ai/free/en/papers/liu-2023-improved-baselines-visual-instruction-tuning/). In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 26296–26306.
47. Yang Liu, Dan Iter, Yichong Xu, Shuohang Wang, Ruochen Xu, and Chenguang Zhu. 2023. [G-eval: Nlg evaluation using gpt-4 with better human alignment](https://md.research.elamur.ai/free/en/papers/liu-2023-g-eval-nlg-evaluation-gpt/). arXiv preprint arXiv:2303.16634.
48. Yuan Liu, Haodong Duan, Yuanhan Zhang, Bo Li, Songyang Zhang, Wangbo Zhao, Yike Yuan, Jiaqi Wang, Conghui He, Ziwei Liu, and 1 others. 2024b. [Mmbench: Is your multi-modal model an all-around player?](https://md.research.elamur.ai/free/en/papers/liu-2023-mmbench-multi-modal-model-all/) In European Conference on Computer Vision, pages 216–233.
49. Pan Lu, Hritik Bansal, Tony Xia, Jiacheng Liu, Chunyuan Li, Hannaneh Hannedige, A Ansari, Kai-Wei Lin, S andchang, and 1 others. 2024. [Mathvista: Evaluating mathematical reasoning of foundation models in visual contexts](https://md.research.elamur.ai/free/en/papers/lu-2023-mathvista-evaluating-mathematical-reasoning-foundation/). In International Conference on Learning Representations (ICLR).
50. Pan Lu, Ran Gong, Shibiao Jiang, Liang Qiu, Siyuan Huang, Xiaodan Liang, and Song-chun Zhu. 2021a. [Inter-gps: Interpretable geometry problem solving with formal language and symbolic reasoning](https://md.research.elamur.ai/free/en/papers/lu-2021-inter-gps-interpretable-geometry-problem/). In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th In…
51. Pan Lu, Swaroop Mishra, Tony Xia, Liang Qiu, Kai-Wei Chang, Song-Chun Zhu, Oyvind Tafjord, Peter Clark, and Ashwin Kalyan. 2022. Learn to explain: Multimodal reasoning via thought chains for science question answering. In The 36th Conference on Neural Information Processing Systems (NeurIPS). [find](https://scholar.google.com/scholar?q=Pan+Lu%2C+Swaroop+Mishra%2C+Tony+Xia%2C+Liang+Qiu%2C+Kai%2DWei+Chang%2C+Song%2DChun+Zhu%2C+Oyvind+Tafjord%2C+Peter+Clark%2C+and+Ashwin+Kalyan%2E+2022%2E+Learn+to+explain%3A+Multimodal+reasoning+via+thought+chains+for+science+q)
52. Pan Lu, Liang Qiu, Kai-Wei Chang, Ying Nian Wu, Song-Chun Zhu, Tanmay Rajpurohit, Peter Clark, and Ashwin Kalyan. 2023. [Dynamic prompt learning via policy gradient for semi-structured mathematical reasoning](https://md.research.elamur.ai/free/en/papers/lu-2022-dynamic-prompt-learning-policy-gradient/). In International Conference on Learning Representations (ICLR).
53. Pan Lu, Liang Qiu, Jiaqi Chen, Tony Xia, Yizhou Zhao, Wei Zhang, Zhou Yu, Xiaodan Liang, and Song-Chun Zhu. 2021b. [Iconqa: A new benchmark for abstract diagram understanding and visual language reasoning](https://md.research.elamur.ai/free/en/papers/lu-2021-iconqa-new-benchmark-abstract-diagram/). arXiv preprint arXiv:2110.13214.
54. Aman Madaan, Niket Tandon, Prakhar Gupta, Skyler Hallinan, Luyu Gao, Sarah Wiegreffe, Uri Alon, Nouha Dziri, Shrimai Prabhumoye, Yiming Yang, and 1 others. 2023. Self-refine: Iterative refinement with self-feedback. Advances in Neural Information Processing Systems, 36:46534–46594. [find](https://scholar.google.com/scholar?q=Aman+Madaan%2C+Niket+Tandon%2C+Prakhar+Gupta%2C+Skyler+Hallinan%2C+Luyu+Gao%2C+Sarah+Wiegreffe%2C+Uri+Alon%2C+Nouha+Dziri%2C+Shrimai+Prabhumoye%2C+Yiming+Yang%2C+and+1+others%2E+2023%2E+Self%2Drefine%3A+Iterative+refinement+with)
55. Ahmed Masry, Xuan Long Do, Jia Qing Tan, Shafiq Joty, and Enamul Hoque. 2022. [Chartqa: A benchmark for question answering about charts with visual and logical reasoning](https://md.research.elamur.ai/free/en/papers/masry-2022-chartqa-benchmark-question-answering-about/). In Findings of the association for computational linguistics: ACL 2022, pages 2263– 2279.
56. Minesh Mathew, Dimosthenis Karatzas, and CV Jawahar. 2021. [Docvqa: A dataset for vqa on document images](https://md.research.elamur.ai/free/en/papers/mathew-2020-docvqa-dataset-vqa-document-images/). In Proceedings of the IEEE/CVF winter conference on applications of computer vision, pages 2200–2209.
57. Nitesh Methani, Pritha Ganguly, Mitesh M. Khapra, and Pratyush Kumar. 2020. Plotqa: Reasoning over scientific plots. In The IEEE Winter Conference on Applications of Computer Vision (WACV). [find](https://scholar.google.com/scholar?q=Nitesh+Methani%2C+Pritha+Ganguly%2C+Mitesh+M%2E+Khapra%2C+and+Pratyush+Kumar%2E+2020%2E+Plotqa%3A+Reasoning+over+scientific+plots%2E+In+The+IEEE+Winter+Conference+on+Applications+of+Computer+Vision+%28WACV%29%2E)
58. Anand Mishra, Shashank Shekhar, Ajeet Kumar Singh, and Anirban Chakraborty. 2019. Ocr-vqa: Visual question answering by reading text in images. In 2019 international conference on document analysis and recognition (ICDAR), pages 947–952. IEEE. [find](https://scholar.google.com/scholar?q=Anand+Mishra%2C+Shashank+Shekhar%2C+Ajeet+Kumar+Singh%2C+and+Anirban+Chakraborty%2E+2019%2E+Ocr%2Dvqa%3A+Visual+question+answering+by+reading+text+in+images%2E+In+2019+international+conference+on+document+analysis+an)
59. Shaghayegh Mobasher, Ghazal Zamaninejad, Maryam Hashemi, Melika Nobakhtian, and Sauleh Eetemadi. 2022. Parsvqa-caps: A benchmark for visual question answering and image captioning in persian. people, 101(404):1. [find](https://scholar.google.com/scholar?q=Shaghayegh+Mobasher%2C+Ghazal+Zamaninejad%2C+Maryam+Hashemi%2C+Melika+Nobakhtian%2C+and+Sauleh+Eetemadi%2E+2022%2E+Parsvqa%2Dcaps%3A+A+benchmark+for+visual+question+answering+and+image+captioning+in+persian%2E+people%2C+)
60. Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, and 1 others. 2022. [Training language models to follow instructions with human feedback](https://md.research.elamur.ai/free/en/papers/ouyang-2022-training-language-models-follow-instructions/). Advances in neural information processing systems, 35:27730–27744.
61. Gemini Team, Petko Georgiev, Ving Ian Lei, Ryan Burnell, Libin Bai, Anmol Gulati, Garrett Tanzer, Damien Vincent, Zhufeng Pan, Shibo Wang, and 1 others. 2024. [Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context](https://md.research.elamur.ai/free/en/papers/gemini-team-2024-gemini-1-5-unlocking-multimodal/). arXiv preprint arXiv:2403.05530.
62. Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. [Bleu: a method for automatic evaluation of machine translation](https://md.research.elamur.ai/free/en/papers/papineni-2002-bleu-method-automatic-evaluation-machine/). In Proceedings of the 40th annual meeting of the Association for Computational Linguistics, pages 311–318.
63. Kimi Team, Angang Du, Bohong Yin, Bowei Xing, Bowen Qu, Bowen Wang, Cheng Chen, Chenlin Zhang, Chenzhuang Du, Chu Wei, and 1 others. 2025. [Kimi-vl technical report](https://md.research.elamur.ai/free/en/papers/kimi-team-2025-kimi-vl-technical-report/). arXiv preprint arXiv:2504.07491.
64. Jie Ren, Yao Zhao, Tu Vu, Peter J Liu, and Balaji Lakshminarayanan. 2023. Self-evaluation improves selective generation in large language models. In Proceedings of the 40th International Conference on Machine Learning. [find](https://scholar.google.com/scholar?q=Jie+Ren%2C+Yao+Zhao%2C+Tu+Vu%2C+Peter+J+Liu%2C+and+Balaji+Lakshminarayanan%2E+2023%2E+Self%2Devaluation+improves+selective+generation+in+large+language+models%2E+In+Proceedings+of+the+40th+International+Conference+on)
65. Alexander Vogel, Omar Moured, Yufan Chen, Jiaming Zhang, and Rainer Stiefelhagen. 2025. Refchartqa: Grounding visual answer on chart images through instruction tuning. In International Conference on Document Analysis and Recognition, pages 523–537. Springer. [find](https://scholar.google.com/scholar?q=Alexander+Vogel%2C+Omar+Moured%2C+Yufan+Chen%2C+Jiaming+Zhang%2C+and+Rainer+Stiefelhagen%2E+2025%2E+Refchartqa%3A+Grounding+visual+answer+on+chart+images+through+instruction+tuning%2E+In+International+Conference+on+D)
66. Jon Saad-Falcon, Omar Khattab, Christopher Potts, and Matei Zaharia. 2023. Ares: An automated evaluation framework for retrieval-augmented generation systems. arXiv preprint arXiv:2311.09476. [find](https://scholar.google.com/scholar?q=Jon+Saad%2DFalcon%2C+Omar+Khattab%2C+Christopher+Potts%2C+and+Matei+Zaharia%2E+2023%2E+Ares%3A+An+automated+evaluation+framework+for+retrieval%2Daugmented+generation+systems%2E+arXiv+preprint+arXiv%3A2311%2E09476%2E)
67. Dustin Schwenk, Apoorv Khandelwal, Christopher Clark, Kenneth Marino, and Roozbeh Mottaghi. 2022. [A-okvqa: A benchmark for visual question answering using world knowledge](https://md.research.elamur.ai/free/en/papers/schwenk-2022-okvqa-benchmark-visual-question-answering/). In European Conference on Computer Vision, pages 146–162.
68. Min Joon Seo, Hannaneh Hajishirzi, Ali Farhadi, Oren Etzioni, and Clint Malcolm. 2015. [Solving geometry problems: Combining text and diagram interpretation](https://md.research.elamur.ai/free/en/papers/seo-2015-solving-geometry-problems-combining-text/). In EMNLP, volume 1, page 3.
69. Sanket Shah, Anand Mishra, Naganand Yadati, and Partha Pratim Talukdar. 2019. Kvqa: Knowledge-aware visual question answering. In AAAI Conference on Artificial Intelligence. [find](https://scholar.google.com/scholar?q=Sanket+Shah%2C+Anand+Mishra%2C+Naganand+Yadati%2C+and+Partha+Pratim+Talukdar%2E+2019%2E+Kvqa%3A+Knowledge%2Daware+visual+question+answering%2E+In+AAAI+Conference+on+Artificial+Intelligence%2E)
70. Yongliang Shen, Kaitao Song, Xu Tan, Dongsheng Li, Weiming Lu, and Yueting Zhuang. 2023. [Hugginggpt: Solving ai tasks with chatgpt and its friends in hugging face](https://md.research.elamur.ai/free/en/papers/shen-2023-hugginggpt-solving-ai-tasks-chatgpt/). Advances in Neural Information Processing Systems, 36:38154–38180.
71. Amanpreet Singh, Vivek Natarajan, Meet Shah, Yu Jiang, Xinlei Chen, Dhruv Batra, Devi Parikh, and Marcus Rohrbach. 2019. Towards vqa models that can read. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 8317–8326. [find](https://scholar.google.com/scholar?q=Amanpreet+Singh%2C+Vivek+Natarajan%2C+Meet+Shah%2C+Yu+Jiang%2C+Xinlei+Chen%2C+Dhruv+Batra%2C+Devi+Parikh%2C+and+Marcus+Rohrbach%2E+2019%2E+Towards+vqa+models+that+can+read%2E+In+Proceedings+of+the+IEEE%2FCVF+Conference+on+)
72. Andrea Sottana, Bin Liang, Kai Zou, and Zheng Yuan. 2023. [Evaluation metrics in the era of gpt-4: reliably evaluating large language models on sequence to sequence tasks](https://md.research.elamur.ai/free/en/papers/sottana-2023-evaluation-metrics-era-gpt-4/). arXiv preprint arXiv:2310.13800.
73. Kai Sun, Yushi Bai, Ji Qi, Lei Hou, and Juanzi Li. 2024. Mm-math: Advancing multimodal math evaluation with process evaluation and fine-grained classification. In Conference on Empirical Methods in Natural Language Processing. [find](https://scholar.google.com/scholar?q=Kai+Sun%2C+Yushi+Bai%2C+Ji+Qi%2C+Lei+Hou%2C+and+Juanzi+Li%2E+2024%2E+Mm%2Dmath%3A+Advancing+multimodal+math+evaluation+with+process+evaluation+and+fine%2Dgrained+classification%2E+In+Conference+on+Empirical+Methods+in+Na)
74. Cunxiang Wang, Sirui Cheng, Qipeng Guo, Yuanhao Yue, Bowen Ding, Zhikun Xu, Yidong Wang, Xiangkun Hu, Zheng Zhang, and Yue Zhang. 2023. Evaluating open-qa evaluation. Advances in Neural Information Processing Systems, 36:77013–77042. [find](https://scholar.google.com/scholar?q=Cunxiang+Wang%2C+Sirui+Cheng%2C+Qipeng+Guo%2C+Yuanhao+Yue%2C+Bowen+Ding%2C+Zhikun+Xu%2C+Yidong+Wang%2C+Xiangkun+Hu%2C+Zheng+Zhang%2C+and+Yue+Zhang%2E+2023%2E+Evaluating+open%2Dqa+evaluation%2E+Advances+in+Neural+Information+Pr)
75. Ke Wang, Junting Pan, Weikang Shi, Zimu Lu, Houxing Ren, Aojun Zhou, Mingjie Zhan, and Hongsheng Li. 2024a. [Measuring multimodal mathematical reasoning with math-vision dataset](https://md.research.elamur.ai/free/en/papers/wang-2024-measuring-multimodal-mathematical-reasoning-math/). Advances in Neural Information Processing Systems, 37:95095–95169.
76. Peng Wang, Shuai Bai, Sinan Tan, Shijie Wang, Zhihao Fan, Jinze Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, and 1 others. 2024b. [Qwen2-vl: Enhancing vision-language model’s perception of the world at any resolution](https://md.research.elamur.ai/free/en/papers/wang-2024-qwen2-vl-enhancing-vision/). arXiv preprint arXiv:2409.12191.
77. Shansong Wang, Mingzhe Hu, Qiang Li, Mojtaba Safari, and Xiaofeng Yang. 2025. Capabilities of gpt-5 on multimodal medical reasoning. arXiv preprint arXiv:2508.08224. — [arXiv:2508.08224](https://arxiv.org/abs/2508.08224) [find](https://scholar.google.com/scholar?q=Shansong+Wang%2C+Mingzhe+Hu%2C+Qiang+Li%2C+Mojtaba+Safari%2C+and+Xiaofeng+Yang%2E+2025%2E+Capabilities+of+gpt%2D5+on+multimodal+medical+reasoning%2E+arXiv+preprint+arXiv%3A2508%2E08224%2E+%E2%80%94+%5BarXiv%3A2508%2E08224%5D%28https%3A%2F%2Farxiv)
78. Xiaoxuan Wang, Ziniu Hu, Pan Lu, Yanqiao Zhu, Jieyu Zhang, Satyen Subramaniam, Arjun R Loomba, Shichang Zhang, Yizhou Sun, and Wei Wang. 2024c. [Scibench: evaluating college-level scientific problem-solving abilities of large language models](https://md.research.elamur.ai/free/en/papers/wang-2023-scibench-evaluating-college-level-scientific/). In Proceedings of the 41st International Conference on Mac…
79. Tianhao Wu, Weizhe Yuan, Olga Golovneva, Jing Xu, Yuandong Tian, Jiantao Jiao, Jason Weston, and Sainbayar Sukhbaatar. 2024. Meta-rewarding language models: Self-improving alignment with llm-as-ameta-judge. arXiv preprint arXiv:2407.19594. — [arXiv:2407.19594](https://arxiv.org/abs/2407.19594) [find](https://scholar.google.com/scholar?q=Tianhao+Wu%2C+Weizhe+Yuan%2C+Olga+Golovneva%2C+Jing+Xu%2C+Yuandong+Tian%2C+Jiantao+Jiao%2C+Jason+Weston%2C+and+Sainbayar+Sukhbaatar%2E+2024%2E+Meta%2Drewarding+language+models%3A+Self%2Dimproving+alignment+with+llm%2Das%2Dameta%2D)
80. Tingyu Xia, Bowen Yu, Yuan Wu, Yi Chang, and Chang Zhou. 2024. Language models can evaluate themselves via probability discrepancy. arXiv preprint arXiv:2405.10516. [find](https://scholar.google.com/scholar?q=Tingyu+Xia%2C+Bowen+Yu%2C+Yuan+Wu%2C+Yi+Chang%2C+and+Chang+Zhou%2E+2024%2E+Language+models+can+evaluate+themselves+via+probability+discrepancy%2E+arXiv+preprint+arXiv%3A2405%2E10516%2E)
81. LLM Xiaomi, Bingquan Xia, Bowen Shen, Dawei Zhu, Di Zhang, Gang Wang, Hailin Zhang, Huaqiu Liu, Jiebao Xiao, Jinhao Dong, and 1 others. 2025. Mimo: Unlocking the reasoning potential of language model– from pretraining to posttraining. arXiv preprint arXiv:2505.07608. [find](https://scholar.google.com/scholar?q=LLM+Xiaomi%2C+Bingquan+Xia%2C+Bowen+Shen%2C+Dawei+Zhu%2C+Di+Zhang%2C+Gang+Wang%2C+Hailin+Zhang%2C+Huaqiu+Liu%2C+Jiebao+Xiao%2C+Jinhao+Dong%2C+and+1+others%2E+2025%2E+Mimo%3A+Unlocking+the+reasoning+potential+of+language+model%E2%80%93)
82. Qianqian Xie, Weiguang Han, Xiao Zhang, Yanzhao Lai, Min Peng, Alejandro Lopez-Lira, and Jimin Huang. 2023. Pixiu: A large language model, instruction data and evaluation benchmark for finance. arXiv preprint arXiv:2306.05443. — [arXiv:2306.05443](https://arxiv.org/abs/2306.05443) [find](https://scholar.google.com/scholar?q=Qianqian+Xie%2C+Weiguang+Han%2C+Xiao+Zhang%2C+Yanzhao+Lai%2C+Min+Peng%2C+Alejandro+Lopez%2DLira%2C+and+Jimin+Huang%2E+2023%2E+Pixiu%3A+A+large+language+model%2C+instruction+data+and+evaluation+benchmark+for+finance%2E+arXiv+)
83. Yiqing Xie, Sheng Zhang, Hao Cheng, Pengfei Liu, Zelalem Gero, Cliff Wong, Tristan Naumann, Hoifung Poon, and Carolyn Rose. 2024. Doclens: Multi-aspect fine-grained evaluation for medical text generation. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Vol… [find](https://scholar.google.com/scholar?q=Yiqing+Xie%2C+Sheng+Zhang%2C+Hao+Cheng%2C+Pengfei+Liu%2C+Zelalem+Gero%2C+Cliff+Wong%2C+Tristan+Naumann%2C+Hoifung+Poon%2C+and+Carolyn+Rose%2E+2024%2E+Doclens%3A+Multi%2Daspect+fine%2Dgrained+evaluation+for+medical+text+generat)
84. Tianyi Xiong, Xiyao Wang, Dong Guo, Qinghao Ye, Haoqi Fan, Quanquan Gu, Heng Huang, and Chunyuan Li. 2024. Llava-critic: Learning to evaluate multimodal models. arXiv preprint arXiv:2410.02712. — [arXiv:2410.02712](https://arxiv.org/abs/2410.02712) [find](https://scholar.google.com/scholar?q=Tianyi+Xiong%2C+Xiyao+Wang%2C+Dong+Guo%2C+Qinghao+Ye%2C+Haoqi+Fan%2C+Quanquan+Gu%2C+Heng+Huang%2C+and+Chunyuan+Li%2E+2024%2E+Llava%2Dcritic%3A+Learning+to+evaluate+multimodal+models%2E+arXiv+preprint+arXiv%3A2410%2E02712%2E+%E2%80%94+%5BarX)
85. Shuying Xu, Junjie Hu, and Ming Jiang. 2024. Large language models are active critics in nlg evaluation. arXiv preprint arXiv:2410.10724. — [arXiv:2410.10724](https://arxiv.org/abs/2410.10724) [find](https://scholar.google.com/scholar?q=Shuying+Xu%2C+Junjie+Hu%2C+and+Ming+Jiang%2E+2024%2E+Large+language+models+are+active+critics+in+nlg+evaluation%2E+arXiv+preprint+arXiv%3A2410%2E10724%2E+%E2%80%94+%5BarXiv%3A2410%2E10724%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2410%2E10724%29)
86. Tan Yue, Rui Mao, Xuzhao Shi, Shuo Zhan, Zuhao Yang, and Dongyan Zhao. 2025a. Qaeval: Mixture of evaluators for question-answering task evaluation. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 14717–14730. [find](https://scholar.google.com/scholar?q=Tan+Yue%2C+Rui+Mao%2C+Xuzhao+Shi%2C+Shuo+Zhan%2C+Zuhao+Yang%2C+and+Dongyan+Zhao%2E+2025a%2E+Qaeval%3A+Mixture+of+evaluators+for+question%2Danswering+task+evaluation%2E+In+Proceedings+of+the+63rd+Annual+Meeting+of+the+Ass)
87. Tan Yue, Xuzhao Shi, Rui Mao, Zilong Song, Zonghai Hu, and Dongyan Zhao. 2025b. Anafig: A human-aligned dataset for scientific figure analysis. In Proceedings of the 33rd ACM International Conference on Multimedia, pages 12837–12843. [find](https://scholar.google.com/scholar?q=Tan+Yue%2C+Xuzhao+Shi%2C+Rui+Mao%2C+Zilong+Song%2C+Zonghai+Hu%2C+and+Dongyan+Zhao%2E+2025b%2E+Anafig%3A+A+human%2Daligned+dataset+for+scientific+figure+analysis%2E+In+Proceedings+of+the+33rd+ACM+International+Conference+)
88. Tan Yue, Qiong Wu, and Dongyan Zhao. 2026b. Mars: Multimodal adaptive reasoning model for avoiding overthinking. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 40, pages 34539– 34547. [find](https://scholar.google.com/scholar?q=Tan+Yue%2C+Qiong+Wu%2C+and+Dongyan+Zhao%2E+2026b%2E+Mars%3A+Multimodal+adaptive+reasoning+model+for+avoiding+overthinking%2E+In+Proceedings+of+the+AAAI+Conference+on+Artificial+Intelligence%2C+volume+40%2C+pages+3453)
89. Tan Yue, Qiong Wu, and Dongyan Zhao. 2026c. We may not need much visual encoding of web data for question answering. In Proceedings of the ACM Web Conference 2026, pages 8437–8440. [find](https://scholar.google.com/scholar?q=Tan+Yue%2C+Qiong+Wu%2C+and+Dongyan+Zhao%2E+2026c%2E+We+may+not+need+much+visual+encoding+of+web+data+for+question+answering%2E+In+Proceedings+of+the+ACM+Web+Conference+2026%2C+pages+8437%E2%80%938440%2E)
90. An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, and 1 others. 2025. [Qwen3 technical report](https://md.research.elamur.ai/free/en/papers/yang-2025-qwen3-technical-report/). arXiv preprint arXiv:2505.09388.
91. Xiang Yue, Yuansheng Ni, Kai Zhang, Tianyu Zheng, Ruoqi Liu, Ge Zhang, Samuel Stevens, Dongfu Jiang, Weiming Ren, Yuxuan Sun, and 1 others. 2024. [Mmmu: A massive multi-discipline multimodal understanding and reasoning benchmark for expert agi](https://md.research.elamur.ai/free/en/papers/yue-2023-mmmu-massive-multi-discipline-multimodal/). In Proceedings of the IEEE/CVF Conference on Computer Vi…
92. Nakyeong Yang, Taegwan Kang, Stanley Jungkyu Choi, Honglak Lee, and Kyomin Jung. 2024. Mitigating biases for instruction-following language models via bias neurons elimination. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 9… [find](https://scholar.google.com/scholar?q=Nakyeong+Yang%2C+Taegwan+Kang%2C+Stanley+Jungkyu+Choi%2C+Honglak+Lee%2C+and+Kyomin+Jung%2E+2024%2E+Mitigating+biases+for+instruction%2Dfollowing+language+models+via+bias+neurons+elimination%2E+In+Proceedings+of+the+6)
93. Kaichen Zhang, Bo Li, Peiyuan Zhang, Fanyi Pu, Joshua Adrian Cahyono, Kairui Hu, Shuai Liu, Yuanhan Zhang, Jingkang Yang, Chunyuan Li, and Ziwei Liu. 2025. [Lmms-eval: Reality check on the evaluation of large multimodal models](https://md.research.elamur.ai/free/en/papers/zhang-2025-lmms-eval-reality-check-evaluation/). In Findings of the Association for Computational Linguistics: NAACL 2025,…
94. Zhongyu Yang, Zuhao Yang, Shuo Zhan, Tan Yue, Wei Pang, and Yingfang Yuan. 2026. Svagent: Storyline-guided long video understanding via cross-modal multi-agent collaboration. In IEEE/CVF Conference on Computer Vision and Pattern Recognition 2026. IEEE. [find](https://scholar.google.com/scholar?q=Zhongyu+Yang%2C+Zuhao+Yang%2C+Shuo+Zhan%2C+Tan+Yue%2C+Wei+Pang%2C+and+Yingfang+Yuan%2E+2026%2E+Svagent%3A+Storyline%2Dguided+long+video+understanding+via+cross%2Dmodal+multi%2Dagent+collaboration%2E+In+IEEE%2FCVF+Conference+on)
95. Renrui Zhang, Dongzhi Jiang, Yichi Zhang, Haokun Lin, Ziyu Guo, Pengshuo Qiu, Aojun Zhou, Pan Lu, Kai-Wei Chang, Yu Qiao, and 1 others. 2024. [Mathverse: Does your multi-modal llm truly see the diagrams in visual math problems?](https://md.research.elamur.ai/free/en/papers/zhang-2024-mathverse-multi-modal-llm-truly/) In European Conference on Computer Vision, pages 169–186.
96. Weihao Yu, Zhengyuan Yang, Linjie Li, Jianfeng Wang, Kevin Lin, Zicheng Liu, Xinchao Wang, and Lijuan Wang. 2024. [Mm-vet: evaluating large multimodal models for integrated capabilities](https://md.research.elamur.ai/free/en/papers/yu-2023-mm-vet-evaluating-large-multimodal/). In Proceedings of the 41st International Conference on Machine Learning, pages 57730–57754.
97. Xiaoman Zhang, Chaoyi Wu, Ziheng Zhao, Weixiong Lin, Ya Zhang, Yanfeng Wang, and Weidi Xie. 2023. Pmc-vqa: Visual instruction tuning for medical visual question answering. arXiv preprint arXiv:2305.10415. — [arXiv:2305.10415](https://arxiv.org/abs/2305.10415) [find](https://scholar.google.com/scholar?q=Xiaoman+Zhang%2C+Chaoyi+Wu%2C+Ziheng+Zhao%2C+Weixiong+Lin%2C+Ya+Zhang%2C+Yanfeng+Wang%2C+and+Weidi+Xie%2E+2023%2E+Pmc%2Dvqa%3A+Visual+instruction+tuning+for+medical+visual+question+answering%2E+arXiv+preprint+arXiv%3A2305%2E10)
98. Shengbin Yue, Wei Chen, Siyuan Wang, Bingxuan Li, Chenchen Shen, Shujun Liu, Yuxuan Zhou, Yao Xiao, Song Yun, Xuanjing Huang, and 1 others. 2023. [Disc-lawllm: Fine-tuning large language models for intelligent legal services](https://md.research.elamur.ai/free/en/papers/yue-2023-disc-lawllm-fine-tuning-large/). arXiv preprint arXiv:2309.11325.
99. Tan Yue, Rui Mao, Xuzhao Shi, and Erik Cambria. 2026a. Interarm: Interpretable affective reasoning model for multimodal sarcasm detection. IEEE Transactions on Affective Computing. [find](https://scholar.google.com/scholar?q=Tan+Yue%2C+Rui+Mao%2C+Xuzhao+Shi%2C+and+Erik+Cambria%2E+2026a%2E+Interarm%3A+Interpretable+affective+reasoning+model+for+multimodal+sarcasm+detection%2E+IEEE+Transactions+on+Affective+Computing%2E)
100. Shuai Zhao, Qika Lin, Yanhao Jia, Xinyi Wu, Yuwen Li, and Luu Anh Tuan. 2026. Unifle: Uniform fusion of multiple lora experts for backdoor defense in large language models. IEEE Transactions on Dependable and Secure Computing. [find](https://scholar.google.com/scholar?q=Shuai+Zhao%2C+Qika+Lin%2C+Yanhao+Jia%2C+Xinyi+Wu%2C+Yuwen+Li%2C+and+Luu+Anh+Tuan%2E+2026%2E+Unifle%3A+Uniform+fusion+of+multiple+lora+experts+for+backdoor+defense+in+large+language+models%2E+IEEE+Transactions+on+Depend)
101. Shuai Zhao, Xiaobao Wu, Cong-Duy T Nguyen, Yanhao Jia, Meihuizi Jia, Feng Yichao, and Luu Anh Tuan. 2025a. Unlearning backdoor attacks for llms with weak-to-strong knowledge distillation. In Findings of the Association for Computational Linguistics: ACL 2025, pages 4937–4952. [find](https://scholar.google.com/scholar?q=Shuai+Zhao%2C+Xiaobao+Wu%2C+Cong%2DDuy+T+Nguyen%2C+Yanhao+Jia%2C+Meihuizi+Jia%2C+Feng+Yichao%2C+and+Luu+Anh+Tuan%2E+2025a%2E+Unlearning+backdoor+attacks+for+llms+with+weak%2Dto%2Dstrong+knowledge+distillation%2E+In+Findings+)
102. Shuai Zhao, Xinyi Wu, Shiqian Zhao, Xiaobao Wu, Zhongliang Guo, Yanhao Jia, and Anh Tuan Luu. 2025b. [P2p: A poison-to-poison remedy for reliable backdoor defense in llms](https://md.research.elamur.ai/free/en/papers/zhao-2026-p2p-poison-poison-remedy-reliable/). arXiv preprint arXiv:2510.04503.
103. Lianghui Zhu, Xinggang Wang, and Xinlong Wang. 2023. [Judgelm: Fine-tuned large language models are scalable judges](https://md.research.elamur.ai/free/en/papers/zhu-2023-judgelm-fine-tuned-large-language/). arXiv preprint arXiv:2310.17631.

*Updated 2 November 2026*