---
title: "DaMo — paper by Shi et al. (2026) — summary and ideas"
description: "The authors from OPPO AI Center propose DaMo. 66 references. Among them: 0."
authors: ["Kai Shi", "Jun Yang", "Ni Yang", "Binqiang Pan", "Qingsong Xie", "Zhangchao", "Zhenyu Yang", "Tianhuang Su", "Haonan Lu"]
year: 2026
venue: "Findings of the Association for Computational Linguistics: ACL 2026"
source: "https://aclanthology.org/2026.findings-acl.1439/"
doi: "10.18653/v1/2026.findings-acl.1439"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/shi-2026-damo-data-mixing-optimizer-fine/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/shi-2026-damo-data-mixing-optimizer-fine/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/26/) / DaMo: Data Mixing Optimizer in Fine-tuning Multimodal LLMs for Mobile Phone Agents

# DaMo: Data Mixing Optimizer in Fine-tuning Multimodal LLMs for Mobile Phone Agents

**Authors:** Kai Shi, [Jun Yang](https://md.research.elamur.ai/free/en/authors/jun-yang/), Ni Yang, Binqiang Pan, Qingsong Xie, Zhangchao, Zhenyu Yang, Tianhuang Su and Haonan Lu · **Year:** 2026 · **Venue:** [Findings of the Association for Computational Linguistics: ACL 2026](https://md.research.elamur.ai/free/en/venues/findings/) ([Findings of the Association for Computational Linguistics 2026](https://md.research.elamur.ai/free/en/venues/findings/2026/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2026.findings-acl.1439/) · [PDF](https://aclanthology.org/2026.findings-acl.1439.pdf) · [DOI 10.18653/v1/2026.findings-acl.1439](https://doi.org/10.18653/v1/2026.findings-acl.1439) · [OpenAlex](https://openalex.org/W4416246765)

**Citations:** 0 citations (OpenAlex, 2026-10-14)

## Summary
Phone assistants built on multimodal LLMs must combine several skills, including planning, tool calling, memory and screen understanding. Multitask SFT is the usual way to teach these skills, but choosing how much of each training dataset to include is expensive to get right. Existing mixture optimizers target pretraining loss, and simple sampling or grid search scale poorly. The authors from OPPO AI Center propose DaMo. It samples a modest number of mixture ratios, fine-tunes a small MLLM on each, and records downstream benchmark scores. An MLP is then trained to map ratios to scores, and the best mixture is found by extrapolating across the full mixing space. They also release PhoneAgentBench, an industrial-style benchmark for mobile agent tasks that goes beyond GUI interaction. DaMo predicts scores well and produces better-performing models than uniform, natural, random and DML mixtures at a similar compute budget. Its chosen mixtures also transfer to other model families. This makes data-ratio tuning for fine-tuning MLLMs more principled and cheaper to reuse.

## Key points
- **PhoneAgentBench:** 1,235 validated QA pairs organized into six datasets, such as MT-Plan and Mobile-FC.
  - It covers four capabilities: task planning, device-native tool use, multimodal memory and screen context understanding.
  - It draws on 50 core mobile interfaces.
  - MT-Plan's complexity score is about 5.4× that of T-Eval.
- **Prediction accuracy:** DaMo reaches R²=0.81 in small-scale pilot runs, with InternVL2.5-4B as the base model.
- **Gains over DML:**
  - +3.06% average across PhoneAgentBench, BFCL-v3, MME-Reasoning, MME-Perception and OCRBench.
  - +6.70% when optimizing only for open-source benchmarks.
  - BFCL-v3 rises from 34.69% (Uniform) to 47.43% when DaMo optimizes for it alone.
- **Cost:** DaMo uses about 4,242 H20 GPU-hours, roughly the same as DML. Uniform and Natural mixtures did not improve even when given longer training.
- **Transfer to other models:** The tested targets were Qwen2.5VL-3B/7B and InternVL3-14B.
  - Reusing the chosen mixture directly yields +0.31–2.87%.
  - Adding a linear correction fitted on 20 calibration samples yields +1.61–3.51%.
  - Correlation between predicted and actual scores rises from r > 0.75 to r > 0.9 with the correction.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

📚 Shelves (3): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [👁️ Computer Vision and Pattern Recognition](https://md.research.elamur.ai/free/en/shelves/computer-science/computer-vision-and-pattern-recognition/) · 2,869 works

🧭 Topics (1): [🖼️ Vision & language](https://md.research.elamur.ai/free/en/topics/multimodal/) · 657 works

## Cite

Cite as: Kai Shi, Jun Yang, Ni Yang, Binqiang Pan, Qingsong Xie, Zhangchao, Zhenyu Yang, Tianhuang Su and Haonan Lu. 2026. DaMo: Data Mixing Optimizer in Fine-tuning Multimodal LLMs for Mobile Phone Agents. Findings of the Association for Computational Linguistics: ACL 2026.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{shi2026damo,
  title = {{DaMo: Data Mixing Optimizer in Fine-tuning Multimodal LLMs for Mobile Phone Agents}},
  author = {Kai Shi, Jun Yang, Ni Yang, Binqiang Pan, Qingsong Xie, Zhangchao, Zhenyu Yang, Tianhuang Su and Haonan Lu},
  booktitle = {Findings of the Association for Computational Linguistics: ACL 2026},
  year = {2026},
  doi = {10.18653/v1/2026.findings-acl.1439},
  url = {https://doi.org/10.18653/v1/2026.findings-acl.1439}
}
```

- [← previous DAMETA: An LLM Benchmark for Danish Metaphor Interpretation with Systematically Varied Distractors](https://md.research.elamur.ai/free/en/papers/schneidermann-2026-dameta-llm-benchmark-danish-metaphor/)
- [next → Danger Depends on the Mind: A Theory-of-Mind Grounded Dataset and Model for Context-Dependent Dangerous Speech](https://md.research.elamur.ai/free/en/papers/shi-2026-danger-depends-mind-theory-mind/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/26/)

## References

66 references, 23 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Abhishek Shrivastava. 2023. sentiment-analysis-dataset. [https://www.kaggle.com/datasets/](https://www.kaggle.com/datasets/) abhi8923shriv/sentiment-analysis-dataset. Accessed: 2025-03-12. [↗](https://www.kaggle.com/datasets/)
2. Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, and 1 others. 2023. [Gpt-4 technical report](https://md.research.elamur.ai/free/en/papers/openai-2023-gpt-4-technical-report/). arXiv preprint arXiv:2303.08774.
3. Alon Albalak, Liangming Pan, Colin Raffel, and William Yang Wang. 2023. [Efficient online data mixing for language model pre-training](https://md.research.elamur.ai/free/en/papers/albalak-2023-efficient-online-data-mixing-language/). arXiv preprint arXiv:2312.02406.
4. Martin Anthony and Peter L Bartlett. 1999. Neural network learning: Theoretical foundations. Cambridge University Press. [find](https://scholar.google.com/scholar?q=Martin+Anthony+and+Peter+L+Bartlett%2E+1999%2E+Neural+network+learning%3A+Theoretical+foundations%2E+Cambridge+University+Press%2E)
5. Shuai Bai, Yuxuan Cai, Ruizhe Chen, Keqin Chen, Xionghui Chen, Zesen Cheng, Lianghao Deng, Wei Ding, Chang Gao, Chunjiang Ge, Wenbin Ge, Zhifang Guo, Qidong Huang, Jie Huang, Fei Huang, Binyuan Hui, Shutong Jiang, Zhaohai Li, Mingsheng Li, and 45 others. 2025a. [Qwen3-vl technical report](https://md.research.elamur.ai/free/en/papers/bai-2025-qwen3-vl-technical-report/). arXiv prepr…
6. Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, and 1 others. 2025b. Qwen2. 5-vl technical report. arXiv preprint arXiv:2502.13923. — [Qwen2.5-VL Technical Report](https://md.research.elamur.ai/free/en/papers/bai-2025-qwen2-5-vl-technical/)
7. Kinjal Basu. 2024. Bridging knowledge gaps in llms via function calls. In Proceedings of the 33rd ACM International Conference on Information and Knowledge Management, pages 5556–5557. [find](https://scholar.google.com/scholar?q=Kinjal+Basu%2E+2024%2E+Bridging+knowledge+gaps+in+llms+via+function+calls%2E+In+Proceedings+of+the+33rd+ACM+International+Conference+on+Information+and+Knowledge+Management%2C+pages+5556%E2%80%935557%2E)
8. Kanzhi Cheng, Qiushi Sun, Yougang Chu, Fangzhi Xu, Yantao Li, Jianbing Zhang, and Zhiyong Wu. 2024. [Seeclick: Harnessing gui grounding for advanced visual gui agents](https://md.research.elamur.ai/free/en/papers/cheng-2024-seeclick-harnessing-gui-grounding-advanced/). arXiv preprint arXiv:2401.10935.
9. Chaoyou Fu, Peixian Chen, Yunhang Shen, Yulei Qin, Mengdan Zhang, Xu Lin, Jinrui Yang, Xiawu Zheng, Ke Li, Xing Sun, and 1 others. 2023. [Mme: A comprehensive evaluation benchmark for multimodal large language models](https://md.research.elamur.ai/free/en/papers/fu-2023-mme-comprehensive-evaluation-benchmark-multimodal/). arXiv preprint arXiv:2306.13394.
10. Longxi Gao, Li Zhang, Shihe Wang, Shangguang Wang, Yuanchun Li, and Mengwei Xu. 2024. Mobileviews: A large-scale mobile gui dataset. arXiv preprint arXiv:2409.14337. — [arXiv:2409.14337](https://arxiv.org/abs/2409.14337) [find](https://scholar.google.com/scholar?q=Longxi+Gao%2C+Li+Zhang%2C+Shihe+Wang%2C+Shangguang+Wang%2C+Yuanchun+Li%2C+and+Mengwei+Xu%2E+2024%2E+Mobileviews%3A+A+large%2Dscale+mobile+gui+dataset%2E+arXiv+preprint+arXiv%3A2409%2E14337%2E+%E2%80%94+%5BarXiv%3A2409%2E14337%5D%28https%3A%2F%2Farxiv)
11. Ce Ge, Zhijian Ma, Daoyuan Chen, Yaliang Li, and Bolin Ding. 2024. Bimix: Bivariate data mixing law for language model pretraining. arXiv preprint arXiv:2405.14908. — [arXiv:2405.14908](https://arxiv.org/abs/2405.14908) [find](https://scholar.google.com/scholar?q=Ce+Ge%2C+Zhijian+Ma%2C+Daoyuan+Chen%2C+Yaliang+Li%2C+and+Bolin+Ding%2E+2024%2E+Bimix%3A+Bivariate+data+mixing+law+for+language+model+pretraining%2E+arXiv+preprint+arXiv%3A2405%2E14908%2E+%E2%80%94+%5BarXiv%3A2405%2E14908%5D%28https%3A%2F%2Farxiv%2E)
12. Longling Geng and Edward Y Chang. 2025. Realm-bench: A real-world planning benchmark for llms and multi-agent systems. arXiv preprint arXiv:2502.18836. — [arXiv:2502.18836](https://arxiv.org/abs/2502.18836) [find](https://scholar.google.com/scholar?q=Longling+Geng+and+Edward+Y+Chang%2E+2025%2E+Realm%2Dbench%3A+A+real%2Dworld+planning+benchmark+for+llms+and+multi%2Dagent+systems%2E+arXiv+preprint+arXiv%3A2502%2E18836%2E+%E2%80%94+%5BarXiv%3A2502%2E18836%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2502%2E)
13. Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, and 1 others. 2024. [The llama 3 herd of models](https://md.research.elamur.ai/free/en/papers/grattafiori-2024-llama-3-herd-models/). arXiv preprint arXiv:2407.21783.
14. Shuhao Gu, Jialing Zhang, Siyuan Zhou, Kevin Yu, Zhaohu Xing, Liangdong Wang, Zhou Cao, Jintao Jia, Zhuoyi Zhang, Yixuan Wang, Zhenchong Hu, Bo-Wen Zhang, Jijie Li, Dong Liang, Yingli Zhao, Yulong Ao, Yaoqi Liu, Fangxiang Feng, and Guang Liu. 2024. [Infinity-mm: Scaling multimodal performance with la…](https://md.research.elamur.ai/free/en/papers/gu-2024-infinity-mm-scaling-multimodal-performance/)
15. David Castillo-Bolado, Joseph Davidson, Finlay Gray, and Marek Rosa. 2024. Beyond prompts: Dynamic conversational benchmarking of large language models. Preprint, arXiv:2409.20222. — [arXiv:2409.20222](https://arxiv.org/abs/2409.20222) [find](https://scholar.google.com/scholar?q=David+Castillo%2DBolado%2C+Joseph+Davidson%2C+Finlay+Gray%2C+and+Marek+Rosa%2E+2024%2E+Beyond+prompts%3A+Dynamic+conversational+benchmarking+of+large+language+models%2E+Preprint%2C+arXiv%3A2409%2E20222%2E+%E2%80%94+%5BarXiv%3A2409%2E20222)
16. Hamed Rahimi. 2024. Sujetfinancevision10k. [https://huggingface.co/datasets/sujet-ai/](https://huggingface.co/datasets/sujet-ai/) Sujet-Finance-Vision-10k. Accessed: 2025-03-12. [↗](https://huggingface.co/datasets/sujet-ai/)
17. Mingyang Chen, Haoze Sun, Tianpeng Li, Fan Yang, Hao Liang, Keer Lu, Bin Cui, Wentao Zhang, Zenan Zhou, and Weipeng Chen. 2024a. Facilitating multi-turn function calling for llms via compositional instruction tuning. arXiv preprint arXiv:2410.12952. — [arXiv:2410.12952](https://arxiv.org/abs/2410.12952) [find](https://scholar.google.com/scholar?q=Mingyang+Chen%2C+Haoze+Sun%2C+Tianpeng+Li%2C+Fan+Yang%2C+Hao+Liang%2C+Keer+Lu%2C+Bin+Cui%2C+Wentao+Zhang%2C+Zenan+Zhou%2C+and+Weipeng+Chen%2E+2024a%2E+Facilitating+multi%2Dturn+function+calling+for+llms+via+compositional+ins)
18. Chen Huang, Shuangfei Zhai, Walter Talbott, Miguel Angel Bautista, Shih-Yu Sun, Carlos Guestrin, and Josh Susskind. 2019. Addressing the loss-metric mismatch with adaptive loss alignment. Preprint, arXiv:1905.05895. — [arXiv:1905.05895](https://arxiv.org/abs/1905.05895) [find](https://scholar.google.com/scholar?q=Chen+Huang%2C+Shuangfei+Zhai%2C+Walter+Talbott%2C+Miguel+Angel+Bautista%2C+Shih%2DYu+Sun%2C+Carlos+Guestrin%2C+and+Josh+Susskind%2E+2019%2E+Addressing+the+loss%2Dmetric+mismatch+with+adaptive+loss+alignment%2E+Preprint%2C+ar)
19. Zehui Chen, Weihua Du, Wenwei Zhang, Kuikun Liu, Jiangning Liu, Miao Zheng, Jingming Zhuo, Songyang Zhang, Dahua Lin, Kai Chen, and 1 others. 2023. T-eval: Evaluating the tool utilization capability of large language models step by step. arXiv preprint arXiv:2312.14033. [find](https://scholar.google.com/scholar?q=Zehui+Chen%2C+Weihua+Du%2C+Wenwei+Zhang%2C+Kuikun+Liu%2C+Jiangning+Liu%2C+Miao+Zheng%2C+Jingming+Zhuo%2C+Songyang+Zhang%2C+Dahua+Lin%2C+Kai+Chen%2C+and+1+others%2E+2023%2E+T%2Deval%3A+Evaluating+the+tool+utilization+capability+o)
20. Tim Ingold. 2021. The perception of the environment: essays on livelihood, dwelling and skill. routledge. [find](https://scholar.google.com/scholar?q=Tim+Ingold%2E+2021%2E+The+perception+of+the+environment%3A+essays+on+livelihood%2C+dwelling+and+skill%2E+routledge%2E)
21. Zhe Chen, Weiyun Wang, Yue Cao, Yangzhou Liu, Zhangwei Gao, Erfei Cui, Jinguo Zhu, Shenglong Ye, Hao Tian, Zhaoyang Liu, and 1 others. 2024b. [Expanding performance boundaries of open-source multimodal models with model, data, and test-time scaling](https://md.research.elamur.ai/free/en/papers/chen-2024-expanding-performance-boundaries-open/). arXiv preprint arXiv:2412.05271.
22. Berivan Isik, Natalia Ponomareva, Hussein Hazimeh, Dimitris Paparas, Sergei Vassilvitskii, and Sanmi Koyejo. 2024. Scaling laws for downstream task performance of large language models. In ICLR 2024 Workshop on Mathematical and Empirical Understanding of Foundation Models. [find](https://scholar.google.com/scholar?q=Berivan+Isik%2C+Natalia+Ponomareva%2C+Hussein+Hazimeh%2C+Dimitris+Paparas%2C+Sergei+Vassilvitskii%2C+and+Sanmi+Koyejo%2E+2024%2E+Scaling+laws+for+downstream+task+performance+of+large+language+models%2E+In+ICLR+2024+W)
23. Andrei N Kolmogorov and Vladimir M Tikhomirov. 1959. ε-entropy and ε-capacity of sets in function spaces. Uspekhi Matematicheskikh Nauk, 14(2):3– 86. [find](https://scholar.google.com/scholar?q=Andrei+N+Kolmogorov+and+Vladimir+M+Tikhomirov%2E+1959%2E+%CE%B5%2Dentropy+and+%CE%B5%2Dcapacity+of+sets+in+function+spaces%2E+Uspekhi+Matematicheskikh+Nauk%2C+14%282%29%3A3%E2%80%93+86%2E)
24. Koushyar Komeilizadeh, Arne Kaps, and Fabian Dud-deck. 2023. Isovolumetric adaptations to space-filling design of experiments. Optimization and Engineering, 24(2):1267–1288. [find](https://scholar.google.com/scholar?q=Koushyar+Komeilizadeh%2C+Arne+Kaps%2C+and+Fabian+Dud%2Ddeck%2E+2023%2E+Isovolumetric+adaptations+to+space%2Dfilling+design+of+experiments%2E+Optimization+and+Engineering%2C+24%282%29%3A1267%E2%80%931288%2E)
25. Hao Li, Chenghao Yang, An Zhang, Yang Deng, Xiang Wang, and Tat-Seng Chua. 2024a. [Hello again! llm-powered personalized agent for long-term dialogue](https://md.research.elamur.ai/free/en/papers/li-2025-hello-again-llm-powered-personalized/). arXiv preprint arXiv:2406.05925.
26. Kaixin Li, Ziyang Meng, Hongzhan Lin, Ziyang Luo, Yuchen Tian, Jing Ma, Zhiyong Huang, and Tat-Seng Chua. 2025a. Screenspot-pro: Gui grounding for professional high-resolution computer use. arXiv preprint arXiv:2504.07981. — [arXiv:2504.07981](https://arxiv.org/abs/2504.07981) [find](https://scholar.google.com/scholar?q=Kaixin+Li%2C+Ziyang+Meng%2C+Hongzhan+Lin%2C+Ziyang+Luo%2C+Yuchen+Tian%2C+Jing+Ma%2C+Zhiyong+Huang%2C+and+Tat%2DSeng+Chua%2E+2025a%2E+Screenspot%2Dpro%3A+Gui+grounding+for+professional+high%2Dresolution+computer+use%2E+arXiv+prep)
27. Lingyu Li, Yixu Wang, Haiquan Zhao, Shuqi Kong, Yan Teng, Chunbo Li, and Yingchun Wang. 2024b. Reflection-bench: probing ai intelligence with reflection. arXiv preprint arXiv:2410.16270. — [arXiv:2410.16270](https://arxiv.org/abs/2410.16270) [find](https://scholar.google.com/scholar?q=Lingyu+Li%2C+Yixu+Wang%2C+Haiquan+Zhao%2C+Shuqi+Kong%2C+Yan+Teng%2C+Chunbo+Li%2C+and+Yingchun+Wang%2E+2024b%2E+Reflection%2Dbench%3A+probing+ai+intelligence+with+reflection%2E+arXiv+preprint+arXiv%3A2410%2E16270%2E+%E2%80%94+%5BarXiv%3A2410)
28. Minghao Li, Yingxiu Zhao, Bowen Yu, Feifan Song, Hangyu Li, Haiyang Yu, Zhoujun Li, Fei Huang, and Yongbin Li. 2023. [Api-bank: A comprehensive benchmark for tool-augmented llms](https://md.research.elamur.ai/free/en/papers/li-2023-api-bank-comprehensive-benchmark-tool/). Preprint, arXiv:2304.08244.
29. Yuan Li, Zhengzhong Liu, and Eric Xing. 2025b. Data mixing optimization for supervised fine-tuning of large language models. arXiv preprint arXiv:2508.11953. — [arXiv:2508.11953](https://arxiv.org/abs/2508.11953) [find](https://scholar.google.com/scholar?q=Yuan+Li%2C+Zhengzhong+Liu%2C+and+Eric+Xing%2E+2025b%2E+Data+mixing+optimization+for+supervised+fine%2Dtuning+of+large+language+models%2E+arXiv+preprint+arXiv%3A2508%2E11953%2E+%E2%80%94+%5BarXiv%3A2508%2E11953%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs)
30. Liang Xu. 2024. Superclue-agent. [https://github.com/CLUEbenchmark/SuperCLUE-Agent](https://github.com/CLUEbenchmark/SuperCLUE-Agent). Accessed: 2025-03-12. [↗](https://github.com/CLUEbenchmark/SuperCLUE-Agent)
31. P Liashchynskyi. 2019. Grid search, random search, genetic algorithm: A big comparison for nas. arXiv preprint ArXiv:1912.06059. — [arXiv:1912.06059](https://arxiv.org/abs/1912.06059) [find](https://scholar.google.com/scholar?q=P+Liashchynskyi%2E+2019%2E+Grid+search%2C+random+search%2C+genetic+algorithm%3A+A+big+comparison+for+nas%2E+arXiv+preprint+ArXiv%3A1912%2E06059%2E+%E2%80%94+%5BarXiv%3A1912%2E06059%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F1912%2E06059%29)
32. Qian Liu, Xiaosen Zheng, Niklas Muennighoff, Guangtao Zeng, Longxu Dou, Tianyu Pang, Jing Jiang, and Min Lin. 2024a. [Regmix: Data mixture as regression for language model pre-training](https://md.research.elamur.ai/free/en/papers/liu-2024-regmix-data-mixture-regression-language/). arXiv preprint arXiv:2407.01492.
33. Xiao Liu, Hao Yu, Hanchen Zhang, Yifan Xu, Xuanyu Lei, Hanyu Lai, Yu Gu, Hangliang Ding, Kaiwen Men, Kejuan Yang, and 1 others. 2023. [Agentbench: Evaluating llms as agents](https://md.research.elamur.ai/free/en/papers/liu-2023-agentbench-evaluating-llms-agents/). arXiv preprint arXiv:2308.03688.
34. Xiao Liu, Tianjie Zhang, Yu Gu, Iat Long Iong, Yifan Xu, Xixuan Song, Shudan Zhang, Hanyu Lai, Xinyi Liu, Hanlin Zhao, and 1 others. 2024b. Visualagentbench: Towards large multimodal models as visual foundation agents. arXiv preprint arXiv:2408.06327. — [arXiv:2408.06327](https://arxiv.org/abs/2408.06327) [find](https://scholar.google.com/scholar?q=Xiao+Liu%2C+Tianjie+Zhang%2C+Yu+Gu%2C+Iat+Long+Iong%2C+Yifan+Xu%2C+Xixuan+Song%2C+Shudan+Zhang%2C+Hanyu+Lai%2C+Xinyi+Liu%2C+Hanlin+Zhao%2C+and+1+others%2E+2024b%2E+Visualagentbench%3A+Towards+large+multimodal+models+as+visual+)
35. Yanming Liu, Xinyue Peng, Jiannan Cao, Shi Bo, Yuwei Zhang, Xuhong Zhang, Sheng Cheng, Xun Wang, Jianwei Yin, and Tianyu Du. 2024c. Tool-planner: Task planning with clusters across multiple tools. arXiv preprint arXiv:2406.03807. — [arXiv:2406.03807](https://arxiv.org/abs/2406.03807) [find](https://scholar.google.com/scholar?q=Yanming+Liu%2C+Xinyue+Peng%2C+Jiannan+Cao%2C+Shi+Bo%2C+Yuwei+Zhang%2C+Xuhong+Zhang%2C+Sheng+Cheng%2C+Xun+Wang%2C+Jianwei+Yin%2C+and+Tianyu+Du%2E+2024c%2E+Tool%2Dplanner%3A+Task+planning+with+clusters+across+multiple+tools%2E+arX)
36. Yuliang Liu, Zhang Li, Mingxin Huang, Biao Yang, Wenwen Yu, Chunyuan Li, Xu-Cheng Yin, Cheng-Lin Liu, Lianwen Jin, and Xiang Bai. 2024d. [Ocrbench: on the hidden mystery of ocr in large multimodal models](https://md.research.elamur.ai/free/en/papers/liu-2023-ocrbench-hidden-mystery-ocr-large/). Science China Information Sciences, 67(12).
37. Pan Lu, Swaroop Mishra, Tanglin Xia, Liang Qiu, Kai-Wei Chang, Song-Chun Zhu, Oyvind Tafjord, Peter Clark, and Ashwin Kalyan. 2022. Learn to explain: Multimodal reasoning via thought chains for science question answering. Advances in Neural Information Processing Systems, 35:2507–2521. [find](https://scholar.google.com/scholar?q=Pan+Lu%2C+Swaroop+Mishra%2C+Tanglin+Xia%2C+Liang+Qiu%2C+Kai%2DWei+Chang%2C+Song%2DChun+Zhu%2C+Oyvind+Tafjord%2C+Peter+Clark%2C+and+Ashwin+Kalyan%2E+2022%2E+Learn+to+explain%3A+Multimodal+reasoning+via+thought+chains+for+scienc)
38. Minesh Mathew, Dimosthenis Karatzas, and CV Jawahar. 2021. [Docvqa: A dataset for vqa on document images](https://md.research.elamur.ai/free/en/papers/mathew-2020-docvqa-dataset-vqa-document-images/). In Proceedings of the IEEE/CVF winter conference on applications of computer vision, pages 2200–2209.
39. Grégoire Mialon, Clémentine Fourrier, Craig Swift, Thomas Wolf, Yann LeCun, and Thomas Scialom. 2023. [Gaia: a benchmark for general ai assistants](https://md.research.elamur.ai/free/en/papers/mialon-2023-gaia-benchmark-general-ai-assistants/). Preprint, arXiv:2311.12983.
40. Paul Michel, Sebastian Ruder, and Dani Yogatama. 2021. Balancing average and worst-case accuracy in multitask learning. arXiv: Learning,arXiv: Learning. minyang. 2024. invoices-and-receipts_ocr_v1. [https://huggingface.co/datasets/mychen76/](https://huggingface.co/datasets/mychen76/) invoices-and-receipts_ocr_v1. Accessed: 2025-03-12. [↗](https://huggingface.co/datasets/mychen76/)
41. Niccolò Zanichelli. 2024. arxiv-ocr-v0.1.2. [https://huggingface.co/datasets/nz/](https://huggingface.co/datasets/nz/) arxiv-ocr-v0.1.2. Accessed: 2025-03-12. [↗](https://huggingface.co/datasets/nz/)
42. Haojie Pan, Zepeng Zhai, Hao Yuan, Yaojia Lv, Ruiji Fu, Ming Liu, Zhongyuan Wang, and Bing Qin. 2023. Kwaiagents: Generalized information-seeking agent system with large language models. arXiv preprint arXiv:2312.04889. — [arXiv:2312.04889](https://arxiv.org/abs/2312.04889) [find](https://scholar.google.com/scholar?q=Haojie+Pan%2C+Zepeng+Zhai%2C+Hao+Yuan%2C+Yaojia+Lv%2C+Ruiji+Fu%2C+Ming+Liu%2C+Zhongyuan+Wang%2C+and+Bing+Qin%2E+2023%2E+Kwaiagents%3A+Generalized+information%2Dseeking+agent+system+with+large+language+models%2E+arXiv+preprin)
43. Shishir G. Patil, Huanzhi Mao, Charlie Cheng-Jie Ji, Fanjia Yan, Vishnu Suresh, Ion Stoica, and Joseph E. Gonzalez. 2025a. The berkeley function calling leaderboard (bfcl): From tool use to agentic evaluation of large language models. In Forty-second International Conference on Machine Learning. [find](https://scholar.google.com/scholar?q=Shishir+G%2E+Patil%2C+Huanzhi+Mao%2C+Charlie+Cheng%2DJie+Ji%2C+Fanjia+Yan%2C+Vishnu+Suresh%2C+Ion+Stoica%2C+and+Joseph+E%2E+Gonzalez%2E+2025a%2E+The+berkeley+function+calling+leaderboard+%28bfcl%29%3A+From+tool+use+to+agentic+ev)
44. Shishir G. Patil, Huanzhi Mao, Charlie Cheng-Jie Ji, Fanjia Yan, Vishnu Suresh, Ion Stoica, and Joseph E. Gonzalez. 2025b. The berkeley function calling leaderboard (bfcl) v4: From tool use to agentic evaluation of large language models. In Forty-second International Conference on Machine Learning. [find](https://scholar.google.com/scholar?q=Shishir+G%2E+Patil%2C+Huanzhi+Mao%2C+Charlie+Cheng%2DJie+Ji%2C+Fanjia+Yan%2C+Vishnu+Suresh%2C+Ion+Stoica%2C+and+Joseph+E%2E+Gonzalez%2E+2025b%2E+The+berkeley+function+calling+leaderboard+%28bfcl%29+v4%3A+From+tool+use+to+agentic)
45. Shishir G Patil, Huanzhi Mao, Fanjia Yan, Charlie Cheng-Jie Ji, Vishnu Suresh, Ion Stoica, and Joseph E Gonzalez. 2025c. The berkeley function calling leaderboard (bfcl): From tool use to agentic evaluation of large language models. In Forty-second International Conference on Machine Learning. qgyd2… [find](https://scholar.google.com/scholar?q=Shishir+G+Patil%2C+Huanzhi+Mao%2C+Fanjia+Yan%2C+Charlie+Cheng%2DJie+Ji%2C+Vishnu+Suresh%2C+Ion+Stoica%2C+and+Joseph+E+Gonzalez%2E+2025c%2E+The+berkeley+function+calling+leaderboard+%28bfcl%29%3A+From+tool+use+to+agentic+eval)
46. Yujia Qin, Shihao Liang, Yining Ye, Kunlun Zhu, Lan Yan, Yaxi Lu, Yankai Lin, Xin Cong, Xiangru Tang, Bill Qian, Sihan Zhao, Lauren Hong, Runchu Tian, Ruobing Xie, Jie Zhou, Mark Gerstein, Dahai Li, Zhiyuan Liu, and Maosong Sun. 2023. [Toolllm: Facilitating large language models to master 16000+ real…](https://md.research.elamur.ai/free/en/papers/qin-2023-toolllm-facilitating-large-language-models/)
47. Shiori Sagawa, PangWei Koh, Tatsunori Hashimoto, and Percy Liang. 2020. Distributionally robust neural networks. International Conference on Learning Representations,International Conference on Learning Representations. shibing624. 2023. sharegpt_gpt4. [https://huggingface.co/datasets/shibing624/](https://huggingface.co/datasets/shibing624/) sh… [↗](https://huggingface.co/datasets/shibing624/)
48. Chan Hee Song, Jiaman Wu, Clayton Washington, Brian M Sadler, Wei-Lun Chao, and Yu Su. 2023. [Llm-planner: Few-shot grounded planning for embodied agents with large language models](https://md.research.elamur.ai/free/en/papers/song-2022-llm-planner-few-shot-grounded/). In Proceedings of the IEEE/CVF international conference on computer vision, pages 2998–3009.
49. Karthik Valmeekam, Matthew Marquez, Alberto Olmo, Sarath Sreedharan, and Subbarao Kambhampati. 2023. [Planbench: An extensible benchmark for evaluating large language models on planning and reasoning about change](https://md.research.elamur.ai/free/en/papers/valmeekam-2022-planbench-extensible-benchmark-evaluating-large/). Advances in Neural Information Processing Systems, 36:38975–38987.
50. Martin J Wainwright. 2019. High-dimensional statistics: A non-asymptotic viewpoint. Cambridge University Press. See Chapter 5 on Metric Entropy and Covering Numbers. [find](https://scholar.google.com/scholar?q=Martin+J+Wainwright%2E+2019%2E+High%2Ddimensional+statistics%3A+A+non%2Dasymptotic+viewpoint%2E+Cambridge+University+Press%2E+See+Chapter+5+on+Metric+Entropy+and+Covering+Numbers%2E)
51. Xuehui Wang, Zhenyu Wu, JingJing Xie, Zichen Ding, Bowen Yang, Zehao Li, Zhaoyang Liu, Qingyun Li, Xuan Dong, Zhe Chen, and 1 others. 2025. Mmbench-gui: Hierarchical multi-platform evaluation framework for gui agents. arXiv preprint arXiv:2507.19478. — [arXiv:2507.19478](https://arxiv.org/abs/2507.19478) [find](https://scholar.google.com/scholar?q=Xuehui+Wang%2C+Zhenyu+Wu%2C+JingJing+Xie%2C+Zichen+Ding%2C+Bowen+Yang%2C+Zehao+Li%2C+Zhaoyang+Liu%2C+Qingyun+Li%2C+Xuan+Dong%2C+Zhe+Chen%2C+and+1+others%2E+2025%2E+Mmbench%2Dgui%3A+Hierarchical+multi%2Dplatform+evaluation+framewor)
52. Yiqi Wang, Wentao Chen, Xiaotian Han, Xudong Lin, Haiteng Zhao, Yongfei Liu, Bohan Zhai, Jianbo Yuan, Quanzeng You, and Hongxia Yang. 2024. Exploring the reasoning abilities of multimodal large language models (mllms): A comprehensive survey on emerging trends in multimodal reasoning. arXiv preprint… — [arXiv:2401.06805](https://arxiv.org/abs/2401.06805) [find](https://scholar.google.com/scholar?q=Yiqi+Wang%2C+Wentao+Chen%2C+Xiaotian+Han%2C+Xudong+Lin%2C+Haiteng+Zhao%2C+Yongfei+Liu%2C+Bohan+Zhai%2C+Jianbo+Yuan%2C+Quanzeng+You%2C+and+Hongxia+Yang%2E+2024%2E+Exploring+the+reasoning+abilities+of+multimodal+large+langua)
53. Sang Michael Xie, Hieu Pham, Xuanyi Dong, Nan Du, Hanxiao Liu, Yifeng Lu, Percy S Liang, Quoc V Le, Tengyu Ma, and Adams Wei Yu. 2023a. Doremi: Optimizing data mixtures speeds up language model pretraining. Advances in Neural Information Processing Systems, 36:69798–69818. [find](https://scholar.google.com/scholar?q=Sang+Michael+Xie%2C+Hieu+Pham%2C+Xuanyi+Dong%2C+Nan+Du%2C+Hanxiao+Liu%2C+Yifeng+Lu%2C+Percy+S+Liang%2C+Quoc+V+Le%2C+Tengyu+Ma%2C+and+Adams+Wei+Yu%2E+2023a%2E+Doremi%3A+Optimizing+data+mixtures+speeds+up+language+model+pretra)
54. Sang Michael Xie, Hieu Pham, Xuanyi Dong, Nan Du, Hanxiao Liu, Yifeng Lu, Percy S Liang, Quoc V Le, Tengyu Ma, and Adams Wei Yu. 2023b. Doremi: Optimizing data mixtures speeds up language model pretraining. In Advances in Neural Information Processing Systems, volume 36, pages 69798–69818. Curran As… [find](https://scholar.google.com/scholar?q=Sang+Michael+Xie%2C+Hieu+Pham%2C+Xuanyi+Dong%2C+Nan+Du%2C+Hanxiao+Liu%2C+Yifeng+Lu%2C+Percy+S+Liang%2C+Quoc+V+Le%2C+Tengyu+Ma%2C+and+Adams+Wei+Yu%2E+2023b%2E+Doremi%3A+Optimizing+data+mixtures+speeds+up+language+model+pretra)
55. Shiming Xie, Hong Chen, Fred Yu, Zeye Sun, and Xiuyu Wu. 2024. Minor sft loss for llm fine-tune to increase performance and reduce model deviation. arXiv preprint arXiv:2408.10642. — [arXiv:2408.10642](https://arxiv.org/abs/2408.10642) [find](https://scholar.google.com/scholar?q=Shiming+Xie%2C+Hong+Chen%2C+Fred+Yu%2C+Zeye+Sun%2C+and+Xiuyu+Wu%2E+2024%2E+Minor+sft+loss+for+llm+fine%2Dtune+to+increase+performance+and+reduce+model+deviation%2E+arXiv+preprint+arXiv%3A2408%2E10642%2E+%E2%80%94+%5BarXiv%3A2408%2E10642)
56. Dmitry Yarotsky. 2017. Error bounds for approximations with deep relu networks. Neural Networks, 94:103–114. [find](https://scholar.google.com/scholar?q=Dmitry+Yarotsky%2E+2017%2E+Error+bounds+for+approximations+with+deep+relu+networks%2E+Neural+Networks%2C+94%3A103%E2%80%93114%2E)
57. Jiasheng Ye, Peiju Liu, Tianxiang Sun, Jun Zhan, Yunhua Zhou, and Xipeng Qiu. 2024. Data mixing laws: Optimizing data mixtures by predicting language modeling performance. arXiv preprint arXiv:2403.16952. [find](https://scholar.google.com/scholar?q=Jiasheng+Ye%2C+Peiju+Liu%2C+Tianxiang+Sun%2C+Jun+Zhan%2C+Yunhua+Zhou%2C+and+Xipeng+Qiu%2E+2024%2E+Data+mixing+laws%3A+Optimizing+data+mixtures+by+predicting+language+modeling+performance%2E+arXiv+preprint+arXiv%3A2403%2E16)
58. Jiakang Yuan, Tianshuo Peng, Yilei Jiang, Yiting Lu, Renrui Zhang, Kaituo Feng, Chaoyou Fu, Tao Chen, Lei Bai, Bo Zhang, and 1 others. 2025. Mme-reasoning: A comprehensive benchmark for logical reasoning in mllms. arXiv preprint arXiv:2505.21327. — [arXiv:2505.21327](https://arxiv.org/abs/2505.21327) [find](https://scholar.google.com/scholar?q=Jiakang+Yuan%2C+Tianshuo+Peng%2C+Yilei+Jiang%2C+Yiting+Lu%2C+Renrui+Zhang%2C+Kaituo+Feng%2C+Chaoyou+Fu%2C+Tao+Chen%2C+Lei+Bai%2C+Bo+Zhang%2C+and+1+others%2E+2025%2E+Mme%2Dreasoning%3A+A+comprehensive+benchmark+for+logical+reason)
59. Ruifeng Yuan, Shichao Sun, Yongqi Li, Zili Wang, Ziqiang Cao, and Wenjie Li. 2023. Personalized large language model assistant with evolving conditional memory. arXiv preprint arXiv:2312.17257. [find](https://scholar.google.com/scholar?q=Ruifeng+Yuan%2C+Shichao+Sun%2C+Yongqi+Li%2C+Zili+Wang%2C+Ziqiang+Cao%2C+and+Wenjie+Li%2E+2023%2E+Personalized+large+language+model+assistant+with+evolving+conditional+memory%2E+arXiv+preprint+arXiv%3A2312%2E17257%2E)
60. Yuanhan Zhang Bo Li Songyang Zhnag Wangbo Zhao Yike Yuan Jiaqi Wang Conghui He Ziwei Liu Kai Chen Dahua Lin Yuan Liu, Haodong Duan. 2023. [Mmbench: Is your multi-modal model an all-around player?](https://md.research.elamur.ai/free/en/papers/liu-2023-mmbench-multi-modal-model-all/) arXiv:2307.06281.
61. Xiang Yue, Yuansheng Ni, Kai Zhang, Tianyu Zheng, Ruoqi Liu, Ge Zhang, Samuel Stevens, Dongfu Jiang, Weiming Ren, Yuxuan Sun, Cong Wei, Botao Yu, Ruibin Yuan, Renliang Sun, Ming Yin, Boyuan Zheng, Zhenzhu Yang, Yibo Liu, Wenhao Huang, and 3 others. 2024. [Mmmu: A massive multi-discipline multimodal u…](https://md.research.elamur.ai/free/en/papers/yue-2023-mmmu-massive-multi-discipline-multimodal/)
62. Sewall Wright. 1921. Correlation and causation. Journal of agricultural research, 20(7):557. [find](https://scholar.google.com/scholar?q=Sewall+Wright%2E+1921%2E+Correlation+and+causation%2E+Journal+of+agricultural+research%2C+20%287%29%3A557%2E)
63. Jin Zhang, Jianyang Xue, and Bochao Cao. 2024. Improving agent performance in fluid environments by perceptual pretraining. Physics of Fluids, 36(12). [find](https://scholar.google.com/scholar?q=Jin+Zhang%2C+Jianyang+Xue%2C+and+Bochao+Cao%2E+2024%2E+Improving+agent+performance+in+fluid+environments+by+perceptual+pretraining%2E+Physics+of+Fluids%2C+36%2812%29%2E)
64. Yuxin Xiao, Shujian Zhang, Wenxuan Zhou, Marzyeh Ghassemi, and Sanqiang Zhao. 2024. [Sftmix: Elevating language model instruction tuning with mixup recipe](https://md.research.elamur.ai/free/en/papers/xiao-2026-sftmix-elevating-language-model-instruction/). arXiv preprint arXiv:2410.05248.
65. Jinguo Zhu, Weiyun Wang, Zhe Chen, Zhaoyang Liu, Shenglong Ye, Lixin Gu, Hao Tian, Yuchen Duan, Weijie Su, Jie Shao, and 1 others. 2025. [Internvl3: Exploring advanced training and test-time recipes for open-source multimodal models](https://md.research.elamur.ai/free/en/papers/zhu-2025-internvl3-exploring-advanced-training/). arXiv preprint arXiv:2504.10479.
66. Tong Zhu, Daize Dong, Xiaoye Qu, Jiacheng Ruan, Wenliang Chen, and Yu Cheng. 2024. Dynamic data mixing maximizes instruction tuning for mixture-of-experts. arXiv preprint arXiv:2406.11256. [find](https://scholar.google.com/scholar?q=Tong+Zhu%2C+Daize+Dong%2C+Xiaoye+Qu%2C+Jiacheng+Ruan%2C+Wenliang+Chen%2C+and+Yu+Cheng%2E+2024%2E+Dynamic+data+mixing+maximizes+instruction+tuning+for+mixture%2Dof%2Dexperts%2E+arXiv+preprint+arXiv%3A2406%2E11256%2E)

*Updated 1 November 2026*