---
title: "Towards Scalable Lightweight GUI Agents via Multi-role"
description: "The authors present LAMO, a framework that teaches a single lightweight MLLM to take on several cooperating… 5.1 points. 60 references."
authors: ["Ziwei Wang", "Junjie Zheng", "Leyang Yang", "Sheng Zhou", "Xiaoxuan Tang", "Fang Zhouhua", "Zhiwei Liu", "Dajun Chen", "Yong Li", "Jiajun Bu"]
year: 2026
venue: "Findings of the Association for Computational Linguistics: ACL 2026"
source: "https://aclanthology.org/2026.findings-acl.1122/"
doi: "10.18653/v1/2026.findings-acl.1122"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/wang-2026-scalable-lightweight-gui-agents-multi/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/wang-2026-scalable-lightweight-gui-agents-multi/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/46/) / Towards Scalable Lightweight GUI Agents via Multi-role Orchestration

# Towards Scalable Lightweight GUI Agents via Multi-role Orchestration

**Authors:** [Ziwei Wang](https://md.research.elamur.ai/free/en/authors/ziwei-wang/), Junjie Zheng, Leyang Yang, Sheng Zhou, Xiaoxuan Tang, [Fang Zhouhua](https://md.research.elamur.ai/free/en/authors/fang-zhouhua/), [Zhiwei Liu](https://md.research.elamur.ai/free/en/authors/zhiwei-liu/), [Dajun Chen](https://md.research.elamur.ai/free/en/authors/dajun-chen/), [Yong Li](https://md.research.elamur.ai/free/en/authors/yong-li/) and [Jiajun Bu](https://md.research.elamur.ai/free/en/authors/jiajun-bu/) · **Year:** 2026 · **Venue:** [Findings of the Association for Computational Linguistics: ACL 2026](https://md.research.elamur.ai/free/en/venues/findings/) ([Findings of the Association for Computational Linguistics 2026](https://md.research.elamur.ai/free/en/venues/findings/2026/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2026.findings-acl.1122/) · [PDF](https://aclanthology.org/2026.findings-acl.1122.pdf) · [DOI 10.18653/v1/2026.findings-acl.1122](https://doi.org/10.18653/v1/2026.findings-acl.1122) · [OpenAlex](https://openalex.org/W7166819443)

**Citations:** 0 citations (OpenAlex, 2026-10-14)

## Summary
Multimodal LLM agents that operate phone and desktop interfaces work best at large scale, but such models are too costly to run on end-user devices. Small models trained end-to-end on whole episodes struggle with realistic, long workflows and cannot easily act as components of multi-agent systems (MAS). The authors present LAMO, a framework that teaches a single lightweight MLLM to take on several cooperating roles, such as planner, observer and executor, through shared parameters and context engineering. Training data is synthesized per role with teacher models. Training then has two stages: SFT with a Perplexity-Weighted Cross-Entropy (PWCE) loss, followed by multi-task RL in which the roles explore jointly. The resulting LAMO-3B, built on Qwen2.5-VL-3B, can run as a standalone agent, as a self-orchestrated MAS, or as a plug-in policy executor under a stronger planner. Paired with GPT-5 as planner, it outperforms earlier state-of-the-art systems on online benchmarks. This suggests that small on-device executors can keep improving as planners improve.

## Key points
- **Role-oriented data synthesis:** GUI automation is split into five skills:
  - Action–Tool Alignment (ATA)
  - Logic-Consistent CoT (LCC)
  - Screen Understanding (SU)
  - Goal Planning (GP)
  - Screen Grounding (SG)

  Qwen-2.5-VL-72B-Instruct served as a teacher for ATA and SG.
- **Benchmarks:** static evaluation used ScreenSpot-pro and AndroidControl; online evaluation used MiniWob++, AndroidWorld and OSWorld.
- **With a GPT-5 planner:** LAMO-3B beats Mobile-Agent-V3 by 5.9% and UI-Venus-Navi-72B by 17.8%. It also exceeds the UI-Ins-7B executor by 3.5 points.
- **As an OSWorld executor** (39 tasks, at most 50 steps):
  - It beats Qwen3-VL-4B by 15.6% and UI-TARS-1.5-7B by 36.5%.
  - It trails Qwen2.5-VL-32B by only 5.1 points while using about 10× fewer parameters.
  - InfiGUI-R1-3B, which shares the same backbone, scores 28.2 points lower.
- **Ablations:**
  - Compared with base Qwen2.5-VL-3B, the full recipe raises scores on ScreenSpot, ScreenSpot-v2, ScreenSpot-pro and MiniWob++ by 7.7%, 6.3%, 51.0% and 44.5%.
  - Replacing PWCE with plain cross-entropy costs about 2.2% on average.
  - Removing the intricate-layout grounding (ILG) data used in RL drops ScreenSpot-pro by 34.7%.
  - Stated limitations are tasks longer than 10 steps and visually complex desktop applications.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🖱️ Human-Computer Interaction](https://md.research.elamur.ai/free/en/shelves/computer-science/human-computer-interaction/) · 141 works · [🌐 Computer Networks and Communications](https://md.research.elamur.ai/free/en/shelves/computer-science/computer-networks-and-communications/) · 64 works

## Cite

Cite as: Ziwei Wang, Junjie Zheng, Leyang Yang, Sheng Zhou, Xiaoxuan Tang, Fang Zhouhua, Zhiwei Liu, Dajun Chen, Yong Li and Jiajun Bu. 2026. Towards Scalable Lightweight GUI Agents via Multi-role Orchestration. Findings of the Association for Computational Linguistics: ACL 2026.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{wang2026scalable,
  title = {{Towards Scalable Lightweight GUI Agents via Multi-role Orchestration}},
  author = {Ziwei Wang, Junjie Zheng, Leyang Yang, Sheng Zhou, Xiaoxuan Tang, Fang Zhouhua, Zhiwei Liu, Dajun Chen, Yong Li and Jiajun Bu},
  booktitle = {Findings of the Association for Computational Linguistics: ACL 2026},
  year = {2026},
  doi = {10.18653/v1/2026.findings-acl.1122},
  url = {https://doi.org/10.18653/v1/2026.findings-acl.1122}
}
```

- [← previous Towards Robust Real-World Spreadsheet Understanding with Multi-Agent Multi-Format Reasoning](https://md.research.elamur.ai/free/en/papers/ren-2026-robust-real-world-spreadsheet-understanding/)
- [next → Towards the First NLP Benchmark for Ladin - an Extremely Low-Resource Language](https://md.research.elamur.ai/free/en/papers/nuha-2026-first-nlp-benchmark-ladin-extremely/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/46/)

## References

60 references, 18 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Saaket Agashe, Kyle Wong, Vincent Tu, Jiachen Yang, Ang Li, and Xin Eric Wang. 2025. Agent s2: A compositional generalist-specialist framework for computer use agents. arXiv preprint arXiv:2504.00906. — [arXiv:2504.00906](https://arxiv.org/abs/2504.00906) [find](https://scholar.google.com/scholar?q=Saaket+Agashe%2C+Kyle+Wong%2C+Vincent+Tu%2C+Jiachen+Yang%2C+Ang+Li%2C+and+Xin+Eric+Wang%2E+2025%2E+Agent+s2%3A+A+compositional+generalist%2Dspecialist+framework+for+computer+use+agents%2E+arXiv+preprint+arXiv%3A2504%2E00906%2E)
2. Anthropic. 2024. Developing a computer use model. [find](https://scholar.google.com/scholar?q=Anthropic%2E+2024%2E+Developing+a+computer+use+model%2E)
3. Shuai Bai, Yuxuan Cai, Ruizhe Chen, Keqin Chen, Xionghui Chen, Zesen Cheng, Lianghao Deng, Wei Ding, Chang Gao, Chunjiang Ge, Wenbin Ge, Zhifang Guo, Qidong Huang, Jie Huang, Fei Huang, Binyuan Hui, Shutong Jiang, Zhaohai Li, Mingsheng Li, and 45 others. 2025a. [Qwen3-vl technical report](https://md.research.elamur.ai/free/en/papers/bai-2025-qwen3-vl-technical-report/). Preprint, a…
4. Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, Humen Zhong, Yuanzhi Zhu, Mingkun Yang, Zhaohai Li, Jianqiang Wan, Pengfei Wang, Wei Ding, Zheren Fu, Yiheng Xu, and 8 others. 2025b. [Qwen2.5-vl technical report](https://md.research.elamur.ai/free/en/papers/bai-2025-qwen2-5-vl-technical/). Preprint, arXiv:2502.1…
5. Yuxiang Chai, Siyuan Huang, Yazhe Niu, Han Xiao, Liang Liu, Dingyu Zhang, Peng Gao, Shuai Ren, and Hongsheng Li. 2024. Amex: Android multi-annotation expo dataset for mobile gui agents. arXiv preprint arXiv:2407.17490. [find](https://scholar.google.com/scholar?q=Yuxiang+Chai%2C+Siyuan+Huang%2C+Yazhe+Niu%2C+Han+Xiao%2C+Liang+Liu%2C+Dingyu+Zhang%2C+Peng+Gao%2C+Shuai+Ren%2C+and+Hongsheng+Li%2E+2024%2E+Amex%3A+Android+multi%2Dannotation+expo+dataset+for+mobile+gui+agents%2E+arXiv+preprint)
6. Liangyu Chen, Hanzhang Zhou, Chenglin Cai, Jianan Zhang, Panrong Tong, Quyu Kong, Xu Zhang, Chen Liu, Yuqi Liu, Wenxuan Wang, and 1 others. 2025a. Ui-ins: Enhancing gui grounding with multi-perspective instruction-as-reasoning. arXiv preprint arXiv:2510.20286. — [arXiv:2510.20286](https://arxiv.org/abs/2510.20286) [find](https://scholar.google.com/scholar?q=Liangyu+Chen%2C+Hanzhang+Zhou%2C+Chenglin+Cai%2C+Jianan+Zhang%2C+Panrong+Tong%2C+Quyu+Kong%2C+Xu+Zhang%2C+Chen+Liu%2C+Yuqi+Liu%2C+Wenxuan+Wang%2C+and+1+others%2E+2025a%2E+Ui%2Dins%3A+Enhancing+gui+grounding+with+multi%2Dperspectiv)
7. Weizhi Chen, Ziwei Wang, Leyang Yang, Sheng Zhou, Xiaoxuan Tang, Jiajun Bu, Yong Li, and Wei Jiang. 2025b. Pg-agent: An agent powered by page graph. In Proceedings of the 33rd ACM International Conference on Multimedia, pages 6878–6887. [find](https://scholar.google.com/scholar?q=Weizhi+Chen%2C+Ziwei+Wang%2C+Leyang+Yang%2C+Sheng+Zhou%2C+Xiaoxuan+Tang%2C+Jiajun+Bu%2C+Yong+Li%2C+and+Wei+Jiang%2E+2025b%2E+Pg%2Dagent%3A+An+agent+powered+by+page+graph%2E+In+Proceedings+of+the+33rd+ACM+International+Confer)
8. Zhe Chen, Jiannan Wu, Wenhai Wang, Weijie Su, Guo Chen, Sen Xing, Muyan Zhong, Qinglong Zhang, Xizhou Zhu, Lewei Lu, and 1 others. 2024. [Internvl: Scaling up vision foundation models and aligning for generic visual-linguistic tasks](https://md.research.elamur.ai/free/en/papers/chen-2023-internvl-scaling-up-vision/). In Proceedings of the IEEE/CVF conference on computer vision and pa…
9. Kanzhi Cheng, Qiushi Sun, Yougang Chu, Fangzhi Xu, Li YanTao, Jianbing Zhang, and Zhiyong Wu. 2024. [Seeclick: Harnessing gui grounding for advanced visual gui agents](https://md.research.elamur.ai/free/en/papers/cheng-2024-seeclick-harnessing-gui-grounding-advanced/). In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 9313–9332.
10. Gemini Team. 2025. [Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilities](https://md.research.elamur.ai/free/en/papers/comanici-2025-gemini-2-5-pushing-frontier/). Technical report, Google DeepMind.
11. Gonzalo Gonzalez-Pumariega, Vincent Tu, Chih-Lun Lee, Jiachen Yang, Ang Li, and Xin Eric Wang. 2025. The unreasonable effectiveness of scaling agents for computer use. arXiv preprint arXiv:2510.02250. — [arXiv:2510.02250](https://arxiv.org/abs/2510.02250) [find](https://scholar.google.com/scholar?q=Gonzalo+Gonzalez%2DPumariega%2C+Vincent+Tu%2C+Chih%2DLun+Lee%2C+Jiachen+Yang%2C+Ang+Li%2C+and+Xin+Eric+Wang%2E+2025%2E+The+unreasonable+effectiveness+of+scaling+agents+for+computer+use%2E+arXiv+preprint+arXiv%3A2510%2E02250%2E)
12. Boyu Gou, Ruohan Wang, Boyuan Zheng, Yanan Xie, Cheng Chang, Yiheng Shu, Huan Sun, and Yu Su. 2025. Navigating the digital world as humans do: Universal visual grounding for gui agents. arXiv preprint arXiv:2410.05243. [find](https://scholar.google.com/scholar?q=Boyu+Gou%2C+Ruohan+Wang%2C+Boyuan+Zheng%2C+Yanan+Xie%2C+Cheng+Chang%2C+Yiheng+Shu%2C+Huan+Sun%2C+and+Yu+Su%2E+2025%2E+Navigating+the+digital+world+as+humans+do%3A+Universal+visual+grounding+for+gui+agents%2E+arXiv+preprint)
13. Ming Gu, Ziwei Wang, Sicen Lai, Zirui Gao, Sheng Zhou, and Jiajun Bu. 2026. Towards scalable web accessibility audit with mllms as copilots. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 40, pages 38515–38523. [find](https://scholar.google.com/scholar?q=Ming+Gu%2C+Ziwei+Wang%2C+Sicen+Lai%2C+Zirui+Gao%2C+Sheng+Zhou%2C+and+Jiajun+Bu%2E+2026%2E+Towards+scalable+web+accessibility+audit+with+mllms+as+copilots%2E+In+Proceedings+of+the+AAAI+Conference+on+Artificial+Intelli)
14. Zhangxuan Gu, Zhengwen Zeng, Zhenyu Xu, Xingran Zhou, Shuheng Shen, Yunfei Liu, Beitong Zhou, Changhua Meng, Tianyu Xia, Weizhi Chen, and 1 others. 2025. Ui-venus technical report: Building high-performance ui agents with rft. arXiv preprint arXiv:2508.10833. — [arXiv:2508.10833](https://arxiv.org/abs/2508.10833) [find](https://scholar.google.com/scholar?q=Zhangxuan+Gu%2C+Zhengwen+Zeng%2C+Zhenyu+Xu%2C+Xingran+Zhou%2C+Shuheng+Shen%2C+Yunfei+Liu%2C+Beitong+Zhou%2C+Changhua+Meng%2C+Tianyu+Xia%2C+Weizhi+Chen%2C+and+1+others%2E+2025%2E+Ui%2Dvenus+technical+report%3A+Building+high%2Dperfo)
15. Wenyi Hong, Weihan Wang, Qingsong Lv, Jiazheng Xu, Wenmeng Yu, Junhui Ji, Yan Wang, Zihan Wang, Yuxiao Dong, Ming Ding, and Jie Tang. 2024. [Cogagent: A visual language model for GUI agents](https://md.research.elamur.ai/free/en/papers/hong-2023-cogagent-visual-language-model/). In IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2024, Seattle, WA, USA, June 16-22, 20…
16. Yuyang Hu, Shichun Liu, Yanwei Yue, Guibin Zhang, Boyang Liu, Fangyi Zhu, Jiahang Lin, Honglin Guo, Shihan Dou, Zhiheng Xi, Senjie Jin, and et al. 2025. Memory in the age of ai agents. Preprint, arXiv:2512.13564. — [arXiv:2512.13564](https://arxiv.org/abs/2512.13564) [find](https://scholar.google.com/scholar?q=Yuyang+Hu%2C+Shichun+Liu%2C+Yanwei+Yue%2C+Guibin+Zhang%2C+Boyang+Liu%2C+Fangyi+Zhu%2C+Jiahang+Lin%2C+Honglin+Guo%2C+Shihan+Dou%2C+Zhiheng+Xi%2C+Senjie+Jin%2C+and+et+al%2E+2025%2E+Memory+in+the+age+of+ai+agents%2E+Preprint%2C+arXiv)
17. Aaron Hurst, Adam Lerer, Adam P Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Ostrow, Akila Welihinda, Alan Hayes, Alec Radford, and 1 others. 2024. [Gpt-4o system card](https://md.research.elamur.ai/free/en/papers/openai-2024-gpt-4o-system-card/). arXiv preprint arXiv:2410.21276.
18. Jared Kaplan, Sam McCandlish, Tom Henighan, Tom B Brown, Benjamin Chess, Rewon Child, Scott Gray, Alec Radford, Jeffrey Wu, and Dario Amodei. 2020. [Scaling laws for neural language models](https://md.research.elamur.ai/free/en/papers/kaplan-2020-scaling-laws-neural-language-models/). arXiv preprint arXiv:2001.08361.
19. Kaixin Li, Ziyang Meng, Hongzhan Lin, Ziyang Luo, Yuchen Tian, Jing Ma, Zhiyong Huang, and Tat-Seng Chua. 2025a. Screenspot-pro: Gui grounding for professional high-resolution computer use. In Proceedings of the 33rd ACM International Conference on Multimedia, pages 8778–8786. [find](https://scholar.google.com/scholar?q=Kaixin+Li%2C+Ziyang+Meng%2C+Hongzhan+Lin%2C+Ziyang+Luo%2C+Yuchen+Tian%2C+Jing+Ma%2C+Zhiyong+Huang%2C+and+Tat%2DSeng+Chua%2E+2025a%2E+Screenspot%2Dpro%3A+Gui+grounding+for+professional+high%2Dresolution+computer+use%2E+In+Proceed)
20. Kaixin Li, Ziyang Meng, Hongzhan Lin, Ziyang Luo, Yuchen Tian, Jing Ma, Zhiyong Huang, and Tat-Seng Chua. 2025b. Screenspot-pro: Gui grounding for professional high-resolution computer use. arXiv preprint arXiv:2504.07981. — [arXiv:2504.07981](https://arxiv.org/abs/2504.07981) [find](https://scholar.google.com/scholar?q=Kaixin+Li%2C+Ziyang+Meng%2C+Hongzhan+Lin%2C+Ziyang+Luo%2C+Yuchen+Tian%2C+Jing+Ma%2C+Zhiyong+Huang%2C+and+Tat%2DSeng+Chua%2E+2025b%2E+Screenspot%2Dpro%3A+Gui+grounding+for+professional+high%2Dresolution+computer+use%2E+arXiv+prep)
21. Wei Li, William Bishop, Alice Li, Chris Rawles, Folawiyo Campbell-Ajala, Divya Tyamagundlu, and Oriana Riva. 2024. On the effects of data scale on ui control agents. Preprint, arXiv:2406.03679. [find](https://scholar.google.com/scholar?q=Wei+Li%2C+William+Bishop%2C+Alice+Li%2C+Chris+Rawles%2C+Folawiyo+Campbell%2DAjala%2C+Divya+Tyamagundlu%2C+and+Oriana+Riva%2E+2024%2E+On+the+effects+of+data+scale+on+ui+control+agents%2E+Preprint%2C+arXiv%3A2406%2E03679%2E)
22. Kevin Qinghong Lin, Linjie Li, Difei Gao, Zhengyuan Yang, Shiwei Wu, Zechen Bai, Stan Weixian Lei, Lijuan Wang, and Mike Zheng Shou. 2025. Showui: One vision-language-action model for gui visual agent. In Proceedings of the Computer Vision and Pattern Recognition Conference, pages 19498– 19508. [find](https://scholar.google.com/scholar?q=Kevin+Qinghong+Lin%2C+Linjie+Li%2C+Difei+Gao%2C+Zhengyuan+Yang%2C+Shiwei+Wu%2C+Zechen+Bai%2C+Stan+Weixian+Lei%2C+Lijuan+Wang%2C+and+Mike+Zheng+Shou%2E+2025%2E+Showui%3A+One+vision%2Dlanguage%2Daction+model+for+gui+visual+agent)
23. Evan Zheran Liu, Kelvin Guu, Panupong Pasupat, Tianlin Shi, and Percy Liang. 2018. Reinforcement learning on web interfaces using workflow-guided exploration. arXiv preprint arXiv:1802.08802. [find](https://scholar.google.com/scholar?q=Evan+Zheran+Liu%2C+Kelvin+Guu%2C+Panupong+Pasupat%2C+Tianlin+Shi%2C+and+Percy+Liang%2E+2018%2E+Reinforcement+learning+on+web+interfaces+using+workflow%2Dguided+exploration%2E+arXiv+preprint+arXiv%3A1802%2E08802%2E)
24. Nelson F. Liu, Kevin Lin, John Hewitt, Ashwin Paranjape, Michele Bevilacqua, Fabio Petroni, and Percy Liang. 2023. [Lost in the middle: How language models use long contexts](https://md.research.elamur.ai/free/en/papers/liu-2023-lost-middle-language-models-use/). Preprint, arXiv:2307.03172.
25. Yuhang Liu, Pengxiang Li, Congkai Xie, Xavier Hu, Xiaotian Han, Shengyu Zhang, Hongxia Yang, and Fei Wu. 2025. Infigui-r1: Advancing multimodal gui agents from reactive actors to deliberative reasoners. Preprint, arXiv:2504.14239. — [arXiv:2504.14239](https://arxiv.org/abs/2504.14239) [find](https://scholar.google.com/scholar?q=Yuhang+Liu%2C+Pengxiang+Li%2C+Congkai+Xie%2C+Xavier+Hu%2C+Xiaotian+Han%2C+Shengyu+Zhang%2C+Hongxia+Yang%2C+and+Fei+Wu%2E+2025%2E+Infigui%2Dr1%3A+Advancing+multimodal+gui+agents+from+reactive+actors+to+deliberative+reasoner)
26. Quanfeng Lu, Wenqi Shao, Zitao Liu, Fanqing Meng, Boxuan Li, Botong Chen, Siyuan Huang, Kaipeng Zhang, Yu Qiao, and Ping Luo. 2024. Gui odyssey: A comprehensive dataset for cross-app gui navigation on mobile devices. arXiv preprint arXiv:2406.08451. — [arXiv:2406.08451](https://arxiv.org/abs/2406.08451) [find](https://scholar.google.com/scholar?q=Quanfeng+Lu%2C+Wenqi+Shao%2C+Zitao+Liu%2C+Fanqing+Meng%2C+Boxuan+Li%2C+Botong+Chen%2C+Siyuan+Huang%2C+Kaipeng+Zhang%2C+Yu+Qiao%2C+and+Ping+Luo%2E+2024%2E+Gui+odyssey%3A+A+comprehensive+dataset+for+cross%2Dapp+gui+navigation+on)
27. Zhengxi Lu, Yuxiang Chai, Yaxuan Guo, Xi Yin, Liang Liu, Hao Wang, Han Xiao, Shuai Ren, Guanjing Xiong, and Hongsheng Li. 2025a. Ui-r1: Enhancing action prediction of gui agents by reinforcement learning. arXiv preprint arXiv:2503.21620. — [arXiv:2503.21620](https://arxiv.org/abs/2503.21620) [find](https://scholar.google.com/scholar?q=Zhengxi+Lu%2C+Yuxiang+Chai%2C+Yaxuan+Guo%2C+Xi+Yin%2C+Liang+Liu%2C+Hao+Wang%2C+Han+Xiao%2C+Shuai+Ren%2C+Guanjing+Xiong%2C+and+Hongsheng+Li%2E+2025a%2E+Ui%2Dr1%3A+Enhancing+action+prediction+of+gui+agents+by+reinforcement+learn)
28. Zhengxi Lu, Jiabo Ye, Fei Tang, Yongliang Shen, Haiyang Xu, Ziwei Zheng, Weiming Lu, Ming Yan, Fei Huang, Jun Xiao, and 1 others. 2025b. Ui-s1: Advancing gui automation via semi-online reinforcement learning. arXiv preprint arXiv:2509.11543. — [arXiv:2509.11543](https://arxiv.org/abs/2509.11543) [find](https://scholar.google.com/scholar?q=Zhengxi+Lu%2C+Jiabo+Ye%2C+Fei+Tang%2C+Yongliang+Shen%2C+Haiyang+Xu%2C+Ziwei+Zheng%2C+Weiming+Lu%2C+Ming+Yan%2C+Fei+Huang%2C+Jun+Xiao%2C+and+1+others%2E+2025b%2E+Ui%2Ds1%3A+Advancing+gui+automation+via+semi%2Donline+reinforcement+l)
29. Run Luo, Lu Wang, Wanwei He, and Xiaobo Xia. 2025. Gui-r1: A generalist r1-style vision-language action model for gui agents. arXiv preprint arXiv:2504.10458. — [arXiv:2504.10458](https://arxiv.org/abs/2504.10458) [find](https://scholar.google.com/scholar?q=Run+Luo%2C+Lu+Wang%2C+Wanwei+He%2C+and+Xiaobo+Xia%2E+2025%2E+Gui%2Dr1%3A+A+generalist+r1%2Dstyle+vision%2Dlanguage+action+model+for+gui+agents%2E+arXiv+preprint+arXiv%3A2504%2E10458%2E+%E2%80%94+%5BarXiv%3A2504%2E10458%5D%28https%3A%2F%2Farxiv%2Eorg%2Fab)
30. China Mobile. 2025. Jt-guiagent-v1: A planner-grounder agent for reliable gui interaction. Project Website. [find](https://scholar.google.com/scholar?q=China+Mobile%2E+2025%2E+Jt%2Dguiagent%2Dv1%3A+A+planner%2Dgrounder+agent+for+reliable+gui+interaction%2E+Project+Website%2E)
31. OpenAI. 2025a. Developing a generalist computer-using agent. [find](https://scholar.google.com/scholar?q=OpenAI%2E+2025a%2E+Developing+a+generalist+computer%2Dusing+agent%2E)
32. OpenAI. 2025b. Gpt-5.1 model overview. Internal model release; no peer-reviewed technical report available at the time of writing. [find](https://scholar.google.com/scholar?q=OpenAI%2E+2025b%2E+Gpt%2D5%2E1+model+overview%2E+Internal+model+release%3B+no+peer%2Dreviewed+technical+report+available+at+the+time+of+writing%2E)
33. Chanwoo Park, Seungju Han, Xingzhi Guo, Asuman Ozdaglar, Kaiqing Zhang, and Joo-Kyung Kim. 2025a. [Maporl: Multi-agent post-co-training for collaborative large language models with reinforcement learning](https://md.research.elamur.ai/free/en/papers/park-2025-maporl-multi-agent-post-co/). Preprint, arXiv:2502.18439.
34. Joonhyung Park, Peng Tang, Sagnik Das, Srikar Appalaraju, Kunwar Yashraj Singh, R. Manmatha, and Shabnam Ghadar. 2025b. R-vlm: Region-aware vision language model for precise gui grounding. Preprint, arXiv:2507.05673. [find](https://scholar.google.com/scholar?q=Joonhyung+Park%2C+Peng+Tang%2C+Sagnik+Das%2C+Srikar+Appalaraju%2C+Kunwar+Yashraj+Singh%2C+R%2E+Manmatha%2C+and+Shabnam+Ghadar%2E+2025b%2E+R%2Dvlm%3A+Region%2Daware+vision+language+model+for+precise+gui+grounding%2E+Preprint%2C+a)
35. Rui Qian, Xin Yin, Chuanhang Deng, Zhiyuan Peng, Jian Xiong, Wei Zhai, and Dejing Dou. 2025. Uground: Towards unified visual grounding with unrolled transformers. CoRR, abs/2510.03853. — [arXiv:2510.03853](https://arxiv.org/abs/2510.03853) [find](https://scholar.google.com/scholar?q=Rui+Qian%2C+Xin+Yin%2C+Chuanhang+Deng%2C+Zhiyuan+Peng%2C+Jian+Xiong%2C+Wei+Zhai%2C+and+Dejing+Dou%2E+2025%2E+Uground%3A+Towards+unified+visual+grounding+with+unrolled+transformers%2E+CoRR%2C+abs%2F2510%2E03853%2E+%E2%80%94+%5BarXiv%3A2510%2E0)
36. Yujia Qin, Yining Ye, Junjie Fang, Haoming Wang, Shihao Liang, Shizuo Tian, Junda Zhang, Jiahao Li, Yunxin Li, Shijue Huang, and 1 others. 2025. [Ui-tars: Pioneering automated gui interaction with native agents](https://md.research.elamur.ai/free/en/papers/qin-2025-ui-tars-pioneering-automated/). arXiv preprint arXiv:2501.12326.
37. Christopher Rawles, Sarah Clinckemaillie, Yifan Chang, Jonathan Waltz, Gabrielle Lau, Marybeth Fair, Alice Li, William Bishop, Wei Li, Folawiyo Campbell-Ajala, and 1 others. 2024. Androidworld: A dynamic benchmarking environment for autonomous agents. arXiv preprint arXiv:2405.14573. [find](https://scholar.google.com/scholar?q=Christopher+Rawles%2C+Sarah+Clinckemaillie%2C+Yifan+Chang%2C+Jonathan+Waltz%2C+Gabrielle+Lau%2C+Marybeth+Fair%2C+Alice+Li%2C+William+Bishop%2C+Wei+Li%2C+Folawiyo+Campbell%2DAjala%2C+and+1+others%2E+2024%2E+Androidworld%3A+A+dyna)
38. Christopher Rawles, Alice Li, Daniel Rodriguez, Oriana Riva, and Timothy Lillicrap. 2023. An-droidinthewild: A large-scale dataset for android device control. Advances in Neural Information Processing Systems, 36:59708–59728. [find](https://scholar.google.com/scholar?q=Christopher+Rawles%2C+Alice+Li%2C+Daniel+Rodriguez%2C+Oriana+Riva%2C+and+Timothy+Lillicrap%2E+2023%2E+An%2Ddroidinthewild%3A+A+large%2Dscale+dataset+for+android+device+control%2E+Advances+in+Neural+Information+Processing)
39. ByteDance Seed. 2025. seed-tars.com/1.5. [find](https://scholar.google.com/scholar?q=ByteDance+Seed%2E+2025%2E+seed%2Dtars%2Ecom%2F1%2E5%2E)
40. Ui-tars-1.5. https://Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Y Wu, and 1 others. 2024. [Deepseekmath: Pushing the limits of mathematical reasoning in open language models](https://md.research.elamur.ai/free/en/papers/shao-2024-deepseekmath-pushing-limits-mathematical-reasoning/). arXiv preprint arXiv:2402.03300.
41. Qiushi Sun, Kanzhi Cheng, Zichen Ding, Chuanyang Jin, Yian Wang, Fangzhi Xu, Zhenyu Wu, Chengyou Jia, Liheng Chen, Zhoumianze Liu, and 1 others. 2025. [Os-genesis: Automating gui agent trajectory construction via reverse task synthesis](https://md.research.elamur.ai/free/en/papers/sun-2025-os-genesis-automating-gui-agent/). In Proceedings of the 63rd Annual Meeting of the Association for…
42. Fei Tang, Haolei Xu, Hang Zhang, Siqi Chen, Xingyu Wu, Yongliang Shen, Wenqi Zhang, Guiyang Hou, Zeqi Tan, Yuchen Yan, Kaitao Song, Jian Shao, Weiming Lu, Jun Xiao, and Yueting Zhuang. 2025. A survey on (m)llm-based gui agents. Preprint, arXiv:2504.13865. — [arXiv:2504.13865](https://arxiv.org/abs/2504.13865) [find](https://scholar.google.com/scholar?q=Fei+Tang%2C+Haolei+Xu%2C+Hang+Zhang%2C+Siqi+Chen%2C+Xingyu+Wu%2C+Yongliang+Shen%2C+Wenqi+Zhang%2C+Guiyang+Hou%2C+Zeqi+Tan%2C+Yuchen+Yan%2C+Kaitao+Song%2C+Jian+Shao%2C+Weiming+Lu%2C+Jun+Xiao%2C+and+Yueting+Zhuang%2E+2025%2E+A+survey+)
43. Kimi Team, Angang Du, Bohong Yin, Bowei Xing, Bowen Qu, Bowen Wang, Cheng Chen, Chenlin Zhang, Chenzhuang Du, Chu Wei, and 1 others. 2025. [Kimi-vl technical report](https://md.research.elamur.ai/free/en/papers/kimi-team-2025-kimi-vl-technical-report/). arXiv preprint arXiv:2504.07491.
44. Tesseract OCR. 2025. Tesseract OCR: Tesseract open source ocr engine. [https://github.com/](https://github.com/) tesseract-ocr/tesseract. Accessed: 2025-12-15. [↗](https://github.com/)
45. Unotools. 2025. Unotools 0.3.3. [https://pypi.org/](https://pypi.org/) project/unotools/. Python Package Index. [↗](https://pypi.org/)
46. Junyang Wang, Haiyang Xu, Haitao Jia, Xi Zhang, Ming Yan, Weizhou Shen, Ji Zhang, Fei Huang, and Jitao Sang. 2024. Mobile-agent-v2: Mobile device operation assistant with effective navigation via multi-agent collaboration. Preprint, arXiv:2406.01014. — [arXiv:2406.01014](https://arxiv.org/abs/2406.01014) [find](https://scholar.google.com/scholar?q=Junyang+Wang%2C+Haiyang+Xu%2C+Haitao+Jia%2C+Xi+Zhang%2C+Ming+Yan%2C+Weizhou+Shen%2C+Ji+Zhang%2C+Fei+Huang%2C+and+Jitao+Sang%2E+2024%2E+Mobile%2Dagent%2Dv2%3A+Mobile+device+operation+assistant+with+effective+navigation+via+mult)
47. Xinyuan Wang, Bowen Wang, Dunjie Lu, Junlin Yang, Tianbao Xie, Junli Wang, Jiaqi Deng, Xiaole Guo, Yiheng Xu, Chen Henry Wu, and 1 others. 2025a. Opencua: Open foundations for computer-use agents. arXiv preprint arXiv:2508.09123. [find](https://scholar.google.com/scholar?q=Xinyuan+Wang%2C+Bowen+Wang%2C+Dunjie+Lu%2C+Junlin+Yang%2C+Tianbao+Xie%2C+Junli+Wang%2C+Jiaqi+Deng%2C+Xiaole+Guo%2C+Yiheng+Xu%2C+Chen+Henry+Wu%2C+and+1+others%2E+2025a%2E+Opencua%3A+Open+foundations+for+computer%2Duse+agents%2E+arX)
48. Ziwei Wang, Weizhi Chen, Leyang Yang, Sheng Zhou, Shengchu Zhao, Hanbei Zhan, Jiongchao Jin, Liangcheng Li, Zirui Shao, and Jiajun Bu. 2025b. Mp-gui: Modality perception with mllms for gui understanding. In Proceedings of the Computer Vision and Pattern Recognition Conference, pages 29711– 29721. [find](https://scholar.google.com/scholar?q=Ziwei+Wang%2C+Weizhi+Chen%2C+Leyang+Yang%2C+Sheng+Zhou%2C+Shengchu+Zhao%2C+Hanbei+Zhan%2C+Jiongchao+Jin%2C+Liangcheng+Li%2C+Zirui+Shao%2C+and+Jiajun+Bu%2E+2025b%2E+Mp%2Dgui%3A+Modality+perception+with+mllms+for+gui+understandi)
49. Ziwei Wang, Leyang Yang, Xiaoxuan Tang, Sheng Zhou, Dajun Chen, Wei Jiang, and Yong Li. 2025c. History-aware reasoning for gui agents. arXiv preprint arXiv:2511.09127. — [arXiv:2511.09127](https://arxiv.org/abs/2511.09127) [find](https://scholar.google.com/scholar?q=Ziwei+Wang%2C+Leyang+Yang%2C+Xiaoxuan+Tang%2C+Sheng+Zhou%2C+Dajun+Chen%2C+Wei+Jiang%2C+and+Yong+Li%2E+2025c%2E+History%2Daware+reasoning+for+gui+agents%2E+arXiv+preprint+arXiv%3A2511%2E09127%2E+%E2%80%94+%5BarXiv%3A2511%2E09127%5D%28https%3A%2F%2Farx)
50. Zhiyong Wu, Zhenyu Wu, Fangzhi Xu, Yian Wang, Qiushi Sun, Chengyou Jia, Kanzhi Cheng, Zichen Ding, Liheng Chen, Paul Pu Liang, and 1 others. 2024. [Os-atlas: A foundation action model for generalist gui agents](https://md.research.elamur.ai/free/en/papers/wu-2024-os-atlas-foundation-action-model/). arXiv preprint arXiv:2410.23218.
51. Tianbao Xie, Danyang Zhang, Jixuan Chen, Xiaochuan Li, Siheng Zhao, Ruisheng Cao, Toh J Hua, Zhoujun Cheng, Dongchan Shin, Fangyu Lei, and 1 others. 2024. [Osworld: Benchmarking multimodal agents for open-ended tasks in real computer environments](https://md.research.elamur.ai/free/en/papers/xie-2024-osworld-benchmarking-multimodal-agents-open/). Advances in Neural Information Processing Systems, 37…
52. Yiheng Xu, Zekun Wang, Junli Wang, Dunjie Lu, Tianbao Xie, Amrita Saha, Doyen Sahoo, Tao Yu, and Caiming Xiong. 2024. Aguvis: Unified pure vision agents for autonomous gui interaction. arXiv preprint arXiv:2412.04454. — [arXiv:2412.04454](https://arxiv.org/abs/2412.04454) [find](https://scholar.google.com/scholar?q=Yiheng+Xu%2C+Zekun+Wang%2C+Junli+Wang%2C+Dunjie+Lu%2C+Tianbao+Xie%2C+Amrita+Saha%2C+Doyen+Sahoo%2C+Tao+Yu%2C+and+Caiming+Xiong%2E+2024%2E+Aguvis%3A+Unified+pure+vision+agents+for+autonomous+gui+interaction%2E+arXiv+preprint+)
53. Leyang Yang, Ziwei Wang, Xiaoxuan Tang, Sheng Zhou, Dajun Chen, Wei Jiang, and Yong Li. 2025a. Probench: Benchmarking gui agents with accurate process information. arXiv preprint arXiv:2511.09157. — [arXiv:2511.09157](https://arxiv.org/abs/2511.09157) [find](https://scholar.google.com/scholar?q=Leyang+Yang%2C+Ziwei+Wang%2C+Xiaoxuan+Tang%2C+Sheng+Zhou%2C+Dajun+Chen%2C+Wei+Jiang%2C+and+Yong+Li%2E+2025a%2E+Probench%3A+Benchmarking+gui+agents+with+accurate+process+information%2E+arXiv+preprint+arXiv%3A2511%2E09157%2E+%E2%80%94+%5B)
54. Yuhao Yang, Yue Wang, Dongxu Li, Ziyang Luo, Bei Chen, Chao Huang, and Junnan Li. 2025b. Aria-ui: Visual grounding for gui instructions. In Findings of the Association for Computational Linguistics: ACL 2025, pages 22418–22433. [find](https://scholar.google.com/scholar?q=Yuhao+Yang%2C+Yue+Wang%2C+Dongxu+Li%2C+Ziyang+Luo%2C+Bei+Chen%2C+Chao+Huang%2C+and+Junnan+Li%2E+2025b%2E+Aria%2Dui%3A+Visual+grounding+for+gui+instructions%2E+In+Findings+of+the+Association+for+Computational+Linguistics%3A+A)
55. Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. 2023. [React: Synergizing reasoning and acting in language models](https://md.research.elamur.ai/free/en/papers/yao-2022-react-synergizing-reasoning-acting-language/). Preprint, arXiv:2210.03629.
56. Jiabo Ye, Xi Zhang, Haiyang Xu, Haowei Liu, Junyang Wang, Zhaoqing Zhu, Ziwei Zheng, Feiyu Gao, Junjie Cao, Zhengxi Lu, and 1 others. 2025. Mobile-agent-v3: Fundamental agents for gui automation. arXiv preprint arXiv:2508.15144. — [arXiv:2508.15144](https://arxiv.org/abs/2508.15144) [find](https://scholar.google.com/scholar?q=Jiabo+Ye%2C+Xi+Zhang%2C+Haiyang+Xu%2C+Haowei+Liu%2C+Junyang+Wang%2C+Zhaoqing+Zhu%2C+Ziwei+Zheng%2C+Feiyu+Gao%2C+Junjie+Cao%2C+Zhengxi+Lu%2C+and+1+others%2E+2025%2E+Mobile%2Dagent%2Dv3%3A+Fundamental+agents+for+gui+automation%2E+arXi)
57. Chi Zhang, Zhao Yang, Jiaxuan Liu, Yanda Li, Yucheng Han, Xin Chen, Zebiao Huang, Bin Fu, and Gang Yu. 2025a. Appagent: Multimodal agents as smartphone users. In Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems, pages 1–20. [find](https://scholar.google.com/scholar?q=Chi+Zhang%2C+Zhao+Yang%2C+Jiaxuan+Liu%2C+Yanda+Li%2C+Yucheng+Han%2C+Xin+Chen%2C+Zebiao+Huang%2C+Bin+Fu%2C+and+Gang+Yu%2E+2025a%2E+Appagent%3A+Multimodal+agents+as+smartphone+users%2E+In+Proceedings+of+the+2025+CHI+Conference)
58. Zhong Zhang, Yaxi Lu, Yikun Fu, Yupeng Huo, Shenzhi Yang, Yesai Wu, Han Si, Xin Cong, Haotian Chen, Yankai Lin, and 1 others. 2025b. Agentcpm-gui: Building mobile-use agents with reinforcement fine-tuning. arXiv preprint arXiv:2506.01391. [find](https://scholar.google.com/scholar?q=Zhong+Zhang%2C+Yaxi+Lu%2C+Yikun+Fu%2C+Yupeng+Huo%2C+Shenzhi+Yang%2C+Yesai+Wu%2C+Han+Si%2C+Xin+Cong%2C+Haotian+Chen%2C+Yankai+Lin%2C+and+1+others%2E+2025b%2E+Agentcpm%2Dgui%3A+Building+mobile%2Duse+agents+with+reinforcement+fine%2Dtu)
59. Yuan Zhao, Hualei Zhu, Tingyu Jiang, Shen Li, Xiaohang Xu, and Hao Henry Wang. 2025. Co-epg: A framework for co-evolution of planning and grounding in autonomous gui agents. arXiv preprint arXiv:2511.10705. — [arXiv:2511.10705](https://arxiv.org/abs/2511.10705) [find](https://scholar.google.com/scholar?q=Yuan+Zhao%2C+Hualei+Zhu%2C+Tingyu+Jiang%2C+Shen+Li%2C+Xiaohang+Xu%2C+and+Hao+Henry+Wang%2E+2025%2E+Co%2Depg%3A+A+framework+for+co%2Devolution+of+planning+and+grounding+in+autonomous+gui+agents%2E+arXiv+preprint+arXiv%3A2511%2E)
60. Jinguo Zhu, Weiyun Wang, Zhe Chen, Zhaoyang Liu, Shenglong Ye, Lixin Gu, Hao Tian, Yuchen Duan, Weijie Su, Jie Shao, Zhangwei Gao, Erfei Cui, Xuehui Wang, Yue Cao, Yangzhou Liu, Xingguang Wei, Hongjie Zhang, Haomin Wang, Weiye Xu, and 4 others. 2025. [Internvl3: Exploring advanced training and test-t…](https://md.research.elamur.ai/free/en/papers/zhu-2025-internvl3-exploring-advanced-training/)

*Updated 1 November 2026*