---
title: "LPO — paper by Tang et al. (2026) — summary and ideas"
description: "The authors propose Location Preference Optimization (LPO), which combines two rewards… 34 references, cited by 2 papers here."
authors: ["Jiaqi Tang", "Yu Xia", "Yi-Feng Wu", "Yuwei Hu", "Chen Yuhui", "Qing-Guo Chen", "Xiaogang Xu", "Xiangyu Wu", "Hao LU", "Yanqing Ma", "Shiyin Lu", "Qifeng Chen"]
year: 2026
venue: "Findings of the Association for Computational Linguistics: ACL 2026"
source: "https://aclanthology.org/2026.findings-acl.718/"
doi: "10.18653/v1/2026.findings-acl.718"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/tang-2026-lpo-accurate-gui-agent-interaction/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/tang-2026-lpo-accurate-gui-agent-interaction/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/5/) / LPO: Towards Accurate GUI Agent Interaction via Location Preference Optimization

# LPO: Towards Accurate GUI Agent Interaction via Location Preference Optimization

**Authors:** [Jiaqi Tang](https://md.research.elamur.ai/free/en/authors/jiaqi-tang/), [Yu Xia](https://md.research.elamur.ai/free/en/authors/yu-xia/), Yi-Feng Wu, [Yuwei Hu](https://md.research.elamur.ai/free/en/authors/yuwei-hu/), Chen Yuhui, Qing-Guo Chen, Xiaogang Xu, [Xiangyu Wu](https://md.research.elamur.ai/free/en/authors/xiangyu-wu/), [Hao LU](https://md.research.elamur.ai/free/en/authors/hao-lu/), Yanqing Ma, Shiyin Lu and [Qifeng Chen](https://md.research.elamur.ai/free/en/authors/qifeng-chen/) · **Year:** 2026 · **Venue:** [Findings of the Association for Computational Linguistics: ACL 2026](https://md.research.elamur.ai/free/en/venues/findings/) ([Findings of the Association for Computational Linguistics 2026](https://md.research.elamur.ai/free/en/venues/findings/2026/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2026.findings-acl.718/) · [PDF](https://aclanthology.org/2026.findings-acl.718.pdf) · [DOI 10.18653/v1/2026.findings-acl.718](https://doi.org/10.18653/v1/2026.findings-acl.718) · [OpenAlex](https://openalex.org/W4417308198)

**Citations:** 1 citations (OpenAlex, 2026-10-14) · [2](https://md.research.elamur.ai/free/en/papers/tang-2026-lpo-accurate-gui-agent-interaction/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
GUI agents built on multimodal LLMs often click or point at the wrong spot. Supervised Fine-Tuning (SFT) handles coordinates poorly. Earlier reinforcement-learning approaches judge location coarsely, using text matching, fixed IoU thresholds, or rigid decision boundaries. UI-TARS-style DPO also needs hand-built positive and negative pairs.

The authors propose Location Preference Optimization (LPO), which combines two rewards:
- **Window-based reward:** the screenshot is split into windows, and each window is scored by its information entropy. This rests on the idea that users mostly act in information-dense regions.
- **Dynamic location reward:** a continuous reward based on the physical distance between the predicted and target coordinates.

Training follows Group Relative Preference Optimization (GRPO), which lets the agent explore the interface space broadly.

LPO is tested offline on Multimodal Mind2Web, VisualWebBench and Screenspot V2, and online on WebVoyager. It beats UI-R1, GUI-R1 and InfiGUI-R1 reward schemes applied to the same SFT base model. This matters because finer-grained spatial rewards translate directly into more reliable real-world web automation. The method's main costs are its need for well-annotated grounding data and heavy training compute.

## Key points
- On WebVoyager's nine live websites (Amazon, Apple, ArXiv, BBC News, etc.), overall task success rises from 48.0 (SFT base) to 57.6 with LPO. The strongest baseline, InfiGUI-R1, reaches 54.1.
- On Multimodal Mind2Web Cross-Task, Step SR improves from 38.2 to 49.5. Cross-Website goes from 38.4 to 46.4, and Cross-Domain from 40.7 to 49.6.
- In ablations on Cross-Task, Step SR drops to 42.3 without the distance reward r_d and to 46.4 without the entropy reward r_w. Removing r_d mainly hurts element accuracy.
- On the Screenspot V2 grounding results, LPO reaches a 90.5 average versus 89.5 for the base model. The competing reward schemes score 88.2–89.5.
- The code is announced for release at github.com/jqtangust/LPO.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Hong Kong University of Science and Technology](https://md.research.elamur.ai/free/en/universities/hong-kong-university-of-science-and-technology/) — University, Hong Kong, China · 142 works · 25,136 citations
- [Alibaba Group](https://md.research.elamur.ai/free/en/companies/alibaba-group/) — Company, Hangzhou, China · 168 works · 15,386 citations
- [Chinese University of Hong Kong](https://md.research.elamur.ai/free/en/universities/chinese-university-of-hong-kong/) — University, Hong Kong, China · 126 works · 14,073 citations
- [The Hong Kong University of Science and Technology (Guangzhou)](https://md.research.elamur.ai/free/en/universities/the-hong-kong-university-of-science-and-technology-guangzhou/) — University, Guangzhou, China · 53 works · 1,367 citations
- [Nanjing University of Science and Technology](https://md.research.elamur.ai/free/en/universities/nanjing-university-of-science-and-technology/) — University, Nanjing, China · 14 works · 1,590 citations

📚 Shelves (5): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works · [🗄️ Information Systems](https://md.research.elamur.ai/free/en/shelves/computer-science/information-systems/) · 1,437 works

🧭 Topics (1): [🖼️ Vision & language](https://md.research.elamur.ai/free/en/topics/multimodal/) · 657 works

## Cite

Cite as: Jiaqi Tang, Yu Xia, Yi-Feng Wu, Yuwei Hu, Chen Yuhui, Qing-Guo Chen, Xiaogang Xu, Xiangyu Wu, Hao LU, Yanqing Ma, Shiyin Lu and Qifeng Chen. 2026. LPO: Towards Accurate GUI Agent Interaction via Location Preference Optimization. Findings of the Association for Computational Linguistics: ACL 2026.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{tang2026lpo,
  title = {{LPO: Towards Accurate GUI Agent Interaction via Location Preference Optimization}},
  author = {Jiaqi Tang, Yu Xia, Yi-Feng Wu, Yuwei Hu, Chen Yuhui, Qing-Guo Chen, Xiaogang Xu, Xiangyu Wu, Hao LU, Yanqing Ma, Shiyin Lu and Qifeng Chen},
  booktitle = {Findings of the Association for Computational Linguistics: ACL 2026},
  year = {2026},
  doi = {10.18653/v1/2026.findings-acl.718},
  url = {https://doi.org/10.18653/v1/2026.findings-acl.718}
}
```

- [← previous Lost in Execution: On the Multilingual Robustness of Tool Calling in Large Language Models](https://md.research.elamur.ai/free/en/papers/luo-2026-lost-execution-multilingual-robustness-tool/)
- [next → MAPRO: Recasting Multi-Agent Prompt Optimization as Maximum a Posteriori Inference](https://md.research.elamur.ai/free/en/papers/zhang-2026-mapro-recasting-multi-agent-prompt/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/5/)

## References

34 references, 11 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Wentong Chen, Junbo Cui, Jinyi Hu, Yujia Qin, Junjie Fang, Yue Zhao, Chongyi Wang, Jun Liu, Guirong Chen, Yupeng Huo, and 1 others. 2024. Guicourse: From general vision language models to versatile gui agents. arXiv preprint arXiv:2406.11317. — [arXiv:2406.11317](https://arxiv.org/abs/2406.11317) [find](https://scholar.google.com/scholar?q=Wentong+Chen%2C+Junbo+Cui%2C+Jinyi+Hu%2C+Yujia+Qin%2C+Junjie+Fang%2C+Yue+Zhao%2C+Chongyi+Wang%2C+Jun+Liu%2C+Guirong+Chen%2C+Yupeng+Huo%2C+and+1+others%2E+2024%2E+Guicourse%3A+From+general+vision+language+models+to+versatile+gu)
2. Kanzhi Cheng, Qiushi Sun, Yougang Chu, Fangzhi Xu, Yantao Li, Jianbing Zhang, and Zhiyong Wu. 2024. [Seeclick: Harnessing gui grounding for advanced visual gui agents](https://md.research.elamur.ai/free/en/papers/cheng-2024-seeclick-harnessing-gui-grounding-advanced/). Preprint, arXiv:2401.10935.
3. Xiang Deng, Yu Gu, Boyuan Zheng, Shijie Chen, Sam Stevens, Boshi Wang, Huan Sun, and Yu Su. 2023a. [Mind2web: Towards a generalist agent for the web](https://md.research.elamur.ai/free/en/papers/deng-2023-mind2web-towards-generalist-agent-web/). Advances in Neural Information Processing Systems, 36:28091–28114.
4. Xiang Deng, Yu Gu, Boyuan Zheng, Shijie Chen, Samuel Stevens, Boshi Wang, Huan Sun, and Yu Su. 2023b. [Mind2web: Towards a generalist agent for the web](https://md.research.elamur.ai/free/en/papers/deng-2023-mind2web-towards-generalist-agent-web/). In Thirty-seventh Conference on Neural Information Processing Systems.
5. Hiroki Furuta, Yutaka Matsuo, Aleksandra Faust, and Izzeddin Gur. 2024. Exposing limitations of language model agents in sequential-task compositions on the web. Transactions on Machine Learning Research. [find](https://scholar.google.com/scholar?q=Hiroki+Furuta%2C+Yutaka+Matsuo%2C+Aleksandra+Faust%2C+and+Izzeddin+Gur%2E+2024%2E+Exposing+limitations+of+language+model+agents+in+sequential%2Dtask+compositions+on+the+web%2E+Transactions+on+Machine+Learning+Resea)
6. Boyu Gou, Ruohan Wang, Boyuan Zheng, Yanan Xie, Cheng Chang, Yiheng Shu, Huan Sun, and Yu Su. 2025. Navigating the digital world as humans do: Universal visual grounding for GUI agents. In The Thirteenth International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Boyu+Gou%2C+Ruohan+Wang%2C+Boyuan+Zheng%2C+Yanan+Xie%2C+Cheng+Chang%2C+Yiheng+Shu%2C+Huan+Sun%2C+and+Yu+Su%2E+2025%2E+Navigating+the+digital+world+as+humans+do%3A+Universal+visual+grounding+for+GUI+agents%2E+In+The+Thirtee)
7. Hongliang He, Wenlin Yao, Kaixin Ma, Wenhao Yu, Yong Dai, Hongming Zhang, Zhenzhong Lan, and Dong Yu. 2024. [Webvoyager: Building an end-to-end web agent with large multimodal models](https://md.research.elamur.ai/free/en/papers/he-2024-webvoyager-building-end-end-web/). Preprint, arXiv:2401.13919.
8. Wenyi Hong, Weihan Wang, Qingsong Lv, Jiazheng Xu, Wenmeng Yu, Junhui Ji, Yan Wang, Zihan Wang, Yuxiao Dong, Ming Ding, and Jie Tang. 2023. [Cogagent: A visual language model for gui agents](https://md.research.elamur.ai/free/en/papers/hong-2023-cogagent-visual-language-model/). Preprint, arXiv:2312.08914.
9. Raghav Kapoor, Yash Parag Butala, Melisa Russak, Jing Yu Koh, Kiran Kamble, Waseem AlShikh, and Ruslan Salakhutdinov. 2024. Omniact: A dataset and benchmark for enabling multimodal generalist autonomous agents for desktop and web. In European Conference on Computer Vision, pages 161– 178. Springer. [find](https://scholar.google.com/scholar?q=Raghav+Kapoor%2C+Yash+Parag+Butala%2C+Melisa+Russak%2C+Jing+Yu+Koh%2C+Kiran+Kamble%2C+Waseem+AlShikh%2C+and+Ruslan+Salakhutdinov%2E+2024%2E+Omniact%3A+A+dataset+and+benchmark+for+enabling+multimodal+generalist+autonomo)
10. Tao Li, Gang Li, Jingjie Zheng, Purple Wang, and Yang Li. 2022. Mug: Interactive multimodal grounding on user interfaces. arXiv preprint arXiv:2209.15099. [find](https://scholar.google.com/scholar?q=Tao+Li%2C+Gang+Li%2C+Jingjie+Zheng%2C+Purple+Wang%2C+and+Yang+Li%2E+2022%2E+Mug%3A+Interactive+multimodal+grounding+on+user+interfaces%2E+arXiv+preprint+arXiv%3A2209%2E15099%2E)
11. Henry Lieberman. 1997. Autonomous interface agents. In Proceedings of the ACM SIGCHI Conference on Human factors in computing systems, pages 67–74. [find](https://scholar.google.com/scholar?q=Henry+Lieberman%2E+1997%2E+Autonomous+interface+agents%2E+In+Proceedings+of+the+ACM+SIGCHI+Conference+on+Human+factors+in+computing+systems%2C+pages+67%E2%80%9374%2E)
12. Junpeng Liu, Yifan Song, Bill Yuchen Lin, Wai Lam, Graham Neubig, Yuanzhi Li, and Xiang Yue. 2024. Visualwebbench: How far have multimodal llms evolved in web page understanding and grounding? Preprint, arXiv:2404.05955. [find](https://scholar.google.com/scholar?q=Junpeng+Liu%2C+Yifan+Song%2C+Bill+Yuchen+Lin%2C+Wai+Lam%2C+Graham+Neubig%2C+Yuanzhi+Li%2C+and+Xiang+Yue%2E+2024%2E+Visualwebbench%3A+How+far+have+multimodal+llms+evolved+in+web+page+understanding+and+grounding%3F+Preprin)
13. Yuhang Liu, Pengxiang Li, Congkai Xie, Xavier Hu, Xiaotian Han, Shengyu Zhang, Hongxia Yang, and Fei Wu. 2025. Infigui-r1: Advancing multimodal gui agents from reactive actors to deliberative reasoners. Preprint, arXiv:2504.14239. — [arXiv:2504.14239](https://arxiv.org/abs/2504.14239) [find](https://scholar.google.com/scholar?q=Yuhang+Liu%2C+Pengxiang+Li%2C+Congkai+Xie%2C+Xavier+Hu%2C+Xiaotian+Han%2C+Shengyu+Zhang%2C+Hongxia+Yang%2C+and+Fei+Wu%2E+2025%2E+Infigui%2Dr1%3A+Advancing+multimodal+gui+agents+from+reactive+actors+to+deliberative+reasoner)
14. Hao Lu, Xuesong Niu, Jiyao Wang, Yin Wang, Qingyong Hu, Jiaqi Tang, Yuting Zhang, Kaishen Yuan, Bin Huang, Zitong Yu, Dengbo He, Shuiguang Deng, Hao Chen, Yingcong Chen, and Shiguang Shan. 2024a. Gpt as psychologist? preliminary evaluations for gpt-4v on visual affective computing. In Proceedings of… [find](https://scholar.google.com/scholar?q=Hao+Lu%2C+Xuesong+Niu%2C+Jiyao+Wang%2C+Yin+Wang%2C+Qingyong+Hu%2C+Jiaqi+Tang%2C+Yuting+Zhang%2C+Kaishen+Yuan%2C+Bin+Huang%2C+Zitong+Yu%2C+Dengbo+He%2C+Shuiguang+Deng%2C+Hao+Chen%2C+Yingcong+Chen%2C+and+Shiguang+Shan%2E+2024a%2E+Gpt+)
15. Shiyin Lu, Yang Li, Qing-Guo Chen, Zhao Xu, Weihua Luo, Kaifu Zhang, and Han-Jia Ye. 2024b. Ovis: Structural embedding alignment for multimodal large language model. arXiv:2405.20797. — [arXiv:2405.20797](https://arxiv.org/abs/2405.20797) [find](https://scholar.google.com/scholar?q=Shiyin+Lu%2C+Yang+Li%2C+Qing%2DGuo+Chen%2C+Zhao+Xu%2C+Weihua+Luo%2C+Kaifu+Zhang%2C+and+Han%2DJia+Ye%2E+2024b%2E+Ovis%3A+Structural+embedding+alignment+for+multimodal+large+language+model%2E+arXiv%3A2405%2E20797%2E+%E2%80%94+%5BarXiv%3A2405%2E20)
16. Yadong Lu, Jianwei Yang, Yelong Shen, and Ahmed Awadallah. 2024c. Omniparser for pure vision based gui agent. Preprint, arXiv:2408.00203. — [arXiv:2408.00203](https://arxiv.org/abs/2408.00203) [find](https://scholar.google.com/scholar?q=Yadong+Lu%2C+Jianwei+Yang%2C+Yelong+Shen%2C+and+Ahmed+Awadallah%2E+2024c%2E+Omniparser+for+pure+vision+based+gui+agent%2E+Preprint%2C+arXiv%3A2408%2E00203%2E+%E2%80%94+%5BarXiv%3A2408%2E00203%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2408%2E00203%29)
17. Zhengxi Lu, Yuxiang Chai, Yaxuan Guo, Xi Yin, Liang Liu, Hao Wang, Guanjing Xiong, and Hongsheng Li. 2025. Ui-r1: Enhancing action prediction of gui agents by reinforcement learning. arXiv preprint arXiv:2503.21620. — [arXiv:2503.21620](https://arxiv.org/abs/2503.21620) [find](https://scholar.google.com/scholar?q=Zhengxi+Lu%2C+Yuxiang+Chai%2C+Yaxuan+Guo%2C+Xi+Yin%2C+Liang+Liu%2C+Hao+Wang%2C+Guanjing+Xiong%2C+and+Hongsheng+Li%2E+2025%2E+Ui%2Dr1%3A+Enhancing+action+prediction+of+gui+agents+by+reinforcement+learning%2E+arXiv+preprint+ar)
18. Xing Han Lù, Zdeněk Kasner, and Siva Reddy. 2024. [Weblinx: Real-world website navigation with multi-turn dialogue](https://md.research.elamur.ai/free/en/papers/lu-2024-weblinx-real-world-website-navigation/). Preprint, arXiv:2402.05930.
19. Yujia Qin, Yining Ye, Junjie Fang, Haoming Wang, Shihao Liang, Shizuo Tian, Junda Zhang, Jiahao Li, Yunxin Li, Shijue Huang, Wanjun Zhong, Kuanye Li, Jiale Yang, Yu Miao, Woyu Lin, Longxiang Liu, Xu Jiang, Qianli Ma, Jingyu Li, and 16 others. 2025. [Ui-tars: Pioneering automated gui interaction with…](https://md.research.elamur.ai/free/en/papers/qin-2025-ui-tars-pioneering-automated/)
20. Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Y Wu, and 1 others. 2024. [Deepseekmath: Pushing the limits of mathematical reasoning in open language models](https://md.research.elamur.ai/free/en/papers/shao-2024-deepseekmath-pushing-limits-mathematical-reasoning/). arXiv preprint arXiv:2402.03300.
21. Qiushi Sun, Kanzhi Cheng, Zichen Ding, Chuanyang Jin, Yian Wang, Fangzhi Xu, Zhenyu Wu, Chengyou Jia, Liheng Chen, Zhoumianze Liu, and 1 others. 2024. [Os-genesis: Automating gui agent trajectory construction via reverse task synthesis](https://md.research.elamur.ai/free/en/papers/sun-2025-os-genesis-automating-gui-agent/). arXiv preprint arXiv:2412.19723.
22. Jiaqi Tang, Jianmin Chen, Wei Wei, Xiaogang Xu, Runtao Liu, Xiangyu Wu, Qipeng Xie, Jiafei Wu, Lei Zhang, and Qifeng Chen. 2026a. Robust-r1: Degradation-aware reasoning for robust visual understanding. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 40, pages 9421–9429. [find](https://scholar.google.com/scholar?q=Jiaqi+Tang%2C+Jianmin+Chen%2C+Wei+Wei%2C+Xiaogang+Xu%2C+Runtao+Liu%2C+Xiangyu+Wu%2C+Qipeng+Xie%2C+Jiafei+Wu%2C+Lei+Zhang%2C+and+Qifeng+Chen%2E+2026a%2E+Robust%2Dr1%3A+Degradation%2Daware+reasoning+for+robust+visual+understanding)
23. Jiaqi Tang, Hao Lu, Ruizheng Wu, Xiaogang Xu, Ke Ma, Cheng Fang, Bin Guo, Jiangbo Lu, Qifeng Chen, and Ying-Cong Chen. 2024a. Hawk: Learning to understand open-world video anomalies. Advances in Neural Information Processing Systems, 37:139751–139785. [find](https://scholar.google.com/scholar?q=Jiaqi+Tang%2C+Hao+Lu%2C+Ruizheng+Wu%2C+Xiaogang+Xu%2C+Ke+Ma%2C+Cheng+Fang%2C+Bin+Guo%2C+Jiangbo+Lu%2C+Qifeng+Chen%2C+and+Ying%2DCong+Chen%2E+2024a%2E+Hawk%3A+Learning+to+understand+open%2Dworld+video+anomalies%2E+Advances+in+Neura)
24. Zhiyong Wu, Zhenyu Wu, Fangzhi Xu, Yian Wang, Qiushi Sun, Chengyou Jia, Kanzhi Cheng, Zichen Ding, Liheng Chen, Paul Pu Liang, and 1 others. 2024. [Os-atlas: A foundation action model for generalist gui agents](https://md.research.elamur.ai/free/en/papers/wu-2024-os-atlas-foundation-action-model/). arXiv preprint arXiv:2410.23218.
25. Xiaobo Xia and Run Luo. 2025. Gui-r1: A generalist r1-style vision-language action model for gui agents. arXiv preprint arXiv:2504.10458. — [arXiv:2504.10458](https://arxiv.org/abs/2504.10458) [find](https://scholar.google.com/scholar?q=Xiaobo+Xia+and+Run+Luo%2E+2025%2E+Gui%2Dr1%3A+A+generalist+r1%2Dstyle+vision%2Dlanguage+action+model+for+gui+agents%2E+arXiv+preprint+arXiv%3A2504%2E10458%2E+%E2%80%94+%5BarXiv%3A2504%2E10458%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2504%2E10458%29)
26. Jianwei Yang, Hao Zhang, Feng Li, Xueyan Zou, Chunyuan Li, and Jianfeng Gao. 2023. Set-of-mark prompting unleashes extraordinary visual grounding in gpt-4v. arXiv preprint arXiv:2310.11441. [find](https://scholar.google.com/scholar?q=Jianwei+Yang%2C+Hao+Zhang%2C+Feng+Li%2C+Xueyan+Zou%2C+Chunyuan+Li%2C+and+Jianfeng+Gao%2E+2023%2E+Set%2Dof%2Dmark+prompting+unleashes+extraordinary+visual+grounding+in+gpt%2D4v%2E+arXiv+preprint+arXiv%3A2310%2E11441%2E)
27. Qiying Yu, Zheng Zhang, Ruofei Zhu, Yufeng Yuan, Xiaochen Zuo, Yu Yue, Tiantian Fan, Gaohong Liu, Lingjun Liu, Xin Liu, Haibin Lin, Zhiqi Lin, Bole Ma, Guangming Sheng, Yuxuan Tong, Chi Zhang, Mofan Zhang, Wang Zhang, Hang Zhu, and 16 others. 2025. [Dapo: An open-source llm reinforcement learning sys…](https://md.research.elamur.ai/free/en/papers/yu-2025-dapo-open-source-llm-reinforcement/)
28. Chaoyun Zhang, Shilin He, Jiaxu Qian, Bowen Li, Liqun Li, Si Qin, Yu Kang, Minghua Ma, Guyue Liu, Qingwei Lin, and 1 others. 2024a. Large language model-brained gui agents: A survey. arXiv preprint arXiv:2411.18279. — [arXiv:2411.18279](https://arxiv.org/abs/2411.18279) [find](https://scholar.google.com/scholar?q=Chaoyun+Zhang%2C+Shilin+He%2C+Jiaxu+Qian%2C+Bowen+Li%2C+Liqun+Li%2C+Si+Qin%2C+Yu+Kang%2C+Minghua+Ma%2C+Guyue+Liu%2C+Qingwei+Lin%2C+and+1+others%2E+2024a%2E+Large+language+model%2Dbrained+gui+agents%3A+A+survey%2E+arXiv+preprint+ar)
29. Jiwen Zhang, Jihao Wu, Yihua Teng, Minghui Liao, Nuo Xu, Xiao Xiao, Zhongyu Wei, and Duyu Tang. 2024b. Android in the zoo: Chain-of-action-thought for gui agents. arXiv preprint arXiv:2403.02713. [find](https://scholar.google.com/scholar?q=Jiwen+Zhang%2C+Jihao+Wu%2C+Yihua+Teng%2C+Minghui+Liao%2C+Nuo+Xu%2C+Xiao+Xiao%2C+Zhongyu+Wei%2C+and+Duyu+Tang%2E+2024b%2E+Android+in+the+zoo%3A+Chain%2Dof%2Daction%2Dthought+for+gui+agents%2E+arXiv+preprint+arXiv%3A2403%2E02713%2E)
30. Zhizheng Zhang, Wenxuan Xie, Xiaoyi Zhang, and Yan Lu. 2023. Reinforced ui instruction grounding: Towards a generic ui task automation api. arXiv preprint arXiv:2310.04716. — [arXiv:2310.04716](https://arxiv.org/abs/2310.04716) [find](https://scholar.google.com/scholar?q=Zhizheng+Zhang%2C+Wenxuan+Xie%2C+Xiaoyi+Zhang%2C+and+Yan+Lu%2E+2023%2E+Reinforced+ui+instruction+grounding%3A+Towards+a+generic+ui+task+automation+api%2E+arXiv+preprint+arXiv%3A2310%2E04716%2E+%E2%80%94+%5BarXiv%3A2310%2E04716%5D%28https%3A)
31. Jiaqi Tang, Hao Lu, Xiaogang Xu, Ruizheng Wu, Sixing Hu, Tong Zhang, Tsz Wa Cheng, Ming Ge, Ying-Cong Chen, and Fugee Tsung. 2024b. An incremental unified framework for small defect inspection. In European conference on computer vision, pages 307– 324. Springer. [find](https://scholar.google.com/scholar?q=Jiaqi+Tang%2C+Hao+Lu%2C+Xiaogang+Xu%2C+Ruizheng+Wu%2C+Sixing+Hu%2C+Tong+Zhang%2C+Tsz+Wa+Cheng%2C+Ming+Ge%2C+Ying%2DCong+Chen%2C+and+Fugee+Tsung%2E+2024b%2E+An+incremental+unified+framework+for+small+defect+inspection%2E+In+Eur)
32. Jiaqi Tang, Ruizheng Wu, Xiaogang Xu, Sixing Hu, and Ying-Cong Chen. 2024c. Learning to remove wrinkled transparent film with polarized prior. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 24987–24996. [find](https://scholar.google.com/scholar?q=Jiaqi+Tang%2C+Ruizheng+Wu%2C+Xiaogang+Xu%2C+Sixing+Hu%2C+and+Ying%2DCong+Chen%2E+2024c%2E+Learning+to+remove+wrinkled+transparent+film+with+polarized+prior%2E+In+Proceedings+of+the+IEEE%2FCVF+Conference+on+Computer+Vis)
33. Jiaqi Tang, Yingying Yan, Qianzhou Wang, Yuyang Xia, Botong Geng, Jianmin Chen, Ke Ma, Youyang Zhai, Qingfeng He, Weigeng Shao, Yunjin Sun, Junwei Dai, Chuxi Chen, Xiaogang Xu, Kelu Yao, Lei Zhang, Wei Wei, Qifeng Chen, Antonio Plaza, and Yanning Zhang. 2026b. Intelligent remote sensing agents: A su… [find](https://scholar.google.com/scholar?q=Jiaqi+Tang%2C+Yingying+Yan%2C+Qianzhou+Wang%2C+Yuyang+Xia%2C+Botong+Geng%2C+Jianmin+Chen%2C+Ke+Ma%2C+Youyang+Zhai%2C+Qingfeng+He%2C+Weigeng+Shao%2C+Yunjin+Sun%2C+Junwei+Dai%2C+Chuxi+Chen%2C+Xiaogang+Xu%2C+Kelu+Yao%2C+Lei+Zhang%2C+We)
34. Shuai Wang, Weiwen Liu, Jingxuan Chen, Yuqi Zhou, Weinan Gan, Xingshan Zeng, Yuhan Che, Shuai Yu, Xinlong Hao, Kun Shao, and 1 others. 2024. Gui agents with foundation models: A comprehensive survey. arXiv preprint arXiv:2411.04890. — [arXiv:2411.04890](https://arxiv.org/abs/2411.04890) [find](https://scholar.google.com/scholar?q=Shuai+Wang%2C+Weiwen+Liu%2C+Jingxuan+Chen%2C+Yuqi+Zhou%2C+Weinan+Gan%2C+Xingshan+Zeng%2C+Yuhan+Che%2C+Shuai+Yu%2C+Xinlong+Hao%2C+Kun+Shao%2C+and+1+others%2E+2024%2E+Gui+agents+with+foundation+models%3A+A+comprehensive+survey%2E+)

*Updated 1 November 2026*