---
title: "Small Agents, Big Gains — paper by Ping et al. (2026)"
description: "The work offers a recipe for aligning agents in open-ended domains where verifiable ground truth cannot be built. 21 references."
authors: ["Qing Ping", "Changyou Chen", "Binxuan Huang"]
year: 2026
venue: "Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 6: Industry Track)"
source: "https://aclanthology.org/2026.acl-industry.39/"
doi: "10.18653/v1/2026.acl-industry.39"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/ping-2026-small-agents-big-gains-journey/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/ping-2026-small-agents-big-gains-journey/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/43/) / Small Agents, Big Gains: Journey-Aware and Critic-Guided Simulation for Long-Horizon Shopping Dialogues

# Small Agents, Big Gains: Journey-Aware and Critic-Guided Simulation for Long-Horizon Shopping Dialogues

**Authors:** Qing Ping, [Changyou Chen](https://md.research.elamur.ai/free/en/authors/changyou-chen/) and [Binxuan Huang](https://md.research.elamur.ai/free/en/authors/binxuan-huang/) · **Year:** 2026 · **Venue:** [Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 6: Industry Track)](https://md.research.elamur.ai/free/en/venues/acl/) ([ACL 2026](https://md.research.elamur.ai/free/en/venues/acl/2026/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2026.acl-industry.39/) · [PDF](https://aclanthology.org/2026.acl-industry.39.pdf) · [DOI 10.18653/v1/2026.acl-industry.39](https://doi.org/10.18653/v1/2026.acl-industry.39) · [OpenAlex](https://openalex.org/W7166841320)

**Citations:** 0 citations (OpenAlex, 2026-10-14)

## Summary
The paper addresses a data bottleneck in building compact e-commerce assistants. Such assistants must help with inspiration, comparison and tool-backed fact-checking across non-linear shopping journeys, but real training trajectories are scarce and good answers have no single reference. The authors, from Amazon, frame trajectory synthesis as a POMDP and implement it as a closed-loop simulation with three agents:

- **User simulator:** journey-aware and stateful, tracking each sub-goal through a staged shopping funnel.
- **Shopping agent:** acts as the teacher, combining tool calls with UI-component layouts.
- **Critic agent:** scores each candidate turn against business rubrics and symbolic checks. It returns textual critiques, used as a "textual gradient," until the turn is accepted.

Accepted outputs serve as SFT data. Rejected and accepted attempts together form preference pairs for DPO. On an in-house shopping benchmark, a small model trained on this data beats same-size baselines and matches or exceeds a larger model, while serving at 8× higher throughput. The work offers a recipe for aligning agents in open-ended domains where verifiable ground truth cannot be built.

## Key points
- The user simulator follows the funnel Broad-Search → Refined-Search → Focused-Search → Evaluation → Fact-Check → Done. It keeps a per-sub-goal table of shortlisted items, current stage and knowledge gaps, and chooses one of three actions: click, chat or end. This design targets three weaknesses of naive simulators: over-revealing goals, role drift and quitting early.
- Missions are drawn from a factorized space covering product domains, mission-specific personas (with knowledge blind spots) and tool-chain complexity. Each mission spans at least k product domains.
- In an open-weight replication on Qwen3-30B-A3B, helpfulness scores were:
  - 0.714 with no SFT;
  - 0.735 with manual SFT data;
  - 0.785 with the multi-agent SFT data;
  - 0.809 with the multi-agent DPO data;
  - 0.852 for Claude Sonnet 4.5 zero-shot, used as an upper reference.
  This equals +6.8% (SFT) and +10.0% (DPO) over manual SFT. Tool-call errors fell to 0%, compared with 0.9% under manual SFT.
- The critic loop costs about 1.09 extra refinement iterations per turn of offline compute.
- The authors list three limitations: rubric-based LLM judging can introduce bias, the simulated users are assumed to be fairly rational, and the pipeline being fully synthetic avoids exposing real customer logs.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works

🧭 Topics (1): [🗨️ Dialogue systems](https://md.research.elamur.ai/free/en/topics/dialogue-systems/) · 624 works

## Cite

Cite as: Qing Ping, Changyou Chen and Binxuan Huang. 2026. Small Agents, Big Gains: Journey-Aware and Critic-Guided Simulation for Long-Horizon Shopping Dialogues. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 6: Industry Track).

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{ping2026small,
  title = {{Small Agents, Big Gains: Journey-Aware and Critic-Guided Simulation for Long-Horizon Shopping Dialogues}},
  author = {Qing Ping, Changyou Chen and Binxuan Huang},
  booktitle = {Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 6: Industry Track)},
  year = {2026},
  doi = {10.18653/v1/2026.acl-industry.39},
  url = {https://doi.org/10.18653/v1/2026.acl-industry.39}
}
```

- [← previous SLPGFJWUInsa at SemEval-2026 Task 1: Enhancing Linguistic Creativity for English Text-Based Humor](https://md.research.elamur.ai/free/en/papers/abbas-2026-slpgfjwuinsa-semeval-2026-task-1/)
- [next → Small Can Be Beautiful in LLMs for SSH: a Case for Bulgarian](https://md.research.elamur.ai/free/en/papers/simov-2026-small-can-be-beautiful-llms/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/43/)

## References

21 references, 4 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Xiangci Li, Zhiyu Chen, Jason Ingyu Choi, Nikhita Vedula, Besnik Fetahu, Oleg Rokhlenko, and Shervin Malmasi. 2025a. Wizard of shopping: Target-oriented E-commerce dialogue generation with decision tree branching. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguis… [find](https://scholar.google.com/scholar?q=Xiangci+Li%2C+Zhiyu+Chen%2C+Jason+Ingyu+Choi%2C+Nikhita+Vedula%2C+Besnik+Fetahu%2C+Oleg+Rokhlenko%2C+and+Shervin+Malmasi%2E+2025a%2E+Wizard+of+shopping%3A+Target%2Doriented+E%2Dcommerce+dialogue+generation+with+decision+tr)
2. Jafar Afzali, Aleksander Mark Drzewiecki, Krisztian Balog, and Shuo Zhang. 2023. Usersimcrs: A user simulation toolkit for evaluating conversational recommender systems. In Proceedings of the Sixteenth Yafu Li, Xuyang Hu, Xiaoye Qu, Linjie Li, and Yu Cheng. 2025b. Test-time preference optimization:… [find](https://scholar.google.com/scholar?q=Jafar+Afzali%2C+Aleksander+Mark+Drzewiecki%2C+Krisztian+Balog%2C+and+Shuo+Zhang%2E+2023%2E+Usersimcrs%3A+A+user+simulation+toolkit+for+evaluating+conversational+recommender+systems%2E+In+Proceedings+of+the+Sixteent)
3. Chen Luo, Dimitri Papadimitriou, Hariharan Muralidharan, Dhineshkumar Ramasubbu, Aakash Kolekar, Wenju Xu, Cong Xu, Anirudh Srinivasan, Mukesh Jain, and Qi He. 2025. Language model alignment for conversational shopping at amazon. In Proceedings of the 48th International ACM SIGIR Conference on Resea… [find](https://scholar.google.com/scholar?q=Chen+Luo%2C+Dimitri+Papadimitriou%2C+Hariharan+Muralidharan%2C+Dhineshkumar+Ramasubbu%2C+Aakash+Kolekar%2C+Wenju+Xu%2C+Cong+Xu%2C+Anirudh+Srinivasan%2C+Mukesh+Jain%2C+and+Qi+He%2E+2025%2E+Language+model+alignment+for+conve)
4. Xiang Luo, Zhiwen Tang, Jin Wang, and Xuejie Zhang. 2024. DuetSim: Building user simulator with dual large language models for task-oriented dialogues. In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024), pages… [find](https://scholar.google.com/scholar?q=Xiang+Luo%2C+Zhiwen+Tang%2C+Jin+Wang%2C+and+Xuejie+Zhang%2E+2024%2E+DuetSim%3A+Building+user+simulator+with+dual+large+language+models+for+task%2Doriented+dialogues%2E+In+Proceedings+of+the+2024+Joint+International+C)
5. Aman Madaan, Niket Tandon, Prakhar Gupta, Skyler Hallinan, Luyu Gao, Sarah Wiegreffe, Uri Alon, Nouha Dziri, Shrimai Prabhumoye, Yiming Yang, Shashank Gupta, Bodhisattwa Prasad Majumder, Katherine Hermann, Sean Welleck, Amir Yazdanbakhsh, and Peter Clark. 2023. Self-refine: iterative refinement with… [find](https://scholar.google.com/scholar?q=Aman+Madaan%2C+Niket+Tandon%2C+Prakhar+Gupta%2C+Skyler+Hallinan%2C+Luyu+Gao%2C+Sarah+Wiegreffe%2C+Uri+Alon%2C+Nouha+Dziri%2C+Shrimai+Prabhumoye%2C+Yiming+Yang%2C+Shashank+Gupta%2C+Bodhisattwa+Prasad+Majumder%2C+Katherine+Her)
6. Shuhaib Mehri, Xiaocheng Yang, Takyoung Kim, Gokhan Tur, Shikib Mehri, and Dilek Hakkani-Tür. 2025. [Goal alignment in LLM-based user simulators for conversational AI](https://md.research.elamur.ai/free/en/papers/mehri-2026-goal-alignment-llm-based-user/). In First Workshop on Multi-Turn Interactions in Large Language Models (MTI-LLM) at NeurIPS 2025.
7. Yu Meng, Mengzhou Xia, and Danqi Chen. 2024. SimPO: Simple preference optimization with a reference-free reward. In The Thirty-eighth Annual Conference on Neural Information Processing Systems. [find](https://scholar.google.com/scholar?q=Yu+Meng%2C+Mengzhou+Xia%2C+and+Danqi+Chen%2E+2024%2E+SimPO%3A+Simple+preference+optimization+with+a+reference%2Dfree+reward%2E+In+The+Thirty%2Deighth+Annual+Conference+on+Neural+Information+Processing+Systems%2E)
8. Tarek Naous, Philippe Laban, Wei Xu, and Jennifer Neville. 2026. Flipping the dialogue: Training and evaluating user language models. In The Fourteenth International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Tarek+Naous%2C+Philippe+Laban%2C+Wei+Xu%2C+and+Jennifer+Neville%2E+2026%2E+Flipping+the+dialogue%3A+Training+and+evaluating+user+language+models%2E+In+The+Fourteenth+International+Conference+on+Learning+Representat)
9. OpenAI. 2025. Introducing shopping research in Chat-GPT. [https://openai.com/index/chatgpt-s](https://openai.com/index/chatgpt-s) hopping-research/. Published: Nov. 24, 2025. Accessed: 2025-12-29. [↗](https://openai.com/index/chatgpt-s)
10. Jeonghoon Shim, Woojung Song, Cheyon Jin, Seung-won KooK, and Yohan Jo. 2026. Non-collaborative user simulators for tool agents. In The Fourteenth International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Jeonghoon+Shim%2C+Woojung+Song%2C+Cheyon+Jin%2C+Seung%2Dwon+KooK%2C+and+Yohan+Jo%2E+2026%2E+Non%2Dcollaborative+user+simulators+for+tool+agents%2E+In+The+Fourteenth+International+Conference+on+Learning+Representations%2E)
11. Noah Shinn, Federico Cassano, Ashwin Gopinath, Karthik Narasimhan, and Shunyu Yao. 2023. [Reflexion: language agents with verbal reinforcement learning](https://md.research.elamur.ai/free/en/papers/shinn-2023-reflexion-language-agents-verbal-reinforcement/). In Proceedings of the 37th International Conference on Neural Information Processing Systems, volume 36, pages 8634–8652.
12. Walmart. 2025. Walmart: The future of shopping is agentic. meet sparky. [https://corporate.walm](https://corporate.walm) art.com/news/2025/06/06/walmart-the-fut ure-of-shopping-is-agentic-meet-sparky. Published: Jun. 6, 2025. Accessed: 2025-12-29. [↗](https://corporate.walm)
13. Kuang Wang, Xianfei Li, Shenghao Yang, Li Zhou, Feng Jiang, and Haizhou Li. 2025. Know you first and be you better: Modeling human-like user simulators via implicit profiles. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 210… [find](https://scholar.google.com/scholar?q=Kuang+Wang%2C+Xianfei+Li%2C+Shenghao+Yang%2C+Li+Zhou%2C+Feng+Jiang%2C+and+Haizhou+Li%2E+2025%2E+Know+you+first+and+be+you+better%3A+Modeling+human%2Dlike+user+simulators+via+implicit+profiles%2E+In+Proceedings+of+the+63r)
14. Liqiang Xiao, Jun Ma, Xin Luna Dong, Pascual Martínez-Gómez, Nasser Zalmout, Chenwei Zhang, Tong Zhao, Hao He, and Yaohui Jin. 2021. End-to-end conversational search for online shopping with utterance transfer. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing… [find](https://scholar.google.com/scholar?q=Liqiang+Xiao%2C+Jun+Ma%2C+Xin+Luna+Dong%2C+Pascual+Mart%C3%ADnez%2DG%C3%B3mez%2C+Nasser+Zalmout%2C+Chenwei+Zhang%2C+Tong+Zhao%2C+Hao+He%2C+and+Yaohui+Jin%2E+2021%2E+End%2Dto%2Dend+conversational+search+for+online+shopping+with+utterance)
15. An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, Chujie Zheng, Dayiheng Liu, Fan Zhou, Fei Huang, Feng Hu, Hao Ge, Haoran Wei, Huan Lin, Jialong Tang, and 41 others. 2025. [Qwen3 technical report](https://md.research.elamur.ai/free/en/papers/yang-2025-qwen3-technical-report/). Preprint, arXiv:2505.09388.
16. Mert Yuksekgonul, Federico Bianchi, Joseph Boen, Sheng Liu, Zhi Huang, Carlos Guestrin, and James Zou. 2024. [Textgrad: Automatic "differentiation" via text](https://md.research.elamur.ai/free/en/papers/yuksekgonul-2024-textgrad-automatic-differentiation-text/). Preprint, arXiv:2406.07496.
17. Qiusi Zhan, Xiaojie Guo, Heng Ji, and Lingfei Wu. 2023. User simulator assisted open-ended conversational recommendation system. In Proceedings of the 5th Workshop on NLP for Conversational AI (NLP4ConvAI 2023), pages 89–101. [find](https://scholar.google.com/scholar?q=Qiusi+Zhan%2C+Xiaojie+Guo%2C+Heng+Ji%2C+and+Lingfei+Wu%2E+2023%2E+User+simulator+assisted+open%2Dended+conversational+recommendation+system%2E+In+Proceedings+of+the+5th+Workshop+on+NLP+for+Conversational+AI+%28NLP4Co)
18. Perplexity. 2024. Shop like a pro: Perplexity’s new AI-powered shopping assistant. [https://www.pe](https://www.pe) rplexity.ai/hub/blog/shop-like-a-pro. Published: Nov. 18, 2024. Accessed: 2025-12-29. [↗](https://www.pe)
19. Zijian Zhang, Shuchang Liu, Ziru Liu, Rui Zhong, Qingpeng Cai, Xiangyu Zhao, Chunxu Zhang, Qidong Liu, and Peng Jiang. 2025. LLM-powered user simulator for recommender system. In Proceedings of the Thirty-Ninth AAAI Conference on Artificial Intelligence, volume 39, pages 13339–13347. [find](https://scholar.google.com/scholar?q=Zijian+Zhang%2C+Shuchang+Liu%2C+Ziru+Liu%2C+Rui+Zhong%2C+Qingpeng+Cai%2C+Xiangyu+Zhao%2C+Chunxu+Zhang%2C+Qidong+Liu%2C+and+Peng+Jiang%2E+2025%2E+LLM%2Dpowered+user+simulator+for+recommender+system%2E+In+Proceedings+of+the+Th)
20. Ivan Sekulic, Silvia Terragni, Victor Guimarães, Nghia Khau, Bruna Guedes, Modestas Filipavicius, Andre Ferreira Manso, and Roland Mathis. 2024. Reliable LLM-based user simulator for task-oriented dialogue systems. In Proceedings of the 1st Workshop on Simulating Conversational Intelligence in Chat… [find](https://scholar.google.com/scholar?q=Ivan+Sekulic%2C+Silvia+Terragni%2C+Victor+Guimar%C3%A3es%2C+Nghia+Khau%2C+Bruna+Guedes%2C+Modestas+Filipavicius%2C+Andre+Ferreira+Manso%2C+and+Roland+Mathis%2E+2024%2E+Reliable+LLM%2Dbased+user+simulator+for+task%2Doriented+dia)
21. Jie Zou, Mohammad Aliannejadi, Evangelos Kanoulas, Shuxi Han, Heli Ma, Zheng Wang, Yang Yang, and Heng Tao Shen. 2025. PSCon: Product search through conversations. In Proceedings of the 48th International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 3659—-3669. [find](https://scholar.google.com/scholar?q=Jie+Zou%2C+Mohammad+Aliannejadi%2C+Evangelos+Kanoulas%2C+Shuxi+Han%2C+Heli+Ma%2C+Zheng+Wang%2C+Yang+Yang%2C+and+Heng+Tao+Shen%2E+2025%2E+PSCon%3A+Product+search+through+conversations%2E+In+Proceedings+of+the+48th+Internati)

*Updated 1 November 2026*