---
title: "SmartAD — paper by Tang and Zhao (2026)"
description: "The authors argue that this ignores what a small student can actually absorb. 21 references, cited by 1 paper here."
authors: ["Guokai Tang", "Feng Zhao"]
year: 2026
venue: "Findings of the Association for Computational Linguistics: ACL 2026"
source: "https://aclanthology.org/2026.findings-acl.1349/"
doi: "10.18653/v1/2026.findings-acl.1349"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/tang-2026-smartad-capacity-aligned-agent-distillation/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/tang-2026-smartad-capacity-aligned-agent-distillation/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/17/) / SmartAD: Capacity-Aligned Agent Distillation for Small Language Models

# SmartAD: Capacity-Aligned Agent Distillation for Small Language Models

**Authors:** Guokai Tang and [Feng Zhao](https://md.research.elamur.ai/free/en/authors/feng-zhao/) · **Year:** 2026 · **Venue:** [Findings of the Association for Computational Linguistics: ACL 2026](https://md.research.elamur.ai/free/en/venues/findings/) ([Findings of the Association for Computational Linguistics 2026](https://md.research.elamur.ai/free/en/venues/findings/2026/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2026.findings-acl.1349/) · [PDF](https://aclanthology.org/2026.findings-acl.1349.pdf) · [DOI 10.18653/v1/2026.findings-acl.1349](https://doi.org/10.18653/v1/2026.findings-acl.1349) · [OpenAlex](https://openalex.org/W7166893590)

**Citations:** 0 citations (OpenAlex, 2026-10-14) · 1 in our library, of the 149,439 papers whose bibliographies we read

## Summary
The paper tackles the transfer of tool-using agent behaviour from large language models to small ones (1.5B and 3B parameters). Existing agent distillation pipelines treat every correct teacher trajectory as equally good training material and apply the same token-level loss across the whole trajectory. The authors argue that this ignores what a small student can actually absorb. Their framework, SmartAD, makes two changes.

1. **Data selection.** Several successful teacher trajectories are sampled per question, and the student keeps the one it already finds most probable, measured by its own negative log-likelihood (NLL).
2. **Weighted supervision.** Training uses a loss that gives increasing weight to reasoning, then action, then final-decision segments.

Trajectories follow the CodeAct thought–action–observation format, and observations are excluded from the loss. On multi-hop QA and math benchmarks, SmartAD beat prompting and distillation baselines at both model sizes and also transferred to out-of-domain tasks. The work matters for building cheaper, faster agents: it shows that matching training signals to student capacity improves what small models learn from large ones.

## Key points
- **Selection procedure.** For each question, K = 10 teacher trajectories are sampled at temperature 1.0. Trajectories with wrong answers or failed tool calls are discarded. The survivor with the lowest student NLL is kept, where NLL is computed as a macro-average over assistant turns so that long turns do not dominate.
- **Segment-weighted loss.** Each trajectory is split into explanatory reasoning, concrete actions (code or search calls) and final-decision spans, with progressively larger weights in that order.
- **Evidence for NLL as a difficulty signal.** Analysis on HotpotQA, MATH-HARD and MATH-MEDIUM shows that trajectories with lower student NLL tend to be executed correctly more often and use fewer tool calls. One exception appears under global bucketing on MATH-HARD, which supports comparing NLL within each question rather than across questions.
- **Results.** SmartAD outperformed all baselines with both 1.5B and 3B students.
- **Limitations.** Sampling multiple trajectories and scoring them with the student raises offline data-generation cost. Test-time scaling methods were not explored.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Huazhong University of Science and Technology](https://md.research.elamur.ai/free/en/universities/huazhong-university-of-science-and-technology/) — University, Wuhan, China · 59 works · 8,310 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 18,614 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 17,965 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 17,450 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 17,296 works

🧭 Topics (1): [🪶 Efficient models & compression](https://md.research.elamur.ai/free/en/topics/efficient-models/) · 297 works

## Cite

Cite as: Guokai Tang and Feng Zhao. 2026. SmartAD: Capacity-Aligned Agent Distillation for Small Language Models. Findings of the Association for Computational Linguistics: ACL 2026.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{tang2026smartad,
  title = {{SmartAD: Capacity-Aligned Agent Distillation for Small Language Models}},
  author = {Guokai Tang and Feng Zhao},
  booktitle = {Findings of the Association for Computational Linguistics: ACL 2026},
  year = {2026},
  doi = {10.18653/v1/2026.findings-acl.1349},
  url = {https://doi.org/10.18653/v1/2026.findings-acl.1349}
}
```

- [← previous Smart_solutions at MEDIQA-SYNUR 2026: A Multi-Stage LLM Pipeline for Nursing Observation Extraction](https://md.research.elamur.ai/free/en/papers/munjal-2026-smart-solutions-mediqa-synur-2026/)
- [next → SMARTER: A Data-efficient Framework to Improve Toxicity Detection with Explanation via Self-augmenting Large Language Models](https://md.research.elamur.ai/free/en/papers/nghiem-2026-smarter-data-efficient-framework-improve/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/17/)

## References

21 references, 12 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. DeepSeek-AI, Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, Xiaokang Zhang, Xingkai Yu, Yu Wu, Z. F. Wu, Zhibin Gou, Zhihong Shao, Zhuoshu Li, Ziyi Gao, and 181 others. 2025. [DeepSeek-R1: Incentivizing reasoning capability in LL…](https://md.research.elamur.ai/free/en/papers/deepseek-ai-2025-deepseek-r1-incentivizing-reasoning-capability/)
2. Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. 2021. [Measuring mathematical problem solving with the MATH dataset](https://md.research.elamur.ai/free/en/papers/hendrycks-2021-measuring-mathematical-problem-solving-math/). In Proceedings of the 35th Conference on Neural Information Processing Systems Datasets and Benchmarks Track, volum…
3. Xanh Ho, Anh-Khoa Duong Nguyen, Saku Sugawara, and Akiko Aizawa. 2020. [Constructing a multi-hop QA dataset for comprehensive evaluation of reasoning steps](https://md.research.elamur.ai/free/en/papers/ho-2020-constructing-multi-hop-qa-dataset/). In Proceedings of the 28th International Conference on Computational Linguistics, pages 6609–6625, Barcelona, Spain (Online). International Comm…
4. Weizhen Li, Jianbo Lin, Zhuosong Jiang, Jingyi Cao, Xinpeng Liu, Jiayu Zhang, Zhenqiang Huang, Qianben Chen, Weichen Sun, Qiexiang Wang, Hongxuan Lu, Tianrui Qin, Chenghao Zhu, Yi Yao, Shuying Fan, Xiaowan Li, Tiannan Wang, Pai Liu, King Zhu, and 11 others. 2025. Chain-of-agents: End-to-end agent fo… — [arXiv:2508.13167](https://arxiv.org/abs/2508.13167) [find](https://scholar.google.com/scholar?q=Weizhen+Li%2C+Jianbo+Lin%2C+Zhuosong+Jiang%2C+Jingyi+Cao%2C+Xinpeng+Liu%2C+Jiayu+Zhang%2C+Zhenqiang+Huang%2C+Qianben+Chen%2C+Weichen+Sun%2C+Qiexiang+Wang%2C+Hongxuan+Lu%2C+Tianrui+Qin%2C+Chenghao+Zhu%2C+Yi+Yao%2C+Shuying+Fan%2C+Xi)
5. Jun Liu, Zhenglun Kong, Peiyan Dong, Changdi Yang, Tianqi Li, Hao Tang, Geng Yuan, Wei Niu, Wenbin Zhang, Pu Zhao, Xue Lin, Dong Huang, and Yanzhi Wang. 2025. Structured agent distillation for large language model. Preprint, arXiv:2505.13820. [find](https://scholar.google.com/scholar?q=Jun+Liu%2C+Zhenglun+Kong%2C+Peiyan+Dong%2C+Changdi+Yang%2C+Tianqi+Li%2C+Hao+Tang%2C+Geng+Yuan%2C+Wei+Niu%2C+Wenbin+Zhang%2C+Pu+Zhao%2C+Xue+Lin%2C+Dong+Huang%2C+and+Yanzhi+Wang%2E+2025%2E+Structured+agent+distillation+for+large+l)
6. Kangzheng Liu, Feng Zhao, Yu Yang, and Guandong Xu. 2024. Dysarl: Dynamic structure-aware representation learning for multimodal knowledge graph reasoning. In Proceedings of the 32nd ACM International Conference on Multimedia, pages 8247–8256, Melbourne, Australia. ACM. [find](https://scholar.google.com/scholar?q=Kangzheng+Liu%2C+Feng+Zhao%2C+Yu+Yang%2C+and+Guandong+Xu%2E+2024%2E+Dysarl%3A+Dynamic+structure%2Daware+representation+learning+for+multimodal+knowledge+graph+reasoning%2E+In+Proceedings+of+the+32nd+ACM+International)
7. OpenAI. 2025. Introducing OpenAI O3 and O4-Mini. [https://openai.com/index/introducing-o3-a](https://openai.com/index/introducing-o3-a) nd-o4-mini. [↗](https://openai.com/index/introducing-o3-a)
8. Ofir Press, Muru Zhang, Sewon Min, Ludwig Schmidt, Noah Smith, and Mike Lewis. 2023. Measuring and narrowing the compositionality gap in language models. In Findings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 5687–5711, Singapore. Association for Computational… [find](https://scholar.google.com/scholar?q=Ofir+Press%2C+Muru+Zhang%2C+Sewon+Min%2C+Ludwig+Schmidt%2C+Noah+Smith%2C+and+Mike+Lewis%2E+2023%2E+Measuring+and+narrowing+the+compositionality+gap+in+language+models%2E+In+Findings+of+the+2023+Conference+on+Empirica)
9. Jiahao Qiu, Xinzhe Juan, Yimin Wang, Ling Yang, Xuan Qi, Tongcheng Zhang, Jiacheng Guo, Yifu Lu, Zixin Yao, Hongru Wang, Shilong Liu, Xun Jiang, Liu Leqi, and Mengdi Wang. 2025. Agentdistill: Training-free agent distillation with generalizable mcp boxes. Preprint, arXiv:2506.14728. — [arXiv:2506.14728](https://arxiv.org/abs/2506.14728) [find](https://scholar.google.com/scholar?q=Jiahao+Qiu%2C+Xinzhe+Juan%2C+Yimin+Wang%2C+Ling+Yang%2C+Xuan+Qi%2C+Tongcheng+Zhang%2C+Jiacheng+Guo%2C+Yifu+Lu%2C+Zixin+Yao%2C+Hongru+Wang%2C+Shilong+Liu%2C+Xun+Jiang%2C+Liu+Leqi%2C+and+Mengdi+Wang%2E+2025%2E+Agentdistill%3A+Training)
10. Weizhou Shen, Chenliang Li, Hongzhan Chen, Ming Yan, Xiaojun Quan, Hehong Chen, Ji Zhang, and Fei Huang. 2024. [Small LLMs are weak tool learners: A multi-LLM agent](https://md.research.elamur.ai/free/en/papers/shen-2024-small-llms-are-weak-tool/). In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 16658–16680, Miami, Florida, USA. Asso…
11. Kumar Shridhar, Alessandro Stolfo, and Mrinmaya Sachan. 2023. [Distilling reasoning capabilities into smaller language models](https://md.research.elamur.ai/free/en/papers/shridhar-2023-distilling-reasoning-capabilities-into-smaller/). In Findings of the 61st Annual Meeting of the Association for Computational Linguistics, pages 7059–7073, Toronto, Canada. Association for Computational Linguistics.
12. Haoxiang Sun, Yingqian Min, Zhipeng Chen, Wayne Xin Zhao, Lei Fang, Zheng Liu, Zhongyuan Wang, and Ji-Rong Wen. 2025. [Challenging the boundaries of reasoning: An olympiad-level math benchmark for large language models](https://md.research.elamur.ai/free/en/papers/sun-2026-challenging-boundaries-reasoning-olympiad-level/). Preprint, arXiv:2503.21380.
13. Zhilin Yang, Peng Qi, Saizheng Zhang, Yoshua Bengio, William Cohen, Ruslan Salakhutdinov, and Christopher D. Manning. 2018. [HotpotQA: A dataset for diverse, explainable multi-hop question answering](https://md.research.elamur.ai/free/en/papers/yang-2018-hotpotqa-dataset-diverse-explainable-multi/). In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 2369…
14. Fan Yin, Zifeng Wang, I-Hung Hsu, Jun Yan, Ke Jiang, Yanfei Chen, Jindong Gu, Long Le, Kai-Wei Chang, Chen-Yu Lee, Hamid Palangi, and Tomas Pfister. 2025. Magnet: Multi-turn tool-use data synthesis and distillation via graph translation. In Proceedings of the 63rd Annual Meeting of the Association f… [find](https://scholar.google.com/scholar?q=Fan+Yin%2C+Zifeng+Wang%2C+I%2DHung+Hsu%2C+Jun+Yan%2C+Ke+Jiang%2C+Yanfei+Chen%2C+Jindong+Gu%2C+Long+Le%2C+Kai%2DWei+Chang%2C+Chen%2DYu+Lee%2C+Hamid+Palangi%2C+and+Tomas+Pfister%2E+2025%2E+Magnet%3A+Multi%2Dturn+tool%2Duse+data+synthesis+an)
15. Shangziqi Zhao, Jiahao Yuan, Guisong Yang, and Usman Naseem. 2025. Can pruning improve reasoning? revisiting long-cot compression with capability in mind for better reasoning. Preprint, arXiv:2505.14582. — [arXiv:2505.14582](https://arxiv.org/abs/2505.14582) [find](https://scholar.google.com/scholar?q=Shangziqi+Zhao%2C+Jiahao+Yuan%2C+Guisong+Yang%2C+and+Usman+Naseem%2E+2025%2E+Can+pruning+improve+reasoning%3F+revisiting+long%2Dcot+compression+with+capability+in+mind+for+better+reasoning%2E+Preprint%2C+arXiv%3A2505%2E145)
16. Wanjun Zhong, Ruixiang Cui, Yiduo Guo, Yaobo Liang, Shuai Lu, Yanlin Wang, Amin Saied, Weizhu Chen, and Nan Duan. 2024. [AGIEval: A human-centric benchmark for evaluating foundation models](https://md.research.elamur.ai/free/en/papers/zhong-2024-agieval-human-centric-benchmark-evaluating/). In Findings of the 2024 Annual Conference of the North American Chapter of the Association for Computational Li…
17. Harsh Trivedi, Niranjan Balasubramanian, Tushar Khot, and Ashish Sabharwal. 2022. Musique: Multi-hop questions via single-hop question composition. Transactions of the Association for Computational Linguistics, 10:539–554. [find](https://scholar.google.com/scholar?q=Harsh+Trivedi%2C+Niranjan+Balasubramanian%2C+Tushar+Khot%2C+and+Ashish+Sabharwal%2E+2022%2E+Musique%3A+Multi%2Dhop+questions+via+single%2Dhop+question+composition%2E+Transactions+of+the+Association+for+Computational+Li)
18. Liang Wang, Nan Yang, Xiaolong Huang, Binxing Jiao, Linjun Yang, Daxin Jiang, Rangan Majumder, and Furu Wei. 2024a. [Text embeddings by weakly-supervised contrastive pre-training](https://md.research.elamur.ai/free/en/papers/wang-2022-text-embeddings-weakly-supervised-contrastive/). Preprint, arXiv:2212.03533.
19. Xingyao Wang, Yangyi Chen, Lifan Yuan, Yizhe Zhang, Yunzhu Li, Hao Peng, and Heng Ji. 2024b. [Executable code actions elicit better LLM agents](https://md.research.elamur.ai/free/en/papers/wang-2024-executable-code-actions-elicit-better/). In Proceedings of the 41st International Conference on Machine Learning, Vienna, Austria. PMLR.
20. Yuyang Wu, Yifei Wang, Tianqi Du, Stefanie Jegelka, and Yisen Wang. 2025. [When more is less: Understanding chain-of-thought length in LLMs](https://md.research.elamur.ai/free/en/papers/wu-2025-more-less-understanding-chain-thought/). In ICLR 2025 Workshop on Reasoning and Planning for Large Language Models, Singapore EXPO. OpenReview.
21. Heming Xia, Chak Tou Leong, Wenjie Wang, Yongqi Li, and Wenjie Li. 2025. [TokenSkip: Controllable chain-of-thought compression in LLMs](https://md.research.elamur.ai/free/en/papers/xia-2025-tokenskip-controllable-chain-thought-compression/). In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pages 3351–3363, Suzhou, China. Association for Computational Linguistics.

*Updated 2 November 2026*