---
title: "ETR — paper by Xiong et al. (2026) — summary and ideas"
description: "Building on this, they propose Entropy Trend Reward (ETR). 32 references. Among them: 0."
authors: ["Xuan Xiong", "Huan Liu", "Li Gu", "Zhixiang Chi", "Yue Qiu", "Yuanhao YU", "Yang Wang"]
year: 2026
venue: "Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)"
source: "https://aclanthology.org/2026.acl-long.799/"
doi: "10.18653/v1/2026.acl-long.799"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/xiong-2026-etr-entropy-trend-reward-efficient/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/xiong-2026-etr-entropy-trend-reward-efficient/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/31/) / ETR: Entropy Trend Reward for Efficient Chain-of-Thought Reasoning

# ETR: Entropy Trend Reward for Efficient Chain-of-Thought Reasoning

**Authors:** Xuan Xiong, [Huan Liu](https://md.research.elamur.ai/free/en/authors/huan-liu/), Li Gu, Zhixiang Chi, Yue Qiu, Yuanhao YU and [Yang Wang](https://md.research.elamur.ai/free/en/authors/yang-wang/) · **Year:** 2026 · **Venue:** [Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)](https://md.research.elamur.ai/free/en/venues/acl/) ([ACL 2026](https://md.research.elamur.ai/free/en/venues/acl/2026/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2026.acl-long.799/) · [PDF](https://aclanthology.org/2026.acl-long.799.pdf) · [DOI 10.18653/v1/2026.acl-long.799](https://doi.org/10.18653/v1/2026.acl-long.799) · [OpenAlex](https://openalex.org/W7166802023)

**Citations:** 0 citations (OpenAlex, 2026-10-14)

## Summary
Reasoning LLMs often overthink and produce very long chains of thought (CoT). Earlier fixes either penalize length directly, which ignores content, or push global entropy down. The second approach treats low uncertainty as good at every step. The authors argue that the shape of uncertainty over time matters more. On MATH500, they measure the Spearman correlation between step index and step entropy. Traces where entropy mostly falls turn out much shorter than traces where it keeps rising. Building on this, they propose Entropy Trend Reward (ETR). This reward uses a momentum-style aggregation of entropy changes across the trace. It favors gradual narrowing of uncertainty while still tolerating brief local exploration. ETR is added to the GRPO reward, and answer correctness remains the primary condition. Across several reasoning models and benchmarks, ETR gives the best accuracy–efficiency balance among the compared methods. Easy problems get short traces, and hard ones get gradual refinement, without hand-set length limits.

## Key points
- On DeepSeek-R1-Distill-7B, ETR raises accuracy by +9.9% and cuts CoT length by 67%, averaged over four benchmarks (including AIME24 and AMC23).
- Baselines include length-penalty methods (LCPO, O1-Pruner), the entropy-based RL method PEAR and the training-free CGRS. ETR obtains the highest AES score in every setting.
- Ablation results (AES): plain entropy minimization (Min. H) scores 1.06 versus 1.53 for ETR. Entropy maximization breaks reasoning entirely. Dropping momentum (No γ) fails to shorten outputs. Dropping R_corr hurts accuracy (65.0% on AMC23).
- After training, the step–entropy Spearman ρ becomes negative for Qwen3-4B, R1-Distill-7B and Qwen3-8B. ETR keeps some self-reflection and makes each step less wordy.
- Limitation: experiments use LoRA and stay at or below 8B parameters. The code is available at github.com/Xuan1030/ETR.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [University of Toronto](https://md.research.elamur.ai/free/en/universities/university-of-toronto/) — University, Toronto, Canada · 110 works · 179,061 citations
- [University of Ottawa](https://md.research.elamur.ai/free/en/universities/university-of-ottawa/) — University, Ottawa, Canada · 13 works · 2,514 citations
- [Concordia University](https://md.research.elamur.ai/free/en/universities/concordia-university/) — University, Montreal, Canada · 13 works · 673 citations
- [McMaster University](https://md.research.elamur.ai/free/en/universities/mcmaster-university/) — University, Hamilton, Canada · 6 works · 656 citations

📚 Shelves (5): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 18,614 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 17,965 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 17,450 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 17,296 works · [👁️ Computer Vision and Pattern Recognition](https://md.research.elamur.ai/free/en/shelves/computer-science/computer-vision-and-pattern-recognition/) · 3,486 works

🧭 Topics (1): [🧮 Reasoning](https://md.research.elamur.ai/free/en/topics/reasoning/) · 1,685 works

## Cite

Cite as: Xuan Xiong, Huan Liu, Li Gu, Zhixiang Chi, Yue Qiu, Yuanhao YU and Yang Wang. 2026. ETR: Entropy Trend Reward for Efficient Chain-of-Thought Reasoning. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{xiong2026etr,
  title = {{ETR: Entropy Trend Reward for Efficient Chain-of-Thought Reasoning}},
  author = {Xuan Xiong, Huan Liu, Li Gu, Zhixiang Chi, Yue Qiu, Yuanhao YU and Yang Wang},
  booktitle = {Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2026},
  doi = {10.18653/v1/2026.acl-long.799},
  url = {https://doi.org/10.18653/v1/2026.acl-long.799}
}
```

- [← previous eSciBench: An Extensible Scientific PDF Extraction Benchmark](https://md.research.elamur.ai/free/en/papers/taillon-2026-escibench-extensible-scientific-pdf-extraction/)
- [next → EVA: Evolving Semantic Adversaries for Red-Teaming GUI Agents Against Environmental Injection Attacks](https://md.research.elamur.ai/free/en/papers/lu-2026-eva-evolving-semantic-adversaries-red/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/31/)

## References

32 references, 19 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Shivam Agarwal, Zimin Zhang, Lifan Yuan, Jiawei Han, and Hao Peng. 2025. The unreasonable effectiveness of entropy minimization in llm reasoning. arXiv preprint arXiv:2505.15134. [find](https://scholar.google.com/scholar?q=Shivam+Agarwal%2C+Zimin+Zhang%2C+Lifan+Yuan%2C+Jiawei+Han%2C+and+Hao+Peng%2E+2025%2E+The+unreasonable+effectiveness+of+entropy+minimization+in+llm+reasoning%2E+arXiv+preprint+arXiv%3A2505%2E15134%2E)
2. Pranjal Aggarwal and Sean Welleck. 2025. [L1: Controlling how long a reasoning model thinks with reinforcement learning](https://md.research.elamur.ai/free/en/papers/aggarwal-2025-l1-controlling-long-reasoning-model/). arXiv preprint arXiv:2503.04697.
3. Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde de Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, Alex Ray, Raul Puri, Gretchen Krueger, Michael Petrov, Heidy Khlaaf, Girish Sastry, Pamela Mishkin, Brooke Chan, Scott Gray, and 39 others. 202… — [Evaluating Large Language Models Trained on Code](https://md.research.elamur.ai/free/en/papers/chen-2021-evaluating-large-language-models-trained/)
4. Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, and 1 others. 2025. [Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning](https://md.research.elamur.ai/free/en/papers/deepseek-ai-2025-deepseek-r1-incentivizing-reasoning-capability/). arXiv preprint arXiv:2501.12948.
5. Zhiwei He, Tian Liang, Jiahao Xu, Qiuzhi Liu, Xingyu Chen, Yue Wang, Linfeng Song, Dian Yu, Zhenwen Liang, Wenxuan Wang, and 1 others. 2025. [Deepmath-103k: A large-scale, challenging, decontaminated, and verifiable mathematical dataset for advancing reasoning](https://md.research.elamur.ai/free/en/papers/he-2025-deepmath-103k-large-scale-challenging/). arXiv preprint arXiv:2504.11456.
6. Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. 2024. [Measuring mathematical problem solving with the math dataset](https://md.research.elamur.ai/free/en/papers/hendrycks-2021-measuring-mathematical-problem-solving-math/), 2021. URL https://arxiv.org/abs/2103.03874, 2. [↗](https://arxiv.org/abs/2103.03874)
7. Edward J Hu, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, Weizhu Chen, and 1 others. 2022. [Lora: Low-rank adaptation of large language models](https://md.research.elamur.ai/free/en/papers/hu-2021-lora-low-rank-adaptation-large/). In International Conference on Learning Representations.
8. Chen Huang, Wei Lu, and Wenxuan Zhang. 2025a. Pear: Phase entropy aware reward for efficient reasoning. arXiv preprint arXiv:2510.08026. — [arXiv:2510.08026](https://arxiv.org/abs/2510.08026) [find](https://scholar.google.com/scholar?q=Chen+Huang%2C+Wei+Lu%2C+and+Wenxuan+Zhang%2E+2025a%2E+Pear%3A+Phase+entropy+aware+reward+for+efficient+reasoning%2E+arXiv+preprint+arXiv%3A2510%2E08026%2E+%E2%80%94+%5BarXiv%3A2510%2E08026%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2510%2E08026%29)
9. Jiameng Huang, Baijiong Lin, Guhao Feng, Jierun Chen, Di He, and Lu Hou. 2025b. Efficient reasoning for large reasoning language models via certainty-guided reflection suppression. arXiv preprint arXiv:2508.05337. — [arXiv:2508.05337](https://arxiv.org/abs/2508.05337) [find](https://scholar.google.com/scholar?q=Jiameng+Huang%2C+Baijiong+Lin%2C+Guhao+Feng%2C+Jierun+Chen%2C+Di+He%2C+and+Lu+Hou%2E+2025b%2E+Efficient+reasoning+for+large+reasoning+language+models+via+certainty%2Dguided+reflection+suppression%2E+arXiv+preprint+arXi)
10. Aaron Jaech, Adam Kalai, Adam Lerer, Adam Richardson, Ahmed El-Kishky, Aiden Low, Alec Helyar, Aleksander Madry, Alex Beutel, Alex Carney, and 1 others. 2024. [Openai o1 system card](https://md.research.elamur.ai/free/en/papers/openai-2024-openai-o1-system-card/). arXiv preprint arXiv:2412.16720.
11. Ayeong Lee, Ethan Che, and Tianyi Peng. 2025. How well do llms compress their own chain-of-thought? a token complexity approach. arXiv preprint arXiv:2503.01141. — [arXiv:2503.01141](https://arxiv.org/abs/2503.01141) [find](https://scholar.google.com/scholar?q=Ayeong+Lee%2C+Ethan+Che%2C+and+Tianyi+Peng%2E+2025%2E+How+well+do+llms+compress+their+own+chain%2Dof%2Dthought%3F+a+token+complexity+approach%2E+arXiv+preprint+arXiv%3A2503%2E01141%2E+%E2%80%94+%5BarXiv%3A2503%2E01141%5D%28https%3A%2F%2Farxiv%2Eorg)
12. Zeju Li, Jianyuan Zhong, Ziyang Zheng, Xiangyu Wen, Zhijian Xu, Yingying Cheng, Fan Zhang, and Qiang Xu. 2025. Compressing chain-of-thought in llms via step entropy. arXiv preprint arXiv:2508.03346. — [arXiv:2508.03346](https://arxiv.org/abs/2508.03346) [find](https://scholar.google.com/scholar?q=Zeju+Li%2C+Jianyuan+Zhong%2C+Ziyang+Zheng%2C+Xiangyu+Wen%2C+Zhijian+Xu%2C+Yingying+Cheng%2C+Fan+Zhang%2C+and+Qiang+Xu%2E+2025%2E+Compressing+chain%2Dof%2Dthought+in+llms+via+step+entropy%2E+arXiv+preprint+arXiv%3A2508%2E03346%2E+%E2%80%94)
13. Hunter Lightman, Vineet Kosaraju, Yuri Burda, Harrison Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever, and Karl Cobbe. 2023. [Let’s verify step by step](https://md.research.elamur.ai/free/en/papers/lightman-2023-let-s-verify-step-step/). In The Twelfth International Conference on Learning Representations.
14. Tengxiao Liu, Qipeng Guo, Xiangkun Hu, Cheng Jiayang, Yue Zhang, Xipeng Qiu, and Zheng Zhang. 2024. Can language models learn to skip steps? Advances in Neural Information Processing Systems, 37:45359–45385. [find](https://scholar.google.com/scholar?q=Tengxiao+Liu%2C+Qipeng+Guo%2C+Xiangkun+Hu%2C+Cheng+Jiayang%2C+Yue+Zhang%2C+Xipeng+Qiu%2C+and+Zheng+Zhang%2E+2024%2E+Can+language+models+learn+to+skip+steps%3F+Advances+in+Neural+Information+Processing+Systems%2C+37%3A45359)
15. Haotian Luo, Li Shen, Haiying He, Yibo Wang, Shiwei Liu, Wei Li, Naiqiang Tan, Xiaochun Cao, and Dacheng Tao. 2025. O1-pruner: Length-harmonizing fine-tuning for o1-like reasoning pruning. arXiv preprint arXiv:2501.12570. [find](https://scholar.google.com/scholar?q=Haotian+Luo%2C+Li+Shen%2C+Haiying+He%2C+Yibo+Wang%2C+Shiwei+Liu%2C+Wei+Li%2C+Naiqiang+Tan%2C+Xiaochun+Cao%2C+and+Dacheng+Tao%2E+2025%2E+O1%2Dpruner%3A+Length%2Dharmonizing+fine%2Dtuning+for+o1%2Dlike+reasoning+pruning%2E+arXiv+prepr)
16. Wenjie Ma, Jingxuan He, Charlie Snell, Tyler Griggs, Sewon Min, and Matei Zaharia. 2025. Reasoning models can be effective without thinking. arXiv preprint arXiv:2504.09858. [find](https://scholar.google.com/scholar?q=Wenjie+Ma%2C+Jingxuan+He%2C+Charlie+Snell%2C+Tyler+Griggs%2C+Sewon+Min%2C+and+Matei+Zaharia%2E+2025%2E+Reasoning+models+can+be+effective+without+thinking%2E+arXiv+preprint+arXiv%3A2504%2E09858%2E)
17. MAA. 2023. American mathematics competitions. [find](https://scholar.google.com/scholar?q=MAA%2E+2023%2E+American+mathematics+competitions%2E)
18. MAA. 2024. American invitational mathematics examination - aime. [find](https://scholar.google.com/scholar?q=MAA%2E+2024%2E+American+invitational+mathematics+examination+%2D+aime%2E)
19. Xiaoye Qu, Yafu Li, Zhaochen Su, Weigao Sun, Jianhao Yan, Dongrui Liu, Ganqu Cui, Daizong Liu, Shuxian Liang, Junxian He, and 1 others. 2025. [A survey of efficient reasoning for large reasoning models: Language, multimodality, and beyond](https://md.research.elamur.ai/free/en/papers/qu-2025-survey-efficient-reasoning-large-reasoning/). arXiv preprint arXiv:2503.21614.
20. David Rein, Betty Li Hou, Asa Cooper Stickland, Jackson Petty, Richard Yuanzhe Pang, Julien Dirani, Julian Michael, and Samuel R Bowman. 2024. [Gpqa: A graduate-level google-proof q&a benchmark](https://md.research.elamur.ai/free/en/papers/rein-2023-gpqa-graduate-level-google-proof/). In First Conference on Language Modeling.
21. John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. 2017. Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347. [find](https://scholar.google.com/scholar?q=John+Schulman%2C+Filip+Wolski%2C+Prafulla+Dhariwal%2C+Alec+Radford%2C+and+Oleg+Klimov%2E+2017%2E+Proximal+policy+optimization+algorithms%2E+arXiv+preprint+arXiv%3A1707%2E06347%2E)
22. Claude E Shannon. 1948. A mathematical theory of communication. The Bell system technical journal, 27(3):379–423. [find](https://scholar.google.com/scholar?q=Claude+E+Shannon%2E+1948%2E+A+mathematical+theory+of+communication%2E+The+Bell+system+technical+journal%2C+27%283%29%3A379%E2%80%93423%2E)
23. Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Yang Wu, and 1 others. 2024. [Deepseekmath: Pushing the limits of mathematical reasoning in open language models](https://md.research.elamur.ai/free/en/papers/shao-2024-deepseekmath-pushing-limits-mathematical-reasoning/). arXiv preprint arXiv:2402.03300.
24. Guangming Sheng, Chi Zhang, Zilingfeng Ye, Xibin Wu, Wang Zhang, Ru Zhang, Yanghua Peng, Haibin Lin, and Chuan Wu. 2025. [Hybridflow: A flexible and efficient rlhf framework](https://md.research.elamur.ai/free/en/papers/sheng-2024-hybridflow-flexible-efficient-rlhf-framework/). In Proceedings of the Twentieth European Conference on Computer Systems, pages 1279–1297.
25. Yang Sui, Yu-Neng Chuang, Guanchu Wang, Jiamu Zhang, Tianyi Zhang, Jiayi Yuan, Hongyi Liu, Andrew Wen, Shaochen Zhong, Na Zou, and 1 others. 2025. [Stop overthinking: A survey on efficient reasoning for large language models](https://md.research.elamur.ai/free/en/papers/sui-2025-stop-overthinking-survey-efficient-reasoning/). arXiv preprint arXiv:2503.16419.
26. Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, and 1 others. 2022. [Chain-of-thought prompting elicits reasoning in large language models](https://md.research.elamur.ai/free/en/papers/wei-2022-chain-thought-prompting-elicits-reasoning/). Advances in neural information processing systems, 35:24824– 24837.
27. Heming Xia, Chak Tou Leong, Wenjie Wang, Yongqi Li, and Wenjie Li. 2025. [Tokenskip: Controllable chain-of-thought compression in llms](https://md.research.elamur.ai/free/en/papers/xia-2025-tokenskip-controllable-chain-thought-compression/). arXiv preprint arXiv:2502.12067.
28. Silei Xu, Wenhao Xie, Lingxiao Zhao, and Pengcheng He. 2025. [Chain of draft: Thinking faster by writing less](https://md.research.elamur.ai/free/en/papers/xu-2025-chain-draft-thinking-faster-writing/). arXiv preprint arXiv:2502.18600.
29. Chenxu Yang, Qingyi Si, Yongjie Duan, Zheliang Zhu, Chenyu Zhu, Qiaowei Li, Minghui Chen, Zheng Lin, and Weiping Wang. 2025. [Dynamic early exit in reasoning models](https://md.research.elamur.ai/free/en/papers/yang-2025-dynamic-early-exit-reasoning-models/). arXiv preprint arXiv:2504.15895.
30. Yixin Ye, Zhen Huang, Yang Xiao, Ethan Chern, Shijie Xia, and Pengfei Liu. 2025. [Limo: Less is more for reasoning](https://md.research.elamur.ai/free/en/papers/ye-2025-limo-less-more-reasoning/). arXiv preprint arXiv:2502.03387.
31. Linan Yue, Yichao Du, Yizhi Wang, Weibo Gao, Fangzhou Yao, Li Wang, Ye Liu, Ziyu Xu, Qi Liu, Shimin Di, and 1 others. 2025. Don’t overthink it: A survey of efficient r1-style large reasoning models. arXiv preprint arXiv:2508.02120. [find](https://scholar.google.com/scholar?q=Linan+Yue%2C+Yichao+Du%2C+Yizhi+Wang%2C+Weibo+Gao%2C+Fangzhou+Yao%2C+Li+Wang%2C+Ye+Liu%2C+Ziyu+Xu%2C+Qi+Liu%2C+Shimin+Di%2C+and+1+others%2E+2025%2E+Don%E2%80%99t+overthink+it%3A+A+survey+of+efficient+r1%2Dstyle+large+reasoning+models%2E+a)
32. Lianmin Zheng, Liangsheng Yin, Zhiqiang Xie, Chuyue Livia Sun, Jeff Huang, Cody Hao Yu, Shiyi Cao, Christos Kozyrakis, Ion Stoica, Joseph E Gonzalez, and 1 others. 2024. [Sglang: Efficient execution of structured language model programs](https://md.research.elamur.ai/free/en/papers/zheng-2023-sglang-efficient-execution-structured-language/). Advances in neural information processing systems, 37:62557– 62…

*Updated 2 November 2026*