---
title: "LogitSpec — paper by Liu et al. (2026) — summary and ideas"
description: "Speculative decoding speeds up LLM inference by drafting several tokens and having the target model… 49 references, cited by 6 papers here."
authors: ["Tianyu Liu", "Qitan Lv", "Hao Li", "Xing Gao", "Xiao Sun", "Xiaoyan Sun"]
year: 2026
venue: "Findings of the Association for Computational Linguistics: ACL 2026"
source: "https://aclanthology.org/2026.findings-acl.1655/"
doi: "10.18653/v1/2026.findings-acl.1655"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/liu-2026-logitspec-accelerating-retrieval-based-speculative/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/liu-2026-logitspec-accelerating-retrieval-based-speculative/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/3/) / LogitSpec: Accelerating Retrieval-based Speculative Decoding via Next Next Token Speculation

# LogitSpec: Accelerating Retrieval-based Speculative Decoding via Next Next Token Speculation

**Authors:** [Tianyu Liu](https://md.research.elamur.ai/free/en/authors/tianyu-liu/), Qitan Lv, [Hao Li](https://md.research.elamur.ai/free/en/authors/hao-li/), [Xing Gao](https://md.research.elamur.ai/free/en/authors/xing-gao/), [Xiao Sun](https://md.research.elamur.ai/free/en/authors/xiao-sun/) and [Xiaoyan Sun](https://md.research.elamur.ai/free/en/authors/xiaoyan-sun/) · **Year:** 2026 · **Venue:** [Findings of the Association for Computational Linguistics: ACL 2026](https://md.research.elamur.ai/free/en/venues/findings/) ([Findings of the Association for Computational Linguistics 2026](https://md.research.elamur.ai/free/en/venues/findings/2026/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2026.findings-acl.1655/) · [PDF](https://aclanthology.org/2026.findings-acl.1655.pdf) · [DOI 10.18653/v1/2026.findings-acl.1655](https://doi.org/10.18653/v1/2026.findings-acl.1655) · [OpenAlex](https://openalex.org/W4417182934)

**Citations:** 0 citations (OpenAlex, 2026-10-14) · [6](https://md.research.elamur.ai/free/en/papers/liu-2026-logitspec-accelerating-retrieval-based-speculative/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
Speculative decoding speeds up LLM inference by drafting several tokens and having the target model verify them in one pass. Retrieval-based variants such as PLD skip the separate draft model and copy n-grams from the context, but they often match the wrong continuation or find no match at all. LogitSpec addresses this with a simple observation: the logit of the last position, which normally predicts the next token, also ranks the token after it fairly well. In more than half of decoding steps, that following token appears among the top-60 candidates, and this holds across Vicuna, LLaMA-3.1 and Qwen-2.5 models of different sizes. The method first guesses this next-next token from the last logit. It then retrieves context references anchored on both the next token and the guessed one, which narrows ambiguous matches and widens the search when no exact match exists. LogitSpec needs no training and works as a plug-in. On Spec-Bench, CNN/DM, GSM8K and HumanEval it gives roughly 1.8–2.6× lossless speedup.

## Key points
- Best result: 2.61× speedup and 3.28 mean accepted tokens (MAT) per step, achieved with Vicuna 7B on CNN/DM.
- Retrieval success rises from about 63–70% for PLD to about 97.6–99.4% for LogitSpec on Spec-Bench and HumanEval (Vicuna 7B/13B/33B).
- Ablations show both components matter. Removing either the last-logit speculation or the retrieval sharply lowers MAT and speedup on every benchmark.
- Overhead is small. Retrieval takes only 1.17% of decoding time on Spec-Bench with Vicuna 7B, while the model forward pass takes 93.92%.
- The method depends on the prompt context, so gains are limited when contexts are short:
  - about 1.4× on translation;
  - 2.01× on LongBench.
- All timings were measured on NVIDIA A100 GPUs. Code is released at github.com/smart-lty/LogitSpec.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [University of Science and Technology of China](https://md.research.elamur.ai/free/en/universities/university-of-science-and-technology-of-china/) — University, Hefei, China · 108 works · 112,698 citations
- [Shanghai Artificial Intelligence Laboratory](https://md.research.elamur.ai/free/en/labs/shanghai-artificial-intelligence-laboratory/) — Research institute, Shanghai, China · 117 works · 11,281 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works

🧭 Topics (1): [🔍 Information retrieval & search](https://md.research.elamur.ai/free/en/topics/information-retrieval/) · 473 works

## Cite

Cite as: Tianyu Liu, Qitan Lv, Hao Li, Xing Gao, Xiao Sun and Xiaoyan Sun. 2026. LogitSpec: Accelerating Retrieval-based Speculative Decoding via Next Next Token Speculation. Findings of the Association for Computational Linguistics: ACL 2026.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{liu2026logitspec,
  title = {{LogitSpec: Accelerating Retrieval-based Speculative Decoding via Next Next Token Speculation}},
  author = {Tianyu Liu, Qitan Lv, Hao Li, Xing Gao, Xiao Sun and Xiaoyan Sun},
  booktitle = {Findings of the Association for Computational Linguistics: ACL 2026},
  year = {2026},
  doi = {10.18653/v1/2026.findings-acl.1655},
  url = {https://doi.org/10.18653/v1/2026.findings-acl.1655}
}
```

- [← previous Lightweight LLM Agent Memory with Small Language Models](https://md.research.elamur.ai/free/en/papers/zhang-2026-lightweight-llm-agent-memory-small/)
- [next → LoopTool: Closing the Data–Training Loop for Robust LLM Tool Calls](https://md.research.elamur.ai/free/en/papers/zhang-2026-looptool-closing-data-training-loop/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/3/)

## References

49 references, 21 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Zachary Ankner, Rishab Parthasarathy, Aniruddha Nrusimha, Christopher Rinard, Jonathan Ragan-Kelley, and William Brandon. 2024. Hydra: Sequentially-dependent draft heads for medusa decoding. In First Conference on Language Modeling. [find](https://scholar.google.com/scholar?q=Zachary+Ankner%2C+Rishab+Parthasarathy%2C+Aniruddha+Nrusimha%2C+Christopher+Rinard%2C+Jonathan+Ragan%2DKelley%2C+and+William+Brandon%2E+2024%2E+Hydra%3A+Sequentially%2Ddependent+draft+heads+for+medusa+decoding%2E+In+First+)
2. Gregor Bachmann, Sotiris Anagnostidis, Albert Pumarola, Markos Georgopoulos, Artsiom Sanakoyeu, Yuming Du, Edgar Schönfeld, Ali Thabet, and Jonas Kohler. 2025. Judge decoding: Faster speculative sampling requires going beyond model alignment. Preprint, arXiv:2501.19309. [find](https://scholar.google.com/scholar?q=Gregor+Bachmann%2C+Sotiris+Anagnostidis%2C+Albert+Pumarola%2C+Markos+Georgopoulos%2C+Artsiom+Sanakoyeu%2C+Yuming+Du%2C+Edgar+Sch%C3%B6nfeld%2C+Ali+Thabet%2C+and+Jonas+Kohler%2E+2025%2E+Judge+decoding%3A+Faster+speculative+sampl)
3. Sangmin Bae, Jongwoo Ko, Hwanjun Song, and Se-Young Yun. 2023. Fast and robust early-exiting framework for autoregressive language models with synchronized parallel decoding. Preprint, arXiv:2310.05424. [find](https://scholar.google.com/scholar?q=Sangmin+Bae%2C+Jongwoo+Ko%2C+Hwanjun+Song%2C+and+Se%2DYoung+Yun%2E+2023%2E+Fast+and+robust+early%2Dexiting+framework+for+autoregressive+language+models+with+synchronized+parallel+decoding%2E+Preprint%2C+arXiv%3A2310%2E0542)
4. Yushi Bai, Xin Lv, Jiajie Zhang, Hongchang Lyu, Jiankai Tang, Zhidian Huang, Zhengxiao Du, Xiao Liu, Aohan Zeng, Lei Hou, Yuxiao Dong, Jie Tang, and Juanzi Li. 2024. LongBench: A bilingual, multi-task benchmark for long context understanding. In Proceedings of the 62nd Annual Meeting of the Associat… [find](https://scholar.google.com/scholar?q=Yushi+Bai%2C+Xin+Lv%2C+Jiajie+Zhang%2C+Hongchang+Lyu%2C+Jiankai+Tang%2C+Zhidian+Huang%2C+Zhengxiao+Du%2C+Xiao+Liu%2C+Aohan+Zeng%2C+Lei+Hou%2C+Yuxiao+Dong%2C+Jie+Tang%2C+and+Juanzi+Li%2E+2024%2E+LongBench%3A+A+bilingual%2C+multi%2Dtask)
5. Tianle Cai, Yuhong Li, Zhengyang Geng, Hongwu Peng, Jason D. Lee, Deming Chen, and Tri Dao. 2024. [Medusa: Simple llm inference acceleration framework with multiple decoding heads](https://md.research.elamur.ai/free/en/papers/cai-2024-medusa-simple-llm-inference-acceleration/). In Proceedings of the 41st International Conference on Machine Learning, ICML’24. JMLR.org.
6. Marco Antonio Calijorne Soares and Fernando Silva Parreiras. 2020. A literature review on question answering techniques, paradigms and systems. Journal of King Saud University - Computer and Information Sciences, 32(6):635–646. [find](https://scholar.google.com/scholar?q=Marco+Antonio+Calijorne+Soares+and+Fernando+Silva+Parreiras%2E+2020%2E+A+literature+review+on+question+answering+techniques%2C+paradigms+and+systems%2E+Journal+of+King+Saud+University+%2D+Computer+and+Informati)
7. Charlie Chen, Sebastian Borgeaud, Geoffrey Irving, Jean-Baptiste Lespiau, Laurent Sifre, and John Jumper. 2023. Accelerating large language model decoding with speculative sampling. Preprint, arXiv:2302.01318. [find](https://scholar.google.com/scholar?q=Charlie+Chen%2C+Sebastian+Borgeaud%2C+Geoffrey+Irving%2C+Jean%2DBaptiste+Lespiau%2C+Laurent+Sifre%2C+and+John+Jumper%2E+2023%2E+Accelerating+large+language+model+decoding+with+speculative+sampling%2E+Preprint%2C+arXiv%3A23)
8. Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde de Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, Alex Ray, Raul Puri, Gretchen Krueger, Michael Petrov, Heidy Khlaaf, Girish Sastry, Pamela Mishkin, Brooke Chan, Scott Gray, and 39 others. 202… — [Evaluating Large Language Models Trained on Code](https://md.research.elamur.ai/free/en/papers/chen-2021-evaluating-large-language-models-trained/)
9. Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman. 2021. [Training verifiers to solve math word problems](https://md.research.elamur.ai/free/en/papers/cobbe-2021-training-verifiers-solve-math-word/). Preprint, arXiv:2110.14168.
10. DeepSeek-AI. 2025. [Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning](https://md.research.elamur.ai/free/en/papers/deepseek-ai-2025-deepseek-r1-incentivizing-reasoning-capability/). Preprint, arXiv:2501.12948.
11. Cunxiao Du, Jing Jiang, Xu Yuanchen, Jiawei Wu, Sicheng Yu, Yongqi Li, Shenggui Li, Kai Xu, Liqiang Nie, Zhaopeng Tu, and Yang You. 2024. Glide with a cape: a low-hassle method to accelerate speculative decoding. In Proceedings of the 41st International Conference on Machine Learning, ICML’24. JMLR.… [find](https://scholar.google.com/scholar?q=Cunxiao+Du%2C+Jing+Jiang%2C+Xu+Yuanchen%2C+Jiawei+Wu%2C+Sicheng+Yu%2C+Yongqi+Li%2C+Shenggui+Li%2C+Kai+Xu%2C+Liqiang+Nie%2C+Zhaopeng+Tu%2C+and+Yang+You%2E+2024%2E+Glide+with+a+cape%3A+a+low%2Dhassle+method+to+accelerate+speculati)
12. Mostafa Elhoushi, Akshat Shrivastava, Diana Liskovich, Basil Hosmer, Bram Wasti, Liangzhen Lai, Anas Mahmoud, Bilge Acun, Saurabh Agarwal, Ahmed Roman, Ahmed A Aly, Beidi Chen, and Carole-Jean Wu. 2024. [Layerskip: Enabling early exit inference and self-speculative decoding](https://md.research.elamur.ai/free/en/papers/elhoushi-2024-layerskip-enabling-early-exit-inference/).
13. Yichao Fu, Peter Bailis, Ion Stoica, and Hao Zhang. 2024. [Break the sequential dependency of llm inference using lookahead decoding](https://md.research.elamur.ai/free/en/papers/fu-2024-break-sequential-dependency-llm-inference/). In Proceedings of the 41st International Conference on Machine Learning, ICML’24. JMLR.org.
14. Fabian Gloeckle, Badr Youbi Idrissi, Baptiste Rozière, David Lopez-Paz, and Gabriel Synnaeve. 2024. Better & faster large language models via multi-token prediction. Preprint, arXiv:2404.19737. [find](https://scholar.google.com/scholar?q=Fabian+Gloeckle%2C+Badr+Youbi+Idrissi%2C+Baptiste+Rozi%C3%A8re%2C+David+Lopez%2DPaz%2C+and+Gabriel+Synnaeve%2E+2024%2E+Better+%26+faster+large+language+models+via+multi%2Dtoken+prediction%2E+Preprint%2C+arXiv%3A2404%2E19737%2E)
15. Zhenyu He, Zexuan Zhong, Tianle Cai, Jason Lee, and Di He. 2024. REST: Retrieval-based speculative decoding. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), pages 1582–1595, Mex… [find](https://scholar.google.com/scholar?q=Zhenyu+He%2C+Zexuan+Zhong%2C+Tianle+Cai%2C+Jason+Lee%2C+and+Di+He%2E+2024%2E+REST%3A+Retrieval%2Dbased+speculative+decoding%2E+In+Proceedings+of+the+2024+Conference+of+the+North+American+Chapter+of+the+Association+for+)
16. Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. 2021. [Measuring mathematical problem solving with the MATH dataset](https://md.research.elamur.ai/free/en/papers/hendrycks-2021-measuring-mathematical-problem-solving-math/). In Thirty-fifth Conference on Neural Information Processing Systems Datasets and Benchmarks Track (Round 2).
17. Coleman Hooper, Sehoon Kim, Hiva Mohammadzadeh, Hasan Genc, Kurt Keutzer, Amir Gholami, and Sophia Shao. 2024. Speed: Speculative pipelined execution for efficient decoding. Preprint, arXiv:2310.12072. — [arXiv:2310.12072](https://arxiv.org/abs/2310.12072) [find](https://scholar.google.com/scholar?q=Coleman+Hooper%2C+Sehoon+Kim%2C+Hiva+Mohammadzadeh%2C+Hasan+Genc%2C+Kurt+Keutzer%2C+Amir+Gholami%2C+and+Sophia+Shao%2E+2024%2E+Speed%3A+Speculative+pipelined+execution+for+efficient+decoding%2E+Preprint%2C+arXiv%3A2310%2E12072)
18. Yuxuan Hu, Ke Wang, Xiaokang Zhang, Fanjin Zhang, Cuiping Li, Hong Chen, and Jing Zhang. 2024. Sam decoding: Speculative decoding via suffix automaton. Preprint, arXiv:2411.10666. [find](https://scholar.google.com/scholar?q=Yuxuan+Hu%2C+Ke+Wang%2C+Xiaokang+Zhang%2C+Fanjin+Zhang%2C+Cuiping+Li%2C+Hong+Chen%2C+and+Jing+Zhang%2E+2024%2E+Sam+decoding%3A+Speculative+decoding+via+suffix+automaton%2E+Preprint%2C+arXiv%3A2411%2E10666%2E)
19. Kaixuan Huang, Xudong Guo, and Mengdi Wang. 2025. Specdec++: Boosting speculative decoding via adaptive candidate lengths. [find](https://scholar.google.com/scholar?q=Kaixuan+Huang%2C+Xudong+Guo%2C+and+Mengdi+Wang%2E+2025%2E+Specdec%2B%2B%3A+Boosting+speculative+decoding+via+adaptive+candidate+lengths%2E)
20. Juyong Jiang, Fan Wang, Jiasi Shen, Sungju Kim, and Sunghun Kim. 2024. [A survey on large language models for code generation](https://md.research.elamur.ai/free/en/papers/jiang-2024-survey-large-language-models-code/). Preprint, arXiv:2406.00515.
21. Yaniv Leviathan, Matan Kalman, and Yossi Matias. 2023. Fast inference from transformers via speculative decoding. In Proceedings of the 40th International Conference on Machine Learning, volume 202 of Proceedings of Machine Learning Research, pages 19274–19286. PMLR. [find](https://scholar.google.com/scholar?q=Yaniv+Leviathan%2C+Matan+Kalman%2C+and+Yossi+Matias%2E+2023%2E+Fast+inference+from+transformers+via+speculative+decoding%2E+In+Proceedings+of+the+40th+International+Conference+on+Machine+Learning%2C+volume+202+of)
22. Jinze Li, Yixing Xu, Haiduo Huang, Xuanwu Yin, Dong Li, Edith C. H. Ngai, and Emad Barsoum. 2025a. Gumiho: A hybrid architecture to prioritize early tokens in speculative decoding. Preprint, arXiv:2503.10135. — [arXiv:2503.10135](https://arxiv.org/abs/2503.10135) [find](https://scholar.google.com/scholar?q=Jinze+Li%2C+Yixing+Xu%2C+Haiduo+Huang%2C+Xuanwu+Yin%2C+Dong+Li%2C+Edith+C%2E+H%2E+Ngai%2C+and+Emad+Barsoum%2E+2025a%2E+Gumiho%3A+A+hybrid+architecture+to+prioritize+early+tokens+in+speculative+decoding%2E+Preprint%2C+arXiv%3A250)
23. Yuhui Li, Fangyun Wei, Chao Zhang, and Hongyang Zhang. 2024. [EAGLE-2: Faster inference of language models with dynamic draft trees](https://md.research.elamur.ai/free/en/papers/li-2024-eagle-2-faster-inference-language/). In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 7421–7432, Miami, Florida, USA. Association for Computational Linguisti…
24. Yuhui Li, Fangyun Wei, Chao Zhang, and Hongyang Zhang. 2025b. Eagle-3: Scaling up inference acceleration of large language models via training-time test. Preprint, arXiv:2503.01840. — [arXiv:2503.01840](https://arxiv.org/abs/2503.01840) [find](https://scholar.google.com/scholar?q=Yuhui+Li%2C+Fangyun+Wei%2C+Chao+Zhang%2C+and+Hongyang+Zhang%2E+2025b%2E+Eagle%2D3%3A+Scaling+up+inference+acceleration+of+large+language+models+via+training%2Dtime+test%2E+Preprint%2C+arXiv%3A2503%2E01840%2E+%E2%80%94+%5BarXiv%3A2503%2E0184)
25. Yuhui Li, Fangyun Wei, Chao Zhang, and Hongyang Zhang. 2025c. [Eagle: Speculative sampling requires rethinking feature uncertainty](https://md.research.elamur.ai/free/en/papers/li-2024-eagle-speculative-sampling-requires-rethinking/). Preprint, arXiv:2401.15077.
26. Jiahao Liu, Qifan Wang, Jingang Wang, and Xunliang Cai. 2024. Speculative decoding via early-exiting for faster llm inference with thompson sampling control mechanism. Preprint, arXiv:2406.03853. [find](https://scholar.google.com/scholar?q=Jiahao+Liu%2C+Qifan+Wang%2C+Jingang+Wang%2C+and+Xunliang+Cai%2E+2024%2E+Speculative+decoding+via+early%2Dexiting+for+faster+llm+inference+with+thompson+sampling+control+mechanism%2E+Preprint%2C+arXiv%3A2406%2E03853%2E)
27. Tianyu Liu, Yun Li, Qitan Lv, Kai Liu, Jianchen Zhu, Winston Hu, and Xiao Sun. 2025. PEARL: Parallel speculative decoding with adaptive draft length. In The Thirteenth International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Tianyu+Liu%2C+Yun+Li%2C+Qitan+Lv%2C+Kai+Liu%2C+Jianchen+Zhu%2C+Winston+Hu%2C+and+Xiao+Sun%2E+2025%2E+PEARL%3A+Parallel+speculative+decoding+with+adaptive+draft+length%2E+In+The+Thirteenth+International+Conference+on+Lear)
28. Xianzhen Luo, Yixuan Wang, Qingfu Zhu, Zhiming Zhang, Xuanyu Zhang, Qing Yang, Dongliang Xu, and Wanxiang Che. 2024. [Turning trash into treasure: Accelerating inference of large language models with token recycling](https://md.research.elamur.ai/free/en/papers/luo-2025-turning-trash-into-treasure-accelerating/). Preprint, arXiv:2408.08696.
29. Mathematical Association of America. 2025. Maa invitational competitions. [https://maa.org/](https://maa.org/) maa-invitational-competitions/. Accessed: 2025-09-09. [↗](https://maa.org/)
30. Xupeng Miao, Gabriele Oliaro, Zhihao Zhang, Xinhao Cheng, Zeyu Wang, Zhengxin Zhang, Rae Ying Yee Wong, Alan Zhu, Lijie Yang, Xiaoxiang Shi, Chunan Shi, Zhuoming Chen, Daiyaan Arfeen, Reyna Abhyankar, and Zhihao Jia. 2024. Specinfer: Accelerating large language model serving with tree-based speculat… [find](https://scholar.google.com/scholar?q=Xupeng+Miao%2C+Gabriele+Oliaro%2C+Zhihao+Zhang%2C+Xinhao+Cheng%2C+Zeyu+Wang%2C+Zhengxin+Zhang%2C+Rae+Ying+Yee+Wong%2C+Alan+Zhu%2C+Lijie+Yang%2C+Xiaoxiang+Shi%2C+Chunan+Shi%2C+Zhuoming+Chen%2C+Daiyaan+Arfeen%2C+Reyna+Abhyankar%2C)
31. Ramesh Nallapati, Bowen Zhou, Cicero dos Santos, Çağlar Gu˙lçehre, and Bing Xiang. 2016. [Abstractive text summarization using sequence-to-sequence RNNs and beyond](https://md.research.elamur.ai/free/en/papers/nallapati-2016-abstractive-text-summarization-sequence-sequence/). In Proceedings of the 20th SIGNLL Conference on Computational Natural Language Learning, pages 280–290, Berlin, Germany. Association fo…
32. NVIDIA, Péter Vingelmann, and Frank H.P. Fitzek. 2020. Cuda, release: 10.2.89. [find](https://scholar.google.com/scholar?q=NVIDIA%2C+P%C3%A9ter+Vingelmann%2C+and+Frank+H%2EP%2E+Fitzek%2E+2020%2E+Cuda%2C+release%3A+10%2E2%2E89%2E)
33. OpenAI. 2024. [Gpt-4 technical report](https://md.research.elamur.ai/free/en/papers/openai-2023-gpt-4-technical-report/). arXiv:2303.08774.
34. Preprint, Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Köpf, Edward Yang, Zach DeVito, Martin Raison, Alykhan Te-jani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, and 2 oth… — [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://md.research.elamur.ai/free/en/papers/paszke-2019-pytorch-imperative-style-high-performance/)
35. Apoorv Saxena. 2023. Prompt lookup decoding. [find](https://scholar.google.com/scholar?q=Apoorv+Saxena%2E+2023%2E+Prompt+lookup+decoding%2E)
36. Llama Team. 2024. [The llama 3 herd of models](https://md.research.elamur.ai/free/en/papers/grattafiori-2024-llama-3-herd-models/). Preprint, arXiv:2407.21783.
37. Qwen Team. 2025. Qwen2.5 technical report. Preprint, arXiv:2412.15115. [find](https://scholar.google.com/scholar?q=Qwen+Team%2E+2025%2E+Qwen2%2E5+technical+report%2E+Preprint%2C+arXiv%3A2412%2E15115%2E)
38. Jikai Wang, Yi Su, Juntao Li, Qingrong Xia, Zi Ye, Xinyu Duan, Zhefeng Wang, and Min Zhang. 2024. [Opt-tree: Speculative decoding with adaptive draft tree structure](https://md.research.elamur.ai/free/en/papers/wang-2025-opt-tree-speculative-decoding-adaptive/). Preprint, arXiv:2406.17276.
39. Yepeng Weng, Dianwen Mei, Huishi Qiu, Xujie Chen, Li Liu, Jiang Tian, and Zhongchao Shi. 2025. Coral: Learning consistent representations across multi-step training with lighter speculative drafter. Preprint, arXiv:2502.16880. [find](https://scholar.google.com/scholar?q=Yepeng+Weng%2C+Dianwen+Mei%2C+Huishi+Qiu%2C+Xujie+Chen%2C+Li+Liu%2C+Jiang+Tian%2C+and+Zhongchao+Shi%2E+2025%2E+Coral%3A+Learning+consistent+representations+across+multi%2Dstep+training+with+lighter+speculative+drafter%2E+P)
40. Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pier-ric Cistac, Tim Rault, Rémi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, and 3 others. 2020. [Transf…](https://md.research.elamur.ai/free/en/papers/wolf-2020-transformers-state-art-natural-language/)
41. Heming Xia, Yongqi Li, Jun Zhang, Cunxiao Du, and Wenjie Li. 2025. SWIFT: On-the-fly self-speculative decoding for LLM inference acceleration. In The Thirteenth International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Heming+Xia%2C+Yongqi+Li%2C+Jun+Zhang%2C+Cunxiao+Du%2C+and+Wenjie+Li%2E+2025%2E+SWIFT%3A+On%2Dthe%2Dfly+self%2Dspeculative+decoding+for+LLM+inference+acceleration%2E+In+The+Thirteenth+International+Conference+on+Learning+Re)
42. Heming Xia, Zhe Yang, Qingxiu Dong, Peiyi Wang, Yongqi Li, Tao Ge, Tianyu Liu, Wenjie Li, and Zhifang Sui. 2024. [Unlocking efficiency in large language model inference: A comprehensive survey of speculative decoding](https://md.research.elamur.ai/free/en/papers/xia-2024-unlocking-efficiency-large-language-model/). In Findings of the Association for Computational Linguistics ACL 2024, pages 7655–7…
43. Bin Xiao, Chunan Shi, Xiaonan Nie, Fan Yang, Xiangwei Deng, Lei Su, Weipeng Chen, and Bin Cui. 2024. Clover: Regressive lightweight speculative decoding with sequential knowledge. Preprint, arXiv:2405.00263. [find](https://scholar.google.com/scholar?q=Bin+Xiao%2C+Chunan+Shi%2C+Xiaonan+Nie%2C+Fan+Yang%2C+Xiangwei+Deng%2C+Lei+Su%2C+Weipeng+Chen%2C+and+Bin+Cui%2E+2024%2E+Clover%3A+Regressive+lightweight+speculative+decoding+with+sequential+knowledge%2E+Preprint%2C+arXiv%3A2405)
44. Siheng Xiong, Joe Zou, Faramarz Fekri, and Yae Jee Cho. Long-context modeling with dynamic hierarchical sparse attention for on-device llms. In NeurIPS 2025 Workshop on Efficient Reasoning. [find](https://scholar.google.com/scholar?q=Siheng+Xiong%2C+Joe+Zou%2C+Faramarz+Fekri%2C+and+Yae+Jee+Cho%2E+Long%2Dcontext+modeling+with+dynamic+hierarchical+sparse+attention+for+on%2Ddevice+llms%2E+In+NeurIPS+2025+Workshop+on+Efficient+Reasoning%2E)
45. Penghui Yang, Cunxiao Du, Fengzhuo Zhang, Haonan Wang, Tianyu Pang, Chao Du, and Bo An. 2026. [Longspec: Long-context lossless speculative decoding with efficient drafting and verification](https://md.research.elamur.ai/free/en/papers/yang-2026-longspec-long-context-lossless-speculative/). Preprint, arXiv:2502.17421.
46. Zihao Yi, Jiarui Ouyang, Yuwen Liu, Tianhao Liao, Zhe Xu, and Ying Shen. 2024. [A survey on recent advances in llm-based multi-turn dialogue systems](https://md.research.elamur.ai/free/en/papers/yi-2024-survey-recent-advances-llm-based/). Preprint, arXiv:2402.18013.
47. Jun Zhang, Jue Wang, Huan Li, Lidan Shou, Ke Chen, Gang Chen, and Sharad Mehrotra. 2024a. [Draft & verify: Lossless large language model acceleration via self-speculative decoding](https://md.research.elamur.ai/free/en/papers/zhang-2024-draft-verify-lossless-large-language/). In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), page…
48. Lefan Zhang, Xiaodan Wang, Yanhua Huang, and Ruiwen Xu. 2025. Learning harmonized representations for speculative sampling. In The Thirteenth International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Lefan+Zhang%2C+Xiaodan+Wang%2C+Yanhua+Huang%2C+and+Ruiwen+Xu%2E+2025%2E+Learning+harmonized+representations+for+speculative+sampling%2E+In+The+Thirteenth+International+Conference+on+Learning+Representations%2E)
49. Situo Zhang, Hankun Wang, Da Ma, Zichen Zhu, Lu Chen, Kunyao Lan, and Kai Yu. 2024b. Adaea-gle: Optimizing speculative decoding via explicit modeling of adaptive draft structures. Preprint, arXiv:2412.18910. — [arXiv:2412.18910](https://arxiv.org/abs/2412.18910) [find](https://scholar.google.com/scholar?q=Situo+Zhang%2C+Hankun+Wang%2C+Da+Ma%2C+Zichen+Zhu%2C+Lu+Chen%2C+Kunyao+Lan%2C+and+Kai+Yu%2E+2024b%2E+Adaea%2Dgle%3A+Optimizing+speculative+decoding+via+explicit+modeling+of+adaptive+draft+structures%2E+Preprint%2C+arXiv%3A2412)

*Updated 1 November 2026*