---
title: "SPIDE: Serial and Parallel Intertwined Speculative Decoding (2026)"
description: "The paper targets two weaknesses of speculative decoding (SD) for LLMs. 37 references."
authors: ["Wenru Xu", "Peixuan Xu", "Ziqi Yang", "Ming Hu", "Zihui Wang", "Jianzhong Qi", "Rongshan Yu", "Xiaoliang Fan", "Cheng Wang"]
year: 2026
venue: "Findings of the Association for Computational Linguistics: ACL 2026"
source: "https://aclanthology.org/2026.findings-acl.1040/"
doi: "10.18653/v1/2026.findings-acl.1040"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/xu-2026-spide-serial-parallel-intertwined-speculative/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/xu-2026-spide-serial-parallel-intertwined-speculative/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/50/) / SPIDE: Serial and Parallel Intertwined Speculative Decoding

# SPIDE: Serial and Parallel Intertwined Speculative Decoding

**Authors:** Wenru Xu, Peixuan Xu, Ziqi Yang, [Ming Hu](https://md.research.elamur.ai/free/en/authors/ming-hu/), Zihui Wang, [Jianzhong Qi](https://md.research.elamur.ai/free/en/authors/jianzhong-qi/), Rongshan Yu, Xiaoliang Fan and [Cheng Wang](https://md.research.elamur.ai/free/en/authors/cheng-wang/) · **Year:** 2026 · **Venue:** [Findings of the Association for Computational Linguistics: ACL 2026](https://md.research.elamur.ai/free/en/venues/findings/) ([Findings of the Association for Computational Linguistics 2026](https://md.research.elamur.ai/free/en/venues/findings/2026/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2026.findings-acl.1040/) · [PDF](https://aclanthology.org/2026.findings-acl.1040.pdf) · [DOI 10.18653/v1/2026.findings-acl.1040](https://doi.org/10.18653/v1/2026.findings-acl.1040) · [OpenAlex](https://openalex.org/W7166865519)

**Citations:** 0 citations (OpenAlex, 2026-10-14)

## Summary

The paper targets two weaknesses of speculative decoding (SD) for LLMs. Classic "draft-then-verify" pipelines run strictly in sequence, which caps the speedup. Fully parallel schemes such as PEARL keep the large target model busy almost constantly, which raises the cost of using it. SPIDE is a training-free framework that switches between two modes as decoding goes on: serial dynamic drafting and parallel draft verification.

The core component is a confidence-acceptance mapping table. Draft-model confidence (the maximum next-token probability) is split into ten intervals, and the table records how often tokens in each interval were actually accepted. While drafting, these estimated acceptance rates are combined into a reliability score for the current draft. When the score drops below a threshold τ, SPIDE starts verifying in parallel while the draft model keeps drafting. After any rejection, it returns to serial mode, and the table is refreshed after every verification.

Across several draft/target model pairs and generation tasks, SPIDE is faster than serial and parallel baselines while keeping target-model calls close to vanilla SD. This makes it a cheaper way to speed up inference.

## Key points

- **Speedup vs. autoregressive decoding:** 3.25× on average and up to 4.56×.
- **Speedup vs. vanilla SD:** up to 2.12×, or an extra 67.7% on average, while target-model calls rise by only 8.2% on average.
- **Comparison with PEARL:** SPIDE is up to 19.6% faster and removes 83.7% of the additional verification cost that parallelism adds, on average.
- **Evaluation setup:**
  - Benchmarks: HumanEval, GSM8K, MGSM (12 multilingual arithmetic tasks) and MT-bench (8 multi-turn dialogue tasks).
  - Model pairs include Vicuna 68M/13B, Pythia 160M/12B, DeepSeek 1.3B/33B, TinyLlama 1.1B with CodeLlama 34B, and Qwen3 1.7B/32B.
  - Baselines: autoregressive decoding, vanilla SD, SVIP, D&V and PEARL.
  - Metrics: walltime speedup (Speed), target-model calls relative to vanilla SD (Cost) and mean accepted tokens (MAT).
- **Ablations and tuning:**
  - Removing either dynamic drafting or parallel verification lowers the speedup.
  - The serial-only variant still beats vanilla SD, SVIP and D&V.
  - Setting τ between 0.5 and 0.9 gives a good balance between speed and cost.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [The University of Melbourne](https://md.research.elamur.ai/free/en/universities/the-university-of-melbourne/) — University, Melbourne, Australia · 102 works · 22,618 citations
- [East China Normal University](https://md.research.elamur.ai/free/en/universities/east-china-normal-university/) — University, Shanghai, China · 69 works · 2,490 citations
- [Peng Cheng Laboratory](https://md.research.elamur.ai/free/en/labs/peng-cheng-laboratory/) — Research institute, Shenzhen, China · 47 works · 2,699 citations
- [Xiamen University](https://md.research.elamur.ai/free/en/universities/xiamen-university/) — University, Xiamen, China · 24 works · 2,317 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 18,614 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 17,965 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 17,450 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 17,296 works

🧭 Topics (1): [🪶 Efficient models & compression](https://md.research.elamur.ai/free/en/topics/efficient-models/) · 297 works

## Cite

Cite as: Wenru Xu, Peixuan Xu, Ziqi Yang, Ming Hu, Zihui Wang, Jianzhong Qi, Rongshan Yu, Xiaoliang Fan and Cheng Wang. 2026. SPIDE: Serial and Parallel Intertwined Speculative Decoding. Findings of the Association for Computational Linguistics: ACL 2026.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{xu2026spide,
  title = {{SPIDE: Serial and Parallel Intertwined Speculative Decoding}},
  author = {Wenru Xu, Peixuan Xu, Ziqi Yang, Ming Hu, Zihui Wang, Jianzhong Qi, Rongshan Yu, Xiaoliang Fan and Cheng Wang},
  booktitle = {Findings of the Association for Computational Linguistics: ACL 2026},
  year = {2026},
  doi = {10.18653/v1/2026.findings-acl.1040},
  url = {https://doi.org/10.18653/v1/2026.findings-acl.1040}
}
```

- [← previous SPENCE: A Syntactic Probe for Detecting Contamination in NL2SQL Benchmarks](https://md.research.elamur.ai/free/en/papers/safarzadeh-2026-spence-syntactic-probe-detecting-contamination/)
- [next → SpidR-Adapt: A Universal Speech Representation Model for Few-Shot Adaptation](https://md.research.elamur.ai/free/en/papers/luthra-2026-spidr-adapt-universal-speech-representation/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/50/)

## References

37 references, 15 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Stella Biderman, Hailey Schoelkopf, Quentin Gregory Anthony, Herbie Bradley, Kyle O’Brien, Eric Hallahan, Mohammad Aflah Khan, Shivanshu Purohit, USVSN Sai Prashanth, Edward Raff, and 1 others. 2023. Pythia: A suite for analyzing large language models across training and scaling. In International Co… [find](https://scholar.google.com/scholar?q=Stella+Biderman%2C+Hailey+Schoelkopf%2C+Quentin+Gregory+Anthony%2C+Herbie+Bradley%2C+Kyle+O%E2%80%99Brien%2C+Eric+Hallahan%2C+Mohammad+Aflah+Khan%2C+Shivanshu+Purohit%2C+USVSN+Sai+Prashanth%2C+Edward+Raff%2C+and+1+others%2E+2023%2E+)
2. Tianle Cai, Yuhong Li, Zhengyang Geng, Hongwu Peng, Jason D Lee, Deming Chen, and Tri Dao. 2024. [Medusa: Simple llm inference acceleration framework with multiple decoding heads](https://md.research.elamur.ai/free/en/papers/cai-2024-medusa-simple-llm-inference-acceleration/). arXiv preprint arXiv:2401.10774.
3. Charlie Chen, Sebastian Borgeaud, Geoffrey Irving, Jean-Baptiste Lespiau, Laurent Sifre, and John Jumper. 2023. Accelerating large language model decoding with speculative sampling. arXiv preprint arXiv:2302.01318. [find](https://scholar.google.com/scholar?q=Charlie+Chen%2C+Sebastian+Borgeaud%2C+Geoffrey+Irving%2C+Jean%2DBaptiste+Lespiau%2C+Laurent+Sifre%2C+and+John+Jumper%2E+2023%2E+Accelerating+large+language+model+decoding+with+speculative+sampling%2E+arXiv+preprint+arX)
4. Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde De Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, and 1 others. 2021. [Evaluating large language models trained on code](https://md.research.elamur.ai/free/en/papers/chen-2021-evaluating-large-language-models-trained/). arXiv preprint arXiv:2107.03374.
5. Ziyi Chen, Xiaocong Yang, Jiacheng Lin, Chenkai Sun, Kevin Chang, and Jie Huang. 2024. Cascade speculative drafting for even faster llm inference. Advances in Neural Information Processing Systems, 37:86226– 86242. [find](https://scholar.google.com/scholar?q=Ziyi+Chen%2C+Xiaocong+Yang%2C+Jiacheng+Lin%2C+Chenkai+Sun%2C+Kevin+Chang%2C+and+Jie+Huang%2E+2024%2E+Cascade+speculative+drafting+for+even+faster+llm+inference%2E+Advances+in+Neural+Information+Processing+Systems%2C+37)
6. Wei-Lin Chiang, Zhuohan Li, Ziqing Lin, Ying Sheng, Zhanghao Wu, Hao Zhang, Lianmin Zheng, Siyuan Zhuang, Yonghao Zhuang, Joseph E Gonzalez, and 1 others. 2023. Vicuna: An open-source chatbot impressing gpt-4 with 90%* chatgpt quality. See [https://vicuna.lmsys](https://vicuna.lmsys). org (accessed 14 April 2023), 2(3):6. [↗](https://vicuna.lmsys)
7. Krishna Teja Chitty-Venkata and Arun K Somani. 2022. Neural architecture search survey: A hardware perspective. ACM Computing Surveys, 55(4):1–36. [find](https://scholar.google.com/scholar?q=Krishna+Teja+Chitty%2DVenkata+and+Arun+K+Somani%2E+2022%2E+Neural+architecture+search+survey%3A+A+hardware+perspective%2E+ACM+Computing+Surveys%2C+55%284%29%3A1%E2%80%9336%2E)
8. Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, and 1 others. 2021. [Training verifiers to solve math word problems](https://md.research.elamur.ai/free/en/papers/cobbe-2021-training-verifiers-solve-math-word/). arXiv preprint arXiv:2110.14168.
9. Cunxiao Du, Jing Jiang, Xu Yuanchen, Jiawei Wu, Sicheng Yu, Yongqi Li, Shenggui Li, Kai Xu, Liqiang Nie, Zhaopeng Tu, and 1 others. 2024. Glide with a cape: A low-hassle method to accelerate speculative decoding. arXiv preprint arXiv:2402.02082. — [arXiv:2402.02082](https://arxiv.org/abs/2402.02082) [find](https://scholar.google.com/scholar?q=Cunxiao+Du%2C+Jing+Jiang%2C+Xu+Yuanchen%2C+Jiawei+Wu%2C+Sicheng+Yu%2C+Yongqi+Li%2C+Shenggui+Li%2C+Kai+Xu%2C+Liqiang+Nie%2C+Zhaopeng+Tu%2C+and+1+others%2E+2024%2E+Glide+with+a+cape%3A+A+low%2Dhassle+method+to+accelerate+speculati)
10. Mostafa Elhoushi, Akshat Shrivastava, Diana Liskovich, Basil Hosmer, Bram Wasti, Liangzhen Lai, Anas Mahmoud, Bilge Acun, Saurabh Agarwal, Ahmed Roman, and 1 others. 2024. [Layerskip: Enabling early exit inference and self-speculative decoding](https://md.research.elamur.ai/free/en/papers/elhoushi-2024-layerskip-enabling-early-exit-inference/). arXiv preprint arXiv:2404.16710.
11. Daya Guo, Qihao Zhu, Dejian Yang, Zhenda Xie, Kai Dong, Wentao Zhang, Guanting Chen, Xiao Bi, Yu Wu, YK Li, and 1 others. 2024. Deepseek-coder: When the large language model meets programming–the rise of code intelligence. arXiv preprint arXiv:2401.14196. [find](https://scholar.google.com/scholar?q=Daya+Guo%2C+Qihao+Zhu%2C+Dejian+Yang%2C+Zhenda+Xie%2C+Kai+Dong%2C+Wentao+Zhang%2C+Guanting+Chen%2C+Xiao+Bi%2C+Yu+Wu%2C+YK+Li%2C+and+1+others%2E+2024%2E+Deepseek%2Dcoder%3A+When+the+large+language+model+meets+programming%E2%80%93the+rise)
12. Geoffrey Hinton, Oriol Vinyals, and Jeff Dean. 2015. [Distilling the knowledge in a neural network](https://md.research.elamur.ai/free/en/papers/hinton-2015-distilling-knowledge-neural-network/). arXiv preprint arXiv:1503.02531.
13. Kaixuan Huang, Xudong Guo, and Mengdi Wang. 2024. Specdec++: Boosting speculative decoding via adaptive candidate lengths. arXiv preprint arXiv:2405.19715. — [arXiv:2405.19715](https://arxiv.org/abs/2405.19715) [find](https://scholar.google.com/scholar?q=Kaixuan+Huang%2C+Xudong+Guo%2C+and+Mengdi+Wang%2E+2024%2E+Specdec%2B%2B%3A+Boosting+speculative+decoding+via+adaptive+candidate+lengths%2E+arXiv+preprint+arXiv%3A2405%2E19715%2E+%E2%80%94+%5BarXiv%3A2405%2E19715%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2)
14. M Kiruthika, Vibhav Krashan Chaurasiya, Balaraman Sundarambal, N Kirubakaran, and 1 others. 2025. A comprehensive review of low-rank decomposition techniques for parameter efficient fine-tuning. In 2025 International Conference on Data Science, Agents & Artificial Intelligence (ICDSAAI), pages 1–6.… [find](https://scholar.google.com/scholar?q=M+Kiruthika%2C+Vibhav+Krashan+Chaurasiya%2C+Balaraman+Sundarambal%2C+N+Kirubakaran%2C+and+1+others%2E+2025%2E+A+comprehensive+review+of+low%2Drank+decomposition+techniques+for+parameter+efficient+fine%2Dtuning%2E+In+20)
15. Jiedong Lang, Zhehao Guo, and Shuyu Huang. 2024. A comprehensive study on quantization techniques for large language models. In 2024 4th International Conference on Artificial Intelligence, Robotics, and Communication (ICAIRC), pages 224–231. IEEE. [find](https://scholar.google.com/scholar?q=Jiedong+Lang%2C+Zhehao+Guo%2C+and+Shuyu+Huang%2E+2024%2E+A+comprehensive+study+on+quantization+techniques+for+large+language+models%2E+In+2024+4th+International+Conference+on+Artificial+Intelligence%2C+Robotics%2C+)
16. Yaniv Leviathan, Matan Kalman, and Yossi Matias. 2023. Fast inference from transformers via speculative decoding. In International Conference on Machine Learning, pages 19274–19286. PMLR. [find](https://scholar.google.com/scholar?q=Yaniv+Leviathan%2C+Matan+Kalman%2C+and+Yossi+Matias%2E+2023%2E+Fast+inference+from+transformers+via+speculative+decoding%2E+In+International+Conference+on+Machine+Learning%2C+pages+19274%E2%80%9319286%2E+PMLR%2E)
17. Yuhui Li, Fangyun Wei, Chao Zhang, and Hongyang Zhang. 2024. [Eagle: Speculative sampling requires rethinking feature uncertainty](https://md.research.elamur.ai/free/en/papers/li-2024-eagle-speculative-sampling-requires-rethinking/). arXiv preprint arXiv:2401.15077.
18. Tianyu Liu, Yun Li, Qitan Lv, Kai Liu, Jianchen Zhu, Winston Hu, and Xiao Sun. 2025. Pearl: Parallel speculative decoding with adaptive draft length. In The Thirteenth International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Tianyu+Liu%2C+Yun+Li%2C+Qitan+Lv%2C+Kai+Liu%2C+Jianchen+Zhu%2C+Winston+Hu%2C+and+Xiao+Sun%2E+2025%2E+Pearl%3A+Parallel+speculative+decoding+with+adaptive+draft+length%2E+In+The+Thirteenth+International+Conference+on+Lear)
19. Zhuang Liu, Mingjie Sun, Tinghui Zhou, Gao Huang, and Trevor Darrell. 2018. Rethinking the value of network pruning. arXiv preprint arXiv:1810.05270. [find](https://scholar.google.com/scholar?q=Zhuang+Liu%2C+Mingjie+Sun%2C+Tinghui+Zhou%2C+Gao+Huang%2C+and+Trevor+Darrell%2E+2018%2E+Rethinking+the+value+of+network+pruning%2E+arXiv+preprint+arXiv%3A1810%2E05270%2E)
20. Jonathan Mamou, Oren Pereg, Daniel Korat, Moshe Berchansky, Nadav Timor, Moshe Wasserblat, and Roy Schwartz. 2024. Dynamic speculation lookahead accelerates speculative decoding of large language models. arXiv preprint arXiv:2405.04304. — [arXiv:2405.04304](https://arxiv.org/abs/2405.04304) [find](https://scholar.google.com/scholar?q=Jonathan+Mamou%2C+Oren+Pereg%2C+Daniel+Korat%2C+Moshe+Berchansky%2C+Nadav+Timor%2C+Moshe+Wasserblat%2C+and+Roy+Schwartz%2E+2024%2E+Dynamic+speculation+lookahead+accelerates+speculative+decoding+of+large+language+mode)
21. Xupeng Miao, Gabriele Oliaro, Zhihao Zhang, Xinhao Cheng, Zeyu Wang, Rae Ying Yee Wong, Zhuoming Chen, Daiyaan Arfeen, Reyna Abhyankar, and Zhihao Jia. 2023. [Specinfer: Accelerating generative llm serving with speculative inference and token tree verification](https://md.research.elamur.ai/free/en/papers/miao-2023-specinfer-accelerating-generative-large-language/). arXiv preprint arXiv:2305.09781, 1(2):4…
22. Jun Zhang, Jue Wang, Huan Li, Lidan Shou, Ke Chen, Gang Chen, and Sharad Mehrotra. 2023. [Draft & verify: Lossless large language model acceleration via self-speculative decoding](https://md.research.elamur.ai/free/en/papers/zhang-2024-draft-verify-lossless-large-language/). arXiv preprint arXiv:2309.08168.
23. Giovanni Monea, Armand Joulin, and Edouard Grave. 2023. Pass: Parallel speculative sampling. arXiv preprint arXiv:2311.13581. — [arXiv:2311.13581](https://arxiv.org/abs/2311.13581) [find](https://scholar.google.com/scholar?q=Giovanni+Monea%2C+Armand+Joulin%2C+and+Edouard+Grave%2E+2023%2E+Pass%3A+Parallel+speculative+sampling%2E+arXiv+preprint+arXiv%3A2311%2E13581%2E+%E2%80%94+%5BarXiv%3A2311%2E13581%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2311%2E13581%29)
24. Peiyuan Zhang, Guangtao Zeng, Tianduo Wang, and Wei Lu. 2024a. [Tinyllama: An open-source small language model](https://md.research.elamur.ai/free/en/papers/zhang-2024-tinyllama-open-source-small-language/). arXiv preprint arXiv:2401.02385.
25. Isaac Ong, Amjad Almahairi, Vincent Wu, Wei-Lin Chiang, Tianhao Wu, Joseph E Gonzalez, M Waleed Kadous, and Ion Stoica. 2024. [Routellm: Learning to route llms with preference data](https://md.research.elamur.ai/free/en/papers/ong-2024-routellm-learning-route-llms-preference/). arXiv preprint arXiv:2406.18665.
26. Ziyin Zhang, Jiahao Xu, Tian Liang, Xingyu Chen, Zhiwei He, Rui Wang, and Zhaopeng Tu. 2024b. Draft model knows when to stop: A self-verification length policy for speculative decoding. arXiv preprint arXiv:2411.18462. — [arXiv:2411.18462](https://arxiv.org/abs/2411.18462) [find](https://scholar.google.com/scholar?q=Ziyin+Zhang%2C+Jiahao+Xu%2C+Tian+Liang%2C+Xingyu+Chen%2C+Zhiwei+He%2C+Rui+Wang%2C+and+Zhaopeng+Tu%2E+2024b%2E+Draft+model+knows+when+to+stop%3A+A+self%2Dverification+length+policy+for+speculative+decoding%2E+arXiv+preprint)
27. Sihyeong Park, Sungryeol Jeon, Chaelyn Lee, Seokhun Jeon, Byung-Soo Kim, and Jemin Lee. 2025. A survey on inference engines for large language models: Perspectives on optimization and efficiency. arXiv preprint arXiv:2505.01658. — [arXiv:2505.01658](https://arxiv.org/abs/2505.01658) [find](https://scholar.google.com/scholar?q=Sihyeong+Park%2C+Sungryeol+Jeon%2C+Chaelyn+Lee%2C+Seokhun+Jeon%2C+Byung%2DSoo+Kim%2C+and+Jemin+Lee%2E+2025%2E+A+survey+on+inference+engines+for+large+language+models%3A+Perspectives+on+optimization+and+efficiency%2E+arXi)
28. Juntao Zhao, Wenhao Lu, Sheng Wang, Lingpeng Kong, and Chuan Wu. 2024a. Qspec: Speculative decoding with complementary quantization schemes. arXiv preprint arXiv:2410.11305. [find](https://scholar.google.com/scholar?q=Juntao+Zhao%2C+Wenhao+Lu%2C+Sheng+Wang%2C+Lingpeng+Kong%2C+and+Chuan+Wu%2E+2024a%2E+Qspec%3A+Speculative+decoding+with+complementary+quantization+schemes%2E+arXiv+preprint+arXiv%3A2410%2E11305%2E)
29. Baptiste Roziere, Jonas Gehring, Fabian Gloeckle, Sten Sootla, Itai Gat, Xiaoqing Ellen Tan, Yossi Adi, Jingyu Liu, Romain Sauvestre, Tal Remez, and 1 others. 2023. [Code llama: Open foundation models for code](https://md.research.elamur.ai/free/en/papers/roziere-2024-code-llama-open-foundation-models/). arXiv preprint arXiv:2308.12950.
30. Yao Zhao, Zhitian Xie, Chen Liang, Chenyi Zhuang, and Jinjie Gu. 2024b. Lookahead: An inference acceleration framework for large language model with lossless generation accuracy. In Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pages 6344–6355. [find](https://scholar.google.com/scholar?q=Yao+Zhao%2C+Zhitian+Xie%2C+Chen+Liang%2C+Chenyi+Zhuang%2C+and+Jinjie+Gu%2E+2024b%2E+Lookahead%3A+An+inference+acceleration+framework+for+large+language+model+with+lossless+generation+accuracy%2E+In+Proceedings+of+the)
31. Noam Shazeer, Azalia Mirhoseini, Krzysztof Maziarz, Andy Davis, Quoc Le, Geoffrey Hinton, and Jeff Dean. 2017. Outrageously large neural networks: The sparsely-gated mixture-of-experts layer. arXiv preprint arXiv:1701.06538. [find](https://scholar.google.com/scholar?q=Noam+Shazeer%2C+Azalia+Mirhoseini%2C+Krzysztof+Maziarz%2C+Andy+Davis%2C+Quoc+Le%2C+Geoffrey+Hinton%2C+and+Jeff+Dean%2E+2017%2E+Outrageously+large+neural+networks%3A+The+sparsely%2Dgated+mixture%2Dof%2Dexperts+layer%2E+arXiv+pr)
32. Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric Xing, and 1 others. 2023. [Judging llm-as-a-judge with mt-bench and chatbot arena](https://md.research.elamur.ai/free/en/papers/zheng-2023-judging-llm-judge-mt-bench/). Advances in neural information processing systems, 36:46595–46623.
33. Freda Shi, Mirac Suzgun, Markus Freitag, Xuezhi Wang, Suraj Srivats, Soroush Vosoughi, Hyung Won Chung, Yi Tay, Sebastian Ruder, Denny Zhou, and 1 others. 2022. [Language models are multilingual chain-of-thought reasoners](https://md.research.elamur.ai/free/en/papers/shi-2022-language-models-multilingual-chain-thought/). arXiv preprint arXiv:2210.03057.
34. A Benjamin Spector and Chris Re. 2023. Accelerating llm inference with staged speculative decoding. arXiv preprint arXiv:2308.04623. — [arXiv:2308.04623](https://arxiv.org/abs/2308.04623) [find](https://scholar.google.com/scholar?q=A+Benjamin+Spector+and+Chris+Re%2E+2023%2E+Accelerating+llm+inference+with+staged+speculative+decoding%2E+arXiv+preprint+arXiv%3A2308%2E04623%2E+%E2%80%94+%5BarXiv%3A2308%2E04623%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2308%2E04623%29)
35. B Wenxiao Wang, Wei Chen, Yicong Luo, Yongliu Long, Zhengkai Lin, Liye Zhang, Binbin Lin, Deng Cai, and Xiaofei He. 2024. Model compression and efficient inference for [large language models: A survey](https://md.research.elamur.ai/free/en/papers/minaee-2024-large-language-models-survey/). arXiv preprint arXiv:2402.09748. — [arXiv:2402.09748](https://arxiv.org/abs/2402.09748)
36. Heming Xia, Zhe Yang, Qingxiu Dong, Peiyi Wang, Yongqi Li, Tao Ge, Tianyu Liu, Wenjie Li, and Zhifang Sui. 2024. [Unlocking efficiency in large language model inference: A comprehensive survey of speculative decoding](https://md.research.elamur.ai/free/en/papers/xia-2024-unlocking-efficiency-large-language-model/). arXiv preprint arXiv:2401.07851.
37. An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, and 1 others. Algorithm Here, we give the whole algorithm of our SPIDE in detail in Algorithm 1. More Observations on Confidence and Acceptance In the figure 5, we provide observati… [find](https://scholar.google.com/scholar?q=An+Yang%2C+Anfeng+Li%2C+Baosong+Yang%2C+Beichen+Zhang%2C+Binyuan+Hui%2C+Bo+Zheng%2C+Bowen+Yu%2C+Chang+Gao%2C+Chengen+Huang%2C+Chenxu+Lv%2C+and+1+others%2E+Algorithm+Here%2C+we+give+the+whole+algorithm+of+our+SPIDE+in+detail+)

*Updated 2 November 2026*