---
title: "RACER — paper by Zhang et al. (2026) — summary and ideas"
description: "Its logits tree builds draft branches by reusing the logit from the most recent earlier occurrence of the same token (called… 37 references."
authors: ["Zihong Zhang", "Zuchao Li", "Lefei Zhang", "Ping Wang", "Hai Zhao"]
year: 2026
venue: "Findings of the Association for Computational Linguistics: ACL 2026"
source: "https://aclanthology.org/2026.findings-acl.998/"
doi: "10.18653/v1/2026.findings-acl.998"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/zhang-2026-racer-retrieval-augmented-contextual-rapid/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/zhang-2026-racer-retrieval-augmented-contextual-rapid/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/16/) / RACER: Retrieval-Augmented Contextual Rapid Speculative Decoding

# RACER: Retrieval-Augmented Contextual Rapid Speculative Decoding

**Authors:** Zihong Zhang, [Zuchao Li](https://md.research.elamur.ai/free/en/authors/zuchao-li/), [Lefei Zhang](https://md.research.elamur.ai/free/en/authors/lefei-zhang/), [Ping Wang](https://md.research.elamur.ai/free/en/authors/ping-wang/) and [Hai Zhao](https://md.research.elamur.ai/free/en/authors/hai-zhao/) · **Year:** 2026 · **Venue:** [Findings of the Association for Computational Linguistics: ACL 2026](https://md.research.elamur.ai/free/en/venues/findings/) ([Findings of the Association for Computational Linguistics 2026](https://md.research.elamur.ai/free/en/venues/findings/2026/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2026.findings-acl.998/) · [PDF](https://aclanthology.org/2026.findings-acl.998.pdf) · [DOI 10.18653/v1/2026.findings-acl.998](https://doi.org/10.18653/v1/2026.findings-acl.998) · [OpenAlex](https://openalex.org/W7166840458)

**Citations:** 1 citations (OpenAlex, 2026-10-14)

## Summary
Large language models decode one token at a time, which makes inference slow. Speculative decoding speeds this up by drafting several tokens and checking them in parallel with the target model. Training-free variants have two weaknesses. Retrieval-based drafters fail when the context contains no exact match. Logit-based drafters have no structural guidance beyond their own predictions. RACER combines both signals without any extra training. Its logits tree builds draft branches by reusing the logit from the most recent earlier occurrence of the same token (called "copy-logit"), and each deeper level gets half its parent's breadth. Its retrieval component is an n-gram trie automaton with failure links and LRU eviction, which keeps memory bounded. The two sources are merged into a single draft tree that is verified using tree attention. Evaluation covers Spec-Bench, HumanEval and the Chinese math benchmark MGSM-ZH. RACER delivers more than 2× speedup over standard autoregressive decoding and beats earlier training-free methods. It works as a plug-in, with no auxiliary draft model to train or host.

## Key points
- **Copy-logit vs. last-logit drafting:** on Spec-Bench with Vicuna-7B, copy-logit raised mean accepted tokens (MAT) from 1.57 to 1.87. Expanding it into a 9-ary analogy tree pushed MAT to 2.34. Trends were similar for OpenPangu-7B and Qwen3-1.7B.
- **Retrieval-only RACER (Vicuna 7B, RTX 3090):** it averaged MAT 2.53 and a 2.05× speedup, against SAMD at 1.98 MAT and PLD at 1.95 MAT. It also outperformed REST. The largest gain was on MGSM-ZH, with MAT 3.32.
- **Integration ablation:** the full "Merge" strategy reached MAT 3.00 and a 2.18× speedup. This beat a fixed budget split ("Half": 2.69 / 1.97×) and fallback switching ("Hard": 2.77 / 2.11×).
- **Model-agnostic logits component:** unlike Token Recycling, whose static tree was tuned for Vicuna-7B, RACER's logits component needs no model-specific tuning and scales better to larger models.
- **Availability and limits:** code is released at github.com/hkr04/RACER. The method has so far been tested only on text and not on multimodal inputs.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Shanghai Jiao Tong University](https://md.research.elamur.ai/free/en/universities/shanghai-jiao-tong-university/) — University, Shanghai, China · 149 works · 16,161 citations
- [Wuhan University](https://md.research.elamur.ai/free/en/universities/wuhan-university/) — University, Wuhan, China · 35 works · 1,127 citations

📚 Shelves (5): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works · [🗄️ Information Systems](https://md.research.elamur.ai/free/en/shelves/computer-science/information-systems/) · 1,437 works

🧭 Topics (2): [🔍 Information retrieval & search](https://md.research.elamur.ai/free/en/topics/information-retrieval/) · 473 works · [🪶 Efficient models & compression](https://md.research.elamur.ai/free/en/topics/efficient-models/) · 242 works

## Cite

Cite as: Zihong Zhang, Zuchao Li, Lefei Zhang, Ping Wang and Hai Zhao. 2026. RACER: Retrieval-Augmented Contextual Rapid Speculative Decoding. Findings of the Association for Computational Linguistics: ACL 2026.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{zhang2026racer,
  title = {{RACER: Retrieval-Augmented Contextual Rapid Speculative Decoding}},
  author = {Zihong Zhang, Zuchao Li, Lefei Zhang, Ping Wang and Hai Zhao},
  booktitle = {Findings of the Association for Computational Linguistics: ACL 2026},
  year = {2026},
  doi = {10.18653/v1/2026.findings-acl.998},
  url = {https://doi.org/10.18653/v1/2026.findings-acl.998}
}
```

- [← previous RA-RRG: Multimodal Retrieval-Augmented Radiology Report Generation with Key Phrase Extraction](https://md.research.elamur.ai/free/en/papers/park-2026-ra-rrg-multimodal-retrieval-augmented/)
- [next → RAFFLES: Reasoning-based Attribution of Faults for LLM Systems](https://md.research.elamur.ai/free/en/papers/zhu-2026-raffles-reasoning-based-attribution-faults/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/16/)

## References

37 references, 14 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Alfred V Aho and Margaret J Corasick. 1975. Efficient string matching: an aid to bibliographic search. Communications of the ACM, 18(6):333–340. [find](https://scholar.google.com/scholar?q=Alfred+V+Aho+and+Margaret+J+Corasick%2E+1975%2E+Efficient+string+matching%3A+an+aid+to+bibliographic+search%2E+Communications+of+the+ACM%2C+18%286%29%3A333%E2%80%93340%2E)
2. Jinze Bai, Shuai Bai, Yunfei Chu, Zeyu Cui, Kai Dang, Xiaodong Deng, Yang Fan, Wenbin Ge, Yu Han, Fei Huang, Binyuan Hui, Luo Ji, Mei Li, Junyang Lin, Runji Lin, Dayiheng Liu, Gao Liu, Chengqiang Lu, Keming Lu, and 29 others. 2023. [Qwen technical report](https://md.research.elamur.ai/free/en/papers/bai-2023-qwen-technical-report/). CoRR, abs/2309.16609.
3. Anselm Blumer, Janet Blumer, Andrzej Ehrenfeucht, David Haussler, and Ross McConnell. 1984. Building the minimal dfa for the set of all subwords of a word on-line in linear time. In International Colloquium on Automata, Languages, and Programming, pages 109–118. Springer. [find](https://scholar.google.com/scholar?q=Anselm+Blumer%2C+Janet+Blumer%2C+Andrzej+Ehrenfeucht%2C+David+Haussler%2C+and+Ross+McConnell%2E+1984%2E+Building+the+minimal+dfa+for+the+set+of+all+subwords+of+a+word+on%2Dline+in+linear+time%2E+In+International+Coll)
4. Tianle Cai, Yuhong Li, Zhengyang Geng, Hongwu Peng, Jason D. Lee, Deming Chen, and Tri Dao. 2024. [Medusa: Simple llm inference acceleration framework with multiple decoding heads](https://md.research.elamur.ai/free/en/papers/cai-2024-medusa-simple-llm-inference-acceleration/). arXiv preprint arXiv: 2401.10774.
5. Charlie Chen, Sebastian Borgeaud, Geoffrey Irving, Jean-Baptiste Lespiau, Laurent Sifre, and John Jumper. 2023. Accelerating large language model decoding with speculative sampling. Preprint, arXiv:2302.01318. [find](https://scholar.google.com/scholar?q=Charlie+Chen%2C+Sebastian+Borgeaud%2C+Geoffrey+Irving%2C+Jean%2DBaptiste+Lespiau%2C+Laurent+Sifre%2C+and+John+Jumper%2E+2023%2E+Accelerating+large+language+model+decoding+with+speculative+sampling%2E+Preprint%2C+arXiv%3A23)
6. Hanting Chen, Yasheng Wang, Kai Han, Dong Li, Lin Li, Zhenni Bi, Jinpeng Li, Haoyu Wang, Fei Mi, Mingjian Zhu, Bin Wang, Kaikai Song, Yifei Fu, Xu He, Yu Luo, Chong Zhu, Quan He, Xueyu Wu, Wei He, and 5 others. 2025. [Pangu embedded: An efficient dual-system LLM reasoner with metacognition](https://md.research.elamur.ai/free/en/papers/chen-2025-pangu-embedded-efficient-dual/). CoRR, abs…
7. Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Pondé de Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, Alex Ray, Raul Puri, Gretchen Krueger, Michael Petrov, Heidy Khlaaf, Girish Sastry, Pamela Mishkin, Brooke Chan, Scott Gray, and 39 others. 202… — [Evaluating Large Language Models Trained on Code](https://md.research.elamur.ai/free/en/papers/chen-2021-evaluating-large-language-models-trained/)
8. Wei-Lin Chiang, Zhuohan Li, Zi Lin, Ying Sheng, Zhanghao Wu, Hao Zhang, Lianmin Zheng, Siyuan Zhuang, Yonghao Zhuang, Joseph E. Gonzalez, Ion Stoica, and Eric P. Xing. 2023. Vicuna: An open-source chatbot impressing gpt-4 with 90%* chatgpt quality. [find](https://scholar.google.com/scholar?q=Wei%2DLin+Chiang%2C+Zhuohan+Li%2C+Zi+Lin%2C+Ying+Sheng%2C+Zhanghao+Wu%2C+Hao+Zhang%2C+Lianmin+Zheng%2C+Siyuan+Zhuang%2C+Yonghao+Zhuang%2C+Joseph+E%2E+Gonzalez%2C+Ion+Stoica%2C+and+Eric+P%2E+Xing%2E+2023%2E+Vicuna%3A+An+open%2Dsource+cha)
9. Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman. 2021. [Training verifiers to solve math word problems](https://md.research.elamur.ai/free/en/papers/cobbe-2021-training-verifiers-solve-math-word/). arXiv preprint arXiv:2110.14168.
10. DeepSeek-AI. 2024. Deepseek-v3 technical report. CoRR, abs/2412.19437. [find](https://scholar.google.com/scholar?q=DeepSeek%2DAI%2E+2024%2E+Deepseek%2Dv3+technical+report%2E+CoRR%2C+abs%2F2412%2E19437%2E)
11. Zhichen Dong, Zhanhui Zhou, Zhixuan Liu, Chao Yang, and Chaochao Lu. 2025. Emergent response planning in LLMs. In Forty-second International Conference on Machine Learning. [find](https://scholar.google.com/scholar?q=Zhichen+Dong%2C+Zhanhui+Zhou%2C+Zhixuan+Liu%2C+Chao+Yang%2C+and+Chaochao+Lu%2E+2025%2E+Emergent+response+planning+in+LLMs%2E+In+Forty%2Dsecond+International+Conference+on+Machine+Learning%2E)
12. Fabian Gloeckle, Badr Youbi Idrissi, Baptiste Rozière, David Lopez-Paz, and Gabriel Synnaeve. 2024. Better & faster large language models via multi-token prediction. arXiv preprint arXiv:2404.19737. [find](https://scholar.google.com/scholar?q=Fabian+Gloeckle%2C+Badr+Youbi+Idrissi%2C+Baptiste+Rozi%C3%A8re%2C+David+Lopez%2DPaz%2C+and+Gabriel+Synnaeve%2E+2024%2E+Better+%26+faster+large+language+models+via+multi%2Dtoken+prediction%2E+arXiv+preprint+arXiv%3A2404%2E19737%2E)
13. Zhenyu He, Zexuan Zhong, Tianle Cai, Jason D Lee, and Di He. 2023. Rest: Retrieval-based speculative decoding. Preprint, arXiv:2311.08252. [find](https://scholar.google.com/scholar?q=Zhenyu+He%2C+Zexuan+Zhong%2C+Tianle+Cai%2C+Jason+D+Lee%2C+and+Di+He%2E+2023%2E+Rest%3A+Retrieval%2Dbased+speculative+decoding%2E+Preprint%2C+arXiv%3A2311%2E08252%2E)
14. Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. 2021. [Measuring mathematical problem solving with the math dataset](https://md.research.elamur.ai/free/en/papers/hendrycks-2021-measuring-mathematical-problem-solving-math/). arXiv preprint arXiv:2103.03874.
15. Yuxuan Hu, Ke Wang, Xiaokang Zhang, Fanjin Zhang, Cuiping Li, Hong Chen, and Jing Zhang. 2025. SAM decoding: Speculative decoding via suffix automaton. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 12187–12204, Vienna, Austr… [find](https://scholar.google.com/scholar?q=Yuxuan+Hu%2C+Ke+Wang%2C+Xiaokang+Zhang%2C+Fanjin+Zhang%2C+Cuiping+Li%2C+Hong+Chen%2C+and+Jing+Zhang%2E+2025%2E+SAM+decoding%3A+Speculative+decoding+via+suffix+automaton%2E+In+Proceedings+of+the+63rd+Annual+Meeting+of+the)
16. Donald E Knuth, James H Morris, Jr, and Vaughan R Pratt. 1977. Fast pattern matching in strings. SIAM journal on computing, 6(2):323–350. [find](https://scholar.google.com/scholar?q=Donald+E+Knuth%2C+James+H+Morris%2C+Jr%2C+and+Vaughan+R+Pratt%2E+1977%2E+Fast+pattern+matching+in+strings%2E+SIAM+journal+on+computing%2C+6%282%29%3A323%E2%80%93350%2E)
17. Yaniv Leviathan, Matan Kalman, and Yossi Matias. 2023. Fast inference from transformers via speculative decoding. In International Conference on Machine Learning, pages 19274–19286. PMLR. [find](https://scholar.google.com/scholar?q=Yaniv+Leviathan%2C+Matan+Kalman%2C+and+Yossi+Matias%2E+2023%2E+Fast+inference+from+transformers+via+speculative+decoding%2E+In+International+Conference+on+Machine+Learning%2C+pages+19274%E2%80%9319286%2E+PMLR%2E)
18. Yuhui Li, Fangyun Wei, Chao Zhang, and Hongyang Zhang. 2024a. [EAGLE-2: Faster inference of language models with dynamic draft trees](https://md.research.elamur.ai/free/en/papers/li-2024-eagle-2-faster-inference-language/). In Empirical Methods in Natural Language Processing.
19. Yuhui Li, Fangyun Wei, Chao Zhang, and Hongyang Zhang. 2024b. [EAGLE: Speculative sampling requires rethinking feature uncertainty](https://md.research.elamur.ai/free/en/papers/li-2024-eagle-speculative-sampling-requires-rethinking/). In International Conference on Machine Learning.
20. Yuhui Li, Fangyun Wei, Chao Zhang, and Hongyang Zhang. 2025. EAGLE-3: Scaling up inference acceleration of large language models via training-time test. Preprint, arXiv:2503.01840. — [arXiv:2503.01840](https://arxiv.org/abs/2503.01840) [find](https://scholar.google.com/scholar?q=Yuhui+Li%2C+Fangyun+Wei%2C+Chao+Zhang%2C+and+Hongyang+Zhang%2E+2025%2E+EAGLE%2D3%3A+Scaling+up+inference+acceleration+of+large+language+models+via+training%2Dtime+test%2E+Preprint%2C+arXiv%3A2503%2E01840%2E+%E2%80%94+%5BarXiv%3A2503%2E01840)
21. Tianyu Liu, Yun Li, Qitan Lv, Kai Liu, Jianchen Zhu, Winston Hu, and Xiao Sun. 2025a. PEARL: Parallel speculative decoding with adaptive draft length. In The Thirteenth International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Tianyu+Liu%2C+Yun+Li%2C+Qitan+Lv%2C+Kai+Liu%2C+Jianchen+Zhu%2C+Winston+Hu%2C+and+Xiao+Sun%2E+2025a%2E+PEARL%3A+Parallel+speculative+decoding+with+adaptive+draft+length%2E+In+The+Thirteenth+International+Conference+on+Lea)
22. Tianyu Liu, Qitan Lv, Hao Li, Xing Gao, and Xiao Sun. 2025b. [Logitspec: Accelerating retrieval-based speculative decoding via next next token speculation](https://md.research.elamur.ai/free/en/papers/liu-2026-logitspec-accelerating-retrieval-based-speculative/). arXiv preprint arXiv:2507.01449.
23. Xianzhen Luo, Yixuan Wang, Qingfu Zhu, Zhiming Zhang, Xuanyu Zhang, Qing Yang, and Dongliang Xu. 2025. [Turning trash into treasure: Accelerating inference of large language models with token recycling](https://md.research.elamur.ai/free/en/papers/luo-2025-turning-trash-into-treasure-accelerating/). In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume…
24. Shi Luohe, Hongyi Zhang, Yao Yao, Zuchao Li, and Hai Zhao. 2024. Keep the cost down: A review on methods to optimize LLM’s KV-cache consumption. In First Conference on Language Modeling. [find](https://scholar.google.com/scholar?q=Shi+Luohe%2C+Hongyi+Zhang%2C+Yao+Yao%2C+Zuchao+Li%2C+and+Hai+Zhao%2E+2024%2E+Keep+the+cost+down%3A+A+review+on+methods+to+optimize+LLM%E2%80%99s+KV%2Dcache+consumption%2E+In+First+Conference+on+Language+Modeling%2E)
25. Ziyang Ma, Zuchao Li, Lefei Zhang, Gui-Song Xia, Bo Du, Liangpei Zhang, and Dacheng Tao. 2025. Model hemorrhage and the robustness limits of large language models. CoRR, abs/2503.23924. models are multilingual chain-of-thought reasoners. Preprint, arXiv:2210.03057. — [arXiv:2503.23924](https://arxiv.org/abs/2503.23924) [find](https://scholar.google.com/scholar?q=Ziyang+Ma%2C+Zuchao+Li%2C+Lefei+Zhang%2C+Gui%2DSong+Xia%2C+Bo+Du%2C+Liangpei+Zhang%2C+and+Dacheng+Tao%2E+2025%2E+Model+hemorrhage+and+the+robustness+limits+of+large+language+models%2E+CoRR%2C+abs%2F2503%2E23924%2E+models+are+mul)
26. Luohe Shi, Zuchao Li, Lefei Zhang, Baoyuan Qi, Guoming Liu, and Hai Zhao. 2026. Scaling llm speculative decoding: Non-autoregressive forecasting in large-batch scenarios. Proceedings of the AAAI Conference on Artificial Intelligence, 40:32947–32955. [find](https://scholar.google.com/scholar?q=Luohe+Shi%2C+Zuchao+Li%2C+Lefei+Zhang%2C+Baoyuan+Qi%2C+Guoming+Liu%2C+and+Hai+Zhao%2E+2026%2E+Scaling+llm+speculative+decoding%3A+Non%2Dautoregressive+forecasting+in+large%2Dbatch+scenarios%2E+Proceedings+of+the+AAAI+Confe)
27. Llama Team. 2024. [The llama 3 herd of models](https://md.research.elamur.ai/free/en/papers/grattafiori-2024-llama-3-herd-models/). CoRR, abs/2407.21783.
28. Qwen Team. 2025. [Qwen3 technical report](https://md.research.elamur.ai/free/en/papers/yang-2025-qwen3-technical-report/). CoRR, abs/2505.09388.
29. Hemish Veeraboina. 2023. Aime problem set 1983-2024. [find](https://scholar.google.com/scholar?q=Hemish+Veeraboina%2E+2023%2E+Aime+problem+set+1983%2D2024%2E)
30. Heming Xia, Zhe Yang, Qingxiu Dong, Peiyi Wang, Yongqi Li, Tao Ge, Tianyu Liu, Wenjie Li, and Zhifang Sui. 2024. [Unlocking efficiency in large language model inference: A comprehensive survey of speculative decoding](https://md.research.elamur.ai/free/en/papers/xia-2024-unlocking-efficiency-large-language-model/). In Findings of the Association for Computational Linguistics ACL 2024, pages 7655–7…
31. Lefan Zhang, Xiaodan Wang, Yanhua Huang, and Ruiwen Xu. 2025. Learning harmonized representations for speculative sampling. In The Thirteenth International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Lefan+Zhang%2C+Xiaodan+Wang%2C+Yanhua+Huang%2C+and+Ruiwen+Xu%2E+2025%2E+Learning+harmonized+representations+for+speculative+sampling%2E+In+The+Thirteenth+International+Conference+on+Learning+Representations%2E)
32. Udi Manber and Gene Myers. 1993. Suffix arrays: a new method for on-line string searches. siam Journal on Computing, 22(5):935–948. [find](https://scholar.google.com/scholar?q=Udi+Manber+and+Gene+Myers%2E+1993%2E+Suffix+arrays%3A+a+new+method+for+on%2Dline+string+searches%2E+siam+Journal+on+Computing%2C+22%285%29%3A935%E2%80%93948%2E)
33. Somesh Mehra, Javier Alonso Garcia, and Lukas Mauch. 2025. On multi-token prediction for efficient llm inference. arXiv preprint arXiv:2502.09419. — [arXiv:2502.09419](https://arxiv.org/abs/2502.09419) [find](https://scholar.google.com/scholar?q=Somesh+Mehra%2C+Javier+Alonso+Garcia%2C+and+Lukas+Mauch%2E+2025%2E+On+multi%2Dtoken+prediction+for+efficient+llm+inference%2E+arXiv+preprint+arXiv%3A2502%2E09419%2E+%E2%80%94+%5BarXiv%3A2502%2E09419%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2502%2E09419)
34. Xupeng Miao, Gabriele Oliaro, Zhihao Zhang, Xinhao Cheng, Zeyu Wang, Zhengxin Zhang, Rae Ying Yee Wong, Alan Zhu, Lijie Yang, Xiaoxiang Shi, Chunan Shi, Zhuoming Chen, Daiyaan Arfeen, Reyna Abhyankar, and Zhihao Jia. 2024. Specinfer: Accelerating large language model serving with tree-based speculat… [find](https://scholar.google.com/scholar?q=Xupeng+Miao%2C+Gabriele+Oliaro%2C+Zhihao+Zhang%2C+Xinhao+Cheng%2C+Zeyu+Wang%2C+Zhengxin+Zhang%2C+Rae+Ying+Yee+Wong%2C+Alan+Zhu%2C+Lijie+Yang%2C+Xiaoxiang+Shi%2C+Chunan+Shi%2C+Zhuoming+Chen%2C+Daiyaan+Arfeen%2C+Reyna+Abhyankar%2C)
35. OpenAI. 2025. [gpt-oss-120b & gpt-oss-20b model card](https://md.research.elamur.ai/free/en/papers/agarwal-2025-gpt-oss-120b-gpt-oss/). Preprint, arXiv:2508.10925.
36. Apoorv Saxena. 2023. Prompt lookup decoding. [find](https://scholar.google.com/scholar?q=Apoorv+Saxena%2E+2023%2E+Prompt+lookup+decoding%2E)
37. Freda Shi, Mirac Suzgun, Markus Freitag, Xuezhi Wang, Suraj Srivats, Soroush Vosoughi, Hyung Won Chung, Yi Tay, Sebastian Ruder, Denny Zhou, Dipanjan Das, and Jason Wei. 2022. Language [find](https://scholar.google.com/scholar?q=Freda+Shi%2C+Mirac+Suzgun%2C+Markus+Freitag%2C+Xuezhi+Wang%2C+Suraj+Srivats%2C+Soroush+Vosoughi%2C+Hyung+Won+Chung%2C+Yi+Tay%2C+Sebastian+Ruder%2C+Denny+Zhou%2C+Dipanjan+Das%2C+and+Jason+Wei%2E+2022%2E+Language)

*Updated 1 November 2026*