---
title: "Who cites Orca: A Distributed Serving — 478 citing papers"
description: "478 papers of the Elamur Research library cite «Orca: A Distributed Serving System for Transformer-Based Generative Models», found in their…"
canonical: "https://research.elamur.ai/free/en/papers/yu-2022-orca-distributed-serving-system-transformer/cited-by/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/yu-2022-orca-distributed-serving-system-transformer/cited-by/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [Orca: A Distributed Serving System for…](https://md.research.elamur.ai/free/en/papers/yu-2022-orca-distributed-serving-system-transformer/) / cited by

# Who cites «Orca: A Distributed Serving System for Transformer-Based Generative Models»

478 papers of our library cite it (of the 149,439 whose bibliographies we read) · 38 with a page here, the most cited first · [← back to «Orca: A Distributed Serving System for Transformer-Based Generative Models»](https://md.research.elamur.ai/free/en/papers/yu-2022-orca-distributed-serving-system-transformer/)

1. **[SGLang: Efficient Execution of Structured Language Model Programs](https://md.research.elamur.ai/free/en/papers/zheng-2023-sglang-efficient-execution-structured-language/) [Conference on Neural Information Processing Systems (NeurIPS)](https://md.research.elamur.ai/free/en/venues/neural-information-processing-systems/)**Lianmin Zheng, Liangsheng Yin, Zhiqiang Xie, et al.*2023 · 729 citations*
2. [**HybridFlow: A Flexible and Efficient RLHF Framework arXiv:2409.19256**Guangming Sheng, Chi Zhang, Zilingfeng Ye, et al.*2024 · 624 citations*](https://md.research.elamur.ai/free/en/papers/sheng-2024-hybridflow-flexible-efficient-rlhf-framework/)
3. [**DistServe: Disaggregating Prefill and Decoding for Goodput-optimized Large Language Model Serving OSDI 2024**Yinmin Zhong, Shengyu Liu, Junda Chen, et al.*2024 · 409 citations*](https://md.research.elamur.ai/free/en/papers/zhong-2024-distserve-disaggregating-prefill-decoding-goodput/)
4. [**Yi: Open Foundation Models by 01.AI arXiv:2403.04652**Alex Young, Bei Chen, Chao Li, et al.*2024 · 408 citations*](https://md.research.elamur.ai/free/en/papers/ai-2024-yi-open-foundation-models-01/)
5. [**FlexGen: High-Throughput Generative Inference of Large Language Models with a Single GPU arXiv:2303.06865**Ying Sheng, Lianmin Zheng, Binhang Yuan, et al.*2023 · 402 citations*](https://md.research.elamur.ai/free/en/papers/sheng-2023-flexgen-high-throughput-generative-inference/)
6. [**Splitwise: Efficient generative LLM inference using phase splitting ISCA 2024**Pratyush Patel, Esha Choukse, Chaojie Zhang, et al.*2023 · 379 citations*](https://md.research.elamur.ai/free/en/papers/patel-2023-splitwise-efficient-generative-llm-inference/)
7. [**Taming Throughput-Latency Tradeoff in LLM Inference with Sarathi-Serve OSDI 2024**Amey Agrawal, Nitin Kedia, Ashish Panwar, et al.*2024 · 369 citations*](https://md.research.elamur.ai/free/en/papers/agrawal-2024-taming-throughput-latency-tradeoff-llm/)
8. **[PowerInfer: Fast Large Language Model Serving with a Consumer-grade GPU](https://md.research.elamur.ai/free/en/papers/song-2023-powerinfer-fast-large-language-model/) [ACM SIGOPS Symposium on Operating Systems Principles (SOSP)](https://md.research.elamur.ai/free/en/venues/acm-sigops-symposium-on-operating-systems-principles-sosp/)**Yixin Song, Zeyu Mi, Haotong Xie, et al.*2023 · 272 citations*
9. **[DynamoLLM: Designing LLM Inference Clusters for Performance and Energy Efficiency](https://md.research.elamur.ai/free/en/papers/stojkovic-2024-dynamollm-designing-llm-inference-clusters/) [Proceedings - International Symposium on High-Performance Computer Architecture/Proceedings](https://md.research.elamur.ai/free/en/venues/proceedings-international-symposium-on-high-performance-computer-architecture/)**Jovan Stojkovic, Chaojie Zhang, Inigo Goiri, et al.*2024 · 218 citations*
10. **[Deja Vu: Contextual Sparsity for Efficient LLMs at Inference Time](https://md.research.elamur.ai/free/en/papers/liu-2023-deja-vu-contextual-sparsity-efficient/) [International Conference on Machine Learning (ICML)](https://md.research.elamur.ai/free/en/venues/international-conference-on-machine-learning/)**Zichang Liu, Jue Wang, Tri Dao, et al.*2023 · 185 citations*
11. [**FlashInfer: Efficient and Customizable Attention Engine for LLM Inference Serving MLSys 2025**Zihao Ye, Lequn Chen, Ruihang Lai, et al.*2025 · 177 citations*](https://md.research.elamur.ai/free/en/papers/ye-2025-flashinfer-efficient-customizable-attention-engine/)
12. [**AlpaServe: Statistical Multiplexing with Model Parallelism for Deep Learning Serving OSDI 2023**Zhuohan Li, Lianmin Zheng, Yinmin Zhong, et al.*2023 · 170 citations*](https://md.research.elamur.ai/free/en/papers/li-2023-alpaserve-statistical-multiplexing-model-parallelism/)
13. [**Mooncake: A KVCache-centric Disaggregated Architecture for LLM Serving FAST 2025**Ruoyu Qin, Zheming Li, Weiran He, et al.*2024 · 161 citations*](https://md.research.elamur.ai/free/en/papers/qin-2024-mooncake-kvcache-centric-disaggregated-architecture/)
14. [**Atom: Low-bit Quantization for Efficient and Accurate LLM Serving MLSys 2024**Yilong Zhao, Chien-Yu Lin, Kan Zhu, et al.*2023 · 156 citations*](https://md.research.elamur.ai/free/en/papers/zhao-2023-atom-low-bit-quantization-efficient/)
15. **[Draft & Verify: Lossless Large Language Model Acceleration via Self-Speculative Decoding](https://md.research.elamur.ai/free/en/papers/zhang-2024-draft-verify-lossless-large-language/) [Annual Meeting of the Association for Computational Linguistics (ACL)](https://md.research.elamur.ai/free/en/venues/acl/)**Jun Zhang, Jue Wang, Huan Li, et al.*2024 · 148 citations*
16. **[Unlocking Efficiency in Large Language Model Inference: A Comprehensive Survey of Speculative Decoding](https://md.research.elamur.ai/free/en/papers/xia-2024-unlocking-efficiency-large-language-model/) [Findings of the Association for Computational Linguistics](https://md.research.elamur.ai/free/en/venues/findings/)**Heming Xia, Zhe Yang, Qingxiu Dong, et al.*2024 · 146 citations*
17. [**SARATHI: Efficient LLM Inference by Piggybacking Decodes with Chunked Prefills arXiv 2023**Amey Agrawal, Ashish Panwar, Jayashree Mohan, et al.*2023 · 144 citations*](https://md.research.elamur.ai/free/en/papers/agrawal-2023-sarathi-efficient-llm-inference-piggybacking/)
18. [**Fast Distributed Inference Serving for Large Language Models arXiv 2023**Bingyang Wu, Yinmin Zhong, Zili Zhang, et al.*2023 · 123 citations*](https://md.research.elamur.ai/free/en/papers/wu-2023-fast-distributed-inference-serving-large/)
19. [**QServe: W4A8KV4 Quantization and System Co-design for Efficient LLM Serving MLSys 2025**Yujun Lin, Haotian Tang, Shang Yang, et al.*2024 · 120 citations*](https://md.research.elamur.ai/free/en/papers/lin-2024-qserve-w4a8kv4-quantization-system-co/)
20. [**Llumnix: Dynamic Scheduling for Large Language Model Serving OSDI 2024**Biao Sun, Ziming Huang, Hanyu Zhao, et al.*2024 · 119 citations*](https://md.research.elamur.ai/free/en/papers/sun-2024-llumnix-dynamic-scheduling-large-language/)
21. [**A Survey on Efficient Inference for Large Language Models arXiv:2404.14294**Zixuan Zhou, Xuefei Ning, Ke Hong, et al.*2024 · 113 citations*](https://md.research.elamur.ai/free/en/papers/zhou-2024-survey-efficient-inference-large-language/)
22. [**Efficient Large Language Models: A Survey arXiv:2312.03863**Zhongwei Wan, Xin Wang, Che Liu, et al.*2023 · 110 citations*](https://md.research.elamur.ai/free/en/papers/wan-2023-efficient-large-language-models-survey/)
23. **[Towards Efficient Generative Large Language Model Serving: A Survey from Algorithms to Systems](https://md.research.elamur.ai/free/en/papers/miao-2023-towards-efficient-generative-large-language/) [ACM Computing Surveys](https://md.research.elamur.ai/free/en/venues/acm-computing-surveys/)**Xupeng Miao, Gabriele Oliaro, Zhihao Zhang, et al.*2023 · 110 citations*
24. [**Cost-Efficient Large Language Model Serving for Multi-turn Conversations with CachedAttention USENIX ATC 2024**Bin Gao, Zhuomin He, Puru Sharma, et al.*2024 · 105 citations*](https://md.research.elamur.ai/free/en/papers/gao-2024-cost-efficient-large-language-model/)
25. [**ServerlessLLM: Low-Latency Serverless Inference for Large Language Models arXiv:2401.14351**Yao Fu, Leyang Xue, Yeqi Huang, et al.*2024 · 105 citations*](https://md.research.elamur.ai/free/en/papers/fu-2024-serverlessllm-low-latency-serverless-inference/)
26. [**Hybrid LLM: Cost-Efficient and Quality-Aware Query Routing arXiv:2404.14618**Dujian Ding, Ankur Mallick, Chi Wang, et al.*2024 · 98 citations*](https://md.research.elamur.ai/free/en/papers/ding-2024-hybrid-llm-cost-efficient-quality/)
27. [**LLM Inference Unveiled: Survey and Roofline Model Insights arXiv 2024**Zhihang Yuan, Yuzhang Shang, Yang Zhou, et al.*2024 · 91 citations*](https://md.research.elamur.ai/free/en/papers/yuan-2024-llm-inference-unveiled-survey-roofline/)
28. [**Vidur: A Large-Scale Simulation Framework For LLM Inference arXiv:2405.05465**Amey Agrawal, Nitin Kedia, Jayashree Mohan, et al.*2024 · 90 citations*](https://md.research.elamur.ai/free/en/papers/agrawal-2024-vidur-large-scale-simulation-framework/)
29. [**Punica: Multi-Tenant LoRA Serving MLSys 2024**Lequn Chen, Zihao Ye, Yongji Wu, et al.*2023 · 81 citations*](https://md.research.elamur.ai/free/en/papers/chen-2023-punica-multi-tenant-lora-serving/)
30. [**S-LoRA: Serving Thousands of Concurrent LoRA Adapters MLSys 2024**Ying Sheng, Shiyi Cao, Dacheng Li, et al.*2023 · 80 citations*](https://md.research.elamur.ai/free/en/papers/sheng-2023-s-lora-serving-thousands-concurrent/)
31. [**DeepSpeed-FastGen: High-throughput Text Generation for LLMs via MII and DeepSpeed-Inference arXiv 2024 (Microsoft DeepSpeed)**Connor Holmes, Masahiro Tanaka, Michael Wyatt, et al.*2024 · 78 citations*](https://md.research.elamur.ai/free/en/papers/holmes-2024-deepspeed-fastgen-high-throughput-text/)
32. [**Skeleton-of-Thought: Prompting LLMs for Efficient Parallel Generation arXiv:2307.15337**Xuefei Ning, Zinan Lin, Zixuan Zhou, et al.*2023 · 72 citations*](https://md.research.elamur.ai/free/en/papers/ning-2023-skeleton-thought-prompting-llms-efficient/)
33. [**From Decoding to Meta-Generation: Inference-time Algorithms for Large Language Models arXiv:2406.16838**Sean Welleck, Amanda Bertsch, Matthew Finlayson, et al.*2024 · 51 citations*](https://md.research.elamur.ai/free/en/papers/welleck-2024-decoding-meta-generation-inference-time/)
34. **[Faster MoE LLM Inference for Extremely Large Models](https://md.research.elamur.ai/free/en/papers/yang-2026-faster-moe-llm-inference-extremely/) [Findings of the Association for Computational Linguistics](https://md.research.elamur.ai/free/en/venues/findings/)**Haoqi Yang, Luohe Shi, Qiwei Li, et al.*2026 · 8 citations*
35. **[MixKVQ: Query-Aware Mixed-Precision KV Cache Quantization for Long-Context Reasoning](https://md.research.elamur.ai/free/en/papers/zhang-2026-mixkvq-query-aware-mixed-precision/) [Annual Meeting of the Association for Computational Linguistics (ACL)](https://md.research.elamur.ai/free/en/venues/acl/)**Tao Zhang, Ziqian Zeng, Hao Peng, et al.*2026 · 5 citations*
36. **[SSSD: Simply-Scalable Speculative Decoding](https://md.research.elamur.ai/free/en/papers/marzollo-2026-sssd-simply-scalable-speculative-decoding/) [Annual Meeting of the Association for Computational Linguistics (ACL)](https://md.research.elamur.ai/free/en/venues/acl/)**Michele Marzollo, Jiawei Zhuang, Niklas Roemer, et al.*2026 · 3 citations*
37. **[Rectified Sparse Attention for Efficient Long-Sequence Generation](https://md.research.elamur.ai/free/en/papers/sun-2026-rectified-sparse-attention-efficient-long/) [Findings of the Association for Computational Linguistics](https://md.research.elamur.ai/free/en/venues/findings/)**Yutao Sun, Tianzhu Ye, Li Dong, et al.*2026 · 1 citation*
38. **[QUARTZ: Quantile-Aware Routing and Queueing for TTFT SLOs in LLM Serving](https://md.research.elamur.ai/free/en/papers/liu-2026-quartz-quantile-aware-routing-queueing/) [Findings of the Association for Computational Linguistics](https://md.research.elamur.ai/free/en/venues/findings/)**Zhipeng Liu, Yifan Zheng, Fanqi Kong, et al.*2026*

440 more papers of our library cite it; they have no page here yet.

- [← back to](https://md.research.elamur.ai/free/en/papers/yu-2022-orca-distributed-serving-system-transformer/) — Orca: A Distributed Serving System for Transformer-Based Generative Models