---
title: "Who cites Efficient Large Scale Language — 99 citing papers"
description: "99 papers of the Elamur Research library cite «Efficient Large Scale Language Modeling with Mixtures of Experts», found in their…"
canonical: "https://research.elamur.ai/free/en/papers/artetxe-2021-efficient-large-scale-language-modeling/cited-by/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/artetxe-2021-efficient-large-scale-language-modeling/cited-by/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [Efficient Large Scale Language Modeling with…](https://md.research.elamur.ai/free/en/papers/artetxe-2021-efficient-large-scale-language-modeling/) / cited by

# Who cites «Efficient Large Scale Language Modeling with Mixtures of Experts»

99 papers of our library cite it (of the 149,439 whose bibliographies we read) · 28 with a page here, the most cited first · [← back to «Efficient Large Scale Language Modeling with Mixtures of Experts»](https://md.research.elamur.ai/free/en/papers/artetxe-2021-efficient-large-scale-language-modeling/)

1. [**Emergent Abilities of Large Language Models arXiv:2206.07682**Jason Wei, Yi Tay, Rishi Bommasani, et al.*2022 · 2,269 citations*](https://md.research.elamur.ai/free/en/papers/wei-2022-emergent-abilities-large-language-models/)
2. **[Training Compute-Optimal Large Language Models](https://md.research.elamur.ai/free/en/papers/hoffmann-2022-training-compute-optimal-large-language/) [Conference on Neural Information Processing Systems (NeurIPS)](https://md.research.elamur.ai/free/en/venues/neural-information-processing-systems/)**Jordan Hoffmann, Sebastian Borgeaud, Arthur Mensch, et al.*2022 · 2,032 citations*
3. [**Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models arXiv:2206.04615 · Transactions on Machine Learning Research, May/2022, https://openreview.net/forum?id=uyTL5Bvosj**Aarohi Srivastava, Abhinav Rastogi, Abhishek Rao, et al.*2022 · 1,756 citations*](https://md.research.elamur.ai/free/en/papers/srivastava-2022-imitation-game-quantifying-extrapolating-capabilities/)
4. [**OPT: Open Pre-trained Transformer Language Models arXiv:2205.01068**Susan Zhang, Stephen Roller, Naman Goyal, et al.*2022 · 1,755 citations*](https://md.research.elamur.ai/free/en/papers/zhang-2022-opt-open-pre-trained-transformer/)
5. **[Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?](https://md.research.elamur.ai/free/en/papers/min-2022-rethinking-role-demonstrations-what-makes/) [Conference on Empirical Methods in Natural Language Processing (EMNLP)](https://md.research.elamur.ai/free/en/venues/emnlp/)**Sewon Min, Xinxi Lyu, Ari Holtzman, et al.*2022 · 1,444 citations*
6. **[GPT-NeoX-20B: An Open-Source Autoregressive Language Model](https://md.research.elamur.ai/free/en/papers/black-2022-gpt-neox-20b-open-source/) [BigScience Episode – Workshop on Challenges & Perspectives in Creating Large Language Models](https://md.research.elamur.ai/free/en/venues/bigscience/)**Sidney Black, Stella Biderman, Eric Hallahan, et al.*2022 · 831 citations*
7. [**GLM-130B: An Open Bilingual Pre-trained Model arXiv:2210.02414**Aohan Zeng, Xiao Liu, Zhengxiao Du, et al.*2022 · 786 citations*](https://md.research.elamur.ai/free/en/papers/zeng-2022-glm-130b-open-bilingual-pre/)
8. [**Orca: A Distributed Serving System for Transformer-Based Generative Models OSDI 2022**Gyeong-In Yu, Joo Seong Jeong, Geon-Woo Kim, et al.*2022 · 478 citations*](https://md.research.elamur.ai/free/en/papers/yu-2022-orca-distributed-serving-system-transformer/)
9. [**The Falcon Series of Open Language Models arXiv:2311.16867**Ebtesam Almazrouei, Hamza Alobeidli, Abdulaziz Alshamsi, et al.*2023 · 443 citations*](https://md.research.elamur.ai/free/en/papers/almazrouei-2023-falcon-series-open-language-models/)
10. [**Taming Throughput-Latency Tradeoff in LLM Inference with Sarathi-Serve OSDI 2024**Amey Agrawal, Nitin Kedia, Ashish Panwar, et al.*2024 · 369 citations*](https://md.research.elamur.ai/free/en/papers/agrawal-2024-taming-throughput-latency-tradeoff-llm/)
11. **[Large Language Models for Data Annotation and Synthesis: A Survey](https://md.research.elamur.ai/free/en/papers/tan-2024-large-language-models-data-annotation/) [Conference on Empirical Methods in Natural Language Processing (EMNLP)](https://md.research.elamur.ai/free/en/venues/emnlp/)**Zhen Tan, Dawei Li, Song Wang, et al.*2024 · 262 citations*
12. [**DeepSpeed-MoE: Advancing Mixture-of-Experts Inference and Training to Power Next-Generation AI Scale arXiv:2201.05596**Samyam Rajbhandari, Conglong Li, Zhewei Yao, et al.*2022 · 261 citations*](https://md.research.elamur.ai/free/en/papers/rajbhandari-2022-deepspeed-moe-advancing-mixture-experts/)
13. [**OPT-IML: Scaling Language Model Instruction Meta Learning through the Lens of Generalization arXiv:2212.12017**Srinivasan Iyer, Xi Victoria Lin, Ramakanth Pasunuru, et al.*2022 · 244 citations*](https://md.research.elamur.ai/free/en/papers/iyer-2022-opt-iml-scaling-language-model/)
14. **[Memorization Without Overfitting: Analyzing the Training Dynamics of Large Language Models](https://md.research.elamur.ai/free/en/papers/tirumala-2022-memorization-overfitting-analyzing-training-dynamics/) [Conference on Neural Information Processing Systems (NeurIPS)](https://md.research.elamur.ai/free/en/venues/neural-information-processing-systems/)**Kushal Tirumala, Aram H. Markosyan, Luke Zettlemoyer, et al.*2022 · 190 citations*
15. [**SARATHI: Efficient LLM Inference by Piggybacking Decodes with Chunked Prefills arXiv 2023**Amey Agrawal, Ashish Panwar, Jayashree Mohan, et al.*2023 · 144 citations*](https://md.research.elamur.ai/free/en/papers/agrawal-2023-sarathi-efficient-llm-inference-piggybacking/)
16. [**Evaluating ChatGPT's Information Extraction Capabilities: An Assessment of Performance, Explainability, Calibration, and Faithfulness arXiv:2304.11633**Bo Li, Gexiang Fang, Yang Yang, et al.*2023 · 142 citations*](https://md.research.elamur.ai/free/en/papers/li-2023-evaluating-chatgpt-s-information-extraction/)
17. [**Branch-Train-Merge: Embarrassingly Parallel Training of Expert Language Models arXiv:2208.03306**Margaret Li, Suchin Gururangan, Tim Dettmers, et al.*2022 · 114 citations*](https://md.research.elamur.ai/free/en/papers/li-2022-branch-train-merge-embarrassingly-parallel/)
18. [**Efficient Large Language Models: A Survey arXiv:2312.03863**Zhongwei Wan, Xin Wang, Che Liu, et al.*2023 · 110 citations*](https://md.research.elamur.ai/free/en/papers/wan-2023-efficient-large-language-models-survey/)
19. **[BLOOM+1: Adding Language Support to BLOOM for Zero-Shot Prompting](https://md.research.elamur.ai/free/en/papers/yong-2023-bloom-1-adding-language-support/) [Annual Meeting of the Association for Computational Linguistics (ACL)](https://md.research.elamur.ai/free/en/venues/acl/)**Zheng Xin Yong, Hailey Schoelkopf, Niklas Muennighoff, et al.*2023 · 102 citations*
20. [**Sparse Upcycling: Training Mixture-of-Experts from Dense Checkpoints arXiv:2212.05055**Aran Komatsuzaki, Joan Puigcerver, James Lee-Thorp, et al.*2022 · 98 citations*](https://md.research.elamur.ai/free/en/papers/komatsuzaki-2022-sparse-upcycling-training-mixture-experts/)
21. [**Accelerating Distributed MoE Training and Inference with Lina arXiv:2210.17223**Jiamin Li, Yimin Jiang, Yibo Zhu, et al.*2022 · 92 citations*](https://md.research.elamur.ai/free/en/papers/li-2022-accelerating-distributed-moe-training-inference/)
22. **[Cross-Lingual Consistency of Factual Knowledge in Multilingual Language Models](https://md.research.elamur.ai/free/en/papers/qi-2023-cross-lingual-consistency-factual-knowledge/) [Conference on Empirical Methods in Natural Language Processing (EMNLP)](https://md.research.elamur.ai/free/en/venues/emnlp/)**Jirui Qi, Raquel Fernández, Arianna Bisazza*2023 · 90 citations*
23. **[Task Contamination: Language Models May Not Be Few-Shot Anymore](https://md.research.elamur.ai/free/en/papers/li-2023-task-contamination-language-models-may/) [Proceedings of the AAAI Conference on Artificial Intelligence](https://md.research.elamur.ai/free/en/venues/proceedings-of-the-aaai-conference-on-artificial-intelligence/)**Changmao Li, Jeffrey Flanigan*2023 · 89 citations*
24. [**Small Language Models: Survey, Measurements, and Insights arXiv:2409.15790**Zhenyan Lu, Xiang Li, Dongqi Cai, et al.*2024 · 88 citations*](https://md.research.elamur.ai/free/en/papers/lu-2024-small-language-models-survey-measurements/)
25. **[D4: Improving LLM Pretraining via Document De-Duplication and Diversification](https://md.research.elamur.ai/free/en/papers/tirumala-2023-d4-improving-llm-pretraining-document/) [Conference on Neural Information Processing Systems (NeurIPS)](https://md.research.elamur.ai/free/en/venues/neural-information-processing-systems/)**Kushal Tirumala, Daniel Simig, Armen Aghajanyan, et al.*2023 · 81 citations*
26. [**Language models scale reliably with over-training and on downstream tasks arXiv:2403.08540**Samir Yitzhak Gadre, Georgios Smyrnis, Vaishaal Shankar, et al.*2024 · 46 citations*](https://md.research.elamur.ai/free/en/papers/gadre-2024-language-models-scale-reliably-over/)
27. [**PanGu-{\Sigma}: Towards Trillion Parameter Language Model with Sparse Heterogeneous Computing arXiv:2303.10845**Xiaozhe Ren, Pingyi Zhou, Xinfan Meng, et al.*2023 · 40 citations*](https://md.research.elamur.ai/free/en/papers/ren-2023-pangu-sigma-trillion-parameter/)
28. [**Skywork-MoE: A Deep Dive into Training Techniques for Mixture-of-Experts Language Models arXiv:2406.06563**Tianwen Wei, Bo Zhu, Liang Zhao, et al.*2024 · 21 citations*](https://md.research.elamur.ai/free/en/papers/wei-2024-skywork-moe-deep-dive/)

71 more papers of our library cite it; they have no page here yet.

- [← back to](https://md.research.elamur.ai/free/en/papers/artetxe-2021-efficient-large-scale-language-modeling/) — Efficient Large Scale Language Modeling with Mixtures of Experts