---
title: "Who cites BASE Layers: Simplifying — 116 citing papers"
description: "116 papers of the Elamur Research library cite «BASE Layers: Simplifying Training of Large, Sparse Models», found in their bibliographies…"
canonical: "https://research.elamur.ai/free/en/papers/lewis-2021-base-layers-simplifying-training-large/cited-by/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/lewis-2021-base-layers-simplifying-training-large/cited-by/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [BASE Layers: Simplifying Training of Large…](https://md.research.elamur.ai/free/en/papers/lewis-2021-base-layers-simplifying-training-large/) / cited by

# Who cites «BASE Layers: Simplifying Training of Large, Sparse Models»

116 papers of our library cite it (of the 149,439 whose bibliographies we read) · 27 with a page here, the most cited first · [← back to «BASE Layers: Simplifying Training of Large, Sparse Models»](https://md.research.elamur.ai/free/en/papers/lewis-2021-base-layers-simplifying-training-large/)

1. [**The Llama 3 Herd of Models arXiv:2407.21783**Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, et al.*2024 · 7,682 citations*](https://md.research.elamur.ai/free/en/papers/grattafiori-2024-llama-3-herd-models/)
2. **[Efficient Transformers: A Survey](https://md.research.elamur.ai/free/en/papers/tay-2020-efficient-transformers-survey/) [ACM Computing Surveys](https://md.research.elamur.ai/free/en/venues/acm-computing-surveys/)**Yi Tay, Mostafa Dehghani, Dara Bahri, et al.*2020 · 1,539 citations*
3. [**Challenges and Applications of Large Language Models arXiv:2307.10169**Jean Kaddour, Joshua Harris, Maximilian Mozes, et al.*2023 · 301 citations*](https://md.research.elamur.ai/free/en/papers/kaddour-2023-challenges-applications-large-language-models/)
4. [**BlenderBot 3: a deployed conversational agent that continually learns to responsibly engage arXiv:2208.03188**Kurt Shuster, Jing Xu, Mojtaba Komeili, et al.*2022 · 241 citations*](https://md.research.elamur.ai/free/en/papers/shuster-2022-blenderbot-3-deployed-conversational-agent/)
5. [**8-bit Optimizers via Block-wise Quantization arXiv:2110.02861**Tim Dettmers, Mike Lewis, Sam Shleifer, et al.*2021 · 181 citations*](https://md.research.elamur.ai/free/en/papers/dettmers-2021-8-bit-optimizers-block-wise/)
6. [**Tutel: Adaptive Mixture-of-Experts at Scale MLSys 2023**Changho Hwang, Wei Cui, Yifan Xiong, et al.*2022 · 168 citations*](https://md.research.elamur.ai/free/en/papers/hwang-2022-tutel-adaptive-mixture-experts-scale/)
7. **[AdaMix: Mixture-of-Adaptations for Parameter-efficient Model Tuning](https://md.research.elamur.ai/free/en/papers/wang-2022-adamix-mixture-adaptations-parameter-efficient/) [Conference on Empirical Methods in Natural Language Processing (EMNLP)](https://md.research.elamur.ai/free/en/venues/emnlp/)**Yaqing Wang, Sahaj Agarwal, Subhabrata Mukherjee, et al.*2022 · 135 citations*
8. **[Language Models that Seek for Knowledge: Modular Search & Generation for Dialogue and Prompt Completion](https://md.research.elamur.ai/free/en/papers/shuster-2022-language-models-that-seek-knowledge/) [Findings of the Association for Computational Linguistics](https://md.research.elamur.ai/free/en/venues/findings/)**Kurt Shuster, Mojtaba Komeili, Leonard Adolphs, et al.*2022 · 132 citations*
9. [**Scaling Vision with Sparse Mixture of Experts arXiv:2106.05974**Carlos Riquelme, Joan Puigcerver, Basil Mustafa, et al.*2021 · 128 citations*](https://md.research.elamur.ai/free/en/papers/riquelme-2021-scaling-vision-sparse-mixture-experts/)
10. **[MoEfication: Transformer Feed-forward Layers are Mixtures of Experts](https://md.research.elamur.ai/free/en/papers/zhang-2022-moefication-transformer-feed-forward-layers/) [Findings of the Association for Computational Linguistics](https://md.research.elamur.ai/free/en/venues/findings/)**Zhengyan Zhang, Yankai Lin, Zhiyuan Liu, et al.*2022 · 115 citations*
11. [**Branch-Train-Merge: Embarrassingly Parallel Training of Expert Language Models arXiv:2208.03306**Margaret Li, Suchin Gururangan, Tim Dettmers, et al.*2022 · 114 citations*](https://md.research.elamur.ai/free/en/papers/li-2022-branch-train-merge-embarrassingly-parallel/)
12. [**A Survey on Efficient Inference for Large Language Models arXiv:2404.14294**Zixuan Zhou, Xuefei Ning, Ke Hong, et al.*2024 · 113 citations*](https://md.research.elamur.ai/free/en/papers/zhou-2024-survey-efficient-inference-large-language/)
13. [**Efficient Large Scale Language Modeling with Mixtures of Experts arXiv:2112.10684**Mikel Artetxe, Shruti Bhosale, Naman Goyal, et al.*2021 · 113 citations*](https://md.research.elamur.ai/free/en/papers/artetxe-2021-efficient-large-scale-language-modeling/)
14. [**Efficient Large Language Models: A Survey arXiv:2312.03863**Zhongwei Wan, Xin Wang, Che Liu, et al.*2023 · 110 citations*](https://md.research.elamur.ai/free/en/papers/wan-2023-efficient-large-language-models-survey/)
15. **[CPM-2: Large-scale Cost-effective Pre-trained Language Models](https://md.research.elamur.ai/free/en/papers/zhang-2021-cpm-2-large-scale/) [AI Open](https://md.research.elamur.ai/free/en/venues/ai-open/)**Zhengyan Zhang, Yuxian Gu, Xu Han, et al.*2021 · 102 citations*
16. [**Sparse Upcycling: Training Mixture-of-Experts from Dense Checkpoints arXiv:2212.05055**Aran Komatsuzaki, Joan Puigcerver, James Lee-Thorp, et al.*2022 · 98 citations*](https://md.research.elamur.ai/free/en/papers/komatsuzaki-2022-sparse-upcycling-training-mixture-experts/)
17. [**Accelerating Distributed MoE Training and Inference with Lina arXiv:2210.17223**Jiamin Li, Yimin Jiang, Yibo Zhu, et al.*2022 · 92 citations*](https://md.research.elamur.ai/free/en/papers/li-2022-accelerating-distributed-moe-training-inference/)
18. **[LLaMA-MoE: Building Mixture-of-Experts from LLaMA with Continual Pre-Training](https://md.research.elamur.ai/free/en/papers/zhu-2024-llama-moe-building-mixture-experts/) [Conference on Empirical Methods in Natural Language Processing (EMNLP)](https://md.research.elamur.ai/free/en/venues/emnlp/)**Tong Zhu, Xiaoye Qu, Daize Dong, et al.*2024 · 85 citations*
19. [**Mixture-of-Experts Meets Instruction Tuning:A Winning Combination for Large Language Models arXiv:2305.14705**Sheng Shen, Le Hou, Yanqi Zhou, et al.*2023 · 70 citations*](https://md.research.elamur.ai/free/en/papers/shen-2023-mixture-experts-meets-instruction-tuning/)
20. [**Pushing Mixture of Experts to the Limit: Extremely Parameter Efficient MoE for Instruction Tuning arXiv:2309.05444**Ted Zadouri, Ahmet Ustun, Arash Ahmadian, et al.*2023 · 69 citations*](https://md.research.elamur.ai/free/en/papers/zadouri-2023-pushing-mixture-experts-limit-extremely/)
21. [**OpenMoE: An Early Effort on Open Mixture-of-Experts Language Models arXiv:2402.01739**Fuzhao Xue, Zian Zheng, Yao Fu, et al.*2024 · 64 citations*](https://md.research.elamur.ai/free/en/papers/xue-2024-openmoe-early-effort-open-mixture/)
22. [**ERNIE 3.0 Titan: Exploring Larger-scale Knowledge Enhanced Pre-training for Language Understanding and Generation arXiv:2112.12731**Shuohuan Wang, Yu Sun, Yang Xiang, et al.*2021 · 63 citations*](https://md.research.elamur.ai/free/en/papers/wang-2021-ernie-3-0-titan/)
23. [**Kimi K3: Open Frontier Intelligence arXiv:2607.24653**Kimi Team, Tongtong Bai, Yifan Bai, et al.*2026 · 40 citations*](https://md.research.elamur.ai/free/en/papers/kimi-team-2026-kimi-k3-open-frontier/)
24. [**Knowledge Card: Filling LLMs' Knowledge Gaps with Plug-in Specialized Language Models arXiv:2305.09955**Shangbin Feng, Weijia Shi, Yuyang Bai, et al.*2023 · 35 citations*](https://md.research.elamur.ai/free/en/papers/feng-2023-knowledge-card-filling-llms-knowledge/)
25. [**ConPET: Continual Parameter-Efficient Tuning for Large Language Models arXiv:2309.14763**Chenyang Song, Xu Han, Zheni Zeng, et al.*2023 · 14 citations*](https://md.research.elamur.ai/free/en/papers/song-2023-conpet-continual-parameter-efficient-tuning/)
26. [**PanGu-π: Enhancing Language Model Architectures via Nonlinearity Compensation arXiv:2312.17276**Yunhe Wang, Hanting Chen, Yehui Tang, et al.*2023 · 10 citations*](https://md.research.elamur.ai/free/en/papers/wang-2023-pangu-pi-enhancing-language/)
27. [**GRIN: GRadient-INformed MoE arXiv:2409.12136**Liyuan Liu, Young Jin Kim, Shuohang Wang, et al.*2024 · 7 citations*](https://md.research.elamur.ai/free/en/papers/liu-2024-grin-gradient-informed-moe/)

89 more papers of our library cite it; they have no page here yet.

- [← back to](https://md.research.elamur.ai/free/en/papers/lewis-2021-base-layers-simplifying-training-large/) — BASE Layers: Simplifying Training of Large, Sparse Models