---
title: "Small Language Models: Survey, Measurements, and Insights"
description: "The analysis shows how design choices have converged, which data strategies produce stronger… 108 references, cited by 69 papers here."
authors: ["Zhenyan Lu", "Xiang Li", "Dongqi Cai", "Rongjie Yi", "Fangming Liu", "Xiwen Zhang", "Nicholas D. Lane", "Mengwei Xu"]
year: 2024
venue: "arXiv:2409.15790"
source: "https://arxiv.org/abs/2409.15790"
doi: "10.48550/arxiv.2409.15790"
arxiv: "2409.15790"
licence: "CC BY 4.0 (arXiv)"
licence_url: "http://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/lu-2024-small-language-models-survey-measurements/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/lu-2024-small-language-models-survey-measurements/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2024](https://md.research.elamur.ai/free/en/papers/2024/page/4/) / Small Language Models: Survey, Measurements, and Insights

# Small Language Models: Survey, Measurements, and Insights

**Authors:** Zhenyan Lu, [Xiang Li](https://md.research.elamur.ai/free/en/authors/xiang-li/), Dongqi Cai, Rongjie Yi, Fangming Liu, Xiwen Zhang, Nicholas D. Lane and Mengwei Xu · **Year:** 2024 · **Venue:** arXiv:2409.15790 · **Licence:** [CC BY 4.0 (arXiv)](http://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://arxiv.org/abs/2409.15790) · [PDF](https://arxiv.org/pdf/2409.15790) · [DOI 10.48550/arxiv.2409.15790](https://doi.org/10.48550/arxiv.2409.15790) · [arXiv 2409.15790](https://arxiv.org/abs/2409.15790) · [OpenAlex](https://openalex.org/W4403786771)

**Citations:** 19 citations (OpenAlex, 2026-10-14) · [69](https://md.research.elamur.ai/free/en/papers/lu-2024-small-language-models-survey-measurements/cited-by/) in our library, of the 141,840 papers whose bibliographies we read

## Summary
Compact language models already run in commercial devices, such as Gemini Nano on Google/Samsung phones and Apple's on-device foundation model. Academic research on them still trails far behind work on cloud-scale LLMs. This paper addresses that gap with a combined survey and measurement study of open-weight, decoder-only transformer SLMs between 100M and 5B parameters. The authors catalogue models released from 2022 to late 2024, including OPT, Pythia, Phi, Qwen, Gemma, MiniCPM, SmolLM and Llama-3.2. They compare these models along three axes: architecture, pre-training data and training recipes. Next, they assess how well the models reason about everyday knowledge, solve mathematical problems, learn from examples supplied in the prompt and locate information within lengthy inputs, and they also profile on-device costs (prefill and decode latency, memory footprint, energy). The analysis shows how design choices have converged, which data strategies produce stronger small models, and how model shape, quantization and hardware affect speed. The study serves as a practical reference for choosing or building models for constrained hardware, and the authors release all results and benchmarking tools publicly.

## Key points
- **Selection criteria:** decoder-only transformers with open weights in the 100M–5B range. Only base models are included, except Phi, StableLM and Dolly-v2, which are available only as instruct versions. RWKV and Mamba are excluded. The 5B cap reflects that 7B models were still mostly cloud-deployed in 2024.
- **Model count inconsistency:** the abstract cites 70 surveyed SLMs, while the introduction mentions 57 fully benchmarked.
- **Architecture trends (2022–2024):** GQA is displacing MHA, and gated FFNs are replacing standard FFNs. MLA, which shrinks the KV cache, appears in MiniCPM3.
- **Training data and scaling:** DCLM and FineWeb-Edu use model-based quality filtering, which largely closes the gap between SLMs trained on open and closed data. Small models are commonly trained on far more tokens than the Chinchilla law suggests.
- **Proposed research directions:**
  - co-designing architectures with hardware such as NPUs;
  - a deployment-aware scaling law;
  - on-device personalization, e.g. zeroth-order finetuning;
  - device–cloud SLM–LLM collaboration;
  - fairer benchmarks, given overfitting to GSM8k;
  - sparse SLMs that keep weights in flash storage.
- **Code and results:** github.com/UbiquitousLearning/SLM_Survey.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [University of Cambridge](https://md.research.elamur.ai/free/en/universities/university-of-cambridge/) — University, Cambridge, United Kingdom · 155 works · 115,562 citations
- [Beijing University of Posts and Telecommunications](https://md.research.elamur.ai/free/en/universities/beijing-university-of-posts-and-telecommunications/) — University, Beijing, China · 22 works · 32,364 citations

📚 Shelves (5): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 9,086 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 8,889 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 9,296 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 9,206 works · [👁️ Computer Vision and Pattern Recognition](https://md.research.elamur.ai/free/en/shelves/computer-science/computer-vision-and-pattern-recognition/) · 1,282 works

🧭 Topics (1): [🧠 Language models & pretraining](https://md.research.elamur.ai/free/en/topics/language-models/) · 1,343 works

## Cite

Cite as: Zhenyan Lu, Xiang Li, Dongqi Cai, Rongjie Yi, Fangming Liu, Xiwen Zhang, Nicholas D. Lane and Mengwei Xu. 2024. Small Language Models: Survey, Measurements, and Insights. arXiv:2409.15790.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@misc{lu2024small,
  title = {{Small Language Models: Survey, Measurements, and Insights}},
  author = {Zhenyan Lu, Xiang Li, Dongqi Cai, Rongjie Yi, Fangming Liu, Xiwen Zhang, Nicholas D. Lane and Mengwei Xu},
  year = {2024},
  doi = {10.48550/arxiv.2409.15790},
  eprint = {2409.15790},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2409.15790}
}
```

- [← previous Not All Experts are Equal: Efficient Expert Pruning and Skipping for Mixture-of-Experts Large Language Models](https://md.research.elamur.ai/free/en/papers/lu-2024-not-all-experts-are-equal/)
- [next → TextGrad: Automatic "Differentiation" via Text](https://md.research.elamur.ai/free/en/papers/yuksekgonul-2024-textgrad-automatic-differentiation-text/)
- [↑ back to papers of 2024](https://md.research.elamur.ai/free/en/papers/2024/page/4/)

## References

108 references, 19 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Google ai edge sdk for gemini nano. [https://developer.android.com/ai/aicore](https://developer.android.com/ai/aicore), 2024. [↗](https://developer.android.com/ai/aicore)
2. Introducing apple’s on-device and server foundation models. [https://machinelearning.apple.com/](https://machinelearning.apple.com/) research/introducing-apple-foundation-models, 2024. [↗](https://machinelearning.apple.com/)
3. Alibaba. Qwen 1. [https://huggingface.co/Qwen/Qwen-1_8B](https://huggingface.co/Qwen/Qwen-1_8B), 2023.11. [↗](https://huggingface.co/Qwen/Qwen-1_8B)
4. Alibaba. Qwen 1.5. [https://huggingface.co/Qwen/Qwen1.5-0.5B](https://huggingface.co/Qwen/Qwen1.5-0.5B), 2024.02. [↗](https://huggingface.co/Qwen/Qwen1.5-0.5B)
5. Alibaba. Qwen 2. [https://huggingface.co/Qwen/Qwen2-0.5B](https://huggingface.co/Qwen/Qwen2-0.5B), 2024.02. [↗](https://huggingface.co/Qwen/Qwen2-0.5B)
6. Alibaba. Qwen 2.5. [https://qwenlm.github.io/blog/qwen2.5/](https://qwenlm.github.io/blog/qwen2.5/), 2024.09. [↗](https://qwenlm.github.io/blog/qwen2.5/)
7. Keivan Alizadeh, Iman Mirzadeh, Dmitry Belenko, Karen Khatamifard, Minsik Cho, Carlo C Del Mundo, Mohammad Rastegari, and Mehrdad Farajtabar. [Llm in a flash: Efficient large language model inference with limited memory](https://md.research.elamur.ai/free/en/papers/alizadeh-2024-llm-flash-efficient-large-language/). arXiv preprint arXiv:2312.11514, 2023.
8. AllenAI. allenai/olmo-1b-hf. [https://huggingface.co/allenai/OLMo-1B-hf](https://huggingface.co/allenai/OLMo-1B-hf), 2024.04. [↗](https://huggingface.co/allenai/OLMo-1B-hf)
9. AMD. Llama. [https://huggingface.co/amd/AMD-Llama-135m](https://huggingface.co/amd/AMD-Llama-135m), 2024.08. [↗](https://huggingface.co/amd/AMD-Llama-135m)
10. Apple. Openelm. [https://huggingface.co/apple/OpenELM-270M](https://huggingface.co/apple/OpenELM-270M), 2024.04. [↗](https://huggingface.co/apple/OpenELM-270M)
11. Mikel Artetxe, Shruti Bhosale, Naman Goyal, Todor Mihaylov, Myle Ott, Sam Shleifer, Xi Victoria Lin, Jingfei Du, Srinivasan Iyer, Ramakanth Pasunuru, et al. Efficient large scale language modeling with mixtures of experts. arXiv preprint arXiv:2112.10684, 2021. [find](https://scholar.google.com/scholar?q=Mikel+Artetxe%2C+Shruti+Bhosale%2C+Naman+Goyal%2C+Todor+Mihaylov%2C+Myle+Ott%2C+Sam+Shleifer%2C+Xi+Victoria+Lin%2C+Jingfei+Du%2C+Srinivasan+Iyer%2C+Ramakanth+Pasunuru%2C+et+al%2E+Efficient+large+scale+language+modeling+wit)
12. Jason Baumgartner, Savvas Zannettou, Brian Keegan, Megan Squire, and Jeremy Blackburn. The pushshift reddit dataset. In Proceedings of the international AAAI conference on web and social media, volume 14, pages 830–839, 2020. [find](https://scholar.google.com/scholar?q=Jason+Baumgartner%2C+Savvas+Zannettou%2C+Brian+Keegan%2C+Megan+Squire%2C+and+Jeremy+Blackburn%2E+The+pushshift+reddit+dataset%2E+In+Proceedings+of+the+international+AAAI+conference+on+web+and+social+media%2C+volume)
13. Loubna Ben Allal, Anton Lozhkov, Guilherme Penedo, Thomas Wolf, and Leandro von Werra. Cosmopedia, 2024. [find](https://scholar.google.com/scholar?q=Loubna+Ben+Allal%2C+Anton+Lozhkov%2C+Guilherme+Penedo%2C+Thomas+Wolf%2C+and+Leandro+von+Werra%2E+Cosmopedia%2C+2024%2E)
14. Milan Bhan, Jean-Noel Vittaut, Nicolas Chesneau, and Marie-Jeanne Lesot. Self-amplify: Improving small language models with self post hoc explanations. arXiv preprint arXiv:2402.12038, 2024. [find](https://scholar.google.com/scholar?q=Milan+Bhan%2C+Jean%2DNoel+Vittaut%2C+Nicolas+Chesneau%2C+and+Marie%2DJeanne+Lesot%2E+Self%2Damplify%3A+Improving+small+language+models+with+self+post+hoc+explanations%2E+arXiv+preprint+arXiv%3A2402%2E12038%2C+2024%2E)
15. BigScience. bigscience/bloom-560m. [https://huggingface.co/bigscience/bloom-560m](https://huggingface.co/bigscience/bloom-560m), 2022.11. [↗](https://huggingface.co/bigscience/bloom-560m)
16. BigScience. bigscience/bloomz-1b1. [https://huggingface.co/bigscience/bloomz-1b1](https://huggingface.co/bigscience/bloomz-1b1), 2022.11. [↗](https://huggingface.co/bigscience/bloomz-1b1)
17. Yonatan Bisk, Rowan Zellers, Ronan Le Bras, Jianfeng Gao, and Yejin Choi. [Piqa: Reasoning about physical commonsense in natural language](https://md.research.elamur.ai/free/en/papers/bisk-2019-piqa-reasoning-about-physical-commonsense/). In Proceedings of the AAAI Conference on Artificial Intelligence, 2020.
18. Cerebras. 2023.03. cerebras/cerebras-gpt-111m. [https://huggingface.co/cerebras/Cerebras-GPT-111M](https://huggingface.co/cerebras/Cerebras-GPT-111M), [↗](https://huggingface.co/cerebras/Cerebras-GPT-111M)
19. Lihu Chen and Gaël Varoquaux. What is the role of small models in the llm era: A survey. arXiv preprint arXiv:2409.06857, 2024. [find](https://scholar.google.com/scholar?q=Lihu+Chen+and+Gae%CC%88l+Varoquaux%2E+What+is+the+role+of+small+models+in+the+llm+era%3A+A+survey%2E+arXiv+preprint+arXiv%3A2409%2E06857%2C+2024%2E)
20. Christopher Clark, Kenton Lee, Ming-Wei Chang, Tom Kwiatkowski, Michael Collins, and Kristina Toutanova. [Boolq: Exploring the surprising difficulty of natural yes/no questions](https://md.research.elamur.ai/free/en/papers/clark-2019-boolq-exploring-surprising-difficulty-natural/). In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human…
21. Peter Clark, Isaac Cowhey, Oren Etzioni, Tushar Khot, Ashish Sabharwal, Carissa Schoenick, and Oyvind Tafjord. Think you have solved question answering? try arc, the ai2 reasoning challenge. In arXiv preprint arXiv:1803.05457, 2018. [find](https://scholar.google.com/scholar?q=Peter+Clark%2C+Isaac+Cowhey%2C+Oren+Etzioni%2C+Tushar+Khot%2C+Ashish+Sabharwal%2C+Carissa+Schoenick%2C+and+Oyvind+Tafjord%2E+Think+you+have+solved+question+answering%3F+try+arc%2C+the+ai2+reasoning+challenge%2E+In+arXiv+)
22. Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman. [Training verifiers to solve math word problems](https://md.research.elamur.ai/free/en/papers/cobbe-2021-training-verifiers-solve-math-word/). arXiv preprint arXiv:2110.14168, 2021.
23. Together Computer. [Redpajama: an open dataset for training large language models](https://md.research.elamur.ai/free/en/papers/weber-2024-redpajama-open-dataset-training-large/), 2023.
24. DataBricks. databricks/dolly-v2-3b. [https://huggingface.co/databricks/dolly-v2-3b](https://huggingface.co/databricks/dolly-v2-3b), 2023.04. [↗](https://huggingface.co/databricks/dolly-v2-3b)
25. Mateusz Dubiel, Yasmine Barghouti, Kristina Kudryavtseva, and Luis A Leiva. On-device query intent prediction with lightweight llms to support ubiquitous conversations. Scientific Reports, 14(1):12731, 2024. [find](https://scholar.google.com/scholar?q=Mateusz+Dubiel%2C+Yasmine+Barghouti%2C+Kristina+Kudryavtseva%2C+and+Luis+A+Leiva%2E+On%2Ddevice+query+intent+prediction+with+lightweight+llms+to+support+ubiquitous+conversations%2E+Scientific+Reports%2C+14%281%29%3A12731)
26. Ronen Eldan and Yuanzhi Li. [Tinystories: How small can language models be and still speak coherent english?](https://md.research.elamur.ai/free/en/papers/eldan-2023-tinystories-small-language-models-still/) arXiv preprint arXiv:2305.07759, 2023.
27. EleutherAI. Eleutherai/pythia-410m. [https://huggingface.co/EleutherAI/pythia-410m](https://huggingface.co/EleutherAI/pythia-410m), 2023.03. [↗](https://huggingface.co/EleutherAI/pythia-410m)
28. Facebook. facebook/opt-125m. [https://huggingface.co/facebook/opt-125m](https://huggingface.co/facebook/opt-125m), 2022.05. [↗](https://huggingface.co/facebook/opt-125m)
29. Facebook. facebook/galactica-125m. [https://huggingface.co/facebook/galactica-125m](https://huggingface.co/facebook/galactica-125m), 2022.11. [↗](https://huggingface.co/facebook/galactica-125m)
30. Leo Gao, Stella Biderman, Sid Black, Laurence Golding, Travis Hoppe, Charles Foster, Jason Phang, Horace He, Anish Thite, Noa Nabeshima, et al. The pile: An 800gb dataset of diverse text for language modeling. arXiv preprint arXiv:2101.00027, 2020. [find](https://scholar.google.com/scholar?q=Leo+Gao%2C+Stella+Biderman%2C+Sid+Black%2C+Laurence+Golding%2C+Travis+Hoppe%2C+Charles+Foster%2C+Jason+Phang%2C+Horace+He%2C+Anish+Thite%2C+Noa+Nabeshima%2C+et+al%2E+The+pile%3A+An+800gb+dataset+of+diverse+text+for+language+)
31. Nathan Godey, Éric de la Clergerie, and Benoı̂t Sagot. Why do small language models underperform? studying language model saturation via the softmax bottleneck. arXiv preprint arXiv:2404.07647, 2024. [find](https://scholar.google.com/scholar?q=Nathan+Godey%2C+E%CC%81ric+de+la+Clergerie%2C+and+Beno%C4%B1%CC%82t+Sagot%2E+Why+do+small+language+models+underperform%3F+studying+language+model+saturation+via+the+softmax+bottleneck%2E+arXiv+preprint+arXiv%3A2404%2E07647%2C+2024%2E)
32. Google. Gemma. [https://huggingface.co/google/Gemma](https://huggingface.co/google/Gemma), 2024.02. [↗](https://huggingface.co/google/Gemma)
33. Google. recurrentgemma. [https://huggingface.co/google/recurrentGemma](https://huggingface.co/google/recurrentGemma), 2024.04. [↗](https://huggingface.co/google/recurrentGemma)
34. Albert Gu and Tri Dao. Mamba: Linear-time sequence modeling with selective state spaces. arXiv preprint arXiv:2312.00752, 2023. [find](https://scholar.google.com/scholar?q=Albert+Gu+and+Tri+Dao%2E+Mamba%3A+Linear%2Dtime+sequence+modeling+with+selective+state+spaces%2E+arXiv+preprint+arXiv%3A2312%2E00752%2C+2023%2E)
35. H2O.ai. h2o-danube3-4b-base. [https://huggingface.co/h2oai/h2o-danube3-4b-base](https://huggingface.co/h2oai/h2o-danube3-4b-base), 2024. [↗](https://huggingface.co/h2oai/h2o-danube3-4b-base)
36. Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. [Measuring massive multitask language understanding](https://md.research.elamur.ai/free/en/papers/hendrycks-2020-measuring-massive-multitask-language-understanding/). arXiv preprint arXiv:2009.03300, 2021.
37. Jordan Hoffmann, Sebastian Borgeaud, Arthur Mensch, Elena Buchatskaya, Trevor Cai, Eliza Rutherford, Diego de Las Casas, Lisa Anne Hendricks, Johannes Welbl, Aidan Clark, et al. [Training compute-optimal large language models](https://md.research.elamur.ai/free/en/papers/hoffmann-2022-training-compute-optimal-large-language/). arXiv preprint arXiv:2203.15556, 2022.
38. HuggingFace. Smollm. [https://huggingface.co/HuggingFaceTB/SmolLM-360M](https://huggingface.co/HuggingFaceTB/SmolLM-360M), 2024.07. [↗](https://huggingface.co/HuggingFaceTB/SmolLM-360M)
39. HuggingFace. Smollm2. [https://huggingface.co/HuggingFaceTB/SmolLM2-360M](https://huggingface.co/HuggingFaceTB/SmolLM2-360M), 2024.11. [↗](https://huggingface.co/HuggingFaceTB/SmolLM2-360M)
40. Jared Kaplan, Sam McCandlish, Tom Henighan, Tom B Brown, Benjamin Chess, Rewon Child, Scott Gray, Alec Radford, Jeffrey Wu, and Dario Amodei. [Scaling laws for neural language models](https://md.research.elamur.ai/free/en/papers/kaplan-2020-scaling-laws-neural-language-models/). arXiv preprint arXiv:2001.08361, 2020.
41. Jongwoo Ko, Sungnyun Kim, Tianyi Chen, and Se-Young Yun. [Distillm: Towards streamlined distillation for large language models](https://md.research.elamur.ai/free/en/papers/ko-2024-distillm-towards-streamlined-distillation-large/). arXiv preprint arXiv:2402.03898, 2024.
42. Jakub Krajewski, Jan Ludziejewski, Kamil Adamczewski, Maciej Pióro, Michał Krutul, Szymon Antoniak, Kamil Ciebiera, Krystian Król, Tomasz Odrzygóźdź, Piotr Sankowski, et al. Scaling laws for fine-grained mixture of experts. arXiv preprint arXiv:2402.07871, 2024. — [arXiv:2402.07871](https://arxiv.org/abs/2402.07871) [find](https://scholar.google.com/scholar?q=Jakub+Krajewski%2C+Jan+Ludziejewski%2C+Kamil+Adamczewski%2C+Maciej+Pio%CC%81ro%2C+Micha%C5%82+Krutul%2C+Szymon+Antoniak%2C+Kamil+Ciebiera%2C+Krystian+Kro%CC%81l%2C+Tomasz+Odrzygo%CC%81z%CC%81dz%CC%81%2C+Piotr+Sankowski%2C+et+al%2E+Scaling+laws+for+fine)
43. Stefanos Laskaridis, Kleomenis Katevas, Lorenzo Minto, and Hamed Haddadi. Mobile and edge evaluation of large language models. In Workshop on Efficient Systems for Foundation Models II@ ICML2024. [find](https://scholar.google.com/scholar?q=Stefanos+Laskaridis%2C+Kleomenis+Katevas%2C+Lorenzo+Minto%2C+and+Hamed+Haddadi%2E+Mobile+and+edge+evaluation+of+large+language+models%2E+In+Workshop+on+Efficient+Systems+for+Foundation+Models+II%40+ICML2024%2E)
44. Pierre Lepagnol, Thomas Gerald, Sahar Ghannay, Christophe Servan, and Sophie Rosset. Small language models are good too: An empirical study of zero-shot classification. arXiv preprint arXiv:2404.11122, 2024. [find](https://scholar.google.com/scholar?q=Pierre+Lepagnol%2C+Thomas+Gerald%2C+Sahar+Ghannay%2C+Christophe+Servan%2C+and+Sophie+Rosset%2E+Small+language+models+are+good+too%3A+An+empirical+study+of+zero%2Dshot+classification%2E+arXiv+preprint+arXiv%3A2404%2E11122)
45. Aitor Lewkowycz, Anders Andreassen, David Dohan, Ethan Dyer, Henryk Michalewski, Vinay Ramasesh, Ambrose Slone, Cem Anil, Imanol Schlag, Theo Gutman-Solo, et al. Minerva: [Solving quantitative reasoning problems with language models](https://md.research.elamur.ai/free/en/papers/lewkowycz-2022-solving-quantitative-reasoning-problems-language/). arXiv preprint arXiv:2206.14858, 2023.
46. Beibin Li, Yi Zhang, Sébastien Bubeck, Jeevan Pathuri, and Ishai Menache. Small language models for application interactions: A case study. arXiv preprint arXiv:2405.20347, 2024. [find](https://scholar.google.com/scholar?q=Beibin+Li%2C+Yi+Zhang%2C+Se%CC%81bastien+Bubeck%2C+Jeevan+Pathuri%2C+and+Ishai+Menache%2E+Small+language+models+for+application+interactions%3A+A+case+study%2E+arXiv+preprint+arXiv%3A2405%2E20347%2C+2024%2E)
47. [Jeffrey Li, Alex Fang, Georgios Smyrnis, Maor Ivgi, Matt Jordan, Samir Gadre, Hritik Bansal, Etash Guha, Sedrick Keh, Kushal Arora, Saurabh Garg, Rui Xin, Niklas Muennighoff, Reinhard Heckel, Jean Mercat, Mayee Chen, Suchin Gururangan, Mitchell Wortsman, Alon Albalak, Yonatan Bitton, Marianna Nezhur…](https://md.research.elamur.ai/free/en/papers/li-2024-datacomp-lm-search-next-generation/)
48. Luchang Li, Sheng Qian, Jie Lu, Lunxi Yuan, Rui Wang, and Qin Xie. Transformer-lite: High-efficiency deployment of large language models on mobile phone gpus. arXiv preprint arXiv:2403.20041, 2024. [find](https://scholar.google.com/scholar?q=Luchang+Li%2C+Sheng+Qian%2C+Jie+Lu%2C+Lunxi+Yuan%2C+Rui+Wang%2C+and+Qin+Xie%2E+Transformer%2Dlite%3A+High%2Defficiency+deployment+of+large+language+models+on+mobile+phone+gpus%2E+arXiv+preprint+arXiv%3A2403%2E20041%2C+2024%2E)
49. Raymond Li, Loubna Ben Allal, Yangtian Zi, Niklas Muennighoff, Denis Kocetkov, Chenghao Mou, Marc Marone, Christopher Akiki, Jia Li, Jenny Chim, Qian Liu, Evgenii Zheltonozhskii, Terry Yue Zhuo, Thomas Wang, Olivier Dehaene, Mishig Davaadorj, Joel Lamy-Poirier, João Monteiro, Oleh Shliazhko, Nicola… [find](https://scholar.google.com/scholar?q=Raymond+Li%2C+Loubna+Ben+Allal%2C+Yangtian+Zi%2C+Niklas+Muennighoff%2C+Denis+Kocetkov%2C+Chenghao+Mou%2C+Marc+Marone%2C+Christopher+Akiki%2C+Jia+Li%2C+Jenny+Chim%2C+Qian+Liu%2C+Evgenii+Zheltonozhskii%2C+Terry+Yue+Zhuo%2C+Thoma)
50. Xiang Li, Zhenyan Lu, Dongqi Cai, Xiao Ma, and Mengwei Xu. Large language models on mobile devices: Measurements, analysis, and insights. In Proceedings of the Workshop on Edge and Mobile Foundation Models, pages 1–6, 2024. [find](https://scholar.google.com/scholar?q=Xiang+Li%2C+Zhenyan+Lu%2C+Dongqi+Cai%2C+Xiao+Ma%2C+and+Mengwei+Xu%2E+Large+language+models+on+mobile+devices%3A+Measurements%2C+analysis%2C+and+insights%2E+In+Proceedings+of+the+Workshop+on+Edge+and+Mobile+Foundation+M)
51. Ji Lin, Jiaming Tang, Haotian Tang, Shang Yang, Wei-Ming Chen, Wei-Chen Wang, Guangxuan Xiao, Xingyu Dang, Chuang Gan, and Song Han. Awq: Activation-aware weight quantization for on-device llm compression and acceleration. Proceedings of Machine Learning and Systems, 6:87–100, 2024. [find](https://scholar.google.com/scholar?q=Ji+Lin%2C+Jiaming+Tang%2C+Haotian+Tang%2C+Shang+Yang%2C+Wei%2DMing+Chen%2C+Wei%2DChen+Wang%2C+Guangxuan+Xiao%2C+Xingyu+Dang%2C+Chuang+Gan%2C+and+Song+Han%2E+Awq%3A+Activation%2Daware+weight+quantization+for+on%2Ddevice+llm+compres)
52. Stephanie Lin, Jacob Hilton, and Owain Evans. [Truthfulqa: Measuring how models mimic human falsehoods](https://md.research.elamur.ai/free/en/papers/lin-2022-truthfulqa-measuring-how-models-mimic/). arXiv preprint arXiv:2109.07958, 2022.
53. Yinhan Liu. Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692, 2019. [find](https://scholar.google.com/scholar?q=Yinhan+Liu%2E+Roberta%3A+A+robustly+optimized+bert+pretraining+approach%2E+arXiv+preprint+arXiv%3A1907%2E11692%2C+2019%2E)
54. Lucie Charlotte Magister, Jonathan Mallinson, Jakub Adamek, Eric Malmi, and Aliaksei Severyn. [Teaching small language models to reason](https://md.research.elamur.ai/free/en/papers/magister-2023-teaching-small-language-models-reason/). arXiv preprint arXiv:2212.08410, 2022.
55. M.A.P. Ct-llm. [https://huggingface.co/m-a-p/CT-LLM-Base](https://huggingface.co/m-a-p/CT-LLM-Base), 2024.04. [↗](https://huggingface.co/m-a-p/CT-LLM-Base)
56. MBZUAI. Mbzuai/lamini-gpt-774m. [https://huggingface.co/MBZUAI/LaMini-GPT-774M](https://huggingface.co/MBZUAI/LaMini-GPT-774M), 2023.04. [↗](https://huggingface.co/MBZUAI/LaMini-GPT-774M)
57. MBZUAI. Mobillama. [https://huggingface.co/mbzuai/MobiLlama](https://huggingface.co/mbzuai/MobiLlama), 2024.02. [↗](https://huggingface.co/mbzuai/MobiLlama)
58. Meituan. Mobilellama. [https://huggingface.co/meituan/MobileLLaMA](https://huggingface.co/meituan/MobileLLaMA), 2023.12. [↗](https://huggingface.co/meituan/MobileLLaMA)
59. Meta. Llama-3.2. [https://huggingface.co/meta-llama/Llama-3.2-3B](https://huggingface.co/meta-llama/Llama-3.2-3B), 2024.09. [↗](https://huggingface.co/meta-llama/Llama-3.2-3B)
60. Microsoft. microsoft/phi-1. [https://huggingface.co/microsoft/phi-1](https://huggingface.co/microsoft/phi-1), 2023.09. [↗](https://huggingface.co/microsoft/phi-1)
61. Microsoft. microsoft/phi-1 5. [https://huggingface.co/microsoft/phi-1_5](https://huggingface.co/microsoft/phi-1_5), 2023.09. [↗](https://huggingface.co/microsoft/phi-1_5)
62. Microsoft. microsoft/phi-2. [https://huggingface.co/microsoft/phi-2](https://huggingface.co/microsoft/phi-2), 2023.12. [↗](https://huggingface.co/microsoft/phi-2)
63. Microsoft. microsoft/phi-3-mini. [https://huggingface.co/microsoft/phi-3-mini](https://huggingface.co/microsoft/phi-3-mini), 2024.04. [↗](https://huggingface.co/microsoft/phi-3-mini)
64. Todor Mihaylov, Peter Clark, Tushar Khot, and Ashish Sabharwal. Can a suit of armor conduct electricity? a new dataset for open book question answering. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, 2018. [find](https://scholar.google.com/scholar?q=Todor+Mihaylov%2C+Peter+Clark%2C+Tushar+Khot%2C+and+Ashish+Sabharwal%2E+Can+a+suit+of+armor+conduct+electricity%3F+a+new+dataset+for+open+book+question+answering%2E+In+Proceedings+of+the+2018+Conference+on+Empiri)
65. Thuat Nguyen, Chien Van Nguyen, Viet Dac Lai, Hieu Man, Nghia Trung Ngo, Franck Dernoncourt, Ryan A Rossi, and Thien Huu Nguyen. [Culturax: A cleaned, enormous, and multilingual dataset for large language models in 167 languages](https://md.research.elamur.ai/free/en/papers/nguyen-2023-culturax-cleaned-enormous-multilingual-dataset/). arXiv preprint arXiv:2309.09400, 2023.
66. Nvidia. Minitron. [https://huggingface.co/nvidia/Minitron-4B-Base](https://huggingface.co/nvidia/Minitron-4B-Base), 2024.07. [↗](https://huggingface.co/nvidia/Minitron-4B-Base)
67. OpenBMB. Minicpm. [https://huggingface.co/openbmb/MiniCPM-1B-sft-bf16](https://huggingface.co/openbmb/MiniCPM-1B-sft-bf16), 2024.04. [↗](https://huggingface.co/openbmb/MiniCPM-1B-sft-bf16)
68. OpenBMB. Minicpm3. [https://huggingface.co/openbmb/MiniCPM3-4B](https://huggingface.co/openbmb/MiniCPM3-4B), 2024.09. [↗](https://huggingface.co/openbmb/MiniCPM3-4B)
69. Guilherme Penedo, Hynek Kydlı́ček, Anton Lozhkov, Margaret Mitchell, Colin Raffel, Leandro Von Werra, Thomas Wolf, et al. [The fineweb datasets: Decanting the web for the finest text data at scale](https://md.research.elamur.ai/free/en/papers/penedo-2024-fineweb-datasets-decanting-web-finest/). arXiv preprint arXiv:2406.17557, 2024.
70. Guilherme Penedo, Quentin Malartic, Daniel Hesslow, Ruxandra Cojocaru, Alessandro Cappelli, Hamza Alobeidli, Baptiste Pannier, Ebtesam Almazrouei, and Julien Launay. The refinedweb dataset for falcon llm: outperforming curated corpora with web data, and web data only. arXiv preprint arXiv:2306.01116… [find](https://scholar.google.com/scholar?q=Guilherme+Penedo%2C+Quentin+Malartic%2C+Daniel+Hesslow%2C+Ruxandra+Cojocaru%2C+Alessandro+Cappelli%2C+Hamza+Alobeidli%2C+Baptiste+Pannier%2C+Ebtesam+Almazrouei%2C+and+Julien+Launay%2E+The+refinedweb+dataset+for+falcon+)
71. Bo Peng, Eric Alcaide, Quentin Anthony, Alon Albalak, Samuel Arcadinho, Stella Biderman, Huanqi Cao, Xin Cheng, Michael Chung, Matteo Grella, et al. Rwkv: Reinventing rnns for the transformer era. arXiv preprint arXiv:2305.13048, 2023. [find](https://scholar.google.com/scholar?q=Bo+Peng%2C+Eric+Alcaide%2C+Quentin+Anthony%2C+Alon+Albalak%2C+Samuel+Arcadinho%2C+Stella+Biderman%2C+Huanqi+Cao%2C+Xin+Cheng%2C+Michael+Chung%2C+Matteo+Grella%2C+et+al%2E+Rwkv%3A+Reinventing+rnns+for+the+transformer+era%2E+arX)
72. Princeton. Sheared-llama. [https://huggingface.co/princeton-nlp/Sheared-LLaMA-1.3B](https://huggingface.co/princeton-nlp/Sheared-LLaMA-1.3B), 2023.11. [↗](https://huggingface.co/princeton-nlp/Sheared-LLaMA-1.3B)
73. Keisuke Sakaguchi, Ronan Le Bras, Chandra Bhagavatula, and Yejin Choi. Winogrande: An adversarial winograd schema challenge at scale. In Proceedings of the AAAI Conference on Artificial Intelligence, 2020. [find](https://scholar.google.com/scholar?q=Keisuke+Sakaguchi%2C+Ronan+Le+Bras%2C+Chandra+Bhagavatula%2C+and+Yejin+Choi%2E+Winogrande%3A+An+adversarial+winograd+schema+challenge+at+scale%2E+In+Proceedings+of+the+AAAI+Conference+on+Artificial+Intelligence%2C+)
74. Timo Schick and Hinrich Schütze. It’s not just size that matters: Small language models are also few-shot learners. arXiv preprint arXiv:2009.[07118, 2020](https://md.research.elamur.ai/free/en/papers/schick-2021-it-s-not-just-size/).
75. Andrii Skliar, Ties van Rozendaal, Romain Lepert, Todor Boinovski, Mart van Baalen, Markus Nagel, Paul Whatmough, and Babak Ehteshami Bejnordi. Mixture of cache-conditional experts for efficient mobile device inference. arXiv preprint arXiv:2412.00099, 2024. — [arXiv:2412.00099](https://arxiv.org/abs/2412.00099) [find](https://scholar.google.com/scholar?q=Andrii+Skliar%2C+Ties+van+Rozendaal%2C+Romain+Lepert%2C+Todor+Boinovski%2C+Mart+van+Baalen%2C+Markus+Nagel%2C+Paul+Whatmough%2C+and+Babak+Ehteshami+Bejnordi%2E+Mixture+of+cache%2Dconditional+experts+for+efficient+mobil)
76. Luca Soldaini, Rodney Kinney, Akshita Bhagia, Dustin Schwenk, David Atkinson, Russell Authur, Ben Bogin, Khyathi Chandu, Jennifer Dumas, Yanai Elazar, Valentin Hofmann, Ananya Harsh Jha, Sachin Kumar, Li Lucy, Xinxi Lyu, Nathan Lambert, Ian Magnusson, Jacob Morrison, Niklas Muennighoff, Aakanksha Na… [find](https://scholar.google.com/scholar?q=Luca+Soldaini%2C+Rodney+Kinney%2C+Akshita+Bhagia%2C+Dustin+Schwenk%2C+David+Atkinson%2C+Russell+Authur%2C+Ben+Bogin%2C+Khyathi+Chandu%2C+Jennifer+Dumas%2C+Yanai+Elazar%2C+Valentin+Hofmann%2C+Ananya+Harsh+Jha%2C+Sachin+Kumar%2C)
77. Jifeng Song, Kai Huang, Xiangyu Yin, Boyuan Yang, and Wei Gao. Achieving sparse activation in small language models. arXiv preprint arXiv:2406.06562, 2024. — [arXiv:2406.06562](https://arxiv.org/abs/2406.06562) [find](https://scholar.google.com/scholar?q=Jifeng+Song%2C+Kai+Huang%2C+Xiangyu+Yin%2C+Boyuan+Yang%2C+and+Wei+Gao%2E+Achieving+sparse+activation+in+small+language+models%2E+arXiv+preprint+arXiv%3A2406%2E06562%2C+2024%2E+%E2%80%94+%5BarXiv%3A2406%2E06562%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2)
78. Yixin Song, Zeyu Mi, Haotong Xie, and Haibo Chen. Powerinfer: Fast large language model serving with a consumer-grade gpu. In Proceedings of the ACM SIGOPS 30th Symposium on Operating Systems Principles, pages 590–606, 2024. [find](https://scholar.google.com/scholar?q=Yixin+Song%2C+Zeyu+Mi%2C+Haotong+Xie%2C+and+Haibo+Chen%2E+Powerinfer%3A+Fast+large+language+model+serving+with+a+consumer%2Dgrade+gpu%2E+In+Proceedings+of+the+ACM+SIGOPS+30th+Symposium+on+Operating+Systems+Principl)
79. StabilityAI. stabilityai/stablelm-zephyr-3b. stablelm-zephyr-3b, 2023.11. [https://huggingface.co/stabilityai/](https://huggingface.co/stabilityai/) [↗](https://huggingface.co/stabilityai/)
80. StabilityAI. stabilityai/stablelm-2-zephyr*. stablelm-2-zephyr, 2024.01. [https://huggingface.co/stabilityai/](https://huggingface.co/stabilityai/) [↗](https://huggingface.co/stabilityai/)
81. Alon Talmor, Jonathan Herzig, Nicholas Lourie, and Jonathan Berant. [Commonsenseqa: A question answering challenge targeting commonsense knowledge](https://md.research.elamur.ai/free/en/papers/talmor-2019-commonsenseqa-question-answering-challenge-targeting/). In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 2019.
82. TensorOpera. Fox-1-1.6b. [https://huggingface.co/tensoropera/Fox-1-1.6B](https://huggingface.co/tensoropera/Fox-1-1.6B), 2024. [↗](https://huggingface.co/tensoropera/Fox-1-1.6B)
83. Toyota. Dclm. [https://huggingface.co/TRI-ML/DCLM-1B](https://huggingface.co/TRI-ML/DCLM-1B), 2024.08. [↗](https://huggingface.co/TRI-ML/DCLM-1B)
84. Mart van Baalen, Andrey Kuzmin, Suparna S Nair, Yuwei Ren, Eric Mahurin, Chirag Patel, Sundar Subramanian, Sanghyuk Lee, Markus Nagel, Joseph Soriaga, et al. Fp8 versus int8 for efficient deep learning inference. arXiv preprint arXiv:2303.17951, 2023. — [arXiv:2303.17951](https://arxiv.org/abs/2303.17951) [find](https://scholar.google.com/scholar?q=Mart+van+Baalen%2C+Andrey+Kuzmin%2C+Suparna+S+Nair%2C+Yuwei+Ren%2C+Eric+Mahurin%2C+Chirag+Patel%2C+Sundar+Subramanian%2C+Sanghyuk+Lee%2C+Markus+Nagel%2C+Joseph+Soriaga%2C+et+al%2E+Fp8+versus+int8+for+efficient+deep+learnin)
85. Tuowei Wang, Ruwen Fan, Minxing Huang, Zixu Hao, Kun Li, Ting Cao, Youyou Lu, Yaoxue Zhang, and Ju Ren. Ripple: Accelerating llm inference on smartphones with correlation-aware neuron management. arXiv preprint arXiv:2410.19274, 2024. — [arXiv:2410.19274](https://arxiv.org/abs/2410.19274) [find](https://scholar.google.com/scholar?q=Tuowei+Wang%2C+Ruwen+Fan%2C+Minxing+Huang%2C+Zixu+Hao%2C+Kun+Li%2C+Ting+Cao%2C+Youyou+Lu%2C+Yaoxue+Zhang%2C+and+Ju+Ren%2E+Ripple%3A+Accelerating+llm+inference+on+smartphones+with+correlation%2Daware+neuron+management%2E+arXi)
86. Taiqiang Wu, Cheng Hou, Shanshan Lao, Jiayi Li, Ngai Wong, Zhe Zhao, and Yujiu Yang. Weight-inherited distillation for task-agnostic bert compression. arXiv preprint arXiv:2305.09098, 2023. [find](https://scholar.google.com/scholar?q=Taiqiang+Wu%2C+Cheng+Hou%2C+Shanshan+Lao%2C+Jiayi+Li%2C+Ngai+Wong%2C+Zhe+Zhao%2C+and+Yujiu+Yang%2E+Weight%2Dinherited+distillation+for+task%2Dagnostic+bert+compression%2E+arXiv+preprint+arXiv%3A2305%2E09098%2C+2023%2E)
87. Nuwa Xi, Yuhan Chen, Sendong Zhao, Haochun Wang, Bing Qin, and Ting Liu. As-es learning: Towards efficient cot learning in small models. arXiv preprint arXiv:2403.01969, 2024. [find](https://scholar.google.com/scholar?q=Nuwa+Xi%2C+Yuhan+Chen%2C+Sendong+Zhao%2C+Haochun+Wang%2C+Bing+Qin%2C+and+Ting+Liu%2E+As%2Des+learning%3A+Towards+efficient+cot+learning+in+small+models%2E+arXiv+preprint+arXiv%3A2403%2E01969%2C+2024%2E)
88. Xiaohongshu. Minima. [https://huggingface.co/GeneZC/MiniMA-3B](https://huggingface.co/GeneZC/MiniMA-3B), 2023.11. [↗](https://huggingface.co/GeneZC/MiniMA-3B)
89. Xiaohongshu. Minima2. [https://huggingface.co/GeneZC/MiniMA-2-1B](https://huggingface.co/GeneZC/MiniMA-2-1B), 2024.07. [↗](https://huggingface.co/GeneZC/MiniMA-2-1B)
90. Daliang Xu, Wangsong Yin, Xin Jin, Ying Zhang, Shiyun Wei, Mengwei Xu, and Xuanzhe Liu. Llmcad: Fast and scalable on-device large language model inference. arXiv preprint arXiv:2309.04255, 2023. — [arXiv:2309.04255](https://arxiv.org/abs/2309.04255) [find](https://scholar.google.com/scholar?q=Daliang+Xu%2C+Wangsong+Yin%2C+Xin+Jin%2C+Ying+Zhang%2C+Shiyun+Wei%2C+Mengwei+Xu%2C+and+Xuanzhe+Liu%2E+Llmcad%3A+Fast+and+scalable+on%2Ddevice+large+language+model+inference%2E+arXiv+preprint+arXiv%3A2309%2E04255%2C+2023%2E+%E2%80%94+%5Bar)
91. Daliang Xu, Hao Zhang, Liming Yang, Ruiqi Liu, Gang Huang, Mengwei Xu, and Xuanzhe Liu. Empowering 1000 tokens/second on-device llm prefilling with mllm-npu. arXiv preprint arXiv:2407.05858, 2024. [find](https://scholar.google.com/scholar?q=Daliang+Xu%2C+Hao+Zhang%2C+Liming+Yang%2C+Ruiqi+Liu%2C+Gang+Huang%2C+Mengwei+Xu%2C+and+Xuanzhe+Liu%2E+Empowering+1000+tokens%2Fsecond+on%2Ddevice+llm+prefilling+with+mllm%2Dnpu%2E+arXiv+preprint+arXiv%3A2407%2E05858%2C+2024%2E)
92. Jiajun Xu, Zhiyuan Li, Wei Chen, Qun Wang, Xin Gao, Qi Cai, and Ziyuan Ling. On-device language models: A comprehensive review. arXiv preprint arXiv:2409.00088, 2024. — [arXiv:2409.00088](https://arxiv.org/abs/2409.00088) [find](https://scholar.google.com/scholar?q=Jiajun+Xu%2C+Zhiyuan+Li%2C+Wei+Chen%2C+Qun+Wang%2C+Xin+Gao%2C+Qi+Cai%2C+and+Ziyuan+Ling%2E+On%2Ddevice+language+models%3A+A+comprehensive+review%2E+arXiv+preprint+arXiv%3A2409%2E00088%2C+2024%2E+%E2%80%94+%5BarXiv%3A2409%2E00088%5D%28https%3A%2F%2Farxi)
93. Mengwei Xu, Dongqi Cai, Yaozong Wu, Xiang Li, and Shangguang Wang. Fwdllm: Efficient fedllm using forward gradient. arXiv preprint arXiv:2308.13894, 2023. — [arXiv:2308.13894](https://arxiv.org/abs/2308.13894) [find](https://scholar.google.com/scholar?q=Mengwei+Xu%2C+Dongqi+Cai%2C+Yaozong+Wu%2C+Xiang+Li%2C+and+Shangguang+Wang%2E+Fwdllm%3A+Efficient+fedllm+using+forward+gradient%2E+arXiv+preprint+arXiv%3A2308%2E13894%2C+2023%2E+%E2%80%94+%5BarXiv%3A2308%2E13894%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F23)
94. Mengwei Xu, Wangsong Yin, Dongqi Cai, Rongjie Yi, Daliang Xu, Qipeng Wang, Bingyang Wu, Yihao Zhao, Chen Yang, Shihe Wang, et al. A survey of resource-efficient llm and multimodal foundation models. arXiv preprint arXiv:2401.08092, 2024. [find](https://scholar.google.com/scholar?q=Mengwei+Xu%2C+Wangsong+Yin%2C+Dongqi+Cai%2C+Rongjie+Yi%2C+Daliang+Xu%2C+Qipeng+Wang%2C+Bingyang+Wu%2C+Yihao+Zhao%2C+Chen+Yang%2C+Shihe+Wang%2C+et+al%2E+A+survey+of+resource%2Defficient+llm+and+multimodal+foundation+models%2E+a)
95. Zhenliang Xue, Yixin Song, Zeyu Mi, Le Chen, Yubin Xia, and Haibo Chen. Powerinfer-2: Fast large language model inference on a smartphone. arXiv preprint arXiv:2406.06282, 2024. [find](https://scholar.google.com/scholar?q=Zhenliang+Xue%2C+Yixin+Song%2C+Zeyu+Mi%2C+Le+Chen%2C+Yubin+Xia%2C+and+Haibo+Chen%2E+Powerinfer%2D2%3A+Fast+large+language+model+inference+on+a+smartphone%2E+arXiv+preprint+arXiv%3A2406%2E06282%2C+2024%2E)
96. Rongjie Yi, Liwei Guo, Shiyun Wei, Ao Zhou, Shangguang Wang, and Mengwei Xu. Edgemoe: Fast on-device inference of moe-based large language models. arXiv preprint arXiv:2308.14352, 2023. — [arXiv:2308.14352](https://arxiv.org/abs/2308.14352) [find](https://scholar.google.com/scholar?q=Rongjie+Yi%2C+Liwei+Guo%2C+Shiyun+Wei%2C+Ao+Zhou%2C+Shangguang+Wang%2C+and+Mengwei+Xu%2E+Edgemoe%3A+Fast+on%2Ddevice+inference+of+moe%2Dbased+large+language+models%2E+arXiv+preprint+arXiv%3A2308%2E14352%2C+2023%2E+%E2%80%94+%5BarXiv%3A2308%2E)
97. Wangsong Yin, Mengwei Xu, Yuanchun Li, and Xuanzhe Liu. Llm as a system service on mobile devices. arXiv preprint arXiv:2403.11805, 2024. — [arXiv:2403.11805](https://arxiv.org/abs/2403.11805) [find](https://scholar.google.com/scholar?q=Wangsong+Yin%2C+Mengwei+Xu%2C+Yuanchun+Li%2C+and+Xuanzhe+Liu%2E+Llm+as+a+system+service+on+mobile+devices%2E+arXiv+preprint+arXiv%3A2403%2E11805%2C+2024%2E+%E2%80%94+%5BarXiv%3A2403%2E11805%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2403%2E11805%29)
98. Wangsong Yin, Rongjie Yi, Daliang Xu, Gang Huang, Mengwei Xu, and Xuanzhe Liu. Elms: Elasticized large language models on mobile devices. arXiv preprint arXiv:2409.09071, 2024. [find](https://scholar.google.com/scholar?q=Wangsong+Yin%2C+Rongjie+Yi%2C+Daliang+Xu%2C+Gang+Huang%2C+Mengwei+Xu%2C+and+Xuanzhe+Liu%2E+Elms%3A+Elasticized+large+language+models+on+mobile+devices%2E+arXiv+preprint+arXiv%3A2409%2E09071%2C+2024%2E)
99. Jinliang Yuan, Chen Yang, Dongqi Cai, Shihe Wang, Xin Yuan, Zeling Zhang, Xiang Li, Dingge Zhang, Hanzi Mei, Xianqing Jia, et al. Mobile foundation model as firmware. arXiv preprint arXiv:2308.14363, 2023. — [arXiv:2308.14363](https://arxiv.org/abs/2308.14363) [find](https://scholar.google.com/scholar?q=Jinliang+Yuan%2C+Chen+Yang%2C+Dongqi+Cai%2C+Shihe+Wang%2C+Xin+Yuan%2C+Zeling+Zhang%2C+Xiang+Li%2C+Dingge+Zhang%2C+Hanzi+Mei%2C+Xianqing+Jia%2C+et+al%2E+Mobile+foundation+model+as+firmware%2E+arXiv+preprint+arXiv%3A2308%2E14363%2C+)
100. Sha Yuan, Hanyu Zhao, Zhengxiao Du, Ming Ding, Xiao Liu, Yukuo Cen, Xu Zou, Zhilin Yang, and Jie Tang. Wudaocorpora: A super large-scale chinese corpora for pre-training language models. AI Open, 2:65–68, 2021. [find](https://scholar.google.com/scholar?q=Sha+Yuan%2C+Hanyu+Zhao%2C+Zhengxiao+Du%2C+Ming+Ding%2C+Xiao+Liu%2C+Yukuo+Cen%2C+Xu+Zou%2C+Zhilin+Yang%2C+and+Jie+Tang%2E+Wudaocorpora%3A+A+super+large%2Dscale+chinese+corpora+for+pre%2Dtraining+language+models%2E+AI+Open%2C+2%3A65)
101. Rowan Zellers, Ari Holtzman, Yonatan Bisk, Ali Farhadi, and Yejin Choi. Hellaswag: Can a machine really finish your sentence? In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, 2019. [find](https://scholar.google.com/scholar?q=Rowan+Zellers%2C+Ari+Holtzman%2C+Yonatan+Bisk%2C+Ali+Farhadi%2C+and+Yejin+Choi%2E+Hellaswag%3A+Can+a+machine+really+finish+your+sentence%3F+In+Proceedings+of+the+57th+Annual+Meeting+of+the+Association+for+Computati)
102. Chenyang Zhang, Feng Zhang, Kuangyu Chen, Mingjun Chen, Bingsheng He, and Xiaoyong Du. Edgenn: Efficient neural network inference for cpu-gpu integrated edge devices. In 2023 IEEE 39th International Conference on Data Engineering (ICDE), pages 1193–1207. IEEE, 2023. [find](https://scholar.google.com/scholar?q=Chenyang+Zhang%2C+Feng+Zhang%2C+Kuangyu+Chen%2C+Mingjun+Chen%2C+Bingsheng+He%2C+and+Xiaoyong+Du%2E+Edgenn%3A+Efficient+neural+network+inference+for+cpu%2Dgpu+integrated+edge+devices%2E+In+2023+IEEE+39th+International+C)
103. Hugh Zhang, Jeff Da, Dean Lee, Vaughn Robinson, Catherine Wu, Will Song, Tiffany Zhao, Pranav Raja, Dylan Slack, Qin Lyu, et al. A careful examination of large language model performance on grade school arithmetic. arXiv preprint arXiv:2405.00332, 2024. [find](https://scholar.google.com/scholar?q=Hugh+Zhang%2C+Jeff+Da%2C+Dean+Lee%2C+Vaughn+Robinson%2C+Catherine+Wu%2C+Will+Song%2C+Tiffany+Zhao%2C+Pranav+Raja%2C+Dylan+Slack%2C+Qin+Lyu%2C+et+al%2E+A+careful+examination+of+large+language+model+performance+on+grade+scho)
104. Kaiyan Zhang, Jianyu Wang, Ning Ding, Biqing Qi, Ermo Hua, Xingtai Lv, and Bowen Zhou. Fast and slow generating: An empirical study on large and small language models collaborative decoding. arXiv preprint arXiv:2406.12295, 2024. [find](https://scholar.google.com/scholar?q=Kaiyan+Zhang%2C+Jianyu+Wang%2C+Ning+Ding%2C+Biqing+Qi%2C+Ermo+Hua%2C+Xingtai+Lv%2C+and+Bowen+Zhou%2E+Fast+and+slow+generating%3A+An+empirical+study+on+large+and+small+language+models+collaborative+decoding%2E+arXiv+pre)
105. Peiyuan Zhang, Guangtao Zeng, Tianduo Wang, and Wei Lu. [Tinyllama: An open-source small language model](https://md.research.elamur.ai/free/en/papers/zhang-2024-tinyllama-open-source-small-language/), 2024.
106. Xinlu Zhang, Zhiyu Zoey Chen, Xi Ye, Xianjun Yang, Lichang Chen, William Yang Wang, and Linda Ruth Petzold. Unveiling the impact of coding data instruction fine-tuning on large language models reasoning. arXiv preprint arXiv:2405.20535, 2024. — [arXiv:2405.20535](https://arxiv.org/abs/2405.20535) [find](https://scholar.google.com/scholar?q=Xinlu+Zhang%2C+Zhiyu+Zoey+Chen%2C+Xi+Ye%2C+Xianjun+Yang%2C+Lichang+Chen%2C+William+Yang+Wang%2C+and+Linda+Ruth+Petzold%2E+Unveiling+the+impact+of+coding+data+instruction+fine%2Dtuning+on+large+language+models+reasoni)
107. Yusen Zhang, Yunchuan Qin, Yufeng Zhang, Xu Zhou, Songlei Jian, Yusong Tan, and Kenli Li. Oncenas: Discovering efficient on-device inference neural networks for edge devices. Information Sciences, 669:120567, 2024. [find](https://scholar.google.com/scholar?q=Yusen+Zhang%2C+Yunchuan+Qin%2C+Yufeng+Zhang%2C+Xu+Zhou%2C+Songlei+Jian%2C+Yusong+Tan%2C+and+Kenli+Li%2E+Oncenas%3A+Discovering+efficient+on%2Ddevice+inference+neural+networks+for+edge+devices%2E+Information+Sciences%2C+669)
108. Zhengping Zhou, Lezhi Li, Xinxi Chen, and Andy Li. Mini-giants:” small” language models and open source win-win. arXiv preprint arXiv:2307.08189, 2023. — [arXiv:2307.08189](https://arxiv.org/abs/2307.08189) [find](https://scholar.google.com/scholar?q=Zhengping+Zhou%2C+Lezhi+Li%2C+Xinxi+Chen%2C+and+Andy+Li%2E+Mini%2Dgiants%3A%E2%80%9D+small%E2%80%9D+language+models+and+open+source+win%2Dwin%2E+arXiv+preprint+arXiv%3A2307%2E08189%2C+2023%2E+%E2%80%94+%5BarXiv%3A2307%2E08189%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2307%2E)

*Updated 24 October 2026*