---
title: "A Comprehensive Survey of Small Language Models in the Era"
description: "The authors define an SLM by two bounds. 463 references, cited by 60 papers here. Among them: 99."
authors: ["Fali Wang", "Zhiwei Zhang", "Xianren Zhang", "Zongyu Wu", "Tzuhao Mo", "Qiuhao Lu", "Wanjing Wang", "Rui Li", "Junjie Xu", "Xianfeng Tang", "Qi He", "Yao Ma", "Ming Huang", "Suhang Wang"]
year: 2024
venue: "arXiv:2411.03350"
source: "https://arxiv.org/abs/2411.03350"
doi: "10.48550/arxiv.2411.03350"
arxiv: "2411.03350"
licence: "arXiv — non-exclusive license to distribute"
licence_url: "http://arxiv.org/licenses/nonexclusive-distrib/1.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/wang-2024-comprehensive-survey-small-language-models/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/wang-2024-comprehensive-survey-small-language-models/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2024](https://md.research.elamur.ai/free/en/papers/2024/page/2/) / A Comprehensive Survey of Small Language Models in the Era of Large Language Models: Techniques, Enhancements, Applications, Collaboration with LLMs, and Trustworthiness

# A Comprehensive Survey of Small Language Models in the Era of Large Language Models: Techniques, Enhancements, Applications, Collaboration with LLMs, and Trustworthiness

**Authors:** [Fali Wang](https://md.research.elamur.ai/free/en/authors/fali-wang/), [Zhiwei Zhang](https://md.research.elamur.ai/free/en/authors/zhiwei-zhang/), [Xianren Zhang](https://md.research.elamur.ai/free/en/authors/xianren-zhang/), [Zongyu Wu](https://md.research.elamur.ai/free/en/authors/zongyu-wu/), Tzuhao Mo, Qiuhao Lu, Wanjing Wang, [Rui Li](https://md.research.elamur.ai/free/en/authors/rui-li/), [Junjie Xu](https://md.research.elamur.ai/free/en/authors/junjie-xu/), [Xianfeng Tang](https://md.research.elamur.ai/free/en/authors/xianfeng-tang/), [Qi He](https://md.research.elamur.ai/free/en/authors/qi-he/), [Yao Ma](https://md.research.elamur.ai/free/en/authors/yao-ma/), Ming Huang and [Suhang Wang](https://md.research.elamur.ai/free/en/authors/suhang-wang/) · **Year:** 2024 · **Venue:** [arXiv:2411.03350](https://md.research.elamur.ai/free/en/venues/acm-transactions-on-intelligent-systems-and-technology/) ([ACM Transactions on Intelligent Systems and Technology 2024](https://md.research.elamur.ai/free/en/venues/acm-transactions-on-intelligent-systems-and-technology/2024/)) · **Publisher:** [Association for Computing Machinery](https://md.research.elamur.ai/free/en/publishers/association-for-computing-machinery/) · **Licence:** [arXiv — non-exclusive license to distribute](http://arxiv.org/licenses/nonexclusive-distrib/1.0/)

**Links:** [Publisher's page](https://arxiv.org/abs/2411.03350) · [PDF](https://arxiv.org/pdf/2411.03350) · [DOI 10.48550/arxiv.2411.03350](https://doi.org/10.48550/arxiv.2411.03350) · [arXiv 2411.03350](https://arxiv.org/abs/2411.03350) · [OpenAlex](https://openalex.org/W4414320956)

**Citations:** 99 citations (OpenAlex, 2026-10-14) · [60](https://md.research.elamur.ai/free/en/papers/wang-2024-comprehensive-survey-small-language-models/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
LLMs such as Llama-3.1 405B are costly to run and fine-tune. They usually sit behind cloud APIs, which creates privacy risks, and they respond too slowly for on-device or real-time use. They can also fall short in specialised fields like medicine or law. This survey examines small language models (SLMs) as an alternative, focusing on models of up to roughly 7B parameters. The authors define an SLM by two bounds. The lower bound is the smallest size at which emergent abilities appear. The upper bound is the largest size that resource-limited hardware can still support. They then organise the field into a taxonomy:
- architectures and training,
- ways to derive SLMs from LLMs (pruning, knowledge distillation, quantization),
- enhancement methods,
- task applications and on-device deployment,
- general and domain-specific model families,
- SLMs that assist LLMs,
- SLM–LLM cooperation,
- trustworthiness.

The survey does not run new experiments. Its value is a structured map and reference collection for anyone building efficient, private or specialised language systems.

## Key points
- The motivating costs are concrete:
  - Llama 3.1 405B was trained on 16K H100 GPUs over 54 days and needs about 202.5 GB of GPU memory even at int4.
  - Llama 2 7B takes around 84 seconds to produce 100 tokens on a Snapdragon 685 phone.
- The survey reviews compression and enhancement methods, including SqueezeLLM, BitNet b1.58, QLoRA, GKD and DistiLLM. It also covers representative SLM families: Phi, Qwen, Gemma, TinyLlama, MiniCPM, BioMedLM, ChemLLM and AstroLLaMA.
- SLMs can support LLMs in several ways:
  - as hallucination detectors and RAG helpers (e.g. Self-RAG, CRAG),
  - as proxies for fine-tuning (Proxy-tuning, Emulated Fine-tuning),
  - as safety guards (Llama Guard),
  - as evaluators (SelfCheckGPT, FActScore).
- Two kinds of SLM–LLM cooperation are described:
  - cloud-edge splits, where private data stays on a local SLM (CoGenesis, CROSSLM),
  - task-centric pipelines, where each model handles part of a task.
- Trustworthiness research has mostly studied models of 7B parameters or larger. The authors therefore call for a TrustLLM-style systematic evaluation of SLMs. The collected models and methods are available at github.com/FairyFali/SLMs-Survey.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [University of Pennsylvania](https://md.research.elamur.ai/free/en/universities/university-of-pennsylvania/) — University, Philadelphia, United States · 233 works · 65,583 citations
- [Amazon](https://md.research.elamur.ai/free/en/companies/amazon/) — Company, Seattle, United States · 128 works · 41,697 citations
- [Pennsylvania State University](https://md.research.elamur.ai/free/en/universities/pennsylvania-state-university/) — University, State College, United States · 67 works · 5,865 citations
- [Rensselaer Polytechnic Institute](https://md.research.elamur.ai/free/en/universities/rensselaer-polytechnic-institute/) — University, Troy, United States · 23 works · 5,092 citations
- [The University of Texas Health Science Center at Houston](https://md.research.elamur.ai/free/en/universities/the-university-of-texas-health-science-center-at-houston/) — University, Houston, United States · 7 works · 195 citations

📚 Shelves (5): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 18,614 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 17,965 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 17,450 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 17,296 works · [👁️ Computer Vision and Pattern Recognition](https://md.research.elamur.ai/free/en/shelves/computer-science/computer-vision-and-pattern-recognition/) · 3,486 works

🧭 Topics (1): [🧠 Language models & pretraining](https://md.research.elamur.ai/free/en/topics/language-models/) · 3,894 works

## Cite

Cite as: Fali Wang, Zhiwei Zhang, Xianren Zhang, Zongyu Wu, Tzuhao Mo, Qiuhao Lu, Wanjing Wang, Rui Li, Junjie Xu, Xianfeng Tang, Qi He, Yao Ma, Ming Huang and Suhang Wang. 2024. A Comprehensive Survey of Small Language Models in the Era of Large Language Models: Techniques, Enhancements, Applications, Collaboration with LLMs, and Trustworthiness. arXiv:2411.03350.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@misc{wang2024comprehensive,
  title = {{A Comprehensive Survey of Small Language Models in the Era of Large Language Models: Techniques, Enhancements, Applications, Collaboration with LLMs, and Trustworthiness}},
  author = {Fali Wang, Zhiwei Zhang, Xianren Zhang, Zongyu Wu, Tzuhao Mo, Qiuhao Lu, Wanjing Wang, Rui Li, Junjie Xu, Xianfeng Tang, Qi He, Yao Ma, Ming Huang and Suhang Wang},
  year = {2024},
  doi = {10.48550/arxiv.2411.03350},
  eprint = {2411.03350},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2411.03350}
}
```

- [← previous Measuring short-form factuality in large language models](https://md.research.elamur.ai/free/en/papers/wei-2024-measuring-short-form-factuality-large/)
- [next → Chameleon: Mixed-Modal Early-Fusion Foundation Models](https://md.research.elamur.ai/free/en/papers/team-2024-chameleon-mixed-modal-early-fusion/)
- [↑ back to papers of 2024](https://md.research.elamur.ai/free/en/papers/2024/page/2/)

## References

463 references, 165 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Marah Abdin, Sam Ade Jacobs, Ammar Ahmad Awan, Jyoti Aneja, Ahmed Awadallah, Hany Awadalla, Nguyen Bach, Amit Bahree, Arash Bakhtiari, Harkirat Behl, et al. 2024. [Phi-3 technical report: A highly capable language model locally on your phone](https://md.research.elamur.ai/free/en/papers/abdin-2024-phi-3-technical-report-highly/). arXiv preprint arXiv:2404.14219 (2024).
2. Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. 2023. [Gpt-4 technical report](https://md.research.elamur.ai/free/en/papers/openai-2023-gpt-4-technical-report/). arXiv preprint arXiv:2303.08774 (2023).
3. Emre Can Acikgoz, Osman Batur İnce, Rayene Bench, Arda Anıl Boz, İlker Kesen, Aykut Erdem, and Erkut Erdem. 2024. Hippocrates: An Open-Source Framework for Advancing Large Language Models in Healthcare. arXiv preprint arXiv:2404.16621 (2024). — [arXiv:2404.16621](https://arxiv.org/abs/2404.16621) [find](https://scholar.google.com/scholar?q=Emre+Can+Acikgoz%2C+Osman+Batur+%C4%B0nce%2C+Rayene+Bench%2C+Arda+An%C4%B1l+Boz%2C+%C4%B0lker+Kesen%2C+Aykut+Erdem%2C+and+Erkut+Erdem%2E+2024%2E+Hippocrates%3A+An+Open%2DSource+Framework+for+Advancing+Large+Language+Models+in+Healthcar)
4. Harshavardhan Adepu, Zhanpeng Zeng, Li Zhang, and Vikas Singh. 2024. FrameQuant: Flexible Low-Bit Quantization for Transformers. arXiv preprint arXiv:2403.06082 (2024). — [arXiv:2403.06082](https://arxiv.org/abs/2403.06082) [find](https://scholar.google.com/scholar?q=Harshavardhan+Adepu%2C+Zhanpeng+Zeng%2C+Li+Zhang%2C+and+Vikas+Singh%2E+2024%2E+FrameQuant%3A+Flexible+Low%2DBit+Quantization+for+Transformers%2E+arXiv+preprint+arXiv%3A2403%2E06082+%282024%29%2E+%E2%80%94+%5BarXiv%3A2403%2E06082%5D%28https%3A%2F%2Far)
5. Abien Fred Agarap. 2018. Deep Learning using Rectified Linear Units (ReLU). CoRR abs/1803.08375 (2018). arXiv:1803.08375 http://arxiv.org/abs/1803.08375 — [arXiv:1803.08375](https://arxiv.org/abs/1803.08375) [↗](http://arxiv.org/abs/1803.08375)
6. Rishabh Agarwal, Nino Vieillard, Yongchao Zhou, Piotr Stanczyk, Sabela Ramos Garea, Matthieu Geist, and Olivier Bachem. 2024. [On-policy distillation of language models: Learning from self-generated mistakes](https://md.research.elamur.ai/free/en/papers/agarwal-2023-policy-distillation-language-models-learning/). In The Twelfth International Conference on Learning Representations.
7. Meta AI. 2024. Llama 3.2: Revolutionizing edge AI and vision with open, customizable models. [https://ai.meta.com/blog/llama-3-2-connect-2024-vision-edge-mobile-devices/](https://ai.meta.com/blog/llama-3-2-connect-2024-vision-edge-mobile-devices/) Accessed: 2024-9-25. [↗](https://ai.meta.com/blog/llama-3-2-connect-2024-vision-edge-mobile-devices/)
8. Joshua Ainslie, James Lee-Thorp, Michiel de Jong, Yury Zemlyanskiy, Federico Lebrón, and Sumit Sanghai. 2023. [GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpoints](https://md.research.elamur.ai/free/en/papers/ainslie-2023-gqa-training-generalized-multi-query/). arXiv:2305.13245 [cs.CL] https://arxiv.org/abs/2305.13245 [↗](https://arxiv.org/abs/2305.13245)
9. Loubna Ben Allal, Anton Lozhkov, Elie Bakouch, Leandro von Werra, and Thomas Wolf. 2024. SmolLM - blazingly fast and remarkably powerful. [find](https://scholar.google.com/scholar?q=Loubna+Ben+Allal%2C+Anton+Lozhkov%2C+Elie+Bakouch%2C+Leandro+von+Werra%2C+and+Thomas+Wolf%2E+2024%2E+SmolLM+%2D+blazingly+fast+and+remarkably+powerful%2E)
10. Ebtesam Almazrouei, Hamza Alobeidli, Abdulaziz Alshamsi, Alessandro Cappelli, Ruxandra Cojocaru, Mérouane Debbah, Étienne Goffinet, Daniel Hesslow, Julien Launay, Quentin Malartic, et al. 2023. [The falcon series of open language models](https://md.research.elamur.ai/free/en/papers/almazrouei-2023-falcon-series-open-language-models/). arXiv preprint arXiv:2311.16867 (2023).
11. Guilherme FCF Almeida, José Luiz Nunes, Neele Engelmann, Alex Wiegmann, and Marcelo de Araújo. 2024. Exploring the psychology of LLMs’ moral and legal reasoning. Artificial Intelligence 333 (2024), 104145. [find](https://scholar.google.com/scholar?q=Guilherme+FCF+Almeida%2C+Jos%C3%A9+Luiz+Nunes%2C+Neele+Engelmann%2C+Alex+Wiegmann%2C+and+Marcelo+de+Ara%C3%BAjo%2E+2024%2E+Exploring+the+psychology+of+LLMs%E2%80%99+moral+and+legal+reasoning%2E+Artificial+Intelligence+333+%282024%29%2C+10)
12. Reza Yazdani Aminabadi, Samyam Rajbhandari, Ammar Ahmad Awan, Cheng Li, Du Li, Elton Zheng, Olatunji Ruwase, Shaden Smith, Minjia Zhang, Jeff Rasley, et al. 2022. Deepspeed-inference: enabling efficient inference of transformer models at unprecedented scale. In SC22: International Conference for Hig… [find](https://scholar.google.com/scholar?q=Reza+Yazdani+Aminabadi%2C+Samyam+Rajbhandari%2C+Ammar+Ahmad+Awan%2C+Cheng+Li%2C+Du+Li%2C+Elton+Zheng%2C+Olatunji+Ruwase%2C+Shaden+Smith%2C+Minjia+Zhang%2C+Jeff+Rasley%2C+et+al%2E+2022%2E+Deepspeed%2Dinference%3A+enabling+efficie)
13. Yongqi An, Xu Zhao, Tao Yu, Ming Tang, and Jinqiao Wang. 2024. Fluctuation-based adaptive structured pruning for large language models. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 38. 10865–10873. [find](https://scholar.google.com/scholar?q=Yongqi+An%2C+Xu+Zhao%2C+Tao+Yu%2C+Ming+Tang%2C+and+Jinqiao+Wang%2E+2024%2E+Fluctuation%2Dbased+adaptive+structured+pruning+for+large+language+models%2E+In+Proceedings+of+the+AAAI+Conference+on+Artificial+Intelligence)
14. Rohan Anil, Andrew M Dai, Orhan Firat, Melvin Johnson, Dmitry Lepikhin, Alexandre Passos, Siamak Shakeri, Emanuel Taropa, Paige Bailey, Zhifeng Chen, et al. 2023. [Palm 2 technical report](https://md.research.elamur.ai/free/en/papers/anil-2023-palm-2-technical-report/). arXiv preprint arXiv:2305.10403 (2023).
15. AI Anthropic. 2024. The claude 3 model family: Opus, sonnet, haiku. Claude-3 Model Card 1 (2024). [find](https://scholar.google.com/scholar?q=AI+Anthropic%2E+2024%2E+The+claude+3+model+family%3A+Opus%2C+sonnet%2C+haiku%2E+Claude%2D3+Model+Card+1+%282024%29%2E)
16. David Anugraha, Genta Indra Winata, Chenyue Li, Patrick Amadeus Irawan, and En-Shiun Annie Lee. 2024. ProxyLM: Predicting Language Model Performance on Multilingual Tasks via Proxy Models. arXiv preprint arXiv:2406.09334 (2024). [find](https://scholar.google.com/scholar?q=David+Anugraha%2C+Genta+Indra+Winata%2C+Chenyue+Li%2C+Patrick+Amadeus+Irawan%2C+and+En%2DShiun+Annie+Lee%2E+2024%2E+ProxyLM%3A+Predicting+Language+Model+Performance+on+Multilingual+Tasks+via+Proxy+Models%2E+arXiv+prepr)
17. Viraat Aryabumi, Yixuan Su, Raymond Ma, Adrien Morisot, Ivan Zhang, Acyr Locatelli, Marzieh Fadaee, Ahmet Üstün, and Sara Hooker. 2024. To Code, or Not To Code? Exploring Impact of Code in Pre-training. arXiv preprint arXiv:2408.10914 (2024). — [arXiv:2408.10914](https://arxiv.org/abs/2408.10914) [find](https://scholar.google.com/scholar?q=Viraat+Aryabumi%2C+Yixuan+Su%2C+Raymond+Ma%2C+Adrien+Morisot%2C+Ivan+Zhang%2C+Acyr+Locatelli%2C+Marzieh+Fadaee%2C+Ahmet+%C3%9Cst%C3%BCn%2C+and+Sara+Hooker%2E+2024%2E+To+Code%2C+or+Not+To+Code%3F+Exploring+Impact+of+Code+in+Pre%2Dtrainin)
18. Akari Asai, Zeqiu Wu, Yizhong Wang, Avirup Sil, and Hannaneh Hajishirzi. 2024. [Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflection](https://md.research.elamur.ai/free/en/papers/asai-2023-self-rag-learning-retrieve-generate/). In The Twelfth International Conference on Learning Representations. [https://openreview.net/forum?id=hSyW5go0v8](https://openreview.net/forum?id=hSyW5go0v8) [↗](https://openreview.net/forum?id=hSyW5go0v8)
19. Saleh Ashkboos, Maximilian L. Croci, Marcelo Gennari do Nascimento, Torsten Hoefler, and James Hensman. 2024. SliceGPT: Compress Large Language Models by Deleting Rows and Columns. In The Twelfth International Conference on Learning Representations. [https://openreview.net/forum?id=vXxardq6db](https://openreview.net/forum?id=vXxardq6db) [↗](https://openreview.net/forum?id=vXxardq6db)
20. Jacob Austin, Augustus Odena, Maxwell Nye, Maarten Bosma, Henryk Michalewski, David Dohan, Ellen Jiang, Carrie Cai, Michael Terry, Quoc Le, et al. 2021. [Program synthesis with large language models](https://md.research.elamur.ai/free/en/papers/austin-2021-program-synthesis-large-language-models/). arXiv preprint arXiv:2108.07732 (2021).
21. Amos Azaria and Tom Mitchell. 2023. [The Internal State of an LLM Knows When It’s Lying](https://md.research.elamur.ai/free/en/papers/azaria-2023-internal-state-llm-knows-when/). In Findings of the Association for Computational Linguistics: EMNLP 2023. 967–976.
22. Zhangir Azerbayev, Hailey Schoelkopf, Keiran Paster, Marco Dos Santos, Stephen McAleer, Albert Q Jiang, Jia Deng, Stella Biderman, and Sean Welleck. 2023. [Llemma: An open language model for mathematics](https://md.research.elamur.ai/free/en/papers/azerbayev-2023-llemma-open-language-model-mathematics/). arXiv preprint arXiv:2310.10631 (2023).
23. Jinze Bai, Shuai Bai, Yunfei Chu, Zeyu Cui, Kai Dang, Xiaodong Deng, Yang Fan, Wenbin Ge, Yu Han, Fei Huang, Binyuan Hui, Luo Ji, Mei Li, Junyang Lin, Runji Lin, Dayiheng Liu, Gao Liu, Chengqiang Lu, Keming Lu, Jianxin Ma, Rui Men, Xingzhang Ren, Xuancheng Ren, Chuanqi Tan, Sinan Tan, Jianhong Tu, P… — [Qwen Technical Report](https://md.research.elamur.ai/free/en/papers/bai-2023-qwen-technical-report/)
24. Jason Baumgartner, Savvas Zannettou, Brian Keegan, Megan Squire, and Jeremy Blackburn. 2020. The pushshift reddit dataset. In Proceedings of the international AAAI conference on web and social media, Vol. 14. 830–839. [find](https://scholar.google.com/scholar?q=Jason+Baumgartner%2C+Savvas+Zannettou%2C+Brian+Keegan%2C+Megan+Squire%2C+and+Jeremy+Blackburn%2E+2020%2E+The+pushshift+reddit+dataset%2E+In+Proceedings+of+the+international+AAAI+conference+on+web+and+social+media%2C+)
25. Maximilian Beck, Korbinian Pöppel, Markus Spanring, Andreas Auer, Oleksandra Prudnikova, Michael K Kopp, Günter Klambauer, Johannes Brandstetter, and Sepp Hochreiter. 2024. [xLSTM: Extended Long Short-Term Memory](https://md.research.elamur.ai/free/en/papers/beck-2024-xlstm-extended-long-short-term/). In The Thirty-eighth Annual Conference on Neural Information Processing Systems. https:…
26. Marco Bellagente, Jonathan Tow, Dakota Mahan, Duy Phung, Maksym Zhuravinskyi, Reshinth Adithyan, James Baicoianu, Ben Brooks, Nathan Cooper, Ashish Datta, et al. 2024. [Stable lm 2 1.6 b technical report](https://md.research.elamur.ai/free/en/papers/bellagente-2024-stable-lm-2-1-6b/). arXiv preprint arXiv:2402.17834 (2024).
27. Loubna Ben Allal, Anton Lozhkov, Guilherme Penedo, Thomas Wolf, and Leandro von Werra. 2024. SmolLM-Corpus. [https://huggingface.co/datasets/HuggingFaceTB/smollm-corpus](https://huggingface.co/datasets/HuggingFaceTB/smollm-corpus) [↗](https://huggingface.co/datasets/HuggingFaceTB/smollm-corpus)
28. Benjamin Bergner, Andrii Skliar, Amelie Royer, Tijmen Blankevoort, Yuki Asano, and Babak Ehteshami Bejnordi. 2024. Think Big, Generate Quick: LLM-to-SLM for Fast Autoregressive Decoding. arXiv preprint arXiv:2402.16844 (2024). [find](https://scholar.google.com/scholar?q=Benjamin+Bergner%2C+Andrii+Skliar%2C+Amelie+Royer%2C+Tijmen+Blankevoort%2C+Yuki+Asano%2C+and+Babak+Ehteshami+Bejnordi%2E+2024%2E+Think+Big%2C+Generate+Quick%3A+LLM%2Dto%2DSLM+for+Fast+Autoregressive+Decoding%2E+arXiv+preprin)
29. Milan Bhan, Jean-Noel Vittaut, Nicolas Chesneau, and Marie-Jeanne Lesot. 2024. Self-AMPLIFY: Improving Small Language Models with Self Post Hoc Explanations. arXiv preprint arXiv:2402.12038 (2024). [find](https://scholar.google.com/scholar?q=Milan+Bhan%2C+Jean%2DNoel+Vittaut%2C+Nicolas+Chesneau%2C+and+Marie%2DJeanne+Lesot%2E+2024%2E+Self%2DAMPLIFY%3A+Improving+Small+Language+Models+with+Self+Post+Hoc+Explanations%2E+arXiv+preprint+arXiv%3A2402%2E12038+%282024%29%2E)
30. Zhen Bi, Ningyu Zhang, Yida Xue, Yixin Ou, Daxiong Ji, Guozhou Zheng, and Huajun Chen. 2023. Oceangpt: A large language model for ocean science tasks. arXiv preprint arXiv:2310.02031 (2023). [find](https://scholar.google.com/scholar?q=Zhen+Bi%2C+Ningyu+Zhang%2C+Yida+Xue%2C+Yixin+Ou%2C+Daxiong+Ji%2C+Guozhou+Zheng%2C+and+Huajun+Chen%2E+2023%2E+Oceangpt%3A+A+large+language+model+for+ocean+science+tasks%2E+arXiv+preprint+arXiv%3A2310%2E02031+%282023%29%2E)
31. Stella Biderman, Hailey Schoelkopf, Quentin Anthony, Herbie Bradley, Kyle O’Brien, Eric Hallahan, Mohammad Aflah Khan, Shivanshu Purohit, USVSN Sai Prashanth, Edward Raff, Aviya Skowron, Lintang Sutawika, and Oskar van der Wal. 2023. Pythia: A Suite for Analyzing Large Language Models Across Trainin… [find](https://scholar.google.com/scholar?q=Stella+Biderman%2C+Hailey+Schoelkopf%2C+Quentin+Anthony%2C+Herbie+Bradley%2C+Kyle+O%E2%80%99Brien%2C+Eric+Hallahan%2C+Mohammad+Aflah+Khan%2C+Shivanshu+Purohit%2C+USVSN+Sai+Prashanth%2C+Edward+Raff%2C+Aviya+Skowron%2C+Lintang+Sutaw)
32. Yonatan Bisk, Rowan Zellers, Jianfeng Gao, Yejin Choi, et al. 2020. [Piqa: Reasoning about physical commonsense in natural language](https://md.research.elamur.ai/free/en/papers/bisk-2019-piqa-reasoning-about-physical-commonsense/). In Proceedings of the AAAI conference on artificial intelligence, Vol. 34. 7432–7439.
33. Sid Black, Leo Gao, Phil Wang, Connor Leahy, and Stella Biderman. 2021. GPT-Neo: Large Scale Autoregressive Language Modeling with Mesh-Tensorflow. https://doi.org/10.5281/zenodo.5297715 If you use this software, please cite it using these metadata.. — [doi:10.5281/zenodo.5297715](https://doi.org/10.5281/zenodo.5297715) [↗](https://doi.org/10.5281/zenodo.5297715)
34. Elliot Bolton, Abhinav Venigalla, Michihiro Yasunaga, David Hall, Betty Xiong, Tony Lee, Roxana Daneshjou, Jonathan Frankle, Percy Liang, Michael Carbin, et al. 2024. [Biomedlm: A 2.7 b parameter language model trained on biomedical text](https://md.research.elamur.ai/free/en/papers/bolton-2024-biomedlm-2-7b-parameter-language/). arXiv preprint arXiv:2403.18421 (2024).
35. William James Bolton, Rafael Poyiadzi, Edward R Morrell, Gabriela van Bergen Gonzalez Bueno, and Lea Goetz. 2024. RAmBLA: A Framework for Evaluating the Reliability of LLMs as Assistants in the Biomedical Domain. arXiv preprint arXiv:2403.14578 (2024). — [arXiv:2403.14578](https://arxiv.org/abs/2403.14578) [find](https://scholar.google.com/scholar?q=William+James+Bolton%2C+Rafael+Poyiadzi%2C+Edward+R+Morrell%2C+Gabriela+van+Bergen+Gonzalez+Bueno%2C+and+Lea+Goetz%2E+2024%2E+RAmBLA%3A+A+Framework+for+Evaluating+the+Reliability+of+LLMs+as+Assistants+in+the+Biomed)
36. Luiz Bonifacio, Hugo Abonizio, Marzieh Fadaee, and Rodrigo Nogueira. 2022. [Inpars: Data augmentation for information retrieval using large language models](https://md.research.elamur.ai/free/en/papers/bonifacio-2022-inpars-data-augmentation-information-retrieval/). arXiv preprint arXiv:2202.05144 (2022).
37. Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel Ziegler, Jeffrey Wu, Clemens Winter, Chri… — [Language Models are Few-Shot Learners](https://md.research.elamur.ai/free/en/papers/brown-2020-language-models-few-shot-learners/)
38. Saikiran Bulusu, Bhavya Kailkhura, Bo Li, Pramod K Varshney, and Dawn Song. 2020. Anomalous example detection in deep learning: A survey. IEEE Access 8 (2020), 132330–132347. [find](https://scholar.google.com/scholar?q=Saikiran+Bulusu%2C+Bhavya+Kailkhura%2C+Bo+Li%2C+Pramod+K+Varshney%2C+and+Dawn+Song%2E+2020%2E+Anomalous+example+detection+in+deep+learning%3A+A+survey%2E+IEEE+Access+8+%282020%29%2C+132330%E2%80%93132347%2E)
39. Weilin Cai, Juyong Jiang, Fan Wang, Jing Tang, Sunghun Kim, and Jiayi Huang. 2024. A survey on mixture of experts. Authorea Preprints (2024). [find](https://scholar.google.com/scholar?q=Weilin+Cai%2C+Juyong+Jiang%2C+Fan+Wang%2C+Jing+Tang%2C+Sunghun+Kim%2C+and+Jiayi+Huang%2E+2024%2E+A+survey+on+mixture+of+experts%2E+Authorea+Preprints+%282024%29%2E)
40. Zheng Cai, Maosong Cao, Haojiong Chen, Kai Chen, Keyu Chen, Xin Chen, Xun Chen, Zehui Chen, Zhi Chen, Pei Chu, et al. 2024. [Internlm2 technical report](https://md.research.elamur.ai/free/en/papers/cai-2024-internlm2-technical-report/). arXiv preprint arXiv:2403.17297 (2024).
41. Nicholas Carlini, Florian Tramer, Eric Wallace, Matthew Jagielski, Ariel Herbert-Voss, Katherine Lee, Adam Roberts, Tom Brown, Dawn Song, Ulfar Erlingsson, et al. 2021. [Extracting training data from large language models](https://md.research.elamur.ai/free/en/papers/carlini-2020-extracting-training-data-large-language/). In 30th USENIX Security Symposium (USENIX Security 21). 2633–2650.
42. Samuel Carreira, Tomás Marques, José Ribeiro, and Carlos Grilo. 2023. Revolutionizing Mobile Interaction: Enabling a 3 Billion Parameter GPT LLM on Mobile. arXiv:2310.01434 [cs.CL] https://arxiv.org/abs/2310.01434 — [arXiv:2310.01434](https://arxiv.org/abs/2310.01434) [↗](https://arxiv.org/abs/2310.01434)
43. Iñigo Casanueva, Tadas Temčinas, Daniela Gerz, Matthew Henderson, and Ivan Vulić. 2020. [Efficient Intent Detection with Dual Sentence Encoders](https://md.research.elamur.ai/free/en/papers/casanueva-2020-efficient-intent-detection-dual-sentence/). In Proceedings of the 2nd Workshop on Natural Language Processing for Conversational AI. Association for Computational Linguistics, Online, 38–45.
44. Wei-Cheng Chang, X Yu Felix, Yin-Wen Chang, Yiming Yang, and Sanjiv Kumar. [n. d.]. Pre-training Tasks for Embedding-based Large-scale Retrieval. In International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Wei%2DCheng+Chang%2C+X+Yu+Felix%2C+Yin%2DWen+Chang%2C+Yiming+Yang%2C+and+Sanjiv+Kumar%2E+%5Bn%2E+d%2E%5D%2E+Pre%2Dtraining+Tasks+for+Embedding%2Dbased+Large%2Dscale+Retrieval%2E+In+International+Conference+on+Learning+Representation)
45. Patrick Chao, Edoardo Debenedetti, Alexander Robey, Maksym Andriushchenko, Francesco Croce, Vikash Sehwag, Edgar Dobriban, Nicolas Flammarion, George J Pappas, Florian Tramer, et al. 2024. [Jailbreakbench: An open robustness benchmark for jailbreaking large language models](https://md.research.elamur.ai/free/en/papers/chao-2024-jailbreakbench-open-robustness-benchmark-jailbreaking/). arXiv preprint arXiv:2404.…
46. Dong Chen, Shuo Zhang, Yueting Zhuang, Siliang Tang, Qidong Liu, Hua Wang, and Mingliang Xu. 2024. Improving Large Models with Small models: Lower Costs and Better Performance. arXiv preprint arXiv:2406.15471 (2024). — [arXiv:2406.15471](https://arxiv.org/abs/2406.15471) [find](https://scholar.google.com/scholar?q=Dong+Chen%2C+Shuo+Zhang%2C+Yueting+Zhuang%2C+Siliang+Tang%2C+Qidong+Liu%2C+Hua+Wang%2C+and+Mingliang+Xu%2E+2024%2E+Improving+Large+Models+with+Small+models%3A+Lower+Costs+and+Better+Performance%2E+arXiv+preprint+arXiv%3A24)
47. Dong Chen, Yueting Zhuang, Shuo Zhang, Jinfeng Liu, Su Dong, and Siliang Tang. 2024. Data Shunt: Collaboration of Small and Large Models for Lower Costs and Better Performance. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 38. 11249–11257. [find](https://scholar.google.com/scholar?q=Dong+Chen%2C+Yueting+Zhuang%2C+Shuo+Zhang%2C+Jinfeng+Liu%2C+Su+Dong%2C+and+Siliang+Tang%2E+2024%2E+Data+Shunt%3A+Collaboration+of+Small+and+Large+Models+for+Lower+Costs+and+Better+Performance%2E+In+Proceedings+of+the+A)
48. Hongzhan Chen, Siyue Wu, Xiaojun Quan, Rui Wang, Ming Yan, and Ji Zhang. 2023. MCC-KD: Multi-CoT Consistent Knowledge Distillation. In Findings of the Association for Computational Linguistics: EMNLP 2023. 6805–6820. [find](https://scholar.google.com/scholar?q=Hongzhan+Chen%2C+Siyue+Wu%2C+Xiaojun+Quan%2C+Rui+Wang%2C+Ming+Yan%2C+and+Ji+Zhang%2E+2023%2E+MCC%2DKD%3A+Multi%2DCoT+Consistent+Knowledge+Distillation%2E+In+Findings+of+the+Association+for+Computational+Linguistics%3A+EMNLP+)
49. Lihu Chen and Gaël Varoquaux. 2024. [What is the role of small models in the llm era: A survey](https://md.research.elamur.ai/free/en/papers/chen-2024-role-small-models-llm-era/). arXiv preprint arXiv:2409.06857 (2024).
50. Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde De Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, et al. 2021. [Evaluating large language models trained on code](https://md.research.elamur.ai/free/en/papers/chen-2021-evaluating-large-language-models-trained/). arXiv preprint arXiv:2107.03374 (2021).
51. Tianyi Chen, Tianyu Ding, Badal Yadav, Ilya Zharkov, and Luming Liang. 2023. Lorashear: Efficient large language model structured pruning and knowledge recovery. arXiv preprint arXiv:2310.18356 (2023). Manuscript submitted to ACM [find](https://scholar.google.com/scholar?q=Tianyi+Chen%2C+Tianyu+Ding%2C+Badal+Yadav%2C+Ilya+Zharkov%2C+and+Luming+Liang%2E+2023%2E+Lorashear%3A+Efficient+large+language+model+structured+pruning+and+knowledge+recovery%2E+arXiv+preprint+arXiv%3A2310%2E18356+%282023%29)
52. Wei Chen, Zhiyuan Li, and Mingyuan Ma. 2024. Octopus: On-device language model for function calling of software APIs. arXiv:2404.01549 [cs.CL] https://arxiv.org/abs/2404.01549 [↗](https://arxiv.org/abs/2404.01549)
53. Yangyi Chen, Fanchao Qi, Hongcheng Gao, Zhiyuan Liu, and Maosong Sun. 2022. Textual Backdoor Attacks Can Be More Harmful via Two Simple Tricks. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, EMNLP. 11215–11221. [find](https://scholar.google.com/scholar?q=Yangyi+Chen%2C+Fanchao+Qi%2C+Hongcheng+Gao%2C+Zhiyuan+Liu%2C+and+Maosong+Sun%2E+2022%2E+Textual+Backdoor+Attacks+Can+Be+More+Harmful+via+Two+Simple+Tricks%2E+In+Proceedings+of+the+2022+Conference+on+Empirical+Metho)
54. Yangyi Chen, Xingyao Wang, Hao Peng, and Heng Ji. 2024. A Single Transformer for Scalable Vision-Language Modeling. Transactions on Machine Learning Research (2024). [https://openreview.net/forum?id=nuzFG0Rbhy](https://openreview.net/forum?id=nuzFG0Rbhy) [↗](https://openreview.net/forum?id=nuzFG0Rbhy)
55. Zeming Chen, Alejandro Hernández Cano, Angelika Romanou, Antoine Bonnet, Kyle Matoba, Francesco Salvi, Matteo Pagliardini, Simin Fan, Andreas Köpf, Amirkeivan Mohtashami, et al. 2023. [MEDITRON-70B: Scaling Medical Pretraining for Large Language Models](https://md.research.elamur.ai/free/en/papers/chen-2023-meditron-70b-scaling-medical-pretraining/). arXiv preprint arXiv:2311.16079 (2023).
56. Zhiyu Chen, Wenhu Chen, Charese Smiley, Sameena Shah, Iana Borova, Dylan Langdon, Reema Moussa, Matt Beane, Ting-Hao Huang, Bryan R Routledge, et al. 2021. [FinQA: A Dataset of Numerical Reasoning over Financial Data](https://md.research.elamur.ai/free/en/papers/chen-2021-finqa-dataset-numerical-reasoning-over/). In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Proc…
57. Zhiyu Chen, Shiyang Li, Charese Smiley, Zhiqiang Ma, Sameena Shah, and William Yang Wang. 2022. [ConvFinQA: Exploring the Chain of Numerical Reasoning in Conversational Finance Question Answering](https://md.research.elamur.ai/free/en/papers/chen-2022-convfinqa-exploring-chain-numerical-reasoning/). In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing. 6279–6292.
58. Xiaoxue Cheng, Junyi Li, Wayne Xin Zhao, Hongzhi Zhang, Fuzheng Zhang, Di Zhang, Kun Gai, and Ji-Rong Wen. 2024. Small Agent Can Also Rock! Empowering Small Language Models as Hallucination Detector. arXiv preprint arXiv:2406.11277 (2024). [find](https://scholar.google.com/scholar?q=Xiaoxue+Cheng%2C+Junyi+Li%2C+Wayne+Xin+Zhao%2C+Hongzhi+Zhang%2C+Fuzheng+Zhang%2C+Di+Zhang%2C+Kun+Gai%2C+and+Ji%2DRong+Wen%2E+2024%2E+Small+Agent+Can+Also+Rock%21+Empowering+Small+Language+Models+as+Hallucination+Detector%2E+)
59. Steffi Chern, Zhulin Hu, Yuqing Yang, Ethan Chern, Yuan Guo, Jiahe Jin, Binjie Wang, and Pengfei Liu. 2024. BeHonest: Benchmarking Honesty of Large Language Models. arXiv preprint arXiv:2406.13261 (2024). — [arXiv:2406.13261](https://arxiv.org/abs/2406.13261) [find](https://scholar.google.com/scholar?q=Steffi+Chern%2C+Zhulin+Hu%2C+Yuqing+Yang%2C+Ethan+Chern%2C+Yuan+Guo%2C+Jiahe+Jin%2C+Binjie+Wang%2C+and+Pengfei+Liu%2E+2024%2E+BeHonest%3A+Benchmarking+Honesty+of+Large+Language+Models%2E+arXiv+preprint+arXiv%3A2406%2E13261+%2820)
60. Yew Ken Chia, Pengfei Hong, Lidong Bing, and Soujanya Poria. 2024. [InstructEval: Towards Holistic Evaluation of Instruction-Tuned Large Language Models](https://md.research.elamur.ai/free/en/papers/chia-2023-instructeval-towards-holistic-evaluation-instruction/). In Proceedings of the First edition of the Workshop on the Scaling Behavior of Large Language Models (SCALE-LLM 2024). 35–64.
61. Wei-Lin Chiang, Zhuohan Li, Zi Lin, Ying Sheng, Zhanghao Wu, Hao Zhang, Lianmin Zheng, Siyuan Zhuang, Yonghao Zhuang, Joseph E. Gonzalez, Ion Stoica, and Eric P. Xing. 2023. Vicuna: An Open-Source Chatbot Impressing GPT-4 with 90%* ChatGPT Quality. [https://lmsys.org/blog/2023-03-30-vicuna/](https://lmsys.org/blog/2023-03-30-vicuna/) [↗](https://lmsys.org/blog/2023-03-30-vicuna/)
62. Yae Jee Cho, Luyang Liu, Zheng Xu, Aldi Fahrezi, and Gauri Joshi. 2024. Heterogeneous LoRA for Federated Fine-tuning of On-Device Foundation Models. arXiv:2401.06432 [cs.LG] https://arxiv.org/abs/2401.06432 [↗](https://arxiv.org/abs/2401.06432)
63. Xiaokai Chu, Jiashu Zhao, Lixin Zou, and Dawei Yin. 2022. H-ERNIE: A multi-granularity pre-trained language model for web search. In Proceedings of the 45th International ACM SIGIR conference on research and development in information retrieval. 1478–1489. [find](https://scholar.google.com/scholar?q=Xiaokai+Chu%2C+Jiashu+Zhao%2C+Lixin+Zou%2C+and+Dawei+Yin%2E+2022%2E+H%2DERNIE%3A+A+multi%2Dgranularity+pre%2Dtrained+language+model+for+web+search%2E+In+Proceedings+of+the+45th+International+ACM+SIGIR+conference+on+resea)
64. Hyung Won Chung, Le Hou, Shayne Longpre, Barret Zoph, Yi Tay, William Fedus, Yunxuan Li, Xuezhi Wang, Mostafa Dehghani, Siddhartha Brahma, et al. 2024. Scaling instruction-finetuned language models. Journal of Machine Learning Research 25, 70 (2024), 1–53. [find](https://scholar.google.com/scholar?q=Hyung+Won+Chung%2C+Le+Hou%2C+Shayne+Longpre%2C+Barret+Zoph%2C+Yi+Tay%2C+William+Fedus%2C+Yunxuan+Li%2C+Xuezhi+Wang%2C+Mostafa+Dehghani%2C+Siddhartha+Brahma%2C+et+al%2E+2024%2E+Scaling+instruction%2Dfinetuned+language+models%2E+J)
65. Peter Clark, Isaac Cowhey, Oren Etzioni, Tushar Khot, Ashish Sabharwal, Carissa Schoenick, and Oyvind Tafjord. 2018. [Think you have solved question answering? try arc, the ai2 reasoning challenge](https://md.research.elamur.ai/free/en/papers/clark-2018-think-have-solved-question-answering/). arXiv preprint arXiv:1803.05457 (2018).
66. Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, et al. 2021. [Training verifiers to solve math word problems](https://md.research.elamur.ai/free/en/papers/cobbe-2021-training-verifiers-solve-math-word/). arXiv preprint arXiv:2110.14168 (2021).
67. Together Computer. 2023. [RedPajama: an Open Dataset for Training Large Language Models](https://md.research.elamur.ai/free/en/papers/weber-2024-redpajama-open-dataset-training-large/). [https://github.com/togethercomputer/RedPajama-Data](https://github.com/togethercomputer/RedPajama-Data) [↗](https://github.com/togethercomputer/RedPajama-Data)
68. Mike Conover, Matt Hayes, Ankit Mathur, Jianwei Xie, Jun Wan, Sam Shah, Ali Ghodsi, Patrick Wendell, Matei Zaharia, and Reynold Xin. 2023. Free Dolly: Introducing the World’s First Truly Open Instruction-Tuned LLM. [https://www.databricks.com/blog/2023/04/12/dolly-first-open-commercially-viable-instr…](https://www.databricks.com/blog/2023/04/12/dolly-first-open-commercially-viable-instr…) [↗](https://www.databricks.com/blog/2023/04/12/dolly-first-open-commercially-viable-instr…)
69. Nick Craswell, Bhaskar Mitra, Emine Yilmaz, and Daniel Campos. 2021. Overview of the TREC 2020 deep learning track. CoRR abs/2102.07662 (2021). arXiv:2102.07662 https://arxiv.org/abs/2102.07662 — [arXiv:2102.07662](https://arxiv.org/abs/2102.07662) [↗](https://arxiv.org/abs/2102.07662)
70. Justin Cui, Wei-Lin Chiang, Ion Stoica, and Cho-Jui Hsieh. 2024. [OR-Bench: An Over-Refusal Benchmark for Large Language Models](https://md.research.elamur.ai/free/en/papers/cui-2024-bench-over-refusal-benchmark-large/). arXiv preprint arXiv:2405.20947 (2024).
71. Shiyao Cui, Zhenyu Zhang, Yilong Chen, Wenyuan Zhang, Tianyun Liu, Siqi Wang, and Tingwen Liu. 2023. Fft: Towards harmlessness evaluation and analysis for llms with factuality, fairness, toxicity. arXiv preprint arXiv:2311.18580 (2023). — [arXiv:2311.18580](https://arxiv.org/abs/2311.18580) [find](https://scholar.google.com/scholar?q=Shiyao+Cui%2C+Zhenyu+Zhang%2C+Yilong+Chen%2C+Wenyuan+Zhang%2C+Tianyun+Liu%2C+Siqi+Wang%2C+and+Tingwen+Liu%2E+2023%2E+Fft%3A+Towards+harmlessness+evaluation+and+analysis+for+llms+with+factuality%2C+fairness%2C+toxicity%2E+arX)
72. Luigi Daniele and Suphavadeeprasit. 2023. Amplify-Instruct: Synthetically Generated Diverse Multi-turn Conversations for efficient LLM Training. arXiv preprint arXiv:(coming soon) (2023). [https://huggingface.co/datasets/LDJnr/Capybara](https://huggingface.co/datasets/LDJnr/Capybara) [↗](https://huggingface.co/datasets/LDJnr/Capybara)
73. Tri Dao. [n. d.]. [FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioning](https://md.research.elamur.ai/free/en/papers/dao-2023-flashattention-2-faster-attention-better/). In The Twelfth International Conference on Learning Representations.
74. Tri Dao, Dan Fu, Stefano Ermon, Atri Rudra, and Christopher Ré. 2022. Flashattention: Fast and memory-efficient exact attention with io-awareness. Advances in Neural Information Processing Systems 35 (2022), 16344–16359. [find](https://scholar.google.com/scholar?q=Tri+Dao%2C+Dan+Fu%2C+Stefano+Ermon%2C+Atri+Rudra%2C+and+Christopher+R%C3%A9%2E+2022%2E+Flashattention%3A+Fast+and+memory%2Defficient+exact+attention+with+io%2Dawareness%2E+Advances+in+Neural+Information+Processing+Systems+35+)
75. Tri Dao and Albert Gu. 2024. Transformers are SSMs: Generalized models and efficient algorithms through structured state space duality. arXiv preprint arXiv:2405.21060 (2024). [find](https://scholar.google.com/scholar?q=Tri+Dao+and+Albert+Gu%2E+2024%2E+Transformers+are+SSMs%3A+Generalized+models+and+efficient+algorithms+through+structured+state+space+duality%2E+arXiv+preprint+arXiv%3A2405%2E21060+%282024%29%2E)
76. Rocktim Jyoti Das, Liqun Ma, and Zhiqiang Shen. 2023. Beyond size: How gradients shape pruning decisions in large language models. arXiv preprint arXiv:2311.04902 (2023). — [arXiv:2311.04902](https://arxiv.org/abs/2311.04902) [find](https://scholar.google.com/scholar?q=Rocktim+Jyoti+Das%2C+Liqun+Ma%2C+and+Zhiqiang+Shen%2E+2023%2E+Beyond+size%3A+How+gradients+shape+pruning+decisions+in+large+language+models%2E+arXiv+preprint+arXiv%3A2311%2E04902+%282023%29%2E+%E2%80%94+%5BarXiv%3A2311%2E04902%5D%28https%3A%2F%2F)
77. Anirban Dasgupta, Ravi Kumar, and Tamás Sarlós. 2011. Fast locality-sensitive hashing. In Proceedings of the 17th ACM SIGKDD international conference on Knowledge discovery and data mining. 1073–1081. Manuscript submitted to ACM [find](https://scholar.google.com/scholar?q=Anirban+Dasgupta%2C+Ravi+Kumar%2C+and+Tam%C3%A1s+Sarl%C3%B3s%2E+2011%2E+Fast+locality%2Dsensitive+hashing%2E+In+Proceedings+of+the+17th+ACM+SIGKDD+international+conference+on+Knowledge+discovery+and+data+mining%2E+1073%E2%80%931081%2E)
78. Pieter Delobelle, Ewoenam Kwaku Tokpo, Toon Calders, and Bettina Berendt. 2022. Measuring fairness with biased rulers: A comparative study on bias metrics for pre-trained language models. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguis… [find](https://scholar.google.com/scholar?q=Pieter+Delobelle%2C+Ewoenam+Kwaku+Tokpo%2C+Toon+Calders%2C+and+Bettina+Berendt%2E+2022%2E+Measuring+fairness+with+biased+rulers%3A+A+comparative+study+on+bias+metrics+for+pre%2Dtrained+language+models%2E+In+Proceedin)
79. Yongheng Deng, Ziqing Qiao, Ju Ren, Yang Liu, and Yaoxue Zhang. 2023. Mutual enhancement of large and small language models with cross-silo knowledge transfer. arXiv preprint arXiv:2312.05842 (2023). — [arXiv:2312.05842](https://arxiv.org/abs/2312.05842) [find](https://scholar.google.com/scholar?q=Yongheng+Deng%2C+Ziqing+Qiao%2C+Ju+Ren%2C+Yang+Liu%2C+and+Yaoxue+Zhang%2E+2023%2E+Mutual+enhancement+of+large+and+small+language+models+with+cross%2Dsilo+knowledge+transfer%2E+arXiv+preprint+arXiv%3A2312%2E05842+%282023%29%2E+)
80. Tim Dettmers, Mike Lewis, Younes Belkada, and Luke Zettlemoyer. 2022. GPT3.int8(): 8-bit Matrix Multiplication for Transformers at Scale. In Advances in Neural Information Processing Systems, Alice H. Oh, Alekh Agarwal, Danielle Belgrave, and Kyunghyun Cho (Eds.). [https://openreview.net/forum?id=dX…](https://openreview.net/forum?id=dX…) [↗](https://openreview.net/forum?id=dX…)
81. Tim Dettmers, Artidoro Pagnoni, Ari Holtzman, and Luke Zettlemoyer. 2024. [Qlora: Efficient finetuning of quantized llms](https://md.research.elamur.ai/free/en/papers/dettmers-2023-qlora-efficient-finetuning-quantized-llms/). Advances in Neural Information Processing Systems 36 (2024).
82. Tim Dettmers and Luke Zettlemoyer. 2023. [The case for 4-bit precision: k-bit inference scaling laws](https://md.research.elamur.ai/free/en/papers/dettmers-2022-case-4-bit-precision-k/). In International Conference on Machine Learning. PMLR, 7750–7774.
83. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://md.research.elamur.ai/free/en/papers/devlin-2019-bert-pre-training-deep-bidirectional/). In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies,…
84. Nolan Dey, Gurpreet Gosal, Zhiming Chen, Hemant Khachane, William Marshall, Ribhu Pathria, Marvin Tom, and Joel Hestness. 2023. Cerebras-GPT: Open Compute-Optimal Language Models Trained on the Cerebras Wafer-Scale Cluster. CoRR abs/2304.03208 (2023). — [arXiv:2304.03208](https://arxiv.org/abs/2304.03208) [find](https://scholar.google.com/scholar?q=Nolan+Dey%2C+Gurpreet+Gosal%2C+Zhiming+Chen%2C+Hemant+Khachane%2C+William+Marshall%2C+Ribhu+Pathria%2C+Marvin+Tom%2C+and+Joel+Hestness%2E+2023%2E+Cerebras%2DGPT%3A+Open+Compute%2DOptimal+Language+Models+Trained+on+the+Cerebr)
85. Richard Diehl Martinez, Pietro Lesci, and Paula Buttery. 2024. Tending Towards Stability: Convergence Challenges in Small Language Models. In Findings of the Association for Computational Linguistics: EMNLP 2024, Yaser Al-Onaizan, Mohit Bansal, and Yun-Nung Chen (Eds.). Association for Computational… [find](https://scholar.google.com/scholar?q=Richard+Diehl+Martinez%2C+Pietro+Lesci%2C+and+Paula+Buttery%2E+2024%2E+Tending+Towards+Stability%3A+Convergence+Challenges+in+Small+Language+Models%2E+In+Findings+of+the+Association+for+Computational+Linguistics%3A)
86. Ning Ding, Yulin Chen, Bokai Xu, Yujia Qin, Zhi Zheng, Shengding Hu, Zhiyuan Liu, Maosong Sun, and Bowen Zhou. 2023. [Enhancing chat language models by scaling high-quality instructional conversations](https://md.research.elamur.ai/free/en/papers/ding-2023-enhancing-chat-language-models-scaling/). arXiv preprint arXiv:2305.14233 (2023).
87. Tinghe Ding. 2024. MobileAgent: enhancing mobile control via human-machine interaction and SOP integration. arXiv:2401.04124 [cs.HC] https://arxiv.org/abs/2401.04124 — [arXiv:2401.04124](https://arxiv.org/abs/2401.04124) [↗](https://arxiv.org/abs/2401.04124)
88. Ricardo Dominguez-Olmedo, Moritz Hardt, and Celestine Mendler-Dünner. 2023. Questioning the survey responses of large language models. arXiv preprint arXiv:2306.07951 (2023). — [arXiv:2306.07951](https://arxiv.org/abs/2306.07951) [find](https://scholar.google.com/scholar?q=Ricardo+Dominguez%2DOlmedo%2C+Moritz+Hardt%2C+and+Celestine+Mendler%2DD%C3%BCnner%2E+2023%2E+Questioning+the+survey+responses+of+large+language+models%2E+arXiv+preprint+arXiv%3A2306%2E07951+%282023%29%2E+%E2%80%94+%5BarXiv%3A2306%2E07951%5D%28http)
89. Qian Dong, Yiding Liu, Qingyao Ai, Haitao Li, Shuaiqiang Wang, Yiqun Liu, Dawei Yin, and Shaoping Ma. 2023. I3 retriever: incorporating implicit interaction in pre-trained language models for passage retrieval. In Proceedings of the 32nd ACM International Conference on Information and Knowledge Mana… [find](https://scholar.google.com/scholar?q=Qian+Dong%2C+Yiding+Liu%2C+Qingyao+Ai%2C+Haitao+Li%2C+Shuaiqiang+Wang%2C+Yiqun+Liu%2C+Dawei+Yin%2C+and+Shaoping+Ma%2E+2023%2E+I3+retriever%3A+incorporating+implicit+interaction+in+pre%2Dtrained+language+models+for+passage+)
90. Xin Dong, Yonggan Fu, Shizhe Diao, Wonmin Byeon, Zijia Chen, Ameya Sunil Mahabaleshwarkar, Shih-Yang Liu, Matthijs Van Keirsbilck, Min-Hung Chen, Yoshi Suhara, et al. 2024. [Hymba: A Hybrid-head Architecture for Small Language Models](https://md.research.elamur.ai/free/en/papers/dong-2024-hymba-hybrid-head-architecture-small/). arXiv preprint arXiv:2411.13676 (2024).
91. Jason Xiaotian Dou, Haiyi Mao, Runxue Bao, Paul Pu Liang, Xiaoqing Tan, Shiyi Zhang, Minxue Jia, Pengfei Zhou, and Zhi-Hong Mao. 2023. The Measurement of Knowledge in Knowledge Graphs. In Proceedings of the AAAI 2023 Workshop on Representation Learning for Responsible Human-Centric AI (R2HCAI). Asso… [find](https://scholar.google.com/scholar?q=Jason+Xiaotian+Dou%2C+Haiyi+Mao%2C+Runxue+Bao%2C+Paul+Pu+Liang%2C+Xiaoqing+Tan%2C+Shiyi+Zhang%2C+Minxue+Jia%2C+Pengfei+Zhou%2C+and+Zhi%2DHong+Mao%2E+2023%2E+The+Measurement+of+Knowledge+in+Knowledge+Graphs%2E+In+Proceedings+)
92. Xinrun Du, Zhouliang Yu, Songyang Gao, Ding Pan, Yuyang Cheng, Ziyang Ma, Ruibin Yuan, Xingwei Qu, Jiaheng Liu, Tianyu Zheng, Xinchen Luo, Guorui Zhou, Wenhu Chen, and Ge Zhang. 2024. [Chinese Tiny LLM: Pretraining a Chinese-Centric Large Language Model](https://md.research.elamur.ai/free/en/papers/du-2024-chinese-tiny-llm-pretraining/). arXiv:2404.04167 [cs.CL] [https://arxiv.org/abs…](https://arxiv.org/abs…) [↗](https://arxiv.org/abs…)
93. Zhengxiao Du, Yujie Qian, Xiao Liu, Ming Ding, Jiezhong Qiu, Zhilin Yang, and Jie Tang. 2022. [GLM: General Language Model Pretraining with Autoregressive Blank Infilling](https://md.research.elamur.ai/free/en/papers/du-2022-glm-general-language-model-pretraining/). In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 320–335.
94. Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al. 2024. [The llama 3 herd of models](https://md.research.elamur.ai/free/en/papers/grattafiori-2024-llama-3-herd-models/). arXiv preprint arXiv:2407.21783 (2024).
95. Kazuki Egashira, Mark Vero, Robin Staab, Jingxuan He, and Martin Vechev. 2024. Exploiting LLM Quantization. arXiv preprint arXiv:2405.18137 (2024). — [arXiv:2405.18137](https://arxiv.org/abs/2405.18137) [find](https://scholar.google.com/scholar?q=Kazuki+Egashira%2C+Mark+Vero%2C+Robin+Staab%2C+Jingxuan+He%2C+and+Martin+Vechev%2E+2024%2E+Exploiting+LLM+Quantization%2E+arXiv+preprint+arXiv%3A2405%2E18137+%282024%29%2E+%E2%80%94+%5BarXiv%3A2405%2E18137%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2405%2E1813)
96. Ronen Eldan and Yuanzhi Li. 2023. [Tinystories: How small can language models be and still speak coherent english?](https://md.research.elamur.ai/free/en/papers/eldan-2023-tinystories-small-language-models-still/) arXiv preprint arXiv:2305.07759 (2023).
97. Stefan Elfwing, Eiji Uchibe, and Kenji Doya. 2018. Sigmoid-weighted linear units for neural network function approximation in reinforcement learning. Neural networks 107 (2018), 3–11. [find](https://scholar.google.com/scholar?q=Stefan+Elfwing%2C+Eiji+Uchibe%2C+and+Kenji+Doya%2E+2018%2E+Sigmoid%2Dweighted+linear+units+for+neural+network+function+approximation+in+reinforcement+learning%2E+Neural+networks+107+%282018%29%2C+3%E2%80%9311%2E)
98. David Esiobu, Xiaoqing Tan, Saghar Hosseini, Megan Ung, Yuchen Zhang, Jude Fernandes, Jane Dwivedi-Yu, Eleonora Presani, Adina Williams, and Eric Smith. 2023. ROBBIE: Robust bias evaluation of large generative language models. In Proceedings of the 2023 Conference on Empirical Methods in Natural Lan… [find](https://scholar.google.com/scholar?q=David+Esiobu%2C+Xiaoqing+Tan%2C+Saghar+Hosseini%2C+Megan+Ung%2C+Yuchen+Zhang%2C+Jude+Fernandes%2C+Jane+Dwivedi%2DYu%2C+Eleonora+Presani%2C+Adina+Williams%2C+and+Eric+Smith%2E+2023%2E+ROBBIE%3A+Robust+bias+evaluation+of+large+g)
99. Hugging Face. 2024. SmolVLM - small yet mighty Vision Language Model. [https://huggingface.co/blog/smolvlm](https://huggingface.co/blog/smolvlm) Accessed: 2024-11-26. [↗](https://huggingface.co/blog/smolvlm)
100. William Fedus, Barret Zoph, and Noam Shazeer. 2022. Switch transformers: Scaling to trillion parameter models with simple and efficient sparsity. Journal of Machine Learning Research 23, 120 (2022), 1–39. [find](https://scholar.google.com/scholar?q=William+Fedus%2C+Barret+Zoph%2C+and+Noam+Shazeer%2E+2022%2E+Switch+transformers%3A+Scaling+to+trillion+parameter+models+with+simple+and+efficient+sparsity%2E+Journal+of+Machine+Learning+Research+23%2C+120+%282022%29%2C+1)
101. Shangbin Feng, Weijia Shi, Yuyang Bai, Vidhisha Balachandran, Tianxing He, and Yulia Tsvetkov. 2023. [Knowledge Card: Filling LLMs’ Knowledge Gaps with Plug-in Specialized Language Models](https://md.research.elamur.ai/free/en/papers/feng-2023-knowledge-card-filling-llms-knowledge/). arXiv preprint arXiv:2305.09955 (2023). Manuscript submitted to ACM
102. Elias Frantar and Dan Alistarh. 2023. Sparsegpt: Massive language models can be accurately pruned in one-shot. In International Conference on Machine Learning. PMLR, 10323–10337. [find](https://scholar.google.com/scholar?q=Elias+Frantar+and+Dan+Alistarh%2E+2023%2E+Sparsegpt%3A+Massive+language+models+can+be+accurately+pruned+in+one%2Dshot%2E+In+International+Conference+on+Machine+Learning%2E+PMLR%2C+10323%E2%80%9310337%2E)
103. Elias Frantar, Saleh Ashkboos, Torsten Hoefler, and Dan Alistarh. 2023. GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers. In The Eleventh International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Elias+Frantar%2C+Saleh+Ashkboos%2C+Torsten+Hoefler%2C+and+Dan+Alistarh%2E+2023%2E+GPTQ%3A+Accurate+Post%2DTraining+Quantization+for+Generative+Pre%2Dtrained+Transformers%2E+In+The+Eleventh+International+Conference+on+L)
104. Hao Fu, Yao; Peng and Tushar Khot. 2022. How does GPT Obtain its Ability? Tracing Emergent Abilities of Language Models to their Sources. Yao Fu’s Notion (Dec 2022). [https://yaofu.notion.site/How-does-GPT-Obtain-its-Ability-Tracing-Emergent-Abilities-of-Language-Models-to-their-Sources-b9a57ac0fcf74…](https://yaofu.notion.site/How-does-GPT-Obtain-its-Ability-Tracing-Emergent-Abilities-of-Language-Models-to-their-Sources-b9a57ac0fcf74…) [↗](https://yaofu.notion.site/How-does-GPT-Obtain-its-Ability-Tracing-Emergent-Abilities-of-Language-Models-to-their-Sources-b9a57ac0fcf74…)
105. Yao Fu, Hao Peng, Litu Ou, Ashish Sabharwal, and Tushar Khot. 2023. [Specializing smaller language models towards multi-step reasoning](https://md.research.elamur.ai/free/en/papers/fu-2023-specializing-smaller-language-models-towards/). In International Conference on Machine Learning. PMLR, 10421–10430.
106. Philip Gage. 1994. A new algorithm for data compression. The C Users Journal 12, 2 (1994), 23–38. [find](https://scholar.google.com/scholar?q=Philip+Gage%2E+1994%2E+A+new+algorithm+for+data+compression%2E+The+C+Users+Journal+12%2C+2+%281994%29%2C+23%E2%80%9338%2E)
107. Chongming Gao, Shiqi Wang, Shijun Li, Jiawei Chen, Xiangnan He, Wenqiang Lei, Biao Li, Yuan Zhang, and Peng Jiang. 2023. CIRS: Bursting filter bubbles by counterfactual interactive recommender system. ACM Transactions on Information Systems 42, 1 (2023), 1–27. [find](https://scholar.google.com/scholar?q=Chongming+Gao%2C+Shiqi+Wang%2C+Shijun+Li%2C+Jiawei+Chen%2C+Xiangnan+He%2C+Wenqiang+Lei%2C+Biao+Li%2C+Yuan+Zhang%2C+and+Peng+Jiang%2E+2023%2E+CIRS%3A+Bursting+filter+bubbles+by+counterfactual+interactive+recommender+system%2E)
108. Leo Gao, Stella Biderman, Sid Black, Laurence Golding, Travis Hoppe, Charles Foster, Jason Phang, Horace He, Anish Thite, Noa Nabeshima, et al. 2020. The Pile: An 800GB dataset of diverse text for language modeling. arXiv preprint arXiv:2101.00027 (2020). [find](https://scholar.google.com/scholar?q=Leo+Gao%2C+Stella+Biderman%2C+Sid+Black%2C+Laurence+Golding%2C+Travis+Hoppe%2C+Charles+Foster%2C+Jason+Phang%2C+Horace+He%2C+Anish+Thite%2C+Noa+Nabeshima%2C+et+al%2E+2020%2E+The+Pile%3A+An+800GB+dataset+of+diverse+text+for+lan)
109. Luyu Gao and Jamie Callan. 2022. [Unsupervised Corpus Aware Language Model Pre-training for Dense Passage Retrieval](https://md.research.elamur.ai/free/en/papers/gao-2022-unsupervised-corpus-aware-language-model/). In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2843–2853.
110. Shangqian Gao, Chi-Heng Lin, Ting Hua, Zheng Tang, Yilin Shen, Hongxia Jin, and Yen-Chang Hsu. 2024. DISP-LLM: Dimension-Independent Structural Pruning for Large Language Models. In The Thirty-eighth Annual Conference on Neural Information Processing Systems. [https://openreview.net/forum?id=YxaY6tH…](https://openreview.net/forum?id=YxaY6tH…) [↗](https://openreview.net/forum?id=YxaY6tH…)
111. Suyu Ge, Yunan Zhang, Liyuan Liu, Minjia Zhang, Jiawei Han, and Jianfeng Gao. 2024. [Model Tells You What to Discard: Adaptive KV Cache Compression for LLMs](https://md.research.elamur.ai/free/en/papers/ge-2023-model-tells-discard-adaptive-kv/). In The Twelfth International Conference on Learning Representations. [https://openreview.net/forum?id=uNrFpDPMyo](https://openreview.net/forum?id=uNrFpDPMyo) [↗](https://openreview.net/forum?id=uNrFpDPMyo)
112. Alex Gichamba, Tewodros Kederalah Idris, Brian Ebiyau, Eric Nyberg, and Teruko Mitamura. 2024. ColBERT Retrieval and Ensemble Response Scoring for Language Model Question Answering. arXiv:2408.10808 [cs.CL] https://arxiv.org/abs/2408.10808 — [arXiv:2408.10808](https://arxiv.org/abs/2408.10808) [↗](https://arxiv.org/abs/2408.10808)
113. Karan Goel. 2024. The On-Device Intelligence Update. [https://www.cartesia.ai/blog/on-device](https://www.cartesia.ai/blog/on-device) [↗](https://www.cartesia.ai/blog/on-device)
114. Aaron Gokaslan, Vanya Cohen, Ellie Pavlick, and Stefanie Tellex. 2019. Openwebtext corpus. [find](https://scholar.google.com/scholar?q=Aaron+Gokaslan%2C+Vanya+Cohen%2C+Ellie+Pavlick%2C+and+Stefanie+Tellex%2E+2019%2E+Openwebtext+corpus%2E)
115. Shreya Goyal, Sumanth Doddapaneni, Mitesh M Khapra, and Balaraman Ravindran. 2023. A survey of adversarial defenses and robustness in nlp. Comput. Surveys 55, 14s (2023), 1–39. [find](https://scholar.google.com/scholar?q=Shreya+Goyal%2C+Sumanth+Doddapaneni%2C+Mitesh+M+Khapra%2C+and+Balaraman+Ravindran%2E+2023%2E+A+survey+of+adversarial+defenses+and+robustness+in+nlp%2E+Comput%2E+Surveys+55%2C+14s+%282023%29%2C+1%E2%80%9339%2E)
116. Dirk Groeneveld, Iz Beltagy, Pete Walsh, Akshita Bhagia, Rodney Kinney, Oyvind Tafjord, Ananya Harsh Jha, Hamish Ivison, Ian Magnusson, Yizhong Wang, et al. 2024. Olmo: Accelerating the science of language models. arXiv preprint arXiv:2402.00838 (2024). [find](https://scholar.google.com/scholar?q=Dirk+Groeneveld%2C+Iz+Beltagy%2C+Pete+Walsh%2C+Akshita+Bhagia%2C+Rodney+Kinney%2C+Oyvind+Tafjord%2C+Ananya+Harsh+Jha%2C+Hamish+Ivison%2C+Ian+Magnusson%2C+Yizhong+Wang%2C+et+al%2E+2024%2E+Olmo%3A+Accelerating+the+science+of+lan)
117. Albert Gu and Tri Dao. 2023. Mamba: Linear-time sequence modeling with selective state spaces. arXiv preprint arXiv:2312.00752 (2023). [find](https://scholar.google.com/scholar?q=Albert+Gu+and+Tri+Dao%2E+2023%2E+Mamba%3A+Linear%2Dtime+sequence+modeling+with+selective+state+spaces%2E+arXiv+preprint+arXiv%3A2312%2E00752+%282023%29%2E)
118. Naibin Gu, Peng Fu, Xiyu Liu, Bowen Shen, Zheng Lin, and Weiping Wang. 2024. Light-PEFT: Lightening Parameter-Efficient Fine-Tuning via Early Pruning. In Findings of the Association for Computational Linguistics: ACL 2024, Lun-Wei Ku, Andre Martins, and Vivek Srikumar (Eds.). Association for Computa… [find](https://scholar.google.com/scholar?q=Naibin+Gu%2C+Peng+Fu%2C+Xiyu+Liu%2C+Bowen+Shen%2C+Zheng+Lin%2C+and+Weiping+Wang%2E+2024%2E+Light%2DPEFT%3A+Lightening+Parameter%2DEfficient+Fine%2DTuning+via+Early+Pruning%2E+In+Findings+of+the+Association+for+Computational+)
119. Yuxian Gu, Li Dong, Furu Wei, and Minlie Huang. 2024. MiniLLM: Knowledge distillation of large language models. In The Twelfth International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Yuxian+Gu%2C+Li+Dong%2C+Furu+Wei%2C+and+Minlie+Huang%2E+2024%2E+MiniLLM%3A+Knowledge+distillation+of+large+language+models%2E+In+The+Twelfth+International+Conference+on+Learning+Representations%2E)
120. Suriya Gunasekar, Yi Zhang, Jyoti Aneja, Caio César Teodoro Mendes, Allie Del Giorno, Sivakanth Gopi, Mojan Javaheripi, Piero Kauffmann, Gustavo de Rosa, Olli Saarikivi, Adil Salim, Shital Shah, Harkirat Singh Behl, Xin Wang, Sébastien Bubeck, Ronen Eldan, Adam Tauman Kalai, Yin Tat Lee, and Yuanzhi… [find](https://scholar.google.com/scholar?q=Suriya+Gunasekar%2C+Yi+Zhang%2C+Jyoti+Aneja%2C+Caio+C%C3%A9sar+Teodoro+Mendes%2C+Allie+Del+Giorno%2C+Sivakanth+Gopi%2C+Mojan+Javaheripi%2C+Piero+Kauffmann%2C+Gustavo+de+Rosa%2C+Olli+Saarikivi%2C+Adil+Salim%2C+Shital+Shah%2C+Harki)
121. Daya Guo, Qihao Zhu, Dejian Yang, Zhenda Xie, Kai Dong, Wentao Zhang, Guanting Chen, Xiao Bi, Yu Wu, YK Li, et al. 2024. DeepSeek-Coder: When the Large Language Model Meets Programming–The Rise of Code Intelligence. arXiv preprint arXiv:2401.14196 (2024). [find](https://scholar.google.com/scholar?q=Daya+Guo%2C+Qihao+Zhu%2C+Dejian+Yang%2C+Zhenda+Xie%2C+Kai+Dong%2C+Wentao+Zhang%2C+Guanting+Chen%2C+Xiao+Bi%2C+Yu+Wu%2C+YK+Li%2C+et+al%2E+2024%2E+DeepSeek%2DCoder%3A+When+the+Large+Language+Model+Meets+Programming%E2%80%93The+Rise+of+Cod)
122. Jinyang Guo, Jianyu Wu, Zining Wang, Jiaheng Liu, Ge Yang, Yifu Ding, Ruihao Gong, Haotong Qin, and Xianglong Liu. 2024. Compressing large language models by joint sparsification and quantization. In Forty-first International Conference on Machine Learning. [find](https://scholar.google.com/scholar?q=Jinyang+Guo%2C+Jianyu+Wu%2C+Zining+Wang%2C+Jiaheng+Liu%2C+Ge+Yang%2C+Yifu+Ding%2C+Ruihao+Gong%2C+Haotong+Qin%2C+and+Xianglong+Liu%2E+2024%2E+Compressing+large+language+models+by+joint+sparsification+and+quantization%2E+In+)
123. Shangwei Guo, Chunlong Xie, Jiwei Li, Lingjuan Lyu, and Tianwei Zhang. 2022. Threats to pre-trained language models: Survey and taxonomy. arXiv preprint arXiv:2202.06862 (2022). — [arXiv:2202.06862](https://arxiv.org/abs/2202.06862) [find](https://scholar.google.com/scholar?q=Shangwei+Guo%2C+Chunlong+Xie%2C+Jiwei+Li%2C+Lingjuan+Lyu%2C+and+Tianwei+Zhang%2E+2022%2E+Threats+to+pre%2Dtrained+language+models%3A+Survey+and+taxonomy%2E+arXiv+preprint+arXiv%3A2202%2E06862+%282022%29%2E+%E2%80%94+%5BarXiv%3A2202%2E06862%5D%28h)
124. Song Guo, Jiahang Xu, Li Lyna Zhang, and Mao Yang. 2023. Compresso: Structured pruning with collaborative prompting learns compact large language models. arXiv preprint arXiv:2310.05015 (2023). — [arXiv:2310.05015](https://arxiv.org/abs/2310.05015) [find](https://scholar.google.com/scholar?q=Song+Guo%2C+Jiahang+Xu%2C+Li+Lyna+Zhang%2C+and+Mao+Yang%2E+2023%2E+Compresso%3A+Structured+pruning+with+collaborative+prompting+learns+compact+large+language+models%2E+arXiv+preprint+arXiv%3A2310%2E05015+%282023%29%2E+%E2%80%94+%5BarX)
125. Zhen Guo, Peiqi Wang, Yanwei Wang, and Shangdi Yu. 2023. Improving Small Language Models on PubMedQA via Generative Data Augmentation. arXiv:2305.07804 [cs.CL] https://arxiv.org/abs/2305.07804 [↗](https://arxiv.org/abs/2305.07804)
126. Song Han, Jeff Pool, John Tran, and William Dally. 2015. Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015). [find](https://scholar.google.com/scholar?q=Song+Han%2C+Jeff+Pool%2C+John+Tran%2C+and+William+Dally%2E+2015%2E+Learning+both+weights+and+connections+for+efficient+neural+network%2E+Advances+in+neural+information+processing+systems+28+%282015%29%2E)
127. Zixu Hao, Huiqiang Jiang, Shiqi Jiang, Ju Ren, and Ting Cao. 2024. Hybrid SLM and LLM for Edge-Cloud Collaborative Inference. In Proceedings of the Workshop on Edge and Mobile Foundation Models. 36–41. [find](https://scholar.google.com/scholar?q=Zixu+Hao%2C+Huiqiang+Jiang%2C+Shiqi+Jiang%2C+Ju+Ren%2C+and+Ting+Cao%2E+2024%2E+Hybrid+SLM+and+LLM+for+Edge%2DCloud+Collaborative+Inference%2E+In+Proceedings+of+the+Workshop+on+Edge+and+Mobile+Foundation+Models%2E+36%E2%80%9341)
128. Tim Hartill, Diana Benavides-Prado, Michael Witbrock, and Patricia J. Riddle. 2023. Answering Unseen Questions With Smaller Language Models Using Rationale Generation and Dense Retrieval. arXiv:2308.04711 [cs.CL] https://arxiv.org/abs/2308.04711 — [arXiv:2308.04711](https://arxiv.org/abs/2308.04711) [↗](https://arxiv.org/abs/2308.04711)
129. Tim Hartill, Neset Tan, Michael Witbrock, and Patricia J. Riddle. 2023. Teaching Smaller Language Models To Generalise To Unseen Compositional Questions. arXiv:2308.00946 [cs.CL] https://arxiv.org/abs/2308.00946 Manuscript submitted to ACM — [arXiv:2308.00946](https://arxiv.org/abs/2308.00946) [↗](https://arxiv.org/abs/2308.00946)
130. Kai He, Rui Mao, Qika Lin, Yucheng Ruan, Xiang Lan, Mengling Feng, and Erik Cambria. 2023. [A survey of large language models for healthcare: from data, technology, and applications to accountability and ethics](https://md.research.elamur.ai/free/en/papers/he-2023-survey-large-language-models-healthcare/). arXiv preprint arXiv:2310.05694 (2023).
131. Pengcheng He, Jianfeng Gao, and Weizhu Chen. 2023. [DeBERTaV3: Improving DeBERTa using ELECTRA-Style Pre-Training with Gradient-Disentangled Embedding Sharing](https://md.research.elamur.ai/free/en/papers/he-2021-debertav3-improving-deberta-using-electra/). In The Eleventh International Conference on Learning Representations. [https://openreview.net/forum?id=sE7-XhLxHA](https://openreview.net/forum?id=sE7-XhLxHA) [↗](https://openreview.net/forum?id=sE7-XhLxHA)
132. Pengcheng He, Xiaodong Liu, Jianfeng Gao, and Weizhu Chen. 2020. [Deberta: Decoding-enhanced bert with disentangled attention](https://md.research.elamur.ai/free/en/papers/he-2020-deberta-decoding-enhanced-bert-disentangled/). arXiv preprint arXiv:2006.03654 (2020).
133. Narges Heidari, Parham Moradi, and Abbas Koochari. 2022. An attention-based deep learning method for solving the cold-start and sparsity issues of recommender systems. Knowledge-Based Systems 256 (2022), 109835. [find](https://scholar.google.com/scholar?q=Narges+Heidari%2C+Parham+Moradi%2C+and+Abbas+Koochari%2E+2022%2E+An+attention%2Dbased+deep+learning+method+for+solving+the+cold%2Dstart+and+sparsity+issues+of+recommender+systems%2E+Knowledge%2DBased+Systems+256+%28202)
134. Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. 2021. [Measuring Massive Multitask Language Understanding](https://md.research.elamur.ai/free/en/papers/hendrycks-2020-measuring-massive-multitask-language-understanding/). In International Conference on Learning Representations. [https://openreview.net/forum?id=d7KBjmI3GmQ](https://openreview.net/forum?id=d7KBjmI3GmQ) [↗](https://openreview.net/forum?id=d7KBjmI3GmQ)
135. Dan Hendrycks and Kevin Gimpel. 2016. Gaussian error linear units (gelus). arXiv preprint arXiv:1606.08415 (2016). [find](https://scholar.google.com/scholar?q=Dan+Hendrycks+and+Kevin+Gimpel%2E+2016%2E+Gaussian+error+linear+units+%28gelus%29%2E+arXiv+preprint+arXiv%3A1606%2E08415+%282016%29%2E)
136. Geoffrey Hinton, Oriol Vinyals, and Jeff Dean. 2015. [Distilling the knowledge in a neural network](https://md.research.elamur.ai/free/en/papers/hinton-2015-distilling-knowledge-neural-network/). arXiv preprint arXiv:1503.02531 (2015).
137. Sepp Hochreiter and Jürgen Schmidhuber. 1996. LSTM can solve hard long time lag problems. Advances in neural information processing systems 9 (1996). [find](https://scholar.google.com/scholar?q=Sepp+Hochreiter+and+J%C3%BCrgen+Schmidhuber%2E+1996%2E+LSTM+can+solve+hard+long+time+lag+problems%2E+Advances+in+neural+information+processing+systems+9+%281996%29%2E)
138. Junyuan Hong, Jinhao Duan, Chenhui Zhang, Zhangheng Li, Chulin Xie, Kelsey Lieberman, James Diffenderfer, Brian R. Bartoldson, Ajay Kumar Jaiswal, Kaidi Xu, Bhavya Kailkhura, Dan Hendrycks, Dawn Song, Zhangyang Wang, and Bo Li. 2024. Decoding Compressed Trust: Scrutinizing the Trustworthiness of Eff… [find](https://scholar.google.com/scholar?q=Junyuan+Hong%2C+Jinhao+Duan%2C+Chenhui+Zhang%2C+Zhangheng+Li%2C+Chulin+Xie%2C+Kelsey+Lieberman%2C+James+Diffenderfer%2C+Brian+R%2E+Bartoldson%2C+Ajay+Kumar+Jaiswal%2C+Kaidi+Xu%2C+Bhavya+Kailkhura%2C+Dan+Hendrycks%2C+Dawn+Song%2C)
139. Yutong Meng Yuhao Wang Hongcheng Liu, Yusheng Liao. 2023. XieZhi: Chinese Law Large Language Model. [https://github.com/LiuHC0428/LAW_GPT](https://github.com/LiuHC0428/LAW_GPT). [↗](https://github.com/LiuHC0428/LAW_GPT)
140. Neil Houlsby, Andrei Giurgiu, Stanislaw Jastrzebski, Bruna Morrone, Quentin De Laroussilhe, Andrea Gesmundo, Mona Attariyan, and Sylvain Gelly. 2019. Parameter-efficient transfer learning for NLP. In International Conference on Machine Learning. PMLR, 2790–2799. [find](https://scholar.google.com/scholar?q=Neil+Houlsby%2C+Andrei+Giurgiu%2C+Stanislaw+Jastrzebski%2C+Bruna+Morrone%2C+Quentin+De+Laroussilhe%2C+Andrea+Gesmundo%2C+Mona+Attariyan%2C+and+Sylvain+Gelly%2E+2019%2E+Parameter%2Defficient+transfer+learning+for+NLP%2E+In+)
141. Cheng-Yu Hsieh, Chun-Liang Li, Chih-kuan Yeh, Hootan Nakhost, Yasuhisa Fujii, Alex Ratner, Ranjay Krishna, Chen-Yu Lee, and Tomas Pfister. 2023. [Distilling Step-by-Step! Outperforming Larger Language Models with Less Training Data and Smaller Model Sizes](https://md.research.elamur.ai/free/en/papers/hsieh-2023-distilling-step-step-outperforming-larger/). In Findings of the Association for Computati…
142. Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2021. [Lora: Low-rank adaptation of large language models](https://md.research.elamur.ai/free/en/papers/hu-2021-lora-low-rank-adaptation-large/). arXiv preprint arXiv:2106.09685 (2021).
143. Shengding Hu, Yuge Tu, Xu Han, Chaoqun He, Ganqu Cui, Xiang Long, Zhi Zheng, Yewei Fang, Yuxiang Huang, Weilin Zhao, et al. 2024. [Minicpm: Unveiling the potential of small language models with scalable training strategies](https://md.research.elamur.ai/free/en/papers/hu-2024-minicpm-unveiling-potential-small-language/). arXiv preprint arXiv:2404.06395 (2024).
144. Xing Hu, Yuan Chen, Dawei Yang, Sifan Zhou, Zhihang Yuan, Jiangyong Yu, and Chen Xu. 2024. I-LLM: Efficient Integer-Only Inference for Fully-Quantized Low-Bit Large Language Models. arXiv preprint arXiv:2405.17849 (2024). — [arXiv:2405.17849](https://arxiv.org/abs/2405.17849) [find](https://scholar.google.com/scholar?q=Xing+Hu%2C+Yuan+Chen%2C+Dawei+Yang%2C+Sifan+Zhou%2C+Zhihang+Yuan%2C+Jiangyong+Yu%2C+and+Chen+Xu%2E+2024%2E+I%2DLLM%3A+Efficient+Integer%2DOnly+Inference+for+Fully%2DQuantized+Low%2DBit+Large+Language+Models%2E+arXiv+preprint+arX)
145. Jiaxin Huang, Shixiang Shane Gu, Le Hou, Yuexin Wu, Xuezhi Wang, Hongkun Yu, and Jiawei Han. [n. d.]. [Large Language Models Can Self-Improve](https://md.research.elamur.ai/free/en/papers/huang-2023-large-language-models-can-self/). In The 2023 Conference on Empirical Methods in Natural Language Processing.
146. Lei Huang, Weijiang Yu, Weitao Ma, Weihong Zhong, Zhangyin Feng, Haotian Wang, Qianglong Chen, Weihua Peng, Xiaocheng Feng, Bing Qin, et al. 2023. [A survey on hallucination in large language models: Principles, taxonomy, challenges, and open questions](https://md.research.elamur.ai/free/en/papers/huang-2023-survey-hallucination-large-language-models/). arXiv preprint arXiv:2311.05232 (2023).
147. Wei Huang, Yangdong Liu, Haotong Qin, Ying Li, Shiming Zhang, Xianglong Liu, Michele Magno, and Xiaojuan Qi. 2024. Billm: Pushing the limit of post-training quantization for llms. arXiv preprint arXiv:2402.04291 (2024). [find](https://scholar.google.com/scholar?q=Wei+Huang%2C+Yangdong+Liu%2C+Haotong+Qin%2C+Ying+Li%2C+Shiming+Zhang%2C+Xianglong+Liu%2C+Michele+Magno%2C+and+Xiaojuan+Qi%2E+2024%2E+Billm%3A+Pushing+the+limit+of+post%2Dtraining+quantization+for+llms%2E+arXiv+preprint+arXiv)
148. Wenyu Huang, Guancheng Zhou, Hongru Wang, Pavlos Vougiouklis, Mirella Lapata, and Jeff Z. Pan. 2024. Less is More: Making Smaller Language Models Competent Subgraph Retrievers for Multi-hop KGQA. In Findings of the Association for Computational Linguistics: EMNLP 2024, Yaser Al-Onaizan, Mohit Bansal… [find](https://scholar.google.com/scholar?q=Wenyu+Huang%2C+Guancheng+Zhou%2C+Hongru+Wang%2C+Pavlos+Vougiouklis%2C+Mirella+Lapata%2C+and+Jeff+Z%2E+Pan%2E+2024%2E+Less+is+More%3A+Making+Smaller+Language+Models+Competent+Subgraph+Retrievers+for+Multi%2Dhop+KGQA%2E+In+F)
149. Yuzhen Huang, Yuzhuo Bai, Zhihao Zhu, Junlei Zhang, Jinghan Zhang, Tangjun Su, Junteng Liu, Chuancheng Lv, Yikai Zhang, Yao Fu, et al. 2024. [C-eval: A multi-level multi-discipline chinese evaluation suite for foundation models](https://md.research.elamur.ai/free/en/papers/huang-2023-c-eval-multi-level-multi/). Advances in Neural Information Processing Systems 36 (2024).
150. Yangsibo Huang, Samyak Gupta, Mengzhou Xia, Kai Li, and Danqi Chen. 2023. [Catastrophic Jailbreak of Open-source LLMs via Exploiting Generation](https://md.research.elamur.ai/free/en/papers/huang-2023-catastrophic-jailbreak-open-source-llms/). arXiv preprint arXiv:2310.06987 (2023).
151. Yuheng Huang, Jiayang Song, Zhijie Wang, Shengming Zhao, Huaming Chen, Felix Juefei-Xu, and Lei Ma. 2023. Look before you leap: An exploratory study of uncertainty measurement for large language models. arXiv preprint arXiv:2307.10236 (2023). — [arXiv:2307.10236](https://arxiv.org/abs/2307.10236) [find](https://scholar.google.com/scholar?q=Yuheng+Huang%2C+Jiayang+Song%2C+Zhijie+Wang%2C+Shengming+Zhao%2C+Huaming+Chen%2C+Felix+Juefei%2DXu%2C+and+Lei+Ma%2E+2023%2E+Look+before+you+leap%3A+An+exploratory+study+of+uncertainty+measurement+for+large+language+model)
152. Samuel Humeau, Kurt Shuster, Marie-Anne Lachaux, and Jason Weston. [n. d.]. Poly-encoders: Architectures and Pre-training Strategies for Fast and Accurate Multi-sentence Scoring. In International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Samuel+Humeau%2C+Kurt+Shuster%2C+Marie%2DAnne+Lachaux%2C+and+Jason+Weston%2E+%5Bn%2E+d%2E%5D%2E+Poly%2Dencoders%3A+Architectures+and+Pre%2Dtraining+Strategies+for+Fast+and+Accurate+Multi%2Dsentence+Scoring%2E+In+International+Conf)
153. Hakan Inan, Kartikeya Upasani, Jianfeng Chi, Rashi Rungta, Krithika Iyer, Yuning Mao, Michael Tontchev, Qing Hu, Brian Fuller, Davide Testuggine, et al. 2023. [Llama guard: Llm-based input-output safeguard for human-ai conversations](https://md.research.elamur.ai/free/en/papers/inan-2023-llama-guard-llm-based-input/). arXiv preprint arXiv:2312.06674 (2023).
154. Robert A Jacobs, Michael I Jordan, Steven J Nowlan, and Geoffrey E Hinton. 1991. Adaptive mixtures of local experts. Neural computation 3, 1 (1991), 79–87. Manuscript submitted to ACM [find](https://scholar.google.com/scholar?q=Robert+A+Jacobs%2C+Michael+I+Jordan%2C+Steven+J+Nowlan%2C+and+Geoffrey+E+Hinton%2E+1991%2E+Adaptive+mixtures+of+local+experts%2E+Neural+computation+3%2C+1+%281991%29%2C+79%E2%80%9387%2E+Manuscript+submitted+to+ACM)
155. Mojan Javaheripi, Sébastien Bubeck, Marah Abdin, Jyoti Aneja, Sebastien Bubeck, Caio César Teodoro Mendes, Weizhu Chen, Allie Del Giorno, Ronen Eldan, Sivakanth Gopi, et al. 2023. Phi-2: The surprising power of small language models. Microsoft Research Blog (2023). [find](https://scholar.google.com/scholar?q=Mojan+Javaheripi%2C+S%C3%A9bastien+Bubeck%2C+Marah+Abdin%2C+Jyoti+Aneja%2C+Sebastien+Bubeck%2C+Caio+C%C3%A9sar+Teodoro+Mendes%2C+Weizhu+Chen%2C+Allie+Del+Giorno%2C+Ronen+Eldan%2C+Sivakanth+Gopi%2C+et+al%2E+2023%2E+Phi%2D2%3A+The+surprisin)
156. Soyeong Jeong, Jinheon Baek, Sukmin Cho, Sung Hwang, and Jong C Park. 2023. Test-Time Self-Adaptive Small Language Models for Question Answering. In Findings of the Association for Computational Linguistics: EMNLP 2023. 15459–15469. [find](https://scholar.google.com/scholar?q=Soyeong+Jeong%2C+Jinheon+Baek%2C+Sukmin+Cho%2C+Sung+Hwang%2C+and+Jong+C+Park%2E+2023%2E+Test%2DTime+Self%2DAdaptive+Small+Language+Models+for+Question+Answering%2E+In+Findings+of+the+Association+for+Computational+Lingu)
157. Ananya Harsh Jha, Tom Sherborne, Evan Pete Walsh, Dirk Groeneveld, Emma Strubell, and Iz Beltagy. 2024. Just CHOP: Embarrassingly Simple LLM Compression. arXiv:2305.14864 [cs.CL] https://arxiv.org/abs/2305.14864 — [arXiv:2305.14864](https://arxiv.org/abs/2305.14864) [↗](https://arxiv.org/abs/2305.14864)
158. Yixin Ji, Yang Xiang, Juntao Li, Wei Chen, Zhongyi Liu, Kehai Chen, and Min Zhang. 2024. Feature-based Low-Rank Compression of Large Language Models via Bayesian Optimization. arXiv preprint arXiv:2405.10616 (2024). — [arXiv:2405.10616](https://arxiv.org/abs/2405.10616) [find](https://scholar.google.com/scholar?q=Yixin+Ji%2C+Yang+Xiang%2C+Juntao+Li%2C+Wei+Chen%2C+Zhongyi+Liu%2C+Kehai+Chen%2C+and+Min+Zhang%2E+2024%2E+Feature%2Dbased+Low%2DRank+Compression+of+Large+Language+Models+via+Bayesian+Optimization%2E+arXiv+preprint+arXiv%3A240)
159. Ziwei Ji, Tiezheng Yu, Yan Xu, Nayeon Lee, Etsuko Ishii, and Pascale Fung. 2023. [Towards mitigating LLM hallucination via self reflection](https://md.research.elamur.ai/free/en/papers/ji-2023-mitigating-llm-hallucination-self-reflection/). In Findings of the Association for Computational Linguistics: EMNLP 2023. 1827–1843.
160. Albert Q Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, et al. 2023. Mistral 7B. arXiv preprint arXiv:2310.06825 (2023). [find](https://scholar.google.com/scholar?q=Albert+Q+Jiang%2C+Alexandre+Sablayrolles%2C+Arthur+Mensch%2C+Chris+Bamford%2C+Devendra+Singh+Chaplot%2C+Diego+de+las+Casas%2C+Florian+Bressand%2C+Gianna+Lengyel%2C+Guillaume+Lample%2C+Lucile+Saulnier%2C+et+al%2E+2023%2E+Mist)
161. Albert Q Jiang, Alexandre Sablayrolles, Antoine Roux, Arthur Mensch, Blanche Savary, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Emma Bou Hanna, Florian Bressand, et al. 2024. Mixtral of experts. arXiv preprint arXiv:2401.04088 (2024). [find](https://scholar.google.com/scholar?q=Albert+Q+Jiang%2C+Alexandre+Sablayrolles%2C+Antoine+Roux%2C+Arthur+Mensch%2C+Blanche+Savary%2C+Chris+Bamford%2C+Devendra+Singh+Chaplot%2C+Diego+de+las+Casas%2C+Emma+Bou+Hanna%2C+Florian+Bressand%2C+et+al%2E+2024%2E+Mixtral+o)
162. Huiqiang Jiang, Qianhui Wu, Xufang Luo, Dongsheng Li, Chin-Yew Lin, Yuqing Yang, and Lili Qiu. 2024. [LongLLMLingua: Accelerating and Enhancing LLMs in Long Context Scenarios via Prompt Compression](https://md.research.elamur.ai/free/en/papers/jiang-2023-longllmlingua-accelerating-enhancing-llms-long/). In ICLR 2024 Workshop on Mathematical and Empirical Understanding of Foundation Models. https://openre…
163. Qiao Jin, Bhuwan Dhingra, Zhengping Liu, William Cohen, and Xinghua Lu. 2019. [PubMedQA: A Dataset for Biomedical Research Question Answering](https://md.research.elamur.ai/free/en/papers/jin-2019-pubmedqa-dataset-biomedical-research-question/). In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Proce…
164. Rudolph Emil Kalman. 1960. A new approach to linear filtering and prediction problems. (1960). [find](https://scholar.google.com/scholar?q=Rudolph+Emil+Kalman%2E+1960%2E+A+new+approach+to+linear+filtering+and+prediction+problems%2E+%281960%29%2E)
165. Hao Kang, Qingru Zhang, Souvik Kundu, Geonhwa Jeong, Zaoxing Liu, Tushar Krishna, and Tuo Zhao. 2024. Gear: An efficient kv cache compression recipefor near-lossless generative inference of llm. arXiv preprint arXiv:2403.05527 (2024). — [arXiv:2403.05527](https://arxiv.org/abs/2403.05527) [find](https://scholar.google.com/scholar?q=Hao+Kang%2C+Qingru+Zhang%2C+Souvik+Kundu%2C+Geonhwa+Jeong%2C+Zaoxing+Liu%2C+Tushar+Krishna%2C+and+Tuo+Zhao%2E+2024%2E+Gear%3A+An+efficient+kv+cache+compression+recipefor+near%2Dlossless+generative+inference+of+llm%2E+arXiv)
166. Jared Kaplan, Sam McCandlish, Tom Henighan, Tom B Brown, Benjamin Chess, Rewon Child, Scott Gray, Alec Radford, Jeffrey Wu, and Dario Amodei. 2020. [Scaling laws for neural language models](https://md.research.elamur.ai/free/en/papers/kaplan-2020-scaling-laws-neural-language-models/). arXiv preprint arXiv:2001.08361 (2020).
167. Omar Khattab, Arnav Singhvi, Paridhi Maheshwari, Zhiyuan Zhang, Keshav Santhanam, Sri Vardhamanan, Saiful Haq, Ashutosh Sharma, Thomas T Joshi, Hanna Moazam, et al. 2023. [Dspy: Compiling declarative language model calls into self-improving pipelines](https://md.research.elamur.ai/free/en/papers/khattab-2023-dspy-compiling-declarative-language-model/). arXiv preprint arXiv:2310.03714 (2023).
168. Jeonghoon Kim, Jung Hyun Lee, Sungdong Kim, Joonsuk Park, Kang Min Yoo, Se Jung Kwon, and Dongsoo Lee. 2024. Memory-efficient fine-tuning of compressed large language models via sub-4-bit integer quantization. Advances in Neural Information Processing Systems 36 (2024). [find](https://scholar.google.com/scholar?q=Jeonghoon+Kim%2C+Jung+Hyun+Lee%2C+Sungdong+Kim%2C+Joonsuk+Park%2C+Kang+Min+Yoo%2C+Se+Jung+Kwon%2C+and+Dongsoo+Lee%2E+2024%2E+Memory%2Defficient+fine%2Dtuning+of+compressed+large+language+models+via+sub%2D4%2Dbit+integer+quan)
169. Minsoo Kim, Sihwa Lee, Janghwan Lee, Sukjin Hong, Du-Seong Chang, Wonyong Sung, and Jungwook Choi. 2024. Token-scaled logit distillation for ternary weight generative language models. Advances in Neural Information Processing Systems 36 (2024). [find](https://scholar.google.com/scholar?q=Minsoo+Kim%2C+Sihwa+Lee%2C+Janghwan+Lee%2C+Sukjin+Hong%2C+Du%2DSeong+Chang%2C+Wonyong+Sung%2C+and+Jungwook+Choi%2E+2024%2E+Token%2Dscaled+logit+distillation+for+ternary+weight+generative+language+models%2E+Advances+in+Neur)
170. Sehoon Kim, Coleman Hooper, Amir Gholami, Zhen Dong, Xiuyu Li, Sheng Shen, Michael W Mahoney, and Kurt Keutzer. 2023. [Squeezellm: Dense-and-sparse quantization](https://md.research.elamur.ai/free/en/papers/kim-2023-squeezellm-dense-sparse-quantization/). arXiv preprint arXiv:2306.07629 (2023).
171. Yoon Kim and Alexander M Rush. 2016. [Sequence-Level Knowledge Distillation](https://md.research.elamur.ai/free/en/papers/kim-2016-sequence-level-knowledge-distillation/). In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing. 1317–1327.
172. Young Jin Kim, Raffy Fahim, and Hany Hassan Awadalla. 2023. Mixture of Quantized Experts (MoQE): Complementary Effect of Low-bit Quantization and Robustness. arXiv preprint arXiv:2310.02410 (2023). [find](https://scholar.google.com/scholar?q=Young+Jin+Kim%2C+Raffy+Fahim%2C+and+Hany+Hassan+Awadalla%2E+2023%2E+Mixture+of+Quantized+Experts+%28MoQE%29%3A+Complementary+Effect+of+Low%2Dbit+Quantization+and+Robustness%2E+arXiv+preprint+arXiv%3A2310%2E02410+%282023%29%2E)
173. Jongwoo Ko, Sungnyun Kim, Tianyi Chen, and Se-Young Yun. 2024. [DistiLLM: Towards Streamlined Distillation for Large Language Models](https://md.research.elamur.ai/free/en/papers/ko-2024-distillm-towards-streamlined-distillation-large/). arXiv preprint arXiv:2402.03898 (2024).
174. Denis Kocetkov, Raymond Li, Loubna Ben Allal, Jia Li, Chenghao Mou, Carlos Muñoz Ferrandis, Yacine Jernite, Margaret Mitchell, Sean Hughes, Thomas Wolf, et al. 2022. The stack: 3 tb of permissively licensed source code. arXiv preprint arXiv:2211.15533 (2022). [find](https://scholar.google.com/scholar?q=Denis+Kocetkov%2C+Raymond+Li%2C+Loubna+Ben+Allal%2C+Jia+Li%2C+Chenghao+Mou%2C+Carlos+Mu%C3%B1oz+Ferrandis%2C+Yacine+Jernite%2C+Margaret+Mitchell%2C+Sean+Hughes%2C+Thomas+Wolf%2C+et+al%2E+2022%2E+The+stack%3A+3+tb+of+permissively+li)
175. Lorenz Kuhn, Yarin Gal, and Sebastian Farquhar. 2023. [Semantic Uncertainty: Linguistic Invariances for Uncertainty Estimation in Natural Language Generation](https://md.research.elamur.ai/free/en/papers/kuhn-2023-semantic-uncertainty-linguistic-invariances-uncertainty/). In Proceedings of the Eleventh International Conference on Learning Representations. [https://openreview.net/forum?id=VD-AYtP0dve](https://openreview.net/forum?id=VD-AYtP0dve) [↗](https://openreview.net/forum?id=VD-AYtP0dve)
176. Divyanshu Kumar, Anurakt Kumar, Sahil Agarwal, and Prashanth Harshangi. 2024. Fine-Tuning, Quantization, and LLMs: Navigating Unintended Outcomes. arXiv preprint arXiv:2404.04392 (2024). — [arXiv:2404.04392](https://arxiv.org/abs/2404.04392) [find](https://scholar.google.com/scholar?q=Divyanshu+Kumar%2C+Anurakt+Kumar%2C+Sahil+Agarwal%2C+and+Prashanth+Harshangi%2E+2024%2E+Fine%2DTuning%2C+Quantization%2C+and+LLMs%3A+Navigating+Unintended+Outcomes%2E+arXiv+preprint+arXiv%3A2404%2E04392+%282024%29%2E+%E2%80%94+%5BarXiv%3A2404)
177. Ohjoon Kwon, Donghyeon Jeon, Nayoung Choi, Gyu-Hwung Cho, Hwiyeol Jo, Changbong Kim, Hyunwoo Lee, Inho Kang, Sun Kim, and Taiwoo Park. 2024. SLM as Guardian: Pioneering AI Safety with Small Language Model. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing: Ind… [find](https://scholar.google.com/scholar?q=Ohjoon+Kwon%2C+Donghyeon+Jeon%2C+Nayoung+Choi%2C+Gyu%2DHwung+Cho%2C+Hwiyeol+Jo%2C+Changbong+Kim%2C+Hyunwoo+Lee%2C+Inho+Kang%2C+Sun+Kim%2C+and+Taiwoo+Park%2E+2024%2E+SLM+as+Guardian%3A+Pioneering+AI+Safety+with+Small+Language+M)
178. Yanis Labrak, Adrien Bazoge, Emmanuel Morin, Pierre-Antoine Gourraud, Mickael Rouvier, and Richard Dufour. 2024. [Biomistral: A collection of open-source pretrained large language models for medical domains](https://md.research.elamur.ai/free/en/papers/labrak-2024-biomistral-collection-open-source-pretrained/). arXiv preprint arXiv:2402.10373 (2024).
179. Stefan Larson, Anish Mahendran, Joseph J. Peper, Christopher Clarke, Andrew Lee, Parker Hill, Jonathan K. Kummerfeld, Kevin Leach, Michael A. Laurenzano, Lingjia Tang, and Jason Mars. 2019. An Evaluation Dataset for Intent Classification and Out-of-Scope Prediction. In Proceedings of the 2019 Confer… [find](https://scholar.google.com/scholar?q=Stefan+Larson%2C+Anish+Mahendran%2C+Joseph+J%2E+Peper%2C+Christopher+Clarke%2C+Andrew+Lee%2C+Parker+Hill%2C+Jonathan+K%2E+Kummerfeld%2C+Kevin+Leach%2C+Michael+A%2E+Laurenzano%2C+Lingjia+Tang%2C+and+Jason+Mars%2E+2019%2E+An+Evaluat)
180. Hugo Laurençon, Lucile Saulnier, Thomas Wang, Christopher Akiki, Albert Villanova del Moral, Teven Le Scao, Leandro Von Werra, Chenghao Mou, Eduardo González Ponferrada, Huu Nguyen, et al. 2022. The bigscience roots corpus: A 1.6 tb composite multilingual dataset. Advances in Neural Information Proc… [find](https://scholar.google.com/scholar?q=Hugo+Lauren%C3%A7on%2C+Lucile+Saulnier%2C+Thomas+Wang%2C+Christopher+Akiki%2C+Albert+Villanova+del+Moral%2C+Teven+Le+Scao%2C+Leandro+Von+Werra%2C+Chenghao+Mou%2C+Eduardo+Gonz%C3%A1lez+Ponferrada%2C+Huu+Nguyen%2C+et+al%2E+2022%2E+The+b)
181. Teven Le Scao, Angela Fan, Christopher Akiki, Ellie Pavlick, Suzana Ilić, Daniel Hesslow, Roman Castagné, Alexandra Sasha Luccioni, François Yvon, Matthias Gallé, et al. 2023. Bloom: A 176b-parameter open-access multilingual language model. (2023). [find](https://scholar.google.com/scholar?q=Teven+Le+Scao%2C+Angela+Fan%2C+Christopher+Akiki%2C+Ellie+Pavlick%2C+Suzana+Ili%C4%87%2C+Daniel+Hesslow%2C+Roman+Castagn%C3%A9%2C+Alexandra+Sasha+Luccioni%2C+Fran%C3%A7ois+Yvon%2C+Matthias+Gall%C3%A9%2C+et+al%2E+2023%2E+Bloom%3A+A+176b%2Dparameter+)
182. Hojae Lee, Junho Kim, and SangKeun Lee. 2024. Mentor-KD: Making Small Language Models Better Multi-step Reasoners. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, Yaser Al-Onaizan, Mohit Bansal, and Yun-Nung Chen (Eds.). Association for Computational Lingui… [find](https://scholar.google.com/scholar?q=Hojae+Lee%2C+Junho+Kim%2C+and+SangKeun+Lee%2E+2024%2E+Mentor%2DKD%3A+Making+Small+Language+Models+Better+Multi%2Dstep+Reasoners%2E+In+Proceedings+of+the+2024+Conference+on+Empirical+Methods+in+Natural+Language+Proces)
183. Jooyoung Lee, Fan Yang, Thanh Tran, Qian Hu, Emre Barut, and Kai-Wei Chang. 2024. Can Small Language Models Help Large Language Models Reason Better?: LM-Guided Chain-of-Thought. In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation… [find](https://scholar.google.com/scholar?q=Jooyoung+Lee%2C+Fan+Yang%2C+Thanh+Tran%2C+Qian+Hu%2C+Emre+Barut%2C+and+Kai%2DWei+Chang%2E+2024%2E+Can+Small+Language+Models+Help+Large+Language+Models+Reason+Better%3F%3A+LM%2DGuided+Chain%2Dof%2DThought%2E+In+Proceedings+of+the)
184. Benjamin Lefaudeux, Francisco Massa, Diana Liskovich, Wenhan Xiong, Vittorio Caggiano, Sean Naren, Min Xu, Jieru Hu, Marta Tintore, Susan Zhang, Patrick Labatut, Daniel Haziza, Luca Wehrstedt, Jeremy Reizenstein, and Grigory Sizov. 2022. xFormers: A modular and hackable Transformer modelling library… [find](https://scholar.google.com/scholar?q=Benjamin+Lefaudeux%2C+Francisco+Massa%2C+Diana+Liskovich%2C+Wenhan+Xiong%2C+Vittorio+Caggiano%2C+Sean+Naren%2C+Min+Xu%2C+Jieru+Hu%2C+Marta+Tintore%2C+Susan+Zhang%2C+Patrick+Labatut%2C+Daniel+Haziza%2C+Luca+Wehrstedt%2C+Jeremy+)
185. Jimmy Lei Ba, Jamie Ryan Kiros, and Geoffrey E Hinton. 2016. [Layer normalization](https://md.research.elamur.ai/free/en/papers/ba-2016-layer-normalization/). ArXiv e-prints (2016), arXiv–1607.
186. Aitor Lewkowycz, Anders Andreassen, David Dohan, Ethan Dyer, Henryk Michalewski, Vinay Ramasesh, Ambrose Slone, Cem Anil, Imanol Schlag, Theo Gutman-Solo, et al. 2022. [Solving quantitative reasoning problems with language models](https://md.research.elamur.ai/free/en/papers/lewkowycz-2022-solving-quantitative-reasoning-problems-language/). Advances in Neural Information Processing Systems 35 (2022), 3843–3857…
187. Chenglin Li, Qianglong Chen, Liangyue Li, Caiyu Wang, Yicheng Li, Zulong Chen, and Yin Zhang. 2023. [Mixed distillation helps smaller language model better reasoning](https://md.research.elamur.ai/free/en/papers/li-2023-mixed-distillation-helps-smaller-language/). arXiv preprint arXiv:2312.10730 (2023).
188. Guangyan Li, Yongqiang Tang, and Wensheng Zhang. 2024. LoRAP: Transformer Sub-Layers Deserve Differentiated Structured Compression for Large Language Models. arXiv preprint arXiv:2404.09695 (2024). — [arXiv:2404.09695](https://arxiv.org/abs/2404.09695) [find](https://scholar.google.com/scholar?q=Guangyan+Li%2C+Yongqiang+Tang%2C+and+Wensheng+Zhang%2E+2024%2E+LoRAP%3A+Transformer+Sub%2DLayers+Deserve+Differentiated+Structured+Compression+for+Large+Language+Models%2E+arXiv+preprint+arXiv%3A2404%2E09695+%282024%29%2E+%E2%80%94+)
189. Haitao Li, Qingyao Ai, Jia Chen, Qian Dong, Zhijing Wu, Yiqun Liu, Chong Chen, and Qi Tian. 2024. BLADE: Enhancing Black-box Large Language Models with Small Domain-Specific Models. arXiv:2403.18365 [cs.CL] https://arxiv.org/abs/2403.18365 — [arXiv:2403.18365](https://arxiv.org/abs/2403.18365) [↗](https://arxiv.org/abs/2403.18365)
190. Haoran Li, Dadi Guo, Donghao Li, Wei Fan, Qi Hu, Xin Liu, Chunkit Chan, Duanyi Yao, Yuan Yao, and Yangqiu Song. 2024. PrivLM-Bench: A Multi-level Privacy Evaluation Benchmark for Language Models. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics. 54–73. [find](https://scholar.google.com/scholar?q=Haoran+Li%2C+Dadi+Guo%2C+Donghao+Li%2C+Wei+Fan%2C+Qi+Hu%2C+Xin+Liu%2C+Chunkit+Chan%2C+Duanyi+Yao%2C+Yuan+Yao%2C+and+Yangqiu+Song%2E+2024%2E+PrivLM%2DBench%3A+A+Multi%2Dlevel+Privacy+Evaluation+Benchmark+for+Language+Models%2E+In+P)
191. Junyi Li, Xiaoxue Cheng, Xin Zhao, Jian-Yun Nie, and Ji-Rong Wen. 2023. [HaluEval: A Large-Scale Hallucination Evaluation Benchmark for Large Language Models](https://md.research.elamur.ai/free/en/papers/li-2023-halueval-large-scale-hallucination-evaluation/). In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, Singapor…
192. Jeffrey Li, Alex Fang, Georgios Smyrnis, Maor Ivgi, Matt Jordan, Samir Gadre, Hritik Bansal, Etash Guha, Sedrick Keh, Kushal Arora, et al. 2024. [Datacomp-lm: In search of the next generation of training sets for language models](https://md.research.elamur.ai/free/en/papers/li-2024-datacomp-lm-search-next-generation/). arXiv preprint arXiv:2406.11794 (2024).
193. Luchang Li, Sheng Qian, Jie Lu, Lunxi Yuan, Rui Wang, and Qin Xie. 2024. Transformer-Lite: High-efficiency Deployment of Large Language Models on Mobile Phone GPUs. arXiv:2403.20041 [cs.CL] https://arxiv.org/abs/2403.20041 [↗](https://arxiv.org/abs/2403.20041)
194. Pingzhi Li, Xiaolong Jin, Yu Cheng, and Tianlong Chen. 2024. Examining Post-Training Quantization for Mixture-of-Experts: A Benchmark. arXiv preprint arXiv:2406.08155 (2024). — [arXiv:2406.08155](https://arxiv.org/abs/2406.08155) [find](https://scholar.google.com/scholar?q=Pingzhi+Li%2C+Xiaolong+Jin%2C+Yu+Cheng%2C+and+Tianlong+Chen%2E+2024%2E+Examining+Post%2DTraining+Quantization+for+Mixture%2Dof%2DExperts%3A+A+Benchmark%2E+arXiv+preprint+arXiv%3A2406%2E08155+%282024%29%2E+%E2%80%94+%5BarXiv%3A2406%2E08155%5D%28http)
195. Raymond Li, Loubna Ben Allal, Yangtian Zi, Niklas Muennighoff, Denis Kocetkov, Chenghao Mou, Marc Marone, Christopher Akiki, Jia Li, Jenny Chim, Qian Liu, Evgenii Zheltonozhskii, Terry Yue Zhuo, Thomas Wang, Olivier Dehaene, Mishig Davaadorj, Joel Lamy-Poirier, João Monteiro, Oleh Shliazhko, Nicolas… [find](https://scholar.google.com/scholar?q=Raymond+Li%2C+Loubna+Ben+Allal%2C+Yangtian+Zi%2C+Niklas+Muennighoff%2C+Denis+Kocetkov%2C+Chenghao+Mou%2C+Marc+Marone%2C+Christopher+Akiki%2C+Jia+Li%2C+Jenny+Chim%2C+Qian+Liu%2C+Evgenii+Zheltonozhskii%2C+Terry+Yue+Zhuo%2C+Thoma)
196. Shengrui Li, Xueting Han, and Jing Bai. 2024. Nuteprune: Efficient progressive pruning with numerous teachers for large language models. arXiv preprint arXiv:2402.09773 (2024). — [arXiv:2402.09773](https://arxiv.org/abs/2402.09773) [find](https://scholar.google.com/scholar?q=Shengrui+Li%2C+Xueting+Han%2C+and+Jing+Bai%2E+2024%2E+Nuteprune%3A+Efficient+progressive+pruning+with+numerous+teachers+for+large+language+models%2E+arXiv+preprint+arXiv%3A2402%2E09773+%282024%29%2E+%E2%80%94+%5BarXiv%3A2402%2E09773%5D%28ht)
197. Tianlin Li, Qian Liu, Tianyu Pang, Chao Du, Qing Guo, Yang Liu, and Min Lin. 2024. Purifying large language models by ensembling a small language model. arXiv preprint arXiv:2402.14845 (2024). — [arXiv:2402.14845](https://arxiv.org/abs/2402.14845) [find](https://scholar.google.com/scholar?q=Tianlin+Li%2C+Qian+Liu%2C+Tianyu+Pang%2C+Chao+Du%2C+Qing+Guo%2C+Yang+Liu%2C+and+Min+Lin%2E+2024%2E+Purifying+large+language+models+by+ensembling+a+small+language+model%2E+arXiv+preprint+arXiv%3A2402%2E14845+%282024%29%2E+%E2%80%94+%5BarXi)
198. Xiang Lisa Li, Ari Holtzman, Daniel Fried, Percy Liang, Jason Eisner, Tatsunori B Hashimoto, Luke Zettlemoyer, and Mike Lewis. 2023. [Contrastive Decoding: Open-ended Text Generation as Optimization](https://md.research.elamur.ai/free/en/papers/li-2022-contrastive-decoding-open-ended-text/). In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1:…
199. Xiang Lisa Li and Percy Liang. 2021. Prefix-tuning: Optimizing continuous prompts for generation. arXiv preprint arXiv:2101.00190 (2021). [find](https://scholar.google.com/scholar?q=Xiang+Lisa+Li+and+Percy+Liang%2E+2021%2E+Prefix%2Dtuning%3A+Optimizing+continuous+prompts+for+generation%2E+arXiv+preprint+arXiv%3A2101%2E00190+%282021%29%2E)
200. Yuanzhi Li, Sébastien Bubeck, Ronen Eldan, Allie Del Giorno, Suriya Gunasekar, and Yin Tat Lee. 2023. [Textbooks Are All You Need II: phi-1.5 technical report](https://md.research.elamur.ai/free/en/papers/li-2023-textbooks-all-need-ii-phi/). arXiv:2309.05463 [cs.CL] https://arxiv.org/abs/2309.05463 Manuscript submitted to ACM [↗](https://arxiv.org/abs/2309.05463)
201. Yun Li, Lin Niu, Xipeng Zhang, Kai Liu, Jianchen Zhu, and Zhanhui Kang. 2023. E-sparse: Boosting the large language model inference through entropy-based n: M sparsity. arXiv preprint arXiv:2310.15929 (2023). [find](https://scholar.google.com/scholar?q=Yun+Li%2C+Lin+Niu%2C+Xipeng+Zhang%2C+Kai+Liu%2C+Jianchen+Zhu%2C+and+Zhanhui+Kang%2E+2023%2E+E%2Dsparse%3A+Boosting+the+large+language+model+inference+through+entropy%2Dbased+n%3A+M+sparsity%2E+arXiv+preprint+arXiv%3A2310%2E15929)
202. Yinheng Li, Shaofei Wang, Han Ding, and Hang Chen. 2023. Large language models in finance: A survey. In Proceedings of the fourth ACM international conference on AI in finance. 374–382. [find](https://scholar.google.com/scholar?q=Yinheng+Li%2C+Shaofei+Wang%2C+Han+Ding%2C+and+Hang+Chen%2E+2023%2E+Large+language+models+in+finance%3A+A+survey%2E+In+Proceedings+of+the+fourth+ACM+international+conference+on+AI+in+finance%2E+374%E2%80%93382%2E)
203. Wing Lian, Guan Wang, Bleys Goodson, Eugene Pentland, Austin Cook, Chanvichet Vong, and "Teknium". 2023. SlimOrca: An Open Dataset of GPT-4 Augmented FLAN Reasoning Traces, with Verification. https://https://huggingface.co/Open-Orca/SlimOrca [find](https://scholar.google.com/scholar?q=Wing+Lian%2C+Guan+Wang%2C+Bleys+Goodson%2C+Eugene+Pentland%2C+Austin+Cook%2C+Chanvichet+Vong%2C+and+%22Teknium%22%2E+2023%2E+SlimOrca%3A+An+Open+Dataset+of+GPT%2D4+Augmented+FLAN+Reasoning+Traces%2C+with+Verification%2E+https%3A%2F%2F)
204. Jinggui Liang, Lizi Liao, Hao Fei, and Jing Jiang. 2024. Synergizing Large Language Models and Pre-Trained Smaller Models for Conversational Intent Discovery. In Findings of the Association for Computational Linguistics ACL 2024. 14133–14147. [find](https://scholar.google.com/scholar?q=Jinggui+Liang%2C+Lizi+Liao%2C+Hao+Fei%2C+and+Jing+Jiang%2E+2024%2E+Synergizing+Large+Language+Models+and+Pre%2DTrained+Smaller+Models+for+Conversational+Intent+Discovery%2E+In+Findings+of+the+Association+for+Comput)
205. Percy Liang, Rishi Bommasani, Tony Lee, Dimitris Tsipras, Dilara Soylu, Michihiro Yasunaga, Yian Zhang, Deepak Narayanan, Yuhuai Wu, Ananya Kumar, Benjamin Newman, Binhang Yuan, Bobby Yan, Ce Zhang, Christian Alexander Cosgrove, Christopher D Manning, Christopher Re, Diana Acosta-Navas, Drew Arad Hu… [find](https://scholar.google.com/scholar?q=Percy+Liang%2C+Rishi+Bommasani%2C+Tony+Lee%2C+Dimitris+Tsipras%2C+Dilara+Soylu%2C+Michihiro+Yasunaga%2C+Yian+Zhang%2C+Deepak+Narayanan%2C+Yuhuai+Wu%2C+Ananya+Kumar%2C+Benjamin+Newman%2C+Binhang+Yuan%2C+Bobby+Yan%2C+Ce+Zhang%2C+C)
206. Jianghao Lin, Rong Shan, Chenxu Zhu, Kounianhua Du, Bo Chen, Shigang Quan, Ruiming Tang, Yong Yu, and Weinan Zhang. 2024. Rella: Retrieval-enhanced large language models for lifelong sequential behavior comprehension in recommendation. In Proceedings of the ACM on Web Conference 2024. 3497–3508. [find](https://scholar.google.com/scholar?q=Jianghao+Lin%2C+Rong+Shan%2C+Chenxu+Zhu%2C+Kounianhua+Du%2C+Bo+Chen%2C+Shigang+Quan%2C+Ruiming+Tang%2C+Yong+Yu%2C+and+Weinan+Zhang%2E+2024%2E+Rella%3A+Retrieval%2Denhanced+large+language+models+for+lifelong+sequential+behavi)
207. Ji Lin, Jiaming Tang, Haotian Tang, Shang Yang, Wei-Ming Chen, Wei-Chen Wang, Guangxuan Xiao, Xingyu Dang, Chuang Gan, and Song Han. 2024. AWQ: Activation-aware Weight Quantization for On-Device LLM Compression and Acceleration. Proceedings of Machine Learning and Systems 6 (2024), 87–100. [find](https://scholar.google.com/scholar?q=Ji+Lin%2C+Jiaming+Tang%2C+Haotian+Tang%2C+Shang+Yang%2C+Wei%2DMing+Chen%2C+Wei%2DChen+Wang%2C+Guangxuan+Xiao%2C+Xingyu+Dang%2C+Chuang+Gan%2C+and+Song+Han%2E+2024%2E+AWQ%3A+Activation%2Daware+Weight+Quantization+for+On%2DDevice+LLM+C)
208. Stephanie Lin, Jacob Hilton, and Owain Evans. 2022. [TruthfulQA: Measuring How Models Mimic Human Falsehoods](https://md.research.elamur.ai/free/en/papers/lin-2022-truthfulqa-measuring-how-models-mimic/). In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 3214–3252.
209. Xi Victoria Lin, Todor Mihaylov, Mikel Artetxe, Tianlu Wang, Shuohui Chen, Daniel Simig, Myle Ott, Naman Goyal, Shruti Bhosale, Jingfei Du, Ramakanth Pasunuru, Sam Shleifer, Punit Singh Koura, Vishrav Chaudhary, Brian O’Horo, Jeff Wang, Luke Zettlemoyer, Zornitsa Kozareva, Mona Diab, Veselin Stoyano… [find](https://scholar.google.com/scholar?q=Xi+Victoria+Lin%2C+Todor+Mihaylov%2C+Mikel+Artetxe%2C+Tianlu+Wang%2C+Shuohui+Chen%2C+Daniel+Simig%2C+Myle+Ott%2C+Naman+Goyal%2C+Shruti+Bhosale%2C+Jingfei+Du%2C+Ramakanth+Pasunuru%2C+Sam+Shleifer%2C+Punit+Singh+Koura%2C+Vishrav)
210. Zhenghao Lin, Zhibin Gou, Yeyun Gong, Xiao Liu, Yelong Shen, Ruochen Xu, Chen Lin, Yujiu Yang, Jian Jiao, Nan Duan, et al. 2024. [Rho-1: Not all tokens are what you need](https://md.research.elamur.ai/free/en/papers/lin-2024-rho-1-all-tokens-need/). arXiv preprint arXiv:2404.07965 (2024).
211. Aixin Liu, Bei Feng, Bin Wang, Bingxuan Wang, Bo Liu, Chenggang Zhao, Chengqi Dengr, Chong Ruan, Damai Dai, Daya Guo, et al. 2024. Deepseek-v2: A strong, economical, and efficient mixture-of-experts language model. arXiv preprint arXiv:2405.04434 (2024). [find](https://scholar.google.com/scholar?q=Aixin+Liu%2C+Bei+Feng%2C+Bin+Wang%2C+Bingxuan+Wang%2C+Bo+Liu%2C+Chenggang+Zhao%2C+Chengqi+Dengr%2C+Chong+Ruan%2C+Damai+Dai%2C+Daya+Guo%2C+et+al%2E+2024%2E+Deepseek%2Dv2%3A+A+strong%2C+economical%2C+and+efficient+mixture%2Dof%2Dexperts+l)
212. Alisa Liu, Xiaochuang Han, Yizhong Wang, Yulia Tsvetkov, Yejin Choi, and Noah A Smith. 2024. [Tuning language models by proxy](https://md.research.elamur.ai/free/en/papers/liu-2024-tuning-language-models-proxy/). arXiv preprint arXiv:2401.08565 (2024).
213. Jiashuo Liu, Zheyan Shen, Yue He, Xingxuan Zhang, Renzhe Xu, Han Yu, and Peng Cui. 2021. Towards out-of-distribution generalization: A survey. arXiv preprint arXiv:2108.13624 (2021). — [arXiv:2108.13624](https://arxiv.org/abs/2108.13624) [find](https://scholar.google.com/scholar?q=Jiashuo+Liu%2C+Zheyan+Shen%2C+Yue+He%2C+Xingxuan+Zhang%2C+Renzhe+Xu%2C+Han+Yu%2C+and+Peng+Cui%2E+2021%2E+Towards+out%2Dof%2Ddistribution+generalization%3A+A+survey%2E+arXiv+preprint+arXiv%3A2108%2E13624+%282021%29%2E+%E2%80%94+%5BarXiv%3A2108%2E136)
214. Qijiong Liu, Nuo Chen, Tetsuya Sakai, and Xiao-Ming Wu. 2024. Once: Boosting content-based recommendation with both open-and closed-source large language models. In Proceedings of the 17th ACM International Conference on Web Search and Data Mining. 452–461. [find](https://scholar.google.com/scholar?q=Qijiong+Liu%2C+Nuo+Chen%2C+Tetsuya+Sakai%2C+and+Xiao%2DMing+Wu%2E+2024%2E+Once%3A+Boosting+content%2Dbased+recommendation+with+both+open%2Dand+closed%2Dsource+large+language+models%2E+In+Proceedings+of+the+17th+ACM+Interna)
215. Suqing Liu, Zezhu Yu, Feiran Huang, Yousef Bulbulia, Andreas Bergen, and Michael Liut. 2024. Can Small Language Models With Retrieval-Augmented Generation Replace Large Language Models When Learning Computer Science? In Proceedings of the 2024 on Innovation and Technology in Computer Science Educati… [find](https://scholar.google.com/scholar?q=Suqing+Liu%2C+Zezhu+Yu%2C+Feiran+Huang%2C+Yousef+Bulbulia%2C+Andreas+Bergen%2C+and+Michael+Liut%2E+2024%2E+Can+Small+Language+Models+With+Retrieval%2DAugmented+Generation+Replace+Large+Language+Models+When+Learning+C)
216. Wei Liu, Weihao Zeng, Keqing He, Yong Jiang, and Junxian He. 2023. [What makes good data for alignment? a comprehensive study of automatic data selection in instruction tuning](https://md.research.elamur.ai/free/en/papers/liu-2023-makes-good-data-alignment-comprehensive/). arXiv preprint arXiv:2312.15685 (2023).
217. Yinpeng Liu, Jiawei Liu, Xiang Shi, Qikai Cheng, and Wei Lu. 2024. Let’s Learn Step by Step: Enhancing In-Context Learning Ability with Curriculum Learning. arXiv preprint arXiv:2402.10738 (2024). — [arXiv:2402.10738](https://arxiv.org/abs/2402.10738) [find](https://scholar.google.com/scholar?q=Yinpeng+Liu%2C+Jiawei+Liu%2C+Xiang+Shi%2C+Qikai+Cheng%2C+and+Wei+Lu%2E+2024%2E+Let%E2%80%99s+Learn+Step+by+Step%3A+Enhancing+In%2DContext+Learning+Ability+with+Curriculum+Learning%2E+arXiv+preprint+arXiv%3A2402%2E10738+%282024%29%2E+%E2%80%94+%5B)
218. Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. [Roberta: A robustly optimized bert pretraining approach](https://md.research.elamur.ai/free/en/papers/liu-2019-roberta-robustly-optimized-bert-pretraining/). arXiv preprint arXiv:1907.11692 (2019).
219. Yanming Liu, Xinyue Peng, Xuhong Zhang, Weihao Liu, Jianwei Yin, Jiannan Cao, and Tianyu Du. 2024. RA-ISF: Learning to Answer and Understand from Retrieval Augmentation via Iterative Self-Feedback. arXiv preprint arXiv:2403.06840 (2024). [find](https://scholar.google.com/scholar?q=Yanming+Liu%2C+Xinyue+Peng%2C+Xuhong+Zhang%2C+Weihao+Liu%2C+Jianwei+Yin%2C+Jiannan+Cao%2C+and+Tianyu+Du%2E+2024%2E+RA%2DISF%3A+Learning+to+Answer+and+Understand+from+Retrieval+Augmentation+via+Iterative+Self%2DFeedback%2E+ar)
220. Zichang Liu, Aditya Desai, Fangshuo Liao, Weitao Wang, Victor Xie, Zhaozhuo Xu, Anastasios Kyrillidis, and Anshumali Shrivastava. 2024. Scissorhands: Exploiting the persistence of importance hypothesis for llm kv cache compression at test time. Advances in Neural Information Processing Systems 36 (2… [find](https://scholar.google.com/scholar?q=Zichang+Liu%2C+Aditya+Desai%2C+Fangshuo+Liao%2C+Weitao+Wang%2C+Victor+Xie%2C+Zhaozhuo+Xu%2C+Anastasios+Kyrillidis%2C+and+Anshumali+Shrivastava%2E+2024%2E+Scissorhands%3A+Exploiting+the+persistence+of+importance+hypothesi)
221. Zechun Liu, Barlas Oguz, Changsheng Zhao, Ernie Chang, Pierre Stock, Yashar Mehdad, Yangyang Shi, Raghuraman Krishnamoorthi, and Vikas Chandra. 2023. [Llm-qat: Data-free quantization aware training for large language models](https://md.research.elamur.ai/free/en/papers/liu-2024-llm-qat-data-free-quantization/). arXiv preprint arXiv:2305.17888 (2023).
222. Zhengxiao Liu, Bowen Shen, Zheng Lin, Fali Wang, and Weiping Wang. 2023. Maximum Entropy Loss, the Silver Bullet Targeting Backdoor Attacks in Pre-trained Language Models. In Findings of the Association for Computational Linguistics: ACL 2023, Anna Rogers, Jordan Boyd-Graber, and Manuscript submitte… [find](https://scholar.google.com/scholar?q=Zhengxiao+Liu%2C+Bowen+Shen%2C+Zheng+Lin%2C+Fali+Wang%2C+and+Weiping+Wang%2E+2023%2E+Maximum+Entropy+Loss%2C+the+Silver+Bullet+Targeting+Backdoor+Attacks+in+Pre%2Dtrained+Language+Models%2E+In+Findings+of+the+Associati)
223. Zechun Liu, Changsheng Zhao, Forrest Iandola, Chen Lai, Yuandong Tian, Igor Fedorov, Yunyang Xiong, Ernie Chang, Yangyang Shi, Raghuraman Krishnamoorthi, et al. 2024. Mobilellm: Optimizing sub-billion parameter language models for on-device use cases. arXiv preprint arXiv:2402.14905 (2024). [find](https://scholar.google.com/scholar?q=Zechun+Liu%2C+Changsheng+Zhao%2C+Forrest+Iandola%2C+Chen+Lai%2C+Yuandong+Tian%2C+Igor+Fedorov%2C+Yunyang+Xiong%2C+Ernie+Chang%2C+Yangyang+Shi%2C+Raghuraman+Krishnamoorthi%2C+et+al%2E+2024%2E+Mobilellm%3A+Optimizing+sub%2Dbillion)
224. Lin Long, Rui Wang, Ruixuan Xiao, Junbo Zhao, Xiao Ding, Gang Chen, and Haobo Wang. 2024. [On LLMs-Driven Synthetic Data Generation, Curation, and Evaluation: A Survey](https://md.research.elamur.ai/free/en/papers/long-2024-llms-driven-synthetic-data-generation/). In Findings of the Association for Computational Linguistics ACL 2024. 11065–11082.
225. Shayne Longpre, Le Hou, Tu Vu, Albert Webson, Hyung Won Chung, Yi Tay, Denny Zhou, Quoc V Le, Barret Zoph, Jason Wei, et al. 2023. [The flan collection: Designing data and methods for effective instruction tuning](https://md.research.elamur.ai/free/en/papers/longpre-2023-flan-collection-designing-data-methods/). In International Conference on Machine Learning. PMLR, 22631–22648.
226. Shayne Longpre, Robert Mahari, Anthony Chen, Naana Obeng-Marnu, Damien Sileo, William Brannon, Niklas Muennighoff, Nathan Khazam, Jad Kabbara, Kartik Perisetla, et al. 2023. The data provenance initiative: A large scale audit of dataset licensing & attribution in ai. arXiv preprint arXiv:2310.16787… — [arXiv:2310.16787](https://arxiv.org/abs/2310.16787) [find](https://scholar.google.com/scholar?q=Shayne+Longpre%2C+Robert+Mahari%2C+Anthony+Chen%2C+Naana+Obeng%2DMarnu%2C+Damien+Sileo%2C+William+Brannon%2C+Niklas+Muennighoff%2C+Nathan+Khazam%2C+Jad+Kabbara%2C+Kartik+Perisetla%2C+et+al%2E+2023%2E+The+data+provenance+initia)
227. Anton Lozhkov, Raymond Li, Loubna Ben Allal, Federico Cassano, Joel Lamy-Poirier, Nouamane Tazi, Ao Tang, Dmytro Pykhtar, Jiawei Liu, Yuxiang Wei, et al. 2024. Starcoder 2 and the stack v2: The next generation. arXiv preprint arXiv:2402.19173 (2024). [find](https://scholar.google.com/scholar?q=Anton+Lozhkov%2C+Raymond+Li%2C+Loubna+Ben+Allal%2C+Federico+Cassano%2C+Joel+Lamy%2DPoirier%2C+Nouamane+Tazi%2C+Ao+Tang%2C+Dmytro+Pykhtar%2C+Jiawei+Liu%2C+Yuxiang+Wei%2C+et+al%2E+2024%2E+Starcoder+2+and+the+stack+v2%3A+The+next+g)
228. Wenhao Lu, Jian Jiao, and Ruofei Zhang. 2020. Twinbert: Distilling knowledge to twin-structured compressed bert models for large-scale retrieval. In Proceedings of the 29th ACM International Conference on Information & Knowledge Management. 2645–2652. [find](https://scholar.google.com/scholar?q=Wenhao+Lu%2C+Jian+Jiao%2C+and+Ruofei+Zhang%2E+2020%2E+Twinbert%3A+Distilling+knowledge+to+twin%2Dstructured+compressed+bert+models+for+large%2Dscale+retrieval%2E+In+Proceedings+of+the+29th+ACM+International+Conferenc)
229. Zhenyan Lu, Xiang Li, Dongqi Cai, Rongjie Yi, Fangming Liu, Xiwen Zhang, Nicholas D Lane, and Mengwei Xu. 2024. [Small language models: Survey, measurements, and insights](https://md.research.elamur.ai/free/en/papers/lu-2024-small-language-models-survey-measurements/). arXiv preprint arXiv:2409.15790 (2024).
230. Renqian Luo, Liai Sun, Yingce Xia, Tao Qin, Sheng Zhang, Hoifung Poon, and Tie-Yan Liu. 2022. BioGPT: generative pre-trained transformer for biomedical text generation and mining. Briefings in bioinformatics 23, 6 (2022), bbac409. [find](https://scholar.google.com/scholar?q=Renqian+Luo%2C+Liai+Sun%2C+Yingce+Xia%2C+Tao+Qin%2C+Sheng+Zhang%2C+Hoifung+Poon%2C+and+Tie%2DYan+Liu%2E+2022%2E+BioGPT%3A+generative+pre%2Dtrained+transformer+for+biomedical+text+generation+and+mining%2E+Briefings+in+bioinfo)
231. Shuming Ma, Hongyu Wang, Lingxiao Ma, Lei Wang, Wenhui Wang, Shaohan Huang, Li Dong, Ruiping Wang, Jilong Xue, and Furu Wei. 2024. The era of 1-bit llms: All large language models are in 1.58 bits. arXiv preprint arXiv:2402.17764 (2024). [find](https://scholar.google.com/scholar?q=Shuming+Ma%2C+Hongyu+Wang%2C+Lingxiao+Ma%2C+Lei+Wang%2C+Wenhui+Wang%2C+Shaohan+Huang%2C+Li+Dong%2C+Ruiping+Wang%2C+Jilong+Xue%2C+and+Furu+Wei%2E+2024%2E+The+era+of+1%2Dbit+llms%3A+All+large+language+models+are+in+1%2E58+bits%2E+ar)
232. Xinyin Ma, Gongfan Fang, and Xinchao Wang. 2023. [Llm-pruner: On the structural pruning of large language models](https://md.research.elamur.ai/free/en/papers/ma-2023-llm-pruner-structural-pruning-large/). Advances in neural information processing systems 36 (2023), 21702–21720.
233. Xinbei Ma, Yeyun Gong, Pengcheng He, Hai Zhao, and Nan Duan. 2023. [Query Rewriting in Retrieval-Augmented Large Language Models](https://md.research.elamur.ai/free/en/papers/ma-2023-query-rewriting-retrieval-augmented-large/). In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing. 5303–5315.
234. Yubo Ma, Yixin Cao, YongChing Hong, and Aixin Sun. 2023. [Large language model is not a good few-shot information extractor, but a good reranker for hard samples!](https://md.research.elamur.ai/free/en/papers/ma-2023-large-language-model-is-not/) arXiv preprint arXiv:2303.08559 (2023).
235. Yuhan Ma, Chenyou Fan, and Haiqi Jiang. 2023. Sci-cot: Leveraging large language models for enhanced knowledge distillation in small models for scientific qa. In 2023 9th International Conference on Computer and Communications (ICCC). IEEE, 2394–2398. [find](https://scholar.google.com/scholar?q=Yuhan+Ma%2C+Chenyou+Fan%2C+and+Haiqi+Jiang%2E+2023%2E+Sci%2Dcot%3A+Leveraging+large+language+models+for+enhanced+knowledge+distillation+in+small+models+for+scientific+qa%2E+In+2023+9th+International+Conference+on+C)
236. YINGWEI MA, Yue Liu, Yue Yu, Yuanliang Zhang, Yu Jiang, Changjian Wang, and Shanshan Li. 2024. At Which Training Stage Does Code Data Help LLMs Reasoning?. In The Twelfth International Conference on Learning Representations. [https://openreview.net/forum?id=KIPJKST4gw](https://openreview.net/forum?id=KIPJKST4gw) [↗](https://openreview.net/forum?id=KIPJKST4gw)
237. Ian Magnusson, Akshita Bhagia, Valentin Hofmann, Luca Soldaini, Ananya Harsh Jha, Oyvind Tafjord, Dustin Schwenk, Evan Pete Walsh, Yanai Elazar, Kyle Lo, et al. 2023. [Paloma: A benchmark for evaluating language model fit](https://md.research.elamur.ai/free/en/papers/magnusson-2023-paloma-benchmark-evaluating-language-model/). arXiv preprint arXiv:2312.10523 (2023).
238. Dakota Mahan, Ryan Carlow, Louis Castricato, Nathan Cooper, and Christian Laforte. [n. d.]. Stable Beluga models. [https://huggingface.co/stabilityai/StableBeluga2](https://huggingface.co/stabilityai/StableBeluga2) [find](https://scholar.google.com/scholar?q=Dakota+Mahan%2C+Ryan+Carlow%2C+Louis+Castricato%2C+Nathan+Cooper%2C+and+Christian+Laforte%2E+%5Bn%2E+d%2E%5D%2E+Stable+Beluga+models%2E+%5Bhttps%3A%2F%2Fhuggingface%2Eco%2Fstabilityai%2FStableBeluga2%5D%28https%3A%2F%2Fhuggingface%2Eco%2Fstabilityai%2F)
239. Vladimir Malinovskii, Denis Mazur, Ivan Ilin, Denis Kuznedelev, Konstantin Burlachenko, Kai Yi, Dan Alistarh, and Peter Richtarik. 2024. PV-Tuning: Beyond Straight-Through Estimation for Extreme LLM Compression. arXiv preprint arXiv:2405.14852 (2024). — [arXiv:2405.14852](https://arxiv.org/abs/2405.14852) [find](https://scholar.google.com/scholar?q=Vladimir+Malinovskii%2C+Denis+Mazur%2C+Ivan+Ilin%2C+Denis+Kuznedelev%2C+Konstantin+Burlachenko%2C+Kai+Yi%2C+Dan+Alistarh%2C+and+Peter+Richtarik%2E+2024%2E+PV%2DTuning%3A+Beyond+Straight%2DThrough+Estimation+for+Extreme+LLM+C)
240. Potsawee Manakul, Adian Liusie, and Mark Gales. 2023. SelfCheckGPT: Zero-Resource Black-Box Hallucination Detection for Generative Large Language Models. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, Houda Bouamor, Juan Pino, and Kalika Bali (Eds.). Assoc… [find](https://scholar.google.com/scholar?q=Potsawee+Manakul%2C+Adian+Liusie%2C+and+Mark+Gales%2E+2023%2E+SelfCheckGPT%3A+Zero%2DResource+Black%2DBox+Hallucination+Detection+for+Generative+Large+Language+Models%2E+In+Proceedings+of+the+2023+Conference+on+Empir)
241. Sachin Mehta, Mohammad Hossein Sekhavat, Qingqing Cao, Maxwell Horton, Yanzi Jin, Chenfan Sun, Seyed Iman Mirzadeh, Mahyar Najibi, Dmitry Belenko, Peter Zatloukal, et al. 2024. Openelm: An efficient language model family with open training and inference framework. In Workshop on Efficient Systems fo… [find](https://scholar.google.com/scholar?q=Sachin+Mehta%2C+Mohammad+Hossein+Sekhavat%2C+Qingqing+Cao%2C+Maxwell+Horton%2C+Yanzi+Jin%2C+Chenfan+Sun%2C+Seyed+Iman+Mirzadeh%2C+Mahyar+Najibi%2C+Dmitry+Belenko%2C+Peter+Zatloukal%2C+et+al%2E+2024%2E+Openelm%3A+An+efficient+l)
242. Dheeraj Mekala, Alex Nguyen, and Jingbo Shang. 2024. Smaller language models are capable of selecting instruction-tuning training data for larger language models. arXiv preprint arXiv:2402.10430 (2024). [find](https://scholar.google.com/scholar?q=Dheeraj+Mekala%2C+Alex+Nguyen%2C+and+Jingbo+Shang%2E+2024%2E+Smaller+language+models+are+capable+of+selecting+instruction%2Dtuning+training+data+for+larger+language+models%2E+arXiv+preprint+arXiv%3A2402%2E10430+%282024)
243. Xin Men, Mingyu Xu, Qingyu Zhang, Bingning Wang, Hongyu Lin, Yaojie Lu, Xianpei Han, and Weipeng Chen. 2024. [Shortgpt: Layers in large language models are more redundant than you expect](https://md.research.elamur.ai/free/en/papers/men-2024-shortgpt-layers-large-language-models/). arXiv preprint arXiv:2403.03853 (2024).
244. Sewon Min, Kalpesh Krishna, Xinxi Lyu, Mike Lewis, Wen-tau Yih, Pang Koh, Mohit Iyyer, Luke Zettlemoyer, and Hannaneh Hajishirzi. 2023. [FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generation](https://md.research.elamur.ai/free/en/papers/min-2023-factscore-fine-grained-atomic-evaluation/). In Proceedings of the 2023 Conference on Empirical Methods in Natural L…
245. Go Min-su. 2024. Deep Learning Bible - 8. Large Language Models. WikiDocs. [https://wikidocs.net/237419](https://wikidocs.net/237419) [↗](https://wikidocs.net/237419)
246. Eric Mitchell, Rafael Rafailov, Archit Sharma, Chelsea Finn, and Christopher D Manning. 2024. [An Emulator for Fine-tuning Large Language Models using Small Language Models](https://md.research.elamur.ai/free/en/papers/mitchell-2023-emulator-fine-tuning-large-language/). In The Twelfth International Conference on Learning Representations. [https://openreview.net/forum?id=Eo7kv0sllr](https://openreview.net/forum?id=Eo7kv0sllr) [↗](https://openreview.net/forum?id=Eo7kv0sllr)
247. Arindam Mitra, Luciano Del Corro, Shweti Mahajan, Andres Codas, Clarisse Simoes, Sahaj Agarwal, Xuxi Chen, Anastasia Razdaibiedina, Erik Jones, Kriti Aggarwal, et al. 2023. [Orca 2: Teaching small language models how to reason](https://md.research.elamur.ai/free/en/papers/mitra-2023-orca-2-teaching-small-language/). arXiv preprint arXiv:2311.11045 (2023). Manuscript submitted to ACM
248. Lingbo Mo, Boshi Wang, Muhao Chen, and Huan Sun. 2024. How Trustworthy are Open-Source LLMs? An Assessment under Malicious Demonstrations Shows their Vulnerabilities. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language… [find](https://scholar.google.com/scholar?q=Lingbo+Mo%2C+Boshi+Wang%2C+Muhao+Chen%2C+and+Huan+Sun%2E+2024%2E+How+Trustworthy+are+Open%2DSource+LLMs%3F+An+Assessment+under+Malicious+Demonstrations+Shows+their+Vulnerabilities%2E+In+Proceedings+of+the+2024+Confer)
249. John Xavier Morris, Wenting Zhao, Justin T Chiu, Vitaly Shmatikov, and Alexander M Rush. 2024. Language Model Inversion. In The Twelfth International Conference on Learning Representations. [https://openreview.net/forum?id=t9dWHpGkPj](https://openreview.net/forum?id=t9dWHpGkPj) [↗](https://openreview.net/forum?id=t9dWHpGkPj)
250. Maximilian Mozes, Xuanli He, Bennett Kleinberg, and Lewis D Griffin. 2023. Use of llms for illicit purposes: Threats, prevention measures, and vulnerabilities. arXiv preprint arXiv:2308.12833 (2023). — [arXiv:2308.12833](https://arxiv.org/abs/2308.12833) [find](https://scholar.google.com/scholar?q=Maximilian+Mozes%2C+Xuanli+He%2C+Bennett+Kleinberg%2C+and+Lewis+D+Griffin%2E+2023%2E+Use+of+llms+for+illicit+purposes%3A+Threats%2C+prevention+measures%2C+and+vulnerabilities%2E+arXiv+preprint+arXiv%3A2308%2E12833+%282023%29%2E+)
251. Subhabrata Mukherjee, Arindam Mitra, Ganesh Jawahar, Sahaj Agarwal, Hamid Palangi, and Ahmed Awadallah. 2023. [Orca: Progressive learning from complex explanation traces of gpt-4](https://md.research.elamur.ai/free/en/papers/mukherjee-2023-orca-progressive-learning-complex-explanation/). arXiv preprint arXiv:2306.02707 (2023).
252. Saurav Muralidharan, Sharath Turuvekere Sreenivas, Raviraj Joshi, Marcin Chochowski, Mostofa Patwary, Mohammad Shoeybi, Bryan Catanzaro, Jan Kautz, and Pavlo Molchanov. 2024. Compact language models via pruning and knowledge distillation. arXiv preprint arXiv:2407.14679 (2024). [find](https://scholar.google.com/scholar?q=Saurav+Muralidharan%2C+Sharath+Turuvekere+Sreenivas%2C+Raviraj+Joshi%2C+Marcin+Chochowski%2C+Mostofa+Patwary%2C+Mohammad+Shoeybi%2C+Bryan+Catanzaro%2C+Jan+Kautz%2C+and+Pavlo+Molchanov%2E+2024%2E+Compact+language+models+v)
253. Rithesh Murthy, Liangwei Yang, Juntao Tan, Tulika Manoj Awalgaonkar, Yilun Zhou, Shelby Heinecke, Sachin Desai, Jason Wu, Ran Xu, Sarah Tan, Jianguo Zhang, Zhiwei Liu, Shirley Kokane, Zuxin Liu, Ming Zhu, Huan Wang, Caiming Xiong, and Silvio Savarese. 2024. MobileAIBench: Benchmarking LLMs and LMMs… — [arXiv:2406.10290](https://arxiv.org/abs/2406.10290) [find](https://scholar.google.com/scholar?q=Rithesh+Murthy%2C+Liangwei+Yang%2C+Juntao+Tan%2C+Tulika+Manoj+Awalgaonkar%2C+Yilun+Zhou%2C+Shelby+Heinecke%2C+Sachin+Desai%2C+Jason+Wu%2C+Ran+Xu%2C+Sarah+Tan%2C+Jianguo+Zhang%2C+Zhiwei+Liu%2C+Shirley+Kokane%2C+Zuxin+Liu%2C+Ming+)
254. Kalyan Nakka, Jimmy Dani, and Nitesh Saxena. 2024. Is On-Device AI Broken and Exploitable? Assessing the Trust and Ethics in Small Language Models. arXiv preprint arXiv:2406.05364 (2024). — [arXiv:2406.05364](https://arxiv.org/abs/2406.05364) [find](https://scholar.google.com/scholar?q=Kalyan+Nakka%2C+Jimmy+Dani%2C+and+Nitesh+Saxena%2E+2024%2E+Is+On%2DDevice+AI+Broken+and+Exploitable%3F+Assessing+the+Trust+and+Ethics+in+Small+Language+Models%2E+arXiv+preprint+arXiv%3A2406%2E05364+%282024%29%2E+%E2%80%94+%5BarXiv%3A240)
255. Daye Nam, Andrew Macvean, Vincent Hellendoorn, Bogdan Vasilescu, and Brad Myers. 2024. Using an llm to help with code understanding. In Proceedings of the IEEE/ACM 46th International Conference on Software Engineering. 1–13. [find](https://scholar.google.com/scholar?q=Daye+Nam%2C+Andrew+Macvean%2C+Vincent+Hellendoorn%2C+Bogdan+Vasilescu%2C+and+Brad+Myers%2E+2024%2E+Using+an+llm+to+help+with+code+understanding%2E+In+Proceedings+of+the+IEEE%2FACM+46th+International+Conference+on+Sof)
256. Piotr Nawrot, Adrian Łańcucki, Marcin Chochowski, David Tarjan, and Edoardo Ponti. 2024. Dynamic Memory Compression: Retrofitting LLMs for Accelerated Inference. In Forty-first International Conference on Machine Learning. [https://openreview.net/forum?id=tDRYrAkOB7](https://openreview.net/forum?id=tDRYrAkOB7) [↗](https://openreview.net/forum?id=tDRYrAkOB7)
257. Thuat Nguyen, Chien Van Nguyen, Viet Dac Lai, Hieu Man, Nghia Trung Ngo, Franck Dernoncourt, Ryan A Rossi, and Thien Huu Nguyen. 2024. [CulturaX: A Cleaned, Enormous, and Multilingual Dataset for Large Language Models in 167 Languages](https://md.research.elamur.ai/free/en/papers/nguyen-2023-culturax-cleaned-enormous-multilingual-dataset/). In Proceedings of the 2024 Joint International Conference on Comp…
258. Tuan Dung Nguyen, Yuan-Sen Ting, Ioana Ciuca, Charles O’Neill, Ze-Chang Sun, Maja Jabłońska, Sandor Kruk, Ernest Perkowski, Jack Miller, Jason Jason Jingsh Li, et al. 2023. AstroLLaMA: Towards Specialized Foundation Models in Astronomy. In Proceedings of the Second Workshop on Information Extraction… [find](https://scholar.google.com/scholar?q=Tuan+Dung+Nguyen%2C+Yuan%2DSen+Ting%2C+Ioana+Ciuca%2C+Charles+O%E2%80%99Neill%2C+Ze%2DChang+Sun%2C+Maja+Jab%C5%82o%C5%84ska%2C+Sandor+Kruk%2C+Ernest+Perkowski%2C+Jack+Miller%2C+Jason+Jason+Jingsh+Li%2C+et+al%2E+2023%2E+AstroLLaMA%3A+Towards+Special)
259. Jianmo Ni, Chen Qu, Jing Lu, Zhuyun Dai, Gustavo Hernandez Abrego, Ji Ma, Vincent Zhao, Yi Luan, Keith Hall, Ming-Wei Chang, and Yinfei Yang. 2022. [Large Dual Encoders Are Generalizable Retrievers](https://md.research.elamur.ai/free/en/papers/ni-2021-large-dual-encoders-generalizable-retrievers/). In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing. Association…
260. Rodrigo Nogueira and Kyunghyun Cho. 2019. [Passage Re-ranking with BERT](https://md.research.elamur.ai/free/en/papers/nogueira-2019-passage-re-ranking-bert/). arXiv preprint arXiv:1901.04085 (2019).
261. A Noorian. 2024. A BERT-based sequential POI recommender system in social media. Computer Standards & Interfaces 87 (2024), 103766. [find](https://scholar.google.com/scholar?q=A+Noorian%2E+2024%2E+A+BERT%2Dbased+sequential+POI+recommender+system+in+social+media%2E+Computer+Standards+%26+Interfaces+87+%282024%29%2C+103766%2E)
262. OpenAI. 2024. GPT-4o mini: advancing cost-efficient intelligence. [https://openai.com/index/gpt-4o-mini-advancing-cost-efficient-intelligence/Accessed:](https://openai.com/index/gpt-4o-mini-advancing-cost-efficient-intelligence/Accessed:) 2024-7-18. [↗](https://openai.com/index/gpt-4o-mini-advancing-cost-efficient-intelligence/Accessed:)
263. OpenAI. 2024. Hello GPT-4o. [https://openai.com/index/hello-gpt-4o/](https://openai.com/index/hello-gpt-4o/) Accessed: 2024-5-13. [↗](https://openai.com/index/hello-gpt-4o/)
264. Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. 2022. [Training language models to follow instructions with human feedback](https://md.research.elamur.ai/free/en/papers/ouyang-2022-training-language-models-follow-instructions/). Advances in neural information processing systems 35 (2022), 27730–27744.
265. Shankar Padmanabhan, Yasumasa Onoe, Michael Zhang, Greg Durrett, and Eunsol Choi. 2024. Propagating knowledge updates to lms through distillation. Advances in Neural Information Processing Systems 36 (2024). [find](https://scholar.google.com/scholar?q=Shankar+Padmanabhan%2C+Yasumasa+Onoe%2C+Michael+Zhang%2C+Greg+Durrett%2C+and+Eunsol+Choi%2E+2024%2E+Propagating+knowledge+updates+to+lms+through+distillation%2E+Advances+in+Neural+Information+Processing+Systems+36+)
266. Jupinder Parmar, Shrimai Prabhumoye, Joseph Jennings, Mostofa Patwary, Sandeep Subramanian, Dan Su, Chen Zhu, Deepak Narayanan, Aastha Jhunjhunwala, Ayush Dattagupta, et al. 2024. [Nemotron-4 15B Technical Report](https://md.research.elamur.ai/free/en/papers/parmar-2024-nemotron-4-15b-technical-report/). arXiv preprint arXiv:2402.16819 (2024).
267. Keiran Paster, Marco Dos Santos, Zhangir Azerbayev, and Jimmy Ba. 2024. [OpenWebMath: An Open Dataset of High-Quality Mathematical Web Text](https://md.research.elamur.ai/free/en/papers/paster-2023-openwebmath-open-dataset-high-quality/). In The Twelfth International Conference on Learning Representations. [https://openreview.net/forum?id=jKHmjlpViu](https://openreview.net/forum?id=jKHmjlpViu) [↗](https://openreview.net/forum?id=jKHmjlpViu)
268. Guilherme Penedo, Hynek Kydlíček, Loubna Ben allal, Anton Lozhkov, Margaret Mitchell, Colin Raffel, Leandro Von Werra, and Thomas Wolf. 2024. [The FineWeb Datasets: Decanting the Web for the Finest Text Data at Scale](https://md.research.elamur.ai/free/en/papers/penedo-2024-fineweb-datasets-decanting-web-finest/). arXiv:2406.17557 [cs.CL]
269. Guilherme Penedo, Quentin Malartic, Daniel Hesslow, Ruxandra Cojocaru, Alessandro Cappelli, Hamza Alobeidli, Baptiste Pannier, Ebtesam Almazrouei, and Julien Launay. 2023. The RefinedWeb dataset for Falcon LLM: outperforming curated corpora with web data, and web data only. arXiv preprint arXiv:2306… [find](https://scholar.google.com/scholar?q=Guilherme+Penedo%2C+Quentin+Malartic%2C+Daniel+Hesslow%2C+Ruxandra+Cojocaru%2C+Alessandro+Cappelli%2C+Hamza+Alobeidli%2C+Baptiste+Pannier%2C+Ebtesam+Almazrouei%2C+and+Julien+Launay%2E+2023%2E+The+RefinedWeb+dataset+for+F)
270. Bo Peng, Eric Alcaide, Quentin Anthony, Alon Albalak, Samuel Arcadinho, Stella Biderman, Huanqi Cao, Xin Cheng, Michael Chung, Leon Derczynski, Xingjian Du, Matteo Grella, Kranthi Gv, Xuzheng He, Haowen Hou, Przemyslaw Kazienko, Jan Kocon, Jiaming Kong, Bartłomiej Koptyra, Hayden Lau, Jiaju Lin, Kri… [find](https://scholar.google.com/scholar?q=Bo+Peng%2C+Eric+Alcaide%2C+Quentin+Anthony%2C+Alon+Albalak%2C+Samuel+Arcadinho%2C+Stella+Biderman%2C+Huanqi+Cao%2C+Xin+Cheng%2C+Michael+Chung%2C+Leon+Derczynski%2C+Xingjian+Du%2C+Matteo+Grella%2C+Kranthi+Gv%2C+Xuzheng+He%2C+Haow)
271. Baolin Peng, Chunyuan Li, Pengcheng He, Michel Galley, and Jianfeng Gao. 2023. [Instruction tuning with gpt-4](https://md.research.elamur.ai/free/en/papers/peng-2023-instruction-tuning-gpt-4/). arXiv preprint arXiv:2304.03277 (2023). Manuscript submitted to ACM
272. Zhiyuan Peng, Xuyang Wu, Qifan Wang, and Yi Fang. 2023. Soft prompt tuning for augmenting dense retrieval with large language models. arXiv preprint arXiv:2307.08303 (2023). — [arXiv:2307.08303](https://arxiv.org/abs/2307.08303) [find](https://scholar.google.com/scholar?q=Zhiyuan+Peng%2C+Xuyang+Wu%2C+Qifan+Wang%2C+and+Yi+Fang%2E+2023%2E+Soft+prompt+tuning+for+augmenting+dense+retrieval+with+large+language+models%2E+arXiv+preprint+arXiv%3A2307%2E08303+%282023%29%2E+%E2%80%94+%5BarXiv%3A2307%2E08303%5D%28https)
273. Ethan Perez, Sam Ringer, Kamile Lukosiute, Karina Nguyen, Edwin Chen, Scott Heiner, Craig Pettit, Catherine Olsson, Sandipan Kundu, Saurav Kadavath, Andy Jones, Anna Chen, Benjamin Mann, Brian Israel, Bryan Seethor, Cameron McKinnon, Christopher Olah, Da Yan, Daniela Amodei, Dario Amodei, Dawn Drain… — [Discovering Language Model Behaviors with Model-Written Evaluations](https://md.research.elamur.ai/free/en/papers/perez-2022-discovering-language-model-behaviors-model/)
274. Pascal Pfeiffer, Philipp Singer, Yauhen Babakhin, Gabor Fodor, Nischay Dhankhar, and Sri Satish Ambati. 2024. H2O-Danube3 Technical Report. arXiv preprint arXiv:2407.09276 (2024). — [arXiv:2407.09276](https://arxiv.org/abs/2407.09276) [find](https://scholar.google.com/scholar?q=Pascal+Pfeiffer%2C+Philipp+Singer%2C+Yauhen+Babakhin%2C+Gabor+Fodor%2C+Nischay+Dhankhar%2C+and+Sri+Satish+Ambati%2E+2024%2E+H2O%2DDanube3+Technical+Report%2E+arXiv+preprint+arXiv%3A2407%2E09276+%282024%29%2E+%E2%80%94+%5BarXiv%3A2407%2E09276%5D)
275. Karmvir Singh Phogat, Sai Akhil Puranam, Sridhar Dasaratha, Chetan Harsha, and Shashishekar Ramakrishna. 2024. Fine-tuning Smaller Language Models for Question Answering over Financial Documents. In Findings of the Association for Computational Linguistics: EMNLP 2024, Yaser Al-Onaizan, Mohit Bansal… [find](https://scholar.google.com/scholar?q=Karmvir+Singh+Phogat%2C+Sai+Akhil+Puranam%2C+Sridhar+Dasaratha%2C+Chetan+Harsha%2C+and+Shashishekar+Ramakrishna%2E+2024%2E+Fine%2Dtuning+Smaller+Language+Models+for+Question+Answering+over+Financial+Documents%2E+In+F)
276. Reiner Pope, Sholto Douglas, Aakanksha Chowdhery, Jacob Devlin, James Bradbury, Jonathan Heek, Kefan Xiao, Shivani Agrawal, and Jeff Dean. 2023. Efficiently scaling transformer inference. Proceedings of Machine Learning and Systems 5 (2023), 606–624. [find](https://scholar.google.com/scholar?q=Reiner+Pope%2C+Sholto+Douglas%2C+Aakanksha+Chowdhery%2C+Jacob+Devlin%2C+James+Bradbury%2C+Jonathan+Heek%2C+Kefan+Xiao%2C+Shivani+Agrawal%2C+and+Jeff+Dean%2E+2023%2E+Efficiently+scaling+transformer+inference%2E+Proceedings+)
277. Ofir Press, Noah Smith, and Mike Lewis. 2022. Train Short, Test Long: Attention with Linear Biases Enables Input Length Extrapolation. In International Conference on Learning Representations. [https://openreview.net/forum?id=R8sQPpGCv0](https://openreview.net/forum?id=R8sQPpGCv0) [↗](https://openreview.net/forum?id=R8sQPpGCv0)
278. Ruiyang Qin, Jun Xia, Zhenge Jia, Meng Jiang, Ahmed Abbasi, Peipei Zhou, Jingtong Hu, and Yiyu Shi. 2023. Enabling on-device large language model personalization with self-supervised data selection and synthesis. arXiv preprint arXiv:2311.12275 (2023). — [arXiv:2311.12275](https://arxiv.org/abs/2311.12275) [find](https://scholar.google.com/scholar?q=Ruiyang+Qin%2C+Jun+Xia%2C+Zhenge+Jia%2C+Meng+Jiang%2C+Ahmed+Abbasi%2C+Peipei+Zhou%2C+Jingtong+Hu%2C+and+Yiyu+Shi%2E+2023%2E+Enabling+on%2Ddevice+large+language+model+personalization+with+self%2Dsupervised+data+selection+an)
279. Yujia Qin, Shihao Liang, Yining Ye, Kunlun Zhu, Lan Yan, Yaxi Lu, Yankai Lin, Xin Cong, Xiangru Tang, Bill Qian, et al. [n. d.]. [ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIs](https://md.research.elamur.ai/free/en/papers/qin-2023-toolllm-facilitating-large-language-models/). In The Twelfth International Conference on Learning Representations.
280. Haohao Qu, Liangbo Ning, Rui An, Wenqi Fan, Tyler Derr, Hui Liu, Xin Xu, and Qing Li. 2024. A survey of mamba. arXiv preprint arXiv:2408.01129 (2024). — [arXiv:2408.01129](https://arxiv.org/abs/2408.01129) [find](https://scholar.google.com/scholar?q=Haohao+Qu%2C+Liangbo+Ning%2C+Rui+An%2C+Wenqi+Fan%2C+Tyler+Derr%2C+Hui+Liu%2C+Xin+Xu%2C+and+Qing+Li%2E+2024%2E+A+survey+of+mamba%2E+arXiv+preprint+arXiv%3A2408%2E01129+%282024%29%2E+%E2%80%94+%5BarXiv%3A2408%2E01129%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2408%2E0)
281. Haohao Qu, Yifeng Zhang, Liangbo Ning, Wenqi Fan, and Qing Li. 2024. Ssd4rec: a structured state space duality model for efficient sequential recommendation. arXiv preprint arXiv:2409.01192 (2024). — [arXiv:2409.01192](https://arxiv.org/abs/2409.01192) [find](https://scholar.google.com/scholar?q=Haohao+Qu%2C+Yifeng+Zhang%2C+Liangbo+Ning%2C+Wenqi+Fan%2C+and+Qing+Li%2E+2024%2E+Ssd4rec%3A+a+structured+state+space+duality+model+for+efficient+sequential+recommendation%2E+arXiv+preprint+arXiv%3A2409%2E01192+%282024%29%2E+%E2%80%94+)
282. Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. 2019. Language models are unsupervised multitask learners. OpenAI blog 1, 8 (2019), 9. [find](https://scholar.google.com/scholar?q=Alec+Radford%2C+Jeffrey+Wu%2C+Rewon+Child%2C+David+Luan%2C+Dario+Amodei%2C+Ilya+Sutskever%2C+et+al%2E+2019%2E+Language+models+are+unsupervised+multitask+learners%2E+OpenAI+blog+1%2C+8+%282019%29%2C+9%2E)
283. Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D Manning, Stefano Ermon, and Chelsea Finn. 2024. [Direct preference optimization: Your language model is secretly a reward model](https://md.research.elamur.ai/free/en/papers/rafailov-2023-direct-preference-optimization-language-model/). Advances in Neural Information Processing Systems 36 (2024).
284. Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu. 2020. [Exploring the limits of transfer learning with a unified text-to-text transformer](https://md.research.elamur.ai/free/en/papers/raffel-2019-exploring-limits-transfer-learning-unified/). Journal of machine learning research 21, 140 (2020), 1–67.
285. Mohammad Wali Ur Rahman, Murad Mehrab Abrar, Hunter Gibbons Copening, Salim Hariri, Sicong Shao, Pratik Satam, and Soheil Salehi. 2023. Quantized Transformer Language Model Implementations on Edge Devices. arXiv:2310.03971 [cs.CL] https://arxiv.org/abs/2310.03971 — [arXiv:2310.03971](https://arxiv.org/abs/2310.03971) [↗](https://arxiv.org/abs/2310.03971)
286. Samyam Rajbhandari, Jeff Rasley, Olatunji Ruwase, and Yuxiong He. 2020. Zero: Memory optimizations toward training trillion parameter models. In SC20: International Conference for High Performance Computing, Networking, Storage and Analysis. IEEE, 1–16. [find](https://scholar.google.com/scholar?q=Samyam+Rajbhandari%2C+Jeff+Rasley%2C+Olatunji+Ruwase%2C+and+Yuxiong+He%2E+2020%2E+Zero%3A+Memory+optimizations+toward+training+trillion+parameter+models%2E+In+SC20%3A+International+Conference+for+High+Performance+Com)
287. Ori Ram, Yoav Levine, Itay Dalmedigos, Dor Muhlgay, Amnon Shashua, Kevin Leyton-Brown, and Yoav Shoham. 2023. [In-Context Retrieval-Augmented Language Models](https://md.research.elamur.ai/free/en/papers/ram-2023-context-retrieval-augmented-language-models/). Transactions of the Association for Computational Linguistics 11 (2023), 1316–1331.
288. Krithika Ramesh, Arnav Chavan, Shrey Pandit, and Sunayana Sitaram. 2023. A Comparative Study on the Impact of Model Compression Techniques on Fairness in Language Models. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Association f… [find](https://scholar.google.com/scholar?q=Krithika+Ramesh%2C+Arnav+Chavan%2C+Shrey+Pandit%2C+and+Sunayana+Sitaram%2E+2023%2E+A+Comparative+Study+on+the+Impact+of+Model+Compression+Techniques+on+Fairness+in+Language+Models%2E+In+Proceedings+of+the+61st+An)
289. Al Mamunur Rashid, George Karypis, and John Riedl. 2008. Learning preferences of new users in recommender systems: an information theoretic approach. Acm Sigkdd Explorations Newsletter 10, 2 (2008), 90–100. [find](https://scholar.google.com/scholar?q=Al+Mamunur+Rashid%2C+George+Karypis%2C+and+John+Riedl%2E+2008%2E+Learning+preferences+of+new+users+in+recommender+systems%3A+an+information+theoretic+approach%2E+Acm+Sigkdd+Explorations+Newsletter+10%2C+2+%282008%29%2C+9)
290. Christopher Rawles, Alice Li, Daniel Rodriguez, Oriana Riva, and Timothy Lillicrap. 2024. Androidinthewild: A large-scale dataset for android device control. Advances in Neural Information Processing Systems 36 (2024). [find](https://scholar.google.com/scholar?q=Christopher+Rawles%2C+Alice+Li%2C+Daniel+Rodriguez%2C+Oriana+Riva%2C+and+Timothy+Lillicrap%2E+2024%2E+Androidinthewild%3A+A+large%2Dscale+dataset+for+android+device+control%2E+Advances+in+Neural+Information+Processing+)
291. Stephen Robertson, Hugo Zaragoza, et al. 2009. The probabilistic relevance framework: BM25 and beyond. Foundations and Trends® in Information Retrieval 3, 4 (2009), 333–389. [find](https://scholar.google.com/scholar?q=Stephen+Robertson%2C+Hugo+Zaragoza%2C+et+al%2E+2009%2E+The+probabilistic+relevance+framework%3A+BM25+and+beyond%2E+Foundations+and+Trends%C2%AE+in+Information+Retrieval+3%2C+4+%282009%29%2C+333%E2%80%93389%2E)
292. Baptiste Roziere, Jonas Gehring, Fabian Gloeckle, Sten Sootla, Itai Gat, Xiaoqing Ellen Tan, Yossi Adi, Jingyu Liu, Tal Remez, Jérémy Rapin, et al. 2023. [Code llama: Open foundation models for code](https://md.research.elamur.ai/free/en/papers/roziere-2024-code-llama-open-foundation-models/). arXiv preprint arXiv:2308.12950 (2023).
293. Caitlin Sadowski and Greg Levin. 2007. Simhash: Hash-based similarity detection. [find](https://scholar.google.com/scholar?q=Caitlin+Sadowski+and+Greg+Levin%2E+2007%2E+Simhash%3A+Hash%2Dbased+similarity+detection%2E)
294. Keisuke Sakaguchi, Ronan Le Bras, Chandra Bhagavatula, and Yejin Choi. 2021. Winogrande: An adversarial winograd schema challenge at scale. Commun. ACM 64, 9 (2021), 99–106. Manuscript submitted to ACM [find](https://scholar.google.com/scholar?q=Keisuke+Sakaguchi%2C+Ronan+Le+Bras%2C+Chandra+Bhagavatula%2C+and+Yejin+Choi%2E+2021%2E+Winogrande%3A+An+adversarial+winograd+schema+challenge+at+scale%2E+Commun%2E+ACM+64%2C+9+%282021%29%2C+99%E2%80%93106%2E+Manuscript+submitted+to+AC)
295. Timo Schick, Jane Dwivedi-Yu, Roberto Dessì, Roberta Raileanu, Maria Lomeli, Eric Hambro, Luke Zettlemoyer, Nicola Cancedda, and Thomas Scialom. 2024. [Toolformer: Language models can teach themselves to use tools](https://md.research.elamur.ai/free/en/papers/schick-2023-toolformer-language-models-can-teach/). Advances in Neural Information Processing Systems 36 (2024).
296. Rico Sennrich, Jannis Vamvas, and Alireza Mohammadshahi. 2023. [Mitigating Hallucinations and Off-target Machine Translation with Source-Contrastive and Language-Contrastive Decoding](https://md.research.elamur.ai/free/en/papers/sennrich-2024-mitigating-hallucinations-off-target-machine/). arXiv preprint arXiv:2309.07098 (2023).
297. Zeyang Sha and Yang Zhang. 2024. Prompt stealing attacks against large language models. arXiv preprint arXiv:2402.12959 (2024). — [arXiv:2402.12959](https://arxiv.org/abs/2402.12959) [find](https://scholar.google.com/scholar?q=Zeyang+Sha+and+Yang+Zhang%2E+2024%2E+Prompt+stealing+attacks+against+large+language+models%2E+arXiv+preprint+arXiv%3A2402%2E12959+%282024%29%2E+%E2%80%94+%5BarXiv%3A2402%2E12959%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2402%2E12959%29)
298. Yu Shang, Yu Li, Fengli Xu, and Yong Li. 2024. Synergy-of-Thoughts: Eliciting Efficient Reasoning in Hybrid Language Models. arXiv preprint arXiv:2402.02563 (2024). [find](https://scholar.google.com/scholar?q=Yu+Shang%2C+Yu+Li%2C+Fengli+Xu%2C+and+Yong+Li%2E+2024%2E+Synergy%2Dof%2DThoughts%3A+Eliciting+Efficient+Reasoning+in+Hybrid+Language+Models%2E+arXiv+preprint+arXiv%3A2402%2E02563+%282024%29%2E)
299. Yuzhang Shang, Zhihang Yuan, Qiang Wu, and Zhen Dong. 2023. PB-LLM: Partially Binarized Large Language Models. arXiv:2310.00034 [cs.LG] https://arxiv.org/abs/2310.00034 — [arXiv:2310.00034](https://arxiv.org/abs/2310.00034) [↗](https://arxiv.org/abs/2310.00034)
300. Hang Shao, Bei Liu, and Yanmin Qian. 2024. One-shot sensitivity-aware mixed sparsity pruning for large language models. In ICASSP 2024-2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 11296–11300. [find](https://scholar.google.com/scholar?q=Hang+Shao%2C+Bei+Liu%2C+and+Yanmin+Qian%2E+2024%2E+One%2Dshot+sensitivity%2Daware+mixed+sparsity+pruning+for+large+language+models%2E+In+ICASSP+2024%2D2024+IEEE+International+Conference+on+Acoustics%2C+Speech+and+Signa)
301. Mrinank Sharma, Meg Tong, Tomasz Korbak, David Duvenaud, Amanda Askell, Samuel R Bowman, Newton Cheng, Esin Durmus, Zac Hatfield-Dodds, Scott R Johnston, et al. 2023. [Towards understanding sycophancy in language models](https://md.research.elamur.ai/free/en/papers/sharma-2023-towards-understanding-sycophancy-language-models/). arXiv preprint arXiv:2310.13548 (2023).
302. Erfan Shayegani, Md Abdullah Al Mamun, Yu Fu, Pedram Zaree, Yue Dong, and Nael Abu-Ghazaleh. 2023. [Survey of vulnerabilities in large language models revealed by adversarial attacks](https://md.research.elamur.ai/free/en/papers/shayegani-2023-survey-vulnerabilities-large-language-models/). arXiv preprint arXiv:2310.10844 (2023).
303. Noam Shazeer. 2019. Fast transformer decoding: One write-head is all you need. arXiv preprint arXiv:1911.02150 (2019). [find](https://scholar.google.com/scholar?q=Noam+Shazeer%2E+2019%2E+Fast+transformer+decoding%3A+One+write%2Dhead+is+all+you+need%2E+arXiv+preprint+arXiv%3A1911%2E02150+%282019%29%2E)
304. Noam Shazeer. 2020. Glu variants improve transformer. arXiv preprint arXiv:2002.05202 (2020). [find](https://scholar.google.com/scholar?q=Noam+Shazeer%2E+2020%2E+Glu+variants+improve+transformer%2E+arXiv+preprint+arXiv%3A2002%2E05202+%282020%29%2E)
305. Bowen Shen, Zheng Lin, Yuanxin Liu, Zhengxiao Liu, Lei Wang, and Weiping Wang. 2022. COST-EFF: Collaborative Optimization of Spatial and Temporal Efficiency with Slenderized Multi-exit Language Models. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, Yoav Go… [find](https://scholar.google.com/scholar?q=Bowen+Shen%2C+Zheng+Lin%2C+Yuanxin+Liu%2C+Zhengxiao+Liu%2C+Lei+Wang%2C+and+Weiping+Wang%2E+2022%2E+COST%2DEFF%3A+Collaborative+Optimization+of+Spatial+and+Temporal+Efficiency+with+Slenderized+Multi%2Dexit+Language+Models)
306. Bowen Shen, Zheng Lin, Daren Zha, Wei Liu, Jian Luan, Bin Wang, and Weiping Wang. 2024. Pruning Large Language Models to Intra-module Low-rank Architecture with Transitional Activations. In Findings of the Association for Computational Linguistics: ACL 2024, Lun-Wei Ku, Andre Martins, and Vivek Srik… [find](https://scholar.google.com/scholar?q=Bowen+Shen%2C+Zheng+Lin%2C+Daren+Zha%2C+Wei+Liu%2C+Jian+Luan%2C+Bin+Wang%2C+and+Weiping+Wang%2E+2024%2E+Pruning+Large+Language+Models+to+Intra%2Dmodule+Low%2Drank+Architecture+with+Transitional+Activations%2E+In+Findings+o)
307. Weizhou Shen, Chenliang Li, Hongzhan Chen, Ming Yan, Xiaojun Quan, Hehong Chen, Ji Zhang, and Fei Huang. 2024. [Small LLMs Are Weak Tool Learners: A Multi-LLM Agent](https://md.research.elamur.ai/free/en/papers/shen-2024-small-llms-are-weak-tool/). In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, M…
308. Ying Sheng, Lianmin Zheng, Binhang Yuan, Zhuohan Li, Max Ryabinin, Beidi Chen, Percy Liang, Christopher Ré, Ion Stoica, and Ce Zhang. 2023. [Flexgen: High-throughput generative inference of large language models with a single gpu](https://md.research.elamur.ai/free/en/papers/sheng-2023-flexgen-high-throughput-generative-inference/). In International Conference on Machine Learning. PMLR, 31094–31116.
309. Wentao Shi, Xiangnan He, Yang Zhang, Chongming Gao, Xinyue Li, Jizhi Zhang, Qifan Wang, and Fuli Feng. 2024. Large language models are learnable planners for long-term recommendation. In Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval.… [find](https://scholar.google.com/scholar?q=Wentao+Shi%2C+Xiangnan+He%2C+Yang+Zhang%2C+Chongming+Gao%2C+Xinyue+Li%2C+Jizhi+Zhang%2C+Qifan+Wang%2C+and+Fuli+Feng%2E+2024%2E+Large+language+models+are+learnable+planners+for+long%2Dterm+recommendation%2E+In+Proceedings+o)
310. Mohammad Shoeybi, Mostofa Patwary, Raul Puri, Patrick LeGresley, Jared Casper, and Bryan Catanzaro. 2019. Megatron-lm: Training multi-billion parameter language models using model parallelism. arXiv preprint arXiv:1909.08053 (2019). [find](https://scholar.google.com/scholar?q=Mohammad+Shoeybi%2C+Mostofa+Patwary%2C+Raul+Puri%2C+Patrick+LeGresley%2C+Jared+Casper%2C+and+Bryan+Catanzaro%2E+2019%2E+Megatron%2Dlm%3A+Training+multi%2Dbillion+parameter+language+models+using+model+parallelism%2E+arXiv+p)
311. Parshin Shojaee, Kazem Meidani, Shashank Gupta, Amir Barati Farimani, and Chandan K Reddy. 2024. Llm-sr: Scientific equation discovery via programming with large language models. arXiv preprint arXiv:2404.18400 (2024). [find](https://scholar.google.com/scholar?q=Parshin+Shojaee%2C+Kazem+Meidani%2C+Shashank+Gupta%2C+Amir+Barati+Farimani%2C+and+Chandan+K+Reddy%2E+2024%2E+Llm%2Dsr%3A+Scientific+equation+discovery+via+programming+with+large+language+models%2E+arXiv+preprint+arXiv%3A)
312. Manli Shu, Weili Nie, De-An Huang, Zhiding Yu, Tom Goldstein, Anima Anandkumar, and Chaowei Xiao. 2022. Test-time prompt tuning for zero-shot generalization in vision-language models. Advances in Neural Information Processing Systems 35 (2022), 14274–14289. [find](https://scholar.google.com/scholar?q=Manli+Shu%2C+Weili+Nie%2C+De%2DAn+Huang%2C+Zhiding+Yu%2C+Tom+Goldstein%2C+Anima+Anandkumar%2C+and+Chaowei+Xiao%2E+2022%2E+Test%2Dtime+prompt+tuning+for+zero%2Dshot+generalization+in+vision%2Dlanguage+models%2E+Advances+in+Neur)
313. Karan Singhal, Shekoofeh Azizi, Tao Tu, S Sara Mahdavi, Jason Wei, Hyung Won Chung, Nathan Scales, Ajay Tanwani, Heather Cole-Lewis, Stephen Pfohl, et al. 2023. [Large language models encode clinical knowledge](https://md.research.elamur.ai/free/en/papers/singhal-2022-large-language-models-encode-clinical/). Nature 620, 7972 (2023), 172–180.
314. Daria Soboleva, Faisal Al-Khateeb, Robert Myers, Jacob R Steeves, Joel Hestness, and Nolan Dey. 2023. SlimPajama: A 627B token cleaned and deduplicated version of RedPajama. [https://cerebras.ai/blog/slimpajama-a-627b-token-cleaned-and-deduplicated-version-of-redpajama](https://cerebras.ai/blog/slimpajama-a-627b-token-cleaned-and-deduplicated-version-of-redpajama). https://huggingface.co/dataset… [↗](https://cerebras.ai/blog/slimpajama-a-627b-token-cleaned-and-deduplicated-version-of-redpajama)
315. Luca Soldaini, Rodney Kinney, Akshita Bhagia, Dustin Schwenk, David Atkinson, Russell Authur, Ben Bogin, Khyathi Chandu, Jennifer Dumas, Yanai Elazar, Valentin Hofmann, Ananya Harsh Jha, Sachin Kumar, Li Lucy, Xinxi Lyu, Nathan Lambert, Ian Magnusson, Jacob Morrison, Niklas Muennighoff, Aakanksha Na… [find](https://scholar.google.com/scholar?q=Luca+Soldaini%2C+Rodney+Kinney%2C+Akshita+Bhagia%2C+Dustin+Schwenk%2C+David+Atkinson%2C+Russell+Authur%2C+Ben+Bogin%2C+Khyathi+Chandu%2C+Jennifer+Dumas%2C+Yanai+Elazar%2C+Valentin+Hofmann%2C+Ananya+Harsh+Jha%2C+Sachin+Kumar%2C)
316. Xinying Song, Alex Salcianu, Yang Song, Dave Dopson, and Denny Zhou. 2021. Fast WordPiece Tokenization. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing. 2089–2103. [find](https://scholar.google.com/scholar?q=Xinying+Song%2C+Alex+Salcianu%2C+Yang+Song%2C+Dave+Dopson%2C+and+Denny+Zhou%2E+2021%2E+Fast+WordPiece+Tokenization%2E+In+Proceedings+of+the+2021+Conference+on+Empirical+Methods+in+Natural+Language+Processing%2E+2089%E2%80%93)
317. Sofia Eleni Spatharioti, David M Rothschild, Daniel G Goldstein, and Jake M Hofman. 2023. Comparing traditional and llm-based search for consumer choice: A randomized experiment. arXiv preprint arXiv:2307.03744 (2023). — [arXiv:2307.03744](https://arxiv.org/abs/2307.03744) [find](https://scholar.google.com/scholar?q=Sofia+Eleni+Spatharioti%2C+David+M+Rothschild%2C+Daniel+G+Goldstein%2C+and+Jake+M+Hofman%2E+2023%2E+Comparing+traditional+and+llm%2Dbased+search+for+consumer+choice%3A+A+randomized+experiment%2E+arXiv+preprint+arXiv%3A)
318. Jianlin Su, Murtadha Ahmed, Yu Lu, Shengfeng Pan, Wen Bo, and Yunfeng Liu. 2024. [Roformer: Enhanced transformer with rotary position embedding](https://md.research.elamur.ai/free/en/papers/su-2021-roformer-enhanced-transformer-rotary-position/). Neurocomputing 568 (2024), 127063. Manuscript submitted to ACM
319. Liangtai Sun, Yang Han, Zihan Zhao, Da Ma, Zhennan Shen, Baocai Chen, Lu Chen, and Kai Yu. 2024. [Scieval: A multi-level large language model evaluation benchmark for scientific research](https://md.research.elamur.ai/free/en/papers/sun-2023-scieval-multi-level-large-language/). In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 38. 19053–19061.
320. Lichao Sun, Yue Huang, Haoran Wang, Siyuan Wu, Qihui Zhang, Chujie Gao, Yixin Huang, Wenhan Lyu, Yixuan Zhang, Xiner Li, et al. 2024. [Trustllm: Trustworthiness in large language models](https://md.research.elamur.ai/free/en/papers/huang-2024-trustllm-trustworthiness-large-language-models/). arXiv preprint arXiv:2401.05561 (2024).
321. Mingjie Sun, Zhuang Liu, Anna Bair, and J Zico Kolter. 2024. A Simple and Effective Pruning Approach for Large Language Models. In Proceedings of the Twelfth International Conference on Learning Representations, ICLR. [find](https://scholar.google.com/scholar?q=Mingjie+Sun%2C+Zhuang+Liu%2C+Anna+Bair%2C+and+J+Zico+Kolter%2E+2024%2E+A+Simple+and+Effective+Pruning+Approach+for+Large+Language+Models%2E+In+Proceedings+of+the+Twelfth+International+Conference+on+Learning+Repre)
322. Zhiqing Sun, Hongkun Yu, Xiaodan Song, Renjie Liu, Yiming Yang, and Denny Zhou. 2020. [Mobilebert: a compact task-agnostic bert for resource-limited devices](https://md.research.elamur.ai/free/en/papers/sun-2020-mobilebert-compact-task-agnostic-bert/). arXiv preprint arXiv:2004.02984 (2020).
323. Swabha Swayamdipta, Roy Schwartz, Nicholas Lourie, Yizhong Wang, Hannaneh Hajishirzi, Noah A Smith, and Yejin Choi. 2020. Dataset Cartography: Mapping and Diagnosing Datasets with Training Dynamics. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP). 92… [find](https://scholar.google.com/scholar?q=Swabha+Swayamdipta%2C+Roy+Schwartz%2C+Nicholas+Lourie%2C+Yizhong+Wang%2C+Hannaneh+Hajishirzi%2C+Noah+A+Smith%2C+and+Yejin+Choi%2E+2020%2E+Dataset+Cartography%3A+Mapping+and+Diagnosing+Datasets+with+Training+Dynamics%2E+I)
324. Alon Talmor and Jonathan Berant. 2018. [The Web as a Knowledge-Base for Answering Complex Questions](https://md.research.elamur.ai/free/en/papers/talmor-2018-web-as-knowledge-base-answering/). In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), Marilyn Walker, Heng Ji, and…
325. Jiejun Tan, Zhicheng Dou, Yutao Zhu, Peidong Guo, Kun Fang, and Ji-Rong Wen. 2024. Small Models, Big Insights: Leveraging Slim Proxy Models To Decide When and What to Retrieve for LLMs. arXiv preprint arXiv:2402.12052 (2024). [find](https://scholar.google.com/scholar?q=Jiejun+Tan%2C+Zhicheng+Dou%2C+Yutao+Zhu%2C+Peidong+Guo%2C+Kun+Fang%2C+and+Ji%2DRong+Wen%2E+2024%2E+Small+Models%2C+Big+Insights%3A+Leveraging+Slim+Proxy+Models+To+Decide+When+and+What+to+Retrieve+for+LLMs%2E+arXiv+preprint)
326. Qiaoyu Tang, Ziliang Deng, Hongyu Lin, Xianpei Han, Qiao Liang, Boxi Cao, and Le Sun. 2023. [Toolalpaca: Generalized tool learning for language models with 3000 simulated cases](https://md.research.elamur.ai/free/en/papers/tang-2023-toolalpaca-generalized-tool-learning-language/). arXiv preprint arXiv:2306.05301 (2023).
327. Xuemei Tang, Jun Wang, and Qi Su. 2024. Small Language Model Is a Good Guide for Large Language Model in Chinese Entity Relation Extraction. arXiv preprint arXiv:2402.14373 (2024). [find](https://scholar.google.com/scholar?q=Xuemei+Tang%2C+Jun+Wang%2C+and+Qi+Su%2E+2024%2E+Small+Language+Model+Is+a+Good+Guide+for+Large+Language+Model+in+Chinese+Entity+Relation+Extraction%2E+arXiv+preprint+arXiv%3A2402%2E14373+%282024%29%2E)
328. Yehui Tang, Fangcheng Liu, Yunsheng Ni, Yuchuan Tian, Zheyuan Bai, Yi-Qi Hu, Sichao Liu, Shangling Jui, Kai Han, and Yunhe Wang. 2024. [Rethinking optimization and architecture for tiny language models](https://md.research.elamur.ai/free/en/papers/tang-2024-pangu-pi-pro-rethinking/). arXiv preprint arXiv:2402.02791 (2024).
329. Rohan Taori, Ishaan Gulrajani, Tianyi Zhang, Yann Dubois, Xuechen Li, Carlos Guestrin, Percy Liang, and Tatsunori B. Hashimoto. 2023. Stanford Alpaca: An Instruction-following LLaMA model. [https://github.com/tatsu-lab/stanford_alpaca](https://github.com/tatsu-lab/stanford_alpaca). [↗](https://github.com/tatsu-lab/stanford_alpaca)
330. Ross Taylor, Marcin Kardas, Guillem Cucurull, Thomas Scialom, Anthony Hartshorn, Elvis Saravia, Andrew Poulton, Viktor Kerkez, and Robert Stojnic. 2022. [Galactica: A large language model for science](https://md.research.elamur.ai/free/en/papers/taylor-2022-galactica-large-language-model-science/). arXiv preprint arXiv:2211.09085 (2022).
331. CodeGemma Team. 2024. [Codegemma: Open code models based on gemma](https://md.research.elamur.ai/free/en/papers/zhao-2024-codegemma-open-code-models-based/). arXiv preprint arXiv:2406.11409 (2024).
332. Gemma Team, Thomas Mesnard, Cassidy Hardin, Robert Dadashi, Surya Bhupatiraju, Shreya Pathak, Laurent Sifre, Morgane Rivière, Mihir Sanjay Kale, Juliette Love, et al. 2024. Gemma: Open models based on gemini research and technology. arXiv preprint arXiv:2403.08295 (2024). [find](https://scholar.google.com/scholar?q=Gemma+Team%2C+Thomas+Mesnard%2C+Cassidy+Hardin%2C+Robert+Dadashi%2C+Surya+Bhupatiraju%2C+Shreya+Pathak%2C+Laurent+Sifre%2C+Morgane+Rivi%C3%A8re%2C+Mihir+Sanjay+Kale%2C+Juliette+Love%2C+et+al%2E+2024%2E+Gemma%3A+Open+models+based+on)
333. Gemma Team, Morgane Riviere, Shreya Pathak, Pier Giuseppe Sessa, Cassidy Hardin, Surya Bhupatiraju, Léonard Hussenot, Thomas Mesnard, Bobak Shahriari, Alexandre Ramé, et al. 2024. Gemma 2: Improving open language models at a practical size. arXiv preprint arXiv:2408.00118 (2024). [find](https://scholar.google.com/scholar?q=Gemma+Team%2C+Morgane+Riviere%2C+Shreya+Pathak%2C+Pier+Giuseppe+Sessa%2C+Cassidy+Hardin%2C+Surya+Bhupatiraju%2C+L%C3%A9onard+Hussenot%2C+Thomas+Mesnard%2C+Bobak+Shahriari%2C+Alexandre+Ram%C3%A9%2C+et+al%2E+2024%2E+Gemma+2%3A+Improving+o)
334. TensorOpera Team. 2024. TensorOpera Unveils Fox Foundation Model: A Pioneering Small Language Model (SLM) for Cloud and Edge. [https://blog.tensoropera.ai/tensoropera-unveils-fox-foundation-model-a-pioneering-open-source-slm-leading-the-way-against-tech-giants/](https://blog.tensoropera.ai/tensoropera-unveils-fox-foundation-model-a-pioneering-open-source-slm-leading-the-way-against-tech-giants/) Accessed: 2024-6-13. [↗](https://blog.tensoropera.ai/tensoropera-unveils-fox-foundation-model-a-pioneering-open-source-slm-leading-the-way-against-tech-giants/)
335. Surat Teerapittayanon, Bradley McDanel, and Hsiang-Tsung Kung. 2016. Branchynet: Fast inference via early exiting from deep neural networks. In 2016 23rd international conference on pattern recognition (ICPR). IEEE, 2464–2469. [find](https://scholar.google.com/scholar?q=Surat+Teerapittayanon%2C+Bradley+McDanel%2C+and+Hsiang%2DTsung+Kung%2E+2016%2E+Branchynet%3A+Fast+inference+via+early+exiting+from+deep+neural+networks%2E+In+2016+23rd+international+conference+on+pattern+recognitio)
336. Omkar Thawakar, Ashmal Vayani, Salman Khan, Hisham Cholakal, Rao M Anwer, Michael Felsberg, Tim Baldwin, Eric P Xing, and Fahad Shahbaz Khan. 2024. Mobillama: Towards accurate and lightweight fully transparent gpt. arXiv preprint arXiv:2402.16840 (2024). — [arXiv:2402.16840](https://arxiv.org/abs/2402.16840) [find](https://scholar.google.com/scholar?q=Omkar+Thawakar%2C+Ashmal+Vayani%2C+Salman+Khan%2C+Hisham+Cholakal%2C+Rao+M+Anwer%2C+Michael+Felsberg%2C+Tim+Baldwin%2C+Eric+P+Xing%2C+and+Fahad+Shahbaz+Khan%2E+2024%2E+Mobillama%3A+Towards+accurate+and+lightweight+fully+tr)
337. Ye Tian, Baolin Peng, Linfeng Song, Lifeng Jin, Dian Yu, Haitao Mi, and Dong Yu. 2024. [Toward Self-Improvement of LLMs via Imagination, Searching, and Criticizing](https://md.research.elamur.ai/free/en/papers/tian-2024-toward-self-improvement-llms-imagination/). arXiv preprint arXivko2024distillm:2404.12253 (2024).
338. Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al. 2023. [Llama: Open and efficient foundation language models](https://md.research.elamur.ai/free/en/papers/touvron-2023-llama-open-efficient-foundation-language/). arXiv preprint arXiv:2302.13971 (2023).
339. Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. 2023. [Llama 2: Open foundation and fine-tuned chat models](https://md.research.elamur.ai/free/en/papers/touvron-2023-llama-2-open-foundation-fine/). arXiv preprint arXiv:2307.09288 (2023).
340. Jonathan Tow, Marco Bellagente, Dakota Mahan, and Carlos Riquelme. [n. d.]. StableLM 3B 4E1T. [https://huggingface.co/stabilityai/stablelm-3b-4e1t](https://huggingface.co/stabilityai/stablelm-3b-4e1t) [find](https://scholar.google.com/scholar?q=Jonathan+Tow%2C+Marco+Bellagente%2C+Dakota+Mahan%2C+and+Carlos+Riquelme%2E+%5Bn%2E+d%2E%5D%2E+StableLM+3B+4E1T%2E+%5Bhttps%3A%2F%2Fhuggingface%2Eco%2Fstabilityai%2Fstablelm%2D3b%2D4e1t%5D%28https%3A%2F%2Fhuggingface%2Eco%2Fstabilityai%2Fstablelm%2D3b%2D4e1t%29)
341. Trieu H Trinh and Quoc V Le. 2018. A simple method for commonsense reasoning. arXiv preprint arXiv:1806.02847 (2018). [find](https://scholar.google.com/scholar?q=Trieu+H+Trinh+and+Quoc+V+Le%2E+2018%2E+A+simple+method+for+commonsense+reasoning%2E+arXiv+preprint+arXiv%3A1806%2E02847+%282018%29%2E)
342. Adina Trufinescu. 2024. Discover the New Multi-Lingual High-Quality Phi-3.5 SLMs. [https://techcommunity.microsoft.com/t5/ai-azure-ai-services-blog/discover-the-new-multi-lingual-high-quality-phi-3-5-slms/ba-p/4225280](https://techcommunity.microsoft.com/t5/ai-azure-ai-services-blog/discover-the-new-multi-lingual-high-quality-phi-3-5-slms/ba-p/4225280). [↗](https://techcommunity.microsoft.com/t5/ai-azure-ai-services-blog/discover-the-new-multi-lingual-high-quality-phi-3-5-slms/ba-p/4225280)
343. Dennis Ulmer, Martin Gubri, Hwaran Lee, Sangdoo Yun, and Seong Joon Oh. 2024. Calibrating Large Language Models Using Their Generations Only. arXiv preprint arXiv:2403.05973 (2024). [find](https://scholar.google.com/scholar?q=Dennis+Ulmer%2C+Martin+Gubri%2C+Hwaran+Lee%2C+Sangdoo+Yun%2C+and+Seong+Joon+Oh%2E+2024%2E+Calibrating+Large+Language+Models+Using+Their+Generations+Only%2E+arXiv+preprint+arXiv%3A2403%2E05973+%282024%29%2E)
344. Sander Van Der Linden. 2022. Misinformation: susceptibility, spread, and interventions to immunize the public. Nature medicine 28, 3 (2022), 460–467. [find](https://scholar.google.com/scholar?q=Sander+Van+Der+Linden%2E+2022%2E+Misinformation%3A+susceptibility%2C+spread%2C+and+interventions+to+immunize+the+public%2E+Nature+medicine+28%2C+3+%282022%29%2C+460%E2%80%93467%2E)
345. Chien Van Nguyen, Xuan Shen, Ryan Aponte, Yu Xia, Samyadeep Basu, Zhengmian Hu, Jian Chen, Mihir Parmar, Sasidhar Kunapuli, Joe Barrow, et al. 2024. [A Survey of Small Language Models](https://md.research.elamur.ai/free/en/papers/nguyen-2024-survey-small-language-models/). arXiv preprint arXiv:2410.20011 (2024).
346. A Vaswani. 2017. [Attention is all you need](https://md.research.elamur.ai/free/en/papers/vaswani-2017-attention-all-need/). Advances in Neural Information Processing Systems (2017). Manuscript submitted to ACM
347. Olga Veksler. 2023. Test time adaptation with regularized loss for weakly supervised salient object detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 7360–7369. [find](https://scholar.google.com/scholar?q=Olga+Veksler%2E+2023%2E+Test+time+adaptation+with+regularized+loss+for+weakly+supervised+salient+object+detection%2E+In+Proceedings+of+the+IEEE%2FCVF+Conference+on+Computer+Vision+and+Pattern+Recognition%2E+736)
348. Paul Voigt and Axel Von dem Bussche. 2017. The eu general data protection regulation (gdpr). A Practical Guide, 1st Ed., Cham: Springer International Publishing 10, 3152676 (2017), 10–5555. [find](https://scholar.google.com/scholar?q=Paul+Voigt+and+Axel+Von+dem+Bussche%2E+2017%2E+The+eu+general+data+protection+regulation+%28gdpr%29%2E+A+Practical+Guide%2C+1st+Ed%2E%2C+Cham%3A+Springer+International+Publishing+10%2C+3152676+%282017%29%2C+10%E2%80%935555%2E)
349. Yuxian Wan, Wenlin Zhang, and Zhen Li. 2023. Multi-Task Feature Self-Distillation for Semi-Supervised Machine Translation. In International Conference on Neural Information Processing. Springer, 238–254. [find](https://scholar.google.com/scholar?q=Yuxian+Wan%2C+Wenlin+Zhang%2C+and+Zhen+Li%2E+2023%2E+Multi%2DTask+Feature+Self%2DDistillation+for+Semi%2DSupervised+Machine+Translation%2E+In+International+Conference+on+Neural+Information+Processing%2E+Springer%2C+238%E2%80%932)
350. Alex Wang, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel Bowman. 2018. [GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding](https://md.research.elamur.ai/free/en/papers/wang-2018-glue-multi-task-benchmark-analysis/). In Proceedings of the 2018 EMNLP Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP. 353–355.
351. Boxin Wang, Weixin Chen, Hengzhi Pei, Chulin Xie, Mintong Kang, Chenhui Zhang, Chejian Xu, Zidi Xiong, Ritik Dutta, Rylan Schaeffer, Sang T. Truong, Simran Arora, Mantas Mazeika, Dan Hendrycks, Zinan Lin, Yu Cheng, Sanmi Koyejo, Dawn Song, and Bo Li. 2023. [DecodingTrust: A Comprehensive Assessment o…](https://md.research.elamur.ai/free/en/papers/wang-2023-decodingtrust-comprehensive-assessment-trustworthiness-gpt/)
352. Boxin Wang, Chejian Xu, Shuohang Wang, Zhe Gan, Yu Cheng, Jianfeng Gao, Ahmed Hassan Awadallah, and Bo Li. 2021. Adversarial glue: A multi-task benchmark for robustness evaluation of language models. arXiv preprint arXiv:2111.02840 (2021). [find](https://scholar.google.com/scholar?q=Boxin+Wang%2C+Chejian+Xu%2C+Shuohang+Wang%2C+Zhe+Gan%2C+Yu+Cheng%2C+Jianfeng+Gao%2C+Ahmed+Hassan+Awadallah%2C+and+Bo+Li%2E+2021%2E+Adversarial+glue%3A+A+multi%2Dtask+benchmark+for+robustness+evaluation+of+language+models%2E+)
353. Fali Wang, Runxue Bao, Suhang Wang, Wenchao Yu, Yanchi Liu, Wei Cheng, and Haifeng Chen. 2024. InfuserKI: Enhancing Large Language Models with Knowledge Graphs via Infuser-Guided Knowledge Integration. arXiv preprint arXiv:2402.11441 (2024). [find](https://scholar.google.com/scholar?q=Fali+Wang%2C+Runxue+Bao%2C+Suhang+Wang%2C+Wenchao+Yu%2C+Yanchi+Liu%2C+Wei+Cheng%2C+and+Haifeng+Chen%2E+2024%2E+InfuserKI%3A+Enhancing+Large+Language+Models+with+Knowledge+Graphs+via+Infuser%2DGuided+Knowledge+Integration)
354. Fali Wang, Zheng Lin, Zhengxiao Liu, Mingyu Zheng, Lei Wang, and Daren Zha. 2021. Macrobert: Maximizing certified region of bert to adversarial word substitutions. In Database Systems for Advanced Applications: 26th International Conference, DASFAA 2021, Taipei, Taiwan, April 11–14, 2021, Proceeding… [find](https://scholar.google.com/scholar?q=Fali+Wang%2C+Zheng+Lin%2C+Zhengxiao+Liu%2C+Mingyu+Zheng%2C+Lei+Wang%2C+and+Daren+Zha%2E+2021%2E+Macrobert%3A+Maximizing+certified+region+of+bert+to+adversarial+word+substitutions%2E+In+Database+Systems+for+Advanced+App)
355. Guan Wang, Sijie Cheng, Qiying Yu, and Changling Liu. 2023. OpenLLMs: Less is More for Open-source Models. https://doi.org/10.5281/zenodo.8105775 — [doi:10.5281/zenodo.8105775](https://doi.org/10.5281/zenodo.8105775) [↗](https://doi.org/10.5281/zenodo.8105775)
356. Guan Wang, Sijie Cheng, Xianyuan Zhan, Xiangang Li, Sen Song, and Yang Liu. 2024. [OpenChat: Advancing Open-source Language Models with Mixed-Quality Data](https://md.research.elamur.ai/free/en/papers/wang-2023-openchat-advancing-open-source-language/). In The Twelfth International Conference on Learning Representations. [https://openreview.net/forum?id=AOJyfhWYHf](https://openreview.net/forum?id=AOJyfhWYHf) [↗](https://openreview.net/forum?id=AOJyfhWYHf)
357. Hongyu Wang, Shuming Ma, Li Dong, Shaohan Huang, Huaijie Wang, Lingxiao Ma, Fan Yang, Ruiping Wang, Yi Wu, and Furu Wei. 2023. Bitnet: Scaling 1-bit transformers for large language models. arXiv preprint arXiv:2310.11453 (2023). [find](https://scholar.google.com/scholar?q=Hongyu+Wang%2C+Shuming+Ma%2C+Li+Dong%2C+Shaohan+Huang%2C+Huaijie+Wang%2C+Lingxiao+Ma%2C+Fan+Yang%2C+Ruiping+Wang%2C+Yi+Wu%2C+and+Furu+Wei%2E+2023%2E+Bitnet%3A+Scaling+1%2Dbit+transformers+for+large+language+models%2E+arXiv+prepr)
358. Jindong Wang, HU Xixu, Wenxin Hou, Hao Chen, Runkai Zheng, Yidong Wang, Linyi Yang, Wei Ye, Haojun Huang, Xiubo Geng, et al. [n. d.]. [On the Robustness of ChatGPT: An Adversarial and Out-of-distribution Perspective](https://md.research.elamur.ai/free/en/papers/wang-2023-robustness-chatgpt-adversarial-out-distribution/). In ICLR 2023 Workshop on Trustworthy and Reliable Large-Scale Machine Learning Model…
359. Liang Wang, Nan Yang, Xiaolong Huang, Linjun Yang, Rangan Majumder, and Furu Wei. 2024. [Improving Text Embeddings with Large Language Models](https://md.research.elamur.ai/free/en/papers/wang-2024-improving-text-embeddings-large-language/). In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Association for Computational Linguistics,…
360. Peifeng Wang, Zhengyang Wang, Zheng Li, Yifan Gao, Bing Yin, and Xiang Ren. 2023. [SCOTT: Self-Consistent Chain-of-Thought Distillation](https://md.research.elamur.ai/free/en/papers/wang-2023-scott-self-consistent-chain-thought/). In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 5546–5558.
361. Wenxiao Wang, Wei Chen, Yicong Luo, Yongliu Long, Zhengkai Lin, Liye Zhang, Binbin Lin, Deng Cai, and Xiaofei He. 2024. Model compression and efficient inference for [large language models: A survey](https://md.research.elamur.ai/free/en/papers/minaee-2024-large-language-models-survey/). arXiv preprint arXiv:2402.09748 (2024). — [arXiv:2402.09748](https://arxiv.org/abs/2402.09748)
362. Wenhui Wang, Furu Wei, Li Dong, Hangbo Bao, Nan Yang, and Ming Zhou. 2020. MiniLM: Deep Self-Attention Distillation for Task-Agnostic Compression of Pre-Trained Transformers. arXiv:2002.10957 [cs.CL] https://arxiv.org/abs/2002.10957 [↗](https://arxiv.org/abs/2002.10957)
363. Xiaoxuan Wang, Ziniu Hu, Pan Lu, Yanqiao Zhu, Jieyu Zhang, Satyen Subramaniam, Arjun R Loomba, Shichang Zhang, Yizhou Sun, and Wei Wang. 2024. [SciBench: Evaluating College-Level Scientific Problem-Solving Abilities of Large Language Models](https://md.research.elamur.ai/free/en/papers/wang-2023-scibench-evaluating-college-level-scientific/). In Forty-first International Conference on Machine Learning…
364. Yuxia Wang, Haonan Li, Xudong Han, Preslav Nakov, and Timothy Baldwin. 2024. [Do-Not-Answer: Evaluating Safeguards in LLMs](https://md.research.elamur.ai/free/en/papers/wang-2024-do-not-answer-evaluating-safeguards/). In Findings of the Association for Computational Linguistics: EACL 2024. Association for Computational Linguistics, 896–911. https://aclanthology.org/2024.findings-eacl.61 [↗](https://aclanthology.org/2024.findings-eacl.61)
365. Yile Wang, Peng Li, Maosong Sun, and Yang Liu. 2023. [Self-Knowledge Guided Retrieval Augmentation for Large Language Models](https://md.research.elamur.ai/free/en/papers/wang-2023-self-knowledge-guided-retrieval-augmentation/). In Findings of the Association for Computational Linguistics: EMNLP 2023. 10303–10315.
366. Yubo Wang, Xueguang Ma, and Wenhu Chen. 2023. Augmenting black-box llms with medical textbooks for clinical question answering. arXiv preprint arXiv:2309.02233 (2023). — [arXiv:2309.02233](https://arxiv.org/abs/2309.02233) [find](https://scholar.google.com/scholar?q=Yubo+Wang%2C+Xueguang+Ma%2C+and+Wenhu+Chen%2E+2023%2E+Augmenting+black%2Dbox+llms+with+medical+textbooks+for+clinical+question+answering%2E+arXiv+preprint+arXiv%3A2309%2E02233+%282023%29%2E+%E2%80%94+%5BarXiv%3A2309%2E02233%5D%28https%3A%2F%2Farx)
367. Yizhong Wang, Swaroop Mishra, Pegah Alipoormolabashi, Yeganeh Kordi, Amirreza Mirzaei, Atharva Naik, Arjun Ashok, Arut Selvan Dhanasekaran, Anjana Arunkumar, David Stap, Eshaan Pathak, Giannis Karamanolakis, Haizhi Lai, Ishan Purohit, Ishani Mondal, Jacob Anderson, Kirby Kuznia, Krima Doshi, Kuntal… [find](https://scholar.google.com/scholar?q=Yizhong+Wang%2C+Swaroop+Mishra%2C+Pegah+Alipoormolabashi%2C+Yeganeh+Kordi%2C+Amirreza+Mirzaei%2C+Atharva+Naik%2C+Arjun+Ashok%2C+Arut+Selvan+Dhanasekaran%2C+Anjana+Arunkumar%2C+David+Stap%2C+Eshaan+Pathak%2C+Giannis+Karaman)
368. Yuyan Wang, Mohit Sharma, Can Xu, Sriraj Badam, Qian Sun, Lee Richardson, Lisa Chung, Ed H. Chi, and Minmin Chen. 2022. Surrogate for Long-Term User Experience in Recommender Systems. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining. Association for Computing M… — [doi:10.1145/3534678.3539073](https://doi.org/10.1145/3534678.3539073) [find](https://scholar.google.com/scholar?q=Yuyan+Wang%2C+Mohit+Sharma%2C+Can+Xu%2C+Sriraj+Badam%2C+Qian+Sun%2C+Lee+Richardson%2C+Lisa+Chung%2C+Ed+H%2E+Chi%2C+and+Minmin+Chen%2E+2022%2E+Surrogate+for+Long%2DTerm+User+Experience+in+Recommender+Systems%2E+In+Proceedings+o)
369. Yuling Wang, Changxin Tian, Binbin Hu, Yanhua Yu, Ziqi Liu, Zhiqiang Zhang, Jun Zhou, Liang Pang, and Xiao Wang. 2024. Can Small Language Models be Good Reasoners for Sequential Recommendation?. In Proceedings of the ACM on Web Conference 2024. 3876–3887. [find](https://scholar.google.com/scholar?q=Yuling+Wang%2C+Changxin+Tian%2C+Binbin+Hu%2C+Yanhua+Yu%2C+Ziqi+Liu%2C+Zhiqiang+Zhang%2C+Jun+Zhou%2C+Liang+Pang%2C+and+Xiao+Wang%2E+2024%2E+Can+Small+Language+Models+be+Good+Reasoners+for+Sequential+Recommendation%3F%2E+In+Pr)
370. Yuqing Wang and Yun Zhao. 2024. RUPBench: Benchmarking Reasoning Under Perturbations for Robustness Evaluation in Large Language Models. arXiv preprint arXiv:2406.11020 (2024). — [arXiv:2406.11020](https://arxiv.org/abs/2406.11020) [find](https://scholar.google.com/scholar?q=Yuqing+Wang+and+Yun+Zhao%2E+2024%2E+RUPBench%3A+Benchmarking+Reasoning+Under+Perturbations+for+Robustness+Evaluation+in+Large+Language+Models%2E+arXiv+preprint+arXiv%3A2406%2E11020+%282024%29%2E+%E2%80%94+%5BarXiv%3A2406%2E11020%5D%28ht)
371. Zhepeng Wang, Runxue Bao, Yawen Wu, Jackson Taylor, Cao Xiao, Feng Zheng, Weiwen Jiang, Shangqian Gao, and Yanfu Zhang. 2024. Unlocking Memorization in Large Language Models with Dynamic Soft Prompting. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, Yaser… [find](https://scholar.google.com/scholar?q=Zhepeng+Wang%2C+Runxue+Bao%2C+Yawen+Wu%2C+Jackson+Taylor%2C+Cao+Xiao%2C+Feng+Zheng%2C+Weiwen+Jiang%2C+Shangqian+Gao%2C+and+Yanfu+Zhang%2E+2024%2E+Unlocking+Memorization+in+Large+Language+Models+with+Dynamic+Soft+Promptin)
372. Jason Wei, Maarten Bosma, Vincent Zhao, Kelvin Guu, Adams Wei Yu, Brian Lester, Nan Du, Andrew M. Dai, and Quoc V Le. 2022. [Finetuned Language Models are Zero-Shot Learners](https://md.research.elamur.ai/free/en/papers/wei-2021-finetuned-language-models-zero-shot/). In International Conference on Learning Representations. [https://openreview.net/forum?id=gEZrGCozdqR](https://openreview.net/forum?id=gEZrGCozdqR) [↗](https://openreview.net/forum?id=gEZrGCozdqR)
373. Yuxiang Wei, Zhe Wang, Jiawei Liu, Yifeng Ding, and Lingming Zhang. 2023. Magicoder: Source code is all you need. arXiv preprint arXiv:2312.02120 (2023). [find](https://scholar.google.com/scholar?q=Yuxiang+Wei%2C+Zhe+Wang%2C+Jiawei+Liu%2C+Yifeng+Ding%2C+and+Lingming+Zhang%2E+2023%2E+Magicoder%3A+Source+code+is+all+you+need%2E+arXiv+preprint+arXiv%3A2312%2E02120+%282023%29%2E)
374. Johannes Welbl, Amelia Glaese, Jonathan Uesato, Sumanth Dathathri, John Mellor, Lisa Anne Hendricks, Kirsty Anderson, Pushmeet Kohli, Ben Coppin, and Po-Sen Huang. 2021. [Challenges in Detoxifying Language Models](https://md.research.elamur.ai/free/en/papers/welbl-2021-challenges-detoxifying-language-models/). In Findings of the Association for Computational Linguistics: EMNLP 2021. Association f…
375. Johannes Welbl, Nelson F Liu, and Matt Gardner. 2017. [Crowdsourcing Multiple Choice Science Questions](https://md.research.elamur.ai/free/en/papers/welbl-2017-crowdsourcing-multiple-choice-science-questions/). In Proceedings of the 3rd Workshop on Noisy User-generated Text. 94–106.
376. Hao Wen, Yuanchun Li, Guohong Liu, Shanhui Zhao, Tao Yu, Toby Jia-Jun Li, Shiqi Jiang, Yunhao Liu, Yaqin Zhang, and Yunxin Liu. 2024. AutoDroid: LLM-powered Task Automation in Android. arXiv:2308.15272 [cs.AI] https://arxiv.org/abs/2308.15272 — [arXiv:2308.15272](https://arxiv.org/abs/2308.15272) [↗](https://arxiv.org/abs/2308.15272)
377. Alexander Wettig, Aatmik Gupta, Saumya Malik, and Danqi Chen. 2024. [QuRating: Selecting High-Quality Data for Training Language Models](https://md.research.elamur.ai/free/en/papers/wettig-2024-qurating-selecting-high-quality-data/). In Forty-first International Conference on Machine Learning. [https://openreview.net/forum?id=GLGYYqPwjy](https://openreview.net/forum?id=GLGYYqPwjy) [↗](https://openreview.net/forum?id=GLGYYqPwjy)
378. Adina Williams, Nikita Nangia, and Samuel Bowman. 2018. [A Broad-Coverage Challenge Corpus for Sentence Understanding through Inference](https://md.research.elamur.ai/free/en/papers/williams-2018-broad-coverage-challenge-corpus-sentence/). In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Pa…
379. Chuhan Wu, Fangzhao Wu, Tao Qi, and Yongfeng Huang. 2021. Empowering news recommendation with pre-trained language models. In Proceedings of the 44th international ACM SIGIR conference on research and development in information retrieval. 1652–1656. [find](https://scholar.google.com/scholar?q=Chuhan+Wu%2C+Fangzhao+Wu%2C+Tao+Qi%2C+and+Yongfeng+Huang%2E+2021%2E+Empowering+news+recommendation+with+pre%2Dtrained+language+models%2E+In+Proceedings+of+the+44th+international+ACM+SIGIR+conference+on+research+and)
380. Minghao Wu, Abdul Waheed, Chiyu Zhang, Muhammad Abdul-Mageed, and Alham Fikri Aji. 2024. [LaMini-LM: A Diverse Herd of Distilled Models from Large-Scale Instructions](https://md.research.elamur.ai/free/en/papers/wu-2023-lamini-lm-diverse-herd-distilled/). In Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers)…
381. Taiqiang Wu, Cheng Hou, Shanshan Lao, Jiayi Li, Ngai Wong, Zhe Zhao, and Yujiu Yang. 2024. Weight-Inherited Distillation for Task-Agnostic BERT Compression. In Findings of the Association for Computational Linguistics: NAACL 2024, Kevin Duh, Helena Gomez, and Steven Bethard (Eds.). Association for C… [find](https://scholar.google.com/scholar?q=Taiqiang+Wu%2C+Cheng+Hou%2C+Shanshan+Lao%2C+Jiayi+Li%2C+Ngai+Wong%2C+Zhe+Zhao%2C+and+Yujiu+Yang%2E+2024%2E+Weight%2DInherited+Distillation+for+Task%2DAgnostic+BERT+Compression%2E+In+Findings+of+the+Association+for+Computat)
382. Xuansheng Wu, Huachi Zhou, Yucheng Shi, Wenlin Yao, Xiao Huang, and Ninghao Liu. 2024. Could Small Language Models Serve as Recommenders? Towards Data-centric Cold-start Recommendation. In Proceedings of the ACM on Web Conference 2024. 3566–3575. [find](https://scholar.google.com/scholar?q=Xuansheng+Wu%2C+Huachi+Zhou%2C+Yucheng+Shi%2C+Wenlin+Yao%2C+Xiao+Huang%2C+and+Ninghao+Liu%2E+2024%2E+Could+Small+Language+Models+Serve+as+Recommenders%3F+Towards+Data%2Dcentric+Cold%2Dstart+Recommendation%2E+In+Proceedings)
383. Zhuofeng Wu, He Bai, Aonan Zhang, Jiatao Gu, VG Vydiswaran, Navdeep Jaitly, and Yizhe Zhang. 2024. Divide-or-Conquer? Which Part Should You Distill Your LLM? arXiv preprint arXiv:2402.15000 (2024). [find](https://scholar.google.com/scholar?q=Zhuofeng+Wu%2C+He+Bai%2C+Aonan+Zhang%2C+Jiatao+Gu%2C+VG+Vydiswaran%2C+Navdeep+Jaitly%2C+and+Yizhe+Zhang%2E+2024%2E+Divide%2Dor%2DConquer%3F+Which+Part+Should+You+Distill+Your+LLM%3F+arXiv+preprint+arXiv%3A2402%2E15000+%282024%29%2E)
384. Nuwa Xi, Yuhan Chen, Sendong Zhao, Haochun Wang, Bing Qin, and Ting Liu. 2024. AS-ES Learning: Towards Efficient CoT Learning in Small Models. arXiv preprint arXiv:2403.01969 (2024). [find](https://scholar.google.com/scholar?q=Nuwa+Xi%2C+Yuhan+Chen%2C+Sendong+Zhao%2C+Haochun+Wang%2C+Bing+Qin%2C+and+Ting+Liu%2E+2024%2E+AS%2DES+Learning%3A+Towards+Efficient+CoT+Learning+in+Small+Models%2E+arXiv+preprint+arXiv%3A2403%2E01969+%282024%29%2E)
385. Mengzhou Xia, Tianyu Gao, Zhiyuan Zeng, and Danqi Chen. 2024. Sheared LLaMA: Accelerating Language Model Pre-training via Structured Pruning. In The Twelfth International Conference on Learning Representations. [https://openreview.net/forum?id=09iOdaeOzp](https://openreview.net/forum?id=09iOdaeOzp) [↗](https://openreview.net/forum?id=09iOdaeOzp)
386. Guangxuan Xiao, Ji Lin, Mickael Seznec, Hao Wu, Julien Demouth, and Song Han. 2023. Smoothquant: Accurate and efficient post-training quantization for large language models. In International Conference on Machine Learning. PMLR, 38087–38099. [find](https://scholar.google.com/scholar?q=Guangxuan+Xiao%2C+Ji+Lin%2C+Mickael+Seznec%2C+Hao+Wu%2C+Julien+Demouth%2C+and+Song+Han%2E+2023%2E+Smoothquant%3A+Accurate+and+efficient+post%2Dtraining+quantization+for+large+language+models%2E+In+International+Conferenc)
387. Tinghao Xie, Xiangyu Qi, Yi Zeng, Yangsibo Huang, Udari Madhushani Sehwag, Kaixuan Huang, Luxi He, Boyi Wei, Dacheng Li, Ying Sheng, et al. 2024. [Sorry-bench: Systematically evaluating large language model safety refusal](https://md.research.elamur.ai/free/en/papers/xie-2024-sorry-bench-systematically-evaluating-large/) behaviors. arXiv preprint arXiv:2406.14598 (2024).
388. Tong Xie, Yuwei Wan, Wei Huang, Zhenyu Yin, Yixuan Liu, Shaozhou Wang, Qingyuan Linghu, Chunyu Kit, Clara Grazian, Wenjie Zhang, et al. 2023. [Darwin series: Domain specific large language models for natural science](https://md.research.elamur.ai/free/en/papers/xie-2023-darwin-series-domain-specific-large/). arXiv preprint arXiv:2308.13565 (2023).
389. Weikai Xie, Li Zhang, Shihe Wang, Rongjie Yi, and Mengwei Xu. 2024. DroidCall: A Dataset for LLM-powered Android Intent Invocation. arXiv preprint arXiv:2412.00402 (2024). [find](https://scholar.google.com/scholar?q=Weikai+Xie%2C+Li+Zhang%2C+Shihe+Wang%2C+Rongjie+Yi%2C+and+Mengwei+Xu%2E+2024%2E+DroidCall%3A+A+Dataset+for+LLM%2Dpowered+Android+Intent+Invocation%2E+arXiv+preprint+arXiv%3A2412%2E00402+%282024%29%2E)
390. Xuan Xie, Jiayang Song, Zhehua Zhou, Yuheng Huang, Da Song, and Lei Ma. 2024. Online Safety Analysis for LLMs: a Benchmark, an Assessment, and a Path Forward. arXiv preprint arXiv:2404.08517 (2024). — [arXiv:2404.08517](https://arxiv.org/abs/2404.08517) [find](https://scholar.google.com/scholar?q=Xuan+Xie%2C+Jiayang+Song%2C+Zhehua+Zhou%2C+Yuheng+Huang%2C+Da+Song%2C+and+Lei+Ma%2E+2024%2E+Online+Safety+Analysis+for+LLMs%3A+a+Benchmark%2C+an+Assessment%2C+and+a+Path+Forward%2E+arXiv+preprint+arXiv%3A2404%2E08517+%282024%29%2E+%E2%80%94)
391. Ji Xin, Raphael Tang, Yaoliang Yu, and Jimmy Lin. 2021. BERxiT: Early Exiting for BERT with Better Fine-Tuning and Extension to Regression. In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume, Paola Merlo, Jorg Tiedemann, and Re… [find](https://scholar.google.com/scholar?q=Ji+Xin%2C+Raphael+Tang%2C+Yaoliang+Yu%2C+and+Jimmy+Lin%2E+2021%2E+BERxiT%3A+Early+Exiting+for+BERT+with+Better+Fine%2DTuning+and+Extension+to+Regression%2E+In+Proceedings+of+the+16th+Conference+of+the+European+Chapte)
392. Can Xu, Qingfeng Sun, Kai Zheng, Xiubo Geng, Pu Zhao, Jiazhan Feng, Chongyang Tao, and Daxin Jiang. 2023. [Wizardlm: Empowering large language models to follow complex instructions](https://md.research.elamur.ai/free/en/papers/xu-2023-wizardlm-empowering-large-pre-trained/). arXiv preprint arXiv:2304.12244 (2023). Manuscript submitted to ACM
393. Canwen Xu, Yichong Xu, Shuohang Wang, Yang Liu, Chenguang Zhu, and Julian McAuley. 2023. Small models are valuable plug-ins for large language models. arXiv preprint arXiv:2305.08848 (2023). [find](https://scholar.google.com/scholar?q=Canwen+Xu%2C+Yichong+Xu%2C+Shuohang+Wang%2C+Yang+Liu%2C+Chenguang+Zhu%2C+and+Julian+McAuley%2E+2023%2E+Small+models+are+valuable+plug%2Dins+for+large+language+models%2E+arXiv+preprint+arXiv%3A2305%2E08848+%282023%29%2E)
394. Daliang Xu, Wangsong Yin, Xin Jin, Ying Zhang, Shiyun Wei, Mengwei Xu, and Xuanzhe Liu. 2023. LLMCad: Fast and Scalable On-device Large Language Model Inference. arXiv:2309.04255 [cs.NI] https://arxiv.org/abs/2309.04255 — [arXiv:2309.04255](https://arxiv.org/abs/2309.04255) [↗](https://arxiv.org/abs/2309.04255)
395. Daliang Xu, Hao Zhang, Liming Yang, Ruiqi Liu, Gang Huang, Mengwei Xu, and Xuanzhe Liu. 2024. Empowering 1000 tokens/second on-device llm prefilling with mllm-npu. arXiv preprint arXiv:2407.05858 (2024). [find](https://scholar.google.com/scholar?q=Daliang+Xu%2C+Hao+Zhang%2C+Liming+Yang%2C+Ruiqi+Liu%2C+Gang+Huang%2C+Mengwei+Xu%2C+and+Xuanzhe+Liu%2E+2024%2E+Empowering+1000+tokens%2Fsecond+on%2Ddevice+llm+prefilling+with+mllm%2Dnpu%2E+arXiv+preprint+arXiv%3A2407%2E05858+%28202)
396. Jiaming Xu, Peng Wang, Guanhua Tian, Bo Xu, Jun Zhao, Fangyuan Wang, and Hongwei Hao. 2015. [Short Text Clustering via Convolutional Neural Networks](https://md.research.elamur.ai/free/en/papers/xu-2015-short-text-clustering-convolutional-neural/). In Proceedings of the 1st Workshop on Vector Space Modeling for Natural Language Processing, Phil Blunsom, Shay Cohen, Paramveer Dhillon, and Percy Lia…
397. Minrui Xu, Niyato Dusit, Jiawen Kang, Zehui Xiong, Shiwen Mao, Zhu Han, Dong In Kim, and Khaled B Letaief. 2024. When large language model agents meet 6G networks: Perception, grounding, and alignment. arXiv preprint arXiv:2401.07764 (2024). — [arXiv:2401.07764](https://arxiv.org/abs/2401.07764) [find](https://scholar.google.com/scholar?q=Minrui+Xu%2C+Niyato+Dusit%2C+Jiawen+Kang%2C+Zehui+Xiong%2C+Shiwen+Mao%2C+Zhu+Han%2C+Dong+In+Kim%2C+and+Khaled+B+Letaief%2E+2024%2E+When+large+language+model+agents+meet+6G+networks%3A+Perception%2C+grounding%2C+and+alignment)
398. Mengwei Xu, Wangsong Yin, Dongqi Cai, Rongjie Yi, Daliang Xu, Qipeng Wang, Bingyang Wu, Yihao Zhao, Chen Yang, Shihe Wang, et al. 2024. A survey of resource-efficient llm and multimodal foundation models. arXiv preprint arXiv:2401.08092 (2024). [find](https://scholar.google.com/scholar?q=Mengwei+Xu%2C+Wangsong+Yin%2C+Dongqi+Cai%2C+Rongjie+Yi%2C+Daliang+Xu%2C+Qipeng+Wang%2C+Bingyang+Wu%2C+Yihao+Zhao%2C+Chen+Yang%2C+Shihe+Wang%2C+et+al%2E+2024%2E+A+survey+of+resource%2Defficient+llm+and+multimodal+foundation+mod)
399. Weijia Xu, Sweta Agrawal, Eleftheria Briakou, Marianna Martindale, and Marine Carpuat. 2023. [Understanding and Detecting Hallucinations in Neural Machine Translation via Model Introspection](https://md.research.elamur.ai/free/en/papers/xu-2023-understanding-detecting-hallucinations-neural-machine/). Transactions of the Association for Computational Linguistics 11 (2023), 546–564.
400. Yuzhuang Xu, Xu Han, Zonghan Yang, Shuo Wang, Qingfu Zhu, Zhiyuan Liu, Weidong Liu, and Wanxiang Che. 2024. OneBit: Towards Extremely Low-bit Large Language Models. arXiv preprint arXiv:2402.11295 (2024). — [arXiv:2402.11295](https://arxiv.org/abs/2402.11295) [find](https://scholar.google.com/scholar?q=Yuzhuang+Xu%2C+Xu+Han%2C+Zonghan+Yang%2C+Shuo+Wang%2C+Qingfu+Zhu%2C+Zhiyuan+Liu%2C+Weidong+Liu%2C+and+Wanxiang+Che%2E+2024%2E+OneBit%3A+Towards+Extremely+Low%2Dbit+Large+Language+Models%2E+arXiv+preprint+arXiv%3A2402%2E11295+%2820)
401. Shi-Qi Yan, Jia-Chen Gu, Yun Zhu, and Zhen-Hua Ling. 2024. [Corrective retrieval augmented generation](https://md.research.elamur.ai/free/en/papers/yan-2024-corrective-retrieval-augmented-generation/). arXiv preprint arXiv:2401.15884 (2024).
402. An Yang, Baosong Yang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Zhou, Chengpeng Li, Chengyuan Li, Dayiheng Liu, Fei Huang, et al. 2024. Qwen2 technical report. arXiv preprint arXiv:2407.10671 (2024). [find](https://scholar.google.com/scholar?q=An+Yang%2C+Baosong+Yang%2C+Binyuan+Hui%2C+Bo+Zheng%2C+Bowen+Yu%2C+Chang+Zhou%2C+Chengpeng+Li%2C+Chengyuan+Li%2C+Dayiheng+Liu%2C+Fei+Huang%2C+et+al%2E+2024%2E+Qwen2+technical+report%2E+arXiv+preprint+arXiv%3A2407%2E10671+%282024%29%2E)
403. Chuanpeng Yang, Wang Lu, Yao Zhu, Yidong Wang, Qian Chen, Chenlong Gao, Bingjie Yan, and Yiqiang Chen. 2024. Survey on Knowledge Distillation for Large Language Models: Methods, Evaluation, and Application. arXiv preprint arXiv:2407.01885 (2024). — [arXiv:2407.01885](https://arxiv.org/abs/2407.01885) [find](https://scholar.google.com/scholar?q=Chuanpeng+Yang%2C+Wang+Lu%2C+Yao+Zhu%2C+Yidong+Wang%2C+Qian+Chen%2C+Chenlong+Gao%2C+Bingjie+Yan%2C+and+Yiqiang+Chen%2E+2024%2E+Survey+on+Knowledge+Distillation+for+Large+Language+Models%3A+Methods%2C+Evaluation%2C+and+Applic)
404. Cheng Yang, Yang Sui, Jinqi Xiao, Lingyi Huang, Yu Gong, Yuanlin Duan, Wenqi Jia, Miao Yin, Yu Cheng, and Bo Yuan. 2024. MoE-I 2 : Compressing Mixture of Experts Models through Inter-Expert Pruning and Intra-Expert Low-Rank Decomposition. In Findings of the Association for Computational Linguistics:… [find](https://scholar.google.com/scholar?q=Cheng+Yang%2C+Yang+Sui%2C+Jinqi+Xiao%2C+Lingyi+Huang%2C+Yu+Gong%2C+Yuanlin+Duan%2C+Wenqi+Jia%2C+Miao+Yin%2C+Yu+Cheng%2C+and+Bo+Yuan%2E+2024%2E+MoE%2DI+2+%3A+Compressing+Mixture+of+Experts+Models+through+Inter%2DExpert+Pruning+an)
405. Kevin Yang, Dan Klein, Asli Celikyilmaz, Nanyun Peng, and Yuandong Tian. 2024. RLCD: Reinforcement Learning from Contrastive Distillation for LM Alignment. In The Twelfth International Conference on Learning Representations. [https://openreview.net/forum?id=v3XXtxWKi6](https://openreview.net/forum?id=v3XXtxWKi6) [↗](https://openreview.net/forum?id=v3XXtxWKi6)
406. Kailai Yang, Tianlin Zhang, Ziyan Kuang, Qianqian Xie, Jimin Huang, and Sophia Ananiadou. 2024. MentaLLaMA: interpretable mental health analysis on social media with large language models. In Proceedings of the ACM on Web Conference 2024. 4489–4500. [find](https://scholar.google.com/scholar?q=Kailai+Yang%2C+Tianlin+Zhang%2C+Ziyan+Kuang%2C+Qianqian+Xie%2C+Jimin+Huang%2C+and+Sophia+Ananiadou%2E+2024%2E+MentaLLaMA%3A+interpretable+mental+health+analysis+on+social+media+with+large+language+models%2E+In+Proceedi)
407. Linyi Yang, Shuibai Zhang, Libo Qin, Yafu Li, Yidong Wang, Hanmeng Liu, Jindong Wang, Xing Xie, and Yue Zhang. 2023. GLUE-X: Evaluating Natural Language Understanding Models from an Out-of-Distribution Generalization Perspective. In Findings of the Association for Computational Linguistics: ACL 2023… [find](https://scholar.google.com/scholar?q=Linyi+Yang%2C+Shuibai+Zhang%2C+Libo+Qin%2C+Yafu+Li%2C+Yidong+Wang%2C+Hanmeng+Liu%2C+Jindong+Wang%2C+Xing+Xie%2C+and+Yue+Zhang%2E+2023%2E+GLUE%2DX%3A+Evaluating+Natural+Language+Understanding+Models+from+an+Out%2Dof%2DDistributio)
408. Linyi Yang, Shuibai Zhang, Zhuohao Yu, Guangsheng Bao, Yidong Wang, Jindong Wang, Ruochen Xu, Wei Ye, Xing Xie, Weizhu Chen, and Yue Zhang. 2024. Supervised Knowledge Makes Large Language Models Better In-context Learners. In The Twelfth International Conference on Learning Representations. https://… [find](https://scholar.google.com/scholar?q=Linyi+Yang%2C+Shuibai+Zhang%2C+Zhuohao+Yu%2C+Guangsheng+Bao%2C+Yidong+Wang%2C+Jindong+Wang%2C+Ruochen+Xu%2C+Wei+Ye%2C+Xing+Xie%2C+Weizhu+Chen%2C+and+Yue+Zhang%2E+2024%2E+Supervised+Knowledge+Makes+Large+Language+Models+Bette)
409. Runming Yang, Taiqiang Wu, Jiahao Wang, Pengfei Hu, Ngai Wong, and Yujiu Yang. 2024. LLM-Neo: Parameter Efficient Knowledge Distillation for Large Language Models. arXiv preprint arXiv:2411.06839 (2024). — [arXiv:2411.06839](https://arxiv.org/abs/2411.06839) [find](https://scholar.google.com/scholar?q=Runming+Yang%2C+Taiqiang+Wu%2C+Jiahao+Wang%2C+Pengfei+Hu%2C+Ngai+Wong%2C+and+Yujiu+Yang%2E+2024%2E+LLM%2DNeo%3A+Parameter+Efficient+Knowledge+Distillation+for+Large+Language+Models%2E+arXiv+preprint+arXiv%3A2411%2E06839+%28202)
410. Yifei Yang, Zouying Cao, and Hai Zhao. 2024. Laco: Large language model pruning via layer collapse. arXiv preprint arXiv:2402.11187 (2024). [find](https://scholar.google.com/scholar?q=Yifei+Yang%2C+Zouying+Cao%2C+and+Hai+Zhao%2E+2024%2E+Laco%3A+Large+language+model+pruning+via+layer+collapse%2E+arXiv+preprint+arXiv%3A2402%2E11187+%282024%29%2E)
411. Yu Yang, Siddhartha Mishra, Jeffrey N Chiang, and Baharan Mirzasoleiman. 2024. SmallToLarge (S2L): Scalable Data Selection for Fine-tuning Large Language Models by Summarizing Training Trajectories of Small Models. In The Thirty-eighth Annual Conference on Neural Information Processing Systems. http… [find](https://scholar.google.com/scholar?q=Yu+Yang%2C+Siddhartha+Mishra%2C+Jeffrey+N+Chiang%2C+and+Baharan+Mirzasoleiman%2E+2024%2E+SmallToLarge+%28S2L%29%3A+Scalable+Data+Selection+for+Fine%2Dtuning+Large+Language+Models+by+Summarizing+Training+Trajectories+of)
412. Yizhe Yang, Huashan Sun, Jiawei Li, Runheng Liu, Yinghao Li, Yuhang Liu, Heyan Huang, and Yang Gao. 2023. Mindllm: Pre-training lightweight large language model from scratch, evaluations and domain applications. arXiv preprint arXiv:2310.15777 (2023). — [arXiv:2310.15777](https://arxiv.org/abs/2310.15777) [find](https://scholar.google.com/scholar?q=Yizhe+Yang%2C+Huashan+Sun%2C+Jiawei+Li%2C+Runheng+Liu%2C+Yinghao+Li%2C+Yuhang+Liu%2C+Heyan+Huang%2C+and+Yang+Gao%2E+2023%2E+Mindllm%3A+Pre%2Dtraining+lightweight+large+language+model+from+scratch%2C+evaluations+and+domain+ap)
413. Zhou Yang, Zhaochun Ren, Wang Yufeng, Shizhong Peng, Haizhou Sun, Xiaofei Zhu, and Xiangwen Liao. 2024. Enhancing Empathetic Response Generation by Augmenting LLMs with Small-scale Empathetic Models. arXiv preprint arXiv:2402.11801 (2024). — [arXiv:2402.11801](https://arxiv.org/abs/2402.11801) [find](https://scholar.google.com/scholar?q=Zhou+Yang%2C+Zhaochun+Ren%2C+Wang+Yufeng%2C+Shizhong+Peng%2C+Haizhou+Sun%2C+Xiaofei+Zhu%2C+and+Xiangwen+Liao%2E+2024%2E+Enhancing+Empathetic+Response+Generation+by+Augmenting+LLMs+with+Small%2Dscale+Empathetic+Models%2E+)
414. Yunzhi Yao, Shaohan Huang, Wenhui Wang, Li Dong, and Furu Wei. 2021. Adapt-and-distill: Developing small, fast and effective pretrained language models for domains. arXiv preprint arXiv:2106.13474 (2021), 460–470. [find](https://scholar.google.com/scholar?q=Yunzhi+Yao%2C+Shaohan+Huang%2C+Wenhui+Wang%2C+Li+Dong%2C+and+Furu+Wei%2E+2021%2E+Adapt%2Dand%2Ddistill%3A+Developing+small%2C+fast+and+effective+pretrained+language+models+for+domains%2E+arXiv+preprint+arXiv%3A2106%2E13474+%2820)
415. Mert Yazan, Suzan Verberne, and Frederik Situmeang. 2024. The Impact of Quantization on Retrieval-Augmented Generation: An Analysis of Small LLMs. arXiv preprint arXiv:2406.10251 (2024). — [arXiv:2406.10251](https://arxiv.org/abs/2406.10251) [find](https://scholar.google.com/scholar?q=Mert+Yazan%2C+Suzan+Verberne%2C+and+Frederik+Situmeang%2E+2024%2E+The+Impact+of+Quantization+on+Retrieval%2DAugmented+Generation%3A+An+Analysis+of+Small+LLMs%2E+arXiv+preprint+arXiv%3A2406%2E10251+%282024%29%2E+%E2%80%94+%5BarXiv%3A2406)
416. Rongjie Yi, Liwei Guo, Shiyun Wei, Ao Zhou, Shangguang Wang, and Mengwei Xu. 2023. EdgeMoE: Fast On-Device Inference of MoE-based Large Language Models. arXiv:2308.14352 [cs.LG] https://arxiv.org/abs/2308.14352 — [arXiv:2308.14352](https://arxiv.org/abs/2308.14352) [↗](https://arxiv.org/abs/2308.14352)
417. Rongjie Yi, Xiang Li, Weikai Xie, Zhenyan Lu, Chenghua Wang, Ao Zhou, Shangguang Wang, Xiwen Zhang, and Mengwei Xu. 2024. PhoneLM: an Efficient and Capable Small Language Model Family through Principled Pre-training. arXiv preprint arXiv:2411.05046 (2024). — [arXiv:2411.05046](https://arxiv.org/abs/2411.05046) [find](https://scholar.google.com/scholar?q=Rongjie+Yi%2C+Xiang+Li%2C+Weikai+Xie%2C+Zhenyan+Lu%2C+Chenghua+Wang%2C+Ao+Zhou%2C+Shangguang+Wang%2C+Xiwen+Zhang%2C+and+Mengwei+Xu%2E+2024%2E+PhoneLM%3A+an+Efficient+and+Capable+Small+Language+Model+Family+through+Principl)
418. Wen-tau Yih, Matthew Richardson, Chris Meek, Ming-Wei Chang, and Jina Suh. 2016. [The Value of Semantic Parse Labeling for Knowledge Base Question Answering](https://md.research.elamur.ai/free/en/papers/yih-2016-value-semantic-parse-labeling-knowledge/). In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), Katrin Erk Manuscript subm…
419. Wangsong Yin, Mengwei Xu, Yuanchun Li, and Xuanzhe Liu. 2024. LLM as a System Service on Mobile Devices. arXiv:2403.11805 [cs.OS] https://arxiv.org/abs/2403.11805 — [arXiv:2403.11805](https://arxiv.org/abs/2403.11805) [↗](https://arxiv.org/abs/2403.11805)
420. Longhui Yu, Weisen Jiang, Han Shi, Jincheng YU, Zhengying Liu, Yu Zhang, James Kwok, Zhenguo Li, Adrian Weller, and Weiyang Liu. 2024. [MetaMath: Bootstrap Your Own Mathematical Questions for Large Language Models](https://md.research.elamur.ai/free/en/papers/yu-2023-metamath-bootstrap-your-own-mathematical/). In The Twelfth International Conference on Learning Representations. https://openrevie…
421. Yue Yu, Wei Ping, Zihan Liu, Boxin Wang, Jiaxuan You, Chao Zhang, Mohammad Shoeybi, and Bryan Catanzaro. 2024. Rankrag: Unifying context ranking with retrieval-augmented generation in llms. arXiv preprint arXiv:2407.02485 (2024). — [arXiv:2407.02485](https://arxiv.org/abs/2407.02485) [find](https://scholar.google.com/scholar?q=Yue+Yu%2C+Wei+Ping%2C+Zihan+Liu%2C+Boxin+Wang%2C+Jiaxuan+You%2C+Chao+Zhang%2C+Mohammad+Shoeybi%2C+and+Bryan+Catanzaro%2E+2024%2E+Rankrag%3A+Unifying+context+ranking+with+retrieval%2Daugmented+generation+in+llms%2E+arXiv+prep)
422. Zhongzhi Yu, Zheng Wang, Yuhan Li, Haoran You, Ruijie Gao, Xiaoya Zhou, Sreenidhi Reedy Bommu, Yang Katie Zhao, and Yingyan Celine Lin. 2024. EDGE-LLM: Enabling Efficient Large Language Model Adaptation on Edge Devices via Layerwise Unified Compression and Adaptive Layer Tuning and Voting. arXiv pre… — [arXiv:2406.15758](https://arxiv.org/abs/2406.15758) [find](https://scholar.google.com/scholar?q=Zhongzhi+Yu%2C+Zheng+Wang%2C+Yuhan+Li%2C+Haoran+You%2C+Ruijie+Gao%2C+Xiaoya+Zhou%2C+Sreenidhi+Reedy+Bommu%2C+Yang+Katie+Zhao%2C+and+Yingyan+Celine+Lin%2E+2024%2E+EDGE%2DLLM%3A+Enabling+Efficient+Large+Language+Model+Adaptati)
423. Jinliang Yuan, Chen Yang, Dongqi Cai, Shihe Wang, Xin Yuan, Zeling Zhang, Xiang Li, Dingge Zhang, Hanzi Mei, Xianqing Jia, et al. 2024. Mobile Foundation Model as Firmware. In Proceedings of the 30th Annual International Conference on Mobile Computing and Networking. 279–295. [find](https://scholar.google.com/scholar?q=Jinliang+Yuan%2C+Chen+Yang%2C+Dongqi+Cai%2C+Shihe+Wang%2C+Xin+Yuan%2C+Zeling+Zhang%2C+Xiang+Li%2C+Dingge+Zhang%2C+Hanzi+Mei%2C+Xianqing+Jia%2C+et+al%2E+2024%2E+Mobile+Foundation+Model+as+Firmware%2E+In+Proceedings+of+the+30th+)
424. Sha Yuan, Hanyu Zhao, Zhengxiao Du, Ming Ding, Xiao Liu, Yukuo Cen, Xu Zou, Zhilin Yang, and Jie Tang. 2021. Wudaocorpora: A super large-scale chinese corpora for pre-training language models. AI Open 2 (2021), 65–68. [find](https://scholar.google.com/scholar?q=Sha+Yuan%2C+Hanyu+Zhao%2C+Zhengxiao+Du%2C+Ming+Ding%2C+Xiao+Liu%2C+Yukuo+Cen%2C+Xu+Zou%2C+Zhilin+Yang%2C+and+Jie+Tang%2E+2021%2E+Wudaocorpora%3A+A+super+large%2Dscale+chinese+corpora+for+pre%2Dtraining+language+models%2E+AI+Open)
425. Xiaohan Yuan, Jinfeng Li, Dongxia Wang, Yuefeng Chen, Xiaofeng Mao, Longtao Huang, Hui Xue, Wenhai Wang, Kui Ren, and Jingyi Wang. 2024. S-Eval: Automatic and Adaptive Test Generation for Benchmarking Safety Evaluation of Large Language Models. arXiv preprint arXiv:2405.14191 (2024). — [arXiv:2405.14191](https://arxiv.org/abs/2405.14191) [find](https://scholar.google.com/scholar?q=Xiaohan+Yuan%2C+Jinfeng+Li%2C+Dongxia+Wang%2C+Yuefeng+Chen%2C+Xiaofeng+Mao%2C+Longtao+Huang%2C+Hui+Xue%2C+Wenhai+Wang%2C+Kui+Ren%2C+and+Jingyi+Wang%2E+2024%2E+S%2DEval%3A+Automatic+and+Adaptive+Test+Generation+for+Benchmarking)
426. Youliang Yuan, Wenxiang Jiao, Wenxuan Wang, Jen tse Huang, Pinjia He, Shuming Shi, and Zhaopeng Tu. 2024. [GPT-4 Is Too Smart To Be Safe: Stealthy Chat with LLMs via Cipher](https://md.research.elamur.ai/free/en/papers/yuan-2023-gpt-4-too-smart-safe/). In The Twelfth International Conference on Learning Representations. [https://openreview.net/forum?id=MbfAK4s61A](https://openreview.net/forum?id=MbfAK4s61A) [↗](https://openreview.net/forum?id=MbfAK4s61A)
427. Shengbin Yue, Wei Chen, Siyuan Wang, Bingxuan Li, Chenchen Shen, Shujun Liu, Yuxuan Zhou, Yao Xiao, Song Yun, Wei Lin, et al. 2023. [Disc-lawllm: Fine-tuning large language models for intelligent legal services](https://md.research.elamur.ai/free/en/papers/yue-2023-disc-lawllm-fine-tuning-large/). arXiv preprint arXiv:2309.11325 (2023).
428. Xiang Yue, Xingwei Qu, Ge Zhang, Yao Fu, Wenhao Huang, Huan Sun, Yu Su, and Wenhu Chen. [n. d.]. [Mammoth: Building math generalist models through hybrid instruction tuning](https://md.research.elamur.ai/free/en/papers/yue-2023-mammoth-building-math-generalist-models/), 2023. URL https://arxiv.org/abs/2309.05653 ([n. d.]). [↗](https://arxiv.org/abs/2309.05653)
429. Rowan Zellers, Ari Holtzman, Yonatan Bisk, Ali Farhadi, and Yejin Choi. 2019. HellaSwag: Can a Machine Really Finish Your Sentence?. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. 4791–4800. [find](https://scholar.google.com/scholar?q=Rowan+Zellers%2C+Ari+Holtzman%2C+Yonatan+Bisk%2C+Ali+Farhadi%2C+and+Yejin+Choi%2E+2019%2E+HellaSwag%3A+Can+a+Machine+Really+Finish+Your+Sentence%3F%2E+In+Proceedings+of+the+57th+Annual+Meeting+of+the+Association+for+Co)
430. Rowan Zellers, Ari Holtzman, Hannah Rashkin, Yonatan Bisk, Ali Farhadi, Franziska Roesner, and Yejin Choi. 2019. [Defending against neural fake news](https://md.research.elamur.ai/free/en/papers/zellers-2019-defending-against-neural-fake-news/). Advances in neural information processing systems 32 (2019).
431. Biao Zhang and Rico Sennrich. 2019. Root mean square layer normalization. Advances in Neural Information Processing Systems 32 (2019). [find](https://scholar.google.com/scholar?q=Biao+Zhang+and+Rico+Sennrich%2E+2019%2E+Root+mean+square+layer+normalization%2E+Advances+in+Neural+Information+Processing+Systems+32+%282019%29%2E)
432. Cheng Zhang, Jianyi Cheng, George A Constantinides, and Yiren Zhao. 2024. LQER: Low-Rank Quantization Error Reconstruction for LLMs. arXiv preprint arXiv:2402.02446 (2024). — [arXiv:2402.02446](https://arxiv.org/abs/2402.02446) [find](https://scholar.google.com/scholar?q=Cheng+Zhang%2C+Jianyi+Cheng%2C+George+A+Constantinides%2C+and+Yiren+Zhao%2E+2024%2E+LQER%3A+Low%2DRank+Quantization+Error+Reconstruction+for+LLMs%2E+arXiv+preprint+arXiv%3A2402%2E02446+%282024%29%2E+%E2%80%94+%5BarXiv%3A2402%2E02446%5D%28https%3A)
433. Collin Zhang, John X Morris, and Vitaly Shmatikov. 2024. Extracting Prompts by Inverting LLM Outputs. arXiv preprint arXiv:2405.15012 (2024). [find](https://scholar.google.com/scholar?q=Collin+Zhang%2C+John+X+Morris%2C+and+Vitaly+Shmatikov%2E+2024%2E+Extracting+Prompts+by+Inverting+LLM+Outputs%2E+arXiv+preprint+arXiv%3A2405%2E15012+%282024%29%2E)
434. Chen Zhang, Dawei Song, Zheyu Ye, and Yan Gao. 2023. [Towards the law of capacity gap in distilling language models](https://md.research.elamur.ai/free/en/papers/zhang-2025-law-capacity-gap-distilling-language/). arXiv preprint arXiv:2311.07052 (2023).
435. Dan Zhang, Ziniu Hu, Sining Zhoubian, Zhengxiao Du, Kaiyu Yang, Zihan Wang, Yisong Yue, Yuxiao Dong, and Jie Tang. 2024. Sciglm: Training scientific language models with self-reflective instruction annotation and tuning. arXiv preprint arXiv:2401.07950 (2024). [find](https://scholar.google.com/scholar?q=Dan+Zhang%2C+Ziniu+Hu%2C+Sining+Zhoubian%2C+Zhengxiao+Du%2C+Kaiyu+Yang%2C+Zihan+Wang%2C+Yisong+Yue%2C+Yuxiao+Dong%2C+and+Jie+Tang%2E+2024%2E+Sciglm%3A+Training+scientific+language+models+with+self%2Dreflective+instruction+an)
436. Di Zhang, Wei Liu, Qian Tan, Jingdan Chen, Hang Yan, Yuliang Yan, Jiatong Li, Weiran Huang, Xiangyu Yue, Dongzhan Zhou, et al. 2024. Chemllm: A chemical large language model. arXiv preprint arXiv:2402.06852 (2024). [find](https://scholar.google.com/scholar?q=Di+Zhang%2C+Wei+Liu%2C+Qian+Tan%2C+Jingdan+Chen%2C+Hang+Yan%2C+Yuliang+Yan%2C+Jiatong+Li%2C+Weiran+Huang%2C+Xiangyu+Yue%2C+Dongzhan+Zhou%2C+et+al%2E+2024%2E+Chemllm%3A+A+chemical+large+language+model%2E+arXiv+preprint+arXiv%3A2402)
437. Kaiyan Zhang, Jianyu Wang, Ermo Hua, Biqing Qi, Ning Ding, and Bowen Zhou. 2024. Cogenesis: A framework collaborating large and small language models for secure context-aware instruction following. arXiv preprint arXiv:2403.03129 (2024). [find](https://scholar.google.com/scholar?q=Kaiyan+Zhang%2C+Jianyu+Wang%2C+Ermo+Hua%2C+Biqing+Qi%2C+Ning+Ding%2C+and+Bowen+Zhou%2E+2024%2E+Cogenesis%3A+A+framework+collaborating+large+and+small+language+models+for+secure+context%2Daware+instruction+following%2E+ar)
438. Mingyang Zhang, Hao Chen, Chunhua Shen, Zhen Yang, Linlin Ou, Xinyi Yu, and Bohan Zhuang. 2023. Loraprune: Pruning meets low-rank parameter-efficient fine-tuning. arXiv preprint arXiv:2305.18403 (2023). — [arXiv:2305.18403](https://arxiv.org/abs/2305.18403) [find](https://scholar.google.com/scholar?q=Mingyang+Zhang%2C+Hao+Chen%2C+Chunhua+Shen%2C+Zhen+Yang%2C+Linlin+Ou%2C+Xinyi+Yu%2C+and+Bohan+Zhuang%2E+2023%2E+Loraprune%3A+Pruning+meets+low%2Drank+parameter%2Defficient+fine%2Dtuning%2E+arXiv+preprint+arXiv%3A2305%2E18403+%282023)
439. Peiyuan Zhang, Guangtao Zeng, Tianduo Wang, and Wei Lu. 2024. [TinyLlama: An Open-Source Small Language Model](https://md.research.elamur.ai/free/en/papers/zhang-2024-tinyllama-open-source-small-language/). arXiv:2401.02385 [cs.CL] https://arxiv.org/abs/2401.02385 [↗](https://arxiv.org/abs/2401.02385)
440. Susan Zhang, Stephen Roller, Naman Goyal, Mikel Artetxe, Moya Chen, Shuohui Chen, Christopher Dewan, Mona Diab, Xian Li, Xi Victoria Lin, et al. 2022. [Opt: Open pre-trained transformer language models](https://md.research.elamur.ai/free/en/papers/zhang-2022-opt-open-pre-trained-transformer/). arXiv preprint arXiv:2205.01068 (2022).
441. Xinran Zhang, Xin Yuan, Yunwei Li, and Yanru Zhang. 2019. Cold-Start representation learning: A recommendation approach with bert4Movie and movie2Vec. In Proceedings of the 27th ACM International Conference on Multimedia. 2612–2616. [find](https://scholar.google.com/scholar?q=Xinran+Zhang%2C+Xin+Yuan%2C+Yunwei+Li%2C+and+Yanru+Zhang%2E+2019%2E+Cold%2DStart+representation+learning%3A+A+recommendation+approach+with+bert4Movie+and+movie2Vec%2E+In+Proceedings+of+the+27th+ACM+International+Conf)
442. Yingtao Zhang, Haoli Bai, Haokun Lin, Jialin Zhao, Lu Hou, and Carlo Vittorio Cannistraci. 2024. Plug-and-play: An efficient post-training pruning method for large language models. In The Twelfth International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Yingtao+Zhang%2C+Haoli+Bai%2C+Haokun+Lin%2C+Jialin+Zhao%2C+Lu+Hou%2C+and+Carlo+Vittorio+Cannistraci%2E+2024%2E+Plug%2Dand%2Dplay%3A+An+efficient+post%2Dtraining+pruning+method+for+large+language+models%2E+In+The+Twelfth+Inte)
443. Yiming Zhang, Nicholas Carlini, and Daphne Ippolito. 2024. Effective Prompt Extraction from Language Models. arXiv:2307.06865 [cs.CL] https://arxiv.org/abs/2307.06865 — [arXiv:2307.06865](https://arxiv.org/abs/2307.06865) [↗](https://arxiv.org/abs/2307.06865)
444. Zhenyu Zhang, Ying Sheng, Tianyi Zhou, Tianlong Chen, Lianmin Zheng, Ruisi Cai, Zhao Song, Yuandong Tian, Christopher Ré, Clark Barrett, et al. 2024. H2o: Heavy-hitter oracle for efficient generative inference of large language models. Advances in Neural Information Processing Systems 36 (2024). Man… [find](https://scholar.google.com/scholar?q=Zhenyu+Zhang%2C+Ying+Sheng%2C+Tianyi+Zhou%2C+Tianlong+Chen%2C+Lianmin+Zheng%2C+Ruisi+Cai%2C+Zhao+Song%2C+Yuandong+Tian%2C+Christopher+R%C3%A9%2C+Clark+Barrett%2C+et+al%2E+2024%2E+H2o%3A+Heavy%2Dhitter+oracle+for+efficient+generative+)
445. Zhiwei Zhang, Fali Wang, Xiaomin Li, Zongyu Wu, Xianfeng Tang, Hui Liu, Qi He, Wenpeng Yin, and Suhang Wang. 2024. Does your LLM truly unlearn? An embarrassingly simple approach to recover unlearned knowledge. arXiv preprint arXiv:2410.16454 (2024). — [arXiv:2410.16454](https://arxiv.org/abs/2410.16454) [find](https://scholar.google.com/scholar?q=Zhiwei+Zhang%2C+Fali+Wang%2C+Xiaomin+Li%2C+Zongyu+Wu%2C+Xianfeng+Tang%2C+Hui+Liu%2C+Qi+He%2C+Wenpeng+Yin%2C+and+Suhang+Wang%2E+2024%2E+Does+your+LLM+truly+unlearn%3F+An+embarrassingly+simple+approach+to+recover+unlearned+k)
446. Bowen Zhao, Hannaneh Hajishirzi, and Qingqing Cao. 2024. APT: Adaptive Pruning and Tuning Pretrained Language Models for Efficient Training and Inference. In Forty-first International Conference on Machine Learning. [find](https://scholar.google.com/scholar?q=Bowen+Zhao%2C+Hannaneh+Hajishirzi%2C+and+Qingqing+Cao%2E+2024%2E+APT%3A+Adaptive+Pruning+and+Tuning+Pretrained+Language+Models+for+Efficient+Training+and+Inference%2E+In+Forty%2Dfirst+International+Conference+on+Ma)
447. Junchen Zhao, Yurun Song, Simeng Liu, Ian G. Harris, and Sangeetha Abdu Jyothi. 2023. LinguaLinked: A Distributed Large Language Model Inference System for Mobile Devices. arXiv:2312.00388 [cs.LG] https://arxiv.org/abs/2312.00388 — [arXiv:2312.00388](https://arxiv.org/abs/2312.00388) [↗](https://arxiv.org/abs/2312.00388)
448. Juntao Zhao, Borui Wan, Yanghua Peng, Haibin Lin, and Chuan Wu. 2024. LLM-PQ: Serving LLM on Heterogeneous Clusters with Phase-Aware Partition and Adaptive Quantization. arXiv preprint arXiv:2403.01136 (2024). [find](https://scholar.google.com/scholar?q=Juntao+Zhao%2C+Borui+Wan%2C+Yanghua+Peng%2C+Haibin+Lin%2C+and+Chuan+Wu%2E+2024%2E+LLM%2DPQ%3A+Serving+LLM+on+Heterogeneous+Clusters+with+Phase%2DAware+Partition+and+Adaptive+Quantization%2E+arXiv+preprint+arXiv%3A2403%2E0113)
449. Kun Zhao, Bohao Yang, Chen Tang, Chenghua Lin, and Liang Zhan. 2024. SLIDE: A Framework Integrating Small and Large Language Models for Open-Domain Dialogues Evaluation. arXiv preprint arXiv:2405.15924 (2024). [find](https://scholar.google.com/scholar?q=Kun+Zhao%2C+Bohao+Yang%2C+Chen+Tang%2C+Chenghua+Lin%2C+and+Liang+Zhan%2E+2024%2E+SLIDE%3A+A+Framework+Integrating+Small+and+Large+Language+Models+for+Open%2DDomain+Dialogues+Evaluation%2E+arXiv+preprint+arXiv%3A2405%2E1592)
450. Theodore Zhao, Mu Wei, J Samuel Preston, and Hoifung Poon. 2023. Automatic calibration and error correction for large language models via pareto optimal self-supervision. arXiv preprint arXiv:2306.16564 (2023). — [arXiv:2306.16564](https://arxiv.org/abs/2306.16564) [find](https://scholar.google.com/scholar?q=Theodore+Zhao%2C+Mu+Wei%2C+J+Samuel+Preston%2C+and+Hoifung+Poon%2E+2023%2E+Automatic+calibration+and+error+correction+for+large+language+models+via+pareto+optimal+self%2Dsupervision%2E+arXiv+preprint+arXiv%3A2306%2E165)
451. Youpeng Zhao, Ming Lin, Huadong Tang, Qiang Wu, and Jun Wang. 2024. Merino: Entropy-driven Design for Generative Language Models on IoT Devices. arXiv:2403.07921 [cs.LG] https://arxiv.org/abs/2403.07921 — [arXiv:2403.07921](https://arxiv.org/abs/2403.07921) [↗](https://arxiv.org/abs/2403.07921)
452. Zhengyun Zhao, Qiao Jin, Fangyuan Chen, Tuorui Peng, and Sheng Yu. 2022. Pmc-patients: A large-scale dataset of patient summaries and relations for benchmarking retrieval-based clinical decision support systems. arXiv preprint arXiv:2202.13876 (2022). — [arXiv:2202.13876](https://arxiv.org/abs/2202.13876) [find](https://scholar.google.com/scholar?q=Zhengyun+Zhao%2C+Qiao+Jin%2C+Fangyuan+Chen%2C+Tuorui+Peng%2C+and+Sheng+Yu%2E+2022%2E+Pmc%2Dpatients%3A+A+large%2Dscale+dataset+of+patient+summaries+and+relations+for+benchmarking+retrieval%2Dbased+clinical+decision+suppo)
453. Zhanhui Zhou, Zhixuan Liu, Jie Liu, Zhichen Dong, Chao Yang, and Yu Qiao. 2024. [Weak-to-Strong Search: Align Large Language Models via Searching over Small Language Models](https://md.research.elamur.ai/free/en/papers/zhou-2024-weak-strong-search-align-large/). In The Thirty-eighth Annual Conference on Neural Information Processing Systems. [https://openreview.net/forum?id=dOJ6CqWDf1](https://openreview.net/forum?id=dOJ6CqWDf1) [↗](https://openreview.net/forum?id=dOJ6CqWDf1)
454. Fengbin Zhu, Wenqiang Lei, Youcheng Huang, Chao Wang, Shuo Zhang, Jiancheng Lv, Fuli Feng, and Tat-Seng Chua. 2021. [TAT-QA: A Question Answering Benchmark on a Hybrid of Tabular and Textual Content in Finance](https://md.research.elamur.ai/free/en/papers/zhu-2021-tat-qa-question-answering-benchmark/). In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics…
455. Jiachen Zhu, Jianghao Lin, Xinyi Dai, Bo Chen, Rong Shan, Jieming Zhu, Ruiming Tang, Yong Yu, and Weinan Zhang. 2024. Lifelong Personalized Low-Rank Adaptation of Large Language Models for Recommendation. arXiv preprint arXiv:2408.03533 (2024). — [arXiv:2408.03533](https://arxiv.org/abs/2408.03533) [find](https://scholar.google.com/scholar?q=Jiachen+Zhu%2C+Jianghao+Lin%2C+Xinyi+Dai%2C+Bo+Chen%2C+Rong+Shan%2C+Jieming+Zhu%2C+Ruiming+Tang%2C+Yong+Yu%2C+and+Weinan+Zhang%2E+2024%2E+Lifelong+Personalized+Low%2DRank+Adaptation+of+Large+Language+Models+for+Recommendat)
456. Kaijie Zhu, Jindong Wang, Jiaheng Zhou, Zichen Wang, Hao Chen, Yidong Wang, Linyi Yang, Wei Ye, Yue Zhang, Neil Zhenqiang Gong, et al. 2023. [Promptbench: Towards evaluating the robustness of large language models on adversarial prompts](https://md.research.elamur.ai/free/en/papers/zhu-2023-promptrobust-towards-evaluating-robustness-large/). arXiv preprint arXiv:2306.04528 (2023).
457. Xunyu Zhu, Jian Li, Yong Liu, Can Ma, and Weiping Wang. 2023. [A survey on model compression for large language models](https://md.research.elamur.ai/free/en/papers/zhu-2024-survey-model-compression-large-language/). arXiv preprint arXiv:2308.07633 (2023).
458. Yun Zhu, Yinxiao Liu, Felix Stahlberg, Shankar Kumar, Yu hui Chen, Liangchen Luo, Lei Shu, Renjie Liu, Jindong Chen, and Lei Meng. 2023. Towards an On-device Agent for Text Rewriting. arXiv:2308.11807 [cs.CL] https://arxiv.org/abs/2308.11807 [↗](https://arxiv.org/abs/2308.11807)
459. Yuanyuan Zhuang and Jaekyeong Kim. 2021. A bert-based multi-criteria recommender system for hotel promotion management. Sustainability 13, 14 (2021), 8039. [find](https://scholar.google.com/scholar?q=Yuanyuan+Zhuang+and+Jaekyeong+Kim%2E+2021%2E+A+bert%2Dbased+multi%2Dcriteria+recommender+system+for+hotel+promotion+management%2E+Sustainability+13%2C+14+%282021%29%2C+8039%2E)
460. Terry Yue Zhuo, Yujin Huang, Chunyang Chen, and Zhenchang Xing. 2023. [Red teaming chatgpt via jailbreaking: Bias, robustness, reliability and toxicity](https://md.research.elamur.ai/free/en/papers/zhuo-2023-red-teaming-chatgpt-jailbreaking-bias/). arXiv preprint arXiv:2301.12867 (2023).
461. Andy Zou, Zifan Wang, Nicholas Carlini, Milad Nasr, J Zico Kolter, and Matt Fredrikson. 2023. [Universal and transferable adversarial attacks on aligned language models](https://md.research.elamur.ai/free/en/papers/zou-2023-universal-transferable-adversarial-attacks-aligned/). arXiv preprint arXiv:2307.15043 (2023).
462. Lixin Zou, Weixue Lu, Yiding Liu, Hengyi Cai, Xiaokai Chu, Dehong Ma, Daiting Shi, Yu Sun, Zhicong Cheng, Simiu Gu, et al. 2022. Pre-trained language model-based retrieval and ranking for web search. ACM Transactions on the Web 17, 1 (2022), 1–36. [find](https://scholar.google.com/scholar?q=Lixin+Zou%2C+Weixue+Lu%2C+Yiding+Liu%2C+Hengyi+Cai%2C+Xiaokai+Chu%2C+Dehong+Ma%2C+Daiting+Shi%2C+Yu+Sun%2C+Zhicong+Cheng%2C+Simiu+Gu%2C+et+al%2E+2022%2E+Pre%2Dtrained+language+model%2Dbased+retrieval+and+ranking+for+web+search%2E+)
463. Jingwei Zuo, Maksim Velikanov, Dhia Eddine Rhaiem, Ilyas Chahed, Younes Belkada, Guillaume Kunsch, and Hakim Hacid. 2024. Falcon mamba: The first competitive attention-free 7b language model. arXiv preprint arXiv:2410.05355 (2024). Received 20 February 2007; revised 12 March 2009; accepted 5 June 20… — [arXiv:2410.05355](https://arxiv.org/abs/2410.05355) [find](https://scholar.google.com/scholar?q=Jingwei+Zuo%2C+Maksim+Velikanov%2C+Dhia+Eddine+Rhaiem%2C+Ilyas+Chahed%2C+Younes+Belkada%2C+Guillaume+Kunsch%2C+and+Hakim+Hacid%2E+2024%2E+Falcon+mamba%3A+The+first+competitive+attention%2Dfree+7b+language+model%2E+arXiv+pr)

*Updated 2 November 2026*