---
title: "GAST — paper by Yao et al. (2026) — summary and ideas"
description: "They propose GAST (Gradient-aligned Sparse Tuning), a PEFT add-on that chooses data and layers jointly at the batch level. 38 references."
authors: ["Kai Yao", "Zhenghan Song", "Kaixin Wu", "Mingjie Zhong", "Danzhao Cheng", "Zhaorui Tan", "Yixin Ji", "Penglei Gao"]
year: 2026
venue: "Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers)"
source: "https://aclanthology.org/2026.eacl-long.206/"
doi: "10.18653/v1/2026.eacl-long.206"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/yao-2026-gast-gradient-aligned-sparse-tuning/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/yao-2026-gast-gradient-aligned-sparse-tuning/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/34/) / GAST: Gradient-aligned Sparse Tuning of Large Language Models with Data-layer Selection

# GAST: Gradient-aligned Sparse Tuning of Large Language Models with Data-layer Selection

**Authors:** Kai Yao, Zhenghan Song, Kaixin Wu, Mingjie Zhong, Danzhao Cheng, Zhaorui Tan, [Yixin Ji](https://md.research.elamur.ai/free/en/authors/yixin-ji/) and Penglei Gao · **Year:** 2026 · **Venue:** [Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers)](https://md.research.elamur.ai/free/en/venues/eacl/) ([EACL 2026](https://md.research.elamur.ai/free/en/venues/eacl/2026/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2026.eacl-long.206/) · [PDF](https://aclanthology.org/2026.eacl-long.206.pdf) · [DOI 10.18653/v1/2026.eacl-long.206](https://doi.org/10.18653/v1/2026.eacl-long.206) · [OpenAlex](https://openalex.org/W7140094610)

**Citations:** 0 citations (OpenAlex, 2026-10-14)

## Summary
Sparse tuning methods for large language models usually work along one axis. Some update only a chosen subset of layers using every sample in a batch. Others train all layers on a filtered subset of the data. The authors argue that this ignores a key fact: individual samples help some layers and hurt others. Discarding "weak" data also throws away signals that could still be useful. They propose GAST (Gradient-aligned Sparse Tuning), a PEFT add-on that chooses data and layers jointly at the batch level. For each layer, it compares every sample's gradient with an averaged gradient from a small held-out support set. It then samples the better-aligned samples to update that layer, which reduces gradient conflicts. A theoretical argument claims that layer-only and data-only selection are both sub-optimal special cases of this joint scheme. Experiments on commonsense and math reasoning benchmarks show consistent gains and faster convergence over LoRA and the layer-selective IST baseline. These results suggest that data–layer interaction is a useful lever for efficient fine-tuning.

## Key points
- On math reasoning with LLaMA3-8B, adding GAST to LoRA raised the average score from 63.4% to 67.5%. Individual benchmarks such as GSM8K and SVAMP also improved.
- The ablation used LLaMA 7B with LoRA on commonsense reasoning:
  - Random data–layer selection already helped.
  - Top-k selection by alignment gave no improvement.
  - Probabilistic sampling based on alignment scores worked best.
- The default data–layer sparsity is 0.5, tested across a range from 0.0 (plain LoRA) to 0.875. Results stayed stable between 0.25 and 0.75 and peaked at 0.5. Even at 0.875, GAST still beat standard LoRA.
- GAST allocates layers unevenly at 50% sparsity. The highest-aligned samples were trained in up to about 70% of LLaMA-7B's 32 layers, and the lowest-aligned in only about 30%. Some samples favoured shallow layers, others deep ones.
- The authors report two limitations:
  - Like GREATS, GAST cannot cut memory and compute at the same time.
  - Models at the scale of LLaMA 3 70B were not tested.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Cornell University](https://md.research.elamur.ai/free/en/universities/cornell-university/) — University, Ithaca, United States · 134 works · 65,159 citations
- [Soochow University](https://md.research.elamur.ai/free/en/universities/soochow-university/) — University, Suzhou, China · 69 works · 4,815 citations
- [University of Liverpool](https://md.research.elamur.ai/free/en/universities/university-of-liverpool/) — University, Liverpool, United Kingdom · 16 works · 745 citations

Also on the byline: [Cleveland Clinic Lerner College of Medicine](https://ror.org/02x4b0932).

📚 Shelves (5): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 18,614 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 17,965 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 17,450 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 17,296 works · [👁️ Computer Vision and Pattern Recognition](https://md.research.elamur.ai/free/en/shelves/computer-science/computer-vision-and-pattern-recognition/) · 3,486 works

🧭 Topics (1): [🧠 Language models & pretraining](https://md.research.elamur.ai/free/en/topics/language-models/) · 3,894 works

## Cite

Cite as: Kai Yao, Zhenghan Song, Kaixin Wu, Mingjie Zhong, Danzhao Cheng, Zhaorui Tan, Yixin Ji and Penglei Gao. 2026. GAST: Gradient-aligned Sparse Tuning of Large Language Models with Data-layer Selection. Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{yao2026gast,
  title = {{GAST: Gradient-aligned Sparse Tuning of Large Language Models with Data-layer Selection}},
  author = {Kai Yao, Zhenghan Song, Kaixin Wu, Mingjie Zhong, Danzhao Cheng, Zhaorui Tan, Yixin Ji and Penglei Gao},
  booktitle = {Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2026},
  doi = {10.18653/v1/2026.eacl-long.206},
  url = {https://doi.org/10.18653/v1/2026.eacl-long.206}
}
```

- [← previous GASim: A Graph-Accelerated Hybrid Framework for Social Simulation](https://md.research.elamur.ai/free/en/papers/zhou-2026-gasim-graph-accelerated-hybrid-framework/)
- [next → GATE-Reranker: A Strong Arabic Cross-Encoder for Document Reranking](https://md.research.elamur.ai/free/en/papers/nacar-2026-gate-reranker-strong-arabic-cross/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/34/)

## References

38 references, 16 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Yonatan Bisk, Rowan Zellers, Ronan Le Bras, Jianfeng Gao, and Yejin Choi. 2020. [Piqa: Reasoning about physical commonsense in natural language](https://md.research.elamur.ai/free/en/papers/bisk-2019-piqa-reasoning-about-physical-commonsense/). In Thirty-Fourth AAAI Conference on Artificial Intelligence.
2. Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. 2020. [Language models are few-shot learners](https://md.research.elamur.ai/free/en/papers/brown-2020-language-models-few-shot-learners/). Advances in neural information processing systems, 33:1877–1901.
3. Christopher Clark, Kenton Lee, Ming-Wei Chang, Tom Kwiatkowski, Michael Collins, and Kristina Toutanova. 2019. [BoolQ: Exploring the surprising difficulty of natural yes/no questions](https://md.research.elamur.ai/free/en/papers/clark-2019-boolq-exploring-surprising-difficulty-natural/). In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics:…
4. Peter Clark, Isaac Cowhey, Oren Etzioni, Tushar Khot, Ashish Sabharwal, Carissa Schoenick, and Oyvind Tafjord. 2018. [Think you have solved question answering? try arc, the ai2 reasoning challenge](https://md.research.elamur.ai/free/en/papers/clark-2018-think-have-solved-question-answering/). arXiv:1803.05457v1.
5. Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman. 2021. [Training verifiers to solve math word problems](https://md.research.elamur.ai/free/en/papers/cobbe-2021-training-verifiers-solve-math-word/). arXiv preprint arXiv:2110.14168.
6. Zhijie Deng, Peng Cui, and Jun Zhu. 2023. Towards accelerated model training via bayesian data selection. In Advances in Neural Information Processing Systems, volume 36, pages 8513–8527. [find](https://scholar.google.com/scholar?q=Zhijie+Deng%2C+Peng+Cui%2C+and+Jun+Zhu%2E+2023%2E+Towards+accelerated+model+training+via+bayesian+data+selection%2E+In+Advances+in+Neural+Information+Processing+Systems%2C+volume+36%2C+pages+8513%E2%80%938527%2E)
7. Mostafa Elhoushi, Akshat Shrivastava, Diana Liskovich, Basil Hosmer, Bram Wasti, Liangzhen Lai, Anas Mahmoud, Bilge Acun, Saurabh Agarwal, Ahmed Roman, et al. 2024. [Layer skip: Enabling early exit inference and self-speculative decoding](https://md.research.elamur.ai/free/en/papers/elhoushi-2024-layerskip-enabling-early-exit-inference/). arXiv preprint arXiv:2404.16710.
8. Chun Fan, Jiwei Li, Xiang Ao, Fei Wu, Yuxian Meng, and Xiaofei Sun. 2021. Layer-wise model pruning based on mutual information. arXiv preprint arXiv:2108.12594. [find](https://scholar.google.com/scholar?q=Chun+Fan%2C+Jiwei+Li%2C+Xiang+Ao%2C+Fei+Wu%2C+Yuxian+Meng%2C+and+Xiaofei+Sun%2E+2021%2E+Layer%2Dwise+model+pruning+based+on+mutual+information%2E+arXiv+preprint+arXiv%3A2108%2E12594%2E)
9. Junxian He, Chunting Zhou, Xuezhe Ma, Taylor Berg-Kirkpatrick, and Graham Neubig. 2022. [Towards a unified view of parameter-efficient transfer learning](https://md.research.elamur.ai/free/en/papers/he-2021-towards-unified-view-parameter-efficient/). In International Conference on Learning Representations.
10. Neil Houlsby, Andrei Giurgiu, Stanislaw Jastrzebski, Bruna Morrone, Quentin De Laroussilhe, Andrea Gesmundo, Mona Attariyan, and Sylvain Gelly. 2019. Parameter-efficient transfer learning for nlp. In International conference on machine learning, pages 2790–2799. [find](https://scholar.google.com/scholar?q=Neil+Houlsby%2C+Andrei+Giurgiu%2C+Stanislaw+Jastrzebski%2C+Bruna+Morrone%2C+Quentin+De+Laroussilhe%2C+Andrea+Gesmundo%2C+Mona+Attariyan%2C+and+Sylvain+Gelly%2E+2019%2E+Parameter%2Defficient+transfer+learning+for+nlp%2E+In+)
11. Jeremy Howard and Sebastian Ruder. 2018. Universal language model fine-tuning for text classification. arXiv preprint arXiv:1801.06146. [find](https://scholar.google.com/scholar?q=Jeremy+Howard+and+Sebastian+Ruder%2E+2018%2E+Universal+language+model+fine%2Dtuning+for+text+classification%2E+arXiv+preprint+arXiv%3A1801%2E06146%2E)
12. Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2021. [Lora: Low-rank adaptation of large language models](https://md.research.elamur.ai/free/en/papers/hu-2021-lora-low-rank-adaptation-large/). arXiv preprint arXiv:2106.09685.
13. Zhiqiang Hu, Yihuai Lan, Lei Wang, Wanyu Xu, Ee-Peng Lim, Roy Ka-Wei Lee, Lidong Bing, and Soujanya Poria. 2023. [Llm-adapters: An adapter family for parameter-efficient fine-tuning of large language models](https://md.research.elamur.ai/free/en/papers/hu-2023-llm-adapters-adapter-family-parameter/). In Empirical Methods in Natural Language Processing.
14. Angela H Jiang, Daniel L-K Wong, Giulio Zhou, David G Andersen, Jeffrey Dean, Gregory R Ganger, Gauri Joshi, Michael Kaminksy, Michael Kozuch, Zachary C Lipton, et al. 2019. Accelerating deep learning by focusing on the biggest losers. arXiv preprint arXiv:1910.00762. — [arXiv:1910.00762](https://arxiv.org/abs/1910.00762) [find](https://scholar.google.com/scholar?q=Angela+H+Jiang%2C+Daniel+L%2DK+Wong%2C+Giulio+Zhou%2C+David+G+Andersen%2C+Jeffrey+Dean%2C+Gregory+R+Ganger%2C+Gauri+Joshi%2C+Michael+Kaminksy%2C+Michael+Kozuch%2C+Zachary+C+Lipton%2C+et+al%2E+2019%2E+Accelerating+deep+learning)
15. Gal Kaplun, Andrey Gurevich, Tal Swisa, Mazor David, Shai Shalev-Shwartz, and Eran Malach. 2023. Less is more: Selective layer finetuning with subtuning. arXiv preprint arXiv:2302.06354. — [arXiv:2302.06354](https://arxiv.org/abs/2302.06354) [find](https://scholar.google.com/scholar?q=Gal+Kaplun%2C+Andrey+Gurevich%2C+Tal+Swisa%2C+Mazor+David%2C+Shai+Shalev%2DShwartz%2C+and+Eran+Malach%2E+2023%2E+Less+is+more%3A+Selective+layer+finetuning+with+subtuning%2E+arXiv+preprint+arXiv%3A2302%2E06354%2E+%E2%80%94+%5BarXiv%3A2302)
16. Angelos Katharopoulos and François Fleuret. 2018. Not all samples are created equal: Deep learning with importance sampling. In International conference on machine learning, pages 2525–2534. PMLR. [find](https://scholar.google.com/scholar?q=Angelos+Katharopoulos+and+Fran%C3%A7ois+Fleuret%2E+2018%2E+Not+all+samples+are+created+equal%3A+Deep+learning+with+importance+sampling%2E+In+International+conference+on+machine+learning%2C+pages+2525%E2%80%932534%2E+PMLR%2E)
17. Rik Koncel-Kedziorski, Subhro Roy, Aida Amini, Nate Kushman, and Hannaneh Hajishirzi. 2016. MAWPS: A math word problem repository. In Proceedings of NAACL, pages 1152–1157. [find](https://scholar.google.com/scholar?q=Rik+Koncel%2DKedziorski%2C+Subhro+Roy%2C+Aida+Amini%2C+Nate+Kushman%2C+and+Hannaneh+Hajishirzi%2E+2016%2E+MAWPS%3A+A+math+word+problem+repository%2E+In+Proceedings+of+NAACL%2C+pages+1152%E2%80%931157%2E)
18. Tao Lei, Junwen Bai, Siddhartha Brahma, Joshua Ainslie, Kenton Lee, Yanqi Zhou, Nan Du, Vincent Zhao, Yuexin Wu, Bo Li, et al. 2024. Conditional adapters: Parameter-efficient transfer learning with fast inference. Advances in Neural Information Processing Systems, 36. [find](https://scholar.google.com/scholar?q=Tao+Lei%2C+Junwen+Bai%2C+Siddhartha+Brahma%2C+Joshua+Ainslie%2C+Kenton+Lee%2C+Yanqi+Zhou%2C+Nan+Du%2C+Vincent+Zhao%2C+Yuexin+Wu%2C+Bo+Li%2C+et+al%2E+2024%2E+Conditional+adapters%3A+Parameter%2Defficient+transfer+learning+with+fa)
19. Xiang Lisa Li and Percy Liang. 2021. Prefix-tuning: Optimizing continuous prompts for generation. arXiv preprint arXiv:2101.00190. [find](https://scholar.google.com/scholar?q=Xiang+Lisa+Li+and+Percy+Liang%2E+2021%2E+Prefix%2Dtuning%3A+Optimizing+continuous+prompts+for+generation%2E+arXiv+preprint+arXiv%3A2101%2E00190%2E)
20. Wang Ling, Dani Yogatama, Chris Dyer, and Phil Blunsom. 2017. Program induction by rationale generation: Learning to solve and explain algebraic word problems. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 158–167. [find](https://scholar.google.com/scholar?q=Wang+Ling%2C+Dani+Yogatama%2C+Chris+Dyer%2C+and+Phil+Blunsom%2E+2017%2E+Program+induction+by+rationale+generation%3A+Learning+to+solve+and+explain+algebraic+word+problems%2E+In+Proceedings+of+the+55th+Annual+Meetin)
21. Shih-Yang Liu, Chien-Yi Wang, Hongxu Yin, Pavlo Molchanov, Yu-Chiang Frank Wang, Kwang-Ting Cheng, and Min-Hung Chen. 2024. Dora: Weight-decomposed low-rank adaptation. In International Conference on Machine Learning. [find](https://scholar.google.com/scholar?q=Shih%2DYang+Liu%2C+Chien%2DYi+Wang%2C+Hongxu+Yin%2C+Pavlo+Molchanov%2C+Yu%2DChiang+Frank+Wang%2C+Kwang%2DTing+Cheng%2C+and+Min%2DHung+Chen%2E+2024%2E+Dora%3A+Weight%2Ddecomposed+low%2Drank+adaptation%2E+In+International+Conference+on+)
22. Xiao Liu, Kaixuan Ji, Yicheng Fu, Weng Lam Tam, Zhengxiao Du, Zhilin Yang, and Jie Tang. 2022. [P-tuning v2: Prompt tuning can be comparable to fine-tuning universally across scales and tasks](https://md.research.elamur.ai/free/en/papers/liu-2021-p-tuning-v2-prompt-tuning/). In Proceedings of the 60th Annual Meeting of the Association of Computational Linguistics.
23. Todor Mihaylov, Peter Clark, Tushar Khot, and Ashish Sabharwal. 2018. Can a suit of armor conduct electricity? a new dataset for open book question answering. In Empirical Methods in Natural Language Processing. [find](https://scholar.google.com/scholar?q=Todor+Mihaylov%2C+Peter+Clark%2C+Tushar+Khot%2C+and+Ashish+Sabharwal%2E+2018%2E+Can+a+suit+of+armor+conduct+electricity%3F+a+new+dataset+for+open+book+question+answering%2E+In+Empirical+Methods+in+Natural+Language+)
24. Rui Pan, Xiang Liu, Shizhe Diao, Renjie Pi, Jipeng Zhang, Chi Han, and Tong Zhang. 2024. Lisa: Layerwise importance sampling for memory-efficient large language model fine-tuning. arXiv preprint arXiv:2403.17919. [find](https://scholar.google.com/scholar?q=Rui+Pan%2C+Xiang+Liu%2C+Shizhe+Diao%2C+Renjie+Pi%2C+Jipeng+Zhang%2C+Chi+Han%2C+and+Tong+Zhang%2E+2024%2E+Lisa%3A+Layerwise+importance+sampling+for+memory%2Defficient+large+language+model+fine%2Dtuning%2E+arXiv+preprint+arXiv)
25. Arkil Patel, Satwik Bhattamishra, and Navin Goyal. 2021. [Are NLP models really able to solve simple math word problems?](https://md.research.elamur.ai/free/en/papers/patel-2021-are-nlp-models-really-able/) In Proceedings of NAACL, pages 2080–2094.
26. Hassan Sajjad, Fahim Dalvi, Nadir Durrani, and Preslav Nakov. 2023. On the effect of dropping layers of pre-trained transformer models. Computer Speech & Language, 77:101429. [find](https://scholar.google.com/scholar?q=Hassan+Sajjad%2C+Fahim+Dalvi%2C+Nadir+Durrani%2C+and+Preslav+Nakov%2E+2023%2E+On+the+effect+of+dropping+layers+of+pre%2Dtrained+transformer+models%2E+Computer+Speech+%26+Language%2C+77%3A101429%2E)
27. Keisuke Sakaguchi, Ronan Le Bras, Chandra Bhagavatula, and Yejin Choi. 2021. Winogrande: An adversarial winograd schema challenge at scale. Communications of the ACM, 64(9):99–106. [find](https://scholar.google.com/scholar?q=Keisuke+Sakaguchi%2C+Ronan+Le+Bras%2C+Chandra+Bhagavatula%2C+and+Yejin+Choi%2E+2021%2E+Winogrande%3A+An+adversarial+winograd+schema+challenge+at+scale%2E+Communications+of+the+ACM%2C+64%289%29%3A99%E2%80%93106%2E)
28. Maarten Sap, Hannah Rashkin, Derek Chen, Ronan LeBras, and Yejin Choi. 2019. [Socialiqa: Commonsense reasoning about social interactions](https://md.research.elamur.ai/free/en/papers/sap-2019-socialiqa-commonsense-reasoning-about-social/). arXiv preprint arXiv:1904.09728.
29. Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al. 2023a. [Llama: Open and efficient foundation language models](https://md.research.elamur.ai/free/en/papers/touvron-2023-llama-open-efficient-foundation-language/). arXiv preprint arXiv:2302.13971.
30. Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al. 2023b. [Llama: Open and efficient foundation language models](https://md.research.elamur.ai/free/en/papers/touvron-2023-llama-open-efficient-foundation-language/). arXiv preprint arXiv:2302.13971.
31. Ben Wang and Aran Komatsuzaki. 2021. GPT-J-6B: A 6 Billion Parameter Autoregressive Language Model. [https://github.com/](https://github.com/) kingoflolz/mesh-transformer-jax. [↗](https://github.com/)
32. Jiachen T Wang, Prateek Mittal, Dawn Song, and Ruoxi Jia. 2024a. Data shapley in one training run. arXiv preprint arXiv:2406.11011. — [arXiv:2406.11011](https://arxiv.org/abs/2406.11011) [find](https://scholar.google.com/scholar?q=Jiachen+T+Wang%2C+Prateek+Mittal%2C+Dawn+Song%2C+and+Ruoxi+Jia%2E+2024a%2E+Data+shapley+in+one+training+run%2E+arXiv+preprint+arXiv%3A2406%2E11011%2E+%E2%80%94+%5BarXiv%3A2406%2E11011%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2406%2E11011%29)
33. Jiachen T Wang, Tong Wu, Dawn Song, Prateek Mittal, and Ruoxi Jia. 2024b. Greats: online selection of high-quality data for llm training in every iteration. In Proceedings of the 38th International Conference on Neural Information Processing Systems, pages 131197–131223. [find](https://scholar.google.com/scholar?q=Jiachen+T+Wang%2C+Tong+Wu%2C+Dawn+Song%2C+Prateek+Mittal%2C+and+Ruoxi+Jia%2E+2024b%2E+Greats%3A+online+selection+of+high%2Dquality+data+for+llm+training+in+every+iteration%2E+In+Proceedings+of+the+38th+International+Co)
34. Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. 2022. [Chain-of-thought prompting elicits reasoning in large language models](https://md.research.elamur.ai/free/en/papers/wei-2022-chain-thought-prompting-elicits-reasoning/). Advances in neural information processing systems, 35:24824–24837.
35. Mengzhou Xia, Sadhika Malladi, Suchin Gururangan, Sanjeev Arora, and Danqi Chen. 2024. [Less: Selecting influential data for targeted instruction tuning](https://md.research.elamur.ai/free/en/papers/xia-2024-less-selecting-influential-data-targeted/). arXiv preprint arXiv:2402.04333.
36. Kai Yao, Penglei Gao, Lichun Li, Yuan Zhao, Xiaofeng Wang, Wei Wang, and Jianke Zhu. 2024. Layer-wise importance matters: Less memory for better performance in parameter-efficient fine-tuning of large language models. In Findings of the Association for Computational Linguistics: EMNLP 2024, pages 19… [find](https://scholar.google.com/scholar?q=Kai+Yao%2C+Penglei+Gao%2C+Lichun+Li%2C+Yuan+Zhao%2C+Xiaofeng+Wang%2C+Wei+Wang%2C+and+Jianke+Zhu%2E+2024%2E+Layer%2Dwise+importance+matters%3A+Less+memory+for+better+performance+in+parameter%2Defficient+fine%2Dtuning+of+large)
37. Rowan Zellers, Ari Holtzman, Yonatan Bisk, Ali Farhadi, and Yejin Choi. 2019. Hellaswag: Can a machine really finish your sentence? arXiv preprint arXiv:1905.07830. [find](https://scholar.google.com/scholar?q=Rowan+Zellers%2C+Ari+Holtzman%2C+Yonatan+Bisk%2C+Ali+Farhadi%2C+and+Yejin+Choi%2E+2019%2E+Hellaswag%3A+Can+a+machine+really+finish+your+sentence%3F+arXiv+preprint+arXiv%3A1905%2E07830%2E)
38. Qingru Zhang, Minshuo Chen, Alexander Bukharin, Pengcheng He, Yu Cheng, Weizhu Chen, and Tuo Zhao. 2023. Adaptive budget allocation for parameter-efficient fine-tuning. In International Conference on Learning Representations. Openreview. [find](https://scholar.google.com/scholar?q=Qingru+Zhang%2C+Minshuo+Chen%2C+Alexander+Bukharin%2C+Pengcheng+He%2C+Yu+Cheng%2C+Weizhu+Chen%2C+and+Tuo+Zhao%2E+2023%2E+Adaptive+budget+allocation+for+parameter%2Defficient+fine%2Dtuning%2E+In+International+Conference+on+)

*Updated 2 November 2026*