---
title: "UORA — paper by Zhang et al. (2025) — summary and ideas"
description: "The authors propose UORA, which keeps VeRA's design of frozen matrices plus scaling vectors and… 98 references, cited by 10 papers here."
authors: ["Xueyan Zhang", "Jinman Zhao", "Zhifei Yang", "Yibo Zhong", "Shuhao Guan", "Linbo Cao", "Yining Wang"]
year: 2025
venue: "Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)"
source: "https://aclanthology.org/2025.acl-long.575/"
doi: "10.18653/v1/2025.acl-long.575"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/zhang-2025-uora-uniform-orthogonal-reinitialization-adaptation/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/zhang-2025-uora-uniform-orthogonal-reinitialization-adaptation/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2025](https://md.research.elamur.ai/free/en/papers/2025/page/6/) / UORA: Uniform Orthogonal Reinitialization Adaptation in Parameter Efficient Fine-Tuning of Large Models

# UORA: Uniform Orthogonal Reinitialization Adaptation in Parameter Efficient Fine-Tuning of Large Models

**Authors:** [Xueyan Zhang](https://md.research.elamur.ai/free/en/authors/xueyan-zhang/), [Jinman Zhao](https://md.research.elamur.ai/free/en/authors/jinman-zhao/), Zhifei Yang, Yibo Zhong, [Shuhao Guan](https://md.research.elamur.ai/free/en/authors/shuhao-guan/), [Linbo Cao](https://md.research.elamur.ai/free/en/authors/linbo-cao/) and [Yining Wang](https://md.research.elamur.ai/free/en/authors/yining-wang/) · **Year:** 2025 · **Venue:** [Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)](https://md.research.elamur.ai/free/en/venues/acl/) ([ACL 2025](https://md.research.elamur.ai/free/en/venues/acl/2025/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2025.acl-long.575/) · [PDF](https://aclanthology.org/2025.acl-long.575.pdf) · [DOI 10.18653/v1/2025.acl-long.575](https://doi.org/10.18653/v1/2025.acl-long.575) · [OpenAlex](https://openalex.org/W4412945377)

**Citations:** 4 citations (OpenAlex, 2026-10-14) · [10](https://md.research.elamur.ai/free/en/papers/zhang-2025-uora-uniform-orthogonal-reinitialization-adaptation/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
Fully fine-tuning large models is expensive in compute and in storage for per-task checkpoints. LoRA lowers this cost by training low-rank matrices. VeRA goes further: it freezes random projection matrices and trains only two scaling vectors, but it needs a high rank to stay competitive, which raises computation. The authors propose UORA, which keeps VeRA's design of frozen matrices plus scaling vectors and adds a selective refresh of the frozen matrices. When a scaling-vector entry stays below a magnitude threshold τ for k consecutive iterations, the matching column of A and row of B are redrawn. Old and new values are blended by linear interpolation with factor α to keep training stable. The frozen matrices use orthogonal uniform initialization rather than Kaiming. Evaluation covers GLUE, E2E, arithmetic-reasoning instruction tuning of LLaMA 7B and LLaMA 3-8B, and ViT image classification. UORA comes close to LoRA with far fewer trainable parameters and beats VeRA at a much lower rank. Its updates merge into the base weights, so inference latency is unchanged.

## Key points
- Compared with LoRA, UORA needs about 15x fewer trainable parameters on GLUE and 8x fewer on E2E, with competitive scores.
- On LLaMA3-8B AddSub, UORA scores 89.11% versus LoRA's 89.62%.
- On ViT-B CIFAR100, UORA reaches 96.08% with 19.2K trainable parameters, versus 95.38% for VeRA (24.6K) and 96.78% for LoRA (294.9K). LoRA was run at rank 8, VeRA at rank 256, and UORA at rank 32.
- In an ablation on MRPC, orthogonal uniform and Xavier initialization did slightly better than Kaiming uniform, while random uniform did poorly.
- Code is released at github.com/zhaojinm/UORA. Hyperparameter tuning guidance for τ, k and α is given in the appendices.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

📚 Shelves (5): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works · [👁️ Computer Vision and Pattern Recognition](https://md.research.elamur.ai/free/en/shelves/computer-science/computer-vision-and-pattern-recognition/) · 2,869 works

## Cite

Cite as: Xueyan Zhang, Jinman Zhao, Zhifei Yang, Yibo Zhong, Shuhao Guan, Linbo Cao and Yining Wang. 2025. UORA: Uniform Orthogonal Reinitialization Adaptation in Parameter Efficient Fine-Tuning of Large Models. Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{zhang2025uora,
  title = {{UORA: Uniform Orthogonal Reinitialization Adaptation in Parameter Efficient Fine-Tuning of Large Models}},
  author = {Xueyan Zhang, Jinman Zhao, Zhifei Yang, Yibo Zhong, Shuhao Guan, Linbo Cao and Yining Wang},
  booktitle = {Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2025},
  doi = {10.18653/v1/2025.acl-long.575},
  url = {https://doi.org/10.18653/v1/2025.acl-long.575}
}
```

- [← previous UnSeenTimeQA: Time-Sensitive Question-Answering Beyond LLMs' Memorization](https://md.research.elamur.ai/free/en/papers/uddin-2025-unseentimeqa-time-sensitive-question-answering/)
- [next → A Reality Check on Context Utilisation for Retrieval-Augmented Generation](https://md.research.elamur.ai/free/en/papers/hagstrom-2025-reality-check-context-utilisation-retrieval/)
- [↑ back to papers of 2025](https://md.research.elamur.ai/free/en/papers/2025/page/6/)

## References

98 references, 34 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Armen Aghajanyan, Luke Zettlemoyer, and Sonal Gupta. 2020. [Intrinsic dimensionality explains the effectiveness of language model fine-tuning](https://md.research.elamur.ai/free/en/papers/aghajanyan-2021-intrinsic-dimensionality-explains-effectiveness-language/). arXiv preprint arXiv:2012.13255.
2. Leonard Berrada, Andrew Zisserman, and M Pawan Kumar. 2020. Training neural networks for and by interpolation. In International conference on machine learning, pages 799–809. PMLR. [find](https://scholar.google.com/scholar?q=Leonard+Berrada%2C+Andrew+Zisserman%2C+and+M+Pawan+Kumar%2E+2020%2E+Training+neural+networks+for+and+by+interpolation%2E+In+International+conference+on+machine+learning%2C+pages+799%E2%80%93809%2E+PMLR%2E)
3. Jinhe Bi, Yujun Wang, Haokun Chen, Xun Xiao, Artur Hecker, Volker Tresp, and Yunpu Ma. 2025a. [Llava steering: Visual instruction tuning with 500x fewer parameters through modality linear representation-steering](https://md.research.elamur.ai/free/en/papers/bi-2025-llava-steering-visual-instruction-tuning/). Preprint, arXiv:2412.12359.
4. Jinhe Bi, Danqi Yan, Yifan Wang, Wenke Huang, Haokun Chen, Guancheng Wan, Mang Ye, Xun Xiao, Hinrich Schuetze, Volker Tresp, and Yunpu Ma. 2025b. Cot-kinetics: A theoretical modeling assessing lrm reasoning process. Preprint, arXiv:2505.13408. — [arXiv:2505.13408](https://arxiv.org/abs/2505.13408) [find](https://scholar.google.com/scholar?q=Jinhe+Bi%2C+Danqi+Yan%2C+Yifan+Wang%2C+Wenke+Huang%2C+Haokun+Chen%2C+Guancheng+Wan%2C+Mang+Ye%2C+Xun+Xiao%2C+Hinrich+Schuetze%2C+Volker+Tresp%2C+and+Yunpu+Ma%2E+2025b%2E+Cot%2Dkinetics%3A+A+theoretical+modeling+assessing+lrm+rea)
5. Lukas Bossard, Matthieu Guillaumin, and Luc Van Gool. 2014. Food-101–mining discriminative components with random forests. In Computer vision–ECCV 2014: 13th European conference, zurich, Switzerland, September 6-12, 2014, proceedings, part VI 13, pages 446–461. Springer. [find](https://scholar.google.com/scholar?q=Lukas+Bossard%2C+Matthieu+Guillaumin%2C+and+Luc+Van+Gool%2E+2014%2E+Food%2D101%E2%80%93mining+discriminative+components+with+random+forests%2E+In+Computer+vision%E2%80%93ECCV+2014%3A+13th+European+conference%2C+zurich%2C+Switzerland%2C+)
6. Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. 2020. [Language models are few-shot learners](https://md.research.elamur.ai/free/en/papers/brown-2020-language-models-few-shot-learners/). Advances in neural information processing systems, 33:1877–1901.
7. Chengkun Cai, Xu Zhao, Yucheng Du, Haoliang Liu, and Lei Li. 2025. t 2 of thoughts: Temperature tree elicits reasoning in large language models. Preprint, arXiv:2405.14075. — [arXiv:2405.14075](https://arxiv.org/abs/2405.14075) [find](https://scholar.google.com/scholar?q=Chengkun+Cai%2C+Xu+Zhao%2C+Yucheng+Du%2C+Haoliang+Liu%2C+and+Lei+Li%2E+2025%2E+t+2+of+thoughts%3A+Temperature+tree+elicits+reasoning+in+large+language+models%2E+Preprint%2C+arXiv%3A2405%2E14075%2E+%E2%80%94+%5BarXiv%3A2405%2E14075%5D%28https%3A)
8. Guanzheng Chen, Fangyu Liu, Zaiqiao Meng, and Shangsong Liang. 2022. Revisiting parameter-efficient tuning: Are we really there yet? arXiv preprint arXiv:2202.07962. [find](https://scholar.google.com/scholar?q=Guanzheng+Chen%2C+Fangyu+Liu%2C+Zaiqiao+Meng%2C+and+Shangsong+Liang%2E+2022%2E+Revisiting+parameter%2Defficient+tuning%3A+Are+we+really+there+yet%3F+arXiv+preprint+arXiv%3A2202%2E07962%2E)
9. Jiali Chen, Xusen Hei, Yuqi Xue, Yuancheng Wei, Jiayuan Xie, Yi Cai, and Qing Li. 2024. Learning to correction: Explainable feedback generation for visual commonsense reasoning distractor. In Proceedings of the 32nd ACM International Conference on Multimedia, MM ’24, page 8209–8218. ACM. [find](https://scholar.google.com/scholar?q=Jiali+Chen%2C+Xusen+Hei%2C+Yuqi+Xue%2C+Yuancheng+Wei%2C+Jiayuan+Xie%2C+Yi+Cai%2C+and+Qing+Li%2E+2024%2E+Learning+to+correction%3A+Explainable+feedback+generation+for+visual+commonsense+reasoning+distractor%2E+In+Proceedi)
10. Jiali Chen, Xusen Hei, Yuqi Xue, Zihan Wu, Jiayuan Xie, and Yi Cai. 2025. Classic4Children: Adapting Chinese literary classics for children with large language model. In Findings of the Association for Computational Linguistics: NAACL 2025, pages 2473– 2488, Albuquerque, New Mexico. Association for… [find](https://scholar.google.com/scholar?q=Jiali+Chen%2C+Xusen+Hei%2C+Yuqi+Xue%2C+Zihan+Wu%2C+Jiayuan+Xie%2C+and+Yi+Cai%2E+2025%2E+Classic4Children%3A+Adapting+Chinese+literary+classics+for+children+with+large+language+model%2E+In+Findings+of+the+Association+fo)
11. Gong Cheng, Junwei Han, and Xiaoqiang Lu. 2017. Remote sensing image scene classification: Benchmark and state of the art. Proceedings of the IEEE, 105(10):1865–1883. [find](https://scholar.google.com/scholar?q=Gong+Cheng%2C+Junwei+Han%2C+and+Xiaoqiang+Lu%2E+2017%2E+Remote+sensing+image+scene+classification%3A+Benchmark+and+state+of+the+art%2E+Proceedings+of+the+IEEE%2C+105%2810%29%3A1865%E2%80%931883%2E)
12. Jingcheng Deng, Zihao Wei, Liang Pang, Hanxing Ding, Huawei Shen, and Xueqi Cheng. 2025. Everything is editable: Extend knowledge editing to unstructured data in large language models. In The Thirteenth International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Jingcheng+Deng%2C+Zihao+Wei%2C+Liang+Pang%2C+Hanxing+Ding%2C+Huawei+Shen%2C+and+Xueqi+Cheng%2E+2025%2E+Everything+is+editable%3A+Extend+knowledge+editing+to+unstructured+data+in+large+language+models%2E+In+The+Thirteen)
13. Tim Dettmers, Artidoro Pagnoni, Ari Holtzman, and Luke Zettlemoyer. 2023. [QLoRA: Efficient finetuning of quantized LLMs](https://md.research.elamur.ai/free/en/papers/dettmers-2023-qlora-efficient-finetuning-quantized-llms/). In Thirty-seventh Conference on Neural Information Processing Systems.
14. Ning Ding, Xingtai Lv, Qiaosen Wang, Yulin Chen, Bowen Zhou, Zhiyuan Liu, and Maosong Sun. 2023. Sparse low-rank adaptation of pre-trained language models. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 4133–4145, Singapore. Association for Computati… [find](https://scholar.google.com/scholar?q=Ning+Ding%2C+Xingtai+Lv%2C+Qiaosen+Wang%2C+Yulin+Chen%2C+Bowen+Zhou%2C+Zhiyuan+Liu%2C+and+Maosong+Sun%2E+2023%2E+Sparse+low%2Drank+adaptation+of+pre%2Dtrained+language+models%2E+In+Proceedings+of+the+2023+Conference+on+Emp)
15. Junhao Dong, Yuan Wang, Xiaohua Xie, Jianhuang Lai, and Yew-Soon Ong. 2024a. Generalizable and discriminative representations for adversarially robust few-shot learning. IEEE Transactions on Neural Networks and Learning Systems. [find](https://scholar.google.com/scholar?q=Junhao+Dong%2C+Yuan+Wang%2C+Xiaohua+Xie%2C+Jianhuang+Lai%2C+and+Yew%2DSoon+Ong%2E+2024a%2E+Generalizable+and+discriminative+representations+for+adversarially+robust+few%2Dshot+learning%2E+IEEE+Transactions+on+Neural+Ne)
16. Junhao Dong, Melvin Wong, Sihan Xia, and Joel Wei En Tay. 2024b. Towards adversarially robust data-efficient learning with generated data. In 2024 IEEE Conference on Artificial Intelligence (CAI), pages 1422–1424. IEEE. [find](https://scholar.google.com/scholar?q=Junhao+Dong%2C+Melvin+Wong%2C+Sihan+Xia%2C+and+Joel+Wei+En+Tay%2E+2024b%2E+Towards+adversarially+robust+data%2Defficient+learning+with+generated+data%2E+In+2024+IEEE+Conference+on+Artificial+Intelligence+%28CAI%29%2C+pag)
17. Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby. 2021. [An image is worth 16x16 words: Transformers for image recognition at scale](https://md.research.elamur.ai/free/en/papers/dosovitskiy-2020-image-worth-16x16-words-transformers/). In Intern…
18. Shihan Dou, Enyu Zhou, Yan Liu, Songyang Gao, Wei Shen, Limao Xiong, Yuhao Zhou, Xiao Wang, Zhiheng Xi, Xiaoran Fan, Shiliang Pu, Jiang Zhu, Rui Zheng, Tao Gui, Qi Zhang, and Xuanjing Huang. 2024. [LoRAMoE: Alleviating world knowledge forgetting in large language models via MoE-style plugin](https://md.research.elamur.ai/free/en/papers/dou-2024-loramoe-alleviating-world-knowledge-forgetting/). In Proce…
19. Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al. 2024. [The llama 3 herd of models](https://md.research.elamur.ai/free/en/papers/grattafiori-2024-llama-3-herd-models/). arXiv preprint arXiv:2407.21783.
20. Benjamin Eysenbach, Vivek Myers, Ruslan Salakhutdinov, and Sergey Levine. 2024. Inference via interpolation: Contrastive representations provably enable planning and inference. In Advances in Neural Information Processing Systems, volume 37, pages 58901–58928. Curran Associates, Inc. [find](https://scholar.google.com/scholar?q=Benjamin+Eysenbach%2C+Vivek+Myers%2C+Ruslan+Salakhutdinov%2C+and+Sergey+Levine%2E+2024%2E+Inference+via+interpolation%3A+Contrastive+representations+provably+enable+planning+and+inference%2E+In+Advances+in+Neural+I)
21. Yuchun Fan, Yongyu Mu, YiLin Wang, Lei Huang, Junhao Ruan, Bei Li, Tong Xiao, Shujian Huang, Xiaocheng Feng, and Jingbo Zhu. 2025. [SLAM: Towards efficient multilingual reasoning via selective language alignment](https://md.research.elamur.ai/free/en/papers/fan-2025-slam-efficient-multilingual-reasoning-selective/). In Proceedings of the 31st International Conference on Computational Linguistics, pages…
22. Han Fang, Zhifei Yang, Yuhan Wei, Xianghao Zang, Chao Ban, Zerun Feng, Zhongjiang He, Yongxiang Li, and Hao Sun. 2023. Alignment and generation adapter for efficient video-text understanding. In ICCV (Workshops), pages 2783–2789. [find](https://scholar.google.com/scholar?q=Han+Fang%2C+Zhifei+Yang%2C+Yuhan+Wei%2C+Xianghao+Zang%2C+Chao+Ban%2C+Zerun+Feng%2C+Zhongjiang+He%2C+Yongxiang+Li%2C+and+Hao+Sun%2E+2023%2E+Alignment+and+generation+adapter+for+efficient+video%2Dtext+understanding%2E+In+ICCV+)
23. Yujie Feng, Bo Liu, Xiaoyu Dong, Zexin Lu, Li-Ming Zhan, Xiao-Ming Wu, and Albert Lam. 2024. Continual dialogue state tracking via reason-of-select distillation. In Findings of the Association for Computational Linguistics ACL 2024, pages 7075–7087. [find](https://scholar.google.com/scholar?q=Yujie+Feng%2C+Bo+Liu%2C+Xiaoyu+Dong%2C+Zexin+Lu%2C+Li%2DMing+Zhan%2C+Xiao%2DMing+Wu%2C+and+Albert+Lam%2E+2024%2E+Continual+dialogue+state+tracking+via+reason%2Dof%2Dselect+distillation%2E+In+Findings+of+the+Association+for+Com)
24. Yujie Feng, Liming Zhan, Zexin Lu, Yongxin Xu, Xu Chu, Yasha Wang, Jiannong Cao, Philip S Yu, and Xiao-Ming Wu. 2025. Geoedit: Geometric knowledge editing for large language models. arXiv preprint arXiv:2502.19953. [find](https://scholar.google.com/scholar?q=Yujie+Feng%2C+Liming+Zhan%2C+Zexin+Lu%2C+Yongxin+Xu%2C+Xu+Chu%2C+Yasha+Wang%2C+Jiannong+Cao%2C+Philip+S+Yu%2C+and+Xiao%2DMing+Wu%2E+2025%2E+Geoedit%3A+Geometric+knowledge+editing+for+large+language+models%2E+arXiv+preprint+arX)
25. Jonathan Frankle and Michael Carbin. 2018. The lottery ticket hypothesis: Finding sparse, trainable neural networks. arXiv preprint arXiv:1803.03635. [find](https://scholar.google.com/scholar?q=Jonathan+Frankle+and+Michael+Carbin%2E+2018%2E+The+lottery+ticket+hypothesis%3A+Finding+sparse%2C+trainable+neural+networks%2E+arXiv+preprint+arXiv%3A1803%2E03635%2E)
26. Ziqi Gao, Qichao Wang, Aochuan Chen, Zijing Liu, Bingzhe Wu, Liang Chen, and Jia Li. 2024. Parameter-efficient fine-tuning with discrete Fourier transform. In Proceedings of the 41st International Conference on Machine Learning, volume 235 of Proceedings of Machine Learning Research, pages 14884–149… [find](https://scholar.google.com/scholar?q=Ziqi+Gao%2C+Qichao+Wang%2C+Aochuan+Chen%2C+Zijing+Liu%2C+Bingzhe+Wu%2C+Liang+Chen%2C+and+Jia+Li%2E+2024%2E+Parameter%2Defficient+fine%2Dtuning+with+discrete+Fourier+transform%2E+In+Proceedings+of+the+41st+International+Con)
27. Han Guo, Philip Greengard, Eric Xing, and Yoon Kim. 2024. LQ-loRA: Low-rank plus quantized matrix decomposition for efficient language model finetuning. In The Twelfth International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Han+Guo%2C+Philip+Greengard%2C+Eric+Xing%2C+and+Yoon+Kim%2E+2024%2E+LQ%2DloRA%3A+Low%2Drank+plus+quantized+matrix+decomposition+for+efficient+language+model+finetuning%2E+In+The+Twelfth+International+Conference+on+Lear)
28. Karen Hambardzumyan, Hrant Khachatrian, and Jonathan May. 2021. Warp: Word-level adversarial reprogramming. arXiv preprint arXiv:2101.00121. [find](https://scholar.google.com/scholar?q=Karen+Hambardzumyan%2C+Hrant+Khachatrian%2C+and+Jonathan+May%2E+2021%2E+Warp%3A+Word%2Dlevel+adversarial+reprogramming%2E+arXiv+preprint+arXiv%3A2101%2E00121%2E)
29. Junxian He, Chunting Zhou, Xuezhe Ma, Taylor Berg-Kirkpatrick, and Graham Neubig. 2021. [Towards a unified view of parameter-efficient transfer learning](https://md.research.elamur.ai/free/en/papers/he-2021-towards-unified-view-parameter-efficient/). arXiv preprint arXiv:2110.04366.
30. Mohammad Javad Hosseini, Hannaneh Hajishirzi, Oren Etzioni, and Nate Kushman. 2014. [Learning to solve arithmetic word problems with verb categorization](https://md.research.elamur.ai/free/en/papers/hosseini-2014-learning-solve-arithmetic-word-problems/). In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 523–533, Doha, Qatar. Association for Comp…
31. Neil Houlsby, Andrei Giurgiu, Stanislaw Jastrzebski, Bruna Morrone, Quentin De Laroussilhe, Andrea Gesmundo, Mona Attariyan, and Sylvain Gelly. 2019. Parameter-efficient transfer learning for nlp. In International conference on machine learning, pages 2790–2799. PMLR. [find](https://scholar.google.com/scholar?q=Neil+Houlsby%2C+Andrei+Giurgiu%2C+Stanislaw+Jastrzebski%2C+Bruna+Morrone%2C+Quentin+De+Laroussilhe%2C+Andrea+Gesmundo%2C+Mona+Attariyan%2C+and+Sylvain+Gelly%2E+2019%2E+Parameter%2Defficient+transfer+learning+for+nlp%2E+In+)
32. Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2021. [Lora: Low-rank adaptation of large language models](https://md.research.elamur.ai/free/en/papers/hu-2021-lora-low-rank-adaptation-large/). arXiv preprint arXiv:2106.09685.
33. Wei Hu, Lechao Xiao, and Jeffrey Pennington. 2020. Provable benefit of orthogonal initialization in optimizing deep linear networks. In International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Wei+Hu%2C+Lechao+Xiao%2C+and+Jeffrey+Pennington%2E+2020%2E+Provable+benefit+of+orthogonal+initialization+in+optimizing+deep+linear+networks%2E+In+International+Conference+on+Learning+Representations%2E)
34. Zhiqiang Hu, Lei Wang, Yihuai Lan, Wanyu Xu, Ee-Peng Lim, Lidong Bing, Xing Xu, Soujanya Poria, and Roy Lee. 2023. [LLM-adapters: An adapter family for parameter-efficient fine-tuning of large language models](https://md.research.elamur.ai/free/en/papers/hu-2023-llm-adapters-adapter-family-parameter/). In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing,…
35. Zhiyuan Hu, Yuliang Liu, Jinman Zhao, Suyuchen Wang, Yan Wang, Wei Shen, Qing Gu, Anh Tuan Luu, See-Kiong Ng, Zhiwei Jiang, and Bryan Hooi. 2024. [Longrecipe: Recipe for efficient long context generalization in large language models](https://md.research.elamur.ai/free/en/papers/hu-2025-longrecipe-recipe-efficient-long-context/). Preprint, arXiv:2409.00509.
36. Wei Huang, Weitao Du, and Richard Yi Da Xu. 2021. On the neural tangent kernel of deep networks with orthogonal initialization. In Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence, IJCAI-21, pages 2577–2583. International Joint Conferences on Artificial Intellig… [find](https://scholar.google.com/scholar?q=Wei+Huang%2C+Weitao+Du%2C+and+Richard+Yi+Da+Xu%2E+2021%2E+On+the+neural+tangent+kernel+of+deep+networks+with+orthogonal+initialization%2E+In+Proceedings+of+the+Thirtieth+International+Joint+Conference+on+Artifi)
37. Gaurav Iyer, Gintare Karolina Dziugaite, and David Rolnick. 2024. Linear weight interpolation leads to transient performance gains. Transactions on Machine Learning Research. [find](https://scholar.google.com/scholar?q=Gaurav+Iyer%2C+Gintare+Karolina+Dziugaite%2C+and+David+Rolnick%2E+2024%2E+Linear+weight+interpolation+leads+to+transient+performance+gains%2E+Transactions+on+Machine+Learning+Research%2E)
38. Chen Jia. 2024. Adversarial moment-matching distillation of large language models. In The Thirty-eighth Annual Conference on Neural Information Processing Systems. [find](https://scholar.google.com/scholar?q=Chen+Jia%2E+2024%2E+Adversarial+moment%2Dmatching+distillation+of+large+language+models%2E+In+The+Thirty%2Deighth+Annual+Conference+on+Neural+Information+Processing+Systems%2E)
39. Yiyao Yu, Yuxiang Zhang, Dongdong Zhang, Xiao Liang, Hengyuan Zhang, Xingxing Zhang, Mahmoud Khademi, Hany Awadalla, Junjie Wang, Yujiu Yang, and Furu Wei. 2025. [Chain-of-Reasoning: Towards Unified Mathematical Reasoning in Large Language Models via a Multi-Paradigm Perspective](https://md.research.elamur.ai/free/en/papers/yu-2025-chain-reasoning-unified-mathematical-reasoning/). Preprint, arXiv:2501…
40. Mingyu Jin, Weidi Luo, Sitao Cheng, Xinyi Wang, Wenyue Hua, Ruixiang Tang, William Yang Wang, and Yongfeng Zhang. 2024. [Disentangling memory and reasoning ability in large language models](https://md.research.elamur.ai/free/en/papers/jin-2025-disentangling-memory-reasoning-ability-large/). Preprint, arXiv:2411.13504.
41. Minsoo Kim, Sihwa Lee, Wonyong Sung, and Jungwook Choi. 2024. RA-LoRA: Rank-adaptive parameter-efficient fine-tuning for accurate 2-bit quantized large language models. In Findings of the Association for Computational Linguistics: ACL 2024, pages 15773–15786, Bangkok, Thailand. Association for Compu… [find](https://scholar.google.com/scholar?q=Minsoo+Kim%2C+Sihwa+Lee%2C+Wonyong+Sung%2C+and+Jungwook+Choi%2E+2024%2E+RA%2DLoRA%3A+Rank%2Dadaptive+parameter%2Defficient+fine%2Dtuning+for+accurate+2%2Dbit+quantized+large+language+models%2E+In+Findings+of+the+Association+)
42. Rik Koncel-Kedziorski, Hannaneh Hajishirzi, Ashish Sabharwal, Oren Etzioni, and Siena Dumas Ang. 2015. Parsing algebraic word problems into equations. Transactions of the Association for Computational Linguistics, 3:585–597. [find](https://scholar.google.com/scholar?q=Rik+Koncel%2DKedziorski%2C+Hannaneh+Hajishirzi%2C+Ashish+Sabharwal%2C+Oren+Etzioni%2C+and+Siena+Dumas+Ang%2E+2015%2E+Parsing+algebraic+word+problems+into+equations%2E+Transactions+of+the+Association+for+Computational)
43. Dawid J Kopiczko, Tijmen Blankevoort, and Yuki M Asano. 2023. [Vera: Vector-based random matrix adaptation](https://md.research.elamur.ai/free/en/papers/kopiczko-2023-vera-vector-based-random-matrix/). arXiv preprint arXiv:2310.11454.
44. Alex Krizhevsky, Geoffrey Hinton, et al. 2009. Learning multiple layers of features from tiny images. [find](https://scholar.google.com/scholar?q=Alex+Krizhevsky%2C+Geoffrey+Hinton%2C+et+al%2E+2009%2E+Learning+multiple+layers+of+features+from+tiny+images%2E)
45. Brian Lester, Rami Al-Rfou, and Noah Constant. 2021. [The power of scale for parameter-efficient prompt tuning](https://md.research.elamur.ai/free/en/papers/lester-2021-power-scale-parameter-efficient-prompt/). arXiv preprint arXiv:2104.08691.
46. Lei Li. 2024. Cpseg: Finer-grained image semantic segmentation via chain-of-thought language prompting. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 513–522. [find](https://scholar.google.com/scholar?q=Lei+Li%2E+2024%2E+Cpseg%3A+Finer%2Dgrained+image+semantic+segmentation+via+chain%2Dof%2Dthought+language+prompting%2E+In+Proceedings+of+the+IEEE%2FCVF+Winter+Conference+on+Applications+of+Computer+Vision%2C+pages+513%E2%80%935)
47. Xuetao Ma, Wenbin Jiang, and Hua Huang. 2025. Problem-solving logic guided curriculum in-context learning for llms complex reasoning. Preprint, arXiv:2502.15401. [find](https://scholar.google.com/scholar?q=Xuetao+Ma%2C+Wenbin+Jiang%2C+and+Hua+Huang%2E+2025%2E+Problem%2Dsolving+logic+guided+curriculum+in%2Dcontext+learning+for+llms+complex+reasoning%2E+Preprint%2C+arXiv%3A2502%2E15401%2E)
48. Humza Naveed, Asad Ullah Khan, Shi Qiu, Muhammad Saqib, Saeed Anwar, Muhammad Usman, Naveed Akhtar, Nick Barnes, and Ajmal Mian. 2023. [A comprehensive overview of large language models](https://md.research.elamur.ai/free/en/papers/naveed-2023-comprehensive-overview-large-language-models/). arXiv preprint arXiv:2307.06435.
49. Wenhao Li, Yuxin Zhang, Gen Luo, Daohai Yu, and Rongrong Ji. 2025a. Training long-context llms efficiently via chunk-wise optimization. Preprint, arXiv:2505.16710. [find](https://scholar.google.com/scholar?q=Wenhao+Li%2C+Yuxin+Zhang%2C+Gen+Luo%2C+Daohai+Yu%2C+and+Rongrong+Ji%2E+2025a%2E+Training+long%2Dcontext+llms+efficiently+via+chunk%2Dwise+optimization%2E+Preprint%2C+arXiv%3A2505%2E16710%2E)
50. Maria-Elena Nilsback and Andrew Zisserman. 2008. Automated flower classification over a large number of classes. In Indian Conference on Computer Vision, Graphics and Image Processing. [find](https://scholar.google.com/scholar?q=Maria%2DElena+Nilsback+and+Andrew+Zisserman%2E+2008%2E+Automated+flower+classification+over+a+large+number+of+classes%2E+In+Indian+Conference+on+Computer+Vision%2C+Graphics+and+Image+Processing%2E)
51. Xiang Lisa Li and Percy Liang. 2021. Prefix-tuning: Optimizing continuous prompts for generation. arXiv preprint arXiv:2101.00190. [find](https://scholar.google.com/scholar?q=Xiang+Lisa+Li+and+Percy+Liang%2E+2021%2E+Prefix%2Dtuning%3A+Optimizing+continuous+prompts+for+generation%2E+arXiv+preprint+arXiv%3A2101%2E00190%2E)
52. Jingcheng Niu, Andrew Liu, Zining Zhu, and Gerald Penn. 2024. What does the knowledge neuron thesis have to do with knowledge? In The Twelfth International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Jingcheng+Niu%2C+Andrew+Liu%2C+Zining+Zhu%2C+and+Gerald+Penn%2E+2024%2E+What+does+the+knowledge+neuron+thesis+have+to+do+with+knowledge%3F+In+The+Twelfth+International+Conference+on+Learning+Representations%2E)
53. Zhong-Zhi Li, Duzhen Zhang, Ming-Liang Zhang, Jiaxin Zhang, Zengyan Liu, Yuxuan Yao, Haotian Xu, Junhao Zheng, Pei-Jie Wang, Xiuyi Chen, Yingying Zhang, Fei Yin, Jiahua Dong, Zhiwei Li, Bao-Long Bi, Ling-Rui Mei, Junfeng Fang, Zhijiang Guo, Le Song, and Cheng-Lin Liu. 2025b. [From system 1 to system…](https://md.research.elamur.ai/free/en/papers/li-2025-system-1-system-2-survey/)
54. Zhaojiang Lin, Andrea Madotto, and Pascale Fung. 2020. [Exploring versatile generative language model via parameter-efficient transfer learning](https://md.research.elamur.ai/free/en/papers/lin-2020-exploring-versatile-generative-language-model/). arXiv preprint arXiv:2004.03829.
55. Wanlong Liu, Junying Chen, Ke Ji, Li Zhou, Wenyu Chen, and Benyou Wang. 2024a. [Rag-instruct: Boosting llms with diverse retrieval-augmented instructions](https://md.research.elamur.ai/free/en/papers/liu-2025-rag-instruct-boosting-llms-diverse/). Preprint, arXiv:2501.00353.
56. Xiao Liu, Kaixuan Ji, Yicheng Fu, Weng Lam Tam, Zhengxiao Du, Zhilin Yang, and Jie Tang. 2021. [Ptuning v2: Prompt tuning can be comparable to fine-tuning universally across scales and tasks](https://md.research.elamur.ai/free/en/papers/liu-2021-p-tuning-v2-prompt-tuning/). arXiv preprint arXiv:2110.07602.
57. Xiao Liu, Yanan Zheng, Zhengxiao Du, Ming Ding, Yujie Qian, Zhilin Yang, and Jie Tang. 2024b. Gpt understands, too. AI Open, 5:208–215. [find](https://scholar.google.com/scholar?q=Xiao+Liu%2C+Yanan+Zheng%2C+Zhengxiao+Du%2C+Ming+Ding%2C+Yujie+Qian%2C+Zhilin+Yang%2C+and+Jie+Tang%2E+2024b%2E+Gpt+understands%2C+too%2E+AI+Open%2C+5%3A208%E2%80%93215%2E)
58. Yinhan Liu. 2019. [Roberta: A robustly optimized bert pretraining approach](https://md.research.elamur.ai/free/en/papers/liu-2019-roberta-robustly-optimized-bert-pretraining/). arXiv preprint arXiv:1907.11692, 364.
59. Zeyu Liu, Souvik Kundu, Anni Li, Junrui Wan, Lianghao Jiang, and Peter Anthony Beerel. 2024c. Aflora: Adaptive freezing of low rank adaptation in parameter efficient fine-tuning of large models. arXiv preprint arXiv:2403.13269. [find](https://scholar.google.com/scholar?q=Zeyu+Liu%2C+Souvik+Kundu%2C+Anni+Li%2C+Junrui+Wan%2C+Lianghao+Jiang%2C+and+Peter+Anthony+Beerel%2E+2024c%2E+Aflora%3A+Adaptive+freezing+of+low+rank+adaptation+in+parameter+efficient+fine%2Dtuning+of+large+models%2E+arXiv)
60. Yingfeng Luo, Tong Zheng, Yongyu Mu, Bei Li, Qinghong Zhang, Yongqi Gao, Ziqiang Xu, Peinan Feng, Xiaoqian Liu, Tong Xiao, et al. 2025. Beyond decoder-only: Large language models can be good encoders for machine translation. arXiv preprint arXiv:2503.06594. [find](https://scholar.google.com/scholar?q=Yingfeng+Luo%2C+Tong+Zheng%2C+Yongyu+Mu%2C+Bei+Li%2C+Qinghong+Zhang%2C+Yongqi+Gao%2C+Ziqiang+Xu%2C+Peinan+Feng%2C+Xiaoqian+Liu%2C+Tong+Xiao%2C+et+al%2E+2025%2E+Beyond+decoder%2Donly%3A+Large+language+models+can+be+good+encoders+)
61. Jingcheng Niu, Wenjie Lu, and Gerald Penn. 2022. Does BERT rediscover a classical NLP pipeline? In Proceedings of the 29th International Conference on Computational Linguistics, pages 3143–3153, Gyeongju, Republic of Korea. International Committee on Computational Linguistics. [find](https://scholar.google.com/scholar?q=Jingcheng+Niu%2C+Wenjie+Lu%2C+and+Gerald+Penn%2E+2022%2E+Does+BERT+rediscover+a+classical+NLP+pipeline%3F+In+Proceedings+of+the+29th+International+Conference+on+Computational+Linguistics%2C+pages+3143%E2%80%933153%2C+Gyeon)
62. Jekaterina Novikova, Ondřej Dušek, and Verena Rieser. 2017. [The e2e dataset: New challenges for end-to-end generation](https://md.research.elamur.ai/free/en/papers/novikova-2017-e2e-dataset-new-challenges-end/). arXiv preprint arXiv:1706.09254.
63. Arkil Patel, Satwik Bhattamishra, and Navin Goyal. 2021. [Are NLP models really able to solve simple math word problems?](https://md.research.elamur.ai/free/en/papers/patel-2021-are-nlp-models-really-able/) In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 2080–2094, Online. Associat…
64. Boci Peng, Yongchao Liu, Xiaohe Bo, Sheng Tian, Baokun Wang, Chuntao Hong, and Yan Zhang. 2024a. Subgraph retrieval enhanced by graph-text alignment for commonsense question answering. In Machine Learning and Knowledge Discovery in Databases. Research Track - European Conference, ECML PKDD 2024, Vil… [find](https://scholar.google.com/scholar?q=Boci+Peng%2C+Yongchao+Liu%2C+Xiaohe+Bo%2C+Sheng+Tian%2C+Baokun+Wang%2C+Chuntao+Hong%2C+and+Yan+Zhang%2E+2024a%2E+Subgraph+retrieval+enhanced+by+graph%2Dtext+alignment+for+commonsense+question+answering%2E+In+Machine+Lear)
65. Boci Peng, Yun Zhu, Yongchao Liu, Xiaohe Bo, Haizhou Shi, Chuntao Hong, Yan Zhang, and Siliang Tang. 2024b. [Graph retrieval-augmented generation: A survey](https://md.research.elamur.ai/free/en/papers/peng-2024-graph-retrieval-augmented-generation-survey/). Preprint, arXiv:2408.08921.
66. Jonas Pfeiffer, Aishwarya Kamath, Andreas Rücklé, Kyunghyun Cho, and Iryna Gurevych. 2020. [Adapterfusion: Non-destructive task composition for transfer learning](https://md.research.elamur.ai/free/en/papers/pfeiffer-2020-adapterfusion-non-destructive-task-composition/). arXiv preprint arXiv:2005.00247.
67. Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. 2019. Language models are unsupervised multitask learners. OpenAI blog, 1(8):9. [find](https://scholar.google.com/scholar?q=Alec+Radford%2C+Jeffrey+Wu%2C+Rewon+Child%2C+David+Luan%2C+Dario+Amodei%2C+Ilya+Sutskever%2C+et+al%2E+2019%2E+Language+models+are+unsupervised+multitask+learners%2E+OpenAI+blog%2C+1%288%29%3A9%2E)
68. Vivek Ramanujan, Mitchell Wortsman, Aniruddha Kembhavi, Ali Farhadi, and Mohammad Rastegari. 2020. What’s hidden in a randomly weighted neural network? In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 11893–11902. [find](https://scholar.google.com/scholar?q=Vivek+Ramanujan%2C+Mitchell+Wortsman%2C+Aniruddha+Kembhavi%2C+Ali+Farhadi%2C+and+Mohammad+Rastegari%2E+2020%2E+What%E2%80%99s+hidden+in+a+randomly+weighted+neural+network%3F+In+Proceedings+of+the+IEEE%2FCVF+conference+on+com)
69. Sylvestre-Alvise Rebuffi, Hakan Bilen, and Andrea Vedaldi. 2017. [Learning multiple visual domains with residual adapters](https://md.research.elamur.ai/free/en/papers/rebuffi-2017-learning-multiple-visual-domains-residual/). Advances in neural information processing systems, 30.
70. Pengjie Ren, Chengshun Shi, Shiguang Wu, Mengqi Zhang, Zhaochun Ren, Maarten de Rijke, Zhumin Chen, and Jiahuan Pei. 2024. MELoRA: Mini-ensemble low-rank adapters for parameter-efficient fine-tuning. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1… [find](https://scholar.google.com/scholar?q=Pengjie+Ren%2C+Chengshun+Shi%2C+Shiguang+Wu%2C+Mengqi+Zhang%2C+Zhaochun+Ren%2C+Maarten+de+Rijke%2C+Zhumin+Chen%2C+and+Jiahuan+Pei%2E+2024%2E+MELoRA%3A+Mini%2Densemble+low%2Drank+adapters+for+parameter%2Defficient+fine%2Dtuning%2E+)
71. Subhro Roy and Dan Roth. 2015. [Solving general arithmetic word problems](https://md.research.elamur.ai/free/en/papers/roy-2015-solving-general-arithmetic-word-problems/). In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 1743–1752, Lisbon, Portugal. Association for Computational Linguistics.
72. Andreas Rücklé, Gregor Geigle, Max Glockner, Tilman Beck, Jonas Pfeiffer, Nils Reimers, and Iryna Gurevych. 2020. [Adapterdrop: On the efficiency of adapters in transformers](https://md.research.elamur.ai/free/en/papers/ruckle-2021-adapterdrop-efficiency-adapters-transformers/). arXiv preprint arXiv:2010.11918.
73. Ying Shen, Zhiyang Xu, Qifan Wang, Yu Cheng, Wenpeng Yin, and Lifu Huang. 2024. Multimodal instruction tuning with conditional mixture of LoRA. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 637–648, Bangkok, Thailand. Associ… [find](https://scholar.google.com/scholar?q=Ying+Shen%2C+Zhiyang+Xu%2C+Qifan+Wang%2C+Yu+Cheng%2C+Wenpeng+Yin%2C+and+Lifu+Huang%2E+2024%2E+Multimodal+instruction+tuning+with+conditional+mixture+of+LoRA%2E+In+Proceedings+of+the+62nd+Annual+Meeting+of+the+Associa)
74. Shuhua Shi, Shaohan Huang, Minghui Song, Zhoujun Li, Zihan Zhang, Haizhen Huang, Furu Wei, Weiwei Deng, Feng Sun, and Qi Zhang. 2024. ResLoRA: Identity residual mapping in low-rank adaption. In Findings of the Association for Computational Linguistics: ACL 2024, pages 8870–8884, Bangkok, Thailand. A… [find](https://scholar.google.com/scholar?q=Shuhua+Shi%2C+Shaohan+Huang%2C+Minghui+Song%2C+Zhoujun+Li%2C+Zihan+Zhang%2C+Haizhen+Huang%2C+Furu+Wei%2C+Weiwei+Deng%2C+Feng+Sun%2C+and+Qi+Zhang%2E+2024%2E+ResLoRA%3A+Identity+residual+mapping+in+low%2Drank+adaption%2E+In+Findin)
75. Mingjie Sun, Zhuang Liu, Anna Bair, and J Zico Kolter. 2024. A simple and effective pruning approach for large language models. In The Twelfth International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Mingjie+Sun%2C+Zhuang+Liu%2C+Anna+Bair%2C+and+J+Zico+Kolter%2E+2024%2E+A+simple+and+effective+pruning+approach+for+large+language+models%2E+In+The+Twelfth+International+Conference+on+Learning+Representations%2E)
76. Zexu Sun, Yiju Guo, Yankai Lin, Xu Chen, Qi Qi, Xing Tang, xiuqiang He, and Ji-Rong Wen. 2025. Uncertainty and influence aware reward model refinement for reinforcement learning from human feedback. In The Thirteenth International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Zexu+Sun%2C+Yiju+Guo%2C+Yankai+Lin%2C+Xu+Chen%2C+Qi+Qi%2C+Xing+Tang%2C+xiuqiang+He%2C+and+Ji%2DRong+Wen%2E+2025%2E+Uncertainty+and+influence+aware+reward+model+refinement+for+reinforcement+learning+from+human+feedback%2E+I)
77. Chunlin Tian, Zhan Shi, Zhijiang Guo, Li Li, and Cheng zhong Xu. 2024. HydraloRA: An asymmetric loRA architecture for efficient fine-tuning. In The Thirty-eighth Annual Conference on Neural Information Processing Systems. [find](https://scholar.google.com/scholar?q=Chunlin+Tian%2C+Zhan+Shi%2C+Zhijiang+Guo%2C+Li+Li%2C+and+Cheng+zhong+Xu%2E+2024%2E+HydraloRA%3A+An+asymmetric+loRA+architecture+for+efficient+fine%2Dtuning%2E+In+The+Thirty%2Deighth+Annual+Conference+on+Neural+Informatio)
78. Mojtaba Valipour, Mehdi Rezagholizadeh, Ivan Kobyzev, and Ali Ghodsi. 2022. [Dylora: Parameter efficient tuning of pre-trained models using dynamic search-free low-rank adaptation](https://md.research.elamur.ai/free/en/papers/valipour-2023-dylora-parameter-efficient-tuning-pre/). arXiv preprint arXiv:2210.07558.
79. Alex Wang. 2018. [Glue: A multi-task benchmark and analysis platform for natural language understanding](https://md.research.elamur.ai/free/en/papers/wang-2018-glue-multi-task-benchmark-analysis/). arXiv preprint arXiv:1804.07461.
80. Ruiyu Wang, Yu Yuan, Shizhao Sun, and Jiang Bian. 2025. Text-to-cad generation through infusing visual feedback in large language models. Preprint, arXiv:2501.19054. — [arXiv:2501.19054](https://arxiv.org/abs/2501.19054) [find](https://scholar.google.com/scholar?q=Ruiyu+Wang%2C+Yu+Yuan%2C+Shizhao+Sun%2C+and+Jiang+Bian%2E+2025%2E+Text%2Dto%2Dcad+generation+through+infusing+visual+feedback+in+large+language+models%2E+Preprint%2C+arXiv%3A2501%2E19054%2E+%E2%80%94+%5BarXiv%3A2501%2E19054%5D%28https%3A%2F%2Farxiv)
81. Tiannan Wang, Wangchunshu Zhou, Yan Zeng, and Xinsong Zhang. 2023. EfficientVLM: Fast and accurate vision-language models via knowledge distillation and modal-adaptive pruning. In Findings of the Association for Computational Linguistics: ACL 2023, pages 13899–13913, Toronto, Canada. Association for… [find](https://scholar.google.com/scholar?q=Tiannan+Wang%2C+Wangchunshu+Zhou%2C+Yan+Zeng%2C+and+Xinsong+Zhang%2E+2023%2E+EfficientVLM%3A+Fast+and+accurate+vision%2Dlanguage+models+via+knowledge+distillation+and+modal%2Dadaptive+pruning%2E+In+Findings+of+the+Asso)
82. Yaqing Wang, Subhabrata Mukherjee, Xiaodong Liu, Jing Gao, Ahmed Hassan Awadallah, and Jianfeng Gao. 2022. Adamix: Mixture-of-adapter for parameter-efficient tuning of large language models. arXiv preprint arXiv:2205.12410, 1(2):4. — [arXiv:2205.12410](https://arxiv.org/abs/2205.12410) [find](https://scholar.google.com/scholar?q=Yaqing+Wang%2C+Subhabrata+Mukherjee%2C+Xiaodong+Liu%2C+Jing+Gao%2C+Ahmed+Hassan+Awadallah%2C+and+Jianfeng+Gao%2E+2022%2E+Adamix%3A+Mixture%2Dof%2Dadapter+for+parameter%2Defficient+tuning+of+large+language+models%2E+arXiv+pre)
83. Hui Wu, Yuanben Zhang, Zhonghe Han, Yingyan Hou, Lei Wang, Siye Liu, Qihang Gong, and Yunping Ge. 2025a. Cot-driven framework for short text classification: Enhancing and transferring capabilities from large to smaller model. Knowledge-Based Systems, page 113057. [find](https://scholar.google.com/scholar?q=Hui+Wu%2C+Yuanben+Zhang%2C+Zhonghe+Han%2C+Yingyan+Hou%2C+Lei+Wang%2C+Siye+Liu%2C+Qihang+Gong%2C+and+Yunping+Ge%2E+2025a%2E+Cot%2Ddriven+framework+for+short+text+classification%3A+Enhancing+and+transferring+capabilities+fro)
84. Jie Wu, Haoling Li, Xin Zhang, Jianwen Luo, Yangyu Huang, Ruihang Chu, Yujiu Yang, and Scarlett Li. 2025b. Iterpref: Focal preference learning for code generation via iterative debugging. Preprint, arXiv:2503.02783. — [arXiv:2503.02783](https://arxiv.org/abs/2503.02783) [find](https://scholar.google.com/scholar?q=Jie+Wu%2C+Haoling+Li%2C+Xin+Zhang%2C+Jianwen+Luo%2C+Yangyu+Huang%2C+Ruihang+Chu%2C+Yujiu+Yang%2C+and+Scarlett+Li%2E+2025b%2E+Iterpref%3A+Focal+preference+learning+for+code+generation+via+iterative+debugging%2E+Preprint%2C+ar)
85. Muling Wu, Wenhao Liu, Xiaohua Wang, Tianlong Li, Changze Lv, Zixuan Ling, Jianhao Zhu, Cenyuan Zhang, Xiaoqing Zheng, and Xuanjing Huang. 2024. Advancing parameter efficiency in fine-tuning via representation editing. arXiv preprint arXiv:2402.15179. [find](https://scholar.google.com/scholar?q=Muling+Wu%2C+Wenhao+Liu%2C+Xiaohua+Wang%2C+Tianlong+Li%2C+Changze+Lv%2C+Zixuan+Ling%2C+Jianhao+Zhu%2C+Cenyuan+Zhang%2C+Xiaoqing+Zheng%2C+and+Xuanjing+Huang%2E+2024%2E+Advancing+parameter+efficiency+in+fine%2Dtuning+via+repre)
86. Haotian Xu, Xing Wu, Weinong Wang, Zhongzhi Li, Da Zheng, Boyuan Chen, Yi Hu, Shijia Kang, Jiaming Ji, Yingying Zhang, Zhijiang Guo, Yaodong Yang, Muhan Zhang, and Debing Zhang. 2025. Redstar: Does scaling long-cot data unlock better slow-reasoning systems? Preprint, arXiv:2501.11284. [find](https://scholar.google.com/scholar?q=Haotian+Xu%2C+Xing+Wu%2C+Weinong+Wang%2C+Zhongzhi+Li%2C+Da+Zheng%2C+Boyuan+Chen%2C+Yi+Hu%2C+Shijia+Kang%2C+Jiaming+Ji%2C+Yingying+Zhang%2C+Zhijiang+Guo%2C+Yaodong+Yang%2C+Muhan+Zhang%2C+and+Debing+Zhang%2E+2025%2E+Redstar%3A+Does+sc)
87. Yuhui Xu, Lingxi Xie, Xiaotao Gu, Xin Chen, Heng Chang, Hengheng Zhang, Zhengsu Chen, XI-AOPENG ZHANG, and Qi Tian. 2024. QA-loRA: Quantization-aware low-rank adaptation of large language models. In The Twelfth International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Yuhui+Xu%2C+Lingxi+Xie%2C+Xiaotao+Gu%2C+Xin+Chen%2C+Heng+Chang%2C+Hengheng+Zhang%2C+Zhengsu+Chen%2C+XI%2DAOPENG+ZHANG%2C+and+Qi+Tian%2E+2024%2E+QA%2DloRA%3A+Quantization%2Daware+low%2Drank+adaptation+of+large+language+models%2E+In+T)
88. An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, et al. 2024a. Qwen2. 5 technical report. arXiv preprint arXiv:2412.15115. [find](https://scholar.google.com/scholar?q=An+Yang%2C+Baosong+Yang%2C+Beichen+Zhang%2C+Binyuan+Hui%2C+Bo+Zheng%2C+Bowen+Yu%2C+Chengyuan+Li%2C+Dayiheng+Liu%2C+Fei+Huang%2C+Haoran+Wei%2C+et+al%2E+2024a%2E+Qwen2%2E+5+technical+report%2E+arXiv+preprint+arXiv%3A2412%2E15115%2E)
89. Hao Yang, Qianghua Zhao, and Lei Li. 2024b. Chain-of-thought in large language models: Decoding, projection, and activation. Preprint, arXiv:2412.03944. — [arXiv:2412.03944](https://arxiv.org/abs/2412.03944) [find](https://scholar.google.com/scholar?q=Hao+Yang%2C+Qianghua+Zhao%2C+and+Lei+Li%2E+2024b%2E+Chain%2Dof%2Dthought+in+large+language+models%3A+Decoding%2C+projection%2C+and+activation%2E+Preprint%2C+arXiv%3A2412%2E03944%2E+%E2%80%94+%5BarXiv%3A2412%2E03944%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2412)
90. Lei Yu, Jingcheng Niu, Zining Zhu, and Gerald Penn. 2024. Functional faithfulness in the wild: Circuit discovery with differentiable computation graph pruning. Preprint, arXiv:2407.03779. — [arXiv:2407.03779](https://arxiv.org/abs/2407.03779) [find](https://scholar.google.com/scholar?q=Lei+Yu%2C+Jingcheng+Niu%2C+Zining+Zhu%2C+and+Gerald+Penn%2E+2024%2E+Functional+faithfulness+in+the+wild%3A+Circuit+discovery+with+differentiable+computation+graph+pruning%2E+Preprint%2C+arXiv%3A2407%2E03779%2E+%E2%80%94+%5BarXiv%3A240)
91. Sheheryar Zaidi, Tudor Berariu, Hyunjik Kim, Jorg Bornschein, Claudia Clopath, Yee Whye Teh, and Razvan Pascanu. 2023. When does re-initialization work? In Proceedings on "I Can’t Believe It’s Not Better! - Understanding Deep Learning Through Empirical Falsification" at NeurIPS 2022 Workshops, volum… [find](https://scholar.google.com/scholar?q=Sheheryar+Zaidi%2C+Tudor+Berariu%2C+Hyunjik+Kim%2C+Jorg+Bornschein%2C+Claudia+Clopath%2C+Yee+Whye+Teh%2C+and+Razvan+Pascanu%2E+2023%2E+When+does+re%2Dinitialization+work%3F+In+Proceedings+on+%22I+Can%E2%80%99t+Believe+It%E2%80%99s+Not+Bet)
92. Elad Ben Zaken, Shauli Ravfogel, and Yoav Goldberg. 2021. [Bitfit: Simple parameter-efficient fine-tuning for transformer-based masked language-models](https://md.research.elamur.ai/free/en/papers/ben-zaken-2021-bitfit-simple-parameter-efficient-fine/). arXiv preprint arXiv:2106.10199.
93. Jingfan Zhang, Yi Zhao, Dan Chen, Xing Tian, Huanran Zheng, and Wei Zhu. 2024. MiLoRA: Efficient mixture of low-rank adaptation for large language models fine-tuning. In Findings of the Association for Computational Linguistics: EMNLP 2024, pages 17071–17084, Miami, Florida, USA. Association for Com… [find](https://scholar.google.com/scholar?q=Jingfan+Zhang%2C+Yi+Zhao%2C+Dan+Chen%2C+Xing+Tian%2C+Huanran+Zheng%2C+and+Wei+Zhu%2E+2024%2E+MiLoRA%3A+Efficient+mixture+of+low%2Drank+adaptation+for+large+language+models+fine%2Dtuning%2E+In+Findings+of+the+Association+fo)
94. Qingru Zhang, Minshuo Chen, Alexander Bukharin, Nikos Karampatziakis, Pengcheng He, Yu Cheng, Weizhu Chen, and Tuo Zhao. 2023a. [Adalora: Adaptive budget allocation for parameter-efficient fine-tuning](https://md.research.elamur.ai/free/en/papers/zhang-2023-adalora-adaptive-budget-allocation-parameter/). arXiv preprint arXiv:2303.10512.
95. Qiyuan Zhang, Fuyuan Lyu, Zexu Sun, Lei Wang, Weixu Zhang, Wenyue Hua, Haolun Wu, Zhihan Guo, Yufei Wang, Niklas Muennighoff, Irwin King, Xue Liu, and Chen Ma. 2025. A survey on test-time scaling in large language models: What, how, where, and how well? Preprint, arXiv:2503.24235. [find](https://scholar.google.com/scholar?q=Qiyuan+Zhang%2C+Fuyuan+Lyu%2C+Zexu+Sun%2C+Lei+Wang%2C+Weixu+Zhang%2C+Wenyue+Hua%2C+Haolun+Wu%2C+Zhihan+Guo%2C+Yufei+Wang%2C+Niklas+Muennighoff%2C+Irwin+King%2C+Xue+Liu%2C+and+Chen+Ma%2E+2025%2E+A+survey+on+test%2Dtime+scaling+in+l)
96. Zhen-Ru Zhang, Chuanqi Tan, Haiyang Xu, Chengyu Wang, Jun Huang, and Songfang Huang. 2023b. Towards adaptive prefix tuning for parameter-efficient language model fine-tuning. arXiv preprint arXiv:2305.15212. [find](https://scholar.google.com/scholar?q=Zhen%2DRu+Zhang%2C+Chuanqi+Tan%2C+Haiyang+Xu%2C+Chengyu+Wang%2C+Jun+Huang%2C+and+Songfang+Huang%2E+2023b%2E+Towards+adaptive+prefix+tuning+for+parameter%2Defficient+language+model+fine%2Dtuning%2E+arXiv+preprint+arXiv%3A2305)
97. Tong Zheng, Yan Wen, Huiwen Bao, Junfeng Guo, and Heng Huang. 2025. Asymmetric conflict and synergy in post-training for llm-based multilingual machine translation. arXiv preprint arXiv:2502.11223. [find](https://scholar.google.com/scholar?q=Tong+Zheng%2C+Yan+Wen%2C+Huiwen+Bao%2C+Junfeng+Guo%2C+and+Heng+Huang%2E+2025%2E+Asymmetric+conflict+and+synergy+in+post%2Dtraining+for+llm%2Dbased+multilingual+machine+translation%2E+arXiv+preprint+arXiv%3A2502%2E11223%2E)
98. Wangchunshu Zhou, Ronan Le Bras, and Yejin Choi. 2023. Modular transformers: Compressing transformers into modularized layers for flexible efficient inference. In Findings of the Association for Computational Linguistics: ACL 2023, pages 10452–10465, Toronto, Canada. Association for Computational Li… [find](https://scholar.google.com/scholar?q=Wangchunshu+Zhou%2C+Ronan+Le+Bras%2C+and+Yejin+Choi%2E+2023%2E+Modular+transformers%3A+Compressing+transformers+into+modularized+layers+for+flexible+efficient+inference%2E+In+Findings+of+the+Association+for+Compu)

*Updated 30 October 2026*