---
title: "Why Do More Experts Fail? A Theoretical Analysis of Model"
description: "Merging several fine-tuned expert models into one multi-task model saves storage and compute. 75 references, cited by 2 papers here."
authors: ["Zijing Wang", "Xingle Xu", "YongKang Liu", "Yiqun Zhang", "Peiqin Lin", "Shi Feng", "Daling Wang", "Xiaocui Yang", "Hinrich Schuetze"]
year: 2026
venue: "Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)"
source: "https://aclanthology.org/2026.acl-long.2108/"
doi: "10.18653/v1/2026.acl-long.2108"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/wang-2026-why-do-more-experts-fail/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/wang-2026-why-do-more-experts-fail/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/9/) / Why Do More Experts Fail? A Theoretical Analysis of Model Merging

# Why Do More Experts Fail? A Theoretical Analysis of Model Merging

**Authors:** [Zijing Wang](https://md.research.elamur.ai/free/en/authors/zijing-wang/), [Xingle Xu](https://md.research.elamur.ai/free/en/authors/xingle-xu/), [YongKang Liu](https://md.research.elamur.ai/free/en/authors/yongkang-liu/), [Yiqun Zhang](https://md.research.elamur.ai/free/en/authors/yiqun-zhang/), [Peiqin Lin](https://md.research.elamur.ai/free/en/authors/peiqin-lin/), [Shi Feng](https://md.research.elamur.ai/free/en/authors/shi-feng/), [Daling Wang](https://md.research.elamur.ai/free/en/authors/daling-wang/), [Xiaocui Yang](https://md.research.elamur.ai/free/en/authors/xiaocui-yang/) and [Hinrich Schuetze](https://md.research.elamur.ai/free/en/authors/hinrich-schuetze/) · **Year:** 2026 · **Venue:** [Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)](https://md.research.elamur.ai/free/en/venues/acl/) ([ACL 2026](https://md.research.elamur.ai/free/en/venues/acl/2026/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2026.acl-long.2108/) · [PDF](https://aclanthology.org/2026.acl-long.2108.pdf) · [DOI 10.18653/v1/2026.acl-long.2108](https://doi.org/10.18653/v1/2026.acl-long.2108) · [OpenAlex](https://openalex.org/W4414887331)

**Citations:** 0 citations (OpenAlex, 2026-10-14) · 2 in our library, of the 149,439 papers whose bibliographies we read

## Summary
Merging several fine-tuned expert models into one multi-task model saves storage and compute. However, adding more experts stops helping quickly: state-of-the-art methods such as GENOME level off at around six merged models on most datasets. This paper asks why that ceiling exists. Using high-dimensional geometry, the authors model expert weights as Gaussian and make three arguments. First, the variance of the merged model converges to a fixed floor, which yields an upper bound and an adaptive stopping rule. Second, the Gaussian Width of the low-loss parameter region grows in a concave way, so each extra expert contributes less. Third, Approximate Kinematics Theory shows that beyond a certain level of parameter redundancy, keeping the loss low becomes impossible. Because the effective parameter space turns out to be very sparse, they propose Reparameterized Heavy-Tailed (RHT). RHT subtracts Gaussian noise from the weights and then applies a nonlinear transform, which gives the weight distribution heavier tails and broader coverage. Experiments on 19 benchmarks support the theory, and RHT continues to improve as more experts are added. The work matters because it gives a principled account of when merging stops paying off.

## Key points
- Empirically, leading merging methods saturate after merging about six experts. Figure 1 illustrates this plateau for GENOME on MMLU, MATH, CSQA and MBPP.
- The theory combines a variance-based upper bound on the number of experts, a Gaussian Width argument for diminishing marginal gains, and a redundancy threshold derived from Approximate Kinematics Theory.
- With fully fine-tuned GPT-2 experts on seven GLUE-style tasks (CoLA, MNLI, MRPC, QNLI, QQP, RTE, SST-2), RHT raised average scores as follows:
  - Ties-Merging: 70.0 → 70.3
  - Task Arithmetic: 70.0 → 70.6
  - EMR-Merging: 80.4 → 81.0
- With LoRA experts, RHT beat GENOME-dare, GENOME-ties and GENOME-dare-ties as the expert count grew. Gains were largest on MMLU, MATH and MGSM.
- Analysis of LLaMA3.1-8B-Instruct suggests LoRA refines the model's existing abilities rather than adding new knowledge. The theory covers only experts that share the same base architecture.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Ludwig-Maximilians-Universität München](https://md.research.elamur.ai/free/en/universities/ludwig-maximilians-universitat-munchen/) — University, Munich, Germany · 107 works · 56,478 citations
- [Munich Center for Machine Learning](https://md.research.elamur.ai/free/en/universities/munich-center-for-machine-learning/) — University, Munich, Germany · 57 works · 1,094 citations
- [Northeastern University](https://md.research.elamur.ai/free/en/universities/northeastern-university-cn/) — University, Shenyang, China · 42 works · 1,940 citations

📚 Shelves (5): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works · [🗄️ Information Systems](https://md.research.elamur.ai/free/en/shelves/computer-science/information-systems/) · 1,437 works

## Cite

Cite as: Zijing Wang, Xingle Xu, YongKang Liu, Yiqun Zhang, Peiqin Lin, Shi Feng, Daling Wang, Xiaocui Yang and Hinrich Schuetze. 2026. Why Do More Experts Fail? A Theoretical Analysis of Model Merging. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{wang2026why,
  title = {{Why Do More Experts Fail? A Theoretical Analysis of Model Merging}},
  author = {Zijing Wang, Xingle Xu, YongKang Liu, Yiqun Zhang, Peiqin Lin, Shi Feng, Daling Wang, Xiaocui Yang and Hinrich Schuetze},
  booktitle = {Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2026},
  doi = {10.18653/v1/2026.acl-long.2108},
  url = {https://doi.org/10.18653/v1/2026.acl-long.2108}
}
```

- [← previous Who Watches the Watchmen? Humans Disagree With Translation Metrics on Unseen Domains](https://md.research.elamur.ai/free/en/papers/schmidt-2026-who-watches-watchmen-humans-disagree/)
- [next → XOXO: Stealthy Cross-Origin Context Poisoning Attacks against AI Coding Assistants](https://md.research.elamur.ai/free/en/papers/storek-2026-xoxo-stealthy-cross-origin-context/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/9/)

## References

75 references, 22 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Takuya Akiba, Makoto Shing, Yujin Tang, Qi Sun, and David Ha. 2025. Evolutionary optimization of model merging recipes. Nature Machine Intelligence, pages 1–10. [find](https://scholar.google.com/scholar?q=Takuya+Akiba%2C+Makoto+Shing%2C+Yujin+Tang%2C+Qi+Sun%2C+and+David+Ha%2E+2025%2E+Evolutionary+optimization+of+model+merging+recipes%2E+Nature+Machine+Intelligence%2C+pages+1%E2%80%9310%2E)
2. Dennis Amelunxen, Martin Lotz, Michael B McCoy, and Joel A Tropp. 2014. Living on the edge: Phase transitions in convex programs with random data. Information and Inference: A Journal of the IMA, 3(3):224–294. [find](https://scholar.google.com/scholar?q=Dennis+Amelunxen%2C+Martin+Lotz%2C+Michael+B+McCoy%2C+and+Joel+A+Tropp%2E+2014%2E+Living+on+the+edge%3A+Phase+transitions+in+convex+programs+with+random+data%2E+Information+and+Inference%3A+A+Journal+of+the+IMA%2C+3%283%29)
3. Jacob Austin, Augustus Odena, Maxwell Nye, Maarten Bosma, Henryk Michalewski, David Dohan, Ellen Jiang, Carrie Cai, Michael Terry, Quoc Le, and 1 others. 2021. [Program synthesis with large language models](https://md.research.elamur.ai/free/en/papers/austin-2021-program-synthesis-large-language-models/). arXiv preprint arXiv:2108.07732.
4. Jiho Choi, Donggyun Kim, Chanhyuk Lee, and Seunghoon Hong. 2024. Revisiting weight averaging for model merging. arXiv preprint arXiv:2412.12153. — [arXiv:2412.12153](https://arxiv.org/abs/2412.12153) [find](https://scholar.google.com/scholar?q=Jiho+Choi%2C+Donggyun+Kim%2C+Chanhyuk+Lee%2C+and+Seunghoon+Hong%2E+2024%2E+Revisiting+weight+averaging+for+model+merging%2E+arXiv+preprint+arXiv%3A2412%2E12153%2E+%E2%80%94+%5BarXiv%3A2412%2E12153%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2412%2E12153%29)
5. Zhuojun Ding, Wei Wei, and Chenghao Fan. 2025. Selecting and merging: Towards adaptable and scalable named entity recognition with large language models. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 9869–9886, Vienna, Austr… [find](https://scholar.google.com/scholar?q=Zhuojun+Ding%2C+Wei+Wei%2C+and+Chenghao+Fan%2E+2025%2E+Selecting+and+merging%3A+Towards+adaptable+and+scalable+named+entity+recognition+with+large+language+models%2E+In+Proceedings+of+the+63rd+Annual+Meeting+of+t)
6. William B. Dolan and Chris Brockett. 2005. [Automatically constructing a corpus of sentential paraphrases](https://md.research.elamur.ai/free/en/papers/dolan-2005-automatically-constructing-corpus-sentential-paraphrases/). In Proceedings of the Third International Workshop on Paraphrasing (IWP2005).
7. Yuxin Fang, Wen Wang, Binhui Xie, Quan Sun, Ledell Wu, Xinggang Wang, Tiejun Huang, Xinlong Wang, and Yue Cao. 2023. Eva: Exploring the limits of masked visual representation learning at scale. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 19358– 19369. [find](https://scholar.google.com/scholar?q=Yuxin+Fang%2C+Wen+Wang%2C+Binhui+Xie%2C+Quan+Sun%2C+Ledell+Wu%2C+Xinggang+Wang%2C+Tiejun+Huang%2C+Xinlong+Wang%2C+and+Yue+Cao%2E+2023%2E+Eva%3A+Exploring+the+limits+of+masked+visual+representation+learning+at+scale%2E+In+Pro)
8. Shangbin Feng, Zifeng Wang, Yike Wang, Sayna Ebrahimi, Hamid Palangi, Lesly Miculicich, Achin Kulshrestha, Nathalie Rauschmayr, Yejin Choi, Yulia Tsvetkov, and 1 others. Model swarms: Collaborative search to adapt llm experts via swarm intelligence. In Forty-second International Conference on Machin… [find](https://scholar.google.com/scholar?q=Shangbin+Feng%2C+Zifeng+Wang%2C+Yike+Wang%2C+Sayna+Ebrahimi%2C+Hamid+Palangi%2C+Lesly+Miculicich%2C+Achin+Kulshrestha%2C+Nathalie+Rauschmayr%2C+Yejin+Choi%2C+Yulia+Tsvetkov%2C+and+1+others%2E+Model+swarms%3A+Collaborative+se)
9. Lei Gao, Yue Niu, Tingting Tang, Salman Avestimehr, and Murali Annavaram. 2024. Ethos: Rectifying language models in orthogonal parameter space. In Findings of the Association for Computational Linguistics: NAACL 2024, pages 2054–2068. [find](https://scholar.google.com/scholar?q=Lei+Gao%2C+Yue+Niu%2C+Tingting+Tang%2C+Salman+Avestimehr%2C+and+Murali+Annavaram%2E+2024%2E+Ethos%3A+Rectifying+language+models+in+orthogonal+parameter+space%2E+In+Findings+of+the+Association+for+Computational+Lingui)
10. Danilo Giampiccolo, Bernardo Magnini, Ido Dagan, and Bill Dolan. 2007. [The third PASCAL recognizing textual entailment challenge](https://md.research.elamur.ai/free/en/papers/giampiccolo-2007-third-pascal-recognizing-textual-entailment/). In Proceedings of the ACL-PASCAL Workshop on Textual Entailment and Paraphrasing, pages 1–9, Prague. Association for Computational Linguistics.
11. Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. [Measuring massive multitask language understanding](https://md.research.elamur.ai/free/en/papers/hendrycks-2020-measuring-massive-multitask-language-understanding/). In International Conference on Learning Representations.
12. Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. 2021. [Measuring mathematical problem solving with the math dataset](https://md.research.elamur.ai/free/en/papers/hendrycks-2021-measuring-mathematical-problem-solving-math/). arXiv preprint arXiv:2103.03874.
13. Chengsong Huang, Qian Liu, Bill Yuchen Lin, Tianyu Pang, Chao Du, and Min Lin. 2024a. [Lorahub: Efficient cross-task generalization via dynamic lora composition](https://md.research.elamur.ai/free/en/papers/huang-2023-lorahub-efficient-cross-task-generalization/). Preprint, arXiv:2307.13269.
14. Chenyu Huang, Peng Ye, Tao Chen, Tong He, Xiangyu Yue, and Wanli Ouyang. 2024b. Emr-merging: Tuning-free high-performance model merging. Advances in Neural Information Processing Systems, 37:122741–122769. [find](https://scholar.google.com/scholar?q=Chenyu+Huang%2C+Peng+Ye%2C+Tao+Chen%2C+Tong+He%2C+Xiangyu+Yue%2C+and+Wanli+Ouyang%2E+2024b%2E+Emr%2Dmerging%3A+Tuning%2Dfree+high%2Dperformance+model+merging%2E+Advances+in+Neural+Information+Processing+Systems%2C+37%3A122741%E2%80%9312)
15. Gabriel Ilharco, Marco Tulio Ribeiro, Mitchell Wortsman, Ludwig Schmidt, Hannaneh Hajishirzi, and Ali Farhadi. Editing models with task arithmetic. In The Eleventh International Conference on Learning Representations. Hamish Ivison, Yizhong Wang, Valentina Pyatkin, Nathan Lambert, Matthew Peters, Pr… — [Camels in a Changing Climate: Enhancing LM Adaptation with Tulu 2](https://md.research.elamur.ai/free/en/papers/ivison-2023-camels-changing-climate-enhancing-lm/)
16. Shankar Iyer, Nikhil Dandekar, Kornél Csernai, and 1 others. 2017. First quora dataset release: Question pairs. [find](https://scholar.google.com/scholar?q=Shankar+Iyer%2C+Nikhil+Dandekar%2C+Korn%C3%A9l+Csernai%2C+and+1+others%2E+2017%2E+First+quora+dataset+release%3A+Question+pairs%2E)
17. Dong-Hwan Jang, Sangdoo Yun, and Dongyoon Han. 2024. Model stock: All we need is just a few fine-tuned models. In European Conference on Computer Vision, pages 207–223. Springer. [find](https://scholar.google.com/scholar?q=Dong%2DHwan+Jang%2C+Sangdoo+Yun%2C+and+Dongyoon+Han%2E+2024%2E+Model+stock%3A+All+we+need+is+just+a+few+fine%2Dtuned+models%2E+In+European+Conference+on+Computer+Vision%2C+pages+207%E2%80%93223%2E+Springer%2E)
18. Joel Jang, Seungone Kim, Bill Yuchen Lin, Yizhong Wang, Jack Hessel, Luke Zettlemoyer, Hannaneh Hajishirzi, Yejin Choi, and Prithviraj Ammanabrolu. 2023. Personalized soups: Personalized large language model alignment via post-hoc parameter merging. arXiv preprint arXiv:2310.11564. [find](https://scholar.google.com/scholar?q=Joel+Jang%2C+Seungone+Kim%2C+Bill+Yuchen+Lin%2C+Yizhong+Wang%2C+Jack+Hessel%2C+Luke+Zettlemoyer%2C+Hannaneh+Hajishirzi%2C+Yejin+Choi%2C+and+Prithviraj+Ammanabrolu%2E+2023%2E+Personalized+soups%3A+Personalized+large+languag)
19. Xiang Lisa Li and Percy Liang. 2021. Prefix-tuning: Optimizing continuous prompts for generation. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers). Association… [find](https://scholar.google.com/scholar?q=Xiang+Lisa+Li+and+Percy+Liang%2E+2021%2E+Prefix%2Dtuning%3A+Optimizing+continuous+prompts+for+generation%2E+In+Proceedings+of+the+59th+Annual+Meeting+of+the+Association+for+Computational+Linguistics+and+the+11t)
20. Deyuan Liu, Zecheng Wang, Bingning Wang, Weipeng Chen, Chunshan Li, Zhiying Tu, Dianhui Chu, Bo Li, and Dianbo Sui. 2025a. Checkpoint merging via bayesian optimization in llm pretraining. Preprint, arXiv:2403.19390. — [arXiv:2403.19390](https://arxiv.org/abs/2403.19390) [find](https://scholar.google.com/scholar?q=Deyuan+Liu%2C+Zecheng+Wang%2C+Bingning+Wang%2C+Weipeng+Chen%2C+Chunshan+Li%2C+Zhiying+Tu%2C+Dianhui+Chu%2C+Bo+Li%2C+and+Dianbo+Sui%2E+2025a%2E+Checkpoint+merging+via+bayesian+optimization+in+llm+pretraining%2E+Preprint%2C+ar)
21. Enshu Liu, Junyi Zhu, Zinan Lin, Xuefei Ning, Shuaiqi Wang, Matthew B. Blaschko, Sergey Yekhanin, Shengen Yan, Guohao Dai, Huazhong Yang, and Yu Wang. 2025b. Linear combination of saved checkpoints makes consistency and diffusion models better. In The Thirteenth International Conference on Learning… [find](https://scholar.google.com/scholar?q=Enshu+Liu%2C+Junyi+Zhu%2C+Zinan+Lin%2C+Xuefei+Ning%2C+Shuaiqi+Wang%2C+Matthew+B%2E+Blaschko%2C+Sergey+Yekhanin%2C+Shengen+Yan%2C+Guohao+Dai%2C+Huazhong+Yang%2C+and+Yu+Wang%2E+2025b%2E+Linear+combination+of+saved+checkpoints+ma)
22. Shuqi Liu, Han Wu, Bowei He, Xiongwei Han, Mingxuan Yuan, and Linqi Song. 2025c. Sens-merging: Sensitivity-guided parameter balancing for merging large language models. In The 63rd Annual Meeting of the Association for Computational Linguistics. ACL Anthology. [find](https://scholar.google.com/scholar?q=Shuqi+Liu%2C+Han+Wu%2C+Bowei+He%2C+Xiongwei+Han%2C+Mingxuan+Yuan%2C+and+Linqi+Song%2E+2025c%2E+Sens%2Dmerging%3A+Sensitivity%2Dguided+parameter+balancing+for+merging+large+language+models%2E+In+The+63rd+Annual+Meeting+of+t)
23. Xiao Liu, Yanan Zheng, Zhengxiao Du, Ming Ding, Yujie Qian, Zhilin Yang, and Jie Tang. 2024a. Gpt understands, too. AI Open, 5:208–215. [find](https://scholar.google.com/scholar?q=Xiao+Liu%2C+Yanan+Zheng%2C+Zhengxiao+Du%2C+Ming+Ding%2C+Yujie+Qian%2C+Zhilin+Yang%2C+and+Jie+Tang%2E+2024a%2E+Gpt+understands%2C+too%2E+AI+Open%2C+5%3A208%E2%80%93215%2E)
24. Xisen Jin, Xiang Ren, Daniel Preotiuc-Pietro, and Pengxiang Cheng. 2023. Dataless knowledge fusion by merging weights of language models. In The Eleventh International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Xisen+Jin%2C+Xiang+Ren%2C+Daniel+Preotiuc%2DPietro%2C+and+Pengxiang+Cheng%2E+2023%2E+Dataless+knowledge+fusion+by+merging+weights+of+language+models%2E+In+The+Eleventh+International+Conference+on+Learning+Represent)
25. Yongkang Liu, Xing Li, Mengjie Zhao, Shanru Zhang, Zijing Wang, Qian Li, Shi Feng, Feiliang Ren, Daling Wang, and Hinrich Schütze. 2026. High-rank structured modulation for parameter-efficient fine-tuning. arXiv preprint arXiv:2601.07507. — [arXiv:2601.07507](https://arxiv.org/abs/2601.07507) [find](https://scholar.google.com/scholar?q=Yongkang+Liu%2C+Xing+Li%2C+Mengjie+Zhao%2C+Shanru+Zhang%2C+Zijing+Wang%2C+Qian+Li%2C+Shi+Feng%2C+Feiliang+Ren%2C+Daling+Wang%2C+and+Hinrich+Sch%C3%BCtze%2E+2026%2E+High%2Drank+structured+modulation+for+parameter%2Defficient+fine%2Dtu)
26. Virginia Klema and Alan Laub. 1980. The singular value decomposition: Its computation and some applications. IEEE Transactions on automatic control, 25(2):164–176. [find](https://scholar.google.com/scholar?q=Virginia+Klema+and+Alan+Laub%2E+1980%2E+The+singular+value+decomposition%3A+Its+computation+and+some+applications%2E+IEEE+Transactions+on+automatic+control%2C+25%282%29%3A164%E2%80%93176%2E)
27. Yongkang Liu, Xingle Xu, Ercong Nie, Zijing Wang, Shi Feng, Daling Wang, Qian Li, and Hinrich Schütze. 2025d. Look within or look beyond? a theoretical comparison between parameter-efficient and full fine-tuning. arXiv preprint arXiv:2505.22355. — [arXiv:2505.22355](https://arxiv.org/abs/2505.22355) [find](https://scholar.google.com/scholar?q=Yongkang+Liu%2C+Xingle+Xu%2C+Ercong+Nie%2C+Zijing+Wang%2C+Shi+Feng%2C+Daling+Wang%2C+Qian+Li%2C+and+Hinrich+Sch%C3%BCtze%2E+2025d%2E+Look+within+or+look+beyond%3F+a+theoretical+comparison+between+parameter%2Defficient+and+full+)
28. Brett W Larsen, Stanislav Fort, Nic Becker, and Surya Ganguli. How many degrees of freedom do we need to train deep networks: a loss landscape perspective. In International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Brett+W+Larsen%2C+Stanislav+Fort%2C+Nic+Becker%2C+and+Surya+Ganguli%2E+How+many+degrees+of+freedom+do+we+need+to+train+deep+networks%3A+a+loss+landscape+perspective%2E+In+International+Conference+on+Learning+Repr)
29. Sanwoo Lee, Jiahao Liu, Qifan Wang, Jingang Wang, Xunliang Cai, and Yunfang Wu. 2025a. Dynamic fisher-weighted model merging via bayesian optimization. In Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Techno… [find](https://scholar.google.com/scholar?q=Sanwoo+Lee%2C+Jiahao+Liu%2C+Qifan+Wang%2C+Jingang+Wang%2C+Xunliang+Cai%2C+and+Yunfang+Wu%2E+2025a%2E+Dynamic+fisher%2Dweighted+model+merging+via+bayesian+optimization%2E+In+Proceedings+of+the+2025+Conference+of+the+Nat)
30. Yu-Ang Lee, Ching-Yun Ko, Tejaswini Pedapati, I-Hsin Chung, Mi-Yen Yeh, and Pin-Yu Chen. 2025b. Star: Spectral truncation and rescale for model merging. In Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Techn… [find](https://scholar.google.com/scholar?q=Yu%2DAng+Lee%2C+Ching%2DYun+Ko%2C+Tejaswini+Pedapati%2C+I%2DHsin+Chung%2C+Mi%2DYen+Yeh%2C+and+Pin%2DYu+Chen%2E+2025b%2E+Star%3A+Spectral+truncation+and+rescale+for+model+merging%2E+In+Proceedings+of+the+2025+Conference+of+the+Na)
31. Yongkang Liu, Yiqun Zhang, Qian Li, Tong Liu, Shi Feng, Daling Wang, Yifei Zhang, and Hinrich Schütze. 2024b. Hift: A hierarchical full parameter fine-tuning strategy. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 18266–18287. [find](https://scholar.google.com/scholar?q=Yongkang+Liu%2C+Yiqun+Zhang%2C+Qian+Li%2C+Tong+Liu%2C+Shi+Feng%2C+Daling+Wang%2C+Yifei+Zhang%2C+and+Hinrich+Sch%C3%BCtze%2E+2024b%2E+Hift%3A+A+hierarchical+full+parameter+fine%2Dtuning+strategy%2E+In+Proceedings+of+the+2024+Confe)
32. Jinliang Lu, Ziliang Pang, Min Xiao, Yaochen Zhu, Rui Xia, and Jiajun Zhang. 2024. Merge, ensemble, and cooperate! a survey on collaborative strategies in the era of large language models. Preprint, arXiv:2407.06089. — [arXiv:2407.06089](https://arxiv.org/abs/2407.06089) [find](https://scholar.google.com/scholar?q=Jinliang+Lu%2C+Ziliang+Pang%2C+Min+Xiao%2C+Yaochen+Zhu%2C+Rui+Xia%2C+and+Jiajun+Zhang%2E+2024%2E+Merge%2C+ensemble%2C+and+cooperate%21+a+survey+on+collaborative+strategies+in+the+era+of+large+language+models%2E+Preprint%2C+a)
33. Tongxu Luo, Jiahe Lei, Fangyu Lei, Weihao Liu, Shizhu He, Jun Zhao, and Kang Liu. 2024. Moelora: Contrastive learning guided mixture of experts on parameter-efficient fine-tuning for large language models. arXiv preprint arXiv:2402.12851. — [arXiv:2402.12851](https://arxiv.org/abs/2402.12851) [find](https://scholar.google.com/scholar?q=Tongxu+Luo%2C+Jiahe+Lei%2C+Fangyu+Lei%2C+Weihao+Liu%2C+Shizhu+He%2C+Jun+Zhao%2C+and+Kang+Liu%2E+2024%2E+Moelora%3A+Contrastive+learning+guided+mixture+of+experts+on+parameter%2Defficient+fine%2Dtuning+for+large+language+mo)
34. Yingfeng Luo, Dingyang Lin, Junxin Wang, Ziqiang Xu, Kaiyan Chang, Tong Zheng, Bei Li, Anxiang Ma, Tong Xiao, Zhengtao Yu, and 1 others. 2025. One size does not fit all: A distribution-aware sparsification for more precise model merging. arXiv preprint arXiv:2508.06163. — [arXiv:2508.06163](https://arxiv.org/abs/2508.06163) [find](https://scholar.google.com/scholar?q=Yingfeng+Luo%2C+Dingyang+Lin%2C+Junxin+Wang%2C+Ziqiang+Xu%2C+Kaiyan+Chang%2C+Tong+Zheng%2C+Bei+Li%2C+Anxiang+Ma%2C+Tong+Xiao%2C+Zhengtao+Yu%2C+and+1+others%2E+2025%2E+One+size+does+not+fit+all%3A+A+distribution%2Daware+sparsific)
35. Daniel Marczak, Simone Magistri, Sebastian Cygert, Bartłomiej Twardowski, Andrew D Bagdanov, and Joost van de Weijer. No task left behind: Isotropic model merging with common and task-specific subspaces. In Forty-second International Conference on Machine Learning. [find](https://scholar.google.com/scholar?q=Daniel+Marczak%2C+Simone+Magistri%2C+Sebastian+Cygert%2C+Bart%C5%82omiej+Twardowski%2C+Andrew+D+Bagdanov%2C+and+Joost+van+de+Weijer%2E+No+task+left+behind%3A+Isotropic+model+merging+with+common+and+task%2Dspecific+subspac)
36. Michael S Matena and Colin A Raffel. 2022. [Merging models with fisher-weighted averaging](https://md.research.elamur.ai/free/en/papers/matena-2021-merging-models-fisher-weighted-averaging/). Advances in Neural Information Processing Systems, 35:17703– 17716.
37. Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. [Bleu: a method for automatic evaluation of machine translation](https://md.research.elamur.ai/free/en/papers/papineni-2002-bleu-method-automatic-evaluation-machine/). In Proceedings of the 40th Annual Meeting on Association for Computational Linguistics, ACL ’02, page 311–318, USA. Association for Computational Linguistics.
38. Ryan Po, Guandao Yang, Kfir Aberman, and Gordon Wetzstein. 2024. Orthogonal adaptation for modular customization of diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 7964–7973. [find](https://scholar.google.com/scholar?q=Ryan+Po%2C+Guandao+Yang%2C+Kfir+Aberman%2C+and+Gordon+Wetzstein%2E+2024%2E+Orthogonal+adaptation+for+modular+customization+of+diffusion+models%2E+In+Proceedings+of+the+IEEE%2FCVF+Conference+on+Computer+Vision+and+P)
39. Akshara Prabhakar, Yuanzhi Li, Karthik Narasimhan, Sham Kakade, Eran Malach, and Samy Jelassi. 2025. Lora soups: Merging loras for practical skill composition tasks. In Proceedings of the 31st International Conference on Computational Linguistics: Industry Track, pages 644–655. [find](https://scholar.google.com/scholar?q=Akshara+Prabhakar%2C+Yuanzhi+Li%2C+Karthik+Narasimhan%2C+Sham+Kakade%2C+Eran+Malach%2C+and+Samy+Jelassi%2E+2025%2E+Lora+soups%3A+Merging+loras+for+practical+skill+composition+tasks%2E+In+Proceedings+of+the+31st+Interna)
40. Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, and 1 others. 2019. Language models are unsupervised multitask learners. OpenAI blog, 1(8):9. [find](https://scholar.google.com/scholar?q=Alec+Radford%2C+Jeffrey+Wu%2C+Rewon+Child%2C+David+Luan%2C+Dario+Amodei%2C+Ilya+Sutskever%2C+and+1+others%2E+2019%2E+Language+models+are+unsupervised+multitask+learners%2E+OpenAI+blog%2C+1%288%29%3A9%2E)
41. Pranav Rajpurkar, Jian Zhang, Konstantin Lopyrev, and Percy Liang. 2016. [Squad: 100,000+ questions for machine comprehension of text](https://md.research.elamur.ai/free/en/papers/rajpurkar-2016-squad-100-000-questions-machine/). In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics.
42. Alexandre Rame, Matthieu Kirchmeyer, Thibaud Rahier, Alain Rakotomamonjy, Patrick Gallinari, and Matthieu Cord. 2022. Diverse weight averaging for out-of-distribution generalization. Advances in Neural Information Processing Systems, 35:10821– 10836. [find](https://scholar.google.com/scholar?q=Alexandre+Rame%2C+Matthieu+Kirchmeyer%2C+Thibaud+Rahier%2C+Alain+Rakotomamonjy%2C+Patrick+Gallinari%2C+and+Matthieu+Cord%2E+2022%2E+Diverse+weight+averaging+for+out%2Dof%2Ddistribution+generalization%2E+Advances+in+Neura)
43. Viraj Shah, Nataniel Ruiz, Forrester Cole, Erika Lu, Svetlana Lazebnik, Yuanzhen Li, and Varun Jampani. 2024. Ziplora: Any subject in any style by effectively merging loras. In European Conference on Computer Vision, pages 422–438. Springer. [find](https://scholar.google.com/scholar?q=Viraj+Shah%2C+Nataniel+Ruiz%2C+Forrester+Cole%2C+Erika+Lu%2C+Svetlana+Lazebnik%2C+Yuanzhen+Li%2C+and+Varun+Jampani%2E+2024%2E+Ziplora%3A+Any+subject+in+any+style+by+effectively+merging+loras%2E+In+European+Conference+on+)
44. Freda Shi, Mirac Suzgun, Markus Freitag, Xuezhi Wang, Suraj Srivats, Soroush Vosoughi, Hyung Won Chung, Yi Tay, Sebastian Ruder, Denny Zhou, and 1 others. [Language models are multilingual chain-of-thought reasoners](https://md.research.elamur.ai/free/en/papers/shi-2022-language-models-multilingual-chain-thought/). In The Eleventh International Conference on Learning Representations.
45. Chongjie Si, Jingjing Jiang, and Wei Shen. 2025a. Unveiling the mystery of weight in large foundation models: Gaussian distribution never fades. Preprint, arXiv:2501.10661. — [arXiv:2501.10661](https://arxiv.org/abs/2501.10661) [find](https://scholar.google.com/scholar?q=Chongjie+Si%2C+Jingjing+Jiang%2C+and+Wei+Shen%2E+2025a%2E+Unveiling+the+mystery+of+weight+in+large+foundation+models%3A+Gaussian+distribution+never+fades%2E+Preprint%2C+arXiv%3A2501%2E10661%2E+%E2%80%94+%5BarXiv%3A2501%2E10661%5D%28https%3A)
46. Chongjie Si, Kangtao Lv, Jingjing Jiang, Yadao Wang, Yongwei Wang, Xiaokang Yang, Wenbo Su, Bo Zheng, and Wei Shen. 2025b. Nan: A training-free solution to coefficient estimation in model merging. Preprint, arXiv:2505.16148. — [arXiv:2505.16148](https://arxiv.org/abs/2505.16148) [find](https://scholar.google.com/scholar?q=Chongjie+Si%2C+Kangtao+Lv%2C+Jingjing+Jiang%2C+Yadao+Wang%2C+Yongwei+Wang%2C+Xiaokang+Yang%2C+Wenbo+Su%2C+Bo+Zheng%2C+and+Wei+Shen%2E+2025b%2E+Nan%3A+A+training%2Dfree+solution+to+coefficient+estimation+in+model+merging%2E+Pre)
47. Richard Socher, Alex Perelygin, Jean Wu, Jason Chuang, Christopher D. Manning, Andrew Ng, and Christopher Potts. 2013. [Recursive deep models for semantic compositionality over a sentiment treebank](https://md.research.elamur.ai/free/en/papers/socher-2013-recursive-deep-models-semantic-compositionality/). In Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, pages 1631–…
48. George Stoica, Pratik Ramesh, Boglarka Ecsedi, Leshem Choshen, and Judy Hoffman. Model merging with svd to tie the knots. In The Thirteenth International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=George+Stoica%2C+Pratik+Ramesh%2C+Boglarka+Ecsedi%2C+Leshem+Choshen%2C+and+Judy+Hoffman%2E+Model+merging+with+svd+to+tie+the+knots%2E+In+The+Thirteenth+International+Conference+on+Learning+Representations%2E)
49. Alon Talmor, Jonathan Herzig, Nicholas Lourie, and Jonathan Berant. 2019. [Commonsenseqa: A question answering challenge targeting commonsense knowledge](https://md.research.elamur.ai/free/en/papers/talmor-2019-commonsenseqa-question-answering-challenge-targeting/). In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies,…
50. Mingxu Tao, Chen Zhang, Quzhe Huang, Tianyao Ma, Songfang Huang, Dongyan Zhao, and Yansong Feng. 2024. Unlocking the potential of model merging for low-resource languages. In Findings of the Association for Computational Linguistics: EMNLP 2024, pages 8705–8720. [find](https://scholar.google.com/scholar?q=Mingxu+Tao%2C+Chen+Zhang%2C+Quzhe+Huang%2C+Tianyao+Ma%2C+Songfang+Huang%2C+Dongyan+Zhao%2C+and+Yansong+Feng%2E+2024%2E+Unlocking+the+potential+of+model+merging+for+low%2Dresource+languages%2E+In+Findings+of+the+Associati)
51. Gemma Team, Morgane Riviere, Shreya Pathak, Pier Giuseppe Sessa, Cassidy Hardin, Surya Bhupatiraju, Léonard Hussenot, Thomas Mesnard, Bobak Shahriari, Alexandre Ramé, and 1 others. 2024. Gemma 2: Improving open language models at a practical size. arXiv preprint arXiv:2408.00118. [find](https://scholar.google.com/scholar?q=Gemma+Team%2C+Morgane+Riviere%2C+Shreya+Pathak%2C+Pier+Giuseppe+Sessa%2C+Cassidy+Hardin%2C+Surya+Bhupatiraju%2C+L%C3%A9onard+Hussenot%2C+Thomas+Mesnard%2C+Bobak+Shahriari%2C+Alexandre+Ram%C3%A9%2C+and+1+others%2E+2024%2E+Gemma+2%3A+Impr)
52. Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, and 1 others. 2023. [Llama: Open and efficient foundation language models](https://md.research.elamur.ai/free/en/papers/touvron-2023-llama-open-efficient-foundation-language/). arXiv preprint arXiv:2302.13971.
53. Roman Vershynin. 2015. Estimation in high dimensions: a geometric perspective. In Sampling Theory, a Renaissance: Compressive Sensing and Other Developments, pages 3–66. Springer. [find](https://scholar.google.com/scholar?q=Roman+Vershynin%2E+2015%2E+Estimation+in+high+dimensions%3A+a+geometric+perspective%2E+In+Sampling+Theory%2C+a+Renaissance%3A+Compressive+Sensing+and+Other+Developments%2C+pages+3%E2%80%9366%2E+Springer%2E)
54. Alex Wang, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel Bowman. 2018. [Glue: A multi-task benchmark and analysis platform for natural language understanding](https://md.research.elamur.ai/free/en/papers/wang-2018-glue-multi-task-benchmark-analysis/). In Proceedings of the 2018 EMNLP Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP, page 353. Asso…
55. Hanqing Wang, Bowen Ping, Shuo Wang, Xu Han, Yun Chen, Zhiyuan Liu, and Maosong Sun. 2024. Lora-flow: Dynamic lora fusion for large language models in generative tasks. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 12871– 12… [find](https://scholar.google.com/scholar?q=Hanqing+Wang%2C+Bowen+Ping%2C+Shuo+Wang%2C+Xu+Han%2C+Yun+Chen%2C+Zhiyuan+Liu%2C+and+Maosong+Sun%2E+2024%2E+Lora%2Dflow%3A+Dynamic+lora+fusion+for+large+language+models+in+generative+tasks%2E+In+Proceedings+of+the+62nd+Annu)
56. Zijing Wang, Yongkang Liu, Yingfeng Luo, Ming Wang, Zhen Song, Shi Feng, Xiaocui Yang, Dingyang Lin, Daling Wang, Yifei Zhang, and 1 others. 2025. Scaling intelligence through model merging: A comprehensive survey. Authorea Preprints. [find](https://scholar.google.com/scholar?q=Zijing+Wang%2C+Yongkang+Liu%2C+Yingfeng+Luo%2C+Ming+Wang%2C+Zhen+Song%2C+Shi+Feng%2C+Xiaocui+Yang%2C+Dingyang+Lin%2C+Daling+Wang%2C+Yifei+Zhang%2C+and+1+others%2E+2025%2E+Scaling+intelligence+through+model+merging%3A+A+compreh)
57. Zijing Wang, Yongkang Liu, Mingyang Wang, Ercong Nie, Deyuan Chen, Zhengjie Zhao, Shi Feng, Daling Wang, Xiaocui Yang, Yifei Zhang, and 1 others. 2026. [Plam: Training-free plateau-guided model merging for better visual grounding in mllms](https://md.research.elamur.ai/free/en/papers/wang-2026-plam-training-free-plateau-guided/). arXiv preprint arXiv:2601.07645.
58. Alex Warstadt, Amanpreet Singh, and Samuel R. Bowman. 2019. Neural network acceptability judgments. Transactions of the Association for Computational Linguistics, 7:625–641. [find](https://scholar.google.com/scholar?q=Alex+Warstadt%2C+Amanpreet+Singh%2C+and+Samuel+R%2E+Bowman%2E+2019%2E+Neural+network+acceptability+judgments%2E+Transactions+of+the+Association+for+Computational+Linguistics%2C+7%3A625%E2%80%93641%2E)
59. Adina Williams, Nikita Nangia, and Samuel Bowman. 2018. [A broad-coverage challenge corpus for sentence understanding through inference](https://md.research.elamur.ai/free/en/papers/williams-2018-broad-coverage-challenge-corpus-sentence/). In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Pa…
60. Svante Wold, Kim Esbensen, and Paul Geladi. 1987. Principal component analysis. Chemometrics and intelligent laboratory systems, 2(1-3):37–52. [find](https://scholar.google.com/scholar?q=Svante+Wold%2C+Kim+Esbensen%2C+and+Paul+Geladi%2E+1987%2E+Principal+component+analysis%2E+Chemometrics+and+intelligent+laboratory+systems%2C+2%281%2D3%29%3A37%E2%80%9352%2E)
61. Mitchell Wortsman, Gabriel Ilharco, Samir Ya Gadre, Rebecca Roelofs, Raphael Gontijo-Lopes, Ari S Morcos, Hongseok Namkoong, Ali Farhadi, Yair Carmon, Simon Kornblith, and 1 others. 2022. [Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference time](https://md.research.elamur.ai/free/en/papers/wortsman-2022-model-soups-averaging-weights-multiple/)…
62. Xun Wu, Shaohan Huang, and Furu Wei. Mixture of lora experts. In The Twelfth International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Xun+Wu%2C+Shaohan+Huang%2C+and+Furu+Wei%2E+Mixture+of+lora+experts%2E+In+The+Twelfth+International+Conference+on+Learning+Representations%2E)
63. Tsendsuren Munkhdalai. 2024. for model merging at scale? arXiv:2410.03617. [find](https://scholar.google.com/scholar?q=Tsendsuren+Munkhdalai%2E+2024%2E+for+model+merging+at+scale%3F+arXiv%3A2410%2E03617%2E)
64. What matters arXiv preprint Enneng Yang, Li Shen, Guibing Guo, Xingwei Wang, Xiaochun Cao, Jie Zhang, and Dacheng Tao. 2024. Model merging in llms, mllms, and beyond: Methods, theories, applications and opportunities. Preprint, arXiv:2408.07666. [find](https://scholar.google.com/scholar?q=What+matters+arXiv+preprint+Enneng+Yang%2C+Li+Shen%2C+Guibing+Guo%2C+Xingwei+Wang%2C+Xiaochun+Cao%2C+Jie+Zhang%2C+and+Dacheng+Tao%2E+2024%2E+Model+merging+in+llms%2C+mllms%2C+and+beyond%3A+Methods%2C+theories%2C+applications+a)
65. Enneng Yang, Zhenyi Wang, Li Shen, Shiwei Liu, Guibing Guo, Xingwei Wang, and Dacheng Tao. Adamerging: Adaptive model merging for multi-task learning. In The Twelfth International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Enneng+Yang%2C+Zhenyi+Wang%2C+Li+Shen%2C+Shiwei+Liu%2C+Guibing+Guo%2C+Xingwei+Wang%2C+and+Dacheng+Tao%2E+Adamerging%3A+Adaptive+model+merging+for+multi%2Dtask+learning%2E+In+The+Twelfth+International+Conference+on+Learni)
66. Le Yu, Bowen Yu, Haiyang Yu, Fei Huang, and Yongbin Li. 2024. [Language models are super mario: Absorbing abilities from homologous models as a free lunch](https://md.research.elamur.ai/free/en/papers/yu-2023-language-models-super-mario-absorbing/). In Forty-first International Conference on Machine Learning.
67. Sayyed M Zahiri and Jinho D Choi. 2018. Emotion detection on tv show transcripts with sequence-based convolutional neural networks. In AAAI Workshops, volume 18, pages 44–52. [find](https://scholar.google.com/scholar?q=Sayyed+M+Zahiri+and+Jinho+D+Choi%2E+2018%2E+Emotion+detection+on+tv+show+transcripts+with+sequence%2Dbased+convolutional+neural+networks%2E+In+AAAI+Workshops%2C+volume+18%2C+pages+44%E2%80%9352%2E)
68. Fanhu Zeng, Haiyang Guo, Fei Zhu, Li Shen, and Hao Tang. 2025. Parameter efficient merging for multimodal large language models with complementary parameter adaptation. Preprint, arXiv:2502.17159. — [arXiv:2502.17159](https://arxiv.org/abs/2502.17159) [find](https://scholar.google.com/scholar?q=Fanhu+Zeng%2C+Haiyang+Guo%2C+Fei+Zhu%2C+Li+Shen%2C+and+Hao+Tang%2E+2025%2E+Parameter+efficient+merging+for+multimodal+large+language+models+with+complementary+parameter+adaptation%2E+Preprint%2C+arXiv%3A2502%2E17159%2E+%E2%80%94+%5B)
69. Haobo Zhang and Jiayu Zhou. 2025. Unraveling lora interference: Orthogonal subspaces for robust model merging. Preprint, arXiv:2505.22934. [find](https://scholar.google.com/scholar?q=Haobo+Zhang+and+Jiayu+Zhou%2E+2025%2E+Unraveling+lora+interference%3A+Orthogonal+subspaces+for+robust+model+merging%2E+Preprint%2C+arXiv%3A2505%2E22934%2E)
70. Jinghan Zhang, Junteng Liu, Junxian He, and 1 others. 2023. Composing parameter-efficient modules with arithmetic operation. Advances in Neural Information Processing Systems, 36:12589–12610. [find](https://scholar.google.com/scholar?q=Jinghan+Zhang%2C+Junteng+Liu%2C+Junxian+He%2C+and+1+others%2E+2023%2E+Composing+parameter%2Defficient+modules+with+arithmetic+operation%2E+Advances+in+Neural+Information+Processing+Systems%2C+36%3A12589%E2%80%9312610%2E)
71. Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q Weinberger, and Yoav Artzi. Bertscore: Evaluating text generation with bert. In International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Tianyi+Zhang%2C+Varsha+Kishore%2C+Felix+Wu%2C+Kilian+Q+Weinberger%2C+and+Yoav+Artzi%2E+Bertscore%3A+Evaluating+text+generation+with+bert%2E+In+International+Conference+on+Learning+Representations%2E)
72. Yiqun Zhang, Peng Ye, Xiaocui Yang, Shi Feng, Shufei Zhang, Lei Bai, Wanli Ouyang, and Shuyue Hu. 2025. [Nature-inspired population-based evolution of large language models](https://md.research.elamur.ai/free/en/papers/zhang-2026-nature-inspired-population-based-evolution/). arXiv preprint arXiv:2503.01155.
73. Yaowei Zheng, Richong Zhang, Junhao Zhang, YeYanhan YeYanhan, and Zheyan Luo. 2024. [Llamafactory: Unified efficient fine-tuning of 100+ language models](https://md.research.elamur.ai/free/en/papers/zheng-2024-llamafactory-unified-efficient-fine-tuning/). In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 3: System Demonstrations), pages 400–410.
74. Prateek Yadav, Derek Tam, Leshem Choshen, Colin A Raffel, and Mohit Bansal. 2023. [Ties-merging: Resolving interference when merging models](https://md.research.elamur.ai/free/en/papers/yadav-2023-ties-merging-resolving-interference-merging/). Advances in Neural Information Processing Systems, 36:7093–7115.
75. Prateek Yadav, Tu Vu, Jonathan Lai, Alexandra Chronopoulou, Manaal Faruqui, Mohit Bansal, and [find](https://scholar.google.com/scholar?q=Prateek+Yadav%2C+Tu+Vu%2C+Jonathan+Lai%2C+Alexandra+Chronopoulou%2C+Manaal+Faruqui%2C+Mohit+Bansal%2C+and)

*Updated 31 October 2026*