---
title: "On the Editability of Delta Parameters in Post-Trained Models"
description: "This offers guidance for compression, merging and edit-friendly post-training. 55 references."
authors: ["Qiaoyu Tang", "Le Yu", "Bowen Yu", "Hongyu Lin", "Keming Lu", "Yaojie Lu", "Xianpei Han", "Le Sun"]
year: 2026
venue: "Findings of the Association for Computational Linguistics: ACL 2026"
source: "https://aclanthology.org/2026.findings-acl.1833/"
doi: "10.18653/v1/2026.findings-acl.1833"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/tang-2026-editability-delta-parameters-post-trained/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/tang-2026-editability-delta-parameters-post-trained/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/38/) / On the Editability of Delta Parameters in Post-Trained Models

# On the Editability of Delta Parameters in Post-Trained Models

**Authors:** [Qiaoyu Tang](https://md.research.elamur.ai/free/en/authors/qiaoyu-tang/), [Le Yu](https://md.research.elamur.ai/free/en/authors/le-yu/), [Bowen Yu](https://md.research.elamur.ai/free/en/authors/bowen-yu/), [Hongyu Lin](https://md.research.elamur.ai/free/en/authors/hongyu-lin/), [Keming Lu](https://md.research.elamur.ai/free/en/authors/keming-lu/), [Yaojie Lu](https://md.research.elamur.ai/free/en/authors/yaojie-lu/), [Xianpei Han](https://md.research.elamur.ai/free/en/authors/xianpei-han/) and [Le Sun](https://md.research.elamur.ai/free/en/authors/le-sun/) · **Year:** 2026 · **Venue:** [Findings of the Association for Computational Linguistics: ACL 2026](https://md.research.elamur.ai/free/en/venues/findings/) ([Findings of the Association for Computational Linguistics 2026](https://md.research.elamur.ai/free/en/venues/findings/2026/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2026.findings-acl.1833/) · [PDF](https://aclanthology.org/2026.findings-acl.1833.pdf) · [DOI 10.18653/v1/2026.findings-acl.1833](https://doi.org/10.18653/v1/2026.findings-acl.1833) · [OpenAlex](https://openalex.org/W7166862872)

**Citations:** 0 citations (OpenAlex, 2026-10-14)

## Summary
The paper asks which properties of delta parameters (post-trained weights minus pre-trained weights) must be kept intact for a model to retain the benefits of post-training. Earlier methods such as DARE, BitDelta and EXPO each changed deltas for their own purposes. The authors instead study editability systematically along two axes: magnitude and sign.

They test instruct LLMs (LLaMA-3-8B-Instruct, Mistral-7B-Instruct-v0.3, Qwen2-7B-Instruct), a reasoning model (Qwen3-1.7B) and a vision model (ViT-B-32). These cover SFT, RLHF and RLVR post-training, evaluated on language, math, code and image-classification benchmarks. They generalize DARE so that a random subset of deltas is multiplied by any coefficient k instead of zero. They also vary distribution shapes and flip signs.

Performance survives large edits. Individual values and distribution shape matter little, while the mean magnitude is the most sensitive factor. Even many signs can be flipped. A second-order Taylor analysis of the loss yields an "editing intensity" measure that explains when edits break models. This offers guidance for compression, merging and edit-friendly post-training.

## Key points
- Generalized DARE: a random fraction p of deltas is scaled by k, and the remaining deltas are rescaled by (1 − kp)/(1 − p) so the expected output is preserved. For small p, performance stays close to the original across many k values.
- Benchmarks include:
  - **Instruct LLMs:** ARC Challenge, GSM8K, HellaSwag, HumanEval, IFEval, MMLU, TruthfulQA and Winogrande.
  - **Reasoning model:** MATH-500, AIME 2025, GPQA Diamond and LiveCodeBench.
  - **Vision model:** eight classification sets, including Cars, DTD, EuroSAT and SVHN.
- Replacing magnitudes with uniform, normal or degenerate distributions that have the same mean barely hurts performance. Changing the mean does hurt it.
- At the same drop rate, sign flipping produces a much higher editing intensity than magnitude-only edits. On LLaMA-3-8B-Instruct/GSM8K, intensity correlates negatively with accuracy, and at I ≤ 2 there is almost no degradation.
- Code is released at github.com/icip-cas/DeltaParameterEdit. The authors describe the Taylor-based analysis as qualitative rather than a precise predictor.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

📚 Shelves (5): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works · [👁️ Computer Vision and Pattern Recognition](https://md.research.elamur.ai/free/en/shelves/computer-science/computer-vision-and-pattern-recognition/) · 2,869 works

## Cite

Cite as: Qiaoyu Tang, Le Yu, Bowen Yu, Hongyu Lin, Keming Lu, Yaojie Lu, Xianpei Han and Le Sun. 2026. On the Editability of Delta Parameters in Post-Trained Models. Findings of the Association for Computational Linguistics: ACL 2026.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{tang2026editability,
  title = {{On the Editability of Delta Parameters in Post-Trained Models}},
  author = {Qiaoyu Tang, Le Yu, Bowen Yu, Hongyu Lin, Keming Lu, Yaojie Lu, Xianpei Han and Le Sun},
  booktitle = {Findings of the Association for Computational Linguistics: ACL 2026},
  year = {2026},
  doi = {10.18653/v1/2026.findings-acl.1833},
  url = {https://doi.org/10.18653/v1/2026.findings-acl.1833}
}
```

- [← previous On the Cultural Anachronism and Temporal Reasoning in Vision Language Models](https://md.research.elamur.ai/free/en/papers/ranjan-2026-cultural-anachronism-temporal-reasoning-vision/)
- [next → On the Effect of Hyperparameters in Language Modeling for Computational Linguistics](https://md.research.elamur.ai/free/en/papers/ning-2026-effect-hyperparameters-language-modeling-computational/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/38/)

## References

55 references, 25 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde De Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, and 1 others. 2021. [Evaluating large language models trained on code](https://md.research.elamur.ai/free/en/papers/chen-2021-evaluating-large-language-models-trained/). arXiv preprint arXiv:2107.03374.
2. Shoufa Chen, Chongjian Ge, Zhan Tong, Jiangliu Wang, Yibing Song, Jue Wang, and Ping Luo. 2022. Adaptformer: Adapting vision transformers for scalable visual recognition. In Advances in Neural Information Processing Systems 35. [find](https://scholar.google.com/scholar?q=Shoufa+Chen%2C+Chongjian+Ge%2C+Zhan+Tong%2C+Jiangliu+Wang%2C+Yibing+Song%2C+Jue+Wang%2C+and+Ping+Luo%2E+2022%2E+Adaptformer%3A+Adapting+vision+transformers+for+scalable+visual+recognition%2E+In+Advances+in+Neural+Informa)
3. Gong Cheng, Junwei Han, and Xiaoqiang Lu. 2017. Remote sensing image scene classification: Benchmark and state of the art. Proceedings of the IEEE, 105(10):1865–1883. [find](https://scholar.google.com/scholar?q=Gong+Cheng%2C+Junwei+Han%2C+and+Xiaoqiang+Lu%2E+2017%2E+Remote+sensing+image+scene+classification%3A+Benchmark+and+state+of+the+art%2E+Proceedings+of+the+IEEE%2C+105%2810%29%3A1865%E2%80%931883%2E)
4. Mircea Cimpoi, Subhransu Maji, Iasonas Kokkinos, Sammy Mohamed, and Andrea Vedaldi. 2014. Describing textures in the wild. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 3606–3613. [find](https://scholar.google.com/scholar?q=Mircea+Cimpoi%2C+Subhransu+Maji%2C+Iasonas+Kokkinos%2C+Sammy+Mohamed%2C+and+Andrea+Vedaldi%2E+2014%2E+Describing+textures+in+the+wild%2E+In+Proceedings+of+the+IEEE+conference+on+computer+vision+and+pattern+recognit)
5. Peter Clark, Isaac Cowhey, Oren Etzioni, Tushar Khot, Ashish Sabharwal, Carissa Schoenick, and Oyvind Tafjord. 2018. [Think you have solved question answering? try arc, the ai2 reasoning challenge](https://md.research.elamur.ai/free/en/papers/clark-2018-think-have-solved-question-answering/). arXiv preprint arXiv:1803.05457.
6. Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, and 1 others. 2021. [Training verifiers to solve math word problems](https://md.research.elamur.ai/free/en/papers/cobbe-2021-training-verifiers-solve-math-word/). arXiv preprint arXiv:2110.14168.
7. Pala Tej Deep, Rishabh Bhardwaj, and Soujanya Poria. 2024. Della-merging: Reducing interference in model merging through magnitude-based sampling. CoRR, abs/2406.11617. — [arXiv:2406.11617](https://arxiv.org/abs/2406.11617) [find](https://scholar.google.com/scholar?q=Pala+Tej+Deep%2C+Rishabh+Bhardwaj%2C+and+Soujanya+Poria%2E+2024%2E+Della%2Dmerging%3A+Reducing+interference+in+model+merging+through+magnitude%2Dbased+sampling%2E+CoRR%2C+abs%2F2406%2E11617%2E+%E2%80%94+%5BarXiv%3A2406%2E11617%5D%28https%3A%2F%2Far)
8. DeepSeek-AI, Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, Xiaokang Zhang, Xingkai Yu, Yu Wu, Z. F. Wu, Zhibin Gou, Zhihong Shao, Zhuoshu Li, Ziyi Gao, and 181 others. 2025. [Deepseek-r1: Incentivizing reasoning capability in ll…](https://md.research.elamur.ai/free/en/papers/deepseek-ai-2025-deepseek-r1-incentivizing-reasoning-capability/)
9. DeepSeek-AI, Aixin Liu, Bei Feng, Bin Wang, Bingxuan Wang, Bo Liu, Chenggang Zhao, Chengqi Dengr, Chong Ruan, Damai Dai, Daya Guo, Dejian Yang, Deli Chen, Dongjie Ji, Erhang Li, Fangyun Lin, Fuli Luo, Guangbo Hao, Guanting Chen, and 138 others. 2024. Deepseek-v2: A strong, economical, and efficient… [find](https://scholar.google.com/scholar?q=DeepSeek%2DAI%2C+Aixin+Liu%2C+Bei+Feng%2C+Bin+Wang%2C+Bingxuan+Wang%2C+Bo+Liu%2C+Chenggang+Zhao%2C+Chengqi+Dengr%2C+Chong+Ruan%2C+Damai+Dai%2C+Daya+Guo%2C+Dejian+Yang%2C+Deli+Chen%2C+Dongjie+Ji%2C+Erhang+Li%2C+Fangyun+Lin%2C+Fuli+Luo%2C)
10. Mucong Ding, Chenghao Deng, Jocelyn Choo, Zichu Wu, Aakriti Agrawal, Avi Schwarzschild, Tianyi Zhou, Tom Goldstein, John Langford, Anima Anandkumar, and 1 others. 2024. Easy2hard-bench: Standardized difficulty labels for profiling llm performance and generalization. Advances in Neural Information Pr… [find](https://scholar.google.com/scholar?q=Mucong+Ding%2C+Chenghao+Deng%2C+Jocelyn+Choo%2C+Zichu+Wu%2C+Aakriti+Agrawal%2C+Avi+Schwarzschild%2C+Tianyi+Zhou%2C+Tom+Goldstein%2C+John+Langford%2C+Anima+Anandkumar%2C+and+1+others%2E+2024%2E+Easy2hard%2Dbench%3A+Standardized+d)
11. Jesse Dodge, Gabriel Ilharco, Roy Schwartz, Ali Farhadi, Hannaneh Hajishirzi, and Noah A. Smith. 2020. Fine-tuning pretrained language models: Weight initializations, data orders, and early stopping. CoRR, abs/2002.06305. [find](https://scholar.google.com/scholar?q=Jesse+Dodge%2C+Gabriel+Ilharco%2C+Roy+Schwartz%2C+Ali+Farhadi%2C+Hannaneh+Hajishirzi%2C+and+Noah+A%2E+Smith%2E+2020%2E+Fine%2Dtuning+pretrained+language+models%3A+Weight+initializations%2C+data+orders%2C+and+early+stopping%2E+)
12. Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, Anirudh Goyal, Anthony Hartshorn, Aobo Yang, Archi Mitra, Archie Sravankumar, Artem Korenev, Arthur Hinsvark, Arun Rao, Aston Zhang, and 82 others. 2024… — [The Llama 3 Herd of Models](https://md.research.elamur.ai/free/en/papers/grattafiori-2024-llama-3-herd-models/)
13. Kawin Ethayarajh, Winnie Xu, Niklas Muennighoff, Dan Jurafsky, and Douwe Kiela. 2024. KTO: model alignment as prospect theoretic optimization. In International Conference on Machine Learning. PMLR. [find](https://scholar.google.com/scholar?q=Kawin+Ethayarajh%2C+Winnie+Xu%2C+Niklas+Muennighoff%2C+Dan+Jurafsky%2C+and+Douwe+Kiela%2E+2024%2E+KTO%3A+model+alignment+as+prospect+theoretic+optimization%2E+In+International+Conference+on+Machine+Learning%2E+PMLR%2E)
14. Zeyu Han, Chao Gao, Jinyang Liu, Jeff Zhang, and Sai Qian Zhang. 2024. [Parameter-efficient fine-tuning for large models: A comprehensive survey](https://md.research.elamur.ai/free/en/papers/han-2024-parameter-efficient-fine-tuning-large/). CoRR, abs/2403.14608.
15. Patrick Helber, Benjamin Bischke, Andreas Dengel, and Damian Borth. 2019. Eurosat: A novel dataset and deep learning benchmark for land use and land cover classification. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 12(7):2217–2226. [find](https://scholar.google.com/scholar?q=Patrick+Helber%2C+Benjamin+Bischke%2C+Andreas+Dengel%2C+and+Damian+Borth%2E+2019%2E+Eurosat%3A+A+novel+dataset+and+deep+learning+benchmark+for+land+use+and+land+cover+classification%2E+IEEE+Journal+of+Selected+Topi)
16. Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. 2020. [Measuring massive multitask language understanding](https://md.research.elamur.ai/free/en/papers/hendrycks-2020-measuring-massive-multitask-language-understanding/). arXiv preprint arXiv:2009.03300.
17. Gabriel Ilharco, Marco Túlio Ribeiro, Mitchell Wortsman, Ludwig Schmidt, Hannaneh Hajishirzi, and Ali Farhadi. 2023. [Editing models with task arithmetic](https://md.research.elamur.ai/free/en/papers/ilharco-2022-editing-models-task-arithmetic/). In The Eleventh International Conference on Learning Representations. OpenReview.net.
18. James Liu, Guangxuan Xiao, Kai Li, Jason D. Lee, Song Han, Tri Dao, and Tianle Cai. 2024. Bitdelta: Your fine-tune may only be worth one bit. CoRR, abs/2402.10193. [find](https://scholar.google.com/scholar?q=James+Liu%2C+Guangxuan+Xiao%2C+Kai+Li%2C+Jason+D%2E+Lee%2C+Song+Han%2C+Tri+Dao%2C+and+Tianle+Cai%2E+2024%2E+Bitdelta%3A+Your+fine%2Dtune+may+only+be+worth+one+bit%2E+CoRR%2C+abs%2F2402%2E10193%2E)
19. Zhenyi Lu, Chenghao Fan, Wei Wei, Xiaoye Qu, Dangyang Chen, and Yu Cheng. 2024. Twin-merging: Dynamic integration of modular expertise in model merging. CoRR, abs/2406.15479. — [arXiv:2406.15479](https://arxiv.org/abs/2406.15479) [find](https://scholar.google.com/scholar?q=Zhenyi+Lu%2C+Chenghao+Fan%2C+Wei+Wei%2C+Xiaoye+Qu%2C+Dangyang+Chen%2C+and+Yu+Cheng%2E+2024%2E+Twin%2Dmerging%3A+Dynamic+integration+of+modular+expertise+in+model+merging%2E+CoRR%2C+abs%2F2406%2E15479%2E+%E2%80%94+%5BarXiv%3A2406%2E15479%5D%28http)
20. Naman Jain, King Han, Alex Gu, Wen-Ding Li, Fanjia Yan, Tianjun Zhang, Sida Wang, Armando Solar-Lezama, Koushik Sen, and Ion Stoica. 2024. [Livecodebench: Holistic and contamination free evaluation of large language models for code](https://md.research.elamur.ai/free/en/papers/jain-2024-livecodebench-holistic-contamination-free-evaluation/). arXiv preprint arXiv:2403.07974.
21. Haipeng Luo, Qingfeng Sun, Can Xu, Pu Zhao, Jian-guang Lou, Chongyang Tao, Xiubo Geng, Qingwei Lin, Shifeng Chen, and Dongmei Zhang. 2023. [Wizardmath: Empowering mathematical reasoning for large language models via reinforced evol-instruct](https://md.research.elamur.ai/free/en/papers/luo-2023-wizardmath-empowering-mathematical-reasoning-large/). CoRR, abs/2308.09583.
22. Albert Q. Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de Las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, Lélio Renard Lavaud, Marie-Anne Lachaux, Pierre Stock, Teven Le Scao, Thibaut Lavril, Thomas Wang, Timothée Lacroix,… [find](https://scholar.google.com/scholar?q=Albert+Q%2E+Jiang%2C+Alexandre+Sablayrolles%2C+Arthur+Mensch%2C+Chris+Bamford%2C+Devendra+Singh+Chaplot%2C+Diego+de+Las+Casas%2C+Florian+Bressand%2C+Gianna+Lengyel%2C+Guillaume+Lample%2C+Lucile+Saulnier%2C+L%C3%A9lio+Renard+Lav)
23. Ivan Moshkov, Darragh Hanley, Ivan Sorokin, Shubham Toshniwal, Christof Henkel, Benedikt Schifferer, Wei Du, and Igor Gitman. 2025. Aimo-2 winning solution: Building state-of-the-art mathematical reasoning models with openmathreasoning dataset. Preprint, arXiv:2504.16891. [find](https://scholar.google.com/scholar?q=Ivan+Moshkov%2C+Darragh+Hanley%2C+Ivan+Sorokin%2C+Shubham+Toshniwal%2C+Christof+Henkel%2C+Benedikt+Schifferer%2C+Wei+Du%2C+and+Igor+Gitman%2E+2025%2E+Aimo%2D2+winning+solution%3A+Building+state%2Dof%2Dthe%2Dart+mathematical+reas)
24. Albert Q. Jiang, Alexandre Sablayrolles, Antoine Roux, Arthur Mensch, Blanche Savary, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Emma Bou Hanna, Florian Bressand, Gianna Lengyel, Guillaume Bour, Guillaume Lample, Lélio Renard Lavaud, Lucile Saulnier, Marie-Anne Lachaux, Pierre Stock,… [find](https://scholar.google.com/scholar?q=Albert+Q%2E+Jiang%2C+Alexandre+Sablayrolles%2C+Antoine+Roux%2C+Arthur+Mensch%2C+Blanche+Savary%2C+Chris+Bamford%2C+Devendra+Singh+Chaplot%2C+Diego+de+las+Casas%2C+Emma+Bou+Hanna%2C+Florian+Bressand%2C+Gianna+Lengyel%2C+Guill)
25. Jonathan Krause, Michael Stark, Jia Deng, and Li Fei-Fei. 2013. 3d object representations for fine-grained categorization. In Proceedings of the IEEE international conference on computer vision workshops, pages 554–561. [find](https://scholar.google.com/scholar?q=Jonathan+Krause%2C+Michael+Stark%2C+Jia+Deng%2C+and+Li+Fei%2DFei%2E+2013%2E+3d+object+representations+for+fine%2Dgrained+categorization%2E+In+Proceedings+of+the+IEEE+international+conference+on+computer+vision+worksh)
26. Nathan Lambert, Jacob Morrison, Valentina Pyatkin, Shengyi Huang, Hamish Ivison, Faeze Brahman, Lester James V. Miranda, Alisa Liu, Nouha Dziri, Shane Lyu, Yuling Gu, Saumya Malik, Victoria Graf, Jena D. Hwang, Jiangjiang Yang, Ronan Le Bras, Oyvind Tafjord, Chris Wilhelm, Luca Soldaini, and 4 other… — [Tulu 3: Pushing Frontiers in Open Language Model Post-Training](https://md.research.elamur.ai/free/en/papers/lambert-2024-tulu-3-pushing-frontiers-open/)
27. Yann LeCun, Corinna Cortes, and CJ Burges. 2010. Mnist handwritten digit database. ATT Labs [Online]. Available: [http://yann.lecun.com/exdb/mnist](http://yann.lecun.com/exdb/mnist), 2. [↗](http://yann.lecun.com/exdb/mnist)
28. Hunter Lightman, Vineet Kosaraju, Yura Burda, Harri Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever, and Karl Cobbe. 2023. [Let’s verify step by step](https://md.research.elamur.ai/free/en/papers/lightman-2023-let-s-verify-step-step/). Preprint, arXiv:2305.20050.
29. Stephanie Lin, Jacob Hilton, and Owain Evans. 2021. [Truthfulqa: Measuring how models mimic human falsehoods](https://md.research.elamur.ai/free/en/papers/lin-2022-truthfulqa-measuring-how-models-mimic/). arXiv preprint arXiv:2109.07958.
30. Yuval Netzer, Tao Wang, Adam Coates, Alessandro Bissacco, Baolin Wu, Andrew Y Ng, and 1 others. 2011. Reading digits in natural images with unsupervised feature learning. In NIPS workshop on deep learning and unsupervised feature learning, volume 2011, page 4. Granada. [find](https://scholar.google.com/scholar?q=Yuval+Netzer%2C+Tao+Wang%2C+Adam+Coates%2C+Alessandro+Bissacco%2C+Baolin+Wu%2C+Andrew+Y+Ng%2C+and+1+others%2E+2011%2E+Reading+digits+in+natural+images+with+unsupervised+feature+learning%2E+In+NIPS+workshop+on+deep+lear)
31. Long Ouyang, Jeff Wu, Xu Jiang, Diogo Almeida, Carroll L. Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, John Schulman, Jacob Hilton, Fraser Kelton, Luke Miller, Maddie Simens, Amanda Askell, Peter Welinder, Paul Christiano, Jan Leike, and Ryan Lowe. 2022. [Train…](https://md.research.elamur.ai/free/en/papers/ouyang-2022-training-language-models-follow-instructions/)
32. Qwen, :, An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, Huan Lin, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jingren Zhou, and 25 others. 2025. Qwen2.5 technical report. Preprint, arXiv:2412.15115. [find](https://scholar.google.com/scholar?q=Qwen%2C+%3A%2C+An+Yang%2C+Baosong+Yang%2C+Beichen+Zhang%2C+Binyuan+Hui%2C+Bo+Zheng%2C+Bowen+Yu%2C+Chengyuan+Li%2C+Dayiheng+Liu%2C+Fei+Huang%2C+Haoran+Wei%2C+Huan+Lin%2C+Jian+Yang%2C+Jianhong+Tu%2C+Jianwei+Zhang%2C+Jianxin+Yang%2C+Jiaxi+)
33. Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, and 1 others. 2021. [Learning transferable visual models from natural language supervision](https://md.research.elamur.ai/free/en/papers/radford-2021-learning-transferable-visual-models-natural/). In International conference on machine learning, pages 8748–8…
34. Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D. Manning, Stefano Ermon, and Chelsea Finn. 2023. [Direct preference optimization: Your language model is secretly a reward model](https://md.research.elamur.ai/free/en/papers/rafailov-2023-direct-preference-optimization-language-model/). In Advances in Neural Information Processing Systems 36.
35. David Rein, Betty Li Hou, Asa Cooper Stickland, Jackson Petty, Richard Yuanzhe Pang, Julien Dirani, Julian Michael, and Samuel R Bowman. 2024. [Gpqa: A graduate-level google-proof q&a benchmark](https://md.research.elamur.ai/free/en/papers/rein-2023-gpqa-graduate-level-google-proof/). In First Conference on Language Modeling.
36. Keisuke Sakaguchi, Ronan Le Bras, Chandra Bhagavatula, and Yejin Choi. 2021. Winogrande: An adversarial winograd schema challenge at scale. Communications of the ACM, 64(9):99–106. [find](https://scholar.google.com/scholar?q=Keisuke+Sakaguchi%2C+Ronan+Le+Bras%2C+Chandra+Bhagavatula%2C+and+Yejin+Choi%2E+2021%2E+Winogrande%3A+An+adversarial+winograd+schema+challenge+at+scale%2E+Communications+of+the+ACM%2C+64%289%29%3A99%E2%80%93106%2E)
37. Mark Sandler, Andrey Zhmoginov, Max Vladymyrov, and Andrew Jackson. 2022. Fine-tuning image transformers using learnable memory. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2022, New Orleans, LA, USA, June 18-24, 2022, pages 12145–12154. IEEE. [find](https://scholar.google.com/scholar?q=Mark+Sandler%2C+Andrey+Zhmoginov%2C+Max+Vladymyrov%2C+and+Andrew+Jackson%2E+2022%2E+Fine%2Dtuning+image+transformers+using+learnable+memory%2E+In+IEEE%2FCVF+Conference+on+Computer+Vision+and+Pattern+Recognition%2C+CVPR)
38. Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, Y. K. Li, Y. Wu, and Daya Guo. 2024. [Deepseekmath: Pushing the limits of mathematical reasoning in open language models](https://md.research.elamur.ai/free/en/papers/shao-2024-deepseekmath-pushing-limits-mathematical-reasoning/). Preprint, arXiv:2402.03300.
39. Johannes Stallkamp, Marc Schlipsing, Jan Salmen, and Christian Igel. 2011. The german traffic sign recognition benchmark: a multi-class classification competition. In The 2011 international joint conference on neural networks, pages 1453–1460. IEEE. [find](https://scholar.google.com/scholar?q=Johannes+Stallkamp%2C+Marc+Schlipsing%2C+Jan+Salmen%2C+and+Christian+Igel%2E+2011%2E+The+german+traffic+sign+recognition+benchmark%3A+a+multi%2Dclass+classification+competition%2E+In+The+2011+international+joint+conf)
40. Prateek Yadav, Derek Tam, Leshem Choshen, Colin A. Raffel, and Mohit Bansal. 2023. [Ties-merging: Resolving interference when merging models](https://md.research.elamur.ai/free/en/papers/yadav-2023-ties-merging-resolving-interference-merging/). In Advances in Neural Information Processing Systems 36.
41. An Yang, Baosong Yang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Zhou, Chengpeng Li, Chengyuan Li, Dayiheng Liu, Fei Huang, Guanting Dong, Haoran Wei, Huan Lin, Jialong Tang, Jialin Wang, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Ma, and 43 others. 2024. Qwen2 technical report. Preprint, arXiv:240… [find](https://scholar.google.com/scholar?q=An+Yang%2C+Baosong+Yang%2C+Binyuan+Hui%2C+Bo+Zheng%2C+Bowen+Yu%2C+Chang+Zhou%2C+Chengpeng+Li%2C+Chengyuan+Li%2C+Dayiheng+Liu%2C+Fei+Huang%2C+Guanting+Dong%2C+Haoran+Wei%2C+Huan+Lin%2C+Jialong+Tang%2C+Jialin+Wang%2C+Jian+Yang%2C+Jian)
42. Le Yu, Bowen Yu, Haiyang Yu, Fei Huang, and Yongbin Li. 2024. [Language models are super mario: Absorbing abilities from homologous models as a free lunch](https://md.research.elamur.ai/free/en/papers/yu-2023-language-models-super-mario-absorbing/). In International Conference on Machine Learning. PMLR.
43. Qwen Team. 2025. [Qwen3 technical report](https://md.research.elamur.ai/free/en/papers/yang-2025-qwen3-technical-report/). Preprint, arXiv:2505.09388.
44. Qiying Yu, Zheng Zhang, Ruofei Zhu, Yufeng Yuan, Xiaochen Zuo, Yu Yue, Weinan Dai, Tiantian Fan, Gaohong Liu, Lingjun Liu, Xin Liu, Haibin Lin, Zhiqi Lin, Bole Ma, Guangming Sheng, Yuxuan Tong, Chi Zhang, Mofan Zhang, Wang Zhang, and 16 others. 2025. [Dapo: An open-source llm reinforcement learning s…](https://md.research.elamur.ai/free/en/papers/yu-2025-dapo-open-source-llm-reinforcement/)
45. Yuxuan Tong, Xiwen Zhang, Rui Wang, Ruidong Wu, and Junxian He. 2024. Dart-math: Difficulty-aware rejection tuning for mathematical problem-solving. CoRR, abs/2407.13690. [find](https://scholar.google.com/scholar?q=Yuxuan+Tong%2C+Xiwen+Zhang%2C+Rui+Wang%2C+Ruidong+Wu%2C+and+Junxian+He%2E+2024%2E+Dart%2Dmath%3A+Difficulty%2Daware+rejection+tuning+for+mathematical+problem%2Dsolving%2E+CoRR%2C+abs%2F2407%2E13690%2E)
46. Rowan Zellers, Ari Holtzman, Yonatan Bisk, Ali Farhadi, and Yejin Choi. 2019. Hellaswag: Can a machine really finish your sentence? arXiv preprint arXiv:1905.07830. [find](https://scholar.google.com/scholar?q=Rowan+Zellers%2C+Ari+Holtzman%2C+Yonatan+Bisk%2C+Ali+Farhadi%2C+and+Yejin+Choi%2E+2019%2E+Hellaswag%3A+Can+a+machine+really+finish+your+sentence%3F+arXiv+preprint+arXiv%3A1905%2E07830%2E)
47. Junqiao Wang, Zeng Zhang, Yangfan He, Zihao Zhang, Xinyuan Song, Yuyang Song, Tianyu Shi, Yuchen Li, Hengyuan Xu, Kunyu Wu, Xin Yi, Zhongwei Wan, Xinhang Yuan, Zijun Wang, Kuan Lu, Menghao Huo, Tang Jingqun, Guangwu Qian, Keqin Li, and 2 others. 2025. Enhancing code llms with reinforcement learning… — [arXiv:2412.20367](https://arxiv.org/abs/2412.20367) [find](https://scholar.google.com/scholar?q=Junqiao+Wang%2C+Zeng+Zhang%2C+Yangfan+He%2C+Zihao+Zhang%2C+Xinyuan+Song%2C+Yuyang+Song%2C+Tianyu+Shi%2C+Yuchen+Li%2C+Hengyuan+Xu%2C+Kunyu+Wu%2C+Xin+Yi%2C+Zhongwei+Wan%2C+Xinhang+Yuan%2C+Zijun+Wang%2C+Kuan+Lu%2C+Menghao+Huo%2C+Tang+J)
48. Wayne Xin Zhao, Kun Zhou, Junyi Li, Tianyi Tang, Xiaolei Wang, Yupeng Hou, Yingqian Min, Beichen Zhang, Junjie Zhang, Zican Dong, Yifan Du, Chen Yang, Yushuo Chen, Zhipeng Chen, Jinhao Jiang, Ruiyang Ren, Yifan Li, Xinyu Tang, Zikang Liu, and 3 others. 2023. [A survey of large language models](https://md.research.elamur.ai/free/en/papers/zhao-2023-survey-large-language-models/). CoRR,…
49. Zhichao Wang, Bin Bi, Shiva Kumar Pentyala, Kiran Ramnath, Sougata Chaudhuri, Shubham Mehrotra, Zixu, Zhu, Xiang-Bo Mao, Sitaram Asur, Na, and Cheng. 2024. A comprehensive survey of llm alignment techniques: Rlhf, rlaif, ppo, dpo and more. Preprint, arXiv:2407.16216. — [arXiv:2407.16216](https://arxiv.org/abs/2407.16216) [find](https://scholar.google.com/scholar?q=Zhichao+Wang%2C+Bin+Bi%2C+Shiva+Kumar+Pentyala%2C+Kiran+Ramnath%2C+Sougata+Chaudhuri%2C+Shubham+Mehrotra%2C+Zixu%2C+Zhu%2C+Xiang%2DBo+Mao%2C+Sitaram+Asur%2C+Na%2C+and+Cheng%2E+2024%2E+A+comprehensive+survey+of+llm+alignment+tech)
50. Jianxiong Xiao, Krista A Ehinger, James Hays, Antonio Torralba, and Aude Oliva. 2016. Sun database: Exploring a large collection of scene categories. International Journal of Computer Vision, 119:3–22. [find](https://scholar.google.com/scholar?q=Jianxiong+Xiao%2C+Krista+A+Ehinger%2C+James+Hays%2C+Antonio+Torralba%2C+and+Aude+Oliva%2E+2016%2E+Sun+database%3A+Exploring+a+large+collection+of+scene+categories%2E+International+Journal+of+Computer+Vision%2C+119%3A3%E2%80%9322)
51. Yi Xin, Siqi Luo, Haodi Zhou, Junlong Du, Xiaohong Liu, Yue Fan, Qing Li, and Yuntao Du. 2024. Parameter-efficient fine-tuning for pre-trained vision models: A survey. CoRR, abs/2402.02242. — [arXiv:2402.02242](https://arxiv.org/abs/2402.02242) [find](https://scholar.google.com/scholar?q=Yi+Xin%2C+Siqi+Luo%2C+Haodi+Zhou%2C+Junlong+Du%2C+Xiaohong+Liu%2C+Yue+Fan%2C+Qing+Li%2C+and+Yuntao+Du%2E+2024%2E+Parameter%2Defficient+fine%2Dtuning+for+pre%2Dtrained+vision+models%3A+A+survey%2E+CoRR%2C+abs%2F2402%2E02242%2E+%E2%80%94+%5BarXiv%3A2)
52. Wenting Zhao, Xiang Ren, Jack Hessel, Claire Cardie, Yejin Choi, and Yuntian Deng. 2024. [Wildchat: 1m chatGPT interaction logs in the wild](https://md.research.elamur.ai/free/en/papers/zhao-2024-wildchat-1m-chatgpt-interaction-logs/). In The Twelfth International Conference on Learning Representations.
53. Chujie Zheng, Ziqi Wang, Heng Ji, Minlie Huang, and Nanyun Peng. 2024. Weak-to-strong extrapolation expedites alignment. CoRR, abs/2404.16792. — [arXiv:2404.16792](https://arxiv.org/abs/2404.16792) [find](https://scholar.google.com/scholar?q=Chujie+Zheng%2C+Ziqi+Wang%2C+Heng+Ji%2C+Minlie+Huang%2C+and+Nanyun+Peng%2E+2024%2E+Weak%2Dto%2Dstrong+extrapolation+expedites+alignment%2E+CoRR%2C+abs%2F2404%2E16792%2E+%E2%80%94+%5BarXiv%3A2404%2E16792%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2404%2E16792%29)
54. Jeffrey Zhou, Tianjian Lu, Swaroop Mishra, Siddhartha Brahma, Sujoy Basu, Yi Luan, Denny Zhou, and Le Hou. 2023. [Instruction-following evaluation for large language models](https://md.research.elamur.ai/free/en/papers/zhou-2023-instruction-following-evaluation-large-language/). Preprint, arXiv:2311.07911.
55. Shusheng Xu, Wei Fu, Jiaxuan Gao, Wenjie Ye, Weilin Liu, Zhiyu Mei, Guangju Wang, Chao Yu, and Yi Wu. 2024. Is DPO superior to PPO for LLM alignment? a comprehensive study. In Forty-first International Conference on Machine Learning. [find](https://scholar.google.com/scholar?q=Shusheng+Xu%2C+Wei+Fu%2C+Jiaxuan+Gao%2C+Wenjie+Ye%2C+Weilin+Liu%2C+Zhiyu+Mei%2C+Guangju+Wang%2C+Chao+Yu%2C+and+Yi+Wu%2E+2024%2E+Is+DPO+superior+to+PPO+for+LLM+alignment%3F+a+comprehensive+study%2E+In+Forty%2Dfirst+Internationa)

*Updated 1 November 2026*