---
title: "LLEOT — paper by Zhong et al. (2026) — summary and ideas"
description: "They propose LLEOT. 49 references. Offsite Tuning (OT) lets a data owner adapt a proprietary LLM without either side sharing its assets."
authors: ["Jin Zhong", "Jinglin Liang", "Tongtong Yang", "Zijian Xie", "Shuangping Huang", "Hanlin Gu"]
year: 2026
venue: "Findings of the Association for Computational Linguistics: ACL 2026"
source: "https://aclanthology.org/2026.findings-acl.1878/"
doi: "10.18653/v1/2026.findings-acl.1878"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/zhong-2026-lleot-privacy-enhancing-offsite-tuning/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/zhong-2026-lleot-privacy-enhancing-offsite-tuning/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/39/) / LLEOT: A Privacy-Enhancing Offsite Tuning Framework via Loss Landscape Elevation

# LLEOT: A Privacy-Enhancing Offsite Tuning Framework via Loss Landscape Elevation

**Authors:** Jin Zhong, Jinglin Liang, Tongtong Yang, Zijian Xie, Shuangping Huang and [Hanlin Gu](https://md.research.elamur.ai/free/en/authors/hanlin-gu/) · **Year:** 2026 · **Venue:** [Findings of the Association for Computational Linguistics: ACL 2026](https://md.research.elamur.ai/free/en/venues/findings/) ([Findings of the Association for Computational Linguistics 2026](https://md.research.elamur.ai/free/en/venues/findings/2026/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2026.findings-acl.1878/) · [PDF](https://aclanthology.org/2026.findings-acl.1878.pdf) · [DOI 10.18653/v1/2026.findings-acl.1878](https://doi.org/10.18653/v1/2026.findings-acl.1878) · [OpenAlex](https://openalex.org/W7166826931)

**Citations:** 0 citations (OpenAlex, 2026-10-14)

## Summary
Offsite Tuning (OT) lets a data owner adapt a proprietary LLM without either side sharing its assets. The model owner sends a reduced "emulator", the data owner trains an adapter on private data, and the adapter is plugged back into the original model. The authors point out a gap: emulators built by OT, CRaSh or GradOT still perform well on their own, so a dishonest data owner could exploit the model's capabilities. They propose LLEOT. It starts from a uniformly layer-dropped copy of the model and applies Loss Landscape Elevation (LLE), which trains the emulator so its loss sits a constant margin H above the original model's on every input. The authors prove this makes the emulator's perplexity grow exponentially while keeping soft-prompt gradients aligned, so prompts tuned on the emulator still transfer. An optional distillation step, CPKD, supports heavier pruning. Experiments on Qwen2-1.5B show that LLEOT emulators score near or below a randomly initialized model, yet the tuned prompts still work well on the original. This protects both data and model privacy.

## Key points
- The paper introduces Capability Privacy Leakage (CPL), the emulator's zero-shot score divided by the original model's. Lower values mean better protection, and protection counts only below 100%.
- In theory, the emulator-to-original perplexity ratio should be about e^H. On Wikitext-2 the measured ratio closely follows this. On downstream sets such as SIQA it still rises exponentially but departs from the prediction, likely due to domain shift.
- Uniform LayerDrop gave better plug-in performance and weaker zero-shot ability than SliceGPT, ShortGPT and BlockPruner, at no extra compute. Uniform elevation also beat directly maximizing the language-modeling loss on plug-in performance.
- At pruning ratio 0.5 on Qwen2-1.5B, CPKD raised plug-in scores substantially:
  - OBQA: 23.00 → 34.20
  - WebQs: 9.35 → 24.15
- Evaluation tasks include OBQA, ARC-c, SIQA and WebQs. Code is released at github.com/Z-eloto/LLEOT. Larger models such as Qwen2-72B remain untested, and stronger theoretical guarantees are left open.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Hong Kong University of Science and Technology](https://md.research.elamur.ai/free/en/universities/hong-kong-university-of-science-and-technology/) — University, Hong Kong, China · 170 works · 26,341 citations
- [South China University of Technology](https://md.research.elamur.ai/free/en/universities/south-china-university-of-technology/) — University, Guangzhou, China · 27 works · 1,554 citations

📚 Shelves (2): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 18,614 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 17,965 works

## Cite

Cite as: Jin Zhong, Jinglin Liang, Tongtong Yang, Zijian Xie, Shuangping Huang and Hanlin Gu. 2026. LLEOT: A Privacy-Enhancing Offsite Tuning Framework via Loss Landscape Elevation. Findings of the Association for Computational Linguistics: ACL 2026.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{zhong2026lleot,
  title = {{LLEOT: A Privacy-Enhancing Offsite Tuning Framework via Loss Landscape Elevation}},
  author = {Jin Zhong, Jinglin Liang, Tongtong Yang, Zijian Xie, Shuangping Huang and Hanlin Gu},
  booktitle = {Findings of the Association for Computational Linguistics: ACL 2026},
  year = {2026},
  doi = {10.18653/v1/2026.findings-acl.1878},
  url = {https://doi.org/10.18653/v1/2026.findings-acl.1878}
}
```

- [← previous LiveCultureBench: a Multi-Agent, Multi-Cultural Benchmark for Large Language Models in Dynamic Social Simulations](https://md.research.elamur.ai/free/en/papers/pham-2026-liveculturebench-multi-agent-multi-cultural/)
- [next → LLM as a Morphological Disambiguator for Belarusian: A Preliminary Study](https://md.research.elamur.ai/free/en/papers/poritski-2026-llm-as-morphological-disambiguator-belarusian/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/39/)

## References

49 references, 14 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Saleh Ashkboos, Maximilian L Croci, Marcelo Gennari do Nascimento, Torsten Hoefler, and James Hensman. 2024. Slicegpt: Compress large language models by deleting rows and columns. In The Twelfth International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Saleh+Ashkboos%2C+Maximilian+L+Croci%2C+Marcelo+Gennari+do+Nascimento%2C+Torsten+Hoefler%2C+and+James+Hensman%2E+2024%2E+Slicegpt%3A+Compress+large+language+models+by+deleting+rows+and+columns%2E+In+The+Twelfth+Inter)
2. Jiamu Bai, Daoyuan Chen, Bingchen Qian, Liuyi Yao, and Yaliang Li. 2024. Federated fine-tuning of large language models under heterogeneous tasks and client resources. Advances in Neural Information Processing Systems, 37:14457–14483. [find](https://scholar.google.com/scholar?q=Jiamu+Bai%2C+Daoyuan+Chen%2C+Bingchen+Qian%2C+Liuyi+Yao%2C+and+Yaliang+Li%2E+2024%2E+Federated+fine%2Dtuning+of+large+language+models+under+heterogeneous+tasks+and+client+resources%2E+Advances+in+Neural+Information+P)
3. Jonathan Berant, Andrew Chou, Roy Frostig, and Percy Liang. 2013. [Semantic parsing on freebase from question-answer pairs](https://md.research.elamur.ai/free/en/papers/berant-2013-semantic-parsing-freebase-question-answer/). In Proceedings of the 2013 conference on empirical methods in natural language processing, pages 1533–1544.
4. Terence Jie Chua, Wenhan Yu, Jun Zhao, and Kwok-Yan Lam. 2023. Fedpeat: Convergence of federated learning, parameter-efficient fine tuning, and emulator assisted tuning for artificial intelligence foundation models with mobile edge computing. arXiv preprint arXiv:2310.17491. — [arXiv:2310.17491](https://arxiv.org/abs/2310.17491) [find](https://scholar.google.com/scholar?q=Terence+Jie+Chua%2C+Wenhan+Yu%2C+Jun+Zhao%2C+and+Kwok%2DYan+Lam%2E+2023%2E+Fedpeat%3A+Convergence+of+federated+learning%2C+parameter%2Defficient+fine+tuning%2C+and+emulator+assisted+tuning+for+artificial+intelligence+fou)
5. Peter Clark, Isaac Cowhey, Oren Etzioni, Tushar Khot, Ashish Sabharwal, Carissa Schoenick, and Oyvind Tafjord. 2018. [Think you have solved question answering? try arc, the ai2 reasoning challenge](https://md.research.elamur.ai/free/en/papers/clark-2018-think-have-solved-question-answering/). arXiv preprint arXiv:1803.05457.
6. Gheorghe Comanici, Eric Bieber, Mike Schaekermann, Ice Pasupat, Noveen Sachdeva, Inderjit Dhillon, Marcel Blistein, Ori Ram, Dan Zhang, Evan Rosen, and 1 others. 2025. [Gemini 2.5: Pushing the frontier with advanced reasoning, multimodality, long context, and next generation agentic capabilities](https://md.research.elamur.ai/free/en/papers/comanici-2025-gemini-2-5-pushing-frontier/). arX…
7. Chenhe Dong, Yuexiang Xie, Bolin Ding, Ying Shen, and Yaliang Li. 2023. Tunable soft prompts are messengers in federated learning. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 14665–14675. [find](https://scholar.google.com/scholar?q=Chenhe+Dong%2C+Yuexiang+Xie%2C+Bolin+Ding%2C+Ying+Shen%2C+and+Yaliang+Li%2E+2023%2E+Tunable+soft+prompts+are+messengers+in+federated+learning%2E+In+Findings+of+the+Association+for+Computational+Linguistics%3A+EMNLP+2)
8. Haonan Duan, Adam Dziedzic, Mohammad Yaghini, Nicolas Papernot, and Franziska Boenisch. 2024. On the privacy risk of in-context learning. arXiv preprint arXiv:2411.10512. — [arXiv:2411.10512](https://arxiv.org/abs/2411.10512) [find](https://scholar.google.com/scholar?q=Haonan+Duan%2C+Adam+Dziedzic%2C+Mohammad+Yaghini%2C+Nicolas+Papernot%2C+and+Franziska+Boenisch%2E+2024%2E+On+the+privacy+risk+of+in%2Dcontext+learning%2E+arXiv+preprint+arXiv%3A2411%2E10512%2E+%E2%80%94+%5BarXiv%3A2411%2E10512%5D%28https%3A%2F%2F)
9. Shengjie Gong, Wenjie Peng, Hongyuan Chen, Gangyu Zhang, Yunqing Hu, Huiyuan Zhang, Shuangping Huang, and Tianshui Chen. 2026. Learning hierarchical and geometry-aware graph representations for text-to-cad. In The Fourteenth International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Shengjie+Gong%2C+Wenjie+Peng%2C+Hongyuan+Chen%2C+Gangyu+Zhang%2C+Yunqing+Hu%2C+Huiyuan+Zhang%2C+Shuangping+Huang%2C+and+Tianshui+Chen%2E+2026%2E+Learning+hierarchical+and+geometry%2Daware+graph+representations+for+text%2Dt)
10. Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, and 1 others. 2024. [The llama 3 herd of models](https://md.research.elamur.ai/free/en/papers/grattafiori-2024-llama-3-herd-models/). arXiv preprint arXiv:2407.21783.
11. Samyak Gupta, Yangsibo Huang, Zexuan Zhong, Tianyu Gao, Kai Li, and Danqi Chen. 2022. Recovering private text in federated learning of language models. Advances in neural information processing systems, 35:8130–8143. [find](https://scholar.google.com/scholar?q=Samyak+Gupta%2C+Yangsibo+Huang%2C+Zexuan+Zhong%2C+Tianyu+Gao%2C+Kai+Li%2C+and+Danqi+Chen%2E+2022%2E+Recovering+private+text+in+federated+learning+of+language+models%2E+Advances+in+neural+information+processing+system)
12. Geoffrey Hinton, Oriol Vinyals, and Jeff Dean. 2015. [Distilling the knowledge in a neural network](https://md.research.elamur.ai/free/en/papers/hinton-2015-distilling-knowledge-neural-network/). arXiv preprint arXiv:1503.02531.
13. Tiffany H Kung, Morgan Cheatham, Arielle Medenilla, Czarina Sillos, Lorie De Leon, Camille Elepaño, Maria Madriaga, Rimel Aggabao, Giezel Diaz-Candido, James Maningo, and 1 others. 2023. [Performance of chatgpt on usmle: potential for ai-assisted medical education using large language models](https://md.research.elamur.ai/free/en/papers/kung-2023-performance-chatgpt-usmle-potential-ai/). PLoS di…
14. Bo Li, Peng Qi, Bo Liu, Shuai Di, Jingen Liu, Jiquan Pei, Jinfeng Yi, and Bowen Zhou. 2023. Trustworthy ai: From principles to practices. ACM Computing Surveys, 55(9):1–46. [find](https://scholar.google.com/scholar?q=Bo+Li%2C+Peng+Qi%2C+Bo+Liu%2C+Shuai+Di%2C+Jingen+Liu%2C+Jiquan+Pei%2C+Jinfeng+Yi%2C+and+Bowen+Zhou%2E+2023%2E+Trustworthy+ai%3A+From+principles+to+practices%2E+ACM+Computing+Surveys%2C+55%289%29%3A1%E2%80%9346%2E)
15. Mengze Hong, Yi Gu, Di Jiang, Hanlin Gu, Chen Jason Zhang, Lu Wang, and Zhiyang Su. 2026a. Federated heterogeneous language model optimization for hybrid automatic speech recognition. Preprint, arXiv:2603.04945. — [arXiv:2603.04945](https://arxiv.org/abs/2603.04945) [find](https://scholar.google.com/scholar?q=Mengze+Hong%2C+Yi+Gu%2C+Di+Jiang%2C+Hanlin+Gu%2C+Chen+Jason+Zhang%2C+Lu+Wang%2C+and+Zhiyang+Su%2E+2026a%2E+Federated+heterogeneous+language+model+optimization+for+hybrid+automatic+speech+recognition%2E+Preprint%2C+arXiv%3A)
16. Zhuohang Li, Chao Yan, Xinmeng Zhang, Gharib Gharibi, Zhijun Yin, Xiaoqian Jiang, and Bradley A Malin. 2024. Split learning for distributed collaborative training of deep learning models in health informatics. In AMIA Annual Symposium Proceedings, volume 2023, page 1047. [find](https://scholar.google.com/scholar?q=Zhuohang+Li%2C+Chao+Yan%2C+Xinmeng+Zhang%2C+Gharib+Gharibi%2C+Zhijun+Yin%2C+Xiaoqian+Jiang%2C+and+Bradley+A+Malin%2E+2024%2E+Split+learning+for+distributed+collaborative+training+of+deep+learning+models+in+health+inf)
17. Mengze Hong, Chen Jason Zhang, Zichang Guo, Hanlin Gu, Di Jiang, and Qing Li. 2026b. Orchestration-free customer service automation: A privacy-preserving and flowchart-guided framework. In Proceedings of the ACM Web Conference 2026, WWW ’26, page 8138–8149, New York, NY, USA. Association for Computi… [find](https://scholar.google.com/scholar?q=Mengze+Hong%2C+Chen+Jason+Zhang%2C+Zichang+Guo%2C+Hanlin+Gu%2C+Di+Jiang%2C+and+Qing+Li%2E+2026b%2E+Orchestration%2Dfree+customer+service+automation%3A+A+privacy%2Dpreserving+and+flowchart%2Dguided+framework%2E+In+Proceedings)
18. Jinglin Liang, Jin Zhong, Shuangping Huang, Yunqing Hu, Huiyuan Zhang, Huifang Li, Lixin Fan, and Hanlin Gu. 2025. Order-level attention similarity across language models: A latent commonality. In The Thirty-ninth Annual Conference on Neural Information Processing Systems. [find](https://scholar.google.com/scholar?q=Jinglin+Liang%2C+Jin+Zhong%2C+Shuangping+Huang%2C+Yunqing+Hu%2C+Huiyuan+Zhang%2C+Huifang+Li%2C+Lixin+Fan%2C+and+Hanlin+Gu%2E+2025%2E+Order%2Dlevel+attention+similarity+across+language+models%3A+A+latent+commonality%2E+In+The)
19. Jiayi Huang, Yuanyuan Zhang, Renwan Bi, Jiayin Lin, and Jinbo Xiong. 2024. Knowledge distillation enables federated learning: A data-free federated aggregation scheme. In 2024 International Joint Conference on Neural Networks (IJCNN), pages 1–7. IEEE. [find](https://scholar.google.com/scholar?q=Jiayi+Huang%2C+Yuanyuan+Zhang%2C+Renwan+Bi%2C+Jiayin+Lin%2C+and+Jinbo+Xiong%2E+2024%2E+Knowledge+distillation+enables+federated+learning%3A+A+data%2Dfree+federated+aggregation+scheme%2E+In+2024+International+Joint+Conf)
20. Haokun Liu, Derek Tam, Mohammed Muqeeth, Jay Mohta, Tenghao Huang, Mohit Bansal, and Colin A Raffel. 2022. [Few-shot parameter-efficient fine-tuning is better and cheaper than in-context learning](https://md.research.elamur.ai/free/en/papers/liu-2022-few-shot-parameter-efficient-fine/). Advances in Neural Information Processing Systems, 35:1950–1965.
21. Gopi Krishna Jagarlamudi, Abbas Yazdinejad, Reza M Parizi, and Seyedamin Pouriyeh. 2024. Exploring privacy measurement in federated learning. The Journal of Supercomputing, 80(8):10511–10551. [find](https://scholar.google.com/scholar?q=Gopi+Krishna+Jagarlamudi%2C+Abbas+Yazdinejad%2C+Reza+M+Parizi%2C+and+Seyedamin+Pouriyeh%2E+2024%2E+Exploring+privacy+measurement+in+federated+learning%2E+The+Journal+of+Supercomputing%2C+80%288%29%3A10511%E2%80%9310551%2E)
22. Yan Kang, Hanlin Gu, Xingxing Tang, Yuanqin He, Yuzhu Zhang, Jinnan He, Yuxing Han, Lixin Fan, Kai Chen, and Qiang Yang. 2024. Optimizing privacy, utility, and efficiency in a constrained multi-objective federated learning framework. ACM Transactions on Intelligent Systems and Technology, 15(6):1–33… [find](https://scholar.google.com/scholar?q=Yan+Kang%2C+Hanlin+Gu%2C+Xingxing+Tang%2C+Yuanqin+He%2C+Yuzhu+Zhang%2C+Jinnan+He%2C+Yuxing+Han%2C+Lixin+Fan%2C+Kai+Chen%2C+and+Qiang+Yang%2E+2024%2E+Optimizing+privacy%2C+utility%2C+and+efficiency+in+a+constrained+multi%2Dobject)
23. Yan Kang, Yuanqin He, Jiahuan Luo, Tao Fan, Yang Liu, and Qiang Yang. 2022. Privacy-preserving federated adversarial domain adaptation over feature groups for interpretability. IEEE Transactions on Big Data, 10(6):879–890. [find](https://scholar.google.com/scholar?q=Yan+Kang%2C+Yuanqin+He%2C+Jiahuan+Luo%2C+Tao+Fan%2C+Yang+Liu%2C+and+Qiang+Yang%2E+2022%2E+Privacy%2Dpreserving+federated+adversarial+domain+adaptation+over+feature+groups+for+interpretability%2E+IEEE+Transactions+on+Bi)
24. Takeshi Kojima, Shixiang Shane Gu, Machel Reid, Yutaka Matsuo, and Yusuke Iwasawa. 2022. [Large language models are zero-shot reasoners](https://md.research.elamur.ai/free/en/papers/kojima-2022-large-language-models-zero-shot/). Advances in neural information processing systems, 35:22199– 22213.
25. Zhiqiang Kou, Jing Wang, Jiawei Tang, Yuheng Jia, Boyu Shi, and Xin Geng. 2024. Exploiting multi-label correlation in label distribution learning. In Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence, pages 4326– 4334. [find](https://scholar.google.com/scholar?q=Zhiqiang+Kou%2C+Jing+Wang%2C+Jiawei+Tang%2C+Yuheng+Jia%2C+Boyu+Shi%2C+and+Xin+Geng%2E+2024%2E+Exploiting+multi%2Dlabel+correlation+in+label+distribution+learning%2E+In+Proceedings+of+the+Thirty%2DThird+International+Join)
26. Lingjuan Lyu, Han Yu, Xingjun Ma, Chen Chen, Lichao Sun, Jun Zhao, Qiang Yang, and Philip S Yu. 2022. Privacy and robustness in federated learning: Attacks and defenses. IEEE transactions on neural networks and learning systems, 35(7):8726–8746. [find](https://scholar.google.com/scholar?q=Lingjuan+Lyu%2C+Han+Yu%2C+Xingjun+Ma%2C+Chen+Chen%2C+Lichao+Sun%2C+Jun+Zhao%2C+Qiang+Yang%2C+and+Philip+S+Yu%2E+2022%2E+Privacy+and+robustness+in+federated+learning%3A+Attacks+and+defenses%2E+IEEE+transactions+on+neural+ne)
27. Brendan McMahan, Eider Moore, Daniel Ramage, Seth Hampson, and Blaise Aguera y Arcas. 2017. Communication-efficient learning of deep networks from decentralized data. In Artificial intelligence and statistics, pages 1273–1282. PMLR. [find](https://scholar.google.com/scholar?q=Brendan+McMahan%2C+Eider+Moore%2C+Daniel+Ramage%2C+Seth+Hampson%2C+and+Blaise+Aguera+y+Arcas%2E+2017%2E+Communication%2Defficient+learning+of+deep+networks+from+decentralized+data%2E+In+Artificial+intelligence+and+st)
28. Xin Men, Mingyu Xu, Qingyu Zhang, Qianhao Yuan, Bingning Wang, Hongyu Lin, Yaojie Lu, Xianpei Han, and Weipeng Chen. 2025. [Shortgpt: Layers in large language models are more redundant than you expect](https://md.research.elamur.ai/free/en/papers/men-2024-shortgpt-layers-large-language-models/). In Findings of the Association for Computational Linguistics: ACL 2025, pages 20192–20204.
29. Stephen Merity, Caiming Xiong, James Bradbury, and Richard Socher. 2016. [Pointer sentinel mixture models](https://md.research.elamur.ai/free/en/papers/merity-2016-pointer-sentinel-mixture-models/). Preprint, arXiv:1609.07843.
30. Todor Mihaylov, Peter Clark, Tushar Khot, and Ashish Sabharwal. 2018. Can a suit of armor conduct electricity? a new dataset for open book question answering. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 2381–2391. [find](https://scholar.google.com/scholar?q=Todor+Mihaylov%2C+Peter+Clark%2C+Tushar+Khot%2C+and+Ashish+Sabharwal%2E+2018%2E+Can+a+suit+of+armor+conduct+electricity%3F+a+new+dataset+for+open+book+question+answering%2E+In+Proceedings+of+the+2018+Conference+on+)
31. Alessio Mora, Irene Tenison, Paolo Bellavista, and Irina Rish. 2024. Knowledge distillation in federated learning: a practical guide. In Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence, pages 8188–8196. [find](https://scholar.google.com/scholar?q=Alessio+Mora%2C+Irene+Tenison%2C+Paolo+Bellavista%2C+and+Irina+Rish%2E+2024%2E+Knowledge+distillation+in+federated+learning%3A+a+practical+guide%2E+In+Proceedings+of+the+Thirty%2DThird+International+Joint+Conference+)
32. An Yang, Baosong Yang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Zhou, Chengpeng Li, Chengyuan Li, Dayiheng Liu, Fei Huang, and 1 others. 2024. Qwen2 technical report. arXiv preprint arXiv:2407.10671. [find](https://scholar.google.com/scholar?q=An+Yang%2C+Baosong+Yang%2C+Binyuan+Hui%2C+Bo+Zheng%2C+Bowen+Yu%2C+Chang+Zhou%2C+Chengpeng+Li%2C+Chengyuan+Li%2C+Dayiheng+Liu%2C+Fei+Huang%2C+and+1+others%2E+2024%2E+Qwen2+technical+report%2E+arXiv+preprint+arXiv%3A2407%2E10671%2E)
33. Niklas Muennighoff, Thomas Wang, Lintang Sutawika, Adam Roberts, Stella Biderman, Teven Le Scao, M Saiful Bari, Sheng Shen, Zheng-Xin Yong, Hailey Schoelkopf, and 1 others. 2023. [Crosslingual generalization through multitask finetuning](https://md.research.elamur.ai/free/en/papers/muennighoff-2022-crosslingual-generalization-through-multitask-finetuning/). In Proceedings of the 61st Annual Meeting of the Association fo…
34. Kai Yao, Zhaorui Tan, Penglei Gao, Lichun Li, Kaixin Wu, Yinggui Wang, Yuan Zhao, Yixin Ji, Jianke Zhu, and Wei Wang. 2025. Gradot: Training-free gradient-preserving offsite-tuning for large language models. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (… [find](https://scholar.google.com/scholar?q=Kai+Yao%2C+Zhaorui+Tan%2C+Penglei+Gao%2C+Lichun+Li%2C+Kaixin+Wu%2C+Yinggui+Wang%2C+Yuan+Zhao%2C+Yixin+Ji%2C+Jianke+Zhu%2C+and+Wei+Wang%2E+2025%2E+Gradot%3A+Training%2Dfree+gradient%2Dpreserving+offsite%2Dtuning+for+large+language+)
35. Dinh C Nguyen, Quoc-Viet Pham, Pubudu N Pathirana, Ming Ding, Aruna Seneviratne, Zihuai Lin, Octavia Dobre, and Won-Joo Hwang. 2022. Federated learning for smart healthcare: A survey. ACM Computing Surveys (Csur), 55(3):1–37. [find](https://scholar.google.com/scholar?q=Dinh+C+Nguyen%2C+Quoc%2DViet+Pham%2C+Pubudu+N+Pathirana%2C+Ming+Ding%2C+Aruna+Seneviratne%2C+Zihuai+Lin%2C+Octavia+Dobre%2C+and+Won%2DJoo+Hwang%2E+2022%2E+Federated+learning+for+smart+healthcare%3A+A+survey%2E+ACM+Computing+Su)
36. Adil Oualid, Youssef Qasmaoui, Youssef Balouki, and Lahcen Moumoun. 2025. Federated learning and open banking for inclusive credit scoring in morocco: A systematic review. In International Conference on intelligent systems and digital applications, pages 242–256. Springer. [find](https://scholar.google.com/scholar?q=Adil+Oualid%2C+Youssef+Qasmaoui%2C+Youssef+Balouki%2C+and+Lahcen+Moumoun%2E+2025%2E+Federated+learning+and+open+banking+for+inclusive+credit+scoring+in+morocco%3A+A+systematic+review%2E+In+International+Conference+)
37. Lang Yu, Qin Chen, Jiaju Lin, and Liang He. 2023. Black-box prompt tuning for vision-language model as a service. In IJCAI, pages 1686–1694. [find](https://scholar.google.com/scholar?q=Lang+Yu%2C+Qin+Chen%2C+Jiaju+Lin%2C+and+Liang+He%2E+2023%2E+Black%2Dbox+prompt+tuning+for+vision%2Dlanguage+model+as+a+service%2E+In+IJCAI%2C+pages+1686%E2%80%931694%2E)
38. Jianyi Zhang, Saeed Vahidian, Martin Kuo, Chunyuan Li, Ruiyi Zhang, Tong Yu, Guoyin Wang, and Yiran Chen. 2024. Towards building the federatedgpt: Federated instruction tuning. In ICASSP 2024-2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 6915–6919. IEE… [find](https://scholar.google.com/scholar?q=Jianyi+Zhang%2C+Saeed+Vahidian%2C+Martin+Kuo%2C+Chunyuan+Li%2C+Ruiyi+Zhang%2C+Tong+Yu%2C+Guoyin+Wang%2C+and+Yiran+Chen%2E+2024%2E+Towards+building+the+federatedgpt%3A+Federated+instruction+tuning%2E+In+ICASSP+2024%2D2024+IEE)
39. Hassan Sajjad, Fahim Dalvi, Nadir Durrani, and Preslav Nakov. 2023. On the effect of dropping layers of pre-trained transformer models. Computer Speech & Language, 77:101429. [find](https://scholar.google.com/scholar?q=Hassan+Sajjad%2C+Fahim+Dalvi%2C+Nadir+Durrani%2C+and+Preslav+Nakov%2E+2023%2E+On+the+effect+of+dropping+layers+of+pre%2Dtrained+transformer+models%2E+Computer+Speech+%26+Language%2C+77%3A101429%2E)
40. Kaiyan Zhang, Ning Ding, Biqing Qi, Xuekai Zhu, Xinwei Long, and Bowen Zhou. 2023. Crash: Clustering, removing, and sharing enhance fine-tuning without full large language model. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 9612–9637. [find](https://scholar.google.com/scholar?q=Kaiyan+Zhang%2C+Ning+Ding%2C+Biqing+Qi%2C+Xuekai+Zhu%2C+Xinwei+Long%2C+and+Bowen+Zhou%2E+2023%2E+Crash%3A+Clustering%2C+removing%2C+and+sharing+enhance+fine%2Dtuning+without+full+large+language+model%2E+In+Proceedings+of+the)
41. Maarten Sap, Hannah Rashkin, Derek Chen, Ronan LeBras, and Yejin Choi. 2019. [Socialiqa: Commonsense reasoning about social interactions](https://md.research.elamur.ai/free/en/papers/sap-2019-socialiqa-commonsense-reasoning-about-social/). arXiv preprint arXiv:1904.09728.
42. Yuanhang Zheng, Zhixing Tan, Peng Li, and Yang Liu. 2024. Black-box prompt tuning with subspace learning. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 32:3002–3013. [find](https://scholar.google.com/scholar?q=Yuanhang+Zheng%2C+Zhixing+Tan%2C+Peng+Li%2C+and+Yang+Liu%2E+2024%2E+Black%2Dbox+prompt+tuning+with+subspace+learning%2E+IEEE%2FACM+Transactions+on+Audio%2C+Speech%2C+and+Language+Processing%2C+32%3A3002%E2%80%933013%2E)
43. Hongrui Shi and Valentin Radu. 2021. Towards federated learning with attention transfer to mitigate system and data heterogeneity of clients. In Proceedings of the 4th international workshop on edge systems, analytics and networking, pages 61–66. [find](https://scholar.google.com/scholar?q=Hongrui+Shi+and+Valentin+Radu%2E+2021%2E+Towards+federated+learning+with+attention+transfer+to+mitigate+system+and+data+heterogeneity+of+clients%2E+In+Proceedings+of+the+4th+international+workshop+on+edge+s)
44. Longguang Zhong, Fanqi Wan, Ruijun Chen, Xiaojun Quan, and Liangzhi Li. 2025. Blockpruner: Fine-grained pruning for large language models. In Findings of the Association for Computational Linguistics: ACL 2025, pages 5065–5080. [find](https://scholar.google.com/scholar?q=Longguang+Zhong%2C+Fanqi+Wan%2C+Ruijun+Chen%2C+Xiaojun+Quan%2C+and+Liangzhi+Li%2E+2025%2E+Blockpruner%3A+Fine%2Dgrained+pruning+for+large+language+models%2E+In+Findings+of+the+Association+for+Computational+Linguistics%3A)
45. Gemma Team, Morgane Riviere, Shreya Pathak, Pier Giuseppe Sessa, Cassidy Hardin, Surya Bhupatiraju, Léonard Hussenot, Thomas Mesnard, Bobak Shahriari, Alexandre Ramé, and 1 others. 2024. Gemma 2: Improving open language models at a practical size. arXiv preprint arXiv:2408.00118. [find](https://scholar.google.com/scholar?q=Gemma+Team%2C+Morgane+Riviere%2C+Shreya+Pathak%2C+Pier+Giuseppe+Sessa%2C+Cassidy+Hardin%2C+Surya+Bhupatiraju%2C+L%C3%A9onard+Hussenot%2C+Thomas+Mesnard%2C+Bobak+Shahriari%2C+Alexandre+Ram%C3%A9%2C+and+1+others%2E+2024%2E+Gemma+2%3A+Impr)
46. Ligeng Zhu, Zhijian Liu, and Song Han. 2019. Deep leakage from gradients. Advances in neural information processing systems, 32. [find](https://scholar.google.com/scholar?q=Ligeng+Zhu%2C+Zhijian+Liu%2C+and+Song+Han%2E+2019%2E+Deep+leakage+from+gradients%2E+Advances+in+neural+information+processing+systems%2C+32%2E)
47. Jason Wei, Maarten Bosma, Vincent Zhao, Kelvin Guu, Adams Wei Yu, Brian Lester, Nan Du, Andrew M Dai, and Quoc V Le. 2023. [Finetuned language models are zero-shot learners](https://md.research.elamur.ai/free/en/papers/wei-2021-finetuned-language-models-zero-shot/). In International Conference on Learning Representations.
48. Guangxuan Xiao, Ji Lin, and Song Han. 2023. Offsite-tuning: Transfer learning without full model. arXiv preprint arXiv:2302.04870. — [arXiv:2302.04870](https://arxiv.org/abs/2302.04870) [find](https://scholar.google.com/scholar?q=Guangxuan+Xiao%2C+Ji+Lin%2C+and+Song+Han%2E+2023%2E+Offsite%2Dtuning%3A+Transfer+learning+without+full+model%2E+arXiv+preprint+arXiv%3A2302%2E04870%2E+%E2%80%94+%5BarXiv%3A2302%2E04870%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2302%2E04870%29)
49. An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, and 1 others. 2025. [Qwen3 technical report](https://md.research.elamur.ai/free/en/papers/yang-2025-qwen3-technical-report/). arXiv preprint arXiv:2505.09388.

*Updated 2 November 2026*