---
title: "Memory-Efficient Structured Backpropagation for On-Device LLM"
description: "The authors propose MeSP (Memory-efficient Structured Backpropagation). 15 references."
authors: ["JuneYoung Park", "Yuri Hong", "Seongwan Kim", "Jaeho Lee"]
year: 2026
venue: "Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 6: Industry Track)"
source: "https://aclanthology.org/2026.acl-industry.62/"
doi: "10.18653/v1/2026.acl-industry.62"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/park-2026-memory-efficient-structured-backpropagation-device/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/park-2026-memory-efficient-structured-backpropagation-device/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/36/) / Memory-Efficient Structured Backpropagation for On-Device LLM Fine-Tuning

# Memory-Efficient Structured Backpropagation for On-Device LLM Fine-Tuning

**Authors:** JuneYoung Park, Yuri Hong, Seongwan Kim and [Jaeho Lee](https://md.research.elamur.ai/free/en/authors/jaeho-lee/) · **Year:** 2026 · **Venue:** [Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 6: Industry Track)](https://md.research.elamur.ai/free/en/venues/acl/) ([ACL 2026](https://md.research.elamur.ai/free/en/venues/acl/2026/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2026.acl-industry.62/) · [PDF](https://aclanthology.org/2026.acl-industry.62.pdf) · [DOI 10.18653/v1/2026.acl-industry.62](https://doi.org/10.18653/v1/2026.acl-industry.62) · [OpenAlex](https://openalex.org/W7166831151)

**Citations:** 0 citations (OpenAlex, 2026-10-14)

## Summary
The paper targets a practical obstacle to personalizing language models on phones and laptops: even with LoRA, the activations kept for backpropagation can exceed what a device can spare. Current options force a compromise. MeBP gives exact gradients but relies on automatic differentiation, which keeps more tensors than needed. MeZO uses little memory, but its gradients are noisy. The authors propose MeSP (Memory-efficient Structured Backpropagation). It uses hand-written backward passes for each transformer component, keeps only block outputs during the forward pass, and recomputes intermediates one layer at a time. The key trick is that the LoRA projection h = xA is very narrow (rank 8–32), so recomputing it is cheaper than caching it. Experiments use Qwen2.5 models (0.5B–3B) on WikiText-2, implemented in MLX on Apple Silicon. MeSP produces the same gradients as MeBP while using 42–62% less peak memory, at roughly 28% extra compute time. This makes exact first-order fine-tuning feasible in memory budgets where it was previously impractical.

## Key points
- **Peak memory at sequence length 256:** on Qwen2.5-0.5B, MeSP drops peak memory from 360.8 MB (MeBP) to 136.2 MB, a 62% cut. The savings shrink as models grow, falling to 49% for 1.5B and 42% for 3B, because weights dominate memory in larger models.
- **Scaling with sequence length and rank:** MeSP saves 51–62% across sequence lengths, while MeBP grows from about 253 MB to 1050 MB. Savings stay near 60–63% across LoRA ranks. By contrast, MeZO actually uses more memory than MeBP at rank 32.
- **Training quality:** over 100K steps, MeSP and MeBP reach the same final loss (about 2.6). MeZO ends roughly 22% higher (about 3.18).
- **Why MeZO is slow:** its gradient estimates barely align with the true gradients, with cosine similarity of about 0.001 and sign agreement near 50%.
- **Ablation and code:** on Qwen2.5-3B, recomputing h instead of storing it saves an extra 7.6% memory for 6.2% more time. The code is at github.com/crinex/acl-mesp.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

📚 Shelves (5): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works · [👁️ Computer Vision and Pattern Recognition](https://md.research.elamur.ai/free/en/shelves/computer-science/computer-vision-and-pattern-recognition/) · 2,869 works

🧭 Topics (2): [🧠 Language models & pretraining](https://md.research.elamur.ai/free/en/topics/language-models/) · 3,320 works · [🪶 Efficient models & compression](https://md.research.elamur.ai/free/en/topics/efficient-models/) · 242 works

## Cite

Cite as: JuneYoung Park, Yuri Hong, Seongwan Kim and Jaeho Lee. 2026. Memory-Efficient Structured Backpropagation for On-Device LLM Fine-Tuning. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 6: Industry Track).

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{park2026memory,
  title = {{Memory-Efficient Structured Backpropagation for On-Device LLM Fine-Tuning}},
  author = {JuneYoung Park, Yuri Hong, Seongwan Kim and Jaeho Lee},
  booktitle = {Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 6: Industry Track)},
  year = {2026},
  doi = {10.18653/v1/2026.acl-industry.62},
  url = {https://doi.org/10.18653/v1/2026.acl-industry.62}
}
```

- [← previous Memory-Driven Role-Playing: Evaluation and Enhancement of Persona Knowledge Utilization in LLMs](https://md.research.elamur.ai/free/en/papers/wang-2026-memory-driven-role-playing-evaluation/)
- [next → Memory-Guided Hard Data Augmentation for Multimodal Named Entity Recognition](https://md.research.elamur.ai/free/en/papers/liu-2026-memory-guided-hard-data-augmentation/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/36/)

## References

15 references, 5 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Apple Machine Learning Research. 2023. MLX: An efficient machine learning framework for apple silicon. [https://github.com/ml-explore/mlx](https://github.com/ml-explore/mlx). [↗](https://github.com/ml-explore/mlx)
2. In Advances in Neural Information Processing Systems. [find](https://scholar.google.com/scholar?q=In+Advances+in+Neural+Information+Processing+Systems%2E)
3. Tim Dettmers, Artidoro Pagnoni, Ari Holtzman, and Luke Zettlemoyer. 2023. [QLoRA: Efficient finetuning of quantized LLMs](https://md.research.elamur.ai/free/en/papers/dettmers-2023-qlora-efficient-finetuning-quantized-llms/). In Advances in Neural Information Processing Systems.
4. Neil Houlsby, Andrei Giurgiu, Stanislaw Jastrzebski, Bruna Morrone, Quentin De Laroussilhe, Andrea Gesmundo, Mona Attariyan, and Sylvain Gelly. 2019. Parameter-efficient transfer learning for NLP. In International Conference on Machine Learning. [find](https://scholar.google.com/scholar?q=Neil+Houlsby%2C+Andrei+Giurgiu%2C+Stanislaw+Jastrzebski%2C+Bruna+Morrone%2C+Quentin+De+Laroussilhe%2C+Andrea+Gesmundo%2C+Mona+Attariyan%2C+and+Sylvain+Gelly%2E+2019%2E+Parameter%2Defficient+transfer+learning+for+NLP%2E+In+)
5. Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2022. [LoRA: Low-rank adaptation of large language models](https://md.research.elamur.ai/free/en/papers/hu-2021-lora-low-rank-adaptation-large/). In International Conference on Learning Representations.
6. Ji Lin, Ligeng Zhu, Wei-Ming Chen, Wei-Chen Wang, Chuang Gan, and Song Han. 2022. On-device training under 256kb memory. In Advances in Neural Information Processing Systems. [find](https://scholar.google.com/scholar?q=Ji+Lin%2C+Ligeng+Zhu%2C+Wei%2DMing+Chen%2C+Wei%2DChen+Wang%2C+Chuang+Gan%2C+and+Song+Han%2E+2022%2E+On%2Ddevice+training+under+256kb+memory%2E+In+Advances+in+Neural+Information+Processing+Systems%2E)
7. Sadhika Malladi, Tianyu Gao, Eshaan Nichani, Alex Damian, Jason D Lee, Danqi Chen, and Sanjeev Arora. 2023. Fine-tuning language models with just forward passes. In Advances in Neural Information Processing Systems. [find](https://scholar.google.com/scholar?q=Sadhika+Malladi%2C+Tianyu+Gao%2C+Eshaan+Nichani%2C+Alex+Damian%2C+Jason+D+Lee%2C+Danqi+Chen%2C+and+Sanjeev+Arora%2E+2023%2E+Fine%2Dtuning+language+models+with+just+forward+passes%2E+In+Advances+in+Neural+Information+Proc)
8. Stephen Merity, Caiming Xiong, James Bradbury, and Richard Socher. 2016. [Pointer sentinel mixture models](https://md.research.elamur.ai/free/en/papers/merity-2016-pointer-sentinel-mixture-models/). arXiv preprint arXiv:1609.07843.
9. Qwen Team. 2024. Qwen2.5 technical report. arXiv preprint arXiv:2412.15115. [find](https://scholar.google.com/scholar?q=Qwen+Team%2E+2024%2E+Qwen2%2E5+technical+report%2E+arXiv+preprint+arXiv%3A2412%2E15115%2E)
10. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. [Attention is all you need](https://md.research.elamur.ai/free/en/papers/vaswani-2017-attention-all-need/). In Advances in Neural Information Processing Systems.
11. Yue Xu, Yuxin Chen, Ligeng Zhu, Ji Lin, and Song Han. 2023. PockEngine: Sparse and efficient fine-tuning in a pocket. arXiv preprint arXiv:2310.17752. [find](https://scholar.google.com/scholar?q=Yue+Xu%2C+Yuxin+Chen%2C+Ligeng+Zhu%2C+Ji+Lin%2C+and+Song+Han%2E+2023%2E+PockEngine%3A+Sparse+and+efficient+fine%2Dtuning+in+a+pocket%2E+arXiv+preprint+arXiv%3A2310%2E17752%2E)
12. Apple Machine Learning Research. 2024. Memory-efficient backpropagation for on-device LLM fine-tuning. [https://github.com/apple/](https://github.com/apple/) ml-mebp. [↗](https://github.com/apple/)
13. Tianqi Chen, Bing Xu, Chiyuan Zhang, and Carlos Guestrin. 2016. Training deep nets with sublinear memory cost. arXiv preprint arXiv:1604.06174. [find](https://scholar.google.com/scholar?q=Tianqi+Chen%2C+Bing+Xu%2C+Chiyuan+Zhang%2C+and+Carlos+Guestrin%2E+2016%2E+Training+deep+nets+with+sublinear+memory+cost%2E+arXiv+preprint+arXiv%3A1604%2E06174%2E)
14. Tri Dao. 2023. [FlashAttention-2: Faster attention with better parallelism and work partitioning](https://md.research.elamur.ai/free/en/papers/dao-2023-flashattention-2-faster-attention-better/). arXiv preprint arXiv:2307.08691.
15. Tri Dao, Dan Fu, Stefano Ermon, Atri Rudra, and Christopher Ré. 2022. FlashAttention: Fast and memory-efficient exact attention with IO-awareness. [find](https://scholar.google.com/scholar?q=Tri+Dao%2C+Dan+Fu%2C+Stefano+Ermon%2C+Atri+Rudra%2C+and+Christopher+R%C3%A9%2E+2022%2E+FlashAttention%3A+Fast+and+memory%2Defficient+exact+attention+with+IO%2Dawareness%2E)

*Updated 1 November 2026*