---
title: "Representation Interventions Enable Lifelong Knowledge Memory"
description: "The authors propose RILKE, which keeps the base model frozen and instead edits hidden states… 54 references, cited by 1 paper here."
authors: ["Xuyuan Liu", "Shengyu Chen", "Xinshuai Dong", "Yanchi Liu", "Xujiang Zhao", "Haoyu Wang", "Yujun Yan", "Haifeng Chen", "Zhengzhang Chen"]
year: 2026
venue: "Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)"
source: "https://aclanthology.org/2026.acl-long.246/"
doi: "10.18653/v1/2026.acl-long.246"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/liu-2026-representation-interventions-enable-lifelong-knowledge/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/liu-2026-representation-interventions-enable-lifelong-knowledge/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/16/) / Representation Interventions Enable Lifelong Knowledge Memory Control in LLMs

# Representation Interventions Enable Lifelong Knowledge Memory Control in LLMs

**Authors:** Xuyuan Liu, Shengyu Chen, [Xinshuai Dong](https://md.research.elamur.ai/free/en/authors/xinshuai-dong/), [Yanchi Liu](https://md.research.elamur.ai/free/en/authors/yanchi-liu/), [Xujiang Zhao](https://md.research.elamur.ai/free/en/authors/xujiang-zhao/), [Haoyu Wang](https://md.research.elamur.ai/free/en/authors/haoyu-wang/), Yujun Yan, [Haifeng Chen](https://md.research.elamur.ai/free/en/authors/haifeng-chen/) and [Zhengzhang Chen](https://md.research.elamur.ai/free/en/authors/zhengzhang-chen/) · **Year:** 2026 · **Venue:** [Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)](https://md.research.elamur.ai/free/en/venues/acl/) ([ACL 2026](https://md.research.elamur.ai/free/en/venues/acl/2026/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2026.acl-long.246/) · [PDF](https://aclanthology.org/2026.acl-long.246.pdf) · [DOI 10.18653/v1/2026.acl-long.246](https://doi.org/10.18653/v1/2026.acl-long.246) · [OpenAlex](https://openalex.org/W4416779194)

**Citations:** 0 citations (OpenAlex, 2026-10-14) · [1](https://md.research.elamur.ai/free/en/papers/liu-2026-representation-interventions-enable-lifelong-knowledge/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
Once deployed, LLMs cannot easily absorb new or corrected facts. Retraining is expensive. Weight-editing methods tend to collapse as edits pile up, especially with long, free-form knowledge rather than subject–relation–object triples. The authors propose RILKE, which keeps the base model frozen and instead edits hidden states through ReFT-style low-rank intervention modules.

The approach rests on two empirically checked observations:
- **Semantic locality:** paraphrases of a query stay close together in representation space.
- **Subspace alignment:** semantically related edits learn similar intervention subspaces.

RILKE builds on these in three ways:
- It trains each module to be robust to paraphrases and confined to a small subspace.
- It stores the original representation as a key, so a query-adaptive router can activate only the matching module at inference.
- It clusters similar knowledge into shared-subspace modules to save memory.

Experiments on LLaMA and Qwen models with large unstructured-editing benchmarks report high edit success, good paraphrase transfer and preserved general ability, at modest memory cost. This makes representation-space intervention a practical route to lifelong knowledge updating.

## Key points
- RILKE stands for Representation Intervention for Lifelong KnowledgE Control. It builds on ReFT (Wu et al., 2024b), uses the last-token hidden state as the query index, and leaves base weights untouched.
- **Subspace alignment evidence:** the cosine similarity of learned projection matrices R^l falls as representation similarity drops. It is about 0.751 for pairs above 0.9 similarity and about 0.586 for pairs below 0.6.
- **Sequential setting (Table 5):** 10 clusters were drawn from the UnKE dataset, averaging 13 items each. Results are BERTScore on original and paraphrased queries:

  | Method | Original | Paraphrased |
  |---|---|---|
  | Batched RILKE | 1.000 | 0.834 |
  | Sequential RILKE | 0.742 | 0.723 |
  | UnKE | 0.603 | 0.572 |
  | AnyEdit | 0.274 | 0.262 |

- **Shared-subspace modules:** grouping semantically similar edits into one module cuts memory. Batching dissimilar items, by contrast, harms edit specificity.
- **Limitations:** the authors defer risk analysis to future work, including malicious edits and bias propagation.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Carnegie Mellon University](https://md.research.elamur.ai/free/en/universities/carnegie-mellon-university/) — University, Pittsburgh, United States · 598 works · 191,226 citations
- [Dartmouth College](https://md.research.elamur.ai/free/en/universities/dartmouth-college/) — University, Hanover, United States · 41 works · 42,719 citations
- [NEC (United States)](https://md.research.elamur.ai/free/en/companies/nec/) — Company, Irving, United States · 10 works · 658 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 18,614 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 17,965 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 17,450 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 17,296 works

🧭 Topics (1): [🧠 Language models & pretraining](https://md.research.elamur.ai/free/en/topics/language-models/) · 3,894 works

## Cite

Cite as: Xuyuan Liu, Shengyu Chen, Xinshuai Dong, Yanchi Liu, Xujiang Zhao, Haoyu Wang, Yujun Yan, Haifeng Chen and Zhengzhang Chen. 2026. Representation Interventions Enable Lifelong Knowledge Memory Control in LLMs. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{liu2026representation,
  title = {{Representation Interventions Enable Lifelong Knowledge Memory Control in LLMs}},
  author = {Xuyuan Liu, Shengyu Chen, Xinshuai Dong, Yanchi Liu, Xujiang Zhao, Haoyu Wang, Yujun Yan, Haifeng Chen and Zhengzhang Chen},
  booktitle = {Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2026},
  doi = {10.18653/v1/2026.acl-long.246},
  url = {https://doi.org/10.18653/v1/2026.acl-long.246}
}
```

- [← previous ReportLogic: Evaluating Logical Quality in Deep Research Reports](https://md.research.elamur.ai/free/en/papers/zhao-2026-reportlogic-evaluating-logical-quality-deep/)
- [next → Representation-Aware Prompting for Zero-Shot Marathi Text Classification: IPA, Romanization, Repetition](https://md.research.elamur.ai/free/en/papers/tran-2026-representation-aware-prompting-zero-shot/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/16/)

## References

54 references, 16 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Andy Arditi, Oscar Obeso, Aaquib Syed, Daniel Paleka, Nina Panickssery, Wes Gurnee, and Neel Nanda. 2024. [Refusal in language models is mediated by a single direction](https://md.research.elamur.ai/free/en/papers/arditi-2024-refusal-language-models-mediated-single/). In Advances in Neural Information Processing Systems 38: Annual Conference on Neural Information Processing Systems 2024, NeurIPS 20…
2. Qin Chen, Yuanyi Ren, Xiaojun Ma, Mugeng Liu, Shi Han, and Dongmei Zhang. 2025a. SheetDesigner: MLLM-powered spreadsheet layout generation with rule-based and vision-based reflection. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pages 18921–18939, Suzhou… [find](https://scholar.google.com/scholar?q=Qin+Chen%2C+Yuanyi+Ren%2C+Xiaojun+Ma%2C+Mugeng+Liu%2C+Shi+Han%2C+and+Dongmei+Zhang%2E+2025a%2E+SheetDesigner%3A+MLLM%2Dpowered+spreadsheet+layout+generation+with+rule%2Dbased+and+vision%2Dbased+reflection%2E+In+Proceedings+o)
3. Qin Chen, Yuanyi Ren, Xiaojun Ma, and Yuyang Shi. 2025b. Large language models for predictive analysis: How far are they? In Findings of the Association for Computational Linguistics: ACL 2025, pages 7961–7978, Vienna, Austria. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Qin+Chen%2C+Yuanyi+Ren%2C+Xiaojun+Ma%2C+and+Yuyang+Shi%2E+2025b%2E+Large+language+models+for+predictive+analysis%3A+How+far+are+they%3F+In+Findings+of+the+Association+for+Computational+Linguistics%3A+ACL+2025%2C+pages+)
4. Qizhou Chen, Taolin Zhang, Xiaofeng He, Dongyang Li, Chengyu Wang, Longtao Huang, and Hui Xue’. 2024. Lifelong knowledge editing for LLMs with retrieval-augmented continuous prompt learning. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 13565–13580,… [find](https://scholar.google.com/scholar?q=Qizhou+Chen%2C+Taolin+Zhang%2C+Xiaofeng+He%2C+Dongyang+Li%2C+Chengyu+Wang%2C+Longtao+Huang%2C+and+Hui+Xue%E2%80%99%2E+2024%2E+Lifelong+knowledge+editing+for+LLMs+with+retrieval%2Daugmented+continuous+prompt+learning%2E+In+Procee)
5. Runjin Chen, Andy Arditi, Henry Sleight, Owain Evans, and Jack Lindsey. 2025c. Persona vectors: Monitoring and controlling character traits in language models. Preprint, arXiv:2507.21509. — [arXiv:2507.21509](https://arxiv.org/abs/2507.21509) [find](https://scholar.google.com/scholar?q=Runjin+Chen%2C+Andy+Arditi%2C+Henry+Sleight%2C+Owain+Evans%2C+and+Jack+Lindsey%2E+2025c%2E+Persona+vectors%3A+Monitoring+and+controlling+character+traits+in+language+models%2E+Preprint%2C+arXiv%3A2507%2E21509%2E+%E2%80%94+%5BarXiv%3A250)
6. Runjin Chen, Zhenyu Zhang, Junyuan Hong, Souvik Kundu, and Zhangyang Wang. 2025d. Seal: Steerable reasoning calibration of large language models for free. arXiv preprint arXiv:2504.07986. — [arXiv:2504.07986](https://arxiv.org/abs/2504.07986) [find](https://scholar.google.com/scholar?q=Runjin+Chen%2C+Zhenyu+Zhang%2C+Junyuan+Hong%2C+Souvik+Kundu%2C+and+Zhangyang+Wang%2E+2025d%2E+Seal%3A+Steerable+reasoning+calibration+of+large+language+models+for+free%2E+arXiv+preprint+arXiv%3A2504%2E07986%2E+%E2%80%94+%5BarXiv%3A250)
7. YuJu Cheng, Yu-Chu Yu, Kai-Po Chang, and Yu-Chiang Frank Wang. 2025. Serial lifelong editing via mixture of knowledge experts. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 30888–30903, Vienna, Austria. Association for Compu… [find](https://scholar.google.com/scholar?q=YuJu+Cheng%2C+Yu%2DChu+Yu%2C+Kai%2DPo+Chang%2C+and+Yu%2DChiang+Frank+Wang%2E+2025%2E+Serial+lifelong+editing+via+mixture+of+knowledge+experts%2E+In+Proceedings+of+the+63rd+Annual+Meeting+of+the+Association+for+Computat)
8. Roi Cohen, Eden Biran, Ori Yoran, Amir Globerson, and Mor Geva. 2024. Evaluating the ripple effects of knowledge editing in language models. Trans. Assoc. Comput. Linguistics, 12:283–298. [find](https://scholar.google.com/scholar?q=Roi+Cohen%2C+Eden+Biran%2C+Ori+Yoran%2C+Amir+Globerson%2C+and+Mor+Geva%2E+2024%2E+Evaluating+the+ripple+effects+of+knowledge+editing+in+language+models%2E+Trans%2E+Assoc%2E+Comput%2E+Linguistics%2C+12%3A283%E2%80%93298%2E)
9. Jingcheng Deng, Zihao Wei, Liang Pang, Hanxing Ding, Huawei Shen, and Xueqi Cheng. 2025. Everything is editable: Extend knowledge editing to unstructured data in large language models. In The Thirteenth International Conference on Learning Representations, ICLR 2025, Singapore, April 24-28, 2025. Op… [find](https://scholar.google.com/scholar?q=Jingcheng+Deng%2C+Zihao+Wei%2C+Liang+Pang%2C+Hanxing+Ding%2C+Huawei+Shen%2C+and+Xueqi+Cheng%2E+2025%2E+Everything+is+editable%3A+Extend+knowledge+editing+to+unstructured+data+in+large+language+models%2E+In+The+Thirteen)
10. Nelson Elhage, Neel Nanda, Catherine Olsson, Tom Henighan, Nicholas Joseph, Ben Mann, Amanda Askell, Yuntao Bai, Anna Chen, Tom Conerly, Nova DasSarma, Dawn Drain, Deep Ganguli, Zac Hatfield-Dodds, Danny Hernandez, Andy Jones, Jackson Kernion, Liane Lovitt, Kamal Ndousse, Dario Amodei, Tom Brown, Ja… [find](https://scholar.google.com/scholar?q=Nelson+Elhage%2C+Neel+Nanda%2C+Catherine+Olsson%2C+Tom+Henighan%2C+Nicholas+Joseph%2C+Ben+Mann%2C+Amanda+Askell%2C+Yuntao+Bai%2C+Anna+Chen%2C+Tom+Conerly%2C+Nova+DasSarma%2C+Dawn+Drain%2C+Deep+Ganguli%2C+Zac+Hatfield%2DDodds%2C+Da)
11. Junfeng Fang, Houcheng Jiang, Kun Wang, Yunshan Ma, Jie Shi, Xiang Wang, Xiangnan He, and Tat-Seng Chua. 2025. Alphaedit: Null-space constrained knowledge editing for language models. In The Thirteenth International Conference on Learning Representations, ICLR 2025, Singapore, April 24-28, 2025. Ope… [find](https://scholar.google.com/scholar?q=Junfeng+Fang%2C+Houcheng+Jiang%2C+Kun+Wang%2C+Yunshan+Ma%2C+Jie+Shi%2C+Xiang+Wang%2C+Xiangnan+He%2C+and+Tat%2DSeng+Chua%2E+2025%2E+Alphaedit%3A+Null%2Dspace+constrained+knowledge+editing+for+language+models%2E+In+The+Thirteent)
12. Mor Geva, Roei Schuster, Jonathan Berant, and Omer Levy. 2021. Transformer feed-forward layers are key-value memories. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 5484–5495, Online and Punta Cana, Dominican Republic. Association for Computational… [find](https://scholar.google.com/scholar?q=Mor+Geva%2C+Roei+Schuster%2C+Jonathan+Berant%2C+and+Omer+Levy%2E+2021%2E+Transformer+feed%2Dforward+layers+are+key%2Dvalue+memories%2E+In+Proceedings+of+the+2021+Conference+on+Empirical+Methods+in+Natural+Language+Pr)
13. Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, et al. 2024. [The llama 3 herd of models](https://md.research.elamur.ai/free/en/papers/grattafiori-2024-llama-3-herd-models/). arXiv preprint arXiv:2407.21783.
14. Bernal Jiménez Gutiérrez, Yiheng Shu, Weijian Qi, Sizhe Zhou, and Yu Su. 2025. [From rag to memory: Non-parametric continual learning for large language models](https://md.research.elamur.ai/free/en/papers/gutierrez-2025-rag-memory-non-parametric-continual/). arXiv preprint arXiv:2502.14802.
15. Chi Han, Jialiang Xu, Manling Li, Yi Fung, Chenkai Sun, Nan Jiang, Tarek F. Abdelzaher, and Heng Ji. 2024. [Word embeddings are steers for language models](https://md.research.elamur.ai/free/en/papers/han-2024-word-embeddings-are-steers-language/). In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), ACL 2024, Bangkok, Thailand,…
16. Tom Hartvigsen, Swami Sankaranarayanan, Hamid Palangi, Yoon Kim, and Marzyeh Ghassemi. 2023a. Aging with grace: Lifelong model editing with discrete key-value adaptors. In Advances in Neural Information Processing Systems, volume 36, pages 47934–47959. Curran Associates, Inc. [find](https://scholar.google.com/scholar?q=Tom+Hartvigsen%2C+Swami+Sankaranarayanan%2C+Hamid+Palangi%2C+Yoon+Kim%2C+and+Marzyeh+Ghassemi%2E+2023a%2E+Aging+with+grace%3A+Lifelong+model+editing+with+discrete+key%2Dvalue+adaptors%2E+In+Advances+in+Neural+Informati)
17. Tom Hartvigsen, Swami Sankaranarayanan, Hamid Palangi, Yoon Kim, and Marzyeh Ghassemi. 2023b. Aging with GRACE: lifelong model editing with discrete key-value adaptors. In Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, NeurIPS 2… [find](https://scholar.google.com/scholar?q=Tom+Hartvigsen%2C+Swami+Sankaranarayanan%2C+Hamid+Palangi%2C+Yoon+Kim%2C+and+Marzyeh+Ghassemi%2E+2023b%2E+Aging+with+GRACE%3A+lifelong+model+editing+with+discrete+key%2Dvalue+adaptors%2E+In+Advances+in+Neural+Informati)
18. Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. 2021. [Measuring massive multitask language understanding](https://md.research.elamur.ai/free/en/papers/hendrycks-2020-measuring-massive-multitask-language-understanding/). In 9th International Conference on Learning Representations, ICLR 2021, Virtual Event, Austria, May 3-7, 2021. OpenReview.net.
19. Lei Hsiung, Tianyu Pang, Yung-Chen Tang, Linyue Song, Tsung-Yi Ho, Pin-Yu Chen, and Yaoqing Yang. 2025. [Why LLM safety guardrails collapse after fine-tuning: A similarity analysis between alignment and fine-tuning datasets](https://md.research.elamur.ai/free/en/papers/hsiung-2026-why-llm-safety-guardrails-collapse/). CoRR, abs/2506.05346.
20. Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2022. [Lora: Low-rank adaptation of large language models](https://md.research.elamur.ai/free/en/papers/hu-2021-lora-low-rank-adaptation-large/). In The Tenth International Conference on Learning Representations, ICLR 2022, Virtual Event, April 25-29, 2022. OpenReview.net.
21. Houcheng Jiang, Junfeng Fang, Ningyu Zhang, Guojun Ma, Mingyang Wan, Xiang Wang, Xiangnan He, and Tat-Seng Chua. 2025. Anyedit: Edit any knowledge encoded in language models. CoRR, abs/2502.05628. — [arXiv:2502.05628](https://arxiv.org/abs/2502.05628) [find](https://scholar.google.com/scholar?q=Houcheng+Jiang%2C+Junfeng+Fang%2C+Ningyu+Zhang%2C+Guojun+Ma%2C+Mingyang+Wan%2C+Xiang+Wang%2C+Xiangnan+He%2C+and+Tat%2DSeng+Chua%2E+2025%2E+Anyedit%3A+Edit+any+knowledge+encoded+in+language+models%2E+CoRR%2C+abs%2F2502%2E05628%2E+%E2%80%94+%5B)
22. Zixuan Ke, Yijia Shao, Haowei Lin, Tatsuya Konishi, Gyuhak Kim, and Bing Liu. 2023. Continual pre-training of language models. In The Eleventh International Conference on Learning Representations, ICLR 2023, Kigali, Rwanda, May 1-5, 2023. Open-Review.net. [find](https://scholar.google.com/scholar?q=Zixuan+Ke%2C+Yijia+Shao%2C+Haowei+Lin%2C+Tatsuya+Konishi%2C+Gyuhak+Kim%2C+and+Bing+Liu%2E+2023%2E+Continual+pre%2Dtraining+of+language+models%2E+In+The+Eleventh+International+Conference+on+Learning+Representations%2C+ICL)
23. Omer Levy, Minjoon Seo, Eunsol Choi, and Luke Zettlemoyer. 2017. [Zero-shot relation extraction via reading comprehension](https://md.research.elamur.ai/free/en/papers/levy-2017-zero-shot-relation-extraction-reading/). In Proceedings of the 21st Conference on Computational Natural Language Learning (CoNLL 2017), Vancouver, Canada, August 3-4, 2017, pages 333–342. Association for Computational L…
24. Gaotang Li, Yuzhong Chen, and Hanghang Tong. 2025. Taming knowledge conflicts in language models. In Forty-second International Conference on Machine Learning. [find](https://scholar.google.com/scholar?q=Gaotang+Li%2C+Yuzhong+Chen%2C+and+Hanghang+Tong%2E+2025%2E+Taming+knowledge+conflicts+in+language+models%2E+In+Forty%2Dsecond+International+Conference+on+Machine+Learning%2E)
25. Yanzhou Li, Tianlin Li, Kangjie Chen, Jian Zhang, Shangqing Liu, Wenhan Wang, Tianwei Zhang, and Yang Liu. 2024. Badedit: Backdooring large language models by model editing. In The Twelfth International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Yanzhou+Li%2C+Tianlin+Li%2C+Kangjie+Chen%2C+Jian+Zhang%2C+Shangqing+Liu%2C+Wenhan+Wang%2C+Tianwei+Zhang%2C+and+Yang+Liu%2E+2024%2E+Badedit%3A+Backdooring+large+language+models+by+model+editing%2E+In+The+Twelfth+Internation)
26. Tianci Liu, Ruirui Li, Yunzhe Qi, Hui Liu, Xianfeng Tang, Tianqi Zheng, Qingyu Yin, Monica Xiao Cheng, Jun Huan, Haoyu Wang, and Jing Gao. 2025a. Unlocking efficient, scalable, and continual knowledge editing with basis-level representation fine-tuning. In The Thirteenth International Conference on… [find](https://scholar.google.com/scholar?q=Tianci+Liu%2C+Ruirui+Li%2C+Yunzhe+Qi%2C+Hui+Liu%2C+Xianfeng+Tang%2C+Tianqi+Zheng%2C+Qingyu+Yin%2C+Monica+Xiao+Cheng%2C+Jun+Huan%2C+Haoyu+Wang%2C+and+Jing+Gao%2E+2025a%2E+Unlocking+efficient%2C+scalable%2C+and+continual+knowledge)
27. Xuyuan Liu, Lei Hsiung, Yaoqing Yang, and Yujun Yan. 2025b. Spectral insights into data-oblivious critical layers in large language models. In Findings of the Association for Computational Linguistics: ACL 2025, pages 4860–4877, Vienna, Austria. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Xuyuan+Liu%2C+Lei+Hsiung%2C+Yaoqing+Yang%2C+and+Yujun+Yan%2E+2025b%2E+Spectral+insights+into+data%2Doblivious+critical+layers+in+large+language+models%2E+In+Findings+of+the+Association+for+Computational+Linguistics)
28. Aman Madaan, Shuyan Zhou, Uri Alon, Yiming Yang, and Graham Neubig. 2022. [Language models of code are few-shot commonsense learners](https://md.research.elamur.ai/free/en/papers/madaan-2022-language-models-code-few-shot/). In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, EMNLP 2022, Abu Dhabi, United Arab Emirates, December 7-11, 2022, pages 1384…
29. Samuel Marks and Max Tegmark. 2023. [The geometry of truth: Emergent linear structure in large language model representations of true/false datasets](https://md.research.elamur.ai/free/en/papers/marks-2023-geometry-truth-emergent-linear-structure/). CoRR, abs/2310.06824.
30. Samuel Marks and Max Tegmark. 2024. [The geometry of truth: Emergent linear structure in large language model representations of true/false datasets](https://md.research.elamur.ai/free/en/papers/marks-2023-geometry-truth-emergent-linear-structure/).
31. Kevin Meng, David Bau, Alex Andonian, and Yonatan Belinkov. 2022a. Locating and editing factual associations in GPT. In Advances in Neural Information Processing Systems 35: Annual Conference on Neural Information Processing Systems 2022, NeurIPS 2022, New Orleans, LA, USA, November 28 - December 9,… [find](https://scholar.google.com/scholar?q=Kevin+Meng%2C+David+Bau%2C+Alex+Andonian%2C+and+Yonatan+Belinkov%2E+2022a%2E+Locating+and+editing+factual+associations+in+GPT%2E+In+Advances+in+Neural+Information+Processing+Systems+35%3A+Annual+Conference+on+Neura)
32. Kevin Meng, Arnab Sen Sharma, Alex Andonian, Yonatan Belinkov, and David Bau. 2022b. Mass-editing memory in a transformer. CoRR, abs/2210.07229. [find](https://scholar.google.com/scholar?q=Kevin+Meng%2C+Arnab+Sen+Sharma%2C+Alex+Andonian%2C+Yonatan+Belinkov%2C+and+David+Bau%2E+2022b%2E+Mass%2Dediting+memory+in+a+transformer%2E+CoRR%2C+abs%2F2210%2E07229%2E)
33. Tomas Mikolov, Wen-tau Yih, and Geoffrey Zweig. 2013. [Linguistic regularities in continuous space word representations](https://md.research.elamur.ai/free/en/papers/mikolov-2013-linguistic-regularities-continuous-space-word/). In Proceedings of the 2013 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 746–751, Atlanta, Georgia.…
34. Eric Mitchell, Charles Lin, Antoine Bosselut, Chelsea Finn, and Christopher D. Manning. 2022. Fast model editing at scale. In The Tenth International Conference on Learning Representations, ICLR 2022, Virtual Event, April 25-29, 2022. OpenReview.net. [find](https://scholar.google.com/scholar?q=Eric+Mitchell%2C+Charles+Lin%2C+Antoine+Bosselut%2C+Chelsea+Finn%2C+and+Christopher+D%2E+Manning%2E+2022%2E+Fast+model+editing+at+scale%2E+In+The+Tenth+International+Conference+on+Learning+Representations%2C+ICLR+2022%2C)
35. Kento Nishi, Maya Okawa, Rahul Ramesh, Mikail Khona, Hidenori Tanaka, and Ekdeep Singh Lubana. 2025. Representation shattering in transformers: A synthetic study with knowledge editing. [find](https://scholar.google.com/scholar?q=Kento+Nishi%2C+Maya+Okawa%2C+Rahul+Ramesh%2C+Mikail+Khona%2C+Hidenori+Tanaka%2C+and+Ekdeep+Singh+Lubana%2E+2025%2E+Representation+shattering+in+transformers%3A+A+synthetic+study+with+knowledge+editing%2E)
36. Kiho Park, Yo Joong Choe, and Victor Veitch. 2024. [The linear representation hypothesis and the geometry of large language models](https://md.research.elamur.ai/free/en/papers/park-2023-linear-representation-hypothesis-geometry-large/). In Proceedings of the 41st International Conference on Machine Learning, ICML’24. JMLR.org.
37. Nina Rimsky, Nick Gabrieli, Julian Schulz, Meg Tong, Evan Hubinger, and Alexander Turner. 2024. Steering llama 2 via contrastive activation addition. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 15504–15522, Bangkok, Thaila… [find](https://scholar.google.com/scholar?q=Nina+Rimsky%2C+Nick+Gabrieli%2C+Julian+Schulz%2C+Meg+Tong%2C+Evan+Hubinger%2C+and+Alexander+Turner%2E+2024%2E+Steering+llama+2+via+contrastive+activation+addition%2E+In+Proceedings+of+the+62nd+Annual+Meeting+of+the+A)
38. Alireza Salemi and Hamed Zamani. 2024. [Evaluating retrieval quality in retrieval-augmented generation](https://md.research.elamur.ai/free/en/papers/salemi-2024-evaluating-retrieval-quality-retrieval-augmented/). In Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR 2024, Washington DC, USA, July 14-18, 2024, pages 2395–2400. ACM.
39. Kai Sun, Yifan Xu, Hanwen Zha, Yue Liu, and Xin Luna Dong. 2024. Head-to-tail: How knowledgeable are large language models (LLMs)? A.K.A. will LLMs replace knowledge graphs? In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human La… [find](https://scholar.google.com/scholar?q=Kai+Sun%2C+Yifan+Xu%2C+Hanwen+Zha%2C+Yue+Liu%2C+and+Xin+Luna+Dong%2E+2024%2E+Head%2Dto%2Dtail%3A+How+knowledgeable+are+large+language+models+%28LLMs%29%3F+A%2EK%2EA%2E+will+LLMs+replace+knowledge+graphs%3F+In+Proceedings+of+the+2024)
40. Lukas Thede, Karsten Roth, Matthias Bethge, Zeynep Akata, and Thomas Hartvigsen. 2025. Wikibigedit: Understanding the limits of lifelong knowledge editing in LLMs. In Forty-second International Conference on Machine Learning. [find](https://scholar.google.com/scholar?q=Lukas+Thede%2C+Karsten+Roth%2C+Matthias+Bethge%2C+Zeynep+Akata%2C+and+Thomas+Hartvigsen%2E+2025%2E+Wikibigedit%3A+Understanding+the+limits+of+lifelong+knowledge+editing+in+LLMs%2E+In+Forty%2Dsecond+International+Confer)
41. Alexander Matt Turner, Lisa Thiergart, Gavin Leech, David Udell, Juan J Vazquez, Ulisse Mini, and Monte MacDiarmid. 2023. [Steering language models with activation engineering](https://md.research.elamur.ai/free/en/papers/turner-2023-steering-language-models-activation-engineering/). arXiv preprint arXiv:2308.10248.
42. Ke Wang, Yiming Qin, Nikolaos Dimitriadis, Alessandro Favero, and Pascal Frossard. 2025. MEMOIR: lifelong model editing with minimal overwrite and informed retention for llms. CoRR, abs/2506.07899. — [arXiv:2506.07899](https://arxiv.org/abs/2506.07899) [find](https://scholar.google.com/scholar?q=Ke+Wang%2C+Yiming+Qin%2C+Nikolaos+Dimitriadis%2C+Alessandro+Favero%2C+and+Pascal+Frossard%2E+2025%2E+MEMOIR%3A+lifelong+model+editing+with+minimal+overwrite+and+informed+retention+for+llms%2E+CoRR%2C+abs%2F2506%2E07899%2E+%E2%80%94+)
43. Peng Wang, Zexi Li, Ningyu Zhang, Ziwen Xu, Yunzhi Yao, Yong Jiang, Pengjun Xie, Fei Huang, and Huajun Chen. 2024. WISE: rethinking the knowledge memory for lifelong model editing of large language models. In Advances in Neural Information Processing Systems 38: Annual Conference on Neural Informati… [find](https://scholar.google.com/scholar?q=Peng+Wang%2C+Zexi+Li%2C+Ningyu+Zhang%2C+Ziwen+Xu%2C+Yunzhi+Yao%2C+Yong+Jiang%2C+Pengjun+Xie%2C+Fei+Huang%2C+and+Huajun+Chen%2E+2024%2E+WISE%3A+rethinking+the+knowledge+memory+for+lifelong+model+editing+of+large+language+mo)
44. Xiaobao Wu, Liangming Pan, William Yang Wang, and Anh Tuan Luu. 2024a. AKEW: assessing knowledge editing in the wild. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, EMNLP 2024, Miami, FL, USA, November 12-16, 2024, pages 15118–15133. Association for Comput… [find](https://scholar.google.com/scholar?q=Xiaobao+Wu%2C+Liangming+Pan%2C+William+Yang+Wang%2C+and+Anh+Tuan+Luu%2E+2024a%2E+AKEW%3A+assessing+knowledge+editing+in+the+wild%2E+In+Proceedings+of+the+2024+Conference+on+Empirical+Methods+in+Natural+Language+Pro)
45. Zhengxuan Wu, Aryaman Arora, Zheng Wang, Atticus Geiger, Dan Jurafsky, Christopher D. Manning, and Christopher Potts. 2024b. Reft: Representation fine-tuning for language models. In Advances in Neural Information Processing Systems 38: Annual Conference on Neural Information Processing Systems 2024,… [find](https://scholar.google.com/scholar?q=Zhengxuan+Wu%2C+Aryaman+Arora%2C+Zheng+Wang%2C+Atticus+Geiger%2C+Dan+Jurafsky%2C+Christopher+D%2E+Manning%2C+and+Christopher+Potts%2E+2024b%2E+Reft%3A+Representation+fine%2Dtuning+for+language+models%2E+In+Advances+in+Neural)
46. Qwen An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Guanting Dong, Haoran Wei, Huan Lin, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxin Yang, Jingren Zhou, Junyang Lin, Kai Dang, Keming Lu, Keqin Bao, Kexin Yang, Le Yu, Mei… [find](https://scholar.google.com/scholar?q=Qwen+An+Yang%2C+Baosong+Yang%2C+Beichen+Zhang%2C+Binyuan+Hui%2C+Bo+Zheng%2C+Bowen+Yu%2C+Chengyuan+Li%2C+Dayiheng+Liu%2C+Fei+Huang%2C+Guanting+Dong%2C+Haoran+Wei%2C+Huan+Lin%2C+Jian+Yang%2C+Jianhong+Tu%2C+Jianwei+Zhang%2C+Jianxin+Y)
47. Wanli Yang, Fei Sun, Xinyu Ma, Xun Liu, Dawei Yin, and Xueqi Cheng. 2024b. The butterfly effect of model editing: Few edits can trigger large language models collapse. In Findings of the Association for Computational Linguistics, ACL 2024, Bangkok, Thailand and virtual meeting, August 11-16, 2024, p… [find](https://scholar.google.com/scholar?q=Wanli+Yang%2C+Fei+Sun%2C+Xinyu+Ma%2C+Xun+Liu%2C+Dawei+Yin%2C+and+Xueqi+Cheng%2E+2024b%2E+The+butterfly+effect+of+model+editing%3A+Few+edits+can+trigger+large+language+models+collapse%2E+In+Findings+of+the+Association+f)
48. Çagatay Yildiz, Nishaanth Kanna Ravichandran, Nitin Sharma, Matthias Bethge, and Beyza Ermis. 2025. Investigating continual pretraining in large language models: Insights and implications. [find](https://scholar.google.com/scholar?q=%C3%87agatay+Yildiz%2C+Nishaanth+Kanna+Ravichandran%2C+Nitin+Sharma%2C+Matthias+Bethge%2C+and+Beyza+Ermis%2E+2025%2E+Investigating+continual+pretraining+in+large+language+models%3A+Insights+and+implications%2E)
49. Trans. Mach. Learn. Res., 2025. [find](https://scholar.google.com/scholar?q=Trans%2E+Mach%2E+Learn%2E+Res%2E%2C+2025%2E)
50. Yue Yu, Wei Ping, Zihan Liu, Boxin Wang, Jiaxuan You, Chao Zhang, Mohammad Shoeybi, and Bryan Catanzaro. 2024. RankRAG: Unifying context ranking with retrieval-augmented generation in LLMs. In The Thirty-eighth Annual Conference on Neural Information Processing Systems. [find](https://scholar.google.com/scholar?q=Yue+Yu%2C+Wei+Ping%2C+Zihan+Liu%2C+Boxin+Wang%2C+Jiaxuan+You%2C+Chao+Zhang%2C+Mohammad+Shoeybi%2C+and+Bryan+Catanzaro%2E+2024%2E+RankRAG%3A+Unifying+context+ranking+with+retrieval%2Daugmented+generation+in+LLMs%2E+In+The+Thi)
51. Binchi Zhang, Zhengzhang Chen, Zaiyi Zheng, Jundong Li, and Haifeng Chen. 2025. Resolving editing-unlearning conflicts: A knowledge codebook framework for large language model updating. CoRR, abs/2502.00158. — [arXiv:2502.00158](https://arxiv.org/abs/2502.00158) [find](https://scholar.google.com/scholar?q=Binchi+Zhang%2C+Zhengzhang+Chen%2C+Zaiyi+Zheng%2C+Jundong+Li%2C+and+Haifeng+Chen%2E+2025%2E+Resolving+editing%2Dunlearning+conflicts%3A+A+knowledge+codebook+framework+for+large+language+model+updating%2E+CoRR%2C+abs%2F2502)
52. Ce Zheng, Lei Li, Qingxiu Dong, Yuxuan Fan, Zhiyong Wu, Jingjing Xu, and Baobao Chang. 2023. Can we edit factual knowledge by in-context learning? In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, EMNLP 2023, Singapore, December 6-10, 2023, pages 4862– 4876.… [find](https://scholar.google.com/scholar?q=Ce+Zheng%2C+Lei+Li%2C+Qingxiu+Dong%2C+Yuxuan+Fan%2C+Zhiyong+Wu%2C+Jingjing+Xu%2C+and+Baobao+Chang%2E+2023%2E+Can+we+edit+factual+knowledge+by+in%2Dcontext+learning%3F+In+Proceedings+of+the+2023+Conference+on+Empirical+Me)
53. Chen Zhu, Ankit Singh Rawat, Manzil Zaheer, Srinadh Bhojanapalli, Daliang Li, Felix Yu, and Sanjiv Kumar. 2020. Modifying memories in transformer models. arXiv preprint arXiv:2012.00363. [find](https://scholar.google.com/scholar?q=Chen+Zhu%2C+Ankit+Singh+Rawat%2C+Manzil+Zaheer%2C+Srinadh+Bhojanapalli%2C+Daliang+Li%2C+Felix+Yu%2C+and+Sanjiv+Kumar%2E+2020%2E+Modifying+memories+in+transformer+models%2E+arXiv+preprint+arXiv%3A2012%2E00363%2E)
54. Andy Zou, Long Phan, Sarah Li Chen, James Campbell, Phillip Guo, Richard Ren, Alexander Pan, Xuwang Yin, Mantas Mazeika, Ann-Kathrin Dombrowski, Shashwat Goel, Nathaniel Li, Michael J. Byun, Zifan Wang, Alex Mallen, Steven Basart, Sanmi Koyejo, Dawn Song, Matt Fredrikson, J. Zico Kolter, and Dan Hen… — [Representation Engineering: A Top-Down Approach to AI Transparency](https://md.research.elamur.ai/free/en/papers/zou-2023-representation-engineering-top-down-approach/)

*Updated 2 November 2026*