---
title: "FAME: Fictional Actors for Multilingual — Savelli 2026"
description: "Machine Unlearning methods remove specific knowledge from a trained LLM without retraining it, which is relevant to privacy rights such as GDPR erasure…"
authors: ["Claudio Savelli", "Moreno La Quatra", "Alkis Koudounas", "Flavio Giobergia"]
year: 2026
venue: "Proceedings of the Fifteenth Language Resources and Evaluation Conference"
source: "https://aclanthology.org/2026.lrec-1.200/"
doi: "10.63317/3npbbsyj2dd7"
licence: "CC BY-NC 4.0 (ELRA, LREC proceedings)"
licence_url: "https://creativecommons.org/licenses/by-nc/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/papers/savelli-2026-fame-fictional-actors-multilingual-erasure/"
---
[Elamur Research](https://md.research.elamur.ai/) · [this page in HTML](https://research.elamur.ai/free/papers/savelli-2026-fame-fictional-actors-multilingual-erasure/)

[home](https://md.research.elamur.ai/) / [papers](https://md.research.elamur.ai/free/papers/) / [2026](https://md.research.elamur.ai/free/papers/2026/page/2/) / FAME: Fictional Actors for Multilingual Erasure

# FAME: Fictional Actors for Multilingual Erasure

**Authors:** Claudio Savelli, Moreno La Quatra, Alkis Koudounas, Flavio Giobergia · **Year:** 2026 · **Venue:** Proceedings of the Fifteenth Language Resources and Evaluation Conference · **Licence:** [CC BY-NC 4.0 (ELRA, LREC proceedings)](https://creativecommons.org/licenses/by-nc/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2026.lrec-1.200/) · [PDF](https://aclanthology.org/2026.lrec-1.200.pdf) · [DOI 10.63317/3npbbsyj2dd7](https://doi.org/10.63317/3npbbsyj2dd7) · [OpenAlex](https://openalex.org/W7160161155)

**Citations:** 1 citations (openalex-snapshot · 2026-10-14) · 6 in our library, of the 140673 papers whose bibliographies we read

## Summary
Machine Unlearning methods remove specific knowledge from a trained LLM without retraining it, which is relevant to privacy rights such as GDPR erasure requests. Existing benchmarks like TOFU have two gaps: they cover English only, and they test only whole-entity forgetting. FAME addresses both. It is a synthetic benchmark of invented actors, so none of its content could have appeared in pretraining data. Each actor profile has exactly 20 atomic facts, grouped under four themes: biography, career, achievements, and personal information. Each fact is paired with one question-answer item, in English, French, German, Italian and Spanish.

This structure supports two kinds of test. One erases an entire identity. The other deletes selected facts while the rest of the profile must survive, much like a real request to remove only contact details. The authors test five unlearning methods on a multilingual LLM in all five languages, scoring utility, efficacy and efficiency. The excerpt provided omits the experimental section, so specific results cannot be reported here.

## Key points
- Size: 1,000 fictional actor biographies and 20,000 QA pairs, one per fact, with 20 facts per actor.
- Languages: English, French, German, Italian, Spanish. The authors present it as the first multilingual text benchmark for LLM unlearning.
- Two splits:
  - an entity-based split for full identity removal;
  - a topic-based split for instance-level (partial) deletion.
- Compared with TOFU's 200 English-only author profiles, FAME has five times as many profiles and adds fine-grained forgetting.
- Data and evaluation code are public at github.com/ClaudioSavelli/FAME.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

Fields: [Linguistics & languages](https://md.research.elamur.ai/free/shelves/linguistics/)

## Cite

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{savelli2026fame,
  title = {{FAME: Fictional Actors for Multilingual Erasure}},
  author = {Savelli, Claudio and Quatra, Moreno La and Koudounas, Alkis and Giobergia, Flavio},
  booktitle = {Proceedings of the Fifteenth Language Resources and Evaluation Conference},
  year = {2026},
  doi = {10.63317/3npbbsyj2dd7},
  url = {https://doi.org/10.63317/3npbbsyj2dd7}
}
```

- [← previous FalAR: A Large-scale Speaker-Annotated European Portuguese Speech Corpus of Parliamentary Sessions](https://md.research.elamur.ai/free/papers/teixeira-2026-falar-large-scale-speaker-annotated/)
- [next → FBK's Long-form SpeechLLMs for IWSLT 2026 Instruction Following](https://md.research.elamur.ai/free/papers/xie-2026-fbk-s-long-form-speechllms/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/papers/2026/page/2/)

## References

34 references, 5 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Kabir Ahuja, Sandipan Dandapat, Sunayana Sitaram, and Monojit Choudhury. 2022. Beyond static models and test sets: Benchmarking the potential of pre-trained models across tasks and languages. In Proceedings of NLP Power! The First Workshop on Efficient Benchmarking in NLP, pages 64–74, Dublin, Irela… [find](https://scholar.google.com/scholar?q=Kabir+Ahuja%2C+Sandipan+Dandapat%2C+Sunayana+Sitaram%2C+and+Monojit+Choudhury%2E+2022%2E+Beyond+static+models+and+test+sets%3A+Benchmarking+the+potential+of+pre%2Dtrained+models+across+tasks+and+languages%2E+In+Proce)
2. Medhi Bourtoule, Varun Chandrasekaran, Christopher A. Choquette-Choo, Haoran Jia, Alex Travers, Baiheng Zhang, David Lie, and Nicolas Papernot. 2021. Machine unlearning. In Proceedings of the IEEE Symposium on Security and Privacy (SP), pages 141–159. [find](https://scholar.google.com/scholar?q=Medhi+Bourtoule%2C+Varun+Chandrasekaran%2C+Christopher+A%2E+Choquette%2DChoo%2C+Haoran+Jia%2C+Alex+Travers%2C+Baiheng+Zhang%2C+David+Lie%2C+and+Nicolas+Papernot%2E+2021%2E+Machine+unlearning%2E+In+Proceedings+of+the+IEEE+Sym)
3. Nicholas Carlini, Daphne Ippolito, Matthew Jagielski, Katherine Lee, Florian Tramer, and Chiyuan Zhang. 2022. Quantifying memorization across neural language models. In The Eleventh International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Nicholas+Carlini%2C+Daphne+Ippolito%2C+Matthew+Jagielski%2C+Katherine+Lee%2C+Florian+Tramer%2C+and+Chiyuan+Zhang%2E+2022%2E+Quantifying+memorization+across+neural+language+models%2E+In+The+Eleventh+International+Conf)
4. Nicholas Carlini, Chang Liu, Úlfar Erlingsson, Jernej Kos, and Dawn Song. 2019. The secret sharer: Evaluating and testing unintended memorization in neural networks. In 28th USENIX security symposium (USENIX security 19), pages 267–284. [find](https://scholar.google.com/scholar?q=Nicholas+Carlini%2C+Chang+Liu%2C+%C3%9Alfar+Erlingsson%2C+Jernej+Kos%2C+and+Dawn+Song%2E+2019%2E+The+secret+sharer%3A+Evaluating+and+testing+unintended+memorization+in+neural+networks%2E+In+28th+USENIX+security+symposium+) [our card](https://md.research.elamur.ai/free/papers/carlini-2018-secret-sharer-evaluating-testing-unintended/)
5. Dasol Choi and Dongbin Na. 2023. Towards machine unlearning benchmarks: Forgetting the personal identities in facial recognition systems. arXiv preprint arXiv:2311.02240. — [arXiv:2311.02240](https://arxiv.org/abs/2311.02240) [find](https://scholar.google.com/scholar?q=Dasol+Choi+and+Dongbin+Na%2E+2023%2E+Towards+machine+unlearning+benchmarks%3A+Forgetting+the+personal+identities+in+facial+recognition+systems%2E+arXiv+preprint+arXiv%3A2311%2E02240%2E+%E2%80%94+%5BarXiv%3A2311%2E02240%5D%28https%3A%2F%2F)
6. Minseok Choi, Kyunghyun Min, and Jaegul Choo. 2024. Cross-lingual unlearning of selective knowledge in multilingual language models. In Findings of the Association for Computational Linguistics: EMNLP 2024, pages 10732–10747, Miami, Florida, USA. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Minseok+Choi%2C+Kyunghyun+Min%2C+and+Jaegul+Choo%2E+2024%2E+Cross%2Dlingual+unlearning+of+selective+knowledge+in+multilingual+language+models%2E+In+Findings+of+the+Association+for+Computational+Linguistics%3A+EMNLP)
7. Minseok Choi, Daniel Rim, Dohyun Lee, and Jaegul Choo. 2025. Opt-out: Investigating entity-level unlearning for large language models via optimal transport. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 28280– 28297, Vienna,… [find](https://scholar.google.com/scholar?q=Minseok+Choi%2C+Daniel+Rim%2C+Dohyun+Lee%2C+and+Jaegul+Choo%2E+2025%2E+Opt%2Dout%3A+Investigating+entity%2Dlevel+unlearning+for+large+language+models+via+optimal+transport%2E+In+Proceedings+of+the+63rd+Annual+Meeting+o)
8. Gheorghe Comanici, Eric Bieber, Mike Schaekermann, Ice Pasupat, Noveen Sachdeva, Inderjit Dhillon, Marcel Blistein, Ori Ram, Dan Zhang, Evan Rosen, et al. 2025. Gemini 2.5: Pushing the frontier with advanced reasoning, multimodality, long context, and next generation agentic capabilities. arXiv prep… [find](https://scholar.google.com/scholar?q=Gheorghe+Comanici%2C+Eric+Bieber%2C+Mike+Schaekermann%2C+Ice+Pasupat%2C+Noveen+Sachdeva%2C+Inderjit+Dhillon%2C+Marcel+Blistein%2C+Ori+Ram%2C+Dan+Zhang%2C+Evan+Rosen%2C+et+al%2E+2025%2E+Gemini+2%2E5%3A+Pushing+the+frontier+with+a) [our card](https://md.research.elamur.ai/free/papers/comanici-2025-gemini-2-5-pushing-frontier/)
9. Kate Crawford. 2022. Atlas of AI: Power, Politics, and the Planetary Costs of Artificial Intelligence. Yale University Press. [find](https://scholar.google.com/scholar?q=Kate+Crawford%2E+2022%2E+Atlas+of+AI%3A+Power%2C+Politics%2C+and+the+Planetary+Costs+of+Artificial+Intelligence%2E+Yale+University+Press%2E)
10. Ronen Eldan and Mark Russinovich. 2023. Who’s harry potter? approximate unlearning in llms. arXiv preprint arXiv:2310.02238. [find](https://scholar.google.com/scholar?q=Ronen+Eldan+and+Mark+Russinovich%2E+2023%2E+Who%E2%80%99s+harry+potter%3F+approximate+unlearning+in+llms%2E+arXiv+preprint+arXiv%3A2310%2E02238%2E)
11. Aditya Golatkar, Alessandro Achille, and Stefano Soatto. 2020. Eternal sunshine of the spotless net: Selective forgetting in deep networks. In CVPR. [find](https://scholar.google.com/scholar?q=Aditya+Golatkar%2C+Alessandro+Achille%2C+and+Stefano+Soatto%2E+2020%2E+Eternal+sunshine+of+the+spotless+net%3A+Selective+forgetting+in+deep+networks%2E+In+CVPR%2E)
12. Jamie Hayes, Ilia Shumailov, Eleni Triantafillou, Amr Khalifa, and Nicolas Papernot. 2024. Inexact unlearning needs more careful evaluations to avoid a false sense of privacy. arXiv preprint arXiv:2403.01218. — [arXiv:2403.01218](https://arxiv.org/abs/2403.01218) [find](https://scholar.google.com/scholar?q=Jamie+Hayes%2C+Ilia+Shumailov%2C+Eleni+Triantafillou%2C+Amr+Khalifa%2C+and+Nicolas+Papernot%2E+2024%2E+Inexact+unlearning+needs+more+careful+evaluations+to+avoid+a+false+sense+of+privacy%2E+arXiv+preprint+arXiv%3A240)
13. Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. 2020. Measuring massive multi-task language understanding. arXiv preprint arXiv:2009.03300. [find](https://scholar.google.com/scholar?q=Dan+Hendrycks%2C+Collin+Burns%2C+Steven+Basart%2C+Andy+Zou%2C+Mantas+Mazeika%2C+Dawn+Song%2C+and+Jacob+Steinhardt%2E+2020%2E+Measuring+massive+multi%2Dtask+language+understanding%2E+arXiv+preprint+arXiv%3A2009%2E03300%2E) [our card](https://md.research.elamur.ai/free/papers/hendrycks-2020-measuring-massive-multitask-language-understanding/)
14. Jordan Hoffmann, Sebastian Borgeaud, Arthur Mensch, Elena Buchatskaya, Trevor Cai, Eliza Rutherford, Diego de Las Casas, Lisa Anne Hendricks, Johannes Welbl, Aidan Clark, et al. 2022. Training compute-optimal large language models. In Proceedings of the 36th International Conference on Neural Inform… [find](https://scholar.google.com/scholar?q=Jordan+Hoffmann%2C+Sebastian+Borgeaud%2C+Arthur+Mensch%2C+Elena+Buchatskaya%2C+Trevor+Cai%2C+Eliza+Rutherford%2C+Diego+de+Las+Casas%2C+Lisa+Anne+Hendricks%2C+Johannes+Welbl%2C+Aidan+Clark%2C+et+al%2E+2022%2E+Training+compute)
15. Shengyuan Hu, Neil Kale, Pratiksha Thaker, Yiwei Fu, Steven Wu, and Virginia Smith. 2025. Blur: A benchmark for llm unlearning robust to forget-retain overlap. arXiv preprint arXiv:2506.15699. — [arXiv:2506.15699](https://arxiv.org/abs/2506.15699) [find](https://scholar.google.com/scholar?q=Shengyuan+Hu%2C+Neil+Kale%2C+Pratiksha+Thaker%2C+Yiwei+Fu%2C+Steven+Wu%2C+and+Virginia+Smith%2E+2025%2E+Blur%3A+A+benchmark+for+llm+unlearning+robust+to+forget%2Dretain+overlap%2E+arXiv+preprint+arXiv%3A2506%2E15699%2E+%E2%80%94+%5BarXi)
16. Xu Huang, Wenhao Zhu, Hanxu Hu, Conghui He, Lei Li, Shujian Huang, and Fei Yuan. 2025. Benchmax: A comprehensive multilingual evaluation suite for large language models. arXiv preprint arXiv:2502.07346. [find](https://scholar.google.com/scholar?q=Xu+Huang%2C+Wenhao+Zhu%2C+Hanxu+Hu%2C+Conghui+He%2C+Lei+Li%2C+Shujian+Huang%2C+and+Fei+Yuan%2E+2025%2E+Benchmax%3A+A+comprehensive+multilingual+evaluation+suite+for+large+language+models%2E+arXiv+preprint+arXiv%3A2502%2E0734)
17. Bjørn Aslak Juliussen, Jon Petter Rui, and Dag Johansen. 2023. Algorithms that forget: Machine unlearning and the right to erasure. Computer Law & Security Review, 51:105885. [find](https://scholar.google.com/scholar?q=Bj%C3%B8rn+Aslak+Juliussen%2C+Jon+Petter+Rui%2C+and+Dag+Johansen%2E+2023%2E+Algorithms+that+forget%3A+Machine+unlearning+and+the+right+to+erasure%2E+Computer+Law+%26+Security+Review%2C+51%3A105885%2E)
18. Alkis Koudounas, Claudio Savelli, Flavio Giobergia, and Elena Baralis. 2025. “Alexa, can you forget me?” Machine Unlearning Benchmark in Spoken Language Understanding. In Interspeech 2025, pages 1768–1772. [find](https://scholar.google.com/scholar?q=Alkis+Koudounas%2C+Claudio+Savelli%2C+Flavio+Giobergia%2C+and+Elena+Baralis%2E+2025%2E+%E2%80%9CAlexa%2C+can+you+forget+me%3F%E2%80%9D+Machine+Unlearning+Benchmark+in+Spoken+Language+Understanding%2E+In+Interspeech+2025%2C+pages+1768%E2%80%93)
19. Meghdad Kurmanji, Peter Triantafillou, Jamie Hayes, and Eleni Triantafillou. 2024. Towards unbounded machine unlearning. NeurIPS, 36. [find](https://scholar.google.com/scholar?q=Meghdad+Kurmanji%2C+Peter+Triantafillou%2C+Jamie+Hayes%2C+and+Eleni+Triantafillou%2E+2024%2E+Towards+unbounded+machine+unlearning%2E+NeurIPS%2C+36%2E)
20. Sijia Liu, Yuanshun Yao, Jinghan Jia, Stephen Casper, Nathalie Baracaldo, Peter Hase, Yuguang Yao, Chris Yuhao Liu, Xiaojun Xu, Hang Li, et al. 2025. Rethinking machine unlearning for large language models. Nature Machine Intelligence, pages 1–14. [find](https://scholar.google.com/scholar?q=Sijia+Liu%2C+Yuanshun+Yao%2C+Jinghan+Jia%2C+Stephen+Casper%2C+Nathalie+Baracaldo%2C+Peter+Hase%2C+Yuguang+Yao%2C+Chris+Yuhao+Liu%2C+Xiaojun+Xu%2C+Hang+Li%2C+et+al%2E+2025%2E+Rethinking+machine+unlearning+for+large+language+m)
21. Weitao Ma, Xiaocheng Feng, Weihong Zhong, Lei Huang, Yangfan Ye, Xiachong Feng, and Bing Qin. 2025. Unveiling entity-level unlearning for large language models: A comprehensive analysis. In Proceedings of the 31st International Conference on Computational Linguistics, pages 5345–5363, Abu Dhabi, UAE… [find](https://scholar.google.com/scholar?q=Weitao+Ma%2C+Xiaocheng+Feng%2C+Weihong+Zhong%2C+Lei+Huang%2C+Yangfan+Ye%2C+Xiachong+Feng%2C+and+Bing+Qin%2E+2025%2E+Unveiling+entity%2Dlevel+unlearning+for+large+language+models%3A+A+comprehensive+analysis%2E+In+Proceeding)
22. Pratyush Maini, Zhili Feng, Avi Schwarzschild, Zachary C Lipton, and J Zico Kolter. 2024. Tofu: A task of fictitious unlearning for llms. arXiv preprint arXiv:2401.06121. [find](https://scholar.google.com/scholar?q=Pratyush+Maini%2C+Zhili+Feng%2C+Avi+Schwarzschild%2C+Zachary+C+Lipton%2C+and+J+Zico+Kolter%2E+2024%2E+Tofu%3A+A+task+of+fictitious+unlearning+for+llms%2E+arXiv+preprint+arXiv%3A2401%2E06121%2E) [our card](https://md.research.elamur.ai/free/papers/maini-2024-tofu-task-fictitious-unlearning-llms/)
23. Alessandro Mantelero. 2013. The eu proposal for a general data protection regulation and the roots of the ‘right to be forgotten’. Computer Law & Security Review, 29(3):229–235. [find](https://scholar.google.com/scholar?q=Alessandro+Mantelero%2E+2013%2E+The+eu+proposal+for+a+general+data+protection+regulation+and+the+roots+of+the+%E2%80%98right+to+be+forgotten%E2%80%99%2E+Computer+Law+%26+Security+Review%2C+29%283%29%3A229%E2%80%93235%2E)
24. Matt Post. 2018. A call for clarity in reporting BLEU scores. In Proceedings of the Third Conference on Machine Translation: Research Papers, pages 186–191, Belgium, Brussels. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Matt+Post%2E+2018%2E+A+call+for+clarity+in+reporting+BLEU+scores%2E+In+Proceedings+of+the+Third+Conference+on+Machine+Translation%3A+Research+Papers%2C+pages+186%E2%80%93191%2C+Belgium%2C+Brussels%2E+Association+for+Computat) [our card](https://md.research.elamur.ai/free/papers/post-2018-call-clarity-reporting-bleu-scores/)
25. Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D Manning, Stefano Ermon, and Chelsea Finn. 2023. Direct preference optimization: Your language model is secretly a reward model. Advances in neural information processing systems, 36:53728–53741. [find](https://scholar.google.com/scholar?q=Rafael+Rafailov%2C+Archit+Sharma%2C+Eric+Mitchell%2C+Christopher+D+Manning%2C+Stefano+Ermon%2C+and+Chelsea+Finn%2E+2023%2E+Direct+preference+optimization%3A+Your+language+model+is+secretly+a+reward+model%2E+Advances+in)
26. John W Ratcliff, David E Metzener, et al. 1988. Pattern matching: The gestalt approach. Dr. Dobb’s Journal, 13(7):46. [find](https://scholar.google.com/scholar?q=John+W+Ratcliff%2C+David+E+Metzener%2C+et+al%2E+1988%2E+Pattern+matching%3A+The+gestalt+approach%2E+Dr%2E+Dobb%E2%80%99s+Journal%2C+13%287%29%3A46%2E)
27. Nils Reimers and Iryna Gurevych. 2019. Sentence-BERT: Sentence embeddings using Siamese BERT-networks. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 3982–3992,… [find](https://scholar.google.com/scholar?q=Nils+Reimers+and+Iryna+Gurevych%2E+2019%2E+Sentence%2DBERT%3A+Sentence+embeddings+using+Siamese+BERT%2Dnetworks%2E+In+Proceedings+of+the+2019+Conference+on+Empirical+Methods+in+Natural+Language+Processing+and+the)
28. Giovanni Sartor, Francesca Lagioia, et al. 2020. The impact of the general data protection regulation (gdpr) on artificial intelligence. [find](https://scholar.google.com/scholar?q=Giovanni+Sartor%2C+Francesca+Lagioia%2C+et+al%2E+2020%2E+The+impact+of+the+general+data+protection+regulation+%28gdpr%29+on+artificial+intelligence%2E)
29. Claudio Savelli, Moreno La Quatra, Alkis Koudounas, and Flavio Giobergia. 2026. Svela at evalita 2026: Overview of the selective verification of erasure from llm answers task. In Proceedings of the Ninth Evaluation Campaign of Natural Language Processing and Speech Tools for Italian. [find](https://scholar.google.com/scholar?q=Claudio+Savelli%2C+Moreno+La+Quatra%2C+Alkis+Koudounas%2C+and+Flavio+Giobergia%2E+2026%2E+Svela+at+evalita+2026%3A+Overview+of+the+selective+verification+of+erasure+from+llm+answers+task%2E+In+Proceedings+of+the+Ni)
30. Final Workshop (EVALITA 2026), Bari, Italy. CEUR.org. [find](https://scholar.google.com/scholar?q=Final+Workshop+%28EVALITA+2026%29%2C+Bari%2C+Italy%2E+CEUR%2Eorg%2E)
31. Weijia Shi, Jaechan Lee, Yangsibo Huang, Sadhika Malladi, Jieyu Zhao, Ari Holtzman, Daogao Liu, Luke Zettlemoyer, Noah A Smith, and Chiyuan Zhang. 2024. Muse: Machine unlearning six-way evaluation for language models. arXiv preprint arXiv:2407.06460. — [arXiv:2407.06460](https://arxiv.org/abs/2407.06460) [find](https://scholar.google.com/scholar?q=Weijia+Shi%2C+Jaechan+Lee%2C+Yangsibo+Huang%2C+Sadhika+Malladi%2C+Jieyu+Zhao%2C+Ari+Holtzman%2C+Daogao+Liu%2C+Luke+Zettlemoyer%2C+Noah+A+Smith%2C+and+Chiyuan+Zhang%2E+2024%2E+Muse%3A+Machine+unlearning+six%2Dway+evaluation+for)
32. Pratiksha Thaker, Shengyuan Hu, Neil Kale, Yash Maurya, Zhiwei Steven Wu, and Virginia Smith. 2025. Position: Llm unlearning benchmarks are weak measures of progress. In 2025 IEEE Conference on Secure and Trustworthy Machine Learning (SaTML), pages 520–533. IEEE. [find](https://scholar.google.com/scholar?q=Pratiksha+Thaker%2C+Shengyuan+Hu%2C+Neil+Kale%2C+Yash+Maurya%2C+Zhiwei+Steven+Wu%2C+and+Virginia+Smith%2E+2025%2E+Position%3A+Llm+unlearning+benchmarks+are+weak+measures+of+progress%2E+In+2025+IEEE+Conference+on+Secure)
33. Qizhou Wang, Bo Han, Puning Yang, Jianing Zhu, Tongliang Liu, and Masashi Sugiyama. 2025. Towards effective evaluations and comparisons for LLM unlearning methods. In The Thirteenth International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Qizhou+Wang%2C+Bo+Han%2C+Puning+Yang%2C+Jianing+Zhu%2C+Tongliang+Liu%2C+and+Masashi+Sugiyama%2E+2025%2E+Towards+effective+evaluations+and+comparisons+for+LLM+unlearning+methods%2E+In+The+Thirteenth+International+Conf)
34. Jin Yao, Eli Chien, Minxin Du, Xinyao Niu, Tianhao Wang, Zezhou Cheng, and Xiang Yue. 2024. Machine unlearning of pre-trained large language models. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 8403–8419, Bangkok, Thailand.… [find](https://scholar.google.com/scholar?q=Jin+Yao%2C+Eli+Chien%2C+Minxin+Du%2C+Xinyao+Niu%2C+Tianhao+Wang%2C+Zezhou+Cheng%2C+and+Xiang+Yue%2E+2024%2E+Machine+unlearning+of+pre%2Dtrained+large+language+models%2E+In+Proceedings+of+the+62nd+Annual+Meeting+of+the+As)