---
title: "Can LLM Watermarks Robustly Prevent Unauthorized Knowledge"
description: "The authors propose three removal attacks: Untargeted paraphrasing (UP): the training data is… 39 references, cited by 7 papers here."
authors: ["Leyi Pan", "Aiwei Liu", "Shiyu Huang", "Yijian Lu", "Xuming Hu", "Lijie Wen", "Irwin King", "Philip S. Yu"]
year: 2025
venue: "Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)"
source: "https://aclanthology.org/2025.acl-long.648/"
doi: "10.18653/v1/2025.acl-long.648"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/pan-2025-can-llm-watermarks-robustly-prevent/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/pan-2025-can-llm-watermarks-robustly-prevent/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2025](https://md.research.elamur.ai/free/en/papers/2025/page/8/) / Can LLM Watermarks Robustly Prevent Unauthorized Knowledge Distillation?

# Can LLM Watermarks Robustly Prevent Unauthorized Knowledge Distillation?

**Authors:** [Leyi Pan](https://md.research.elamur.ai/free/en/authors/leyi-pan/), [Aiwei Liu](https://md.research.elamur.ai/free/en/authors/aiwei-liu/), [Shiyu Huang](https://md.research.elamur.ai/free/en/authors/shiyu-huang/), Yijian Lu, [Xuming Hu](https://md.research.elamur.ai/free/en/authors/xuming-hu/), [Lijie Wen](https://md.research.elamur.ai/free/en/authors/lijie-wen/), [Irwin King](https://md.research.elamur.ai/free/en/authors/irwin-king/) and [Philip S. Yu](https://md.research.elamur.ai/free/en/authors/philip-s-yu/) · **Year:** 2025 · **Venue:** [Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)](https://md.research.elamur.ai/free/en/venues/acl/) ([ACL 2025](https://md.research.elamur.ai/free/en/venues/acl/2025/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2025.acl-long.648/) · [PDF](https://aclanthology.org/2025.acl-long.648.pdf) · [DOI 10.18653/v1/2025.acl-long.648](https://doi.org/10.18653/v1/2025.acl-long.648) · [OpenAlex](https://openalex.org/W4412945330)

**Citations:** 4 citations (OpenAlex, 2026-10-14) · [7](https://md.research.elamur.ai/free/en/papers/pan-2025-can-llm-watermarks-robustly-prevent/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
Watermarking LLM outputs is promoted as a way to catch unauthorized knowledge distillation. It relies on "radioactivity": student models trained on watermarked teacher text inherit a detectable signal. This paper tests whether that signal survives a deliberate attacker. The authors propose three removal attacks:
- **Untargeted paraphrasing (UP):** the training data is rewritten before distillation.
- **Targeted paraphrasing (TP):** the paraphraser (e.g., Dipper) is steered by inverted watermark rules.
- **Watermark neutralization (WN):** inverse rules are applied while the trained student decodes.

TP and WN depend on a new watermark-stealing method. It works without knowing the scheme or its hyper-parameters and weights the extracted rules by factors that affect radioactivity.

The experiments use GLM-4-9b-chat as teacher and Llama-7b and Llama-3.2-1b as students, testing KGW and SynthID-Text. Both TP and WN push detection down to non-watermarked levels. WN also keeps benchmark performance close to that of normally distilled students, at low cost.

Mixing data from several watermarked teachers weakens detection even without any attack. The results suggest that current n-gram watermarks give weak protection against distillation.

## Key points
- **Setup:** 2 teacher–student pairs × 2 schemes (KGW, SynthID-Text) × 3 hyper-parameter settings, with 200,000 training samples. Extra results for KGW-Minhash and KGW-SkipHash are in Appendix D.
- **Effectiveness:** TP and WN raise detection p-values above 10⁻² in every setting. WN matches baseline students on ARC challenge, TruthfulQA and MTBench.
- **Cost (Llama-7b, 8× H800):** preprocessing takes 31.8 h for UP, 37.1 h for TP and 3.6 h for WN. Only WN adds inference latency, about 0.0068 s/token.
- **Multi-source collision:** two KGW teachers with opposing keys cut detection from 4.21e-28 to roughly 2.8e-01 and 7.2e-01. Using more teachers further hides the watermark while MT-Bench scores remain stable.
- **Recommendation:** the authors call for a more diverse set of token-level watermarking paradigms. They note that sentence-level alternatives add too much latency for real-time deployment.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Tsinghua University](https://md.research.elamur.ai/free/en/universities/tsinghua-university/) — University, Beijing, China · 415 works · 94,028 citations
- [Chinese University of Hong Kong](https://md.research.elamur.ai/free/en/universities/chinese-university-of-hong-kong/) — University, Hong Kong, China · 159 works · 16,154 citations
- [University of Illinois Chicago](https://md.research.elamur.ai/free/en/universities/university-of-illinois-chicago/) — University, Chicago, United States · 70 works · 20,363 citations
- [The Hong Kong University of Science and Technology (Guangzhou)](https://md.research.elamur.ai/free/en/universities/the-hong-kong-university-of-science-and-technology-guangzhou/) — University, Guangzhou, China · 67 works · 1,765 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 18,614 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 17,965 works · [👁️ Computer Vision and Pattern Recognition](https://md.research.elamur.ai/free/en/shelves/computer-science/computer-vision-and-pattern-recognition/) · 3,486 works · [🔌 Hardware and Architecture](https://md.research.elamur.ai/free/en/shelves/computer-science/hardware-and-architecture/) · 167 works

🧭 Topics (2): [🧠 Language models & pretraining](https://md.research.elamur.ai/free/en/topics/language-models/) · 3,894 works · [🪶 Efficient models & compression](https://md.research.elamur.ai/free/en/topics/efficient-models/) · 297 works

## Cite

Cite as: Leyi Pan, Aiwei Liu, Shiyu Huang, Yijian Lu, Xuming Hu, Lijie Wen, Irwin King and Philip S. Yu. 2025. Can LLM Watermarks Robustly Prevent Unauthorized Knowledge Distillation?. Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{pan2025can,
  title = {{Can LLM Watermarks Robustly Prevent Unauthorized Knowledge Distillation?}},
  author = {Leyi Pan, Aiwei Liu, Shiyu Huang, Yijian Lu, Xuming Hu, Lijie Wen, Irwin King and Philip S. Yu},
  booktitle = {Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2025},
  doi = {10.18653/v1/2025.acl-long.648},
  url = {https://doi.org/10.18653/v1/2025.acl-long.648}
}
```

- [← previous Calibrating Verbal Uncertainty as a Linear Feature to Reduce Hallucinations](https://md.research.elamur.ai/free/en/papers/ji-2025-calibrating-verbal-uncertainty-as-linear/)
- [next → Can LLMs Identify Critical Limitations within Scientific Research? A Systematic Evaluation on AI Research Papers](https://md.research.elamur.ai/free/en/papers/xu-2025-can-llms-identify-critical-limitations/)
- [↑ back to papers of 2025](https://md.research.elamur.ai/free/en/papers/2025/page/8/)

## References

39 references, 16 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. S. Aaronson and H. Kirchner. 2022. Watermarking gpt outputs. [https://www.scottaaronson.com/](https://www.scottaaronson.com/) talks/watermark.ppt. [↗](https://www.scottaaronson.com/)
2. Yapei Chang, Kalpesh Krishna, Amir Houmansadr, John Wieting, and Mohit Iyyer. 2024. Postmark: A robust blackbox watermark for large language models. arXiv preprint arXiv:2406.14517. [find](https://scholar.google.com/scholar?q=Yapei+Chang%2C+Kalpesh+Krishna%2C+Amir+Houmansadr%2C+John+Wieting%2C+and+Mohit+Iyyer%2E+2024%2E+Postmark%3A+A+robust+blackbox+watermark+for+large+language+models%2E+arXiv+preprint+arXiv%3A2406%2E14517%2E)
3. Peter Clark, Isaac Cowhey, Oren Etzioni, Tushar Khot, Ashish Sabharwal, Carissa Schoenick, and Oyvind Tafjord. 2018. [Think you have solved question answering? try arc, the ai2 reasoning challenge](https://md.research.elamur.ai/free/en/papers/clark-2018-think-have-solved-question-answering/). ArXiv, abs/1803.05457.
4. Sumanth Dathathri, Abigail See, Sumedh Ghaisas, Po-Sen Huang, Rob McAdam, Johannes Welbl, Vandana Bachani, Alex Kaskasoli, Robert Stanforth, Tatiana Matejovicova, Jamie Hayes, Nidhi Vyas, Majd Al Merey, Jonah Brown-Cohen, Rudy Bunel, Borja Balle, Taylan Cemgil, Zahra Ahmed, Kitty Stacpoole, Ilia Shu… [find](https://scholar.google.com/scholar?q=Sumanth+Dathathri%2C+Abigail+See%2C+Sumedh+Ghaisas%2C+Po%2DSen+Huang%2C+Rob+McAdam%2C+Johannes+Welbl%2C+Vandana+Bachani%2C+Alex+Kaskasoli%2C+Robert+Stanforth%2C+Tatiana+Matejovicova%2C+Jamie+Hayes%2C+Nidhi+Vyas%2C+Majd+Al+Mere)
5. Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al. 2024. [The llama 3 herd of models](https://md.research.elamur.ai/free/en/papers/grattafiori-2024-llama-3-herd-models/). arXiv preprint arXiv:2407.21783.
6. Team GLM, Aohan Zeng, Bin Xu, Bowen Wang, Chenhui Zhang, Da Yin, Diego Rojas, Guanyu Feng, Hanlin Zhao, Hanyu Lai, Hao Yu, Hongning Wang, Jiadai Sun, Jiajie Zhang, Jiale Cheng, Jiayi Gui, Jie Tang, Jing Zhang, Juanzi Li, Lei Zhao, Lindong Wu, Lucen Zhong, Mingdao Liu, Minlie Huang, Peng Zhang, Qinka… — [ChatGLM: A Family of Large Language Models from GLM-130B to GLM-4 All Tools](https://md.research.elamur.ai/free/en/papers/glm-2024-chatglm-family-large-language-models/)
7. Chenchen Gu, Xiang Lisa Li, Percy Liang, and Tatsunori Hashimoto. 2024. On the learnability of watermarks for language models. In The Twelfth International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Chenchen+Gu%2C+Xiang+Lisa+Li%2C+Percy+Liang%2C+and+Tatsunori+Hashimoto%2E+2024%2E+On+the+learnability+of+watermarks+for+language+models%2E+In+The+Twelfth+International+Conference+on+Learning+Representations%2E)
8. Zhiwei He, Binglin Zhou, Hongkun Hao, Aiwei Liu, Xing Wang, Zhaopeng Tu, Zhuosheng Zhang, and Rui Wang. 2024. Can watermarks survive translation? on the cross-lingual consistency of text watermark for large language models. In Proceedings of the 62nd Annual Meeting of the Association for Computation… [find](https://scholar.google.com/scholar?q=Zhiwei+He%2C+Binglin+Zhou%2C+Hongkun+Hao%2C+Aiwei+Liu%2C+Xing+Wang%2C+Zhaopeng+Tu%2C+Zhuosheng+Zhang%2C+and+Rui+Wang%2E+2024%2E+Can+watermarks+survive+translation%3F+on+the+cross%2Dlingual+consistency+of+text+watermark+for)
9. Abe Hou, Jingyu Zhang, Yichen Wang, Daniel Khashabi, and Tianxing He. 2024. k-SemStamp: A clustering-based semantic watermark for detection of machine-generated text. In Findings of the Association for Computational Linguistics: ACL 2024, pages 1706–1715, Bangkok, Thailand. Association for Computati… [find](https://scholar.google.com/scholar?q=Abe+Hou%2C+Jingyu+Zhang%2C+Yichen+Wang%2C+Daniel+Khashabi%2C+and+Tianxing+He%2E+2024%2E+k%2DSemStamp%3A+A+clustering%2Dbased+semantic+watermark+for+detection+of+machine%2Dgenerated+text%2E+In+Findings+of+the+Association+fo)
10. Abe Bohan Hou, Jingyu Zhang, Tianxing He, Yichen Wang, Yung-Sung Chuang, Hongwei Wang, Lingfeng Shen, Benjamin Van Durme, Daniel Khashabi, and Yulia Tsvetkov. 2023. Semstamp: A semantic watermark with paraphrastic robustness for text generation. arXiv preprint arXiv:2310.03991. [find](https://scholar.google.com/scholar?q=Abe+Bohan+Hou%2C+Jingyu+Zhang%2C+Tianxing+He%2C+Yichen+Wang%2C+Yung%2DSung+Chuang%2C+Hongwei+Wang%2C+Lingfeng+Shen%2C+Benjamin+Van+Durme%2C+Daniel+Khashabi%2C+and+Yulia+Tsvetkov%2E+2023%2E+Semstamp%3A+A+semantic+watermark+with)
11. Zhengmian Hu, Lichang Chen, Xidong Wu, Yihan Wu, Hongyang Zhang, and Heng Huang. 2024. Unbiased watermark for large language models. In The Twelfth International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Zhengmian+Hu%2C+Lichang+Chen%2C+Xidong+Wu%2C+Yihan+Wu%2C+Hongyang+Zhang%2C+and+Heng+Huang%2E+2024%2E+Unbiased+watermark+for+large+language+models%2E+In+The+Twelfth+International+Conference+on+Learning+Representations)
12. Nikola Jovanović, Robin Staab, and Martin Vechev. Watermark stealing in large language models. In Forty-first International Conference on Machine Learning. [find](https://scholar.google.com/scholar?q=Nikola+Jovanovic%CC%81%2C+Robin+Staab%2C+and+Martin+Vechev%2E+Watermark+stealing+in+large+language+models%2E+In+Forty%2Dfirst+International+Conference+on+Machine+Learning%2E)
13. John Kirchenbauer, Jonas Geiping, Yuxin Wen, Jonathan Katz, Ian Miers, and Tom Goldstein. 2023. [A watermark for large language models](https://md.research.elamur.ai/free/en/papers/kirchenbauer-2023-watermark-large-language-models/). In International Conference on Machine Learning, ICML 2023, 23-29 July 2023, Honolulu, Hawaii, USA, volume 202 of Proceedings of Machine Learning Research, pages 170…
14. Linguistics (Volume 1: Long Papers), pages 11724– 11735, Bangkok, Thailand. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Linguistics+%28Volume+1%3A+Long+Papers%29%2C+pages+11724%E2%80%93+11735%2C+Bangkok%2C+Thailand%2E+Association+for+Computational+Linguistics%2E)
15. John Kirchenbauer, Jonas Geiping, Yuxin Wen, Manli Shu, Khalid Saifullah, Kezhi Kong, Kasun Fernando, Aniruddha Saha, Micah Goldblum, and Tom Goldstein. 2024. [On the reliability of watermarks for large language models](https://md.research.elamur.ai/free/en/papers/kirchenbauer-2023-reliability-watermarks-large-language-models/). In The Twelfth International Conference on Learning Representations.
16. Travis Munyer and Xin Zhong. 2023. Deeptextmark: Deep learning based text watermarking for detection of large language model generated text. arXiv preprint arXiv:2305.05773. — [arXiv:2305.05773](https://arxiv.org/abs/2305.05773) [find](https://scholar.google.com/scholar?q=Travis+Munyer+and+Xin+Zhong%2E+2023%2E+Deeptextmark%3A+Deep+learning+based+text+watermarking+for+detection+of+large+language+model+generated+text%2E+arXiv+preprint+arXiv%3A2305%2E05773%2E+%E2%80%94+%5BarXiv%3A2305%2E05773%5D%28https)
17. Kalpesh Krishna, Yixiao Song, Marzena Karpinska, John Wieting, and Mohit Iyyer. 2023. [Paraphrasing evades detectors of ai-generated text, but retrieval is an effective defense](https://md.research.elamur.ai/free/en/papers/krishna-2023-paraphrasing-evades-detectors-ai-generated/). arXiv preprint arXiv:2303.13408.
18. Rohith Kuditipudi, John Thickstun, Tatsunori Hashimoto, and Percy Liang. 2024. [Robust distortion-free watermarks for language models](https://md.research.elamur.ai/free/en/papers/kuditipudi-2023-robust-distortion-free-watermarks-language/). Transactions on Machine Learning Research.
19. Taehyun Lee, Seokhee Hong, Jaewoo Ahn, Ilgee Hong, Hwaran Lee, Sangdoo Yun, Jamin Shin, and Gunhee Kim. 2024. [Who wrote this code? watermarking for code generation](https://md.research.elamur.ai/free/en/papers/lee-2024-who-wrote-this-code-watermarking/). In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 4890–4911, Ba…
20. Stephanie Lin, Jacob Hilton, and Owain Evans. 2022. [TruthfulQA: Measuring how models mimic human falsehoods](https://md.research.elamur.ai/free/en/papers/lin-2022-truthfulqa-measuring-how-models-mimic/). In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 3214–3252, Dublin, Ireland. Association for Computational Linguistics…
21. Aiwei Liu, Sheng Guan, Yiming Liu, Leyi Pan, Yifei Zhang, Liancheng Fang, Lijie Wen, Philip S Yu, and Xuming Hu. 2024a. Can watermarked llms be identified by users via crafted prompts? arXiv preprint arXiv:2410.03168. — [arXiv:2410.03168](https://arxiv.org/abs/2410.03168) [find](https://scholar.google.com/scholar?q=Aiwei+Liu%2C+Sheng+Guan%2C+Yiming+Liu%2C+Leyi+Pan%2C+Yifei+Zhang%2C+Liancheng+Fang%2C+Lijie+Wen%2C+Philip+S+Yu%2C+and+Xuming+Hu%2E+2024a%2E+Can+watermarked+llms+be+identified+by+users+via+crafted+prompts%3F+arXiv+preprint+)
22. Aiwei Liu, Leyi Pan, Xuming Hu, Shuang Li, Lijie Wen, Irwin King, and Philip S. Yu. 2024b. An unforgeable publicly verifiable watermark for large language models. In The Twelfth International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Aiwei+Liu%2C+Leyi+Pan%2C+Xuming+Hu%2C+Shuang+Li%2C+Lijie+Wen%2C+Irwin+King%2C+and+Philip+S%2E+Yu%2E+2024b%2E+An+unforgeable+publicly+verifiable+watermark+for+large+language+models%2E+In+The+Twelfth+International+Conferen)
23. Aiwei Liu, Leyi Pan, Xuming Hu, Shiao Meng, and Lijie Wen. 2024c. A semantic invariant robust watermark for large language models. In The Twelfth International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Aiwei+Liu%2C+Leyi+Pan%2C+Xuming+Hu%2C+Shiao+Meng%2C+and+Lijie+Wen%2E+2024c%2E+A+semantic+invariant+robust+watermark+for+large+language+models%2E+In+The+Twelfth+International+Conference+on+Learning+Representations%2E)
24. Aiwei Liu, Leyi Pan, Yijian Lu, Jingjing Li, Xuming Hu, Xi Zhang, Lijie Wen, Irwin King, Hui Xiong, and Philip Yu. 2024d. A survey of text watermarking in the era of large language models. ACM Computing Surveys, 57(2):1–36. [find](https://scholar.google.com/scholar?q=Aiwei+Liu%2C+Leyi+Pan%2C+Yijian+Lu%2C+Jingjing+Li%2C+Xuming+Hu%2C+Xi+Zhang%2C+Lijie+Wen%2C+Irwin+King%2C+Hui+Xiong%2C+and+Philip+Yu%2E+2024d%2E+A+survey+of+text+watermarking+in+the+era+of+large+language+models%2E+ACM+Computi)
25. Yijian Lu, Aiwei Liu, Dianzhi Yu, Jingjing Li, and Irwin King. 2024. An entropy-based text watermarking detection method. In Proceedings of the 62nd Annual Meeting of the Association for Computational Leyi Pan, Aiwei Liu, Zhiwei He, Zitian Gao, Xuandong Zhao, Yijian Lu, Binglin Zhou, Shuliang Liu, X… [find](https://scholar.google.com/scholar?q=Yijian+Lu%2C+Aiwei+Liu%2C+Dianzhi+Yu%2C+Jingjing+Li%2C+and+Irwin+King%2E+2024%2E+An+entropy%2Dbased+text+watermarking+detection+method%2E+In+Proceedings+of+the+62nd+Annual+Meeting+of+the+Association+for+Computational)
26. Leyi Pan, Aiwei Liu, Yijian Lu, Zitian Gao, Yichen Di, Lijie Wen, Irwin King, and Philip S. Yu. 2025. WaterSeeker: Pioneering efficient detection of watermarked segments in large documents. In Findings of the Association for Computational Linguistics: NAACL 2025, pages 2866–2882, Albuquerque, New Me… [find](https://scholar.google.com/scholar?q=Leyi+Pan%2C+Aiwei+Liu%2C+Yijian+Lu%2C+Zitian+Gao%2C+Yichen+Di%2C+Lijie+Wen%2C+Irwin+King%2C+and+Philip+S%2E+Yu%2E+2025%2E+WaterSeeker%3A+Pioneering+efficient+detection+of+watermarked+segments+in+large+documents%2E+In+Finding)
27. Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu. 2020. [Exploring the limits of transfer learning with a unified text-to-text transformer](https://md.research.elamur.ai/free/en/papers/raffel-2019-exploring-limits-transfer-learning-unified/). The Journal of Machine Learning Research, 21(1):5485–5551.
28. Tom Sander, Pierre Fernandez, Alain Oliviero Durmus, Matthijs Douze, and Teddy Furon. 2024. Watermarking makes language models radioactive. In The Thirty-eighth Annual Conference on Neural Information Processing Systems. [find](https://scholar.google.com/scholar?q=Tom+Sander%2C+Pierre+Fernandez%2C+Alain+Oliviero+Durmus%2C+Matthijs+Douze%2C+and+Teddy+Furon%2E+2024%2E+Watermarking+makes+language+models+radioactive%2E+In+The+Thirty%2Deighth+Annual+Conference+on+Neural+Information)
29. Gemini Team, Rohan Anil, Sebastian Borgeaud, Jean-Baptiste Alayrac, Jiahui Yu, Radu Soricut, Johan Schalkwyk, Andrew M Dai, Anja Hauth, Katie Millican, et al. 2023. [Gemini: a family of highly capable multimodal models](https://md.research.elamur.ai/free/en/papers/gemini-team-2023-gemini-family-highly-capable-multimodal/). arXiv preprint arXiv:2312.11805.
30. Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al. 2023. [Llama: Open and efficient foundation language models](https://md.research.elamur.ai/free/en/papers/touvron-2023-llama-open-efficient-foundation-language/). arXiv preprint arXiv:2302.13971.
31. Qilong Wu and Varun Chandrasekaran. 2024. Bypassing LLM watermarks with color-aware substitutions. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 8549–8581, Bangkok, Thailand. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Qilong+Wu+and+Varun+Chandrasekaran%2E+2024%2E+Bypassing+LLM+watermarks+with+color%2Daware+substitutions%2E+In+Proceedings+of+the+62nd+Annual+Meeting+of+the+Association+for+Computational+Linguistics+%28Volume+1%3A)
32. Yihan Wu, Zhengmian Hu, Hongyang Zhang, and Heng Huang. 2023. Dipmark: A stealthy, efficient and resilient watermark for large language models. arXiv preprint arXiv:2310.07710. — [arXiv:2310.07710](https://arxiv.org/abs/2310.07710) [find](https://scholar.google.com/scholar?q=Yihan+Wu%2C+Zhengmian+Hu%2C+Hongyang+Zhang%2C+and+Heng+Huang%2E+2023%2E+Dipmark%3A+A+stealthy%2C+efficient+and+resilient+watermark+for+large+language+models%2E+arXiv+preprint+arXiv%3A2310%2E07710%2E+%E2%80%94+%5BarXiv%3A2310%2E07710%5D%28ht)
33. Xi Yang, Jie Zhang, Kejiang Chen, Weiming Zhang, Zehua Ma, Feng Wang, and Nenghai Yu. 2022. Tracing text provenance via context-aware lexical substitution. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 36, pages 11613–11621. [find](https://scholar.google.com/scholar?q=Xi+Yang%2C+Jie+Zhang%2C+Kejiang+Chen%2C+Weiming+Zhang%2C+Zehua+Ma%2C+Feng+Wang%2C+and+Nenghai+Yu%2E+2022%2E+Tracing+text+provenance+via+context%2Daware+lexical+substitution%2E+In+Proceedings+of+the+AAAI+Conference+on+Art)
34. Zhaoxi Zhang, Xiaomei Zhang, Yanjun Zhang, Leo Yu Zhang, Chao Chen, Shengshan Hu, Asif Gill, and Shirui Pan. 2024. Large language model watermark stealing with mixed integer programming. arXiv preprint arXiv:2405.19677. — [arXiv:2405.19677](https://arxiv.org/abs/2405.19677) [find](https://scholar.google.com/scholar?q=Zhaoxi+Zhang%2C+Xiaomei+Zhang%2C+Yanjun+Zhang%2C+Leo+Yu+Zhang%2C+Chao+Chen%2C+Shengshan+Hu%2C+Asif+Gill%2C+and+Shirui+Pan%2E+2024%2E+Large+language+model+watermark+stealing+with+mixed+integer+programming%2E+arXiv+preprin)
35. Xuandong Zhao, Prabhanjan Vijendra Ananth, Lei Li, and Yu-Xiang Wang. 2024. [Provable robust watermarking for AI-generated text](https://md.research.elamur.ai/free/en/papers/zhao-2023-provable-robust-watermarking-ai-generated/). In The Twelfth International Conference on Learning Representations.
36. Xuandong Zhao, Yu-Xiang Wang, and Lei Li. 2023. Protecting language generation models via invisible watermarking. arXiv preprint arXiv:2302.03162. — [arXiv:2302.03162](https://arxiv.org/abs/2302.03162) [find](https://scholar.google.com/scholar?q=Xuandong+Zhao%2C+Yu%2DXiang+Wang%2C+and+Lei+Li%2E+2023%2E+Protecting+language+generation+models+via+invisible+watermarking%2E+arXiv+preprint+arXiv%3A2302%2E03162%2E+%E2%80%94+%5BarXiv%3A2302%2E03162%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2302%2E03162)
37. Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric Xing, et al. 2023. [Judging llm-as-a-judge with mt-bench and chatbot arena](https://md.research.elamur.ai/free/en/papers/zheng-2023-judging-llm-judge-mt-bench/). arXiv preprint arXiv:2306.05685.
38. Yaowei Zheng, Richong Zhang, Junhao Zhang, Yanhan Ye, Zheyan Luo, Zhangchi Feng, and Yongqiang Ma. 2024. [Llamafactory: Unified efficient fine-tuning of 100+ language models](https://md.research.elamur.ai/free/en/papers/zheng-2024-llamafactory-unified-efficient-fine-tuning/). In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 3: System Demonstrations),…
39. Barret Zoph, Colin Raffel, Dale Schuurmans, Dani Yogatama, Denny Zhou, Don Metzler, Ed H Chi, Jason Wei, Jeff Dean, Liam B Fedus, et al. 2022. [Emergent abilities of large language models](https://md.research.elamur.ai/free/en/papers/wei-2022-emergent-abilities-large-language-models/). TMLR.

*Updated 2 November 2026*