---
title: "EntroBench — paper by Qin et al. (2026) — summary and ideas"
description: "Some test only open-ended news continuation on C4, and others test only low-entropy code generation. 48 references."
authors: ["Pengyuan Qin", "Linnan Tu", "Yuhan Ke", "Hefei Ling"]
year: 2026
venue: "Findings of the Association for Computational Linguistics: ACL 2026"
source: "https://aclanthology.org/2026.findings-acl.2089/"
doi: "10.18653/v1/2026.findings-acl.2089"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/qin-2026-entrobench-evaluating-llm-watermarking-under/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/qin-2026-entrobench-evaluating-llm-watermarking-under/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/29/) / EntroBench: Evaluating LLM Watermarking Under Multi-Entropy Scenarios and Practical User Operations

# EntroBench: Evaluating LLM Watermarking Under Multi-Entropy Scenarios and Practical User Operations

**Authors:** Pengyuan Qin, Linnan Tu, Yuhan Ke and [Hefei Ling](https://md.research.elamur.ai/free/en/authors/hefei-ling/) · **Year:** 2026 · **Venue:** [Findings of the Association for Computational Linguistics: ACL 2026](https://md.research.elamur.ai/free/en/venues/findings/) ([Findings of the Association for Computational Linguistics 2026](https://md.research.elamur.ai/free/en/venues/findings/2026/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2026.findings-acl.2089/) · [PDF](https://aclanthology.org/2026.findings-acl.2089.pdf) · [DOI 10.18653/v1/2026.findings-acl.2089](https://doi.org/10.18653/v1/2026.findings-acl.2089) · [OpenAlex](https://openalex.org/W7166881796)

**Citations:** 0 citations (OpenAlex, 2026-10-14)

## Summary
Most evaluations of LLM watermarking use a single entropy regime. Some test only open-ended news continuation on C4, and others test only low-entropy code generation. Robustness tests usually cover deliberate attacks such as paraphrasing, not the everyday edits users make to model output. EntroBench addresses both gaps. It groups tasks into low, medium and high entropy using the spike entropy measure from Kirchenbauer et al., and it covers seven tasks, including mixed text-and-code outputs. To compare eight watermarking schemes fairly, the authors align watermark strength on a high-entropy anchor dataset through hyper-parameter search. They then measure detectability and downstream quality across all entropy levels. A component called LLM-as-a-User simulates harmless post-processing: summarizing, converting to bullet points, and keeping only code. No method performs consistently across entropy levels, and stronger detectability tends to cost output quality. Ordinary user edits can largely erase watermark signals. These results show that practical, non-adversarial use is a serious and under-studied threat to provenance verification.

## Key points
- The benchmark draws on C4 RealNewsLike, WMT23, GSM8K, MATH-500, MMLU, MBPP and a Mix (code plus explanation) setting. Generation models are Llama2-7B/13B-chat and Qwen-2.5-7B/14B-Instruct.
- The eight methods evaluated are KGW, Unigram, SynthID-Text, DiPMark, MorphMark, SWEET, EWD and IE.
- EWD detects watermarks well across entropy levels but lowers output quality. DiPMark preserves downstream quality best, which the authors attribute to its distribution-preserving design.
- Robustness is measured with TPR@1%FPR, using GPT-4o and Llama2-13B-chat as simulated users. On C4, many methods fall from about 0.98 to roughly 0.1–0.2. Unigram holds up best under Summarize and Bullet Points.
- Entropy-aware methods lose more detectability after user edits, likely because removing surrounding context disrupts their entropy estimates. Code and data: github.com/py-qin/EntroBench.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Huazhong University of Science and Technology](https://md.research.elamur.ai/free/en/universities/huazhong-university-of-science-and-technology/) — University, Wuhan, China · 45 works · 6,345 citations
- [Wuhan University](https://md.research.elamur.ai/free/en/universities/wuhan-university/) — University, Wuhan, China · 35 works · 1,127 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [👁️ Computer Vision and Pattern Recognition](https://md.research.elamur.ai/free/en/shelves/computer-science/computer-vision-and-pattern-recognition/) · 2,869 works · [🔌 Hardware and Architecture](https://md.research.elamur.ai/free/en/shelves/computer-science/hardware-and-architecture/) · 81 works

🧭 Topics (1): [🧠 Language models & pretraining](https://md.research.elamur.ai/free/en/topics/language-models/) · 3,320 works

## Cite

Cite as: Pengyuan Qin, Linnan Tu, Yuhan Ke and Hefei Ling. 2026. EntroBench: Evaluating LLM Watermarking Under Multi-Entropy Scenarios and Practical User Operations. Findings of the Association for Computational Linguistics: ACL 2026.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{qin2026entrobench,
  title = {{EntroBench: Evaluating LLM Watermarking Under Multi-Entropy Scenarios and Practical User Operations}},
  author = {Pengyuan Qin, Linnan Tu, Yuhan Ke and Hefei Ling},
  booktitle = {Findings of the Association for Computational Linguistics: ACL 2026},
  year = {2026},
  doi = {10.18653/v1/2026.findings-acl.2089},
  url = {https://doi.org/10.18653/v1/2026.findings-acl.2089}
}
```

- [← previous entity-linkings: A Unified Library for Entity Linking](https://md.research.elamur.ai/free/en/papers/sawada-2026-entity-linkings-unified-library-entity/)
- [next → Entropy Scheduling in Reinforcement Learning for Large Language Models](https://md.research.elamur.ai/free/en/papers/wang-2026-entropy-scheduling-reinforcement-learning-large/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/29/)

## References

48 references, 16 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Scott Aaronson. 2022. My AI safety lecture for UT effective altruism. [https://scottaaronson.blog/](https://scottaaronson.blog/) ?p=6823. [↗](https://scottaaronson.blog/)
2. Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, and 1 others. 2023. [Gpt-4 technical report](https://md.research.elamur.ai/free/en/papers/openai-2023-gpt-4-technical-report/). arXiv preprint arXiv:2303.08774.
3. Mansour Al Ghanim, Jiaqi Xue, Rochana Prih Hastuti, Mengxin Zheng, Yan Solihin, and Qian Lou. 2025. Evaluating the robustness and accuracy of text watermarking under real-world cross-lingual manipulations. In Findings of the Association for Computational Linguistics: EMNLP 2025, pages 7396–7416, Suz… [find](https://scholar.google.com/scholar?q=Mansour+Al+Ghanim%2C+Jiaqi+Xue%2C+Rochana+Prih+Hastuti%2C+Mengxin+Zheng%2C+Yan+Solihin%2C+and+Qian+Lou%2E+2025%2E+Evaluating+the+robustness+and+accuracy+of+text+watermarking+under+real%2Dworld+cross%2Dlingual+manipulat)
4. Hussain Alibrahim and Simone A Ludwig. 2021. Hyperparameter optimization: Comparing genetic algorithm against grid search and bayesian optimization. In 2021 IEEE congress on evolutionary computation (CEC), pages 1551–1559. IEEE. [find](https://scholar.google.com/scholar?q=Hussain+Alibrahim+and+Simone+A+Ludwig%2E+2021%2E+Hyperparameter+optimization%3A+Comparing+genetic+algorithm+against+grid+search+and+bayesian+optimization%2E+In+2021+IEEE+congress+on+evolutionary+computation+%28)
5. Jacob Austin, Augustus Odena, Maxwell Nye, Maarten Bosma, Henryk Michalewski, David Dohan, Ellen Jiang, Carrie Cai, Michael Terry, Quoc Le, and 1 others. 2021. [Program synthesis with large language models](https://md.research.elamur.ai/free/en/papers/austin-2021-program-synthesis-large-language-models/). arXiv preprint arXiv:2108.07732.
6. Miranda Christ, Sam Gunn, and Or Zamir. 2024. Undetectable watermarks for language models. In Proceedings of Thirty Seventh Conference on Learning Theory, volume 247 of Proceedings of Machine Learning Research, pages 1125–1139. PMLR. [find](https://scholar.google.com/scholar?q=Miranda+Christ%2C+Sam+Gunn%2C+and+Or+Zamir%2E+2024%2E+Undetectable+watermarks+for+language+models%2E+In+Proceedings+of+Thirty+Seventh+Conference+on+Learning+Theory%2C+volume+247+of+Proceedings+of+Machine+Learning)
7. Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, and 1 others. 2021. [Training verifiers to solve math word problems](https://md.research.elamur.ai/free/en/papers/cobbe-2021-training-verifiers-solve-math-word/). arXiv preprint arXiv:2110.14168.
8. Sumanth Dathathri, Abigail See, Sumedh Ghaisas, Po-Sen Huang, Rob McAdam, Johannes Welbl, Vandana Bachani, Alex Kaskasoli, Robert Stanforth, Tatiana Matejovicova, and 1 others. 2024. Scalable watermarking for identifying large language model outputs. Nature, 634(8035):818–823. [find](https://scholar.google.com/scholar?q=Sumanth+Dathathri%2C+Abigail+See%2C+Sumedh+Ghaisas%2C+Po%2DSen+Huang%2C+Rob+McAdam%2C+Johannes+Welbl%2C+Vandana+Bachani%2C+Alex+Kaskasoli%2C+Robert+Stanforth%2C+Tatiana+Matejovicova%2C+and+1+others%2E+2024%2E+Scalable+watermar)
9. Tianle Gu, Zongqi Wang, Kexin Huang, Yuanqi Yao, Xiangliang Zhang, Yujiu Yang, and Xiuying Chen. 2025. Invisible entropy: Towards safe and efficient low-entropy LLM watermarking. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pages 6716–6733, Suzhou, China… [find](https://scholar.google.com/scholar?q=Tianle+Gu%2C+Zongqi+Wang%2C+Kexin+Huang%2C+Yuanqi+Yao%2C+Xiangliang+Zhang%2C+Yujiu+Yang%2C+and+Xiuying+Chen%2E+2025%2E+Invisible+entropy%3A+Towards+safe+and+efficient+low%2Dentropy+LLM+watermarking%2E+In+Proceedings+of+the)
10. Biyang Guo, Xin Zhang, Ziyuan Wang, Minqi Jiang, Jinran Nie, Yuxuan Ding, Jianwei Yue, and Yupeng Wu. 2023. [How close is chatgpt to human experts? comparison corpus, evaluation, and detection](https://md.research.elamur.ai/free/en/papers/guo-2023-close-chatgpt-human-experts-comparison/). arXiv preprint arXiv:2301.07597.
11. Zhiwei He, Binglin Zhou, Hongkun Hao, Aiwei Liu, Xing Wang, Zhaopeng Tu, Zhuosheng Zhang, and Rui Wang. 2024. Can watermarks survive translation? on the cross-lingual consistency of text watermark for large language models. In Proceedings of the 62nd Annual Meeting of the Association for Computation… [find](https://scholar.google.com/scholar?q=Zhiwei+He%2C+Binglin+Zhou%2C+Hongkun+Hao%2C+Aiwei+Liu%2C+Xing+Wang%2C+Zhaopeng+Tu%2C+Zhuosheng+Zhang%2C+and+Rui+Wang%2E+2024%2E+Can+watermarks+survive+translation%3F+on+the+cross%2Dlingual+consistency+of+text+watermark+for)
12. Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. 2020. [Measuring massive multitask language understanding](https://md.research.elamur.ai/free/en/papers/hendrycks-2020-measuring-massive-multitask-language-understanding/). arXiv preprint arXiv:2009.03300.
13. Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. 2021. [Measuring mathematical problem solving with the math dataset](https://md.research.elamur.ai/free/en/papers/hendrycks-2021-measuring-mathematical-problem-solving-math/). arXiv preprint arXiv:2103.03874.
14. Abe Hou, Jingyu Zhang, Tianxing He, Yichen Wang, Yung-Sung Chuang, Hongwei Wang, Lingfeng Shen, Benjamin Van Durme, Daniel Khashabi, and Yulia Tsvetkov. 2024a. SemStamp: A semantic watermark with paraphrastic robustness for text generation. In Proceedings of the 2024 Conference of the North American… [find](https://scholar.google.com/scholar?q=Abe+Hou%2C+Jingyu+Zhang%2C+Tianxing+He%2C+Yichen+Wang%2C+Yung%2DSung+Chuang%2C+Hongwei+Wang%2C+Lingfeng+Shen%2C+Benjamin+Van+Durme%2C+Daniel+Khashabi%2C+and+Yulia+Tsvetkov%2E+2024a%2E+SemStamp%3A+A+semantic+watermark+with+para)
15. Abe Hou, Jingyu Zhang, Yichen Wang, Daniel Khashabi, and Tianxing He. 2024b. k-SemStamp: A clustering-based semantic watermark for detection of machine-generated text. In Findings of the Association for Computational Linguistics: ACL 2024, pages 1706–1715, Bangkok, Thailand. Association for Computat… [find](https://scholar.google.com/scholar?q=Abe+Hou%2C+Jingyu+Zhang%2C+Yichen+Wang%2C+Daniel+Khashabi%2C+and+Tianxing+He%2E+2024b%2E+k%2DSemStamp%3A+A+clustering%2Dbased+semantic+watermark+for+detection+of+machine%2Dgenerated+text%2E+In+Findings+of+the+Association+f)
16. Beining Huang, Du Su, Fei Sun, Qi Cao, Huawei Shen, and Xueqi Cheng. 2025. Low-entropy watermark detection via Bayes’ rule derived detector. In Findings of the Association for Computational Linguistics: ACL 2025, pages 14330–14344, Vienna, Austria. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Beining+Huang%2C+Du+Su%2C+Fei+Sun%2C+Qi+Cao%2C+Huawei+Shen%2C+and+Xueqi+Cheng%2E+2025%2E+Low%2Dentropy+watermark+detection+via+Bayes%E2%80%99+rule+derived+detector%2E+In+Findings+of+the+Association+for+Computational+Linguistic)
17. Jungin Kim, Shinwoo Park, and Yo-Sub Han. 2025. Marking code without breaking it: Code watermarking for detecting llm-generated code. arXiv preprint arXiv:2502.18851. [find](https://scholar.google.com/scholar?q=Jungin+Kim%2C+Shinwoo+Park%2C+and+Yo%2DSub+Han%2E+2025%2E+Marking+code+without+breaking+it%3A+Code+watermarking+for+detecting+llm%2Dgenerated+code%2E+arXiv+preprint+arXiv%3A2502%2E18851%2E)
18. John Kirchenbauer, Jonas Geiping, Yuxin Wen, Jonathan Katz, Ian Miers, and Tom Goldstein. 2023. [A watermark for large language models](https://md.research.elamur.ai/free/en/papers/kirchenbauer-2023-watermark-large-language-models/). In International Conference on Machine Learning, pages 17061–17084. PMLR.
19. Tom Kocmi, Eleftherios Avramidis, Rachel Bawden, Ondřej Bojar, Anton Dvorkovich, Christian Federmann, Mark Fishel, Markus Freitag, Thamme Gowda, Roman Grundkiewicz, Barry Haddow, Philipp Koehn, Benjamin Marie, Christof Monz, Makoto Morishita, Kenton Murray, Masaaki Nagata, Toshiaki Nakazawa, Martin… [find](https://scholar.google.com/scholar?q=Tom+Kocmi%2C+Eleftherios+Avramidis%2C+Rachel+Bawden%2C+Ondr%CC%8Cej+Bojar%2C+Anton+Dvorkovich%2C+Christian+Federmann%2C+Mark+Fishel%2C+Markus+Freitag%2C+Thamme+Gowda%2C+Roman+Grundkiewicz%2C+Barry+Haddow%2C+Philipp+Koehn%2C+Benja)
20. Rohith Kuditipudi, John Thickstun, Tatsunori Hashimoto, and Percy Liang. 2023. Robust distortion-free watermarks for language models. arXiv preprint arXiv:2307.15593. [find](https://scholar.google.com/scholar?q=Rohith+Kuditipudi%2C+John+Thickstun%2C+Tatsunori+Hashimoto%2C+and+Percy+Liang%2E+2023%2E+Robust+distortion%2Dfree+watermarks+for+language+models%2E+arXiv+preprint+arXiv%3A2307%2E15593%2E)
21. Taehyun Lee, Seokhee Hong, Jaewoo Ahn, Ilgee Hong, Hwaran Lee, Sangdoo Yun, Jamin Shin, and Gunhee Kim. 2024. [Who wrote this code? watermarking for code generation](https://md.research.elamur.ai/free/en/papers/lee-2024-who-wrote-this-code-watermarking/). In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 4890–4911, Ba…
22. Yafu Li, Qintong Li, Leyang Cui, Wei Bi, Zhilin Wang, Longyue Wang, Linyi Yang, Shuming Shi, and Yue Zhang. 2024. [Mage: Machine-generated text detection in the wild](https://md.research.elamur.ai/free/en/papers/li-2024-mage-machine-generated-text-detection/). In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 36–53.
23. Jiacheng Liang, Zian Wang, Spencer Hong, Shouling Ji, and Ting Wang. 2025. Watermark under fire: A robustness evaluation of LLM watermarking. In Findings of the Association for Computational Linguistics: EMNLP 2025, pages 21050–21074, Suzhou, China. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Jiacheng+Liang%2C+Zian+Wang%2C+Spencer+Hong%2C+Shouling+Ji%2C+and+Ting+Wang%2E+2025%2E+Watermark+under+fire%3A+A+robustness+evaluation+of+LLM+watermarking%2E+In+Findings+of+the+Association+for+Computational+Linguisti)
24. Hunter Lightman, Vineet Kosaraju, Yuri Burda, Harrison Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever, and Karl Cobbe. 2023. [Let’s verify step by step](https://md.research.elamur.ai/free/en/papers/lightman-2023-let-s-verify-step-step/). In The Twelfth International Conference on Learning Representations.
25. Aiwei Liu, Leyi Pan, Xuming Hu, Shuang Li, Lijie Wen, Irwin King, and Philip S Yu. 2024a. An unforgeable publicly verifiable watermark for large language models. In ICLR. [find](https://scholar.google.com/scholar?q=Aiwei+Liu%2C+Leyi+Pan%2C+Xuming+Hu%2C+Shuang+Li%2C+Lijie+Wen%2C+Irwin+King%2C+and+Philip+S+Yu%2E+2024a%2E+An+unforgeable+publicly+verifiable+watermark+for+large+language+models%2E+In+ICLR%2E)
26. Aiwei Liu, Leyi Pan, Xuming Hu, Shiao Meng, and Lijie Wen. 2023. A semantic invariant robust watermark for large language models. arXiv preprint arXiv:2310.06356. — [arXiv:2310.06356](https://arxiv.org/abs/2310.06356) [find](https://scholar.google.com/scholar?q=Aiwei+Liu%2C+Leyi+Pan%2C+Xuming+Hu%2C+Shiao+Meng%2C+and+Lijie+Wen%2E+2023%2E+A+semantic+invariant+robust+watermark+for+large+language+models%2E+arXiv+preprint+arXiv%3A2310%2E06356%2E+%E2%80%94+%5BarXiv%3A2310%2E06356%5D%28https%3A%2F%2Farxiv%2Eor)
27. Aiwei Liu, Leyi Pan, Yijian Lu, Jingjing Li, Xuming Hu, Xi Zhang, Lijie Wen, Irwin King, Hui Xiong, and Philip Yu. 2024b. A survey of text watermarking in the era of large language models. ACM Comput. Surv., 57(2). [find](https://scholar.google.com/scholar?q=Aiwei+Liu%2C+Leyi+Pan%2C+Yijian+Lu%2C+Jingjing+Li%2C+Xuming+Hu%2C+Xi+Zhang%2C+Lijie+Wen%2C+Irwin+King%2C+Hui+Xiong%2C+and+Philip+Yu%2E+2024b%2E+A+survey+of+text+watermarking+in+the+era+of+large+language+models%2E+ACM+Comput%2E)
28. Yepeng Liu and Yuheng Bu. 2024. Adaptive text watermark for large language models. arXiv preprint arXiv:2401.13927. — [arXiv:2401.13927](https://arxiv.org/abs/2401.13927) [find](https://scholar.google.com/scholar?q=Yepeng+Liu+and+Yuheng+Bu%2E+2024%2E+Adaptive+text+watermark+for+large+language+models%2E+arXiv+preprint+arXiv%3A2401%2E13927%2E+%E2%80%94+%5BarXiv%3A2401%2E13927%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2401%2E13927%29)
29. Yijian Lu, Aiwei Liu, Dianzhi Yu, Jingjing Li, and Irwin King. 2024. An entropy-based text watermarking detection method. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 11724– 11735, Bangkok, Thailand. Association for Computa… [find](https://scholar.google.com/scholar?q=Yijian+Lu%2C+Aiwei+Liu%2C+Dianzhi+Yu%2C+Jingjing+Li%2C+and+Irwin+King%2E+2024%2E+An+entropy%2Dbased+text+watermarking+detection+method%2E+In+Proceedings+of+the+62nd+Annual+Meeting+of+the+Association+for+Computational)
30. James B McQueen. 1967. Some methods of classification and analysis of multivariate observations. In Proc. of 5th Berkeley Symposium on Math. Stat. and Prob., pages 281–297. [find](https://scholar.google.com/scholar?q=James+B+McQueen%2E+1967%2E+Some+methods+of+classification+and+analysis+of+multivariate+observations%2E+In+Proc%2E+of+5th+Berkeley+Symposium+on+Math%2E+Stat%2E+and+Prob%2E%2C+pages+281%E2%80%93297%2E)
31. Eric Mitchell, Yoonho Lee, Alexander Khazatsky, Christopher D Manning, and Chelsea Finn. 2023. [Detectgpt: Zero-shot machine-generated text detection using probability curvature](https://md.research.elamur.ai/free/en/papers/mitchell-2023-detectgpt-zero-shot-machine-generated/). In International conference on machine learning, pages 24950–24962. PMLR.
32. OpenAI. 2023. New AI Classifier for Indicating AI-Written Text. [find](https://scholar.google.com/scholar?q=OpenAI%2E+2023%2E+New+AI+Classifier+for+Indicating+AI%2DWritten+Text%2E)
33. Leyi Pan, Aiwei Liu, Zhiwei He, Zitian Gao, Xuandong Zhao, Yijian Lu, Binglin Zhou, Shuliang Liu, Xuming Hu, Lijie Wen, Irwin King, and Philip S. Yu. 2024. MarkLLM: An open-source toolkit for LLM watermarking. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing:… [find](https://scholar.google.com/scholar?q=Leyi+Pan%2C+Aiwei+Liu%2C+Zhiwei+He%2C+Zitian+Gao%2C+Xuandong+Zhao%2C+Yijian+Lu%2C+Binglin+Zhou%2C+Shuliang+Liu%2C+Xuming+Hu%2C+Lijie+Wen%2C+Irwin+King%2C+and+Philip+S%2E+Yu%2E+2024%2E+MarkLLM%3A+An+open%2Dsource+toolkit+for+LLM+wate)
34. Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. [Bleu: a method for automatic evaluation of machine translation](https://md.research.elamur.ai/free/en/papers/papineni-2002-bleu-method-automatic-evaluation-machine/). In Proceedings of the 40th annual meeting of the Association for Computational Linguistics, pages 311–318.
35. Julien Piet, Chawin Sitawarin, Vivian Fang, Norman Mu, and David Wagner. 2025. Markmywords: Analyzing and evaluating language model watermarks. In 2025 IEEE Conference on Secure and Trustworthy Machine Learning (SaTML), pages 68–91. IEEE. [find](https://scholar.google.com/scholar?q=Julien+Piet%2C+Chawin+Sitawarin%2C+Vivian+Fang%2C+Norman+Mu%2C+and+David+Wagner%2E+2025%2E+Markmywords%3A+Analyzing+and+evaluating+language+model+watermarks%2E+In+2025+IEEE+Conference+on+Secure+and+Trustworthy+Machin)
36. Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu. 2020. [Exploring the limits of transfer learning with a unified text-to-text transformer](https://md.research.elamur.ai/free/en/papers/raffel-2019-exploring-limits-transfer-learning-unified/). Journal of machine learning research, 21(140):1–67.
37. Jie Ren, Han Xu, Yiding Liu, Yingqian Cui, Shuaiqiang Wang, Dawei Yin, and Jiliang Tang. 2024a. A robust semantics-based watermark for large language model against paraphrasing. In Findings of the Association for Computational Linguistics: NAACL 2024, pages 613–625, Mexico City, Mexico. Association… [find](https://scholar.google.com/scholar?q=Jie+Ren%2C+Han+Xu%2C+Yiding+Liu%2C+Yingqian+Cui%2C+Shuaiqiang+Wang%2C+Dawei+Yin%2C+and+Jiliang+Tang%2E+2024a%2E+A+robust+semantics%2Dbased+watermark+for+large+language+model+against+paraphrasing%2E+In+Findings+of+the+Ass)
38. Yubing Ren, Ping Guo, Yanan Cao, and Wei Ma. 2024b. Subtle signatures, strong shields: Advancing robust and imperceptible watermarking in large language models. In Findings of the Association for Computational Linguistics: ACL 2024, pages 5508–5519, Bangkok, Thailand. Association for Computational L… [find](https://scholar.google.com/scholar?q=Yubing+Ren%2C+Ping+Guo%2C+Yanan+Cao%2C+and+Wei+Ma%2E+2024b%2E+Subtle+signatures%2C+strong+shields%3A+Advancing+robust+and+imperceptible+watermarking+in+large+language+models%2E+In+Findings+of+the+Association+for+Comp)
39. Vinu Sankar Sadasivan, Aounon Kumar, Sriram Balasubramanian, Wenxiao Wang, and Soheil Feizi. 2023. [Can ai-generated text be reliably detected?](https://md.research.elamur.ai/free/en/papers/sadasivan-2023-ai-generated-text-reliably-detected/) arXiv preprint arXiv:2303.11156.
40. Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, and 1 others. 2023. [Llama 2: Open foundation and fine-tuned chat models](https://md.research.elamur.ai/free/en/papers/touvron-2023-llama-2-open-foundation-fine/). arXiv preprint arXiv:2307.09288.
41. Shangqing Tu, Yuliang Sun, Yushi Bai, Jifan Yu, Lei Hou, and Juanzi Li. 2024. WaterBench: Towards holistic evaluation of watermarks for large language models. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1517–1542, Bangkok,… [find](https://scholar.google.com/scholar?q=Shangqing+Tu%2C+Yuliang+Sun%2C+Yushi+Bai%2C+Jifan+Yu%2C+Lei+Hou%2C+and+Juanzi+Li%2E+2024%2E+WaterBench%3A+Towards+holistic+evaluation+of+watermarks+for+large+language+models%2E+In+Proceedings+of+the+62nd+Annual+Meeting)
42. Yidan Wang, Yubing Ren, Yanan Cao, and Binxing Fang. 2025a. From trade-off to synergy: A versatile symbiotic watermarking framework for large language models. arXiv preprint arXiv:2505.09924. [find](https://scholar.google.com/scholar?q=Yidan+Wang%2C+Yubing+Ren%2C+Yanan+Cao%2C+and+Binxing+Fang%2E+2025a%2E+From+trade%2Doff+to+synergy%3A+A+versatile+symbiotic+watermarking+framework+for+large+language+models%2E+arXiv+preprint+arXiv%3A2505%2E09924%2E)
43. Zongqi Wang, Tianle Gu, Baoyuan Wu, and Yujiu Yang. 2025b. [MorphMark: Flexible adaptive watermarking for large language models](https://md.research.elamur.ai/free/en/papers/wang-2025-morphmark-flexible-adaptive-watermarking-large/). In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 4842–4860, Vienna, Austria. Association for Comput…
44. John Wieting, Kevin Gimpel, Graham Neubig, and Taylor Berg-kirkpatrick. 2022. Paraphrastic representations at scale. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pages 379–388, Abu Dhabi, UAE. Association for Computational Linguist… [find](https://scholar.google.com/scholar?q=John+Wieting%2C+Kevin+Gimpel%2C+Graham+Neubig%2C+and+Taylor+Berg%2Dkirkpatrick%2E+2022%2E+Paraphrastic+representations+at+scale%2E+In+Proceedings+of+the+2022+Conference+on+Empirical+Methods+in+Natural+Language+Proc)
45. BenLong Wu, Kejiang Chen, Yanru He, Guoqiang Chen, Weiming Zhang, and Nenghai Yu. 2024a. Codewm-bench: An automated benchmark for code watermarking evaluation. In Proceedings of the ACM Turing Award Celebration Conference-China 2024, pages 120–125. [find](https://scholar.google.com/scholar?q=BenLong+Wu%2C+Kejiang+Chen%2C+Yanru+He%2C+Guoqiang+Chen%2C+Weiming+Zhang%2C+and+Nenghai+Yu%2E+2024a%2E+Codewm%2Dbench%3A+An+automated+benchmark+for+code+watermarking+evaluation%2E+In+Proceedings+of+the+ACM+Turing+Award+C)
46. Yihan Wu, Zhengmian Hu, Junfeng Guo, Hongyang Zhang, and Heng Huang. 2024b. A resilient and accessible distribution-preserving watermark for large language models. In International Conference on Machine Learning, pages 53443–53470. PMLR. [find](https://scholar.google.com/scholar?q=Yihan+Wu%2C+Zhengmian+Hu%2C+Junfeng+Guo%2C+Hongyang+Zhang%2C+and+Heng+Huang%2E+2024b%2E+A+resilient+and+accessible+distribution%2Dpreserving+watermark+for+large+language+models%2E+In+International+Conference+on+Machi)
47. An Yang, Baosong Yang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Zhou, Chengpeng Li, Chengyuan Li, Dayiheng Liu, Fei Huang, Guanting Dong, Haoran Wei, Huan Lin, Jialong Tang, Jialin Wang, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Ma, and 40 others. 2024. Qwen2 technical report. arXiv preprint arXi… [find](https://scholar.google.com/scholar?q=An+Yang%2C+Baosong+Yang%2C+Binyuan+Hui%2C+Bo+Zheng%2C+Bowen+Yu%2C+Chang+Zhou%2C+Chengpeng+Li%2C+Chengyuan+Li%2C+Dayiheng+Liu%2C+Fei+Huang%2C+Guanting+Dong%2C+Haoran+Wei%2C+Huan+Lin%2C+Jialong+Tang%2C+Jialin+Wang%2C+Jian+Yang%2C+Jian)
48. Jingyuan Yi, Zeqiu Xu, Tianyi Huang, and Peiyang Yu. 2025. Challenges and innovations in llm-powered fake news detection: A synthesis of approaches and future directions. In Proceedings of the 2025 2nd International Conference on Generative Artificial Intelligence and Information Security, pages 87–… [find](https://scholar.google.com/scholar?q=Jingyuan+Yi%2C+Zeqiu+Xu%2C+Tianyi+Huang%2C+and+Peiyang+Yu%2E+2025%2E+Challenges+and+innovations+in+llm%2Dpowered+fake+news+detection%3A+A+synthesis+of+approaches+and+future+directions%2E+In+Proceedings+of+the+2025+2n)

*Updated 1 November 2026*