--- title: "Red Teaming Language Model Detectors with Language Models" description: "They introduce two attacks. 3 references, cited by 33 papers here. Among them: 29." authors: ["Zhouxing Shi", "Yihan Wang", "Fan Yin", "Xiangning Chen", "Kai-Wei Chang", "Cho-Jui Hsieh"] year: 2024 venue: "Transactions of the Association for Computational Linguistics" source: "https://aclanthology.org/2024.tacl-1.10/" doi: "10.1162/tacl_a_00639" licence: "CC BY 4.0 (ACL Anthology)" licence_url: "https://creativecommons.org/licenses/by/4.0/" summary_licence: "Elamur Cosmic License" canonical: "https://research.elamur.ai/free/en/papers/shi-2024-red-teaming-language-model-detectors/" --- [Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/shi-2024-red-teaming-language-model-detectors/) [home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2024](https://md.research.elamur.ai/free/en/papers/2024/page/7/) / Red Teaming Language Model Detectors with Language Models # Red Teaming Language Model Detectors with Language Models **Authors:** [Zhouxing Shi](https://md.research.elamur.ai/free/en/authors/zhouxing-shi/), [Yihan Wang](https://md.research.elamur.ai/free/en/authors/yihan-wang/), [Fan Yin](https://md.research.elamur.ai/free/en/authors/fan-yin/), Xiangning Chen, [Kai-Wei Chang](https://md.research.elamur.ai/free/en/authors/kai-wei-chang/) and [Cho-Jui Hsieh](https://md.research.elamur.ai/free/en/authors/cho-jui-hsieh/) · **Year:** 2024 · **Venue:** [Transactions of the Association for Computational Linguistics](https://md.research.elamur.ai/free/en/venues/tacl/) ([Transactions of the Association for Computational Linguistics (TACL) 2024](https://md.research.elamur.ai/free/en/venues/tacl/2024/)) · **Publisher:** [The MIT Press](https://md.research.elamur.ai/free/en/publishers/the-mit-press/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/) **Links:** [Publisher's page](https://aclanthology.org/2024.tacl-1.10/) · [PDF](https://aclanthology.org/2024.tacl-1.10.pdf) · [DOI 10.1162/tacl\_a\_00639](https://doi.org/10.1162/tacl_a_00639) · [OpenAlex](https://openalex.org/W4391876565) **Citations:** 29 citations (OpenAlex, 2026-10-14) · [33](https://md.research.elamur.ai/free/en/papers/shi-2024-red-teaming-language-model-detectors/cited-by/) in our library, of the 149,439 papers whose bibliographies we read ## Summary Detectors that flag LLM-generated text are proposed as a safeguard against misuse such as plagiarism or mass misinformation, but it was unclear how well they resist deliberate evasion. The authors stress-test three detector families: a trained classifier (OpenAI's AI Text Classifier), the watermarking scheme of Kirchenbauer et al. (2023), and the likelihood-based DetectGPT. Their setting is harder than earlier paraphrasing attacks because the attacker's helper LLM is assumed to be protected by a detector too. They introduce two attacks. The first asks the auxiliary LLM for context-aware synonym candidates and chooses replacements either without consulting the detector or through a query-based evolutionary search. The second targets instruction-following models like ChatGPT. It uses a small training subset to find one fixed extra instruction that pushes the writing style toward text the classifier misses. On XSum and ELI5, every detector loses substantial AUROC and detection rate. Human raters still judge the attacked outputs as reasonably fluent and plausible. The work shows current detection tools are not yet trustworthy. ## Key points - Attacks cover all three detector types. Word substitution works against classifier, watermark, and likelihood detectors. Instructional prompts target classifier-based detection of instruction-tuned LLMs and need no test-time queries. - The prompt search drew on a training set of n = 50 examples and ran for T = 5 rounds, considering K = 5 candidates in each round. The OpenAI classifier was evaluated at a 0.9 threshold for detection rate. The found prompts beat paraphrasing with ChatGPT. - Watermarking proved the most robust of the three, which the authors attribute to its statistical nature. Query-based substitution generally gave the strongest attacks, while query-free substitution tended to preserve fluency better. - Human evaluation ran on Amazon MTurk. It covered the first 20 test examples per dataset, with 3 workers each rating fluency and plausibility on a 1–3 scale. Quality dropped only slightly. - Proposed defence: evading the watermark typically requires swapping about 20% of tokens from greenlist to redlist. Checking redlist-token likelihood or perplexity under the watermarked model could therefore expose such tampering. Code is available at github.com/shizhouxing/LLM-Detector-Robustness. *Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.* ## Institutions - [University of California, Los Angeles](https://md.research.elamur.ai/free/en/universities/university-of-california-los-angeles/) — University, Los Angeles, United States · 99 works · 56,330 citations 📚 Shelves (5): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 18,614 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 17,965 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 17,450 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 17,296 works · [👁️ Computer Vision and Pattern Recognition](https://md.research.elamur.ai/free/en/shelves/computer-science/computer-vision-and-pattern-recognition/) · 3,486 works 🧭 Topics (2): [🧠 Language models & pretraining](https://md.research.elamur.ai/free/en/topics/language-models/) · 3,894 works · [🛡️ Bias, fairness & safety](https://md.research.elamur.ai/free/en/topics/bias-fairness-safety/) · 587 works ## Cite Cite as: Zhouxing Shi, Yihan Wang, Fan Yin, Xiangning Chen, Kai-Wei Chang and Cho-Jui Hsieh. 2024. Red Teaming Language Model Detectors with Language Models. Transactions of the Association for Computational Linguistics. The paper itself, not this page. BibTeX from the library's record of the work. ```bibtex @article{shi2024red, title = {{Red Teaming Language Model Detectors with Language Models}}, author = {Zhouxing Shi, Yihan Wang, Fan Yin, Xiangning Chen, Kai-Wei Chang and Cho-Jui Hsieh}, journal = {Transactions of the Association for Computational Linguistics}, year = {2024}, doi = {10.1162/tacl\_a\_00639}, url = {https://doi.org/10.1162/tacl_a_00639} } ``` - [← previous OpenScholar: Synthesizing Scientific Literature with Retrieval-augmented LMs](https://md.research.elamur.ai/free/en/papers/asai-2024-openscholar-synthesizing-scientific-literature/) - [next → ToMBench: Benchmarking Theory of Mind in Large Language Models](https://md.research.elamur.ai/free/en/papers/chen-2024-tombench-benchmarking-theory-mind-large/) - [↑ back to papers of 2024](https://md.research.elamur.ai/free/en/papers/2024/page/7/) ## References 3 references, 1 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet. 1. Moustafa Alzantot, Yash Sharma, Ahmed Elgohary, Bo-Jhang Ho, Mani Srivastava, and Kai-Wei Chang. 2018. [Generating natural language adversarial examples](https://md.research.elamur.ai/free/en/papers/alzantot-2018-generating-natural-language-adversarial-examples/). In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 2890–2896, Brussels, Belgium. https://doi.org/10.1… 2. Anas Awadalla, Mitchell Wortsman, Gabriel Ilharco, Sewon Min, Ian Magnusson, Hannaneh Hajishirzi, and Ludwig Schmidt. 2022. Exploring the landscape of distributional robustness for question answering models. In Findings of the Association for Computational Linguistics: EMNLP 2022, pages 5971–5987. h… [find](https://scholar.google.com/scholar?q=Anas+Awadalla%2C+Mitchell+Wortsman%2C+Gabriel+Ilharco%2C+Sewon+Min%2C+Ian+Magnusson%2C+Hannaneh+Hajishirzi%2C+and+Ludwig+Schmidt%2E+2022%2E+Exploring+the+landscape+of+distributional+robustness+for+question+answering+) 3. Conclusion and Limitations Melika Behjati, Seyed-Mohsen Moosavi-Dezfooli, Mahdieh Soleymani Baghshah, and Pascal Frossard. 2019. Universal adversarial attacks on text classifiers. In ICASSP 2019-2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 7345–7349.… — [doi:10.1109/icassp.2019.8682430](https://doi.org/10.1109/icassp.2019.8682430) [find](https://scholar.google.com/scholar?q=Conclusion+and+Limitations+Melika+Behjati%2C+Seyed%2DMohsen+Moosavi%2DDezfooli%2C+Mahdieh+Soleymani+Baghshah%2C+and+Pascal+Frossard%2E+2019%2E+Universal+adversarial+attacks+on+text+classifiers%2E+In+ICASSP+2019%2D2019+) *Updated 31 October 2026*