---
title: "Does DetectGPT Fully Utilize Perturbation? — Liu 2024"
description: "The paper tackles detection of machine-generated text (MGT) and addresses three weaknesses of DetectGPT."
authors: ["Shengchao Liu", "Xiaoming Liu", "Yichen Wang", "Zehua Cheng", "Chengzhengxu Li", "Zhaohan Zhang", "Yu Lan", "Chao Shen"]
year: 2024
venue: "Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)"
source: "https://aclanthology.org/2024.acl-long.103/"
doi: "10.18653/v1/2024.acl-long.103"
arxiv: "2402.00263"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/papers/liu-2024-does-detectgpt-fully-utilize-perturbation/"
---
[Elamur Research](https://md.research.elamur.ai/) · [this page in HTML](https://research.elamur.ai/free/papers/liu-2024-does-detectgpt-fully-utilize-perturbation/)

[home](https://md.research.elamur.ai/) / [papers](https://md.research.elamur.ai/free/papers/) / [2024](https://md.research.elamur.ai/free/papers/2024/page/2/) / Does DetectGPT Fully Utilize Perturbation? Bridging Selective Perturbation to Fine-tuned Contrastive Learning Detector would be Better

# Does DetectGPT Fully Utilize Perturbation? Bridging Selective Perturbation to Fine-tuned Contrastive Learning Detector would be Better

**Authors:** Shengchao Liu, Xiaoming Liu, Yichen Wang, Zehua Cheng, Chengzhengxu Li, Zhaohan Zhang, Yu Lan, Chao Shen · **Year:** 2024 · **Venue:** Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2024.acl-long.103/) · [PDF](https://aclanthology.org/2024.acl-long.103.pdf) · [DOI 10.18653/v1/2024.acl-long.103](https://doi.org/10.18653/v1/2024.acl-long.103) · [arXiv 2402.00263](https://arxiv.org/abs/2402.00263) · [OpenAlex](https://openalex.org/W4402671968)

**Citations:** 5 citations (openalex-snapshot · 2026-10-14) · [11](#cited-by) in our library, of the 140673 papers whose bibliographies we read

## Summary
The paper tackles detection of machine-generated text (MGT) and addresses three weaknesses of DetectGPT. Its random perturbations can inject noise. It reduces perturbation to a single probability-difference score. Its threshold-based logit regression generalizes poorly and fails on single or small-batch inputs. The authors propose PECOLA, a fine-tuned detector that combines the metric-based and fine-tuned approaches in two stages. First, Selective Strategy Perturbation uses a token-level extension of YAKE to score token importance. Unimportant tokens are more likely to be masked or removed, and a mask-filling LM rewrites the masked spans. Second, a token-level weighted multi-pairwise contrastive learning module compares original and perturbed texts. It reuses the importance scores as weights and is trained jointly with cross-entropy loss. In few-shot experiments on four public datasets, PECOLA beats the previous state of the art by 1.20% accuracy on average. It also proves robust across domains, generators, mask-filling models and short inputs. The result matters because PECOLA needs no threshold, works on individual texts and requires little labeled data.

## Key points
- Average few-shot gains are +1.20% over SOTA, +3.84% over the newest metric-based detectors and +1.62% over fine-tuned detectors. PECOLA outperforms all eight baselines.
- Importance-based selective masking has the lowest mask-filling failure ratio, 5.56% versus 9.20% for random masking. Keyword and probability-rank strategies fall in between.
- Every tested mask-filling model beats the baseline, with only small variation between them. The models were Bert, Bart, GPT-2, Twhin-bert, XLM, XLNet, RoBERTa and LLaMA-2 (7B).
- Generalization:
  - Cross-domain on HC3 (Medicine, Finance, Comp. Sci., 64-shot): 84.51% average accuracy, against 82.88% for DetectGPT and 79.90% for RoBERTa.
  - Cross-generator between the GPT3.5 and HC3 datasets: margins of 10.21% and 6.98% over the best competitor.
- On short chunks of 50–200 tokens, PECOLA beats RoBERTa by 4.16% on GPT-2 data and 2.13% on HC3 data. Code and data: github.com/lsc-1/Pecola.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

Fields: [Linguistics & languages](https://md.research.elamur.ai/free/shelves/linguistics/)

## Cited by

Cited by

11 papers of our library cite it (of the 140673 whose bibliographies we read); 1 of them have a page here, 10 not yet.

- [SearchLLM: Detecting LLM Paraphrased Text by Measuring the Similarity with Regeneration of the Candidate Source via Search Engine](https://md.research.elamur.ai/free/papers/son-2026-searchllm-detecting-llm-paraphrased-text/)

## Cite

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{liu2024does,
  title = {{Does DetectGPT Fully Utilize Perturbation? Bridging Selective Perturbation to Fine-tuned Contrastive Learning Detector would be Better}},
  author = {Liu, Shengchao and Liu, Xiaoming and Wang, Yichen and Cheng, Zehua and Li, Chengzhengxu and Zhang, Zhaohan and Lan, Yu and Shen, Chao},
  booktitle = {Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2024},
  doi = {10.18653/v1/2024.acl-long.103},
  eprint = {2402.00263},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.18653/v1/2024.acl-long.103}
}
```

- [← previous Do NOT Think That Much for 2+3=? On the Overthinking of o1-Like LLMs](https://md.research.elamur.ai/free/papers/chen-2024-think-that-much-2-3/)
- [next → Dreaming Out Loud: A Self-Synthesis Approach For Training Vision-Language Models With Developmentally Plausible Data](https://md.research.elamur.ai/free/papers/alkhamissi-2024-dreaming-out-loud-self-synthesis/)
- [↑ back to papers of 2024](https://md.research.elamur.ai/free/papers/2024/page/2/)

## References

61 references, 16 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Conneau Alexis, Khandelwal Kartikay, Goyal Naman, Chaudhary Vishrav, Wenzek Guillaume, Guzmán Francisco, Grave Edouard, Ott Myle, Zettlemoyer Luke, and Stoyanov Veselin. 2020. Unsupervised cross-lingual representation learning at scale. In Annual Meeting of the Association for Computational Linguist… [find](https://scholar.google.com/scholar?q=Conneau+Alexis%2C+Khandelwal+Kartikay%2C+Goyal+Naman%2C+Chaudhary+Vishrav%2C+Wenzek+Guillaume%2C+Guzm%C3%A1n+Francisco%2C+Grave+Edouard%2C+Ott+Myle%2C+Zettlemoyer+Luke%2C+and+Stoyanov+Veselin%2E+2020%2E+Unsupervised+cross%2Dlingu)
2. Guangsheng Bao, Yanbin Zhao, Zhiyang Teng, Linyi Yang, and Yue Zhang. 2024. Fast-detectGPT: Efficient zero-shot detection of machine-generated text via conditional probability curvature. In The Twelfth International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Guangsheng+Bao%2C+Yanbin+Zhao%2C+Zhiyang+Teng%2C+Linyi+Yang%2C+and+Yue+Zhang%2E+2024%2E+Fast%2DdetectGPT%3A+Efficient+zero%2Dshot+detection+of+machine%2Dgenerated+text+via+conditional+probability+curvature%2E+In+The+Twelft) [our card](https://md.research.elamur.ai/free/papers/bao-2023-fast-detectgpt-efficient-zero-shot/)
3. Stella Biderman, Hailey Schoelkopf, Quentin Anthony, Herbie Bradley, Kyle O’Brien, Eric Hallahan, Mohammad Aflah Khan, Shivanshu Purohit, USVSN Sai Prashanth, Edward Raff, Aviya Skowron, Lintang Sutawika, and Oskar van der Wal. 2023. Pythia: A suite for analyzing large language models across trainin… [find](https://scholar.google.com/scholar?q=Stella+Biderman%2C+Hailey+Schoelkopf%2C+Quentin+Anthony%2C+Herbie+Bradley%2C+Kyle+O%E2%80%99Brien%2C+Eric+Hallahan%2C+Mohammad+Aflah+Khan%2C+Shivanshu+Purohit%2C+USVSN+Sai+Prashanth%2C+Edward+Raff%2C+Aviya+Skowron%2C+Lintang+Sutaw)
4. Sid Black, Stella Biderman, Eric Hallahan, Quentin Anthony, Leo Gao, Laurence Golding, Horace He, Connor Leahy, Kyle McDonell, Jason Phang, et al. 2022. Gpt-neox-20b: An open-source autoregressive language model. arXiv preprint arXiv:2204.06745. [find](https://scholar.google.com/scholar?q=Sid+Black%2C+Stella+Biderman%2C+Eric+Hallahan%2C+Quentin+Anthony%2C+Leo+Gao%2C+Laurence+Golding%2C+Horace+He%2C+Connor+Leahy%2C+Kyle+McDonell%2C+Jason+Phang%2C+et+al%2E+2022%2E+Gpt%2Dneox%2D20b%3A+An+open%2Dsource+autoregressive+lan) [our card](https://md.research.elamur.ai/free/papers/black-2022-gpt-neox-20b-open-source/)
5. Ricardo Campos, Vítor Mangaravite, Arian Pasquali, Alípio Jorge, Célia Nunes, and Adam Jatowt. 2020. Yake! keyword extraction from single documents using multiple local features. Information Sciences, 509:257–289. [find](https://scholar.google.com/scholar?q=Ricardo+Campos%2C+V%C3%ADtor+Mangaravite%2C+Arian+Pasquali%2C+Al%C3%ADpio+Jorge%2C+C%C3%A9lia+Nunes%2C+and+Adam+Jatowt%2E+2020%2E+Yake%21+keyword+extraction+from+single+documents+using+multiple+local+features%2E+Information+Sciences%2C)
6. Asli Celikyilmaz, Elizabeth Clark, and Jianfeng Gao. 2020. Evaluation of text generation: A survey. Computing Research Repository, abs/2006.14799. [find](https://scholar.google.com/scholar?q=Asli+Celikyilmaz%2C+Elizabeth+Clark%2C+and+Jianfeng+Gao%2E+2020%2E+Evaluation+of+text+generation%3A+A+survey%2E+Computing+Research+Repository%2C+abs%2F2006%2E14799%2E)
7. Jiaao Chen, Zichao Yang, and Diyi Yang. 2020. Mixtext: Linguistically-informed interpolation of hidden space for semi-supervised text classification. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 2147–2157. [find](https://scholar.google.com/scholar?q=Jiaao+Chen%2C+Zichao+Yang%2C+and+Diyi+Yang%2E+2020%2E+Mixtext%3A+Linguistically%2Dinformed+interpolation+of+hidden+space+for+semi%2Dsupervised+text+classification%2E+In+Proceedings+of+the+58th+Annual+Meeting+of+the+A)
8. Yutian Chen, Hao Kang, Vivian Zhai, Liangze Li, Rita Singh, and Bhiksha Ramakrishnan. 2023. Gpt-sentinel: Distinguishing human and chatgpt generated content. arXiv preprint arXiv:2305.07969. — [arXiv:2305.07969](https://arxiv.org/abs/2305.07969) [find](https://scholar.google.com/scholar?q=Yutian+Chen%2C+Hao+Kang%2C+Vivian+Zhai%2C+Liangze+Li%2C+Rita+Singh%2C+and+Bhiksha+Ramakrishnan%2E+2023%2E+Gpt%2Dsentinel%3A+Distinguishing+human+and+chatgpt+generated+content%2E+arXiv+preprint+arXiv%3A2305%2E07969%2E+%E2%80%94+%5BarXiv%3A)
9. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. Bert: Pre-training of deep bidirectional transformers for language understanding. In North American Chapter of the Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Jacob+Devlin%2C+Ming%2DWei+Chang%2C+Kenton+Lee%2C+and+Kristina+Toutanova%2E+2019%2E+Bert%3A+Pre%2Dtraining+of+deep+bidirectional+transformers+for+language+understanding%2E+In+North+American+Chapter+of+the+Association+f)
10. Yingtong Dou, Guixiang Ma, Philip S Yu, and Sihong Xie. 2020. Robust spammer detection by nash reinforcement learning. In Proceedings of the 26th ACM SIGKDD international conference on knowledge discovery & data mining, pages 924–933. [find](https://scholar.google.com/scholar?q=Yingtong+Dou%2C+Guixiang+Ma%2C+Philip+S+Yu%2C+and+Sihong+Xie%2E+2020%2E+Robust+spammer+detection+by+nash+reinforcement+learning%2E+In+Proceedings+of+the+26th+ACM+SIGKDD+international+conference+on+knowledge+disco)
11. Ethan Fetaya, Joern-Henrik Jacobsen, Will Grathwohl, and Richard Zemel. 2020. Understanding the limitations of conditional generative models. In International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Ethan+Fetaya%2C+Joern%2DHenrik+Jacobsen%2C+Will+Grathwohl%2C+and+Richard+Zemel%2E+2020%2E+Understanding+the+limitations+of+conditional+generative+models%2E+In+International+Conference+on+Learning+Representations%2E)
12. Jun Gao, Changlong Yu, Wei Wang, Huan Zhao, and Ruifeng Xu. 2022. Mask-then-fill: A flexible and effective data augmentation framework for event extraction. In Conference on Empirical Methods in Natural Language Processing. [find](https://scholar.google.com/scholar?q=Jun+Gao%2C+Changlong+Yu%2C+Wei+Wang%2C+Huan+Zhao%2C+and+Ruifeng+Xu%2E+2022%2E+Mask%2Dthen%2Dfill%3A+A+flexible+and+effective+data+augmentation+framework+for+event+extraction%2E+In+Conference+on+Empirical+Methods+in+Natur)
13. Sebastian Gehrmann, Hendrik Strobelt, and Alexander M Rush. 2019. Gltr: Statistical detection and visualization of generated text. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics: System Demonstrations, pages 111–116. [find](https://scholar.google.com/scholar?q=Sebastian+Gehrmann%2C+Hendrik+Strobelt%2C+and+Alexander+M+Rush%2E+2019%2E+Gltr%3A+Statistical+detection+and+visualization+of+generated+text%2E+In+Proceedings+of+the+57th+Annual+Meeting+of+the+Association+for+Comp) [our card](https://md.research.elamur.ai/free/papers/gehrmann-2019-gltr-statistical-detection-visualization-generated/)
14. Beliz Gunel, Jingfei Du, Alexis Conneau, and Ves Stoyanov. 2021. Supervised contrastive learning for pre-trained language model fine-tuning. In International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Beliz+Gunel%2C+Jingfei+Du%2C+Alexis+Conneau%2C+and+Ves+Stoyanov%2E+2021%2E+Supervised+contrastive+learning+for+pre%2Dtrained+language+model+fine%2Dtuning%2E+In+International+Conference+on+Learning+Representations%2E) [our card](https://md.research.elamur.ai/free/papers/gunel-2020-supervised-contrastive-learning-pre-trained/)
15. Biyang Guo, Xin Zhang, Ziyuan Wang, Minqi Jiang, Jinran Nie, Yuxuan Ding, Jianwei Yue, and Yupeng Wu. 2023. How close is chatgpt to human experts? comparison corpus, evaluation, and detection. arXiv preprint arXiv:2301.07597. [find](https://scholar.google.com/scholar?q=Biyang+Guo%2C+Xin+Zhang%2C+Ziyuan+Wang%2C+Minqi+Jiang%2C+Jinran+Nie%2C+Yuxuan+Ding%2C+Jianwei+Yue%2C+and+Yupeng+Wu%2E+2023%2E+How+close+is+chatgpt+to+human+experts%3F+comparison+corpus%2C+evaluation%2C+and+detection%2E+arXiv+p) [our card](https://md.research.elamur.ai/free/papers/guo-2023-close-chatgpt-human-experts-comparison/)
16. Xiaomeng Hu, Pin-Yu Chen, and Tsung-Yi Ho. 2023. Radar: Robust ai-text detection via adversarial learning. arXiv preprint arXiv:2307.03838. [find](https://scholar.google.com/scholar?q=Xiaomeng+Hu%2C+Pin%2DYu+Chen%2C+and+Tsung%2DYi+Ho%2E+2023%2E+Radar%3A+Robust+ai%2Dtext+detection+via+adversarial+learning%2E+arXiv+preprint+arXiv%3A2307%2E03838%2E)
17. Hazel H Kim, Daecheol Woo, Seong Joon Oh, Jeong-Won Cha, and Yo-Sub Han. 2022. Alp: Data augmentation using lexicalized pcfgs for few-shot text classification. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 36, pages 10894– 10902. [find](https://scholar.google.com/scholar?q=Hazel+H+Kim%2C+Daecheol+Woo%2C+Seong+Joon+Oh%2C+Jeong%2DWon+Cha%2C+and+Yo%2DSub+Han%2E+2022%2E+Alp%3A+Data+augmentation+using+lexicalized+pcfgs+for+few%2Dshot+text+classification%2E+In+Proceedings+of+the+AAAI+Conference+on)
18. John Kirchenbauer, Jonas Geiping, Yuxin Wen, Jonathan Katz, Ian Miers, and Tom Goldstein. 2023. A watermark for large language models. arXiv preprint arXiv:2301.10226. [find](https://scholar.google.com/scholar?q=John+Kirchenbauer%2C+Jonas+Geiping%2C+Yuxin+Wen%2C+Jonathan+Katz%2C+Ian+Miers%2C+and+Tom+Goldstein%2E+2023%2E+A+watermark+for+large+language+models%2E+arXiv+preprint+arXiv%3A2301%2E10226%2E) [our card](https://md.research.elamur.ai/free/papers/kirchenbauer-2023-watermark-large-language-models/)
19. Robert Kirk, Ishita Mediratta, Christoforos Nalmpantis, Jelena Luketina, Eric Hambro, Edward Grefenstette, and Roberta Raileanu. 2023. Understanding the effects of rlhf on llm generalisation and diversity. arXiv preprint arXiv:2310.06452. [find](https://scholar.google.com/scholar?q=Robert+Kirk%2C+Ishita+Mediratta%2C+Christoforos+Nalmpantis%2C+Jelena+Luketina%2C+Eric+Hambro%2C+Edward+Grefenstette%2C+and+Roberta+Raileanu%2E+2023%2E+Understanding+the+effects+of+rlhf+on+llm+generalisation+and+diver) [our card](https://md.research.elamur.ai/free/papers/kirk-2023-understanding-effects-rlhf-llm-generalisation/)
20. Xiaoming Liu, Zhaohan Zhang, Yichen Wang, Hang Pu, Yu Lan, and Chao Shen. 2023. Coco: Coherence-enhanced machine-generated text detection under low resource with contrastive learning. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 16167–16188. [find](https://scholar.google.com/scholar?q=Xiaoming+Liu%2C+Zhaohan+Zhang%2C+Yichen+Wang%2C+Hang+Pu%2C+Yu+Lan%2C+and+Chao+Shen%2E+2023%2E+Coco%3A+Coherence%2Denhanced+machine%2Dgenerated+text+detection+under+low+resource+with+contrastive+learning%2E+In+Proceedings+o)
21. Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692. [find](https://scholar.google.com/scholar?q=Yinhan+Liu%2C+Myle+Ott%2C+Naman+Goyal%2C+Jingfei+Du%2C+Mandar+Joshi%2C+Danqi+Chen%2C+Omer+Levy%2C+Mike+Lewis%2C+Luke+Zettlemoyer%2C+and+Veselin+Stoyanov%2E+2019%2E+Roberta%3A+A+robustly+optimized+bert+pretraining+approach%2E+a)
22. David Machado, Tiago Barbosa, Sebastião Pais, Bruno Martins, and Gaël Dias. 2009. Universal mobile information retrieval. In Universal Access in Human-Computer Interaction. Intelligent and Ubiquitous Interaction Environments: 5th International Conference, UAHCI 2009, Held as Part of HCI Internationa… [find](https://scholar.google.com/scholar?q=David+Machado%2C+Tiago+Barbosa%2C+Sebasti%C3%A3o+Pais%2C+Bruno+Martins%2C+and+Ga%C3%ABl+Dias%2E+2009%2E+Universal+mobile+information+retrieval%2E+In+Universal+Access+in+Human%2DComputer+Interaction%2E+Intelligent+and+Ubiquitous+)
23. Chengzhi Mao, Carl Vondrick, Hao Wang, and Junfeng Yang. 2024. Raidar: generative AI detection via rewriting. In The Twelfth International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Chengzhi+Mao%2C+Carl+Vondrick%2C+Hao+Wang%2C+and+Junfeng+Yang%2E+2024%2E+Raidar%3A+generative+AI+detection+via+rewriting%2E+In+The+Twelfth+International+Conference+on+Learning+Representations%2E)
24. Lewis Mike, Liu Yinhan, Goyal Naman, Ghazvininejad Marjan, Mohamed Abdelrahman, Levy Omer, Stoyanov Ves, and Zettlemoyer Luke. 2020. Bart: Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension. In Annual Meeting of the Association for Computation… [find](https://scholar.google.com/scholar?q=Lewis+Mike%2C+Liu+Yinhan%2C+Goyal+Naman%2C+Ghazvininejad+Marjan%2C+Mohamed+Abdelrahman%2C+Levy+Omer%2C+Stoyanov+Ves%2C+and+Zettlemoyer+Luke%2E+2020%2E+Bart%3A+Denoising+sequence%2Dto%2Dsequence+pre%2Dtraining+for+natural+langu) [our card](https://md.research.elamur.ai/free/papers/lewis-2020-bart-denoising-sequence-sequence-pre/)
25. George A. Miller. 1992. WordNet: A lexical database for English. In Speech and Natural Language: Proceedings of a Workshop Held at Harriman, New York, February 23-26, 1992. [find](https://scholar.google.com/scholar?q=George+A%2E+Miller%2E+1992%2E+WordNet%3A+A+lexical+database+for+English%2E+In+Speech+and+Natural+Language%3A+Proceedings+of+a+Workshop+Held+at+Harriman%2C+New+York%2C+February+23%2D26%2C+1992%2E) [our card](https://md.research.elamur.ai/free/papers/lex-1994-miller-wordnet/)
26. Fatemehsadat Mireshghallah, Justus Mattern, Sicun Gao, Reza Shokri, and Taylor Berg-Kirkpatrick. 2023. Smaller language models are better black-box machine-generated text detectors. arXiv preprint arXiv:2305.09859. — [arXiv:2305.09859](https://arxiv.org/abs/2305.09859) [find](https://scholar.google.com/scholar?q=Fatemehsadat+Mireshghallah%2C+Justus+Mattern%2C+Sicun+Gao%2C+Reza+Shokri%2C+and+Taylor+Berg%2DKirkpatrick%2E+2023%2E+Smaller+language+models+are+better+black%2Dbox+machine%2Dgenerated+text+detectors%2E+arXiv+preprint+arX)
27. Eric Mitchell, Yoonho Lee, Alexander Khazatsky, Christopher D. Manning, and Chelsea Finn. 2023. Detectgpt: Zero-shot machine-generated text detection using probability curvature. [find](https://scholar.google.com/scholar?q=Eric+Mitchell%2C+Yoonho+Lee%2C+Alexander+Khazatsky%2C+Christopher+D%2E+Manning%2C+and+Chelsea+Finn%2E+2023%2E+Detectgpt%3A+Zero%2Dshot+machine%2Dgenerated+text+detection+using+probability+curvature%2E) [our card](https://md.research.elamur.ai/free/papers/mitchell-2023-detectgpt-zero-shot-machine-generated/)
28. OpenAI. 2019. Gpt-2 output dataset. Website. [find](https://scholar.google.com/scholar?q=OpenAI%2E+2019%2E+Gpt%2D2+output+dataset%2E+Website%2E)
29. OpenAI. 2023. Ai text classifier. Website. [find](https://scholar.google.com/scholar?q=OpenAI%2E+2023%2E+Ai+text+classifier%2E+Website%2E)
30. Piotr P˛ezik, Agnieszka Mikołajczyk-Bareła, Adam Wawrzyński, Bartłomiej Nitoń, and Maciej Ogrodniczuk. 2022. Keyword extraction from short texts with a text-to-text transfer transformer. ACIIDS (Companion), 1716:530–542. [find](https://scholar.google.com/scholar?q=Piotr+P%CB%9Bezik%2C+Agnieszka+Miko%C5%82ajczyk%2DBare%C5%82a%2C+Adam+Wawrzyn%CC%81ski%2C+Bart%C5%82omiej+Niton%CC%81%2C+and+Maciej+Ogrodniczuk%2E+2022%2E+Keyword+extraction+from+short+texts+with+a+text%2Dto%2Dtext+transfer+transformer%2E+ACIIDS+%28Com)
31. Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. 2019. Language models are unsupervised multitask learners. OpenAI blog, 1:9. [find](https://scholar.google.com/scholar?q=Alec+Radford%2C+Jeffrey+Wu%2C+Rewon+Child%2C+David+Luan%2C+Dario+Amodei%2C+Ilya+Sutskever%2C+et+al%2E+2019%2E+Language+models+are+unsupervised+multitask+learners%2E+OpenAI+blog%2C+1%3A9%2E)
32. Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu. 2020. Exploring the limits of transfer learning with a unified text-to-text transformer. The Journal of Machine Learning Research, 21:5485–5551. [find](https://scholar.google.com/scholar?q=Colin+Raffel%2C+Noam+Shazeer%2C+Adam+Roberts%2C+Katherine+Lee%2C+Sharan+Narang%2C+Michael+Matena%2C+Yanqi+Zhou%2C+Wei+Li%2C+and+Peter+J+Liu%2E+2020%2E+Exploring+the+limits+of+transfer+learning+with+a+unified+text%2Dto%2Dtext)
33. Yuhui Shi, Qiang Sheng, Juan Cao, Hao Mi, Beizhe Hu, and Danding Wang. 2024. Ten words only still help: Improving black-box ai-generated text detection via proxy-guided efficient re-sampling. CoRR, abs/2402.09199. — [arXiv:2402.09199](https://arxiv.org/abs/2402.09199) [find](https://scholar.google.com/scholar?q=Yuhui+Shi%2C+Qiang+Sheng%2C+Juan+Cao%2C+Hao+Mi%2C+Beizhe+Hu%2C+and+Danding+Wang%2E+2024%2E+Ten+words+only+still+help%3A+Improving+black%2Dbox+ai%2Dgenerated+text+detection+via+proxy%2Dguided+efficient+re%2Dsampling%2E+CoRR%2C+ab)
34. Eyal Shnarch, Ariel Gera, Alon Halfon, Lena Dankin, Leshem Choshen, Ranit Aharonov, and Noam Slonim. 2022. Cluster & tune: Boost cold start performance in text classification. In Annual Meeting of the Association for Computational Linguistics, page 7639–7653. [find](https://scholar.google.com/scholar?q=Eyal+Shnarch%2C+Ariel+Gera%2C+Alon+Halfon%2C+Lena+Dankin%2C+Leshem+Choshen%2C+Ranit+Aharonov%2C+and+Noam+Slonim%2E+2022%2E+Cluster+%26+tune%3A+Boost+cold+start+performance+in+text+classification%2E+In+Annual+Meeting+of+the)
35. KaShun Shum, Shizhe Diao, and Tong Zhang. 2023. Automatic prompt augmentation and selection with chain-of-thought from labeled data. arXiv preprint arXiv:2302.12822. [find](https://scholar.google.com/scholar?q=KaShun+Shum%2C+Shizhe+Diao%2C+and+Tong+Zhang%2E+2023%2E+Automatic+prompt+augmentation+and+selection+with+chain%2Dof%2Dthought+from+labeled+data%2E+arXiv+preprint+arXiv%3A2302%2E12822%2E)
36. Jihoon Tack, Sangwoo Mo, Jongheon Jeong, and Jin-woo Shin. 2020. Csi: Novelty detection via contrastive learning on distributionally shifted instances. Advances in neural information processing systems, 33:11839–11852. [find](https://scholar.google.com/scholar?q=Jihoon+Tack%2C+Sangwoo+Mo%2C+Jongheon+Jeong%2C+and+Jin%2Dwoo+Shin%2E+2020%2E+Csi%3A+Novelty+detection+via+contrastive+learning+on+distributionally+shifted+instances%2E+Advances+in+neural+information+processing+system)
37. Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. 2023. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288. [find](https://scholar.google.com/scholar?q=Hugo+Touvron%2C+Louis+Martin%2C+Kevin+Stone%2C+Peter+Albert%2C+Amjad+Almahairi%2C+Yasmine+Babaei%2C+Nikolay+Bashlykov%2C+Soumya+Batra%2C+Prajjwal+Bhargava%2C+Shruti+Bhosale%2C+et+al%2E+2023%2E+Llama+2%3A+Open+foundation+and+fi)
38. Christoforos Vasilatos, Manaar Alam, Talal Rahwan, Yasir Zaki, and Michail Maniatakos. 2023. Howkgpt: Investigating the detection of chatgpt-generated university student homework through context-aware perplexity analysis. arXiv preprint arXiv:2305.18226. — [arXiv:2305.18226](https://arxiv.org/abs/2305.18226) [find](https://scholar.google.com/scholar?q=Christoforos+Vasilatos%2C+Manaar+Alam%2C+Talal+Rahwan%2C+Yasir+Zaki%2C+and+Michail+Maniatakos%2E+2023%2E+Howkgpt%3A+Investigating+the+detection+of+chatgpt%2Dgenerated+university+student+homework+through+context%2Daware)
39. Saranya Venkatraman, Adaku Uchendu, and Dongwon Lee. 2023. Gpt-who: An information density-based machine-generated text detector. arXiv preprint arXiv:2310.06202. [find](https://scholar.google.com/scholar?q=Saranya+Venkatraman%2C+Adaku+Uchendu%2C+and+Dongwon+Lee%2E+2023%2E+Gpt%2Dwho%3A+An+information+density%2Dbased+machine%2Dgenerated+text+detector%2E+arXiv+preprint+arXiv%3A2310%2E06202%2E)
40. Rakesh Verma and Daniel Lee. 2017. Extractive summarization: Limits, compression, generalized model and heuristics. Computación y Sistemas, 21:787–798. [find](https://scholar.google.com/scholar?q=Rakesh+Verma+and+Daniel+Lee%2E+2017%2E+Extractive+summarization%3A+Limits%2C+compression%2C+generalized+model+and+heuristics%2E+Computaci%C3%B3n+y+Sistemas%2C+21%3A787%E2%80%93798%2E)
41. Vivek Verma, Eve Fleisig, Nicholas Tomlin, and Dan Klein. 2023. Ghostbuster: Detecting text ghostwritten by large language models. arXiv preprint arXiv:2305.15047. [find](https://scholar.google.com/scholar?q=Vivek+Verma%2C+Eve+Fleisig%2C+Nicholas+Tomlin%2C+and+Dan+Klein%2E+2023%2E+Ghostbuster%3A+Detecting+text+ghostwritten+by+large+language+models%2E+arXiv+preprint+arXiv%3A2305%2E15047%2E)
42. Jan Philip Wahle, Terry Ruas, Frederic Kirstein, and Bela Gipp. 2022. How large language models are transforming machine-paraphrase plagiarism. In Conference on Empirical Methods in Natural Language Processing, page 952–963. [find](https://scholar.google.com/scholar?q=Jan+Philip+Wahle%2C+Terry+Ruas%2C+Frederic+Kirstein%2C+and+Bela+Gipp%2E+2022%2E+How+large+language+models+are+transforming+machine%2Dparaphrase+plagiarism%2E+In+Conference+on+Empirical+Methods+in+Natural+Language+P)
43. Ben Wang. 2021. Mesh-Transformer-JAX: Model-Parallel Implementation of Transformer Language Model with JAX. [https://github.com/](https://github.com/) kingoflolz/mesh-transformer-jax. [↗](https://github.com/)
44. Pengyu Wang, Linyang Li, Ke Ren, Botian Jiang, Dong Zhang, and Xipeng Qiu. 2023. SeqXGPT: Sentence-level AI-generated text detection. In The 2023 Conference on Empirical Methods in Natural Language Processing. [find](https://scholar.google.com/scholar?q=Pengyu+Wang%2C+Linyang+Li%2C+Ke+Ren%2C+Botian+Jiang%2C+Dong+Zhang%2C+and+Xipeng+Qiu%2E+2023%2E+SeqXGPT%3A+Sentence%2Dlevel+AI%2Dgenerated+text+detection%2E+In+The+2023+Conference+on+Empirical+Methods+in+Natural+Language+Pr)
45. Sheng Wang, Jinjiao Lian, Yuzhong Peng, Baoqing Hu, and Hongsong Chen. 2019. Generalized reference evapotranspiration models with limited climatic data based on random forest and gene expression programming in guangxi, china. Agricultural Water Management, 221:220–230. [find](https://scholar.google.com/scholar?q=Sheng+Wang%2C+Jinjiao+Lian%2C+Yuzhong+Peng%2C+Baoqing+Hu%2C+and+Hongsong+Chen%2E+2019%2E+Generalized+reference+evapotranspiration+models+with+limited+climatic+data+based+on+random+forest+and+gene+expression+progr)
46. Xinyi Wang, Hieu Pham, Zihang Dai, and Graham Neubig. 2018. Switchout: an efficient data augmentation algorithm for neural machine translation. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 856–861. [find](https://scholar.google.com/scholar?q=Xinyi+Wang%2C+Hieu+Pham%2C+Zihang+Dai%2C+and+Graham+Neubig%2E+2018%2E+Switchout%3A+an+efficient+data+augmentation+algorithm+for+neural+machine+translation%2E+In+Proceedings+of+the+2018+Conference+on+Empirical+Metho)
47. Yichen Wang, Shangbin Feng, Abe Bohan Hou, Xiao Pu, Chao Shen, Xiaoming Liu, Yulia Tsvetkov, and Tianxing He. 2024. Stumbling blocks: Stress testing the robustness of machine-generated text detectors under attacks. arXiv preprint arXiv:2402.11638. [find](https://scholar.google.com/scholar?q=Yichen+Wang%2C+Shangbin+Feng%2C+Abe+Bohan+Hou%2C+Xiao+Pu%2C+Chao+Shen%2C+Xiaoming+Liu%2C+Yulia+Tsvetkov%2C+and+Tianxing+He%2E+2024%2E+Stumbling+blocks%3A+Stress+testing+the+robustness+of+machine%2Dgenerated+text+detectors+)
48. Jason Wei, Chengyu Huang, Soroush Vosoughi, Yu Cheng, and Shiqi Xu. 2021. Few-shot text classification with triplet networks, data augmentation, and curriculum learning. arXiv preprint arXiv:2103.07552. [find](https://scholar.google.com/scholar?q=Jason+Wei%2C+Chengyu+Huang%2C+Soroush+Vosoughi%2C+Yu+Cheng%2C+and+Shiqi+Xu%2E+2021%2E+Few%2Dshot+text+classification+with+triplet+networks%2C+data+augmentation%2C+and+curriculum+learning%2E+arXiv+preprint+arXiv%3A2103%2E0755)
49. Jason Wei and Kai Zou. 2019. Eda: Easy data augmentation techniques for boosting performance on text classification tasks. In Conference on Empirical Methods in Natural Language Processing, pages 6381–6387. [find](https://scholar.google.com/scholar?q=Jason+Wei+and+Kai+Zou%2E+2019%2E+Eda%3A+Easy+data+augmentation+techniques+for+boosting+performance+on+text+classification+tasks%2E+In+Conference+on+Empirical+Methods+in+Natural+Language+Processing%2C+pages+6381) [our card](https://md.research.elamur.ai/free/papers/wei-2019-eda-easy-data-augmentation-techniques/)
50. Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pier-ric Cistac, Tim Rault, Rémi Louf, Morgan Funtowicz, et al. 2020. Transformers: State-of-the-art natural language processing. In Proceedings of the 2020 conference on empirical methods in natural language p… [find](https://scholar.google.com/scholar?q=Thomas+Wolf%2C+Lysandre+Debut%2C+Victor+Sanh%2C+Julien+Chaumond%2C+Clement+Delangue%2C+Anthony+Moi%2C+Pier%2Dric+Cistac%2C+Tim+Rault%2C+R%C3%A9mi+Louf%2C+Morgan+Funtowicz%2C+et+al%2E+2020%2E+Transformers%3A+State%2Dof%2Dthe%2Dart+natural+l) [our card](https://md.research.elamur.ai/free/papers/wolf-2020-transformers-state-art-natural-language/)
51. Junchao Wu, Shu Yang, Runzhe Zhan, Yulin Yuan, Derek F Wong, and Lidia S Chao. 2023a. A survey on llm-gernerated text detection: Necessity, methods, and future directions. arXiv preprint arXiv:2310.14724. — [arXiv:2310.14724](https://arxiv.org/abs/2310.14724) [find](https://scholar.google.com/scholar?q=Junchao+Wu%2C+Shu+Yang%2C+Runzhe+Zhan%2C+Yulin+Yuan%2C+Derek+F+Wong%2C+and+Lidia+S+Chao%2E+2023a%2E+A+survey+on+llm%2Dgernerated+text+detection%3A+Necessity%2C+methods%2C+and+future+directions%2E+arXiv+preprint+arXiv%3A2310%2E14)
52. Kangxi Wu, Liang Pang, Huawei Shen, Xueqi Cheng, and Tat-Seng Chua. 2023b. LLMDet: A third party large language models generated text detection tool. In The 2023 Conference on Empirical Methods in Natural Language Processing. [find](https://scholar.google.com/scholar?q=Kangxi+Wu%2C+Liang+Pang%2C+Huawei+Shen%2C+Xueqi+Cheng%2C+and+Tat%2DSeng+Chua%2E+2023b%2E+LLMDet%3A+A+third+party+large+language+models+generated+text+detection+tool%2E+In+The+2023+Conference+on+Empirical+Methods+in+Nat)
53. Qizhe Xie, Zihang Dai, Eduard Hovy, Thang Luong, and Quoc Le. 2020. Unsupervised data augmentation for consistency training. Advances in neural information processing systems, 33:6256–6268. [find](https://scholar.google.com/scholar?q=Qizhe+Xie%2C+Zihang+Dai%2C+Eduard+Hovy%2C+Thang+Luong%2C+and+Quoc+Le%2E+2020%2E+Unsupervised+data+augmentation+for+consistency+training%2E+Advances+in+neural+information+processing+systems%2C+33%3A6256%E2%80%936268%2E) [our card](https://md.research.elamur.ai/free/papers/xie-2019-unsupervised-data-augmentation-consistency-training/)
54. Xianjun Yang, Wei Cheng, Linda Petzold, William Yang Wang, and Haifeng Chen. 2023. Dna-gpt: Divergent n-gram analysis for training-free detection of gpt-generated text. arXiv preprint arXiv:2305.17359. — [arXiv:2305.17359](https://arxiv.org/abs/2305.17359) [find](https://scholar.google.com/scholar?q=Xianjun+Yang%2C+Wei+Cheng%2C+Linda+Petzold%2C+William+Yang+Wang%2C+and+Haifeng+Chen%2E+2023%2E+Dna%2Dgpt%3A+Divergent+n%2Dgram+analysis+for+training%2Dfree+detection+of+gpt%2Dgenerated+text%2E+arXiv+preprint+arXiv%3A2305%2E17359)
55. Zhilin Yang, Zihang Dai, Yiming Yang, Jaime Carbonell, Russ R Salakhutdinov, and Quoc V Le. 2019. Xlnet: Generalized autoregressive pretraining for language understanding. Advances in neural information processing systems, 32. [find](https://scholar.google.com/scholar?q=Zhilin+Yang%2C+Zihang+Dai%2C+Yiming+Yang%2C+Jaime+Carbonell%2C+Russ+R+Salakhutdinov%2C+and+Quoc+V+Le%2E+2019%2E+Xlnet%3A+Generalized+autoregressive+pretraining+for+language+understanding%2E+Advances+in+neural+informati) [our card](https://md.research.elamur.ai/free/papers/yang-2019-xlnet-generalized-autoregressive-pretraining-language/)
56. Rowan Zellers, Ari Holtzman, Hannah Rashkin, Yonatan Bisk, Ali Farhadi, Franziska Roesner, and Yejin Choi. 2019. Defending against neural fake news. Advances in neural information processing systems, 32. [find](https://scholar.google.com/scholar?q=Rowan+Zellers%2C+Ari+Holtzman%2C+Hannah+Rashkin%2C+Yonatan+Bisk%2C+Ali+Farhadi%2C+Franziska+Roesner%2C+and+Yejin+Choi%2E+2019%2E+Defending+against+neural+fake+news%2E+Advances+in+neural+information+processing+systems%2C+) [our card](https://md.research.elamur.ai/free/papers/zellers-2019-defending-against-neural-fake-news/)
57. Xiang Zhang, Junbo Zhao, and Yann LeCun. 2015. Character-level convolutional networks for text classification. Advances in neural information processing systems, 28. [find](https://scholar.google.com/scholar?q=Xiang+Zhang%2C+Junbo+Zhao%2C+and+Yann+LeCun%2E+2015%2E+Character%2Dlevel+convolutional+networks+for+text+classification%2E+Advances+in+neural+information+processing+systems%2C+28%2E) [our card](https://md.research.elamur.ai/free/papers/zhang-2015-character-level-convolutional-networks-text/)
58. Xinyang Zhang, Yury Malkov, Omar Florez, Serim Park, Brian McWilliams, Jiawei Han, and Ahmed El-Kishky. 2023. Twhin-bert: A socially-enriched pre-trained language model for multilingual tweet representations at twitter. In Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data… [find](https://scholar.google.com/scholar?q=Xinyang+Zhang%2C+Yury+Malkov%2C+Omar+Florez%2C+Serim+Park%2C+Brian+McWilliams%2C+Jiawei+Han%2C+and+Ahmed+El%2DKishky%2E+2023%2E+Twhin%2Dbert%3A+A+socially%2Denriched+pre%2Dtrained+language+model+for+multilingual+tweet+represen)
59. Wenxuan Zhou, Fangyu Liu, and Muhao Chen. 2021. Contrastive out-of-distribution detection for pre-trained transformers. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 1100–1111. [find](https://scholar.google.com/scholar?q=Wenxuan+Zhou%2C+Fangyu+Liu%2C+and+Muhao+Chen%2E+2021%2E+Contrastive+out%2Dof%2Ddistribution+detection+for+pre%2Dtrained+transformers%2E+In+Proceedings+of+the+2021+Conference+on+Empirical+Methods+in+Natural+Language+P)
60. Implementation Details B.1 This part mentions the hyperparameter settings and meta-information of the HC3 dataset. A.1 Hyperparameter Details Experiments evaluating competitors and P ECOLA follow the setting of CoCo (Liu et al., 2023). The hyperparameter settings of all the methods in the experiment… [find](https://scholar.google.com/scholar?q=Implementation+Details+B%2E1+This+part+mentions+the+hyperparameter+settings+and+meta%2Dinformation+of+the+HC3+dataset%2E+A%2E1+Hyperparameter+Details+Experiments+evaluating+competitors+and+P+ECOLA+follow+the+)
61. Value 30 AdamW 1e-5 0.01 16 2 10% 0.4 RoBERTa-base Training Epochs Optimizer Learning rate Weight Decay Batch Size Mask Gap Mask Proportion Score threshold Pre-trained model [find](https://scholar.google.com/scholar?q=Value+30+AdamW+1e%2D5+0%2E01+16+2+10%25+0%2E4+RoBERTa%2Dbase+Training+Epochs+Optimizer+Learning+rate+Weight+Decay+Batch+Size+Mask+Gap+Mask+Proportion+Score+threshold+Pre%2Dtrained+model)