---
title: "Automatic Detection of Generated Text is — Ippolito 2019"
description: "The paper asks why people and automatic classifiers disagree about whether a text came from a person or a language model, and how the decoding strategy…"
authors: ["Daphne Ippolito", "Daniel Duckworth", "Chris Callison-Burch", "Douglas Eck"]
year: 2019
venue: "arXiv:1911.00650"
source: "https://arxiv.org/abs/1911.00650"
doi: "10.48550/arxiv.1911.00650"
arxiv: "1911.00650"
licence: "arXiv — non-exclusive license to distribute"
licence_url: "http://arxiv.org/licenses/nonexclusive-distrib/1.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/ippolito-2019-automatic-detection-generated-text-easiest/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/ippolito-2019-automatic-detection-generated-text-easiest/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2019](https://md.research.elamur.ai/free/en/papers/2019/) / Automatic Detection of Generated Text is Easiest when Humans are Fooled

# Automatic Detection of Generated Text is Easiest when Humans are Fooled

**Authors:** [Daphne Ippolito](https://md.research.elamur.ai/free/en/authors/daphne-ippolito/), [Daniel Duckworth](https://md.research.elamur.ai/free/en/authors/daniel-duckworth/), [Chris Callison-Burch](https://md.research.elamur.ai/free/en/authors/chris-callison-burch/), Douglas Eck · **Year:** 2019 · **Venue:** arXiv:1911.00650 · **Licence:** [arXiv — non-exclusive license to distribute](http://arxiv.org/licenses/nonexclusive-distrib/1.0/)

**Links:** [Publisher's page](https://arxiv.org/abs/1911.00650) · [PDF](https://arxiv.org/pdf/1911.00650) · [DOI 10.48550/arxiv.1911.00650](https://doi.org/10.48550/arxiv.1911.00650) · [arXiv 1911.00650](https://arxiv.org/abs/1911.00650) · [OpenAlex](https://openalex.org/W3021153741)

**Citations:** 243 citations (openalex-snapshot · 2026-10-14) · [163](#cited-by) in our library, of the 140884 papers whose bibliographies we read

## Summary
The paper asks why people and automatic classifiers disagree about whether a text came from a person or a language model, and how the decoding strategy shapes that gap. The authors generated excerpts with GPT-2 Large (774M parameters) using three sampling schemes: untruncated random sampling, top-k (k=40) and nucleus sampling (p=0.96). They paired these with human web text drawn from the same distribution as GPT-2's training data. For each scheme and several excerpt lengths, they fine-tuned BERT-based binary classifiers and also ran evaluations with trained human raters.

The main finding is an inverse relationship. Cutting off the low-probability tail, especially with top-k, makes output more convincing to readers. It also leaves statistical traces, such as a skewed token distribution, that classifiers detect easily. Automatic detectors beat humans in every setting, but they transferred poorly across decoding methods. The authors conclude that judging how human-like generated text is requires both human and automatic detectors.

## Key points
- Per decoding method, the training set contains 250,000 generated excerpts paired with 250,000 webtext excerpts from the gpt-2-output-dataset. An extra 5,000 pairs are held out for validation and testing, and excerpts shorter than 192 WordPiece tokens are removed.
- Top-k output is the hardest for raters but the easiest for classifiers. Raters found untruncated and nucleus (p=0.96) samples equally hard, suggesting that p values well below the 0.92–0.98 range would be needed to seem noticeably more human.
- Human rater accuracy has a median of 74%, which is below the best discriminator. Even multi-sentence excerpts fooled expert raters more than 30% of the time.
- Random pairs of raters agreed on only 59% of excerpts, while raters and discriminators agreed on 61–70%. Humans tend to notice semantic errors such as topic drift and falsehoods, whereas classifiers pick up statistical artifacts.
- Classifiers trained on one strategy generalize poorly to another, but training on mixed data helps. Accuracy grows with excerpt length for both humans and machines. All experiments use English only.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

Fields: [Other](https://md.research.elamur.ai/free/en/shelves/other/)

## Cited by

Cited by

163 papers of our library cite it (of the 140884 whose bibliographies we read); 14 of them have a page here, 149 not yet.

- [The AraGenEval Shared Task on Arabic Authorship Style Transfer and AI Generated Text Detection](https://md.research.elamur.ai/free/en/papers/abudalfa-2025-arageneval-shared-task-arabic-authorship/)
- [Towards a Human-like Open-Domain Chatbot](https://md.research.elamur.ai/free/en/papers/adiwardana-2020-towards-human-like-open-domain/)
- [Fast-DetectGPT: Efficient Zero-Shot Detection of Machine-Generated Text via Conditional Probability Curvature](https://md.research.elamur.ai/free/en/papers/bao-2023-fast-detectgpt-efficient-zero-shot/)
- [On the Opportunities and Risks of Foundation Models](https://md.research.elamur.ai/free/en/papers/bommasani-2021-opportunities-risks-foundation-models/)
- [Extracting Training Data from Large Language Models](https://md.research.elamur.ai/free/en/papers/carlini-2020-extracting-training-data-large-language/)
- [Automatic Detection of Machine Generated Text: A Critical Survey](https://md.research.elamur.ai/free/en/papers/jawahar-2020-automatic-detection-machine-generated-text/)
- [Paraphrasing evades detectors of AI-generated text, but retrieval is an effective defense](https://md.research.elamur.ai/free/en/papers/krishna-2023-paraphrasing-evades-detectors-ai-generated/)
- [GPT detectors are biased against non-native English writers](https://md.research.elamur.ai/free/en/papers/liang-2023-gpt-detectors-biased-against-non/)
- [DetectGPT: Zero-Shot Machine-Generated Text Detection using Probability Curvature](https://md.research.elamur.ai/free/en/papers/mitchell-2023-detectgpt-zero-shot-machine-generated/)
- [MAUVE: Measuring the Gap Between Neural Text and Human Text using Divergence Frontiers](https://md.research.elamur.ai/free/en/papers/pillutla-2021-mauve-measuring-gap-between-neural/)
- [When Translations Surprise: Human Awareness of Predictability in Translations](https://md.research.elamur.ai/free/en/papers/romero-2026-when-translations-surprise-human-awareness/)
- [Can AI-Generated Text be Reliably Detected?](https://md.research.elamur.ai/free/en/papers/sadasivan-2023-ai-generated-text-reliably-detected/)
- [Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models](https://md.research.elamur.ai/free/en/papers/srivastava-2022-imitation-game-quantifying-extrapolating-capabilities/)
- [DetectLLM: Leveraging Log Rank Information for Zero-Shot Detection of Machine-Generated Text](https://md.research.elamur.ai/free/en/papers/su-2023-detectllm-leveraging-log-rank-information/)

## Cite

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@misc{ippolito2019automatic,
  title = {{Automatic Detection of Generated Text is Easiest when Humans are Fooled}},
  author = {Ippolito, Daphne and Duckworth, Daniel and Callison-Burch, Chris and Eck, Douglas},
  year = {2019},
  doi = {10.48550/arxiv.1911.00650},
  eprint = {1911.00650},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.1911.00650}
}
```

- [← previous AudioCaps: Generating Captions for Audios in The Wild](https://md.research.elamur.ai/free/en/papers/kim-2019-audiocaps-generating-captions-audios-wild/)
- [next → Automatic Language Identification in Texts: A Survey](https://md.research.elamur.ai/free/en/papers/jauhiainen-2019-automatic-language-identification-texts/)
- [↑ back to papers of 2019](https://md.research.elamur.ai/free/en/papers/2019/)

## References

34 references, 6 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. David Ifeoluwa Adelani, Haotian Mai, Fuming Fang, Huy H Nguyen, Junichi Yamagishi, and Isao Echizen. 2020. Generating sentiment-preserving fake online reviews using neural language models and their human-and machine-based detection. In International Conference on Advanced Information Networking and… [find](https://scholar.google.com/scholar?q=David+Ifeoluwa+Adelani%2C+Haotian+Mai%2C+Fuming+Fang%2C+Huy+H+Nguyen%2C+Junichi+Yamagishi%2C+and+Isao+Echizen%2E+2020%2E+Generating+sentiment%2Dpreserving+fake+online+reviews+using+neural+language+models+and+their+hu)
2. Hunt Allcott and Matthew Gentzkow. 2017. Social media and fake news in the 2016 election. Journal of economic perspectives, 31(2):211–36. [find](https://scholar.google.com/scholar?q=Hunt+Allcott+and+Matthew+Gentzkow%2E+2017%2E+Social+media+and+fake+news+in+the+2016+election%2E+Journal+of+economic+perspectives%2C+31%282%29%3A211%E2%80%9336%2E)
3. Anton Bakhtin, Sam Gross, Myle Ott, Yuntian Deng, Marc’Aurelio Ranzato, and Arthur Szlam. 2019. [find](https://scholar.google.com/scholar?q=Anton+Bakhtin%2C+Sam+Gross%2C+Myle+Ott%2C+Yuntian+Deng%2C+Marc%E2%80%99Aurelio+Ranzato%2C+and+Arthur+Szlam%2E+2019%2E)
4. Real or fake? learning to discriminate machine from human generated text. arXiv preprint arXiv:1906.03351. — [arXiv:1906.03351](https://arxiv.org/abs/1906.03351) [find](https://scholar.google.com/scholar?q=Real+or+fake%3F+learning+to+discriminate+machine+from+human+generated+text%2E+arXiv+preprint+arXiv%3A1906%2E03351%2E+%E2%80%94+%5BarXiv%3A1906%2E03351%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F1906%2E03351%29)
5. Nicole A Cooke. 2018. Fake news and alternative facts: Information literacy in a post-truth era. American Library Association. [find](https://scholar.google.com/scholar?q=Nicole+A+Cooke%2E+2018%2E+Fake+news+and+alternative+facts%3A+Information+literacy+in+a+post%2Dtruth+era%2E+American+Library+Association%2E)
6. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies,… [find](https://scholar.google.com/scholar?q=Jacob+Devlin%2C+Ming%2DWei+Chang%2C+Kenton+Lee%2C+and+Kristina+Toutanova%2E+2019%2E+BERT%3A+Pre%2Dtraining+of+deep+bidirectional+transformers+for+language+understanding%2E+In+Proceedings+of+the+2019+Conference+of+the+N)
7. Angela Fan, Mike Lewis, and Yann Dauphin. 2018. Hierarchical neural story generation. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 889–898. [find](https://scholar.google.com/scholar?q=Angela+Fan%2C+Mike+Lewis%2C+and+Yann+Dauphin%2E+2018%2E+Hierarchical+neural+story+generation%2E+In+Proceedings+of+the+56th+Annual+Meeting+of+the+Association+for+Computational+Linguistics+%28Volume+1%3A+Long+Papers%29)
8. Sebastian Gehrmann, Hendrik Strobelt, and Alexander M Rush. 2019. Gltr: Statistical detection and visualization of generated text. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics: System Demonstrations, pages 111–116. [find](https://scholar.google.com/scholar?q=Sebastian+Gehrmann%2C+Hendrik+Strobelt%2C+and+Alexander+M+Rush%2E+2019%2E+Gltr%3A+Statistical+detection+and+visualization+of+generated+text%2E+In+Proceedings+of+the+57th+Annual+Meeting+of+the+Association+for+Comp) [our card](https://md.research.elamur.ai/free/en/papers/gehrmann-2019-gltr-statistical-detection-visualization-generated/)
9. Ari Holtzman, Jan Buys, Li Du, Maxwell Forbes, and Yejin Choi. 2020. The curious case of neural text degeneration. In International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Ari+Holtzman%2C+Jan+Buys%2C+Li+Du%2C+Maxwell+Forbes%2C+and+Yejin+Choi%2E+2020%2E+The+curious+case+of+neural+text+degeneration%2E+In+International+Conference+on+Learning+Representations%2E)
10. Anjuli Kannan and Oriol Vinyals. 2017. Adversarial evaluation of dialogue models. arXiv preprint arXiv:1701.08198. — [arXiv:1701.08198](https://arxiv.org/abs/1701.08198) [find](https://scholar.google.com/scholar?q=Anjuli+Kannan+and+Oriol+Vinyals%2E+2017%2E+Adversarial+evaluation+of+dialogue+models%2E+arXiv+preprint+arXiv%3A1701%2E08198%2E+%E2%80%94+%5BarXiv%3A1701%2E08198%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F1701%2E08198%29)
11. Sarah E Kreps, Miles McCain, and Miles Brundage. 2020. All the news thats fit to fabricate: Ai-generated text as a tool of media misinformation. Social Science Research Network. [find](https://scholar.google.com/scholar?q=Sarah+E+Kreps%2C+Miles+McCain%2C+and+Miles+Brundage%2E+2020%2E+All+the+news+thats+fit+to+fabricate%3A+Ai%2Dgenerated+text+as+a+tool+of+media+misinformation%2E+Social+Science+Research+Network%2E)
12. Chris van der Lee, Albert Gatt, Emiel van Miltenburg, Sander Wubben, and Emiel Krahmer. 2019. Best practices for the human evaluation of automatically generated text. In Proceedings of the 12th International Conference on Natural Language Generation, pages 355–368. [find](https://scholar.google.com/scholar?q=Chris+van+der+Lee%2C+Albert+Gatt%2C+Emiel+van+Miltenburg%2C+Sander+Wubben%2C+and+Emiel+Krahmer%2E+2019%2E+Best+practices+for+the+human+evaluation+of+automatically+generated+text%2E+In+Proceedings+of+the+12th+Intern)
13. Jiwei Li, Will Monroe, Tianlin Shi, Sébastien Jean, Alan Ritter, and Dan Jurafsky. 2017. Adversarial learning for neural dialogue generation. arXiv preprint arXiv:1701.06547. [find](https://scholar.google.com/scholar?q=Jiwei+Li%2C+Will+Monroe%2C+Tianlin+Shi%2C+Se%CC%81bastien+Jean%2C+Alan+Ritter%2C+and+Dan+Jurafsky%2E+2017%2E+Adversarial+learning+for+neural+dialogue+generation%2E+arXiv+preprint+arXiv%3A1701%2E06547%2E) [our card](https://md.research.elamur.ai/free/en/papers/li-2017-adversarial-learning-neural-dialogue-generation/)
14. Kevin Lin, Dianqi Li, Xiaodong He, Zhengyou Zhang, and Ming-Ting Sun. 2017. Adversarial ranking for language generation. In Advances in Neural Information Processing Systems, pages 3155–3165. [find](https://scholar.google.com/scholar?q=Kevin+Lin%2C+Dianqi+Li%2C+Xiaodong+He%2C+Zhengyou+Zhang%2C+and+Ming%2DTing+Sun%2E+2017%2E+Adversarial+ranking+for+language+generation%2E+In+Advances+in+Neural+Information+Processing+Systems%2C+pages+3155%E2%80%933165%2E)
15. Peter J. Liu, Mohammad Saleh, Etienne Pot, Ben Goodrich, Ryan Sepassi, Lukasz Kaiser, and Noam Shazeer. 2018. Generating wikipedia by summarizing long sequences. In International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Peter+J%2E+Liu%2C+Mohammad+Saleh%2C+Etienne+Pot%2C+Ben+Goodrich%2C+Ryan+Sepassi%2C+Lukasz+Kaiser%2C+and+Noam+Shazeer%2E+2018%2E+Generating+wikipedia+by+summarizing+long+sequences%2E+In+International+Conference+on+Learnin) [our card](https://md.research.elamur.ai/free/en/papers/liu-2018-generating-wikipedia-summarizing-long-sequences/)
16. Ryan Lowe, Michael Noseworthy, Iulian Vlad Serban, Nicolas Angelard-Gontier, Yoshua Bengio, and Joelle Pineau. 2017. Towards an automatic turing test: Learning to evaluate dialogue responses. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long P… [find](https://scholar.google.com/scholar?q=Ryan+Lowe%2C+Michael+Noseworthy%2C+Iulian+Vlad+Serban%2C+Nicolas+Angelard%2DGontier%2C+Yoshua+Bengio%2C+and+Joelle+Pineau%2E+2017%2E+Towards+an+automatic+turing+test%3A+Learning+to+evaluate+dialogue+responses%2E+In+Proce) [our card](https://md.research.elamur.ai/free/en/papers/lowe-2017-towards-automatic-turing-test-learning/)
17. Lesly Miculicich, Marc Marone, and Hany Hassan. 2019. Selecting, planning, and rewriting: A modular approach for data-to-document generation and translation. [find](https://scholar.google.com/scholar?q=Lesly+Miculicich%2C+Marc+Marone%2C+and+Hany+Hassan%2E+2019%2E+Selecting%2C+planning%2C+and+rewriting%3A+A+modular+approach+for+data%2Dto%2Ddocument+generation+and+translation%2E)
18. EMNLP-IJCNLP 2019, page 289. [find](https://scholar.google.com/scholar?q=EMNLP%2DIJCNLP+2019%2C+page+289%2E)
19. Timothy Niven and Hung-Yu Kao. 2019. Probing neural network comprehension of natural language arguments. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 4658–4664, Florence, Italy. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Timothy+Niven+and+Hung%2DYu+Kao%2E+2019%2E+Probing+neural+network+comprehension+of+natural+language+arguments%2E+In+Proceedings+of+the+57th+Annual+Meeting+of+the+Association+for+Computational+Linguistics%2C+pag)
20. Jekaterina Novikova, Ondřej Dušek, Amanda Cercas Curry, and Verena Rieser. 2017. Why we need new evaluation metrics for nlg. arXiv preprint arXiv:1707.06875. [find](https://scholar.google.com/scholar?q=Jekaterina+Novikova%2C+Ondr%CC%8Cej+Dus%CC%8Cek%2C+Amanda+Cercas+Curry%2C+and+Verena+Rieser%2E+2017%2E+Why+we+need+new+evaluation+metrics+for+nlg%2E+arXiv+preprint+arXiv%3A1707%2E06875%2E)
21. Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. 2019. Language models are unsupervised multitask learners. OpenAI Blog, 1(8). [find](https://scholar.google.com/scholar?q=Alec+Radford%2C+Jeffrey+Wu%2C+Rewon+Child%2C+David+Luan%2C+Dario+Amodei%2C+and+Ilya+Sutskever%2E+2019%2E+Language+models+are+unsupervised+multitask+learners%2E+OpenAI+Blog%2C+1%288%29%2E)
22. Tal Schuster, Roei Schuster, Darsh J Shah, and Regina Barzilay. 2019. Are we safe yet? the limitations of distributional features for fake news detection. arXiv preprint arXiv:1908.09805. — [arXiv:1908.09805](https://arxiv.org/abs/1908.09805) [find](https://scholar.google.com/scholar?q=Tal+Schuster%2C+Roei+Schuster%2C+Darsh+J+Shah%2C+and+Regina+Barzilay%2E+2019%2E+Are+we+safe+yet%3F+the+limitations+of+distributional+features+for+fake+news+detection%2E+arXiv+preprint+arXiv%3A1908%2E09805%2E+%E2%80%94+%5BarXiv%3A190)
23. Abigail See, Aneesh Pappu, Rohun Saxena, Akhila Yerukola, and Christopher D Manning. 2019. Do massively pretrained language models make better storytellers? In Proceedings of the 23rd Conference on Computational Natural Language Learning (CoNLL), pages 843–861. [find](https://scholar.google.com/scholar?q=Abigail+See%2C+Aneesh+Pappu%2C+Rohun+Saxena%2C+Akhila+Yerukola%2C+and+Christopher+D+Manning%2E+2019%2E+Do+massively+pretrained+language+models+make+better+storytellers%3F+In+Proceedings+of+the+23rd+Conference+on+Co)
24. Rico Sennrich, Barry Haddow, and Alexandra Birch. 2016. Neural machine translation of rare words with subword units. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1715– 1725. [find](https://scholar.google.com/scholar?q=Rico+Sennrich%2C+Barry+Haddow%2C+and+Alexandra+Birch%2E+2016%2E+Neural+machine+translation+of+rare+words+with+subword+units%2E+In+Proceedings+of+the+54th+Annual+Meeting+of+the+Association+for+Computational+Ling)
25. Irene Solaiman, Miles Brundage, Jack Clark, Amanda Askell, Ariel Herbert-Voss, Jeff Wu, Alec Radford, and Jasmine Wang. 2019. Release strategies and the social impacts of language models. arXiv preprint arXiv:1908.09203. [find](https://scholar.google.com/scholar?q=Irene+Solaiman%2C+Miles+Brundage%2C+Jack+Clark%2C+Amanda+Askell%2C+Ariel+Herbert%2DVoss%2C+Jeff+Wu%2C+Alec+Radford%2C+and+Jasmine+Wang%2E+2019%2E+Release+strategies+and+the+social+impacts+of+language+models%2E+arXiv+prepri) [our card](https://md.research.elamur.ai/free/en/papers/solaiman-2019-release-strategies-social-impacts-language/)
26. Jonghyuk Song, Sangho Lee, and Jong Kim. 2015. Crowdtarget: Target-based detection of crowdturfing in online social networks. In Proceedings of the 22nd ACM SIGSAC Conference on Computer and Communications Security, pages 793–804. ACM. [find](https://scholar.google.com/scholar?q=Jonghyuk+Song%2C+Sangho+Lee%2C+and+Jong+Kim%2E+2015%2E+Crowdtarget%3A+Target%2Dbased+detection+of+crowdturfing+in+online+social+networks%2E+In+Proceedings+of+the+22nd+ACM+SIGSAC+Conference+on+Computer+and+Communica)
27. Alan Turing. 1950. Computing machinery and intelligence-am turing. Mind, 59(236):433. [find](https://scholar.google.com/scholar?q=Alan+Turing%2E+1950%2E+Computing+machinery+and+intelligence%2Dam+turing%2E+Mind%2C+59%28236%29%3A433%2E)
28. Chris J Vargo, Lei Guo, and Michelle A Amazeen. 2018. The agenda-setting power of fake news: A big data analysis of the online media landscape from 2014 to 2016. New media & society, 20(5):2028– 2049. [find](https://scholar.google.com/scholar?q=Chris+J+Vargo%2C+Lei+Guo%2C+and+Michelle+A+Amazeen%2E+2018%2E+The+agenda%2Dsetting+power+of+fake+news%3A+A+big+data+analysis+of+the+online+media+landscape+from+2014+to+2016%2E+New+media+%26+society%2C+20%285%29%3A2028%E2%80%93+2049%2E)
29. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Advances in neural information processing systems, pages 5998–6008. [find](https://scholar.google.com/scholar?q=Ashish+Vaswani%2C+Noam+Shazeer%2C+Niki+Parmar%2C+Jakob+Uszkoreit%2C+Llion+Jones%2C+Aidan+N+Gomez%2C+%C5%81ukasz+Kaiser%2C+and+Illia+Polosukhin%2E+2017%2E+Attention+is+all+you+need%2E+In+Advances+in+neural+information+processi)
30. Soroush Vosoughi, Deb Roy, and Sinan Aral. 2018. The spread of true and false news online. Science, 359(6380):1146–1151. [find](https://scholar.google.com/scholar?q=Soroush+Vosoughi%2C+Deb+Roy%2C+and+Sinan+Aral%2E+2018%2E+The+spread+of+true+and+false+news+online%2E+Science%2C+359%286380%29%3A1146%E2%80%931151%2E)
31. Gang Wang, Christo Wilson, Xiaohan Zhao, Yibo Zhu, Manish Mohanlal, Haitao Zheng, and Ben Y Zhao. 2012. Serf and turf: crowdturfing for fun and profit. In Proceedings of the 21st international conference on World Wide Web, pages 679–688. ACM. [find](https://scholar.google.com/scholar?q=Gang+Wang%2C+Christo+Wilson%2C+Xiaohan+Zhao%2C+Yibo+Zhu%2C+Manish+Mohanlal%2C+Haitao+Zheng%2C+and+Ben+Y+Zhao%2E+2012%2E+Serf+and+turf%3A+crowdturfing+for+fun+and+profit%2E+In+Proceedings+of+the+21st+international+confere)
32. Yonghui Wu, Mike Schuster, Zhifeng Chen, Quoc V Le, Mohammad Norouzi, Wolfgang Macherey, Maxim Krikun, Yuan Cao, Qin Gao, Klaus Macherey, et al. 2016. Google’s neural machine translation system: Bridging the gap between human and machine translation. arXiv preprint arXiv:1609.08144. [find](https://scholar.google.com/scholar?q=Yonghui+Wu%2C+Mike+Schuster%2C+Zhifeng+Chen%2C+Quoc+V+Le%2C+Mohammad+Norouzi%2C+Wolfgang+Macherey%2C+Maxim+Krikun%2C+Yuan+Cao%2C+Qin+Gao%2C+Klaus+Macherey%2C+et+al%2E+2016%2E+Google%E2%80%99s+neural+machine+translation+system%3A+Bridg)
33. Rowan Zellers, Ari Holtzman, Hannah Rashkin, Yonatan Bisk, Ali Farhadi, Franziska Roesner, and Yejin Choi. 2019. Defending against neural fake news. CoRR, abs/1905.12616. [find](https://scholar.google.com/scholar?q=Rowan+Zellers%2C+Ari+Holtzman%2C+Hannah+Rashkin%2C+Yonatan+Bisk%2C+Ali+Farhadi%2C+Franziska+Roesner%2C+and+Yejin+Choi%2E+2019%2E+Defending+against+neural+fake+news%2E+CoRR%2C+abs%2F1905%2E12616%2E) [our card](https://md.research.elamur.ai/free/en/papers/zellers-2019-defending-against-neural-fake-news/)
34. Tianyi Zhang, Varsha Kishore, Felix Wu*, Kilian Q. Weinberger, and Yoav Artzi. 2020. Bertscore: Evaluating text generation with bert. In International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Tianyi+Zhang%2C+Varsha+Kishore%2C+Felix+Wu%2A%2C+Kilian+Q%2E+Weinberger%2C+and+Yoav+Artzi%2E+2020%2E+Bertscore%3A+Evaluating+text+generation+with+bert%2E+In+International+Conference+on+Learning+Representations%2E)