---
title: "TURINGBENCH — paper by Uchendu et al. (2021)"
description: "The authors present TURINGBENCH, which has three parts: a dataset, two tasks, and a public website with… 0.62 F1. 73 references, cited by 66 papers here."
authors: ["Adaku Uchendu", "Zeyu Ma", "Thai Le", "Rui Zhang", "Dongwon Lee"]
year: 2021
venue: "Findings of the Association for Computational Linguistics: EMNLP 2021"
source: "https://aclanthology.org/2021.findings-emnlp.172/"
doi: "10.18653/v1/2021.findings-emnlp.172"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/uchendu-2021-turingbench-benchmark-environment-turing-test/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/uchendu-2021-turingbench-benchmark-environment-turing-test/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2021](https://md.research.elamur.ai/free/en/papers/2021/page/3/) / TURINGBENCH: A Benchmark Environment for Turing Test in the Age of Neural Text Generation

# TURINGBENCH: A Benchmark Environment for Turing Test in the Age of Neural Text Generation

**Authors:** [Adaku Uchendu](https://md.research.elamur.ai/free/en/authors/adaku-uchendu/), Zeyu Ma, [Thai Le](https://md.research.elamur.ai/free/en/authors/thai-le/), [Rui Zhang](https://md.research.elamur.ai/free/en/authors/rui-zhang/) and [Dongwon Lee](https://md.research.elamur.ai/free/en/authors/dongwon-lee/) · **Year:** 2021 · **Venue:** [Findings of the Association for Computational Linguistics: EMNLP 2021](https://md.research.elamur.ai/free/en/venues/findings/) ([Findings of the Association for Computational Linguistics 2021](https://md.research.elamur.ai/free/en/venues/findings/2021/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2021.findings-emnlp.172/) · [PDF](https://aclanthology.org/2021.findings-emnlp.172.pdf) · [DOI 10.18653/v1/2021.findings-emnlp.172](https://doi.org/10.18653/v1/2021.findings-emnlp.172) · [OpenAlex](https://openalex.org/W3204318298)

**Citations:** 74 citations (OpenAlex, 2026-10-14) · [66](https://md.research.elamur.ai/free/en/papers/uchendu-2021-turingbench-benchmark-environment-turing-test/cited-by/) in our library, of the 141,840 papers whose bibliographies we read

## Summary
As neural language models produce increasingly fluent text, telling machine output apart from human writing has become a practical concern, for example in countering fake news and propaganda. Before this work, no shared benchmark existed for studying the problem systematically. The authors present TURINGBENCH, which has three parts: a dataset, two tasks, and a public website with leaderboards.

To build the dataset, they collected 10K human-written news articles, mostly political, from CNN, Washington Post and Kaggle. Each article's title served as a prompt for 19 generators built on 10 architectures, including GPT-1/2/3, GROVER, CTRL, XLM, XLNET, FAIR, TRANSFORMER_XL and PPLM. The two tasks are:
- **Turing Test (TT):** binary human-vs-machine classification.
- **Authorship Attribution (AA):** identifying which of 20 authors wrote a text.

Five state-of-the-art TT detectors were tested. FAIR_wmt20 and GPT-3 produced the most human-like text, earning the lowest detection F1. Classical and neural AA models also could not fully separate the authors' styles. The benchmark gives the community a common testbed for building stronger, more explainable detectors.

## Key points
- **Dataset:** 200K samples with 20 labels (Human plus 19 generators). Human source articles were filtered to 200–500 words, leaving a 130K pool, of which 10K were used as prompts.
- **TT task:** split into 19 separate human-vs-generator subtasks. **AA task:** a single 20-way classification.
- **Hardest to detect:** FAIR_wmt20 was the most difficult. GPT-3 and the GPT-2 variants (small, medium, large, XL) also scored below 0.62 F1.
- **Detector behavior:** GPT-2 detector and GLTR fell below 0.4 F1 on some subtasks and often labeled sophisticated machine text as human. Text length (100–400 words) showed no clear link to performance, except for RoBERTa.
- **Style analysis:** a PCA of LIWC psycholinguistic features shows the 20 authors overlapping heavily. Human texts spread widely across the plot, while each generator stays in a narrow region.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Carnegie Mellon University](https://md.research.elamur.ai/free/en/universities/carnegie-mellon-university/) — University, Pittsburgh, United States · 365 works · 158,048 citations
- [Pennsylvania State University](https://md.research.elamur.ai/free/en/universities/pennsylvania-state-university/) — University, State College, United States · 22 works · 4,878 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 9,086 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 8,889 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 9,296 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 9,206 works

🧭 Topics (2): [📏 Evaluation & benchmarks](https://md.research.elamur.ai/free/en/topics/evaluation-benchmarks/) · 685 works · [✍️ Text generation](https://md.research.elamur.ai/free/en/topics/text-generation/) · 145 works

## Cite

Cite as: Adaku Uchendu, Zeyu Ma, Thai Le, Rui Zhang and Dongwon Lee. 2021. TURINGBENCH: A Benchmark Environment for Turing Test in the Age of Neural Text Generation. Findings of the Association for Computational Linguistics: EMNLP 2021.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{uchendu2021turingbench,
  title = {{TURINGBENCH: A Benchmark Environment for Turing Test in the Age of Neural Text Generation}},
  author = {Adaku Uchendu, Zeyu Ma, Thai Le, Rui Zhang and Dongwon Lee},
  booktitle = {Findings of the Association for Computational Linguistics: EMNLP 2021},
  year = {2021},
  doi = {10.18653/v1/2021.findings-emnlp.172},
  url = {https://doi.org/10.18653/v1/2021.findings-emnlp.172}
}
```

- [← previous Paragraph-level Simplification of Medical Texts](https://md.research.elamur.ai/free/en/papers/devaraj-2021-paragraph-level-simplification-medical-texts/)
- [next → All Bark and No Bite: Rogue Dimensions in Transformer Language Models Obscure Representational Quality](https://md.research.elamur.ai/free/en/papers/timkey-2021-all-bark-no-bite-rogue/)
- [↑ back to papers of 2021](https://md.research.elamur.ai/free/en/papers/2021/page/3/)

## References

73 references, 19 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. David Ifeoluwa Adelani, Haotian Mai, Fuming Fang, Huy H Nguyen, Junichi Yamagishi, and Isao Echizen. 2020. Generating sentiment-preserving fake online reviews using neural language models and their human-and machine-based detection. In International Conference on Advanced Information Networking and… [find](https://scholar.google.com/scholar?q=David+Ifeoluwa+Adelani%2C+Haotian+Mai%2C+Fuming+Fang%2C+Huy+H+Nguyen%2C+Junichi+Yamagishi%2C+and+Isao+Echizen%2E+2020%2E+Generating+sentiment%2Dpreserving+fake+online+reviews+using+neural+language+models+and+their+hu)
2. Uri Alon, Shaked Brody, Omer Levy, and Eran Yahav. 2018. code2seq: Generating sequences from structured representations of code. In International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Uri+Alon%2C+Shaked+Brody%2C+Omer+Levy%2C+and+Eran+Yahav%2E+2018%2E+code2seq%3A+Generating+sequences+from+structured+representations+of+code%2E+In+International+Conference+on+Learning+Representations%2E)
3. Anton Bakhtin, Sam Gross, Myle Ott, Yuntian Deng, Marc’Aurelio Ranzato, and Arthur Szlam. 2019. Real or fake? learning to discriminate machine from human generated text. arXiv preprint arXiv:1906.03351. — [arXiv:1906.03351](https://arxiv.org/abs/1906.03351) [find](https://scholar.google.com/scholar?q=Anton+Bakhtin%2C+Sam+Gross%2C+Myle+Ott%2C+Yuntian+Deng%2C+Marc%E2%80%99Aurelio+Ranzato%2C+and+Arthur+Szlam%2E+2019%2E+Real+or+fake%3F+learning+to+discriminate+machine+from+human+generated+text%2E+arXiv+preprint+arXiv%3A1906%2E0335)
4. Meghana Moorthy Bhat and Srinivasan Parthasarathy. 2020. How effectively can machines defend against machine-generated fake news? an empirical study. In Proceedings of the First Workshop on Insights from Negative Results in NLP, pages 48–53. [find](https://scholar.google.com/scholar?q=Meghana+Moorthy+Bhat+and+Srinivasan+Parthasarathy%2E+2020%2E+How+effectively+can+machines+defend+against+machine%2Dgenerated+fake+news%3F+an+empirical+study%2E+In+Proceedings+of+the+First+Workshop+on+Insights+f)
5. Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel Ziegler, Jeffrey Wu, Clemens Winter, Chri… [our card](https://md.research.elamur.ai/free/en/papers/brown-2020-language-models-few-shot-learners/)
6. Erion Çano and Ondřej Bojar. 2020. Human or machine: Automating human likeliness evaluation of nlg texts. arXiv preprint arXiv:2006.03189. — [arXiv:2006.03189](https://arxiv.org/abs/2006.03189) [find](https://scholar.google.com/scholar?q=Erion+%C3%87ano+and+Ondr%CC%8Cej+Bojar%2E+2020%2E+Human+or+machine%3A+Automating+human+likeliness+evaluation+of+nlg+texts%2E+arXiv+preprint+arXiv%3A2006%2E03189%2E+%E2%80%94+%5BarXiv%3A2006%2E03189%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2006%2E03189%29)
7. Peng-Jen Chen, Ann Lee, Changhan Wang, Naman Goyal, Angela Fan, Mary Williamson, and Jiatao Gu. 2020. Facebook ai’s wmt20 news translation task submission. In Proceedings of the Fifth Conference on Machine Translation, pages 113–125. [find](https://scholar.google.com/scholar?q=Peng%2DJen+Chen%2C+Ann+Lee%2C+Changhan+Wang%2C+Naman+Goyal%2C+Angela+Fan%2C+Mary+Williamson%2C+and+Jiatao+Gu%2E+2020%2E+Facebook+ai%E2%80%99s+wmt20+news+translation+task+submission%2E+In+Proceedings+of+the+Fifth+Conference+on+Ma)
8. Rosa María Coyotl-Morales, Luis Villaseñor-Pineda, Manuel Montes-y Gómez, and Paolo Rosso. 2006. Authorship attribution using word sequences. In Iberoamerican Congress on Pattern Recognition, pages 844–853. Springer. [find](https://scholar.google.com/scholar?q=Rosa+Mar%C3%ADa+Coyotl%2DMorales%2C+Luis+Villase%C3%B1or%2DPineda%2C+Manuel+Montes%2Dy+G%C3%B3mez%2C+and+Paolo+Rosso%2E+2006%2E+Authorship+attribution+using+word+sequences%2E+In+Iberoamerican+Congress+on+Pattern+Recognition%2C+pages+84)
9. Zihang Dai, Zhilin Yang, Yiming Yang, Jaime G Carbonell, Quoc Le, and Ruslan Salakhutdinov. 2019. Transformer-xl: Attentive language models beyond a fixed-length context. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 2978–2988. [find](https://scholar.google.com/scholar?q=Zihang+Dai%2C+Zhilin+Yang%2C+Yiming+Yang%2C+Jaime+G+Carbonell%2C+Quoc+Le%2C+and+Ruslan+Salakhutdinov%2E+2019%2E+Transformer%2Dxl%3A+Attentive+language+models+beyond+a+fixed%2Dlength+context%2E+In+Proceedings+of+the+57th+An)
10. Avisha Das and Rakesh Verma. 2018. Automated email generation for targeted attacks using natural language. In TA-COS 2018: 2nd Workshop on Text Analytics for Cybersecurity and Online Safety, page 23. [find](https://scholar.google.com/scholar?q=Avisha+Das+and+Rakesh+Verma%2E+2018%2E+Automated+email+generation+for+targeted+attacks+using+natural+language%2E+In+TA%2DCOS+2018%3A+2nd+Workshop+on+Text+Analytics+for+Cybersecurity+and+Online+Safety%2C+page+23%2E)
11. Sumanth Dathathri, Andrea Madotto, Janice Lan, Jane Hung, Eric Frank, Piero Molino, Jason Yosinski, and Rosanne Liu. 2020. Plug and play language models: A simple approach to controlled text generation. In International Conference on Learning Representations. [our card](https://md.research.elamur.ai/free/en/papers/dathathri-2019-plug-play-language-models-simple/)
12. Yuntian Deng, Anton Bakhtin, Myle Ott, Arthur Szlam, and Marc’Aurelio Ranzato. 2019. Residual energy-based models for text generation. In International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Yuntian+Deng%2C+Anton+Bakhtin%2C+Myle+Ott%2C+Arthur+Szlam%2C+and+Marc%E2%80%99Aurelio+Ranzato%2E+2019%2E+Residual+energy%2Dbased+models+for+text+generation%2E+In+International+Conference+on+Learning+Representations%2E)
13. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies,… [our card](https://md.research.elamur.ai/free/en/papers/devlin-2019-bert-pre-training-deep-bidirectional/)
14. H. Jair Escalante, Thamar Solorio, and Manuel Montes y Gómez. 2011. Local histograms of character ngrams for authorship attribution. In Proceedings of the 49th Annual Meeting for the Association for Computational Linguistics: Human Language Technologies, pages 288–298, Portland, Oregon. ACL. [find](https://scholar.google.com/scholar?q=H%2E+Jair+Escalante%2C+Thamar+Solorio%2C+and+Manuel+Montes+y+G%C3%B3mez%2E+2011%2E+Local+histograms+of+character+ngrams+for+authorship+attribution%2E+In+Proceedings+of+the+49th+Annual+Meeting+for+the+Association+for+C)
15. Maël Fabien, Esa ú Villatoro-Tello, Petr Motlicek, and Shantipriya Parida. 2020. Bertaa: Bert fine-tuning for authorship attribution. In Proceedings of the 17th International Conference on Natural Language Processing, CONF. ACL. [find](https://scholar.google.com/scholar?q=Ma%C3%ABl+Fabien%2C+Esa+%C3%BA+Villatoro%2DTello%2C+Petr+Motlicek%2C+and+Shantipriya+Parida%2E+2020%2E+Bertaa%3A+Bert+fine%2Dtuning+for+authorship+attribution%2E+In+Proceedings+of+the+17th+International+Conference+on+Natural+Lan)
16. Angela Fan, Mike Lewis, and Yann Dauphin. 2018. Hierarchical neural story generation. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 889–898. [find](https://scholar.google.com/scholar?q=Angela+Fan%2C+Mike+Lewis%2C+and+Yann+Dauphin%2E+2018%2E+Hierarchical+neural+story+generation%2E+In+Proceedings+of+the+56th+Annual+Meeting+of+the+Association+for+Computational+Linguistics+%28Volume+1%3A+Long+Papers%29)
17. William Fedus, Barret Zoph, and Noam Shazeer. 2021. Switch transformers: Scaling to trillion parameter models with simple and efficient sparsity. arXiv preprint arXiv:2101.03961. [find](https://scholar.google.com/scholar?q=William+Fedus%2C+Barret+Zoph%2C+and+Noam+Shazeer%2E+2021%2E+Switch+transformers%3A+Scaling+to+trillion+parameter+models+with+simple+and+efficient+sparsity%2E+arXiv+preprint+arXiv%3A2101%2E03961%2E)
18. Elisa Ferracane, Su Wang, and Raymond Mooney. 2017. Leveraging discourse information effectively for authorship attribution. In Proceedings of the Eighth International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 584–593. [find](https://scholar.google.com/scholar?q=Elisa+Ferracane%2C+Su+Wang%2C+and+Raymond+Mooney%2E+2017%2E+Leveraging+discourse+information+effectively+for+authorship+attribution%2E+In+Proceedings+of+the+Eighth+International+Joint+Conference+on+Natural+Lang)
19. Tommaso Fornaciari and Massimo Poesio. 2014. Identifying fake amazon reviews as learning from crowds. In Proceedings of the 14th Conference of the European Chapter of the Association for Computational Linguistics, pages 279–287. [find](https://scholar.google.com/scholar?q=Tommaso+Fornaciari+and+Massimo+Poesio%2E+2014%2E+Identifying+fake+amazon+reviews+as+learning+from+crowds%2E+In+Proceedings+of+the+14th+Conference+of+the+European+Chapter+of+the+Association+for+Computational)
20. Cristina Garbacea and Qiaozhu Mei. 2020. Neural language generation: Formulation, methods, and evaluation. arXiv preprint arXiv:2007.15780. — [arXiv:2007.15780](https://arxiv.org/abs/2007.15780) [find](https://scholar.google.com/scholar?q=Cristina+Garbacea+and+Qiaozhu+Mei%2E+2020%2E+Neural+language+generation%3A+Formulation%2C+methods%2C+and+evaluation%2E+arXiv+preprint+arXiv%3A2007%2E15780%2E+%E2%80%94+%5BarXiv%3A2007%2E15780%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2007%2E15780%29)
21. Zhenhao Ge, Yufang Sun, and Mark Smith. 2016. Authorship attribution using a neural network language model. In Proceedings of the AAAI Conference on Artificial Intelligence. [find](https://scholar.google.com/scholar?q=Zhenhao+Ge%2C+Yufang+Sun%2C+and+Mark+Smith%2E+2016%2E+Authorship+attribution+using+a+neural+network+language+model%2E+In+Proceedings+of+the+AAAI+Conference+on+Artificial+Intelligence%2E)
22. Maarten Lambers and Cor J Veenman. 2009. Forensic authorship attribution using compression distances to prototypes. In International Workshop on Computational Forensics, pages 13–24. Springer. [find](https://scholar.google.com/scholar?q=Maarten+Lambers+and+Cor+J+Veenman%2E+2009%2E+Forensic+authorship+attribution+using+compression+distances+to+prototypes%2E+In+International+Workshop+on+Computational+Forensics%2C+pages+13%E2%80%9324%2E+Springer%2E)
23. Sebastian Gehrmann, Hendrik Strobelt, and Alexander M Rush. 2019. Gltr: Statistical detection and visualization of generated text. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics: System Demonstrations, pages 111–116. [our card](https://md.research.elamur.ai/free/en/papers/gehrmann-2019-gltr-statistical-detection-visualization-generated/)
24. Guillaume Lample and Alexis Conneau. 2019. Cross-lingual language model pretraining. arXiv preprint arXiv:1901.07291. [find](https://scholar.google.com/scholar?q=Guillaume+Lample+and+Alexis+Conneau%2E+2019%2E+Cross%2Dlingual+language+model+pretraining%2E+arXiv+preprint+arXiv%3A1901%2E07291%2E)
25. Jade Goldstein, Ransom Winder, and Roberta Sabin. 2009. Person identification from text and speech genre samples. In Proceedings of the 12th Conference of the European Chapter of the ACL (EACL 2009), pages 336–344. [find](https://scholar.google.com/scholar?q=Jade+Goldstein%2C+Ransom+Winder%2C+and+Roberta+Sabin%2E+2009%2E+Person+identification+from+text+and+speech+genre+samples%2E+In+Proceedings+of+the+12th+Conference+of+the+European+Chapter+of+the+ACL+%28EACL+2009%29%2C+)
26. Oren Halvani, Lukas Graner, Roey Regev, and Philipp Marquardt. 2020. An improved topic masking technique for authorship analysis. arXiv preprint arXiv:2005.06605. — [arXiv:2005.06605](https://arxiv.org/abs/2005.06605) [find](https://scholar.google.com/scholar?q=Oren+Halvani%2C+Lukas+Graner%2C+Roey+Regev%2C+and+Philipp+Marquardt%2E+2020%2E+An+improved+topic+masking+technique+for+authorship+analysis%2E+arXiv+preprint+arXiv%3A2005%2E06605%2E+%E2%80%94+%5BarXiv%3A2005%2E06605%5D%28https%3A%2F%2Farxiv%2Eor)
27. Julian Hitschler, Esther Van Den Berg, and Ines Rehbein. 2017. Authorship attribution with convolutional neural networks and pos-eliding. In Proceedings of the Workshop on Stylistic Variation, pages 53–58. [find](https://scholar.google.com/scholar?q=Julian+Hitschler%2C+Esther+Van+Den+Berg%2C+and+Ines+Rehbein%2E+2017%2E+Authorship+attribution+with+convolutional+neural+networks+and+pos%2Deliding%2E+In+Proceedings+of+the+Workshop+on+Stylistic+Variation%2C+pages+5)
28. John Houvardas and Efstathios Stamatatos. 2006. Ngram feature selection for authorship identification. In International conference on artificial intelligence: Methodology, systems, and applications, pages 77– 86. Springer. [find](https://scholar.google.com/scholar?q=John+Houvardas+and+Efstathios+Stamatatos%2E+2006%2E+Ngram+feature+selection+for+authorship+identification%2E+In+International+conference+on+artificial+intelligence%3A+Methodology%2C+systems%2C+and+applications%2C+p)
29. Daphne Ippolito, Daniel Duckworth, Chris Callison-Burch, and Douglas Eck. 2020. Automatic detection of generated text is easiest when humans are fooled. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. [our card](https://md.research.elamur.ai/free/en/papers/ippolito-2019-automatic-detection-generated-text-easiest/)
30. Fereshteh Jafariakinabad and Kien A Hua. 2019. Style-aware neural model with application in authorship attribution. In 2019 18th IEEE International Conference On Machine Learning And Applications (ICMLA), pages 325–328. IEEE. [find](https://scholar.google.com/scholar?q=Fereshteh+Jafariakinabad+and+Kien+A+Hua%2E+2019%2E+Style%2Daware+neural+model+with+application+in+authorship+attribution%2E+In+2019+18th+IEEE+International+Conference+On+Machine+Learning+And+Applications+%28ICM)
31. Fereshteh Jafariakinabad and Kien A Hua. 2020. A self-supervised representation learning of sentence structure for authorship attribution. arXiv preprint arXiv:2010.06786. — [arXiv:2010.06786](https://arxiv.org/abs/2010.06786) [find](https://scholar.google.com/scholar?q=Fereshteh+Jafariakinabad+and+Kien+A+Hua%2E+2020%2E+A+self%2Dsupervised+representation+learning+of+sentence+structure+for+authorship+attribution%2E+arXiv+preprint+arXiv%3A2010%2E06786%2E+%E2%80%94+%5BarXiv%3A2010%2E06786%5D%28https%3A%2F)
32. Fereshteh Jafariakinabad, Sansiri Tarnpradab, and Kien A Hua. 2019. Syntactic recurrent neural network for authorship attribution. arXiv preprint arXiv:1902.09723. — [arXiv:1902.09723](https://arxiv.org/abs/1902.09723) [find](https://scholar.google.com/scholar?q=Fereshteh+Jafariakinabad%2C+Sansiri+Tarnpradab%2C+and+Kien+A+Hua%2E+2019%2E+Syntactic+recurrent+neural+network+for+authorship+attribution%2E+arXiv+preprint+arXiv%3A1902%2E09723%2E+%E2%80%94+%5BarXiv%3A1902%2E09723%5D%28https%3A%2F%2Farxiv%2Eo)
33. Ganesh Jawahar, Muhammad Abdul-Mageed, and VS Laks Lakshmanan. 2020. Automatic detection of machine generated text: A critical survey. In Proceedings of the 28th International Conference on Computational Linguistics, pages 2296–2309. [our card](https://md.research.elamur.ai/free/en/papers/jawahar-2020-automatic-detection-machine-generated-text/)
34. Nitish Shirish Keskar, Bryan McCann, Lav R Varshney, Caiming Xiong, and Richard Socher. 2019. Ctrl: A conditional transformer language model for controllable generation. arXiv preprint arXiv:1909.05858. [our card](https://md.research.elamur.ai/free/en/papers/keskar-2019-ctrl-conditional-transformer-language-model/)
35. Rémi Lebret, David Grangier, and Michael Auli. 2016. Neural text generation from structured data with application to the biography domain. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 1203–1213. [our card](https://md.research.elamur.ai/free/en/papers/lebret-2016-neural-text-generation-structured-data/)
36. Jiwei Li, Myle Ott, Claire Cardie, and Eduard Hovy. 2014. Towards a general rule for identifying deceptive opinion spam. In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1566– 1576. [find](https://scholar.google.com/scholar?q=Jiwei+Li%2C+Myle+Ott%2C+Claire+Cardie%2C+and+Eduard+Hovy%2E+2014%2E+Towards+a+general+rule+for+identifying+deceptive+opinion+spam%2E+In+Proceedings+of+the+52nd+Annual+Meeting+of+the+Association+for+Computational+)
37. Guanxiong Liu, Tzu-Ming Harry Hsu, Matthew Mc-Dermott, Willie Boag, Wei-Hung Weng, Peter Szolovits, and Marzyeh Ghassemi. 2019a. Clinically accurate chest x-ray report generation. In Machine Learning for Healthcare Conference, pages 249–269. PMLR. [find](https://scholar.google.com/scholar?q=Guanxiong+Liu%2C+Tzu%2DMing+Harry+Hsu%2C+Matthew+Mc%2DDermott%2C+Willie+Boag%2C+Wei%2DHung+Weng%2C+Peter+Szolovits%2C+and+Marzyeh+Ghassemi%2E+2019a%2E+Clinically+accurate+chest+x%2Dray+report+generation%2E+In+Machine+Learning+)
38. Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019b. Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692. [find](https://scholar.google.com/scholar?q=Yinhan+Liu%2C+Myle+Ott%2C+Naman+Goyal%2C+Jingfei+Du%2C+Mandar+Joshi%2C+Danqi+Chen%2C+Omer+Levy%2C+Mike+Lewis%2C+Luke+Zettlemoyer%2C+and+Veselin+Stoyanov%2E+2019b%2E+Roberta%3A+A+robustly+optimized+bert+pretraining+approach%2E+)
39. A Pastor López-Monroy, Fabio A González, and Thamar Solorio. 2020. Early author profiling on twitter using profile features with multi-resolution. Expert Systems with Applications, 140:112909. [find](https://scholar.google.com/scholar?q=A+Pastor+L%C3%B3pez%2DMonroy%2C+Fabio+A+Gonz%C3%A1lez%2C+and+Thamar+Solorio%2E+2020%2E+Early+author+profiling+on+twitter+using+profile+features+with+multi%2Dresolution%2E+Expert+Systems+with+Applications%2C+140%3A112909%2E)
40. Asad Mahmood, Faizan Ahmad, Zubair Shafiq, Padmini Srinivasan, and Fareed Zaffar. 2019. A girl has no name: Automated authorship obfuscation using mutant-x. Proceedings on Privacy Enhancing Technologies, 2019(4):54–71. [find](https://scholar.google.com/scholar?q=Asad+Mahmood%2C+Faizan+Ahmad%2C+Zubair+Shafiq%2C+Padmini+Srinivasan%2C+and+Fareed+Zaffar%2E+2019%2E+A+girl+has+no+name%3A+Automated+authorship+obfuscation+using+mutant%2Dx%2E+Proceedings+on+Privacy+Enhancing+Technologi)
41. Enrique Manjavacas, Jeroen De Gussem, Walter Daelemans, and Mike Kestemont. 2017. Assessing the stylistic properties of neurally generated text in authorship attribution. arXiv preprint arXiv:1708.05536. [find](https://scholar.google.com/scholar?q=Enrique+Manjavacas%2C+Jeroen+De+Gussem%2C+Walter+Daelemans%2C+and+Mike+Kestemont%2E+2017%2E+Assessing+the+stylistic+properties+of+neurally+generated+text+in+authorship+attribution%2E+arXiv+preprint+arXiv%3A1708%2E055)
42. Ahmadreza Mosallanezhad, Kai Shu, and Huan Liu. 2020. Topic-preserving synthetic news generation: An adversarial deep reinforcement learning approach. arXiv preprint arXiv:2010.16324. — [arXiv:2010.16324](https://arxiv.org/abs/2010.16324) [find](https://scholar.google.com/scholar?q=Ahmadreza+Mosallanezhad%2C+Kai+Shu%2C+and+Huan+Liu%2E+2020%2E+Topic%2Dpreserving+synthetic+news+generation%3A+An+adversarial+deep+reinforcement+learning+approach%2E+arXiv+preprint+arXiv%3A2010%2E16324%2E+%E2%80%94+%5BarXiv%3A2010%2E16)
43. Shaoor Munir, Brishna Batool, Zubair Shafiq, Padmini Srinivasan, and Fareed Zaffar. 2021. Through the looking glass: Learning to attribute synthetic text generated by language models. In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main… [find](https://scholar.google.com/scholar?q=Shaoor+Munir%2C+Brishna+Batool%2C+Zubair+Shafiq%2C+Padmini+Srinivasan%2C+and+Fareed+Zaffar%2E+2021%2E+Through+the+looking+glass%3A+Learning+to+attribute+synthetic+text+generated+by+language+models%2E+In+Proceedings+o)
44. Nathan Ng, Kyra Yee, Alexei Baevski, Myle Ott, Michael Auli, and Sergey Edunov. 2019. Facebook fair’s wmt19 news translation task submission. arXiv preprint arXiv:1907.06616. [our card](https://md.research.elamur.ai/free/en/papers/ng-2019-facebook-fair-s-wmt19-news/)
45. Kai Shu, Yichuan Li, Kaize Ding, and Huan Liu. 2021. Fact-enhanced synthetic news generation. In AAAI. [find](https://scholar.google.com/scholar?q=Kai+Shu%2C+Yichuan+Li%2C+Kaize+Ding%2C+and+Huan+Liu%2E+2021%2E+Fact%2Denhanced+synthetic+news+generation%2E+In+AAAI%2E)
46. Kai Shu, Amy Sliva, Suhang Wang, Jiliang Tang, and Huan Liu. 2017. Fake news detection on social media: A data mining perspective. ACM SIGKDD explorations newsletter, 19(1):22–36. [find](https://scholar.google.com/scholar?q=Kai+Shu%2C+Amy+Sliva%2C+Suhang+Wang%2C+Jiliang+Tang%2C+and+Huan+Liu%2E+2017%2E+Fake+news+detection+on+social+media%3A+A+data+mining+perspective%2E+ACM+SIGKDD+explorations+newsletter%2C+19%281%29%3A22%E2%80%9336%2E)
47. Alec Radford, Karthik Narasimhan, Tim Salimans, and Ilya Sutskever. 2018. Improving language understanding by generative pre-training. URL Irene Solaiman, Miles Brundage, Jack Clark, Amanda [https://s3-us-west-2.amazonaws](https://s3-us-west-2.amazonaws). com/openai-Askell, Ariel Herbert-Voss, Jeff Wu, Alec Rad-assets/researchcover… [↗](https://s3-us-west-2.amazonaws)
48. Kreps, et al. 2019. Release strategies and the social impacts of language models. arXiv preprint Alec Radford, Jeffrey Wu, Rewon Child, David Luan, arXiv:1908.09203. [our card](https://md.research.elamur.ai/free/en/papers/solaiman-2019-release-strategies-social-impacts-language/)
49. Dario Amodei, and Ilya Sutskever. 2019. Language models are unsupervised multitask learners. [find](https://scholar.google.com/scholar?q=Dario+Amodei%2C+and+Ilya+Sutskever%2E+2019%2E+Language+models+are+unsupervised+multitask+learners%2E)
50. OpenAI Efstathios Stamatatos. 2009a. Intrinsic plagiarism de-blog, 1(8):9. tection using character n-gram profiles. In 3rd PAN WORKSHOP. [find](https://scholar.google.com/scholar?q=OpenAI+Efstathios+Stamatatos%2E+2009a%2E+Intrinsic+plagiarism+de%2Dblog%2C+1%288%29%3A9%2E+tection+using+character+n%2Dgram+profiles%2E+In+3rd+PAN+WORKSHOP%2E)
51. UNCOVERING PLAGIARISM, AU-Pranav Rajpurkar, Jian Zhang, Konstantin Lopyrev, and THORSHIP AND SOCIAL SOFTWARE MISUSE. Percy Liang. 2016. Squad: 100,000+ questions for machine comprehension of text. In Proceedings of Efstathios Stamatatos. 2009b. A survey of modern au-the 2016 Conference on Empirical… [our card](https://md.research.elamur.ai/free/en/papers/rajpurkar-2016-squad-100-000-questions-machine/)
52. Hannah Rashkin, Eunsol Choi, Jin Yea Jang, Svitlana Volkova, and Yejin Choi. 2017. Truth of varying Alan M Turing. 2009. Computing machinery and in-shades: Analyzing language in fake news and polit-telligence. In Parsing the turing test, pages 23–65. ical fact-checking. In Proceedings of the 2017 co… [find](https://scholar.google.com/scholar?q=Hannah+Rashkin%2C+Eunsol+Choi%2C+Jin+Yea+Jang%2C+Svitlana+Volkova%2C+and+Yejin+Choi%2E+2017%2E+Truth+of+varying+Alan+M+Turing%2E+2009%2E+Computing+machinery+and+in%2Dshades%3A+Analyzing+language+in+fake+news+and+polit%2Dte)
53. Adaku Uchendu, Jeffery Cao, Qiaozhi Wang, Bo Luo, and Dongwon Lee. 2019. Characterizing man-made vs. machine-made chatbot dialogs. In Proceedings of the Int’l Conf. on Truth and Trust Online (TTO). [find](https://scholar.google.com/scholar?q=Adaku+Uchendu%2C+Jeffery+Cao%2C+Qiaozhi+Wang%2C+Bo+Luo%2C+and+Dongwon+Lee%2E+2019%2E+Characterizing+man%2Dmade+vs%2E+machine%2Dmade+chatbot+dialogs%2E+In+Proceedings+of+the+Int%E2%80%99l+Conf%2E+on+Truth+and+Trust+Online+%28TTO%29%2E)
54. Upendra Sapkota, Steven Bethard, Manuel Montes y Gomez, and Thamar Solorio. 2015. Not all character n-grams are created equal: A study in authorship attribution. In Proceedings of the 2015 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Techn… [find](https://scholar.google.com/scholar?q=Upendra+Sapkota%2C+Steven+Bethard%2C+Manuel+Montes+y+Gomez%2C+and+Thamar+Solorio%2E+2015%2E+Not+all+character+n%2Dgrams+are+created+equal%3A+A+study+in+authorship+attribution%2E+In+Proceedings+of+the+2015+Conference+)
55. Adaku Uchendu, Thai Le, Kai Shu, and Dongwon Lee. 2020. Authorship attribution for neural text generation. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP). [our card](https://md.research.elamur.ai/free/en/papers/uchendu-2020-authorship-attribution-neural-text-generation/)
56. Upendra Sapkota, Thamar Solorio, Manuel Montes y Gomez, and Steven Bethard. 2016. Domain adaptation for authorship attribution: Improved structural correspondence learning. In Association for Computational Linguistics (ACL), Berlin, Germany. ACL. [find](https://scholar.google.com/scholar?q=Upendra+Sapkota%2C+Thamar+Solorio%2C+Manuel+Montes+y+Gomez%2C+and+Steven+Bethard%2E+2016%2E+Domain+adaptation+for+authorship+attribution%3A+Improved+structural+correspondence+learning%2E+In+Association+for+Computat)
57. Onur Varol, Emilio Ferrara, Clayton Davis, Filippo Menczer, and Alessandro Flammini. 2017. Online human-bot interactions: Detection, estimation, and characterization. In Proceedings of the International AAAI Conference on Web and Social Media. [find](https://scholar.google.com/scholar?q=Onur+Varol%2C+Emilio+Ferrara%2C+Clayton+Davis%2C+Filippo+Menczer%2C+and+Alessandro+Flammini%2E+2017%2E+Online+human%2Dbot+interactions%3A+Detection%2C+estimation%2C+and+characterization%2E+In+Proceedings+of+the+Internation)
58. Tal Schuster, Roei Schuster, Darsh J Shah, and Regina Barzilay. 2020. The limitations of stylometry for detecting machine-generated fake news. Computational Linguistics, 46(2):499–510. [find](https://scholar.google.com/scholar?q=Tal+Schuster%2C+Roei+Schuster%2C+Darsh+J+Shah%2C+and+Regina+Barzilay%2E+2020%2E+The+limitations+of+stylometry+for+detecting+machine%2Dgenerated+fake+news%2E+Computational+Linguistics%2C+46%282%29%3A499%E2%80%93510%2E)
59. Lav R Varshney, Nitish Shirish Keskar, and Richard Socher. 2020. Limits of detecting text generated by large-scale language models. In 2020 Information Theory and Applications Workshop (ITA), pages 1– 5. IEEE. [find](https://scholar.google.com/scholar?q=Lav+R+Varshney%2C+Nitish+Shirish+Keskar%2C+and+Richard+Socher%2E+2020%2E+Limits+of+detecting+text+generated+by+large%2Dscale+language+models%2E+In+2020+Information+Theory+and+Applications+Workshop+%28ITA%29%2C+pages+1%E2%80%93)
60. Abigail See, Aneesh Pappu, Rohun Saxena, Akhila Yerukola, and Christopher D Manning. 2019. Do massively pretrained language models make better storytellers? In Proceedings of the 23rd Conference on Computational Natural Language Learning (CoNLL), pages 843–861. [find](https://scholar.google.com/scholar?q=Abigail+See%2C+Aneesh+Pappu%2C+Rohun+Saxena%2C+Akhila+Yerukola%2C+and+Christopher+D+Manning%2E+2019%2E+Do+massively+pretrained+language+models+make+better+storytellers%3F+In+Proceedings+of+the+23rd+Conference+on+Co)
61. Prasha Shrestha, Sebastian Sierra, Fabio Gonzalez, Manuel Montes, Paolo Rosso, and Thamar Solorio. 2017. Convolutional neural networks for authorship attribution of short texts. In Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 2,… [find](https://scholar.google.com/scholar?q=Prasha+Shrestha%2C+Sebastian+Sierra%2C+Fabio+Gonzalez%2C+Manuel+Montes%2C+Paolo+Rosso%2C+and+Thamar+Solorio%2E+2017%2E+Convolutional+neural+networks+for+authorship+attribution+of+short+texts%2E+In+Proceedings+of+the+)
62. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. Advances in neural information processing systems, 30:5998–6008. [our card](https://md.research.elamur.ai/free/en/papers/vaswani-2017-attention-all-need/)
63. Alex Wang, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel Bowman. 2018. Glue: A multi-task benchmark and analysis platform for natural language understanding. In Proceedings of the 2018 EMNLP Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP, pages 353–355. [our card](https://md.research.elamur.ai/free/en/papers/wang-2018-glue-multi-task-benchmark-analysis/)
64. William Yang Wang. 2017. “liar, liar pants on fire”: A new benchmark dataset for fake news detection. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 422–426. [our card](https://md.research.elamur.ai/free/en/papers/wang-2017-liar-liar-pants-fire-new/)
65. Joseph Weizenbaum. 1966. Eliza—a computer program for the study of natural language communication between man and machine. Communications of the ACM, 9(1):36–45. [find](https://scholar.google.com/scholar?q=Joseph+Weizenbaum%2E+1966%2E+Eliza%E2%80%94a+computer+program+for+the+study+of+natural+language+communication+between+man+and+machine%2E+Communications+of+the+ACM%2C+9%281%29%3A36%E2%80%9345%2E)
66. Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pier-ric Cistac, Tim Rault, Rémi Louf, Morgan Funtowicz, et al. 2019. Huggingface’s transformers: State-of-the-art natural language processing. ArXiv, pages arXiv–1910. [our card](https://md.research.elamur.ai/free/en/papers/wolf-2020-transformers-state-art-natural-language/)
67. Zhilin Yang, Zihang Dai, Yiming Yang, Jaime Carbonell, Russ R Salakhutdinov, and Quoc V Le. 2019. Xlnet: Generalized autoregressive pretraining for language understanding. In Advances in neural information processing systems, pages 5754–5764. [our card](https://md.research.elamur.ai/free/en/papers/yang-2019-xlnet-generalized-autoregressive-pretraining-language/)
68. Rowan Zellers, Ari Holtzman, Hannah Rashkin, Yonatan Bisk, Ali Farhadi, Franziska Roesner, and Yejin Choi. 2019. Defending against neural fake news. In Advances in Neural Information Processing Systems, pages 9051–9062. [our card](https://md.research.elamur.ai/free/en/papers/zellers-2019-defending-against-neural-fake-news/)
69. Chunxia Zhang, Xindong Wu, Zhendong Niu, and Wei Ding. 2014. Authorship identification from unstructured texts. Knowledge-Based Systems, 66:99–111. [find](https://scholar.google.com/scholar?q=Chunxia+Zhang%2C+Xindong+Wu%2C+Zhendong+Niu%2C+and+Wei+Ding%2E+2014%2E+Authorship+identification+from+unstructured+texts%2E+Knowledge%2DBased+Systems%2C+66%3A99%E2%80%93111%2E)
70. Richong Zhang, Zhiyuan Hu, Hongyu Guo, and Yongyi Mao. 2018. Syntax encoding with application in authorship attribution. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 2742–2753. [find](https://scholar.google.com/scholar?q=Richong+Zhang%2C+Zhiyuan+Hu%2C+Hongyu+Guo%2C+and+Yongyi+Mao%2E+2018%2E+Syntax+encoding+with+application+in+authorship+attribution%2E+In+Proceedings+of+the+2018+Conference+on+Empirical+Methods+in+Natural+Language+)
71. Yizhe Zhang, Siqi Sun, Michel Galley, Yen-Chun Chen, Chris Brockett, Xiang Gao, Jianfeng Gao, Jingjing Liu, and William B Dolan. 2020. Dialogpt: Large-scale generative pre-training for conversational response generation. In Proceedings of the 58th Annual Meeting of the Association for Computational… [our card](https://md.research.elamur.ai/free/en/papers/zhang-2019-dialogpt-large-scale-generative-pre/)
72. Wanjun Zhong, Duyu Tang, Zenan Xu, Ruize Wang, Nan Duan, Ming Zhou, Jiahai Wang, and Jian Yin. 2020. Neural deepfake detection with factual structure of text. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP). [find](https://scholar.google.com/scholar?q=Wanjun+Zhong%2C+Duyu+Tang%2C+Zenan+Xu%2C+Ruize+Wang%2C+Nan+Duan%2C+Ming+Zhou%2C+Jiahai+Wang%2C+and+Jian+Yin%2E+2020%2E+Neural+deepfake+detection+with+factual+structure+of+text%2E+In+Proceedings+of+the+2020+Conference+on+)
73. Zhixuan Zhou, Huankang Guan, Meghana Moorthy Bhat, and Justin Hsu. 2019. Fake news detection via nlp is vulnerable to adversarial attacks. arXiv preprint arXiv:1901.09657. — [arXiv:1901.09657](https://arxiv.org/abs/1901.09657) [find](https://scholar.google.com/scholar?q=Zhixuan+Zhou%2C+Huankang+Guan%2C+Meghana+Moorthy+Bhat%2C+and+Justin+Hsu%2E+2019%2E+Fake+news+detection+via+nlp+is+vulnerable+to+adversarial+attacks%2E+arXiv+preprint+arXiv%3A1901%2E09657%2E+%E2%80%94+%5BarXiv%3A1901%2E09657%5D%28https%3A%2F)

*Updated 24 October 2026*