---
title: "Reevaluating Adversarial Examples in Natural Language — paper by Morris et al. (2020)"
description: "The authors propose a common framework. 39 references, cited by 59 papers here."
authors: ["John Morris", "Eli Lifland", "Jack Lanchantin", "Yangfeng Ji", "Yanjun Qi"]
year: 2020
venue: "Findings of the Association for Computational Linguistics: EMNLP 2020"
source: "https://aclanthology.org/2020.findings-emnlp.341/"
doi: "10.18653/v1/2020.findings-emnlp.341"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/morris-2020-reevaluating-adversarial-examples-natural-language/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/morris-2020-reevaluating-adversarial-examples-natural-language/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2020](https://md.research.elamur.ai/free/en/papers/2020/page/4/) / Reevaluating Adversarial Examples in Natural Language

# Reevaluating Adversarial Examples in Natural Language

**Authors:** [John Morris](https://md.research.elamur.ai/free/en/authors/john-x-morris/), [Eli Lifland](https://md.research.elamur.ai/free/en/authors/eli-lifland/), [Jack Lanchantin](https://md.research.elamur.ai/free/en/authors/jack-lanchantin/), [Yangfeng Ji](https://md.research.elamur.ai/free/en/authors/yangfeng-ji/) and [Yanjun Qi](https://md.research.elamur.ai/free/en/authors/yanjun-qi/) · **Year:** 2020 · **Venue:** [Findings of the Association for Computational Linguistics: EMNLP 2020](https://md.research.elamur.ai/free/en/venues/findings/) ([Findings of the Association for Computational Linguistics 2020](https://md.research.elamur.ai/free/en/venues/findings/2020/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2020.findings-emnlp.341/) · [PDF](https://aclanthology.org/2020.findings-emnlp.341.pdf) · [DOI 10.18653/v1/2020.findings-emnlp.341](https://doi.org/10.18653/v1/2020.findings-emnlp.341) · [OpenAlex](https://openalex.org/W3021148167)

**Citations:** 92 citations (OpenAlex, 2026-10-14) · [59](https://md.research.elamur.ai/free/en/papers/morris-2020-reevaluating-adversarial-examples-natural-language/cited-by/) in our library, of the 141,840 papers whose bibliographies we read

## Summary
Research on adversarial attacks against NLP models has no agreed notion of what makes an attack successful. This makes published attacks hard to compare and limits their usefulness for analysing models. The authors propose a common framework. A valid textual adversarial example must change the model's output while satisfying four kinds of linguistic constraint: semantics, grammaticality, overlap and non-suspicion. They classify earlier attacks by these constraints and suggest human and automatic ways to check each one.

Applying these checks to two leading synonym-substitution attacks, GENETICATTACK and TEXTFOOLER, shows that many of their perturbations change meaning, look suspicious or introduce grammar errors. Human studies were used to set stricter thresholds for word-embedding and sentence-encoder similarity, and a grammar checker was added. The result is TFADJUSTED, which yields better examples but succeeds far less often.

Using TextAttack, the authors kept the constraints fixed and compared only the search strategies. This revealed that TEXTFOOLER's reported advantage came from looser constraints, not from a better search. The work argues that new search methods should be evaluated separately from changes to constraints.

## Key points
- **Error rates:** About 38% of perturbations from the evaluated attacks introduce grammatical errors. A grammar checker flags 39% more errors in perturbed texts than in the originals.
- **TFADJUSTED trade-off:** It keeps TEXTFOOLER's greedy search but tightens the similarity constraints and adds grammar validation. The attack success rate drops from above 80% to below 20%, a fall of more than 70 percentage points. Its examples reach semantic-preservation Likert scores above 4.
- **Most restrictive constraint:** The word-embedding distance constraint reduced attack success the most, followed by the sentence-encoder constraint.
- **Search comparison (BERT fine-tuned on MR, 1000 samples):** GENETICATTACK's genetic algorithm beats TEXTFOOLER's greedy search under both constraint sets.
  - TFADJUSTED constraints: 12.0% vs 10.6% success.
  - TEXTFOOLER constraints: 95.0% vs 91.1% success.
  - The genetic search needs more than 40x as many model queries.
- **Adversarial training:** Training on TEXTFOOLER's examples lowered model accuracy, while training on TFADJUSTED's examples did not. Code and data are released.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [University of Virginia](https://md.research.elamur.ai/free/en/universities/university-of-virginia/) — University, Charlottesville, United States · 18 works · 8,786 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 9,086 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 8,889 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 9,296 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 9,206 works

## Cite

Cite as: John Morris, Eli Lifland, Jack Lanchantin, Yangfeng Ji and Yanjun Qi. 2020. Reevaluating Adversarial Examples in Natural Language. Findings of the Association for Computational Linguistics: EMNLP 2020.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{morris2020reevaluating,
  title = {{Reevaluating Adversarial Examples in Natural Language}},
  author = {John Morris, Eli Lifland, Jack Lanchantin, Yangfeng Ji and Yanjun Qi},
  booktitle = {Findings of the Association for Computational Linguistics: EMNLP 2020},
  year = {2020},
  doi = {10.18653/v1/2020.findings-emnlp.341},
  url = {https://doi.org/10.18653/v1/2020.findings-emnlp.341}
}
```

- [← previous Exploring Versatile Generative Language Model Via Parameter-Efficient Transfer Learning](https://md.research.elamur.ai/free/en/papers/lin-2020-exploring-versatile-generative-language-model/)
- [next → Will-They-Won't-They: A Very Large Dataset for Stance Detection on Twitter](https://md.research.elamur.ai/free/en/papers/conforti-2020-will-they-won-t-they/)
- [↑ back to papers of 2020](https://md.research.elamur.ai/free/en/papers/2020/page/4/)

## References

39 references, 21 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Moustafa Alzantot, Yash Sharma, Ahmed Elgohary, Bo-Jhang Ho, Mani Srivastava, and Kai-Wei Chang. 2018. Generating natural language adversarial examples. arXiv preprint arXiv:1804.07998. [our card](https://md.research.elamur.ai/free/en/papers/alzantot-2018-generating-natural-language-adversarial-examples/)
2. Samuel R. Bowman, Gabor Angeli, Christopher Potts, and Christopher D. Manning. 2015. A large annotated corpus for learning natural language inference. CoRR, abs/1508.05326. [find](https://scholar.google.com/scholar?q=Samuel+R%2E+Bowman%2C+Gabor+Angeli%2C+Christopher+Potts%2C+and+Christopher+D%2E+Manning%2E+2015%2E+A+large+annotated+corpus+for+learning+natural+language+inference%2E+CoRR%2C+abs%2F1508%2E05326%2E)
3. Daniel Cer, Mona Diab, Eneko Agirre, Iñigo Lopez-Gazpio, and Lucia Specia. 2017. SemEval-2017 task 1: Semantic textual similarity multilingual and crosslingual focused evaluation. In Proceedings of the 11th International Workshop on Semantic Evaluation (SemEval-2017), pages 1–14, Vancouver, Canada.… [our card](https://md.research.elamur.ai/free/en/papers/cer-2017-semeval-2017-task-1-semantic/)
4. Daniel Cer, Yinfei Yang, Sheng yi Kong, Nan Hua, Nicole Limtiaco, Rhomni St. John, Noah Constant, Mario Guajardo-Cespedes, Steve Yuan, Chris Tar, Yun-Hsuan Sung, Brian Strope, and Ray Kurzweil. 2018. Universal sentence encoder. ArXiv, abs/1803.11175. [our card](https://md.research.elamur.ai/free/en/papers/cer-2018-universal-sentence-encoder/)
5. Minhao Cheng, Jinfeng Yi, Huan Zhang, Pin-Yu Chen, and Cho-Jui Hsieh. 2018. Seq2sick: Evaluating the robustness of sequence-to-sequence models with adversarial examples. arXiv preprint arXiv:1803.01128. — [arXiv:1803.01128](https://arxiv.org/abs/1803.01128) [find](https://scholar.google.com/scholar?q=Minhao+Cheng%2C+Jinfeng+Yi%2C+Huan+Zhang%2C+Pin%2DYu+Chen%2C+and+Cho%2DJui+Hsieh%2E+2018%2E+Seq2sick%3A+Evaluating+the+robustness+of+sequence%2Dto%2Dsequence+models+with+adversarial+examples%2E+arXiv+preprint+arXiv%3A1803%2E0112)
6. Michael Denkowski and Alon Lavie. 2014. Meteor universal: Language specific translation evaluation for any target language. In Proceedings of the Ninth Workshop on Statistical Machine Translation, pages 376–380, Baltimore, Maryland, USA. Association for Computational Linguistics. [our card](https://md.research.elamur.ai/free/en/papers/denkowski-2014-meteor-universal-language-specific-translation/)
7. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805. [our card](https://md.research.elamur.ai/free/en/papers/devlin-2019-bert-pre-training-deep-bidirectional/)
8. Javid Ebrahimi, Anyi Rao, Daniel Lowd, and Dejing Dou. 2017. Hotflip: White-box adversarial examples for text classification. arXiv preprint arXiv:1712.06751. [our card](https://md.research.elamur.ai/free/en/papers/ebrahimi-2018-hotflip-white-box-adversarial-examples/)
9. Ji Gao, Jack Lanchantin, Mary Lou Soffa, and Yanjun Qi. 2018. Black-box generation of adversarial text sequences to evade deep learning classifiers. In IEEE Security and Privacy Workshops (SPW). [our card](https://md.research.elamur.ai/free/en/papers/gao-2018-black-box-generation-adversarial-text/)
10. Justin Gilmer, Ryan P. Adams, Ian J. Goodfellow, David Andersen, and George E. Dahl. 2018. Motivating the rules of the game for adversarial example research. CoRR, abs/1807.06732. — [arXiv:1807.06732](https://arxiv.org/abs/1807.06732) [find](https://scholar.google.com/scholar?q=Justin+Gilmer%2C+Ryan+P%2E+Adams%2C+Ian+J%2E+Goodfellow%2C+David+Andersen%2C+and+George+E%2E+Dahl%2E+2018%2E+Motivating+the+rules+of+the+game+for+adversarial+example+research%2E+CoRR%2C+abs%2F1807%2E06732%2E+%E2%80%94+%5BarXiv%3A1807%2E06732%5D)
11. Ian Goodfellow, Nicolas Papernot, Sandy Huang, Rocky Duan, Pieter Abbeel, and Jack Clark. Attacking machine learning with adversarial examples [online]. 2017. [find](https://scholar.google.com/scholar?q=Ian+Goodfellow%2C+Nicolas+Papernot%2C+Sandy+Huang%2C+Rocky+Duan%2C+Pieter+Abbeel%2C+and+Jack+Clark%2E+Attacking+machine+learning+with+adversarial+examples+%5Bonline%5D%2E+2017%2E)
12. Ian J Goodfellow, Jonathon Shlens, and Christian Szegedy. 2014. Explaining and harnessing adversarial examples. arXiv preprint arXiv:1412.6572. [our card](https://md.research.elamur.ai/free/en/papers/goodfellow-2014-explaining-harnessing-adversarial-examples/)
13. Po-Sen Huang, Robert Stanforth, Johannes Welbl, Chris Dyer, Dani Yogatama, Sven Gowal, Krishnamurthy Dvijotham, and Pushmeet Kohli. 2019. Achieving verified robustness to symbol substitutions via interval bound propagation. ArXiv, abs/1909.01492. [our card](https://md.research.elamur.ai/free/en/papers/huang-2019-achieving-verified-robustness-symbol-substitutions/)
14. Mohit Iyyer, John Wieting, Kevin Gimpel, and Luke Zettlemoyer. 2018. Adversarial example generation with syntactically controlled paraphrase networks. CoRR, abs/1804.06059. [our card](https://md.research.elamur.ai/free/en/papers/iyyer-2018-adversarial-example-generation-syntactically-controlled/)
15. Di Jin, Zhijing Jin, Joey Tianyi Zhou, and Peter Szolovits. 2019. Is BERT Really Robust? A Strong Baseline for Natural Language Attack on Text Classification and Entailment. arXiv e-prints, page arXiv:1907.11932. [our card](https://md.research.elamur.ai/free/en/papers/jin-2019-bert-really-robust-strong-baseline/)
16. Volodymyr Kuleshov, Shantanu Thakoor, Tingfung Lau, and Stefano Ermon. 2018. Adversarial examples for natural language classification problems. [find](https://scholar.google.com/scholar?q=Volodymyr+Kuleshov%2C+Shantanu+Thakoor%2C+Tingfung+Lau%2C+and+Stefano+Ermon%2E+2018%2E+Adversarial+examples+for+natural+language+classification+problems%2E)
17. Jinfeng Li, Shouling Ji, Tianyu Du, Bo Li, and Ting Wang. 2018. Textbugger: Generating adversarial text against real-world applications. arXiv preprint arXiv:1812.05271. [our card](https://md.research.elamur.ai/free/en/papers/li-2018-textbugger-generating-adversarial-text-against/)
18. Bin Liang, Hongcheng Li, Miaoqiang Su, Pan Bian, Xirong Li, and Wenchang Shi. 2017. Deep text classification can be fooled. arXiv preprint arXiv:1704.08006. — [arXiv:1704.08006](https://arxiv.org/abs/1704.08006) [find](https://scholar.google.com/scholar?q=Bin+Liang%2C+Hongcheng+Li%2C+Miaoqiang+Su%2C+Pan+Bian%2C+Xirong+Li%2C+and+Wenchang+Shi%2E+2017%2E+Deep+text+classification+can+be+fooled%2E+arXiv+preprint+arXiv%3A1704%2E08006%2E+%E2%80%94+%5BarXiv%3A1704%2E08006%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F)
19. R. Likert. 1932. A Technique for the Measurement of Attitudes. Number nos. 136-165 in A Technique for the Measurement of Attitudes. publisher not identified. [find](https://scholar.google.com/scholar?q=R%2E+Likert%2E+1932%2E+A+Technique+for+the+Measurement+of+Attitudes%2E+Number+nos%2E+136%2D165+in+A+Technique+for+the+Measurement+of+Attitudes%2E+publisher+not+identified%2E)
20. Zachary Chase Lipton and Jacob Steinhardt. 2018. Troubling trends in machine learning scholarship. ArXiv, abs/1807.03341. [find](https://scholar.google.com/scholar?q=Zachary+Chase+Lipton+and+Jacob+Steinhardt%2E+2018%2E+Troubling+trends+in+machine+learning+scholarship%2E+ArXiv%2C+abs%2F1807%2E03341%2E)
21. Paul Michel, Xian Li, Graham Neubig, and Juan Miguel Pino. 2019. On evaluation of adversarial perturbations for sequence-to-sequence models. CoRR, abs/1903.06620. [our card](https://md.research.elamur.ai/free/en/papers/michel-2019-evaluation-adversarial-perturbations-sequence-sequence/)
22. John X. Morris, Eli Lifland, Jin Yong Yoo, and Yanjun Qi. 2020. Textattack: A framework for adversarial attacks in natural language processing. [find](https://scholar.google.com/scholar?q=John+X%2E+Morris%2C+Eli+Lifland%2C+Jin+Yong+Yoo%2C+and+Yanjun+Qi%2E+2020%2E+Textattack%3A+A+framework+for+adversarial+attacks+in+natural+language+processing%2E)
23. Nikola Mrksic, Diarmuid Ó Séaghdha, Blaise Thomson, Milica Gasic, Lina Maria Rojas-Barahona, Pei hao Su, David Vandyke, Tsung-Hsien Wen, and Steve J. Young. 2016. Counter-fitting word vectors to linguistic constraints. In HLT-NAACL. [find](https://scholar.google.com/scholar?q=Nikola+Mrksic%2C+Diarmuid+O%CC%81+Se%CC%81aghdha%2C+Blaise+Thomson%2C+Milica+Gasic%2C+Lina+Maria+Rojas%2DBarahona%2C+Pei+hao+Su%2C+David+Vandyke%2C+Tsung%2DHsien+Wen%2C+and+Steve+J%2E+Young%2E+2016%2E+Counter%2Dfitting+word+vectors+to+lin)
24. Daniel Naber. 2003. A rule-based style and grammar checker. [find](https://scholar.google.com/scholar?q=Daniel+Naber%2E+2003%2E+A+rule%2Dbased+style+and+grammar+checker%2E)
25. Bo Pang and Lillian Lee. 2005. Seeing stars: Exploiting class relationships for sentiment categorization with respect to rating scales. In Proceedings of the 43rd Annual Meeting of the Association for Computational Linguistics (ACL’05), pages 115– 124, Ann Arbor, Michigan. Association for Computatio… [our card](https://md.research.elamur.ai/free/en/papers/pang-2005-seeing-stars-exploiting-class-relationships/)
26. Nicolas Papernot, Patrick McDaniel, Ananthram Swami, and Richard Harang. 2016. Crafting adversarial input sequences for recurrent neural networks. In Military Communications Conference, MILCOM 2016-2016 IEEE, pages 49–54. IEEE. [find](https://scholar.google.com/scholar?q=Nicolas+Papernot%2C+Patrick+McDaniel%2C+Ananthram+Swami%2C+and+Richard+Harang%2E+2016%2E+Crafting+adversarial+input+sequences+for+recurrent+neural+networks%2E+In+Military+Communications+Conference%2C+MILCOM+2016%2D20)
27. Rowan Zellers, Ari Holtzman, Hannah Rashkin, Yonatan Bisk, Ali Farhadi, Franziska Roesner, and Yejin Choi. 2019. Defending against neural fake news. CoRR, abs/1905.12616. [our card](https://md.research.elamur.ai/free/en/papers/zellers-2019-defending-against-neural-fake-news/)
28. Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. Bleu: a method for automatic evaluation of machine translation. In Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, pages 311–318, Philadelphia, Pennsylvania, USA. Association for Computational… [our card](https://md.research.elamur.ai/free/en/papers/papineni-2002-bleu-method-automatic-evaluation-machine/)
29. Maja Popović. 2015. chrF: character n-gram f-score for automatic MT evaluation. In Proceedings of the Tenth Workshop on Statistical Machine Translation, pages 392–395, Lisbon, Portugal. Association for Computational Linguistics. [our card](https://md.research.elamur.ai/free/en/papers/popovic-2015-chrf-character-n-gram-f/)
30. Wei Emma Zhang, Quan Z. Sheng, and Ahoud Abdul-rahmn F. Alhazmi. 2019. Generating textual adversarial examples for deep learning models: A survey. CoRR, abs/1901.06796. [find](https://scholar.google.com/scholar?q=Wei+Emma+Zhang%2C+Quan+Z%2E+Sheng%2C+and+Ahoud+Abdul%2Drahmn+F%2E+Alhazmi%2E+2019%2E+Generating+textual+adversarial+examples+for+deep+learning+models%3A+A+survey%2E+CoRR%2C+abs%2F1901%2E06796%2E)
31. Xiang Zhang, Junbo Zhao, and Yann LeCun. 2015. Character-level convolutional networks for text classification. In Advances in neural information processing systems, pages 649–657. [our card](https://md.research.elamur.ai/free/en/papers/zhang-2015-character-level-convolutional-networks-text/)
32. Zhengli Zhao, Dheeru Dua, and Sameer Singh. 2017. Generating natural adversarial examples. arXiv preprint arXiv:1710.11342. [find](https://scholar.google.com/scholar?q=Zhengli+Zhao%2C+Dheeru+Dua%2C+and+Sameer+Singh%2E+2017%2E+Generating+natural+adversarial+examples%2E+arXiv+preprint+arXiv%3A1710%2E11342%2E)
33. Nils Reimers and Iryna Gurevych. 2019. Sentence-bert: Sentence embeddings using siamese bert-networks. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics. [our card](https://md.research.elamur.ai/free/en/papers/reimers-2019-sentence-bert-sentence-embeddings-siamese/)
34. Shuhuai Ren, Yihe Deng, Kun He, and Wanxiang Che. 2019. Generating natural language adversarial examples through probability weighted word saliency. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 1085–1097, Florence, Italy. Association for Computati… [our card](https://md.research.elamur.ai/free/en/papers/ren-2019-generating-natural-language-adversarial-examples/)
35. Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. 2018. Semantically equivalent adversarial rules for debugging nlp models. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 856–865. [find](https://scholar.google.com/scholar?q=Marco+Tulio+Ribeiro%2C+Sameer+Singh%2C+and+Carlos+Guestrin%2E+2018%2E+Semantically+equivalent+adversarial+rules+for+debugging+nlp+models%2E+In+Proceedings+of+the+56th+Annual+Meeting+of+the+Association+for+Compu)
36. Suranjana Samanta and Sameep Mehta. 2017. Towards crafting text adversarial samples. arXiv preprint arXiv:1707.02812. [find](https://scholar.google.com/scholar?q=Suranjana+Samanta+and+Sameep+Mehta%2E+2017%2E+Towards+crafting+text+adversarial+samples%2E+arXiv+preprint+arXiv%3A1707%2E02812%2E)
37. Alex Warstadt, Amanpreet Singh, and Samuel R. Bowman. 2018. Neural network acceptability judgments. CoRR, abs/1805.12471. [find](https://scholar.google.com/scholar?q=Alex+Warstadt%2C+Amanpreet+Singh%2C+and+Samuel+R%2E+Bowman%2E+2018%2E+Neural+network+acceptability+judgments%2E+CoRR%2C+abs%2F1805%2E12471%2E)
38. John Wieting and Kevin Gimpel. 2018. Paranmt-50m: Pushing the limits of paraphrastic sentence embeddings with millions of machine translations. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 451–462. [our card](https://md.research.elamur.ai/free/en/papers/wieting-2018-paranmt-50m-pushing-limits-paraphrastic/)
39. Adina Williams, Nikita Nangia, and Samuel Bowman. 2018. A broad-coverage challenge corpus for sentence understanding through inference. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume [find](https://scholar.google.com/scholar?q=Adina+Williams%2C+Nikita+Nangia%2C+and+Samuel+Bowman%2E+2018%2E+A+broad%2Dcoverage+challenge+corpus+for+sentence+understanding+through+inference%2E+In+Proceedings+of+the+2018+Conference+of+the+North+American+Chap)

*Updated 24 October 2026*