---
title: "Gradient-based Adversarial Attacks against Text — Guo 2021"
description: "Crafting adversarial inputs for text models is difficult because tokens are discrete, which rules out ordinary gradient descent, and because there is no…"
authors: ["Chuan Guo", "Alexandre Sablayrolles", "Hervé Jégou", "Douwe Kiela"]
year: 2021
venue: "Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing"
source: "https://aclanthology.org/2021.emnlp-main.464/"
doi: "10.18653/v1/2021.emnlp-main.464"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/guo-2021-gradient-based-adversarial-attacks-against/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/guo-2021-gradient-based-adversarial-attacks-against/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2021](https://md.research.elamur.ai/free/en/papers/2021/page/3/) / Gradient-based Adversarial Attacks against Text Transformers

# Gradient-based Adversarial Attacks against Text Transformers

**Authors:** Chuan Guo, [Alexandre Sablayrolles](https://md.research.elamur.ai/free/en/authors/alexandre-sablayrolles/), [Hervé Jégou](https://md.research.elamur.ai/free/en/authors/herve-jegou/), [Douwe Kiela](https://md.research.elamur.ai/free/en/authors/douwe-kiela/) · **Year:** 2021 · **Venue:** Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2021.emnlp-main.464/) · [PDF](https://aclanthology.org/2021.emnlp-main.464.pdf) · [DOI 10.18653/v1/2021.emnlp-main.464](https://doi.org/10.18653/v1/2021.emnlp-main.464) · [OpenAlex](https://openalex.org/W3158360872)

**Citations:** 106 citations (openalex-snapshot · 2026-10-14) · [81](#cited-by) in our library, of the 140884 papers whose bibliographies we read

## Summary
Crafting adversarial inputs for text models is difficult because tokens are discrete, which rules out ordinary gradient descent, and because there is no simple measure of how noticeable a change is. Earlier attacks relied on heuristic word or character swaps and black-box greedy or beam search. These often produced unnatural text and missed stronger attacks. The authors introduce GBDA (Gradient-based Distributional Attack), a white-box method for transformers. Rather than editing one sentence, it learns a continuous matrix that defines a distribution over token sequences, relaxed with Gumbel-softmax so that gradients can pass through. Two differentiable soft penalties guide the optimization: language-model perplexity keeps the text fluent, and BERTScore preserves meaning. GBDA was tested on news categorization, sentiment analysis and natural language inference. It reached state-of-the-art success rates with comparable imperceptibility under automatic and human evaluation. Samples drawn from the learned distribution also enable a black-box transfer attack that needs only hard-label outputs. This narrows the gap between text attacks and the much stronger attacks used on images.

## Key points
- On AG News, GBDA lowers the target model's accuracy from 95.1 to 3.5. BERT-Attack only brings it down to 10.6.
- In an Amazon Mechanical Turk study with more than 3000 samples (Yelp, BERT target), annotators identified BAE examples 78.04% of the time and GBDA examples 76.06% of the time. Accuracy after the attack was 4.7% for GBDA and 12.0% for BAE.
- Tokens that change after re-tokenization occur in about 2% of cases. Reported results are computed after re-tokenization, and the affected examples generally stay adversarial.
- Each adversarial example takes about 20 seconds to generate, which is comparable to BERT-Attack.
- The method has two main limitations. It supports only token substitutions, with no insertions or deletions. Its parameter matrix Θ has size n×V, which is heavily over-parameterized.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

Fields: [Natural language processing](https://md.research.elamur.ai/free/en/shelves/nlp/)

## Cited by

Cited by

81 papers of our library cite it (of the 140884 whose bibliographies we read); 8 of them have a page here, 73 not yet.

- [Security and Privacy Challenges of Large Language Models: A Survey](https://md.research.elamur.ai/free/en/papers/das-2024-security-privacy-challenges-large-language/)
- [Catastrophic Jailbreak of Open-source LLMs via Exploiting Generation](https://md.research.elamur.ai/free/en/papers/huang-2023-catastrophic-jailbreak-open-source-llms/)
- [Baseline Defenses for Adversarial Attacks Against Aligned Language Models](https://md.research.elamur.ai/free/en/papers/jain-2023-baseline-defenses-adversarial-attacks-against/)
- [HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusal](https://md.research.elamur.ai/free/en/papers/mazeika-2024-harmbench-standardized-evaluation-framework-automated/)
- [Tree of Attacks: Jailbreaking Black-Box LLMs Automatically](https://md.research.elamur.ai/free/en/papers/mehrotra-2023-tree-attacks-jailbreaking-black-box/)
- [How Johnny Can Persuade LLMs to Jailbreak Them: Rethinking Persuasion to Challenge AI Safety by Humanizing LLMs](https://md.research.elamur.ai/free/en/papers/zeng-2024-how-johnny-can-persuade-llms/)
- [SPARTA: Evaluating Reasoning Segmentation Robustness through Black-Box Adversarial Paraphrasing in Text Autoencoder Latent Space](https://md.research.elamur.ai/free/en/papers/zinkovich-2026-sparta-evaluating-reasoning-segmentation-robustness/)
- [Universal and Transferable Adversarial Attacks on Aligned Language Models](https://md.research.elamur.ai/free/en/papers/zou-2023-universal-transferable-adversarial-attacks-aligned/)

## Cite

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{guo2021gradient,
  title = {{Gradient-based Adversarial Attacks against Text Transformers}},
  author = {Guo, Chuan and Sablayrolles, Alexandre and Jégou, Hervé and Kiela, Douwe},
  booktitle = {Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing},
  year = {2021},
  doi = {10.18653/v1/2021.emnlp-main.464},
  url = {https://doi.org/10.18653/v1/2021.emnlp-main.464}
}
```

- [← previous GPT3Mix: Leveraging Large-scale Language Models for Text Augmentation](https://md.research.elamur.ai/free/en/papers/yoo-2021-gpt3mix-leveraging-large-scale-language/)
- [next → Grammar-Based Concept Alignment for Domain-Specific Machine Translation](https://md.research.elamur.ai/free/en/papers/masciolini-2021-grammar-based-concept-alignment-domain/)
- [↑ back to papers of 2021](https://md.research.elamur.ai/free/en/papers/2021/page/3/)

## References

39 references, 18 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Naveed Akhtar and Ajmal Mian. 2018. Threat of adversarial attacks on deep learning in computer vision: A survey. Ieee Access, 6:14410–14430. [find](https://scholar.google.com/scholar?q=Naveed+Akhtar+and+Ajmal+Mian%2E+2018%2E+Threat+of+adversarial+attacks+on+deep+learning+in+computer+vision%3A+A+survey%2E+Ieee+Access%2C+6%3A14410%E2%80%9314430%2E)
2. Moustafa Alzantot, Yash Sharma, Ahmed Elgohary, Bo-Jhang Ho, Mani Srivastava, and Kai-Wei Chang. 2018. Generating natural language adversarial examples. arXiv preprint arXiv:1804.07998. [find](https://scholar.google.com/scholar?q=Moustafa+Alzantot%2C+Yash+Sharma%2C+Ahmed+Elgohary%2C+Bo%2DJhang+Ho%2C+Mani+Srivastava%2C+and+Kai%2DWei+Chang%2E+2018%2E+Generating+natural+language+adversarial+examples%2E+arXiv+preprint+arXiv%3A1804%2E07998%2E) [our card](https://md.research.elamur.ai/free/en/papers/alzantot-2018-generating-natural-language-adversarial-examples/)
3. Battista Biggio, Igino Corona, Davide Maiorca, Blaine Nelson, Nedim Šrndić, Pavel Laskov, Giorgio Giacinto, and Fabio Roli. 2013. Evasion attacks against machine learning at test time. In Joint European conference on machine learning and knowledge discovery in databases, pages 387–402. Springer. [find](https://scholar.google.com/scholar?q=Battista+Biggio%2C+Igino+Corona%2C+Davide+Maiorca%2C+Blaine+Nelson%2C+Nedim+%C5%A0rndic%CC%81%2C+Pavel+Laskov%2C+Giorgio+Giacinto%2C+and+Fabio+Roli%2E+2013%2E+Evasion+attacks+against+machine+learning+at+test+time%2E+In+Joint+Europ)
4. Nicholas Carlini and David Wagner. 2017. Towards evaluating the robustness of neural networks. In 2017 ieee symposium on security and privacy (sp), pages 39–57. IEEE. [find](https://scholar.google.com/scholar?q=Nicholas+Carlini+and+David+Wagner%2E+2017%2E+Towards+evaluating+the+robustness+of+neural+networks%2E+In+2017+ieee+symposium+on+security+and+privacy+%28sp%29%2C+pages+39%E2%80%9357%2E+IEEE%2E)
5. Nicholas Carlini and David Wagner. 2018. Audio adversarial examples: Targeted attacks on speech-to-text. In 2018 IEEE Security and Privacy Workshops (SPW), pages 1–7. IEEE. [find](https://scholar.google.com/scholar?q=Nicholas+Carlini+and+David+Wagner%2E+2018%2E+Audio+adversarial+examples%3A+Targeted+attacks+on+speech%2Dto%2Dtext%2E+In+2018+IEEE+Security+and+Privacy+Workshops+%28SPW%29%2C+pages+1%E2%80%937%2E+IEEE%2E)
6. Daniel Cer, Yinfei Yang, Sheng-yi Kong, Nan Hua, Nicole Limtiaco, Rhomni St John, Noah Constant, Mario Guajardo-Céspedes, Steve Yuan, Chris Tar, et al. 2018. Universal sentence encoder. arXiv preprint arXiv:1803.11175. [find](https://scholar.google.com/scholar?q=Daniel+Cer%2C+Yinfei+Yang%2C+Sheng%2Dyi+Kong%2C+Nan+Hua%2C+Nicole+Limtiaco%2C+Rhomni+St+John%2C+Noah+Constant%2C+Mario+Guajardo%2DC%C3%A9spedes%2C+Steve+Yuan%2C+Chris+Tar%2C+et+al%2E+2018%2E+Universal+sentence+encoder%2E+arXiv+preprint) [our card](https://md.research.elamur.ai/free/en/papers/cer-2018-universal-sentence-encoder/)
7. Hanjun Dai, Hui Li, Tian Tian, Xin Huang, Lin Wang, Jun Zhu, and Le Song. 2018. Adversarial attack on graph structured data. In International conference on machine learning, pages 1115–1124. PMLR. [find](https://scholar.google.com/scholar?q=Hanjun+Dai%2C+Hui+Li%2C+Tian+Tian%2C+Xin+Huang%2C+Lin+Wang%2C+Jun+Zhu%2C+and+Le+Song%2E+2018%2E+Adversarial+attack+on+graph+structured+data%2E+In+International+conference+on+machine+learning%2C+pages+1115%E2%80%931124%2E+PMLR%2E)
8. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies,… [find](https://scholar.google.com/scholar?q=Jacob+Devlin%2C+Ming%2DWei+Chang%2C+Kenton+Lee%2C+and+Kristina+Toutanova%2E+2019%2E+BERT%3A+Pre%2Dtraining+of+deep+bidirectional+transformers+for+language+understanding%2E+In+Proceedings+of+the+2019+Conference+of+the+N)
9. Javid Ebrahimi, Anyi Rao, Daniel Lowd, and Dejing Dou. 2017. Hotflip: White-box adversarial examples for text classification. arXiv preprint arXiv:1712.06751. [find](https://scholar.google.com/scholar?q=Javid+Ebrahimi%2C+Anyi+Rao%2C+Daniel+Lowd%2C+and+Dejing+Dou%2E+2017%2E+Hotflip%3A+White%2Dbox+adversarial+examples+for+text+classification%2E+arXiv+preprint+arXiv%3A1712%2E06751%2E) [our card](https://md.research.elamur.ai/free/en/papers/ebrahimi-2018-hotflip-white-box-adversarial-examples/)
10. Ji Gao, Jack Lanchantin, Mary Lou Soffa, and Yanjun Qi. 2018. Black-box generation of adversarial text sequences to evade deep learning classifiers. In 2018 IEEE Security and Privacy Workshops (SPW), pages 50–56. IEEE. [find](https://scholar.google.com/scholar?q=Ji+Gao%2C+Jack+Lanchantin%2C+Mary+Lou+Soffa%2C+and+Yanjun+Qi%2E+2018%2E+Black%2Dbox+generation+of+adversarial+text+sequences+to+evade+deep+learning+classifiers%2E+In+2018+IEEE+Security+and+Privacy+Workshops+%28SPW%29%2C+) [our card](https://md.research.elamur.ai/free/en/papers/gao-2018-black-box-generation-adversarial-text/)
11. Siddhant Garg and Goutham Ramakrishnan. 2020. BAE: BERT-based adversarial examples for text classification. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 6174–6181, Online. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Siddhant+Garg+and+Goutham+Ramakrishnan%2E+2020%2E+BAE%3A+BERT%2Dbased+adversarial+examples+for+text+classification%2E+In+Proceedings+of+the+2020+Conference+on+Empirical+Methods+in+Natural+Language+Processing+%28E) [our card](https://md.research.elamur.ai/free/en/papers/garg-2020-bae-bert-based-adversarial-examples/)
12. Eric Jang, Shixiang Gu, and Ben Poole. 2016. Categorical reparameterization with gumbel-softmax. arXiv preprint arXiv:1611.01144. [find](https://scholar.google.com/scholar?q=Eric+Jang%2C+Shixiang+Gu%2C+and+Ben+Poole%2E+2016%2E+Categorical+reparameterization+with+gumbel%2Dsoftmax%2E+arXiv+preprint+arXiv%3A1611%2E01144%2E) [our card](https://md.research.elamur.ai/free/en/papers/jang-2016-categorical-reparameterization-gumbel-softmax/)
13. Di Jin, Zhijing Jin, Joey Tianyi Zhou, and Peter Szolovits. 2020. Is bert really robust? a strong baseline for natural language attack on text classification and entailment. In Proceedings of the AAAI conference on artificial intelligence, volume 34, pages 8018–8025. [find](https://scholar.google.com/scholar?q=Di+Jin%2C+Zhijing+Jin%2C+Joey+Tianyi+Zhou%2C+and+Peter+Szolovits%2E+2020%2E+Is+bert+really+robust%3F+a+strong+baseline+for+natural+language+attack+on+text+classification+and+entailment%2E+In+Proceedings+of+the+AAAI) [our card](https://md.research.elamur.ai/free/en/papers/jin-2019-bert-really-robust-strong-baseline/)
14. Diederik P Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980. [find](https://scholar.google.com/scholar?q=Diederik+P+Kingma+and+Jimmy+Ba%2E+2014%2E+Adam%3A+A+method+for+stochastic+optimization%2E+arXiv+preprint+arXiv%3A1412%2E6980%2E)
15. Guillaume Lample and Alexis Conneau. 2019. Cross-lingual language model pretraining. arXiv preprint arXiv:1901.07291. [find](https://scholar.google.com/scholar?q=Guillaume+Lample+and+Alexis+Conneau%2E+2019%2E+Cross%2Dlingual+language+model+pretraining%2E+arXiv+preprint+arXiv%3A1901%2E07291%2E)
16. Zhenzhong Lan, Mingda Chen, Sebastian Goodman, Kevin Gimpel, Piyush Sharma, and Radu Soricut. 2019. Albert: A lite bert for self-supervised learning of language representations. arXiv preprint arXiv:1909.11942. [find](https://scholar.google.com/scholar?q=Zhenzhong+Lan%2C+Mingda+Chen%2C+Sebastian+Goodman%2C+Kevin+Gimpel%2C+Piyush+Sharma%2C+and+Radu+Soricut%2E+2019%2E+Albert%3A+A+lite+bert+for+self%2Dsupervised+learning+of+language+representations%2E+arXiv+preprint+arXiv%3A1)
17. Dianqi Li, Yizhe Zhang, Hao Peng, Liqun Chen, Chris Brockett, Ming-Ting Sun, and Bill Dolan. 2020a. Contextualized perturbation for textual adversarial attack. arXiv preprint arXiv:2009.07502. [find](https://scholar.google.com/scholar?q=Dianqi+Li%2C+Yizhe+Zhang%2C+Hao+Peng%2C+Liqun+Chen%2C+Chris+Brockett%2C+Ming%2DTing+Sun%2C+and+Bill+Dolan%2E+2020a%2E+Contextualized+perturbation+for+textual+adversarial+attack%2E+arXiv+preprint+arXiv%3A2009%2E07502%2E) [our card](https://md.research.elamur.ai/free/en/papers/li-2021-contextualized-perturbation-textual-adversarial-attack/)
18. Jinfeng Li, Shouling Ji, Tianyu Du, Bo Li, and Ting Wang. 2018. Textbugger: Generating adversarial text against real-world applications. arXiv preprint arXiv:1812.05271. [find](https://scholar.google.com/scholar?q=Jinfeng+Li%2C+Shouling+Ji%2C+Tianyu+Du%2C+Bo+Li%2C+and+Ting+Wang%2E+2018%2E+Textbugger%3A+Generating+adversarial+text+against+real%2Dworld+applications%2E+arXiv+preprint+arXiv%3A1812%2E05271%2E) [our card](https://md.research.elamur.ai/free/en/papers/li-2018-textbugger-generating-adversarial-text-against/)
19. Linyang Li, Ruotian Ma, Qipeng Guo, Xiangyang Xue, and Xipeng Qiu. 2020b. BERT-ATTACK: Adversarial attack against BERT using BERT. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 6193–6202, Online. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Linyang+Li%2C+Ruotian+Ma%2C+Qipeng+Guo%2C+Xiangyang+Xue%2C+and+Xipeng+Qiu%2E+2020b%2E+BERT%2DATTACK%3A+Adversarial+attack+against+BERT+using+BERT%2E+In+Proceedings+of+the+2020+Conference+on+Empirical+Methods+in+Natural) [our card](https://md.research.elamur.ai/free/en/papers/li-2020-bert-attack-adversarial-attack-against/)
20. Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692. [find](https://scholar.google.com/scholar?q=Yinhan+Liu%2C+Myle+Ott%2C+Naman+Goyal%2C+Jingfei+Du%2C+Mandar+Joshi%2C+Danqi+Chen%2C+Omer+Levy%2C+Mike+Lewis%2C+Luke+Zettlemoyer%2C+and+Veselin+Stoyanov%2E+2019%2E+Roberta%3A+A+robustly+optimized+bert+pretraining+approach%2E+a)
21. Andrew Maas, Raymond E Daly, Peter T Pham, Dan Huang, Andrew Y Ng, and Christopher Potts. 2011. Learning word vectors for sentiment analysis. In Proceedings of the 49th annual meeting of the association for computational linguistics: Human language technologies, pages 142–150. [find](https://scholar.google.com/scholar?q=Andrew+Maas%2C+Raymond+E+Daly%2C+Peter+T+Pham%2C+Dan+Huang%2C+Andrew+Y+Ng%2C+and+Christopher+Potts%2E+2011%2E+Learning+word+vectors+for+sentiment+analysis%2E+In+Proceedings+of+the+49th+annual+meeting+of+the+associati) [our card](https://md.research.elamur.ai/free/en/papers/maas-2011-learning-word-vectors-sentiment-analysis/)
22. Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. 2017. Towards deep learning models resistant to adversarial attacks. arXiv preprint arXiv:1706.06083. [find](https://scholar.google.com/scholar?q=Aleksander+Madry%2C+Aleksandar+Makelov%2C+Ludwig+Schmidt%2C+Dimitris+Tsipras%2C+and+Adrian+Vladu%2E+2017%2E+Towards+deep+learning+models+resistant+to+adversarial+attacks%2E+arXiv+preprint+arXiv%3A1706%2E06083%2E) [our card](https://md.research.elamur.ai/free/en/papers/madry-2017-towards-deep-learning-models-resistant/)
23. Rishabh Maheshwary, Saket Maheshwary, and Vikram Pudi. 2020. Generating natural language attacks in a hard label black box setting. arXiv preprint arXiv:2012.14956. — [arXiv:2012.14956](https://arxiv.org/abs/2012.14956) [find](https://scholar.google.com/scholar?q=Rishabh+Maheshwary%2C+Saket+Maheshwary%2C+and+Vikram+Pudi%2E+2020%2E+Generating+natural+language+attacks+in+a+hard+label+black+box+setting%2E+arXiv+preprint+arXiv%3A2012%2E14956%2E+%E2%80%94+%5BarXiv%3A2012%2E14956%5D%28https%3A%2F%2Farxiv%2E)
24. Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg Corrado, and Jeffrey Dean. 2013. Distributed representations of words and phrases and their compositionality. arXiv preprint arXiv:1310.4546. [find](https://scholar.google.com/scholar?q=Tomas+Mikolov%2C+Ilya+Sutskever%2C+Kai+Chen%2C+Greg+Corrado%2C+and+Jeffrey+Dean%2E+2013%2E+Distributed+representations+of+words+and+phrases+and+their+compositionality%2E+arXiv+preprint+arXiv%3A1310%2E4546%2E)
25. John Morris, Eli Lifland, Jack Lanchantin, Yangfeng Ji, and Yanjun Qi. 2020a. Reevaluating adversarial examples in natural language. In Findings of the Association for Computational Linguistics: EMNLP 2020, pages 3829–3839, Online. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=John+Morris%2C+Eli+Lifland%2C+Jack+Lanchantin%2C+Yangfeng+Ji%2C+and+Yanjun+Qi%2E+2020a%2E+Reevaluating+adversarial+examples+in+natural+language%2E+In+Findings+of+the+Association+for+Computational+Linguistics%3A+EMNLP)
26. Yuan Zang, Fanchao Qi, Chenghao Yang, Zhiyuan Liu, Meng Zhang, Qun Liu, and Maosong Sun. 2020. Word-level textual adversarial attacking as combinatorial optimization. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 6066–6080, Online. Association for… [find](https://scholar.google.com/scholar?q=Yuan+Zang%2C+Fanchao+Qi%2C+Chenghao+Yang%2C+Zhiyuan+Liu%2C+Meng+Zhang%2C+Qun+Liu%2C+and+Maosong+Sun%2E+2020%2E+Word%2Dlevel+textual+adversarial+attacking+as+combinatorial+optimization%2E+In+Proceedings+of+the+58th+Annual) [our card](https://md.research.elamur.ai/free/en/papers/zang-2020-word-level-textual-adversarial-attacking/)
27. Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q Weinberger, and Yoav Artzi. 2019. Bertscore: Evaluating text generation with bert. arXiv preprint arXiv:1904.09675. [find](https://scholar.google.com/scholar?q=Tianyi+Zhang%2C+Varsha+Kishore%2C+Felix+Wu%2C+Kilian+Q+Weinberger%2C+and+Yoav+Artzi%2E+2019%2E+Bertscore%3A+Evaluating+text+generation+with+bert%2E+arXiv+preprint+arXiv%3A1904%2E09675%2E)
28. John X Morris, Eli Lifland, Jin Yong Yoo, and Yanjun Qi. 2020b. Textattack: A framework for adversarial attacks in natural language processing. arXiv preprint arXiv:2005.05909. [find](https://scholar.google.com/scholar?q=John+X+Morris%2C+Eli+Lifland%2C+Jin+Yong+Yoo%2C+and+Yanjun+Qi%2E+2020b%2E+Textattack%3A+A+framework+for+adversarial+attacks+in+natural+language+processing%2E+arXiv+preprint+arXiv%3A2005%2E05909%2E) [our card](https://md.research.elamur.ai/free/en/papers/morris-2020-textattack-framework-adversarial-attacks-data/)
29. Xiang Zhang, Junbo Zhao, and Yann LeCun. 2015. Character-level convolutional networks for text classification. arXiv preprint arXiv:1509.01626. [find](https://scholar.google.com/scholar?q=Xiang+Zhang%2C+Junbo+Zhao%2C+and+Yann+LeCun%2E+2015%2E+Character%2Dlevel+convolutional+networks+for+text+classification%2E+arXiv+preprint+arXiv%3A1509%2E01626%2E) [our card](https://md.research.elamur.ai/free/en/papers/zhang-2015-character-level-convolutional-networks-text/)
30. Jeffrey Pennington, Richard Socher, and Christopher D Manning. 2014. Glove: Global vectors for word representation. In Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP), pages 1532–1543. [find](https://scholar.google.com/scholar?q=Jeffrey+Pennington%2C+Richard+Socher%2C+and+Christopher+D+Manning%2E+2014%2E+Glove%3A+Global+vectors+for+word+representation%2E+In+Proceedings+of+the+2014+conference+on+empirical+methods+in+natural+language+proce)
31. Daniel Zügner, Amir Akbarnejad, and Stephan Günnemann. 2018. Adversarial attacks on neural networks for graph data. In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pages 2847–2856. [find](https://scholar.google.com/scholar?q=Daniel+Z%C3%BCgner%2C+Amir+Akbarnejad%2C+and+Stephan+G%C3%BCnnemann%2E+2018%2E+Adversarial+attacks+on+neural+networks+for+graph+data%2E+In+Proceedings+of+the+24th+ACM+SIGKDD+International+Conference+on+Knowledge+Discover)
32. Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. 2019. Language models are unsupervised multitask learners. OpenAI blog, 1(8):9. [find](https://scholar.google.com/scholar?q=Alec+Radford%2C+Jeffrey+Wu%2C+Rewon+Child%2C+David+Luan%2C+Dario+Amodei%2C+and+Ilya+Sutskever%2E+2019%2E+Language+models+are+unsupervised+multitask+learners%2E+OpenAI+blog%2C+1%288%29%3A9%2E)
33. Shuhuai Ren, Yihe Deng, Kun He, and Wanxiang Che. 2019. Generating natural language adversarial examples through probability weighted word saliency. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 1085–1097, Florence, Italy. Association for Computati… [find](https://scholar.google.com/scholar?q=Shuhuai+Ren%2C+Yihe+Deng%2C+Kun+He%2C+and+Wanxiang+Che%2E+2019%2E+Generating+natural+language+adversarial+examples+through+probability+weighted+word+saliency%2E+In+Proceedings+of+the+57th+Annual+Meeting+of+the+As) [our card](https://md.research.elamur.ai/free/en/papers/ren-2019-generating-natural-language-adversarial-examples/)
34. Suranjana Samanta and Sameep Mehta. 2017. Towards crafting text adversarial samples. arXiv preprint arXiv:1707.02812. — [arXiv:1707.02812](https://arxiv.org/abs/1707.02812) [find](https://scholar.google.com/scholar?q=Suranjana+Samanta+and+Sameep+Mehta%2E+2017%2E+Towards+crafting+text+adversarial+samples%2E+arXiv+preprint+arXiv%3A1707%2E02812%2E+%E2%80%94+%5BarXiv%3A1707%2E02812%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F1707%2E02812%29)
35. Liwei Song, Xinwei Yu, Hsuan-Tung Peng, and Karthik Narasimhan. 2020. Universal adversarial attacks with natural triggers for text classification. arXiv preprint arXiv:2005.00174. [find](https://scholar.google.com/scholar?q=Liwei+Song%2C+Xinwei+Yu%2C+Hsuan%2DTung+Peng%2C+and+Karthik+Narasimhan%2E+2020%2E+Universal+adversarial+attacks+with+natural+triggers+for+text+classification%2E+arXiv+preprint+arXiv%3A2005%2E00174%2E)
36. Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, and Rob Fergus. 2013. Intriguing properties of neural networks. arXiv preprint arXiv:1312.6199. [find](https://scholar.google.com/scholar?q=Christian+Szegedy%2C+Wojciech+Zaremba%2C+Ilya+Sutskever%2C+Joan+Bruna%2C+Dumitru+Erhan%2C+Ian+Goodfellow%2C+and+Rob+Fergus%2E+2013%2E+Intriguing+properties+of+neural+networks%2E+arXiv+preprint+arXiv%3A1312%2E6199%2E) [our card](https://md.research.elamur.ai/free/en/papers/szegedy-2013-intriguing-properties-neural-networks/)
37. Eric Wallace, Shi Feng, Nikhil Kandpal, Matt Gardner, and Sameer Singh. 2019. Universal adversarial triggers for attacking and analyzing nlp. arXiv preprint arXiv:1908.07125. [find](https://scholar.google.com/scholar?q=Eric+Wallace%2C+Shi+Feng%2C+Nikhil+Kandpal%2C+Matt+Gardner%2C+and+Sameer+Singh%2E+2019%2E+Universal+adversarial+triggers+for+attacking+and+analyzing+nlp%2E+arXiv+preprint+arXiv%3A1908%2E07125%2E)
38. Adina Williams, Nikita Nangia, and Samuel R Bowman. 2017. A broad-coverage challenge corpus for sentence understanding through inference. arXiv preprint arXiv:1704.05426. [find](https://scholar.google.com/scholar?q=Adina+Williams%2C+Nikita+Nangia%2C+and+Samuel+R+Bowman%2E+2017%2E+A+broad%2Dcoverage+challenge+corpus+for+sentence+understanding+through+inference%2E+arXiv+preprint+arXiv%3A1704%2E05426%2E)
39. Zhilin Yang, Zihang Dai, Yiming Yang, Jaime Carbonell, Ruslan Salakhutdinov, and Quoc V Le. 2019. Xlnet: Generalized autoregressive pretraining for language understanding. arXiv preprint arXiv:1906.08237. [find](https://scholar.google.com/scholar?q=Zhilin+Yang%2C+Zihang+Dai%2C+Yiming+Yang%2C+Jaime+Carbonell%2C+Ruslan+Salakhutdinov%2C+and+Quoc+V+Le%2E+2019%2E+Xlnet%3A+Generalized+autoregressive+pretraining+for+language+understanding%2E+arXiv+preprint+arXiv%3A1906%2E08) [our card](https://md.research.elamur.ai/free/en/papers/yang-2019-xlnet-generalized-autoregressive-pretraining-language/)