---
title: "Data Augmentation using Pre-trained Transformer — Kumar 2020"
description: "Augmenting text data helps classifiers trained on very small datasets."
authors: ["Varun Kumar", "Ashutosh Choudhary", "Eunah Cho"]
year: 2020
venue: "arXiv:2003.02245"
source: "https://arxiv.org/abs/2003.02245"
doi: "10.48550/arxiv.2003.02245"
arxiv: "2003.02245"
licence: "arXiv — non-exclusive license to distribute"
licence_url: "http://arxiv.org/licenses/nonexclusive-distrib/1.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/kumar-2020-data-augmentation-using-pre-trained/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/kumar-2020-data-augmentation-using-pre-trained/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2020](https://md.research.elamur.ai/free/en/papers/2020/page/2/) / Data Augmentation using Pre-trained Transformer Models

# Data Augmentation using Pre-trained Transformer Models

**Authors:** Varun Kumar, Ashutosh Choudhary, Eunah Cho · **Year:** 2020 · **Venue:** arXiv:2003.02245 · **Licence:** [arXiv — non-exclusive license to distribute](http://arxiv.org/licenses/nonexclusive-distrib/1.0/)

**Links:** [Publisher's page](https://arxiv.org/abs/2003.02245) · [PDF](https://arxiv.org/pdf/2003.02245) · [DOI 10.48550/arxiv.2003.02245](https://doi.org/10.48550/arxiv.2003.02245) · [arXiv 2003.02245](https://arxiv.org/abs/2003.02245) · [OpenAlex](https://openalex.org/W3120948048)

**Citations:** 294 citations (openalex-snapshot · 2026-10-14) · [174](#cited-by) in our library, of the 140673 papers whose bibliographies we read

## Summary
Augmenting text data helps classifiers trained on very small datasets. Earlier generators often ignored class labels, so a synthetic sentence could invert its meaning (for example, flip its sentiment). Others relied on architecture-specific tricks, such as CBERT's reuse of BERT segment embeddings, and prior comparisons between methods were not run on equal terms. The authors propose a single generic recipe: fine-tune any pre-trained transformer on training examples with the class label prepended, then generate one synthetic example per original. They test three model families: an auto-encoder (BERT), an auto-regressive model (GPT2) and a seq2seq denoiser (BART). Experiments simulate a low-resource regime of 10 examples per class on SST-2, SNIPS and TREC, using a BERT classifier. Baselines are EDA, back-translation and CBERT. Every family improves accuracy, and the pre-trained seq2seq model performs best overall. The paper also analyses diversity and label preservation and gives practical guidelines. This makes it a useful model-agnostic starting point for augmenting small labeled corpora.

## Key points
- Two labeling schemes are compared. In *prepend*, the label is split into ordinary subwords. In *expand*, the label becomes a new vocabulary token, which needed about 150 training epochs for BERT. Plain prepending proved competitive and works for any model.
- Plain GPT2 generation tends to lose the class. Seeding the prompt with the label plus the first k = 3 words of an original example (GPT2 context) is needed to get useful data.
- BART is fine-tuned to reconstruct text with 40% of words masked, in two variants: BART word (single-word masks) and BART span (continuous chunks). Word-level masking beat subword masking. The masking ratio also controls output diversity.
- Type-token ratio analysis shows EDA gives the most diverse tri-grams and back-translation the most diverse unigrams. Seq2seq models balance diversity against semantic fidelity. RoBERTa prepend outperformed BERT prepend on TREC (66.12 vs 64.74 mean accuracy).
- The authors note two limitations: gains are hard to achieve in rich-resource settings, and no single generator wins on every dataset. Code is available at github.com/varinf/TransformersDataAugmentation.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

Fields: [Other](https://md.research.elamur.ai/free/en/shelves/other/)

## Cited by

Cited by

174 papers of our library cite it (of the 140673 whose bibliographies we read); 16 of them have a page here, 158 not yet.

- [An Empirical Survey of Data Augmentation for Limited Data Learning in NLP](https://md.research.elamur.ai/free/en/papers/chen-2023-empirical-survey-data-augmentation-limited/)
- [AugGPT: Leveraging ChatGPT for Text Data Augmentation](https://md.research.elamur.ai/free/en/papers/dai-2023-auggpt-leveraging-chatgpt-text-data/)
- [A Survey on Recent Approaches for Natural Language Processing in Low-Resource Scenarios](https://md.research.elamur.ai/free/en/papers/hedderich-2021-survey-recent-approaches-natural-language/)
- [Target-Aware Data Augmentation for Stance Detection](https://md.research.elamur.ai/free/en/papers/li-2021-target-aware-data-augmentation-stance/)
- [BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation](https://md.research.elamur.ai/free/en/papers/li-2022-blip-bootstrapping-language-image-pre/)
- [Synthetic Data Generation with Large Language Models for Text Classification: Potential and Limitations](https://md.research.elamur.ai/free/en/papers/li-2023-synthetic-data-generation-large-language/)
- [A Holistic Approach to Undesired Content Detection in the Real World](https://md.research.elamur.ai/free/en/papers/markov-2022-holistic-approach-undesired-content-detection/)
- [Recent Advances in Natural Language Processing via Large Pre-Trained Language Models: A Survey](https://md.research.elamur.ai/free/en/papers/min-2021-recent-advances-natural-language-processing/)
- [Pre-trained Models for Natural Language Processing: A Survey](https://md.research.elamur.ai/free/en/papers/qiu-2020-pre-trained-models-natural-language/)
- [What happens before and after: Multi-Event Commonsense in Event Coreference Resolution](https://md.research.elamur.ai/free/en/papers/ravi-2023-what-happens-before-after-multi/)
- [Generating Datasets with Pretrained Language Models](https://md.research.elamur.ai/free/en/papers/schick-2021-generating-datasets-pretrained-language-models/)
- [Symbolic Knowledge Distillation: from General Language Models to Commonsense Models](https://md.research.elamur.ai/free/en/papers/west-2022-symbolic-knowledge-distillation-general-language/)
- [Generative Data Augmentation for Commonsense Reasoning](https://md.research.elamur.ai/free/en/papers/yang-2020-generative-data-augmentation-commonsense-reasoning/)
- [ZeroGen: Efficient Zero-shot Learning via Dataset Generation](https://md.research.elamur.ai/free/en/papers/ye-2022-zerogen-efficient-zero-shot-learning/)
- [GPT3Mix: Leveraging Large-scale Language Models for Text Augmentation](https://md.research.elamur.ai/free/en/papers/yoo-2021-gpt3mix-leveraging-large-scale-language/)
- [Big Bird: Transformers for Longer Sequences](https://md.research.elamur.ai/free/en/papers/zaheer-2020-big-bird-transformers-longer-sequences/)

## Cite

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@misc{kumar2020data,
  title = {{Data Augmentation using Pre-trained Transformer Models}},
  author = {Kumar, Varun and Choudhary, Ashutosh and Cho, Eunah},
  year = {2020},
  doi = {10.48550/arxiv.2003.02245},
  eprint = {2003.02245},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2003.02245}
}
```

- [← previous DART: Open-Domain Structured Data Record to Text Generation](https://md.research.elamur.ai/free/en/papers/nan-2020-dart-open-domain-structured-data/)
- [next → Data-Driven Sentence Simplification: Survey and Benchmark](https://md.research.elamur.ai/free/en/papers/manchego-2020-data-driven-sentence-simplification-survey/)
- [↑ back to papers of 2020](https://md.research.elamur.ai/free/en/papers/2020/page/2/)

## References

30 references, 10 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Ateret Anaby-Tavor, Boaz Carmeli, Esther Goldbraich, Amir Kantor, George Kour, Segev Shlomov, Naama Tepper, and Naama Zwerdling. 2019. Not enough data? deep learning to the rescue! arXiv preprint arXiv:1911.03118. — [arXiv:1911.03118](https://arxiv.org/abs/1911.03118) [find](https://scholar.google.com/scholar?q=Ateret+Anaby%2DTavor%2C+Boaz+Carmeli%2C+Esther+Goldbraich%2C+Amir+Kantor%2C+George+Kour%2C+Segev+Shlomov%2C+Naama+Tepper%2C+and+Naama+Zwerdling%2E+2019%2E+Not+enough+data%3F+deep+learning+to+the+rescue%21+arXiv+preprint+arXi)
2. Alice Coucke, Alaa Saade, Adrien Ball, Théodore Bluche, Alexandre Caulier, David Leroy, Clément Doumouro, Thibault Gisselbrecht, Francesco Caltagirone, Thibaut Lavril, et al. 2018. Snips voice platform: an embedded spoken language understanding system for private-by-design voice interfaces. arXiv… [find](https://scholar.google.com/scholar?q=Alice+Coucke%2C+Alaa+Saade%2C+Adrien+Ball%2C+The%CC%81odore+Bluche%2C+Alexandre+Caulier%2C+David+Leroy%2C+Cle%CC%81ment+Doumouro%2C+Thibault+Gisselbrecht%2C+Francesco+Caltagirone%2C+Thibaut+Lavril%2C+et+al%2E+2018%2E+Snips+voice+platf) [our card](https://md.research.elamur.ai/free/en/papers/coucke-2018-snips-voice-platform-embedded-spoken/)
3. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805. [find](https://scholar.google.com/scholar?q=Jacob+Devlin%2C+Ming%2DWei+Chang%2C+Kenton+Lee%2C+and+Kristina+Toutanova%2E+2018%2E+Bert%3A+Pre%2Dtraining+of+deep+bidirectional+transformers+for+language+understanding%2E+arXiv+preprint+arXiv%3A1810%2E04805%2E)
4. Sergey Edunov, Myle Ott, M. Auli, and David Grangier. 2018. Understanding back-translation at scale. ArXiv, abs/1808.09381. [find](https://scholar.google.com/scholar?q=Sergey+Edunov%2C+Myle+Ott%2C+M%2E+Auli%2C+and+David+Grangier%2E+2018%2E+Understanding+back%2Dtranslation+at+scale%2E+ArXiv%2C+abs%2F1808%2E09381%2E)
5. Ari Holtzman, Jan Buys, Maxwell Forbes, and Yejin Choi. 2019. The curious case of neural text degeneration. [find](https://scholar.google.com/scholar?q=Ari+Holtzman%2C+Jan+Buys%2C+Maxwell+Forbes%2C+and+Yejin+Choi%2E+2019%2E+The+curious+case+of+neural+text+degeneration%2E)
6. Zhiting Hu, Bowen Tan, Russ R Salakhutdinov, Tom M Mitchell, and Eric P Xing. 2019. Learning data manipulation for augmentation and weighting. In Advances in Neural Information Processing Systems, pages 15738–15749. [find](https://scholar.google.com/scholar?q=Zhiting+Hu%2C+Bowen+Tan%2C+Russ+R+Salakhutdinov%2C+Tom+M+Mitchell%2C+and+Eric+P+Xing%2E+2019%2E+Learning+data+manipulation+for+augmentation+and+weighting%2E+In+Advances+in+Neural+Information+Processing+Systems%2C+pag)
7. Melvin Johnson, Mike Schuster, Quoc V Le, Maxim Krikun, Yonghui Wu, Zhifeng Chen, Nikhil Thorat, Fernanda Viégas, Martin Wattenberg, Greg Corrado, et al. 2017. Google’s multilingual neural machine translation system: Enabling zero-shot translation. Transactions of the Association for Computational… [find](https://scholar.google.com/scholar?q=Melvin+Johnson%2C+Mike+Schuster%2C+Quoc+V+Le%2C+Maxim+Krikun%2C+Yonghui+Wu%2C+Zhifeng+Chen%2C+Nikhil+Thorat%2C+Fernanda+Vie%CC%81gas%2C+Martin+Wattenberg%2C+Greg+Corrado%2C+et+al%2E+2017%2E+Google%E2%80%99s+multilingual+neural+machine+tr)
8. Nitish Shirish Keskar, Bryan McCann, Lav R Varshney, Caiming Xiong, and Richard Socher. 2019. Ctrl: A conditional transformer language model for controllable generation. arXiv preprint arXiv:1909.05858. [find](https://scholar.google.com/scholar?q=Nitish+Shirish+Keskar%2C+Bryan+McCann%2C+Lav+R+Varshney%2C+Caiming+Xiong%2C+and+Richard+Socher%2E+2019%2E+Ctrl%3A+A+conditional+transformer+language+model+for+controllable+generation%2E+arXiv+preprint+arXiv%3A1909%2E0585) [our card](https://md.research.elamur.ai/free/en/papers/keskar-2019-ctrl-conditional-transformer-language-model/)
9. Diederik P Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980. [find](https://scholar.google.com/scholar?q=Diederik+P+Kingma+and+Jimmy+Ba%2E+2014%2E+Adam%3A+A+method+for+stochastic+optimization%2E+arXiv+preprint+arXiv%3A1412%2E6980%2E)
10. Sosuke Kobayashi. 2018. Contextual augmentation: Data augmentation by words with paradigmatic relations. arXiv preprint arXiv:1805.06201. [find](https://scholar.google.com/scholar?q=Sosuke+Kobayashi%2E+2018%2E+Contextual+augmentation%3A+Data+augmentation+by+words+with+paradigmatic+relations%2E+arXiv+preprint+arXiv%3A1805%2E06201%2E) [our card](https://md.research.elamur.ai/free/en/papers/kobayashi-2018-contextual-augmentation-data-augmentation-words/)
11. Taku Kudo and John Richardson. 2018. Sentencepiece: A simple and language independent subword tokenizer and detokenizer for neural text processing. arXiv preprint arXiv:1808.06226. [find](https://scholar.google.com/scholar?q=Taku+Kudo+and+John+Richardson%2E+2018%2E+Sentencepiece%3A+A+simple+and+language+independent+subword+tokenizer+and+detokenizer+for+neural+text+processing%2E+arXiv+preprint+arXiv%3A1808%2E06226%2E) [our card](https://md.research.elamur.ai/free/en/papers/kudo-2018-sentencepiece-simple-language-independent-subword/)
12. Varun Kumar, Hadrien Glaude, Cyprien de Lichy, and William Campbell. 2019. A closer look at feature space data augmentation for few-shot intent classification. arXiv preprint arXiv:1910.04176. [find](https://scholar.google.com/scholar?q=Varun+Kumar%2C+Hadrien+Glaude%2C+Cyprien+de+Lichy%2C+and+William+Campbell%2E+2019%2E+A+closer+look+at+feature+space+data+augmentation+for+few%2Dshot+intent+classification%2E+arXiv+preprint+arXiv%3A1910%2E04176%2E)
13. Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Veselin Stoyanov, and Luke Zettlemoyer. 2019. Bart: Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension. arXiv preprint arXiv:1910.13461. [find](https://scholar.google.com/scholar?q=Mike+Lewis%2C+Yinhan+Liu%2C+Naman+Goyal%2C+Marjan+Ghazvininejad%2C+Abdelrahman+Mohamed%2C+Omer+Levy%2C+Veselin+Stoyanov%2C+and+Luke+Zettlemoyer%2E+2019%2E+Bart%3A+Denoising+sequence%2Dto%2Dsequence+pre%2Dtraining+for+natural+l) [our card](https://md.research.elamur.ai/free/en/papers/lewis-2020-bart-denoising-sequence-sequence-pre/)
14. Xin Li and Dan Roth. 2002. Learning question classifiers. In Proceedings of the 19th international conference on Computational linguistics-Volume 1, pages 1–7. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Xin+Li+and+Dan+Roth%2E+2002%2E+Learning+question+classifiers%2E+In+Proceedings+of+the+19th+international+conference+on+Computational+linguistics%2DVolume+1%2C+pages+1%E2%80%937%2E+Association+for+Computational+Linguistic) [our card](https://md.research.elamur.ai/free/en/papers/li-2002-learning-question-classifiers/)
15. Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692. [find](https://scholar.google.com/scholar?q=Yinhan+Liu%2C+Myle+Ott%2C+Naman+Goyal%2C+Jingfei+Du%2C+Mandar+Joshi%2C+Danqi+Chen%2C+Omer+Levy%2C+Mike+Lewis%2C+Luke+Zettlemoyer%2C+and+Veselin+Stoyanov%2E+2019%2E+Roberta%3A+A+robustly+optimized+bert+pretraining+approach%2E+a)
16. George A Miller. 1998. WordNet: An electronic lexical database. MIT press. [find](https://scholar.google.com/scholar?q=George+A+Miller%2E+1998%2E+WordNet%3A+An+electronic+lexical+database%2E+MIT+press%2E)
17. Nathan Ng, Kyra Yee, Alexei Baevski, Myle Ott, Michael Auli, and Sergey Edunov. 2019. Facebook fair’s wmt19 news translation task submission. In Proc. of WMT. [find](https://scholar.google.com/scholar?q=Nathan+Ng%2C+Kyra+Yee%2C+Alexei+Baevski%2C+Myle+Ott%2C+Michael+Auli%2C+and+Sergey+Edunov%2E+2019%2E+Facebook+fair%E2%80%99s+wmt19+news+translation+task+submission%2E+In+Proc%2E+of+WMT%2E) [our card](https://md.research.elamur.ai/free/en/papers/ng-2019-facebook-fair-s-wmt19-news/)
18. Myle Ott, Sergey Edunov, Alexei Baevski, Angela Fan, Sam Gross, Nathan Ng, David Grangier, and Michael Auli. 2019. fairseq: A fast, extensible toolkit for sequence modeling. In Proceedings of NAACL-HLT 2019: Demonstrations. [find](https://scholar.google.com/scholar?q=Myle+Ott%2C+Sergey+Edunov%2C+Alexei+Baevski%2C+Angela+Fan%2C+Sam+Gross%2C+Nathan+Ng%2C+David+Grangier%2C+and+Michael+Auli%2E+2019%2E+fairseq%3A+A+fast%2C+extensible+toolkit+for+sequence+modeling%2E+In+Proceedings+of+NAACL%2DHL) [our card](https://md.research.elamur.ai/free/en/papers/ott-2019-fairseq-fast-extensible-toolkit-sequence/)
19. Alec Radford, Karthik Narasimhan, Tim Salimans, Qizhe Xie, Zihang Dai, Eduard Hovy, Minh-Thang Lu-and Ilya Sutskever. 2018. Improving language ong, and Quoc V. Le. 2019. Unsupervised data aug-understanding by generative pre-training. URL mentation for consistency training. https://s3-us-west-2. amaz… [find](https://scholar.google.com/scholar?q=Alec+Radford%2C+Karthik+Narasimhan%2C+Tim+Salimans%2C+Qizhe+Xie%2C+Zihang+Dai%2C+Eduard+Hovy%2C+Minh%2DThang+Lu%2Dand+Ilya+Sutskever%2E+2018%2E+Improving+language+ong%2C+and+Quoc+V%2E+Le%2E+2019%2E+Unsupervised+data+aug%2Dundersta)
20. Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. 2019. Language models are unsupervised multitask learners. OpenAI Blog, 1(8). [find](https://scholar.google.com/scholar?q=Alec+Radford%2C+Jeffrey+Wu%2C+Rewon+Child%2C+David+Luan%2C+Dario+Amodei%2C+and+Ilya+Sutskever%2E+2019%2E+Language+models+are+unsupervised+multitask+learners%2E+OpenAI+Blog%2C+1%288%29%2E)
21. Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu. 2019. Exploring the limits of transfer learning with a unified text-to-text transformer. arXiv preprint arXiv:1910.10683. [find](https://scholar.google.com/scholar?q=Colin+Raffel%2C+Noam+Shazeer%2C+Adam+Roberts%2C+Katherine+Lee%2C+Sharan+Narang%2C+Michael+Matena%2C+Yanqi+Zhou%2C+Wei+Li%2C+and+Peter+J+Liu%2E+2019%2E+Exploring+the+limits+of+transfer+learning+with+a+unified+text%2Dto%2Dtext)
22. Melissa Roemmele, Andrew S Gordon, and Reid Swanson. 2017. Evaluating story generation systems using automated linguistic analyses. In SIGKDD 2017 Workshop on Machine Learning for Creativity. [find](https://scholar.google.com/scholar?q=Melissa+Roemmele%2C+Andrew+S+Gordon%2C+and+Reid+Swanson%2E+2017%2E+Evaluating+story+generation+systems+using+automated+linguistic+analyses%2E+In+SIGKDD+2017+Workshop+on+Machine+Learning+for+Creativity%2E)
23. Rico Sennrich, Barry Haddow, and Alexandra Birch. 2015a. Improving neural machine translation models with monolingual data. arXiv preprint arXiv:1511.06709. [find](https://scholar.google.com/scholar?q=Rico+Sennrich%2C+Barry+Haddow%2C+and+Alexandra+Birch%2E+2015a%2E+Improving+neural+machine+translation+models+with+monolingual+data%2E+arXiv+preprint+arXiv%3A1511%2E06709%2E)
24. Rico Sennrich, Barry Haddow, and Alexandra Birch. 2015b. Neural machine translation of rare words with subword units. arXiv preprint arXiv:1508.07909. [find](https://scholar.google.com/scholar?q=Rico+Sennrich%2C+Barry+Haddow%2C+and+Alexandra+Birch%2E+2015b%2E+Neural+machine+translation+of+rare+words+with+subword+units%2E+arXiv+preprint+arXiv%3A1508%2E07909%2E)
25. Sam Shleifer. 2019. Low resource text classification with ulmfit and backtranslation. [find](https://scholar.google.com/scholar?q=Sam+Shleifer%2E+2019%2E+Low+resource+text+classification+with+ulmfit+and+backtranslation%2E)
26. Richard Socher, Alex Perelygin, Jean Wu, Jason Chuang, Christopher D Manning, Andrew Ng, and Christopher Potts. 2013. Recursive deep models for semantic compositionality over a sentiment treebank. In Proceedings of the 2013 conference on empirical methods in natural language processing, pages 1631–1… [find](https://scholar.google.com/scholar?q=Richard+Socher%2C+Alex+Perelygin%2C+Jean+Wu%2C+Jason+Chuang%2C+Christopher+D+Manning%2C+Andrew+Ng%2C+and+Christopher+Potts%2E+2013%2E+Recursive+deep+models+for+semantic+compositionality+over+a+sentiment+treebank%2E+In+)
27. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Advances in neural information processing systems, pages 5998–6008. [find](https://scholar.google.com/scholar?q=Ashish+Vaswani%2C+Noam+Shazeer%2C+Niki+Parmar%2C+Jakob+Uszkoreit%2C+Llion+Jones%2C+Aidan+N+Gomez%2C+%C5%81ukasz+Kaiser%2C+and+Illia+Polosukhin%2E+2017%2E+Attention+is+all+you+need%2E+In+Advances+in+neural+information+processi)
28. Jason W Wei and Kai Zou. 2019. Eda: Easy data augmentation techniques for boosting performance on text classification tasks. arXiv preprint arXiv:1901.11196. [find](https://scholar.google.com/scholar?q=Jason+W+Wei+and+Kai+Zou%2E+2019%2E+Eda%3A+Easy+data+augmentation+techniques+for+boosting+performance+on+text+classification+tasks%2E+arXiv+preprint+arXiv%3A1901%2E11196%2E) [our card](https://md.research.elamur.ai/free/en/papers/wei-2019-eda-easy-data-augmentation-techniques/)
29. Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pier-ric Cistac, Tim Rault, R’emi Louf, Morgan Funtowicz, and Jamie Brew. 2019. Huggingface’s transformers: State-of-the-art natural language processing. ArXiv, abs/1910.03771. [find](https://scholar.google.com/scholar?q=Thomas+Wolf%2C+Lysandre+Debut%2C+Victor+Sanh%2C+Julien+Chaumond%2C+Clement+Delangue%2C+Anthony+Moi%2C+Pier%2Dric+Cistac%2C+Tim+Rault%2C+R%E2%80%99emi+Louf%2C+Morgan+Funtowicz%2C+and+Jamie+Brew%2E+2019%2E+Huggingface%E2%80%99s+transformers%3A+St) [our card](https://md.research.elamur.ai/free/en/papers/wolf-2020-transformers-state-art-natural-language/)
30. Xing Wu, Shangwen Lv, Liangjun Zang, Jizhong Han, and Songlin Hu. 2019. Conditional bert contextual augmentation. In International Conference on Computational Science, pages 84–95. Springer. [find](https://scholar.google.com/scholar?q=Xing+Wu%2C+Shangwen+Lv%2C+Liangjun+Zang%2C+Jizhong+Han%2C+and+Songlin+Hu%2E+2019%2E+Conditional+bert+contextual+augmentation%2E+In+International+Conference+on+Computational+Science%2C+pages+84%E2%80%9395%2E+Springer%2E)