--- title: "Who cites Cloze-driven Pretraining of Self-attention Networks" description: "74 papers of the Elamur Research library cite «Cloze-driven Pretraining of Self-attention Networks», found in their bibliographies; the most cited first." canonical: "https://research.elamur.ai/free/en/papers/baevski-2019-cloze-driven-pretraining-self-attention/cited-by/" --- [Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/baevski-2019-cloze-driven-pretraining-self-attention/cited-by/) [home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [Cloze-driven Pretraining of Self-attention…](https://md.research.elamur.ai/free/en/papers/baevski-2019-cloze-driven-pretraining-self-attention/) / cited by # Who cites «Cloze-driven Pretraining of Self-attention Networks» 74 papers of our library cite it (of the 149,439 whose bibliographies we read) · 13 with a page here, the most cited first · [← back to «Cloze-driven Pretraining of Self-attention Networks»](https://md.research.elamur.ai/free/en/papers/baevski-2019-cloze-driven-pretraining-self-attention/) 1. [**Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer arXiv:1910.10683**Colin Raffel, Noam Shazeer, Adam Roberts, et al.*2019 · 11,325 citations*](https://md.research.elamur.ai/free/en/papers/raffel-2019-exploring-limits-transfer-learning-unified/) 2. [**RoBERTa: A Robustly Optimized BERT Pretraining Approach arXiv:1907.11692**Yinhan Liu, Myle Ott, Naman Goyal, et al.*2019 · 9,837 citations*](https://md.research.elamur.ai/free/en/papers/liu-2019-roberta-robustly-optimized-bert-pretraining/) 3. [**Pre-trained Models for Natural Language Processing: A Survey arXiv:2003.08271 · SCIENCE CHINA Technological Sciences,2020, 63, 1872-1897**Xipeng Qiu, Tianxiang Sun, Yige Xu, et al.*2020 · 1,759 citations*](https://md.research.elamur.ai/free/en/papers/qiu-2020-pre-trained-models-natural-language/) 4. [**Multitask Prompted Training Enables Zero-Shot Task Generalization arXiv:2110.08207**Victor Sanh, Albert Webson, Colin Raffel, et al.*2021 · 1,612 citations*](https://md.research.elamur.ai/free/en/papers/sanh-2021-multitask-prompted-training-enables-zero/) 5. [**Unified Language Model Pre-training for Natural Language Understanding and Generation arXiv:1905.03197**Li Dong, Nan Yang, Wenhui Wang, et al.*2019 · 1,422 citations*](https://md.research.elamur.ai/free/en/papers/dong-2019-unified-language-model-pre-training/) 6. **[XLS-R: Self-supervised Cross-lingual Speech Representation Learning at Scale](https://md.research.elamur.ai/free/en/papers/babu-2021-xls-r-self-supervised-cross/) [Interspeech](https://md.research.elamur.ai/free/en/venues/interspeech-usb/)**Arun Babu, Changhan Wang, Andros Tjandra, et al.*2021 · 807 citations* 7. **[CCNet: Extracting High Quality Monolingual Datasets from Web Crawl Data](https://md.research.elamur.ai/free/en/papers/wenzek-2020-ccnet-extracting-high-quality-monolingual/) [Language Resources and Evaluation Conference (LREC)](https://md.research.elamur.ai/free/en/venues/lrec/)**Guillaume Wenzek, Marie-Anne Lachaux, Alexis Conneau, et al.*2020 · 666 citations* 8. **[Are All Languages Created Equal in Multilingual BERT?](https://md.research.elamur.ai/free/en/papers/wu-2020-are-all-languages-created-equal/) [Workshop on Representation Learning for NLP](https://md.research.elamur.ai/free/en/venues/repl4nlp/)**Shijie Wu, Mark Dredze*2020 · 460 citations* 9. **[Visualizing and Understanding the Effectiveness of BERT](https://md.research.elamur.ai/free/en/papers/hao-2019-visualizing-understanding-effectiveness-bert/) [Conference on Empirical Methods in Natural Language Processing (EMNLP)](https://md.research.elamur.ai/free/en/venues/emnlp/)**Yaru Hao, Li Dong, Furu Wei, et al.*2019 · 222 citations* 10. **[Automated Concatenation of Embeddings for Structured Prediction](https://md.research.elamur.ai/free/en/papers/wang-2021-automated-concatenation-embeddings-structured-prediction/) [Annual Meeting of the Association for Computational Linguistics (ACL)](https://md.research.elamur.ai/free/en/venues/acl/)**Xinyu Wang, Yong Jiang, Nguyen Bach, et al.*2021 · 206 citations* 11. **[Improving Named Entity Recognition by External Context Retrieving and Cooperative Learning](https://md.research.elamur.ai/free/en/papers/wang-2021-improving-named-entity-recognition-external/) [Annual Meeting of the Association for Computational Linguistics (ACL)](https://md.research.elamur.ai/free/en/venues/acl/)**Xinyu Wang, Yong Jiang, Nguyen Bach, et al.*2021 · 206 citations* 12. [**NeoBERT: A Next-Generation BERT arXiv:2502.19587**Lola Le Breton, Quentin Fournier, Mariam El Mezouar, et al.*2025 · 20 citations*](https://md.research.elamur.ai/free/en/papers/breton-2025-neobert-next-generation-bert/) 13. [**DeformAr: Rethinking NER Evaluation through Component Analysis and Visual Analytics arXiv:2512.00938**Ahmed Mustafa Younes*2025*](https://md.research.elamur.ai/free/en/papers/younes-2025-deformar-rethinking-ner-evaluation-through/) 61 more papers of our library cite it; they have no page here yet. - [← back to](https://md.research.elamur.ai/free/en/papers/baevski-2019-cloze-driven-pretraining-self-attention/) — Cloze-driven Pretraining of Self-attention Networks