---
title: "Workshop on Representation Learning for NLP 2021 — accepted"
description: "Workshop on Representation Learning for NLP 2021 — accepted papers: 2 papers in the Elamur Research library (Online), with our…"
canonical: "https://research.elamur.ai/free/en/venues/repl4nlp/2021/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/venues/repl4nlp/2021/)

[home](https://md.research.elamur.ai/free/en/) / [venues](https://md.research.elamur.ai/free/en/venues/) / [Workshop on Representation Learning for…](https://md.research.elamur.ai/free/en/venues/repl4nlp/) / 2021

# Workshop on Representation Learning for NLP 2021 — accepted papers

Workshop on Representation Learning for NLP 2021 is the 2021 edition of the Workshop on Representation Learning for NLP, held in Online, August 2021 (publisher: Association for Computational Linguistics). The ACL Anthology lists 32 papers of it; our library holds 2 with our summaries. Below: the most cited papers, every accepted paper, and ours with their topics, authors and institutions.

[**32** papers in the proceedings](#all) · [**2** works in our library](https://md.research.elamur.ai/free/en/venues/repl4nlp/2021/#works) · [**290** citations of these works](https://md.research.elamur.ai/free/en/venues/repl4nlp/2021/#works)**Online** where it was held**August 2021** when

## Most cited papers

By their OpenAlex citations; a paper with our summary links to its page here, the others to the original.

1. **[In-Batch Negatives for Knowledge Distillation with Tightly-Coupled Teachers for Dense Retrieval](https://aclanthology.org/2021.repl4nlp-1.17/)**Sheng-Chieh Lin, Jheng-Hong Yang, Jimmy Lin*122 citations*
2. **[Larger-Scale Transformers for Multilingual Masked Language Modeling](https://md.research.elamur.ai/free/en/papers/goyal-2021-larger-scale-transformers-multilingual-masked/)**Naman Goyal, Jingfei Du, Myle Ott et al.*83 citations · our summary*
3. **[Scaling Deep Contrastive Learning Batch Size under Memory Limited Setup](https://md.research.elamur.ai/free/en/papers/gao-2021-scaling-deep-contrastive-learning-batch/)**Luyu Gao, Yunyi Zhang, Jiawei Han et al.*64 citations · our summary*
4. **[Entity and Evidence Guided Document-Level Relation Extraction](https://aclanthology.org/2021.repl4nlp-1.30/)**Kevin Huang, Peng Qi, Guangtao Wang et al.*39 citations*
5. **[Deriving Contextualised Semantic Features from BERT (and Other Transformer Model) Embeddings](https://aclanthology.org/2021.repl4nlp-1.26/)**Jacob Turton, Robert Elliott Smith, David Vinson*29 citations*
6. **[Preserving Cross-Linguality of Pre-trained Models via Continual Learning](https://aclanthology.org/2021.repl4nlp-1.8/)**Zihan Liu, Genta Indra Winata, Andrea Madotto et al.*14 citations*
7. **[Inductively Representing Out-of-Knowledge-Graph Entities by Optimal Estimation Under Translational Assumptions](https://aclanthology.org/2021.repl4nlp-1.10/)**Damai Dai, Hua Zheng, Fuli Luo et al.*13 citations*
8. **[Syntactic Perturbations Reveal Representational Correlates of Hierarchical Phrase Structure in Pretrained Language Models](https://aclanthology.org/2021.repl4nlp-1.27/)**Matteo Alleman, Jonathan Mamou, Miguel A Del Rio et al.*11 citations*
9. **[NPVec1: Word Embeddings for Nepali - Construction and Evaluation](https://aclanthology.org/2021.repl4nlp-1.18/)**Pravesh Koirala, Nobal B. Niraula*10 citations*
10. **[Probing Multilingual Language Models for Discourse](https://aclanthology.org/2021.repl4nlp-1.2/)**Murathan Kurfalı, Robert Östling*10 citations*

## All accepted papers (32)

The most cited first; 2 with our summary link to their page here, the others to the original.

1. **[In-Batch Negatives for Knowledge Distillation with Tightly-Coupled Teachers for Dense Retrieval](https://aclanthology.org/2021.repl4nlp-1.17/)**Sheng-Chieh Lin, Jheng-Hong Yang, Jimmy Lin*122 citations*
2. **[Larger-Scale Transformers for Multilingual Masked Language Modeling](https://md.research.elamur.ai/free/en/papers/goyal-2021-larger-scale-transformers-multilingual-masked/)**Naman Goyal, Jingfei Du, Myle Ott et al.*83 citations · our summary*
3. **[Scaling Deep Contrastive Learning Batch Size under Memory Limited Setup](https://md.research.elamur.ai/free/en/papers/gao-2021-scaling-deep-contrastive-learning-batch/)**Luyu Gao, Yunyi Zhang, Jiawei Han et al.*64 citations · our summary*
4. **[Entity and Evidence Guided Document-Level Relation Extraction](https://aclanthology.org/2021.repl4nlp-1.30/)**Kevin Huang, Peng Qi, Guangtao Wang et al.*39 citations*
5. **[Deriving Contextualised Semantic Features from BERT (and Other Transformer Model) Embeddings](https://aclanthology.org/2021.repl4nlp-1.26/)**Jacob Turton, Robert Elliott Smith, David Vinson*29 citations*
6. **[Preserving Cross-Linguality of Pre-trained Models via Continual Learning](https://aclanthology.org/2021.repl4nlp-1.8/)**Zihan Liu, Genta Indra Winata, Andrea Madotto et al.*14 citations*
7. **[Inductively Representing Out-of-Knowledge-Graph Entities by Optimal Estimation Under Translational Assumptions](https://aclanthology.org/2021.repl4nlp-1.10/)**Damai Dai, Hua Zheng, Fuli Luo et al.*13 citations*
8. **[Syntactic Perturbations Reveal Representational Correlates of Hierarchical Phrase Structure in Pretrained Language Models](https://aclanthology.org/2021.repl4nlp-1.27/)**Matteo Alleman, Jonathan Mamou, Miguel A Del Rio et al.*11 citations*
9. **[NPVec1: Word Embeddings for Nepali - Construction and Evaluation](https://aclanthology.org/2021.repl4nlp-1.18/)**Pravesh Koirala, Nobal B. Niraula*10 citations*
10. **[Probing Multilingual Language Models for Discourse](https://aclanthology.org/2021.repl4nlp-1.2/)**Murathan Kurfalı, Robert Östling*10 citations*
11. **[Revisiting Pretraining with Adapters](https://aclanthology.org/2021.repl4nlp-1.11/)**Seungwon Kim, Alex Shum, Nathan Susanj et al.*9 citations*
12. **[Structure-aware Sentence Encoder in Bert-Based Siamese Network](https://aclanthology.org/2021.repl4nlp-1.7/)**Qiwei Peng, David Weir, Julie Weeds*7 citations*
13. **[X2Parser: Cross-Lingual and Cross-Domain Framework for Task-Oriented Compositional Semantic Parsing](https://aclanthology.org/2021.repl4nlp-1.13/)**Zihan Liu, Genta Indra Winata, Peng Xu et al.*7 citations*
14. **[Comprehension Based Question Answering using Bloom's Taxonomy](https://aclanthology.org/2021.repl4nlp-1.3/)**Pritish Sahu, Michael Cogswell, Ajay Divakaran et al.*6 citations*
15. **[An Overview of Uncertainty Calibration for Text Classification and the Role of Distillation](https://aclanthology.org/2021.repl4nlp-1.29/)**Han Guo, Ramakanth Pasunuru, Mohit Bansal*5 citations*
16. **[Direction is what you need: Improving Word Embedding Compression in Large Language Models](https://aclanthology.org/2021.repl4nlp-1.32/)**Klaudia Bałazy, Mohammadreza Banaei, Rémi Lebret et al.*5 citations*
17. **[Zero-shot Sequence Labeling for Transformer-based Sentence Classifiers](https://aclanthology.org/2021.repl4nlp-1.20/)**Kamil Bujel, Helen Yannakoudakis, Marek Rei*5 citations*
18. **[Knodle: Modular Weakly Supervised Learning with PyTorch](https://aclanthology.org/2021.repl4nlp-1.12/)**Anastasiia Sedova, Andreas Stephan, Marina Speranskaya et al.*4 citations*
19. **[Knowledge Informed Semantic Parsing for Conversational Question Answering](https://aclanthology.org/2021.repl4nlp-1.24/)**Raghuveer Thirukovalluru, Mukund Sridhar, Dung Thai et al.*4 citations*
20. **[Improving Cross-lingual Text Classification with Zero-shot Instance-Weighting](https://aclanthology.org/2021.repl4nlp-1.1/)**Irene Li, Prithviraj Sen, Huaiyu Zhu et al.*3 citations*
21. **[Learning Sparse Sentence Encoding without Supervision: An Exploration of Sparsity in Variational Autoencoders](https://aclanthology.org/2021.repl4nlp-1.5/)**Victor Prokhorov, Yingzhen Li, Ehsan Shareghi et al.*3 citations*
22. **[Unsupervised Representation Disentanglement of Text: An Evaluation on Synthetic Datasets](https://aclanthology.org/2021.repl4nlp-1.14/)**Lan Zhang, Victor Prokhorov, Ehsan Shareghi*3 citations*
23. **[Box-To-Box Transformations for Modeling Joint Hierarchies](https://aclanthology.org/2021.repl4nlp-1.28/)**Shib Sankar Dasgupta, Xiang Lorraine Li, Michael Boratko et al.*2 citations*
24. **[Deriving Word Vectors from Contextualized Language Models using Topic-Aware Mention Selection](https://aclanthology.org/2021.repl4nlp-1.19/)**Yixiao Wang, Zied Bouraoui, Luis Espinosa Anke et al.*2 citations*
25. **[Predicting the Success of Domain Adaptation in Text Similarity](https://aclanthology.org/2021.repl4nlp-1.21/)**Nick Pogrebnyakov, Shohreh Shaghaghian*2 citations*

[See all 32 papers in Workshop on Representation Learning for NLP 2021 — accepted papers →](https://md.research.elamur.ai/free/en/venues/repl4nlp/2021/papers/)

## Papers with our summary in our library (2)

The most cited first (OpenAlex citations plus our library's).

1. **[Larger-Scale Transformers for Multilingual Masked Language Modeling](https://md.research.elamur.ai/free/en/papers/goyal-2021-larger-scale-transformers-multilingual-masked/) [Workshop on Representation Learning for NLP](https://md.research.elamur.ai/free/en/venues/repl4nlp/)**Naman Goyal, Jingfei Du, Myle Ott, et al.*2021 · 160 citations*
2. **[Scaling Deep Contrastive Learning Batch Size under Memory Limited Setup](https://md.research.elamur.ai/free/en/papers/gao-2021-scaling-deep-contrastive-learning-batch/) [Workshop on Representation Learning for NLP](https://md.research.elamur.ai/free/en/venues/repl4nlp/)**Luyu Gao, Yunyi Zhang, Jiawei Han, et al.*2021 · 130 citations*

## Top topics

Topics of its works in our library, with the number of works on each.

[Language models & pretraining (1)](https://md.research.elamur.ai/free/en/topics/language-models/) · [Multilingual & cross-lingual NLP (1)](https://md.research.elamur.ai/free/en/topics/multilingual-nlp/)

## Fields

[Arts and Humanities (2)](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · [Computer Science (2)](https://md.research.elamur.ai/free/en/shelves/computer-science/)

## Top authors

Authors with a page who published here most often: their works here, and the citations of those works.

[🧑‍🔬**Alexis Conneau**1 work here · 160 citations](https://md.research.elamur.ai/free/en/authors/alexis-conneau/) · [🧑‍🔬**Jingfei Du**1 work here · 160 citations](https://md.research.elamur.ai/free/en/authors/jingfei-du/) · [🧑‍🔬**Myle Ott**1 work here · 160 citations](https://md.research.elamur.ai/free/en/authors/myle-ott/) · [🧑‍🔬**Naman Goyal**1 work here · 160 citations](https://md.research.elamur.ai/free/en/authors/naman-goyal/) · [🧑‍🔬**Jamie Callan**1 work here · 130 citations](https://md.research.elamur.ai/free/en/authors/jamie-callan/) · [🧑‍🔬**Jiawei Han**1 work here · 130 citations](https://md.research.elamur.ai/free/en/authors/jiawei-han/) · [🧑‍🔬**Luyu Gao**1 work here · 130 citations](https://md.research.elamur.ai/free/en/authors/luyu-gao/) · [🧑‍🔬**Yunyi Zhang**1 work here · 130 citations](https://md.research.elamur.ai/free/en/authors/yunyi-zhang/)

## Top institutions

Institutions on the bylines of its works here, by the number of works.

[🎓**University of Illinois Urbana-Champaign** Urbana · United States 1 work here](https://md.research.elamur.ai/free/en/universities/university-of-illinois-urbana-champaign/) · [🎓**Carnegie Mellon University** Pittsburgh · United States 1 work here](https://md.research.elamur.ai/free/en/universities/carnegie-mellon-university/)

Place and dates from the event's site or the ACL Anthology; works and citations from OpenAlex and our library.

- [← previous Workshop on Representation Learning for NLP 2020](https://md.research.elamur.ai/free/en/venues/repl4nlp/2020/)
- [next → Workshop on Representation Learning for NLP 2025](https://md.research.elamur.ai/free/en/venues/repl4nlp/2025/)
- [↑ back to Workshop on Representation Learning for NLP](https://md.research.elamur.ai/free/en/venues/repl4nlp/)