--- title: "Microsoft — works of 2021: 16 papers, the most cited first" description: "16 works of Microsoft from 2021 in our library, the most cited first. Among them: DeBERTaV3: Improving DeBERTa using ELECTRA-Style…" canonical: "https://research.elamur.ai/free/en/companies/microsoft/year/2021/" --- [Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/companies/microsoft/year/2021/) [home](https://md.research.elamur.ai/free/en/) / [institutions](https://md.research.elamur.ai/free/en/orgs/) / [companies](https://md.research.elamur.ai/free/en/companies/) / [Microsoft](https://md.research.elamur.ai/free/en/companies/microsoft/) / 2021 # Works of Microsoft from 2021 16 works · the most cited first · [← Microsoft](https://md.research.elamur.ai/free/en/companies/microsoft/) 1. [**DeBERTaV3: Improving DeBERTa using ELECTRA-Style Pre-Training with Gradient-Disentangled Embedding Sharing arXiv:2111.09543**Pengcheng He, Jianfeng Gao, Weizhu Chen*2021 · 1,316 citations*](https://md.research.elamur.ai/free/en/papers/he-2021-debertav3-improving-deberta-using-electra/) 2. [**Are NLP Models really able to Solve Simple Math Word Problems?**Arkil Patel, Satwik Bhattamishra, Navin Goyal*2021 · 883 citations*](https://md.research.elamur.ai/free/en/papers/patel-2021-are-nlp-models-really-able/) 3. **[LayoutLMv2: Multi-modal Pre-training for Visually-rich Document Understanding](https://md.research.elamur.ai/free/en/papers/xu-2021-layoutlmv2-multi-modal-pre-training/) [Annual Meeting of the Association for Computational Linguistics (ACL)](https://md.research.elamur.ai/free/en/venues/acl/)**Yang Xu, Yiheng Xu, Tengchao Lv, et al.*2021 · 623 citations* 4. **[QMSum: A New Benchmark for Query-based Multi-domain Meeting Summarization](https://md.research.elamur.ai/free/en/papers/zhong-2021-qmsum-new-benchmark-query-based/) [Conference of the North American Chapter of the Association for Computational Linguistics (NAACL)](https://md.research.elamur.ai/free/en/venues/naacl/)**Ming Zhong, Da Yin, Tao Yu, et al.*2021 · 400 citations* 5. [**GraphCodeBERT: Pre-training Code Representations with Data Flow arXiv:2009.08366**Daya Guo, Shuo Ren, Shuai Lu, et al.*2021 · 331 citations*](https://md.research.elamur.ai/free/en/papers/guo-2021-graphcodebert-pre-training-code-representations/) 6. [**ZeRO-Offload: Democratizing Billion-Scale Model Training arXiv:2101.06840**Jie Ren, Samyam Rajbhandari, Reza Yazdani Aminabadi, et al.*2021 · 303 citations*](https://md.research.elamur.ai/free/en/papers/ren-2021-zero-offload-democratizing-billion-scale/) 7. **[Want To Reduce Labeling Cost? GPT-3 Can Help](https://md.research.elamur.ai/free/en/papers/wang-2021-want-reduce-labeling-cost-gpt/) [Findings of the Association for Computational Linguistics](https://md.research.elamur.ai/free/en/venues/findings/)**Shuohang Wang, Yang Liu, Yichong Xu, et al.*2021 · 298 citations* 8. **[Polyjuice: Generating Counterfactuals for Explaining, Evaluating, and Improving Models](https://md.research.elamur.ai/free/en/papers/wu-2021-polyjuice-generating-counterfactuals-explaining-evaluating/) [Annual Meeting of the Association for Computational Linguistics (ACL)](https://md.research.elamur.ai/free/en/venues/acl/)**Tongshuang Wu, Marco Tulio Ribeiro, Jeffrey Heer, et al.*2021 · 272 citations* 9. [**TAPEX: Table Pre-training via Learning a Neural SQL Executor arXiv:2107.07653**Qian Liu, Bei Chen, Jiaqi Guo, et al.*2021 · 254 citations*](https://md.research.elamur.ai/free/en/papers/liu-2021-tapex-table-pre-training-learning/) 10. **[The Curious Case of Hallucinations in Neural Machine Translation](https://md.research.elamur.ai/free/en/papers/raunak-2021-curious-case-hallucinations-neural-machine/) [Conference of the North American Chapter of the Association for Computational Linguistics (NAACL)](https://md.research.elamur.ai/free/en/venues/naacl/)**Vikas Raunak, Arul Menezes, Marcin Junczys-Dowmunt*2021 · 241 citations* 11. **[Challenges in Detoxifying Language Models](https://md.research.elamur.ai/free/en/papers/welbl-2021-challenges-detoxifying-language-models/) [Findings of the Association for Computational Linguistics](https://md.research.elamur.ai/free/en/venues/findings/)**Johannes Welbl, Amelia Glaese, Jonathan Uesato, et al.*2021 · 207 citations* 12. [**COCO-LM: Correcting and Contrasting Text Sequences for Language Model Pretraining arXiv:2102.08473**Yu Meng, Chenyan Xiong, Payal Bajaj, et al.*2021 · 196 citations*](https://md.research.elamur.ai/free/en/papers/meng-2021-coco-lm-correcting-contrasting-text/) 13. **[Enhancing Factual Consistency of Abstractive Summarization](https://md.research.elamur.ai/free/en/papers/zhu-2021-enhancing-factual-consistency-abstractive-summarization/) [Conference of the North American Chapter of the Association for Computational Linguistics (NAACL)](https://md.research.elamur.ai/free/en/venues/naacl/)**Chenguang Zhu, William Hinthorn, Ruochen Xu, et al.*2021 · 188 citations* 14. **[MediaSum: A Large-scale Media Interview Dataset for Dialogue Summarization](https://md.research.elamur.ai/free/en/papers/zhu-2021-mediasum-large-scale-media-interview/) [Conference of the North American Chapter of the Association for Computational Linguistics (NAACL)](https://md.research.elamur.ai/free/en/venues/naacl/)**Chenguang Zhu, Yang Liu, Jie Mei, et al.*2021 · 180 citations* 15. **[Structure-Grounded Pretraining for Text-to-SQL](https://md.research.elamur.ai/free/en/papers/deng-2021-structure-grounded-pretraining-text-sql/) [Conference of the North American Chapter of the Association for Computational Linguistics (NAACL)](https://md.research.elamur.ai/free/en/venues/naacl/)**Xiang Deng, Ahmed Hassan Awadallah, Christopher Meek, et al.*2021 · 166 citations* 16. **[Joint Universal Syntactic and Semantic Parsing](https://md.research.elamur.ai/free/en/papers/eskin-2021-joint-universal-syntactic-semantic-parsing/) [Transactions of the Association for Computational Linguistics (TACL)](https://md.research.elamur.ai/free/en/venues/tacl/)**Elias Stengel-Eskin, Kenton Murray, Sheng Zhang, et al.*2021 · 15 citations* - [← back to](https://md.research.elamur.ai/free/en/companies/microsoft/) — Microsoft