---
title: "ExT5: Towards Extreme Multi-Task Scaling for Transfer Learning"
description: "The authors build ExMix, a set of 107 English supervised tasks with about 18M labeled examples… 154 references, cited by 106 papers here."
authors: ["Vamsi Aribandi", "Yi Tay", "Tal Schuster", "Jinfeng Rao", "Huaixiu Steven Zheng", "Sanket Vaibhav Mehta", "Honglei Zhuang", "Vinh Q. Tran", "Dara Bahri", "Jianmo Ni", "Jai Gupta", "Kai Hui", "Sebastian Ruder", "Donald Metzler"]
year: 2021
venue: "arXiv:2111.10952"
source: "https://arxiv.org/abs/2111.10952"
doi: "10.48550/arxiv.2111.10952"
arxiv: "2111.10952"
licence: "CC BY 4.0 (arXiv)"
licence_url: "http://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/aribandi-2021-ext5-towards-extreme-multi-task/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/aribandi-2021-ext5-towards-extreme-multi-task/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2021](https://md.research.elamur.ai/free/en/papers/2021/page/4/) / ExT5: Towards Extreme Multi-Task Scaling for Transfer Learning

# ExT5: Towards Extreme Multi-Task Scaling for Transfer Learning

**Authors:** Vamsi Aribandi, [Yi Tay](https://md.research.elamur.ai/free/en/authors/yi-tay/), [Tal Schuster](https://md.research.elamur.ai/free/en/authors/tal-schuster/), [Jinfeng Rao](https://md.research.elamur.ai/free/en/authors/jinfeng-rao/), [Huaixiu Steven Zheng](https://md.research.elamur.ai/free/en/authors/huaixiu-steven-zheng/), [Sanket Vaibhav Mehta](https://md.research.elamur.ai/free/en/authors/sanket-vaibhav-mehta/), [Honglei Zhuang](https://md.research.elamur.ai/free/en/authors/honglei-zhuang/), [Vinh Q. Tran](https://md.research.elamur.ai/free/en/authors/vinh-q-tran/), [Dara Bahri](https://md.research.elamur.ai/free/en/authors/dara-bahri/), [Jianmo Ni](https://md.research.elamur.ai/free/en/authors/jianmo-ni/), [Jai Gupta](https://md.research.elamur.ai/free/en/authors/jai-gupta/), [Kai Hui](https://md.research.elamur.ai/free/en/authors/kai-hui/), [Sebastian Ruder](https://md.research.elamur.ai/free/en/authors/sebastian-ruder/) and [Donald Metzler](https://md.research.elamur.ai/free/en/authors/donald-metzler/) · **Year:** 2021 · **Venue:** arXiv:2111.10952 · **Licence:** [CC BY 4.0 (arXiv)](http://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://arxiv.org/abs/2111.10952) · [PDF](https://arxiv.org/pdf/2111.10952) · [DOI 10.48550/arxiv.2111.10952](https://doi.org/10.48550/arxiv.2111.10952) · [arXiv 2111.10952](https://arxiv.org/abs/2111.10952) · [OpenAlex](https://openalex.org/W3217187998)

**Citations:** 8 citations (OpenAlex, 2026-10-14) · [106](https://md.research.elamur.ai/free/en/papers/aribandi-2021-ext5-towards-extreme-multi-task/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
Multi-task learning helps transfer in NLP, but no one had systematically tested what happens when the number of supervised tasks used during pre-training is pushed very high. Hand-picking helpful task subsets is also expensive, because negative transfer is common. The authors build ExMix, a set of 107 English supervised tasks with about 18M labeled examples, all converted to text-to-text format. The tasks span classification, NLI, reading comprehension, closed-book QA, commonsense reasoning, semantic parsing, dialogue and summarization. Co-training experiments between task families show that transfer is uneven. Choosing a mixture from fine-tuning results does not reliably beat simply using every task. The authors then pre-train ExT5, a T5 model trained jointly on C4 span denoising and ExMix. ExT5 outperforms competitive T5 baselines across closed-book QA, Rainbow, GEM, SuperGLUE and several tasks not included in ExMix. It also reaches strong scores in fewer pre-training steps. The work suggests that broad task coverage is a cheaper and more robust alternative to searching for an ideal task set.

## Key points
- **ExMix size and sampling:** ExMix has 107 tasks and roughly 18M examples, about twice as many tasks as the largest earlier study (Aghajanyan et al., 2021). Datasets are sampled in proportion to their size, with each capped at 3×10⁵ effective examples.
- **Transfer study setup:** 8 task families, each represented by 3 datasets, produced 34 co-trained models. All were trained on the T5.1.1 BASE checkpoint for 200k steps with batch size 128.
- **SuperGLUE average scores:**
  - vanilla T5: 76.1
  - manually curated "best-effort" mixture: 76.4
  - 55 random tasks: 77.0
  - full ExMix in multi-task pre-training (ExT5): 79.9
  - ExMix used only for pre-finetuning: 78.1
- **More tasks and mixing ratio:** At batch size 512, scores generally rise with task count (30 tasks: 75.8; 80 tasks: 77.6; 107 tasks: 79.9). A C4-to-ExMix mixing ratio of R=2 worked best.
- **Sample efficiency:** After 20k pre-training steps, ExT5 scores 75.8 on SuperGLUE versus 73.8 for T5.1.1. After 200k steps, the gap is 85.3 versus 83.8.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Google](https://md.research.elamur.ai/free/en/companies/google/) — Company, Mountain View, United States · 492 works · 440,463 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 18,614 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 17,965 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 17,450 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 17,296 works

🧭 Topics (1): [🧠 Language models & pretraining](https://md.research.elamur.ai/free/en/topics/language-models/) · 3,894 works

## Cite

Cite as: Vamsi Aribandi, Yi Tay, Tal Schuster, Jinfeng Rao, Huaixiu Steven Zheng, Sanket Vaibhav Mehta, Honglei Zhuang, Vinh Q. Tran, Dara Bahri, Jianmo Ni, Jai Gupta, Kai Hui, Sebastian Ruder and Donald Metzler. 2021. ExT5: Towards Extreme Multi-Task Scaling for Transfer Learning. arXiv:2111.10952.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@misc{aribandi2021ext5,
  title = {{ExT5: Towards Extreme Multi-Task Scaling for Transfer Learning}},
  author = {Vamsi Aribandi, Yi Tay, Tal Schuster, Jinfeng Rao, Huaixiu Steven Zheng, Sanket Vaibhav Mehta, Honglei Zhuang, Vinh Q. Tran, Dara Bahri, Jianmo Ni, Jai Gupta, Kai Hui, Sebastian Ruder and Donald Metzler},
  year = {2021},
  doi = {10.48550/arxiv.2111.10952},
  eprint = {2111.10952},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2111.10952}
}
```

- [← previous Aspect-Controllable Opinion Summarization](https://md.research.elamur.ai/free/en/papers/amplayo-2021-aspect-controllable-opinion-summarization/)
- [next → SIMMC 2.0: A Task-oriented Dialog Dataset for Immersive Multimodal Conversations](https://md.research.elamur.ai/free/en/papers/kottur-2021-simmc-2-0-task-oriented/)
- [↑ back to papers of 2021](https://md.research.elamur.ai/free/en/papers/2021/page/4/)

## References

154 references, 87 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Amazon reviews dataset. readme.html. — [Muppet: Massive Multi-task Representations with Pre-Finetuning](https://md.research.elamur.ai/free/en/papers/aghajanyan-2021-muppet-massive-multi-task-representations/) [https://s3.amazonaws.com/amazon-reviews-pds/Armen](https://s3.amazonaws.com/amazon-reviews-pds/Armen) Aghajanyan, Anchit Gupta, Akshat Shrivastava, Xilun Chen, Luke Zettlemoyer, and Sonal Gupta. Muppet: Massive multi-task representations with pre-finetuning, 2021. [↗](https://s3.amazonaws.com/amazon-reviews-pds/Armen)
2. Payal Bajaj, Daniel Campos, Nick Craswell, Li Deng, Jianfeng Gao, Xiaodong Liu, Rangan Majumder, Andrew McNamara, Bhaskar Mitra, Tri Nguyen, Mir Rosenberg, Xia Song, Alina Stoica, Saurabh Tiwary, and Tong Wang. [Ms marco: A human generated machine reading comprehension dataset](https://md.research.elamur.ai/free/en/papers/bajaj-2016-ms-marco-human-generated/), 2018.
3. Francesco Barbieri, Jose Camacho-Collados, Francesco Ronzano, Luis Espinosa-Anke, Miguel Ballesteros, Valerio Basile, Viviana Patti, and Horacio Saggion. [Semeval 2018 task 2: Multilingual emoji prediction](https://md.research.elamur.ai/free/en/papers/barbieri-2018-semeval-2018-task-2-multilingual/). In Proceedings of The 12th International Workshop on Semantic Evaluation, pp. 24–33, 2018.
4. Francesco Barbieri, Jose Camacho-Collados, Luis Espinosa Anke, and Leonardo Neves. Tweet-Eval: Unified benchmark and comparative evaluation for tweet classification. In Findings of the Association for Computational Linguistics: EMNLP 2020, pp. 1644–1650, Online, 2020. Association for Computational L… — [doi:10.18653/v1/2020.findings-emnlp.148](https://doi.org/10.18653/v1/2020.findings-emnlp.148) [find](https://scholar.google.com/scholar?q=Francesco+Barbieri%2C+Jose+Camacho%2DCollados%2C+Luis+Espinosa+Anke%2C+and+Leonardo+Neves%2E+Tweet%2DEval%3A+Unified+benchmark+and+comparative+evaluation+for+tweet+classification%2E+In+Findings+of+the+Association+for)
5. Valerio Basile, Cristina Bosco, Elisabetta Fersini, Debora Nozza, Viviana Patti, Francisco Manuel Rangel Pardo, Paolo Rosso, and Manuela Sanguinetti. [SemEval-2019 task 5: Multilingual detection of hate speech against immigrants and women in Twitter](https://md.research.elamur.ai/free/en/papers/basile-2019-semeval-2019-task-5-multilingual/). In Proceedings of the 13th International Workshop…
6. Jonathan Baxter. A model of inductive bias learning. Journal of artificial intelligence research, 12: 149–198, 2000. [find](https://scholar.google.com/scholar?q=Jonathan+Baxter%2E+A+model+of+inductive+bias+learning%2E+Journal+of+artificial+intelligence+research%2C+12%3A+149%E2%80%93198%2C+2000%2E)
7. Emily Bender, Timnit Gebru, Angelina McMillan-Major, and Shmargaret Shmitchell. On the dangers of stochastic parrots: can language models be too big? In Proceedings of the ACM/IEEE Joint Conference on Digital Libraries, volume 1, pp. 271–278. Association for Computing Machinery, 2020. ISBN 978145037… — [doi:10.1145/3442188.3445922](https://doi.org/10.1145/3442188.3445922) [find](https://scholar.google.com/scholar?q=Emily+Bender%2C+Timnit+Gebru%2C+Angelina+McMillan%2DMajor%2C+and+Shmargaret+Shmitchell%2E+On+the+dangers+of+stochastic+parrots%3A+can+language+models+be+too+big%3F+In+Proceedings+of+the+ACM%2FIEEE+Joint+Conference+on)
8. Jonathan Berant, Andrew Chou, Roy Frostig, and Percy Liang. [Semantic parsing on Freebase from question-answer pairs](https://md.research.elamur.ai/free/en/papers/berant-2013-semantic-parsing-freebase-question-answer/). In Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, pp. 1533–1544, Seattle, Washington, USA, 2013. Association for Computational Linguistics. UR…
9. Chandra Bhagavatula, Ronan Le Bras, Chaitanya Malaviya, Keisuke Sakaguchi, Ari Holtzman, Hannah Rashkin, Doug Downey, Wen tau Yih, and Yejin Choi. [Abductive commonsense reasoning](https://md.research.elamur.ai/free/en/papers/bhagavatula-2019-abductive-commonsense-reasoning/). In International Conference on Learning Representations, 2020. URL [https://openreview.net/forum?id=Byg1v1HKDB](https://openreview.net/forum?id=Byg1v1HKDB). [↗](https://openreview.net/forum?id=Byg1v1HKDB)
10. Joachim Bingel and Anders Søgaard. [Identifying beneficial task relations for multi-task learning in deep neural networks](https://md.research.elamur.ai/free/en/papers/bingel-2017-identifying-beneficial-task-relations-multi/). In Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 2, Short Papers, pp. 164–169, Valencia, Spain, April 2017.…
11. Yonatan Bisk, Rowan Zellers, Ronan Le Bras, Jianfeng Gao, and Yejin Choi. [Piqa: Reasoning about physical commonsense in natural language](https://md.research.elamur.ai/free/en/papers/bisk-2019-piqa-reasoning-about-physical-commonsense/). In Thirty-Fourth AAAI Conference on Artificial Intelligence, 2020.
12. Ondřej Bojar, Christian Buck, Christian Federmann, Barry Haddow, Philipp Koehn, Johannes Leveling, Christof Monz, Pavel Pecina, Matt Post, Herve Saint-Amand, Radu Soricut, Lucia Specia, and Aleš Tamchyna. [Findings of the 2014 workshop on statistical machine translation](https://md.research.elamur.ai/free/en/papers/bojar-2014-findings-2014-workshop-statistical-machine/). In Proceedings of the Ninth…
13. Ondřej Bojar, Rajen Chatterjee, Christian Federmann, Barry Haddow, Matthias Huck, Chris Hokamp, Philipp Koehn, Varvara Logacheva, Christof Monz, Matteo Negri, Matt Post, Carolina Scarton, Lucia Specia, and Marco Turchi. [Findings of the 2015 workshop on statistical machine translation](https://md.research.elamur.ai/free/en/papers/bojar-2015-findings-2015-workshop-statistical-machine/). In Proceeding…
14. Ondřej Bojar, Rajen Chatterjee, Christian Federmann, Yvette Graham, Barry Haddow, Matthias Huck, Antonio Jimeno Yepes, Philipp Koehn, Varvara Logacheva, Christof Monz, Matteo Negri, Aurélie Névéol, Mariana Neves, Martin Popel, Matt Post, Raphael Rubino, Carolina Scarton, Lucia Specia, Marco Turc… — [Findings of the 2016 Conference on Machine Translation](https://md.research.elamur.ai/free/en/papers/bojar-2016-findings-2016-conference-machine-translation/)
15. Daniel Borkan, Lucas Dixon, Jeffrey Sorensen, Nithum Thain, and Lucy Vasserman. Nuanced metrics for measuring unintended bias with real data for text classification. In Companion Proceedings of The 2019 World Wide Web Conference, WWW ’19, pp. 491–500, New York, NY, USA, 2019. Association for Computi… — [doi:10.1145/3308560.3317593](https://doi.org/10.1145/3308560.3317593) [find](https://scholar.google.com/scholar?q=Daniel+Borkan%2C+Lucas+Dixon%2C+Jeffrey+Sorensen%2C+Nithum+Thain%2C+and+Lucy+Vasserman%2E+Nuanced+metrics+for+measuring+unintended+bias+with+real+data+for+text+classification%2E+In+Companion+Proceedings+of+The+20)
16. Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter,… — [Language Models are Few-Shot Learners](https://md.research.elamur.ai/free/en/papers/brown-2020-language-models-few-shot-learners/)
17. R. Caruana, D. L. Silver, J. Baxter, T. M. Mitchell, L. Y. Pratt, and S. Thrun. Learning to learn: knowledge consolidation and transfer in inductive systems, 1995. [find](https://scholar.google.com/scholar?q=R%2E+Caruana%2C+D%2E+L%2E+Silver%2C+J%2E+Baxter%2C+T%2E+M%2E+Mitchell%2C+L%2E+Y%2E+Pratt%2C+and+S%2E+Thrun%2E+Learning+to+learn%3A+knowledge+consolidation+and+transfer+in+inductive+systems%2C+1995%2E)
18. Rich Caruana. Multitask learning. Machine learning, 28(1):41–75, 1997. [find](https://scholar.google.com/scholar?q=Rich+Caruana%2E+Multitask+learning%2E+Machine+learning%2C+28%281%29%3A41%E2%80%9375%2C+1997%2E)
19. Iñigo Casanueva, Tadas Temčinas, Daniela Gerz, Matthew Henderson, and Ivan Vulić. [Efficient intent detection with dual sentence encoders](https://md.research.elamur.ai/free/en/papers/casanueva-2020-efficient-intent-detection-dual-sentence/), 2020.
20. Thiago Castro Ferreira, Claire Gardent, Nikolai Ilinykh, Chris van der Lee, Simon Mille, Diego Moussallem, and Anastasia Shimorina. The 2020 [bilingual, bi-directional webnlg+ shared task overview and evaluation results (webnlg+ 2020)](https://md.research.elamur.ai/free/en/papers/ferreira-2020-2020-bilingual-bi-directional-webnlg/). In Proceedings of the 3rd WebNLG Workshop on Natural Language Gen…
21. Daniel Cer, Mona Diab, Eneko Agirre, Iñigo Lopez-Gazpio, and Lucia Specia. [SemEval-2017 task 1: Semantic textual similarity multilingual and crosslingual focused evaluation](https://md.research.elamur.ai/free/en/papers/cer-2017-semeval-2017-task-1-semantic/). In Proceedings of the 11th International Workshop on Semantic Evaluation (SemEval-2017), pp. 1–14, Vancouver, Canada, 2017. A…
22. Wei-Cheng Chang, Felix X Yu, Yin-Wen Chang, Yiming Yang, and Sanjiv Kumar. Pre-training tasks for embedding-based large-scale retrieval. arXiv preprint arXiv:2002.03932, 2020. — [arXiv:2002.03932](https://arxiv.org/abs/2002.03932) [find](https://scholar.google.com/scholar?q=Wei%2DCheng+Chang%2C+Felix+X+Yu%2C+Yin%2DWen+Chang%2C+Yiming+Yang%2C+and+Sanjiv+Kumar%2E+Pre%2Dtraining+tasks+for+embedding%2Dbased+large%2Dscale+retrieval%2E+arXiv+preprint+arXiv%3A2002%2E03932%2C+2020%2E+%E2%80%94+%5BarXiv%3A2002%2E03932%5D%28htt)
23. Eunsol Choi, He He, Mohit Iyyer, Mark Yatskar, Wen-tau Yih, Yejin Choi, Percy Liang, and Luke Zettlemoyer. [QuAC: Question answering in context](https://md.research.elamur.ai/free/en/papers/choi-2018-quac-question-answering-context/). In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pp. 2174–2184, Brussels, Belgium, 2018. Association for Computati…
24. Christopher Clark, Kenton Lee, Ming-Wei Chang, Tom Kwiatkowski, Michael Collins, and Kristina Toutanova. [BoolQ: Exploring the surprising difficulty of natural yes/no questions](https://md.research.elamur.ai/free/en/papers/clark-2019-boolq-exploring-surprising-difficulty-natural/). In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human…
25. Peter Clark, Isaac Cowhey, Oren Etzioni, Tushar Khot, Ashish Sabharwal, Carissa Schoenick, and Oyvind Tafjord. [Think you have solved question answering? try arc, the ai2 reasoning challenge](https://md.research.elamur.ai/free/en/papers/clark-2018-think-have-solved-question-answering/). ArXiv, abs/1803.05457, 2018.
26. Ronan Collobert and Jason Weston. A unified architecture for natural language processing: Deep neural networks with multitask learning. In Proceedings of the 25th International Conference on Machine Learning, ICML ’08, pp. 160–167, New York, NY, USA, 2008. Association for Computing Machinery. ISBN 9… — [doi:10.1145/1390156.1390177](https://doi.org/10.1145/1390156.1390177) [find](https://scholar.google.com/scholar?q=Ronan+Collobert+and+Jason+Weston%2E+A+unified+architecture+for+natural+language+processing%3A+Deep+neural+networks+with+multitask+learning%2E+In+Proceedings+of+the+25th+International+Conference+on+Machine+L)
27. Ido Dagan, Oren Glickman, and Bernardo Magnini. The pascal recognising textual entailment challenge. In Joaquin Quiñonero-Candela, Ido Dagan, Bernardo Magnini, and Florence d’Alché Buc (eds.), Machine Learning Challenges. Evaluating Predictive Uncertainty, Visual Object Classification, and Recogni… [find](https://scholar.google.com/scholar?q=Ido+Dagan%2C+Oren+Glickman%2C+and+Bernardo+Magnini%2E+The+pascal+recognising+textual+entailment+challenge%2E+In+Joaquin+Quin%CC%83onero%2DCandela%2C+Ido+Dagan%2C+Bernardo+Magnini%2C+and+Florence+d%E2%80%99Alche%CC%81+Buc+%28eds%2E%29%2C+Machi)
28. Marie-Catherine De Marneffe, Mandy Simons, and Judith Tonhauser. [The CommitmentBank: Investigating projection in naturally occurring discourse](https://md.research.elamur.ai/free/en/papers/marneffe-2019-commitmentbank-investigating-projection-naturally/). 2019. To appear in proceedings of Sinn und Bedeutung 23. Data can be found at [https://github.com/mcdm/CommitmentBank/](https://github.com/mcdm/CommitmentBank/). [↗](https://github.com/mcdm/CommitmentBank/)
29. Dorottya Demszky, Dana Movshovitz-Attias, Jeongwoo Ko, Alan Cowen, Gaurav Nemade, and Sujith Ravi. GoEmotions: A dataset of fine-grained emotions. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp. 4040–4054, Online, 2020. Association for Computational Li… [find](https://scholar.google.com/scholar?q=Dorottya+Demszky%2C+Dana+Movshovitz%2DAttias%2C+Jeongwoo+Ko%2C+Alan+Cowen%2C+Gaurav+Nemade%2C+and+Sujith+Ravi%2E+GoEmotions%3A+A+dataset+of+fine%2Dgrained+emotions%2E+In+Proceedings+of+the+58th+Annual+Meeting+of+the+Asso)
30. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. [BERT: Pre-training of deep bidirectional transformers for language understanding](https://md.research.elamur.ai/free/en/papers/devlin-2019-bert-pre-training-deep-bidirectional/). In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volum…
31. Jay DeYoung, Sarthak Jain, Nazneen Fatema Rajani, Eric Lehman, Caiming Xiong, Richard Socher, and Byron C. Wallace. [ERASER: A benchmark to evaluate rationalized NLP models](https://md.research.elamur.ai/free/en/papers/deyoung-2020-eraser-benchmark-evaluate-rationalized-nlp/). In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp. 4443–4458, Online, 2020. Associ…
32. Emily Dinan, Stephen Roller, Kurt Shuster, Angela Fan, Michael Auli, and Jason Weston. [Wizard of Wikipedia: Knowledge-powered conversational agents](https://md.research.elamur.ai/free/en/papers/dinan-2018-wizard-wikipedia-knowledge-powered-conversational/). In Proceedings of the International Conference on Learning Representations (ICLR), 2019.
33. William B. Dolan and Chris Brockett. [Automatically constructing a corpus of sentential paraphrases](https://md.research.elamur.ai/free/en/papers/dolan-2005-automatically-constructing-corpus-sentential-paraphrases/). In Proceedings of the Third International Workshop on Paraphrasing (IWP2005), 2005. URL https://aclanthology.org/I05-5002. [↗](https://aclanthology.org/I05-5002)
34. Dheeru Dua, Yizhong Wang, Pradeep Dasigi, Gabriel Stanovsky, Sameer Singh, and Matt Gardner. [DROP: A reading comprehension benchmark requiring discrete reasoning over paragraphs](https://md.research.elamur.ai/free/en/papers/dua-2019-drop-reading-comprehension-benchmark-requiring/). In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Hum…
35. Ondřej Dušek, David M Howcroft, and Verena Rieser. [Semantic Noise Matters for Neural Natural Language Generation](https://md.research.elamur.ai/free/en/papers/dusek-2019-semantic-noise-matters-neural-natural/). In Proceedings of the 12th International Conference on Natural Language Generation (INLG 2019), pp. 421–426, Tokyo, Japan, 2019. URL [https://www.aclweb.org/anthology/W19-8652/](https://www.aclweb.org/anthology/W19-8652/). [↗](https://www.aclweb.org/anthology/W19-8652/)
36. Alexander Fabbri, Irene Li, Tianwei She, Suyi Li, and Dragomir Radev. [Multi-news: A large-scale multi-document summarization dataset and abstractive hierarchical model](https://md.research.elamur.ai/free/en/papers/fabbri-2019-multi-news-large-scale-multi/). In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pp. 1074–1084, Florence, Italy, 2019. A…
37. Claire Cardie Faisal Ladhak, Esin Durmus and Kathleen McKeown. Wikilingua: A new benchmark dataset for multilingual abstractive summarization. In Findings of EMNLP, 2020, 2020. [find](https://scholar.google.com/scholar?q=Claire+Cardie+Faisal+Ladhak%2C+Esin+Durmus+and+Kathleen+McKeown%2E+Wikilingua%3A+A+new+benchmark+dataset+for+multilingual+abstractive+summarization%2E+In+Findings+of+EMNLP%2C+2020%2C+2020%2E)
38. Angela Fan, Yacine Jernite, Ethan Perez, David Grangier, Jason Weston, and Michael Auli. [Eli5: Long form question answering](https://md.research.elamur.ai/free/en/papers/fan-2019-eli5-long-form-question-answering/), 2019.
39. Christopher Fifty, Ehsan Amid, Zhe Zhao, Tianhe Yu, Rohan Anil, and Chelsea Finn. Efficiently identifying task groupings for multi-task learning. arXiv preprint arXiv:2109.04617, 2021. — [arXiv:2109.04617](https://arxiv.org/abs/2109.04617) [find](https://scholar.google.com/scholar?q=Christopher+Fifty%2C+Ehsan+Amid%2C+Zhe+Zhao%2C+Tianhe+Yu%2C+Rohan+Anil%2C+and+Chelsea+Finn%2E+Efficiently+identifying+task+groupings+for+multi%2Dtask+learning%2E+arXiv+preprint+arXiv%3A2109%2E04617%2C+2021%2E+%E2%80%94+%5BarXiv%3A2109%2E0)
40. Claire Gardent, Anastasia Shimorina, Shashi Narayan, and Laura Perez-Beltrachini. [Creating training corpora for nlg micro-planners](https://md.research.elamur.ai/free/en/papers/gardent-2017-creating-training-corpora-nlg-micro/). In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 179–188. Association for Computational Linguisti…
41. Sebastian Gehrmann, Tosin Adewumi, Karmanya Aggarwal, Pawan Sasanka Ammanamanchi, Anuoluwapo Aremu, Antoine Bosselut, Khyathi Raghavi Chandu, Miruna-Adriana Clinciu, Dipanjan Das, Kaustubh Dhole, Wanyu Du, Esin Durmus, Ondřej Dušek, Chris Chinenye Emezue, Varun Gangal, Cristina Garbacea, Tatsunori… — [The GEM Benchmark: Natural Language Generation, its Evaluation and Metrics](https://md.research.elamur.ai/free/en/papers/gehrmann-2021-gem-benchmark-natural-language-generation/)
42. Mor Geva, Uri Katz, Aviv Ben-Arie, and Jonathan Berant. What’s in your head? emergent behaviour in multi-task transformer models. In EMNLP, 2021. [find](https://scholar.google.com/scholar?q=Mor+Geva%2C+Uri+Katz%2C+Aviv+Ben%2DArie%2C+and+Jonathan+Berant%2E+What%E2%80%99s+in+your+head%3F+emergent+behaviour+in+multi%2Dtask+transformer+models%2E+In+EMNLP%2C+2021%2E)
43. Bogdan Gliwa, Iwona Mochol, Maciej Biesek, and Aleksander Wawer. [SAMSum corpus: A human-annotated dialogue dataset for abstractive summarization](https://md.research.elamur.ai/free/en/papers/gliwa-2019-samsum-corpus-human-annotated-dialogue/). In Proceedings of the 2nd Workshop on New Frontiers in Summarization, pp. 70–79, Hong Kong, China, 2019. Association for Computational Linguistics. doi: 1…
44. Alec Go, Richa Bhayani, and Lei Huang. Twitter sentiment classification using distant supervision, 2009. URL [http://help.sentiment140.com/home](http://help.sentiment140.com/home). [↗](http://help.sentiment140.com/home)
45. Han Guo, Ramakanth Pasunuru, and Mohit Bansal. AutoSeM: Automatic task selection and mixing in multi-task learning. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pp.… [find](https://scholar.google.com/scholar?q=Han+Guo%2C+Ramakanth+Pasunuru%2C+and+Mohit+Bansal%2E+AutoSeM%3A+Automatic+task+selection+and+mixing+in+multi%2Dtask+learning%2E+In+Proceedings+of+the+2019+Conference+of+the+North+American+Chapter+of+the+Associati)
46. Jiaqi Guo, Qian Liu, Jian-Guang Lou, Zhenwen Li, Xueqing Liu, Tao Xie, and Ting Liu. Benchmarking meaning representations in neural semantic parsing. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 1520–1540, Online, 2020. Association for Comput… [find](https://scholar.google.com/scholar?q=Jiaqi+Guo%2C+Qian+Liu%2C+Jian%2DGuang+Lou%2C+Zhenwen+Li%2C+Xueqing+Liu%2C+Tao+Xie%2C+and+Ting+Liu%2E+Benchmarking+meaning+representations+in+neural+semantic+parsing%2E+In+Proceedings+of+the+2020+Conference+on+Empirical)
47. Zhaochen Guo and Denilson Barbosa. Robust entity linking via random walks. In Proceedings of the 23rd ACM International Conference on Conference on Information and Knowledge Management, CIKM 2014, Shanghai, China, November 3-7, 2014, pp. 499–508, 2014. doi: 10.1145/2661829. 2661887. URL [http://doi.a…](http://doi.a…) — [doi:10.1145/2661829](https://doi.org/10.1145/2661829) [↗](http://doi.a…)
48. Sonal Gupta, Rushin Shah, Mrinal Mohit, Anuj Kumar, and Mike Lewis. [Semantic parsing for task oriented dialog using hierarchical representations](https://md.research.elamur.ai/free/en/papers/gupta-2018-semantic-parsing-task-oriented-dialog/), 2018.
49. Suchin Gururangan, Ana Marasović, Swabha Swayamdipta, Kyle Lo, Iz Beltagy, Doug Downey, and Noah A. Smith. Don’t stop pretraining: Adapt language models to domains and tasks. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp. 8342–8360, Online, July 2020… [find](https://scholar.google.com/scholar?q=Suchin+Gururangan%2C+Ana+Marasovic%CC%81%2C+Swabha+Swayamdipta%2C+Kyle+Lo%2C+Iz+Beltagy%2C+Doug+Downey%2C+and+Noah+A%2E+Smith%2E+Don%E2%80%99t+stop+pretraining%3A+Adapt+language+models+to+domains+and+tasks%2E+In+Proceedings+of+the+58)
50. David Ha, Andrew Dai, and Quoc V. Le. Hypernetworks, 2016. [find](https://scholar.google.com/scholar?q=David+Ha%2C+Andrew+Dai%2C+and+Quoc+V%2E+Le%2E+Hypernetworks%2C+2016%2E)
51. Jonathan Herzig, Peter Shaw, Ming-Wei Chang, Kelvin Guu, Panupong Pasupat, and Yuan Zhang. Unlocking compositional generalization in pre-trained models using intermediate representations, 2021. [find](https://scholar.google.com/scholar?q=Jonathan+Herzig%2C+Peter+Shaw%2C+Ming%2DWei+Chang%2C+Kelvin+Guu%2C+Panupong+Pasupat%2C+and+Yuan+Zhang%2E+Unlocking+compositional+generalization+in+pre%2Dtrained+models+using+intermediate+representations%2C+2021%2E)
52. Eduard Hovy, Laurie Gerber, Ulf Hermjakob, Chin-Yew Lin, and Deepak Ravichandran. [Toward semantics-based answer pinpointing](https://md.research.elamur.ai/free/en/papers/hovy-2001-semantics-based-answer-pinpointing/). In Proceedings of the First International Conference on Human Language Technology Research, 2001. URL https://aclanthology.org/ H01-1069. [↗](https://aclanthology.org/)
53. Junjie Hu, Sebastian Ruder, Aditya Siddhant, Graham Neubig, Orhan Firat, and Melvin Johnson. [XTREME: A Massively Multilingual Multi-task Benchmark for Evaluating Cross-lingual Generalization](https://md.research.elamur.ai/free/en/papers/hu-2020-xtreme-massively-multilingual-multi-task/). In Proceedings of ICML 2020, 2020.
54. Lifu Huang, Ronan Le Bras, Chandra Bhagavatula, and Yejin Choi. [Cosmos QA: Machine reading comprehension with contextual commonsense reasoning](https://md.research.elamur.ai/free/en/papers/huang-2019-cosmos-qa-machine-reading-comprehension/). In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Pro…
55. Harsh Jhamtani, Varun Gangal, Eduard Hovy, and Eric Nyberg. [Shakespearizing modern language using copy-enriched sequence to sequence models](https://md.research.elamur.ai/free/en/papers/jhamtani-2017-shakespearizing-modern-language-copy-enriched/). In Proceedings of the Workshop on Stylistic Variation, pp. 10–19, Copenhagen, Denmark, 2017. Association for Computational Linguistics. doi: 10.18653/v1/W17-49…
56. Robin Jia, Mike Lewis, and Luke Zettlemoyer. Question answering infused pre-training of general-purpose contextualized representations, 2021. [find](https://scholar.google.com/scholar?q=Robin+Jia%2C+Mike+Lewis%2C+and+Luke+Zettlemoyer%2E+Question+answering+infused+pre%2Dtraining+of+general%2Dpurpose+contextualized+representations%2C+2021%2E)
57. Chao Jiang, Mounica Maddela, Wuwei Lan, Yang Zhong, and Wei Xu. [Neural CRF model for sentence alignment in text simplification](https://md.research.elamur.ai/free/en/papers/jiang-2020-neural-crf-model-sentence-alignment/). In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp. 7943–7960, Online, July 2020. Association for Computational Linguistics. doi…
58. Mandar Joshi, Eunsol Choi, Daniel Weld, and Luke Zettlemoyer. [TriviaQA: A large scale distantly supervised challenge dataset for reading comprehension](https://md.research.elamur.ai/free/en/papers/joshi-2017-triviaqa-large-scale-distantly-supervised/). In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 1601–1611, Vancouver, Canada…
59. Mandar Joshi, Danqi Chen, Yinhan Liu, Daniel S Weld, Luke Zettlemoyer, and Omer Levy. [Spanbert: Improving pre-training by representing and predicting spans](https://md.research.elamur.ai/free/en/papers/joshi-2019-spanbert-improving-pre-training-representing/). Transactions of the Association for Computational Linguistics, 8:64–77, 2020.
60. Jared Kaplan, Sam McCandlish, Tom Henighan, Tom B Brown, Benjamin Chess, Rewon Child, Scott Gray, Alec Radford, Jeffrey Wu, and Dario Amodei. [Scaling laws for neural language models](https://md.research.elamur.ai/free/en/papers/kaplan-2020-scaling-laws-neural-language-models/). arXiv preprint arXiv:2001.08361, 2020.
61. Daniel Khashabi, Snigdha Chaturvedi, Michael Roth, Shyam Upadhyay, and Dan Roth. [Looking beyond the surface: A challenge set for reading comprehension over multiple sentences](https://md.research.elamur.ai/free/en/papers/khashabi-2018-looking-beyond-surface-challenge-set/). In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human…
62. Daniel Khashabi, Sewon Min, Tushar Khot, Ashish Sabharwal, Oyvind Tafjord, Peter Clark, and Hannaneh Hajishirzi. [UNIFIEDQA: Crossing format boundaries with a single QA system](https://md.research.elamur.ai/free/en/papers/khashabi-2020-unifiedqa-crossing-format-boundaries-single/). In Findings of the Association for Computational Linguistics: EMNLP 2020, pp. 1896–1907, Online, 2020. Association for Compu…
63. Najoung Kim and Tal Linzen. COGS: A compositional generalization challenge based on semantic interpretation. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 9087–9105, Online, 2020. Association for Computational Linguistics. doi: 10.18653/v1/202… [find](https://scholar.google.com/scholar?q=Najoung+Kim+and+Tal+Linzen%2E+COGS%3A+A+compositional+generalization+challenge+based+on+semantic+interpretation%2E+In+Proceedings+of+the+2020+Conference+on+Empirical+Methods+in+Natural+Language+Processing+%28)
64. Sneha Kudugunta, Ankur Bapna, Isaac Caswell, and Orhan Firat. [Investigating multilingual NMT representations at scale](https://md.research.elamur.ai/free/en/papers/kudugunta-2019-investigating-multilingual-nmt-representations-scale/). In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), p…
65. Tom Kwiatkowski, Jennimaria Palomaki, Olivia Redfield, Michael Collins, Ankur Parikh, Chris Alberti, Danielle Epstein, Illia Polosukhin, Jacob Devlin, Kenton Lee, Kristina Toutanova, Llion Jones, Matthew Kelcey, Ming-Wei Chang, Andrew M. Dai, Jakob Uszkoreit, Quoc Le, and Slav Petrov. Natural Questi… — [Natural Questions: A Benchmark for Question Answering Research](https://md.research.elamur.ai/free/en/papers/kwiatkowski-2019-natural-questions-benchmark-question-answering/)
66. Guokun Lai, Qizhe Xie, Hanxiao Liu, Yiming Yang, and Eduard Hovy. [RACE: Large-scale ReAding comprehension dataset from examinations](https://md.research.elamur.ai/free/en/papers/lai-2017-race-large-scale-reading-comprehension/). In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pp. 785–794, Copenhagen, Denmark, 2017. Association for Computational Lingui…
67. Zhenzhong Lan, Mingda Chen, Sebastian Goodman, Kevin Gimpel, Piyush Sharma, and Radu Soricut. [Albert: A lite bert for self-supervised learning of language representations](https://md.research.elamur.ai/free/en/papers/lan-2019-albert-lite-bert-self-supervised/). arXiv preprint arXiv:1909.11942, 2019.
68. Stefan Larson, Anish Mahendran, Joseph J. Peper, Christopher Clarke, Andrew Lee, Parker Hill, Jonathan K. Kummerfeld, Kevin Leach, Michael A. Laurenzano, Lingjia Tang, and Jason Mars. An evaluation dataset for intent classification and out-of-scope prediction. In Proceedings of the 2019 Conference o… [find](https://scholar.google.com/scholar?q=Stefan+Larson%2C+Anish+Mahendran%2C+Joseph+J%2E+Peper%2C+Christopher+Clarke%2C+Andrew+Lee%2C+Parker+Hill%2C+Jonathan+K%2E+Kummerfeld%2C+Kevin+Leach%2C+Michael+A%2E+Laurenzano%2C+Lingjia+Tang%2C+and+Jason+Mars%2E+An+evaluation+da)
69. Adam D. Lelkes, Vinh Q. Tran, and Cong Yu. Quiz-Style Question Generation for News Stories, pp. 2501–2511. Association for Computing Machinery, New York, NY, USA, 2021. ISBN 9781450383127. URL https://doi.org/10.1145/3442381.3449892. — [doi:10.1145/3442381.3449892](https://doi.org/10.1145/3442381.3449892) [↗](https://doi.org/10.1145/3442381.3449892)
70. Hector J Levesque, Ernest Davis, and Leora Morgenstern. The Winograd schema challenge. In AAAI Spring Symposium: Logical Formalizations of Commonsense Reasoning, volume 46, pp. 47, 2011. [find](https://scholar.google.com/scholar?q=Hector+J+Levesque%2C+Ernest+Davis%2C+and+Leora+Morgenstern%2E+The+Winograd+schema+challenge%2E+In+AAAI+Spring+Symposium%3A+Logical+Formalizations+of+Commonsense+Reasoning%2C+volume+46%2C+pp%2E+47%2C+2011%2E)
71. Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Ves Stoyanov, and Luke Zettlemoyer. [Bart: Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension](https://md.research.elamur.ai/free/en/papers/lewis-2020-bart-denoising-sequence-sequence-pre/). arXiv preprint arXiv:1910.13461, 2019.
72. Junlong Li, Zhuosheng Zhang, Hai Zhao, Xi Zhou, and Xiang Zhou. Task-specific objectives of pre-trained language models for dialogue adaptation, 2020. [find](https://scholar.google.com/scholar?q=Junlong+Li%2C+Zhuosheng+Zhang%2C+Hai+Zhao%2C+Xi+Zhou%2C+and+Xiang+Zhou%2E+Task%2Dspecific+objectives+of+pre%2Dtrained+language+models+for+dialogue+adaptation%2C+2020%2E)
73. Ya Li, Xinmei Tian, Tongliang Liu, and Dacheng Tao. On better exploring and exploiting task relationships in multitask learning: Joint model and feature learning. IEEE Transactions on Neural Networks and Learning Systems, 29(5):1975–1985, May 2018. ISSN 2162-2388. doi: 10.1109/ tnnls.2017.2690683. U… — [doi:10.1109/tnnls.2017.2690683](https://doi.org/10.1109/tnnls.2017.2690683) [find](https://scholar.google.com/scholar?q=Ya+Li%2C+Xinmei+Tian%2C+Tongliang+Liu%2C+and+Dacheng+Tao%2E+On+better+exploring+and+exploiting+task+relationships+in+multitask+learning%3A+Joint+model+and+feature+learning%2E+IEEE+Transactions+on+Neural+Networks+)
74. Bill Yuchen Lin, Wangchunshu Zhou, Ming Shen, Pei Zhou, Chandra Bhagavatula, Yejin Choi, and Xiang Ren. CommonGen: A constrained text generation challenge for generative commonsense reasoning. In Findings of the Association for Computational Linguistics: EMNLP 2020, pp. 1823–1840, Online, November 2… [find](https://scholar.google.com/scholar?q=Bill+Yuchen+Lin%2C+Wangchunshu+Zhou%2C+Ming+Shen%2C+Pei+Zhou%2C+Chandra+Bhagavatula%2C+Yejin+Choi%2C+and+Xiang+Ren%2E+CommonGen%3A+A+constrained+text+generation+challenge+for+generative+commonsense+reasoning%2E+In+Find)
75. Yu-Hsiang Lin, Chian-Yu Chen, Jean Lee, Zirui Li, Yuyan Zhang, Mengzhou Xia, Shruti Rijhwani, Junxian He, Zhisong Zhang, Xuezhe Ma, Antonios Anastasopoulos, Patrick Littell, and Graham Neubig. [Choosing Transfer Languages for Cross-Lingual Learning](https://md.research.elamur.ai/free/en/papers/lin-2019-choosing-transfer-languages-cross-lingual/). In Proceedings of ACL 2019, 2019. URL [http://arxiv.…](http://arxiv.…) [↗](http://arxiv.…)
76. Xiaodong Liu, Pengcheng He, Weizhu Chen, and Jianfeng Gao. [Multi-task deep neural networks for natural language understanding](https://md.research.elamur.ai/free/en/papers/liu-2019-multi-task-deep-neural-networks/). In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pp. 4487–4496, Florence, Italy, 2019a. Association for Computational Linguistics.…
77. Xingkun Liu, Arash Eshghi, Pawel Swietojanski, and Verena Rieser. Benchmarking natural language understanding services for building conversational agents, 2019b. [find](https://scholar.google.com/scholar?q=Xingkun+Liu%2C+Arash+Eshghi%2C+Pawel+Swietojanski%2C+and+Verena+Rieser%2E+Benchmarking+natural+language+understanding+services+for+building+conversational+agents%2C+2019b%2E)
78. Nicholas Lourie, Ronan Le Bras, Chandra Bhagavatula, and Yejin Choi. Unicorn on rainbow: A universal commonsense reasoning model on a new multitask benchmark. AAAI, 2021. [find](https://scholar.google.com/scholar?q=Nicholas+Lourie%2C+Ronan+Le+Bras%2C+Chandra+Bhagavatula%2C+and+Yejin+Choi%2E+Unicorn+on+rainbow%3A+A+universal+commonsense+reasoning+model+on+a+new+multitask+benchmark%2E+AAAI%2C+2021%2E)
79. Yiwei Lyu, Paul Pu Liang, Hai Pham, Eduard Hovy, Barnabás Póczos, Ruslan Salakhutdinov, and Louis-Philippe Morency. StylePTB: A compositional benchmark for fine-grained controllable text style transfer. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Comp… [find](https://scholar.google.com/scholar?q=Yiwei+Lyu%2C+Paul+Pu+Liang%2C+Hai+Pham%2C+Eduard+Hovy%2C+Barnaba%CC%81s+Po%CC%81czos%2C+Ruslan+Salakhutdinov%2C+and+Louis%2DPhilippe+Morency%2E+StylePTB%3A+A+compositional+benchmark+for+fine%2Dgrained+controllable+text+style+trans)
80. Andrew L. Maas, Raymond E. Daly, Peter T. Pham, Dan Huang, Andrew Y. Ng, and Christopher Potts. [Learning word vectors for sentiment analysis](https://md.research.elamur.ai/free/en/papers/maas-2011-learning-word-vectors-sentiment-analysis/). In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, pp. 142–150, Portland, Oregon, USA, 2…
81. Sam McCandlish, Jared Kaplan, Dario Amodei, and OpenAI Dota Team. An empirical model of large-batch training, 2018. [find](https://scholar.google.com/scholar?q=Sam+McCandlish%2C+Jared+Kaplan%2C+Dario+Amodei%2C+and+OpenAI+Dota+Team%2E+An+empirical+model+of+large%2Dbatch+training%2C+2018%2E)
82. Bryan McCann, Nitish Shirish Keskar, Caiming Xiong, and Richard Socher. [The natural language decathlon: Multitask learning as question answering](https://md.research.elamur.ai/free/en/papers/mccann-2018-natural-language-decathlon-multitask-learning/), 2018.
83. Shikib Mehri, Mihail Eric, and Dilek Hakkani-Tur. Dialoglue: A natural language understanding benchmark for task-oriented dialogue, 2020. [find](https://scholar.google.com/scholar?q=Shikib+Mehri%2C+Mihail+Eric%2C+and+Dilek+Hakkani%2DTur%2E+Dialoglue%3A+A+natural+language+understanding+benchmark+for+task%2Doriented+dialogue%2C+2020%2E)
84. Saif Mohammad, Svetlana Kiritchenko, Parinaz Sobhani, Xiaodan Zhu, and Colin Cherry. [Semeval-2016 task 6: Detecting stance in tweets](https://md.research.elamur.ai/free/en/papers/mohammad-2016-semeval-2016-task-6-detecting/). In Proceedings of the 10th International Workshop on Semantic Evaluation (SemEval-2016), pp. 31–41, 2016.
85. Saif Mohammad, Felipe Bravo-Marquez, Mohammad Salameh, and Svetlana Kiritchenko. [Semeval-2018 task 1: Affect in tweets](https://md.research.elamur.ai/free/en/papers/mohammad-2018-semeval-2018-task-1-affect/). In Proceedings of the 12th international workshop on semantic evaluation, pp. 1–17, 2018.
86. Shashi Narayan, Shay B. Cohen, and Mirella Lapata. Don’t give me the details, just the summary! topic-aware convolutional neural networks for extreme summarization. In Proceedings of the 2018 [Conference on Empirical Methods in Natural Language Processing, pp](https://md.research.elamur.ai/free/en/papers/narayan-2018-don-t-give-me-details/). 1797–1807, Brussels, Belgium, 2018. Asso…
87. Yixin Nie, Adina Williams, Emily Dinan, Mohit Bansal, Jason Weston, and Douwe Kiela. [Adversarial NLI: A new benchmark for natural language understanding](https://md.research.elamur.ai/free/en/papers/nie-2020-adversarial-nli-new-benchmark-natural/). In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp. 4885–4901, Online, 2020. Association for Computati…
88. Rodrigo Nogueira, Zhiying Jiang, Ronak Pradeep, and Jimmy Lin. [Document ranking with a pretrained sequence-to-sequence model](https://md.research.elamur.ai/free/en/papers/nogueira-2020-document-ranking-pretrained-sequence-sequence/). In Findings of the Association for Computational Linguistics: EMNLP 2020, pp. 708–718, Online, 2020. Association for Computational Linguistics. doi: 10.18653/v1/2020.findings…
89. Richard Yuanzhe Pang, Adam Lelkes, Vinh Tran, and Cong Yu. AgreeSum: Agreement-oriented multi-document summarization. In Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, pp. 3377–3391, Online, 2021. Association for Computational Linguistics. doi: 10.18653/v1/2021.findings-… [find](https://scholar.google.com/scholar?q=Richard+Yuanzhe+Pang%2C+Adam+Lelkes%2C+Vinh+Tran%2C+and+Cong+Yu%2E+AgreeSum%3A+Agreement%2Doriented+multi%2Ddocument+summarization%2E+In+Findings+of+the+Association+for+Computational+Linguistics%3A+ACL%2DIJCNLP+2021%2C+pp%2E)
90. Ankur P Parikh, Xuezhi Wang, Sebastian Gehrmann, Manaal Faruqui, Bhuwan Dhingra, Diyi Yang, and Dipanjan Das. [Totto: A controlled table-to-text generation dataset](https://md.research.elamur.ai/free/en/papers/parikh-2020-totto-controlled-table-text-generation/). arXiv preprint arXiv:2004.14373, 2020.
91. Fabio Petroni, Aleksandra Piktus, Angela Fan, Patrick Lewis, Majid Yazdani, Nicola De Cao, James Thorne, Yacine Jernite, Vladimir Karpukhin, Jean Maillard, Vassilis Plachouras, Tim Rocktäschel, and Sebastian Riedel. [KILT: a benchmark for knowledge intensive language tasks](https://md.research.elamur.ai/free/en/papers/petroni-2020-kilt-benchmark-knowledge-intensive-language/). In Proceedings of the 202…
92. Jason Phang, Thibault Févry, and Samuel R. Bowman. [Sentence encoders on stilts: Supplementary training on intermediate labeled-data tasks](https://md.research.elamur.ai/free/en/papers/phang-2018-sentence-encoders-stilts-supplementary-training/), 2019.
93. Mohammad Taher Pilehvar and Jose Camacho-Collados. [WiC: the word-in-context dataset for evaluating context-sensitive meaning representations](https://md.research.elamur.ai/free/en/papers/pilehvar-2019-wic-word-context-dataset-evaluating/). In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (L…
94. Lorien Y Pratt, Jack Mostow, Candace A Kamm, Ace A Kamm, et al. Direct transfer of learned information among neural networks. In Aaai, volume 91, pp. 584–589, 1991. [find](https://scholar.google.com/scholar?q=Lorien+Y+Pratt%2C+Jack+Mostow%2C+Candace+A+Kamm%2C+Ace+A+Kamm%2C+et+al%2E+Direct+transfer+of+learned+information+among+neural+networks%2E+In+Aaai%2C+volume+91%2C+pp%2E+584%E2%80%93589%2C+1991%2E)
95. Dragomir Radev, Rui Zhang, Amrit Rau, Abhinand Sivaprasad, Chiachun Hsieh, Nazneen Fatema Rajani, Xiangru Tang, Aadit Vyas, Neha Verma, Pranav Krishna, Yangxiaokang Liu, Nadia Irwanto, Jessica Pan, Faiaz Rahman, Ahmad Zaidi, Murori Mutuma, Yasin Tarabar, Ankit Gupta, Tao Yu, Yi Chern Tan, Xi Victori… — [DART: Open-Domain Structured Data Record to Text Generation](https://md.research.elamur.ai/free/en/papers/nan-2020-dart-open-domain-structured-data/)
96. Alec Radford, Jong Wook Kim, and Jeff Wu. Gpt-2 output dataset. [https://github.com/](https://github.com/) openai/gpt-2-output-dataset, 2019. [↗](https://github.com/)
97. Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. [Exploring the limits of transfer learning with a unified text-to-text transformer](https://md.research.elamur.ai/free/en/papers/raffel-2019-exploring-limits-transfer-learning-unified/). Journal of Machine Learning Research, 21(140):1–67, 2020. URL [http://jmlr.org/papers/v21/2…](http://jmlr.org/papers/v21/2…) [↗](http://jmlr.org/papers/v21/2…)
98. Pranav Rajpurkar, Jian Zhang, Konstantin Lopyrev, and Percy Liang. [SQuAD: 100,000+ questions for machine comprehension of text](https://md.research.elamur.ai/free/en/papers/rajpurkar-2016-squad-100-000-questions-machine/). In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pp. 2383–2392, Austin, Texas, 2016. Association for Computational Linguistics. do…
99. Ori Ram, Yuval Kirstain, Jonathan Berant, Amir Globerson, and Omer Levy. Few-shot question answering by pretraining span selection. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (V… [find](https://scholar.google.com/scholar?q=Ori+Ram%2C+Yuval+Kirstain%2C+Jonathan+Berant%2C+Amir+Globerson%2C+and+Omer+Levy%2E+Few%2Dshot+question+answering+by+pretraining+span+selection%2E+In+Proceedings+of+the+59th+Annual+Meeting+of+the+Association+for+Com)
100. Abhinav Rastogi, Xiaoxue Zang, Srinivas Sunkara, Raghav Gupta, and Pranav Khaitan. [Towards scalable multi-domain conversational agents: The schema-guided dialogue dataset](https://md.research.elamur.ai/free/en/papers/rastogi-2019-towards-scalable-multi-domain-conversational/), 2020.
101. Adam Roberts, Colin Raffel, and Noam Shazeer. How much knowledge can you pack into the parameters of a language model? In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 5418–5426, Online, 2020. Association for Computational Linguistics. doi: 10.18… [find](https://scholar.google.com/scholar?q=Adam+Roberts%2C+Colin+Raffel%2C+and+Noam+Shazeer%2E+How+much+knowledge+can+you+pack+into+the+parameters+of+a+language+model%3F+In+Proceedings+of+the+2020+Conference+on+Empirical+Methods+in+Natural+Language+Pr)
102. Melissa Roemmele, Cosmin Adrian Bejan, and Andrew S. Gordon. Choice of plausible alternatives: An evaluation of commonsense causal reasoning. In 2011 AAAI Spring Symposium Series, 2011. [find](https://scholar.google.com/scholar?q=Melissa+Roemmele%2C+Cosmin+Adrian+Bejan%2C+and+Andrew+S%2E+Gordon%2E+Choice+of+plausible+alternatives%3A+An+evaluation+of+commonsense+causal+reasoning%2E+In+2011+AAAI+Spring+Symposium+Series%2C+2011%2E)
103. Michael T. Rosenstein, Zvika Marx, Leslie Pack Kaelbling, and Thomas G. Dietterich. To transfer or not to transfer. In In NIPS’05 Workshop, Inductive Transfer: 10 Years Later, 2005. [find](https://scholar.google.com/scholar?q=Michael+T%2E+Rosenstein%2C+Zvika+Marx%2C+Leslie+Pack+Kaelbling%2C+and+Thomas+G%2E+Dietterich%2E+To+transfer+or+not+to+transfer%2E+In+In+NIPS%E2%80%9905+Workshop%2C+Inductive+Transfer%3A+10+Years+Later%2C+2005%2E)
104. Sara Rosenthal, Noura Farra, and Preslav Nakov. [Semeval-2017 task 4: Sentiment analysis in twitter](https://md.research.elamur.ai/free/en/papers/rosenthal-2017-semeval-2017-task-4-sentiment/). In Proceedings of the 11th international workshop on semantic evaluation (SemEval-2017), pp. 502–518, 2017.
105. Sebastian Ruder and Barbara Plank. [Learning to select data for transfer learning with Bayesian optimization](https://md.research.elamur.ai/free/en/papers/ruder-2017-learning-select-data-transfer-learning/). In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pp. 372–382, Copenhagen, Denmark, September 2017. Association for Computational Linguistics. doi: 10…
106. Sebastian Ruder, Matthew E Peters, Swabha Swayamdipta, and Thomas Wolf. [Transfer learning in natural language processing](https://md.research.elamur.ai/free/en/papers/ruder-2019-transfer-learning-natural-language-processing/). In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Tutorials, pp. 15–18, 2019.
107. Alexander M. Rush, Sumit Chopra, and Jason Weston. [A neural attention model for abstractive sentence summarization](https://md.research.elamur.ai/free/en/papers/rush-2015-neural-attention-model-abstractive-sentence/). In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pp. 379–389, Lisbon, Portugal, 2015. Association for Computational Linguistics. doi: 10.18653…
108. Keisuke Sakaguchi, Ronan Le Bras, Chandra Bhagavatula, and Yejin Choi. Winogrande: An adversarial winograd schema challenge at scale. ArXiv, abs/1907.10641, 2020. [find](https://scholar.google.com/scholar?q=Keisuke+Sakaguchi%2C+Ronan+Le+Bras%2C+Chandra+Bhagavatula%2C+and+Yejin+Choi%2E+Winogrande%3A+An+adversarial+winograd+schema+challenge+at+scale%2E+ArXiv%2C+abs%2F1907%2E10641%2C+2020%2E)
109. Keisuke Sakaguchi, Ronan Le Bras, Chandra Bhagavatula, and Yejin Choi. Winogrande: An adversarial winograd schema challenge at scale. Commun. ACM, 64(9):99–106, August 2021. ISSN 0001-0782. doi: 10.1145/3474381. URL https://doi.org/10.1145/3474381. [↗](https://doi.org/10.1145/3474381)
110. Victor Sanh, Thomas Wolf, and Sebastian Ruder. A Hierarchical Multi-task Approach for Learning Embeddings from Semantic Tasks. In Proceedings of AAAI 2019, 2019. [find](https://scholar.google.com/scholar?q=Victor+Sanh%2C+Thomas+Wolf%2C+and+Sebastian+Ruder%2E+A+Hierarchical+Multi%2Dtask+Approach+for+Learning+Embeddings+from+Semantic+Tasks%2E+In+Proceedings+of+AAAI+2019%2C+2019%2E)
111. Maarten Sap, Hannah Rashkin, Derek Chen, Ronan Le Bras, and Yejin Choi. Social IQa: Commonsense reasoning about social interactions. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EM… [find](https://scholar.google.com/scholar?q=Maarten+Sap%2C+Hannah+Rashkin%2C+Derek+Chen%2C+Ronan+Le+Bras%2C+and+Yejin+Choi%2E+Social+IQa%3A+Commonsense+reasoning+about+social+interactions%2E+In+Proceedings+of+the+2019+Conference+on+Empirical+Methods+in+Natur)
112. J. H. Schmidhuber. Evolutionary principles in self-referential learning, or on learning how to learn: the meta-meta-... hook. 1987. [find](https://scholar.google.com/scholar?q=J%2E+H%2E+Schmidhuber%2E+Evolutionary+principles+in+self%2Dreferential+learning%2C+or+on+learning+how+to+learn%3A+the+meta%2Dmeta%2D%2E%2E%2E+hook%2E+1987%2E)
113. Tal Schuster, Adam Fisch, and Regina Barzilay. [Get your vitamin C! robust fact verification with contrastive evidence](https://md.research.elamur.ai/free/en/papers/schuster-2021-get-your-vitamin-c-robust/). In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 624–643, Online, 2021. Associa…
114. Abigail See, Peter J. Liu, and Christopher D. Manning. Get to the point: Summarization with pointer-generator networks. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 1073–1083, Vancouver, Canada, 2017. Association for Computat… [find](https://scholar.google.com/scholar?q=Abigail+See%2C+Peter+J%2E+Liu%2C+and+Christopher+D%2E+Manning%2E+Get+to+the+point%3A+Summarization+with+pointer%2Dgenerator+networks%2E+In+Proceedings+of+the+55th+Annual+Meeting+of+the+Association+for+Computational+L)
115. Noam Shazeer. Glu variants improve transformer, 2020. [find](https://scholar.google.com/scholar?q=Noam+Shazeer%2E+Glu+variants+improve+transformer%2C+2020%2E)
116. Noam Shazeer, Youlong Cheng, Niki Parmar, Dustin Tran, Ashish Vaswani, Penporn Koanantakool, Peter Hawkins, HyoukJoong Lee, Mingsheng Hong, Cliff Young, Ryan Sepassi, and Blake A. Hechtman. [Mesh-tensorflow: Deep learning for supercomputers](https://md.research.elamur.ai/free/en/papers/shazeer-2018-mesh-tensorflow-deep-learning-supercomputers/). In Samy Bengio, Hanna M. Wallach, Hugo Larochelle, Kristen…
117. Samuel L. Smith, Pieter-Jan Kindermans, Chris Ying, and Quoc V. Le. Don’t decay the learning rate, increase the batch size. In 6th International Conference on Learning Representations, ICLR 2018, Vancouver, BC, Canada, April 30 - May 3, 2018, Conference Track Proceedings. [find](https://scholar.google.com/scholar?q=Samuel+L%2E+Smith%2C+Pieter%2DJan+Kindermans%2C+Chris+Ying%2C+and+Quoc+V%2E+Le%2E+Don%E2%80%99t+decay+the+learning+rate%2C+increase+the+batch+size%2E+In+6th+International+Conference+on+Learning+Representations%2C+ICLR+2018%2C+Vanc)
118. OpenReview.net, 2018. URL [https://openreview.net/forum?id=B1Yy1BxCZ](https://openreview.net/forum?id=B1Yy1BxCZ). [↗](https://openreview.net/forum?id=B1Yy1BxCZ)
119. Richard Socher, Alex Perelygin, Jean Wu, Jason Chuang, Christopher D. Manning, Andrew Ng, and Christopher Potts. [Recursive deep models for semantic compositionality over a sentiment treebank](https://md.research.elamur.ai/free/en/papers/socher-2013-recursive-deep-models-semantic-compositionality/). In Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, pp. 1631–1642, Se…
120. Irene Solaiman and Christy Dennison. [Process for adapting language models to society (palms) with values-targeted datasets](https://md.research.elamur.ai/free/en/papers/solaiman-2021-process-adapting-language-models-society/). arXiv preprint arXiv:2106.10328, 2021.
121. Christian Stab, Tristan Miller, Pranav Rai, Benjamin Schiller, and Iryna Gurevych. Ukp sentential argument mining corpus, 2018. URL [https://tudatalib.ulb.tu-darmstadt.de/](https://tudatalib.ulb.tu-darmstadt.de/) handle/tudatalib/2345. [↗](https://tudatalib.ulb.tu-darmstadt.de/)
122. Emma Strubell, Ananya Ganesh, and Andrew McCallum. [Energy and policy considerations for deep learning in nlp](https://md.research.elamur.ai/free/en/papers/strubell-2019-energy-policy-considerations-deep-learning/). arXiv preprint arXiv:1906.02243, 2019.
123. Jimin Sun, Hwijeen Ahn, Chan Young Park, Yulia Tsvetkov, and David R. Mortensen. Ranking Transfer Languages with Pragmatically-Motivated Features for Multilingual Sentiment Analysis. In Proceedings of EACL 2021, 2021. [find](https://scholar.google.com/scholar?q=Jimin+Sun%2C+Hwijeen+Ahn%2C+Chan+Young+Park%2C+Yulia+Tsvetkov%2C+and+David+R%2E+Mortensen%2E+Ranking+Transfer+Languages+with+Pragmatically%2DMotivated+Features+for+Multilingual+Sentiment+Analysis%2E+In+Proceedings+of)
124. Yi Tay, Mostafa Dehghani, Jinfeng Rao, William Fedus, Samira Abnar, Hyung Won Chung, Sharan Narang, Dani Yogatama, Ashish Vaswani, and Donald Metzler. Scale efficiently: Insights from pre-training and fine-tuning transformers. arXiv preprint arXiv:2109.10686, 2021a. [find](https://scholar.google.com/scholar?q=Yi+Tay%2C+Mostafa+Dehghani%2C+Jinfeng+Rao%2C+William+Fedus%2C+Samira+Abnar%2C+Hyung+Won+Chung%2C+Sharan+Narang%2C+Dani+Yogatama%2C+Ashish+Vaswani%2C+and+Donald+Metzler%2E+Scale+efficiently%3A+Insights+from+pre%2Dtraining+and)
125. Yi Tay, Zhe Zhao, Dara Bahri, Donald Metzler, and Da-Cheng Juan. Hypergrid transformers: Towards a single model for multiple tasks. In International Conference on Learning Representations, 2021b. URL [https://openreview.net/forum?id=hiq1rHO8pNT](https://openreview.net/forum?id=hiq1rHO8pNT). [↗](https://openreview.net/forum?id=hiq1rHO8pNT)
126. James Thorne, Andreas Vlachos, Oana Cocarascu, Christos Christodoulopoulos, and Arpit Mittal. The FEVER2.0 shared task. In Proceedings of the Second Workshop on Fact Extraction and VERification (FEVER), 2018. [find](https://scholar.google.com/scholar?q=James+Thorne%2C+Andreas+Vlachos%2C+Oana+Cocarascu%2C+Christos+Christodoulopoulos%2C+and+Arpit+Mittal%2E+The+FEVER2%2E0+shared+task%2E+In+Proceedings+of+the+Second+Workshop+on+Fact+Extraction+and+VERification+%28FEVER)
127. Erik F. Tjong Kim Sang and Fien De Meulder. [Introduction to the conll-2003 shared task: Language-independent named entity recognition](https://md.research.elamur.ai/free/en/papers/sang-2003-introduction-conll-2003-shared-task/). In Proceedings of the Seventh Conference on Natural Language Learning at HLT-NAACL 2003 - Volume 4, CONLL ’03, pp. 142–147, USA, 2003. Association for Computational…
128. Cynthia Van Hee, Els Lefever, and Véronique Hoste. [Semeval-2018 task 3: Irony detection in english tweets](https://md.research.elamur.ai/free/en/papers/hee-2018-semeval-2018-task-3-irony/). In Proceedings of The 12th International Workshop on Semantic Evaluation, pp. 39–50, 2018.
129. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. [Attention is all you need](https://md.research.elamur.ai/free/en/papers/vaswani-2017-attention-all-need/). In Isabelle Guyon, Ulrike von Luxburg, Samy Bengio, Hanna M. Wallach, Rob Fergus, S. V. N. Vishwanathan, and Roman Garnett (eds.), Advances in Neur…
130. Tu Vu, Tong Wang, Tsendsuren Munkhdalai, Alessandro Sordoni, Adam Trischler, Andrew Mattarella-Micke, Subhransu Maji, and Mohit Iyyer. [Exploring and predicting transferability across NLP tasks](https://md.research.elamur.ai/free/en/papers/vu-2020-exploring-predicting-transferability-across-nlp/). In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 788…
131. Alex Wang, Yada Pruksachatkun, Nikita Nangia, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel R. Bowman. [Superglue: A stickier benchmark for general-purpose language understanding systems](https://md.research.elamur.ai/free/en/papers/wang-2019-superglue-stickier-benchmark-general-purpose/). In Hanna M. Wallach, Hugo Larochelle, Alina Beygelzimer, Florence d’Alché-Buc, Emily B. Fox…
132. Alex Wang, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel R. Bowman. [GLUE: A multi-task benchmark and analysis platform for natural language understanding](https://md.research.elamur.ai/free/en/papers/wang-2018-glue-multi-task-benchmark-analysis/). In 7th International Conference on Learning Representations, ICLR 2019, New Orleans, LA, USA, May 6-9, 2019.
133. OpenReview.net, 2019b. URL [https://openreview.net/forum?id=](https://openreview.net/forum?id=) rJ4km2R5t7. [↗](https://openreview.net/forum?id=)
134. Zirui Wang, Yulia Tsvetkov, Orhan Firat, and Yuan Cao. Gradient vaccine: Investigating and improving multi-task optimization in massively multilingual models. In International Conference on Learning Representations, 2021. URL [https://openreview.net/forum?id=](https://openreview.net/forum?id=) F1vEjWK-lH_. [↗](https://openreview.net/forum?id=)
135. Alex Warstadt, Amanpreet Singh, and Samuel R. Bowman. Neural network acceptability judgments. Transactions of the Association for Computational Linguistics, 7:625–641, 2019. doi: 10.1162/ tacl a 00290. URL https://aclanthology.org/Q19-1040. [↗](https://aclanthology.org/Q19-1040)
136. Jason Wei, Maarten Bosma, Vincent Y. Zhao, Kelvin Guu, Adams Wei Yu, Brian Lester, Nan Du, Andrew M. Dai, and Quoc V. Le. [Finetuned language models are zero-shot learners](https://md.research.elamur.ai/free/en/papers/wei-2021-finetuned-language-models-zero-shot/), 2021.
137. Adina Williams, Nikita Nangia, and Samuel Bowman. [A broad-coverage challenge corpus for sentence understanding through inference](https://md.research.elamur.ai/free/en/papers/williams-2018-broad-coverage-challenge-corpus-sentence/). In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers),…
138. Ellery Wulczyn, Nithum Thain, and Lucas Dixon. Ex machina: Personal attacks seen at scale. In Rick Barrett, Rick Cummings, Eugene Agichtein, and Evgeniy Gabrilovich (eds.), Proceedings of the 26th International Conference on World Wide Web, WWW 2017, Perth, Australia, April 3-7, 2017, pp. 1391–1399. [find](https://scholar.google.com/scholar?q=Ellery+Wulczyn%2C+Nithum+Thain%2C+and+Lucas+Dixon%2E+Ex+machina%3A+Personal+attacks+seen+at+scale%2E+In+Rick+Barrett%2C+Rick+Cummings%2C+Eugene+Agichtein%2C+and+Evgeniy+Gabrilovich+%28eds%2E%29%2C+Proceedings+of+the+26th+Int)
139. ACM, 2017. doi: 10.1145/3038912.3052591. URL https://doi.org/10.1145/3038912.3052591. — [doi:10.1145/3038912.3052591](https://doi.org/10.1145/3038912.3052591) [↗](https://doi.org/10.1145/3038912.3052591)
140. Wenhan Xiong, Jiawei Wu, Hong Wang, Vivek Kulkarni, Mo Yu, Shiyu Chang, Xiaoxiao Guo, and William Yang Wang. TWEETQA: A social media focused question answering dataset. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pp. 5020–5031, Florence, Italy, 2019. A… [find](https://scholar.google.com/scholar?q=Wenhan+Xiong%2C+Jiawei+Wu%2C+Hong+Wang%2C+Vivek+Kulkarni%2C+Mo+Yu%2C+Shiyu+Chang%2C+Xiaoxiao+Guo%2C+and+William+Yang+Wang%2E+TWEETQA%3A+A+social+media+focused+question+answering+dataset%2E+In+Proceedings+of+the+57th+Annu)
141. Wei Xu, Courtney Napoles, Ellie Pavlick, Quanze Chen, and Chris Callison-Burch. [Optimizing statistical machine translation for text simplification](https://md.research.elamur.ai/free/en/papers/xu-2016-optimizing-statistical-machine-translation-text/). Transactions of the Association for Computational Linguistics, 4:401–415, 2016. URL [https://cocoxu.github.io/](https://cocoxu.github.io/) publications/tacl2016-smt-simplification.p… [↗](https://cocoxu.github.io/)
142. Zhilin Yang, Peng Qi, Saizheng Zhang, Yoshua Bengio, William W. Cohen, Ruslan Salakhutdinov, and Christopher D. Manning. [HotpotQA: A dataset for diverse, explainable multi-hop question answering](https://md.research.elamur.ai/free/en/papers/yang-2018-hotpotqa-dataset-diverse-explainable-multi/). In Conference on Empirical Methods in Natural Language Processing (EMNLP), 2018.
143. Wenpeng Yin, Dragomir Radev, and Caiming Xiong. [DocNLI: A large-scale dataset for document-level natural language inference](https://md.research.elamur.ai/free/en/papers/yin-2021-docnli-large-scale-dataset-document/). In Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, pp. 4913–4922, Online, 2021. Association for Computational Linguistics. doi: 10.18653/v1/2021.fi…
144. Mohamed Amir Yosef, Johannes Hoffart, Ilaria Bordino, Marc Spaniol, and Gerhard Weikum. Aida: An online tool for accurate disambiguation of named entities in text and tables. Proc. VLDB Endow., 4(12):1450–1453, August 2011. ISSN 2150-8097. doi: 10.14778/3402755.3402793. URL https://doi.org/10.14778/… — [doi:10.14778/3402755.3402793](https://doi.org/10.14778/3402755.3402793) [find](https://scholar.google.com/scholar?q=Mohamed+Amir+Yosef%2C+Johannes+Hoffart%2C+Ilaria+Bordino%2C+Marc+Spaniol%2C+and+Gerhard+Weikum%2E+Aida%3A+An+online+tool+for+accurate+disambiguation+of+named+entities+in+text+and+tables%2E+Proc%2E+VLDB+Endow%2E%2C+4%2812%29%3A)
145. Tianhe Yu, Saurabh Kumar, Abhishek Gupta, Sergey Levine, Karol Hausman, and Chelsea Finn. [Gradient surgery for multi-task learning](https://md.research.elamur.ai/free/en/papers/yu-2020-gradient-surgery-multi-task-learning/), 2020.
146. Marcos Zampieri, Shervin Malmasi, Preslav Nakov, Sara Rosenthal, Noura Farra, and Ritesh Kumar. [Semeval-2019 task 6: Identifying and categorizing offensive language in social media (offenseval)](https://md.research.elamur.ai/free/en/papers/zampieri-2019-semeval-2019-task-6-identifying/). In Proceedings of the 13th International Workshop on Semantic Evaluation, pp. 75–86, 2019.
147. Rowan Zellers, Ari Holtzman, Yonatan Bisk, Ali Farhadi, and Yejin Choi. HellaSwag: Can a machine really finish your sentence? In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pp. 4791–4800, Florence, Italy, July 2019. Association for Computational Linguisti… [find](https://scholar.google.com/scholar?q=Rowan+Zellers%2C+Ari+Holtzman%2C+Yonatan+Bisk%2C+Ali+Farhadi%2C+and+Yejin+Choi%2E+HellaSwag%3A+Can+a+machine+really+finish+your+sentence%3F+In+Proceedings+of+the+57th+Annual+Meeting+of+the+Association+for+Computati)
148. Xiaohua Zhai, Alexander Kolesnikov, Neil Houlsby, and Lucas Beyer. Scaling vision transformers. arXiv preprint arXiv:2106.04560, 2021. [find](https://scholar.google.com/scholar?q=Xiaohua+Zhai%2C+Alexander+Kolesnikov%2C+Neil+Houlsby%2C+and+Lucas+Beyer%2E+Scaling+vision+transformers%2E+arXiv+preprint+arXiv%3A2106%2E04560%2C+2021%2E)
149. Rui Zhang and Joel Tetreault. This email could save your life: Introducing the task of email subject line generation. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pp. 446–456, Florence, Italy, 2019. Association for Computational Linguistics. doi: 10.186… [find](https://scholar.google.com/scholar?q=Rui+Zhang+and+Joel+Tetreault%2E+This+email+could+save+your+life%3A+Introducing+the+task+of+email+subject+line+generation%2E+In+Proceedings+of+the+57th+Annual+Meeting+of+the+Association+for+Computational+Lin)
150. Sheng Zhang, Xiaodong Liu, Jingjing Liu, Jianfeng Gao, Kevin Duh, and Benjamin Van Durme. [ReCoRD: Bridging the gap between human and machine commonsense reading comprehension](https://md.research.elamur.ai/free/en/papers/zhang-2018-record-bridging-gap-between-human/). arXiv preprint 1810.12885, 2018.
151. Xiang Zhang, Junbo Jake Zhao, and Yann LeCun. [Character-level convolutional networks for text classification](https://md.research.elamur.ai/free/en/papers/zhang-2015-character-level-convolutional-networks-text/). In Corinna Cortes, Neil D. Lawrence, Daniel D. Lee, Masashi Sugiyama, and Roman Garnett (eds.), Advances in Neural Information Processing Systems 28: Annual Conference on Neural Information…
152. Yian Zhang, Alex Warstadt, Xiaocheng Li, and Samuel R. Bowman. When do you need billions of words of pretraining data? In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long… [find](https://scholar.google.com/scholar?q=Yian+Zhang%2C+Alex+Warstadt%2C+Xiaocheng+Li%2C+and+Samuel+R%2E+Bowman%2E+When+do+you+need+billions+of+words+of+pretraining+data%3F+In+Proceedings+of+the+59th+Annual+Meeting+of+the+Association+for+Computational+Li)
153. Yu Zhang and Dit-Yan Yeung. A convex formulation for learning task relationships in multi-task learning. In Proceedings of the Twenty-Sixth Conference on Uncertainty in Artificial Intelligence, UAI’10, pp. 733–742, Arlington, Virginia, USA, 2010. AUAI Press. ISBN 9780974903965. [find](https://scholar.google.com/scholar?q=Yu+Zhang+and+Dit%2DYan+Yeung%2E+A+convex+formulation+for+learning+task+relationships+in+multi%2Dtask+learning%2E+In+Proceedings+of+the+Twenty%2DSixth+Conference+on+Uncertainty+in+Artificial+Intelligence%2C+UAI%E2%80%9910)
154. Zhengyan Zhang, Xu Han, Zhiyuan Liu, Xin Jiang, Maosong Sun, and Qun Liu. Ernie: Enhanced language representation with informative entities. arXiv preprint arXiv:1905.07129, 2019. [find](https://scholar.google.com/scholar?q=Zhengyan+Zhang%2C+Xu+Han%2C+Zhiyuan+Liu%2C+Xin+Jiang%2C+Maosong+Sun%2C+and+Qun+Liu%2E+Ernie%3A+Enhanced+language+representation+with+informative+entities%2E+arXiv+preprint+arXiv%3A1905%2E07129%2C+2019%2E)

*Updated 2 November 2026*