---
title: "PEER — paper by Schick et al. (2022) — summary and ideas"
description: "The authors propose PEER (Plan, Edit, Explain, Repeat), a model that learns the writing process… 67 references, cited by 82 papers here."
authors: ["Timo Schick", "Jane Dwivedi-Yu", "Zhengbao Jiang", "Fabio Petroni", "Patrick Lewis", "Gautier Izacard", "Qingfei You", "Christoforos Nalmpantis", "Edouard Grave", "Sebastian Riedel"]
year: 2022
venue: "arXiv:2208.11663"
source: "https://arxiv.org/abs/2208.11663"
doi: "10.48550/arxiv.2208.11663"
arxiv: "2208.11663"
licence: "arXiv — non-exclusive licence to distribute"
licence_url: "http://arxiv.org/licenses/nonexclusive-distrib/1.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/schick-2022-peer-collaborative-language-model/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/schick-2022-peer-collaborative-language-model/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2022](https://md.research.elamur.ai/free/en/papers/2022/page/4/) / PEER: A Collaborative Language Model

# PEER: A Collaborative Language Model

**Authors:** [Timo Schick](https://md.research.elamur.ai/free/en/authors/timo-schick/), [Jane Dwivedi-Yu](https://md.research.elamur.ai/free/en/authors/jane-dwivedi-yu/), [Zhengbao Jiang](https://md.research.elamur.ai/free/en/authors/zhengbao-jiang/), [Fabio Petroni](https://md.research.elamur.ai/free/en/authors/fabio-petroni/), [Patrick Lewis](https://md.research.elamur.ai/free/en/authors/patrick-lewis/), [Gautier Izacard](https://md.research.elamur.ai/free/en/authors/gautier-izacard/), Qingfei You, [Christoforos Nalmpantis](https://md.research.elamur.ai/free/en/authors/christoforos-nalmpantis/), [Edouard Grave](https://md.research.elamur.ai/free/en/authors/edouard-grave/) and [Sebastian Riedel](https://md.research.elamur.ai/free/en/authors/sebastian-riedel/) · **Year:** 2022 · **Venue:** arXiv:2208.11663 · **Licence:** [arXiv — non-exclusive licence to distribute](http://arxiv.org/licenses/nonexclusive-distrib/1.0/)

**Links:** [Publisher's page](https://arxiv.org/abs/2208.11663) · [PDF](https://arxiv.org/pdf/2208.11663) · [DOI 10.48550/arxiv.2208.11663](https://doi.org/10.48550/arxiv.2208.11663) · [arXiv 2208.11663](https://arxiv.org/abs/2208.11663) · [OpenAlex](https://openalex.org/W4293138840)

**Citations:** 44 citations (OpenAlex, 2026-10-14) · [82](https://md.research.elamur.ai/free/en/papers/schick-2022-peer-collaborative-language-model/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
Conventional language models generate a finished text in one left-to-right pass. As a result, they cannot go back and revise their own output, are difficult to steer, and offer no justification for what they produce. All of this hampers human–machine co-writing. The authors propose PEER (Plan, Edit, Explain, Repeat), a model that learns the writing process rather than only its end product. In each round, a plan from the user or the model is applied to the current text as an edit. The edit can then be justified with a comment and with citations to retrieved background documents, and rounds continue until no further changes are needed. Training data comes mainly from Wikipedia revision histories, where editor comments stand in for plans and explanations. Because this data is noisy, Wikipedia-specific and often unsourced, the authors train extra PEER variants that reconstruct missing pieces and use their outputs for self-training. This extends PEER to domains without edit histories and strengthens instruction following, citing and explaining. PEER outperforms several baselines across editing tasks, and the models, data and code are released.

## Key points
- Four roles are trained:
  - **PEER-Edit** turns a text, a plan and documents into a revised text plus an explanation.
  - **PEER-Undo** recovers the earlier version of an edited text, which turns plain text into synthetic edit histories.
  - **PEER-Explain** writes plans or explanations for a given edit. Control sequences set their length and word overlap with the edit.
  - **PEER-Document** generates plausible supporting documents, though these are often repetitive.
- Training draws primarily on Wikipedia edit histories. Background passages are retrieved with the system of Petroni et al. (2022).
- On the Natural Edits evaluation, domain-adapted PEER variants beat regular PEER on every subset.
- In intro generation, PEER gets comparatively high QuestEval scores. Manual inspection, however, still finds false or unsupported claims.
- The authors note several limitations:
  - The retriever has access to the target text, which is unrealistic in practice.
  - Each edit rewrites the whole paragraph, which is inefficient and rules out full documents.
  - All evaluations are in English only.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Carnegie Mellon University](https://md.research.elamur.ai/free/en/universities/carnegie-mellon-university/) — University, Pittsburgh, United States · 598 works · 191,226 citations
- [University College London](https://md.research.elamur.ai/free/en/universities/university-college-london/) — University, London, United Kingdom · 107 works · 22,924 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 18,614 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 17,965 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 17,450 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 17,296 works

🧭 Topics (1): [🧠 Language models & pretraining](https://md.research.elamur.ai/free/en/topics/language-models/) · 3,894 works

## Cite

Cite as: Timo Schick, Jane Dwivedi-Yu, Zhengbao Jiang, Fabio Petroni, Patrick Lewis, Gautier Izacard, Qingfei You, Christoforos Nalmpantis, Edouard Grave and Sebastian Riedel. 2022. PEER: A Collaborative Language Model. arXiv:2208.11663.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@misc{schick2022peer,
  title = {{PEER: A Collaborative Language Model}},
  author = {Timo Schick, Jane Dwivedi-Yu, Zhengbao Jiang, Fabio Petroni, Patrick Lewis, Gautier Izacard, Qingfei You, Christoforos Nalmpantis, Edouard Grave and Sebastian Riedel},
  year = {2022},
  doi = {10.48550/arxiv.2208.11663},
  eprint = {2208.11663},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2208.11663}
}
```

- [← previous XLM-E: Cross-lingual Language Model Pre-training via ELECTRA](https://md.research.elamur.ai/free/en/papers/chi-2022-xlm-e-cross-lingual-language/)
- [next → A Systematic Evaluation of Large Language Models of Code](https://md.research.elamur.ai/free/en/papers/xu-2022-systematic-evaluation-large-language-models/)
- [↑ back to papers of 2022](https://md.research.elamur.ai/free/en/papers/2022/page/4/)

## References

67 references, 38 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Fernando Alva-Manchego, Louis Martin, Antoine Bordes, Carolina Scarton, Benoît Sagot, and Lucia Specia. 2020. [Asset: A dataset for tuning and evaluation of sentence simplification models with multiple rewriting transformations](https://md.research.elamur.ai/free/en/papers/manchego-2020-asset-dataset-tuning-evaluation-sentence/). In Proceedings of the 58th Annual Meeting of the Association for Computa…
2. Fernando Alva-Manchego, Louis Martin, Carolina Scarton, and Lucia Specia. 2019. [EASSE: Easier automatic sentence simplification evaluation](https://md.research.elamur.ai/free/en/papers/manchego-2019-easse-easier-automatic-sentence-simplification/). In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Process…
3. Abhijeet Awasthi, Sunita Sarawagi, Rasna Goyal, Sabyasachi Ghosh, and Vihari Piratla. 2019. [Parallel iterative edit models for local sequence transduction](https://md.research.elamur.ai/free/en/papers/awasthi-2019-parallel-iterative-edit-models-local/). In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural…
4. Stephen Bach, Victor Sanh, Zheng Xin Yong, Albert Webson, Colin Raffel, Nihal V. Nayak, Abheesht Sharma, Taewoon Kim, M Saiful Bari, Thibault Fevry, Zaid Alyafeai, Manan Dey, Andrea Santilli, Zhiqing Sun, Srulik Ben-david, Canwen Xu, Gunjan Chhablani, Han Wang, Jason Fries, Maged Al-shaibani, Shanya… [find](https://scholar.google.com/scholar?q=Stephen+Bach%2C+Victor+Sanh%2C+Zheng+Xin+Yong%2C+Albert+Webson%2C+Colin+Raffel%2C+Nihal+V%2E+Nayak%2C+Abheesht+Sharma%2C+Taewoon+Kim%2C+M+Saiful+Bari%2C+Thibault+Fevry%2C+Zaid+Alyafeai%2C+Manan+Dey%2C+Andrea+Santilli%2C+Zhiqing+)
5. S Borgeaud, A Mensch, J Hoffmann, T Cai, E Rutherford, and K Millican. 2021. Improving language models by retrieving from trillions of tokens. arXiv preprint arXiv:2112.04426. [find](https://scholar.google.com/scholar?q=S+Borgeaud%2C+A+Mensch%2C+J+Hoffmann%2C+T+Cai%2C+E+Rutherford%2C+and+K+Millican%2E+2021%2E+Improving+language+models+by+retrieving+from+trillions+of+tokens%2E+arXiv+preprint+arXiv%3A2112%2E04426%2E)
6. Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel Ziegler, Jeffrey Wu, Clemens Winter, Chri… — [Language Models are Few-Shot Learners](https://md.research.elamur.ai/free/en/papers/brown-2020-language-models-few-shot-learners/)
7. Nicola De Cao, Gautier Izacard, Sebastian Riedel, and Fabio Petroni. 2021. Autoregressive entity retrieval. In International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Nicola+De+Cao%2C+Gautier+Izacard%2C+Sebastian+Riedel%2C+and+Fabio+Petroni%2E+2021%2E+Autoregressive+entity+retrieval%2E+In+International+Conference+on+Learning+Representations%2E)
8. Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung, Charles Sutton, Sebastian Gehrmann, Parker Schuh, Kensen Shi, Sasha Tsvyashchenko, Joshua Maynez, Abhishek Rao, Parker Barnes, Yi Tay, Noam Shazeer, Vinodkumar Prabhakaran, Emi… — [PaLM: Scaling Language Modeling with Pathways](https://md.research.elamur.ai/free/en/papers/chowdhery-2022-palm-scaling-language-modeling-pathways/)
9. Zhuyun Dai, Arun Tejasvi Chaganty, Vincent Y Zhao, Aida Amini, Qazi Mamunur Rashid, Mike Green, and Kelvin Guu. 2022. Dialog inpainting: Turning documents into dialogs. In International Conference on Machine Learning, pages 4558–4586. PMLR. [find](https://scholar.google.com/scholar?q=Zhuyun+Dai%2C+Arun+Tejasvi+Chaganty%2C+Vincent+Y+Zhao%2C+Aida+Amini%2C+Qazi+Mamunur+Rashid%2C+Mike+Green%2C+and+Kelvin+Guu%2E+2022%2E+Dialog+inpainting%3A+Turning+documents+into+dialogs%2E+In+International+Conference+on+)
10. David Dohan, Winnie Xu, Aitor Lewkowycz, Jacob Austin, David Bieber, Raphael Gontijo Lopes, Yuhuai Wu, Henryk Michalewski, Rif A Saurous, Jascha Sohl-dickstein, et al. 2022. [Language model cascades](https://md.research.elamur.ai/free/en/papers/dohan-2022-language-model-cascades/). arXiv preprint arXiv:2207.10342.
11. Chris Donahue, Mina Lee, and Percy Liang. 2020. [Enabling language models to fill in the blanks](https://md.research.elamur.ai/free/en/papers/donahue-2020-enabling-language-models-fill-blanks/). In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 2492– 2501, Online. Association for Computational Linguistics.
12. Wanyu Du, Zae Myung Kim, Vipul Raheja, Dhruv Kumar, and Dongyeop Kang. 2022a. Read, revise, repeat: A system demonstration for human-in-the-loop iterative text revision. In Proceedings of the First Workshop on Intelligent and Interactive Writing Assistants (In2Writing 2022), pages 96–108, Dublin, Ir… [find](https://scholar.google.com/scholar?q=Wanyu+Du%2C+Zae+Myung+Kim%2C+Vipul+Raheja%2C+Dhruv+Kumar%2C+and+Dongyeop+Kang%2E+2022a%2E+Read%2C+revise%2C+repeat%3A+A+system+demonstration+for+human%2Din%2Dthe%2Dloop+iterative+text+revision%2E+In+Proceedings+of+the+First+Wo)
13. Wanyu Du, Vipul Raheja, Dhruv Kumar, Zae Myung Kim, Melissa Lopez, and Dongyeop Kang. 2022b. Understanding iterative revision from human-written text. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 3573–3590. [find](https://scholar.google.com/scholar?q=Wanyu+Du%2C+Vipul+Raheja%2C+Dhruv+Kumar%2C+Zae+Myung+Kim%2C+Melissa+Lopez%2C+and+Dongyeop+Kang%2E+2022b%2E+Understanding+iterative+revision+from+human%2Dwritten+text%2E+In+Proceedings+of+the+60th+Annual+Meeting+of+the+)
14. Felix Faltings, Michel Galley, Gerold Hintz, Chris Brockett, Chris Quirk, Jianfeng Gao, and Bill Dolan. 2021. Text editing by command. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 5259–5274, O… [find](https://scholar.google.com/scholar?q=Felix+Faltings%2C+Michel+Galley%2C+Gerold+Hintz%2C+Chris+Brockett%2C+Chris+Quirk%2C+Jianfeng+Gao%2C+and+Bill+Dolan%2E+2021%2E+Text+editing+by+command%2E+In+Proceedings+of+the+2021+Conference+of+the+North+American+Chapt)
15. Tao Ge, Furu Wei, and Ming Zhou. 2018. Reaching human-level performance in automatic grammatical error correction: An empirical study. arXiv preprint arXiv:1807.01270. [find](https://scholar.google.com/scholar?q=Tao+Ge%2C+Furu+Wei%2C+and+Ming+Zhou%2E+2018%2E+Reaching+human%2Dlevel+performance+in+automatic+grammatical+error+correction%3A+An+empirical+study%2E+arXiv+preprint+arXiv%3A1807%2E01270%2E)
16. Kelvin Guu, Kenton Lee, Zora Tung, Panupong Pasupat, and Ming-Wei Chang. 2020. [Realm: Retrieval-augmented language model pre-training](https://md.research.elamur.ai/free/en/papers/guu-2020-realm-retrieval-augmented-language-model/). In Proceedings of the 37th International Conference on Machine Learning, ICML’20. JMLR.org.
17. Junxian He, Jiatao Gu, Jiajun Shen, and Marc’Aurelio Ranzato. 2020a. Revisiting self-training for neural sequence generation. In International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Junxian+He%2C+Jiatao+Gu%2C+Jiajun+Shen%2C+and+Marc%E2%80%99Aurelio+Ranzato%2E+2020a%2E+Revisiting+self%2Dtraining+for+neural+sequence+generation%2E+In+International+Conference+on+Learning+Representations%2E)
18. Junxian He, Wojciech Kryściński, Bryan McCann, Nazneen Rajani, and Caiming Xiong. 2020b. [Ctrlsum: Towards generic controllable text summarization](https://md.research.elamur.ai/free/en/papers/he-2022-ctrlsum-generic-controllable-text-summarization/). arXiv preprint arXiv:2012.04281.
19. Vu Cong Duy Hoang, Philipp Koehn, Gholamreza Haffari, and Trevor Cohn. 2018. [Iterative back-translation for neural machine translation](https://md.research.elamur.ai/free/en/papers/hoang-2018-iterative-back-translation-neural-machine/). In Proceedings of the 2nd Workshop on Neural Machine Translation and Generation, pages 18–24, Melbourne, Australia. Association for Computational Linguistics.
20. Ari Holtzman, Jan Buys, Li Du, Maxwell Forbes, and Yejin Choi. 2020. [The curious case of neural text degeneration](https://md.research.elamur.ai/free/en/papers/holtzman-2019-curious-case-neural-text-degeneration/). In International Conference on Learning Representations.
21. Vladimir Karpukhin, Barlas Oğuz, Sewon Min, Patrick Lewis, Ledell Wu, Sergey Edunov, Danqi Chen, and Wen-tau Yih. 2020. [Dense passage retrieval for open-domain question answering](https://md.research.elamur.ai/free/en/papers/karpukhin-2020-dense-passage-retrieval-open-domain/). arXiv preprint arXiv:2004.04906.
22. Nitish Shirish Keskar, Bryan McCann, Lav R Varshney, Caiming Xiong, and Richard Socher. 2019. [Ctrl: A conditional transformer language model for controllable generation](https://md.research.elamur.ai/free/en/papers/keskar-2019-ctrl-conditional-transformer-language-model/). arXiv preprint arXiv:1909.05858.
23. Tomasz Korbak, Hady Elsahar, German Kruszewski, and Marc Dymetman. 2022. Controlling conditional language models without catastrophic forgetting. In Proceedings of the 39th International Conference on Machine Learning, volume 162 of Proceedings of Machine Learning Research, pages 11499–11528. PMLR. [find](https://scholar.google.com/scholar?q=Tomasz+Korbak%2C+Hady+Elsahar%2C+German+Kruszewski%2C+and+Marc+Dymetman%2E+2022%2E+Controlling+conditional+language+models+without+catastrophic+forgetting%2E+In+Proceedings+of+the+39th+International+Conference+on)
24. Mina Lee, Percy Liang, and Qian Yang. 2022. [Coauthor: Designing a human-ai collaborative writing dataset for exploring language model capabilities](https://md.research.elamur.ai/free/en/papers/lee-2022-coauthor-designing-human-ai-collaborative/). In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems, CHI ’22, New York, NY, USA. Association for Computing Machinery.
25. Xiang Lisa Li, John Thickstun, Ishaan Gulrajani, Percy Liang, and Tatsunori B Hashimoto. 2022. [Diffusion-lm improves controllable text generation](https://md.research.elamur.ai/free/en/papers/li-2022-diffusion-lm-improves-controllable-text/). arXiv preprint arXiv:2205.14217.
26. Chin-Yew Lin. 2004. [Rouge: A package for automatic evaluation of summaries](https://md.research.elamur.ai/free/en/papers/lin-2004-rouge-package-automatic-evaluation-summaries/). In Text summarization branches out, pages 74–81.
27. Hui Liu, Qingyu Yin, and William Yang Wang. 2019. Towards explainable NLP: A generative explanation framework for text classification. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 5570–5581, Florence, Italy. Association for Computational Linguisti… [find](https://scholar.google.com/scholar?q=Hui+Liu%2C+Qingyu+Yin%2C+and+William+Yang+Wang%2E+2019%2E+Towards+explainable+NLP%3A+A+generative+explanation+framework+for+text+classification%2E+In+Proceedings+of+the+57th+Annual+Meeting+of+the+Association+for+)
28. Robert L Logan IV, Alexandre Passos, Sameer Singh, and Ming-Wei Chang. 2021. Fruit: Faithfully reflecting updated information in text. arXiv preprint arXiv:2112.08634. [find](https://scholar.google.com/scholar?q=Robert+L+Logan+IV%2C+Alexandre+Passos%2C+Sameer+Singh%2C+and+Ming%2DWei+Chang%2E+2021%2E+Fruit%3A+Faithfully+reflecting+updated+information+in+text%2E+arXiv+preprint+arXiv%3A2112%2E08634%2E)
29. Jonathan Mallinson, Aliaksei Severyn, Eric Malmi, and Guillermo Garrido. 2020. FELIX: Flexible text editing through tagging and insertion. In Findings of the Association for Computational Linguistics: EMNLP 2020, pages 1244–1255, Online. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Jonathan+Mallinson%2C+Aliaksei+Severyn%2C+Eric+Malmi%2C+and+Guillermo+Garrido%2E+2020%2E+FELIX%3A+Flexible+text+editing+through+tagging+and+insertion%2E+In+Findings+of+the+Association+for+Computational+Linguistics%3A)
30. Louis Martin, Angela Fan, Éric de la Clergerie, Antoine Bordes, and Benoît Sagot. 2020. [Muss: multilingual unsupervised sentence simplification by mining paraphrases](https://md.research.elamur.ai/free/en/papers/martin-2022-muss-multilingual-unsupervised-sentence-simplification/). arXiv preprint arXiv:2005.00352.
31. Joshua Maynez, Shashi Narayan, Bernd Bohnet, and Ryan McDonald. 2020. [On faithfulness and factuality in abstractive summarization](https://md.research.elamur.ai/free/en/papers/maynez-2020-faithfulness-factuality-abstractive-summarization/). In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 1906–1919, Online. Association for Computational Linguistics.
32. Amit Moryossef, Yoav Goldberg, and Ido Dagan. 2019. [Step-by-step: Separating planning from realization in neural data-to-text generation](https://md.research.elamur.ai/free/en/papers/moryossef-2019-step-step-separating-planning-realization/). In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long…
33. Kenton Murray and David Chiang. 2018. [Correcting length bias in neural machine translation](https://md.research.elamur.ai/free/en/papers/murray-2018-correcting-length-bias-neural-machine/). In Proceedings of the Third Conference on Machine Translation: Research Papers, pages 212–223, Brussels, Belgium. Association for Computational Linguistics.
34. Reiichiro Nakano, Jacob Hilton, Suchir Balaji, Jeff Wu, Long Ouyang, Christina Kim, Christopher Hesse, Shantanu Jain, Vineet Kosaraju, William Saunders, et al. 2021. [Webgpt: Browser-assisted question-answering with human feedback](https://md.research.elamur.ai/free/en/papers/nakano-2021-webgpt-browser-assisted-question-answering/). arXiv preprint arXiv:2112.09332.
35. Courtney Napoles, Keisuke Sakaguchi, Matt Post, and Joel Tetreault. 2015. Ground truth for grammatical error correction metrics. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volum… [find](https://scholar.google.com/scholar?q=Courtney+Napoles%2C+Keisuke+Sakaguchi%2C+Matt+Post%2C+and+Joel+Tetreault%2E+2015%2E+Ground+truth+for+grammatical+error+correction+metrics%2E+In+Proceedings+of+the+53rd+Annual+Meeting+of+the+Association+for+Comput)
36. Courtney Napoles, Keisuke Sakaguchi, and Joel Tetreault. 2017. Jfleg: A fluency corpus and benchmark for grammatical error correction. In Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 2, Short Papers, pages 229–234. [find](https://scholar.google.com/scholar?q=Courtney+Napoles%2C+Keisuke+Sakaguchi%2C+and+Joel+Tetreault%2E+2017%2E+Jfleg%3A+A+fluency+corpus+and+benchmark+for+grammatical+error+correction%2E+In+Proceedings+of+the+15th+Conference+of+the+European+Chapter+of+)
37. Shashi Narayan, Yao Zhao, Joshua Maynez, Gonçalo Simões, Vitaly Nikolaev, and Ryan McDonald. 2021. [Planning with Learned Entity Prompts for Abstractive Summarization](https://md.research.elamur.ai/free/en/papers/narayan-2021-planning-learned-entity-prompts-abstractive/). Transactions of the Association for Computational Linguistics, 9:1475–1492.
38. Long Ouyang, Jeff Wu, Xu Jiang, Diogo Almeida, Carroll L Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. 2022. [Training language models to follow instructions with human feedback](https://md.research.elamur.ai/free/en/papers/ouyang-2022-training-language-models-follow-instructions/). arXiv preprint arXiv:2203.02155.
39. Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. [Bleu: a method for automatic evaluation of machine translation](https://md.research.elamur.ai/free/en/papers/papineni-2002-bleu-method-automatic-evaluation-machine/). In Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, pages 311–318, Philadelphia, Pennsylvania, USA. Association for Computational…
40. Fabio Petroni, Samuel Broscheit, Aleksandra Piktus, Patrick Lewis, Gautier Izacard, Lucas Hosseini, Jane Dwivedi-Yu, Maria Lomeli, Timo Schick, Pierre-Emmanuel Mazaré, Armand Joulin, Edouard Grave, and Sebastian Riedel. 2022. Improving wikipedia verifiability with ai. [find](https://scholar.google.com/scholar?q=Fabio+Petroni%2C+Samuel+Broscheit%2C+Aleksandra+Piktus%2C+Patrick+Lewis%2C+Gautier+Izacard%2C+Lucas+Hosseini%2C+Jane+Dwivedi%2DYu%2C+Maria+Lomeli%2C+Timo+Schick%2C+Pierre%2DEmmanuel+Mazar%C3%A9%2C+Armand+Joulin%2C+Edouard+Grave%2C+an)
41. Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Dmytro Okhonko, Samuel Broscheit, Gautier Izacard, Patrick Lewis, Barlas Oğuz, Edouard Grave, Wen-tau Yih, et al. 2021. The web is your oyster– knowledge-intensive nlp against a very large web corpus. arXiv preprint arXiv:2112.09924. — [arXiv:2112.09924](https://arxiv.org/abs/2112.09924) [find](https://scholar.google.com/scholar?q=Aleksandra+Piktus%2C+Fabio+Petroni%2C+Vladimir+Karpukhin%2C+Dmytro+Okhonko%2C+Samuel+Broscheit%2C+Gautier+Izacard%2C+Patrick+Lewis%2C+Barlas+Og%CC%86uz%2C+Edouard+Grave%2C+Wen%2Dtau+Yih%2C+et+al%2E+2021%2E+The+web+is+your+oyster%E2%80%93+k)
42. Martin Potthast, Benno Stein, and Robert Gerling. 2008. Automatic vandalism detection in wikipedia. In European conference on information retrieval, pages 663–668. Springer. [find](https://scholar.google.com/scholar?q=Martin+Potthast%2C+Benno+Stein%2C+and+Robert+Gerling%2E+2008%2E+Automatic+vandalism+detection+in+wikipedia%2E+In+European+conference+on+information+retrieval%2C+pages+663%E2%80%93668%2E+Springer%2E)
43. Reid Pryzant, Richard Diehl Martinez, Nathan Dass, Sadao Kurohashi, Dan Jurafsky, and Diyi Yang. 2020. Automatically neutralizing subjective bias in text. In Proceedings of the aaai conference on artificial intelligence, volume 34, pages 480–489. [find](https://scholar.google.com/scholar?q=Reid+Pryzant%2C+Richard+Diehl+Martinez%2C+Nathan+Dass%2C+Sadao+Kurohashi%2C+Dan+Jurafsky%2C+and+Diyi+Yang%2E+2020%2E+Automatically+neutralizing+subjective+bias+in+text%2E+In+Proceedings+of+the+aaai+conference+on+arti)
44. Alec Radford, Jeff Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. 2019. Language models are unsupervised multitask learners. Technical report, Open AI. [find](https://scholar.google.com/scholar?q=Alec+Radford%2C+Jeff+Wu%2C+Rewon+Child%2C+David+Luan%2C+Dario+Amodei%2C+and+Ilya+Sutskever%2E+2019%2E+Language+models+are+unsupervised+multitask+learners%2E+Technical+report%2C+Open+AI%2E)
45. Jack W. Rae, Sebastian Borgeaud, Trevor Cai, Katie Millican, Jordan Hoffmann, Francis Song, John Aslanides, Sarah Henderson, Roman Ring, Susannah Young, Eliza Rutherford, Tom Hennigan, Jacob Menick, Albin Cassirer, Richard Powell, George van den Driessche, Lisa Anne Hendricks, Maribeth Rauh, Po-Sen… [find](https://scholar.google.com/scholar?q=Jack+W%2E+Rae%2C+Sebastian+Borgeaud%2C+Trevor+Cai%2C+Katie+Millican%2C+Jordan+Hoffmann%2C+Francis+Song%2C+John+Aslanides%2C+Sarah+Henderson%2C+Roman+Ring%2C+Susannah+Young%2C+Eliza+Rutherford%2C+Tom+Hennigan%2C+Jacob+Menick%2C+A)
46. Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. 2020. [Exploring the limits of transfer learning with a unified text-to-text transformer](https://md.research.elamur.ai/free/en/papers/raffel-2019-exploring-limits-transfer-learning-unified/). Journal of Machine Learning Research, 21(140):1–67.
47. Jeff Rasley, Samyam Rajbhandari, Olatunji Ruwase, and Yuxiong He. 2020. Deepspeed: System optimizations enable training deep learning models with over 100 billion parameters. In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, KDD ’20, page 3505–3506,… [find](https://scholar.google.com/scholar?q=Jeff+Rasley%2C+Samyam+Rajbhandari%2C+Olatunji+Ruwase%2C+and+Yuxiong+He%2E+2020%2E+Deepspeed%3A+System+optimizations+enable+training+deep+learning+models+with+over+100+billion+parameters%2E+In+Proceedings+of+the+26t)
48. Machel Reid and Graham Neubig. 2022. Learning to model editing processes. [find](https://scholar.google.com/scholar?q=Machel+Reid+and+Graham+Neubig%2E+2022%2E+Learning+to+model+editing+processes%2E)
49. Stephen E Robertson, Steve Walker, Susan Jones, Micheline M Hancock-Beaulieu, Mike Gatford, et al. 1995. [Okapi at trec-3](https://md.research.elamur.ai/free/en/papers/robertson-1994-okapi-trec-3/). Nist Special Publication Sp, 109:109.
50. Victor Sanh, Albert Webson, Colin Raffel, Stephen Bach, Lintang Sutawika, Zaid Alyafeai, Antoine Chaffin, Arnaud Stiegler, Arun Raja, Manan Dey, M Saiful Bari, Canwen Xu, Urmish Thakker, Shanya Sharma Sharma, Eliza Szczechla, Taewoon Kim, Gunjan Chhablani, Nihal Nayak, Debajyoti Datta, Jonathan Chan… — [Multitask Prompted Training Enables Zero-Shot Task Generalization](https://md.research.elamur.ai/free/en/papers/sanh-2021-multitask-prompted-training-enables-zero/)
51. Timo Schick and Hinrich Schütze. 2021a. [Exploiting cloze-questions for few-shot text classification and natural language inference](https://md.research.elamur.ai/free/en/papers/schick-2021-exploiting-cloze-questions-few-shot/). In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume, pages 255–269, Online. Association for Comp…
52. Timo Schick and Hinrich Schütze. 2021b. [Few-shot text generation with natural language instructions](https://md.research.elamur.ai/free/en/papers/schick-2021-few-shot-text-generation-natural/). In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 390–402, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics.
53. Thomas Scialom, Paul-Alexis Dray, Patrick Gallinari, Sylvain Lamprier, Benjamin Piwowarski, Jacopo Staiano, and Alex Wang. 2021. [Questeval: Summarization asks for fact-based evaluation](https://md.research.elamur.ai/free/en/papers/scialom-2021-questeval-summarization-asks-fact-based/). arXiv preprint arXiv:2103.12693.
54. Rico Sennrich, Barry Haddow, and Alexandra Birch. 2016. Improving neural machine translation models with monolingual data. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 86–96, Berlin, Germany. Association for Computational L… [find](https://scholar.google.com/scholar?q=Rico+Sennrich%2C+Barry+Haddow%2C+and+Alexandra+Birch%2E+2016%2E+Improving+neural+machine+translation+models+with+monolingual+data%2E+In+Proceedings+of+the+54th+Annual+Meeting+of+the+Association+for+Computationa)
55. Tianxiao Shen, Victor Quach, Regina Barzilay, and Tommi Jaakkola. 2020. Blank language models. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 5186–5198, Online. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Tianxiao+Shen%2C+Victor+Quach%2C+Regina+Barzilay%2C+and+Tommi+Jaakkola%2E+2020%2E+Blank+language+models%2E+In+Proceedings+of+the+2020+Conference+on+Empirical+Methods+in+Natural+Language+Processing+%28EMNLP%29%2C+pages+)
56. Kurt Shuster, Spencer Poff, Moya Chen, Douwe Kiela, and Jason Weston. 2021. [Retrieval augmentation reduces hallucination in conversation](https://md.research.elamur.ai/free/en/papers/shuster-2021-retrieval-augmentation-reduces-hallucination-conversation/). In Findings of the Association for Computational Linguistics: EMNLP 2021, pages 3784–3803, Punta Cana, Dominican Republic. Association for Computational Linguisti…
57. Yizhong Wang, Swaroop Mishra, Pegah Alipoormolabashi, Yeganeh Kordi, Amirreza Mirzaei, Anjana Arunkumar, Arjun Ashok, Arut Selvan Dhanasekaran, Atharva Naik, David Stap, et al. 2022. Benchmarking generalization via in-context instructions on 1,600+ language tasks. arXiv preprint arXiv:2204.07705. [find](https://scholar.google.com/scholar?q=Yizhong+Wang%2C+Swaroop+Mishra%2C+Pegah+Alipoormolabashi%2C+Yeganeh+Kordi%2C+Amirreza+Mirzaei%2C+Anjana+Arunkumar%2C+Arjun+Ashok%2C+Arut+Selvan+Dhanasekaran%2C+Atharva+Naik%2C+David+Stap%2C+et+al%2E+2022%2E+Benchmarking+gene)
58. Jason Wei, Maarten Bosma, Vincent Zhao, Kelvin Guu, Adams Wei Yu, Brian Lester, Nan Du, Andrew M. Dai, and Quoc V Le. 2022a. [Finetuned language models are zero-shot learners](https://md.research.elamur.ai/free/en/papers/wei-2021-finetuned-language-models-zero-shot/). In International Conference on Learning Representations.
59. Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Ed Chi, Quoc Le, and Denny Zhou. 2022b. [Chain of thought prompting elicits reasoning in large language models](https://md.research.elamur.ai/free/en/papers/wei-2022-chain-thought-prompting-elicits-reasoning/). arXiv preprint arXiv:2201.11903.
60. Guillaume Wenzek, Marie-Anne Lachaux, Alexis Conneau, Vishrav Chaudhary, Francisco Guzmán, Armand Joulin, and Edouard Grave. 2020. [CCNet: Extracting high quality monolingual datasets from web crawl data](https://md.research.elamur.ai/free/en/papers/wenzek-2020-ccnet-extracting-high-quality-monolingual/). In Proceedings of the 12th Language Resources and Evaluation Conference, pages 4003–4012, Marsei…
61. Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pier-ric Cistac, Tim Rault, Remi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Mariama Drame, Quentin Lho… — [Transformers: State-of-the-Art Natural Language Processing](https://md.research.elamur.ai/free/en/papers/wolf-2020-transformers-state-art-natural-language/)
62. Zeqiu Wu, Michel Galley, Chris Brockett, Yizhe Zhang, and Bill Dolan. 2021. Automatic document sketching: Generating drafts from analogous texts. In Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, pages 2102–2113, Online. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Zeqiu+Wu%2C+Michel+Galley%2C+Chris+Brockett%2C+Yizhe+Zhang%2C+and+Bill+Dolan%2E+2021%2E+Automatic+document+sketching%3A+Generating+drafts+from+analogous+texts%2E+In+Findings+of+the+Association+for+Computational+Lingu)
63. Wei Xu, Courtney Napoles, Ellie Pavlick, Quanze Chen, and Chris Callison-Burch. 2016. [Optimizing statistical machine translation for text simplification](https://md.research.elamur.ai/free/en/papers/xu-2016-optimizing-statistical-machine-translation-text/). Transactions of the Association for Computational Linguistics, 4:401–415.
64. Diyi Yang, Aaron Halfaker, Robert Kraut, and Eduard Hovy. 2017. Identifying semantic edit intentions from revisions in Wikipedia. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 2000–2010, Copenhagen, Denmark. Association for Computational Linguistics… [find](https://scholar.google.com/scholar?q=Diyi+Yang%2C+Aaron+Halfaker%2C+Robert+Kraut%2C+and+Eduard+Hovy%2E+2017%2E+Identifying+semantic+edit+intentions+from+revisions+in+Wikipedia%2E+In+Proceedings+of+the+2017+Conference+on+Empirical+Methods+in+Natural+)
65. Lili Yao, Nanyun Peng, Ralph Weischedel, Kevin Knight, Dongyan Zhao, and Rui Yan. 2019. Plan-and-write: Towards better automatic storytelling. Proceedings of the AAAI Conference on Artificial Intelligence, 33(01):7378–7385. [find](https://scholar.google.com/scholar?q=Lili+Yao%2C+Nanyun+Peng%2C+Ralph+Weischedel%2C+Kevin+Knight%2C+Dongyan+Zhao%2C+and+Rui+Yan%2E+2019%2E+Plan%2Dand%2Dwrite%3A+Towards+better+automatic+storytelling%2E+Proceedings+of+the+AAAI+Conference+on+Artificial+Intellig)
66. David Yarowsky. 1995. [Unsupervised word sense disambiguation rivaling supervised methods](https://md.research.elamur.ai/free/en/papers/yarowsky-1995-unsupervised-word-sense-disambiguation-rivaling/). In 33rd Annual Meeting of the Association for Computational Linguistics, pages 189–196, Cambridge, Massachusetts, USA. Association for Computational Linguistics.
67. Susan Zhang, Stephen Roller, Naman Goyal, Mikel Artetxe, Moya Chen, Shuohui Chen, Christopher Dewan, Mona Diab, Xian Li, Xi Victoria Lin, Todor Mihaylov, Myle Ott, Sam Shleifer, Kurt Shuster, Daniel Simig, Punit Singh Koura, Anjali Sridhar, Tianlu Wang, and Luke Zettlemoyer. 2022. [Opt: Open pre-trai…](https://md.research.elamur.ai/free/en/papers/zhang-2022-opt-open-pre-trained-transformer/)

*Updated 2 November 2026*