---
title: "Interpretability for Language Learners Using — Kaneko 2022"
description: "Neural grammatical error correction (GEC) systems have become quite accurate, but they rarely tell learners why a correction was made."
authors: ["Masahiro Kaneko", "Sho Takase", "Ayana Niwa", "Naoaki Okazaki"]
year: 2022
venue: "Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)"
source: "https://aclanthology.org/2022.acl-long.496/"
doi: "10.18653/v1/2022.acl-long.496"
arxiv: "2203.07085"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/papers/kaneko-2022-interpretability-language-learners-example-based/"
---
[Elamur Research](https://md.research.elamur.ai/) · [this page in HTML](https://research.elamur.ai/free/papers/kaneko-2022-interpretability-language-learners-example-based/)

[home](https://md.research.elamur.ai/) / [papers](https://md.research.elamur.ai/free/papers/) / [2022](https://md.research.elamur.ai/free/papers/2022/page/2/) / Interpretability for Language Learners Using Example-Based Grammatical Error Correction

# Interpretability for Language Learners Using Example-Based Grammatical Error Correction

**Authors:** Masahiro Kaneko, Sho Takase, Ayana Niwa, [Naoaki Okazaki](https://md.research.elamur.ai/free/authors/naoaki-okazaki/) · **Year:** 2022 · **Venue:** Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2022.acl-long.496/) · [PDF](https://aclanthology.org/2022.acl-long.496.pdf) · [DOI 10.18653/v1/2022.acl-long.496](https://doi.org/10.18653/v1/2022.acl-long.496) · [arXiv 2203.07085](https://arxiv.org/abs/2203.07085) · [OpenAlex](https://openalex.org/W4226366719)

**Citations:** 20 citations (openalex-snapshot · 2026-10-14) · 27 in our library, of the 140673 papers whose bibliographies we read

## Summary

Neural grammatical error correction (GEC) systems have become quite accurate, but they rarely tell learners why a correction was made. This leaves learners unsure whether to trust a suggestion. Kaneko et al. propose EB-GEC, which justifies each edit with a retrieved pair of incorrect and corrected sentences. It builds on kNN-MT. Decoder hidden states from the training data serve as keys, and the values are the next token plus its source and target sentences. At inference, the model mixes the usual softmax output with a distribution built from the nearest stored neighbours. The closest neighbour behind each edited token is shown to the learner as supporting evidence.

The approach works with any trained autoregressive encoder-decoder and needs no extra retrieval training. Compared with vanilla GEC, EB-GEC was more accurate on three test sets and on par on the fourth. In human evaluation, its examples were judged significantly more useful than those from earlier surface-matching retrieval baselines. The authors present this as the first evidence that retrieved examples help actual users, not just task accuracy.

## Key points

- **Training data and datastore:** BEA-2019 data (W&I-train, NUCLE, FCE-train, Lang-8) was used, with identical source/target pairs removed. This left about 0.6M sentence pairs, which also form the datastore. W&I-dev served as development data.
- **Evaluation:** Tested on W&I-test and FCE-test (ERRANT), CoNLL2014 (M²), and JFLEG-test (GLEU).
- **Setup:** Transformer-big with an 8,000-subword BPE vocabulary and beam width 5. Settings were λ = 0.5, k = 16, and temperature T = 1,000. Faiss handled the nearest-neighbour search, and results are averaged over five random seeds.
- **Edit alignment:** Edits were located with Gestalt pattern matching, and one nearest example is displayed per edited token.
- **Comparison and code:** The baseline example-retrieval methods were taken from the work of Arai et al. (2020), Yen et al. (2015), and Matsubara et al. (2008). Code is available at github.com/kanekomasahiro/eb-gec.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

Fields: [Linguistics & languages](https://md.research.elamur.ai/free/shelves/linguistics/)

## Cite

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{kaneko2022interpretability,
  title = {{Interpretability for Language Learners Using Example-Based Grammatical Error Correction}},
  author = {Kaneko, Masahiro and Takase, Sho and Niwa, Ayana and Okazaki, Naoaki},
  booktitle = {Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2022},
  doi = {10.18653/v1/2022.acl-long.496},
  eprint = {2203.07085},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.18653/v1/2022.acl-long.496}
}
```

- [← previous Internet-augmented language models through few-shot prompting for open-domain question answering](https://md.research.elamur.ai/free/papers/lazaridou-2022-internet-augmented-language-models-through/)
- [next → Introducing BEREL: BERT Embeddings for Rabbinic-Encoded Language](https://md.research.elamur.ai/free/papers/hbo-2022-shmidman-introducing-berel-bert-embeddings-rabbinic/)
- [↑ back to papers of 2022](https://md.research.elamur.ai/free/papers/2022/page/2/)

## References

59 references, 3 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Mio Arai, Masahiro Kaneko, and Mamoru Komachi. 2019. Grammatical-error-aware incorrect example retrieval system for learners of Japanese as a second language. In Proceedings of the Fourteenth Workshop on Innovative Use of NLP for Building Educational Applications, pages 296–305, Florence, Italy. Ass… [find](https://scholar.google.com/scholar?q=Mio+Arai%2C+Masahiro+Kaneko%2C+and+Mamoru+Komachi%2E+2019%2E+Grammatical%2Derror%2Daware+incorrect+example+retrieval+system+for+learners+of+Japanese+as+a+second+language%2E+In+Proceedings+of+the+Fourteenth+Workshop)
2. Mio Arai, Masahiro Kaneko, and Mamoru Komachi. 2020. Example retrieval system using grammatical error detection for japanese as a second language learners. Transactions of the Japanese Society for Artificial Intelligence, 35(5):A–K23. [find](https://scholar.google.com/scholar?q=Mio+Arai%2C+Masahiro+Kaneko%2C+and+Mamoru+Komachi%2E+2020%2E+Example+retrieval+system+using+grammatical+error+detection+for+japanese+as+a+second+language+learners%2E+Transactions+of+the+Japanese+Society+for+Art)
3. Christopher Bryant, Mariano Felice, Øistein E. Andersen, and Ted Briscoe. 2019. The BEA-2019 Shared Task on Grammatical Error Correction. In BEA, pages 52–75, Florence, Italy. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Christopher+Bryant%2C+Mariano+Felice%2C+%C3%98istein+E%2E+Andersen%2C+and+Ted+Briscoe%2E+2019%2E+The+BEA%2D2019+Shared+Task+on+Grammatical+Error+Correction%2E+In+BEA%2C+pages+52%E2%80%9375%2C+Florence%2C+Italy%2E+Association+for+Computat) [our card](https://md.research.elamur.ai/free/papers/bryant-2019-bea-2019-shared-task-grammatical/)
4. Christopher Bryant, Mariano Felice, and Ted Briscoe. 2017. Automatic Annotation and Evaluation of Error Types for Grammatical Error Correction. In ACL, pages 793–805, Vancouver, Canada. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Christopher+Bryant%2C+Mariano+Felice%2C+and+Ted+Briscoe%2E+2017%2E+Automatic+Annotation+and+Evaluation+of+Error+Types+for+Grammatical+Error+Correction%2E+In+ACL%2C+pages+793%E2%80%93805%2C+Vancouver%2C+Canada%2E+Association+fo) [our card](https://md.research.elamur.ai/free/papers/bryant-2017-automatic-annotation-evaluation-error-types/)
5. Yuchang Cheng and Tomoki Nagase. 2012. An example-based Japanese proofreading system for offshore development. In Proceedings of COLING 2012: Demonstration Papers, pages 67–76, Mumbai, India. [find](https://scholar.google.com/scholar?q=Yuchang+Cheng+and+Tomoki+Nagase%2E+2012%2E+An+example%2Dbased+Japanese+proofreading+system+for+offshore+development%2E+In+Proceedings+of+COLING+2012%3A+Demonstration+Papers%2C+pages+67%E2%80%9376%2C+Mumbai%2C+India%2E)
6. The COLING 2012 Organizing Committee. [find](https://scholar.google.com/scholar?q=The+COLING+2012+Organizing+Committee%2E)
7. Shamil Chollampatt and Hwee Tou Ng. 2018. A Multilayer Convolutional Encoder-Decoder Neural Network for Grammatical Error Correction. In Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence, pages 5755–5762, New Orleans, Louisiana. Association for the Advancement of Artificial… [find](https://scholar.google.com/scholar?q=Shamil+Chollampatt+and+Hwee+Tou+Ng%2E+2018%2E+A+Multilayer+Convolutional+Encoder%2DDecoder+Neural+Network+for+Grammatical+Error+Correction%2E+In+Proceedings+of+the+Thirty%2DSecond+AAAI+Conference+on+Artificial+)
8. Daniel Dahlmeier and Hwee Tou Ng. 2012. Better Evaluation for Grammatical Error Correction. In NAACL, pages 568–572, Montréal, Canada. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Daniel+Dahlmeier+and+Hwee+Tou+Ng%2E+2012%2E+Better+Evaluation+for+Grammatical+Error+Correction%2E+In+NAACL%2C+pages+568%E2%80%93572%2C+Montr%C3%A9al%2C+Canada%2E+Association+for+Computational+Linguistics%2E)
9. Daniel Dahlmeier, Hwee Tou Ng, and Siew Mei Wu. 2013. Building a Large Annotated Corpus of Learner English: The NUS Corpus of Learner English. In BEA, pages 22–31, Atlanta, Georgia. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Daniel+Dahlmeier%2C+Hwee+Tou+Ng%2C+and+Siew+Mei+Wu%2E+2013%2E+Building+a+Large+Annotated+Corpus+of+Learner+English%3A+The+NUS+Corpus+of+Learner+English%2E+In+BEA%2C+pages+22%E2%80%9331%2C+Atlanta%2C+Georgia%2E+Association+for+Co)
10. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies,… [find](https://scholar.google.com/scholar?q=Jacob+Devlin%2C+Ming%2DWei+Chang%2C+Kenton+Lee%2C+and+Kristina+Toutanova%2E+2019%2E+BERT%3A+Pre%2Dtraining+of+deep+bidirectional+transformers+for+language+understanding%2E+In+Proceedings+of+the+2019+Conference+of+the+N)
11. Takao Doi, Hirofumi Yamamoto, and Eiichiro Sumita. 2005. Example-based machine translation using efficient sentence retrieval based on edit-distance. ACM Transactions on Asian Language Information Processing, 4(4):377–399. [find](https://scholar.google.com/scholar?q=Takao+Doi%2C+Hirofumi+Yamamoto%2C+and+Eiichiro+Sumita%2E+2005%2E+Example%2Dbased+machine+translation+using+efficient+sentence+retrieval+based+on+edit%2Ddistance%2E+ACM+Transactions+on+Asian+Language+Information+Pro)
12. Finale Doshi-Velez and Been Kim. 2017. Towards a rigorous science of interpretable machine learning. arXiv preprint arXiv:1702.08608. [find](https://scholar.google.com/scholar?q=Finale+Doshi%2DVelez+and+Been+Kim%2E+2017%2E+Towards+a+rigorous+science+of+interpretable+machine+learning%2E+arXiv+preprint+arXiv%3A1702%2E08608%2E) [our card](https://md.research.elamur.ai/free/papers/doshi-2017-velez-towards-rigorous-science-interpretable-machine/)
13. Mariano Felice, Christopher Bryant, and Ted Briscoe. 2016. Automatic Extraction of Learner Errors in ESL Sentences Using Linguistically Enhanced Alignments. In COLING, pages 825–835, Osaka, Japan. [find](https://scholar.google.com/scholar?q=Mariano+Felice%2C+Christopher+Bryant%2C+and+Ted+Briscoe%2E+2016%2E+Automatic+Extraction+of+Learner+Errors+in+ESL+Sentences+Using+Linguistically+Enhanced+Alignments%2E+In+COLING%2C+pages+825%E2%80%93835%2C+Osaka%2C+Japan%2E)
14. The COLING 2016 Organizing Committee. [find](https://scholar.google.com/scholar?q=The+COLING+2016+Organizing+Committee%2E)
15. Sylviane Granger. 1998. Developing an Automated Writing Placement System for ESL Learners. In LEC, pages 3–18. [find](https://scholar.google.com/scholar?q=Sylviane+Granger%2E+1998%2E+Developing+an+Automated+Writing+Placement+System+for+ESL+Learners%2E+In+LEC%2C+pages+3%E2%80%9318%2E)
16. Jiatao Gu, Yong Wang, Kyunghyun Cho, and Victor OK Li. 2018. Search engine guided neural machine translation. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 32. [find](https://scholar.google.com/scholar?q=Jiatao+Gu%2C+Yong+Wang%2C+Kyunghyun+Cho%2C+and+Victor+OK+Li%2E+2018%2E+Search+engine+guided+neural+machine+translation%2E+In+Proceedings+of+the+AAAI+Conference+on+Artificial+Intelligence%2C+volume+32%2E)
17. Kelvin Guu, Kenton Lee, Zora Tung, Panupong Pasupat, and Ming-Wei Chang. 2020. Realm: Retrieval-augmented language model pre-training. arXiv preprint arXiv:2002.08909. [find](https://scholar.google.com/scholar?q=Kelvin+Guu%2C+Kenton+Lee%2C+Zora+Tung%2C+Panupong+Pasupat%2C+and+Ming%2DWei+Chang%2E+2020%2E+Realm%3A+Retrieval%2Daugmented+language+model+pre%2Dtraining%2E+arXiv+preprint+arXiv%3A2002%2E08909%2E)
18. Rejwanul Haque, Sudip Kumar Naskar, Yanjun Ma, and Andy Way. 2009. Using supertags as source language context in SMT. In Proceedings of the 13th Annual conference of the European Association for Machine Translation, Barcelona, Spain. European Association for Machine Translation. [find](https://scholar.google.com/scholar?q=Rejwanul+Haque%2C+Sudip+Kumar+Naskar%2C+Yanjun+Ma%2C+and+Andy+Way%2E+2009%2E+Using+supertags+as+source+language+context+in+SMT%2E+In+Proceedings+of+the+13th+Annual+conference+of+the+European+Association+for+Machi)
19. Kengo Hotate, Masahiro Kaneko, Satoru Katsumata, and Mamoru Komachi. 2019. Controlling grammatical error correction using word edit rate. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics: Student Research Workshop, pages 149–154, Florence, Italy. Association… [find](https://scholar.google.com/scholar?q=Kengo+Hotate%2C+Masahiro+Kaneko%2C+Satoru+Katsumata%2C+and+Mamoru+Komachi%2E+2019%2E+Controlling+grammatical+error+correction+using+word+edit+rate%2E+In+Proceedings+of+the+57th+Annual+Meeting+of+the+Association+f)
20. Kengo Hotate, Masahiro Kaneko, and Mamoru Komachi. 2020. Generating diverse corrections with local beam search for grammatical error correction. In Proceedings of the 28th International Conference on Computational Linguistics, pages 2132–2137, Barcelona, Spain (Online). International Committee on Co… [find](https://scholar.google.com/scholar?q=Kengo+Hotate%2C+Masahiro+Kaneko%2C+and+Mamoru+Komachi%2E+2020%2E+Generating+diverse+corrections+with+local+beam+search+for+grammatical+error+correction%2E+In+Proceedings+of+the+28th+International+Conference+on+)
21. Tim Johns. 1994. From printout to handout: Grammar and vocabulary teaching in the context of data-driven learning. Perspectives on pedagogical grammar, 293. [find](https://scholar.google.com/scholar?q=Tim+Johns%2E+1994%2E+From+printout+to+handout%3A+Grammar+and+vocabulary+teaching+in+the+context+of+data%2Ddriven+learning%2E+Perspectives+on+pedagogical+grammar%2C+293%2E)
22. Jeff Johnson, Matthijs Douze, and Hervé Jégou. 2021. Billion-scale similarity search with gpus. IEEE Transactions on Big Data, 7(3):535–547. [find](https://scholar.google.com/scholar?q=Jeff+Johnson%2C+Matthijs+Douze%2C+and+Herv%C3%A9+J%C3%A9gou%2E+2021%2E+Billion%2Dscale+similarity+search+with+gpus%2E+IEEE+Transactions+on+Big+Data%2C+7%283%29%3A535%E2%80%93547%2E)
23. Marcin Junczys-Dowmunt, Roman Grundkiewicz, Shubha Guha, and Kenneth Heafield. 2018. Approaching neural grammatical error correction as a low-resource machine translation task. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human… [find](https://scholar.google.com/scholar?q=Marcin+Junczys%2DDowmunt%2C+Roman+Grundkiewicz%2C+Shubha+Guha%2C+and+Kenneth+Heafield%2E+2018%2E+Approaching+neural+grammatical+error+correction+as+a+low%2Dresource+machine+translation+task%2E+In+Proceedings+of+the+2)
24. Megumi Makino and Kazuhide Yamamoto. 2008. Summarization by analogy: An example-based approach for news articles. In Proceedings of the Third International Joint Conference on Natural Language Processing: Volume-II. [find](https://scholar.google.com/scholar?q=Megumi+Makino+and+Kazuhide+Yamamoto%2E+2008%2E+Summarization+by+analogy%3A+An+example%2Dbased+approach+for+news+articles%2E+In+Proceedings+of+the+Third+International+Joint+Conference+on+Natural+Language+Process)
25. Shigeki Matsubara, Yoshihide Kato, and Seiji Egawa. 2008. Escort: example sentence retrieval system as support tool for english writing. Journal of Information Processing and Management, 51(4):251–259. [find](https://scholar.google.com/scholar?q=Shigeki+Matsubara%2C+Yoshihide+Kato%2C+and+Seiji+Egawa%2E+2008%2E+Escort%3A+example+sentence+retrieval+system+as+support+tool+for+english+writing%2E+Journal+of+Information+Processing+and+Management%2C+51%284%29%3A251%E2%80%93259)
26. Atsushi Mizumoto and Kiyomi Chujo. 2015. A meta-analysis of data-driven learning approach in the japanese efl classroom. English Corpus Studies, 22:1–18. [find](https://scholar.google.com/scholar?q=Atsushi+Mizumoto+and+Kiyomi+Chujo%2E+2015%2E+A+meta%2Danalysis+of+data%2Ddriven+learning+approach+in+the+japanese+efl+classroom%2E+English+Corpus+Studies%2C+22%3A1%E2%80%9318%2E)
27. Masahiro Kaneko, Kengo Hotate, Satoru Katsumata, and Mamoru Komachi. 2019. TMU transformer system using BERT for re-ranking at BEA 2019 grammatical error correction on restricted track. In Proceedings of the Fourteenth Workshop on Innovative Use of NLP for Building Educational Applications, pages 20… [find](https://scholar.google.com/scholar?q=Masahiro+Kaneko%2C+Kengo+Hotate%2C+Satoru+Katsumata%2C+and+Mamoru+Komachi%2E+2019%2E+TMU+transformer+system+using+BERT+for+re%2Dranking+at+BEA+2019+grammatical+error+correction+on+restricted+track%2E+In+Proceedings)
28. Tomoya Mizumoto, Mamoru Komachi, Masaaki Nagata, and Yuji Matsumoto. 2011. Mining Revision Log of Language Learning SNS for Automated Japanese Error Correction of Second Language Learners. In IJCNLP, pages 147–155, Chiang Mai, Thailand. Asian Federation of Natural Language Processing. [find](https://scholar.google.com/scholar?q=Tomoya+Mizumoto%2C+Mamoru+Komachi%2C+Masaaki+Nagata%2C+and+Yuji+Matsumoto%2E+2011%2E+Mining+Revision+Log+of+Language+Learning+SNS+for+Automated+Japanese+Error+Correction+of+Second+Language+Learners%2E+In+IJCNLP%2C+)
29. Masahiro Kaneko and Mamoru Komachi. 2019. Multi-head multi-layer attention to deep language representations for grammatical error detection. Computación y Sistemas, 23(3):883–891. [find](https://scholar.google.com/scholar?q=Masahiro+Kaneko+and+Mamoru+Komachi%2E+2019%2E+Multi%2Dhead+multi%2Dlayer+attention+to+deep+language+representations+for+grammatical+error+detection%2E+Computaci%C3%B3n+y+Sistemas%2C+23%283%29%3A883%E2%80%93891%2E)
30. Makoto Nagao. 1984. A framework of a mechanical translation between japanese and english by analogy principle. Artificial and human intelligence, pages 351–354. [find](https://scholar.google.com/scholar?q=Makoto+Nagao%2E+1984%2E+A+framework+of+a+mechanical+translation+between+japanese+and+english+by+analogy+principle%2E+Artificial+and+human+intelligence%2C+pages+351%E2%80%93354%2E)
31. Masahiro Kaneko, Masato Mita, Shun Kiyono, Jun Suzuki, and Kentaro Inui. 2020. Encoder-decoder models can benefit from pre-trained masked language models in grammatical error correction. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 4248– 4254, Onl… [find](https://scholar.google.com/scholar?q=Masahiro+Kaneko%2C+Masato+Mita%2C+Shun+Kiyono%2C+Jun+Suzuki%2C+and+Kentaro+Inui%2E+2020%2E+Encoder%2Ddecoder+models+can+benefit+from+pre%2Dtrained+masked+language+models+in+grammatical+error+correction%2E+In+Proceeding)
32. Ryo Nagata. 2019. Toward a task of feedback comment generation for writing learning. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 3206–3215, Hong Kong, China.… [find](https://scholar.google.com/scholar?q=Ryo+Nagata%2E+2019%2E+Toward+a+task+of+feedback+comment+generation+for+writing+learning%2E+In+Proceedings+of+the+2019+Conference+on+Empirical+Methods+in+Natural+Language+Processing+and+the+9th+International)
33. Masahiro Kaneko, Yuya Sakaizawa, and Mamoru Komachi. 2017. Grammatical error detection using error- and grammaticality-specific word embeddings. In Proceedings of the Eighth International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 40–48, Taipei, Taiwan. Asian Fede… [find](https://scholar.google.com/scholar?q=Masahiro+Kaneko%2C+Yuya+Sakaizawa%2C+and+Mamoru+Komachi%2E+2017%2E+Grammatical+error+detection+using+error%2D+and+grammaticality%2Dspecific+word+embeddings%2E+In+Proceedings+of+the+Eighth+International+Joint+Confer)
34. Urvashi Khandelwal, Angela Fan, Dan Jurafsky, Luke Zettlemoyer, and Mike Lewis. 2021. Nearest neighbor machine translation. In International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Urvashi+Khandelwal%2C+Angela+Fan%2C+Dan+Jurafsky%2C+Luke+Zettlemoyer%2C+and+Mike+Lewis%2E+2021%2E+Nearest+neighbor+machine+translation%2E+In+International+Conference+on+Learning+Representations%2E)
35. Urvashi Khandelwal, Omer Levy, Dan Jurafsky, Luke Zettlemoyer, and Mike Lewis. 2020. Generalization through memorization: Nearest neighbor language models. In International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Urvashi+Khandelwal%2C+Omer+Levy%2C+Dan+Jurafsky%2C+Luke+Zettlemoyer%2C+and+Mike+Lewis%2E+2020%2E+Generalization+through+memorization%3A+Nearest+neighbor+language+models%2E+In+International+Conference+on+Learning+Repr)
36. Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, Sebastian Riedel, and Douwe Kiela. 2020. Retrieval-augmented generation for knowledge-intensive nlp tasks. In Advances in Neural Information Proc… [find](https://scholar.google.com/scholar?q=Patrick+Lewis%2C+Ethan+Perez%2C+Aleksandra+Piktus%2C+Fabio+Petroni%2C+Vladimir+Karpukhin%2C+Naman+Goyal%2C+Heinrich+K%C3%BCttler%2C+Mike+Lewis%2C+Wen%2Dtau+Yih%2C+Tim+Rockt%C3%A4schel%2C+Sebastian+Riedel%2C+and+Douwe+Kiela%2E+2020%2E+Retr)
37. Ryo Nagata, Kentaro Inui, and Shin’ichiro Ishikawa. 2020. Creating corpora for research in feedback comment generation. In Proceedings of the 12th Language Resources and Evaluation Conference, pages 340–345, Marseille, France. European Language Resources Association. [find](https://scholar.google.com/scholar?q=Ryo+Nagata%2C+Kentaro+Inui%2C+and+Shin%E2%80%99ichiro+Ishikawa%2E+2020%2E+Creating+corpora+for+research+in+feedback+comment+generation%2E+In+Proceedings+of+the+12th+Language+Resources+and+Evaluation+Conference%2C+pages+3)
38. Courtney Napoles, Keisuke Sakaguchi, Matt Post, and Joel Tetreault. 2015. Ground Truth for Grammatical Error Correction Metrics. In NAACL, pages 588– 593, Beijing, China. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Courtney+Napoles%2C+Keisuke+Sakaguchi%2C+Matt+Post%2C+and+Joel+Tetreault%2E+2015%2E+Ground+Truth+for+Grammatical+Error+Correction+Metrics%2E+In+NAACL%2C+pages+588%E2%80%93+593%2C+Beijing%2C+China%2E+Association+for+Computational)
39. Courtney Napoles, Keisuke Sakaguchi, and Joel Tetreault. 2017. JFLEG: A Fluency Corpus and Benchmark for Grammatical Error Correction. In EACL, pages 229–234, Valencia, Spain. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Courtney+Napoles%2C+Keisuke+Sakaguchi%2C+and+Joel+Tetreault%2E+2017%2E+JFLEG%3A+A+Fluency+Corpus+and+Benchmark+for+Grammatical+Error+Correction%2E+In+EACL%2C+pages+229%E2%80%93234%2C+Valencia%2C+Spain%2E+Association+for+Computat)
40. Hwee Tou Ng, Siew Mei Wu, Ted Briscoe, Christian Hadiwinoto, Raymond Hendy Susanto, and Christopher Bryant. 2014. The CoNLL-2014 Shared Task on Grammatical Error Correction. In CoNLL, pages 1–14, Baltimore, Maryland. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Hwee+Tou+Ng%2C+Siew+Mei+Wu%2C+Ted+Briscoe%2C+Christian+Hadiwinoto%2C+Raymond+Hendy+Susanto%2C+and+Christopher+Bryant%2E+2014%2E+The+CoNLL%2D2014+Shared+Task+on+Grammatical+Error+Correction%2E+In+CoNLL%2C+pages+1%E2%80%9314%2C+Balt)
41. Kiyonori Ohtake and Kazuhide Yamamoto. 2003. Applicability analysis of corpus-derived paraphrases toward example-based paraphrasing. In Proceedings of the 17th Pacific Asia Conference on Language, Information and Computation, pages 380–391, Sen-tosa, Singapore. COLIPS PUBLICATIONS. [find](https://scholar.google.com/scholar?q=Kiyonori+Ohtake+and+Kazuhide+Yamamoto%2E+2003%2E+Applicability+analysis+of+corpus%2Dderived+paraphrases+toward+example%2Dbased+paraphrasing%2E+In+Proceedings+of+the+17th+Pacific+Asia+Conference+on+Language%2C+Inf)
42. Kostiantyn Omelianchuk, Vitaliy Atrasevych, Artem Chernodub, and Oleksandr Skurzhanskyi. 2020. GECToR – grammatical error correction: Tag, not rewrite. In Proceedings of the Fifteenth Workshop on Innovative Use of NLP for Building Educational Applications, pages 163–170, Seattle, WA, USA → Online. A… [find](https://scholar.google.com/scholar?q=Kostiantyn+Omelianchuk%2C+Vitaliy+Atrasevych%2C+Artem+Chernodub%2C+and+Oleksandr+Skurzhanskyi%2E+2020%2E+GECToR+%E2%80%93+grammatical+error+correction%3A+Tag%2C+not+rewrite%2E+In+Proceedings+of+the+Fifteenth+Workshop+on+Inno)
43. Hiroki Ouchi, Jun Suzuki, Sosuke Kobayashi, Sho Yokoi, Tatsuki Kuribayashi, Ryuto Konno, and Kentaro Inui. 2020. Instance-based learning of span representations: A case study through named entity recognition. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics,… [find](https://scholar.google.com/scholar?q=Hiroki+Ouchi%2C+Jun+Suzuki%2C+Sosuke+Kobayashi%2C+Sho+Yokoi%2C+Tatsuki+Kuribayashi%2C+Ryuto+Konno%2C+and+Kentaro+Inui%2E+2020%2E+Instance%2Dbased+learning+of+span+representations%3A+A+case+study+through+named+entity+reco)
44. John W Ratcliff and David E Metzener. 1988. Pattern-matching-the gestalt approach. Dr Dobbs Journal, 13(7):46. [find](https://scholar.google.com/scholar?q=John+W+Ratcliff+and+David+E+Metzener%2E+1988%2E+Pattern%2Dmatching%2Dthe+gestalt+approach%2E+Dr+Dobbs+Journal%2C+13%287%29%3A46%2E)
45. Rico Sennrich, Barry Haddow, and Alexandra Birch. 2016. Neural machine translation of rare words with subword units. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1715– 1725, Berlin, Germany. Association for Computational Li… [find](https://scholar.google.com/scholar?q=Rico+Sennrich%2C+Barry+Haddow%2C+and+Alexandra+Birch%2E+2016%2E+Neural+machine+translation+of+rare+words+with+subword+units%2E+In+Proceedings+of+the+54th+Annual+Meeting+of+the+Association+for+Computational+Ling)
46. Nicolas Stroppa, Antal van den Bosch, and Andy Way. 2007. Exploiting source similarity for SMT using context-informed features. In Proceedings of the 11th Conference on Theoretical and Methodological Issues in Machine Translation of Natural Languages: Papers, Skövde, Sweden. [find](https://scholar.google.com/scholar?q=Nicolas+Stroppa%2C+Antal+van+den+Bosch%2C+and+Andy+Way%2E+2007%2E+Exploiting+source+similarity+for+SMT+using+context%2Dinformed+features%2E+In+Proceedings+of+the+11th+Conference+on+Theoretical+and+Methodological+)
47. Eiichiro Sumita and Hitoshi Iida. 1991. Experiments and prospects of example-based machine translation. In 29th Annual Meeting of the Association for Computational Linguistics, pages 185–192, Berkeley, California, USA. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Eiichiro+Sumita+and+Hitoshi+Iida%2E+1991%2E+Experiments+and+prospects+of+example%2Dbased+machine+translation%2E+In+29th+Annual+Meeting+of+the+Association+for+Computational+Linguistics%2C+pages+185%E2%80%93192%2C+Berkeley)
48. Antal Van Den Bosch. 2007. A memory-based classification approach to marker-based ebmt. In Proceedings of the METIS-II Workshop on New Approaches to Machine Translation. [find](https://scholar.google.com/scholar?q=Antal+Van+Den+Bosch%2E+2007%2E+A+memory%2Dbased+classification+approach+to+marker%2Dbased+ebmt%2E+In+Proceedings+of+the+METIS%2DII+Workshop+on+New+Approaches+to+Machine+Translation%2E)
49. Maarten Van Gompel, Antal Van Den Bosch, and Peter Berck. 2009. Extending memory-based machine translation to phrases. In Proceedings of the Third Workshop on Example-Based Machine Translation, pages 79–86. [find](https://scholar.google.com/scholar?q=Maarten+Van+Gompel%2C+Antal+Van+Den+Bosch%2C+and+Peter+Berck%2E+2009%2E+Extending+memory%2Dbased+machine+translation+to+phrases%2E+In+Proceedings+of+the+Third+Workshop+on+Example%2DBased+Machine+Translation%2C+pages+)
50. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Ł ukasz Kaiser, and Illia Polosukhin. 2017. Attention is All you Need. In I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett, editors, NeurIPS, pages 5998–6008. Curran Asso… [find](https://scholar.google.com/scholar?q=Ashish+Vaswani%2C+Noam+Shazeer%2C+Niki+Parmar%2C+Jakob+Uszkoreit%2C+Llion+Jones%2C+Aidan+N+Gomez%2C+%C5%81+ukasz+Kaiser%2C+and+Illia+Polosukhin%2E+2017%2E+Attention+is+All+you+Need%2E+In+I%2E+Guyon%2C+U%2E+V%2E+Luxburg%2C+S%2E+Bengio%2C+H%2E)
51. Mary E Webb, Andrew Fluck, Johannes Magenheim, Joyce Malyn-Smith, Juliet Waters, Michelle De-schênes, and Jason Zagami. 2020. Machine learning for human learners: opportunities, issues, tensions and threats. Educational Technology Research and Development, pages 1–22. [find](https://scholar.google.com/scholar?q=Mary+E+Webb%2C+Andrew+Fluck%2C+Johannes+Magenheim%2C+Joyce+Malyn%2DSmith%2C+Juliet+Waters%2C+Michelle+De%2Dsch%C3%AAnes%2C+and+Jason+Zagami%2E+2020%2E+Machine+learning+for+human+learners%3A+opportunities%2C+issues%2C+tensions+and+t)
52. Sam Wiseman and Karl Stratos. 2019. Label-agnostic sequence labeling by copying nearest neighbors. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 5363–5369, Florence, Italy. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Sam+Wiseman+and+Karl+Stratos%2E+2019%2E+Label%2Dagnostic+sequence+labeling+by+copying+nearest+neighbors%2E+In+Proceedings+of+the+57th+Annual+Meeting+of+the+Association+for+Computational+Linguistics%2C+pages+536)
53. Ziang Xie, Guillaume Genthial, Stanley Xie, Andrew Ng, and Dan Jurafsky. 2018. Noising and denoising natural language: Diverse backtranslation for grammar correction. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language… [find](https://scholar.google.com/scholar?q=Ziang+Xie%2C+Guillaume+Genthial%2C+Stanley+Xie%2C+Andrew+Ng%2C+and+Dan+Jurafsky%2E+2018%2E+Noising+and+denoising+natural+language%3A+Diverse+backtranslation+for+grammar+correction%2E+In+Proceedings+of+the+2018+Confer)
54. Helen Yannakoudakis, Ted Briscoe, and Ben Medlock. 2011. A New Dataset and Method for Automatically Grading ESOL Texts. In NAACL, pages 180–189, Portland, Oregon, USA. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Helen+Yannakoudakis%2C+Ted+Briscoe%2C+and+Ben+Medlock%2E+2011%2E+A+New+Dataset+and+Method+for+Automatically+Grading+ESOL+Texts%2E+In+NAACL%2C+pages+180%E2%80%93189%2C+Portland%2C+Oregon%2C+USA%2E+Association+for+Computational+Li)
55. Helen Yannakoudakis, Øistein E. Andersen, Geran-payeh Ardeshir, Briscoe Ted, and Nicholls Diane. 2018. Developing an Automated Writing Placement System for ESL Learners. In Applied Measurement in Education, pages 251–267. [find](https://scholar.google.com/scholar?q=Helen+Yannakoudakis%2C+%C3%98istein+E%2E+Andersen%2C+Geran%2Dpayeh+Ardeshir%2C+Briscoe+Ted%2C+and+Nicholls+Diane%2E+2018%2E+Developing+an+Automated+Writing+Placement+System+for+ESL+Learners%2E+In+Applied+Measurement+in+Educ)
56. Tzu-Hsi Yen, Jian-Cheng Wu, Jim Chang, Joanne Boisson, and Jason Chang. 2015. WriteAhead: Mining grammar patterns in corpora for assisted writing. In Proceedings of ACL-IJCNLP 2015 System Demonstrations, pages 139–144, Beijing, China. Association for Computational Linguistics and The Asian Federatio… [find](https://scholar.google.com/scholar?q=Tzu%2DHsi+Yen%2C+Jian%2DCheng+Wu%2C+Jim+Chang%2C+Joanne+Boisson%2C+and+Jason+Chang%2E+2015%2E+WriteAhead%3A+Mining+grammar+patterns+in+corpora+for+assisted+writing%2E+In+Proceedings+of+ACL%2DIJCNLP+2015+System+Demonstratio)
57. Zheng Yuan and Ted Briscoe. 2016. Grammatical error correction using neural machine translation. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 380–386, San Diego, California. Association for Co… [find](https://scholar.google.com/scholar?q=Zheng+Yuan+and+Ted+Briscoe%2E+2016%2E+Grammatical+error+correction+using+neural+machine+translation%2E+In+Proceedings+of+the+2016+Conference+of+the+North+American+Chapter+of+the+Association+for+Computationa)
58. Jingyi Zhang, Masao Utiyama, Eiichro Sumita, Graham Neubig, and Satoshi Nakamura. 2018. Guiding neural machine translation with retrieved translation pieces. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technolog… [find](https://scholar.google.com/scholar?q=Jingyi+Zhang%2C+Masao+Utiyama%2C+Eiichro+Sumita%2C+Graham+Neubig%2C+and+Satoshi+Nakamura%2E+2018%2E+Guiding+neural+machine+translation+with+retrieved+translation+pieces%2E+In+Proceedings+of+the+2018+Conference+of+t)
59. Wei Zhao, Liang Wang, Kewei Shen, Ruoyu Jia, and Jingming Liu. 2019. Improving grammatical error correction via pre-training a copy-augmented architecture with unlabeled data. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human… [find](https://scholar.google.com/scholar?q=Wei+Zhao%2C+Liang+Wang%2C+Kewei+Shen%2C+Ruoyu+Jia%2C+and+Jingming+Liu%2E+2019%2E+Improving+grammatical+error+correction+via+pre%2Dtraining+a+copy%2Daugmented+architecture+with+unlabeled+data%2E+In+Proceedings+of+the+20)