---
title: "Neural Network Translation Models for — Chollampatt 2016"
description: "The paper addresses grammatical error correction (GEC) for English learner writing."
authors: ["Shamil Chollampatt", "Kaveh Taghipour", "Hwee Tou Ng"]
year: 2016
venue: "arXiv:1606.00189"
source: "https://arxiv.org/abs/1606.00189"
doi: "10.48550/arxiv.1606.00189"
arxiv: "1606.00189"
licence: "arXiv — non-exclusive licence to distribute"
licence_url: "http://arxiv.org/licenses/nonexclusive-distrib/1.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/papers/chollampatt-2016-neural-network-translation-models-grammatical/"
---
[Elamur Research](https://md.research.elamur.ai/) · [this page in HTML](https://research.elamur.ai/free/papers/chollampatt-2016-neural-network-translation-models-grammatical/)

[home](https://md.research.elamur.ai/) / [papers](https://md.research.elamur.ai/free/papers/) / [2016](https://md.research.elamur.ai/free/papers/2016/page/2/) / Neural Network Translation Models for Grammatical Error Correction

# Neural Network Translation Models for Grammatical Error Correction

**Authors:** Shamil Chollampatt, Kaveh Taghipour, [Hwee Tou Ng](https://md.research.elamur.ai/free/authors/hwee-tou-ng/) · **Year:** 2016 · **Venue:** arXiv:1606.00189 · **Licence:** [arXiv — non-exclusive licence to distribute](http://arxiv.org/licenses/nonexclusive-distrib/1.0/)

**Links:** [Publisher's page](https://arxiv.org/abs/1606.00189) · [PDF](https://arxiv.org/pdf/1606.00189) · [DOI 10.48550/arxiv.1606.00189](https://doi.org/10.48550/arxiv.1606.00189) · [arXiv 1606.00189](https://arxiv.org/abs/1606.00189) · [OpenAlex](https://openalex.org/W2410156476)

**Citations:** 50 citations (openalex-snapshot · 2026-10-14) · 29 in our library, of the 140673 papers whose bibliographies we read

## Summary
The paper addresses grammatical error correction (GEC) for English learner writing. It frames GEC as translation from erroneous to corrected English. Phrase-based SMT systems were already state of the art for this task, since they learn rewrites from parallel data without needing error-type-specific rules. However, they handle words as discrete symbols, rely on linear feature combinations, and see little sentence-wide context.

The authors add two neural features to a monotone phrase-based SMT system tuned with MERT on F0.5:
- **NNGLM** is a feed-forward global lexicon model. It reads a bag-of-words of the whole source sentence and predicts which target words should appear. Its outputs are recalibrated with a class-weighted logistic mapping.
- **NNJM** is a joint model. It scores each target word using a window of aligned source words and the preceding target words.

On the CoNLL 2014 shared task test set, combining both models improves F0.5 by 1.17 over the baseline. The final system surpasses the previous best published result without extra training data. This shows that continuous representations help SMT-based correction, especially for word choice and collocation errors.

## Key points
- The final system (Baseline + NNGLM + NNJM) reaches P 52.34, R 23.07 and F0.5 41.75 on CoNLL 2014 test data. The baseline scores 40.58 F0.5.
- It beats [Yuan and Briscoe, 2016] (neural MT, 39.90 F0.5) by 1.85, even though that system used much more training data. It also beats [Susanto et al., 2014] (39.39) and the top shared-task teams CAMB (37.33), CUUI (36.79) and AMU (35.01).
- NNGLM uses binary cross-entropy training. A post-hoc logistic rescaling, fitted on the development set, counters the model's tendency to output probabilities near zero.
- The two models complement each other. NNGLM steers word selection using whole-sentence context, while NNJM favours fluent local corrections.
- The main weakness is spelling. Rare and misspelled words are mapped to UNK, so some typos fixed by the baseline go uncorrected (e.g., "hugh" → "huge").

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

Fields: [Machine translation](https://md.research.elamur.ai/free/shelves/machine-translation/)

Topics: [Machine translation](https://md.research.elamur.ai/free/topics/machine-translation/)

## Cite

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@misc{chollampatt2016neural,
  title = {{Neural Network Translation Models for Grammatical Error Correction}},
  author = {Chollampatt, Shamil and Taghipour, Kaveh and Ng, Hwee Tou},
  year = {2016},
  doi = {10.48550/arxiv.1606.00189},
  eprint = {1606.00189},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.1606.00189}
}
```

- [← previous Neural Architecture Search with Reinforcement Learning](https://md.research.elamur.ai/free/papers/zoph-2016-neural-architecture-search-reinforcement-learning/)
- [next → Neural Paraphrase Generation with Stacked Residual LSTM Networks](https://md.research.elamur.ai/free/papers/prakash-2016-neural-paraphrase-generation-stacked-residual/)
- [↑ back to papers of 2016](https://md.research.elamur.ai/free/papers/2016/page/2/)

## References

22 references, 0 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Yoshua Bengio, Réjean Ducharme, Pascal Vincent, and Christian Janvin. A neural probabilistic language model. Journal of Machine Learning Research, 3:1137–1155, 2003. [find](https://scholar.google.com/scholar?q=Yoshua+Bengio%2C+Re%CC%81jean+Ducharme%2C+Pascal+Vincent%2C+and+Christian+Janvin%2E+A+neural+probabilistic+language+model%2E+Journal+of+Machine+Learning+Research%2C+3%3A1137%E2%80%931155%2C+2003%2E)
2. Chris Brockett, William B. Dolan, and Michael Gamon. Correcting ESL errors using phrasal SMT techniques. In Proc. of ACL, 2006. [find](https://scholar.google.com/scholar?q=Chris+Brockett%2C+William+B%2E+Dolan%2C+and+Michael+Gamon%2E+Correcting+ESL+errors+using+phrasal+SMT+techniques%2E+In+Proc%2E+of+ACL%2C+2006%2E)
3. Daniel Dahlmeier and Hwee Tou Ng. Better evaluation for grammatical error correction. In Proc. of NAACL, 2012. [find](https://scholar.google.com/scholar?q=Daniel+Dahlmeier+and+Hwee+Tou+Ng%2E+Better+evaluation+for+grammatical+error+correction%2E+In+Proc%2E+of+NAACL%2C+2012%2E)
4. Daniel Dahlmeier, Hwee Tou Ng, and Eric Jun Feng Ng. NUS at the HOO 2012 shared task. In Proceedings of the Seventh Workshop on Building Educational Applications Using NLP, 2012. [find](https://scholar.google.com/scholar?q=Daniel+Dahlmeier%2C+Hwee+Tou+Ng%2C+and+Eric+Jun+Feng+Ng%2E+NUS+at+the+HOO+2012+shared+task%2E+In+Proceedings+of+the+Seventh+Workshop+on+Building+Educational+Applications+Using+NLP%2C+2012%2E)
5. Daniel Dahlmeier, Hwee Tou Ng, and Siew Mei Wu. Building a large annotated corpus of learner English: The NUS corpus of learner English. In Proceedings of the Eighth Workshop on Innovative Use of NLP for Building Educational Applications, 2013. [find](https://scholar.google.com/scholar?q=Daniel+Dahlmeier%2C+Hwee+Tou+Ng%2C+and+Siew+Mei+Wu%2E+Building+a+large+annotated+corpus+of+learner+English%3A+The+NUS+corpus+of+learner+English%2E+In+Proceedings+of+the+Eighth+Workshop+on+Innovative+Use+of+NLP+)
6. Robert Dale and Adam Kilgarriff. Helping Our Own: The HOO 2011 pilot shared task. In Proceedings of the 13th European Workshop on Natural Language Generation, 2011. [find](https://scholar.google.com/scholar?q=Robert+Dale+and+Adam+Kilgarriff%2E+Helping+Our+Own%3A+The+HOO+2011+pilot+shared+task%2E+In+Proceedings+of+the+13th+European+Workshop+on+Natural+Language+Generation%2C+2011%2E)
7. Robert Dale, Ilya Anisimoff, and George Narroway. HOO 2012: A report on the preposition and determiner error correction shared task. In Proceedings of the Seventh Workshop on Building Educational Applications Using NLP, 2012. [find](https://scholar.google.com/scholar?q=Robert+Dale%2C+Ilya+Anisimoff%2C+and+George+Narroway%2E+HOO+2012%3A+A+report+on+the+preposition+and+determiner+error+correction+shared+task%2E+In+Proceedings+of+the+Seventh+Workshop+on+Building+Educational+Appl)
8. Jacob Devlin, Rabih Zbib, Zhongqiang Huang, Thomas Lamar, Richard Schwartz, and John Makhoul. Fast and robust neural network joint models for statistical machine translation. In Proc. of ACL, 2014. [find](https://scholar.google.com/scholar?q=Jacob+Devlin%2C+Rabih+Zbib%2C+Zhongqiang+Huang%2C+Thomas+Lamar%2C+Richard+Schwartz%2C+and+John+Makhoul%2E+Fast+and+robust+neural+network+joint+models+for+statistical+machine+translation%2E+In+Proc%2E+of+ACL%2C+2014%2E)
9. Mariano Felice, Zheng Yuan, Øistein E. Andersen, Helen Yannakoudakis, and Ekaterina Kochmar. Grammatical error correction using hybrid systems and type filtering. In Proc. of CoNLL Shared Task, 2014. [find](https://scholar.google.com/scholar?q=Mariano+Felice%2C+Zheng+Yuan%2C+%C3%98istein+E%2E+Andersen%2C+Helen+Yannakoudakis%2C+and+Ekaterina+Kochmar%2E+Grammatical+error+correction+using+hybrid+systems+and+type+filtering%2E+In+Proc%2E+of+CoNLL+Shared+Task%2C+2014%2E)
10. Thanh-Le Ha, Jan Niehues, and Alex Waibel. Lexical translation model using a deep neural network architecture. In Proceedings of the 11th International Workshop on Spoken Language Translation, 2014. [↗](https://doi.org/10.48550/arxiv.1504.07395)
11. Marcin Junczys-Dowmunt and Roman Grundkiewicz. The AMU system in the CoNLL-2014 shared task: Grammatical error correction by data-intensive and feature-rich statistical machine translation. In Proc. of CoNLL Shared Task, 2014. [find](https://scholar.google.com/scholar?q=Marcin+Junczys%2DDowmunt+and+Roman+Grundkiewicz%2E+The+AMU+system+in+the+CoNLL%2D2014+shared+task%3A+Grammatical+error+correction+by+data%2Dintensive+and+feature%2Drich+statistical+machine+translation%2E+In+Proc%2E+o)
12. Philipp Koehn, Franz Josef Och, and Daniel Marcu. Statistical phrase-based translation. In Proc. of NAACL, 2003. [↗](https://doi.org/10.21236/ada461156)
13. Arne Mauser, Saša Hasan, and Hermann Ney. Extending statistical machine translation with discriminative and trigger-based lexicon models. In Proc. of EMNLP, 2009. [find](https://scholar.google.com/scholar?q=Arne+Mauser%2C+Sas%CC%8Ca+Hasan%2C+and+Hermann+Ney%2E+Extending+statistical+machine+translation+with+discriminative+and+trigger%2Dbased+lexicon+models%2E+In+Proc%2E+of+EMNLP%2C+2009%2E)
14. Tomoya Mizumoto, Mamoru Komachi, Masaaki Nagata, and Yuji Matsumoto. Mining revision log of language learning SNS for automated Japanese error correction of second language learners. In Proc. of IJCNLP, 2011. [find](https://scholar.google.com/scholar?q=Tomoya+Mizumoto%2C+Mamoru+Komachi%2C+Masaaki+Nagata%2C+and+Yuji+Matsumoto%2E+Mining+revision+log+of+language+learning+SNS+for+automated+Japanese+error+correction+of+second+language+learners%2E+In+Proc%2E+of+IJCNL)
15. Hwee Tou Ng, Siew Mei Wu, Yuanbin Wu, Christian Hadiwinoto, and Joel Tetreault. The CoNLL-2013 shared task on grammatical error correction. In Proc. of CoNLL Shared Task, 2013. [find](https://scholar.google.com/scholar?q=Hwee+Tou+Ng%2C+Siew+Mei+Wu%2C+Yuanbin+Wu%2C+Christian+Hadiwinoto%2C+and+Joel+Tetreault%2E+The+CoNLL%2D2013+shared+task+on+grammatical+error+correction%2E+In+Proc%2E+of+CoNLL+Shared+Task%2C+2013%2E)
16. Hwee Tou Ng, Siew Mei Wu, Ted Briscoe, Christian Hadiwinoto, Raymond Hendy Susanto, and Christopher Bryant. The CoNLL-2014 shared task on grammatical error correction. In Proc. of CoNLL Shared Task, 2014. [find](https://scholar.google.com/scholar?q=Hwee+Tou+Ng%2C+Siew+Mei+Wu%2C+Ted+Briscoe%2C+Christian+Hadiwinoto%2C+Raymond+Hendy+Susanto%2C+and+Christopher+Bryant%2E+The+CoNLL%2D2014+shared+task+on+grammatical+error+correction%2E+In+Proc%2E+of+CoNLL+Shared+Task%2C+2)
17. Alla Rozovskaya, Kai-Wei Chang, Mark Sammons, Dan Roth, and Nizar Habash. The Illinois-Columbia system in the CoNLL-2014 shared task. In Proc. of CoNLL Shared Task, 2014. [find](https://scholar.google.com/scholar?q=Alla+Rozovskaya%2C+Kai%2DWei+Chang%2C+Mark+Sammons%2C+Dan+Roth%2C+and+Nizar+Habash%2E+The+Illinois%2DColumbia+system+in+the+CoNLL%2D2014+shared+task%2E+In+Proc%2E+of+CoNLL+Shared+Task%2C+2014%2E)
18. Chengjie Sun, Xiaoqiang Jin, Lei Lin, Yuming Zhao, and Xiaolong Wang. Convolutional neural networks for correcting English article errors. In Proceedings of the 4th CCF Conference on Natural Language Processing and Chinese Computing, 2015. [find](https://scholar.google.com/scholar?q=Chengjie+Sun%2C+Xiaoqiang+Jin%2C+Lei+Lin%2C+Yuming+Zhao%2C+and+Xiaolong+Wang%2E+Convolutional+neural+networks+for+correcting+English+article+errors%2E+In+Proceedings+of+the+4th+CCF+Conference+on+Natural+Language+)
19. Raymond Hendy Susanto, Peter Phandi, and Hwee Tou Ng. System combination for grammatical error correction. In Proc. of EMNLP, 2014. [find](https://scholar.google.com/scholar?q=Raymond+Hendy+Susanto%2C+Peter+Phandi%2C+and+Hwee+Tou+Ng%2E+System+combination+for+grammatical+error+correction%2E+In+Proc%2E+of+EMNLP%2C+2014%2E)
20. Ashish Vaswani, Yinggong Zhao, Victoria Fossum, and David Chiang. Decoding with large-scale neural language models improves translation. In Proc. of EMNLP, 2013. [find](https://scholar.google.com/scholar?q=Ashish+Vaswani%2C+Yinggong+Zhao%2C+Victoria+Fossum%2C+and+David+Chiang%2E+Decoding+with+large%2Dscale+neural+language+models+improves+translation%2E+In+Proc%2E+of+EMNLP%2C+2013%2E)
21. Jian-Cheng Wu, Tzu-Hsi Yen, Jim Chang, Guan-Cheng Huang, Jimmy Chang, Hsiang-Ling Hsu, Yu-Wei Chang, and Jason S. Chang. NTHU at the CoNLL-2014 shared task. In Proc. of CoNLL Shared Task, 2014. [find](https://scholar.google.com/scholar?q=Jian%2DCheng+Wu%2C+Tzu%2DHsi+Yen%2C+Jim+Chang%2C+Guan%2DCheng+Huang%2C+Jimmy+Chang%2C+Hsiang%2DLing+Hsu%2C+Yu%2DWei+Chang%2C+and+Jason+S%2E+Chang%2E+NTHU+at+the+CoNLL%2D2014+shared+task%2E+In+Proc%2E+of+CoNLL+Shared+Task%2C+2014%2E)
22. Zheng Yuan and Ted Briscoe. Grammatical error correction using neural machine translation. In Proc. of NAACL, 2016. [find](https://scholar.google.com/scholar?q=Zheng+Yuan+and+Ted+Briscoe%2E+Grammatical+error+correction+using+neural+machine+translation%2E+In+Proc%2E+of+NAACL%2C+2016%2E)