---
title: "Compositional Sequence Labeling Models for Error Detection in"
description: "The best design is a multi-layer bidirectional LSTM, which conditions every label on the whole sentence. 48 references, cited by 52 papers here."
authors: ["Marek Rei", "Helen Yannakoudakis"]
year: 2016
venue: "Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)"
source: "https://aclanthology.org/P16-1112/"
doi: "10.18653/v1/p16-1112"
arxiv: "1607.06153"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/rei-2016-compositional-sequence-labeling-models-error/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/rei-2016-compositional-sequence-labeling-models-error/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2016](https://md.research.elamur.ai/free/en/papers/2016/page/3/) / Compositional Sequence Labeling Models for Error Detection in Learner Writing

# Compositional Sequence Labeling Models for Error Detection in Learner Writing

**Authors:** [Marek Rei](https://md.research.elamur.ai/free/en/authors/marek-rei/) and [Helen Yannakoudakis](https://md.research.elamur.ai/free/en/authors/helen-yannakoudakis/) · **Year:** 2016 · **Venue:** [Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)](https://md.research.elamur.ai/free/en/venues/acl/) ([ACL 2016](https://md.research.elamur.ai/free/en/venues/acl/2016/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/P16-1112/) · [PDF](https://aclanthology.org/P16-1112.pdf) · [DOI 10.18653/v1/p16-1112](https://doi.org/10.18653/v1/p16-1112) · [arXiv 1607.06153](https://arxiv.org/abs/1607.06153) · [OpenAlex](https://openalex.org/W2478432301)

**Citations:** 102 citations (OpenAlex, 2026-10-14) · [52](https://md.research.elamur.ai/free/en/papers/rei-2016-compositional-sequence-labeling-models-error/cited-by/) in our library, of the 141,840 papers whose bibliographies we read

## Summary
The paper treats grammatical error detection in learner English as a task of its own rather than a side effect of error correction. Each token is labeled as correct or incorrect given its sentence context. The authors compare six neural composition architectures for building token-in-context representations: convolutional, bidirectional Elman-type recurrent, and bidirectional LSTM networks, each in single- and multi-layer form. A CRF++ baseline is included for reference. Experiments use the FCE-public corpus, with error spans converted into token-level labels.

The best design is a multi-layer bidirectional LSTM, which conditions every label on the whole sentence. With extra training data, it surpasses all CoNLL-14 shared task participants on error detection. This is notable because it is a single model, whereas competing systems are mostly ensembles of several components.

When its token probabilities are added as a feature to an existing automated essay scoring system, the scorer improves. With the larger training set, its correlation with gold scores approaches or exceeds that of human annotators. This suggests error detection is useful for real learner-feedback tools.

## Key points
- **Data split:** FCE-public has 77 error types. The authors use 30,953 training sentences (2,222 of them held out for development) and 2,720 test sentences. For a missing word, the error label goes on the token that follows the gap.
- **CoNLL-14 result:** trained on all available data, the Bi-LSTM reaches the highest F0.5 on both CoNLL-14 test annotations. This is roughly a 3% absolute gain over the previous best. The best correction system (P1+P2+S1+S2) was not the best detector (CAMB).
- **Training data:** most of the gain from additional data came from about 8 million tokens written by learners of mixed proficiency. Adding more text from higher-proficiency writers helped only slightly.
- **Essay scoring:** the Bi-LSTM output was added to the deployed SAT scorer (Andersen et al., 2013).

  | System | Pearson r | Spearman ρ |
  |---|---|---|
  | SAT alone | 75.1 | 76.0 |
  | SAT + Bi-LSTM (FCE-public only) | 76.0 | 77.0 |
  | SAT + Bi-LSTM (full data) | 78.0 | 79.9 |
  | Human annotators | 79.6 | 79.2 |

- **Error reduction:** the full-data combination cuts the gap to human performance on Pearson's correlation by 64% (relative error reduction) and beats humans on Spearman's correlation.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [University of Cambridge](https://md.research.elamur.ai/free/en/universities/university-of-cambridge/) — University, Cambridge, United Kingdom · 155 works · 115,562 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 9,085 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 8,888 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 9,296 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 9,206 works

## Cite

Cite as: Marek Rei and Helen Yannakoudakis. 2016. Compositional Sequence Labeling Models for Error Detection in Learner Writing. Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{rei2016compositional,
  title = {{Compositional Sequence Labeling Models for Error Detection in Learner Writing}},
  author = {Marek Rei and Helen Yannakoudakis},
  booktitle = {Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2016},
  doi = {10.18653/v1/p16-1112},
  eprint = {1607.06153},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.18653/v1/p16-1112}
}
```

- [← previous pkudblab at SemEval-2016 Task 6 : A Specific Convolutional Neural Network System for Effective Stance Detection](https://md.research.elamur.ai/free/en/papers/wei-2016-pkudblab-semeval-2016-task-6/)
- [next → The MGB-2 Challenge: Arabic Multi-Dialect Broadcast Media Recognition](https://md.research.elamur.ai/free/en/papers/ali-2016-mgb-2-challenge-arabic-multi/)
- [↑ back to papers of 2016](https://md.research.elamur.ai/free/en/papers/2016/page/3/)

## References

48 references, 12 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Øistein E. Andersen, Helen Yannakoudakis, Fiona Barker, and Tim Parish. 2013. Developing and testing a self-assessment and tutoring system. Proceedings of the Eighth Workshop on Innovative Use of NLP for Building Educational Applications. 53rd Annual Meeting of the Association for Computational Ling… [find](https://scholar.google.com/scholar?q=%C3%98istein+E%2E+Andersen%2C+Helen+Yannakoudakis%2C+Fiona+Barker%2C+and+Tim+Parish%2E+2013%2E+Developing+and+testing+a+self%2Dassessment+and+tutoring+system%2E+Proceedings+of+the+Eighth+Workshop+on+Innovative+Use+of+NLP+)
2. Ciprian Chelba, Tomáš Mikolov, Mike Schuster, Qi Ge, Thorsten Brants, Phillipp Koehn, and Tony Robinson. 2013. [One Billion Word Benchmark for Measuring Progress in Statistical Language Modeling](https://md.research.elamur.ai/free/en/papers/chelba-2013-one-billion-word-benchmark-measuring/). In arXiv preprint.
3. Kyunghyun Cho, Bart van Merrienboer, Dzmitry Bahdanau, and Yoshua Bengio. 2014a. [On the Properties of Neural Machine Translation: Encoder-Decoder Approaches](https://md.research.elamur.ai/free/en/papers/cho-2014-properties-neural-machine-translation-encoder/). In Eighth Workshop on Syntax, Semantics and Structure in Statistical Translation.
4. Kyunghyun Cho, Bart van Merrienboer, Caglar Gulcehre, Dzmitry Bahdanau, Fethi Bougares, Holger Schwenk, and Yoshua Bengio. 2014b. Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation. In Conference on Empirical Methods in Natural Language Processing (EMNLP 20… [find](https://scholar.google.com/scholar?q=Kyunghyun+Cho%2C+Bart+van+Merrienboer%2C+Caglar+Gulcehre%2C+Dzmitry+Bahdanau%2C+Fethi+Bougares%2C+Holger+Schwenk%2C+and+Yoshua+Bengio%2E+2014b%2E+Learning+Phrase+Representations+using+RNN+Encoder%2DDecoder+for+Statisti)
5. Martin Chodorow and Claudia Leacock. 2000. An unsupervised method for detecting grammatical errors. In Proceedings of the first conference on North American chapter of the Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Martin+Chodorow+and+Claudia+Leacock%2E+2000%2E+An+unsupervised+method+for+detecting+grammatical+errors%2E+In+Proceedings+of+the+first+conference+on+North+American+chapter+of+the+Association+for+Computationa)
6. Martin Chodorow, Joel R. Tetreault, and Na-Rae Han. 2007. Detection of grammatical errors involving prepositions. In Proceedings of the 4th ACL-SIGSEM Workshop on Prepositions. [find](https://scholar.google.com/scholar?q=Martin+Chodorow%2C+Joel+R%2E+Tetreault%2C+and+Na%2DRae+Han%2E+2007%2E+Detection+of+grammatical+errors+involving+prepositions%2E+In+Proceedings+of+the+4th+ACL%2DSIGSEM+Workshop+on+Prepositions%2E)
7. Martin Chodorow, Markus Dickinson, Ross Israel, and Joel Tetreault. 2012. Problems in evaluating grammatical error detection systems. In COLING 2012. [find](https://scholar.google.com/scholar?q=Martin+Chodorow%2C+Markus+Dickinson%2C+Ross+Israel%2C+and+Joel+Tetreault%2E+2012%2E+Problems+in+evaluating+grammatical+error+detection+systems%2E+In+COLING+2012%2E)
8. George. E. Dahl, Dong Yu, Li Deng, and Alex Acero. 2012. Context-Dependent Pre-Trained Deep Neural Networks for Large-Vocabulary Speech Recognition. IEEE Transactions on Audio, Speech, and Language Processing, 20. [find](https://scholar.google.com/scholar?q=George%2E+E%2E+Dahl%2C+Dong+Yu%2C+Li+Deng%2C+and+Alex+Acero%2E+2012%2E+Context%2DDependent+Pre%2DTrained+Deep+Neural+Networks+for+Large%2DVocabulary+Speech+Recognition%2E+IEEE+Transactions+on+Audio%2C+Speech%2C+and+Language+Pr)
9. Daniel Dahlmeier, Hwee Tou Ng, and Siew Mei Wu. 2013. Building a large annotated corpus of learner English: The NUS corpus of learner English. Proceedings of the Eighth Workshop on Innovative Use of NLP for Building Educational Applications. [find](https://scholar.google.com/scholar?q=Daniel+Dahlmeier%2C+Hwee+Tou+Ng%2C+and+Siew+Mei+Wu%2E+2013%2E+Building+a+large+annotated+corpus+of+learner+English%3A+The+NUS+corpus+of+learner+English%2E+Proceedings+of+the+Eighth+Workshop+on+Innovative+Use+of+N)
10. Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. 2015. [Neural Machine Translation by Jointly Learning to Align and Translate](https://md.research.elamur.ai/free/en/papers/bahdanau-2014-neural-machine-translation-jointly-learning/). In International Conference on Learning Representations.
11. Robert Dale and Adam Kilgarriff. 2011. [find](https://scholar.google.com/scholar?q=Robert+Dale+and+Adam+Kilgarriff%2E+2011%2E)
12. [Helping Our Own: The HOO 2011 Pilot Shared Task](https://md.research.elamur.ai/free/en/papers/dale-2011-helping-our-own-hoo-2011/). In Proceedings of the 13th European Workshop on Natural Language Generation.
13. Yoshua Bengio, Réjean Ducharme, Pascal Vincent, and Christian Janvin. 2003. [A Neural Probabilistic Language Model](https://md.research.elamur.ai/free/en/papers/bengio-2003-neural-probabilistic-language-model/) Yoshua. Journal of Machine Learning Research, 3.
14. Robert Dale, Ilya Anisimoff, and George Narroway. 2012. [find](https://scholar.google.com/scholar?q=Robert+Dale%2C+Ilya+Anisimoff%2C+and+George+Narroway%2E+2012%2E)
15. [HOO 2012: A report on the Preposition and Determiner Error Correction Shared Task](https://md.research.elamur.ai/free/en/papers/dale-2012-hoo-2012-report-preposition-determiner/). In The Seventh Workshop on Building Educational Applications Using NLP.
16. Christopher Bryant and Hwee Tou Ng. 2015. How Far are We from Fully Automatic High Quality Grammatical Error Correction? In Proceedings of the Jeffrey L. Elman. 1990. Finding structure in time. Cognitive science, 14(2). [find](https://scholar.google.com/scholar?q=Christopher+Bryant+and+Hwee+Tou+Ng%2E+2015%2E+How+Far+are+We+from+Fully+Automatic+High+Quality+Grammatical+Error+Correction%3F+In+Proceedings+of+the+Jeffrey+L%2E+Elman%2E+1990%2E+Finding+structure+in+time%2E+Cognit)
17. Mariano Felice and Ted Briscoe. 2015. Towards a standard evaluation method for grammatical error detection and correction. In The 2015 Annual Conference of the North American Chapter of the ACL. [find](https://scholar.google.com/scholar?q=Mariano+Felice+and+Ted+Briscoe%2E+2015%2E+Towards+a+standard+evaluation+method+for+grammatical+error+detection+and+correction%2E+In+The+2015+Annual+Conference+of+the+North+American+Chapter+of+the+ACL%2E)
18. Mariano Felice, Zheng Yuan, Øistein E. Andersen, Helen Yannakoudakis, and Ekaterina Kochmar. 2014. Grammatical error correction using hybrid systems and type filtering. Conference on Computational Natural Language Learning: Shared Task (CoNLL-2014). [find](https://scholar.google.com/scholar?q=Mariano+Felice%2C+Zheng+Yuan%2C+%C3%98istein+E%2E+Andersen%2C+Helen+Yannakoudakis%2C+and+Ekaterina+Kochmar%2E+2014%2E+Grammatical+error+correction+using+hybrid+systems+and+type+filtering%2E+Conference+on+Computational+Nat)
19. Michael Gamon. 2011. High-Order Sequence Modeling for Language Learner Error Detection. Proceedings of the Sixth Workshop on Innovative Use of NLP for Building Educational Applications. [find](https://scholar.google.com/scholar?q=Michael+Gamon%2E+2011%2E+High%2DOrder+Sequence+Modeling+for+Language+Learner+Error+Detection%2E+Proceedings+of+the+Sixth+Workshop+on+Innovative+Use+of+NLP+for+Building+Educational+Applications%2E)
20. Alex Graves, Navdeep Jaitly, and Abdel Rahman Mohamed. 2013. Hybrid speech recognition with Deep Bidirectional LSTM. In IEEE Workshop on Automatic Speech Recognition and Understanding (ASRU 2013). [find](https://scholar.google.com/scholar?q=Alex+Graves%2C+Navdeep+Jaitly%2C+and+Abdel+Rahman+Mohamed%2E+2013%2E+Hybrid+speech+recognition+with+Deep+Bidirectional+LSTM%2E+In+IEEE+Workshop+on+Automatic+Speech+Recognition+and+Understanding+%28ASRU+2013%29%2E)
21. Na-Rae Han, Martin Chodorow, and Claudia Leacock. 2004. Detecting Errors in English Article Usage with a Maximum Entropy Classifier Trained on a Large, Diverse Corpus. Proceedings of the 4th International Conference on Language Resources and Evaluation. [find](https://scholar.google.com/scholar?q=Na%2DRae+Han%2C+Martin+Chodorow%2C+and+Claudia+Leacock%2E+2004%2E+Detecting+Errors+in+English+Article+Usage+with+a+Maximum+Entropy+Classifier+Trained+on+a+Large%2C+Diverse+Corpus%2E+Proceedings+of+the+4th+Internati)
22. Na-Rae Han, Martin Chodorow, and Claudia Leacock. 2006. Detecting errors in English article usage by non-native speakers. Natural Language Engineering, 12. [find](https://scholar.google.com/scholar?q=Na%2DRae+Han%2C+Martin+Chodorow%2C+and+Claudia+Leacock%2E+2006%2E+Detecting+errors+in+English+article+usage+by+non%2Dnative+speakers%2E+Natural+Language+Engineering%2C+12%2E)
23. Sepp Hochreiter and Jürgen Schmidhuber. 1997. Long Short-term Memory. Neural Computation, 9. [find](https://scholar.google.com/scholar?q=Sepp+Hochreiter+and+Ju%CC%88rgen+Schmidhuber%2E+1997%2E+Long+Short%2Dterm+Memory%2E+Neural+Computation%2C+9%2E)
24. Ozan Irsoy and Claire Cardie. 2014. Opinion Mining with Deep Recurrent Neural Networks. In EMNLP-2014. [find](https://scholar.google.com/scholar?q=Ozan+Irsoy+and+Claire+Cardie%2E+2014%2E+Opinion+Mining+with+Deep+Recurrent+Neural+Networks%2E+In+EMNLP%2D2014%2E)
25. Marcin Junczys-Dowmunt and Roman Grundkiewicz. 2014. The AMU System in the CoNLL-2014 Shared Task: Grammatical Error Correction by Data-Intensive and Feature-Rich Statistical Machine Translation. Proceedings of the Eighteenth Conference on Computational Natural Language Learning: Shared Task (CoNLL-… [find](https://scholar.google.com/scholar?q=Marcin+Junczys%2DDowmunt+and+Roman+Grundkiewicz%2E+2014%2E+The+AMU+System+in+the+CoNLL%2D2014+Shared+Task%3A+Grammatical+Error+Correction+by+Data%2DIntensive+and+Feature%2DRich+Statistical+Machine+Translation%2E+Proc)
26. Nal Kalchbrenner and Phil Blunsom. 2013. [Recurrent Continuous Translation Models](https://md.research.elamur.ai/free/en/papers/kalchbrenner-2013-recurrent-continuous-translation-models/). In Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing (EMNLP 2013).
27. Diederik P. Kingma and Jimmy Lei Ba. 2015. Adam: a Method for Stochastic Optimization. In International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Diederik+P%2E+Kingma+and+Jimmy+Lei+Ba%2E+2015%2E+Adam%3A+a+Method+for+Stochastic+Optimization%2E+In+International+Conference+on+Learning+Representations%2E)
28. Ekaterina Kochmar and Ted Briscoe. 2014. Detecting Learner Errors in the Choice of Content Words Using Compositional Distributional Semantics. In Proceedings of COLING 2014, the 25th International Conference on Computational Linguistics: Technical Papers. [find](https://scholar.google.com/scholar?q=Ekaterina+Kochmar+and+Ted+Briscoe%2E+2014%2E+Detecting+Learner+Errors+in+the+Choice+of+Content+Words+Using+Compositional+Distributional+Semantics%2E+In+Proceedings+of+COLING+2014%2C+the+25th+International+Con)
29. John Lafferty, Andrew McCallum, and Fernando Pereira. 2001. Conditional random fields: Probabilistic models for segmenting and labeling sequence data. In Proceedings of the 18th International Conference on Machine Learning. Citeseer. [find](https://scholar.google.com/scholar?q=John+Lafferty%2C+Andrew+McCallum%2C+and+Fernando+Pereira%2E+2001%2E+Conditional+random+fields%3A+Probabilistic+models+for+segmenting+and+labeling+sequence+data%2E+In+Proceedings+of+the+18th+International+Conferen)
30. Claudia Leacock, Martin Chodorow, Michael Gamon, and Joel R. Tetreault. 2014. Automated Grammatical Error Detection for Language Learners: Second Edition. [find](https://scholar.google.com/scholar?q=Claudia+Leacock%2C+Martin+Chodorow%2C+Michael+Gamon%2C+and+Joel+R%2E+Tetreault%2E+2014%2E+Automated+Grammatical+Error+Detection+for+Language+Learners%3A+Second+Edition%2E)
31. John Lee and Stephanie Seneff. 2008. Correcting misuse of verb forms. In Proceedings of the 46th Annual Meeting of the ACL. [find](https://scholar.google.com/scholar?q=John+Lee+and+Stephanie+Seneff%2E+2008%2E+Correcting+misuse+of+verb+forms%2E+In+Proceedings+of+the+46th+Annual+Meeting+of+the+ACL%2E)
32. Mnih-Thang Luong, Hieu Pham, and Christopher D. Manning. 2015. [Effective Approaches to Attention-based Neural Machine Translation](https://md.research.elamur.ai/free/en/papers/luong-2015-effective-approaches-attention-based-neural/). In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing.
33. Tomáš Mikolov, Stefan Kombrink, Anoop Deoras, Lukáš Burget, and Jan Černocký. 2011. RNNLM-Recurrent neural network language modeling toolkit. In ASRU 2011 Demo Session. [find](https://scholar.google.com/scholar?q=Toma%CC%81s%CC%8C+Mikolov%2C+Stefan+Kombrink%2C+Anoop+Deoras%2C+Luka%CC%81s%CC%8C+Burget%2C+and+Jan+C%CC%8Cernocky%CC%81%2E+2011%2E+RNNLM%2DRecurrent+neural+network+language+modeling+toolkit%2E+In+ASRU+2011+Demo+Session%2E)
34. Tomáš Mikolov, Greg Corrado, Kai Chen, and Jeffrey Dean. 2013. [Efficient Estimation of Word Representations in Vector Space](https://md.research.elamur.ai/free/en/papers/mikolov-2013-efficient-estimation-word-representations-vector/). Proceedings of the International Conference on Learning Representations (ICLR 2013).
35. Ryo Nagata and Kazuhide Nakatani. 2010. Evaluating performance of grammatical error detection to maximize learning effect. [find](https://scholar.google.com/scholar?q=Ryo+Nagata+and+Kazuhide+Nakatani%2E+2010%2E+Evaluating+performance+of+grammatical+error+detection+to+maximize+learning+effect%2E)
36. Coling 2010: Poster Volume. [find](https://scholar.google.com/scholar?q=Coling+2010%3A+Poster+Volume%2E)
37. Hwee Tou Ng, Yuanbin Wu, and Christian Hadiwinoto. 2013. The CoNLL-2013 Shared Task on Grammatical Error Correction. Computational Natural Language Learning (CoNLL), Shared Task. [find](https://scholar.google.com/scholar?q=Hwee+Tou+Ng%2C+Yuanbin+Wu%2C+and+Christian+Hadiwinoto%2E+2013%2E+The+CoNLL%2D2013+Shared+Task+on+Grammatical+Error+Correction%2E+Computational+Natural+Language+Learning+%28CoNLL%29%2C+Shared+Task%2E)
38. Hwee Tou Ng, Siew Mei Wu, Ted Briscoe, Christian Hadiwinoto, Raymond Hendy Susanto, and Christopher Bryant. 2014. The CoNLL-2014 Shared Task on Grammatical Error Correction. In Proceedings of the Eighteenth Conference on Computational Natural Language Learning: Shared Task. [find](https://scholar.google.com/scholar?q=Hwee+Tou+Ng%2C+Siew+Mei+Wu%2C+Ted+Briscoe%2C+Christian+Hadiwinoto%2C+Raymond+Hendy+Susanto%2C+and+Christopher+Bryant%2E+2014%2E+The+CoNLL%2D2014+Shared+Task+on+Grammatical+Error+Correction%2E+In+Proceedings+of+the+Eigh)
39. Diane Nicholls. 2003. The Cambridge Learner Corpus - error coding and analysis for lexicography and ELT. Proceedings of the Corpus Linguistics 2003 Conference. [find](https://scholar.google.com/scholar?q=Diane+Nicholls%2E+2003%2E+The+Cambridge+Learner+Corpus+%2D+error+coding+and+analysis+for+lexicography+and+ELT%2E+Proceedings+of+the+Corpus+Linguistics+2003+Conference%2E)
40. Alla Rozovskaya and Dan Roth. 2010. Training Paradigms for Correcting Errors in Grammar and Usage. [find](https://scholar.google.com/scholar?q=Alla+Rozovskaya+and+Dan+Roth%2E+2010%2E+Training+Paradigms+for+Correcting+Errors+in+Grammar+and+Usage%2E)
41. Human Language Technologies: The 2010 Annual Conference of the North American Chapter of the Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Human+Language+Technologies%3A+The+2010+Annual+Conference+of+the+North+American+Chapter+of+the+Association+for+Computational+Linguistics%2E)
42. Alla Rozovskaya, Mark Sammons, Joshua Gioja, and Dan Roth. 2011. University of Illinois System in HOO Text Correction Shared Task. Proceedings of the 13th European Workshop on Natural Language Generation (ENLG). [find](https://scholar.google.com/scholar?q=Alla+Rozovskaya%2C+Mark+Sammons%2C+Joshua+Gioja%2C+and+Dan+Roth%2E+2011%2E+University+of+Illinois+System+in+HOO+Text+Correction+Shared+Task%2E+Proceedings+of+the+13th+European+Workshop+on+Natural+Language+Generat)
43. Alla Rozovskaya, Kai-Wei Chang, Mark Sammons, and Dan Roth. 2013. The University of Illinois System in the CoNLL-2013 Shared Task. In Proceedings of the Seventeenth Conference on Computational Natural Language Learning: Shared Task. [find](https://scholar.google.com/scholar?q=Alla+Rozovskaya%2C+Kai%2DWei+Chang%2C+Mark+Sammons%2C+and+Dan+Roth%2E+2013%2E+The+University+of+Illinois+System+in+the+CoNLL%2D2013+Shared+Task%2E+In+Proceedings+of+the+Seventeenth+Conference+on+Computational+Natural)
44. Alla Rozovskaya, Kai-Wei Chang, Mark Sammons, Dan Roth, and Nizar Habash. 2014. The Illinois-Columbia System in the CoNLL-2014 Shared Task. Proceedings of the Eighteenth Conference on Computational Natural Language Learning: Shared Task (CoNLL-2014). [find](https://scholar.google.com/scholar?q=Alla+Rozovskaya%2C+Kai%2DWei+Chang%2C+Mark+Sammons%2C+Dan+Roth%2C+and+Nizar+Habash%2E+2014%2E+The+Illinois%2DColumbia+System+in+the+CoNLL%2D2014+Shared+Task%2E+Proceedings+of+the+Eighteenth+Conference+on+Computational+Na)
45. Raymond Hendy Susanto, Peter Phandi, and Hwee Tou Ng. 2014. System Combination for Grammatical Error Correction. Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP-2014). [find](https://scholar.google.com/scholar?q=Raymond+Hendy+Susanto%2C+Peter+Phandi%2C+and+Hwee+Tou+Ng%2E+2014%2E+System+Combination+for+Grammatical+Error+Correction%2E+Proceedings+of+the+2014+Conference+on+Empirical+Methods+in+Natural+Language+Processing+)
46. Joel R. Tetreault and Martin Chodorow. 2008. [The Ups and Downs of Preposition Error Detection in ESL Writing](https://md.research.elamur.ai/free/en/papers/tetreault-2008-ups-downs-preposition-error-detection/). Proceedings of the 22nd International Conference on Computational Linguistics (Coling 2008).
47. Tsung-Hsien Wen, Milica Gasic, Nikola Mrksic, Pei-Hao Su, David Vandyke, and Steve Young. 2015. [Semantically Conditioned LSTM-based Natural Language Generation for Spoken Dialogue Systems](https://md.research.elamur.ai/free/en/papers/wen-2015-semantically-conditioned-lstm-based-natural/).
48. Helen Yannakoudakis, Ted Briscoe, and Ben Medlock. 2011. [A New Dataset and Method for Automatically Grading ESOL Texts](https://md.research.elamur.ai/free/en/papers/yannakoudakis-2011-new-dataset-method-automatically-grading/). In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies.

*Updated 24 October 2026*