---
title: "Using Contextual Speller Techniques and Language Modeling for"
description: "The paper tackles automatic proofing for non-native (ESL) writers. 58.36% accuracy on Reuters. 23 references, cited by 65 papers here."
authors: ["Michael Gamon", "Jianfeng Gao", "Chris Brockett", "Alexandre Klementiev", "William B. Dolan", "Dmitriy Belenko", "Lucy Vanderwende"]
year: 2008
venue: "Proceedings of the Third International Joint Conference on Natural Language Processing: Volume-I"
source: "https://aclanthology.org/I08-1059/"
licence: "CC BY-NC-SA 3.0 (ACL Anthology, before 2016)"
licence_url: "https://creativecommons.org/licenses/by-nc-sa/3.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/gamon-2008-contextual-speller-techniques-language-modeling/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/gamon-2008-contextual-speller-techniques-language-modeling/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2008](https://md.research.elamur.ai/free/en/papers/2008/) / Using Contextual Speller Techniques and Language Modeling for ESL Error Correction

# Using Contextual Speller Techniques and Language Modeling for ESL Error Correction

**Authors:** [Michael Gamon](https://md.research.elamur.ai/free/en/authors/michael-gamon/), [Jianfeng Gao](https://md.research.elamur.ai/free/en/authors/jianfeng-gao/), [Chris Brockett](https://md.research.elamur.ai/free/en/authors/chris-brockett/), [Alexandre Klementiev](https://md.research.elamur.ai/free/en/authors/alexandre-klementiev/), [William B. Dolan](https://md.research.elamur.ai/free/en/authors/william-b-dolan/), Dmitriy Belenko and [Lucy Vanderwende](https://md.research.elamur.ai/free/en/authors/lucy-vanderwende/) · **Year:** 2008 · **Venue:** [Proceedings of the Third International Joint Conference on Natural Language Processing: Volume-I](https://md.research.elamur.ai/free/en/venues/ijcnlp/) ([IJCNLP 2008](https://md.research.elamur.ai/free/en/venues/ijcnlp/2008/)) · **Licence:** [CC BY-NC-SA 3.0 (ACL Anthology, before 2016)](https://creativecommons.org/licenses/by-nc-sa/3.0/)

**Links:** [Publisher's page](https://aclanthology.org/I08-1059/) · [PDF](https://aclanthology.org/I08-1059.pdf) · [OpenAlex](https://openalex.org/W2401451642)

**Citations:** 139 citations (OpenAlex, 2026-10-14) · [65](https://md.research.elamur.ai/free/en/papers/gamon-2008-contextual-speller-techniques-language-modeling/cited-by/) in our library, of the 141,840 papers whose bibliographies we read

## Summary
The paper tackles automatic proofing for non-native (ESL) writers. It focuses on determiner and preposition errors, which standard grammar checkers built for native speakers mostly ignore. Annotated learner-error data was not available, so the authors trained on well-formed native text: Encarta and Reuters sentences. They borrowed the contextual-speller idea. At each candidate slot, one decision-tree classifier decides whether a determiner or preposition belongs there, and a second picks which one, using words and POS tags from the surrounding context. A large 5-gram language model built on Gigaword then checks the suggestions and drops those that do not beat the original sentence. An optional module retrieves web sentences that show the suggested wording in use. A judge reviewed the system's flags on CLEC learner text and found that about 55% of article corrections and 46% of preposition corrections were valid fixes. The work matters because it combines detection, correction and example-based support into one practical tool for non-native writers.

## Key points
- Determiner and preposition errors dominate learner data. In the NICT JLE corpus, 26.6% of errors involve determiners and about 10% involve prepositions.
- **Classifiers:**
  - Training data: Encarta (560k sentences) plus 1M Reuters sentences.
  - Features: a ±6-token window of words, POS tags and relative positions.
  - Feature pruning: a count cutoff of 10, then the top 75K features by log likelihood ratio.
  - Model: WinMine decision trees, which beat linear SVMs and were cheaper to train.
- **Language model:** a 5-gram LM on English Gigaword (LDC2005T12) with interpolated Kneser-Ney smoothing and no count cutoff. It has a 120K-word vocabulary and about 12 billion 5-grams.
- **Effect of the LM filter:**
  - Keeping only suggestions that score higher than the original cut preposition flags by 66.8% and determiner flags by 50.7%, and raised precision sharply.
  - Ranking prepositions by LM score alone reached only 58.36% accuracy on Reuters.
- **Human evaluation on CLEC** (6k sentences for articles, 8k for prepositions):
  - Valid corrections: 55% for articles and 46% for prepositions.
  - Wrongly flagged correct text: 16% for articles and 21% for prepositions.
  - Most article fixes were insertions (79%), while most preposition fixes were substitutions (76%).

## Interesting ideas
- **Native text as a substitute for error-annotated data.** The paper treats correction like a contextual speller: it learns what normally appears at a slot from clean text, then flags mismatches in user text. This fits any low-resource language where learner corpora are missing but clean text is plentiful.
- **Separate "should something be here?" from "which item?".** Splitting presence/absence from choice lets each classifier specialize. It also maps directly onto insertion, deletion and substitution edits.
- **Two-stage filtering: classifier, then LM.** A cheap local classifier proposes candidates, and a large LM vetoes those that do not improve fluency. Combining class probability and LM score into one tunable threshold lets you trade recall for precision, which matters for user-facing tools.
- **Show real examples instead of only verdicts.** Some choices depend on meaning (such as *a* vs *the*), so the system offers retrieved example sentences. It favours short ones and those sharing words with the user's input, so the writer can decide.
- **Fine-grained human error categories.** The evaluation goes beyond correct or incorrect. It separates cases such as "right spot, wrong fix", "both versions fine" and "nearby spelling error", which shows more clearly where a system fails.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Microsoft](https://md.research.elamur.ai/free/en/companies/microsoft/) — Company, Redmond, United States · 240 works · 123,537 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 9,086 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 8,889 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 9,296 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 9,206 works

🧭 Topics (2): [🧠 Language models & pretraining](https://md.research.elamur.ai/free/en/topics/language-models/) · 1,343 works · [🖍️ Grammatical error correction & learner language](https://md.research.elamur.ai/free/en/topics/grammatical-error-correction/) · 80 works

## Cite

Cite as: Michael Gamon, Jianfeng Gao, Chris Brockett, Alexandre Klementiev, William B. Dolan, Dmitriy Belenko and Lucy Vanderwende. 2008. Using Contextual Speller Techniques and Language Modeling for ESL Error Correction. Proceedings of the Third International Joint Conference on Natural Language Processing: Volume-I.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{gamon2008using,
  title = {{Using Contextual Speller Techniques and Language Modeling for ESL Error Correction}},
  author = {Michael Gamon, Jianfeng Gao, Chris Brockett, Alexandre Klementiev, William B. Dolan, Dmitriy Belenko and Lucy Vanderwende},
  booktitle = {Proceedings of the Third International Joint Conference on Natural Language Processing: Volume-I},
  year = {2008},
  url = {https://aclanthology.org/I08-1059/}
}
```

- [← previous Lattice Minimum Bayes-Risk Decoding for Statistical Machine Translation](https://md.research.elamur.ai/free/en/papers/tromble-2008-lattice-minimum-bayes-risk-decoding/)
- [next → Tree Kernels for Semantic Role Labeling](https://md.research.elamur.ai/free/en/papers/moschitti-2008-tree-kernels-semantic-role-labeling/)
- [↑ back to papers of 2008](https://md.research.elamur.ai/free/en/papers/2008/)

## References

23 references, 2 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Bond, Francis. 2005. Translating the Untranslatable: A Solution to the Problem of Generating English Determiners. CSLI Publications. [find](https://scholar.google.com/scholar?q=Bond%2C+Francis%2E+2005%2E+Translating+the+Untranslatable%3A+A+Solution+to+the+Problem+of+Generating+English+Determiners%2E+CSLI+Publications%2E)
2. Charniak, Eugene. 2001. [Immediate-head parsing for language models](https://md.research.elamur.ai/free/en/papers/charniak-2001-immediate-head-parsing-language-models/). In Proceedingsof the 39th Annual Meeting of the Association for Computational Linguistics, pp 116-123.
3. Chickering, David Maxwell. 2002. The WinMine Toolkit. Microsoft Technical Report 2002-103. [find](https://scholar.google.com/scholar?q=Chickering%2C+David+Maxwell%2E+2002%2E+The+WinMine+Toolkit%2E+Microsoft+Technical+Report+2002%2D103%2E)
4. Chodorow, Martin, Joel R. Tetreault and Na-Rae Han. 2007. Detection of Grammatical Errors Involving Prepositions. In Proceedings of the 4th ACL-SIGSEM Workshop on Prepositions, pp 25-30. [find](https://scholar.google.com/scholar?q=Chodorow%2C+Martin%2C+Joel+R%2E+Tetreault+and+Na%2DRae+Han%2E+2007%2E+Detection+of+Grammatical+Errors+Involving+Prepositions%2E+In+Proceedings+of+the+4th+ACL%2DSIGSEM+Workshop+on+Prepositions%2C+pp+25%2D30%2E)
5. Crystal, David. 1997. University Press. [find](https://scholar.google.com/scholar?q=Crystal%2C+David%2E+1997%2E+University+Press%2E)
6. Global English. Cambridge Rachele De Felice and Stephen G Pulman. 2007. Automatically acquiring models of preposition use. Proceedings of the ACL-07 Workshop on Prepositions. [find](https://scholar.google.com/scholar?q=Global+English%2E+Cambridge+Rachele+De+Felice+and+Stephen+G+Pulman%2E+2007%2E+Automatically+acquiring+models+of+preposition+use%2E+Proceedings+of+the+ACL%2D07+Workshop+on+Prepositions%2E)
7. Dunning, Ted. 1993. [Accurate Methods for the Statistics of Surprise and Coincidence](https://md.research.elamur.ai/free/en/papers/dunning-1993-accurate-methods-statistics-surprise-coincidence/). Computational Linguistics, 19:61-74.
8. Gamon, Michael, Eric Ringger, and Simon Corston-Oliver. 2002. Amalgam: A machine-learned generation module. Microsoft Technical Report, MSR-TR-2002-57. [find](https://scholar.google.com/scholar?q=Gamon%2C+Michael%2C+Eric+Ringger%2C+and+Simon+Corston%2DOliver%2E+2002%2E+Amalgam%3A+A+machine%2Dlearned+generation+module%2E+Microsoft+Technical+Report%2C+MSR%2DTR%2D2002%2D57%2E)
9. Golding, Andrew R. and Dan Roth. 1999. A Winnow Based Approach to Context-Sensitive Spelling Correction. Machine Learning, pp. 107-130. [find](https://scholar.google.com/scholar?q=Golding%2C+Andrew+R%2E+and+Dan+Roth%2E+1999%2E+A+Winnow+Based+Approach+to+Context%2DSensitive+Spelling+Correction%2E+Machine+Learning%2C+pp%2E+107%2D130%2E)
10. Gui, Shicun and Huizhong Yang (eds.). 2003. Zhongguo Xuexizhe Yingyu Yuliaohu. (Chinese Learner English Corpus). Shanghai Waiyu Jiaoyu Chubanshe.. [find](https://scholar.google.com/scholar?q=Gui%2C+Shicun+and+Huizhong+Yang+%28eds%2E%29%2E+2003%2E+Zhongguo+Xuexizhe+Yingyu+Yuliaohu%2E+%28Chinese+Learner+English+Corpus%29%2E+Shanghai+Waiyu+Jiaoyu+Chubanshe%2E%2E)
11. Han, Na-Rae., Chodorow, Martin and Claudia Leacock. 2004. Detecting errors in English article usage with a maximum entropy classifier trained on a large, diverse corpus. Proceedings of the 4th international conference on language resources and evaluation, Lisbon, Portugal. [find](https://scholar.google.com/scholar?q=Han%2C+Na%2DRae%2E%2C+Chodorow%2C+Martin+and+Claudia+Leacock%2E+2004%2E+Detecting+errors+in+English+article+usage+with+a+maximum+entropy+classifier+trained+on+a+large%2C+diverse+corpus%2E+Proceedings+of+the+4th+interna)
12. Han, Na-Rae. Chodorow, Martin., and Claudia Leacock. (2006). Detecting errors in English article usage by non-native speakers. Natural Language Engineering, 12(2), 115-129. [find](https://scholar.google.com/scholar?q=Han%2C+Na%2DRae%2E+Chodorow%2C+Martin%2E%2C+and+Claudia+Leacock%2E+%282006%29%2E+Detecting+errors+in+English+article+usage+by+non%2Dnative+speakers%2E+Natural+Language+Engineering%2C+12%282%29%2C+115%2D129%2E)
13. Heidorn, George. 2000. Intelligent Writing Assistance. In Robert Dale, Herman Moisl, and Harold Somers (eds.). Handbook of Natural Language Processing. Marcel Dekker. pp 181 -207. [find](https://scholar.google.com/scholar?q=Heidorn%2C+George%2E+2000%2E+Intelligent+Writing+Assistance%2E+In+Robert+Dale%2C+Herman+Moisl%2C+and+Harold+Somers+%28eds%2E%29%2E+Handbook+of+Natural+Language+Processing%2E+Marcel+Dekker%2E+pp+181+%2D207%2E)
14. Izumi, Emi, Kiyotaka Uchimoto and Hitoshi Isahara. 2004. The NICT JLE Corpus: Exploiting the Language Learner’s Speech Database for Research and Education. International Journal of the Computer, the Internet and Management 12:2, pp 119 -125. [find](https://scholar.google.com/scholar?q=Izumi%2C+Emi%2C+Kiyotaka+Uchimoto+and+Hitoshi+Isahara%2E+2004%2E+The+NICT+JLE+Corpus%3A+Exploiting+the+Language+Learner%E2%80%99s+Speech+Database+for+Research+and+Education%2E+International+Journal+of+the+Computer%2C+the+I)
15. Kneser, Reinhard. and Hermann Ney. 1995. Improved backing-off for m-gram language modeling. Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing, volume 1. 1995. pp. 181–184. [find](https://scholar.google.com/scholar?q=Kneser%2C+Reinhard%2E+and+Hermann+Ney%2E+1995%2E+Improved+backing%2Doff+for+m%2Dgram+language+modeling%2E+Proceedings+of+the+IEEE+International+Conference+on+Acoustics%2C+Speech%2C+and+Signal+Processing%2C+volume+1%2E+1995)
16. Knight, Kevin and Ishwar Chander. 1994. Automatic Postediting of Documents. Proceedings of the American Association of Artificial Intelligence, pp 779-784. [find](https://scholar.google.com/scholar?q=Knight%2C+Kevin+and+Ishwar+Chander%2E+1994%2E+Automatic+Postediting+of+Documents%2E+Proceedings+of+the+American+Association+of+Artificial+Intelligence%2C+pp+779%2D784%2E)
17. Lee, John. 2004. Automatic Article Restoration. Proceedings of the Human Language Technology Conference of the North American Chapter of the Association for Computational Linguistics, pp. 31-36. [find](https://scholar.google.com/scholar?q=Lee%2C+John%2E+2004%2E+Automatic+Article+Restoration%2E+Proceedings+of+the+Human+Language+Technology+Conference+of+the+North+American+Chapter+of+the+Association+for+Computational+Linguistics%2C+pp%2E+31%2D36%2E)
18. Liu, Ting, Mingh Zhou, JianfengGao, Endong Xun, and Changning Huan. 2000. PENS: A Machine-Aided English Writing System for Chinese Users. Proceedings of ACL 2000, pp 529-536. [find](https://scholar.google.com/scholar?q=Liu%2C+Ting%2C+Mingh+Zhou%2C+JianfengGao%2C+Endong+Xun%2C+and+Changning+Huan%2E+2000%2E+PENS%3A+A+Machine%2DAided+English+Writing+System+for+Chinese+Users%2E+Proceedings+of+ACL+2000%2C+pp+529%2D536%2E)
19. Minnen, Guido, Francis Bond and Ann Copestake. 2000. Memory-Based Learning for Article Generation. Proceedings of the Fourth Conference on Computational Natural Language Learning and of the Second Learning Language in Logic Workshop, pp 43-48. [find](https://scholar.google.com/scholar?q=Minnen%2C+Guido%2C+Francis+Bond+and+Ann+Copestake%2E+2000%2E+Memory%2DBased+Learning+for+Article+Generation%2E+Proceedings+of+the+Fourth+Conference+on+Computational+Natural+Language+Learning+and+of+the+Second+Lea)
20. Suzuki, Hisami and Kristina Toutanova. 2006. Learning to Predict Case Markers in Japanese. Proceedings of COLING-ACL, pp. 1049-1056. [find](https://scholar.google.com/scholar?q=Suzuki%2C+Hisami+and+Kristina+Toutanova%2E+2006%2E+Learning+to+Predict+Case+Markers+in+Japanese%2E+Proceedings+of+COLING%2DACL%2C+pp%2E+1049%2D1056%2E)
21. Toutanova, Kristina and Hisami Suzuki. 2007 Generating Case Markers in Machine Translation. Proceedings of NAACL-HLT. [find](https://scholar.google.com/scholar?q=Toutanova%2C+Kristina+and+Hisami+Suzuki%2E+2007+Generating+Case+Markers+in+Machine+Translation%2E+Proceedings+of+NAACL%2DHLT%2E)
22. Turner, Jenine and Eugene Charniak. 2007. Language Modeling for Determiner Selection. In Human Language Technologies 2007: The Conference of the North American Chapter of the Association for Computational Linguistics; Companion Volume, Short Papers, pp 177-180. [find](https://scholar.google.com/scholar?q=Turner%2C+Jenine+and+Eugene+Charniak%2E+2007%2E+Language+Modeling+for+Determiner+Selection%2E+In+Human+Language+Technologies+2007%3A+The+Conference+of+the+North+American+Chapter+of+the+Association+for+Computati)
23. Yi, Xing, Jianfeng Gao and William B. Dolan. 2008. Web-Based English Proofing System for English as a Second Language Users. To be presented at IJCNLP 2008. [find](https://scholar.google.com/scholar?q=Yi%2C+Xing%2C+Jianfeng+Gao+and+William+B%2E+Dolan%2E+2008%2E+Web%2DBased+English+Proofing+System+for+English+as+a+Second+Language+Users%2E+To+be+presented+at+IJCNLP+2008%2E)

*Updated 24 October 2026*