---
title: "A Systematic Review of Automated Grammar Checking in"
description: "The survey asks how far automatic detection and correction of English grammar errors had progressed… 26 references, cited by 3 papers here."
authors: ["Madhvi Soni", "Jitendra Singh Thakur"]
year: 2018
venue: "arXiv:1804.00540"
source: "https://arxiv.org/abs/1804.00540"
doi: "10.48550/arxiv.1804.00540"
arxiv: "1804.00540"
licence: "CC BY 4.0 (arXiv)"
licence_url: "http://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/soni-2018-systematic-review-automated-grammar-checking/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/soni-2018-systematic-review-automated-grammar-checking/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2018](https://md.research.elamur.ai/free/en/papers/2018/page/5/) / A Systematic Review of Automated Grammar Checking in English Language

# A Systematic Review of Automated Grammar Checking in English Language

**Authors:** Madhvi Soni and Jitendra Singh Thakur · **Year:** 2018 · **Venue:** arXiv:1804.00540 · **Licence:** [CC BY 4.0 (arXiv)](http://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://arxiv.org/abs/1804.00540) · [PDF](https://arxiv.org/pdf/1804.00540) · [DOI 10.48550/arxiv.1804.00540](https://doi.org/10.48550/arxiv.1804.00540) · [arXiv 1804.00540](https://arxiv.org/abs/1804.00540) · [OpenAlex](https://openalex.org/W2796193757)

**Citations:** 25 citations (OpenAlex, 2026-10-14) · 3 in our library, of the 141,840 papers whose bibliographies we read

## Summary
The survey asks how far automatic detection and correction of English grammar errors had progressed, mainly to serve non-native writers, and where the gaps remained. The authors ran a formal systematic review. They wrote ten research questions, built a Google Scholar query from population, intervention and outcome keyword groups, and filtered the hits down to 12 primary studies. For each study they recorded the technique, workflow, target errors, data, results, tool availability, strengths and weaknesses in a shared table. They sorted the approaches into rule-based, machine-learning and hybrid families. They also proposed their own error taxonomy, because earlier taxonomies from Naber, Wagner et al., Lee et al. and Yuan disagree with one another.

The main finding is that the field is fragmented. Every system targets a different subset of errors and reports results on its own test data, so the numbers cannot be compared. Most tools are not released. Errors at the sentence level and the semantic level are rarely tackled. The authors argue for shared test sets and evaluation broken down by error type.

## Key points
- **Paper selection:** 113 candidate papers were found. After removing 35 duplicates, 36 papers rejected on abstract or introduction, 29 rejected on full text, and 1 more rejected for missing technical detail, 12 studies remained. For the CoNLL-2013 and CoNLL-2014 shared tasks, only the top system was kept.
- **Error taxonomy:** five top-level categories, each with subtypes:
  - sentence structure,
  - punctuation,
  - spelling,
  - syntax,
  - semantic.
- **Criteria behind the taxonomy:** how frequent the error is, what makes the text invalid, the span needed to detect it (word window vs. whole sentence), how hard it is to detect, and keeping overlap between categories small.
- **Technique trade-offs:**
  - Rule-based systems suit language learning, but writing the rules is laborious.
  - Statistical systems save that labour, but how well they work hinges on how large the training corpus is and what kind of text it contains.
  - Hybrids need each component placed where it fits best.
- **Data and results:** the reviewed work draws on NUCLE, Gigaword, Google Web 1T 5-gram, CLC, FCE, EVP and the CoNLL-2014 data. The hybrid system of Felice et al. handles 28 error types. Reported precision is typically well under 70% and recall around 20–35%. Run-on sentences remain essentially unaddressed.

## Interesting ideas
- Tie each error type to the smallest span that reveals it: a preposition slip shows up in a few neighbouring words, while a fragment needs the shape of the full parse tree. This lets cheap local checks run before expensive global ones.
- Label an error by what it violates (spelling conventions, grammar rules, sentence-structure rules, or meaning) rather than by surface symptom. The label then points toward the right fix.
- Instead of a single overall score, evaluate on a shared test set where every faulty sentence carries an error-type label, so the best method for each type becomes visible.
- Rule-based checking is valued for teaching because every flag can be explained, so statistical parts should be kept only where rules are impractical.

## For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

- **I1** (tree): sort tree sanity checks into local head/dependent window rules and whole-tree pattern rules, and run the cheap ones first so each flagged problem says which scope caught it.
- **I2** (learn): tag every self-induced repair rule and every silver-data disagreement with a violation cause, so repairs can be grouped and audited by the kind of rule broken.
- **I3** (learn): add type-labelled corrupted sentences as negative controls and report accuracy per corruption type, showing which pipeline stage fails on which kind of damage.
- **I4** (answer): when a student-facing reader marks an answer wrong, prefer an explicit rule trace and fall back to learned scores only with a stated reason.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 9,085 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 8,888 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 9,296 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 9,206 works

🧭 Topics (2): [English](https://md.research.elamur.ai/free/en/topics/english/) · 1,373 works · [🖍️ Grammatical error correction & learner language](https://md.research.elamur.ai/free/en/topics/grammatical-error-correction/) · 80 works

## Cite

Cite as: Madhvi Soni and Jitendra Singh Thakur. 2018. A Systematic Review of Automated Grammar Checking in English Language. arXiv:1804.00540.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@misc{soni2018systematic,
  title = {{A Systematic Review of Automated Grammar Checking in English Language}},
  author = {Madhvi Soni and Jitendra Singh Thakur},
  year = {2018},
  doi = {10.48550/arxiv.1804.00540},
  eprint = {1804.00540},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.1804.00540}
}
```

- [← previous Primary and secondary discourse connectives: definitions and lexicons](https://md.research.elamur.ai/free/en/papers/danlos-2018-primary-secondary-discourse-connectives-definitions/)
- [next → IronyMagnet at SemEval-2018 Task 3: A Siamese network for Irony detection in Social media](https://md.research.elamur.ai/free/en/papers/ghosh-2018-ironymagnet-semeval-2018-task-3/)
- [↑ back to papers of 2018](https://md.research.elamur.ai/free/en/papers/2018/page/5/)

## References

26 references, 1 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Emily M Bender, Dan Flickinger, Stephan Oepen, Annemarie Walsh, and Timothy Baldwin. 2004. Arboretum: Using a precision grammar for grammar checking in CALL. In Instil/icall symposium 2004. [find](https://scholar.google.com/scholar?q=Emily+M+Bender%2C+Dan+Flickinger%2C+Stephan+Oepen%2C+Annemarie+Walsh%2C+and+Timothy+Baldwin%2E+2004%2E+Arboretum%3A+Using+a+precision+grammar+for+grammar+checking+in+CALL%2E+In+Instil%2Ficall+symposium+2004%2E)
2. Johnny Bigert. 2004. Probabilistic Detection of Context-Sensitive Spelling Errors.. In LREC. [find](https://scholar.google.com/scholar?q=Johnny+Bigert%2E+2004%2E+Probabilistic+Detection+of+Context%2DSensitive+Spelling+Errors%2E%2E+In+LREC%2E)
3. Chris Brockett, William B Dolan, and Michael Gamon. 2006. [Correcting ESL errors using phrasal SMT techniques](https://md.research.elamur.ai/free/en/papers/brockett-2006-correcting-esl-errors-phrasal-smt/). In Proceedings of the 21st International Conference on Computational Linguistics and the 44th annual meeting of the Association for Computational Linguistics. Association for Computational Linguistics, 249–256.
4. Martin Chodorow, Joel R Tetreault, and Na-Rae Han. 2007. Detection of grammatical errors involving prepositions. In Proceedings of the fourth ACL-SIGSEM workshop on prepositions. Association for Computational Linguistics, 25–30. [find](https://scholar.google.com/scholar?q=Martin+Chodorow%2C+Joel+R+Tetreault%2C+and+Na%2DRae+Han%2E+2007%2E+Detection+of+grammatical+errors+involving+prepositions%2E+In+Proceedings+of+the+fourth+ACL%2DSIGSEM+workshop+on+prepositions%2E+Association+for+Compu)
5. Daniel Dahlmeier and Hwee Tou Ng. 2011. Grammatical error correction with alternating structure optimization. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies-Volume 1. Association for Computational Linguistics, 915–923. [find](https://scholar.google.com/scholar?q=Daniel+Dahlmeier+and+Hwee+Tou+Ng%2E+2011%2E+Grammatical+error+correction+with+alternating+structure+optimization%2E+In+Proceedings+of+the+49th+Annual+Meeting+of+the+Association+for+Computational+Linguistics)
6. Mariano Felice, Zheng Yuan, Øistein E Andersen, Helen Yannakoudakis, and Ekaterina Kochmar. 2014. Grammatical error correction using hybrid systems and type filtering.. In CoNLL Shared Task. 15–24. [find](https://scholar.google.com/scholar?q=Mariano+Felice%2C+Zheng+Yuan%2C+%C3%98istein+E+Andersen%2C+Helen+Yannakoudakis%2C+and+Ekaterina+Kochmar%2E+2014%2E+Grammatical+error+correction+using+hybrid+systems+and+type+filtering%2E%2E+In+CoNLL+Shared+Task%2E+15%E2%80%9324%2E)
7. AS Hornby. 1995. Guide to Patterns and Usage in English. (1995). [find](https://scholar.google.com/scholar?q=AS+Hornby%2E+1995%2E+Guide+to+Patterns+and+Usage+in+English%2E+%281995%29%2E)
8. An-Ta Huang, Tsung-Ting Kuo, Ying-Chun Lai, and Shou-De Lin. 2010. Discovering Correction Rules for Auto Editing. Computational Linguistics and Chinese Language Processing 15, 3-4 (2010), 219–236. [↗](https://doi.org/10.30019/ijclclp.201009.0004)
9. Staffs Keele et al. 2007. Guidelines for performing systematic literature reviews in software engineering. In Technical report, Ver. 2.3 EBSE Technical Report. EBSE. sn. [find](https://scholar.google.com/scholar?q=Staffs+Keele+et+al%2E+2007%2E+Guidelines+for+performing+systematic+literature+reviews+in+software+engineering%2E+In+Technical+report%2C+Ver%2E+2%2E3+EBSE+Technical+Report%2E+EBSE%2E+sn%2E)
10. Akshat Kumar and Shivashankar Nair. 2007. An artificial immune system based approach for English grammar checking. Artificial immune systems (2007), 348–357. [find](https://scholar.google.com/scholar?q=Akshat+Kumar+and+Shivashankar+Nair%2E+2007%2E+An+artificial+immune+system+based+approach+for+English+grammar+checking%2E+Artificial+immune+systems+%282007%29%2C+348%E2%80%93357%2E)
11. John Lee and Stephanie Seneff. 2008. Correcting Misuse of Verb Forms.. In ACL. 174–182. [find](https://scholar.google.com/scholar?q=John+Lee+and+Stephanie+Seneff%2E+2008%2E+Correcting+Misuse+of+Verb+Forms%2E%2E+In+ACL%2E+174%E2%80%93182%2E)
12. Nina H Macdonald. 1983. Human factors and behavioral science: The UNIXâĎć Writer’s Workbench software: Rationale and design. Bell Labs Technical Journal 62, 6 (1983), 1891–1908. [find](https://scholar.google.com/scholar?q=Nina+H+Macdonald%2E+1983%2E+Human+factors+and+behavioral+science%3A+The+UNIX%C3%A2%C4%8E%C4%87+Writer%E2%80%99s+Workbench+software%3A+Rationale+and+design%2E+Bell+Labs+Technical+Journal+62%2C+6+%281983%29%2C+1891%E2%80%931908%2E)
13. Maxim Mozgovoy. 2011. Dependency-based rules for grammar checking with LanguageTool. In Computer Science and Information Systems (FedCSIS), 2011 Federated Conference on. IEEE, 209–212. [find](https://scholar.google.com/scholar?q=Maxim+Mozgovoy%2E+2011%2E+Dependency%2Dbased+rules+for+grammar+checking+with+LanguageTool%2E+In+Computer+Science+and+Information+Systems+%28FedCSIS%29%2C+2011+Federated+Conference+on%2E+IEEE%2C+209%E2%80%93212%2E)
14. Daniel Naber. 2003. A rule-based style and grammar checker. (2003). [find](https://scholar.google.com/scholar?q=Daniel+Naber%2E+2003%2E+A+rule%2Dbased+style+and+grammar+checker%2E+%282003%29%2E)
15. Michael Neuman. 1991. RightWriter 3.1. (1991). [find](https://scholar.google.com/scholar?q=Michael+Neuman%2E+1991%2E+RightWriter+3%2E1%2E+%281991%29%2E)
16. Jong C Park, Martha Stone Palmer, and Clay Washburn. 1997. An English Grammar Checker as a Writing Aid for Students of English as a Second Language.. In ANLP. 24. [find](https://scholar.google.com/scholar?q=Jong+C+Park%2C+Martha+Stone+Palmer%2C+and+Clay+Washburn%2E+1997%2E+An+English+Grammar+Checker+as+a+Writing+Aid+for+Students+of+English+as+a+Second+Language%2E%2E+In+ANLP%2E+24%2E)
17. Alla Rozovskaya, Kai-Wei Chang, Mark Sammons, and Dan Roth. 2013. The University of Illinois system in the CoNLL-2013 shared task. In Proceedings of the Seventeenth Conference on Computational Natural Language Learning: Shared Task. 13–19. [find](https://scholar.google.com/scholar?q=Alla+Rozovskaya%2C+Kai%2DWei+Chang%2C+Mark+Sammons%2C+and+Dan+Roth%2E+2013%2E+The+University+of+Illinois+system+in+the+CoNLL%2D2013+shared+task%2E+In+Proceedings+of+the+Seventeenth+Conference+on+Computational+Natural)
18. Alla Rozovskaya, Kai-Wei Chang, Mark Sammons, Dan Roth, and Nizar Habash. 2014. The Illinois-Columbia System in the CoNLL-2014 Shared Task.. In CoNLL Shared Task. 34–42. [find](https://scholar.google.com/scholar?q=Alla+Rozovskaya%2C+Kai%2DWei+Chang%2C+Mark+Sammons%2C+Dan+Roth%2C+and+Nizar+Habash%2E+2014%2E+The+Illinois%2DColumbia+System+in+the+CoNLL%2D2014+Shared+Task%2E%2E+In+CoNLL+Shared+Task%2E+34%E2%80%9342%2E)
19. Grigori Sidorov, Anubhav Gupta, Martin Tozer, Dolors Catala, Angels Catena, and Sandrine Fuentes. 2013. Rule-based System for Automatic Grammar Correction Using Syntactic N-grams for English Language Learning (L2).. In CoNLL Shared Task. 96–101. [find](https://scholar.google.com/scholar?q=Grigori+Sidorov%2C+Anubhav+Gupta%2C+Martin+Tozer%2C+Dolors+Catala%2C+Angels+Catena%2C+and+Sandrine+Fuentes%2E+2013%2E+Rule%2Dbased+System+for+Automatic+Grammar+Correction+Using+Syntactic+N%2Dgrams+for+English+Language+)
20. Guihua Sun, Xiaohua Liu, Gao Cong, Ming Zhou, Zhongyang Xiong, John Lee, and Chin-Yew Lin. 2007. Detecting erroneous sentences using automatically mined sequential patterns. In ACL. 23–30. [find](https://scholar.google.com/scholar?q=Guihua+Sun%2C+Xiaohua+Liu%2C+Gao+Cong%2C+Ming+Zhou%2C+Zhongyang+Xiong%2C+John+Lee%2C+and+Chin%2DYew+Lin%2E+2007%2E+Detecting+erroneous+sentences+using+automatically+mined+sequential+patterns%2E+In+ACL%2E+23%E2%80%9330%2E)
21. Cornelia Tschichold, Franck Bodmer, Etienne Cornu, Franqois Grosjean, Lysiane Grosjean, N Kübler, N Léwy, and Corinne Tschumi. 1997. Developing a new grammar checker for English as a second language. Proc. From Research to Commercial Applications: Making NLP Work in Practice (1997), 7–12. [find](https://scholar.google.com/scholar?q=Cornelia+Tschichold%2C+Franck+Bodmer%2C+Etienne+Cornu%2C+Franqois+Grosjean%2C+Lysiane+Grosjean%2C+N+K%C3%BCbler%2C+N+L%C3%A9wy%2C+and+Corinne+Tschumi%2E+1997%2E+Developing+a+new+grammar+checker+for+English+as+a+second+language%2E+)
22. Joachim Wagner, Jennifer Foster, and Josef van Genabith. 2007. A comparative evaluation of deep and shallow approaches to the automatic detection of common grammatical errors. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Joachim+Wagner%2C+Jennifer+Foster%2C+and+Josef+van+Genabith%2E+2007%2E+A+comparative+evaluation+of+deep+and+shallow+approaches+to+the+automatic+detection+of+common+grammatical+errors%2E+Association+for+Computat)
23. PC Wren and H Martin. 2000. English Grammar & Composition. S. Chand & Company Ltd (2000). [find](https://scholar.google.com/scholar?q=PC+Wren+and+H+Martin%2E+2000%2E+English+Grammar+%26+Composition%2E+S%2E+Chand+%26+Company+Ltd+%282000%29%2E)
24. Chak Yan Yeung and John Lee. 2015. Automatic Detection of Sentence Fragments.. In ACL (2). 599–603. [find](https://scholar.google.com/scholar?q=Chak+Yan+Yeung+and+John+Lee%2E+2015%2E+Automatic+Detection+of+Sentence+Fragments%2E%2E+In+ACL+%282%29%2E+599%E2%80%93603%2E)
25. Zheng Yuan. 2017. Grammatical error correction in non-native English. Technical Report. University of Cambridge, Computer Laboratory. [find](https://scholar.google.com/scholar?q=Zheng+Yuan%2E+2017%2E+Grammatical+error+correction+in+non%2Dnative+English%2E+Technical+Report%2E+University+of+Cambridge%2C+Computer+Laboratory%2E)
26. Tao Yue, Lionel C Briand, and Yvan Labiche. 2011. A systematic review of transformation approaches between user requirements and analysis models. Requirements Engineering 16, 2 (2011), 75–99. [↗](https://doi.org/10.1007/s00766-010-0111-y)

*Updated 24 October 2026*