---
title: "Experiments on Automatic Error Detection and Correction for"
description: "The authors annotated small samples by hand to build a typology of errors. 0.82 accuracy. 20 references, cited by 3 papers here."
authors: ["Romina Brown", "Santiago Paez", "Gonzalo Herrera", "Luis Chiruzzo", "Aiala Rosá"]
year: 2023
venue: "Proceedings of the 12th Workshop on NLP for Computer Assisted Language Learning"
source: "https://aclanthology.org/2023.nlp4call-1.6/"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/brown-2023-experiments-automatic-error-detection-correction/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/brown-2023-experiments-automatic-error-detection-correction/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2023](https://md.research.elamur.ai/free/en/papers/2023/page/12/) / Experiments on Automatic Error Detection and Correction for Uruguayan Learners of English

# Experiments on Automatic Error Detection and Correction for Uruguayan Learners of English

**Authors:** Romina Brown, Santiago Paez, Gonzalo Herrera, [Luis Chiruzzo](https://md.research.elamur.ai/free/en/authors/luis-chiruzzo/) and [Aiala Rosá](https://md.research.elamur.ai/free/en/authors/aiala-rosa/) · **Year:** 2023 · **Venue:** [Proceedings of the 12th Workshop on NLP for Computer Assisted Language Learning](https://md.research.elamur.ai/free/en/venues/nlp4call/) ([Workshop on Natural Language Processing for Computer Assisted Language Learning 2023](https://md.research.elamur.ai/free/en/venues/nlp4call/2023/)) · **Publisher:** [LiU Electronic Press](https://md.research.elamur.ai/free/en/publishers/liu-electronic-press/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2023.nlp4call-1.6/) · [PDF](https://aclanthology.org/2023.nlp4call-1.6.pdf) · [OpenAlex](https://openalex.org/W4376645554)

**Citations:** 3 citations (OpenAlex, 2026-10-14) · [3](https://md.research.elamur.ai/free/en/papers/brown-2023-experiments-automatic-error-detection-correction/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
The paper looks at automatic grammatical error correction (GEC) and automatic grading for very short English texts by Uruguayan schoolchildren. These are beginners whose mistakes reflect Spanish, and specifically Rioplatense Spanish. The data comes from Ceibal en Inglés: answers from a 2017 national exam, in which children aged 9–11 described a girl in a picture along with her likes and dislikes. Teachers graded each answer from 0 to 6, roughly A0 to B1 on the CEFR scale.

The authors annotated small samples by hand to build a typology of errors. They then wrote rule-based heuristics to detect and fix the most frequent ones, using SymSpell for spelling. The counts produced by these heuristics, together with text length, became features for scikit-learn grading classifiers, which were compared against bag-of-words features.

Combined features grade best. Classifiers that use only error counts and length come close, and they ignore the exercise's vocabulary, which suggests they could transfer to other prompts. The authors stress that this is a proof of concept, since the data covers a single exercise.

## Key points
- **Corpus size and balance.** The corpus has 65,528 texts, about 54k after removing empty and ungraded ones. It is heavily skewed toward low grades: nearly half score 0 or 1, and fewer than 150 score 5 or 6.
- **Annotation samples.** A 53-text development sample was used to build the error typology. A separate 42-text evaluation sample was then annotated by two other researchers using the compiled error list.
- **Dialect-specific spelling errors.**
  - The silent Spanish "h" produces real-word mistakes such as *hare*.
  - The Rioplatense pronunciation of "ll" as /ʃ/ yields non-words such as *llor* for "shorts".
  - Spanish word order also shows up, with adjectives placed after nouns.
- **Features and models.**
  - Spelling: SymSpell alone was both the most accurate and the fastest option tested.
  - Error-count features: eight in total, covering capitalization types, verb form, subject-verb agreement, determiners and others, plus a length feature.
  - Bag-of-words features: the 750 most frequent unigrams and bigrams.
  - Classifiers: NB, RF, ME, SVM and MLP.
- **Results on three grade ranges.**
  - All features combined: RF reaches macro-F1 0.76.
  - Exercise-independent features only: MLP reaches 0.82 accuracy and 0.68 macro-F1.
  - Bag-of-words features overfit to the prompt; top features include "Andrea", "pizza" and "14".

## Interesting ideas
- Spelling candidates can be generated from the writer's L1 sound system (silent h, "ll" read as /ʃ/), not just from edit distance, which recovers words a generic speller would miss.
- The paper separates non-word errors, which a dictionary lookup catches, from real-word errors, which only context reveals.
- Describing a text by counts of error types rather than by its words cuts the grader loose from the prompt vocabulary.
- Inspecting which bag-of-words features dominate ("Andrea", "pizza") exposes overfitting to the prompt, and an exercise-independent feature set serves as a deliberate comparison.
- The annotation protocol has one pair of annotators split the data, cross-check and resolve disagreements to build an error list, after which a different pair applies that list to a held-out sample.

## For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

- **I1** (tokens): a learner-text normalizer could add Spanish-phonology confusion pairs to its candidate list before tagging and lemmatization, logging each rewrite so it stays explainable.
- **I2** (tree): real-word slips like *hare* could be flagged when the UD tree shows an implausible attachment or agreement pattern, and routed to repair instead of being parsed as-is.
- **I3** (answer): student readers could score answers with structural features such as agreement and missing-subject counts, so that topic words cannot inflate a score.
- **I4** (learn): evaluations could routinely include a lexical-only baseline and a lexicon-blind model as negative controls, showing whether a gain comes from structure or from memorized topic words.
- **I5** (learn): the same two-stage scheme could validate LLM-teacher or silver error labels, defining categories on one slice and testing them on a fresh slice with separate annotators.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Universidad de la República de Uruguay](https://md.research.elamur.ai/free/en/universities/universidad-de-la-republica-de-uruguay/) — University, Montevideo, Uruguay · 9 works · 450 citations

📚 Shelves (6): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works · [💭 Psychology](https://md.research.elamur.ai/free/en/shelves/psychology/) · 1,552 works · [💭 Developmental and Educational Psychology](https://md.research.elamur.ai/free/en/shelves/psychology/developmental-and-educational-psychology/) · 499 works

🧭 Topics (1): [English](https://md.research.elamur.ai/free/en/topics/english/) · 2,263 works

## Cite

Cite as: Romina Brown, Santiago Paez, Gonzalo Herrera, Luis Chiruzzo and Aiala Rosá. 2023. Experiments on Automatic Error Detection and Correction for Uruguayan Learners of English. Proceedings of the 12th Workshop on NLP for Computer Assisted Language Learning.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{brown2023experiments,
  title = {{Experiments on Automatic Error Detection and Correction for Uruguayan Learners of English}},
  author = {Romina Brown, Santiago Paez, Gonzalo Herrera, Luis Chiruzzo and Aiala Rosá},
  booktitle = {Proceedings of the 12th Workshop on NLP for Computer Assisted Language Learning},
  year = {2023},
  doi = {10.3384/ecp197006},
  url = {https://doi.org/10.3384/ecp197006}
}
```

- [← previous Evaluating pragmatic abilities of image captioners on A3DS](https://md.research.elamur.ai/free/en/papers/tsvilodub-2023-evaluating-pragmatic-abilities-image-captioners/)
- [next → Exploring the Jungle of Bias: Political Bias Attribution in Language Models via Dependency Analysis](https://md.research.elamur.ai/free/en/papers/jenny-2023-exploring-jungle-bias-political-bias/)
- [↑ back to papers of 2023](https://md.research.elamur.ai/free/en/papers/2023/page/12/)

## References

20 references, 7 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Sagar Ailani, Ashwini Dalvi, and Irfan Siddavatam. 2019. Grammatical error correction (gec): research approaches till now. International Journal of Computer Application, 178(40):1–3. [↗](https://doi.org/10.5120/ijca2019919275)
2. Nicolas Ballier, Thomas Gaillat, Andrew Simpkin, Bernardo Stearns, Manon Bouyé, and Manel Zarrouk. 2019. A supervised learning model for the automatic assessment of language levels based on learner errors. In Transforming Learning with Meaningful Technologies: 14th European Conference on Technology Enhanced Learning, EC-TEL 2019, Delft, The Netherlands, September 16–19, 2019, Proceedings 14, pages 308–320. Springer. [find](https://scholar.google.com/scholar?q=Nicolas+Ballier%2C+Thomas+Gaillat%2C+Andrew+Simpkin%2C+Bernardo+Stearns%2C+Manon+Bouye%CC%81%2C+and+Manel+Zarrouk%2E+2019%2E+A+supervised+learning+model+for+the+automatic+assessment+of+language+levels+based+on+learner+e)
3. Christopher Bryant, Mariano Felice, Øistein E. Andersen, and Ted Briscoe. 2019. [The BEA-2019 shared task on grammatical error correction](https://md.research.elamur.ai/free/en/papers/bryant-2019-bea-2019-shared-task-grammatical/). In Proceedings of the Fourteenth Workshop on Innovative Use of NLP for Building Educational Applications, pages 52–75, Florence, Italy. Association for Computational Linguistics.
4. Luis Chiruzzo, Laura Musto, Santiago Góngora, Brian Carpenter, Juan Filevich, and Aiala Rosá. 2022. [Using nlp to support english teaching in rural schools](https://md.research.elamur.ai/free/en/papers/chiruzzo-2022-nlp-support-english-teaching-rural/). In Proceedings of the Second Workshop on NLP for Positive Impact (NLP4PI), pages 113–121. [https://ceibal.edu.uy/](https://ceibal.edu.uy/) [↗](https://ceibal.edu.uy/)
5. Burr Settles, Chris Brust, Erin Gustafson, Masato Hagiwara, and Nitin Madnani. 2018. Second language acquisition modeling. In Proceedings of the thirteenth workshop on innovative use of NLP for building educational applications, pages 56–65. [find](https://scholar.google.com/scholar?q=Burr+Settles%2C+Chris+Brust%2C+Erin+Gustafson%2C+Masato+Hagiwara%2C+and+Nitin+Madnani%2E+2018%2E+Second+language+acquisition+modeling%2E+In+Proceedings+of+the+thirteenth+workshop+on+innovative+use+of+NLP+for+buildi)
6. Luis Morgado Da Costa, Francis Bond, and Xiaoling He. 2016. Syntactic well-formedness diagnosis and error-based coaching in computer assisted language learning using machine translation. In Proceedings of the 3rd Workshop on Natural Language Processing Techniques for Educational Applications (NLPTEA2016), pages 107–116. [find](https://scholar.google.com/scholar?q=Luis+Morgado+Da+Costa%2C+Francis+Bond%2C+and+Xiaoling+He%2E+2016%2E+Syntactic+well%2Dformedness+diagnosis+and+error%2Dbased+coaching+in+computer+assisted+language+learning+using+machine+translation%2E+In+Proceeding)
7. Michael Swan and Catherine Walter. 2011. Oxford English grammar course. Oxford University Press Oxford. [find](https://scholar.google.com/scholar?q=Michael+Swan+and+Catherine+Walter%2E+2011%2E+Oxford+English+grammar+course%2E+Oxford+University+Press+Oxford%2E)
8. Daniel Dahlmeier, Hwee Tou Ng, and Siew Mei Wu. 2013. [Building a large annotated corpus of learner english: The nus corpus of learner english](https://md.research.elamur.ai/free/en/papers/dahlmeier-2013-building-large-annotated-corpus-learner/). In Proceedings of the eighth workshop on innovative use of NLP for building educational applications, pages 22–31.
9. Joanna Turnbull, D Lea, D Parkinson, P Phillips, B Francis, S Webb, V Bull, and M Ashby. 2010. Oxford advanced learner’s dictionary. International Student’s Edition. [find](https://scholar.google.com/scholar?q=Joanna+Turnbull%2C+D+Lea%2C+D+Parkinson%2C+P+Phillips%2C+B+Francis%2C+S+Webb%2C+V+Bull%2C+and+M+Ashby%2E+2010%2E+Oxford+advanced+learner%E2%80%99s+dictionary%2E+International+Student%E2%80%99s+Edition%2E)
10. Yuzhu Wang and Hai Zhao. 2015. A light rule-based approach to english subject-verb agreement errors on the third person singular forms. In Proceedings of the 29th Pacific Asia Conference on Language, Information and Computation: Posters, pages 345– 353. [find](https://scholar.google.com/scholar?q=Yuzhu+Wang+and+Hai+Zhao%2E+2015%2E+A+light+rule%2Dbased+approach+to+english+subject%2Dverb+agreement+errors+on+the+third+person+singular+forms%2E+In+Proceedings+of+the+29th+Pacific+Asia+Conference+on+Language%2C+)
11. Annette MB De Groot and Rineke Keijzer. 2000. What is hard to learn is easy to forget: The roles of word concreteness, cognate status, and word frequency in foreign-language vocabulary learning and forgetting. Language learning, 50(1):1–56. [↗](https://doi.org/10.1111/0023-8333.00110)
12. Rodrigo Wilkens, Daiane Seibert, Xiaoou Wang, and Thomas François. 2022. [Mwe for essay scoring english as a foreign language](https://md.research.elamur.ai/free/en/papers/wilkens-2022-mwe-essay-scoring-english-as/). In Proceedings of the 2nd Workshop on Tools and Resources to Empower People with REAding DIfficulties (READI) within the 13th Language Resources and Evaluation Conference, pages 62–69.
13. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. [Bert: Pre-training of deep bidirectional transformers for language understanding](https://md.research.elamur.ai/free/en/papers/devlin-2019-bert-pre-training-deep-bidirectional/). arXiv preprint arXiv:1810.04805. [↗](https://arxiv.org/abs/1810.04805)
14. Samuel Gehman, Suchin Gururangan, Maarten Sap, Yejin Choi, and Noah A Smith. 2020. [Realtoxicityprompts: Evaluating neural toxic degeneration in language models](https://md.research.elamur.ai/free/en/papers/gehman-2020-realtoxicityprompts-evaluating-neural-toxic-degeneration/). arXiv preprint arXiv:2009.11462. [↗](https://arxiv.org/abs/2009.11462)
15. Helen Yannakoudakis, Ted Briscoe, and Ben Medlock. 2011. [A new dataset and method for automatically grading esol texts](https://md.research.elamur.ai/free/en/papers/yannakoudakis-2011-new-dataset-method-automatically-grading/). In Proceedings of the 49th annual meeting of the association for computational linguistics: human language technologies, pages 180–189.
16. Ekaterina Kochmar and Ted Briscoe. 2014. Detecting learner errors in the choice of content words using compositional distributional semantics. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Ekaterina+Kochmar+and+Ted+Briscoe%2E+2014%2E+Detecting+learner+errors+in+the+choice+of+content+words+using+compositional+distributional+semantics%2E+Association+for+Computational+Linguistics%2E)
17. Tomoya Mizumoto, Yuta Hayashibe, Mamoru Komachi, Masaaki Nagata, and Yuji Matsumoto. 2012. The effect of learner corpus size in grammatical error correction of esl writings. In Proceedings of COLING 2012: Posters, pages 863–872. [find](https://scholar.google.com/scholar?q=Tomoya+Mizumoto%2C+Yuta+Hayashibe%2C+Mamoru+Komachi%2C+Masaaki+Nagata%2C+and+Yuji+Matsumoto%2E+2012%2E+The+effect+of+learner+corpus+size+in+grammatical+error+correction+of+esl+writings%2E+In+Proceedings+of+COLING+2)
18. Hwee Tou Ng, Siew Mei Wu, Ted Briscoe, Christian Hadiwinoto, Raymond Hendy Susanto, and Christopher Bryant. 2014. The CoNLL-2014 shared task on grammatical error correction. In Proceedings of the Eighteenth Conference on Computational Natural Language Learning: Shared Task, pages 1– 14, Baltimore, Maryland. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Hwee+Tou+Ng%2C+Siew+Mei+Wu%2C+Ted+Briscoe%2C+Christian+Hadiwinoto%2C+Raymond+Hendy+Susanto%2C+and+Christopher+Bryant%2E+2014%2E+The+CoNLL%2D2014+shared+task+on+grammatical+error+correction%2E+In+Proceedings+of+the+Eigh)
19. Diane Nicholls. 2003. The cambridge learner corpus: Error coding and analysis for lexicography and elt. In Proceedings of the Corpus Linguistics 2003 conference, volume 16, pages 572–581. Cambridge University Press Cambridge. [find](https://scholar.google.com/scholar?q=Diane+Nicholls%2E+2003%2E+The+cambridge+learner+corpus%3A+Error+coding+and+analysis+for+lexicography+and+elt%2E+In+Proceedings+of+the+Corpus+Linguistics+2003+conference%2C+volume+16%2C+pages+572%E2%80%93581%2E+Cambridge+Un)
20. Fabian Pedregosa, Gaël Varoquaux, Alexandre Gramfort, Vincent Michel, Bertrand Thirion, Olivier Grisel, Mathieu Blondel, Peter Prettenhofer, Ron Weiss, Vincent Dubourg, et al. 2011. Scikit-learn: Machine learning in python. Journal of machine learning research, 12(Oct):2825–2830. [↗](https://doi.org/10.48550/arxiv.1201.0490)

*Updated 1 November 2026*