---
title: "Building a Large Annotated Corpus of Learner English: The NUS"
description: "The paper describes NUCLE, a fully error-annotated English corpus of texts written by learners of… 15 references, cited by 227 papers here."
authors: ["Daniel Dahlmeier", "Hwee Tou Ng", "Siew Mei Wu"]
year: 2013
venue: "Proceedings of the Eighth Workshop on Innovative Use of NLP for Building Educational Applications"
source: "https://aclanthology.org/W13-1703/"
licence: "CC BY-NC-SA 3.0 (ACL Anthology, before 2016)"
licence_url: "https://creativecommons.org/licenses/by-nc-sa/3.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/dahlmeier-2013-building-large-annotated-corpus-learner/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/dahlmeier-2013-building-large-annotated-corpus-learner/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2013](https://md.research.elamur.ai/free/en/papers/2013/) / Building a Large Annotated Corpus of Learner English: The NUS Corpus of Learner English

# Building a Large Annotated Corpus of Learner English: The NUS Corpus of Learner English

**Authors:** [Daniel Dahlmeier](https://md.research.elamur.ai/free/en/authors/daniel-dahlmeier/), [Hwee Tou Ng](https://md.research.elamur.ai/free/en/authors/hwee-tou-ng/) and Siew Mei Wu · **Year:** 2013 · **Venue:** [Proceedings of the Eighth Workshop on Innovative Use of NLP for Building Educational Applications](https://md.research.elamur.ai/free/en/venues/bea/) ([BEA 2013](https://md.research.elamur.ai/free/en/venues/bea/2013/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY-NC-SA 3.0 (ACL Anthology, before 2016)](https://creativecommons.org/licenses/by-nc-sa/3.0/)

**Links:** [Publisher's page](https://aclanthology.org/W13-1703/) · [PDF](https://aclanthology.org/W13-1703.pdf) · [OpenAlex](https://openalex.org/W2170527467)

**Citations:** 334 citations (OpenAlex, 2026-10-14) · [227](https://md.research.elamur.ai/free/en/papers/dahlmeier-2013-building-large-annotated-corpus-learner/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
The paper describes NUCLE, a fully error-annotated English corpus of texts written by learners of English. It is freely available for research. The corpus was created because training and evaluating automatic grammatical error correction systems lacked a large open resource: existing annotated corpora were either small or closed. NUCLE consists of essays written by students at the National University of Singapore. They were annotated by professional English instructors from CELC using WAMP, a web-based tool developed specifically for this corpus. Each error is assigned a category and a proposed correction, and the marked span is kept as short as possible. The authors also publish, for the first time, a study of inter-annotator agreement. To compare different ways of marking errors fairly, they reduce the annotations to the level of individual tokens. The study showed that grammatical error correction is difficult even for humans. According to the corpus statistics, errors are rare and their distribution has a long tail. The most frequent errors involve prepositions, collocations, and articles. The paper has become the reference description of the corpus and makes it easier for other researchers to work with it.

## Key points
- **Size:** about 1,400 essays, more than one million words. Errors are annotated and corrected throughout the entire corpus.
- **Annotation scheme:** 27 error categories in 13 groups, including Vt, Vform, SVA, ArtOrDet, Nn, Wcip, Rloc, Mec. The scheme is based on a tag set previously developed at CELC. For comparison, the Cambridge Learner Corpus (CLC) has 88 categories.
- **Agreement study:** 100 essays were taken from the pilot data, 4 of which were discarded because of very low quality. Each of the remaining 96 was annotated independently by two annotators, and these essays were not included in the official NUCLE. Before comparison, the marked spans were normalized: they were extended to word boundaries, identical tokens at the edges were trimmed, and the spans were projected onto individual tokens. Tokens were aligned by minimum edit distance.
- **Most frequent errors:** Wcip 15.69 %, Rloc 13.71 %, ArtOrDet 12.88 %, Nn 8.49 %, Mec 7.06 %. These five categories account for 57.83 % of all annotations, and the ten most frequent for 79.66 %.
- **Comparison with other resources:** the annotated portion of ICLE and the Chinese Learner English Corpus in Rozovskaya & Roth has 63 thousand words. The HOO 2011 corpus has roughly 20 thousand words each for training and testing and covers only NLP papers. FCE essays are shorter than NUCLE essays and contain more errors per unit of text.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [National University of Singapore](https://md.research.elamur.ai/free/en/universities/national-university-of-singapore/) — University, Singapore, Singapore · 180 works · 34,304 citations

📚 Shelves (6): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works · [💭 Psychology](https://md.research.elamur.ai/free/en/shelves/psychology/) · 1,552 works · [💭 Developmental and Educational Psychology](https://md.research.elamur.ai/free/en/shelves/psychology/developmental-and-educational-psychology/) · 499 works

🧭 Topics (2): [English](https://md.research.elamur.ai/free/en/topics/english/) · 2,263 works · [🖍️ Grammatical error correction & learner language](https://md.research.elamur.ai/free/en/topics/grammatical-error-correction/) · 156 works

💡 Concepts (1): [💡 Linguistic annotation](https://md.research.elamur.ai/free/en/concepts/annotation/) · 26 works

## Cite

Cite as: Daniel Dahlmeier, Hwee Tou Ng and Siew Mei Wu. 2013. Building a Large Annotated Corpus of Learner English: The NUS Corpus of Learner English. Proceedings of the Eighth Workshop on Innovative Use of NLP for Building Educational Applications.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{dahlmeier2013building,
  title = {{Building a Large Annotated Corpus of Learner English: The NUS Corpus of Learner English}},
  author = {Daniel Dahlmeier, Hwee Tou Ng and Siew Mei Wu},
  booktitle = {Proceedings of the Eighth Workshop on Innovative Use of NLP for Building Educational Applications},
  year = {2013},
  url = {https://aclanthology.org/W13-1703/}
}
```

- [← previous Weakly Supervised Learning of Semantic Parsers for Mapping Instructions to Actions](https://md.research.elamur.ai/free/en/papers/artzi-2013-weakly-supervised-learning-semantic-parsers/)
- [next → Deterministic Coreference Resolution Based on Entity-Centric, Precision-Ranked Rules](https://md.research.elamur.ai/free/en/papers/lee-2013-deterministic-coreference-resolution-based-entity/)
- [↑ back to papers of 2013](https://md.research.elamur.ai/free/en/papers/2013/)

## References

15 references, 3 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. J. Cohen. 1960. A coefficient of agreement for nominal scales. Educational and Psychological Measurement, 20(1):37–46. [↗](https://doi.org/10.1177/001316446002000104)
2. D. Dahlmeier and H.T. Ng. 2012. Better evaluation for grammatical error correction. In Proceedings of HLT-NAACL, pages 568–572. [find](https://scholar.google.com/scholar?q=D%2E+Dahlmeier+and+H%2ET%2E+Ng%2E+2012%2E+Better+evaluation+for+grammatical+error+correction%2E+In+Proceedings+of+HLT%2DNAACL%2C+pages+568%E2%80%93572%2E)
3. R. Dale and A. Kilgarriff. 2011. [Helping Our Own: The HOO 2011 pilot shared task](https://md.research.elamur.ai/free/en/papers/dale-2011-helping-our-own-hoo-2011/). In Proceedings of the Generation Challenges Session at the 13th European Workshop on Natural Language Generation, pages 242–249.
4. R. Dale, I. Anisimoff, and G. Narroway. 2012. [HOO 2012: A report on the preposition and determiner error correction shared task](https://md.research.elamur.ai/free/en/papers/dale-2012-hoo-2012-report-preposition-determiner/). In Proceedings of the Seventh Workshop on Innovative Use of NLP for Building Educational Applications, pages 54–62.
5. S. Granger, F. Dagneaux, E. Meunier, and M. Paquot. 2002. The International Corpus of Learner English. Presses Universitaires de Louvain, Louvain-la-Neuve, Belgium. [find](https://scholar.google.com/scholar?q=S%2E+Granger%2C+F%2E+Dagneaux%2C+E%2E+Meunier%2C+and+M%2E+Paquot%2E+2002%2E+The+International+Corpus+of+Learner+English%2E+Presses+Universitaires+de+Louvain%2C+Louvain%2Dla%2DNeuve%2C+Belgium%2E)
6. S. Gui and H. Yang. 2003. Zhongguo Xuexizhe Yingyu Yuliaohu (Chinese Learner English Corpus). Shanghai Waiyu Jiaoyu Chubanshe. In Chinese. [find](https://scholar.google.com/scholar?q=S%2E+Gui+and+H%2E+Yang%2E+2003%2E+Zhongguo+Xuexizhe+Yingyu+Yuliaohu+%28Chinese+Learner+English+Corpus%29%2E+Shanghai+Waiyu+Jiaoyu+Chubanshe%2E+In+Chinese%2E)
7. J.R. Landis and G.G Koch. 1977. The measurement of observer agreement for categorical data. Biometrics, 33(1):159–174. [↗](https://doi.org/10.2307/2529310)
8. C. Leacock, M. Chodorow, M. Gamon, and J. Tetreault. 2010. Automated Grammatical Error Detection for Language Learners. Morgan & Claypool Publishers. [↗](https://doi.org/10.2200/s00275ed1v01y201006hlt009)
9. J. Lee, J. Tetreault, and M. Chodorow. 2009. Human evaluation of article and noun number usage: Influences of context and construction variability. In Proceedings of the Linguistic Annotation Workshop III (LAW3), pages 60–63. [find](https://scholar.google.com/scholar?q=J%2E+Lee%2C+J%2E+Tetreault%2C+and+M%2E+Chodorow%2E+2009%2E+Human+evaluation+of+article+and+noun+number+usage%3A+Influences+of+context+and+construction+variability%2E+In+Proceedings+of+the+Linguistic+Annotation+Workshop)
10. N. Madnani, J. Tetreault, M. Chodorow, and R. Rozovskaya. 2011. They can help: using crowdsourcing to improve the evaluation of grammatical error detection systems. In Proceedings of ACL:HLT, pages 508–513. [find](https://scholar.google.com/scholar?q=N%2E+Madnani%2C+J%2E+Tetreault%2C+M%2E+Chodorow%2C+and+R%2E+Rozovskaya%2E+2011%2E+They+can+help%3A+using+crowdsourcing+to+improve+the+evaluation+of+grammatical+error+detection+systems%2E+In+Proceedings+of+ACL%3AHLT%2C+pages+50)
11. D. Nicholls. 2003. The Cambridge learner corpus: Error coding and analysis for lexicography and ELT. In Proceedings of the Corpus Linguistics 2003 Conference, pages 572–581. [find](https://scholar.google.com/scholar?q=D%2E+Nicholls%2E+2003%2E+The+Cambridge+learner+corpus%3A+Error+coding+and+analysis+for+lexicography+and+ELT%2E+In+Proceedings+of+the+Corpus+Linguistics+2003+Conference%2C+pages+572%E2%80%93581%2E)
12. A. Rozovskaya and D. Roth. 2010. Annotating ESL errors: Challenges and rewards. In Proceedings of the Fifth Workshop on Innovative Use of NLP for Building Educational Applications, pages 28–36. [find](https://scholar.google.com/scholar?q=A%2E+Rozovskaya+and+D%2E+Roth%2E+2010%2E+Annotating+ESL+errors%3A+Challenges+and+rewards%2E+In+Proceedings+of+the+Fifth+Workshop+on+Innovative+Use+of+NLP+for+Building+Educational+Applications%2C+pages+28%E2%80%9336%2E)
13. J. Tetreault and M. Chodorow. 2008. Native judgments of non-native usage: Experiments in preposition error detection. In Proceedings of the Workshop on Human Judgements in Computational Linguistics, pages 24– 32. [find](https://scholar.google.com/scholar?q=J%2E+Tetreault+and+M%2E+Chodorow%2E+2008%2E+Native+judgments+of+non%2Dnative+usage%3A+Experiments+in+preposition+error+detection%2E+In+Proceedings+of+the+Workshop+on+Human+Judgements+in+Computational+Linguistics%2C+p)
14. J. Tetreault, E. Filatova, and M. Chodorow. 2010. Rethinking grammatical error annotation and evaluation with the Amazon Mechanical Turk. In Proceedings of the Fifth Workshop on Innovative Use of NLP for Building Educational Applications, pages 45–48. [find](https://scholar.google.com/scholar?q=J%2E+Tetreault%2C+E%2E+Filatova%2C+and+M%2E+Chodorow%2E+2010%2E+Rethinking+grammatical+error+annotation+and+evaluation+with+the+Amazon+Mechanical+Turk%2E+In+Proceedings+of+the+Fifth+Workshop+on+Innovative+Use+of+NLP+)
15. H. Yannakoudakis, T. Briscoe, and B. Medlock. 2011. [A new dataset and method for automatically grading ESOL texts](https://md.research.elamur.ai/free/en/papers/yannakoudakis-2011-new-dataset-method-automatically-grading/). In Proceedings of ACL:HLT, pages 180– 189.

*Updated 1 November 2026*