---
title: "Exploring the Influence of Spelling Errors on Lexical Variation"
description: "Lexical richness scores such as Type-Token Ratio (TTR) and Yule's K are commonly used to analyse… 15 references, cited by 3 papers here."
authors: ["Ryo Nagata", "Taisei Sato", "Hiroya Takamura"]
year: 2018
venue: "Proceedings of the 27th International Conference on Computational Linguistics"
source: "https://aclanthology.org/C18-1202/"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/nagata-2018-exploring-influence-spelling-errors-lexical/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/nagata-2018-exploring-influence-spelling-errors-lexical/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2018](https://md.research.elamur.ai/free/en/papers/2018/page/7/) / Exploring the Influence of Spelling Errors on Lexical Variation Measures

# Exploring the Influence of Spelling Errors on Lexical Variation Measures

**Authors:** [Ryo Nagata](https://md.research.elamur.ai/free/en/authors/ryo-nagata/), Taisei Sato and [Hiroya Takamura](https://md.research.elamur.ai/free/en/authors/hiroya-takamura/) · **Year:** 2018 · **Venue:** [Proceedings of the 27th International Conference on Computational Linguistics](https://md.research.elamur.ai/free/en/venues/coling/) ([COLING 2018](https://md.research.elamur.ai/free/en/venues/coling/2018/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/C18-1202/) · [PDF](https://aclanthology.org/C18-1202.pdf) · [OpenAlex](https://openalex.org/W2834585486)

**Citations:** 2 citations (OpenAlex, 2026-10-14) · 3 in our library, of the 149,439 papers whose bibliographies we read

## Summary
Lexical richness scores such as Type-Token Ratio (TTR) and Yule's K are commonly used to analyse and grade learner English. Both depend on how many distinct word types a text contains, and misspellings inflate that count. A word like *because* spelt three wrong ways would count as four types. The authors test whether this makes the measures unreliable. They use essays by Japanese-L1 learners at three levels (middle school, high school, college), compute TTR and K on the raw text, and compare them with versions where the spelling errors were corrected by hand. They try two correction policies built on a 13-category error taxonomy.

TTR shifts by at least 16% in every group. K changes by no more than 1%. A short derivation explains this: K is driven by squared frequencies of common words, which are rarely misspelt. The authors conclude that K is the safer choice for measuring vocabulary in noisy learner text.

## Key points
- **Corpus:** 1,073 essays, 82,659 words and 2,534 spelling errors across the three groups. Mean edit distance of SP-type errors is 1.38. Part of the data is public as the Konan-JIEM (KJ) learner corpus.
- **Taxonomy:** 13 error types, including SP, PC, OC, GC, NM, RE and romanized Japanese (RO/SR).
  - The "all" policy corrects every type.
  - The "selected" policy corrects SP, PC, OC, GC and NM, keeps real-word errors (RE) unchanged, and drops the remaining types from both counts.
- **Preprocessing:** Stanford Parser 3.5.0 splits sentences and tokens. Text is lowercased, and tokens with no Latin letters are removed. Merged-word and split-word errors are excluded.
- **Results:** TTR differs by at least about 16% between original and corrected text. K differs by at most 1%.
- **Misspelt share r:** This is the fraction of a word's occurrences that are misspelt.
  - For words with frequency 10–99, r has mean 0.06 (SD 0.10).
  - For words with frequency ≥100, r has mean 0.01 (SD 0.024).

## Interesting ideas
- **Before/after manual correction as a controlled experiment.** Compute a metric on the original text and again on a hand-corrected copy, so the gap isolates the effect of the noise itself. This shows directly how much a statistic depends on surface errors.
- **Each error class gets one of three treatments: fix, keep, or exclude.** The choice depends on whether the error reflects what the writer actually knows. A real-word mix-up like their/there may show genuine ignorance, so it is kept, while invented forms are removed from the counts.
- **Analytic sensitivity check.** If a fraction r of a word's occurrences is misspelt, that word's squared-frequency contribution scales by (1−r)²+r², which is close to 1−2r when r is small. A quick derivation like this predicts which statistics will be robust to noise before running experiments.
- **Measuring r empirically per frequency band.** Splitting words into frequency bands showed that frequent words are almost never misspelt. This supports any assumption that each word has one dominant spelling.
- **Choose the metric that survives noise.** Rather than only cleaning data, pick a statistic that is inherently stable under the expected noise, as K is here.

## For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

- **I1** (learn): Our evaluations could include noisy-vs-clean paired runs, checking how much parser LAS or frame extraction drops on misspelt inputs.
- **I2** (tokens): The normalizer could tag each suspect token with a treatment class, so later stages know whether to re-lemmatize it, trust it, or skip it.
- **I3** (world): When compiling frequency-based knowledge (measures, verb-class statistics), prefer frequency-weighted aggregates dominated by common items, and document their noise sensitivity formally.
- **I4** (tokens): Banded misspelling rates could set thresholds for the spelling-variant lexicon, mapping rare variants to a dominant form only when the canonical form clearly dominates.
- **I5** (answer): When scoring student or school-test answers for vocabulary or style, use K-like measures so that misspellings do not distort the assessment.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Tokyo Institute of Technology](https://md.research.elamur.ai/free/en/universities/tokyo-institute-of-technology/) — University, Tokyo, Japan · 22 works · 2,137 citations
- [Konan University](https://md.research.elamur.ai/free/en/universities/konan-university/) — University, Kobe, Japan · 3 works · 39 citations

📚 Shelves (6): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works · [💭 Psychology](https://md.research.elamur.ai/free/en/shelves/psychology/) · 1,552 works · [💭 Developmental and Educational Psychology](https://md.research.elamur.ai/free/en/shelves/psychology/developmental-and-educational-psychology/) · 499 works

💡 Concepts (1): [💡 Type–token ratio](https://md.research.elamur.ai/free/en/concepts/type-token-ratio/) · 14 works

## Cite

Cite as: Ryo Nagata, Taisei Sato and Hiroya Takamura. 2018. Exploring the Influence of Spelling Errors on Lexical Variation Measures. Proceedings of the 27th International Conference on Computational Linguistics.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{nagata2018exploring,
  title = {{Exploring the Influence of Spelling Errors on Lexical Variation Measures}},
  author = {Ryo Nagata, Taisei Sato and Hiroya Takamura},
  booktitle = {Proceedings of the 27th International Conference on Computational Linguistics},
  year = {2018},
  url = {https://aclanthology.org/C18-1202/}
}
```

- [← previous Approximate Dynamic Oracle for Dependency Parsing with Reinforcement Learning](https://md.research.elamur.ai/free/en/papers/yu-2018-approximate-dynamic-oracle-dependency-parsing/)
- [next → Predicting CEFRL levels in learner English on the basis of metrics and full texts](https://md.research.elamur.ai/free/en/papers/arnold-2018-predicting-cefrl-levels-learner-english/)
- [↑ back to papers of 2018](https://md.research.elamur.ai/free/en/papers/2018/page/7/)

## References

15 references, 1 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Pierrre J.L. Arnaud. 1984. The lexical richness of L2 written productions and the validity of vocabulary tests. In Proc. of International Symposium on Language Testing, pages 14–28. [find](https://scholar.google.com/scholar?q=Pierrre+J%2EL%2E+Arnaud%2E+1984%2E+The+lexical+richness+of+L2+written+productions+and+the+validity+of+vocabulary+tests%2E+In+Proc%2E+of+International+Symposium+on+Language+Testing%2C+pages+14%E2%80%9328%2E)
2. Yigal Attali and Jill Burstein. 2006. Automated essay scoring with E-rater v.2.0. The Journal of Technology, Learning, and Assessment, 4(3):3–30. [find](https://scholar.google.com/scholar?q=Yigal+Attali+and+Jill+Burstein%2E+2006%2E+Automated+essay+scoring+with+E%2Drater+v%2E2%2E0%2E+The+Journal+of+Technology%2C+Learning%2C+and+Assessment%2C+4%283%29%3A3%E2%80%9330%2E)
3. Danqi Chen and Christopher Manning. 2014. [A fast and accurate dependency parser using neural networks](https://md.research.elamur.ai/free/en/papers/chen-2014-fast-accurate-dependency-parser-neural/). In Proc. of 2014 Conference on Empirical Methods in Natural Language Processing, pages 740–750.
4. Sylviane Granger and Martin Wynne, 1999. Optimising measures of lexical variation in EFL learner corpora, pages 249–257. Corpora Galore. Rodopi. [find](https://scholar.google.com/scholar?q=Sylviane+Granger+and+Martin+Wynne%2C+1999%2E+Optimising+measures+of+lexical+variation+in+EFL+learner+corpora%2C+pages+249%E2%80%93257%2E+Corpora+Galore%2E+Rodopi%2E)
5. Carmen Gregori-Signes and Begoñ Clavel-Arroitia. 2015. Analysing lexical density and lexical diversity in university. In Proc. of 7th International Conference on Corpus Linguistics: Current Work in Corpus Linguistics: Working with Traditionally-conceived Corpora and Beyond, pages 546–556. [find](https://scholar.google.com/scholar?q=Carmen+Gregori%2DSignes+and+Begon%CC%83+Clavel%2DArroitia%2E+2015%2E+Analysing+lexical+density+and+lexical+diversity+in+university%2E+In+Proc%2E+of+7th+International+Conference+on+Corpus+Linguistics%3A+Current+Work+in+C)
6. Pierre Guiraud. 1959. Problèmes Et Méthodes De La Statistique Linguistique. D. Reidel Publishing Company, Dordrecht. [find](https://scholar.google.com/scholar?q=Pierre+Guiraud%2E+1959%2E+Proble%CC%80mes+Et+Me%CC%81thodes+De+La+Statistique+Linguistique%2E+D%2E+Reidel+Publishing+Company%2C+Dordrecht%2E)
7. Gustav Herdan. 1960. Type-Token Mathematics: A Textbook of Mathematical Linguistics. The Hague: Mouton, Amsterdam. [find](https://scholar.google.com/scholar?q=Gustav+Herdan%2E+1960%2E+Type%2DToken+Mathematics%3A+A+Textbook+of+Mathematical+Linguistics%2E+The+Hague%3A+Mouton%2C+Amsterdam%2E)
8. Shin’ichiro Ishikawa. 2015. Lexical development in L2 English learners’ speeches and writings. In Proc. 7th International Conference on Corpus Linguistics: Current Work in Corpus Linguistics: Working with Traditionally-conceived Corpora and Beyond, pages 202–210. [find](https://scholar.google.com/scholar?q=Shin%E2%80%99ichiro+Ishikawa%2E+2015%2E+Lexical+development+in+L2+English+learners%E2%80%99+speeches+and+writings%2E+In+Proc%2E+7th+International+Conference+on+Corpus+Linguistics%3A+Current+Work+in+Corpus+Linguistics%3A+Working+)
9. Daisuke Kimura and Kumiko Tanaka-Ishii. 2011. A study on constants of natural language texts. Journal of Natural Language Processing, 18(2):119–137. [↗](https://doi.org/10.5715/jnlp.18.119)
10. David D. Malvern, Brian J. Richards, Ngoni Chipere, and Pilar Duràn. 2004. Lexical Diversity and Language Development. Palgrave Macmillan, London. [↗](https://doi.org/10.1057/9780230511804)
11. Ildikó Pilán, Elena Volodina, and Torsten Zesch. 2016. Predicting proficiency levels in learner writings by transferring a linguistic complexity model from expert-written coursebooks. In Proc. of 26th International Conference on Computational Linguistics, pages 2101–2111. [↗](https://doi.org/10.17185/duepublico/72093)
12. Anaı̈s Tack, Thomas François, Sophie Roekhaut, and Cédrick Fairon. 2017. Human and automated CEFR-based grading of short answers. In Proc. of 12th Workshop on Innovative Use of NLP for Building Educational Applications, pages 169–179. [find](https://scholar.google.com/scholar?q=Ana%C4%B1%CC%88s+Tack%2C+Thomas+Franc%CC%A7ois%2C+Sophie+Roekhaut%2C+and+Ce%CC%81drick+Fairon%2E+2017%2E+Human+and+automated+CEFR%2Dbased+grading+of+short+answers%2E+In+Proc%2E+of+12th+Workshop+on+Innovative+Use+of+NLP+for+Building+Educ)
13. Kumiko Tanaka-Ishii and Shunsuke Aihara. 2015. Computational constancy measures of texts–Yule’s K and Rèyi’s entropy. Computational Linguistics, 1(3). [find](https://scholar.google.com/scholar?q=Kumiko+Tanaka%2DIshii+and+Shunsuke+Aihara%2E+2015%2E+Computational+constancy+measures+of+texts%E2%80%93Yule%E2%80%99s+K+and+Re%CC%80yi%E2%80%99s+entropy%2E+Computational+Linguistics%2C+1%283%29%2E)
14. Zdislava Šišková. 2012. Lexical richness in EFL students’ narratives. Language Studies Working Papers, 4:26–36. [find](https://scholar.google.com/scholar?q=Zdislava+S%CC%8Cis%CC%8Ckova%CC%81%2E+2012%2E+Lexical+richness+in+EFL+students%E2%80%99+narratives%2E+Language+Studies+Working+Papers%2C+4%3A26%E2%80%9336%2E)
15. G. U. Yule. 1944. The Statistical Study of Literary Vocabulary. Cambridge. [find](https://scholar.google.com/scholar?q=G%2E+U%2E+Yule%2E+1944%2E+The+Statistical+Study+of+Literary+Vocabulary%2E+Cambridge%2E)

*Updated 1 November 2026*