---
title: "Pronunciation Modeling for Improved Spelling Correction (2002)"
description: "The paper targets spelling correction for errors that come from not knowing how a word is spelled… 10 references, cited by 65 papers here."
authors: ["Kristina Toutanova", "Robert Moore"]
year: 2002
venue: "Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics"
source: "https://aclanthology.org/P02-1019/"
doi: "10.3115/1073083.1073109"
licence: "CC BY-NC-SA 3.0 (ACL Anthology, before 2016)"
licence_url: "https://creativecommons.org/licenses/by-nc-sa/3.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/toutanova-2002-pronunciation-modeling-improved-spelling-correction/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/toutanova-2002-pronunciation-modeling-improved-spelling-correction/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2002](https://md.research.elamur.ai/free/en/papers/2002/) / Pronunciation Modeling for Improved Spelling Correction

# Pronunciation Modeling for Improved Spelling Correction

**Authors:** [Kristina Toutanova](https://md.research.elamur.ai/free/en/authors/kristina-toutanova/) and [Robert Moore](https://md.research.elamur.ai/free/en/authors/robert-c-moore/) · **Year:** 2002 · **Venue:** [Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics](https://md.research.elamur.ai/free/en/venues/acl/) ([ACL 2002](https://md.research.elamur.ai/free/en/venues/acl/2002/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY-NC-SA 3.0 (ACL Anthology, before 2016)](https://creativecommons.org/licenses/by-nc-sa/3.0/)

**Links:** [Publisher's page](https://aclanthology.org/P02-1019/) · [PDF](https://aclanthology.org/P02-1019.pdf) · [DOI 10.3115/1073083.1073109](https://doi.org/10.3115/1073083.1073109) · [OpenAlex](https://openalex.org/W2144226312)

**Citations:** 187 citations (OpenAlex, 2026-10-14) · [65](https://md.research.elamur.ai/free/en/papers/toutanova-2002-pronunciation-modeling-improved-spelling-correction/cited-by/) in our library, of the 141,840 papers whose bibliographies we read

## Summary
The paper targets spelling correction for errors that come from not knowing how a word is spelled, where the writer produces a form that sounds like the intended word (e.g., *latecks* for *latex*). The strong letter-based noisy channel model of Brill and Moore (2000) still fails on many such cases. Toutanova and Moore keep the Brill and Moore learning algorithm but apply it twice on the same misspelling/correct-word pairs. The first application gives a letter-level error model (LTR). The second gives a phone-sequence error model (PH), trained on pronunciations of correct words and automatically predicted pronunciations of misspellings. Those predictions come from an improved N-gram letter-to-phone converter learned from the Microsoft Speech dictionary. A log-linear combination (CMB) rescores the merged candidate lists of both models. On roughly 10,000 English misspelling pairs, CMB reduces 1-best errors by about 24% relative to LTR, and the reduction approaches 47% at 4-best. This shows that explicitly modeling sound-alike errors complements letter-edit models.

## Key points
- Accuracy on the test set:
  - LTR alone: 1-best 94.21%, 4-best 99.06%.
  - PH alone: 86.36%, 4-best 96.63%.
  - CMB: 95.58%, 4-best 99.50%.
  - Relative error reduction of CMB over LTR: 23.8% (1-best), 39.6% (2-best), 40% (3-best), 46.8% (4-best).
- Data: about 10,000 misspelling pairs, split into 7,385 training and 1,812 test pairs. Pairs whose correct word was absent from the pronunciation dictionary were removed.
- The letter-to-phone model extends Fisher's (1999) N-gram rule model with five extensions, which together cut its error rate by roughly 20%. The extensions include:
  - interpolating all applicable context rules;
  - marking word-initial and word-final positions explicitly;
  - rescoring N-best pronunciations with a phone trigram LM;
  - rescoring with a vowel 4-gram LM, which had not been proposed before.
- Pronunciation resources:
  - NETtalk (14,876 train / 4,964 test words), which is phone-aligned and was used to check the automatic aligner.
  - Microsoft Speech dictionary (106,650 / 30,003), used to train the final model.
- Letter–phone alignment uses hard EM (Viterbi training). Each letter may map to zero, one, or two phones.

## Interesting ideas
- **Same learner, different representation.** Training an identical error-model algorithm once on characters and once on phones yields two complementary views at almost no extra design cost. The same trick could pair surface forms with transliterations or with lemma and morpheme sequences.
- **Union of candidate lists before rescoring.** Only words in either model's top-N list are scored with the combined model. This avoids a full dictionary search while still letting each model recover candidates the other missed. The pattern suits any ensemble of retrieval-style components.
- **Using several predicted pronunciations instead of one.** Considering multiple letter-to-phone hypotheses for a misspelling worked better than using the single best one, because the converter's N-best accuracy is much higher. Propagating N-best outputs of a noisy upstream component is generally safer than committing early.
- **Vowel-only sequence LM for rescoring.** A 4-gram model over just the vowel phones catches errors where predicted pronunciations are most often wrong. Focusing an auxiliary LM on the most error-prone subset of symbols is a cheap, transferable idea.
- **N-best accuracy as an evaluation lens.** Reporting accuracy at 1–4 best shows that gains grow when a user picks from a short list. This is more informative than top-1 alone for interactive correction tools.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Stanford University](https://md.research.elamur.ai/free/en/universities/stanford-university/) — University, Stanford, United States · 310 works · 284,480 citations
- [Microsoft](https://md.research.elamur.ai/free/en/companies/microsoft/) — Company, Redmond, United States · 240 works · 123,537 citations

📚 Shelves (5): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 9,085 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 8,888 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 9,296 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 9,206 works · [👁️ Computer Vision and Pattern Recognition](https://md.research.elamur.ai/free/en/shelves/computer-science/computer-vision-and-pattern-recognition/) · 1,282 works

🧭 Topics (1): [🖍️ Grammatical error correction & learner language](https://md.research.elamur.ai/free/en/topics/grammatical-error-correction/) · 80 works

## Cite

Cite as: Kristina Toutanova and Robert Moore. 2002. Pronunciation Modeling for Improved Spelling Correction. Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{toutanova2002pronunciation,
  title = {{Pronunciation Modeling for Improved Spelling Correction}},
  author = {Kristina Toutanova and Robert Moore},
  booktitle = {Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics},
  year = {2002},
  doi = {10.3115/1073083.1073109},
  url = {https://doi.org/10.3115/1073083.1073109}
}
```

- [← previous Generative Models for Statistical Parsing with Combinatory Categorial Grammar](https://md.research.elamur.ai/free/en/papers/hockenmaier-2002-generative-models-statistical-parsing-combinatory/)
- [next → Phrasal Cohesion and Statistical Machine Translation](https://md.research.elamur.ai/free/en/papers/fox-2002-phrasal-cohesion-statistical-machine-translation/)
- [↑ back to papers of 2002](https://md.research.elamur.ai/free/en/papers/2002/)

## References

10 references, 2 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. E. Brill and R. C. Moore. 2000. [An improved error model for noisy channel spelling correction](https://md.research.elamur.ai/free/en/papers/brill-2000-improved-error-model-noisy-channel/). In Proc. of the 38th Annual Meeting of the ACL, pages 286– 293.
2. K. Church and W. Gale. 1991. Probability scoring for spelling correction. In Statistics and Computing, volume 1, pages 93–103. [find](https://scholar.google.com/scholar?q=K%2E+Church+and+W%2E+Gale%2E+1991%2E+Probability+scoring+for+spelling+correction%2E+In+Statistics+and+Computing%2C+volume+1%2C+pages+93%E2%80%93103%2E)
3. W. Daelemans and A. van den Bosch. 1996. Language-independent data-oriented grapheme-to-phoneme conversion. In Progress in Speech Synthesis, pages 77–90. [find](https://scholar.google.com/scholar?q=W%2E+Daelemans+and+A%2E+van+den+Bosch%2E+1996%2E+Language%2Dindependent+data%2Doriented+grapheme%2Dto%2Dphoneme+conversion%2E+In+Progress+in+Speech+Synthesis%2C+pages+77%E2%80%9390%2E)
4. F. J. Damerau. 1964. A technique for computer detection and correction of spelling errors. In Communications of the ACM, volume 7(3), pages 171–176. [find](https://scholar.google.com/scholar?q=F%2E+J%2E+Damerau%2E+1964%2E+A+technique+for+computer+detection+and+correction+of+spelling+errors%2E+In+Communications+of+the+ACM%2C+volume+7%283%29%2C+pages+171%E2%80%93176%2E)
5. W. M. Fisher. 1999. A statistical text-to-phone function using ngrams and rules. In Proc. of the IEEE International Conference on Acoustics, Speech and Signal Processing, pages 649–652. [find](https://scholar.google.com/scholar?q=W%2E+M%2E+Fisher%2E+1999%2E+A+statistical+text%2Dto%2Dphone+function+using+ngrams+and+rules%2E+In+Proc%2E+of+the+IEEE+International+Conference+on+Acoustics%2C+Speech+and+Signal+Processing%2C+pages+649%E2%80%93652%2E)
6. L. Jiang, H.W. Hon, and X. Huang. 1997. Improvements on a trainable letter-to-sound converter. In Proceedings of the 5th European Conference on Speech Communication and Technology. [find](https://scholar.google.com/scholar?q=L%2E+Jiang%2C+H%2EW%2E+Hon%2C+and+X%2E+Huang%2E+1997%2E+Improvements+on+a+trainable+letter%2Dto%2Dsound+converter%2E+In+Proceedings+of+the+5th+European+Conference+on+Speech+Communication+and+Technology%2E)
7. K. Kuckich. 1992. Techniques for automatically correcting words in text. In ACM Computing Surveys, volume 24(4), pages 377–439. [find](https://scholar.google.com/scholar?q=K%2E+Kuckich%2E+1992%2E+Techniques+for+automatically+correcting+words+in+text%2E+In+ACM+Computing+Surveys%2C+volume+24%284%29%2C+pages+377%E2%80%93439%2E)
8. W. Church M. D. Kernigan and W. A. Gale. 1990. [A spelling correction program based on a noisy channel model](https://md.research.elamur.ai/free/en/papers/kernighan-1990-spelling-correction-program-based-noisy/). In Proc. of COLING-90, volume II, pages 205– 211.
9. F. Mayes and et al. F. Damerau. 1991. Conext based spelling correction. In Information Processing and Management, volume 27(5), pages 517–522. [find](https://scholar.google.com/scholar?q=F%2E+Mayes+and+et+al%2E+F%2E+Damerau%2E+1991%2E+Conext+based+spelling+correction%2E+In+Information+Processing+and+Management%2C+volume+27%285%29%2C+pages+517%E2%80%93522%2E)
10. T. J. Sejnowski and C. R. Rosenberg. 1987. Parallel networks that learn to pronounce english text. In Complex Systems, pages 145–168. [find](https://scholar.google.com/scholar?q=T%2E+J%2E+Sejnowski+and+C%2E+R%2E+Rosenberg%2E+1987%2E+Parallel+networks+that+learn+to+pronounce+english+text%2E+In+Complex+Systems%2C+pages+145%E2%80%93168%2E)

*Updated 24 October 2026*