---
title: "Unsupervised Discovery of Morphemes — Creutz 2002"
description: "The authors propose an unsupervised method for automatically segmenting words into… 15 references, cited by 120 papers here."
authors: ["Mathias Creutz", "Krista Lagus"]
year: 2002
venue: "Proceedings of the ACL-02 Workshop on Morphological and Phonological Learning"
source: "https://aclanthology.org/W02-0603/"
doi: "10.3115/1118647.1118650"
arxiv: "cs/0205057"
licence: "CC BY-NC-SA 3.0 (ACL Anthology, before 2016)"
licence_url: "https://creativecommons.org/licenses/by-nc-sa/3.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/creutz-2002-unsupervised-discovery-morphemes/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/creutz-2002-unsupervised-discovery-morphemes/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2002](https://md.research.elamur.ai/free/en/papers/2002/) / Unsupervised Discovery of Morphemes

# Unsupervised Discovery of Morphemes

**Authors:** Mathias Creutz and Krista Lagus · **Year:** 2002 · **Venue:** [Proceedings of the ACL-02 Workshop on Morphological and Phonological Learning](https://md.research.elamur.ai/free/en/venues/acl-02-workshop-on-morphological-and-phonological-learning/) ([ACL-02 Workshop on Morphological and Phonological Learning 2002](https://md.research.elamur.ai/free/en/venues/acl-02-workshop-on-morphological-and-phonological-learning/2002/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY-NC-SA 3.0 (ACL Anthology, before 2016)](https://creativecommons.org/licenses/by-nc-sa/3.0/)

**Links:** [Publisher's page](https://aclanthology.org/W02-0603/) · [PDF](https://aclanthology.org/W02-0603.pdf) · [DOI 10.3115/1118647.1118650](https://doi.org/10.3115/1118647.1118650) · [arXiv cs/0205057](https://arxiv.org/abs/cs/0205057) · [OpenAlex](https://openalex.org/W2117621558)

**Citations:** 377 citations (OpenAlex, 2026-10-14) · [120](https://md.research.elamur.ai/free/en/papers/creutz-2002-unsupervised-discovery-morphemes/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary

The authors propose an unsupervised method for automatically segmenting words into morpheme-like units. Their main target is languages with rich agglutinative morphology, Finnish in particular. In such languages a word-based language model (for example, an n-gram model in speech recognition) suffers from data sparsity, because the number of word forms is very large. The model treats a word as an arbitrarily long sequence of "morphs" and does not distinguish between stems and affixes. Two methods are compared. The first is an online algorithm that recursively splits a word into two parts and optimizes a cost function based on the MDL principle (the cost of the corpus plus the cost of the morph lexicon). The second is batch sequential segmentation with an ML criterion. Quality is evaluated by comparing the segmentation with an existing linguistic morphological analysis, on Finnish and English data. Goldsmith's Linguistica serves as the baseline. The recursive MDL method performed best, especially for Finnish: it yields a more compact morph lexicon and more correct segmentations. For English the margin is small. The work is significant as an approach to morphological segmentation without annotated data, which is useful for low-resource languages.

## Key points

- **Two methods:** Recursive MDL (online, binary recursive splitting, lexicon cost of 5 bits per character) and Sequential ML (batch training with Viterbi segmentation). They are compared with Linguistica.
- **Morph lexicon size:** for Finnish, Rec. MDL yields 6,302 morphs, Seq. ML 10,977, and Linguistica 22,075. For English, the figures are 3,836, 4,888 and 8,153, respectively.
- **Manual evaluation of 2,500 Finnish tokens:** fully correct segmentations amount to 49.6% for Rec. MDL, 47.3% for Seq. ML and 43.1% for Linguistica. Seq. ML has the most incorrect boundaries, at 37.4%.
- **Typical errors:** the MDL method does not split very frequent words and oversegments rare ones. Seq. ML tends toward oversegmentation (affecti + on + at + e). Linguistica often leaves Finnish compound words unsplit.
- **Limitations and plans:** the model does not take context (allomorphy) or morpheme order (morphotactics) into account. The authors plan to transfer the MDL-like cost to the sequential method. The English data were taken from the Brown corpus.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works

🧭 Topics (1): [🧩 Morphology](https://md.research.elamur.ai/free/en/topics/morphology/) · 405 works

💡 Concepts (2): [💡 Natural language processing](https://md.research.elamur.ai/free/en/concepts/natural-language-processing/) · 131 works · [💡 Morphology](https://md.research.elamur.ai/free/en/concepts/morphology/) · 50 works

## Cite

Cite as: Mathias Creutz and Krista Lagus. 2002. Unsupervised Discovery of Morphemes. Proceedings of the ACL-02 Workshop on Morphological and Phonological Learning.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{creutz2002unsupervised,
  title = {{Unsupervised Discovery of Morphemes}},
  author = {Mathias Creutz and Krista Lagus},
  booktitle = {Proceedings of the ACL-02 Workshop on Morphological and Phonological Learning},
  year = {2002},
  doi = {10.3115/1118647.1118650},
  eprint = {cs/0205057},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.3115/1118647.1118650}
}
```

- [← previous An Unsupervised Approach to Recognizing Discourse Relations](https://md.research.elamur.ai/free/en/papers/marcu-2002-unsupervised-approach-recognizing-discourse-relations/)
- [next → A Bootstrapping Method for Learning Semantic Lexicons using Extraction Pattern Contexts](https://md.research.elamur.ai/free/en/papers/thelen-2002-bootstrapping-method-learning-semantic-lexicons/)
- [↑ back to papers of 2002](https://md.research.elamur.ai/free/en/papers/2002/)

## References

15 references, 0 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Jerome Bellegarda. 2000. Exploiting latent semantic information in statistical language modeling. Proceedings of the IEEE, 88(8):1279–1296. [find](https://scholar.google.com/scholar?q=Jerome+Bellegarda%2E+2000%2E+Exploiting+latent+semantic+information+in+statistical+language+modeling%2E+Proceedings+of+the+IEEE%2C+88%288%29%3A1279%E2%80%931296%2E)
2. Michael R. Brent. 1999. An efficient, probabilistically sound algorithm for segmentation and word discovery. Machine Learning, 34:71–105. [find](https://scholar.google.com/scholar?q=Michael+R%2E+Brent%2E+1999%2E+An+efficient%2C+probabilistically+sound+algorithm+for+segmentation+and+word+discovery%2E+Machine+Learning%2C+34%3A71%E2%80%93105%2E)
3. Carl de Marcken. 1995. The unsupervised acquisition of a lexicon from continuous speech. Technical Report A.I. Memo 1558, MIT Artificial Intelligence Lab., Cambridge, Massachusetts. [find](https://scholar.google.com/scholar?q=Carl+de+Marcken%2E+1995%2E+The+unsupervised+acquisition+of+a+lexicon+from+continuous+speech%2E+Technical+Report+A%2EI%2E+Memo+1558%2C+MIT+Artificial+Intelligence+Lab%2E%2C+Cambridge%2C+Massachusetts%2E)
4. Carl de Marcken. 1996. Linguistic structure as composition and perturbation. In Meeting of the Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Carl+de+Marcken%2E+1996%2E+Linguistic+structure+as+composition+and+perturbation%2E+In+Meeting+of+the+Association+for+Computational+Linguistics%2E)
5. Hervé Déjean. 1998. Morphemes as necessary concept for structures discovery from untagged corpora. In Workshop on Paradigms and Grounding in Natural Language Learning, pages 295–299, Adelaide, Jan. 22. [find](https://scholar.google.com/scholar?q=Herve%CC%81+De%CC%81jean%2E+1998%2E+Morphemes+as+necessary+concept+for+structures+discovery+from+untagged+corpora%2E+In+Workshop+on+Paradigms+and+Grounding+in+Natural+Language+Learning%2C+pages+295%E2%80%93299%2C+Adelaide%2C+Jan%2E+)
6. Sabine Deligne and Frédéric Bimbot. 1997. Inference of variable-length linguistic and acoustic units by multigrams. Speech Communication, 23:223–241. [find](https://scholar.google.com/scholar?q=Sabine+Deligne+and+Fre%CC%81de%CC%81ric+Bimbot%2E+1997%2E+Inference+of+variable%2Dlength+linguistic+and+acoustic+units+by+multigrams%2E+Speech+Communication%2C+23%3A223%E2%80%93241%2E)
7. John Goldsmith. 2001. Unsupervised learning of the morphology of a natural language. Computational Linguistics, 27(2):153–198. [find](https://scholar.google.com/scholar?q=John+Goldsmith%2E+2001%2E+Unsupervised+learning+of+the+morphology+of+a+natural+language%2E+Computational+Linguistics%2C+27%282%29%3A153%E2%80%93198%2E)
8. Linguistica affect affect + ing affect + ing + ly affect + ion affect + ion + ate affect + ion + s affect + s eläinlääkäri eläinlääkäri + lle eläinmuseo + n eläinmuseo + on eläinpuisto + n eläinpuisto + sta eläintarh + an Yu Hua. 2000. Unsupervised word induction using MDL criterion. In… [find](https://scholar.google.com/scholar?q=Linguistica+affect+affect+%2B+ing+affect+%2B+ing+%2B+ly+affect+%2B+ion+affect+%2B+ion+%2B+ate+affect+%2B+ion+%2B+s+affect+%2B+s+ela%CC%88inla%CC%88a%CC%88ka%CC%88ri+ela%CC%88inla%CC%88a%CC%88ka%CC%88ri+%2B+lle+ela%CC%88inmuseo+%2B+n+ela%CC%88inmuseo+%2B+on+ela%CC%88inpuisto+%2B+n+)
9. Fred Karlsson. 1987. Finnish Grammar. WSOY, Juva, second edition. [find](https://scholar.google.com/scholar?q=Fred+Karlsson%2E+1987%2E+Finnish+Grammar%2E+WSOY%2C+Juva%2C+second+edition%2E)
10. Fred Karlsson. 1998. Yleinen Yliopistopaino/Helsinki University Press. kielitiede. [find](https://scholar.google.com/scholar?q=Fred+Karlsson%2E+1998%2E+Yleinen+Yliopistopaino%2FHelsinki+University+Press%2E+kielitiede%2E)
11. Chunyu Kit and Yorick Wilks. 1999. Unsupervised learning of word boundary with description length gain. In Proceedings of CoNLL99 ACL Workshop, Bergen. [find](https://scholar.google.com/scholar?q=Chunyu+Kit+and+Yorick+Wilks%2E+1999%2E+Unsupervised+learning+of+word+boundary+with+description+length+gain%2E+In+Proceedings+of+CoNLL99+ACL+Workshop%2C+Bergen%2E)
12. Kimmo Koskenniemi. 1983. Two-level morphology: A general computational model for word-form recognition and production. Ph.D. thesis, University of Helsinki. [find](https://scholar.google.com/scholar?q=Kimmo+Koskenniemi%2E+1983%2E+Two%2Dlevel+morphology%3A+A+general+computational+model+for+word%2Dform+recognition+and+production%2E+Ph%2ED%2E+thesis%2C+University+of+Helsinki%2E)
13. A. Norman Redlich. 1993. Redundancy reduction as a strategy for unsupervised learning. Neural Computation, 5:289–304. [find](https://scholar.google.com/scholar?q=A%2E+Norman+Redlich%2E+1993%2E+Redundancy+reduction+as+a+strategy+for+unsupervised+learning%2E+Neural+Computation%2C+5%3A289%E2%80%93304%2E)
14. Jorma Rissanen. 1989. Stochastic complexity in statistical inquiry. World Scientific Series in Computer Science, 15:79–93. [find](https://scholar.google.com/scholar?q=Jorma+Rissanen%2E+1989%2E+Stochastic+complexity+in+statistical+inquiry%2E+World+Scientific+Series+in+Computer+Science%2C+15%3A79%E2%80%9393%2E)
15. Patrick Schone and Daniel Jurafsky. 2000. Knowledge-free induction of morphology using latent semantic analysis. In Proceedings of CoNLL-2000 and LLL-2000, pages 67–72, Lisbon. [find](https://scholar.google.com/scholar?q=Patrick+Schone+and+Daniel+Jurafsky%2E+2000%2E+Knowledge%2Dfree+induction+of+morphology+using+latent+semantic+analysis%2E+In+Proceedings+of+CoNLL%2D2000+and+LLL%2D2000%2C+pages+67%E2%80%9372%2C+Lisbon%2E)

*Updated 29 October 2026*