---
title: "Modeling Word Forms Using Latent Underlying Morphs and"
description: "The authors also propose an evaluation setup based on predicting held-out surface forms. 62 references, cited by 23 papers here."
authors: ["Ryan Cotterell", "Nanyun Peng", "Jason Eisner"]
year: 2015
venue: "Transactions of the Association for Computational Linguistics"
source: "https://aclanthology.org/Q15-1031/"
doi: "10.1162/tacl_a_00149"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/cotterell-2015-modeling-word-forms-latent-underlying/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/cotterell-2015-modeling-word-forms-latent-underlying/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2015](https://md.research.elamur.ai/free/en/papers/2015/page/2/) / Modeling Word Forms Using Latent Underlying Morphs and Phonology

# Modeling Word Forms Using Latent Underlying Morphs and Phonology

**Authors:** [Ryan Cotterell](https://md.research.elamur.ai/free/en/authors/ryan-cotterell/), [Nanyun Peng](https://md.research.elamur.ai/free/en/authors/nanyun-peng/) and [Jason Eisner](https://md.research.elamur.ai/free/en/authors/jason-eisner/) · **Year:** 2015 · **Venue:** [Transactions of the Association for Computational Linguistics](https://md.research.elamur.ai/free/en/venues/tacl/) ([Transactions of the Association for Computational Linguistics (TACL) 2015](https://md.research.elamur.ai/free/en/venues/tacl/2015/)) · **Publisher:** [The MIT Press](https://md.research.elamur.ai/free/en/publishers/the-mit-press/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/Q15-1031/) · [PDF](https://aclanthology.org/Q15-1031.pdf) · [DOI 10.1162/tacl\_a\_00149](https://doi.org/10.1162/tacl_a_00149) · [OpenAlex](https://openalex.org/W1908676432)

**Citations:** 75 citations (OpenAlex, 2026-10-14) · [23](https://md.research.elamur.ai/free/en/papers/cotterell-2015-modeling-word-forms-latent-underlying/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
The paper addresses a classic phonology puzzle: how to automatically infer the single abstract "underlying" form of each morpheme, and the sound rules that turn concatenated morphemes into the pronunciations actually observed. Linguists usually solve this by hand. The authors treat it as joint inference in a directed graphical model (a Bayesian network) whose variables are unknown strings. A lexicon model, a probabilistic finite-state automaton, generates morph URs. These are joined with boundary symbols into word URs. A stochastic phonology, encoded as a weighted finite-state transducer with trainable weights, then maps each word UR to its surface form. Inference uses loopy belief propagation over weighted finite-state machines, so candidate URs are not restricted to a small preselected set. Parameters are tuned with MAP-EM. The input is surface word types paired with their abstract morpheme sequences. The authors also propose an evaluation setup based on predicting held-out surface forms. Across subsets of 7 languages, the learner predicted unseen forms with low error. This matters for generating and analyzing novel words in morphologically rich languages and for comparing phonological theories.

## Key points
- The model has four layers: morpheme URs, concatenated word URs (joined by #), word SRs produced by the phonology, and observations. In the paper's experiments, the SR and observation layers are merged.
- The phonology S_θ is a contextual edit process implemented as a probabilistic finite-state transducer. It uses roughly 50,000 features, and exceptions are handled as low-probability edits.
- Inference relies on loopy BP: finite-state "messages" score every possible string. Learning uses EM, which may get stuck in local optima.
- The task requires observed morpheme annotation. Unlike Dreyer and Eisner's paradigm models, this directed model is productive and can generate unboundedly many new words, including derivational ones.
- Evaluation covers 7 languages on both small and larger datasets, using a newly defined held-out SR prediction paradigm.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Johns Hopkins University](https://md.research.elamur.ai/free/en/universities/johns-hopkins-university/) — University, Baltimore, United States · 240 works · 96,487 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works

🧭 Topics (1): [🧩 Morphology](https://md.research.elamur.ai/free/en/topics/morphology/) · 405 works

💡 Concepts (1): [💡 Word form](https://md.research.elamur.ai/free/en/concepts/word-form/) · 16 works

## Cite

Cite as: Ryan Cotterell, Nanyun Peng and Jason Eisner. 2015. Modeling Word Forms Using Latent Underlying Morphs and Phonology. Transactions of the Association for Computational Linguistics.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@article{cotterell2015modeling,
  title = {{Modeling Word Forms Using Latent Underlying Morphs and Phonology}},
  author = {Ryan Cotterell, Nanyun Peng and Jason Eisner},
  journal = {Transactions of the Association for Computational Linguistics},
  year = {2015},
  doi = {10.1162/tacl\_a\_00149},
  url = {https://doi.org/10.1162/tacl_a_00149}
}
```

- [← previous Hierarchical Low-Rank Tensors for Multilingual Transfer Parsing](https://md.research.elamur.ai/free/en/papers/zhang-2015-hierarchical-low-rank-tensors-multilingual/)
- [next → Discourse parsing for multi-party chat dialogues](https://md.research.elamur.ai/free/en/papers/afantenos-2015-discourse-parsing-multi-party-chat/)
- [↑ back to papers of 2015](https://md.research.elamur.ai/free/en/papers/2015/page/2/)

## References

62 references, 3 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Diana Apoussidou. 2006. On-line learning of underlying forms. Technical Report ROA-835, Rutgers Optimality Archive. [find](https://scholar.google.com/scholar?q=Diana+Apoussidou%2E+2006%2E+On%2Dline+learning+of+underlying+forms%2E+Technical+Report+ROA%2D835%2C+Rutgers+Optimality+Archive%2E)
2. R. Harald Baayen, Richard Piepenbrock, and Leon Gulikers. 1995. The CELEX lexical database on CD-ROM. [find](https://scholar.google.com/scholar?q=R%2E+Harald+Baayen%2C+Richard+Piepenbrock%2C+and+Leon+Gulikers%2E+1995%2E+The+CELEX+lexical+database+on+CD%2DROM%2E)
3. Juliette Blevins. 1994. A phonological and morphological reanalysis of the Maori passive. Te Reo, 37:29–53. [find](https://scholar.google.com/scholar?q=Juliette+Blevins%2E+1994%2E+A+phonological+and+morphological+reanalysis+of+the+Maori+passive%2E+Te+Reo%2C+37%3A29%E2%80%9353%2E)
4. George N. Clements and Elizabeth V. Hume. 1995. The internal organization of speech sounds. In John Goldsmith, editor, Handbook of Phonological Theory. Oxford University Press, Oxford. [find](https://scholar.google.com/scholar?q=George+N%2E+Clements+and+Elizabeth+V%2E+Hume%2E+1995%2E+The+internal+organization+of+speech+sounds%2E+In+John+Goldsmith%2C+editor%2C+Handbook+of+Phonological+Theory%2E+Oxford+University+Press%2C+Oxford%2E)
5. Ryan Cotterell and Jason Eisner. 2015. Penalized expectation propagation for graphical models over strings. In Proceedings of NAACL-HLT, pages 932– 942, Denver, June. Supplementary material (11 pages) also available. [find](https://scholar.google.com/scholar?q=Ryan+Cotterell+and+Jason+Eisner%2E+2015%2E+Penalized+expectation+propagation+for+graphical+models+over+strings%2E+In+Proceedings+of+NAACL%2DHLT%2C+pages+932%E2%80%93+942%2C+Denver%2C+June%2E+Supplementary+material+%2811+pages%29)
6. Ryan Cotterell, Nanyun Peng, and Jason Eisner. 2014. Stochastic contextual edit distance and probabilistic FSTs. In Proceedings of ACL. [find](https://scholar.google.com/scholar?q=Ryan+Cotterell%2C+Nanyun+Peng%2C+and+Jason+Eisner%2E+2014%2E+Stochastic+contextual+edit+distance+and+probabilistic+FSTs%2E+In+Proceedings+of+ACL%2E)
7. Mathias Creutz and Krista Lagus. 2005. Inducing the morphological lexicon of a natural language from unannotated text. In Proceedings of the International and Interdisciplinary Conference on Adaptive Knowledge Representation and Reasoning (AKRR05), volume 1. [find](https://scholar.google.com/scholar?q=Mathias+Creutz+and+Krista+Lagus%2E+2005%2E+Inducing+the+morphological+lexicon+of+a+natural+language+from+unannotated+text%2E+In+Proceedings+of+the+International+and+Interdisciplinary+Conference+on+Adaptive+)
8. Rina Dechter, Bozhena Bidyuk, Robert Mateescu, and Emma Rollon. 2010. On the power of belief propagation: A constraint propagation perspective. In Rina Dechter, Hector Geffner, and Joseph Y. Halpern, editors, Heuristics, Probability and Causality: A Tribute to Judea Pearl. College Publications. [find](https://scholar.google.com/scholar?q=Rina+Dechter%2C+Bozhena+Bidyuk%2C+Robert+Mateescu%2C+and+Emma+Rollon%2E+2010%2E+On+the+power+of+belief+propagation%3A+A+constraint+propagation+perspective%2E+In+Rina+Dechter%2C+Hector+Geffner%2C+and+Joseph+Y%2E+Halpern%2C+)
9. Rina Dechter. 2003. Constraint Processing. Morgan Kaufmann. [find](https://scholar.google.com/scholar?q=Rina+Dechter%2E+2003%2E+Constraint+Processing%2E+Morgan+Kaufmann%2E)
10. Paul Boersma and Bruce Hayes. 2001. Empirical tests of the Gradual Learning Algorithm. Linguistic Inquiry, 32(1):45–86. [find](https://scholar.google.com/scholar?q=Paul+Boersma+and+Bruce+Hayes%2E+2001%2E+Empirical+tests+of+the+Gradual+Learning+Algorithm%2E+Linguistic+Inquiry%2C+32%281%29%3A45%E2%80%9386%2E)
11. Markus Dreyer and Jason Eisner. 2009. Graphical models over multiple strings. In Proceedings of EMNLP, pages 101–110. [find](https://scholar.google.com/scholar?q=Markus+Dreyer+and+Jason+Eisner%2E+2009%2E+Graphical+models+over+multiple+strings%2E+In+Proceedings+of+EMNLP%2C+pages+101%E2%80%93110%2E)
12. Paul Boersma. 1997. How we learn variation, optionality, and probability. In Proceedings of the Institute of Phonetic Sciences of the University of Amsterdam, volume 21, pages 43–58. [find](https://scholar.google.com/scholar?q=Paul+Boersma%2E+1997%2E+How+we+learn+variation%2C+optionality%2C+and+probability%2E+In+Proceedings+of+the+Institute+of+Phonetic+Sciences+of+the+University+of+Amsterdam%2C+volume+21%2C+pages+43%E2%80%9358%2E)
13. Markus Dreyer and Jason Eisner. 2011. Discovering morphological paradigms from plain text using a Dirichlet process mixture model. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 616–627, Edinburgh, July. Supplementary material (9 pages) also avail… [find](https://scholar.google.com/scholar?q=Markus+Dreyer+and+Jason+Eisner%2E+2011%2E+Discovering+morphological+paradigms+from+plain+text+using+a+Dirichlet+process+mixture+model%2E+In+Proceedings+of+the+Conference+on+Empirical+Methods+in+Natural+Lang)
14. Paul Boersma. 1998. How we learn variation, optionality, and probability. In Functional Phonology: Formalizing the Interactions Between Articulatory and Perceptual Drives, chapter 15. Ph.D. Dissertation, University of Amsterdam. Previously appeared in IFA Proceedings (1997), pp. 43–58. [find](https://scholar.google.com/scholar?q=Paul+Boersma%2E+1998%2E+How+we+learn+variation%2C+optionality%2C+and+probability%2E+In+Functional+Phonology%3A+Formalizing+the+Interactions+Between+Articulatory+and+Perceptual+Drives%2C+chapter+15%2E+Ph%2ED%2E+Dissertati)
15. Alexandre Bouchard-Côté, Percy Liang, Thomas L. Griffiths, and Dan Klein. 2007. A probabilistic approach to language change. In Proceedings of NIPS. [find](https://scholar.google.com/scholar?q=Alexandre+Bouchard%2DCo%CC%82te%CC%81%2C+Percy+Liang%2C+Thomas+L%2E+Griffiths%2C+and+Dan+Klein%2E+2007%2E+A+probabilistic+approach+to+language+change%2E+In+Proceedings+of+NIPS%2E)
16. Alexandre Bouchard-Côté, David Hall, Thomas L. Griffiths, and Dan Klein. 2013. Automated reconstruction of ancient languages using probabilistic models of sound change. Proceedings of the National Academy of Sciences. [find](https://scholar.google.com/scholar?q=Alexandre+Bouchard%2DCo%CC%82te%CC%81%2C+David+Hall%2C+Thomas+L%2E+Griffiths%2C+and+Dan+Klein%2E+2013%2E+Automated+reconstruction+of+ancient+languages+using+probabilistic+models+of+sound+change%2E+Proceedings+of+the+National+A)
17. Noam Chomsky and Morris Halle. 1968. The Sound Pattern of English. Harper and Row. [find](https://scholar.google.com/scholar?q=Noam+Chomsky+and+Morris+Halle%2E+1968%2E+The+Sound+Pattern+of+English%2E+Harper+and+Row%2E)
18. Christos Christodoulopoulos, Sharon Goldwater, and Mark Steedman. 2010. [Two decades of unsupervised POS induction: How far have we come?](https://md.research.elamur.ai/free/en/papers/christodoulopoulos-2010-two-decades-unsupervised-pos-induction/) In Proceedings of EMNLP, pages 575–584.
19. Markus Dreyer, Jason R. Smith, and Jason Eisner. 2008. Latent-variable modeling of string transductions with finite-state methods. In Proceedings of EMNLP, pages 1080–1089. [find](https://scholar.google.com/scholar?q=Markus+Dreyer%2C+Jason+R%2E+Smith%2C+and+Jason+Eisner%2E+2008%2E+Latent%2Dvariable+modeling+of+string+transductions+with+finite%2Dstate+methods%2E+In+Proceedings+of+EMNLP%2C+pages+1080%E2%80%931089%2E)
20. Markus Dreyer. 2011. A Non-Parametric Model for the Discovery of Inflectional Paradigms from Plain Text Using Graphical Models over Strings. Ph.D. thesis, Johns Hopkins University, Baltimore, MD, April. [find](https://scholar.google.com/scholar?q=Markus+Dreyer%2E+2011%2E+A+Non%2DParametric+Model+for+the+Discovery+of+Inflectional+Paradigms+from+Plain+Text+Using+Graphical+Models+over+Strings%2E+Ph%2ED%2E+thesis%2C+Johns+Hopkins+University%2C+Baltimore%2C+MD%2C+Apri)
21. Sarah Eisenstat. 2009. Learning underlying forms with MaxEnt. Master’s thesis, Brown University, Providence, RI. [find](https://scholar.google.com/scholar?q=Sarah+Eisenstat%2E+2009%2E+Learning+underlying+forms+with+MaxEnt%2E+Master%E2%80%99s+thesis%2C+Brown+University%2C+Providence%2C+RI%2E)
22. Jason Eisner. 2002a. Comprehension and compilation in Optimality Theory. In Proceedings of ACL, pages 56–63, Philadelphia, July. [find](https://scholar.google.com/scholar?q=Jason+Eisner%2E+2002a%2E+Comprehension+and+compilation+in+Optimality+Theory%2E+In+Proceedings+of+ACL%2C+pages+56%E2%80%9363%2C+Philadelphia%2C+July%2E)
23. Jason Eisner. 2002b. Discovering syntactic deep structure via Bayesian statistics. Cognitive Science, 26(3):255–268, May-June. [find](https://scholar.google.com/scholar?q=Jason+Eisner%2E+2002b%2E+Discovering+syntactic+deep+structure+via+Bayesian+statistics%2E+Cognitive+Science%2C+26%283%29%3A255%E2%80%93268%2C+May%2DJune%2E)
24. Jason Eisner. 2015. Should linguists evaluate grammars or grammar learners? In preparation. [find](https://scholar.google.com/scholar?q=Jason+Eisner%2E+2015%2E+Should+linguists+evaluate+grammars+or+grammar+learners%3F+In+preparation%2E)
25. C. Douglas Johnson. 1972. Formal Aspects of Phonological Description. Mouton. [find](https://scholar.google.com/scholar?q=C%2E+Douglas+Johnson%2E+1972%2E+Formal+Aspects+of+Phonological+Description%2E+Mouton%2E)
26. Micha Elsner, Sharon Goldwater, and Jacob Eisenstein. 2012. Bootstrapping a unified model of lexical and phonetic acquisition. In Proceedings of ACL, pages 184–193. [find](https://scholar.google.com/scholar?q=Micha+Elsner%2C+Sharon+Goldwater%2C+and+Jacob+Eisenstein%2E+2012%2E+Bootstrapping+a+unified+model+of+lexical+and+phonetic+acquisition%2E+In+Proceedings+of+ACL%2C+pages+184%E2%80%93193%2E)
27. René Kager. 1999. Optimality Theory, volume 2. MIT Press. [find](https://scholar.google.com/scholar?q=Rene%CC%81+Kager%2E+1999%2E+Optimality+Theory%2C+volume+2%2E+MIT+Press%2E)
28. Micha Elsner, Sharon Goldwater, Naomi Feldman, and Frank Wood. 2013. A joint learning model of word segmentation, lexical acquisition, and phonetic variability. In Proceedings of EMNLP, pages 42–54. [find](https://scholar.google.com/scholar?q=Micha+Elsner%2C+Sharon+Goldwater%2C+Naomi+Feldman%2C+and+Frank+Wood%2E+2013%2E+A+joint+learning+model+of+word+segmentation%2C+lexical+acquisition%2C+and+phonetic+variability%2E+In+Proceedings+of+EMNLP%2C+pages+42%E2%80%9354%2E)
29. Alexander M. Fraser, Marion Weller, Aoife Cahill, and Fabienne Cap. 2012. Modeling inflection and word-formation in SMT. In Proceedings of EACL, pages 664–674. [find](https://scholar.google.com/scholar?q=Alexander+M%2E+Fraser%2C+Marion+Weller%2C+Aoife+Cahill%2C+and+Fabienne+Cap%2E+2012%2E+Modeling+inflection+and+word%2Dformation+in+SMT%2E+In+Proceedings+of+EACL%2C+pages+664%E2%80%93674%2E)
30. J. Goldsmith. 2006. An algorithm for the unsupervised learning of morphology. Natural Language Engineering, 12(4):353–371. [find](https://scholar.google.com/scholar?q=J%2E+Goldsmith%2E+2006%2E+An+algorithm+for+the+unsupervised+learning+of+morphology%2E+Natural+Language+Engineering%2C+12%284%29%3A353%E2%80%93371%2E)
31. Sharon Goldwater and Mark Johnson. 2003. Learning OT constraint rankings using a maximum entropy model. In Jennifer Spenader, Anders Eriksson, and Osten Dahl, editors, Proceedings of the Workshop on Variation within Optimality Theory, pages 113–122, Stockholm University. [find](https://scholar.google.com/scholar?q=Sharon+Goldwater+and+Mark+Johnson%2E+2003%2E+Learning+OT+constraint+rankings+using+a+maximum+entropy+model%2E+In+Jennifer+Spenader%2C+Anders+Eriksson%2C+and+Osten+Dahl%2C+editors%2C+Proceedings+of+the+Workshop+on+V)
32. David Hall and Dan Klein. 2010. Finding cognate groups using phylogenies. In Proceedings of ACL. [find](https://scholar.google.com/scholar?q=David+Hall+and+Dan+Klein%2E+2010%2E+Finding+cognate+groups+using+phylogenies%2E+In+Proceedings+of+ACL%2E)
33. Bruce Hayes and Colin Wilson. 2008. A maximum entropy model of phonotactics and phonotactic learning. Linguistic Inquiry, 39(3):379–440. [find](https://scholar.google.com/scholar?q=Bruce+Hayes+and+Colin+Wilson%2E+2008%2E+A+maximum+entropy+model+of+phonotactics+and+phonotactic+learning%2E+Linguistic+Inquiry%2C+39%283%29%3A379%E2%80%93440%2E)
34. Matt Hohensee and Emily M. Bender. 2012. Getting more from morphology in multilingual dependency parsing. In Proceedings of NAACL-HLT, pages 315– 326. [find](https://scholar.google.com/scholar?q=Matt+Hohensee+and+Emily+M%2E+Bender%2E+2012%2E+Getting+more+from+morphology+in+multilingual+dependency+parsing%2E+In+Proceedings+of+NAACL%2DHLT%2C+pages+315%E2%80%93+326%2E)
35. Mans Hulden. 2009. [Revisiting multi-tape automata for Semitic morphological analysis and generation](https://md.research.elamur.ai/free/en/papers/hulden-2009-revisiting-multi-tape-automata-semitic/). In Proceedings of the EACL 2009 Workshop on Computational Approaches to Semitic Languages, pages 19–26, March.
36. Sharon Inkelas and Cheryl Zoll. 2005. Reduplication: Doubling in Morphology. Number 106 in Cambridge Studies in Linguistics. Cambridge University Press. [find](https://scholar.google.com/scholar?q=Sharon+Inkelas+and+Cheryl+Zoll%2E+2005%2E+Reduplication%3A+Doubling+in+Morphology%2E+Number+106+in+Cambridge+Studies+in+Linguistics%2E+Cambridge+University+Press%2E)
37. Roman Jakobson. 1948. Russian conjugation. Word, 4:155–167. [find](https://scholar.google.com/scholar?q=Roman+Jakobson%2E+1948%2E+Russian+conjugation%2E+Word%2C+4%3A155%E2%80%93167%2E)
38. Gaja Jarosz. 2006. Richness of the base and probabilistic unsupervised learning in Optimality Theory. In Proceedings of the Eighth Meeting of the ACL Special Interest Group on Computational Phonology and Morphology, pages 50–59. [find](https://scholar.google.com/scholar?q=Gaja+Jarosz%2E+2006%2E+Richness+of+the+base+and+probabilistic+unsupervised+learning+in+Optimality+Theory%2E+In+Proceedings+of+the+Eighth+Meeting+of+the+ACL+Special+Interest+Group+on+Computational+Phonology+)
39. Gaja Jarosz. 2013. Learning with hidden structure in optimality theory and harmonic grammar: Beyond robust interpretive parsing. Phonology, 30(01):27– 71. [find](https://scholar.google.com/scholar?q=Gaja+Jarosz%2E+2013%2E+Learning+with+hidden+structure+in+optimality+theory+and+harmonic+grammar%3A+Beyond+robust+interpretive+parsing%2E+Phonology%2C+30%2801%29%3A27%E2%80%93+71%2E)
40. Ronald M. Kaplan and Martin Kay. 1994. Regular models of phonological rule systems. Computational Linguistics, 20(3):331–378. [find](https://scholar.google.com/scholar?q=Ronald+M%2E+Kaplan+and+Martin+Kay%2E+1994%2E+Regular+models+of+phonological+rule+systems%2E+Computational+Linguistics%2C+20%283%29%3A331%E2%80%93378%2E)
41. Michael J. Kenstowicz and Charles W. Kisseberth. 1979. Generative Phonology. Academic Press San Diego. [find](https://scholar.google.com/scholar?q=Michael+J%2E+Kenstowicz+and+Charles+W%2E+Kisseberth%2E+1979%2E+Generative+Phonology%2E+Academic+Press+San+Diego%2E)
42. András Kornai. 1995. Formal Phonology. Garland Publishing, New York. [find](https://scholar.google.com/scholar?q=Andra%CC%81s+Kornai%2E+1995%2E+Formal+Phonology%2E+Garland+Publishing%2C+New+York%2E)
43. Zhifei Li and Jason Eisner. 2009. [First- and second-order expectation semirings with applications to minimum-risk training on translation forests](https://md.research.elamur.ai/free/en/papers/li-2009-first-second-order-expectation-semirings/). In Proceedings of EMNLP, pages 40–51, Singapore, August.
44. Dong C. Liu and Jorge Nocedal. 1989. On the limited memory BFGS method for large scale optimization. Mathematical Programming, 45(1-3):503–528. [find](https://scholar.google.com/scholar?q=Dong+C%2E+Liu+and+Jorge+Nocedal%2E+1989%2E+On+the+limited+memory+BFGS+method+for+large+scale+optimization%2E+Mathematical+Programming%2C+45%281%2D3%29%3A503%E2%80%93528%2E)
45. Andrew McCallum, Dayne Freitag, and Fernando C. N. Pereira. 2000. Maximum entropy Markov models for information extraction and segmentation. In Proceedings of ICML, pages 591–598. [find](https://scholar.google.com/scholar?q=Andrew+McCallum%2C+Dayne+Freitag%2C+and+Fernando+C%2E+N%2E+Pereira%2E+2000%2E+Maximum+entropy+Markov+models+for+information+extraction+and+segmentation%2E+In+Proceedings+of+ICML%2C+pages+591%E2%80%93598%2E)
46. Navarré Merchant. 2008. Discovering Underlying Forms: Contrast Pairs and Ranking. Ph.D. thesis, Rutgers University. Available on the Rutgers Optimality Archive as ROA-964. [find](https://scholar.google.com/scholar?q=Navarre%CC%81+Merchant%2E+2008%2E+Discovering+Underlying+Forms%3A+Contrast+Pairs+and+Ranking%2E+Ph%2ED%2E+thesis%2C+Rutgers+University%2E+Available+on+the+Rutgers+Optimality+Archive+as+ROA%2D964%2E)
47. Kevin P. Murphy, Yair Weiss, and Michael I. Jordan. 1999. Loopy belief propagation for approximate inference: An empirical study. In Proceedings of UAI, pages 467–475. [find](https://scholar.google.com/scholar?q=Kevin+P%2E+Murphy%2C+Yair+Weiss%2C+and+Michael+I%2E+Jordan%2E+1999%2E+Loopy+belief+propagation+for+approximate+inference%3A+An+empirical+study%2E+In+Proceedings+of+UAI%2C+pages+467%E2%80%93475%2E)
48. Joe Pater, Karen Jesney, Robert Staubs, and Brian Smith. 2012. Learning probabilities over underlying representations. In Proceedings of the Twelfth Meeting of the Special Interest Group on Computational Morphology and Phonology, pages 62–71. [find](https://scholar.google.com/scholar?q=Joe+Pater%2C+Karen+Jesney%2C+Robert+Staubs%2C+and+Brian+Smith%2E+2012%2E+Learning+probabilities+over+underlying+representations%2E+In+Proceedings+of+the+Twelfth+Meeting+of+the+Special+Interest+Group+on+Computatio)
49. Judea Pearl. 1988. Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference. Morgan Kaufmann Publishers Inc., San Francisco, CA, USA. [find](https://scholar.google.com/scholar?q=Judea+Pearl%2E+1988%2E+Probabilistic+Reasoning+in+Intelligent+Systems%3A+Networks+of+Plausible+Inference%2E+Morgan+Kaufmann+Publishers+Inc%2E%2C+San+Francisco%2C+CA%2C+USA%2E)
50. Nanyun Peng, Ryan Cotterell, and Jason Eisner. 2015. Dual decomposition inference for graphical models over strings. In Proceedings of EMNLP, Lisbon, September. To appear. [find](https://scholar.google.com/scholar?q=Nanyun+Peng%2C+Ryan+Cotterell%2C+and+Jason+Eisner%2E+2015%2E+Dual+decomposition+inference+for+graphical+models+over+strings%2E+In+Proceedings+of+EMNLP%2C+Lisbon%2C+September%2E+To+appear%2E)
51. Janet Pierrehumbert. 2003. Probabilistic phonology: Discrimination and robustness. In Probabilistic Linguistics, pages 177–228. MIT Press. [find](https://scholar.google.com/scholar?q=Janet+Pierrehumbert%2E+2003%2E+Probabilistic+phonology%3A+Discrimination+and+robustness%2E+In+Probabilistic+Linguistics%2C+pages+177%E2%80%93228%2E+MIT+Press%2E)
52. Alan Prince and Paul Smolensky. 2004. Optimality Theory: Constraint Interaction in Generative Grammar. Wiley-Blackwell. [find](https://scholar.google.com/scholar?q=Alan+Prince+and+Paul+Smolensky%2E+2004%2E+Optimality+Theory%3A+Constraint+Interaction+in+Generative+Grammar%2E+Wiley%2DBlackwell%2E)
53. Jason A. Riggle. 2004. Generation, Recognition, and Learning in Finite State Optimality Theory. Ph.D. thesis, University of California at Los Angeles. [find](https://scholar.google.com/scholar?q=Jason+A%2E+Riggle%2E+2004%2E+Generation%2C+Recognition%2C+and+Learning+in+Finite+State+Optimality+Theory%2E+Ph%2ED%2E+thesis%2C+University+of+California+at+Los+Angeles%2E)
54. Jason Riggle. 2012. Phonological features chart (version 12.12). Available online at [https://dl.dropboxusercontent.com/u/](https://dl.dropboxusercontent.com/u/) 5956329/Riggle/PhonChart_v1212.pdf (retrieved 2015-08-07). [↗](https://dl.dropboxusercontent.com/u/)
55. David Sankoff. 1978. Probability and linguistic variation. Synthese, 37(2):217–238. [find](https://scholar.google.com/scholar?q=David+Sankoff%2E+1978%2E+Probability+and+linguistic+variation%2E+Synthese%2C+37%282%29%3A217%E2%80%93238%2E)
56. Ferdinand de Saussure. 1916. Course in General Linguistics. Columbia University Press. English edition of June 2011, based on the 1959 translation by Wade Baskin. [find](https://scholar.google.com/scholar?q=Ferdinand+de+Saussure%2E+1916%2E+Course+in+General+Linguistics%2E+Columbia+University+Press%2E+English+edition+of+June+2011%2C+based+on+the+1959+translation+by+Wade+Baskin%2E)
57. Paul Smolensky and Géraldine Legendre. 2006. The Harmonic Mind: From Neural Computation to Optimality-Theoretic Grammar (Vol. 1: Cognitive Architecture). MIT Press. [find](https://scholar.google.com/scholar?q=Paul+Smolensky+and+Ge%CC%81raldine+Legendre%2E+2006%2E+The+Harmonic+Mind%3A+From+Neural+Computation+to+Optimality%2DTheoretic+Grammar+%28Vol%2E+1%3A+Cognitive+Architecture%29%2E+MIT+Press%2E)
58. Bruce Tesar and Paul Smolensky. 1998. Learnability in Optimality theory. Linguistic Inquiry, 29(2):229– 268. [find](https://scholar.google.com/scholar?q=Bruce+Tesar+and+Paul+Smolensky%2E+1998%2E+Learnability+in+Optimality+theory%2E+Linguistic+Inquiry%2C+29%282%29%3A229%E2%80%93+268%2E)
59. Bruce Tesar. 2004. Contrast analysis in phonological learning. Technical Report ROA-695, Rutgers Optimality Archive. [find](https://scholar.google.com/scholar?q=Bruce+Tesar%2E+2004%2E+Contrast+analysis+in+phonological+learning%2E+Technical+Report+ROA%2D695%2C+Rutgers+Optimality+Archive%2E)
60. Bruce Tesar. 2014. Output-Driven Phonology: Theory and Learning. Cambridge University Press. [find](https://scholar.google.com/scholar?q=Bruce+Tesar%2E+2014%2E+Output%2DDriven+Phonology%3A+Theory+and+Learning%2E+Cambridge+University+Press%2E)
61. Yi Yang and Jacob Eisenstein. 2013. A log-linear model for unsupervised text normalization. In EMNLP, pages 61–72. [find](https://scholar.google.com/scholar?q=Yi+Yang+and+Jacob+Eisenstein%2E+2013%2E+A+log%2Dlinear+model+for+unsupervised+text+normalization%2E+In+EMNLP%2C+pages+61%E2%80%9372%2E)
62. Reyyan Yeniterzi. 2011. Exploiting morphology in Turkish named entity recognition system. In Proceedings of the ACL Student Session, pages 105– 110. [find](https://scholar.google.com/scholar?q=Reyyan+Yeniterzi%2E+2011%2E+Exploiting+morphology+in+Turkish+named+entity+recognition+system%2E+In+Proceedings+of+the+ACL+Student+Session%2C+pages+105%E2%80%93+110%2E)

*Updated 29 October 2026*