---
title: "Probabilistic Top-Down Parsing and Language — Roark 2001"
description: "Roark examines whether syntactic parsing can produce better language models for speech recognition than the shallow n-gram models that dominate the field."
authors: ["Brian Roark"]
year: 2001
venue: "Computational Linguistics"
source: "https://aclanthology.org/J01-2004/"
doi: "10.1162/089120101750300526"
licence: "CC BY-NC-SA 3.0 (ACL Anthology, before 2016)"
licence_url: "https://creativecommons.org/licenses/by-nc-sa/3.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/roark-2001-probabilistic-top-down-parsing-language/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/roark-2001-probabilistic-top-down-parsing-language/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2001](https://md.research.elamur.ai/free/en/papers/2001/) / Probabilistic Top-Down Parsing and Language Modeling

# Probabilistic Top-Down Parsing and Language Modeling

**Authors:** [Brian Roark](https://md.research.elamur.ai/free/en/authors/brian-roark/) · **Year:** 2001 · **Venue:** Computational Linguistics · **Licence:** [CC BY-NC-SA 3.0 (ACL Anthology, before 2016)](https://creativecommons.org/licenses/by-nc-sa/3.0/)

**Links:** [Publisher's page](https://aclanthology.org/J01-2004/) · [PDF](https://aclanthology.org/J01-2004.pdf) · [DOI 10.1162/089120101750300526](https://doi.org/10.1162/089120101750300526) · [OpenAlex](https://openalex.org/W2123893795)

**Citations:** 329 citations (openalex-snapshot · 2026-10-14) · [95](#cited-by) in our library, of the 140884 papers whose bibliographies we read

## Summary
Roark examines whether syntactic parsing can produce better language models for speech recognition than the shallow n-gram models that dominate the field. He builds a broad-coverage, lexicalized probabilistic top-down parser. It grows rooted, fully connected candidate trees incrementally from left to right, using beam search over a left-factored PCFG trained on the Penn Treebank. Because every partial derivation is rooted, the parser can compute a generative probability for each prefix string. That yields a true conditional probability for each next word, which can be interpolated word by word with a trigram model. Head-first and shift-reduce parsers cannot do this. The fully specified left context also lets the model condition on many features, extracted by tree-walking functions such as c-commanding heads, at no extra search cost. The parser's accuracy and efficiency are competitive with leading statistical parsers. On the Chelba and Jelinek corpora, the resulting model beats earlier syntactic language models in perplexity. Interpolating it with a trigram gives unusually large gains, which suggests that syntax captures information largely complementary to local n-grams.

## Key points
- The core idea is rooted left-to-right derivations, which allow incremental generative word probabilities. Earley and left-corner parsers share this property, but they leave parts of the left context underspecified.
- Grammar rules are binarized through left-factorization, the same transform as "right binarization" in Roark and Johnson (1999). PCFGs are estimated with relative frequency, which guarantees they are tight.
- Perplexity is evaluated on the Chelba and Jelinek (1998) treebank-derived corpora. The model improves on prior syntax-based language models, and trigram interpolation gives large additional reductions.
- The default base beam factor is 10⁻¹¹. Narrowing the beam sharply hurts parse accuracy, coverage, and uninterpolated perplexity. Even so, interpolated perplexity stays well below the trigram baseline.
- A small speech recognition experiment supports the model's usefulness. Proposed future work includes predicting empty categories, using finer-grained conditioning, and running larger word error rate tests.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

Fields: [Natural language processing](https://md.research.elamur.ai/free/en/shelves/nlp/)

## Cited by

Cited by

95 papers of our library cite it (of the 140884 whose bibliographies we read); 4 of them have a page here, 91 not yet.

- [Head-Driven Statistical Models for Natural Language Parsing](https://md.research.elamur.ai/free/en/papers/cl-2003-collins-head-driven-statistical-models-natural/)
- [Incremental Parsing with the Perceptron Algorithm](https://md.research.elamur.ai/free/en/papers/collins-2004-incremental-parsing-perceptron-algorithm/)
- [Top-down Discourse Parsing via Sequence Labelling](https://md.research.elamur.ai/free/en/papers/koto-2021-top-down-discourse-parsing-sequence/)
- [Reading Level Assessment Using Support Vector Machines and Statistical Language Models](https://md.research.elamur.ai/free/en/papers/schwarm-2005-reading-level-assessment-support-vector/)

## Cite

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@article{roark2001probabilistic,
  title = {{Probabilistic Top-Down Parsing and Language Modeling}},
  author = {Roark, Brian},
  journal = {Computational Linguistics},
  year = {2001},
  doi = {10.1162/089120101750300526},
  url = {https://doi.org/10.1162/089120101750300526}
}
```

- [← previous Knowledge-Free Induction of Inflectional Morphologies](https://md.research.elamur.ai/free/en/papers/aclbulk-2001-schone-knowledge-free-induction-inflectional-morphologies/)
- [next → Scaling to Very Very Large Corpora for Natural Language Disambiguation](https://md.research.elamur.ai/free/en/papers/banko-2001-scaling-very-very-large-corpora/)
- [↑ back to papers of 2001](https://md.research.elamur.ai/free/en/papers/2001/)

## References

41 references, 5 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Aho, Alfred V., Ravi Sethi, and Jeffrey D. Ullman. 1986. Compilers, Principles, Techniques, and Tools. Addison-Wesley, Reading, MA. [find](https://scholar.google.com/scholar?q=Aho%2C+Alfred+V%2E%2C+Ravi+Sethi%2C+and+Jeffrey+D%2E+Ullman%2E+1986%2E+Compilers%2C+Principles%2C+Techniques%2C+and+Tools%2E+Addison%2DWesley%2C+Reading%2C+MA%2E)
2. Black, Ezra, Steven Abney, Dan Flickenger, Claudia Gdaniec, Ralph Grishman, Philip Harrison, Donald Hindle, Robert Ingria, Frederick Jelinek, Judith Klavans, Mark Liberman, Mitchell P. Marcus, Salim Roukos, Beatrice Santorini, and Tomek Strzalkowski. 1991. A procedure for quantitatively comparing th… [find](https://scholar.google.com/scholar?q=Black%2C+Ezra%2C+Steven+Abney%2C+Dan+Flickenger%2C+Claudia+Gdaniec%2C+Ralph+Grishman%2C+Philip+Harrison%2C+Donald+Hindle%2C+Robert+Ingria%2C+Frederick+Jelinek%2C+Judith+Klavans%2C+Mark+Liberman%2C+Mitchell+P%2E+Marcus%2C+Salim+R)
3. Briscoe, Ted and John Carroll. 1993. Generalized probabilistic LR parsing of natural language (corpora) with unification-based grammars. Computational Linguistics, 19(1):25-60. [find](https://scholar.google.com/scholar?q=Briscoe%2C+Ted+and+John+Carroll%2E+1993%2E+Generalized+probabilistic+LR+parsing+of+natural+language+%28corpora%29+with+unification%2Dbased+grammars%2E+Computational+Linguistics%2C+19%281%29%3A25%2D60%2E)
4. Caraballo, Sharon and Eugene Charniak. 1998. New figures of merit for best-first probabilistic chart parsing. Computational Linguistics, 24(2):275-298. [find](https://scholar.google.com/scholar?q=Caraballo%2C+Sharon+and+Eugene+Charniak%2E+1998%2E+New+figures+of+merit+for+best%2Dfirst+probabilistic+chart+parsing%2E+Computational+Linguistics%2C+24%282%29%3A275%2D298%2E)
5. Charniak, Eugene. 1997. Statistical parsing with a context-flee grammar and word statistics. In Proceedings of the Fourteenth National Conference on Arti~'cial Intelligence, pages 598-603, Menlo Park. AAAI Press/MIT Press. [find](https://scholar.google.com/scholar?q=Charniak%2C+Eugene%2E+1997%2E+Statistical+parsing+with+a+context%2Dflee+grammar+and+word+statistics%2E+In+Proceedings+of+the+Fourteenth+National+Conference+on+Arti%7E%27cial+Intelligence%2C+pages+598%2D603%2C+Menlo+Park%2E)
6. Charniak, Eugene. 2000. A maximum-entropy-inspired parser. In Proceedings of the ls t Conference of the North American Chapter of the Association for Computational Linguistics, pages 132-139. [find](https://scholar.google.com/scholar?q=Charniak%2C+Eugene%2E+2000%2E+A+maximum%2Dentropy%2Dinspired+parser%2E+In+Proceedings+of+the+ls+t+Conference+of+the+North+American+Chapter+of+the+Association+for+Computational+Linguistics%2C+pages+132%2D139%2E) [our card](https://md.research.elamur.ai/free/en/papers/charniak-2000-maximum-entropy-inspired-parser/)
7. Charniak, Eugene, Sharon Goldwater, and Mark Johnson. 1998. Edge-based best-first chart parsing. In Proceedings of the Sixth Workshop on Very Large Corpora, pages 127-133. [find](https://scholar.google.com/scholar?q=Charniak%2C+Eugene%2C+Sharon+Goldwater%2C+and+Mark+Johnson%2E+1998%2E+Edge%2Dbased+best%2Dfirst+chart+parsing%2E+In+Proceedings+of+the+Sixth+Workshop+on+Very+Large+Corpora%2C+pages+127%2D133%2E)
8. Chelba, Ciprian. 2000. Exploiting Syntactic Structure for Natural Language Modeling. Ph.D. thesis, The Johns Hopkins University. [find](https://scholar.google.com/scholar?q=Chelba%2C+Ciprian%2E+2000%2E+Exploiting+Syntactic+Structure+for+Natural+Language+Modeling%2E+Ph%2ED%2E+thesis%2C+The+Johns+Hopkins+University%2E)
9. Chelba, Ciprian and Frederick Jelinek. 1998a. Exploiting syntactic structure for language modeling. In Proceedings of the 36th Annual Meeting of the Association for Computational Linguistics and 17 th International Conference on Computational Linguistics, pages 225-231. [find](https://scholar.google.com/scholar?q=Chelba%2C+Ciprian+and+Frederick+Jelinek%2E+1998a%2E+Exploiting+syntactic+structure+for+language+modeling%2E+In+Proceedings+of+the+36th+Annual+Meeting+of+the+Association+for+Computational+Linguistics+and+17+th)
10. Chelba, Ciprian and Frederick Jelinek. 1998b. Refinement of a structured language model. In International Conference on Advances in Pattern Recognition, pages 275-284. [find](https://scholar.google.com/scholar?q=Chelba%2C+Ciprian+and+Frederick+Jelinek%2E+1998b%2E+Refinement+of+a+structured+language+model%2E+In+International+Conference+on+Advances+in+Pattern+Recognition%2C+pages+275%2D284%2E)
11. Chelba, Ciprian and Frederick Jelinek. 1999. Recognition performance of a structured language model. In Proceedings of the 6th European Conference on Speech Communication and Technology (Eurospeech), pages 1,567-1,570. [find](https://scholar.google.com/scholar?q=Chelba%2C+Ciprian+and+Frederick+Jelinek%2E+1999%2E+Recognition+performance+of+a+structured+language+model%2E+In+Proceedings+of+the+6th+European+Conference+on+Speech+Communication+and+Technology+%28Eurospeech%29%2C+)
12. Chi, Zhiyi and Stuart Geman. 1998. Estimation of probabilistic context-free grammars. Computational Linguistics, 24(2):299-305. [find](https://scholar.google.com/scholar?q=Chi%2C+Zhiyi+and+Stuart+Geman%2E+1998%2E+Estimation+of+probabilistic+context%2Dfree+grammars%2E+Computational+Linguistics%2C+24%282%29%3A299%2D305%2E)
13. Collins, Michael J. 1997. Three generative, lexicalised models for statistical parsing. In Proceedings of the 35th Annual Meeting, pages 16-23. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Collins%2C+Michael+J%2E+1997%2E+Three+generative%2C+lexicalised+models+for+statistical+parsing%2E+In+Proceedings+of+the+35th+Annual+Meeting%2C+pages+16%2D23%2E+Association+for+Computational+Linguistics%2E) [our card](https://md.research.elamur.ai/free/en/papers/collins-1997-three-generative-lexicalised-models-statistical/)
14. Collins, Michael J. 1999. Head-Driven Statistical Models for Natural Language Parsing. Ph.D. thesis, University of Pennsylvania. [find](https://scholar.google.com/scholar?q=Collins%2C+Michael+J%2E+1999%2E+Head%2DDriven+Statistical+Models+for+Natural+Language+Parsing%2E+Ph%2ED%2E+thesis%2C+University+of+Pennsylvania%2E)
15. Collins, Michael J. 2000. Discriminative reranking for natural language parsing. In The Proceedings of the 17th International Conference on Machine Learning, pages 175-182. [find](https://scholar.google.com/scholar?q=Collins%2C+Michael+J%2E+2000%2E+Discriminative+reranking+for+natural+language+parsing%2E+In+The+Proceedings+of+the+17th+International+Conference+on+Machine+Learning%2C+pages+175%2D182%2E) [our card](https://md.research.elamur.ai/free/en/papers/collins-2000-discriminative-reranking-natural-language-parsing/)
16. Cover, Thomas M. and Joy A. Thomas. 1991. Elements of Information Theory. John Wiley and Sons, Inc., New York. [find](https://scholar.google.com/scholar?q=Cover%2C+Thomas+M%2E+and+Joy+A%2E+Thomas%2E+1991%2E+Elements+of+Information+Theory%2E+John+Wiley+and+Sons%2C+Inc%2E%2C+New+York%2E)
17. Earley, Jay. 1970. An efficient context-free parsing algorithm. Communications of the ACM, 6(8):451-455. [find](https://scholar.google.com/scholar?q=Earley%2C+Jay%2E+1970%2E+An+efficient+context%2Dfree+parsing+algorithm%2E+Communications+of+the+ACM%2C+6%288%29%3A451%2D455%2E)
18. Eisner, J. and G. Satta. 1999. Efficient parsing for bilexical context-free grammars and head automaton grammars. In Proceedings of the 37th Annual Meeting, pages 457-464. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Eisner%2C+J%2E+and+G%2E+Satta%2E+1999%2E+Efficient+parsing+for+bilexical+context%2Dfree+grammars+and+head+automaton+grammars%2E+In+Proceedings+of+the+37th+Annual+Meeting%2C+pages+457%2D464%2E+Association+for+Computationa)
19. Goddeau, David. 1992. Using probabilistic shift-reduce parsing in speech recognition systems. In Proceedings of the 2nd International Conference on Spoken Language Processing, pages 321-324. [find](https://scholar.google.com/scholar?q=Goddeau%2C+David%2E+1992%2E+Using+probabilistic+shift%2Dreduce+parsing+in+speech+recognition+systems%2E+In+Proceedings+of+the+2nd+International+Conference+on+Spoken+Language+Processing%2C+pages+321%2D324%2E)
20. Goodman, Joshua. 1997. Global thresholding and multiple-pass parsing. In Proceedings of the Second Conference on Empirical Methods in Natural Language Processing (EMNLP-97), pages 11-25. [find](https://scholar.google.com/scholar?q=Goodman%2C+Joshua%2E+1997%2E+Global+thresholding+and+multiple%2Dpass+parsing%2E+In+Proceedings+of+the+Second+Conference+on+Empirical+Methods+in+Natural+Language+Processing+%28EMNLP%2D97%29%2C+pages+11%2D25%2E)
21. Henderson, John C. and Eric Brill. 1999. Exploiting diversity in natural language processing: Combining parsers. In Proceedings of the Fourth Conference on Empirical Methods in Natural Language Processing (EMNLP-99), pages 187-194. [find](https://scholar.google.com/scholar?q=Henderson%2C+John+C%2E+and+Eric+Brill%2E+1999%2E+Exploiting+diversity+in+natural+language+processing%3A+Combining+parsers%2E+In+Proceedings+of+the+Fourth+Conference+on+Empirical+Methods+in+Natural+Language+Proces)
22. Hopcroft, John E. and Jeffrey D. Ullman. 1979. Introduction to Automata Theory, Languages and Computation. Addison-Wesley. [find](https://scholar.google.com/scholar?q=Hopcroft%2C+John+E%2E+and+Jeffrey+D%2E+Ullman%2E+1979%2E+Introduction+to+Automata+Theory%2C+Languages+and+Computation%2E+Addison%2DWesley%2E)
23. Inui, Kentaro, Virach Sornlertlarnvanich, Hozurnmi Tanaka, and Takenobu Tokunaga. 1997. A new formalization of probabilistic GLR parsing. In Proceedings of the 5th International Workshop on Parsing Technologies, pages 123-134. [find](https://scholar.google.com/scholar?q=Inui%2C+Kentaro%2C+Virach+Sornlertlarnvanich%2C+Hozurnmi+Tanaka%2C+and+Takenobu+Tokunaga%2E+1997%2E+A+new+formalization+of+probabilistic+GLR+parsing%2E+In+Proceedings+of+the+5th+International+Workshop+on+Parsing+Te)
24. Jelinek, Frederick. 1997. Statistical Methods for Speech Recognition. MIT Press, Cambridge, MA. [find](https://scholar.google.com/scholar?q=Jelinek%2C+Frederick%2E+1997%2E+Statistical+Methods+for+Speech+Recognition%2E+MIT+Press%2C+Cambridge%2C+MA%2E)
25. Jelinek, Frederick and Ciprian Chelba. 1999. Putting language into language modeling. In Proceedings of the 6th European Conference on Speech Communication and Technology (Eurospeech), pages KN1-6. [find](https://scholar.google.com/scholar?q=Jelinek%2C+Frederick+and+Ciprian+Chelba%2E+1999%2E+Putting+language+into+language+modeling%2E+In+Proceedings+of+the+6th+European+Conference+on+Speech+Communication+and+Technology+%28Eurospeech%29%2C+pages+KN1%2D6%2E)
26. Jelinek, Frederick and John D. Lafferty. 1991. Computation of the probability of initial substring generation by stochastic context-free grammars. Computational Linguistics, 17(3):315-323. [find](https://scholar.google.com/scholar?q=Jelinek%2C+Frederick+and+John+D%2E+Lafferty%2E+1991%2E+Computation+of+the+probability+of+initial+substring+generation+by+stochastic+context%2Dfree+grammars%2E+Computational+Linguistics%2C+17%283%29%3A315%2D323%2E)
27. Jelinek, Frederick and Robert L. Mercer. 1980. Interpolated estimation of Markov source parameters from sparse data. In Proceedings of the Workshop on Pattern Recognition in Practice, pages 381-397. [find](https://scholar.google.com/scholar?q=Jelinek%2C+Frederick+and+Robert+L%2E+Mercer%2E+1980%2E+Interpolated+estimation+of+Markov+source+parameters+from+sparse+data%2E+In+Proceedings+of+the+Workshop+on+Pattern+Recognition+in+Practice%2C+pages+381%2D397%2E)
28. Johnson, Mark. 1998. PCFG models of linguistic tree representations. Computational Linguistics, 24(4):617-636. [find](https://scholar.google.com/scholar?q=Johnson%2C+Mark%2E+1998%2E+PCFG+models+of+linguistic+tree+representations%2E+Computational+Linguistics%2C+24%284%29%3A617%2D636%2E) [our card](https://md.research.elamur.ai/free/en/papers/johnson-1998-pcfg-models-linguistic-tree-representations/)
29. Johnson, Mark, Stuart Geman, Stephen Canon, Zhiyi Chi, and Stefan Riezler. 1999. Estimators for stochastic "unification-based" grammars. In Proceedings of the 37th Annual Meeting, pages 535-541. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Johnson%2C+Mark%2C+Stuart+Geman%2C+Stephen+Canon%2C+Zhiyi+Chi%2C+and+Stefan+Riezler%2E+1999%2E+Estimators+for+stochastic+%22unification%2Dbased%22+grammars%2E+In+Proceedings+of+the+37th+Annual+Meeting%2C+pages+535%2D541%2E+Assoc)
30. Jurafsky, Daniel, Chuck Wooters, Jonathan Segal, Andreas Stolcke, Eric Fosler, Gary Tajchman, and Nelson Morgan. 1995. Using a stochastic context-free grammar as a language model for speech recognition. In Proceedings of the IEEE Conference on Acoustics, Speech, and Signal Processing, pages 189-192. [find](https://scholar.google.com/scholar?q=Jurafsky%2C+Daniel%2C+Chuck+Wooters%2C+Jonathan+Segal%2C+Andreas+Stolcke%2C+Eric+Fosler%2C+Gary+Tajchman%2C+and+Nelson+Morgan%2E+1995%2E+Using+a+stochastic+context%2Dfree+grammar+as+a+language+model+for+speech+recognitio)
31. Manning, Christopher D. and Hinrich Schiitze. 1999. Foundations of Statistical Natural Language Processing. MIT Press, Cambridge, MA. [find](https://scholar.google.com/scholar?q=Manning%2C+Christopher+D%2E+and+Hinrich+Schiitze%2E+1999%2E+Foundations+of+Statistical+Natural+Language+Processing%2E+MIT+Press%2C+Cambridge%2C+MA%2E)
32. Marcus, Mitchell P., Beatrice Santorini, and Mary Ann Marcinkiewicz. 1993. Building a large annotated corpus of English: The Penn Treebank. Computational Linguistics, 19(2):313-330. [find](https://scholar.google.com/scholar?q=Marcus%2C+Mitchell+P%2E%2C+Beatrice+Santorini%2C+and+Mary+Ann+Marcinkiewicz%2E+1993%2E+Building+a+large+annotated+corpus+of+English%3A+The+Penn+Treebank%2E+Computational+Linguistics%2C+19%282%29%3A313%2D330%2E) [our card](https://md.research.elamur.ai/free/en/papers/bank-1993-marcus-building-large-annotated-corpus-english/)
33. Ratnaparkhi, Adwait. 1997. A linear observed time statistical parser based on maximum entropy models. In Proceedings of the Second Conference on Empirical Methods in Natural Language Processing (EMNLP-97), pages 1-10. [find](https://scholar.google.com/scholar?q=Ratnaparkhi%2C+Adwait%2E+1997%2E+A+linear+observed+time+statistical+parser+based+on+maximum+entropy+models%2E+In+Proceedings+of+the+Second+Conference+on+Empirical+Methods+in+Natural+Language+Processing+%28EMNLP)
34. Roark, Brian and Eugene Charniak. 2000. Measuring efficiency in high-accuracy, broad-coverage statistical parsing. In Proceedings of the COLING-2000 Workshop on Efficiency in Large-scale Parsing Systems, pages 29-36. [find](https://scholar.google.com/scholar?q=Roark%2C+Brian+and+Eugene+Charniak%2E+2000%2E+Measuring+efficiency+in+high%2Daccuracy%2C+broad%2Dcoverage+statistical+parsing%2E+In+Proceedings+of+the+COLING%2D2000+Workshop+on+Efficiency+in+Large%2Dscale+Parsing+Syste)
35. Roark, Brian and Mark Johnson. 1999. Efficient probabilistic top-down and left-corner parsing. In Proceedings of the 37th Annual Meeting, pages 421--428. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Roark%2C+Brian+and+Mark+Johnson%2E+1999%2E+Efficient+probabilistic+top%2Ddown+and+left%2Dcorner+parsing%2E+In+Proceedings+of+the+37th+Annual+Meeting%2C+pages+421%2D%2D428%2E+Association+for+Computational+Linguistics%2E)
36. Rosenfeld, Ronald. 1996. A maximum entropy approach to adaptive statistical language modeling. Computer, Speech and Language, 10:187-228. [find](https://scholar.google.com/scholar?q=Rosenfeld%2C+Ronald%2E+1996%2E+A+maximum+entropy+approach+to+adaptive+statistical+language+modeling%2E+Computer%2C+Speech+and+Language%2C+10%3A187%2D228%2E)
37. Rosenfeld, Ronald. 1997. A whole sentence maximum entropy language model. In Proceedings of lEEE Workshop on Speech Recognition and Understanding, pages 230-237, Rosenkrantz, Daniel J. and Philip M. Lewis II. 1970. Deterministic left corner parsing. In IEEE Conference Record of the 11th Annual Sympo… [find](https://scholar.google.com/scholar?q=Rosenfeld%2C+Ronald%2E+1997%2E+A+whole+sentence+maximum+entropy+language+model%2E+In+Proceedings+of+lEEE+Workshop+on+Speech+Recognition+and+Understanding%2C+pages+230%2D237%2C+Rosenkrantz%2C+Daniel+J%2E+and+Philip+M%2E+L)
38. Saul, Lawrence and Fernando C. N. Pereira. 1997. Aggregate and mixed-order Markov models for statistical language processing. [find](https://scholar.google.com/scholar?q=Saul%2C+Lawrence+and+Fernando+C%2E+N%2E+Pereira%2E+1997%2E+Aggregate+and+mixed%2Dorder+Markov+models+for+statistical+language+processing%2E)
39. In Proceedings of the Second Conference on Empirical Methods in Natural Language Processing (EMNLP-97), pages 81-89. [find](https://scholar.google.com/scholar?q=In+Proceedings+of+the+Second+Conference+on+Empirical+Methods+in+Natural+Language+Processing+%28EMNLP%2D97%29%2C+pages+81%2D89%2E)
40. Stolcke, Andreas. 1995. An efficient probabilistic context-free parsing algorithm that computes prefix probabilities. Computational Linguistics, 21(2):165-202. [find](https://scholar.google.com/scholar?q=Stolcke%2C+Andreas%2E+1995%2E+An+efficient+probabilistic+context%2Dfree+parsing+algorithm+that+computes+prefix+probabilities%2E+Computational+Linguistics%2C+21%282%29%3A165%2D202%2E)
41. Stolcke, Andreas and Jonathan Segal. 1994. Precise n-gram probabilities from stochastic context-free grammars. In Proceedings of the 32nd Annual Meeting, pages 74-79. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Stolcke%2C+Andreas+and+Jonathan+Segal%2E+1994%2E+Precise+n%2Dgram+probabilities+from+stochastic+context%2Dfree+grammars%2E+In+Proceedings+of+the+32nd+Annual+Meeting%2C+pages+74%2D79%2E+Association+for+Computational+Lin)