---
title: "Bayesian Unsupervised Topic Segmentation — Eisenstein 2008"
description: "The paper tackles unsupervised linear topic segmentation: splitting a document into a sequence of… 32 references, cited by 62 papers here."
authors: ["Jacob Eisenstein", "Regina Barzilay"]
year: 2008
venue: "Proceedings of the 2008 Conference on Empirical Methods in Natural Language Processing"
source: "https://aclanthology.org/D08-1035/"
licence: "CC BY-NC-SA 3.0 (ACL Anthology, before 2016)"
licence_url: "https://creativecommons.org/licenses/by-nc-sa/3.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/eisenstein-2008-bayesian-unsupervised-topic-segmentation/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/eisenstein-2008-bayesian-unsupervised-topic-segmentation/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2008](https://md.research.elamur.ai/free/en/papers/2008/) / Bayesian Unsupervised Topic Segmentation

# Bayesian Unsupervised Topic Segmentation

**Authors:** [Jacob Eisenstein](https://md.research.elamur.ai/free/en/authors/jacob-eisenstein/) and [Regina Barzilay](https://md.research.elamur.ai/free/en/authors/regina-barzilay/) · **Year:** 2008 · **Venue:** [Proceedings of the 2008 Conference on Empirical Methods in Natural Language Processing](https://md.research.elamur.ai/free/en/venues/emnlp/) ([EMNLP 2008](https://md.research.elamur.ai/free/en/venues/emnlp/2008/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY-NC-SA 3.0 (ACL Anthology, before 2016)](https://creativecommons.org/licenses/by-nc-sa/3.0/)

**Links:** [Publisher's page](https://aclanthology.org/D08-1035/) · [PDF](https://aclanthology.org/D08-1035.pdf) · [OpenAlex](https://openalex.org/W2027823133)

**Citations:** 235 citations (OpenAlex, 2026-10-14) · [62](https://md.research.elamur.ai/free/en/papers/eisenstein-2008-bayesian-unsupervised-topic-segmentation/cited-by/) in our library, of the 141,840 papers whose bibliographies we read

## Summary
The paper tackles unsupervised linear topic segmentation: splitting a document into a sequence of topically coherent segments without labeled data. Earlier unsupervised systems measured lexical cohesion with hand-designed similarity scores, which are hard to combine with other cues and often need tuning. Eisenstein and Barzilay instead cast cohesion as a generative model. Each segment has its own multinomial language model, and the sentences in a segment are drawn from it. A segmentation with high likelihood therefore concentrates each segment's vocabulary. The language models are either point-estimated or integrated out via the Dirichlet Compound Multinomial. Dynamic programming then finds the exact best segmentation. Cue phrases are added as emissions from segment boundaries, drawn from one shared multinomial, and a sampler handles inference for that version. On the ICSI meeting corpus and a new medical-textbook corpus, the system (BayesSeg) beats U&I, MCS and LCSeg on both Pk and WindowDiff. The work matters because it turns an ad hoc heuristic into a principled model that extends naturally to new features and needs no tuning.

## Key points
- Evaluation used the ICSI meeting transcripts and a newly built text corpus taken from a medical textbook. All systems were given the desired number of segments.
- On the textbook corpus, BayesSeg scored Pk .339 and WD .353, against roughly .368–.385 for the baselines.
- On the meeting corpus, the best variant (BayesSeg-cue-prop) reached Pk .258 and WD .312. The strongest baseline results were U&I's Pk of .297 and LCSeg's WD of .322. Purver et al. (2006) reported .289 and .329.
- Cue phrases helped on meetings but not on the textbook.
  - On meetings, the top induced cues included "okay", "so", "um" and "but". Four of the top ten matched the supervised list of Galley et al. (2003).
  - On the textbook, mostly very frequent words such as "the" were selected.
- The model needs no development-set tuning because its priors are re-estimated. Code is released at groups.csail.mit.edu/rbg/code/bayesseg/.

## Interesting ideas
- **Replace a heuristic score with a likelihood.** Reframing "cohesion" as the likelihood under per-segment language models makes the objective principled. It also makes the objective extensible, since new evidence becomes just another generated variable. The same move could work for other tasks driven by similarity heuristics, such as chunking or document clustering.
- **Recover old methods as special cases.** With a zero prior, the objective becomes a length-weighted sum of segment entropies. U&I's smoothing corresponds to one particular prior choice. Showing that an existing method falls out of a general model is a useful way to justify and unify prior work.
- **Use a shared multinomial to induce markers without labels.** Generating boundary words from one distribution shared across documents pushes the model toward a compact set of markers. Ranking the results by chi-squared then gives an interpretable sanity check. This could help discover discourse markers in under-resourced languages.
- **Inject knowledge through the proposal, not the model.** Biasing the sampler toward boundaries that contain known cue phrases steers the search while leaving the probabilistic model untouched. This is a cheap way to add linguistic knowledge without hurting the model's validity.
- **Exploit linear structure for exact inference.** Restricting the search to segmentation points and marginalizing out the language models allows exact dynamic programming. This avoids the cost of sampling many layers of hidden variables, as LDA-style segmenters must.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Massachusetts Institute of Technology](https://md.research.elamur.ai/free/en/universities/massachusetts-institute-of-technology/) — University, Cambridge, United States · 159 works · 65,157 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 9,086 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 8,889 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 9,296 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 9,206 works

## Cite

Cite as: Jacob Eisenstein and Regina Barzilay. 2008. Bayesian Unsupervised Topic Segmentation. Proceedings of the 2008 Conference on Empirical Methods in Natural Language Processing.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{eisenstein2008bayesian,
  title = {{Bayesian Unsupervised Topic Segmentation}},
  author = {Jacob Eisenstein and Regina Barzilay},
  booktitle = {Proceedings of the 2008 Conference on Empirical Methods in Natural Language Processing},
  year = {2008},
  doi = {10.3115/1613715.1613760},
  url = {https://doi.org/10.3115/1613715.1613760}
}
```

- [← previous Finding Contradictions in Text](https://md.research.elamur.ai/free/en/papers/marneffe-2008-finding-contradictions-text/)
- [next → Cross-Language Parser Adaptation between Related Languages](https://md.research.elamur.ai/free/en/papers/zeman-2008-cross-language-parser-adaptation-between/)
- [↑ back to papers of 2008](https://md.research.elamur.ai/free/en/papers/2008/)

## References

32 references, 11 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Doug Beeferman, Adam Berger, and John D. Lafferty. 1999. Statistical models for text segmentation. Machine Learning, 34(1-3):177–210. [find](https://scholar.google.com/scholar?q=Doug+Beeferman%2C+Adam+Berger%2C+and+John+D%2E+Lafferty%2E+1999%2E+Statistical+models+for+text+segmentation%2E+Machine+Learning%2C+34%281%2D3%29%3A177%E2%80%93210%2E)
2. José M. Bernardo and Adrian F. M. Smith. 2000. Bayesian Theory. Wiley. [find](https://scholar.google.com/scholar?q=Jose%CC%81+M%2E+Bernardo+and+Adrian+F%2E+M%2E+Smith%2E+2000%2E+Bayesian+Theory%2E+Wiley%2E)
3. Christopher M. Bishop. 2006. Pattern Recognition and Machine Learning. Springer. [find](https://scholar.google.com/scholar?q=Christopher+M%2E+Bishop%2E+2006%2E+Pattern+Recognition+and+Machine+Learning%2E+Springer%2E)
4. David M. Blei, Andrew Y. Ng, and Michael I. Jordan. 2003. [Latent Dirichlet allocation](https://md.research.elamur.ai/free/en/papers/blei-2003-latent-dirichlet-allocation/). Journal of Machine Learning Research, 3:993–1022.
5. Freddy Y. Y. Choi. 2000. [Advances in domain independent linear text segmentation](https://md.research.elamur.ai/free/en/papers/choi-2000-advances-domain-independent-linear-text/). In Proceedings of NAACL, pages 26–33.
6. Micha Elsner and Eugene Charniak. 2008. You Talking to Me? A Corpus and Algorithm for Conversation Disentanglement. In Proceedings of ACL. [find](https://scholar.google.com/scholar?q=Micha+Elsner+and+Eugene+Charniak%2E+2008%2E+You+Talking+to+Me%3F+A+Corpus+and+Algorithm+for+Conversation+Disentanglement%2E+In+Proceedings+of+ACL%2E)
7. Jenny Rose Finkel, Trond Grenager, and Christopher Manning. 2005. [Incorporating non-local information into information extraction systems by gibbs sampling](https://md.research.elamur.ai/free/en/papers/finkel-2005-incorporating-non-local-information-into/). In Proceedings of ACL, pages 363–370.
8. Michel Galley, Kathleen R. McKeown, Eric Fosler-Lussier, and Hongyan Jing. 2003. [Discourse segmentation of multi-party conversation](https://md.research.elamur.ai/free/en/papers/galley-2003-discourse-segmentation-multi-party-conversation/). Proceedings of ACL, pages 562–569.
9. Jean-Luc Gauvain and Chin-Hui Lee. 1994. Maximum a posteriori estimation for multivariate Gaussian mixture observations of Markov chains. IEEE Transactions on Speech and Audio Processing, 2(2):291–298. [find](https://scholar.google.com/scholar?q=Jean%2DLuc+Gauvain+and+Chin%2DHui+Lee%2E+1994%2E+Maximum+a+posteriori+estimation+for+multivariate+Gaussian+mixture+observations+of+Markov+chains%2E+IEEE+Transactions+on+Speech+and+Audio+Processing%2C+2%282%29%3A291%E2%80%93298)
10. Dmitriy Genzel and Eugene Charniak. 2002. [Entropy rate constancy in text](https://md.research.elamur.ai/free/en/papers/genzel-2002-entropy-rate-constancy-text/). In Proceedings of ACL, pages 199–206.
11. Sharon Goldwater and Tom Griffiths. 2007. [A fully bayesian approach to unsupervised part-of-speech tagging](https://md.research.elamur.ai/free/en/papers/goldwater-2007-fully-bayesian-approach-unsupervised-part/). In Proceedings of ACL, pages 744–751.
12. Barbara Grosz and Candace Sidner. 1986. [Attention, intentions, and the structure of discourse](https://md.research.elamur.ai/free/en/papers/grosz-1986-attention-intentions-structure-discourse/). Computational Linguistics, 12(3):175–204.
13. Barbara Grosz. 1977. The representation and use of focus in dialogue understanding. Technical Report 151, Artificial Intelligence Center, SRI International. [find](https://scholar.google.com/scholar?q=Barbara+Grosz%2E+1977%2E+The+representation+and+use+of+focus+in+dialogue+understanding%2E+Technical+Report+151%2C+Artificial+Intelligence+Center%2C+SRI+International%2E)
14. M. A. K. Halliday and Ruqaiya Hasan. 1976. Cohesion in English. Longman. [find](https://scholar.google.com/scholar?q=M%2E+A%2E+K%2E+Halliday+and+Ruqaiya+Hasan%2E+1976%2E+Cohesion+in+English%2E+Longman%2E)
15. Marti A. Hearst. 1994. Multi-paragraph segmentation of expository text. In Proceedings of ACL, pages 9–16. [find](https://scholar.google.com/scholar?q=Marti+A%2E+Hearst%2E+1994%2E+Multi%2Dparagraph+segmentation+of+expository+text%2E+In+Proceedings+of+ACL%2C+pages+9%E2%80%9316%2E)
16. Julia Hirschberg and Diane Litman. 1993. [Empirical studies on the disambiguation of cue phrases](https://md.research.elamur.ai/free/en/papers/hirschberg-1993-empirical-studies-disambiguation-cue-phrases/). Computational Linguistics, 19(3):501–530.
17. A. Janin, D. Baron, J. Edwards, D. Ellis, D. Gelbart, N. Morgan, B. Peskin, T. Pfau, E. Shriberg, A. Stolcke, et al. 2003. [find](https://scholar.google.com/scholar?q=A%2E+Janin%2C+D%2E+Baron%2C+J%2E+Edwards%2C+D%2E+Ellis%2C+D%2E+Gelbart%2C+N%2E+Morgan%2C+B%2E+Peskin%2C+T%2E+Pfau%2C+E%2E+Shriberg%2C+A%2E+Stolcke%2C+et+al%2E+2003%2E)
18. The ICSI Meeting Corpus. Acoustics, Speech, and Signal Processing, 2003. Proceedings.(ICASSP’03). 2003 IEEE International Conference on, 1. [find](https://scholar.google.com/scholar?q=The+ICSI+Meeting+Corpus%2E+Acoustics%2C+Speech%2C+and+Signal+Processing%2C+2003%2E+Proceedings%2E%28ICASSP%E2%80%9903%29%2E+2003+IEEE+International+Conference+on%2C+1%2E)
19. Norman L. Johnson, Samuel Kotz, and N. Balakrishnan. 1997. Discrete Multivariate Distributions. Wiley. [find](https://scholar.google.com/scholar?q=Norman+L%2E+Johnson%2C+Samuel+Kotz%2C+and+N%2E+Balakrishnan%2E+1997%2E+Discrete+Multivariate+Distributions%2E+Wiley%2E)
20. Alistair Knott. 1996. A Data-Driven Methodology for Motivating a Set of Coherence Relations. Ph.D. thesis, University of Edinburgh. [find](https://scholar.google.com/scholar?q=Alistair+Knott%2E+1996%2E+A+Data%2DDriven+Methodology+for+Motivating+a+Set+of+Coherence+Relations%2E+Ph%2ED%2E+thesis%2C+University+of+Edinburgh%2E)
21. Diane J. Litman and Rebecca J. Passonneau. 1995. Combining multiple knowledge sources for discourse segmentation. In Proceedings of the ACL, pages 108–115. [find](https://scholar.google.com/scholar?q=Diane+J%2E+Litman+and+Rebecca+J%2E+Passonneau%2E+1995%2E+Combining+multiple+knowledge+sources+for+discourse+segmentation%2E+In+Proceedings+of+the+ACL%2C+pages+108%E2%80%93115%2E)
22. Igor Malioutov and Regina Barzilay. 2006. [Minimum cut model for spoken lecture segmentation](https://md.research.elamur.ai/free/en/papers/malioutov-2006-minimum-cut-model-spoken-lecture/). In Proceedings of ACL, pages 25–32.
23. Christopher D. Manning and Hinrich Schütze. 1999. Foundations of Statistical Natural Language Processing. The MIT Press. [find](https://scholar.google.com/scholar?q=Christopher+D%2E+Manning+and+Hinrich+Schu%CC%88tze%2E+1999%2E+Foundations+of+Statistical+Natural+Language+Processing%2E+The+MIT+Press%2E)
24. Thomas P. Minka. 2003. Estimating a dirichlet distribution. Technical report, Massachusetts Institute of Technology. [find](https://scholar.google.com/scholar?q=Thomas+P%2E+Minka%2E+2003%2E+Estimating+a+dirichlet+distribution%2E+Technical+report%2C+Massachusetts+Institute+of+Technology%2E)
25. Rebecca Passonneau and Diane Litman. 1993. Intention-based segmentation: Human reliability and correlation with linguistic cues. In Proceedings of ACL, pages 148–155. [find](https://scholar.google.com/scholar?q=Rebecca+Passonneau+and+Diane+Litman%2E+1993%2E+Intention%2Dbased+segmentation%3A+Human+reliability+and+correlation+with+linguistic+cues%2E+In+Proceedings+of+ACL%2C+pages+148%E2%80%93155%2E)
26. Lev Pevzner and Marti A. Hearst. 2002. [A critique and improvement of an evaluation metric for text segmentation](https://md.research.elamur.ai/free/en/papers/pevzner-2002-critique-improvement-evaluation-metric-text/). Computational Linguistics, 28(1):19–36.
27. M. F. Porter. 1980. An algorithm for suffix stripping. Program, 14:130–137. [find](https://scholar.google.com/scholar?q=M%2E+F%2E+Porter%2E+1980%2E+An+algorithm+for+suffix+stripping%2E+Program%2C+14%3A130%E2%80%93137%2E)
28. M. Purver, T.L. Griffiths, K.P. Körding, and J.B. Tenenbaum. 2006. Unsupervised topic modelling for multi-party spoken discourse. In Proceedings of ACL, pages 17–24. [find](https://scholar.google.com/scholar?q=M%2E+Purver%2C+T%2EL%2E+Griffiths%2C+K%2EP%2E+Ko%CC%88rding%2C+and+J%2EB%2E+Tenenbaum%2E+2006%2E+Unsupervised+topic+modelling+for+multi%2Dparty+spoken+discourse%2E+In+Proceedings+of+ACL%2C+pages+17%E2%80%9324%2E)
29. Caroline Sporleder and Mirella Lapata. 2006. Broad coverage paragraph segmentation across languages and domains. ACM Transactions on Speech and Language Processing, 3(2):1–35. [find](https://scholar.google.com/scholar?q=Caroline+Sporleder+and+Mirella+Lapata%2E+2006%2E+Broad+coverage+paragraph+segmentation+across+languages+and+domains%2E+ACM+Transactions+on+Speech+and+Language+Processing%2C+3%282%29%3A1%E2%80%9335%2E)
30. Masao Utiyama and Hitoshi Isahara. 2001. [A statistical model for domain-independent text segmentation](https://md.research.elamur.ai/free/en/papers/utiyama-2001-statistical-model-domain-independent-text/). In Proceedings of ACL, pages 491–498.
31. H. Kenneth Walker, W. Dallas Hall, and J. Willis Hurst, editors. 1990. Clinical Methods : The History, Physical, and Laboratory Examinations. Butterworths. [find](https://scholar.google.com/scholar?q=H%2E+Kenneth+Walker%2C+W%2E+Dallas+Hall%2C+and+J%2E+Willis+Hurst%2C+editors%2E+1990%2E+Clinical+Methods+%3A+The+History%2C+Physical%2C+and+Laboratory+Examinations%2E+Butterworths%2E)
32. Greg C. G. Wei and Martin A. Tanner. 1990. A monte carlo implementation of the EM algorithm and the poor man’s data augmentation algorithms. Journal of the American Statistical Association, 85(411), September. [find](https://scholar.google.com/scholar?q=Greg+C%2E+G%2E+Wei+and+Martin+A%2E+Tanner%2E+1990%2E+A+monte+carlo+implementation+of+the+EM+algorithm+and+the+poor+man%E2%80%99s+data+augmentation+algorithms%2E+Journal+of+the+American+Statistical+Association%2C+85%28411%29%2C+S)

*Updated 24 October 2026*