---
title: "Unsupervised Modeling of Twitter Conversations — Ritter 2010"
description: "The paper addresses how to find dialogue acts (shallow conversational roles such as questions or replies) in open-domain chat without a hand-built tag…"
authors: ["Alan Ritter", "Colin Cherry", "Bill Dolan"]
year: 2010
venue: "Human Language Technologies: The 2010 Annual Conference of the North American Chapter of the Association for Computational Linguistics"
source: "https://aclanthology.org/N10-1020/"
licence: "CC BY-NC-SA 3.0 (ACL Anthology, before 2016)"
licence_url: "https://creativecommons.org/licenses/by-nc-sa/3.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/ritter-2010-unsupervised-modeling-twitter-conversations/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/ritter-2010-unsupervised-modeling-twitter-conversations/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2010](https://md.research.elamur.ai/free/en/papers/2010/) / Unsupervised Modeling of Twitter Conversations

# Unsupervised Modeling of Twitter Conversations

**Authors:** [Alan Ritter](https://md.research.elamur.ai/free/en/authors/alan-ritter/), [Colin Cherry](https://md.research.elamur.ai/free/en/authors/colin-cherry/), [Bill Dolan](https://md.research.elamur.ai/free/en/authors/bill-dolan/) · **Institutions:** [University of Washington](https://md.research.elamur.ai/free/en/universities/university-of-washington/) · [Microsoft](https://md.research.elamur.ai/free/en/companies/microsoft/) · [National Research Council Canada](https://md.research.elamur.ai/free/en/government/national-research-council-canada/) · **Year:** 2010 · **Venue:** Human Language Technologies: The 2010 Annual Conference of the North American Chapter of the Association for Computational Linguistics · **Licence:** [CC BY-NC-SA 3.0 (ACL Anthology, before 2016)](https://creativecommons.org/licenses/by-nc-sa/3.0/)

**Links:** [Publisher's page](https://aclanthology.org/N10-1020/) · [PDF](https://aclanthology.org/N10-1020.pdf) · [OpenAlex](https://openalex.org/W1654173042)

**Citations:** 405 citations (OpenAlex, 2026-10-14) · [152](https://md.research.elamur.ai/free/en/papers/ritter-2010-unsupervised-modeling-twitter-conversations/cited-by/) in our library, of the 140884 papers whose bibliographies we read

## Summary
The paper addresses how to find dialogue acts (shallow conversational roles such as questions or replies) in open-domain chat without a hand-built tag inventory or annotated data. The authors crawled Twitter for two months in summer 2009 and assembled about 1.3 million English reply-chain conversations, which they planned to release publicly. Their starting point is an HMM borrowed from Barzilay and Lee's content models: each post is emitted by one hidden act as a unigram bag of words, and acts follow a Markov chain. Trained with EM, this model tended to group conversations by topic (for example, food) rather than by dialogue function. A Conversation+Topic model fixes this with an LDA-style split: each word comes from the post's act, the conversation's topic, or general English. Bayesian variants are trained with Gibbs sampling. Evaluation uses visualization plus a new intrinsic task: recovering the original order of shuffled posts. Bayesian inference beat EM. The topic-aware model produced the clearest acts, but it was not the best at ordering. The work shows that dialogue structure can be learned quickly for a new medium.

## Key points
- Corpus: roughly 1.3M Twitter conversations, each with 2 to 243 posts. Two-post exchanges make up 69%, and conversation lengths follow a power law. Threads were rebuilt from reply links in the Twitter API, so no disentanglement step was needed.
- Twitter text is very noisy, with even pronouns and prepositions spelled many ways. Jcluster word clustering exposed spelling variants such as b4/be4/befor.
- Models compared: the EM Conversation model (an HMM trained with forward-backward), the Conversation+Topic model, and the Bayesian Conversation model. Each was tested with 5 to 40 acts.
- Ordering evaluation:
  - Setup: 1,000 held-out conversations, with every n! permutation of each one scored and Kendall's τ computed against the true order.
  - Baseline: a bigram model reached only τ = 0.0358, compared with 0.19–0.74 reported on news data.
- Further results:
  - Bayesian models achieved much higher held-out likelihood than EM, and EM overfit as the number of states grew.
  - The Bayesian Conversation model ordered posts better than Conversation+Topic, but the latter was far more interpretable.
  - Learned acts included Status, Question and Response, which differ in their use of the pronouns "I" and "you".

## Interesting ideas
- **Remove topic through the model, not the data.** Giving each word a choice of source (act, topic, or background language) separates content from function. This could suit other noisy domains where masking words with NER or POS tools fails.
- **Use reordering as a label-free evaluation.** Asking a model to restore the order of shuffled units tests its sequential knowledge without any annotated tag set. The idea could transfer to evaluating discourse or document-structure models in low-resource languages.
- **Report interpretability and prediction separately.** The model that predicted order best was not the most interpretable one. Projects should therefore not rely on a single intrinsic metric when choosing models.
- **Borrow models across genres.** A summarization content model, with its hidden states reinterpreted as dialogue acts, became a dialogue model. Architectures from neighboring tasks are worth checking before designing new ones.
- **Use platform metadata for structure.** Reply links supplied clean conversation threads for free. Similar metadata on other platforms could avoid costly reconstruction steps when building corpora.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

Fields: [Natural language processing](https://md.research.elamur.ai/free/en/shelves/nlp/)

## Cite

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{ritter2010unsupervised,
  title = {{Unsupervised Modeling of Twitter Conversations}},
  author = {Ritter, Alan and Cherry, Colin and Dolan, Bill},
  booktitle = {Human Language Technologies: The 2010 Annual Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2010},
  url = {https://aclanthology.org/N10-1020/}
}
```

- [← previous Collecting Image Annotations Using Amazon's Mechanical Turk](https://md.research.elamur.ai/free/en/papers/rashtchian-2010-collecting-image-annotations-amazon-s/)
- [next → A Monolingual Tree-based Translation Model for Sentence Simplification](https://md.research.elamur.ai/free/en/papers/zhu-2010-monolingual-tree-based-translation-model/)
- [↑ back to papers of 2010](https://md.research.elamur.ai/free/en/papers/2010/)

## References

28 references, 6 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. James Allen, Nathanael Chambers, George Ferguson, Lucian Galescu, Hyuckchul Jung, Mary Swift, and William Taysom. 2007. Plow: a collaborative task learning agent. In Proceedings of AAAI. [find](https://scholar.google.com/scholar?q=James+Allen%2C+Nathanael+Chambers%2C+George+Ferguson%2C+Lucian+Galescu%2C+Hyuckchul+Jung%2C+Mary+Swift%2C+and+William+Taysom%2E+2007%2E+Plow%3A+a+collaborative+task+learning+agent%2E+In+Proceedings+of+AAAI%2E)
2. Regina Barzilay and Lillian Lee. 2004. Catching the drift: Probabilistic content models, with applications to generation and summarization. In Proceedings of HLT-NAACL, pages 113–120. [find](https://scholar.google.com/scholar?q=Regina+Barzilay+and+Lillian+Lee%2E+2004%2E+Catching+the+drift%3A+Probabilistic+content+models%2C+with+applications+to+generation+and+summarization%2E+In+Proceedings+of+HLT%2DNAACL%2C+pages+113%E2%80%93120%2E) [our card](https://md.research.elamur.ai/free/en/papers/barzilay-2004-catching-drift-probabilistic-content-models/)
3. David M. Blei, Andrew Y. Ng, and Michael I. Jordan. 2003. Latent dirichlet allocation. J. Mach. Learn. Res., 3:993–1022. [find](https://scholar.google.com/scholar?q=David+M%2E+Blei%2C+Andrew+Y%2E+Ng%2C+and+Michael+I%2E+Jordan%2E+2003%2E+Latent+dirichlet+allocation%2E+J%2E+Mach%2E+Learn%2E+Res%2E%2C+3%3A993%E2%80%931022%2E) [our card](https://md.research.elamur.ai/free/en/papers/blei-2003-latent-dirichlet-allocation/)
4. William W. Cohen, Vitor R. Carvalho, and Tom M. Mitchell. 2004. Learning to classify email into “speech acts”. In Proceedings of EMNLP. [find](https://scholar.google.com/scholar?q=William+W%2E+Cohen%2C+Vitor+R%2E+Carvalho%2C+and+Tom+M%2E+Mitchell%2E+2004%2E+Learning+to+classify+email+into+%E2%80%9Cspeech+acts%E2%80%9D%2E+In+Proceedings+of+EMNLP%2E) [our card](https://md.research.elamur.ai/free/en/papers/cohen-2004-learning-classify-email-into-speech/)
5. Nigel Crook, Ramon Granell, and Stephen Pulman. 2009. Unsupervised classification of dialogue acts using a Dirichlet process mixture model. In Proceedings of SIGDIAL, pages 341–348. [find](https://scholar.google.com/scholar?q=Nigel+Crook%2C+Ramon+Granell%2C+and+Stephen+Pulman%2E+2009%2E+Unsupervised+classification+of+dialogue+acts+using+a+Dirichlet+process+mixture+model%2E+In+Proceedings+of+SIGDIAL%2C+pages+341%E2%80%93348%2E)
6. Hal Daumé III and Daniel Marcu. 2006. Bayesian query-focused summarization. In Proceedings of ACL. [find](https://scholar.google.com/scholar?q=Hal+Daume%CC%81+III+and+Daniel+Marcu%2E+2006%2E+Bayesian+query%2Dfocused+summarization%2E+In+Proceedings+of+ACL%2E)
7. Rajdip Dhillon, Sonali Bhagat, Hannah Carvey, and Elizabeth Shriberg. 2004. Meeting recorder project: Dialog act labeling guide. Technical report, International Computer Science Institute. [find](https://scholar.google.com/scholar?q=Rajdip+Dhillon%2C+Sonali+Bhagat%2C+Hannah+Carvey%2C+and+Elizabeth+Shriberg%2E+2004%2E+Meeting+recorder+project%3A+Dialog+act+labeling+guide%2E+Technical+report%2C+International+Computer+Science+Institute%2E)
8. Micha Elsner and Eugene Charniak. 2008. You talking to me? A corpus and algorithm for conversation disentanglement. In Proceedings of ACL-HLT. [find](https://scholar.google.com/scholar?q=Micha+Elsner+and+Eugene+Charniak%2E+2008%2E+You+talking+to+me%3F+A+corpus+and+algorithm+for+conversation+disentanglement%2E+In+Proceedings+of+ACL%2DHLT%2E)
9. Eric N. Forsyth and Craig H. Martell. 2007. Lexical and discourse analysis of online chat dialog. In Proceedings of ICSC. [find](https://scholar.google.com/scholar?q=Eric+N%2E+Forsyth+and+Craig+H%2E+Martell%2E+2007%2E+Lexical+and+discourse+analysis+of+online+chat+dialog%2E+In+Proceedings+of+ICSC%2E)
10. Michael Gamon, Sumit Basu, Dmitriy Belenko, Danyel Fisher, Matthew Hurst, and Arnd Christian Knig. 2008. Blews: Using blogs to provide context for news articles. In Proceedings of ICWSM. [find](https://scholar.google.com/scholar?q=Michael+Gamon%2C+Sumit+Basu%2C+Dmitriy+Belenko%2C+Danyel+Fisher%2C+Matthew+Hurst%2C+and+Arnd+Christian+Knig%2E+2008%2E+Blews%3A+Using+blogs+to+provide+context+for+news+articles%2E+In+Proceedings+of+ICWSM%2E)
11. Sharon Goldwater and Tom Griffiths. 2007. A fully bayesian approach to unsupervised part-of-speech tagging. In Proceedings of ACL, pages 744–751. [find](https://scholar.google.com/scholar?q=Sharon+Goldwater+and+Tom+Griffiths%2E+2007%2E+A+fully+bayesian+approach+to+unsupervised+part%2Dof%2Dspeech+tagging%2E+In+Proceedings+of+ACL%2C+pages+744%E2%80%93751%2E) [our card](https://md.research.elamur.ai/free/en/papers/goldwater-2007-fully-bayesian-approach-unsupervised-part/)
12. Joshua T. Goodman. 2001. A bit of progress in language modeling. Technical report. [find](https://scholar.google.com/scholar?q=Joshua+T%2E+Goodman%2E+2001%2E+A+bit+of+progress+in+language+modeling%2E+Technical+report%2E)
13. T. L. Griffiths and M. Steyvers. 2004. Finding scientific topics. Proc Natl Acad Sci, 101 Suppl 1:5228–5235. [find](https://scholar.google.com/scholar?q=T%2E+L%2E+Griffiths+and+M%2E+Steyvers%2E+2004%2E+Finding+scientific+topics%2E+Proc+Natl+Acad+Sci%2C+101+Suppl+1%3A5228%E2%80%935235%2E)
14. Aria Haghighi and Lucy Vanderwende. 2009. Exploring content models for multi-document summarization. In Proceedings of HLT-NAACL, pages 362–370. [find](https://scholar.google.com/scholar?q=Aria+Haghighi+and+Lucy+Vanderwende%2E+2009%2E+Exploring+content+models+for+multi%2Ddocument+summarization%2E+In+Proceedings+of+HLT%2DNAACL%2C+pages+362%E2%80%93370%2E) [our card](https://md.research.elamur.ai/free/en/papers/haghighi-2009-exploring-content-models-multi-document/)
15. Yijue How and Min-Yen Kan. 2005. Optimizing predictive text entry for short message service on mobile phones. In Proceedings of HCII. [find](https://scholar.google.com/scholar?q=Yijue+How+and+Min%2DYen+Kan%2E+2005%2E+Optimizing+predictive+text+entry+for+short+message+service+on+mobile+phones%2E+In+Proceedings+of+HCII%2E)
16. Minwoo Jeong, Chin-Yew Lin, and Gary Geunbae Lee. 2009. Semi-supervised speech act recognition in emails and forums. In Proceedings of EMNLP, pages 1250–1259. [find](https://scholar.google.com/scholar?q=Minwoo+Jeong%2C+Chin%2DYew+Lin%2C+and+Gary+Geunbae+Lee%2E+2009%2E+Semi%2Dsupervised+speech+act+recognition+in+emails+and+forums%2E+In+Proceedings+of+EMNLP%2C+pages+1250%E2%80%931259%2E)
17. Dan Jurafsky, Liz Shriberg, and Debra Biasca. 1997. Switchboard swbd-damsl shallow-discourse-function annotation coders manual, draft 13. Technical report, University of Colorado Institute of Cognitive Science. [find](https://scholar.google.com/scholar?q=Dan+Jurafsky%2C+Liz+Shriberg%2C+and+Debra+Biasca%2E+1997%2E+Switchboard+swbd%2Ddamsl+shallow%2Ddiscourse%2Dfunction+annotation+coders+manual%2C+draft+13%2E+Technical+report%2C+University+of+Colorado+Institute+of+Cognitiv)
18. Ryan Kelly. 2009. Pear analytics twitter study. Whitepaper, August. [find](https://scholar.google.com/scholar?q=Ryan+Kelly%2E+2009%2E+Pear+analytics+twitter+study%2E+Whitepaper%2C+August%2E)
19. Iain Murray and Ruslan Salakhutdinov. 2008. Evaluating probabilities under high-dimensional latent variable models. In Proceedings of NIPS, pages 1137– 1144. [find](https://scholar.google.com/scholar?q=Iain+Murray+and+Ruslan+Salakhutdinov%2E+2008%2E+Evaluating+probabilities+under+high%2Ddimensional+latent+variable+models%2E+In+Proceedings+of+NIPS%2C+pages+1137%E2%80%93+1144%2E)
20. Gabriel Murray, Steve Renals, Jean Carletta, and Johanna Moore. 2006. Incorporating speaker and discourse features into speech summarization. In Proceedings of HLT-NAACL, pages 367–374. [find](https://scholar.google.com/scholar?q=Gabriel+Murray%2C+Steve+Renals%2C+Jean+Carletta%2C+and+Johanna+Moore%2E+2006%2E+Incorporating+speaker+and+discourse+features+into+speech+summarization%2E+In+Proceedings+of+HLT%2DNAACL%2C+pages+367%E2%80%93374%2E)
21. Radford M. Neal. 2003. Slice sampling. Annals of Statistics, 31:705–767. [find](https://scholar.google.com/scholar?q=Radford+M%2E+Neal%2E+2003%2E+Slice+sampling%2E+Annals+of+Statistics%2C+31%3A705%E2%80%93767%2E)
22. Rajesh Ranganath, Dan Jurafsky, and Dan Mcfarland. 2009. It’s not you, it’s me: Detecting flirting and its misperception in speed-dates. In Proceedings of EMNLP, pages 334–342. [find](https://scholar.google.com/scholar?q=Rajesh+Ranganath%2C+Dan+Jurafsky%2C+and+Dan+Mcfarland%2E+2009%2E+It%E2%80%99s+not+you%2C+it%E2%80%99s+me%3A+Detecting+flirting+and+its+misperception+in+speed%2Ddates%2E+In+Proceedings+of+EMNLP%2C+pages+334%E2%80%93342%2E)
23. Andreas Stolcke, Noah Coccaro, Rebecca Bates, Paul Taylor, Carol Van Ess-Dykema, Klaus Ries, Elizabeth Shriberg, Daniel Jurafsky, Rachel Martin, and Marie Meteer. 2000. Dialogue act modeling for automatic tagging and recognition of conversational speech. Computational Linguistics, 26(3):339–373. [find](https://scholar.google.com/scholar?q=Andreas+Stolcke%2C+Noah+Coccaro%2C+Rebecca+Bates%2C+Paul+Taylor%2C+Carol+Van+Ess%2DDykema%2C+Klaus+Ries%2C+Elizabeth+Shriberg%2C+Daniel+Jurafsky%2C+Rachel+Martin%2C+and+Marie+Meteer%2E+2000%2E+Dialogue+act+modeling+for+autom) [our card](https://md.research.elamur.ai/free/en/papers/prag-2000-stolcke-dialogue-act-modeling-automatic-tagging/)
24. Hanna M. Wallach, Iain Murray, Ruslan Salakhutdinov, and David M. Mimno. 2009. Evaluation methods for topic models. In Proceedings of ICML, page 139. [find](https://scholar.google.com/scholar?q=Hanna+M%2E+Wallach%2C+Iain+Murray%2C+Ruslan+Salakhutdinov%2C+and+David+M%2E+Mimno%2E+2009%2E+Evaluation+methods+for+topic+models%2E+In+Proceedings+of+ICML%2C+page+139%2E)
25. Yorick Wilks. 2006. Artificial companions as a new kind of interface to the future internet. In OII Research Report No. 13. [find](https://scholar.google.com/scholar?q=Yorick+Wilks%2E+2006%2E+Artificial+companions+as+a+new+kind+of+interface+to+the+future+internet%2E+In+OII+Research+Report+No%2E+13%2E)
26. M. Woszczyna and A. Waibel. 1994. Inferring linguistic structure in spoken language. In Proceedings of IC-SLP. [find](https://scholar.google.com/scholar?q=M%2E+Woszczyna+and+A%2E+Waibel%2E+1994%2E+Inferring+linguistic+structure+in+spoken+language%2E+In+Proceedings+of+IC%2DSLP%2E)
27. Tae Yano, William W. Cohen, and Noah A. Smith. 2009. Predicting response to political blog posts with topic models. In Proceedings of NAACL, pages 477–485. [find](https://scholar.google.com/scholar?q=Tae+Yano%2C+William+W%2E+Cohen%2C+and+Noah+A%2E+Smith%2E+2009%2E+Predicting+response+to+political+blog+posts+with+topic+models%2E+In+Proceedings+of+NAACL%2C+pages+477%E2%80%93485%2E)
28. Limin Yao, David Mimno, and Andrew McCallum. 2009. Efficient methods for topic model inference on streaming document collections. In Proceedings of KDD, pages 937–946. [find](https://scholar.google.com/scholar?q=Limin+Yao%2C+David+Mimno%2C+and+Andrew+McCallum%2E+2009%2E+Efficient+methods+for+topic+model+inference+on+streaming+document+collections%2E+In+Proceedings+of+KDD%2C+pages+937%E2%80%93946%2E)