---
title: "Classifying Dialogue Acts in Multi-party Live — Kim 2012"
description: "Kim, Cavedon and Baldwin study how to automatically label chat messages with dialogue acts when many people talk at once."
authors: ["Su Nam Kim", "Lawrence Cavedon", "Timothy Baldwin"]
year: 2012
venue: "Proceedings of the 26th Pacific Asia Conference on Language, Information, and Computation"
source: "https://aclanthology.org/Y12-1050/"
licence: "CC BY-NC-SA 3.0 (ACL Anthology, before 2016)"
licence_url: "https://creativecommons.org/licenses/by-nc-sa/3.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/papers/kim-2012-classifying-dialogue-acts-multi-party/"
---
[Elamur Research](https://md.research.elamur.ai/) · [this page in HTML](https://research.elamur.ai/free/papers/kim-2012-classifying-dialogue-acts-multi-party/)

[home](https://md.research.elamur.ai/) / [papers](https://md.research.elamur.ai/free/papers/) / [2012](https://md.research.elamur.ai/free/papers/2012/) / Classifying Dialogue Acts in Multi-party Live Chats

# Classifying Dialogue Acts in Multi-party Live Chats

**Authors:** [Su Nam Kim](https://md.research.elamur.ai/free/authors/su-nam-kim/), [Lawrence Cavedon](https://md.research.elamur.ai/free/authors/lawrence-cavedon/), [Timothy Baldwin](https://md.research.elamur.ai/free/authors/timothy-baldwin/) · **Year:** 2012 · **Venue:** Proceedings of the 26th Pacific Asia Conference on Language, Information, and Computation · **Licence:** [CC BY-NC-SA 3.0 (ACL Anthology, before 2016)](https://creativecommons.org/licenses/by-nc-sa/3.0/)

**Links:** [Publisher's page](https://aclanthology.org/Y12-1050/) · [PDF](https://aclanthology.org/Y12-1050.pdf) · [OpenAlex](https://openalex.org/W2177147982)

**Citations:** 23 citations (openalex-snapshot · 2026-10-14) · [11](#cited-by) in our library, of the 140673 papers whose bibliographies we read

## Summary
Kim, Cavedon and Baldwin study how to automatically label chat messages with dialogue acts when many people talk at once. In this setting, replies from different users interleave ("entanglement"), which makes the task harder than in two-person chats. The authors built a new gold-standard corpus from topic-focused online forum chats run by the US Library of Congress. They also reused Forsyth's NPS casual chat data for comparison. Each utterance was represented with several feature groups: unigram bag-of-words, structural position, hand-picked keywords and dialogue interaction cues. These were tested with Naive Bayes, SVM and CRF classifiers. Unigrams plus keywords worked best, with CRF reaching an F-score of about 82.8 on the library data. Structural and interaction features gave little benefit, unlike earlier findings for one-to-one chats. Experiments using gold labels for preceding turns suggest that these features would become very useful if threads were first separated. This points to chat disentanglement as a key step for multi-party dialogue act tagging.

## Key points
- **New Library of Congress corpus:** 5,276 manually segmented utterances from 15 forum chats.
  - The tagset has 14 dialogue acts: Ivanovic's (2008) 12 labels, plus BACKGROUND and OTHER.
  - Inter-annotator agreement was 81.4% (kappa 0.74).
  - STATEMENT dominates the label distribution at about 48%.
- **NPS casual chats (Forsyth, 2007):** 10,567 utterances across 15 conversations, with 15 dialogue act labels and modified Penn Treebank POS tags. No agreement figures are available for this dataset.
- **Effect of keyword features:** On the library data, keywords lifted CRF from 79.26 to 82.77 F-score. Statistical significance testing by randomized estimation gave p \< 0.05.
- **Interaction features with gold labels:** When the previous utterance's gold-standard label was supplied, CRF scored 97.80 on the library data and 99.03 on NPS. Using predicted labels instead brought no gain. This indicates that noisy predictions and entanglement are the main obstacles.
- **Per-class results:** EXPRESSION, OPENING and THANKING were easy to recognise. NO-ANSWER, REQUEST and RESPONSE-ACK were hard, mainly due to low recall. REQUEST was frequently mistaken for STATEMENT.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

Fields: [Linguistics & languages](https://md.research.elamur.ai/free/shelves/linguistics/)

Topics: [Speech act theory](https://md.research.elamur.ai/free/topics/speech-act-theory/)

## Cited by

Cited by

11 papers of our library cite it (of the 140673 whose bibliographies we read); 2 of them have a page here, 9 not yet.

- [Multimodality and Dialogue Act Classification in the RoboHelper Project](https://md.research.elamur.ai/free/papers/chen-2013-multimodality-dialogue-act-classification-robohelper/)
- [Extractive Summarization and Dialogue Act Modeling on Email Threads: An Integrated Probabilistic Approach](https://md.research.elamur.ai/free/papers/oya-2014-extractive-summarization-dialogue-act-modeling/)

## Cite

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{kim2012classifying,
  title = {{Classifying Dialogue Acts in Multi-party Live Chats}},
  author = {Kim, Su Nam and Cavedon, Lawrence and Baldwin, Timothy},
  booktitle = {Proceedings of the 26th Pacific Asia Conference on Language, Information, and Computation},
  year = {2012},
  url = {https://aclanthology.org/Y12-1050/}
}
```

- [← previous Building Large Monolingual Dictionaries at the Leipzig Corpora Collection: From 100 to 200 Languages](https://md.research.elamur.ai/free/papers/goldhahn-2012-building-large-monolingual-dictionaries-leipzig/)
- [next → Coarse Lexical Semantic Annotation with Supersenses: An Arabic Case Study](https://md.research.elamur.ai/free/papers/aclbulk-2012-schneider-coarse-lexical-semantic-annotation-supersenses/)
- [↑ back to papers of 2012](https://md.research.elamur.ai/free/papers/2012/)

## References

21 references, 4 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. James Allen and Mark Core. 1997. Draft of damsl: Dialog act markup in several layers. Technical report, University of Rochester, Rochester, USA. The Multi-party Discourse Group. [find](https://scholar.google.com/scholar?q=James+Allen+and+Mark+Core%2E+1997%2E+Draft+of+damsl%3A+Dialog+act+markup+in+several+layers%2E+Technical+report%2C+University+of+Rochester%2C+Rochester%2C+USA%2E+The+Multi%2Dparty+Discourse+Group%2E)
2. Srinivas Bangalore, Giuseppe Di Fabbrizio, and Amanda Stent. 2006. Learning the structure of task-driven human-human dialogs. In Proceedings of the 21st International Conference on Computational Linguistics and 44th Annual Meeting of the Association for Computational Linguistics, pages 201–208, Sydney, Australia. [find](https://scholar.google.com/scholar?q=Srinivas+Bangalore%2C+Giuseppe+Di+Fabbrizio%2C+and+Amanda+Stent%2E+2006%2E+Learning+the+structure+of+task%2Ddriven+human%2Dhuman+dialogs%2E+In+Proceedings+of+the+21st+International+Conference+on+Computational+Lingu)
3. William W. Cohen, Vitor R. Carvalho, and Tom M. Mitchell. 2004. Learning to classify email into speech acts. In Proceedings of the 2004 Conference on Empirical Methods in Natural Language Processing (EMNLP 2004), pages 309–316, Barcelona, Spain. [find](https://scholar.google.com/scholar?q=William+W%2E+Cohen%2C+Vitor+R%2E+Carvalho%2C+and+Tom+M%2E+Mitchell%2E+2004%2E+Learning+to+classify+email+into+speech+acts%2E+In+Proceedings+of+the+2004+Conference+on+Empirical+Methods+in+Natural+Language+Processing+%28) [our card](https://md.research.elamur.ai/free/papers/cohen-2004-learning-classify-email-into-speech/)
4. Gao Cong, Long Wang, Chin-Yew Lin, Young-In Song, and Yueheng Sun. 2008. Finding question-answer pairs from online forums. In Proceedings of 31st International ACM-SIGIR Conference on Research and Development in Information Retrieval (SIGIR08), pages 467–474, Singapore. [find](https://scholar.google.com/scholar?q=Gao+Cong%2C+Long+Wang%2C+Chin%2DYew+Lin%2C+Young%2DIn+Song%2C+and+Yueheng+Sun%2E+2008%2E+Finding+question%2Danswer+pairs+from+online+forums%2E+In+Proceedings+of+31st+International+ACM%2DSIGIR+Conference+on+Research+and+Dev)
5. Micha Elsner and Eugene Charniak. 2008. You talking to me? a corpus and algorithm for conversation disentanglement. In Proceedings of the 46th Annual Meeting of the ACL: HLT (ACL 2008), pages 834–842, Columbus, USA. [find](https://scholar.google.com/scholar?q=Micha+Elsner+and+Eugene+Charniak%2E+2008%2E+You+talking+to+me%3F+a+corpus+and+algorithm+for+conversation+disentanglement%2E+In+Proceedings+of+the+46th+Annual+Meeting+of+the+ACL%3A+HLT+%28ACL+2008%29%2C+pages+834%E2%80%93842%2C)
6. Eric N. Forsyth. 2007. Improving automated lexical and discourse analysis of online chat dialog. Master’s thesis, Naval Postgraduate School. features used are stemmed uni-Edward Ivanovic. 2008. Automatic instant messaging dialogue using statistical models and dialogue acts. Master’s thesis, The University of Melbourne. [find](https://scholar.google.com/scholar?q=Eric+N%2E+Forsyth%2E+2007%2E+Improving+automated+lexical+and+discourse+analysis+of+online+chat+dialog%2E+Master%E2%80%99s+thesis%2C+Naval+Postgraduate+School%2E+features+used+are+stemmed+uni%2DEdward+Ivanovic%2E+2008%2E+Automa)
7. Daniel Jurafsky, Elizabeth Shriberg, Barbara Fox, and Traci Curl. 1998. Lexical, prosodic, and syntactic cues for dialog acts. In Proceedings of ACL/COLING-98 Workshop on Discourse Relations and Discourse Markers, pages 114–120, Montreal, Canada. [find](https://scholar.google.com/scholar?q=Daniel+Jurafsky%2C+Elizabeth+Shriberg%2C+Barbara+Fox%2C+and+Traci+Curl%2E+1998%2E+Lexical%2C+prosodic%2C+and+syntactic+cues+for+dialog+acts%2E+In+Proceedings+of+ACL%2FCOLING%2D98+Workshop+on+Discourse+Relations+and+Disco) [our card](https://md.research.elamur.ai/free/papers/jurafsky-1998-lexical-prosodic-syntactic-cues-dialog/)
8. Dan Jurafsky, Rajesh Ranganath, and Dan McFarland. 2009. Extracting social meaning: identifying interactional style in spoken conversation. In Proceedings of Human Language Technologies: The 2009 Annual Conference of the North American Chapter of the Association for Computational Linguistics (NAACL HLT 2009), pages 638–646, Boulder, USA. [find](https://scholar.google.com/scholar?q=Dan+Jurafsky%2C+Rajesh+Ranganath%2C+and+Dan+McFarland%2E+2009%2E+Extracting+social+meaning%3A+identifying+interactional+style+in+spoken+conversation%2E+In+Proceedings+of+Human+Language+Technologies%3A+The+2009+Annu)
9. Su Nam Kim, Lawrence Cavedon, and Timothy Baldwin. 2010a. Classifying dialogue acts in 1-to-1 live chats. In Proceedings of the 2010 Conference on Empirical Methods in Natural Language Processing (EMNLP 2010), pages 862–871, Boston, USA. [find](https://scholar.google.com/scholar?q=Su+Nam+Kim%2C+Lawrence+Cavedon%2C+and+Timothy+Baldwin%2E+2010a%2E+Classifying+dialogue+acts+in+1%2Dto%2D1+live+chats%2E+In+Proceedings+of+the+2010+Conference+on+Empirical+Methods+in+Natural+Language+Processing+%28EMN)
10. Su Nam Kim, Li Wang, and Timothy Baldwin. 2010b. Tagging and linking web forum posts. In Proceedings of the Fourteenth Conference on Computational Natural Language Learning, pages 192–202, Uppsala, Sweden. [find](https://scholar.google.com/scholar?q=Su+Nam+Kim%2C+Li+Wang%2C+and+Timothy+Baldwin%2E+2010b%2E+Tagging+and+linking+web+forum+posts%2E+In+Proceedings+of+the+Fourteenth+Conference+on+Computational+Natural+Language+Learning%2C+pages+192%E2%80%93202%2C+Uppsala%2C+Sw)
11. Andrew Lampert, Robert Dale, and Cecile Paris. 2008. The nature of requests and ecommitments in email messages. In Proceedings of the AAAI 2008 Workshop on Enhanced Messaging, pages 42–47, Chicago, USA. [find](https://scholar.google.com/scholar?q=Andrew+Lampert%2C+Robert+Dale%2C+and+Cecile+Paris%2E+2008%2E+The+nature+of+requests+and+ecommitments+in+email+messages%2E+In+Proceedings+of+the+AAAI+2008+Workshop+on+Enhanced+Messaging%2C+pages+42%E2%80%9347%2C+Chicago%2C+US)
12. Andrew Kachites McCallum. 2002. MALLET: A machine learning for language toolkit. [http://mallet.cs.umass.edu](http://mallet.cs.umass.edu). [↗](http://mallet.cs.umass.edu)
13. Guido Minnen, John Carroll, and Darren Pearce. 2001. Applied morphological processing of English. Natural Language Engineering, 7(3):207–223. [↗](https://doi.org/10.1017/s1351324901002728)
14. Gabriel Murray, Steve Renals, Jean Carletta, and Johanna Moore. 2006. Incorporating speaker and discourse features into speech summarization. In Proceedings of the Main Conference on Human Language Technology Conference of the North American Chapter of the Association of Computational Linguistics, pages 367– 374, New York, USA. [find](https://scholar.google.com/scholar?q=Gabriel+Murray%2C+Steve+Renals%2C+Jean+Carletta%2C+and+Johanna+Moore%2E+2006%2E+Incorporating+speaker+and+discourse+features+into+speech+summarization%2E+In+Proceedings+of+the+Main+Conference+on+Human+Language+Te)
15. Ken Samuel, Sandra Carberry, and K. Vijay-Shanker. 1998. Dialogue act tagging with transformation-based learning. In Proceedings of COLING/ACL 1998, pages 1150–1156, Montreal, Canada. [↗](https://doi.org/10.48550/arxiv.cmp-lg/9806006) [our card](https://md.research.elamur.ai/free/papers/samuel-1998-dialogue-act-tagging-transformation-based/)
16. Andreas Stolcke, Klaus Ries, Noah Coccaro, Elizabeth Shriberg, Rebecca Bates, Daniel Jurafsky, Paul Taylor, Rachel Martin, Carol Van Ess-Dykema, and Marie Meteer. 2000. Dialogue act modeling for automatic tagging and recognition of conversational speech. Computational Linguistics, 26(3):339–373. [↗](https://doi.org/10.1162/089120100561737) [our card](https://md.research.elamur.ai/free/papers/prag-2000-stolcke-dialogue-act-modeling-automatic-tagging/)
17. Li Wang, Marco Lui, Su Nam Kim, Joakim Nivre, and Timothy Baldwin. 2011. Predicting thread discourse structure over technical web forums. In Proceedings of the 2011 Conference on Empirical Methods in Natural Language Processing (EMNLP 2011), pages 13– 25, Edinburgh, UK. [find](https://scholar.google.com/scholar?q=Li+Wang%2C+Marco+Lui%2C+Su+Nam+Kim%2C+Joakim+Nivre%2C+and+Timothy+Baldwin%2E+2011%2E+Predicting+thread+discourse+structure+over+technical+web+forums%2E+In+Proceedings+of+the+2011+Conference+on+Empirical+Methods+in+)
18. Christopher C. Werry. 1996. Linguistic and interactional features of internet relay chat. In Susan C. Herring, editor, Computer-Mediated Communication. John Benjamins, Amsterdam, the Netherlands. [↗](https://doi.org/10.1075/pbns.39.06wer)
19. Ian Witten and Eibe Frank. 2005. Data Mining: Practical Machine Learning Tools and Techniques. Morgan Kaufmann, San Francisco, USA. [find](https://scholar.google.com/scholar?q=Ian+Witten+and+Eibe+Frank%2E+2005%2E+Data+Mining%3A+Practical+Machine+Learning+Tools+and+Techniques%2E+Morgan+Kaufmann%2C+San+Francisco%2C+USA%2E)
20. Tianhao Wu, Faisal M. Khan, Todd A. Fisher, Lori A. Shuler, and William M. Pottenger. 2002. Posting act tagging using transformation-based learning. In Proceedings of the Workshop on Foundations of Data Mining and Discovery, IEEE International Conference on Data Mining, Maebashi City, Japan. [find](https://scholar.google.com/scholar?q=Tianhao+Wu%2C+Faisal+M%2E+Khan%2C+Todd+A%2E+Fisher%2C+Lori+A%2E+Shuler%2C+and+William+M%2E+Pottenger%2E+2002%2E+Posting+act+tagging+using+transformation%2Dbased+learning%2E+In+Proceedings+of+the+Workshop+on+Foundations+of+Da)
21. Alexander Yeh. 2000. More accurate tests for the statistical significance of result differences. In Proceedings of the 18th International Conference on Computational Linguistics (COLING 2000), pages 947–953, Saarbrücken, Germany. [↗](https://doi.org/10.48550/arxiv.cs/0008005)