--- title: "Classifying Dialogue Acts in One-on-One Live — Kim 2010" description: "The paper addresses automatic dialogue act tagging in written one-on-one customer-service chats." authors: ["Su Nam Kim", "Lawrence Cavedon", "Timothy Baldwin"] year: 2010 venue: "Proceedings of the 2010 Conference on Empirical Methods in Natural Language Processing" source: "https://aclanthology.org/D10-1084/" licence: "CC BY-NC-SA 3.0 (ACL Anthology, before 2016)" licence_url: "https://creativecommons.org/licenses/by-nc-sa/3.0/" summary_licence: "Elamur Cosmic License" canonical: "https://research.elamur.ai/free/papers/kim-2010-classifying-dialogue-acts-one-one/" --- [Elamur Research](https://md.research.elamur.ai/) · [this page in HTML](https://research.elamur.ai/free/papers/kim-2010-classifying-dialogue-acts-one-one/) [home](https://md.research.elamur.ai/) / [papers](https://md.research.elamur.ai/free/papers/) / [2010](https://md.research.elamur.ai/free/papers/2010/) / Classifying Dialogue Acts in One-on-One Live Chats # Classifying Dialogue Acts in One-on-One Live Chats **Authors:** [Su Nam Kim](https://md.research.elamur.ai/free/authors/su-nam-kim/), [Lawrence Cavedon](https://md.research.elamur.ai/free/authors/lawrence-cavedon/), [Timothy Baldwin](https://md.research.elamur.ai/free/authors/timothy-baldwin/) · **Year:** 2010 · **Venue:** Proceedings of the 2010 Conference on Empirical Methods in Natural Language Processing · **Licence:** [CC BY-NC-SA 3.0 (ACL Anthology, before 2016)](https://creativecommons.org/licenses/by-nc-sa/3.0/) **Links:** [Publisher's page](https://aclanthology.org/D10-1084/) · [PDF](https://aclanthology.org/D10-1084.pdf) · [OpenAlex](https://openalex.org/W1542146969) **Citations:** 85 citations (openalex-snapshot · 2026-10-14) · [40](#cited-by) in our library, of the 140673 papers whose bibliographies we read ## Summary The paper addresses automatic dialogue act tagging in written one-on-one customer-service chats. This setting has received much less study than spoken dialogue, and it lacks the acoustic and prosodic cues that speech-based classifiers rely on. The authors use MSN Online Shopping chats between customers and sales agents, annotated with the 12-label dialogue act scheme of Ivanovic (2008), and take that work's n-gram approach as their starting point. They first refine bag-of-words features by varying n-gram size, comparing raw words with lemmas, and testing several term weighting methods. They then add two new feature groups: - **Structural cues:** where an utterance sits within the chat and within a turn, and who wrote it. - **Inter-utterance dependencies:** the dialogue act labels of earlier utterances. Naive Bayes, SVM, HMM and CRF learners are compared. CRFs clearly perform best because they model sequence directly. Combining structural and dependency features pushes accuracy to about 97%. Results with predicted rather than gold labels are nearly as good, which suggests the method could support partial automation of live-chat customer service. ## Key points - **Data and labels:** MSN Online Shopping chat data, with the 12 DAMSL-derived acts from Ivanovic (2008), such as CONVENTIONAL OPENING, THANKS, YES NO QUESTION and STATEMENT. The earlier n-gram baseline on this data reached about 80% F-score. - **BoW variants:** 1-, 2- and 3-grams were tested alone and in combination, using words or lemmas and Boolean, TF, TF·IDF or Information Gain weights. Lemmas gave a slight gain. - **Dependency features:** LabelList, LabelPrev, LabelAuthor, LabelPrev_t and LabelAuthor_t were added to BoW. They helped only with CRF, raising accuracy from .862 to up to .878 (BoW+P+P_t, gold labels). The full label history (LabelList) hurt performance. - **Best result:** CRF with lemmatized unigrams, relative position, author, and LabelAuthor or LabelPrev_t achieved 96.86% accuracy. SVM and NB stayed below 0.64. - **Errors:** STATEMENT, the majority class, absorbs many misclassified REQUEST and RESPONSE ACK utterances. Punctuation features were not used and might improve results. *Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.* Fields: [Linguistics & languages](https://md.research.elamur.ai/free/shelves/linguistics/) Topics: [Speech act theory](https://md.research.elamur.ai/free/topics/speech-act-theory/) ## Cited by Cited by 40 papers of our library cite it (of the 140673 whose bibliographies we read); 4 of them have a page here, 36 not yet. - [Multimodality and Dialogue Act Classification in the RoboHelper Project](https://md.research.elamur.ai/free/papers/chen-2013-multimodality-dialogue-act-classification-robohelper/) - [Multi-level Gated Recurrent Neural Network for dialog act classification](https://md.research.elamur.ai/free/papers/li-2016-multi-level-gated-recurrent-neural/) - [Speaker-change Aware CRF for Dialogue Act Classification](https://md.research.elamur.ai/free/papers/shang-2020-speaker-change-aware-crf-dialogue/) - [Dialogue Act Recognition in Synchronous and Asynchronous Conversations](https://md.research.elamur.ai/free/papers/tavafi-2013-dialogue-act-recognition-synchronous-asynchronous/) ## Cite The paper itself, not this page. BibTeX from the library's record of the work. ```bibtex @inproceedings{kim2010classifying, title = {{Classifying Dialogue Acts in One-on-One Live Chats}}, author = {Kim, Su Nam and Cavedon, Lawrence and Baldwin, Timothy}, booktitle = {Proceedings of the 2010 Conference on Empirical Methods in Natural Language Processing}, year = {2010}, url = {https://aclanthology.org/D10-1084/} } ``` - [← previous CCASH: A Web Application Framework for Efficient, Distributed Language Resource Development](https://md.research.elamur.ai/free/papers/lingbulk-2010-felt-ccash-web-application-framework-efficient/) - [next → Clitics in Arabic Language: A Statistical Study](https://md.research.elamur.ai/free/papers/aclbulk-2010-alotaiby-clitics-arabic-language-statistical-study/) - [↑ back to papers of 2010](https://md.research.elamur.ai/free/papers/2010/) ## References 21 references, 3 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet. 1. J.Allen and M.Core. Draft of DAMSL: Dialog Act Markup in Several Layers. The Multiparty Discourse Group. University of Rochester, Rochester, USA. 1997. A. Anderson, M. Bader, E. Bard, E. Boyle G.M. Doherty, S. Garrod, S. Isard, J. Kowtko, J. McAllister, J. Miller, C. Sotillo, H.S. Thompson, R. and Weinert. The HCRC Map Task Corpus. [find](https://scholar.google.com/scholar?q=J%2EAllen+and+M%2ECore%2E+Draft+of+DAMSL%3A+Dialog+Act+Markup+in+Several+Layers%2E+The+Multiparty+Discourse+Group%2E+University+of+Rochester%2C+Rochester%2C+USA%2E+1997%2E+A%2E+Anderson%2C+M%2E+Bader%2C+E%2E+Bard%2C+E%2E+Boyle+G%2EM%2E+Do) 2. Language and Speech. 1991, 34, pp. 351–366. J. Ang, Y. Liu and E. Shriberg. Automatic Dialog Act Segmentation and Classification in Multiparty Meetings. IEEE International Conference on Acoustics, Speech, and Signal Processing. 2005, pp, 1061–1064. S. Bangalore, G. Di Fabbrizio and A. Stent. Learning the Structure of Task-Driven Human-Human Dialogs. Proceedings of the 21st COLING and 44th ACL. 2006, pp. 201–208. [find](https://scholar.google.com/scholar?q=Language+and+Speech%2E+1991%2C+34%2C+pp%2E+351%E2%80%93366%2E+J%2E+Ang%2C+Y%2E+Liu+and+E%2E+Shriberg%2E+Automatic+Dialog+Act+Segmentation+and+Classification+in+Multiparty+Meetings%2E+IEEE+International+Conference+on+Acoustics%2C+Spe) 3. H. H. Bui. A general model for online probabilistic plan recognition. IJCAI. 2003, pp. 1309–1318. [find](https://scholar.google.com/scholar?q=H%2E+H%2E+Bui%2E+A+general+model+for+online+probabilistic+plan+recognition%2E+IJCAI%2E+2003%2C+pp%2E+1309%E2%80%931318%2E) 4. G.Y Chung. Sentence retrieval for abstracts of randomized controlled trials. BMC Medical Informatics and Decision Making. 2009, 9(10), pp. 1–13. [↗](https://doi.org/10.1186/1472-6947-9-10) 5. F. Debole and F. Sebastiani. Supervised term weighting for automated text categorization. 18th ACM Symposium on Applied Computing. 2003, pp. 784–788. [↗](https://doi.org/10.1007/978-3-540-45219-5_7) 6. Cl 0 0 0 0 0 0 0 0 0 1 0 0 Op 0 0 0 0 0 0 0 0 0 0 0 1 Dp 0 0 0 0 0 0 0 1 0 0 0 0 Ex 0 0 0 0 0 0 0 0 0 0 0 0 No 0 0 0 0 0 0 0 1 0 0 0 0 Op Cl Dp Ex No Qu Rq Ack St Ta Yes YN Table 9: Confusion matrix for errors from the CRF with BoW+Position+Author+LabelAuthor (rows = correct classification; columns = misclassification; C ONVENTIONAL C LOSING = Cl; C ONVENTIONAL O PENING = Op; D OWN - PLAYER = Dp; E XPRESSIVE = Ex; N O A NSWER = No; O PEN Q UESTION = Qu; R EQUEST = Rq; R ESPONSE A CK = Ack; S TATEMENT = St; T HANKS = Ta; Y ES A NSWER = Yes; and Y ES N O Q UESTION = YN) E. N. Forsyth. Improving Automated Lexical and Discourse Analysis of Online Chat Dialog. Master’s thesis. Naval Postgraduate School, 2007. [find](https://scholar.google.com/scholar?q=Cl+0+0+0+0+0+0+0+0+0+1+0+0+Op+0+0+0+0+0+0+0+0+0+0+0+1+Dp+0+0+0+0+0+0+0+1+0+0+0+0+Ex+0+0+0+0+0+0+0+0+0+0+0+0+No+0+0+0+0+0+0+0+1+0+0+0+0+Op+Cl+Dp+Ex+No+Qu+Rq+Ack+St+Ta+Yes+YN+Table+9%3A+Confusion+matrix+f) 7. J. Godfrey and E. Holliman and J. McDaniel. SWITCH-BOARD: Telephone speech corpus for research and development. Proceedings of IEEE International Conference on Acoustics, Speech, and Signal Processing. 1992, pp. 517–520. [find](https://scholar.google.com/scholar?q=J%2E+Godfrey+and+E%2E+Holliman+and+J%2E+McDaniel%2E+SWITCH%2DBOARD%3A+Telephone+speech+corpus+for+research+and+development%2E+Proceedings+of+IEEE+International+Conference+on+Acoustics%2C+Speech%2C+and+Signal+Processing) 8. S. Grau, E. Sanchis, M. Jose and D. Vilar. Dialogue act classification using a Bayesian approach. Proceedings of the 9th Conference on Speech and Computer. 2004. P. A. Heeman and J. Allen. The Trains 93 Dialogues. Trains Technical Note 94-2. Computer Science Dept., University of Rochester, March 1995. [find](https://scholar.google.com/scholar?q=S%2E+Grau%2C+E%2E+Sanchis%2C+M%2E+Jose+and+D%2E+Vilar%2E+Dialogue+act+classification+using+a+Bayesian+approach%2E+Proceedings+of+the+9th+Conference+on+Speech+and+Computer%2E+2004%2E+P%2E+A%2E+Heeman+and+J%2E+Allen%2E+The+Trains+) 9. T. Joachims. Text categorization with support vector machines: Learning with many relevant features. Proceedings of European Conference on Machine Learning. 1998, pp. 137–142. M. Johnston, S. Bangalore, G. Vasireddy, A. Stent, P. Ehlen, M. Walker, S. Whittaker and P. Maloor. MATCH: An Architecture for Multimodal Dialogue Systems. Proceedings of 40th ACL. 2002, pp. 376– 383. F. N. Julia and K. M. Iftekharuddin. Dialog Act classification using acoustic and discourse information of MapTask Data. Proceedings of the International Joint Conference on Neural Networks. 2008, pp. 1472– 1479. D. Jurafsky, E. Shriberg, B Fox and T. Curl. Lexical, Prosodic, and Syntactic Cues for Dialog Acts. Proceedings of ACL/COLING-98 Workshop on Discourse Relations and Discourse Markers. 1998, pp. 114–120. [↗](https://doi.org/10.17877/de290r-5097) [our card](https://md.research.elamur.ai/free/papers/joachims-1998-text-categorization-support-vector-machines/) 10. Qu 0 0 0 0 0 0 0 0 0 0 0 0 Rq 0 0 0 0 0 0 0 0 1 0 0 0 Ack 0 0 0 0 0 0 0 0 0 0 0 0 St 0 1 0 0 0 0 3 5 0 0 0 0 Ta 0 1 0 0 0 0 0 0 0 0 0 0 Yes 0 0 0 0 0 0 0 0 0 0 0 0 YN 2 0 0 0 0 0 0 0 0 0 0 0 E. Ivanovic. Automatic instant messaging dialogue using statistical models and dialogue acts. Master’s Thesis. The University of Melbourne. 2008. [find](https://scholar.google.com/scholar?q=Qu+0+0+0+0+0+0+0+0+0+0+0+0+Rq+0+0+0+0+0+0+0+0+1+0+0+0+Ack+0+0+0+0+0+0+0+0+0+0+0+0+St+0+1+0+0+0+0+3+5+0+0+0+0+Ta+0+1+0+0+0+0+0+0+0+0+0+0+Yes+0+0+0+0+0+0+0+0+0+0+0+0+YN+2+0+0+0+0+0+0+0+0+0+0+0+E%2E+Ivanov) 11. S. Keizer. A Bayesian Approach to Dialogue Act Classification. 5th Workshop on Formal Semantics and Pragmatics of Dialogue. 2001, pp. 210–218. S.N. Kim and L. Wang and T. Baldwin. Tagging and Linking Web Forum Posts. Fourteenth Conference on Computational Natural Language Learning. 2010. J. Lafferty, A. McCallum and F. Pereira. Conditional random fields: Probabilistic models for segmenting and labeling sequence data. Proceedings of ICML. 2001, pp. 282–289. [find](https://scholar.google.com/scholar?q=S%2E+Keizer%2E+A+Bayesian+Approach+to+Dialogue+Act+Classification%2E+5th+Workshop+on+Formal+Semantics+and+Pragmatics+of+Dialogue%2E+2001%2C+pp%2E+210%E2%80%93218%2E+S%2EN%2E+Kim+and+L%2E+Wang+and+T%2E+Baldwin%2E+Tagging+and+Linking+) 12. D. J. Litman and S. Silliman. ITSPOKE: An Intelligent Tutoring Spoken Dialogue SYstem. Proceedings of the HLT/NAACL. 2004. [find](https://scholar.google.com/scholar?q=D%2E+J%2E+Litman+and+S%2E+Silliman%2E+ITSPOKE%3A+An+Intelligent+Tutoring+Spoken+Dialogue+SYstem%2E+Proceedings+of+the+HLT%2FNAACL%2E+2004%2E) 13. M. M. Louwerse and S. Crossley. Dialog Act Classification Using N -Gram Algorithms. FLAIRS Conference, 2006, pp. 758–763. [find](https://scholar.google.com/scholar?q=M%2E+M%2E+Louwerse+and+S%2E+Crossley%2E+Dialog+Act+Classification+Using+N+%2DGram+Algorithms%2E+FLAIRS+Conference%2C+2006%2C+pp%2E+758%E2%80%93763%2E) 14. G. Minnen, J. Carroll and D. Pearce. Applied morphological processing of English Natural Language Engineering 2000, 7(3), pp. 77–80. M. Purver, J. Niekrasz and S. Peters. Ontology-Based Discourse Understanding for a Persistent Meeting Assistant. Proc. CHI 2005 Workshop on The Virtuality Continuum Revisited. 2005. [find](https://scholar.google.com/scholar?q=G%2E+Minnen%2C+J%2E+Carroll+and+D%2E+Pearce%2E+Applied+morphological+processing+of+English+Natural+Language+Engineering+2000%2C+7%283%29%2C+pp%2E+77%E2%80%9380%2E+M%2E+Purver%2C+J%2E+Niekrasz+and+S%2E+Peters%2E+Ontology%2DBased+Discourse+Unde) 15. K. Samuel, Sandra Carberry and K. Vijay-Shanker. Dialogue Act Tagging with Transformation-Based Learning. Proceedings of COLING/ACL 1998. 1998, pp. 1150-1156. [↗](https://doi.org/10.48550/arxiv.cmp-lg/9806006) [our card](https://md.research.elamur.ai/free/papers/samuel-1998-dialogue-act-tagging-transformation-based/) 16. E. Shriberg, R. Bates, P. Taylor, A. Stolcke, D. Jurafsky, K. Ries, N. Coccaro, R. Martin, M. Meteer and C. Van Ess-Dykema. Can Prosody Aid the Automatic Classification of Dialog Acts in Conversational Speech?. Language and Speech. 1998, 41(3-4), pp. 439–487. [↗](https://doi.org/10.1177/002383099804100410) 17. V. R. Sridhar, S. Bangalore and S. Narayanan. Combining lexical, syntactic and prosodic cues for improved online dialog act tagging. Computer Speech and Language. 2009, 23(4), pp. 407–422. [↗](https://doi.org/10.1016/j.csl.2008.12.001) 18. A. Stolcke, K. Ries, N. Coccaro, E. Shriberg, R. Bates, D. Jurafsky, P. Taylor, R. Martin, C. Van Ess-Dykema and M. Meteer. Dialogue Act Modeling for Automatic Tagging and Recognition of Conversational Speech. Computational Linguistics. 2000, 26(3), pp. 339–373. [↗](https://doi.org/10.1162/089120100561737) [our card](https://md.research.elamur.ai/free/papers/prag-2000-stolcke-dialogue-act-modeling-automatic-tagging/) 19. A. Stolcke and E. Shriberg. Markovian Combination of Language and Prosodic Models for better Speech Understanding and Recognition . Invited talk at the IEEE Workshop on Speech Recognition and Understanding, Madonna di Campiglio, Italy, December 2001 2001, C. C. Werry. Linguistic and interactional features of Internet Relay Chat. In S. C. Herring (ed.). Computer-Mediated Communication. [find](https://scholar.google.com/scholar?q=A%2E+Stolcke+and+E%2E+Shriberg%2E+Markovian+Combination+of+Language+and+Prosodic+Models+for+better+Speech+Understanding+and+Recognition+%2E+Invited+talk+at+the+IEEE+Workshop+on+Speech+Recognition+and+Understa) 20. I. Witten and E. Frank. Data Mining: Practical Machine Learning Tools and Techniques. Morgan Kaufmann, 2005. [find](https://scholar.google.com/scholar?q=I%2E+Witten+and+E%2E+Frank%2E+Data+Mining%3A+Practical+Machine+Learning+Tools+and+Techniques%2E+Morgan+Kaufmann%2C+2005%2E) 21. T. Wu, F. M. Khan, T. A. Fisher, L. A. Shuler and W. M. Pottenger. Posting act tagging using transformation-based learning. Proceedings of the Workshop on Foundations of Data Mining and Discovery, IEEE International Conference on Data Mining. 2002. [find](https://scholar.google.com/scholar?q=T%2E+Wu%2C+F%2E+M%2E+Khan%2C+T%2E+A%2E+Fisher%2C+L%2E+A%2E+Shuler+and+W%2E+M%2E+Pottenger%2E+Posting+act+tagging+using+transformation%2Dbased+learning%2E+Proceedings+of+the+Workshop+on+Foundations+of+Data+Mining+and+Discovery%2C+IEE)