---
title: "Experiments with Domain Dependent Dialogue Act Classification"
description: "They build a CRF sequence labeler in MALLET. 22 references. Among them: 0."
authors: ["Swapnil Hingmire", "Apoorv Shrivastava", "Girish Palshikar", "Saurabh Srivastava"]
year: 2017
venue: "Proceedings of the 14th International Conference on Natural Language Processing (ICON-2017)"
source: "https://aclanthology.org/W17-7538/"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/hingmire-2017-experiments-domain-dependent-dialogue-act/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/hingmire-2017-experiments-domain-dependent-dialogue-act/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2017](https://md.research.elamur.ai/free/en/papers/2017/page/5/) / Experiments with Domain Dependent Dialogue Act Classification using Open-Domain Dialogue Corpora

# Experiments with Domain Dependent Dialogue Act Classification using Open-Domain Dialogue Corpora

**Authors:** Swapnil Hingmire, Apoorv Shrivastava, Girish Palshikar and [Saurabh Srivastava](https://md.research.elamur.ai/free/en/authors/saurabh-srivastava/) · **Year:** 2017 · **Venue:** [Proceedings of the 14th International Conference on Natural Language Processing (ICON-2017)](https://md.research.elamur.ai/free/en/venues/international-conference-natural-language-processing/) ([International conference natural language processing 2017](https://md.research.elamur.ai/free/en/venues/international-conference-natural-language-processing/2017/)) · **Publisher:** [NLP Association of India](https://md.research.elamur.ai/free/en/publishers/nlp-association-of-india/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/W17-7538/) · [PDF](https://aclanthology.org/W17-7538.pdf) · [OpenAlex](https://openalex.org/W2955382734)

**Citations:** 0 citations (OpenAlex, 2026-10-14)

## Summary
Dialogue act (DA) classifiers usually need large annotated corpora, and they are usually tested only in the domain they were trained on. Domain-specific chatbots (finance, tourism, helpdesks) rarely have such labeled data. The authors ask whether a classifier trained on open-domain dialogue can be carried over to a new domain without new annotation. They build a CRF sequence labeler in MALLET. Its features combine lemmas, PoS tags, bigrams of both, and hand-designed cue indicators for questions, gratitude, apology, conversation openings and agreement. On top of the CRF's probability output, a domain expert can write simple priority rules that flip a near-tied prediction toward a preferred act. They train on SwDA or DSTC2 and test on DSTC2 and a small Mutual Funds chat corpus. The comparison system is an LSTM classifier following Khanpour et al. (2016). Both models degrade sharply across domains, but the CRF transfers far better than the RNN, and the rules add further gains. This suggests that cheap expert knowledge can partly replace domain annotation.

## Key points
- **Data:**
  - SwDA: 1,155 Switchboard dialogues, 43 DA tags, 19 test conversations.
  - DSTC2: 2,118 restaurant-assistant conversations, with 19 user goals mapped to DAs.
  - Mutual Funds: 26 customer-service chats, 572 utterances.
- **Training on SwDA, accuracy (RNN / CRF / CRF + Rules):**
  - DSTC2: 21.9 / 46.4 / 61.7.
  - Mutual Funds: 14.5 / 58.0 / 63.2.
- **In-domain results are close:**
  - SwDA→SwDA: 68.9 / 66.9 / 67.1.
  - DSTC2→DSTC2: 94.1 / 89.8 / 90.1.
- **Training on DSTC2 transfers poorly:**
  - SwDA: about 11–22.
  - Mutual Funds: 33.2 / 32.9, rising to 43.3 with rules.
- **Priority rule format:** DA1→DA2 means "prefer DA2 if it is second and within a probability threshold of DA1." Example: STATEMENT→WH-QUESTION with threshold 0.3.

## Interesting ideas
- Experts can correct a probabilistic classifier only where it is unsure: an override fires only when the top two labels are within a margin. This keeps rules from damaging confident decisions and makes each correction traceable.
- Small, interpretable cue features carry much of the cross-domain signal and survive domain shift better than learned embeddings. Examples include a WH-word at utterance start, question collocations like "can I", and thanks, apology or greeting lexicons.
- DA labeling is treated as sequence labeling over the whole conversation, so the act of the previous turn constrains the next one (a yes/no question predicts an answer).
- Reporting a full train-corpus × test-corpus matrix exposes domain fragility that same-domain scores hide.

## For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

- **I1** (frames): speech-act decisions in prag could apply ordered "prefer X over Y when close" rules to ties such as statement versus question, logging which rule fired.
- **I2** (frames): speech-act detection can key on tree positions (fronted wh-word, subject–aux inversion) plus small lexicons rather than bag-of-words.
- **I3** (frames): prag could score speech acts jointly across turns in dialogues within fables and tales, using adjacency-pair expectations.
- **I4** (learn): evaluations should routinely include out-of-domain test sets, such as school texts versus tales, as a control against overfitting to one genre.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Indian Institute of Technology Patna](https://md.research.elamur.ai/free/en/universities/indian-institute-of-technology-patna/) — University, Patna, India · 10 works · 262 citations
- [Tata Consultancy Services](https://md.research.elamur.ai/free/en/companies/tata-consultancy-services/) — Company, Pune, India · 3 works · 32 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 9,086 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 8,889 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 9,296 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 9,206 works

🧭 Topics (2): [🗨️ Dialogue systems](https://md.research.elamur.ai/free/en/topics/dialogue-systems/) · 401 works · [💬 Speech act theory](https://md.research.elamur.ai/free/en/topics/speech-act-theory/) · 160 works

## Cite

Cite as: Swapnil Hingmire, Apoorv Shrivastava, Girish Palshikar and Saurabh Srivastava. 2017. Experiments with Domain Dependent Dialogue Act Classification using Open-Domain Dialogue Corpora. Proceedings of the 14th International Conference on Natural Language Processing (ICON-2017).

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{hingmire2017experiments,
  title = {{Experiments with Domain Dependent Dialogue Act Classification using Open-Domain Dialogue Corpora}},
  author = {Swapnil Hingmire, Apoorv Shrivastava, Girish Palshikar and Saurabh Srivastava},
  booktitle = {Proceedings of the 14th International Conference on Natural Language Processing (ICON-2017)},
  year = {2017},
  url = {https://aclanthology.org/W17-7538/}
}
```

- [← previous Evaluative Features of the Image "Friend" in Phrasal Units of the English Language](https://md.research.elamur.ai/free/en/papers/lyzlov-2017-evaluative-features-image-friend-phrasal/)
- [next → Fronting and exhaustive exclusion in Biblical Hebrew](https://md.research.elamur.ai/free/en/papers/merwe-2018-fronting-exhaustive-exclusion-biblical-hebrew/)
- [↑ back to papers of 2017](https://md.research.elamur.ai/free/en/papers/2017/page/5/)

## References

22 references, 8 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. S. Bangalore, G. Di Fabbrizio, and A. Stent. 2006. Learning the structure of task-driven human-human dialogs. In Proc. 21st COLING, and 44th ACL, pages 201–208. [find](https://scholar.google.com/scholar?q=S%2E+Bangalore%2C+G%2E+Di+Fabbrizio%2C+and+A%2E+Stent%2E+2006%2E+Learning+the+structure+of+task%2Ddriven+human%2Dhuman+dialogs%2E+In+Proc%2E+21st+COLING%2C+and+44th+ACL%2C+pages+201%E2%80%93208%2E)
2. Sumit Bhatia, Prakhar Biyani, and Prasenjit Mitra. 2014. [Summarizing online forum discussions – can dialog acts of individual messages help?](https://md.research.elamur.ai/free/en/papers/bhatia-2014-summarizing-online-forum-discussions-can/) In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 2127–2131, October.
3. Kristy Boyer, Joseph Grafsgaard, Eun Young Ha, Robert Phillips, and James Lester. 2011. [An affect-enriched dialogue act classification model for task-oriented dialogue](https://md.research.elamur.ai/free/en/papers/boyer-2011-affect-enriched-dialogue-act-classification/). In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, pages 1190–1199, June.
4. Tamitha Carpenter and Emi Fujioka. 2011. The role and identification of dialog acts in online chat. In Proc. Workshop on Analyzing Microtext at the 25th AAAI, Conference on Artificial Intelligence. [find](https://scholar.google.com/scholar?q=Tamitha+Carpenter+and+Emi+Fujioka%2E+2011%2E+The+role+and+identification+of+dialog+acts+in+online+chat%2E+In+Proc%2E+Workshop+on+Analyzing+Microtext+at+the+25th+AAAI%2C+Conference+on+Artificial+Intelligence%2E)
5. W.W. Cohen, V.R. Carvalho, and T.M. Mitchell. 2004. [Learning to classify email into ”speech acts”](https://md.research.elamur.ai/free/en/papers/cohen-2004-learning-classify-email-into-speech/). In Proc. Empirical Methods in Natural Language Processing (EMNLP-2004), pages 309–316.
6. Oliver Ferschke, Iryna Gurevych, and Yevgen Chebotar. 2012. Behind the article: Recognizing dialog 310 acts in wikipedia talk pages. In Proceedings of the 13th Conference of the European Chapter of the Association for Computational Linguistics, pages 777– 786, April. [find](https://scholar.google.com/scholar?q=Oliver+Ferschke%2C+Iryna+Gurevych%2C+and+Yevgen+Chebotar%2E+2012%2E+Behind+the+article%3A+Recognizing+dialog+310+acts+in+wikipedia+talk+pages%2E+In+Proceedings+of+the+13th+Conference+of+the+European+Chapter+of+th)
7. John J. Godfrey, Edward C. Holliman, and Jane Mc-Daniel. 1992. Switchboard: Telephone speech corpus for research and development. In Proceedings of the 1992 IEEE International Conference on Acoustics, Speech and Signal Processing - Volume 1, ICASSP’92, pages 517–520, Washington, DC, USA. IEEE Computer Society. [find](https://scholar.google.com/scholar?q=John+J%2E+Godfrey%2C+Edward+C%2E+Holliman%2C+and+Jane+Mc%2DDaniel%2E+1992%2E+Switchboard%3A+Telephone+speech+corpus+for+research+and+development%2E+In+Proceedings+of+the+1992+IEEE+International+Conference+on+Acoustics%2C)
8. D. Jurafsky, R. Bates, N. Coccaro, R. Martin, M. Meteer, K. Ries, E. Shriberg, A. Stolcke, P. Taylor, and C. Van Ess-Dykema. 1997. Automatic detection of discourse structure for speech recognition and understanding. In 1997 IEEE Workshop on Automatic Speech Recognition and Understanding Proceedings, pages 88–95, December. [↗](https://doi.org/10.5445/ir/1000166547)
9. Simon Keizer. 2001. A bayesian approach to dialogue act classification. In Proc. BI-DIALOG. [find](https://scholar.google.com/scholar?q=Simon+Keizer%2E+2001%2E+A+bayesian+approach+to+dialogue+act+classification%2E+In+Proc%2E+BI%2DDIALOG%2E)
10. Hamed Khanpour, Nishitha Guntakandla, and Rodney Nielsen. 2016. [Dialogue act classification in domain-independent conversations using a deep recurrent neural network](https://md.research.elamur.ai/free/en/papers/khanpour-2016-dialogue-act-classification-domain-independent/). In Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers, pages 2012– 2021, Osaka, Japan, December.
11. The COLING 2016 Organizing Committee. [find](https://scholar.google.com/scholar?q=The+COLING+2016+Organizing+Committee%2E)
12. Jihie Kim, Jia Li, and Taehwan Kim. 2010a. Towards identifying unresolved discussions in student online forums. In Proc. [find](https://scholar.google.com/scholar?q=Jihie+Kim%2C+Jia+Li%2C+and+Taehwan+Kim%2E+2010a%2E+Towards+identifying+unresolved+discussions+in+student+online+forums%2E+In+Proc%2E)
13. NAACL HLT, 2010 Fifth Workshop on Innovative Use of NLP, for Building Educational Applications, pages 84–91. [find](https://scholar.google.com/scholar?q=NAACL+HLT%2C+2010+Fifth+Workshop+on+Innovative+Use+of+NLP%2C+for+Building+Educational+Applications%2C+pages+84%E2%80%9391%2E)
14. Su Nam Kim, Lawrence Cavedon, and Timothy Baldwin. 2010b. [Classifying dialogue acts in one-on-one live chats](https://md.research.elamur.ai/free/en/papers/kim-2010-classifying-dialogue-acts-one-one/). In Proceedings of the 2010 Conference on Empirical Methods in Natural Language Processing, EMNLP ’10, pages 862–871, Stroudsburg, PA, USA. Association for Computational Linguistics.
15. Tao Lei, Regina Barzilay, and Tommi Jaakkola. 2015a. Molding cnns for text: non-linear, non-consecutive convolutions. arXiv preprint arXiv:1508.04112. Tao Lei, Hrishikesh Joshi, Regina Barzilay, Tommi S. Jaakkola, Kateryna Tymoshenko, Alessandro Moschitti, and Lluı́s Màrquez i Villodre. 2015b. Denoising bodies to titles: Retrieving similar questions with recurrent convolutional models. CoRR, abs/1512.05726. [↗](https://arxiv.org/abs/1508.04112)
16. Wei Li and Yunfang Wu. 2016. [Multi-level gated recurrent neural network for dialog act classification](https://md.research.elamur.ai/free/en/papers/li-2016-multi-level-gated-recurrent-neural/). In Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers, pages 1970–1979, Osaka, Japan, December.
17. The COLING 2016 Organizing Committee. [find](https://scholar.google.com/scholar?q=The+COLING+2016+Organizing+Committee%2E)
18. D. J. Litman and S. Silliman. 2004. ITSPOKE: An intelligent tutoring spoken dialogue system. In Proc. HLT/NAACL. [find](https://scholar.google.com/scholar?q=D%2E+J%2E+Litman+and+S%2E+Silliman%2E+2004%2E+ITSPOKE%3A+An+intelligent+tutoring+spoken+dialogue+system%2E+In+Proc%2E+HLT%2FNAACL%2E)
19. Jeffrey Pennington, Richard Socher, and Christopher D. Manning. 2014. [Glove: Global vectors for word representation](https://md.research.elamur.ai/free/en/papers/pennington-2014-glove-global-vectors-word-representation/). In Empirical Methods in Natural Language Processing (EMNLP), pages 1532– 1543. [↗](https://doi.org/10.3115/v1/d14-1162)
20. Norbert Reithinger and Martin Klesen. 1997. Dialog act classification using language models. In Proc. EuroSpeech-97, pages 2235–2238. [find](https://scholar.google.com/scholar?q=Norbert+Reithinger+and+Martin+Klesen%2E+1997%2E+Dialog+act+classification+using+language+models%2E+In+Proc%2E+EuroSpeech%2D97%2C+pages+2235%E2%80%932238%2E)
21. Andreas Stolcke, Klaus Ries, Noah Coccaro, Elizabeth Shriberg, Rebecca Bates, Daniel Jurafsky, Paul Taylor, Rachel Martin, Carol Van Ess-Dykema, and Marie Meteer. 2000. [Dialogue act modeling for automatic tagging and recognition of conversational speech](https://md.research.elamur.ai/free/en/papers/stolcke-2000-dialogue-act-modeling-automatic-tagging/). Computational Linguistics, 26(3):339–373. [↗](https://doi.org/10.1162/089120100561737)
22. Helen Wright. 1998. Automatic utterance type detection using suprasegmental features. In Proceedings of the International Conference on Spoken Language Processing 1998, page 1403. [find](https://scholar.google.com/scholar?q=Helen+Wright%2E+1998%2E+Automatic+utterance+type+detection+using+suprasegmental+features%2E+In+Proceedings+of+the+International+Conference+on+Spoken+Language+Processing+1998%2C+page+1403%2E)

*Updated 24 October 2026*