---
title: "Dialogue Act Recognition via CRF-Attentive Structured Network"
description: "The authors propose CRF-ASN, a fully differentiable model built in two stages. 39 references, cited by 40 papers here."
authors: ["Zheqian Chen", "Rongqin Yang", "Zhou Zhao", "Deng Cai", "Xiaofei He"]
year: 2017
venue: "arXiv:1711.05568"
source: "https://arxiv.org/abs/1711.05568"
doi: "10.48550/arxiv.1711.05568"
arxiv: "1711.05568"
licence: "arXiv non-exclusive distribution licence (read only, no redistribution)"
licence_url: "http://arxiv.org/licenses/nonexclusive-distrib/1.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/chen-2017-dialogue-act-recognition-crf-attentive/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/chen-2017-dialogue-act-recognition-crf-attentive/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2017](https://md.research.elamur.ai/free/en/papers/2017/page/3/) / Dialogue Act Recognition via CRF-Attentive Structured Network

# Dialogue Act Recognition via CRF-Attentive Structured Network

**Authors:** Zheqian Chen, Rongqin Yang, [Zhou Zhao](https://md.research.elamur.ai/free/en/authors/zhou-zhao/), [Deng Cai](https://md.research.elamur.ai/free/en/authors/deng-cai/) and Xiaofei He · **Year:** 2017 · **Venue:** arXiv:1711.05568 · **Licence:** [arXiv non-exclusive distribution licence (read only, no redistribution)](http://arxiv.org/licenses/nonexclusive-distrib/1.0/)

**Links:** [Publisher's page](https://arxiv.org/abs/1711.05568) · [PDF](https://arxiv.org/pdf/1711.05568) · [DOI 10.48550/arxiv.1711.05568](https://doi.org/10.48550/arxiv.1711.05568) · [arXiv 1711.05568](https://arxiv.org/abs/1711.05568) · [OpenAlex](https://openalex.org/W2963550483)

**Citations:** 93 citations (OpenAlex, 2026-10-14) · [40](https://md.research.elamur.ai/free/en/papers/chen-2017-dialogue-act-recognition-crf-attentive/cited-by/) in our library, of the 141,840 papers whose bibliographies we read

## Summary
The paper tackles Dialogue Act Recognition (DAR), which means tagging each utterance in a conversation with a label such as Question, Answer, Greeting or Backchannel that reflects the speaker's intent. Earlier systems had two weaknesses. Some relied on handcrafted features and classified each utterance in isolation with SVM or Naive Bayes. Others treated the whole dialogue as one long flat sequence, which blurs the difference between links among utterances and links among act labels. The authors propose CRF-ASN, a fully differentiable model built in two stages. First, a hierarchical encoder works at the word, utterance and conversation levels. It combines character- and word-level embeddings, a BiGRU, and a memory-network-style component that lets each utterance draw on the whole dialogue, not only nearby context. Second, a linear-chain CRF output layer is extended with structured attention, so dependencies between neighbouring acts are weighted softly according to context. On SWDA and MRDA the model outperforms earlier state-of-the-art systems. On SWDA it comes within roughly 2% of human annotator performance. This suggests that explicit label structure still helps inside end-to-end neural models.

## Key points
- DAR is framed as sequence labeling. A conversation is a sequence of utterances, each aligned with one act label, and each utterance is a sequence of words.
- Evaluation uses two benchmark corpora: SWDA (the Switchboard Dialogue Act corpus) and MRDA (the Meeting Recorder Dialogue Act corpus). It is compared with deep baselines including DRLM-Conditional, LSTM-Softmax and RCNN.
- The reported results beat prior state of the art on both corpora, with a gap below 2% to human annotators on SWDA.
- The utterance encoder has two parts:
  - an embedding layer that combines character-level and word-level inputs;
  - a BiGRU plus input and output memories, with attention over the previous and following utterances (one memory hop in the illustrated setup).
- The structured attention builds on Kim et al.'s structured attention networks. It is repurposed from segmentation-style selection to modeling how dialogue acts influence one another along a linear-chain CRF.

## Interesting ideas
- The model keeps two kinds of dependency apart: what the utterance text says, and which act tends to follow which (e.g. Question → Answer, Greeting → Greeting). Mixing them into one flat stream loses structure.
- Instead of feeding a dialogue as one huge word chain, it is encoded in layers (words → utterance → conversation). This avoids losing signal over long distances and mirrors how discourse is actually organised.
- An utterance is interpreted using both the preceding and the following turn, because a reply often reveals what the earlier turn was doing.
- Results are judged against human annotator agreement as a practical ceiling, not just against other systems, which shows how much room is really left.

## For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

- **I1** (frames): the speech-act component in prag could combine per-sentence evidence with an explicit, readable table of act-to-act transitions, so every act decision can be explained as "content says X, previous act makes Y likely".
- **I2** (world): the world of a text could be built sentence by sentence and then aggregated per paragraph or text, rather than tracking state over a raw token stream.
- **I3** (frames): implicature and irony detection could look ahead to the next sentence or reply before fixing the reading of a turn.
- **I4** (learn): evaluation could report agreement between witness parsers or teacher tiers as the ceiling, so a gain is read relative to that upper bound.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Alibaba Group](https://md.research.elamur.ai/free/en/companies/alibaba-group/) — Company, Hangzhou, China · 64 works · 14,113 citations
- [Zhejiang University](https://md.research.elamur.ai/free/en/universities/zhejiang-university/) — University, Hangzhou, China · 46 works · 11,542 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 9,086 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 8,889 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 9,296 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 9,206 works

🧭 Topics (2): [🗨️ Dialogue systems](https://md.research.elamur.ai/free/en/topics/dialogue-systems/) · 401 works · [💬 Speech act theory](https://md.research.elamur.ai/free/en/topics/speech-act-theory/) · 160 works

## Cite

Cite as: Zheqian Chen, Rongqin Yang, Zhou Zhao, Deng Cai and Xiaofei He. 2017. Dialogue Act Recognition via CRF-Attentive Structured Network. arXiv:1711.05568.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@misc{chen2017dialogue,
  title = {{Dialogue Act Recognition via CRF-Attentive Structured Network}},
  author = {Zheqian Chen, Rongqin Yang, Zhou Zhao, Deng Cai and Xiaofei He},
  year = {2017},
  doi = {10.48550/arxiv.1711.05568},
  eprint = {1711.05568},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.1711.05568}
}
```

- [← previous The Parallel Meaning Bank: Towards a Multilingual Corpus of Translations Annotated with Compositional Meaning Representations](https://md.research.elamur.ai/free/en/papers/abzianidze-2017-parallel-meaning-bank-multilingual-corpus/)
- [next → Answering Complex Questions Using Open Information Extraction](https://md.research.elamur.ai/free/en/papers/khot-2017-answering-complex-questions-open-information/)
- [↑ back to papers of 2017](https://md.research.elamur.ai/free/en/papers/2017/page/3/)

## References

39 references, 18 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Miltiadis Allamanis, Hao Peng, and Charles Sutton. 2016. A convolutional attention network for extreme summarization of source code. In International Conference on Machine Learning. 2091–2100. [↗](https://doi.org/10.48550/arxiv.1602.03001)
2. Jeremy Ang, Yang Liu, and Elizabeth Shriberg. 2005. Automatic Dialog Act Segmentation and Classification in Multiparty Meetings. In ICASSP. [find](https://scholar.google.com/scholar?q=Jeremy+Ang%2C+Yang+Liu%2C+and+Elizabeth+Shriberg%2E+2005%2E+Automatic+Dialog+Act+Segmentation+and+Classification+in+Multiparty+Meetings%2E+In+ICASSP%2E)
3. Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. 2014. [Neural Machine Translation by Jointly Learning to Align and Translate](https://md.research.elamur.ai/free/en/papers/bahdanau-2014-neural-machine-translation-jointly-learning/). Computer Science (2014). [↗](https://doi.org/10.48550/arxiv.1409.0473)
4. Phil Blunsom and Nal Kalchbrenner. 2013. [Recurrent Convolutional Neural Networks for Discourse Compositionality](https://md.research.elamur.ai/free/en/papers/kalchbrenner-2013-recurrent-convolutional-neural-networks-discourse/). In Proceedings of the 2013 Workshop on Continuous Vector Space Models and their Compositionality. Proceedings of the 2013 Workshop on Continuous Vector Space Models and their Compositionality.
5. Kristy Elizabeth Boyer, Eunyoung Ha, Michael D Wallis, Robert Phillips, Mladen A Vouk, and James C Lester. 2009. Discovering Tutorial Dialogue Strategies with Hidden Markov Models.. In AIED. 141–148. [↗](https://doi.org/10.3233/978-1-60750-028-5-141)
6. Yun-Nung Chen, William Yang Wang, and Alexander I Rudnicky. 2013. An empirical investigation of sparse log-linear models for improved dialogue act classification. In Acoustics, Speech and Signal Processing (ICASSP), 2013 IEEE International Conference on. IEEE, 8317–8321. [find](https://scholar.google.com/scholar?q=Yun%2DNung+Chen%2C+William+Yang+Wang%2C+and+Alexander+I+Rudnicky%2E+2013%2E+An+empirical+investigation+of+sparse+log%2Dlinear+models+for+improved+dialogue+act+classification%2E+In+Acoustics%2C+Speech+and+Signal+Proce)
7. Kyunghyun Cho, Bart van Merriënboer, Dzmitry Bahdanau, and Yoshua Bengio. 2014. [On the Properties of Neural Machine Translation: Encoder–Decoder Approaches](https://md.research.elamur.ai/free/en/papers/cho-2014-properties-neural-machine-translation-encoder/). Syntax, Semantics and Structure in Statistical Translation (2014), 103.
8. Orhan Firat, Kyunghyun Cho, and Yoshua Bengio. 2016. [Multi-Way, Multilingual Neural Machine Translation with a Shared Attention Mechanism](https://md.research.elamur.ai/free/en/papers/firat-2016-multi-way-multilingual-neural-machine/). In Proceedings of NAACL-HLT. 866–875.
9. Michel Galley. 2006. A skip-chain conditional random field for ranking meeting utterances by importance. In Proceedings of the 2006 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, 364–372. [find](https://scholar.google.com/scholar?q=Michel+Galley%2E+2006%2E+A+skip%2Dchain+conditional+random+field+for+ranking+meeting+utterances+by+importance%2E+In+Proceedings+of+the+2006+Conference+on+Empirical+Methods+in+Natural+Language+Processing%2E+Asso)
10. Jeroen Geertzen, Volha Petukhova, and Harry Bunt. 2007. [A multidimensional approach to utterance segmentation and dialogue act classification](https://md.research.elamur.ai/free/en/papers/geertzen-2007-multidimensional-approach-utterance-segmentation-dialogue/). In Proceedings of the 8th SIGdial Workshop on Discourse and Dialogue, Antwerp. 140–149.
11. Sergio Grau, Emilio Sanchis, Maria Jose Castro, and David Vilar. 2004. Dialogue act classification using a Bayesian approach. In 9th Conference Speech and Computer. [find](https://scholar.google.com/scholar?q=Sergio+Grau%2C+Emilio+Sanchis%2C+Maria+Jose+Castro%2C+and+David+Vilar%2E+2004%2E+Dialogue+act+classification+using+a+Bayesian+approach%2E+In+9th+Conference+Speech+and+Computer%2E)
12. Karl Moritz Hermann, Tomas Kocisky, Edward Grefenstette, Lasse Espeholt, Will Kay, Mustafa Suleyman, and Phil Blunsom. 2015. [Teaching machines to read and comprehend](https://md.research.elamur.ai/free/en/papers/hermann-2015-teaching-machines-read-comprehend/). In Advances in Neural Information Processing Systems. 1693–1701. [↗](https://doi.org/10.48550/arxiv.1506.03340)
13. Ryuichiro Higashinaka, Kenji Imamura, Toyomi Meguro, Chiaki Miyazaki, Nozomi Kobayashi, Hiroaki Sugiyama, Toru Hirano, Toshiro Makino, and Yoshihiro Matsuo. 2014. Towards an open-domain conversational system fully based on natural language processing.. In COLING. 928–939. [find](https://scholar.google.com/scholar?q=Ryuichiro+Higashinaka%2C+Kenji+Imamura%2C+Toyomi+Meguro%2C+Chiaki+Miyazaki%2C+Nozomi+Kobayashi%2C+Hiroaki+Sugiyama%2C+Toru+Hirano%2C+Toshiro+Makino%2C+and+Yoshihiro+Matsuo%2E+2014%2E+Towards+an+open%2Ddomain+conversational)
14. Sepp Hochreiter and Jürgen Schmidhuber. 1997. Long short-term memory. Neural computation 9, 8 (1997), 1735–1780. [↗](https://doi.org/10.1162/neco.1997.9.8.1735)
15. Yangfeng Ji, Gholamreza Haffari, and Jacob Eisenstein. 2016. A Latent Variable Recurrent Neural Network for Discourse-Driven Language Models. In HLT-NAACL. [find](https://scholar.google.com/scholar?q=Yangfeng+Ji%2C+Gholamreza+Haffari%2C+and+Jacob+Eisenstein%2E+2016%2E+A+Latent+Variable+Recurrent+Neural+Network+for+Discourse%2DDriven+Language+Models%2E+In+HLT%2DNAACL%2E)
16. Yangfeng Ji, Gholamreza Haffari, and Jacob Eisenstein. 2016. A Latent Variable Recurrent Neural Network for Discourse Relation Language Models. In Proceedings of NAACL-HLT. 332–342. [↗](https://doi.org/10.48550/arxiv.1603.01913)
17. Nal Kalchbrenner and Phil Blunsom. 2013. [Recurrent Convolutional Neural Networks for Discourse Compositionality](https://md.research.elamur.ai/free/en/papers/kalchbrenner-2013-recurrent-convolutional-neural-networks-discourse/). ACL 2013 (2013), 119.
18. Hamed Khanpour, Nishitha Guntakandla, and Rodney Nielsen. 2016. [Dialogue Act Classification in Domain-Independent Conversations Using a Deep Recurrent Neural Network](https://md.research.elamur.ai/free/en/papers/khanpour-2016-dialogue-act-classification-domain-independent/). In COLING.
19. Yoon Kim, Yacine Jernite, David Sontag, and Alexander M Rush. 2016. [Character-Aware Neural Language Models](https://md.research.elamur.ai/free/en/papers/kim-2015-character-aware-neural-language-models/).. In AAAI. 2741–2749.
20. Pavel Král and Christophe Cerisara. 2012. Dialogue act recognition approaches. Computing and Informatics 29, 2 (2012), 227–250. [find](https://scholar.google.com/scholar?q=Pavel+Kr%C3%A1l+and+Christophe+Cerisara%2E+2012%2E+Dialogue+act+recognition+approaches%2E+Computing+and+Informatics+29%2C+2+%282012%29%2C+227%E2%80%93250%2E)
21. Harshit Kumar, Arvind Agarwal, Riddhiman Dasgupta, Sachindra Joshi, and Arun Kumar. 2017. [Dialogue Act Sequence Labeling using Hierarchical encoder with CRF](https://md.research.elamur.ai/free/en/papers/kumar-2017-dialogue-act-sequence-labeling-using/). (2017). [↗](https://doi.org/10.1609/aaai.v32i1.11701)
22. Harshit Kumar, Arvind Agarwal, Riddhiman Dasgupta, Sachindra Joshi, and Arun Kumar. 2017. [Dialogue Act Sequence Labeling using Hierarchical encoder with CRF](https://md.research.elamur.ai/free/en/papers/kumar-2017-dialogue-act-sequence-labeling-using/). CoRR abs/1709.04250 (2017). [↗](https://arxiv.org/abs/1709.04250)
23. Ji Young Lee and Franck Dernoncourt. 2016. [Sequential Short-Text Classification with Recurrent and Convolutional Neural Networks](https://md.research.elamur.ai/free/en/papers/lee-2016-sequential-short-text-classification-recurrent/). In Proceedings of NAACL-HLT. 515–520.
24. Piroska Lendvai and Jeroen Geertzen. 2007. [Token-based chunking of turn-internal dialogue act sequences](https://md.research.elamur.ai/free/en/papers/lendvai-2007-token-based-chunking-turn-internal/). In Proceedings of the 8th SIGDIAL Workshop on Discourse and Dialogue. 174–181.
25. Edward Loper and Steven Bird. 2002. NLTK: The natural language toolkit. In Proceedings of the ACL-02 Workshop on Effective tools and methodologies for teaching natural language processing and computational linguistics-Volume 1. Association for Computational Linguistics, 63–70. [↗](https://doi.org/10.48550/arxiv.cs/0205028)
26. James M Lucas and Michael S Saccucci. 1990. Exponentially weighted moving average control schemes: properties and enhancements. Technometrics 32, 1 (1990), 1–12. [↗](https://doi.org/10.1080/00401706.1990.10484583)
27. Dmitrijs Milajevs and Matthew Purver. 2014. [Investigating the contribution of distributional semantic information for dialogue act classification](https://md.research.elamur.ai/free/en/papers/milajevs-2014-investigating-contribution-distributional-semantic-information/). In Proceedings of the 2nd Workshop on Continuous Vector Space Models and their Compositionality (CVSC). 40–47.
28. Jeffrey Pennington, Richard Socher, and Christopher Manning. 2014. [Glove: Global vectors for word representation](https://md.research.elamur.ai/free/en/papers/pennington-2014-glove-global-vectors-word-representation/). In Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP). 1532–1543.
29. Norbert Reithinger and Martin Klesen. 1997. Dialogue act classification using language models.. In EuroSpeech. [find](https://scholar.google.com/scholar?q=Norbert+Reithinger+and+Martin+Klesen%2E+1997%2E+Dialogue+act+classification+using+language+models%2E%2E+In+EuroSpeech%2E)
30. Riccardo Serafin, Barbara Di Eugenio, and Michael Glass. 2003. Latent Semantic Analysis for dialogue act classification. In Proceedings of the 2003 Conference of the North American Chapter of the Association for Computational Linguistics on Human Language Technology: companion volume of the Proceedings of HLT-NAACL 2003–short papers-Volume 2. Association for Computational Linguistics, 94–96. [find](https://scholar.google.com/scholar?q=Riccardo+Serafin%2C+Barbara+Di+Eugenio%2C+and+Michael+Glass%2E+2003%2E+Latent+Semantic+Analysis+for+dialogue+act+classification%2E+In+Proceedings+of+the+2003+Conference+of+the+North+American+Chapter+of+the+Asso)
31. Nitish Srivastava, Geoffrey E Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. 2014. Dropout: a simple way to prevent neural networks from overfitting. Journal of machine learning research 15, 1 (2014), 1929–1958. [find](https://scholar.google.com/scholar?q=Nitish+Srivastava%2C+Geoffrey+E+Hinton%2C+Alex+Krizhevsky%2C+Ilya+Sutskever%2C+and+Ruslan+Salakhutdinov%2E+2014%2E+Dropout%3A+a+simple+way+to+prevent+neural+networks+from+overfitting%2E+Journal+of+machine+learning+re)
32. Andreas Stolcke, Klaus Ries, Noah Coccaro, Elizabeth Shriberg, Rebecca Bates, Daniel Jurafsky, Paul Taylor, Rachel Martin, Carol Van Ess-Dykema, and Marie Meteer. 2006. Dialogue act modeling for automatic tagging and recognition of conversational speech. Dialogue 26, 3 (2006). [find](https://scholar.google.com/scholar?q=Andreas+Stolcke%2C+Klaus+Ries%2C+Noah+Coccaro%2C+Elizabeth+Shriberg%2C+Rebecca+Bates%2C+Daniel+Jurafsky%2C+Paul+Taylor%2C+Rachel+Martin%2C+Carol+Van+Ess%2DDykema%2C+and+Marie+Meteer%2E+2006%2E+Dialogue+act+modeling+for+autom)
33. Dinoj Surendran and Gina-Anne Levow. 2006. Dialog act tagging with support vector machines and hidden Markov models.. In Interspeech. [find](https://scholar.google.com/scholar?q=Dinoj+Surendran+and+Gina%2DAnne+Levow%2E+2006%2E+Dialog+act+tagging+with+support+vector+machines+and+hidden+Markov+models%2E%2E+In+Interspeech%2E)
34. Linlin Wang, Zhu Cao, Gerard de Melo, and Zhiyuan Liu. 2016. [Relation Classification via Multi-Level Attention CNNs](https://md.research.elamur.ai/free/en/papers/wang-2016-relation-classification-multi-level-attention/).. In ACL (1).
35. Nick Webb, Mark Hepple, and Yorick Wilks. 2005. Dialogue act classification based on intra-utterance features. In Proceedings of the AAAI Workshop on Spoken Language Understanding, Vol. 4. 5. [find](https://scholar.google.com/scholar?q=Nick+Webb%2C+Mark+Hepple%2C+and+Yorick+Wilks%2E+2005%2E+Dialogue+act+classification+based+on+intra%2Dutterance+features%2E+In+Proceedings+of+the+AAAI+Workshop+on+Spoken+Language+Understanding%2C+Vol%2E+4%2E+5%2E)
36. Zichao Yang, Diyi Yang, Chris Dyer, Xiaodong He, Alexander J Smola, and Eduard H Hovy. 2016. [Hierarchical Attention Networks for Document Classification](https://md.research.elamur.ai/free/en/papers/yang-2016-hierarchical-attention-networks-document-classification/).. In HLT-NAACL. 1480–1489.
37. Peng Zhou, Wei Shi, Jun Tian, Zhenyu Qi, Bingchen Li, Hongwei Hao, and Bo Xu. 2016. [Attention-based bidirectional long short-term memory networks for relation classification](https://md.research.elamur.ai/free/en/papers/zhou-2016-attention-based-bidirectional-long-short/). In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), Vol. 2. 207–212.
38. Yucan Zhou, Qinghua Hu, Jie Liu, and Yuan Jia. 2015. Combining heterogeneous deep neural networks with conditional random fields for Chinese dialogue act recognition. Neurocomputing 168 (2015), 408–417. [↗](https://doi.org/10.1016/j.neucom.2015.05.086)
39. Matthias Zimmermann. 2009. Joint segmentation and classification of dialog acts using conditional random fields. In Tenth Annual Conference of the International Speech Communication Association. [find](https://scholar.google.com/scholar?q=Matthias+Zimmermann%2E+2009%2E+Joint+segmentation+and+classification+of+dialog+acts+using+conditional+random+fields%2E+In+Tenth+Annual+Conference+of+the+International+Speech+Communication+Association%2E)

*Updated 24 October 2026*