---
title: "Guiding attention in Sequence-to-sequence models for Dialogue"
description: "Colombo and colleagues address dialogue act (DA) classification, which… 85.0% accuracy on SwDA. 30 references, cited by 27 papers here."
authors: ["Pierre Colombo", "Emile Chapuis", "Matteo Manica", "Emmanuel Vignon", "Giovanna Varni", "Chloe Clavel"]
year: 2020
venue: "AAAI 2020"
source: "https://arxiv.org/abs/2002.08801"
doi: "10.48550/arxiv.2002.08801"
arxiv: "2002.08801"
licence: "arXiv non-exclusive distribution licence (read only, no redistribution)"
licence_url: "http://arxiv.org/licenses/nonexclusive-distrib/1.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/colombo-2020-guiding-attention-sequence-sequence-models/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/colombo-2020-guiding-attention-sequence-sequence-models/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2020](https://md.research.elamur.ai/free/en/papers/2020/page/6/) / Guiding attention in Sequence-to-sequence models for Dialogue Act prediction

# Guiding attention in Sequence-to-sequence models for Dialogue Act prediction

**Authors:** [Pierre Colombo](https://md.research.elamur.ai/free/en/authors/pierre-colombo/), Emile Chapuis, Matteo Manica, Emmanuel Vignon, Giovanna Varni and [Chloe Clavel](https://md.research.elamur.ai/free/en/authors/chloe-clavel/) · **Year:** 2020 · **Venue:** AAAI 2020 · **Licence:** [arXiv non-exclusive distribution licence (read only, no redistribution)](http://arxiv.org/licenses/nonexclusive-distrib/1.0/)

**Links:** [Publisher's page](https://arxiv.org/abs/2002.08801) · [PDF](https://arxiv.org/pdf/2002.08801) · [DOI 10.48550/arxiv.2002.08801](https://doi.org/10.48550/arxiv.2002.08801) · [arXiv 2002.08801](https://arxiv.org/abs/2002.08801) · [OpenAlex](https://openalex.org/W3008671257)

**Citations:** 7 citations (OpenAlex, 2026-10-14) · [27](https://md.research.elamur.ai/free/en/papers/colombo-2020-guiding-attention-sequence-sequence-models/cited-by/) in our library, of the 141,840 papers whose bibliographies we read

## Summary
Colombo and colleagues address dialogue act (DA) classification, which assigns each utterance a label for the speaker's intention, such as question, backchannel or opinion. Earlier top systems pair a recurrent encoder with a linear CRF. A CRF links only neighbouring labels, so it misses cases where an answer refers back to a question several turns earlier. The authors instead treat the task like neural machine translation. A window of utterances is the source sequence, and the matching tag sequence is the target. Their seq2seq model has three parts:
- a hierarchical GRU encoder, with one level over words and one over utterances;
- a new "guided attention" in the decoder, which uses the fact that each utterance has exactly one tag;
- beam search, used in inference and also in fine-tuning with a sequence-level loss.

The model is trained end-to-end without hand-engineered features. It reaches 85.0% accuracy on SwDA, about two points above the earlier best of 82.9%. On MRDA it scores 91.6%, which is close to the best reported systems. The work shows that NMT decoding tools carry over to structured labelling tasks with a small tag set.

## Key points
- Benchmarks: SwDA (42 tags) and MRDA (5 tags). Test accuracy was 85.0% on SwDA and 91.6% on MRDA. On MRDA, Li et al. (2018a) report 92.2% and Chen et al. 91.7%.
- Three encoders were compared: a vanilla averaged-embedding BiGRU, the hierarchical HGRU and a speaker-aware PersoHGRU. HGRU won in most settings. The persona layer brought no reliable gain and lowered accuracy on MRDA.
- Hard guided attention gave the best overall results on both corpora and adds no parameters. On SwDA, Wilcoxon tests over 10 seeds gave p \< 0.01 against the other decoders.
- Beam search used only at inference added at most 0.2%. Fine-tuning with a beam-based sequence-level loss added about 1 point on SwDA (84→85) and 1.2 points on MRDA (90.4→91.6).
- Larger beams made the decoder conservative. It drifted towards frequent labels such as sd, or towards runs of one tag like "s s s s s".

## Interesting ideas
- Model the label sequence with its own autoregressive decoder rather than a first-order CRF, so that a tag can depend on tags several steps back (for example, an answer that follows a backchannel and points back to an earlier question).
- When input and output units are aligned one-to-one, attention can be fixed to the aligned position (the hard version) instead of being learned freely, which costs no parameters and keeps attention interpretable.
- With a tiny output vocabulary, plain beam search at inference barely helps, but using the beam inside a sequence-level training loss gives a real gain.
- In imbalanced, small label spaces, wider beams collapse onto majority-class sequences, which is a failure mode worth testing for explicitly.
- Report weak or harmful additions (here the speaker-persona layer) and back decoder choices with Wilcoxon tests over many seeds.

## For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

- **I1** (frames): the speech-act layer in prag could score acts over a whole dialogue window, so that answer/question pairing survives intervening backchannels; the reason for each act would be shown as an attended earlier turn.
- **I2** (tokens): per-token taggers in crate en that use contextual decoders could fix attention to the aligned token, making each tag decision traceable to one input position.
- **I3** (learn): when training small-label-set components (speech acts, roles), a sequence-level fine-tuning stage may pay off more than larger inference beams.
- **I4** (learn): add a negative control that measures how much a decoder over-predicts the dominant tag as beam width grows, and flag models that drift towards it.
- **I5** (learn): adopt multi-seed significance testing in evaluation, so that a new feature such as speaker turns is kept only if it beats the baseline across seeds.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [IBM](https://md.research.elamur.ai/free/en/companies/ibm/) — Company, Armonk, United States · 85 works · 66,118 citations
- [Institut Polytechnique de Paris](https://md.research.elamur.ai/free/en/universities/institut-polytechnique-de-paris/) — University, Palaiseau, France · 10 works · 1,555 citations
- [Laboratoire Traitement et Communication de l’Information](https://md.research.elamur.ai/free/en/labs/laboratoire-traitement-et-communication-de-l-information/) — Research institute, Palaiseau, France · 5 works · 318 citations
- [Télécom Paris](https://md.research.elamur.ai/free/en/universities/telecom-paris/) — University, Palaiseau, France · 6 works · 759 citations
- [IBM Research - Zurich](https://md.research.elamur.ai/free/en/labs/ibm-research-zurich/) — Research institute, Rüschlikon, Switzerland · 2 works · 1,615 citations

Also on the byline: [IBM (France)](https://ror.org/02wnbr922).

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 9,086 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 8,889 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 9,296 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 9,206 works

🧭 Topics (2): [🗨️ Dialogue systems](https://md.research.elamur.ai/free/en/topics/dialogue-systems/) · 401 works · [💬 Speech act theory](https://md.research.elamur.ai/free/en/topics/speech-act-theory/) · 160 works

## Cite

Cite as: Pierre Colombo, Emile Chapuis, Matteo Manica, Emmanuel Vignon, Giovanna Varni and Chloe Clavel. 2020. Guiding attention in Sequence-to-sequence models for Dialogue Act prediction. AAAI 2020.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{colombo2020guiding,
  title = {{Guiding attention in Sequence-to-sequence models for Dialogue Act prediction}},
  author = {Pierre Colombo, Emile Chapuis, Matteo Manica, Emmanuel Vignon, Giovanna Varni and Chloe Clavel},
  booktitle = {AAAI 2020},
  year = {2020},
  doi = {10.48550/arxiv.2002.08801},
  eprint = {2002.08801},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2002.08801}
}
```

- [← previous Transformer-based Context-aware Sarcasm Detection in Conversation Threads from Social Media](https://md.research.elamur.ai/free/en/papers/dong-2020-transformer-based-context-aware-sarcasm/)
- [next → Transformers on Sarcasm Detection with Context](https://md.research.elamur.ai/free/en/papers/kumar-2020-transformers-sarcasm-detection-context/)
- [↑ back to papers of 2020](https://md.research.elamur.ai/free/en/papers/2020/page/6/)

## References

30 references, 16 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. [2014] Bahdanau, D.; Cho, K.; and Bengio, Y. 2014. [Neural machine translation by jointly learning to align and translate](https://md.research.elamur.ai/free/en/papers/bahdanau-2014-neural-machine-translation-jointly-learning/). CoRR abs/1409.0473. [↗](https://arxiv.org/abs/1409.0473)
2. [2017] Bojanowski, P.; Grave, E.; Joulin, A.; and Mikolov, T. 2017. [Enriching word vectors with subword information](https://md.research.elamur.ai/free/en/papers/bojanowski-2017-enriching-word-vectors-subword-information/). Transactions of ACL 5:135–146. [↗](https://doi.org/10.1162/tacl_a_00051)
3. [2018] Bothe, C.; Weber, C.; Magg, S.; and Wermter, S. 2018. [A context-based approach for dialogue act recognition using simple recurrent neural networks](https://md.research.elamur.ai/free/en/papers/bothe-2018-context-based-approach-dialogue-act/). CoRR abs/1805.06280. [↗](https://arxiv.org/abs/1805.06280)
4. [2018] Chen, Z.; Yang, R.; Zhao, Z.; Cai, D.; and He, X. 2018. [Dialogue act recognition via crf-attentive structured network](https://md.research.elamur.ai/free/en/papers/chen-2017-dialogue-act-recognition-crf-attentive/). In The 41st International ACM SIGIR Conference on Research & Development in Information Retrieval, 225– 234. ACM.
5. [2014] Cho, K.; van Merrienboer, B.; Gülçehre, Ç.; Bougares, F.; Schwenk, H.; and Bengio, Y. 2014. Learning phrase representations using RNN encoder-decoder for statistical machine translation. CoRR abs/1406.1078. [↗](https://arxiv.org/abs/1406.1078)
6. [2013] Gimpel, K.; Batra, D.; Dyer, C.; and Shakhnarovich, G. 2013. A systematic exploration of diversity in machine translation. In Proceedings of EMNLP 2013, 1100–1111. [find](https://scholar.google.com/scholar?q=%5B2013%5D+Gimpel%2C+K%2E%3B+Batra%2C+D%2E%3B+Dyer%2C+C%2E%3B+and+Shakhnarovich%2C+G%2E+2013%2E+A+systematic+exploration+of+diversity+in+machine+translation%2E+In+Proceedings+of+EMNLP+2013%2C+1100%E2%80%931111%2E)
7. [2003] Janin, A.; Baron, D.; Edwards, J.; Ellis, D.; Gelbart, D.; Morgan, N.; Peskin, B.; Pfau, T.; Shriberg, E.; Stolcke, A.; et al. 2003. The icsi meeting corpus. In 2003 IEEE International Conference on Acoustics, Speech, and Signal Processing, 2003. Proceedings.(ICASSP’03)., volume 1, I–I. IEEE. [find](https://scholar.google.com/scholar?q=%5B2003%5D+Janin%2C+A%2E%3B+Baron%2C+D%2E%3B+Edwards%2C+J%2E%3B+Ellis%2C+D%2E%3B+Gelbart%2C+D%2E%3B+Morgan%2C+N%2E%3B+Peskin%2C+B%2E%3B+Pfau%2C+T%2E%3B+Shriberg%2C+E%2E%3B+Stolcke%2C+A%2E%3B+et+al%2E+2003%2E+The+icsi+meeting+corpus%2E+In+2003+IEEE+International+Conferen)
8. [2015] Jozefowicz, R.; Zaremba, W.; and Sutskever, I. 2015. An empirical exploration of recurrent network architectures. In International Conference on Machine Learning, 2342– 2350. [find](https://scholar.google.com/scholar?q=%5B2015%5D+Jozefowicz%2C+R%2E%3B+Zaremba%2C+W%2E%3B+and+Sutskever%2C+I%2E+2015%2E+An+empirical+exploration+of+recurrent+network+architectures%2E+In+International+Conference+on+Machine+Learning%2C+2342%E2%80%93+2350%2E)
9. [2002] Keizer, S.; op den Akker, R.; and Nijholt, A. 2002. [Dialogue act recognition with bayesian networks for dutch dialogues](https://md.research.elamur.ai/free/en/papers/keizer-2002-dialogue-act-recognition-bayesian-networks/). In Proceedings of the Third SIGdial Workshop on Discourse and Dialogue.
10. [2016] Khanpour, H.; Guntakandla, N.; and Nielsen, R. 2016. [Dialogue act classification in domain-independent conversations using a deep recurrent neural network](https://md.research.elamur.ai/free/en/papers/khanpour-2016-dialogue-act-classification-domain-independent/). In COLING.
11. [2014] Kingma, D. P., and Ba, J. 2014. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980. [↗](https://arxiv.org/abs/1412.6980)
12. [2018] Kumar, H.; Agarwal, A.; Dasgupta, R.; and Joshi, S. 2018. [Dialogue act sequence labeling using hierarchical encoder with crf](https://md.research.elamur.ai/free/en/papers/kumar-2017-dialogue-act-sequence-labeling-using/). In Thirty-Second AAAI Conference on Artificial Intelligence. [↗](https://doi.org/10.1609/aaai.v32i1.11701)
13. [2015] LeCun, Y.; Bengio, Y.; and Hinton, G. 2015. Deep learning. nature 521(7553):436. [find](https://scholar.google.com/scholar?q=%5B2015%5D+LeCun%2C+Y%2E%3B+Bengio%2C+Y%2E%3B+and+Hinton%2C+G%2E+2015%2E+Deep+learning%2E+nature+521%287553%29%3A436%2E)
14. [2016a] Li, J.; Galley, M.; Brockett, C.; Gao, J.; and Dolan, W. B. 2016a. [A diversity-promoting objective function for neural conversation models](https://md.research.elamur.ai/free/en/papers/li-2016-diversity-promoting-objective-function-neural/). In HLT-NAACL. [↗](https://doi.org/10.48550/arxiv.1510.03055)
15. [2016b] Li, J.; Galley, M.; Brockett, C.; Spithourakis, G.; Gao, J.; and Dolan, B. 2016b. [A persona-based neural conversation model](https://md.research.elamur.ai/free/en/papers/li-2016-persona-based-neural-conversation-model/). In Proceedings of the 54th Annual Meeting of ACL (Volume 1: Long Papers), 994–1003. Association for Computational Linguistics. [↗](https://doi.org/10.48550/arxiv.1603.06155)
16. [2018a] Li, R.; Lin, C.; Collinson, M.; Li, X.; and Chen, G. 2018a. A dual-attention hierarchical recurrent neural network for dialogue act classification. CoRR. [↗](https://doi.org/10.48550/arxiv.1810.09154)
17. [2018b] Li, Z.; Cai, J.; He, S.; and Zhao, H. 2018b. [Seq2seq dependency parsing](https://md.research.elamur.ai/free/en/papers/li-2018-seq2seq-dependency-parsing/). In Proceedings of the 27th International Conference on Computational Linguistics, 3203– 3214.
18. [2017] Loshchilov, I., and Hutter, F. 2017. Fixing weight decay regularization in adam. arXiv preprint arXiv:1711.05101. [↗](https://arxiv.org/abs/1711.05101)
19. [2015] Luong, T.; Pham, H.; and Manning, C. D. 2015. [Effective approaches to attention-based neural machine translation](https://md.research.elamur.ai/free/en/papers/luong-2015-effective-approaches-attention-based-neural/). In Proceedings of EMNLP 2015, 1412–1421. Lisbon, Portugal: Association for Computational Linguistics. [↗](https://doi.org/10.48550/arxiv.1508.04025)
20. [2013] Mikolov, T.; Sutskever, I.; Chen, K.; Corrado, G. S.; and Dean, J. 2013. [Distributed representations of words and phrases and their compositionality](https://md.research.elamur.ai/free/en/papers/mikolov-2013-distributed-representations-words-phrases-their/). In NeurIPS, 3111–3119. [↗](https://doi.org/10.48550/arxiv.1310.4546)
21. [2014] Pennington, J.; Socher, R.; and Manning, C. 2014. [Glove: Global vectors for word representation](https://md.research.elamur.ai/free/en/papers/pennington-2014-glove-global-vectors-word-representation/). In Proceedings of EMNLP 2014, 1532–1543.
22. [2019] Raheja, and Tetreault. 2019. [Dialogue act classification with context-aware self-attention](https://md.research.elamur.ai/free/en/papers/raheja-2019-dialogue-act-classification-context-aware/). CoRR abs/1904.02594. [↗](https://arxiv.org/abs/1904.02594)
23. [2004] Shriberg, E.; Dhillon, R.; Bhagat, S.; Ang, J.; and Carvey, H. 2004. [The icsi meeting recorder dialog act (mrda) corpus](https://md.research.elamur.ai/free/en/papers/shriberg-2004-icsi-meeting-recorder-dialog-act/). In Proceedings of the 5th SIGdial Workshop on Discourse and Dialogue at HLT-NAACL 2004. [↗](https://doi.org/10.21236/ada460980)
24. [2015] Sordoni, A.; Bengio, Y.; Vahabi, H.; Lioma, C.; Grue Simonsen, J.; and Nie, J.-Y. 2015. A hierarchical recurrent encoder-decoder for generative context-aware query suggestion. In Proceedings of the 24th ACM International on Conference on Information and Knowledge Management, 553–562. ACM. [find](https://scholar.google.com/scholar?q=%5B2015%5D+Sordoni%2C+A%2E%3B+Bengio%2C+Y%2E%3B+Vahabi%2C+H%2E%3B+Lioma%2C+C%2E%3B+Grue+Simonsen%2C+J%2E%3B+and+Nie%2C+J%2E%2DY%2E+2015%2E+A+hierarchical+recurrent+encoder%2Ddecoder+for+generative+context%2Daware+query+suggestion%2E+In+Proceedings+of)
25. [1998] Stolcke, A.; Shriberg, E.; Bates, R.; Coccaro, N.; Jurafsky, D.; Martin, R.; Meteer, M.; Ries, K.; Taylor, P.; Van Ess-Dykema, C.; et al. 1998. Dialog act modeling for conversational speech. In AAAI Spring Symposium on Applying Machine Learning to Discourse Processing, 98–105. [find](https://scholar.google.com/scholar?q=%5B1998%5D+Stolcke%2C+A%2E%3B+Shriberg%2C+E%2E%3B+Bates%2C+R%2E%3B+Coccaro%2C+N%2E%3B+Jurafsky%2C+D%2E%3B+Martin%2C+R%2E%3B+Meteer%2C+M%2E%3B+Ries%2C+K%2E%3B+Taylor%2C+P%2E%3B+Van+Ess%2DDykema%2C+C%2E%3B+et+al%2E+1998%2E+Dialog+act+modeling+for+conversational+speech%2E+In)
26. [2000] Stolcke, A.; Ries, K.; Coccaro, N.; Shriberg, E.; Bates, R.; Jurafsky, D.; Taylor, P.; Martin, R.; Ess-Dykema, C. V.; and Meteer, M. 2000. [Dialogue act modeling for automatic tagging and recognition of conversational speech](https://md.research.elamur.ai/free/en/papers/stolcke-2000-dialogue-act-modeling-automatic-tagging/). Computational linguistics 26(3):339–373. [↗](https://doi.org/10.1162/089120100561737)
27. [2006] Surendran, D., and Levow, G.-A. 2006. Dialog act tagging with support vector machines and hidden markov models. In Ninth International Conference on Spoken Language Processing. [find](https://scholar.google.com/scholar?q=%5B2006%5D+Surendran%2C+D%2E%2C+and+Levow%2C+G%2E%2DA%2E+2006%2E+Dialog+act+tagging+with+support+vector+machines+and+hidden+markov+models%2E+In+Ninth+International+Conference+on+Spoken+Language+Processing%2E)
28. [2014] Sutskever, I.; Vinyals, O.; and Le, Q. V. 2014. Sequence to sequence learning with neural networks. In NeurIPS, 3104–3112. [↗](https://doi.org/10.48550/arxiv.1409.3215)
29. [2016] Wiseman, S., and Rush, A. M. 2016. Sequence-to-sequence learning as beam-search optimization. In EMNLP. [find](https://scholar.google.com/scholar?q=%5B2016%5D+Wiseman%2C+S%2E%2C+and+Rush%2C+A%2E+M%2E+2016%2E+Sequence%2Dto%2Dsequence+learning+as+beam%2Dsearch+optimization%2E+In+EMNLP%2E)
30. [2016] Wu, Y.; Schuster, M.; Chen, Z.; Le, Q. V.; Norouzi, M.; Macherey, W.; Krikun, M.; Cao, Y.; Gao, Q.; Macherey, K.; et al. 2016. Google’s neural machine translation system: Bridging the gap between human and machine translation. arXiv preprint arXiv:1609.08144. [↗](https://arxiv.org/abs/1609.08144)

*Updated 24 October 2026*