---
title: "EDA — paper by Bothe et al. (2020) — summary and ideas"
description: "Conversational emotion corpora label each utterance with an emotion but say nothing… 75.1% accuracy. 33 references, cited by 3 papers here."
authors: ["Chandrakant Bothe", "Cornelius Weber", "Sven Magg", "Stefan Wermter"]
year: 2020
venue: "Proceedings of the Twelfth Language Resources and Evaluation Conference"
source: "https://aclanthology.org/2020.lrec-1.78/"
arxiv: "1912.00819"
licence: "CC BY-NC 4.0 (ELRA, LREC proceedings)"
licence_url: "https://creativecommons.org/licenses/by-nc/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/bothe-2020-eda-enriching-emotional-dialogue-acts/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/bothe-2020-eda-enriching-emotional-dialogue-acts/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2020](https://md.research.elamur.ai/free/en/papers/2020/page/8/) / EDA: Enriching Emotional Dialogue Acts using an Ensemble of Neural Annotators

# EDA: Enriching Emotional Dialogue Acts using an Ensemble of Neural Annotators

**Authors:** [Chandrakant Bothe](https://md.research.elamur.ai/free/en/authors/chandrakant-bothe/), [Cornelius Weber](https://md.research.elamur.ai/free/en/authors/cornelius-weber/), [Sven Magg](https://md.research.elamur.ai/free/en/authors/sven-magg/) and [Stefan Wermter](https://md.research.elamur.ai/free/en/authors/stefan-wermter/) · **Year:** 2020 · **Venue:** [Proceedings of the Twelfth Language Resources and Evaluation Conference](https://md.research.elamur.ai/free/en/venues/lrec/) ([LREC 2020](https://md.research.elamur.ai/free/en/venues/lrec/2020/)) · **Publisher:** [European Language Resources Association (ELRA)](https://md.research.elamur.ai/free/en/publishers/european-language-resources-association-elra/) · **Licence:** [CC BY-NC 4.0 (ELRA, LREC proceedings)](https://creativecommons.org/licenses/by-nc/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2020.lrec-1.78/) · [PDF](https://aclanthology.org/2020.lrec-1.78.pdf) · [arXiv 1912.00819](https://arxiv.org/abs/1912.00819) · [OpenAlex](https://openalex.org/W3012348221)

**Citations:** 2 citations (OpenAlex, 2026-10-14) · 3 in our library, of the 141,840 papers whose bibliographies we read

## Summary
Conversational emotion corpora label each utterance with an emotion but say nothing about its dialogue act, the intention or function an utterance performs, such as asking, agreeing, or thanking. Dialogue systems and discourse analysis need both layers. The authors fill this gap automatically. They trained five recurrent neural classifiers with attention on the DAMSL-tagged Switchboard Dialogue Act (SwDA) corpus, using pre-trained ELMo embeddings. Some classifiers look only at the current utterance, and others also see the two preceding turns. An ensemble merges their outputs, first by majority vote and then by ranking model confidence. On the SwDA test set it scored slightly above the best single model. The ensemble then labelled two multimodal emotion corpora, IEMOCAP and MELD. The authors checked label reliability with inter-annotator agreement measures and released the result as the Emotional Dialogue Acts (EDA) corpus. Cross-tabulating the two label layers revealed systematic pairings between intentions and feelings. This makes EDA a resource for building dialogue agents that respond to both what the user means and how they feel.

## Key points
- The annotator pool has five members trained on SwDA:
  - Utt-level-1 (RNN + attention), 75.1% accuracy.
  - Context-1 (hierarchical RNN over the current and two prior utterances), 77.55% accuracy, the strongest single model.
  - Utt-level-2 (mean of embeddings + feed-forward layer), 72.6% accuracy.
  - Context-2, 75.9% accuracy.
  - Context-3, 74.9% accuracy.
- The ensemble reached 0.778 accuracy on the SwDA test set. Spearman correlation with gold labels was also reported for every model (ensemble 0.822).
- Words are represented by 1024-dimensional pre-trained ELMo vectors. The public Discourse-Wizard demo uses cheaper, model-trained embeddings instead.
- The target corpora are IEMOCAP (Ekman categories plus Frustration and Excited, and valence/arousal/dominance) and MELD (7 emotion categories plus sentiment). Their utterances are not split into DAMSL segments.
- Observed pairings between dialogue acts and emotions:
  - Accept/Agree with Joy
  - Reject with Anger
  - Acknowledgement with Surprise
  - Thanking with Joy
  - Apology with Sadness

## Interesting ideas
- The pool deliberately mixes architectures (attention RNN, embedding averaging, context vs. no context), so model errors are less correlated and a majority vote becomes more meaningful. Confidence ranking only breaks ties when no majority exists.
- Before projecting labels onto an unlabelled corpus, the authors scored each annotator and the ensemble against a gold test set from the source domain. They reported a rank correlation alongside accuracy.
- Dialogue-act recognition improves with a small, fixed window of the two preceding utterances, without encoding the whole dialogue.
- Overlaying an automatically added label layer on an existing human-labelled one turns two separate corpora into a resource for studying how the layers relate. Examples are apologies co-occurring with sadness and rejections with anger.
- The emotion corpora keep multi-function utterances unsegmented, so a statement followed by a question gets a single label. The paper explicitly flags this as a granularity mismatch with DAMSL.

## For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

- **I1** (learn): Witness parsers for silver data could be chosen for architectural diversity, with a two-step merge rule: agreement first, highest calibrated confidence as a fallback.
- **I2** (learn): Every silver-labelling run could be preceded by a gold-set check per witness and for the merged output, with the numbers logged next to the silver data.
- **I3** (frames): The speech-act classifier in prag could use features from the previous two turns or sentences, which keeps its decisions local and explainable.
- **I4** (frames): Speech-act labels and emotion or attitude cues in tales could be cross-tabulated to derive explainable priors, for example treating an apology as evidence of a sad or regretful speaker state.
- **I5** (frames): Speech acts should be assigned per clause or segment from the UD tree rather than per sentence, so mixed utterances yield several acts.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Universität Hamburg](https://md.research.elamur.ai/free/en/universities/universitat-hamburg/) — University, Hamburg, Germany · 39 works · 4,669 citations

📚 Shelves (6): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 9,086 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 8,889 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 9,296 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 9,206 works · [💭 Psychology](https://md.research.elamur.ai/free/en/shelves/psychology/) · 798 works · [💭 Experimental and Cognitive Psychology](https://md.research.elamur.ai/free/en/shelves/psychology/experimental-and-cognitive-psychology/) · 463 works

🧭 Topics (3): [😊 Sentiment & emotion](https://md.research.elamur.ai/free/en/topics/sentiment-emotion/) · 608 works · [🗨️ Dialogue systems](https://md.research.elamur.ai/free/en/topics/dialogue-systems/) · 401 works · [💬 Speech act theory](https://md.research.elamur.ai/free/en/topics/speech-act-theory/) · 160 works

## Cite

Cite as: Chandrakant Bothe, Cornelius Weber, Sven Magg and Stefan Wermter. 2020. EDA: Enriching Emotional Dialogue Acts using an Ensemble of Neural Annotators. Proceedings of the Twelfth Language Resources and Evaluation Conference.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{bothe2020eda,
  title = {{EDA: Enriching Emotional Dialogue Acts using an Ensemble of Neural Annotators}},
  author = {Chandrakant Bothe, Cornelius Weber, Sven Magg and Stefan Wermter},
  booktitle = {Proceedings of the Twelfth Language Resources and Evaluation Conference},
  year = {2020},
  eprint = {1912.00819},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1912.00819}
}
```

- [← previous Eavesdropping: What is it good for?](https://md.research.elamur.ai/free/en/papers/phillips-2020-eavesdropping-what-is-it-good/)
- [next → Efficient Arabic emotion recognition using deep neural networks](https://md.research.elamur.ai/free/en/papers/ali-2020-efficient-arabic-emotion-recognition-using/)
- [↑ back to papers of 2020](https://md.research.elamur.ai/free/en/papers/2020/page/8/)

## References

33 references, 6 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Allen, J. and Core, M. (1997). Draft of DAMSL: Dialogue Act Markup in Several Layers. Carnegie Mellon University. [find](https://scholar.google.com/scholar?q=Allen%2C+J%2E+and+Core%2C+M%2E+%281997%29%2E+Draft+of+DAMSL%3A+Dialogue+Act+Markup+in+Several+Layers%2E+Carnegie+Mellon+University%2E)
2. Austin, J. L. (1962). How to Do Things with Words. Oxford University Press. [find](https://scholar.google.com/scholar?q=Austin%2C+J%2E+L%2E+%281962%29%2E+How+to+Do+Things+with+Words%2E+Oxford+University+Press%2E)
3. Berg, M. M. (2015). NADIA: A Simplified Approach Towards the Development of Natural Dialogue Systems. In International Conference on Applications of Natural Language to Information Systems, pages 144–150. Springer. [find](https://scholar.google.com/scholar?q=Berg%2C+M%2E+M%2E+%282015%29%2E+NADIA%3A+A+Simplified+Approach+Towards+the+Development+of+Natural+Dialogue+Systems%2E+In+International+Conference+on+Applications+of+Natural+Language+to+Information+Systems%2C+pages+144%E2%80%93)
4. Bothe, C., Magg, S., Weber, C., and Wermter, S. (2018a). Conversational Analysis using Utterance-level Attention-based Bidirectional Recurrent Neural Networks. In Proceedings of the International Conference INTERSPEECH 2018, pages 996–1000. International Speech Communication Association (ISCA). [find](https://scholar.google.com/scholar?q=Bothe%2C+C%2E%2C+Magg%2C+S%2E%2C+Weber%2C+C%2E%2C+and+Wermter%2C+S%2E+%282018a%29%2E+Conversational+Analysis+using+Utterance%2Dlevel+Attention%2Dbased+Bidirectional+Recurrent+Neural+Networks%2E+In+Proceedings+of+the+International+Conf)
5. Bothe, C., Magg, S., Weber, C., and Wermter, S. (2018b). Discourse-Wizard: Discovering Deep Discourse Structure in your Conversation with RNNs. preprint arXiv:1806.11420. [↗](https://arxiv.org/abs/1806.11420)
6. Bothe, C., Weber, C., Magg, S., and Wermter, S. (2018c). [A Context-based Approach for Dialogue Act Recognition using Simple Recurrent Neural Networks](https://md.research.elamur.ai/free/en/papers/bothe-2018-context-based-approach-dialogue-act/). In Proceedings of the Eleventh International Conference on Language Resources and Evaluation, LREC 2018, pages 1952–1957. European Language Resources Association (ERLA).
7. Busso, C., Bulut, M., Lee, C.-C., Kazemzadeh, A., Mower, E., Kim, S., Chang, J. N., Lee, S., and Narayanan, S. S. (2008). IEMOCAP: Interactive emotional dyadic motion capture database. Language Resources and Evaluation, 42(4):335–359. [↗](https://doi.org/10.1007/s10579-008-9076-6)
8. Chen, S.-Y., Hsu, C.-C., Kuo, C.-C., Huang, T.-H., and Ku, L.-W. (2018). EmotionLines: An Emotion Corpus of Multi-Party Conversations. In Proceedings of the Eleventh International Conference on Language Resources and Evaluation, LREC 2018, pages 1597–1601. [↗](https://doi.org/10.48550/arxiv.1802.08379)
9. Cowie, R. and Cornelius, R. R. (2003). Describing the Emotional States That Are Expressed in Speech. Speech Communication, 40(1-2):5–32. [↗](https://doi.org/10.1016/s0167-6393(02)00071-7)
10. Ekman, P., Friesen, W. V., O’sullivan, M., Chan, A., Diacoyanni-Tarlatzis, I., Heider, K., Krause, R., LeCompte, W. A., Pitcairn, T., Ricci-Bitti, P. E., et al. (1987). Universals and Cultural Differences in the Judgments of Facial Expressions of Emotion. Journal of Personality and Social Psychology, 53(4):712–717. [↗](https://doi.org/10.1037//0022-3514.53.4.712)
11. Fleiss, J. L. (1971). Measuring nominal scale agreement among many raters. Psychological Bulletin, 76(5):378. [↗](https://doi.org/10.1037/h0031619)
12. Godfrey, J. J., Holliman, E. C., and McDaniel, J. (1992). SWITCHBOARD: Telephone speech corpus for research and development. In Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing, volume 1, pages 517–520. [find](https://scholar.google.com/scholar?q=Godfrey%2C+J%2E+J%2E%2C+Holliman%2C+E%2E+C%2E%2C+and+McDaniel%2C+J%2E+%281992%29%2E+SWITCHBOARD%3A+Telephone+speech+corpus+for+research+and+development%2E+In+Proceedings+of+the+IEEE+International+Conference+on+Acoustics%2C+Speech+an)
13. Ihasz, P. L. and Kryssanov, V. (2018). Emotions and Intentions Mediated with Dialogue Acts. In Proceedings of the 5th International Conference on Business and Industrial Research (ICBIR), pages 125–130. IEEE. [find](https://scholar.google.com/scholar?q=Ihasz%2C+P%2E+L%2E+and+Kryssanov%2C+V%2E+%282018%29%2E+Emotions+and+Intentions+Mediated+with+Dialogue+Acts%2E+In+Proceedings+of+the+5th+International+Conference+on+Business+and+Industrial+Research+%28ICBIR%29%2C+pages+125%E2%80%9313)
14. Jurafsky, D., Shriberg, E., and Biasca, D. (1997). Switchboard Dialog Act Corpus. Technical report, International Computer Science Inst. Berkeley CA. [find](https://scholar.google.com/scholar?q=Jurafsky%2C+D%2E%2C+Shriberg%2C+E%2E%2C+and+Biasca%2C+D%2E+%281997%29%2E+Switchboard+Dialog+Act+Corpus%2E+Technical+report%2C+International+Computer+Science+Inst%2E+Berkeley+CA%2E)
15. Jurafsky, D. (1997). Switchboard SWBD-DAMSL Shallow-Discourse-Function Annotation Coders Manual, draft 13. Technical Report 97-01, University of Colorado Institute of Cognitive Science, pages 225–233. [find](https://scholar.google.com/scholar?q=Jurafsky%2C+D%2E+%281997%29%2E+Switchboard+SWBD%2DDAMSL+Shallow%2DDiscourse%2DFunction+Annotation+Coders+Manual%2C+draft+13%2E+Technical+Report+97%2D01%2C+University+of+Colorado+Institute+of+Cognitive+Science%2C+pages+225%E2%80%93233%2E)
16. Kalchbrenner, N., Espeholt, L., Simonyan, K., Oord, A. v. d., Graves, A., and Kavukcuoglu, K. (2016). Neural Machine Translation in Linear Time. arXiv:1610.10099. [↗](https://arxiv.org/abs/1610.10099)
17. Krippendorff, K. (1970). Estimating the Reliability, Systematic Error and Random Error of Interval Data. Educational and Psychological Measurement, 30(1):61–70. [↗](https://doi.org/10.1177/001316447003000105)
18. Lakomkin, E., Zamani, M. A., Weber, C., Magg, S., and Wermter, S. (2019). Incorporating End-to-End Speech Recognition Models for Sentiment Analysis. In International Conference on Robotics and Automation (ICRA), pages 7976–7982. IEEE. [find](https://scholar.google.com/scholar?q=Lakomkin%2C+E%2E%2C+Zamani%2C+M%2E+A%2E%2C+Weber%2C+C%2E%2C+Magg%2C+S%2E%2C+and+Wermter%2C+S%2E+%282019%29%2E+Incorporating+End%2Dto%2DEnd+Speech+Recognition+Models+for+Sentiment+Analysis%2E+In+International+Conference+on+Robotics+and+Automat)
19. McHugh, M. L. (2012). Interrater reliability: the kappa statistic. Biochemia Medica, 22(3):276–282. [↗](https://doi.org/10.11613/bm.2012.031)
20. McKeown, G., Valstar, M., Cowie, R., Pantic, M., and Schroder, M. (2012). The SEMAINE Database: Annotated Multimodal Records of Emotionally Colored Conversations between a Person and a Limited Agent. IEEE Transactions on Affective Computing, 3(1):5–17. [↗](https://doi.org/10.1109/t-affc.2011.20)
21. Osgood, C. E., May, W. H., Miron, M. S., and Miron, M. S. (1975). Cross-cultural Universals of Affective Meaning, volume 1. University of Illinois Press. [find](https://scholar.google.com/scholar?q=Osgood%2C+C%2E+E%2E%2C+May%2C+W%2E+H%2E%2C+Miron%2C+M%2E+S%2E%2C+and+Miron%2C+M%2E+S%2E+%281975%29%2E+Cross%2Dcultural+Universals+of+Affective+Meaning%2C+volume+1%2E+University+of+Illinois+Press%2E)
22. Peters, M. E., Neumann, M., Iyyer, M., Gardner, M., Clark, C., Lee, K., and Zettlemoyer, L. (2018). [Deep Contextualized Word Representations](https://md.research.elamur.ai/free/en/papers/peters-2018-deep-contextualized-word-representations/). North American Chapter of the Association for Computational Linguistics (NAACL 2018), pages 2227–2237.
23. Poria, S., Hazarika, D., Majumder, N., Naik, G., Cambria, E., and Mihalcea, R. (2019). [MELD: A Multimodal Multi-Party Dataset for Emotion Recognition in Conversations](https://md.research.elamur.ai/free/en/papers/poria-2018-meld-multimodal-multi-party-dataset/). In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 527–536. Association for Computational Linguistics.
24. Powers, D. M. (2012). The Problem with Kappa. In Proceedings of the 13th Conference of the European Chapter of the Association for Computational Linguistics, pages 345–355. Association for Computational Linguistics. [↗](https://doi.org/10.13140/rg.2.1.3492.0487)
25. Russell, J. A. and Mehrabian, A. (1977). Evidence for a Three-Factor Theory of Emotions. Journal of Research in Personality, 11(3):273–294. [↗](https://doi.org/10.1016/0092-6566(77)90037-x)
26. Searle, J. R. (1979). Expression and Meaning: Studies in the Theory of Speech Acts. Cambridge University Press. [find](https://scholar.google.com/scholar?q=Searle%2C+J%2E+R%2E+%281979%29%2E+Expression+and+Meaning%3A+Studies+in+the+Theory+of+Speech+Acts%2E+Cambridge+University+Press%2E)
27. Skantze, G. (2007). Error Handling in Spoken Dialogue Systems-Managing Uncertainty, Grounding and Miscommunication: Chapter 2, Spoken Dialogue Systems. KTH Computer Science and Communication. [find](https://scholar.google.com/scholar?q=Skantze%2C+G%2E+%282007%29%2E+Error+Handling+in+Spoken+Dialogue+Systems%2DManaging+Uncertainty%2C+Grounding+and+Miscommunication%3A+Chapter+2%2C+Spoken+Dialogue+Systems%2E+KTH+Computer+Science+and+Communication%2E)
28. Stolcke, A., Ries, K., Coccaro, N., Shriberg, E., Bates, R., Jurafsky, D., Taylor, P., Martin, R., Van Ess-Dykema, C., and Meteer, M. (2000). [Dialogue Act Modeling for Automatic Tagging and Recognition of Conversational Speech](https://md.research.elamur.ai/free/en/papers/stolcke-2000-dialogue-act-modeling-automatic-tagging/). Computational Linguistics, 26(3):339–373. [↗](https://doi.org/10.1162/089120100561737)
29. Ultes, S., Rojas Barahona, L. M., Su, P.-H., Vandyke, D., Kim, D., Casanueva, I., Budzianowski, P., Mrkšić, N., Wen, T.-H., Gasic, M., and Young, S. (2017). PyDial: A Multi-domain Statistical Dialogue System Toolkit. In Proceedings of the ACL 2017, System Demonstrations, pages 73–78. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Ultes%2C+S%2E%2C+Rojas+Barahona%2C+L%2E+M%2E%2C+Su%2C+P%2E%2DH%2E%2C+Vandyke%2C+D%2E%2C+Kim%2C+D%2E%2C+Casanueva%2C+I%2E%2C+Budzianowski%2C+P%2E%2C+Mrks%CC%8Cic%CC%81%2C+N%2E%2C+Wen%2C+T%2E%2DH%2E%2C+Gasic%2C+M%2E%2C+and+Young%2C+S%2E+%282017%29%2E+PyDial%3A+A+Multi%2Ddomain+Statistical+Dialog)
30. Wang, A., Singh, A., Michael, J., Hill, F., Levy, O., and Bowman, S. R. (2018). [GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding](https://md.research.elamur.ai/free/en/papers/wang-2018-glue-multi-task-benchmark-analysis/). arXiv preprint arXiv:1804.07461. [↗](https://arxiv.org/abs/1804.07461)
31. Wermter, S. and Löchel, M. (1996). Learning dialog act processing. In Proceedings of the 16th Conference on Computational Linguistics, volume 2, pages 740–745. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Wermter%2C+S%2E+and+Lo%CC%88chel%2C+M%2E+%281996%29%2E+Learning+dialog+act+processing%2E+In+Proceedings+of+the+16th+Conference+on+Computational+Linguistics%2C+volume+2%2C+pages+740%E2%80%93745%2E+Association+for+Computational+Linguisti)
32. Wood, I., McCrae, J. P., Andryushechkin, V., and Buitelaar, P. (2018). A Comparison Of Emotion Annotation Schemes And A New Annotated Data Set. In Proceedings of the Eleventh International Conference on Language Resources and Evaluation, LREC 2018, pages 1197–1202. [find](https://scholar.google.com/scholar?q=Wood%2C+I%2E%2C+McCrae%2C+J%2E+P%2E%2C+Andryushechkin%2C+V%2E%2C+and+Buitelaar%2C+P%2E+%282018%29%2E+A+Comparison+Of+Emotion+Annotation+Schemes+And+A+New+Annotated+Data+Set%2E+In+Proceedings+of+the+Eleventh+International+Conference+)
33. Yang, Z., Dai, Z., Yang, Y., Carbonell, J., Salakhutdinov, R. R., and Le, Q. V. (2019). [XLNet: Generalized Autoregressive Pretraining for Language Understanding](https://md.research.elamur.ai/free/en/papers/yang-2019-xlnet-generalized-autoregressive-pretraining-language/). In Advances in neural information processing systems, pages 5754–5764. [↗](https://doi.org/10.48550/arxiv.1906.08237)

*Updated 24 October 2026*