---
title: "Adaptation of Hierarchical Structured Models for Speech Act"
description: "First, they build an end-to-end hierarchical model: a sentence-level bi-LSTM feeds a conversation-level… 34 references, cited by 3 papers here."
authors: ["Tasnim Mohiuddin", "Thanh-Tung Nguyen", "Shafiq Joty"]
year: 2019
venue: "Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)"
source: "https://aclanthology.org/N19-1134/"
doi: "10.18653/v1/n19-1134"
arxiv: "1904.04021"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/mohiuddin-2019-adaptation-hierarchical-structured-models-speech/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/mohiuddin-2019-adaptation-hierarchical-structured-models-speech/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2019](https://md.research.elamur.ai/free/en/papers/2019/page/6/) / Adaptation of Hierarchical Structured Models for Speech Act Recognition in Asynchronous Conversation

# Adaptation of Hierarchical Structured Models for Speech Act Recognition in Asynchronous Conversation

**Authors:** [Tasnim Mohiuddin](https://md.research.elamur.ai/free/en/authors/tasnim-mohiuddin/), [Thanh-Tung Nguyen](https://md.research.elamur.ai/free/en/authors/thanh-tung-nguyen/) and [Shafiq Joty](https://md.research.elamur.ai/free/en/authors/shafiq-joty/) · **Year:** 2019 · **Venue:** [Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)](https://md.research.elamur.ai/free/en/venues/naacl/) ([NAACL 2019](https://md.research.elamur.ai/free/en/venues/naacl/2019/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/N19-1134/) · [PDF](https://aclanthology.org/N19-1134.pdf) · [DOI 10.18653/v1/n19-1134](https://doi.org/10.18653/v1/n19-1134) · [arXiv 1904.04021](https://arxiv.org/abs/1904.04021) · [OpenAlex](https://openalex.org/W2935047982)

**Citations:** 4 citations (OpenAlex, 2026-10-14) · [3](https://md.research.elamur.ai/free/en/papers/mohiuddin-2019-adaptation-hierarchical-structured-models-speech/cited-by/) in our library, of the 141,840 papers whose bibliographies we read

## Summary
Speech act recognition (SAR) labels each sentence in a conversation with its communicative function, such as Question, Suggestion, Response or Polite. Asynchronous conversations like forums and emails have little labeled data, while spoken synchronous corpora such as MRDA are large. The authors tackle this gap in three steps. First, they build an end-to-end hierarchical model: a sentence-level bi-LSTM feeds a conversation-level bi-LSTM, topped by either a Softmax or a CRF layer (H-LSTM and H-LSTM-CRF). Second, they train word embeddings on a large unlabeled conversational corpus. Third, they apply domain adversarial training (Ganin et al., 2016) so the encoder learns features that are useful for classifying acts but do not reveal whether a sentence comes from meetings or from written threads. They evaluate on the forum and email datasets QC3, TA and BC3, and on MRDA. Trained on in-domain data alone, the hierarchical models already set a new state of the art. Adversarial adaptation adds clear further gains. This matters because it shows how to reuse labeled data from a mismatched domain without simply pooling it.

## Key points
- Datasets: QC3, TA and BC3 (forum and email threads) are the targets, and the MRDA meeting corpus is the source. Code, conversational embeddings and data are publicly released.
- Simply concatenating MRDA with the target data (the "Merge" baseline) or fine-tuning on it works worse than adversarial adaptation, for every model and every dataset.
- **Semi-supervised adaptation (50% of target labels):**
  - It gains about 11 F1 points absolute over Merge on QC3 and BC3, and 7–8 points over Fine-tune.
  - It also beats in-domain models trained on 100% of the target labels.
- **Full supervision and no supervision:**
  - With full target labels, adaptation adds roughly 3 points over the baselines and 3–4 points over in-domain training.
  - With no target labels, adaptation underperforms the transfer baseline, because the much larger MRDA set dominates training.
- Neural SCL performed worst, possibly because pivot features do not match across the two domains. Most errors confuse Statement with Response or with Suggestion, the same pairs that troubled human annotators.

## Interesting ideas
- An adversarial domain classifier on the shared encoder pushes features toward domain invariance, which beats naive data pooling when source and target differ in style and vocabulary
- A two-level encoder (words into sentences, sentences into a conversation) combined with a CRF over act labels captures dependencies between consecutive acts, for example that a Response tends to follow a Question
- The CRF variant adapts better when target labels are scarce, because tag-transition structure learned on the source domain carries over to the target
- Learning curves show most of the benefit arrives with the first quarter of target labels, which suggests concentrating annotation effort early
- A failure check that adapts with zero target labels shows the source domain can overwhelm the task, a useful negative control

## For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

- **I1** (frames): When training speech-act or implicature classifiers on dialogue-style data, a domain-adversarial head could keep the features usable for written fables and school texts.
- **I2** (frames): The prag crate could score speech-act sequences across a discourse with explicit, inspectable transition weights instead of labeling each sentence in isolation.
- **I3** (learn): Low-resource stages could favor structured output layers whose transition tables are learned on silver data and then reused across genres.
- **I4** (learn): Budgets for the LLM teacher tiers could be planned around such curves, labeling a small in-genre seed before scaling up.
- **I5** (learn): Evaluations should include a "source-only" run to detect cases where borrowed treebank or silver data actually hurts the target genre.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Salesforce](https://md.research.elamur.ai/free/en/companies/salesforce/) — Company, San Francisco, United States · 37 works · 15,019 citations
- [Nanyang Technological University](https://md.research.elamur.ai/free/en/universities/nanyang-technological-university/) — University, Singapore, Singapore · 73 works · 24,390 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 9,085 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 8,888 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 9,296 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 9,206 works

🧭 Topics (1): [💬 Speech act theory](https://md.research.elamur.ai/free/en/topics/speech-act-theory/) · 160 works

## Cite

Cite as: Tasnim Mohiuddin, Thanh-Tung Nguyen and Shafiq Joty. 2019. Adaptation of Hierarchical Structured Models for Speech Act Recognition in Asynchronous Conversation. Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers).

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{mohiuddin2019adaptation,
  title = {{Adaptation of Hierarchical Structured Models for Speech Act Recognition in Asynchronous Conversation}},
  author = {Tasnim Mohiuddin, Thanh-Tung Nguyen and Shafiq Joty},
  booktitle = {Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)},
  year = {2019},
  doi = {10.18653/v1/n19-1134},
  eprint = {1904.04021},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.18653/v1/n19-1134}
}
```

- [← previous A Probabilistic Approach to Syntactic Variation in Biblical Hebrew](https://md.research.elamur.ai/free/en/papers/bijl-2019-probabilistic-approach-syntactic-variation-biblical/)
- [next → ANETAC: Arabic Named Entity Transliteration and Classification Dataset](https://md.research.elamur.ai/free/en/papers/ameur-2019-anetac-arabic-named-entity-transliteration/)
- [↑ back to papers of 2019](https://md.research.elamur.ai/free/en/papers/2019/page/6/)

## References

34 references, 16 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Martı́n Arjovsky, Soumith Chintala, and Léon Bottou. 2017. Wasserstein GAN. CoRR, abs/1701.07875. [↗](https://arxiv.org/abs/1701.07875)
2. John Langshaw Austin. 1962. How to do things with words. Harvard University Press. [find](https://scholar.google.com/scholar?q=John+Langshaw+Austin%2E+1962%2E+How+to+do+things+with+words%2E+Harvard+University+Press%2E)
3. Vitor R. Carvalho and William W. Cohen. 2005. On the collective classification of email ”speech acts”. In SIGIR ’05, pages 345–352, New York, NY, USA. [find](https://scholar.google.com/scholar?q=Vitor+R%2E+Carvalho+and+William+W%2E+Cohen%2E+2005%2E+On+the+collective+classification+of+email+%E2%80%9Dspeech+acts%E2%80%9D%2E+In+SIGIR+%E2%80%9905%2C+pages+345%E2%80%93352%2C+New+York%2C+NY%2C+USA%2E)
4. Kyunghyun Cho, Bart van Merriënboer, Çalar Gülçehre, Dzmitry Bahdanau, Fethi Bougares, Holger Schwenk, and Yoshua Bengio. 2014. Learning phrase representations using rnn encoder–decoder for statistical machine translation. In EMNLP’14, Doha, Qatar. [↗](https://doi.org/10.48550/arxiv.1406.1078)
5. Chenhui Chu and Rui Wang. 2018. [A survey of domain adaptation for neural machine translation](https://md.research.elamur.ai/free/en/papers/chu-2018-survey-domain-adaptation-neural-machine/). In Proceedings of the 27th International Conference on Computational Linguistics, pages 1304–1319. Association for Computational Linguistics. [↗](https://doi.org/10.48550/arxiv.1806.00258)
6. William W. Cohen, Vitor R. Carvalho, and Tom M. Mitchell. 2004. [Learning to classify email into “speech acts”](https://md.research.elamur.ai/free/en/papers/cohen-2004-learning-classify-email-into-speech/). In EMNLP’04, pages 309–316.
7. Ronan Collobert, Jason Weston, Léon Bottou, Michael Karlen, Koray Kavukcuoglu, and Pavel Kuksa. 2011. [Natural language processing (almost) from scratch](https://md.research.elamur.ai/free/en/papers/collobert-2011-natural-language-processing-almost-scratch/). JMLR, 12:2493–2537.
8. Matt Crane. 2018. Questionable answers in question answering research: Reproducibility and variability of published results. TACL’18, 6:241–252. [↗](https://doi.org/10.1162/tacl_a_00018)
9. Rajdip Dhillon, Sonali Bhagat, Hannah Carvey, and Elizabeth Shriberg. 2004. Meeting Recorder Project: Dialog Act Labeling Guide. Technical report, ICSI Tech. Report. [↗](https://doi.org/10.21236/ada607947)
10. Yaroslav Ganin, Evgeniya Ustinova, Hana Ajakan, Pascal Germain, Hugo Larochelle, François Lavi-olette, Mario Marchand, and Victor Lempitsky. 2016. Domain-adversarial training of neural networks. JMLR, 17(1):2096–2030. [↗](https://doi.org/10.1007/978-3-319-58347-1_10)
11. Kevin Gimpel, Nathan Schneider, Brendan O’Connor, Dipanjan Das, Daniel Mills, Jacob Eisenstein, Michael Heilman, Dani Yogatama, Jeffrey Flanigan, and Noah A Smith. 2011. [Part-of-speech tagging for twitter: Annotation, features, and experiments](https://md.research.elamur.ai/free/en/papers/gimpel-2011-part-speech-tagging-twitter-annotation/). In HLT’11: short papers-Volume 2, pages 42–47. [↗](https://doi.org/10.21236/ada547371)
12. Xavier Glorot and Yoshua Bengio. 2010. Understanding the difficulty of training deep feedforward neural networks. In JMLR, Sardinia, Italy. [find](https://scholar.google.com/scholar?q=Xavier+Glorot+and+Yoshua+Bengio%2E+2010%2E+Understanding+the+difficulty+of+training+deep+feedforward+neural+networks%2E+In+JMLR%2C+Sardinia%2C+Italy%2E)
13. Sepp Hochreiter and Jürgen Schmidhuber. 1997. Long short-term memory. Neural computation, 9(8):1735–1780. [↗](https://doi.org/10.1162/neco.1997.9.8.1735)
14. Minwoo Jeong, Chin-Yew Lin, and Gary Geunbae Lee. 2009. Semi-supervised speech act recognition in emails and forums. In EMNLP’09, pages 1250– 1259, Singapore. [find](https://scholar.google.com/scholar?q=Minwoo+Jeong%2C+Chin%2DYew+Lin%2C+and+Gary+Geunbae+Lee%2E+2009%2E+Semi%2Dsupervised+speech+act+recognition+in+emails+and+forums%2E+In+EMNLP%E2%80%9909%2C+pages+1250%E2%80%93+1259%2C+Singapore%2E)
15. Shafiq Joty, Giuseppe Carenini, and Chin-Yew Lin. 2011. Unsupervised Modeling of Dialog Acts in Asynchronous Conversations. In IJCAI’11, pages 1–130, Barcelona. [find](https://scholar.google.com/scholar?q=Shafiq+Joty%2C+Giuseppe+Carenini%2C+and+Chin%2DYew+Lin%2E+2011%2E+Unsupervised+Modeling+of+Dialog+Acts+in+Asynchronous+Conversations%2E+In+IJCAI%E2%80%9911%2C+pages+1%E2%80%93130%2C+Barcelona%2E)
16. Shafiq Joty and Enamul Hoque. 2016. [Speech act modeling of written asynchronous conversations with task-specific embeddings and conditional structured models](https://md.research.elamur.ai/free/en/papers/joty-2016-speech-act-modeling-written-asynchronous/). In ACL ’16, pages 1746–1756, Berlin, Germany.
17. Dan Jurafsky, Liz Shriberg, and Debra Biasca. 1997. Switchboard SWBD-DAMSL Shallow-Discourse-Function Annotation Coders Manual. Technical report, UC Boulder & SRI International. [find](https://scholar.google.com/scholar?q=Dan+Jurafsky%2C+Liz+Shriberg%2C+and+Debra+Biasca%2E+1997%2E+Switchboard+SWBD%2DDAMSL+Shallow%2DDiscourse%2DFunction+Annotation+Coders+Manual%2E+Technical+report%2C+UC+Boulder+%26+SRI+International%2E)
18. Nal Kalchbrenner and Phil Blunsom. 2013. [Recurrent convolutional neural networks for discourse compositionality](https://md.research.elamur.ai/free/en/papers/kalchbrenner-2013-recurrent-convolutional-neural-networks-discourse/). In CVSC’13, pages 119–126.
19. Hamed Khanpour, Nishitha Guntakandla, and Rodney Nielsen. 2016. [Dialogue act classification in domain-independent conversations using a deep recurrent neural network](https://md.research.elamur.ai/free/en/papers/khanpour-2016-dialogue-act-classification-domain-independent/). In COLING’16, pages 2012–2021.
20. Diederik P. Kingma and Jimmy Ba. 2014. A method for stochastic optimization. abs/1412.6980. Adam: CoRR, Ryan Kiros, Yukun Zhu, Ruslan Salakhutdinov, Richard S. Zemel, Antonio Torralba, Raquel Urtasun, and Sanja Fidler. 2015. Skip-thought vectors. In Proceedings of the 28th International Conference on Neural Information Processing Systems -Volume 2, NIPS’15, pages 3294–3302, Cambridge, MA, USA. [↗](https://arxiv.org/abs/1412.6980)
21. Harshit Kumar, Arvind Agarwal, Riddhiman Dasgupta, Sachindra Joshi, and Arun Kumar. 2018. [Dialogue act sequence labeling using hierarchical encoder with CRF](https://md.research.elamur.ai/free/en/papers/kumar-2017-dialogue-act-sequence-labeling-using/). In AAAI’18. [↗](https://doi.org/10.1609/aaai.v32i1.11701)
22. John Lafferty, Andrew K. McCallum, and Fernando Pereira. 2001. Conditional Random Fields: Probabilistic models for segmenting and labeling sequence data. In ICML. [find](https://scholar.google.com/scholar?q=John+Lafferty%2C+Andrew+K%2E+McCallum%2C+and+Fernando+Pereira%2E+2001%2E+Conditional+Random+Fields%3A+Probabilistic+models+for+segmenting+and+labeling+sequence+data%2E+In+ICML%2E)
23. Ji Young Lee and Franck Dernoncourt. 2016. [Sequential short-text classification with recurrent and convolutional neural networks](https://md.research.elamur.ai/free/en/papers/lee-2016-sequential-short-text-classification-recurrent/). In HLT’16, pages 515– 520.
24. Annie P Louis and Shay B Cohen. 2015. Conversation trees: A grammar model for topic structure in forums. In ACL ’15. [find](https://scholar.google.com/scholar?q=Annie+P+Louis+and+Shay+B+Cohen%2E+2015%2E+Conversation+trees%3A+A+grammar+model+for+topic+structure+in+forums%2E+In+ACL+%E2%80%9915%2E)
25. Tomas Mikolov, Wen-tau Yih, and Geoffrey Zweig. 2013. [Linguistic Regularities in Continuous Space Word Representations](https://md.research.elamur.ai/free/en/papers/mikolov-2013-linguistic-regularities-continuous-space-word/). In NAACL-HLT ’13, pages 746–751, Atlanta, GA, USA.
26. Tatsuro Oya and Giuseppe Carenini. 2014. [Extractive summarization and dialogue act modeling on email threads: An integrated probabilistic approach](https://md.research.elamur.ai/free/en/papers/oya-2014-extractive-summarization-dialogue-act-modeling/). In SIGDIAL’14, page 133–140, Philadelphia, PA, U.S.A.
27. Michael J. Paul. 2012. Mixed Membership Markov Models for Unsupervised Conversation Modeling. In EMNLP-CoNLL ’12, pages 94–104, Stroudsburg, PA, USA. [find](https://scholar.google.com/scholar?q=Michael+J%2E+Paul%2E+2012%2E+Mixed+Membership+Markov+Models+for+Unsupervised+Conversation+Modeling%2E+In+EMNLP%2DCoNLL+%E2%80%9912%2C+pages+94%E2%80%93104%2C+Stroudsburg%2C+PA%2C+USA%2E)
28. Jeffrey Pennington, Richard Socher, and Christopher Manning. 2014. [Glove: Global vectors for word representation](https://md.research.elamur.ai/free/en/papers/pennington-2014-glove-global-vectors-word-representation/). In EMNLP’14, pages 1532–1543, Doha, Qatar.
29. Ashequl Qadir and Ellen Riloff. 2011. [Classifying sentences as speech acts in message board posts](https://md.research.elamur.ai/free/en/papers/qadir-2011-classifying-sentences-as-speech-acts/). In EMNLP’11, pages 748–758, Edinburgh, Scotland, UK.
30. A. Ritter, C. Cherry, and B. Dolan. 2010. [Unsupervised modeling of twitter conversations](https://md.research.elamur.ai/free/en/papers/ritter-2010-unsupervised-modeling-twitter-conversations/). In HLT: NAACL’10, LA, California.
31. Iulian V. Serban, Alessandro Sordoni, Yoshua Bengio, Aaron Courville, and Joelle Pineau. 2016. [Building end-to-end dialogue systems using generative hierarchical neural network models](https://md.research.elamur.ai/free/en/papers/serban-2015-building-end-end-dialogue-systems/). In AAAI’16. [↗](https://doi.org/10.1609/aaai.v30i1.9883)
32. Maryam Tavafi, Yashar Mehdad, Shafiq Joty, Giuseppe Carenini, and Raymond Ng. 2013. [Dialogue act recognition in synchronous and asynchronous conversations](https://md.research.elamur.ai/free/en/papers/tavafi-2013-dialogue-act-recognition-synchronous-asynchronous/). In SIGDIAL’13, Metz, France. [↗](https://doi.org/10.14288/1.0052186)
33. Jan Ulrich, Gabriel Murray, and Giuseppe Carenini. 2008. A publicly available annotated corpus for supervised email summarization. In AAAI’08, Chicago, USA. [find](https://scholar.google.com/scholar?q=Jan+Ulrich%2C+Gabriel+Murray%2C+and+Giuseppe+Carenini%2E+2008%2E+A+publicly+available+annotated+corpus+for+supervised+email+summarization%2E+In+AAAI%E2%80%9908%2C+Chicago%2C+USA%2E)
34. Yftah Ziser and Roi Reichart. 2017. Neural structural correspondence learning for domain adaptation. In CoNLL 2017, pages 400–410. [find](https://scholar.google.com/scholar?q=Yftah+Ziser+and+Roi+Reichart%2E+2017%2E+Neural+structural+correspondence+learning+for+domain+adaptation%2E+In+CoNLL+2017%2C+pages+400%E2%80%93410%2E)

*Updated 24 October 2026*