---
title: "A Study on Dialog Act Recognition using Character-Level"
description: "The authors argue that affixes, punctuation and similar sub-word cues also signal intention. 0.7 points. 27 references, cited by 2 papers here."
authors: ["Eugenio Ribeiro; Ricardo Ribeiro;", "David Martins de Matos"]
year: 2018
venue: "arXiv:1805.07231"
source: "https://arxiv.org/abs/1805.07231"
doi: "10.48550/arxiv.1805.07231"
arxiv: "1805.07231"
licence: "arXiv non-exclusive distribution licence (read only, no redistribution)"
licence_url: "http://arxiv.org/licenses/nonexclusive-distrib/1.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/ribeiro-2018-study-dialog-act-recognition-using/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/ribeiro-2018-study-dialog-act-recognition-using/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2018](https://md.research.elamur.ai/free/en/papers/2018/page/6/) / A Study on Dialog Act Recognition using Character-Level Tokenization

# A Study on Dialog Act Recognition using Character-Level Tokenization

**Authors:** [Eugenio Ribeiro; Ricardo Ribeiro;](https://md.research.elamur.ai/free/en/authors/eugenio-ribeiro/) and [David Martins de Matos](https://md.research.elamur.ai/free/en/authors/david-martins-de-matos/) · **Year:** 2018 · **Venue:** [arXiv:1805.07231](https://md.research.elamur.ai/free/en/venues/lecture-notes-in-computer-science/) ([Lecture notes in computer science 2018](https://md.research.elamur.ai/free/en/venues/lecture-notes-in-computer-science/2018/)) · **Publisher:** [Springer Science+Business Media](https://md.research.elamur.ai/free/en/publishers/springer-science-business-media/) · **Licence:** [arXiv non-exclusive distribution licence (read only, no redistribution)](http://arxiv.org/licenses/nonexclusive-distrib/1.0/)

**Links:** [Publisher's page](https://arxiv.org/abs/1805.07231) · [PDF](https://arxiv.org/pdf/1805.07231) · [DOI 10.48550/arxiv.1805.07231](https://doi.org/10.48550/arxiv.1805.07231) · [arXiv 1805.07231](https://arxiv.org/abs/1805.07231) · [OpenAlex](https://openalex.org/W2804771341)

**Citations:** 3 citations (OpenAlex, 2026-10-14) · [2](https://md.research.elamur.ai/free/en/papers/ribeiro-2018-study-dialog-act-recognition-using/cited-by/) in our library, of the 141,840 papers whose bibliographies we read

## Summary
The paper asks whether dialog act recognition, which identifies the intention behind an utterance, gains from looking below the word level. Most neural systems treat a segment as a sequence of words. The authors argue that affixes, punctuation and similar sub-word cues also signal intention. They took the word-level CNN architecture of Liu et al., which runs parallel convolutions with different window widths, applies max pooling, reduces dimensionality and ends in a softmax, and fed it characters instead of words. Experiments covered two contrasting corpora: Switchboard (English, open-domain human-human talk) and DIHANA (Spanish, users talking to a Wizard of Oz train-information system). They varied the window sizes, and tested capitalization, punctuation and lemmatization. Character-level models matched or beat the word-level baselines. Adding word-level representations improved results further, which shows the two views carry complementary information. With context features included, the combined model exceeded the earlier state of the art on Switchboard and came close to perfect on DIHANA. This suggests sub-word cues for intention hold across languages and domains.

## Key points
- Data: Switchboard has 1,155 dialogs, 223,606 segments and 42 labels. DIHANA has 900 dialogs and 23,542 segments, scored on the 11 domain-independent Level 1 tags, of which only 5 are shared by user and system turns.
- The best character setup used three parallel windows of 3, 5 and 7 characters. It reached .7208 accuracy on the Switchboard test set and .9244 on DIHANA, beating any single window.
- Keeping punctuation added about 1 percentage point (Switchboard test .7317). Lemmatizing the input lowered accuracy (.7140), but only slightly.
- Capitalization made no difference on Switchboard. On DIHANA it helped (.9425), because system turns lack mid-segment capitals, so capitalization revealed which side was speaking.
- Combining character and word representations gave .7401 on Switchboard and .9568 on DIHANA. Adding context raised this to .7901 and .9910, about 0.7 points above the prior Switchboard best.

## Interesting ideas
- Running several character windows of different widths in parallel lets one model pick up short affixes, whole stems and cross-word fragments at once, without choosing a single n-gram size.
- Feeding the model lemmatized input is a cheap ablation that measures how much of the signal lives in inflection rather than in the lexeme.
- When capitalization unexpectedly helped, the authors traced the gain to a transcription artifact that exposed who was speaking, rather than accepting it as linguistic signal.
- Punctuation works both as a direct cue to intention (a question mark) and as an indirect structural one (commas organizing the segment).
- Character-level and word-level representations proved complementary, so merging them beat either one alone.

## For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

- **I1** (tokens): Char windows of 3/5/7 could supply features for tagging and lemmatizing unknown or misspelled English words, with each window's contribution kept inspectable.
- **I2** (learn): The same ablation could act as a negative control when evaluating whether a parser or speech-act classifier really uses morphology.
- **I3** (learn): Evaluation could routinely check whether surprising gains come from formatting quirks of a corpus source, especially when silver data mixes treebanks with different conventions.
- **I4** (frames): The speech-act detector in prag should treat terminal and internal punctuation as explicit, explainable evidence alongside the UD tree.
- **I5** (frames): Speech-act and irony decisions could combine lexical/tree features with a small character-level cue channel, recording which channel drove each decision.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Instituto de Engenharia de Sistemas e Computadores Investigação e Desenvolvimento](https://md.research.elamur.ai/free/en/labs/instituto-de-engenharia-de-sistemas-e-computadores-investigacao-e/) — Research institute, Lisbon, Portugal · 21 works · 3,723 citations
- [Instituto Superior Técnico](https://md.research.elamur.ai/free/en/universities/instituto-superior-tecnico/) — University, Lisbon, Portugal · 28 works · 3,944 citations
- [University of Lisbon](https://md.research.elamur.ai/free/en/universities/university-of-lisbon/) — University, Lisbon, Portugal · 20 works · 3,477 citations
- [Iscte – Instituto Universitário de Lisboa](https://md.research.elamur.ai/free/en/universities/iscte-instituto-universitario-de-lisboa/) — University, Lisbon, Portugal · 5 works · 34 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 9,086 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 8,889 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 9,296 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 9,206 works

🧭 Topics (3): [🗨️ Dialogue systems](https://md.research.elamur.ai/free/en/topics/dialogue-systems/) · 401 works · [✂️ Tokenization & segmentation](https://md.research.elamur.ai/free/en/topics/tokenization/) · 77 works · [💬 Speech act theory](https://md.research.elamur.ai/free/en/topics/speech-act-theory/) · 160 works

## Cite

Cite as: Eugenio Ribeiro; Ricardo Ribeiro; and David Martins de Matos. 2018. A Study on Dialog Act Recognition using Character-Level Tokenization. arXiv:1805.07231.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@misc{ribeiro2018study,
  title = {{A Study on Dialog Act Recognition using Character-Level Tokenization}},
  author = {Eugenio Ribeiro; Ricardo Ribeiro; and David Martins {de Matos}},
  year = {2018},
  doi = {10.48550/arxiv.1805.07231},
  eprint = {1805.07231},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.1805.07231}
}
```

- [← previous Rollenwechsel-English: a large-scale semantic role corpus](https://md.research.elamur.ai/free/en/papers/sayeed-2018-rollenwechsel-english-large-scale-semantic/)
- [next → Approximate Dynamic Oracle for Dependency Parsing with Reinforcement Learning](https://md.research.elamur.ai/free/en/papers/yu-2018-approximate-dynamic-oracle-dependency-parsing/)
- [↑ back to papers of 2018](https://md.research.elamur.ai/free/en/papers/2018/page/6/)

## References

27 references, 9 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Abadi, M., et al.: TensorFlow: Large-Scale Machine Learning on Heterogeneous Systems. [https://www.tensorflow.org/](https://www.tensorflow.org/) (2015) [↗](https://www.tensorflow.org/)
2. Alcácer, N., Benedı́, J.M., Blat, F., Granell, R., Martı́nez, C.D., Torres, F.: Acquisition and Labelling of a Spontaneous Speech Dialogue Corpus. In: SPECOM. pp. 583–586 (2005) [find](https://scholar.google.com/scholar?q=Alca%CC%81cer%2C+N%2E%2C+Bened%C4%B1%CC%81%2C+J%2EM%2E%2C+Blat%2C+F%2E%2C+Granell%2C+R%2E%2C+Mart%C4%B1%CC%81nez%2C+C%2ED%2E%2C+Torres%2C+F%2E%3A+Acquisition+and+Labelling+of+a+Spontaneous+Speech+Dialogue+Corpus%2E+In%3A+SPECOM%2E+pp%2E+583%E2%80%93586+%282005%29)
3. Benedı́, J.M., Lleida, E., Varona, A., Castro, M.J., Galiano, I., Justo, R., de Let-ona, I.L., Miguel, A.: Design and Acquisition of a Telephone Spontaneous Speech Dialogue Corpus in Spanish: DIHANA. In: LREC. pp. 1636–1639 (2006) [find](https://scholar.google.com/scholar?q=Bened%C4%B1%CC%81%2C+J%2EM%2E%2C+Lleida%2C+E%2E%2C+Varona%2C+A%2E%2C+Castro%2C+M%2EJ%2E%2C+Galiano%2C+I%2E%2C+Justo%2C+R%2E%2C+de+Let%2Dona%2C+I%2EL%2E%2C+Miguel%2C+A%2E%3A+Design+and+Acquisition+of+a+Telephone+Spontaneous+Speech+Dialogue+Corpus+in+Spanish%3A+DIHANA%2E+)
4. Cardellino, C.: Spanish Billion Words Corpus and Embeddings. [http://crscardellino.me/SBWCE/](http://crscardellino.me/SBWCE/) (2016) [↗](http://crscardellino.me/SBWCE/)
5. Chollet, F., et al.: Keras: The Python Deep Learning Library. [https://keras.io/(2015)](https://keras.io/(2015)) [↗](https://keras.io/(2015))
6. Gambäck, B., Olsson, F., Täckström, O.: Active Learning for Dialogue Act Classification. In: INTERSPEECH. pp. 1329–1332 (2011) [find](https://scholar.google.com/scholar?q=Gamba%CC%88ck%2C+B%2E%2C+Olsson%2C+F%2E%2C+Ta%CC%88ckstro%CC%88m%2C+O%2E%3A+Active+Learning+for+Dialogue+Act+Classification%2E+In%3A+INTERSPEECH%2E+pp%2E+1329%E2%80%931332+%282011%29)
7. Goldberg, Y.: A primer on neural network models for natural language processing. Journal of Artificial Intelligence Research 57, 345–420 (2016) [find](https://scholar.google.com/scholar?q=Goldberg%2C+Y%2E%3A+A+primer+on+neural+network+models+for+natural+language+processing%2E+Journal+of+Artificial+Intelligence+Research+57%2C+345%E2%80%93420+%282016%29)
8. Jaech, A., Mulcaire, G., Hathi, S., Ostendorf, M., Smith, N.A.: Hierarchical Character-Word Models for Language Identification. In: International Workshop on Natural Language Processing for Social Media. pp. 84–93 (2016) [find](https://scholar.google.com/scholar?q=Jaech%2C+A%2E%2C+Mulcaire%2C+G%2E%2C+Hathi%2C+S%2E%2C+Ostendorf%2C+M%2E%2C+Smith%2C+N%2EA%2E%3A+Hierarchical+Character%2DWord+Models+for+Language+Identification%2E+In%3A+International+Workshop+on+Natural+Language+Processing+for+Social+Med)
9. Ji, Y., Haffari, G., Eisenstein, J.: A Latent Variable Recurrent Neural Network for Discourse Relation Language Models. In: NAACL-HLT. pp. 332–342 (2016) [find](https://scholar.google.com/scholar?q=Ji%2C+Y%2E%2C+Haffari%2C+G%2E%2C+Eisenstein%2C+J%2E%3A+A+Latent+Variable+Recurrent+Neural+Network+for+Discourse+Relation+Language+Models%2E+In%3A+NAACL%2DHLT%2E+pp%2E+332%E2%80%93342+%282016%29)
10. Jurafsky, D., Shriberg, E., Biasca, D.: Switchboard SWBD-DAMSL Shallow-Discourse-Function Annotation Coders Manual. Tech. Rep. Draft 13, University of Colorado, Institute of Cognitive Science (1997) [find](https://scholar.google.com/scholar?q=Jurafsky%2C+D%2E%2C+Shriberg%2C+E%2E%2C+Biasca%2C+D%2E%3A+Switchboard+SWBD%2DDAMSL+Shallow%2DDiscourse%2DFunction+Annotation+Coders+Manual%2E+Tech%2E+Rep%2E+Draft+13%2C+University+of+Colorado%2C+Institute+of+Cognitive+Science+%281997%29)
11. Kalchbrenner, N., Blunsom, P.: [Recurrent Convolutional Neural Networks for Discourse Compositionality](https://md.research.elamur.ai/free/en/papers/kalchbrenner-2013-recurrent-convolutional-neural-networks-discourse/). In: Workshop on Continuous Vector Space Models and their Compositionality. pp. 119–126 (2013)
12. Khanpour, H., Guntakandla, N., Nielsen, R.: [Dialogue Act Classification in Domain-Independent Conversations Using a Deep Recurrent Neural Network](https://md.research.elamur.ai/free/en/papers/khanpour-2016-dialogue-act-classification-domain-independent/). In: COLING. pp. 2012–2021 (2016)
13. Král, P., Cerisara, C.: Dialogue Act Recognition Approaches. Computing and Informatics 29(2), 227–250 (2010) [find](https://scholar.google.com/scholar?q=Kra%CC%81l%2C+P%2E%2C+Cerisara%2C+C%2E%3A+Dialogue+Act+Recognition+Approaches%2E+Computing+and+Informatics+29%282%29%2C+227%E2%80%93250+%282010%29)
14. Lee, J.Y., Dernoncourt, F.: [Sequential Short-Text Classification with Recurrent and Convolutional Neural Networks](https://md.research.elamur.ai/free/en/papers/lee-2016-sequential-short-text-classification-recurrent/). In: NAACL-HLT. pp. 515–520 (2016)
15. Liu, Y., Han, K., Tan, Z., Lei, Y.: [Using Context Information for Dialog Act Classification in DNN Framework](https://md.research.elamur.ai/free/en/papers/liu-2017-context-information-dialog-act-classification/). In: EMNLP. pp. 2160–2168 (2017)
16. Manning, C.D.: Computational Linguistics and Deep Learning. Computational Linguistics 41(4), 701–707 (2015) [find](https://scholar.google.com/scholar?q=Manning%2C+C%2ED%2E%3A+Computational+Linguistics+and+Deep+Learning%2E+Computational+Linguistics+41%284%29%2C+701%E2%80%93707+%282015%29)
17. Martı́nez-Hinarejos, C.D., Benedı́, J.M., Granell, R.: Statistical Framework for a Spanish Spoken Dialogue Corpus. Speech Communication 50(11–12), 992–1008 (2008) [find](https://scholar.google.com/scholar?q=Mart%C4%B1%CC%81nez%2DHinarejos%2C+C%2ED%2E%2C+Bened%C4%B1%CC%81%2C+J%2EM%2E%2C+Granell%2C+R%2E%3A+Statistical+Framework+for+a+Spanish+Spoken+Dialogue+Corpus%2E+Speech+Communication+50%2811%E2%80%9312%29%2C+992%E2%80%931008+%282008%29)
18. Martı́nez-Hinarejos, C.D., Sanchis, E., Garcı́a-Granada, F., Aibar, P.: A Labelling Proposal to Annotate Dialogues. In: LREC. vol. 5, pp. 1566–1582 (2002) [find](https://scholar.google.com/scholar?q=Mart%C4%B1%CC%81nez%2DHinarejos%2C+C%2ED%2E%2C+Sanchis%2C+E%2E%2C+Garc%C4%B1%CC%81a%2DGranada%2C+F%2E%2C+Aibar%2C+P%2E%3A+A+Labelling+Proposal+to+Annotate+Dialogues%2E+In%3A+LREC%2E+vol%2E+5%2C+pp%2E+1566%E2%80%931582+%282002%29)
19. Mikolov, T., Karafit, M., Burget, L., Cernock, J., Khudanpur, S.: Recurrent Neural Network Based Language Model. In: INTERSPEECH. pp. 1045–1048 (2010) [find](https://scholar.google.com/scholar?q=Mikolov%2C+T%2E%2C+Karafit%2C+M%2E%2C+Burget%2C+L%2E%2C+Cernock%2C+J%2E%2C+Khudanpur%2C+S%2E%3A+Recurrent+Neural+Network+Based+Language+Model%2E+In%3A+INTERSPEECH%2E+pp%2E+1045%E2%80%931048+%282010%29)
20. Mikolov, T., Sutskever, I., Chen, K., Corrado, G.S., Dean, J.: [Distributed Representations of Words and Phrases and their Compositionality](https://md.research.elamur.ai/free/en/papers/mikolov-2013-distributed-representations-words-phrases-their/). In: NIPS. pp. 3111–3119 (2013)
21. Pennington, J., Socher, R., Manning, C.D.: [GloVe: Global Vectors for Word Representation](https://md.research.elamur.ai/free/en/papers/pennington-2014-glove-global-vectors-word-representation/). In: EMNLP. pp. 1532–1543 (2014)
22. Ribeiro, E., Ribeiro, R., de Matos, D.M.: [The Influence of Context on Dialogue Act Recognition](https://md.research.elamur.ai/free/en/papers/ribeiro-2015-influence-context-dialogue-act-recognition/). CoRR abs/1506.00839 (2015), http://arxiv.org/abs/1506.00839 [↗](https://arxiv.org/abs/1506.00839)
23. Santos, C.D., Zadrozny, B.: Learning Character-level Representations for Part-of-Speech Tagging. In: ICML. pp. 1818–1826 (2014) [find](https://scholar.google.com/scholar?q=Santos%2C+C%2ED%2E%2C+Zadrozny%2C+B%2E%3A+Learning+Character%2Dlevel+Representations+for+Part%2Dof%2DSpeech+Tagging%2E+In%3A+ICML%2E+pp%2E+1818%E2%80%931826+%282014%29)
24. Searle, J.R.: Speech Acts: An Essay in the Philosophy of Language. Cambridge University Press, Cambridge, London (1969) [find](https://scholar.google.com/scholar?q=Searle%2C+J%2ER%2E%3A+Speech+Acts%3A+An+Essay+in+the+Philosophy+of+Language%2E+Cambridge+University+Press%2C+Cambridge%2C+London+%281969%29)
25. Stolcke, A., Coccaro, N., Bates, R., Taylor, P., Van Ess-Dykema, C., Ries, K., Shriberg, E., Jurafsky, D., Martin, R., Meteer, M.: [Dialogue Act Modeling for Automatic Tagging and Recognition of Conversational Speech](https://md.research.elamur.ai/free/en/papers/stolcke-2000-dialogue-act-modeling-automatic-tagging/). Computational Linguistics 26(3), 339–373 (2000)
26. Tamarit, V., Martı́nez-Hinarejos, C.D.: Dialog Act Labeling in the DIHANA Corpus using Prosody Information. In: V Jornadas en Tecnologı́a del Habla. pp. 183– 186 (2008) [find](https://scholar.google.com/scholar?q=Tamarit%2C+V%2E%2C+Mart%C4%B1%CC%81nez%2DHinarejos%2C+C%2ED%2E%3A+Dialog+Act+Labeling+in+the+DIHANA+Corpus+using+Prosody+Information%2E+In%3A+V+Jornadas+en+Tecnolog%C4%B1%CC%81a+del+Habla%2E+pp%2E+183%E2%80%93+186+%282008%29)
27. Zhang, X., Zhao, J., LeCun, Y.: [Character-level Convolutional Networks for Text Classification](https://md.research.elamur.ai/free/en/papers/zhang-2015-character-level-convolutional-networks-text/). In: NIPS. vol. 1, pp. 649–657 (2015) Character-Level Dialog Act Recognition

*Updated 24 October 2026*