---
title: "Collaborative Annotation of Dialogue Acts: Application of a New"
description: "The paper examines whether the then-new ISO DIS 24617-2 standard for dialogue act annotation can be applied to real… 11 references."
authors: ["Alex C. Fang", "Harry Bunt", "Jing Cao", "Xiaoyue Liu"]
year: 2012
venue: "Proceedings of the Workshop on Innovative Hybrid Approaches to the Processing of Textual Data"
source: "https://aclanthology.org/W12-0509/"
licence: "CC BY-NC-SA 3.0 (ACL Anthology, before 2016)"
licence_url: "https://creativecommons.org/licenses/by-nc-sa/3.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/fang-2012-collaborative-annotation-dialogue-acts-application/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/fang-2012-collaborative-annotation-dialogue-acts-application/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2012](https://md.research.elamur.ai/free/en/papers/2012/page/2/) / Collaborative Annotation of Dialogue Acts: Application of a New ISO Standard to the Switchboard Corpus

# Collaborative Annotation of Dialogue Acts: Application of a New ISO Standard to the Switchboard Corpus

**Authors:** [Alex C. Fang](https://md.research.elamur.ai/free/en/authors/alex-chengyu-fang/), [Harry Bunt](https://md.research.elamur.ai/free/en/authors/harry-bunt/), Jing Cao and Xiaoyue Liu · **Year:** 2012 · **Venue:** Proceedings of the Workshop on Innovative Hybrid Approaches to the Processing of Textual Data · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY-NC-SA 3.0 (ACL Anthology, before 2016)](https://creativecommons.org/licenses/by-nc-sa/3.0/)

**Links:** [Publisher's page](https://aclanthology.org/W12-0509/) · [PDF](https://aclanthology.org/W12-0509.pdf) · [OpenAlex](https://openalex.org/W56941735)

**Citations:** 3 citations (OpenAlex, 2026-10-14)

## Summary
The paper examines whether the then-new ISO DIS 24617-2 standard for dialogue act annotation can be applied to real transcribed conversation. The authors convert the Switchboard Dialogue Act (SWBD-DA) Corpus from its original SWBD-DAMSL scheme to ISO tags. SWBD-DAMSL assigns a single label to each slash-unit. The ISO scheme instead tags "functional segments" and allows several acts per segment, each placed in a dimension such as turn management or feedback.

The authors grouped the 303 SWBD-DAMSL tags into 60 clusters and mapped most of them to ISO tags automatically. To check the result, they trained classifiers on lexical features. On the ISO-mapped labels, these reached a weighted F1 of about 0.83.

Four tags cannot be converted mechanically because their ISO equivalent depends on context. For these, the authors defined rules keyed on the preceding act and built a graphical tool for manual mapping. The work matters because it moves a widely used dialogue corpus toward an interoperable standard and enables direct comparison of the two schemes.

## Key points
- **Corpus scale:** SWBD-DA contains 1,155 five-minute conversations, about 1.5M word tokens and 223,606 slash-units. All conversations were used, not only the 1,115 standard training dialogues.
- **Tag clustering and skew:** 303 distinct tags were found and grouped into 60 functions. Statement-non-opinion alone covers 51.79% of tokens, and the top 10 types cover 93.38%.
- **ISO DIS 24617-2 design:**
  - It defines nine dimensions: Task, Auto-/Allo-Feedback, Turn, Time, Discourse Structuring, Social Obligations, Own and Partner Communication Management.
  - Functional segments may be discontinuous or overlapping.
  - Tags are stored as stand-off annotation.
- **Classification on ISO labels (task 2):**
  - Weighted F1 was 0.831, more than 10 points above task 1.
  - Inform reached F1 0.93 and Answer 0.775. InitialGoodbye had the highest precision (0.972).
  - Four rare classes scored F1 = 0.
- **Manual mapping:**
  - Four tags need manual treatment: action-directive (ad), other answers (no), and the accept (aa) and reject (ar) tags.
  - A GUI shows adjacent turns, the previous five turns, or the whole history.
  - Three annotators will each label 115 random files, and agreement will be measured with kappa.

## Interesting ideas
- The meaning of a reactive act like "accept" depends on the act it answers: accepting an offer, a thanking or an apology are different ISO acts. Mapping the act by looking back at the previous act resolves this deterministically.
- One utterance can carry several acts in separate dimensions, such as giving the turn while also asking a question. Storing these acts as parallel stand-off layers avoids forcing a single label.
- The conversion is a hybrid. Rules handle the bulk of tags automatically, and humans handle only the context-dependent residue, through a tool that shows exactly the needed context window.
- Training a classifier on the converted labels serves as a sanity check: if the new tags are learnable from simple features, the mapping is probably coherent.
- The authors flag that the most frequent tags include non-communicative labels such as "segment" or "abandoned". This exposes cases where an annotation scheme mixes form-level bookkeeping with function.

## For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

- **I1** (frames): prag could label replies like "okay"/"yeah" by first classifying the antecedent act, giving an explainable rule rather than a lexical guess.
- **I2** (frames): speech-act output in prag could be a set of (dimension, function) pairs per segment, so turn-taking and social moves sit beside the main task act.
- **I3** (learn): when converting treebanks or silver data, route only the ambiguous categories to an LLM teacher or human, with a minimal context window attached.
- **I4** (learn): after any UD dialect or label conversion, train a cheap model on the result and inspect per-class F1, especially zero-F1 classes, as a signal of broken mappings.
- **I5** (world): keep structural markers like abandoned or quoted material out of the event/state layer, so the world of a dialogue only receives genuine communicative acts.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Tilburg University](https://md.research.elamur.ai/free/en/universities/tilburg-university/) — University, Tilburg, Netherlands · 40 works · 6,231 citations
- [City University of Hong Kong](https://md.research.elamur.ai/free/en/universities/city-university-of-hong-kong/) — University, Hong Kong, China · 15 works · 2,314 citations

Also on the byline: [Zhongnan University of Economics and Law](https://ror.org/04yqxxq63).

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 9,086 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 8,889 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 9,296 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 9,206 works

🧭 Topics (2): [🗨️ Dialogue systems](https://md.research.elamur.ai/free/en/topics/dialogue-systems/) · 401 works · [💬 Speech act theory](https://md.research.elamur.ai/free/en/topics/speech-act-theory/) · 160 works

## Cite

Cite as: Alex C. Fang, Harry Bunt, Jing Cao and Xiaoyue Liu. 2012. Collaborative Annotation of Dialogue Acts: Application of a New ISO Standard to the Switchboard Corpus. Proceedings of the Workshop on Innovative Hybrid Approaches to the Processing of Textual Data.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{fang2012collaborative,
  title = {{Collaborative Annotation of Dialogue Acts: Application of a New ISO Standard to the Switchboard Corpus}},
  author = {Alex C. Fang, Harry Bunt, Jing Cao and Xiaoyue Liu},
  booktitle = {Proceedings of the Workshop on Innovative Hybrid Approaches to the Processing of Textual Data},
  year = {2012},
  url = {https://aclanthology.org/W12-0509/}
}
```

- [← previous Automatic Extraction and Evaluation of Arabic LFG Resources](https://md.research.elamur.ai/free/en/papers/attia-2012-automatic-extraction-evaluation-arabic-lfg/)
- [next → Event Nominals: Annotation Guidelines and a Manually Annotated Corpus in French](https://md.research.elamur.ai/free/en/papers/arnulphy-2012-event-nominals-annotation-guidelines-manually/)
- [↑ back to papers of 2012](https://md.research.elamur.ai/free/en/papers/2012/page/2/)

## References

11 references, 3 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Bunt, H. (2009). Multifunctionality and multidimensional dialogue semantics. In Proceedings of DiaHolmia Workshop on the Semantics and Pragmatics of Dialogue, Stockholm, 2009. [find](https://scholar.google.com/scholar?q=Bunt%2C+H%2E+%282009%29%2E+Multifunctionality+and+multidimensional+dialogue+semantics%2E+In+Proceedings+of+DiaHolmia+Workshop+on+the+Semantics+and+Pragmatics+of+Dialogue%2C+Stockholm%2C+2009%2E)
2. Bunt, H. (2011). Multifunctionality in dialogue and its interpretation. Computer, Speech and Language, 25 (2), pp. 225--245. [find](https://scholar.google.com/scholar?q=Bunt%2C+H%2E+%282011%29%2E+Multifunctionality+in+dialogue+and+its+interpretation%2E+Computer%2C+Speech+and+Language%2C+25+%282%29%2C+pp%2E+225%2D%2D245%2E)
3. Bunt, H., Alexandersson, J., Carletta, J., Choe, J.-W., Fang, A.C., Hasida, K., Lee, K., Petukhova, V., Popescu-Belis, A., Romary, L., Soria, C. and Traum, D. (2010). [Towards an ISO standard for dialogue act annotation](https://md.research.elamur.ai/free/en/papers/bunt-2010-iso-standard-dialogue-act-annotation/). In Proceedings of the Seventh International Conference on Language Resources and Evaluation. Valletta, MALTA, 17-23 May 2010.
4. Hall, M., Frank, E., Holmes, G., Pfahringer, B., Reutemann, P. and Witten, I. H. (2009). The WEKA data mining software: an update. SIGKDD Explorations, 11 (1), pp. 10--18. [↗](https://doi.org/10.1145/1656274.1656278)
5. ISO DIS 24617-2. (2010). Language resource management – Semantic annotation framework (SemAF), Part 2: Dialogue acts. [find](https://scholar.google.com/scholar?q=ISO+DIS+24617%2D2%2E+%282010%29%2E+Language+resource+management+%E2%80%93+Semantic+annotation+framework+%28SemAF%29%2C+Part+2%3A+Dialogue+acts%2E)
6. ISO, Geneva, January 2010. [find](https://scholar.google.com/scholar?q=ISO%2C+Geneva%2C+January+2010%2E)
7. Jurafsky, D., Shriberg, E. and Biasca, D. (1997). Switchboard SWBD-DAMSL shallow-discourse-function annotation coders manual, Draft 13. University of Colorado, Boulder Institute of Cognitive Science Technical Report 97-02. [find](https://scholar.google.com/scholar?q=Jurafsky%2C+D%2E%2C+Shriberg%2C+E%2E+and+Biasca%2C+D%2E+%281997%29%2E+Switchboard+SWBD%2DDAMSL+shallow%2Ddiscourse%2Dfunction+annotation+coders+manual%2C+Draft+13%2E+University+of+Colorado%2C+Boulder+Institute+of+Cognitive+Science+T)
8. Jurafsky, D., Bates, R., Coccaro, N., Martin, R., Meteer, M., Ries, K., Shriberg, E., Stolcke, A., Taylor, P. and Ess-Dykema, C. V. (1998a). Switchbaod Discourse Language Modeling Project and Report. Research Note 30, Center for Language and Speech Processing, Johns Hopkins University, Baltimore, MD, January. [find](https://scholar.google.com/scholar?q=Jurafsky%2C+D%2E%2C+Bates%2C+R%2E%2C+Coccaro%2C+N%2E%2C+Martin%2C+R%2E%2C+Meteer%2C+M%2E%2C+Ries%2C+K%2E%2C+Shriberg%2C+E%2E%2C+Stolcke%2C+A%2E%2C+Taylor%2C+P%2E+and+Ess%2DDykema%2C+C%2E+V%2E+%281998a%29%2E+Switchbaod+Discourse+Language+Modeling+Project+and+Report%2E+)
9. Jurafsky, D., Shriberg, E., Fox B. and Curl, T. (1998b). [Lexical, prosodic, and syntactic cues for dialog acts](https://md.research.elamur.ai/free/en/papers/jurafsky-1998-lexical-prosodic-syntactic-cues-dialog/). ACL/COLING-98 Workshop on Discourse Relations and Discourse Markers.
10. Meeter, M., Taylor, A. (1995). Dysfluency annotation stylebook for the Switchboard Corpus. Available at ftp://ftp.cis.upenn.edu/pub/treebank/swbd/doc/DFL-bo ok.ps. [find](https://scholar.google.com/scholar?q=Meeter%2C+M%2E%2C+Taylor%2C+A%2E+%281995%29%2E+Dysfluency+annotation+stylebook+for+the+Switchboard+Corpus%2E+Available+at+ftp%3A%2F%2Fftp%2Ecis%2Eupenn%2Eedu%2Fpub%2Ftreebank%2Fswbd%2Fdoc%2FDFL%2Dbo+ok%2Eps%2E)
11. Stolcke, A., Ries, K., Coccaro, N., Shriberg, E., Bates, R., Jurfsky, D., Taylor, P., Martin, R., Ess-Dykema, C.V. and Meteer, M. (2000). [Dialogue Act Modeling for Automatic Tagging and Recognition of Conversational Speech](https://md.research.elamur.ai/free/en/papers/stolcke-2000-dialogue-act-modeling-automatic-tagging/). Computational Linguistics, 26 (3), pp. 339--373. [↗](https://doi.org/10.1162/089120100561737)

*Updated 24 October 2026*