---
title: "Combining Acoustic and Pragmatic Features to — Gabsdil 2004"
description: "Spoken dialogue systems must decide whether to accept, reject, or ignore each incoming recognition hypothesis, and sometimes whether to ask for…"
authors: ["Malte Gabsdil", "Oliver Lemon"]
year: 2004
venue: "Proceedings of the 42nd Annual Meeting of the Association for Computational Linguistics (ACL-04)"
source: "https://aclanthology.org/P04-1044/"
doi: "10.3115/1218955.1218999"
licence: "CC BY-NC-SA 3.0 (ACL Anthology, before 2016)"
licence_url: "https://creativecommons.org/licenses/by-nc-sa/3.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/papers/gabsdil-2004-combining-acoustic-pragmatic-features-predict/"
---
[Elamur Research](https://md.research.elamur.ai/) · [this page in HTML](https://research.elamur.ai/free/papers/gabsdil-2004-combining-acoustic-pragmatic-features-predict/)

[home](https://md.research.elamur.ai/) / [papers](https://md.research.elamur.ai/free/papers/) / [2004](https://md.research.elamur.ai/free/papers/2004/) / Combining Acoustic and Pragmatic Features to Predict Recognition Performance in Spoken Dialogue Systems

# Combining Acoustic and Pragmatic Features to Predict Recognition Performance in Spoken Dialogue Systems

**Authors:** Malte Gabsdil, Oliver Lemon · **Year:** 2004 · **Venue:** Proceedings of the 42nd Annual Meeting of the Association for Computational Linguistics (ACL-04) · **Licence:** [CC BY-NC-SA 3.0 (ACL Anthology, before 2016)](https://creativecommons.org/licenses/by-nc-sa/3.0/)

**Links:** [Publisher's page](https://aclanthology.org/P04-1044/) · [PDF](https://aclanthology.org/P04-1044.pdf) · [DOI 10.3115/1218955.1218999](https://doi.org/10.3115/1218955.1218999) · [OpenAlex](https://openalex.org/W2159250980)

**Citations:** 59 citations (openalex-snapshot · 2026-10-14) · 15 in our library, of the 140673 papers whose bibliographies we read

## Summary
Spoken dialogue systems must decide whether to accept, reject, or ignore each incoming recognition hypothesis, and sometimes whether to ask for clarification. Wrong choices either cause misunderstandings or make the system overly cautious. Gabsdil and Lemon tackle this with machine learning. Their classifiers combine acoustic confidence scores with pragmatic plausibility features drawn from dialogue context. They apply these classifiers to n-best hypotheses rather than only the top one, then pick a best candidate.

The data come from a user study with the WITAS multimodal system for commanding a simulated UAV, which uses the Information State Update (ISU) approach. The authors compare TiMBL and RIPPER against a grammar-switching baseline that relies on fixed rejection thresholds. With optimized parameters, TiMBL raises weighted f-score by about 25% over the baseline. This result comes close to an oracle upper bound.

The work matters because it shows that context-derived features can substantially improve how a dialogue system reacts to user input. The authors also argue the method should carry over to other ISU-based systems.

## Key points
- **Data:** 6 subjects completed 5 tasks each with the WITAS system (Nuance 8.0 recognizer, Gemini grammar), giving 30 dialogues and 303 utterances. Each utterance was labelled in-grammar, out-of-grammar, or crosstalk.
- **Baseline:** the grammar-switching system reached 65.68% accuracy and 61.81% weighted f-score. Its main weakness was accepting too many utterances.
- **Best result:** TiMBL with optimized parameters, using recognition, utterance, dialogue, and task features (REC+UTT+DIAL+TASK), on 10-best output:
  - 86.14% accuracy and 86.39% weighted f-score for 3 classes;
  - 82.51% accuracy and 83.29% weighted f-score for 4 classes (including clarify).
- **Upper bound:** the oracle scored about 94%, because the 10-best lists of 18 in-grammar utterances contained no correct logical form. A χ² test found no significant difference between the oracle and optimized TiMBL.
- **Tuning effect:** parameter optimization added roughly 9% over default TiMBL settings, and all learned classifiers significantly outperformed the baseline.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

Fields: [Linguistics & languages](https://md.research.elamur.ai/free/shelves/linguistics/)

## Cite

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{gabsdil2004combining,
  title = {{Combining Acoustic and Pragmatic Features to Predict Recognition Performance in Spoken Dialogue Systems}},
  author = {Gabsdil, Malte and Lemon, Oliver},
  booktitle = {Proceedings of the 42nd Annual Meeting of the Association for Computational Linguistics (ACL-04)},
  year = {2004},
  doi = {10.3115/1218955.1218999},
  url = {https://doi.org/10.3115/1218955.1218999}
}
```

- [← previous Chapter 3: Minimalist assumptions on negation in Biblical Hebrew syntax](https://md.research.elamur.ai/free/papers/hbo-2004-snyman-chapter-3-minimalist-assumptions-negation/)
- [next → Coreference Resolution for Information Extraction](https://md.research.elamur.ai/free/papers/zelenko-2004-coreference-resolution-information-extraction/)
- [↑ back to papers of 2004](https://md.research.elamur.ai/free/papers/2004/)

## References

15 references, 0 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. M. Boros, W. Eckert, F. Gallwitz, G. Görz, G. Han-rieder, and H. Niemann. 1996. Towards Understanding Spontaneous Speech: Word Accuracy vs. Concept Accuracy. In Proc. ICSLP-96. [find](https://scholar.google.com/scholar?q=M%2E+Boros%2C+W%2E+Eckert%2C+F%2E+Gallwitz%2C+G%2E+Go%CC%88rz%2C+G%2E+Han%2Drieder%2C+and+H%2E+Niemann%2E+1996%2E+Towards+Understanding+Spontaneous+Speech%3A+Word+Accuracy+vs%2E+Concept+Accuracy%2E+In+Proc%2E+ICSLP%2D96%2E)
2. Ananlada Chotimongkol and Alexander I. Rudnicky. 2001. N-best Speech Hypotheses Reordering Using Linear Regression. In Proceedings of EuroSpeech 2001, pages 1829–1832. [find](https://scholar.google.com/scholar?q=Ananlada+Chotimongkol+and+Alexander+I%2E+Rudnicky%2E+2001%2E+N%2Dbest+Speech+Hypotheses+Reordering+Using+Linear+Regression%2E+In+Proceedings+of+EuroSpeech+2001%2C+pages+1829%E2%80%931832%2E)
3. William W. Cohen. 1995. Fast Effective Rule Induction. In Proceedings of the 12th International Conference on Machine Learning. [find](https://scholar.google.com/scholar?q=William+W%2E+Cohen%2E+1995%2E+Fast+Effective+Rule+Induction%2E+In+Proceedings+of+the+12th+International+Conference+on+Machine+Learning%2E)
4. EC FP6 IST-507802, [http://www.talk-project.org](http://www.talk-project.org) Walter Daelemans and Véronique Hoste. 2002. Evaluation of Machine Learning Methods for Natural Language Processing Tasks. In Proceedings of LREC-02. [↗](http://www.talk-project.org)
5. Walter Daelemans, Jakub Zavrel, Ko van der Sloot, and Antal van den Bosch. 2002. TIMBL: Tilburg Memory Based Learner, version 4.2, Reference Guide. In ILK Technical Report 02-01. [find](https://scholar.google.com/scholar?q=Walter+Daelemans%2C+Jakub+Zavrel%2C+Ko+van+der+Sloot%2C+and+Antal+van+den+Bosch%2E+2002%2E+TIMBL%3A+Tilburg+Memory+Based+Learner%2C+version+4%2E2%2C+Reference+Guide%2E+In+ILK+Technical+Report+02%2D01%2E)
6. John Dowding, Jean Mark Gawron, Doug Appelt, John Bear, Lynn Cherny, Robert Moore, and Douglas Moran. 1993. GEMINI: a natural language system for spoken-language understanding. In Proceedings of ACL-93. [find](https://scholar.google.com/scholar?q=John+Dowding%2C+Jean+Mark+Gawron%2C+Doug+Appelt%2C+John+Bear%2C+Lynn+Cherny%2C+Robert+Moore%2C+and+Douglas+Moran%2E+1993%2E+GEMINI%3A+a+natural+language+system+for+spoken%2Dlanguage+understanding%2E+In+Proceedings+of+ACL%2D9)
7. Malte Gabsdil. 2003. Classifying Recognition Results for Spoken Dialogue Systems. In Proceedings of the Student Research Workshop at ACL-03. [find](https://scholar.google.com/scholar?q=Malte+Gabsdil%2E+2003%2E+Classifying+Recognition+Results+for+Spoken+Dialogue+Systems%2E+In+Proceedings+of+the+Student+Research+Workshop+at+ACL%2D03%2E)
8. Perry R. Hinton. 1995. Statistics Explained – A Guide For Social Science Students. Routledge. [find](https://scholar.google.com/scholar?q=Perry+R%2E+Hinton%2E+1995%2E+Statistics+Explained+%E2%80%93+A+Guide+For+Social+Science+Students%2E+Routledge%2E)
9. Oliver Lemon and Alexander Gruenstein. 2004. Multithreaded context for robust conversational interfaces: context-sensitive speech recognition and interpretation of corrective fragments. ACM Transactions on Computer-Human Interaction. (to appear). [find](https://scholar.google.com/scholar?q=Oliver+Lemon+and+Alexander+Gruenstein%2E+2004%2E+Multithreaded+context+for+robust+conversational+interfaces%3A+context%2Dsensitive+speech+recognition+and+interpretation+of+corrective+fragments%2E+ACM+Transactio)
10. Oliver Lemon, Alexander Gruenstein, and Stanley Peters. 2002. Collaborative activities and multi-tasking in dialogue systems. Traitement Automatique des Langues, 43(2):131–154. [find](https://scholar.google.com/scholar?q=Oliver+Lemon%2C+Alexander+Gruenstein%2C+and+Stanley+Peters%2E+2002%2E+Collaborative+activities+and+multi%2Dtasking+in+dialogue+systems%2E+Traitement+Automatique+des+Langues%2C+43%282%29%3A131%E2%80%93154%2E)
11. Oliver Lemon. 2004. Context-sensitive speech recognition in ISU dialogue systems: results for the grammar switching approach. In Proceedings of the 8th Workshop on the Semantics and Pragmatics of Dialogue, CATALOG’04. [find](https://scholar.google.com/scholar?q=Oliver+Lemon%2E+2004%2E+Context%2Dsensitive+speech+recognition+in+ISU+dialogue+systems%3A+results+for+the+grammar+switching+approach%2E+In+Proceedings+of+the+8th+Workshop+on+the+Semantics+and+Pragmatics+of+Dial)
12. Diane J. Litman, Julia Hirschberg, and Marc Swerts. 2000. Predicting Automatic Speech Recognition Performance Using Prosodic Cues. In Proceedings of NAACL-00. [find](https://scholar.google.com/scholar?q=Diane+J%2E+Litman%2C+Julia+Hirschberg%2C+and+Marc+Swerts%2E+2000%2E+Predicting+Automatic+Speech+Recognition+Performance+Using+Prosodic+Cues%2E+In+Proceedings+of+NAACL%2D00%2E)
13. Erwin Marsi, Martin Reynaert, Antal van den Bosch, Walter Daelemans, and Véronique Hoste. 2003. Learning to predict pitch accents and prosodic boundaries in Dutch. In Proceedings of ACL-03. [find](https://scholar.google.com/scholar?q=Erwin+Marsi%2C+Martin+Reynaert%2C+Antal+van+den+Bosch%2C+Walter+Daelemans%2C+and+Ve%CC%81ronique+Hoste%2E+2003%2E+Learning+to+predict+pitch+accents+and+prosodic+boundaries+in+Dutch%2E+In+Proceedings+of+ACL%2D03%2E)
14. David Traum, Johan Bos, Robin Cooper, Staffan Larsson, Ian Lewin, Colin Matheson, and Massimo Poesio. 1999. A Model of Dialogue Moves and Information State Revision. Technical Report D2.1, Trindi Project. [find](https://scholar.google.com/scholar?q=David+Traum%2C+Johan+Bos%2C+Robin+Cooper%2C+Staffan+Larsson%2C+Ian+Lewin%2C+Colin+Matheson%2C+and+Massimo+Poesio%2E+1999%2E+A+Model+of+Dialogue+Moves+and+Information+State+Revision%2E+Technical+Report+D2%2E1%2C+Trindi+Proj)
15. Marilyn Walker, Jerry Wright, and Irene Langkilde. 2000. Using Natural Language Processing and Discourse Features to Identify Understanding Errors in a Spoken Dialogue System. In Proceedings of ICML-2000. [find](https://scholar.google.com/scholar?q=Marilyn+Walker%2C+Jerry+Wright%2C+and+Irene+Langkilde%2E+2000%2E+Using+Natural+Language+Processing+and+Discourse+Features+to+Identify+Understanding+Errors+in+a+Spoken+Dialogue+System%2E+In+Proceedings+of+ICML%2D2)