---
title: "Detecting Inappropriate Clarification Requests in Spoken"
description: "Spoken dialogue systems that suspect a misrecognition ask the user for clarification. 88.5% accuracy. 6 references, cited by 2 papers here."
authors: ["Alex Liu", "Rose Sloan", "Mei-Vern Then", "Svetlana Stoyanchev", "Julia Hirschberg", "Elizabeth Shriberg"]
year: 2014
venue: "Proceedings of the 15th Annual Meeting of the Special Interest Group on Discourse and Dialogue (SIGDIAL)"
source: "https://aclanthology.org/W14-4331/"
doi: "10.3115/v1/w14-4331"
licence: "CC BY-NC-SA 3.0 (ACL Anthology, before 2016)"
licence_url: "https://creativecommons.org/licenses/by-nc-sa/3.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/liu-2014-detecting-inappropriate-clarification-requests-spoken/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/liu-2014-detecting-inappropriate-clarification-requests-spoken/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2014](https://md.research.elamur.ai/free/en/papers/2014/page/3/) / Detecting Inappropriate Clarification Requests in Spoken Dialogue Systems

# Detecting Inappropriate Clarification Requests in Spoken Dialogue Systems

**Authors:** Alex Liu, Rose Sloan, Mei-Vern Then, [Svetlana Stoyanchev](https://md.research.elamur.ai/free/en/authors/svetlana-stoyanchev/), [Julia Hirschberg](https://md.research.elamur.ai/free/en/authors/julia-hirschberg/) and [Elizabeth Shriberg](https://md.research.elamur.ai/free/en/authors/elizabeth-shriberg/) · **Year:** 2014 · **Venue:** [Proceedings of the 15th Annual Meeting of the Special Interest Group on Discourse and Dialogue (SIGDIAL)](https://md.research.elamur.ai/free/en/venues/sigdial/) ([SIGDIAL 2014](https://md.research.elamur.ai/free/en/venues/sigdial/2014/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY-NC-SA 3.0 (ACL Anthology, before 2016)](https://creativecommons.org/licenses/by-nc-sa/3.0/)

**Links:** [Publisher's page](https://aclanthology.org/W14-4331/) · [PDF](https://aclanthology.org/W14-4331.pdf) · [DOI 10.3115/v1/w14-4331](https://doi.org/10.3115/v1/w14-4331) · [OpenAlex](https://openalex.org/W22104849)

**Citations:** 6 citations (OpenAlex, 2026-10-14) · 2 in our library, of the 141,840 papers whose bibliographies we read

## Summary
Spoken dialogue systems that suspect a misrecognition ask the user for clarification. When they misjudge where the error lies or what kind it is, the question confuses the user. Examples include asking for a synonym of a proper name or demanding that a long phrase be spelled in the NATO alphabet. The authors frame spotting such flawed questions as a new classification task, decided from the user's reply rather than from the system's internal state. They built a laboratory corpus with a simulated system. Participants read sentences aloud, then answered scripted clarification requests of six types modelled on SRI's Thunder-BOLT translator. Each type had appropriate variants and deliberately flawed ones, all derived from TRANSTAC material. A Weka J48 decision tree combines lexical, POS-bigram, LIWC, numeric and request-type features. It reaches 88.5% accuracy and an F-measure of .885, well above a 63.2% majority baseline. A system that notices it has asked a bad question can switch to another strategy or a generic fallback instead of letting the dialogue go off track.

## Key points
- Corpus design:
  - 228 scripted requests (84 appropriate, 144 inappropriate), 12 for each of 19 condition categories, built from TRANSTAC data.
  - 17 native American English speakers each answered 114 requests, giving 714 responses to appropriate and 1224 to inappropriate requests (63% inappropriate).
- Request types: confirmation; rephrase-part; spell; disambiguate; targeted reprise; and targeted-reprise-rephrase. Confirmation was always treated as appropriate.
- Features: word unigrams and bigrams with frequency cutoffs, Stanford-tagger POS bigrams, LIWC 2007 categories, function-word ratio, total spelled-letter count, and the type of the preceding request.
- Results (J48, 10-fold cross-validation; J48 beat JRip and LibSVM):
  - full feature set: 88.5% accuracy, F .885;
  - request-type-only baseline: 73.7%, F .686;
  - majority baseline: 63.2%, F .489.
- Leave-one-out ablation: removing request type costs 7.6 points, LIWC 2.3, and POS bigrams 2.0. The top tree splits use "name", the collapsed SPELL token, the bigram of start marker plus "neither", a LIWC category, and request type.

## Interesting ideas
- The system's own previous move is judged by the reaction it provokes, so the user's reply becomes a diagnostic signal for an earlier mistake.
- Before tagging, spelled-out letter runs and disfluencies are collapsed into single placeholder tokens, and a handful of highly diagnostic words (negations, "word", "name") get their own POS classes; the resulting POS bigrams then carry start and end markers.
- Errors are injected by construction in a simulated system, so every item's gold label is known, and each error condition gets the same number of items.
- The paper reports a context-only baseline (request type alone) next to the full model, then runs leave-one-feature-out ablation and inspects the top tree splits.
- Error analysis shows users often silently ignore a bad question and just give the right information, so the lack of a complaint does not prove the question was fine.

## For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

- **I1** (frames): prag could recognise a correction speech act that rejects a presupposition of the prior turn (e.g. "not a name, a phrase") and flag which earlier frame or label was wrong.
- **I2** (tokens): the en tokenizer could merge letter-by-letter spellings into one token with a "spelled" feature, so the tagger and parser are not derailed by streams of single letters.
- **I3** (learn): negative-control sets could be built the same way, by deliberately corrupting a referent, role or event order, to check that evaluators and repair rules actually catch known faults.
- **I4** (learn): every evaluation could include a "prior-only" baseline plus per-component ablations, to show how much each explainable stage really contributes.
- **I5** (answer): in dialogue-style QA, a reader should not treat an uncontested earlier statement as confirmed, only as not yet challenged.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Columbia University](https://md.research.elamur.ai/free/en/universities/columbia-university/) — University, New York, United States · 149 works · 48,852 citations
- [AT&T](https://md.research.elamur.ai/free/en/companies/at-t/) — Company, Bedminster, United States · 29 works · 15,393 citations
- [Yale University](https://md.research.elamur.ai/free/en/universities/yale-university/) — University, New Haven, United States · 54 works · 11,964 citations
- [International Computer Science Institute](https://md.research.elamur.ai/free/en/labs/international-computer-science-institute/) — Research institute, Berkeley, United States · 12 works · 8,813 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 9,086 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 8,889 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 9,296 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 9,206 works

🧭 Topics (1): [🗨️ Dialogue systems](https://md.research.elamur.ai/free/en/topics/dialogue-systems/) · 401 works

## Cite

Cite as: Alex Liu, Rose Sloan, Mei-Vern Then, Svetlana Stoyanchev, Julia Hirschberg and Elizabeth Shriberg. 2014. Detecting Inappropriate Clarification Requests in Spoken Dialogue Systems. Proceedings of the 15th Annual Meeting of the Special Interest Group on Discourse and Dialogue (SIGDIAL).

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{liu2014detecting,
  title = {{Detecting Inappropriate Clarification Requests in Spoken Dialogue Systems}},
  author = {Alex Liu, Rose Sloan, Mei-Vern Then, Svetlana Stoyanchev, Julia Hirschberg and Elizabeth Shriberg},
  booktitle = {Proceedings of the 15th Annual Meeting of the Special Interest Group on Discourse and Dialogue (SIGDIAL)},
  year = {2014},
  doi = {10.3115/v1/w14-4331},
  url = {https://doi.org/10.3115/v1/w14-4331}
}
```

- [← previous Croatian Dependency Treebank 2.0: New Annotation Guidelines for Improved Parsing](https://md.research.elamur.ai/free/en/papers/agic-2014-croatian-dependency-treebank-2-0/)
- [next → Information Structure Prediction for Visual-world Referring Expressions](https://md.research.elamur.ai/free/en/papers/elsner-2014-information-structure-prediction-visual-world/)
- [↑ back to papers of 2014](https://md.research.elamur.ai/free/en/papers/2014/page/3/)

## References

6 references, 0 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. M. Akbacak et al. 2009. Recent advances in SRI’s IraqCommtm Iraqi Arabic-English speech-to-speech translation system. In ICASSP, pages 4809–4812. [find](https://scholar.google.com/scholar?q=M%2E+Akbacak+et+al%2E+2009%2E+Recent+advances+in+SRI%E2%80%99s+IraqCommtm+Iraqi+Arabic%2DEnglish+speech%2Dto%2Dspeech+translation+system%2E+In+ICASSP%2C+pages+4809%E2%80%934812%2E)
2. N. F. Ayan et al. 2013. “Can you give me another word for hyperbaric?”: Improving speech translation using targeted clarification questions. In Acoustics, Speech and Signal Processing (ICASSP), 2013 IEEE International Conference on, pages 8391–8395. IEEE. [find](https://scholar.google.com/scholar?q=N%2E+F%2E+Ayan+et+al%2E+2013%2E+%E2%80%9CCan+you+give+me+another+word+for+hyperbaric%3F%E2%80%9D%3A+Improving+speech+translation+using+targeted+clarification+questions%2E+In+Acoustics%2C+Speech+and+Signal+Processing+%28ICASSP%29%2C+2013+I)
3. D. Bohus and A. I. Rudnicky. 2005. A principled approach for rejection threshold optimization in spoken dialog systems. In INTERSPEECH, pages 2781–2784. [find](https://scholar.google.com/scholar?q=D%2E+Bohus+and+A%2E+I%2E+Rudnicky%2E+2005%2E+A+principled+approach+for+rejection+threshold+optimization+in+spoken+dialog+systems%2E+In+INTERSPEECH%2C+pages+2781%E2%80%932784%2E)
4. J. Ginzburg and R. Cooper. 2004. Clarification, ellip-sism and the nature of contextual updates. Linguistics and Philosophy, 27(3). [find](https://scholar.google.com/scholar?q=J%2E+Ginzburg+and+R%2E+Cooper%2E+2004%2E+Clarification%2C+ellip%2Dsism+and+the+nature+of+contextual+updates%2E+Linguistics+and+Philosophy%2C+27%283%29%2E)
5. J. Goldberg, M. Ostendorf, and K. Kirchhoff. 2003. The impact of response wording in error correction subdialogs. In ISCA Tutorial and Research Workshop on Error Handling in Spoken Dialogue Systems. [find](https://scholar.google.com/scholar?q=J%2E+Goldberg%2C+M%2E+Ostendorf%2C+and+K%2E+Kirchhoff%2E+2003%2E+The+impact+of+response+wording+in+error+correction+subdialogs%2E+In+ISCA+Tutorial+and+Research+Workshop+on+Error+Handling+in+Spoken+Dialogue+Systems%2E)
6. P. Lendvai, A. van den Bosch, E. Krahmer, and M. Swerts. 2002. Improving machine-learned detection of miscommunications in human-machine dialogues through informed data splitting. In Proceedings of the ESSLLI Workshop on Machine Learning Approaches in Computational Linguistics, pages 1–15. [find](https://scholar.google.com/scholar?q=P%2E+Lendvai%2C+A%2E+van+den+Bosch%2C+E%2E+Krahmer%2C+and+M%2E+Swerts%2E+2002%2E+Improving+machine%2Dlearned+detection+of+miscommunications+in+human%2Dmachine+dialogues+through+informed+data+splitting%2E+In+Proceedings+of+th)

*Updated 24 October 2026*