---
title: "Word-Based Dialog State Tracking with — Henderson 2014"
description: "Spoken dialog systems must keep a probability distribution over what the user wants, a task called dialog state tracking."
authors: ["Matthew Henderson", "Blaise Thomson", "Steve Young"]
year: 2014
venue: "Proceedings of the 15th Annual Meeting of the Special Interest Group on Discourse and Dialogue (SIGDIAL)"
source: "https://aclanthology.org/W14-4340/"
doi: "10.3115/v1/w14-4340"
licence: "CC BY-NC-SA 3.0 (ACL Anthology, before 2016)"
licence_url: "https://creativecommons.org/licenses/by-nc-sa/3.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/henderson-2014-word-based-dialog-state-tracking/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/henderson-2014-word-based-dialog-state-tracking/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2014](https://md.research.elamur.ai/free/en/papers/2014/page/2/) / Word-Based Dialog State Tracking with Recurrent Neural Networks

# Word-Based Dialog State Tracking with Recurrent Neural Networks

**Authors:** [Matthew Henderson](https://md.research.elamur.ai/free/en/authors/matthew-henderson/), [Blaise Thomson](https://md.research.elamur.ai/free/en/authors/blaise-thomson/), [Steve Young](https://md.research.elamur.ai/free/en/authors/steve-young/) · **Year:** 2014 · **Venue:** Proceedings of the 15th Annual Meeting of the Special Interest Group on Discourse and Dialogue (SIGDIAL) · **Licence:** [CC BY-NC-SA 3.0 (ACL Anthology, before 2016)](https://creativecommons.org/licenses/by-nc-sa/3.0/)

**Links:** [Publisher's page](https://aclanthology.org/W14-4340/) · [PDF](https://aclanthology.org/W14-4340.pdf) · [DOI 10.3115/v1/w14-4340](https://doi.org/10.3115/v1/w14-4340) · [OpenAlex](https://openalex.org/W2250297846)

**Citations:** 337 citations (openalex-snapshot · 2026-10-14) · [94](#cited-by) in our library, of the 140673 papers whose bibliographies we read

## Summary
Spoken dialog systems must keep a probability distribution over what the user wants, a task called dialog state tracking. Most trackers consume the output of a separate SLU semantic decoder and depend on hand-designed features. Henderson, Thomson and Young skip the SLU stage and update the belief state directly from ASR N-best hypotheses and the last system act. Their model uses one recurrent neural network per slot. It takes simple weighted n-gram features and keeps an internal memory across turns. To cope with rare or unseen slot values, part of the network works on "tagged" features, where specific values and slots are replaced by generic placeholders. This lets it learn patterns that hold for any value. Training relies on careful initialisation, namely denoising autoencoders and shared initialisation. Several trackers are combined by averaging their scores. Evaluated blind on the DSTC2 restaurant-domain corpus, the word-based RNN was among the strongest entries, especially for the quality of its probability estimates. This suggests that removing the explicit semantic layer can reduce information loss without heavy feature engineering.

## Key points
- The data is DSTC2 (Cambridge restaurant search). It has 1612 training dialogs and 506 dev dialogs, plus 1117 test dialogs collected with a dialog manager that does not appear in the labelled data.
- Four variants were submitted under "team4". They cross two choices: word-based (ASR) vs. SLU input, and with vs. without the value-specific sub-network h. Each variant is a score-averaged ensemble of 12 trackers.
- The best system (h-ASR) reached 0.768 joint goal accuracy with an L2 of 0.346 on the test set. It ranked first for joint goal L2 and for requested-slot accuracy and L2.
- Across 318 DSTC2 metrics, h-ASR came top in 89, more than any other tracker. The ASR variant without h was second with 33.
- Variants without h do not depend on specific values, so they can handle new values at runtime. An unoptimised Python implementation processes about 50 turns per second.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

Fields: [Linguistics & languages](https://md.research.elamur.ai/free/en/shelves/linguistics/)

## Cited by

Cited by

94 papers of our library cite it (of the 140673 whose bibliographies we read); 6 of them have a page here, 88 not yet.

- [Creating Annotated Dialogue Resources: Cross-domain Dialogue Act Classification](https://md.research.elamur.ai/free/en/papers/amanova-2016-creating-annotated-dialogue-resources-cross/)
- [Common Ground Tracking in Multimodal Dialogue](https://md.research.elamur.ai/free/en/papers/khebour-2024-common-ground-tracking-multimodal-dialogue/)
- [SUMBT: Slot-Utterance Matching for Universal and Scalable Belief Tracking](https://md.research.elamur.ai/free/en/papers/lee-2019-sumbt-slot-utterance-matching-universal/)
- [The Ubuntu Dialogue Corpus: A Large Dataset for Research in Unstructured Multi-Turn Dialogue Systems](https://md.research.elamur.ai/free/en/papers/lowe-2015-ubuntu-dialogue-corpus-large-dataset/)
- [Building End-To-End Dialogue Systems Using Generative Hierarchical Neural Network Models](https://md.research.elamur.ai/free/en/papers/serban-2015-building-end-end-dialogue-systems/)
- [Find or Classify? Dual Strategy for Slot-Value Predictions on Multi-Domain Dialog State Tracking](https://md.research.elamur.ai/free/en/papers/zhang-2020-find-or-classify-dual-strategy/)

## Cite

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{henderson2014word,
  title = {{Word-Based Dialog State Tracking with Recurrent Neural Networks}},
  author = {Henderson, Matthew and Thomson, Blaise and Young, Steve},
  booktitle = {Proceedings of the 15th Annual Meeting of the Special Interest Group on Discourse and Dialogue (SIGDIAL)},
  year = {2014},
  doi = {10.3115/v1/w14-4340},
  url = {https://doi.org/10.3115/v1/w14-4340}
}
```

- [← previous Why are You Taking this Stance? Identifying and Classifying Reasons in Ideological Debates](https://md.research.elamur.ai/free/en/papers/hasan-2014-why-are-you-taking-this/)
- [next → WordNet—Wikipedia—Wiktionary: Construction of a Three-way Alignment](https://md.research.elamur.ai/free/en/papers/lex-2014-miller-wordnet-wikipedia-wiktionary/)
- [↑ back to papers of 2014](https://md.research.elamur.ai/free/en/papers/2014/page/2/)

## References

16 references, 0 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Dan Bohus and Alex Rudnicky. 2006. A K-hypotheses+ Other Belief Updating Model. Proc. of the AAAI Workshop on Statistical and Empirical Methods in Spoken Dialogue Systems. [find](https://scholar.google.com/scholar?q=Dan+Bohus+and+Alex+Rudnicky%2E+2006%2E+A+K%2Dhypotheses%2B+Other+Belief+Updating+Model%2E+Proc%2E+of+the+AAAI+Workshop+on+Statistical+and+Empirical+Methods+in+Spoken+Dialogue+Systems%2E)
2. Léon Bottou. 1991. Stochastic gradient learning in neural networks. In Proceedings of Neuro-Nı̂mes 91, Nı̂mes, France. EC2. [find](https://scholar.google.com/scholar?q=Le%CC%81on+Bottou%2E+1991%2E+Stochastic+gradient+learning+in+neural+networks%2E+In+Proceedings+of+Neuro%2DN%C4%B1%CC%82mes+91%2C+N%C4%B1%CC%82mes%2C+France%2E+EC2%2E)
3. Yun-Nung Chen, William Yang Wang, and Alexander I Rudnicky. 2013. An empirical investigation of sparse log-linear models for improved dialogue act classification. In Acoustics, Speech and Signal Processing (ICASSP), 2013 IEEE International Conference on. [find](https://scholar.google.com/scholar?q=Yun%2DNung+Chen%2C+William+Yang+Wang%2C+and+Alexander+I+Rudnicky%2E+2013%2E+An+empirical+investigation+of+sparse+log%2Dlinear+models+for+improved+dialogue+act+classification%2E+In+Acoustics%2C+Speech+and+Signal+Proce)
4. Matthew Henderson, Milica Gašić, Blaise Thomson, Pirros Tsiakoulis, Kai Yu, and Steve Young. 2012. [find](https://scholar.google.com/scholar?q=Matthew+Henderson%2C+Milica+Gas%CC%8Cic%CC%81%2C+Blaise+Thomson%2C+Pirros+Tsiakoulis%2C+Kai+Yu%2C+and+Steve+Young%2E+2012%2E)
5. Discriminative Spoken Language Understanding Using Word Confusion Networks. In Spoken Language Technology Workshop, 2012. IEEE. [find](https://scholar.google.com/scholar?q=Discriminative+Spoken+Language+Understanding+Using+Word+Confusion+Networks%2E+In+Spoken+Language+Technology+Workshop%2C+2012%2E+IEEE%2E)
6. Matthew Henderson, Blaise Thomson, and Jason Williams. 2013a. Dialog State Tracking Challenge 2 & 3 Handbook. camdial.org/˜mh521/dstc/ . [find](https://scholar.google.com/scholar?q=Matthew+Henderson%2C+Blaise+Thomson%2C+and+Jason+Williams%2E+2013a%2E+Dialog+State+Tracking+Challenge+2+%26+3+Handbook%2E+camdial%2Eorg%2F%CB%9Cmh521%2Fdstc%2F+%2E)
7. Matthew Henderson, Blaise Thomson, and Steve Young. 2013b. Deep Neural Network Approach for the Dialog State Tracking Challenge. In Proceedings of SIGdial, Metz, France, August. [find](https://scholar.google.com/scholar?q=Matthew+Henderson%2C+Blaise+Thomson%2C+and+Steve+Young%2E+2013b%2E+Deep+Neural+Network+Approach+for+the+Dialog+State+Tracking+Challenge%2E+In+Proceedings+of+SIGdial%2C+Metz%2C+France%2C+August%2E)
8. Matthew Henderson, Blaise Thomson, and Jason Williams. 2014. The second dialog state tracking challenge. In Proceedings of the SIGdial 2014 Conference, Baltimore, U.S.A., June. [find](https://scholar.google.com/scholar?q=Matthew+Henderson%2C+Blaise+Thomson%2C+and+Jason+Williams%2E+2014%2E+The+second+dialog+state+tracking+challenge%2E+In+Proceedings+of+the+SIGdial+2014+Conference%2C+Baltimore%2C+U%2ES%2EA%2E%2C+June%2E)
9. Sungjin Lee and Maxine Eskenazi. 2013. Recipe for building robust spoken dialog state trackers: Dialog state tracking challenge system description. In Proceedings of the SIGDIAL 2013 Conference, Metz, France, August. [find](https://scholar.google.com/scholar?q=Sungjin+Lee+and+Maxine+Eskenazi%2E+2013%2E+Recipe+for+building+robust+spoken+dialog+state+trackers%3A+Dialog+state+tracking+challenge+system+description%2E+In+Proceedings+of+the+SIGDIAL+2013+Conference%2C+Metz%2C)
10. Sungjin Lee. 2013. Structured discriminative model for dialog state tracking. In Proceedings of the SIG-DIAL 2013 Conference, Metz, France, August. [find](https://scholar.google.com/scholar?q=Sungjin+Lee%2E+2013%2E+Structured+discriminative+model+for+dialog+state+tracking%2E+In+Proceedings+of+the+SIG%2DDIAL+2013+Conference%2C+Metz%2C+France%2C+August%2E)
11. Razvan Pascanu, Tomas Mikolov, and Yoshua Bengio. 2012. Understanding the exploding gradient problem. CoRR. [find](https://scholar.google.com/scholar?q=Razvan+Pascanu%2C+Tomas+Mikolov%2C+and+Yoshua+Bengio%2E+2012%2E+Understanding+the+exploding+gradient+problem%2E+CoRR%2E)
12. Hang Ren, Weiqun Xu, Yan Zhang, and Yonghong Yan. 2013. Dialog state tracking using conditional random fields. In Proceedings of the SIGDIAL 2013 Conference, Metz, France, August. [find](https://scholar.google.com/scholar?q=Hang+Ren%2C+Weiqun+Xu%2C+Yan+Zhang%2C+and+Yonghong+Yan%2E+2013%2E+Dialog+state+tracking+using+conditional+random+fields%2E+In+Proceedings+of+the+SIGDIAL+2013+Conference%2C+Metz%2C+France%2C+August%2E)
13. Gökhan Tür, Anoop Deoras, and Dilek Hakkani-Tür. 2013. Semantic parsing using word confusion networks with conditional random fields. In INTER-SPEECH. [find](https://scholar.google.com/scholar?q=Go%CC%88khan+Tu%CC%88r%2C+Anoop+Deoras%2C+and+Dilek+Hakkani%2DTu%CC%88r%2E+2013%2E+Semantic+parsing+using+word+confusion+networks+with+conditional+random+fields%2E+In+INTER%2DSPEECH%2E)
14. Pascal Vincent, Hugo Larochelle, Yoshua Bengio, and Pierre-Antoine Manzagol. 2008. Extracting and composing robust features with denoising autoencoders. In Proceedings of the 25th International Conference on Machine Learning, Helsinki, Finland. [find](https://scholar.google.com/scholar?q=Pascal+Vincent%2C+Hugo+Larochelle%2C+Yoshua+Bengio%2C+and+Pierre%2DAntoine+Manzagol%2E+2008%2E+Extracting+and+composing+robust+features+with+denoising+autoencoders%2E+In+Proceedings+of+the+25th+International+Confer)
15. Jason Williams. 2013. Multi-domain learning and generalization in dialog state tracking. In Proceedings of the SIGDIAL 2013 Conference, Metz, France, August. [find](https://scholar.google.com/scholar?q=Jason+Williams%2E+2013%2E+Multi%2Ddomain+learning+and+generalization+in+dialog+state+tracking%2E+In+Proceedings+of+the+SIGDIAL+2013+Conference%2C+Metz%2C+France%2C+August%2E)
16. Steve Young, Milica Gašić, Simon Keizer, François Mairesse, Jost Schatzmann, Blaise Thomson, and Kai Yu. 2009. The Hidden Information State model: A practical framework for POMDP-based spoken dialogue management. Computer Speech & Language. [find](https://scholar.google.com/scholar?q=Steve+Young%2C+Milica+Gas%CC%8Cic%CC%81%2C+Simon+Keizer%2C+Franc%CC%A7ois+Mairesse%2C+Jost+Schatzmann%2C+Blaise+Thomson%2C+and+Kai+Yu%2E+2009%2E+The+Hidden+Information+State+model%3A+A+practical+framework+for+POMDP%2Dbased+spoken+dialo)