---
title: "SUMBT: Slot-Utterance Matching for Universal and — Lee 2019"
description: "Goal-oriented dialog systems must work out, at every turn, which value the user intends for each slot."
authors: ["Hwaran Lee", "Jinsik Lee", "Tae-Yoon Kim"]
year: 2019
venue: "Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics"
source: "https://aclanthology.org/P19-1546/"
doi: "10.18653/v1/p19-1546"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/lee-2019-sumbt-slot-utterance-matching-universal/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/lee-2019-sumbt-slot-utterance-matching-universal/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2019](https://md.research.elamur.ai/free/en/papers/2019/page/4/) / SUMBT: Slot-Utterance Matching for Universal and Scalable Belief Tracking

# SUMBT: Slot-Utterance Matching for Universal and Scalable Belief Tracking

**Authors:** [Hwaran Lee](https://md.research.elamur.ai/free/en/authors/hwaran-lee/), Jinsik Lee, Tae-Yoon Kim · **Year:** 2019 · **Venue:** Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/P19-1546/) · [PDF](https://aclanthology.org/P19-1546.pdf) · [DOI 10.18653/v1/p19-1546](https://doi.org/10.18653/v1/p19-1546) · [OpenAlex](https://openalex.org/W2962831269)

**Citations:** 172 citations (openalex-snapshot · 2026-10-14) · [85](#cited-by) in our library, of the 140884 papers whose bibliographies we read

## Summary
Goal-oriented dialog systems must work out, at every turn, which value the user intends for each slot. This problem is known as dialog state tracking (DST), also referred to as belief tracking. Earlier neural trackers either built separate components per domain or slot, or used output layers tied to a fixed list of candidate values, so changing the ontology meant retraining.

The authors propose SUMBT, a single tracker shared by all domains and slot-types. Following ideas from machine reading comprehension, it treats a domain-slot-type phrase (e.g., "restaurant – food") as a query. Multi-head attention then locates the relevant words in BERT-encoded system and user utterances. An RNN carries this attended context across turns. Its output is compared, through a distance metric, with frozen BERT embeddings of candidate slot-values. Because prediction is non-parametric, new slots or values can be supported simply by encoding them.

On WOZ 2.0 and MultiWOZ, SUMBT achieved state-of-the-art joint goal accuracy. Training one model across all slots outperformed slot-specific variants. This matters for assistants where new domains and skills keep appearing.

## Key points
- Evaluation corpora: WOZ 2.0 (restaurant domain, slots area, food and price range) and MultiWOZ (35 slots across 7 domains).
- WOZ 2.0 joint accuracy:
  - SUMBT: 0.910.
  - StateNetPSI: 0.889.
  - BERT+RNN, BERT+RNN+Ontology and slot-dependent SUMBT baselines: 0.891–0.893.
- MultiWOZ joint accuracy:
  - SUMBT: 0.4240.
  - GCE: 0.3558.
  - GLAD: 0.3557.
  - Benchmark baseline: 0.2583.
- Design and setup:
  - The utterance BERT is fine-tuned, while a separate BERT sv for slots and values stays frozen.
  - The tracker is a GRU or LSTM.
  - Euclidean and negative cosine distances were both tested.
  - Results are averaged over 20 random seeds.
- Released resources and analysis:
  - The PyTorch code and the preprocessed MultiWOZ data are available at github.com/SKTBrain/SUMBT.
  - Attention maps show the model focusing on "reasonably priced" when the label is 'moderate'.
  - When no value is present, attention shifts to [CLS]/[SEP] tokens.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

Fields: [Natural language processing](https://md.research.elamur.ai/free/en/shelves/nlp/)

## Cited by

Cited by

85 papers of our library cite it (of the 140884 whose bibliographies we read); 5 of them have a page here, 80 not yet.

- [Efficient Dialogue State Tracking by Selectively Overwriting Memory](https://md.research.elamur.ai/free/en/papers/kim-2020-efficient-dialogue-state-tracking-selectively/)
- [MinTL: Minimalist Transfer Learning for Task-Oriented Dialogue Systems](https://md.research.elamur.ai/free/en/papers/lin-2020-mintl-minimalist-transfer-learning-task/)
- [Event-Guided Slot Interaction for Multi-Domain Dialogue State Tracking](https://md.research.elamur.ai/free/en/papers/xia-2026-event-guided-slot-interaction-multi/)
- [MultiWOZ 2.4: A Multi-Domain Task-Oriented Dialogue Dataset with Essential Annotation Corrections to Improve State Tracking Evaluation](https://md.research.elamur.ai/free/en/papers/ye-2022-multiwoz-2-4-multi-domain/)
- [Find or Classify? Dual Strategy for Slot-Value Predictions on Multi-Domain Dialog State Tracking](https://md.research.elamur.ai/free/en/papers/zhang-2020-find-or-classify-dual-strategy/)

## Cite

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{lee2019sumbt,
  title = {{SUMBT: Slot-Utterance Matching for Universal and Scalable Belief Tracking}},
  author = {Lee, Hwaran and Lee, Jinsik and Kim, Tae-Yoon},
  booktitle = {Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics},
  year = {2019},
  doi = {10.18653/v1/p19-1546},
  url = {https://doi.org/10.18653/v1/p19-1546}
}
```

- [← previous Submodular Optimization-based Diverse Paraphrasing and its Effectiveness in Data Augmentation](https://md.research.elamur.ai/free/en/papers/kumar-2019-submodular-optimization-based-diverse-paraphrasing/)
- [next → SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systems](https://md.research.elamur.ai/free/en/papers/wang-2019-superglue-stickier-benchmark-general-purpose/)
- [↑ back to papers of 2019](https://md.research.elamur.ai/free/en/papers/2019/page/4/)

## References

18 references, 4 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Jimmy Lei Ba, Jamie Ryan Kiros, and Geoffrey E. Hinton. 2016. Layer normalization. arXiv preprint, arXiv:1607.06450. [find](https://scholar.google.com/scholar?q=Jimmy+Lei+Ba%2C+Jamie+Ryan+Kiros%2C+and+Geoffrey+E%2E+Hinton%2E+2016%2E+Layer+normalization%2E+arXiv+preprint%2C+arXiv%3A1607%2E06450%2E) [our card](https://md.research.elamur.ai/free/en/papers/ba-2016-layer-normalization/)
2. Paweł Budzianowski, Tsung Hsien Wen, Bo-Hsiang Tseng, Iñigo Casanueva, Stefan Ultes, Osman Ramadan, and Milica Gašić. 2018. MultiWOZ - a large-scale multi-domain Wizard-of-Oz dataset for task-oriented dialogue modelling. In Proceedings of the 2018 Conference on Empirical Methods in Natural Langua… [find](https://scholar.google.com/scholar?q=Pawe%C5%82+Budzianowski%2C+Tsung+Hsien+Wen%2C+Bo%2DHsiang+Tseng%2C+In%CC%83igo+Casanueva%2C+Stefan+Ultes%2C+Osman+Ramadan%2C+and+Milica+Gas%CC%8Cic%CC%81%2E+2018%2E+MultiWOZ+%2D+a+large%2Dscale+multi%2Ddomain+Wizard%2Dof%2DOz+dataset+for+task%2Dorien)
3. Danqi Chen, Adam Fisch, Jason Weston, and Antoine Bordes. 2017. Reading Wikipedia to answer open-domain questions. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, pages 1870–1879. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Danqi+Chen%2C+Adam+Fisch%2C+Jason+Weston%2C+and+Antoine+Bordes%2E+2017%2E+Reading+Wikipedia+to+answer+open%2Ddomain+questions%2E+In+Proceedings+of+the+55th+Annual+Meeting+of+the+Association+for+Computational+Lingui) [our card](https://md.research.elamur.ai/free/en/papers/chen-2017-reading-wikipedia-answer-open-domain/)
4. Yun-Nung Chen, Dilek Hakkani-Tür, and Xiaodong He. 2016. Zero-shot learning of intent embeddings for expansion by convolutional deep structured semantic models. In Proceedings of the 41st IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 6045–6049. IEEE. [find](https://scholar.google.com/scholar?q=Yun%2DNung+Chen%2C+Dilek+Hakkani%2DTu%CC%88r%2C+and+Xiaodong+He%2E+2016%2E+Zero%2Dshot+learning+of+intent+embeddings+for+expansion+by+convolutional+deep+structured+semantic+models%2E+In+Proceedings+of+the+41st+IEEE+Intern)
5. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. BERT: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint, arXiv:1810.04805. [find](https://scholar.google.com/scholar?q=Jacob+Devlin%2C+Ming%2DWei+Chang%2C+Kenton+Lee%2C+and+Kristina+Toutanova%2E+2018%2E+BERT%3A+Pre%2Dtraining+of+deep+bidirectional+transformers+for+language+understanding%2E+arXiv+preprint%2C+arXiv%3A1810%2E04805%2E)
6. Rahul Goel, Shachi Paul, Tagyoung Chung, Jeremie Lecomte, Arindam Mandal, and Dilek Hakkani-Tur. 2018. Flexible and scalable state tracking framework for goal-oriented dialogue systems. arXiv preprint, arXiv:1811.12891. — [arXiv:1811.12891](https://arxiv.org/abs/1811.12891) [find](https://scholar.google.com/scholar?q=Rahul+Goel%2C+Shachi+Paul%2C+Tagyoung+Chung%2C+Jeremie+Lecomte%2C+Arindam+Mandal%2C+and+Dilek+Hakkani%2DTur%2E+2018%2E+Flexible+and+scalable+state+tracking+framework+for+goal%2Doriented+dialogue+systems%2E+arXiv+preprint)
7. Matthew Henderson, Blaise Thomson, and Steve Young. 2014. Word-based dialog state tracking with recurrent neural networks. In Proceedings of the SIGDIAL 2014 Conference, pages 292–299. Association for Computational Linguistics. system. In Proceedings of the 15th Conference of the European Chapter of… [find](https://scholar.google.com/scholar?q=Matthew+Henderson%2C+Blaise+Thomson%2C+and+Steve+Young%2E+2014%2E+Word%2Dbased+dialog+state+tracking+with+recurrent+neural+networks%2E+In+Proceedings+of+the+SIGDIAL+2014+Conference%2C+pages+292%E2%80%93299%2E+Association+for) [our card](https://md.research.elamur.ai/free/en/papers/henderson-2014-word-based-dialog-state-tracking/)
8. Young-Bum Kim, Dongchan Kim, Anjishnu Kumar, and Ruhi Sarikaya. 2018. Efficient large-scale neural domain classification with personalized attention. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Long Papers), pages 2214–2224. Association for Computation… [find](https://scholar.google.com/scholar?q=Young%2DBum+Kim%2C+Dongchan+Kim%2C+Anjishnu+Kumar%2C+and+Ruhi+Sarikaya%2E+2018%2E+Efficient+large%2Dscale+neural+domain+classification+with+personalized+attention%2E+In+Proceedings+of+the+56th+Annual+Meeting+of+the+A)
9. Victor Zhong, Caiming Xiong, and Richard Socher. 2018. Global-locally self-attentive dialogue state tracker. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Long Papers), pages 1458–1467. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Victor+Zhong%2C+Caiming+Xiong%2C+and+Richard+Socher%2E+2018%2E+Global%2Dlocally+self%2Dattentive+dialogue+state+tracker%2E+In+Proceedings+of+the+56th+Annual+Meeting+of+the+Association+for+Computational+Linguistics+)
10. Nikola Mrkšić and Ivan Vulić. 2018. Fully statistical neural belief tracking. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Short Papers), pages 108–113. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Nikola+Mrks%CC%8Cic%CC%81+and+Ivan+Vulic%CC%81%2E+2018%2E+Fully+statistical+neural+belief+tracking%2E+In+Proceedings+of+the+56th+Annual+Meeting+of+the+Association+for+Computational+Linguistics+%28Short+Papers%29%2C+pages+108%E2%80%9311)
11. Nikola Mrkšić, Diarmuid Ó Séaghdha, Tsung Hsien Wen, Blaise Thomson, and Steve Young. 2017. Neural belief tracker: Data-driven dialogue state tracking. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, pages 1777–1788. Association for Computational Lingu… [find](https://scholar.google.com/scholar?q=Nikola+Mrks%CC%8Cic%CC%81%2C+Diarmuid+O%CC%81+Se%CC%81aghdha%2C+Tsung+Hsien+Wen%2C+Blaise+Thomson%2C+and+Steve+Young%2E+2017%2E+Neural+belief+tracker%3A+Data%2Ddriven+dialogue+state+tracking%2E+In+Proceedings+of+the+55th+Annual+Meeting+of)
12. Elnaz Nouri and Ehsan Hosseini-Asl. 2018. Toward scalable neural dialogue state tracking model. arXiv preprint, arXiv:1812.00899. — [arXiv:1812.00899](https://arxiv.org/abs/1812.00899) [find](https://scholar.google.com/scholar?q=Elnaz+Nouri+and+Ehsan+Hosseini%2DAsl%2E+2018%2E+Toward+scalable+neural+dialogue+state+tracking+model%2E+arXiv+preprint%2C+arXiv%3A1812%2E00899%2E+%E2%80%94+%5BarXiv%3A1812%2E00899%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F1812%2E00899%29)
13. Osman Ramadan, Paweł Budzianowski, and Milica Gašić. 2018. Large-scale multi-domain belief tracking with knowledge sharing. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Short Papers), pages 432–437. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Osman+Ramadan%2C+Pawe%C5%82+Budzianowski%2C+and+Milica+Gas%CC%8Cic%CC%81%2E+2018%2E+Large%2Dscale+multi%2Ddomain+belief+tracking+with+knowledge+sharing%2E+In+Proceedings+of+the+56th+Annual+Meeting+of+the+Association+for+Computati)
14. Liliang Ren, Kaige Xie, Lu Chen, and Kai Yu. 2018. Towards universal dialogue state tracking. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 2780– 2786. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Liliang+Ren%2C+Kaige+Xie%2C+Lu+Chen%2C+and+Kai+Yu%2E+2018%2E+Towards+universal+dialogue+state+tracking%2E+In+Proceedings+of+the+2018+Conference+on+Empirical+Methods+in+Natural+Language+Processing%2C+pages+2780%E2%80%93+278)
15. Minjoon Seo, Aniruddha Kembhavi, Ali Farhadi, and Hannaneh Hajishirzi. 2017. Bidirectional attention flow for machine comprehension. In International Conference on Learning Representations (ICLR). [find](https://scholar.google.com/scholar?q=Minjoon+Seo%2C+Aniruddha+Kembhavi%2C+Ali+Farhadi%2C+and+Hannaneh+Hajishirzi%2E+2017%2E+Bidirectional+attention+flow+for+machine+comprehension%2E+In+International+Conference+on+Learning+Representations+%28ICLR%29%2E) [our card](https://md.research.elamur.ai/free/en/papers/seo-2016-bidirectional-attention-flow-machine-comprehension/)
16. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Advances in Neural Information Processing Systems, pages 5998–6008. [find](https://scholar.google.com/scholar?q=Ashish+Vaswani%2C+Noam+Shazeer%2C+Niki+Parmar%2C+Jakob+Uszkoreit%2C+Llion+Jones%2C+Aidan+N+Gomez%2C+%C5%81ukasz+Kaiser%2C+and+Illia+Polosukhin%2E+2017%2E+Attention+is+all+you+need%2E+In+Advances+in+Neural+Information+Processi)
17. Oriol Vinyals, Charles Blundell, Timothy Lillicrap, Daan Wierstra, et al. 2016. Matching networks for one shot learning. In Advances in neural information processing systems, pages 3630–3638. [find](https://scholar.google.com/scholar?q=Oriol+Vinyals%2C+Charles+Blundell%2C+Timothy+Lillicrap%2C+Daan+Wierstra%2C+et+al%2E+2016%2E+Matching+networks+for+one+shot+learning%2E+In+Advances+in+neural+information+processing+systems%2C+pages+3630%E2%80%933638%2E)
18. Tsung-Hsien Wen, David Vandyke, Nikola Mrkšić, Milica Gasic, Lina M. Rojas Barahona, Pei-Hao Su, Stefan Ultes, and Steve Young. 2017. A network-based end-to-end trainable task-oriented dialogue [find](https://scholar.google.com/scholar?q=Tsung%2DHsien+Wen%2C+David+Vandyke%2C+Nikola+Mrks%CC%8Cic%CC%81%2C+Milica+Gasic%2C+Lina+M%2E+Rojas+Barahona%2C+Pei%2DHao+Su%2C+Stefan+Ultes%2C+and+Steve+Young%2E+2017%2E+A+network%2Dbased+end%2Dto%2Dend+trainable+task%2Doriented+dialogue)