---
title: "Efficient Dialogue State Tracking by Selectively — Kim 2020"
description: "Dialogue state tracking (DST) records what a user wants during a task-oriented conversation as a set of slot-value pairs."
authors: ["Sungdong Kim", "Sohee Yang", "Gyuwan Kim", "Sang-Woo Lee"]
year: 2020
venue: "Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics"
source: "https://aclanthology.org/2020.acl-main.53/"
doi: "10.18653/v1/2020.acl-main.53"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/kim-2020-efficient-dialogue-state-tracking-selectively/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/kim-2020-efficient-dialogue-state-tracking-selectively/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2020](https://md.research.elamur.ai/free/en/papers/2020/page/2/) / Efficient Dialogue State Tracking by Selectively Overwriting Memory

# Efficient Dialogue State Tracking by Selectively Overwriting Memory

**Authors:** [Sungdong Kim](https://md.research.elamur.ai/free/en/authors/sungdong-kim/), [Sohee Yang](https://md.research.elamur.ai/free/en/authors/sohee-yang/), Gyuwan Kim, [Sang-Woo Lee](https://md.research.elamur.ai/free/en/authors/sang-woo-lee/) · **Year:** 2020 · **Venue:** Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2020.acl-main.53/) · [PDF](https://aclanthology.org/2020.acl-main.53.pdf) · [DOI 10.18653/v1/2020.acl-main.53](https://doi.org/10.18653/v1/2020.acl-main.53) · [OpenAlex](https://openalex.org/W3034573951)

**Citations:** 186 citations (openalex-snapshot · 2026-10-14) · [82](#cited-by) in our library, of the 140884 papers whose bibliographies we read

## Summary
Dialogue state tracking (DST) records what a user wants during a task-oriented conversation as a set of slot-value pairs. Open vocabulary-based trackers remove the need for a predefined ontology, but most of them rebuild the full state at every turn, which wastes computation. The authors, from Clova AI at NAVER, treat the state as a fixed-size memory that only needs partial rewriting. In SOM-DST, a BERT-based encoder reads the previous and current turn utterances along with the previous state. It then assigns each slot one of four operations: CARRYOVER, DELETE, DONTCARE or UPDATE. A decoder generates values autoregressively, but only for slots marked UPDATE. Splitting the task this way gives each component a narrower job and makes the decoder's work easier. On MultiWOZ 2.0 and MultiWOZ 2.1, the model achieved the best joint goal accuracy reported for the open vocabulary setting and ran much faster than TRADE. An error analysis shows that operation prediction is the main weak spot, which points to a clear direction for future work.

## Key points
- Joint goal accuracy is 51.72% on MultiWOZ 2.0 and 53.01% on MultiWOZ 2.1, rising to 53.68% with BERT-large. Code is released at github.com/clovaai/som-dst.
- On the MultiWOZ 2.1 test set, SOM-DST generates values for 1.14 slots per turn on average and 9 at most. TRADE and ML-BST generate all 30 slots every turn, and COMER averages 5.72.
- Latency on a Tesla V100 with batch size 1 is about 27 ms per turn, roughly 12.5 times faster than TRADE at 340 ms. This is close to NADST's 26 ms, while accuracy is clearly higher (53.01% vs 49.04%).
- When the gold previous state is supplied, accuracy reaches 81.00%. Feeding back predicted states therefore increases the error about 2.47 times.
- The state operation predictor accounts for roughly 78–93% of errors. Rare operations are handled poorly, with test F1 of 32.51 for DONTCARE and 2.86 for DELETE, which reflects strong class imbalance.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

Fields: [Natural language processing](https://md.research.elamur.ai/free/en/shelves/nlp/)

## Cited by

Cited by

82 papers of our library cite it (of the 140884 whose bibliographies we read); 5 of them have a page here, 77 not yet.

- [In-Context Learning for Few-Shot Dialogue State Tracking](https://md.research.elamur.ai/free/en/papers/hu-2022-context-learning-few-shot-dialogue/)
- [MinTL: Minimalist Transfer Learning for Task-Oriented Dialogue Systems](https://md.research.elamur.ai/free/en/papers/lin-2020-mintl-minimalist-transfer-learning-task/)
- [Disentangling Indirect Answers to Yes-No Questions in Real Conversations](https://md.research.elamur.ai/free/en/papers/sanagavarapu-2022-disentangling-indirect-answers-yes-no/)
- [Event-Guided Slot Interaction for Multi-Domain Dialogue State Tracking](https://md.research.elamur.ai/free/en/papers/xia-2026-event-guided-slot-interaction-multi/)
- [MultiWOZ 2.4: A Multi-Domain Task-Oriented Dialogue Dataset with Essential Annotation Corrections to Improve State Tracking Evaluation](https://md.research.elamur.ai/free/en/papers/ye-2022-multiwoz-2-4-multi-domain/)

## Cite

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{kim2020efficient,
  title = {{Efficient Dialogue State Tracking by Selectively Overwriting Memory}},
  author = {Kim, Sungdong and Yang, Sohee and Kim, Gyuwan and Lee, Sang-Woo},
  booktitle = {Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics},
  year = {2020},
  doi = {10.18653/v1/2020.acl-main.53},
  url = {https://doi.org/10.18653/v1/2020.acl-main.53}
}
```

- [← previous Efficient Content-Based Sparse Attention with Routing Transformers](https://md.research.elamur.ai/free/en/papers/roy-2020-efficient-content-based-sparse-attention/)
- [next → Efficient Intent Detection with Dual Sentence Encoders](https://md.research.elamur.ai/free/en/papers/casanueva-2020-efficient-intent-detection-dual-sentence/)
- [↑ back to papers of 2020](https://md.research.elamur.ai/free/en/papers/2020/page/2/)

## References

27 references, 6 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Samuel R. Bowman, Luke Vilnis, Oriol Vinyals, Andrew Dai, Rafal Jozefowicz, and Samy Bengio. 2016. Generating sentences from a continuous space. In Proceedings of The 20th SIGNLL Conference on Computational Natural Language Learning, pages 10–21, Berlin, Germany. Association for Computational Lingui… [find](https://scholar.google.com/scholar?q=Samuel+R%2E+Bowman%2C+Luke+Vilnis%2C+Oriol+Vinyals%2C+Andrew+Dai%2C+Rafal+Jozefowicz%2C+and+Samy+Bengio%2E+2016%2E+Generating+sentences+from+a+continuous+space%2E+In+Proceedings+of+The+20th+SIGNLL+Conference+on+Computa) [our card](https://md.research.elamur.ai/free/en/papers/bowman-2016-generating-sentences-continuous-space/)
2. Paweł Budzianowski, Tsung-Hsien Wen, Bo-Hsiang Tseng, Inigo Casanueva, Stefan Ultes, Osman Ramadan, and Milica Gasic. 2018. Multiwoz - a large-scale multi-domain wizard-of-oz dataset for task-oriented dialogue modelling. In EMNLP. [find](https://scholar.google.com/scholar?q=Pawe%C5%82+Budzianowski%2C+Tsung%2DHsien+Wen%2C+Bo%2DHsiang+Tseng%2C+Inigo+Casanueva%2C+Stefan+Ultes%2C+Osman+Ramadan%2C+and+Milica+Gasic%2E+2018%2E+Multiwoz+%2D+a+large%2Dscale+multi%2Ddomain+wizard%2Dof%2Doz+dataset+for+task%2Doriented)
3. Hongshen Chen, Xiaorui Liu, Dawei Yin, and Jiliang Tang. 2017. A survey on dialogue systems: Recent advances and new frontiers. ACM SIGKDD Explorations Newsletter, 19(2):25–35. [find](https://scholar.google.com/scholar?q=Hongshen+Chen%2C+Xiaorui+Liu%2C+Dawei+Yin%2C+and+Jiliang+Tang%2E+2017%2E+A+survey+on+dialogue+systems%3A+Recent+advances+and+new+frontiers%2E+ACM+SIGKDD+Explorations+Newsletter%2C+19%282%29%3A25%E2%80%9335%2E)
4. Kyunghyun Cho, Bart van Merrienboer, Caglar Gulcehre, Dzmitry Bahdanau, Fethi Bougares, Holger Schwenk, and Yoshua Bengio. 2014a. Learning phrase representations using rnn encoder–decoder for statistical machine translation. In EMNLP. [find](https://scholar.google.com/scholar?q=Kyunghyun+Cho%2C+Bart+van+Merrienboer%2C+Caglar+Gulcehre%2C+Dzmitry+Bahdanau%2C+Fethi+Bougares%2C+Holger+Schwenk%2C+and+Yoshua+Bengio%2E+2014a%2E+Learning+phrase+representations+using+rnn+encoder%E2%80%93decoder+for+statisti)
5. Kyunghyun Cho, Bart Van Merriënboer, Dzmitry Bahdanau, and Yoshua Bengio. 2014b. On the properties of neural machine translation: Encoder-decoder approaches. In Proceedings of SSST-8, Eighth Workshop on Syntax, Semantics and Structure in Statistical Translation. [find](https://scholar.google.com/scholar?q=Kyunghyun+Cho%2C+Bart+Van+Merrie%CC%88nboer%2C+Dzmitry+Bahdanau%2C+and+Yoshua+Bengio%2E+2014b%2E+On+the+properties+of+neural+machine+translation%3A+Encoder%2Ddecoder+approaches%2E+In+Proceedings+of+SSST%2D8%2C+Eighth+Workshop) [our card](https://md.research.elamur.ai/free/en/papers/cho-2014-properties-neural-machine-translation-encoder/)
6. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. Bert: Pre-training of deep bidirectional transformers for language understanding. In NAACL-HLT. [find](https://scholar.google.com/scholar?q=Jacob+Devlin%2C+Ming%2DWei+Chang%2C+Kenton+Lee%2C+and+Kristina+Toutanova%2E+2019%2E+Bert%3A+Pre%2Dtraining+of+deep+bidirectional+transformers+for+language+understanding%2E+In+NAACL%2DHLT%2E)
7. Mihail Eric, Rahul Goel, Shachi Paul, Abhishek Sethi, Sanchit Agarwal, Shuyag Gao, and Dilek Hakkani-Tur. 2019. Multiwoz 2.1: Multi-domain dialogue state corrections and state tracking baselines. arXiv preprint arXiv:1907.01669. [find](https://scholar.google.com/scholar?q=Mihail+Eric%2C+Rahul+Goel%2C+Shachi+Paul%2C+Abhishek+Sethi%2C+Sanchit+Agarwal%2C+Shuyag+Gao%2C+and+Dilek+Hakkani%2DTur%2E+2019%2E+Multiwoz+2%2E1%3A+Multi%2Ddomain+dialogue+state+corrections+and+state+tracking+baselines%2E+arXi)
8. Shuyang Gao, Abhishek Sethi, Sanchit Agarwal, Tagyoung Chung, and Dilek Hakkani-Tur. 2019. Dialog state tracking: A neural reading comprehension approach. In SIGDIAL. [find](https://scholar.google.com/scholar?q=Shuyang+Gao%2C+Abhishek+Sethi%2C+Sanchit+Agarwal%2C+Tagyoung+Chung%2C+and+Dilek+Hakkani%2DTur%2E+2019%2E+Dialog+state+tracking%3A+A+neural+reading+comprehension+approach%2E+In+SIGDIAL%2E) [our card](https://md.research.elamur.ai/free/en/papers/gao-2019-dialog-state-tracking-neural-reading/)
9. Rahul Goel, Shachi Paul, and Dilek Hakkani-Tür. 2019. Hyst: A hybrid approach for flexible and accurate dialogue state tracking. In Interspeech. [find](https://scholar.google.com/scholar?q=Rahul+Goel%2C+Shachi+Paul%2C+and+Dilek+Hakkani%2DTu%CC%88r%2E+2019%2E+Hyst%3A+A+hybrid+approach+for+flexible+and+accurate+dialogue+state+tracking%2E+In+Interspeech%2E)
10. Hung Le, Doyen Sahoo, Chenghao Liu, Nancy F. Chen, and Steven C.H. Hoi. 2020a. End-to-end multi-domain task-oriented dialogue systems with multi-level neural belief tracker. In Submitted to ICLR 2020. [find](https://scholar.google.com/scholar?q=Hung+Le%2C+Doyen+Sahoo%2C+Chenghao+Liu%2C+Nancy+F%2E+Chen%2C+and+Steven+C%2EH%2E+Hoi%2E+2020a%2E+End%2Dto%2Dend+multi%2Ddomain+task%2Doriented+dialogue+systems+with+multi%2Dlevel+neural+belief+tracker%2E+In+Submitted+to+ICLR+2020%2E)
11. Hung Le, Richard Socher, and Steven C.H. Hoi. 2020b. Non-autoregressive dialog state tracking. In ICLR. [find](https://scholar.google.com/scholar?q=Hung+Le%2C+Richard+Socher%2C+and+Steven+C%2EH%2E+Hoi%2E+2020b%2E+Non%2Dautoregressive+dialog+state+tracking%2E+In+ICLR%2E)
12. Hwaran Lee, Jinsik Lee, and Tae-Yoon Kim. 2019. Sumbt: Slot-utterance matching for universal and scalable belief tracking. In ACL. [find](https://scholar.google.com/scholar?q=Hwaran+Lee%2C+Jinsik+Lee%2C+and+Tae%2DYoon+Kim%2E+2019%2E+Sumbt%3A+Slot%2Dutterance+matching+for+universal+and+scalable+belief+tracking%2E+In+ACL%2E) [our card](https://md.research.elamur.ai/free/en/papers/lee-2019-sumbt-slot-utterance-matching-universal/)
13. Wenqiang Lei, Xisen Jin, Min-Yen Kan, Zhaochun Ren, Xiangnan He, and Dawei Yin. 2018. Sequicity: Simplifying task-oriented dialogue systems with single sequence-to-sequence architectures. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Paper… [find](https://scholar.google.com/scholar?q=Wenqiang+Lei%2C+Xisen+Jin%2C+Min%2DYen+Kan%2C+Zhaochun+Ren%2C+Xiangnan+He%2C+and+Dawei+Yin%2E+2018%2E+Sequicity%3A+Simplifying+task%2Doriented+dialogue+systems+with+single+sequence%2Dto%2Dsequence+architectures%2E+In+Proceedin) [our card](https://md.research.elamur.ai/free/en/papers/lei-2018-sequicity-simplifying-task-oriented-dialogue/)
14. Nikola Mrkšić, Diarmuid Ó Séaghdha, Tsung-Hsien Wen, Blaise Thomson, and Steve Young. 2017. Neural belief tracker: Data-driven dialogue state tracking. In ACL. [find](https://scholar.google.com/scholar?q=Nikola+Mrks%CC%8Cic%CC%81%2C+Diarmuid+O%CC%81+Se%CC%81aghdha%2C+Tsung%2DHsien+Wen%2C+Blaise+Thomson%2C+and+Steve+Young%2E+2017%2E+Neural+belief+tracker%3A+Data%2Ddriven+dialogue+state+tracking%2E+In+ACL%2E)
15. Elnaz Nouri and Ehsan Hosseini-Asl. 2018. Toward scalable neural dialogue state tracking model. In 2nd Conversational AI workshop on NeurIPS 2018. [find](https://scholar.google.com/scholar?q=Elnaz+Nouri+and+Ehsan+Hosseini%2DAsl%2E+2018%2E+Toward+scalable+neural+dialogue+state+tracking+model%2E+In+2nd+Conversational+AI+workshop+on+NeurIPS+2018%2E)
16. Liliang Ren, Jianmo Ni, and Julian McAuley. 2019. Scalable and accurate dialogue state tracking via hierarchical sequence generation. In EMNLP-IJCNLP. [find](https://scholar.google.com/scholar?q=Liliang+Ren%2C+Jianmo+Ni%2C+and+Julian+McAuley%2E+2019%2E+Scalable+and+accurate+dialogue+state+tracking+via+hierarchical+sequence+generation%2E+In+EMNLP%2DIJCNLP%2E)
17. Abigail See, Peter J Liu, and Christopher D Manning. 2017. Get to the point: Summarization with pointer-generator networks. In ACL. [find](https://scholar.google.com/scholar?q=Abigail+See%2C+Peter+J+Liu%2C+and+Christopher+D+Manning%2E+2017%2E+Get+to+the+point%3A+Summarization+with+pointer%2Dgenerator+networks%2E+In+ACL%2E)
18. Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. 2014. Dropout: a simple way to prevent neural networks from overfitting. JMLR, 15(1):1929–1958. [find](https://scholar.google.com/scholar?q=Nitish+Srivastava%2C+Geoffrey+Hinton%2C+Alex+Krizhevsky%2C+Ilya+Sutskever%2C+and+Ruslan+Salakhutdinov%2E+2014%2E+Dropout%3A+a+simple+way+to+prevent+neural+networks+from+overfitting%2E+JMLR%2C+15%281%29%3A1929%E2%80%931958%2E)
19. Nako Sung, Minkyu Kim, Hyunwoo Jo, Youngil Yang, Jingwoong Kim, Leonard Lausen, Youngkwan Kim, Gayoung Lee, Donghyun Kwak, Jung-Woo Ha, et al. 2017. Nsml: A machine learning platform that enables you to focus on your models. arXiv preprint arXiv:1712.05902. [find](https://scholar.google.com/scholar?q=Nako+Sung%2C+Minkyu+Kim%2C+Hyunwoo+Jo%2C+Youngil+Yang%2C+Jingwoong+Kim%2C+Leonard+Lausen%2C+Youngkwan+Kim%2C+Gayoung+Lee%2C+Donghyun+Kwak%2C+Jung%2DWoo+Ha%2C+et+al%2E+2017%2E+Nsml%3A+A+machine+learning+platform+that+enables+you+)
20. Matthew Henderson, Blaise Thomson, and Jason D Williams. 2014. The second dialog state tracking challenge. In SIGDIAL. [find](https://scholar.google.com/scholar?q=Matthew+Henderson%2C+Blaise+Thomson%2C+and+Jason+D+Williams%2E+2014%2E+The+second+dialog+state+tracking+challenge%2E+In+SIGDIAL%2E)
21. Tsung-Hsien Wen, David Vandyke, Nikola Mrkšić, Milica Gasic, Lina M Rojas Barahona, Pei-Hao Su, Stefan Ultes, and Steve Young. 2017. A network-based end-to-end trainable task-oriented dialogue system. In EACL. [find](https://scholar.google.com/scholar?q=Tsung%2DHsien+Wen%2C+David+Vandyke%2C+Nikola+Mrks%CC%8Cic%CC%81%2C+Milica+Gasic%2C+Lina+M+Rojas+Barahona%2C+Pei%2DHao+Su%2C+Stefan+Ultes%2C+and+Steve+Young%2E+2017%2E+A+network%2Dbased+end%2Dto%2Dend+trainable+task%2Doriented+dialogue+syste)
22. Hanjoo Kim, Minkyu Kim, Dongjoo Seo, Jinwoong Kim, Heungseok Park, Soeun Park, Hyunwoo Jo, KyungHyun Kim, Youngil Yang, Youngkwan Kim, et al. 2018. Nsml: Meet the mlaas platform with a real-world case study. arXiv preprint arXiv:1810.09957. — [arXiv:1810.09957](https://arxiv.org/abs/1810.09957) [find](https://scholar.google.com/scholar?q=Hanjoo+Kim%2C+Minkyu+Kim%2C+Dongjoo+Seo%2C+Jinwoong+Kim%2C+Heungseok+Park%2C+Soeun+Park%2C+Hyunwoo+Jo%2C+KyungHyun+Kim%2C+Youngil+Yang%2C+Youngkwan+Kim%2C+et+al%2E+2018%2E+Nsml%3A+Meet+the+mlaas+platform+with+a+real%2Dworld+case)
23. Chien-Sheng Wu, Andrea Madotto, Ehsan Hosseini-Asl, Caiming Xiong, Richard Socher, and Pascale Fung. 2019. Transferable multi-domain state generator for task-oriented dialogue systems. In ACL. [find](https://scholar.google.com/scholar?q=Chien%2DSheng+Wu%2C+Andrea+Madotto%2C+Ehsan+Hosseini%2DAsl%2C+Caiming+Xiong%2C+Richard+Socher%2C+and+Pascale+Fung%2E+2019%2E+Transferable+multi%2Ddomain+state+generator+for+task%2Doriented+dialogue+systems%2E+In+ACL%2E)
24. Diederik P Kingma and Jimmy Ba. 2015. Adam: A method for stochastic optimization. In ICLR. [find](https://scholar.google.com/scholar?q=Diederik+P+Kingma+and+Jimmy+Ba%2E+2015%2E+Adam%3A+A+method+for+stochastic+optimization%2E+In+ICLR%2E)
25. Puyang Xu and Qi Hu. 2018. An end-to-end approach for handling unknown slot values in dialogue state tracking. In ACL. [find](https://scholar.google.com/scholar?q=Puyang+Xu+and+Qi+Hu%2E+2018%2E+An+end%2Dto%2Dend+approach+for+handling+unknown+slot+values+in+dialogue+state+tracking%2E+In+ACL%2E)
26. Jian-Guo Zhang, Kazuma Hashimoto, Chien-Sheng Wu, Philip S. Yu, Richard Socher, and Caiming Xiong. 2019. Find or classify? dual strategy for slot-value predictions on multi-domain dialog state tracking. arXiv preprint arXiv:1910.03544. [find](https://scholar.google.com/scholar?q=Jian%2DGuo+Zhang%2C+Kazuma+Hashimoto%2C+Chien%2DSheng+Wu%2C+Philip+S%2E+Yu%2C+Richard+Socher%2C+and+Caiming+Xiong%2E+2019%2E+Find+or+classify%3F+dual+strategy+for+slot%2Dvalue+predictions+on+multi%2Ddomain+dialog+state+trackin) [our card](https://md.research.elamur.ai/free/en/papers/zhang-2020-find-or-classify-dual-strategy/)
27. Victor Zhong, Caiming Xiong, and Richard Socher. 2018. Global-locally self-attentive encoder for dialogue state tracking. In ACL. [find](https://scholar.google.com/scholar?q=Victor+Zhong%2C+Caiming+Xiong%2C+and+Richard+Socher%2E+2018%2E+Global%2Dlocally+self%2Dattentive+encoder+for+dialogue+state+tracking%2E+In+ACL%2E)