---
title: "Large-Scale Multi-Domain Belief Tracking with Knowledge Sharing"
description: "The authors propose a tracker that compares encoded user and system utterances… 93.7% accuracy. 17 references, cited by 55 papers here."
authors: ["Osman Ramadan", "Paweł Budzianowski", "Milica Gašić"]
year: 2018
venue: "Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)"
source: "https://aclanthology.org/P18-2069/"
doi: "10.18653/v1/p18-2069"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/ramadan-2018-large-scale-multi-domain-belief/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/ramadan-2018-large-scale-multi-domain-belief/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2018](https://md.research.elamur.ai/free/en/papers/2018/page/3/) / Large-Scale Multi-Domain Belief Tracking with Knowledge Sharing

# Large-Scale Multi-Domain Belief Tracking with Knowledge Sharing

**Authors:** Osman Ramadan, Paweł Budzianowski and [Milica Gašić](https://md.research.elamur.ai/free/en/authors/milica-gasic/) · **Year:** 2018 · **Venue:** [Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)](https://md.research.elamur.ai/free/en/venues/acl/) ([ACL 2018](https://md.research.elamur.ai/free/en/venues/acl/2018/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/P18-2069/) · [PDF](https://aclanthology.org/P18-2069.pdf) · [DOI 10.18653/v1/p18-2069](https://doi.org/10.18653/v1/p18-2069) · [OpenAlex](https://openalex.org/W2798434138)

**Citations:** 136 citations (OpenAlex, 2026-10-14) · [55](https://md.research.elamur.ai/free/en/papers/ramadan-2018-large-scale-multi-domain-belief/cited-by/) in our library, of the 141,840 papers whose bibliographies we read

## Summary
The paper tackles dialogue belief tracking when a conversation can move between several domains, such as hotels, restaurants, trains and taxis. Earlier trackers either depend on hand-built semantic dictionaries for delexicalisation or, like the Neural Belief Tracker (NBT), train each slot separately with parameters tied to the ontology. Both approaches scale poorly as domains are added. The authors propose a tracker that compares encoded user and system utterances with embeddings of ontology terms (domains, slots, values) in a shared semantic space. Because of this design, the parameter count does not grow with the ontology, and knowledge carries over between slots and domains. Domain detection and slot-value tracking are trained separately and then combined. A learned recurrent update across turns replaces hand-tuned update rules. The authors also release a new human-human Wizard-of-Oz corpus that is roughly ten times larger than earlier resources. The model beats NBT on single-domain tracking and handles multi-domain dialogues well, which makes it a practical basis for scalable task-oriented systems.

## Key points
- **New corpus:** It contains 2480 single-domain and 7375 multi-domain dialogues, each covering 1 to 5 domains. The combined ontology has 5 domains, 27 slots and 663 values. Test and development sets hold 1k dialogues each.
- **Architecture:**
  - Encoders are either Bi-LSTMs or CNNs built on semantically specialised word embeddings, with 7 separate encoders in the Bi-LSTM variant.
  - Inform, request and confirm cases are modelled separately.
  - An RNN with a memory cell handles belief updates across turns.
- **WOZ 2.0 results (joint goal accuracy):** NBT-CNN reaches 84.2, the Bi-LSTM variant 85.1 and the CNN variant 85.5.
- **New corpus, restaurant-only results (joint goal accuracy):** NBT-CNN reaches 57.7, Bi-LSTM 59.9 and CNN 63.7. Scores are lower than on WOZ 2.0, which the authors attribute to noisier, richer dialogues.
- **Multi-domain test results:** The Bi-LSTM variant scores F1 0.876 with 93.7% accuracy, and the CNN variant scores F1 0.878 with 93.2% accuracy. A uniform random baseline reaches only F1 0.108. The data and model are publicly available.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [University of Cambridge](https://md.research.elamur.ai/free/en/universities/university-of-cambridge/) — University, Cambridge, United Kingdom · 155 works · 115,562 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 9,085 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 8,888 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 9,296 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 9,206 works

🧭 Topics (1): [🕸️ Knowledge graphs & ontologies](https://md.research.elamur.ai/free/en/topics/knowledge-graphs/) · 103 works

## Cite

Cite as: Osman Ramadan, Paweł Budzianowski and Milica Gašić. 2018. Large-Scale Multi-Domain Belief Tracking with Knowledge Sharing. Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers).

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{ramadan2018large,
  title = {{Large-Scale Multi-Domain Belief Tracking with Knowledge Sharing}},
  author = {Osman Ramadan, Paweł Budzianowski and Milica Gašić},
  booktitle = {Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)},
  year = {2018},
  doi = {10.18653/v1/p18-2069},
  url = {https://doi.org/10.18653/v1/p18-2069}
}
```

- [← previous Fine-Grained Arabic Dialect Identification](https://md.research.elamur.ai/free/en/papers/salameh-2018-fine-grained-arabic-dialect-identification/)
- [next → OneStopEnglish corpus: A new corpus for automatic readability assessment and text simplification](https://md.research.elamur.ai/free/en/papers/vajjala-2018-onestopenglish-corpus-new-corpus-automatic/)
- [↑ back to papers of 2018](https://md.research.elamur.ai/free/en/papers/2018/page/3/)

## References

17 references, 4 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Alex Graves and Jürgen Schmidhuber. 2005. Frame-wise phoneme classification with bidirectional lstm and other neural network architectures. Neural Networks 18(5-6):602–610. [find](https://scholar.google.com/scholar?q=Alex+Graves+and+Ju%CC%88rgen+Schmidhuber%2E+2005%2E+Frame%2Dwise+phoneme+classification+with+bidirectional+lstm+and+other+neural+network+architectures%2E+Neural+Networks+18%285%2D6%29%3A602%E2%80%93610%2E)
2. Tsung-Hsien Wen, Milica Gašić, Nikola Mrkšić, Lina M. Rojas-Barahona, Pei-Hao Su, Stefan Ultes, David Vandyke, and Steve Young. 2017. A network-based end-to-end trainable task-oriented dialogue system. In Proceedings on EACL . [find](https://scholar.google.com/scholar?q=Tsung%2DHsien+Wen%2C+Milica+Gas%CC%8Cic%CC%81%2C+Nikola+Mrks%CC%8Cic%CC%81%2C+Lina+M%2E+Rojas%2DBarahona%2C+Pei%2DHao+Su%2C+Stefan+Ultes%2C+David+Vandyke%2C+and+Steve+Young%2E+2017%2E+A+network%2Dbased+end%2Dto%2Dend+trainable+task%2Doriented+dialogue+sy)
3. Matthew Henderson, Blaise Thomson, and Steve Young. 2014a. Robust dialog state tracking using delexicalised recurrent neural networks and unsupervised adaptation. In Spoken Language Technology Workshop (SLT), 2014 IEEE. IEEE, pages 360– 365. [find](https://scholar.google.com/scholar?q=Matthew+Henderson%2C+Blaise+Thomson%2C+and+Steve+Young%2E+2014a%2E+Robust+dialog+state+tracking+using+delexicalised+recurrent+neural+networks+and+unsupervised+adaptation%2E+In+Spoken+Language+Technology+Worksho)
4. John Wieting, Mohit Bansal, Kevin Gimpel, Karen Livescu, and Dan Roth. 2015. [From paraphrase database to compositional paraphrase model and back](https://md.research.elamur.ai/free/en/papers/wieting-2015-paraphrase-database-compositional-paraphrase-model/). Transactions of the Association for Computational Linguistics 3:345–358.
5. Jason Williams, Antoine Raux, and Matthew Henderson. 2016. The dialog state tracking challenge series: A review. Dialogue & Discourse 7(3):4–33. [find](https://scholar.google.com/scholar?q=Jason+Williams%2C+Antoine+Raux%2C+and+Matthew+Henderson%2E+2016%2E+The+dialog+state+tracking+challenge+series%3A+A+review%2E+Dialogue+%26+Discourse+7%283%29%3A4%E2%80%9333%2E)
6. Matthew Henderson, Blaise Thomson, and Steve Young. 2014b. [Word-based dialog state tracking with recurrent neural networks](https://md.research.elamur.ai/free/en/papers/henderson-2014-word-based-dialog-state-tracking/). In Proceedings of the 15th Annual Meeting of the Special Interest Group on Discourse and Dialogue (SIGDIAL). pages 292–299.
7. Jason D Williams. 2014. Web-style ranking and slu combination for dialog state tracking. In Proceedings of the 15th Annual Meeting of the Special Interest Group on Discourse and Dialogue (SIGDIAL). pages 282–291. [find](https://scholar.google.com/scholar?q=Jason+D+Williams%2E+2014%2E+Web%2Dstyle+ranking+and+slu+combination+for+dialog+state+tracking%2E+In+Proceedings+of+the+15th+Annual+Meeting+of+the+Special+Interest+Group+on+Discourse+and+Dialogue+%28SIGDIAL%29%2E+pa)
8. Nal Kalchbrenner, Edward Grefenstette, and Phil Blunsom. 2014. [A convolutional neural network for modelling sentences](https://md.research.elamur.ai/free/en/papers/kalchbrenner-2014-convolutional-neural-network-modelling-sentences/). In Proceedings of ACL .
9. Wenpeng Yin, Katharina Kann, Mo Yu, and Hinrich Schütze. 2017. Comparative study of cnn and rnn for natural language processing. arXiv preprint arXiv:1702.01923 . [find](https://scholar.google.com/scholar?q=Wenpeng+Yin%2C+Katharina+Kann%2C+Mo+Yu%2C+and+Hinrich+Schu%CC%88tze%2E+2017%2E+Comparative+study+of+cnn+and+rnn+for+natural+language+processing%2E+arXiv+preprint+arXiv%3A1702%2E01923+%2E)
10. John F Kelley. 1984. An iterative design methodology for user-friendly natural language office information applications. ACM Transactions on Information Systems (TOIS) 2(1):26–41. [find](https://scholar.google.com/scholar?q=John+F+Kelley%2E+1984%2E+An+iterative+design+methodology+for+user%2Dfriendly+natural+language+office+information+applications%2E+ACM+Transactions+on+Information+Systems+%28TOIS%29+2%281%29%3A26%E2%80%9341%2E)
11. Steve Young, Milica Gašić, Simon Keizer, François Mairesse, Jost Schatzmann, Blaise Thomson, and Kai Yu. 2010. The hidden information state model: A practical framework for pomdp-based spoken dialogue management. Computer Speech & Language 24(2):150–174. [find](https://scholar.google.com/scholar?q=Steve+Young%2C+Milica+Gas%CC%8Cic%CC%81%2C+Simon+Keizer%2C+Franc%CC%A7ois+Mairesse%2C+Jost+Schatzmann%2C+Blaise+Thomson%2C+and+Kai+Yu%2E+2010%2E+The+hidden+information+state+model%3A+A+practical+framework+for+pomdp%2Dbased+spoken+dialo)
12. Yoon Kim. 2014. [Convolutional neural networks for sentence classification](https://md.research.elamur.ai/free/en/papers/kim-2014-convolutional-neural-networks-sentence-classification/). In Proceedings of EMNLP .
13. Diederik P Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. ICLR . [find](https://scholar.google.com/scholar?q=Diederik+P+Kingma+and+Jimmy+Ba%2E+2014%2E+Adam%3A+A+method+for+stochastic+optimization%2E+ICLR+%2E)
14. Lukas Zilka and Filip Jurcicek. 2015. Incremental lstm-based dialog state tracker. In Automatic Speech Recognition and Understanding (ASRU), 2015 IEEE Workshop on. IEEE, pages 757–762. [find](https://scholar.google.com/scholar?q=Lukas+Zilka+and+Filip+Jurcicek%2E+2015%2E+Incremental+lstm%2Dbased+dialog+state+tracker%2E+In+Automatic+Speech+Recognition+and+Understanding+%28ASRU%29%2C+2015+IEEE+Workshop+on%2E+IEEE%2C+pages+757%E2%80%93762%2E)
15. Nikola Mrkšić, Diarmuid Ó Séaghdha, Tsung-Hsien Wen, Blaise Thomson, and Steve Young. 2017. Neural belief tracker: Data-driven dialogue state tracking. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). volume 1, pages 1777– 1788. [find](https://scholar.google.com/scholar?q=Nikola+Mrks%CC%8Cic%CC%81%2C+Diarmuid+O%CC%81+Se%CC%81aghdha%2C+Tsung%2DHsien+Wen%2C+Blaise+Thomson%2C+and+Steve+Young%2E+2017%2E+Neural+belief+tracker%3A+Data%2Ddriven+dialogue+state+tracking%2E+In+Proceedings+of+the+55th+Annual+Meeting+of)
16. Nikola Mrkvsić and Ivan Vulić. 2018. Fully statistical neural belief tracking. In Proceedings of ACL. [find](https://scholar.google.com/scholar?q=Nikola+Mrkvsic%CC%81+and+Ivan+Vulic%CC%81%2E+2018%2E+Fully+statistical+neural+belief+tracking%2E+In+Proceedings+of+ACL%2E)
17. Julien Perez and Fei Liu. 2016. Dialog state tracking, a machine reading approach using memory network. arXiv preprint arXiv:1606.04052 . [find](https://scholar.google.com/scholar?q=Julien+Perez+and+Fei+Liu%2E+2016%2E+Dialog+state+tracking%2C+a+machine+reading+approach+using+memory+network%2E+arXiv+preprint+arXiv%3A1606%2E04052+%2E)

*Updated 24 October 2026*