---
title: "A Simple Language Model for Task-Oriented Dialogue (2020)"
description: "Researchers from Salesforce Research propose SimpleTOD, a single causal language model that… 64 references, cited by 255 papers here."
authors: ["Ehsan Hosseini-Asl", "Bryan McCann", "Chien-Sheng Wu", "Semih Yavuz", "Richard Socher"]
year: 2020
venue: "arXiv:2005.00796"
source: "https://arxiv.org/abs/2005.00796"
doi: "10.48550/arxiv.2005.00796"
arxiv: "2005.00796"
licence: "arXiv — non-exclusive licence to distribute"
licence_url: "http://arxiv.org/licenses/nonexclusive-distrib/1.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/hosseini-asl-2020-simple-language-model-task-oriented/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/hosseini-asl-2020-simple-language-model-task-oriented/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2020](https://md.research.elamur.ai/free/en/papers/2020/page/2/) / A Simple Language Model for Task-Oriented Dialogue

# A Simple Language Model for Task-Oriented Dialogue

**Authors:** [Ehsan Hosseini-Asl](https://md.research.elamur.ai/free/en/authors/ehsan-hosseini-asl/), [Bryan McCann](https://md.research.elamur.ai/free/en/authors/bryan-mccann/), [Chien-Sheng Wu](https://md.research.elamur.ai/free/en/authors/chien-sheng-wu/), [Semih Yavuz](https://md.research.elamur.ai/free/en/authors/semih-yavuz/) and [Richard Socher](https://md.research.elamur.ai/free/en/authors/richard-socher/) · **Year:** 2020 · **Venue:** arXiv:2005.00796 · **Licence:** [arXiv — non-exclusive licence to distribute](http://arxiv.org/licenses/nonexclusive-distrib/1.0/)

**Links:** [Publisher's page](https://arxiv.org/abs/2005.00796) · [PDF](https://arxiv.org/pdf/2005.00796) · [DOI 10.48550/arxiv.2005.00796](https://doi.org/10.48550/arxiv.2005.00796) · [arXiv 2005.00796](https://arxiv.org/abs/2005.00796) · [OpenAlex](https://openalex.org/W3021016503)

**Citations:** 144 citations (OpenAlex, 2026-10-14) · [255](https://md.research.elamur.ai/free/en/papers/hosseini-asl-2020-simple-language-model-task-oriented/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
Dialogue systems that help a user accomplish a specific task (task-oriented dialogue, TOD) are usually built as a pipeline of separate modules. One module tracks the dialogue state (belief state), a second selects system actions, and a third generates the response. Because of this division, errors from one module propagate to the subsequent ones. Researchers from Salesforce Research propose SimpleTOD, a single causal language model that solves all three subtasks as a single sequence prediction task. The model reads the dialogue history and generates the belief state. It then receives aggregated results of the database query and generates the actions and a delexicalized response. The weights are taken from GPT-2 or DistilGPT2, and the model is fine-tuned on MultiWOZ 2.1. Greedy decoding is sufficient for generation. SimpleTOD outperformed previous methods both in dialogue state tracking and in end-to-end evaluation of actions and responses. Thus, the knowledge the model acquired during pre-training on general text transfers well to target dialogues, and a cumbersome modular architecture can be replaced with a simple one. The authors released the code, a list of erroneous annotations in MultiWOZ, and a cleaned version of the test set.

## Key points
- Joint goal accuracy on MultiWOZ 2.1 is 55.76, the best result for state tracking at the time of publication. In the end-to-end setting, the inform rate increased by 8.1 points, the success rate by 9.7, and the combined score by 7.2.
- Special tokens are critically important. Without end-of-segment markers and without user/system turn tagging, joint accuracy drops to 16.79, and the combined score falls from 92.98 to 26.73.
- Without pre-training, the model performs much worse: a 12-layer model with random weights achieves a joint accuracy of 20.17 versus 55.76 for GPT-2. A larger model is not always better: DistilGPT2 yields a combined score of 92.98, while GPT-2 yields 90.75.
- The authors identified four types of erroneous annotations in MultiWOZ 2.1 and showed that SimpleTOD is robust to such noise.
- The code, the cleaned test set, and the list of annotation errors were released on GitHub: salesforce/simpletod. The paper was published at NeurIPS 2020.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Salesforce](https://md.research.elamur.ai/free/en/companies/salesforce/) — Company, San Francisco, United States · 56 works · 17,867 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works

🧭 Topics (2): [🧠 Language models & pretraining](https://md.research.elamur.ai/free/en/topics/language-models/) · 3,320 works · [🗨️ Dialogue systems](https://md.research.elamur.ai/free/en/topics/dialogue-systems/) · 624 works

💡 Concepts (1): [💡 Chatbots](https://md.research.elamur.ai/free/en/concepts/chatbot/) · 54 works

## Cite

Cite as: Ehsan Hosseini-Asl, Bryan McCann, Chien-Sheng Wu, Semih Yavuz and Richard Socher. 2020. A Simple Language Model for Task-Oriented Dialogue. arXiv:2005.00796.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@misc{hosseiniasl2020simple,
  title = {{A Simple Language Model for Task-Oriented Dialogue}},
  author = {Ehsan Hosseini-Asl, Bryan McCann, Chien-Sheng Wu, Semih Yavuz and Richard Socher},
  year = {2020},
  doi = {10.48550/arxiv.2005.00796},
  eprint = {2005.00796},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2005.00796}
}
```

- [← previous Findings of the 2020 Conference on Machine Translation (WMT20)](https://md.research.elamur.ai/free/en/papers/barrault-2020-findings-2020-conference-machine-translation/)
- [next → Dependency Graph Enhanced Dual-transformer Structure for Aspect-based Sentiment Classification](https://md.research.elamur.ai/free/en/papers/tang-2020-dependency-graph-enhanced-dual-transformer/)
- [↑ back to papers of 2020](https://md.research.elamur.ai/free/en/papers/2020/page/2/)

## References

64 references, 32 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. D. Adiwardana, M.-T. Luong, D. R. So, J. Hall, N. Fiedel, R. Thoppilan, Z. Yang, A. Kulshreshtha, G. Nemade, Y. Lu, et al. [Towards a human-like open-domain chatbot](https://md.research.elamur.ai/free/en/papers/adiwardana-2020-towards-human-like-open-domain/). arXiv preprint arXiv:2001.09977, 2020.
2. J. Ba, R. Kiros, and G. E. Hinton. [Layer normalization](https://md.research.elamur.ai/free/en/papers/ba-2016-layer-normalization/). CoRR, abs/1607.06450, 2016.
3. S. Bao, H. He, F. Wang, and H. Wu. [Plato: Pre-trained dialogue generation model with discrete latent variable](https://md.research.elamur.ai/free/en/papers/bao-2020-plato-pre-trained-dialogue-generation/). arXiv preprint arXiv:1910.07931, 2019.
4. Y. Bengio, R. Ducharme, P. Vincent, and C. Jauvin. [A neural probabilistic language model](https://md.research.elamur.ai/free/en/papers/bengio-2003-neural-probabilistic-language-model/). Journal of machine learning research, 3(Feb):1137–1155, 2003.
5. A. Bordes, Y.-L. Boureau, and J. Weston. Learning end-to-end goal-oriented dialog. In International Conference on Learning Representations, 2017. [find](https://scholar.google.com/scholar?q=A%2E+Bordes%2C+Y%2E%2DL%2E+Boureau%2C+and+J%2E+Weston%2E+Learning+end%2Dto%2Dend+goal%2Doriented+dialog%2E+In+International+Conference+on+Learning+Representations%2C+2017%2E)
6. P. Budzianowski and I. Vulić. Hello, it’s gpt-2–how can i help you? towards the use of pretrained language models for task-oriented dialogue systems. arXiv preprint arXiv:1907.05774, 2019. [find](https://scholar.google.com/scholar?q=P%2E+Budzianowski+and+I%2E+Vulic%CC%81%2E+Hello%2C+it%E2%80%99s+gpt%2D2%E2%80%93how+can+i+help+you%3F+towards+the+use+of+pretrained+language+models+for+task%2Doriented+dialogue+systems%2E+arXiv+preprint+arXiv%3A1907%2E05774%2C+2019%2E)
7. P. Budzianowski, I. Casanueva, B.-H. Tseng, and M. Gasic. Towards end-to-end multi-domain dialogue modelling. 2018. [find](https://scholar.google.com/scholar?q=P%2E+Budzianowski%2C+I%2E+Casanueva%2C+B%2E%2DH%2E+Tseng%2C+and+M%2E+Gasic%2E+Towards+end%2Dto%2Dend+multi%2Ddomain+dialogue+modelling%2E+2018%2E)
8. L. Chen, B. Lv, C. Wang, S. Zhu, B. Tan, and K. Yu. Schema-guided multi-domain dialogue state tracking with graph attention neural networks. 2020. [find](https://scholar.google.com/scholar?q=L%2E+Chen%2C+B%2E+Lv%2C+C%2E+Wang%2C+S%2E+Zhu%2C+B%2E+Tan%2C+and+K%2E+Yu%2E+Schema%2Dguided+multi%2Ddomain+dialogue+state+tracking+with+graph+attention+neural+networks%2E+2020%2E)
9. W. Chen, J. Chen, P. Qin, X. Yan, and W. Y. Wang. Semantically conditioned dialog response generation via hierarchical disentangled self-attention. arXiv preprint arXiv:1905.12866, 2019. [find](https://scholar.google.com/scholar?q=W%2E+Chen%2C+J%2E+Chen%2C+P%2E+Qin%2C+X%2E+Yan%2C+and+W%2E+Y%2E+Wang%2E+Semantically+conditioned+dialog+response+generation+via+hierarchical+disentangled+self%2Dattention%2E+arXiv+preprint+arXiv%3A1905%2E12866%2C+2019%2E)
10. R. Child, S. Gray, A. Radford, and I. Sutskever. [Generating long sequences with sparse transformers](https://md.research.elamur.ai/free/en/papers/child-2019-generating-long-sequences-sparse-transformers/). arXiv preprint arXiv:1904.10509, 2019.
11. J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova. [Bert: Pre-training of deep bidirectional transformers for language understanding](https://md.research.elamur.ai/free/en/papers/devlin-2019-bert-pre-training-deep-bidirectional/). arXiv preprint arXiv:1810.04805, 2018.
12. L. Dong, N. Yang, W. Wang, F. Wei, X. Liu, Y. Wang, J. Gao, M. Zhou, and H.-W. Hon. [Unified language model pre-training for natural language understanding and generation](https://md.research.elamur.ai/free/en/papers/dong-2019-unified-language-model-pre-training/). In Advances in Neural Information Processing Systems, pages 13042–13054, 2019.
13. M. Eric and C. D. Manning. [Key-value retrieval networks for task-oriented dialogue](https://md.research.elamur.ai/free/en/papers/eric-2017-key-value-retrieval-networks-task/). arXiv preprint arXiv:1705.05414, 2017.
14. M. Eric, R. Goel, S. Paul, A. Sethi, S. Agarwal, S. Gao, and D. Hakkani-Tur. Multiwoz 2.1: Multi-domain dialogue state corrections and state tracking baselines. arXiv preprint arXiv:1907.01669, 2019. [find](https://scholar.google.com/scholar?q=M%2E+Eric%2C+R%2E+Goel%2C+S%2E+Paul%2C+A%2E+Sethi%2C+S%2E+Agarwal%2C+S%2E+Gao%2C+and+D%2E+Hakkani%2DTur%2E+Multiwoz+2%2E1%3A+Multi%2Ddomain+dialogue+state+corrections+and+state+tracking+baselines%2E+arXiv+preprint+arXiv%3A1907%2E01669%2C+2019%2E)
15. J. Gao, M. Galley, L. Li, et al. Neural approaches to conversational ai. Foundations and Trends® in Information Retrieval, 13(2-3):127–298, 2019. [find](https://scholar.google.com/scholar?q=J%2E+Gao%2C+M%2E+Galley%2C+L%2E+Li%2C+et+al%2E+Neural+approaches+to+conversational+ai%2E+Foundations+and+Trends%C2%AE+in+Information+Retrieval%2C+13%282%2D3%29%3A127%E2%80%93298%2C+2019%2E)
16. S. Gao, Y. Zhang, Z. Ou, and Z. Yu. Paraphrase augmented task-oriented dialog generation. arXiv preprint arXiv:2004.07462, 2020. [find](https://scholar.google.com/scholar?q=S%2E+Gao%2C+Y%2E+Zhang%2C+Z%2E+Ou%2C+and+Z%2E+Yu%2E+Paraphrase+augmented+task%2Doriented+dialog+generation%2E+arXiv+preprint+arXiv%3A2004%2E07462%2C+2020%2E)
17. D. Ham, J.-G. Lee, Y. Jang, and K.-E. Kim. End-to-end neural pipeline for goal-oriented dialogue system using gpt-2. ACL, 2020. [find](https://scholar.google.com/scholar?q=D%2E+Ham%2C+J%2E%2DG%2E+Lee%2C+Y%2E+Jang%2C+and+K%2E%2DE%2E+Kim%2E+End%2Dto%2Dend+neural+pipeline+for+goal%2Doriented+dialogue+system+using+gpt%2D2%2E+ACL%2C+2020%2E)
18. K. He, X. Zhang, S. Ren, and J. Sun. [Deep residual learning for image recognition](https://md.research.elamur.ai/free/en/papers/he-2015-deep-residual-learning-image-recognition/). In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770–778, 2016.
19. M. Heck, C. van Niekerk, N. Lubis, C. Geishauser, H.-C. Lin, M. Moresi, and M. Gašić. Trippy: A triple copy strategy for value independent neural dialog state tracking. arXiv preprint arXiv:2005.02877, 2020. [find](https://scholar.google.com/scholar?q=M%2E+Heck%2C+C%2E+van+Niekerk%2C+N%2E+Lubis%2C+C%2E+Geishauser%2C+H%2E%2DC%2E+Lin%2C+M%2E+Moresi%2C+and+M%2E+Ga%C5%A1ic%CC%81%2E+Trippy%3A+A+triple+copy+strategy+for+value+independent+neural+dialog+state+tracking%2E+arXiv+preprint+arXiv%3A2005%2E0287)
20. M. Henderson, B. Thomson, and S. Young. Deep neural network approach for the dialog state tracking challenge. In Proceedings of the SIGDIAL 2013 Conference, 2013. [find](https://scholar.google.com/scholar?q=M%2E+Henderson%2C+B%2E+Thomson%2C+and+S%2E+Young%2E+Deep+neural+network+approach+for+the+dialog+state+tracking+challenge%2E+In+Proceedings+of+the+SIGDIAL+2013+Conference%2C+2013%2E)
21. M. Henderson, I. Casanueva, N. Mrkšić, P.-H. Su, I. Vulić, et al. [Convert: Efficient and accurate conversational representations from transformers](https://md.research.elamur.ai/free/en/papers/henderson-2020-convert-efficient-accurate-conversational-representations/). arXiv preprint arXiv:1911.03688, 2019.
22. A. Holtzman, J. Buys, L. Du, M. Forbes, and Y. Choi. [The curious case of neural text degeneration](https://md.research.elamur.ai/free/en/papers/holtzman-2019-curious-case-neural-text-degeneration/). arXiv preprint arXiv:1904.09751, 2019.
23. N. S. Keskar, B. McCann, L. R. Varshney, C. Xiong, and R. Socher. [Ctrl: A conditional transformer language model for controllable generation](https://md.research.elamur.ai/free/en/papers/keskar-2019-ctrl-conditional-transformer-language-model/). arXiv preprint arXiv:1909.05858, 2019.
24. W. Lei, X. Jin, M.-Y. Kan, Z. Ren, X. He, and D. Yin. [Sequicity: Simplifying task-oriented dialogue systems with single sequence-to-sequence architectures](https://md.research.elamur.ai/free/en/papers/lei-2018-sequicity-simplifying-task-oriented-dialogue/). In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics, 2018.
25. W. Liang, Y. Tian, C. Chen, and Z. Yu. Moss: End-to-end dialog system framework with modular supervision. arXiv preprint arXiv:1909.05528, 2019. — [arXiv:1909.05528](https://arxiv.org/abs/1909.05528) [find](https://scholar.google.com/scholar?q=W%2E+Liang%2C+Y%2E+Tian%2C+C%2E+Chen%2C+and+Z%2E+Yu%2E+Moss%3A+End%2Dto%2Dend+dialog+system+framework+with+modular+supervision%2E+arXiv+preprint+arXiv%3A1909%2E05528%2C+2019%2E+%E2%80%94+%5BarXiv%3A1909%2E05528%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F1909%2E05528%29)
26. B. Liu and I. Lane. Attention-based recurrent neural network models for joint intent detection and slot filling. In INTERSPEECH, 2016. [find](https://scholar.google.com/scholar?q=B%2E+Liu+and+I%2E+Lane%2E+Attention%2Dbased+recurrent+neural+network+models+for+joint+intent+detection+and+slot+filling%2E+In+INTERSPEECH%2C+2016%2E)
27. B. Liu and I. Lane. End-to-end learning of task-oriented dialogs. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Student Research Workshop, pages 67–73, 2018. [find](https://scholar.google.com/scholar?q=B%2E+Liu+and+I%2E+Lane%2E+End%2Dto%2Dend+learning+of+task%2Doriented+dialogs%2E+In+Proceedings+of+the+2018+Conference+of+the+North+American+Chapter+of+the+Association+for+Computational+Linguistics%3A+Student+Research)
28. B. Liu, G. Tür, D. Hakkani-Tür, P. Shah, and L. Heck. [Dialogue learning with human teaching and feedback in end-to-end trainable task-oriented dialogue systems](https://md.research.elamur.ai/free/en/papers/liu-2018-dialogue-learning-human-teaching-feedback/). In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Techno…
29. A. Madotto, C.-S. Wu, and P. Fung. [Mem2seq: Effectively incorporating knowledge bases into end-to-end task-oriented dialog systems](https://md.research.elamur.ai/free/en/papers/madotto-2018-mem2seq-effectively-incorporating-knowledge-bases/). arXiv preprint arXiv:1804.08217, 2018.
30. B. McCann, J. Bradbury, C. Xiong, and R. Socher. [Learned in translation: Contextualized word vectors](https://md.research.elamur.ai/free/en/papers/mccann-2017-learned-translation-contextualized-word-vectors/). In Advances in Neural Information Processing Systems, pages 6294–6305, 2017.
31. S. Mehri, T. Srinivasan, and M. Eskenazi. Structured fusion networks for dialog. arXiv preprint arXiv:1907.10016, 2019. [find](https://scholar.google.com/scholar?q=S%2E+Mehri%2C+T%2E+Srinivasan%2C+and+M%2E+Eskenazi%2E+Structured+fusion+networks+for+dialog%2E+arXiv+preprint+arXiv%3A1907%2E10016%2C+2019%2E)
32. N. Mrkšić, D. Ó Séaghdha, T.-H. Wen, B. Thomson, and S. Young. Neural belief tracker: Data-driven dialogue state tracking. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, 2017. [find](https://scholar.google.com/scholar?q=N%2E+Mrk%C5%A1ic%CC%81%2C+D%2E+%C3%93+S%C3%A9aghdha%2C+T%2E%2DH%2E+Wen%2C+B%2E+Thomson%2C+and+S%2E+Young%2E+Neural+belief+tracker%3A+Data%2Ddriven+dialogue+state+tracking%2E+In+Proceedings+of+the+55th+Annual+Meeting+of+the+Association+for+Computation)
33. A. Neelakantan, S. Yavuz, S. Narang, V. Prasad, B. Goodrich, D. Duckworth, C. Sankar, and X. Yan. Neural assistant: Joint action prediction, response generation, and latent knowledge reasoning. In NeurIPS 2019 Converstional AI Workshop, 2019. [find](https://scholar.google.com/scholar?q=A%2E+Neelakantan%2C+S%2E+Yavuz%2C+S%2E+Narang%2C+V%2E+Prasad%2C+B%2E+Goodrich%2C+D%2E+Duckworth%2C+C%2E+Sankar%2C+and+X%2E+Yan%2E+Neural+assistant%3A+Joint+action+prediction%2C+response+generation%2C+and+latent+knowledge+reasoning%2E+In+Neu)
34. E. Nouri and E. Hosseini-Asl. Toward scalable neural dialogue state tracking model. In NeurIPS 2018 Conversational AI Workshop, 2018. [find](https://scholar.google.com/scholar?q=E%2E+Nouri+and+E%2E+Hosseini%2DAsl%2E+Toward+scalable+neural+dialogue+state+tracking+model%2E+In+NeurIPS+2018+Conversational+AI+Workshop%2C+2018%2E)
35. K. Papineni, S. Roukos, T. Ward, and W.-J. Zhu. [Bleu: A method for automatic evaluation of machine translation](https://md.research.elamur.ai/free/en/papers/papineni-2002-bleu-method-automatic-evaluation-machine/). In ACL, 2002.
36. B. Peng, C. Li, J. Li, S. Shayandeh, L. Liden, and J. Gao. Soloist: Few-shot task-oriented dialog with a single pre-trained auto-regressive model. arXiv preprint arXiv:2005.05298, 2020. [find](https://scholar.google.com/scholar?q=B%2E+Peng%2C+C%2E+Li%2C+J%2E+Li%2C+S%2E+Shayandeh%2C+L%2E+Liden%2C+and+J%2E+Gao%2E+Soloist%3A+Few%2Dshot+task%2Doriented+dialog+with+a+single+pre%2Dtrained+auto%2Dregressive+model%2E+arXiv+preprint+arXiv%3A2005%2E05298%2C+2020%2E)
37. B. Peng, C. Zhu, C. Li, X. Li, J. Li, M. Zeng, and J. Gao. [Few-shot natural language generation for task-oriented dialog](https://md.research.elamur.ai/free/en/papers/peng-2020-few-shot-natural-language-generation/), 2020.
38. M. E. Peters, M. Neumann, M. Iyyer, M. Gardner, C. Clark, K. Lee, and L. Zettlemoyer. [Deep contextualized word representations](https://md.research.elamur.ai/free/en/papers/peters-2018-deep-contextualized-word-representations/). arXiv preprint arXiv:1802.05365, 2018.
39. A. Radford, J. Wu, R. Child, D. Luan, D. Amodei, and I. Sutskever. Language models are unsupervised multitask learners. OpenAI Blog, 1(8):9, 2019. [find](https://scholar.google.com/scholar?q=A%2E+Radford%2C+J%2E+Wu%2C+R%2E+Child%2C+D%2E+Luan%2C+D%2E+Amodei%2C+and+I%2E+Sutskever%2E+Language+models+are+unsupervised+multitask+learners%2E+OpenAI+Blog%2C+1%288%29%3A9%2C+2019%2E)
40. P. Rajpurkar, J. Zhang, K. Lopyrev, and P. Liang. [Squad: 100,000+ questions for machine comprehension of text](https://md.research.elamur.ai/free/en/papers/rajpurkar-2016-squad-100-000-questions-machine/). arXiv preprint arXiv:1606.05250, 2016.
41. A. Rastogi, D. Hakkani-Tur, and L. Heck. Scalable multi-domain dialogue state tracking. In Proceedings of IEEE ASRU, 2017. [find](https://scholar.google.com/scholar?q=A%2E+Rastogi%2C+D%2E+Hakkani%2DTur%2C+and+L%2E+Heck%2E+Scalable+multi%2Ddomain+dialogue+state+tracking%2E+In+Proceedings+of+IEEE+ASRU%2C+2017%2E)
42. S. Roller, E. Dinan, N. Goyal, D. Ju, M. Williamson, Y. Liu, J. Xu, M. Ott, K. Shuster, E. M. Smith, et al. [Recipes for building an open-domain chatbot](https://md.research.elamur.ai/free/en/papers/roller-2021-recipes-building-open-domain-chatbot/). arXiv preprint arXiv:2004.13637, 2020.
43. V. Sanh, L. Debut, J. Chaumond, and T. Wolf. Distilbert, a distilled version of bert: smaller, faster, cheaper and lighter. arXiv preprint arXiv:1910.01108, 2019. [find](https://scholar.google.com/scholar?q=V%2E+Sanh%2C+L%2E+Debut%2C+J%2E+Chaumond%2C+and+T%2E+Wolf%2E+Distilbert%2C+a+distilled+version+of+bert%3A+smaller%2C+faster%2C+cheaper+and+lighter%2E+arXiv+preprint+arXiv%3A1910%2E01108%2C+2019%2E)
44. R. Sennrich, B. Haddow, and A. Birch. [Neural machine translation of rare words with subword units](https://md.research.elamur.ai/free/en/papers/sennrich-2016-neural-machine-translation-rare-words/). In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1715–1725, Berlin, Germany, Aug. 2016. Association for Computational Linguistic…
45. L. Shu, P. Molino, M. Namazifar, B. Liu, H. Xu, H. Zheng, , and G. Tur. Incorporating the structure of the belief state in end-to-end task-oriented dialogue systems. In NeurIPS 2018 Converstional AI Workshop, 2018. [find](https://scholar.google.com/scholar?q=L%2E+Shu%2C+P%2E+Molino%2C+M%2E+Namazifar%2C+B%2E+Liu%2C+H%2E+Xu%2C+H%2E+Zheng%2C+%2C+and+G%2E+Tur%2E+Incorporating+the+structure+of+the+belief+state+in+end%2Dto%2Dend+task%2Doriented+dialogue+systems%2E+In+NeurIPS+2018+Converstional+AI+W)
46. R. W. Smith and D. R. Hipp. Spoken natural language dialog systems: A practical approach. Oxford University Press on Demand, 1994. [find](https://scholar.google.com/scholar?q=R%2E+W%2E+Smith+and+D%2E+R%2E+Hipp%2E+Spoken+natural+language+dialog+systems%3A+A+practical+approach%2E+Oxford+University+Press+on+Demand%2C+1994%2E)
47. A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin. [Attention is all you need](https://md.research.elamur.ai/free/en/papers/vaswani-2017-attention-all-need/). In Advances in neural information processing systems, pages 5998–6008, 2017.
48. A. Wang, A. Singh, J. Michael, F. Hill, O. Levy, and S. R. Bowman. [Glue: A multi-task benchmark and analysis platform for natural language understanding](https://md.research.elamur.ai/free/en/papers/wang-2018-glue-multi-task-benchmark-analysis/). arXiv preprint arXiv:1804.07461, 2018.
49. T.-H. Wen, M. Gašić, N. Mrkšić, P.-H. Su, D. Vandyke, and S. Young. [Semantically conditioned LSTM-based natural language generation for spoken dialogue systems](https://md.research.elamur.ai/free/en/papers/wen-2015-semantically-conditioned-lstm-based-natural/). In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, 2015.
50. T.-H. Wen, D. Vandyke, N. Mrksic, M. Gasic, L. M. Rojas-Barahona, P.-H. Su, S. Ultes, and S. Young. [A network-based end-to-end trainable task-oriented dialogue system](https://md.research.elamur.ai/free/en/papers/wen-2017-network-based-end-end-trainable/). arXiv preprint arXiv:1604.04562, 2016.
51. T.-H. Wen, Y. Miao, P. Blunsom, and S. Young. Latent intention dialogue models. In Proceedings of the 34th International Conference on Machine Learning, 2017. [find](https://scholar.google.com/scholar?q=T%2E%2DH%2E+Wen%2C+Y%2E+Miao%2C+P%2E+Blunsom%2C+and+S%2E+Young%2E+Latent+intention+dialogue+models%2E+In+Proceedings+of+the+34th+International+Conference+on+Machine+Learning%2C+2017%2E)
52. T. Wolf, L. Debut, V. Sanh, J. Chaumond, C. Delangue, A. Moi, P. Cistac, T. Rault, R. Louf, M. Funtowicz, et al. [Transformers: State-of-the-art natural language processing](https://md.research.elamur.ai/free/en/papers/wolf-2020-transformers-state-art-natural-language/). arXiv preprint arXiv:1910.03771, 2019.
53. C.-S. Wu, A. Madotto, E. Hosseini-Asl, C. Xiong, R. Socher, and P. Fung. [Transferable multi-domain state generator for task-oriented dialogue systems](https://md.research.elamur.ai/free/en/papers/wu-2019-transferable-multi-domain-state-generator/). arXiv preprint arXiv:1905.08743, 2019.
54. C.-S. Wu, R. Socher, and C. Xiong. Global-to-local memory pointer networks for task-oriented dialogue. arXiv preprint arXiv:1901.04713, 2019. [find](https://scholar.google.com/scholar?q=C%2E%2DS%2E+Wu%2C+R%2E+Socher%2C+and+C%2E+Xiong%2E+Global%2Dto%2Dlocal+memory+pointer+networks+for+task%2Doriented+dialogue%2E+arXiv+preprint+arXiv%3A1901%2E04713%2C+2019%2E)
55. C.-S. Wu, S. Hoi, R. Socher, and C. Xiong. Tod-bert: Pre-trained natural language understanding for task-oriented dialogues. arXiv preprint arXiv:2004.06871, 2020. — [arXiv:2004.06871](https://arxiv.org/abs/2004.06871) [find](https://scholar.google.com/scholar?q=C%2E%2DS%2E+Wu%2C+S%2E+Hoi%2C+R%2E+Socher%2C+and+C%2E+Xiong%2E+Tod%2Dbert%3A+Pre%2Dtrained+natural+language+understanding+for+task%2Doriented+dialogues%2E+arXiv+preprint+arXiv%3A2004%2E06871%2C+2020%2E+%E2%80%94+%5BarXiv%3A2004%2E06871%5D%28https%3A%2F%2Farxiv%2Eo)
56. Q. Wu, Y. Zhang, Y. Li, and Z. Yu. Alternating recurrent dialog model with large-scale pre-trained language models. arXiv preprint arXiv:1910.03756, 2019. [find](https://scholar.google.com/scholar?q=Q%2E+Wu%2C+Y%2E+Zhang%2C+Y%2E+Li%2C+and+Z%2E+Yu%2E+Alternating+recurrent+dialog+model+with+large%2Dscale+pre%2Dtrained+language+models%2E+arXiv+preprint+arXiv%3A1910%2E03756%2C+2019%2E)
57. Z. Yang, Z. Dai, Y. Yang, J. Carbonell, R. R. Salakhutdinov, and Q. V. Le. [Xlnet: Generalized autoregressive pretraining for language understanding](https://md.research.elamur.ai/free/en/papers/yang-2019-xlnet-generalized-autoregressive-pretraining-language/). In Advances in neural information processing systems, pages 5754–5764, 2019.
58. S. Young, M. Gašić, B. Thomson, and J. D. Williams. Pomdp-based statistical spoken dialog systems: A review. Proceedings of the IEEE, 101(5):1160–1179, 2013. [find](https://scholar.google.com/scholar?q=S%2E+Young%2C+M%2E+Ga%C5%A1ic%CC%81%2C+B%2E+Thomson%2C+and+J%2E+D%2E+Williams%2E+Pomdp%2Dbased+statistical+spoken+dialog+systems%3A+A+review%2E+Proceedings+of+the+IEEE%2C+101%285%29%3A1160%E2%80%931179%2C+2013%2E)
59. J.-G. Zhang, K. Hashimoto, C.-S. Wu, Y. Wan, P. S. Yu, R. Socher, and C. Xiong. [Find or classify? dual strategy for slot-value predictions on multi-domain dialog state tracking](https://md.research.elamur.ai/free/en/papers/zhang-2020-find-or-classify-dual-strategy/). arXiv preprint arXiv:1910.03544, 2019.
60. Y. Zhang, Z. Ou, and Z. Yu. Task-oriented dialog systems that consider multiple appropriate responses under the same context. arXiv preprint arXiv:1911.10484, 2019. — [arXiv:1911.10484](https://arxiv.org/abs/1911.10484) [find](https://scholar.google.com/scholar?q=Y%2E+Zhang%2C+Z%2E+Ou%2C+and+Z%2E+Yu%2E+Task%2Doriented+dialog+systems+that+consider+multiple+appropriate+responses+under+the+same+context%2E+arXiv+preprint+arXiv%3A1911%2E10484%2C+2019%2E+%E2%80%94+%5BarXiv%3A1911%2E10484%5D%28https%3A%2F%2Farxiv%2E)
61. Y. Zhang, S. Sun, M. Galley, Y.-C. Chen, C. Brockett, X. Gao, J. Gao, J. Liu, and B. Dolan. [Dialogpt: Large-scale generative pre-training for conversational response generation](https://md.research.elamur.ai/free/en/papers/zhang-2019-dialogpt-large-scale-generative-pre/). arXiv preprint arXiv:1911.00536, 2019.
62. T. Zhao, A. Lu, K. Lee, and M. Eskenazi. Generative encoder-decoder models for task-oriented spoken dialog systems with chatting capability. In Proceedings of the 18th Annual SIGdial Meeting on Discourse and Dialogue, 2017. [find](https://scholar.google.com/scholar?q=T%2E+Zhao%2C+A%2E+Lu%2C+K%2E+Lee%2C+and+M%2E+Eskenazi%2E+Generative+encoder%2Ddecoder+models+for+task%2Doriented+spoken+dialog+systems+with+chatting+capability%2E+In+Proceedings+of+the+18th+Annual+SIGdial+Meeting+on+Discou)
63. T. Zhao, K. Xie, and M. Eskenazi. [Rethinking action spaces for reinforcement learning in end-to-end dialog agents with latent variable models](https://md.research.elamur.ai/free/en/papers/zhao-2019-rethinking-action-spaces-reinforcement-learning/). arXiv preprint arXiv:1902.08858, 2019.
64. L. Zhou and K. Small. Multi-domain dialogue state tracking as dynamic knowledge graph enhanced question answering. arXiv preprint arXiv:1911.06192, 2019. — [arXiv:1911.06192](https://arxiv.org/abs/1911.06192) [find](https://scholar.google.com/scholar?q=L%2E+Zhou+and+K%2E+Small%2E+Multi%2Ddomain+dialogue+state+tracking+as+dynamic+knowledge+graph+enhanced+question+answering%2E+arXiv+preprint+arXiv%3A1911%2E06192%2C+2019%2E+%E2%80%94+%5BarXiv%3A1911%2E06192%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F191)

*Updated 29 October 2026*