---
title: "Dialogue Learning with Human Teaching and Feedback in"
description: "To address this, they build a neural agent trainable end-to-end, covering language understanding… 40 references, cited by 67 papers here."
authors: ["Bing Liu", "Gokhan Tür", "Dilek Hakkani-Tür", "Pararth Shah", "Larry Heck"]
year: 2018
venue: "Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers)"
source: "https://aclanthology.org/N18-1187/"
doi: "10.18653/v1/n18-1187"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/liu-2018-dialogue-learning-human-teaching-feedback/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/liu-2018-dialogue-learning-human-teaching-feedback/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2018](https://md.research.elamur.ai/free/en/papers/2018/page/3/) / Dialogue Learning with Human Teaching and Feedback in End-to-End Trainable Task-Oriented Dialogue Systems

# Dialogue Learning with Human Teaching and Feedback in End-to-End Trainable Task-Oriented Dialogue Systems

**Authors:** [Bing Liu](https://md.research.elamur.ai/free/en/authors/bing-liu/), [Gokhan Tür](https://md.research.elamur.ai/free/en/authors/gokhan-tur/), [Dilek Hakkani-Tür](https://md.research.elamur.ai/free/en/authors/dilek-hakkani-tur/), [Pararth Shah](https://md.research.elamur.ai/free/en/authors/pararth-shah/) and Larry Heck · **Year:** 2018 · **Venue:** [Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers)](https://md.research.elamur.ai/free/en/venues/naacl/) ([NAACL 2018](https://md.research.elamur.ai/free/en/venues/naacl/2018/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/N18-1187/) · [PDF](https://aclanthology.org/N18-1187.pdf) · [DOI 10.18653/v1/n18-1187](https://doi.org/10.18653/v1/n18-1187) · [OpenAlex](https://openalex.org/W2963567240)

**Citations:** 145 citations (OpenAlex, 2026-10-14) · [67](https://md.research.elamur.ai/free/en/papers/liu-2018-dialogue-learning-human-teaching-feedback/cited-by/) in our library, of the 141,840 papers whose bibliographies we read

## Summary
Task-oriented dialogue agents are often pre-trained with supervised learning on dialogue corpora and then refined with reinforcement learning (RL) from user feedback. The authors argue that this pipeline suffers from a distribution mismatch. Once the agent talks to users, its own small errors lead it into dialogue states that rarely occur in human-curated data, so mistakes compound. To address this, they build a neural agent trainable end-to-end, covering language understanding, belief tracking, and policy. It uses a bidirectional LSTM utterance encoder, a dialogue-level LSTM state, per-slot value distributions, symbolic knowledge-base queries, and a policy network. Training proceeds in three stages. Supervised pre-training comes first. Next is imitation learning, in which users correct the agent's wrong actions turn by turn and these corrected dialogues are aggregated into the training data (DAgger-style). Finally, RL uses only a binary success signal at the end of each dialogue. On a movie booking task, adding imitation learning before RL produced higher task success and state-tracking accuracy than SL+RL alone. End-to-end RL updates also outperformed updating only the policy.

## Key points
- The hybrid scheme follows dataset aggregation (Ross et al., 2011). User demonstrations are collected along trajectories produced by the agent's own policy, which directly targets the recovery states missing from offline corpora.
- On the movie booking dataset, joint slot tracking accuracy of the SL model is 84.57% on the fixed corpus. It falls to 50.51% in interactive evaluation with unseen user NLG templates. After just 500 imitation learning sessions, it rises to 67.47%.
- Settings with 500 and 1000 imitation episodes (SL + IL 500/1000 + RL) beat SL + RL on task success rate. RL trained from scratch stayed at a very low success rate after 10K simulated dialogues.
- SL + RL reached the shortest average dialogue length. However, this reflects the model solving mainly easy tasks, while the IL-based models handled harder goals at the cost of slightly longer dialogues.
- Propagating RL gradients through the whole network, rather than only the policy layer, gave higher task success. It also further improved belief tracking.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Google](https://md.research.elamur.ai/free/en/companies/google/) — Company, Mountain View, United States · 353 works · 364,806 citations
- [Carnegie Mellon University](https://md.research.elamur.ai/free/en/universities/carnegie-mellon-university/) — University, Pittsburgh, United States · 365 works · 158,048 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 9,086 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 8,889 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 9,296 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 9,206 works

🧭 Topics (1): [🗨️ Dialogue systems](https://md.research.elamur.ai/free/en/topics/dialogue-systems/) · 401 works

## Cite

Cite as: Bing Liu, Gokhan Tür, Dilek Hakkani-Tür, Pararth Shah and Larry Heck. 2018. Dialogue Learning with Human Teaching and Feedback in End-to-End Trainable Task-Oriented Dialogue Systems. Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{liu2018dialogue,
  title = {{Dialogue Learning with Human Teaching and Feedback in End-to-End Trainable Task-Oriented Dialogue Systems}},
  author = {Bing Liu, Gokhan Tür, Dilek Hakkani-Tür, Pararth Shah and Larry Heck},
  booktitle = {Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers)},
  year = {2018},
  doi = {10.18653/v1/n18-1187},
  url = {https://doi.org/10.18653/v1/n18-1187}
}
```

- [← previous Content Selection in Deep Learning Models of Summarization](https://md.research.elamur.ai/free/en/papers/kedzie-2018-content-selection-deep-learning-models/)
- [next → Syntax for Semantic Role Labeling, To Be, Or Not To Be](https://md.research.elamur.ai/free/en/papers/he-2018-syntax-semantic-role-labeling-be/)
- [↑ back to papers of 2018](https://md.research.elamur.ai/free/en/papers/2018/page/3/)

## References

40 references, 7 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Bhuwan Dhingra, Lihong Li, Xiujun Li, Jianfeng Gao, Yun-Nung Chen, Faisal Ahmed, and Li Deng. 2017. [Towards end-to-end reinforcement learning of dialogue agents for information access](https://md.research.elamur.ai/free/en/papers/dhingra-2017-end-end-reinforcement-learning-dialogue/). In ACL.
2. Mihail Eric and Christopher D Manning. 2017. A copy-augmented sequence-to-sequence architecture gives good performance on task-oriented dialogue. In EACL. [find](https://scholar.google.com/scholar?q=Mihail+Eric+and+Christopher+D+Manning%2E+2017%2E+A+copy%2Daugmented+sequence%2Dto%2Dsequence+architecture+gives+good+performance+on+task%2Doriented+dialogue%2E+In+EACL%2E)
3. Milica Gašić, Catherine Breslin, Matthew Henderson, Dongho Kim, Martin Szummer, Blaise Thomson, Pirros Tsiakoulis, and Steve Young. 2013. Online policy optimisation of bayesian spoken dialogue systems via human interaction. In ICASSP. [find](https://scholar.google.com/scholar?q=Milica+Gas%CC%8Cic%CC%81%2C+Catherine+Breslin%2C+Matthew+Henderson%2C+Dongho+Kim%2C+Martin+Szummer%2C+Blaise+Thomson%2C+Pirros+Tsiakoulis%2C+and+Steve+Young%2E+2013%2E+Online+policy+optimisation+of+bayesian+spoken+dialogue+syste)
4. Milica Gasic and Steve Young. 2014. Gaussian processes for pomdp-based dialogue manager optimization. IEEE/ACM Transactions on Audio, Speech, and Language Processing . [find](https://scholar.google.com/scholar?q=Milica+Gasic+and+Steve+Young%2E+2014%2E+Gaussian+processes+for+pomdp%2Dbased+dialogue+manager+optimization%2E+IEEE%2FACM+Transactions+on+Audio%2C+Speech%2C+and+Language+Processing+%2E)
5. Score 3.987 ± 0.086 4.378 ± 0.082 4.603 ± 0.067 *Table 3: Human evaluation results. Mean and standard deviation of crowd worker scores (between 1 to 5).* Antoine Bordes and Jason Weston. 2017. Learning end-to-end goal-oriented dialog. In International Conference on Learning Representations. \#\# Concl… [find](https://scholar.google.com/scholar?q=Score+3%2E987+%C2%B1+0%2E086+4%2E378+%C2%B1+0%2E082+4%2E603+%C2%B1+0%2E067+%2ATable+3%3A+Human+evaluation+results%2E+Mean+and+standard+deviation+of+crowd+worker+scores+%28between+1+to+5%29%2E%2A+Antoine+Bordes+and+Jason+Weston%2E+2017%2E+Learnin)
6. Dilek Hakkani-Tür, Gökhan Tür, Asli Celikyilmaz, Yun-Nung Chen, Jianfeng Gao, Li Deng, and Ye-Yi Wang. 2016. Multi-domain joint semantic frame parsing using bi-directional rnn-lstm. In Interspeech. [find](https://scholar.google.com/scholar?q=Dilek+Hakkani%2DTu%CC%88r%2C+Go%CC%88khan+Tu%CC%88r%2C+Asli+Celikyilmaz%2C+Yun%2DNung+Chen%2C+Jianfeng+Gao%2C+Li+Deng%2C+and+Ye%2DYi+Wang%2E+2016%2E+Multi%2Ddomain+joint+semantic+frame+parsing+using+bi%2Ddirectional+rnn%2Dlstm%2E+In+Interspeech%2E)
7. James Henderson, Oliver Lemon, and Kallirroi Georgila. 2008. Hybrid reinforcement/supervised learning of dialogue policies from fixed data sets. Computational Linguistics . [find](https://scholar.google.com/scholar?q=James+Henderson%2C+Oliver+Lemon%2C+and+Kallirroi+Georgila%2E+2008%2E+Hybrid+reinforcement%2Fsupervised+learning+of+dialogue+policies+from+fixed+data+sets%2E+Computational+Linguistics+%2E)
8. Matthew Henderson, Blaise Thomson, and Jason Williams. 2013. Dialog state tracking challenge 2 & 3. [http://camdial.org/˜mh521/dstc/](http://camdial.org/˜mh521/dstc/). [↗](http://camdial.org/˜mh521/dstc/)
9. Matthew Henderson, Blaise Thomson, and Jason Williams. 2014a. The second dialog state tracking challenge. In SIGDIAL. [find](https://scholar.google.com/scholar?q=Matthew+Henderson%2C+Blaise+Thomson%2C+and+Jason+Williams%2E+2014a%2E+The+second+dialog+state+tracking+challenge%2E+In+SIGDIAL%2E)
10. Matthew Henderson, Blaise Thomson, and Steve Young. 2014b. Robust dialog state tracking using delexicalised recurrent neural networks and unsupervised gate. In IEEE SLT. [find](https://scholar.google.com/scholar?q=Matthew+Henderson%2C+Blaise+Thomson%2C+and+Steve+Young%2E+2014b%2E+Robust+dialog+state+tracking+using+delexicalised+recurrent+neural+networks+and+unsupervised+gate%2E+In+IEEE+SLT%2E)
11. Matthew Henderson, Blaise Thomson, and Steve Young. 2014c. [Word-based dialog state tracking with recurrent neural networks](https://md.research.elamur.ai/free/en/papers/henderson-2014-word-based-dialog-state-tracking/). In SIGDIAL.
12. Filip Jurčı́ček, Blaise Thomson, and Steve Young. 2012. Reinforcement learning for parameter estimation in statistical spoken dialogue systems. Computer Speech & Language 26(3):168–192. [find](https://scholar.google.com/scholar?q=Filip+Jurc%CC%8C%C4%B1%CC%81c%CC%8Cek%2C+Blaise+Thomson%2C+and+Steve+Young%2E+2012%2E+Reinforcement+learning+for+parameter+estimation+in+statistical+spoken+dialogue+systems%2E+Computer+Speech+%26+Language+26%283%29%3A168%E2%80%93192%2E)
13. Diederik Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. In International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Diederik+Kingma+and+Jimmy+Ba%2E+2014%2E+Adam%3A+A+method+for+stochastic+optimization%2E+In+International+Conference+on+Learning+Representations%2E)
14. Jiwei Li, Michel Galley, Chris Brockett, Georgios P Spithourakis, Jianfeng Gao, and Bill Dolan. 2016. [A persona-based neural conversation model](https://md.research.elamur.ai/free/en/papers/li-2016-persona-based-neural-conversation-model/). In ACL.
15. Xuijun Li, Yun-Nung Chen, Lihong Li, and Jianfeng Gao. 2017. [End-to-end task-completion neural dialogue systems](https://md.research.elamur.ai/free/en/papers/li-2017-end-end-task-completion-neural/). arXiv preprint arXiv:1703.01008 .
16. Bing Liu and Ian Lane. 2016. Joint online spoken language understanding and language modeling with recurrent neural networks. In SIGDIAL. [find](https://scholar.google.com/scholar?q=Bing+Liu+and+Ian+Lane%2E+2016%2E+Joint+online+spoken+language+understanding+and+language+modeling+with+recurrent+neural+networks%2E+In+SIGDIAL%2E)
17. Bing Liu and Ian Lane. 2017a. An end-to-end trainable neural network model with belief tracking for task-oriented dialog. In Interspeech. [find](https://scholar.google.com/scholar?q=Bing+Liu+and+Ian+Lane%2E+2017a%2E+An+end%2Dto%2Dend+trainable+neural+network+model+with+belief+tracking+for+task%2Doriented+dialog%2E+In+Interspeech%2E)
18. Bing Liu and Ian Lane. 2017b. Iterative policy learning in end-to-end trainable task-oriented neural dialog models. In Proceedings of IEEE ASRU. [find](https://scholar.google.com/scholar?q=Bing+Liu+and+Ian+Lane%2E+2017b%2E+Iterative+policy+learning+in+end%2Dto%2Dend+trainable+task%2Doriented+neural+dialog+models%2E+In+Proceedings+of+IEEE+ASRU%2E)
19. Bing Liu, Gokhan Tur, Dilek Hakkani-Tur, Pararth Shah, and Larry Heck. 2017. End-to-end optimization of task-oriented dialogue model with deep reinforcement learning. In NIPS Workshop on Conversational AI. [find](https://scholar.google.com/scholar?q=Bing+Liu%2C+Gokhan+Tur%2C+Dilek+Hakkani%2DTur%2C+Pararth+Shah%2C+and+Larry+Heck%2E+2017%2E+End%2Dto%2Dend+optimization+of+task%2Doriented+dialogue+model+with+deep+reinforcement+learning%2E+In+NIPS+Workshop+on+Conversationa)
20. Fei Liu and Julien Perez. 2017. memory networks. In EACL. [find](https://scholar.google.com/scholar?q=Fei+Liu+and+Julien+Perez%2E+2017%2E+memory+networks%2E+In+EACL%2E)
21. Gated end-to-end Grégoire Mesnil, Yann Dauphin, Kaisheng Yao, Yoshua Bengio, Li Deng, Dilek Hakkani-Tur, Xiaodong He, Larry Heck, Gokhan Tur, Dong Yu, et al. 2015. Using recurrent neural networks for slot filling in spoken language understanding. IEEE/ACM Transactions on Audio, Speech and Language… [find](https://scholar.google.com/scholar?q=Gated+end%2Dto%2Dend+Gre%CC%81goire+Mesnil%2C+Yann+Dauphin%2C+Kaisheng+Yao%2C+Yoshua+Bengio%2C+Li+Deng%2C+Dilek+Hakkani%2DTur%2C+Xiaodong+He%2C+Larry+Heck%2C+Gokhan+Tur%2C+Dong+Yu%2C+et+al%2E+2015%2E+Using+recurrent+neural+networks+for)
22. Nikola Mrkšić, Diarmuid O Séaghdha, Tsung-Hsien Wen, Blaise Thomson, and Steve Young. 2016. Neural belief tracker: Data-driven dialogue state tracking. arXiv preprint arXiv:1606.03777 . [find](https://scholar.google.com/scholar?q=Nikola+Mrks%CC%8Cic%CC%81%2C+Diarmuid+O+Se%CC%81aghdha%2C+Tsung%2DHsien+Wen%2C+Blaise+Thomson%2C+and+Steve+Young%2E+2016%2E+Neural+belief+tracker%3A+Data%2Ddriven+dialogue+state+tracking%2E+arXiv+preprint+arXiv%3A1606%2E03777+%2E)
23. Baolin Peng, Xiujun Li, Lihong Li, Jianfeng Gao, Asli Celikyilmaz, Sungjin Lee, and Kam-Fai Wong. 2017. Composite task-completion dialogue policy learning via hierarchical deep reinforcement learning. In Proceedings of EMNLP. [find](https://scholar.google.com/scholar?q=Baolin+Peng%2C+Xiujun+Li%2C+Lihong+Li%2C+Jianfeng+Gao%2C+Asli+Celikyilmaz%2C+Sungjin+Lee%2C+and+Kam%2DFai+Wong%2E+2017%2E+Composite+task%2Dcompletion+dialogue+policy+learning+via+hierarchical+deep+reinforcement+learning%2E)
24. Antoine Raux, Brian Langner, Dan Bohus, Alan W Black, and Maxine Eskenazi. 2005. Lets go public! taking a spoken dialog system to the real world. In Interspeech. [find](https://scholar.google.com/scholar?q=Antoine+Raux%2C+Brian+Langner%2C+Dan+Bohus%2C+Alan+W+Black%2C+and+Maxine+Eskenazi%2E+2005%2E+Lets+go+public%21+taking+a+spoken+dialog+system+to+the+real+world%2E+In+Interspeech%2E)
25. Stéphane Ross and Drew Bagnell. 2010. Efficient reductions for imitation learning. In Proceedings of the thirteenth international conference on artificial intelligence and statistics. pages 661–668. [find](https://scholar.google.com/scholar?q=Ste%CC%81phane+Ross+and+Drew+Bagnell%2E+2010%2E+Efficient+reductions+for+imitation+learning%2E+In+Proceedings+of+the+thirteenth+international+conference+on+artificial+intelligence+and+statistics%2E+pages+661%E2%80%93668%2E)
26. Stéphane Ross, Geoffrey J Gordon, and Drew Bagnell. 2011. A reduction of imitation learning and structured prediction to no-regret online learning. In International Conference on Artificial Intelligence and Statistics. pages 627–635. [find](https://scholar.google.com/scholar?q=Ste%CC%81phane+Ross%2C+Geoffrey+J+Gordon%2C+and+Drew+Bagnell%2E+2011%2E+A+reduction+of+imitation+learning+and+structured+prediction+to+no%2Dregret+online+learning%2E+In+International+Conference+on+Artificial+Intellige)
27. Alexander I Rudnicky, Eric H Thayer, Paul C Constantinides, Chris Tchou, R Shern, Kevin A Lenzo, Wei Xu, and Alice Oh. 1999. Creating natural dialogs in the carnegie mellon communicator system. In Eurospeech. [find](https://scholar.google.com/scholar?q=Alexander+I+Rudnicky%2C+Eric+H+Thayer%2C+Paul+C+Constantinides%2C+Chris+Tchou%2C+R+Shern%2C+Kevin+A+Lenzo%2C+Wei+Xu%2C+and+Alice+Oh%2E+1999%2E+Creating+natural+dialogs+in+the+carnegie+mellon+communicator+system%2E+In+Eur)
28. Jost Schatzmann, Blaise Thomson, Karl Weilhammer, Hui Ye, and Steve Young. 2007. [Agenda-based user simulation for bootstrapping a pomdp dialogue system](https://md.research.elamur.ai/free/en/papers/schatzmann-2007-agenda-based-user-simulation-bootstrapping/). In NAACL-HLT.
29. Minjoon Seo, Ali Farhadi, and Hannaneh Hajishirzi. 2016. Query-regression networks for machine comprehension. arXiv preprint arXiv:1606.04582 . — [arXiv:1606.04582](https://arxiv.org/abs/1606.04582) [find](https://scholar.google.com/scholar?q=Minjoon+Seo%2C+Ali+Farhadi%2C+and+Hannaneh+Hajishirzi%2E+2016%2E+Query%2Dregression+networks+for+machine+comprehension%2E+arXiv+preprint+arXiv%3A1606%2E04582+%2E+%E2%80%94+%5BarXiv%3A1606%2E04582%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F1606%2E04582%29)
30. Iulian V Serban, Alessandro Sordoni, Yoshua Bengio, Aaron Courville, and Joelle Pineau. 2015. [Building end-to-end dialogue systems using generative hierarchical neural network models](https://md.research.elamur.ai/free/en/papers/serban-2015-building-end-end-dialogue-systems/). arXiv preprint arXiv:1507.04808 .
31. Pararth Shah, Dilek Hakkani-Tür, Liu Bing, and Gokhan Tür. 2018. [Bootstrapping a neural conversational agent with dialogue self-play, crowdsourcing and on-line reinforcement learning](https://md.research.elamur.ai/free/en/papers/shah-2018-bootstrapping-neural-conversational-agent-dialogue/). In NAACL-HLT.
32. Pararth Shah, Dilek Hakkani-Tür, and Larry Heck. 2016. Interactive reinforcement learning for task-oriented dialogue management. In NIPS 2016 Deep Learning for Action and Interaction Workshop. [find](https://scholar.google.com/scholar?q=Pararth+Shah%2C+Dilek+Hakkani%2DTu%CC%88r%2C+and+Larry+Heck%2E+2016%2E+Interactive+reinforcement+learning+for+task%2Doriented+dialogue+management%2E+In+NIPS+2016+Deep+Learning+for+Action+and+Interaction+Workshop%2E)
33. Pei-Hao Su, Pawel Budzianowski, Stefan Ultes, Milica Gasic, and Steve Young. 2017. Sample-efficient actor-critic reinforcement learning with supervised data for dialogue management. In SIGDIAL. [find](https://scholar.google.com/scholar?q=Pei%2DHao+Su%2C+Pawel+Budzianowski%2C+Stefan+Ultes%2C+Milica+Gasic%2C+and+Steve+Young%2E+2017%2E+Sample%2Defficient+actor%2Dcritic+reinforcement+learning+with+supervised+data+for+dialogue+management%2E+In+SIGDIAL%2E)
34. Pei-Hao Su, Milica Gasic, Nikola Mrksic, Lina Rojas-Barahona, Stefan Ultes, David Vandyke, Tsung-Hsien Wen, and Steve Young. 2016. On-line active reward learning for policy optimisation in spoken dialogue systems. In ACL. [find](https://scholar.google.com/scholar?q=Pei%2DHao+Su%2C+Milica+Gasic%2C+Nikola+Mrksic%2C+Lina+Rojas%2DBarahona%2C+Stefan+Ultes%2C+David+Vandyke%2C+Tsung%2DHsien+Wen%2C+and+Steve+Young%2E+2016%2E+On%2Dline+active+reward+learning+for+policy+optimisation+in+spoken+dial)
35. Tsung-Hsien Wen, David Vandyke, Nikola Mrkšić, Milica Gašić, Lina M. Rojas-Barahona, Pei-Hao Su, Stefan Ultes, and Steve Young. 2017. A network-based end-to-end trainable task-oriented dialogue system. In EACL. [find](https://scholar.google.com/scholar?q=Tsung%2DHsien+Wen%2C+David+Vandyke%2C+Nikola+Mrks%CC%8Cic%CC%81%2C+Milica+Gas%CC%8Cic%CC%81%2C+Lina+M%2E+Rojas%2DBarahona%2C+Pei%2DHao+Su%2C+Stefan+Ultes%2C+and+Steve+Young%2E+2017%2E+A+network%2Dbased+end%2Dto%2Dend+trainable+task%2Doriented+dialogue+sy)
36. Jason D Williams, Kavosh Asadi, and Geoffrey Zweig. 2017. Hybrid code networks: practical and efficient end-to-end dialog control with supervised and reinforcement learning. In ACL. [find](https://scholar.google.com/scholar?q=Jason+D+Williams%2C+Kavosh+Asadi%2C+and+Geoffrey+Zweig%2E+2017%2E+Hybrid+code+networks%3A+practical+and+efficient+end%2Dto%2Dend+dialog+control+with+supervised+and+reinforcement+learning%2E+In+ACL%2E)
37. Jason D Williams and Geoffrey Zweig. 2016. End-to-end lstm-based dialog control optimized with supervised and reinforcement learning. arXiv preprint arXiv:1606.01269 . — [arXiv:1606.01269](https://arxiv.org/abs/1606.01269) [find](https://scholar.google.com/scholar?q=Jason+D+Williams+and+Geoffrey+Zweig%2E+2016%2E+End%2Dto%2Dend+lstm%2Dbased+dialog+control+optimized+with+supervised+and+reinforcement+learning%2E+arXiv+preprint+arXiv%3A1606%2E01269+%2E+%E2%80%94+%5BarXiv%3A1606%2E01269%5D%28https%3A%2F%2Farx)
38. Ronald J Williams. 1992. Simple statistical gradient-following algorithms for connectionist reinforcement learning. Machine learning . [find](https://scholar.google.com/scholar?q=Ronald+J+Williams%2E+1992%2E+Simple+statistical+gradient%2Dfollowing+algorithms+for+connectionist+reinforcement+learning%2E+Machine+learning+%2E)
39. Steve Young, Milica Gašić, Blaise Thomson, and Jason D Williams. 2013. Pomdp-based statistical spoken dialog systems: A review. Proceedings of the IEEE 101(5):1160–1179. [find](https://scholar.google.com/scholar?q=Steve+Young%2C+Milica+Gas%CC%8Cic%CC%81%2C+Blaise+Thomson%2C+and+Jason+D+Williams%2E+2013%2E+Pomdp%2Dbased+statistical+spoken+dialog+systems%3A+A+review%2E+Proceedings+of+the+IEEE+101%285%29%3A1160%E2%80%931179%2E)
40. Tiancheng Zhao and Maxine Eskenazi. 2016. Towards end-to-end learning for dialog state tracking and management using deep reinforcement learning. In SIGDIAL. [find](https://scholar.google.com/scholar?q=Tiancheng+Zhao+and+Maxine+Eskenazi%2E+2016%2E+Towards+end%2Dto%2Dend+learning+for+dialog+state+tracking+and+management+using+deep+reinforcement+learning%2E+In+SIGDIAL%2E)

*Updated 24 October 2026*