---
title: "Stochastic Language Generation in Dialogue using — Wen 2015"
description: "Spoken dialogue systems must turn an abstract dialogue act (an act type plus slot-value pairs) into a sentence."
authors: ["Tsung-Hsien Wen", "Milica Gašić", "Dongho Kim", "Nikola Mrkšić", "Pei-Hao Su", "David Vandyke", "Steve Young"]
year: 2015
venue: "Proceedings of the 16th Annual Meeting of the Special Interest Group on Discourse and Dialogue"
source: "https://aclanthology.org/W15-4639/"
doi: "10.18653/v1/w15-4639"
licence: "CC BY-NC-SA 3.0 (ACL Anthology, before 2016)"
licence_url: "https://creativecommons.org/licenses/by-nc-sa/3.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/papers/wen-2015-stochastic-language-generation-dialogue-recurrent/"
---
[Elamur Research](https://md.research.elamur.ai/) · [this page in HTML](https://research.elamur.ai/free/papers/wen-2015-stochastic-language-generation-dialogue-recurrent/)

[home](https://md.research.elamur.ai/) / [papers](https://md.research.elamur.ai/free/papers/) / [2015](https://md.research.elamur.ai/free/papers/2015/page/2/) / Stochastic Language Generation in Dialogue using Recurrent Neural Networks with Convolutional Sentence Reranking

# Stochastic Language Generation in Dialogue using Recurrent Neural Networks with Convolutional Sentence Reranking

**Authors:** Tsung-Hsien Wen, Milica Gašić, Dongho Kim, Nikola Mrkšić, Pei-Hao Su, David Vandyke, [Steve Young](https://md.research.elamur.ai/free/authors/steve-young/) · **Year:** 2015 · **Venue:** Proceedings of the 16th Annual Meeting of the Special Interest Group on Discourse and Dialogue · **Licence:** [CC BY-NC-SA 3.0 (ACL Anthology, before 2016)](https://creativecommons.org/licenses/by-nc-sa/3.0/)

**Links:** [Publisher's page](https://aclanthology.org/W15-4639/) · [PDF](https://aclanthology.org/W15-4639.pdf) · [DOI 10.18653/v1/w15-4639](https://doi.org/10.18653/v1/w15-4639) · [OpenAlex](https://openalex.org/W1947758080)

**Citations:** 151 citations (openalex-snapshot · 2026-10-14) · [61](#cited-by) in our library, of the 140673 papers whose bibliographies we read

## Summary
Spoken dialogue systems must turn an abstract dialogue act (an act type plus slot-value pairs) into a sentence. This step usually depends on handwritten rules or on corpora with careful semantic alignment, which makes new domains and languages expensive to support. The authors build a generator that learns only from pairs of dialogue acts and utterances, with no alignments or grammar trees. A forward RNNLM is trained on a delexicalised corpus, in which slot values are replaced by slot placeholders. At every time step it receives a 1-hot control vector that encodes the dialogue act. A decaying gate switches off each slot's feature once that slot has been produced. The RNN over-generates candidate sentences. A CNN sentence model then reranks them for semantic consistency, and a backward RNNLM reranks them for fluency. In a San Francisco restaurant information domain, the system beats the n-gram generator of Oh and Rudnicky (2000) on objective metrics and is cheaper to run. Human judges also preferred it to a rule-based generator. The work shows that varied, data-driven NLG is possible without costly annotation.

## Key points
- The architecture has three parts: a forward RNNLM generator conditioned on the dialogue act, a CNN reranker that checks semantic coverage, and a backward RNNLM reranker. Objective evaluation uses BLEU and slot error (ERR).
- The feature gate uses a decay factor δ. Values of δ = 1 (no gate), 0.7 and 0 were compared. Gating clearly improved both BLEU and slot error, and δ = 0 gave the best results. With δ = 0, a slot's feature is cut as soon as the slot is generated.
- The CNN reranker helped most on a "hard" subset made of binary slots and *dont care* values, gaining about 1.5% there. These slot-value pairs have no explicit surface value to delexicalise.
- The human evaluation recruited about 60 Amazon Mechanical Turk judges and used 2571 unseen dialogues.
  - Four systems were compared pairwise: RNN 1-best, RNN sampling from the top 5, O&R sampling from the top 5, and a handcrafted baseline.
  - Judges preferred both RNN variants over the rule-based system, and this preference was statistically significant.
  - The RNN systems scored higher on informativeness, but that difference was not significant.
- Training data size mattered differently for the two output modes:
  - About 2000 utterances (roughly half the data) were enough for 1-best output.
  - Varied top-5 output appeared to need more than the 4156 training utterances available.

## Interesting ideas
- **Re-inject the conditioning vector at every step.** Feeding the control vector at every step, rather than only at the start, stops the conditioning signal from fading over a long sequence. This applies to any conditional sequence generator, such as captioning or data-to-text.
- **Gate each slot after it is mentioned.** A per-slot gate that shrinks a slot's feature once it has been expressed is a cheap way to discourage repeated content and encourage full coverage. It needs no extra training objective.
- **Over-generate, then rerank with specialised models.** The generator proposes many candidates, and separate models score each candidate for meaning and for fluency (a CNN checking semantics, a backward LM checking fluency). This keeps the generator simple and lets weaknesses be fixed in a modular way.
- **Handle what delexicalisation cannot.** Delexicalising slot values shrinks the vocabulary and helps with sparse data. A separate classifier-style check covers the cases that have no explicit value in the text, such as *dont care* or boolean slots.
- **Sample from the top n candidates for variety.** Choosing randomly among the top-n reranked outputs trades a little BLEU for less repetitive replies. The authors argue this matters for dialogue systems, and it could serve any user-facing generator.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

Fields: [Linguistics & languages](https://md.research.elamur.ai/free/shelves/linguistics/)

Topics: [Speech act theory](https://md.research.elamur.ai/free/topics/speech-act-theory/)

## Cited by

Cited by

61 papers of our library cite it (of the 140673 whose bibliographies we read); 1 of them have a page here, 60 not yet.

- [Survey of Hallucination in Natural Language Generation](https://md.research.elamur.ai/free/papers/ji-2022-survey-hallucination-natural-language-generation/)

## Cite

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{wen2015stochastic,
  title = {{Stochastic Language Generation in Dialogue using Recurrent Neural Networks with Convolutional Sentence Reranking}},
  author = {Wen, Tsung-Hsien and Gašić, Milica and Kim, Dongho and Mrkšić, Nikola and Su, Pei-Hao and Vandyke, David and Young, Steve},
  booktitle = {Proceedings of the 16th Annual Meeting of the Special Interest Group on Discourse and Dialogue},
  year = {2015},
  doi = {10.18653/v1/w15-4639},
  url = {https://doi.org/10.18653/v1/w15-4639}
}
```

- [← previous Statistical Parsing by Machine Learning from a Classical Arabic Treebank](https://md.research.elamur.ai/free/papers/langstream-2015-dukes-statistical-parsing-machine-learning-classical/)
- [next → Suspense and Authority amid Biblical Hebrew Front Dislocation](https://md.research.elamur.ai/free/papers/hbo-2015-korchin-suspense-authority-amid-biblical-hebrew/)
- [↑ back to papers of 2015](https://md.research.elamur.ai/free/papers/2015/page/2/)

## References

46 references, 4 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Gabor Angeli, Percy Liang, and Dan Klein. 2010. A simple domain-independent probabilistic approach to generation. In Proceedings of the 2010 Conference on Empirical Methods in Natural Language Processing, EMNLP ’10, pages 502–512. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Gabor+Angeli%2C+Percy+Liang%2C+and+Dan+Klein%2E+2010%2E+A+simple+domain%2Dindependent+probabilistic+approach+to+generation%2E+In+Proceedings+of+the+2010+Conference+on+Empirical+Methods+in+Natural+Language+Process)
2. Frédéric Bastien, Pascal Lamblin, Razvan Pascanu, James Bergstra, Ian J. Goodfellow, Arnaud Bergeron, Nicolas Bouchard, and Yoshua Bengio. 2012. Theano: new features and speed improvements. [find](https://scholar.google.com/scholar?q=Fre%CC%81de%CC%81ric+Bastien%2C+Pascal+Lamblin%2C+Razvan+Pascanu%2C+James+Bergstra%2C+Ian+J%2E+Goodfellow%2C+Arnaud+Bergeron%2C+Nicolas+Bouchard%2C+and+Yoshua+Bengio%2E+2012%2E+Theano%3A+new+features+and+speed+improvements%2E)
3. Deep Learning and Unsupervised Feature Learning NIPS 2012 Workshop. [find](https://scholar.google.com/scholar?q=Deep+Learning+and+Unsupervised+Feature+Learning+NIPS+2012+Workshop%2E)
4. Anja Belz. 2008. Automatic generation of weather forecast texts using comprehensive probabilistic generation-space models. Natural Language Engineering, 14(4):431–455, October. [find](https://scholar.google.com/scholar?q=Anja+Belz%2E+2008%2E+Automatic+generation+of+weather+forecast+texts+using+comprehensive+probabilistic+generation%2Dspace+models%2E+Natural+Language+Engineering%2C+14%284%29%3A431%E2%80%93455%2C+October%2E)
5. Yoshua Bengio, Patrice Simard, and Paolo Frasconi. 1994. Learning long-term dependencies with gradient descent is difficult. Neural Networks, IEEE Transactions on, 5(2):157–166. [find](https://scholar.google.com/scholar?q=Yoshua+Bengio%2C+Patrice+Simard%2C+and+Paolo+Frasconi%2E+1994%2E+Learning+long%2Dterm+dependencies+with+gradient+descent+is+difficult%2E+Neural+Networks%2C+IEEE+Transactions+on%2C+5%282%29%3A157%E2%80%93166%2E)
6. James Bergstra, Olivier Breuleux, Frédéric Bastien, Pascal Lamblin, Razvan Pascanu, Guillaume Desjardins, Joseph Turian, David Warde-Farley, and Yoshua Bengio. 2010. Theano: a CPU and GPU math expression compiler. In Proceedings of the Python for Scientific Computing Conference. [find](https://scholar.google.com/scholar?q=James+Bergstra%2C+Olivier+Breuleux%2C+Fre%CC%81de%CC%81ric+Bastien%2C+Pascal+Lamblin%2C+Razvan+Pascanu%2C+Guillaume+Desjardins%2C+Joseph+Turian%2C+David+Warde%2DFarley%2C+and+Yoshua+Bengio%2E+2010%2E+Theano%3A+a+CPU+and+GPU+math+expre)
7. Dan Bohus and Alexander I. Rudnicky. 2009. The ravenclaw dialog management framework: Architecture and systems. Computer Speech and Language, 23(3):332–361, July. [find](https://scholar.google.com/scholar?q=Dan+Bohus+and+Alexander+I%2E+Rudnicky%2E+2009%2E+The+ravenclaw+dialog+management+framework%3A+Architecture+and+systems%2E+Computer+Speech+and+Language%2C+23%283%29%3A332%E2%80%93361%2C+July%2E)
8. Adam Cheyer and Didier Guzzoni. 2007. Method and apparatus for building an intelligent automated assistant. US Patent App. 11/518,292. [find](https://scholar.google.com/scholar?q=Adam+Cheyer+and+Didier+Guzzoni%2E+2007%2E+Method+and+apparatus+for+building+an+intelligent+automated+assistant%2E+US+Patent+App%2E+11%2F518%2C292%2E)
9. Ronan Collobert and Jason Weston. 2008. A unified architecture for natural language processing: Deep neural networks with multitask learning. In Proceedings of the 25th International Conference on Machine Learning, ICML ’08, pages 160–167. [find](https://scholar.google.com/scholar?q=Ronan+Collobert+and+Jason+Weston%2E+2008%2E+A+unified+architecture+for+natural+language+processing%3A+Deep+neural+networks+with+multitask+learning%2E+In+Proceedings+of+the+25th+International+Conference+on+Mac)
10. Nina Dethlefs, Helen Hastie, Heriberto Cuayhuitl, and Oliver Lemon. 2013. Conditional random fields for responsive surface realisation using global features. In In Proceedings of ACL. [find](https://scholar.google.com/scholar?q=Nina+Dethlefs%2C+Helen+Hastie%2C+Heriberto+Cuayhuitl%2C+and+Oliver+Lemon%2E+2013%2E+Conditional+random+fields+for+responsive+surface+realisation+using+global+features%2E+In+In+Proceedings+of+ACL%2E)
11. Milica Gašić, Dongho Kim, Pirros Tsiakoulis, Catherine Breslin, Matthew Henderson, Martin Szummer, Blaise Thomson, and Steve Young. 2014. Incremental on-line adaptation of pomdp-based dialogue managers to extended domains. In In Proceedings on InterSpeech. [find](https://scholar.google.com/scholar?q=Milica+Gas%CC%8Cic%CC%81%2C+Dongho+Kim%2C+Pirros+Tsiakoulis%2C+Catherine+Breslin%2C+Matthew+Henderson%2C+Martin+Szummer%2C+Blaise+Thomson%2C+and+Steve+Young%2E+2014%2E+Incremental+on%2Dline+adaptation+of+pomdp%2Dbased+dialogue+manag)
12. Alex Graves, Marcus Liwicki, Horst Bunke, Jürgen Schmidhuber, and Santiago Fernández. 2008. Unconstrained on-line handwriting recognition with recurrent neural networks. In Advances in Neural Information Processing Systems, pages 577–584. [find](https://scholar.google.com/scholar?q=Alex+Graves%2C+Marcus+Liwicki%2C+Horst+Bunke%2C+Ju%CC%88rgen+Schmidhuber%2C+and+Santiago+Ferna%CC%81ndez%2E+2008%2E+Unconstrained+on%2Dline+handwriting+recognition+with+recurrent+neural+networks%2E+In+Advances+in+Neural+Inform)
13. Alex Graves, A-R Mohamed, and Geoffrey Hinton. 2013. Speech recognition with deep recurrent neural networks. In Acoustics, Speech and Signal Processing (ICASSP), 2013 IEEE International Conference on, pages 6645–6649. IEEE. [find](https://scholar.google.com/scholar?q=Alex+Graves%2C+A%2DR+Mohamed%2C+and+Geoffrey+Hinton%2E+2013%2E+Speech+recognition+with+deep+recurrent+neural+networks%2E+In+Acoustics%2C+Speech+and+Signal+Processing+%28ICASSP%29%2C+2013+IEEE+International+Conference+on%2C)
14. Matthew Henderson, Blaise Thomson, and Steve Young. 2014. Robust dialog state tracking using delexicalised recurrent neural networks and unsupervised adaptation. In Proceedings of IEEE Spoken Language Technology. [find](https://scholar.google.com/scholar?q=Matthew+Henderson%2C+Blaise+Thomson%2C+and+Steve+Young%2E+2014%2E+Robust+dialog+state+tracking+using+delexicalised+recurrent+neural+networks+and+unsupervised+adaptation%2E+In+Proceedings+of+IEEE+Spoken+Language)
15. Nal Kalchbrenner, Edward Grefenstette, and Phil Blunsom. 2014. A convolutional neural network for modelling sentences. CoRR, abs/1404.2188. [find](https://scholar.google.com/scholar?q=Nal+Kalchbrenner%2C+Edward+Grefenstette%2C+and+Phil+Blunsom%2E+2014%2E+A+convolutional+neural+network+for+modelling+sentences%2E+CoRR%2C+abs%2F1404%2E2188%2E) [our card](https://md.research.elamur.ai/free/papers/kalchbrenner-2014-convolutional-neural-network-modelling-sentences/)
16. Andrej Karpathy and Li Fei-Fei. 2014. Deep visual-semantic alignments for generating image descriptions. CoRR, abs/1412.2306. [find](https://scholar.google.com/scholar?q=Andrej+Karpathy+and+Li+Fei%2DFei%2E+2014%2E+Deep+visual%2Dsemantic+alignments+for+generating+image+descriptions%2E+CoRR%2C+abs%2F1412%2E2306%2E) [our card](https://md.research.elamur.ai/free/papers/karpathy-2014-deep-visual-semantic-alignments-generating/)
17. Yoon Kim. 2014. Convolutional neural networks for sentence classification. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 1746– 1751, Doha, Qatar, October. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Yoon+Kim%2E+2014%2E+Convolutional+neural+networks+for+sentence+classification%2E+In+Proceedings+of+the+2014+Conference+on+Empirical+Methods+in+Natural+Language+Processing+%28EMNLP%29%2C+pages+1746%E2%80%93+1751%2C+Doha%2C+Qa) [our card](https://md.research.elamur.ai/free/papers/kim-2014-convolutional-neural-networks-sentence-classification/)
18. Ravi Kondadadi, Blake Howald, and Frank Schilder. 2013. A statistical nlg framework for aggregated planning and realization. In Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1406–1415, Sofia, Bulgaria, August. Association for C… [find](https://scholar.google.com/scholar?q=Ravi+Kondadadi%2C+Blake+Howald%2C+and+Frank+Schilder%2E+2013%2E+A+statistical+nlg+framework+for+aggregated+planning+and+realization%2E+In+Proceedings+of+the+51st+Annual+Meeting+of+the+Association+for+Computatio)
19. Karen Kukich. 1987. Where do phrases come from: Some preliminary experiments in connectionist phrase generation. In Natural Language Generation, volume 135 of NATO ASI Series, pages 405– 421. Springer Netherlands. [find](https://scholar.google.com/scholar?q=Karen+Kukich%2E+1987%2E+Where+do+phrases+come+from%3A+Some+preliminary+experiments+in+connectionist+phrase+generation%2E+In+Natural+Language+Generation%2C+volume+135+of+NATO+ASI+Series%2C+pages+405%E2%80%93+421%2E+Springer)
20. Irene Langkilde and Kevin Knight. 1998. Generation that exploits corpus-based statistical knowledge. In Proceedings of the 36th Annual Meeting of the Association for Computational Linguistics, ACL ’98, pages 704–710. [find](https://scholar.google.com/scholar?q=Irene+Langkilde+and+Kevin+Knight%2E+1998%2E+Generation+that+exploits+corpus%2Dbased+statistical+knowledge%2E+In+Proceedings+of+the+36th+Annual+Meeting+of+the+Association+for+Computational+Linguistics%2C+ACL+%E2%80%9998)
21. Yann Lecun, León Bottou, Yoshua Bengio, and Patrick Haffner. 1998. Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(11):2278–2324, Nov. [find](https://scholar.google.com/scholar?q=Yann+Lecun%2C+Leo%CC%81n+Bottou%2C+Yoshua+Bengio%2C+and+Patrick+Haffner%2E+1998%2E+Gradient%2Dbased+learning+applied+to+document+recognition%2E+Proceedings+of+the+IEEE%2C+86%2811%29%3A2278%E2%80%932324%2C+Nov%2E)
22. François Mairesse and Steve Young. 2014. Stochastic language generation in dialogue using factored language models. Computer Linguistics, 40(4):763– 799. [find](https://scholar.google.com/scholar?q=Franc%CC%A7ois+Mairesse+and+Steve+Young%2E+2014%2E+Stochastic+language+generation+in+dialogue+using+factored+language+models%2E+Computer+Linguistics%2C+40%284%29%3A763%E2%80%93+799%2E)
23. François Mairesse, Milica Gašić, Filip Jurčı́ček, Simon Keizer, Blaise Thomson, Kai Yu, and Steve Young. 2010. Phrase-based statistical language generation using graphical models and active learning. In Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics, ACL… [find](https://scholar.google.com/scholar?q=Franc%CC%A7ois+Mairesse%2C+Milica+Gas%CC%8Cic%CC%81%2C+Filip+Jurc%CC%8C%C4%B1%CC%81c%CC%8Cek%2C+Simon+Keizer%2C+Blaise+Thomson%2C+Kai+Yu%2C+and+Steve+Young%2E+2010%2E+Phrase%2Dbased+statistical+language+generation+using+graphical+models+and+active+learn)
24. Tomáš Mikolov and Geoffrey Zweig. 2012. Context dependent recurrent neural network language model. In In Proceedings on IEEE SLT workshop. [find](https://scholar.google.com/scholar?q=Toma%CC%81s%CC%8C+Mikolov+and+Geoffrey+Zweig%2E+2012%2E+Context+dependent+recurrent+neural+network+language+model%2E+In+In+Proceedings+on+IEEE+SLT+workshop%2E)
25. Tomáš Mikolov, Martin Karafit, Lukáš Burget, Jan Černocký, and Sanjeev Khudanpur. 2010. Recurrent neural network based language model. In In Proceedings on InterSpeech. [find](https://scholar.google.com/scholar?q=Toma%CC%81s%CC%8C+Mikolov%2C+Martin+Karafit%2C+Luka%CC%81s%CC%8C+Burget%2C+Jan+C%CC%8Cernocky%CC%81%2C+and+Sanjeev+Khudanpur%2E+2010%2E+Recurrent+neural+network+based+language+model%2E+In+In+Proceedings+on+InterSpeech%2E)
26. Tomáš Mikolov, Stefan Kombrink, Lukáš Burget, Jan H. Černocký, and Sanjeev Khudanpur. 2011a. Extensions of recurrent neural network language model. In Acoustics, Speech and Signal Processing (ICASSP), 2011 IEEE International Conference on. [find](https://scholar.google.com/scholar?q=Toma%CC%81s%CC%8C+Mikolov%2C+Stefan+Kombrink%2C+Luka%CC%81s%CC%8C+Burget%2C+Jan+H%2E+C%CC%8Cernocky%CC%81%2C+and+Sanjeev+Khudanpur%2E+2011a%2E+Extensions+of+recurrent+neural+network+language+model%2E+In+Acoustics%2C+Speech+and+Signal+Processing+%28IC)
27. Tomáš Mikolov, Stefan Kombrink, Anoop Deoras, Lukáš Burget, and Jan Černocký. 2011b. Rnnlm - recurrent neural network language modeling toolkit. In In Proceedings on ASRU. [find](https://scholar.google.com/scholar?q=Toma%CC%81s%CC%8C+Mikolov%2C+Stefan+Kombrink%2C+Anoop+Deoras%2C+Luka%CC%81s%CC%8C+Burget%2C+and+Jan+C%CC%8Cernocky%CC%81%2E+2011b%2E+Rnnlm+%2D+recurrent+neural+network+language+modeling+toolkit%2E+In+In+Proceedings+on+ASRU%2E)
28. Tomáš Mikolov, Ilya Sutskever, Kai Chen, Greg S. Corrado, and Jeff Dean. 2013. Distributed representations of words and phrases and their compositionality. In Advances in Neural Information Processing Systems 26, pages 3111–3119. [find](https://scholar.google.com/scholar?q=Toma%CC%81s%CC%8C+Mikolov%2C+Ilya+Sutskever%2C+Kai+Chen%2C+Greg+S%2E+Corrado%2C+and+Jeff+Dean%2E+2013%2E+Distributed+representations+of+words+and+phrases+and+their+compositionality%2E+In+Advances+in+Neural+Information+Processi)
29. Danilo Mirkovic and Lawrence Cavedon. 2011. Dialogue management using scripts, February 16. EP Patent 1,891,625. [find](https://scholar.google.com/scholar?q=Danilo+Mirkovic+and+Lawrence+Cavedon%2E+2011%2E+Dialogue+management+using+scripts%2C+February+16%2E+EP+Patent+1%2C891%2C625%2E)
30. Alice H. Oh and Alexander I. Rudnicky. 2000. Stochastic language generation for spoken dialogue systems. In Proceedings of the 2000 ANLP/NAACL Workshop on Conversational Systems - Volume 3, ANLP/NAACL-ConvSyst ’00, pages 27–32. [find](https://scholar.google.com/scholar?q=Alice+H%2E+Oh+and+Alexander+I%2E+Rudnicky%2E+2000%2E+Stochastic+language+generation+for+spoken+dialogue+systems%2E+In+Proceedings+of+the+2000+ANLP%2FNAACL+Workshop+on+Conversational+Systems+%2D+Volume+3%2C+ANLP%2FNAACL)
31. Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. Bleu: a method for automatic evaluation of machine translation. In Proceedings of the 40th annual meeting on association for computational linguistics, pages 311–318. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Kishore+Papineni%2C+Salim+Roukos%2C+Todd+Ward%2C+and+Wei%2DJing+Zhu%2E+2002%2E+Bleu%3A+a+method+for+automatic+evaluation+of+machine+translation%2E+In+Proceedings+of+the+40th+annual+meeting+on+association+for+computat) [our card](https://md.research.elamur.ai/free/papers/papineni-2002-bleu-method-automatic-evaluation-machine/)
32. Jeffrey Pennington, Richard Socher, and Christopher Manning. 2014. Glove: Global vectors for word representation. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 1532–1543. Association for Computational Linguistics, October. [find](https://scholar.google.com/scholar?q=Jeffrey+Pennington%2C+Richard+Socher%2C+and+Christopher+Manning%2E+2014%2E+Glove%3A+Global+vectors+for+word+representation%2E+In+Proceedings+of+the+2014+Conference+on+Empirical+Methods+in+Natural+Language+Process)
33. Heather Pon-Barry, Karl Schultz, Elizabeth Owen Bratt, Brady Clark, and Stanley Peters. 2006. Responding to student uncertainty in spoken tutorial dialogue systems. International Journal of Artificial Intelligence in Education. [find](https://scholar.google.com/scholar?q=Heather+Pon%2DBarry%2C+Karl+Schultz%2C+Elizabeth+Owen+Bratt%2C+Brady+Clark%2C+and+Stanley+Peters%2E+2006%2E+Responding+to+student+uncertainty+in+spoken+tutorial+dialogue+systems%2E+International+Journal+of+Artificial)
34. Adwait Ratnaparkhi. 2002. Trainable approaches to surface natural language generation and their application to conversational dialog systems. Computer Speech and Language. Spoken Language Generation. [find](https://scholar.google.com/scholar?q=Adwait+Ratnaparkhi%2E+2002%2E+Trainable+approaches+to+surface+natural+language+generation+and+their+application+to+conversational+dialog+systems%2E+Computer+Speech+and+Language%2E+Spoken+Language+Generation%2E)
35. Verena Rieser and Oliver Lemon. 2010. Natural language generation as planning under uncertainty for spoken dialogue systems. In Empirical Methods in Natural Language Generation, pages 105–120. [find](https://scholar.google.com/scholar?q=Verena+Rieser+and+Oliver+Lemon%2E+2010%2E+Natural+language+generation+as+planning+under+uncertainty+for+spoken+dialogue+systems%2E+In+Empirical+Methods+in+Natural+Language+Generation%2C+pages+105%E2%80%93120%2E)
36. Tara N Sainath, A-r Mohamed, Brian Kingsbury, and Bhuvana Ramabhadran. 2013. Deep convolutional neural networks for lvcsr. In Acoustics, Speech and Signal Processing (ICASSP), 2013 IEEE International Conference on, pages 8614–8618. IEEE. [find](https://scholar.google.com/scholar?q=Tara+N+Sainath%2C+A%2Dr+Mohamed%2C+Brian+Kingsbury%2C+and+Bhuvana+Ramabhadran%2E+2013%2E+Deep+convolutional+neural+networks+for+lvcsr%2E+In+Acoustics%2C+Speech+and+Signal+Processing+%28ICASSP%29%2C+2013+IEEE+International+)
37. Mike Schuster and Kuldip K Paliwal. 1997. Bidirectional recurrent neural networks. Signal Processing, IEEE Transactions on, 45(11):2673–2681. [find](https://scholar.google.com/scholar?q=Mike+Schuster+and+Kuldip+K+Paliwal%2E+1997%2E+Bidirectional+recurrent+neural+networks%2E+Signal+Processing%2C+IEEE+Transactions+on%2C+45%2811%29%3A2673%E2%80%932681%2E)
38. Richard Socher, Alex Perelygin, Jean Y. Wu, Jason Chuang, Christopher D. Manning, Andrew Y. Ng, and Christopher Potts. 2013. Recursive deep models for semantic compositionality over a sentiment treebank. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP… [find](https://scholar.google.com/scholar?q=Richard+Socher%2C+Alex+Perelygin%2C+Jean+Y%2E+Wu%2C+Jason+Chuang%2C+Christopher+D%2E+Manning%2C+Andrew+Y%2E+Ng%2C+and+Christopher+Potts%2E+2013%2E+Recursive+deep+models+for+semantic+compositionality+over+a+sentiment+treeba)
39. Amanda Stent, Rashmi Prasad, and Marilyn Walker. 2004. Trainable sentence planning for complex information presentation in spoken dialog systems. In In Proceedings of the Annual Meeting of the Association for Computational Linguistics, pages 79–86. [find](https://scholar.google.com/scholar?q=Amanda+Stent%2C+Rashmi+Prasad%2C+and+Marilyn+Walker%2E+2004%2E+Trainable+sentence+planning+for+complex+information+presentation+in+spoken+dialog+systems%2E+In+In+Proceedings+of+the+Annual+Meeting+of+the+Associa)
40. Martin Sundermeyer, Tamer Alkhouli, Joern Wuebker, and Hermann Ney. 2014. Translation modeling with bidirectional recurrent neural networks. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 14–25. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Martin+Sundermeyer%2C+Tamer+Alkhouli%2C+Joern+Wuebker%2C+and+Hermann+Ney%2E+2014%2E+Translation+modeling+with+bidirectional+recurrent+neural+networks%2E+In+Proceedings+of+the+2014+Conference+on+Empirical+Methods+)
41. Ilya Sutskever, James Martens, and Geoffrey E. Hinton. 2011. Generating text with recurrent neural networks. In Proceedings of the 28th International Conference on Machine Learning (ICML-11), pages 1017–1024, New York, NY, USA. ACM. [find](https://scholar.google.com/scholar?q=Ilya+Sutskever%2C+James+Martens%2C+and+Geoffrey+E%2E+Hinton%2E+2011%2E+Generating+text+with+recurrent+neural+networks%2E+In+Proceedings+of+the+28th+International+Conference+on+Machine+Learning+%28ICML%2D11%29%2C+pages+10)
42. Marilyn A Walker, Owen C Rambow, and Monica Rogati. 2002. Training a sentence planner for spoken dialogue using boosting. Computer Speech and Language, 16(3):409–433. [find](https://scholar.google.com/scholar?q=Marilyn+A+Walker%2C+Owen+C+Rambow%2C+and+Monica+Rogati%2E+2002%2E+Training+a+sentence+planner+for+spoken+dialogue+using+boosting%2E+Computer+Speech+and+Language%2C+16%283%29%3A409%E2%80%93433%2E)
43. Wayne Ward and Sunil Issar. 1994. Recent improvements in the cmu spoken language understanding system. In Proceedings of the Workshop on Human Language Technology, HLT ’94, pages 213– 216. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Wayne+Ward+and+Sunil+Issar%2E+1994%2E+Recent+improvements+in+the+cmu+spoken+language+understanding+system%2E+In+Proceedings+of+the+Workshop+on+Human+Language+Technology%2C+HLT+%E2%80%9994%2C+pages+213%E2%80%93+216%2E+Association)
44. Paul J Werbos. 1990. Backpropagation through time: what it does and how to do it. Proceedings of the IEEE, 78(10):1550–1560. [find](https://scholar.google.com/scholar?q=Paul+J+Werbos%2E+1990%2E+Backpropagation+through+time%3A+what+it+does+and+how+to+do+it%2E+Proceedings+of+the+IEEE%2C+78%2810%29%3A1550%E2%80%931560%2E)
45. Steve Young, Milica Gašić, Blaise Thomson, and Jason D. Williams. 2013. Pomdp-based statistical spoken dialog systems: A review. Proceedings of the IEEE, 101(5):1160–1179, May. [find](https://scholar.google.com/scholar?q=Steve+Young%2C+Milica+Gas%CC%8Cic%CC%81%2C+Blaise+Thomson%2C+and+Jason+D%2E+Williams%2E+2013%2E+Pomdp%2Dbased+statistical+spoken+dialog+systems%3A+A+review%2E+Proceedings+of+the+IEEE%2C+101%285%29%3A1160%E2%80%931179%2C+May%2E)
46. Xingxing Zhang and Mirella Lapata. 2014. Chinese poetry generation with recurrent neural networks. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 670–680. Association for Computational Linguistics, October. [find](https://scholar.google.com/scholar?q=Xingxing+Zhang+and+Mirella+Lapata%2E+2014%2E+Chinese+poetry+generation+with+recurrent+neural+networks%2E+In+Proceedings+of+the+2014+Conference+on+Empirical+Methods+in+Natural+Language+Processing+%28EMNLP%29%2C+pa)