---
title: "Learning Neural Templates for Text Generation — Wiseman 2018"
description: "Neural encoder-decoder systems produce fluent text from structured data, but they work as black boxes."
authors: ["Sam Wiseman", "Stuart Shieber", "Alexander Rush"]
year: 2018
venue: "Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing"
source: "https://aclanthology.org/D18-1356/"
doi: "10.18653/v1/d18-1356"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/wiseman-2018-learning-neural-templates-text-generation/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/wiseman-2018-learning-neural-templates-text-generation/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2018](https://md.research.elamur.ai/free/en/papers/2018/page/3/) / Learning Neural Templates for Text Generation

# Learning Neural Templates for Text Generation

**Authors:** [Sam Wiseman](https://md.research.elamur.ai/free/en/authors/sam-wiseman/), [Stuart Shieber](https://md.research.elamur.ai/free/en/authors/stuart-shieber/), [Alexander Rush](https://md.research.elamur.ai/free/en/authors/alexander-rush/) · **Year:** 2018 · **Venue:** Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/D18-1356/) · [PDF](https://aclanthology.org/D18-1356.pdf) · [DOI 10.18653/v1/d18-1356](https://doi.org/10.18653/v1/d18-1356) · [OpenAlex](https://openalex.org/W2889009749)

**Citations:** 184 citations (openalex-snapshot · 2026-10-14) · [95](#cited-by) in our library, of the 140884 papers whose bibliographies we read

## Summary
Neural encoder-decoder systems produce fluent text from structured data, but they work as black boxes. It is hard to tell which input facts a sentence expresses, and hard to steer what is said or how. Wiseman, Shieber and Rush address this by replacing the usual decoder with a neural hidden semi-markov model (HSMM). The HSMM splits the output into segments, and each segment is labeled with a discrete latent state. A sequence of such states acts as a learned "template," and record values are inserted into its slots through copy attention. Training needs no hand-written rules: the exact marginal likelihood is computed with a dynamic program, and gradients are backpropagated through it. The authors test the approach on two data-to-text benchmarks, the E2E restaurant dataset and WikiBio biographies. Its automatic scores come close to standard encoder-decoder baselines. In exchange, swapping templates changes which facts appear and in what order. The learned states also map consistently onto record types, which makes the outputs easier to interpret.

## Key points
- The model comes in two variants, NTemp and an autoregressive version, NTemp+AR. Both learn discrete templates jointly with generation.
- Experiments are run on two benchmarks, WikiBio (Lebret et al., 2016) and E2E (Novikova et al., 2017). WikiBio has about 1700 record types, while the model is limited to 135 latent states.
- State purity is the share of a state's words that come from its most frequent record type. NTemp scores 89.2% on E2E and 43.2% on WikiBio. NTemp+AR scores lower, 85.4% and 39.9%, because the autoregressive RNN takes over part of the typing.
- On WikiBio, individual states correspond to specific fields, for example state 132 for birth names, 101 for birth dates, 82 for nationalities and 20 for occupations.
- Keeping the input fixed and choosing a different template changes the word order and the set of mentioned fields. The authors argue this kind of control is much harder to achieve with encoder-decoder models.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

Fields: [Natural language processing](https://md.research.elamur.ai/free/en/shelves/nlp/)

## Cited by

Cited by

95 papers of our library cite it (of the 140884 whose bibliographies we read); 9 of them have a page here, 86 not yet.

- [Knowledge Graph Based Synthetic Corpus Generation for Knowledge-Enhanced Language Model Pre-training](https://md.research.elamur.ai/free/en/papers/agarwal-2021-knowledge-graph-based-synthetic-corpus/)
- [Controllable Paraphrase Generation with a Syntactic Exemplar](https://md.research.elamur.ai/free/en/papers/chen-2019-controllable-paraphrase-generation-syntactic-exemplar/)
- [Logical Natural Language Generation from Open-Domain Tables](https://md.research.elamur.ai/free/en/papers/chen-2020-logical-natural-language-generation-open/)
- [Reformulating Unsupervised Style Transfer as Paraphrase Generation](https://md.research.elamur.ai/free/en/papers/krishna-2020-reformulating-unsupervised-style-transfer-as/)
- [A Token-level Reference-free Hallucination Detection Benchmark for Free-form Text Generation](https://md.research.elamur.ai/free/en/papers/liu-2022-token-level-reference-free-hallucination/)
- [Encode, Tag, Realize: High-Precision Text Editing](https://md.research.elamur.ai/free/en/papers/malmi-2019-encode-tag-realize-high-precision/)
- [Step-by-Step: Separating Planning from Realization in Neural Data-to-Text Generation](https://md.research.elamur.ai/free/en/papers/moryossef-2019-step-step-separating-planning-realization/)
- [Planning with Learned Entity Prompts for Abstractive Summarization](https://md.research.elamur.ai/free/en/papers/narayan-2021-planning-learned-entity-prompts-abstractive/)
- [MoverScore: Text Generation Evaluating with Contextualized Embeddings and Earth Mover Distance](https://md.research.elamur.ai/free/en/papers/zhao-2019-moverscore-text-generation-evaluating-contextualized/)

## Cite

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{wiseman2018learning,
  title = {{Learning Neural Templates for Text Generation}},
  author = {Wiseman, Sam and Shieber, Stuart and Rush, Alexander},
  booktitle = {Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing},
  year = {2018},
  doi = {10.18653/v1/d18-1356},
  url = {https://doi.org/10.18653/v1/d18-1356}
}
```

- [← previous Learning Named Entity Tagger using Domain-Specific Dictionary](https://md.research.elamur.ai/free/en/papers/shang-2018-learning-named-entity-tagger-domain/)
- [next → Learning Scalar Adjective Intensity from Paraphrases](https://md.research.elamur.ai/free/en/papers/cocos-2018-learning-scalar-adjective-intensity-paraphrases/)
- [↑ back to papers of 2018](https://md.research.elamur.ai/free/en/papers/2018/page/3/)

## References

57 references, 16 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Gabor Angeli, Percy Liang, and Dan Klein. 2010. A simple domain-independent probabilistic approach to generation. In Proceedings of the 2010 Conference on Empirical Methods in Natural Language Processing, pages 502–512. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Gabor+Angeli%2C+Percy+Liang%2C+and+Dan+Klein%2E+2010%2E+A+simple+domain%2Dindependent+probabilistic+approach+to+generation%2E+In+Proceedings+of+the+2010+Conference+on+Empirical+Methods+in+Natural+Language+Process) [our card](https://md.research.elamur.ai/free/en/papers/angeli-2010-simple-domain-independent-probabilistic-approach/)
2. Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. 2015. Neural machine translation by jointly learning to align and translate. In ICLR. [find](https://scholar.google.com/scholar?q=Dzmitry+Bahdanau%2C+Kyunghyun+Cho%2C+and+Yoshua+Bengio%2E+2015%2E+Neural+machine+translation+by+jointly+learning+to+align+and+translate%2E+In+ICLR%2E) [our card](https://md.research.elamur.ai/free/en/papers/bahdanau-2014-neural-machine-translation-jointly-learning/)
3. Satanjeev Banerjee and Alon Lavie. 2005. Meteor: An automatic metric for mt evaluation with improved correlation with human judgments. In Proceedings of the acl workshop on intrinsic and extrinsic evaluation measures for machine translation and/or summarization, pages 65–72. [find](https://scholar.google.com/scholar?q=Satanjeev+Banerjee+and+Alon+Lavie%2E+2005%2E+Meteor%3A+An+automatic+metric+for+mt+evaluation+with+improved+correlation+with+human+judgments%2E+In+Proceedings+of+the+acl+workshop+on+intrinsic+and+extrinsic+eva) [our card](https://md.research.elamur.ai/free/en/papers/banerjee-2005-meteor-automatic-metric-mt-evaluation/)
4. Anja Belz. 2008. Automatic generation of weather forecast texts using comprehensive probabilistic generation-space models. Natural Language Engineering, 14(04):431–455. [find](https://scholar.google.com/scholar?q=Anja+Belz%2E+2008%2E+Automatic+generation+of+weather+forecast+texts+using+comprehensive+probabilistic+generation%2Dspace+models%2E+Natural+Language+Engineering%2C+14%2804%29%3A431%E2%80%93455%2E)
5. Anja Belz and Ehud Reiter. 2006. Comparing automatic and human evaluation of nlg systems. In 11th Conference of the European Chapter of the Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Anja+Belz+and+Ehud+Reiter%2E+2006%2E+Comparing+automatic+and+human+evaluation+of+nlg+systems%2E+In+11th+Conference+of+the+European+Chapter+of+the+Association+for+Computational+Linguistics%2E)
6. Andrew Chisholm, Will Radford, and Ben Hachey. 2017. Learning to generate one-sentence biographies from wikidata. CoRR, abs/1702.06235. [find](https://scholar.google.com/scholar?q=Andrew+Chisholm%2C+Will+Radford%2C+and+Ben+Hachey%2E+2017%2E+Learning+to+generate+one%2Dsentence+biographies+from+wikidata%2E+CoRR%2C+abs%2F1702%2E06235%2E)
7. KyungHyun Cho, Bart van Merrienboer, Dzmitry Bahdanau, and Yoshua Bengio. 2014. On the properties of neural machine translation: Encoder-decoder approaches. Eighth Workshop on Syntax, Semantics and Structure in Statistical Translation. [find](https://scholar.google.com/scholar?q=KyungHyun+Cho%2C+Bart+van+Merrienboer%2C+Dzmitry+Bahdanau%2C+and+Yoshua+Bengio%2E+2014%2E+On+the+properties+of+neural+machine+translation%3A+Encoder%2Ddecoder+approaches%2E+Eighth+Workshop+on+Syntax%2C+Semantics+and+St) [our card](https://md.research.elamur.ai/free/en/papers/cho-2014-properties-neural-machine-translation-encoder/)
8. Hanjun Dai, Bo Dai, Yan-Ming Zhang, Shuang Li, and Le Song. 2017. Recurrent hidden semi-markov model. In International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Hanjun+Dai%2C+Bo+Dai%2C+Yan%2DMing+Zhang%2C+Shuang+Li%2C+and+Le+Song%2E+2017%2E+Recurrent+hidden+semi%2Dmarkov+model%2E+In+International+Conference+on+Learning+Representations%2E)
9. Çaglar Gülçehre, Sungjin Ahn, Ramesh Nallapati, Bowen Zhou, and Yoshua Bengio. 2016. Pointing the unknown words. In ACL. [find](https://scholar.google.com/scholar?q=C%CC%A7aglar+Gu%CC%88lc%CC%A7ehre%2C+Sungjin+Ahn%2C+Ramesh+Nallapati%2C+Bowen+Zhou%2C+and+Yoshua+Bengio%2E+2016%2E+Pointing+the+unknown+words%2E+In+ACL%2E) [our card](https://md.research.elamur.ai/free/en/papers/gulcehre-2016-pointing-unknown-words/)
10. Sepp Hochreiter and Jürgen Schmidhuber. 1997. Long short-term memory. Neural Comput., 9:1735–1780. [find](https://scholar.google.com/scholar?q=Sepp+Hochreiter+and+Ju%CC%88rgen+Schmidhuber%2E+1997%2E+Long+short%2Dterm+memory%2E+Neural+Comput%2E%2C+9%3A1735%E2%80%931780%2E)
11. Blake Howald, Ravikumar Kondadadi, and Frank Schilder. 2013. Domain adaptable semantic clustering in statistical nlg. In Proceedings of the 10th International Conference on Computational Semantics (IWCS 2013)–Long Papers, pages 143–154. [find](https://scholar.google.com/scholar?q=Blake+Howald%2C+Ravikumar+Kondadadi%2C+and+Frank+Schilder%2E+2013%2E+Domain+adaptable+semantic+clustering+in+statistical+nlg%2E+In+Proceedings+of+the+10th+International+Conference+on+Computational+Semantics+%28IW)
12. Zhiting Hu, Zichao Yang, Xiaodan Liang, Ruslan Salakhutdinov, and Eric P Xing. 2017. Toward controlled generation of text. In International Conference on Machine Learning, pages 1587–1596. [find](https://scholar.google.com/scholar?q=Zhiting+Hu%2C+Zichao+Yang%2C+Xiaodan+Liang%2C+Ruslan+Salakhutdinov%2C+and+Eric+P+Xing%2E+2017%2E+Toward+controlled+generation+of+text%2E+In+International+Conference+on+Machine+Learning%2C+pages+1587%E2%80%931596%2E) [our card](https://md.research.elamur.ai/free/en/papers/hu-2017-toward-controlled-generation-text/)
13. Po-Sen Huang, Chong Wang, Sitao Huang, Dengyong Zhou, and Li Deng. 2018. Towards neural phrase-based machine translation. In International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Po%2DSen+Huang%2C+Chong+Wang%2C+Sitao+Huang%2C+Dengyong+Zhou%2C+and+Li+Deng%2E+2018%2E+Towards+neural+phrase%2Dbased+machine+translation%2E+In+International+Conference+on+Learning+Representations%2E)
14. Dan Jurafsky and James H Martin. 2014. Speech and language processing. Pearson London. [find](https://scholar.google.com/scholar?q=Dan+Jurafsky+and+James+H+Martin%2E+2014%2E+Speech+and+language+processing%2E+Pearson+London%2E)
15. Ravi Kondadadi, Blake Howald, and Frank Schilder. 2013. A statistical nlg framework for aggregated planning and realization. In Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), volume 1, pages 1406–1415. [find](https://scholar.google.com/scholar?q=Ravi+Kondadadi%2C+Blake+Howald%2C+and+Frank+Schilder%2E+2013%2E+A+statistical+nlg+framework+for+aggregated+planning+and+realization%2E+In+Proceedings+of+the+51st+Annual+Meeting+of+the+Association+for+Computatio)
16. Lingpeng Kong, Chris Dyer, and Noah A Smith. 2016. Segmental recurrent neural networks. In International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Lingpeng+Kong%2C+Chris+Dyer%2C+and+Noah+A+Smith%2E+2016%2E+Segmental+recurrent+neural+networks%2E+In+International+Conference+on+Learning+Representations%2E)
17. Ioannis Konstas and Mirella Lapata. 2013. A global model for concept-to-text generation. J. Artif. Intell. Res.(JAIR), 48:305–346. [find](https://scholar.google.com/scholar?q=Ioannis+Konstas+and+Mirella+Lapata%2E+2013%2E+A+global+model+for+concept%2Dto%2Dtext+generation%2E+J%2E+Artif%2E+Intell%2E+Res%2E%28JAIR%29%2C+48%3A305%E2%80%93346%2E)
18. Karen Kukich. 1983. Design of a knowledge-based report generator. In ACL, pages 145–150. [find](https://scholar.google.com/scholar?q=Karen+Kukich%2E+1983%2E+Design+of+a+knowledge%2Dbased+report+generator%2E+In+ACL%2C+pages+145%E2%80%93150%2E)
19. Rémi Lebret, David Grangier, and Michael Auli. 2016. Neural text generation from structured data with application to the biography domain. In EMNLP, pages 1203–1213. [find](https://scholar.google.com/scholar?q=Re%CC%81mi+Lebret%2C+David+Grangier%2C+and+Michael+Auli%2E+2016%2E+Neural+text+generation+from+structured+data+with+application+to+the+biography+domain%2E+In+EMNLP%2C+pages+1203%E2%80%931213%2E) [our card](https://md.research.elamur.ai/free/en/papers/lebret-2016-neural-text-generation-structured-data/)
20. Ondrej Dušek and Filip Jurcıcek. 2016. Sequence-to-sequence generation for spoken dialogue via deep syntax trees and strings. In The 54th Annual Meeting of the Association for Computational Linguistics, page 45. [find](https://scholar.google.com/scholar?q=Ondrej+Dus%CC%8Cek+and+Filip+Jurc%C4%B1cek%2E+2016%2E+Sequence%2Dto%2Dsequence+generation+for+spoken+dialogue+via+deep+syntax+trees+and+strings%2E+In+The+54th+Annual+Meeting+of+the+Association+for+Computational+Linguisti)
21. J. Li, R. Jia, H. He, and P. Liang. 2018. Delete, retrieve, generate: A simple approach to sentiment and style transfer. In North American Association for Computational Linguistics (NAACL). [find](https://scholar.google.com/scholar?q=J%2E+Li%2C+R%2E+Jia%2C+H%2E+He%2C+and+P%2E+Liang%2E+2018%2E+Delete%2C+retrieve%2C+generate%3A+A+simple+approach+to+sentiment+and+style+transfer%2E+In+North+American+Association+for+Computational+Linguistics+%28NAACL%29%2E) [our card](https://md.research.elamur.ai/free/en/papers/li-2018-delete-retrieve-generate-simple-approach/)
22. Mark JF Gales and Steve J Young. 1993. The theory of segmental hidden Markov models. University of Cambridge, Department of Engineering. [find](https://scholar.google.com/scholar?q=Mark+JF+Gales+and+Steve+J+Young%2E+1993%2E+The+theory+of+segmental+hidden+Markov+models%2E+University+of+Cambridge%2C+Department+of+Engineering%2E)
23. Percy Liang, Michael I Jordan, and Dan Klein. 2009. Learning semantic correspondences with less supervision. In ACL, pages 91–99. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Percy+Liang%2C+Michael+I+Jordan%2C+and+Dan+Klein%2E+2009%2E+Learning+semantic+correspondences+with+less+supervision%2E+In+ACL%2C+pages+91%E2%80%9399%2E+Association+for+Computational+Linguistics%2E) [our card](https://md.research.elamur.ai/free/en/papers/liang-2009-learning-semantic-correspondences-less-supervision/)
24. Albert Gatt and Ehud Reiter. 2009. Simplenlg: A realisation engine for practical applications. In Proceedings of the 12th European Workshop on Natural Language Generation, pages 90–93. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Albert+Gatt+and+Ehud+Reiter%2E+2009%2E+Simplenlg%3A+A+realisation+engine+for+practical+applications%2E+In+Proceedings+of+the+12th+European+Workshop+on+Natural+Language+Generation%2C+pages+90%E2%80%9393%2E+Association+for)
25. Chin-Yew Lin. 2004. Rouge: A package for automatic evaluation of summaries. Text Summarization Branches Out. [find](https://scholar.google.com/scholar?q=Chin%2DYew+Lin%2E+2004%2E+Rouge%3A+A+package+for+automatic+evaluation+of+summaries%2E+Text+Summarization+Branches+Out%2E)
26. Jiatao Gu, Zhengdong Lu, Hang Li, and Victor O. K. Li. 2016. Incorporating copying mechanism in sequence-to-sequence learning. In ACL. [find](https://scholar.google.com/scholar?q=Jiatao+Gu%2C+Zhengdong+Lu%2C+Hang+Li%2C+and+Victor+O%2E+K%2E+Li%2E+2016%2E+Incorporating+copying+mechanism+in+sequence%2Dto%2Dsequence+learning%2E+In+ACL%2E) [our card](https://md.research.elamur.ai/free/en/papers/gu-2016-incorporating-copying-mechanism-sequence-sequence/)
27. Tianyu Liu, Kexiang Wang, Lei Sha, Baobao Chang, and Zhifang Sui. 2018. Table-to-text generation by structure-aware seq2seq learning. In Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence. [find](https://scholar.google.com/scholar?q=Tianyu+Liu%2C+Kexiang+Wang%2C+Lei+Sha%2C+Baobao+Chang%2C+and+Zhifang+Sui%2E+2018%2E+Table%2Dto%2Dtext+generation+by+structure%2Daware+seq2seq+learning%2E+In+Proceedings+of+the+Thirty%2DSecond+AAAI+Conference+on+Artificial+)
28. Liang Lu, Lingpeng Kong, Chris Dyer, Noah A Smith, and Steve Renals. 2016. Segmental recurrent neural networks for end-to-end speech recognition. [find](https://scholar.google.com/scholar?q=Liang+Lu%2C+Lingpeng+Kong%2C+Chris+Dyer%2C+Noah+A+Smith%2C+and+Steve+Renals%2E+2016%2E+Segmental+recurrent+neural+networks+for+end%2Dto%2Dend+speech+recognition%2E)
29. Interspeech 2016, pages 385–389. [find](https://scholar.google.com/scholar?q=Interspeech+2016%2C+pages+385%E2%80%93389%2E)
30. Thang Luong, Hieu Pham, and Christopher D. Manning. 2015. Effective approaches to attention-based neural machine translation. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, EMNLP 2015, pages 1412– 1421. [find](https://scholar.google.com/scholar?q=Thang+Luong%2C+Hieu+Pham%2C+and+Christopher+D%2E+Manning%2E+2015%2E+Effective+approaches+to+attention%2Dbased+neural+machine+translation%2E+In+Proceedings+of+the+2015+Conference+on+Empirical+Methods+in+Natural+Lang)
31. Kathleen McKeown. 1992. Text generation - using discourse strategies and focus constraints to generate natural language text. Studies in natural language processing. Cambridge University Press. [find](https://scholar.google.com/scholar?q=Kathleen+McKeown%2E+1992%2E+Text+generation+%2D+using+discourse+strategies+and+focus+constraints+to+generate+natural+language+text%2E+Studies+in+natural+language+processing%2E+Cambridge+University+Press%2E)
32. Susan W McRoy, Songsak Channarukul, and Syed S Ali. 2000. Yag: A template-based generator for real-time systems. In Proceedings of the first international conference on Natural language generation-Volume 14, pages 264–267. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Susan+W+McRoy%2C+Songsak+Channarukul%2C+and+Syed+S+Ali%2E+2000%2E+Yag%3A+A+template%2Dbased+generator+for+real%2Dtime+systems%2E+In+Proceedings+of+the+first+international+conference+on+Natural+language+generation%2DVol)
33. Hongyuan Mei, Mohit Bansal, and Matthew R. Walter. 2016. What to talk about and how? selective generation using lstms with coarse-to-fine alignment. In NAACL HLT, pages 720–730. [find](https://scholar.google.com/scholar?q=Hongyuan+Mei%2C+Mohit+Bansal%2C+and+Matthew+R%2E+Walter%2E+2016%2E+What+to+talk+about+and+how%3F+selective+generation+using+lstms+with+coarse%2Dto%2Dfine+alignment%2E+In+NAACL+HLT%2C+pages+720%E2%80%93730%2E) [our card](https://md.research.elamur.ai/free/en/papers/mei-2016-what-talk-about-how-selective/)
34. Kevin P Murphy. 2002. Hidden semi-markov models (hsmms). unpublished notes, 2. [find](https://scholar.google.com/scholar?q=Kevin+P+Murphy%2E+2002%2E+Hidden+semi%2Dmarkov+models+%28hsmms%29%2E+unpublished+notes%2C+2%2E)
35. Vinod Nair and Geoffrey E Hinton. 2010. Rectified linear units improve restricted boltzmann machines. In Proceedings of the 27th international conference on machine learning (ICML-10), pages 807–814. [find](https://scholar.google.com/scholar?q=Vinod+Nair+and+Geoffrey+E+Hinton%2E+2010%2E+Rectified+linear+units+improve+restricted+boltzmann+machines%2E+In+Proceedings+of+the+27th+international+conference+on+machine+learning+%28ICML%2D10%29%2C+pages+807%E2%80%93814%2E)
36. Jekaterina Novikova, Ondrej Dušek, and Verena Rieser. 2017. The E2E dataset: New challenges for end-to-end generation. In Proceedings of the 18th Annual Meeting of the Special Interest Group on Discourse and Dialogue, Saarbrücken, Germany. [find](https://scholar.google.com/scholar?q=Jekaterina+Novikova%2C+Ondrej+Dus%CC%8Cek%2C+and+Verena+Rieser%2E+2017%2E+The+E2E+dataset%3A+New+challenges+for+end%2Dto%2Dend+generation%2E+In+Proceedings+of+the+18th+Annual+Meeting+of+the+Special+Interest+Group+on+Disco) [our card](https://md.research.elamur.ai/free/en/papers/novikova-2017-e2e-dataset-new-challenges-end/)
37. Mari Ostendorf, Vassilios V Digalakis, and Owen A Kimball. 1996. From hmm’s to segment models: A unified view of stochastic modeling for speech recognition. IEEE Transactions on speech and audio processing, 4(5):360–378. [find](https://scholar.google.com/scholar?q=Mari+Ostendorf%2C+Vassilios+V+Digalakis%2C+and+Owen+A+Kimball%2E+1996%2E+From+hmm%E2%80%99s+to+segment+models%3A+A+unified+view+of+stochastic+modeling+for+speech+recognition%2E+IEEE+Transactions+on+speech+and+audio+proce)
38. Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. Bleu: a method for automatic evaluation of machine translation. In Proceedings of the 40th annual meeting on association for computational linguistics, pages 311–318. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Kishore+Papineni%2C+Salim+Roukos%2C+Todd+Ward%2C+and+Wei%2DJing+Zhu%2E+2002%2E+Bleu%3A+a+method+for+automatic+evaluation+of+machine+translation%2E+In+Proceedings+of+the+40th+annual+meeting+on+association+for+computat) [our card](https://md.research.elamur.ai/free/en/papers/papineni-2002-bleu-method-automatic-evaluation-machine/)
39. Adam Paszke, Sam Gross, Soumith Chintala, Gregory Chanan, Edward Yang, Zachary DeVito, Zeming Lin, Alban Desmaison, Luca Antiga, and Adam Lerer. 2017. Automatic differentiation in pytorch. [find](https://scholar.google.com/scholar?q=Adam+Paszke%2C+Sam+Gross%2C+Soumith+Chintala%2C+Gregory+Chanan%2C+Edward+Yang%2C+Zachary+DeVito%2C+Zeming+Lin%2C+Alban+Desmaison%2C+Luca+Antiga%2C+and+Adam+Lerer%2E+2017%2E+Automatic+differentiation+in+pytorch%2E)
40. NIPS 2017 Autodiff Workshop. [find](https://scholar.google.com/scholar?q=NIPS+2017+Autodiff+Workshop%2E)
41. Lawrence R Rabiner. 1989. A tutorial on hidden markov models and selected applications in speech recognition. Proceedings of the IEEE, 77(2):257– 286. [find](https://scholar.google.com/scholar?q=Lawrence+R+Rabiner%2E+1989%2E+A+tutorial+on+hidden+markov+models+and+selected+applications+in+speech+recognition%2E+Proceedings+of+the+IEEE%2C+77%282%29%3A257%E2%80%93+286%2E)
42. Ehud Reiter and Robert Dale. 1997. Building applied natural language generation systems. Natural Language Engineering, 3(1):57–87. [find](https://scholar.google.com/scholar?q=Ehud+Reiter+and+Robert+Dale%2E+1997%2E+Building+applied+natural+language+generation+systems%2E+Natural+Language+Engineering%2C+3%281%29%3A57%E2%80%9387%2E)
43. Tianxiao Shen, Tao Lei, Regina Barzilay, and Tommi Jaakkola. 2017. Style transfer from non-parallel text by cross-alignment. In Advances in Neural Information Processing Systems, pages 6833–6844. [find](https://scholar.google.com/scholar?q=Tianxiao+Shen%2C+Tao+Lei%2C+Regina+Barzilay%2C+and+Tommi+Jaakkola%2E+2017%2E+Style+transfer+from+non%2Dparallel+text+by+cross%2Dalignment%2E+In+Advances+in+Neural+Information+Processing+Systems%2C+pages+6833%E2%80%936844%2E) [our card](https://md.research.elamur.ai/free/en/papers/shen-2017-style-transfer-non-parallel-text/)
44. Yikang Shen, Zhouhan Lin, Chin wei Huang, and Aaron Courville. 2018. Neural language modeling by jointly learning syntax and lexicon. In International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Yikang+Shen%2C+Zhouhan+Lin%2C+Chin+wei+Huang%2C+and+Aaron+Courville%2E+2018%2E+Neural+language+modeling+by+jointly+learning+syntax+and+lexicon%2E+In+International+Conference+on+Learning+Representations%2E)
45. Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. 2014. Dropout: A simple way to prevent neural networks from overfitting. The Journal of Machine Learning Research, 15(1):1929–1958. [find](https://scholar.google.com/scholar?q=Nitish+Srivastava%2C+Geoffrey+Hinton%2C+Alex+Krizhevsky%2C+Ilya+Sutskever%2C+and+Ruslan+Salakhutdinov%2E+2014%2E+Dropout%3A+A+simple+way+to+prevent+neural+networks+from+overfitting%2E+The+Journal+of+Machine+Learning+)
46. Ilya Sutskever, Oriol Vinyals, and Quoc VV Le. 2014. Sequence to sequence learning with neural networks. In Advances in Neural Information Processing Systems (NIPS), pages 3104–3112. [find](https://scholar.google.com/scholar?q=Ilya+Sutskever%2C+Oriol+Vinyals%2C+and+Quoc+VV+Le%2E+2014%2E+Sequence+to+sequence+learning+with+neural+networks%2E+In+Advances+in+Neural+Information+Processing+Systems+%28NIPS%29%2C+pages+3104%E2%80%933112%2E)
47. Hao Tang, Weiran Wang, Kevin Gimpel, and Karen Livescu. 2016. End-to-end training approaches for discriminative segmental models. In Spoken Language Technology Workshop (SLT), 2016 IEEE, pages 496–502. IEEE. [find](https://scholar.google.com/scholar?q=Hao+Tang%2C+Weiran+Wang%2C+Kevin+Gimpel%2C+and+Karen+Livescu%2E+2016%2E+End%2Dto%2Dend+training+approaches+for+discriminative+segmental+models%2E+In+Spoken+Language+Technology+Workshop+%28SLT%29%2C+2016+IEEE%2C+pages+496%E2%80%93502)
48. Ke M Tran, Yonatan Bisk, Ashish Vaswani, Daniel Marcu, and Kevin Knight. 2016. Unsupervised neural hidden markov models. In Proceedings of the Workshop on Structured Prediction for NLP, pages 63–71. [find](https://scholar.google.com/scholar?q=Ke+M+Tran%2C+Yonatan+Bisk%2C+Ashish+Vaswani%2C+Daniel+Marcu%2C+and+Kevin+Knight%2E+2016%2E+Unsupervised+neural+hidden+markov+models%2E+In+Proceedings+of+the+Workshop+on+Structured+Prediction+for+NLP%2C+pages+63%E2%80%9371%2E)
49. Ramakrishna Vedantam, C Lawrence Zitnick, and Devi Parikh. 2015. Cider: Consensus-based image description evaluation. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 4566–4575. [find](https://scholar.google.com/scholar?q=Ramakrishna+Vedantam%2C+C+Lawrence+Zitnick%2C+and+Devi+Parikh%2E+2015%2E+Cider%3A+Consensus%2Dbased+image+description+evaluation%2E+In+Proceedings+of+the+IEEE+conference+on+computer+vision+and+pattern+recognition%2C+) [our card](https://md.research.elamur.ai/free/en/papers/vedantam-2014-cider-consensus-based-image-description/)
50. Chong Wang, Yining Wang, Po-Sen Huang, Abdelrahman Mohamed, Dengyong Zhou, and Li Deng. 2017. Sequence modeling via segmentations. In International Conference on Machine Learning, pages 3674–3683. [find](https://scholar.google.com/scholar?q=Chong+Wang%2C+Yining+Wang%2C+Po%2DSen+Huang%2C+Abdelrahman+Mohamed%2C+Dengyong+Zhou%2C+and+Li+Deng%2E+2017%2E+Sequence+modeling+via+segmentations%2E+In+International+Conference+on+Machine+Learning%2C+pages+3674%E2%80%933683%2E)
51. Lu Wang and Claire Cardie. 2013. Domain-independent abstract generation for focused meeting summarization. In Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), volume 1, pages 1395–1405. [find](https://scholar.google.com/scholar?q=Lu+Wang+and+Claire+Cardie%2E+2013%2E+Domain%2Dindependent+abstract+generation+for+focused+meeting+summarization%2E+In+Proceedings+of+the+51st+Annual+Meeting+of+the+Association+for+Computational+Linguistics+%28V)
52. Sam Wiseman, Stuart Shieber, and Alexander Rush. 2017. Challenges in data-to-document generation. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 2253–2263. [find](https://scholar.google.com/scholar?q=Sam+Wiseman%2C+Stuart+Shieber%2C+and+Alexander+Rush%2E+2017%2E+Challenges+in+data%2Dto%2Ddocument+generation%2E+In+Proceedings+of+the+2017+Conference+on+Empirical+Methods+in+Natural+Language+Processing%2C+pages+2253%E2%80%93) [our card](https://md.research.elamur.ai/free/en/papers/wiseman-2017-challenges-data-document-generation/)
53. Zhilin Yang, Zihang Dai, Ruslan Salakhutdinov, and William W. Cohen. 2018. Breaking the softmax bottleneck: A high-rank RNN language model. In International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Zhilin+Yang%2C+Zihang+Dai%2C+Ruslan+Salakhutdinov%2C+and+William+W%2E+Cohen%2E+2018%2E+Breaking+the+softmax+bottleneck%3A+A+high%2Drank+RNN+language+model%2E+In+International+Conference+on+Learning+Representations%2E)
54. Zichao Yang, Phil Blunsom, Chris Dyer, and Wang Ling. 2016. Reference-aware language models. CoRR, abs/1611.01628. [find](https://scholar.google.com/scholar?q=Zichao+Yang%2C+Phil+Blunsom%2C+Chris+Dyer%2C+and+Wang+Ling%2E+2016%2E+Reference%2Daware+language+models%2E+CoRR%2C+abs%2F1611%2E01628%2E)
55. Lei Yu, Jan Buys, and Phil Blunsom. 2016. Online segment to segment neural transduction. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 1307–1316. [find](https://scholar.google.com/scholar?q=Lei+Yu%2C+Jan+Buys%2C+and+Phil+Blunsom%2E+2016%2E+Online+segment+to+segment+neural+transduction%2E+In+Proceedings+of+the+2016+Conference+on+Empirical+Methods+in+Natural+Language+Processing%2C+pages+1307%E2%80%931316%2E)
56. U 1 ∈ R m 3 ×d and U 2 ∈ R K×m 2 ×m 3 ; D(x) is defined analogously. For all experiments, m 1 = 64, m 2 = 32, and m 3 = 64. [find](https://scholar.google.com/scholar?q=U+1+%E2%88%88+R+m+3+%C3%97d+and+U+2+%E2%88%88+R+K%C3%97m+2+%C3%97m+3+%3B+D%28x%29+is+defined+analogously%2E+For+all+experiments%2C+m+1+%3D+64%2C+m+2+%3D+32%2C+and+m+3+%3D+64%2E)
57. Junbo Jake Zhao, Yoon Kim, Kelly Zhang, Alexander M. Rush, and Yann LeCun. 2018. Adversarially regularized autoencoders. In Proceedings of the 35th International Conference on Machine Learning, ICML 2018, pages 5897–5906. [find](https://scholar.google.com/scholar?q=Junbo+Jake+Zhao%2C+Yoon+Kim%2C+Kelly+Zhang%2C+Alexander+M%2E+Rush%2C+and+Yann+LeCun%2E+2018%2E+Adversarially+regularized+autoencoders%2E+In+Proceedings+of+the+35th+International+Conference+on+Machine+Learning%2C+ICML+2)