--- title: "Data-to-text Generation with Macro Planning — Puduppully 2021" description: "Neural encoder-decoder systems for data-to-text generation write fluent prose. 59 references, cited by 48 papers here." authors: ["Ratish Puduppully", "Mirella Lapata"] year: 2021 venue: "Transactions of the Association for Computational Linguistics" source: "https://aclanthology.org/2021.tacl-1.31/" doi: "10.1162/tacl_a_00381" licence: "CC BY 4.0 (ACL Anthology)" licence_url: "https://creativecommons.org/licenses/by/4.0/" summary_licence: "Elamur Cosmic License" canonical: "https://research.elamur.ai/free/en/papers/puduppully-2021-data-text-generation-macro-planning/" --- [Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/puduppully-2021-data-text-generation-macro-planning/) [home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2021](https://md.research.elamur.ai/free/en/papers/2021/page/4/) / Data-to-text Generation with Macro Planning # Data-to-text Generation with Macro Planning **Authors:** [Ratish Puduppully](https://md.research.elamur.ai/free/en/authors/ratish-puduppully/) and [Mirella Lapata](https://md.research.elamur.ai/free/en/authors/mirella-lapata/) · **Year:** 2021 · **Venue:** [Transactions of the Association for Computational Linguistics](https://md.research.elamur.ai/free/en/venues/tacl/) ([Transactions of the Association for Computational Linguistics (TACL) 2021](https://md.research.elamur.ai/free/en/venues/tacl/2021/)) · **Publisher:** [The MIT Press](https://md.research.elamur.ai/free/en/publishers/the-mit-press/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/) **Links:** [Publisher's page](https://aclanthology.org/2021.tacl-1.31/) · [PDF](https://aclanthology.org/2021.tacl-1.31.pdf) · [DOI 10.1162/tacl\_a\_00381](https://doi.org/10.1162/tacl_a_00381) · [OpenAlex](https://openalex.org/W3128366560) **Citations:** 50 citations (OpenAlex, 2026-10-14) · [48](https://md.research.elamur.ai/free/en/papers/puduppully-2021-data-text-generation-macro-planning/cited-by/) in our library, of the 141,840 papers whose bibliographies we read ## Summary Neural encoder-decoder systems for data-to-text generation write fluent prose. However, they often hallucinate facts, select content poorly and order it incoherently, and these problems worsen in long, multi-paragraph outputs such as sports reports. Puduppully and Lapata revisit the classic pipeline idea of separating planning from surface realization. They focus on macro planning: deciding at a high level which entities and events a document covers and how they are distributed across paragraphs. Benchmark summaries contain no explicit plans, so the authors derive them from training data. Each paragraph serves as a unit and is aligned with verbalized records about players, teams and game events, such as innings in baseball. At test time, the model first predicts a macro plan from a set of candidate paragraph plans. A generator then produces text conditioned on that plan instead of the raw table. On ROTOWIRE (NBA) and MLB (baseball), this approach outperforms strong baselines in automatic metrics and human evaluation, producing more factual, coherent and fluent summaries. It also makes generation easier to interpret and control, because plans can be edited or written by hand. ## Key points - Benchmarks: MLB (about 25K baseball game records paired with human summaries, built from box scores and play-by-play) and ROTOWIRE (about 5K NBA games with descriptions). - A macro plan is a sequence of paragraph plans separated by `

`. Each paragraph plan verbalizes entities (``) and half-inning events (``). - The method needs no post-processing (e.g., smoothing referring expressions) and no extra tools such as parsers or information extraction. Code, trained models and the macro-plan dataset are available at github.com/ratishsp/data2text-macro-plan-py. - Limitations: RG precision still has room to improve. MLB inputs average about 3100 tokens in the development set, which spreads attention thin and causes selection errors such as naming the wrong pitcher. On ROTOWIRE, the model handles numbers poorly. The authors suggest noisy channel decomposition and better number representations as possible remedies. - The authors argue macro planning pays off mainly for long texts with multiple discourse units, while micro planning (record-level) may suit short outputs better. *Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.* ## Institutions - [University of Edinburgh](https://md.research.elamur.ai/free/en/universities/university-of-edinburgh/) — University, Edinburgh, United Kingdom · 255 works · 68,771 citations 📚 Shelves (5): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 9,086 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 8,889 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 9,296 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 9,206 works · [👁️ Computer Vision and Pattern Recognition](https://md.research.elamur.ai/free/en/shelves/computer-science/computer-vision-and-pattern-recognition/) · 1,282 works 🧭 Topics (1): [✍️ Text generation](https://md.research.elamur.ai/free/en/topics/text-generation/) · 145 works ## Cite Cite as: Ratish Puduppully and Mirella Lapata. 2021. Data-to-text Generation with Macro Planning. Transactions of the Association for Computational Linguistics. The paper itself, not this page. BibTeX from the library's record of the work. ```bibtex @article{puduppully2021data, title = {{Data-to-text Generation with Macro Planning}}, author = {Ratish Puduppully and Mirella Lapata}, journal = {Transactions of the Association for Computational Linguistics}, year = {2021}, doi = {10.1162/tacl\_a\_00381}, url = {https://doi.org/10.1162/tacl_a_00381} } ``` - [← previous Do Long-Range Language Models Actually Use Long-Range Context?](https://md.research.elamur.ai/free/en/papers/sun-2021-do-long-range-language-models/) - [next → Understanding by Understanding Not: Modeling Negation in Language Models](https://md.research.elamur.ai/free/en/papers/hosseini-2021-understanding-understanding-not-modeling-negation/) - [↑ back to papers of 2021](https://md.research.elamur.ai/free/en/papers/2021/page/4/) ## References 59 references, 21 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet. 1. Robert Dale. 1989. Generating referring expressions in a domain of objects and processes. [find](https://scholar.google.com/scholar?q=Robert+Dale%2E+1989%2E+Generating+referring+expressions+in+a+domain+of+objects+and+processes%2E) 2. Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. 2015. [Neural machine translation by jointly learning to align and translate](https://md.research.elamur.ai/free/en/papers/bahdanau-2014-neural-machine-translation-jointly-learning/). In 3rd International Conference on Learning Pablo Duboue and Kathleen McKeown. 2002. Content planner construction via evolutionary algorithms and a corpus-based fitness fu… 3. Jiatao Gu, Zhengdong Lu, Hang Li, and Victor O. K. Li. 2016. [Incorporating copying mechanism in sequence-to-sequence learning](https://md.research.elamur.ai/free/en/papers/gu-2016-incorporating-copying-mechanism-sequence-sequence/). In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1631–1640, Berlin, Germany. Association for Computa… 4. Pablo A. Duboue and Kathleen R. McKeown. 2001. Empirically estimating order constraints for content planning in generation. In Proceedings of the 39th Annual Meeting of the Association for Computational Linguistics, pages 172–179, Toulouse, France. Association for Computational Linguistics. DOI: htt… [find](https://scholar.google.com/scholar?q=Pablo+A%2E+Duboue+and+Kathleen+R%2E+McKeown%2E+2001%2E+Empirically+estimating+order+constraints+for+content+planning+in+generation%2E+In+Proceedings+of+the+39th+Annual+Meeting+of+the+Association+for+Computation) 5. Caglar Gulcehre, Sungjin Ahn, Ramesh Nallapati, Bowen Zhou, and Yoshua Bengio. 2016. [Pointing the unknown words](https://md.research.elamur.ai/free/en/papers/gulcehre-2016-pointing-unknown-words/). In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 140–149, Berlin, Germany. Association for Computational Linguisti… 6. John C. Duchi, Elad Hazan, and Yoram Singer. 2011. Adaptive subgradient methods for online learning and stochastic optimization. Journal of Machine Learning Research, 12:2121–2159. [find](https://scholar.google.com/scholar?q=John+C%2E+Duchi%2C+Elad+Hazan%2C+and+Yoram+Singer%2E+2011%2E+Adaptive+subgradient+methods+for+online+learning+and+stochastic+optimization%2E+Journal+of+Machine+Learning+Research%2C+12%3A2121%E2%80%932159%2E) 7. Angela Fan, David Grangier, and Michael Auli. 2018. [Controllable abstractive summarization](https://md.research.elamur.ai/free/en/papers/fan-2018-controllable-abstractive-summarization/). In Proceedings of the 2nd Workshop on Neural Machine Translation and Generation, pages 45–54, Melbourne, Australia. Association for Computational Linguistics. 8. M. A. K. Halliday and Ruqaiya Hasan. 1976. Cohesion in English, London. Longman. DOI: https://doi.org/10.1162/neco .1997.9.8.1735 — [doi:10.1162/neco](https://doi.org/10.1162/neco) [↗](https://doi.org/10.1162/neco) 9. Claire Gardent, Anastasia Shimorina, Shashi Narayan, and Laura Perez-Beltrachini. 2017. [Creating training corpora for NLG micro-planners](https://md.research.elamur.ai/free/en/papers/gardent-2017-creating-training-corpora-nlg-micro/). In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 179–188, Vancouver, Canada. Association… 10. Sepp Hochreiter and Jürgen Schmidhuber. 1997. Long short-term memory. Neural Computation, 9:1735–1780. [find](https://scholar.google.com/scholar?q=Sepp+Hochreiter+and+Ju%CC%88rgen+Schmidhuber%2E+1997%2E+Long+short%2Dterm+memory%2E+Neural+Computation%2C+9%3A1735%E2%80%931780%2E) 11. Eduard H. Hovy. 1993. Automated discourse generation using discourse structure relations. Artificial Intelligence, 63(1-2):341–385. DOI: https://doi.org/10.1016/0004-3702 (93)90021-3 — [doi:10.1016/0004-3702](https://doi.org/10.1016/0004-3702) [↗](https://doi.org/10.1016/0004-3702) 12. Hayate Iso, Yui Uehara, Tatsuya Ishigaki, Hiroshi Noji, Eiji Aramaki, Ichiro Kobayashi, Yusuke Miyao, Naoaki Okazaki, and Hiroya Takamura. 2019. Learning to select, track, and generate for data-to-text. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages… [find](https://scholar.google.com/scholar?q=Hayate+Iso%2C+Yui+Uehara%2C+Tatsuya+Ishigaki%2C+Hiroshi+Noji%2C+Eiji+Aramaki%2C+Ichiro+Kobayashi%2C+Yusuke+Miyao%2C+Naoaki+Okazaki%2C+and+Hiroya+Takamura%2E+2019%2E+Learning+to+select%2C+track%2C+and+generate+for+data%2Dto%2Dtex) 13. Albert Gatt and Emiel Krahmer. 2018. Survey of the state of the art in natural language generation: Core tasks, applications and evaluation. J. Artif. Intell. Res., 61:65–170. DOI: https://doi.org/10.1613/jair.5477 [↗](https://doi.org/10.1613/jair.5477) 14. Heng Gong, Xiaocheng Feng, Bing Qin, and Ting Liu. 2019. Table-to-text generation with effective hierarchical encoder on three dimensions (row, column and time). In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Na… [find](https://scholar.google.com/scholar?q=Heng+Gong%2C+Xiaocheng+Feng%2C+Bing+Qin%2C+and+Ting+Liu%2E+2019%2E+Table%2Dto%2Dtext+generation+with+effective+hierarchical+encoder+on+three+dimensions+%28row%2C+column+and+time%29%2E+In+Proceedings+of+the+2019+Conference+) 15. Natural Language Generation Conference, pages 1–8, Harriman, New York, USA. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Natural+Language+Generation+Conference%2C+pages+1%E2%80%938%2C+Harriman%2C+New+York%2C+USA%2E+Association+for+Computational+Linguistics%2E) 16. Colin Kelly, Ann Copestake, and Nikiforos Karamanis. 2009. Investigating content selection for language generation using machine learning. In Proceedings of the 12th European Workshop on Natural Language Generation (ENLG 2009), pages 130–137, Athens, Greece. Association for Computational Linguistics… [find](https://scholar.google.com/scholar?q=Colin+Kelly%2C+Ann+Copestake%2C+and+Nikiforos+Karamanis%2E+2009%2E+Investigating+content+selection+for+language+generation+using+machine+learning%2E+In+Proceedings+of+the+12th+European+Workshop+on+Natural+Langu) 17. R. E. Longacre. 1979. The paragraph as a grammatical unit. In Talmy Givón, editor, Syntax and Semantics, volume 12, Academic Press Inc., pages 115–133. [find](https://scholar.google.com/scholar?q=R%2E+E%2E+Longacre%2E+1979%2E+The+paragraph+as+a+grammatical+unit%2E+In+Talmy+Givo%CC%81n%2C+editor%2C+Syntax+and+Semantics%2C+volume+12%2C+Academic+Press+Inc%2E%2C+pages+115%E2%80%93133%2E) 18. Yuta Kikuchi, Graham Neubig, Ryohei Sasano, Hiroya Takamura, and Manabu Okumura. 2016. [Controlling output length in neural encoder-decoders](https://md.research.elamur.ai/free/en/papers/kikuchi-2016-controlling-output-length-neural-encoder/). In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 1328–1338, Austin, Texas. Association for Computational Lingui… 19. Jordan J. Louviere, Terry N. Flynn, and A. A. J. Marley. 2015. Best-Worst Scaling: Theory, Methods and Applications, Cambridge University Press. DOI: https://doi.org/10.1017/CBO9781107337855 — [doi:10.1017/cbo9781107337855](https://doi.org/10.1017/cbo9781107337855) [↗](https://doi.org/10.1017/CBO9781107337855) 20. Jordan J. Louviere and George G. Woodworth. 1991. Best-worst scaling: A model for the largest difference judgments. University of Alberta: Working Paper. [find](https://scholar.google.com/scholar?q=Jordan+J%2E+Louviere+and+George+G%2E+Woodworth%2E+1991%2E+Best%2Dworst+scaling%3A+A+model+for+the+largest+difference+judgments%2E+University+of+Alberta%3A+Working+Paper%2E) 21. Guillaume Klein, Yoon Kim, Yuntian Deng, Jean Senellart, and Alexander Rush. 2017. [OpenNMT: Open-source toolkit for neural machine translation](https://md.research.elamur.ai/free/en/papers/klein-2017-opennmt-open-source-toolkit-neural/). In Proceedings of ACL 2017, System Demonstrations, pages 67–72, Vancouver, Canada. Association for Computational Linguistics. DOI: [https://doi.org](https://doi.org) /10.18653… [↗](https://doi.org) 22. Thang Luong, Hieu Pham, and Christopher D. Manning. 2015. [Effective approaches to attention-based neural machine translation](https://md.research.elamur.ai/free/en/papers/luong-2015-effective-approaches-attention-based-neural/). In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 1412–1421, Lisbon, Portugal. Association for Computational Linguistics. DOI:… 23. Ioannis Konstas and Mirella Lapata. 2013. Inducing document plans for concept-to-text generation. In Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, pages 1503–1514, Seattle, Washington, USA. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Ioannis+Konstas+and+Mirella+Lapata%2E+2013%2E+Inducing+document+plans+for+concept%2Dto%2Dtext+generation%2E+In+Proceedings+of+the+2013+Conference+on+Empirical+Methods+in+Natural+Language+Processing%2C+pages+1503%E2%80%93) 24. Kathleen R. McKeown. 1992. Text Generation. Studies in Natural Language Processing, Cambridge University Press. [find](https://scholar.google.com/scholar?q=Kathleen+R%2E+McKeown%2E+1992%2E+Text+Generation%2E+Studies+in+Natural+Language+Processing%2C+Cambridge+University+Press%2E) 25. Kathleen R. McKeown, Desmond A. Jordan, Shimei Pan, James Shaw, and Barry A. Allen. 1997. Language generation for multimedia healthcare briefings. In Fifth Conference on Applied Natural Language Processing, pages 277–282, Washington, DC, USA. Association for Computational Linguistics. DOI: https://d… [find](https://scholar.google.com/scholar?q=Kathleen+R%2E+McKeown%2C+Desmond+A%2E+Jordan%2C+Shimei+Pan%2C+James+Shaw%2C+and+Barry+A%2E+Allen%2E+1997%2E+Language+generation+for+multimedia+healthcare+briefings%2E+In+Fifth+Conference+on+Applied+Natural+Language+Proce) 26. Karen Kukich. 1983. [Design of a knowledge-based report generator](https://md.research.elamur.ai/free/en/papers/kukich-1983-design-knowledge-based-report-generator/). In 21st Annual Meeting of the Association for Computational Linguistics. DOI: https://doi.org/10.3115/981311.981340 [↗](https://doi.org/10.3115/981311.981340) 27. Anirban Laha, Parag Jain, Abhijit Mishra, and Karthik Sankaranarayanan. 2020. Scalable micro-planned generation of discourse from structured data. Computational Linguistics, 45(4):737–763. DOI: https://doi.org/10.1162/coli a 00363 [↗](https://doi.org/10.1162/coli) 28. Hongyuan Mei, Mohit Bansal, and Matthew R. Walter. 2016. [What to talk about and how? Selective generation using LSTMs with coarse-to-fine alignment](https://md.research.elamur.ai/free/en/papers/mei-2016-what-talk-about-how-selective/). In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, page… 29. Romain Paulus, Caiming Xiong, and Richard Socher. 2018. [A deep reinforced model for abstractive summarization](https://md.research.elamur.ai/free/en/papers/paulus-2017-deep-reinforced-model-abstractive-summarization/). In International Conference on Learning Representations. 30. Laura Perez-Beltrachini and Mirella Lapata. 2018. Bootstrapping generators from noisy data. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), pages 1516–1527, New Orleans, Louisia… [find](https://scholar.google.com/scholar?q=Laura+Perez%2DBeltrachini+and+Mirella+Lapata%2E+2018%2E+Bootstrapping+generators+from+noisy+data%2E+In+Proceedings+of+the+2018+Conference+of+the+North+American+Chapter+of+the+Association+for+Computational+Lin) 31. Amit Moryossef, Yoav Goldberg, and Ido Dagan. 2019. [Step-by-step: Separating planning from realization in neural data-to-text generation](https://md.research.elamur.ai/free/en/papers/moryossef-2019-step-step-separating-planning-realization/). In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long… 32. Ratish Puduppully, Li Dong, and Mirella Lapata. 2019a. Data-to-text generation with content selection and planning. In Proceedings of the 33rd AAAI Conference on Artificial Intelligence. Honolulu, Hawaii. DOI: https://doi.org/10.1609/aaai.v33i01.330 16908 — [doi:10.1609/aaai.v33i01.330](https://doi.org/10.1609/aaai.v33i01.330) [↗](https://doi.org/10.1609/aaai.v33i01.330) 33. Feng Nie, Jinpeng Wang, Jin-Ge Yao, Rong Pan, and Chin-Yew Lin. 2018. Operation-guided neural networks for high fidelity data-to-text generation. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 3879–3889, Brussels, Belgium. Association for Computation… [find](https://scholar.google.com/scholar?q=Feng+Nie%2C+Jinpeng+Wang%2C+Jin%2DGe+Yao%2C+Rong+Pan%2C+and+Chin%2DYew+Lin%2E+2018%2E+Operation%2Dguided+neural+networks+for+high+fidelity+data%2Dto%2Dtext+generation%2E+In+Proceedings+of+the+2018+Conference+on+Empirical+Met) 34. Ratish Puduppully, Li Dong, and Mirella Lapata. 2019b. [Data-to-text generation with entity modeling](https://md.research.elamur.ai/free/en/papers/puduppully-2019-data-text-generation-entity-modeling/). In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 2023–2035, Florence, Italy. Association for Computational Linguistics. DOI: https://doi.org/10.18653/… 35. Jekaterina Novikova, Ondřej Dušek, and Verena Rieser. 2017. [The E2E dataset: New challenges for end-to-end generation](https://md.research.elamur.ai/free/en/papers/novikova-2017-e2e-dataset-new-challenges-end/). In Proceedings of the 18th Annual SIGdial Meeting on Discourse and Dialogue, pages 201–206, Saarbrücken, Germany. Association for Computational Linguistics. DOI: https://doi.org/… 36. Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. 2019. Language models are unsupervised multitask learners. OpenAI blog, 1(8):9. DOI: https://doi.org/10.18653/v1/P19 -1195 [↗](https://doi.org/10.18653/v1/P19) 37. Clément Rebuffel, Laure Soulier, Geoffrey Scoutheeten, and Patrick Gallinari. 2020. A hierarchical model for data-to-text generation. In European Conference on Information Retrieval, pages 65–80. Springer. DOI: https://doi.org/10.1007/978-3-030 -45439-5 5, PMCID: PMC7148215 — [doi:10.1007/978-3-030](https://doi.org/10.1007/978-3-030) [↗](https://doi.org/10.1007/978-3-030) 38. Bryan Orme. 2009. Maxdiff analysis: Simple counting, individual-level logit, and HB. Sawtooth Software. [find](https://scholar.google.com/scholar?q=Bryan+Orme%2E+2009%2E+Maxdiff+analysis%3A+Simple+counting%2C+individual%2Dlevel+logit%2C+and+HB%2E+Sawtooth+Software%2E) 39. Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. [BLEU: a method for automatic evaluation of machine translation](https://md.research.elamur.ai/free/en/papers/papineni-2002-bleu-method-automatic-evaluation-machine/). In Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, pages 311–318, Philadelphia, Pennsylvania, USA. Association for Computational… 40. Ehud Reiter. 1995. NLG vs. templates. CoRR, cmp-lg/9504013v1. DOI: https://doi.org/10.1017/S1351324997001502 — [doi:10.1017/s1351324997001502](https://doi.org/10.1017/s1351324997001502) [↗](https://doi.org/10.1017/S1351324997001502) 41. Ehud Reiter and Robert Dale. 1997. Building applied natural language generation systems. Natural Language Engineering, 3(1):57–87. [find](https://scholar.google.com/scholar?q=Ehud+Reiter+and+Robert+Dale%2E+1997%2E+Building+applied+natural+language+generation+systems%2E+Natural+Language+Engineering%2C+3%281%29%3A57%E2%80%9387%2E) 42. Ehud Reiter and Robert Dale. 2000. Building Natural Language Generation Systems. Studies in Natural Language Processing, Cambridge University Press. DOI: https://doi.org/10.1017/CBO9780511519857 — [doi:10.1017/cbo9780511519857](https://doi.org/10.1017/cbo9780511519857) [↗](https://doi.org/10.1017/CBO9780511519857) 43. Ran Tian, Shashi Narayan, Thibault Sellam, and Ankur P. Parikh. 2019. Sticking to the facts: Confident decoding for faithful data-to-text generation. CoRR, abs/1910.08684v2. — [arXiv:1910.08684](https://arxiv.org/abs/1910.08684) [find](https://scholar.google.com/scholar?q=Ran+Tian%2C+Shashi+Narayan%2C+Thibault+Sellam%2C+and+Ankur+P%2E+Parikh%2E+2019%2E+Sticking+to+the+facts%3A+Confident+decoding+for+faithful+data%2Dto%2Dtext+generation%2E+CoRR%2C+abs%2F1910%2E08684v2%2E+%E2%80%94+%5BarXiv%3A1910%2E08684%5D%28https) 44. Chris van der Lee, Albert Gatt, Emiel van Miltenburg, Sander Wubben, and Emiel Krahmer. 2019. Best practices for the human evaluation of automatically generated text. In Proceedings of the 12th International Conference on Natural Language Generation, pages 355–368, Tokyo, Japan. Association for Comp… [find](https://scholar.google.com/scholar?q=Chris+van+der+Lee%2C+Albert+Gatt%2C+Emiel+van+Miltenburg%2C+Sander+Wubben%2C+and+Emiel+Krahmer%2E+2019%2E+Best+practices+for+the+human+evaluation+of+automatically+generated+text%2E+In+Proceedings+of+the+12th+Intern) 45. Fahimeh Saleh, Alexandre Berard, Ioan Calapodescu, and Laurent Besacier. 2019. Naver Labs Europe’s systems for the document-level generation and translation task at WNGT 2019. In Proceedings of the 3rd Workshop on Neural Generation and Translation, pages 273–279, Hong Kong. Association for Computati… [find](https://scholar.google.com/scholar?q=Fahimeh+Saleh%2C+Alexandre+Berard%2C+Ioan+Calapodescu%2C+and+Laurent+Besacier%2E+2019%2E+Naver+Labs+Europe%E2%80%99s+systems+for+the+document%2Dlevel+generation+and+translation+task+at+WNGT+2019%2E+In+Proceedings+of+the+3r) 46. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. [Attention is all you need](https://md.research.elamur.ai/free/en/papers/vaswani-2017-attention-all-need/). In I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett, editors, Advances in Neural Information Proces… 47. Rico Sennrich, Barry Haddow, and Alexandra Birch. 2016. [Neural machine translation of rare words with subword units](https://md.research.elamur.ai/free/en/papers/sennrich-2016-neural-machine-translation-rare-words/). In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1715–1725, Berlin, Germany. Association for Computational Lin… 48. Oriol Vinyals, Meire Fortunato, and Navdeep Jaitly. 2015. Pointer networks. In C. Cortes, N. D. Lawrence, D. D. Lee, M. Sugiyama, and R. Garnett, editors, Advances in Neural Information Processing Systems 28, pages 2692–2700, Curran Associates, Inc. [find](https://scholar.google.com/scholar?q=Oriol+Vinyals%2C+Meire+Fortunato%2C+and+Navdeep+Jaitly%2E+2015%2E+Pointer+networks%2E+In+C%2E+Cortes%2C+N%2E+D%2E+Lawrence%2C+D%2E+D%2E+Lee%2C+M%2E+Sugiyama%2C+and+R%2E+Garnett%2C+editors%2C+Advances+in+Neural+Information+Processing+Sys) 49. Zhihong Shao, Minlie Huang, Jiangtao Wen, Wenfei Xu, and Xiaoyan Zhu. 2019. Long and diverse text generation with planning-based hierarchical variational model. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Nat… [find](https://scholar.google.com/scholar?q=Zhihong+Shao%2C+Minlie+Huang%2C+Jiangtao+Wen%2C+Wenfei+Xu%2C+and+Xiaoyan+Zhu%2E+2019%2E+Long+and+diverse+text+generation+with+planning%2Dbased+hierarchical+variational+model%2E+In+Proceedings+of+the+2019+Conference+o) 50. Eric Wallace, Yizhong Wang, Sujian Li, Sameer Singh, and Matt Gardner. 2019. [Do NLP models know numbers? Probing numeracy in embeddings](https://md.research.elamur.ai/free/en/papers/wallace-2019-do-nlp-models-know-numbers/). In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing… 51. Ilya Sutskever, Oriol Vinyals, and Quoc V. Le. 2014. Sequence to sequence learning with neural networks. In Advances in Neural Information Processing Systems, volume 27, pages 3104–3112. Curran Associates, Inc. [find](https://scholar.google.com/scholar?q=Ilya+Sutskever%2C+Oriol+Vinyals%2C+and+Quoc+V%2E+Le%2E+2014%2E+Sequence+to+sequence+learning+with+neural+networks%2E+In+Advances+in+Neural+Information+Processing+Systems%2C+volume+27%2C+pages+3104%E2%80%933112%2E+Curran+Associ) 52. Ronald J. Williams and Jing Peng. 1990. An efficient gradient-based algorithm for on-line training of recurrent network trajectories. Neural Computation, 2(4):490–501. DOI: https://doi.org/10.1162/neco.1990 .2.4.490 — [doi:10.1162/neco.1990](https://doi.org/10.1162/neco.1990) [↗](https://doi.org/10.1162/neco.1990) 53. Shunsuke Takeno, Masaaki Nagata, and Kazuhide Yamamoto. 2017. Controlling target features in neural machine translation via prefix constraints. In Proceedings of the 4th Workshop on Asian Translation (WAT 2017), pages 55–63, Taipei, Taiwan. Asian Federation of Natural Language Processing. [find](https://scholar.google.com/scholar?q=Shunsuke+Takeno%2C+Masaaki+Nagata%2C+and+Kazuhide+Yamamoto%2E+2017%2E+Controlling+target+features+in+neural+machine+translation+via+prefix+constraints%2E+In+Proceedings+of+the+4th+Workshop+on+Asian+Translation+) 54. Sam Wiseman, Stuart Shieber, and Alexander Rush. 2017. [Challenges in data-to-document generation](https://md.research.elamur.ai/free/en/papers/wiseman-2017-challenges-data-document-generation/). In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 2253–2263, Copenhagen, Denmark. Association for Computational Linguistics. DOI: https://doi .org/10.18653… 55. Linguistics: Human Language Technologies, pages 1480–1489, San Diego, California. Association for Computational Linguistics. DOI: https://doi.org/10.18653/v1/N16 -1174 — [doi:10.18653/v1/n16](https://doi.org/10.18653/v1/n16) [↗](https://doi.org/10.18653/v1/N16) 56. Kyra Yee, Yann Dauphin, and Michael Auli. 2019. [Simple and effective noisy channel modeling for neural machine translation](https://md.research.elamur.ai/free/en/papers/yee-2019-simple-effective-noisy-channel-modeling/). In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNL… 57. Yonghui Wu, Mike Schuster, Zhifeng Chen, Quoc V. Le, Mohammad Norouzi, Wolfgang Macherey, Maxim Krikun, Yuan Cao, Qin Gao, Klaus Macherey, Jeff Klingner, Apurva Shah, Melvin Johnson, Xiaobing Liu, Lukasz Kaiser, Stephan Gouws, Yoshikiyo Kato, Taku Kudo, Hideto Kazawa, Keith Stevens, George Kurian, N… [find](https://scholar.google.com/scholar?q=Yonghui+Wu%2C+Mike+Schuster%2C+Zhifeng+Chen%2C+Quoc+V%2E+Le%2C+Mohammad+Norouzi%2C+Wolfgang+Macherey%2C+Maxim+Krikun%2C+Yuan+Cao%2C+Qin+Gao%2C+Klaus+Macherey%2C+Jeff+Klingner%2C+Apurva+Shah%2C+Melvin+Johnson%2C+Xiaobing+Liu%2C+Luk) 58. Lei Yu, Laurent Sartran, Wojciech Stokowiec, Wang Ling, Lingpeng Kong, Phil Blunsom, and Chris Dyer. 2020. Better document-level machine translation with Bayes’ rule. Transactions of the Association for Computational Linguistics, 8:346–360. DOI: https://doi.org/10.1162/tacl a 00319 [↗](https://doi.org/10.1162/tacl) 59. Zichao Yang, Diyi Yang, Chris Dyer, Xiaodong He, Alex Smola, and Eduard Hovy. 2016. [Hierarchical attention networks for document classification](https://md.research.elamur.ai/free/en/papers/yang-2016-hierarchical-attention-networks-document-classification/). In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Wlodek Zadrozny and Karen Jensen. 1991. Semantics… *Updated 24 October 2026*