--- title: "Hierarchical Neural Story Generation — paper by Fan et al. (2018)" description: "To avoid this, the authors propose fusion: a second seq2seq model is trained on top of an already… 6 references, cited by 794 papers here." authors: ["Angela Fan", "Mike Lewis", "Yann Dauphin"] year: 2018 venue: "Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)" source: "https://aclanthology.org/P18-1082/" doi: "10.18653/v1/p18-1082" arxiv: "1805.04833" licence: "CC BY 4.0 (ACL Anthology)" licence_url: "https://creativecommons.org/licenses/by/4.0/" summary_licence: "Elamur Cosmic License" canonical: "https://research.elamur.ai/free/en/papers/fan-2018-hierarchical-neural-story-generation/" --- [Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/fan-2018-hierarchical-neural-story-generation/) [home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2018](https://md.research.elamur.ai/free/en/papers/2018/) / Hierarchical Neural Story Generation # Hierarchical Neural Story Generation **Authors:** [Angela Fan](https://md.research.elamur.ai/free/en/authors/angela-fan/), [Mike Lewis](https://md.research.elamur.ai/free/en/authors/mike-lewis/) and [Yann Dauphin](https://md.research.elamur.ai/free/en/authors/yann-dauphin/) · **Year:** 2018 · **Venue:** [Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)](https://md.research.elamur.ai/free/en/venues/acl/) ([ACL 2018](https://md.research.elamur.ai/free/en/venues/acl/2018/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/) **Links:** [Publisher's page](https://aclanthology.org/P18-1082/) · [PDF](https://aclanthology.org/P18-1082.pdf) · [DOI 10.18653/v1/p18-1082](https://doi.org/10.18653/v1/p18-1082) · [arXiv 1805.04833](https://arxiv.org/abs/1805.04833) · [OpenAlex](https://openalex.org/W2963096510) **Citations:** 1,215 citations (OpenAlex, 2026-10-14) · [794](https://md.research.elamur.ai/free/en/papers/fan-2018-hierarchical-neural-story-generation/cited-by/) in our library, of the 149,439 papers whose bibliographies we read ## Summary The paper addresses the automatic generation of long works of fiction. Such texts must be coherent and fluent, and must also stay on a given topic. The authors collected the WritingPrompts corpus from the Reddit forum r/WritingPrompts. It contains about 300K human-written stories, each tied to a short prompt. Generation proceeds on two levels: first, a convolutional language model (GCNN) produces a prompt, then a convolutional seq2seq model expands it into a full story. A standard seq2seq model often ignores the prompt and behaves like plain language modeling. To avoid this, the authors propose fusion: a second seq2seq model is trained on top of an already trained first one, and it learns what the first one failed to capture. To let the model take distant context into account, the decoder is augmented with a gated multi-scale self-attention mechanism, in which each head attends to the preceding text with a different downsampling step. Together, these components markedly improve both automatic metrics and human judgments. The work matters for two reasons: it provides an open resource, and it demonstrates a practical way to plan long text through an intermediate level. ## Key points - **WritingPrompts corpus:** 303,358 "prompt–story" pairs collected over three years via the Reddit API. Split: 272,600 for training, 15,620 for validation, 15,138 for test. A prompt averages 28.4 words, a story 734.5 words. Code and data are available in the fairseq repository. - **Vocabularies:** 19,025 tokens for prompts and 104,960 for stories. Only words occurring more than 10 times were included. Story length is capped at 1000 words. - **Perplexity:** self-attention and fusion together reduce it by about 9 points compared to the baseline Conv seq2seq. Fusion outperforms an ensemble despite having fewer parameters. - **Human evaluation (Amazon Mechanical Turk):** hierarchical stories are preferred twice as often as stories from a strong model without a prompt. Thanks to fusion, evaluators match a story to its prompt 7% more accurately. - **Text originality:** in 150-word generated stories, the longest common subsequence with the training set averages 8.9 words. For the baseline Conv seq2seq it is 10.2 words, while KNN copies all 150. Known weaknesses of the model: repetition, and the generated prompts come out formulaic. *Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.* ## Institutions - [Meta](https://md.research.elamur.ai/free/en/companies/meta/) — Company, Menlo Park, United States · 183 works · 228,558 citations 📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works 🧭 Topics (1): [✍️ Text generation](https://md.research.elamur.ai/free/en/topics/text-generation/) · 202 works ## Cite Cite as: Angela Fan, Mike Lewis and Yann Dauphin. 2018. Hierarchical Neural Story Generation. Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). The paper itself, not this page. BibTeX from the library's record of the work. ```bibtex @inproceedings{fan2018hierarchical, title = {{Hierarchical Neural Story Generation}}, author = {Angela Fan, Mike Lewis and Yann Dauphin}, booktitle = {Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)}, year = {2018}, doi = {10.18653/v1/p18-1082}, eprint = {1805.04833}, archivePrefix = {arXiv}, url = {https://doi.org/10.18653/v1/p18-1082} } ``` - [← previous Don't Give Me the Details, Just the Summary! Topic-Aware Convolutional Neural Networks for Extreme Summarization](https://md.research.elamur.ai/free/en/papers/narayan-2018-don-t-give-me-details/) - [next → Personalizing Dialogue Agents: I have a dog, do you have pets too?](https://md.research.elamur.ai/free/en/papers/zhang-2018-personalizing-dialogue-agents-i-have/) - [↑ back to papers of 2018](https://md.research.elamur.ai/free/en/papers/2018/) ## References 6 references, 4 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet. 1. Jimmy Lei Ba, Jamie Ryan Kiros, and Geoffrey E Hinton. 2016. [Layer normalization](https://md.research.elamur.ai/free/en/papers/ba-2016-layer-normalization/). arXiv preprint arXiv:1607.06450. 2. Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. 2015. [Neural machine translation by jointly learning to align and translate](https://md.research.elamur.ai/free/en/papers/bahdanau-2014-neural-machine-translation-jointly-learning/). International Conference on Learning Representation (ICLR). 3. Piotr Bojanowski, Edouard Grave, Armand Joulin, and Tomas Mikolov. 2016. [Enriching word vectors with subword information](https://md.research.elamur.ai/free/en/papers/bojanowski-2017-enriching-word-vectors-subword-information/). arXiv preprint arXiv:1607.04606. 4. Jan Chorowski and Navdeep Jaitly. 2016. Towards better decoding and language model integration in sequence to sequence models. arXiv preprint arXiv:1612.02695. — [arXiv:1612.02695](https://arxiv.org/abs/1612.02695) [find](https://scholar.google.com/scholar?q=Jan+Chorowski+and+Navdeep+Jaitly%2E+2016%2E+Towards+better+decoding+and+language+model+integration+in+sequence+to+sequence+models%2E+arXiv+preprint+arXiv%3A1612%2E02695%2E+%E2%80%94+%5BarXiv%3A1612%2E02695%5D%28https%3A%2F%2Farxiv%2Eorg%2Fa) 5. Yann N. Dauphin, Angela Fan, Michael Auli, and David Grangier. 2017. [Language modeling with gated convolutional networks](https://md.research.elamur.ai/free/en/papers/dauphin-2016-language-modeling-gated-convolutional-networks/). 6. Generation with Fusion of Conv seq2seq + Self-Attention Example Prompt 1: Aliens start abducting humans [find](https://scholar.google.com/scholar?q=Generation+with+Fusion+of+Conv+seq2seq+%2B+Self%2DAttention+Example+Prompt+1%3A+Aliens+start+abducting+humans) *Updated 29 October 2026*