--- title: "Visual Writing Prompts — paper by Hong et al. (2023)" description: "The authors introduce the notion of \"visual tellability\": a good image prompt should be coherent, diverse… 1 reference, cited by 22 papers here." authors: ["Xudong Hong", "Asad Sayeed", "Khushboo Mehra", "Vera Demberg", "Bernt Schiele"] year: 2023 venue: "Transactions of the Association for Computational Linguistics" source: "https://aclanthology.org/2023.tacl-1.33/" doi: "10.1162/tacl_a_00553" licence: "CC BY 4.0 (ACL Anthology)" licence_url: "https://creativecommons.org/licenses/by/4.0/" summary_licence: "Elamur Cosmic License" canonical: "https://research.elamur.ai/free/en/papers/hong-2023-visual-writing-prompts-character-grounded/" --- [Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/hong-2023-visual-writing-prompts-character-grounded/) [home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2023](https://md.research.elamur.ai/free/en/papers/2023/page/10/) / Visual Writing Prompts: Character-Grounded Story Generation with Curated Image Sequences # Visual Writing Prompts: Character-Grounded Story Generation with Curated Image Sequences **Authors:** Xudong Hong, [Asad Sayeed](https://md.research.elamur.ai/free/en/authors/asad-sayeed/), Khushboo Mehra, [Vera Demberg](https://md.research.elamur.ai/free/en/authors/vera-demberg/) and [Bernt Schiele](https://md.research.elamur.ai/free/en/authors/bernt-schiele/) · **Year:** 2023 · **Venue:** [Transactions of the Association for Computational Linguistics](https://md.research.elamur.ai/free/en/venues/tacl/) ([Transactions of the Association for Computational Linguistics (TACL) 2023](https://md.research.elamur.ai/free/en/venues/tacl/2023/)) · **Publisher:** [The MIT Press](https://md.research.elamur.ai/free/en/publishers/the-mit-press/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/) **Links:** [Publisher's page](https://aclanthology.org/2023.tacl-1.33/) · [PDF](https://aclanthology.org/2023.tacl-1.33.pdf) · [DOI 10.1162/tacl\_a\_00553](https://doi.org/10.1162/tacl_a_00553) · [OpenAlex](https://openalex.org/W4381611941) **Citations:** 25 citations (OpenAlex, 2026-10-14) · [22](https://md.research.elamur.ai/free/en/papers/hong-2023-visual-writing-prompts-character-grounded/cited-by/) in our library, of the 149,439 papers whose bibliographies we read ## Summary Visual story generation models are often trained on image sequences such as VIST's Flickr albums. These sequences tend to lack a real plot, and the people in them change from picture to picture, so the stories written for them are weak training data. The authors introduce the notion of "visual tellability": a good image prompt should be coherent, diverse, and visually grounded. To get such prompts, they built an automatic pipeline that draws shots from MovieNet scenes tied to a single synopsis paragraph and filters out unsuitable genres, near-duplicate frames, and adult content. Crowdworkers then wrote stories for these sequences while seeing the main characters' names and cropped images. The resulting dataset, Visual Writing Prompts (VWP), yields stories that are more coherent, diverse, and grounded than earlier collections. The authors also propose CharGrid, a generation model with a coherence component modeled on entity grids that tracks character continuity. CharGrid outperforms the previous state of the art, TAPM, on both automatic and human evaluations. This shows that careful data curation and explicit character modeling both improve image-based storytelling. ## Key points - **Dataset size:** VWP contains almost 2K movie-shot sequences of 5–10 images each, paired with about 12K crowdsourced stories of 50–300 words collected on Amazon Mechanical Turk. - **Character grounding:** Each story is linked to at most five main characters, and every mention of a character is tied to a bounding box marking that character in the matching image (taken from MovieNet annotations). - **Filtering pipeline:** - Fantasy, sci-fi, horror, and animated films were excluded. - Near-duplicate frames were removed when the cosine similarity of their ResNet-50 features exceeded 0.89. - **Automatic results:** On the VWP test set, CharGrid reaches a CIDEr score of about 4.83, compared with roughly 1.4–1.9 for TAPM variants. Baselines also include GPT-2 with object and character features. - **Human evaluation:** 28 workers compared 50 story pairs. CharGrid beat TAPM + char significantly on every criterion: +8.41 points on coherence, +11.06 on diversity, +6.49 on grammaticality, and +6.25 on visual groundedness. Code and data are available at vwprompt.github.io. *Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.* ## Institutions - [Saarland University](https://md.research.elamur.ai/free/en/universities/saarland-university/) — University, Saarbrücken, Germany · 149 works · 18,353 citations - [University of Gothenburg](https://md.research.elamur.ai/free/en/universities/university-of-gothenburg/) — University, Gothenburg, Sweden · 34 works · 1,625 citations 📚 Shelves (5): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 18,614 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 17,965 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 17,450 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 17,296 works · [👁️ Computer Vision and Pattern Recognition](https://md.research.elamur.ai/free/en/shelves/computer-science/computer-vision-and-pattern-recognition/) · 3,486 works 🧭 Topics (1): [✍️ Text generation](https://md.research.elamur.ai/free/en/topics/text-generation/) · 214 works ## Cite Cite as: Xudong Hong, Asad Sayeed, Khushboo Mehra, Vera Demberg and Bernt Schiele. 2023. Visual Writing Prompts: Character-Grounded Story Generation with Curated Image Sequences. Transactions of the Association for Computational Linguistics. The paper itself, not this page. BibTeX from the library's record of the work. ```bibtex @article{hong2023visual, title = {{Visual Writing Prompts: Character-Grounded Story Generation with Curated Image Sequences}}, author = {Xudong Hong, Asad Sayeed, Khushboo Mehra, Vera Demberg and Bernt Schiele}, journal = {Transactions of the Association for Computational Linguistics}, year = {2023}, doi = {10.1162/tacl\_a\_00553}, url = {https://doi.org/10.1162/tacl_a_00553} } ``` - [← previous Towards Mitigating Hallucination in Large Language Models via Self-Reflection](https://md.research.elamur.ai/free/en/papers/ji-2023-towards-mitigating-hallucination-large-language/) - [next → AlGhafa Evaluation Benchmark for Arabic Language Models](https://md.research.elamur.ai/free/en/papers/almazrouei-2023-alghafa-evaluation-benchmark-arabic-language/) - [↑ back to papers of 2023](https://md.research.elamur.ai/free/en/papers/2023/page/10/) ## References 1 references, 0 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet. 1. Nader Akoury, Shufan Wang, Josh Whiting, Stephen Hood, Nanyun Peng, and Mohit Iyyer. 2020. STORIUM: A dataset and evaluation platform for machine-in-the-loop story generation. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 6470–6484, Online.… [find](https://scholar.google.com/scholar?q=Nader+Akoury%2C+Shufan+Wang%2C+Josh+Whiting%2C+Stephen+Hood%2C+Nanyun+Peng%2C+and+Mohit+Iyyer%2E+2020%2E+STORIUM%3A+A+dataset+and+evaluation+platform+for+machine%2Din%2Dthe%2Dloop+story+generation%2E+In+Proceedings+of+the+20) *Updated 30 October 2026*