--- title: "DREAM: A Multicultural Multimodal Dataset — Mallo 2026" description: "Multimodal dialogue corpora rarely combine conversations with image sequences that keep each speaker's appearance stable across turns, and they seldom…" authors: ["Juan Mallo", "Marcos Estecha-Garitagoitia", "Ricardo Cordoba", "Luis Fernando D'Haro"] year: 2026 venue: "Proceedings of the Fifteenth Language Resources and Evaluation Conference" source: "https://aclanthology.org/2026.lrec-1.728/" doi: "10.63317/2v7b4xhs2d5g" licence: "CC BY-NC 4.0 (ELRA, LREC proceedings)" licence_url: "https://creativecommons.org/licenses/by-nc/4.0/" summary_licence: "Elamur Cosmic License" canonical: "https://research.elamur.ai/free/papers/mallo-2026-dream-multicultural-multimodal-dataset-linking/" --- [Elamur Research](https://md.research.elamur.ai/) · [this page in HTML](https://research.elamur.ai/free/papers/mallo-2026-dream-multicultural-multimodal-dataset-linking/) [home](https://md.research.elamur.ai/) / [papers](https://md.research.elamur.ai/free/papers/) / [2026](https://md.research.elamur.ai/free/papers/2026/page/2/) / DREAM: A Multicultural Multimodal Dataset Linking Dialogues and Realistic Image Sequences # DREAM: A Multicultural Multimodal Dataset Linking Dialogues and Realistic Image Sequences **Authors:** Juan Mallo, Marcos Estecha-Garitagoitia, Ricardo Cordoba, [Luis Fernando D'Haro](https://md.research.elamur.ai/free/authors/luis-fernando-d-haro/) · **Year:** 2026 · **Venue:** Proceedings of the Fifteenth Language Resources and Evaluation Conference · **Licence:** [CC BY-NC 4.0 (ELRA, LREC proceedings)](https://creativecommons.org/licenses/by-nc/4.0/) **Links:** [Publisher's page](https://aclanthology.org/2026.lrec-1.728/) · [PDF](https://aclanthology.org/2026.lrec-1.728.pdf) · [DOI 10.63317/2v7b4xhs2d5g](https://doi.org/10.63317/2v7b4xhs2d5g) · [OpenAlex](https://openalex.org/W7160169963) **Citations:** 0 citations (openalex-snapshot · 2026-10-14) ## Summary Multimodal dialogue corpora rarely combine conversations with image sequences that keep each speaker's appearance stable across turns, and they seldom control demographic diversity in a systematic way. DREAM addresses this gap. It is a synthetic resource of 1,000 persona-grounded dialogues taken from PersonaChat and ComperDial. Each speaker gets an expanded profile with structured and narrative layers, plus a photorealistic portrait. Selected dialogue turns are then illustrated as storyboard-style scenes, using the portraits as identity anchors. The authors built a model-agnostic, batchable pipeline: o4-mini handles persona expansion and gpt-image-1 renders portraits and scenes. Prompts are culturally aware, and turn selection targets visually salient moments. Everything is packaged in one JSON schema. Quality is assessed in two ways: human ratings of realism, coherence, consistency and perceived demographics, and automated portrait analysis with GPT-5. Annotators' comments are grouped into eight recurring failure categories, such as identity drift and anatomy errors. The resource enables studies of visual grounding, character continuity, and bias in appearance-based perception, while avoiding real personal data. ## Key points - The corpus has 1,000 dialogues: 900 from PersonaChat (train_both_revised.txt, out of 8,939 available) and 100 from ComperDial (its full public set). Each dialogue has two portraits and a set of scene images. - Each persona combines two layers: - **profile_struct:** age, gender_identity, sexual_orientation, a 10-cluster appearance-based ethnicity, nationality, and residence_country. Values come from textual evidence where available, otherwise from controlled fallback distributions. - **profile_narrative:** five sections of descriptive text. - Nationality and residence are kept separate, which allows diaspora-like profiles. Visual turns are tagged with a scene type (shared, memory, imagined, cutaway, montage) and a speaker focus. - The generation stack uses o4-mini for text and gpt-image-1 for images. GPT-5 serves as the vision–language evaluator and returns strict JSON labels for ethnicity, gender and age. - Free-text human feedback falls into eight error types: anatomy, object physics, text/iconography, identity drift, wardrobe continuity, gaze/interaction, scene–dialogue mismatch, and object semantics. *Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.* Fields: [Linguistics & languages](https://md.research.elamur.ai/free/shelves/linguistics/) ## Cite The paper itself, not this page. BibTeX from the library's record of the work. ```bibtex @inproceedings{mallo2026dream, title = {{DREAM: A Multicultural Multimodal Dataset Linking Dialogues and Realistic Image Sequences}}, author = {Mallo, Juan and Estecha-Garitagoitia, Marcos and Cordoba, Ricardo and D'Haro, Luis Fernando}, booktitle = {Proceedings of the Fifteenth Language Resources and Evaluation Conference}, year = {2026}, doi = {10.63317/2v7b4xhs2d5g}, url = {https://doi.org/10.63317/2v7b4xhs2d5g} } ``` - [← previous Domain-Aware Error Correction for Citation NER in Medieval Hebrew Responsa](https://md.research.elamur.ai/free/papers/aclbulk-2026-liebeskind-domain-aware-error-correction-citation/) - [next → DRInQ: Evaluating Conversational Implicature with Controlled Context Variation](https://md.research.elamur.ai/free/papers/arai-2026-drinq-evaluating-conversational-implicature-controlled/) - [↑ back to papers of 2026](https://md.research.elamur.ai/free/papers/2026/page/2/) ## References 15 references, 2 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet. 1. American Psychological Association. 2023. Gender identity (apa dictionary of psychology). [https://dictionary.apa.org/gender-ident](https://dictionary.apa.org/gender-ident) ity. [↗](https://dictionary.apa.org/gender-ident) 2. Peter Anderson, Qi Wu, Damien Teney, Jake Bruce, Mark Johnson, Niko Sünderhauf, Ian Reid, Stephen Gould, and Anton van den Hengel. 2018. Vision-and-language navigation: Interpreting visually-grounded navigation instructions in real environments. In CVPR. [find](https://scholar.google.com/scholar?q=Peter+Anderson%2C+Qi+Wu%2C+Damien+Teney%2C+Jake+Bruce%2C+Mark+Johnson%2C+Niko+S%C3%BCnderhauf%2C+Ian+Reid%2C+Stephen+Gould%2C+and+Anton+van+den+Hengel%2E+2018%2E+Vision%2Dand%2Dlanguage+navigation%3A+Interpreting+visually%2Dgrounded+) [our card](https://md.research.elamur.ai/free/papers/anderson-2017-vision-language-navigation-interpreting-visually/) 3. Emily M. Bender and Batya Friedman. 2018. Data statements for NLP: Toward mitigating system bias and enabling better science. Transactions of the Association for Computational Linguistics, 6:587–604. [find](https://scholar.google.com/scholar?q=Emily+M%2E+Bender+and+Batya+Friedman%2E+2018%2E+Data+statements+for+NLP%3A+Toward+mitigating+system+bias+and+enabling+better+science%2E+Transactions+of+the+Association+for+Computational+Linguistics%2C+6%3A587%E2%80%93604%2E) 4. Stephen Brade, Bryan Wang, Mauricio Sousa, Sageev Oore, and Tovi Grossman. 2023. Promptify: Text-to-image generation through interactive prompt exploration with large language models. In UIST. [find](https://scholar.google.com/scholar?q=Stephen+Brade%2C+Bryan+Wang%2C+Mauricio+Sousa%2C+Sageev+Oore%2C+and+Tovi+Grossman%2E+2023%2E+Promptify%3A+Text%2Dto%2Dimage+generation+through+interactive+prompt+exploration+with+large+language+models%2E+In+UIST%2E) 5. Centers for Disease Control and Prevention. 2024. Preferred terms for select population groups & communities. [https://www.cdc.gov/heal](https://www.cdc.gov/heal) th-communication/php/toolkit/prefe rred-terms.html. [↗](https://www.cdc.gov/heal) 6. Abhishek Das, Satwik Kottur, Khushi Gupta, Amanpreet Singh, Deshraj Yadav, José M. F. Moura, Devi Parikh, and Dhruv Batra. 2017. Visual dialog. In CVPR. [find](https://scholar.google.com/scholar?q=Abhishek+Das%2C+Satwik+Kottur%2C+Khushi+Gupta%2C+Amanpreet+Singh%2C+Deshraj+Yadav%2C+Jos%C3%A9+M%2E+F%2E+Moura%2C+Devi+Parikh%2C+and+Dhruv+Batra%2E+2017%2E+Visual+dialog%2E+In+CVPR%2E) 7. Harm de Vries, Florian Strub, Sarath Chandar, Olivier Pietquin, Hugo Larochelle, and Aaron Courville. 2017. Guesswhat?! visual object discovery through multi-modal dialogue. In CVPR. [find](https://scholar.google.com/scholar?q=Harm+de+Vries%2C+Florian+Strub%2C+Sarath+Chandar%2C+Olivier+Pietquin%2C+Hugo+Larochelle%2C+and+Aaron+Courville%2E+2017%2E+Guesswhat%3F%21+visual+object+discovery+through+multi%2Dmodal+dialogue%2E+In+CVPR%2E) 8. Gallup. 2023. Lgbt identification in the u.s. [https://news.gallup.com/poll/656708/lgbt](https://news.gallup.com/poll/656708/lgbt) q-identification-rises.aspx. [↗](https://news.gallup.com/poll/656708/lgbt) 9. Mao Xun Huang and Hen-Hsen Huang. 2024. Integrating text-to-image and vision language models for synergistic dataset generation. In IJCAI. [find](https://scholar.google.com/scholar?q=Mao+Xun+Huang+and+Hen%2DHsen+Huang%2E+2024%2E+Integrating+text%2Dto%2Dimage+and+vision+language+models+for+synergistic+dataset+generation%2E+In+IJCAI%2E) 10. Office for National Statistics. 2023. Sexual orientation, uk: annual estimates. [https://www.on](https://www.on) s.gov.uk/peoplepopulationandcommun ity/culturalidentity/sexuality/bul letins/sexualidentityuk/2023. [↗](https://www.on) 11. Kurt Shuster, Samuel Humeau, Antoine Bordes, and Jason Weston. 2018. Image chat: Engaging grounded conversations. In ACL. [find](https://scholar.google.com/scholar?q=Kurt+Shuster%2C+Samuel+Humeau%2C+Antoine+Bordes%2C+and+Jason+Weston%2E+2018%2E+Image+chat%3A+Engaging+grounded+conversations%2E+In+ACL%2E) 12. United Nations Department of Economic and Social Affairs. 2020. International migrant stock. [https://www.un.org/development/desa/pd/](https://www.un.org/development/desa/pd/). [↗](https://www.un.org/development/desa/pd/) 13. United Nations Statistics Division. 2024. Standard country or area codes for statistical use (m49). [https://unstats.un.org/unsd/method](https://unstats.un.org/unsd/method) ology/m49/. [↗](https://unstats.un.org/unsd/method) 14. Hiromi Wakaki, Yuki Mitsufuji, Yoshinori Maeda, Yukiko Nishimura, Silin Gao, Mengjie Zhao, Keiichi Yamada, and Antoine Bosselut. 2024. Comperdial: Commonsense persona-grounded dialogue dataset and benchmark. arXiv preprint arXiv:2406.11228. — [arXiv:2406.11228](https://arxiv.org/abs/2406.11228) [find](https://scholar.google.com/scholar?q=Hiromi+Wakaki%2C+Yuki+Mitsufuji%2C+Yoshinori+Maeda%2C+Yukiko+Nishimura%2C+Silin+Gao%2C+Mengjie+Zhao%2C+Keiichi+Yamada%2C+and+Antoine+Bosselut%2E+2024%2E+Comperdial%3A+Commonsense+persona%2Dgrounded+dialogue+dataset+and+ben) 15. Saizheng Zhang, Emily Dinan, Jack Urbanek, Arthur Szlam, Douwe Kiela, and Jason Weston. 2018. Personalizing dialogue agents: I have a dog, do you have pets too? arXiv preprint arXiv:1801.07243. 12. Language Resource References Awsaf49. 2018. PersonaChat (Hugging Face mirror) train_both_revised.txt.… [find](https://scholar.google.com/scholar?q=Saizheng+Zhang%2C+Emily+Dinan%2C+Jack+Urbanek%2C+Arthur+Szlam%2C+Douwe+Kiela%2C+and+Jason+Weston%2E+2018%2E+Personalizing+dialogue+agents%3A+I+have+a+dog%2C+do+you+have+pets+too%3F+arXiv+preprint+arXiv%3A1801%2E07243%2E+12%2E+La) [our card](https://md.research.elamur.ai/free/papers/zhang-2018-personalizing-dialogue-agents-i-have/)