---
title: "SIMMC 2.0 — paper by Kottur et al. (2021) — summary and ideas"
description: "The authors build scenes in Unity by hand-crafting seed stores and then… 0.192 BLEU. 35 references, cited by 57 papers here."
authors: ["Satwik Kottur", "Seungwhan Moon", "Alborz Geramifard", "Babak Damavandi"]
year: 2021
venue: "Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing"
source: "https://aclanthology.org/2021.emnlp-main.401/"
doi: "10.18653/v1/2021.emnlp-main.401"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/kottur-2021-simmc-2-0-task-oriented/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/kottur-2021-simmc-2-0-task-oriented/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2021](https://md.research.elamur.ai/free/en/papers/2021/page/4/) / SIMMC 2.0: A Task-oriented Dialog Dataset for Immersive Multimodal Conversations

# SIMMC 2.0: A Task-oriented Dialog Dataset for Immersive Multimodal Conversations

**Authors:** Satwik Kottur, [Seungwhan Moon](https://md.research.elamur.ai/free/en/authors/seungwhan-moon/), Alborz Geramifard and Babak Damavandi · **Year:** 2021 · **Venue:** [Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing](https://md.research.elamur.ai/free/en/venues/emnlp/) ([EMNLP 2021](https://md.research.elamur.ai/free/en/venues/emnlp/2021/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2021.emnlp-main.401/) · [PDF](https://aclanthology.org/2021.emnlp-main.401.pdf) · [DOI 10.18653/v1/2021.emnlp-main.401](https://doi.org/10.18653/v1/2021.emnlp-main.401) · [OpenAlex](https://openalex.org/W3156555225)

**Citations:** 57 citations (OpenAlex, 2026-10-14) · [57](https://md.research.elamur.ai/free/en/papers/kottur-2021-simmc-2-0-task-oriented/cited-by/) in our library, of the 141,840 papers whose bibliographies we read

## Summary
Task-oriented virtual assistants increasingly need to reason about what the user sees as well as what they say. The earlier SIMMC benchmarks relied on simplistic visual contexts. SIMMC 2.0 addresses this with a larger, more realistic resource for shopping conversations in the fashion and furniture domains, set in cluttered, photo-realistic VR store scenes.

The authors build scenes in Unity by hand-crafting seed stores and then programmatically swapping semantically compatible assets. They capture random camera viewpoints and automatically extract scene graphs and bounding boxes. An agenda-based multimodal simulator, consisting of a goal generator, a user simulator and an assistant simulator, produces annotated dialog flows. Crowd workers then rewrite these flows into natural language.

The resulting corpus supports four benchmarks:
- Multimodal Disambiguation
- MM-Coref
- MM-DST
- Response Generation

A multi-task GPT-2 baseline reaches only about 37% coreference F1, versus about 86% on SIMMC 1.0. This gap shows that visual grounding in realistic, cluttered scenes remains a hard open problem.

## Key points
- **Dataset size:** 11,244 dialogs, 117,236 utterances, 1,566 scene snapshots. Each dialog has about 10.4 utterances and 19.7 objects in the scene on average. Snapshots showing fewer than 5 objects were filtered out.
- **Two-phase collection:** simulated multimodal self-play first, then manual paraphrasing. This lowers annotation time and cost while keeping full ground-truth labels.
- **Data split:** train 65%, dev 5%, dev-test 15%, and a hidden test-std set of 15%.
- **Baselines:**
  - A GPT-2 model (12-layer, 117M parameters) that reads flattened scene metadata scores 73.9 disambiguation accuracy, 36.64 coref F1, DST F1 of 81.72 / 94.53 and 0.192 BLEU.
  - The image-based MTN model scores 0.217 BLEU.
- **Availability:** code and data are released at github.com/facebookresearch/simmc2.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Meta](https://md.research.elamur.ai/free/en/companies/meta/) — Company, Menlo Park, United States · 158 works · 211,459 citations

📚 Shelves (5): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 9,085 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 8,888 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 9,296 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 9,206 works · [👁️ Computer Vision and Pattern Recognition](https://md.research.elamur.ai/free/en/shelves/computer-science/computer-vision-and-pattern-recognition/) · 1,282 works

🧭 Topics (2): [🖼️ Vision & language](https://md.research.elamur.ai/free/en/topics/multimodal/) · 269 works · [🗨️ Dialogue systems](https://md.research.elamur.ai/free/en/topics/dialogue-systems/) · 401 works

## Cite

Cite as: Satwik Kottur, Seungwhan Moon, Alborz Geramifard and Babak Damavandi. 2021. SIMMC 2.0: A Task-oriented Dialog Dataset for Immersive Multimodal Conversations. Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{kottur2021simmc,
  title = {{SIMMC 2.0: A Task-oriented Dialog Dataset for Immersive Multimodal Conversations}},
  author = {Satwik Kottur, Seungwhan Moon, Alborz Geramifard and Babak Damavandi},
  booktitle = {Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing},
  year = {2021},
  doi = {10.18653/v1/2021.emnlp-main.401},
  url = {https://doi.org/10.18653/v1/2021.emnlp-main.401}
}
```

- [← previous Aspect-Controllable Opinion Summarization](https://md.research.elamur.ai/free/en/papers/amplayo-2021-aspect-controllable-opinion-summarization/)
- [next → Overview of the WANLP 2021 Shared Task on Sarcasm and Sentiment Detection in Arabic](https://md.research.elamur.ai/free/en/papers/farha-2021-overview-wanlp-2021-shared-task/)
- [↑ back to papers of 2021](https://md.research.elamur.ai/free/en/papers/2021/page/4/)

## References

35 references, 3 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Stanislaw Antol, Aishwarya Agrawal, Jiasen Lu, Margaret Mitchell, Dhruv Batra, C Lawrence Zitnick, and Devi Parikh. 2015. VQA: Visual question answering. In ICCV. [find](https://scholar.google.com/scholar?q=Stanislaw+Antol%2C+Aishwarya+Agrawal%2C+Jiasen+Lu%2C+Margaret+Mitchell%2C+Dhruv+Batra%2C+C+Lawrence+Zitnick%2C+and+Devi+Parikh%2E+2015%2E+VQA%3A+Visual+question+answering%2E+In+ICCV%2E)
2. Paweł Budzianowski, Tsung-Hsien Wen, Bo-Hsiang Tseng, Iñigo Casanueva, Stefan Ultes, Osman Ramadan, and Milica Gašić. 2018. MultiWOZ - a large-scale multi-domain wizard-of-Oz dataset for task-oriented dialogue modelling. In Proceedings of the Conference on Empirical Methods in Natural Language Proc… [find](https://scholar.google.com/scholar?q=Pawe%C5%82+Budzianowski%2C+Tsung%2DHsien+Wen%2C+Bo%2DHsiang+Tseng%2C+I%C3%B1igo+Casanueva%2C+Stefan+Ultes%2C+Osman+Ramadan%2C+and+Milica+Ga%C5%A1ic%CC%81%2E+2018%2E+MultiWOZ+%2D+a+large%2Dscale+multi%2Ddomain+wizard%2Dof%2DOz+dataset+for+task%2Doriente)
3. Guan-Lin Chao and Ian Lane. 2019. Bert-dst: Scalable end-to-end dialogue state tracking with bidirectional encoder representations from transformer. In Annual Conference of the International Speech Communication Association (INTERSPEECH). [find](https://scholar.google.com/scholar?q=Guan%2DLin+Chao+and+Ian+Lane%2E+2019%2E+Bert%2Ddst%3A+Scalable+end%2Dto%2Dend+dialogue+state+tracking+with+bidirectional+encoder+representations+from+transformer%2E+In+Annual+Conference+of+the+International+Speech+Co)
4. Paul A. Crook, Satwik Kottur, Seungwhan Moon, Ahmad Beirami, Eunjoon Cho, Rajen Subba, and Alborz Geramifard. 2021. Situated interactive multimodal conversations (simmc) track at dstc9. AAAI DSTC9 Workshop. [find](https://scholar.google.com/scholar?q=Paul+A%2E+Crook%2C+Satwik+Kottur%2C+Seungwhan+Moon%2C+Ahmad+Beirami%2C+Eunjoon+Cho%2C+Rajen+Subba%2C+and+Alborz+Geramifard%2E+2021%2E+Situated+interactive+multimodal+conversations+%28simmc%29+track+at+dstc9%2E+AAAI+DSTC9+Wor)
5. Abhishek Das, Satwik Kottur, Khushi Gupta, Avi Singh, Deshraj Yadav, José MF Moura, Devi Parikh, and Dhruv Batra. 2017. Visual dialog. In CVPR. [find](https://scholar.google.com/scholar?q=Abhishek+Das%2C+Satwik+Kottur%2C+Khushi+Gupta%2C+Avi+Singh%2C+Deshraj+Yadav%2C+Jos%C3%A9+MF+Moura%2C+Devi+Parikh%2C+and+Dhruv+Batra%2E+2017%2E+Visual+dialog%2E+In+CVPR%2E)
6. Mihail Eric, Rahul Goel, Shachi Paul, Adarsh Kumar, Abhishek Sethi, Peter Ku, Anuj Kumar Goyal, Sanchit Agarwal, Shuyag Gao, and Dilek Hakkani-Tur. 2019. Multiwoz 2.1: Multi-domain dialogue state corrections and state tracking baselines. arXiv preprint arXiv:1907.01669. [find](https://scholar.google.com/scholar?q=Mihail+Eric%2C+Rahul+Goel%2C+Shachi+Paul%2C+Adarsh+Kumar%2C+Abhishek+Sethi%2C+Peter+Ku%2C+Anuj+Kumar+Goyal%2C+Sanchit+Agarwal%2C+Shuyag+Gao%2C+and+Dilek+Hakkani%2DTur%2E+2019%2E+Multiwoz+2%2E1%3A+Multi%2Ddomain+dialogue+state+corr)
7. Matthew Fisher, Daniel Ritchie, Manolis Savva, Thomas Funkhouser, and Pat Hanrahan. 2012. Example-based synthesis of 3d object arrangements. In ACM SIGGRAPH Asia 2012 papers, SIGGRAPH Asia ’12. [find](https://scholar.google.com/scholar?q=Matthew+Fisher%2C+Daniel+Ritchie%2C+Manolis+Savva%2C+Thomas+Funkhouser%2C+and+Pat+Hanrahan%2E+2012%2E+Example%2Dbased+synthesis+of+3d+object+arrangements%2E+In+ACM+SIGGRAPH+Asia+2012+papers%2C+SIGGRAPH+Asia+%E2%80%9912%2E)
8. Matthew Fisher, Manolis Savva, Yangyan Li, Pat Hanrahan, and Matthias Nießner. 2015. Activity-centric scene synthesis for functional 3d scene modeling. ACM Transactions on Graphics (TOG), 34(6). [find](https://scholar.google.com/scholar?q=Matthew+Fisher%2C+Manolis+Savva%2C+Yangyan+Li%2C+Pat+Hanrahan%2C+and+Matthias+Nie%C3%9Fner%2E+2015%2E+Activity%2Dcentric+scene+synthesis+for+functional+3d+scene+modeling%2E+ACM+Transactions+on+Graphics+%28TOG%29%2C+34%286%29%2E)
9. Shuyang Gao, Sanchit Agarwal Abhishek Seth and, Tagyoung Chun, and Dilek Hakkani-Ture. 2019. [Dialog state tracking: A neural reading comprehension approach](https://md.research.elamur.ai/free/en/papers/gao-2019-dialog-state-tracking-neural-reading/). In Special Interest Group on Discourse and Dialogue (SIGDIAL).
10. Chiori Hori, Anoop Cherian, Tim K. Marks, and Florian Metze. 2018. Audio visual scene-aware dialog track in dstc8. DSTC Track Proposal. [find](https://scholar.google.com/scholar?q=Chiori+Hori%2C+Anoop+Cherian%2C+Tim+K%2E+Marks%2C+and+Florian+Metze%2E+2018%2E+Audio+visual+scene%2Daware+dialog+track+in+dstc8%2E+DSTC+Track+Proposal%2E)
11. Ehsan Hosseini-Asl, Bryan McCann, Chien-Sheng Wu, Semih Yavuz, and Richard Socher. 2020. A simple language model for task-oriented dialogue. arXiv preprint arXiv:2005.00796. [find](https://scholar.google.com/scholar?q=Ehsan+Hosseini%2DAsl%2C+Bryan+McCann%2C+Chien%2DSheng+Wu%2C+Semih+Yavuz%2C+and+Richard+Socher%2E+2020%2E+A+simple+language+model+for+task%2Doriented+dialogue%2E+arXiv+preprint+arXiv%3A2005%2E00796%2E)
12. Xin Huang, Chor Seng Tan, Yan Bin Ng, Wei Shi, Kheng Hui Yeo, Ridong Jiang, and Jung Jae Kim. 2021. Joint generation and bi-encoder for situated interactive multimodal conversations. [find](https://scholar.google.com/scholar?q=Xin+Huang%2C+Chor+Seng+Tan%2C+Yan+Bin+Ng%2C+Wei+Shi%2C+Kheng+Hui+Yeo%2C+Ridong+Jiang%2C+and+Jung+Jae+Kim%2E+2021%2E+Joint+generation+and+bi%2Dencoder+for+situated+interactive+multimodal+conversations%2E)
13. AAAI 2021 DSTC9 Workshop. [find](https://scholar.google.com/scholar?q=AAAI+2021+DSTC9+Workshop%2E)
14. Younghoon Jeong, Se Jin Lee, Youngjoong Ko, and Jungyun Seo. 2021. Tom : End-to-end task-oriented multimodal dialog system with gpt-2. [find](https://scholar.google.com/scholar?q=Younghoon+Jeong%2C+Se+Jin+Lee%2C+Youngjoong+Ko%2C+and+Jungyun+Seo%2E+2021%2E+Tom+%3A+End%2Dto%2Dend+task%2Doriented+multimodal+dialog+system+with+gpt%2D2%2E)
15. AAAI 2021 DSTC9 Workshop. [find](https://scholar.google.com/scholar?q=AAAI+2021+DSTC9+Workshop%2E)
16. Byoungjae Kim, Inkwon Lee, Yeonseok Jeong, Ko Youngjoong, Myoung-Wan Koo, and Jungyun Seo. 2021. Improving multimodal api prediction via adding dialog state and various multimodal gates. [find](https://scholar.google.com/scholar?q=Byoungjae+Kim%2C+Inkwon+Lee%2C+Yeonseok+Jeong%2C+Ko+Youngjoong%2C+Myoung%2DWan+Koo%2C+and+Jungyun+Seo%2E+2021%2E+Improving+multimodal+api+prediction+via+adding+dialog+state+and+various+multimodal+gates%2E)
17. AAAI 2021 DSTC9 Workshop. [find](https://scholar.google.com/scholar?q=AAAI+2021+DSTC9+Workshop%2E)
18. Satwik Kottur, José MF Moura, Devi Parikh, Dhruv Batra, and Marcus Rohrbach. 2019. Clevr-dialog: A diagnostic dataset for multi-round reasoning in visual dialog. arXiv preprint arXiv:1903.03166. [find](https://scholar.google.com/scholar?q=Satwik+Kottur%2C+Jos%C3%A9+MF+Moura%2C+Devi+Parikh%2C+Dhruv+Batra%2C+and+Marcus+Rohrbach%2E+2019%2E+Clevr%2Ddialog%3A+A+diagnostic+dataset+for+multi%2Dround+reasoning+in+visual+dialog%2E+arXiv+preprint+arXiv%3A1903%2E03166%2E)
19. Po-Nien Kung, Tse-Hsuan Yang, Chung-Cheng Chang, Hsin-Kai Hsu, Yu-Jia Liou, and Yun-Nung Chen. 2021. Multi-task learning for situated multi-domain end-to-end dialogue systems. [find](https://scholar.google.com/scholar?q=Po%2DNien+Kung%2C+Tse%2DHsuan+Yang%2C+Chung%2DCheng+Chang%2C+Hsin%2DKai+Hsu%2C+Yu%2DJia+Liou%2C+and+Yun%2DNung+Chen%2E+2021%2E+Multi%2Dtask+learning+for+situated+multi%2Ddomain+end%2Dto%2Dend+dialogue+systems%2E)
20. AAAI 2021 DSTC9 Workshop. [find](https://scholar.google.com/scholar?q=AAAI+2021+DSTC9+Workshop%2E)
21. Hung Le, Doyen Sahoo, Nancy Chen, and Steven Hoi. 2019. Multimodal transformer networks for end-to-end video-grounded dialogue systems. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 5612–5623. [find](https://scholar.google.com/scholar?q=Hung+Le%2C+Doyen+Sahoo%2C+Nancy+Chen%2C+and+Steven+Hoi%2E+2019%2E+Multimodal+transformer+networks+for+end%2Dto%2Dend+video%2Dgrounded+dialogue+systems%2E+In+Proceedings+of+the+57th+Annual+Meeting+of+the+Association+for)
22. Seungwhan Moon, Satwik Kottur, Paul A Crook, Ankita De, Shivani Poddar, Theodore Levin, David Whitney, Daniel Difranco, Ahmad Beirami, Eunjoon Cho, Rajen Subba, and Alborz Geramifard. 2020. Situated and interactive multimodal conversations. arXiv preprint arXiv:2006.01460. [find](https://scholar.google.com/scholar?q=Seungwhan+Moon%2C+Satwik+Kottur%2C+Paul+A+Crook%2C+Ankita+De%2C+Shivani+Poddar%2C+Theodore+Levin%2C+David+Whitney%2C+Daniel+Difranco%2C+Ahmad+Beirami%2C+Eunjoon+Cho%2C+Rajen+Subba%2C+and+Alborz+Geramifard%2E+2020%2E+Situated+a)
23. Chulaka Gunasekara, Seokhwan Kim, Luis Fernando D’Haro, Abhinav Rastogi, Yun-Nung Chen, Mihail Eric, Behnam Hedayatnia, Karthik Gopalakrishnan, Yang Liu, Chao-Wei Huang, et al. 2020. Overview of the ninth dialog system technology challenge: Dstc9. arXiv preprint arXiv:2011.06486. — [arXiv:2011.06486](https://arxiv.org/abs/2011.06486) [find](https://scholar.google.com/scholar?q=Chulaka+Gunasekara%2C+Seokhwan+Kim%2C+Luis+Fernando+D%E2%80%99Haro%2C+Abhinav+Rastogi%2C+Yun%2DNung+Chen%2C+Mihail+Eric%2C+Behnam+Hedayatnia%2C+Karthik+Gopalakrishnan%2C+Yang+Liu%2C+Chao%2DWei+Huang%2C+et+al%2E+2020%2E+Overview+of+the+n)
24. Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. [Bleu: a method for automatic evaluation of machine translation](https://md.research.elamur.ai/free/en/papers/papineni-2002-bleu-method-automatic-evaluation-machine/). In Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, pages 311–318, Philadelphia, Pennsylvania, USA. Association for Computational…
25. Matthew Henderson, Blaise Thomson, and Jason D Williams. 2014. The second dialog state tracking challenge. In Proceedings of the 15th annual meeting of the special interest group on discourse and dialogue (SIGDIAL), pages 263–272. [find](https://scholar.google.com/scholar?q=Matthew+Henderson%2C+Blaise+Thomson%2C+and+Jason+D+Williams%2E+2014%2E+The+second+dialog+state+tracking+challenge%2E+In+Proceedings+of+the+15th+annual+meeting+of+the+special+interest+group+on+discourse+and+dial)
26. Baolin Peng, Chunyuan Li, Jinchao Li, Shahin Shayandeh, Lars Liden, and Jianfeng Gao. 2020. Soloist: Few-shot task-oriented dialog with a single pre-trained auto-regressive model. arXiv preprint arXiv:2005.05298. [find](https://scholar.google.com/scholar?q=Baolin+Peng%2C+Chunyuan+Li%2C+Jinchao+Li%2C+Shahin+Shayandeh%2C+Lars+Liden%2C+and+Jianfeng+Gao%2E+2020%2E+Soloist%3A+Few%2Dshot+task%2Doriented+dialog+with+a+single+pre%2Dtrained+auto%2Dregressive+model%2E+arXiv+preprint+arXiv)
27. Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. 2019. Language models are unsupervised multitask learners. OpenAI blog, 1(8):9. [find](https://scholar.google.com/scholar?q=Alec+Radford%2C+Jeffrey+Wu%2C+Rewon+Child%2C+David+Luan%2C+Dario+Amodei%2C+and+Ilya+Sutskever%2E+2019%2E+Language+models+are+unsupervised+multitask+learners%2E+OpenAI+blog%2C+1%288%29%3A9%2E)
28. Abhinav Rastogi, Xiaoxue Zang, Srinivas Sunkara, Raghav Gupta, and Pranav Khaitan. 2019. Towards scalable multi-domain conversational agents: The schema-guided dialogue dataset. In Association for the Advancement of Artificial Intelligence (AAAI). [find](https://scholar.google.com/scholar?q=Abhinav+Rastogi%2C+Xiaoxue+Zang%2C+Srinivas+Sunkara%2C+Raghav+Gupta%2C+and+Pranav+Khaitan%2E+2019%2E+Towards+scalable+multi%2Ddomain+conversational+agents%3A+The+schema%2Dguided+dialogue+dataset%2E+In+Association+for+the)
29. Jost Schatzmann, Blaise Thomson, Karl Weilhammer, Hui Ye, and Steve Young. 2007. [Agenda-based user simulation for bootstrapping a pomdp dialogue system](https://md.research.elamur.ai/free/en/papers/schatzmann-2007-agenda-based-user-simulation-bootstrapping/). In Human Language Technologies 2007: The Conference of the North American Chapter of the Association for Computational Linguistics; Companion Volum…
30. Matteo Antonio Senese, Giuseppe Rizzo, Alberto Benincasa, and Barbara Caputo. 2021. A response retrieval approach for dialogue using a multi-attentive transformer. [find](https://scholar.google.com/scholar?q=Matteo+Antonio+Senese%2C+Giuseppe+Rizzo%2C+Alberto+Benincasa%2C+and+Barbara+Caputo%2E+2021%2E+A+response+retrieval+approach+for+dialogue+using+a+multi%2Dattentive+transformer%2E)
31. AAAI 2021 DSTC9 Workshop. [find](https://scholar.google.com/scholar?q=AAAI+2021+DSTC9+Workshop%2E)
32. Pararth Shah, Dilek Hakkani-Tür, Gokhan Tür, Abhinav Rastogi, Ankur Bapna, Neha Nayak, and Larry Heck. 2018. Building a conversational agent overnight with dialogue self-play. arXiv preprint arXiv:1801.04871. [find](https://scholar.google.com/scholar?q=Pararth+Shah%2C+Dilek+Hakkani%2DT%C3%BCr%2C+Gokhan+T%C3%BCr%2C+Abhinav+Rastogi%2C+Ankur+Bapna%2C+Neha+Nayak%2C+and+Larry+Heck%2E+2018%2E+Building+a+conversational+agent+overnight+with+dialogue+self%2Dplay%2E+arXiv+preprint+arXiv%3A180)
33. Unity Technologies. 2019. Unity. [https://unity.com/](https://unity.com/). [↗](https://unity.com/)
34. Harm de Vries, Kurt Shuster, Dhruv Batra, Devi Parikh, Jason Weston, and Douwe Kiela. 2018. Talk the walk: Navigating new york city through grounded dialogue. arXiv preprint arXiv:1807.03367. — [arXiv:1807.03367](https://arxiv.org/abs/1807.03367) [find](https://scholar.google.com/scholar?q=Harm+de+Vries%2C+Kurt+Shuster%2C+Dhruv+Batra%2C+Devi+Parikh%2C+Jason+Weston%2C+and+Douwe+Kiela%2E+2018%2E+Talk+the+walk%3A+Navigating+new+york+city+through+grounded+dialogue%2E+arXiv+preprint+arXiv%3A1807%2E03367%2E+%E2%80%94+%5BarXiv)
35. Harm de Vries, Florian Strub, Sarath Chandar, Olivier Pietquin, Hugo Larochelle, and Aaron Courville. 2017. Guesswhat?! visual object discovery through multi-modal dialogue. In CVPR. [find](https://scholar.google.com/scholar?q=Harm+de+Vries%2C+Florian+Strub%2C+Sarath+Chandar%2C+Olivier+Pietquin%2C+Hugo+Larochelle%2C+and+Aaron+Courville%2E+2017%2E+Guesswhat%3F%21+visual+object+discovery+through+multi%2Dmodal+dialogue%2E+In+CVPR%2E)

*Updated 24 October 2026*