---
title: "doc2dial — paper by Feng et al. (2020)"
description: "The authors build the dialogues in two stages. 30 references, cited by 69 papers here."
authors: ["Song Feng", "Hui Wan", "Chulaka Gunasekara", "Siva Patel", "Sachindra Joshi", "Luis Lastras"]
year: 2020
venue: "Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)"
source: "https://aclanthology.org/2020.emnlp-main.652/"
doi: "10.18653/v1/2020.emnlp-main.652"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/feng-2020-doc2dial-goal-oriented-document-grounded/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/feng-2020-doc2dial-goal-oriented-document-grounded/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2020](https://md.research.elamur.ai/free/en/papers/2020/page/4/) / doc2dial: A Goal-Oriented Document-Grounded Dialogue Dataset

# doc2dial: A Goal-Oriented Document-Grounded Dialogue Dataset

**Authors:** [Song Feng](https://md.research.elamur.ai/free/en/authors/song-feng/), Hui Wan, [Chulaka Gunasekara](https://md.research.elamur.ai/free/en/authors/chulaka-gunasekara/), Siva Patel, [Sachindra Joshi](https://md.research.elamur.ai/free/en/authors/sachindra-joshi/) and [Luis Lastras](https://md.research.elamur.ai/free/en/authors/luis-lastras/) · **Year:** 2020 · **Venue:** [Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)](https://md.research.elamur.ai/free/en/venues/emnlp/) ([EMNLP 2020](https://md.research.elamur.ai/free/en/venues/emnlp/2020/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2020.emnlp-main.652/) · [PDF](https://aclanthology.org/2020.emnlp-main.652.pdf) · [DOI 10.18653/v1/2020.emnlp-main.652](https://doi.org/10.18653/v1/2020.emnlp-main.652) · [OpenAlex](https://openalex.org/W3099590177)

**Citations:** 83 citations (OpenAlex, 2026-10-14) · [69](https://md.research.elamur.ai/free/en/papers/feng-2020-doc2dial-goal-oriented-document-grounded/cited-by/) in our library, of the 141,840 papers whose bibliographies we read

## Summary

Most existing conversational QA resources, such as QuAC, CoQA and ShARC, ground dialogues in short text snippets and often restrict answers, for example to yes/no. This limits how realistically they reflect information-seeking support. doc2dial addresses this with goal-oriented conversations between an agent and a user, each tied to full documents from US government service websites. The authors build the dialogues in two stages. First, a semi-automatic pipeline turns each document into a graph of text spans linked by relations derived from document structure (headers, list items) and from discourse connectives. Second, the pipeline samples dialogue flows from this graph, where each turn specifies a role, a dialogue act and a grounding span. Crowd workers then write natural utterances for every turn, so annotations come built in rather than added afterward. The resulting dialogues cover varied scenes: under-specified questions, agent verification questions, multi-turn conditions and off-topic queries. Baselines are reported for three tasks: user utterance understanding, agent response generation and relevant document identification. The dataset gives a testbed for models that must reason over long, structured documents during dialogue.

## Key points

- The corpus contains 4470 dialogues averaging about 14 turns, grounded in 458 documents from four domains: ssa.gov, va.gov, dmv.gov and cdc.gov. Documents are provided as plain text and HTML.
- Each turn is annotated with its role, its dialogue act and the document span it is grounded in.
- About 70 qualified contributors from appen.com wrote the utterances, each playing both roles for a whole dialogue. They were paid $1.5–$2 per conversation.
- For single-document retrieval against a pool of 594 documents, a BertForMultipleChoice classifier based on bert-base-uncased consistently beat BM25. Recall improved as more dialogue turns were included.
- A multi-document variant has 2051 conversations, split 1640/206/205 into train/dev/test. Baseline results on it leave considerable room for improvement.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [IBM](https://md.research.elamur.ai/free/en/companies/ibm/) — Company, Armonk, United States · 85 works · 66,118 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 9,086 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 8,889 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 9,296 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 9,206 works

🧭 Topics (1): [🗨️ Dialogue systems](https://md.research.elamur.ai/free/en/topics/dialogue-systems/) · 401 works

## Cite

Cite as: Song Feng, Hui Wan, Chulaka Gunasekara, Siva Patel, Sachindra Joshi and Luis Lastras. 2020. doc2dial: A Goal-Oriented Document-Grounded Dialogue Dataset. Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP).

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{feng2020doc2dial,
  title = {{doc2dial: A Goal-Oriented Document-Grounded Dialogue Dataset}},
  author = {Song Feng, Hui Wan, Chulaka Gunasekara, Siva Patel, Sachindra Joshi and Luis Lastras},
  booktitle = {Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)},
  year = {2020},
  doi = {10.18653/v1/2020.emnlp-main.652},
  url = {https://doi.org/10.18653/v1/2020.emnlp-main.652}
}
```

- [← previous Unsupervised Quality Estimation for Neural Machine Translation](https://md.research.elamur.ai/free/en/papers/fomicheva-2020-unsupervised-quality-estimation-neural-machine/)
- [next → Continual Relation Learning via Episodic Memory Activation and Reconsolidation](https://md.research.elamur.ai/free/en/papers/han-2020-continual-relation-learning-episodic-memory/)
- [↑ back to papers of 2020](https://md.research.elamur.ai/free/en/papers/2020/page/4/)

## References

30 references, 15 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Leif Azzopardi, Mateusz Dubiel, Martin Halvey, and Jeffery Dalton. 2018. Conceptualizing agent-human interactions during the conversational search process. In The Second International Workshop on Conversational Approaches to Information Retrieval. [find](https://scholar.google.com/scholar?q=Leif+Azzopardi%2C+Mateusz+Dubiel%2C+Martin+Halvey%2C+and+Jeffery+Dalton%2E+2018%2E+Conceptualizing+agent%2Dhuman+interactions+during+the+conversational+search+process%2E+In+The+Second+International+Workshop+on+Conv)
2. Paweł Budzianowski, Tsung-Hsien Wen, Bo-Hsiang Tseng, Iñigo Casanueva, Stefan Ultes, Osman Ramadan, and Milica Gasic. 2018. Multiwoz-a large-scale multi-domain wizard-of-oz dataset for task-oriented dialogue modelling. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language P… [find](https://scholar.google.com/scholar?q=Pawe%C5%82+Budzianowski%2C+Tsung%2DHsien+Wen%2C+Bo%2DHsiang+Tseng%2C+In%CC%83igo+Casanueva%2C+Stefan+Ultes%2C+Osman+Ramadan%2C+and+Milica+Gasic%2E+2018%2E+Multiwoz%2Da+large%2Dscale+multi%2Ddomain+wizard%2Dof%2Doz+dataset+for+task%2Doriented+)
3. Bill Byrne, Karthik Krishnamoorthi, Chinnadhurai Sankar, Arvind Neelakantan, Ben Goodrich, Daniel Duckworth, Semih Yavuz, Amit Dubey, Kyu-Young Kim, and Andy Cedilnik. 2019. Taskmaster-1: Toward a realistic and diverse dialog dataset. In Proceedings of the 2019 Conference on Empirical Methods in Nat… [our card](https://md.research.elamur.ai/free/en/papers/byrne-2019-taskmaster-1-realistic-diverse-dialog/)
4. Jon Ander Campos, Arantxa Otegi, Aitor Soroa, Jan Deriu, Mark Cieliebak, and Eneko Agirre. 2020. DoQA - accessing domain-specific FAQs via conversational QA. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 7302–7314, Online. Association for Computati… [find](https://scholar.google.com/scholar?q=Jon+Ander+Campos%2C+Arantxa+Otegi%2C+Aitor+Soroa%2C+Jan+Deriu%2C+Mark+Cieliebak%2C+and+Eneko+Agirre%2E+2020%2E+DoQA+%2D+accessing+domain%2Dspecific+FAQs+via+conversational+QA%2E+In+Proceedings+of+the+58th+Annual+Meeting+)
5. Eunsol Choi, He He, Mohit Iyyer, Mark Yatskar, Wen-tau Yih, Yejin Choi, Percy Liang, and Luke Zettlemoyer. 2018. QuAC: Question answering in context. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 2174–2184, Brussels, Belgium. Association for Computa… [our card](https://md.research.elamur.ai/free/en/papers/choi-2018-quac-question-answering-context/)
6. Debopam Das, Tatjana Scheffler, Peter Bourgonje, and Manfred Stede. 2018. Constructing a lexicon of english discourse connectives. In Proceedings of the 19th Annual SIGdial Meeting on Discourse and Dialogue, pages 360–365. [our card](https://md.research.elamur.ai/free/en/papers/das-2018-constructing-lexicon-english-discourse-connectives/)
7. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies,… [our card](https://md.research.elamur.ai/free/en/papers/devlin-2019-bert-pre-training-deep-bidirectional/)
8. Song Feng, Kshitij Fadnis, Q Vera Liao, and Luis A Lastras. 2020. Doc2dial: a framework for dialogue composition grounded in documents. In Thirty-Fourth AAAI Conference on Artificial Intelligence. [find](https://scholar.google.com/scholar?q=Song+Feng%2C+Kshitij+Fadnis%2C+Q+Vera+Liao%2C+and+Luis+A+Lastras%2E+2020%2E+Doc2dial%3A+a+framework+for+dialogue+composition+grounded+in+documents%2E+In+Thirty%2DFourth+AAAI+Conference+on+Artificial+Intelligence%2E)
9. Jeroen Geertzen and Harry Bunt. 2009. Measuring annotator agreement in a complex hierarchical dialogue act annotation scheme. In Proceedings of the 7th SIGdial Workshop on Discourse and Dialogue, pages 126–133. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Jeroen+Geertzen+and+Harry+Bunt%2E+2009%2E+Measuring+annotator+agreement+in+a+complex+hierarchical+dialogue+act+annotation+scheme%2E+In+Proceedings+of+the+7th+SIGdial+Workshop+on+Discourse+and+Dialogue%2C+page)
10. Mor Geva and Jonathan Berant. 2018. Learning to search in long documents using document structure. In Proceedings of the 27th International Conference on Computational Linguistics, pages 161–176, Santa Fe, New Mexico, USA. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Mor+Geva+and+Jonathan+Berant%2E+2018%2E+Learning+to+search+in+long+documents+using+document+structure%2E+In+Proceedings+of+the+27th+International+Conference+on+Computational+Linguistics%2C+pages+161%E2%80%93176%2C+Sant)
11. Vidur Joshi, Matthew Peters, and Mark Hopkins. 2018. Extending a parser to distant domains using a few dozen partially annotated examples. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1190–1199, Melbourne, Australia. Associ… [our card](https://md.research.elamur.ai/free/en/papers/joshi-2018-extending-parser-distant-domains-few/)
12. Seokhwan Kim, Mihail Eric, Karthik Gopalakrishnan, Behnam Hedayatnia, Yang Liu, and Dilek Hakkani-Tur. 2020. Beyond domain apis: Task-oriented conversational modeling with unstructured knowledge access. arXiv preprint arXiv:2006.03533. [find](https://scholar.google.com/scholar?q=Seokhwan+Kim%2C+Mihail+Eric%2C+Karthik+Gopalakrishnan%2C+Behnam+Hedayatnia%2C+Yang+Liu%2C+and+Dilek+Hakkani%2DTur%2E+2020%2E+Beyond+domain+apis%3A+Task%2Doriented+conversational+modeling+with+unstructured+knowledge+acces)
13. Tina Klüwer. 2011. “i like your shirt”-dialogue acts for enabling social talk in conversational agents. In International Workshop on Intelligent Virtual Agents, pages 14–27. Springer. [find](https://scholar.google.com/scholar?q=Tina+Klu%CC%88wer%2E+2011%2E+%E2%80%9Ci+like+your+shirt%E2%80%9D%2Ddialogue+acts+for+enabling+social+talk+in+conversational+agents%2E+In+International+Workshop+on+Intelligent+Virtual+Agents%2C+pages+14%E2%80%9327%2E+Springer%2E)
14. Yang Liu, Kun Han, Zhao Tan, and Yun Lei. 2017. Using context information for dialog act classification in dnn framework. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 2170–2178. [our card](https://md.research.elamur.ai/free/en/papers/liu-2017-context-information-dialog-act-classification/)
15. Saikat Mukherjee, Guizhen Yang, Wenfang Tan, and IV Ramakrishnan. 2003. Automatic discovery of semantic structures in html documents. In Seventh International Conference on Document Analysis and Recognition, 2003. Proceedings., pages 245–249. IEEE. [find](https://scholar.google.com/scholar?q=Saikat+Mukherjee%2C+Guizhen+Yang%2C+Wenfang+Tan%2C+and+IV+Ramakrishnan%2E+2003%2E+Automatic+discovery+of+semantic+structures+in+html+documents%2E+In+Seventh+International+Conference+on+Document+Analysis+and+Recog)
16. Silvia Pareti and Tatiana Lando. 2018. Dialog intent structure: A hierarchical schema of linked dialog acts. In Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC-2018). [our card](https://md.research.elamur.ai/free/en/papers/pareti-2018-dialog-intent-structure-hierarchical-schema/)
17. Gustavo Penha, Alexandru Balan, and Claudia Hauff. 2019. Introducing mantis: a novel multi-domain information seeking dialogues dataset. arXiv preprint arXiv:1912.04639. — [arXiv:1912.04639](https://arxiv.org/abs/1912.04639) [find](https://scholar.google.com/scholar?q=Gustavo+Penha%2C+Alexandru+Balan%2C+and+Claudia+Hauff%2E+2019%2E+Introducing+mantis%3A+a+novel+multi%2Ddomain+information+seeking+dialogues+dataset%2E+arXiv+preprint+arXiv%3A1912%2E04639%2E+%E2%80%94+%5BarXiv%3A1912%2E04639%5D%28https%3A%2F%2Fa)
18. Rashmi Prasad, Nikhil Dinesh, Alan Lee, Eleni Miltsakaki, Livio Robaldo, Aravind Joshi, and Bonnie Webber. 2008. The Penn discourse TreeBank 2.0. In Proceedings of the Sixth International Conference on Language Resources and Evaluation (LREC’08), Marrakech, Morocco. European Language Resources Assoc… [our card](https://md.research.elamur.ai/free/en/papers/prasad-2008-penn-discourse-treebank-2-0/)
19. Rashmi Prasad, Bonnie Webber, Alan Lee, and Aravind Joshi. 2019. Penn Discourse Treebank Version 3.0. In LDC2019T05. Philadelphia: Linguistic Data Consortium. [find](https://scholar.google.com/scholar?q=Rashmi+Prasad%2C+Bonnie+Webber%2C+Alan+Lee%2C+and+Aravind+Joshi%2E+2019%2E+Penn+Discourse+Treebank+Version+3%2E0%2E+In+LDC2019T05%2E+Philadelphia%3A+Linguistic+Data+Consortium%2E)
20. Pranav Rajpurkar, Robin Jia, and Percy Liang. 2018. Know what you don’t know: Unanswerable questions for SQuAD. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 784– 789, Melbourne, Australia. Association for Computational Lin… [our card](https://md.research.elamur.ai/free/en/papers/rajpurkar-2018-know-what-you-don-t/)
21. Pranav Rajpurkar, Jian Zhang, Konstantin Lopyrev, and Percy Liang. 2016. SQuAD: 100,000+ questions for machine comprehension of text. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 2383–2392, Austin, Texas. Association for Computational Linguistics. [our card](https://md.research.elamur.ai/free/en/papers/rajpurkar-2016-squad-100-000-questions-machine/)
22. Siva Reddy, Danqi Chen, and Christopher D. Manning. 2019. CoQA: A conversational question answering challenge. Transactions of the Association for Computational Linguistics, 7:249–266. [our card](https://md.research.elamur.ai/free/en/papers/reddy-2019-coqa-conversational-question-answering-challenge/)
23. Stephen Robertson and Hugo Zaragoza. 2009. The probabilistic relevance framework: Bm25 and beyond. Found. Trends Inf. Retr., 3(4):333–389. [find](https://scholar.google.com/scholar?q=Stephen+Robertson+and+Hugo+Zaragoza%2E+2009%2E+The+probabilistic+relevance+framework%3A+Bm25+and+beyond%2E+Found%2E+Trends+Inf%2E+Retr%2E%2C+3%284%29%3A333%E2%80%93389%2E)
24. Marzieh Saeidi, Max Bartolo, Patrick Lewis, Sameer Singh, Tim Rocktäschel, Mike Sheldon, Guillaume Bouchard, and Sebastian Riedel. 2018. Interpretation of natural language rules in conversational machine reading. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Process… [our card](https://md.research.elamur.ai/free/en/papers/saeidi-2018-interpretation-natural-language-rules-conversational/)
25. M Stede, T Scheffler, and A Mendes. 2019. Connective-lex: A web-based multilingual lexical resource for connectives. discours. Revue de linguistique, psycholinguistique et informatique. A journal of linguistics, psycholinguistics and computational linguistics,(24). [find](https://scholar.google.com/scholar?q=M+Stede%2C+T+Scheffler%2C+and+A+Mendes%2E+2019%2E+Connective%2Dlex%3A+A+web%2Dbased+multilingual+lexical+resource+for+connectives%2E+discours%2E+Revue+de+linguistique%2C+psycholinguistique+et+informatique%2E+A+journal+of+l)
26. Quan Hung Tran, Gholamreza Haffari, and Ingrid Zukerman. 2017. A generative attentional neural network model for dialogue act classification. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 524–529. [our card](https://md.research.elamur.ai/free/en/papers/tran-2017-generative-attentional-neural-network-model/)
27. Alex Wang, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel Bowman. 2018. Glue: A multi-task benchmark and analysis platform for natural language understanding. In Proceedings of the 2018 EMNLP Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP, pages 353–355. [our card](https://md.research.elamur.ai/free/en/papers/wang-2018-glue-multi-task-benchmark-analysis/)
28. Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pier-ric Cistac, Tim Rault, R’emi Louf, Morgan Funtowicz, and Jamie Brew. 2019. Huggingface’s transformers: State-of-the-art natural language processing. ArXiv, abs/1910.03771. [our card](https://md.research.elamur.ai/free/en/papers/wolf-2020-transformers-state-art-natural-language/)
29. Rowan Zellers, Yonatan Bisk, Roy Schwartz, and Yejin Choi. 2018. Swag: A large-scale adversarial dataset for grounded commonsense inference. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 93–104. [find](https://scholar.google.com/scholar?q=Rowan+Zellers%2C+Yonatan+Bisk%2C+Roy+Schwartz%2C+and+Yejin+Choi%2E+2018%2E+Swag%3A+A+large%2Dscale+adversarial+dataset+for+grounded+commonsense+inference%2E+In+Proceedings+of+the+2018+Conference+on+Empirical+Methods+)
30. Victor Zhong and Luke Zettlemoyer. 2019. E3: Entailment-driven extracting and editing for conversational machine reading. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 2310–2320, Florence, Italy. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Victor+Zhong+and+Luke+Zettlemoyer%2E+2019%2E+E3%3A+Entailment%2Ddriven+extracting+and+editing+for+conversational+machine+reading%2E+In+Proceedings+of+the+57th+Annual+Meeting+of+the+Association+for+Computational)

*Updated 24 October 2026*