--- title: "HotpotQA: A Dataset for Diverse, Explainable — Yang 2018" description: "Earlier QA benchmarks such as SQuAD, TriviaQA and SearchQA can mostly be solved by matching a question to one or a few nearby sentences." authors: ["Zhilin Yang", "Peng Qi", "Saizheng Zhang", "Yoshua Bengio", "William Cohen", "Ruslan Salakhutdinov", "Christopher D. Manning"] year: 2018 venue: "Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing" source: "https://aclanthology.org/D18-1259/" doi: "10.18653/v1/d18-1259" arxiv: "1809.09600" licence: "CC BY 4.0 (ACL Anthology)" licence_url: "https://creativecommons.org/licenses/by/4.0/" summary_licence: "Elamur Cosmic License" canonical: "https://research.elamur.ai/free/en/papers/yang-2018-hotpotqa-dataset-diverse-explainable-multi/" --- [Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/yang-2018-hotpotqa-dataset-diverse-explainable-multi/) [home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2018](https://md.research.elamur.ai/free/en/papers/2018/page/2/) / HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering # HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering **Authors:** [Zhilin Yang](https://md.research.elamur.ai/free/en/authors/zhilin-yang/), [Peng Qi](https://md.research.elamur.ai/free/en/authors/peng-qi/), [Saizheng Zhang](https://md.research.elamur.ai/free/en/authors/saizheng-zhang/), [Yoshua Bengio](https://md.research.elamur.ai/free/en/authors/yoshua-bengio/), [William Cohen](https://md.research.elamur.ai/free/en/authors/william-cohen/), [Ruslan Salakhutdinov](https://md.research.elamur.ai/free/en/authors/ruslan-salakhutdinov/), [Christopher D. Manning](https://md.research.elamur.ai/free/en/authors/christopher-d-manning/) · **Year:** 2018 · **Venue:** Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/) **Links:** [Publisher's page](https://aclanthology.org/D18-1259/) · [PDF](https://aclanthology.org/D18-1259.pdf) · [DOI 10.18653/v1/d18-1259](https://doi.org/10.18653/v1/d18-1259) · [arXiv 1809.09600](https://arxiv.org/abs/1809.09600) · [OpenAlex](https://openalex.org/W2889787757) **Citations:** 1735 citations (openalex-snapshot · 2026-10-14) · [1754](#cited-by) in our library, of the 140884 papers whose bibliographies we read ## Summary Earlier QA benchmarks such as SQuAD, TriviaQA and SearchQA can mostly be solved by matching a question to one or a few nearby sentences. Multi-hop sets such as QAngaroo and COMPLEXWEBQUESTIONS are limited by the schemas of the knowledge bases used to build them. None of these datasets show a model which evidence supports an answer. HotpotQA fills these gaps with about 113k crowdsourced question-answer pairs based on English Wikipedia. The authors build a graph from the hyperlinks in each article's opening paragraph. They then pair linked paragraphs through hand-picked "bridge" entities. They also draw pairs of entities from curated lists to get comparison questions, some of them yes/no. Mechanical Turk workers write the questions and also mark the sentences their answers depend on. A baseline model trained with these sentence labels gains about 2 F1. With distractor paragraphs added, it still trails humans by roughly 30 points on answer EM and F1. HotpotQA therefore offers a hard, interpretable benchmark for reasoning across documents and for checking whether a model can justify its answers. ## Key points - The dataset has 112,779 examples, collected on Amazon Mechanical Turk through the ParlAI interface. The splits are train-easy (18,089, single-hop), train-medium (56,814), train-hard (15,661), and dev, test-distractor and test-fullwiki (7,405 each). - There are two question types. Bridge-entity questions (about 75% of collection) come from edges in the Wikipedia hyperlink graph. Comparison questions come from 42 manually curated entity lists, and half of them are yes/no questions. - Every example has sentence-level supporting facts. These give strong supervision during training and let evaluators score explainability by comparing predicted evidence to gold evidence. - On 1,000 sampled examples in the distractor setting, the baseline reached 68.99 answer F1 and 52.37 joint F1. Humans reached 91.40 and 82.55, and the human upper bound for answer F1 is 98.77. - Using the gold supporting facts as the only context adds 10+ F1. This suggests the paper's way of using the supervision (about 2 F1) is far from optimal. Training on all three train splits together gave the best results. *Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.* Fields: [Natural language processing](https://md.research.elamur.ai/free/en/shelves/nlp/) ## Cited by Cited by 1754 papers of our library cite it (of the 140884 whose bibliographies we read); 144 of them have a page here, the first 100 below, 1610 not yet. - [Explanations for CommonsenseQA: New Dataset and Models](https://md.research.elamur.ai/free/en/papers/aggarwal-2021-explanations-commonsenseqa-new-dataset-models/) - [ETC: Encoding Long and Structured Inputs in Transformers](https://md.research.elamur.ai/free/en/papers/ainslie-2020-etc-encoding-long-structured-inputs/) - [Aya 23: Open Weight Releases to Further Multilingual Progress](https://md.research.elamur.ai/free/en/papers/aryabumi-2024-aya-23-open-weight-releases/) - [Learning to Retrieve Reasoning Paths over Wikipedia Graph for Question Answering](https://md.research.elamur.ai/free/en/papers/asai-2019-learning-retrieve-reasoning-paths-over/) - [Evidentiality-guided Generation for Knowledge-Intensive NLP Tasks](https://md.research.elamur.ai/free/en/papers/asai-2022-evidentiality-guided-generation-knowledge-intensive/) - [Task-aware Retrieval with Instructions](https://md.research.elamur.ai/free/en/papers/asai-2023-task-aware-retrieval-instructions/) - [LongBench: A Bilingual, Multitask Benchmark for Long Context Understanding](https://md.research.elamur.ai/free/en/papers/bai-2023-longbench-bilingual-multitask-benchmark/) - [A Multitask, Multilingual, Multimodal Evaluation of ChatGPT on Reasoning, Hallucination, and Interactivity](https://md.research.elamur.ai/free/en/papers/bang-2023-multitask-multilingual-multimodal-evaluation-chatgpt/) - [Revisiting Joint Modeling of Cross-document Entity and Event Coreference Resolution](https://md.research.elamur.ai/free/en/papers/barhom-2019-revisiting-joint-modeling-cross-document/) - [LLM2Vec: Large Language Models Are Secretly Powerful Text Encoders](https://md.research.elamur.ai/free/en/papers/behnamghader-2024-llm2vec-large-language-models-secretly/) - [PyramidKV: Dynamic KV Cache Compression based on Pyramidal Information Funneling](https://md.research.elamur.ai/free/en/papers/cai-2024-pyramidkv-dynamic-kv-cache-compression/) - [HybridQA: A Dataset of Multi-Hop Question Answering over Tabular and Textual Data](https://md.research.elamur.ai/free/en/papers/chen-2020-hybridqa-dataset-multi-hop-question/) - [FinQA: A Dataset of Numerical Reasoning over Financial Data](https://md.research.elamur.ai/free/en/papers/chen-2021-finqa-dataset-numerical-reasoning-over/) - [M3-Embedding: Multi-Linguality, Multi-Functionality, Multi-Granularity Text Embeddings Through Self-Knowledge Distillation](https://md.research.elamur.ai/free/en/papers/chen-2024-m3-embedding-multi-linguality/) - [BoolQ: Exploring the Surprising Difficulty of Natural Yes/No Questions](https://md.research.elamur.ai/free/en/papers/clark-2019-boolq-exploring-surprising-difficulty-natural/) - [Transformers as Soft Reasoners over Language](https://md.research.elamur.ai/free/en/papers/clark-2020-transformers-soft-reasoners-over-language/) - [Promptagator: Few-shot Dense Retrieval From 8 Examples](https://md.research.elamur.ai/free/en/papers/dai-2022-promptagator-few-shot-dense-retrieval/) - [Explaining Answers with Entailment Trees](https://md.research.elamur.ai/free/en/papers/dalvi-2021-explaining-answers-entailment-trees/) - [A Dataset of Information-Seeking Questions and Answers Anchored in Research Papers](https://md.research.elamur.ai/free/en/papers/dasigi-2021-dataset-information-seeking-questions-answers/) - [Few-NERD: A Few-shot Named Entity Recognition Dataset](https://md.research.elamur.ai/free/en/papers/ding-2021-few-nerd-few-shot-named/) - [A Rationale-centric Counterfactual Data Augmentation Method for Cross-Document Event Coreference Resolution](https://md.research.elamur.ai/free/en/papers/ding-2024-rationale-centric-counterfactual-data-augmentation/) - [DROP: A Reading Comprehension Benchmark Requiring Discrete Reasoning Over Paragraphs](https://md.research.elamur.ai/free/en/papers/dua-2019-drop-reading-comprehension-benchmark-requiring/) - [English Machine Reading Comprehension Datasets: A Survey](https://md.research.elamur.ai/free/en/papers/dzendzik-2021-english-machine-reading-comprehension-datasets/) - [Olmo 3](https://md.research.elamur.ai/free/en/papers/ettinger-2025-olmo-3/) - [ELI5: Long Form Question Answering](https://md.research.elamur.ai/free/en/papers/fan-2019-eli5-long-form-question-answering/) - [Retrieval-Augmented Generation for Large Language Models: A Survey](https://md.research.elamur.ai/free/en/papers/gao-2023-retrieval-augmented-generation-large-language/) - [CRITIC: Large Language Models Can Self-Correct with Tool-Interactive Critiquing](https://md.research.elamur.ai/free/en/papers/gou-2023-critic-large-language-models-self/) - [Evaluating Large Language Models: A Comprehensive Survey](https://md.research.elamur.ai/free/en/papers/guo-2023-evaluating-large-language-models-comprehensive/) - [HippoRAG: Neurobiologically Inspired Long-Term Memory for Large Language Models](https://md.research.elamur.ai/free/en/papers/gutierrez-2024-hipporag-neurobiologically-inspired-long-term/) - [FOLIO: Natural Language Reasoning with First-Order Logic](https://md.research.elamur.ai/free/en/papers/han-2022-folio-natural-language-reasoning-first/) - [Constructing A Multi-hop QA Dataset for Comprehensive Evaluation of Reasoning Steps](https://md.research.elamur.ai/free/en/papers/ho-2020-constructing-multi-hop-qa-dataset/) - [RULER: What's the Real Context Size of Your Long-Context Language Models?](https://md.research.elamur.ai/free/en/papers/hsieh-2024-ruler-s-real-context-size/) - [Large Language Models Cannot Self-Correct Reasoning Yet](https://md.research.elamur.ai/free/en/papers/huang-2023-large-language-models-cannot-self/) - [A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questions](https://md.research.elamur.ai/free/en/papers/huang-2023-survey-hallucination-large-language-models/) - [TrustLLM: Trustworthiness in Large Language Models](https://md.research.elamur.ai/free/en/papers/huang-2024-trustllm-trustworthiness-large-language-models/) - [Adaptive-RAG: Learning to Adapt Retrieval-Augmented Large Language Models through Question Complexity](https://md.research.elamur.ai/free/en/papers/jeong-2024-adaptive-rag-learning-adapt-retrieval/) - [Survey of Hallucination in Natural Language Generation](https://md.research.elamur.ai/free/en/papers/ji-2022-survey-hallucination-natural-language-generation/) - [PubMedQA: A Dataset for Biomedical Research Question Answering](https://md.research.elamur.ai/free/en/papers/jin-2019-pubmedqa-dataset-biomedical-research-question/) - [BIDER: Bridging Knowledge Inconsistency for Efficient Retrieval-Augmented LLMs via Key Supporting Evidence](https://md.research.elamur.ai/free/en/papers/jin-2024-bider-bridging-knowledge-inconsistency-efficient/) - [Search-R1: Training LLMs to Reason and Leverage Search Engines with Reinforcement Learning](https://md.research.elamur.ai/free/en/papers/jin-2025-search-r1-training-llms-reason/) - [SpanBERT: Improving Pre-training by Representing and Predicting Spans](https://md.research.elamur.ai/free/en/papers/joshi-2019-spanbert-improving-pre-training-representing/) - [Evaluating Open-Domain Question Answering in the Era of Large Language Models](https://md.research.elamur.ai/free/en/papers/kamalloo-2023-evaluating-open-domain-question-answering/) - [Are Pretrained Language Models Symbolic Reasoners over Knowledge?](https://md.research.elamur.ai/free/en/papers/kassner-2020-are-pretrained-language-models-symbolic/) - [Bridging the Preference Gap between Retrievers and LLMs](https://md.research.elamur.ai/free/en/papers/ke-2024-bridging-preference-gap-between-retrievers/) - [CTRL: A Conditional Transformer Language Model for Controllable Generation](https://md.research.elamur.ai/free/en/papers/keskar-2019-ctrl-conditional-transformer-language-model/) - [Demonstrate-Search-Predict: Composing retrieval and language models for knowledge-intensive NLP](https://md.research.elamur.ai/free/en/papers/khattab-2022-demonstrate-search-predict-composing-retrieval/) - [DSPy: Compiling Declarative Language Model Calls into Self-Improving Pipelines](https://md.research.elamur.ai/free/en/papers/khattab-2023-dspy-compiling-declarative-language-model/) - [JGLUE: Japanese General Language Understanding Evaluation](https://md.research.elamur.ai/free/en/papers/kurihara-2022-jglue-japanese-general-language-understanding/) - [Natural Questions: A Benchmark for Question Answering Research](https://md.research.elamur.ai/free/en/papers/kwiatkowski-2019-natural-questions-benchmark-question-answering/) - [Internet-augmented language models through few-shot prompting for open-domain question answering](https://md.research.elamur.ai/free/en/papers/lazaridou-2022-internet-augmented-language-models-through/) - [Robustifying Multi-hop QA through Pseudo-Evidentiality Training](https://md.research.elamur.ai/free/en/papers/lee-2021-robustifying-multi-hop-qa-through/) - [NV-Embed: Improved Techniques for Training LLMs as Generalist Embedding Models](https://md.research.elamur.ai/free/en/papers/lee-2024-nv-embed-improved-techniques-training/) - [HaluEval: A Large-Scale Hallucination Evaluation Benchmark for Large Language Models](https://md.research.elamur.ai/free/en/papers/li-2023-halueval-large-scale-hallucination-evaluation/) - [LooGLE: Can Long-Context Language Models Understand Long Contexts?](https://md.research.elamur.ai/free/en/papers/li-2023-loogle-long-context-language-models/) - [Making Language Models Better Reasoners with Step-Aware Verifier](https://md.research.elamur.ai/free/en/papers/li-2023-making-language-models-better-reasoners/) - [The Dawn After the Dark: An Empirical Study on Factuality Hallucination in Large Language Models](https://md.research.elamur.ai/free/en/papers/li-2024-dawn-after-dark-empirical-study/) - [SnapKV: LLM Knows What You are Looking for Before Generation](https://md.research.elamur.ai/free/en/papers/li-2024-snapkv-llm-knows-looking-before/) - [Search-o1: Agentic Search-Enhanced Large Reasoning Models](https://md.research.elamur.ai/free/en/papers/li-2025-search-o1-agentic-search-enhanced/) - [Reasoning Over Paragraph Effects in Situations](https://md.research.elamur.ai/free/en/papers/lin-2019-reasoning-over-paragraph-effects-situations/) - [RA-DIT: Retrieval-Augmented Dual Instruction Tuning](https://md.research.elamur.ai/free/en/papers/lin-2023-ra-dit-retrieval-augmented-dual/) - [LogiQA: A Challenge Dataset for Machine Reading Comprehension with Logical Reasoning](https://md.research.elamur.ai/free/en/papers/liu-2020-logiqa-challenge-dataset-machine-reading/) - [Towards Faithful Model Explanation in NLP: A Survey](https://md.research.elamur.ai/free/en/papers/lyu-2024-faithful-model-explanation-nlp-survey/) - [Query Rewriting in Retrieval-Augmented Large Language Models](https://md.research.elamur.ai/free/en/papers/ma-2023-query-rewriting-retrieval-augmented-large/) - [Hallucination-Free? Assessing the Reliability of Leading AI Legal Research Tools](https://md.research.elamur.ai/free/en/papers/magesh-2024-hallucination-free-assessing-reliability-leading/) - [Augmenting LLM Reasoning with Dynamic Notes Writing for Complex MultiHop QA](https://md.research.elamur.ai/free/en/papers/maheshwary-2026-augmenting-llm-reasoning-dynamic-notes/) - [Quantifying the Accuracy and Cost Impact of Design Decisions in Budget-Constrained Agentic LLM Search](https://md.research.elamur.ai/free/en/papers/mccleary-2026-quantifying-accuracy-cost-impact-design/) - [Compositional Questions Do Not Necessitate Multi-hop Reasoning](https://md.research.elamur.ai/free/en/papers/min-2019-compositional-questions-do-not-necessitate/) - [A Discrete Hard EM Approach for Weakly Supervised Question Answering](https://md.research.elamur.ai/free/en/papers/min-2019-discrete-hard-em-approach-weakly/) - [Multi-hop Reading Comprehension through Question Decomposition and Rescoring](https://md.research.elamur.ai/free/en/papers/min-2019-multi-hop-reading-comprehension-through/) - [Large Language Models: A Survey](https://md.research.elamur.ai/free/en/papers/minaee-2024-large-language-models-survey/) - [SPARTQA: A Textual Question Answering Benchmark for Spatial Reasoning](https://md.research.elamur.ai/free/en/papers/mirzaee-2021-spartqa-textual-question-answering-benchmark/) - [SGPT: GPT Sentence Embeddings for Semantic Search](https://md.research.elamur.ai/free/en/papers/muennighoff-2022-sgpt-gpt-sentence-embeddings-semantic/) - [Generative Representational Instruction Tuning](https://md.research.elamur.ai/free/en/papers/muennighoff-2024-generative-representational-instruction-tuning/) - [Sentiment Analysis and Language Models for Kwanyama](https://md.research.elamur.ai/free/en/papers/nakashole-2026-sentiment-analysis-language-models-kwanyama/) - [AccurateRAG: A Framework for Building Accurate Retrieval-Augmented Question-Answering Applications](https://md.research.elamur.ai/free/en/papers/nguyen-2026-accuraterag-framework-building-accurate-retrieval/) - [Nomic Embed: Training a Reproducible Long Context Text Embedder](https://md.research.elamur.ai/free/en/papers/nussbaum-2024-nomic-embed-training-reproducible-long/) - [MedMCQA : A Large-scale Multi-Subject Multi-Choice Dataset for Medical domain Question Answering](https://md.research.elamur.ai/free/en/papers/pal-2022-medmcqa-large-scale-multi-subject/) - [Fact-Checking Complex Claims with Program-Guided Reasoning](https://md.research.elamur.ai/free/en/papers/pan-2023-fact-checking-complex-claims-program/) - [QuALITY: Question Answering with Long Input Texts, Yes!](https://md.research.elamur.ai/free/en/papers/pang-2022-quality-question-answering-long-input/) - [HistoriQA-ThirdRepublic: Multi-Hop Question Answering Corpus for Historical Research, Parliamentary Debates from the French Third Republic (1870-1940)](https://md.research.elamur.ai/free/en/papers/pellet-2026-historiqa-thirdrepublic-multi-hop-question/) - [Unsupervised Question Decomposition for Question Answering](https://md.research.elamur.ai/free/en/papers/perez-2020-unsupervised-question-decomposition-question-answering/) - [KILT: a Benchmark for Knowledge Intensive Language Tasks](https://md.research.elamur.ai/free/en/papers/petroni-2020-kilt-benchmark-knowledge-intensive-language/) - [Humanity's Last Exam](https://md.research.elamur.ai/free/en/papers/phan-2025-humanity-s-last-exam/) - [Training Question Answering Models From Synthetic Data](https://md.research.elamur.ai/free/en/papers/puri-2020-training-question-answering-models-synthetic/) - [PragmatiCQA: A Dataset for Pragmatic Question Answering in Conversations](https://md.research.elamur.ai/free/en/papers/qi-2023-pragmaticqa-dataset-pragmatic-question-answering/) - [Tool Learning with Foundation Models](https://md.research.elamur.ai/free/en/papers/qin-2023-tool-learning-foundation-models/) - [Pre-trained Models for Natural Language Processing: A Survey](https://md.research.elamur.ai/free/en/papers/qiu-2020-pre-trained-models-natural-language/) - [GPQA: A Graduate-Level Google-Proof Q&A Benchmark](https://md.research.elamur.ai/free/en/papers/rein-2023-gpqa-graduate-level-google-proof/) - [A guide to the dataset explosion in QA, NLI, and commonsense reasoning](https://md.research.elamur.ai/free/en/papers/rogers-2020-guide-dataset-explosion-qa-nli/) - [ColBERTv2: Effective and Efficient Retrieval via Lightweight Late Interaction](https://md.research.elamur.ai/free/en/papers/santhanam-2022-colbertv2-effective-efficient-retrieval-lightweight/) - [Enhancing Retrieval-Augmented Large Language Models with Iterative Retrieval-Generation Synergy](https://md.research.elamur.ai/free/en/papers/shao-2023-enhancing-retrieval-augmented-large-language/) - [StepGame: A New Benchmark for Robust Multi-Hop Spatial Reasoning in Texts](https://md.research.elamur.ai/free/en/papers/shi-2022-stepgame-new-benchmark-robust-multi/) - [Reflexion: Language Agents with Verbal Reinforcement Learning](https://md.research.elamur.ai/free/en/papers/shinn-2023-reflexion-language-agents-verbal-reinforcement/) - [Prompting GPT-3 To Be Reliable](https://md.research.elamur.ai/free/en/papers/si-2022-prompting-gpt-3-reliable/) - [Aya Dataset: An Open-Access Collection for Multilingual Instruction Tuning](https://md.research.elamur.ai/free/en/papers/singh-2024-aya-dataset-open-access-collection/) - [R1-Searcher: Incentivizing the Search Capability in LLMs via Reinforcement Learning](https://md.research.elamur.ai/free/en/papers/song-2025-r1-searcher-incentivizing-search/) - [ASQA: Factoid Questions Meet Long-Form Answers](https://md.research.elamur.ai/free/en/papers/stelmakh-2022-asqa-factoid-questions-meet-long/) - [One Embedder, Any Task: Instruction-Finetuned Text Embeddings](https://md.research.elamur.ai/free/en/papers/su-2022-one-embedder-any-task-instruction/) - [MultiQA: An Empirical Investigation of Generalization and Transfer in Reading Comprehension](https://md.research.elamur.ai/free/en/papers/talmor-2019-multiqa-empirical-investigation-generalization-transfer/) - [oLMpics -- On what Language Model Pre-training Captures](https://md.research.elamur.ai/free/en/papers/talmor-2019-olmpics-language-model-pre-training/) ## Cite The paper itself, not this page. BibTeX from the library's record of the work. ```bibtex @inproceedings{yang2018hotpotqa, title = {{HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering}}, author = {Yang, Zhilin and Qi, Peng and Zhang, Saizheng and Bengio, Yoshua and Cohen, William and Salakhutdinov, Ruslan and Manning, Christopher D.}, booktitle = {Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing}, year = {2018}, doi = {10.18653/v1/d18-1259}, eprint = {1809.09600}, archivePrefix = {arXiv}, url = {https://doi.org/10.18653/v1/d18-1259} } ``` - [← previous HotFlip: White-Box Adversarial Examples for Text Classification](https://md.research.elamur.ai/free/en/papers/ebrahimi-2018-hotflip-white-box-adversarial-examples/) - [next → How Much Reading Does Reading Comprehension Require? A Critical Investigation of Popular Benchmarks](https://md.research.elamur.ai/free/en/papers/kaushik-2018-how-much-reading-does-reading/) - [↑ back to papers of 2018](https://md.research.elamur.ai/free/en/papers/2018/page/2/) ## References 19 references, 12 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet. 1. Danqi Chen, Adam Fisch, Jason Weston, and Antoine Bordes. 2017. Reading Wikipedia to answer open-domain questions. In Association for Computational Linguistics (ACL). [find](https://scholar.google.com/scholar?q=Danqi+Chen%2C+Adam+Fisch%2C+Jason+Weston%2C+and+Antoine+Bordes%2E+2017%2E+Reading+Wikipedia+to+answer+open%2Ddomain+questions%2E+In+Association+for+Computational+Linguistics+%28ACL%29%2E) [our card](https://md.research.elamur.ai/free/en/papers/chen-2017-reading-wikipedia-answer-open-domain/) 2. Pranav Rajpurkar, Jian Zhang, Konstantin Lopyrev, and Percy Liang. 2016. SQuAD: 100,000+ questions for machine comprehension of text. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing (EMNLP). [find](https://scholar.google.com/scholar?q=Pranav+Rajpurkar%2C+Jian+Zhang%2C+Konstantin+Lopyrev%2C+and+Percy+Liang%2E+2016%2E+SQuAD%3A+100%2C000%2B+questions+for+machine+comprehension+of+text%2E+In+Proceedings+of+the+2016+Conference+on+Empirical+Methods+in+Natu) [our card](https://md.research.elamur.ai/free/en/papers/rajpurkar-2016-squad-100-000-questions-machine/) 3. Christopher Clark and Matt Gardner. 2017. Simple and effective multi-paragraph reading comprehension. In Proceedings of the 55th Annual Meeting of the Association of Computational Linguistics. [find](https://scholar.google.com/scholar?q=Christopher+Clark+and+Matt+Gardner%2E+2017%2E+Simple+and+effective+multi%2Dparagraph+reading+comprehension%2E+In+Proceedings+of+the+55th+Annual+Meeting+of+the+Association+of+Computational+Linguistics%2E) [our card](https://md.research.elamur.ai/free/en/papers/clark-2017-simple-effective-multi-paragraph-reading/) 4. Shimi Salant and Jonathan Berant. 2018. Contextualized word representations for reading comprehension. In Proceedings of the 16th Annual Conference of the North American Chapter of the Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Shimi+Salant+and+Jonathan+Berant%2E+2018%2E+Contextualized+word+representations+for+reading+comprehension%2E+In+Proceedings+of+the+16th+Annual+Conference+of+the+North+American+Chapter+of+the+Association+for) 5. Matthew Dunn, Levent Sagun, Mike Higgins, Ugur Guney, Volkan Cirik, and Kyunghyun Cho. 2017. SearchQA: A new Q&A dataset augmented with context from a search engine. arXiv preprint arXiv:1704.05179. [find](https://scholar.google.com/scholar?q=Matthew+Dunn%2C+Levent+Sagun%2C+Mike+Higgins%2C+Ugur+Guney%2C+Volkan+Cirik%2C+and+Kyunghyun+Cho%2E+2017%2E+SearchQA%3A+A+new+Q%26A+dataset+augmented+with+context+from+a+search+engine%2E+arXiv+preprint+arXiv%3A1704%2E05179%2E) [our card](https://md.research.elamur.ai/free/en/papers/dunn-2017-searchqa-new-q-dataset-augmented/) 6. Minjoon Seo, Aniruddha Kembhavi, Ali Farhadi, and Hannaneh Hajishirzi. 2017. Bidirectional attention flow for machine comprehension. In Proceedings of the International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Minjoon+Seo%2C+Aniruddha+Kembhavi%2C+Ali+Farhadi%2C+and+Hannaneh+Hajishirzi%2E+2017%2E+Bidirectional+attention+flow+for+machine+comprehension%2E+In+Proceedings+of+the+International+Conference+on+Learning+Represen) [our card](https://md.research.elamur.ai/free/en/papers/seo-2016-bidirectional-attention-flow-machine-comprehension/) 7. Mandar Joshi, Eunsol Choi, Daniel S. Weld, and Luke Zettlemoyer. 2017. TriviaQA: A large scale distantly supervised challenge dataset for reading comprehension. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Mandar+Joshi%2C+Eunsol+Choi%2C+Daniel+S%2E+Weld%2C+and+Luke+Zettlemoyer%2E+2017%2E+TriviaQA%3A+A+large+scale+distantly+supervised+challenge+dataset+for+reading+comprehension%2E+In+Proceedings+of+the+55th+Annual+Meeti) [our card](https://md.research.elamur.ai/free/en/papers/joshi-2017-triviaqa-large-scale-distantly-supervised/) 8. Alon Talmor and Jonathan Berant. 2018. The web as a knowledge-base for answering complex questions. In Proceedings of the 16th Annual Conference of the North American Chapter of the Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Alon+Talmor+and+Jonathan+Berant%2E+2018%2E+The+web+as+a+knowledge%2Dbase+for+answering+complex+questions%2E+In+Proceedings+of+the+16th+Annual+Conference+of+the+North+American+Chapter+of+the+Association+for+Co) 9. Xiaodong Liu, Yelong Shen, Kevin Duh, and Jianfeng Gao. 2018. Stochastic answer networks for machine reading comprehension. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Xiaodong+Liu%2C+Yelong+Shen%2C+Kevin+Duh%2C+and+Jianfeng+Gao%2E+2018%2E+Stochastic+answer+networks+for+machine+reading+comprehension%2E+In+Proceedings+of+the+56th+Annual+Meeting+of+the+Association+for+Computation) 10. Wenhui Wang, Nan Yang, Furu Wei, Baobao Chang, and Ming Zhou. 2017. Gated self-matching networks for reading comprehension and question answering. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), volume 1, pages 189–198. [find](https://scholar.google.com/scholar?q=Wenhui+Wang%2C+Nan+Yang%2C+Furu+Wei%2C+Baobao+Chang%2C+and+Ming+Zhou%2E+2017%2E+Gated+self%2Dmatching+networks+for+reading+comprehension+and+question+answering%2E+In+Proceedings+of+the+55th+Annual+Meeting+of+the+Asso) [our card](https://md.research.elamur.ai/free/en/papers/wang-2017-gated-self-matching-networks-reading/) 11. Christopher D. Manning, Mihai Surdeanu, John Bauer, Jenny Finkel, Steven J. Bethard, and David Mc-Closky. 2014. The Stanford CoreNLP natural language processing toolkit. In Association for Computational Linguistics (ACL) System Demonstrations, pages 55–60. [find](https://scholar.google.com/scholar?q=Christopher+D%2E+Manning%2C+Mihai+Surdeanu%2C+John+Bauer%2C+Jenny+Finkel%2C+Steven+J%2E+Bethard%2C+and+David+Mc%2DClosky%2E+2014%2E+The+Stanford+CoreNLP+natural+language+processing+toolkit%2E+In+Association+for+Computation) [our card](https://md.research.elamur.ai/free/en/papers/manning-2014-stanford-corenlp-natural-language-processing/) 12. Johannes Welbl, Pontus Stenetorp, and Sebastian Riedel. 2018. Constructing datasets for multi-hop reading comprehension across documents. Transactions of the Association of Computational Linguistics. [find](https://scholar.google.com/scholar?q=Johannes+Welbl%2C+Pontus+Stenetorp%2C+and+Sebastian+Riedel%2E+2018%2E+Constructing+datasets+for+multi%2Dhop+reading+comprehension+across+documents%2E+Transactions+of+the+Association+of+Computational+Linguistics%2E) [our card](https://md.research.elamur.ai/free/en/papers/welbl-2018-constructing-datasets-multi-hop-reading/) 13. Alexander H Miller, Will Feng, Adam Fisch, Jiasen Lu, Dhruv Batra, Antoine Bordes, Devi Parikh, and Jason Weston. 2017. ParlAI: A dialog research software platform. arXiv preprint arXiv:1705.06476. [find](https://scholar.google.com/scholar?q=Alexander+H+Miller%2C+Will+Feng%2C+Adam+Fisch%2C+Jiasen+Lu%2C+Dhruv+Batra%2C+Antoine+Bordes%2C+Devi+Parikh%2C+and+Jason+Weston%2E+2017%2E+ParlAI%3A+A+dialog+research+software+platform%2E+arXiv+preprint+arXiv%3A1705%2E06476%2E) [our card](https://md.research.elamur.ai/free/en/papers/miller-2017-parlai-dialog-research-software-platform/) 14. Caiming Xiong, Victor Zhong, and Richard Socher. 2018. DCN+: Mixed objective and deep residual coattention for question answering. In Proceedings of the International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Caiming+Xiong%2C+Victor+Zhong%2C+and+Richard+Socher%2E+2018%2E+DCN%2B%3A+Mixed+objective+and+deep+residual+coattention+for+question+answering%2E+In+Proceedings+of+the+International+Conference+on+Learning+Representa) 15. Tri Nguyen, Mir Rosenberg, Xia Song, Jianfeng Gao, Saurabh Tiwary, Rangan Majumder, and Li Deng. 2016. MS MARCO: A human generated machine reading comprehension dataset. In Proceedings of the 30th Annual Conference on Neural Information Processing Systems (NIPS). [find](https://scholar.google.com/scholar?q=Tri+Nguyen%2C+Mir+Rosenberg%2C+Xia+Song%2C+Jianfeng+Gao%2C+Saurabh+Tiwary%2C+Rangan+Majumder%2C+and+Li+Deng%2E+2016%2E+MS+MARCO%3A+A+human+generated+machine+reading+comprehension+dataset%2E+In+Proceedings+of+the+30th+Ann) [our card](https://md.research.elamur.ai/free/en/papers/bajaj-2016-ms-marco-human-generated/) 16. Zhilin Yang, Saizheng Zhang, Jack Urbanek, Will Feng, Alexander H Miller, Arthur Szlam, Douwe Kiela, and Jason Weston. 2018. Mastering the dungeon: Grounded language learning by mechanical turker descent. In Proceedings of the International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Zhilin+Yang%2C+Saizheng+Zhang%2C+Jack+Urbanek%2C+Will+Feng%2C+Alexander+H+Miller%2C+Arthur+Szlam%2C+Douwe+Kiela%2C+and+Jason+Weston%2E+2018%2E+Mastering+the+dungeon%3A+Grounded+language+learning+by+mechanical+turker+desc) 17. Jekaterina Novikova, Ondřej Dušek, Amanda Cercas Curry, and Verena Rieser. 2017. Why we need new evaluation metrics for NLG. In Proceedings of the Conference on Empirical Methods in Natural Language Processing. [find](https://scholar.google.com/scholar?q=Jekaterina+Novikova%2C+Ondr%CC%8Cej+Dus%CC%8Cek%2C+Amanda+Cercas+Curry%2C+and+Verena+Rieser%2E+2017%2E+Why+we+need+new+evaluation+metrics+for+NLG%2E+In+Proceedings+of+the+Conference+on+Empirical+Methods+in+Natural+Language) 18. Boyuan Pan, Hao Li, Zhou Zhao, Bin Cao, Deng Cai, and Xiaofei He. 2017. Memen: Multi-layer embedding with memory networks for machine comprehension. arXiv preprint arXiv:1707.09098. — [arXiv:1707.09098](https://arxiv.org/abs/1707.09098) [find](https://scholar.google.com/scholar?q=Boyuan+Pan%2C+Hao+Li%2C+Zhou+Zhao%2C+Bin+Cao%2C+Deng+Cai%2C+and+Xiaofei+He%2E+2017%2E+Memen%3A+Multi%2Dlayer+embedding+with+memory+networks+for+machine+comprehension%2E+arXiv+preprint+arXiv%3A1707%2E09098%2E+%E2%80%94+%5BarXiv%3A1707%2E0909) 19. Pranav Rajpurkar, Robin Jia, and Percy Liang. 2018. Know what you don’t know: Unanswerable questions for SQuAD. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Pranav+Rajpurkar%2C+Robin+Jia%2C+and+Percy+Liang%2E+2018%2E+Know+what+you+don%E2%80%99t+know%3A+Unanswerable+questions+for+SQuAD%2E+In+Proceedings+of+the+56th+Annual+Meeting+of+the+Association+for+Computational+Linguisti) [our card](https://md.research.elamur.ai/free/en/papers/rajpurkar-2018-know-what-you-don-t/)