---
title: "Multi-hop Reading Comprehension through Question — Min 2019"
description: "Multi-hop reading comprehension asks a system to combine evidence from several paragraphs."
authors: ["Sewon Min", "Victor Zhong", "Luke Zettlemoyer", "Hannaneh Hajishirzi"]
year: 2019
venue: "Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics"
source: "https://aclanthology.org/P19-1613/"
doi: "10.18653/v1/p19-1613"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/min-2019-multi-hop-reading-comprehension-through/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/min-2019-multi-hop-reading-comprehension-through/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2019](https://md.research.elamur.ai/free/en/papers/2019/page/3/) / Multi-hop Reading Comprehension through Question Decomposition and Rescoring

# Multi-hop Reading Comprehension through Question Decomposition and Rescoring

**Authors:** [Sewon Min](https://md.research.elamur.ai/free/en/authors/sewon-min/), [Victor Zhong](https://md.research.elamur.ai/free/en/authors/victor-zhong/), [Luke Zettlemoyer](https://md.research.elamur.ai/free/en/authors/luke-zettlemoyer/), [Hannaneh Hajishirzi](https://md.research.elamur.ai/free/en/authors/hannaneh-hajishirzi/) · **Year:** 2019 · **Venue:** Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/P19-1613/) · [PDF](https://aclanthology.org/P19-1613.pdf) · [DOI 10.18653/v1/p19-1613](https://doi.org/10.18653/v1/p19-1613) · [OpenAlex](https://openalex.org/W2949134692)

**Citations:** 194 citations (OpenAlex, 2026-10-14) · [112](https://md.research.elamur.ai/free/en/papers/min-2019-multi-hop-reading-comprehension-through/cited-by/) in our library, of the 140884 papers whose bibliographies we read

## Summary

Multi-hop reading comprehension asks a system to combine evidence from several paragraphs. The paper presents DecompRC, which splits a compositional question into simpler single-hop sub-questions. An ordinary off-the-shelf single-hop RC model then answers these sub-questions.

Writing sub-questions word by word would need a lot of annotated data. The authors avoid this by treating decomposition as span prediction. A BERT-based model, Pointer_c, marks a few boundary positions inside the question. Light rule-based editing then turns those spans into sub-questions for three reasoning types: bridging, intersection and comparison. Only 400 annotated decompositions are needed to train it.

All reasoning types are tried in parallel, so the system does not commit to one interpretation up front. A decomposition scorer then judges each candidate together with its answers and evidence, and picks the final answer.

On HotpotQA, DecompRC reached state-of-the-art results in both the distractor and full wiki settings. It also proved more robust than an end-to-end BERT baseline under adversarial distractors and questions. The sub-questions make each prediction explainable.

## Key points

- **Reasoning types:** The authors manually inspected 200 HotpotQA dev questions, of which 184 needed multi-hop reasoning. Of these, 47% were bridging, 23% intersection and 22% comparison; the remaining 8% fit none of the three types.
- **Sample efficiency:** Sub-questions produced by the span-based decomposer, trained on just 400 examples, worked about as well as sub-questions written by humans.
- **Rescoring vs. alternatives:** The answer-aware scorer beat a pipeline that chooses the reasoning type beforehand, and also beat selection by RC-model confidence. It still trails an oracle by 6.2 F1.
- **Human upper bound:** On 50 bridging questions, humans answered the annotated sub-questions without seeing the original question and reached 72.67 F1. Of the 15 failures, 53% came from groundtruth or human issues and 47% from genuine decomposition problems.
- **Limitations and resources:** The method fails on questions that need implicit reasoning or commonsense inference, and on operations such as counting. Code and a demo are available at github.com/shmsw25/DecompRC.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

Fields: [Natural language processing](https://md.research.elamur.ai/free/en/shelves/nlp/)

Topics: [Question answering](https://md.research.elamur.ai/free/en/topics/question-answering/)

## Cite

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{min2019multi,
  title = {{Multi-hop Reading Comprehension through Question Decomposition and Rescoring}},
  author = {Min, Sewon and Zhong, Victor and Zettlemoyer, Luke and Hajishirzi, Hannaneh},
  booktitle = {Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics},
  year = {2019},
  doi = {10.18653/v1/p19-1613},
  url = {https://doi.org/10.18653/v1/p19-1613}
}
```

- [← previous A Transformer-based approach to Irony and Sarcasm detection](https://md.research.elamur.ai/free/en/papers/potamias-2019-transformer-based-approach-irony-sarcasm/)
- [next → Challenges and frontiers in abusive content detection](https://md.research.elamur.ai/free/en/papers/vidgen-2019-challenges-frontiers-abusive-content-detection/)
- [↑ back to papers of 2019](https://md.research.elamur.ai/free/en/papers/2019/page/3/)

## References

26 references, 10 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Jonathan Berant, Andrew Chou, Roy Frostig, and Percy Liang. 2013. Semantic parsing on freebase from question-answer pairs. In EMNLP. [find](https://scholar.google.com/scholar?q=Jonathan+Berant%2C+Andrew+Chou%2C+Roy+Frostig%2C+and+Percy+Liang%2E+2013%2E+Semantic+parsing+on+freebase+from+question%2Danswer+pairs%2E+In+EMNLP%2E)
2. Nicola De Cao, Wilker Aziz, and Ivan Titov. 2019. Question answering by reasoning across documents with graph convolutional networks. In NAACL. [find](https://scholar.google.com/scholar?q=Nicola+De+Cao%2C+Wilker+Aziz%2C+and+Ivan+Titov%2E+2019%2E+Question+answering+by+reasoning+across+documents+with+graph+convolutional+networks%2E+In+NAACL%2E)
3. Danqi Chen, Adam Fisch, Jason Weston, and Antoine Bordes. 2017. Reading Wikipedia to answer open-domain questions. In ACL. [find](https://scholar.google.com/scholar?q=Danqi+Chen%2C+Adam+Fisch%2C+Jason+Weston%2C+and+Antoine+Bordes%2E+2017%2E+Reading+Wikipedia+to+answer+open%2Ddomain+questions%2E+In+ACL%2E) [our card](https://md.research.elamur.ai/free/en/papers/chen-2017-reading-wikipedia-answer-open-domain/)
4. Matthew Richardson, Christopher JC Burges, and Erin Renshaw. 2013. MCTest: A challenge dataset for the open-domain machine comprehension of text. In EMNLP. [find](https://scholar.google.com/scholar?q=Matthew+Richardson%2C+Christopher+JC+Burges%2C+and+Erin+Renshaw%2E+2013%2E+MCTest%3A+A+challenge+dataset+for+the+open%2Ddomain+machine+comprehension+of+text%2E+In+EMNLP%2E) [our card](https://md.research.elamur.ai/free/en/papers/missing-2013-richardson-mctest-challenge-dataset-open-domain/)
5. Minjoon Seo, Aniruddha Kembhavi, Ali Farhadi, and Hannaneh Hajishirzi. 2017. Bidirectional attention flow for machine comprehension. In ICLR. [find](https://scholar.google.com/scholar?q=Minjoon+Seo%2C+Aniruddha+Kembhavi%2C+Ali+Farhadi%2C+and+Hannaneh+Hajishirzi%2E+2017%2E+Bidirectional+attention+flow+for+machine+comprehension%2E+In+ICLR%2E) [our card](https://md.research.elamur.ai/free/en/papers/seo-2016-bidirectional-attention-flow-machine-comprehension/)
6. Alon Talmor and Jonathan Berant. 2018. The web as a knowledge-base for answering complex questions. In NAACL. [find](https://scholar.google.com/scholar?q=Alon+Talmor+and+Jonathan+Berant%2E+2018%2E+The+web+as+a+knowledge%2Dbase+for+answering+complex+questions%2E+In+NAACL%2E)
7. Christopher Clark and Matt Gardner. 2018. Simple and effective multi-paragraph reading comprehension. In ACL. [find](https://scholar.google.com/scholar?q=Christopher+Clark+and+Matt+Gardner%2E+2018%2E+Simple+and+effective+multi%2Dparagraph+reading+comprehension%2E+In+ACL%2E) [our card](https://md.research.elamur.ai/free/en/papers/clark-2017-simple-effective-multi-paragraph-reading/)
8. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In NIPS. [find](https://scholar.google.com/scholar?q=Ashish+Vaswani%2C+Noam+Shazeer%2C+Niki+Parmar%2C+Jakob+Uszkoreit%2C+Llion+Jones%2C+Aidan+N+Gomez%2C+Lukasz+Kaiser%2C+and+Illia+Polosukhin%2E+2017%2E+Attention+is+all+you+need%2E+In+NIPS%2E)
9. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. In NAACL. [find](https://scholar.google.com/scholar?q=Jacob+Devlin%2C+Ming%2DWei+Chang%2C+Kenton+Lee%2C+and+Kristina+Toutanova%2E+2019%2E+BERT%3A+Pre%2Dtraining+of+deep+bidirectional+transformers+for+language+understanding%2E+In+NAACL%2E)
10. Johannes Welbl, Pontus Stenetorp, and Sebastian Riedel. 2017. Constructing Datasets for Multi-hop Reading Comprehension Across Documents. In TACL. [find](https://scholar.google.com/scholar?q=Johannes+Welbl%2C+Pontus+Stenetorp%2C+and+Sebastian+Riedel%2E+2017%2E+Constructing+Datasets+for+Multi%2Dhop+Reading+Comprehension+Across+Documents%2E+In+TACL%2E) [our card](https://md.research.elamur.ai/free/en/papers/welbl-2018-constructing-datasets-multi-hop-reading/)
11. Bhuwan Dhingra, Qiao Jin, Zhilin Yang, William W Cohen, and Ruslan Salakhutdinov. 2018. Neural models for reasoning over multiple mentions using coreference. In NAACL. [find](https://scholar.google.com/scholar?q=Bhuwan+Dhingra%2C+Qiao+Jin%2C+Zhilin+Yang%2C+William+W+Cohen%2C+and+Ruslan+Salakhutdinov%2E+2018%2E+Neural+models+for+reasoning+over+multiple+mentions+using+coreference%2E+In+NAACL%2E) [our card](https://md.research.elamur.ai/free/en/papers/dhingra-2018-neural-models-reasoning-over-multiple/)
12. Caiming Xiong, Victor Zhong, and Richard Socher. 2018. DCN+: Mixed objective and deep residual coattention for question answering. In ICLR. [find](https://scholar.google.com/scholar?q=Caiming+Xiong%2C+Victor+Zhong%2C+and+Richard+Socher%2E+2018%2E+DCN%2B%3A+Mixed+objective+and+deep+residual+coattention+for+question+answering%2E+In+ICLR%2E)
13. Albert Gatt and Emiel Krahmer. 2018. Survey of the state of the art in natural language generation: Core tasks, applications and evaluation. Artificial Intelligence Research. [find](https://scholar.google.com/scholar?q=Albert+Gatt+and+Emiel+Krahmer%2E+2018%2E+Survey+of+the+state+of+the+art+in+natural+language+generation%3A+Core+tasks%2C+applications+and+evaluation%2E+Artificial+Intelligence+Research%2E)
14. Zhilin Yang, Peng Qi, Saizheng Zhang, Yoshua Bengio, William W Cohen, Ruslan Salakhutdinov, and Christopher D Manning. 2018. HotpotQA: A dataset for diverse, explainable multi-hop question answering. In EMNLP. [find](https://scholar.google.com/scholar?q=Zhilin+Yang%2C+Peng+Qi%2C+Saizheng+Zhang%2C+Yoshua+Bengio%2C+William+W+Cohen%2C+Ruslan+Salakhutdinov%2C+and+Christopher+D+Manning%2E+2018%2E+HotpotQA%3A+A+dataset+for+diverse%2C+explainable+multi%2Dhop+question+answering%2E+) [our card](https://md.research.elamur.ai/free/en/papers/yang-2018-hotpotqa-dataset-diverse-explainable-multi/)
15. Karl Moritz Hermann, Tomas Kocisky, Edward Grefenstette, Lasse Espeholt, Will Kay, Mustafa Suleyman, and Phil Blunsom. 2015. Teaching machines to read and comprehend. In NIPS. [find](https://scholar.google.com/scholar?q=Karl+Moritz+Hermann%2C+Tomas+Kocisky%2C+Edward+Grefenstette%2C+Lasse+Espeholt%2C+Will+Kay%2C+Mustafa+Suleyman%2C+and+Phil+Blunsom%2E+2015%2E+Teaching+machines+to+read+and+comprehend%2E+In+NIPS%2E) [our card](https://md.research.elamur.ai/free/en/papers/hermann-2015-teaching-machines-read-comprehend/)
16. Mandar Joshi, Eunsol Choi, Daniel S Weld, and Luke Zettlemoyer. 2017. TriviaQA: A large scale distantly supervised challenge dataset for reading comprehension. In ACL. [find](https://scholar.google.com/scholar?q=Mandar+Joshi%2C+Eunsol+Choi%2C+Daniel+S+Weld%2C+and+Luke+Zettlemoyer%2E+2017%2E+TriviaQA%3A+A+large+scale+distantly+supervised+challenge+dataset+for+reading+comprehension%2E+In+ACL%2E) [our card](https://md.research.elamur.ai/free/en/papers/joshi-2017-triviaqa-large-scale-distantly-supervised/)
17. Diederik P Kingma and Jimmy Ba. 2015. Adam: A method for stochastic optimization. In ICLR. [find](https://scholar.google.com/scholar?q=Diederik+P+Kingma+and+Jimmy+Ba%2E+2015%2E+Adam%3A+A+method+for+stochastic+optimization%2E+In+ICLR%2E)
18. Percy Liang, Michael Jordan, and Dan Klein. 2011. Learning dependency-based compositional semantics. In ACL. [find](https://scholar.google.com/scholar?q=Percy+Liang%2C+Michael+Jordan%2C+and+Dan+Klein%2E+2011%2E+Learning+dependency%2Dbased+compositional+semantics%2E+In+ACL%2E)
19. Sewon Min, Victor Zhong, Richard Socher, and Caiming Xiong. 2018. Efficient and robust question answering from minimal context over documents. In ACL. [find](https://scholar.google.com/scholar?q=Sewon+Min%2C+Victor+Zhong%2C+Richard+Socher%2C+and+Caiming+Xiong%2E+2018%2E+Efficient+and+robust+question+answering+from+minimal+context+over+documents%2E+In+ACL%2E)
20. Adams Wei Yu, David Dohan, Quoc Le, Thang Luong, Rui Zhao, and Kai Chen. 2018. Fast and accurate reading comprehension by combining self-attention and convolution. In ICLR. [find](https://scholar.google.com/scholar?q=Adams+Wei+Yu%2C+David+Dohan%2C+Quoc+Le%2C+Thang+Luong%2C+Rui+Zhao%2C+and+Kai+Chen%2E+2018%2E+Fast+and+accurate+reading+comprehension+by+combining+self%2Dattention+and+convolution%2E+In+ICLR%2E)
21. John M. Zelle and Raymond J. Mooney. 1996. Learning to parse database queries using inductive logic programming. In AAAI/IAAI. [find](https://scholar.google.com/scholar?q=John+M%2E+Zelle+and+Raymond+J%2E+Mooney%2E+1996%2E+Learning+to+parse+database+queries+using+inductive+logic+programming%2E+In+AAAI%2FIAAI%2E)
22. Luke S. Zettlemoyer and Michael Collins. 2005. Learning to map sentences to logical form: Structured classification with probabilistic categorial grammars. In UAI. [find](https://scholar.google.com/scholar?q=Luke+S%2E+Zettlemoyer+and+Michael+Collins%2E+2005%2E+Learning+to+map+sentences+to+logical+form%3A+Structured+classification+with+probabilistic+categorial+grammars%2E+In+UAI%2E)
23. Victor Zhong, Caiming Xiong, Nitish Shirish Keskar, and Richard Socher. 2019. Coarse-grain fine-grain coattention network for multi-evidence question answering. In ICLR. [find](https://scholar.google.com/scholar?q=Victor+Zhong%2C+Caiming+Xiong%2C+Nitish+Shirish+Keskar%2C+and+Richard+Socher%2E+2019%2E+Coarse%2Dgrain+fine%2Dgrain+coattention+network+for+multi%2Devidence+question+answering%2E+In+ICLR%2E)
24. Jekaterina Novikova, Ondej Duek, Amanda Cercas Curry, and Verena Rieser. 2017. Why we need new evaluation metrics for NLG. In EMNLP. [find](https://scholar.google.com/scholar?q=Jekaterina+Novikova%2C+Ondej+Duek%2C+Amanda+Cercas+Curry%2C+and+Verena+Rieser%2E+2017%2E+Why+we+need+new+evaluation+metrics+for+NLG%2E+In+EMNLP%2E)
25. Adam Paszke, Sam Gross, Soumith Chintala, Gregory Chanan, Edward Yang, Zachary DeVito, Zeming Lin, Alban Desmaison, Luca Antiga, and Adam Lerer. 2017. Automatic differentiation in PyTorch. [find](https://scholar.google.com/scholar?q=Adam+Paszke%2C+Sam+Gross%2C+Soumith+Chintala%2C+Gregory+Chanan%2C+Edward+Yang%2C+Zachary+DeVito%2C+Zeming+Lin%2C+Alban+Desmaison%2C+Luca+Antiga%2C+and+Adam+Lerer%2E+2017%2E+Automatic+differentiation+in+PyTorch%2E)
26. Pranav Rajpurkar, Jian Zhang, Konstantin Lopyrev, and Percy Liang. 2016. SQuAD: 100,000+ questions for machine comprehension of text. In EMNLP. [find](https://scholar.google.com/scholar?q=Pranav+Rajpurkar%2C+Jian+Zhang%2C+Konstantin+Lopyrev%2C+and+Percy+Liang%2E+2016%2E+SQuAD%3A+100%2C000%2B+questions+for+machine+comprehension+of+text%2E+In+EMNLP%2E) [our card](https://md.research.elamur.ai/free/en/papers/rajpurkar-2016-squad-100-000-questions-machine/)