---
title: "Efficient and Robust Question Answering from Minimal Context"
description: "Neural reading-comprehension models that jointly encode a question and a whole document… 83.1 F1. 40 references, cited by 51 papers here."
authors: ["Sewon Min", "Victor Zhong", "Richard Socher", "Caiming Xiong"]
year: 2018
venue: "Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)"
source: "https://aclanthology.org/P18-1160/"
doi: "10.18653/v1/p18-1160"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/min-2018-efficient-robust-question-answering-minimal/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/min-2018-efficient-robust-question-answering-minimal/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2018](https://md.research.elamur.ai/free/en/papers/2018/page/3/) / Efficient and Robust Question Answering from Minimal Context over Documents

# Efficient and Robust Question Answering from Minimal Context over Documents

**Authors:** [Sewon Min](https://md.research.elamur.ai/free/en/authors/sewon-min/), [Victor Zhong](https://md.research.elamur.ai/free/en/authors/victor-zhong/), [Richard Socher](https://md.research.elamur.ai/free/en/authors/richard-socher/) and [Caiming Xiong](https://md.research.elamur.ai/free/en/authors/caiming-xiong/) · **Year:** 2018 · **Venue:** [Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)](https://md.research.elamur.ai/free/en/venues/acl/) ([ACL 2018](https://md.research.elamur.ai/free/en/venues/acl/2018/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/P18-1160/) · [PDF](https://aclanthology.org/P18-1160.pdf) · [DOI 10.18653/v1/p18-1160](https://doi.org/10.18653/v1/p18-1160) · [OpenAlex](https://openalex.org/W2963159735)

**Citations:** 145 citations (OpenAlex, 2026-10-14) · [51](https://md.research.elamur.ai/free/en/papers/min-2018-efficient-robust-question-answering-minimal/cited-by/) in our library, of the 141,840 papers whose bibliographies we read

## Summary
Neural reading-comprehension models that jointly encode a question and a whole document, for example through coattention, become slow on long or multi-document inputs. They are also easily misled by distracting adversarial sentences. Min et al. first measure how much text a question really needs. Manual inspection of samples from SQuAD and TriviaQA (Wikipedia) shows that one or two sentences usually suffice. DCN+ trained only on the answer-bearing sentence even slightly beats its full-document version.

Building on this, the authors add a lightweight sentence selector that shares its encoder with the S-Reader QA model. It scores all sentences in parallel and forwards a variable, question-dependent number of them to the reader. Three techniques make the selector effective: weight transfer, data modification and score normalization.

On SQuAD, NewsQA, TriviaQA (Wikipedia) and SQuAD-Open, this MINIMAL setup matches or exceeds the FULL baseline while training and inferring much faster. It sets new state-of-the-art results on three of these datasets. It also resists adversarial distractors far better, showing that pruning context helps both scalability and robustness.

## Key points
- **Human study (50 examples per dataset):** On SQuAD, 98% of questions are answerable, and 92% of those need just one sentence. TriviaQA documents average 488 sentences versus about 5 for SQuAD, yet 95% of its answerable questions need at most two sentences.
- **Oracle-sentence experiment:** DCN+ reaches 83.1 F1 with the full document and 85.1 F1 with only the oracle sentence on the SQuAD dev set. The ORACLE model answers 93% (SQuAD) and 86% (NewsQA) of the questions FULL gets right.
- **Efficiency:** MINIMAL gives up to 15× faster training and up to 13× faster inference than FULL. Dynamic sentence selection outperforms TF-IDF retrieval at a similar number of sentences (59.5 vs 51.9 F1).
- **Adversarial robustness (SQuAD-Adversarial dev, top-1 sentence):** DCN+ MINIMAL scores 59.7 F1 on AddSent and 67.5 F1 on AddOneSent. This beats the earlier best, Mnemonic Reader (46.6 and 56.0), and improves over FULL by roughly 11 F1 on both sets.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Seoul National University](https://md.research.elamur.ai/free/en/universities/seoul-national-university/) — University, Seoul, South Korea · 20 works · 45,282 citations

📚 Shelves (5): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 9,086 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 8,889 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 9,296 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 9,206 works · [🗄️ Information Systems](https://md.research.elamur.ai/free/en/shelves/computer-science/information-systems/) · 523 works

🧭 Topics (1): [❓ Question answering](https://md.research.elamur.ai/free/en/topics/question-answering/) · 242 works

## Cite

Cite as: Sewon Min, Victor Zhong, Richard Socher and Caiming Xiong. 2018. Efficient and Robust Question Answering from Minimal Context over Documents. Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{min2018efficient,
  title = {{Efficient and Robust Question Answering from Minimal Context over Documents}},
  author = {Sewon Min, Victor Zhong, Richard Socher and Caiming Xiong},
  booktitle = {Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2018},
  doi = {10.18653/v1/p18-1160},
  url = {https://doi.org/10.18653/v1/p18-1160}
}
```

- [← previous Design Challenges and Misconceptions in Neural Sequence Labeling](https://md.research.elamur.ai/free/en/papers/yang-2018-design-challenges-misconceptions-neural-sequence/)
- [next → Retrieval of the Best Counterargument without Prior Topic Knowledge](https://md.research.elamur.ai/free/en/papers/wachsmuth-2018-retrieval-best-counterargument-without-prior/)
- [↑ back to papers of 2018](https://md.research.elamur.ai/free/en/papers/2018/page/3/)

## References

40 references, 19 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Jonathan Berant, Andrew Chou, Roy Frostig, and Percy Liang. 2013. [Semantic parsing on freebase from question-answer pairs](https://md.research.elamur.ai/free/en/papers/berant-2013-semantic-parsing-freebase-question-answer/). In EMNLP.
2. Danqi Chen, Adam Fisch, Jason Weston, and Antoine Bordes. 2017. [Reading wikipedia to answer open-domain questions](https://md.research.elamur.ai/free/en/papers/chen-2017-reading-wikipedia-answer-open-domain/). In ACL.
3. Eunsol Choi, Daniel Hewlett, Jakob Uszkoreit, Illia Polosukhin, Alexandre Lacoste, and Jonathan Berant. 2017. Coarse-to-fine question answering for long documents. In ACL. [find](https://scholar.google.com/scholar?q=Eunsol+Choi%2C+Daniel+Hewlett%2C+Jakob+Uszkoreit%2C+Illia+Polosukhin%2C+Alexandre+Lacoste%2C+and+Jonathan+Berant%2E+2017%2E+Coarse%2Dto%2Dfine+question+answering+for+long+documents%2E+In+ACL%2E)
4. Christopher Clark and Matt Gardner. 2017. [Simple and effective multi-paragraph reading comprehension](https://md.research.elamur.ai/free/en/papers/clark-2017-simple-effective-multi-paragraph-reading/). arXiv preprint arXiv:1710.10723 .
5. Bhuwan Dhingra, Kathryn Mazaitis, and William W Cohen. 2017. Quasar: Datasets for question answering by search and reading. arXiv preprint arXiv:1707.03904 . [find](https://scholar.google.com/scholar?q=Bhuwan+Dhingra%2C+Kathryn+Mazaitis%2C+and+William+W+Cohen%2E+2017%2E+Quasar%3A+Datasets+for+question+answering+by+search+and+reading%2E+arXiv+preprint+arXiv%3A1707%2E03904+%2E)
6. Matthew Dunn, Levent Sagun, Mike Higgins, Ugur Guney, Volkan Cirik, and Kyunghyun Cho. 2017. [Searchqa: A new q&a dataset augmented with context from a search engine](https://md.research.elamur.ai/free/en/papers/dunn-2017-searchqa-new-q-dataset-augmented/). arXiv preprint arXiv:1704.05179 .
7. Kazuma Hashimoto, Caiming Xiong, Yoshimasa Tsuruoka, and Richard Socher. 2017. [A joint many-task model: Growing a neural network for multiple nlp tasks](https://md.research.elamur.ai/free/en/papers/hashimoto-2016-joint-many-task-model-growing/). In EMNLP.
8. Karl Moritz Hermann, Tomas Kocisky, Edward Grefenstette, Lasse Espeholt, Will Kay, Mustafa Suleyman, and Phil Blunsom. 2015. [Teaching machines to read and comprehend](https://md.research.elamur.ai/free/en/papers/hermann-2015-teaching-machines-read-comprehend/). In Advances in Neural Information Processing Systems.
9. Sepp Hochreiter and Jürgen Schmidhuber. 1997. Long short-term memory. Neural computation . [find](https://scholar.google.com/scholar?q=Sepp+Hochreiter+and+Ju%CC%88rgen+Schmidhuber%2E+1997%2E+Long+short%2Dterm+memory%2E+Neural+computation+%2E)
10. Minghao Hu, Yuxing Peng, and Xipeng Qiu. 2017. Mnemonic reader for machine comprehension. arXiv preprint arXiv:1705.02798 . [find](https://scholar.google.com/scholar?q=Minghao+Hu%2C+Yuxing+Peng%2C+and+Xipeng+Qiu%2E+2017%2E+Mnemonic+reader+for+machine+comprehension%2E+arXiv+preprint+arXiv%3A1705%2E02798+%2E)
11. Hsin-Yuan Huang, Chenguang Zhu, Yelong Shen, and Weizhu Chen. 2018. Fusionnet: Fusing via fully-aware attention with application to machine comprehension. In ICLR. [find](https://scholar.google.com/scholar?q=Hsin%2DYuan+Huang%2C+Chenguang+Zhu%2C+Yelong+Shen%2C+and+Weizhu+Chen%2E+2018%2E+Fusionnet%3A+Fusing+via+fully%2Daware+attention+with+application+to+machine+comprehension%2E+In+ICLR%2E)
12. Robin Jia and Percy Liang. 2017. [Adversarial examples for evaluating reading comprehension systems](https://md.research.elamur.ai/free/en/papers/jia-2017-adversarial-examples-evaluating-reading-comprehension/). In EMNLP.
13. Mandar Joshi, Eunsol Choi, Daniel S Weld, and Luke Zettlemoyer. 2017. [Triviaqa: A large scale distantly supervised challenge dataset for reading comprehension](https://md.research.elamur.ai/free/en/papers/joshi-2017-triviaqa-large-scale-distantly-supervised/). In ACL.
14. Guokun Lai, Qizhe Xie, Hanxiao Liu, Yiming Yang, and Eduard Hovy. 2017. [Race: Large-scale reading comprehension dataset from examinations](https://md.research.elamur.ai/free/en/papers/lai-2017-race-large-scale-reading-comprehension/). In EMNLP.
15. Kenton Lee, Shimi Salant, Tom Kwiatkowski, Ankur Parikh, Dipanjan Das, and Jonathan Berant. 2016. Learning recurrent span representations for extractive question answering. arXiv preprint arXiv:1611.01436 . — [arXiv:1611.01436](https://arxiv.org/abs/1611.01436) [find](https://scholar.google.com/scholar?q=Kenton+Lee%2C+Shimi+Salant%2C+Tom+Kwiatkowski%2C+Ankur+Parikh%2C+Dipanjan+Das%2C+and+Jonathan+Berant%2E+2016%2E+Learning+recurrent+span+representations+for+extractive+question+answering%2E+arXiv+preprint+arXiv%3A1611%2E0)
16. Christopher Manning, Mihai Surdeanu, John Bauer, Jenny Finkel, Steven Bethard, and David McClosky. 2014. [The stanford corenlp natural language processing toolkit](https://md.research.elamur.ai/free/en/papers/manning-2014-stanford-corenlp-natural-language-processing/). In ACL.
17. Bryan McCann, James Bradbury, Caiming Xiong, and Richard Socher. 2017. [Learned in translation: Contextualized word vectors](https://md.research.elamur.ai/free/en/papers/mccann-2017-learned-translation-contextualized-word-vectors/). In NIPS.
18. Alexander Miller, Adam Fisch, Jesse Dodge, Amir-Hossein Karimi, Antoine Bordes, and Jason Weston. 2016. Key-value memory networks for directly reading documents. In EMNLP. [find](https://scholar.google.com/scholar?q=Alexander+Miller%2C+Adam+Fisch%2C+Jesse+Dodge%2C+Amir%2DHossein+Karimi%2C+Antoine+Bordes%2C+and+Jason+Weston%2E+2016%2E+Key%2Dvalue+memory+networks+for+directly+reading+documents%2E+In+EMNLP%2E)
19. Sewon Min, Minjoon Seo, and Hannaneh Hajishirzi. 2017. Question answering through transfer learning from large fine-grained supervision data. In ACL. [find](https://scholar.google.com/scholar?q=Sewon+Min%2C+Minjoon+Seo%2C+and+Hannaneh+Hajishirzi%2E+2017%2E+Question+answering+through+transfer+learning+from+large+fine%2Dgrained+supervision+data%2E+In+ACL%2E)
20. Boyuan Pan, Hao Li, Zhou Zhao, Bin Cao, Deng Cai, and Xiaofei He. 2017. Memen: Multi-layer embedding with memory networks for machine comprehension. arXiv preprint arXiv:1707.09098 . — [arXiv:1707.09098](https://arxiv.org/abs/1707.09098) [find](https://scholar.google.com/scholar?q=Boyuan+Pan%2C+Hao+Li%2C+Zhou+Zhao%2C+Bin+Cao%2C+Deng+Cai%2C+and+Xiaofei+He%2E+2017%2E+Memen%3A+Multi%2Dlayer+embedding+with+memory+networks+for+machine+comprehension%2E+arXiv+preprint+arXiv%3A1707%2E09098+%2E+%E2%80%94+%5BarXiv%3A1707%2E090)
21. Jeffrey Pennington, Richard Socher, and Christopher Manning. 2014. [Glove: Global vectors for word representation](https://md.research.elamur.ai/free/en/papers/pennington-2014-glove-global-vectors-word-representation/). In EMNLP.
22. Jonathan Raiman and John Miller. 2017. Globally normalized reader. In EMNLP. [find](https://scholar.google.com/scholar?q=Jonathan+Raiman+and+John+Miller%2E+2017%2E+Globally+normalized+reader%2E+In+EMNLP%2E)
23. Pranav Rajpurkar, Jian Zhang, Konstantin Lopyrev, and Percy Liang. 2016. [Squad: 100,000+ questions for machine comprehension of text](https://md.research.elamur.ai/free/en/papers/rajpurkar-2016-squad-100-000-questions-machine/). In EMNLP.
24. Matthew Richardson, Christopher JC Burges, and Erin Renshaw. 2013. [Mctest: A challenge dataset for the open-domain machine comprehension of text](https://md.research.elamur.ai/free/en/papers/richardson-2013-mctest-challenge-dataset-open-domain/). In EMNLP.
25. Minjoon Seo, Aniruddha Kembhavi, Ali Farhadi, and Hannaneh Hajishirzi. 2017. [Bidirectional attention flow for machine comprehension](https://md.research.elamur.ai/free/en/papers/seo-2016-bidirectional-attention-flow-machine-comprehension/). In ICLR.
26. Diederik P Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. arXiv preprint arXiv:1706.02596v2 . [find](https://scholar.google.com/scholar?q=Diederik+P+Kingma+and+Jimmy+Ba%2E+2014%2E+Adam%3A+A+method+for+stochastic+optimization%2E+arXiv+preprint+arXiv%3A1706%2E02596v2+%2E)
27. Yelong Shen, Po-Sen Huang, Jianfeng Gao, and Weizhu Chen. 2017. Reasonet: Learning to stop reading in machine comprehension. In ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. [find](https://scholar.google.com/scholar?q=Yelong+Shen%2C+Po%2DSen+Huang%2C+Jianfeng+Gao%2C+and+Weizhu+Chen%2E+2017%2E+Reasonet%3A+Learning+to+stop+reading+in+machine+comprehension%2E+In+ACM+SIGKDD+International+Conference+on+Knowledge+Discovery+and+Data+Mini)
28. Tomáš Kočiskỳ, Jonathan Schwarz, Phil Blunsom, Chris Dyer, Karl Moritz Hermann, Gábor Melis, and Edward Grefenstette. 2017. [The narrativeqa reading comprehension challenge](https://md.research.elamur.ai/free/en/papers/kocisky-2018-narrativeqa-reading-comprehension-challenge/). arXiv preprint arXiv:1712.07040 .
29. Nitish Srivastava, Geoffrey E Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. 2014. Dropout: a simple way to prevent neural networks from overfitting. Journal of machine learning research . [find](https://scholar.google.com/scholar?q=Nitish+Srivastava%2C+Geoffrey+E+Hinton%2C+Alex+Krizhevsky%2C+Ilya+Sutskever%2C+and+Ruslan+Salakhutdinov%2E+2014%2E+Dropout%3A+a+simple+way+to+prevent+neural+networks+from+overfitting%2E+Journal+of+machine+learning+re)
30. Swabha Swayamdipta, Ankur P Parikh, and Tom Kwiatkowski. 2018. Multi-mention learning for reading comprehension with neural cascades. In ICLR. [find](https://scholar.google.com/scholar?q=Swabha+Swayamdipta%2C+Ankur+P+Parikh%2C+and+Tom+Kwiatkowski%2E+2018%2E+Multi%2Dmention+learning+for+reading+comprehension+with+neural+cascades%2E+In+ICLR%2E)
31. Chuanqi Tan, Furu Wei, Qingyu Zhou, Nan Yang, Bowen Du, Weifeng Lv, and Ming Zhou. 2018. Context-aware answer sentence selection with hierarchical gated recurrent neural networks. IEEE/ACM Transactions on Audio, Speech, and Language Processing . [find](https://scholar.google.com/scholar?q=Chuanqi+Tan%2C+Furu+Wei%2C+Qingyu+Zhou%2C+Nan+Yang%2C+Bowen+Du%2C+Weifeng+Lv%2C+and+Ming+Zhou%2E+2018%2E+Context%2Daware+answer+sentence+selection+with+hierarchical+gated+recurrent+neural+networks%2E+IEEE%2FACM+Transaction)
32. Adam Trischler, Tong Wang, Xingdi Yuan, Justin Harris, Alessandro Sordoni, Philip Bachman, and Kaheer Suleman. 2016. [Newsqa: A machine comprehension dataset](https://md.research.elamur.ai/free/en/papers/trischler-2017-newsqa-machine-comprehension-dataset/). arXiv preprint arXiv:1611.09830 .
33. Shuohang Wang, Mo Yu, Xiaoxiao Guo, Zhiguo Wang, Tim Klinger, Wei Zhang, Shiyu Chang, Gerald Tesauro, Bowen Zhou, and Jing Jiang. 2018. R3: Reinforced reader-ranker for open-domain question answering. In AAAI. [find](https://scholar.google.com/scholar?q=Shuohang+Wang%2C+Mo+Yu%2C+Xiaoxiao+Guo%2C+Zhiguo+Wang%2C+Tim+Klinger%2C+Wei+Zhang%2C+Shiyu+Chang%2C+Gerald+Tesauro%2C+Bowen+Zhou%2C+and+Jing+Jiang%2E+2018%2E+R3%3A+Reinforced+reader%2Dranker+for+open%2Ddomain+question+answering%2E)
34. Dirk Weissenborn. 2017. Reading twice for natural language understanding. CoRR abs/1706.02596. — [arXiv:1706.02596](https://arxiv.org/abs/1706.02596) [find](https://scholar.google.com/scholar?q=Dirk+Weissenborn%2E+2017%2E+Reading+twice+for+natural+language+understanding%2E+CoRR+abs%2F1706%2E02596%2E+%E2%80%94+%5BarXiv%3A1706%2E02596%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F1706%2E02596%29)
35. Dirk Weissenborn, Georg Wiese, and Laura Seiffe. 2017. Making neural qa as simple as possible but not simpler. In CoNLL. [find](https://scholar.google.com/scholar?q=Dirk+Weissenborn%2C+Georg+Wiese%2C+and+Laura+Seiffe%2E+2017%2E+Making+neural+qa+as+simple+as+possible+but+not+simpler%2E+In+CoNLL%2E)
36. Qizhe Xie, Guokun Lai, Zihang Dai, and Eduard Hovy. 2017. Large-scale cloze test dataset designed by teachers. arXiv preprint arXiv:1711.03225 . — [arXiv:1711.03225](https://arxiv.org/abs/1711.03225) [find](https://scholar.google.com/scholar?q=Qizhe+Xie%2C+Guokun+Lai%2C+Zihang+Dai%2C+and+Eduard+Hovy%2E+2017%2E+Large%2Dscale+cloze+test+dataset+designed+by+teachers%2E+arXiv+preprint+arXiv%3A1711%2E03225+%2E+%E2%80%94+%5BarXiv%3A1711%2E03225%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F1711%2E03225%29)
37. Caiming Xiong, Victor Zhong, and Richard Socher. 2018. Dcn+: Mixed objective and deep residual coattention for question answering. In ICLR. [find](https://scholar.google.com/scholar?q=Caiming+Xiong%2C+Victor+Zhong%2C+and+Richard+Socher%2E+2018%2E+Dcn%2B%3A+Mixed+objective+and+deep+residual+coattention+for+question+answering%2E+In+ICLR%2E)
38. Yi Yang, Wen-tau Yih, and Christopher Meek. 2015. [Wikiqa: A challenge dataset for open-domain question answering](https://md.research.elamur.ai/free/en/papers/yang-2015-wikiqa-challenge-dataset-open-domain/). In EMNLP.
39. Wenpeng Yin, Hinrich Schtze, Bing Xiang, and Bowen Zhou. 2016. [Abcnn: Attention-based convolutional neural network for modeling sentence pairs](https://md.research.elamur.ai/free/en/papers/yin-2016-abcnn-attention-based-convolutional-neural/). TACL .
40. Adams Wei Yu, David Dohan, Quoc Le, Thang Luong, Rui Zhao, and Kai Chen. 2018. Fast and accurate reading comprehension by combining self-attention and convolution. In ICLR. [find](https://scholar.google.com/scholar?q=Adams+Wei+Yu%2C+David+Dohan%2C+Quoc+Le%2C+Thang+Luong%2C+Rui+Zhao%2C+and+Kai+Chen%2E+2018%2E+Fast+and+accurate+reading+comprehension+by+combining+self%2Dattention+and+convolution%2E+In+ICLR%2E)

*Updated 24 October 2026*