---
title: "Commonsense for Generative Multi-Hop Question Answering"
description: "The authors first build a strong baseline, the Multi-Hop Pointer-Generator Model… 41.49 ROUGE-L. 38 references, cited by 68 papers here."
authors: ["Lisa Bauer", "Yicheng Wang", "Mohit Bansal"]
year: 2018
venue: "Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing"
source: "https://aclanthology.org/D18-1454/"
doi: "10.18653/v1/d18-1454"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/bauer-2018-commonsense-generative-multi-hop-question/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/bauer-2018-commonsense-generative-multi-hop-question/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2018](https://md.research.elamur.ai/free/en/papers/2018/page/3/) / Commonsense for Generative Multi-Hop Question Answering Tasks

# Commonsense for Generative Multi-Hop Question Answering Tasks

**Authors:** Lisa Bauer, [Yicheng Wang](https://md.research.elamur.ai/free/en/authors/yicheng-wang/) and [Mohit Bansal](https://md.research.elamur.ai/free/en/authors/mohit-bansal/) · **Year:** 2018 · **Venue:** [Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing](https://md.research.elamur.ai/free/en/venues/emnlp/) ([EMNLP 2018](https://md.research.elamur.ai/free/en/venues/emnlp/2018/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/D18-1454/) · [PDF](https://aclanthology.org/D18-1454.pdf) · [DOI 10.18653/v1/d18-1454](https://doi.org/10.18653/v1/d18-1454) · [OpenAlex](https://openalex.org/W2892280852)

**Citations:** 161 citations (OpenAlex, 2026-10-14) · [68](https://md.research.elamur.ai/free/en/papers/bauer-2018-commonsense-generative-multi-hop-question/cited-by/) in our library, of the 141,840 papers whose bibliographies we read

## Summary
Most reading-comprehension QA research targets extractive, fact-lookup questions. This EMNLP 2018 paper addresses the harder generative setting of NarrativeQA. There, answers must be composed from several scattered parts of a long story, and they often depend on implicit links that people bridge with everyday background knowledge. The authors first build a strong baseline, the Multi-Hop Pointer-Generator Model (MHPGM). It combines learned and ELMo embeddings, a stack of k reasoning cells based on BiDAF attention, a self-attention layer, and a pointer-generator decoder that can copy rare words from the context. They then propose a way to extract multi-hop relation paths from ConceptNet that are anchored in the context and question, ranked by a PMI and term-frequency scoring function. These paths are injected between reasoning hops through the Necessary and Optional Information Cell (NOIC), a selectively gated attention unit. The baseline already beats earlier generative systems, and NOIC sets a new state of the art. Human judges confirm the gain, and it also transfers to QAngaroo-WikiHop. This shows that structured commonsense measurably helps generative multi-hop reasoning.

## Key points
- On the NarrativeQA summary subtask, MHPGM reaches 41.49 Rouge-L and 17.33 METEOR. Adding NOIC with ConceptNet paths raises this to 44.16 Rouge-L and 19.03 METEOR.
- On QAngaroo-WikiHop, the commonsense-enhanced model gains 1.5% accuracy over the MHPGM baseline.
- Ablations show the importance of the full design:
  - Limiting the model to a single attention hop (k=1) costs over 12 Rouge-L points.
  - Simpler commonsense variants give no significant improvement (p ≥ 0.15). These include NumberBatch embeddings, random in-domain relations, and single-hop extraction.
- Manual check of 50 test samples:
  - Half of the questions needed external commonsense.
  - The selector supplied relevant knowledge in 34% of all samples.
  - Overall, the selector was useful 48% of the time.
- Human comparison on 100 examples by 3 annotators (Fleiss κ = 0.831):
  - MHPGM+NOIC was judged better in 23% of cases.
  - MHPGM was judged better in 15% of cases.
  - Code, models, and data are released at github.com/yicheng-w/CommonSenseMultiHopQA.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [University of North Carolina at Chapel Hill](https://md.research.elamur.ai/free/en/universities/university-of-north-carolina-at-chapel-hill/) — University, Chapel Hill, United States · 37 works · 14,332 citations

📚 Shelves (5): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 9,086 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 8,889 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 9,296 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 9,206 works · [👁️ Computer Vision and Pattern Recognition](https://md.research.elamur.ai/free/en/shelves/computer-science/computer-vision-and-pattern-recognition/) · 1,282 works

🧭 Topics (2): [🧮 Reasoning](https://md.research.elamur.ai/free/en/topics/reasoning/) · 510 works · [❓ Question answering](https://md.research.elamur.ai/free/en/topics/question-answering/) · 242 works

## Cite

Cite as: Lisa Bauer, Yicheng Wang and Mohit Bansal. 2018. Commonsense for Generative Multi-Hop Question Answering Tasks. Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{bauer2018commonsense,
  title = {{Commonsense for Generative Multi-Hop Question Answering Tasks}},
  author = {Lisa Bauer, Yicheng Wang and Mohit Bansal},
  booktitle = {Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing},
  year = {2018},
  doi = {10.18653/v1/d18-1454},
  url = {https://doi.org/10.18653/v1/d18-1454}
}
```

- [← previous Harvesting Paragraph-level Question-Answer Pairs from Wikipedia](https://md.research.elamur.ai/free/en/papers/du-2018-harvesting-paragraph-level-question-answer/)
- [next → Cross-topic Argument Mining from Heterogeneous Sources](https://md.research.elamur.ai/free/en/papers/stab-2018-cross-topic-argument-mining-heterogeneous/)
- [↑ back to papers of 2018](https://md.research.elamur.ai/free/en/papers/2018/page/3/)

## References

38 references, 19 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Short Papers), volume 2, pages 42–48. [find](https://scholar.google.com/scholar?q=the+North+American+Chapter+of+the+Association+for+Computational+Linguistics%3A+Human+Language+Technologies%2C+Volume+2+%28Short+Papers%29%2C+volume+2%2C+pages+42%E2%80%9348%2E)
2. Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. 2015. [Neural machine translation by jointly learning to align and translate](https://md.research.elamur.ai/free/en/papers/bahdanau-2014-neural-machine-translation-jointly-learning/). In ICLR.
3. Satanjeev Banerjee and Alon Lavie. 2005. [Meteor: An automatic metric for mt evaluation with improved correlation with human judgments](https://md.research.elamur.ai/free/en/papers/banerjee-2005-meteor-automatic-metric-mt-evaluation/). In Proceedings of the ACL Workshop on intrinsic and extrinsic evaluation measures for machine translation and/or summarization, pages 65–72.
4. Bhuwan Dhingra, Hanxiao Liu, Zhilin Yang, William Cohen, and Ruslan Salakhutdinov. 2017. Gated-attention readers for text comprehension. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), volume 1, pages 1832–1846. [find](https://scholar.google.com/scholar?q=Bhuwan+Dhingra%2C+Hanxiao+Liu%2C+Zhilin+Yang%2C+William+Cohen%2C+and+Ruslan+Salakhutdinov%2E+2017%2E+Gated%2Dattention+readers+for+text+comprehension%2E+In+Proceedings+of+the+55th+Annual+Meeting+of+the+Association+fo)
5. Junwei Bao, Nan Duan, Zhao Yan, Ming Zhou, and Tiejun Zhao. 2016. [Constraint-based question answering with knowledge graph](https://md.research.elamur.ai/free/en/papers/bao-2016-constraint-based-question-answering-knowledge/). In Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers, pages 2503–2514.
6. Bradley Efron and Robert J Tibshirani. 1994. An introduction to the bootstrap. CRC press. [find](https://scholar.google.com/scholar?q=Bradley+Efron+and+Robert+J+Tibshirani%2E+1994%2E+An+introduction+to+the+bootstrap%2E+CRC+press%2E)
7. Kurt Bollacker, Colin Evans, Praveen Paritosh, Tim Sturge, and Jamie Taylor. 2008. Freebase: a collaboratively created graph database for structuring human knowledge. In Proceedings of the 2008 ACM SIGMOD international conference on Management of data, pages 1247–1250. AcM. [find](https://scholar.google.com/scholar?q=Kurt+Bollacker%2C+Colin+Evans%2C+Praveen+Paritosh%2C+Tim+Sturge%2C+and+Jamie+Taylor%2E+2008%2E+Freebase%3A+a+collaboratively+created+graph+database+for+structuring+human+knowledge%2E+In+Proceedings+of+the+2008+ACM+SI)
8. Antoine Bordes, Sumit Chopra, and Jason Weston. 2014. Question answering with subgraph embeddings. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 615–620. [find](https://scholar.google.com/scholar?q=Antoine+Bordes%2C+Sumit+Chopra%2C+and+Jason+Weston%2E+2014%2E+Question+answering+with+subgraph+embeddings%2E+In+Proceedings+of+the+2014+Conference+on+Empirical+Methods+in+Natural+Language+Processing+%28EMNLP%29%2C+pa)
9. Gerlof Bouma. 2009. Normalized (pointwise) mutual information in collocation extraction. Proceedings of GSCL, pages 31–40. [find](https://scholar.google.com/scholar?q=Gerlof+Bouma%2E+2009%2E+Normalized+%28pointwise%29+mutual+information+in+collocation+extraction%2E+Proceedings+of+GSCL%2C+pages+31%E2%80%9340%2E)
10. Erik Cambria, Amir Hussain, Tariq Durrani, Catherine Havasi, Chris Eckl, and James Munro. 2010. Sentic computing for patient centered applications. In Signal Processing (ICSP), 2010 IEEE 10th International Conference on, pages 1279–1282. IEEE. [find](https://scholar.google.com/scholar?q=Erik+Cambria%2C+Amir+Hussain%2C+Tariq+Durrani%2C+Catherine+Havasi%2C+Chris+Eckl%2C+and+James+Munro%2E+2010%2E+Sentic+computing+for+patient+centered+applications%2E+In+Signal+Processing+%28ICSP%29%2C+2010+IEEE+10th+Internat)
11. Qian Chen, Xiaodan Zhu, Zhen-Hua Ling, Diana Inkpen, and Si Wei. 2018. [Neural natural language inference models enhanced with external knowledge](https://md.research.elamur.ai/free/en/papers/chen-2018-neural-natural-language-inference-models/). In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 2406–2417. Association for Compu…
12. Jianpeng Cheng, Li Dong, and Mirella Lapata. 2016. [Long short-term memory-networks for machine reading](https://md.research.elamur.ai/free/en/papers/cheng-2016-long-short-term-memory-networks/). In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 551–561.
13. Christopher Clark and Matt Gardner. 2018. [Simple and effective multi-paragraph reading comprehension](https://md.research.elamur.ai/free/en/papers/clark-2017-simple-effective-multi-paragraph-reading/). In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Long Papers), pages 845–855.
14. Bhuwan Dhingra, Qiao Jin, Zhilin Yang, William Cohen, and Ruslan Salakhutdinov. 2018. [Neural models for reasoning over multiple mentions using coreference](https://md.research.elamur.ai/free/en/papers/dhingra-2018-neural-models-reasoning-over-multiple/). In Proceedings of the 2018 Conference of Marjan Ghazvininejad, Chris Brockett, Ming-Wei Chang, Bill Dolan, Jianfeng Gao, Wen-tau Yih, and Michel…
15. Karl Moritz Hermann, Tomas Kocisky, Edward Grefenstette, Lasse Espeholt, Will Kay, Mustafa Suleyman, and Phil Blunsom. 2015. [Teaching machines to read and comprehend](https://md.research.elamur.ai/free/en/papers/hermann-2015-teaching-machines-read-comprehend/). In Advances in Neural Information Processing Systems, pages 1693– 1701.
16. Rudolf Kadlec, Martin Schmid, Ondrej Bajgar, and Jan Kleindienst. 2016. [Text understanding with the attention sum reader network](https://md.research.elamur.ai/free/en/papers/kadlec-2016-text-understanding-attention-sum-reader/). In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, pages 908–918.
17. Tomáš Kočiskỳ, Jonathan Schwarz, Phil Blunsom, Chris Dyer, Karl Moritz Hermann, Gáabor Melis, and Edward Grefenstette. 2018. [The narrativeqa reading comprehension challenge](https://md.research.elamur.ai/free/en/papers/kocisky-2018-narrativeqa-reading-comprehension-challenge/). Transactions of the Association of Computational Linguistics, 6:317–328.
18. J Richard Landis and Gary G Koch. 1977. The measurement of observer agreement for categorical data. biometrics, pages 159–174. [find](https://scholar.google.com/scholar?q=J+Richard+Landis+and+Gary+G+Koch%2E+1977%2E+The+measurement+of+observer+agreement+for+categorical+data%2E+biometrics%2C+pages+159%E2%80%93174%2E)
19. Chin-Yew Lin. 2004. [Rouge: A package for automatic evaluation of summaries](https://md.research.elamur.ai/free/en/papers/lin-2004-rouge-package-automatic-evaluation-summaries/). Text Summarization Branches Out.
20. Todor Mihaylov and Anette Frank. 2018. [Knowledgeable reader: enhancing cloze-style reading comprehension with external commonsense knowledge](https://md.research.elamur.ai/free/en/papers/mihaylov-2018-knowledgeable-reader-enhancing-cloze-style/). In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 821–832. Association for Computation…
21. Eric W Noreen. 1989. Computer-intensive methods for testing hypotheses. Wiley New York. [find](https://scholar.google.com/scholar?q=Eric+W+Noreen%2E+1989%2E+Computer%2Dintensive+methods+for+testing+hypotheses%2E+Wiley+New+York%2E)
22. Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. [Bleu: a method for automatic evaluation of machine translation](https://md.research.elamur.ai/free/en/papers/papineni-2002-bleu-method-automatic-evaluation-machine/). In Proceedings of the 40th Annual Meeting on Association for Computational Linguistics, pages 311–318. Association for Computational Linguistics.
23. Matthew E Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke Zettlemoyer. 2018. [Deep contextualized word representations](https://md.research.elamur.ai/free/en/papers/peters-2018-deep-contextualized-word-representations/). In Proceedings of NAACL.
24. Soujanya Poria, Erik Cambria, Alexander Gelbukh, Federica Bisio, and Amir Hussain. 2015. Sentiment data flow analysis by means of dynamic linguistic patterns. IEEE Computational Intelligence Magazine, 10(4):26–36. [find](https://scholar.google.com/scholar?q=Soujanya+Poria%2C+Erik+Cambria%2C+Alexander+Gelbukh%2C+Federica+Bisio%2C+and+Amir+Hussain%2E+2015%2E+Sentiment+data+flow+analysis+by+means+of+dynamic+linguistic+patterns%2E+IEEE+Computational+Intelligence+Magazine%2C)
25. Johannes Welbl, Pontus Stenetorp, and Sebastian Riedel. 2018. [Constructing datasets for multi-hop reading comprehension across documents](https://md.research.elamur.ai/free/en/papers/welbl-2018-constructing-datasets-multi-hop-reading/). Transactions of the Association of Computational Linguistics, 6:287–302.
26. Jason Weston, Antoine Bordes, Sumit Chopra, Alexander M Rush, Bart van Merriënboer, Armand Joulin, and Tomas Mikolov. 2016. [Towards ai-complete question answering: A set of prerequisite toy tasks](https://md.research.elamur.ai/free/en/papers/weston-2015-towards-ai-complete-question-answering/). In ICLR.
27. Soujanya Poria, Iti Chaturvedi, Erik Cambria, and Federica Bisio. 2016. Sentic LDA: Improving on LDA with semantic similarity for aspect-based sentiment analysis. In Neural Networks (IJCNN), 2016 International Joint Conference on, pages 4465–4473. IEEE. [find](https://scholar.google.com/scholar?q=Soujanya+Poria%2C+Iti+Chaturvedi%2C+Erik+Cambria%2C+and+Federica+Bisio%2E+2016%2E+Sentic+LDA%3A+Improving+on+LDA+with+semantic+similarity+for+aspect%2Dbased+sentiment+analysis%2E+In+Neural+Networks+%28IJCNN%29%2C+2016+Inte)
28. Zhen Xu, Bingquan Liu, Baoxun Wang, Chengjie Sun, and Xiaolong Wang. 2017. Incorporating loose-structured knowledge into LSTM with recall gate for conversation modeling. In International Joint Conference on Neural Networks. [find](https://scholar.google.com/scholar?q=Zhen+Xu%2C+Bingquan+Liu%2C+Baoxun+Wang%2C+Chengjie+Sun%2C+and+Xiaolong+Wang%2E+2017%2E+Incorporating+loose%2Dstructured+knowledge+into+LSTM+with+recall+gate+for+conversation+modeling%2E+In+International+Joint+Confere)
29. Pranav Rajpurkar, Jian Zhang, Konstantin Lopyrev, and Percy Liang. 2016. [Squad: 100,000+ questions for machine comprehension of text](https://md.research.elamur.ai/free/en/papers/rajpurkar-2016-squad-100-000-questions-machine/). In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 2383–2392.
30. Tom Young, Erik Cambria, Iti Chaturvedi, Minlie Huang, Hao Zhou, and Subham Biswas. 2018. Augmenting end-to-end dialog systems with commonsense knowledge. In Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence (AAAI-18). [find](https://scholar.google.com/scholar?q=Tom+Young%2C+Erik+Cambria%2C+Iti+Chaturvedi%2C+Minlie+Huang%2C+Hao+Zhou%2C+and+Subham+Biswas%2E+2018%2E+Augmenting+end%2Dto%2Dend+dialog+systems+with+commonsense+knowledge%2E+In+Proceedings+of+the+Thirty%2DSecond+AAAI+Conf)
31. Abigail See, Peter J Liu, and Christopher D Manning. 2017. Get to the point: Summarization with pointer-generator networks. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), volume 1, pages 1073–1083. [find](https://scholar.google.com/scholar?q=Abigail+See%2C+Peter+J+Liu%2C+and+Christopher+D+Manning%2E+2017%2E+Get+to+the+point%3A+Summarization+with+pointer%2Dgenerator+networks%2E+In+Proceedings+of+the+55th+Annual+Meeting+of+the+Association+for+Computation)
32. Minjoon Seo, Aniruddha Kembhavi, Ali Farhadi, and Hannaneh Hajishirzi. 2017. [Bidirectional attention flow for machine comprehension](https://md.research.elamur.ai/free/en/papers/seo-2016-bidirectional-attention-flow-machine-comprehension/). In ICLR.
33. Robert Speer and Catherine Havasi. 2012. Representing general relational knowledge in ConceptNet 5. In LREC, pages 3679–3686. [find](https://scholar.google.com/scholar?q=Robert+Speer+and+Catherine+Havasi%2E+2012%2E+Representing+general+relational+knowledge+in+ConceptNet+5%2E+In+LREC%2C+pages+3679%E2%80%933686%2E)
34. Sainbayar Sukhbaatar, Jason Weston, Rob Fergus, et al. 2015. End-to-end memory networks. In Advances in Neural Information Processing Systems, pages 2440–2448. [find](https://scholar.google.com/scholar?q=Sainbayar+Sukhbaatar%2C+Jason+Weston%2C+Rob+Fergus%2C+et+al%2E+2015%2E+End%2Dto%2Dend+memory+networks%2E+In+Advances+in+Neural+Information+Processing+Systems%2C+pages+2440%E2%80%932448%2E)
35. Yi Tay, Luu Anh Tuan, and Siu Cheung Hui. 2018. Multi-range reasoning for machine comprehension. arXiv preprint arXiv:1803.09074. — [arXiv:1803.09074](https://arxiv.org/abs/1803.09074) [find](https://scholar.google.com/scholar?q=Yi+Tay%2C+Luu+Anh+Tuan%2C+and+Siu+Cheung+Hui%2E+2018%2E+Multi%2Drange+reasoning+for+machine+comprehension%2E+arXiv+preprint+arXiv%3A1803%2E09074%2E+%E2%80%94+%5BarXiv%3A1803%2E09074%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F1803%2E09074%29)
36. Ramakrishna Vedantam, C Lawrence Zitnick, and Devi Parikh. 2015. [Cider: Consensus-based image description evaluation](https://md.research.elamur.ai/free/en/papers/vedantam-2014-cider-consensus-based-image-description/). In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 4566–4575.
37. Qiu-Feng Wang, Erik Cambria, Cheng-Lin Liu, and Amir Hussain. 2013. Common sense knowledge for handwritten chinese text recognition. Cognitive Computation, 5(2):234–242. [find](https://scholar.google.com/scholar?q=Qiu%2DFeng+Wang%2C+Erik+Cambria%2C+Cheng%2DLin+Liu%2C+and+Amir+Hussain%2E+2013%2E+Common+sense+knowledge+for+handwritten+chinese+text+recognition%2E+Cognitive+Computation%2C+5%282%29%3A234%E2%80%93242%2E)
38. Dirk Weissenborn, Tomáš Kočiskỳ, and Chris Dyer. 2017. Dynamic integration of background knowledge in neural NLU systems. arXiv preprint arXiv:1706.02596. — [arXiv:1706.02596](https://arxiv.org/abs/1706.02596) [find](https://scholar.google.com/scholar?q=Dirk+Weissenborn%2C+Toma%CC%81s%CC%8C+Koc%CC%8Cisky%CC%80%2C+and+Chris+Dyer%2E+2017%2E+Dynamic+integration+of+background+knowledge+in+neural+NLU+systems%2E+arXiv+preprint+arXiv%3A1706%2E02596%2E+%E2%80%94+%5BarXiv%3A1706%2E02596%5D%28https%3A%2F%2Farxiv%2Eorg%2Fab)

*Updated 24 October 2026*