---
title: "Avoiding Reasoning Shortcuts — paper by Jiang and Bansal (2019)"
description: "To expose this, they introduce ADDDOC, an adversary that copies the answer-bearing supporting document and alters… 34 references, cited by 57 papers here."
authors: ["Yichen Jiang", "Mohit Bansal"]
year: 2019
venue: "Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics"
source: "https://aclanthology.org/P19-1262/"
doi: "10.18653/v1/p19-1262"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/jiang-2019-avoiding-reasoning-shortcuts-adversarial-evaluation/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/jiang-2019-avoiding-reasoning-shortcuts-adversarial-evaluation/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2019](https://md.research.elamur.ai/free/en/papers/2019/page/4/) / Avoiding Reasoning Shortcuts: Adversarial Evaluation, Training, and Model Development for Multi-Hop QA

# Avoiding Reasoning Shortcuts: Adversarial Evaluation, Training, and Model Development for Multi-Hop QA

**Authors:** [Yichen Jiang](https://md.research.elamur.ai/free/en/authors/yichen-jiang/) and [Mohit Bansal](https://md.research.elamur.ai/free/en/authors/mohit-bansal/) · **Year:** 2019 · **Venue:** [Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics](https://md.research.elamur.ai/free/en/venues/acl/) ([ACL 2019](https://md.research.elamur.ai/free/en/venues/acl/2019/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/P19-1262/) · [PDF](https://aclanthology.org/P19-1262.pdf) · [DOI 10.18653/v1/p19-1262](https://doi.org/10.18653/v1/p19-1262) · [OpenAlex](https://openalex.org/W2951328433)

**Citations:** 83 citations (OpenAlex, 2026-10-14) · [57](https://md.research.elamur.ai/free/en/papers/jiang-2019-avoiding-reasoning-shortcuts-adversarial-evaluation/cited-by/) in our library, of the 141,840 papers whose bibliographies we read

## Summary
Multi-hop QA benchmarks are meant to test whether a system can chain several pieces of evidence. Jiang and Bansal show that many HotpotQA questions can be solved by matching question keywords against a single sentence, skipping the intermediate hop entirely. To expose this, they introduce ADDDOC, an adversary that copies the answer-bearing supporting document and alters its title and answer span. The result is a plausible distractor that offers a conflicting answer along the shortcut path while the true answer stays valid. From this they build adversarial dev and training sets (adv-dev, adv-train) for the HotpotQA distractor setting.

The strong Bi-attention + Self-attention baseline loses a large share of its EM score on adv-dev. Adversarial training and supporting-fact supervision help but leave a clear gap. The authors then add a recurrent control unit, adapted from Hudson and Manning (2018), which shifts attention across question words at each reasoning hop. This 2-hop model is more robust even with ordinary training, and after adversarial training it surpasses the adversarially trained baseline. The work exposes dataset artifacts and argues for pairing compositional architectures with adversarial data.

## Key points
- HotpotQA has about 113k crowd-written questions. Each context holds 10 Wikipedia documents: 2 supporting ones plus 8 distractors chosen by bigram TF-IDF similarity to the question.
- In a manual check of 50 randomly sampled bridge-type dev questions, 26 could be answered through a single-hop reasoning shortcut.
- Comparison questions (roughly 21% of the dev set) were left unchanged, since they inherently require aggregating information from two documents.
- ADDDOC builds fake answers from close substitutes (top-10 nearest candidates) for answer tokens. It also replaces titles with sampled ones, so the shortcut now points to two competing answers.
- Supervision on supporting facts noticeably improves robustness. Code and data are released at github.com/jiangycTarheel/Adversarial-MultiHopQA.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [University of North Carolina at Chapel Hill](https://md.research.elamur.ai/free/en/universities/university-of-north-carolina-at-chapel-hill/) — University, Chapel Hill, United States · 37 works · 14,332 citations

📚 Shelves (5): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 9,086 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 8,889 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 9,296 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 9,206 works · [👁️ Computer Vision and Pattern Recognition](https://md.research.elamur.ai/free/en/shelves/computer-science/computer-vision-and-pattern-recognition/) · 1,282 works

🧭 Topics (2): [🧮 Reasoning](https://md.research.elamur.ai/free/en/topics/reasoning/) · 510 works · [❓ Question answering](https://md.research.elamur.ai/free/en/topics/question-answering/) · 242 works

## Cite

Cite as: Yichen Jiang and Mohit Bansal. 2019. Avoiding Reasoning Shortcuts: Adversarial Evaluation, Training, and Model Development for Multi-Hop QA. Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{jiang2019avoiding,
  title = {{Avoiding Reasoning Shortcuts: Adversarial Evaluation, Training, and Model Development for Multi-Hop QA}},
  author = {Yichen Jiang and Mohit Bansal},
  booktitle = {Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics},
  year = {2019},
  doi = {10.18653/v1/p19-1262},
  url = {https://doi.org/10.18653/v1/p19-1262}
}
```

- [← previous Vecalign: Improved Sentence Alignment in Linear Time and Space](https://md.research.elamur.ai/free/en/papers/thompson-2019-vecalign-improved-sentence-alignment-linear/)
- [next → Investigating Entity Knowledge in BERT with Simple Neural End-To-End Entity Linking](https://md.research.elamur.ai/free/en/papers/broscheit-2019-investigating-entity-knowledge-bert-simple/)
- [↑ back to papers of 2019](https://md.research.elamur.ai/free/en/papers/2019/page/4/)

## References

34 references, 17 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Danqi Chen, Adam Fisch, Jason Weston, and Antoine Bordes. 2017. Reading wikipedia to answer open-domain questions. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). [our card](https://md.research.elamur.ai/free/en/papers/chen-2017-reading-wikipedia-answer-open-domain/)
2. Christopher Clark and Matt Gardner. 2018. Simple and effective multi-paragraph reading comprehension. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). [our card](https://md.research.elamur.ai/free/en/papers/clark-2017-simple-effective-multi-paragraph-reading/)
3. Bradley Efron and Robert J Tibshirani. 1994. An introduction to the bootstrap. CRC press. [find](https://scholar.google.com/scholar?q=Bradley+Efron+and+Robert+J+Tibshirani%2E+1994%2E+An+introduction+to+the+bootstrap%2E+CRC+press%2E)
4. Karl Moritz Hermann, Tomas Kocisky, Edward Grefenstette, Lasse Espeholt, Will Kay, Mustafa Suleyman, and Phil Blunsom. 2015. Teaching machines to read and comprehend. In Advances in Neural Information Processing Systems, pages 1693– 1701. [our card](https://md.research.elamur.ai/free/en/papers/hermann-2015-teaching-machines-read-comprehend/)
5. Sepp Hochreiter and Jürgen Schmidhuber. 1997. Long short-term memory. Neural computation, 9(8):1735–1780. [find](https://scholar.google.com/scholar?q=Sepp+Hochreiter+and+Ju%CC%88rgen+Schmidhuber%2E+1997%2E+Long+short%2Dterm+memory%2E+Neural+computation%2C+9%288%29%3A1735%E2%80%931780%2E)
6. Minghao Hu, Yuxing Peng, Zhen Huang, Xipeng Qiu, Furu Wei, and Ming Zhou. 2018. Reinforced mnemonic reader for machine reading comprehension. In IJCAI. [find](https://scholar.google.com/scholar?q=Minghao+Hu%2C+Yuxing+Peng%2C+Zhen+Huang%2C+Xipeng+Qiu%2C+Furu+Wei%2C+and+Ming+Zhou%2E+2018%2E+Reinforced+mnemonic+reader+for+machine+reading+comprehension%2E+In+IJCAI%2E)
7. Hsin-Yuan Huang, Chenguang Zhu, Yelong Shen, and Weizhu Chen. 2018. Fusionnet: Fusing via fully-aware attention with application to machine comprehension. In ICLR. [find](https://scholar.google.com/scholar?q=Hsin%2DYuan+Huang%2C+Chenguang+Zhu%2C+Yelong+Shen%2C+and+Weizhu+Chen%2E+2018%2E+Fusionnet%3A+Fusing+via+fully%2Daware+attention+with+application+to+machine+comprehension%2E+In+ICLR%2E)
8. Drew A Hudson and Christopher D Manning. 2018. Compositional attention networks for machine reasoning. In Proceedings of ICLR. [find](https://scholar.google.com/scholar?q=Drew+A+Hudson+and+Christopher+D+Manning%2E+2018%2E+Compositional+attention+networks+for+machine+reasoning%2E+In+Proceedings+of+ICLR%2E)
9. Sarthak Jain. 2016. Question answering over knowledge base using factual memory networks. In Proceedings of the NAACL Student Research Workshop. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Sarthak+Jain%2E+2016%2E+Question+answering+over+knowledge+base+using+factual+memory+networks%2E+In+Proceedings+of+the+NAACL+Student+Research+Workshop%2E+Association+for+Computational+Linguistics%2E)
10. Robin Jia and Percy Liang. 2017. Adversarial examples for evaluating reading comprehension systems. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP). [our card](https://md.research.elamur.ai/free/en/papers/jia-2017-adversarial-examples-evaluating-reading-comprehension/)
11. Mandar Joshi, Eunsol Choi, Daniel S Weld, and Luke Zettlemoyer. 2017. Triviaqa: A large scale distantly supervised challenge dataset for reading comprehension. arXiv preprint arXiv:1705.03551. [our card](https://md.research.elamur.ai/free/en/papers/joshi-2017-triviaqa-large-scale-distantly-supervised/)
12. Diederik P. Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. CoRR. [find](https://scholar.google.com/scholar?q=Diederik+P%2E+Kingma+and+Jimmy+Ba%2E+2014%2E+Adam%3A+A+method+for+stochastic+optimization%2E+CoRR%2E)
13. Xi Victoria Lin, Richard Socher, and Caiming Xiong. 2018. Multi-hop knowledge graph reasoning with reward shaping. In EMNLP. [find](https://scholar.google.com/scholar?q=Xi+Victoria+Lin%2C+Richard+Socher%2C+and+Caiming+Xiong%2E+2018%2E+Multi%2Dhop+knowledge+graph+reasoning+with+reward+shaping%2E+In+EMNLP%2E)
14. Tri Nguyen, Mir Rosenberg, Xia Song, Jianfeng Gao, Saurabh Tiwary, Rangan Majumder, and Li Deng. 2016. Ms marco: A human generated machine reading comprehension dataset. arXiv preprint arXiv:1611.09268. [our card](https://md.research.elamur.ai/free/en/papers/bajaj-2016-ms-marco-human-generated/)
15. Jeffrey Pennington, Richard Socher, and Christopher D Manning. 2014. Glove: Global vectors for word representation. In Conference on Empirical Methods in Natural Language Processing (EMNLP). [our card](https://md.research.elamur.ai/free/en/papers/pennington-2014-glove-global-vectors-word-representation/)
16. P. Rajpurkar, R. Jia, and P. Liang. 2018. Know what you don’t know: Unanswerable questions for SQuAD. In Association for Computational Linguistics (ACL). [our card](https://md.research.elamur.ai/free/en/papers/rajpurkar-2018-know-what-you-don-t/)
17. Pranav Rajpurkar, Jian Zhang, Konstantin Lopyrev, and Percy Liang. 2016. Squad: 100,000+ questions for machine comprehension of text. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP). [our card](https://md.research.elamur.ai/free/en/papers/rajpurkar-2016-squad-100-000-questions-machine/)
18. Minjoon Seo, Aniruddha Kembhavi, Ali Farhadi, and Hannaneh Hajishirzi. 2017. Bidirectional attention flow for machine comprehension. In International Conference on Learning Representations (ICLR). [our card](https://md.research.elamur.ai/free/en/papers/seo-2016-bidirectional-attention-flow-machine-comprehension/)
19. Rupesh Kumar Srivastava, Klaus Greff, and Jürgen Schmidhuber. 2015. Highway networks. In International Conference on Machine Learning (ICML). [our card](https://md.research.elamur.ai/free/en/papers/srivastava-2015-highway-networks/)
20. Wenhui Wang, Nan Yang, Furu Wei, Baobao Chang, and Ming Zhou. 2017. Gated self-matching networks for reading comprehension and question answering. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), volume 1, pages 189–198. [our card](https://md.research.elamur.ai/free/en/papers/wang-2017-gated-self-matching-networks-reading/)
21. Yicheng Wang and Mohit Bansal. 2018. Robust machine comprehension models via adversarial training. In NAACL. [find](https://scholar.google.com/scholar?q=Yicheng+Wang+and+Mohit+Bansal%2E+2018%2E+Robust+machine+comprehension+models+via+adversarial+training%2E+In+NAACL%2E)
22. Johannes Welbl, Pontus Stenetorp, and Sebastian Riedel. 2017. Constructing datasets for multi-hop reading comprehension across documents. In TACL. [our card](https://md.research.elamur.ai/free/en/papers/welbl-2018-constructing-datasets-multi-hop-reading/)
23. Jason Weston, Antoine Bordes, Sumit Chopra, Alexander M Rush, Bart van Merriënboer, Armand Joulin, and Tomas Mikolov. 2016. Towards ai-complete question answering: A set of prerequisite toy tasks. In ICLR. [our card](https://md.research.elamur.ai/free/en/papers/weston-2015-towards-ai-complete-question-answering/)
24. Caiming Xiong, Victor Zhong, and Richard Socher. 2017. Dynamic coattention networks for question answering. In ICLR. [our card](https://md.research.elamur.ai/free/en/papers/xiong-2016-dynamic-coattention-networks-question-answering/)
25. Yi Yang, Wen-tau Yih, and Christopher Meek. 2015. Wikiqa: A challenge dataset for open-domain question answering. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 2013–2018. [our card](https://md.research.elamur.ai/free/en/papers/yang-2015-wikiqa-challenge-dataset-open-domain/)
26. Zhilin Yang, Peng Qi, Saizheng Zhang, Yoshua Bengio, William W Cohen, Ruslan Salakhutdinov, and Christopher D Manning. 2018. Hotpotqa: A dataset for diverse, explainable multi-hop question answering. In Conference on Empirical Methods in Natural Language Processing (EMNLP). [our card](https://md.research.elamur.ai/free/en/papers/yang-2018-hotpotqa-dataset-diverse-explainable-multi/)
27. Pengcheng Yin, Zhengdong Lu, Hang Li, and Ben Kao. 2015. Neural enquirer: Learning to query tables. arXiv preprint. [find](https://scholar.google.com/scholar?q=Pengcheng+Yin%2C+Zhengdong+Lu%2C+Hang+Li%2C+and+Ben+Kao%2E+2015%2E+Neural+enquirer%3A+Learning+to+query+tables%2E+arXiv+preprint%2E)
28. Mantong Zhou, Minlie Huang, and Xiaoyan Zhu. 2018. An interpretable reasoning network for multi-relation question answering. In Proceedings of the 27th International Conference on Computational Linguistics. [find](https://scholar.google.com/scholar?q=Mantong+Zhou%2C+Minlie+Huang%2C+and+Xiaoyan+Zhu%2E+2018%2E+An+interpretable+reasoning+network+for+multi%2Drelation+question+answering%2E+In+Proceedings+of+the+27th+International+Conference+on+Computational+Lingui)
29. Question Golden Reasoning Chain Docs Distractor Docs Adversarial Doc Who produced the film that was Jennifer Kent 's directorial debut ? Jennifer Kent is an Australian actress, writer and director, best known for her horror film "The Babadook" (2014), which was her directorial debut. She is currentl… [find](https://scholar.google.com/scholar?q=Question+Golden+Reasoning+Chain+Docs+Distractor+Docs+Adversarial+Doc+Who+produced+the+film+that+was+Jennifer+Kent+%27s+directorial+debut+%3F+Jennifer+Kent+is+an+Australian+actress%2C+writer+and+director%2C+be)
30. The Babadook is a 2014 Australian psychological horror film written and directed by Jennifer Kent in her directorial debut , and produced byKristina Ceyton and Kristian Moliere . The film stars Essie Davis, Noah Wiseman, Daniel Henshall, Hayley McElhinney, Barbara West, and Ben Winspear.', ' It is b… [find](https://scholar.google.com/scholar?q=The+Babadook+is+a+2014+Australian+psychological+horror+film+written+and+directed+by+Jennifer+Kent+in+her+directorial+debut+%2C+and+produced+byKristina+Ceyton+and+Kristian+Moliere+%2E+The+film+stars+Essie+)
31. You Can't Kill Stephen King is a 2012 American comedy horror film that was directed by Monroe Mann, Ronnie Khalil, and Jorge Valdés-Iga, and is the directorial debut of Khalil and the feature film directorial debut of Mann ... [find](https://scholar.google.com/scholar?q=You+Can%27t+Kill+Stephen+King+is+a+2012+American+comedy+horror+film+that+was+directed+by+Monroe+Mann%2C+Ronnie+Khalil%2C+and+Jorge+Vald%C3%A9s%2DIga%2C+and+is+the+directorial+debut+of+Khalil+and+the+feature+film+dir)
32. The Iron Giant is a 1999 American animated science-fiction comedy-drama action film using both traditional animation and computer animation, produced by and directed by Brad Bird in his directorial debut. [find](https://scholar.google.com/scholar?q=The+Iron+Giant+is+a+1999+American+animated+science%2Dfiction+comedy%2Ddrama+action+film+using+both+traditional+animation+and+computer+animation%2C+produced+by+and+directed+by+Brad+Bird+in+his+directorial+de)
33. The Aphra Behn is a 2014 Australian psychological horror film written and directed by Scott Hahn in her directorial debut, and produced by Kristina Mutrux and Kristian ionesco. ... [find](https://scholar.google.com/scholar?q=The+Aphra+Behn+is+a+2014+Australian+psychological+horror+film+written+and+directed+by+Scott+Hahn+in+her+directorial+debut%2C+and+produced+by+Kristina+Mutrux+and+Kristian+ionesco%2E+%2E%2E%2E)
34. Prediction : Kristina Ceyton and Kristian Moliere Prediction under adversary : Kristina Ceyton and Kristian Moliere *Figure 4: A single-hop HotpotQA example that cannot be fixed with our adversary.* [find](https://scholar.google.com/scholar?q=Prediction+%3A+Kristina+Ceyton+and+Kristian+Moliere+Prediction+under+adversary+%3A+Kristina+Ceyton+and+Kristian+Moliere+%2AFigure+4%3A+A+single%2Dhop+HotpotQA+example+that+cannot+be+fixed+with+our+adversary%2E%2A)

*Updated 24 October 2026*