---
title: "Gated-Attention Readers for Text Comprehension — Dhingra 2016"
description: "The authors propose the Gated-Attention (GA) Reader, which combines two ideas that earlier work… 32 references, cited by 107 papers here."
authors: ["Bhuwan Dhingra", "Hanxiao Liu", "Zhilin Yang", "William W. Cohen", "Ruslan Salakhutdinov"]
year: 2016
venue: "arXiv:1606.01549"
source: "https://arxiv.org/abs/1606.01549"
doi: "10.48550/arxiv.1606.01549"
arxiv: "1606.01549"
licence: "arXiv — non-exclusive license to distribute"
licence_url: "http://arxiv.org/licenses/nonexclusive-distrib/1.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/dhingra-2016-gated-attention-readers-text-comprehension/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/dhingra-2016-gated-attention-readers-text-comprehension/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2016](https://md.research.elamur.ai/free/en/papers/2016/page/2/) / Gated-Attention Readers for Text Comprehension

# Gated-Attention Readers for Text Comprehension

**Authors:** [Bhuwan Dhingra](https://md.research.elamur.ai/free/en/authors/bhuwan-dhingra/), [Hanxiao Liu](https://md.research.elamur.ai/free/en/authors/hanxiao-liu/), [Zhilin Yang](https://md.research.elamur.ai/free/en/authors/zhilin-yang/), [William W. Cohen](https://md.research.elamur.ai/free/en/authors/william-w-cohen/) and [Ruslan Salakhutdinov](https://md.research.elamur.ai/free/en/authors/ruslan-salakhutdinov/) · **Year:** 2016 · **Venue:** [arXiv:1606.01549](https://md.research.elamur.ai/free/en/venues/acl/) ([ACL 2016](https://md.research.elamur.ai/free/en/venues/acl/2016/)) · **Licence:** [arXiv — non-exclusive license to distribute](http://arxiv.org/licenses/nonexclusive-distrib/1.0/)

**Links:** [Publisher's page](https://arxiv.org/abs/1606.01549) · [PDF](https://arxiv.org/pdf/1606.01549) · [DOI 10.48550/arxiv.1606.01549](https://doi.org/10.48550/arxiv.1606.01549) · [arXiv 1606.01549](https://arxiv.org/abs/1606.01549) · [OpenAlex](https://openalex.org/W2415755012)

**Citations:** 356 citations (OpenAlex, 2026-10-14) · [107](https://md.research.elamur.ai/free/en/papers/dhingra-2016-gated-attention-readers-text-comprehension/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
The paper addresses cloze-style reading comprehension: given a document and a query with one phrase blanked out, a model must choose the correct filler from candidate entities found in the document. The authors propose the Gated-Attention (GA) Reader, which combines two ideas that earlier work had developed separately: multi-hop reading and attention. The model stacks several layers of bidirectional GRUs over the document. At each layer, every document token attends over the query to build its own query summary, and that summary is multiplied element-wise with the token's hidden state. This acts as a filter that suppresses query-irrelevant features before the next pass. The final layer uses an attention-sum step, as in the AS Reader, to score candidates. Experiments on CNN, Daily Mail and Who Did What (WDW) show state-of-the-art accuracy, with gains above 4% over strong baselines. Ablations and statistical tests confirm that the gating itself drives much of the improvement. This makes the mechanism a candidate for other tasks where several information sources interact.

## Key points
- The GA module computes a token-specific query vector by softmax attention, then gates the document token by element-wise multiplication. On WDW, multiplication clearly beat addition and concatenation as the combining operator.
- With one hop (K = 1), the model reduces to the AS Reader without gating. Accuracy climbs steadily up to K = 3 and then levels off.
- In the WDW ablation (val/test), the full GA scored 68.3/68.0. Removing character embeddings dropped it to 66.9/66.9, and removing token-level query attention dropped it to 65.7/65.0. Replacing GloVe vectors (trained on about 6B tokens) with corpus-trained embeddings gave the largest drop, to 64.0/62.5.
- The benefit of gating was assessed across 50%, 75% and 100% training-set sizes using an exact one-sided McNemar test. The gains were mostly significant (p \< 0.01) on CNN, especially with more data.
- Attention visualizations suggest that different layers focus on different clue words in the query. Code was released at github.com/bdhingra/ga-reader.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Carnegie Mellon University](https://md.research.elamur.ai/free/en/universities/carnegie-mellon-university/) — University, Pittsburgh, United States · 598 works · 191,226 citations
- [Microsoft](https://md.research.elamur.ai/free/en/companies/microsoft/) — Company, Redmond, United States · 378 works · 161,630 citations

📚 Shelves (6): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 18,614 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 17,965 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 17,450 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 17,296 works · [👁️ Computer Vision and Pattern Recognition](https://md.research.elamur.ai/free/en/shelves/computer-science/computer-vision-and-pattern-recognition/) · 3,486 works · [🗄️ Information Systems](https://md.research.elamur.ai/free/en/shelves/computer-science/information-systems/) · 1,730 works

## Cite

Cite as: Bhuwan Dhingra, Hanxiao Liu, Zhilin Yang, William W. Cohen and Ruslan Salakhutdinov. 2016. Gated-Attention Readers for Text Comprehension. arXiv:1606.01549.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@misc{dhingra2016gated,
  title = {{Gated-Attention Readers for Text Comprehension}},
  author = {Bhuwan Dhingra, Hanxiao Liu, Zhilin Yang, William W. Cohen and Ruslan Salakhutdinov},
  year = {2016},
  doi = {10.48550/arxiv.1606.01549},
  eprint = {1606.01549},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.1606.01549}
}
```

- [← previous CLEVR: A Diagnostic Dataset for Compositional Language and Elementary Visual Reasoning](https://md.research.elamur.ai/free/en/papers/johnson-2016-clevr-diagnostic-dataset-compositional-language/)
- [next → A Persona-Based Neural Conversation Model](https://md.research.elamur.ai/free/en/papers/li-2016-persona-based-neural-conversation-model/)
- [↑ back to papers of 2016](https://md.research.elamur.ai/free/en/papers/2016/page/2/)

## References

32 references, 12 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. [Neural machine translation by jointly learning to align and translate](https://md.research.elamur.ai/free/en/papers/bahdanau-2014-neural-machine-translation-jointly-learning/). arXiv preprint arXiv:1409.0473, 2014.
2. Ondrej Bajgar, Rudolf Kadlec, and Jan Kleindienst. Embracing data abundance: Booktest dataset for reading comprehension. arXiv preprint arXiv:1610.00956, 2016. — [arXiv:1610.00956](https://arxiv.org/abs/1610.00956) [find](https://scholar.google.com/scholar?q=Ondrej+Bajgar%2C+Rudolf+Kadlec%2C+and+Jan+Kleindienst%2E+Embracing+data+abundance%3A+Booktest+dataset+for+reading+comprehension%2E+arXiv+preprint+arXiv%3A1610%2E00956%2C+2016%2E+%E2%80%94+%5BarXiv%3A1610%2E00956%5D%28https%3A%2F%2Farxiv%2Eorg%2Fa)
3. Danqi Chen, Jason Bolton, and Christopher D Manning. [A thorough examination of the cnn/daily mail reading comprehension task](https://md.research.elamur.ai/free/en/papers/chen-2016-thorough-examination-cnn-daily-mail/). ACL, 2016.
4. Kyunghyun Cho, Bart Van Merriënboer, Caglar Gulcehre, Dzmitry Bahdanau, Fethi Bougares, Holger Schwenk, and Yoshua Bengio. Learning phrase representations using rnn encoder-decoder for statistical machine translation. ACL, 2015. — [Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation](https://md.research.elamur.ai/free/en/papers/cho-2014-learning-phrase-representations-rnn-encoder/)
5. Yiming Cui, Zhipeng Chen, Si Wei, Shijin Wang, Ting Liu, and Guoping Hu. Attention-over-attention neural networks for reading comprehension. ACL, 2017. [find](https://scholar.google.com/scholar?q=Yiming+Cui%2C+Zhipeng+Chen%2C+Si+Wei%2C+Shijin+Wang%2C+Ting+Liu%2C+and+Guoping+Hu%2E+Attention%2Dover%2Dattention+neural+networks+for+reading+comprehension%2E+ACL%2C+2017%2E)
6. Bhuwan Dhingra, Zhong Zhou, Dylan Fitzpatrick, Michael Muehl, and William W Cohen. Tweet2vec: Character-based distributed representations for social media. ACL, 2016. [find](https://scholar.google.com/scholar?q=Bhuwan+Dhingra%2C+Zhong+Zhou%2C+Dylan+Fitzpatrick%2C+Michael+Muehl%2C+and+William+W+Cohen%2E+Tweet2vec%3A+Character%2Dbased+distributed+representations+for+social+media%2E+ACL%2C+2016%2E)
7. Karl Moritz Hermann, Tomas Kocisky, Edward Grefenstette, Lasse Espeholt, Will Kay, Mustafa Suleyman, and Phil Blunsom. [Teaching machines to read and comprehend](https://md.research.elamur.ai/free/en/papers/hermann-2015-teaching-machines-read-comprehend/). In Advances in Neural Information Processing Systems, pp. 1684–1692, 2015.
8. Felix Hill, Antoine Bordes, Sumit Chopra, and Jason Weston. The goldilocks principle: Reading children’s books with explicit memory representations. ICLR, 2016. — [The Goldilocks Principle: Reading Children's Books with Explicit Memory Representations](https://md.research.elamur.ai/free/en/papers/hill-2015-goldilocks-principle-reading-children-s/)
9. Sepp Hochreiter and Jürgen Schmidhuber. Long short-term memory. Neural computation, 9(8):1735– 1780, 1997. [find](https://scholar.google.com/scholar?q=Sepp+Hochreiter+and+Ju%CC%88rgen+Schmidhuber%2E+Long+short%2Dterm+memory%2E+Neural+computation%2C+9%288%29%3A1735%E2%80%93+1780%2C+1997%2E)
10. Rudolf Kadlec, Martin Schmid, Ondrej Bajgar, and Jan Kleindienst. [Text understanding with the attention sum reader network](https://md.research.elamur.ai/free/en/papers/kadlec-2016-text-understanding-attention-sum-reader/). ACL, 2016.
11. Diederik Kingma and Jimmy Ba. [Adam: A method for stochastic optimization](https://md.research.elamur.ai/free/en/papers/kingma-2014-adam-method-stochastic-optimization/). ICLR, 2015.
12. Ryan Kiros, Richard Zemel, and Ruslan R Salakhutdinov. A multiplicative model for learning distributed text-based attribute representations. In Advances in Neural Information Processing Systems, pp. 2348– 2356, 2014. [find](https://scholar.google.com/scholar?q=Ryan+Kiros%2C+Richard+Zemel%2C+and+Ruslan+R+Salakhutdinov%2E+A+multiplicative+model+for+learning+distributed+text%2Dbased+attribute+representations%2E+In+Advances+in+Neural+Information+Processing+Systems%2C+pp%2E+2)
13. Sosuke Kobayashi, Ran Tian, Naoaki Okazaki, and Kentaro Inui. Dynamic entity representations with max-pooling improves machine reading. In NAACL-HLT, 2016. [find](https://scholar.google.com/scholar?q=Sosuke+Kobayashi%2C+Ran+Tian%2C+Naoaki+Okazaki%2C+and+Kentaro+Inui%2E+Dynamic+entity+representations+with+max%2Dpooling+improves+machine+reading%2E+In+NAACL%2DHLT%2C+2016%2E)
14. Ankit Kumar, Ozan Irsoy, Jonathan Su, James Bradbury, Robert English, Brian Pierce, Peter Ondruska, Ishaan Gulrajani, and Richard Socher. [Ask me anything: Dynamic memory networks for natural language processing](https://md.research.elamur.ai/free/en/papers/kumar-2015-ask-me-anything-dynamic-memory/). ICML, 2016.
15. Peng Li, Wei Li, Zhengyan He, Xuguang Wang, Ying Cao, Jie Zhou, and Wei Xu. Dataset and neural recurrent sequence labeling model for open-domain factoid question answering. arXiv preprint arXiv:1607.06275, 2016. — [arXiv:1607.06275](https://arxiv.org/abs/1607.06275) [find](https://scholar.google.com/scholar?q=Peng+Li%2C+Wei+Li%2C+Zhengyan+He%2C+Xuguang+Wang%2C+Ying+Cao%2C+Jie+Zhou%2C+and+Wei+Xu%2E+Dataset+and+neural+recurrent+sequence+labeling+model+for+open%2Ddomain+factoid+question+answering%2E+arXiv+preprint+arXiv%3A1607%2E0)
16. Jeff Mitchell and Mirella Lapata. Vector-based models of semantic composition. In ACL, pp. 236–244, 2008. [find](https://scholar.google.com/scholar?q=Jeff+Mitchell+and+Mirella+Lapata%2E+Vector%2Dbased+models+of+semantic+composition%2E+In+ACL%2C+pp%2E+236%E2%80%93244%2C+2008%2E)
17. Volodymyr Mnih, Nicolas Heess, Alex Graves, et al. Recurrent models of visual attention. In Advances in Neural Information Processing Systems, pp. 2204– 2212, 2014. [find](https://scholar.google.com/scholar?q=Volodymyr+Mnih%2C+Nicolas+Heess%2C+Alex+Graves%2C+et+al%2E+Recurrent+models+of+visual+attention%2E+In+Advances+in+Neural+Information+Processing+Systems%2C+pp%2E+2204%E2%80%93+2212%2C+2014%2E)
18. Tsendsuren Munkhdalai and Hong Yu. Neural semantic encoders. EACL, 2017a. [find](https://scholar.google.com/scholar?q=Tsendsuren+Munkhdalai+and+Hong+Yu%2E+Neural+semantic+encoders%2E+EACL%2C+2017a%2E)
19. Tsendsuren Munkhdalai and Hong Yu. Reasoning with memory augmented neural networks for language comprehension. ICLR, 2017b. [find](https://scholar.google.com/scholar?q=Tsendsuren+Munkhdalai+and+Hong+Yu%2E+Reasoning+with+memory+augmented+neural+networks+for+language+comprehension%2E+ICLR%2C+2017b%2E)
20. Takeshi Onishi, Hai Wang, Mohit Bansal, Kevin Gimpel, and David McAllester. [Who did what: A large-scale person-centered cloze dataset](https://md.research.elamur.ai/free/en/papers/onishi-2016-who-did-what-large-scale/). EMNLP, 2016.
21. Razvan Pascanu, Tomas Mikolov, and Yoshua Bengio. On the difficulty of training recurrent neural networks. ICML (3), 28:1310–1318, 2013. [find](https://scholar.google.com/scholar?q=Razvan+Pascanu%2C+Tomas+Mikolov%2C+and+Yoshua+Bengio%2E+On+the+difficulty+of+training+recurrent+neural+networks%2E+ICML+%283%29%2C+28%3A1310%E2%80%931318%2C+2013%2E)
22. Jeffrey Pennington, Richard Socher, and Christopher D. Manning. [Glove: Global vectors for word representation](https://md.research.elamur.ai/free/en/papers/pennington-2014-glove-global-vectors-word-representation/). In Empirical Methods in Natural Language Processing (EMNLP), pp. 1532– 1543, 2014. URL [http://www.aclweb.org/anthology/D14-1162](http://www.aclweb.org/anthology/D14-1162). [↗](http://www.aclweb.org/anthology/D14-1162)
23. Minjoon Seo, Aniruddha Kembhavi, Ali Farhadi, and Hannaneh Hajishirzi. [Bidirectional attention flow for machine comprehension](https://md.research.elamur.ai/free/en/papers/seo-2016-bidirectional-attention-flow-machine-comprehension/). ICLR, 2017.
24. Yelong Shen, Po-Sen Huang, Jianfeng Gao, and Weizhu Chen. Reasonet: Learning to stop reading in machine comprehension. arXiv preprint arXiv:1609.05284, 2016. — [arXiv:1609.05284](https://arxiv.org/abs/1609.05284) [find](https://scholar.google.com/scholar?q=Yelong+Shen%2C+Po%2DSen+Huang%2C+Jianfeng+Gao%2C+and+Weizhu+Chen%2E+Reasonet%3A+Learning+to+stop+reading+in+machine+comprehension%2E+arXiv+preprint+arXiv%3A1609%2E05284%2C+2016%2E+%E2%80%94+%5BarXiv%3A1609%2E05284%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs)
25. Alessandro Sordoni, Phillip Bachman, and Yoshua Bengio. Iterative alternating neural attention for machine reading. arXiv preprint arXiv:1606.02245, 2016. — [arXiv:1606.02245](https://arxiv.org/abs/1606.02245) [find](https://scholar.google.com/scholar?q=Alessandro+Sordoni%2C+Phillip+Bachman%2C+and+Yoshua+Bengio%2E+Iterative+alternating+neural+attention+for+machine+reading%2E+arXiv+preprint+arXiv%3A1606%2E02245%2C+2016%2E+%E2%80%94+%5BarXiv%3A1606%2E02245%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F16)
26. Sainbayar Sukhbaatar, Jason Weston, Rob Fergus, et al. End-to-end memory networks. In Advances in Neural Information Processing Systems, pp. 2431–2439, 2015. [find](https://scholar.google.com/scholar?q=Sainbayar+Sukhbaatar%2C+Jason+Weston%2C+Rob+Fergus%2C+et+al%2E+End%2Dto%2Dend+memory+networks%2E+In+Advances+in+Neural+Information+Processing+Systems%2C+pp%2E+2431%E2%80%932439%2C+2015%2E)
27. Theano Development Team. [Theano: A Python framework for fast computation of mathematical expressions](https://md.research.elamur.ai/free/en/papers/the-theano-development-team-2016-theano-python-framework-fast-computation/). arXiv e-prints, abs/1605.02688, May 2016. URL http://arxiv.org/abs/1605.02688. [↗](http://arxiv.org/abs/1605.02688)
28. Adam Trischler, Zheng Ye, Xingdi Yuan, and Kaheer Suleman. Natural language comprehension with the epireader. EMNLP, 2016. [find](https://scholar.google.com/scholar?q=Adam+Trischler%2C+Zheng+Ye%2C+Xingdi+Yuan%2C+and+Kaheer+Suleman%2E+Natural+language+comprehension+with+the+epireader%2E+EMNLP%2C+2016%2E)
29. Jason Weston, Sumit Chopra, and Antoine Bordes. Memory networks. ICLR, 2015. [find](https://scholar.google.com/scholar?q=Jason+Weston%2C+Sumit+Chopra%2C+and+Antoine+Bordes%2E+Memory+networks%2E+ICLR%2C+2015%2E)
30. Yuhuai Wu, Saizheng Zhang, Ying Zhang, Yoshua Bengio, and Ruslan Salakhutdinov. On multiplicative integration with recurrent neural networks. Advances in Neural Information Processing Systems, 2016. [find](https://scholar.google.com/scholar?q=Yuhuai+Wu%2C+Saizheng+Zhang%2C+Ying+Zhang%2C+Yoshua+Bengio%2C+and+Ruslan+Salakhutdinov%2E+On+multiplicative+integration+with+recurrent+neural+networks%2E+Advances+in+Neural+Information+Processing+Systems%2C+2016%2E)
31. Bishan Yang, Wen-tau Yih, Xiaodong He, Jianfeng Gao, and Li Deng. Learning multi-relational semantics using neural-embedding models. NIPS Workshop on Learning Semantics, 2014. [find](https://scholar.google.com/scholar?q=Bishan+Yang%2C+Wen%2Dtau+Yih%2C+Xiaodong+He%2C+Jianfeng+Gao%2C+and+Li+Deng%2E+Learning+multi%2Drelational+semantics+using+neural%2Dembedding+models%2E+NIPS+Workshop+on+Learning+Semantics%2C+2014%2E)
32. Zhilin Yang, Ruslan Salakhutdinov, and William Cohen. Multi-task cross-lingual sequence tagging from scratch. arXiv preprint arXiv:1603.06270, 2016. [find](https://scholar.google.com/scholar?q=Zhilin+Yang%2C+Ruslan+Salakhutdinov%2C+and+William+Cohen%2E+Multi%2Dtask+cross%2Dlingual+sequence+tagging+from+scratch%2E+arXiv+preprint+arXiv%3A1603%2E06270%2C+2016%2E)

*Updated 2 November 2026*