---
title: "Recurrent Attention Network on Memory for Aspect — Chen 2017"
description: "The paper addresses aspect-level sentiment classification: deciding whether a review or comment is negative, neutral or positive towards one specific…"
authors: ["Peng Chen", "Zhongqian Sun", "Lidong Bing", "Wei Yang"]
year: 2017
venue: "Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing"
source: "https://aclanthology.org/D17-1047/"
doi: "10.18653/v1/d17-1047"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/chen-2017-recurrent-attention-network-memory-aspect/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/chen-2017-recurrent-attention-network-memory-aspect/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2017](https://md.research.elamur.ai/free/en/papers/2017/page/3/) / Recurrent Attention Network on Memory for Aspect Sentiment Analysis

# Recurrent Attention Network on Memory for Aspect Sentiment Analysis

**Authors:** [Peng Chen](https://md.research.elamur.ai/free/en/authors/peng-chen/), Zhongqian Sun, [Lidong Bing](https://md.research.elamur.ai/free/en/authors/lidong-bing/), [Wei Yang](https://md.research.elamur.ai/free/en/authors/wei-yang/) · **Year:** 2017 · **Venue:** Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/D17-1047/) · [PDF](https://aclanthology.org/D17-1047.pdf) · [DOI 10.18653/v1/d17-1047](https://doi.org/10.18653/v1/d17-1047) · [OpenAlex](https://openalex.org/W2757541972)

**Citations:** 995 citations (openalex-snapshot · 2026-10-14) · [118](#cited-by) in our library, of the 140884 papers whose bibliographies we read

## Summary
The paper addresses aspect-level sentiment classification: deciding whether a review or comment is negative, neutral or positive towards one specific opinion target, when a sentence may mention several targets with different polarities. The authors argue that TD-LSTM-style models lose sentiment cues that are far from the target, and that a single attention blurs several scattered cues into one average. Their model, RAM, has four main stages. A deep bidirectional LSTM turns word embeddings into a memory. That memory is reweighted by each word's distance to the target, so every target gets its own memory. Several attention passes then read from it, and a GRU combines their outputs non-linearly before a softmax classifier. They test RAM on SemEval 2014 laptop and restaurant reviews, a Twitter set, and a new Chinese news-comment dataset where people are the targets. RAM consistently beats the state-of-the-art baselines on all four. This suggests that recurrent multi-hop attention over a target-specific memory handles contrastive, comparative and long-distance constructions better.

## Key points
- RAM has five modules: input embeddings (e.g. GloVe or CBOW), a DBLSTM memory, a position-weighted memory, a recurrent attention module (several attention layers fused by a GRU), and a softmax output.
- Evaluation uses four datasets:
  - SemEval 2014 Laptop and Restaurant reviews,
  - tweets collected by Dong et al. (2014),
  - a Chinese news-comment corpus with person mentions as targets, built to check how sensitive the model is to language.
- In the embedding study (Table 4), frozen pre-trained embeddings (RAM-3AL-NT) scored best everywhere: Laptop 0.7449, Restaurant 0.8023, Tweet 0.6936, Comment 0.7389.
- Randomly initialised, tuned embeddings (RAM-3AL-T-R) were clearly worse, for example 0.5806 on Laptop. Fine-tuning pre-trained vectors (RAM-3AL-T) landed in between.
- The authors explain the embedding results by three factors: small training sets, many test words unseen in training, and overfitting once embedding parameters are added.

## Interesting ideas
- **Per-target memory through distance weighting:** Encode the sentence once, then scale each memory slot by its distance to the target. This gives each target its own view of the sentence cheaply, which suits any task with several entities per sentence, such as relation or stance classification.
- **Fusing attention hops with a GRU:** Feeding the output of each attention pass into a GRU lets the model keep, forget or transform evidence across hops. Simply summing the hops cannot do this, and the design could carry over to multi-hop reading or QA models.
- **Reading attention over contextual states:** Attending over BiLSTM states rather than raw word vectors lets one memory slot capture a phrase like "not wonderful enough". This is a useful choice wherever negation or intensifiers change meaning.
- **Freezing embeddings on small datasets:** Keeping pre-trained vectors fixed keeps unseen test words in the same space as training words. This is a practical default for low-resource settings, such as small Ukrainian corpora.
- **A simplified model for interpretation:** For visualisation, the authors drop the BLSTM so that attention weights map directly onto words. Interpreting a cut-down variant like this is a neat way to sanity-check what attention focuses on.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

Fields: [Natural language processing](https://md.research.elamur.ai/free/en/shelves/nlp/)

## Cited by

Cited by

118 papers of our library cite it (of the 140884 whose bibliographies we read); 10 of them have a page here, 108 not yet.

- [An Interactive Multi-Task Learning Network for End-to-End Aspect-Based Sentiment Analysis](https://md.research.elamur.ai/free/en/papers/he-2019-interactive-multi-task-learning-network/)
- [Knowledge Enhanced Masked Language Model for Stance Detection](https://md.research.elamur.ai/free/en/papers/kawintiranon-2021-knowledge-enhanced-masked-language-model/)
- [Transformation Networks for Target-Oriented Sentiment Classification](https://md.research.elamur.ai/free/en/papers/li-2018-transformation-networks-target-oriented-sentiment/)
- [Arabic aspect based sentiment analysis using bidirectional GRU based models](https://md.research.elamur.ai/free/en/papers/m-2021-abdelgwad-arabic-aspect-based-sentiment-analysis/)
- [Arabic aspect sentiment polarity classification using BERT](https://md.research.elamur.ai/free/en/papers/m-2021-abdelgwad-arabic-aspect-sentiment-polarity-classification/)
- [Aspect-Level Sentiment Analysis Via Convolution over Dependency Tree](https://md.research.elamur.ai/free/en/papers/sun-2019-aspect-level-sentiment-analysis-convolution/)
- [Reasoning with Sarcasm by Reading In-Between](https://md.research.elamur.ai/free/en/papers/tay-2018-reasoning-sarcasm-reading-between/)
- [Relational Graph Attention Network for Aspect-based Sentiment Analysis](https://md.research.elamur.ai/free/en/papers/wang-2020-relational-graph-attention-network-aspect/)
- [Position-Aware Tagging for Aspect Sentiment Triplet Extraction](https://md.research.elamur.ai/free/en/papers/xu-2020-position-aware-tagging-aspect-sentiment/)
- [Aspect-based Sentiment Classification with Aspect-specific Graph Convolutional Networks](https://md.research.elamur.ai/free/en/papers/zhang-2019-aspect-based-sentiment-classification-aspect/)

## Cite

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{chen2017recurrent,
  title = {{Recurrent Attention Network on Memory for Aspect Sentiment Analysis}},
  author = {Chen, Peng and Sun, Zhongqian and Bing, Lidong and Yang, Wei},
  booktitle = {Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing},
  year = {2017},
  doi = {10.18653/v1/d17-1047},
  url = {https://doi.org/10.18653/v1/d17-1047}
}
```

- [← previous Reading Wikipedia to Answer Open-Domain Questions](https://md.research.elamur.ai/free/en/papers/chen-2017-reading-wikipedia-answer-open-domain/)
- [next → Reference-based Metrics can be Replaced with Reference-less Metrics in Evaluating Grammatical Error Correction Systems](https://md.research.elamur.ai/free/en/papers/asano-2017-reference-based-metrics-can-be/)
- [↑ back to papers of 2017](https://md.research.elamur.ai/free/en/papers/2017/page/3/)

## References

26 references, 10 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Orestes Appel, Francisco Chiclana, Jenny Carter, and Hamido Fujita. 2016. A hybrid approach to the sentiment analysis problem at the sentence level. Knowl.-Based Syst., 108:110–124. [find](https://scholar.google.com/scholar?q=Orestes+Appel%2C+Francisco+Chiclana%2C+Jenny+Carter%2C+and+Hamido+Fujita%2E+2016%2E+A+hybrid+approach+to+the+sentiment+analysis+problem+at+the+sentence+level%2E+Knowl%2E%2DBased+Syst%2E%2C+108%3A110%E2%80%93124%2E)
2. Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. 2014. Neural machine translation by jointly learning to align and translate. CoRR, abs/1409.0473. [find](https://scholar.google.com/scholar?q=Dzmitry+Bahdanau%2C+Kyunghyun+Cho%2C+and+Yoshua+Bengio%2E+2014%2E+Neural+machine+translation+by+jointly+learning+to+align+and+translate%2E+CoRR%2C+abs%2F1409%2E0473%2E) [our card](https://md.research.elamur.ai/free/en/papers/bahdanau-2014-neural-machine-translation-jointly-learning/)
3. Conclusions and Future Work Peng Chen, Bing Xu, Muyun Yang, and Sheng Li. 2016. Clause sentiment identification based on convolutional neural network with context embedding. [find](https://scholar.google.com/scholar?q=Conclusions+and+Future+Work+Peng+Chen%2C+Bing+Xu%2C+Muyun+Yang%2C+and+Sheng+Li%2E+2016%2E+Clause+sentiment+identification+based+on+convolutional+neural+network+with+context+embedding%2E)
4. In this paper, we proposed a framework to identify the sentiment of opinion targets. The model Jeffrey Pennington, Richard Socher, and Christopher D. Manning. 2014. Glove: Global vectors for word representation. In Empirical Methods in Natural Language Processing (EMNLP), pages 1532– 1543. [find](https://scholar.google.com/scholar?q=In+this+paper%2C+we+proposed+a+framework+to+identify+the+sentiment+of+opinion+targets%2E+The+model+Jeffrey+Pennington%2C+Richard+Socher%2C+and+Christopher+D%2E+Manning%2E+2014%2E+Glove%3A+Global+vectors+for+word+repr)
5. In Natural Computation, Fuzzy Systems and Knowledge Discovery (ICNC-FSKD), 2016 12th International Conference on, pages 1532–1538. [find](https://scholar.google.com/scholar?q=In+Natural+Computation%2C+Fuzzy+Systems+and+Knowledge+Discovery+%28ICNC%2DFSKD%29%2C+2016+12th+International+Conference+on%2C+pages+1532%E2%80%931538%2E)
6. KyungHyun Cho, Bart van Merrienboer, Dzmitry Bahdanau, and Yoshua Bengio. 2014. On the properties of neural machine translation: Encoder-decoder approaches. CoRR, abs/1409.1259. [find](https://scholar.google.com/scholar?q=KyungHyun+Cho%2C+Bart+van+Merrienboer%2C+Dzmitry+Bahdanau%2C+and+Yoshua+Bengio%2E+2014%2E+On+the+properties+of+neural+machine+translation%3A+Encoder%2Ddecoder+approaches%2E+CoRR%2C+abs%2F1409%2E1259%2E) [our card](https://md.research.elamur.ai/free/en/papers/cho-2014-properties-neural-machine-translation-encoder/)
7. Maria Pontiki, Dimitris Galanis, John Pavlopoulos, Harris Papageorgiou, Ion Androutsopoulos, and Suresh Manandhar. 2014. Semeval-2014 task 4: Aspect based sentiment analysis. In Proceedings of the 8th international workshop on semantic evaluation (SemEval 2014), pages 27–35. [find](https://scholar.google.com/scholar?q=Maria+Pontiki%2C+Dimitris+Galanis%2C+John+Pavlopoulos%2C+Harris+Papageorgiou%2C+Ion+Androutsopoulos%2C+and+Suresh+Manandhar%2E+2014%2E+Semeval%2D2014+task+4%3A+Aspect+based+sentiment+analysis%2E+In+Proceedings+of+the+8th) [our card](https://md.research.elamur.ai/free/en/papers/pontiki-2014-semeval-2014-task-4-aspect/)
8. Lingjia Deng and Janyce Wiebe. 2015. Joint prediction for entity/event-level sentiment analysis using probabilistic soft logic models. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing. [find](https://scholar.google.com/scholar?q=Lingjia+Deng+and+Janyce+Wiebe%2E+2015%2E+Joint+prediction+for+entity%2Fevent%2Dlevel+sentiment+analysis+using+probabilistic+soft+logic+models%2E+In+Proceedings+of+the+2015+Conference+on+Empirical+Methods+in+Nat)
9. Alexander M. Rush, Sumit Chopra, and Jason Weston. 2015. A neural attention model for abstractive sentence summarization. CoRR, abs/1509.00685. [find](https://scholar.google.com/scholar?q=Alexander+M%2E+Rush%2C+Sumit+Chopra%2C+and+Jason+Weston%2E+2015%2E+A+neural+attention+model+for+abstractive+sentence+summarization%2E+CoRR%2C+abs%2F1509%2E00685%2E)
10. Xiaowen Ding, Bing Liu, and Philip S Yu. 2008. A holistic lexicon-based approach to opinion mining. In Proceedings of the 2008 international conference on web search and data mining, pages 231–240. [find](https://scholar.google.com/scholar?q=Xiaowen+Ding%2C+Bing+Liu%2C+and+Philip+S+Yu%2E+2008%2E+A+holistic+lexicon%2Dbased+approach+to+opinion+mining%2E+In+Proceedings+of+the+2008+international+conference+on+web+search+and+data+mining%2C+pages+231%E2%80%93240%2E)
11. Richard Socher, Christopher D Manning, and Andrew Y Ng. 2010. Learning continuous phrase representations and syntactic parsing with recursive neural networks. In Proceedings of the NIPS-2010 Deep Learning and Unsupervised Feature Learning Workshop, pages 1–9. [find](https://scholar.google.com/scholar?q=Richard+Socher%2C+Christopher+D+Manning%2C+and+Andrew+Y+Ng%2E+2010%2E+Learning+continuous+phrase+representations+and+syntactic+parsing+with+recursive+neural+networks%2E+In+Proceedings+of+the+NIPS%2D2010+Deep+Lear)
12. Li Dong, Furu Wei, Chuanqi Tan, Duyu Tang, Ming Zhou, and Ke Xu. 2014. Adaptive recursive neural network for target-dependent twitter sentiment classification. In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers). [find](https://scholar.google.com/scholar?q=Li+Dong%2C+Furu+Wei%2C+Chuanqi+Tan%2C+Duyu+Tang%2C+Ming+Zhou%2C+and+Ke+Xu%2E+2014%2E+Adaptive+recursive+neural+network+for+target%2Ddependent+twitter+sentiment+classification%2E+In+Proceedings+of+the+52nd+Annual+Meetin) [our card](https://md.research.elamur.ai/free/en/papers/dong-2014-adaptive-recursive-neural-network-target/)
13. Richard Socher, Jeffrey Pennington, Eric H. Huang, Andrew Y. Ng, and Christopher D. Manning. 2011. Semi-supervised recursive autoencoders for predicting sentiment distributions. In Proceedings of the 2011 Conference on Empirical Methods in Natural Language Processing, EMNLP 2011, pages 151– 161. [find](https://scholar.google.com/scholar?q=Richard+Socher%2C+Jeffrey+Pennington%2C+Eric+H%2E+Huang%2C+Andrew+Y%2E+Ng%2C+and+Christopher+D%2E+Manning%2E+2011%2E+Semi%2Dsupervised+recursive+autoencoders+for+predicting+sentiment+distributions%2E+In+Proceedings+of+the+) [our card](https://md.research.elamur.ai/free/en/papers/socher-2011-semi-supervised-recursive-autoencoders-predicting/)
14. Murthy Ganapathibhotla and Bing Liu. 2008. Mining opinions in comparative sentences. In Proceedings of the 22nd International Conference on Computational Linguistics-Volume 1, pages 241–248. [find](https://scholar.google.com/scholar?q=Murthy+Ganapathibhotla+and+Bing+Liu%2E+2008%2E+Mining+opinions+in+comparative+sentences%2E+In+Proceedings+of+the+22nd+International+Conference+on+Computational+Linguistics%2DVolume+1%2C+pages+241%E2%80%93248%2E)
15. Alex Graves, Greg Wayne, and Ivo Danihelka. 2014. Neural turing machines. CoRR, abs/1410.5401. [find](https://scholar.google.com/scholar?q=Alex+Graves%2C+Greg+Wayne%2C+and+Ivo+Danihelka%2E+2014%2E+Neural+turing+machines%2E+CoRR%2C+abs%2F1410%2E5401%2E)
16. Richard Socher, Alex Perelygin, Jean Wu, Jason Chuang, Christopher D. Manning, Andrew Ng, and Christopher Potts. 2013. Recursive deep models for semantic compositionality over a sentiment treebank. In Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing. [find](https://scholar.google.com/scholar?q=Richard+Socher%2C+Alex+Perelygin%2C+Jean+Wu%2C+Jason+Chuang%2C+Christopher+D%2E+Manning%2C+Andrew+Ng%2C+and+Christopher+Potts%2E+2013%2E+Recursive+deep+models+for+semantic+compositionality+over+a+sentiment+treebank%2E+In)
17. Long Jiang, Mo Yu, Ming Zhou, Xiaohua Liu, and Tiejun Zhao. 2011. Target-dependent twitter sentiment classification. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies-Volume 1, pages 151–160. [find](https://scholar.google.com/scholar?q=Long+Jiang%2C+Mo+Yu%2C+Ming+Zhou%2C+Xiaohua+Liu%2C+and+Tiejun+Zhao%2E+2011%2E+Target%2Ddependent+twitter+sentiment+classification%2E+In+Proceedings+of+the+49th+Annual+Meeting+of+the+Association+for+Computational+Ling)
18. Duyu Tang, Bing Qin, Xiaocheng Feng, and Ting Liu. 2015. Target-dependent sentiment classification with long short term memory. CoRR, abs/1512.01100. — [arXiv:1512.01100](https://arxiv.org/abs/1512.01100) [find](https://scholar.google.com/scholar?q=Duyu+Tang%2C+Bing+Qin%2C+Xiaocheng+Feng%2C+and+Ting+Liu%2E+2015%2E+Target%2Ddependent+sentiment+classification+with+long+short+term+memory%2E+CoRR%2C+abs%2F1512%2E01100%2E+%E2%80%94+%5BarXiv%3A1512%2E01100%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F1512%2E01)
19. Andrej Karpathy, Justin Johnson, and Fei-Fei Li. 2015. Visualizing and understanding recurrent networks. CoRR, abs/1506.02078. [find](https://scholar.google.com/scholar?q=Andrej+Karpathy%2C+Justin+Johnson%2C+and+Fei%2DFei+Li%2E+2015%2E+Visualizing+and+understanding+recurrent+networks%2E+CoRR%2C+abs%2F1506%2E02078%2E) [our card](https://md.research.elamur.ai/free/en/papers/karpathy-2015-visualizing-understanding-recurrent-networks/)
20. Svetlana Kiritchenko, Xiaodan Zhu, Colin Cherry, and Saif Mohammad. 2014. Nrc-canada-2014: Detecting aspects and sentiment in customer reviews. In Proceedings of the 8th International Workshop on Semantic Evaluation (SemEval 2014), pages 437– 442. [find](https://scholar.google.com/scholar?q=Svetlana+Kiritchenko%2C+Xiaodan+Zhu%2C+Colin+Cherry%2C+and+Saif+Mohammad%2E+2014%2E+Nrc%2Dcanada%2D2014%3A+Detecting+aspects+and+sentiment+in+customer+reviews%2E+In+Proceedings+of+the+8th+International+Workshop+on+Sema) [our card](https://md.research.elamur.ai/free/en/papers/kiritchenko-2014-nrc-canada-2014-detecting-aspects/)
21. Duyu Tang, Bing Qin, and Ting Liu. 2016. Aspect level sentiment classification with deep memory network. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing. [find](https://scholar.google.com/scholar?q=Duyu+Tang%2C+Bing+Qin%2C+and+Ting+Liu%2E+2016%2E+Aspect+level+sentiment+classification+with+deep+memory+network%2E+In+Proceedings+of+the+2016+Conference+on+Empirical+Methods+in+Natural+Language+Processing%2E) [our card](https://md.research.elamur.ai/free/en/papers/tang-2016-aspect-level-sentiment-classification-deep/)
22. Ankit Kumar, Ozan Irsoy, Jonathan Su, James Bradbury, Robert English, Brian Pierce, Peter Ondruska, Ishaan Gulrajani, and Richard Socher. 2015. Ask me anything: Dynamic memory networks for natural language processing. CoRR, abs/1506.07285. [find](https://scholar.google.com/scholar?q=Ankit+Kumar%2C+Ozan+Irsoy%2C+Jonathan+Su%2C+James+Bradbury%2C+Robert+English%2C+Brian+Pierce%2C+Peter+Ondruska%2C+Ishaan+Gulrajani%2C+and+Richard+Socher%2E+2015%2E+Ask+me+anything%3A+Dynamic+memory+networks+for+natural+lan) [our card](https://md.research.elamur.ai/free/en/papers/kumar-2015-ask-me-anything-dynamic-memory/)
23. Yequan Wang, Minlie Huang, xiaoyan zhu, and Li Zhao. 2016. Attention-based lstm for aspect-level sentiment classification. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing. [find](https://scholar.google.com/scholar?q=Yequan+Wang%2C+Minlie+Huang%2C+xiaoyan+zhu%2C+and+Li+Zhao%2E+2016%2E+Attention%2Dbased+lstm+for+aspect%2Dlevel+sentiment+classification%2E+In+Proceedings+of+the+2016+Conference+on+Empirical+Methods+in+Natural+Languag) [our card](https://md.research.elamur.ai/free/en/papers/wang-2016-attention-based-lstm-aspect-level/)
24. Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean. 2013. Efficient estimation of word representations in vector space. CoRR, abs/1301.3781. [find](https://scholar.google.com/scholar?q=Tomas+Mikolov%2C+Kai+Chen%2C+Greg+Corrado%2C+and+Jeffrey+Dean%2E+2013%2E+Efficient+estimation+of+word+representations+in+vector+space%2E+CoRR%2C+abs%2F1301%2E3781%2E)
25. Wayne Xin Zhao, Jing Jiang, Hongfei Yan, and Xiaoming Li. 2010. Jointly modeling aspects and opinions with a maxent-lda hybrid. In Proceedings of the 2010 Conference on Empirical Methods in Natural Language Processing, EMNLP ’10, pages 56–65. [find](https://scholar.google.com/scholar?q=Wayne+Xin+Zhao%2C+Jing+Jiang%2C+Hongfei+Yan%2C+and+Xiaoming+Li%2E+2010%2E+Jointly+modeling+aspects+and+opinions+with+a+maxent%2Dlda+hybrid%2E+In+Proceedings+of+the+2010+Conference+on+Empirical+Methods+in+Natural+La)
26. Thien Hai Nguyen and Kiyoaki Shirai. 2015. Phrasernn: Phrase recursive neural network for aspect-based sentiment analysis. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing. [find](https://scholar.google.com/scholar?q=Thien+Hai+Nguyen+and+Kiyoaki+Shirai%2E+2015%2E+Phrasernn%3A+Phrase+recursive+neural+network+for+aspect%2Dbased+sentiment+analysis%2E+In+Proceedings+of+the+2015+Conference+on+Empirical+Methods+in+Natural+Languag)