---
title: "Utilizing BERT for Aspect-Based Sentiment — Sun 2019"
description: "Aspect-based sentiment analysis (ABSA) assigns a polarity to each aspect mentioned in a text."
authors: ["Chi Sun", "Luyao Huang", "Xipeng Qiu"]
year: 2019
venue: "Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)"
source: "https://aclanthology.org/N19-1035/"
doi: "10.18653/v1/n19-1035"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/sun-2019-utilizing-bert-aspect-based-sentiment/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/sun-2019-utilizing-bert-aspect-based-sentiment/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2019](https://md.research.elamur.ai/free/en/papers/2019/page/5/) / Utilizing BERT for Aspect-Based Sentiment Analysis via Constructing Auxiliary Sentence

# Utilizing BERT for Aspect-Based Sentiment Analysis via Constructing Auxiliary Sentence

**Authors:** [Chi Sun](https://md.research.elamur.ai/free/en/authors/chi-sun/), [Luyao Huang](https://md.research.elamur.ai/free/en/authors/luyao-huang/), [Xipeng Qiu](https://md.research.elamur.ai/free/en/authors/xipeng-qiu/) · **Year:** 2019 · **Venue:** Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/N19-1035/) · [PDF](https://aclanthology.org/N19-1035.pdf) · [DOI 10.18653/v1/n19-1035](https://doi.org/10.18653/v1/n19-1035) · [OpenAlex](https://openalex.org/W2954278700)

**Citations:** 235 citations (openalex-snapshot · 2026-10-14) · [121](#cited-by) in our library, of the 140673 papers whose bibliographies we read

## Summary
Aspect-based sentiment analysis (ABSA) assigns a polarity to each aspect mentioned in a text. Its targeted variant (TABSA) also links aspects to specific entities, which matters when one review discusses several targets. The authors found that fine-tuning BERT directly on single sentences gives little gain on these tasks. They instead recast the problem as sentence-pair classification, similar to QA or NLI. For each target–aspect combination they generate a second input sentence in one of four styles: a natural question or a terse pseudo-sentence, each with or without the candidate polarity label. Label-bearing versions become yes/no decisions per polarity. They fine-tune uncased BERT-base on these pairs and test on SentiHood (TABSA) and SemEval-2014 Task 4 (aspect category detection and polarity). All four pair-based variants clearly beat earlier feature-based, LSTM and memory-network systems, as well as a single-sentence BERT baseline. This shows the gain comes from the reformulation, not just from BERT. The idea is simple and transfers readily to other aspect-level classification problems. Code is released on GitHub (HSLCY/ABSA-BERT-pair).

## Key points
- Four auxiliary-sentence models: BERT-pair-QA-M, BERT-pair-NLI-M, BERT-pair-QA-B, BERT-pair-NLI-B. The "-B" variants add the polarity label and choose the class with the highest "yes" probability.
- SentiHood has 5,215 sentences, of which 3,862 have a single target. Evaluation covers four aspects: general, price, transit-location and safety.
- On SentiHood, the best pair models reach 87.9 aspect Macro-F1 (QA-B) and 93.6 sentiment accuracy (QA-M). This is +9.4 F1 and +2.6 accuracy over Dmu-Entnet. BERT-single gets only 85.5 sentiment accuracy.
- On SemEval-2014 Task 4, BERT-pair-NLI-B reaches 92.18 F1 on Subtask 3, compared with 88.58 for NRC-Canada. BERT-pair-QA-B gets 85.9 / 89.9 / 95.6 accuracy on Subtask 4 in the 4-way / 3-way / binary settings.
- Setup: uncased BERT-base (12 layers, hidden size 768, 12 heads, 110M parameters), fine-tuned for 4 epochs with learning rate 2e-5, batch size 24 and dropout 0.1.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

Fields: [Natural language processing](https://md.research.elamur.ai/free/en/shelves/nlp/)

## Cited by

Cited by

121 papers of our library cite it (of the 140673 whose bibliographies we read); 10 of them have a page here, 111 not yet.

- [Affective and Contextual Embedding for Sarcasm Detection](https://md.research.elamur.ai/free/en/papers/babanejad-2020-affective-contextual-embedding-sarcasm-detection/)
- [Template-Based Named Entity Recognition Using BART](https://md.research.elamur.ai/free/en/papers/cui-2021-template-based-named-entity-recognition/)
- [GlossBERT: BERT for Word Sense Disambiguation with Gloss Knowledge](https://md.research.elamur.ai/free/en/papers/huang-2019-glossbert-bert-word-sense-disambiguation/)
- [Target-Aware Data Augmentation for Stance Detection](https://md.research.elamur.ai/free/en/papers/li-2021-target-aware-data-augmentation-stance/)
- [Arabic aspect sentiment polarity classification using BERT](https://md.research.elamur.ai/free/en/papers/m-2021-abdelgwad-arabic-aspect-sentiment-polarity-classification/)
- [Pre-trained Models for Natural Language Processing: A Survey](https://md.research.elamur.ai/free/en/papers/qiu-2020-pre-trained-models-natural-language/)
- [Relational Graph Attention Network for Aspect-based Sentiment Analysis](https://md.research.elamur.ai/free/en/papers/wang-2020-relational-graph-attention-network-aspect/)
- [Big Bird: Transformers for Longer Sequences](https://md.research.elamur.ai/free/en/papers/zaheer-2020-big-bird-transformers-longer-sequences/)
- [Aspect Sentiment Quad Prediction as Paraphrase Generation](https://md.research.elamur.ai/free/en/papers/zhang-2021-aspect-sentiment-quad-prediction-as/)
- [Knowledge-Grounded Dialogue Generation with Pre-trained Language Models](https://md.research.elamur.ai/free/en/papers/zhao-2020-knowledge-grounded-dialogue-generation-pre/)

## Cite

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{sun2019utilizing,
  title = {{Utilizing BERT for Aspect-Based Sentiment Analysis via Constructing Auxiliary Sentence}},
  author = {Sun, Chi and Huang, Luyao and Qiu, Xipeng},
  booktitle = {Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)},
  year = {2019},
  doi = {10.18653/v1/n19-1035},
  url = {https://doi.org/10.18653/v1/n19-1035}
}
```

- [← previous Using Syntax to Resolve NPE in English](https://md.research.elamur.ai/free/en/papers/aclbulk-2019-khullar-syntax-resolve-npe-english/)
- [next → Vecalign: Improved Sentence Alignment in Linear Time and Space](https://md.research.elamur.ai/free/en/papers/thompson-2019-vecalign-improved-sentence-alignment-linear/)
- [↑ back to papers of 2019](https://md.research.elamur.ai/free/en/papers/2019/page/5/)

## References

20 references, 7 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Maria Pontiki, Dimitris Galanis, Haris Papageorgiou, Suresh Manandhar, and Ion Androutsopoulos. 2015. Semeval-2015 task 12: Aspect based sentiment analysis. In Proceedings of the 9th International Workshop on Semantic Evaluation (SemEval 2015), pages 486–495. [find](https://scholar.google.com/scholar?q=Maria+Pontiki%2C+Dimitris+Galanis%2C+Haris+Papageorgiou%2C+Suresh+Manandhar%2C+and+Ion+Androutsopoulos%2E+2015%2E+Semeval%2D2015+task+12%3A+Aspect+based+sentiment+analysis%2E+In+Proceedings+of+the+9th+International+Wor) [our card](https://md.research.elamur.ai/free/en/papers/pontiki-2015-semeval-2015-task-12-aspect/)
2. Caroline Brun, Diana Nicoleta Popa, and Claude Roux. 2014. Xrce: Hybrid classification for aspect-based sentiment analysis. In Proceedings of the 8th International Workshop on Semantic Evaluation (SemEval 2014), pages 838–842. [find](https://scholar.google.com/scholar?q=Caroline+Brun%2C+Diana+Nicoleta+Popa%2C+and+Claude+Roux%2E+2014%2E+Xrce%3A+Hybrid+classification+for+aspect%2Dbased+sentiment+analysis%2E+In+Proceedings+of+the+8th+International+Workshop+on+Semantic+Evaluation+%28Sem)
3. Maria Pontiki, Dimitris Galanis, John Pavlopoulos, Harris Papageorgiou, Ion Androutsopoulos, and Suresh Manandhar. 2014. Semeval-2014 task 4: Aspect based sentiment analysis. In Proceedings of the 8th International Workshop on Semantic Evaluation (SemEval 2014), pages 27–35. Association for Computat… [find](https://scholar.google.com/scholar?q=Maria+Pontiki%2C+Dimitris+Galanis%2C+John+Pavlopoulos%2C+Harris+Papageorgiou%2C+Ion+Androutsopoulos%2C+and+Suresh+Manandhar%2E+2014%2E+Semeval%2D2014+task+4%3A+Aspect+based+sentiment+analysis%2E+In+Proceedings+of+the+8th) [our card](https://md.research.elamur.ai/free/en/papers/pontiki-2014-semeval-2014-task-4-aspect/)
4. Erik Cambria, Soujanya Poria, Rajiv Bajpai, and Björn Schuller. 2016. Senticnet 4: A semantic resource for sentiment analysis based on conceptual primitives. In Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers, pages 2666–2677. [find](https://scholar.google.com/scholar?q=Erik+Cambria%2C+Soujanya+Poria%2C+Rajiv+Bajpai%2C+and+Bjo%CC%88rn+Schuller%2E+2016%2E+Senticnet+4%3A+A+semantic+resource+for+sentiment+analysis+based+on+conceptual+primitives%2E+In+Proceedings+of+COLING+2016%2C+the+26th+I)
5. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805. [find](https://scholar.google.com/scholar?q=Jacob+Devlin%2C+Ming%2DWei+Chang%2C+Kenton+Lee%2C+and+Kristina+Toutanova%2E+2018%2E+Bert%3A+Pre%2Dtraining+of+deep+bidirectional+transformers+for+language+understanding%2E+arXiv+preprint+arXiv%3A1810%2E04805%2E)
6. Alec Radford, Karthik Narasimhan, Tim Salimans, and Ilya Sutskever. 2018. Improving language understanding by generative pre-training. URL [https://s3-us-west-2.amazonaws](https://s3-us-west-2.amazonaws). com/openai-assets/research-covers/language-unsupervised/language understanding paper. pdf. [↗](https://s3-us-west-2.amazonaws)
7. Mikael Henaff, Jason Weston, Arthur Szlam, Antoine Bordes, and Yann LeCun. 2016. Tracking the world state with recurrent entity networks. arXiv preprint arXiv:1612.03969. [find](https://scholar.google.com/scholar?q=Mikael+Henaff%2C+Jason+Weston%2C+Arthur+Szlam%2C+Antoine+Bordes%2C+and+Yann+LeCun%2E+2016%2E+Tracking+the+world+state+with+recurrent+entity+networks%2E+arXiv+preprint+arXiv%3A1612%2E03969%2E)
8. Marzieh Saeidi, Guillaume Bouchard, Maria Liakata, and Sebastian Riedel. 2016. Sentihood: targeted aspect based sentiment analysis dataset for urban neighbourhoods. arXiv preprint arXiv:1610.03771. [find](https://scholar.google.com/scholar?q=Marzieh+Saeidi%2C+Guillaume+Bouchard%2C+Maria+Liakata%2C+and+Sebastian+Riedel%2E+2016%2E+Sentihood%3A+targeted+aspect+based+sentiment+analysis+dataset+for+urban+neighbourhoods%2E+arXiv+preprint+arXiv%3A1610%2E03771%2E)
9. Yohan Jo and Alice H Oh. 2011. Aspect and sentiment unification model for online review analysis. In Proceedings of the fourth ACM international conference on Web search and data mining, pages 815– 824. ACM. [find](https://scholar.google.com/scholar?q=Yohan+Jo+and+Alice+H+Oh%2E+2011%2E+Aspect+and+sentiment+unification+model+for+online+review+analysis%2E+In+Proceedings+of+the+fourth+ACM+international+conference+on+Web+search+and+data+mining%2C+pages+815%E2%80%93+82)
10. Duyu Tang, Bing Qin, Xiaocheng Feng, and Ting Liu. 2015. Effective lstms for target-dependent sentiment classification. arXiv preprint arXiv:1512.01100. [find](https://scholar.google.com/scholar?q=Duyu+Tang%2C+Bing+Qin%2C+Xiaocheng+Feng%2C+and+Ting+Liu%2E+2015%2E+Effective+lstms+for+target%2Ddependent+sentiment+classification%2E+arXiv+preprint+arXiv%3A1512%2E01100%2E) [our card](https://md.research.elamur.ai/free/en/papers/tang-2016-effective-lstms-target-dependent-sentiment/)
11. Svetlana Kiritchenko, Xiaodan Zhu, Colin Cherry, and Saif Mohammad. 2014. Nrc-canada-2014: Detecting aspects and sentiment in customer reviews. In Proceedings of the 8th International Workshop on Semantic Evaluation (SemEval 2014), pages 437– 442. [find](https://scholar.google.com/scholar?q=Svetlana+Kiritchenko%2C+Xiaodan+Zhu%2C+Colin+Cherry%2C+and+Saif+Mohammad%2E+2014%2E+Nrc%2Dcanada%2D2014%3A+Detecting+aspects+and+sentiment+in+customer+reviews%2E+In+Proceedings+of+the+8th+International+Workshop+on+Sema) [our card](https://md.research.elamur.ai/free/en/papers/kiritchenko-2014-nrc-canada-2014-detecting-aspects/)
12. Duyu Tang, Bing Qin, and Ting Liu. 2016. Aspect level sentiment classification with deep memory network. arXiv preprint arXiv:1605.08900. [find](https://scholar.google.com/scholar?q=Duyu+Tang%2C+Bing+Qin%2C+and+Ting+Liu%2E+2016%2E+Aspect+level+sentiment+classification+with+deep+memory+network%2E+arXiv+preprint+arXiv%3A1605%2E08900%2E) [our card](https://md.research.elamur.ai/free/en/papers/tang-2016-aspect-level-sentiment-classification-deep/)
13. Joachim Wagner, Piyush Arora, Santiago Cortes, Utsab Barman, Dasha Bogdanova, Jennifer Foster, and Lamia Tounsi. 2014. Dcu: Aspect-based polarity classification for semeval task 4. In Proceedings of the 8th international workshop on semantic evaluation (SemEval 2014), pages 223–229. [find](https://scholar.google.com/scholar?q=Joachim+Wagner%2C+Piyush+Arora%2C+Santiago+Cortes%2C+Utsab+Barman%2C+Dasha+Bogdanova%2C+Jennifer+Foster%2C+and+Lamia+Tounsi%2E+2014%2E+Dcu%3A+Aspect%2Dbased+polarity+classification+for+semeval+task+4%2E+In+Proceedings+of+t)
14. Fei Liu, Trevor Cohn, and Timothy Baldwin. 2018. Recurrent entity networks with delayed memory update for targeted aspect-based sentiment analysis. arXiv preprint arXiv:1804.11019. [find](https://scholar.google.com/scholar?q=Fei+Liu%2C+Trevor+Cohn%2C+and+Timothy+Baldwin%2E+2018%2E+Recurrent+entity+networks+with+delayed+memory+update+for+targeted+aspect%2Dbased+sentiment+analysis%2E+arXiv+preprint+arXiv%3A1804%2E11019%2E)
15. Yukun Ma, Haiyun Peng, and Erik Cambria. 2018. Targeted aspect-based sentiment analysis via embedding commonsense knowledge into an attentive lstm. In Proceedings of AAAI. [find](https://scholar.google.com/scholar?q=Yukun+Ma%2C+Haiyun+Peng%2C+and+Erik+Cambria%2E+2018%2E+Targeted+aspect%2Dbased+sentiment+analysis+via+embedding+commonsense+knowledge+into+an+attentive+lstm%2E+In+Proceedings+of+AAAI%2E)
16. Bo Wang, Maria Liakata, Arkaitz Zubiaga, and Rob Procter. 2017. Tdparse: Multi-target-specific sentiment recognition on twitter. In Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 1, Long Papers, volume 1, pages 483–493. [find](https://scholar.google.com/scholar?q=Bo+Wang%2C+Maria+Liakata%2C+Arkaitz+Zubiaga%2C+and+Rob+Procter%2E+2017%2E+Tdparse%3A+Multi%2Dtarget%2Dspecific+sentiment+recognition+on+twitter%2E+In+Proceedings+of+the+15th+Conference+of+the+European+Chapter+of+the+As)
17. Thien Hai Nguyen and Kiyoaki Shirai. 2015. Phrasernn: Phrase recursive neural network for aspect-based sentiment analysis. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 2509–2514. [find](https://scholar.google.com/scholar?q=Thien+Hai+Nguyen+and+Kiyoaki+Shirai%2E+2015%2E+Phrasernn%3A+Phrase+recursive+neural+network+for+aspect%2Dbased+sentiment+analysis%2E+In+Proceedings+of+the+2015+Conference+on+Empirical+Methods+in+Natural+Languag)
18. Yequan Wang, Minlie Huang, Li Zhao, et al. 2016. Attention-based lstm for aspect-level sentiment classification. In Proceedings of the 2016 conference on empirical methods in natural language processing, pages 606–615. [find](https://scholar.google.com/scholar?q=Yequan+Wang%2C+Minlie+Huang%2C+Li+Zhao%2C+et+al%2E+2016%2E+Attention%2Dbased+lstm+for+aspect%2Dlevel+sentiment+classification%2E+In+Proceedings+of+the+2016+conference+on+empirical+methods+in+natural+language+processi) [our card](https://md.research.elamur.ai/free/en/papers/wang-2016-attention-based-lstm-aspect-level/)
19. Matthew E Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke Zettlemoyer. 2018. Deep contextualized word representations. arXiv preprint arXiv:1802.05365. [find](https://scholar.google.com/scholar?q=Matthew+E+Peters%2C+Mark+Neumann%2C+Mohit+Iyyer%2C+Matt+Gardner%2C+Christopher+Clark%2C+Kenton+Lee%2C+and+Luke+Zettlemoyer%2E+2018%2E+Deep+contextualized+word+representations%2E+arXiv+preprint+arXiv%3A1802%2E05365%2E)
20. Maria Pontiki, Dimitris Galanis, Haris Papageorgiou, Ion Androutsopoulos, Suresh Manandhar, AL-Smadi Mohammad, Mahmoud Al-Ayyoub, Yanyan Zhao, Bing Qin, Orphée De Clercq, et al. 2016. Semeval-2016 task 5: Aspect based sentiment analysis. In Proceedings of the 10th international workshop on semantic… [find](https://scholar.google.com/scholar?q=Maria+Pontiki%2C+Dimitris+Galanis%2C+Haris+Papageorgiou%2C+Ion+Androutsopoulos%2C+Suresh+Manandhar%2C+AL%2DSmadi+Mohammad%2C+Mahmoud+Al%2DAyyoub%2C+Yanyan+Zhao%2C+Bing+Qin%2C+Orphe%CC%81e+De+Clercq%2C+et+al%2E+2016%2E+Semeval%2D2016+ta) [our card](https://md.research.elamur.ai/free/en/papers/pontiki-2016-semeval-2016-task-5-aspect/)