---
title: "SemEval-2016 Task 4 — Nakov 2016"
description: "This paper reports on the fourth edition of the SemEval shared task on Twitter sentiment, which drew 43 teams."
authors: ["Preslav Nakov", "Alan Ritter", "Sara Rosenthal", "Fabrizio Sebastiani", "Veselin Stoyanov"]
year: 2016
venue: "Proceedings of the 10th International Workshop on Semantic Evaluation (SemEval-2016)"
source: "https://aclanthology.org/S16-1001/"
doi: "10.18653/v1/s16-1001"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/nakov-2016-semeval-2016-task-4-sentiment/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/nakov-2016-semeval-2016-task-4-sentiment/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2016](https://md.research.elamur.ai/free/en/papers/2016/page/3/) / SemEval-2016 Task 4: Sentiment Analysis in Twitter

# SemEval-2016 Task 4: Sentiment Analysis in Twitter

**Authors:** [Preslav Nakov](https://md.research.elamur.ai/free/en/authors/preslav-nakov/), [Alan Ritter](https://md.research.elamur.ai/free/en/authors/alan-ritter/), [Sara Rosenthal](https://md.research.elamur.ai/free/en/authors/sara-rosenthal/), [Fabrizio Sebastiani](https://md.research.elamur.ai/free/en/authors/fabrizio-sebastiani/), [Veselin Stoyanov](https://md.research.elamur.ai/free/en/authors/veselin-stoyanov/) · **Year:** 2016 · **Venue:** Proceedings of the 10th International Workshop on Semantic Evaluation (SemEval-2016) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/S16-1001/) · [PDF](https://aclanthology.org/S16-1001.pdf) · [DOI 10.18653/v1/s16-1001](https://doi.org/10.18653/v1/s16-1001) · [OpenAlex](https://openalex.org/W2460474657)

**Citations:** 966 citations (openalex-snapshot · 2026-10-14) · [115](#cited-by) in our library, of the 140884 papers whose bibliographies we read

## Summary
This paper reports on the fourth edition of the SemEval shared task on Twitter sentiment, which drew 43 teams. Two subtasks repeat earlier setups. Subtask A is three-way polarity classification of a tweet, and Subtask B is binary sentiment towards a given topic. Three subtasks are new. Subtask C uses an ordered five-point scale, which makes it ordinal classification. Subtask D asks for the share of positive and negative tweets about a topic, which is known as quantification. Subtask E combines both ideas by estimating a five-class distribution.

The organisers built a new Tweet 2016 dataset. They selected sentiment-rich tweets on 200 manually chosen topics and labelled them through crowdsourcing. Topics in the training and test sets do not overlap. Each subtask got its own evaluation measure; for example, Subtask E used EMD.

Most top-ranked teams relied on deep learning, such as CNNs, RNNs and word embeddings. These systems often scored well without being adapted to the ordinal or quantification nature of a subtask. The task matters because many social-science and market-research uses of Twitter care about aggregate opinion, not individual labels.

## Key points
- Five subtasks (A–E). Subtasks B–E form a 2×2 design: classification vs. quantification crossed with a two-point vs. five-point scale. The expression-level subtask from 2013–2015 was dropped.
- Data collection:
  - Named entities were extracted with a Twitter-tuned NER system.
  - Tweets without sentiment words were removed, using SentiWordNet 3.0 with a score threshold above 0.3.
  - The final set covers 200 topics with at least 100 tweets each.
  - Training and development tweets come from July–October 2015; test tweets come from October–December 2015.
- Annotation: TRAIN, DEV and DEVTEST were labelled on Amazon's Mechanical Turk by five annotators per HIT. TEST was labelled on CrowdFlower, which used hidden quality checks to exclude unreliable annotators.
- Subtask E results (10 teams, lower EMD is better):
  - QCRI ranked first with EMD 0.243, using an ordinal hierarchy of binary probabilistic quantifiers.
  - The training-distribution baseline scored 0.474, and the majority-class baseline scored 0.734.
- Suggested future directions:
  - Create artificial test topics by shifting class prevalences.
  - Separate NEUTRAL tweets from lukewarm FAIR opinions.
  - Add multilingual and cross-lingual subtasks, for example English–Arabic.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

Fields: [Natural language processing](https://md.research.elamur.ai/free/en/shelves/nlp/)

## Cited by

Cited by

115 papers of our library cite it (of the 140884 whose bibliographies we read); 8 of them have a page here, 107 not yet.

- [DataStories at SemEval-2017 Task 4: Deep LSTM with Attention for Message-level and Topic-based Sentiment Analysis](https://md.research.elamur.ai/free/en/papers/baziotis-2017-datastories-semeval-2017-task-4/)
- [Multi-task dialog act and sentiment recognition on Mastodon](https://md.research.elamur.ai/free/en/papers/cerisara-2018-multi-task-dialog-act-sentiment/)
- [Using millions of emoji occurrences to learn any-domain representations for detecting sentiment, emotion and sarcasm](https://md.research.elamur.ai/free/en/papers/felbo-2017-millions-emoji-occurrences-learn-any/)
- [Using Convolutional Neural Networks to Classify Hate-Speech](https://md.research.elamur.ai/free/en/papers/gamback-2017-convolutional-neural-networks-classify-hate/)
- [SemEval-2018 Task 3: Irony Detection in English Tweets](https://md.research.elamur.ai/free/en/papers/hee-2018-semeval-2018-task-3-irony/)
- [WLV at SemEval-2018 Task 3: Dissecting Tweets in Search of Irony](https://md.research.elamur.ai/free/en/papers/rohanian-2018-wlv-semeval-2018-task-3/)
- [SemEval-2017 Task 4: Sentiment Analysis in Twitter](https://md.research.elamur.ai/free/en/papers/rosenthal-2017-semeval-2017-task-4-sentiment/)
- [Annotation, Modelling and Analysis of Fine-Grained Emotions on a Stance and Sentiment Detection Corpus](https://md.research.elamur.ai/free/en/papers/schuff-2017-annotation-modelling-analysis-fine-grained/)

## Cite

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{nakov2016semeval,
  title = {{SemEval-2016 Task 4: Sentiment Analysis in Twitter}},
  author = {Nakov, Preslav and Ritter, Alan and Rosenthal, Sara and Sebastiani, Fabrizio and Stoyanov, Veselin},
  booktitle = {Proceedings of the 10th International Workshop on Semantic Evaluation (SemEval-2016)},
  year = {2016},
  doi = {10.18653/v1/s16-1001},
  url = {https://doi.org/10.18653/v1/s16-1001}
}
```

- [← previous SemEval-2016 Task 3: Community Question Answering](https://md.research.elamur.ai/free/en/papers/nakov-2016-semeval-2016-task-3-community/)
- [next → SemEval-2016 Task 5: Aspect Based Sentiment Analysis](https://md.research.elamur.ai/free/en/papers/pontiki-2016-semeval-2016-task-5-aspect/)
- [↑ back to papers of 2016](https://md.research.elamur.ai/free/en/papers/2016/page/3/)

## References

56 references, 6 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Omar Abdelwahab and Adel Elmaghraby. 2016. UofL at SemEval-2016 Task 4: Multi domain word2vec for Twitter sentiment classification. In Proceedings of the 10th International Workshop on Semantic Evaluation (SemEval 2016), San Diego, US. Ramón Astudillo and Silvio Amir. 2016. INESC-ID at SemEval-2016… [find](https://scholar.google.com/scholar?q=Omar+Abdelwahab+and+Adel+Elmaghraby%2E+2016%2E+UofL+at+SemEval%2D2016+Task+4%3A+Multi+domain+word2vec+for+Twitter+sentiment+classification%2E+In+Proceedings+of+the+10th+International+Workshop+on+Semantic+Evalua)
2. Giuseppe Attardi and Daniele Sartiano. 2016. UniPI at SemEval-2016 Task 4: Convolutional neural networks for sentiment classification. In Proceedings of the 10th International Workshop on Semantic Evaluation (SemEval 2016), San Diego, US. [find](https://scholar.google.com/scholar?q=Giuseppe+Attardi+and+Daniele+Sartiano%2E+2016%2E+UniPI+at+SemEval%2D2016+Task+4%3A+Convolutional+neural+networks+for+sentiment+classification%2E+In+Proceedings+of+the+10th+International+Workshop+on+Semantic+Eva)
3. Stefano Baccianella, Andrea Esuli, and Fabrizio Sebastiani. 2009. Evaluation measures for ordinal regression. In Proceedings of the 9th IEEE International Conference on Intelligent Systems Design and Applications (ISDA 2009), pages 283–287, Pisa, IT. [find](https://scholar.google.com/scholar?q=Stefano+Baccianella%2C+Andrea+Esuli%2C+and+Fabrizio+Sebastiani%2E+2009%2E+Evaluation+measures+for+ordinal+regression%2E+In+Proceedings+of+the+9th+IEEE+International+Conference+on+Intelligent+Systems+Design+and+)
4. Stefano Baccianella, Andrea Esuli, and Fabrizio Sebastiani. 2010. SentiWordNet 3.0: An enhanced lexical resource for sentiment analysis and opinion mining. In Proceedings of the 7th Conference on Language Resources and Evaluation (LREC 2010), Valletta, MT. [find](https://scholar.google.com/scholar?q=Stefano+Baccianella%2C+Andrea+Esuli%2C+and+Fabrizio+Sebastiani%2E+2010%2E+SentiWordNet+3%2E0%3A+An+enhanced+lexical+resource+for+sentiment+analysis+and+opinion+mining%2E+In+Proceedings+of+the+7th+Conference+on+Lang) [our card](https://md.research.elamur.ai/free/en/papers/baccianella-2010-sentiwordnet-3-0-enhanced-lexical/)
5. Alexandra Balahur. 2016. OPAL at SemEval-2016 Task 4: the Challenge of Porting a Sentiment Analysis System to the ”Real” World. In Proceedings of the 10th International Workshop on Semantic Evaluation (SemEval 2016), San Diego, US. [find](https://scholar.google.com/scholar?q=Alexandra+Balahur%2E+2016%2E+OPAL+at+SemEval%2D2016+Task+4%3A+the+Challenge+of+Porting+a+Sentiment+Analysis+System+to+the+%E2%80%9DReal%E2%80%9D+World%2E+In+Proceedings+of+the+10th+International+Workshop+on+Semantic+Evaluation)
6. Georgios Balikas and Massih-Reza Amini. 2016. TwiSE at SemEval-2016 Task 4: Twitter sentiment classification. In Proceedings of the 10th International Workshop on Semantic Evaluation (SemEval 2016), San Diego, US. [find](https://scholar.google.com/scholar?q=Georgios+Balikas+and+Massih%2DReza+Amini%2E+2016%2E+TwiSE+at+SemEval%2D2016+Task+4%3A+Twitter+sentiment+classification%2E+In+Proceedings+of+the+10th+International+Workshop+on+Semantic+Evaluation+%28SemEval+2016%29%2C+S)
7. Johan Bollen, Huina Mao, and Xiao-Jun Zeng. 2011. Twitter mood predicts the stock market. Journal of Computational Science, 2(1):1–8. [find](https://scholar.google.com/scholar?q=Johan+Bollen%2C+Huina+Mao%2C+and+Xiao%2DJun+Zeng%2E+2011%2E+Twitter+mood+predicts+the+stock+market%2E+Journal+of+Computational+Science%2C+2%281%29%3A1%E2%80%938%2E)
8. Javier Borge-Holthoefer, Walid Magdy, Kareem Darwish, and Ingmar Weber. 2015. Content and network dynamics behind Egyptian political polarization on Twitter. In Proceedings of the 18th ACM Conference on Computer Supported Cooperative Work and Social Computing (CSCW 2015), pages 700–711, Vancouver, C… [find](https://scholar.google.com/scholar?q=Javier+Borge%2DHolthoefer%2C+Walid+Magdy%2C+Kareem+Darwish%2C+and+Ingmar+Weber%2E+2015%2E+Content+and+network+dynamics+behind+Egyptian+political+polarization+on+Twitter%2E+In+Proceedings+of+the+18th+ACM+Conference+)
9. Gerard Briones and Kasun Amarasinghe. 2016. VCU-TSA at SemEval-2016 Task 4: Sentiment analysis in Twitter. In Proceedings of the 10th International Workshop on Semantic Evaluation (SemEval 2016), San Diego, US. [find](https://scholar.google.com/scholar?q=Gerard+Briones+and+Kasun+Amarasinghe%2E+2016%2E+VCU%2DTSA+at+SemEval%2D2016+Task+4%3A+Sentiment+analysis+in+Twitter%2E+In+Proceedings+of+the+10th+International+Workshop+on+Semantic+Evaluation+%28SemEval+2016%29%2C+San+)
10. S. Burton and A. Soboleva. 2011. Interactive or reactive? Marketing with Twitter. Journal of Consumer Marketing, 28(7):491–499. Esteban Castillo, Ofelia Cervantes, Darnes Vilariño, and David Báez. 2016. UDLAP at SemEval-2016 Task 4: Sentiment quantification using a graph based representation. In P… [find](https://scholar.google.com/scholar?q=S%2E+Burton+and+A%2E+Soboleva%2E+2011%2E+Interactive+or+reactive%3F+Marketing+with+Twitter%2E+Journal+of+Consumer+Marketing%2C+28%287%29%3A491%E2%80%93499%2E+Esteban+Castillo%2C+Ofelia+Cervantes%2C+Darnes+Vilarin%CC%83o%2C+and+David+Ba%CC%81ez%2E+2)
11. Calin-Cristian Ciubotariu, Marius-Valentin Hrisca, Mihail Gliga, Diana Darabana, Diana Trandabat, and Adrian Iftene. 2016. Minions at SemEval-2016 Task 4: Or how to boost a student’s self esteem. In Proceedings of the 10th International Workshop on Semantic Evaluation (SemEval 2016), San Diego, US. [find](https://scholar.google.com/scholar?q=Calin%2DCristian+Ciubotariu%2C+Marius%2DValentin+Hrisca%2C+Mihail+Gliga%2C+Diana+Darabana%2C+Diana+Trandabat%2C+and+Adrian+Iftene%2E+2016%2E+Minions+at+SemEval%2D2016+Task+4%3A+Or+how+to+boost+a+student%E2%80%99s+self+esteem%2E+In+P)
12. Vittoria Cozza and Marinella Petrocchi. 2016. mib at SemEval-2016 Task 4: Exploiting lexicon based features for sentiment analysis in Twitter. In Proceedings of the 10th International Workshop on Semantic Evaluation (SemEval 2016), San Diego, US. [find](https://scholar.google.com/scholar?q=Vittoria+Cozza+and+Marinella+Petrocchi%2E+2016%2E+mib+at+SemEval%2D2016+Task+4%3A+Exploiting+lexicon+based+features+for+sentiment+analysis+in+Twitter%2E+In+Proceedings+of+the+10th+International+Workshop+on+Sema)
13. Giovanni Da San Martino, Wei Gao, and Fabrizio Sebastiani. 2016. QCRI at SemEval-2016 Task 4: Probabilistic methods for binary and ordinal quantification. In Proceedings of the 10th International Workshop on Semantic Evaluation (SemEval 2016), San Diego, US. Forthcoming. [find](https://scholar.google.com/scholar?q=Giovanni+Da+San+Martino%2C+Wei+Gao%2C+and+Fabrizio+Sebastiani%2E+2016%2E+QCRI+at+SemEval%2D2016+Task+4%3A+Probabilistic+methods+for+binary+and+ordinal+quantification%2E+In+Proceedings+of+the+10th+International+Work)
14. Jan Deriu, Maurice Gonzenbach, Fatih Uzdilli, Aurelien Lucchi, Valeria De Luca, and Martin Jaggi. 2016. SwissCheese at SemEval-2016 Task 4: Sentiment classification using an ensemble of convolutional neural networks with distant supervision. In Proceedings of the 10th International Workshop on Seman… [find](https://scholar.google.com/scholar?q=Jan+Deriu%2C+Maurice+Gonzenbach%2C+Fatih+Uzdilli%2C+Aurelien+Lucchi%2C+Valeria+De+Luca%2C+and+Martin+Jaggi%2E+2016%2E+SwissCheese+at+SemEval%2D2016+Task+4%3A+Sentiment+classification+using+an+ensemble+of+convolutional+)
15. Peter S. Dodds, Kameron D. Harris, Isabel M. Kloumann, Catherine A. Bliss, and Christopher M. Danforth. 2011. Temporal patterns of happiness and information in a global social network: Hedonometrics and Twitter. PLoS ONE, 6(12). [find](https://scholar.google.com/scholar?q=Peter+S%2E+Dodds%2C+Kameron+D%2E+Harris%2C+Isabel+M%2E+Kloumann%2C+Catherine+A%2E+Bliss%2C+and+Christopher+M%2E+Danforth%2E+2011%2E+Temporal+patterns+of+happiness+and+information+in+a+global+social+network%3A+Hedonometrics+a)
16. Steven Du and Xi Zhang. 2016. Aicyber at SemEval-2016 Task 4: i-vector based sentence representation. In Proceedings of the 10th International Workshop on Semantic Evaluation (SemEval 2016), San Diego, US. [find](https://scholar.google.com/scholar?q=Steven+Du+and+Xi+Zhang%2E+2016%2E+Aicyber+at+SemEval%2D2016+Task+4%3A+i%2Dvector+based+sentence+representation%2E+In+Proceedings+of+the+10th+International+Workshop+on+Semantic+Evaluation+%28SemEval+2016%29%2C+San+Diego)
17. Andrea Esuli and Fabrizio Sebastiani. 2010. Sentiment quantification. IEEE Intelligent Systems, 25(4):72–75. [find](https://scholar.google.com/scholar?q=Andrea+Esuli+and+Fabrizio+Sebastiani%2E+2010%2E+Sentiment+quantification%2E+IEEE+Intelligent+Systems%2C+25%284%29%3A72%E2%80%9375%2E)
18. Andrea Esuli and Fabrizio Sebastiani. 2015. Optimizing text quantifiers for multivariate loss functions. ACM Transactions on Knowledge Discovery and Data, 9(4):Article 27. [find](https://scholar.google.com/scholar?q=Andrea+Esuli+and+Fabrizio+Sebastiani%2E+2015%2E+Optimizing+text+quantifiers+for+multivariate+loss+functions%2E+ACM+Transactions+on+Knowledge+Discovery+and+Data%2C+9%284%29%3AArticle+27%2E)
19. Andrea Esuli. 2016. ISTI-CNR at SemEval-2016 Task 4: Quantification on an ordinal scale. In Proceedings of the 10th International Workshop on Semantic Evaluation (SemEval 2016), San Diego, US. [find](https://scholar.google.com/scholar?q=Andrea+Esuli%2E+2016%2E+ISTI%2DCNR+at+SemEval%2D2016+Task+4%3A+Quantification+on+an+ordinal+scale%2E+In+Proceedings+of+the+10th+International+Workshop+on+Semantic+Evaluation+%28SemEval+2016%29%2C+San+Diego%2C+US%2E)
20. Cosmin Florean, Oana Bejenaru, Eduard Apostol, Octavian Ciobanu, Adrian Iftene, and Diana Trandabat. 2016. SentimentalITists at SemEval-2016 Task 4: Building a Twitter sentiment analyzer in your backyard. In Proceedings of the 10th International Workshop on Semantic Evaluation (SemEval 2016), San Di… [find](https://scholar.google.com/scholar?q=Cosmin+Florean%2C+Oana+Bejenaru%2C+Eduard+Apostol%2C+Octavian+Ciobanu%2C+Adrian+Iftene%2C+and+Diana+Trandabat%2E+2016%2E+SentimentalITists+at+SemEval%2D2016+Task+4%3A+Building+a+Twitter+sentiment+analyzer+in+your+backy)
21. George Forman. 2005. Counting positives accurately despite inaccurate classification. In Proceedings of the 16th European Conference on Machine Learning (ECML 2005), pages 564–575, Porto, PT. [find](https://scholar.google.com/scholar?q=George+Forman%2E+2005%2E+Counting+positives+accurately+despite+inaccurate+classification%2E+In+Proceedings+of+the+16th+European+Conference+on+Machine+Learning+%28ECML+2005%29%2C+pages+564%E2%80%93575%2C+Porto%2C+PT%2E)
22. George Forman. 2008. Quantifying counts and costs via classification. Data Mining and Knowledge Discovery, 17(2):164–206. [find](https://scholar.google.com/scholar?q=George+Forman%2E+2008%2E+Quantifying+counts+and+costs+via+classification%2E+Data+Mining+and+Knowledge+Discovery%2C+17%282%29%3A164%E2%80%93206%2E)
23. Jasper Friedrichs. 2016. IIP at SemEval-2016 Task 4: Prioritizing classes in ensemble classification for sentiment analysis of tweets. In Proceedings of the 10th International Workshop on Semantic Evaluation (SemEval 2016), San Diego, US. [find](https://scholar.google.com/scholar?q=Jasper+Friedrichs%2E+2016%2E+IIP+at+SemEval%2D2016+Task+4%3A+Prioritizing+classes+in+ensemble+classification+for+sentiment+analysis+of+tweets%2E+In+Proceedings+of+the+10th+International+Workshop+on+Semantic+Eva)
24. Hang Gao and Tim Oates. 2016. MDSENT at SemEval-2016 Task 4: Supervised system for message polarity classification. In Proceedings of the 10th International Workshop on Semantic Evaluation (SemEval 2016), San Diego, US. [find](https://scholar.google.com/scholar?q=Hang+Gao+and+Tim+Oates%2E+2016%2E+MDSENT+at+SemEval%2D2016+Task+4%3A+Supervised+system+for+message+polarity+classification%2E+In+Proceedings+of+the+10th+International+Workshop+on+Semantic+Evaluation+%28SemEval+20)
25. Stavros Giorgis, Apostolos Rousas, John Pavlopoulos, Prodromos Malakasiotis, and Ion Androutsopoulos. 2016. aueb.twitter.sentiment at SemEval-2016 Task 4: A weighted ensemble of SVMs for Twitter sentiment analysis. In Proceedings of the 10th International Workshop on Semantic Evaluation (SemEval 201… [find](https://scholar.google.com/scholar?q=Stavros+Giorgis%2C+Apostolos+Rousas%2C+John+Pavlopoulos%2C+Prodromos+Malakasiotis%2C+and+Ion+Androutsopoulos%2E+2016%2E+aueb%2Etwitter%2Esentiment+at+SemEval%2D2016+Task+4%3A+A+weighted+ensemble+of+SVMs+for+Twitter+senti)
26. Hussam Hamdan. 2016. SentiSys at SemEval-2016 Task 4: Feature-based system for sentiment analysis in Twitter. In Proceedings of the 10th International Workshop on Semantic Evaluation (SemEval 2016), San Diego, US. [find](https://scholar.google.com/scholar?q=Hussam+Hamdan%2E+2016%2E+SentiSys+at+SemEval%2D2016+Task+4%3A+Feature%2Dbased+system+for+sentiment+analysis+in+Twitter%2E+In+Proceedings+of+the+10th+International+Workshop+on+Semantic+Evaluation+%28SemEval+2016%29%2C+S)
27. Yunchao He, Liang-Chih Yu, Chin-Sheng Yang, K. Robert Lai, and Weiyi Liu. 2016. YZU-NLP at SemEval-2016 Task 4: Ordinal sentiment classification using a recurrent convolutional network. In Proceedings of the 10th International Workshop on Semantic Evaluation (SemEval 2016), San Diego, US. Brage Ekro… [find](https://scholar.google.com/scholar?q=Yunchao+He%2C+Liang%2DChih+Yu%2C+Chin%2DSheng+Yang%2C+K%2E+Robert+Lai%2C+and+Weiyi+Liu%2E+2016%2E+YZU%2DNLP+at+SemEval%2D2016+Task+4%3A+Ordinal+sentiment+classification+using+a+recurrent+convolutional+network%2E+In+Proceedings)
28. Nikolay Karpov, Alexander Porshnev, and Kirill Rudakov. 2016. NRU-HSE at SemEval-2016 Task 4: The open quantification library with two iterative methods. In Proceedings of the 10th International Workshop on Semantic Evaluation (SemEval 2016), San Diego, US. [find](https://scholar.google.com/scholar?q=Nikolay+Karpov%2C+Alexander+Porshnev%2C+and+Kirill+Rudakov%2E+2016%2E+NRU%2DHSE+at+SemEval%2D2016+Task+4%3A+The+open+quantification+library+with+two+iterative+methods%2E+In+Proceedings+of+the+10th+International+Works)
29. Mesut Kaya, Guven Fidan, and Ismail Hakki Toroslu. 2013. Transfer learning using Twitter data for improving sentiment classification of Turkish political news. In Proceedings of the 28th International Symposium on Computer and Information Sciences (ISCIS 2013), pages 139–148, Paris, FR. [find](https://scholar.google.com/scholar?q=Mesut+Kaya%2C+Guven+Fidan%2C+and+Ismail+Hakki+Toroslu%2E+2013%2E+Transfer+learning+using+Twitter+data+for+improving+sentiment+classification+of+Turkish+political+news%2E+In+Proceedings+of+the+28th+International)
30. Mateusz Lango, Dariusz Brzezinski, and Jerzy Ste-fanowski. 2016. PUT at SemEval-2016 Task 4: The ABC of Twitter sentiment analysis. In Proceedings of the 10th International Workshop on Semantic Evaluation (SemEval 2016), San Diego, US. [find](https://scholar.google.com/scholar?q=Mateusz+Lango%2C+Dariusz+Brzezinski%2C+and+Jerzy+Ste%2Dfanowski%2E+2016%2E+PUT+at+SemEval%2D2016+Task+4%3A+The+ABC+of+Twitter+sentiment+analysis%2E+In+Proceedings+of+the+10th+International+Workshop+on+Semantic+Evalua)
31. Micol Marchetti-Bowick and Nathanael Chambers. 2012. Learning for microblogs with distant supervision: Political forecasting with Twitter. In Proceedings of the 13th Conference of the European Chapter of the Association for Computational Linguistics (EACL 2012), pages 603–612, Avignon, FR. Eugenio M… [find](https://scholar.google.com/scholar?q=Micol+Marchetti%2DBowick+and+Nathanael+Chambers%2E+2012%2E+Learning+for+microblogs+with+distant+supervision%3A+Political+forecasting+with+Twitter%2E+In+Proceedings+of+the+13th+Conference+of+the+European+Chapter)
32. Twitter: A Digital Socioscope. Cambridge University Press, Cambridge, UK. Tomas Mikolov, Wen-Tau Yih, and Geoffrey Zweig. 2013. Linguistic Regularities in Continuous Space Word Representations. In Proceedings of the 2013 Conference of the North American Chapter of the Association for Computational L… [find](https://scholar.google.com/scholar?q=Twitter%3A+A+Digital+Socioscope%2E+Cambridge+University+Press%2C+Cambridge%2C+UK%2E+Tomas+Mikolov%2C+Wen%2DTau+Yih%2C+and+Geoffrey+Zweig%2E+2013%2E+Linguistic+Regularities+in+Continuous+Space+Word+Representations%2E+In+Pro) [our card](https://md.research.elamur.ai/free/en/papers/mikolov-2013-linguistic-regularities-continuous-space-word/)
33. Hurtado, and Ferran Pla. 2016. DSIC-ELIRF at SemEval-2016 Task 4: Message polarity classification in Twitter using a support vector machine approach. In Proceedings of the 10th International Workshop on Semantic Evaluation (SemEval 2016), San Diego, US. [find](https://scholar.google.com/scholar?q=Hurtado%2C+and+Ferran+Pla%2E+2016%2E+DSIC%2DELIRF+at+SemEval%2D2016+Task+4%3A+Message+polarity+classification+in+Twitter+using+a+support+vector+machine+approach%2E+In+Proceedings+of+the+10th+International+Workshop+)
34. Mahmoud Nabil, Amir Atyia, and Mohamed Aly. 2016. CUFE at SemEval-2016 Task 4: A gated recurrent model for sentiment classification. In Proceedings of the 10th International Workshop on Semantic Evaluation (SemEval 2016), San Diego, US. [find](https://scholar.google.com/scholar?q=Mahmoud+Nabil%2C+Amir+Atyia%2C+and+Mohamed+Aly%2E+2016%2E+CUFE+at+SemEval%2D2016+Task+4%3A+A+gated+recurrent+model+for+sentiment+classification%2E+In+Proceedings+of+the+10th+International+Workshop+on+Semantic+Evalu)
35. Preslav Nakov, Sara Rosenthal, Zornitsa Kozareva, Veselin Stoyanov, Alan Ritter, and Theresa Wilson. 2013. SemEval-2013 Task 2: Sentiment analysis in Twitter. In Proceedings of the 7th International Workshop on Semantic Evaluation (SemEval 2013), pages 312–320, Atlanta, US. [find](https://scholar.google.com/scholar?q=Preslav+Nakov%2C+Sara+Rosenthal%2C+Zornitsa+Kozareva%2C+Veselin+Stoyanov%2C+Alan+Ritter%2C+and+Theresa+Wilson%2E+2013%2E+SemEval%2D2013+Task+2%3A+Sentiment+analysis+in+Twitter%2E+In+Proceedings+of+the+7th+International+W) [our card](https://md.research.elamur.ai/free/en/papers/nakov-2013-semeval-2013-task-2-sentiment/)
36. Preslav Nakov, Alan Ritter, Sara Rosenthal, Fabrizio Sebastiani, and Veselin Stoyanov. 2016a. Evaluation measures for the SemEval-2016 Task 4 “Sentiment analysis in Twitter”. Available from [http://alt.qcri.org/semeval2016/task4/](http://alt.qcri.org/semeval2016/task4/). [↗](http://alt.qcri.org/semeval2016/task4/)
37. Preslav Nakov, Sara Rosenthal, Svetlana Kiritchenko, Saif M. Mohammad, Zornitsa Kozareva, Alan Ritter, Veselin Stoyanov, and Xiaodan Zhu. 2016b. Developing a successful SemEval task in sentiment analysis of Twitter and other social media texts. Language Resources and Evaluation, 50(1):35–65. [find](https://scholar.google.com/scholar?q=Preslav+Nakov%2C+Sara+Rosenthal%2C+Svetlana+Kiritchenko%2C+Saif+M%2E+Mohammad%2C+Zornitsa+Kozareva%2C+Alan+Ritter%2C+Veselin+Stoyanov%2C+and+Xiaodan+Zhu%2E+2016b%2E+Developing+a+successful+SemEval+task+in+sentiment+analy)
38. Brendan O’Connor, Ramnath Balasubramanyan, Bryan R. Routledge, and Noah A. Smith. 2010. From tweets to polls: Linking text sentiment to public opinion time series. In Proceedings of the 4th AAAI Conference on Weblogs and Social Media (ICWSM 2010), Washington, US. [find](https://scholar.google.com/scholar?q=Brendan+O%E2%80%99Connor%2C+Ramnath+Balasubramanyan%2C+Bryan+R%2E+Routledge%2C+and+Noah+A%2E+Smith%2E+2010%2E+From+tweets+to+polls%3A+Linking+text+sentiment+to+public+opinion+time+series%2E+In+Proceedings+of+the+4th+AAAI+Confe)
39. Elisavet Palogiannidi, Athanasia Kolovou, Fenia Christopoulou, Filippos Kokkinos, Elias Iosif, Nikolaos Malandrakis, Haris Papageorgiou, Shrikanth Narayanan, and Alexandros Potamianos. 2016. Tweester at SemEval-2016 Task 4: Sentiment analysis in Twitter using semantic-affective model adaptation. In… [find](https://scholar.google.com/scholar?q=Elisavet+Palogiannidi%2C+Athanasia+Kolovou%2C+Fenia+Christopoulou%2C+Filippos+Kokkinos%2C+Elias+Iosif%2C+Nikolaos+Malandrakis%2C+Haris+Papageorgiou%2C+Shrikanth+Narayanan%2C+and+Alexandros+Potamianos%2E+2016%2E+Tweester+)
40. Jeffrey Pennington, Richard Socher, and Christopher D. Manning. 2014. GloVe: Global vectors for word representation. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP 2014), pages 1532–1543, Doha, QA. [find](https://scholar.google.com/scholar?q=Jeffrey+Pennington%2C+Richard+Socher%2C+and+Christopher+D%2E+Manning%2E+2014%2E+GloVe%3A+Global+vectors+for+word+representation%2E+In+Proceedings+of+the+Conference+on+Empirical+Methods+in+Natural+Language+Processin)
41. Muhammad A. Qureshi, Colm O’Riordan, and Gabriella Pasi. 2013. Clustering with error estimation for monitoring reputation of companies on Twitter. In Proceedings of the 9th Asia Information Retrieval Societies Conference (AIRS 2013), pages 170–180, Singapore, SN. Stefan Räbiger, Mishal Kazmi, Yüce… [find](https://scholar.google.com/scholar?q=Muhammad+A%2E+Qureshi%2C+Colm+O%E2%80%99Riordan%2C+and+Gabriella+Pasi%2E+2013%2E+Clustering+with+error+estimation+for+monitoring+reputation+of+companies+on+Twitter%2E+In+Proceedings+of+the+9th+Asia+Information+Retrieval+)
42. Alan Ritter, Sam Clark, and Oren Etzioni. 2011. Named entity recognition in tweets: An experimental study. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP 2011), pages 1524–1534, Edinburgh, UK. [find](https://scholar.google.com/scholar?q=Alan+Ritter%2C+Sam+Clark%2C+and+Oren+Etzioni%2E+2011%2E+Named+entity+recognition+in+tweets%3A+An+experimental+study%2E+In+Proceedings+of+the+Conference+on+Empirical+Methods+in+Natural+Language+Processing+%28EMNLP+2) [our card](https://md.research.elamur.ai/free/en/papers/ritter-2011-named-entity-recognition-tweets-experimental/)
43. Sara Rosenthal, Alan Ritter, Preslav Nakov, and Veselin Stoyanov. 2014. SemEval-2014 Task 9: Sentiment analysis in Twitter. In Proceedings of the 8th International Workshop on Semantic Evaluation (SemEval 2014), pages 73–80, Dublin, IE. [find](https://scholar.google.com/scholar?q=Sara+Rosenthal%2C+Alan+Ritter%2C+Preslav+Nakov%2C+and+Veselin+Stoyanov%2E+2014%2E+SemEval%2D2014+Task+9%3A+Sentiment+analysis+in+Twitter%2E+In+Proceedings+of+the+8th+International+Workshop+on+Semantic+Evaluation+%28Sem) [our card](https://md.research.elamur.ai/free/en/papers/rosenthal-2014-semeval-2014-task-9-sentiment/)
44. Sara Rosenthal, Preslav Nakov, Svetlana Kiritchenko, Saif Mohammad, Alan Ritter, and Veselin Stoyanov. 2015. SemEval-2015 Task 10: Sentiment analysis in Twitter. In Proceedings of the 9th International Workshop on Semantic Evaluation (SemEval 2015), pages 451–463, Denver, US. [find](https://scholar.google.com/scholar?q=Sara+Rosenthal%2C+Preslav+Nakov%2C+Svetlana+Kiritchenko%2C+Saif+Mohammad%2C+Alan+Ritter%2C+and+Veselin+Stoyanov%2E+2015%2E+SemEval%2D2015+Task+10%3A+Sentiment+analysis+in+Twitter%2E+In+Proceedings+of+the+9th+Internationa) [our card](https://md.research.elamur.ai/free/en/papers/rosenthal-2015-semeval-2015-task-10-sentiment/)
45. Mickael Rouvier and Benoit Favre. 2016. SENSEI-LIF at SemEval-2016 Task 4: Polarity embedding fusion for robust sentiment analysis. In Proceedings of the 10th International Workshop on Semantic Evaluation (SemEval 2016), San Diego, US. [find](https://scholar.google.com/scholar?q=Mickael+Rouvier+and+Benoit+Favre%2E+2016%2E+SENSEI%2DLIF+at+SemEval%2D2016+Task+4%3A+Polarity+embedding+fusion+for+robust+sentiment+analysis%2E+In+Proceedings+of+the+10th+International+Workshop+on+Semantic+Evalua)
46. Yossi Rubner, Carlo Tomasi, and Leonidas J. Guibas. 2000. The Earth Mover’s Distance as a metric for image retrieval. International Journal of Computer Vision, 40(2):99–121. [find](https://scholar.google.com/scholar?q=Yossi+Rubner%2C+Carlo+Tomasi%2C+and+Leonidas+J%2E+Guibas%2E+2000%2E+The+Earth+Mover%E2%80%99s+Distance+as+a+metric+for+image+retrieval%2E+International+Journal+of+Computer+Vision%2C+40%282%29%3A99%E2%80%93121%2E)
47. Sebastian Ruder, Parsa Ghaffari, and John G. Breslin. 2016. INSIGHT-1 at SemEval-2016 Task 4: Convolutional Neural Networks for Sentiment Classification and Quantification. In Proceedings of the 10th International Workshop on Semantic Evaluation (SemEval 2016), San Diego, US. [find](https://scholar.google.com/scholar?q=Sebastian+Ruder%2C+Parsa+Ghaffari%2C+and+John+G%2E+Breslin%2E+2016%2E+INSIGHT%2D1+at+SemEval%2D2016+Task+4%3A+Convolutional+Neural+Networks+for+Sentiment+Classification+and+Quantification%2E+In+Proceedings+of+the+10th+)
48. Ludger Rüschendorf. 2001. Wasserstein metric. In Michiel Hazewinkel, editor, Encyclopaedia of Mathematics. [find](https://scholar.google.com/scholar?q=Ludger+Ru%CC%88schendorf%2E+2001%2E+Wasserstein+metric%2E+In+Michiel+Hazewinkel%2C+editor%2C+Encyclopaedia+of+Mathematics%2E)
49. Kluwer Academic Publishers, Dordrecht, NL. Abeed Sarker. 2016. DIEGOLab16 at SemEval-2016 Task 4: Sentiment analysis in Twitter using centroids, clusters, and sentiment lexicons. In Proceedings of the 10th International Workshop on Semantic Evaluation (SemEval 2016), San Diego, US. [find](https://scholar.google.com/scholar?q=Kluwer+Academic+Publishers%2C+Dordrecht%2C+NL%2E+Abeed+Sarker%2E+2016%2E+DIEGOLab16+at+SemEval%2D2016+Task+4%3A+Sentiment+analysis+in+Twitter+using+centroids%2C+clusters%2C+and+sentiment+lexicons%2E+In+Proceedings+of+the)
50. Fabrizio Sebastiani. 2015. An axiomatically derived measure for the evaluation of classification algorithms. In Proceedings of the 5th ACM International Conference on the Theory of Information Retrieval (ICTIR 2015), pages 11–20, Northampton, US. [find](https://scholar.google.com/scholar?q=Fabrizio+Sebastiani%2E+2015%2E+An+axiomatically+derived+measure+for+the+evaluation+of+classification+algorithms%2E+In+Proceedings+of+the+5th+ACM+International+Conference+on+the+Theory+of+Information+Retriev)
51. Uladzimir Sidarenka. 2016. PotTS at SemEval-2016 Task 4: Sentiment analysis of Twitter using character-level convolutional neural networks. In Proceedings of the 10th International Workshop on Semantic Evaluation (SemEval 2016), San Diego, US. [find](https://scholar.google.com/scholar?q=Uladzimir+Sidarenka%2E+2016%2E+PotTS+at+SemEval%2D2016+Task+4%3A+Sentiment+analysis+of+Twitter+using+character%2Dlevel+convolutional+neural+networks%2E+In+Proceedings+of+the+10th+International+Workshop+on+Semanti)
52. Dario Stojanovski, Gjorgji Strezoski, Gjorgji Madjarov, and Ivica Dimitrovski. 2016. Finki at SemEval-2016 Task 4: Deep learning architecture for Twitter sentiment analysis. In Proceedings of the 10th International Workshop on Semantic Evaluation (SemEval 2016), San Diego, US. David Vilares, Yerai D… [find](https://scholar.google.com/scholar?q=Dario+Stojanovski%2C+Gjorgji+Strezoski%2C+Gjorgji+Madjarov%2C+and+Ivica+Dimitrovski%2E+2016%2E+Finki+at+SemEval%2D2016+Task+4%3A+Deep+learning+architecture+for+Twitter+sentiment+analysis%2E+In+Proceedings+of+the+10th)
53. Steven Xu, HuiZhi Liang, and Tim Baldwin. 2016. UNIMELB at SemEval-2016 Task 4: An ensemble of neural networks and a word2vec based model for sentiment classification. In Proceedings of the 10th International Workshop on Semantic Evaluation (SemEval 2016), San Diego, US. [find](https://scholar.google.com/scholar?q=Steven+Xu%2C+HuiZhi+Liang%2C+and+Tim+Baldwin%2E+2016%2E+UNIMELB+at+SemEval%2D2016+Task+4%3A+An+ensemble+of+neural+networks+and+a+word2vec+based+model+for+sentiment+classification%2E+In+Proceedings+of+the+10th+Inter)
54. Vikrant Yadav. 2016. thecerealkiller at SemEval-2016 Task 4: Deep learning based system for classifying sentiment of tweets on two point scale. In Proceedings of the 10th International Workshop on Semantic Evaluation (SemEval 2016), San Diego, US. [find](https://scholar.google.com/scholar?q=Vikrant+Yadav%2E+2016%2E+thecerealkiller+at+SemEval%2D2016+Task+4%3A+Deep+learning+based+system+for+classifying+sentiment+of+tweets+on+two+point+scale%2E+In+Proceedings+of+the+10th+International+Workshop+on+Sem)
55. Zhengchen Zhang, Chen Zhang, Dongyan Huang, Wu Fuxiang, Weisi Lin, and Minghui Dong. 2016. I2RNTU at SemEval-2016 Task 4: Classifier fusion for polarity classification in Twitter. In Proceedings of the 10th International Workshop on Semantic Evaluation (SemEval 2016), San Diego, US. [find](https://scholar.google.com/scholar?q=Zhengchen+Zhang%2C+Chen+Zhang%2C+Dongyan+Huang%2C+Wu+Fuxiang%2C+Weisi+Lin%2C+and+Minghui+Dong%2E+2016%2E+I2RNTU+at+SemEval%2D2016+Task+4%3A+Classifier+fusion+for+polarity+classification+in+Twitter%2E+In+Proceedings+of+th)
56. Yunxiao Zhou, Zhihua Zhang, and Man Lan. 2016. ECNU at SemEval-2016 Task 4: An empirical investigation of traditional NLP features and word embedding features for sentence-level and topic-level sentiment analysis in Twitter. In Proceedings of the 10th International Workshop on Semantic Evaluation (S… [find](https://scholar.google.com/scholar?q=Yunxiao+Zhou%2C+Zhihua+Zhang%2C+and+Man+Lan%2E+2016%2E+ECNU+at+SemEval%2D2016+Task+4%3A+An+empirical+investigation+of+traditional+NLP+features+and+word+embedding+features+for+sentence%2Dlevel+and+topic%2Dlevel+sentim)