--- title: "Robust Sentiment Detection on Twitter from — Barbosa 2010" description: "Barbosa and Feng address sentiment classification of tweets." authors: ["Luciano Barbosa", "Junlan Feng"] year: 2010 venue: "Coling 2010: Posters" source: "https://aclanthology.org/C10-2005/" licence: "CC BY-NC-SA 3.0 (ACL Anthology, before 2016)" licence_url: "https://creativecommons.org/licenses/by-nc-sa/3.0/" summary_licence: "Elamur Cosmic License" canonical: "https://research.elamur.ai/free/en/papers/barbosa-2010-robust-sentiment-detection-twitter-biased/" --- [Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/barbosa-2010-robust-sentiment-detection-twitter-biased/) [home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2010](https://md.research.elamur.ai/free/en/papers/2010/) / Robust Sentiment Detection on Twitter from Biased and Noisy Data # Robust Sentiment Detection on Twitter from Biased and Noisy Data **Authors:** Luciano Barbosa, Junlan Feng · **Year:** 2010 · **Venue:** Coling 2010: Posters · **Licence:** [CC BY-NC-SA 3.0 (ACL Anthology, before 2016)](https://creativecommons.org/licenses/by-nc-sa/3.0/) **Links:** [Publisher's page](https://aclanthology.org/C10-2005/) · [PDF](https://aclanthology.org/C10-2005.pdf) · [OpenAlex](https://openalex.org/W2124156373) **Citations:** 865 citations (openalex-snapshot · 2026-10-14) · [87](#cited-by) in our library, of the 140884 papers whose bibliographies we read ## Summary Barbosa and Feng address sentiment classification of tweets. Tweets are too short for the classic n-gram models built for long texts such as reviews and news, and hand-labelled training data is expensive. They use a two-stage pipeline: first separate subjective from objective tweets, then label subjective ones as positive or negative. For training data they crawled tweets with automatic sentiment labels from three public websites (Twendz, Twitter Sentiment, TweetFeel) and analysed how good, biased and complementary these labels are. Instead of raw words, the features are abstract: - word meta-information: POS tags, plus prior subjectivity and polarity from the MPQA lexicon extended with slang; - tweet-specific syntax: retweets, hashtags, replies, links, punctuation, emoticons and capitalisation. Their system, TwitterSA, combines classifiers trained on each source. It beats unigram baselines on polarity, and its advantage grows when training data is small, biased or noisy. The work matters because it shows that cheap, imperfect labels can produce a usable classifier when the features generalise beyond specific words. ## Key points - **Data collection:** tweets were gathered over three weeks with the generic query "of". - Twendz: 254,081 tweets, 3 classes. - Twitter Sentiment: 79,696 tweets, 3 classes. - TweetFeel: 13,122 tweets, positive/negative only. - **Label quality and bias:** - Estimated polarity label quality was 0.77 (Twendz), 0.82 (Twitter Sentiment) and 0.89 (TweetFeel). - TweetFeel was heavily biased: 4 words cover 95% of its positive examples and 6 words cover 96% of its negatives. - **Polarity error rates:** - TwitterSA(maxconf): 18.7%. - Unigrams: 20.9%. - ReviewSA: 21.7%. - Unigrams-TS: 24.3%. - Majority voting was the weakest way to combine the per-source classifiers. - **Small training sets:** with 2,000 training tweets, polarity error was 46% for Unigrams versus 23.8% for TwitterSA. - **Top features for subjectivity** (by information gain): the most informative were positive-polarity words, the presence of a link, strongly subjective words, capitalised text and verbs. ## Interesting ideas - **Treat each automatic labeller as a separate annotator.** Measure each source's quality and bias, then combine them rather than trusting any single one. Different sources make different mistakes, so ensembles over weak labels can cancel out errors. This fits any project that relies on silver-standard data. - **Clean noisy training data with heuristics:** - drop items on which the sources disagree; - keep only one message per user, so that prolific accounts posting ads or job offers do not flood the data; - remove tweets containing the strongest opinion words from the objective set. These rules are cheap and can transfer to other weakly supervised corpora. - **Prefer abstract features to lexical ones when data is scarce or skewed.** Representing text via POS tags, lexicon polarity and formatting cues makes the model less sensitive to the narrow vocabulary of a biased source. This could help in low-resource languages such as Ukrainian, where labelled data is limited. - **Use entropy and word coverage to diagnose bias.** Checking how few words cover most examples of a class quickly reveals a labelling source that keys on a handful of trigger words. - **Flip prior polarity after a preceding negation.** This simple rule adds some compositional awareness to lexicon features without any parsing. *Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.* Fields: [Natural language processing](https://md.research.elamur.ai/free/en/shelves/nlp/) Topics: [Part-of-speech tagging](https://md.research.elamur.ai/free/en/topics/part-of-speech-tagging/) ## Cited by Cited by 87 papers of our library cite it (of the 140884 whose bibliographies we read); 8 of them have a page here, 79 not yet. - [Sentiment Analysis of Twitter Data](https://md.research.elamur.ai/free/en/papers/agarwal-2011-sentiment-analysis-twitter-data/) - [Part-of-Speech Tagging for Twitter: Annotation, Features, and Experiments](https://md.research.elamur.ai/free/en/papers/gimpel-2011-part-speech-tagging-twitter-annotation/) - [Open Domain Targeted Sentiment](https://md.research.elamur.ai/free/en/papers/mitchell-2013-open-domain-targeted-sentiment/) - [SemEval-2013 Task 2: Sentiment Analysis in Twitter](https://md.research.elamur.ai/free/en/papers/nakov-2013-semeval-2013-task-2-sentiment/) - [SemEval-2014 Task 9: Sentiment Analysis in Twitter](https://md.research.elamur.ai/free/en/papers/rosenthal-2014-semeval-2014-task-9-sentiment/) - [SemEval-2015 Task 10: Sentiment Analysis in Twitter](https://md.research.elamur.ai/free/en/papers/rosenthal-2015-semeval-2015-task-10-sentiment/) - [Deep Convolutional Neural Networks for Sentiment Analysis of Short Texts](https://md.research.elamur.ai/free/en/papers/santos-2014-deep-convolutional-neural-networks-sentiment/) - [Learning Sentiment-Specific Word Embedding for Twitter Sentiment Classification](https://md.research.elamur.ai/free/en/papers/tang-2014-learning-sentiment-specific-word-embedding/) ## Cite The paper itself, not this page. BibTeX from the library's record of the work. ```bibtex @inproceedings{barbosa2010robust, title = {{Robust Sentiment Detection on Twitter from Biased and Noisy Data}}, author = {Barbosa, Luciano and Feng, Junlan}, booktitle = {Coling 2010: Posters}, year = {2010}, url = {https://aclanthology.org/C10-2005/} } ``` - [← previous Resolving Object and Attribute Coreference in Opinion Mining](https://md.research.elamur.ai/free/en/papers/ding-2010-resolving-object-attribute-coreference-opinion/) - [next → SemEval-2010 Task 5 : Automatic Keyphrase Extraction from Scientific Articles](https://md.research.elamur.ai/free/en/papers/kim-2010-semeval-2010-task-5-automatic/) - [↑ back to papers of 2010](https://md.research.elamur.ai/free/en/papers/2010/) ## References 17 references, 5 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet. 1. Cohen, J. 1960. A coefficient of agreement for nominal scales. Educational and psychological measurement, 20(1):37. [find](https://scholar.google.com/scholar?q=Cohen%2C+J%2E+1960%2E+A+coefficient+of+agreement+for+nominal+scales%2E+Educational+and+psychological+measurement%2C+20%281%29%3A37%2E) 2. Glance, N., M. Hurst, K. Nigam, M. Siegler, R. Stockton, and T. Tomokiyo. 2005. Deriving marketing intelligence from online discussion. In Proceedings of the eleventh ACM SIGKDD, pages 419–428. ACM. [find](https://scholar.google.com/scholar?q=Glance%2C+N%2E%2C+M%2E+Hurst%2C+K%2E+Nigam%2C+M%2E+Siegler%2C+R%2E+Stockton%2C+and+T%2E+Tomokiyo%2E+2005%2E+Deriving+marketing+intelligence+from+online+discussion%2E+In+Proceedings+of+the+eleventh+ACM+SIGKDD%2C+pages+419%E2%80%93428%2E+ACM%2E) 3. Go, A., R. Bhayani, and L. Huang. 2009. Twitter sentiment classification using distant supervision. Technical report, Stanford Digital Library Technologies Project. [find](https://scholar.google.com/scholar?q=Go%2C+A%2E%2C+R%2E+Bhayani%2C+and+L%2E+Huang%2E+2009%2E+Twitter+sentiment+classification+using+distant+supervision%2E+Technical+report%2C+Stanford+Digital+Library+Technologies+Project%2E) 4. Landis, J.R. and G.G. Koch. 1977. The measurement of observer agreement for categorical data. Biometrics, pages 159–174. [find](https://scholar.google.com/scholar?q=Landis%2C+J%2ER%2E+and+G%2EG%2E+Koch%2E+1977%2E+The+measurement+of+observer+agreement+for+categorical+data%2E+Biometrics%2C+pages+159%E2%80%93174%2E) 5. LingPipe 3.9.1. [http://alias-i.com/lingpipe](http://alias-i.com/lingpipe) . [↗](http://alias-i.com/lingpipe) 6. Pang, B. and L. Lee. 2004. A sentimental education: Sentiment analysis using subjectivity summarization based on minimum cuts. In Proceedings of the ACL, volume 2004. [find](https://scholar.google.com/scholar?q=Pang%2C+B%2E+and+L%2E+Lee%2E+2004%2E+A+sentimental+education%3A+Sentiment+analysis+using+subjectivity+summarization+based+on+minimum+cuts%2E+In+Proceedings+of+the+ACL%2C+volume+2004%2E) [our card](https://md.research.elamur.ai/free/en/papers/pang-2004-sentimental-education-sentiment-analysis-subjectivity/) 7. Pang, B., L. Lee, and S. Vaithyanathan. 2002. Thumbs up?: sentiment classification using machine learning techniques. In Proceedings of the ACL, pages 79–86. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Pang%2C+B%2E%2C+L%2E+Lee%2C+and+S%2E+Vaithyanathan%2E+2002%2E+Thumbs+up%3F%3A+sentiment+classification+using+machine+learning+techniques%2E+In+Proceedings+of+the+ACL%2C+pages+79%E2%80%9386%2E+Association+for+Computational+Linguistics%2E) [our card](https://md.research.elamur.ai/free/en/papers/pang-2002-thumbs-up-sentiment-classification-machine/) 8. Polikar, R. 2006. Ensemble based systems in decision making. IEEE Circuits and systems magazine, 6(3):21–45. [find](https://scholar.google.com/scholar?q=Polikar%2C+R%2E+2006%2E+Ensemble+based+systems+in+decision+making%2E+IEEE+Circuits+and+systems+magazine%2C+6%283%29%3A21%E2%80%9345%2E) 9. Prentice, S. and E. Huffman. 2008. Social Medias New Role In Emergency Management. Idaho National Laboratory, pages 1–5. [find](https://scholar.google.com/scholar?q=Prentice%2C+S%2E+and+E%2E+Huffman%2E+2008%2E+Social+Medias+New+Role+In+Emergency+Management%2E+Idaho+National+Laboratory%2C+pages+1%E2%80%935%2E) 10. Riloff, E. and J. Wiebe. 2003. Learning extraction patterns for subjective expressions. In Proceedings of the 2003 conference on Empirical methods in natural language processing, pages 105–112. [find](https://scholar.google.com/scholar?q=Riloff%2C+E%2E+and+J%2E+Wiebe%2E+2003%2E+Learning+extraction+patterns+for+subjective+expressions%2E+In+Proceedings+of+the+2003+conference+on+Empirical+methods+in+natural+language+processing%2C+pages+105%E2%80%93112%2E) [our card](https://md.research.elamur.ai/free/en/papers/riloff-2003-learning-extraction-patterns-subjective-expressions/) 11. Riloff, E., J. Wiebe, and T. Wilson. 2003. Learning subjective nouns using extraction pattern bootstrapping. In Proceedings of the 7th Conference on Natural Language Learning, pages 25–32. [find](https://scholar.google.com/scholar?q=Riloff%2C+E%2E%2C+J%2E+Wiebe%2C+and+T%2E+Wilson%2E+2003%2E+Learning+subjective+nouns+using+extraction+pattern+bootstrapping%2E+In+Proceedings+of+the+7th+Conference+on+Natural+Language+Learning%2C+pages+25%E2%80%9332%2E) [our card](https://md.research.elamur.ai/free/en/papers/riloff-2003-learning-subjective-nouns-extraction-pattern/) 12. Riloff, E., S. Patwardhan, and J. Wiebe. 2006. Feature subsumption for opinion analysis. In Proceedings of the 2006 Conference on Empirical Methods in Natural Language Processing, pages 440–448. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Riloff%2C+E%2E%2C+S%2E+Patwardhan%2C+and+J%2E+Wiebe%2E+2006%2E+Feature+subsumption+for+opinion+analysis%2E+In+Proceedings+of+the+2006+Conference+on+Empirical+Methods+in+Natural+Language+Processing%2C+pages+440%E2%80%93448%2E+Assoc) 13. Sheng, V.S., F. Provost, and P.G. Ipeirotis. 2008. Get another label? Improving data quality and data mining using multiple, noisy labelers. In Proceeding of the 14th ACM SIGKDD international conference on Knowledge discovery and data mining, pages 614– 622. ACM. [find](https://scholar.google.com/scholar?q=Sheng%2C+V%2ES%2E%2C+F%2E+Provost%2C+and+P%2EG%2E+Ipeirotis%2E+2008%2E+Get+another+label%3F+Improving+data+quality+and+data+mining+using+multiple%2C+noisy+labelers%2E+In+Proceeding+of+the+14th+ACM+SIGKDD+international+conferen) 14. Wiebe, J. and E. Riloff. 2005. Creating subjective and objective sentence classifiers from unannotated texts. Computational Linguistics and Intelligent Text Processing, pages 486–497. [find](https://scholar.google.com/scholar?q=Wiebe%2C+J%2E+and+E%2E+Riloff%2E+2005%2E+Creating+subjective+and+objective+sentence+classifiers+from+unannotated+texts%2E+Computational+Linguistics+and+Intelligent+Text+Processing%2C+pages+486%E2%80%93497%2E) 15. Wiebe, J.M., RF Brace, and T.P. O’Hara. 1999. Development and use of a gold-standard data set for subjectivity classifications. In Proceedings of the ACL, pages 246–253. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Wiebe%2C+J%2EM%2E%2C+RF+Brace%2C+and+T%2EP%2E+O%E2%80%99Hara%2E+1999%2E+Development+and+use+of+a+gold%2Dstandard+data+set+for+subjectivity+classifications%2E+In+Proceedings+of+the+ACL%2C+pages+246%E2%80%93253%2E+Association+for+Computational+) 16. Wilson, T., J. Wiebe, and P. Hoffmann. 2005. Recognizing contextual polarity in phrase-level sentiment analysis. In EMNLP, page 354. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Wilson%2C+T%2E%2C+J%2E+Wiebe%2C+and+P%2E+Hoffmann%2E+2005%2E+Recognizing+contextual+polarity+in+phrase%2Dlevel+sentiment+analysis%2E+In+EMNLP%2C+page+354%2E+Association+for+Computational+Linguistics%2E) [our card](https://md.research.elamur.ai/free/en/papers/wilson-2005-recognizing-contextual-polarity-phrase-level/) 17. Witten, Ian H. and Eibe Frank. 2005. Data Mining: Practical machine learning tools and techniques. Morgan Kaufmann. [find](https://scholar.google.com/scholar?q=Witten%2C+Ian+H%2E+and+Eibe+Frank%2E+2005%2E+Data+Mining%3A+Practical+machine+learning+tools+and+techniques%2E+Morgan+Kaufmann%2E)