--- title: "Sentiment Analysis of Twitter Data — Agarwal 2011" description: "The paper addresses polarity classification of tweets in two settings: positive vs." authors: ["Apoorv Agarwal", "Boyi Xie", "Ilia Vovsha", "Owen Rambow", "Rebecca Passonneau"] year: 2011 venue: "Proceedings of the Workshop on Language in Social Media (LSM 2011)" source: "https://aclanthology.org/W11-0705/" licence: "CC BY-NC-SA 3.0 (ACL Anthology, before 2016)" licence_url: "https://creativecommons.org/licenses/by-nc-sa/3.0/" summary_licence: "Elamur Cosmic License" canonical: "https://research.elamur.ai/free/en/papers/agarwal-2011-sentiment-analysis-twitter-data/" --- [Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/agarwal-2011-sentiment-analysis-twitter-data/) [home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2011](https://md.research.elamur.ai/free/en/papers/2011/) / Sentiment Analysis of Twitter Data # Sentiment Analysis of Twitter Data **Authors:** [Apoorv Agarwal](https://md.research.elamur.ai/free/en/authors/apoorv-agarwal/), Boyi Xie, Ilia Vovsha, [Owen Rambow](https://md.research.elamur.ai/free/en/authors/owen-rambow/), [Rebecca Passonneau](https://md.research.elamur.ai/free/en/authors/rebecca-passonneau/) · **Year:** 2011 · **Venue:** Proceedings of the Workshop on Language in Social Media (LSM 2011) · **Licence:** [CC BY-NC-SA 3.0 (ACL Anthology, before 2016)](https://creativecommons.org/licenses/by-nc-sa/3.0/) **Links:** [Publisher's page](https://aclanthology.org/W11-0705/) · [PDF](https://aclanthology.org/W11-0705.pdf) · [OpenAlex](https://openalex.org/W1743243001) **Citations:** 1338 citations (openalex-snapshot · 2026-10-14) · [119](#cited-by) in our library, of the 140884 papers whose bibliographies we read ## Summary The paper addresses polarity classification of tweets in two settings: positive vs. negative, and a 3-way task that adds neutral. The authors use 11,875 tweets labelled by hand. A commercial provider collected them from the live Twitter stream without keyword filtering, so the sample is less biased than data gathered with search queries or emoticon-based distant labels. Preprocessing relies on two new resources, an emoticon polarity dictionary and an acronym expansion list. Three model types are compared: - a unigram baseline; - a feature-based model with about 100 "Senti-features" that combine prior word polarity with POS tags; - an SVM tree kernel over a custom tree representation of each tweet. The 100 features match unigrams, which need over 10,000 features. The tree kernel and the model combinations beat the baseline by roughly 4% absolute. Feature analysis shows that polarity-by-POS features matter most, while Twitter-specific cues add little. The authors conclude that standard NLP tools and lexicons transfer to tweets reasonably well. ## Key points - **Data:** 11,875 annotated tweets. After removing "junk" tweets, often caused by poor Google Translate output, 8,753 remain. Stratified sampling then gives a balanced set of 5,127 tweets, 1,709 per class. - **Resources:** - an emoticon dictionary of 170 Wikipedia-listed emoticons with five labels, from Extremely-positive to Extremely-negative plus Neutral; - an acronym dictionary of 5,184 entries, e.g. lol → laughing out loud. - **Preprocessing:** emoticons are replaced by their polarity, and URLs and @-targets are replaced by tags. Negations are normalised to NOT, and character runs are shortened to three letters. - **3-way results (5-fold CV, chance 33.33%):** | Model | Accuracy | |---|---| | Unigram | 56.58% | | Senti-features | 56.31% | | Unigram + Senti-features | 60.50% | | Tree kernel | 60.60% | | Tree kernel + Senti-features | ≈60.83% (best) | - **Feature ranking:** the strongest unigrams include clearly polar words, emoticon tags and the URL tag. Unexpectedly, "for" also ranks high, through patterns like "for you". ## Interesting ideas - **Polarity crossed with POS:** pairing a word's lexicon polarity with its part of speech gives a small, interpretable feature set that rivals large n-gram models. This could suit low-resource languages, where a sentiment lexicon plus a POS tagger are easier to obtain than lots of labelled data. - **Tree kernels instead of feature engineering:** encoding each tweet as a tree and letting an SVM kernel compare structures removes most manual feature design. It still performs about as well as the best hand-built features, which makes it a quick strong baseline for new genres. - **Normalising noise into meaningful tags:** mapping emoticons, URLs, mentions and negations to placeholder tokens, while keeping three-letter elongations, preserves emphasis and still shrinks the vocabulary. The step is cheap and reusable for any social-media corpus. - **Incremental ablation plus learning curves:** adding feature groups one at a time shows which families actually help. Plotting accuracy against training-set size showed the gains over unigrams grow when data is scarce. Both checks are worth copying when justifying compact feature sets. - **Sampling from the stream rather than by query:** a random sample of the stream avoids topic and emoticon biases in evaluation data. That is useful whenever a benchmark must reflect real-world language use. *Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.* Fields: [Natural language processing](https://md.research.elamur.ai/free/en/shelves/nlp/) Topics: [Part-of-speech tagging](https://md.research.elamur.ai/free/en/topics/part-of-speech-tagging/) ## Cited by Cited by 119 papers of our library cite it (of the 140884 whose bibliographies we read); 3 of them have a page here, 116 not yet. - [A Comparative Study of Feature Selection Methods for Dialectal Arabic Sentiment Classification Using Support Vector Machine](https://md.research.elamur.ai/free/en/papers/al-2019-harbi-comparative-study-feature-selection-methods/) - [Open Domain Targeted Sentiment](https://md.research.elamur.ai/free/en/papers/mitchell-2013-open-domain-targeted-sentiment/) - [That's So Annoying!!!: A Lexical and Frame-Semantic Embedding Based Data Augmentation Approach to Automatic Categorization of Annoying Behaviors using #petpeeve Tweets](https://md.research.elamur.ai/free/en/papers/wang-2015-that-s-so-annoying-lexical/) ## Cite The paper itself, not this page. BibTeX from the library's record of the work. ```bibtex @inproceedings{agarwal2011sentiment, title = {{Sentiment Analysis of Twitter Data}}, author = {Agarwal, Apoorv and Xie, Boyi and Vovsha, Ilia and Rambow, Owen and Passonneau, Rebecca}, booktitle = {Proceedings of the Workshop on Language in Social Media (LSM 2011)}, year = {2011}, url = {https://aclanthology.org/W11-0705/} } ``` - [← previous Semi-Supervised Recursive Autoencoders for Predicting Sentiment Distributions](https://md.research.elamur.ai/free/en/papers/socher-2011-semi-supervised-recursive-autoencoders-predicting/) - [next → Shallow Discourse Parsing with Conditional Random Fields](https://md.research.elamur.ai/free/en/papers/ghosh-2011-shallow-discourse-parsing-conditional-random/) - [↑ back to papers of 2011](https://md.research.elamur.ai/free/en/papers/2011/) ## References 16 references, 6 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet. 1. Apoorv Agarwal, Fadi Biadsy, and Kathleen Mckeown. 2009. Contextual phrase-level polarity analysis using lexical affect scoring and syntactic n-grams. Proceedings of the 12th Conference of the European Chapter of the ACL (EACL 2009), pages 24–32, March. [find](https://scholar.google.com/scholar?q=Apoorv+Agarwal%2C+Fadi+Biadsy%2C+and+Kathleen+Mckeown%2E+2009%2E+Contextual+phrase%2Dlevel+polarity+analysis+using+lexical+affect+scoring+and+syntactic+n%2Dgrams%2E+Proceedings+of+the+12th+Conference+of+the+Europea) 2. Luciano Barbosa and Junlan Feng. 2010. Robust sentiment detection on twitter from biased and noisy data. Proceedings of the 23rd International Conference on Computational Linguistics: Posters, pages 36–44. [find](https://scholar.google.com/scholar?q=Luciano+Barbosa+and+Junlan+Feng%2E+2010%2E+Robust+sentiment+detection+on+twitter+from+biased+and+noisy+data%2E+Proceedings+of+the+23rd+International+Conference+on+Computational+Linguistics%3A+Posters%2C+pages+3) [our card](https://md.research.elamur.ai/free/en/papers/barbosa-2010-robust-sentiment-detection-twitter-biased/) 3. Adam Bermingham and Alan Smeaton. 2010. Classifying sentiment in microblogs: is brevity an advantage is brevity an advantage? ACM, pages 1833–1836. [find](https://scholar.google.com/scholar?q=Adam+Bermingham+and+Alan+Smeaton%2E+2010%2E+Classifying+sentiment+in+microblogs%3A+is+brevity+an+advantage+is+brevity+an+advantage%3F+ACM%2C+pages+1833%E2%80%931836%2E) 4. C. Fellbaum. 1998. Wordnet, an electronic lexical database. MIT Press. [find](https://scholar.google.com/scholar?q=C%2E+Fellbaum%2E+1998%2E+Wordnet%2C+an+electronic+lexical+database%2E+MIT+Press%2E) 5. Michael Gamon. 2004. Sentiment classification on customer feedback data: noisy data, large feature vectors, and the role of linguistic analysis. Proceedings of the 20th international conference on Computational Linguistics. [find](https://scholar.google.com/scholar?q=Michael+Gamon%2E+2004%2E+Sentiment+classification+on+customer+feedback+data%3A+noisy+data%2C+large+feature+vectors%2C+and+the+role+of+linguistic+analysis%2E+Proceedings+of+the+20th+international+conference+on+Com) 6. Alec Go, Richa Bhayani, and Lei Huang. 2009. Twitter sentiment classification using distant supervision. Technical report, Stanford. [find](https://scholar.google.com/scholar?q=Alec+Go%2C+Richa+Bhayani%2C+and+Lei+Huang%2E+2009%2E+Twitter+sentiment+classification+using+distant+supervision%2E+Technical+report%2C+Stanford%2E) 7. David Haussler. 1999. Convolution kernels on discrete structures. Technical report, University of California at Santa Cruz. [find](https://scholar.google.com/scholar?q=David+Haussler%2E+1999%2E+Convolution+kernels+on+discrete+structures%2E+Technical+report%2C+University+of+California+at+Santa+Cruz%2E) 8. M Hu and B Liu. 2004. Mining and summarizing customer reviews. KDD. [find](https://scholar.google.com/scholar?q=M+Hu+and+B+Liu%2E+2004%2E+Mining+and+summarizing+customer+reviews%2E+KDD%2E) 9. S M Kim and E Hovy. 2004. Determining the sentiment of opinions. Coling. [find](https://scholar.google.com/scholar?q=S+M+Kim+and+E+Hovy%2E+2004%2E+Determining+the+sentiment+of+opinions%2E+Coling%2E) [our card](https://md.research.elamur.ai/free/en/papers/kim-2004-determining-sentiment-opinions/) 10. Dan Klein and Christopher D. Manning. 2003. Accurate unlexicalized parsing. Proceedings of the 41st Meeting of the Association for Computational Linguistics, pages 423–430. [find](https://scholar.google.com/scholar?q=Dan+Klein+and+Christopher+D%2E+Manning%2E+2003%2E+Accurate+unlexicalized+parsing%2E+Proceedings+of+the+41st+Meeting+of+the+Association+for+Computational+Linguistics%2C+pages+423%E2%80%93430%2E) [our card](https://md.research.elamur.ai/free/en/papers/aclbulk-2003-klein-accurate-unlexicalized-parsing/) 11. Alessandro Moschitti. 2006. Efficient convolution kernels for dependency and constituent syntactic trees. In Proceedings of the 17th European Conference on Machine Learning. [find](https://scholar.google.com/scholar?q=Alessandro+Moschitti%2E+2006%2E+Efficient+convolution+kernels+for+dependency+and+constituent+syntactic+trees%2E+In+Proceedings+of+the+17th+European+Conference+on+Machine+Learning%2E) 12. Alexander Pak and Patrick Paroubek. 2010. Twitter as a corpus for sentiment analysis and opinion mining. Proceedings of LREC. [find](https://scholar.google.com/scholar?q=Alexander+Pak+and+Patrick+Paroubek%2E+2010%2E+Twitter+as+a+corpus+for+sentiment+analysis+and+opinion+mining%2E+Proceedings+of+LREC%2E) [our card](https://md.research.elamur.ai/free/en/papers/pak-2010-twitter-as-corpus-sentiment-analysis/) 13. B. Pang and L. Lee. 2004. A sentimental education: Sentiment analysis using subjectivity analysis using subjectivity summarization based on minimum cuts. ACL. [find](https://scholar.google.com/scholar?q=B%2E+Pang+and+L%2E+Lee%2E+2004%2E+A+sentimental+education%3A+Sentiment+analysis+using+subjectivity+analysis+using+subjectivity+summarization+based+on+minimum+cuts%2E+ACL%2E) 14. P. Turney. 2002. Thumbs up or thumbs down? semantic orientation applied to unsupervised classification of reviews. ACL. [find](https://scholar.google.com/scholar?q=P%2E+Turney%2E+2002%2E+Thumbs+up+or+thumbs+down%3F+semantic+orientation+applied+to+unsupervised+classification+of+reviews%2E+ACL%2E) [our card](https://md.research.elamur.ai/free/en/papers/turney-2002-thumbs-up-or-thumbs-down/) 15. C M Whissel. 1989. The dictionary of Affect in Language. Emotion: theory research and experience, Acad press London. [find](https://scholar.google.com/scholar?q=C+M+Whissel%2E+1989%2E+The+dictionary+of+Affect+in+Language%2E+Emotion%3A+theory+research+and+experience%2C+Acad+press+London%2E) 16. T. Wilson, J. Wiebe, and P. Hoffman. 2005. Recognizing contextual polarity in phrase level sentiment analysis. ACL. [find](https://scholar.google.com/scholar?q=T%2E+Wilson%2C+J%2E+Wiebe%2C+and+P%2E+Hoffman%2E+2005%2E+Recognizing+contextual+polarity+in+phrase+level+sentiment+analysis%2E+ACL%2E) [our card](https://md.research.elamur.ai/free/en/papers/wilson-2005-recognizing-contextual-polarity-phrase-level/)