---
title: "Enhanced Sentiment Learning Using Twitter — Davidov 2010"
description: "The paper tackles fine-grained sentiment classification of very short texts."
authors: ["Dmitry Davidov", "Oren Tsur", "Ari Rappoport"]
year: 2010
venue: "Coling 2010: Posters"
source: "https://aclanthology.org/C10-2028/"
licence: "CC BY-NC-SA 3.0 (ACL Anthology, before 2016)"
licence_url: "https://creativecommons.org/licenses/by-nc-sa/3.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/davidov-2010-enhanced-sentiment-learning-twitter-hashtags/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/davidov-2010-enhanced-sentiment-learning-twitter-hashtags/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2010](https://md.research.elamur.ai/free/en/papers/2010/) / Enhanced Sentiment Learning Using Twitter Hashtags and Smileys

# Enhanced Sentiment Learning Using Twitter Hashtags and Smileys

**Authors:** [Dmitry Davidov](https://md.research.elamur.ai/free/en/authors/dmitry-davidov/), [Oren Tsur](https://md.research.elamur.ai/free/en/authors/oren-tsur/), [Ari Rappoport](https://md.research.elamur.ai/free/en/authors/ari-rappoport/) · **Year:** 2010 · **Venue:** Coling 2010: Posters · **Licence:** [CC BY-NC-SA 3.0 (ACL Anthology, before 2016)](https://creativecommons.org/licenses/by-nc-sa/3.0/)

**Links:** [Publisher's page](https://aclanthology.org/C10-2028/) · [PDF](https://aclanthology.org/C10-2028.pdf) · [OpenAlex](https://openalex.org/W2112251034)

**Citations:** 688 citations (OpenAlex, 2026-10-14) · [88](https://md.research.elamur.ai/free/en/papers/davidov-2010-enhanced-sentiment-learning-twitter-hashtags/cited-by/) in our library, of the 140884 papers whose bibliographies we read

## Summary
The paper tackles fine-grained sentiment classification of very short texts. It moves beyond positive/negative polarity to dozens of mood and opinion types, and it avoids manual annotation. The authors treat Twitter users' own markup as free supervision: 50 sentiment-bearing hashtags and 15 ASCII smileys serve as class labels for tweets. Each tweet becomes one feature vector combining four feature groups: single words, 2–5-word n-grams, surface patterns built from high-frequency words with content-word slots, and simple punctuation and length statistics. A kNN-style classifier with outlier removal assigns the label. The authors measure each feature group's contribution and confirm that every group helps. Mechanical Turk judges agreed with the predicted labels for most untagged tweets, and agreement was higher for smileys than for hashtags. Two techniques, tag co-occurrence and feature overlap, are used to explore how sentiment types relate to each other. The work shows that noisy user-generated tags can replace costly hand-labelled sentiment lexicons or datasets.

## Key points
- Labels come from 50 Twitter hashtags and 15 smileys. Tweets carrying more than one label are excluded from training to reduce ambiguity.
- **Features:**
  - Words and n-grams are weighted by inverse corpus count.
  - Features found in under 0.5% of training sentences are dropped.
  - Patterns contain 2–6 HFWs and 1–5 CW slots and must begin and end with an HFW. The HFW/CW frequency thresholds are 100 and 1000 words per million.
  - Partial pattern matches get reduced scores (α = γ = 0.1).
- **Classifier:** kNN with k = 10 and Euclidean distance. Only training vectors sharing at least one feature are compared. Neighbours farther than twice the mean distance are discarded before the majority vote.
- **Human evaluation:** Four MTurk judges rated each task set, and five control items filtered out unreliable workers.
  - Smiley labels were judged correct 84% of the time, versus 77% for hashtags.
  - Judges found no fitting tag in 6% (smileys) and 10% (hashtags) of cases.
  - Inter-judge agreement was κ = 0.41.
- Tags with opposite meanings (e.g., #sad and #happy) co-occur surprisingly often. Feature-overlap statistics show the same trends as direct co-occurrence, though the shared-feature percentages are higher.

## Interesting ideas
- **User markup as distant supervision.** Hashtags, emoji, or forum tags can act as noisy labels for training a multi-class classifier cheaply. This transfers to any language or platform where such labels exist, including Ukrainian social media.
- **Graded partial pattern matching.** Exact, sparse, and incomplete matches receive progressively smaller weights. This reduces sparsity for long patterns in short texts and could suit any short-text pattern classifier.
- **Staged feature selection.** Frequent words are found first, then only n-grams built from them, then only patterns built from those. This keeps feature extraction affordable on large corpora.
- **Control questions in crowdsourcing.** Each task set includes a few unambiguous gold items, and workers who miss them are excluded. This is a simple quality filter for any annotation campaign.
- **Feature overlap as a label-similarity measure.** Label relatedness is estimated from shared features rather than direct co-occurrence. This works even when co-occurrence data is sparse and could support clustering of categories in other taxonomies.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

Fields: [Natural language processing](https://md.research.elamur.ai/free/en/shelves/nlp/)

Topics: [Sentiment & emotion](https://md.research.elamur.ai/free/en/topics/sentiment-emotion/)

## Cite

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{davidov2010enhanced,
  title = {{Enhanced Sentiment Learning Using Twitter Hashtags and Smileys}},
  author = {Davidov, Dmitry and Tsur, Oren and Rappoport, Ari},
  booktitle = {Coling 2010: Posters},
  year = {2010},
  url = {https://aclanthology.org/C10-2028/}
}
```

- [← previous Automatic Evaluation of Topic Coherence](https://md.research.elamur.ai/free/en/papers/newman-2010-automatic-evaluation-topic-coherence/)
- [next → Open Information Extraction Using Wikipedia](https://md.research.elamur.ai/free/en/papers/wu-2010-open-information-extraction-wikipedia/)
- [↑ back to papers of 2010](https://md.research.elamur.ai/free/en/papers/2010/)

## References

26 references, 9 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Akkaya, Cem, Janyce Wiebe, and Rada Mihalcea. 2009. Subjectivity word sense disambiguation. In EMNLP. [find](https://scholar.google.com/scholar?q=Akkaya%2C+Cem%2C+Janyce+Wiebe%2C+and+Rada+Mihalcea%2E+2009%2E+Subjectivity+word+sense+disambiguation%2E+In+EMNLP%2E)
2. Andreevskaia, A. and S. Bergler. 2006. Mining wordnet for fuzzy sentiment: Sentiment tag extraction from wordnet glosses. In EACL. [find](https://scholar.google.com/scholar?q=Andreevskaia%2C+A%2E+and+S%2E+Bergler%2E+2006%2E+Mining+wordnet+for+fuzzy+sentiment%3A+Sentiment+tag+extraction+from+wordnet+glosses%2E+In+EACL%2E)
3. Balog, Krisztian, Gilad Mishne, and Maarten de Rijke. 2006. Why are they excited? identifying and explaining spikes in blog mood levels. In EACL. [find](https://scholar.google.com/scholar?q=Balog%2C+Krisztian%2C+Gilad+Mishne%2C+and+Maarten+de+Rijke%2E+2006%2E+Why+are+they+excited%3F+identifying+and+explaining+spikes+in+blog+mood+levels%2E+In+EACL%2E)
4. Bloom, Kenneth, Navendu Garg, and Shlomo Argamon. 2007. Extracting appraisal expressions. In HLT/NAACL. [find](https://scholar.google.com/scholar?q=Bloom%2C+Kenneth%2C+Navendu+Garg%2C+and+Shlomo+Argamon%2E+2007%2E+Extracting+appraisal+expressions%2E+In+HLT%2FNAACL%2E)
5. Davidov, D. and A. Rappoport. 2006. Efficient unsupervised discovery of word categories using symmetric patterns and high frequency words. In COLING-ACL. [find](https://scholar.google.com/scholar?q=Davidov%2C+D%2E+and+A%2E+Rappoport%2E+2006%2E+Efficient+unsupervised+discovery+of+word+categories+using+symmetric+patterns+and+high+frequency+words%2E+In+COLING%2DACL%2E)
6. Davidov, D. and A. Rappoport. 2008. Unsupervised discovery of generic relationships using pattern clusters and its evaluation by automatically generated sat analogy questions. In ACL. [find](https://scholar.google.com/scholar?q=Davidov%2C+D%2E+and+A%2E+Rappoport%2E+2008%2E+Unsupervised+discovery+of+generic+relationships+using+pattern+clusters+and+its+evaluation+by+automatically+generated+sat+analogy+questions%2E+In+ACL%2E)
7. Davidov, D., O. Tsur, and A. Rappoport. 2010. Semi-supervised recognition of sarcastic sentences in twitter and amazon. In CoNLL. [find](https://scholar.google.com/scholar?q=Davidov%2C+D%2E%2C+O%2E+Tsur%2C+and+A%2E+Rappoport%2E+2010%2E+Semi%2Dsupervised+recognition+of+sarcastic+sentences+in+twitter+and+amazon%2E+In+CoNLL%2E) [our card](https://md.research.elamur.ai/free/en/papers/davidov-2010-semi-supervised-recognition-sarcasm-twitter/)
8. Esuli, Andrea and Fabrizio Sebastiani. 2006. Sentiwordnet: A publicly available lexical resource for opinion mining. In LREC. [find](https://scholar.google.com/scholar?q=Esuli%2C+Andrea+and+Fabrizio+Sebastiani%2E+2006%2E+Sentiwordnet%3A+A+publicly+available+lexical+resource+for+opinion+mining%2E+In+LREC%2E) [our card](https://md.research.elamur.ai/free/en/papers/esuli-2006-sentiwordnet-publicly-available-lexical-resource/)
9. Jansen, B.J., M. Zhang, K. Sobel, and A. Chowdury. 2009. Twitter power: Tweets as electronic word of mouth. Journal of the American Society for Information Science and Technology. [find](https://scholar.google.com/scholar?q=Jansen%2C+B%2EJ%2E%2C+M%2E+Zhang%2C+K%2E+Sobel%2C+and+A%2E+Chowdury%2E+2009%2E+Twitter+power%3A+Tweets+as+electronic+word+of+mouth%2E+Journal+of+the+American+Society+for+Information+Science+and+Technology%2E)
10. Kim, S.M. and E. Hovy. 2004. Determining the sentiment of opinions. In COLING. [find](https://scholar.google.com/scholar?q=Kim%2C+S%2EM%2E+and+E%2E+Hovy%2E+2004%2E+Determining+the+sentiment+of+opinions%2E+In+COLING%2E) [our card](https://md.research.elamur.ai/free/en/papers/kim-2004-determining-sentiment-opinions/)
11. McDonald, Ryan, Kerry Hannan, Tyler Neylon, Mike Wells, and Jeff Reynar. 2007. Structured models for fine-to-coarse sentiment analysis. In ACL. [find](https://scholar.google.com/scholar?q=McDonald%2C+Ryan%2C+Kerry+Hannan%2C+Tyler+Neylon%2C+Mike+Wells%2C+and+Jeff+Reynar%2E+2007%2E+Structured+models+for+fine%2Dto%2Dcoarse+sentiment+analysis%2E+In+ACL%2E)
12. Melville, Prem, Wojciech Gryc, and Richard D. Lawrence. 2009. Sentiment analysis of blogs by combining lexical knowledge with text classification. In KDD. ACM. [find](https://scholar.google.com/scholar?q=Melville%2C+Prem%2C+Wojciech+Gryc%2C+and+Richard+D%2E+Lawrence%2E+2009%2E+Sentiment+analysis+of+blogs+by+combining+lexical+knowledge+with+text+classification%2E+In+KDD%2E+ACM%2E)
13. Mihalcea, Rada and Hugo Liu. 2006. A corpus-based approach to finding happiness. In In AAAI 2006 Symposium on Computational Approaches to Analysing Weblogs. AAAI Press. [find](https://scholar.google.com/scholar?q=Mihalcea%2C+Rada+and+Hugo+Liu%2E+2006%2E+A+corpus%2Dbased+approach+to+finding+happiness%2E+In+In+AAAI+2006+Symposium+on+Computational+Approaches+to+Analysing+Weblogs%2E+AAAI+Press%2E)
14. Mishne, Gilad. 2005. Experiments with mood classification in blog posts. In Proceedings of the 1st Workshop on Stylistic Analysis Of Text. [find](https://scholar.google.com/scholar?q=Mishne%2C+Gilad%2E+2005%2E+Experiments+with+mood+classification+in+blog+posts%2E+In+Proceedings+of+the+1st+Workshop+on+Stylistic+Analysis+Of+Text%2E)
15. Riloff, Ellen. 2003. Learning extraction patterns for subjective expressions. In EMNLP. [find](https://scholar.google.com/scholar?q=Riloff%2C+Ellen%2E+2003%2E+Learning+extraction+patterns+for+subjective+expressions%2E+In+EMNLP%2E) [our card](https://md.research.elamur.ai/free/en/papers/riloff-2003-learning-extraction-patterns-subjective-expressions/)
16. Strapparava, Carlo and Rada Mihalcea. 2008. Learning to identify emotions in text. In SAC. [find](https://scholar.google.com/scholar?q=Strapparava%2C+Carlo+and+Rada+Mihalcea%2E+2008%2E+Learning+to+identify+emotions+in+text%2E+In+SAC%2E)
17. Titov, Ivan and Ryan McDonald. 2008a. A joint model of text and aspect ratings for sentiment summarization. In ACL/HLT, June. [find](https://scholar.google.com/scholar?q=Titov%2C+Ivan+and+Ryan+McDonald%2E+2008a%2E+A+joint+model+of+text+and+aspect+ratings+for+sentiment+summarization%2E+In+ACL%2FHLT%2C+June%2E) [our card](https://md.research.elamur.ai/free/en/papers/titov-2008-joint-model-text-aspect-ratings/)
18. Titov, Ivan and Ryan McDonald. 2008b. Modeling online reviews with multi-grain topic models. In WWW, pages 111–120, New York, NY, USA. ACM. [find](https://scholar.google.com/scholar?q=Titov%2C+Ivan+and+Ryan+McDonald%2E+2008b%2E+Modeling+online+reviews+with+multi%2Dgrain+topic+models%2E+In+WWW%2C+pages+111%E2%80%93120%2C+New+York%2C+NY%2C+USA%2E+ACM%2E)
19. Tsur, Oren, Dmitry Davidov, and Ari Rappoport. 2010. Icwsm – a great catchy name: Semi-supervised recognition of sarcastic sentences in product reviews. In AAAI-ICWSM. [find](https://scholar.google.com/scholar?q=Tsur%2C+Oren%2C+Dmitry+Davidov%2C+and+Ari+Rappoport%2E+2010%2E+Icwsm+%E2%80%93+a+great+catchy+name%3A+Semi%2Dsupervised+recognition+of+sarcastic+sentences+in+product+reviews%2E+In+AAAI%2DICWSM%2E)
20. Turney, Peter D. 2002. Thumbs up or thumbs down? semantic orientation applied to unsupervised classification of reviews. In ACL ’02, volume 40. [find](https://scholar.google.com/scholar?q=Turney%2C+Peter+D%2E+2002%2E+Thumbs+up+or+thumbs+down%3F+semantic+orientation+applied+to+unsupervised+classification+of+reviews%2E+In+ACL+%E2%80%9902%2C+volume+40%2E) [our card](https://md.research.elamur.ai/free/en/papers/turney-2002-thumbs-up-or-thumbs-down/)
21. Whitelaw, Casey, Navendu Garg, and Shlomo Argamon. 2005. Using appraisal groups for sentiment analysis. In CIKM. [find](https://scholar.google.com/scholar?q=Whitelaw%2C+Casey%2C+Navendu+Garg%2C+and+Shlomo+Argamon%2E+2005%2E+Using+appraisal+groups+for+sentiment+analysis%2E+In+CIKM%2E)
22. Wiebe, Janyce and Rada Mihalcea. 2006. Word sense and subjectivity. In COLING/ACL, Sydney, AUS. [find](https://scholar.google.com/scholar?q=Wiebe%2C+Janyce+and+Rada+Mihalcea%2E+2006%2E+Word+sense+and+subjectivity%2E+In+COLING%2FACL%2C+Sydney%2C+AUS%2E)
23. Wiebe, Janyce M. 2000. Learning subjective adjectives from corpora. In AAAI. [find](https://scholar.google.com/scholar?q=Wiebe%2C+Janyce+M%2E+2000%2E+Learning+subjective+adjectives+from+corpora%2E+In+AAAI%2E)
24. Wilson, Theresa, Janyce Wiebe, and Paul Hoffmann. 2005. Recognizing contextual polarity in phrase-level sentiment analysis. In HLT/EMNLP. [find](https://scholar.google.com/scholar?q=Wilson%2C+Theresa%2C+Janyce+Wiebe%2C+and+Paul+Hoffmann%2E+2005%2E+Recognizing+contextual+polarity+in+phrase%2Dlevel+sentiment+analysis%2E+In+HLT%2FEMNLP%2E) [our card](https://md.research.elamur.ai/free/en/papers/wilson-2005-recognizing-contextual-polarity-phrase-level/)
25. Wilson, Theresa, Janyce Wiebe, and Paul Hoffmann. 2009. Recognizing contextual polarity: An exploration of features for phrase-level sentiment analysis. Computational Linguistics, 35(3):399–433. [find](https://scholar.google.com/scholar?q=Wilson%2C+Theresa%2C+Janyce+Wiebe%2C+and+Paul+Hoffmann%2E+2009%2E+Recognizing+contextual+polarity%3A+An+exploration+of+features+for+phrase%2Dlevel+sentiment+analysis%2E+Computational+Linguistics%2C+35%283%29%3A399%E2%80%93433%2E) [our card](https://md.research.elamur.ai/free/en/papers/wilson-2009-recognizing-contextual-polarity-exploration-features/)
26. Yu, Hong and Vasileios Hatzivassiloglou. 2003. Towards answering opinion questions: Separating facts from opinions and identifying the polarity of opinion sentences. In EMNLP. [find](https://scholar.google.com/scholar?q=Yu%2C+Hong+and+Vasileios+Hatzivassiloglou%2E+2003%2E+Towards+answering+opinion+questions%3A+Separating+facts+from+opinions+and+identifying+the+polarity+of+opinion+sentences%2E+In+EMNLP%2E) [our card](https://md.research.elamur.ai/free/en/papers/yu-2003-answering-opinion-questions-separating-facts/)