---
title: "An Arabic Twitter Corpus for Subjectivity and Sentiment Analysis"
description: "Sentiment work on Arabic lags far behind English. 19 references, cited by 32 papers here."
authors: ["Eshrag Refaee", "Verena Rieser"]
year: 2014
venue: "Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC'14)"
source: "https://aclanthology.org/L14-1280/"
licence: "CC BY-NC 4.0 (ELRA, LREC proceedings)"
licence_url: "https://creativecommons.org/licenses/by-nc/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/refaee-2014-arabic-twitter-corpus-subjectivity-sentiment/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/refaee-2014-arabic-twitter-corpus-subjectivity-sentiment/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2014](https://md.research.elamur.ai/free/en/papers/2014/page/2/) / An Arabic Twitter Corpus for Subjectivity and Sentiment Analysis

# An Arabic Twitter Corpus for Subjectivity and Sentiment Analysis

**Authors:** [Eshrag Refaee](https://md.research.elamur.ai/free/en/authors/eshrag-refaee/) and [Verena Rieser](https://md.research.elamur.ai/free/en/authors/verena-rieser/) · **Year:** 2014 · **Venue:** [Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC'14)](https://md.research.elamur.ai/free/en/venues/lrec/) ([LREC 2014](https://md.research.elamur.ai/free/en/venues/lrec/2014/)) · **Publisher:** [European Language Resources Association (ELRA)](https://md.research.elamur.ai/free/en/publishers/european-language-resources-association-elra/) · **Licence:** [CC BY-NC 4.0 (ELRA, LREC proceedings)](https://creativecommons.org/licenses/by-nc/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/L14-1280/) · [PDF](https://aclanthology.org/L14-1280.pdf) · [OpenAlex](https://openalex.org/W2250522473)

**Citations:** 140 citations (OpenAlex, 2026-10-14) · [32](https://md.research.elamur.ai/free/en/papers/refaee-2014-arabic-twitter-corpus-subjectivity-sentiment/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
Sentiment work on Arabic lags far behind English. Part of the reason is the language's rich morphology. Another part is that annotated social-media data was rarely shared. Refaee and Rieser address this gap by building a hand-annotated collection of Arabic tweets for analysing subjectivity and sentiment (SSA).

They collected tweets through the Twitter Search API using topic-oriented queries about products, public figures, sport, social issues and similar subjects. The data come from two separate time windows, giving a development set and a test set. Two native speakers labelled each tweet as positive, negative, mixed or neutral and reached κ = 0.816.

Each tweet also carries automatically extracted features:
- morphology from MADA+TOKAN, plus a flag marking whether analysis succeeded,
- word n-grams,
- hits in a merged sentiment lexicon,
- emoticons,
- social-signal cues.

Comparing the two time slices shows strong topic drift: frequent content words barely overlap. In related work, classifiers trained on one slice generalised poorly to the other. The corpus matters as an early publicly released Arabic SSA resource and as evidence that Twitter's shifting topics undermine word-based models.

## Key points
- The corpus has 8,868 gold tweets. The development set holds 7,503 tweets (20 Jan–21 Feb 2014) and the test set 1,365 (6–15 Nov 2013). Observed agreement was 91.74%, with weighted κ = 0.816.
- Polar tweets are those labelled positive or negative. Any tweet that even one annotator marked other/uncertain was discarded, as were ads and redundant tweets given the skip label.
- The combined sentiment lexicon has 4,422 entries, built from three sources:
  - ArabSenti,
  - MPQA machine-translated into Arabic and hand-corrected (2,627 entries),
  - a new 484-word dialectal lexicon, which is released.
- Morphological features come from MADA+TOKAN v3.2. Because that tool covers only MSA, a "has morphological analysis" feature marks dialectal or misspelled words it cannot handle.
- The release goes through ELRA in CSV and ARFF formats. Tweets appear as IDs only, for privacy reasons. The training portion has 96,493 tokens and 26,724 unique word types.

## Interesting ideas
- Annotators had to state a reason for each label, and those reasons were used to refine the scheme in rounds on a small subset; this also shows when people agree on a label but for different reasons.
- Instead of silently trusting an analyser outside its domain, a separate feature records whether it produced any analysis at all, so downstream models know when the morphology is unreliable.
- Training and test data were drawn from separate time windows, and comparing their frequent content words revealed topic drift that random splits would hide.
- When translating an English polarity lexicon, the authors corrected entries whose context-free polarity changed in translation, for instance a phrase that is negative in English but neutral in Arabic.
- An error analysis found that emoticon-based distant labels became noisy because emoticon direction is ambiguous in right-to-left text; even "obvious" weak signals depend on the writing conventions of the text.

## For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

- **I1** (learn): LLM teacher tiers and human checks could return a short justification with each silver label, so label-level agreement can be separated from reason-level agreement before data enters training.
- **I2** (tokens): the en tagger and lemmatizer could expose an explicit "no analysis / out-of-lexicon" flag per token, letting later stages lower confidence instead of acting on a guessed lemma.
- **I3** (learn): evaluation sets could be split by source or period rather than at random, with a vocabulary-overlap check acting as a negative control against topic memorisation.
- **I4** (world): when compiling value or attitude knowledge from external resources, entries should be checked for polarity that depends on context, so the ontology records default evaluations rather than connotations of a particular usage.
- **I5** (frames): irony and speech-act cues such as emoticons or laugh tokens should be normalised for orientation and treated as weak evidence, never as decisive labels for stance.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Heriot-Watt University](https://md.research.elamur.ai/free/en/universities/heriot-watt-university/) — University, Edinburgh, United Kingdom · 17 works · 5,994 citations

📚 Shelves (6): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 18,614 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 17,965 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 17,450 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 17,296 works · [💭 Psychology](https://md.research.elamur.ai/free/en/shelves/psychology/) · 1,649 works · [💭 Experimental and Cognitive Psychology](https://md.research.elamur.ai/free/en/shelves/psychology/experimental-and-cognitive-psychology/) · 930 works

🧭 Topics (3): [😊 Sentiment & emotion](https://md.research.elamur.ai/free/en/topics/sentiment-emotion/) · 1,035 works · [Semitic languages](https://md.research.elamur.ai/free/en/topics/semitic-languages/) · 3,066 works · [🧩 Morphology](https://md.research.elamur.ai/free/en/topics/morphology/) · 420 works

## Cite

Cite as: Eshrag Refaee and Verena Rieser. 2014. An Arabic Twitter Corpus for Subjectivity and Sentiment Analysis. Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC'14).

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{refaee2014arabic,
  title = {{An Arabic Twitter Corpus for Subjectivity and Sentiment Analysis}},
  author = {Eshrag Refaee and Verena Rieser},
  booktitle = {Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC'14)},
  year = {2014},
  doi = {10.63317/3zem5qb5stkg},
  url = {https://doi.org/10.63317/3zem5qb5stkg}
}
```

- [← previous An Enhanced Lesk Word Sense Disambiguation Algorithm through a Distributional Semantic Model](https://md.research.elamur.ai/free/en/papers/basile-2014-enhanced-lesk-word-sense-disambiguation/)
- [next → Linguistically debatable or just plain wrong?](https://md.research.elamur.ai/free/en/papers/plank-2014-linguistically-debatable-or-just-plain/)
- [↑ back to papers of 2014](https://md.research.elamur.ai/free/en/papers/2014/page/2/)

## References

19 references, 10 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Abdul-Mageed, M. and Diab, M. (2012a). [Awatif: A multi-genre corpus for modern standard arabic subjectivity and sentiment analysis](https://md.research.elamur.ai/free/en/papers/mageed-2012-awatif-multi-genre-corpus-modern/). In Proceedings of the Eight International Conference on Language Resources and Evaluation (LREC’12), Istanbul, Turkey. European Language Resources Association (ELRA).
2. Abdul-Mageed, M. and Diab, M. (2012b). [Awatif: A multi-genre corpus for modern standard arabic subjectivity and sentiment analysis](https://md.research.elamur.ai/free/en/papers/mageed-2012-awatif-multi-genre-corpus-modern/). In Proceedings of the Eight International Conference on Language Resources and  Example Ð Q m ' (respect) Gen: m mod: i num: per: stt: s 3 na vox: a Evaluation (LREC’12), Istanbul, Turkey. European Language Resources Association (ELRA).
3. Abdul-Mageed, M., Diab, M. T., and Korayem, M. (2011). [Subjectivity and sentiment analysis of modern standard arabic](https://md.research.elamur.ai/free/en/papers/mageed-2011-subjectivity-sentiment-analysis-modern-standard/). In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies: short papers - Volume 2, HLT ’11, pages 587–591, Stroudsburg, PA, USA. Association for Computational Linguistics.
4. Al-Sabbagh, R. and Girju, R. (2012). [Yadac: Yet another dialectal arabic corpus](https://md.research.elamur.ai/free/en/papers/sabbagh-2012-yadac-yet-another-dialectal-arabic/). In Proceedings of the Eight International Conference on Language Resources and Evaluation (LREC’12), Istanbul, Turkey. European Language Resources Association (ELRA).
5. Bifet, A. and Frank, E. (2010). Sentiment knowledge discovery in twitter streaming data. In Discovery Science, pages 1–15. Springer. [find](https://scholar.google.com/scholar?q=Bifet%2C+A%2E+and+Frank%2C+E%2E+%282010%29%2E+Sentiment+knowledge+discovery+in+twitter+streaming+data%2E+In+Discovery+Science%2C+pages+1%E2%80%9315%2E+Springer%2E)
6. Carletta, J. (1996). [Assessing agreement on classification tasks: the kappa statistic](https://md.research.elamur.ai/free/en/papers/carletta-1996-assessing-agreement-classification-tasks-kappa/). Computational Linguistics, 22(2):249–254. [↗](https://doi.org/10.48550/arxiv.cmp-lg/9602004)
7. Cohen, J. (1960). A coefficient of agreement for nominal scales. Educational and psychological measurement, 20(1):37–46. [↗](https://doi.org/10.1177/001316446002000104)
8. Diab, M., Habash, N., Rambow, O., Altantawy, M., and Benajiba, Y. (2010). COLABA: Arabic dialect annotation and processing. In Proceedings of the LREC Workshop on Semitic Language Processing). European Language Resources Association (ELRA). [find](https://scholar.google.com/scholar?q=Diab%2C+M%2E%2C+Habash%2C+N%2E%2C+Rambow%2C+O%2E%2C+Altantawy%2C+M%2E%2C+and+Benajiba%2C+Y%2E+%282010%29%2E+COLABA%3A+Arabic+dialect+annotation+and+processing%2E+In+Proceedings+of+the+LREC+Workshop+on+Semitic+Language+Processing%29%2E+Europea)
9. Eisenstein, J. (2013). [What to do about bad language on the internet](https://md.research.elamur.ai/free/en/papers/eisenstein-2013-what-do-about-bad-language/). In Proceedings of NAACL-HLT, pages 359– 369.
10. Habash, N. and Rambow, O. (2005). [Arabic Tokenization, Part-of-Speech Tagging and Morphological Disambiguation in One Fell Swoop](https://md.research.elamur.ai/free/en/papers/habash-2005-arabic-tokenization-part-speech-tagging/). In Proceedings of the 43rd Annual Meeting of the Association for Computational Linguistics (ACL’05), pages 573–580, Ann Arbor, Michigan, June. Association for Computational Linguistics.
11. Habash, N. (2010). Introduction to Arabic Natural Language Processing. Synthesis Lectures on Human Language Technologies. Morgan & Claypool Publishers. [find](https://scholar.google.com/scholar?q=Habash%2C+N%2E+%282010%29%2E+Introduction+to+Arabic+Natural+Language+Processing%2E+Synthesis+Lectures+on+Human+Language+Technologies%2E+Morgan+%26+Claypool+Publishers%2E)
12. Liu, B. (2012). Sentiment Analysis and Opinion Mining. Synthesis Lectures on Human Language Technologies. Morgan & Claypool Publishers. [↗](https://doi.org/10.2200/s00416ed1v01y201204hlt016)
13. Mourad, A. and Darwish, K. (2013). [Subjectivity and sentiment analysis of modern standard arabic and arabic microblogs](https://md.research.elamur.ai/free/en/papers/mourad-2013-subjectivity-sentiment-analysis-modern-standard/). WASSA 2013, page 55.
14. Nizar Habash, O. R. and Roth, R. (2009). MADA+TOKAN: A toolkit for arabic tokenization, diacritization, morphological disambiguation, pos tagging, stemming and lemmatization. In Choukri, K. and Maegaard, B., editors, Proceedings of the Second International Conference on Arabic Language Resources and Tools, Cairo, Egypt, April. The MEDAR Consortium. [find](https://scholar.google.com/scholar?q=Nizar+Habash%2C+O%2E+R%2E+and+Roth%2C+R%2E+%282009%29%2E+MADA%2BTOKAN%3A+A+toolkit+for+arabic+tokenization%2C+diacritization%2C+morphological+disambiguation%2C+pos+tagging%2C+stemming+and+lemmatization%2E+In+Choukri%2C+K%2E+and+Maegaa)
15. Refaee, E. and Rieser, V. (2014a). Can we read emotions from a smiley face? Emoticon-based distant supervision for subjectivity and sentiment analysis of arabic twitter feeds. In 5th International Workshop on Emotion, Social Signal, Sentiment & Linked Open Data. [find](https://scholar.google.com/scholar?q=Refaee%2C+E%2E+and+Rieser%2C+V%2E+%282014a%29%2E+Can+we+read+emotions+from+a+smiley+face%3F+Emoticon%2Dbased+distant+supervision+for+subjectivity+and+sentiment+analysis+of+arabic+twitter+feeds%2E+In+5th+International+Wor)
16. Refaee, E. and Rieser, V. (2014b). Subjectivity and sentiment analysis of arabic twitter feeds with limited resources. In 9th International Conference on Language Resources and Evaluation (LREC’14). [find](https://scholar.google.com/scholar?q=Refaee%2C+E%2E+and+Rieser%2C+V%2E+%282014b%29%2E+Subjectivity+and+sentiment+analysis+of+arabic+twitter+feeds+with+limited+resources%2E+In+9th+International+Conference+on+Language+Resources+and+Evaluation+%28LREC%E2%80%9914%29%2E)
17. Wilson, T., Wiebe, J., and Hoffmann, P. (2009). [Recognizing contextual polarity: An exploration of features for phrase-level sentiment analysis](https://md.research.elamur.ai/free/en/papers/wilson-2009-recognizing-contextual-polarity-exploration-features/). Computational Linguistics, 35(3):399–433. [↗](https://doi.org/10.1162/coli.08-012-r1-06-90)
18. Zaidan, O. F. and Callison-Burch, C. (2013). [Arabic dialect identification](https://md.research.elamur.ai/free/en/papers/zaidan-2014-arabic-dialect-identification/). Computational Linguistics. [↗](https://doi.org/10.1162/coli_a_00169)
19. Zhang, L., Ghosh, R., Dekhil, M., Hsu, M., and Liu, B. (2011). Combining lexiconbased and learning-based methods for twitter sentiment analysis. HP Laboratories, Technical Report HPL-2011, 89. [find](https://scholar.google.com/scholar?q=Zhang%2C+L%2E%2C+Ghosh%2C+R%2E%2C+Dekhil%2C+M%2E%2C+Hsu%2C+M%2E%2C+and+Liu%2C+B%2E+%282011%29%2E+Combining+lexiconbased+and+learning%2Dbased+methods+for+twitter+sentiment+analysis%2E+HP+Laboratories%2C+Technical+Report+HPL%2D2011%2C+89%2E)

*Updated 2 November 2026*