---
title: "Machine Learning-Based Model for Sentiment and Sarcasm"
description: "The paper addresses two related tasks on Arabic tweets: deciding whether a tweet is sarcastic, and… 14 references, cited by 5 papers here."
authors: ["Hamada Nayel", "Eslam Amer", "Aya Allam", "Hanya Abdallah"]
year: 2021
venue: "Proceedings of the Sixth Arabic Natural Language Processing Workshop"
source: "https://aclanthology.org/2021.wanlp-1.51/"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/nayel-2021-machine-learning-based-model-sentiment/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/nayel-2021-machine-learning-based-model-sentiment/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2021](https://md.research.elamur.ai/free/en/papers/2021/page/5/) / Machine Learning-Based Model for Sentiment and Sarcasm Detection

# Machine Learning-Based Model for Sentiment and Sarcasm Detection

**Authors:** [Hamada Nayel](https://md.research.elamur.ai/free/en/authors/hamada-nayel/), Eslam Amer, Aya Allam and Hanya Abdallah · **Year:** 2021 · **Venue:** [Proceedings of the Sixth Arabic Natural Language Processing Workshop](https://md.research.elamur.ai/free/en/venues/wanlp/) ([WANLP 2021](https://md.research.elamur.ai/free/en/venues/wanlp/2021/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2021.wanlp-1.51/) · [PDF](https://aclanthology.org/2021.wanlp-1.51.pdf) · [OpenAlex](https://openalex.org/W3154295458)

**Citations:** 20 citations (OpenAlex, 2026-10-14) · [5](https://md.research.elamur.ai/free/en/papers/nayel-2021-machine-learning-based-model-sentiment/cited-by/) in our library, of the 141,840 papers whose bibliographies we read

## Summary
The paper addresses two related tasks on Arabic tweets: deciding whether a tweet is sarcastic, and labelling its sentiment as positive, negative, or neutral. Sarcasm is hard because the literal wording differs from what the writer means. Arabic adds difficulty through rich morphology and many dialects alongside Modern Standard Arabic. The work was entered in a shared competition covering Arabic sentiment and sarcasm identification, using the ArSarcasm-v2 dataset. The pipeline has three stages. First, preprocessing re-encodes the data as UTF-8 and strips hashtags, emojis, numbers, and URLs. Second, unigram/bigram features are weighted with TF/IDF. Third, a document-level classifier is applied. In five-fold cross-validation, an SVM beat Complement Naïve Bayes, Naïve Bayes, and Linear Regression on both tasks, so it was chosen for the blind test. Official test scores were far lower than the development accuracies, and the authors call the results moderate. The paper matters mainly as a simple, reproducible classical baseline for Arabic sarcasm and sentiment. It also shows how optimistic accuracy-based development numbers can be on imbalanced data.

## Key points
- Data: ArSarcasm-v2 contains 12,548 labelled training tweets and a blind test set of 3,000 tweets. Each tweet also carries dialect information.
- The training set is strongly imbalanced:
  - Sarcasm: only 2,168 tweets (17.28%) are sarcastic.
  - Sentiment: 17.37% positive, 36.83% negative, and 45.80% neutral.
- Five-fold CV results (sentiment / sarcasm):
  - SVM: 85.55% / 84.22%
  - CNB: 83.47% / 82.97%
  - NB: 78.92% / 78.10%
  - LR: 65.89% / 64.03%
  - SGD was also tried, but no score is shown.
- Official metrics differ by task. Sarcasm is scored by F1 of the sarcastic class (F1-sarcastic). Sentiment is scored by the macro F-score over the positive and negative classes only (F-PN).
- Blind-test submissions scored accuracies of 0.7460 and 0.5980, and official scores of 0.5936 and 0.2440. The extracted table is garbled, so which score belongs to which task is unclear from this copy.

## Interesting ideas
- Score a skewed task by the F1 of the rare class you care about, not overall accuracy; this exposes systems that mostly predict the majority label.
- The sentiment metric averages only over the positive and negative classes and ignores neutral, so a model gains nothing by defaulting to "neutral."
- The large drop from cross-validation accuracy to blind-test official scores shows that development numbers on the same distribution can badly overstate quality.
- Sarcasm is framed as cases where the literal content and the intended meaning diverge.
- A TF/IDF n-gram SVM is a cheap, transparent bag-of-words reference that several classifiers were benchmarked against.

## For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

- **I1** (learn): Mova's evaluation of rare phenomena (irony, negation, unusual frames) should report F1 on the rare class itself so frequent easy cases cannot mask failures.
- **I2** (answer): QA and test scoring could likewise exclude an "unknown/no answer" fallback from the main score, so readers are rewarded for committing to correct answers.
- **I3** (learn): Mova should always keep a held-out blind set and a trivial-baseline negative control, and report the gap between them alongside development results.
- **I4** (frames): crate prag could flag irony when the literal frame's evaluation clashes with world or context expectations, giving an explainable signal rather than an n-gram guess.
- **I5** (learn): such a model is useful as a negative-control baseline that Mova's structured, tree-and-frame approach to irony and sentiment must clearly beat.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 9,085 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 8,888 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 9,296 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 9,206 works

🧭 Topics (2): [😊 Sentiment & emotion](https://md.research.elamur.ai/free/en/topics/sentiment-emotion/) · 609 works · [Semitic languages](https://md.research.elamur.ai/free/en/topics/semitic-languages/) · 1,485 works

## Cite

Cite as: Hamada Nayel, Eslam Amer, Aya Allam and Hanya Abdallah. 2021. Machine Learning-Based Model for Sentiment and Sarcasm Detection. Proceedings of the Sixth Arabic Natural Language Processing Workshop.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{nayel2021machine,
  title = {{Machine Learning-Based Model for Sentiment and Sarcasm Detection}},
  author = {Hamada Nayel, Eslam Amer, Aya Allam and Hanya Abdallah},
  booktitle = {Proceedings of the Sixth Arabic Natural Language Processing Workshop},
  year = {2021},
  url = {https://aclanthology.org/2021.wanlp-1.51/}
}
```

- [← previous Improving Arabic Diacritization with Regularized Decoding and Adversarial Training](https://md.research.elamur.ai/free/en/papers/qin-2021-improving-arabic-diacritization-regularized-decoding/)
- [next → OntoGUM: Evaluating Contextualized SOTA Coreference Resolution on 12 More Genres](https://md.research.elamur.ai/free/en/papers/zhu-2021-ontogum-evaluating-contextualized-sota-coreference/)
- [↑ back to papers of 2021](https://md.research.elamur.ai/free/en/papers/2021/page/5/)

## References

14 references, 2 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Muhammad Abdul-Mageed, Mona Diab, and Mohammed Korayem. 2011. Subjectivity and sentiment analysis of Modern Standard Arabic. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, pages 587–591, Portland, Oregon, USA. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Muhammad+Abdul%2DMageed%2C+Mona+Diab%2C+and+Mohammed+Korayem%2E+2011%2E+Subjectivity+and+sentiment+analysis+of+Modern+Standard+Arabic%2E+In+Proceedings+of+the+49th+Annual+Meeting+of+the+Association+for+Computatio)
2. Hamada A. Nayel and Shashirekha H. L. 2019. DEEP at HASOC2019: A machine learning framework for hate speech and offensive language detection. In Working Notes of FIRE 2019 - Forum for Information Retrieval Evaluation, Kolkata, India, December 12-15, 2019, volume 2517 of CEUR Workshop Proceedings, pages 336–343. CEUR-WS.org. [find](https://scholar.google.com/scholar?q=Hamada+A%2E+Nayel+and+Shashirekha+H%2E+L%2E+2019%2E+DEEP+at+HASOC2019%3A+A+machine+learning+framework+for+hate+speech+and+offensive+language+detection%2E+In+Working+Notes+of+FIRE+2019+%2D+Forum+for+Information+Retr)
3. Ibrahim Abu Farha, Wajdi Zaghouani, and Walid Magdy. 2021. [Overview of the WANLP 2021 Shared Task on Sarcasm and Sentiment Detection in Arabic](https://md.research.elamur.ai/free/en/papers/farha-2021-overview-wanlp-2021-shared-task/). In Proceedings of the Sixth Arabic Natural Language Processing Workshop.
4. Hamada A. Nayel, Walaa Medhat, and Metwally Rashad. 2019. BENHA@IDAT: Improving Irony Detection in Arabic Tweets using Ensemble Approach. In Working Notes of FIRE 2019 - Forum for Information Retrieval Evaluation, Kolkata, India, December 12-15, 2019, volume 2517 of CEUR Workshop Proceedings, pages 401–408. CEUR-WS.org. [find](https://scholar.google.com/scholar?q=Hamada+A%2E+Nayel%2C+Walaa+Medhat%2C+and+Metwally+Rashad%2E+2019%2E+BENHA%40IDAT%3A+Improving+Irony+Detection+in+Arabic+Tweets+using+Ensemble+Approach%2E+In+Working+Notes+of+FIRE+2019+%2D+Forum+for+Information+Retrieva)
5. D. Al-Ghadhban, E. Alnkhilan, L. Tatwany, and M. Al-razgan. 2017. Arabic sarcasm detection in twitter. In 2017 International Conference on Engineering MIS (ICEMIS), pages 1–7. [find](https://scholar.google.com/scholar?q=D%2E+Al%2DGhadhban%2C+E%2E+Alnkhilan%2C+L%2E+Tatwany%2C+and+M%2E+Al%2Drazgan%2E+2017%2E+Arabic+sarcasm+detection+in+twitter%2E+In+2017+International+Conference+on+Engineering+MIS+%28ICEMIS%29%2C+pages+1%E2%80%937%2E)
6. Sergios Theodoridis and Konstantinos Koutroumbas. 2009. Chapter 3 - Linear Classifiers. In Pattern Recognition (Fourth Edition), fourth edition edition, pages 91 – 150. Academic Press, Boston. [find](https://scholar.google.com/scholar?q=Sergios+Theodoridis+and+Konstantinos+Koutroumbas%2E+2009%2E+Chapter+3+%2D+Linear+Classifiers%2E+In+Pattern+Recognition+%28Fourth+Edition%29%2C+fourth+edition+edition%2C+pages+91+%E2%80%93+150%2E+Academic+Press%2C+Boston%2E)
7. Ahmed Alsayat and Nouh Elmitwally. 2020. A comprehensive study for arabic sentiment analysis (challenges and applications). Egyptian Informatics Journal, 21(1):7–12. [↗](https://doi.org/10.1016/j.eij.2019.06.001)
8. Deirdre Wilson. 2006. The pragmatics of verbal irony: Echo or pretence? Lingua, 116(10):1722–1743. Language in Mind: A Tribute to Neil Smith on the Occasion of his Retirement. [↗](https://doi.org/10.1016/j.lingua.2006.05.001)
9. Naaima Boudad, Rdouan Faizi, Rachid Oulad Haj Thami, and Raddouane Chiheb. 2018. Sentiment analysis in arabic: A review of the literature. Ain Shams Engineering Journal, 9(4):2479–2490. [↗](https://doi.org/10.1016/j.asej.2017.04.007)
10. Jihen Karoui, Farah Banamara Zitoune, and Véronique Moriceau. 2017. Soukhria: Towards an irony detection system for arabic in social media. Procedia Computer Science, 117:161–168. Arabic Computational Linguistics. [find](https://scholar.google.com/scholar?q=Jihen+Karoui%2C+Farah+Banamara+Zitoune%2C+and+Ve%CC%81ronique+Moriceau%2E+2017%2E+Soukhria%3A+Towards+an+irony+detection+system+for+arabic+in+social+media%2E+Procedia+Computer+Science%2C+117%3A161%E2%80%93168%2E+Arabic+Computationa)
11. E. Lunando and A. Purwarianti. 2013. Indonesian social media sentiment analysis with sarcasm detection. In 2013 International Conference on Advanced Computer Science and Information Systems (ICAC-SIS), pages 195–198. [find](https://scholar.google.com/scholar?q=E%2E+Lunando+and+A%2E+Purwarianti%2E+2013%2E+Indonesian+social+media+sentiment+analysis+with+sarcasm+detection%2E+In+2013+International+Conference+on+Advanced+Computer+Science+and+Information+Systems+%28ICAC%2DSIS%29)
12. Diana Maynard and Mark Greenwood. 2014. [Who cares about sarcastic tweets? investigating the impact of sarcasm on sentiment analysis](https://md.research.elamur.ai/free/en/papers/maynard-2014-who-cares-about-sarcastic-tweets/). In Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC’14), pages 4238–4243, Reykjavik, Iceland. European Language Resources Association (ELRA). [↗](https://doi.org/10.63317/2ios4bdd5xas)
13. Hamada Nayel. 2020. NAYEL at SemEval-2020 task 12: TF/IDF-based approach for automatic offensive language detection in Arabic tweets. In Proceedings of the Fourteenth Workshop on Semantic Evaluation, pages 2086–2089, Barcelona (online). International Committee for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Hamada+Nayel%2E+2020%2E+NAYEL+at+SemEval%2D2020+task+12%3A+TF%2FIDF%2Dbased+approach+for+automatic+offensive+language+detection+in+Arabic+tweets%2E+In+Proceedings+of+the+Fourteenth+Workshop+on+Semantic+Evaluation%2C+)
14. Hamada A. Nayel. 2019. NAYEL@APDA: Machine Learning Approach for Author Profiling and Deception Detection in Arabic Texts. In Working Notes of FIRE 2019 - Forum for Information Retrieval Evaluation, Kolkata, India, December 12-15, 2019, volume 2517 of CEUR Workshop Proceedings, pages 92–99. CEUR-WS.org. [find](https://scholar.google.com/scholar?q=Hamada+A%2E+Nayel%2E+2019%2E+NAYEL%40APDA%3A+Machine+Learning+Approach+for+Author+Profiling+and+Deception+Detection+in+Arabic+Texts%2E+In+Working+Notes+of+FIRE+2019+%2D+Forum+for+Information+Retrieval+Evaluation%2C+K)

*Updated 24 October 2026*