---
title: "DeepBlueAI at WANLP-EACL2021 task 2 — Song 2021"
description: "This paper describes DeepBlueAI's entry to the WANLP-2021 shared task on Arabic tweets."
authors: ["Bingyan Song", "Chunguang Pan", "Shengguang Wang", "Zhipeng Luo"]
year: 2021
venue: "Proceedings of the Sixth Arabic Natural Language Processing Workshop"
source: "https://aclanthology.org/2021.wanlp-1.52/"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/song-2021-deepblueai-wanlp-eacl2021-task-2/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/song-2021-deepblueai-wanlp-eacl2021-task-2/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2021](https://md.research.elamur.ai/free/en/papers/2021/page/2/) / DeepBlueAI at WANLP-EACL2021 task 2: A Deep Ensemble-based Method for Sarcasm and Sentiment Detection in Arabic

# DeepBlueAI at WANLP-EACL2021 task 2: A Deep Ensemble-based Method for Sarcasm and Sentiment Detection in Arabic

**Authors:** Bingyan Song, Chunguang Pan, Shengguang Wang, Zhipeng Luo · **Year:** 2021 · **Venue:** Proceedings of the Sixth Arabic Natural Language Processing Workshop · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2021.wanlp-1.52/) · [PDF](https://aclanthology.org/2021.wanlp-1.52.pdf) · [OpenAlex](https://openalex.org/W3155259807)

**Citations:** 2 citations (openalex-snapshot · 2026-10-14) · [3](#cited-by) in our library, of the 140884 papers whose bibliographies we read

## Summary
This paper describes DeepBlueAI's entry to the WANLP-2021 shared task on Arabic tweets. The task has two subtasks: binary sarcasm detection and three-way sentiment classification (positive, negative, neutral). Sarcasm makes sentiment hard to read because positive wording often carries a complaint, and Arabic adds dialect variation.

The team used ArSarcasm-v2 and fine-tuned two families of pre-trained encoders, XLM-R and AraBERT. The dialect label was placed in front of each tweet as a separate segment. A dense layer with Multi-Sample Dropout sat on top of the [CLS] vector. Each model was improved in two ways: further masked-LM pre-training on the task tweets (TAPT), and then knowledge distillation from soft targets. Many variants were trained under 7-fold cross-validation and combined by a learned meta-estimator: linear regression for sarcasm and an SVM for sentiment.

On the official leaderboard the system placed third in sarcasm and second in sentiment. The work shows that cheap ensembling and adaptation tricks reliably add gains on top of strong pre-trained models.

## Key points
- **Data:** ArSarcasm-v2 has 12,548 annotated training tweets and 3,000 unlabelled test tweets.
  - About 17.28% of the training tweets are sarcastic.
  - Most tweets are MSA or Egyptian; Maghrebi has only 43 examples.
  - Around 89% of sarcastic tweets carry negative sentiment.
- **Architecture:** The input is built as [CLS] dialect [SEP] tweet. A 256-dimensional ReLU dense layer follows. Multi-Sample Dropout uses 5 dropout samples with averaged losses.
- **Cross-validation gains:** Each strategy added on top of the previous one.
  - For XLM-R LARGE, sarcasm F1 rose from 0.5725 with plain fine-tuning to 0.6027 with TAPT and 0.6687 with TAPT plus distillation.
  - Sentiment F-PN rose from 0.7348 to 0.7919 over the same steps.
- **Stacking:** The learned meta-estimators beat simple averaging of the models.
  - Sarcasm: linear regression with a decision threshold of 0.41 reached F1 0.6899, against 0.5964 for the mean.
  - Sentiment: the SVM reached F-PN 0.8024, against 0.7782 for the mean.
- **Official test results:** F1-sarcastic was 0.6127 (3rd place) and F-PN was 0.7392 (2nd place). Both are noticeably below the cross-validation figures.

## Interesting ideas
- The authors feed metadata (the dialect tag) to the encoder as plain text in its own segment before the tweet, which conditions the model without any architectural change. — For Mova (tree): gparse could receive a short genre or dialect prefix (e.g., tweet, fable, school text) so one parser adapts to its input source instead of needing separate models.
- The data shows that sarcasm almost always pairs with negative sentiment, so a sarcasm signal can serve as evidence that positive wording should be read with flipped polarity. — For Mova (frames): the irony component in prag could treat detected sarcasm as an explicit, explainable rule that inverts the evaluative stance of a frame, rather than as a separate opaque label.
- A learned meta-estimator over many diverse models beats plain averaging, and even weaker models contribute useful signal once they are weighted properly. — For Mova (learn): when merging silver data from several witness parsers, learned per-witness (or per-relation) reliability weights could replace simple majority agreement.
- Distillation from a teacher's full probability distribution helped beyond training on hard labels, suggesting that uncertainty among classes carries information. — For Mova (learn): LLM teacher tiers could pass graded confidence over alternative heads or roles, not just one answer, to the student parser.
- Moving the decision threshold to 0.41 for the rare class, plus the sizeable gap between cross-validation and official test scores, both point to calibrating on held-out data and not trusting in-sample estimates. — For Mova (learn): evaluation should keep a truly held-out split alongside negative controls and tune thresholds for rare labels (e.g., irony, rare UD relations) only on that split.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

Fields: [Linguistics & languages](https://md.research.elamur.ai/free/en/shelves/linguistics/)

## Cited by

Cited by

3 papers of our library cite it (of the 140884 whose bibliographies we read); 1 of them have a page here, 2 not yet.

- [Overview of the WANLP 2021 Shared Task on Sarcasm and Sentiment Detection in Arabic](https://md.research.elamur.ai/free/en/papers/farha-2021-overview-wanlp-2021-shared-task/)

## Cite

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{song2021deepblueai,
  title = {{DeepBlueAI at WANLP-EACL2021 task 2: A Deep Ensemble-based Method for Sarcasm and Sentiment Detection in Arabic}},
  author = {Song, Bingyan and Pan, Chunguang and Wang, Shengguang and Luo, Zhipeng},
  booktitle = {Proceedings of the Sixth Arabic Natural Language Processing Workshop},
  year = {2021},
  url = {https://aclanthology.org/2021.wanlp-1.52/}
}
```

- [← previous Deep Multi-Task Model for Sarcasm Detection and Sentiment Analysis in Arabic Language](https://md.research.elamur.ai/free/en/papers/mahdaouy-2021-deep-multi-task-model-sarcasm/)
- [next → Defining Definiteness in Turoyo](https://md.research.elamur.ai/free/en/papers/lingbulk-2021-yifrach-defining-definiteness-turoyo/)
- [↑ back to papers of 2021](https://md.research.elamur.ai/free/en/papers/2021/page/2/)

## References

19 references, 7 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Ibrahim Abu Farha and Walid Magdy. 2020. From Arabic sentiment analysis to sarcasm detection: The Ar-Sarcasm dataset. In Proceedings of the 4th Workshop on Open-Source Arabic Corpora and Processing Tools, with a Shared Task on Offensive Language Detection, pages 32–39, Marseille, France. European Language Resource Association. [find](https://scholar.google.com/scholar?q=Ibrahim+Abu+Farha+and+Walid+Magdy%2E+2020%2E+From+Arabic+sentiment+analysis+to+sarcasm+detection%3A+The+Ar%2DSarcasm+dataset%2E+In+Proceedings+of+the+4th+Workshop+on+Open%2DSource+Arabic+Corpora+and+Processing+To)
2. Ibrahim Abu Farha, Wajdi Zaghouani, and Walid Magdy. 2021. Overview of the wanlp 2021 shared task on sarcasm and sentiment detection in arabic. In Proceedings of the Sixth Arabic Natural Language Processing Workshop. [find](https://scholar.google.com/scholar?q=Ibrahim+Abu+Farha%2C+Wajdi+Zaghouani%2C+and+Walid+Magdy%2E+2021%2E+Overview+of+the+wanlp+2021+shared+task+on+sarcasm+and+sentiment+detection+in+arabic%2E+In+Proceedings+of+the+Sixth+Arabic+Natural+Language+Proc) [our card](https://md.research.elamur.ai/free/en/papers/farha-2021-overview-wanlp-2021-shared-task/)
3. Mahmoud Al-Ayyoub, Safa Bani Essa, and Izzat Als-madi. 2015. Lexicon-based sentiment analysis of arabic tweets. International Journal of Social Network Mining, 2(2):101–114. [find](https://scholar.google.com/scholar?q=Mahmoud+Al%2DAyyoub%2C+Safa+Bani+Essa%2C+and+Izzat+Als%2Dmadi%2E+2015%2E+Lexicon%2Dbased+sentiment+analysis+of+arabic+tweets%2E+International+Journal+of+Social+Network+Mining%2C+2%282%29%3A101%E2%80%93114%2E)
4. Ahmad Al-Sallab, Ramy Baly, Hazem Hajj, Khaled Bashir Shaban, Wassim El-Hajj, and Gilbert Badaro. 2017. Aroma: A recursive deep learning model for opinion mining in arabic as a low resource language. ACM Transactions on Asian and Low-Resource Language Information Processing (TALLIP), 16(4):1–20. [find](https://scholar.google.com/scholar?q=Ahmad+Al%2DSallab%2C+Ramy+Baly%2C+Hazem+Hajj%2C+Khaled+Bashir+Shaban%2C+Wassim+El%2DHajj%2C+and+Gilbert+Badaro%2E+2017%2E+Aroma%3A+A+recursive+deep+learning+model+for+opinion+mining+in+arabic+as+a+low+resource+language%2E+)
5. Mohammad Al-Smadi, Bashar Talafha, Mahmoud Al-Ayyoub, and Yaser Jararweh. 2019. Using long short-term memory deep neural networks for aspect-based sentiment analysis of arabic reviews. International Journal of Machine Learning and Cybernetics, 10(8):2163–2175. [find](https://scholar.google.com/scholar?q=Mohammad+Al%2DSmadi%2C+Bashar+Talafha%2C+Mahmoud+Al%2DAyyoub%2C+and+Yaser+Jararweh%2E+2019%2E+Using+long+short%2Dterm+memory+deep+neural+networks+for+aspect%2Dbased+sentiment+analysis+of+arabic+reviews%2E+International+J)
6. Abdulaziz M Alayba, Vasile Palade, Matthew England, and Rahat Iqbal. 2017. Arabic language sentiment analysis on health services. In 2017 1st international workshop on arabic script analysis and recognition (asar), pages 114–118. IEEE. [find](https://scholar.google.com/scholar?q=Abdulaziz+M+Alayba%2C+Vasile+Palade%2C+Matthew+England%2C+and+Rahat+Iqbal%2E+2017%2E+Arabic+language+sentiment+analysis+on+health+services%2E+In+2017+1st+international+workshop+on+arabic+script+analysis+and+recog)
7. Wissam Antoun, Fady Baly, and Hazem Hajj. 2020. Arabert: Transformer-based model for arabic language understanding. In Proceedings of the 4th Workshop on Open-Source Arabic Corpora and Processing Tools, with a Shared Task on Offensive Language Detection, pages 9–15. [find](https://scholar.google.com/scholar?q=Wissam+Antoun%2C+Fady+Baly%2C+and+Hazem+Hajj%2E+2020%2E+Arabert%3A+Transformer%2Dbased+model+for+arabic+language+understanding%2E+In+Proceedings+of+the+4th+Workshop+on+Open%2DSource+Arabic+Corpora+and+Processing+Tool) [our card](https://md.research.elamur.ai/free/en/papers/antoun-2020-arabert-transformer-based-model-arabic/)
8. Alexis Conneau, Kartikay Khandelwal, Naman Goyal, Vishrav Chaudhary, Guillaume Wenzek, Francisco Guzmán, Édouard Grave, Myle Ott, Luke Zettlemoyer, and Veselin Stoyanov. 2020. Unsupervised cross-lingual representation learning at scale. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 8440– 8451. [find](https://scholar.google.com/scholar?q=Alexis+Conneau%2C+Kartikay+Khandelwal%2C+Naman+Goyal%2C+Vishrav+Chaudhary%2C+Guillaume+Wenzek%2C+Francisco+Guzma%CC%81n%2C+E%CC%81douard+Grave%2C+Myle+Ott%2C+Luke+Zettlemoyer%2C+and+Veselin+Stoyanov%2E+2020%2E+Unsupervised+cross%2Dlin)
9. Rehab Duwairi and Mahmoud El-Orfali. 2014. A study of the effects of preprocessing strategies on sentiment analysis for arabic text. Journal of Information Science, 40(4):501–513. [find](https://scholar.google.com/scholar?q=Rehab+Duwairi+and+Mahmoud+El%2DOrfali%2E+2014%2E+A+study+of+the+effects+of+preprocessing+strategies+on+sentiment+analysis+for+arabic+text%2E+Journal+of+Information+Science%2C+40%284%29%3A501%E2%80%93513%2E)
10. Kariman Elshakankery and Mona F Ahmed. 2019. Hi-latsa: A hybrid incremental learning approach for arabic tweets sentiment analysis. Egyptian Informatics Journal, 20(3):163–171. [find](https://scholar.google.com/scholar?q=Kariman+Elshakankery+and+Mona+F+Ahmed%2E+2019%2E+Hi%2Dlatsa%3A+A+hybrid+incremental+learning+approach+for+arabic+tweets+sentiment+analysis%2E+Egyptian+Informatics+Journal%2C+20%283%29%3A163%E2%80%93171%2E)
11. Suchin Gururangan, Ana Marasović, Swabha Swayamdipta, Kyle Lo, Iz Beltagy, Doug Downey, and Noah A Smith. 2020. Don’t stop pretraining: Adapt language models to domains and tasks. arXiv preprint arXiv:2004.10964. [↗](https://arxiv.org/abs/2004.10964)
12. Geoffrey Hinton, Oriol Vinyals, and Jeff Dean. 2015. Distilling the knowledge in a neural network. arXiv preprint arXiv:1503.02531. [↗](https://arxiv.org/abs/1503.02531) [our card](https://md.research.elamur.ai/free/en/papers/hinton-2015-distilling-knowledge-neural-network/)
13. Hiroshi Inoue. 2019. Multi-sample dropout for accelerated training and better generalization. arXiv preprint arXiv:1905.09788. [↗](https://arxiv.org/abs/1905.09788)
14. Navonil Majumder, Soujanya Poria, Haiyun Peng, Niyati Chhaya, Erik Cambria, and Alexander Gelbukh. 2019. Sentiment and sarcasm classification with multitask learning. IEEE Intelligent Systems, 34(3):38–43. [find](https://scholar.google.com/scholar?q=Navonil+Majumder%2C+Soujanya+Poria%2C+Haiyun+Peng%2C+Niyati+Chhaya%2C+Erik+Cambria%2C+and+Alexander+Gelbukh%2E+2019%2E+Sentiment+and+sarcasm+classification+with+multitask+learning%2E+IEEE+Intelligent+Systems%2C+34%283%29%3A3)
15. Mahmoud Nabil, Mohamed Aly, and Amir Atiya. 2015. Astd: Arabic sentiment tweets dataset. In Proceedings of the 2015 conference on empirical methods in natural language processing, pages 2515–2519. [find](https://scholar.google.com/scholar?q=Mahmoud+Nabil%2C+Mohamed+Aly%2C+and+Amir+Atiya%2E+2015%2E+Astd%3A+Arabic+sentiment+tweets+dataset%2E+In+Proceedings+of+the+2015+conference+on+empirical+methods+in+natural+language+processing%2C+pages+2515%E2%80%932519%2E) [our card](https://md.research.elamur.ai/free/en/papers/nabil-2015-astd-arabic-sentiment-tweets-dataset/)
16. Shereen Oraby, Yasser El-Sonbaty, and Mohamad Abou El-Nasr. 2013. Finding opinion strength using rule-based parsing for arabic sentiment analysis. In Mexican International Conference on Artificial Intelligence, pages 509–520. Springer. [find](https://scholar.google.com/scholar?q=Shereen+Oraby%2C+Yasser+El%2DSonbaty%2C+and+Mohamad+Abou+El%2DNasr%2E+2013%2E+Finding+opinion+strength+using+rule%2Dbased+parsing+for+arabic+sentiment+analysis%2E+In+Mexican+International+Conference+on+Artificial+Int)
17. Oumaima Oueslati, Erik Cambria, Moez Ben HajH-mida, and Habib Ounelli. 2020. A review of sentiment analysis research in arabic language. Future Generation Computer Systems, 112:408–430. [find](https://scholar.google.com/scholar?q=Oumaima+Oueslati%2C+Erik+Cambria%2C+Moez+Ben+HajH%2Dmida%2C+and+Habib+Ounelli%2E+2020%2E+A+review+of+sentiment+analysis+research+in+arabic+language%2E+Future+Generation+Computer+Systems%2C+112%3A408%E2%80%93430%2E) [our card](https://md.research.elamur.ai/free/en/papers/oueslati-2020-review-sentiment-analysis-research-arabic/)
18. Sara Rosenthal, Noura Farra, and Preslav Nakov. 2017. Semeval-2017 task 4: Sentiment analysis in twitter. In Proceedings of the 11th international workshop on semantic evaluation (SemEval-2017), pages 502– 518. [find](https://scholar.google.com/scholar?q=Sara+Rosenthal%2C+Noura+Farra%2C+and+Preslav+Nakov%2E+2017%2E+Semeval%2D2017+task+4%3A+Sentiment+analysis+in+twitter%2E+In+Proceedings+of+the+11th+international+workshop+on+semantic+evaluation+%28SemEval%2D2017%29%2C+pages) [our card](https://md.research.elamur.ai/free/en/papers/rosenthal-2017-semeval-2017-task-4-sentiment/)
19. Daniel Zeman, Jan Hajic, Martin Popel, Martin Potthast, Milan Straka, Filip Ginter, Joakim Nivre, and Slav Petrov. 2018. Conll 2018 shared task: Multilingual parsing from raw text to universal dependencies. In Proceedings of the CoNLL 2018 Shared Task: Multilingual parsing from raw text to universal dependencies, pages 1–21. [find](https://scholar.google.com/scholar?q=Daniel+Zeman%2C+Jan+Hajic%2C+Martin+Popel%2C+Martin+Potthast%2C+Milan+Straka%2C+Filip+Ginter%2C+Joakim+Nivre%2C+and+Slav+Petrov%2E+2018%2E+Conll+2018+shared+task%3A+Multilingual+parsing+from+raw+text+to+universal+depende) [our card](https://md.research.elamur.ai/free/en/papers/ud-2018-zeman-conll-2018-shared-task-multilingual/)