---
title: "DoTheMath at SemEval-2020 Task 12 : Deep Neural Networks"
description: "The paper describes the DoTheMath team's entry for the Arabic track (Sub-Task A) of… 0.85 precision. 16 references, cited by 1 paper here."
authors: ["Zoher Orabe", "Bushr Haddad", "Nada Ghneim", "Anas Al-Abood"]
year: 2020
venue: "Proceedings of the Fourteenth Workshop on Semantic Evaluation"
source: "https://aclanthology.org/2020.semeval-1.254/"
doi: "10.18653/v1/2020.semeval-1.254"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/orabe-2020-dothemath-semeval-2020-task-12/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/orabe-2020-dothemath-semeval-2020-task-12/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2020](https://md.research.elamur.ai/free/en/papers/2020/page/9/) / DoTheMath at SemEval-2020 Task 12 : Deep Neural Networks with Self Attention for Arabic Offensive Language Detection

# DoTheMath at SemEval-2020 Task 12 : Deep Neural Networks with Self Attention for Arabic Offensive Language Detection

**Authors:** [Zoher Orabe](https://md.research.elamur.ai/free/en/authors/zoher-orabe/), [Bushr Haddad](https://md.research.elamur.ai/free/en/authors/bushr-haddad/), [Nada Ghneim](https://md.research.elamur.ai/free/en/authors/nada-ghneim/) and [Anas Al-Abood](https://md.research.elamur.ai/free/en/authors/anas-al-abood/) · **Year:** 2020 · **Venue:** [Proceedings of the Fourteenth Workshop on Semantic Evaluation](https://md.research.elamur.ai/free/en/venues/semeval/) ([SemEval 2020](https://md.research.elamur.ai/free/en/venues/semeval/2020/)) · **Publisher:** [International Committee on Computational Linguistics (ICCL)](https://md.research.elamur.ai/free/en/publishers/international-committee-on-computational-linguistics-iccl/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2020.semeval-1.254/) · [PDF](https://aclanthology.org/2020.semeval-1.254.pdf) · [DOI 10.18653/v1/2020.semeval-1.254](https://doi.org/10.18653/v1/2020.semeval-1.254) · [OpenAlex](https://openalex.org/W3117825923)

**Citations:** 5 citations (OpenAlex, 2026-10-14) · 1 in our library, of the 149,439 papers whose bibliographies we read

## Summary
The paper describes the DoTheMath team's entry for the Arabic track (Sub-Task A) of OffensEval at SemEval-2020. The task is to label Arabic tweets as offensive (OFF) or not (NOT OFF). The training data is heavily skewed, with only about one tweet in five offensive.

The team cleaned the tweets by removing some stop words, punctuation and repeated characters, and also tried stemming. They built two families of attention-augmented neural classifiers:
- **Character-level:** a ConvNet-style CNN combined with a BiLSTM.
- **Word-level:** a CNN and a bidirectional GRU, fed with Word2Vec, FastText, GloVe or AraVec vectors, optionally concatenated with emoji2vec vectors.

To counter the class imbalance, they added comments from an external YouTube corpus and generated extra samples by shuffling word order.

Word-level models clearly beat the character model. Adding emoji vectors gave a further gain. The BI GRU ATT model reached a macro F1 of 0.85 on the official test set.

The study is a useful side-by-side comparison of representation levels and embedding choices for noisy, dialect-rich social media text.

## Key points
- **Data:** the Arabic OffensEval tweets with a 70/10/20 train/validation/test split. About 19% of training tweets are OFF and 81% are NOT OFF. Extra samples came from YouTube comments (Alakrot 2018) and word-shuffled copies of existing tweets.
- **Embeddings without emojis:** Word2Vec gave the best word-level validation score (F1 ≈ 0.82). Unigram embeddings did better than bigram ones.
- **Emoji vectors:** concatenating 300-dimensional emoji2vec vectors (1661 emojis, learned from Unicode descriptions) with the word vectors raised validation F1 to about 0.84.
- **After balancing, with AraVec (skip-gram, 100 dimensions):**
  - BI GRU ATT reached F1 0.86 and precision 0.91 on validation.
  - CNN ATT reached F1 0.82.
  - The character model ((CNN+LSTM) ATT: 6 conv layers, 3 max-pooling layers, input length 2028) scored F1 0.76 before balancing and 0.75 after.
- **Official test result for BI GRU ATT:** macro F1 0.85, precision 0.88, recall 0.83, accuracy 0.91.

## Interesting ideas
- Emojis get their own vectors, built from their Unicode textual descriptions and joined to ordinary word vectors, so pictographs carry meaning instead of being discarded as noise.
- Balancing by shuffling the word order of minority-class sentences actually lowered recall and F1 for the character model, so the authors flagged it as ineffective. Checking per-class recall after any augmentation catches this kind of hidden harm.
- Character-level input is motivated as a way to cope with misspellings, dialectal spellings and rare new insult words that no word vocabulary covers.
- Minority-class data was borrowed from a different platform (YouTube comments) with the same labels, as a cheap way to enlarge a scarce class.
- The attention layer scores each input position and builds the sentence vector from those weights, so the weights show which positions the classifier relied on.

## For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

- **I1** (tokens): the tokenizer could map each emoji to a gloss lemma taken from its Unicode name, letting the tagger and later stages treat it as a word.
- **I2** (learn): every synthetic or augmented silver batch should be tested against a negative control on the minority labels before it enters training.
- **I3** (tokens): a character-based fallback for out-of-vocabulary social-media forms could propose lemma and POS when dictionary lookup fails.
- **I4** (learn): rare constructions or speech-act types could be topped up from other genres, as long as the borrowed examples are tracked and evaluated separately.
- **I5** (frames): as a diagnostic only, such weights could point to candidate insult or irony trigger words that an explicit speech-act rule should then explain.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Damascus University](https://md.research.elamur.ai/free/en/universities/damascus-university/) — University, Damascus, Syria · 4 works · 43 citations

Also on the byline: [Al-Sham Private University](https://ror.org/026csjr38).

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 18,614 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 17,965 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 17,450 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 17,296 works

🧭 Topics (2): [😊 Sentiment & emotion](https://md.research.elamur.ai/free/en/topics/sentiment-emotion/) · 1,035 works · [Semitic languages](https://md.research.elamur.ai/free/en/topics/semitic-languages/) · 3,066 works

## Cite

Cite as: Zoher Orabe, Bushr Haddad, Nada Ghneim and Anas Al-Abood. 2020. DoTheMath at SemEval-2020 Task 12 : Deep Neural Networks with Self Attention for Arabic Offensive Language Detection. Proceedings of the Fourteenth Workshop on Semantic Evaluation.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{orabe2020dothemath,
  title = {{DoTheMath at SemEval-2020 Task 12 : Deep Neural Networks with Self Attention for Arabic Offensive Language Detection}},
  author = {Zoher Orabe, Bushr Haddad, Nada Ghneim and Anas Al-Abood},
  booktitle = {Proceedings of the Fourteenth Workshop on Semantic Evaluation},
  year = {2020},
  doi = {10.18653/v1/2020.semeval-1.254},
  url = {https://doi.org/10.18653/v1/2020.semeval-1.254}
}
```

- [← previous Developing a Corpus of Indirect Speech Act Schemas](https://md.research.elamur.ai/free/en/papers/roque-2020-developing-corpus-indirect-speech-act/)
- [next → Evaluating Amharic Machine Translation](https://md.research.elamur.ai/free/en/papers/hadgu-2020-evaluating-amharic-machine-translation/)
- [↑ back to papers of 2020](https://md.research.elamur.ai/free/en/papers/2020/page/9/)

## References

16 references, 8 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Azalden Alakrot, Liam Murray, and Nikola S Nikolov. 2018. Dataset construction for the detection of anti-social behaviour in online communication in arabic. Procedia Computer Science, 142:174–181. [find](https://scholar.google.com/scholar?q=Azalden+Alakrot%2C+Liam+Murray%2C+and+Nikola+S+Nikolov%2E+2018%2E+Dataset+construction+for+the+detection+of+anti%2Dsocial+behaviour+in+online+communication+in+arabic%2E+Procedia+Computer+Science%2C+142%3A174%E2%80%93181%2E)
2. Pinkesh Badjatiya, Shashank Gupta, Manish Gupta, and Vasudeva Varma. 2017. Deep learning for hate speech detection in tweets. In Proceedings of the 26th International Conference on World Wide Web Companion, pages 759–760. [find](https://scholar.google.com/scholar?q=Pinkesh+Badjatiya%2C+Shashank+Gupta%2C+Manish+Gupta%2C+and+Vasudeva+Varma%2E+2017%2E+Deep+learning+for+hate+speech+detection+in+tweets%2E+In+Proceedings+of+the+26th+International+Conference+on+World+Wide+Web+Comp)
3. Piotr Bojanowski, Edouard Grave, Armand Joulin, and Tomas Mikolov. 2017. [Enriching word vectors with subword information](https://md.research.elamur.ai/free/en/papers/bojanowski-2017-enriching-word-vectors-subword-information/). arXiv:1609.04606. [↗](https://arxiv.org/abs/1609.04606)
4. Aleix Cambray and Norbert Podsadowski. 2019. Bidirectional recurrent models for offensive tweet classification. arXiv:1903.08808. [↗](https://arxiv.org/abs/1903.08808)
5. Yuxiao Chen, Jianbo Yuan, Quanzeng You, and Jiebo Luo. 2018. Twitter sentiment analysis via bi-sense emoji embedding and attention-based lstm. In Proceedings of the 26th ACM International Conference on Multimedia, pages 117–125. [find](https://scholar.google.com/scholar?q=Yuxiao+Chen%2C+Jianbo+Yuan%2C+Quanzeng+You%2C+and+Jiebo+Luo%2E+2018%2E+Twitter+sentiment+analysis+via+bi%2Dsense+emoji+embedding+and+attention%2Dbased+lstm%2E+In+Proceedings+of+the+26th+ACM+International+Conference+o)
6. Ronan Collobert and Jason Weston. 2008. A unified architecture for natural language processing: Deep neural networks with multitask learning. In Proceedings of the 25th International Conference on Machine Learning, pages 160–167. [find](https://scholar.google.com/scholar?q=Ronan+Collobert+and+Jason+Weston%2E+2008%2E+A+unified+architecture+for+natural+language+processing%3A+Deep+neural+networks+with+multitask+learning%2E+In+Proceedings+of+the+25th+International+Conference+on+Mac)
7. Ben Eisner, Tim Rocktäschel, Isabelle Augenstein, Matko Bošnjak, and Sebastian Riedel. 2016. [emoji2vec: Learning emoji representations from their description](https://md.research.elamur.ai/free/en/papers/eisner-2016-emoji2vec-learning-emoji-representations-their/). arXiv:1609.08359. [↗](https://arxiv.org/abs/1609.08359)
8. Björn Gambäck and Utpal Kumar Sikdar. 2017. [Using convolutional neural networks to classify hate-speech](https://md.research.elamur.ai/free/en/papers/gamback-2017-convolutional-neural-networks-classify-hate/). In Proceedings of the First Workshop On Abusive Language Online, pages 85–90.
9. Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean. 2013. [Efficient estimation of word representations in vector space](https://md.research.elamur.ai/free/en/papers/mikolov-2013-efficient-estimation-word-representations-vector/). arXiv:1301.3781. [↗](https://arxiv.org/abs/1301.3781)
10. Hamdy Mubarak, Ammar Rashed, Kareem Darwish, Younes Samih, and Ahmed Abdelali. 2020. [Arabic offensive language on twitter: Analysis and experiments](https://md.research.elamur.ai/free/en/papers/mubarak-2021-arabic-offensive-language-twitter-analysis/). arXiv:2004.02192. [↗](https://arxiv.org/abs/2004.02192)
11. Jeffrey Pennington, Richard Socher, and Christopher D. Manning. 2014. [Glove: Global vectors for word representation](https://md.research.elamur.ai/free/en/papers/pennington-2014-glove-global-vectors-word-representation/). In Empirical Methods in Natural Language Processing (EMNLP), pages 1532–1543. [↗](https://doi.org/10.3115/v1/d14-1162)
12. Abu Bakr Soliman, Kareem Eissa, and Samhaa R El-Beltagy. 2017. Aravec: A set of arabic word embedding models for use in arabic nlp. Procedia Computer Science, 117:256–265. [find](https://scholar.google.com/scholar?q=Abu+Bakr+Soliman%2C+Kareem+Eissa%2C+and+Samhaa+R+El%2DBeltagy%2E+2017%2E+Aravec%3A+A+set+of+arabic+word+embedding+models+for+use+in+arabic+nlp%2E+Procedia+Computer+Science%2C+117%3A256%E2%80%93265%2E)
13. Marcos Zampieri, Preslav Nakov, Sara Rosenthal, Pepa Atanasova, Georgi Karadzhov, Hamdy Mubarak, Leon Derczynski, Zeses Pitenis, and Cagrı Cöltekin. 2020. Semeval-2020 task 12: Multilingual offensive language identification in social media (offenseval 2020). Proceedings of SemEval. [find](https://scholar.google.com/scholar?q=Marcos+Zampieri%2C+Preslav+Nakov%2C+Sara+Rosenthal%2C+Pepa+Atanasova%2C+Georgi+Karadzhov%2C+Hamdy+Mubarak%2C+Leon+Derczynski%2C+Zeses+Pitenis%2C+and+Cagr%C4%B1+Co%CC%88ltekin%2E+2020%2E+Semeval%2D2020+task+12%3A+Multilingual+offensive)
14. Ziqi Zhang and Lei Luo. 2019. Hate speech detection: A solved problem? the challenging case of long tail on twitter. Semantic Web, 10(5):925–945. [find](https://scholar.google.com/scholar?q=Ziqi+Zhang+and+Lei+Luo%2E+2019%2E+Hate+speech+detection%3A+A+solved+problem%3F+the+challenging+case+of+long+tail+on+twitter%2E+Semantic+Web%2C+10%285%29%3A925%E2%80%93945%2E)
15. Xiang Zhang, Junbo Zhao, and Yann LeCun. 2015. [Character-level convolutional networks for text classification](https://md.research.elamur.ai/free/en/papers/zhang-2015-character-level-convolutional-networks-text/). In Advances in Neural Information Processing Systems, pages 649–657.
16. Peng Zhou, Wei Shi, Jun Tian, Zhenyu Qi, Bingchen Li, Hongwei Hao, and Bo Xu. 2016. [Attention-based bidirectional long short-term memory networks for relation classification](https://md.research.elamur.ai/free/en/papers/zhou-2016-attention-based-bidirectional-long-short/). In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (volume 2: Short papers), pages 207–212.

*Updated 2 November 2026*