---
title: "BhamNLP at SemEval-2020 Task 12: An Ensemble of Different"
description: "The paper describes the BhamNLP entry for the Arabic track of OffensEval 2020… 0.87 F1. 27 references, cited by 3 papers here."
authors: ["Abdullah I. Alharbi", "Mark Lee"]
year: 2020
venue: "Proceedings of the Fourteenth Workshop on Semantic Evaluation"
source: "https://aclanthology.org/2020.semeval-1.200/"
doi: "10.18653/v1/2020.semeval-1.200"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/alharbi-2020-bhamnlp-semeval-2020-task-12/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/alharbi-2020-bhamnlp-semeval-2020-task-12/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2020](https://md.research.elamur.ai/free/en/papers/2020/page/8/) / BhamNLP at SemEval-2020 Task 12: An Ensemble of Different Word Embeddings and Emotion Transfer Learning for Arabic Offensive Language Identification in Social Media

# BhamNLP at SemEval-2020 Task 12: An Ensemble of Different Word Embeddings and Emotion Transfer Learning for Arabic Offensive Language Identification in Social Media

**Authors:** [Abdullah I. Alharbi](https://md.research.elamur.ai/free/en/authors/abdullah-i-alharbi/) and [Mark Lee](https://md.research.elamur.ai/free/en/authors/mark-lee/) · **Year:** 2020 · **Venue:** [Proceedings of the Fourteenth Workshop on Semantic Evaluation](https://md.research.elamur.ai/free/en/venues/semeval/) ([SemEval 2020](https://md.research.elamur.ai/free/en/venues/semeval/2020/)) · **Publisher:** [International Committee on Computational Linguistics (ICCL)](https://md.research.elamur.ai/free/en/publishers/international-committee-on-computational-linguistics-iccl/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2020.semeval-1.200/) · [PDF](https://aclanthology.org/2020.semeval-1.200.pdf) · [DOI 10.18653/v1/2020.semeval-1.200](https://doi.org/10.18653/v1/2020.semeval-1.200) · [OpenAlex](https://openalex.org/W3115879895)

**Citations:** 6 citations (OpenAlex, 2026-10-14) · [3](https://md.research.elamur.ai/free/en/papers/alharbi-2020-bhamnlp-semeval-2020-task-12/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
The paper describes the BhamNLP entry for the Arabic track of OffensEval 2020 (SemEval-2020 Task 12). The task is to label tweets as offensive or not. It is hard because only about a fifth of the tweets are offensive and they mix many Arabic dialects with no fixed spelling norms.

The authors built three classifiers and combined them:
- **CWE:** a convolutional + LSTM network whose input layer joins two word-level embeddings and one character-level embedding.
- **AraBERT:** a frozen model whose sentence vectors are classified by an SVM.
- **Affect:** a model trained on Arabic emotion data from SemEval 2018 Task 1, used through a hand-written anger rule.

The Affect model only decides tweets on which the two offensive-trained models disagree. Preprocessing normalises letter variants and uses a regex to split the fused vocative "ya" from the word that follows it.

The ensemble reached macro-F1 0.90 on Dev and 0.87 on Test, placing 9th of 52 teams. The work suggests that mixing embedding granularities and borrowing emotion knowledge helps in noisy, dialect-heavy social media text.

## Key points
- **Data and setup:** OffensEval 2020 Arabic, with roughly 19% offensive tweets. The official metric is macro-F1. Models were developed on the organisers' training and Dev splits.
- **CWE embeddings:** the input concatenates three 300-dimensional pretrained embeddings, all fine-tuned during training:
  - Ara2Vec (77M tweets);
  - Mazajak (250M tweets);
  - a FastText character-level model trained on 10M emotion-rich dialectal tweets.
- **CWE architecture:** two 1-D convolutions (128 filters, kernel 3), then LSTMs of 256 and 128 units with 0.2 dropout, a sigmoid output and the Adam optimizer.
- **Results:**

  | Model | Dev F1 | Test F1 |
  |---|---|---|
  | CWE | 0.88 | 0.86 |
  | AraBERT | 0.81 | 0.81 |
  | Ensemble | 0.90 | 0.87 |

  The authors blame AraBERT's weaker score on using frozen [CLS] vectors without fine-tuning.
- **Embedding comparison and Affect rule:**
  - On Dev, the char-level embedding alone matched Mazajak (both 0.87 F1) and beat Ara2Vec (0.86), despite far less training text.
  - The char-level embedding also caught an out-of-vocabulary dialectal insult (Alklbwbh, "female puppy") that both word-level models missed.
  - The Affect rule marks a tweet OFF when it is predicted as negative sentiment, anger, and high intensity.

## Interesting ideas
- Use a third, differently-trained model only as a tie-breaker when the two main classifiers disagree, so its weaker signal never overrides a consensus.
- Split a frequent fused pattern (vocative particle glued to the next word) with an explicit regex, and admit known false hits like personal names as a documented gap.
- A character-level embedding trained on a small but domain-matched corpus can rival a much larger word-level one and recover meanings of unseen spelling variants.
- Turn a combination of sentiment polarity, emotion type and emotion intensity into one readable rule for spotting hostile language, rather than treating sentiment as a single score.
- Error analysis showed a static model tagging tweets offensive just because a politician's name often co-occurs with insults, while the contextual model avoided this.

## For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

- **I1** (learn): when witness parsers split on an arc or label, a structurally different witness could be consulted only for that conflict before the item enters silver data.
- **I2** (tokens): fused informal forms such as "gimme" or "wanna" could get transparent split rules backed by an exception lexicon, keeping every split explainable.
- **I3** (tokens): character or subword cues could back off the tagger and lemmatizer on unknown or misspelled words, with the nearest known forms shown as the explanation.
- **I4** (frames): an insult or hostility speech act could be inferred from explicit affect attributes on the frame, making the decision traceable.
- **I5** (learn): negative controls could swap or insert named entities into neutral sentences and check that labels do not flip on the name alone.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [University of Birmingham](https://md.research.elamur.ai/free/en/universities/university-of-birmingham/) — University, Birmingham, United Kingdom · 25 works · 62,564 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🤝 Social Sciences](https://md.research.elamur.ai/free/en/shelves/social-sciences/) · 1,772 works · [🤝 Communication](https://md.research.elamur.ai/free/en/shelves/social-sciences/communication/) · 97 works

🧭 Topics (2): [😊 Sentiment & emotion](https://md.research.elamur.ai/free/en/topics/sentiment-emotion/) · 972 works · [Semitic languages](https://md.research.elamur.ai/free/en/topics/semitic-languages/) · 2,908 works

## Cite

Cite as: Abdullah I. Alharbi and Mark Lee. 2020. BhamNLP at SemEval-2020 Task 12: An Ensemble of Different Word Embeddings and Emotion Transfer Learning for Arabic Offensive Language Identification in Social Media. Proceedings of the Fourteenth Workshop on Semantic Evaluation.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{alharbi2020bhamnlp,
  title = {{BhamNLP at SemEval-2020 Task 12: An Ensemble of Different Word Embeddings and Emotion Transfer Learning for Arabic Offensive Language Identification in Social Media}},
  author = {Abdullah I. Alharbi and Mark Lee},
  booktitle = {Proceedings of the Fourteenth Workshop on Semantic Evaluation},
  year = {2020},
  doi = {10.18653/v1/2020.semeval-1.200},
  url = {https://doi.org/10.18653/v1/2020.semeval-1.200}
}
```

- [← previous ASU_OPTO at OSACT4 - Offensive Language Detection for Arabic text](https://md.research.elamur.ai/free/en/papers/keleg-2020-asu-opto-osact4-offensive-language/)
- [next → Comparison by Conversion: Reverse-Engineering UCCA from Syntax and Lexical Semantics](https://md.research.elamur.ai/free/en/papers/hershcovich-2020-comparison-conversion-reverse-engineering-ucca/)
- [↑ back to papers of 2020](https://md.research.elamur.ai/free/en/papers/2020/page/8/)

## References

27 references, 14 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Ahmed Abdelali, Kareem Darwish, Nadir Durrani, and Hamdy Mubarak. 2016. [Farasa: A fast and furious segmenter for Arabic](https://md.research.elamur.ai/free/en/papers/abdelali-2016-farasa-fast-furious-segmenter-arabic/). In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Demonstrations, pages 11–16, San Diego, California, June. Association for Computational Linguistics.
2. Ibrahim Abu Farha and Walid Magdy. 2019. [Mazajak: An online Arabic sentiment analyser](https://md.research.elamur.ai/free/en/papers/farha-2019-mazajak-online-arabic-sentiment-analyser/). In Proceedings of the Fourth Arabic Natural Language Processing Workshop, pages 192–198, Florence, Italy, August. Association for Computational Linguistics.
3. Areej Al-Hassan and Hmood Al-Dossari. 2019. Detection of hate speech in social networks: a survey on multilingual corpus. In 6th International Conference on Computer Science and Information Technology. [find](https://scholar.google.com/scholar?q=Areej+Al%2DHassan+and+Hmood+Al%2DDossari%2E+2019%2E+Detection+of+hate+speech+in+social+networks%3A+a+survey+on+multilingual+corpus%2E+In+6th+International+Conference+on+Computer+Science+and+Information+Technology)
4. Azalden Alakrot, Liam Murray, and Nikola S. Nikolov. 2018. Towards accurate detection of offensive language in online communication in Arabic. Procedia Computer Science, 142:315 – 320. Arabic Computational Linguistics. [find](https://scholar.google.com/scholar?q=Azalden+Alakrot%2C+Liam+Murray%2C+and+Nikola+S%2E+Nikolov%2E+2018%2E+Towards+accurate+detection+of+offensive+language+in+online+communication+in+Arabic%2E+Procedia+Computer+Science%2C+142%3A315+%E2%80%93+320%2E+Arabic+Computat)
5. N. Albadi, M. Kurdi, and S. Mishra. 2018. Are they our brothers? analysis and detection of religious hate speech in the Arabic twittersphere. In 2018 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining (ASONAM), pages 69–76. [find](https://scholar.google.com/scholar?q=N%2E+Albadi%2C+M%2E+Kurdi%2C+and+S%2E+Mishra%2E+2018%2E+Are+they+our+brothers%3F+analysis+and+detection+of+religious+hate+speech+in+the+Arabic+twittersphere%2E+In+2018+IEEE%2FACM+International+Conference+on+Advances+in+S)
6. Abdullah I. Alharbi and Mark Lee. 2020. Combining character and word embeddings for affect in Arabic informal social media microblogs. In Elisabeth Métais, Farid Meziane, Helmut Horacek, and Philipp Cimiano, editors, Natural Language Processing and Information Systems, pages 213–224, Cham. Springer International Publishing. [find](https://scholar.google.com/scholar?q=Abdullah+I%2E+Alharbi+and+Mark+Lee%2E+2020%2E+Combining+character+and+word+embeddings+for+affect+in+Arabic+informal+social+media+microblogs%2E+In+Elisabeth+Me%CC%81tais%2C+Farid+Meziane%2C+Helmut+Horacek%2C+and+Philipp+)
7. Wissam Antoun, Fady Baly, and Hazem Hajj. 2020. [Arabert: Transformer-based model for Arabic language understanding](https://md.research.elamur.ai/free/en/papers/antoun-2020-arabert-transformer-based-model-arabic/). [↗](https://doi.org/10.48550/arxiv.2003.00104)
8. Piotr Bojanowski, Edouard Grave, Armand Joulin, and Tomas Mikolov. 2017. [Enriching word vectors with subword information](https://md.research.elamur.ai/free/en/papers/bojanowski-2017-enriching-word-vectors-subword-information/). Transactions of the Association for Computational Linguistics, 5:135–146. [↗](https://doi.org/10.1162/tacl_a_00051)
9. Antoine Bordes, Sumit Chopra, and Jason Weston. 2014. Question Answering with Subgraph Embeddings. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 615–620. [↗](https://doi.org/10.48550/arxiv.1406.3676)
10. Thomas Davidson, Dana Warmsley, Michael Macy, and Ingmar Weber. 2017. [Automated hate speech detection and the problem of offensive language](https://md.research.elamur.ai/free/en/papers/davidson-2017-automated-hate-speech-detection-problem/). In Eleventh international aaai conference on web and social media, pages 512–515. [↗](https://doi.org/10.48550/arxiv.1703.04009)
11. Jacob Devlin, Rabih Zbib, Zhongqiang Huang, Thomas Lamar, Richard Schwartz, and John Makhoul. 2014. [Fast and robust neural network joint models for statistical machine translation](https://md.research.elamur.ai/free/en/papers/devlin-2014-fast-robust-neural-network-joint/). In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), volume 1, pages 1370– 1380.
12. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. [BERT: pre-training of deep bidirectional transformers for language understanding](https://md.research.elamur.ai/free/en/papers/devlin-2019-bert-pre-training-deep-bidirectional/). In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 4171–4186, Minneapolis. Association for Computational Linguistics.
13. Rehab Duwairi and Mahmoud El-Orfali. 2014. A study of the effects of preprocessing strategies on sentiment analysis for Arabic text. Journal of Information Science, 40(4):501–513. [↗](https://doi.org/10.1177/0165551514534143)
14. Sepp Hochreiter and Jürgen Schmidhuber. 1997. Long short-term memory. Neural Comput., 9(8):1735–1780, November. Ritesh Kumar, Atul Kr. [↗](https://doi.org/10.1162/neco.1997.9.8.1735)
15. Ojha, Shervin Malmasi, and Marcos Zampieri. 2018. [Benchmarking aggression identification in social media](https://md.research.elamur.ai/free/en/papers/kumar-2018-benchmarking-aggression-identification-social-media/). In Proceedings of the First Workshop on Trolling, Aggression and Cyberbullying (TRAC-2018), pages 1–11, Santa Fe, New Mexico, USA, August. Association for Computational Linguistics.
16. Chu-Cheng Lin, Waleed Ammar, Chris Dyer, and Lori Levin. 2015. Unsupervised POS Induction with Word Embeddings. In Proceedings of the 2015 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 1311–1316. [↗](https://doi.org/10.48550/arxiv.1503.06760)
17. Shervin Malmasi and Marcos Zampieri. 2017. Detecting hate speech in social media. In Proceedings of the International Conference Recent Advances in Natural Language Processing, RANLP 2017, pages 467–472, Varna, Bulgaria, September. INCOMA Ltd. [↗](https://doi.org/10.48550/arxiv.1712.06427)
18. Saif Mohammad, Felipe Bravo-Marquez, Mohammad Salameh, and Svetlana Kiritchenko. 2018. [SemEval-2018 task 1: Affect in Tweets](https://md.research.elamur.ai/free/en/papers/mohammad-2018-semeval-2018-task-1-affect/). In Proceedings of The 12th International Workshop on Semantic Evaluation, pages 1–17, New Orleans, Louisiana. Association for Computational Linguistics.
19. Hamdy Mubarak and Kareem Darwish. 2019. Arabic offensive language classification on twitter. In International Conference on Social Informatics, pages 269–276. Springer. [find](https://scholar.google.com/scholar?q=Hamdy+Mubarak+and+Kareem+Darwish%2E+2019%2E+Arabic+offensive+language+classification+on+twitter%2E+In+International+Conference+on+Social+Informatics%2C+pages+269%E2%80%93276%2E+Springer%2E)
20. Hamdy Mubarak, Kareem Darwish, and Walid Magdy. 2017. [Abusive language detection on Arabic social media](https://md.research.elamur.ai/free/en/papers/mubarak-2017-abusive-language-detection-arabic-social/). In Proceedings of the First Workshop on Abusive Language Online, pages 52–56.
21. Hamdy Mubarak, Ammar Rashed, Kareem Darwish, Younes Samih, and Ahmed Abdelali. 2020. [Arabic offensive language on twitter: Analysis and experiments](https://md.research.elamur.ai/free/en/papers/mubarak-2021-arabic-offensive-language-twitter-analysis/). arXiv preprint arXiv:2004.02192. [↗](https://arxiv.org/abs/2004.02192)
22. Arfath Pasha, Mohamed Al-Badrashiny, Mona Diab, Ahmed El Kholy, Ramy Eskander, Nizar Habash, Manoj Pooleery, Owen Rambow, and Ryan Roth. 2014. [MADAMIRA: A fast, comprehensive tool for morphological analysis and disambiguation of Arabic](https://md.research.elamur.ai/free/en/papers/pasha-2014-madamira-fast-comprehensive-tool-morphological/). In Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC’14), pages 1094–1101, Reykjavik, Iceland, May. European Language Resources Association (ELRA).
23. Abu Bakr Soliman, Kareem Eissa, and Samhaa R El-Beltagy. 2017. AraVec: A set of Arabic word embedding models for use in Arabic NLP. Procedia Computer Science, 117:256–265. [find](https://scholar.google.com/scholar?q=Abu+Bakr+Soliman%2C+Kareem+Eissa%2C+and+Samhaa+R+El%2DBeltagy%2E+2017%2E+AraVec%3A+A+set+of+Arabic+word+embedding+models+for+use+in+Arabic+NLP%2E+Procedia+Computer+Science%2C+117%3A256%E2%80%93265%2E)
24. Zeerak Waseem, Thomas Davidson, Dana Warmsley, and Ingmar Weber. 2017. [Understanding abuse: A typology of abusive language detection subtasks](https://md.research.elamur.ai/free/en/papers/waseem-2017-understanding-abuse-typology-abusive-language/). In Proceedings of the First Workshop on Abusive Language Online, pages 78–84, Vancouver, BC, Canada, August. Association for Computational Linguistics.
25. Marcos Zampieri, Shervin Malmasi, Preslav Nakov, Sara Rosenthal, Noura Farra, and Ritesh Kumar. 2019. [SemEval-2019 task 6: Identifying and categorizing offensive language in social media (OffensEval)](https://md.research.elamur.ai/free/en/papers/zampieri-2019-semeval-2019-task-6-identifying/). In Proceedings of the 13th International Workshop on Semantic Evaluation, pages 75–86, Minneapolis, Minnesota, USA, June. Association for Computational Linguistics.
26. Marcos Zampieri, Preslav Nakov, Sara Rosenthal, Pepa Atanasova, Georgi Karadzhov, Hamdy Mubarak, Leon Derczynski, Zeses Pitenis, and Çağrı Çöltekin. 2020. SemEval-2020 Task 12: Multilingual Offensive Language Identification in Social Media (OffensEval 2020). In Proceedings of SemEval. [↗](https://doi.org/10.48550/arxiv.2006.07235)
27. Jiajun Zhang, Shujie Liu, Mu Li, Ming Zhou, and Chengqing Zong. 2014. Bilingually-constrained phrase embeddings for machine translation. In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), volume 1, pages 111–121. [find](https://scholar.google.com/scholar?q=Jiajun+Zhang%2C+Shujie+Liu%2C+Mu+Li%2C+Ming+Zhou%2C+and+Chengqing+Zong%2E+2014%2E+Bilingually%2Dconstrained+phrase+embeddings+for+machine+translation%2E+In+Proceedings+of+the+52nd+Annual+Meeting+of+the+Association+fo)

*Updated 1 November 2026*