--- title: "Transfer Learning Approach for Arabic — Husain 2021" description: "The paper asks whether an offensive-language classifier for Arabic social media can carry what it learns from one labelled collection over to others." authors: ["Fatemah Husain", "Ozlem Uzuner"] year: 2021 venue: "arXiv:2102.05708" source: "https://arxiv.org/abs/2102.05708" doi: "10.48550/arxiv.2102.05708" arxiv: "2102.05708" licence: "CC BY 4.0 (arXiv)" licence_url: "http://creativecommons.org/licenses/by/4.0/" summary_licence: "Elamur Cosmic License" canonical: "https://research.elamur.ai/free/en/papers/husain-2021-transfer-learning-approach-arabic-offensive/" --- [Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/husain-2021-transfer-learning-approach-arabic-offensive/) [home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2021](https://md.research.elamur.ai/free/en/papers/2021/page/5/) / Transfer Learning Approach for Arabic Offensive Language Detection System -- BERT-Based Model # Transfer Learning Approach for Arabic Offensive Language Detection System -- BERT-Based Model **Authors:** [Fatemah Husain](https://md.research.elamur.ai/free/en/authors/fatemah-husain/), [Ozlem Uzuner](https://md.research.elamur.ai/free/en/authors/ozlem-uzuner/) · **Year:** 2021 · **Venue:** arXiv:2102.05708 · **Licence:** [CC BY 4.0 (arXiv)](http://creativecommons.org/licenses/by/4.0/) **Links:** [Publisher's page](https://arxiv.org/abs/2102.05708) · [PDF](https://arxiv.org/pdf/2102.05708) · [DOI 10.48550/arxiv.2102.05708](https://doi.org/10.48550/arxiv.2102.05708) · [arXiv 2102.05708](https://arxiv.org/abs/2102.05708) · [OpenAlex](https://openalex.org/W3128004393) **Citations:** 10 citations (openalex-snapshot · 2026-10-14) · [2](#cited-by) in our library, of the 140884 papers whose bibliographies we read ## Summary The paper asks whether an offensive-language classifier for Arabic social media can carry what it learns from one labelled collection over to others. This is hard because user comments are written mostly in diverse regional dialects rather than Modern Standard Arabic. The authors first run a short pilot comparing four Transformer encoders that support Arabic. The monolingual AraBERT wins, so they adopt it. They then fine-tune AraBERT on each of four public corpora, collected from news-site comments, YouTube and Twitter, and evaluate every trained model on all four test sets. They also try a version fine-tuned on all corpora concatenated. All labels are reduced to a binary offensive/not-offensive scheme, and the text is left unpreprocessed. Models score well on their own corpus, but performance drops noticeably when they are tested on a different source. The drop is especially sharp on the strongly dialectal Aljazeera deleted-comments data. The authors conclude that transfer across these datasets helps only a little and that Arabic offensive-language detection likely needs new pre-training strategies. ## Key points - Four corpora are used: - Aljazeera.net Deleted Comments (31,692 comments). - YouTube Comments (15,050). - L-HSAB, a Levantine Twitter set (5,846 tweets). - OSACT (10,000 tweets). - Hate, abusive and obscene labels are merged into a single "offensive" class. Each corpus is pooled and re-split at random into 80% train and 20% test. - In the pilot on OSACT and L-HSAB, AraBERT beat Arabic-base-BERT, M-BERT and XLM-R on macro F1. The Arabic-only models outperformed the multilingual ones overall. - Setup and in-domain results: - Fine-tuning used a pooled encoder output, a simple FFNN head, max length 128, batch size 16, 5 epochs and learning rate 2e-5. - In-domain macro F1 was 0.91 on OSACT, 0.87 on L-HSAB, 0.87 on YouTube and 0.74 on Aljazeera. - Cross-corpus results are much weaker. Testing on Aljazeera gave only 0.39–0.52 macro F1 for models trained on the other sets. The Aljazeera-trained model reached just 0.54–0.65 macro F1 on the others. ## Interesting ideas - Train one model per corpus and score each on every other corpus; the resulting grid shows which sources generalise and which form isolated islands. — For Mova (learn): Run the perceptron parser and gparse through a train-on-X/test-on-Y grid over genres (news, fiction, school texts, fables) to find which domains need their own silver data. - Map several incompatible annotation schemes onto one shared coarse label set before comparing them, so that differences reflect the data rather than the label design. — For Mova (tree): Apply the same idea to UD dialect conversion: normalise treebanks to a common core relation set first, then measure cross-treebank LAS without scheme noise. - Merge any official splits and re-split every dataset with one fixed protocol, so that comparisons across datasets are made under identical conditions. — For Mova (learn): Use a single, seeded split procedure for all treebanks and test suites so that evaluation numbers across corpora are directly comparable. - Run a cheap pilot on a couple of datasets to choose the backbone encoder before committing to the full experimental grid. — For Mova (tree): Before training gparse at scale, compare candidate encoders on two small treebanks and keep only the winner. - Error analysis in the cited AraBERT work found that most mistakes involved offensive words used harmlessly, sarcasm, or quoting someone else's insult, i.e. cases where surface vocabulary misleads. — For Mova (frames): Use this as a motivating test set for crate prag: mark quoted or reported speech and irony in frames so that a word's literal polarity is not taken as the speaker's own act. *Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.* Fields: [Linguistics & languages](https://md.research.elamur.ai/free/en/shelves/linguistics/) ## Cited by Cited by 2 papers of our library cite it (of the 140884 whose bibliographies we read); each has a page here. - [Spam and Sentiment Detection in Arabic Tweets Using MARBERT Model](https://md.research.elamur.ai/free/en/papers/alotaibi-2026-spam-sentiment-detection-arabic-tweets/) - [Arabic Hate Speech Identification and Masking in Social Media using Deep Learning Models and Pre-trained Models Fine-tuning](https://md.research.elamur.ai/free/en/papers/doghmash-2025-arabic-hate-speech-identification-masking/) ## Cite The paper itself, not this page. BibTeX from the library's record of the work. ```bibtex @misc{husain2021transfer, title = {{Transfer Learning Approach for Arabic Offensive Language Detection System -- BERT-Based Model}}, author = {Husain, Fatemah and Uzuner, Ozlem}, year = {2021}, doi = {10.48550/arxiv.2102.05708}, eprint = {2102.05708}, archivePrefix = {arXiv}, url = {https://doi.org/10.48550/arxiv.2102.05708} } ``` - [← previous Trankit: A Light-Weight Transformer-based Toolkit for Multilingual Natural Language Processing](https://md.research.elamur.ai/free/en/papers/ud-2021-nguyen-trankit-light-weight-transformer-based/) - [next → Transition-based Bubble Parsing: Improvements on Coordination Structure Prediction](https://md.research.elamur.ai/free/en/papers/shi-2021-transition-based-bubble-parsing-improvements/) - [↑ back to papers of 2021](https://md.research.elamur.ai/free/en/papers/2021/page/5/)