--- title: "Fine-Tuning Approach for Arabic Offensive — Husain 2022" description: "The paper looks at automatic detection of offensive comments in Arabic social-media text." authors: ["Fatemah Husain", "Ozlem Uzuner"] year: 2022 venue: "arXiv:2203.03542" source: "https://arxiv.org/abs/2203.03542" doi: "10.48550/arxiv.2203.03542" arxiv: "2203.03542" licence: "CC BY 4.0 (arXiv)" licence_url: "http://creativecommons.org/licenses/by/4.0/" summary_licence: "Elamur Cosmic License" canonical: "https://research.elamur.ai/free/en/papers/husain-2022-fine-tuning-approach-arabic-offensive/" --- [Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/husain-2022-fine-tuning-approach-arabic-offensive/) [home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2022](https://md.research.elamur.ai/free/en/papers/2022/page/4/) / Fine-Tuning Approach for Arabic Offensive Language Detection System: BERT-Based Model # Fine-Tuning Approach for Arabic Offensive Language Detection System: BERT-Based Model **Authors:** [Fatemah Husain](https://md.research.elamur.ai/free/en/authors/fatemah-husain/), [Ozlem Uzuner](https://md.research.elamur.ai/free/en/authors/ozlem-uzuner/) · **Institutions:** [George Mason University](https://md.research.elamur.ai/free/en/universities/george-mason-university/) · [Kuwait University](https://md.research.elamur.ai/free/en/universities/kuwait-university/) · **Year:** 2022 · **Venue:** arXiv:2203.03542 · **Licence:** [CC BY 4.0 (arXiv)](http://creativecommons.org/licenses/by/4.0/) **Links:** [Publisher's page](https://arxiv.org/abs/2203.03542) · [PDF](https://arxiv.org/pdf/2203.03542) · [DOI 10.48550/arxiv.2203.03542](https://doi.org/10.48550/arxiv.2203.03542) · [arXiv 2203.03542](https://arxiv.org/abs/2203.03542) · [OpenAlex](https://openalex.org/W4221149035) **Citations:** 2 citations (OpenAlex, 2026-10-14) ## Summary The paper looks at automatic detection of offensive comments in Arabic social-media text. This content is mostly dialectal, and dialect diversity makes it hard for NLP tools. The authors first compared four pretrained transformers on two datasets using identical hyperparameters: XLM-RoBERTa, M-BERT, Arabic-BERT and AraBERT. The Arabic-only models did better than the multilingual ones, and AraBERT came out on top. They then fine-tuned AraBERT on four public corpora, Aljazeera deleted comments, YouTube comments, L-HSAB and OSACT. All labels were reduced to offensive versus not offensive. Each corpus was tested separately, then across corpora, and finally with all training sets pooled together. Within a single corpus, scores were strong on OSACT and weakest on Aljazeera. Pooling the data brought no improvement and slightly hurt the Levantine L-HSAB set. Transfer between datasets helped little, especially for heavily dialectal comments. The practical lesson is that a large pretrained model plus more pooled data will not automatically cover dialect and domain shifts in abuse detection. ## Key points - Four models were compared on OSACT and L-HSAB: XLM-RoBERTa, M-BERT, Arabic-BERT and AraBERT. AraBERT scored best on both, and the monolingual models beat the multilingual ones. - The same settings were used everywhere: - max length 128, batch size 16, 5 epochs, learning rate 2e-5, epsilon 1e-8 - the classifier is the pooled encoder output with a simple FFNN layer on top - no text preprocessing was applied - Each dataset was merged from any official splits and re-split randomly 80/20. Multi-class labels (hate, abusive, obscene) were collapsed into a single offensive class. - In-domain macro-F1 / accuracy results: - OSACT 0.91 / 0.94 - L-HSAB 0.87 / 0.88 - YouTube 0.87 / 0.88 - Aljazeera 0.74 / 0.85 - Fine-tuning on the concatenated corpus gave no gains over single-corpus training. L-HSAB lost about 3 points of macro-F1, which the authors attribute to its Levantine vocabulary being rare in the other sets. ## Interesting ideas - Training on each corpus and testing on every other one shows how far a model really transfers, which a single in-domain score hides. — For Mova (learn): Run a full treebank-by-treebank train/test grid for the perceptron parser and gparse to see which genres and dialects the parser actually generalizes to. - Merging every dataset can drag down the most distinctive subset, as happened with the dialect-heavy L-HSAB. Pooled results should therefore always be broken down per source. — For Mova (tree): When gparse is trained on mixed treebanks, report LAS per treebank so that losses on unusual registers are not masked by the overall average. - The model choice was made with a cheap pilot on two datasets under frozen, shared hyperparameters, so differences come from the models rather than from tuning. — For Mova (learn): Compare candidate witness parsers or LLM teacher tiers on a small fixed slice with one shared configuration before investing in full training runs. - Mapping several incompatible label schemes onto one coarse shared label makes heterogeneous corpora directly comparable and combinable. — For Mova (tree): Use the same move in UD dialect conversion, collapsing fine treebank-specific relations onto a common core before mixing data or comparing scores. - Related work cited in the paper finds that errors cluster on insulting words used harmlessly, sarcasm, and quoting someone else's offensive statement. These cases need context rather than keywords. — For Mova (frames): Mark reported or quoted speech and ironic speech acts in prag, so that a quoted insult is attributed to its original speaker rather than asserted by the writer. *Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.* Fields: [Linguistics & languages](https://md.research.elamur.ai/free/en/shelves/linguistics/) Topics: [Sentiment & emotion](https://md.research.elamur.ai/free/en/topics/sentiment-emotion/) · [Semitic languages](https://md.research.elamur.ai/free/en/topics/semitic-languages/) ## Cite The paper itself, not this page. BibTeX from the library's record of the work. ```bibtex @misc{husain2022fine, title = {{Fine-Tuning Approach for Arabic Offensive Language Detection System: BERT-Based Model}}, author = {Husain, Fatemah and Uzuner, Ozlem}, year = {2022}, doi = {10.48550/arxiv.2203.03542}, eprint = {2203.03542}, archivePrefix = {arXiv}, url = {https://doi.org/10.48550/arxiv.2203.03542} } ``` - [← previous FII UAIC at SemEval-2022 Task 6: iSarcasmEval - Intended Sarcasm Detection in English and Arabic](https://md.research.elamur.ai/free/en/papers/manoleasa-2022-fii-uaic-semeval-2022-task/) - [next → Handling Stress in Finite-State Morphological Analyzers for Ancient Greek and Ancient Hebrew](https://md.research.elamur.ai/free/en/papers/aclbulk-2022-swanson-handling-stress-finite-state-morphological/) - [↑ back to papers of 2022](https://md.research.elamur.ai/free/en/papers/2022/page/4/)