---
title: "Optimizer Choice and Calibration for QARiB on Arabic-Script"
description: "The paper asks how optimizer choice and post-hoc calibration affect fine-tuned QARiB on Arabic offensive-language detection. 31 references."
authors: ["Auda Elshokry", "Mohammed Alhanjouri"]
year: 2026
venue: "Proceedings of the 2nd Workshop on NLP for Languages Using Arabic Script"
source: "https://aclanthology.org/2026.abjadnlp-1.4/"
doi: "10.18653/v1/2026.abjadnlp-1.4"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/elshokry-2026-optimizer-choice-calibration-qarib-arabic/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/elshokry-2026-optimizer-choice-calibration-qarib-arabic/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/38/) / Optimizer Choice and Calibration for QARiB on Arabic-Script Social Media Offensive Language Detection

# Optimizer Choice and Calibration for QARiB on Arabic-Script Social Media Offensive Language Detection

**Authors:** Auda Elshokry and Mohammed Alhanjouri · **Year:** 2026 · **Venue:** [Proceedings of the 2nd Workshop on NLP for Languages Using Arabic Script](https://md.research.elamur.ai/free/en/venues/abjadnlp/) ([Workshop on NLP for Languages Using Arabic Script 2026](https://md.research.elamur.ai/free/en/venues/abjadnlp/2026/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2026.abjadnlp-1.4/) · [PDF](https://aclanthology.org/2026.abjadnlp-1.4.pdf) · [DOI 10.18653/v1/2026.abjadnlp-1.4](https://doi.org/10.18653/v1/2026.abjadnlp-1.4) · [OpenAlex](https://openalex.org/W7140104619)

**Citations:** 0 citations (OpenAlex, 2026-10-14)

## Summary
The paper asks how optimizer choice and post-hoc calibration affect fine-tuned QARiB on Arabic offensive-language detection. This setting is hard because the texts are noisy, dialect-heavy user posts and the offensive class is small. The authors fine-tune QARiB, a 12-layer BERT-style encoder, as a binary OFF/NOT classifier on two benchmarks: the Arabic part of OffensEval 2020 and MPOLD. Preprocessing is deliberately minimal so that real script variation is kept.

They compare Adam, AdamW and SGD over a small controlled grid of weight decay and warmup, with three matched seeds per setting. Evaluation covers both sides of model quality:
- **Discrimination:** macro-F1 and AUC-PR for the OFF class.
- **Calibration:** ECE, reliability diagrams, Brier score and NLL.

When tuned, Adam and AdamW perform almost identically and well. SGD, given the same tuning budget, falls clearly behind and varies more between seeds, especially on MPOLD. All optimizers show some miscalibration. Temperature scaling changes calibration only slightly, in a way that depends on the dataset, and it leaves rankings untouched. Choosing the decision threshold on validation data proved unstable. The practical lesson is to start from a tuned AdamW baseline and to check probability quality separately from F1.

## Key points
- **Data:** OffensEval20 Arabic is split into stratified train/val/test of 6,743/1,446/1,446 posts, with about 19.9% OFF. MPOLD is split 2,800/600/600, with about 16.8–16.9% OFF.
- **Setup:** QARiB with a linear head on [CLS], batch size 16, maximum length 256, up to 5 epochs, gradient clipping at 1.0, and early stopping on validation macro-F1 with patience 2.
- **Grid:**
  - Learning rate is 2e-5 for Adam/AdamW and 1e-4 for SGD with momentum 0.9.
  - Weight decay is 0 or 0.01, and warmup ratio is 0 or 0.06.
  - Seeds are 42, 1978 and 2025.
- **Preprocessing:** emojis, elongations and non-standard spellings are kept. URLs and mentions are removed, and only whitespace is normalized, with no Arabic letter normalization.
- **Calibration and thresholds:** ECE uses 15 equal-width bins on p(OFF). On OffensEval20 the threshold tuned for AdamW swung widely across seeds (0.34 ± 0.57) and gave no dependable test gain. For SGD the tuned threshold stayed close to 0.5.

## Interesting ideas
- Measure ranking quality (AUC-PR for the rare class) separately from probability quality (ECE plus Brier/NLL), since a model can rank well yet be badly over- or under-confident.
- Do not trust a single binned ECE number: confirm it with proper scoring rules and reliability diagrams, because binning choices can distort the picture.
- Temperature scaling is a cheap, ranking-preserving fix for confidence, fitted on held-out data after training.
- Re-pick the decision threshold on validation data, but report how much the optimal threshold moves across seeds, as an instability warning.
- Keep real-world orthographic noise in the input instead of normalizing it away, so the evaluation matches deployment conditions.

## For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

- **I1** (learn): Mova's evaluation could report, for each confidence-bearing component, both how well the scores order correct choices and how honest the numbers themselves are.
- **I2** (learn): when the confidences of witness parsers are compared for silver-data agreement, reliability plots plus NLL would show whether a parser's stated certainty can actually be used for voting weights.
- **I3** (tree): gparse arc and label scores could be temperature-scaled on a dev treebank, so that downstream frames receive meaningful attachment probabilities without changing the parses.
- **I4** (frames): thresholds for flagging irony or implicature should come with seed or fold variance, so a fragile cutoff is noticed before it is trusted.
- **I5** (tokens): the tokenizer and tagger could be tested on raw, unnormalized text (elongations, emoji, odd spellings) alongside clean text, to expose brittleness early.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

📚 Shelves (2): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works

🧭 Topics (2): [😊 Sentiment & emotion](https://md.research.elamur.ai/free/en/topics/sentiment-emotion/) · 972 works · [Semitic languages](https://md.research.elamur.ai/free/en/topics/semitic-languages/) · 2,908 works

## Cite

Cite as: Auda Elshokry and Mohammed Alhanjouri. 2026. Optimizer Choice and Calibration for QARiB on Arabic-Script Social Media Offensive Language Detection. Proceedings of the 2nd Workshop on NLP for Languages Using Arabic Script.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{elshokry2026optimizer,
  title = {{Optimizer Choice and Calibration for QARiB on Arabic-Script Social Media Offensive Language Detection}},
  author = {Auda Elshokry and Mohammed Alhanjouri},
  booktitle = {Proceedings of the 2nd Workshop on NLP for Languages Using Arabic Script},
  year = {2026},
  doi = {10.18653/v1/2026.abjadnlp-1.4},
  url = {https://doi.org/10.18653/v1/2026.abjadnlp-1.4}
}
```

- [← previous Optical Character Recognition for the International Phonetic Alphabet](https://md.research.elamur.ai/free/en/papers/okabe-2026-optical-character-recognition-international-phonetic/)
- [next → Optimizing Efficiency in Multi-Stage Semantic Re-ranking Architectures](https://md.research.elamur.ai/free/en/papers/novais-2026-optimizing-efficiency-multi-stage-semantic/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/38/)

## References

31 references, 10 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Jansen, and Joni Salminen. 2020. [A multi-platform arabic news comment dataset for offensive language detection](https://md.research.elamur.ai/free/en/papers/chowdhury-2020-multi-platform-arabic-news-comment/). In Proceedings of the 12th Language Resources and Evaluation Conference (LREC 2020).
2. Ahmed Abdelali, Hamdy Mubarak, Kareem Darwish, and Walid Magdy. 2021. QARiB: QCRI arabic and dialectal BERT. arXiv preprint. [find](https://scholar.google.com/scholar?q=Ahmed+Abdelali%2C+Hamdy+Mubarak%2C+Kareem+Darwish%2C+and+Walid+Magdy%2E+2021%2E+QARiB%3A+QCRI+arabic+and+dialectal+BERT%2E+arXiv+preprint%2E)
3. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. [BERT: Pre-training of deep bidirectional transformers for language understanding](https://md.research.elamur.ai/free/en/papers/devlin-2019-bert-pre-training-deep-bidirectional/). In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 4171–4186, Minneapolis, Minnesota. Association for Computational Linguistics. [↗](https://doi.org/10.18653/v1/n19-1423)
4. Muhammad Abdul-Mageed, AbdelRahim Elmadany, and El Moatez Billah Nagoudi. 2021. [ARBERT & MARBERT: Deep bidirectional transformers for Arabic](https://md.research.elamur.ai/free/en/papers/abdul-mageed-2021-arbert-marbert-deep-bidirectional-transformers/). In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers). [↗](https://doi.org/10.48550/arxiv.2101.01785)
5. AbjadNLP Organizers. 2026. Abjadnlp workshop. [https://wp.lancs.ac.uk/abjad/](https://wp.lancs.ac.uk/abjad/). Accessed: 2025-12-18. [↗](https://wp.lancs.ac.uk/abjad/)
6. Raiomond Doctor, Alexander Gutkin, Cibu Johny, Brian Roark, and Richard Sproat. 2022. Graphemic normalization of the perso-arabic script. arXiv preprint arXiv:2210.12273. [↗](https://arxiv.org/abs/2210.12273)
7. Bashar Alhafni, Sarah Al-Towaity, Ziyad Fawzy, Fatema Nassar, Fadhl Eryani, Houda Bouamor, and Nizar Habash. 2024. [Exploiting dialect identification in automatic dialectal text normalization](https://md.research.elamur.ai/free/en/papers/alhafni-2024-exploiting-dialect-identification-automatic-dialectal/). arXiv preprint.
8. Isabel O. Gallegos, Ryan A. Rossi, Joe Barrow, Md Mehrab Tanjim, Sungchul Kim, Franck Dernoncourt, Tong Yu, Ruiyi Zhang, and Nesreen K. Ahmed. 2023. [Bias and fairness in large language models: A survey](https://md.research.elamur.ai/free/en/papers/gallegos-2023-bias-fairness-large-language-models/). arXiv preprint arXiv:2309.00770. [↗](https://arxiv.org/abs/2309.00770)
9. Wissam Antoun, Fady Baly, and Hazem Hajj. 2020. [AraBERT: Transformer-based model for Arabic language understanding](https://md.research.elamur.ai/free/en/papers/antoun-2020-arabert-transformer-based-model-arabic/). In Proceedings of the 4th Workshop on Open-Source Arabic Corpora and Processing Tools, with a Shared Task on Offensive Language Detection, pages 9–15, Marseille, France. European Language Resource Association. [↗](https://doi.org/10.48550/arxiv.2003.00104)
10. Nefeli Gkouti, Prodromos Malakasiotis, Stavros Toumpis, and Ion Androutsopoulos. 2024. Should I try multiple optimizers when fine-tuning a pre-trained transformer for NLP tasks? should I tune their hyperparameters? In Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers), pages 2555– 2574, St. Julian’s, Malta. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Nefeli+Gkouti%2C+Prodromos+Malakasiotis%2C+Stavros+Toumpis%2C+and+Ion+Androutsopoulos%2E+2024%2E+Should+I+try+multiple+optimizers+when+fine%2Dtuning+a+pre%2Dtrained+transformer+for+NLP+tasks%3F+should+I+tune+their+hy)
11. Mohammed Attia. 2007. [Arabic tokenization system](https://md.research.elamur.ai/free/en/papers/attia-2007-arabic-tokenization-system/). In Proceedings of the 2007 Workshop on Computational Approaches to Semitic Languages: Common Issues and Resources, pages 65–72, Prague, Czech Republic. Association for Computational Linguistics.
12. Chuan Guo, Geoff Pleiss, Yu Sun, and Kilian Q. Weinberger. 2017. On calibration of modern neural networks. In Proceedings of the 34th International Conference on Machine Learning, pages 1321–1330. [↗](https://doi.org/10.48550/arxiv.1706.04599)
13. Emily M. Bender, Timnit Gebru, Angelina McMillan-Major, and Shmargaret Shmitchell. 2021. On the dangers of stochastic parrots: Can language models be too big? In Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency (FAccT ’21). [find](https://scholar.google.com/scholar?q=Emily+M%2E+Bender%2C+Timnit+Gebru%2C+Angelina+McMillan%2DMajor%2C+and+Shmargaret+Shmitchell%2E+2021%2E+On+the+dangers+of+stochastic+parrots%3A+Can+language+models+be+too+big%3F+In+Proceedings+of+the+2021+ACM+Conference)
14. Nizar Habash, Fadhl Eryani, Salam Khalifa, Owen Rambow, Dana Abdulrahim, Alexander Erdmann, Nadir Habash, Fahad Al-Obaidli, Mahmoud Al-Taie, and Wajdi Zaghouani. 2018. [Unified guidelines and resources for arabic dialect orthography](https://md.research.elamur.ai/free/en/papers/habash-2018-unified-guidelines-resources-arabic-dialect/). In Sabit Hassan Chowdhury, Hamdy Mubarak, Ahmed Abdelali, Soon-Gyo Jung, Bernard J.
15. Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018). [find](https://scholar.google.com/scholar?q=Proceedings+of+the+Eleventh+International+Conference+on+Language+Resources+and+Evaluation+%28LREC+2018%29%2E)
16. Takaya Saito and Marc Rehmsmeier. 2015. The precision–recall plot is more informative than the ROC plot when evaluating binary classifiers on imbalanced datasets. PLOS ONE, 10(3):e0118432. [↗](https://doi.org/10.1371/journal.pone.0118432)
17. Alon Halfon, Shai Gretz, Ofir Arviv, Artem Spector, Orith Toledo-Ronen, Yoav Katz, Liat Ein-Dor, Michal Shmueli-Scheuer, and Noam Slonim. 2024. Stay tuned: An empirical study of the impact of hyperparameters on LLM tuning in real-world applications. arXiv preprint. [↗](https://doi.org/10.48550/arxiv.2407.18990)
18. Maohao Shen, Subhro Das, Kristjan Greenewald, Prasanna Sattigeri, Gregory W. Wornell, and Soumya Ghosh. 2024. Thermometer: Towards universal calibration for large language models. In Proceedings of the 41st International Conference on Machine Learning, volume 235 of Proceedings of Machine Learning Research, pages 44687– 44711. [↗](https://doi.org/10.48550/arxiv.2403.08819)
19. Diederik P. Kingma and Jimmy Ba. 2015. [Adam: A method for stochastic optimization](https://md.research.elamur.ai/free/en/papers/kingma-2014-adam-method-stochastic-optimization/). In International Conference on Learning Representations (ICLR). [↗](https://doi.org/10.48550/arxiv.1412.6980)
20. Ashia C. Wilson, Rebecca Roelofs, Mitchell Stern, Nathan Srebro, and Benjamin Recht. 2017. The marginal value of adaptive gradient methods in machine learning. In Advances in Neural Information Processing Systems. [find](https://scholar.google.com/scholar?q=Ashia+C%2E+Wilson%2C+Rebecca+Roelofs%2C+Mitchell+Stern%2C+Nathan+Srebro%2C+and+Benjamin+Recht%2E+2017%2E+The+marginal+value+of+adaptive+gradient+methods+in+machine+learning%2E+In+Advances+in+Neural+Information+Proces)
21. Ananya Kumar, Percy Liang, and Tengyu Ma. 2019. Verified uncertainty calibration. In Advances in Neural Information Processing Systems (NeurIPS). [↗](https://doi.org/10.48550/arxiv.1909.10155)
22. Ilya Loshchilov and Frank Hutter. 2019. Decoupled weight decay regularization. In International Conference on Learning Representations (ICLR). [find](https://scholar.google.com/scholar?q=Ilya+Loshchilov+and+Frank+Hutter%2E+2019%2E+Decoupled+weight+decay+regularization%2E+In+International+Conference+on+Learning+Representations+%28ICLR%29%2E)
23. Johnathan Xie, Annie S. Chen, Yoonho Lee, Eric Mitchell, and Chelsea Finn. 2024. Calibrating language models with adaptive temperature scaling. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 18128–18138, Miami, Florida, USA. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Johnathan+Xie%2C+Annie+S%2E+Chen%2C+Yoonho+Lee%2C+Eric+Mitchell%2C+and+Chelsea+Finn%2E+2024%2E+Calibrating+language+models+with+adaptive+temperature+scaling%2E+In+Proceedings+of+the+2024+Conference+on+Empirical+Metho)
24. Marius Mosbach, Maksym Andriushchenko, and Dietrich Klakow. 2021. On the stability of fine-tuning BERT: Misconceptions, explanations, and strong baselines. In International Conference on Learning Representations (ICLR). [↗](https://doi.org/10.48550/arxiv.2006.04884)
25. Marcos Zampieri, Preslav Nakov, Sara Rosenthal, Pepa Atanasova, Georgi Karadzhov, Hamdy Mubarak, Leon Derczynski, Zeses Pitenis, and Çağrı Çöltekin. 2020. SemEval-2020 task 12: Multilingual offensive language identification in social media (OffensEval 2020). In Proceedings of the Fourteenth Workshop on Semantic Evaluation, pages 1425–1447, Barcelona (online). International Committee for Computational Linguistics. [↗](https://doi.org/10.48550/arxiv.2006.07235)
26. Hamdy Mubarak, Ammar Rashed, Kareem Darwish, Younes Samih, and Ahmed Abdelali. 2021. [Arabic offensive language on twitter: Analysis and experiments](https://md.research.elamur.ai/free/en/papers/mubarak-2021-arabic-offensive-language-twitter-analysis/). In Proceedings of the Sixth Arabic Natural Language Processing Workshop, pages 126–135, Kyiv, Ukraine (Virtual). Association for Computational Linguistics. [↗](https://doi.org/10.48550/arxiv.2004.02192)
27. Balamurali Murugesan, Julio Silva-Rodríguez, Ismail Ben Ayed, and Jose Dolz. 2024. Robust calibration of large vision–language adapters. arXiv preprint. [find](https://scholar.google.com/scholar?q=Balamurali+Murugesan%2C+Julio+Silva%2DRodr%C3%ADguez%2C+Ismail+Ben+Ayed%2C+and+Jose+Dolz%2E+2024%2E+Robust+calibration+of+large+vision%E2%80%93language+adapters%2E+arXiv+preprint%2E)
28. Jingzhao Zhang, Sai Praneeth Karimireddy, Andreas Veit, Seungyeon Kim, Sashank J. Reddi, Sanjiv Kumar, and Suvrit Sra. 2020. Why are adaptive methods good for attention models? In Advances in Neural Information Processing Systems. [↗](https://doi.org/10.48550/arxiv.1912.03194)
29. Rebecca Roelofs, Nicholas Cain, Jonathon Shlens, and Michael C. Mozer. 2022. Mitigating bias in calibration error estimation. In Proceedings of the 25th International Conference on Artificial Intelligence and Statistics (AISTATS). [find](https://scholar.google.com/scholar?q=Rebecca+Roelofs%2C+Nicholas+Cain%2C+Jonathon+Shlens%2C+and+Michael+C%2E+Mozer%2E+2022%2E+Mitigating+bias+in+calibration+error+estimation%2E+In+Proceedings+of+the+25th+International+Conference+on+Artificial+Intellig)
30. Yushun Zhang, Congliang Chen, Tian Ding, Ziniu Li, Ruoyu Sun, and Zhi-Quan Luo. 2024. Why transformers need Adam: A hessian perspective. In Advances in Neural Information Processing Systems, volume 37. [find](https://scholar.google.com/scholar?q=Yushun+Zhang%2C+Congliang+Chen%2C+Tian+Ding%2C+Ziniu+Li%2C+Ruoyu+Sun%2C+and+Zhi%2DQuan+Luo%2E+2024%2E+Why+transformers+need+Adam%3A+A+hessian+perspective%2E+In+Advances+in+Neural+Information+Processing+Systems%2C+volume+37)
31. Pan Zhou, Jiashi Feng, Chao Ma, Caiming Xiong, Steven Hoi, and Weinan E. 2020. Towards theoretically understanding why SGD generalizes better than Adam in deep learning. In Advances in Neural Information Processing Systems. [↗](https://doi.org/10.48550/arxiv.2010.05627)

*Updated 1 November 2026*