---
title: "Supachoke at AbjadMed — paper by Nguyen et al. (2026)"
description: "The paper addresses the classification of Arabic medical question–answer pairs into one of 82 disease categories, as defined… 15 references."
authors: ["Thanh Phu Nguyen", "Tuan Thai Huy Nguyen Cu", "Son Thai Pham", "Tri Duy Ho Nguyen"]
year: 2026
venue: "Proceedings of the 2nd Workshop on NLP for Languages Using Arabic Script"
source: "https://aclanthology.org/2026.abjadnlp-1.18/"
doi: "10.18653/v1/2026.abjadnlp-1.18"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/nguyen-2026-supachoke-abjadmed-enhancing-arabic-medical/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/nguyen-2026-supachoke-abjadmed-enhancing-arabic-medical/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/44/) / Supachoke at AbjadMed: Enhancing Arabic Medical Text Classification Using Fine-Tuned AraBERT

# Supachoke at AbjadMed: Enhancing Arabic Medical Text Classification Using Fine-Tuned AraBERT

**Authors:** Thanh Phu Nguyen, Tuan Thai Huy Nguyen Cu, Son Thai Pham and Tri Duy Ho Nguyen · **Year:** 2026 · **Venue:** [Proceedings of the 2nd Workshop on NLP for Languages Using Arabic Script](https://md.research.elamur.ai/free/en/venues/abjadnlp/) ([Workshop on NLP for Languages Using Arabic Script 2026](https://md.research.elamur.ai/free/en/venues/abjadnlp/2026/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2026.abjadnlp-1.18/) · [PDF](https://aclanthology.org/2026.abjadnlp-1.18.pdf) · [DOI 10.18653/v1/2026.abjadnlp-1.18](https://doi.org/10.18653/v1/2026.abjadnlp-1.18) · [OpenAlex](https://openalex.org/W7140107739)

**Citations:** 0 citations (OpenAlex, 2026-10-14)

## Summary
The paper addresses the classification of Arabic medical question–answer pairs into one of 82 disease categories, as defined by the EACL 2026 AbjadNLP shared task. The domain is difficult for several reasons. Specialist vocabulary appears alongside dialectal and misspelled user text, Arabic morphology increases sparsity, some categories overlap in meaning, and the label distribution is strongly skewed.

The Supachoke team (UIT, VNU-HCM) cleaned the data with ArabertPreprocessor. They then fine-tuned bert-base-arabertv02 end to end, with a linear head on the [CLS] representation. To counter imbalance, they weighted the cross-entropy loss by inverse class frequency. They compared this system against a TF-IDF + linear SVM baseline, SaudiBERT, CamelBERT and an unweighted AraBERT.

Weighted AraBERT performed best, scoring 0.4076 on the private leaderboard and placing 13th. This is still far below the winning score of 0.6732. The authors attribute the gap to three choices: no domain-adaptive pretraining, no ensembling, and merging question and answer into a single flat input. The study provides a simple, reproducible reference point for Arabic medical NLP and shows a measurable benefit from loss reweighting.

## Key points
- **Task setup:** single-label, 82-class classification of Arabic medical QA pairs. The training data was split 90/10 with stratification for training and validation.
- **Model and training:** bert-base-arabertv02 (12 layers, 768 hidden size, 12 heads, ~135M parameters) was trained with these settings:
  - AdamW at learning rate 5e-5;
  - up to 8 epochs, with early stopping (patience 3);
  - gradient accumulation of 2 steps;
  - FP16 mixed precision.
- **Private scores:**

  | System | Score |
  |---|---|
  | TF-IDF + SVM | 0.3508 |
  | SaudiBERT | 0.3570 |
  | AraBERT without class weights | 0.3759 |
  | CamelBERT | 0.3823 |
  | AraBERT with class weights | 0.4076 |

- **Ranking:** the system placed 13th, 0.2656 behind the top team, F.A.H (0.6732). The authors suggest SaudiBERT's Twitter-heavy pretraining suits this mixed formal/informal medical text poorly.
- **Code and future work:** the code is released on GitHub (PhuThanh3012/Supachoke_AbjadNLP). Planned next steps are domain-adaptive pretraining on an Arabic medical corpus, voting ensembles and hierarchical classifiers.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works

🧭 Topics (1): [Semitic languages](https://md.research.elamur.ai/free/en/topics/semitic-languages/) · 2,908 works

## Cite

Cite as: Thanh Phu Nguyen, Tuan Thai Huy Nguyen Cu, Son Thai Pham and Tri Duy Ho Nguyen. 2026. Supachoke at AbjadMed: Enhancing Arabic Medical Text Classification Using Fine-Tuned AraBERT. Proceedings of the 2nd Workshop on NLP for Languages Using Arabic Script.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{nguyen2026supachoke,
  title = {{Supachoke at AbjadMed: Enhancing Arabic Medical Text Classification Using Fine-Tuned AraBERT}},
  author = {Thanh Phu Nguyen, Tuan Thai Huy Nguyen Cu, Son Thai Pham and Tri Duy Ho Nguyen},
  booktitle = {Proceedings of the 2nd Workshop on NLP for Languages Using Arabic Script},
  year = {2026},
  doi = {10.18653/v1/2026.abjadnlp-1.18},
  url = {https://doi.org/10.18653/v1/2026.abjadnlp-1.18}
}
```

- [← previous Sujith Kanakkassery at AbjadMed: Imbalance-Aware Transformer Fine-tuning for Arabic Medical Text Classification](https://md.research.elamur.ai/free/en/papers/kanakkassery-2026-sujith-kanakkassery-abjadmed-imbalance-aware/)
- [next → Superficial Success vs. Internal Breakdown: An Empirical Study of Generalization in Adaptive Multi-Agent Systems](https://md.research.elamur.ai/free/en/papers/so-2026-superficial-success-vs-internal-breakdown/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/44/)

## References

15 references, 6 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Muhammad Abdul-Mageed, AbdelRahim Elmadany, and El Moatez Billah Nagoudi. 2021. [ARBERT & MARBERT: Deep bidirectional transformers for Arabic](https://md.research.elamur.ai/free/en/papers/abdul-mageed-2021-arbert-marbert-deep-bidirectional-transformers/). In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 7088–7105, Online. Association for Computational Linguistics. [↗](https://doi.org/10.18653/v1/2021.acl-long.551)
2. Ali Saleh Alammary. 2022. Bert models for arabic text classification: A systematic review. Applied Sciences, 12(11). [↗](https://doi.org/10.3390/app12115720)
3. Asmaa Alrayzah, Fawaz Alsolami, and Mostafa Saleh. 2023. Challenges and opportunities for arabic question-answering systems: current techniques and future directions. PeerJ Computer Science, 9:e1633. [↗](https://doi.org/10.7717/peerj-cs.1633)
4. Wissam Antoun, Fady Baly, and Hazem Hajj. 2021. [Arabert: Transformer-based model for arabic language understanding](https://md.research.elamur.ai/free/en/papers/antoun-2020-arabert-transformer-based-model-arabic/). [↗](https://doi.org/10.48550/arxiv.2003.00104)
5. Seyyed Mohammad Hossein Dadgar, Mohammad Shirzad Araghi, and Morteza Mastery Farahani. 2016. A novel text mining approach based on tf-idf and support vector machine for news classification. In 2016 IEEE International Conference on Engineering and Technology (ICETECH), pages 112–116. IEEE. [find](https://scholar.google.com/scholar?q=Seyyed+Mohammad+Hossein+Dadgar%2C+Mohammad+Shirzad+Araghi%2C+and+Morteza+Mastery+Farahani%2E+2016%2E+A+novel+text+mining+approach+based+on+tf%2Didf+and+support+vector+machine+for+news+classification%2E+In+2016+IE)
6. In this paper, we presented an effective baseline for Arabic medical text classification using a fine-tuned AraBERT model with a class-weighted loss function. Our approach achieved a private leaderboard Jacob Devlin, Ming-Wei Chang, Kenton Lee, and score of 0.4076, [significantly outperforming the Kristina Toutanova](https://md.research.elamur.ai/free/en/papers/devlin-2019-bert-pre-training-deep-bidirectional/). 2018. BERT: pre-training of TF-IDF baseline (0.3508) and other Arabic trans-deep bidirectional transformers for language understanding. CoRR, abs/1810.04805. former variants like SaudiBERT (0.3570). Despite these promising results, the performance Pranav Gupta, Niranjan Kumar M, Balaji Nagarajan, gap with state-of-the-art systems highlights the lim-Imed Zitouni, and Mo El-Haj. 2026. Abjadmed: Ara-itations of standard fine-tuning on small, imbal-bic medical text classification at abjadnlp 2026. In 130 [↗](https://arxiv.org/abs/1810.04805)
7. Proceedings of the 2nd Workshop on NLP for Languages Using Arabic Script (AbjadNLP 2026), co-located with the 19th Conference of the European Chapter of the Association for Computational Linguistics (EACL 2026), Rabat, Morocco. [find](https://scholar.google.com/scholar?q=Proceedings+of+the+2nd+Workshop+on+NLP+for+Languages+Using+Arabic+Script+%28AbjadNLP+2026%29%2C+co%2Dlocated+with+the+19th+Conference+of+the+European+Chapter+of+the+Association+for+Computational+Linguistics+%28)
8. KHALED Wail Hamood, AL-SARRAYRIH Haytham Saleem, and KNIPPING Lars. 2014. Arabic text categorization using improved k-nearest neighbour algorithm. Journal of Applied Computer Science & Mathematics, 8(3):9–12. [find](https://scholar.google.com/scholar?q=KHALED+Wail+Hamood%2C+AL%2DSARRAYRIH+Haytham+Saleem%2C+and+KNIPPING+Lars%2E+2014%2E+Arabic+text+categorization+using+improved+k%2Dnearest+neighbour+algorithm%2E+Journal+of+Applied+Computer+Science+%26+Mathematics%2C+8%28)
9. Fouzi Harrag and Eyas El-Qawasmah. 2009. Neural network for arabic text classification. In 2009 Second International Conference on the Applications of Digital Information and Web Technologies, pages 778–783. IEEE. [find](https://scholar.google.com/scholar?q=Fouzi+Harrag+and+Eyas+El%2DQawasmah%2E+2009%2E+Neural+network+for+arabic+text+classification%2E+In+2009+Second+International+Conference+on+the+Applications+of+Digital+Information+and+Web+Technologies%2C+pages+7)
10. Go Inoue, Bashar Alhafni, Nurpeiis Baimukan, Houda Bouamor, and Nizar Habash. 2021. [The interplay of variant, size, and task type in Arabic pre-trained language models](https://md.research.elamur.ai/free/en/papers/inoue-2021-interplay-variant-size-task-type/). In Proceedings of the Sixth Arabic Natural Language Processing Workshop, Kyiv, Ukraine (Online). Association for Computational Linguistics.
11. Man Lan. 2026. Natural language processing and text mining. In Artificial Intelligence for Drug Design, pages 189–215. Springer. [↗](https://doi.org/10.1007/978-981-95-2525-6_18)
12. Ilya Loshchilov and Frank Hutter. 2019. Decoupled weight decay regularization. [find](https://scholar.google.com/scholar?q=Ilya+Loshchilov+and+Frank+Hutter%2E+2019%2E+Decoupled+weight+decay+regularization%2E)
13. Faisal Qarah. 2024. [Saudibert: A large language model pretrained on saudi dialect corpora](https://md.research.elamur.ai/free/en/papers/qarah-2024-saudibert-large-language-model-pretrained/). arXiv preprint arXiv:2405.06239. [↗](https://arxiv.org/abs/2405.06239)
14. Ali Safaya, Moutasem Abdullatif, and Deniz Yuret. 2020. [KUISAIL at SemEval-2020 task 12: BERT-CNN for offensive speech identification in social media](https://md.research.elamur.ai/free/en/papers/safaya-2020-kuisail-semeval-2020-task-12/). In Proceedings of the Fourteenth Workshop on Semantic Evaluation, pages 2054–2059, Barcelona (online). International Committee for Computational Linguistics.
15. Eric A Stone. 2014. Predictor performance with stratified data and imbalanced classes. Nature methods, 11(8):782–783. [↗](https://doi.org/10.1038/nmeth.3045)

*Updated 1 November 2026*