--- title: "KvochurHegel at AbjadMed — paper by Le (2026)" description: "This system paper addresses the AbjadMed shared task: assigning Arabic medical question-answer pairs to one of 82 categories. 8 references." authors: ["Minh-Hoang Le"] year: 2026 venue: "Proceedings of the 2nd Workshop on NLP for Languages Using Arabic Script" source: "https://aclanthology.org/2026.abjadnlp-1.16/" doi: "10.18653/v1/2026.abjadnlp-1.16" licence: "CC BY 4.0 (ACL Anthology)" licence_url: "https://creativecommons.org/licenses/by/4.0/" summary_licence: "Elamur Cosmic License" canonical: "https://research.elamur.ai/free/en/papers/le-2026-kvochurhegel-abjadmed-combining-ldam-loss/" --- [Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/le-2026-kvochurhegel-abjadmed-combining-ldam-loss/) [home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/34/) / KvochurHegel at AbjadMed: Combining LDAM Loss and Adversarial Training for Arabic Medical Question-Answer Classification # KvochurHegel at AbjadMed: Combining LDAM Loss and Adversarial Training for Arabic Medical Question-Answer Classification **Authors:** Minh-Hoang Le · **Year:** 2026 · **Venue:** [Proceedings of the 2nd Workshop on NLP for Languages Using Arabic Script](https://md.research.elamur.ai/free/en/venues/abjadnlp/) ([Workshop on NLP for Languages Using Arabic Script 2026](https://md.research.elamur.ai/free/en/venues/abjadnlp/2026/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/) **Links:** [Publisher's page](https://aclanthology.org/2026.abjadnlp-1.16/) · [PDF](https://aclanthology.org/2026.abjadnlp-1.16.pdf) · [DOI 10.18653/v1/2026.abjadnlp-1.16](https://doi.org/10.18653/v1/2026.abjadnlp-1.16) · [OpenAlex](https://openalex.org/W7140110288) **Citations:** 0 citations (OpenAlex, 2026-10-14) ## Summary This system paper addresses the AbjadMed shared task: assigning Arabic medical question-answer pairs to one of 82 categories. The label set is strongly long-tailed and has several near-synonymous classes. Arabic medical encoders mostly target NER or span extraction, so the author fine-tunes the general-purpose ARBERTv2 and focuses on optimization rather than domain pre-training. The pipeline has five parts: - **Head-tail truncation**, which keeps both the patient's question and the doctor's closing advice within 512 tokens. - **Layer-wise learning rate decay.** - **FGM adversarial perturbations** on embeddings. - **LDAM margin loss** with deferred re-weighting. - **A manual tweak** that widens margins only for confusable dental, pediatric and psychiatric classes. The system scores Macro-F1 0.4028 on the private test set, placing 16th of 42. This is slightly above its public score, which the author reads as evidence that the regularizers limited overfitting. Per-class analysis shows the method's limits. Classes with fewer than ten examples are never learned, and lexically overlapping categories stay confused. The author concludes that better data, not further loss engineering, is likely needed. The work matters as a practical baseline for imbalanced, fine-grained classification in a lower-resource language. ## Key points - Data: 27,951 training items and a blind test set of 18,634, split evenly into public and private halves. Head classes have 600 examples each, while classes such as Biochemistry have only 7, an imbalance of roughly 86:1. - Backbone: ARBERTv2, a BERT-Base model pre-trained on 243GB of MSA text. The [CLS] vector (768 dimensions) feeds a linear head. - Hyperparameters: - LLRD decay factor ξ = 0.9 over 12 layers. - FGM perturbation size ϵ = 0.2. - LDAM margin constant C = 1.0, with cross-entropy for the first 5 epochs before switching to LDAM. - Training ran for 8,000 steps (about 20 epochs) at batch size 64 on one A100 in BF16. - Results: Macro-F1 of 0.4028 private versus 0.3968 public, ranking 16 of 42. - Validation F1 by class: - Addiction 0.86 and Diabetes 0.84. - Child Health 0.50, Dentistry 0.48 and Dental Diseases 0.27. - Biochemistry and Vascular Surgery both 0.00. ## Interesting ideas - When an input must be cut to fit, keep the question plus the opening and closing parts of the answer, with a guaranteed minimum budget for the answer. Conclusions in such texts tend to sit at the end. - The class counts used to compute margins can be separated from the counts used for loss weighting. This lets known-confusable but frequent classes get wider decision boundaries without destabilizing the overall weighting. - Deferred re-weighting trains with plain cross-entropy first so that representations settle, and only then switches to an imbalance-aware loss. This avoids distorting early feature learning toward rare classes. - Small gradient-direction perturbations of input embeddings act as a regularizer that smooths boundaries and tolerates noisy labels. - Per-class results are grouped into clusters: distinct frequent classes, ambiguous frequent classes and extreme-rare classes. This separates failures caused by scarcity from failures caused by overlap. ## For Mova How these ideas could serve Mova, our small open language model — its files, stages and tests: - **I1** (answer): Readers for fables and tales could keep the opening setup and the final moral when a story exceeds a model window, instead of cutting the ending. - **I2** (tree): gparse could widen margins only for chronically confused deprels such as obl vs nmod or advcl vs xcomp, while keeping normal frequency weighting. - **I3** (learn): Training on silver treebanks could start with an unweighted phase and then switch to emphasizing rare constructions and labels. - **I4** (learn): Silver data agreed by witness parsers still contains errors, so FGM-style perturbation during gparse training could reduce memorization of those residual errors. - **I5** (learn): Evaluation reports could bucket labels or frames the same way, showing whether errors call for more data or for sharper definitions of near-duplicate categories. *Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.* ## Institutions - [Vietnam National University Ho Chi Minh City](https://md.research.elamur.ai/free/en/universities/vietnam-national-university-ho-chi-minh-city/) — University, Ho Chi Minh City, Vietnam · 21 works · 14 citations - [University of Information Technology](https://md.research.elamur.ai/free/en/universities/university-of-information-technology/) — University, Ho Chi Minh City, Vietnam · 13 works · 7 citations 📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works 🧭 Topics (1): [Semitic languages](https://md.research.elamur.ai/free/en/topics/semitic-languages/) · 2,908 works ## Cite Cite as: Minh-Hoang Le. 2026. KvochurHegel at AbjadMed: Combining LDAM Loss and Adversarial Training for Arabic Medical Question-Answer Classification. Proceedings of the 2nd Workshop on NLP for Languages Using Arabic Script. The paper itself, not this page. BibTeX from the library's record of the work. ```bibtex @inproceedings{le2026kvochurhegel, title = {{KvochurHegel at AbjadMed: Combining LDAM Loss and Adversarial Training for Arabic Medical Question-Answer Classification}}, author = {Minh-Hoang Le}, booktitle = {Proceedings of the 2nd Workshop on NLP for Languages Using Arabic Script}, year = {2026}, doi = {10.18653/v1/2026.abjadnlp-1.16}, url = {https://doi.org/10.18653/v1/2026.abjadnlp-1.16} } ``` - [← previous Konkani Wordnet Resources](https://md.research.elamur.ai/free/en/papers/redkar-2026-konkani-wordnet-resources/) - [next → KWIC view on Constructed Action (CA) and its Collocates in German Sign Language (DGS) – Possibilities and Limitations](https://md.research.elamur.ai/free/en/papers/wahl-2026-kwic-view-constructed-action-ca/) - [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/34/) ## References 8 references, 5 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet. 1. Muhammad Abdul-Mageed, AbdelRahim Elmadany, and El Moatez Billah Nagoudi. 2021. [ARBERT & MARBERT: Deep bidirectional transformers for Arabic](https://md.research.elamur.ai/free/en/papers/abdul-mageed-2021-arbert-marbert-deep-bidirectional-transformers/). In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 7088–7105, Online. Association for Computational Linguistics. Discussion and Conclusion Our results on the AbjadMed (Gupta et al., 2026) demonstrate that general-purpose Arabic encoders can achieve competitive performance (Rank 16/42) [↗](https://doi.org/10.48550/arxiv.2101.01785) 2. Kaidi Cao, Colin Wei, Adrien Gaidon, Nikos Arechiga, and Tengyu Ma. 2019. Learning imbalanced datasets with label-distribution-aware margin loss. In Advances in Neural Information Processing Systems. [↗](https://doi.org/10.48550/arxiv.1906.07413) 3. Yin Cui, Menglin Jia, Tsung-Yi Lin, Yang Song, and Serge Belongie. 2019. Class-balanced loss based on effective number of samples. Preprint, arXiv:1901.05555. [↗](https://arxiv.org/abs/1901.05555) 4. AbdelRahim Elmadany, El Moatez Billah Nagoudi, and Muhammad Abdul-Mageed. 2022. [Orca: A challenging benchmark for arabic language understanding](https://md.research.elamur.ai/free/en/papers/elmadany-2023-orca-challenging-benchmark-arabic-language/). arXiv preprint arXiv:2212.10758. [↗](https://arxiv.org/abs/2212.10758) 5. Pranav Gupta, Niranjan Kumar M, Balaji Nagarajan, Imed Zitouni, and Mo El-Haj. 2026. [Abjadmed: Arabic medical text classification at abjadnlp 2026](https://md.research.elamur.ai/free/en/papers/gupta-2026-abjadmed-arabic-medical-text-classification/). In Proceedings of the 2nd Workshop on NLP for Languages Using Arabic Script (AbjadNLP 2026), co-located with the 19th Conference of the European Chapter of the Association for Computational Linguistics (EACL 2026), Rabat, Morocco. 6. Jeremy Howard and Sebastian Ruder. 2018. Universal language model fine-tuning for text classification. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 328–339, Melbourne, Australia. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Jeremy+Howard+and+Sebastian+Ruder%2E+2018%2E+Universal+language+model+fine%2Dtuning+for+text+classification%2E+In+Proceedings+of+the+56th+Annual+Meeting+of+the+Association+for+Computational+Linguistics+%28Volum) 7. Tsung-Yi Lin, Priya Goyal, Ross Girshick, Kaiming He, and Piotr Dollár. 2017. [Focal loss for dense object detection](https://md.research.elamur.ai/free/en/papers/lin-2017-focal-loss-dense-object-detection/). In Proceedings of the IEEE international conference on computer vision, pages 2980–2988. [↗](https://doi.org/10.1109/iccv.2017.324) 8. Takeru Miyato, Andrew M. Dai, and Ian Goodfellow. 2017. [Adversarial training methods for semi-supervised text classification](https://md.research.elamur.ai/free/en/papers/miyato-2016-adversarial-training-methods-semi-supervised/). In International Conference on Learning Representations. [↗](https://doi.org/10.48550/arxiv.1605.07725) *Updated 1 November 2026*