---
title: "Tashkees-AI at AbjadMed 2026 — paper by Mohamed (2026)"
description: "This system paper, submitted to the AbjadMed 2026 shared task, assigns Arabic patient questions from medical… 65.4% F1. 5 references."
authors: ["Emad Eldin, Fatimah Mohamed"]
year: 2026
venue: "Proceedings of the 2nd Workshop on NLP for Languages Using Arabic Script"
source: "https://aclanthology.org/2026.abjadnlp-1.20/"
doi: "10.18653/v1/2026.abjadnlp-1.20"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/eldin-2026-tashkees-ai-abjadmed-2026-flat/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/eldin-2026-tashkees-ai-abjadmed-2026-flat/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/44/) / Tashkees-AI at AbjadMed 2026: Flat vs. Hierarchical Classification for Fine-Grained Arabic Medical QA

# Tashkees-AI at AbjadMed 2026: Flat vs. Hierarchical Classification for Fine-Grained Arabic Medical QA

**Authors:** [Emad Eldin, Fatimah Mohamed](https://md.research.elamur.ai/free/en/authors/emad-eldin-fatimah-mohamed/) · **Year:** 2026 · **Venue:** [Proceedings of the 2nd Workshop on NLP for Languages Using Arabic Script](https://md.research.elamur.ai/free/en/venues/abjadnlp/) ([Workshop on NLP for Languages Using Arabic Script 2026](https://md.research.elamur.ai/free/en/venues/abjadnlp/2026/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2026.abjadnlp-1.20/) · [PDF](https://aclanthology.org/2026.abjadnlp-1.20.pdf) · [DOI 10.18653/v1/2026.abjadnlp-1.20](https://doi.org/10.18653/v1/2026.abjadnlp-1.20) · [OpenAlex](https://openalex.org/W7140094932)

**Citations:** 0 citations (OpenAlex, 2026-10-14)

## Summary
This system paper, submitted to the AbjadMed 2026 shared task, assigns Arabic patient questions from medical forums to one of 82 specialty labels. The labels are highly unbalanced and some overlap in meaning. The organizers supplied 27,951 labelled training records and 18,634 blind test items. The author compares three families of approaches:

* **Flat fine-tuning.** Arabic encoders (MARBERTv2, CAMeLBERT-DA, AraBERTv2) are trained with class-weighted cross-entropy.
* **Two-stage hierarchy.** A first model picks one of 12 broad domains, then a domain-specific model chooses the fine label.
* **Retrieval and ensembles.** These include a k-NN retrieval approach and ensembles that mix the transformer with LightGBM or SVM classifiers trained on TF-IDF features.

The simplest option wins: MARBERTv2 fine-tuned on normalized text reaches 0.3659 macro F1 on the blind test, which placed 26th. The hierarchy scores lower (0.332) because first-stage mistakes cannot be corrected later. Adding lexical models to the ensemble does not help. The study matters because it shows that, with noisy labels and extreme imbalance, extra architectural complexity can hurt rather than help.

## Key points
- The data has 82 classes, with between 7 and 600 training examples per class (roughly an 85:1 ratio). Validation used a 90/10 stratified split (25,156 / 2,795).
- On the blind test, cleaned MARBERTv2 scored 0.3659 macro F1, against 0.357 without cleaning. AraBERTv2 reached 0.358 and CAMeLBERT-DA 0.293.
- In the hierarchical system, Stage 1 reached about 65.4% F1 (69.8% accuracy), so roughly 30% of items were already lost before Stage 2. The full pipeline scored 0.332 and ever predicted only 68 of the 82 labels.
- Weaker alternatives:
  - k-NN retrieval (k = 20): 0.230 F1.
  - LightGBM on character 1–3-gram TF-IDF: 0.315 on validation.
  - MARBERT+LGBM mixture: 0.358.
  - 2-fold CV ensemble: 0.3213.
  - MARBERT+TF-IDF SVM hybrid: 0.3150.
- The error analysis found:
  - pairs of near-duplicate labels, for example Pediatric diseases vs Child health with 34 confusions;
  - 14 rare classes that were never predicted;
  - greetings in the questions acting as noise;
  - about 12% label noise among overlapping categories.

## Interesting ideas
- Instead of only scoring a cascade end to end, measure how many items the first stage dooms before later stages run; this shows the ceiling a pipeline can never exceed.
- Report "coverage," meaning how many output labels a model ever produces, alongside the score; it cheaply reveals classes the system has silently given up on.
- When an ensemble of different models brings no gain, read it as a sign that their mistakes are correlated rather than complementary.
- Compare how often each label is predicted with how often it occurs in training; a large gap, such as Pharmacology being over-predicted, exposes a bias toward a few attractor classes.
- Mine the most frequent confusion pairs to separate true model errors from label pairs that annotators themselves cannot tell apart.

## For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

- **I1** (learn): For each step of text → tree → frames → world → answer, report the share of answer failures already caused upstream, so repair effort goes to the stage that really blocks.
- **I2** (frames): Count which VerbNet classes and semantic roles prag ever assigns on a corpus, to find frames that exist in the inventory but are never chosen.
- **I3** (learn): Before trusting agreement among witness parsers as silver data, check how often they fail on the same tokens; correlated witnesses give false confidence.
- **I4** (tokens): Track predicted vs. gold frequency of each POS tag and morphological feature in the en tagger to catch tags it overuses.
- **I5** (tree): Apply the same check to frequently confused UD relations, such as obl vs nmod, during dialect conversion, to decide whether to fix the parser or the conversion rules.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Cairo University](https://md.research.elamur.ai/free/en/universities/cairo-university/) — University, Giza, Egypt · 42 works · 1,350 citations

📚 Shelves (5): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works · [🗄️ Information Systems](https://md.research.elamur.ai/free/en/shelves/computer-science/information-systems/) · 1,437 works

🧭 Topics (2): [Semitic languages](https://md.research.elamur.ai/free/en/topics/semitic-languages/) · 2,908 works · [❓ Question answering](https://md.research.elamur.ai/free/en/topics/question-answering/) · 441 works

## Cite

Cite as: Emad Eldin, Fatimah Mohamed. 2026. Tashkees-AI at AbjadMed 2026: Flat vs. Hierarchical Classification for Fine-Grained Arabic Medical QA. Proceedings of the 2nd Workshop on NLP for Languages Using Arabic Script.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{mohamed2026tashkees,
  title = {{Tashkees-AI at AbjadMed 2026: Flat vs. Hierarchical Classification for Fine-Grained Arabic Medical QA}},
  author = {Emad Eldin, Fatimah Mohamed},
  booktitle = {Proceedings of the 2nd Workshop on NLP for Languages Using Arabic Script},
  year = {2026},
  doi = {10.18653/v1/2026.abjadnlp-1.20},
  url = {https://doi.org/10.18653/v1/2026.abjadnlp-1.20}
}
```

- [← previous TARo: Token-level Adaptive Routing for LLM Test-time Alignment](https://md.research.elamur.ai/free/en/papers/rai-2026-taro-token-level-adaptive-routing/)
- [next → Task Assignment meets Annotator Modeling: Human-LLM Collaborative Annotation with Constraints](https://md.research.elamur.ai/free/en/papers/moriyama-2026-task-assignment-meets-annotator-modeling/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/44/)

## References

5 references, 1 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Muhammad Abdul-Mageed and 1 others. 2021. MAR-BERT: Deep bidirectional transformers for arabic sentiment analysis. In ACL-IJCNLP 2021. [find](https://scholar.google.com/scholar?q=Muhammad+Abdul%2DMageed+and+1+others%2E+2021%2E+MAR%2DBERT%3A+Deep+bidirectional+transformers+for+arabic+sentiment+analysis%2E+In+ACL%2DIJCNLP+2021%2E)
2. Wissam Antoun and 1 others. 2020. [AraBERT: Transformer-based model for arabic language understanding](https://md.research.elamur.ai/free/en/papers/antoun-2020-arabert-transformer-based-model-arabic/). In LREC 2020.
3. The analysis reveals that hierarchical classification Pranav Gupta, Niranjan Kumar M, Balaji Nagarajan, fails primarily due to error cascades, where the Imed Zitouni, and Mo El-Haj. 2026. Abjadmed: Ara-30.2% error rate in Stage 1 creates an insurmount-bic medical text classification at abjadnlp 2026. In able ceiling for Stage 2 models. Furthermore, data Proceedings of the 2nd Workshop on NLP for Languages Using Arabic Script (AbjadNLP 2026), co-fragmentation across specialized models exacer-located with the 19th Conference of the European bates sparsity; some categories contain fewer than Chapter of the Association for Computational Lin-20 samples, preventing effective learning. Flat clas-guistics (EACL 2026), Rabat, Morocco. sification using MARBERTv2 proves superior as Go Inoue, Bashar Alhafni, Nurpeiis Baimukan, Houda it avoids these bottlenecks while preserving the Bouamor, and Nizar Habash. 2021. The interplay semantic relationships across all 82 categories. of variant, size, and task type in Arabic pre-trained language models. In Proceedings of the Sixth Arabic Prioritizing data augmentation for minority Natural Language Processing Workshop, pages 92– classes and utilizing fine-tuned encoders with 104, Kyiv, Ukraine (Virtual). Association for Compu-weighted loss over complex hierarchical archi-tational Linguistics. tectures is recommended. Hierarchical methods Jinhyuk Lee, Wonjin Yoon, Sungdong Kim, Donghyeon should be avoided unless Stage 1 accuracy exceeds Kim, Sunkyu Kim, Chan Ho So, and Jaewoo Kang. 85%, as the loss of morphological information and 2019. BioBERT: a pre-trained biomedical language training data density outweighs the benefits of a representation model for biomedical text mining. narrower decision space. Bioinformatics. 140 [find](https://scholar.google.com/scholar?q=The+analysis+reveals+that+hierarchical+classification+Pranav+Gupta%2C+Niranjan+Kumar+M%2C+Balaji+Nagarajan%2C+fails+primarily+due+to+error+cascades%2C+where+the+Imed+Zitouni%2C+and+Mo+El%2DHaj%2E+2026%2E+Abjadmed%3A+Ar)
4. Carlos N. Silla and Alex A. Freitas. 2011. A survey of hierarchical classification across different application domains. Data Mining and Knowledge Discovery, 22(1):31–72. [find](https://scholar.google.com/scholar?q=Carlos+N%2E+Silla+and+Alex+A%2E+Freitas%2E+2011%2E+A+survey+of+hierarchical+classification+across+different+application+domains%2E+Data+Mining+and+Knowledge+Discovery%2C+22%281%29%3A31%E2%80%9372%2E)
5. Jesse Zhang, Haonan Yu, and Wei Xu. 2021. Hierarchical reinforcement learning by discovering intrinsic options. In International Conference on Learning Representations. [↗](https://doi.org/10.48550/arxiv.2101.06521)

*Updated 1 November 2026*