--- title: "Qais at BAREC Shared Task 2025 — paper by Ahmad (2025)" description: "The author argues that earlier Arabic work used coarse scales of 3 to 7 levels or shallow surface counts… 8 references, cited by 1 paper here." authors: ["Samar Ahmad"] year: 2025 venue: "Proceedings of The Third Arabic Natural Language Processing Conference: Shared Tasks" source: "https://aclanthology.org/2025.arabicnlp-sharedtasks.43/" doi: "10.18653/v1/2025.arabicnlp-sharedtasks.43" licence: "CC BY 4.0 (ACL Anthology)" licence_url: "https://creativecommons.org/licenses/by/4.0/" summary_licence: "Elamur Cosmic License" canonical: "https://research.elamur.ai/free/en/papers/ahmad-2025-qais-barec-shared-task-2025/" --- [Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/ahmad-2025-qais-barec-shared-task-2025/) [home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2025](https://md.research.elamur.ai/free/en/papers/2025/page/15/) / Qais at BAREC Shared Task 2025: A Fine-Grained Approach for Arabic Readability Classification Using a pre-trained model # Qais at BAREC Shared Task 2025: A Fine-Grained Approach for Arabic Readability Classification Using a pre-trained model **Authors:** [Samar Ahmad](https://md.research.elamur.ai/free/en/authors/samar-ahmad/) · **Year:** 2025 · **Venue:** [Proceedings of The Third Arabic Natural Language Processing Conference: Shared Tasks](https://md.research.elamur.ai/free/en/venues/arabicnlp/) ([ArabicNLP 2025](https://md.research.elamur.ai/free/en/venues/arabicnlp/2025/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/) **Links:** [Publisher's page](https://aclanthology.org/2025.arabicnlp-sharedtasks.43/) · [PDF](https://aclanthology.org/2025.arabicnlp-sharedtasks.43.pdf) · [DOI 10.18653/v1/2025.arabicnlp-sharedtasks.43](https://doi.org/10.18653/v1/2025.arabicnlp-sharedtasks.43) · [OpenAlex](https://openalex.org/W4416037768) **Citations:** 0 citations (OpenAlex, 2026-10-14) · [1](https://md.research.elamur.ai/free/en/papers/ahmad-2025-qais-barec-shared-task-2025/cited-by/) in our library, of the 149,439 papers whose bibliographies we read ## Summary This system description comes from the BAREC 2025 shared task on Arabic readability. The task is to assign each sentence one of 19 difficulty levels, from kindergarten to postgraduate. The author argues that earlier Arabic work used coarse scales of 3 to 7 levels or shallow surface counts, and that a fine-grained scale paired with a transformer could give more precise estimates. The system fine-tunes AraBERT (aubmindlab/bert-arabertv2) on segmented, tokenized sentences from the BAREC corpus. There are two settings. The strict track uses the original data, with class weights to counter imbalance. The open track rebalances the training set: ChatGPT generates extra sentences for the rare extreme levels, and the dominant middle levels are cut down. The submissions reach QWK of 82.5% (strict) and 83.0% (open), so rebalancing gives only a small gain. The paper also reports that the base model outperformed the large one and that an ordinal CORAL loss did not help. A brief manual error analysis shows that numbers and Arabic-script transliterations of foreign words mislead the classifier. This matters for tools that grade educational Arabic text. ## Key points - The BAREC corpus has 69,441 sentences (over 1M words) labelled on 19 levels inspired by the Taha/Arabi21 scheme. Inter-annotator agreement is QWK 81.8%. The split is 80/10/10, plus a blind test set of 3,420 items. - Open-track rebalancing worked as follows: - levels 1, 2, 17, 18 and 19 were up-sampled with GPT-generated sentences; - levels 10, 12 and 14 were down-sampled to 7,000 each; - the final set had 59,236 rows. - arabertv2-base scored QWK of about 80–83, while bert-large-arabertv2 dropped to 70–78. The author attributes this to the large model needing more data and compute. - A learning rate of 2e-5 beat 5e-5. Other settings were weight decay 0.01, batch size 4 or 8, up to 20 epochs, and AdamW on Colab A100/T4 GPUs. CORAL ordinal loss reduced accuracy. - Evaluation uses the following metrics, with QWK as the primary one: - QWK; - exact accuracy on 19 levels; - accuracy on scales collapsed to 7, 5 and 3 levels; - ±1-level adjacent accuracy; - mean absolute distance. ## Interesting ideas - An ordinal label is scored from several angles at once: exact match, off-by-one tolerance, coarser collapsed scales, and a distance-weighted agreement such as QWK. This shows whether mistakes are near-misses or gross errors. - When a few classes dominate, cap them at the size of the largest class that remains once those dominant ones are set aside. This gives a principled, non-arbitrary target for down-sampling. - The error analysis found that digits and Arabic-script transliterations of foreign names push difficulty predictions far too high, because the model reads unfamiliar surface forms as hard vocabulary. - The paper reports two negative results: a bigger checkpoint and an ordinal-aware loss both underperformed a plain base model on mid-sized data. This argues for always keeping the simple configuration as a control. - Sentence-level readability on a fine school-grade scale is a usable signal for choosing texts that match a learner's level. ## For Mova How these ideas could serve Mova, our small open language model — its files, stages and tests: - **I1** (learn): Any graded output Mova produces, such as confidence tiers or difficulty levels, could be evaluated with exact, ±1 and collapsed-scale scores so regressions in severity are visible, not just in hit rate. - **I2** (learn): When building silver training sets from witness parsers, frequent constructions could be capped this way so rare relations are not drowned out. - **I3** (tokens): The tagger could mark numerals, dates and foreign proper names as an explicit class, so later stages treat them as names or quantities rather than rare difficult words. - **I4** (learn): Before adopting a larger gparse variant or a fancier loss, Mova should test it against the plain baseline on the same split as a standing check. - **I5** (answer): Mova's school-course readers could tag reading passages with a grade level and check that test passages match the targeted course stage. *Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.* 📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works 🧭 Topics (1): [Semitic languages](https://md.research.elamur.ai/free/en/topics/semitic-languages/) · 2,908 works ## Cite Cite as: Samar Ahmad. 2025. Qais at BAREC Shared Task 2025: A Fine-Grained Approach for Arabic Readability Classification Using a pre-trained model. Proceedings of The Third Arabic Natural Language Processing Conference: Shared Tasks. The paper itself, not this page. BibTeX from the library's record of the work. ```bibtex @inproceedings{ahmad2025qais, title = {{Qais at BAREC Shared Task 2025: A Fine-Grained Approach for Arabic Readability Classification Using a pre-trained model}}, author = {Samar Ahmad}, booktitle = {Proceedings of The Third Arabic Natural Language Processing Conference: Shared Tasks}, year = {2025}, doi = {10.18653/v1/2025.arabicnlp-sharedtasks.43}, url = {https://doi.org/10.18653/v1/2025.arabicnlp-sharedtasks.43} } ``` - [← previous PTUK-HULAT at AraGenEval Shared Task: Fine-tuning XLM-RoBERTa for AI-Generated Arabic News Detection](https://md.research.elamur.ai/free/en/papers/duridi-2025-ptuk-hulat-arageneval-shared-task/) - [next → Quasar at MAHED Shared Task : Decoding Emotions and Offense in Arabic Text using LLM and Transformer-Based Approaches](https://md.research.elamur.ai/free/en/papers/chowdhury-2025-quasar-mahed-shared-task-decoding/) - [↑ back to papers of 2025](https://md.research.elamur.ai/free/en/papers/2025/page/15/) ## References 8 references, 6 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet. 1. Muhamed Al Khalil, Nizar Habash, and Zhengyang Jiang. 2020. [A large-scale leveled readability lexicon for Standard Arabic](https://md.research.elamur.ai/free/en/papers/khalil-2020-large-scale-leveled-readability-lexicon/). In Proceedings of the Twelfth Language Resources and Evaluation Conference, pages 3053–3062, Marseille, France. European Language Resources Association. 2. Nora Al-Twairesh, Abeer Al-Dayel, Hend Al-Khalifa, Maha Al-Yahya, Sinaa Alageel, Nora Abanmy, and Nouf Al-Shenaifi. 2016. [Madad: a readability annotation tool for arabic text](https://md.research.elamur.ai/free/en/papers/twairesh-2016-madad-readability-annotation-tool-arabic/). In Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC’16), pages 4093–4097. 3. Bashar Alhafni, Reem Hazim, Juan David Pineros Liberato, Muhamed Al Khalil, and Nizar Habash. 2025. [The SAMER Arabic text simplification corpus](https://md.research.elamur.ai/free/en/papers/alhafni-2024-samer-arabic-text-simplification-corpus/). In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024), pages 16079–16093, Torino, Italia. ELRA and ICCL. 4. Wissam Antoun, Fady Baly, and Hazem Hajj. 2020. [AraBERT: Transformer-based model for Arabic language understanding](https://md.research.elamur.ai/free/en/papers/antoun-2020-arabert-transformer-based-model-arabic/). In Proceedings of the 4th Workshop on Open-Source Arabic Corpora and Processing Tools, with a Shared Task on Offensive Language Detection, pages 9–15, Marseille, France. European Language Resource Association. [↗](https://doi.org/10.48550/arxiv.2003.00104) 5. Reem Hazim, Hind Saddiki, Bashar Alhafni, Muhamed Al Khalil, and Nizar Habash. 2022. [Arabic word-level readability visualization for assisted text simplification](https://md.research.elamur.ai/free/en/papers/hazim-2022-arabic-word-level-readability-visualization/). In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pages 242–249, Abu Dhabi, UAE. Association for Computational Linguistics. 6. Juan Liberato, Bashar Alhafni, Muhamed Khalil, and Nizar Habash. 2024. [Strategies for Arabic readability modeling](https://md.research.elamur.ai/free/en/papers/liberato-2024-strategies-arabic-readability-modeling/). In Proceedings of the Second Arabic Natural Language Processing Conference, pages 55–66, Bangkok, Thailand. Association for Computational Linguistics. 7. Naoual Nassiri, Abdelhak Lakhouaja, and Violetta Cavalli-Sforza. 2022. Arabic l2 readability assessment: Dimensionality reduction study. Journal of King Saud University-Computer and Information Sciences, 34(6):3789–3799. [↗](https://doi.org/10.1016/j.jksuci.2020.12.021) 8. Hanada Taha-Thomure. 2017. Arabic Language Text Leveling. Educational Book House. [find](https://scholar.google.com/scholar?q=Hanada+Taha%2DThomure%2E+2017%2E+Arabic+Language+Text+Leveling%2E+Educational+Book+House%2E) *Updated 1 November 2026*