---
title: "!MSA at BAREC Shared Task 2025: Ensembling Arabic"
description: "The paper describes the winning entry to the BAREC 2025 Shared Task on Arabic readability. 22 references, cited by 1 paper here."
authors: ["Mohamed Basem", "Mohamed Younes", "Seif Ahmed", "Abdelrahman Moustafa"]
year: 2025
venue: "Proceedings of The Third Arabic Natural Language Processing Conference: Shared Tasks"
source: "https://aclanthology.org/2025.arabicnlp-sharedtasks.42/"
doi: "10.18653/v1/2025.arabicnlp-sharedtasks.42"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/basem-2025-msa-barec-shared-task-2025/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/basem-2025-msa-barec-shared-task-2025/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2025](https://md.research.elamur.ai/free/en/papers/2025/page/14/) / !MSA at BAREC Shared Task 2025: Ensembling Arabic Transformers for Readability Assessment

# !MSA at BAREC Shared Task 2025: Ensembling Arabic Transformers for Readability Assessment

**Authors:** [Mohamed Basem](https://md.research.elamur.ai/free/en/authors/mohamed-basem/), [Mohamed Younes](https://md.research.elamur.ai/free/en/authors/mohamed-younes/), [Seif Ahmed](https://md.research.elamur.ai/free/en/authors/seif-ahmed/) and Abdelrahman Moustafa · **Year:** 2025 · **Venue:** [Proceedings of The Third Arabic Natural Language Processing Conference: Shared Tasks](https://md.research.elamur.ai/free/en/venues/arabicnlp/) ([ArabicNLP 2025](https://md.research.elamur.ai/free/en/venues/arabicnlp/2025/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2025.arabicnlp-sharedtasks.42/) · [PDF](https://aclanthology.org/2025.arabicnlp-sharedtasks.42.pdf) · [DOI 10.18653/v1/2025.arabicnlp-sharedtasks.42](https://doi.org/10.18653/v1/2025.arabicnlp-sharedtasks.42) · [OpenAlex](https://openalex.org/W4416037821)

**Citations:** 0 citations (OpenAlex, 2026-10-14) · [1](https://md.research.elamur.ai/free/en/papers/basem-2025-msa-barec-shared-task-2025/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
The paper describes the winning entry to the BAREC 2025 Shared Task on Arabic readability. The task asks systems to place sentences and documents on a fine-grained 1–19 difficulty scale, under Strict, Constrained and Open resource conditions. The main obstacles are a strongly skewed label distribution and limited training data. The authors fine-tune four Arabic encoders: AraBERTv2, AraELECTRA, MARBERT and CAMeL-BERT. They train variants with classification, regression and ordinal (CORAL) objectives and fuse their outputs with confidence weights. Supporting steps include:
- inverse-frequency class weights;
- D3TOK morphological tokenization from CAMeL Tools;
- relabeling the SAMER corpus with their best BAREC model instead of rescaling its labels;
- about 10k synthetic sentences from Gemini 2.5 Flash for the Open track.

Post-processing that fixes skewed document-level prediction distributions adds 6.3 QWK points. The system ranks first in all six tracks in the results table, reaching 87.5 QWK for sentences and 87.4 for documents. This shows that loss diversity and careful data handling pay off on ordinal, imbalanced labels.

## Key points
- BAREC has over 1M words in 68k+ sentences and 1,900+ documents, labeled on 19 levels. Rare levels include 1, 18 and 19, while levels 12 and 14 dominate.
- Single models scored 81.0–84.8 QWK, with MARBERT+COR the strongest. The ensemble reached 87.5 QWK at sentence level (Strict).
- Training on min-max rescaled SAMER labels (3–6 mapped to 1–19) gave only 50% QWK. Relabeling SAMER with the BAREC model worked much better, and mapping the relabels back to the original scale stayed within ±0.5.
- Document-level QWK was 87.4 (Strict), 84.3 (Constrained) and 82.2 (Open). The sentence-level runs also placed first in fields of 39, 20 and 22 teams. The abstract claims six of six wins, while the conclusion says five of six.
- Document scores come from the maximum over sentence predictions. Two fixes in later averaging-based aggregation each helped: a floor instead of a ceiling for borderline cases, and an override toward labels 16/17 when any model predicts them.

## Interesting ideas
- Training the same backbone with cross-entropy, MSE and ordinal (CORAL) losses gives ensemble members that make different errors on an ordered scale, which pure architecture variety may not provide.
- Instead of mechanically mapping one corpus's label scheme onto another, label the foreign data with your in-domain model and check that a reverse mapping roughly recovers the original labels.
- Confidence for fusion is computed per model type: softmax probability for classifiers and inverse variance for regressors, so heterogeneous outputs can be averaged on a common footing.
- Plotting the histogram of predicted labels against the expected distribution exposed an aggregation artifact, where level 10 was never predicted because of ceiling rounding.
- A document is treated as only as easy as its hardest sentence, so difficulty is aggregated with max rather than mean.

## For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

- **I1** (learn): Witness parsers or taggers could be trained with differing objectives, so that silver-data agreement reflects genuinely independent evidence rather than shared biases.
- **I2** (tree): When converting a treebank in another UD dialect, parse it with our parser and accept a conversion only where the result round-trips back to the source annotation.
- **I3** (learn): Votes from the perceptron parser and gparse could be weighted by each witness's own calibrated certainty when building silver arcs, instead of counting raw votes.
- **I4** (learn): Evaluation could routinely compare predicted deprel, role or frame distributions with gold, flagging labels that never appear as a cheap detector of pipeline bugs.
- **I5** (answer): When selecting fables or school-course texts for a target level, the system could rate each text by its hardest sentence to avoid passages that hide one very difficult clause.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [October University of Modern Sciences and Arts](https://md.research.elamur.ai/free/en/universities/october-university-of-modern-sciences-and-arts/) — University, Cairo, Egypt · 10 works · 27 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works

🧭 Topics (1): [Semitic languages](https://md.research.elamur.ai/free/en/topics/semitic-languages/) · 2,908 works

## Cite

Cite as: Mohamed Basem, Mohamed Younes, Seif Ahmed and Abdelrahman Moustafa. 2025. !MSA at BAREC Shared Task 2025: Ensembling Arabic Transformers for Readability Assessment. Proceedings of The Third Arabic Natural Language Processing Conference: Shared Tasks.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{basem2025msa,
  title = {{!MSA at BAREC Shared Task 2025: Ensembling Arabic Transformers for Readability Assessment}},
  author = {Mohamed Basem, Mohamed Younes, Seif Ahmed and Abdelrahman Moustafa},
  booktitle = {Proceedings of The Third Arabic Natural Language Processing Conference: Shared Tasks},
  year = {2025},
  doi = {10.18653/v1/2025.arabicnlp-sharedtasks.42},
  url = {https://doi.org/10.18653/v1/2025.arabicnlp-sharedtasks.42}
}
```

- [← previous !MSA at AraHealthQA 2025 Shared Task: Enhancing LLM Performance for Arabic Clinical Question Answering through Prompt Engineering and Ensemble Learning](https://md.research.elamur.ai/free/en/papers/younes-2025-msa-arahealthqa-2025-shared-task/)
- [next → 912 at TAQEEM 2025: A Distribution-aware Approach to Arabic Essay Scoring](https://md.research.elamur.ai/free/en/papers/vu-2025-912-taqeem-2025-distribution-aware/)
- [↑ back to papers of 2025](https://md.research.elamur.ai/free/en/papers/2025/page/14/)

## References

22 references, 13 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Muhammad Abdul-Mageed, AbdelRahim Elmadany, and El Moatez Billah Nagoudi. 2021. [ARBERT & MARBERT: Deep bidirectional transformers for Arabic](https://md.research.elamur.ai/free/en/papers/abdul-mageed-2021-arbert-marbert-deep-bidirectional-transformers/). In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 7088–7105, Online. Association for Computational Linguistics. [↗](https://doi.org/10.18653/v1/2021.acl-long.551)
2. Alfred V. Aho and Jeffrey D. Ullman. 1972. The Theory of Parsing, Translation and Compiling, volume 1. Prentice-Hall, Englewood Cliffs, NJ. [find](https://scholar.google.com/scholar?q=Alfred+V%2E+Aho+and+Jeffrey+D%2E+Ullman%2E+1972%2E+The+Theory+of+Parsing%2C+Translation+and+Compiling%2C+volume+1%2E+Prentice%2DHall%2C+Englewood+Cliffs%2C+NJ%2E)
3. Muhamed Al Khalil, Nizar Habash, and Zhengyang Jiang. 2020. [A large-scale leveled readability lexicon for Standard Arabic](https://md.research.elamur.ai/free/en/papers/khalil-2020-large-scale-leveled-readability-lexicon/). In Proceedings of the Twelfth Language Resources and Evaluation Conference, pages 3053– 3062, Marseille, France. European Language Resources Association.
4. Bashar Alhafni, Reem Hazim, Juan David Pineros Liberato, Muhamed Al Khalil, and Nizar Habash. 2025. [The SAMER Arabic text simplification corpus](https://md.research.elamur.ai/free/en/papers/alhafni-2024-samer-arabic-text-simplification-corpus/). In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024), pages 16079–16093, Torino, Italia. ELRA and ICCL.
5. American Psychological Association. 1983. Publications Manual. American Psychological Association, Washington, DC. [find](https://scholar.google.com/scholar?q=American+Psychological+Association%2E+1983%2E+Publications+Manual%2E+American+Psychological+Association%2C+Washington%2C+DC%2E)
6. Rie Kubota Ando and Tong Zhang. 2005. A framework for learning predictive structures from multiple tasks and unlabeled data. Journal of Machine Learning Research, 6:1817–1853. [↗](https://doi.org/10.5555/1046920.1194905)
7. Galen Andrew and Jianfeng Gao. 2007. Scalable training of L1-regularized log-linear models. In Proceedings of the 24th International Conference on Machine Learning, pages 33–40. [find](https://scholar.google.com/scholar?q=Galen+Andrew+and+Jianfeng+Gao%2E+2007%2E+Scalable+training+of+L1%2Dregularized+log%2Dlinear+models%2E+In+Proceedings+of+the+24th+International+Conference+on+Machine+Learning%2C+pages+33%E2%80%9340%2E)
8. Wissam Antoun, Fady Baly, and Hazem Hajj. 2020. [AraBERT: Transformer-based model for Arabic language understanding](https://md.research.elamur.ai/free/en/papers/antoun-2020-arabert-transformer-based-model-arabic/). In Proceedings of the 4th Workshop on Open-Source Arabic Corpora and Processing Tools, with a Shared Task on Offensive Language Detection, pages 9– 15, Marseille, France. European Language Resource Association. [↗](https://doi.org/10.48550/arxiv.2003.00104)
9. Wissam Antoun, Fady Baly, and Hazem Hajj. 2021. [AraELECTRA: Pre-training text discriminators for Arabic language understanding](https://md.research.elamur.ai/free/en/papers/antoun-2021-araelectra-pre-training-text-discriminators/). In Proceedings of the Sixth Arabic Natural Language Processing Workshop, pages 191–195, Kyiv, Ukraine (Virtual). Association for Computational Linguistics. [↗](https://doi.org/10.48550/arxiv.2012.15516)
10. Wenzhi Cao, Vahid Mirjalili, and Sebastian Raschka. 2019. CORAL: Rank-consistent ordinal regression for neural networks. arXiv preprint arXiv:1901.07884. [↗](https://arxiv.org/abs/1901.07884)
11. Ashok K. Chandra, Dexter C. Kozen, and Larry J. Stockmeyer. 1981. Alternation. Journal of the Association for Computing Machinery, 28(1):114–133. [find](https://scholar.google.com/scholar?q=Ashok+K%2E+Chandra%2C+Dexter+C%2E+Kozen%2C+and+Larry+J%2E+Stockmeyer%2E+1981%2E+Alternation%2E+Journal+of+the+Association+for+Computing+Machinery%2C+28%281%29%3A114%E2%80%93133%2E)
12. Mo El-Haj, Sultan Almujaiwel, Damith Premasiri, Tharindu Ranasinghe, and Ruslan Mitkov. 2024. [DARES: Dataset for Arabic readability estimation of school materials](https://md.research.elamur.ai/free/en/papers/haj-2024-dares-dataset-arabic-readability-estimation/). In Proceedings of the Workshop on DeTermIt! Evaluating Text Difficulty in a Multilingual Context @ LREC-COLING 2024, pages 103–113, Torino, Italia. ELRA and ICCL.
13. Khalid N. Elmadani, Bashar Alhafni, Hanada Taha, and Nizar Habash. 2025a. [BAREC shared task 2025 on Arabic readability assessment](https://md.research.elamur.ai/free/en/papers/elmadani-2025-barec-shared-task-2025-arabic/). In Proceedings of the Third Arabic Natural Language Processing Conference, Suzhou, China. Association for Computational Linguistics.
14. Khalid N. Elmadani, Nizar Habash, and Hanada Taha-Thomure. 2025b. [A large and balanced corpus for fine-grained Arabic readability assessment](https://md.research.elamur.ai/free/en/papers/elmadani-2025-large-balanced-corpus-fine-grained/). In Findings of the Association for Computational Linguistics: ACL 2025, pages 16376–16400, Vienna, Austria. Association for Computational Linguistics.
15. In Proceedings of the 19th Linguistic Annotation Workshop (LAW-XIX-2025), pages 359– 376, Vienna, Austria. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=In+Proceedings+of+the+19th+Linguistic+Annotation+Workshop+%28LAW%2DXIX%2D2025%29%2C+pages+359%E2%80%93+376%2C+Vienna%2C+Austria%2E+Association+for+Computational+Linguistics%2E)
16. Go Inoue, Bashar Alhafni, Nurpeiis Baimukan, Houda Bouamor, and Nizar Habash. 2021. [The interplay of variant, size, and task type in Arabic pre-trained language models](https://md.research.elamur.ai/free/en/papers/inoue-2021-interplay-variant-size-task-type/). In Proceedings of the Sixth Arabic Natural Language Processing Workshop, pages 92–104, Kyiv, Ukraine (Virtual). Association for Computational Linguistics.
17. Abdullah Khered, Ingy Abdelhalim Abdelhalim, and Riza Batista-Navarro. 2022. [Building an ensemble of transformer models for Arabic dialect classification and sentiment analysis](https://md.research.elamur.ai/free/en/papers/khered-2022-building-ensemble-transformer-models-arabic/). In Proceedings of the Seventh Arabic Natural Language Processing Workshop (WANLP), pages 479–484, Abu Dhabi, United Arab Emirates (Hybrid). Association for Computational Linguistics.
18. Juan Liberato, Bashar Alhafni, Muhamed Khalil, and Nizar Habash. 2024. [Strategies for Arabic readability modeling](https://md.research.elamur.ai/free/en/papers/liberato-2024-strategies-arabic-readability-modeling/). In Proceedings of the Second Arabic Natural Language Processing Conference, pages 55–66, Bangkok, Thailand. Association for Computational Linguistics.
19. Ossama Obeid, Nasser Zalmout, Salam Khalifa, Dima Taji, Mai Oudah, Bashar Alhafni, Go Inoue, Fadhl Eryani, Alexander Erdmann, and Nizar Habash. 2020. [CAMeL tools: An open source Python toolkit for Arabic natural language processing](https://md.research.elamur.ai/free/en/papers/obeid-2020-camel-tools-open-source-python/). In Proceedings of the Twelfth Language Resources and Evaluation Conference, pages 7022–7032, Marseille, France. European Language Resources Association.
20. Mohammad Sadegh Rasooli and Joel R. Tetreault. 2015. Yara parser: A fast and accurate dependency parser. Computing Research Repository, arXiv:1503.06733, version 2. [↗](https://arxiv.org/abs/1503.06733)
21. Dan Gusfield. 1997. Algorithms on Strings, Trees and Sequences. Cambridge University Press, Cambridge, UK. [↗](https://doi.org/10.1017/cbo9780511574931)
22. Nizar Habash, Hanada Taha-Thomure, Khalid N. Elmadani, Zeina Zeino, and Abdallah Abushmaes. 2025. [Guidelines for fine-grained sentence-level Arabic readability annotation](https://md.research.elamur.ai/free/en/papers/habash-2025-guidelines-fine-grained-sentence-level/).

*Updated 1 November 2026*