--- title: "A Large and Balanced Corpus for Fine-grained Arabic Readability Assessment — Elmadani 2025" description: "Arabic readability research has lacked a large, carefully balanced, finely graded dataset. 5 references, cited by 26 papers here." authors: ["Khalid N. Elmadani", "Nizar Habash", "Hanada Taha-Thomure"] year: 2025 venue: "Findings of the Association for Computational Linguistics: ACL 2025" source: "https://aclanthology.org/2025.findings-acl.842/" doi: "10.18653/v1/2025.findings-acl.842" licence: "CC BY 4.0 (ACL Anthology)" licence_url: "https://creativecommons.org/licenses/by/4.0/" summary_licence: "Elamur Cosmic License" canonical: "https://research.elamur.ai/free/en/papers/elmadani-2025-large-balanced-corpus-fine-grained/" --- [Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/elmadani-2025-large-balanced-corpus-fine-grained/) [home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2025](https://md.research.elamur.ai/free/en/papers/2025/) / A Large and Balanced Corpus for Fine-grained Arabic Readability Assessment # A Large and Balanced Corpus for Fine-grained Arabic Readability Assessment **Authors:** [Khalid N. Elmadani](https://md.research.elamur.ai/free/en/authors/khalid-n-elmadani/), [Nizar Habash](https://md.research.elamur.ai/free/en/authors/nizar-habash/) and [Hanada Taha-Thomure](https://md.research.elamur.ai/free/en/authors/hanada-taha-thomure/) · **Year:** 2025 · **Venue:** [Findings of the Association for Computational Linguistics: ACL 2025](https://md.research.elamur.ai/free/en/venues/findings/) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/) **Links:** [Publisher's page](https://aclanthology.org/2025.findings-acl.842/) · [PDF](https://aclanthology.org/2025.findings-acl.842.pdf) · [DOI 10.18653/v1/2025.findings-acl.842](https://doi.org/10.18653/v1/2025.findings-acl.842) · [OpenAlex](https://openalex.org/W4412888009) **Citations:** 2 citations (OpenAlex, 2026-10-14) · [26](https://md.research.elamur.ai/free/en/papers/elmadani-2025-large-balanced-corpus-fine-grained/cited-by/) in our library, of the 140,884 papers whose bibliographies we read ## Summary Arabic readability research has lacked a large, carefully balanced, finely graded dataset. The authors address this with BAREC (Balanced Arabic Readability Evaluation Corpus). It contains 69,441 sentence-level segments, totalling over a million words, drawn from many genres, topics and audiences. Each segment was manually labelled on a 19-level scale that runs from kindergarten to postgraduate reading. The scale adapts the Taha/Arabi21 leveling system. A team of experienced Arabic educators did the labelling, and their pairwise agreement reached a Quadratic Weighted Kappa of 81.8%. The 19 levels nest cleanly into coarser 7-, 5- and 3-level versions. The authors benchmark fine-tuned Arabic BERT models with several input representations and loss functions. The best single classifier is AraBERTv2 on D3Tok input, with roughly 56% exact 19-level accuracy and about 80% QWK. A regression variant pushes QWK to about 84%. An oracle combination of systems shows large headroom for ensembling. The resource, guidelines and benchmarks are released openly to support research, education and tools that help authors write for specific reading levels. ## Key points - BAREC has 69,441 segments and 1M+ words. A "sentence" here can also be a phrase or a single word, such as a book title. - Annotators judge each segment on six dimensions: spelling, word count, morphology, syntax, vocabulary and content. The assigned level is set by the single hardest phenomenon found in the segment. - Six native-speaker educators formed the team. One led segmentation and text selection, and five assigned levels. Average pairwise QWK was 81.8%. - The authors compared four models (CamelBERT-msa, MARBERTv2, AraBERTv02, AraBERTv2) and four inputs (Word, Lex, D3Tok, D3Lex). - On the test set, AraBERTv2+D3Tok with cross-entropy gets 55.9% Acc19 and 80.2% QWK. - Regression gets 84.4% QWK but only 41.4% Acc19. - An oracle combination reaches 75.9% Acc19 and 94.7% QWK. - Error analysis used dev sentences with five annotations. In 87% of cases, the model's prediction could be compared with at least one annotator's label. Of the remaining 13%, the model mostly guessed too easy a level (9%) rather than too hard (3%). ## Interesting ideas - The level of a sentence is set by its single most difficult feature, not by an average, which yields a clear rule annotators can defend. - The fine scale nests exactly into coarser scales (19→7→5→3), so one annotation pass serves many uses and results can be reported at every granularity. - Comparing many loss functions shows a trade-off: ordinal or regression objectives improve distance-aware metrics like QWK but lose exact-match accuracy. Reporting both kinds of metric exposes this. - Error analysis judges the model against the spread of several human labels (inside, above or below the range) rather than against one gold label. - Varying the input representation (raw words, lemmas, clitic-segmented tokens) on a fixed model shows that segmentation choices alone shift results noticeably. ## For Mova How these ideas could serve Mova, our small open language model — its files, stages and tests: - **I1** (answer): Mova could grade fables and school texts this way, flagging the exact frame or construction that makes a passage hard, which keeps the difficulty score explainable. - **I2** (learn): Mova's evaluations, such as role labels or error categories, could use nested label sets so that coarse and fine scores come from the same gold data. - **I3** (learn): when training gparse or the taggers, Mova could report both exact scores and "near-miss" distance scores, so that a gain on one metric does not hide a loss on the other. - **I4** (learn): Mova's silver data from several witness parsers could be scored the same way, treating outputs outside the witnesses' range as the priority errors for repair rules. - **I5** (tokens): Mova could run a similar ablation on how the en crate splits contractions and possessive clitics, measuring the downstream effect on parsing before committing to a scheme. *Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.* ## Institutions - [New York University Abu Dhabi](https://md.research.elamur.ai/free/en/universities/new-york-university-abu-dhabi/) — University, Abu Dhabi, United Arab Emirates · 45 works · 2,055 citations 💡 Concepts (1): [📚 Corpus linguistics](https://md.research.elamur.ai/free/en/concepts/corpus-linguistics/) · 23 works 📚 Shelves (6): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 7,249 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 7,084 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 7,398 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 7,340 works · [💭 Psychology](https://md.research.elamur.ai/free/en/shelves/psychology/) · 569 works · [💭 Developmental and Educational Psychology](https://md.research.elamur.ai/free/en/shelves/psychology/developmental-and-educational-psychology/) · 189 works 🧭 Topics (1): [Semitic languages](https://md.research.elamur.ai/free/en/topics/semitic-languages/) · 1,336 works ## Cite Cite as: Khalid N. Elmadani, Nizar Habash and Hanada Taha-Thomure. 2025. A Large and Balanced Corpus for Fine-grained Arabic Readability Assessment. Findings of the Association for Computational Linguistics: ACL 2025. The paper itself, not this page. BibTeX from the library's record of the work. ```bibtex @inproceedings{elmadani2025large, title = {{A Large and Balanced Corpus for Fine-grained Arabic Readability Assessment}}, author = {Khalid N. Elmadani, Nizar Habash and Hanada Taha-Thomure}, booktitle = {Findings of the Association for Computational Linguistics: ACL 2025}, year = {2025}, doi = {10.18653/v1/2025.findings-acl.842}, url = {https://doi.org/10.18653/v1/2025.findings-acl.842} } ``` - [← previous SemEval-2025 Task 1: AdMIRe - Advancing Multimodal Idiomaticity Representation](https://md.research.elamur.ai/free/en/papers/pickard-2025-semeval-2025-task-1-admire/) - [next → Machine Translation Meta Evaluation through Translation Accuracy Challenge Sets](https://md.research.elamur.ai/free/en/papers/moghe-2025-machine-translation-meta-evaluation-through/) - [↑ back to papers of 2025](https://md.research.elamur.ai/free/en/papers/2025/) ## References 5 references, 0 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet. 1. Wikipedia A subset of 168 Arabic wikipedia articles covering Culture, Figures, Geography, History, Mathematics, Sciences, Society, Philosophy, Religions and Technologies. 14 Kalima The first 500 words of 62 books from Kalima project. 7 [https://chatgpt.com/](https://chatgpt.com/) https://www.hindawi.org/books/categories/ 7 https://alc.ae/publications/kalima/https://archive.org/details/201409_201409 http://al-nada.eb2a.com/1000lela&lela/ 10 https://www.hindawi.org/books/90463596/ 11 https://www.hindawi.org/books/72707304/  Ï@ 12  ®ÊªÖ https://ar.wikipedia.org/wiki/ HA https://archive.org/details/majid_magazine https://ar.wikipedia.org/Prompt Targeted #Words per Sentence Prompt Text % Discarded 2-4 I am creating a children's textbook to practice reading in Arabic. I need short sentences containing 2 to 4 words that are limited to children's vocabulary. Give me 200 sentences in Standard Arabic -- no need to include English. 1.5% Prompt 1 .ةقرشم سمشلا Examples 5-7 Prompt 2 .ةهكافلا لكأت تنبلا I am creating a children's textbook to practice reading in Arabic. I need 5-word, 6-word, and 7-word sentences that are limited to children's vocabulary. Give me 150 sentences in Standard Arabic -- no need to include English. .ةريبك ةرجش تحت ماني دسلأا Examples 8-10 Prompt 3 1.3% .ةريبك ةداعسب نوكحضيو بعلملا يف نوبعلي لافطلأا I am creating a children's textbook to practice reading in Arabic. I need long sentences (8-word, 9-word, and 10-word sentences) that are limited to children's vocabulary. Give me 100 sentences in Standard Arabic -- no need to include English. 1.0% .ركابلا حابصلا يف رضخلأا بشعلا قوف زفقي بنرلأا Examples .عرف ىلإ عرف نم ةعاربب زفقيو ةعرسب راجشلأا قلستي درقلا *Table 9: ChatGPT Prompts. % Discarded is the percentage of discarded sentences due to grammatical errors.* Constitutions The first 2000 words of the Arabic constitutions from 16 Arabic speaking countries, collected from MCWC dataset (El-Haj and Ezzini, 2024). [↗](https://chatgpt.com/) 2. UN The Arabic translation of the Universal Declaration of Human Rights. 15 C.1.5 C.2 Arts & Humanities The Arts and Humanities domain comprised the following subdomains. • Literature and Fiction: Encompasses novels, short stories, poetry, and other creative writing forms that emphasize narrative and artistic expression. Religion • Religion and Philosophy: Contains religious texts, philosophical works, and related writings that explore spiritual beliefs, ethics, and metaphysical ideas. Old Testament The first 20 chapters of the Book of Genesis (Smith and Van Dyck, 1865). 16 [find](https://scholar.google.com/scholar?q=UN+The+Arabic+translation+of+the+Universal+Declaration+of+Human+Rights%2E+15+C%2E1%2E5+C%2E2+Arts+%26+Humanities+The+Arts+and+Humanities+domain+comprised+the+following+subdomains%2E+%E2%80%A2+Literature+and+Fiction%3A+Enco) 3. New Testament The first 16 chapters of the Book of Matthew (Smith and Van Dyck, 1860). 16 [find](https://scholar.google.com/scholar?q=New+Testament+The+first+16+chapters+of+the+Book+of+Matthew+%28Smith+and+Van+Dyck%2C+1860%29%2E+16) 4. Education and Academic Texts (on Arts and Humanities): Includes textbooks, scholarly articles, and educational materials that are often structured for learning and academic purposes. Quran The first three Surahs and the last 14 Surahs from the Holy Quran. We selected the text from the Quran Corpus Project (Dukes et al., 2013). 17 [find](https://scholar.google.com/scholar?q=Education+and+Academic+Texts+%28on+Arts+and+Humanities%29%3A+Includes+textbooks%2C+scholarly+articles%2C+and+educational+materials+that+are+often+structured+for+learning+and+academic+purposes%2E+Quran+The+first+t) 5. Hadith The first 75 Hadiths from Sahih Bukhari (al Bukhari, 846). We selected the text from the LK Hadith Corpus 18 (Altammami et al., 2019). Some datasets are chosen because they already have annotations available for other tasks. For example, dependency treebank annotations exist for Odes, Quran, Hadith, 1001, Hayy, OT, NT, Sara,WikiNews, ALC, BTEC, and ZAEBUC (Habash et al., 2022a). [https://www.un.org/ar/about-us/](https://www.un.org/ar/about-us/) universal-declaration-of-human-rights 16 https://www.arabicbible.com/ 17 https://corpus.quran.com/ 18 https://github.com/ShathaTm/LK-Hadith-Corpus [↗](https://www.un.org/ar/about-us/) *Updated 23 October 2026*