---
title: "Efficient Measuring of Readability to Improve — Bessou 2021"
description: "The paper tackles automatic difficulty grading for Arabic texts so that learners can be given material that fits their level."
authors: ["Sadik Bessou", "Ghozlane Chenni"]
year: 2021
venue: "arXiv:2109.08648"
source: "https://arxiv.org/abs/2109.08648"
doi: "10.48550/arxiv.2109.08648"
arxiv: "2109.08648"
licence: "CC BY 4.0 (arXiv)"
licence_url: "http://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/bessou-2021-efficient-measuring-readability-improve-documents/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/bessou-2021-efficient-measuring-readability-improve-documents/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2021](https://md.research.elamur.ai/free/en/papers/2021/page/4/) / Efficient Measuring of Readability to Improve Documents Accessibility for Arabic Language Learners

# Efficient Measuring of Readability to Improve Documents Accessibility for Arabic Language Learners

**Authors:** [Sadik Bessou](https://md.research.elamur.ai/free/en/authors/sadik-bessou/), Ghozlane Chenni · **Year:** 2021 · **Venue:** arXiv:2109.08648 · **Licence:** [CC BY 4.0 (arXiv)](http://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://arxiv.org/abs/2109.08648) · [PDF](https://arxiv.org/pdf/2109.08648) · [DOI 10.48550/arxiv.2109.08648](https://doi.org/10.48550/arxiv.2109.08648) · [arXiv 2109.08648](https://arxiv.org/abs/2109.08648) · [OpenAlex](https://openalex.org/W4400420757)

**Citations:** 8 citations (OpenAlex, 2026-10-14) · 2 in our library, of the 140884 papers whose bibliographies we read

## Summary
The paper tackles automatic difficulty grading for Arabic texts so that learners can be given material that fits their level. The authors argue that Arabic has received far less attention here than English. They built a corpus of 39,792 web-collected Arabic documents, each labelled with one of four levels. Labels follow the intended audience of each source: children's books and primary textbooks at the bottom, pre-Islamic classical writing at the top. Texts were cleaned, normalized for Hamza/Alif spelling variants, stripped of stop words and light-stemmed. The authors then compared Count and TF-IDF vectors built from unigrams, bigrams or both, feeding each into five standard classifiers. The strongest setup was an SVM on TF-IDF with combined unigrams and bigrams, at 87.14% accuracy over four classes. Multinomial Naïve Bayes on Count vectors came close. The work matters because it provides a sizeable labelled Arabic readability resource. It also shows that the best classifier depends on how features are represented.

## Key points
- The corpus has 39,792 documents and 475,454 tokens. They split into Easy (6,691), Medium (8,844), Difficult (9,326) and Very difficult (14,931).
- The "Very difficult" class consists of pre-Islamic texts in a variety that differs from MSA. The top level therefore partly reflects a change of language variety, not only complexity.
- Five classifiers were tested: Multinomial NB, Bernoulli NB, Logistic Regression, SVM and Random Forest. Each was run across 2 representations × 3 n-gram settings, giving six experiments in total.
- The best result was SVM with TF-IDF on unigrams plus bigrams, at 87.14% accuracy and F1 0.87. With Count vectors, Multinomial NB led at 86.47%, while Bernoulli NB was consistently weakest (as low as 44.74%).
- Bigrams on their own performed much worse (best 76.74%). Adding them to unigrams helped every classifier slightly, and SVM did better with TF-IDF while Multinomial NB did better with raw counts.

## Interesting ideas
- Difficulty labels come cheaply from where a text was published (primary textbook, student magazine, university material), so no per-document human rating is needed; this provenance-as-label trick scales annotation for graded corpora. — For Mova (answer): school-course test passages could be tagged by the grade of their source book, letting us report reader accuracy per difficulty band instead of one flat score.
- A full grid of representation × n-gram setting × classifier showed that rankings flip between setups, e.g. NB wins on counts while SVM wins on TF-IDF, which guards against declaring a model best from a single configuration. — For Mova (learn): evaluations of parser or frame-extractor variants should cross feature/encoding choices with model types before crediting any one component.
- Sparse bigram features fail on their own yet add a little on top of unigrams, a reminder that higher-order features work as supplements rather than replacements. — For Mova (tokens): tagger feature templates could add word-pair context only alongside single-word features, with an ablation that keeps the bigram-only setting as a check.
- A shallow n-gram classifier already reaches high accuracy on level prediction, which makes it a natural baseline that deeper analyses must beat. — For Mova (learn): use such a bag-of-n-grams grader as a negative control, so that any difficulty signal derived from UD trees or frames has to show gains beyond surface lexis.
- Spelling variants (Hamza/Alif forms) are unified before features are counted, so that one word does not split into several sparse entries. — For Mova (tokens): map orthographic variants such as British/American spellings or quote and dash forms to one key before lemma lookup, while keeping the original form for explainability.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

Fields: [Linguistics & languages](https://md.research.elamur.ai/free/en/shelves/linguistics/)

Topics: [Semitic languages](https://md.research.elamur.ai/free/en/topics/semitic-languages/) · [Grammatical error correction & learner language](https://md.research.elamur.ai/free/en/topics/grammatical-error-correction/)

## Cite

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@misc{bessou2021efficient,
  title = {{Efficient Measuring of Readability to Improve Documents Accessibility for Arabic Language Learners}},
  author = {Bessou, Sadik and Chenni, Ghozlane},
  year = {2021},
  doi = {10.48550/arxiv.2109.08648},
  eprint = {2109.08648},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2109.08648}
}
```

- [← previous Anaphora Resolution in Dialogue: Description of the DFKI-TalkingRobots System for the CODI-CRAC 2021 Shared-Task](https://md.research.elamur.ai/free/en/papers/anikina-2021-anaphora-resolution-dialogue-description-dfki/)
- [next → The IDC System for Sentiment Classification and Sarcasm Detection in Arabic](https://md.research.elamur.ai/free/en/papers/israeli-2021-idc-system-sentiment-classification-sarcasm/)
- [↑ back to papers of 2021](https://md.research.elamur.ai/free/en/papers/2021/page/4/)

## References

20 references, 1 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Zambrano, J. O., Tapia, E. V. (2018). Reading comprehension in university texts: the metrics of lexical complexity in corpus analysis in Spanish. In: International Conference on Computer and Communication Engineering (p. 111-123). Springer, Cham, (October). [find](https://scholar.google.com/scholar?q=Zambrano%2C+J%2E+O%2E%2C+Tapia%2C+E%2E+V%2E+%282018%29%2E+Reading+comprehension+in+university+texts%3A+the+metrics+of+lexical+complexity+in+corpus+analysis+in+Spanish%2E+In%3A+International+Conference+on+Computer+and+Communica)
2. Zhang, Lixiao, Liu, Zaiying., Ni, Jun. (2013). Feature-based assessment of text readability. In: 2013 Seventh International Conference on Internet Computing for Engineering and Science, p. 51-54. IEEE. [find](https://scholar.google.com/scholar?q=Zhang%2C+Lixiao%2C+Liu%2C+Zaiying%2E%2C+Ni%2C+Jun%2E+%282013%29%2E+Feature%2Dbased+assessment+of+text+readability%2E+In%3A+2013+Seventh+International+Conference+on+Internet+Computing+for+Engineering+and+Science%2C+p%2E+51%2D54%2E+IEEE)
3. Silpa, K. S., Irshad, M. (2018). Lexical Simplification of Complex Scientific Terms. In: 2018 International Conference on Emerging Trends and Innovations In Engineering and Technological Research (ICETIETR), p. 1-5. IEEE. [find](https://scholar.google.com/scholar?q=Silpa%2C+K%2E+S%2E%2C+Irshad%2C+M%2E+%282018%29%2E+Lexical+Simplification+of+Complex+Scientific+Terms%2E+In%3A+2018+International+Conference+on+Emerging+Trends+and+Innovations+In+Engineering+and+Technological+Research+%28ICE)
4. Wibowo, Satrio, Muhammad., Romadhony, Ade., Siti Sa’adah. (2019). Lexical and Syntactic Simplification for Indonesian Text. In: 2019 International Seminar on Research of Information Technology and Intelligent Systems (ISRITI), p. 64-68. IEEE. [find](https://scholar.google.com/scholar?q=Wibowo%2C+Satrio%2C+Muhammad%2E%2C+Romadhony%2C+Ade%2E%2C+Siti+Sa%E2%80%99adah%2E+%282019%29%2E+Lexical+and+Syntactic+Simplification+for+Indonesian+Text%2E+In%3A+2019+International+Seminar+on+Research+of+Information+Technology+and+Int)
5. Nandhini, Kumaresh., Ramakrishnan Balasundaram, Sadhu. (2012). Significance of learner dependent features for improving text readability using extractive summarization. In: 2012 4th International Conference on Intelligent Human Computer Interaction (IHCI), p. 1-5. IEEE. [find](https://scholar.google.com/scholar?q=Nandhini%2C+Kumaresh%2E%2C+Ramakrishnan+Balasundaram%2C+Sadhu%2E+%282012%29%2E+Significance+of+learner+dependent+features+for+improving+text+readability+using+extractive+summarization%2E+In%3A+2012+4th+International+Conf)
6. Daowadung, Patcharanut., Yaw-Huei Chen. (2012). Stop word in readability assessment of Thai text. In: 2012 IEEE 12 th International Conference on Advanced Learning Technologies, p. 497-499. IEEE. [find](https://scholar.google.com/scholar?q=Daowadung%2C+Patcharanut%2E%2C+Yaw%2DHuei+Chen%2E+%282012%29%2E+Stop+word+in+readability+assessment+of+Thai+text%2E+In%3A+2012+IEEE+12+th+International+Conference+on+Advanced+Learning+Technologies%2C+p%2E+497%2D499%2E+IEEE%2E)
7. Luong, An-Vinh, Nguyen, Diep., DienDinh. (2017). Examining the text-length factor in evaluating the readability of literary texts in Vietnamese textbooks. In: 2017 9th International Conference on Knowledge and Systems Engineering (KSE), p. 36-41. IEEE. [find](https://scholar.google.com/scholar?q=Luong%2C+An%2DVinh%2C+Nguyen%2C+Diep%2E%2C+DienDinh%2E+%282017%29%2E+Examining+the+text%2Dlength+factor+in+evaluating+the+readability+of+literary+texts+in+Vietnamese+textbooks%2E+In%3A+2017+9th+International+Conference+on+Know)
8. Luong, An-Vinh, Nguyen, Diep., DienDinh. (2018). Assessing the Readability of Literary Texts in Vietnamese Textbooks. In: 2018 5 th NAFOSTED Conference on Information and Computer Science (NICS), p. 231-236. IEEE. [find](https://scholar.google.com/scholar?q=Luong%2C+An%2DVinh%2C+Nguyen%2C+Diep%2E%2C+DienDinh%2E+%282018%29%2E+Assessing+the+Readability+of+Literary+Texts+in+Vietnamese+Textbooks%2E+In%3A+2018+5+th+NAFOSTED+Conference+on+Information+and+Computer+Science+%28NICS%29%2C+p%2E+2)
9. Mukherjee, Partha, Leroy, Gondy., Kauchak, David. (2018). Using Lexical Chains to Identify Text Difficulty: A Corpus Statistics and Classification Study. IEEE journal of biomedical and health informatics, 23 (5) 2164-2173. [find](https://scholar.google.com/scholar?q=Mukherjee%2C+Partha%2C+Leroy%2C+Gondy%2E%2C+Kauchak%2C+David%2E+%282018%29%2E+Using+Lexical+Chains+to+Identify+Text+Difficulty%3A+A+Corpus+Statistics+and+Classification+Study%2E+IEEE+journal+of+biomedical+and+health+informat)
10. Naderi, Babak, SalarMohtaj, Karan, Karan., Möller, Sebastian. (2019). Automated Text Readability Assessment for German Language: A Quality of Experience Approach. In: 2019 Eleventh International Conference on Quality of Multimedia Experience (QoMEX), p. 1-3. IEEE. [find](https://scholar.google.com/scholar?q=Naderi%2C+Babak%2C+SalarMohtaj%2C+Karan%2C+Karan%2E%2C+M%C3%B6ller%2C+Sebastian%2E+%282019%29%2E+Automated+Text+Readability+Assessment+for+German+Language%3A+A+Quality+of+Experience+Approach%2E+In%3A+2019+Eleventh+International+Confe)
11. Narasinh, Vishwaas. (2019). Readability Analysis of Kannada Language. In: 2019 1 st International Conference on Advances in Information Technology (ICAIT), p. 45-49. IEEE. [find](https://scholar.google.com/scholar?q=Narasinh%2C+Vishwaas%2E+%282019%29%2E+Readability+Analysis+of+Kannada+Language%2E+In%3A+2019+1+st+International+Conference+on+Advances+in+Information+Technology+%28ICAIT%29%2C+p%2E+45%2D49%2E+IEEE%2E)
12. Zhang, Yusha, Lin, Nankai., Jiang, Shengyi. (2019). A Study on Syntactic Complexity and Text Readability of ASEAN English News. In: 2019 International Conference on Asian Language Processing (IALP), p. 313-318. IEEE. [find](https://scholar.google.com/scholar?q=Zhang%2C+Yusha%2C+Lin%2C+Nankai%2E%2C+Jiang%2C+Shengyi%2E+%282019%29%2E+A+Study+on+Syntactic+Complexity+and+Text+Readability+of+ASEAN+English+News%2E+In%3A+2019+International+Conference+on+Asian+Language+Processing+%28IALP%29%2C+p)
13. Al-Ajlan, Amani, A., Hend, S., Al-Khalifa, Abdul Malik S., Al-Salman. (2008). Towards the development of an automatic readability measurements for Arabic language. In: 2008 Third International Conference on Digital Information Management, p. 506-511. IEEE. [find](https://scholar.google.com/scholar?q=Al%2DAjlan%2C+Amani%2C+A%2E%2C+Hend%2C+S%2E%2C+Al%2DKhalifa%2C+Abdul+Malik+S%2E%2C+Al%2DSalman%2E+%282008%29%2E+Towards+the+development+of+an+automatic+readability+measurements+for+Arabic+language%2E+In%3A+2008+Third+International+Confere)
14. Saddiki, Hind, Bouzoubaa, Karim., Cavalli-Sforza, Violetta. (2015). Text readability for Arabic as a foreign language. In: 2015 IEEE/ACS 12 th International Conference of Computer Systems and Applications (AICCSA), p. 1-8. IEEE. [find](https://scholar.google.com/scholar?q=Saddiki%2C+Hind%2C+Bouzoubaa%2C+Karim%2E%2C+Cavalli%2DSforza%2C+Violetta%2E+%282015%29%2E+Text+readability+for+Arabic+as+a+foreign+language%2E+In%3A+2015+IEEE%2FACS+12+th+International+Conference+of+Computer+Systems+and+Applicat)
15. Wilkens, R., DallaVecchia, A., Boito, M. Z., Padró, M., Villavicencio, A. (2014). Size does not matter. Frequency does. A study of features for measuring lexical complexity. In: Ibero-American Conference on Artificial Intelligence (p. 129-140). Springer, Cham, (November). [find](https://scholar.google.com/scholar?q=Wilkens%2C+R%2E%2C+DallaVecchia%2C+A%2E%2C+Boito%2C+M%2E+Z%2E%2C+Padr%C3%B3%2C+M%2E%2C+Villavicencio%2C+A%2E+%282014%29%2E+Size+does+not+matter%2E+Frequency+does%2E+A+study+of+features+for+measuring+lexical+complexity%2E+In%3A+Ibero%2DAmerican+Confere)
16. Wang, Huiping., Yang, Lijiao., Xiao, Huimin. (2019). Construction of Quantitative Index System of Vocabulary Difficulty in Chinese Grade Reading. In: 2019 International Conference on Asian Language Processing (IALP), p. 480-486. IEEE. [find](https://scholar.google.com/scholar?q=Wang%2C+Huiping%2E%2C+Yang%2C+Lijiao%2E%2C+Xiao%2C+Huimin%2E+%282019%29%2E+Construction+of+Quantitative+Index+System+of+Vocabulary+Difficulty+in+Chinese+Grade+Reading%2E+In%3A+2019+International+Conference+on+Asian+Language+Pr)
17. Khoja, Shereen., Garside, Roger. (1999). Stemming arabic text. Lancaster, UK, Computing Department, Lancaster University. [find](https://scholar.google.com/scholar?q=Khoja%2C+Shereen%2E%2C+Garside%2C+Roger%2E+%281999%29%2E+Stemming+arabic+text%2E+Lancaster%2C+UK%2C+Computing+Department%2C+Lancaster+University%2E)
18. Larkey, Leah, S., Margaret, E., Connell. (2001). Arabic information retrieval at UMass in TREC-10. In TREC. 2001. [find](https://scholar.google.com/scholar?q=Larkey%2C+Leah%2C+S%2E%2C+Margaret%2C+E%2E%2C+Connell%2E+%282001%29%2E+Arabic+information+retrieval+at+UMass+in+TREC%2D10%2E+In+TREC%2E+2001%2E)
19. Bessou, Sadik., Touahria, Mohamed. (2014). An accuracy-enhanced stemming algorithm for Arabic information retrieval. Neural Network World, 24 (2) 117. [find](https://scholar.google.com/scholar?q=Bessou%2C+Sadik%2E%2C+Touahria%2C+Mohamed%2E+%282014%29%2E+An+accuracy%2Denhanced+stemming+algorithm+for+Arabic+information+retrieval%2E+Neural+Network+World%2C+24+%282%29+117%2E) [our card](https://md.research.elamur.ai/free/en/papers/bessou-2019-accuracy-enhanced-stemming-algorithm-arabic/)
20. Schütze, Hinrich., Christopher, D., Manning, Raghavan, Prabhakar. (2008). Introduction to information retrieval. Vol. 39. Cambridge: Cambridge University Press. [find](https://scholar.google.com/scholar?q=Sch%C3%BCtze%2C+Hinrich%2E%2C+Christopher%2C+D%2E%2C+Manning%2C+Raghavan%2C+Prabhakar%2E+%282008%29%2E+Introduction+to+information+retrieval%2E+Vol%2E+39%2E+Cambridge%3A+Cambridge+University+Press%2E)