---
title: "Readability Assessment for Text — Aluisio 2010"
description: "The paper tackles automatic readability assessment for Portuguese."
authors: ["Sandra Aluisio", "Lucia Specia", "Caroline Gasperin", "Carolina Scarton"]
year: 2010
venue: "Proceedings of the NAACL HLT 2010 Fifth Workshop on Innovative Use of NLP for Building Educational Applications"
source: "https://aclanthology.org/W10-1001/"
licence: "CC BY-NC-SA 3.0 (ACL Anthology, before 2016)"
licence_url: "https://creativecommons.org/licenses/by-nc-sa/3.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/papers/aluisio-2010-readability-assessment-text-simplification/"
---
[Elamur Research](https://md.research.elamur.ai/) · [this page in HTML](https://research.elamur.ai/free/papers/aluisio-2010-readability-assessment-text-simplification/)

[home](https://md.research.elamur.ai/) / [papers](https://md.research.elamur.ai/free/papers/) / [2010](https://md.research.elamur.ai/free/papers/2010/) / Readability Assessment for Text Simplification

# Readability Assessment for Text Simplification

**Authors:** Sandra Aluisio, [Lucia Specia](https://md.research.elamur.ai/free/authors/lucia-specia/), [Caroline Gasperin](https://md.research.elamur.ai/free/authors/caroline-gasperin/), [Carolina Scarton](https://md.research.elamur.ai/free/authors/carolina-scarton/) · **Year:** 2010 · **Venue:** Proceedings of the NAACL HLT 2010 Fifth Workshop on Innovative Use of NLP for Building Educational Applications · **Licence:** [CC BY-NC-SA 3.0 (ACL Anthology, before 2016)](https://creativecommons.org/licenses/by-nc-sa/3.0/)

**Links:** [Publisher's page](https://aclanthology.org/W10-1001/) · [PDF](https://aclanthology.org/W10-1001.pdf) · [OpenAlex](https://openalex.org/W7097038257)

**Citations:** 0 citations (openalex-snapshot · 2026-10-14) · [68](#cited-by) in our library, of the 140673 papers whose bibliographies we read

## Summary
The paper tackles automatic readability assessment for Portuguese. Its goal is to help authors who simplify texts for adults with limited literacy. Instead of school grade levels, texts are mapped onto three literacy levels from Brazil's INAF index: advanced (original texts), basic (natural simplification) and rudimentary (strong simplification). The work belongs to the PorSimples project. The training material is a manually built corpus of original texts paired with naturally and strongly simplified versions.

Each text is described with several feature groups:
- cognitively motivated metrics from Coh-Metrix-PORT, a Portuguese adaptation of Coh-Metrix 2.0
- basic surface counts
- syntactic features tied to specific simplification operations
- n-gram language model scores
- the Flesch index

The authors compare standard classification, ordinal classification and regression using 10-fold cross-validation. Using all features together gives the best results for every model. Ordinal classification has the lowest error, and RBF-kernel regression has the highest correlation. Telling natural from strong simplification is the hardest part of the task. The simplest classifier was built into the SIMPLIFICA authoring tool. There, authors can recheck a text's level while they edit until it suits the intended readers.

## Key points
- Motivation from INAF 2009: 7% of Brazilians are illiterate, 21% have rudimentary literacy, 47% basic and only 25% advanced. Before this work, the only Portuguese readability tool was an adapted Flesch Reading Ease index.
- The features most correlated with level were words per sentence (0.693), incidence of apposition (0.688), incidence of clauses (0.614) and the Flesch index (0.580).
- Standard classification with all features reached F-measures of 0.913 (original), 0.483 (natural) and 0.732 (strong), with correlation 0.84 and MAE 0.276.
- Ordinal classification gave similar F-measure and correlation (0.83) but a lower MAE (0.163). RBF-kernel regression had the best correlation (0.8502) but a higher MAE (0.3478). A linear SVM regressor performed poorly.
- Among single feature groups, syntactic features and Coh-Metrix-PORT correlated best with the true level, while the language model features did worst.

## Interesting ideas
- **Literacy levels instead of grades.** The target scale is a population literacy framework rather than school grades. This could suit adult-oriented or accessibility applications in other languages, where grade levels describe the audience poorly.
- **Labels from simplification degree.** Training labels come from how strongly a text was simplified (original, natural, strong), not from human readability ratings. This is a cheap way to obtain graded data wherever parallel simplified corpora exist.
- **Features that mirror the simplifier's rules.** Syntactic features count the same phenomena the rule-based simplifier targets, such as appositions, relative clauses and passives. The readability score therefore reacts directly to the edits an author makes.
- **Comparing nominal, ordinal and interval framings.** Running all three on the same data and judging them by both correlation and MAE shows trade-offs that a single metric would hide. This is useful for any task with ordered labels.
- **A live readability check during editing.** Placing the classifier inside the authoring tool turns it into feedback the author can request at any point. The same design could guide human post-editing or steer automatic simplification toward a target level.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

Fields: [Machine translation](https://md.research.elamur.ai/free/shelves/machine-translation/) · [Natural language processing](https://md.research.elamur.ai/free/shelves/nlp/)

## Cited by

Cited by

68 papers of our library cite it (of the 140673 whose bibliographies we read); 3 of them have a page here, 65 not yet.

- [Modélisation de la lisibilité en français pour les personnes en situation d'illettrisme](https://md.research.elamur.ai/free/papers/aissa-2025-modelisation-de-la-lisibilite-en/)
- [READ–IT: Assessing Readability of Italian Texts with a View to Text Simplification](https://md.research.elamur.ai/free/papers/orletta-2011-read-it-assessing-readability-italian/)
- [Translation as Augmentation: Effect of Translated Data on Assessment of Difficulty](https://md.research.elamur.ai/free/papers/wu-2026-translation-as-augmentation-effect-translated/)

## Cite

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{aluisio2010readability,
  title = {{Readability Assessment for Text Simplification}},
  author = {Aluisio, Sandra and Specia, Lucia and Gasperin, Caroline and Scarton, Carolina},
  booktitle = {Proceedings of the NAACL HLT 2010 Fifth Workshop on Innovative Use of NLP for Building Educational Applications},
  year = {2010},
  url = {https://aclanthology.org/W10-1001/}
}
```

- [← previous Probabilistic Frame-Semantic Parsing](https://md.research.elamur.ai/free/papers/das-2010-probabilistic-frame-semantic-parsing/)
- [next → Realization of Discourse Relations by Other Means: Alternative Lexicalizations](https://md.research.elamur.ai/free/papers/prasad-2010-realization-discourse-relations-other-means/)
- [↑ back to papers of 2010](https://md.research.elamur.ai/free/papers/2010/)

## References

30 references, 0 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Sandra M. Aluísio, Lucia Specia, Thiago A. S. Pardo, Erick G. Maziero, Renata P. M. Fortes (2008). Towards Brazilian Portuguese Automatic Text Simplification Systems. In the Proceedings of the 8th ACM Symposium on Document Engineering, pp. 240-248. [find](https://scholar.google.com/scholar?q=Sandra+M%2E+Alu%C3%ADsio%2C+Lucia+Specia%2C+Thiago+A%2E+S%2E+Pardo%2C+Erick+G%2E+Maziero%2C+Renata+P%2E+M%2E+Fortes+%282008%29%2E+Towards+Brazilian+Portuguese+Automatic+Text+Simplification+Systems%2E+In+the+Proceedings+of+the+8th+ACM)
2. Eckhard Bick (2000). The Parsing System "Palavras": Automatic Grammatical Analysis of Portuguese in a Constraint Grammar Framework. PhD Thesis. University of Århus, Denmark. [find](https://scholar.google.com/scholar?q=Eckhard+Bick+%282000%29%2E+The+Parsing+System+%22Palavras%22%3A+Automatic+Grammatical+Analysis+of+Portuguese+in+a+Constraint+Grammar+Framework%2E+PhD+Thesis%2E+University+of+%C3%85rhus%2C+Denmark%2E)
3. Jill Burstein, Martin Chodorow and Claudia Leacock (2003). CriterionSM Online Essay Evaluation: An Application for Automated Evaluation of Student Essays. In the Proceedings of the Fifteenth Annual Conference on Innovative Applications of Artificial Intelligence, Acapulco, Mexico. [find](https://scholar.google.com/scholar?q=Jill+Burstein%2C+Martin+Chodorow+and+Claudia+Leacock+%282003%29%2E+CriterionSM+Online+Essay+Evaluation%3A+An+Application+for+Automated+Evaluation+of+Student+Essays%2E+In+the+Proceedings+of+the+Fifteenth+Annual+Co)
4. Arnaldo Candido Jr., Erick Maziero, Caroline Gasperin, Thiago A. S. Pardo, Lucia Specia, and Sandra M. Aluisio (2009). Supporting the Adaptation of Texts for Poor Literacy Readers: a Text Simplification Editor for Brazilian Portuguese. In NAACL-HLT Workshop on Innovative Use of NLP for Building Educational Applications, pages 34–42, Boulder’. [find](https://scholar.google.com/scholar?q=Arnaldo+Candido+Jr%2E%2C+Erick+Maziero%2C+Caroline+Gasperin%2C+Thiago+A%2E+S%2E+Pardo%2C+Lucia+Specia%2C+and+Sandra+M%2E+Aluisio+%282009%29%2E+Supporting+the+Adaptation+of+Texts+for+Poor+Literacy+Readers%3A+a+Text+Simplificati)
5. Helena de M. Caseli, Tiago de F. Pereira, Lúcia Specia, Thiago A. S. Pardo, Caroline Gasperin and Sandra Maria Aluísio (2009). Building a Brazilian Portuguese Parallel Corpus of Original and Simplified Texts. In the Proceedings of CICLing. [find](https://scholar.google.com/scholar?q=Helena+de+M%2E+Caseli%2C+Tiago+de+F%2E+Pereira%2C+L%C3%BAcia+Specia%2C+Thiago+A%2E+S%2E+Pardo%2C+Caroline+Gasperin+and+Sandra+Maria+Alu%C3%ADsio+%282009%29%2E+Building+a+Brazilian+Portuguese+Parallel+Corpus+of+Original+and+Simplifie)
6. Max Coltheart (1981). The MRC psycholinguistic database. In Quartely Jounal of Experimental Psychology, 33A, pages 497-505. [↗](https://doi.org/10.1080/14640748108400805)
7. Scott Deerwester, Susan T. Dumais, George W. Furnas, Thomas K. Landauer e Richard Harshman (1990). Indexing By Latent Semantic Analysis. In Journal of the American Society For Information Science, V. 41, pages 391-407. [find](https://scholar.google.com/scholar?q=Scott+Deerwester%2C+Susan+T%2E+Dumais%2C+George+W%2E+Furnas%2C+Thomas+K%2E+Landauer+e+Richard+Harshman+%281990%29%2E+Indexing+By+Latent+Semantic+Analysis%2E+In+Journal+of+the+American+Society+For+Information+Science%2C+V%2E+)
8. Bento C. Dias-da-Silva and Helio R. Moraes (2003). A construção de um thesaurus eletrônico para o português do Brasil. In ALFA- Revista de Lingüística, V. 47, N. 2, pages 101-115. [find](https://scholar.google.com/scholar?q=Bento+C%2E+Dias%2Dda%2DSilva+and+Helio+R%2E+Moraes+%282003%29%2E+A+constru%C3%A7%C3%A3o+de+um+thesaurus+eletr%C3%B4nico+para+o+portugu%C3%AAs+do+Brasil%2E+In+ALFA%2D+Revista+de+Ling%C3%BC%C3%ADstica%2C+V%2E+47%2C+N%2E+2%2C+pages+101%2D115%2E)
9. Bento C Dias-da-Silva, Ariani Di Felippo and Maria das Graças V. Nunes (2008). The automatic mapping of Princeton WordNet lexical conceptual relations onto the Brazilian Portuguese WordNet database. In Proceedings of the 6th LREC, Marrakech, Morocco. [find](https://scholar.google.com/scholar?q=Bento+C+Dias%2Dda%2DSilva%2C+Ariani+Di+Felippo+and+Maria+das+Gra%C3%A7as+V%2E+Nunes+%282008%29%2E+The+automatic+mapping+of+Princeton+WordNet+lexical+conceptual+relations+onto+the+Brazilian+Portuguese+WordNet+database%2E+I)
10. William H. DuBay (2004). The principles of readability. Costa Mesa, CA: Impact Information: [http://www.i](http://www.i) mpact-information.com/impactinfo/readability02.pdf Christiane Fellbaum (1998). WordNet: An electronic lexical database. Cambridge, MA: MIT Press. [↗](http://www.i)
11. Lijun Feng, Noémie Elhadad and Matt Huenerfauth (2009). Cognitively Motivated Features for Readability Assessment. In the Proceedings of EACL 2009, pages 229-237. [find](https://scholar.google.com/scholar?q=Lijun+Feng%2C+No%C3%A9mie+Elhadad+and+Matt+Huenerfauth+%282009%29%2E+Cognitively+Motivated+Features+for+Readability+Assessment%2E+In+the+Proceedings+of+EACL+2009%2C+pages+229%2D237%2E)
12. Ingo Glöckner, Sven Hartrumpf, Hermann Helbig, Johannes Leveling and Rainer Osswald (2006b). An architecture for rating and controlling text readability. In Proceedings of KONVENS 2006, pages 32-35. Konstanz, Germany. [find](https://scholar.google.com/scholar?q=Ingo+Gl%C3%B6ckner%2C+Sven+Hartrumpf%2C+Hermann+Helbig%2C+Johannes+Leveling+and+Rainer+Osswald+%282006b%29%2E+An+architecture+for+rating+and+controlling+text+readability%2E+In+Proceedings+of+KONVENS+2006%2C+pages+32%2D35%2E+K)
13. Arthur C. Graesser, Danielle S. McNamara, Max M. Louwerse and Zhiqiang Cai (2004). Coh-Metrix: Analysis of text on cohesion and language. In Behavioral Research Methods, Instruments, and Computers, V. 36, pages 193-202. [↗](https://doi.org/10.3758/bf03195564)
14. Ronald K. Hambleton, H. Swaminathan and H. Jane Rogers (1991). Fundamentals of item response theory. Newbury Park, CA: Sage Press. [find](https://scholar.google.com/scholar?q=Ronald+K%2E+Hambleton%2C+H%2E+Swaminathan+and+H%2E+Jane+Rogers+%281991%29%2E+Fundamentals+of+item+response+theory%2E+Newbury+Park%2C+CA%3A+Sage+Press%2E)
15. Michael Heilman, Kevyn Collins-Thompson, Jamie Callan and Max Eskenazi (2007). Combining lexical and grammatical features to improve readability measures for first and second language texts. In the Proceedings of NAACL HLT 2007, pages 460-467. [find](https://scholar.google.com/scholar?q=Michael+Heilman%2C+Kevyn+Collins%2DThompson%2C+Jamie+Callan+and+Max+Eskenazi+%282007%29%2E+Combining+lexical+and+grammatical+features+to+improve+readability+measures+for+first+and+second+language+texts%2E+In+the+Pr)
16. Michael Heilman, Kevyn Collins-Thompson and Maxine Eskenazi (2008). An Analysis of Statistical Models and Features for Reading Difficulty Prediction. In Proceedings of the 3rd Workshop on Innovative Use of NLP for Building Educational Applications, pages 71-79. [find](https://scholar.google.com/scholar?q=Michael+Heilman%2C+Kevyn+Collins%2DThompson+and+Maxine+Eskenazi+%282008%29%2E+An+Analysis+of+Statistical+Models+and+Features+for+Reading+Difficulty+Prediction%2E+In+Proceedings+of+the+3rd+Workshop+on+Innovative+U)
17. INAF (2009). Instituto P. Montenegro and Ação Educativa. INAF Brasil - Indicador de Alfabetismo Funcional - 2009. Available online at [http://www.ibope](http://www.ibope). com.br/ipm/relatorios/relatorio_inaf_2009.pdf Teresa B. F. Martins, Claudete M. Ghiraldelo, Maria das Graças V. Nunes e Osvaldo N. de Oliveira Jr. (1996). Readability formulas applied to textbooks in brazilian portuguese. ICMC Technical Report, N. 28, 11p. [↗](http://www.ibope)
18. Aurélien Max (2006). Writing for Language-impaired Readers. In Proceedings of CICLing, pages 567-570. [find](https://scholar.google.com/scholar?q=Aur%C3%A9lien+Max+%282006%29%2E+Writing+for+Language%2Dimpaired+Readers%2E+In+Proceedings+of+CICLing%2C+pages+567%2D570%2E)
19. Danielle McNamara, Max Louwerse, and Art Graesser, 2002. Coh-Metrix: Automated cohesion and coherence scores to predict text readability and facilitate comprehension. Grant proposal. [http://cohmetrix.memphis.edu/cohmetrixpr/publications.html](http://cohmetrix.memphis.edu/cohmetrixpr/publications.html) Eleni Miltsakaki and Audrey Troutt (2007). Read-X: Automatic Evaluation of Reading Difficulty of Web Text. In the Proceedings of E-Learn 2007, Quebec, Canada. [↗](http://cohmetrix.memphis.edu/cohmetrixpr/publications.html)
20. Eleni Miltsakaki and Audrey Troutt (2008). Real Time Web Text Classification and Analysis of Reading Difficulty. In the Proceedings of the 3rd Workshop on Innovative Use of NLP for Building Educational Applications, Columbus, OH. [find](https://scholar.google.com/scholar?q=Eleni+Miltsakaki+and+Audrey+Troutt+%282008%29%2E+Real+Time+Web+Text+Classification+and+Analysis+of+Reading+Difficulty%2E+In+the+Proceedings+of+the+3rd+Workshop+on+Innovative+Use+of+NLP+for+Building+Educationa)
21. Cláudia Oliveira, Maria C. Freitas, Violeta Quental, Cícero N. dos Santos, Renato P. L. and Lucas Souza (2006). A Set of NP-extraction rules for Portuguese: defining and learning. In 7th Workshop on Computational Processing of Written and Spoken Portuguese, Itatiaia, Brazil. [find](https://scholar.google.com/scholar?q=Cl%C3%A1udia+Oliveira%2C+Maria+C%2E+Freitas%2C+Violeta+Quental%2C+C%C3%ADcero+N%2E+dos+Santos%2C+Renato+P%2E+L%2E+and+Lucas+Souza+%282006%29%2E+A+Set+of+NP%2Dextraction+rules+for+Portuguese%3A+defining+and+learning%2E+In+7th+Workshop+on+C)
22. Sarah E. Petersen and Mari Ostendorf (2009). A machine learning approach to reading level assessment. Computer Speech and Language 23, 89-106. [↗](https://doi.org/10.1016/j.csl.2008.04.003)
23. Emily Pitler and Ani Nenkova (2008). Revisiting readability: A unified framework for predicting text quality. In Proceedings of EMNLP, 2008. [find](https://scholar.google.com/scholar?q=Emily+Pitler+and+Ani+Nenkova+%282008%29%2E+Revisiting+readability%3A+A+unified+framework+for+predicting+text+quality%2E+In+Proceedings+of+EMNLP%2C+2008%2E)
24. Adwait Ratnaparkhi (1996). A Maximum Entropy Part-of-Speech Tagger. In Proceedings of the First Empirical Methods in Natural Language Processing Conference, pages133-142. [find](https://scholar.google.com/scholar?q=Adwait+Ratnaparkhi+%281996%29%2E+A+Maximum+Entropy+Part%2Dof%2DSpeech+Tagger%2E+In+Proceedings+of+the+First+Empirical+Methods+in+Natural+Language+Processing+Conference%2C+pages133%2D142%2E)
25. Brian Roark, Margaret Mitchell and Kristy Hollingshead (2007). Syntactic complexity measures for detecting mild cognitive impairment. In the Proceedings of the Workshop on BioNLP 2007: Biological, Translational, and Clinical Language Processing, Prague, Czech Republic. [find](https://scholar.google.com/scholar?q=Brian+Roark%2C+Margaret+Mitchell+and+Kristy+Hollingshead+%282007%29%2E+Syntactic+complexity+measures+for+detecting+mild+cognitive+impairment%2E+In+the+Proceedings+of+the+Workshop+on+BioNLP+2007%3A+Biological%2C+Tra)
26. Caroline E. Scarton, Daniel M. Almeida, Sandra M. Aluísio (2009). Análise da Inteligibilidade de textos via ferramentas de Processamento de Língua Natural: adaptando as métricas do Coh-Metrix para o Português. In Proceedings of STIL-2009, São Carlos, Brazil. [find](https://scholar.google.com/scholar?q=Caroline+E%2E+Scarton%2C+Daniel+M%2E+Almeida%2C+Sandra+M%2E+Alu%C3%ADsio+%282009%29%2E+An%C3%A1lise+da+Inteligibilidade+de+textos+via+ferramentas+de+Processamento+de+L%C3%ADngua+Natural%3A+adaptando+as+m%C3%A9tricas+do+Coh%2DMetrix+para+o+P)
27. Sarah E. Schwarm and Mari Ostendorf (2005). Reading Level Assessment Using Support Vector Machines and Statistical Language Models. In the Proceedings of the 43rd Annual Meeting of the ACL, pp 523–530. [find](https://scholar.google.com/scholar?q=Sarah+E%2E+Schwarm+and+Mari+Ostendorf+%282005%29%2E+Reading+Level+Assessment+Using+Support+Vector+Machines+and+Statistical+Language+Models%2E+In+the+Proceedings+of+the+43rd+Annual+Meeting+of+the+ACL%2C+pp+523%E2%80%93530)
28. Kathleen M. Sheehan, Irene Kostin and Yoko Futagi (2007). Reading Level Assessment for Literary and Expository Texts. In D. S. McNamara and J. G. Trafton (Eds.), Proceedings of the 29th Annual Cognitive Science Society, page 1853. Austin, TX: Cognitive Science Society. [find](https://scholar.google.com/scholar?q=Kathleen+M%2E+Sheehan%2C+Irene+Kostin+and+Yoko+Futagi+%282007%29%2E+Reading+Level+Assessment+for+Literary+and+Expository+Texts%2E+In+D%2E+S%2E+McNamara+and+J%2E+G%2E+Trafton+%28Eds%2E%29%2C+Proceedings+of+the+29th+Annual+Cogniti)
29. Advaith Siddharthan (2003). Syntactic Simplification and Text Cohesion. PhD Thesis. University of Cambridge. [find](https://scholar.google.com/scholar?q=Advaith+Siddharthan+%282003%29%2E+Syntactic+Simplification+and+Text+Cohesion%2E+PhD+Thesis%2E+University+of+Cambridge%2E)
30. Andreas Stolcke. SRILM -- an extensible language modeling toolkit. In Proceedings of the International Conference on Spoken Language Processing, 2002. [find](https://scholar.google.com/scholar?q=Andreas+Stolcke%2E+SRILM+%2D%2D+an+extensible+language+modeling+toolkit%2E+In+Proceedings+of+the+International+Conference+on+Spoken+Language+Processing%2C+2002%2E)