---
title: "Criterial Features in German — paper by Loefflad et al. (2026)"
description: "Automatic readability assessment for language learners usually assigns one level to a whole text. 29 references."
authors: ["Denise Loefflad", "Sofia Kathmann", "Heiko Holz", "Detmar Meurers"]
year: 2026
venue: "Proceedings of the 21st Workshop on Innovative Use of NLP for Building Educational Applications (BEA 2026)"
source: "https://aclanthology.org/2026.bea-1.6/"
doi: "10.18653/v1/2026.bea-1.6"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/loefflad-2026-criterial-features-german-interpretable-nlp/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/loefflad-2026-criterial-features-german-interpretable-nlp/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/28/) / Criterial Features in German: Towards Interpretable NLP in Readability Assessment

# Criterial Features in German: Towards Interpretable NLP in Readability Assessment

**Authors:** Denise Loefflad, Sofia Kathmann, Heiko Holz and [Detmar Meurers](https://md.research.elamur.ai/free/en/authors/detmar-meurers/) · **Year:** 2026 · **Venue:** [Proceedings of the 21st Workshop on Innovative Use of NLP for Building Educational Applications (BEA 2026)](https://md.research.elamur.ai/free/en/venues/bea/) ([BEA 2026](https://md.research.elamur.ai/free/en/venues/bea/2026/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2026.bea-1.6/) · [PDF](https://aclanthology.org/2026.bea-1.6.pdf) · [DOI 10.18653/v1/2026.bea-1.6](https://doi.org/10.18653/v1/2026.bea-1.6) · [OpenAlex](https://openalex.org/W7166839947)

**Citations:** 0 citations (OpenAlex, 2026-10-14)

## Summary
Automatic readability assessment for language learners usually assigns one level to a whole text. It does not reveal which constructions make the text hard. The authors ask whether criterial features (CFs), grammatical structures linked to CEFR levels, can make this more interpretable for German. They evaluate the German Grammar Profile (GGP), a theory-driven inventory of 153 CEFR-mapped features, and its rule-based extractor PALME.

First, three experts built a test suite with five positive and five deliberately tricky negative sentences for every feature. PALME performed reliably: over nine in ten features reached precision and recall above 0.8. Most errors came from ambiguous case morphology on nouns and adjectives.

Second, PALME was applied to a graded reader corpus, and an Explainable Boosting Machine was trained to predict levels. It reached precision 0.75 and recall 0.73. Inspecting the model showed that verb-construction features partly follow the stages predicted by Processability Theory.

The work gives German a validated, transparent tool for choosing and diagnosing texts. It also provides an evaluation approach that could be adapted for use with languages beyond German.

## Key points
- The GGP covers 153 CFs: 47 at A1, 49 at A2, 35 at B1 and 22 at B2. It is derived from Profile Deutsch and modeled on the English Grammar Profile (EGP).
- PALME (Pädagogisch Ausgerichtete Linguistische Merkmalsextraktion) is rule-based. It is comparable to POLKE for English and SABER for Portuguese.
- The test suite contains 10 sentences per feature, and more than 90% of features score above 0.8. The weak spots are noun and adjective case ambiguity.
- On graded readers, the EBM achieves precision 0.75 and recall 0.73. Separable-verb (split verb) patterns partially match Processability Theory, while morphological features also matter.
- A complexity-based classifier (Weiss et al., 2021: 300+ features, SVM) scores higher on the same corpus. CFs, however, are more interpretable and pedagogically grounded. The authors also suggest reusing the test suite to probe LLMs.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [University of Tübingen](https://md.research.elamur.ai/free/en/universities/university-of-tubingen/) — University, Tübingen, Germany · 78 works · 41,280 citations
- [Leibniz-Institut für Wissensmedien](https://md.research.elamur.ai/free/en/labs/leibniz-institut-fur-wissensmedien/) — Research institute, Tübingen, Germany · 2 works

Also on the byline: [Ludwigsburg University of Education](https://ror.org/02j0n6s98).

📚 Shelves (6): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 18,614 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 17,965 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 17,450 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 17,296 works · [💭 Psychology](https://md.research.elamur.ai/free/en/shelves/psychology/) · 1,649 works · [💭 Developmental and Educational Psychology](https://md.research.elamur.ai/free/en/shelves/psychology/developmental-and-educational-psychology/) · 523 works

🧭 Topics (1): [🖍️ Grammatical error correction & learner language](https://md.research.elamur.ai/free/en/topics/grammatical-error-correction/) · 162 works

## Cite

Cite as: Denise Loefflad, Sofia Kathmann, Heiko Holz and Detmar Meurers. 2026. Criterial Features in German: Towards Interpretable NLP in Readability Assessment. Proceedings of the 21st Workshop on Innovative Use of NLP for Building Educational Applications (BEA 2026).

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{loefflad2026criterial,
  title = {{Criterial Features in German: Towards Interpretable NLP in Readability Assessment}},
  author = {Denise Loefflad, Sofia Kathmann, Heiko Holz and Detmar Meurers},
  booktitle = {Proceedings of the 21st Workshop on Innovative Use of NLP for Building Educational Applications (BEA 2026)},
  year = {2026},
  doi = {10.18653/v1/2026.bea-1.6},
  url = {https://doi.org/10.18653/v1/2026.bea-1.6}
}
```

- [← previous CRiT-QA: Evaluating Multi-hop Reasoning with Counterfactual Chains and Distractor Traps](https://md.research.elamur.ai/free/en/papers/yun-2026-crit-qa-evaluating-multi-hop/)
- [next → Critic Rule Induction: Improving Temporal Knowledge Graph Forecasting with Generator-Critic Language Models](https://md.research.elamur.ai/free/en/papers/ning-2026-critic-rule-induction-improving-temporal/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/28/)

## References

29 references, 1 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Soroosh Akef, Detmar Meurers, Amália Mendes, and Patrick Rebuschat. 2025. [Interpretable Machine Learning for Societal Language Identification: Modeling English and German Influences on Portuguese Heritage Language](https://md.research.elamur.ai/free/en/papers/akef-2025-interpretable-machine-learning-societal-language/). In Proceedings of the 14th Workshop on Natural Language Processing for Computer Assisted Language Learning, pages 50–62.
2. Lorna Balkan, Siety Meijer, Doug Arnold, Eva Dauphin, Dominique Estival, Kirsten Falkedal, Sabine Lehmann, Klaus Netter, and Sylvie Regnier-Prost. 1994. Issues in Test Suite Design. Report to LRE, 62:089. [find](https://scholar.google.com/scholar?q=Lorna+Balkan%2C+Siety+Meijer%2C+Doug+Arnold%2C+Eva+Dauphin%2C+Dominique+Estival%2C+Kirsten+Falkedal%2C+Sabine+Lehmann%2C+Klaus+Netter%2C+and+Sylvie+Regnier%2DProst%2E+1994%2E+Issues+in+Test+Suite+Design%2E+Report+to+LRE%2C+62%3A)
3. Denise Löfflad, Benedikt Beuttler, and Detmar Meurers. 2025. German Grammar Profile for Learners: Pedagogical Feature Definition and Automated Extraction. In Proceedings of the 21st Conference on Natural Language Processing (KONVENS 2025): Workshops, pages 212–223. [find](https://scholar.google.com/scholar?q=Denise+L%C3%B6fflad%2C+Benedikt+Beuttler%2C+and+Detmar+Meurers%2E+2025%2E+German+Grammar+Profile+for+Learners%3A+Pedagogical+Feature+Definition+and+Automated+Extraction%2E+In+Proceedings+of+the+21st+Conference+on+Natu)
4. Council of Europe. 2020. Common European Framework of Reference for Languages: Learning, Teaching, Assessment – Companion Volume. Council of Europe Publishing, Strasbourg, France. [find](https://scholar.google.com/scholar?q=Council+of+Europe%2E+2020%2E+Common+European+Framework+of+Reference+for+Languages%3A+Learning%2C+Teaching%2C+Assessment+%E2%80%93+Companion+Volume%2E+Council+of+Europe+Publishing%2C+Strasbourg%2C+France%2E)
5. Yin Lou, Rich Caruana, Johannes Gehrke, and Giles Hooker. 2013. Accurate Intelligible Models with Pairwise Interactions. In Proceedings of the 19th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD ’13), pages 623–631. [find](https://scholar.google.com/scholar?q=Yin+Lou%2C+Rich+Caruana%2C+Johannes+Gehrke%2C+and+Giles+Hooker%2E+2013%2E+Accurate+Intelligible+Models+with+Pairwise+Interactions%2E+In+Proceedings+of+the+19th+ACM+SIGKDD+International+Conference+on+Knowledge+Dis)
6. Rocio Cuberos Vicente and Barbara De Cock. 2023. Towards a Graded Lexical Inventory in L2 Spanish: Insights from Productive Vocabulary Knowledge. In Proceedings of the Workshop on Linking Lexicographic and Language Learning Resources (4LR). [find](https://scholar.google.com/scholar?q=Rocio+Cuberos+Vicente+and+Barbara+De+Cock%2E+2023%2E+Towards+a+Graded+Lexical+Inventory+in+L2+Spanish%3A+Insights+from+Productive+Vocabulary+Knowledge%2E+In+Proceedings+of+the+Workshop+on+Linking+Lexicographi)
7. Salwa Mohamed. 2023. The Development of an Arabic Curriculum Framework based on a Compilation of Salient Features from CEFR Level Descriptors. The Language Learning Journal, 51(1):33–47. [find](https://scholar.google.com/scholar?q=Salwa+Mohamed%2E+2023%2E+The+Development+of+an+Arabic+Curriculum+Framework+based+on+a+Compilation+of+Salient+Features+from+CEFR+Level+Descriptors%2E+The+Language+Learning+Journal%2C+51%281%29%3A33%E2%80%9347%2E)
8. Mark Davies. 2009. The 385+ Million Word Corpus of Contemporary American English (1990–2008+): Design, Architecture, and Linguistic Insights. International Journal of Corpus Linguistics, 14(2):159–190. [↗](https://doi.org/10.1075/ijcl.14.2.02dav)
9. Harsha Nori, Samuel Jenkins, Paul Koch, and Rich Caruana. 2019. InterpretML: A Unified Framework for Machine Learning Interpretability. arXiv preprint arXiv:1909.09223. [↗](https://arxiv.org/abs/1909.09223)
10. Thomas Gaillat, Andrew Simpkin, Nicolas Ballier, Bernardo Stearns, Annanda Sousa, Manon Bouyé, and Manel Zarrouk. 2022. Predicting CEFR levels in learners of English: The Use of Microsystem Criterial Features in a Machine Learning Approach. ReCALL, 34(2):130–146. [↗](https://doi.org/10.1017/s095834402100029x)
11. Anne O’Keeffe and Geraldine Mark. 2017. The English Grammar Profile of Learner Competence: Methodology and Key Findings. International Journal of Corpus Linguistics, 22(4):457–489. Publisher: John Benjamins Publishing Company Amsterdam/Philadelphia. [↗](https://doi.org/10.1075/ijcl.14086.oke)
12. Manuela Glaboniat, editor. 2010. Profile Deutsch: Gemeinsamer europäischer Referenzrahmen; Lernzielbestimmungen; Kannbeschreibungen; Kommunikative Mittel; Niveau A1-A2, B1-B2; C1-C2; [CD-ROM Version 2.0 mit Begleitbuch], nachdr. edition. Langenscheidt. [find](https://scholar.google.com/scholar?q=Manuela+Glaboniat%2C+editor%2E+2010%2E+Profile+Deutsch%3A+Gemeinsamer+europ%C3%A4ischer+Referenzrahmen%3B+Lernzielbestimmungen%3B+Kannbeschreibungen%3B+Kommunikative+Mittel%3B+Niveau+A1%2DA2%2C+B1%2DB2%3B+C1%2DC2%3B+%5BCD%2DROM+Version+2)
13. Manfred Pienemann. 1989. Is language teachable? psycholinguistic experiments and hypotheses. Applied Linguistics, 10(1):52–79. [↗](https://doi.org/10.1093/applin/10.1.52)
14. Manfred Pienemann. 1998. Developmental Dynamics in L1 and L2 Acquisition: Processability Theory and Generative Entrenchment. Bilingualism: Language and Cognition, 1(1):1–20. [↗](https://doi.org/10.1017/s1366728998000017)
15. John A. Hawkins and Paula Buttery. 2010. Criterial Features in Learner Corpora: Theory and Illustrations. English Profile Journal, 1:e5. [↗](https://doi.org/10.1017/s2041536210000103)
16. Manfred Pienemann. 2015. An outline of processability theory and its relationship to other approaches to SLA. Language Learning, 65(1):123–151. [↗](https://doi.org/10.1111/lang.12095)
17. John A. Hawkins and Luna Filipović. 2012. Criterial Features in L2 English: Specifying the Reference Levels of the Common European Framework, volume 1 of English Profile Studies. Cambridge University Press. [find](https://scholar.google.com/scholar?q=John+A%2E+Hawkins+and+Luna+Filipovic%CC%81%2E+2012%2E+Criterial+Features+in+L2+English%3A+Specifying+the+Reference+Levels+of+the+Common+European+Framework%2C+volume+1+of+English+Profile+Studies%2E+Cambridge+University)
18. Rashmi Prasad and Anoop Sarkar. 2000. Comparing Test-Suite based Evaluation and Corpus-based Evaluation of a wide-coverage Grammar for English. In Proceedings of the Workshop on Using Evaluation within Human Language Technology Programs: Results and Trends. (LREC), pages 7–12, Athens, Greece. [find](https://scholar.google.com/scholar?q=Rashmi+Prasad+and+Anoop+Sarkar%2E+2000%2E+Comparing+Test%2DSuite+based+Evaluation+and+Corpus%2Dbased+Evaluation+of+a+wide%2Dcoverage+Grammar+for+English%2E+In+Proceedings+of+the+Workshop+on+Using+Evaluation+withi)
19. Stephen D Krashen. 1985. The input hypothesis: Issues and implications. Longman, New York. [find](https://scholar.google.com/scholar?q=Stephen+D+Krashen%2E+1985%2E+The+input+hypothesis%3A+Issues+and+implications%2E+Longman%2C+New+York%2E)
20. Nadezhda Stanislavovna Lagutina, Kseniya Vladimirovna Lagutina, Anasta-sya Mikhailovna Brederman, and Natalia Nikolaevna Kasatkina. 2023. Text Classification by CEFR Levels using Machine Learning Methods and BERT Language Model. Modeling and Analysis of Information Systems, 30(3):202–213. [↗](https://doi.org/10.18255/1818-1015-2023-3-202-213)
21. Nelly Sagirov and Xiaobin Chen. 2025. POLKE: A System for Comprehensively Annotating Pedagogically-Oriented Grammatical Structure Use in Language Production. [↗](https://doi.org/10.1016/j.rmal.2026.100336)
22. Angeliki Salamoura and Nick Saville. 2010. Exemplifying the CEFR: Criterial Features of Written Learner English from the English Profile Programme. In Inge Bartning, Maisa Martin, and Ineke Vedder, editors, Communicative proficiency and linguistic development: Intersections between SLA and language testing research, volume 1 of Eurosla Monographs Series, pages 101–131. European Second Language Association. [find](https://scholar.google.com/scholar?q=Angeliki+Salamoura+and+Nick+Saville%2E+2010%2E+Exemplifying+the+CEFR%3A+Criterial+Features+of+Written+Learner+English+from+the+English+Profile+Programme%2E+In+Inge+Bartning%2C+Maisa+Martin%2C+and+Ineke+Vedder%2C+ed)
23. Sabine Lehmann, Stephan Oepen, Sylvie Regnier-Prost, Klaus Netter, Veronika Lux, Judith Klein, Kirsten Falkedal, Frederik Fouvry, Dominique Estival, and Eva Dauphin. 1996. TSNLP - Test Suites for Natural Language Processing. In Proceedings of the 16th International Conference on Computational Linguistics (COLING), pages 711–716. [find](https://scholar.google.com/scholar?q=Sabine+Lehmann%2C+Stephan+Oepen%2C+Sylvie+Regnier%2DProst%2C+Klaus+Netter%2C+Veronika+Lux%2C+Judith+Klein%2C+Kirsten+Falkedal%2C+Frederik+Fouvry%2C+Dominique+Estival%2C+and+Eva+Dauphin%2E+1996%2E+TSNLP+%2D+Test+Suites+for+Natu)
24. Andrea Vázquez-Ingelmo, Denise Löfflad, Alicia García-Holgado, Roberto Therón-Sánchez, and Detmar Meurers. 2026. From CEFR Classification to Generative AI Materials: Designing and Validating the LATILL Platform. Universal Access in the Information Society, 25(2):67. [↗](https://doi.org/10.1007/s10209-026-01333-8)
25. Daniela Verratti-Souto, Nelly Sagirov, and Xiaobin Chen. 2025. NLP-Powered Quantitative Verification of the English Grammar Profile’s Structure-Level Assignment. Annual Review of Applied Linguistics, pages 1–22. [↗](https://doi.org/10.1017/s0267190525100093)
26. Zarah Weiss. 2024. An Integrative Approach to Linguistic Complexity Analysis for German. PhD Thesis, Universität Tübingen. [find](https://scholar.google.com/scholar?q=Zarah+Weiss%2E+2024%2E+An+Integrative+Approach+to+Linguistic+Complexity+Analysis+for+German%2E+PhD+Thesis%2C+Universit%C3%A4t+T%C3%BCbingen%2E)
27. Zarah Weiss, Xiaobin Chen, and Detmar Meurers. 2021. Using Broad Linguistic Complexity Modeling for Cross-Lingual Readability Assessment. In Proceedings of the 10th Workshop on NLP for Computer Assisted Language Learning, pages 38–54. LiU Electronic Press. [find](https://scholar.google.com/scholar?q=Zarah+Weiss%2C+Xiaobin+Chen%2C+and+Detmar+Meurers%2E+2021%2E+Using+Broad+Linguistic+Complexity+Modeling+for+Cross%2DLingual+Readability+Assessment%2E+In+Proceedings+of+the+10th+Workshop+on+NLP+for+Computer+Assist)
28. Zarah Weiss and Detmar Meurers. 2021. Analyzing the Linguistic Complexity of German Learner Language in a Reading Comprehension Task: Using Proficiency Classification to Investigate Short Answer Data, Cross-Data Generalizability, and the Impact of Linguistic Analysis Quality. International Journal of Learner Corpus Research, 7(1):84–131. [find](https://scholar.google.com/scholar?q=Zarah+Weiss+and+Detmar+Meurers%2E+2021%2E+Analyzing+the+Linguistic+Complexity+of+German+Learner+Language+in+a+Reading+Comprehension+Task%3A+Using+Proficiency+Classification+to+Investigate+Short+Answer+Data%2C)
29. Katrin Wisniewski. 2020. SLA Developmental Stages in the CEFR-Related Learner Corpus MERLIN: Inversion and Verb-End Structures in German A2 and B1 Learner Texts. International Journal of Learner Corpus Research, 6(1):1–37. [find](https://scholar.google.com/scholar?q=Katrin+Wisniewski%2E+2020%2E+SLA+Developmental+Stages+in+the+CEFR%2DRelated+Learner+Corpus+MERLIN%3A+Inversion+and+Verb%2DEnd+Structures+in+German+A2+and+B1+Learner+Texts%2E+International+Journal+of+Learner+Corpu)

*Updated 1 November 2026*