---
title: "Un prototype en ligne pour la prédiction du niveau de"
description: "This demonstration paper presents a proof-of-concept tool that automatically places English learners' essays… 82% accuracy. 14 references."
authors: ["Thomas Gaillat", "Nicolas Ballier", "Annanda Sousa", "Manon Bouyé", "Andrew Simpkin", "Bernardo Stearns", "Manel Zarrouk"]
year: 2020
venue: "Actes de la 6e conférence conjointe Journées d'Études sur la Parole (JEP, 33e édition), Traitement Automatique des Langues Naturelles (TALN, 27e édition), Rencontre des Étudiants Chercheurs en Informatique pour le Traitement Automatique des Langues (RÉCITAL, 22e édition). Volume 4 : Démonstrations et résumés d'articles internationaux"
source: "https://aclanthology.org/2020.jeptalnrecital-demos.8/"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/gaillat-2020-un-prototype-en-ligne-pour/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/gaillat-2020-un-prototype-en-ligne-pour/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2020](https://md.research.elamur.ai/free/en/papers/2020/page/11/) / Un prototype en ligne pour la prédiction du niveau de compétence en anglais des productions écrites (A prototype for web-based prediction of English proficiency levels in writings)

# Un prototype en ligne pour la prédiction du niveau de compétence en anglais des productions écrites (A prototype for web-based prediction of English proficiency levels in writings)

**Authors:** [Thomas Gaillat](https://md.research.elamur.ai/free/en/authors/thomas-gaillat/), [Nicolas Ballier](https://md.research.elamur.ai/free/en/authors/nicolas-ballier/), [Annanda Sousa](https://md.research.elamur.ai/free/en/authors/annanda-sousa/), Manon Bouyé, [Andrew Simpkin](https://md.research.elamur.ai/free/en/authors/andrew-simpkin/), [Bernardo Stearns](https://md.research.elamur.ai/free/en/authors/bernardo-stearns/) and [Manel Zarrouk](https://md.research.elamur.ai/free/en/authors/manel-zarrouk/) · **Year:** 2020 · **Venue:** [Actes de la 6e conférence conjointe Journées d'Études sur la Parole (JEP, 33e édition), Traitement Automatique des Langues Naturelles (TALN, 27e édition), Rencontre des Étudiants Chercheurs en Informatique pour le Traitement Automatique des Langues (RÉCITAL, 22e édition). Volume 4 : Démonstrations et résumés d'articles internationaux](https://md.research.elamur.ai/free/en/venues/jeptalnrecital/) ([JEP-TALN-RECITAL (the French-speaking conferences on speech and natural language processing) 2020](https://md.research.elamur.ai/free/en/venues/jeptalnrecital/2020/)) · **Publisher:** [ATALA](https://md.research.elamur.ai/free/en/publishers/atala/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2020.jeptalnrecital-demos.8/) · [PDF](https://aclanthology.org/2020.jeptalnrecital-demos.8.pdf) · [OpenAlex](https://openalex.org/W3036993351)

**Citations:** 0 citations (OpenAlex, 2026-10-14)

## Summary
This demonstration paper presents a proof-of-concept tool that automatically places English learners' essays on CEFR proficiency levels. The authors trained a supervised classifier on EFCAMDAT, a corpus of over 40,000 level-annotated learner texts. Each text is described by 769 features from a chain of existing analysers covering syntactic, lexical, discourse and psycholinguistic complexity, plus readability and spelling. The features describe what the learner actually produces and do not rely on annotated errors. A novel feature family, "microsystems", records how often competing linguistic forms are used relative to one another, independent of the rest of the text. Among the models tried, logistic regression performed best, reaching 82% accuracy. The system has two parts: a public web interface that returns a predicted CEFR class with its probability, and a command-line batch pipeline that builds metric datasets. The work shows how criterial features from second-language research can be turned into a usable assessment tool for teachers.

## Key points
- Training data: EFCAMDAT, more than 40,000 texts and about 3,298,343 tokens, already annotated with proficiency levels (Geertzen et al., 2013).
- There are 769 features, and the best model is logistic regression, with a reported accuracy of 82%.
- The feature tools are a modified L2SCA, LCA, TAALES, TAASC, TAACO, Pyenchant 2.0.0 (an Aspell-based spelling check) and Textstat (readability indices such as Dale_Chall, plus n-gram frequencies).
- The architecture uses interconnected Docker modules written in Python 3.6. Because one tool depends on a proprietary corpus, only a reduced version of the pipeline is publicly released.
- Planned extensions include:
  - speech input through the SpeechRecognition library;
  - morphological microsystem metrics;
  - visualisations comparing an individual learner with typical learner profiles.

## Interesting ideas
- Measuring "microsystems", i.e. the usage ratios among a small set of competing forms, captures paradigmatic choices that length-based syntactic measures miss, and it can be applied to any closed set of alternatives.
- Features describe the positive properties a writer shows rather than counting errors, which avoids dependence on costly error annotation.
- Many heterogeneous off-the-shelf analysers are concatenated into one feature vector and fed to a transparent linear model, so each prediction can be traced back to named metrics.
- The same metric pipeline serves both an interactive interface (class plus probability) and a batch mode that writes per-text metric datasets for research.

## For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

- **I1** (tree): compute ratios between competing constructions in parsed UD trees, so a skewed distribution in a new parser's output signals a dialect or conversion problem.
- **I2** (answer): grade student answers on school-course tests by which structures and relations they correctly express, giving an explainable profile rather than a list of mistakes.
- **I3** (learn): estimate text difficulty from interpretable tree and lexical metrics, then order self-play or training texts from easy to hard.
- **I4** (learn): add a batch mode that dumps per-text metric tables, so evaluation runs and negative controls can be compared with simple statistics.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Université Paris Cité](https://md.research.elamur.ai/free/en/universities/universite-paris-cite/) — University, Paris, France · 20 works · 1,426 citations
- [Laboratoire d'Informatique de Paris-Nord](https://md.research.elamur.ai/free/en/labs/laboratoire-d-informatique-de-paris-nord/) — Research institute, Villetaneuse, France · 10 works · 510 citations
- [Université Sorbonne Paris Nord](https://md.research.elamur.ai/free/en/universities/universite-sorbonne-paris-nord/) — University, Villetaneuse, France · 9 works · 468 citations
- [Approches Linguistiques Théoriques, Appliquées et Expérimentales : langues et cultures connectées](https://md.research.elamur.ai/free/en/labs/approches-linguistiques-theoriques-appliquees-et-experimentales-langues-et/) — Research institute, Paris, France · 10 works · 260 citations
- [Linguistique Ingénierie et Didactique des Langues](https://md.research.elamur.ai/free/en/labs/linguistique-ingenierie-et-didactique-des-langues/) — Research institute, Rennes, France · 5 works · 14 citations
- [Université Rennes 2](https://md.research.elamur.ai/free/en/universities/universite-rennes-2/) — University, Rennes, France · 4 works · 9 citations

📚 Shelves (6): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 18,614 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 17,965 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 17,450 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 17,296 works · [💭 Psychology](https://md.research.elamur.ai/free/en/shelves/psychology/) · 1,649 works · [💭 Developmental and Educational Psychology](https://md.research.elamur.ai/free/en/shelves/psychology/developmental-and-educational-psychology/) · 523 works

🧭 Topics (1): [English](https://md.research.elamur.ai/free/en/topics/english/) · 2,379 works

## Cite

Cite as: Thomas Gaillat, Nicolas Ballier, Annanda Sousa, Manon Bouyé, Andrew Simpkin, Bernardo Stearns and Manel Zarrouk. 2020. Un prototype en ligne pour la prédiction du niveau de compétence en anglais des productions écrites (A prototype for web-based prediction of English proficiency levels in writings). Actes de la 6e conférence conjointe Journées d'Études sur la Parole (JEP, 33e édition), Traitement Automatique des Langues Naturelles (TALN, 27e édition), Rencontre des Étudiants Chercheurs en Informatique pour le Traitement Automatique des Langues (RÉCITAL, 22e édition). Volume 4 : Démonstrations et résumés d'articles internationaux.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{gaillat2020un,
  title = {{Un prototype en ligne pour la prédiction du niveau de compétence en anglais des productions écrites (A prototype for web-based prediction of English proficiency levels in writings)}},
  author = {Thomas Gaillat, Nicolas Ballier, Annanda Sousa, Manon Bouyé, Andrew Simpkin, Bernardo Stearns and Manel Zarrouk},
  booktitle = {Actes de la 6e conférence conjointe Journées d'Études sur la Parole (JEP, 33e édition), Traitement Automatique des Langues Naturelles (TALN, 27e édition), Rencontre des Étudiants Chercheurs en Informatique pour le Traitement Automatique des Langues (RÉCITAL, 22e édition). Volume 4 : Démonstrations et résumés d'articles internationaux},
  year = {2020},
  url = {https://aclanthology.org/2020.jeptalnrecital-demos.8/}
}
```

- [← previous Transitivity and intransitivity in the grammar of Bar Hebraeus](https://md.research.elamur.ai/free/en/papers/bohas-2020-transitivite-et-intransitivite-dans-la/)
- [next → Vom x-qatal zur Informationsstruktur – und retour. Bemerkungen zur Analyse althebräischer Sätze in der Poesie (From x-qatal to information structure – and back again)](https://md.research.elamur.ai/free/en/papers/schiller-2020-x-qatal-information-structure-back/)
- [↑ back to papers of 2020](https://md.research.elamur.ai/free/en/papers/2020/page/11/)

## References

14 references, 0 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. B ALLIER N., G AILLAT T., S IMPKIN A., S TEARNS B., B OUYÉ M. & Z ARROUK M. (2019). A supervised learning model for the automatic assessment of language levels based on learner errors. In European Conference on Technology Enhanced Learning, p. 308–320 : Springer. [find](https://scholar.google.com/scholar?q=B+ALLIER+N%2E%2C+G+AILLAT+T%2E%2C+S+IMPKIN+A%2E%2C+S+TEARNS+B%2E%2C+B+OUY%C3%89+M%2E+%26+Z+ARROUK+M%2E+%282019%29%2E+A+supervised+learning+model+for+the+automatic+assessment+of+language+levels+based+on+learner+errors%2E+In+European+Con)
2. B ANSAL S. (2018). Textstat Python package. shivam5992/textstat. Accessible sur [https://github.com/](https://github.com/) [↗](https://github.com/)
3. B REZINA V. & P ALLOTTI G. (2019). Morphological complexity in written l2 texts. Second language research, 35(1), 99–119. [find](https://scholar.google.com/scholar?q=B+REZINA+V%2E+%26+P+ALLOTTI+G%2E+%282019%29%2E+Morphological+complexity+in+written+l2+texts%2E+Second+language+research%2C+35%281%29%2C+99%E2%80%93119%2E)
4. D ECHERT , H ANS -W ILHELM (1983). How a story is done in a second language. In C. F ÆRCH & G. K ASPER , Éds., Strategies in interlanguage communication, p. 175–195. London ; New York : Longman. OCLC : 644977107. [find](https://scholar.google.com/scholar?q=D+ECHERT+%2C+H+ANS+%2DW+ILHELM+%281983%29%2E+How+a+story+is+done+in+a+second+language%2E+In+C%2E+F+%C3%86RCH+%26+G%2E+K+ASPER+%2C+%C3%89ds%2E%2C+Strategies+in+interlanguage+communication%2C+p%2E+175%E2%80%93195%2E+London+%3B+New+York+%3A+Longman%2E+OCLC+)
5. E UROPEAN C OUNCIL (2001). Common European Framework of Reference for Languages : Learning, teaching, assessment. Cambridge : Cambridge University Press. G AILLAT T., S IMPKIN A., B ALLIER N., S TEARNS B., S OUSA A., B OUYÉ M. & Z ARROUK M. (submitted). Predicting CEFR levels in learners of English : the use of microsystem criterial features in a machine learning approach. Journal With Anonymous Submission. [find](https://scholar.google.com/scholar?q=E+UROPEAN+C+OUNCIL+%282001%29%2E+Common+European+Framework+of+Reference+for+Languages+%3A+Learning%2C+teaching%2C+assessment%2E+Cambridge+%3A+Cambridge+University+Press%2E+G+AILLAT+T%2E%2C+S+IMPKIN+A%2E%2C+B+ALLIER+N%2E%2C+S+TEARN)
6. G EERTZEN J., A LEXOPOULOU T. & K ORHONEN A. (2013). Automatic Linguistic Annotation of Large Scale L2 Databases : The EF-Cambridge Open Language Database (EFCamDat). In R. T. M ILLER , K. I. M ARTIN , C. M. E DDINGTON , A. H ENERY , N. M IGUEL , A. T SENG , A. T UNINETTI & D. W ALTER , Éds., Proceeedings of the 31st Second Language Research Forum, Carnegie Mellon : Cascadilla Press. [find](https://scholar.google.com/scholar?q=G+EERTZEN+J%2E%2C+A+LEXOPOULOU+T%2E+%26+K+ORHONEN+A%2E+%282013%29%2E+Automatic+Linguistic+Annotation+of+Large+Scale+L2+Databases+%3A+The+EF%2DCambridge+Open+Language+Database+%28EFCamDat%29%2E+In+R%2E+T%2E+M+ILLER+%2C+K%2E+I%2E+M+ARTIN+)
7. H AWKINS J. A. & F ILIPOVI Ć L. (2012). Criterial Features in L2 English : Specifying the Reference Levels of the Common European Framework. United Kingdom : Cambridge University Press. [find](https://scholar.google.com/scholar?q=H+AWKINS+J%2E+A%2E+%26+F+ILIPOVI+C%CC%81+L%2E+%282012%29%2E+Criterial+Features+in+L2+English+%3A+Specifying+the+Reference+Levels+of+the+Common+European+Framework%2E+United+Kingdom+%3A+Cambridge+University+Press%2E)
8. H OUSEN A., K UIKEN F. & V EDDER I., Éds. (2012). Dimensions of L2 performance and proficiency : complexity, accuracy and fluency in SLA, volume 32 de Language Learning & Language Teaching (LL&LT). Amsterdam, Pays-Bas, Etats-Unis d’Amérique : John Benjamins Publishing Company. [find](https://scholar.google.com/scholar?q=H+OUSEN+A%2E%2C+K+UIKEN+F%2E+%26+V+EDDER+I%2E%2C+%C3%89ds%2E+%282012%29%2E+Dimensions+of+L2+performance+and+proficiency+%3A+complexity%2C+accuracy+and+fluency+in+SLA%2C+volume+32+de+Language+Learning+%26+Language+Teaching+%28LL%26LT%29%2E+Am)
9. K ELLY R. (2016). PyEnchant a spellchecking library for Python. Accessible sur [https://pythonhosted.org/pyenchant](https://pythonhosted.org/pyenchant). [↗](https://pythonhosted.org/pyenchant)
10. K YLE K., C ROSSLEY S. & B ERGER C. (2018). The tool for the automatic analysis of lexical sophistication (TAALES) : version 2.0. Behavior Research Methods, 50(3), 1030–1046. [find](https://scholar.google.com/scholar?q=K+YLE+K%2E%2C+C+ROSSLEY+S%2E+%26+B+ERGER+C%2E+%282018%29%2E+The+tool+for+the+automatic+analysis+of+lexical+sophistication+%28TAALES%29+%3A+version+2%2E0%2E+Behavior+Research+Methods%2C+50%283%29%2C+1030%E2%80%931046%2E)
11. L U X. (2014). Computational Methods for Corpus Annotation and Analysis. Dordrecht : Springer. [find](https://scholar.google.com/scholar?q=L+U+X%2E+%282014%29%2E+Computational+Methods+for+Corpus+Annotation+and+Analysis%2E+Dordrecht+%3A+Springer%2E)
12. M ERKEL D. (2014). Docker : lightweight linux containers for consistent development and deployment. Linux journal, 2014(239), 2. [find](https://scholar.google.com/scholar?q=M+ERKEL+D%2E+%282014%29%2E+Docker+%3A+lightweight+linux+containers+for+consistent+development+and+deployment%2E+Linux+journal%2C+2014%28239%29%2C+2%2E)
13. S OUSA A., B ALLIER N., G AILLAT T., S TEARNS B., Z ARROUK M., S IMPKIN A. & B OUYÉ M. (2020). From Linguistic Research Projects to Language Technology Platforms : A Case Study in learner data. In Proceedings of the 1st International Workshop on Language Technology Platforms IWLTP 2020, Marseille : LREC / ACL. [find](https://scholar.google.com/scholar?q=S+OUSA+A%2E%2C+B+ALLIER+N%2E%2C+G+AILLAT+T%2E%2C+S+TEARNS+B%2E%2C+Z+ARROUK+M%2E%2C+S+IMPKIN+A%2E+%26+B+OUY%C3%89+M%2E+%282020%29%2E+From+Linguistic+Research+Projects+to+Language+Technology+Platforms+%3A+A+Case+Study+in+learner+data%2E+In+Pro)
14. Z HANG A. (2017). Speech recognition (version 3.8) python library. Accessible sur [https://pypi.org/project/SpeechRecognition/](https://pypi.org/project/SpeechRecognition/). [↗](https://pypi.org/project/SpeechRecognition/)

*Updated 2 November 2026*