---
title: "Assessing the validity of new paradigmatic complexity measures"
description: "The study asks whether learner proficiency in written L2 English can be read from how learners choose among competing… 46 references."
authors: ["Cyriel Mallart", "Andrew Simpkin", "Nicolas Ballier )", "Paula Lisson", "Remi Venant", "Jen-Yu Li", "Bernardo Stearns", "Thomas Gaillat"]
year: 2025
venue: "arXiv:2503.10220"
source: "https://arxiv.org/abs/2503.10220"
doi: "10.48550/arxiv.2503.10220"
arxiv: "2503.10220"
licence: "arXiv — non-exclusive license to distribute"
licence_url: "http://arxiv.org/licenses/nonexclusive-distrib/1.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/mallart-2025-assessing-validity-new-paradigmatic-complexity/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/mallart-2025-assessing-validity-new-paradigmatic-complexity/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2025](https://md.research.elamur.ai/free/en/papers/2025/page/17/) / Assessing the validity of new paradigmatic complexity measures as criterial features for proficiency in L2 writings in English

# Assessing the validity of new paradigmatic complexity measures as criterial features for proficiency in L2 writings in English

**Authors:** [Cyriel Mallart](https://md.research.elamur.ai/free/en/authors/cyriel-mallart/), [Andrew Simpkin](https://md.research.elamur.ai/free/en/authors/andrew-simpkin/), [Nicolas Ballier )](https://md.research.elamur.ai/free/en/authors/nicolas-ballier/), [Paula Lisson](https://md.research.elamur.ai/free/en/authors/paula-lisson/), [Remi Venant](https://md.research.elamur.ai/free/en/authors/remi-venant/), [Jen-Yu Li](https://md.research.elamur.ai/free/en/authors/jen-yu-li/), [Bernardo Stearns](https://md.research.elamur.ai/free/en/authors/bernardo-stearns/) and [Thomas Gaillat](https://md.research.elamur.ai/free/en/authors/thomas-gaillat/) · **Year:** 2025 · **Venue:** arXiv:2503.10220 · **Licence:** [arXiv — non-exclusive license to distribute](http://arxiv.org/licenses/nonexclusive-distrib/1.0/)

**Links:** [Publisher's page](https://arxiv.org/abs/2503.10220) · [PDF](https://arxiv.org/pdf/2503.10220) · [DOI 10.48550/arxiv.2503.10220](https://doi.org/10.48550/arxiv.2503.10220) · [arXiv 2503.10220](https://arxiv.org/abs/2503.10220) · [OpenAlex](https://openalex.org/W4415261768)

**Citations:** 0 citations (OpenAlex, 2026-10-14)

## Summary
The study asks whether learner proficiency in written L2 English can be read from how learners choose among competing forms that serve one function, rather than from global "omnibus" complexity scores that are hard to interpret. The authors call such groups of competing forms *microsystems* (MS), for example the proforms it/this/that. An earlier version of the idea used raw proportions of each form. That approach can be gamed: a text that uses only the form linked to higher levels would score top. The new version instead estimates, with generalized linear models such as logistic regression, how likely each form is compared with its competitors given its local context. Seven MS models were trained on EFCAMDAT and tested on an internal split and on an external corpus of French learners. Ordinal regression then related the MS probabilities to CEFR levels. Each MS was significant on its own but contributed little. Taken together, they had a substantial effect. This makes them interpretable features that CALL systems could use for proficiency assessment and for explaining feedback.

## Key points
- Seven microsystems were operationalised. The examples focus on nominal structures and determination, such as it/this/that and a/the/zero article.
- EFCAMDAT served as the gold-standard training data. A corpus of French learners of English was the external test set.
- Evaluation used ordinal regression against CEFR levels. All MS were significant, with low individual impact but strong combined influence.
- A task-effect check on IT-THAT-THIS found 14 A1 outliers with normalised frequency above 10. Nine came from one EnglishTown prompt (Topic ID 2), a task about listing the items found in an office, where learners wrote "That are…" in place of "there are". Three clothing tasks with pictures (Topic IDs 7, 12, 21) also seemed to elicit THAT.
- Modals were not implemented as an MS because the UD scheme gives no way to query the epistemic vs root distinction.

## Interesting ideas
- Score a choice by its probability against a closed set of competitors in context, not by how often it occurs, so that overusing one "good" form is not rewarded.
- Define a paradigm as a small set of forms competing for one function, and extract it with treebank queries over UD trees, keeping every feature tied to a named linguistic choice.
- Treat a feature that is weak alone but strong in a group as normal, and test MS jointly rather than discarding them one by one.
- Inspect outliers by the prompt that produced them, as the authors did when one office-inventory task explained most A1 anomalies, to separate task effects from real developmental signal.
- Note where the annotation scheme cannot express a needed distinction, as UD cannot for epistemic vs root modals, and mark that as the place where a semantic layer is required.

## For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

- **I1** (answer): Graders of student answers could judge each determiner or pronoun choice against its alternatives in context instead of counting forms, which also gives an explainable reason for each judgement.
- **I2** (tree): Mova could store "microsystem" queries over its UD output (proforms, articles, dative alternation) as explainable diagnostics of parser and text behaviour.
- **I3** (learn): When evaluating self-induced repair rules or witness-parser signals, assess them as bundles with negative controls, so that individually small but jointly useful cues are not discarded.
- **I4** (learn): Silver data and test results could be grouped by source text or task, so that prompt-driven artefacts are not mistaken for parser or reader errors.
- **I5** (frames): The prag crate could supply the modal-sense distinction that UD lacks and pass it down as an annotated feature, making modal-based microsystems queryable.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Laboratoire d’Informatique de l’Université du Mans](https://md.research.elamur.ai/free/en/labs/laboratoire-d-informatique-de-l-universite-du-mans/) — Research institute, Le Mans, France · 13 works · 2,995 citations
- [Ollscoil na Gaillimhe – University of Galway](https://md.research.elamur.ai/free/en/universities/ollscoil-na-gaillimhe-university-of-galway/) — University, Galway, Ireland · 25 works · 2,751 citations
- [Université Paris Cité](https://md.research.elamur.ai/free/en/universities/universite-paris-cite/) — University, Paris, France · 20 works · 1,426 citations
- [Approches Linguistiques Théoriques, Appliquées et Expérimentales : langues et cultures connectées](https://md.research.elamur.ai/free/en/labs/approches-linguistiques-theoriques-appliquees-et-experimentales-langues-et/) — Research institute, Paris, France · 10 works · 260 citations
- [Linguistique Ingénierie et Didactique des Langues](https://md.research.elamur.ai/free/en/labs/linguistique-ingenierie-et-didactique-des-langues/) — Research institute, Rennes, France · 5 works · 14 citations
- [Université Rennes 2](https://md.research.elamur.ai/free/en/universities/universite-rennes-2/) — University, Rennes, France · 4 works · 9 citations

Also on the byline: [Universidade Federal de Rondônia](https://ror.org/02842cb31) · [Universidad Internacional de La Rioja](https://ror.org/029gnnp81) · [Laboratoire d'Électronique et d'Electromagnétisme](https://ror.org/02v8v7830).

📚 Shelves (6): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 18,614 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 17,965 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 17,450 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 17,296 works · [💭 Psychology](https://md.research.elamur.ai/free/en/shelves/psychology/) · 1,649 works · [💭 Developmental and Educational Psychology](https://md.research.elamur.ai/free/en/shelves/psychology/developmental-and-educational-psychology/) · 523 works

🧭 Topics (1): [English](https://md.research.elamur.ai/free/en/topics/english/) · 2,379 works

## Cite

Cite as: Cyriel Mallart, Andrew Simpkin, Nicolas Ballier ), Paula Lisson, Remi Venant, Jen-Yu Li, Bernardo Stearns and Thomas Gaillat. 2025. Assessing the validity of new paradigmatic complexity measures as criterial features for proficiency in L2 writings in English. arXiv:2503.10220.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@misc{mallart2025assessing,
  title = {{Assessing the validity of new paradigmatic complexity measures as criterial features for proficiency in L2 writings in English}},
  author = {Cyriel Mallart, Andrew Simpkin, Nicolas Ballier ), Paula Lisson, Remi Venant, Jen-Yu Li, Bernardo Stearns and Thomas Gaillat},
  year = {2025},
  doi = {10.48550/arxiv.2503.10220},
  eprint = {2503.10220},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2503.10220}
}
```

- [← previous Assessing the Agreement Competence of Large Language Models](https://md.research.elamur.ai/free/en/papers/garcia-2025-assessing-agreement-competence-large-language/)
- [next → AtlasIA at SemEval-2025 Task 11: FastText-Based Emotion Detection in Moroccan Arabic for Low-Resource Settings](https://md.research.elamur.ai/free/en/papers/majjodi-2025-atlasia-semeval-2025-task-11/)
- [↑ back to papers of 2025](https://md.research.elamur.ai/free/en/papers/2025/page/17/)

## References

46 references, 4 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Alexopoulou, Theodora, Marije Michel, Akira Murakami, and Detmar Meurers. 2017. “Task effects on linguistic complexity and accuracy: A large-scale learner corpus analysis employing natural language processing techniques.” Language Learning 67 (S1): 180–208. [↗](https://doi.org/10.1111/lang.12232)
2. Artstein, Ron, and Massimo Poesio. 2008. [“Inter-coder Agreement for Computational Linguistics.” Comput](https://md.research.elamur.ai/free/en/papers/artstein-2008-survey-article-inter-coder-agreement/). Linguist. 34 (4): 555–596. [↗](https://doi.org/10.1162/coli.07-034-r2)
3. Biber, Douglas, and Bethany Gray. 2011. “Grammatical change in the noun phrase: the influence of written language use.” English Language & Linguistics 15 (2): 223–250. Publisher: Cambridge University Press. [↗](https://doi.org/10.1017/s1360674311000025)
4. Biber, Douglas, Bethany Gray, and Kornwipa Poonpon. 2011. “Should We Use Characteristics of Conversation to Measure Grammatical Complexity in L2 Writing Development?” TESOL Quarterly 45 (1): 5–35. [↗](https://doi.org/10.5054/tq.2011.244483)
5. Biber, Douglas, Bethany Gray, Shelley Staples, and Jesse Egbert. 2020. “Investigating grammatical complexity in L2 English writing research: Linguistic description versus predictive measurement.” Journal of English for Academic Purposes 46: 100869. [↗](https://doi.org/10.1016/j.jeap.2020.100869)
6. Biber, Douglas, Tove Larsson, and Gregory R. Hancock. 2023. “The linguistic organization of grammatical text complexity: comparing the empirical adequacy of theory-based models.” Corpus Linguistics and Linguistic Theory Publisher: De Gruyter Mouton. [↗](https://doi.org/10.1515/cllt-2023-0016)
7. Biber, Douglas, Benedikt Szmrecsanyi, Randi Reppen, and Tove Larsson. 2023. “Expanding the scope of grammatical variation: towards a comprehensive account of genitive variation across registers.” English Language & Linguistics 1–39. Cambridge University Press. [↗](https://doi.org/10.1017/s1360674323000497)
8. Bresnan, Joan, Anna Cueni, Tatiana Nikitina, and R. Harald Baayen. 2007. “Predicting the dative alternation.” In Cognitive foundations of interpretation, edited by Bouma Gerlof, Irene Kramer, and Joost Swarts, 69–94. Amsterdam: Royal Netherlands Academy of Arts and Sciences. [find](https://scholar.google.com/scholar?q=Bresnan%2C+Joan%2C+Anna+Cueni%2C+Tatiana+Nikitina%2C+and+R%2E+Harald+Baayen%2E+2007%2E+%E2%80%9CPredicting+the+dative+alternation%2E%E2%80%9D+In+Cognitive+foundations+of+interpretation%2C+edited+by+Bouma+Gerlof%2C+Irene+Kramer%2C+and+Joos)
9. Bult, Bram, and Alex Housen. 2012. Defining and Operationalising L2 Complexity. John Benjamins Publishing Company. [find](https://scholar.google.com/scholar?q=Bult%2C+Bram%2C+and+Alex+Housen%2E+2012%2E+Defining+and+Operationalising+L2+Complexity%2E+John+Benjamins+Publishing+Company%2E)
10. Council of Europe. 2018. Common European Framework of Reference for Languages: Learning, teaching, assessment: Companion volume with new descriptors. Strasbourg: Council of Europe. [find](https://scholar.google.com/scholar?q=Council+of+Europe%2E+2018%2E+Common+European+Framework+of+Reference+for+Languages%3A+Learning%2C+teaching%2C+assessment%3A+Companion+volume+with+new+descriptors%2E+Strasbourg%3A+Council+of+Europe%2E)
11. Crossley, Scott A., Kristopher Kyle, and Mihai Dascalu. 2019. “The Tool for the Automatic Analysis of Cohesion 2.0: Integrating semantic similarity and text overlap.” Behavior Research Methods 51 (1): 14–27. [↗](https://doi.org/10.3758/s13428-018-1142-4)
12. Cushing Weigle, Sara. 2013. “English language learners and automated scoring of essays: Critical considerations.” Assessing Writing 18 (1): 85–99. [↗](https://doi.org/10.1016/j.asw.2012.10.006)
13. Cushing Weigle, Sara. 2017. “Corpus linguistics in language testing research.” Language Testing 34 (4): 441–449. Publisher: SAGE Publications Ltd. [↗](https://doi.org/10.1177/0265532217713044)
14. de Marneffe, Marie-Catherine, Christopher D. Manning, Joakim Nivre, and Daniel Zeman. 2021. “[Universal Dependencies](https://md.research.elamur.ai/free/en/papers/marneffe-2021-universal-dependencies-2/).” Computational Linguistics 47 (2): 255–308. Place: Cambridge, MA Publisher: MIT Press.
15. Depraetere, Ilse, and Chad Langford. 2012. Advanced English grammar: a linguistic approach. London: Continuum. [find](https://scholar.google.com/scholar?q=Depraetere%2C+Ilse%2C+and+Chad+Langford%2E+2012%2E+Advanced+English+grammar%3A+a+linguistic+approach%2E+London%3A+Continuum%2E)
16. Ellis, Rod. 1994. The Study of Second Language Acquisition. Oxford, United Kingdom: Oxford University Press. [find](https://scholar.google.com/scholar?q=Ellis%2C+Rod%2E+1994%2E+The+Study+of+Second+Language+Acquisition%2E+Oxford%2C+United+Kingdom%3A+Oxford+University+Press%2E)
17. Fleiss, Joseph L., Bruce Levin, and Myunghee Cho Paik. 2003. “The Measurement of Interrater Agreement.” In Statistical Methods for Rates and Proportions, 598–626. John Wiley & Sons, Ltd. [↗](https://doi.org/10.1002/0471445428.ch18)
18. Gaillat, Thomas, Andrew Simpkin, Nicolas Ballier, Bernardo Stearns, Annanda Sousa, Manon Bouy, and Manel Zarrouk. 2022. “Predicting CEFR levels in learners of English: The use of microsystem criterial features in a machine learning approach.” ReCALL 34 (2): 130–146. Publisher: Cambridge University Press. [↗](https://doi.org/10.1017/s095834402100029x)
19. Geertzen, Jeroen, Theodora Alexopoulou, and Anna Korhonen. 2013. “Automatic Linguistic Annotation of Large Scale L2 Databases: The EF-Cambridge Open Language Database (EFCamDat).” In Proceedings of the 31st Second Language Research Forum, edited by R. T. Miller, K. I. Martin, C. M. Eddington, A. Henery, N. Miguel, A. Tseng, A. Tuninetti, and D Walter, Carnegie Mellon. Cascadilla Press. [find](https://scholar.google.com/scholar?q=Geertzen%2C+Jeroen%2C+Theodora+Alexopoulou%2C+and+Anna+Korhonen%2E+2013%2E+%E2%80%9CAutomatic+Linguistic+Annotation+of+Large+Scale+L2+Databases%3A+The+EF%2DCambridge+Open+Language+Database+%28EFCamDat%29%2E%E2%80%9D+In+Proceedings+of+th)
20. Gentilhomme, Yves. 1980. “Microsystmes et acquisition des langues.” Encrages (Numro spcial): 79–84. [find](https://scholar.google.com/scholar?q=Gentilhomme%2C+Yves%2E+1980%2E+%E2%80%9CMicrosystmes+et+acquisition+des+langues%2E%E2%80%9D+Encrages+%28Numro+spcial%29%3A+79%E2%80%9384%2E)
21. Gries, Stefan Th. 2022. “Two (more) suggestions for improvement: MuPDAR for corpusbased learner and variety studies.” In Broadening the Spectrum of Corpus Linguistics: New approaches to variability and change, edited by Susanne Flach and Martin Hilpert, Studies in Corpus Linguistics, 256–283. John Benjamins Publishing Company. [find](https://scholar.google.com/scholar?q=Gries%2C+Stefan+Th%2E+2022%2E+%E2%80%9CTwo+%28more%29+suggestions+for+improvement%3A+MuPDAR+for+corpusbased+learner+and+variety+studies%2E%E2%80%9D+In+Broadening+the+Spectrum+of+Corpus+Linguistics%3A+New+approaches+to+variability+an)
22. Gries, Stefan Th, Santa Barbara, Justus Liebig, and Sandra C. Deshors. 2020. “Theres more to alternations than the main diagonal of a 22 confusion matrix: Improvements of MuPDAR and other classificatory alternation studies.” ICAME Journal 44 (1): 69–96. [find](https://scholar.google.com/scholar?q=Gries%2C+Stefan+Th%2C+Santa+Barbara%2C+Justus+Liebig%2C+and+Sandra+C%2E+Deshors%2E+2020%2E+%E2%80%9CTheres+more+to+alternations+than+the+main+diagonal+of+a+22+confusion+matrix%3A+Improvements+of+MuPDAR+and+other+classificato)
23. Guillaume, Bruno. 2021. “Graph Matching and Graph Rewriting: GREW tools for corpus exploration, maintenance and conversion.” In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: System Demonstrations, 168–175. [find](https://scholar.google.com/scholar?q=Guillaume%2C+Bruno%2E+2021%2E+%E2%80%9CGraph+Matching+and+Graph+Rewriting%3A+GREW+tools+for+corpus+exploration%2C+maintenance+and+conversion%2E%E2%80%9D+In+Proceedings+of+the+16th+Conference+of+the+European+Chapter+of+the+Associ)
24. Halliday, M. A. K., and Christian M. I. M. Matthiessen. 2004. An introduction to functional grammar. 3rd ed. London: Hodder Arnold. [find](https://scholar.google.com/scholar?q=Halliday%2C+M%2E+A%2E+K%2E%2C+and+Christian+M%2E+I%2E+M%2E+Matthiessen%2E+2004%2E+An+introduction+to+functional+grammar%2E+3rd+ed%2E+London%3A+Hodder+Arnold%2E)
25. Hawkins, John A, and Luna Filipovi. 2012. Criterial features in L2 English: specifying the reference levels of the Common European Framework. Cambridge: Cambridge University Press. [find](https://scholar.google.com/scholar?q=Hawkins%2C+John+A%2C+and+Luna+Filipovi%2E+2012%2E+Criterial+features+in+L2+English%3A+specifying+the+reference+levels+of+the+Common+European+Framework%2E+Cambridge%3A+Cambridge+University+Press%2E)
26. Knoch, Ute. 2011. “Rating scales for diagnostic assessment of writing: What should they look like and where should the criteria come from?” Assessing Writing 16 (2): 81–96. https://doi.org/10.1016/j.asw.2011.02.003. [↗](https://doi.org/10.1016/j.asw.2011.02.003)
27. Kyle, Kristopher. 2016. “Measuring syntactic development in L2 writing: Fine grained indices of syntactic complexity and usage-based indices of syntactic sophistication.” Dissertation, Georgia State University, Georgia. [find](https://scholar.google.com/scholar?q=Kyle%2C+Kristopher%2E+2016%2E+%E2%80%9CMeasuring+syntactic+development+in+L2+writing%3A+Fine+grained+indices+of+syntactic+complexity+and+usage%2Dbased+indices+of+syntactic+sophistication%2E%E2%80%9D+Dissertation%2C+Georgia+State+U)
28. Lan, Ge, Kyle Lucas, and Yachao Sun. 2019. “Does L2 writing proficiency influence noun phrase complexity? A case analysis of argumentative essays written by Chinese students in a first-year composition course.” System 85: 102116. [↗](https://doi.org/10.1016/j.system.2019.102116)
29. Larsson, Tove, Tony Berber Sardinha, Bethany Gray, and Douglas Biber. 2023. “Exploring early L2 writing development through the lens of grammatical complexity.” Applied Corpus Linguistics 3 (3): 100077. [↗](https://doi.org/10.1016/j.acorp.2023.100077)
30. Larsson, Tove, M. Paquot, and Luke D. Plonsky. 2020. “Inter-rater reliability in Learner Corpus Research: Insights from a collaborative study on adverb placement.” . [↗](https://doi.org/10.1075/ijlcr.20001.lar)
31. Lu, Xiaofei. 2014. Computational Methods for Corpus Annotation and Analysis. Dordrecht: Springer. [↗](https://doi.org/10.1007/978-94-017-8645-4)
32. Mallart, Cyriel, Andrew Simpkin, Rmi Venant, Nicolas Ballier, Bernardo Stearns, Jen Yu Li, and Thomas Gaillat. 2023. “[A new learner language data set for the study of English for Specific Purposes at university](https://md.research.elamur.ai/free/en/papers/mallart-2023-new-learner-language-data-set/) level.” In Proceedings of the 4th Conference on Language, Data and Knowledge - LDK 2023, Vol. 1, Vienna, Austria, September, 281–287.
33. Michel, Marije, Akira Murakami, Theodora Alexopoulou, and Detmar Meurers. 2019. “Effects of task type on morphosyntactic complexity across proficiency: Evidence from a large learner corpus of A1 to C2 writings.” Journal of Instructed Second Language Acquisition 3 (2): 124– 152. [find](https://scholar.google.com/scholar?q=Michel%2C+Marije%2C+Akira+Murakami%2C+Theodora+Alexopoulou%2C+and+Detmar+Meurers%2E+2019%2E+%E2%80%9CEffects+of+task+type+on+morphosyntactic+complexity+across+proficiency%3A+Evidence+from+a+large+learner+corpus+of+A1+to+C2)
34. Norris, John, and Lourdes Ortega. 2008. “Defining and Measuring SLA.” In The Handbook of Second Language Acquisition, edited by Catherine J. Doughty and Michael H. Long, 716–761. Oxford: John Wiley & Sons, Ltd. [find](https://scholar.google.com/scholar?q=Norris%2C+John%2C+and+Lourdes+Ortega%2E+2008%2E+%E2%80%9CDefining+and+Measuring+SLA%2E%E2%80%9D+In+The+Handbook+of+Second+Language+Acquisition%2C+edited+by+Catherine+J%2E+Doughty+and+Michael+H%2E+Long%2C+716%E2%80%93761%2E+Oxford%3A+John+Wiley+%26+)
35. OKeeffe, Anne, and Geraldine Mark. 2017. “The English Grammar Profile of learner competence: Methodology and key findings.” International Journal of Corpus Linguistics 22 (4): 457–489. Publisher: John Benjamins. [↗](https://doi.org/10.1075/ijcl.14086.oke)
36. Page, Ellis B. 1968. “The Use of the Computer in Analyzing Student Essays.” International Review of Education / Internationale Zeitschrift fr Erziehungswissenschaft / Revue Internationale de l’Education 14 (2): 210–225. [↗](https://doi.org/10.1007/bf01419938)
37. Piln, Ildik. 2018. Automatic proficiency level prediction for Intelligent Computer-Assisted Language Learning. Gothenburg, Sweden: Gothenburg University Publications Electronic Archive. [find](https://scholar.google.com/scholar?q=Piln%2C+Ildik%2E+2018%2E+Automatic+proficiency+level+prediction+for+Intelligent+Computer%2DAssisted+Language+Learning%2E+Gothenburg%2C+Sweden%3A+Gothenburg+University+Publications+Electronic+Archive%2E)
38. Piln, Ildik, Elena Volodina, and Torsten Zesch. 2016. “Predicting proficiency levels in learner writings by transferring a linguistic complexity model from expert-written coursebooks.” In Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers, Osaka, Japan, December, 2101–2111. [find](https://scholar.google.com/scholar?q=Piln%2C+Ildik%2C+Elena+Volodina%2C+and+Torsten+Zesch%2E+2016%2E+%E2%80%9CPredicting+proficiency+levels+in+learner+writings+by+transferring+a+linguistic+complexity+model+from+expert%2Dwritten+coursebooks%2E%E2%80%9D+In+Proceedings+)
39. The COLING 2016 Organizing Committee. [find](https://scholar.google.com/scholar?q=The+COLING+2016+Organizing+Committee%2E)
40. Py, Bernard. 1980. “Quelques rflexions sur la notion d’interlangue.” Revue Tranel (Travaux neuchtelois de linguistique) 1: 31–54. [find](https://scholar.google.com/scholar?q=Py%2C+Bernard%2E+1980%2E+%E2%80%9CQuelques+rflexions+sur+la+notion+d%E2%80%99interlangue%2E%E2%80%9D+Revue+Tranel+%28Travaux+neuchtelois+de+linguistique%29+1%3A+31%E2%80%9354%2E)
41. Safari, Faeze, and Alireza Ahmadi. 2023. “Developing and evaluating an empirically-based diagnostic checklist for assessing second language integrated writing.” Journal of Second Language Writing 60: 101007. [↗](https://doi.org/10.1016/j.jslw.2023.101007)
42. Shatz, Itamar. 2020. “Refining and modifying the EFCAMDAT: Lessons from creating a new corpus from an existing large-scale English learner language database.” International Journal of Learner Corpus Research 6 (2): 220–236. Publisher: John Benjamins. [find](https://scholar.google.com/scholar?q=Shatz%2C+Itamar%2E+2020%2E+%E2%80%9CRefining+and+modifying+the+EFCAMDAT%3A+Lessons+from+creating+a+new+corpus+from+an+existing+large%2Dscale+English+learner+language+database%2E%E2%80%9D+International+Journal+of+Learner+Corpus+R)
43. Staples, Shelley, Jesse Egbert, Douglas Biber, and Bethany Gray. 2016. “Academic Writing Development at the University Level: Phrasal and Clausal Complexity Across Level of Study, Discipline, and Genre.” Written Communication 33 (2): 149–183. Publisher: SAGE Publications Inc. [find](https://scholar.google.com/scholar?q=Staples%2C+Shelley%2C+Jesse+Egbert%2C+Douglas+Biber%2C+and+Bethany+Gray%2E+2016%2E+%E2%80%9CAcademic+Writing+Development+at+the+University+Level%3A+Phrasal+and+Clausal+Complexity+Across+Level+of+Study%2C+Discipline%2C+and+Genr)
44. Staples, Shelley, Bethany Gray, Douglas Biber, and Jesse Egbert. 2022. “Writing Trajectories of Grammatical Complexity at the University: Comparing L1 and L2 English Writers in BAWE.” Applied Linguistics amac047. [↗](https://doi.org/10.1093/applin/amac047)
45. Straka, Milan, Jan Haji, and Jana Strakov. 2016. “[UDPipe: Trainable Pipeline for Processing CoNLL-U Files Performing Tokenization, Morphological Analysis, POS Tagging and Parsing](https://md.research.elamur.ai/free/en/papers/straka-2016-udpipe-trainable-pipeline-processing-conll/).” In Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC’16), Portoro, Slovenia, May, 4290–4297. European Language Resources Association (ELRA).
46. Wolfe-Quintero, Kate, Shunji Inagaki, and Hae-Young Kim. 1998. Second language development in writing: measures of fluency, accuracy, & complexity. Honolulu: Second Language Teaching & Curriculum Center, University of Hawaii at Manoa. OCLC: 40664312. 7. Appendices [find](https://scholar.google.com/scholar?q=Wolfe%2DQuintero%2C+Kate%2C+Shunji+Inagaki%2C+and+Hae%2DYoung+Kim%2E+1998%2E+Second+language+development+in+writing%3A+measures+of+fluency%2C+accuracy%2C+%26+complexity%2E+Honolulu%3A+Second+Language+Teaching+%26+Curriculum+Cent)

*Updated 1 November 2026*