--- title: "A Corpus of Spontaneous L2 English Speech for Real-situation" description: "In the exam, 260 students argue opposing sides of a contested topic and negotiate a compromise, and… 17 references, cited by 3 papers here." authors: ["Sylvain Coulange", "Marie-Hélène Fries", "Monica Masperi", "Solange Rossato"] year: 2024 venue: "Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024)" source: "https://aclanthology.org/2024.lrec-main.27/" licence: "CC BY-NC 4.0 (ELRA, LREC proceedings)" licence_url: "https://creativecommons.org/licenses/by-nc/4.0/" summary_licence: "Elamur Cosmic License" canonical: "https://research.elamur.ai/free/en/papers/coulange-2024-corpus-spontaneous-l2-english-speech/" --- [Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/coulange-2024-corpus-spontaneous-l2-english-speech/) [home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2024](https://md.research.elamur.ai/free/en/papers/2024/page/14/) / A Corpus of Spontaneous L2 English Speech for Real-situation Speaking Assessment # A Corpus of Spontaneous L2 English Speech for Real-situation Speaking Assessment **Authors:** Sylvain Coulange, Marie-Hélène Fries, Monica Masperi and Solange Rossato · **Year:** 2024 · **Venue:** [Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024)](https://md.research.elamur.ai/free/en/venues/lrec/) ([LREC 2024](https://md.research.elamur.ai/free/en/venues/lrec/2024/)) · **Publisher:** [European Language Resources Association (ELRA)](https://md.research.elamur.ai/free/en/publishers/european-language-resources-association-elra/) · **Licence:** [CC BY-NC 4.0 (ELRA, LREC proceedings)](https://creativecommons.org/licenses/by-nc/4.0/) **Links:** [Publisher's page](https://aclanthology.org/2024.lrec-main.27/) · [PDF](https://aclanthology.org/2024.lrec-main.27.pdf) · [OpenAlex](https://openalex.org/W4402426972) **Citations:** 0 citations (OpenAlex, 2026-10-14) · [3](https://md.research.elamur.ai/free/en/papers/coulange-2024-corpus-spontaneous-l2-english-speech/cited-by/) in our library, of the 149,439 papers whose bibliographies we read ## Summary Automated speaking-assessment tools are usually built on tightly controlled tasks, such as reading words or sentences aloud, so they say little about how learners perform in real conversation. The authors address this with a 20-hour corpus of spontaneous L2 English recorded during CLES certification exams at Université Grenoble Alpes. In the exam, 260 students argue opposing sides of a contested topic and negotiate a compromise, and experts rate each one as B2, B1 or non-validated. Most of the corpus is publicly released. As a first use of the data, the authors built an open, interpretable pipeline: diarization, then ASR with word alignment, then syllable-nucleus detection, then speaker-normalised prosody. They used it to check whether 176 French-L1 speakers put primary stress where the CMU dictionary expects it. Only about a third of the words carried stress on the right syllable, and learners tended to shift prominence to the last syllable, a pattern consistent with French. B2 speakers did slightly better than B1 speakers. The work matters because it supplies real-situation data and transparent rhythm measures that current scoring systems rarely include. ## Key points - **Corpus scope:** 260 speakers in 134 recording groups, 20 hours in total. - Mostly 2-speaker B2-level role-plays of about 10 minutes each. - A few 3-speaker conversations and 13 B1-level monologues. - 83% of speakers have French as L1. - The public portion (128 students) is available at hdl.handle.net/11403/cles-spontaneous-english. - **Level split:** 58% B2, 29% B1, 13% non-validated. - B2 raters score eight dimensions. - Failing any one dimension drops the candidate to B1 or no validation. - **Pipeline:** - Pyannote for diarization. - WhisperX for transcription and word alignment. - SpaCy for POS tagging. - A Praat syllable-nuclei script with a 300–3300 Hz bandpass filter. - CMU Pronouncing Dictionary for expected stress. - The pipeline is released as open source (plspp). - **Main result:** 35.4% of 6,350 target words were stressed correctly. - Per-speaker accuracy ranged from 0% to 68.4%. - Median accuracy was 36% for B2 and 29.6% for B1 (p \< .0001). - Accuracy above 50% was reached by 22% of B2 speakers but only 3% of B1 speakers. - **How prominence is made:** accurate speakers marked stress with F0 and intensity, while weak ones lengthened final syllables. - Stressed–unstressed contrast correlates with accuracy (R = .82). - A manual check of 100 random words found 92 correctly recognised and 95 correctly aligned. ## Interesting ideas - **Self-consistency filter:** a word was kept only if the dictionary's syllable count matched the number of nuclei detected inside its aligned boundaries, so misaligned items are dropped without any gold annotation. This is a cheap internal check that guards automatic pipelines against garbage input. - **Per-speaker percentiles:** every prosodic value was converted to a percentile within that speaker's own distribution, which removes differences in speech rate, voice and microphone. Normalising each source against itself makes heterogeneous inputs comparable. - **Expected vs. observed diagnosis:** the authors deliberately avoided a trained black-box classifier. They compared what each speaker produced against a reference lexicon pattern and read off how errors arise (duration vs. F0/intensity), which explains a systematic L1-driven shift rather than just scoring it. - **Small random audit:** pipeline quality was estimated by hand-checking 100 randomly sampled words, separately for recognition and for alignment. Such a cheap per-stage audit can show which step is the weak link. - **Pooling cues with equal weight:** stress was decided from F0, intensity and duration combined with equal weight, and each cue was also inspected separately. This keeps the aggregate decision simple while still allowing per-cue explanation. ## For Mova How these ideas could serve Mova, our small open language model — its files, stages and tests: - **I1** (learn): when building silver data, keep only sentences where independent signals agree (for example, token counts or a tagger's POS against the parser's head structure), and reject the rest instead of training on them. - **I2** (learn): express parser confidence or feature scores as percentiles within each witness parser or each document genre, so that thresholds for agreement do not depend on one parser's scale. - **I3** (answer): grade student or learner answers by aligning the produced frame against the expected frame and reporting which role or relation diverged, giving an explainable diagnosis instead of a single score. - **I4** (learn): add per-stage random audits (for example, 100 tokens for tagging, 100 arcs for attachment, 100 frames for roles) to locate where errors enter the text → answer chain. - **I5** (frames): combine evidence for a speech-act or irony decision from several interpretable cues with transparent weights, and expose each cue's contribution in the explanation. *Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.* ## Institutions - [Université Grenoble Alpes](https://md.research.elamur.ai/free/en/universities/universite-grenoble-alpes/) — University, Saint-Martin-d'Hères, France · 24 works · 8,259 citations - [Laboratoire d'Informatique de Grenoble](https://md.research.elamur.ai/free/en/labs/laboratoire-d-informatique-de-grenoble/) — Research institute, Grenoble, France · 24 works · 689 citations - [Laboratoire de Linguistique et Didactique des Langues Etrangères et Maternelles](https://md.research.elamur.ai/free/en/labs/laboratoire-de-linguistique-et-didactique-des-langues-etrangeres-et-maternelles/) — Research institute, Grenoble, France · 7 works · 113 citations - [Institut des langues et cultures d'Europe, Amérique, Afrique, Asie et Australie](https://md.research.elamur.ai/free/en/labs/institut-des-langues-et-cultures-d-europe-amerique-afrique-asie-et-australie/) — Research institute, Grenoble, France · 2 works · 7 citations 📚 Shelves (4): [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 17,450 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 17,296 works · [💭 Psychology](https://md.research.elamur.ai/free/en/shelves/psychology/) · 1,649 works · [💭 Developmental and Educational Psychology](https://md.research.elamur.ai/free/en/shelves/psychology/developmental-and-educational-psychology/) · 523 works 🧭 Topics (1): [English](https://md.research.elamur.ai/free/en/topics/english/) · 2,379 works ## Cite Cite as: Sylvain Coulange, Marie-Hélène Fries, Monica Masperi and Solange Rossato. 2024. A Corpus of Spontaneous L2 English Speech for Real-situation Speaking Assessment. Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024). The paper itself, not this page. BibTeX from the library's record of the work. ```bibtex @inproceedings{coulange2024corpus, title = {{A Corpus of Spontaneous L2 English Speech for Real-situation Speaking Assessment}}, author = {Sylvain Coulange, Marie-Hélène Fries, Monica Masperi and Solange Rossato}, booktitle = {Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024)}, year = {2024}, doi = {10.63317/3dg43n4y98yx}, url = {https://doi.org/10.63317/3dg43n4y98yx} } ``` - [← previous A Canonical Form for Flexible Multiword Expressions](https://md.research.elamur.ai/free/en/papers/odijk-2024-canonical-form-flexible-multiword-expressions/) - [next → A Dataset for Metaphor Detection in Early Medieval Hebrew Poetry](https://md.research.elamur.ai/free/en/papers/toker-2024-dataset-metaphor-detection-early-medieval/) - [↑ back to papers of 2024](https://md.research.elamur.ai/free/en/papers/2024/page/14/) ## References 17 references, 0 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet. 1. Corine Astesano. 2001. Rythme et accentuation en français: invariance et variabilité stylistique. Collection Langue & parole. L’Harmattan. [find](https://scholar.google.com/scholar?q=Corine+Astesano%2E+2001%2E+Rythme+et+accentuation+en+fran%C3%A7ais%3A+invariance+et+variabilit%C3%A9+stylistique%2E+Collection+Langue+%26+parole%2E+L%E2%80%99Harmattan%2E) 2. Max Bain, Jaesung Huh, Tengda Han, and Andrew Zisserman. 2023. Whisperx: Time-accurate speech transcription of long-form audio. Interspeech. [find](https://scholar.google.com/scholar?q=Max+Bain%2C+Jaesung+Huh%2C+Tengda+Han%2C+and+Andrew+Zisserman%2E+2023%2E+Whisperx%3A+Time%2Daccurate+speech+transcription+of+long%2Dform+audio%2E+Interspeech%2E) 3. Hervé Bredin and Antoine Laurent. 2021. End-to-end speaker segmentation for overlap-aware resegmentation. In Interspeech. [find](https://scholar.google.com/scholar?q=Herv%C3%A9+Bredin+and+Antoine+Laurent%2E+2021%2E+End%2Dto%2Dend+speaker+segmentation+for+overlap%2Daware+resegmentation%2E+In+Interspeech%2E) 4. Sylvain Coulange. 2023. Computer-aided pronunciation training in 2022: When pedagogy struggles to catch up. In Alice Henderson and Anastazija Kirkova-Naskova, editors, Proc. of the 7th International Conf. on English Pronunciation: Issues and Practices, pages 11–22. [find](https://scholar.google.com/scholar?q=Sylvain+Coulange%2E+2023%2E+Computer%2Daided+pronunciation+training+in+2022%3A+When+pedagogy+struggles+to+catch+up%2E+In+Alice+Henderson+and+Anastazija+Kirkova%2DNaskova%2C+editors%2C+Proc%2E+of+the+7th+International+C) 5. Council of Europe. 2020. Common European framework of reference for languages. Council of Europe, Strasbourg, France. [find](https://scholar.google.com/scholar?q=Council+of+Europe%2E+2020%2E+Common+European+framework+of+reference+for+languages%2E+Council+of+Europe%2C+Strasbourg%2C+France%2E) 6. Anne Cutler. 2015. Lexical stress in english pronunciation. In The Handbook of English Pronunciation, pages 106– 124. John Wiley & Sons, Inc, Hoboken, NJ. [↗](https://doi.org/10.1002/9781118346952.ch6) 7. Nivja H. de Jong, Jos Pacilly, and Willemijn Heeren. 2021. Praat scripts to measure speed fluency and breakdown fluency in speech automatically. Assessment in Education: Principles, Policy & Practice, 28:456–476. [↗](https://doi.org/10.1080/0969594x.2021.1951162) 8. Keelan Evanini and Klaus Zechner. 2019. Overview of automated speech scoring, Innovations in Language Learning and Assessment at ETS, pages 3–20. Routledge, London, England. [find](https://scholar.google.com/scholar?q=Keelan+Evanini+and+Klaus+Zechner%2E+2019%2E+Overview+of+automated+speech+scoring%2C+Innovations+in+Language+Learning+and+Assessment+at+ETS%2C+pages+3%E2%80%9320%2E+Routledge%2C+London%2C+England%2E) 9. Luciana Ferrer, Harry Bratt, Colleen Richey, Horacio Franco, Victor Abrash, and Kristin Precoda. 2015. Classification of lexical stress using spectral and prosodic features for computer-assisted language learning systems. Speech Comm., 69:31–45. [↗](https://doi.org/10.1016/j.specom.2015.02.002) 10. Matthew Honnibal, Ines Montani, Sofie Van Landeghem, and Adriane Boyd. 2020. spacy: Industrial-strength natural language processing in python. [find](https://scholar.google.com/scholar?q=Matthew+Honnibal%2C+Ines+Montani%2C+Sofie+Van+Landeghem%2C+and+Adriane+Boyd%2E+2020%2E+spacy%3A+Industrial%2Dstrength+natural+language+processing+in+python%2E) 11. Talia Isaacs, Pavel Trofimovich, and Jennifer Ann Foote. 2018. Developing a user-oriented second language comprehensibility scale for english-medium universities. Language Testing, 35(2):193–216. [↗](https://doi.org/10.1177/0265532217703433) 12. David O. Johnson and Okim Kang. 2015. Automatic prominent syllable detection with machine learning classifiers. Int. J. Speech Technol., 18(4):583–592. [↗](https://doi.org/10.1007/s10772-015-9299-z) 13. Kun Li, Shaoguang Mao, Xu Li, Zhiyong Wu, and Helen Meng. 2018. Automatic lexical stress and pitch accent detection for L2 English speech using multi-distribution deep neural networks. Speech Comm., 96:28–36. [↗](https://doi.org/10.1016/j.specom.2017.11.003) 14. Kazuya Saito, Konstantinos Macmillan, Magdalena Kach-licka, Takuya Kunihara, and Nobuaki Minematsu. 2022. Automated assessment of second language comprehensibility: Review, training, validation, and generalization studies. Second Lang. Acquis., pages 1–30. [↗](https://doi.org/10.1017/s0272263122000080) 15. Anne Tortel. 2021. Le rythme en anglais oral : considérations théoriques et illustrations sur corpus. Recherche et pratiques pédagogiques en langues - Cahiers de l’APLIUT, (Vol. 40 N°1). [↗](https://doi.org/10.4000/apliut.8857) 16. Anne Tortel and Sophie Herment. 2018. La voyelle inaccentuée e en position initiale : analyses acoustiques et enjeux pédagogiques pour l’anglais L2. In XXXIIe Journées d’Études sur la Parole, ISCA. ISCA. [find](https://scholar.google.com/scholar?q=Anne+Tortel+and+Sophie+Herment%2E+2018%2E+La+voyelle+inaccentu%C3%A9e+e+en+position+initiale+%3A+analyses+acoustiques+et+enjeux+p%C3%A9dagogiques+pour+l%E2%80%99anglais+L2%2E+In+XXXIIe+Journ%C3%A9es+d%E2%80%99%C3%89tudes+sur+la+Parole%2C+ISCA%2E+IS) 17. Emily Di Zhang. 2020. Automated speaking assessment: Using language technologies to score spontaneous speech. Lang. Assessment Quarterly, 17(3):327–330. [find](https://scholar.google.com/scholar?q=Emily+Di+Zhang%2E+2020%2E+Automated+speaking+assessment%3A+Using+language+technologies+to+score+spontaneous+speech%2E+Lang%2E+Assessment+Quarterly%2C+17%283%29%3A327%E2%80%93330%2E) *Updated 1 November 2026*