---
title: "Discourse Analysis of Argumentative Essays of English Learners"
description: "The authors ask whether learners of English organise their argumentative writing differently as their proficiency grows. 25 references."
authors: ["Blaise Hanel", "Leila Kosseim"]
year: 2023
venue: "Proceedings of the 14th International Conference on Recent Advances in Natural Language Processing"
source: "https://aclanthology.org/2023.ranlp-1.52/"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/hanel-2023-discourse-analysis-argumentative-essays-english/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/hanel-2023-discourse-analysis-argumentative-essays-english/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2023](https://md.research.elamur.ai/free/en/papers/2023/page/17/) / Discourse Analysis of Argumentative Essays of English Learners Based on CEFR Level

# Discourse Analysis of Argumentative Essays of English Learners Based on CEFR Level

**Authors:** Blaise Hanel and [Leila Kosseim](https://md.research.elamur.ai/free/en/authors/leila-kosseim/) · **Year:** 2023 · **Venue:** [Proceedings of the 14th International Conference on Recent Advances in Natural Language Processing](https://md.research.elamur.ai/free/en/venues/ranlp/) ([RANLP 2023](https://md.research.elamur.ai/free/en/venues/ranlp/2023/)) · **Publisher:** [INCOMA Ltd.](https://md.research.elamur.ai/free/en/publishers/incoma-ltd/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2023.ranlp-1.52/) · [PDF](https://aclanthology.org/2023.ranlp-1.52.pdf) · [OpenAlex](https://openalex.org/W4390623405)

**Citations:** 0 citations (OpenAlex, 2026-10-14)

## Summary
The authors ask whether learners of English organise their argumentative writing differently as their proficiency grows. They use two essay collections, ICNALE and CROW, with CEFR levels A2, B1, B2 and C2. CROW originally carries TOEFL scores, which were converted with the ETS mapping. Each essay was run through two RST parsers and two PDTB parsers. For RST, only relations on which both parsers agreed for the same aligned span were kept. For PDTB, the two parsers' outputs were averaged. Relation frequencies at each learner level were compared to C2 with two-tailed t-tests. A published PDTB-to-RST mapping linked the two frameworks.

Lower-level writers lean more on RST EXPLANATION and BACKGROUND. In ICNALE, PDTB CONTINGENCY is used less as level rises, and the matching RST causal group shows the same pattern. The study is presented as the first look at discourse structure across proficiency levels. It suggests discourse-relation profiles could serve as an extra signal for automatic CEFR assessment.

## Key points
- Data: ICNALE has 5600 essays averaging about 231 words. CROW has 1429 argumentative essays averaging about 1057 words. Neither has A1 or C1, and C2 texts come from countries where English is official. ICNALE's B1.1 and B1.2 were merged into B1.
- Parsers: the RST parsers were Heilman and Sagae (2015), with 57.4% F on RST-DT, and Wang et al. (2017), with 59.7% F, against a human score of 65.8%. The PDTB-2.0 parsers were Lin et al. (2014) and Wang and Lan (2015), restricted to level-1 senses.
- Filtering: relations found in under 10% of documents were dropped (EVALUATION, SUMMARY, TOPIC-COMMENT, TOPIC-CHANGE). This left 12 RST relations and all 4 PDTB level-1 senses.
- Parser disagreement: the two RST parsers clashed most often on CAUSE, which was frequently tagged as EXPLANATION. ENABLEMENT was often tagged as ELABORATION.
- Results: in ICNALE, CONTINGENCY drops from about 40% at A2 to about 30% at C2. EXPLANATION is used more by lower-level writers in both corpora. Cross-framework comparison used the Demberg et al. (2017) mapping. Code is at GitHub CLaC-Lab/discourse-parsers-and-agreement.

## Interesting ideas
- Two parsers with the same label set are aligned on identical text spans, ignoring punctuation, and only labels both agree on are kept; this turns noisy automatic analyses into a cleaner sample without manual annotation.
- Labels that occur in less than a set share of documents are excluded before significance testing, so conclusions are not driven by rare, unstable categories.
- A result from one framework (PDTB CONTINGENCY) is checked against a grouped equivalent in another (RST CAUSE + CONDITION + EXPLANATION) via an explicit mapping table; agreement across independent schemes makes the finding more credible.
- Recording which labels the parsers systematically confuse, such as cause versus explanation or enablement versus elaboration, shows exactly where the category boundary is fuzzy.
- Each learner group is profiled against a reference group (C2) by relation frequencies, turning discourse structure into a measurable style fingerprint.

## For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

- **I1** (learn): The same span-alignment-plus-agreement filter could gate silver data from witness parsers, keeping only arcs or clause relations that independent parsers produce over identical token spans.
- **I2** (learn): Evaluation reports could apply a document-frequency floor before claiming that one model or rule set changes a label's rate, avoiding false wins on sparse categories.
- **I3** (frames): Clause-to-clause causal and conditional links in prag could be cross-checked against a second inventory through an explicit mapping, flagging links where the two views disagree.
- **I4** (frames): A confusion log for discourse and frame relations could point to where explicit cue words or rules are needed to separate causal from explanatory links.
- **I5** (answer): When Mova reads or grades student essays, comparing their relation profile with a reference profile could give an explainable signal of writing maturity.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Concordia University](https://md.research.elamur.ai/free/en/universities/concordia-university/) — University, Montreal, Canada · 13 works · 673 citations

📚 Shelves (6): [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 17,450 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 17,296 works · [💭 Psychology](https://md.research.elamur.ai/free/en/shelves/psychology/) · 1,649 works · [🤔 Philosophy](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/philosophy/) · 287 works · [💭 Developmental and Educational Psychology](https://md.research.elamur.ai/free/en/shelves/psychology/developmental-and-educational-psychology/) · 523 works · [📚 Literature and Literary Theory](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/literature-and-literary-theory/) · 541 works

🧭 Topics (2): [English](https://md.research.elamur.ai/free/en/topics/english/) · 2,379 works · [🧵 Discourse analysis](https://md.research.elamur.ai/free/en/topics/discourse-analysis/) · 471 works

## Cite

Cite as: Blaise Hanel and Leila Kosseim. 2023. Discourse Analysis of Argumentative Essays of English Learners Based on CEFR Level. Proceedings of the 14th International Conference on Recent Advances in Natural Language Processing.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{hanel2023discourse,
  title = {{Discourse Analysis of Argumentative Essays of English Learners Based on CEFR Level}},
  author = {Blaise Hanel and Leila Kosseim},
  booktitle = {Proceedings of the 14th International Conference on Recent Advances in Natural Language Processing},
  year = {2023},
  doi = {10.26615/978-954-452-092-2\_052},
  url = {https://doi.org/10.26615/978-954-452-092-2_052}
}
```

- [← previous Difficulties in Handling Mathematical Expressions in Universal Dependencies](https://md.research.elamur.ai/free/en/papers/levine-2023-difficulties-handling-mathematical-expressions-universal/)
- [next → DUTIR at SemEval-2023 Task 10: Semi-supervised Learning for Sexism Detection in English](https://md.research.elamur.ai/free/en/papers/yu-2023-dutir-semeval-2023-task-10/)
- [↑ back to papers of 2023](https://md.research.elamur.ai/free/en/papers/2023/page/17/)

## References

25 references, 10 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Brian MacWhinney, Davida Fromm, Margaret Forbes, and Audrey Holland. 2011. Aphasiabank: Methods for studying discourse. Aphasiology, 25(11):1286– 1307. PMID: 22923879. [↗](https://doi.org/10.1080/02687038.2011.589893)
2. Mohamed Abdalla, Frank Rudzicz, and Graeme Hirst. 2018. Rhetorical structure and Alzheimer’s disease. Aphasiology, 32(1):41–60. [↗](https://doi.org/10.1080/02687038.2017.1355439)
3. William C. Mann and Sandra A. Thompson. 1988. [Rhetorical structure theory: Toward a functional theory of text organization](https://md.research.elamur.ai/free/en/papers/mann-1988-rhetorical-structure-theory-functional/). Text - Interdisciplinary Journal for the Study of Discourse, 8:243–281. [↗](https://doi.org/10.1515/text.1.1988.8.3.243)
4. Tatsuya Aoyama. 2022. [Comparing native and learner englishes using a large pre-trained language model](https://md.research.elamur.ai/free/en/papers/aoyama-2022-comparing-native-learner-englishes-large/). In Proceedings of the 11th Workshop on NLP for Computer Assisted Language Learning, pages 1–9, Louvain-la-Neuve, Belgium. LiU Electronic Press.
5. Margot Mieskes and Ulrike Padó. 2018. Work smart -reducing effort in short-answer grading. In Proceedings of the 7th workshop on NLP for Computer Assisted Language Learning, pages 57–68, Stockholm, Sweden. LiU Electronic Press. [find](https://scholar.google.com/scholar?q=Margot+Mieskes+and+Ulrike+Pado%CC%81%2E+2018%2E+Work+smart+%2Dreducing+effort+in+short%2Danswer+grading%2E+In+Proceedings+of+the+7th+workshop+on+NLP+for+Computer+Assisted+Language+Learning%2C+pages+57%E2%80%9368%2C+Stockholm%2C+S)
6. Félix-Hervé Bachand, Elnaz Davoodi, and Leila Kosseim. 2014. An Investigation on the Influence of Genres and Textual Organisation on the Use of Discourse Relations. In International Conference on Intelligent Text Processing and Computational Linguistics (CICLing-2014), pages 454–468. [find](https://scholar.google.com/scholar?q=Fe%CC%81lix%2DHerve%CC%81+Bachand%2C+Elnaz+Davoodi%2C+and+Leila+Kosseim%2E+2014%2E+An+Investigation+on+the+Influence+of+Genres+and+Textual+Organisation+on+the+Use+of+Discourse+Relations%2E+In+International+Conference+on+In)
7. Charlene Pope and Boyd H. Davis. 2011. Finding a balance: The carolinas conversation collection. 7(1):143–161. [↗](https://doi.org/10.1515/cllt.2011.007)
8. Lynn Carlson, Daniel Marcu, and Mary Ellen Okurowski. 2001. [Building a discourse-tagged corpus in the framework of rhetorical structure theory](https://md.research.elamur.ai/free/en/papers/carlson-2001-building-discourse-tagged-corpus-framework/). In Proceedings of Second SIGdial Workshop on Discourse and Dialogue (SIGDIAL 2001), pages 1–10, Aalborg, Denmark. [↗](https://doi.org/10.21236/ada460581)
9. Rashmi Prasad, Nikhil Dinesh, Alan Lee, and Eleni Miltsakaki. 2008. [The Penn Discourse TreeBank 2.0](https://md.research.elamur.ai/free/en/papers/prasad-2008-penn-discourse-treebank-2-0/). In Proceedings of the Sixth International Conference on Language Resources and Evaluation (LREC’08), Marrakech.
10. Nelson Filipe Costa, Nadia Sheikh, and Leila Kosseim. 2023. [Mapping explicit and implicit discourse relations between the RST-DT and the PDTB 3.0](https://md.research.elamur.ai/free/en/papers/costa-2023-mapping-explicit-implicit-discourse-relations/). In Proceedings of the International Conference on Recent Advances in Natural Language Processing (RANLP 2023), Varna, Bulgaria.
11. Rashmi Prasad, Eleni Miltsakaki, Nikhil Dinesh, and Alan Lee. 2006. The Penn Discourse TreeBank 1.0 Annotation Manual. [find](https://scholar.google.com/scholar?q=Rashmi+Prasad%2C+Eleni+Miltsakaki%2C+Nikhil+Dinesh%2C+and+Alan+Lee%2E+2006%2E+The+Penn+Discourse+TreeBank+1%2E0+Annotation+Manual%2E)
12. Rashmi Prasad, Bonnie Webber, Alan Lee, and Aravind Joshi. 2019. Penn Discourse Treebank Version 3.0. Linguistic Data Consortium. [↗](https://doi.org/10.35111/qebf-gk47)
13. Elnaz Davoodi. 2017. Computational Discourse Analysis Across Complexity Levels. Ph.D. thesis, Concordia University Department of Computer Science and Software Engineering. [find](https://scholar.google.com/scholar?q=Elnaz+Davoodi%2E+2017%2E+Computational+Discourse+Analysis+Across+Complexity+Levels%2E+Ph%2ED%2E+thesis%2C+Concordia+University+Department+of+Computer+Science+and+Software+Engineering%2E)
14. Veronica Juliana Schmalz and Alessio Brutti. December 2021. [Automatic Assessment of English CEFR Levels Using BERT Embeddings](https://md.research.elamur.ai/free/en/papers/schmalz-2021-automatic-assessment-english-cefr-levels/). In Proceedings of 2021 Italian Conference on Computational Linguistics 2021 (CLiC), volume 3033.
15. Vera Demberg, Fatemeh Torabi Asr, and Merel C. J. Scholman. 2017. How consistent are our discourse annotations? Insights from mapping RST-DT and PDTB annotations. CoRR, abs/1704.08893. [↗](https://arxiv.org/abs/1704.08893)
16. Shelley Staples and Bradley Dilger. 2018. Corpus and Repository Of Writing [Learner corpus articulated with repository]. [find](https://scholar.google.com/scholar?q=Shelley+Staples+and+Bradley+Dilger%2E+2018%2E+Corpus+and+Repository+Of+Writing+%5BLearner+corpus+articulated+with+repository%5D%2E)
17. Vanessa Wei Feng and Graeme Hirst. 2014. [A Linear-Time Bottom-Up Discourse Parser with Constraints and Post-Editing](https://md.research.elamur.ai/free/en/papers/feng-2014-linear-time-bottom-up-discourse/). In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (ACL-2014), pages 511–521, Baltimore.
18. Jianxiang Wang and Man Lan. 2015. [A Refined End-to-End Discourse Parser](https://md.research.elamur.ai/free/en/papers/wang-2015-refined-end-end-discourse-parser/). In Proceedings of the 19 th Conference on Computational Natural Language Learning (CoNLL-2015), pages 17–24, Beijing.
19. Michael Heilman and Kenji Sagae. 2015. [Fast Rhetorical Structure Theory Discourse Parsing](https://md.research.elamur.ai/free/en/papers/heilman-2015-fast-rhetorical-structure-theory-discourse/). Computing Research Repository, abs/1505.02425. [↗](https://arxiv.org/abs/1505.02425)
20. Yizhong Wang, Sujian Li, and Houfeng Wang. 2017. [A Two-Stage Parsing Method for Text-Level Discourse Analysis](https://md.research.elamur.ai/free/en/papers/wang-2017-two-stage-parsing-method-text/). pages 184–188, Vancouver.
21. Shin Ishikawa. 2013. The ICNALE and sophisticated contrastive interlanguage analysis of Asian learners of English. In Learner corpus studies in Asia and the world, volume 1, pages 91–118. [↗](https://doi.org/10.24546/81006678)
22. Bonnie Webber. 2009. Genre distinctions for discourse in the Penn TreeBank. In Proceedings of the Joint Conference of the 47th Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing of the AFNLP (ACL/AFNLP-2009), pages 674–682, Suntec, Singapore. [find](https://scholar.google.com/scholar?q=Bonnie+Webber%2E+2009%2E+Genre+distinctions+for+discourse+in+the+Penn+TreeBank%2E+In+Proceedings+of+the+Joint+Conference+of+the+47th+Meeting+of+the+ACL+and+the+4th+International+Joint+Conference+on+Natural+)
23. Ziheng Lin, Hwee Tou Ng, and Min-Yen Kan. 2014. A PDTB-Styled End-to-End Discourse Parser. Natural Language Engineering, 20:151 – 184. [find](https://scholar.google.com/scholar?q=Ziheng+Lin%2C+Hwee+Tou+Ng%2C+and+Min%2DYen+Kan%2E+2014%2E+A+PDTB%2DStyled+End%2Dto%2DEnd+Discourse+Parser%2E+Natural+Language+Engineering%2C+20%3A151+%E2%80%93+184%2E)
24. S. Magalı́ López Cortez and Cassandra L. Jacobs. 2023. The distribution of discourse relations within and across turns in spontaneous conversation. In Proceedings of the 4th Workshop on Computational Approaches to Discourse (CODI 2023), pages 156– 162, Toronto, Canada. [find](https://scholar.google.com/scholar?q=S%2E+Magal%C4%B1%CC%81+Lo%CC%81pez+Cortez+and+Cassandra+L%2E+Jacobs%2E+2023%2E+The+distribution+of+discourse+relations+within+and+across+turns+in+spontaneous+conversation%2E+In+Proceedings+of+the+4th+Workshop+on+Computational)
25. Bonnie L. Webber and Aravind K. Joshi. 1998. Anchoring a lexicalized tree-adjoining grammar for discourse. CoRR, cmp-lg/9806017. [↗](https://doi.org/10.48550/arxiv.cmp-lg/9806017)

*Updated 2 November 2026*