---
title: "Interaction Matters — paper by Gao et al. (2025)"
description: "The authors propose a two-tier annotation framework for ESL dialogues. 23 references, cited by 5 papers here."
authors: ["Rena Gao", "Carsten Roever", "Jey Han Lau"]
year: 2025
venue: "Proceedings of the 31st International Conference on Computational Linguistics"
source: "https://aclanthology.org/2025.coling-main.729/"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/gao-2025-interaction-matters-evaluation-framework-interactive/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/gao-2025-interaction-matters-evaluation-framework-interactive/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2025](https://md.research.elamur.ai/free/en/papers/2025/page/10/) / Interaction Matters: An Evaluation Framework for Interactive Dialogue Assessment on English Second Language Conversations

# Interaction Matters: An Evaluation Framework for Interactive Dialogue Assessment on English Second Language Conversations

**Authors:** [Rena Gao](https://md.research.elamur.ai/free/en/authors/rena-gao/), Carsten Roever and [Jey Han Lau](https://md.research.elamur.ai/free/en/authors/jey-han-lau/) · **Year:** 2025 · **Venue:** [Proceedings of the 31st International Conference on Computational Linguistics](https://md.research.elamur.ai/free/en/venues/coling/) ([COLING 2025](https://md.research.elamur.ai/free/en/venues/coling/2025/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2025.coling-main.729/) · [PDF](https://aclanthology.org/2025.coling-main.729.pdf) · [OpenAlex](https://openalex.org/W4400518740)

**Citations:** 1 citations (OpenAlex, 2026-10-14) · 5 in our library, of the 149,439 papers whose bibliographies we read

## Summary
Most dialogue datasets and quality metrics come from native-speaker conversations and focus on fluency or coherence. They overlook how English as a Second Language (ESL) speakers handle interaction, for example how they develop topics, adjust register, or open and close a conversation. The authors propose a two-tier annotation framework for ESL dialogues. At the dialogue level, each conversation receives four interactivity scores on a 1–5 scale. Below that, 17 fine-grained linguistic features are marked as spans, split into token-level and utterance-level phenomena. Using this scheme they build and release SLEDE, an annotated ESL dialogue corpus. They then train simple classifiers that predict the interactivity scores from the micro-level features. These feature-based models outperform a BERT baseline that reads the raw dialogue. Some features, such as reference words, turn out to be strongly linked to interactivity quality. The work therefore connects low-level linguistic signals to higher-level communicative skill and points toward tools for assessing spoken ESL interaction.

## Key points
- **Interactivity labels:** the four dialogue-level labels are topic management, tone appropriateness, conversation opening and conversation closing.
- **Micro-level features:** the 17 features include 7 token-level ones (e.g., reference word, noun & verb collocation, code-switching, routinized resources) and utterance-level ones (e.g., backchannels, feedback in next turn, collaborative finishes, epistemic copulas).
- **SLEDE corpus:** SLEDE (Second Language English Dialogue Evaluation) contains 120 dialogues. The authors report strong inter-annotator agreement and very long dialogues. Data and code are on GitHub (RenaGao/2024InteractiveMetrics).
- **Models and ablation:** the classifiers are Logistic Regression, Random Forest and Naive Bayes, compared with a BERT baseline. In the ablation, token-level features slightly beat utterance-level ones. Combining both sets works best in most settings; the exceptions are RF for topic and tone.
- **Main limitation:** the models are not end-to-end, because they need manually annotated micro-level features as input. Automating the extraction of these features is left for future work.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [The University of Melbourne](https://md.research.elamur.ai/free/en/universities/the-university-of-melbourne/) — University, Melbourne, Australia · 100 works · 22,618 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works

🧭 Topics (3): [English](https://md.research.elamur.ai/free/en/topics/english/) · 2,263 works · [🗨️ Dialogue systems](https://md.research.elamur.ai/free/en/topics/dialogue-systems/) · 624 works · [🖍️ Grammatical error correction & learner language](https://md.research.elamur.ai/free/en/topics/grammatical-error-correction/) · 156 works

## Cite

Cite as: Rena Gao, Carsten Roever and Jey Han Lau. 2025. Interaction Matters: An Evaluation Framework for Interactive Dialogue Assessment on English Second Language Conversations. Proceedings of the 31st International Conference on Computational Linguistics.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{gao2025interaction,
  title = {{Interaction Matters: An Evaluation Framework for Interactive Dialogue Assessment on English Second Language Conversations}},
  author = {Rena Gao, Carsten Roever and Jey Han Lau},
  booktitle = {Proceedings of the 31st International Conference on Computational Linguistics},
  year = {2025},
  url = {https://aclanthology.org/2025.coling-main.729/}
}
```

- [← previous Insight Over Sight: Exploring the Vision-Knowledge Conflicts in Multimodal LLMs](https://md.research.elamur.ai/free/en/papers/liu-2025-insight-over-sight-exploring-vision/)
- [next → IPA CHILDES & G2P+: Feature-Rich Resources for Cross-Lingual Phonology and Phonemic Language Modeling](https://md.research.elamur.ai/free/en/papers/goriely-2025-ipa-childes-g2p-feature-rich/)
- [↑ back to papers of 2025](https://md.research.elamur.ai/free/en/papers/2025/page/10/)

## References

23 references, 1 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Makoto Abe and Carsten Roever. 2019. Interactional competence in l2 text-chat interactions: First-idea proffering in task openings. Journal of Pragmatics, 144:1–14. [↗](https://doi.org/10.1016/j.pragma.2019.03.001)
2. Evelina D Galaczi. 2014. Interactional competence across proficiency levels: How do learners manage interaction in paired speaking tests? Applied linguistics, 35(5):553–574. [↗](https://doi.org/10.1093/applin/amt017)
3. Sarik Ghazarian, Nuan Wen, Aram Galstyan, and Nanyun Peng. 2022. Deam: Dialogue coherence evaluation using amr-based semantic manipulations. arXiv preprint arXiv:2203.09711. [↗](https://arxiv.org/abs/2203.09711)
4. Yugo Hayashi. 2020. Gaze awareness and metacognitive suggestions by a pedagogical conversational agent: an experimental investigation on interventions to support collaborative learning process and performance. International Journal of Computer-Supported Collaborative Learning, 15(4):469–498. [↗](https://doi.org/10.1007/s11412-020-09333-3)
5. Klaus Krippendorff. 2018. Content analysis: An introduction to its methodology. Sage publications. [↗](https://doi.org/10.4135/9781071878781)
6. Daniel MK Lam. 2018. What counts as “responding”? contingency on previous speaker contribution as a feature of interactional competence. Language Testing, 35(3):377–401. [find](https://scholar.google.com/scholar?q=Daniel+MK+Lam%2E+2018%2E+What+counts+as+%E2%80%9Cresponding%E2%80%9D%3F+contingency+on+previous+speaker+contribution+as+a+feature+of+interactional+competence%2E+Language+Testing%2C+35%283%29%3A377%E2%80%93401%2E)
7. Holy Lovenia, Bryan Wilie, Willy Chung, Min Zeng, Samuel Cahyawijaya, Su Dan, and Pascale Fung. 2022. Clozer: Adaptable data augmentation for cloze-style reading comprehension. arXiv preprint arXiv:2203.16027. [↗](https://arxiv.org/abs/2203.16027)
8. Shikib Mehri, Yulan Feng, Carla Gordon, Seyed Hossein Alavi, David Traum, and Maxine Eskenazi. 2022. Interactive evaluation of dialog track at dstc9. arXiv preprint arXiv:2207.14403. [↗](https://arxiv.org/abs/2207.14403)
9. John Pill. 2016. Drawing on indigenous criteria for more authentic assessment in a specific-purpose language test: Health professionals interacting with patients. Language Testing, 33(2):175–193. [↗](https://doi.org/10.1177/0265532215607400)
10. Thomas Warren. 2017. Cross-cultural Communication: Perspectives in theory and practice. Routledge. [find](https://scholar.google.com/scholar?q=Thomas+Warren%2E+2017%2E+Cross%2Dcultural+Communication%3A+Perspectives+in+theory+and+practice%2E+Routledge%2E)
11. Steve Whittaker, Yvonne Rogers, Elena Petrovskaya, and Hongbin Zhuang. 2021. Designing personas for expressive robots: Personality in the new breed of moving, speaking, and colorful social home robots. J. Hum.-Robot Interact., 10(1). [find](https://scholar.google.com/scholar?q=Steve+Whittaker%2C+Yvonne+Rogers%2C+Elena+Petrovskaya%2C+and+Hongbin+Zhuang%2E+2021%2E+Designing+personas+for+expressive+robots%3A+Personality+in+the+new+breed+of+moving%2C+speaking%2C+and+colorful+social+home+robots)
12. Jingxuan Wu and Carsten Roever. 2021. Proficiency and preference organization in second language mandarin chinese refusals. The Modern Language Journal, 105(4):897–918. [↗](https://doi.org/10.1111/modl.12736)
13. Foong Ha Yap and Anindita Sahoo. 2024. Versatile copulas and their stance-marking uses in conversational odia, an indo-aryan language. Lingua, 297:103641. [↗](https://doi.org/10.1016/j.lingua.2023.103641)
14. Juan-Pedro Rica-Peromingo. 2009. The Status of English in Spain, pages 168–174. [find](https://scholar.google.com/scholar?q=Juan%2DPedro+Rica%2DPeromingo%2E+2009%2E+The+Status+of+English+in+Spain%2C+pages+168%E2%80%93174%2E)
15. Carsten Roever and David W Dai. 2021. Reconceptualizing interactional competence for language testing. Assessing speaking in context: Expanding the construct and its applications, pages 23–49. [find](https://scholar.google.com/scholar?q=Carsten+Roever+and+David+W+Dai%2E+2021%2E+Reconceptualizing+interactional+competence+for+language+testing%2E+Assessing+speaking+in+context%3A+Expanding+the+construct+and+its+applications%2C+pages+23%E2%80%9349%2E)
16. Carsten Roever and Naoki Ikeda. 2023. The relationship between l2 interactional competence and proficiency. Applied Linguistics, page amad053. [↗](https://doi.org/10.1093/applin/amad053)
17. Emanuel A Schegloff. 1968. Sequencing in conversational openings 1. American anthropologist, 70(6):1075–1095. [find](https://scholar.google.com/scholar?q=Emanuel+A+Schegloff%2E+1968%2E+Sequencing+in+conversational+openings+1%2E+American+anthropologist%2C+70%286%29%3A1075%E2%80%931095%2E)
18. Emanuel A Schegloff and Harvey Sacks. 1973. Opening up closings. [↗](https://doi.org/10.1515/semi.1973.8.4.289)
19. Isis H Settles, Martinque K Jones, NiCole T Buchanan, and Kristie Dotson. 2021. Epistemic exclusion: Scholar (ly) devaluation that marginalizes faculty of color. Journal of Diversity in Higher Education, 14(4):493. [↗](https://doi.org/10.1037/dhe0000174)
20. Abdullayev Shaxobiddin. 2024. A discourse analysis of modal verbs in modern english: Patterns and functions. Journal of new century innovations, 50(2):145– 147. [find](https://scholar.google.com/scholar?q=Abdullayev+Shaxobiddin%2E+2024%2E+A+discourse+analysis+of+modal+verbs+in+modern+english%3A+Patterns+and+functions%2E+Journal+of+new+century+innovations%2C+50%282%29%3A145%E2%80%93+147%2E)
21. Koustuv Sinha, Prasanna Parthasarathi, Jasmine Wang, Ryan Lowe, William L Hamilton, and Joelle Pineau. 2020. Learning an unreferenced metric for online dialogue evaluation. arXiv preprint arXiv:2005.00583. [↗](https://arxiv.org/abs/2005.00583)
22. Andreas Stolcke, Klaus Ries, Noah Coccaro, Elizabeth Shriberg, Rebecca Bates, Daniel Jurafsky, Paul Taylor, Rachel Martin, Carol Van Ess-Dykema, and Marie Meteer. 2000. [Dialogue act modeling for automatic tagging and recognition of conversational speech](https://md.research.elamur.ai/free/en/papers/stolcke-2000-dialogue-act-modeling-automatic-tagging/). Computational linguistics, 26(3):339–373. [↗](https://doi.org/10.1162/089120100561737)
23. Chongyang Tao, Lili Mou, Dongyan Zhao, and Rui Yan. 2018. Ruber: An unsupervised method for automatic evaluation of open-domain dialog systems. In Proceedings of the AAAI conference on artificial intelligence, volume 32. [find](https://scholar.google.com/scholar?q=Chongyang+Tao%2C+Lili+Mou%2C+Dongyan+Zhao%2C+and+Rui+Yan%2E+2018%2E+Ruber%3A+An+unsupervised+method+for+automatic+evaluation+of+open%2Ddomain+dialog+systems%2E+In+Proceedings+of+the+AAAI+conference+on+artificial+inte)

*Updated 1 November 2026*