---
title: "Using Adaptive Empathetic Responses for Teaching English (2024)"
description: "English-teaching chatbots seldom respond to how learners feel, even though teacher empathy is linked… 26 references, cited by 3 papers here."
authors: ["Li Siyan", "Teresa Shao", "Julia Hirschberg", "Zhou Yu"]
year: 2024
venue: "Proceedings of the 19th Workshop on Innovative Use of NLP for Building Educational Applications (BEA 2024)"
source: "https://aclanthology.org/2024.bea-1.4/"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/siyan-2024-adaptive-empathetic-responses-teaching-english/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/siyan-2024-adaptive-empathetic-responses-teaching-english/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2024](https://md.research.elamur.ai/free/en/papers/2024/page/11/) / Using Adaptive Empathetic Responses for Teaching English

# Using Adaptive Empathetic Responses for Teaching English

**Authors:** [Li Siyan](https://md.research.elamur.ai/free/en/authors/li-siyan/), [Teresa Shao](https://md.research.elamur.ai/free/en/authors/teresa-shao/), [Julia Hirschberg](https://md.research.elamur.ai/free/en/authors/julia-hirschberg/) and [Zhou Yu](https://md.research.elamur.ai/free/en/authors/zhou-yu/) · **Year:** 2024 · **Venue:** [Proceedings of the 19th Workshop on Innovative Use of NLP for Building Educational Applications (BEA 2024)](https://md.research.elamur.ai/free/en/venues/bea/) ([BEA 2024](https://md.research.elamur.ai/free/en/venues/bea/2024/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2024.bea-1.4/) · [PDF](https://aclanthology.org/2024.bea-1.4.pdf) · [OpenAlex](https://openalex.org/W4395065276)

**Citations:** 3 citations (OpenAlex, 2026-10-14) · [3](https://md.research.elamur.ai/free/en/papers/siyan-2024-adaptive-empathetic-responses-teaching-english/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
English-teaching chatbots seldom respond to how learners feel, even though teacher empathy is linked to higher engagement and lower anxiety. The authors frame a new task: spotting negative emotion in learners' speech, using audio alone, as a cue for empathetic intervention. They also build what they describe as the first spoken English-tutoring chatbot that adapts empathetic feedback without sensors or cameras. The data are recordings of native Mandarin speakers talking to a practice chatbot. Clips are transcribed with Whisper medium and hand-labeled for negative affect and long pauses. An off-the-shelf speech emotion classifier, which was not retrained, is reconfigured to flag distress. When distress is detected, ChatGPT with automatically optimized prompts writes the encouragement. Otherwise, a fine-tuned grammar-correction model gives recasts with templated explanations. A small preliminary user study found that participants did perceive affective support. Ratings for encouragement and praise were mixed. This matters because accented learner speech is poorly served by emotion models trained on native speakers.

## Key points
- Dataset: 3,200 recordings of Mandarin-accented English, drawn from 163 users across 613 conversations. The release includes ASR transcripts and labels from four classes: Negative, Pauses, Neutral, Unusable. Two annotators double-labeled about 10% of the clips (Kappa 0.893), and only clips with agreed labels were kept.
- Grammar correction models were trained on ErAConD:
  - Flan-T5-XXL reached exact match 0.62, substring match 0.72 and a GPT-judged correctness of 0.58.
  - Llama-2-chat-7b scored only 0.30 on exact match, because it tends to keep generating past the correction.
- ChatGPT was tried first for grammar feedback but was dropped, since its corrections were often hallucinated or malformed.
- Pipeline:
  - Speech is transcribed with Whisper medium.
  - Replies are spoken by SpeechT5 with a CMU Arctic "slt" speaker embedding.
  - The system is built on the dialogue framework of Li et al. (2022).
  - A cooldown requires at least 2 turns between grammar feedback and 4 turns between empathetic feedback.
- The user study used perceived affective support (PAS) as its measure. The longer-term aim is to test whether the system raises L2 grit.

## Interesting ideas
- Report substring match next to exact match, and discard any correction longer than one sentence; together these catch generative models that pad their output without hiding real errors.
- Feedback recasts only the dependency-parse constituent that contains the error, not the whole sentence, which makes the correction shorter and more natural.
- Different feedback types get separate cooldowns (2 turns for grammar, 4 for empathy) so the learner is not overwhelmed.
- Prolonged pauses are treated as a distinct signal of struggle, and mixed cases are folded into the rarer Negative class to improve label balance.
- When gold corrections are missing, as with the 100 transcribed utterances, a stronger LLM (GPT-4-Turbo) acts as judge of grammaticality.

## For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

- **I1** (learn): Evaluations of LLM-teacher outputs could separate "correct but verbose" from "wrong," and length-based rejection could filter silver data before training.
- **I2** (back): Generation could realize a minimal subtree around the changed node, so explanations point to exactly the phrase that differs.
- **I3** (answer): Tutoring-style readers could throttle how often each kind of correction or hint appears in a dialogue with a student.
- **I4** (frames): Hesitation markers in text (ellipses, fillers, self-repairs) could feed a pragmatic "speaker is struggling" state, with ambiguous cases resolved toward the rarer class on purpose.
- **I5** (learn): An LLM judge could score outputs on unannotated domains, provided it is checked against negative controls so the judge is not simply trusted.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Columbia University](https://md.research.elamur.ai/free/en/universities/columbia-university/) — University, New York, United States · 241 works · 54,053 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works

🧭 Topics (1): [English](https://md.research.elamur.ai/free/en/topics/english/) · 2,263 works

## Cite

Cite as: Li Siyan, Teresa Shao, Julia Hirschberg and Zhou Yu. 2024. Using Adaptive Empathetic Responses for Teaching English. Proceedings of the 19th Workshop on Innovative Use of NLP for Building Educational Applications (BEA 2024).

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{siyan2024using,
  title = {{Using Adaptive Empathetic Responses for Teaching English}},
  author = {Li Siyan, Teresa Shao, Julia Hirschberg and Zhou Yu},
  booktitle = {Proceedings of the 19th Workshop on Innovative Use of NLP for Building Educational Applications (BEA 2024)},
  year = {2024},
  url = {https://aclanthology.org/2024.bea-1.4/}
}
```

- [← previous Upaya at ArabicNLU Shared-Task: Arabic Lexical Disambiguation using Large Language Models](https://md.research.elamur.ai/free/en/papers/rajpoot-2024-upaya-arabicnlu-shared-task-arabic/)
- [next → A systematic investigation of learnability from single child linguistic input](https://md.research.elamur.ai/free/en/papers/qin-2024-systematic-investigation-learnability-single-child/)
- [↑ back to papers of 2024](https://md.research.elamur.ai/free/en/papers/2024/page/11/)

## References

26 references, 3 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Luke Fryer, David Coniam, Rollo Carpenter, and Diana Lăpus , neanu. 2020. Bots for language learning now: Current and future directions. [find](https://scholar.google.com/scholar?q=Luke+Fryer%2C+David+Coniam%2C+Rollo+Carpenter%2C+and+Diana+La%CC%86pus+%2C+neanu%2E+2020%2E+Bots+for+language+learning+now%3A+Current+and+future+directions%2E)
2. Junyi Ao, Rui Wang, Long Zhou, Chengyi Wang, Shuo Ren, Yu Wu, Shujie Liu, Tom Ko, Qing Li, Yu Zhang, Zhihua Wei, Yao Qian, Jinyu Li, and Furu Wei. 2022. SpeechT5: Unified-modal encoder-decoder Fabrizio Gilardi, Meysam Alizadeh, and Maël Kubli. 2023. [Chatgpt outperforms crowd-workers for text-annotation tasks](https://md.research.elamur.ai/free/en/papers/gilardi-2023-chatgpt-outperforms-crowd-workers-text/). arXiv preprint arXiv:2303.15056. [↗](https://arxiv.org/abs/2303.15056)
3. Yoshiko Goda, Masanori Yamada, Hideya Matsukawa, Kojiro Hata, and Seisuke Yasunami. 2014. Conversation with a chatbot before an online efl group discussion and the effects on critical thinking. The Journal of Information and Systems in Education, 13(1):1–7. [↗](https://doi.org/10.12937/ejsise.13.1)
4. Hao-Chiang Koong Lin, Ching-Ju Chao, and Tsu-Ching Huang. 2015. From a perspective on foreign language learning anxiety to develop an affective tutoring system. Educational Technology Research and Development, 63:727–747. [↗](https://doi.org/10.1007/s11423-015-9385-6)
5. Jonatas Grosman. 2021. Fine-tuned XLSR-53 large model for speech recognition in English. [https://huggingface.co/jonatasgrosman/](https://huggingface.co/jonatasgrosman/) wav2vec2-large-xlsr-53-english. [↗](https://huggingface.co/jonatasgrosman/)
6. Liang-Chen Lin. 2014. Understanding pronunciation variations facing esl students. International Journal of Humanities and Social Science, 4(5):16–20. [find](https://scholar.google.com/scholar?q=Liang%2DChen+Lin%2E+2014%2E+Understanding+pronunciation+variations+facing+esl+students%2E+International+Journal+of+Humanities+and+Social+Science%2C+4%285%29%3A16%E2%80%9320%2E)
7. Michael Pin-Chuan Lin and Daniel Chang. 2020. Enhancing post-secondary writers’ writing skills with a chatbot. Journal of Educational Technology & Society, 23(1):78–92. [find](https://scholar.google.com/scholar?q=Michael+Pin%2DChuan+Lin+and+Daniel+Chang%2E+2020%2E+Enhancing+post%2Dsecondary+writers%E2%80%99+writing+skills+with+a+chatbot%2E+Journal+of+Educational+Technology+%26+Society%2C+23%281%29%3A78%E2%80%9392%2E)
8. Suleyman Kasap. 2019. Anxiety in the efl speaking classrooms. The Journal of Language Learning and Teaching, 9(2):23–36. [↗](https://doi.org/10.66887/jltl.v9i2.169)
9. Steven R Livingstone and Frank A Russo. 2018. The ryerson audio-visual database of emotional speech and song (ravdess): A dynamic, multimodal set of facial and vocal expressions in north american english. PloS one, 13(5):e0196391. [↗](https://doi.org/10.1371/journal.pone.0196391)
10. Omar Khattab, Arnav Singhvi, Paridhi Maheshwari, Zhiyuan Zhang, Keshav Santhanam, Sri Vardhamanan, Saiful Haq, Ashutosh Sharma, Thomas T. Joshi, Hanna Moazam, Heather Miller, Matei Zaharia, and Christopher Potts. 2023. [Dspy: Compiling declarative language model calls into self-improving pipelines](https://md.research.elamur.ai/free/en/papers/khattab-2023-dspy-compiling-declarative-language-model/). arXiv preprint arXiv:2310.03714. [↗](https://arxiv.org/abs/2310.03714)
11. Reza Lotfian and Carlos Busso. 2017. Building naturalistic emotionally balanced speech corpus by retrieving emotional speech from existing podcast recordings. IEEE Transactions on Affective Computing, 10(4):471–483. [↗](https://doi.org/10.1109/taffc.2017.2736999)
12. Na-Young Kim. 2018. A study on chatbots for developing korean college students’ english listening and reading skills. Journal of Digital Convergence, 16(8). [↗](https://doi.org/10.14400/jdc.2018.16.8.019)
13. Roy Lyster, Kazuya Saito, and Masatoshi Sato. 2013. Oral corrective feedback in second language classrooms. Language teaching, 46(1):1–40. [↗](https://doi.org/10.1017/s0261444812000365)
14. Na-Young Kim, Yoonjung Cha, and Hea-Suk Kim. 2019. Future english learning: Chatbots and artificial intelligence. Multimedia-Assisted Language Learning, 22(3). [find](https://scholar.google.com/scholar?q=Na%2DYoung+Kim%2C+Yoonjung+Cha%2C+and+Hea%2DSuk+Kim%2E+2019%2E+Future+english+learning%3A+Chatbots+and+artificial+intelligence%2E+Multimedia%2DAssisted+Language+Learning%2C+22%283%29%2E)
15. Michal Mazur, Rafal Rzepka, and Kenji Araki. 2011. Proposal for a conversational english tutoring system that encourages user engagement. In Proceedings of the 19th International Conference on Computers in Education, pages 10–12. [find](https://scholar.google.com/scholar?q=Michal+Mazur%2C+Rafal+Rzepka%2C+and+Kenji+Araki%2E+2011%2E+Proposal+for+a+conversational+english+tutoring+system+that+encourages+user+engagement%2E+In+Proceedings+of+the+19th+International+Conference+on+Compute)
16. John Kominek and Alan W Black. 2004. The cmu arctic speech databases. In Fifth ISCA workshop on speech synthesis. [find](https://scholar.google.com/scholar?q=John+Kominek+and+Alan+W+Black%2E+2004%2E+The+cmu+arctic+speech+databases%2E+In+Fifth+ISCA+workshop+on+speech+synthesis%2E)
17. Susana Pérez Castillejo. 2019. The role of foreign language anxiety on l2 utterance fluency during a final exam. Language Testing, 36(3):327–345. [↗](https://doi.org/10.1177/0265532218777783)
18. Tony Chiu Ming Lam, Klodiana Kolomitro, and Flanny C Alamparambil. 2011. Empathy training: Methods, evaluation practices, and validity. Journal of Multidisciplinary Evaluation, 7(16):162–200. [↗](https://doi.org/10.56645/jmde.v7i16.314)
19. Alec Radford, Jong Wook Kim, Tao Xu, Greg Brockman, Christine McLeavey, and Ilya Sutskever. 2023. [Robust speech recognition via large-scale weak supervision](https://md.research.elamur.ai/free/en/papers/radford-2022-robust-speech-recognition-large-scale/). In International Conference on Machine Learning, pages 28492–28518. PMLR. [↗](https://doi.org/10.48550/arxiv.2212.04356)
20. Yu Li, Chun-Yen Chen, Dian Yu, Sam Davidson, Ryan Hou, Xun Yuan, Yinghua Tan, Derek Pham, and Zhou Yu. 2022. Using chatbots to teach languages. In Proceedings of the Ninth ACM Conference on Learning@ Scale, pages 451–455. [find](https://scholar.google.com/scholar?q=Yu+Li%2C+Chun%2DYen+Chen%2C+Dian+Yu%2C+Sam+Davidson%2C+Ryan+Hou%2C+Xun+Yuan%2C+Yinghua+Tan%2C+Derek+Pham%2C+and+Zhou+Yu%2E+2022%2E+Using+chatbots+to+teach+languages%2E+In+Proceedings+of+the+Ninth+ACM+Conference+on+Learning%40+)
21. Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. 2019. Language models are unsupervised multitask learners. OpenAI blog, 1(8):9. [find](https://scholar.google.com/scholar?q=Alec+Radford%2C+Jeffrey+Wu%2C+Rewon+Child%2C+David+Luan%2C+Dario+Amodei%2C+Ilya+Sutskever%2C+et+al%2E+2019%2E+Language+models+are+unsupervised+multitask+learners%2E+OpenAI+blog%2C+1%288%29%3A9%2E)
22. Yu Li, Shang Qu, Jili Shen, Shangchao Min, and Zhou Yu. 2023. Curriculum-driven edubot: A framework for developing language learning chatbots through synthesizing conversational data. arXiv preprint arXiv:2309.16804. [↗](https://arxiv.org/abs/2309.16804)
23. Gonul Sakiz. 2007. Does teacher affective support matter? An investigation of the relationship among perceived teacher affective support, sense of belonging, academic emotions, academic self-efficacy beliefs, and academic effort in middle school mathematics classrooms. Ph.D. thesis, The Ohio State University. [find](https://scholar.google.com/scholar?q=Gonul+Sakiz%2E+2007%2E+Does+teacher+affective+support+matter%3F+An+investigation+of+the+relationship+among+perceived+teacher+affective+support%2C+sense+of+belonging%2C+academic+emotions%2C+academic+self%2Defficacy+)
24. Kai-Hui Liang, Sam Davidson, Xun Yuan, Shehan Pan-ditharatne, Chun-Yen Chen, Ryan Shea, Derek Pham, Yinghua Tan, Erik Voss, and Luke Fryer. 2023. Chat-Back: Investigating methods of providing grammatical error feedback in a GUI-based language learning chatbot. In Proceedings of the 18th Workshop on Innovative Use of NLP for Building Educational Applications (BEA 2023), pages 83–99, Toronto, Canada. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Kai%2DHui+Liang%2C+Sam+Davidson%2C+Xun+Yuan%2C+Shehan+Pan%2Dditharatne%2C+Chun%2DYen+Chen%2C+Ryan+Shea%2C+Derek+Pham%2C+Yinghua+Tan%2C+Erik+Voss%2C+and+Luke+Fryer%2E+2023%2E+Chat%2DBack%3A+Investigating+methods+of+providing+grammati)
25. Olga C Santos, Mar Saneiro, Jesus G Boticario, and María Cristina Rodriguez-Sanchez. 2016. Toward interactive context-aware affective educational recommendations in computer-assisted language learning. New Review of Hypermedia and Multimedia, 22(1-2):27–57. [↗](https://doi.org/10.1080/13614568.2015.1058428)
26. A Nuobei Shi, Qin Zeng, and Raymond Lee. 2020. The design and implementation of language learning chatbot with xai using ontology and transfer learning. arXiv preprint arXiv:2009.13984. Chatbot Design Discussion [↗](https://arxiv.org/abs/2009.13984)

*Updated 1 November 2026*