---
title: "Double-Edged Sword or Sharp Tool? Designing and Evaluating"
description: "Here the authors build and deploy a three-party workflow. 27 references. Among them: 0."
authors: ["Canran Wang", "Yuwen Yang", "Zhen Wang", "Ming MA", "Ding Yu", "Chentai Wang", "Keman Huang", "Xiaoyong Du"]
year: 2026
venue: "Findings of the Association for Computational Linguistics: ACL 2026"
source: "https://aclanthology.org/2026.findings-acl.597/"
doi: "10.18653/v1/2026.findings-acl.597"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/wang-2026-double-edged-sword-or-sharp/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/wang-2026-double-edged-sword-or-sharp/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/28/) / Double-Edged Sword or Sharp Tool? Designing and Evaluating Triadic LLM-Teacher Collaboration for K-12 Writing at Scale

# Double-Edged Sword or Sharp Tool? Designing and Evaluating Triadic LLM-Teacher Collaboration for K-12 Writing at Scale

**Authors:** Canran Wang, Yuwen Yang, [Zhen Wang](https://md.research.elamur.ai/free/en/authors/zhen-wang/), Ming MA, Ding Yu, Chentai Wang, Keman Huang and [Xiaoyong Du](https://md.research.elamur.ai/free/en/authors/xiaoyong-du/) · **Year:** 2026 · **Venue:** [Findings of the Association for Computational Linguistics: ACL 2026](https://md.research.elamur.ai/free/en/venues/findings/) ([Findings of the Association for Computational Linguistics 2026](https://md.research.elamur.ai/free/en/venues/findings/2026/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2026.findings-acl.597/) · [PDF](https://aclanthology.org/2026.findings-acl.597.pdf) · [DOI 10.18653/v1/2026.findings-acl.597](https://doi.org/10.18653/v1/2026.findings-acl.597) · [OpenAlex](https://openalex.org/W7166876926)

**Citations:** 0 citations (OpenAlex, 2026-10-14)

## Summary
The paper asks how LLMs and teachers can share the work of giving writing feedback to school-age learners. Most prior studies look at a single adult user working with a model in a lab. Here the authors build and deploy a three-party workflow. An LLM scores a student draft and proposes suggestions. A teacher then vets, edits and supplements those suggestions before the student revises, and the revision is graded again.

To measure progress, they draw on Halliday's Systemic Functional Linguistics. Their metrics cover vocabulary and syntax, coherence, and emotional and moral expression. They also follow individual suggestions to see whether students act on them.

Deployed across real classrooms for two years, the setup raised revision quality by more than 5%. The clearest gains were interpersonal, with a pro-social shift. The LLM mostly handled volume, while teachers filtered suggestions and drove adoption. Richer language helped weaker writers, but its benefit faded for stronger ones, and some metrics even turned negative. The authors therefore argue that support should adapt to student proficiency.

## Key points
- **Dataset:** 57,954 essays written by 10,195 students in 120 schools over two years. The authors describe it as the largest empirical collection on LLM-assisted K-12 learning.
- **Four-stage cycle:**
  - The student writes a draft (D_pre).
  - The LLM produces a grade and initial suggestions (S_initial).
  - The teacher refines these into S_final and gives a grade.
  - The student revises the draft (D_post), and both the LLM and the teacher grade it again.
- **Evaluation metrics, by SFL metafunction:**
  - Ideational: MATTR-based Lexical Richness, and Syntactic Diversity via a WL graph kernel.
  - Textual: Sentence-BERT coherence measures for dispersion and shift.
  - Interpersonal: 8 Plutchik emotions and 5 Moral Foundations Theory dimensions.
- **Interpersonal labelling:** done with two LLM agents (annotator and verifier) aligned to human labels. Prompts were refined until Cohen's kappa exceeded 0.6.
- **Teacher vs. LLM contributions:** teacher-edited suggestions were adopted more often and had larger marginal effects on improvement. A proficiency-dependent ceiling effect also emerged. The stated limitations are genre generalizability, short-term scope and possible automation bias.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Renmin University of China](https://md.research.elamur.ai/free/en/universities/renmin-university-of-china/) — University, Beijing, China · 114 works · 16,118 citations
- [Central University of Finance and Economics](https://md.research.elamur.ai/free/en/universities/central-university-of-finance-and-economics/) — University, Beijing, China · 2 works

📚 Shelves (5): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works · [🗄️ Information Systems](https://md.research.elamur.ai/free/en/shelves/computer-science/information-systems/) · 1,437 works

🧭 Topics (2): [🧠 Language models & pretraining](https://md.research.elamur.ai/free/en/topics/language-models/) · 3,320 works · [📏 Evaluation & benchmarks](https://md.research.elamur.ai/free/en/topics/evaluation-benchmarks/) · 1,220 works

## Cite

Cite as: Canran Wang, Yuwen Yang, Zhen Wang, Ming MA, Ding Yu, Chentai Wang, Keman Huang and Xiaoyong Du. 2026. Double-Edged Sword or Sharp Tool? Designing and Evaluating Triadic LLM-Teacher Collaboration for K-12 Writing at Scale. Findings of the Association for Computational Linguistics: ACL 2026.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{wang2026double,
  title = {{Double-Edged Sword or Sharp Tool? Designing and Evaluating Triadic LLM-Teacher Collaboration for K-12 Writing at Scale}},
  author = {Canran Wang, Yuwen Yang, Zhen Wang, Ming MA, Ding Yu, Chentai Wang, Keman Huang and Xiaoyong Du},
  booktitle = {Findings of the Association for Computational Linguistics: ACL 2026},
  year = {2026},
  doi = {10.18653/v1/2026.findings-acl.597},
  url = {https://doi.org/10.18653/v1/2026.findings-acl.597}
}
```

- [← previous DORA: A Dual-Objective Reinforcement Learning Framework for Effective and Efficient Multimodal Agentic Search](https://md.research.elamur.ai/free/en/papers/qin-2026-dora-dual-objective-reinforcement-learning/)
- [next → DPLoRA: A Dual-Pruning Framework based on ILP Optimization and Progressive Pruning for Parameter-Efficient LoRA Fine-Tuning](https://md.research.elamur.ai/free/en/papers/park-2026-dplora-dual-pruning-framework-based/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/28/)

## References

27 references, 3 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Marcos Florencio and Francielle Prieto. 2025. Vibe learning: Education in the age of ai. arXiv preprint arXiv:2511.01956. — [arXiv:2511.01956](https://arxiv.org/abs/2511.01956) [find](https://scholar.google.com/scholar?q=Marcos+Florencio+and+Francielle+Prieto%2E+2025%2E+Vibe+learning%3A+Education+in+the+age+of+ai%2E+arXiv+preprint+arXiv%3A2511%2E01956%2E+%E2%80%94+%5BarXiv%3A2511%2E01956%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2511%2E01956%29)
2. Yanzhu Guo, Guokan Shang, Michalis Vazirgiannis, and Chloé Clavel. 2024. The curious decline of linguistic diversity: Training language models on synthetic text. In Findings of the Association for Computational Linguistics: NAACL 2024, pages 3589– 3604. [find](https://scholar.google.com/scholar?q=Yanzhu+Guo%2C+Guokan+Shang%2C+Michalis+Vazirgiannis%2C+and+Chlo%C3%A9+Clavel%2E+2024%2E+The+curious+decline+of+linguistic+diversity%3A+Training+language+models+on+synthetic+text%2E+In+Findings+of+the+Association+for+Com)
3. Michael Alexander Kirkwood Halliday and Christian MIM Matthiessen. 2013. Halliday’s introduction to functional grammar. Routledge. [find](https://scholar.google.com/scholar?q=Michael+Alexander+Kirkwood+Halliday+and+Christian+MIM+Matthiessen%2E+2013%2E+Halliday%E2%80%99s+introduction+to+functional+grammar%2E+Routledge%2E)
4. Maurice Jakesch, Advait Bhat, Daniel Buschek, Lior Zalmanson, and Mor Naaman. 2023. Co-writing with opinionated language models affects users’ views. In Proceedings of the 2023 CHI conference on human factors in computing systems, pages 1–15. [find](https://scholar.google.com/scholar?q=Maurice+Jakesch%2C+Advait+Bhat%2C+Daniel+Buschek%2C+Lior+Zalmanson%2C+and+Mor+Naaman%2E+2023%2E+Co%2Dwriting+with+opinionated+language+models+affects+users%E2%80%99+views%2E+In+Proceedings+of+the+2023+CHI+conference+on+human)
5. Marwa Abdulhai, Gregory Serapio-Garcia, Clément Crepy, Daria Valter, John Canny, and Natasha Jaques. 2024. [Moral foundations of large language models](https://md.research.elamur.ai/free/en/papers/abdulhai-2024-moral-foundations-large-language-models/). In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 17737–17752.
6. Kowe Kadoma, Marianne Aubin Le Quere, Xiyu Jenny Fu, Christin Munsch, Danaë Metaxa, and Mor Naaman. 2024. The role of inclusion, control, and ownership in workplace ai-mediated communication. In Proceedings of the 2024 CHI Conference on Human Factors in Computing Systems, pages 1–10. [find](https://scholar.google.com/scholar?q=Kowe+Kadoma%2C+Marianne+Aubin+Le+Quere%2C+Xiyu+Jenny+Fu%2C+Christin+Munsch%2C+Dana%C3%AB+Metaxa%2C+and+Mor+Naaman%2E+2024%2E+The+role+of+inclusion%2C+control%2C+and+ownership+in+workplace+ai%2Dmediated+communication%2E+In+Proce)
7. Dhruv Agarwal, Mor Naaman, and Aditya Vashistha. 2025. Ai suggestions homogenize writing toward western styles and diminish cultural nuances. In Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems, pages 1–21. [find](https://scholar.google.com/scholar?q=Dhruv+Agarwal%2C+Mor+Naaman%2C+and+Aditya+Vashistha%2E+2025%2E+Ai+suggestions+homogenize+writing+toward+western+styles+and+diminish+cultural+nuances%2E+In+Proceedings+of+the+2025+CHI+Conference+on+Human+Factors)
8. Jinwei Lu, Yikuan Yan, Keman Huang, Ming Yin, and Fang Zhang. 2025. Do we learn from each other: Understanding the human-ai co-learning process embedded in human-ai collaboration. Group Decision and Negotiation, 34(2):235–271. [find](https://scholar.google.com/scholar?q=Jinwei+Lu%2C+Yikuan+Yan%2C+Keman+Huang%2C+Ming+Yin%2C+and+Fang+Zhang%2E+2025%2E+Do+we+learn+from+each+other%3A+Understanding+the+human%2Dai+co%2Dlearning+process+embedded+in+human%2Dai+collaboration%2E+Group+Decision+and+N)
9. Eman AlGhamdi, Yuheng Li, Dragan Gašević, and Guanliang Chen. 2025. Leveraging prompt-based llms for automated scoring and feedback generation in higher education. Computers & Education, page 105511. [find](https://scholar.google.com/scholar?q=Eman+AlGhamdi%2C+Yuheng+Li%2C+Dragan+Ga%C5%A1evi%C4%87%2C+and+Guanliang+Chen%2E+2025%2E+Leveraging+prompt%2Dbased+llms+for+automated+scoring+and+feedback+generation+in+higher+education%2E+Computers+%26+Education%2C+page+105511%2E)
10. Vishakh Padmakumar and He He. 2023. Does writing with language models reduce content diversity? arXiv preprint arXiv:2309.05196. [find](https://scholar.google.com/scholar?q=Vishakh+Padmakumar+and+He+He%2E+2023%2E+Does+writing+with+language+models+reduce+content+diversity%3F+arXiv+preprint+arXiv%3A2309%2E05196%2E)
11. Wanxiang Che, Yunlong Feng, Libo Qin, and Ting Liu. 2021. N-ltp: An open-source neural language technology platform for chinese. In Proceedings of the 2021 conference on empirical methods in natural language processing: System demonstrations, pages 42–49. [find](https://scholar.google.com/scholar?q=Wanxiang+Che%2C+Yunlong+Feng%2C+Libo+Qin%2C+and+Ting+Liu%2E+2021%2E+N%2Dltp%3A+An+open%2Dsource+neural+language+technology+platform+for+chinese%2E+In+Proceedings+of+the+2021+conference+on+empirical+methods+in+natural+l)
12. Minju Park, Sojung Kim, Seunghyun Lee, Soonwoo Kwon, and Kyuseok Kim. 2024. Empowering personalized learning through a conversation-based tutoring system with student modeling. In Extended Abstracts of the CHI Conference on Human Factors in Computing Systems, pages 1–10. [find](https://scholar.google.com/scholar?q=Minju+Park%2C+Sojung+Kim%2C+Seunghyun+Lee%2C+Soonwoo+Kwon%2C+and+Kyuseok+Kim%2E+2024%2E+Empowering+personalized+learning+through+a+conversation%2Dbased+tutoring+system+with+student+modeling%2E+In+Extended+Abstracts+o)
13. Zhendong Chu, Shen Wang, Jian Xie, Tinghui Zhu, Yibo Yan, Jinheng Ye, Aoxiao Zhong, Xuming Hu, Jing Liang, Philip S Yu, and 1 others. 2025. Llm agents for education: Advances and applications. arXiv preprint arXiv:2503.11733. [find](https://scholar.google.com/scholar?q=Zhendong+Chu%2C+Shen+Wang%2C+Jian+Xie%2C+Tinghui+Zhu%2C+Yibo+Yan%2C+Jinheng+Ye%2C+Aoxiao+Zhong%2C+Xuming+Hu%2C+Jing+Liang%2C+Philip+S+Yu%2C+and+1+others%2E+2025%2E+Llm+agents+for+education%3A+Advances+and+applications%2E+arXiv+p)
14. Flor Miriam Plaza-del Arco, Alba A Cercas Curry, Amanda Cercas Curry, and Dirk Hovy. 2024. Emotion analysis in nlp: Trends, gaps and roadmap for future directions. In Proceedings of the 2024 joint international conference on computational linguistics, language resources and evaluation (LREC-COLING 2… [find](https://scholar.google.com/scholar?q=Flor+Miriam+Plaza%2Ddel+Arco%2C+Alba+A+Cercas+Curry%2C+Amanda+Cercas+Curry%2C+and+Dirk+Hovy%2E+2024%2E+Emotion+analysis+in+nlp%3A+Trends%2C+gaps+and+roadmap+for+future+directions%2E+In+Proceedings+of+the+2024+joint+int)
15. Paramveer S Dhillon, Somayeh Molaei, Jiaqi Li, Maximilian Golub, Shaochun Zheng, and Lionel Peter Robert. 2024. Shaping human-ai collaboration: Varied scaffolding levels in co-writing with language models. In Proceedings of the 2024 CHI Conference on Human Factors in Computing Systems, pages 1– 18. [find](https://scholar.google.com/scholar?q=Paramveer+S+Dhillon%2C+Somayeh+Molaei%2C+Jiaqi+Li%2C+Maximilian+Golub%2C+Shaochun+Zheng%2C+and+Lionel+Peter+Robert%2E+2024%2E+Shaping+human%2Dai+collaboration%3A+Varied+scaffolding+levels+in+co%2Dwriting+with+language+mo)
16. Nils Reimers and Iryna Gurevych. 2019. [Sentence-bert: Sentence embeddings using siamese bert-networks](https://md.research.elamur.ai/free/en/papers/reimers-2019-sentence-bert-sentence-embeddings-siamese/). In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 3982–3992.
17. Anil R Doshi and Oliver P Hauser. 2024. Generative ai enhances individual creativity but reduces the collective diversity of novel content. Science advances, 10(28):eadn5290. [find](https://scholar.google.com/scholar?q=Anil+R+Doshi+and+Oliver+P+Hauser%2E+2024%2E+Generative+ai+enhances+individual+creativity+but+reduces+the+collective+diversity+of+novel+content%2E+Science+advances%2C+10%2828%29%3Aeadn5290%2E)
18. Nino Shervashidze, Pascal Schweitzer, Erik Jan Van Leeuwen, Kurt Mehlhorn, and Karsten M Borgwardt. 2011. Weisfeiler-lehman graph kernels. Journal of Machine Learning Research, 12(9). [find](https://scholar.google.com/scholar?q=Nino+Shervashidze%2C+Pascal+Schweitzer%2C+Erik+Jan+Van+Leeuwen%2C+Kurt+Mehlhorn%2C+and+Karsten+M+Borgwardt%2E+2011%2E+Weisfeiler%2Dlehman+graph+kernels%2E+Journal+of+Machine+Learning+Research%2C+12%289%29%2E)
19. Mostafa Faghih Shojaei, Rahul Gulati, Benjamin A Jasperson, Shangshang Wang, Simone Cimolato, Dangli Cao, Willie Neiswanger, and Krishna Garikipati. 2025. Ai-university: An llm-based platform for instructional alignment to scientific classrooms. arXiv preprint arXiv:2504.08846. — [arXiv:2504.08846](https://arxiv.org/abs/2504.08846) [find](https://scholar.google.com/scholar?q=Mostafa+Faghih+Shojaei%2C+Rahul+Gulati%2C+Benjamin+A+Jasperson%2C+Shangshang+Wang%2C+Simone+Cimolato%2C+Dangli+Cao%2C+Willie+Neiswanger%2C+and+Krishna+Garikipati%2E+2025%2E+Ai%2Duniversity%3A+An+llm%2Dbased+platform+for+inst)
20. Shirish Sundaresan and Isin Guler. 2025. Algorithmic recommendation tools and experiential learning in clinical care. Organization Science. [find](https://scholar.google.com/scholar?q=Shirish+Sundaresan+and+Isin+Guler%2E+2025%2E+Algorithmic+recommendation+tools+and+experiential+learning+in+clinical+care%2E+Organization+Science%2E)
21. Ruixiang Tang, Yu-Neng Chuang, and Xia Hu. 2024. The science of detecting llm-generated text. Communications of the ACM, 67(4):50–59. [find](https://scholar.google.com/scholar?q=Ruixiang+Tang%2C+Yu%2DNeng+Chuang%2C+and+Xia+Hu%2E+2024%2E+The+science+of+detecting+llm%2Dgenerated+text%2E+Communications+of+the+ACM%2C+67%284%29%3A50%E2%80%9359%2E)
22. Dawei Wang, Difang Huang, Haipeng Shen, and Brian Uzzi. 2025. A large-scale comparison of divergent creativity in humans and large language models. Nature Human Behaviour, pages 1–10. [find](https://scholar.google.com/scholar?q=Dawei+Wang%2C+Difang+Huang%2C+Haipeng+Shen%2C+and+Brian+Uzzi%2E+2025%2E+A+large%2Dscale+comparison+of+divergent+creativity+in+humans+and+large+language+models%2E+Nature+Human+Behaviour%2C+pages+1%E2%80%9310%2E)
23. Shen Wang, Tianlong Xu, Hang Li, Chaoli Zhang, Joleen Liang, Jiliang Tang, Philip S Yu, and Qingsong Wen. 2024. Large language models for education: A survey and outlook. arXiv preprint arXiv:2403.18105. [find](https://scholar.google.com/scholar?q=Shen+Wang%2C+Tianlong+Xu%2C+Hang+Li%2C+Chaoli+Zhang%2C+Joleen+Liang%2C+Jiliang+Tang%2C+Philip+S+Yu%2C+and+Qingsong+Wen%2E+2024%2E+Large+language+models+for+education%3A+A+survey+and+outlook%2E+arXiv+preprint+arXiv%3A2403%2E181)
24. Yiran Wu, Feiran Jia, Shaokun Zhang, Hangyu Li, Erkang Zhu, Yue Wang, Yin Tat Lee, Richard Peng, Qingyun Wu, and Chi Wang. 2023. [Mathchat: Converse to tackle challenging math problems with llm agents](https://md.research.elamur.ai/free/en/papers/wu-2023-mathchat-converse-tackle-challenging-math/). arXiv preprint arXiv:2306.01337.
25. Songlin Xu, Hao-Ning Wen, Hongyi Pan, Dallas Dominguez, Dongyin Hu, and Xinyu Zhang. 2025. Classroom simulacra: Building contextual student generative agents in online education for learning behavioral simulation. In Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems, pages… [find](https://scholar.google.com/scholar?q=Songlin+Xu%2C+Hao%2DNing+Wen%2C+Hongyi+Pan%2C+Dallas+Dominguez%2C+Dongyin+Hu%2C+and+Xinyu+Zhang%2E+2025%2E+Classroom+simulacra%3A+Building+contextual+student+generative+agents+in+online+education+for+learning+behaviora)
26. Bo Yang, Yongqiang Sun, Zihan Zeng, and Qinwei Li. 2026. Deskilling, reskilling, or upskilling? unpacking the pathways of student adaptation to generative artificial intelligence. International Journal of Information Management, 87:103002. [find](https://scholar.google.com/scholar?q=Bo+Yang%2C+Yongqiang+Sun%2C+Zihan+Zeng%2C+and+Qinwei+Li%2E+2026%2E+Deskilling%2C+reskilling%2C+or+upskilling%3F+unpacking+the+pathways+of+student+adaptation+to+generative+artificial+intelligence%2E+International+Journa)
27. Ann Yuan, Andy Coenen, Emily Reif, and Daphne Ippolito. 2022. Wordcraft: story writing with large language models. In Proceedings of the 27th International Conference on Intelligent User Interfaces, pages 841–852. [find](https://scholar.google.com/scholar?q=Ann+Yuan%2C+Andy+Coenen%2C+Emily+Reif%2C+and+Daphne+Ippolito%2E+2022%2E+Wordcraft%3A+story+writing+with+large+language+models%2E+In+Proceedings+of+the+27th+International+Conference+on+Intelligent+User+Interfaces%2C+p)

*Updated 1 November 2026*