---
title: "Utterance-level Detection Framework for LLM-Involved Content"
description: "Most detectors of machine-generated text are built for static, document-level inputs. 98.14% accuracy. 24 references."
authors: ["Muyang Zhou", "Huaxia Rui"]
year: 2026
venue: "Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers)"
source: "https://aclanthology.org/2026.eacl-long.63/"
doi: "10.18653/v1/2026.eacl-long.63"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/zhou-2026-utterance-level-detection-framework-llm/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/zhou-2026-utterance-level-detection-framework-llm/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/55/) / Utterance-level Detection Framework for LLM-Involved Content Detection in Conversational Setting

# Utterance-level Detection Framework for LLM-Involved Content Detection in Conversational Setting

**Authors:** Muyang Zhou and Huaxia Rui · **Year:** 2026 · **Venue:** [Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers)](https://md.research.elamur.ai/free/en/venues/eacl/) ([EACL 2026](https://md.research.elamur.ai/free/en/venues/eacl/2026/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2026.eacl-long.63/) · [PDF](https://aclanthology.org/2026.eacl-long.63.pdf) · [DOI 10.18653/v1/2026.eacl-long.63](https://doi.org/10.18653/v1/2026.eacl-long.63) · [OpenAlex](https://openalex.org/W7140113150)

**Citations:** 0 citations (OpenAlex, 2026-10-14)

## Summary
Most detectors of machine-generated text are built for static, document-level inputs. They handle short, interactive exchanges poorly, especially when human and model turns are mixed in one dialogue. The authors reframe detection at the utterance level within conversations and supply both a dataset and a model. The dataset combines real human-human dialogues from Wizard of Wikipedia, Persona-Chat and MuTual with two kinds of synthetic dialogues. In LLM-LLM dialogues, two persona-prompted assistants converse turn by turn. In human-LLM dialogues, people chat with a model on summarized topics.

The detector is lightweight. It uses the LLaMA3 tokenizer with a small custom transformer, arranged recurrently over dialogue turns. It mixes self-attention and cross-attention between the two speakers. A detection focus augmentation strategy emphasizes the second speaker's replies while keeping the first speaker's turns as context.

On human-LLM dialogues it reaches 98.14% accuracy with fast inference and outperforms baseline detectors. It also generalizes reasonably in zero-shot tests on out-of-domain data. This makes it a practical candidate for real-time transparency checks in chat systems.

## Key points
- **Dataset:** over 80,000 training dialogues in three parts (human-human, LLM-LLM, human-LLM).
  - Synthetic turns come from GPT-4o, LLaMA-3.1-70B-Instruct and Qwen2.5-72B-Instruct.
  - Turn counts match those of the real human conversations.
- **Model:** LLaMA3 tokenizer feeding a custom 4-layer transformer for feature extraction, followed by self-attention and cross-attention blocks in a recurrent arrangement with a decoder at each stage.
- **Results and ablations:**
  - 98.14% accuracy on human-LLM dialogues.
  - The LLM tokenizer adds about 9% accuracy compared with a traditional tokenizer.
  - Accuracy exceeds 90% on some out-of-domain tasks.
- **Dialogue length:** training uses 5-round dialogues.
  - Shorter inputs are padded with blank content.
  - Longer ones are split into 5-round sliding windows whose predictions are combined by voting.
  - Compared with models trained on 1–10 rounds, performance remains comparable.
- **Limitations:**
  - Covers only two-party dialogues.
  - May be weaker in specialized domains such as medical or legal.
  - Has notable compute cost.
  - Cross-lingual and multi-party detection are left as future work, which matters for applying it to Ukrainian.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [University of Rochester](https://md.research.elamur.ai/free/en/universities/university-of-rochester/) — University, Rochester, United States · 43 works · 7,874 citations

📚 Shelves (6): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 18,614 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 17,965 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 17,450 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 17,296 works · [👁️ Computer Vision and Pattern Recognition](https://md.research.elamur.ai/free/en/shelves/computer-science/computer-vision-and-pattern-recognition/) · 3,486 works · [🗄️ Information Systems](https://md.research.elamur.ai/free/en/shelves/computer-science/information-systems/) · 1,730 works

🧭 Topics (1): [🗨️ Dialogue systems](https://md.research.elamur.ai/free/en/topics/dialogue-systems/) · 654 works

## Cite

Cite as: Muyang Zhou and Huaxia Rui. 2026. Utterance-level Detection Framework for LLM-Involved Content Detection in Conversational Setting. Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{zhou2026utterance,
  title = {{Utterance-level Detection Framework for LLM-Involved Content Detection in Conversational Setting}},
  author = {Muyang Zhou and Huaxia Rui},
  booktitle = {Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2026},
  doi = {10.18653/v1/2026.eacl-long.63},
  url = {https://doi.org/10.18653/v1/2026.eacl-long.63}
}
```

- [← previous Uslub at AbjadAuthorID Shared Task: A Comparative Analysis of Traditional Machine Learning and Transformer-Based Models for Authorship Attribution in Arabic and Urdu](https://md.research.elamur.ai/free/en/papers/alsuhaibani-2026-uslub-abjadauthorid-shared-task-comparative/)
- [next → UWB at SemEval-2026 Task 5: Synsets and their contexts](https://md.research.elamur.ai/free/en/papers/taylor-2026-uwb-semeval-2026-task-5/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/55/)

## References

24 references, 11 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. 2023. [Gpt-4 technical report](https://md.research.elamur.ai/free/en/papers/openai-2023-gpt-4-technical-report/). arXiv preprint arXiv:2303.08774.
2. Guangsheng Bao, Yanbin Zhao, Zhiyang Teng, Linyi Yang, and Yue Zhang. 2023. [Fast-detectgpt: Efficient zero-shot detection of machine-generated text via conditional probability curvature](https://md.research.elamur.ai/free/en/papers/bao-2023-fast-detectgpt-efficient-zero-shot/). arXiv preprint arXiv:2310.05130.
3. Yutian Chen, Hao Kang, Vivian Zhai, Liangze Li, Rita Singh, and Bhiksha Raj. 2023. Token prediction as implicit classification to identify llm-generated text. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, page 13112–13120. Association for Computational Li… [find](https://scholar.google.com/scholar?q=Yutian+Chen%2C+Hao+Kang%2C+Vivian+Zhai%2C+Liangze+Li%2C+Rita+Singh%2C+and+Bhiksha+Raj%2E+2023%2E+Token+prediction+as+implicit+classification+to+identify+llm%2Dgenerated+text%2E+In+Proceedings+of+the+2023+Conference+on+)
4. Leyang Cui, Yu Wu, Shujie Liu, Yue Zhang, and Ming Zhou. 2020. [MuTual: A dataset for multi-turn dialogue reasoning](https://md.research.elamur.ai/free/en/papers/cui-2020-mutual-dataset-multi-turn-dialogue/). In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 1406–1416, Online. Association for Computational Linguistics.
5. Emily Dinan et al. 2018. [Wizard of wikipedia: Knowledge-powered conversational agents](https://md.research.elamur.ai/free/en/papers/dinan-2018-wizard-wikipedia-knowledge-powered-conversational/). In Proceedings of ICLR.
6. Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al. 2024. [The llama 3 herd of models](https://md.research.elamur.ai/free/en/papers/grattafiori-2024-llama-3-herd-models/). arXiv preprint arXiv:2407.21783.
7. Tiziano Fagni, Fabrizio Falchi, Margherita Gambini, Antonio Martella, and Maurizio Tesconi. 2021. [Tweepfake: About detecting deepfake tweets](https://md.research.elamur.ai/free/en/papers/fagni-2020-tweepfake-about-detecting-deepfake-tweets/). Plos one, 16(5):e0251415.
8. Pengyu Wang, Linyang Li, Ke Ren, Botian Jiang, Dong Zhang, and Xipeng Qiu. 2023. Seqxgpt: Sentence-level ai-generated text detection. arXiv preprint arXiv:2310.08903. [find](https://scholar.google.com/scholar?q=Pengyu+Wang%2C+Linyang+Li%2C+Ke+Ren%2C+Botian+Jiang%2C+Dong+Zhang%2C+and+Xipeng+Qiu%2E+2023%2E+Seqxgpt%3A+Sentence%2Dlevel+ai%2Dgenerated+text+detection%2E+arXiv+preprint+arXiv%3A2310%2E08903%2E)
9. Deepanway Ghosal, Navonil Majumder, Soujanya Poria, Niyati Chhaya, and Alexander Gelbukh. 2019. [Dialoguegcn: A graph convolutional neural network for emotion recognition in conversation](https://md.research.elamur.ai/free/en/papers/ghosal-2019-dialoguegcn-graph-convolutional-neural-network/). Preprint, arXiv:1908.11540.
10. An Yang, Baosong Yang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Zhou, Chengpeng Li, Chengyuan Li, Dayiheng Liu, Fei Huang, et al. 2024. Qwen2 technical report. arXiv preprint arXiv:2407.10671. [find](https://scholar.google.com/scholar?q=An+Yang%2C+Baosong+Yang%2C+Binyuan+Hui%2C+Bo+Zheng%2C+Bowen+Yu%2C+Chang+Zhou%2C+Chengpeng+Li%2C+Chengyuan+Li%2C+Dayiheng+Liu%2C+Fei+Huang%2C+et+al%2E+2024%2E+Qwen2+technical+report%2E+arXiv+preprint+arXiv%3A2407%2E10671%2E)
11. Google. 2025. Gemini 2.0 flash. [https://blog.google/technology/google-deepmind/](https://blog.google/technology/google-deepmind/) google-gemini-ai-update-december-2024/ \#gemini-2-0-flash. [↗](https://blog.google/technology/google-deepmind/)
12. Xiaodong Gu, Kang Min Yoo, and Jung-Woo Ha. 2021. Dialogbert: Discourse-aware response generation via learning to recover and rank utterances. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pages 12911–12919. [find](https://scholar.google.com/scholar?q=Xiaodong+Gu%2C+Kang+Min+Yoo%2C+and+Jung%2DWoo+Ha%2E+2021%2E+Dialogbert%3A+Discourse%2Daware+response+generation+via+learning+to+recover+and+rank+utterances%2E+In+Proceedings+of+the+AAAI+Conference+on+Artificial+Intel)
13. Biyang Guo, Xin Zhang, Ziyuan Wang, Minqi Jiang, Jinran Nie, Yuxuan Ding, Jianwei Yue, and Yupeng Wu. 2023. [How close is chatgpt to human experts? comparison corpus, evaluation, and detection](https://md.research.elamur.ai/free/en/papers/guo-2023-close-chatgpt-human-experts-comparison/). Preprint, arXiv:2301.07597.
14. Aixin Liu, Bei Feng, Bing Xue, Bingxuan Wang, Bochao Wu, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, et al. 2024. Deepseek-v3 technical report. arXiv preprint arXiv:2412.19437. [find](https://scholar.google.com/scholar?q=Aixin+Liu%2C+Bei+Feng%2C+Bing+Xue%2C+Bingxuan+Wang%2C+Bochao+Wu%2C+Chengda+Lu%2C+Chenggang+Zhao%2C+Chengqi+Deng%2C+Chenyu+Zhang%2C+Chong+Ruan%2C+et+al%2E+2024%2E+Deepseek%2Dv3+technical+report%2E+arXiv+preprint+arXiv%3A2412%2E19437%2E)
15. Navonil Majumder, Soujanya Poria, Devamanyu Hazarika, Rada Mihalcea, Alexander Gelbukh, and Erik Cambria. 2019. Dialoguernn: An attentive rnn for emotion detection in conversations. In Proceedings of the AAAI conference on artificial intelligence, volume 33, pages 6818–6825. [find](https://scholar.google.com/scholar?q=Navonil+Majumder%2C+Soujanya+Poria%2C+Devamanyu+Hazarika%2C+Rada+Mihalcea%2C+Alexander+Gelbukh%2C+and+Erik+Cambria%2E+2019%2E+Dialoguernn%3A+An+attentive+rnn+for+emotion+detection+in+conversations%2E+In+Proceedings+of+)
16. Peipeng Yu, Jiahan Chen, Xuan Feng, and Zhihua Xia. 2024. Cheat: A large-scale dataset for detecting chatgpt-written abstracts. Preprint, arXiv:2304.12008. — [arXiv:2304.12008](https://arxiv.org/abs/2304.12008) [find](https://scholar.google.com/scholar?q=Peipeng+Yu%2C+Jiahan+Chen%2C+Xuan+Feng%2C+and+Zhihua+Xia%2E+2024%2E+Cheat%3A+A+large%2Dscale+dataset+for+detecting+chatgpt%2Dwritten+abstracts%2E+Preprint%2C+arXiv%3A2304%2E12008%2E+%E2%80%94+%5BarXiv%3A2304%2E12008%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2)
17. Rowan Zellers, Ari Holtzman, Hannah Rashkin, Yonatan Bisk, Ali Farhadi, Franziska Roesner, and Yejin Choi. 2019. [Defending against neural fake news](https://md.research.elamur.ai/free/en/papers/zellers-2019-defending-against-neural-fake-news/). Advances in neural information processing systems, 32.
18. Zijie Zeng, Lele Sha, Yuheng Li, Kaixun Yang, Dragan Gašević, and Guanliang Chen. 2023. Towards automatic boundary detection for human-ai collaborative hybrid essay in education. Preprint, arXiv:2307.12267. — [arXiv:2307.12267](https://arxiv.org/abs/2307.12267) [find](https://scholar.google.com/scholar?q=Zijie+Zeng%2C+Lele+Sha%2C+Yuheng+Li%2C+Kaixun+Yang%2C+Dragan+Ga%C5%A1evic%CC%81%2C+and+Guanliang+Chen%2E+2023%2E+Towards+automatic+boundary+detection+for+human%2Dai+collaborative+hybrid+essay+in+education%2E+Preprint%2C+arXiv%3A2307)
19. Bowen Zhang, Xiaofei Xu, Xutao Li, Yunming Ye, Xiaojun Chen, and Zhongjie Wang. 2020. A memory network based end-to-end personalized task-oriented dialogue generation. Knowledge-Based Systems, 207:106398. [find](https://scholar.google.com/scholar?q=Bowen+Zhang%2C+Xiaofei+Xu%2C+Xutao+Li%2C+Yunming+Ye%2C+Xiaojun+Chen%2C+and+Zhongjie+Wang%2E+2020%2E+A+memory+network+based+end%2Dto%2Dend+personalized+task%2Doriented+dialogue+generation%2E+Knowledge%2DBased+Systems%2C+207%3A106)
20. Saizheng Zhang et al. 2018. [Personalizing dialogue agents: I have a dog, do you have pets too?](https://md.research.elamur.ai/free/en/papers/zhang-2018-personalizing-dialogue-agents-i-have/) In Proceedings of ACL.
21. Eric Mitchell, Yoonho Lee, Alexander Khazatsky, Christopher D. Manning, and Chelsea Finn. 2023. [Detectgpt: Zero-shot machine-generated text detection using probability curvature](https://md.research.elamur.ai/free/en/papers/mitchell-2023-detectgpt-zero-shot-machine-generated/). Preprint, arXiv:2301.11305.
22. Zhenpeng Su, Xing Wu, Wei Zhou, Guangyuan Ma, and Songlin Hu. 2024. Hc3 plus: A semantic-invariant human chatgpt comparison corpus. Preprint, arXiv:2309.02731. — [arXiv:2309.02731](https://arxiv.org/abs/2309.02731) [find](https://scholar.google.com/scholar?q=Zhenpeng+Su%2C+Xing+Wu%2C+Wei+Zhou%2C+Guangyuan+Ma%2C+and+Songlin+Hu%2E+2024%2E+Hc3+plus%3A+A+semantic%2Dinvariant+human+chatgpt+comparison+corpus%2E+Preprint%2C+arXiv%3A2309%2E02731%2E+%E2%80%94+%5BarXiv%3A2309%2E02731%5D%28https%3A%2F%2Farxiv%2Eorg%2Fa)
23. Jiamin Wang, Xiao Sun, Qian Chen, and Meng Wang. 2022. Information-enhanced hierarchical self-attention network for multiturn dialog generation. IEEE Transactions on Computational Social Systems, 10(5):2686–2697. [find](https://scholar.google.com/scholar?q=Jiamin+Wang%2C+Xiao+Sun%2C+Qian+Chen%2C+and+Meng+Wang%2E+2022%2E+Information%2Denhanced+hierarchical+self%2Dattention+network+for+multiturn+dialog+generation%2E+IEEE+Transactions+on+Computational+Social+Systems%2C+10%285)
24. Jian Wang, Junhao Liu, Wei Bi, Xiaojiang Liu, Kejing He, Ruifeng Xu, and Min Yang. 2020. Dual dynamic memory network for end-to-end multi-turn task-oriented dialog systems. In Proceedings of the 28th international conference on computational linguistics, pages 4100–4110. [find](https://scholar.google.com/scholar?q=Jian+Wang%2C+Junhao+Liu%2C+Wei+Bi%2C+Xiaojiang+Liu%2C+Kejing+He%2C+Ruifeng+Xu%2C+and+Min+Yang%2E+2020%2E+Dual+dynamic+memory+network+for+end%2Dto%2Dend+multi%2Dturn+task%2Doriented+dialog+systems%2E+In+Proceedings+of+the+28th+)

*Updated 2 November 2026*