---
title: "DIDECO: An Annotated Dataset for Intent — Popovic 2026"
description: "Phishing and other social-engineering emails are increasingly written by LLMs."
authors: ["Senaid Popovic", "Damien Riquet", "Maxime Meyer", "Fabien Lauer", "Yannick Parmentier"]
year: 2026
venue: "Proceedings of the Fifteenth Language Resources and Evaluation Conference"
source: "https://aclanthology.org/2026.lrec-1.542/"
doi: "10.63317/3xsu5rdnw8hi"
licence: "CC BY-NC 4.0 (ELRA, LREC proceedings)"
licence_url: "https://creativecommons.org/licenses/by-nc/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/popovic-2026-dideco-annotated-dataset-intent-detection/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/popovic-2026-dideco-annotated-dataset-intent-detection/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/5/) / DIDECO: An Annotated Dataset for Intent Detection in Digital Communications

# DIDECO: An Annotated Dataset for Intent Detection in Digital Communications

**Authors:** Senaid Popovic, Damien Riquet, Maxime Meyer, Fabien Lauer, [Yannick Parmentier](https://md.research.elamur.ai/free/en/authors/yannick-parmentier/) · **Year:** 2026 · **Venue:** Proceedings of the Fifteenth Language Resources and Evaluation Conference · **Licence:** [CC BY-NC 4.0 (ELRA, LREC proceedings)](https://creativecommons.org/licenses/by-nc/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2026.lrec-1.542/) · [PDF](https://aclanthology.org/2026.lrec-1.542.pdf) · [DOI 10.63317/3xsu5rdnw8hi](https://doi.org/10.63317/3xsu5rdnw8hi) · [OpenAlex](https://openalex.org/W7160193713)

**Citations:** 0 citations (OpenAlex, 2026-10-14)

## Summary
Phishing and other social-engineering emails are increasingly written by LLMs. They can look fully legitimate, so keyword- or signature-based filters miss them. The authors argue that detectors need to understand intent. Until now, no annotated resource has covered both the stated goal of a message and the hidden pressure tactics behind it. To fill this gap, they built DIDECO.

They first designed a hierarchical taxonomy with two branches:
- **Explicit intents**, meaning what the sender asks for, informed by Speech Act Theory.
- **Implicit intents**, meaning how the sender pushes the reader to comply, drawn from Cialdini's persuasion principles and research on scams.

Six trained annotators then applied this taxonomy to LLM-generated spear-phishing emails. Each email could receive several labels, and annotators followed published guidelines.

The analysis shows that a typical malicious message stacks several intents, pairing an explicit request with multiple persuasion levers. Annotators agreed far more on explicit labels than on implicit ones, which highlights how difficult it is to annotate pragmatic, indirect meaning. The resource is intended as a basis for intent-aware security classifiers.

## Key points
- **Taxonomy:** 20 intent categories in total. The explicit branch has four groups: Information Manipulation, Resource Acquisition, Trust Establishment and Call to Action. The implicit branch has five: Authority, Social Proof, Liking/Similarity/Deception, Distraction, and Commitment/Integrity/Reciprocation.
- **Corpus:** 220 LLM-generated spear-phishing emails, labelled by six annotators with a multi-label scheme, yielding 2,162 intent annotations.
- **Agreement:** measured on 169 emails that had at least two annotators. Krippendorff's α was 0.407 and mean pairwise Cohen's κ was 0.355.
- **Explicit vs. implicit agreement:** κ reached 0.577 for explicit intents but only 0.111 for implicit ones.
- **Availability:** the full annotation guidelines, with category definitions, identification criteria and examples, are released in the DIDECO repository.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

Fields: [Linguistics & languages](https://md.research.elamur.ai/free/en/shelves/linguistics/)

## Cite

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{popovic2026dideco,
  title = {{DIDECO: An Annotated Dataset for Intent Detection in Digital Communications}},
  author = {Popovic, Senaid and Riquet, Damien and Meyer, Maxime and Lauer, Fabien and Parmentier, Yannick},
  booktitle = {Proceedings of the Fifteenth Language Resources and Evaluation Conference},
  year = {2026},
  doi = {10.63317/3xsu5rdnw8hi},
  url = {https://doi.org/10.63317/3xsu5rdnw8hi}
}
```

- [← previous DialogueSidon: Recovering Full-Duplex Dialogue Tracks from In-the-Wild Dialogue Audio](https://md.research.elamur.ai/free/en/papers/nakata-2026-dialoguesidon-recovering-full-duplex-dialogue/)
- [next → Diff4TST: Masked Diffusion Language Model for Text Style Transfer](https://md.research.elamur.ai/free/en/papers/ma-2026-diff4tst-masked-diffusion-language-model/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/5/)

## References

31 references, 7 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. John Langshaw Austin. 1962. How to Do Things with Words. Clarendon Press, Oxford [Eng.]. [find](https://scholar.google.com/scholar?q=John+Langshaw+Austin%2E+1962%2E+How+to+Do+Things+with+Words%2E+Clarendon+Press%2C+Oxford+%5BEng%2E%5D%2E)
2. Davide Bassi, Søren Fomsgaard, and Martín Pereira-Fariña. 2024. Decoding persuasion: a survey on ML and NLP methods for the study of online persuasion. Frontiers in Communication, 9. [find](https://scholar.google.com/scholar?q=Davide+Bassi%2C+S%C3%B8ren+Fomsgaard%2C+and+Mart%C3%ADn+Pereira%2DFari%C3%B1a%2E+2024%2E+Decoding+persuasion%3A+a+survey+on+ML+and+NLP+methods+for+the+study+of+online+persuasion%2E+Frontiers+in+Communication%2C+9%2E)
3. Paul N. Bennett and Jaime Carbonell. 2005. Detecting action-items in e-mail. In Proceedings of the 28th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR ’05, page 585–586, New York, NY, USA. Association for Computing Machinery. [find](https://scholar.google.com/scholar?q=Paul+N%2E+Bennett+and+Jaime+Carbonell%2E+2005%2E+Detecting+action%2Ditems+in+e%2Dmail%2E+In+Proceedings+of+the+28th+Annual+International+ACM+SIGIR+Conference+on+Research+and+Development+in+Information+Retrieval%2C+)
4. Caleb T Carr and Rebecca A Hayes. 2017. Social media: Defining, developing, and divining. Atlantic Journal of Communication, 23(1):46–65. [find](https://scholar.google.com/scholar?q=Caleb+T+Carr+and+Rebecca+A+Hayes%2E+2017%2E+Social+media%3A+Defining%2C+developing%2C+and+divining%2E+Atlantic+Journal+of+Communication%2C+23%281%29%3A46%E2%80%9365%2E)
5. Vitor R. Carvalho and William W. Cohen. 2005. On the collective classification of email "speech acts". In Proceedings of the 28th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR ’05, page 345–352, New York, NY, USA. Association for Computing Mach… [find](https://scholar.google.com/scholar?q=Vitor+R%2E+Carvalho+and+William+W%2E+Cohen%2E+2005%2E+On+the+collective+classification+of+email+%22speech+acts%22%2E+In+Proceedings+of+the+28th+Annual+International+ACM+SIGIR+Conference+on+Research+and+Development+)
6. Iñigo Casanueva, Tadas Temčinas, Daniela Gerz, Matthew Henderson, and Ivan Vulić. 2020. Efficient intent detection with dual sentence encoders. In Proceedings of the 2nd Workshop on Natural Language Processing for Conversational AI, pages 38–45, Online. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=I%C3%B1igo+Casanueva%2C+Tadas+Tem%C4%8Dinas%2C+Daniela+Gerz%2C+Matthew+Henderson%2C+and+Ivan+Vuli%C4%87%2E+2020%2E+Efficient+intent+detection+with+dual+sentence+encoders%2E+In+Proceedings+of+the+2nd+Workshop+on+Natural+Language+P) [our card](https://md.research.elamur.ai/free/en/papers/casanueva-2020-efficient-intent-detection-dual-sentence/)
7. Robert B. Cialdini. 1993. Influence: the psychology of persuasion, rev. edition. Quill, New York. [find](https://scholar.google.com/scholar?q=Robert+B%2E+Cialdini%2E+1993%2E+Influence%3A+the+psychology+of+persuasion%2C+rev%2E+edition%2E+Quill%2C+New+York%2E)
8. William W. Cohen, Vitor R. Carvalho, and Tom M. Mitchell. 2004. Learning to classify email into “speech acts”. In Proceedings of the 2004 Conference on Empirical Methods in Natural Language Processing, pages 309–316, Barcelona, Spain. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=William+W%2E+Cohen%2C+Vitor+R%2E+Carvalho%2C+and+Tom+M%2E+Mitchell%2E+2004%2E+Learning+to+classify+email+into+%E2%80%9Cspeech+acts%E2%80%9D%2E+In+Proceedings+of+the+2004+Conference+on+Empirical+Methods+in+Natural+Language+Processing) [our card](https://md.research.elamur.ai/free/en/papers/cohen-2004-learning-classify-email-into-speech/)
9. Alice Coucke, Alaa Saade, Adrien Ball, Théodore Bluche, Alexandre Caulier, David Leroy, Clément Doumouro, Thibault Gisselbrecht, Francesco Caltagirone, Thibaut Lavril, et al. 2018. Snips voice platform: an embedded spoken language understanding system for private-by-design voice interfaces. In Priva… [find](https://scholar.google.com/scholar?q=Alice+Coucke%2C+Alaa+Saade%2C+Adrien+Ball%2C+Th%C3%A9odore+Bluche%2C+Alexandre+Caulier%2C+David+Leroy%2C+Cl%C3%A9ment+Doumouro%2C+Thibault+Gisselbrecht%2C+Francesco+Caltagirone%2C+Thibaut+Lavril%2C+et+al%2E+2018%2E+Snips+voice+platfor) [our card](https://md.research.elamur.ai/free/en/papers/coucke-2018-snips-voice-platform-embedded-spoken/)
10. Giovanni Da San Martino, Alberto Barrón-Cedeño, and Preslav Nakov. 2019. Findings of the NLP4IF-2019 shared task on fine-grained propaganda detection. In Proceedings of the Second Workshop on Natural Language Processing for Internet Freedom: Censorship, Disinformation, and Propaganda, pages 162–170. [find](https://scholar.google.com/scholar?q=Giovanni+Da+San+Martino%2C+Alberto+Barr%C3%B3n%2DCede%C3%B1o%2C+and+Preslav+Nakov%2E+2019%2E+Findings+of+the+NLP4IF%2D2019+shared+task+on+fine%2Dgrained+propaganda+detection%2E+In+Proceedings+of+the+Second+Workshop+on+Natural+)
11. Giovanni Da San Martino, Alberto Barrón-Cedeño, Henning Wachsmuth, Rostislav Petrov, and Preslav Nakov. 2020. Semeval-2020 task 11: Detection of propaganda techniques in news articles. In Proceedings of the Fourteenth Workshop on Semantic Evaluation, pages 1377–1414. [find](https://scholar.google.com/scholar?q=Giovanni+Da+San+Martino%2C+Alberto+Barr%C3%B3n%2DCede%C3%B1o%2C+Henning+Wachsmuth%2C+Rostislav+Petrov%2C+and+Preslav+Nakov%2E+2020%2E+Semeval%2D2020+task+11%3A+Detection+of+propaganda+techniques+in+news+articles%2E+In+Proceedings+) [our card](https://md.research.elamur.ai/free/en/papers/martino-2020-semeval-2020-task-11-detection/)
12. Laura A. Dabbish and Robert E. Kraut. 2006. Email overload at work: an analysis of factors associated with email strain. In Proceedings of the 2006 20th Anniversary Conference on Computer Supported Cooperative Work, CSCW ’06, page 431–440, New York, NY, USA. Association for Computing Machinery. [find](https://scholar.google.com/scholar?q=Laura+A%2E+Dabbish+and+Robert+E%2E+Kraut%2E+2006%2E+Email+overload+at+work%3A+an+analysis+of+factors+associated+with+email+strain%2E+In+Proceedings+of+the+2006+20th+Anniversary+Conference+on+Computer+Supported+Co)
13. Laura A. Dabbish, Robert E. Kraut, Susan Fussell, and Sara Kiesler. 2005. Understanding email use: predicting action on a message. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI ’05, page 691–700, New York, NY, USA. Association for Computing Machinery. [find](https://scholar.google.com/scholar?q=Laura+A%2E+Dabbish%2C+Robert+E%2E+Kraut%2C+Susan+Fussell%2C+and+Sara+Kiesler%2E+2005%2E+Understanding+email+use%3A+predicting+action+on+a+message%2E+In+Proceedings+of+the+SIGCHI+Conference+on+Human+Factors+in+Computing)
14. Federal Bureau of Investigation. 2023. Internet crime complaint center (ic3) \| business e-mail compromise (bec) \| ic3.gov. [find](https://scholar.google.com/scholar?q=Federal+Bureau+of+Investigation%2E+2023%2E+Internet+crime+complaint+center+%28ic3%29+%5C%7C+business+e%2Dmail+compromise+%28bec%29+%5C%7C+ic3%2Egov%2E)
15. Ana Ferreira and Soraia Teles. 2019. Persuasion: How phishing emails can influence users and bypass security measures. Int. J. Hum.-Comput. Stud., 125(C):19–31. [find](https://scholar.google.com/scholar?q=Ana+Ferreira+and+Soraia+Teles%2E+2019%2E+Persuasion%3A+How+phishing+emails+can+influence+users+and+bypass+security+measures%2E+Int%2E+J%2E+Hum%2E%2DComput%2E+Stud%2E%2C+125%28C%29%3A19%E2%80%9331%2E)
16. Michael C. Frank and Noah D. Goodman. 2012. Predicting pragmatic reasoning in language games. Science, 336(6084):998. [find](https://scholar.google.com/scholar?q=Michael+C%2E+Frank+and+Noah+D%2E+Goodman%2E+2012%2E+Predicting+pragmatic+reasoning+in+language+games%2E+Science%2C+336%286084%29%3A998%2E) [our card](https://md.research.elamur.ai/free/en/papers/frank-2012-predicting-pragmatic-reasoning-language-games/)
17. David Gragg. 2003. A multi-level defense against social engineering. [find](https://scholar.google.com/scholar?q=David+Gragg%2E+2003%2E+A+multi%2Dlevel+defense+against+social+engineering%2E)
18. Charles T. Hemphill, John J. Godfrey, and George R. Doddington. 1990. The ATIS spoken language systems pilot corpus. In Speech and Natural Language: Proceedings of a Workshop Held at Hidden Valley, Pennsylvania, June 24-27,1990. [find](https://scholar.google.com/scholar?q=Charles+T%2E+Hemphill%2C+John+J%2E+Godfrey%2C+and+George+R%2E+Doddington%2E+1990%2E+The+ATIS+spoken+language+systems+pilot+corpus%2E+In+Speech+and+Natural+Language%3A+Proceedings+of+a+Workshop+Held+at+Hidden+Valley%2C+Pe)
19. Bryan Klimt and Yiming Yang. 2004. The enron corpus: A new dataset for email classification research. In Machine Learning: ECML 2004, pages 217–226, Berlin, Heidelberg. Springer Berlin Heidelberg. [find](https://scholar.google.com/scholar?q=Bryan+Klimt+and+Yiming+Yang%2E+2004%2E+The+enron+corpus%3A+A+new+dataset+for+email+classification+research%2E+In+Machine+Learning%3A+ECML+2004%2C+pages+217%E2%80%93226%2C+Berlin%2C+Heidelberg%2E+Springer+Berlin+Heidelberg%2E)
20. Andrew Lampert, Robert Dale, and Cecile Paris. 2010. Detecting emails containing requests for action. In Human Language Technologies: The 2010 Annual Conference of the North American Chapter of the Association for Computational Linguistics, pages 984–992. [find](https://scholar.google.com/scholar?q=Andrew+Lampert%2C+Robert+Dale%2C+and+Cecile+Paris%2E+2010%2E+Detecting+emails+containing+requests+for+action%2E+In+Human+Language+Technologies%3A+The+2010+Annual+Conference+of+the+North+American+Chapter+of+the+As)
21. Daniel Nahmias, Gal Engelberg, Dan Klein, and Asaf Shabtai. 2024. Prompted contextual vectors for spear-phishing detection. ArXiv, abs/2402.08309. — [arXiv:2402.08309](https://arxiv.org/abs/2402.08309) [find](https://scholar.google.com/scholar?q=Daniel+Nahmias%2C+Gal+Engelberg%2C+Dan+Klein%2C+and+Asaf+Shabtai%2E+2024%2E+Prompted+contextual+vectors+for+spear%2Dphishing+detection%2E+ArXiv%2C+abs%2F2402%2E08309%2E+%E2%80%94+%5BarXiv%3A2402%2E08309%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2402%2E08309)
22. Ellie Pavlick and Tom Kwiatkowski. 2019. Inherent disagreements in human textual inferences. Transactions of the Association for Computational Linguistics, 7:677–694. [find](https://scholar.google.com/scholar?q=Ellie+Pavlick+and+Tom+Kwiatkowski%2E+2019%2E+Inherent+disagreements+in+human+textual+inferences%2E+Transactions+of+the+Association+for+Computational+Linguistics%2C+7%3A677%E2%80%93694%2E) [our card](https://md.research.elamur.ai/free/en/papers/pavlick-2019-inherent-disagreements-human-textual-inferences/)
23. Jiaxin Pei, Aparna Ananthasubramaniam, Xingyao Wang, Naitian Zhou, Apostolos Dedeloudis, Jackson Sargent, and David Jurgens. 2022. POTATO: The portable text annotation tool. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pages 327–33… [find](https://scholar.google.com/scholar?q=Jiaxin+Pei%2C+Aparna+Ananthasubramaniam%2C+Xingyao+Wang%2C+Naitian+Zhou%2C+Apostolos+Dedeloudis%2C+Jackson+Sargent%2C+and+David+Jurgens%2E+2022%2E+POTATO%3A+The+portable+text+annotation+tool%2E+In+Proceedings+of+the+2022)
24. Maya Sappelli, Gabriella Pasi, Suzan Verberne, Maaike de Boer, and Wessel Kraaij. 2016. Assessing e-mail intent and tasks in e-mail messages. Information Sciences, 358:1–17. [find](https://scholar.google.com/scholar?q=Maya+Sappelli%2C+Gabriella+Pasi%2C+Suzan+Verberne%2C+Maaike+de+Boer%2C+and+Wessel+Kraaij%2E+2016%2E+Assessing+e%2Dmail+intent+and+tasks+in+e%2Dmail+messages%2E+Information+Sciences%2C+358%3A1%E2%80%9317%2E)
25. John R. Searle. 1969. Speech Acts: An Essay in the Philosophy of Language. Cambridge University Press. [find](https://scholar.google.com/scholar?q=John+R%2E+Searle%2E+1969%2E+Speech+Acts%3A+An+Essay+in+the+Philosophy+of+Language%2E+Cambridge+University+Press%2E)
26. Frank Stajano and Paul Wilson. 2011. Understanding scam victims: seven principles for systems security. Commun. ACM, 54(3):70–75. [find](https://scholar.google.com/scholar?q=Frank+Stajano+and+Paul+Wilson%2E+2011%2E+Understanding+scam+victims%3A+seven+principles+for+systems+security%2E+Commun%2E+ACM%2C+54%283%29%3A70%E2%80%9375%2E)
27. Zhen Tan, Dawei Li, Song Wang, Alimohammad Beigi, Bohan Jiang, Amrita Bhattacharjee, Mansooreh Karami, Jundong Li, Lu Cheng, and Huan Liu. 2024. Large language models for data annotation and synthesis: A survey. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processin… [find](https://scholar.google.com/scholar?q=Zhen+Tan%2C+Dawei+Li%2C+Song+Wang%2C+Alimohammad+Beigi%2C+Bohan+Jiang%2C+Amrita+Bhattacharjee%2C+Mansooreh+Karami%2C+Jundong+Li%2C+Lu+Cheng%2C+and+Huan+Liu%2E+2024%2E+Large+language+models+for+data+annotation+and+synthesis) [our card](https://md.research.elamur.ai/free/en/papers/tan-2024-large-language-models-data-annotation/)
28. Alexandra Uma, Tommaso Fornaciari, Dirk Hovy, Silviu Paun, Barbara Plank, and Massimo Poesio. 2021. Learning from disagreement: A survey. Journal of Artificial Intelligence Research, 72:1385–1470. [find](https://scholar.google.com/scholar?q=Alexandra+Uma%2C+Tommaso+Fornaciari%2C+Dirk+Hovy%2C+Silviu+Paun%2C+Barbara+Plank%2C+and+Massimo+Poesio%2E+2021%2E+Learning+from+disagreement%3A+A+survey%2E+Journal+of+Artificial+Intelligence+Research%2C+72%3A1385%E2%80%931470%2E)
29. Wei Wang, Saghar Hosseini, Ahmed Hassan Awadallah, Paul N. Bennett, and Chris Quirk. 2019. Context-aware intent identification in email conversations. In Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval, SI-GIR’19, page 585–594. [find](https://scholar.google.com/scholar?q=Wei+Wang%2C+Saghar+Hosseini%2C+Ahmed+Hassan+Awadallah%2C+Paul+N%2E+Bennett%2C+and+Chris+Quirk%2E+2019%2E+Context%2Daware+intent+identification+in+email+conversations%2E+In+Proceedings+of+the+42nd+International+ACM+SIGI)
30. Liu Yang, Susan T. Dumais, Paul N. Bennett, and Ahmed Hassan Awadallah. 2017. Characterizing and predicting enterprise email reply behavior. In Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR ’17, page 235–244, New York, NY, USA.… [find](https://scholar.google.com/scholar?q=Liu+Yang%2C+Susan+T%2E+Dumais%2C+Paul+N%2E+Bennett%2C+and+Ahmed+Hassan+Awadallah%2E+2017%2E+Characterizing+and+predicting+enterprise+email+reply+behavior%2E+In+Proceedings+of+the+40th+International+ACM+SIGIR+Conferen)
31. Language Resource References Oard, Douglas W and Webber, William and Kirsch, David and Golitsynskiy, Sergey. 2015. Avocado Research Email Collection. Philadelphia: Linguistic Data Consortium, ISLRN 102-408-869-995-0. PID [https://catalog.ldc.upenn.edu/LDC2015T03](https://catalog.ldc.upenn.edu/LDC2015T03). [↗](https://catalog.ldc.upenn.edu/LDC2015T03)