---
title: "TONI-NLP at PsyDefDetect — paper by Paul et al. (2026)"
description: "This system paper describes Team TONI-NLP's submission to PsyDefDetect 2026, a shared task held… 14 references, cited by 1 paper here."
authors: ["Durjoy Paul", "Arshitha Basavaraj", "Callum Chan", "Veronica Perez-Rosas", "Diana Inkpen", "Francisco Pereira", "Juan Antonio Lossio-Ventura"]
year: 2026
venue: "Proceedings of the BioNLP 2026 (Shared Tasks)"
source: "https://aclanthology.org/2026.bionlp-2.19/"
doi: "10.18653/v1/2026.bionlp-2.19"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/paul-2026-toni-nlp-psydefdetect-defense-mechanism/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/paul-2026-toni-nlp-psydefdetect-defense-mechanism/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/18/) / TONI-NLP at PsyDefDetect: Defense Mechanism Detection via LLM-based Ensemble Methods

# TONI-NLP at PsyDefDetect: Defense Mechanism Detection via LLM-based Ensemble Methods

**Authors:** Durjoy Paul, Arshitha Basavaraj, Callum Chan, [Veronica Perez-Rosas](https://md.research.elamur.ai/free/en/authors/veronica-perez-rosas/), [Diana Inkpen](https://md.research.elamur.ai/free/en/authors/diana-inkpen/), Francisco Pereira and Juan Antonio Lossio-Ventura · **Year:** 2026 · **Venue:** [Proceedings of the BioNLP 2026 (Shared Tasks)](https://md.research.elamur.ai/free/en/venues/bionlp/) ([BioNLP 2026](https://md.research.elamur.ai/free/en/venues/bionlp/2026/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2026.bionlp-2.19/) · [PDF](https://aclanthology.org/2026.bionlp-2.19.pdf) · [DOI 10.18653/v1/2026.bionlp-2.19](https://doi.org/10.18653/v1/2026.bionlp-2.19) · [OpenAlex](https://openalex.org/W7166877994)

**Citations:** 0 citations (OpenAlex, 2026-10-14) · [1](https://md.research.elamur.ai/free/en/papers/paul-2026-toni-nlp-psydefdetect-defense-mechanism/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
This system paper describes Team TONI-NLP's submission to PsyDefDetect 2026, a shared task held at BioNLP 2026. The task is to label help-seeker turns in emotional support conversations with one of nine classes based on the Defense Mechanism Rating Scales (DMRS). Seven classes are defense levels of increasing maturity, one means no defense, and one marks turns that need more information. The data come from the PSYDEFCONV corpus, which is small and heavily skewed toward the High-Adaptive class.

The authors compared four families of methods:
- zero-shot and few-shot prompting of Llama, Claude and GPT models;
- a two-stage hierarchical classifier that combines TF-IDF features with MentalBERT embeddings;
- fine-tuning of encoders, of decoder LLMs with a classification head, and of generative Llama models;
- ensembles of these systems.

Hard majority voting over their three strongest fine-tuned Llama systems gave the best result, a macro-F1 of 0.320, which placed 9th of 21 teams. Two broader findings are useful for clinical-dialogue NLP. Feeding the model only the target utterance worked as well as or better than adding dialogue context. Parameter-efficient LoRA tuning beat both full fine-tuning and prompting.

## Key points
- **Data:** PSYDEFCONV has 200 dialogues, with 1,864 training turns and 472 test turns. High-Adaptive covers 968 of the training turns. The authors built their own conversation-level validation split of 20 dialogues (272 turns), leaving 180 dialogues (1,592 turns) for training.
- **Prompting:** Dynamic few-shot prompting retrieved examples by embedding similarity. It worked best with k = 20 but reached only 0.197 macro-F1, against 0.193 for the zero-shot baseline. Randomly chosen examples did worse than both.
- **Best single model:** LoRA-tuned Llama-3.1-8B-Instruct with a classification head on the utterance alone scored 0.303 macro-F1. Among encoders, MentalRoBERTa (0.263) and MentalBERT (0.252) outperformed generic BERT models.
- **Hierarchical approach:** The four-group then fine-grained MLR pipeline peaked at 0.238. Adding the previous conversational turn lowered its score.
- **Ensembles:** Voting reached 0.320, while probability averaging dropped to 0.105. The winning ensemble combined LoRA on the utterance, LoRA on the full dialogue, and a fully fine-tuned generative 8B model.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [National Institutes of Health](https://md.research.elamur.ai/free/en/government/national-institutes-of-health/) — Government body, Bethesda, United States · 12 works · 3,846 citations
- [University of Ottawa](https://md.research.elamur.ai/free/en/universities/university-of-ottawa/) — University, Ottawa, Canada · 11 works · 2,401 citations
- [Texas State University](https://md.research.elamur.ai/free/en/universities/texas-state-university/) — University, San Marcos, United States · 3 works · 190 citations
- [International Institute of Information Technology Bangalore](https://md.research.elamur.ai/free/en/universities/international-institute-of-information-technology-bangalore/) — University, Bengaluru, India · 2 works · 19 citations

Also on the byline: [National Institute of Mental Health](https://ror.org/04xeg9z08).

📚 Shelves (6): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works · [💭 Psychology](https://md.research.elamur.ai/free/en/shelves/psychology/) · 1,552 works · [💭 Social Psychology](https://md.research.elamur.ai/free/en/shelves/psychology/social-psychology/) · 306 works

## Cite

Cite as: Durjoy Paul, Arshitha Basavaraj, Callum Chan, Veronica Perez-Rosas, Diana Inkpen, Francisco Pereira and Juan Antonio Lossio-Ventura. 2026. TONI-NLP at PsyDefDetect: Defense Mechanism Detection via LLM-based Ensemble Methods. Proceedings of the BioNLP 2026 (Shared Tasks).

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{paul2026toni,
  title = {{TONI-NLP at PsyDefDetect: Defense Mechanism Detection via LLM-based Ensemble Methods}},
  author = {Durjoy Paul, Arshitha Basavaraj, Callum Chan, Veronica Perez-Rosas, Diana Inkpen, Francisco Pereira and Juan Antonio Lossio-Ventura},
  booktitle = {Proceedings of the BioNLP 2026 (Shared Tasks)},
  year = {2026},
  doi = {10.18653/v1/2026.bionlp-2.19},
  url = {https://doi.org/10.18653/v1/2026.bionlp-2.19}
}
```

- [← previous Tokenization Cost, Retention, and Orthography Robustness for Ladin and Italian Varieties](https://md.research.elamur.ai/free/en/papers/staffini-2026-tokenization-cost-retention-orthography-robustness/)
- [next → ToolDreamer: Instilling LLM Reasoning Into Tool Retrievers](https://md.research.elamur.ai/free/en/papers/sengupta-2026-tooldreamer-instilling-llm-reasoning-into/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/18/)

## References

14 references, 3 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Callum Chan, Sunveer Khunkhun, Diana Inkpen, and Juan Antonio Lossio-Ventura. 2025. Prompt engineering for capturing dynamic mental health self states from social media posts. In Proceedings of the 10th Workshop on Computational Linguistics and Clinical Psychology (CLPsych 2025), pages 256–267, Albu… [find](https://scholar.google.com/scholar?q=Callum+Chan%2C+Sunveer+Khunkhun%2C+Diana+Inkpen%2C+and+Juan+Antonio+Lossio%2DVentura%2E+2025%2E+Prompt+engineering+for+capturing+dynamic+mental+health+self+states+from+social+media+posts%2E+In+Proceedings+of+the+10)
2. Tim Dettmers, Artidoro Pagnoni, Ari Holtzman, and Luke Zettlemoyer. 2023. [Qlora: Efficient finetuning of quantized llms](https://md.research.elamur.ai/free/en/papers/dettmers-2023-qlora-efficient-finetuning-quantized-llms/). In Advances in Neural Information Processing Systems (NeurIPS).
3. Mariagrazia Di Giuseppe and J Christopher Perry. 2021. The hierarchy of defense mechanisms: Assessing defensive functioning with the defense mechanisms rating scales q-sort. Frontiers in psychology, 12:718440. [find](https://scholar.google.com/scholar?q=Mariagrazia+Di+Giuseppe+and+J+Christopher+Perry%2E+2021%2E+The+hierarchy+of+defense+mechanisms%3A+Assessing+defensive+functioning+with+the+defense+mechanisms+rating+scales+q%2Dsort%2E+Frontiers+in+psychology%2C+1)
4. Thomas G Dietterich. 2000. Ensemble methods in machine learning. In International workshop on multiple classifier systems, pages 1–15. Springer. [find](https://scholar.google.com/scholar?q=Thomas+G+Dietterich%2E+2000%2E+Ensemble+methods+in+machine+learning%2E+In+International+workshop+on+multiple+classifier+systems%2C+pages+1%E2%80%9315%2E+Springer%2E)
5. Muskan Garg. 2023. Mental health analysis in social media posts: A survey. Archives of Computational Methods in Engineering, 30. [find](https://scholar.google.com/scholar?q=Muskan+Garg%2E+2023%2E+Mental+health+analysis+in+social+media+posts%3A+A+survey%2E+Archives+of+Computational+Methods+in+Engineering%2C+30%2E)
6. Shaoxiong Ji, Tianlin Zhang, Luna Ansari, Jie Fu, Prayag Tiwari, and Erik Cambria. 2022. Mentalbert: Publicly available pretrained language models for mental healthcare. In proceedings of the thirteenth language resources and evaluation conference, pages 7184–7190. [find](https://scholar.google.com/scholar?q=Shaoxiong+Ji%2C+Tianlin+Zhang%2C+Luna+Ansari%2C+Jie+Fu%2C+Prayag+Tiwari%2C+and+Erik+Cambria%2E+2022%2E+Mentalbert%3A+Publicly+available+pretrained+language+models+for+mental+healthcare%2E+In+proceedings+of+the+thirteen)
7. Juan Antonio Lossio-Ventura, Callum Chan, Arshitha Basavaraj, Hugo Alatrista-Salas, Francisco Pereira, and Diana Inkpen. 2025. 5cNLP at BioLay-Summ2025: Prompts, retrieval, and multimodal fusion. In Proceedings of the 24th Workshop on Biomedical Language Processing (Shared Tasks), pages 215–231, Vie… [find](https://scholar.google.com/scholar?q=Juan+Antonio+Lossio%2DVentura%2C+Callum+Chan%2C+Arshitha+Basavaraj%2C+Hugo+Alatrista%2DSalas%2C+Francisco+Pereira%2C+and+Diana+Inkpen%2E+2025%2E+5cNLP+at+BioLay%2DSumm2025%3A+Prompts%2C+retrieval%2C+and+multimodal+fusion%2E+In+P)
8. Hongbin Na, Yining Hua, Zimu Wang, Tao Shen, Beibei Yu, Lilin Wang, Wei Wang, John Torous, and Ling Chen. 2025. A survey of large language models in psychotherapy: Current landscape and future directions. In Findings of the Association for Computational Linguistics: ACL 2025, pages 7362–7376, Vienna… [find](https://scholar.google.com/scholar?q=Hongbin+Na%2C+Yining+Hua%2C+Zimu+Wang%2C+Tao+Shen%2C+Beibei+Yu%2C+Lilin+Wang%2C+Wei+Wang%2C+John+Torous%2C+and+Ling+Chen%2E+2025%2E+A+survey+of+large+language+models+in+psychotherapy%3A+Current+landscape+and+future+directi)
9. Hongbin Na, Zimu Wang, Zhaoming Chen, Yining Hua, Rena Gao, Kailai Yang, Ling Chen, Wei Wang, Shaoxiong Ji, John Torous, and Sophia Ananiadou. 2026a. [Overview of the psydefdetect shared task at bionlp 2026: Detecting levels of psychological defense mechanisms in supportive conversations](https://md.research.elamur.ai/free/en/papers/na-2026-overview-psydefdetect-shared-task-bionlp/). In Proceedi…
10. Hongbin Na, Zimu Wang, Zhaoming Chen, Peilin Zhou, Yining Hua, Grace Ziqi Zhou, Haiyang Zhang, Tao Shen, Wei Wang, John Torous, Shaoxiong Ji, and Ling Chen. 2026b. [You never know a person, you only know their defenses: Detecting levels of psychological defense mechanisms in supportive conversations](https://md.research.elamur.ai/free/en/papers/na-2026-you-never-know-person-you/).…
11. J. Christopher Perry and Melissa Henry. 2004. Chapter 9 - studying defense mechanisms in psychotherapy using the defense mechanism rating scales. In Uwe Hentschel, Gudmund Smith, Juris G. Draguns, and Wolfram Ehlers, editors, Defense Mechanisms, volume 136 of Advances in Psychology, pages 165–192. N… [find](https://scholar.google.com/scholar?q=J%2E+Christopher+Perry+and+Melissa+Henry%2E+2004%2E+Chapter+9+%2D+studying+defense+mechanisms+in+psychotherapy+using+the+defense+mechanism+rating+scales%2E+In+Uwe+Hentschel%2C+Gudmund+Smith%2C+Juris+G%2E+Draguns%2C+and)
12. Ruba Skaik and Diana Inkpen. 2020. Using social media for mental health surveillance: A review. ACM Comput. Surv., 53(6). [find](https://scholar.google.com/scholar?q=Ruba+Skaik+and+Diana+Inkpen%2E+2020%2E+Using+social+media+for+mental+health+surveillance%3A+A+review%2E+ACM+Comput%2E+Surv%2E%2C+53%286%29%2E)
13. Bin Wang and C-C Jay Kuo. 2020. Sbert-wk: A sentence embedding method by dissecting bert-based word models. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 28:2146–2157. [find](https://scholar.google.com/scholar?q=Bin+Wang+and+C%2DC+Jay+Kuo%2E+2020%2E+Sbert%2Dwk%3A+A+sentence+embedding+method+by+dissecting+bert%2Dbased+word+models%2E+IEEE%2FACM+Transactions+on+Audio%2C+Speech%2C+and+Language+Processing%2C+28%3A2146%E2%80%932157%2E)
14. Hongzhi Zhang and M Omair Shafiq. 2024. Survey of transformers and towards ensemble learning using transformers for natural language processing. Journal of big Data, 11(1):25. [find](https://scholar.google.com/scholar?q=Hongzhi+Zhang+and+M+Omair+Shafiq%2E+2024%2E+Survey+of+transformers+and+towards+ensemble+learning+using+transformers+for+natural+language+processing%2E+Journal+of+big+Data%2C+11%281%29%3A25%2E)

*Updated 1 November 2026*