---
title: "On the Role of Context in LLM Alignment to Mental Health"
description: "The authors test LLaMA-3-8B-Instruct, LLaMA-3-70B-Instruct and the medically tuned OpenBioLLM-70B on… 63% accuracy. 21 references."
authors: ["Sadiya Sayara Chowdhury Puspo", "Marcos Zampieri", "Özlem Uzuner"]
year: 2026
venue: "Proceedings of the 11th Workshop on Computational Linguistics and Clinical Psychology (CLPsych 2026)"
source: "https://aclanthology.org/2026.clpsych-1.6/"
doi: "10.18653/v1/2026.clpsych-1.6"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/puspo-2026-role-context-llm-alignment-mental/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/puspo-2026-role-context-llm-alignment-mental/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/38/) / On the Role of Context in LLM Alignment to Mental Health Counseling Competencies

# On the Role of Context in LLM Alignment to Mental Health Counseling Competencies

**Authors:** [Sadiya Sayara Chowdhury Puspo](https://md.research.elamur.ai/free/en/authors/sadiya-sayara-chowdhury-puspo/), [Marcos Zampieri](https://md.research.elamur.ai/free/en/authors/marcos-zampieri/) and [Özlem Uzuner](https://md.research.elamur.ai/free/en/authors/ozlem-uzuner/) · **Year:** 2026 · **Venue:** [Proceedings of the 11th Workshop on Computational Linguistics and Clinical Psychology (CLPsych 2026)](https://md.research.elamur.ai/free/en/venues/clpsych/) ([Workshop on Computational Linguistics and Clinical Psychology 2026](https://md.research.elamur.ai/free/en/venues/clpsych/2026/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2026.clpsych-1.6/) · [PDF](https://aclanthology.org/2026.clpsych-1.6.pdf) · [DOI 10.18653/v1/2026.clpsych-1.6](https://doi.org/10.18653/v1/2026.clpsych-1.6) · [OpenAlex](https://openalex.org/W7166882839)

**Citations:** 0 citations (OpenAlex, 2026-10-14)

## Summary
Large language models can reach passing scores on mental health counseling exams. This paper asks whether those scores come from reasoning about the individual patient or from general symptom-to-diagnosis associations. The authors test LLaMA-3-8B-Instruct, LLaMA-3-70B-Instruct and the medically tuned OpenBioLLM-70B on CounselingBench, a benchmark of case vignettes drawn from NCMHCE licensing exam practice material.

They split each vignette's patient-specific context into four parts: demographics, presenting problem, mental status examination and other contexts. They then remove these parts step by step, shuffle answer options, reorder question and answer, assign the model different roles, and apply Thread-of-Thought (ThoT) prompting.

Removing context lowers accuracy only slightly. Option shuffling barely changes predictions, while question ordering has a larger effect. Neither role framing nor ThoT helps. Even when ThoT reasoning surfaces the relevant cue, the final answer often ignores it.

The work matters because benchmark-level competence may overstate how well models integrate context. This argues for evaluations built around context, such as contrastive vignettes, before LLMs are deployed in mental health care.

## Key points
- **Dataset:** CounselingBench has 1,612 questions across 138 case studies, covering five competencies:
  - CS&I: 599 questions
  - IA&D: 460 questions
  - PP&E: 274 questions
  - TP: 253 questions
  - CCA: 23 questions
- **Passing threshold:** The typical NCMHCE pass mark is about 63% accuracy, and the chosen models had already reached or exceeded it in earlier work.
- **Role framing:** Assigned personas had no measurable effect on accuracy. In 30.2% of cases, models gave the same wrong answer regardless of persona.
- **Error analysis:** The authors examined 490 cases where all four role-framed prompts agreed on an incorrect answer. They found three recurring failure patterns:
  - broad-to-specific errors when selecting assessments
  - substitution of generic priorities at intake
  - neglect of relational context in ethics questions
- **Limitations:** Only open models were tested, and the benchmark is US-specific, English-only and multiple-choice. Errors clustered in PP&E, and the authors flag possible benchmark contamination.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [George Mason University](https://md.research.elamur.ai/free/en/universities/george-mason-university/) — University, Fairfax, United States · 24 works · 1,288 citations

📚 Shelves (6): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works · [🩺 Medicine](https://md.research.elamur.ai/free/en/shelves/medicine/) · 241 works · [🩺 Health Informatics](https://md.research.elamur.ai/free/en/shelves/medicine/health-informatics/) · 184 works

🧭 Topics (1): [🧠 Language models & pretraining](https://md.research.elamur.ai/free/en/topics/language-models/) · 3,320 works

## Cite

Cite as: Sadiya Sayara Chowdhury Puspo, Marcos Zampieri and Özlem Uzuner. 2026. On the Role of Context in LLM Alignment to Mental Health Counseling Competencies. Proceedings of the 11th Workshop on Computational Linguistics and Clinical Psychology (CLPsych 2026).

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{puspo2026role,
  title = {{On the Role of Context in LLM Alignment to Mental Health Counseling Competencies}},
  author = {Sadiya Sayara Chowdhury Puspo, Marcos Zampieri and Özlem Uzuner},
  booktitle = {Proceedings of the 11th Workshop on Computational Linguistics and Clinical Psychology (CLPsych 2026)},
  year = {2026},
  doi = {10.18653/v1/2026.clpsych-1.6},
  url = {https://doi.org/10.18653/v1/2026.clpsych-1.6}
}
```

- [← previous On the Robustness of Morphosyntactic Transformation with Large Language Models: The Case of Quechua Collao](https://md.research.elamur.ai/free/en/papers/pocco-2026-robustness-morphosyntactic-transformation-large-language/)
- [next → On the source of distributive inferences](https://md.research.elamur.ai/free/en/papers/marty-2026-source-distributive-inferences/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/38/)

## References

21 references, 1 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Mohsen Ghorbian and Mostafa Ghobaei-Arani. 2025. Large language models for mental health diagnosis and treatment: a survey. Artificial Intelligence Review, 59(1):9. [find](https://scholar.google.com/scholar?q=Mohsen+Ghorbian+and+Mostafa+Ghobaei%2DArani%2E+2025%2E+Large+language+models+for+mental+health+diagnosis+and+treatment%3A+a+survey%2E+Artificial+Intelligence+Review%2C+59%281%29%3A9%2E)
2. Hiba Arnaout, Anmol Goel, H Andrew Schwartz, Steffen T Eberhardt, Dana Atzil-Slonim, Gavin Doherty, Brian Schwartz, Wolfgang Lutz, Tim Althoff, Munmun De Choudhury, and 1 others. 2026. Responsible evaluation of ai for mental health. arXiv preprint arXiv:2602.00065. [find](https://scholar.google.com/scholar?q=Hiba+Arnaout%2C+Anmol+Goel%2C+H+Andrew+Schwartz%2C+Steffen+T+Eberhardt%2C+Dana+Atzil%2DSlonim%2C+Gavin+Doherty%2C+Brian+Schwartz%2C+Wolfgang+Lutz%2C+Tim+Althoff%2C+Munmun+De+Choudhury%2C+and+1+others%2E+2026%2E+Responsible+eva)
3. Candida M Greco, Andrea Simeri, Andrea Tagarelli, and Ester Zumpano. 2023. Transformer-based language models for mental health issues: a survey. Pattern Recognition Letters, 167:204–211. [find](https://scholar.google.com/scholar?q=Candida+M+Greco%2C+Andrea+Simeri%2C+Andrea+Tagarelli%2C+and+Ester+Zumpano%2E+2023%2E+Transformer%2Dbased+language+models+for+mental+health+issues%3A+a+survey%2E+Pattern+Recognition+Letters%2C+167%3A204%E2%80%93211%2E)
4. Ana-Maria Bucur. 2026. Computational approaches to mental health disorders detection from social media texts, images and videos. Procesamiento del Lenguaje Natural, 76:311–314. [find](https://scholar.google.com/scholar?q=Ana%2DMaria+Bucur%2E+2026%2E+Computational+approaches+to+mental+health+disorders+detection+from+social+media+texts%2C+images+and+videos%2E+Procesamiento+del+Lenguaje+Natural%2C+76%3A311%E2%80%93314%2E)
5. Zhijun Guo, Alvina Lai, Johan H Thygesen, Joseph Farrington, Thomas Keen, Kezhi Li, and 1 others. 2024. Large language models for mental health applications: systematic review. JMIR mental health, 11(1):e57400. [find](https://scholar.google.com/scholar?q=Zhijun+Guo%2C+Alvina+Lai%2C+Johan+H+Thygesen%2C+Joseph+Farrington%2C+Thomas+Keen%2C+Kezhi+Li%2C+and+1+others%2E+2024%2E+Large+language+models+for+mental+health+applications%3A+systematic+review%2E+JMIR+mental+health%2C+11%28)
6. Ana-Maria Bucur, Andreea Moldovan, Krutika Parvatikar, Marcos Zampieri, Ashiqur Khudabukhsh, and Liviu P Dinu. 2025a. Datasets for depression modeling in social media: An overview. In Proceedings of the 10th Workshop on Computational Linguistics and Clinical Psychology (CLPsych 2025), pages 116–126. [find](https://scholar.google.com/scholar?q=Ana%2DMaria+Bucur%2C+Andreea+Moldovan%2C+Krutika+Parvatikar%2C+Marcos+Zampieri%2C+Ashiqur+Khudabukhsh%2C+and+Liviu+P+Dinu%2E+2025a%2E+Datasets+for+depression+modeling+in+social+media%3A+An+overview%2E+In+Proceedings+of+t)
7. Zainab Iftikhar, Amy Xiao, Sean Ransom, Jeff Huang, and Harini Suresh. 2025. How llm counselors violate ethical standards in mental health practice: A practitioner-informed framework. In Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society, volume 8, pages 1311–1323. [find](https://scholar.google.com/scholar?q=Zainab+Iftikhar%2C+Amy+Xiao%2C+Sean+Ransom%2C+Jeff+Huang%2C+and+Harini+Suresh%2E+2025%2E+How+llm+counselors+violate+ethical+standards+in+mental+health+practice%3A+A+practitioner%2Dinformed+framework%2E+In+Proceedings+o)
8. Ana-Maria Bucur, Andreea-Codrina Moldovan, Kru-tika Parvatikar, Marcos Zampieri, Ashiqur R KhudaBukhsh, and Liviu P Dinu. 2025b. On the state of nlp approaches to modeling depression in social media: A post-covid-19 outlook. IEEE Journal of Biomedical and Health Informatics. [find](https://scholar.google.com/scholar?q=Ana%2DMaria+Bucur%2C+Andreea%2DCodrina+Moldovan%2C+Kru%2Dtika+Parvatikar%2C+Marcos+Zampieri%2C+Ashiqur+R+KhudaBukhsh%2C+and+Liviu+P+Dinu%2E+2025b%2E+On+the+state+of+nlp+approaches+to+modeling+depression+in+social+media%3A+)
9. Di Jin, Eileen Pan, Nassim Oufattole, Wei-Hung Weng, Hanyi Fang, and Peter Szolovits. 2021. [What disease does this patient have? a large-scale open domain question answering dataset from medical exams](https://md.research.elamur.ai/free/en/papers/jin-2020-disease-this-patient-have-large/). Applied Sciences, 11(14):6421.
10. Ana-Maria Bucur, Marcos Zampieri, Tharindu Ranasinghe, and Fabio Crestani. 2026. A survey on multilingual mental disorders detection from social media data. In Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers), pages 8… [find](https://scholar.google.com/scholar?q=Ana%2DMaria+Bucur%2C+Marcos+Zampieri%2C+Tharindu+Ranasinghe%2C+and+Fabio+Crestani%2E+2026%2E+A+survey+on+multilingual+mental+disorders+detection+from+social+media+data%2E+In+Proceedings+of+the+19th+Conference+of+th)
11. Yoon Kyung Lee, Inju Lee, Minjung Shin, Seoyeon Bae, and Sowon Hahn. 2023. Chain of empathy: Enhancing empathetic response of large language models based on psychotherapy models. arXiv preprint arXiv:2311.04915. — [arXiv:2311.04915](https://arxiv.org/abs/2311.04915) [find](https://scholar.google.com/scholar?q=Yoon+Kyung+Lee%2C+Inju+Lee%2C+Minjung+Shin%2C+Seoyeon+Bae%2C+and+Sowon+Hahn%2E+2023%2E+Chain+of+empathy%3A+Enhancing+empathetic+response+of+large+language+models+based+on+psychotherapy+models%2E+arXiv+preprint+arXiv%3A)
12. Inbar Levkovich. 2025. Evaluating diagnostic accuracy and treatment efficacy in mental health: A comparative analysis of large language model tools and mental health professionals. European Journal of Investigation in Health, Psychology and Education, 15(1):9. [find](https://scholar.google.com/scholar?q=Inbar+Levkovich%2E+2025%2E+Evaluating+diagnostic+accuracy+and+treatment+efficacy+in+mental+health%3A+A+comparative+analysis+of+large+language+model+tools+and+mental+health+professionals%2E+European+Journal+of)
13. Mohit Chandra, Siddharth Sriraman, Harneet Singh Khanuja, Yiqiao Jin, and Munmun De Choudhury. 2025. Reasoning is not all you need: examining llms for multi-turn mental health conversations. arXiv preprint arXiv:2505.20201. [find](https://scholar.google.com/scholar?q=Mohit+Chandra%2C+Siddharth+Sriraman%2C+Harneet+Singh+Khanuja%2C+Yiqiao+Jin%2C+and+Munmun+De+Choudhury%2E+2025%2E+Reasoning+is+not+all+you+need%3A+examining+llms+for+multi%2Dturn+mental+health+conversations%2E+arXiv+pre)
14. Siyuan Brandon Loh and Aravind Sesagiri Raamkumar. 2023. Harnessing large language models’ empathetic response generation capabilities for online mental health counselling support. arXiv preprint arXiv:2310.08017. — [arXiv:2310.08017](https://arxiv.org/abs/2310.08017) [find](https://scholar.google.com/scholar?q=Siyuan+Brandon+Loh+and+Aravind+Sesagiri+Raamkumar%2E+2023%2E+Harnessing+large+language+models%E2%80%99+empathetic+response+generation+capabilities+for+online+mental+health+counselling+support%2E+arXiv+preprint+arXi)
15. Ha Na Cho, Jiayuan Wang, Di Hu, and Kai Zheng. 2026. Large language model–based chatbots and agentic ai for mental health counseling: Systematic review of methodologies, evaluation frameworks, and ethical safeguards. JMIR AI, 5(1):e80348. [find](https://scholar.google.com/scholar?q=Ha+Na+Cho%2C+Jiayuan+Wang%2C+Di+Hu%2C+and+Kai+Zheng%2E+2026%2E+Large+language+model%E2%80%93based+chatbots+and+agentic+ai+for+mental+health+counseling%3A+Systematic+review+of+methodologies%2C+evaluation+frameworks%2C+and+eth)
16. Danilo Moggia, Wolfgang Lutz, Eva-Lotta Brakemeier, and Leonard Bickman. 2024. Treatment personalization and precision mental health care: Where are we and where do we want to go? Administration and Policy in Mental Health and Mental Health Services Research, 51(5):611–616. [find](https://scholar.google.com/scholar?q=Danilo+Moggia%2C+Wolfgang+Lutz%2C+Eva%2DLotta+Brakemeier%2C+and+Leonard+Bickman%2E+2024%2E+Treatment+personalization+and+precision+mental+health+care%3A+Where+are+we+and+where+do+we+want+to+go%3F+Administration+and+P)
17. Sarah C Cook, Ann C Schwartz, and Nadine J Kaslow. 2017. Evidence-based psychotherapy: Advantages and challenges. Neurotherapeutics, 14(3):537–545. [find](https://scholar.google.com/scholar?q=Sarah+C+Cook%2C+Ann+C+Schwartz%2C+and+Nadine+J+Kaslow%2E+2017%2E+Evidence%2Dbased+psychotherapy%3A+Advantages+and+challenges%2E+Neurotherapeutics%2C+14%283%29%3A537%E2%80%93545%2E)
18. Ahmed Tamer Elboardy, Ziad Mohamed, Ghada Khoriba, Tamer Arafa, and Essam A Rashed. 2025. Bridging gender disparities in mental health: A bilingual large language model for multilingual therapeutic chatbots. In 2025 22nd International Learning and Technology Conference (L&T), volume 22, pages 251–25… [find](https://scholar.google.com/scholar?q=Ahmed+Tamer+Elboardy%2C+Ziad+Mohamed%2C+Ghada+Khoriba%2C+Tamer+Arafa%2C+and+Essam+A+Rashed%2E+2025%2E+Bridging+gender+disparities+in+mental+health%3A+A+bilingual+large+language+model+for+multilingual+therapeutic+ch)
19. Viet Cuong Nguyen, Mohammad Taher, Dongwan Hong, Vinicius Konkolics Possobom, Vibha Thirunel-layi Gopalakrishnan, Ekta Raj, Zihang Li, Heather J Soled, Michael L Birnbaum, Srijan Kumar, and 1 others. 2024. Do large language models align with core mental health counseling competencies? arXiv preprint… [find](https://scholar.google.com/scholar?q=Viet+Cuong+Nguyen%2C+Mohammad+Taher%2C+Dongwan+Hong%2C+Vinicius+Konkolics+Possobom%2C+Vibha+Thirunel%2Dlayi+Gopalakrishnan%2C+Ekta+Raj%2C+Zihang+Li%2C+Heather+J+Soled%2C+Michael+L+Birnbaum%2C+Srijan+Kumar%2C+and+1+others%2E+)
20. Zhuohan Ge, Nicole Hu, Darian Li, Yubo Wang, Shihao Qi, Yuming Xu, Han Shi, and Jason Zhang. 2025. A survey of large language models in mental health disorder detection on social media. In 2025 IEEE 41st International Conference on Data Engineering Workshops (ICDEW), pages 164–176. IEEE. [find](https://scholar.google.com/scholar?q=Zhuohan+Ge%2C+Nicole+Hu%2C+Darian+Li%2C+Yubo+Wang%2C+Shihao+Qi%2C+Yuming+Xu%2C+Han+Shi%2C+and+Jason+Zhang%2E+2025%2E+A+survey+of+large+language+models+in+mental+health+disorder+detection+on+social+media%2E+In+2025+IEEE+4)
21. APA Presidential Task Force on Evidence-Based Practice and 1 others. 2006. Evidence-based practice in psychology. The American Psychologist, 61(4):271– 285. [find](https://scholar.google.com/scholar?q=APA+Presidential+Task+Force+on+Evidence%2DBased+Practice+and+1+others%2E+2006%2E+Evidence%2Dbased+practice+in+psychology%2E+The+American+Psychologist%2C+61%284%29%3A271%E2%80%93+285%2E)

*Updated 1 November 2026*