--- title: "Finetuning Large Language Models for Automated Depression" description: "Many Nigerians describe distress in Nigerian Pidgin, using idioms that general-purpose LLMs tend to misread. 12 references." authors: ["Isaac Iyinoluwa Olufadewa", "Miracle Ayomikun Adesina", "Ezekiel Ayodeji Oladejo", "Uthman Babatunde Usman", "Owen Kolade Adeniyi", "Matthew Tolulope Olawoyin"] year: 2025 venue: "arXiv:2601.00004" source: "https://arxiv.org/abs/2601.00004" doi: "10.48550/arxiv.2601.00004" arxiv: "2601.00004" licence: "CC BY 4.0 (arXiv)" licence_url: "http://creativecommons.org/licenses/by/4.0/" summary_licence: "Elamur Cosmic License" canonical: "https://research.elamur.ai/free/en/papers/olufadewa-2025-finetuning-large-language-models-automated/" --- [Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/olufadewa-2025-finetuning-large-language-models-automated/) [home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2025](https://md.research.elamur.ai/free/en/papers/2025/page/17/) / Finetuning Large Language Models for Automated Depression Screening in Nigerian Pidgin English: GENSCORE Pilot Study # Finetuning Large Language Models for Automated Depression Screening in Nigerian Pidgin English: GENSCORE Pilot Study **Authors:** Isaac Iyinoluwa Olufadewa, Miracle Ayomikun Adesina, Ezekiel Ayodeji Oladejo, Uthman Babatunde Usman, Owen Kolade Adeniyi and Matthew Tolulope Olawoyin · **Year:** 2025 · **Venue:** arXiv:2601.00004 · **Licence:** [CC BY 4.0 (arXiv)](http://creativecommons.org/licenses/by/4.0/) **Links:** [Publisher's page](https://arxiv.org/abs/2601.00004) · [PDF](https://arxiv.org/pdf/2601.00004) · [DOI 10.48550/arxiv.2601.00004](https://doi.org/10.48550/arxiv.2601.00004) · [arXiv 2601.00004](https://arxiv.org/abs/2601.00004) · [OpenAlex](https://openalex.org/W7118370466) **Citations:** 0 citations (OpenAlex, 2026-10-14) ## Summary Depression screening in Nigeria is limited by a severe shortage of psychiatrists, by stigma, and by the PHQ-9 itself. That questionnaire was designed for English-speaking, high-income settings. Many Nigerians describe distress in Nigerian Pidgin, using idioms that general-purpose LLMs tend to misread. The GENSCORE pilot collected 432 spoken Pidgin answers from adults aged 18–40 through the authors' GENCURATE platform, with prompts tied to each of the nine PHQ-9 items. After transcription and spelling normalization, a mixed team of linguists, psychologists and native speakers annotated each answer with three things: a PHQ-9 severity score (0–3), whether the wording was literal or figurative, and a clinical gloss for slang and idioms. Three models were trained on this data with SFT: Phi-3-mini-4k-instruct, Gemma-3-4B-it and GPT-4.1. Training was paired with system prompts that set tone, forbade diagnosis and defined crisis escalation. GPT-4.1 performed best, with 94.5% severity accuracy, and human raters also judged its outputs most culturally fitting. The study suggests that culturally grounded fine-tuning could make cheap, scalable preliminary screening feasible for underserved Pidgin speakers. ## Key points - The dataset has 432 transcribed Pidgin audio responses collected via GENCURATE. The reported split is 345/44/42 (train/validation/test), and the test portion is described as a 10% held-out set. - Each response has three annotation layers: - a PHQ-9 severity category (0–3); - semantic intent (literal vs. metaphorical or idiomatic); - an idiom-to-clinical-meaning interpretation, e.g. "my head just dey heavy" glossed as low energy or cognitive fatigue. - Severity classification results (accuracy / precision / F1, %): - Phi-3-mini-4k-instruct: 71.8 / 69.1 / 60.9 - Gemma-3-4B-it: 82.7 / 80.3 / 74.5 - GPT-4.1: 94.5 / 93.2 / 89.1 - Semantic alignment with expert interpretations, measured by BERTScore F1, was 0.72 for Phi-3, 0.85 for Gemma and 0.95 for GPT-4.1. - Safety was assessed on two measures: correct escalation of self-harm content (PHQ-9 Item 9) and compliance with the non-diagnostic rule. Phi-3 reached 82% on escalation; the other safety and human-evaluation values are not legible in the extracted text. A rule-based escalation protocol backed up the models, and the study had ethics approval from the FCT Health Ethics Research Committee. ## Interesting ideas - Each response carries a separate tag saying whether it is meant literally or figuratively, annotated independently of the severity label, so meaning is not forced through a literal reading. - Slang and idioms are mapped explicitly to the clinical state they express, so the model's interpretation can be checked against a human gloss. - The model's free-text interpretation is scored against expert annotations as a separate measure, alongside label accuracy. This separates "got the right score" from "understood why". - High-risk cases (self-harm disclosures) are handled by a deterministic escalation rule that sits outside the learned model. Safety therefore does not rest on model behaviour alone. - Spelling variation in a non-standard written variety is normalized before modeling. The paper also notes that an English-centric tokenizer struggles with such variation. ## For Mova How these ideas could serve Mova, our small open language model — its files, stages and tests: - **I1** (frames): A literal/figurative flag on predicates before role assignment would keep phrases like "head dey heavy" from producing a frame about physical weight. - **I2** (world): A compiled idiom table linking fixed expressions to ontology states (fatigue, low mood, anhedonia) would let the world model record the intended state instead of the surface event. - **I3** (back): Generated paraphrases from the meaning graph could be compared against reference glosses, so that a correct final answer reached through a wrong reading gets caught. - **I4** (answer): Readers could have hard-coded guard rules for critical question types that override the learned path and stay fully explainable. - **I5** (tokens): A normalization layer could map variant spellings to canonical lemmas while keeping the original form, which would make the tagger more robust on dialectal or informal English. *Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.* 📚 Shelves (5): [💭 Psychology](https://md.research.elamur.ai/free/en/shelves/psychology/) · 1,649 works · [💭 Experimental and Cognitive Psychology](https://md.research.elamur.ai/free/en/shelves/psychology/experimental-and-cognitive-psychology/) · 930 works · [💭 Social Psychology](https://md.research.elamur.ai/free/en/shelves/psychology/social-psychology/) · 330 works · [🏥 Health Professions](https://md.research.elamur.ai/free/en/shelves/health-professions/) · 75 works · [🏥 Health Information Management](https://md.research.elamur.ai/free/en/shelves/health-professions/health-information-management/) · 56 works 🧭 Topics (3): [🧠 Language models & pretraining](https://md.research.elamur.ai/free/en/topics/language-models/) · 3,894 works · [English](https://md.research.elamur.ai/free/en/topics/english/) · 2,379 works · [🧷 Idioms & multiword expressions](https://md.research.elamur.ai/free/en/topics/multiword-expressions/) · 222 works ## Cite Cite as: Isaac Iyinoluwa Olufadewa, Miracle Ayomikun Adesina, Ezekiel Ayodeji Oladejo, Uthman Babatunde Usman, Owen Kolade Adeniyi and Matthew Tolulope Olawoyin. 2025. Finetuning Large Language Models for Automated Depression Screening in Nigerian Pidgin English: GENSCORE Pilot Study. arXiv:2601.00004. The paper itself, not this page. BibTeX from the library's record of the work. ```bibtex @misc{olufadewa2025finetuning, title = {{Finetuning Large Language Models for Automated Depression Screening in Nigerian Pidgin English: GENSCORE Pilot Study}}, author = {Isaac Iyinoluwa Olufadewa, Miracle Ayomikun Adesina, Ezekiel Ayodeji Oladejo, Uthman Babatunde Usman, Owen Kolade Adeniyi and Matthew Tolulope Olawoyin}, year = {2025}, doi = {10.48550/arxiv.2601.00004}, eprint = {2601.00004}, archivePrefix = {arXiv}, url = {https://doi.org/10.48550/arxiv.2601.00004} } ``` - [← previous Fine-Grained Arabic Offensive Language Classification with Taxonomy, Sentiment, and Emotions](https://md.research.elamur.ai/free/en/papers/vanetik-2025-fine-grained-arabic-offensive-language/) - [next → From Human Preference to Autonomous Alignment: A Survey of Large Language Model Alignment](https://md.research.elamur.ai/free/en/papers/dou-2025-human-preference-autonomous/) - [↑ back to papers of 2025](https://md.research.elamur.ai/free/en/papers/2025/page/17/) ## References 12 references, 2 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet. 1. Adewoyin, T., Okoye, S., & Eze, J. (2023). Leveraging low-resource language data for NLP in African mental health contexts. Journal of African Language Technologies, 5(2), 45–59. [find](https://scholar.google.com/scholar?q=Adewoyin%2C+T%2E%2C+Okoye%2C+S%2E%2C+%26+Eze%2C+J%2E+%282023%29%2E+Leveraging+low%2Dresource+language+data+for+NLP+in+African+mental+health+contexts%2E+Journal+of+African+Language+Technologies%2C+5%282%29%2C+45%E2%80%9359%2E) 2. Alyousef, Awad & Omari, Asem & Mahafdah, Rund. (2025). Ethical Alignment in Large Language Models: Interpreting Moral Reasoning in Transformer-Based AI Systems. Journal of Posthumanism. 5. 10.63332/joph.v5i4.1089. [↗](https://doi.org/10.63332/joph.v5i4.1089) 3. Biega, A. J., Gummadi, K., & Weikum, G. (2022). Privacy-preserving natural language processing: Challenges and opportunities. Proceedings of the 2022 Conference on Fairness, Accountability, and Transparency, 112–123. [find](https://scholar.google.com/scholar?q=Biega%2C+A%2E+J%2E%2C+Gummadi%2C+K%2E%2C+%26+Weikum%2C+G%2E+%282022%29%2E+Privacy%2Dpreserving+natural+language+processing%3A+Challenges+and+opportunities%2E+Proceedings+of+the+2022+Conference+on+Fairness%2C+Accountability%2C+and+Transp) 4. Fadele, K. P., Igwe, S. C., Toluwalogo, N. O., Udokang, E. I., Ogaya, J. B., & Lucero-Prisno, D. E., 3rd (2024). Mental health challenges in Nigeria: Bridging the gap between demand and resources. Global mental health (Cambridge, England), 11, e29. https://doi.org/10.1017/gmh.2024.19 [↗](https://doi.org/10.1017/gmh.2024.19) 5. Hedderich A., Lukas Lange, Heike Adel, Jannik Strötgen, and Dietrich Klakow. 2021. [A Survey on Recent Approaches for Natural Language Processing in Low-Resource Scenarios](https://md.research.elamur.ai/free/en/papers/hedderich-2021-survey-recent-approaches-natural-language/). In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 2545–2568. 6. Kaiser, Bonnie & Ticao, Cynthia & Anoje, C. & Minto, J. & Boglosa, J. & Kohrt, Brandon. (2019). Adapting culturally appropriate mental health screening tools for use among conflict-affected and other vulnerable adolescents in Nigeria. Global Mental Health. 6. 10.1017/gmh.2019.8. [↗](https://doi.org/10.1017/gmh.2019.8) 7. Li, Yuxin & Kumbale, Sinchana & Chen, Yanru & Surana, Tanmay & Chng, Eng & Guan, Cuntai. (2025). Automated Depression Detection From Text and Audio: A Systematic Review. IEEE journal of biomedical and health informatics. PP. 10.1109/JBHI.2025.3570900. [↗](https://doi.org/10.1109/jbhi.2025.3570900) 8. Lin, H., Zhao, Q., & Liu, Y. (2022). Automated depression detection using conversational AI: A systematic review. Frontiers in Digital Health, 4, 815623. [find](https://scholar.google.com/scholar?q=Lin%2C+H%2E%2C+Zhao%2C+Q%2E%2C+%26+Liu%2C+Y%2E+%282022%29%2E+Automated+depression+detection+using+conversational+AI%3A+A+systematic+review%2E+Frontiers+in+Digital+Health%2C+4%2C+815623%2E) 9. Malgaroli, Matteo & Hull, Thomas & Zech, James & Althoff, Tim. (2023). Natural language processing for mental health interventions: a systematic review and research framework. Translational psychiatry. 13. 309. 10.1038/s41398-023-02592-2. [↗](https://doi.org/10.1038/s41398-023-02592-2) 10. Ouyang, L., Wu, J., Jiang, X., Almeida, D., Wainwright, C., Mishkin, P., … Leike, J. (2022). [Training language models to follow instructions with human feedback](https://md.research.elamur.ai/free/en/papers/ouyang-2022-training-language-models-follow-instructions/). Advances in Neural Information Processing Systems, 35, 27730–27744. https://doi.org/10.48550/arXiv.2203.02155 [↗](https://doi.org/10.48550/arxiv.2203.02155) 11. Sanh, V., Wenzek, G., Xu, Z., & Raffel, C. (2023). Instruction fine-tuning large language models for domain adaptation. arXiv preprint arXiv:2305.00428. https://arxiv.org/abs/2305.00428 [↗](https://arxiv.org/abs/2305.00428) 12. Zhang, T., Kishore, V., Wu, F., Weinberger, K. Q., & Artzi, Y. (2020). BERTScore: Evaluating text generation with BERT. International Conference on Learning Representations (ICLR 2020). [https://openreview.net/forum?id=Syg0sYHFwS](https://openreview.net/forum?id=Syg0sYHFwS) [↗](https://openreview.net/forum?id=Syg0sYHFwS) *Updated 29 October 2026*