--- title: "Sindbad at AraHealthQA Track 1 — paper by Morsy et al. (2025)" description: "This paper describes a system for the mental-health track of the AraHealthQA shared task. 7 references, cited by 2 papers here." authors: ["AbdulRahman A. Morsy", "Saad Mankarious", "Ayah Zirikly"] year: 2025 venue: "Proceedings of The Third Arabic Natural Language Processing Conference: Shared Tasks" source: "https://aclanthology.org/2025.arabicnlp-sharedtasks.26/" doi: "10.18653/v1/2025.arabicnlp-sharedtasks.26" licence: "CC BY 4.0 (ACL Anthology)" licence_url: "https://creativecommons.org/licenses/by/4.0/" summary_licence: "Elamur Cosmic License" canonical: "https://research.elamur.ai/free/en/papers/morsy-2025-sindbad-arahealthqa-track-1-leveraging/" --- [Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/morsy-2025-sindbad-arahealthqa-track-1-leveraging/) [home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2025](https://md.research.elamur.ai/free/en/papers/2025/page/13/) / Sindbad at AraHealthQA Track 1: Leveraging Large Language Models for Mental Health Q&A # Sindbad at AraHealthQA Track 1: Leveraging Large Language Models for Mental Health Q&A **Authors:** [AbdulRahman A. Morsy](https://md.research.elamur.ai/free/en/authors/abdulrahman-a-morsy/), [Saad Mankarious](https://md.research.elamur.ai/free/en/authors/saad-mankarious/) and [Ayah Zirikly](https://md.research.elamur.ai/free/en/authors/ayah-zirikly/) · **Year:** 2025 · **Venue:** [Proceedings of The Third Arabic Natural Language Processing Conference: Shared Tasks](https://md.research.elamur.ai/free/en/venues/arabicnlp/) ([ArabicNLP 2025](https://md.research.elamur.ai/free/en/venues/arabicnlp/2025/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/) **Links:** [Publisher's page](https://aclanthology.org/2025.arabicnlp-sharedtasks.26/) · [PDF](https://aclanthology.org/2025.arabicnlp-sharedtasks.26.pdf) · [DOI 10.18653/v1/2025.arabicnlp-sharedtasks.26](https://doi.org/10.18653/v1/2025.arabicnlp-sharedtasks.26) · [OpenAlex](https://openalex.org/W4416037788) **Citations:** 0 citations (OpenAlex, 2026-10-14) · [2](https://md.research.elamur.ai/free/en/papers/morsy-2025-sindbad-arahealthqa-track-1-leveraging/cited-by/) in our library, of the 149,439 papers whose bibliographies we read ## Summary This paper describes a system for the mental-health track of the AraHealthQA shared task. The track has three subtasks: multi-label classification of Arabic patient questions into seven categories, multi-label classification of professional answers into three strategy types, and answer generation. The training data is small, only 350 annotated question–answer pairs from the MentalQA forum corpus, and it is heavily skewed: some question categories have fewer than ten examples. The authors balanced the data by having GPT-4o write extra examples for rare labels and rare label combinations, and they checked roughly a third of these by hand. They then fine-tuned ALLaM 7B for questions and MARBERT for answers. For generation, gpt-3.5-turbo receives few-shot exemplars and the question category predicted by the classifier. Its system prompt is refined with GrIPS, a gradient-free edit-and-score search judged by BERTScore. The team placed 1st in answer classification and 3rd in the other two subtasks. The results suggest that targeted synthetic oversampling combined with prompt search is a practical recipe for tiny, imbalanced, low-resource datasets. ## Key points - The dataset has 350 QA pairs from the MentalQA corpus. Question labels are A–F plus Z, which in order cover diagnosing a condition, treating it, the body's anatomy and physiology, disease epidemiology, lifestyle habits for good health, selecting a care provider, and a catch-all "other" class. Answer labels are information, direct guidance and emotional support. - GPT-4o (temperature 0.7) produced synthetic data for both subtasks: - Subtask 1: 300 examples for classes C, D, E and F, growing the set to 650. - Subtask 2: 160 examples, 40 for each rare configuration (3, 1+3, 2+3, 1+2+3), growing the set to 510. - Question classification used ALLaM 7B with LoRA and reached Weighted-F1 0.53 and Jaccard 0.49 (3rd place). - Answer classification used MARBERT with BCE loss and AdamW and reached Weighted-F1 0.79 and Jaccard 0.71 (1st place). The authors credit the gain mainly to augmenting under-represented label combinations. - The QA system used gpt-3.5-turbo at temperature 0 with three hand-picked exemplars and the predicted category in the prompt. GrIPS ran for six rounds, scored by BERTScore F1 on a training subset. The system scored 0.668 BERTScore on the test set (3rd place). ## Interesting ideas - Oversample rare label *combinations*, not only rare single labels, because multi-label co-occurrences can be far scarcer than any one label alone. - Accept synthetic data only after a human inspects a random third of each batch for fluency, label fit and constraint adherence, which is a cheap audit that catches drifting generators. - Feed an upstream classifier's predicted category into the generator's prompt so the output is steered by an explicit, inspectable decision rather than left implicit. - Improve an instruction without gradients by mutating it, scoring each variant on a held-out slice, and keeping only the best one per round. - Constrain generation prompts explicitly with fixed counts, varied lengths and lexical-variety controls to reduce near-duplicate synthetic samples. ## For Mova How these ideas could serve Mova, our small open language model — its files, stages and tests: - **I1** (learn): when building LLM-teacher silver data, target rare co-occurring phenomena (e.g. a rare deprel attached to a rare construction) rather than only rare individual tags. - **I2** (learn): add a sampled manual audit gate per generated silver batch, paired with negative controls, before it enters training. - **I3** (answer): pass the question type detected upstream (e.g. a why/how/yes-no speech act) to the readers so the answer form follows a stated, explainable choice. - **I4** (learn): apply the same mutate-score-keep loop to teacher-tier prompts, using agreement with gold treebank trees as the score. - **I5** (back): use similar diversity constraints when generating paraphrases from the meaning graph, so test sets for the compressor do not collapse into repetitive surface forms. *Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.* ## Institutions - [George Washington University](https://md.research.elamur.ai/free/en/universities/george-washington-university/) — University, Washington, United States · 51 works · 8,148 citations 📚 Shelves (6): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works · [💭 Psychology](https://md.research.elamur.ai/free/en/shelves/psychology/) · 1,552 works · [💭 Social Psychology](https://md.research.elamur.ai/free/en/shelves/psychology/social-psychology/) · 306 works 🧭 Topics (2): [🧠 Language models & pretraining](https://md.research.elamur.ai/free/en/topics/language-models/) · 3,320 works · [Semitic languages](https://md.research.elamur.ai/free/en/topics/semitic-languages/) · 2,908 works ## Cite Cite as: AbdulRahman A. Morsy, Saad Mankarious and Ayah Zirikly. 2025. Sindbad at AraHealthQA Track 1: Leveraging Large Language Models for Mental Health Q&A. Proceedings of The Third Arabic Natural Language Processing Conference: Shared Tasks. The paper itself, not this page. BibTeX from the library's record of the work. ```bibtex @inproceedings{morsy2025sindbad, title = {{Sindbad at AraHealthQA Track 1: Leveraging Large Language Models for Mental Health Q\&A}}, author = {AbdulRahman A. Morsy, Saad Mankarious and Ayah Zirikly}, booktitle = {Proceedings of The Third Arabic Natural Language Processing Conference: Shared Tasks}, year = {2025}, doi = {10.18653/v1/2025.arabicnlp-sharedtasks.26}, url = {https://doi.org/10.18653/v1/2025.arabicnlp-sharedtasks.26} } ``` - [← previous SHAMI-MT: A Syrian Arabic Dialect to Modern Standard Arabic Bidirectional Machine Translation System](https://md.research.elamur.ai/free/en/papers/sibaee-2025-shami-mt-syrian-arabic-dialect/) - [next → SmolLab_SEU at MAHED Shared Task: Do Arabic-Native Encoders Surpass Multilingual Models in Detecting the Nuances of Hope, Hate, and Emotion?](https://md.research.elamur.ai/free/en/papers/rahman-2025-smollab-seu-mahed-shared-task/) - [↑ back to papers of 2025](https://md.research.elamur.ai/free/en/papers/2025/page/13/) ## References 7 references, 5 of them with a card of ours. ↗ leads to the work. 1. M. Abdul-Mageed, A. Elmadany, and E. M. B. Nagoudi. 2021. [ARBERT & MARBERT: Deep bidirectional transformers for arabic](https://md.research.elamur.ai/free/en/papers/abdul-mageed-2021-arbert-marbert-deep-bidirectional-transformers/). In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 7088–7105, Virtual. [↗](https://doi.org/10.48550/arxiv.2101.01785) 2. Mouath Abu Daoud, Chaimae Abouzahir, Leen Kharouf, Walid Al-Eisawi, Nizar Habash, and Farah E Shamout. 2025. [Medarabiq: Benchmarking large language models on arabic medical tasks](https://md.research.elamur.ai/free/en/papers/daoud-2025-medarabiq-benchmarking-large-language-models/). arXiv e-prints, pages arXiv–2505. 3. Hania El Khatib, Aisha Alyafei, and Madiha Shaikh. 2023. Understanding experiences of mental health help-seeking in arab populations around the world: A systematic review and narrative synthesis. BMC Psychiatry, 23(1):324. [↗](https://doi.org/10.1186/s12888-023-04827-4) 4. Archiki Prasad, Pratyush Venkatesh, Yixin Xu, Rui Zhang, Junyi Jessy Li, Surya Kallumadi, and Bill Yuchen Lin. 2022. [GrIPS: Gradient-free, edit-based instruction search for prompting large language models](https://md.research.elamur.ai/free/en/papers/prasad-2023-grips-gradient-free-edit-based/). arXiv preprint arXiv:2203.07281. [↗](https://arxiv.org/abs/2203.07281) 5. Damian F. Santomauro, Ana M. Mantilla Herrera, Junfang Shadid, Peng Zheng, Charlie Ashbaugh, Giorgia Pigott, Harish G. Sheena, and et al. 2021. Global prevalence and burden of depressive and anxiety disorders in 204 countries and territories in 2020 due to the covid-19 pandemic. The Lancet, 398(10312):1700–1712. [↗](https://doi.org/10.1016/s0140-6736(21)02143-7) 6. Han-Chin Shing, Suraj Nair, Ayah Zirikly, Meir Friedenberg, Hal Daumé III, and Philip Resnik. 2018. [Expert, crowdsourced, and machine assessment of suicide risk via online postings](https://md.research.elamur.ai/free/en/papers/shing-2018-expert-crowdsourced-machine-assessment-suicide/). In Proceedings of the Fifth Workshop on Computational Linguistics and Clinical Psychology: From Keyboard to Clinic, pages 25–36, New Orleans, LA. Association for Computational Linguistics. 7. Ayah Zirikly, Philip Resnik, Özlem Uzuner, and Kristy Hollingshead. 2019. [CLPsych 2019 shared task: Predicting the degree of suicide risk in Reddit posts](https://md.research.elamur.ai/free/en/papers/zirikly-2019-clpsych-2019-shared-task-predicting/). In Proceedings of the Sixth Workshop on Computational Linguistics and Clinical Psychology, pages 24–33, Minneapolis, Minnesota. Association for Computational Linguistics. *Updated 1 November 2026*