--- title: "AraMed: Arabic Medical Question Answering — Alasmari 2024" description: "AraMed targets a resource gap: medical question answering research has mostly used English data, and the Arabic medical QA corpora that exist are either…" authors: ["Ashwag Alasmari", "Sarah Alhumoud", "Waad Alshammari"] year: 2024 venue: "Proceedings of the 6th Workshop on Open-Source Arabic Corpora and Processing Tools (OSACT) with Shared Tasks on Arabic LLMs Hallucination and Dialect to MSA Machine Translation @ LREC-COLING 2024" source: "https://aclanthology.org/2024.osact-1.6/" licence: "CC BY-NC 4.0 (ELRA, LREC proceedings)" licence_url: "https://creativecommons.org/licenses/by-nc/4.0/" summary_licence: "Elamur Cosmic License" canonical: "https://research.elamur.ai/free/en/papers/alasmari-2024-aramed-arabic-medical-question-answering/" --- [Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/alasmari-2024-aramed-arabic-medical-question-answering/) [home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2024](https://md.research.elamur.ai/free/en/papers/2024/) / AraMed: Arabic Medical Question Answering using Pretrained Transformer Language Models # AraMed: Arabic Medical Question Answering using Pretrained Transformer Language Models **Authors:** Ashwag Alasmari, Sarah Alhumoud, Waad Alshammari · **Year:** 2024 · **Venue:** Proceedings of the 6th Workshop on Open-Source Arabic Corpora and Processing Tools (OSACT) with Shared Tasks on Arabic LLMs Hallucination and Dialect to MSA Machine Translation @ LREC-COLING 2024 · **Licence:** [CC BY-NC 4.0 (ELRA, LREC proceedings)](https://creativecommons.org/licenses/by-nc/4.0/) **Links:** [Publisher's page](https://aclanthology.org/2024.osact-1.6/) · [PDF](https://aclanthology.org/2024.osact-1.6.pdf) · [OpenAlex](https://openalex.org/W7131794521) **Citations:** 3 citations (openalex-snapshot · 2026-10-14) · [13](#cited-by) in our library, of the 140884 papers whose bibliographies we read ## Summary AraMed targets a resource gap: medical question answering research has mostly used English data, and the Arabic medical QA corpora that exist are either tiny (ARmed, 350 questions) or small and skewed (CQA-MD, where truly relevant answers are about 4% of pairs). The authors gathered consumer health questions and physician answers from Altibbi, a widely used MENA medical platform, covering 2020–2021 and favouring posts that users voted most useful. Each record keeps metadata such as category, questioner age and gender, physician specialty, and date. Because the platform holds only correct answers, they generated an irrelevant answer for every relevant one with a rule-based sliding-window procedure that avoids answers from the same category. The resulting balanced corpus of roughly 270k question–answer pairs was used to benchmark answer selection, framed as binary classification, with AraBERT, CAMeLBERT, and a hybrid AraBERTv0.2 + Sentence-BERT encoder. AraBERTv2 performed best, reaching an F1 of 96.73%. The corpus gives Arabic medical QA a training set large enough for deep learning. ## Key points - Split sizes: train has 109,834 questions with 219,668 answers across 85 categories, half relevant and half irrelevant. Test has 27,459 questions with 54,918 answers across 78 categories. The largest category is gynecological diseases, with 24,399 questions. - Negative sampling works as follows. Start the candidate 10 rows past the current answer. If it shares the current answer's category, move 5 rows further. Accept it only if it has more than 7 words. - Preprocessing steps: - remove diacritics and elongation with Pyarabic; - strip links, Latin letters and digits with regular expressions; - map alef variants (ٱ إ أ آ) to ا; - replace "?" with the Arabic question mark "؟". - Maximum input length is 256 tokens. Average question length is 23.9 and average answer length is 34.2. Earlier work by the authors found that 512 tokens doubled runtime without improving accuracy. - Results: AraBERTv2 reached 96.71% accuracy and 96.73% F1, with the other variants mostly in the 93–96% range. The weakest reported system scored 86.29% accuracy and 89.49% F1. For comparison, CQA-MD inputs average 1,101 tokens, which is too long for typical transformers. ## Interesting ideas - **Metadata-based negatives:** wrong answers are drawn from a different topical category at a fixed row offset, which turns a positives-only forum dump into a balanced classification set at no annotation cost. — For Mova (answer): distractor answers for fable/tale QA tests could be harvested automatically from other stories or other question types, keeping a known "clearly wrong" baseline. - **Easy negatives inflate scores:** the authors themselves propose not skipping same-category answers in future work, admitting that cross-category negatives are easy and help explain the very high F1. — For Mova (learn): pair every evaluation with an easy-negative and a hard-negative condition as a control, so we can tell real understanding from topic matching. - **Filter uninformative negatives:** candidate negatives with too few words (7 or fewer) are discarded because short answers carry little signal. — For Mova (learn): when building silver data or contrastive examples, drop near-empty sentences so they cannot become trivially separable noise. - **Set context size from measured lengths:** the 256-token limit was chosen from average question and answer lengths, after 512 had shown no gain at double the cost. — For Mova (tree): size gparse's input window from treebank sentence-length statistics rather than a default maximum, saving GPU time. - **Script-level normalization before modelling:** variant letters and Latin vs. native punctuation are unified so equivalent strings look identical to the model. — For Mova (tokens): a small, explicit normalization table (quote styles, dashes, Unicode look-alikes) in crate en would reduce spurious tokenizer and lemma variants while staying fully explainable. *Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.* Fields: [Linguistics & languages](https://md.research.elamur.ai/free/en/shelves/linguistics/) · [Machine translation](https://md.research.elamur.ai/free/en/shelves/machine-translation/) ## Cited by Cited by 13 papers of our library cite it (of the 140884 whose bibliographies we read); 4 of them have a page here, 9 not yet. - [MedAraBench: Large-Scale Arabic Medical Question Answering Dataset and Benchmark](https://md.research.elamur.ai/free/en/papers/abu-2026-daoud-medarabench-large-scale-arabic-medical/) - [Integrating AI into Arabic Linguistics: A Methodological Framework for Reproducible and Interpretable Research](https://md.research.elamur.ai/free/en/papers/akzam-2026-integrating-ai-arabic-linguistics-methodological/) - [Benchmarking the Medical Understanding and Reasoning of Large Language Models in Arabic Healthcare Tasks](https://md.research.elamur.ai/free/en/papers/aldahoul-2025-benchmarking-medical-understanding-reasoning-large/) - [MedArabiQ: Benchmarking Large Language Models on Arabic Medical Tasks](https://md.research.elamur.ai/free/en/papers/daoud-2025-medarabiq-benchmarking-large-language-models/) ## Cite The paper itself, not this page. BibTeX from the library's record of the work. ```bibtex @inproceedings{alasmari2024aramed, title = {{AraMed: Arabic Medical Question Answering using Pretrained Transformer Language Models}}, author = {Alasmari, Ashwag and Alhumoud, Sarah and Alshammari, Waad}, booktitle = {Proceedings of the 6th Workshop on Open-Source Arabic Corpora and Processing Tools (OSACT) with Shared Tasks on Arabic LLMs Hallucination and Dialect to MSA Machine Translation @ LREC-COLING 2024}, year = {2024}, doi = {10.63317/2k278cznpbxt}, url = {https://doi.org/10.63317/2k278cznpbxt} } ``` - [← previous ArabicMMLU: Assessing Massive Multitask Language Understanding in Arabic](https://md.research.elamur.ai/free/en/papers/koto-2024-arabicmmlu-assessing-massive-multitask-language/) - [next → AraPoemBERT: A Pretrained Language Model for Arabic Poetry Analysis](https://md.research.elamur.ai/free/en/papers/qarah-2024-arapoembert-pretrained-language-model-arabic/) - [↑ back to papers of 2024](https://md.research.elamur.ai/free/en/papers/2024/) ## References 7 references, 0 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet. 1. A Silver Standard Gutub, A. (2023). Analysis of community question-Biomedical Corpus for Arabic Language. answering issues via machine learning and deep Complexity, 2020, 8896659. learning: State-of-the-art review. CAAI https://doi.org/10.1155/2020/8896659 Transactions on Intelligence Technology, 8(1), Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. 95–117. https://doi.org/10.1049/cit2.12081 (2019). BERT: Pre-training of Deep Bidirectional Thuwaini, W. A., & Alhumoud, S. (2022). TAQS: An Transformers for Language Understanding. ArXiv, Arabic Question Similarity System Using Transfer abs/1810.0. Learning of BERT With BiLSTM. IEEE Access, 10, Faris, H., Habib, M., Faris, M., Alomari, A., Castillo, P. 91509–91523. [↗](https://doi.org/10.1155/2020/8896659) 2. A., & Alomari, M. (2022). Classification of Arabic https://doi.org/10.1109/ACCESS.2022.3198955 healthcare questions based on word embeddings Wicks, P., Massagli, M., Frost, J., Brownstein, C., learned from massive consultations: A deep Okun, S., Vaughan, T., Bradley, R., & Heywood, J. learning approach. Journal of Ambient Intelligence (2010). Sharing Health Data for Better Outcomes and Humanized Computing, 13(4), 1811–1827. on PatientsLikeMe. Journal of Medical Internet https://doi.org/10.1007/s12652-021-02948-w Research, 12(2), e19. [↗](https://doi.org/10.1109/access.2022.3198955) 3. Fehri, H., Dardour, S., & Haddar, K. (2022). ARmed https://doi.org/10.2196/jmir.1549 question answering system. Concurrency and Wolf, T., Debut, L., Sanh, V., Chaumond, J., Computation: Practice and Experience, 34(21), Delangue, C., Moi, A., Cistac, P., Rault, T., Louf, e7054. https://doi.org/10.1002/cpe.7054 R., Funtowicz, M., & others. (2020). Transformers: Guo, Q., Cao, S., & Yi, Z. (2022). A medical question State-of-the-art natural language processing. answering system using large language models Proceedings of the 2020 Conference on Empirical and knowledge graphs. International Journal of Methods in Natural Language Processing: System Intelligent Systems, 37(11), 8548–8564. Demonstrations, 38–45. https://doi.org/10.1002/int.22955 [↗](https://doi.org/10.2196/jmir.1549) 4. Zhang, W., Chen, Z., Dong, C., Wang, W., Zha, H., & Inoue, G., Alhafni, B., Baimukan, N., Bouamor, H., & Wang, J. (2021). [find](https://scholar.google.com/scholar?q=Zhang%2C+W%2E%2C+Chen%2C+Z%2E%2C+Dong%2C+C%2E%2C+Wang%2C+W%2E%2C+Zha%2C+H%2E%2C+%26+Inoue%2C+G%2E%2C+Alhafni%2C+B%2E%2C+Baimukan%2C+N%2E%2C+Bouamor%2C+H%2E%2C+%26+Wang%2C+J%2E+%282021%29%2E) 5. Graph-Based Tri-Attention Habash, N. (2021). The Interplay of Variant, Size, Network for Answer Ranking in CQA. ArXiv, and Task Type in Arabic Pre-trained Language abs/2103.03583. Models. WANLP. https://api.semanticscholar.org/CorpusID:232135 [↗](https://arxiv.org/abs/2103.03583) 6. Lai, T. M., Bui, T., & Li, S. (2018). A Review on Deep 063 Learning Techniques Applied to Answer Selection. Proceedings of the 27th International Conference on Computational Linguistics, 2132–2144. 8. Appendices Liu, Q., Agichtein, E., Dror, G., Maarek, Y., & 8.1 Dataset Description Szpektor, I. (2012). When web search fails, Each question “Q” has question ID, question searchers become askers. Proceedings of the category, gender, age, question time, question title, 35th International ACM SIGIR Conference on question description, number of answers, relevant Research and Development in Information answer ID, date of relevant answer, relevant answer, Retrieval - SIGIR ’12, 801. the answer specialty of the physician, irrelevant https://doi.org/10.1145/2348283.2348390 answer ID, doctor specialty of irrelevant answer, date Mishra, A., & Jain, S. K. (2016). A survey on question of irrelevant answer, irrelevant answer. answering systems with classification. Journal of King Saud University - Computer and Information To offer a deeper understanding of the AraMed Sciences, 28(3), 345–361. corpus, let's delve into the nature of the questions https://doi.org/10.1016/j.jksuci.2014.10.007 and answers. The AraMed corpus is primarily Nakov, P., Hoogeveen, D., Màrquez, L., Moschitti, A., composed of questions and answers formulated in Mubarak, H., Baldwin, T., & Verspoor, K. (2017). Modern Standard Arabic (MSA). This reflects the platform's focus on providing reliable and SemEval-2017 Task 3: Community Question standardized health information. However, the Answering. Proceedings of the 11th International corpus also encompasses a variety of question Workshop on Semantic Evaluation (SemEval-types. Users engage in both knowledge-seeking 2017), 27–48. https://doi.org/10.18653/v1/S17-queries, such as "What are the symptoms of the 2003 common cold?", and those seeking specific Othman, N., Faiz, R., & Smaïli, K. (2020). Improving consultations, like "I have a persistent cough. What the Community Question Retrieval Performance could it be?". This diversity enriches the data by Using Attention-Based Siamese LSTM. In E. reflecting real-world information needs within Arabic-Métais, F. Meziane, H. Horacek, & P. Cimiano speaking communities. (Eds.), Natural Language Processing and Information Systems (pp. 252–263). Springer International Publishing. [↗](https://doi.org/10.1145/2348283.2348390) 7. Reimers, N., & Gurevych, I. (2019). Sentence-bert: Sentence embeddings using siamese bert-networks. arXiv Preprint arXiv:1908.10084. Roy, P. K., Saumya, S., Singh, J. P., Banerjee, S., & 55 Categories Number of Questions 24399 18569 10656 7898 5574 5567 5063 4980 4266 4113 3513 2410 3364 3140 2753 2713 2684 2277 Gynecological diseases Sexual health General Medicine Musculoskeletal and joint diseases Skin disease Gastrointestinal diseases Urinary and venereal diseases Sexually transmitted diseases Pharmacology Cardiovascular disease General Surgery Dental disease Otolaryngology Pregnancy and birth Psychiatric illness Children's diseases Internal medicine Ophthalmology Table 3: Total number of questions for each category. [↗](https://arxiv.org/abs/1908.10084)