--- title: "Structured Prompting for Arabic Essay — Mandhari 2026" description: "Arabic Automatic Essay Scoring (AES) lags behind English." authors: ["Salim Al Mandhari", "Hieu Pham Dinh", "Mo El-Haj", "Paul Rayson"] year: 2026 venue: "Proceedings of the Fifteenth Language Resources and Evaluation Conference" source: "https://aclanthology.org/2026.lrec-1.826/" doi: "10.63317/4st85zet2p6h" licence: "CC BY-NC 4.0 (ELRA, LREC proceedings)" licence_url: "https://creativecommons.org/licenses/by-nc/4.0/" summary_licence: "Elamur Cosmic License" canonical: "https://research.elamur.ai/free/papers/mandhari-2026-structured-prompting-arabic-essay-proficiency/" --- [Elamur Research](https://md.research.elamur.ai/) · [this page in HTML](https://research.elamur.ai/free/papers/mandhari-2026-structured-prompting-arabic-essay-proficiency/) [home](https://md.research.elamur.ai/) / [papers](https://md.research.elamur.ai/free/papers/) / [2026](https://md.research.elamur.ai/free/papers/2026/page/4/) / Structured Prompting for Arabic Essay Proficiency: A Trait-Centric Evaluation Approach # Structured Prompting for Arabic Essay Proficiency: A Trait-Centric Evaluation Approach **Authors:** Salim Al Mandhari, Hieu Pham Dinh, [Mo El-Haj](https://md.research.elamur.ai/free/authors/mo-el-haj/), [Paul Rayson](https://md.research.elamur.ai/free/authors/paul-rayson/) · **Year:** 2026 · **Venue:** Proceedings of the Fifteenth Language Resources and Evaluation Conference · **Licence:** [CC BY-NC 4.0 (ELRA, LREC proceedings)](https://creativecommons.org/licenses/by-nc/4.0/) **Links:** [Publisher's page](https://aclanthology.org/2026.lrec-1.826/) · [PDF](https://aclanthology.org/2026.lrec-1.826.pdf) · [DOI 10.63317/4st85zet2p6h](https://doi.org/10.63317/4st85zet2p6h) · [OpenAlex](https://openalex.org/W7160195193) **Citations:** 0 citations (openalex-snapshot · 2026-10-14) ## Summary Arabic Automatic Essay Scoring (AES) lags behind English. Annotated data are scarce, and most existing systems produce one holistic or content-correctness score rather than ratings for separate proficiency traits. The authors test whether large language models can score individual traits of Arabic student essays through prompting alone, without any fine-tuning. They design three prompt levels: 1. A standard zero-shot prompt that rates all seven traits in one pass. 2. A hybrid prompt in which five simulated specialist raters each judge only their own area. Each trait's score is the average of the raters assigned to it. 3. A rubric-guided few-shot prompt that supplies an Arabic scoring guide and low/mid/high scored examples, and asks for a JSON score with a justification. Eight LLMs are tested on QAES, the first public Arabic AES corpus with trait-level labels. Evaluation uses QWK and confidence intervals. Fanar-1-9B-Instruct agrees most closely with human raters. Rubric guidance improves every model and trait. The findings suggest that careful prompt design matters more than model size for low-resource educational assessment. ## Key points - **Dataset and traits:** QAES (Bashendy et al., 2024) builds on the QCAW argumentative-writing corpus. Its rubrics come from Qatar University Testing Center guidelines and cover seven traits: Organization, Vocabulary, Style, Development, Mechanics, Structure, Relevance. - **Hybrid level raters:** The five simulated raters cover: - Organization & Coherence - Vocabulary & Lexical Variety - Mechanics (grammar, spelling, punctuation) - Content Development & Reasoning - Style/Tone & Contextual Appropriateness Each trait is mapped to three raters, whose scores are averaged. - **Best model:** Fanar-1-9B-Instruct performed best in both zero- and few-shot settings, with a reported QWK of 0.28 and CI of 0.41. For reference, the earlier QAES baseline was a holistic LR model at QWK 0.26. - **Effect of rubric guidance:** Rubric-guided prompting gave consistent improvements. The largest gains were on discourse-level traits, Development and Style. - **Availability:** Code and prompts are on GitHub (dinhieufam/Arabic_AES). *Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.* Fields: [Linguistics & languages](https://md.research.elamur.ai/free/shelves/linguistics/) ## Cite The paper itself, not this page. BibTeX from the library's record of the work. ```bibtex @inproceedings{mandhari2026structured, title = {{Structured Prompting for Arabic Essay Proficiency: A Trait-Centric Evaluation Approach}}, author = {Mandhari, Salim Al and Dinh, Hieu Pham and El-Haj, Mo and Rayson, Paul}, booktitle = {Proceedings of the Fifteenth Language Resources and Evaluation Conference}, year = {2026}, doi = {10.63317/4st85zet2p6h}, url = {https://doi.org/10.63317/4st85zet2p6h} } ``` - [← previous Structured Legal Document Generation in India: A Model-Agnostic Wrapper Approach with VidhikDastaavej](https://md.research.elamur.ai/free/papers/nigam-2026-structured-legal-document-generation-india/) - [next → Subject to change: Agreement and case](https://md.research.elamur.ai/free/papers/lingbulk-2026-rimon-subject-change-agreement-case/) - [↑ back to papers of 2026](https://md.research.elamur.ai/free/papers/2026/page/4/) ## References 26 references, 5 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet. 1. Ayad R Abbas and Ahmed S Al-qaza. 2014. Automated arabic essay scoring (aaes) using vector space model (vsm). Journal of Al-Turath University College, 15:25–39. [find](https://scholar.google.com/scholar?q=Ayad+R+Abbas+and+Ahmed+S+Al%2Dqaza%2E+2014%2E+Automated+arabic+essay+scoring+%28aaes%29+using+vector+space+model+%28vsm%29%2E+Journal+of+Al%2DTurath+University+College%2C+15%3A25%E2%80%9339%2E) 2. Ayad R Abbas and Ahmed S Al-qazaz. 2015. Automated arabic essay scoring (aaes) using vectors space model (vsm) and latent semantics indexing (lsi). Eng. Technol. J., 33(3):410–426. [↗](https://doi.org/10.30684/etj.33.3b.4) 3. Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. 2023. Gpt-4 technical report. arXiv preprint arXiv:2303.08774. [↗](https://arxiv.org/abs/2303.08774) [our card](https://md.research.elamur.ai/free/papers/openai-2023-gpt-4-technical-report/) 4. Imtiaz Ahmed, Sadman Islam, Partha Protim Datta, Imran Kabir, Naseef Ur Rahman Chowdhury, and Ahshanul Haque. 2025. Qwen 2.5: A comprehensive review of the leading resource-efficient llm with potentioal to surpass all competitors. Authorea Preprints. [↗](https://doi.org/10.36227/techrxiv.174060306.65738406/v1) 5. Sultan Almujaiwel, Damith Premasiri, Tharindu Ranasinghe, Mo El-Haj, and Ruslan Mitkov. 2025. Complex concept-based readability estimation from arabic curriculum. ACM Trans. Asian Low-Resour. Lang. Inf. Process. Just Accepted. [↗](https://doi.org/10.1145/3770070) 6. Jinze Bai, Shuai Bai, Yunfei Chu, Zeyu Cui, Kai Dang, Xiaodong Deng, Yang Fan, Wenbin Ge, Yu Han, Fei Huang, et al. 2023. Qwen technical report. arXiv preprint arXiv:2309.16609. [↗](https://arxiv.org/abs/2309.16609) [our card](https://md.research.elamur.ai/free/papers/bai-2023-qwen-technical-report/) 7. M Saiful Bari, Yazeed Alnumay, Norah A Alzahrani, Nouf M Alotaibi, Hisham A Alyahya, Sultan Al-Rashed, Faisal A Mirza, Shaykhah Z Alsubaie, Hassan A Alahmed, Ghadah Alabduljabbar, et al. 2024. Allam: Large language models for arabic and english. arXiv preprint arXiv:2407.15390. [↗](https://arxiv.org/abs/2407.15390) [our card](https://md.research.elamur.ai/free/papers/bari-2024-allam-large-language-models-arabic/) 8. May Bashendy, Salam Albatarni, Sohaila Eltanbouly, Eman Zahran, Hamdo Elhuseyin, Tamer Elsayed, Walid Massoud, and Houda Bouamor. 2024. Qaes: First publicly-available trait-specific annotations for automated scoring of arabic essays. In Proceedings of The Second Arabic Natural Language Processing Conference, pages 337–351. [find](https://scholar.google.com/scholar?q=May+Bashendy%2C+Salam+Albatarni%2C+Sohaila+Eltanbouly%2C+Eman+Zahran%2C+Hamdo+Elhuseyin%2C+Tamer+Elsayed%2C+Walid+Massoud%2C+and+Houda+Bouamor%2E+2024%2E+Qaes%3A+First+publicly%2Davailable+trait%2Dspecific+annotations+for+au) 9. Afrizal Doewes, Nughthoh Kurdhi, and Akrati Saxena. 2023. Evaluating quadratic weighted kappa as the standard performance metric for automated essay scoring. In 16th International Conference on Educational Data Mining, EDM 2023, pages 103–113. International Educational Data Mining Society (IEDMS). [find](https://scholar.google.com/scholar?q=Afrizal+Doewes%2C+Nughthoh+Kurdhi%2C+and+Akrati+Saxena%2E+2023%2E+Evaluating+quadratic+weighted+kappa+as+the+standard+performance+metric+for+automated+essay+scoring%2E+In+16th+International+Conference+on+Educat) 10. Bradley Efron and Robert J. Tibshirani. 1993. An Introduction to the Bootstrap. Chapman & Hall/CRC. [↗](https://doi.org/10.1007/978-1-4899-4541-9) 11. Mo El-Haj, Sultan Almujaiwel, Damith Premasiri, Tharindu Ranasinghe, and Ruslan Mitkov. 2024. DARES: Dataset for Arabic readability estimation of school materials. In Proceedings of the Workshop on DeTermIt! Evaluating Text Difficulty in a Multilingual Context @ LREC-COLING 2024, pages 103–113, Torino, Italia. ELRA and ICCL. [find](https://scholar.google.com/scholar?q=Mo+El%2DHaj%2C+Sultan+Almujaiwel%2C+Damith+Premasiri%2C+Tharindu+Ranasinghe%2C+and+Ruslan+Mitkov%2E+2024%2E+DARES%3A+Dataset+for+Arabic+readability+estimation+of+school+materials%2E+In+Proceedings+of+the+Workshop+on+De) 12. Sohaila Eltanbouly, Salam Albatarni, and Tamer Elsayed. 2025. Trates: Trait-specific rubric-assisted cross-prompt essay scoring. arXiv preprint arXiv:2505.14577. [↗](https://arxiv.org/abs/2505.14577) 13. Rayed Ghazawi and Edwin Simpson. 2024. Automated essay scoring in arabic: a dataset and analysis of a bert-based system. arXiv preprint arXiv:2407.11212. [↗](https://arxiv.org/abs/2407.11212) 14. Rayed Ghazawi and Edwin Simpson. 2025. How well can llms grade essays in arabic? arXiv preprint arXiv:2501.16516. [↗](https://arxiv.org/abs/2501.16516) 15. Sanwoo Lee, Yida Cai, Desong Meng, Ziyang Wang, and Yunfang Wu. 2024. Unleashing large language models’ proficiency in zero-shot essay scoring. arXiv preprint arXiv:2404.04941. [↗](https://arxiv.org/abs/2404.04941) 16. Nourmeen Lotfy, Abdulaziz Shehab, Mohammed Elhoseny, and Ahmed Abu-Elfetouh. 2023. An enhanced automatic arabic essay scoring system based on machine learning algorithms. CMC-COMPUTERS MATERIALS & CONTINUA, 77(1):1227–1249. [↗](https://doi.org/10.32604/cmc.2023.039185) 17. Watheq Mansour, Salam Albatarni, Sohaila Eltanbouly, and Tamer Elsayed. 2024. Can large language models automatically score proficiency of written essays? In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024), pages 2777–2786, Torino, Italia. ELRA and ICCL. [find](https://scholar.google.com/scholar?q=Watheq+Mansour%2C+Salam+Albatarni%2C+Sohaila+Eltanbouly%2C+and+Tamer+Elsayed%2E+2024%2E+Can+large+language+models+automatically+score+proficiency+of+written+essays%3F+In+Proceedings+of+the+2024+Joint+Internationa) 18. Chatrine Qwaider, Bashar Alhafni, Kirill Chirkunov, Nizar Habash, and Ted Briscoe. 2025. Enhancing arabic automated essay scoring with synthetic data and error injection. arXiv preprint arXiv:2503.17739. [↗](https://arxiv.org/abs/2503.17739) 19. Dadi Ramesh and Suresh Kumar Sanampudi. 2024. Coherence-based automatic short answer scoring using sentence embedding. European Journal of Education, 59(3):e12684. [↗](https://doi.org/10.1111/ejed.12684) 20. Neha Sengupta, Sunil Kumar Sahu, Bokang Jia, Satheesh Katipomu, Haonan Li, Fajri Koto, William Marshall, Gurpreet Gosal, Cynthia Liu, Zhiming Chen, et al. 2023. Jais and jais-chat: Arabic-centric foundation and instruction-tuned open generative large language models. arXiv preprint arXiv:2308.16149. [↗](https://arxiv.org/abs/2308.16149) [our card](https://md.research.elamur.ai/free/papers/sengupta-2023-jais-jais-chat-arabic-centric/) 21. Burr Settles, Geoffrey T. LaFlair, and Masato Hagiwara. 2020. Machine learning–driven language assessment. Transactions of the Association for computational Linguistics, 8:247–263. [↗](https://doi.org/10.1162/tacl_a_00310) 22. Abdulaziz Shehab, Mahmoud Faroun, and Magdi Rashad. 2018. An automatic arabic essay grading system based on text similarity algorithms. International Journal of Advanced Computer Science and Applications, 9(3). [↗](https://doi.org/10.14569/ijacsa.2018.090337) 23. Fanar Team, Ummar Abbas, Mohammad Shahmeer Ahmad, Firoj Alam, Enes Altinisik, Ehsannedin Asgari, Yazan Boshmaf, Sabri Boughorbel, Sanjay Chawla, Shammur Chowdhury, Fahim Dalvi, Kareem Darwish, Nadir Durrani, Mohamed Elfeky, Ahmed Elmagarmid, Mohamed Eltabakh, Masoomali Fatehkia, Anastasios Fragkopoulos, Maram Hasanain, Majd Hawasly, Mus’ab Husaini, Soon-Gyo Jung, Ji Kim Lucas, Walid Magdy, Safa Messaoud, Abubakr Mohamed, Tasnim Mohiuddin, Basel Mousi, Hamdy Mubarak, Ahmad Musleh, Zan Naeem, Mourad Ouzzani, Dorde Popovic, Amin Sadeghi, Husrev Taha Sencar, Mohammed Shinoy, Omar Sinan, Yifan Zhang, Ahmed Ali, Yassine El Kheir, Xiaosong Ma, and Chaoyi Ruan. 2025. Fanar: An arabic-centric multimodal generative ai platform. [↗](https://doi.org/10.48550/arxiv.2501.13944) [our card](https://md.research.elamur.ai/free/papers/abbas-2025-fanar-arabic-centric-multimodal-generative/) 24. QwenLM Team. 2024. Qwen3-VL: Multimodal Large Language Model. [https://github.com/QwenLM/Qwen3-VL](https://github.com/QwenLM/Qwen3-VL). Accessed: October 20, 2025. [↗](https://github.com/QwenLM/Qwen3-VL) 25. Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. 2023. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288. [↗](https://arxiv.org/abs/2307.09288) 26. Wajdi Zaghouani, Abdelhamid Ahmed, Xiao Zhang, and Lameya Rezk. 2024. Qcaw 1.0: building a qatari corpus of student argumentative writing. In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024), pages 13382–13394. [find](https://scholar.google.com/scholar?q=Wajdi+Zaghouani%2C+Abdelhamid+Ahmed%2C+Xiao+Zhang%2C+and+Lameya+Rezk%2E+2024%2E+Qcaw+1%2E0%3A+building+a+qatari+corpus+of+student+argumentative+writing%2E+In+Proceedings+of+the+2024+Joint+International+Conference+on+)