---
title: "Enhancing Arabic Automated Essay Scoring with Synthetic Data"
description: "These are concentrated at B1–B2, with none at A1 or C2. The authors build a synthetic dataset to fill… 29 references, cited by 8 papers here."
authors: ["Chatrine Qwaider", "Bashar Alhafni", "Kirill Chirkunov", "Nizar Habash", "Ted Briscoe"]
year: 2025
venue: "Proceedings of the 20th Workshop on Innovative Use of NLP for Building Educational Applications (BEA 2025)"
source: "https://aclanthology.org/2025.bea-1.40/"
doi: "10.18653/v1/2025.bea-1.40"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/qwaider-2025-enhancing-arabic-automated-essay-scoring/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/qwaider-2025-enhancing-arabic-automated-essay-scoring/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2025](https://md.research.elamur.ai/free/en/papers/2025/page/6/) / Enhancing Arabic Automated Essay Scoring with Synthetic Data and Error Injection

# Enhancing Arabic Automated Essay Scoring with Synthetic Data and Error Injection

**Authors:** [Chatrine Qwaider](https://md.research.elamur.ai/free/en/authors/chatrine-qwaider/), [Bashar Alhafni](https://md.research.elamur.ai/free/en/authors/bashar-alhafni/), [Kirill Chirkunov](https://md.research.elamur.ai/free/en/authors/kirill-chirkunov/), [Nizar Habash](https://md.research.elamur.ai/free/en/authors/nizar-habash/) and [Ted Briscoe](https://md.research.elamur.ai/free/en/authors/ted-briscoe/) · **Year:** 2025 · **Venue:** [Proceedings of the 20th Workshop on Innovative Use of NLP for Building Educational Applications (BEA 2025)](https://md.research.elamur.ai/free/en/venues/bea/) ([BEA 2025](https://md.research.elamur.ai/free/en/venues/bea/2025/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2025.bea-1.40/) · [PDF](https://aclanthology.org/2025.bea-1.40.pdf) · [DOI 10.18653/v1/2025.bea-1.40](https://doi.org/10.18653/v1/2025.bea-1.40) · [OpenAlex](https://openalex.org/W4412889250)

**Citations:** 5 citations (OpenAlex, 2026-10-14) · [8](https://md.research.elamur.ai/free/en/papers/qwaider-2025-enhancing-arabic-automated-essay-scoring/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
Arabic automated essay scoring (AES) has little annotated training data. The main resource, ZAEBUC, has only 216 CEFR-rated essays. These are concentrated at B1–B2, with none at A1 or C2. The authors build a synthetic dataset to fill this gap in three phases:

- They compile a level-balanced bank of essay prompts.
- They profile each CEFR level in ZAEBUC by length, lexical diversity, syntactic depth and POS usage.
- They ask GPT-4o to write clean essays per level, then add learner-like errors.

They compare two ways of adding errors. One prompts GPT-4o through multiple steps. The other is a controlled two-stage model: a fine-tuned CAMeLBERT MSA predicts an error type for each word, and a bigram MLE model chooses the concrete edit. The result is 3,040 CEFR-labelled essays, used to train a BERT-based scorer. The hybrid of GPT text plus controlled injection beats the baseline and is easier to interpret. This suggests that profile-guided synthetic data can partly replace scarce learner corpora.

## Key points
- The seed corpus is ZAEBUC, with splits from Alhafni et al. (2023). It covers three prompts (Social Media, Tolerance, Development) and is distributed as 51% B1, 37% B2, 5% C1 and 3% A2, with no A1 or C2.
- The authors drafted prompts with GPT-4o, Gemini and Copilot, then reviewed them by hand. The final 152 prompts span 17 topics and three bands (Beginner, Intermediate, Advanced).
- Each level profile includes word and sentence counts, word and sentence lengths, type–token ratio, mean syntactic tree depth, and frequent POS tags from ZAEBUC annotations.
- The released synthetic dataset has 3,040 essays and is available at github.com/mbzuai-nlp/arabic-aes-bea25. The scorer is a fine-tuned BERT classifier evaluated with QWK and F1; the conclusion reports QWK 27.87% and F1 38.02% in its baseline comparison.
- The authors list several limitations:
  - GPT essays match the linguistic profiles only about 27.5% of the time.
  - There is no gold data for A1 or C2.
  - No human evaluation of the synthetic essays was carried out.

## Interesting ideas
- Split synthetic error generation into two stages: first decide which word gets which error type, then pick the surface edit from corpus-estimated transformation statistics. This keeps the error mix proportional to real data and makes each injected error traceable.
- Build a quantitative profile of each target class from gold data (length, TTR, tree depth, POS mix) and use it to check whether generated text actually resembles that class.
- Use average dependency-tree depth as a simple, explainable measure of sentence complexity.
- Write prompts per difficulty level, reserving abstract topics for higher levels and general themes for all levels.
- Compare a free-form LLM corruption pass against a controlled, statistics-driven one, and prefer the controlled one for its reliability and interpretability.

## For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

- **I1** (tokens): Generate realistic misspelled or misagreed English inputs with known corruption labels, to stress-test and harden the tokenizer, tagger and lemmatizer.
- **I2** (learn): Compare LLM-teacher or self-play outputs against profiles from real treebank text as a cheap sanity check, rejecting silver data that drifts too far.
- **I3** (tree): Stratify parser evaluation (perceptron vs gparse) by tree-depth buckets to see where accuracy degrades on deeper structures.
- **I4** (answer): Build graded reading and test passages for the school-course readers, so comprehension can be measured as text difficulty rises.
- **I5** (learn): When choosing LLM teacher tiers versus rule-driven generators for training data, run the same head-to-head comparison rather than assuming the LLM is better.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [New York University Abu Dhabi](https://md.research.elamur.ai/free/en/universities/new-york-university-abu-dhabi/) — University, Abu Dhabi, United Arab Emirates · 122 works · 4,602 citations

📚 Shelves (5): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works · [🗄️ Information Systems](https://md.research.elamur.ai/free/en/shelves/computer-science/information-systems/) · 1,437 works

🧭 Topics (2): [Semitic languages](https://md.research.elamur.ai/free/en/topics/semitic-languages/) · 2,908 works · [🖍️ Grammatical error correction & learner language](https://md.research.elamur.ai/free/en/topics/grammatical-error-correction/) · 156 works

## Cite

Cite as: Chatrine Qwaider, Bashar Alhafni, Kirill Chirkunov, Nizar Habash and Ted Briscoe. 2025. Enhancing Arabic Automated Essay Scoring with Synthetic Data and Error Injection. Proceedings of the 20th Workshop on Innovative Use of NLP for Building Educational Applications (BEA 2025).

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{qwaider2025enhancing,
  title = {{Enhancing Arabic Automated Essay Scoring with Synthetic Data and Error Injection}},
  author = {Chatrine Qwaider, Bashar Alhafni, Kirill Chirkunov, Nizar Habash and Ted Briscoe},
  booktitle = {Proceedings of the 20th Workshop on Innovative Use of NLP for Building Educational Applications (BEA 2025)},
  year = {2025},
  doi = {10.18653/v1/2025.bea-1.40},
  url = {https://doi.org/10.18653/v1/2025.bea-1.40}
}
```

- [← previous Embracing Imperfection: Simulating Students with Diverse Cognitive Levels Using LLM-based Agents](https://md.research.elamur.ai/free/en/papers/wu-2025-embracing-imperfection-simulating-students-diverse/)
- [next → Enhancing Automated Interpretability with Output-Centric Feature Descriptions](https://md.research.elamur.ai/free/en/papers/arieh-2025-enhancing-automated-interpretability-output-centric/)
- [↑ back to papers of 2025](https://md.research.elamur.ai/free/en/papers/2025/page/6/)

## References

29 references, 7 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. D Blanchard. 2013. Toefl11: A corpus of non-native english. Educational Testing Service. [↗](https://doi.org/10.1002/j.2333-8504.2013.tb02331.x)
2. Abdelhamid M Ahmed, Xiao Zhang, Lameya M Rezk, and Wajdi Zaghouani. 2024. Building an annotated l1 arabic/l2 english bilingual writer corpus: The qatari Jacob Cohen. 1968. Weighted kappa: Nominal scale agreement provision for scaled disagreement or partial credit. Psychological bulletin, 70(4):213. [find](https://scholar.google.com/scholar?q=Abdelhamid+M+Ahmed%2C+Xiao+Zhang%2C+Lameya+M+Rezk%2C+and+Wajdi+Zaghouani%2E+2024%2E+Building+an+annotated+l1+arabic%2Fl2+english+bilingual+writer+corpus%3A+The+qatari+Jacob+Cohen%2E+1968%2E+Weighted+kappa%3A+Nominal+scal)
3. C. o. E. Council of Europe. 2001. Common european framework of reference for languages: learning, teaching, assessment. [find](https://scholar.google.com/scholar?q=C%2E+o%2E+E%2E+Council+of+Europe%2E+2001%2E+Common+european+framework+of+reference+for+languages%3A+learning%2C+teaching%2C+assessment%2E)
4. ACL 2024, pages 5622–5640, Bangkok, Thailand. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=ACL+2024%2C+pages+5622%E2%80%935640%2C+Bangkok%2C+Thailand%2E+Association+for+Computational+Linguistics%2E)
5. Chima Elhaddadi, Imad Zeroual, and Anoual El Kah. 2024. Automatic arabic essays scoring: A scoping review. In International Conference on Arabic Language Processing, pages 38–48. Springer. [find](https://scholar.google.com/scholar?q=Chima+Elhaddadi%2C+Imad+Zeroual%2C+and+Anoual+El+Kah%2E+2024%2E+Automatic+arabic+essays+scoring%3A+A+scoping+review%2E+In+International+Conference+on+Arabic+Language+Processing%2C+pages+38%E2%80%9348%2E+Springer%2E)
6. Paraskevas Lagakis and Stavros Demetriadis. 2021. Automated essay scoring: A review of the field. In 2021 International Conference on Computer, Information and Telecommunication Systems (CITS), pages 1–6. IEEE. [find](https://scholar.google.com/scholar?q=Paraskevas+Lagakis+and+Stavros+Demetriadis%2E+2021%2E+Automated+essay+scoring%3A+A+review+of+the+field%2E+In+2021+International+Conference+on+Computer%2C+Information+and+Telecommunication+Systems+%28CITS%29%2C+pages+)
7. S Elliott, MD Shermis, and J Burstein. 2003. Overview of intellimetric. Automated Essay Scoring: A Cross-Disciplinary Perspective, pages 67–70. [find](https://scholar.google.com/scholar?q=S+Elliott%2C+MD+Shermis%2C+and+J+Burstein%2E+2003%2E+Overview+of+intellimetric%2E+Automated+Essay+Scoring%3A+A+Cross%2DDisciplinary+Perspective%2C+pages+67%E2%80%9370%2E)
8. Bohan Li, Yutai Hou, and Wanxiang Che. 2022. Data augmentation approaches in natural language processing: A survey. Ai Open, 3:71–90. [↗](https://doi.org/10.1016/j.aiopen.2022.03.001)
9. Marwa M Gaheen, Rania M ElEraky, and Ahmed A Ewees. 2021. Automated students arabic essay scoring using trained neural network by e-jaya optimization to support personalized system of instruction. Education and Information Technologies, 26:1165– 1181. [↗](https://doi.org/10.1007/s10639-020-10300-6)
10. Chun Then Lim, Chih How Bong, Wee Sian Wong, and Nung Kion Lee. 2021. A comprehensive review of automated essay scoring (aes) research and development. Pertanika Journal of Science & Technology, 29(3):1875–1899. [↗](https://doi.org/10.47836/pjst.29.3.27)
11. Rayed Ghazawi and Edwin Simpson. 2024. [Automated essay scoring in arabic: a dataset and analysis of a bert-based system](https://md.research.elamur.ai/free/en/papers/ghazawi-2024-automated-essay-scoring-arabic-dataset/). arXiv preprint arXiv:2407.11212. [↗](https://arxiv.org/abs/2407.11212)
12. Lin Long, Rui Wang, Ruixuan Xiao, Junbo Zhao, Xiao Ding, Gang Chen, and Haobo Wang. 2024. [On llms-driven synthetic data generation, curation, and evaluation: A survey](https://md.research.elamur.ai/free/en/papers/long-2024-llms-driven-synthetic-data-generation/). arXiv preprint arXiv:2406.15126. [↗](https://arxiv.org/abs/2406.15126)
13. Sylviane Granger. 2003. The international corpus of learner english: a new resource for foreign language learning and teaching and second language acquisition research. Tesol Quarterly, 37(3):538–546. [↗](https://doi.org/10.2307/3588404)
14. Dadi Ramesh and Suresh Kumar Sanampudi. 2022. An automated essay scoring systems: a systematic literature review. Artificial Intelligence Review, 55(3):2495–2527. [↗](https://doi.org/10.1007/s10462-021-10068-2)
15. Nizar Habash and David Palfreyman. 2022. ZAEBUC: An annotated Arabic-English bilingual writer corpus. In Proceedings of the Thirteenth Language Resources and Evaluation Conference, pages 79–88, Marseille, France. European Language Resources Association. [find](https://scholar.google.com/scholar?q=Nizar+Habash+and+David+Palfreyman%2E+2022%2E+ZAEBUC%3A+An+annotated+Arabic%2DEnglish+bilingual+writer+corpus%2E+In+Proceedings+of+the+Thirteenth+Language+Resources+and+Evaluation+Conference%2C+pages+79%E2%80%9388%2C+Marsei)
16. Dania Refai, Saleh Abu-Soud, and Mohammad J Abdel-Rahman. 2023. [Data augmentation using transformers and similarity measures for improving arabic text classification](https://md.research.elamur.ai/free/en/papers/refai-2022-data-augmentation-using-transformers-similarity/). IEEE Access, 11:132516–132531. [↗](https://doi.org/10.1109/access.2023.3336311)
17. Marcelo Guerra Hahn, Silvia Margarita Baldiris Navarro, Luis De La Fuente Valentín, and Daniel Burgos. 2021. A systematic review of the effects of automatic scoring and automatic feedback in educational settings. IEEE Access, 9:108190–108198. [↗](https://doi.org/10.1109/access.2021.3100890)
18. Go Inoue, Bashar Alhafni, Nurpeiis Baimukan, Houda Bouamor, and Nizar Habash. 2021. [The interplay of variant, size, and task type in Arabic pre-trained language models](https://md.research.elamur.ai/free/en/papers/inoue-2021-interplay-variant-size-task-type/). In Proceedings of the Sixth Arabic Natural Language Processing Workshop, pages 92– 104, Kyiv, Ukraine (Virtual). Association for Computational Linguistics.
19. Zixuan Ke and Vincent Ng. 2019. Automated essay scoring: A survey of the state of the art. In Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence, IJCAI-19, pages 6300–6308. International Joint Conferences on Artificial Intelligence Organization. [find](https://scholar.google.com/scholar?q=Zixuan+Ke+and+Vincent+Ng%2E+2019%2E+Automated+essay+scoring%3A+A+survey+of+the+state+of+the+art%2E+In+Proceedings+of+the+Twenty%2DEighth+International+Joint+Conference+on+Artificial+Intelligence%2C+IJCAI%2D19%2C+page)
20. Mohamed Bayan Kmainasi, Rakif Khan, Ali Ezzat Shahroor, Boushra Bendou, Maram Hasanain, and Firoj Alam. 2024. Native vs non-native language prompting: A comparative analysis. In International Conference on Web Information Systems Engineering, pages 406–420. Springer. [find](https://scholar.google.com/scholar?q=Mohamed+Bayan+Kmainasi%2C+Rakif+Khan%2C+Ali+Ezzat+Shahroor%2C+Boushra+Bendou%2C+Maram+Hasanain%2C+and+Firoj+Alam%2E+2024%2E+Native+vs+non%2Dnative+language+prompting%3A+A+comparative+analysis%2E+In+International+Conferen)
21. Fajri Koto, Haonan Li, Sara Shatnawi, Jad Doughman, Abdelrahman Sadallah, Aisha Alraeesi, Khalid Almubarak, Zaid Alyafeai, Neha Sengupta, Shady Shehata, Nizar Habash, Preslav Nakov, and Timothy Baldwin. 2024. [ArabicMMLU: Assessing massive multitask language understanding in Arabic](https://md.research.elamur.ai/free/en/papers/koto-2024-arabicmmlu-assessing-massive-multitask-language/). In Findings of the Association for Computational Linguistics: Caroline Sabty, Islam Omar, Fady Wasfalla, Mohamed Islam, and Slim Abdennadher. 2021. Data augmentation techniques on arabic data for named entity recognition. Procedia Computer Science, 189:292– 299. [↗](https://doi.org/10.48550/arxiv.2402.12840)
22. Vinay Samuel, Houda Aynaou, Arijit Chowdhury, Karthik Venkat Ramanan, and Aman Chadha. 2024. [Can LLMs augment low-resource reading comprehension datasets? opportunities and challenges](https://md.research.elamur.ai/free/en/papers/samuel-2023-llms-augment-low-resource-reading/). In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 4: Student Research Workshop), pages 307–317, Bangkok, Thailand. Association for Computational Linguistics. [↗](https://doi.org/10.13140/rg.2.2.27139.07205)
23. Mark D Shermis and Jill C Burstein. 2003. Automated essay scoring: A cross-disciplinary perspective. Routledge. [find](https://scholar.google.com/scholar?q=Mark+D+Shermis+and+Jill+C+Burstein%2E+2003%2E+Automated+essay+scoring%3A+A+cross%2Ddisciplinary+perspective%2E+Routledge%2E)
24. Aiman Solyman, Marco Zappatore, Wang Zhenyu, Zeinab Mahmoud, Ali Alfatemi, Ashraf Osman Ibrahim, and Lubna Abdelkareim Gabralla. 2023. Optimizing the impact of data augmentation for low-resource grammatical error correction. Journal of King Saud University-Computer and Information Sciences, 35(6):101572. [↗](https://doi.org/10.1016/j.jksuci.2023.101572)
25. Meilia Nur Indah Susanti, Arief Ramadhan, and Harco Leslie Hendric Spit Warnars. 2023. Automatic essay exam scoring system: A systematic literature review. Procedia Computer Science, 216:531–538. [↗](https://doi.org/10.1016/j.procs.2022.12.166)
26. Ke Wang, Jiahui Zhu, Minjie Ren, Zeming Liu, Shiwei Li, Zongye Zhang, Chenkai Zhang, Xiaoyu Wu, Qiqi Zhan, Qingjie Liu, et al. 2024a. A survey on data synthesis and augmentation for large language models. arXiv preprint arXiv:2410.12896. [↗](https://arxiv.org/abs/2410.12896)
27. Yidong Wang, Qi Guo, Wenjin Yao, Hongbo Zhang, Xin Zhang, Zhen Wu, Meishan Zhang, Xinyu Dai, Min Zhang, Qingsong Wen, et al. 2024b. Autosurvey: Large language models can automatically write surveys. arXiv preprint arXiv:2406.10252. [↗](https://arxiv.org/abs/2406.10252)
28. Helen Yannakoudakis, Øistein E Andersen, Ardeshir Geranpayeh, Ted Briscoe, and Diane Nicholls. 2018. Developing an automated writing placement system for esl learners. Applied Measurement in Education, 31(3):251–267. [↗](https://doi.org/10.1080/08957347.2018.1464447)
29. Helen Yannakoudakis, Ted Briscoe, and Ben Medlock. 2011. [A new dataset and method for automatically grading esol texts](https://md.research.elamur.ai/free/en/papers/yannakoudakis-2011-new-dataset-method-automatically-grading/). In Proceedings of the 49th annual meeting of the association for computational linguistics: human language technologies, pages 180–189.

*Updated 1 November 2026*