---
title: "Enhancing Arabic NLP Tasks through Character-Level Models"
description: "They compare three architectures: a CharCNN, a CharBiLSTM and the pre-trained character… 27 references, cited by 1 paper here."
authors: ["Mohanad Mohamed", "Sadam Al-Azani"]
year: 2025
venue: "Proceedings of the 31st International Conference on Computational Linguistics"
source: "https://aclanthology.org/2025.coling-main.186/"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/mohamed-2025-enhancing-arabic-nlp-tasks-through/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/mohamed-2025-enhancing-arabic-nlp-tasks-through/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2025](https://md.research.elamur.ai/free/en/papers/2025/page/14/) / Enhancing Arabic NLP Tasks through Character-Level Models and Data Augmentation

# Enhancing Arabic NLP Tasks through Character-Level Models and Data Augmentation

**Authors:** Mohanad Mohamed and [Sadam Al-Azani](https://md.research.elamur.ai/free/en/authors/sadam-al-azani/) · **Year:** 2025 · **Venue:** [Proceedings of the 31st International Conference on Computational Linguistics](https://md.research.elamur.ai/free/en/venues/coling/) ([COLING 2025](https://md.research.elamur.ai/free/en/venues/coling/2025/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2025.coling-main.186/) · [PDF](https://aclanthology.org/2025.coling-main.186.pdf)

**Citations:** 1 in our library, of the 149,439 papers whose bibliographies we read

## Summary
Arabic is difficult for systems built on word or subword tokens because of its rich morphology, root-based word formation, missing diacritics and spelling variation. The authors ask whether small character-level models, given extra synthetic training data, can match large BERT-style classifiers. They compare three architectures: a CharCNN, a CharBiLSTM and the pre-trained character transformer CANINE-s. The test task is the Saudi Privacy Policy Dataset, where text lines are labelled with one of ten principles of the Saudi PDPL.

They test three generic augmentation methods: back translation, and contextual embedding substitution and insertion. They also propose two methods specific to Arabic:
- **Vowel deletion:** removing a random, first or last vowel letter.
- **Style transfer:** rewriting nominal sentences as verbal ones and the reverse.

CANINE-s with last-vowel deletion scored the best micro-F1, 93.8%. That narrowly beats the earlier AraBERT, MARBERT and CamelBERT results. A CharCNN under one million parameters came close behind. The work shows that cheap character models plus language-aware augmentation can compete with much heavier models. The evidence comes from a single classification task.

## Key points
- **Data:** 4,638 annotated lines from 1,000 Saudi privacy policies. Lengths are strongly right-skewed: mean about 790 characters, median 459.5, maximum 16,258. The split is 2,968 train, 742 validation and 928 test, and the test size matches the dataset creators' setup.
- **Tokenization:** CharCNN and CharBiLSTM use a fixed inventory of 46 symbols (Arabic letters, digits, punctuation). CANINE-s reads raw characters with no vocabulary.
- **Baselines without augmentation:** micro-F1 was 0.9224 for both CharCNN and CANINE-s, and 0.8405 for CharBiLSTM.
- **Best result per model:**

  | Model | Best augmentation | Micro-F1 |
  |---|---|---|
  | CANINE-s | Last vowel deletion | 0.938 |
  | CharCNN | Last vowel deletion | 0.931 |
  | CharBiLSTM | Back translation + insertion + substitution combined | 0.879 |

  For the CharBiLSTM, vowel deletion and style transfer gave little or no gain and could hurt; style transfer scored 0.811.
- **Model size:** CharCNN has 943,946 parameters and CANINE-s about 132M. The prior BERT models are roughly 300M. Their published scores are 0.932 (AraBERT) and 0.933 (MARBERT, CamelBERT).

## Interesting ideas
- Deleting a vowel letter at a controlled position (random, first, last) gives label-preserving noise that teaches a character model to rely on the consonant skeleton; the position choice itself is a tunable variable worth comparing.
- Syntactic paraphrase as augmentation: converting nominal sentences to verbal ones (and back) changes structure while keeping meaning and label.
- The same augmentation can help one architecture and harm another, as vowel deletion boosted CANINE-s and CharCNN but hurt the BiLSTM, so each method must be scored per model rather than assumed beneficial.
- Results are reported next to parameter counts, which shows a sub-million-parameter CharCNN nearly matching ~300M-parameter BERTs and makes cost-versus-accuracy explicit.
- The test set is sized to match the dataset creators' original setup so that comparison with earlier results stays fair.

## For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

- **I1** (tokens): A similar position-controlled character corruption (dropped letters, typo-like edits) could stress-test and harden the tokenizer and tagger against misspelled input, reporting results per corruption type.
- **I2** (frames): English analogues such as active↔passive or nominalization↔clause rewrites should leave who-did-what-to-whom unchanged, so they make a natural invariance test for role assignment.
- **I3** (learn): When adding synthetic or silver data, evaluate its effect separately for the perceptron parser and gparse, and keep a no-augmentation run as a control.
- **I4** (learn): Logging model size and cost beside every accuracy figure would help decide where a small explainable component is enough instead of a heavy one.
- **I5** (answer): When scoring readers on school tests or fable QA, reuse the exact official splits so comparisons with published systems remain meaningful.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

📚 Shelves (2): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works

🧭 Topics (1): [Semitic languages](https://md.research.elamur.ai/free/en/topics/semitic-languages/) · 2,908 works

## Cite

Cite as: Mohanad Mohamed and Sadam Al-Azani. 2025. Enhancing Arabic NLP Tasks through Character-Level Models and Data Augmentation. Proceedings of the 31st International Conference on Computational Linguistics.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{mohamed2025enhancing,
  title = {{Enhancing Arabic NLP Tasks through Character-Level Models and Data Augmentation}},
  author = {Mohanad Mohamed and Sadam Al-Azani},
  booktitle = {Proceedings of the 31st International Conference on Computational Linguistics},
  year = {2025},
  url = {https://aclanthology.org/2025.coling-main.186/}
}
```

- [← previous Enhancing Arabic Dialectal Sentiment Analysis through Advanced Data Augmentation Techniques](https://md.research.elamur.ai/free/en/papers/biswas-2025-enhancing-arabic-dialectal-sentiment-analysis/)
- [next → Enhancing the Automatic Classification of Metadiscourse in Low-Proficiency Learners' Spoken and Written English Texts Using XLNet](https://md.research.elamur.ai/free/en/papers/guan-2025-enhancing-automatic-classification-metadiscourse-low/)
- [↑ back to papers of 2025](https://md.research.elamur.ai/free/en/papers/2025/page/14/)

## References

27 references, 8 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Sadam Al-Azani, Nora Alturayeif, Haneen Abouelresh, and Alhanoof Alhunief. 2024. A comprehensive framework and empirical analysis for evaluating large language models in arabic dialect identification. In 2024 International Joint Conference on Neural Networks (IJCNN), pages 1–7. IEEE. [find](https://scholar.google.com/scholar?q=Sadam+Al%2DAzani%2C+Nora+Alturayeif%2C+Haneen+Abouelresh%2C+and+Alhanoof+Alhunief%2E+2024%2E+A+comprehensive+framework+and+empirical+analysis+for+evaluating+large+language+models+in+arabic+dialect+identification%2E)
2. Hend Al-Khalifa, Malak Mashaabi, Ghadi Al-Yahya, and Raghad Alnashwan. 2023. The saudi privacy policy dataset. arXiv preprint arXiv:2304.02757. [↗](https://arxiv.org/abs/2304.02757)
3. Abdullah I Alharbi, Phillip Smith, and Mark Lee. 2022. Integrating character-level and word-level representation for affect in arabic tweets. Data & Knowledge Engineering, 138:101973. [↗](https://doi.org/10.1016/j.datak.2021.101973)
4. Abdullah M Alkadri, Abeer Elkorany, and Cherry Ahmed. 2022. Enhancing detection of arabic social spam using data augmentation and machine learning. Applied Sciences, 12(22):11388. [↗](https://doi.org/10.3390/app122211388)
5. Abdulelah Alkesaiberi, Ali Alkhathlan, and Ahmed Abdelali. 2024. Ara–canine: Character-based pre-trained language model for arabic language understanding. International Journal on Cybernetics & Informatics (IJCI), 13(13):45. [↗](https://doi.org/10.5121/ijci.2024.130204)
6. Tahani Alqurashi. 2022. Applying a character-level model to a short arabic dialect sentence: A saudi dialect as a case study. Applied Sciences, 12(23). [↗](https://doi.org/10.3390/app122312435)
7. Zaid Alyafeai, Maged S Al-shaibani, Mustafa Ghaleb, and Irfan Ahmad. 2023. [Evaluating various tokenizers for arabic text classification](https://md.research.elamur.ai/free/en/papers/alyafeai-2021-evaluating-various-tokenizers-arabic-text/). Neural Processing Letters, 55(3):2911–2933. [↗](https://doi.org/10.1007/s11063-022-10990-8)
8. Wissam Antoun, Fady Baly, and Hazem Hajj. 2020a. [Arabert: Transformer-based model for arabic language understanding](https://md.research.elamur.ai/free/en/papers/antoun-2020-arabert-transformer-based-model-arabic/). arXiv preprint arXiv:2003.00104. [↗](https://arxiv.org/abs/2003.00104)
9. Wissam Antoun, Fady Baly, and Hazem Hajj. 2020b. [Aragpt2: Pre-trained transformer for arabic language generation](https://md.research.elamur.ai/free/en/papers/antoun-2021-aragpt2-pre-trained-transformer-arabic/). arXiv preprint arXiv:2012.15520. [↗](https://arxiv.org/abs/2012.15520)
10. Markus Bayer, Marc-André Kaufhold, Björn Buchhold, Marcel Keller, Jörg Dallmeyer, and Christian Reuter. 2023. Data augmentation in natural language processing: a novel text generation approach for long and short text classifiers. International journal of machine learning and cybernetics, 14(1):135–150. [↗](https://doi.org/10.1007/s13042-022-01553-3)
11. Tyler A Chang and Benjamin K Bergen. 2024. [Language model behavior: A comprehensive survey](https://md.research.elamur.ai/free/en/papers/chang-2024-language-model-behavior-comprehensive-survey/). Computational Linguistics, 50(1):293–350. [↗](https://doi.org/10.1162/coli_a_00492)
12. Jonathan H Clark, Dan Garrette, Iulia Turc, and John Wieting. 2022. [Canine: Pre-training an efficient tokenization-free encoder for language representation](https://md.research.elamur.ai/free/en/papers/clark-2022-canine-pre-training-efficient-tokenization/). Transactions of the Association for Computational Linguistics, 10:73–91. [↗](https://doi.org/10.1162/tacl_a_00448)
13. Huu-Thanh Duong and Tram-Anh Nguyen-Thi. 2021. A review: preprocessing techniques and data augmentation for sentiment analysis. Computational Social Networks, 8(1):1. [↗](https://doi.org/10.1186/s40649-020-00080-x)
14. Jeremy Howard and Sebastian Ruder. 2018. Universal language model fine-tuning for text classification. arXiv preprint arXiv:1801.06146. [↗](https://arxiv.org/abs/1801.06146)
15. Rafal Jozefowicz, Oriol Vinyals, Mike Schuster, Noam Shazeer, and Yonghui Wu. 2016. Exploring the limits of language modeling. arXiv preprint arXiv:1602.02410. [↗](https://arxiv.org/abs/1602.02410)
16. Yoon Kim, Yacine Jernite, David Sontag, and Alexander Rush. 2016. [Character-aware neural language models](https://md.research.elamur.ai/free/en/papers/kim-2015-character-aware-neural-language-models/). In Proceedings of the AAAI conference on artificial intelligence, volume 30.
17. Sosuke Kobayashi. 2018. [Contextual augmentation: Data augmentation by words with paradigmatic relations](https://md.research.elamur.ai/free/en/papers/kobayashi-2018-contextual-augmentation-data-augmentation-words/). arXiv preprint arXiv:1805.06201. [↗](https://arxiv.org/abs/1805.06201)
18. Malak Mashaabi, Ghadi Al-Yahya, Raghad Alnashwan, and Hend Al-Khalifa. 2023. Arabic privacy policy corpus and classification. In Natural Language Processing and Information Systems, pages 94–108, Cham. Springer Nature Switzerland. [find](https://scholar.google.com/scholar?q=Malak+Mashaabi%2C+Ghadi+Al%2DYahya%2C+Raghad+Alnashwan%2C+and+Hend+Al%2DKhalifa%2E+2023%2E+Arabic+privacy+policy+corpus+and+classification%2E+In+Natural+Language+Processing+and+Information+Systems%2C+pages+94%E2%80%93108%2C+Cham)
19. Ebtsam A Mohamed, Walaa N Ismail, Osman Ali Sadek Ibrahim, and Eman MG Younis. 2024. A two-stage framework for arabic social media text misinformation detection combining data augmentation and arabert. Social Network Analysis and Mining, 14(1):53. [↗](https://doi.org/10.1007/s13278-024-01201-4)
20. Eslam Omara, Mervat Mousa, and Nabil Ismail. 2022. Character gated recurrent neural networks for arabic sentiment analysis. Scientific Reports, 12(1):9779. [↗](https://doi.org/10.1038/s41598-022-13153-w)
21. Dania Refai, Saleh Abu-Soud, and Mohammad J Abdel-Rahman. 2023. [Data augmentation using transformers and similarity measures for improving arabic text classification](https://md.research.elamur.ai/free/en/papers/refai-2022-data-augmentation-using-transformers-similarity/). IEEE Access, 11:132516–132531. [↗](https://doi.org/10.1109/access.2023.3336311)
22. Gözde Gül Şahin. 2022. To augment or not to augment? a comparative study on text augmentation techniques for low-resource nlp. Computational Linguistics, 48(1):5–42. and 4, and classes 1 and 4. The confusion matrix highlights that even with the impressive performance of the model, there’s room for further improvement, especially in distinguishing between specific classes. [↗](https://doi.org/10.48448/n37p-t696)
23. Rico Sennrich, Barry Haddow, and Alexandra Birch. 2015. Improving neural machine translation models with monolingual data. arXiv preprint arXiv:1511.06709. [↗](https://arxiv.org/abs/1511.06709)
24. Khaled Shaalan, Sanjeera Siddiqui, Manar Alkhatib, and Azza Abdel Monem. 2019. Challenges in arabic natural language processing. In Computational linguistics, speech and image processing for Arabic language, pages 59–83. World Scientific. [↗](https://doi.org/10.1142/9789813229396_0003)
25. Clara Vania. 2020. On understanding character-level models for representing morphology. [find](https://scholar.google.com/scholar?q=Clara+Vania%2E+2020%2E+On+understanding+character%2Dlevel+models+for+representing+morphology%2E)
26. Linting Xue, Aditya Barua, Noah Constant, Rami Al-Rfou, Sharan Narang, Mihir Kale, Adam Roberts, and Colin Raffel. 2022. Byt5: Towards a token-free future with pre-trained byte-to-byte models. Transactions of the Association for Computational Linguistics, 10:291–306. [↗](https://doi.org/10.1162/tacl_a_00461)
27. Farida Yasser, Souzan Hatem, Laila Ayman, Loujain Mohamed, and Ammar Mohammed. 2024. Data augmentation for sentiment analysis enhancement: A comparative study. In 2024 6th International Conference on Computing and Informatics (ICCI), pages 53–58. IEEE. [find](https://scholar.google.com/scholar?q=Farida+Yasser%2C+Souzan+Hatem%2C+Laila+Ayman%2C+Loujain+Mohamed%2C+and+Ammar+Mohammed%2E+2024%2E+Data+augmentation+for+sentiment+analysis+enhancement%3A+A+comparative+study%2E+In+2024+6th+International+Conference+on+)

*Updated 1 November 2026*