--- title: "ARBML: Democratizing Arabic Natural Language Processing Tools" description: "Arabic NLP lags behind English in open tooling. 1 reference, cited by 3 papers here. Among them: 11." authors: ["Zaid Alyafeai", "Maged S. Al-Shaibani"] year: 2020 venue: "Proceedings of Second Workshop for NLP Open Source Software (NLP-OSS)" source: "https://aclanthology.org/2020.nlposs-1.2/" doi: "10.18653/v1/2020.nlposs-1.2" licence: "CC BY 4.0 (ACL Anthology)" licence_url: "https://creativecommons.org/licenses/by/4.0/" summary_licence: "Elamur Cosmic License" canonical: "https://research.elamur.ai/free/en/papers/alyafeai-2020-arbml-democratizing-arabic-natural-language/" --- [Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/alyafeai-2020-arbml-democratizing-arabic-natural-language/) [home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2020](https://md.research.elamur.ai/free/en/papers/2020/page/8/) / ARBML: Democratizing Arabic Natural Language Processing Tools # ARBML: Democratizing Arabic Natural Language Processing Tools **Authors:** [Zaid Alyafeai](https://md.research.elamur.ai/free/en/authors/zaid-alyafeai/) and [Maged S. Al-Shaibani](https://md.research.elamur.ai/free/en/authors/maged-s-al-shaibani/) · **Year:** 2020 · **Venue:** [Proceedings of Second Workshop for NLP Open Source Software (NLP-OSS)](https://md.research.elamur.ai/free/en/venues/nlposs/) ([NLP-OSS 2020](https://md.research.elamur.ai/free/en/venues/nlposs/2020/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/) **Links:** [Publisher's page](https://aclanthology.org/2020.nlposs-1.2/) · [PDF](https://aclanthology.org/2020.nlposs-1.2.pdf) · [DOI 10.18653/v1/2020.nlposs-1.2](https://doi.org/10.18653/v1/2020.nlposs-1.2) · [OpenAlex](https://openalex.org/W3106209420) **Citations:** 11 citations (OpenAlex, 2026-10-14) · [3](https://md.research.elamur.ai/free/en/papers/alyafeai-2020-arbml-democratizing-arabic-natural-language/cited-by/) in our library, of the 149,439 papers whose bibliographies we read ## Summary Arabic NLP lags behind English in open tooling. Existing Arabic toolkits (MADAMIRA, FARASA, CAMeL NLP, Adawat) mostly cover classic linguistic analysis, seldom use recent deep learning, and are often closed or limited to educational licenses. ARBML responds with a fully open GitHub project that covers the whole workflow, from raw web text to models running in a browser. It has three parts: - **Tnqeeb** scrapes and collects corpora. - **Tnkeeh** cleans, normalizes, segments and splits data. - **Tkseem** provides six interchangeable tokenizers behind one shared interface. On top of these, the authors gathered eight datasets and trained a dozen Keras models, ranging from diacritization and translation to poem-meter classification. Each model comes with a Colab notebook and runs client-side via TensorFlow.js. The paper reports no benchmark scores. Its contribution is the reproducible, openly licensed pipeline, which the authors argue can be carried over to other languages. It is aimed at learners, developers prototyping for industry, and researchers who want a starting point. ## Key points - **Tnqeeb** uses scrapy to gather poetry from three large Arabic sites (aldiwan, poetry encyclopedia, poets gate). It then groups and de-duplicates the data into separate repositories. - **Tnkeeh** handles preprocessing: - cleaning, which removes diacritics, Tatweel, extra spaces and English characters; - FARASA morpheme segmentation; - a dictionary-based Unicode normalization; - task-aware splitting for unsupervised, classification or parallel data. - **Tkseem** offers six tokenizers: word, character, sentencepiece, morphological (trained with Madamira on a news corpus), random, and disjoint-letter. - All share one base class with tokenize, encode, decode and detokenize. - Tokenizers can be serialized, and caching and memory-mapped files speed them up. - They were tested on sentiment analysis, poem-meter classification and NMT. - **Datasets** include: - 146,604 poems; - 55,440 verses labeled with meters; - 100,000 Arabic–English pairs from OpenSubtitles; - 30,000 translated captions from Lin et al. (2014); - 1,648 product reviews; - digit, letter and font image sets. - **Models and comparison:** twelve models were built in Keras/TensorFlow (e.g. seq2seq with attention, Bidirectional GRU meter classifier, Aravec embeddings, SSD detector) and exported to TensorFlow.js for offline in-browser use. A comparison table against Stanford CoreNLP and the Arabic tools above focuses on code openness and model availability. ## Interesting ideas - **Random tokenizer as a baseline.** Placing a random splitter next to linguistically informed tokenizers, behind the same interface, gives a cheap floor for judging whether segmentation actually helps a downstream task. - **Extrinsic tokenizer evaluation.** Tokenizers are judged by plugging them into several different downstream tasks rather than by intrinsic metrics alone. This shows which segmentation actually serves applications. - **Canonicalization table for Unicode variants.** A plain mapping table turns visually identical but differently encoded characters into one canonical form before any other processing. - **Serialized tokenizers with caching.** Trained tokenizers are saved and reloaded on demand, and caching plus memory-mapped files make repeated runs over big corpora cheaper. - **Collection pipeline with de-duplication.** Scraped data from each source is grouped, lightly processed and de-duplicated into its own repository before use. This keeps provenance clear and avoids duplicates inflating results. ## For Mova How these ideas could serve Mova, our small open language model — its files, stages and tests: - **I1** (learn): Add a random-segmentation or random-attachment witness as a negative control when scoring silver data, so a component counts as useful only if it clearly beats chance. - **I2** (tokens): Compare tokenizer or lemmatizer variants by their effect on parser LAS and on answer accuracy, not only by token-level agreement. - **I3** (tokens): A small explicit table for quotes, dashes, ligatures and look-alike letters, applied first and logged, would keep the tokenizer explainable and stop spurious mismatches with treebank text. - **I4** (learn): Cache tokenized and parsed corpora as memory-mapped artifacts, so repeated witness-parser agreement runs do not redo the tokenization stage. - **I5** (learn): Store silver data per source with de-duplication, so train/test leakage and over-represented sources can be traced and controlled during evaluation. *Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.* 📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 18,614 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 17,965 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 17,450 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 17,296 works 🧭 Topics (1): [Semitic languages](https://md.research.elamur.ai/free/en/topics/semitic-languages/) · 3,066 works ## Cite Cite as: Zaid Alyafeai and Maged S. Al-Shaibani. 2020. ARBML: Democratizing Arabic Natural Language Processing Tools. Proceedings of Second Workshop for NLP Open Source Software (NLP-OSS). The paper itself, not this page. BibTeX from the library's record of the work. ```bibtex @inproceedings{alyafeai2020arbml, title = {{ARBML: Democratizing Arabic Natural Language Processing Tools}}, author = {Zaid Alyafeai and Maged S. Al-Shaibani}, booktitle = {Proceedings of Second Workshop for NLP Open Source Software (NLP-OSS)}, year = {2020}, doi = {10.18653/v1/2020.nlposs-1.2}, url = {https://doi.org/10.18653/v1/2020.nlposs-1.2} } ``` - [← previous Arabic Dialect Identification Using BERT Fine-Tuning](https://md.research.elamur.ai/free/en/papers/mansour-2020-arabic-dialect-identification-bert-fine/) - [next → Bert Transformer model for Detecting Arabic GPT2 Auto-Generated Tweets](https://md.research.elamur.ai/free/en/papers/harrag-2020-bert-transformer-model-detecting-arabic/) - [↑ back to papers of 2020](https://md.research.elamur.ai/free/en/papers/2020/page/8/) ## References 1 references, 1 of them with a card of ours. ↗ leads to the work. 1. Conclusion and future plans Martı́n Abadi, Paul Barham, Jianmin Chen, Zhifeng Chen, Andy Davis, Jeffrey Dean, Matthieu Devin, Sanjay Ghemawat, Geoffrey Irving, Michael Isard, et al. 2016. [Tensorflow: A system for large-scale machine learning](https://md.research.elamur.ai/free/en/papers/abadi-2016-tensorflow-system-large-scale-machine/). In 12th {USENIX} symposium on operating systems design and implementation ({OSDI} 16), pages 265–283. [↗](https://doi.org/10.48550/arxiv.1605.08695) *Updated 1 November 2026*