---
title: "Arabic Named Entity Recognition Using Transformer-based-CRF"
description: "The authors build a three-part tagger. 30 references, cited by 7 papers here. Among them: 7."
authors: ["Muhammad Saleh Al-Qurishi", "Riad Souissi"]
year: 2021
venue: "Proceedings of the 4th International Conference on Natural Language and Speech Processing (ICNLSP 2021)"
source: "https://aclanthology.org/2021.icnlsp-1.31/"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/qurishi-2021-arabic-named-entity-recognition-transformer/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/qurishi-2021-arabic-named-entity-recognition-transformer/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2021](https://md.research.elamur.ai/free/en/papers/2021/page/7/) / Arabic Named Entity Recognition Using Transformer-based-CRF Model

# Arabic Named Entity Recognition Using Transformer-based-CRF Model

**Authors:** Muhammad Saleh Al-Qurishi and [Riad Souissi](https://md.research.elamur.ai/free/en/authors/riad-souissi/) · **Year:** 2021 · **Venue:** [Proceedings of the 4th International Conference on Natural Language and Speech Processing (ICNLSP 2021)](https://md.research.elamur.ai/free/en/venues/icnlsp/) ([International Conference on Natural Language and Speech Processing 2021](https://md.research.elamur.ai/free/en/venues/icnlsp/2021/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2021.icnlsp-1.31/) · [PDF](https://aclanthology.org/2021.icnlsp-1.31.pdf)

**Citations:** [7](https://md.research.elamur.ai/free/en/papers/qurishi-2021-arabic-named-entity-recognition-transformer/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
The paper addresses named entity recognition for Arabic, where rich morphology, ambiguous spelling and limited tooling make the task hard. The authors build a three-part tagger. A pretrained transformer encoder (AraBERT, AraELECTRA or XLM-RoBERTa) is fine-tuned. A linear projection then maps its token vectors to tag scores, and a CRF layer picks the best overall label sequence. Instead of using only the encoder's final layer, they also try fusing several hidden layers, either by concatenating them or by adding them up, and report that this helps. Models are trained and tested on ANERCorp and AQMAR, with an extra check on CANERCorpus, a classical-Arabic corpus built from hadith texts. On ANERCorp the best AraBERTv2+CRF variant reaches about 0.896 F1-macro, clearly above the earlier BERT-family baselines listed. On AQMAR it gets roughly 0.88, and on CANERCorpus it beats a previously published system. The work matters because a simple, resource-free design (no gazetteers or knowledge bases) gives strong Arabic NER and is easy to reproduce.

## Key points
- The architecture has three stages: a transformer language model, a fully connected layer, and a CRF trained with CRF loss. Training uses AdamW, learning rate 5e-5, 5 epochs and dropout 0.1, with max sequence lengths of 128, 256 and 512.
- On ANERCorp, the best setup is AraBERTv2+CRF concatenating 6 hidden states at sequence length 256. It scores F1-macro 0.8956 and F1-measure 0.913, against 0.828 F1-macro for AraBERTv1 and 0.804 for AraBERTv2 without a CRF.
- AraELECTRA+CRF (concat 5) and XLM-RoBERTa+CRF (concat 9) score lower than AraBERTv2+CRF on ANERCorp (≈0.872 and ≈0.876 F1-macro).
- On AQMAR, the bert-large-arabertv2+crf model ranges from about 0.85 to 0.88 F1-macro. Concat and sum fusion always beat last-layer-only output at every sequence length.
- On CANERCorpus, the model reaches F1-measure 98.26 vs 94.76 for Alsaaran and Alrabiah (2021). The corpus has about 7,000 hadiths, ~250k words, ~13,000 entities, 20 classes and IOB2 labels, with majority vote among three annotators.

## Interesting ideas
- **Fusing intermediate encoder layers:** Concatenating or summing several hidden states, instead of trusting only the top layer, recovers information that deeper layers may have washed out; this is a cheap gain for any classifier head on a transformer.
- **CRF on top of contextual embeddings:** A CRF scores whole label sequences and models dependencies between neighbouring tags, so impossible transitions (e.g. an I- tag after O) are discouraged.
- **Grid ablation over sequence length × fusion mode × layer count:** Reporting every combination (last / concat / sum at 128, 256 and 512) shows which gains are robust and which are noise.
- **Testing on a stylistically distant, fine-grained corpus:** Checking the model on classical religious texts with 20 specialised classes probes generalisation beyond newswire.
- **Entity-type ambiguity as an explicit problem:** The authors' planned future work targets words that can be both a place and a person, treating this as a distinct issue rather than a tagging error.

## For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

- **I1** (tree): gparse could feed its arc and label scorers a sum or concatenation of several encoder layers, and pick the best combination on a dev treebank.
- **I2** (tokens): the crate en tagger or an entity-span tagger could add a transition layer so BIO-style outputs stay well-formed and the chosen path can be explained by its transition scores.
- **I3** (learn): evaluation could adopt such small factorial grids, with last-layer-only as the built-in control, before accepting an architectural change.
- **I4** (learn): adding an out-of-domain test set, such as older literary prose or fables, would show whether parser or tagger gains carry beyond the training genre.
- **I5** (world): the world model could keep competing entity types for such names and resolve them later from frames and context, logging why each choice was made.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

📚 Shelves (2): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works

🧭 Topics (2): [Semitic languages](https://md.research.elamur.ai/free/en/topics/semitic-languages/) · 2,908 works · [🏷️ Named entity recognition](https://md.research.elamur.ai/free/en/topics/named-entity-recognition/) · 200 works

## Cite

Cite as: Muhammad Saleh Al-Qurishi and Riad Souissi. 2021. Arabic Named Entity Recognition Using Transformer-based-CRF Model. Proceedings of the 4th International Conference on Natural Language and Speech Processing (ICNLSP 2021).

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{alqurishi2021arabic,
  title = {{Arabic Named Entity Recognition Using Transformer-based-CRF Model}},
  author = {Muhammad Saleh Al-Qurishi and Riad Souissi},
  booktitle = {Proceedings of the 4th International Conference on Natural Language and Speech Processing (ICNLSP 2021)},
  year = {2021},
  url = {https://aclanthology.org/2021.icnlsp-1.31/}
}
```

- [← previous A pragmatic account of the weak evidence effect](https://md.research.elamur.ai/free/en/papers/barnett-2021-pragmatic-account-weak-evidence-effect/)
- [next → Country-level Arabic Dialect Identification Using Small Datasets with Integrated Machine Learning Techniques and Deep Learning Models](https://md.research.elamur.ai/free/en/papers/althobaiti-2021-country-level-arabic-dialect-identification/)
- [↑ back to papers of 2021](https://md.research.elamur.ai/free/en/papers/2021/page/7/)

## References

30 references, 12 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Ahmed Abdelali, Kareem Darwish, Nadir Durrani, and Hamdy Mubarak. 2016. [Farasa: A fast and furious segmenter for arabic](https://md.research.elamur.ai/free/en/papers/abdelali-2016-farasa-fast-furious-segmenter-arabic/). In Proceedings of the 2016 conference of the North American chapter of the association for computational linguistics: Demonstrations, pages 11–16.
2. Muhammad Abdul-Mageed, AbdelRahim Elmadany, and El Moatez Billah Nagoudi. 2020. [Arbert & marbert: deep bidirectional transformers for arabic](https://md.research.elamur.ai/free/en/papers/abdul-mageed-2021-arbert-marbert-deep-bidirectional-transformers/). arXiv preprint arXiv:2101.01785. [↗](https://arxiv.org/abs/2101.01785)
3. Mohammad Al-Smadi, Saad Al-Zboon, Yaser Jararweh, and Patrick Juola. 2020. Transfer learning for arabic named entity recognition with deep neural networks. Ieee Access, 8:37736–37745. [↗](https://doi.org/10.1109/access.2020.2973319)
4. Manar Alkhatib and Khaled Shaalan. 2020. Boosting arabic named entity recognition transliteration with deep learning. In The thirty-third international flairs conference. [find](https://scholar.google.com/scholar?q=Manar+Alkhatib+and+Khaled+Shaalan%2E+2020%2E+Boosting+arabic+named+entity+recognition+transliteration+with+deep+learning%2E+In+The+thirty%2Dthird+international+flairs+conference%2E)
5. Norah Alsaaran and Maha Alrabiah. 2021. Classical arabic named entity recognition using variant deep neural network architectures and bert. IEEE Access, 9:91537–91547. [↗](https://doi.org/10.1109/access.2021.3092261)
6. Nasser Alshammari and Saad Alanazi. 2020. The impact of using different annotation schemes on named entity recognition. Egyptian Informatics Journal. [↗](https://doi.org/10.1016/j.eij.2020.10.004)
7. Wissam Antoun, Fady Baly, and Hazem Hajj. 2020. [Arabert: Transformer-based model for arabic language understanding](https://md.research.elamur.ai/free/en/papers/antoun-2020-arabert-transformer-based-model-arabic/). In Proceedings of the 4th Workshop on Open-Source Arabic Corpora and Processing Tools, with a Shared Task on Offensive Language Detection, pages 9–15. [↗](https://doi.org/10.48550/arxiv.2003.00104)
8. Wissam Antoun, Fady Baly, and Hazem Hajj. 2021. [Araelectra: Pre-training text discriminators for arabic language understanding](https://md.research.elamur.ai/free/en/papers/antoun-2021-araelectra-pre-training-text-discriminators/). In Proceedings of the Sixth Arabic Natural Language Processing Workshop, pages 191–195. [↗](https://doi.org/10.48550/arxiv.2012.15516)
9. Husamelddin AM Balla and Sarah Jane Delany. 2020. Exploration of approaches to arabic named entity recognition. In CLEOPATRA@ ESWC, pages 2–16. [find](https://scholar.google.com/scholar?q=Husamelddin+AM+Balla+and+Sarah+Jane+Delany%2E+2020%2E+Exploration+of+approaches+to+arabic+named+entity+recognition%2E+In+CLEOPATRA%40+ESWC%2C+pages+2%E2%80%9316%2E)
10. Yassine Benajiba and Paolo Rosso. 2008. Arabic named entity recognition using conditional random fields. In Proc. of Workshop on HLT & NLP within the Arabic World, LREC, volume 8, pages 143–153. Citeseer. [find](https://scholar.google.com/scholar?q=Yassine+Benajiba+and+Paolo+Rosso%2E+2008%2E+Arabic+named+entity+recognition+using+conditional+random+fields%2E+In+Proc%2E+of+Workshop+on+HLT+%26+NLP+within+the+Arabic+World%2C+LREC%2C+volume+8%2C+pages+143%E2%80%93153%2E+Cites)
11. Yassine Benajiba, Imed Zitouni, Mona Diab, and Paolo Rosso. 2010. [Arabic named entity recognition: using features extracted from noisy data](https://md.research.elamur.ai/free/en/papers/benajiba-2010-arabic-named-entity-recognition-features/). In Proceedings of the ACL 2010 conference short papers, pages 281– 285.
12. Brahim Ait Benali, Soukaina Mihi, Ismail El Bazi, and Nabil Laachfoubi. 2021. New approach for arabic named entity recognition on social media based on feature selection using genetic algorithm. International Journal of Electrical and Computer Engineering, 11(2):1485. [find](https://scholar.google.com/scholar?q=Brahim+Ait+Benali%2C+Soukaina+Mihi%2C+Ismail+El+Bazi%2C+and+Nabil+Laachfoubi%2E+2021%2E+New+approach+for+arabic+named+entity+recognition+on+social+media+based+on+feature+selection+using+genetic+algorithm%2E+Inter)
13. Alexis Conneau, Kartikay Khandelwal, Naman Goyal, Vishrav Chaudhary, Guillaume Wenzek, Francisco Guzmán, Edouard Grave, Myle Ott, Luke Zettlemoyer, and Veselin Stoyanov. 2019. Unsupervised Model last concat sum last concat sum Last concat Sum \#states 1 10 2 1 2 3 1 10 2 Seq 128 128 128 256 256 256 512 512 512 F1-Macro 0.869587 0.865022 0.859811 0.862752 0.875771 0.861978 0.872761 0.862209 0.8559 Accuracy 0.978834 0.97758 0.978417 0.978032 0.978344 0.978552 0.979593 0.977719 0.977719 Precision 0.875871 0.86192 0.853556 0.864902 0.87883 0.864902 0.862116 0.86908 0.866295 Recall 0.86620689 0.8583333 0.8547486 0.8758815 0.87638888 0.8601108 0.8792613 0.85714285 0.859116 F1-measure 0.871012482 0.860125261 0.854152128 0.870357393 0.877607788 0.8625 0.870604781 0.863070539 0.862690707 *Table 6: The performance of the proposed Arabic NER model using araelectra-base-discriminator +crf test results on AQMAR dataset* cross-lingual representation learning at scale. arXiv preprint arXiv:1911.02116. [↗](https://arxiv.org/abs/1911.02116)
14. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. [Bert: Pre-training of deep bidirectional transformers for language understanding](https://md.research.elamur.ai/free/en/papers/devlin-2019-bert-pre-training-deep-bidirectional/). arXiv preprint arXiv:1810.04805. [↗](https://arxiv.org/abs/1810.04805)
15. Marwa Elgamal, Mohamed Abou-Kreisha, Reda Abo Elezz, and Salwa Hamada. 2020. An ontology-based name entity recognition ner and nlp systems in arabic storytelling. Al-Azhar Bulletin of Science, 31(2-B):31–38. [↗](https://doi.org/10.21608/absb.2020.44367.1088)
16. Archana Goyal, Vishal Gupta, and Manish Kumar. 2018. Recent named entity recognition and classification techniques: a systematic review. Computer Science Review, 29:21–43. [↗](https://doi.org/10.1016/j.cosrev.2018.06.001)
17. Mourad Gridach. 2016. [Character-aware neural networks for arabic named entity recognition for social media](https://md.research.elamur.ai/free/en/papers/gridach-2016-character-aware-neural-networks-arabic/). In Proceedings of the 6th workshop on South and Southeast Asian natural language processing (WSSANLP2016), pages 23–32.
18. Chadi Helwe and Shady Elbassuoni. 2019. Arabic named entity recognition via deep co-learning. Artificial Intelligence Review, 52(1):197–215. [↗](https://doi.org/10.1007/s10462-019-09688-6)
19. Mohammad Hudhud, Hamed Abdelhaq, and Fadi Mohsen. 2021. Arabianer: A system to extract named entities from arabic content. In ICAART (1), pages 489–497. [find](https://scholar.google.com/scholar?q=Mohammad+Hudhud%2C+Hamed+Abdelhaq%2C+and+Fadi+Mohsen%2E+2021%2E+Arabianer%3A+A+system+to+extract+named+entities+from+arabic+content%2E+In+ICAART+%281%29%2C+pages+489%E2%80%93497%2E)
20. Hussein Khalil, Taha Osman, and Mohammed Miltan. 2020. Extracting arabic composite names using genitive principles of arabic grammar. ACM Transactions on Asian and Low-Resource Language Information Processing (TALLIP), 19(4):1–16. [↗](https://doi.org/10.1145/3382187)
21. John Lafferty, Andrew McCallum, and Fernando CN Pereira. 2001. Conditional random fields: Probabilistic models for segmenting and labeling sequence data. [find](https://scholar.google.com/scholar?q=John+Lafferty%2C+Andrew+McCallum%2C+and+Fernando+CN+Pereira%2E+2001%2E+Conditional+random+fields%3A+Probabilistic+models+for+segmenting+and+labeling+sequence+data%2E)
22. Hugo Larochelle. 2021. lectures on conditional random fields. [find](https://scholar.google.com/scholar?q=Hugo+Larochelle%2E+2021%2E+lectures+on+conditional+random+fields%2E)
23. Jing Li, Aixin Sun, Jianglei Han, and Chenliang Li. 2020. A survey on deep learning for named entity recognition. IEEE Transactions on Knowledge and Data Engineering. [↗](https://doi.org/10.1109/tkde.2020.2981314)
24. Marwa Muhammad, Muhammad Rohaim, Alaa Hamouda, and Salah Abdel-Mageid. 2020. A comparison between conditional random field and structured support vector machine for arabic named entity recognition. [↗](https://doi.org/10.3844/jcssp.2020.117.125)
25. Ossama Obeid, Nasser Zalmout, Salam Khalifa, Dima Taji, Mai Oudah, Bashar Alhafni, Go Inoue, Fadhl Eryani, Alexander Erdmann, and Nizar Habash. 2020. [Camel tools: An open source python toolkit for arabic natural language processing](https://md.research.elamur.ai/free/en/papers/obeid-2020-camel-tools-open-source-python/). In Proceedings of the 12th language resources and evaluation conference, pages 7022–7032.
26. Mai Oudah and Khaled Shaalan. 2012. [A pipeline arabic named entity recognition using a hybrid approach](https://md.research.elamur.ai/free/en/papers/oudah-2012-pipeline-arabic-named-entity-recognition/). In Proceedings of COLING 2012, pages 2159–2176.
27. Arfath Pasha, Mohamed Al-Badrashiny, Mona T Diab, Ahmed El Kholy, Ramy Eskander, Nizar Habash, Manoj Pooleery, Owen Rambow, and Ryan Roth. 2014. [Madamira: A fast, comprehensive tool for morphological analysis and disambiguation of arabic](https://md.research.elamur.ai/free/en/papers/pasha-2014-madamira-fast-comprehensive-tool-morphological/). In Lrec, volume 14, pages 1094–1101. Citeseer.
28. Ramzi Esmail Salah and Lailatul Qadri Binti Zakaria. 2018. Building the classical arabic named entity recognition corpus (canercorpus). In 2018 Fourth International Conference on Information Retrieval and Knowledge Management (CAMP), pages 1–8. IEEE. [find](https://scholar.google.com/scholar?q=Ramzi+Esmail+Salah+and+Lailatul+Qadri+Binti+Zakaria%2E+2018%2E+Building+the+classical+arabic+named+entity+recognition+corpus+%28canercorpus%29%2E+In+2018+Fourth+International+Conference+on+Information+Retrieval)
29. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. [Attention is all you need](https://md.research.elamur.ai/free/en/papers/vaswani-2017-attention-all-need/). In Advances in neural information processing systems, pages 5998–6008.
30. Ayah Zirikly and Mona Diab. 2015. [Named entity recognition for arabic social media](https://md.research.elamur.ai/free/en/papers/zirikly-2015-named-entity-recognition-arabic-social/). In Proceedings of the 1st workshop on vector space modeling for natural language processing, pages 176–185.

*Updated 1 November 2026*