---
title: "NERDz: A Preliminary Dataset of Named Entities for Algerian"
description: "This paper presents NERDz, an initial NER dataset built on top of an extended NArabizi treebank. 19 references, cited by 3 papers here."
authors: ["Samia Touileb"]
year: 2022
venue: "Proceedings of the 2nd Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 12th International Joint Conference on Natural Language Processing (Volume 2: Short Papers)"
source: "https://aclanthology.org/2022.aacl-short.13/"
doi: "10.18653/v1/2022.aacl-short.13"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/touileb-2022-nerdz-preliminary-dataset-named-entities/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/touileb-2022-nerdz-preliminary-dataset-named-entities/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2022](https://md.research.elamur.ai/free/en/papers/2022/page/6/) / NERDz: A Preliminary Dataset of Named Entities for Algerian

# NERDz: A Preliminary Dataset of Named Entities for Algerian

**Authors:** [Samia Touileb](https://md.research.elamur.ai/free/en/authors/samia-touileb/) · **Year:** 2022 · **Venue:** [Proceedings of the 2nd Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 12th International Joint Conference on Natural Language Processing (Volume 2: Short Papers)](https://md.research.elamur.ai/free/en/venues/aacl/) ([AACL 2022](https://md.research.elamur.ai/free/en/venues/aacl/2022/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2022.aacl-short.13/) · [PDF](https://aclanthology.org/2022.aacl-short.13.pdf) · [DOI 10.18653/v1/2022.aacl-short.13](https://doi.org/10.18653/v1/2022.aacl-short.13) · [OpenAlex](https://openalex.org/W4404783711)

**Citations:** 7 citations (OpenAlex, 2026-10-14) · [3](https://md.research.elamur.ai/free/en/papers/touileb-2022-nerdz-preliminary-dataset-named-entities/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
Algerian Arabic is a non-standardized spoken variety. It mixes heavily with French and is written in both Latin script (NArabizi) and Arabic script, yet it had no public resource for named entity recognition. This paper presents NERDz, an initial NER dataset built on top of an extended NArabizi treebank. In that treebank, every sentence already exists in three parallel forms: the original NArabizi, a manual Alg-Arabic transliteration, and a code-switched version. Two native-speaker annotators labelled all sentences in BRAT with eight entity types in IOB2 format, following guidelines adapted from ACE, CoNLL and OntoNotes. Agreement between them was high, and most disagreements involved entity spans, which were often traceable to inconsistent tokenization in the treebank. The author reports baselines from two systems: an NCRF++ tagger (char-CNN, BiLSTM, CRF) and a fine-tuned DziriBERT. NCRF++ performed better on every script. Scores were highest on the code-switched version and lowest on Latin-script NArabizi. Because the entities are parallel across scripts, the dataset also lets researchers study how writing system affects NER for a dialect with few resources.

## Key points
- **Size and splits:**
  - 1,276 sentences and 19,258 tokens in total; the train/dev/test splits are inherited from the NArabizi treebank (997/136/143 sentences).
  - 1,566 entities: 1,229 in train, 180 in dev, 157 in test.
- **Entity types:** the annotation scheme defines eight categories in total: PER, ORG, GPE, LOC, NORP, EVT, PROD and MISC. PER, GPE, ORG and NORP together make up more than 90% of all entities.
- **Inter-annotator agreement:**
  - Krippendorff's α = 0.87.
  - Strict micro F1 = 86.3, using the SemEval 2013 task 9 scheme and ignoring tokens both annotators marked as O.
- **Test results (strict F1):**

  | Script | NCRF++ | DziriBERT |
  |---|---|---|
  | NArabizi | 65.89 | 56.56 |
  | Alg-Arabic | 72.4 | 68.91 |
  | Code-switched | 77.46 | 61.63 |

  Keeping only the four most frequent types raised NCRF++ scores by about 1–3 points.
- **Out-of-vocabulary rates:** for the FastText Algerian embeddings, OOV was 95.95% on NArabizi, compared with 22.02% on Alg-Arabic and 33.36% on code-switched text. This helps explain why the Latin-script version scored worst.

## Interesting ideas
- Adding entity labels as an extra layer on an existing treebank that has parallel script versions allows the same content to be compared across writing forms, which separates the effect of orthography from the effect of the task itself.
- The author chose not to repair inconsistent tokenization because doing so would break the existing dependency trees, and instead documented how annotators dealt with it. Preserving the integrity of one layer while recording its known flaws is a practical policy for multi-layer resources.
- Agreement was measured in two ways, a chance-corrected α and a strict span-plus-type F1 that ignores shared O labels, so the score is not inflated by easy negatives.
- For nested entities, a simple deterministic rule (always keep the longest span) resolved recurring disagreements without requiring a nested annotation scheme.
- Two diagnostics separated data scarcity from model weakness: retraining on only the frequent types, and reporting embedding OOV rates per script.

## For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

- **I1** (world): Entity spans could be anchored to UD token IDs, so that spelling or normalization variants of one text map to the same world entities and can be checked for consistency.
- **I2** (tree): During treebank or dialect conversion, known tokenization quirks (such as clitic articles) could be logged as flagged exceptions rather than silently re-tokenized, which keeps the trees valid and the problems traceable.
- **I3** (learn): When several witness parsers or taggers vote on silver data, agreement could be scored only on the contested non-default labels, using a strict exact-match criterion.
- **I4** (world): A fixed longest-span tie-break could be applied when entity mentions overlap, such as an event versus its parent event, with the inner span kept as a linked sub-entity.
- **I5** (learn): Evaluation could routinely include frequent-class-only reruns and coverage and OOV statistics, so that a drop in score is attributed to sparsity or vocabulary rather than to the method.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [University of Bergen](https://md.research.elamur.ai/free/en/universities/university-of-bergen/) — University, Bergen, Norway · 9 works · 583 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works

🧭 Topics (1): [Semitic languages](https://md.research.elamur.ai/free/en/topics/semitic-languages/) · 2,908 works

## Cite

Cite as: Samia Touileb. 2022. NERDz: A Preliminary Dataset of Named Entities for Algerian. Proceedings of the 2nd Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 12th International Joint Conference on Natural Language Processing (Volume 2: Short Papers).

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{touileb2022nerdz,
  title = {{NERDz: A Preliminary Dataset of Named Entities for Algerian}},
  author = {Samia Touileb},
  booktitle = {Proceedings of the 2nd Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 12th International Joint Conference on Natural Language Processing (Volume 2: Short Papers)},
  year = {2022},
  doi = {10.18653/v1/2022.aacl-short.13},
  url = {https://doi.org/10.18653/v1/2022.aacl-short.13}
}
```

- [← previous How Much Does Lookahead Matter for Disambiguation? Partial Arabic Diacritization Case Study](https://md.research.elamur.ai/free/en/papers/esmail-2022-how-much-does-lookahead-matter/)
- [next → Plumeria at SemEval-2022 Task 6: Sarcasm Detection for English and Arabic Using Transformers and Data Augmentation](https://md.research.elamur.ai/free/en/papers/shaheen-2022-plumeria-semeval-2022-task-6/)
- [↑ back to papers of 2022](https://md.research.elamur.ai/free/en/papers/2022/page/6/)

## References

19 references, 8 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Conclusion and Future works Amine Abdaoui, Mohamed Berrimi, Mourad Oussalah, and Abdelouahab Moussaoui. 2021. [Dziribert: a pre-trained language model for the algerian dialect](https://md.research.elamur.ai/free/en/papers/abdaoui-2021-dziribert-pre-trained-language-model/).
2. We present our annotations to expand the NArabizi treebank (Seddah et al., 2020) with named en-Wafia Adouane, Jean-Philippe Bernardy, and Simon Dobnik. 2019. Normalising non-standardised orthography in Algerian code-switched user-generated data. In Proceedings of the 5th Workshop on Noisy User-generated Text (W-NUT 2019), pages 131–140, Hong Kong, China. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=We+present+our+annotations+to+expand+the+NArabizi+treebank+%28Seddah+et+al%2E%2C+2020%29+with+named+en%2DWafia+Adouane%2C+Jean%2DPhilippe+Bernardy%2C+and+Simon+Dobnik%2E+2019%2E+Normalising+non%2Dstandardised+orthography+i)
3. Wafia Adouane, Samia Touileb, and Jean-Philippe Bernardy. 2020. Identifying sentiments in Algerian code-switched user-generated comments. In Proceedings of the Twelfth Language Resources and Evaluation Conference, pages 2698–2705, Marseille, France. European Language Resources Association. [find](https://scholar.google.com/scholar?q=Wafia+Adouane%2C+Samia+Touileb%2C+and+Jean%2DPhilippe+Bernardy%2E+2020%2E+Identifying+sentiments+in+Algerian+code%2Dswitched+user%2Dgenerated+comments%2E+In+Proceedings+of+the+Twelfth+Language+Resources+and+Evaluatio)
4. Rami Al-Rfou, Vivek Kulkarni, Bryan Perozzi, and Steven Skiena. 2015. Polyglot-ner: Massive multilingual named en-tity recognition. In Proceedings of the 2015 SIAM International Conference on Data Mining, pages 586–594. SIAM. [find](https://scholar.google.com/scholar?q=Rami+Al%2DRfou%2C+Vivek+Kulkarni%2C+Bryan+Perozzi%2C+and+Steven+Skiena%2E+2015%2E+Polyglot%2Dner%3A+Massive+multilingual+named+en%2Dtity+recognition%2E+In+Proceedings+of+the+2015+SIAM+International+Conference+on+Data+Min)
5. Jason P.C. Chiu and Eric Nichols. 2016. [Named entity recognition with bidirectional LSTM-CNNs](https://md.research.elamur.ai/free/en/papers/chiu-2015-named-entity-recognition-bidirectional-lstm/). Transactions of the Association for Computational Linguistics, 4:357–370. [↗](https://doi.org/10.1162/tacl_a_00104)
6. Ryan Cotterell, Adithya Renduchintala, Naomi Saphra, and Chris Callison-Burch. 2014. An algerian arabic-french code-switched corpus. In Workshop on free/open-source Arabic corpora and corpora processing tools workshop programme, page 34. [find](https://scholar.google.com/scholar?q=Ryan+Cotterell%2C+Adithya+Renduchintala%2C+Naomi+Saphra%2C+and+Chris+Callison%2DBurch%2E+2014%2E+An+algerian+arabic%2Dfrench+code%2Dswitched+corpus%2E+In+Workshop+on+free%2Fopen%2Dsource+Arabic+corpora+and+corpora+processi)
7. Fredrik Jørgensen, Tobias Aasmoe, Anne-Stine Ruud Husevåg, Lilja Øvrelid, and Erik Velldal. 2020. NorNE: Annotating named entities for Norwegian. In Proceedings of the 12th Language Resources and Evaluation Conference, pages 4547–4556, Marseille, France. European Language Resources Association. [↗](https://doi.org/10.48550/arxiv.1911.12146)
8. Guillaume Lample, Miguel Ballesteros, Sandeep Subramanian, Kazuya Kawakami, and Chris Dyer. 2016. [Neural architectures for named entity recognition](https://md.research.elamur.ai/free/en/papers/lample-2016-neural-architectures-named-entity-recognition/). In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 260–270, San Diego, California. Association for Computational Linguistics. [↗](https://doi.org/10.48550/arxiv.1603.01360)
9. A. Mitchell, S. Strassel, M. Przybocki, J. Davis, G. Doddington, A. Brunstein A. Ferro L. Grishman, R. a Meyers, and B. Sundheim. 2003. Ace-2 version 1.0. In Web Download. LDC Catalog No. LDC2003T11. [find](https://scholar.google.com/scholar?q=A%2E+Mitchell%2C+S%2E+Strassel%2C+M%2E+Przybocki%2C+J%2E+Davis%2C+G%2E+Doddington%2C+A%2E+Brunstein+A%2E+Ferro+L%2E+Grishman%2C+R%2E+a+Meyers%2C+and+B%2E+Sundheim%2E+2003%2E+Ace%2D2+version+1%2E0%2E+In+Web+Download%2E+LDC+Catalog+No%2E+LDC2003T11%2E)
10. Djamé Seddah, Farah Essaidi, Amal Fethi, Matthieu Futeral, Benjamin Muller, Pedro Javier Ortiz Suárez, Benoît Sagot, and Abhishek Srivastava. 2020. Building a user-generated content North-African Arabizi treebank: Tackling hell. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 1139–1150, Online. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Djam%C3%A9+Seddah%2C+Farah+Essaidi%2C+Amal+Fethi%2C+Matthieu+Futeral%2C+Benjamin+Muller%2C+Pedro+Javier+Ortiz+Su%C3%A1rez%2C+Beno%C3%AEt+Sagot%2C+and+Abhishek+Srivastava%2E+2020%2E+Building+a+user%2Dgenerated+content+North%2DAfrican+Arab)
11. (DDIExtraction 2013). In Second Joint Conference on Lexical and Computational Semantics (*SEM), Volume 2: Proceedings of the Seventh International Workshop on Semantic Evaluation (SemEval 2013), pages 341–350, Atlanta, Georgia, USA. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=%28DDIExtraction+2013%29%2E+In+Second+Joint+Conference+on+Lexical+and+Computational+Semantics+%28%2ASEM%29%2C+Volume+2%3A+Proceedings+of+the+Seventh+International+Workshop+on+Semantic+Evaluation+%28SemEval+2013%29%2C+pages)
12. Pontus Stenetorp, Sampo Pyysalo, Goran Topić, Tomoko Ohta, Sophia Ananiadou, and Jun’ichi Tsujii. 2012. [brat: a web-based tool for NLP-assisted text annotation](https://md.research.elamur.ai/free/en/papers/stenetorp-2012-brat-web-based-tool-nlp/). In Proceedings of the Demonstrations at the 13th Conference of the European Chapter of the Association for Computational Linguistics, pages 102–107, Avignon, France. Association for Computational Linguistics.
13. Erik F. Tjong Kim Sang and Fien De Meulder. 2003. [Introduction to the CoNLL-2003 shared task: Language-independent named entity recognition](https://md.research.elamur.ai/free/en/papers/sang-2003-introduction-conll-2003-shared-task/). In Proceedings of the Seventh Conference on Natural Language Learning at HLT-NAACL 2003, pages 142– 147. [↗](https://doi.org/10.48550/arxiv.cs/0306050)
14. Samia Touileb and Jeremy Barnes. 2021. The interplay between morphological typology and script on a novel multi-layer algerian dialect corpus. arXiv, arXiv:2105.07400. [↗](https://arxiv.org/abs/2105.07400)
15. Reut Tsarfaty, Djamé Seddah, Yoav Goldberg, Sandra Kübler, Yannick Versley, Marie Candito, Jennifer Foster, Ines Rehbein, and Lamia Tounsi. 2010. [Statistical parsing of morphologically rich languages (spmrl) what, how and whither](https://md.research.elamur.ai/free/en/papers/tsarfaty-2010-statistical-parsing-morphologically-rich-languages/). In Proceedings of the NAACL HLT 2010 First Workshop on Statistical Parsing of Morphologically-Rich Languages, pages 1–12.
16. Ralph Weischedel, Martha Palmer, Mitchell Marcus, Eduard Hovy, Sameer Pradhan, Lance Ramshaw, Nianwen Xue, Ann Taylor, Jeff Kaufman, Michelle Franchini, et al. 2013. Ontonotes release 5.0 ldc2013t19. Linguistic Data Consortium, Philadelphia, PA, 23. [find](https://scholar.google.com/scholar?q=Ralph+Weischedel%2C+Martha+Palmer%2C+Mitchell+Marcus%2C+Eduard+Hovy%2C+Sameer+Pradhan%2C+Lance+Ramshaw%2C+Nianwen+Xue%2C+Ann+Taylor%2C+Jeff+Kaufman%2C+Michelle+Franchini%2C+et+al%2E+2013%2E+Ontonotes+release+5%2E0+ldc2013t19%2E+)
17. Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pier-ric Cistac, Tim Rault, Remi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Mariama Drame, Quentin Lhoest, and Alexander Rush. 2020. [Transformers: State-of-the-art natural language processing](https://md.research.elamur.ai/free/en/papers/wolf-2020-transformers-state-art-natural-language/). In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pages 38–45, Online. Association for Computational Linguistics. [↗](https://doi.org/10.18653/v1/2020.emnlp-demos.6)
18. Jie Yang and Yue Zhang. 2018. [Ncrf++: An open-source neural sequence labeling toolkit](https://md.research.elamur.ai/free/en/papers/yang-2018-ncrf-open-source-neural-sequence/). In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics.
19. Isabel Segura-Bedmar, Paloma Martínez, and María Herrero-Zazo. 2013. SemEval-2013 task 9 : Extraction of drug-drug interactions from biomedical texts [find](https://scholar.google.com/scholar?q=Isabel+Segura%2DBedmar%2C+Paloma+Mart%C3%ADnez%2C+and+Mar%C3%ADa+Herrero%2DZazo%2E+2013%2E+SemEval%2D2013+task+9+%3A+Extraction+of+drug%2Ddrug+interactions+from+biomedical+texts)

*Updated 1 November 2026*