---
title: "Simplified Feature Set for Arabic Named Entity Recognition (2010)"
description: "The paper tackles named entity recognition (persons, locations, organizations) for Arabic. 16 references, cited by 22 papers here."
authors: ["Ahmed Abdul-Hamid", "Kareem Darwish"]
year: 2010
venue: "Proceedings of the 2010 Named Entities Workshop"
source: "https://aclanthology.org/W10-2417/"
licence: "CC BY-NC-SA 3.0 (ACL Anthology, before 2016)"
licence_url: "https://creativecommons.org/licenses/by-nc-sa/3.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/hamid-2010-simplified-feature-set-arabic-named/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/hamid-2010-simplified-feature-set-arabic-named/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2010](https://md.research.elamur.ai/free/en/papers/2010/) / Simplified Feature Set for Arabic Named Entity Recognition

# Simplified Feature Set for Arabic Named Entity Recognition

**Authors:** Ahmed Abdul-Hamid and [Kareem Darwish](https://md.research.elamur.ai/free/en/authors/kareem-darwish/) · **Year:** 2010 · **Venue:** Proceedings of the 2010 Named Entities Workshop · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY-NC-SA 3.0 (ACL Anthology, before 2016)](https://creativecommons.org/licenses/by-nc-sa/3.0/)

**Links:** [Publisher's page](https://aclanthology.org/W10-2417/) · [PDF](https://aclanthology.org/W10-2417.pdf) · [OpenAlex](https://openalex.org/W1814730318)

**Citations:** 76 citations (OpenAlex, 2026-10-14) · [22](https://md.research.elamur.ai/free/en/papers/hamid-2010-simplified-feature-set-arabic-named/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
The paper tackles named entity recognition (persons, locations, organizations) for Arabic. Arabic attaches clitics such as conjunctions, prepositions and the determiner directly to words, has no capitalization, and often uses ordinary words as names. Earlier systems relied on POS tagging, chunking, tokenization and gazetteers. The authors ask whether cheap surface features can do as well. They train a CRF sequence labeler whose main inputs are character bigrams, trigrams and 4-grams taken from the start and end of each word. They add word-level statistics: unigram and bigram probabilities, probability ratios to neighbouring words, t-test and mutual information, per-class character trigram language models, word length and relative position. The only Arabic-specific step is stripping two coordinating conjunctions. Experiments use the Benajiba newswire corpus and ACE 2005 (newswire and broadcast news). Compared with Benajiba and Rosso (2008) on the same data and CRF implementation, the method loses 2 F-points on locations but gains 8 on organizations and 9 on persons. This shows that affix-position n-grams can stand in for morphological and syntactic analysis.

## Key points
- Data:
  - The Benajiba corpus has over 150,000 words, with 878 LOC, 342 ORG and 689 PER entities.
  - ACE 2005 provides 98,530 words (66,590 newswire, 31,940 broadcast news), with 867 LOC, 269 ORG and 524 PER entities.
  - ACE weblogs were dropped because they contain colloquial Arabic.
  - 13 of the 40 ACE sub-entity types were excluded, such as nominal, pronominal and vehicle.
- Best run per dataset:
  - On Benajiba, the best run is 3s_6bi_6tri_6quad_1gP: stems, leading/trailing bi-, tri- and 4-grams, and unigram probability.
  - On ACE, the best run is 3w_6bi_6tri_6quad_WL: surface words plus the same affix n-grams and word length.
- Stems versus words:
  - Using words alone on Benajiba, stemming lifted recall for LOC by 7 points, for ORG by 13, and for PER by 14.
  - Once the affix n-grams were added, that gap shrank to about 1 recall point.
- Feature value:
  - The extra statistical features (1gP, 2gP, WL, WP, 3gCLM) added at most about 1 F-point, mostly through recall.
  - t-test and mutual information gave no benefit and sometimes hurt.
- Comparison with prior work:
  - Benajiba and Rosso (2008) reported 90/66/73 F on this corpus using POS tags, base-phrase chunking, tokenization and nationality adjectives; this paper scores 2 lower on LOC but 8 and 9 higher on ORG and PER.
  - The authors use macro-averages, while that work used micro-averages.
  - The authors attribute the weaker location score to having no gazetteer.

## Interesting ideas
- Instead of running a morphological analyzer, feed the model the first three and last three character n-grams of each word, with boundary markers; this lets the learner decide which clitics and affixes matter, which also avoids errors from a stemmer or tagger.
- A sharp drop in unigram probability compared with the preceding and following word is used as evidence that a token is a name, since names tend to be rare or out-of-vocabulary.
- Separate character trigram language models are trained for each entity class and for non-entities, and their scores become features; this works because transliterated foreign names have distinctive letter patterns.
- The ablation checks whether a preprocessing step (stemming) still pays off once richer surface features are present, and finds its benefit nearly disappears.
- Before comparing with published numbers, the authors harmonize the data and spell out what limits the comparison. They collapse subtypes, remove non-specific and anaphoric mentions, skip the ACE comparison because of tagging differences, and flag macro-average versus micro-average.

## For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

- **I1** (tokens): positional affix n-grams could help the tagger and lemma guesser with unknown English words, catching cues like -ness, -ed, un- or name-like endings without hand-written suffix rules.
- **I2** (world): when building the entity inventory of a tale, such probability dips could flag candidate proper names (an invented character's name) that no lexicon or Wikidata entry covers.
- **I3** (tokens): class-specific character models could estimate whether an unseen capitalized token looks like a person, place or organization, giving the tagger an explainable score.
- **I4** (learn): ablations should routinely re-test each pipeline stage against stronger feature sets, so that components which no longer add value are dropped.
- **I5** (learn): evaluation reports could carry an explicit comparability note listing label conversions, excluded categories and averaging method, so that scores stay honest across treebanks or test sets.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Microsoft](https://md.research.elamur.ai/free/en/companies/microsoft/) — Company, Redmond, United States · 325 works · 142,931 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works

🧭 Topics (2): [Semitic languages](https://md.research.elamur.ai/free/en/topics/semitic-languages/) · 2,908 works · [🏷️ Named entity recognition](https://md.research.elamur.ai/free/en/topics/named-entity-recognition/) · 200 works

## Cite

Cite as: Ahmed Abdul-Hamid and Kareem Darwish. 2010. Simplified Feature Set for Arabic Named Entity Recognition. Proceedings of the 2010 Named Entities Workshop.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{abdulhamid2010simplified,
  title = {{Simplified Feature Set for Arabic Named Entity Recognition}},
  author = {Ahmed Abdul-Hamid and Kareem Darwish},
  booktitle = {Proceedings of the 2010 Named Entities Workshop},
  year = {2010},
  url = {https://aclanthology.org/W10-2417/}
}
```

- [← previous Arabic Dialect Handling in Hybrid Machine Translation](https://md.research.elamur.ai/free/en/papers/sawaf-2010-arabic-dialect-handling-hybrid-machine/)
- [next → The Leeds Arabic Discourse Treebank: Annotating Discourse Connectives for Arabic](https://md.research.elamur.ai/free/en/papers/saif-2010-leeds-arabic-discourse-treebank-annotating/)
- [↑ back to papers of 2010](https://md.research.elamur.ai/free/en/papers/2010/)

## References

16 references, 4 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Y. Benajiba, M. Diab, and P. Rosso. 2008. [Arabic Named Entity Recognition using Optimized Feature Sets](https://md.research.elamur.ai/free/en/papers/benajiba-2008-arabic-named-entity-recognition-optimized/). Proceedings of the 2008 Conference on Empirical Methods in Natural Language Processing, pages 284–293, Honolulu, October 2008.
2. Y. Benajiba and P. Rosso. 2008. Arabic Named Entity Recognition using Conditional Random Fields. In Proc. of Workshop on HLT & NLP within the Arabic World, LREC’08. [find](https://scholar.google.com/scholar?q=Y%2E+Benajiba+and+P%2E+Rosso%2E+2008%2E+Arabic+Named+Entity+Recognition+using+Conditional+Random+Fields%2E+In+Proc%2E+of+Workshop+on+HLT+%26+NLP+within+the+Arabic+World%2C+LREC%E2%80%9908%2E)
3. Y. Benajiba, P. Rosso and J. M. Benedí. 2007. AN-ERsys: An Arabic Named Entity Recognition system based on Maximum Entropy. In Proc. of CI-CLing-2007, Springer-Verlag, LNCS(4394), pp. 143-153. [find](https://scholar.google.com/scholar?q=Y%2E+Benajiba%2C+P%2E+Rosso+and+J%2E+M%2E+Bened%C3%AD%2E+2007%2E+AN%2DERsys%3A+An+Arabic+Named+Entity+Recognition+system+based+on+Maximum+Entropy%2E+In+Proc%2E+of+CI%2DCLing%2D2007%2C+Springer%2DVerlag%2C+LNCS%284394%29%2C+pp%2E+143%2D153%2E)
4. Y. Benajiba and P. Rosso. 2007. ANERsys 2.0: Conquering the NER task for the Arabic language by combining the Maximum Entropy with POS-tag information. In Proc. of Workshop on Natural Language-Independent Engineering, IICAI-2007. [find](https://scholar.google.com/scholar?q=Y%2E+Benajiba+and+P%2E+Rosso%2E+2007%2E+ANERsys+2%2E0%3A+Conquering+the+NER+task+for+the+Arabic+language+by+combining+the+Maximum+Entropy+with+POS%2Dtag+information%2E+In+Proc%2E+of+Workshop+on+Natural+Language%2DIndepen)
5. M. Diab. 2009. Second Generation Tools (AMIRA 2.0): Fast and Robust Tokenization, POS tagging, and Base Phrase Chunking. Proceedings of the Second International Conference on Arabic Language Resources and Tools, 2009. [find](https://scholar.google.com/scholar?q=M%2E+Diab%2E+2009%2E+Second+Generation+Tools+%28AMIRA+2%2E0%29%3A+Fast+and+Robust+Tokenization%2C+POS+tagging%2C+and+Base+Phrase+Chunking%2E+Proceedings+of+the+Second+International+Conference+on+Arabic+Language+Resources)
6. B. Farber, D. Freitag, N. Habash, and O. Rambow. 2008. [Improving NER in Arabic Using a Morphological Tagger](https://md.research.elamur.ai/free/en/papers/farber-2008-improving-ner-arabic-morphological-tagger/). In Proc. of LREC’08.
7. F. Huang. 2005. Multilingual Named Entity Extraction and Translation from Text and Speech. Ph.D. Thesis. Pittsburgh: Carnegie Mellon University. [↗](https://doi.org/10.5445/ir/1000166653)
8. J. Lafferty, A. McCallum, and F. Pereira. 2001. Conditional random fields: Probabilistic models for segmenting and labeling sequence data, In Proc. of ICML, pp.282-289, 2001. [find](https://scholar.google.com/scholar?q=J%2E+Lafferty%2C+A%2E+McCallum%2C+and+F%2E+Pereira%2E+2001%2E+Conditional+random+fields%3A+Probabilistic+models+for+segmenting+and+labeling+sequence+data%2C+In+Proc%2E+of+ICML%2C+pp%2E282%2D289%2C+2001%2E)
9. Young-Suk Lee, Kishore Papineni, Salim Roukos, Ossama Emam, Hany Hassan. 2003. [Language Model Based Arabic Word Segmentation](https://md.research.elamur.ai/free/en/papers/lee-2003-language-model-based-arabic-word/). ACL 2003: 399-406
10. C. Manning and H. Schutze. 1999. Foundations of Statistical Natural Language Processing. Cambridge, Massachusetts: The MIT Press. [find](https://scholar.google.com/scholar?q=C%2E+Manning+and+H%2E+Schutze%2E+1999%2E+Foundations+of+Statistical+Natural+Language+Processing%2E+Cambridge%2C+Massachusetts%3A+The+MIT+Press%2E)
11. J. Mayfield, P. McNamee, and C. Piatko. 2003. Named Entity Recognition using Hundreds of Thousands of Features. HLT-NAACL 2003-Volume 4, 2003. [find](https://scholar.google.com/scholar?q=J%2E+Mayfield%2C+P%2E+McNamee%2C+and+C%2E+Piatko%2E+2003%2E+Named+Entity+Recognition+using+Hundreds+of+Thousands+of+Features%2E+HLT%2DNAACL+2003%2DVolume+4%2C+2003%2E)
12. A. McCallum and W. Li. 2003. Early Results for Named Entity Recognition with Conditional Random Fields, Features Induction and Web-Enhanced Lexicons. In Proc. Conference on Computational Natural Language Learning. [find](https://scholar.google.com/scholar?q=A%2E+McCallum+and+W%2E+Li%2E+2003%2E+Early+Results+for+Named+Entity+Recognition+with+Conditional+Random+Fields%2C+Features+Induction+and+Web%2DEnhanced+Lexicons%2E+In+Proc%2E+Conference+on+Computational+Natural+Langu)
13. P. McNamee and J. Mayfield. 2002. Entity extraction without language-specific. Proceedings of CoNLL, 2002. [find](https://scholar.google.com/scholar?q=P%2E+McNamee+and+J%2E+Mayfield%2E+2002%2E+Entity+extraction+without+language%2Dspecific%2E+Proceedings+of+CoNLL%2C+2002%2E)
14. D. Nadeau and S. Sekine. 2009. A survey of named entity recognition and classification. Named entities: recognition, classification and use, ed. S. Sekine and E. Ranchhod, John Benjamins Publishing Company. [↗](https://doi.org/10.1075/bct.19.03nad)
15. F. Sha and F. Pereira. 2003. Shallow parsing with conditional random fields, In Proc. of HLT/NAACL-2003. [find](https://scholar.google.com/scholar?q=F%2E+Sha+and+F%2E+Pereira%2E+2003%2E+Shallow+parsing+with+conditional+random+fields%2C+In+Proc%2E+of+HLT%2FNAACL%2D2003%2E)
16. K. Shaalan and H. Raza. 2007. [Person Name Entity Recognition for Arabic](https://md.research.elamur.ai/free/en/papers/shaalan-2007-person-name-entity-recognition-arabic/). Proceedings of the 5th Workshop on Important Unresolved Matters, pages 17–24, Prague, Czech Republic, June 2007.

*Updated 1 November 2026*