---
title: "Simple Effective Microblog Named Entity Recognition: Arabic as"
description: "The authors build a CRF baseline with simple character-affix and word-window features. 35.3 points. 22 references, cited by 10 papers here."
authors: ["Kareem Darwish", "Wei Gao"]
year: 2014
venue: "Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC'14)"
source: "https://aclanthology.org/L14-1192/"
licence: "CC BY-NC 4.0 (ELRA, LREC proceedings)"
licence_url: "https://creativecommons.org/licenses/by-nc/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/darwish-2014-simple-effective-microblog-named-entity/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/darwish-2014-simple-effective-microblog-named-entity/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2014](https://md.research.elamur.ai/free/en/papers/2014/page/3/) / Simple Effective Microblog Named Entity Recognition: Arabic as an Example

# Simple Effective Microblog Named Entity Recognition: Arabic as an Example

**Authors:** [Kareem Darwish](https://md.research.elamur.ai/free/en/authors/kareem-darwish/) and [Wei Gao](https://md.research.elamur.ai/free/en/authors/wei-gao/) · **Year:** 2014 · **Venue:** [Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC'14)](https://md.research.elamur.ai/free/en/venues/lrec/) ([LREC 2014](https://md.research.elamur.ai/free/en/venues/lrec/2014/)) · **Publisher:** [European Language Resources Association (ELRA)](https://md.research.elamur.ai/free/en/publishers/european-language-resources-association-elra/) · **Licence:** [CC BY-NC 4.0 (ELRA, LREC proceedings)](https://creativecommons.org/licenses/by-nc/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/L14-1192/) · [PDF](https://aclanthology.org/L14-1192.pdf) · [OpenAlex](https://openalex.org/W2251715652)

**Citations:** 26 citations (OpenAlex, 2026-10-14) · [10](https://md.research.elamur.ai/free/en/papers/darwish-2014-simple-effective-microblog-named-entity/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
The paper addresses named entity recognition (NER) on Arabic tweets. Newswire-trained systems perform poorly on tweets because the language is informal and dialectal, spelling varies, entity names are often shortened, and Arabic has no capitalization cue. The authors build a CRF baseline with simple character-affix and word-window features. They then test three language-independent improvements: a large gazetteer mined from Wikipedia, domain adaptation that mixes news and tweet training data, and a semi-supervised two-pass scheme. In the two-pass scheme, entities found automatically in a large unlabeled tweet collection become a confidence-weighted gazetteer feature. They also annotate a new Arabic tweet NER dataset, with training and test tweets taken from different time periods. Combining all three techniques raises overall F-measure to 65.2. That is 35.3 points above the news-trained baseline and 19.9 points above the tweet-trained baseline. The study shows which cheap techniques help most for noisy social-media text when linguistic tools are weak.

## Key points
- The baseline uses CRF++ with the leading and trailing 1–4 letters of each word plus the raw current, previous and next words. It is trained either on ANERCORP (about 150k news tokens) or on 3,646 newly annotated tweets from May 2012.
- The test set has 1,423 tweets (about 26k tokens) from November 2011. Only 25% of its entities appear in the tweet training data. Overall F rises from 29.9 with news training to 45.3 with tweet training, so in-domain data had the largest effect of any single factor.
- The Wikipedia gazetteer holds 70,908 locations, 26,391 organizations and 81,880 persons. Titles were typed via category trigger words, and redirects were added. It covers 60% of test entities and lifts F to 37.2 with news training and 55.0 with tweet training. It mainly boosts precision.
- Domain adaptation replicates the tweet set alongside ANERCORP. A factor of 7 was chosen on an 80/20 split, testing values from 1 to 9. This reaches F 54.6 alone and 59.5 with the gazetteer, and it mainly boosts recall.
- The two-pass method tags 65M unlabeled tweets. The resulting automatic gazetteer covers 74% of test entities. It adds 3.2 F over the tweet baseline and 5.3 F over the baseline with the Wikipedia gazetteer, so the method works better when the first-pass system is stronger.

## Interesting ideas
- Run a tagger over a huge unlabeled corpus, drop items found only once, and feed the survivors back as a lexicon feature weighted by how often each was given each type. This turns noisy self-labels into a soft prior rather than hard training labels.
- Continuous confidence values are rounded to one significant figure and fused into the label string (e.g. a B-ORG tag with a .8 suffix), so a learner that only handles symbolic features can still use graded evidence.
- Domain adaptation is done simply by repeating the small in-domain set several times next to a large out-of-domain set, with the repeat count tuned on held-out data.
- The test tweets come from a different time window than the training tweets, which checks generalization to new entities rather than memorization of recurring ones.
- Wikipedia titles are typed as person, location or organization by checking whether their categories contain trigger words such as "cities" or "births", which gives a large typed lexicon with almost no manual work.

## For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

- **I1** (learn): Silver entity or lemma lists harvested from Mova's own runs could become weighted features for the next training round, instead of being trusted outright.
- **I2** (tokens): The perceptron tagger could take binned lexicon-confidence features the same way, and each feature would stay readable when explaining a decision.
- **I3** (tree): When mixing a small target-dialect treebank with large UD treebanks for the parser, a tuned replication factor is a cheap and transparent way to weight the domains.
- **I4** (learn): Evaluation splits for Mova could be separated by source or period, giving a harder control that exposes overfitting to frequent names.
- **I5** (world): Similar category-keyword rules could cheaply assign coarse ontology types to Wikidata or Wikipedia entities in Mova's knowledge layer.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Qatar Foundation](https://md.research.elamur.ai/free/en/ngos/qatar-foundation/) — Nonprofit, Doha, Qatar · 17 works · 2,111 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works

🧭 Topics (2): [Semitic languages](https://md.research.elamur.ai/free/en/topics/semitic-languages/) · 2,908 works · [🏷️ Named entity recognition](https://md.research.elamur.ai/free/en/topics/named-entity-recognition/) · 200 works

## Cite

Cite as: Kareem Darwish and Wei Gao. 2014. Simple Effective Microblog Named Entity Recognition: Arabic as an Example. Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC'14).

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{darwish2014simple,
  title = {{Simple Effective Microblog Named Entity Recognition: Arabic as an Example}},
  author = {Kareem Darwish and Wei Gao},
  booktitle = {Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC'14)},
  year = {2014},
  doi = {10.63317/5imzhbt74xh4},
  url = {https://doi.org/10.63317/5imzhbt74xh4}
}
```

- [← previous Semantic Query Expansion for Arabic Information Retrieval](https://md.research.elamur.ai/free/en/papers/mahgoub-2014-semantic-query-expansion-arabic-information/)
- [next → ANCOR_Centre, a large free spoken French coreference corpus: description of the resource and reliability measures](https://md.research.elamur.ai/free/en/papers/muzerelle-2014-ancor-centre-large-free-spoken/)
- [↑ back to papers of 2014](https://md.research.elamur.ai/free/en/papers/2014/page/3/)

## References

22 references, 10 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. A. Abdul-Hamid and K. Darwish. 2010. [Simplified Feature Set for Arabic Named Entity Recognition](https://md.research.elamur.ai/free/en/papers/hamid-2010-simplified-feature-set-arabic-named/). In 2010 NEWS Workshop, ACL-2010, pages 110–115.
2. Y. Benajiba, M. Diab, and P. Rosso. 2008. [Arabic Named Entity Recognition using Optimized Feature Sets](https://md.research.elamur.ai/free/en/papers/benajiba-2008-arabic-named-entity-recognition-optimized/). EMNLP-2008, pages 284–293.
3. Y. Benajiba and P. Rosso. 2008. Arabic Named Entity Recognition using Conditional Random Fields. Workshop on HLT & NLP within the Arabic World, LREC-2008. [find](https://scholar.google.com/scholar?q=Y%2E+Benajiba+and+P%2E+Rosso%2E+2008%2E+Arabic+Named+Entity+Recognition+using+Conditional+Random+Fields%2E+Workshop+on+HLT+%26+NLP+within+the+Arabic+World%2C+LREC%2D2008%2E)
4. Y. Benajiba, P. Rosso and J. M. Benedi Ruiz. 2007. ANERsys: An Arabic Named Entity Recognition system based on Maximum Entropy. CICLing-2007, Springer-Verlag, LNCS(4394), pages 143–153 [find](https://scholar.google.com/scholar?q=Y%2E+Benajiba%2C+P%2E+Rosso+and+J%2E+M%2E+Benedi+Ruiz%2E+2007%2E+ANERsys%3A+An+Arabic+Named+Entity+Recognition+system+based+on+Maximum+Entropy%2E+CICLing%2D2007%2C+Springer%2DVerlag%2C+LNCS%284394%29%2C+pages+143%E2%80%93153)
5. Y. Benajiba and P. Rosso. 2007. ANERsys 2.0: Conquering the NER task for the Arabic language by combining the Maximum Entropy with POS-tag information. Workshop on Natural Language-Independent Engineering, IICAI 2007. [find](https://scholar.google.com/scholar?q=Y%2E+Benajiba+and+P%2E+Rosso%2E+2007%2E+ANERsys+2%2E0%3A+Conquering+the+NER+task+for+the+Arabic+language+by+combining+the+Maximum+Entropy+with+POS%2Dtag+information%2E+Workshop+on+Natural+Language%2DIndependent+Enginee)
6. Kareem Darwish, Walid Magdy, Ahmed Mourad. 2012. Language Processing for Arabic Microblog Retrieval. CIKM-2012, pages 2427–2430 [find](https://scholar.google.com/scholar?q=Kareem+Darwish%2C+Walid+Magdy%2C+Ahmed+Mourad%2E+2012%2E+Language+Processing+for+Arabic+Microblog+Retrieval%2E+CIKM%2D2012%2C+pages+2427%E2%80%932430)
7. Kareem Darwish. 2013. [Named Entity Recognition using Cross-lingual Resources: Arabic as an Example](https://md.research.elamur.ai/free/en/papers/darwish-2013-named-entity-recognition-cross-lingual/). ACL-2013.
8. Hal Daumé III. 2007. [Frustratingly Easy Domain Adaptation](https://md.research.elamur.ai/free/en/papers/iii-2007-frustratingly-easy-domain-adaptation/). ACL-2007, pages 256–263.
9. B. Farber, D. Freitag, N. Habash, and O. Rambow. 2008. [Improving NER in Arabic Using a Morphological Tagger](https://md.research.elamur.ai/free/en/papers/farber-2008-improving-ner-arabic-morphological-tagger/). LREC-2008.
10. Mena Habib, Maurice van Keulen. 2012. Unsupervised Improvement of Named Entity Extraction in Short Informal Context Using Disambiguation Clues. EKAW Workshop on Semantic Web and Information Extraction, pp. 1–10. [find](https://scholar.google.com/scholar?q=Mena+Habib%2C+Maurice+van+Keulen%2E+2012%2E+Unsupervised+Improvement+of+Named+Entity+Extraction+in+Short+Informal+Context+Using+Disambiguation+Clues%2E+EKAW+Workshop+on+Semantic+Web+and+Information+Extraction)
11. F. Huang. 2005. Multilingual Named Entity Extraction and Translation from Text and Speech. Ph.D. Thesis. Carnegie Mellon University, Pittsburgh, USA. [↗](https://doi.org/10.5445/ir/1000166653)
12. Jason J. Jung. 2012. Online Named Entity Recognition Method for Microtexts in Social Networking Services: A Case Study on Twitter. Journal of Expert Systems with Applications, 39(9):8066–8070. [find](https://scholar.google.com/scholar?q=Jason+J%2E+Jung%2E+2012%2E+Online+Named+Entity+Recognition+Method+for+Microtexts+in+Social+Networking+Services%3A+A+Case+Study+on+Twitter%2E+Journal+of+Expert+Systems+with+Applications%2C+39%289%29%3A8066%E2%80%938070%2E)
13. J. Lafferty, A. McCallum, and F. Pereira. 2001. Conditional random fields: Probabilistic models for segmenting and labeling sequence data. ICML-2001, pp.282–289. [find](https://scholar.google.com/scholar?q=J%2E+Lafferty%2C+A%2E+McCallum%2C+and+F%2E+Pereira%2E+2001%2E+Conditional+random+fields%3A+Probabilistic+models+for+segmenting+and+labeling+sequence+data%2E+ICML%2D2001%2C+pp%2E282%E2%80%93289%2E)
14. Chenliang Li, Jianshu Weng, Qi He, Yuxia Yao, Anwitaman Datta, Aixin Sun and Bu-Sung Lee. 2012. TwiNER: Named Entity Recognition in Targeted Twitter Stream. SIGIR-2012, pages 721–730. [find](https://scholar.google.com/scholar?q=Chenliang+Li%2C+Jianshu+Weng%2C+Qi+He%2C+Yuxia+Yao%2C+Anwitaman+Datta%2C+Aixin+Sun+and+Bu%2DSung+Lee%2E+2012%2E+TwiNER%3A+Named+Entity+Recognition+in+Targeted+Twitter+Stream%2E+SIGIR%2D2012%2C+pages+721%E2%80%93730%2E)
15. Wenhui Liao, Sriharsha Veeramachaneni. 2009. A simple semi-supervised algorithm for named entity recognition. In NAACL-HLT 2009 Workshop on Semi-Supervised Learning for Natural Language Processing, pages 58–65. [find](https://scholar.google.com/scholar?q=Wenhui+Liao%2C+Sriharsha+Veeramachaneni%2E+2009%2E+A+simple+semi%2Dsupervised+algorithm+for+named+entity+recognition%2E+In+NAACL%2DHLT+2009+Workshop+on+Semi%2DSupervised+Learning+for+Natural+Language+Processing%2C+pa)
16. Xiaohua Liu, Shaodian Zhang, Furu Wei, Ming Zhou. 2011. [Recognizing Named Entities in Tweets](https://md.research.elamur.ai/free/en/papers/liu-2011-recognizing-named-entities-tweets/). ACL-2011, pages 359–367.
17. A. McCallum and W. Li. 2003. Early Results for Named Entity Recognition with Conditional Random Fields, Features Induction and Web-Enhanced Lexicons. CoNLL-2003, pages 188-191. [find](https://scholar.google.com/scholar?q=A%2E+McCallum+and+W%2E+Li%2E+2003%2E+Early+Results+for+Named+Entity+Recognition+with+Conditional+Random+Fields%2C+Features+Induction+and+Web%2DEnhanced+Lexicons%2E+CoNLL%2D2003%2C+pages+188%2D191%2E)
18. Behrang Mohit, Nathan Schneider, Rishav Bhowmick, Kemal Oflazer, Noah A. Smith. 2012. [Recall-oriented learning of named entities in Arabic Wikipedia](https://md.research.elamur.ai/free/en/papers/mohit-2012-recall-oriented-learning-named-entities/). EACL-2012, pages 162–173.
19. D. Nadeau and S. Sekine. 2009. A Survey of Named Entity Recognition and Classification. S. Sekine and E. Ranchhod (ed.), Named Entities: Recognition, Classification and Use, John Benjamins Publishing Co. [↗](https://doi.org/10.1075/bct.19.03nad)
20. L. Ratinov and D. Roth. 2009. [Design Challenges and Misconceptions in Named Entity Recognition](https://md.research.elamur.ai/free/en/papers/ratinov-2009-design-challenges-misconceptions-named-entity/). CoNLL-2009, pages 147–155.
21. Alan Ritter, Sam Clark, Oren Etzioni. 2011. [Named entity recognition in tweets: an experimental study](https://md.research.elamur.ai/free/en/papers/ritter-2011-named-entity-recognition-tweets-experimental/). In EMNLP-2011, pages 1524–1534.
22. K. Shaalan and H. Raza. 2007. [Person Name Entity Recognition for Arabic](https://md.research.elamur.ai/free/en/papers/shaalan-2007-person-name-entity-recognition-arabic/). In 2007 Workshop on Computational Approaches to Semitic Languages: Common Issues and Resources (Semitic 2007), pages 17–24.

*Updated 1 November 2026*