---
title: "Recognizing Named Entities in Tweets — Liu 2011"
description: "The paper tackles named entity recognition on Twitter, where messages are short, noisy and ungrammatical. 45.8% F1. 28 references, cited by 71 papers here."
authors: ["Xiaohua Liu", "Shaodian Zhang", "Furu Wei", "Ming Zhou"]
year: 2011
venue: "Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies"
source: "https://aclanthology.org/P11-1037/"
licence: "CC BY-NC-SA 3.0 (ACL Anthology, before 2016)"
licence_url: "https://creativecommons.org/licenses/by-nc-sa/3.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/liu-2011-recognizing-named-entities-tweets/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/liu-2011-recognizing-named-entities-tweets/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2011](https://md.research.elamur.ai/free/en/papers/2011/) / Recognizing Named Entities in Tweets

# Recognizing Named Entities in Tweets

**Authors:** [Xiaohua Liu](https://md.research.elamur.ai/free/en/authors/xiaohua-liu/), Shaodian Zhang, [Furu Wei](https://md.research.elamur.ai/free/en/authors/furu-wei/) and [Ming Zhou](https://md.research.elamur.ai/free/en/authors/ming-zhou/) · **Year:** 2011 · **Venue:** Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies · **Licence:** [CC BY-NC-SA 3.0 (ACL Anthology, before 2016)](https://creativecommons.org/licenses/by-nc-sa/3.0/)

**Links:** [Publisher's page](https://aclanthology.org/P11-1037/) · [PDF](https://aclanthology.org/P11-1037.pdf) · [OpenAlex](https://openalex.org/W2119759918)

**Citations:** 364 citations (OpenAlex, 2026-10-14) · [71](https://md.research.elamur.ai/free/en/papers/liu-2011-recognizing-named-entities-tweets/cited-by/) in our library, of the 140,884 papers whose bibliographies we read

## Summary
The paper tackles named entity recognition on Twitter, where messages are short, noisy and ungrammatical. Systems built for news text break down there: Stanford NER, trained on CoNLL03, falls from 90.8% to 45.8% F1 on tweets, and annotated tweet data is scarce. The authors combine two models. First, a K-Nearest Neighbors classifier assigns a provisional label to each word, using similar tweets that were labeled recently. Then a linear CRF performs sequence labeling with the BILOU scheme, using those provisional labels as extra features alongside orthographic, lexical and gazetteer features (30 gazetteers). Both models are retrained in a bootstrapping loop: tweets labeled with high confidence are added to the training pool as new data arrives. For evaluation, 12,245 tweets were manually annotated for four types: person, organization, product and location. The full system reaches 80.2% F1 and beats the baselines on every type. Ablations show that both the KNN stage and self-training help. The work matters because it shows a practical route to NER on noisy social-media text when little labeled data exists.

## Key points
- Off-the-shelf tools degrade sharply on tweets. Stanford NER drops from 90.8% to 45.8% F1, and the OpenNLP POS tagger reaches only 74.0% accuracy on the test set.
- The annotated set has 12,245 tweets:
  - 46.8% contain at least one entity.
  - The four target types make up 89.0% of all entities.
- Contribution of each component (overall F1):
  - Full system NER_CB: 80.2%.
  - Without KNN: 78.5%.
  - Without semi-supervised learning: 76.7%.
  - CRF beat alternative labelers by 2.1% F1.
- Feature ablation for the CRF: orthographic features alone give 53.5% F1, and adding lexical plus gazetteer features raises it to 78.5%. Gazetteers give the largest gain.
- The errors fall into three groups:
  - slang and abbreviations: 35.5%;
  - data sparseness: 37.2%;
  - extreme noise: 27.3%.

## Interesting ideas
- **Two-stage stacking of a global and a local model.** A cheap instance-based classifier looks across many similar texts, and its output becomes a feature for a sequence model that sees only one short text. This could help any domain with very short inputs, such as search queries, product titles or SMS.
- **Use only the confident part of the first stage.** The KNN's high-confidence predictions cover about 22% of words but are more accurate than the CRF baseline. Selective, confidence-filtered evidence can be more valuable than full coverage.
- **Streaming self-training with human seeds.** Retraining on confidently labeled items as data arrives lets the system adapt to new entities over time. The authors evaluate this by feeding the test set in 10 sequential chunks and tracking F1 per chunk, a simple way to measure gains from online learning.
- **Prefer a fast-to-retrain component.** Alternative classifiers performed about the same as KNN, but KNN retrains much faster. That is a sensible design priority when a model must be updated repeatedly.
- **Gazetteers as cross-domain knowledge.** Generic name lists compensate strongly for missing training data. Enriching them continuously with newly found entities is a cheap lever for low-resource NER, including in languages like Ukrainian.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

Fields: [Natural language processing](https://md.research.elamur.ai/free/en/shelves/nlp/)

## Institutions

- [Microsoft Research Asia](https://md.research.elamur.ai/free/en/companies/microsoft-research-asia/) — Company, Beijing, China · 41 works · 57,263 citations
- [Harbin Institute of Technology](https://md.research.elamur.ai/free/en/universities/harbin-institute-of-technology/) — University, Harbin, China · 58 works · 25,250 citations
- [Shanghai Jiao Tong University](https://md.research.elamur.ai/free/en/universities/shanghai-jiao-tong-university/) — University, Shanghai, China · 43 works · 12,961 citations

## Cite

Cite as: Xiaohua Liu, Shaodian Zhang, Furu Wei and Ming Zhou. 2011. Recognizing Named Entities in Tweets. Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{liu2011recognizing,
  title = {{Recognizing Named Entities in Tweets}},
  author = {Xiaohua Liu, Shaodian Zhang, Furu Wei and Ming Zhou},
  booktitle = {Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies},
  year = {2011},
  url = {https://aclanthology.org/P11-1037/}
}
```

- [← previous Multi-Source Transfer of Delexicalized Dependency Parsers](https://md.research.elamur.ai/free/en/papers/mcdonald-2011-multi-source-transfer-delexicalized-dependency/)
- [next → CoNLL-2011 Shared Task: Modeling Unrestricted Coreference in OntoNotes](https://md.research.elamur.ai/free/en/papers/pradhan-2011-conll-2011-shared-task-modeling/)
- [↑ back to papers of 2011](https://md.research.elamur.ai/free/en/papers/2011/)

## References

28 references, 10 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Peter F. Brown, Peter V. deSouza, Robert L. Mercer, Vincent J. Della Pietra, and Jenifer C. Lai. 1992. Class-based n-gram models of natural language. Comput. Linguist., 18:467–479. [find](https://scholar.google.com/scholar?q=Peter+F%2E+Brown%2C+Peter+V%2E+deSouza%2C+Robert+L%2E+Mercer%2C+Vincent+J%2E+Della+Pietra%2C+and+Jenifer+C%2E+Lai%2E+1992%2E+Class%2Dbased+n%2Dgram+models+of+natural+language%2E+Comput%2E+Linguist%2E%2C+18%3A467%E2%80%93479%2E)
2. Laura Chiticariu, Rajasekar Krishnamurthy, Yunyao Li, Frederick Reiss, and Shivakumar Vaithyanathan. 2010. Domain adaptation of rule-based annotators for named-entity recognition tasks. In EMNLP, pages 1002–1012. [find](https://scholar.google.com/scholar?q=Laura+Chiticariu%2C+Rajasekar+Krishnamurthy%2C+Yunyao+Li%2C+Frederick+Reiss%2C+and+Shivakumar+Vaithyanathan%2E+2010%2E+Domain+adaptation+of+rule%2Dbased+annotators+for+named%2Dentity+recognition+tasks%2E+In+EMNLP%2C+page)
3. Michael Collins. 2002. Discriminative training methods for hidden markov models: theory and experiments with perceptron algorithms. In EMNLP, pages 1–8. [our card](https://md.research.elamur.ai/free/en/papers/collins-2002-discriminative-training-methods-hidden-markov/)
4. Doug Downey, Matthew Broadhead, and Oren Etzioni. 2007. Locating Complex Named Entities in Web Text. In IJCAI. [find](https://scholar.google.com/scholar?q=Doug+Downey%2C+Matthew+Broadhead%2C+and+Oren+Etzioni%2E+2007%2E+Locating+Complex+Named+Entities+in+Web+Text%2E+In+IJCAI%2E)
5. Oren Etzioni, Michael Cafarella, Doug Downey, Ana-Maria Popescu, Tal Shaked, Stephen Soderland, Daniel S. Weld, and Alexander Yates. 2005. Unsupervised named-entity extraction from the web: an experimental study. Artif. Intell., 165(1):91–134. [find](https://scholar.google.com/scholar?q=Oren+Etzioni%2C+Michael+Cafarella%2C+Doug+Downey%2C+Ana%2DMaria+Popescu%2C+Tal+Shaked%2C+Stephen+Soderland%2C+Daniel+S%2E+Weld%2C+and+Alexander+Yates%2E+2005%2E+Unsupervised+named%2Dentity+extraction+from+the+web%3A+an+experim)
6. Tim Finin, Will Murnane, Anand Karandikar, Nicholas Keller, Justin Martineau, and Mark Dredze. 2010. Annotating named entities in twitter data with crowdsourcing. In CSLDAMT, pages 80–88. [find](https://scholar.google.com/scholar?q=Tim+Finin%2C+Will+Murnane%2C+Anand+Karandikar%2C+Nicholas+Keller%2C+Justin+Martineau%2C+and+Mark+Dredze%2E+2010%2E+Annotating+named+entities+in+twitter+data+with+crowdsourcing%2E+In+CSLDAMT%2C+pages+80%E2%80%9388%2E)
7. Jenny Rose Finkel and Christopher D. Manning. 2009. Nested named entity recognition. In EMNLP, pages 141–150. [our card](https://md.research.elamur.ai/free/en/papers/finkel-2009-nested-named-entity-recognition/)
8. Jenny Rose Finkel, Trond Grenager, and Christopher Manning. 2005. Incorporating non-local information into information extraction systems by gibbs sampling. In ACL, pages 363–370. [our card](https://md.research.elamur.ai/free/en/papers/finkel-2005-incorporating-non-local-information-into/)
9. Honglei Guo, Huijia Zhu, Zhili Guo, Xiaoxun Zhang, Xian Wu, and Zhong Su. 2009. Domain adaptation with latent semantic association for named entity recognition. In NAACL, pages 281–289. [find](https://scholar.google.com/scholar?q=Honglei+Guo%2C+Huijia+Zhu%2C+Zhili+Guo%2C+Xiaoxun+Zhang%2C+Xian+Wu%2C+and+Zhong+Su%2E+2009%2E+Domain+adaptation+with+latent+semantic+association+for+named+entity+recognition%2E+In+NAACL%2C+pages+281%E2%80%93289%2E)
10. Martin Jansche and Steven P. Abney. 2002. Information extraction from voicemail transcripts. In EMNLP, pages 320–327. [find](https://scholar.google.com/scholar?q=Martin+Jansche+and+Steven+P%2E+Abney%2E+2002%2E+Information+extraction+from+voicemail+transcripts%2E+In+EMNLP%2C+pages+320%E2%80%93327%2E)
11. Jing Jiang and ChengXiang Zhai. 2007. Instance weighting for domain adaptation in nlp. In ACL, pages 264– 271. [our card](https://md.research.elamur.ai/free/en/papers/jiang-2007-instance-weighting-domain-adaptation-nlp/)
12. Dan Klein and Christopher D. Manning. 2003. Accurate unlexicalized parsing. In ACL, pages 423–430. [our card](https://md.research.elamur.ai/free/en/papers/klein-2003-accurate-unlexicalized-parsing/)
13. Vijay Krishnan and Christopher D. Manning. 2006. An effective two-stage model for exploiting non-local dependencies in named entity recognition. In ACL, pages 1121–1128. [find](https://scholar.google.com/scholar?q=Vijay+Krishnan+and+Christopher+D%2E+Manning%2E+2006%2E+An+effective+two%2Dstage+model+for+exploiting+non%2Dlocal+dependencies+in+named+entity+recognition%2E+In+ACL%2C+pages+1121%E2%80%931128%2E)
14. George R. Krupka and Kevin Hausman. 1998. Isoquest: Description of the netowl T M extractor system as used in muc-7. In MUC-7. [find](https://scholar.google.com/scholar?q=George+R%2E+Krupka+and+Kevin+Hausman%2E+1998%2E+Isoquest%3A+Description+of+the+netowl+T+M+extractor+system+as+used+in+muc%2D7%2E+In+MUC%2D7%2E)
15. John D. Lafferty, Andrew McCallum, and Fernando C. N. Pereira. 2001. Conditional random fields: Probabilistic models for segmenting and labeling sequence data. In ICML, pages 282–289. [find](https://scholar.google.com/scholar?q=John+D%2E+Lafferty%2C+Andrew+McCallum%2C+and+Fernando+C%2E+N%2E+Pereira%2E+2001%2E+Conditional+random+fields%3A+Probabilistic+models+for+segmenting+and+labeling+sequence+data%2E+In+ICML%2C+pages+282%E2%80%93289%2E)
16. Andrew Mccallum and Wei Li. 2003. Early results for named entity recognition with conditional random fields, feature induction and web-enhanced lexicons. In HLT-NAACL, pages 188–191. [our card](https://md.research.elamur.ai/free/en/papers/mccallum-2003-early-results-named-entity-recognition/)
17. Scott Miller, Jethran Guinness, and Alex Zamanian. 2004. Name tagging with word clusters and discriminative training. In HLT-NAACL, pages 337–342. [our card](https://md.research.elamur.ai/free/en/papers/miller-2004-name-tagging-word-clusters-discriminative/)
18. Einat Minkov, Richard C. Wang, and William W. Cohen. 2005. Extracting personal names from email: applying named entity recognition to informal text. In HLT, pages 443–450. [find](https://scholar.google.com/scholar?q=Einat+Minkov%2C+Richard+C%2E+Wang%2C+and+William+W%2E+Cohen%2E+2005%2E+Extracting+personal+names+from+email%3A+applying+named+entity+recognition+to+informal+text%2E+In+HLT%2C+pages+443%E2%80%93450%2E)
19. David Nadeau and Satoshi Sekine. 2007. A survey of named entity recognition and classification. Linguisticae Investigationes, 30:3–26. [find](https://scholar.google.com/scholar?q=David+Nadeau+and+Satoshi+Sekine%2E+2007%2E+A+survey+of+named+entity+recognition+and+classification%2E+Linguisticae+Investigationes%2C+30%3A3%E2%80%9326%2E)
20. Lev Ratinov and Dan Roth. 2009. Design challenges and misconceptions in named entity recognition. In CoNLL, pages 147–155. [our card](https://md.research.elamur.ai/free/en/papers/ratinov-2009-design-challenges-misconceptions-named-entity/)
21. Sameer Singh, Dustin Hillard, and Chris Leggetter. 2010. Minimally-supervised extraction of entities from text advertisements. In HLT-NAACL, pages 73–81. [find](https://scholar.google.com/scholar?q=Sameer+Singh%2C+Dustin+Hillard%2C+and+Chris+Leggetter%2E+2010%2E+Minimally%2Dsupervised+extraction+of+entities+from+text+advertisements%2E+In+HLT%2DNAACL%2C+pages+73%E2%80%9381%2E)
22. Jun Suzuki and Hideki Isozaki. 2008. Semi-supervised sequential labeling and segmentation using giga-word scale unlabeled data. In ACL, pages 665–673. [find](https://scholar.google.com/scholar?q=Jun+Suzuki+and+Hideki+Isozaki%2E+2008%2E+Semi%2Dsupervised+sequential+labeling+and+segmentation+using+giga%2Dword+scale+unlabeled+data%2E+In+ACL%2C+pages+665%E2%80%93673%2E)
23. Erik F. Tjong Kim Sang and Fien De Meulder. 2003. Introduction to the CoNLL-2003 shared task: language-independent named entity recognition. In HLT-NAACL, pages 142–147. [our card](https://md.research.elamur.ai/free/en/papers/sang-2003-introduction-conll-2003-shared-task/)
24. Yefeng Wang. 2009. Annotating and recognising named entities in clinical notes. In ACL-IJCNLP, pages 18– 26. [find](https://scholar.google.com/scholar?q=Yefeng+Wang%2E+2009%2E+Annotating+and+recognising+named+entities+in+clinical+notes%2E+In+ACL%2DIJCNLP%2C+pages+18%E2%80%93+26%2E)
25. Dan Wu, Wee Sun Lee, Nan Ye, and Hai Leong Chieu. 2009. Domain adaptive bootstrapping for named entity recognition. In EMNLP, pages 1523–1532. [find](https://scholar.google.com/scholar?q=Dan+Wu%2C+Wee+Sun+Lee%2C+Nan+Ye%2C+and+Hai+Leong+Chieu%2E+2009%2E+Domain+adaptive+bootstrapping+for+named+entity+recognition%2E+In+EMNLP%2C+pages+1523%E2%80%931532%2E)
26. Kazuhiro Yoshida and Jun’ichi Tsujii. 2007. Reranking for biomedical named-entity recognition. In BioNLP, pages 209–216. [find](https://scholar.google.com/scholar?q=Kazuhiro+Yoshida+and+Jun%E2%80%99ichi+Tsujii%2E+2007%2E+Reranking+for+biomedical+named%2Dentity+recognition%2E+In+BioNLP%2C+pages+209%E2%80%93216%2E)
27. Tong Zhang and David Johnson. 2003. A robust risk minimization based named entity recognition system. In HLT-NAACL, pages 204–207. [find](https://scholar.google.com/scholar?q=Tong+Zhang+and+David+Johnson%2E+2003%2E+A+robust+risk+minimization+based+named+entity+recognition+system%2E+In+HLT%2DNAACL%2C+pages+204%E2%80%93207%2E)
28. GuoDong Zhou and Jian Su. 2002. Named entity recognition using an hmm-based chunk tagger. In ACL, pages 473–480. [our card](https://md.research.elamur.ai/free/en/papers/zhou-2002-named-entity-recognition-hmm-based/)