--- title: "Part-of-Speech Tagging for Twitter — Gimpel 2011" description: "Taggers trained on newswire, such as those built from the Penn Treebank WSJ corpus, perform badly on tweets." authors: ["Kevin Gimpel", "Nathan Schneider", "Brendan O'Connor", "Dipanjan Das", "Daniel Mills", "Jacob Eisenstein", "Michael Heilman", "Dani Yogatama", "Jeffrey Flanigan", "Noah A. Smith"] year: 2011 venue: "Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies" source: "https://aclanthology.org/P11-2008/" licence: "CC BY-NC-SA 3.0 (ACL Anthology, before 2016)" licence_url: "https://creativecommons.org/licenses/by-nc-sa/3.0/" summary_licence: "Elamur Cosmic License" canonical: "https://research.elamur.ai/free/en/papers/gimpel-2011-part-speech-tagging-twitter-annotation/" --- [Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/gimpel-2011-part-speech-tagging-twitter-annotation/) [home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2011](https://md.research.elamur.ai/free/en/papers/2011/) / Part-of-Speech Tagging for Twitter: Annotation, Features, and Experiments # Part-of-Speech Tagging for Twitter: Annotation, Features, and Experiments **Authors:** [Kevin Gimpel](https://md.research.elamur.ai/free/en/authors/kevin-gimpel/), [Nathan Schneider](https://md.research.elamur.ai/free/en/authors/nathan-schneider/), [Brendan O'Connor](https://md.research.elamur.ai/free/en/authors/brendan-o-connor/), [Dipanjan Das](https://md.research.elamur.ai/free/en/authors/dipanjan-das/), Daniel Mills, [Jacob Eisenstein](https://md.research.elamur.ai/free/en/authors/jacob-eisenstein/), [Michael Heilman](https://md.research.elamur.ai/free/en/authors/michael-heilman/), [Dani Yogatama](https://md.research.elamur.ai/free/en/authors/dani-yogatama/), [Jeffrey Flanigan](https://md.research.elamur.ai/free/en/authors/jeffrey-flanigan/), [Noah A. Smith](https://md.research.elamur.ai/free/en/authors/noah-a-smith/) · **Year:** 2011 · **Venue:** Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies · **Licence:** [CC BY-NC-SA 3.0 (ACL Anthology, before 2016)](https://creativecommons.org/licenses/by-nc-sa/3.0/) **Links:** [Publisher's page](https://aclanthology.org/P11-2008/) · [PDF](https://aclanthology.org/P11-2008.pdf) · [OpenAlex](https://openalex.org/W4233787372) **Citations:** 491 citations (openalex-snapshot · 2026-10-14) · [276](#cited-by) in our library, of the 140884 papers whose bibliographies we read ## Summary Taggers trained on newswire, such as those built from the Penn Treebank WSJ corpus, perform badly on tweets. Tweets are conversational, often misspelled, short, and full of platform-specific tokens. The authors built an English POS tagger designed for Twitter. They defined a coarse 25-tag inventory that adds categories for hashtags, at-mentions, URLs, emoticons, discourse markers like RT, and fused forms such as noun+verb contractions. Seventeen annotators corrected automatic pre-tags on 1,827 tweets, followed by review and consistency passes. The tagger is a CRF that combines base lexical features with Twitter-oriented ones: orthographic rules, capitalization gazetteers, a soft PTB tag dictionary, distributional clusters from unlabeled tweets, and Metaphone phonetic keys. It reaches about 89.4% test accuracy, cutting errors by roughly a quarter compared with a retrained Stanford tagger. The whole effort took about 200 person-hours over two months. The corpus and tagger were released as TweetNLP. The work shows that a careful tagset and well-chosen features can quickly produce a usable tool for a messy new domain. ## Key points - **Corpus:** 1,827 English tweets (26,436 tokens), sampled from one day in October 2010 and split into 1,000 train, 327 dev and 500 test tweets. - **Agreement:** On 72 independently re-annotated tweets, inter-annotator agreement was 92.2%, with Cohen's κ = 0.914. - **Accuracy:** - Full model: 89.37% test. - Base features only: 83.38%. - Stanford tagger retrained on the same data: 85.85%. - **Ablations:** - Removing DISTSIM or TAGDICT costs about 1.06 points each. - Removing TWORTH costs about 1.00. - Removing METAPH costs about 0.42. - NAMES gave essentially no gain. - **Weak spots and data efficiency:** Proper-noun recall is only 71%, and the miscellaneous G class is the hardest. Training on just 500 tweets still yields 87.66%. ## Interesting ideas - **Pre-tag, then correct:** Annotators fixed the output of an existing out-of-domain tagger plus heuristics rather than labeling from scratch. This can sharply cut annotation time when building a corpus for a new domain or a low-resource variety. - **Leave fused forms whole:** Instead of forcing contractions and possessives to split, the tagset adds combined tags such as nominal+verbal. This avoids brittle tokenization of nonstandard spellings and could suit colloquial text in other languages with clitics or slang fusions. - **Soft external dictionaries:** Tag-dictionary entries from an external treebank enter the model as features rather than hard constraints. This is a cheap form of type-level domain adaptation that tolerates poor lexical coverage and dialect drift. - **Phonetic keys for spelling variants:** Metaphone keys group variants (e.g., several spellings of "thanks") into one feature, and the model also checks the most common PTB tag for words sharing that key. A language-appropriate phonetic or transliteration normalizer could do the same for noisy user text elsewhere. - **Context-based features for rare words:** Predecessor and successor statistics from about 134,000 unlabeled in-domain tweets help with words seen rarely or never in training. This is useful whenever labeled data is tiny but raw text is plentiful. *Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.* Fields: [Natural language processing](https://md.research.elamur.ai/free/en/shelves/nlp/) Topics: [Part-of-speech tagging](https://md.research.elamur.ai/free/en/topics/part-of-speech-tagging/) ## Cited by Cited by 276 papers of our library cite it (of the 140884 whose bibliographies we read); 43 of them have a page here, 233 not yet. - [A Dependency Parser for Tweets](https://md.research.elamur.ai/free/en/papers/aclbulk-2014-kong-dependency-parser-tweets/) - [An Evaluation of Parser Robustness for Ungrammatical Sentences](https://md.research.elamur.ai/free/en/papers/aclbulk-2016-hashemi-evaluation-parser-robustness-ungrammatical-sentences/) - [Anaphora Resolution for Twitter Conversations: An Exploratory Study](https://md.research.elamur.ai/free/en/papers/akta-2018-anaphora-resolution-twitter-conversations-exploratory/) - [We Need to Consider Disagreement in Evaluation](https://md.research.elamur.ai/free/en/papers/basile-2021-we-need-consider-disagreement-evaluation/) - [DataStories at SemEval-2017 Task 4: Deep LSTM with Attention for Message-level and Topic-based Sentiment Analysis](https://md.research.elamur.ai/free/en/papers/baziotis-2017-datastories-semeval-2017-task-4/) - [Text: now in 2D! A framework for lexical expansion with contextual similarity](https://md.research.elamur.ai/free/en/papers/biemann-2013-text-now-2d-framework-lexical/) - [Demographic Dialectal Variation in Social Media: A Case Study of African-American English](https://md.research.elamur.ai/free/en/papers/blodgett-2016-demographic-dialectal-variation-social-media/) - [Annotation Error Detection: Analyzing the Past and Present for a More Coherent Future](https://md.research.elamur.ai/free/en/papers/cl-2023-klie-annotation-error-detection-analyzing-past/) - [Adaptive Recursive Neural Network for Target-dependent Twitter Sentiment Classification](https://md.research.elamur.ai/free/en/papers/dong-2014-adaptive-recursive-neural-network-target/) - [What to do about bad language on the internet](https://md.research.elamur.ai/free/en/papers/eisenstein-2013-what-do-about-bad-language/) - [Sarcastic or Not: Word Embeddings to Predict the Literal or Sarcastic Meaning of Words](https://md.research.elamur.ai/free/en/papers/ghosh-2015-sarcastic-or-not-word-embeddings/) - [Sarcasm Analysis Using Conversation Context](https://md.research.elamur.ai/free/en/papers/ghosh-2018-sarcasm-analysis-conversation-context/) - [Monday mornings are my fave :) #not Exploring the Automatic Recognition of Irony in English tweets](https://md.research.elamur.ai/free/en/papers/hee-2016-monday-mornings-are-my-fave/) - [We Usually Don't Like Going to the Dentist: Using Common Sense to Detect Irony on Twitter](https://md.research.elamur.ai/free/en/papers/hee-2018-we-usually-don-t-like/) - [Deep contextualized word representations for detecting sarcasm and irony](https://md.research.elamur.ai/free/en/papers/ilic-2018-deep-contextualized-word-representations-detecting/) - [Learning a POS tagger for AAVE-like language](https://md.research.elamur.ai/free/en/papers/jorgensen-2016-learning-pos-tagger-aave-like/) - [Speech Act Modeling of Written Asynchronous Conversations with Task-Specific Embeddings and Conditional Structured Models](https://md.research.elamur.ai/free/en/papers/joty-2016-speech-act-modeling-written-asynchronous/) - [Modeling Speech Acts in Asynchronous Conversations: A Neural-CRF Approach](https://md.research.elamur.ai/free/en/papers/joty-2018-modeling-speech-acts-asynchronous-conversations/) - [NRC-Canada-2014: Detecting Aspects and Sentiment in Customer Reviews](https://md.research.elamur.ai/free/en/papers/kiritchenko-2014-nrc-canada-2014-detecting-aspects/) - [A Continuously Growing Dataset of Sentential Paraphrases](https://md.research.elamur.ai/free/en/papers/lan-2017-continuously-growing-dataset-sentential-paraphrases/) - [Agreeing to Disagree: Annotating Offensive Language Datasets with Annotators' Disagreement](https://md.research.elamur.ai/free/en/papers/leonardelli-2021-agreeing-disagree-annotating-offensive-language/) - [Two/Too Simple Adaptations of Word2Vec for Syntax Problems](https://md.research.elamur.ai/free/en/papers/ling-2015-two-too-simple-adaptations-word2vec/) - [The Denoised Web Treebank: Evaluating Dependency Parsing under Noisy Input Conditions](https://md.research.elamur.ai/free/en/papers/lingbulk-2016-daiber-denoised-web-treebank-evaluating-dependency/) - [NRC-Canada: Building the State-of-the-Art in Sentiment Analysis of Tweets](https://md.research.elamur.ai/free/en/papers/mohammad-2013-nrc-canada-building-state-art/) - [Emotion Intensities in Tweets](https://md.research.elamur.ai/free/en/papers/mohammad-2017-emotion-intensities-tweets/) - [WASSA-2017 Shared Task on Emotion Intensity](https://md.research.elamur.ai/free/en/papers/mohammad-2017-wassa-2017-shared-task-emotion/) - [BERTweet: A pre-trained language model for English Tweets](https://md.research.elamur.ai/free/en/papers/nguyen-2020-bertweet-pre-trained-language-model/) - [A review of sentiment analysis research in Arabic language](https://md.research.elamur.ai/free/en/papers/oueslati-2020-review-sentiment-analysis-research-arabic/) - [Improved Part-of-Speech Tagging for Online Conversational Text with Word Clusters](https://md.research.elamur.ai/free/en/papers/owoputi-2013-improved-part-speech-tagging-online/) - [Learning part-of-speech taggers with inter-annotator agreement loss](https://md.research.elamur.ai/free/en/papers/plank-2014-learning-part-speech-taggers-inter/) - [Linguistically debatable or just plain wrong?](https://md.research.elamur.ai/free/en/papers/plank-2014-linguistically-debatable-or-just-plain/) - [Neural Unsupervised Domain Adaptation in NLP–-A Survey](https://md.research.elamur.ai/free/en/papers/ramponi-2020-neural-unsupervised-domain-adaptation-nlp/) - [Named Entity Recognition in Tweets: An Experimental Study](https://md.research.elamur.ai/free/en/papers/ritter-2011-named-entity-recognition-tweets-experimental/) - [SemEval-2014 Task 9: Sentiment Analysis in Twitter](https://md.research.elamur.ai/free/en/papers/rosenthal-2014-semeval-2014-task-9-sentiment/) - [A Dataset for Multi-Target Stance Detection](https://md.research.elamur.ai/free/en/papers/sobhani-2017-dataset-multi-target-stance-detection/) - [Now We Stronger than Ever: African-American English Syntax in Twitter](https://md.research.elamur.ai/free/en/papers/stewart-2014-now-we-stronger-than-ever/) - [Learning Sentiment-Specific Word Embedding for Twitter Sentiment Classification](https://md.research.elamur.ai/free/en/papers/tang-2014-learning-sentiment-specific-word-embedding/) - [Parsing Tweets into Universal Dependencies](https://md.research.elamur.ai/free/en/papers/ud-2018-liu-parsing-tweets-into-universal-dependencies/) - [POS Tagging of English-Hindi Code-Mixed Social Media Content](https://md.research.elamur.ai/free/en/papers/vyas-2014-pos-tagging-english-hindi-code/) - [That's So Annoying!!!: A Lexical and Frame-Semantic Embedding Based Data Augmentation Approach to Automatic Categorization of Annoying Behaviors using #petpeeve Tweets](https://md.research.elamur.ai/free/en/papers/wang-2015-that-s-so-annoying-lexical/) - [Are You a Racist or Am I Seeing Things? Annotator Influence on Hate Speech Detection on Twitter](https://md.research.elamur.ai/free/en/papers/waseem-2016-are-you-racist-or-am/) - [From Shakespeare to Twitter: What are Language Styles all about?](https://md.research.elamur.ai/free/en/papers/xu-2017-shakespeare-twitter-what-are-language/) - [SemEval-2019 Task 6: Identifying and Categorizing Offensive Language in Social Media (OffensEval)](https://md.research.elamur.ai/free/en/papers/zampieri-2019-semeval-2019-task-6-identifying/) ## Cite The paper itself, not this page. BibTeX from the library's record of the work. ```bibtex @inproceedings{gimpel2011part, title = {{Part-of-Speech Tagging for Twitter: Annotation, Features, and Experiments}}, author = {Gimpel, Kevin and Schneider, Nathan and O'Connor, Brendan and Das, Dipanjan and Mills, Daniel and Eisenstein, Jacob and Heilman, Michael and Yogatama, Dani and Flanigan, Jeffrey and Smith, Noah A.}, booktitle = {Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies}, year = {2011}, doi = {10.21236/ada547371}, url = {https://doi.org/10.21236/ada547371} } ``` - [← previous Parsing Noun Phrases in the Penn Treebank](https://md.research.elamur.ai/free/en/papers/bank-2011-vadas-parsing-noun-phrases-penn-treebank/) - [next → Part-of-Speech Tagging from 97% to 100%: Is It Time for Some Linguistics?](https://md.research.elamur.ai/free/en/papers/enud-2011-manning-part-speech-tagging-97-100/) - [↑ back to papers of 2011](https://md.research.elamur.ai/free/en/papers/2011/) ## References 18 references, 5 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet. 1. Sitaram Asur and Bernardo A. Huberman. 2010. Predicting the future with social media. In Proc. of WI-IAT. [find](https://scholar.google.com/scholar?q=Sitaram+Asur+and+Bernardo+A%2E+Huberman%2E+2010%2E+Predicting+the+future+with+social+media%2E+In+Proc%2E+of+WI%2DIAT%2E) 2. Luciano Barbosa and Junlan Feng. 2010. Robust sentiment detection on Twitter from biased and noisy data. In Proc. of COLING. [find](https://scholar.google.com/scholar?q=Luciano+Barbosa+and+Junlan+Feng%2E+2010%2E+Robust+sentiment+detection+on+Twitter+from+biased+and+noisy+data%2E+In+Proc%2E+of+COLING%2E) [our card](https://md.research.elamur.ai/free/en/papers/barbosa-2010-robust-sentiment-detection-twitter-biased/) 3. Lauren Collister. 2010. Meaning variation of the iconic deictics ˆ and \\<— in an online community. In New Ways of Analyzing Variation. [find](https://scholar.google.com/scholar?q=Lauren+Collister%2E+2010%2E+Meaning+variation+of+the+iconic+deictics+%CB%86+and+%5C%3C%E2%80%94+in+an+online+community%2E+In+New+Ways+of+Analyzing+Variation%2E) 4. Christiane Fellbaum. 1998. WordNet: An Electronic Lexical Database. Bradford Books. [find](https://scholar.google.com/scholar?q=Christiane+Fellbaum%2E+1998%2E+WordNet%3A+An+Electronic+Lexical+Database%2E+Bradford+Books%2E) 5. Tim Finin, Will Murnane, Anand Karandikar, Nicholas Keller, Justin Martineau, and Mark Dredze. 2010. Annotating named entities in Twitter data with crowdsourcing. In Proceedings of the NAACL HLT 2010 Workshop on Creating Speech and Language Data with Amazon’s Mechanical Turk. [find](https://scholar.google.com/scholar?q=Tim+Finin%2C+Will+Murnane%2C+Anand+Karandikar%2C+Nicholas+Keller%2C+Justin+Martineau%2C+and+Mark+Dredze%2E+2010%2E+Annotating+named+entities+in+Twitter+data+with+crowdsourcing%2E+In+Proceedings+of+the+NAACL+HLT+2010+) 6. John Lafferty, Andrew McCallum, and Fernando Pereira. 2001. Conditional random fields: Probabilistic models for segmenting and labeling sequence data. In Proc. of ICML. [find](https://scholar.google.com/scholar?q=John+Lafferty%2C+Andrew+McCallum%2C+and+Fernando+Pereira%2E+2001%2E+Conditional+random+fields%3A+Probabilistic+models+for+segmenting+and+labeling+sequence+data%2E+In+Proc%2E+of+ICML%2E) 7. Mitchell P. Marcus, Beatrice Santorini, and Mary Ann Marcinkiewicz. 1993. Building a large annotated corpus of English: The Penn Treebank. Computational Linguistics, 19:313–330. [find](https://scholar.google.com/scholar?q=Mitchell+P%2E+Marcus%2C+Beatrice+Santorini%2C+and+Mary+Ann+Marcinkiewicz%2E+1993%2E+Building+a+large+annotated+corpus+of+English%3A+The+Penn+Treebank%2E+Computational+Linguistics%2C+19%3A313%E2%80%93330%2E) [our card](https://md.research.elamur.ai/free/en/papers/bank-1993-marcus-building-large-annotated-corpus-english/) 8. Brendan O’Connor, Ramnath Balasubramanyan, Bryan R. Routledge, and Noah A. Smith. 2010a. From tweets to polls: Linking text sentiment to public opinion time series. In Proc. of ICWSM. [find](https://scholar.google.com/scholar?q=Brendan+O%E2%80%99Connor%2C+Ramnath+Balasubramanyan%2C+Bryan+R%2E+Routledge%2C+and+Noah+A%2E+Smith%2E+2010a%2E+From+tweets+to+polls%3A+Linking+text+sentiment+to+public+opinion+time+series%2E+In+Proc%2E+of+ICWSM%2E) 9. Brendan O’Connor, Michel Krieger, and David Ahn. 2010b. TweetMotif: Exploratory search and topic summarization for Twitter. In Proc. of ICWSM (demo track). [find](https://scholar.google.com/scholar?q=Brendan+O%E2%80%99Connor%2C+Michel+Krieger%2C+and+David+Ahn%2E+2010b%2E+TweetMotif%3A+Exploratory+search+and+topic+summarization+for+Twitter%2E+In+Proc%2E+of+ICWSM+%28demo+track%29%2E) 10. Slav Petrov, Dipanjan Das, and Ryan McDonald. 2011. A universal part-of-speech tagset. ArXiv:1104.2086. [find](https://scholar.google.com/scholar?q=Slav+Petrov%2C+Dipanjan+Das%2C+and+Ryan+McDonald%2E+2011%2E+A+universal+part%2Dof%2Dspeech+tagset%2E+ArXiv%3A1104%2E2086%2E) 11. Lawrence Philips. 1990. Hanging on the Metaphone. Computer Language, 7(12). [find](https://scholar.google.com/scholar?q=Lawrence+Philips%2E+1990%2E+Hanging+on+the+Metaphone%2E+Computer+Language%2C+7%2812%29%2E) 12. Alan Ritter, Colin Cherry, and Bill Dolan. 2010. Unsupervised modeling of Twitter conversations. In Proc. of NAACL. [find](https://scholar.google.com/scholar?q=Alan+Ritter%2C+Colin+Cherry%2C+and+Bill+Dolan%2E+2010%2E+Unsupervised+modeling+of+Twitter+conversations%2E+In+Proc%2E+of+NAACL%2E) [our card](https://md.research.elamur.ai/free/en/papers/ritter-2010-unsupervised-modeling-twitter-conversations/) 13. Hinrich Schütze and Jan Pedersen. 1993. A vector model for syntagmatic and paradigmatic relatedness. In Proceedings of the 9th Annual Conference of the UW Centre for the New OED and Text Research. [find](https://scholar.google.com/scholar?q=Hinrich+Schu%CC%88tze+and+Jan+Pedersen%2E+1993%2E+A+vector+model+for+syntagmatic+and+paradigmatic+relatedness%2E+In+Proceedings+of+the+9th+Annual+Conference+of+the+UW+Centre+for+the+New+OED+and+Text+Research%2E) 14. Beaux Sharifi, Mark-Anthony Hutton, and Jugal Kalita. 2010. Summarizing microblogs automatically. In Proc. of NAACL. [find](https://scholar.google.com/scholar?q=Beaux+Sharifi%2C+Mark%2DAnthony+Hutton%2C+and+Jugal+Kalita%2E+2010%2E+Summarizing+microblogs+automatically%2E+In+Proc%2E+of+NAACL%2E) 15. Mike Thelwall, Kevan Buckley, and Georgios Paltoglou. 2011. Sentiment in Twitter events. Journal of the American Society for Information Science and Technology, 62(2):406–418. [find](https://scholar.google.com/scholar?q=Mike+Thelwall%2C+Kevan+Buckley%2C+and+Georgios+Paltoglou%2E+2011%2E+Sentiment+in+Twitter+events%2E+Journal+of+the+American+Society+for+Information+Science+and+Technology%2C+62%282%29%3A406%E2%80%93418%2E) 16. Kristina Toutanova, Dan Klein, Christopher D. Manning, and Yoram Singer. 2003. Feature-rich part-of-speech tagging with a cyclic dependency network. In Proc. of HLT-NAACL. [find](https://scholar.google.com/scholar?q=Kristina+Toutanova%2C+Dan+Klein%2C+Christopher+D%2E+Manning%2C+and+Yoram+Singer%2E+2003%2E+Feature%2Drich+part%2Dof%2Dspeech+tagging+with+a+cyclic+dependency+network%2E+In+Proc%2E+of+HLT%2DNAACL%2E) [our card](https://md.research.elamur.ai/free/en/papers/toutanova-2003-feature-rich-part-speech-tagging/) 17. Joseph Turian, Lev Ratinov, and Yoshua Bengio. 2010. Word representations: a simple and general method for semi-supervised learning. In Proc. of ACL. [find](https://scholar.google.com/scholar?q=Joseph+Turian%2C+Lev+Ratinov%2C+and+Yoshua+Bengio%2E+2010%2E+Word+representations%3A+a+simple+and+general+method+for+semi%2Dsupervised+learning%2E+In+Proc%2E+of+ACL%2E) [our card](https://md.research.elamur.ai/free/en/papers/turian-2010-word-representations-simple-general-method/) 18. Grady Ward. 1996. Moby lexicon. [http://icon.shef.ac.uk/Moby](http://icon.shef.ac.uk/Moby). [↗](http://icon.shef.ac.uk/Moby)