---
title: "emoji2vec: Learning Emoji Representations from their Description"
description: "Widely used pre-trained word embeddings such as word2vec and GloVe have few… 85.5% accuracy. 23 references, cited by 69 papers here."
authors: ["Ben Eisner", "Tim Rocktäschel", "Isabelle Augenstein", "Matko Bošnjak", "Sebastian Riedel"]
year: 2016
venue: "Proceedings of the Fourth International Workshop on Natural Language Processing for Social Media"
source: "https://aclanthology.org/W16-6208/"
doi: "10.18653/v1/w16-6208"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/eisner-2016-emoji2vec-learning-emoji-representations-their/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/eisner-2016-emoji2vec-learning-emoji-representations-their/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2016](https://md.research.elamur.ai/free/en/papers/2016/page/2/) / emoji2vec: Learning Emoji Representations from their Description

# emoji2vec: Learning Emoji Representations from their Description

**Authors:** Ben Eisner, [Tim Rocktäschel](https://md.research.elamur.ai/free/en/authors/tim-rocktaschel/), [Isabelle Augenstein](https://md.research.elamur.ai/free/en/authors/isabelle-augenstein/), Matko Bošnjak and [Sebastian Riedel](https://md.research.elamur.ai/free/en/authors/sebastian-riedel/) · **Year:** 2016 · **Venue:** [Proceedings of the Fourth International Workshop on Natural Language Processing for Social Media](https://md.research.elamur.ai/free/en/venues/socialnlp/) ([International Workshop on Natural Language Processing for Social Media 2016](https://md.research.elamur.ai/free/en/venues/socialnlp/2016/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/W16-6208/) · [PDF](https://aclanthology.org/W16-6208.pdf) · [DOI 10.18653/v1/w16-6208](https://doi.org/10.18653/v1/w16-6208) · [OpenAlex](https://openalex.org/W2963291843)

**Citations:** 291 citations (OpenAlex, 2026-10-14) · [69](https://md.research.elamur.ai/free/en/papers/eisner-2016-emoji2vec-learning-emoji-representations-their/cited-by/) in our library, of the 141,840 papers whose bibliographies we read

## Summary
Widely used pre-trained word embeddings such as word2vec and GloVe have few or no vectors for emoji, even though emoji are now everywhere on social media. Earlier emoji vectors were learned with skip-gram on very large tweet collections. That approach works only for emoji that appear often: one such corpus covered roughly 700 of the 1600+ Unicode emoji.

This paper learns emoji vectors from the official Unicode emoji list instead. Names and keyword phrases give 6088 descriptions for 1661 symbols. Each description becomes the sum of its Google News word2vec vectors. A separate vector per emoji is then trained so that a sigmoid over its dot product with the description vector separates correct descriptions from randomly paired wrong ones. The result, emoji2vec, sits in the same 300-dimensional space as word2vec, so it can be added to existing pipelines without changes.

On hand-labelled emoji–phrase pairs the model scores 85.5% accuracy. In three-class Twitter sentiment classification it beats both plain word2vec and tweet-trained emoji vectors, with the largest gains on tweets containing emoji. This shows that a small, cheap resource can cover rarely used symbols.

## Key points
- **Training data and setup:** 6088 Unicode descriptions (names and keywords) for 1661 emoji, mapped into the 300-dim Google News word2vec space. The model was trained in TensorFlow with Adam, using a 1:1 ratio of negative to positive samples and about 80 epochs. Training takes under 3 minutes on a 2013 MacBook Pro.
- **Intrinsic test:** emoji–description pair classification reaches 85.5% accuracy at a 0.5 threshold, with ROC AUC 0.933.
- **Downstream test:** three-way sentiment classification on the Kralj Novak et al. (2015) corpus of over 67k English tweets (46% neutral, 29% positive, 25% negative). Tweet features are summed word and emoji vectors, fed to a Random Forest or Linear SVM.
- **Linear SVM results:**

  | Test subset | Google News only | + Barbieri et al. (2016) | + emoji2vec |
  |---|---|---|---|
  | Full test set (N = 12,920) | 58.5 | 60.0 | 60.5 |
  | Tweets with emoji (N = 2295) | 47.1 | 57.4 | 59.2 |
  | 10% least frequent emoji (N = 308) | 43.2 | 52.9 | 55.2 |

  Significance was checked with McNemar's test.
- **Release:** embeddings, training data and code are at github.com/uclmr/emoji2vec, in word2vec-compatible format (loadable with gensim). Known weaknesses include poorly separated number symbols in t-SNE plots and no modelling of cultural or contextual meaning shifts such as sarcasm.

## Interesting ideas
- **Learn from definitions, not usage:** training vectors from short dictionary-style descriptions removes the need for many usage contexts. This could give coverage for rare tokens, symbols or terms in low-resource settings where large corpora are unavailable.
- **Project into an existing embedding space:** fitting new vectors into an existing pre-trained space makes them a drop-in extension. Downstream models need no retraining or alignment step, which keeps adoption cheap.
- **Use mismatched pairs as negatives:** when only correct pairs exist, randomly re-pairing items with wrong descriptions supplies negative examples. A tuned positive-to-negative ratio (1:1 worked best here) turns the task into simple binary matching.
- **Split evaluation by frequency:** reporting results separately for frequent and rare items, plus the subset where the new tokens actually occur, shows whether a method helps the long tail. Full-corpus averages can hide this, as the small overall gains here show.
- **Pair intrinsic and extrinsic tests:** a small hand-labelled pair-matching set evaluated with ROC/AUC, combined with a deliberately simple downstream classifier and McNemar's significance test, isolates the value of the embeddings themselves rather than the model built on top of them.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

📚 Shelves (5): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 9,086 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 8,889 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 9,296 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 9,206 works · [🖱️ Human-Computer Interaction](https://md.research.elamur.ai/free/en/shelves/computer-science/human-computer-interaction/) · 61 works

## Cite

Cite as: Ben Eisner, Tim Rocktäschel, Isabelle Augenstein, Matko Bošnjak and Sebastian Riedel. 2016. emoji2vec: Learning Emoji Representations from their Description. Proceedings of the Fourth International Workshop on Natural Language Processing for Social Media.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{eisner2016emoji2vec,
  title = {{emoji2vec: Learning Emoji Representations from their Description}},
  author = {Ben Eisner, Tim Rocktäschel, Isabelle Augenstein, Matko Bošnjak and Sebastian Riedel},
  booktitle = {Proceedings of the Fourth International Workshop on Natural Language Processing for Social Media},
  year = {2016},
  doi = {10.18653/v1/w16-6208},
  url = {https://doi.org/10.18653/v1/w16-6208}
}
```

- [← previous Analyzing the Behavior of Visual Question Answering Models](https://md.research.elamur.ai/free/en/papers/agrawal-2016-analyzing-behavior-visual-question-answering/)
- [next → Question Answering on Freebase via Relation Extraction and Textual Evidence](https://md.research.elamur.ai/free/en/papers/xu-2016-question-answering-freebase-relation-extraction/)
- [↑ back to papers of 2016](https://md.research.elamur.ai/free/en/papers/2016/page/2/)

## References

23 references, 9 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Martın Abadi, Ashish Agarwal, Paul Barham, Eugene Brevdo, Zhifeng Chen, Craig Citro, Greg S Corrado, Andy Davis, Jeffrey Dean, Matthieu Devin, et al. 2015. [TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems](https://md.research.elamur.ai/free/en/papers/abadi-2016-tensorflow-large-scale-machine-learning/). Software available from tensorflow. org, 1. Isabelle Augenstein,…
2. Francesco Barbieri, Francesco Ronzano, and Horacio Saggion. 2016. What does this Emoji Mean? A Vector Space Skip-Gram Model for Twitter Emojis. In Proceedings of LREC, May. [find](https://scholar.google.com/scholar?q=Francesco+Barbieri%2C+Francesco+Ronzano%2C+and+Horacio+Saggion%2E+2016%2E+What+does+this+Emoji+Mean%3F+A+Vector+Space+Skip%2DGram+Model+for+Twitter+Emojis%2E+In+Proceedings+of+LREC%2C+May%2E)
3. Joe Cruse. 2015. Emoji usage in TV conversation. [find](https://scholar.google.com/scholar?q=Joe+Cruse%2E+2015%2E+Emoji+usage+in+TV+conversation%2E)
4. Bhuwan Dhingra, Zhong Zhou, Dylan Fitzpatrick, Michael Muehl, and William Cohen. 2016. Tweet2Vec: Character-Based Distributed Representations for Social Media. In Proceedings of ACL, pages 269–274. [find](https://scholar.google.com/scholar?q=Bhuwan+Dhingra%2C+Zhong+Zhou%2C+Dylan+Fitzpatrick%2C+Michael+Muehl%2C+and+William+Cohen%2E+2016%2E+Tweet2Vec%3A+Character%2DBased+Distributed+Representations+for+Social+Media%2E+In+Proceedings+of+ACL%2C+pages+269%E2%80%93274%2E)
5. Thomas Dimson. 2015. Machine Learning for Emoji Trends. [http://instagram-engineering.tumblr.com/post/](http://instagram-engineering.tumblr.com/post/) 117889701472/emojineering-part-1-machine-learning-for-emoji. Accessed: 2016-09-05. [↗](http://instagram-engineering.tumblr.com/post/)
6. Li Dong, Furu Wei, Chuanqi Tan, Duyu Tang, Ming Zhou, and Ke Xu. 2014. Adaptive Recursive Neural emojipedia.org [https://radimrehurek.com/gensim/models/](https://radimrehurek.com/gensim/models/) word2vec.html Network for Target-dependent Twitter Sentiment Classification. In Proceedings of ACL, pages 49–54. [↗](https://radimrehurek.com/gensim/models/)
7. Rong-En Fan, Kai-Wei Chang, Cho-Jui Hsieh, Xiang-Rui Wang, and Chih-Jen Lin. 2008. Liblinear: A library for large linear classification. Journal of machine learning research, 9(Aug):1871–1874. [find](https://scholar.google.com/scholar?q=Rong%2DEn+Fan%2C+Kai%2DWei+Chang%2C+Cho%2DJui+Hsieh%2C+Xiang%2DRui+Wang%2C+and+Chih%2DJen+Lin%2E+2008%2E+Liblinear%3A+A+library+for+large+linear+classification%2E+Journal+of+machine+learning+research%2C+9%28Aug%29%3A1871%E2%80%931874%2E)
8. Abhishek Gattani, Digvijay S Lamba, Nikesh Garera, Mitul Tiwari, Xiaoyong Chai, Sanjib Das, Sri Subramaniam, Anand Rajaraman, Venky Harinarayan, and AnHai Doan. 2013. Entity Extraction, Linking, Classification, and Tagging for Social Media: A Wikipedia-Based Approach. In Proceedings of the VLDB Endo… [find](https://scholar.google.com/scholar?q=Abhishek+Gattani%2C+Digvijay+S+Lamba%2C+Nikesh+Garera%2C+Mitul+Tiwari%2C+Xiaoyong+Chai%2C+Sanjib+Das%2C+Sri+Subramaniam%2C+Anand+Rajaraman%2C+Venky+Harinarayan%2C+and+AnHai+Doan%2E+2013%2E+Entity+Extraction%2C+Linking%2C+Class)
9. Felix Hill, Kyunghyun Cho, Anna Korhonen, and Yoshua Bengio. 2016. [Learning to Understand Phrases by Embedding the Dictionary](https://md.research.elamur.ai/free/en/papers/hill-2016-learning-understand-phrases-embedding-dictionary/).
10. TACL. Tin Kam Ho. 1995. Random decision forests. In Document Analysis and Recognition, 1995., Proceedings of the Third International Conference on, volume 1, pages 278–282. [find](https://scholar.google.com/scholar?q=TACL%2E+Tin+Kam+Ho%2E+1995%2E+Random+decision+forests%2E+In+Document+Analysis+and+Recognition%2C+1995%2E%2C+Proceedings+of+the+Third+International+Conference+on%2C+volume+1%2C+pages+278%E2%80%93282%2E)
11. IEEE. Ryan Kelly and Leon Watts. 2015. Characterising the inventive appropriation of emoji as relationally meaningful in mediated close personal relationships. Experiences of Technology Appropriation: Unanticipated Users, Usage, Circumstances, and Design. [find](https://scholar.google.com/scholar?q=IEEE%2E+Ryan+Kelly+and+Leon+Watts%2E+2015%2E+Characterising+the+inventive+appropriation+of+emoji+as+relationally+meaningful+in+mediated+close+personal+relationships%2E+Experiences+of+Technology+Appropriation%3A)
12. Diederik Kingma and Jimmy Ba. 2015. Adam: A Method for Stochastic Optimization. In Proceedings of ICLR. Petra Kralj Novak, Jasmina Smailović, Borut Sluban, and Igor Mozetič. 2015. Sentiment of Emojis. PLoS ONE, 10(12):1–22, 12. [find](https://scholar.google.com/scholar?q=Diederik+Kingma+and+Jimmy+Ba%2E+2015%2E+Adam%3A+A+Method+for+Stochastic+Optimization%2E+In+Proceedings+of+ICLR%2E+Petra+Kralj+Novak%2C+Jasmina+Smailovic%CC%81%2C+Borut+Sluban%2C+and+Igor+Mozetic%CC%8C%2E+2015%2E+Sentiment+of+Emoji)
13. Lisa Lebduska. 2014. Emoji, Emoji, What for Art Thou? Harlot: A Revealing Look at the Arts of Persuasion, 1(12). [find](https://scholar.google.com/scholar?q=Lisa+Lebduska%2E+2014%2E+Emoji%2C+Emoji%2C+What+for+Art+Thou%3F+Harlot%3A+A+Revealing+Look+at+the+Arts+of+Persuasion%2C+1%2812%29%2E)
14. Omer Levy and Yoav Goldberg. 2014. [Linguistic Regularities in Sparse and Explicit Word Representations](https://md.research.elamur.ai/free/en/papers/levy-2014-linguistic-regularities-sparse-explicit-word/). In Proceedings of ConLL, pages 171–180.
15. Laurens van der Maaten and Geoffrey Hinton. 2008. Visualizing data using t-sne. Journal of Machine Learning Research, 9(Nov):2579–2605. [find](https://scholar.google.com/scholar?q=Laurens+van+der+Maaten+and+Geoffrey+Hinton%2E+2008%2E+Visualizing+data+using+t%2Dsne%2E+Journal+of+Machine+Learning+Research%2C+9%28Nov%29%3A2579%E2%80%932605%2E)
16. Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S Corrado, and Jeff Dean. 2013a. [Distributed Representations of Words and Phrases and their Compositionality](https://md.research.elamur.ai/free/en/papers/mikolov-2013-distributed-representations-words-phrases-their/). In Proceedings of NIPS, pages 3111–3119.
17. Tomas Mikolov, Wen-tau Yih, and Geoffrey Zweig. 2013b. [Linguistic Regularities in Continuous Space Word Representations](https://md.research.elamur.ai/free/en/papers/mikolov-2013-linguistic-regularities-continuous-space-word/). In Proceedings of NAACL-HLT, pages 746–751.
18. Hannah Miller, Jacob Thebault-Spieker, Shuo Chang, Isaac Johnson, Loren Terveen, and Brent Hecht. 2016. “Blissfully happy” or “ready to fight”: Varying Interpretations of Emoji. In Proceedings of ICWSM. [find](https://scholar.google.com/scholar?q=Hannah+Miller%2C+Jacob+Thebault%2DSpieker%2C+Shuo+Chang%2C+Isaac+Johnson%2C+Loren+Terveen%2C+and+Brent+Hecht%2E+2016%2E+%E2%80%9CBlissfully+happy%E2%80%9D+or+%E2%80%9Cready+to+fight%E2%80%9D%3A+Varying+Interpretations+of+Emoji%2E+In+Proceedings+of+ICWS)
19. Jaram Park, Vladimir Barash, Clay Fink, and Meeyoung Cha. 2013. Emoticon style: Interpreting differences in emoticons across cultures. In ICWSM. [find](https://scholar.google.com/scholar?q=Jaram+Park%2C+Vladimir+Barash%2C+Clay+Fink%2C+and+Meeyoung+Cha%2E+2013%2E+Emoticon+style%3A+Interpreting+differences+in+emoticons+across+cultures%2E+In+ICWSM%2E)
20. Jeffrey Pennington, Richard Socher, and Christopher Manning. 2014. [Glove: Global Vectors for Word Representation](https://md.research.elamur.ai/free/en/papers/pennington-2014-glove-global-vectors-word-representation/). In Proceedings of EMNLP, pages 1532– 1543, October.
21. Alan Ritter, Sam Clark, Mausam, and Oren Etzioni. 2011. [Named Entity Recognition in Tweets: An Experimental Study](https://md.research.elamur.ai/free/en/papers/ritter-2011-named-entity-recognition-tweets-experimental/). In Proceedings of EMNLP, pages 1524–1534.
22. Sara Rosenthal, Preslav Nakov, Svetlana Kiritchenko, Saif Mohammad, Alan Ritter, and Veselin Stoyanov. 2015. [SemEval-2015 Task 10: Sentiment Analysis in Twitter](https://md.research.elamur.ai/free/en/papers/rosenthal-2015-semeval-2015-task-10-sentiment/). In Proceedings of the SemEval, pages 451– 463.
23. Duyu Tang, Furu Wei, Nan Yang, Ming Zhou, Ting Liu, and Bing Qin. 2014. [Learning Sentiment-Specific Word Embedding for Twitter Sentiment Classification](https://md.research.elamur.ai/free/en/papers/tang-2014-learning-sentiment-specific-word-embedding/). In Proceedings of ACL, pages 1555–1565.

*Updated 24 October 2026*