---
title: "Deep contextualized word representations for — Ilić 2018"
description: "Sarcasm and irony often flip the literal polarity of a text, which makes them hard for sentiment and opinion-mining systems."
authors: ["Suzana Ilić", "Edison Marrese-Taylor", "Jorge Balazs", "Yutaka Matsuo"]
year: 2018
venue: "Proceedings of the 9th Workshop on Computational Approaches to Subjectivity, Sentiment and Social Media Analysis"
source: "https://aclanthology.org/W18-6202/"
doi: "10.18653/v1/w18-6202"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/papers/ilic-2018-deep-contextualized-word-representations-detecting/"
---
[Elamur Research](https://md.research.elamur.ai/) · [this page in HTML](https://research.elamur.ai/free/papers/ilic-2018-deep-contextualized-word-representations-detecting/)

[home](https://md.research.elamur.ai/) / [papers](https://md.research.elamur.ai/free/papers/) / [2018](https://md.research.elamur.ai/free/papers/2018/) / Deep contextualized word representations for detecting sarcasm and irony

# Deep contextualized word representations for detecting sarcasm and irony

**Authors:** Suzana Ilić, Edison Marrese-Taylor, Jorge Balazs, [Yutaka Matsuo](https://md.research.elamur.ai/free/authors/yutaka-matsuo/) · **Year:** 2018 · **Venue:** Proceedings of the 9th Workshop on Computational Approaches to Subjectivity, Sentiment and Social Media Analysis · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/W18-6202/) · [PDF](https://aclanthology.org/W18-6202.pdf) · [DOI 10.18653/v1/w18-6202](https://doi.org/10.18653/v1/w18-6202) · [OpenAlex](https://openalex.org/W2964101952)

**Citations:** 122 citations (openalex-snapshot · 2026-10-14) · [19](#cited-by) in our library, of the 140673 papers whose bibliographies we read

## Summary
Sarcasm and irony often flip the literal polarity of a text, which makes them hard for sentiment and opinion-mining systems. Earlier neural detectors relied on word-level embeddings and could miss surface signals such as capitalization, quotation marks, emoticons, emojis and hashtags. Ilić et al. address this with a character-aware model. Pre-trained ELMo embeddings, which are built from characters via CNNs and contextualized by a deep biLM, are kept frozen and fed into a BiLSTM. The BiLSTM states are max-pooled and passed through a two-layer feed-forward classifier for binary prediction. Preprocessing is deliberately light: no lowercasing, stemming or punctuation stripping.

The model was tested on seven benchmarks drawn from Twitter, Reddit and online debate dialogues. It achieved state-of-the-art results on six of them without attention, handcrafted features or multi-task learning. The only exception was SemEval-2018, where the winning system kept higher recall and F1. Adding hashtag-based, soft-labeled tweets did not help. This suggests that more manually annotated data, rather than noisy data, is needed. Code is released as elmo4irony.

## Key points
- **Architecture:** frozen ELMo (1,024-dim vectors, pre-trained on the 1 Billion Word Benchmark) → BiLSTM → max-pooling → 2 feed-forward layers of 512 units. Training used Adam with lr 0.001 and a modified Twokenizer for emoji handling.
- **Datasets:**
  - Twitter: SemEval-2018 Task 3, Riloff, Ptáček
  - Reddit: SARC 2.0, SARC 2.0 pol
  - IAC dialogues: SC-V1, SC-V2
- **Results:**
  - Ptáček: F1 0.872 vs 0.860 (Tay et al.)
  - Riloff: F1 0.759 vs 0.732
  - SC-V2: F1 0.760 vs 0.728
  - SC-V1: F1 0.644 vs 0.632
  - SARC 2.0: accuracy 0.773 vs 0.758 (Khodak et al.)
  - SARC 2.0 pol: accuracy 0.785 vs 0.765
- **SemEval-2018:**
  - A 10-model ensemble reached F1 0.696, behind the Wu et al. winner at 0.705, but with higher precision (0.696 vs 0.630).
  - As a single model, it beats Wu et al.'s single-model F1 of 0.674.
- **Data augmentation:** about 225k #sarcasm/#irony tweets were filtered by hashtag overlap and added to the three Twitter sets. This gave no gains in any of them.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

Fields: [Linguistics & languages](https://md.research.elamur.ai/free/shelves/linguistics/)

## Cited by

Cited by

19 papers of our library cite it (of the 140673 whose bibliographies we read); 3 of them have a page here, 16 not yet.

- [Affective and Contextual Embedding for Sarcasm Detection](https://md.research.elamur.ai/free/papers/babanejad-2020-affective-contextual-embedding-sarcasm-detection/)
- [A Transformer-based approach to Irony and Sarcasm detection](https://md.research.elamur.ai/free/papers/potamias-2019-transformer-based-approach-irony-sarcasm/)
- [A Survey in Automatic Irony Processing: Linguistic, Cognitive, and Multi-X Perspectives](https://md.research.elamur.ai/free/papers/zeng-2022-survey-automatic-irony-processing-linguistic/)

## Cite

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{ilic2018deep,
  title = {{Deep contextualized word representations for detecting sarcasm and irony}},
  author = {Ilić, Suzana and Marrese-Taylor, Edison and Balazs, Jorge and Matsuo, Yutaka},
  booktitle = {Proceedings of the 9th Workshop on Computational Approaches to Subjectivity, Sentiment and Social Media Analysis},
  year = {2018},
  doi = {10.18653/v1/w18-6202},
  url = {https://doi.org/10.18653/v1/w18-6202}
}
```

- [← previous Dear Sir or Madam, May I Introduce the GYAFC Dataset: Corpus, Benchmarks and Metrics for Formality Style Transfer](https://md.research.elamur.ai/free/papers/rao-2018-dear-sir-or-madam-may/)
- [next → Deep Enhanced Representation for Implicit Discourse Relation Recognition](https://md.research.elamur.ai/free/papers/bai-2018-deep-enhanced-representation-implicit-discourse/)
- [↑ back to papers of 2018](https://md.research.elamur.ai/free/papers/2018/)

## References

29 references, 13 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Mikhail Khodak, Nikunj Saunshi, and Kiran Vodra-halli. 2017. A Large Self-Annotated Corpus for Sarcasm. [find](https://scholar.google.com/scholar?q=Mikhail+Khodak%2C+Nikunj+Saunshi%2C+and+Kiran+Vodra%2Dhalli%2E+2017%2E+A+Large+Self%2DAnnotated+Corpus+for+Sarcasm%2E) [our card](https://md.research.elamur.ai/free/papers/khodak-2018-large-self-annotated-corpus-sarcasm/)
2. S. K. Bharti, B. Vachha, R. K. Pradhan, K. S. Babu, and S. K. Jena. 2016. Sarcastic sentiment detection in tweets streamed in real time: a big data approach. Digital Communications and Networks, 2(3):108– 121. [↗](https://doi.org/10.1016/j.dcan.2016.06.002)
3. Florian Kunneman, Christine Liebrecht, Margotvan Mulken, and Antalvan den Bosch. 2015. Signalling sarcasm : From hyperbole to hashtag. Information Processing & Management. [find](https://scholar.google.com/scholar?q=Florian+Kunneman%2C+Christine+Liebrecht%2C+Margotvan+Mulken%2C+and+Antalvan+den+Bosch%2E+2015%2E+Signalling+sarcasm+%3A+From+hyperbole+to+hashtag%2E+Information+Processing+%26+Management%2E)
4. Santosh Kumar Bharti, Korra Sathya Babu, and Sanjay Kumar Jena. 2015. Parsing-based Sarcasm Sentiment Recognition in Twitter Data. Proceedings of the 2015 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining 2015 - ASONAM ’15, pages 1373–1380. [find](https://scholar.google.com/scholar?q=Santosh+Kumar+Bharti%2C+Korra+Sathya+Babu%2C+and+Sanjay+Kumar+Jena%2E+2015%2E+Parsing%2Dbased+Sarcasm+Sentiment+Recognition+in+Twitter+Data%2E+Proceedings+of+the+2015+IEEE%2FACM+International+Conference+on+Advances)
5. Joan Lucariello. 1994. Situational Irony: A Concept of Events Gone Awry. Journal of Experimental Psychology: General, 123(2):129–145. [↗](https://doi.org/10.1037//0096-3445.123.2.129)
6. Pranali Chaudhari and Chaitali Chandankhede. 2017. Literature Survey of Sarcasm Detection. pages 2041–2046. [find](https://scholar.google.com/scholar?q=Pranali+Chaudhari+and+Chaitali+Chandankhede%2E+2017%2E+Literature+Survey+of+Sarcasm+Detection%2E+pages+2041%E2%80%932046%2E)
7. Saif M. Mohammad, Felipe Bravo-Marquez, Salameh Mohammad, and Svetlana Kiritchenko. 2018. SemEval-2018 Task 1 : Affect in Tweets. In Proceedings of the 12th International Workshop on Semantic Evaluation (SemEval-2018), pages 1–17. [find](https://scholar.google.com/scholar?q=Saif+M%2E+Mohammad%2C+Felipe+Bravo%2DMarquez%2C+Salameh+Mohammad%2C+and+Svetlana+Kiritchenko%2E+2018%2E+SemEval%2D2018+Task+1+%3A+Affect+in+Tweets%2E+In+Proceedings+of+the+12th+International+Workshop+on+Semantic+Evaluati) [our card](https://md.research.elamur.ai/free/papers/mohammad-2018-semeval-2018-task-1-affect/)
8. Ciprian Chelba, Tomas Mikolov, Mike Schuster, Qi Ge, Thorsten Brants, Phillipp Koehn, and Tony Robinson. 2014. One billion word benchmark for measuring progress in statistical language modeling. Proceedings of the Annual Conference of the International Speech Communication Association, INTER-SPEECH, pages 2635–2639. [find](https://scholar.google.com/scholar?q=Ciprian+Chelba%2C+Tomas+Mikolov%2C+Mike+Schuster%2C+Qi+Ge%2C+Thorsten+Brants%2C+Phillipp+Koehn%2C+and+Tony+Robinson%2E+2014%2E+One+billion+word+benchmark+for+measuring+progress+in+statistical+language+modeling%2E+Proce) [our card](https://md.research.elamur.ai/free/papers/chelba-2013-one-billion-word-benchmark-measuring/)
9. Shereen Oraby, Vrindavan Harrison, Lena Reed, Ernesto Hernandez, Ellen Riloff, and Marilyn Walker. 2016. Creating and Characterizing a Diverse Corpus of Sarcasm in Dialogue. Proceedings of the 17th Annual Meeting of the Special Interest Group on Discourse and Dialogue, (September):31– 41. [find](https://scholar.google.com/scholar?q=Shereen+Oraby%2C+Vrindavan+Harrison%2C+Lena+Reed%2C+Ernesto+Hernandez%2C+Ellen+Riloff%2C+and+Marilyn+Walker%2E+2016%2E+Creating+and+Characterizing+a+Diverse+Corpus+of+Sarcasm+in+Dialogue%2E+Proceedings+of+the+17th+An) [our card](https://md.research.elamur.ai/free/papers/oraby-2016-creating-characterizing-diverse-corpus-sarcasm/)
10. Elena Filatova. 2012. Irony and Sarcasm: Corpus Generation and Analysis Using Crowdsourcing. Lrec, pages 392–398. [find](https://scholar.google.com/scholar?q=Elena+Filatova%2E+2012%2E+Irony+and+Sarcasm%3A+Corpus+Generation+and+Analysis+Using+Crowdsourcing%2E+Lrec%2C+pages+392%E2%80%93398%2E) [our card](https://md.research.elamur.ai/free/papers/filatova-2012-irony-sarcasm-corpus-generation-analysis/)
11. Bo Pang and Lillian Lee. 2006. Opinion Mining and Sentiment Analysis. Foundations and Trends in Information Retrieval, 1(2):91–231. [find](https://scholar.google.com/scholar?q=Bo+Pang+and+Lillian+Lee%2E+2006%2E+Opinion+Mining+and+Sentiment+Analysis%2E+Foundations+and+Trends+in+Information+Retrieval%2C+1%282%29%3A91%E2%80%93231%2E)
12. Kevin Gimpel, Nathan Schneider, Brendan O’Connor, Dipanjan Das, Daniel Mills, Jacob Eisenstein, Michael Heilman, Dani Yogatama, Jeffrey Flanigan, and Noah A. Smith. 2011. Part-of-speech tagging for twitter: Annotation, features, and experiments. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, pages 42–47, Portland, Oregon, USA. Association for Computational Linguistics. [↗](https://doi.org/10.21236/ada547371)
13. Matthew E. Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke Zettlemoyer. 2018. Deep contextualized word representations. [find](https://scholar.google.com/scholar?q=Matthew+E%2E+Peters%2C+Mark+Neumann%2C+Mohit+Iyyer%2C+Matt+Gardner%2C+Christopher+Clark%2C+Kenton+Lee%2C+and+Luke+Zettlemoyer%2E+2018%2E+Deep+contextualized+word+representations%2E)
14. Tomáš Ptáček, Ivan Habernal, and Jun Hong. 2014. Sarcasm Detection on Czech and English Twitter. Proceedings of the 25th International Conference on Computational Linguistics: Technical Papers (COL-ING 2014), Citeseer, pages 213–223. [find](https://scholar.google.com/scholar?q=Toma%CC%81s%CC%8C+Pta%CC%81c%CC%8Cek%2C+Ivan+Habernal%2C+and+Jun+Hong%2E+2014%2E+Sarcasm+Detection+on+Czech+and+English+Twitter%2E+Proceedings+of+the+25th+International+Conference+on+Computational+Linguistics%3A+Technical+Papers+%28CO) [our card](https://md.research.elamur.ai/free/papers/ptacek-2014-sarcasm-detection-czech-english-twitter/)
15. Antonio Reyes, Paolo Rosso, and Tony Veale. 2013. A multidimensional approach for detecting irony in Twitter. Language Resources and Evaluation, 47(1):239–268. [↗](https://doi.org/10.1007/s10579-012-9196-x)
16. H. Paul Grice. 1975. Logic and Conversation. In Peter Cole and Jerry L. Morgan, editors, Syntax and Semantics, volume 3, pages 41–58. Academic Press, New York. [↗](https://doi.org/10.1163/9789004368811_003)
17. Ellen Riloff, Ashequl Qadir, Prafulla Surve, Lalin-dra De Silva, Nathan Gilbert, and Ruihong Huang. 2013. Sarcasm as Contrast between a Positive Sentiment and Negative Situation. Emnlp, (Emnlp):704– 714. [find](https://scholar.google.com/scholar?q=Ellen+Riloff%2C+Ashequl+Qadir%2C+Prafulla+Surve%2C+Lalin%2Ddra+De+Silva%2C+Nathan+Gilbert%2C+and+Ruihong+Huang%2E+2013%2E+Sarcasm+as+Contrast+between+a+Positive+Sentiment+and+Negative+Situation%2E+Emnlp%2C+%28Emnlp%29%3A704%E2%80%93+7) [our card](https://md.research.elamur.ai/free/papers/prag-2013-riloff-sarcasm-as-contrast-between-positive/)
18. Jeremy Howard and Sebastian Ruder. 2018. Universal language model fine-tuning for text classification. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 328–339. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Jeremy+Howard+and+Sebastian+Ruder%2E+2018%2E+Universal+language+model+fine%2Dtuning+for+text+classification%2E+In+Proceedings+of+the+56th+Annual+Meeting+of+the+Association+for+Computational+Linguistics+%28Volum)
19. Cameron Shelley. 2001. The bicoherence theory of situational irony. Cognitive Science, 25(5):775–818. [↗](https://doi.org/10.1207/s15516709cog2505_7)
20. Aditya Joshi, Pushpak Bhattacharyya, and Mark James Carman. 2016. Automatic Sarcasm Detection: A Survey. 50(5). [↗](https://doi.org/10.48550/arxiv.1602.03426) [our card](https://md.research.elamur.ai/free/papers/joshi-2016-automatic-sarcasm-detection-survey/)
21. Yi Tay, Anh Tuan Luu, , Siu Cheung Hui, and Jian Su. 2018. Reasoning with sarcasm by reading in-between. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1010–1020. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Yi+Tay%2C+Anh+Tuan+Luu%2C+%2C+Siu+Cheung+Hui%2C+and+Jian+Su%2E+2018%2E+Reasoning+with+sarcasm+by+reading+in%2Dbetween%2E+In+Proceedings+of+the+56th+Annual+Meeting+of+the+Association+for+Computational+Linguistics+%28Vol) [our card](https://md.research.elamur.ai/free/papers/tay-2018-reasoning-sarcasm-reading-between/)
22. Aditya Joshi, Vinita Sharma, and Pushpak Bhattacharyya. 2015. Harnessing Context Incongruity for Sarcasm Detection. Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Short Papers), 51(4):757–762. [find](https://scholar.google.com/scholar?q=Aditya+Joshi%2C+Vinita+Sharma%2C+and+Pushpak+Bhattacharyya%2E+2015%2E+Harnessing+Context+Incongruity+for+Sarcasm+Detection%2E+Proceedings+of+the+53rd+Annual+Meeting+of+the+Association+for+Computational+Linguist) [our card](https://md.research.elamur.ai/free/papers/joshi-2015-harnessing-context-incongruity-sarcasm-detection/)
23. Piyoros Tungthamthiti, Kiyoaki Shirai, and Masnizah Mohd. 2010. Recognition of Sarcasm in Microblog-ging Based on Sentiment Analysis and Coherence Identification. Journal of Natural Language Processing, 23(5):383–405. [find](https://scholar.google.com/scholar?q=Piyoros+Tungthamthiti%2C+Kiyoaki+Shirai%2C+and+Masnizah+Mohd%2E+2010%2E+Recognition+of+Sarcasm+in+Microblog%2Dging+Based+on+Sentiment+Analysis+and+Coherence+Identification%2E+Journal+of+Natural+Language+Processin)
24. Cynthia Van Hee, Els Lefever, and Veronique Hoste. 2018. SemEval-2018 Task 3: Irony Detection in English Tweets. Proceedings of the 12th International Workshop on Semantic Evaluation (SemEval-2018), pages 39–50. [find](https://scholar.google.com/scholar?q=Cynthia+Van+Hee%2C+Els+Lefever%2C+and+Veronique+Hoste%2E+2018%2E+SemEval%2D2018+Task+3%3A+Irony+Detection+in+English+Tweets%2E+Proceedings+of+the+12th+International+Workshop+on+Semantic+Evaluation+%28SemEval%2D2018%29%2C+p) [our card](https://md.research.elamur.ai/free/papers/hee-2018-semeval-2018-task-3-irony/)
25. Tony Veale. 2016. Fracking Sarcasm using Neural Network Fracking Sarcasm using Neural Network. Acl, (May):161–169. [find](https://scholar.google.com/scholar?q=Tony+Veale%2E+2016%2E+Fracking+Sarcasm+using+Neural+Network+Fracking+Sarcasm+using+Neural+Network%2E+Acl%2C+%28May%29%3A161%E2%80%93169%2E) [our card](https://md.research.elamur.ai/free/papers/ghosh-2016-fracking-sarcasm-neural-network/)
26. Byron C. Wallace. 2015. Computational irony: A survey and new perspectives. Artificial Intelligence Review, 43(4):467–483. [find](https://scholar.google.com/scholar?q=Byron+C%2E+Wallace%2E+2015%2E+Computational+irony%3A+A+survey+and+new+perspectives%2E+Artificial+Intelligence+Review%2C+43%284%29%3A467%E2%80%93483%2E)
27. Chuhan Wu, Fangzhao Wu, Sixing Wu, Junxin Liu, Zhigang Yuan, and Yongfeng Huang. 2018. THU NGN at SemEval-2018 Task 3 : Tweet Irony Detection with Densely THU NGN at SemEval-2018 Task 3 : Tweet Irony Detection with Densely Connected LSTM and Multi-task Learning. (March):51–56. [find](https://scholar.google.com/scholar?q=Chuhan+Wu%2C+Fangzhao+Wu%2C+Sixing+Wu%2C+Junxin+Liu%2C+Zhigang+Yuan%2C+and+Yongfeng+Huang%2E+2018%2E+THU+NGN+at+SemEval%2D2018+Task+3+%3A+Tweet+Irony+Detection+with+Densely+THU+NGN+at+SemEval%2D2018+Task+3+%3A+Tweet+Irony+)
28. Meishan Zhang, Yue Zhang, and Guohong Fu. 2016. Tweet sarcasm detection using deep neural network. In Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers, pages 2449–2460. [find](https://scholar.google.com/scholar?q=Meishan+Zhang%2C+Yue+Zhang%2C+and+Guohong+Fu%2E+2016%2E+Tweet+sarcasm+detection+using+deep+neural+network%2E+In+Proceedings+of+COLING+2016%2C+the+26th+International+Conference+on+Computational+Linguistics%3A+Techni) [our card](https://md.research.elamur.ai/free/papers/zhang-2016-tweet-sarcasm-detection-deep-neural/)
29. The COLING 2016 Organizing Committee. [find](https://scholar.google.com/scholar?q=The+COLING+2016+Organizing+Committee%2E)