---
title: "Exploring the Realization of Irony in Twitter — Hee 2016"
description: "Irony in social media can turn an apparently positive message into a negative one, which misleads sentiment analysis systems."
authors: ["Cynthia Van Hee", "Els Lefever", "Véronique Hoste"]
year: 2016
venue: "Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC'16)"
source: "https://aclanthology.org/L16-1283/"
licence: "CC BY-NC 4.0 (ELRA, LREC proceedings)"
licence_url: "https://creativecommons.org/licenses/by-nc/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/papers/hee-2016-exploring-realization-irony-twitter-data/"
---
[Elamur Research](https://md.research.elamur.ai/) · [this page in HTML](https://research.elamur.ai/free/papers/hee-2016-exploring-realization-irony-twitter-data/)

[home](https://md.research.elamur.ai/) / [papers](https://md.research.elamur.ai/free/papers/) / [2016](https://md.research.elamur.ai/free/papers/2016/page/2/) / Exploring the Realization of Irony in Twitter Data

# Exploring the Realization of Irony in Twitter Data

**Authors:** [Cynthia Van Hee](https://md.research.elamur.ai/free/authors/cynthia-van-hee/), [Els Lefever](https://md.research.elamur.ai/free/authors/els-lefever/), [Véronique Hoste](https://md.research.elamur.ai/free/authors/veronique-hoste/) · **Year:** 2016 · **Venue:** Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC'16) · **Licence:** [CC BY-NC 4.0 (ELRA, LREC proceedings)](https://creativecommons.org/licenses/by-nc/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/L16-1283/) · [PDF](https://aclanthology.org/L16-1283.pdf) · [OpenAlex](https://openalex.org/W2394787500)

**Citations:** 29 citations (openalex-snapshot · 2026-10-14) · [19](#cited-by) in our library, of the 140673 papers whose bibliographies we read

## Summary
Irony in social media can turn an apparently positive message into a negative one, which misleads sentiment analysis systems. Earlier computational work mostly relied on author-supplied hashtags such as #irony as noisy labels. No guidelines existed for finer-grained manual annotation. The authors address this gap with a Twitter corpus for English and Dutch, gathered via irony-related hashtags, and a new annotation scheme.

The scheme treats irony as an evaluative utterance whose polarity changes between the literal and the intended meaning. It deliberately does not separate irony from sarcasm. Annotators label each tweet's irony status, note extra properties of ironic tweets, and mark the contrasting evaluations in brat. An inter-annotator agreement study was used to validate the guidelines.

Corpus statistics show a typical pattern: ironic tweets most often state something positive while meaning something negative. This supports the hypothesis that contrasting evaluations are a useful cue for detecting irony. The human-labeled data reduces hashtag noise and is intended as training material for automatic irony detection.

## Key points
- The corpus contains roughly 3,000 tweets in English and Dutch, collected with #irony, #sarcasm and #not (Dutch: #ironie, #sarcasme, #not).
- Tweets are labeled as ironic, possibly ironic or not ironic. For ironic tweets, annotators record:
  - whether a hashtag is needed to notice the irony,
  - harshness on a 0–1 scale,
  - the contrasting evaluation spans.
- The authors prefer "polarity change" over "polarity reversal" so that ironic hyperbole and understatement are also covered.
- In the ironic data, literal sentiment is mostly positive and intended sentiment negative. Dutch ironic tweets contained more evaluations than English ones.
- The guidelines are published as LT3 technical report 15-02 (Van Hee et al., 2015). The work was funded within the AMiCA project (IWT SBO 120007).

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

Fields: [Linguistics & languages](https://md.research.elamur.ai/free/shelves/linguistics/)

## Cited by

Cited by

19 papers of our library cite it (of the 140673 whose bibliographies we read); 6 of them have a page here, 13 not yet.

- [Application and Analysis of a Multi-layered Scheme for Irony on the Italian Twitter Corpus TWITTIRÒ](https://md.research.elamur.ai/free/papers/cignarella-2018-application-analysis-multi-layered-scheme/)
- [EPIC: Multi-Perspective Annotation of a Corpus of Irony](https://md.research.elamur.ai/free/papers/frenda-2023-epic-multi-perspective-annotation-corpus/)
- [Monday mornings are my fave :) #not Exploring the Automatic Recognition of Irony in English tweets](https://md.research.elamur.ai/free/papers/hee-2016-monday-mornings-are-my-fave/)
- [We Usually Don't Like Going to the Dentist: Using Common Sense to Detect Irony on Twitter](https://md.research.elamur.ai/free/papers/hee-2018-we-usually-don-t-like/)
- [THU_NGN at SemEval-2018 Task 1: Fine-grained Tweet Sentiment Intensity Analysis with Attention CNN-LSTM](https://md.research.elamur.ai/free/papers/wu-2018-thu-ngn-semeval-2018-task/)
- [A Survey in Automatic Irony Processing: Linguistic, Cognitive, and Multi-X Perspectives](https://md.research.elamur.ai/free/papers/zeng-2022-survey-automatic-irony-processing-linguistic/)

## Cite

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{hee2016exploring,
  title = {{Exploring the Realization of Irony in Twitter Data}},
  author = {Hee, Cynthia Van and Lefever, Els and Hoste, Véronique},
  booktitle = {Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC'16)},
  year = {2016},
  doi = {10.63317/2u588fkk7io3},
  url = {https://doi.org/10.63317/2u588fkk7io3}
}
```

- [← previous Exploiting Source-side Monolingual Data in Neural Machine Translation](https://md.research.elamur.ai/free/papers/zhang-2016-exploiting-source-side-monolingual-data/)
- [next → Exploring the steps of Verb Phrase Ellipsis](https://md.research.elamur.ai/free/papers/aclbulk-2016-liu-exploring-steps-verb-phrase-ellipsis/)
- [↑ back to papers of 2016](https://md.research.elamur.ai/free/papers/2016/page/2/)

## References

27 references, 5 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Attardo, S., Eisterhold, J., Hay, J., and Poggi, I. (2003). Multimodal markers of irony and sarcasm. Humor-international Journal of Humor Research, 16:243–260. [find](https://scholar.google.com/scholar?q=Attardo%2C+S%2E%2C+Eisterhold%2C+J%2E%2C+Hay%2C+J%2E%2C+and+Poggi%2C+I%2E+%282003%29%2E+Multimodal+markers+of+irony+and+sarcasm%2E+Humor%2Dinternational+Journal+of+Humor+Research%2C+16%3A243%E2%80%93260%2E)
2. Attardo, S. (2000). Irony as relevant inappropriateness. Journal of Pragmatics, 32(6):793–826. [find](https://scholar.google.com/scholar?q=Attardo%2C+S%2E+%282000%29%2E+Irony+as+relevant+inappropriateness%2E+Journal+of+Pragmatics%2C+32%286%29%3A793%E2%80%93826%2E)
3. Barbieri, F. and Saggion, H. (2014). Modelling Irony in Twitter, Feature Analysis and Evaluation. In LREC, Reykjavik, Iceland. [find](https://scholar.google.com/scholar?q=Barbieri%2C+F%2E+and+Saggion%2C+H%2E+%282014%29%2E+Modelling+Irony+in+Twitter%2C+Feature+Analysis+and+Evaluation%2E+In+LREC%2C+Reykjavik%2C+Iceland%2E) [our card](https://md.research.elamur.ai/free/papers/barbieri-2014-modelling-irony-twitter-feature-analysis/)
4. Burgers, C. (2010). Verbal irony: Use and effects in written discourse. UB Nijmegen [Host]. [find](https://scholar.google.com/scholar?q=Burgers%2C+C%2E+%282010%29%2E+Verbal+irony%3A+Use+and+effects+in+written+discourse%2E+UB+Nijmegen+%5BHost%5D%2E)
5. Camp, E. (2012). Sarcasm, Pretense, and The Semantics/Pragmatics Distinction. Nous, 46(4):587–634. [find](https://scholar.google.com/scholar?q=Camp%2C+E%2E+%282012%29%2E+Sarcasm%2C+Pretense%2C+and+The+Semantics%2FPragmatics+Distinction%2E+Nous%2C+46%284%29%3A587%E2%80%93634%2E)
6. Carvalho, P., Sarmento, L., Silva, M. J., and de Oliveira, E. (2009). Clues for Detecting Irony in User-generated Contents: Oh...!! It’s “So Easy” ;-). In Proceedings of the 1st International CIKM Workshop on Topic-sentiment Analysis for Mass Opinion, TSA ’09, pages 53–56, New York, NY, USA. [find](https://scholar.google.com/scholar?q=Carvalho%2C+P%2E%2C+Sarmento%2C+L%2E%2C+Silva%2C+M%2E+J%2E%2C+and+de+Oliveira%2C+E%2E+%282009%29%2E+Clues+for+Detecting+Irony+in+User%2Dgenerated+Contents%3A+Oh%2E%2E%2E%21%21+It%E2%80%99s+%E2%80%9CSo+Easy%E2%80%9D+%3B%2D%29%2E+In+Proceedings+of+the+1st+International+CIKM+Wor)
7. Clift, R. (1999). Irony in conversation. Language in Society, 28(4):523–553. [find](https://scholar.google.com/scholar?q=Clift%2C+R%2E+%281999%29%2E+Irony+in+conversation%2E+Language+in+Society%2C+28%284%29%3A523%E2%80%93553%2E)
8. Cohen, J. (1960). A coefficient of agreement for nominal scales. Educational and Psychological Measurement, 20:37–46. Curcó, C. (2007). Irony: Negation, echo, and metarepre-sentation. Irony in language and thought, pages 269– 296. [find](https://scholar.google.com/scholar?q=Cohen%2C+J%2E+%281960%29%2E+A+coefficient+of+agreement+for+nominal+scales%2E+Educational+and+Psychological+Measurement%2C+20%3A37%E2%80%9346%2E+Curco%CC%81%2C+C%2E+%282007%29%2E+Irony%3A+Negation%2C+echo%2C+and+metarepre%2Dsentation%2E+Irony+in+langua)
9. Davidov, D., Tsur, O., and Rappoport, A. (2010). Semi-supervised Recognition of Sarcastic Sentences in Twitter and Amazon. In Proceedings of the Fourteenth Conference on Computational Natural Language Learning, CoNLL ’10, pages 107–116, Stroudsburg, PA, USA. Association for Computational Linguistics… [find](https://scholar.google.com/scholar?q=Davidov%2C+D%2E%2C+Tsur%2C+O%2E%2C+and+Rappoport%2C+A%2E+%282010%29%2E+Semi%2Dsupervised+Recognition+of+Sarcastic+Sentences+in+Twitter+and+Amazon%2E+In+Proceedings+of+the+Fourteenth+Conference+on+Computational+Natural+Language) [our card](https://md.research.elamur.ai/free/papers/davidov-2010-semi-supervised-recognition-sarcasm-twitter/)
10. Giora, R. (1995). On irony and negation. Discourse Processes, 19(2):239–264. [find](https://scholar.google.com/scholar?q=Giora%2C+R%2E+%281995%29%2E+On+irony+and+negation%2E+Discourse+Processes%2C+19%282%29%3A239%E2%80%93264%2E)
11. Grice, H. P. (1975). Logic and Conversation. In Peter Cole et al., editors, Syntax and Semantics: Vol. 3, pages 41– 58. [find](https://scholar.google.com/scholar?q=Grice%2C+H%2E+P%2E+%281975%29%2E+Logic+and+Conversation%2E+In+Peter+Cole+et+al%2E%2C+editors%2C+Syntax+and+Semantics%3A+Vol%2E+3%2C+pages+41%E2%80%93+58%2E)
12. Academic Press, New York. Grice, H. P. (1978). Further Notes on Locgic and Conversation. In Peter Cole, editor, Syntax and Semantics: Vol. 9, pages 113–127. [find](https://scholar.google.com/scholar?q=Academic+Press%2C+New+York%2E+Grice%2C+H%2E+P%2E+%281978%29%2E+Further+Notes+on+Locgic+and+Conversation%2E+In+Peter+Cole%2C+editor%2C+Syntax+and+Semantics%3A+Vol%2E+9%2C+pages+113%E2%80%93127%2E)
13. Academic Press, New York. Kreuz, R. J. and Glucksberg, S. (1989). How to be sarcastic: The echoic reminder theory of verbal irony. Journal of Experimental Psychology: General, 118(4):374. [find](https://scholar.google.com/scholar?q=Academic+Press%2C+New+York%2E+Kreuz%2C+R%2E+J%2E+and+Glucksberg%2C+S%2E+%281989%29%2E+How+to+be+sarcastic%3A+The+echoic+reminder+theory+of+verbal+irony%2E+Journal+of+Experimental+Psychology%3A+General%2C+118%284%29%3A374%2E)
14. Kunneman, F., Liebrecht, C., van Mulken, M., and van den Bosch, A. (2015). Signaling sarcasm: From hyperbole to hashtag. Information Processing & Management, 51(4):500–509. [find](https://scholar.google.com/scholar?q=Kunneman%2C+F%2E%2C+Liebrecht%2C+C%2E%2C+van+Mulken%2C+M%2E%2C+and+van+den+Bosch%2C+A%2E+%282015%29%2E+Signaling+sarcasm%3A+From+hyperbole+to+hashtag%2E+Information+Processing+%26+Management%2C+51%284%29%3A500%E2%80%93509%2E)
15. Lee, Christopher J.and Albert N. Katz, A. N. (1998). The Differential Role of Ridicule in Sarcasm and Irony. Metaphor and Symbol, 13(1):1–15. [find](https://scholar.google.com/scholar?q=Lee%2C+Christopher+J%2Eand+Albert+N%2E+Katz%2C+A%2E+N%2E+%281998%29%2E+The+Differential+Role+of+Ridicule+in+Sarcasm+and+Irony%2E+Metaphor+and+Symbol%2C+13%281%29%3A1%E2%80%9315%2E)
16. Lucariello, J. (1994). Situational Irony: A Concept of Events Gone Awry. Journal of Experimental Psychology: General, 123(2):129–145. [find](https://scholar.google.com/scholar?q=Lucariello%2C+J%2E+%281994%29%2E+Situational+Irony%3A+A+Concept+of+Events+Gone+Awry%2E+Journal+of+Experimental+Psychology%3A+General%2C+123%282%29%3A129%E2%80%93145%2E)
17. Maynard, D. and Greenwood, M. (2014). Who cares about Sarcastic Tweets? Investigating the Impact of Sarcasm on Sentiment Analysis. In Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC’14), Reykjavik, Iceland. [find](https://scholar.google.com/scholar?q=Maynard%2C+D%2E+and+Greenwood%2C+M%2E+%282014%29%2E+Who+cares+about+Sarcastic+Tweets%3F+Investigating+the+Impact+of+Sarcasm+on+Sentiment+Analysis%2E+In+Proceedings+of+the+Ninth+International+Conference+on+Language+Reso) [our card](https://md.research.elamur.ai/free/papers/maynard-2014-who-cares-about-sarcastic-tweets/)
18. McQuarrie, E. F. and Mick, D. G. (1996). Figures of Rhetoric in Advertising Language. Journal of Consumer Research, 22(4):424–438. [find](https://scholar.google.com/scholar?q=McQuarrie%2C+E%2E+F%2E+and+Mick%2C+D%2E+G%2E+%281996%29%2E+Figures+of+Rhetoric+in+Advertising+Language%2E+Journal+of+Consumer+Research%2C+22%284%29%3A424%E2%80%93438%2E)
19. Reyes, A., Rosso, P., and Veale, T. (2013). A multidimensional approach for detecting irony in Twitter. Language Resources and Evaluation, pages 238–269. [find](https://scholar.google.com/scholar?q=Reyes%2C+A%2E%2C+Rosso%2C+P%2E%2C+and+Veale%2C+T%2E+%282013%29%2E+A+multidimensional+approach+for+detecting+irony+in+Twitter%2E+Language+Resources+and+Evaluation%2C+pages+238%E2%80%93269%2E)
20. Riloff, E., Qadir, A., Surve, P., Silva, L. D., Gilbert, N., and Huang, R. (2013). Sarcasm as Contrast between a Positive Sentiment and Negative Situation. In EMNLP, pages 704–714. [find](https://scholar.google.com/scholar?q=Riloff%2C+E%2E%2C+Qadir%2C+A%2E%2C+Surve%2C+P%2E%2C+Silva%2C+L%2E+D%2E%2C+Gilbert%2C+N%2E%2C+and+Huang%2C+R%2E+%282013%29%2E+Sarcasm+as+Contrast+between+a+Positive+Sentiment+and+Negative+Situation%2E+In+EMNLP%2C+pages+704%E2%80%93714%2E) [our card](https://md.research.elamur.ai/free/papers/prag-2013-riloff-sarcasm-as-contrast-between-positive/)
21. ACL. Searle, J. R. (1978). Literal meaning. Erkenntnis, 13(1):207–224. [find](https://scholar.google.com/scholar?q=ACL%2E+Searle%2C+J%2E+R%2E+%281978%29%2E+Literal+meaning%2E+Erkenntnis%2C+13%281%29%3A207%E2%80%93224%2E)
22. Shelley, C. (2001). The bicoherence theory of situational irony. Cognitive Science, 25(5):775–818. [find](https://scholar.google.com/scholar?q=Shelley%2C+C%2E+%282001%29%2E+The+bicoherence+theory+of+situational+irony%2E+Cognitive+Science%2C+25%285%29%3A775%E2%80%93818%2E)
23. Sperber, D. and Wilson, D. (1981). Irony and the Use - Mention Distinction. In Peter Cole, editor, Radical Pragmatics, pages 295–318. [find](https://scholar.google.com/scholar?q=Sperber%2C+D%2E+and+Wilson%2C+D%2E+%281981%29%2E+Irony+and+the+Use+%2D+Mention+Distinction%2E+In+Peter+Cole%2C+editor%2C+Radical+Pragmatics%2C+pages+295%E2%80%93318%2E)
24. Academic Press, New York. Utsumi, A. (1996). A Unified Theory of Irony and Its Computational Formalization. In Proceedings of the 16th Conference on Computational Linguistics - Volume 2, COLING ’96, pages 962–967, Stroudsburg, PA, USA. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Academic+Press%2C+New+York%2E+Utsumi%2C+A%2E+%281996%29%2E+A+Unified+Theory+of+Irony+and+Its+Computational+Formalization%2E+In+Proceedings+of+the+16th+Conference+on+Computational+Linguistics+%2D+Volume+2%2C+COLING+%E2%80%9996%2C+p) [our card](https://md.research.elamur.ai/free/papers/utsumi-1996-unified-theory-irony-its-computational/)
25. Van Hee, C., Lefever, E., and Hoste, V. (2015). Guidelines for Annotating Irony in Social Media Text, version 1.0. Technical Report LT3 15-02, LT3, Language and Translation Technology Team–Ghent University. [find](https://scholar.google.com/scholar?q=Van+Hee%2C+C%2E%2C+Lefever%2C+E%2E%2C+and+Hoste%2C+V%2E+%282015%29%2E+Guidelines+for+Annotating+Irony+in+Social+Media+Text%2C+version+1%2E0%2E+Technical+Report+LT3+15%2D02%2C+LT3%2C+Language+and+Translation+Technology+Team%E2%80%93Ghent+Unive)
26. Veale, T. and Hao, Y. (2010). Detecting Ironic Intent in Creative Comparisons. In Proceedings of the 2010 Conference on ECAI 2010: 19th European Conference on Artificial Intelligence, pages 765–770, Amsterdam, The Netherlands, The Netherlands. [find](https://scholar.google.com/scholar?q=Veale%2C+T%2E+and+Hao%2C+Y%2E+%282010%29%2E+Detecting+Ironic+Intent+in+Creative+Comparisons%2E+In+Proceedings+of+the+2010+Conference+on+ECAI+2010%3A+19th+European+Conference+on+Artificial+Intelligence%2C+pages+765%E2%80%93770%2C+A)
27. IOS Press. Wilson, D. and Sperber, D. (1992). On verbal irony. Lingua, 87(1):53–76. [find](https://scholar.google.com/scholar?q=IOS+Press%2E+Wilson%2C+D%2E+and+Sperber%2C+D%2E+%281992%29%2E+On+verbal+irony%2E+Lingua%2C+87%281%29%3A53%E2%80%9376%2E)