---
title: "Does Commonsense help in detecting Sarcasm? (2021)"
description: "The paper tests whether social commonsense knowledge improves sarcasm detection. 34 references, cited by 3 papers here."
authors: ["Somnath Basu Roy Chowdhury", "Snigdha Chaturvedi"]
year: 2021
venue: "Proceedings of the Second Workshop on Insights from Negative Results in NLP"
source: "https://aclanthology.org/2021.insights-1.2/"
doi: "10.18653/v1/2021.insights-1.2"
arxiv: "2109.08588"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/chowdhury-2021-does-commonsense-help-detecting-sarcasm/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/chowdhury-2021-does-commonsense-help-detecting-sarcasm/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2021](https://md.research.elamur.ai/free/en/papers/2021/page/6/) / Does Commonsense help in detecting Sarcasm?

# Does Commonsense help in detecting Sarcasm?

**Authors:** Somnath Basu Roy Chowdhury and [Snigdha Chaturvedi](https://md.research.elamur.ai/free/en/authors/snigdha-chaturvedi/) · **Year:** 2021 · **Venue:** [Proceedings of the Second Workshop on Insights from Negative Results in NLP](https://md.research.elamur.ai/free/en/venues/insights/) ([Workshop on Insights from Negative Results in NLP 2021](https://md.research.elamur.ai/free/en/venues/insights/2021/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2021.insights-1.2/) · [PDF](https://aclanthology.org/2021.insights-1.2.pdf) · [DOI 10.18653/v1/2021.insights-1.2](https://doi.org/10.18653/v1/2021.insights-1.2) · [arXiv 2109.08588](https://arxiv.org/abs/2109.08588) · [OpenAlex](https://openalex.org/W3200043970)

**Citations:** 14 citations (OpenAlex, 2026-10-14) · 3 in our library, of the 141,840 papers whose bibliographies we read

## Summary
The paper tests whether social commonsense knowledge improves sarcasm detection. For each input sentence, the authors use COMET to generate inferences about the subject: what they want (xWant) and what happens to them (xEffect). These raw fragments become full sentences by placing the subject before them and letting a pre-trained BERT fill a masked verb slot.

The model is a graph:
- the sentence and its COMET sentences are nodes, embedded with DistilBERT;
- a GraphSage network links them, and a feed-forward layer classifies the result.

The baseline is DistilBERT plus a feed-forward network. Experiments cover SemEval irony tweets, the News Headlines dataset and FigLang 2020 Reddit.

The outcome is negative. The knowledge-augmented model only matches the baseline, and its predictions largely coincide with the baseline's. Saliency and occlusion tests show the classifier mostly ignores the commonsense nodes. Error analysis suggests the inferences help mainly with irony built on polarity contrast. For non-sarcastic texts they act as distracting context.

This is a well-instrumented negative result: it shows how to check whether injected knowledge is actually used.

## Key points
- **Data:** SemEval Irony has 3833 training and 958 test samples, and FigLang 2020 Reddit has 3520 and 880. Both are balanced. Accuracy is averaged over 5 random splits, and the Reddit context was ignored.
- **Accuracy, baseline vs. the three GCN edge setups:**
  - News Headlines: 96.13% vs. 96.14–96.18%.
  - SemEval: 69.09% vs. 67.88–68.66%.
  - Reddit: 67.95% vs. 67.35–67.54%.
- **Agreement with the baseline:** the two models give identical labels on 99.5%, 91.6% and 92.7% of test items.
- **Occlusion:** removing the sentence representation shifts output confidence by 27.99%, while removing the COMET representations shifts it by only 1.38%. Without the sentence node, the COMET→input edge setup drops to 55.00%.
- **Error coverage:** among items the GCN got wrong but the baseline got right, the non-sarcastic share is 83.8%, 64.6% and 96.8%. On a SemEval subset restricted to polarity-contrast irony plus non-sarcastic items, it falls to 57.1%.

## Interesting ideas
- **Occlusion as a usage test:** To test whether an added knowledge channel is actually used, zero it out and measure how much the output confidence moves; this exposes decorative inputs that accuracy alone hides.
- **Agreement rate as a novelty check:** Comparing prediction overlap between the augmented system and the baseline is a cheap way to see whether the new component solves any new cases at all.
- **Class-conditioned error attribution:** Look only at cases where the new model fails and the old one succeeds, then ask which gold class dominates; here it showed the extra knowledge misleads on the negative (literal) class.
- **Turning fragments into sentences:** Raw knowledge-base fragments like "to save money" become full sentences by adding the subject and letting a masked LM supply the linking verb, which puts them in the same form as the input.
- **Intent-versus-outcome contrast as an irony cue:** Sarcasm built on polarity contrast often shows up as a clash between what the subject wants and what actually results, which is the one case where commonsense helped.

## For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

- **I1** (learn): Run the same occlusion on each knowledge source feeding a decision (VerbNet roles, WordNet ontology) as a standing negative control, and flag sources whose removal changes nothing.
- **I2** (learn): When a new repair rule or witness parser is added, report its label agreement with the previous version beside accuracy, so improvements that only reshuffle errors are caught early.
- **I3** (learn): Apply this lost-cases breakdown to each pipeline change to locate the input class it harms, for example literal sentences misread as ironic.
- **I4** (world): Compiled knowledge snippets could likewise be turned into explicit subject-predicate statements, so they enter the world model as ordinary clauses rather than loose phrases.
- **I5** (frames): The irony detector in prag could compare the subject's stated or inferred goal frame with the resulting state frame, and raise a polarity-contrast irony flag with an explainable reason when they conflict.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [University of North Carolina at Chapel Hill](https://md.research.elamur.ai/free/en/universities/university-of-north-carolina-at-chapel-hill/) — University, Chapel Hill, United States · 37 works · 14,332 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 9,086 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 8,889 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 9,296 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 9,206 works

🧭 Topics (2): [😊 Sentiment & emotion](https://md.research.elamur.ai/free/en/topics/sentiment-emotion/) · 608 works · [🧮 Reasoning](https://md.research.elamur.ai/free/en/topics/reasoning/) · 510 works

## Cite

Cite as: Somnath Basu Roy Chowdhury and Snigdha Chaturvedi. 2021. Does Commonsense help in detecting Sarcasm?. Proceedings of the Second Workshop on Insights from Negative Results in NLP.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{chowdhury2021does,
  title = {{Does Commonsense help in detecting Sarcasm?}},
  author = {Somnath Basu Roy Chowdhury and Snigdha Chaturvedi},
  booktitle = {Proceedings of the Second Workshop on Insights from Negative Results in NLP},
  year = {2021},
  doi = {10.18653/v1/2021.insights-1.2},
  eprint = {2109.08588},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.18653/v1/2021.insights-1.2}
}
```

- [← previous Coreference Resolution for the Biomedical Domain: A Survey](https://md.research.elamur.ai/free/en/papers/lu-2021-coreference-resolution-biomedical-domain-survey/)
- [next → Exploring Transformer Based Models to Identify Hate Speech and Offensive Content in English and Indo-Aryan Languages](https://md.research.elamur.ai/free/en/papers/banerjee-2021-exploring-transformer-based-models-identify/)
- [↑ back to papers of 2021](https://md.research.elamur.ai/free/en/papers/2021/page/6/)

## References

34 references, 20 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Aniruddha Ghosh and Tony Veale. 2017. [Magnets for sarcasm: Making sarcasm detection timely, contextual and very personal](https://md.research.elamur.ai/free/en/papers/ghosh-2017-magnets-sarcasm-making-sarcasm-detection/). In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 482–491, Copenhagen, Denmark. Association for Computational Linguistics.
2. Silvio Amir, Byron C. Wallace, Hao Lyu, Paula Carvalho, and Mário J. Silva. 2016. [Modelling context with user embeddings for sarcasm detection in social media](https://md.research.elamur.ai/free/en/papers/amir-2016-modelling-context-user-embeddings-sarcasm/). In Proceedings of The 20th SIGNLL Conference on Computational Natural Language Learning, pages 167–177, Berlin, Germany. Association for Computational Linguistics.
3. Debanjan Ghosh, Weiwei Guo, and Smaranda Muresan. 2015. [Sarcastic or not: Word embeddings to predict the literal or sarcastic meaning of words](https://md.research.elamur.ai/free/en/papers/ghosh-2015-sarcastic-or-not-word-embeddings/). In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 1003– 1012, Lisbon, Portugal. Association for Computational Linguistics. [↗](https://doi.org/10.18653/v1/d15-1116)
4. Jasmijn Bastings and Katja Filippova. 2020. [The elephant in the interpretability room: Why use attention as explanation when we have saliency methods?](https://md.research.elamur.ai/free/en/papers/bastings-2020-elephant-interpretability-room-why-use/) In Proceedings of the Third BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP, pages 149–155, Online. Association for Computational Linguistics.
5. Debanjan Ghosh, Avijit Vajpayee, and Smaranda Muresan. 2020. [A report on the 2020 sarcasm detection shared task](https://md.research.elamur.ai/free/en/papers/ghosh-2020-report-2020-sarcasm-detection-shared/). In Proceedings of the Second Workshop on Figurative Language Processing, pages 1– 11, Online. Association for Computational Linguistics. [↗](https://doi.org/10.18653/v1/2020.figlang-1.1)
6. Antoine Bosselut, Hannah Rashkin, Maarten Sap, Chaitanya Malaviya, Asli Celikyilmaz, and Yejin Choi. 2019. [COMET: Commonsense transformers for automatic knowledge graph construction](https://md.research.elamur.ai/free/en/papers/bosselut-2019-comet-commonsense-transformers-automatic-knowledge/). In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 4762–4779, Florence, Italy. Association for Computational Linguistics.
7. Raymond W Gibbs. 1986. On the psycholinguistics of sarcasm. Journal of experimental psychology: general, 115(1):3. [↗](https://doi.org/10.1037/0096-3445.115.1.3)
8. Faeze Brahman and Snigdha Chaturvedi. 2020. Modeling protagonist emotions for emotion-aware storytelling. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 5277–5294. [find](https://scholar.google.com/scholar?q=Faeze+Brahman+and+Snigdha+Chaturvedi%2E+2020%2E+Modeling+protagonist+emotions+for+emotion%2Daware+storytelling%2E+In+Proceedings+of+the+2020+Conference+on+Empirical+Methods+in+Natural+Language+Processing+%28EMN)
9. Raymond W Gibbs Jr, Raymond W Gibbs, and Herbert L Colston. 2007. Irony in language and thought: A cognitive science reader. Psychology Press. [↗](https://doi.org/10.4324/9781410616685)
10. Paula Carvalho, Luı́s Sarmento, Mário J Silva, and Eugénio De Oliveira. 2009. Clues for detecting irony in user-generated contents: oh...!! it’s” so easy”;-. In Proceedings of the 1st international CIKM workshop on Topic-sentiment analysis for mass opinion, pages 53–56. [find](https://scholar.google.com/scholar?q=Paula+Carvalho%2C+Lu%C4%B1%CC%81s+Sarmento%2C+Ma%CC%81rio+J+Silva%2C+and+Euge%CC%81nio+De+Oliveira%2E+2009%2E+Clues+for+detecting+irony+in+user%2Dgenerated+contents%3A+oh%2E%2E%2E%21%21+it%E2%80%99s%E2%80%9D+so+easy%E2%80%9D%3B%2D%2E+In+Proceedings+of+the+1st+international+)
11. Roberto González-Ibáñez, Smaranda Muresan, and Nina Wacholder. 2011. [Identifying sarcasm in Twitter: A closer look](https://md.research.elamur.ai/free/en/papers/ibanez-2011-identifying-sarcasm-twitter-closer-look/). In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, pages 581–586, Portland, Oregon, USA. Association for Computational Linguistics.
12. Tuhin Chakrabarty, Debanjan Ghosh, Smaranda Muresan, and Nanyun Peng. 2020. [r 3 : Reverse, retrieve, and rank for sarcasm generation with commonsense knowledge](https://md.research.elamur.ai/free/en/papers/chakrabarty-2020-r-textasciicircum3-reverse-retrieve-rank/). arXiv preprint arXiv:2004.13248. [↗](https://arxiv.org/abs/2004.13248)
13. William L. Hamilton, Zhitao Ying, and Jure Leskovec. 2017. [Inductive representation learning on large graphs](https://md.research.elamur.ai/free/en/papers/hamilton-2017-inductive-representation-learning-large-graphs/). In Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, December 4-9, 2017, Long Beach, CA, USA, pages 1024–1034. [↗](https://doi.org/10.48550/arxiv.1706.02216)
14. Jindong Chen, Yizhou Hu, Jingping Liu, Yanghua Xiao, and Haiyun Jiang. 2019. Deep short text classification with knowledge powered attention. In The Thirty-Third AAAI Conference on Artificial Intelligence, AAAI 2019, The Thirty-First Innovative Applications of Artificial Intelligence Conference, IAAI 2019, The Ninth AAAI Symposium on Educational Advances in Artificial Intelligence, EAAI 2019, Honolulu, Hawaii, USA, January 27 - February 1, 2019, pages 6252–6259. AAAI Press. [find](https://scholar.google.com/scholar?q=Jindong+Chen%2C+Yizhou+Hu%2C+Jingping+Liu%2C+Yanghua+Xiao%2C+and+Haiyun+Jiang%2E+2019%2E+Deep+short+text+classification+with+knowledge+powered+attention%2E+In+The+Thirty%2DThird+AAAI+Conference+on+Artificial+Intellig)
15. Aditya Joshi, Vinita Sharma, and Pushpak Bhattacharyya. 2015. [Harnessing context incongruity for sarcasm detection](https://md.research.elamur.ai/free/en/papers/joshi-2015-harnessing-context-incongruity-sarcasm-detection/). In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 2: Short Papers), pages 757–762, Beijing, China. Association for Computational Linguistics. [↗](https://doi.org/10.3115/v1/p15-2124)
16. Dmitry Davidov, Oren Tsur, and Ari Rappoport. 2010. [Semi-supervised recognition of sarcasm in Twitter and Amazon](https://md.research.elamur.ai/free/en/papers/davidov-2010-semi-supervised-recognition-sarcasm-twitter/). In Proceedings of the Fourteenth Conference on Computational Natural Language Learning, pages 107–116, Uppsala, Sweden. Association for Computational Linguistics.
17. Annervaz K M, Somnath Basu Roy Chowdhury, and Ambedkar Dukkipati. 2018. Learning beyond datasets: Knowledge graph augmented neural networks for natural language processing. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), pages 313–322, New Orleans, Louisiana. Association for Computational Linguistics. [↗](https://doi.org/10.48550/arxiv.1802.05930)
18. Daria Dzendzik, Carl Vogel, and Jennifer Foster. 2020. Q. can knowledge graphs be used to answer boolean questions? a. it’s complicated! Association for Computational Linguistics (ACL). [find](https://scholar.google.com/scholar?q=Daria+Dzendzik%2C+Carl+Vogel%2C+and+Jennifer+Foster%2E+2020%2E+Q%2E+can+knowledge+graphs+be+used+to+answer+boolean+questions%3F+a%2E+it%E2%80%99s+complicated%21+Association+for+Computational+Linguistics+%28ACL%29%2E)
19. Akira Utsumi. 2000. Verbal irony as implicit display of ironic environment: Distinguishing ironic utterances from nonirony. Journal of Pragmatics, 32(12):1777–1806. [↗](https://doi.org/10.1016/s0378-2166(99)00116-2)
20. Thomas N. Kipf and Max Welling. 2017. [Semi-supervised classification with graph convolutional networks](https://md.research.elamur.ai/free/en/papers/kipf-2016-semi-supervised-classification-graph-convolutional/). In 5th International Conference on Learning Representations, ICLR 2017, Toulon, France, April 24-26, 2017, Conference Track Proceedings. OpenReview.net. [↗](https://doi.org/10.48550/arxiv.1609.02907)
21. Roger J Kreuz and Sam Glucksberg. 1989. How to be sarcastic: The echoic reminder theory of verbal irony. Journal of experimental psychology: General, 118(4):374. [↗](https://doi.org/10.1037/0096-3445.118.4.374)
22. Cynthia Van Hee, Els Lefever, and Véronique Hoste. 2018. [SemEval-2018 task 3: Irony detection in English tweets](https://md.research.elamur.ai/free/en/papers/hee-2018-semeval-2018-task-3-irony/). In Proceedings of The 12th International Workshop on Semantic Evaluation, pages 39– 50, New Orleans, Louisiana. Association for Computational Linguistics. [↗](https://doi.org/10.18653/v1/s18-1005)
23. Stephanie Lukin and Marilyn Walker. 2013. [Really? well. apparently bootstrapping improves the performance of sarcasm and nastiness classifiers for online dialogue](https://md.research.elamur.ai/free/en/papers/lukin-2013-really-well-apparently-bootstrapping-improves/). In Proceedings of the Workshop on Language Analysis in Social Media, pages 30–40, Atlanta, Georgia. Association for Computational Linguistics.
24. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. [Attention is all you need](https://md.research.elamur.ai/free/en/papers/vaswani-2017-attention-all-need/). In Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, December 4-9, 2017, Long Beach, CA, USA, pages 5998–6008.
25. Diana Maynard and Mark Greenwood. 2014. [Who cares about sarcastic tweets? investigating the impact of sarcasm on sentiment analysis](https://md.research.elamur.ai/free/en/papers/maynard-2014-who-cares-about-sarcastic-tweets/). In Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC’14), pages 4238–4243, Reykjavik, Iceland. European Language Resources Association (ELRA). [↗](https://doi.org/10.63317/2ios4bdd5xas)
26. Byron C. Wallace, Do Kook Choe, Laura Kertz, and Eugene Charniak. 2014. [Humans require context to infer ironic intent (so computers probably do, too)](https://md.research.elamur.ai/free/en/papers/wallace-2014-humans-require-context-infer-ironic/). In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 512–516, Baltimore, Maryland. Association for Computational Linguistics. [↗](https://doi.org/10.3115/v1/p14-2084)
27. Abhijit Mishra, Diptesh Kanojia, Seema Nagar, Kuntal Dey, and Pushpak Bhattacharyya. 2016. [Harnessing cognitive features for sarcasm detection](https://md.research.elamur.ai/free/en/papers/mishra-2016-harnessing-cognitive-features-sarcasm-detection/). In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1095–1104, Berlin, Germany. Association for Computational Linguistics. [↗](https://doi.org/10.48550/arxiv.1701.05574)
28. Xiaoyan Wang, Pavan Kapanipathi, Ryan Musa, Mo Yu, Kartik Talamadupula, Ibrahim Abdelaziz, Maria Chang, Achille Fokoue, Bassem Makni, Nicholas Mattei, et al. 2019. Improving natural language inference using external knowledge in the science questions domain. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 33, pages 7208–7215. [find](https://scholar.google.com/scholar?q=Xiaoyan+Wang%2C+Pavan+Kapanipathi%2C+Ryan+Musa%2C+Mo+Yu%2C+Kartik+Talamadupula%2C+Ibrahim+Abdelaziz%2C+Maria+Chang%2C+Achille+Fokoue%2C+Bassem+Makni%2C+Nicholas+Mattei%2C+et+al%2E+2019%2E+Improving+natural+language+inference)
29. Rishabh Misra and Prahal Arora. 2019. Sarcasm detection using hybrid neural network. arXiv preprint arXiv:1908.07414. [↗](https://arxiv.org/abs/1908.07414)
30. Mingda Zhang, Keren Ye, Rebecca Hwa, and Adriana Kovashka. 2020. Story completion with explicit modeling of commonsense knowledge. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pages 376– 377. [find](https://scholar.google.com/scholar?q=Mingda+Zhang%2C+Keren+Ye%2C+Rebecca+Hwa%2C+and+Adriana+Kovashka%2E+2020%2E+Story+completion+with+explicit+modeling+of+commonsense+knowledge%2E+In+Proceedings+of+the+IEEE%2FCVF+Conference+on+Computer+Vision+and+Patt)
31. Smaranda Muresan, Roberto Gonzalez-Ibanez, Debanjan Ghosh, and Nina Wacholder. 2016. Identification of nonliteral language in social media: A case study on sarcasm. Journal of the Association for Information Science and Technology, 67(11):2725– 2737. [↗](https://doi.org/10.1002/asi.23624)
32. Fabio Petroni, Patrick Lewis, Aleksandra Piktus, Tim Rocktäschel, Yuxiang Wu, Alexander H Miller, and Sebastian Riedel. 2020. [How context affects language models’ factual predictions](https://md.research.elamur.ai/free/en/papers/petroni-2020-context-affects-language-models-factual/). arXiv preprint arXiv:2005.04611. [↗](https://arxiv.org/abs/2005.04611)
33. Ellen Riloff, Ashequl Qadir, Prafulla Surve, Lalindra De Silva, Nathan Gilbert, and Ruihong Huang. 2013. [Sarcasm as contrast between a positive sentiment and negative situation](https://md.research.elamur.ai/free/en/papers/riloff-2013-sarcasm-as-contrast-between-positive/). In Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, pages 704–714, Seattle, Washington, USA. Association for Computational Linguistics. [↗](https://doi.org/10.18653/v1/d13-1066)
34. Victor Sanh, Lysandre Debut, Julien Chaumond, and Thomas Wolf. 2019. Distilbert, a distilled version of bert: smaller, faster, cheaper and lighter. arXiv preprint arXiv:1910.01108. [↗](https://arxiv.org/abs/1910.01108)

*Updated 24 October 2026*