---
title: "A Transformer Approach to Contextual Sarcasm Detection in"
description: "The paper tackles automatic sarcasm detection in tweets for the FigLang 2020 shared task, where… 17 references, cited by 7 papers here."
authors: ["Hunter Gregory", "Steven Li", "Pouya Mohammadi", "Natalie Tarn", "Rachel Draelos", "Cynthia Rudin"]
year: 2020
venue: "Proceedings of the Second Workshop on Figurative Language Processing"
source: "https://aclanthology.org/2020.figlang-1.37/"
doi: "10.18653/v1/2020.figlang-1.37"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/gregory-2020-transformer-approach-contextual-sarcasm-detection/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/gregory-2020-transformer-approach-contextual-sarcasm-detection/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2020](https://md.research.elamur.ai/free/en/papers/2020/page/6/) / A Transformer Approach to Contextual Sarcasm Detection in Twitter

# A Transformer Approach to Contextual Sarcasm Detection in Twitter

**Authors:** Hunter Gregory, Steven Li, Pouya Mohammadi, Natalie Tarn, Rachel Draelos and [Cynthia Rudin](https://md.research.elamur.ai/free/en/authors/cynthia-rudin/) · **Year:** 2020 · **Venue:** [Proceedings of the Second Workshop on Figurative Language Processing](https://md.research.elamur.ai/free/en/venues/figlang/) ([Workshop on Figurative Language Processing 2020](https://md.research.elamur.ai/free/en/venues/figlang/2020/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2020.figlang-1.37/) · [PDF](https://aclanthology.org/2020.figlang-1.37.pdf) · [DOI 10.18653/v1/2020.figlang-1.37](https://doi.org/10.18653/v1/2020.figlang-1.37) · [OpenAlex](https://openalex.org/W3046053481)

**Citations:** 31 citations (OpenAlex, 2026-10-14) · [7](https://md.research.elamur.ai/free/en/papers/gregory-2020-transformer-approach-contextual-sarcasm-detection/cited-by/) in our library, of the 141,840 papers whose bibliographies we read

## Summary
The paper tackles automatic sarcasm detection in tweets for the FigLang 2020 shared task, where each reply comes with its conversation context. The authors used the workshop's balanced Twitter (5,000 items) and Reddit (4,400 items) sets. They cleaned duplicates and repaired apostrophe spacing, slang and hashtags (via ekphrasis), and added hand-made cues such as all-caps words, quotation marks, exclamation counts, letter elongation, TextBlob sentiment, emoji sentiment and a keyword-based political flag. Their baseline followed Ghosh et al. (2018): LSTMs with attention over context and reply. They also tried GRUs, a stacked ensemble and frozen pretrained transformers. The best system combined BERT, RoBERTa, XLNet, RoBERTa-large and ALBERT without fine-tuning. Each model learned a weighting of its hidden layers, and the result was joined with the hand features before a dense classifier. It reached 0.756 F1-macro on the test set, against 0.669 for the LSTM baseline. The work matters mainly for its candid diagnosis of a train/test distribution shift and of spurious word cues in small sarcasm corpora.

## Key points
- Data: the Twitter set was labelled through #sarcasm/#sarcastic hashtags. Both test sets held 1,800 items, and about 300 duplicate Twitter observations were dropped. Only Twitter was targeted, with Reddit used as auxiliary data.
- Test F1-macro: transformer ensemble 0.756, solo RoBERTa 0.733, BERT embeddings + LSTM 0.692, stacked Logistic Regression ensemble 0.686, LSTM with attention 0.669.
- Context handling: encoding context and reply separately and concatenating the outputs beat joint input, which had to cut the front of long contexts.
- Unidirectional vs bidirectional LSTMs, GRU vs LSTM, and attention vs no attention made little difference. LDA topics were nearly useless, at about 53% training accuracy.
- Large validation-to-test drops occurred (e.g. 0.767 to 0.669 for the LSTM). Feature distributions also shifted: the political share differed by about 0.08 between train and test.

## Interesting ideas
- Lexical cues that look highly predictive in a small training set were checked against an independent 65,000-tweet corpus collected the same way (Ptáček et al., 2014), and many patterns failed. This cheaply exposes dataset artifacts before they enter a model.
- Train/test covariate shift was diagnosed by tabulating simple interpretable features on both splits rather than just reporting the score gap. This explains why validation numbers overpromise.
- Context and the reply to it were encoded separately and then combined. This keeps long context intact instead of truncating it to fit a single input window.
- Out-of-domain batches (Reddit) were down-weighted in a weighted binary cross-entropy. The model thus absorbs general sarcasm signals while staying tuned to the target domain.
- Frozen pretrained encoders were used through a learned weighting of their hidden layers, then combined with transparent hand-crafted cues (caps, quotes, exclamations, sentiment contrast).

## For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

- **I1** (learn): before trusting any induced repair rule or cue learned from silver data, re-measure its statistics on a separately sourced corpus as a negative control.
- **I2** (learn): compare distributions of tree and frame features between training and evaluation sets so that score drops come with an explicit cause.
- **I3** (frames): irony detection in prag can analyse the prior utterance and the reaction as separate frame structures and then look for a clash between them.
- **I4** (tree): when gparse trains on mixed treebanks or converted UD dialects, the loss weight of foreign-style material could be lowered relative to the target treebank.
- **I5** (frames): explicit typographic and sentiment cues can serve as explainable irony evidence next to frames, so a sarcasm verdict can point to the cues that triggered it.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Duke University](https://md.research.elamur.ai/free/en/universities/duke-university/) — University, Durham, United States · 18 works · 14,944 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 9,086 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 8,889 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 9,296 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 9,206 works

🧭 Topics (1): [😊 Sentiment & emotion](https://md.research.elamur.ai/free/en/topics/sentiment-emotion/) · 608 works

## Cite

Cite as: Hunter Gregory, Steven Li, Pouya Mohammadi, Natalie Tarn, Rachel Draelos and Cynthia Rudin. 2020. A Transformer Approach to Contextual Sarcasm Detection in Twitter. Proceedings of the Second Workshop on Figurative Language Processing.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{gregory2020transformer,
  title = {{A Transformer Approach to Contextual Sarcasm Detection in Twitter}},
  author = {Hunter Gregory, Steven Li, Pouya Mohammadi, Natalie Tarn, Rachel Draelos and Cynthia Rudin},
  booktitle = {Proceedings of the Second Workshop on Figurative Language Processing},
  year = {2020},
  doi = {10.18653/v1/2020.figlang-1.37},
  url = {https://doi.org/10.18653/v1/2020.figlang-1.37}
}
```

- [← previous STARC: Structured Annotations for Reading Comprehension](https://md.research.elamur.ai/free/en/papers/berzak-2020-starc-structured-annotations-reading-comprehension/)
- [next → All-in-One: A Deep Attentive Multi-task Learning Framework for Humour, Sarcasm, Offensive, Motivation, and Sentiment on Memes](https://md.research.elamur.ai/free/en/papers/chauhan-2020-all-one-deep-attentive-multi/)
- [↑ back to papers of 2020](https://md.research.elamur.ai/free/en/papers/2020/page/6/)

## References

17 references, 11 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Christos Baziotis, Nikos Pelekis, and Christos Doulk-eridis. 2017. [Datastories at semeval-2017 task 4: Deep LSTM with attention for message-level and topic-based sentiment analysis](https://md.research.elamur.ai/free/en/papers/baziotis-2017-datastories-semeval-2017-task-4/). In Proceedings of the 11th International Workshop on Semantic Evaluation (SemEval-2017), pages 747–754, Vancouver, Canada. Association for Computational Linguistics.
2. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. [BERT: Pre-training of deep bidirectional transformers for language understanding](https://md.research.elamur.ai/free/en/papers/devlin-2019-bert-pre-training-deep-bidirectional/). In Proceedings of NAACL-HLT 2019, pages 4171–4186.
3. Aniruddha Ghosh and Tony Veale. 2017. [Magnets for sarcasm: Making sarcasm detection timely, contextual and very personal](https://md.research.elamur.ai/free/en/papers/ghosh-2017-magnets-sarcasm-making-sarcasm-detection/). In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 482–491. [↗](https://doi.org/10.18653/v1/d17-1050)
4. Debanjan Ghosh, Alexander R. Fabbri, and Smaranda Muresan. 2018. [Sarcasm analysis using conversation context](https://md.research.elamur.ai/free/en/papers/ghosh-2018-sarcasm-analysis-conversation-context/). Computational Linguistics, 44(4):755– 792. [↗](https://doi.org/10.1162/coli_a_00336)
5. Edouard Grave, Piotr Bojanowski, Prakhar Gupta, Armand Joulin, and Tomas Mikolov. 2018. [Learning word vectors for 157 languages](https://md.research.elamur.ai/free/en/papers/grave-2018-learning-word-vectors-157-languages/). In Proceedings of the International Conference on Language Resources and Evaluation (LREC 2018).
6. Klaus Greff, Rupesh K Srivastava, Jan Koutnı́k, Bas R Steunebrink, and Jürgen Schmidhuber. 2016. LSTM: A search space odyssey. IEEE transactions on neural networks and learning systems, 28(10):2222–2232. [↗](https://doi.org/10.1109/tnnls.2016.2582924)
7. S. Kayalvizhi, D Thenmozhi, B Senthil Kumar, and Chandrabose Aravindan. 2019. SSN NLP@IDAT-FIRE-2019: Irony detection in arabic tweets using deep learning and features-based approaches. In FIRE. [find](https://scholar.google.com/scholar?q=S%2E+Kayalvizhi%2C+D+Thenmozhi%2C+B+Senthil+Kumar%2C+and+Chandrabose+Aravindan%2E+2019%2E+SSN+NLP%40IDAT%2DFIRE%2D2019%3A+Irony+detection+in+arabic+tweets+using+deep+learning+and+features%2Dbased+approaches%2E+In+FIRE%2E)
8. Mikhail Khodak, Nikunj Saunshi, and Kiran Vodrahalli. 2017. [A large self-annotated corpus for sarcasm](https://md.research.elamur.ai/free/en/papers/khodak-2018-large-self-annotated-corpus-sarcasm/). arXiv preprint arXiv:1704.05579. [↗](https://arxiv.org/abs/1704.05579)
9. Petra Kralj Novak, Jasmina Smailović, Borut Sluban, and Igor Mozetič. 2015. Sentiment of emojis. PLoS ONE, 10(12):e0144296. [↗](https://doi.org/10.1371/journal.pone.0144296)
10. Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. RoBERTa: A robustly optimized BERT pretraining approach. arXiv preprint arXiv:1907.11692. [↗](https://arxiv.org/abs/1907.11692)
11. Steven Loria. 2018. textblob documentation. Release 0.15, 2. [find](https://scholar.google.com/scholar?q=Steven+Loria%2E+2018%2E+textblob+documentation%2E+Release+0%2E15%2C+2%2E)
12. Matthew Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke Zettlemoyer. 2018. [Deep contextualized word representations](https://md.research.elamur.ai/free/en/papers/peters-2018-deep-contextualized-word-representations/). In NAACL.
13. Rolandos Alexandros Potamias, Georgios Siolas, and Andreas Georgios Stafylopatis. 2019. [A transformer-based approach to irony and sarcasm detection](https://md.research.elamur.ai/free/en/papers/potamias-2019-transformer-based-approach-irony-sarcasm/). [↗](https://doi.org/10.1007/s00521-020-05102-3)
14. Tomáš Ptáček, Ivan Habernal, and Jun Hong. 2014. [Sarcasm detection on czech and english twitter](https://md.research.elamur.ai/free/en/papers/ptacek-2014-sarcasm-detection-czech-english-twitter/). pages 213–223.
15. Hong Wang, Qingsong Xu, and Lifeng Zhou. 2015. Large unbalanced credit scoring using lasso-logistic regression ensemble. PloS one, 10(2). [↗](https://doi.org/10.1371/journal.pone.0117844)
16. Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pier-ric Cistac, Tim Rault, R’emi Louf, Morgan Funtow-icz, and Jamie Brew. 2019. Huggingface’s [transformers: State-of-the-art natural language processing](https://md.research.elamur.ai/free/en/papers/wolf-2020-transformers-state-art-natural-language/). ArXiv, abs/1910.03771. [↗](https://arxiv.org/abs/1910.03771)
17. Zichao Yang, Diyi Yang, Chris Dyer, Xiaodong He, Alex Smola, and Eduard Hovy. 2016. [Hierarchical attention networks for document classification](https://md.research.elamur.ai/free/en/papers/yang-2016-hierarchical-attention-networks-document-classification/). In Proceedings of the 2016 conference of the North American chapter of the association for computational linguistics: human language technologies, pages 1480–1489.

*Updated 24 October 2026*