---
title: "Transformers on Sarcasm Detection with Context — Kumar 2020"
description: "This paper is a system description for the Sarcasm Detection with Context shared task at the Second Workshop on Figurative Language Processing (ACL 2020)."
authors: ["Amardeep Kumar", "Vivek Anand"]
year: 2020
venue: "Proceedings of the Second Workshop on Figurative Language Processing"
source: "https://aclanthology.org/2020.figlang-1.13/"
doi: "10.18653/v1/2020.figlang-1.13"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/papers/kumar-2020-transformers-sarcasm-detection-context/"
---
[Elamur Research](https://md.research.elamur.ai/) · [this page in HTML](https://research.elamur.ai/free/papers/kumar-2020-transformers-sarcasm-detection-context/)

[home](https://md.research.elamur.ai/) / [papers](https://md.research.elamur.ai/free/papers/) / [2020](https://md.research.elamur.ai/free/papers/2020/page/4/) / Transformers on Sarcasm Detection with Context

# Transformers on Sarcasm Detection with Context

**Authors:** Amardeep Kumar, Vivek Anand · **Year:** 2020 · **Venue:** Proceedings of the Second Workshop on Figurative Language Processing · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2020.figlang-1.13/) · [PDF](https://aclanthology.org/2020.figlang-1.13.pdf) · [DOI 10.18653/v1/2020.figlang-1.13](https://doi.org/10.18653/v1/2020.figlang-1.13) · [OpenAlex](https://openalex.org/W3046050780)

**Citations:** 28 citations (openalex-snapshot · 2026-10-14) · [6](#cited-by) in our library, of the 140673 papers whose bibliographies we read

## Summary
This paper is a system description for the Sarcasm Detection with Context shared task at the Second Workshop on Figurative Language Processing (ACL 2020). It asks whether earlier turns of a social-media conversation help decide if a reply is sarcastic. The authors use two balanced English corpora: Twitter conversations (Ghosh et al., 2018) and Reddit threads (Khodak et al., 2018). They lightly clean the text and build the context by joining the earlier turns from newest to oldest. They then fine-tune BERT, RoBERTa and spanBERT in two setups: one where the reply and its context form a single input, and one where they are fed as a pair. They also test three custom designs:
- an LSTM placed on top of the transformer's final hidden states
- a Siamese Transformer
- a Dual Transformer with a Bi-LSTM

The LSTM-over-Transformer model with roberta-large performs best on both corpora. Adding context clearly beats classifying the reply alone. The work matters because it shows that conversational context measurably improves sarcasm detection with pre-trained transformers.

## Key points
- **Data:** 5,000 tweets and 4,400 Reddit posts, each split evenly between SARCASM and NOT SARCASM. Every item comes in JSON with response, context and label fields.
- **Best model:** LSTM over Transformer with roberta-large reached F1 77.22 on Twitter and 69.11 on Reddit.
- **Reply only, no context:** the best F1 was 67.50 on Twitter and about 63.8 on Reddit. The paper's text gives 63.2 for Reddit, while its table shows 63.83.
- **Model comparisons:** larger checkpoints consistently beat base ones. spanbert-large outperformed both BERT variants, and roberta-large was the strongest single-input baseline. The pair-input Siamese and Dual Transformer models did not beat the best model, possibly because of limited training data.
- **Setup:** the best maximum sequence lengths were 126 tokens for Twitter and 80 for Reddit. Models were built with the Hugging Face transformers library. The authors made no comparison with prior systems, citing limited compute.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

Fields: [Linguistics & languages](https://md.research.elamur.ai/free/shelves/linguistics/)

## Cited by

Cited by

6 papers of our library cite it (of the 140673 whose bibliographies we read); 1 of them have a page here, 5 not yet.

- [A Report on the 2020 Sarcasm Detection Shared Task](https://md.research.elamur.ai/free/papers/ghosh-2020-report-2020-sarcasm-detection-shared/)

## Cite

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{kumar2020transformers,
  title = {{Transformers on Sarcasm Detection with Context}},
  author = {Kumar, Amardeep and Anand, Vivek},
  booktitle = {Proceedings of the Second Workshop on Figurative Language Processing},
  year = {2020},
  doi = {10.18653/v1/2020.figlang-1.13},
  url = {https://doi.org/10.18653/v1/2020.figlang-1.13}
}
```

- [← previous Transformers as Soft Reasoners over Language](https://md.research.elamur.ai/free/papers/clark-2020-transformers-soft-reasoners-over-language/)
- [next → Transformers: State-of-the-Art Natural Language Processing](https://md.research.elamur.ai/free/papers/wolf-2020-transformers-state-art-natural-language/)
- [↑ back to papers of 2020](https://md.research.elamur.ai/free/papers/2020/page/4/)

## References

12 references, 6 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 4171–4186, Minneapolis, Minnesota. Association for Computational Linguistics. *Table 2: Result on Reddit* Debanjan Ghosh, Alexander R. Fabbri, and Smaranda Muresan. 2018. Sarcasm analysis using conversation context. Computational Linguistics, 44(4):755– 792. [find](https://scholar.google.com/scholar?q=Jacob+Devlin%2C+Ming%2DWei+Chang%2C+Kenton+Lee%2C+and+Kristina+Toutanova%2E+2019%2E+BERT%3A+Pre%2Dtraining+of+deep+bidirectional+transformers+for+language+understanding%2E+In+Proceedings+of+the+2019+Conference+of+the+N) [our card](https://md.research.elamur.ai/free/papers/ghosh-2018-sarcasm-analysis-conversation-context/)
2. Sepp Hochreiter and Jürgen Schmidhuber. 1997. Long short-term memory. Neural Computation, 9(8):1735–1780. [↗](https://doi.org/10.1162/neco.1997.9.8.1735)
3. Aditya Joshi, Vaibhav Tripathi, Pushpak Bhattacharyya, and Mark J. Carman. 2016. Harnessing sequence labeling for sarcasm detection in dialogue from TV series ‘Friends’. In Proceedings of The 20th SIGNLL Conference on Computational Natural Language Learning, pages 146–155, Berlin, Germany. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Aditya+Joshi%2C+Vaibhav+Tripathi%2C+Pushpak+Bhattacharyya%2C+and+Mark+J%2E+Carman%2E+2016%2E+Harnessing+sequence+labeling+for+sarcasm+detection+in+dialogue+from+TV+series+%E2%80%98Friends%E2%80%99%2E+In+Proceedings+of+The+20th+SIG) [our card](https://md.research.elamur.ai/free/papers/joshi-2016-harnessing-sequence-labeling-sarcasm-detection/)
4. Mandar Joshi, Danqi Chen, Yinhan Liu, Daniel S. Weld, Luke Zettlemoyer, and Omer Levy. 2020. Spanbert: Improving pre-training by representing and predicting spans. Transactions of the Association for Computational Linguistics, 8:64–77. [↗](https://doi.org/10.1162/tacl_a_00300) [our card](https://md.research.elamur.ai/free/papers/joshi-2019-spanbert-improving-pre-training-representing/)
5. Mikhail Khodak, Nikunj Saunshi, and Kiran Vodra-halli. 2018. A large self-annotated corpus for sarcasm. In Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC-2018), Miyazaki, Japan. European Languages Resources Association (ELRA). [find](https://scholar.google.com/scholar?q=Mikhail+Khodak%2C+Nikunj+Saunshi%2C+and+Kiran+Vodra%2Dhalli%2E+2018%2E+A+large+self%2Dannotated+corpus+for+sarcasm%2E+In+Proceedings+of+the+Eleventh+International+Conference+on+Language+Resources+and+Evaluation+%28LR) [our card](https://md.research.elamur.ai/free/papers/khodak-2018-large-self-annotated-corpus-sarcasm/)
6. Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. Roberta: A robustly optimized BERT pretraining approach. CoRR, abs/1907.11692. [↗](https://arxiv.org/abs/1907.11692)
7. Jonas Mueller and Aditya Thyagarajan. 2016. Siamese recurrent architectures for learning sentence similarity. In thirtieth AAAI conference on artificial intelligence. [find](https://scholar.google.com/scholar?q=Jonas+Mueller+and+Aditya+Thyagarajan%2E+2016%2E+Siamese+recurrent+architectures+for+learning+sentence+similarity%2E+In+thirtieth+AAAI+conference+on+artificial+intelligence%2E)
8. Mike Schuster and Kuldip K Paliwal. 1997. Bidirectional recurrent neural networks. IEEE transactions on Signal Processing, 45(11):2673–2681. [↗](https://doi.org/10.1109/78.650093)
9. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Ł ukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett, editors, Advances in Neural Information Processing Systems 30, pages 5998–6008. Curran Associates, Inc. [find](https://scholar.google.com/scholar?q=Ashish+Vaswani%2C+Noam+Shazeer%2C+Niki+Parmar%2C+Jakob+Uszkoreit%2C+Llion+Jones%2C+Aidan+N+Gomez%2C+%C5%81+ukasz+Kaiser%2C+and+Illia+Polosukhin%2E+2017%2E+Attention+is+all+you+need%2E+In+I%2E+Guyon%2C+U%2E+V%2E+Luxburg%2C+S%2E+Bengio%2C+H%2E)
10. Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pier-ric Cistac, Tim Rault, R’emi Louf, Morgan Funtow-icz, and Jamie Brew. 2019. Huggingface’s transformers: State-of-the-art natural language processing. ArXiv, abs/1910.03771. [↗](https://arxiv.org/abs/1910.03771) [our card](https://md.research.elamur.ai/free/papers/wolf-2020-transformers-state-art-natural-language/)
11. Meishan Zhang, Yue Zhang, and Guohong Fu. 2016. Tweet sarcasm detection using deep neural network. In Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers, pages 2449–2460, Osaka, Japan. [find](https://scholar.google.com/scholar?q=Meishan+Zhang%2C+Yue+Zhang%2C+and+Guohong+Fu%2E+2016%2E+Tweet+sarcasm+detection+using+deep+neural+network%2E+In+Proceedings+of+COLING+2016%2C+the+26th+International+Conference+on+Computational+Linguistics%3A+Techni) [our card](https://md.research.elamur.ai/free/papers/zhang-2016-tweet-sarcasm-detection-deep-neural/)
12. The COLING 2016 Organizing Committee. [find](https://scholar.google.com/scholar?q=The+COLING+2016+Organizing+Committee%2E)