--- title: "connotation_clashers at SemEval-2022 Task 6: The effect — paper by Hantsch and Chkroun (2022)" description: "The paper tests whether sentiment signals help a classifier detect sarcasm in tweets. 10 references." authors: ["Patrick Hantsch", "Nadav Chkroun"] year: 2022 venue: "Proceedings of the 16th International Workshop on Semantic Evaluation (SemEval-2022)" source: "https://aclanthology.org/2022.semeval-1.132/" doi: "10.18653/v1/2022.semeval-1.132" licence: "CC BY 4.0 (ACL Anthology)" licence_url: "https://creativecommons.org/licenses/by/4.0/" summary_licence: "Elamur Cosmic License" canonical: "https://research.elamur.ai/free/en/papers/hantsch-2022-connotation-clashers-semeval-2022-task/" --- [Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/hantsch-2022-connotation-clashers-semeval-2022-task/) [home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2022](https://md.research.elamur.ai/free/en/papers/2022/page/6/) / connotation\_clashers at SemEval-2022 Task 6: The effect of sentiment analysis on sarcasm detection # connotation\_clashers at SemEval-2022 Task 6: The effect of sentiment analysis on sarcasm detection **Authors:** Patrick Hantsch and Nadav Chkroun · **Year:** 2022 · **Venue:** [Proceedings of the 16th International Workshop on Semantic Evaluation (SemEval-2022)](https://md.research.elamur.ai/free/en/venues/semeval/) ([SemEval 2022](https://md.research.elamur.ai/free/en/venues/semeval/2022/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/) **Links:** [Publisher's page](https://aclanthology.org/2022.semeval-1.132/) · [PDF](https://aclanthology.org/2022.semeval-1.132.pdf) · [DOI 10.18653/v1/2022.semeval-1.132](https://doi.org/10.18653/v1/2022.semeval-1.132) · [OpenAlex](https://openalex.org/W4287891165) **Citations:** 2 citations (OpenAlex, 2026-10-14) ## Summary The paper tests whether sentiment signals help a classifier detect sarcasm in tweets. The idea is that sarcasm often pairs a positive word with a negative situation, as in loving that one got ill. The authors entered only subtask A (binary sarcastic vs. non-sarcastic) of SemEval-2022 Task 6, with English and Arabic tweet data. They built two pipelines and ran each with and without sentiment: - **CNN:** a Keras model with its own learned embeddings. The sentiment variant was first pretrained on IMDB movie reviews and then fine-tuned on sarcasm. - **SVM:** a model over tf-idf word n-grams. The sentiment variant received extra one-hot features built from binned Flair sentiment predictions. Adding sentiment slightly hurt both pipelines. The SVM without sentiment scored best overall. The authors conclude that labelling a whole sentence as positive or negative is too coarse to capture an internal polarity clash. They suggest scoring sentence sub-parts or using a sentiment lexicon. The result is a useful negative finding: the polarity-contrast intuition only pays off if the contrast is located inside the sentence. ## Key points - **Data:** the English training set has 3468 tweets, 867 of them sarcastic. The Arabic set has 3102 tweets, 745 of them sarcastic. - **Official submission:** the plain CNN, without sentiment. - English: sarcastic-class F1 0.2024, rank 37 of 43. - Arabic: F1 0.3013, rank 23 of 32. - **CNN with sentiment:** pretraining on Keras IMDB (25,000 reviews, split 20000/5000) dropped English F1 to 0.1679. - **SVM setup:** - Data balanced by downsampling to 867 tweets per class (694 train / 173 held-out each). - Features: tf-idf over unigrams to 4-grams. - **Model choice:** Naive Bayes won the grid search but labelled nearly all test tweets sarcastic. The authors therefore used the SVM with scikit-learn default settings. - **SVM results:** F1 0.2738 without the Flair features vs. 0.2721 with them. ## Interesting ideas - Sarcasm as a polarity clash should be checked between parts of a sentence, not on the sentence as a whole, since a global score can average the contrast away. - A near-neutral overall score can itself be evidence of mixed content. The authors propose treating mid-range outputs (around 0.35–0.65) as a possible contradiction instead of forcing a positive/negative label. - A model's confidence can be turned into a few discrete bands (very/quite/rather positive or negative) that feed later steps as readable one-hot features, rather than an opaque float. - The best grid-search model was rejected after a check of its prediction distribution showed it labelled almost everything as one class. - The authors blame part of the failure on a genre mismatch between the auxiliary data (movie reviews) and the target data (tweets). ## For Mova How these ideas could serve Mova, our small open language model — its files, stages and tests: - **I1** (frames): the irony module in prag could score polarity separately on the predicate and on its complement or situation in the UD tree. It could then flag a strong positive-vs-negative mismatch and cite the two clashing constituents as its explanation. - **I2** (frames): an implicature or irony detector could treat "uncertain overall polarity combined with strongly polar parts" as a trigger for an ironic reading, not as noise. - **I3** (frames): speech-act or sentiment judgments in prag could be passed on as named confidence bands. Downstream rules and explanations could then mention them explicitly. - **I4** (learn): add a label-distribution sanity check to evaluation as a cheap negative control, so that degenerate classifiers or repair rules are caught before they enter training. - **I5** (learn): when choosing silver data or LLM-teacher examples for a skill like irony, prefer sources close to the target genre and record genre so mismatches can be audited. *Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.* ## Institutions - [University of Tübingen](https://md.research.elamur.ai/free/en/universities/university-of-tubingen/) — University, Tübingen, Germany · 30 works · 40,834 citations 📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 9,086 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 8,889 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 9,296 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 9,206 works 🧭 Topics (1): [😊 Sentiment & emotion](https://md.research.elamur.ai/free/en/topics/sentiment-emotion/) · 608 works ## Cite Cite as: Patrick Hantsch and Nadav Chkroun. 2022. connotation\_clashers at SemEval-2022 Task 6: The effect of sentiment analysis on sarcasm detection. Proceedings of the 16th International Workshop on Semantic Evaluation (SemEval-2022). The paper itself, not this page. BibTeX from the library's record of the work. ```bibtex @inproceedings{hantsch2022connotation, title = {{connotation\_clashers at SemEval-2022 Task 6: The effect of sentiment analysis on sarcasm detection}}, author = {Patrick Hantsch and Nadav Chkroun}, booktitle = {Proceedings of the 16th International Workshop on Semantic Evaluation (SemEval-2022)}, year = {2022}, doi = {10.18653/v1/2022.semeval-1.132}, url = {https://doi.org/10.18653/v1/2022.semeval-1.132} } ``` - [← previous Comparing Native and Learner Englishes Using a Large Pre-trained Language Model](https://md.research.elamur.ai/free/en/papers/aoyama-2022-comparing-native-learner-englishes-large/) - [next → Contextualized Sensorimotor Norms: multi-dimensional measures of sensorimotor strength for ambiguous English words, in context](https://md.research.elamur.ai/free/en/papers/trott-2022-contextualized-sensorimotor-norms-multi-dimensional/) - [↑ back to papers of 2022](https://md.research.elamur.ai/free/en/papers/2022/page/6/) ## References 10 references, 2 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet. 1. Ibrahim Abu Farha, Silviu Oprea, Steven Wilson, and Walid Magdy. 2022. SemEval-2022 Task 6: iSar-casmEval, Intended Sarcasm Detection in English and Arabic. In Proceedings of the 16th International Workshop on Semantic Evaluation (SemEval-2022). Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Ibrahim+Abu+Farha%2C+Silviu+Oprea%2C+Steven+Wilson%2C+and+Walid+Magdy%2E+2022%2E+SemEval%2D2022+Task+6%3A+iSar%2DcasmEval%2C+Intended+Sarcasm+Detection+in+English+and+Arabic%2E+In+Proceedings+of+the+16th+International+Wo) 2. Alan Akbik, Tanja Bergmann, Duncan Blythe, Kashif Rasul, Stefan Schweter, and Roland Vollgraf. 2019. Flair: An easy-to-use framework for state-of-the-art in natural language processing. In NAACL 2019, 2019 Annual Conference of the North American Chapter of the Association for Computational Linguistics (Demonstrations), pages 54–59. [find](https://scholar.google.com/scholar?q=Alan+Akbik%2C+Tanja+Bergmann%2C+Duncan+Blythe%2C+Kashif+Rasul%2C+Stefan+Schweter%2C+and+Roland+Vollgraf%2E+2019%2E+Flair%3A+An+easy%2Dto%2Duse+framework+for+state%2Dof%2Dthe%2Dart+in+natural+language+processing%2E+In+NAACL+2019%2C) 3. Jason Brownlee. 2019. A gentle introduction to the rectified linear unit (relu). [https://machinelearningmastery.com/rectified-linear-activation-function-for-deep-learning-neural-networks/](https://machinelearningmastery.com/rectified-linear-activation-function-for-deep-learning-neural-networks/). [↗](https://machinelearningmastery.com/rectified-linear-activation-function-for-deep-learning-neural-networks/) 4. François Chollet et al. 2015. keras.io. Keras. [https://](https://) [↗](https://) 5. Simona Frenda. 2018. The role of sarcasm in hate speech. pages 14–16. [find](https://scholar.google.com/scholar?q=Simona+Frenda%2E+2018%2E+The+role+of+sarcasm+in+hate+speech%2E+pages+14%E2%80%9316%2E) 6. Cynthia Van Hee, Els Lefever, and Véronique Hoste. 2018. Semeval-2018 task 3: Irony detection in english tweets. In Proceedings of the 12th International Workshop on Semantic Evaluation (SemEval-2018). Association for Computational Linguistics. [our card](https://md.research.elamur.ai/free/en/papers/hee-2018-semeval-2018-task-3-irony/) 7. Diederik Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. International Conference on Learning Representations. [↗](https://doi.org/10.48550/arxiv.1412.6980) 8. F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, M. Blondel, P. Prettenhofer, R. Weiss, V. Dubourg, J. Vanderplas, A. Passos, D. Cournapeau, M. Brucher, M. Perrot, and E. Duchesnay. 2011. Scikit-learn: Machine learning in Python. Journal of Machine Learning Research, 12:2825–2830. [↗](https://doi.org/10.48550/arxiv.1201.0490) 9. Soujanya Poria, Erik Cambria, Devamanyu Hazarika, and Prateek Vij. 2017. A deeper look into sarcastic tweets using deep convolutional neural networks. [↗](https://doi.org/10.48550/arxiv.1610.08815) [our card](https://md.research.elamur.ai/free/en/papers/poria-2016-deeper-look-into-sarcastic-tweets/) 10. Cambridge University Press. 2022. Cambridge dictionary. [find](https://scholar.google.com/scholar?q=Cambridge+University+Press%2E+2022%2E+Cambridge+dictionary%2E) *Updated 24 October 2026*