---
title: "PALI-NLP at SemEval-2022 Task 6 — paper by Du et al. (2022)"
description: "The paper presents the system behind PALI-NLP's entry to SemEval-2022 Task 6 (iSarcasmEval). 19 references, cited by 4 papers here."
authors: ["Xiyang Du", "Dou Hu", "Jin Zhi", "Lianxin Jiang", "Xiaofeng Shi"]
year: 2022
venue: "Proceedings of the 16th International Workshop on Semantic Evaluation (SemEval-2022)"
source: "https://aclanthology.org/2022.semeval-1.112/"
doi: "10.18653/v1/2022.semeval-1.112"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/du-2022-pali-nlp-semeval-2022-task/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/du-2022-pali-nlp-semeval-2022-task/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2022](https://md.research.elamur.ai/free/en/papers/2022/page/7/) / PALI-NLP at SemEval-2022 Task 6: iSarcasmEval- Fine-tuning the Pre-trained Model for Detecting Intended Sarcasm

# PALI-NLP at SemEval-2022 Task 6: iSarcasmEval- Fine-tuning the Pre-trained Model for Detecting Intended Sarcasm

**Authors:** Xiyang Du, Dou Hu, Jin Zhi, [Lianxin Jiang](https://md.research.elamur.ai/free/en/authors/lianxin-jiang/) and [Xiaofeng Shi](https://md.research.elamur.ai/free/en/authors/xiaofeng-shi/) · **Year:** 2022 · **Venue:** [Proceedings of the 16th International Workshop on Semantic Evaluation (SemEval-2022)](https://md.research.elamur.ai/free/en/venues/semeval/) ([SemEval 2022](https://md.research.elamur.ai/free/en/venues/semeval/2022/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2022.semeval-1.112/) · [PDF](https://aclanthology.org/2022.semeval-1.112.pdf) · [DOI 10.18653/v1/2022.semeval-1.112](https://doi.org/10.18653/v1/2022.semeval-1.112) · [OpenAlex](https://openalex.org/W4287889069)

**Citations:** 3 citations (OpenAlex, 2026-10-14) · [4](https://md.research.elamur.ai/free/en/papers/du-2022-pali-nlp-semeval-2022-task/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
The paper presents the system behind PALI-NLP's entry to SemEval-2022 Task 6 (iSarcasmEval). The team worked on Subtask B, a multi-label problem: assign a sarcastic English tweet one or more of six categories (sarcasm, irony, satire, understatement, overstatement, rhetorical question), and give a non-sarcastic tweet no label. The approach is deliberately simple. Pre-trained encoders (BERT, RoBERTa, BERTweet) are fine-tuned with a plain classifier on the [CLS] representation. The training data combines the official iSarcasm-2022 set with extra sarcastic tweets from the earlier iSarcasm corpus.

Most of the gains come from training choices rather than architecture:
- a re-weighted BCE loss,
- multi-sample dropout,
- PGD adversarial perturbation of the embeddings,
- optionally a supervised contrastive loss.

Final predictions come from a rank-weighted ensemble that votes separately on primary and secondary labels. The system placed 1st in Subtask B with macro-F1 0.1630. Its best single model reached 0.1675. The work shows that matching the pre-training domain and tuning the training recipe can outperform complex classifier designs on noisy social-media text.

## Key points
- BERTweet clearly outperformed the general-domain encoders (macro-F1 0.1417 vs 0.0991 for RoBERTa-base and 0.0766 for BERT-base-uncased), which the authors attribute to its Twitter pre-training corpus.
- 777 extra samples from iSarcasm (Oprea and Magdy, 2019) were added. Re-weighting the BCE loss raised BERTweet from 0.1417 to 0.1564, and this became the baseline.
- Each technique was tested against that baseline:

  | Setting | Macro-F1 |
  |---|---|
  | Baseline | 0.1564 |
  | 6-layer multi-sample dropout | 0.1578 |
  | PGD (adversarial rate 0.5) | 0.1595 |
  | PGD + multi-sample dropout | 0.1675 |
  | SupCon loss alone | 0.1529 |
  | SupCon + multi-sample dropout | 0.1670 |

- Training setup: batch size 128, sequence length 64, decision threshold 0.2, layer-wise learning-rate decay, AdamW with Lookahead, early stopping on three validation checks per epoch.
- The label distribution has a long tail. Understatement received an F1 of 0 in every reported configuration.

## Interesting ideas
- The authors noticed that the six labels form a two-level hierarchy (sarcasm/irony as primary, the other four as secondary) and ran separate ensemble votes for each level, which makes use of label structure without changing the model.
- Ensemble members get voting weights equal to the square root of the inverse of their validation rank, a cheap scheme that favours stronger models without letting one dominate.
- Mentions, hashtags and URLs are normalized to placeholder tokens, while emojis are deliberately kept because they may signal a gap between literal wording and intended sentiment.
- The ablations show that components interact: the contrastive loss hurt on its own but helped when paired with multi-sample dropout, so single-addition tests can be misleading.
- Pre-training on in-domain text (BERTweet) mattered more than any later fine-tuning trick in this task.

## For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

- **I1** (frames): the irony module in prag could first decide whether an utterance is ironic or sarcastic and only then choose a rhetorical subtype, keeping each decision separately explainable.
- **I2** (learn): when several witness parsers disagree while building silver data, their votes could be weighted by rank on a held-out treebank instead of counted equally.
- **I3** (tokens): the en tokenizer could map social-media artefacts to typed placeholders but keep emojis as tokens whose sentiment the irony step can compare with the literal text.
- **I4** (learn): evaluation of new training tricks or repair rules should include pairwise combinations alongside negative controls before a component is discarded.
- **I5** (tree): for informal or tweet-like input, gparse could swap in a domain-matched encoder, or be adapted on such text, before effort goes into parser-side tuning.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Ping An](https://md.research.elamur.ai/free/en/companies/ping-an/) — Company, Shenzhen, China · 3 works · 228 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 18,614 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 17,965 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 17,450 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 17,296 works

🧭 Topics (2): [English](https://md.research.elamur.ai/free/en/topics/english/) · 2,379 works · [😊 Sentiment & emotion](https://md.research.elamur.ai/free/en/topics/sentiment-emotion/) · 1,035 works

## Cite

Cite as: Xiyang Du, Dou Hu, Jin Zhi, Lianxin Jiang and Xiaofeng Shi. 2022. PALI-NLP at SemEval-2022 Task 6: iSarcasmEval- Fine-tuning the Pre-trained Model for Detecting Intended Sarcasm. Proceedings of the 16th International Workshop on Semantic Evaluation (SemEval-2022).

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{du2022pali,
  title = {{PALI-NLP at SemEval-2022 Task 6: iSarcasmEval- Fine-tuning the Pre-trained Model for Detecting Intended Sarcasm}},
  author = {Xiyang Du, Dou Hu, Jin Zhi, Lianxin Jiang and Xiaofeng Shi},
  booktitle = {Proceedings of the 16th International Workshop on Semantic Evaluation (SemEval-2022)},
  year = {2022},
  doi = {10.18653/v1/2022.semeval-1.112},
  url = {https://doi.org/10.18653/v1/2022.semeval-1.112}
}
```

- [← previous Offensive language detection in Hebrew: can other languages help?](https://md.research.elamur.ai/free/en/papers/litvak-2022-offensive-language-detection-hebrew-can/)
- [next → ParCorFull2.0: a Parallel Corpus Annotated with Full Coreference](https://md.research.elamur.ai/free/en/papers/koltunski-2022-parcorfull2-0-parallel-corpus-annotated/)
- [↑ back to papers of 2022](https://md.research.elamur.ai/free/en/papers/2022/page/7/)

## References

19 references, 13 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Ibrahim Abu Farha, Silviu Oprea, Steven Wilson, and Walid Magdy. 2022. [SemEval-2022 Task 6: iSarcasmEval, Intended Sarcasm Detection in English and Arabic](https://md.research.elamur.ai/free/en/papers/farha-2022-semeval-2022-task-6-isarcasmeval/). In Proceedings of the 16th International Workshop on Semantic Evaluation (SemEval-2022). Association for Computational Linguistics.
2. Yitao Cai, Huiyu Cai, and Xiaojun Wan. 2019. [Multimodal sarcasm detection in twitter with hierarchical fusion model](https://md.research.elamur.ai/free/en/papers/cai-2019-multi-modal-sarcasm-detection-twitter/). In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 2506–2515.
3. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. [Bert: Pre-training of deep bidirectional transformers for language understanding](https://md.research.elamur.ai/free/en/papers/devlin-2019-bert-pre-training-deep-bidirectional/). arXiv preprint arXiv:1810.04805. [↗](https://arxiv.org/abs/1810.04805)
4. Raymond W Gibbs Jr, Raymond W Gibbs, and Jr Gibbs. 1994. The poetics of mind: Figurative thought, language, and understanding. Cambridge University Press. [find](https://scholar.google.com/scholar?q=Raymond+W+Gibbs+Jr%2C+Raymond+W+Gibbs%2C+and+Jr+Gibbs%2E+1994%2E+The+poetics+of+mind%3A+Figurative+thought%2C+language%2C+and+understanding%2E+Cambridge+University+Press%2E)
5. Hiroshi Inoue. 2019. Multi-sample dropout for accelerated training and better generalization. arXiv preprint arXiv:1905.09788. [↗](https://arxiv.org/abs/1905.09788)
6. Prannay Khosla, Piotr Teterwak, Chen Wang, Aaron Sarna, Yonglong Tian, Phillip Isola, Aaron Maschinot, Ce Liu, and Dilip Krishnan. 2020. Supervised contrastive learning. Advances in Neural Information Processing Systems, 33:18661–18673. [find](https://scholar.google.com/scholar?q=Prannay+Khosla%2C+Piotr+Teterwak%2C+Chen+Wang%2C+Aaron+Sarna%2C+Yonglong+Tian%2C+Phillip+Isola%2C+Aaron+Maschinot%2C+Ce+Liu%2C+and+Dilip+Krishnan%2E+2020%2E+Supervised+contrastive+learning%2E+Advances+in+Neural+Information)
7. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. [Attention is all you need](https://md.research.elamur.ai/free/en/papers/vaswani-2017-attention-all-need/). Advances in neural information processing systems, 30.
8. Deirdre Wilson. 2006. The pragmatics of verbal irony: Echo or pretence? Lingua, 116(10):1722–1743. [↗](https://doi.org/10.1016/j.lingua.2006.05.001)
9. Tao Xiong, Peiran Zhang, Hongbo Zhu, and Yihui Yang. 2019. Sarcasm detection with self-matching networks and low-rank bilinear pooling. In The World Wide Web Conference, pages 2115–2124. [find](https://scholar.google.com/scholar?q=Tao+Xiong%2C+Peiran+Zhang%2C+Hongbo+Zhu%2C+and+Yihui+Yang%2E+2019%2E+Sarcasm+detection+with+self%2Dmatching+networks+and+low%2Drank+bilinear+pooling%2E+In+The+World+Wide+Web+Conference%2C+pages+2115%E2%80%932124%2E)
10. Hamed Yaghoobian, Hamid R Arabnia, and Khaled Rasheed. 2021. Sarcasm detection: A comparative study. arXiv preprint arXiv:2107.02276. [↗](https://arxiv.org/abs/2107.02276)
11. Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. [Roberta: A robustly optimized bert pretraining approach](https://md.research.elamur.ai/free/en/papers/liu-2019-roberta-robustly-optimized-bert-pretraining/). arXiv preprint arXiv:1907.11692. [↗](https://arxiv.org/abs/1907.11692)
12. Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. 2017. [Towards deep learning models resistant to adversarial attacks](https://md.research.elamur.ai/free/en/papers/madry-2017-towards-deep-learning-models-resistant/). arXiv preprint arXiv:1706.06083. [↗](https://arxiv.org/abs/1706.06083)
13. Diana G Maynard and Mark A Greenwood. 2014. [Who cares about sarcastic tweets? investigating the impact of sarcasm on sentiment analysis](https://md.research.elamur.ai/free/en/papers/maynard-2014-who-cares-about-sarcastic-tweets/). In Lrec 2014 proceedings. ELRA.
14. Takeru Miyato, Andrew M Dai, and Ian Goodfellow. 2016. [Adversarial training methods for semi-supervised text classification](https://md.research.elamur.ai/free/en/papers/miyato-2016-adversarial-training-methods-semi-supervised/). arXiv preprint arXiv:1605.07725. [↗](https://arxiv.org/abs/1605.07725)
15. Dat Quoc Nguyen, Thanh Vu, and Anh Tuan Nguyen. 2020. [Bertweet: A pre-trained language model for english tweets](https://md.research.elamur.ai/free/en/papers/nguyen-2020-bertweet-pre-trained-language-model/). arXiv preprint arXiv:2005.10200. [↗](https://arxiv.org/abs/2005.10200)
16. Silviu Oprea and Walid Magdy. 2019. [isarcasm: A dataset of intended sarcasm](https://md.research.elamur.ai/free/en/papers/oprea-2020-isarcasm-dataset-intended-sarcasm/). arXiv preprint arXiv:1911.03123. [↗](https://arxiv.org/abs/1911.03123)
17. Hongliang Pan, Zheng Lin, Peng Fu, Yatao Qi, and Weiping Wang. 2020. [Modeling intra and inter-modality incongruity for multi-modal sarcasm detection](https://md.research.elamur.ai/free/en/papers/pan-2020-modeling-intra-inter-modality-incongruity/). In Findings of the Association for Computational Linguistics: EMNLP 2020, pages 1383–1392.
18. Rossano Schifanella, Paloma de Juan, Joel Tetreault, and Liangliang Cao. 2016. [Detecting sarcasm in multimodal social platforms](https://md.research.elamur.ai/free/en/papers/schifanella-2016-detecting-sarcasm-multimodal-social-platforms/). In Proceedings of the 24th ACM international conference on Multimedia, pages 1136–1145. [↗](https://doi.org/10.1145/2964284.2964321)
19. Yi Tay, Luu Anh Tuan, Siu Cheung Hui, and Jian Su. 2018. [Reasoning with sarcasm by reading in-between](https://md.research.elamur.ai/free/en/papers/tay-2018-reasoning-sarcasm-reading-between/). arXiv preprint arXiv:1805.02856. [↗](https://arxiv.org/abs/1805.02856)

*Updated 2 November 2026*