--- title: "High Tech team at SemEval-2022 Task 6 — Alami 2022" description: "The authors join iSarcasmEval (SemEval-2022 Task 6) for Arabic only." authors: ["Hamza Alami", "Abdessamad Benlahbib", "Ahmed Alami"] year: 2022 venue: "Proceedings of the 16th International Workshop on Semantic Evaluation (SemEval-2022)" source: "https://aclanthology.org/2022.semeval-1.116/" doi: "10.18653/v1/2022.semeval-1.116" licence: "CC BY 4.0 (ACL Anthology)" licence_url: "https://creativecommons.org/licenses/by/4.0/" summary_licence: "Elamur Cosmic License" canonical: "https://research.elamur.ai/free/en/papers/alami-2022-high-tech-team-semeval-2022/" --- [Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/alami-2022-high-tech-team-semeval-2022/) [home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2022](https://md.research.elamur.ai/free/en/papers/2022/page/2/) / High Tech team at SemEval-2022 Task 6: Intended Sarcasm Detection for Arabic texts # High Tech team at SemEval-2022 Task 6: Intended Sarcasm Detection for Arabic texts **Authors:** [Hamza Alami](https://md.research.elamur.ai/free/en/authors/hamza-alami/), [Abdessamad Benlahbib](https://md.research.elamur.ai/free/en/authors/abdessamad-benlahbib/), Ahmed Alami · **Year:** 2022 · **Venue:** Proceedings of the 16th International Workshop on Semantic Evaluation (SemEval-2022) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/) **Links:** [Publisher's page](https://aclanthology.org/2022.semeval-1.116/) · [PDF](https://aclanthology.org/2022.semeval-1.116.pdf) · [DOI 10.18653/v1/2022.semeval-1.116](https://doi.org/10.18653/v1/2022.semeval-1.116) · [OpenAlex](https://openalex.org/W4287890126) **Citations:** 1 citations (openalex-snapshot · 2026-10-14) · [1](#cited-by) in our library, of the 140884 papers whose bibliographies we read ## Summary The authors join iSarcasmEval (SemEval-2022 Task 6) for Arabic only. They take part in subtask A, which asks whether a tweet is sarcastic, and subtask C, which asks which of two texts with the same meaning is the sarcastic one. Both systems fine-tune Arabic BERT models (AraBERTv02-twitter and CAMEL-Mix) with a softmax output layer and cross-entropy loss. The training data are imbalanced, and emojis are much rarer in training than in test. To handle this, the authors build four validation splits that vary how emoji-bearing sarcastic tweets are represented. They pick models by averaging sarcastic-class F1 across these splits. An extra preprocessing step that replaces emojis with their meanings did not help. For subtask C, the two texts are joined with [SEP]. The data are doubled by swapping the order of the pair and flipping the label. The submissions placed 4th in subtask A (46.84% sarcastic-class F1) and 2nd in subtask C (88.5% accuracy). The work shows that simple, well-validated fine-tuning can compete in Arabic sarcasm detection. ## Key points - Training set: 3102 Arabic tweets, of which 745 (24.02%) are sarcastic and 2357 (75.98%) are not. The test set has 1400 tweets, and every tweet also carries a dialect label. - Emoji share differs sharply between sets: 6.9% of training samples versus 20.86% of test samples. This gap motivated validation splits A–D, ranging from "all emoji-sarcastic tweets held out" to "none held out", plus a plain stratified 20% split. - Averaged sarcastic-class F1 across the four splits: - AraBERTv02-twitter: 77.32 - CAMEL-MIX: 76.36 - AraBERTv02-twitter with emoji-to-meaning preprocessing (Alami et al., 2020): 70.75 - Setup: HuggingFace, AdamW, batch size 8, learning rate 1e-5, run on Google Colab. - Results: subtask A ranked 4th with 46.84% F1 on the sarcastic class, well below the validation figures. Subtask C, a pairwise [SEP]-concatenated classifier with swap augmentation, ranked 2nd with 88.5% accuracy. ## Interesting ideas - Build validation splits that deliberately over- or under-represent a feature known to shift between training and deployment (here, emojis), and pick the model that holds up across all of them. This gives a cheap robustness check whenever the test distribution is known to differ. — For Mova (learn): Evaluation could hold out sentences rich in a rare phenomenon (e.g., emoji, ellipsis, quoted speech) in several proportions, and select models by their averaged score rather than one random split. - Recast "is this sarcastic?" as a forced choice between a sarcastic text and its literal paraphrase. This minimal-pair setup isolates the sarcasm signal from topic and content and turned out much easier than absolute classification. — For Mova (frames): Irony detection in prag could be tested on pairs of ironic and literal rephrasings, checking that the frame-level clash between surface and implied evaluation appears only in the ironic member. - Double pairwise data by swapping the two inputs and flipping the label. This removes any positional shortcut and is essentially free. — For Mova (answer): Readers answering multiple-choice school tests could be evaluated, and trained, with permuted option orders as a negative control against position bias. - Treat an intuitive preprocessing step (replacing emojis with their textual meaning) as a hypothesis to ablate, not a default; here it actually lowered the score. — For Mova (tokens): Emoji and symbol normalization in the en tokenizer should be kept switchable and measured downstream, rather than assumed to help. *Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.* Fields: [Linguistics & languages](https://md.research.elamur.ai/free/en/shelves/linguistics/) ## Cited by Cited by 1 paper of our library cites it (of the 140884 whose bibliographies we read); each has a page here. - [SemEval-2022 Task 6: iSarcasmEval, Intended Sarcasm Detection in English and Arabic](https://md.research.elamur.ai/free/en/papers/farha-2022-semeval-2022-task-6-isarcasmeval/) ## Cite The paper itself, not this page. BibTeX from the library's record of the work. ```bibtex @inproceedings{alami2022high, title = {{High Tech team at SemEval-2022 Task 6: Intended Sarcasm Detection for Arabic texts}}, author = {Alami, Hamza and Benlahbib, Abdessamad and Alami, Ahmed}, booktitle = {Proceedings of the 16th International Workshop on Semantic Evaluation (SemEval-2022)}, year = {2022}, doi = {10.18653/v1/2022.semeval-1.116}, url = {https://doi.org/10.18653/v1/2022.semeval-1.116} } ``` - [← previous High Quality Rather than High Model Probability: Minimum Bayes Risk Decoding with Neural Metrics](https://md.research.elamur.ai/free/en/papers/freitag-2022-high-quality-rather-than-high/) - [next → How do we get there? Evaluating transformer neural networks as cognitive models for English past tense inflection](https://md.research.elamur.ai/free/en/papers/ma-2022-how-do-we-get-there/) - [↑ back to papers of 2022](https://md.research.elamur.ai/free/en/papers/2022/page/2/) ## References 11 references, 5 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet. 1. Ibrahim Abu Farha, Silviu Oprea, Steven Wilson, and Walid Magdy. 2022. SemEval-2022 Task 6: iSarcasmEval, Intended Sarcasm Detection in English and Arabic. In Proceedings of the 16th International Workshop on Semantic Evaluation (SemEval-2022). Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Ibrahim+Abu+Farha%2C+Silviu+Oprea%2C+Steven+Wilson%2C+and+Walid+Magdy%2E+2022%2E+SemEval%2D2022+Task+6%3A+iSarcasmEval%2C+Intended+Sarcasm+Detection+in+English+and+Arabic%2E+In+Proceedings+of+the+16th+International+Wor) [our card](https://md.research.elamur.ai/free/en/papers/farha-2022-semeval-2022-task-6-isarcasmeval/) 2. Hamza Alami, Said Ouatik El Alaoui, Abdessamad Benlahbib, and Noureddine En-nahnahi. 2020. LISAC FSDM-USMBA team at SemEval-2020 task 12: Overcoming AraBERT’s pretrain-finetune discrepancy for Arabic offensive language identification. In Proceedings of the Fourteenth Workshop on Semantic Evaluation, pages 2080–2085, Barcelona (online). International Committee for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Hamza+Alami%2C+Said+Ouatik+El+Alaoui%2C+Abdessamad+Benlahbib%2C+and+Noureddine+En%2Dnahnahi%2E+2020%2E+LISAC+FSDM%2DUSMBA+team+at+SemEval%2D2020+task+12%3A+Overcoming+AraBERT%E2%80%99s+pretrain%2Dfinetune+discrepancy+for+Arabic+) 3. Wissam Antoun, Fady Baly, and Hazem Hajj. 2020. AraBERT: Transformer-based model for Arabic language understanding. In Proceedings of the 4th Workshop on Open-Source Arabic Corpora and Processing Tools, with a Shared Task on Offensive Language Detection, pages 9–15, Marseille, France. European Language Resource Association. [find](https://scholar.google.com/scholar?q=Wissam+Antoun%2C+Fady+Baly%2C+and+Hazem+Hajj%2E+2020%2E+AraBERT%3A+Transformer%2Dbased+model+for+Arabic+language+understanding%2E+In+Proceedings+of+the+4th+Workshop+on+Open%2DSource+Arabic+Corpora+and+Processing+Tool) [our card](https://md.research.elamur.ai/free/en/papers/antoun-2020-arabert-transformer-based-model-arabic/) 4. Sara Rosenthal, Alan Ritter, Preslav Nakov, and Veselin Stoyanov. 2014. SemEval-2014 task 9: Sentiment analysis in Twitter. In Proceedings of the 8th International Workshop on Semantic Evaluation (SemEval 2014), pages 73–80, Dublin, Ireland. Association for Computational Linguistics. [↗](https://doi.org/10.3115/v1/s14-2009) [our card](https://md.research.elamur.ai/free/en/papers/rosenthal-2014-semeval-2014-task-9-sentiment/) 5. Mike Schuster and Kaisuke Nakajima. 2012. Japanese and korean voice search. In 2012 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 5149–5152. [find](https://scholar.google.com/scholar?q=Mike+Schuster+and+Kaisuke+Nakajima%2E+2012%2E+Japanese+and+korean+voice+search%2E+In+2012+IEEE+International+Conference+on+Acoustics%2C+Speech+and+Signal+Processing+%28ICASSP%29%2C+pages+5149%E2%80%935152%2E) 6. Cynthia Van Hee, Gilles Jacobs, Chris Emmery, Bart Desmet, Els Lefever, Ben Verhoeven, Guy De Pauw, Walter Daelemans, and Véronique Hoste. 2018. Automatic detection of cyberbullying in social media text. PLOS ONE, 13(10):1–22. [find](https://scholar.google.com/scholar?q=Cynthia+Van+Hee%2C+Gilles+Jacobs%2C+Chris+Emmery%2C+Bart+Desmet%2C+Els+Lefever%2C+Ben+Verhoeven%2C+Guy+De+Pauw%2C+Walter+Daelemans%2C+and+V%C3%A9ronique+Hoste%2E+2018%2E+Automatic+detection+of+cyberbullying+in+social+media+te) 7. Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pier-ric Cistac, Tim Rault, Remi Louf, Morgan Funtow-icz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Mariama Drame, Quentin Lhoest, and Alexander Rush. 2020. Transformers: State-of-the-art natural language processing. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pages 38–45, Online. Association for Computational Linguistics. [↗](https://doi.org/10.18653/v1/2020.emnlp-demos.6) [our card](https://md.research.elamur.ai/free/en/papers/wolf-2020-transformers-state-art-natural-language/) 8. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 4171–4186, Minneapolis, Minnesota. Association for Computational Linguistics. [↗](https://doi.org/10.18653/v1/n19-1423) 9. Go Inoue, Bashar Alhafni, Nurpeiis Baimukan, Houda Bouamor, and Nizar Habash. 2021. The interplay of variant, size, and task type in Arabic pre-trained language models. In Proceedings of the Sixth Arabic Natural Language Processing Workshop, Kyiv, Ukraine (Online). Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Go+Inoue%2C+Bashar+Alhafni%2C+Nurpeiis+Baimukan%2C+Houda+Bouamor%2C+and+Nizar+Habash%2E+2021%2E+The+interplay+of+variant%2C+size%2C+and+task+type+in+Arabic+pre%2Dtrained+language+models%2E+In+Proceedings+of+the+Sixth+Ara) [our card](https://md.research.elamur.ai/free/en/papers/inoue-2021-interplay-variant-size-task-type/) 10. Bing Liu. 2012. Sentiment analysis and opinion mining. Synthesis lectures on human language technologies, 5(1):1–167. [↗](https://doi.org/10.1007/978-3-031-02145-9) 11. Ilya Loshchilov and Frank Hutter. 2017. weight decay regularization in adam. abs/1711.05101. Fixing ArXiv, [↗](https://arxiv.org/abs/1711.05101)