--- title: "ISD at SemEval-2022 Task 6 — paper by Huang et al. (2022)" description: "The paper describes a system for Subtask A of SemEval-2022 Task 6, a binary task that labels a tweet as sarcastic… 0.2403 F1. 3 references." authors: ["Samantha Huang", "Ethan Chi", "Nathan Chi"] year: 2022 venue: "Proceedings of the 16th International Workshop on Semantic Evaluation (SemEval-2022)" source: "https://aclanthology.org/2022.semeval-1.129/" doi: "10.18653/v1/2022.semeval-1.129" licence: "CC BY 4.0 (ACL Anthology)" licence_url: "https://creativecommons.org/licenses/by/4.0/" summary_licence: "Elamur Cosmic License" canonical: "https://research.elamur.ai/free/en/papers/huang-2022-isd-semeval-2022-task-6/" --- [Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/huang-2022-isd-semeval-2022-task-6/) [home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2022](https://md.research.elamur.ai/free/en/papers/2022/page/7/) / ISD at SemEval-2022 Task 6: Sarcasm Detection Using Lightweight Models # ISD at SemEval-2022 Task 6: Sarcasm Detection Using Lightweight Models **Authors:** Samantha Huang, Ethan Chi and Nathan Chi · **Year:** 2022 · **Venue:** [Proceedings of the 16th International Workshop on Semantic Evaluation (SemEval-2022)](https://md.research.elamur.ai/free/en/venues/semeval/) ([SemEval 2022](https://md.research.elamur.ai/free/en/venues/semeval/2022/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/) **Links:** [Publisher's page](https://aclanthology.org/2022.semeval-1.129/) · [PDF](https://aclanthology.org/2022.semeval-1.129.pdf) · [DOI 10.18653/v1/2022.semeval-1.129](https://doi.org/10.18653/v1/2022.semeval-1.129) · [OpenAlex](https://openalex.org/W4287854412) **Citations:** 0 citations (OpenAlex, 2026-10-14) ## Summary The paper describes a system for Subtask A of SemEval-2022 Task 6, a binary task that labels a tweet as sarcastic or not. The tweet labels come from the people who wrote them. The authors ask whether cheap, low-compute classifiers can handle the task, and compare them with fine-tuned BERT. The lightweight models were Logistic Regression, GaussianNB, SVM, AdaBoost and XGBClassifier, all built with scikit-learn. Each was trained on two kinds of input: GloVe vectors, or ten hand-picked count features such as specific punctuation, emojis and cue words. Because the official test labels were hidden during the competition, the authors held out a quarter of the training data for validation. On this internal split, hand-crafted features gave clearly better sarcastic-class F1 than GloVe, but they also lowered accuracy, which reflects the skewed class balance. The submitted system was XGBClassifier on the manual features. It scored 0.2403 F1 on the official test set and placed 32nd of 43. The study suggests that small, transparent feature sets can match or beat fine-tuned BERT on a small, imbalanced sarcasm dataset. ## Key points - The dataset has 4868 tweets: 3468 for training and 1400 for test. The training portion holds 867 sarcastic and 2601 non-sarcastic items, and the internal 75:25 split gave 2774 train and 693 validation examples. - There are ten manual features, each a count of occurrences: - mentions of political leaders, the word "gay", "definitely", "thank" and "haha"; - the characters "!", "@" and "#"; - the pleading-face and grinning-face emojis. - Best validation results by positive-class F1: - XGBClassifier with manual features: 0.45, but accuracy only 0.32; - Logistic Regression with manual features: 0.44; - GaussianNB with manual features: 0.43; - BERT base uncased, the best of four BERT variants: 0.40, with accuracy 0.75. - The official test F1 of 0.2403 is far below the internal validation score, so the home-made split overestimated real performance. - The authors note that the top XGB model did not clearly outperform plain Logistic Regression. ## Interesting ideas - A tiny lexicon of surface cues (emoji, hashtags, "definitely", "haha"), counted and fed to a linear model, yields per-cue coefficients that show which signals push toward a sarcastic reading. - Running each classifier on two feature types, hand-crafted counts versus GloVe embeddings, isolates how much the representation matters compared with the learner. - Reporting minority-class F1 together with accuracy exposed a sharp trade-off: features that raised sarcastic-class F1 dragged accuracy well under what always predicting the majority class would give. - The labels encode the author's own intention to be sarcastic rather than a reader's perception, which is a deliberate choice about what ground truth means for irony. - On a small imbalanced set, simple boosted trees and Naive Bayes rivalled fine-tuned BERT, so trying cheap models first is worthwhile. ## For Mova How these ideas could serve Mova, our small open language model — its files, stages and tests: - **I1** (frames): The irony module in prag could keep a similar small, inspectable cue inventory as explicit evidence, combined with tree-level contrast between literal sentiment and context, so every irony flag can cite its triggers. - **I2** (learn): Evaluations of any learned component could routinely pair "explainable features" against "opaque embedding" runs with the model held fixed, to show whether transparency costs anything. - **I3** (learn): Rare-phenomenon detectors such as irony or speech-act classifiers should be scored against a majority-class negative control plus per-class F1, so a metric gain cannot hide a degenerate model. - **I4** (frames): When building irony data for prag, Mova could record whether a label reflects speaker intent or listener interpretation, since implicature reasoning differs between the two. - **I5** (learn): Before training GPU models for a new pragmatic label, a lightweight baseline on explicit features could set the bar and might already be good enough. *Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.* ## Institutions - [Stanford University](https://md.research.elamur.ai/free/en/universities/stanford-university/) — University, Stanford, United States · 310 works · 284,480 citations Also on the byline: [De Anza College](https://ror.org/01dfsmx55). 📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 9,086 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 8,889 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 9,296 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 9,206 works 🧭 Topics (1): [😊 Sentiment & emotion](https://md.research.elamur.ai/free/en/topics/sentiment-emotion/) · 608 works ## Cite Cite as: Samantha Huang, Ethan Chi and Nathan Chi. 2022. ISD at SemEval-2022 Task 6: Sarcasm Detection Using Lightweight Models. Proceedings of the 16th International Workshop on Semantic Evaluation (SemEval-2022). The paper itself, not this page. BibTeX from the library's record of the work. ```bibtex @inproceedings{huang2022isd, title = {{ISD at SemEval-2022 Task 6: Sarcasm Detection Using Lightweight Models}}, author = {Samantha Huang, Ethan Chi and Nathan Chi}, booktitle = {Proceedings of the 16th International Workshop on Semantic Evaluation (SemEval-2022)}, year = {2022}, doi = {10.18653/v1/2022.semeval-1.129}, url = {https://doi.org/10.18653/v1/2022.semeval-1.129} } ``` - [← previous Investigating the improvement of students' performance and autonomy in problem-solving styles of learning English grammar under the flipped learning pattern](https://md.research.elamur.ai/free/en/papers/soleymani-2022-investigating-improvement-students-performance-autonomy/) - [next → Lip̄nē 'in the face of': A Locative Preposition with a Threatening Connotation](https://md.research.elamur.ai/free/en/papers/staps-2022-lipne-face-locative-preposition-threatening/) - [↑ back to papers of 2022](https://md.research.elamur.ai/free/en/papers/2022/page/7/) ## References 3 references, 0 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet. 1. Ibrahim Abu Farha, Silviu Oprea, Steven Wilson, and Walid Magdy. 2022. SemEval-2022 Task 6: iSar-casmEval, Intended Sarcasm Detection in English and Arabic. In Proceedings of the 16th International Workshop on Semantic Evaluation (SemEval-2022). Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Ibrahim+Abu+Farha%2C+Silviu+Oprea%2C+Steven+Wilson%2C+and+Walid+Magdy%2E+2022%2E+SemEval%2D2022+Task+6%3A+iSar%2DcasmEval%2C+Intended+Sarcasm+Detection+in+English+and+Arabic%2E+In+Proceedings+of+the+16th+International+Wo) 2. Dimitri P Solomatine and Durga L Shrestha. 2004. Ad-aboost. rt: a boosting algorithm for regression problems. In 2004 IEEE International Joint Conference on Neural Networks (IEEE Cat. No. 04CH37541), volume 2, pages 1163–1168. IEEE. [find](https://scholar.google.com/scholar?q=Dimitri+P+Solomatine+and+Durga+L+Shrestha%2E+2004%2E+Ad%2Daboost%2E+rt%3A+a+boosting+algorithm+for+regression+problems%2E+In+2004+IEEE+International+Joint+Conference+on+Neural+Networks+%28IEEE+Cat%2E+No%2E+04CH37541%29%2C+) 3. Hwanjo Yu and Sungchul Kim. 2012. Svm tutorial-classification, regression and ranking. Handbook of Natural computing, 1:479–506. Model Hyperparameter Task 1a GaussianNB priors var_smoothing 0.025, 0.975 1e-09 SVM class_weight C kernel degree balanced 1.0 rbf 50 AdaBoostClassifier base_estimator n estimators learning_rate loss max_depth = 1 class_weight = {0: 0.1, 1: 0.9} 50 1.0 linear n estimators max_depth eta min_child_weight booster 100 5 0.3 1 gbtree XGBClassifier *Table 5: Hyperparameters for best-performing Manual models.* Sentence Sarcastic yeah your girl is fine but does she pass out while giving blood just impulse bought a mandolin and in 3-5 buisness days i will impulse learn some jigs *Table 6: Examples that are sarcastic and not sarcastic, respectively.* [↗](https://doi.org/10.1007/978-3-540-92910-9_15) *Updated 24 October 2026*