--- title: "Irony Detector at SemEval-2018 Task 3: Irony Detection in" description: "This system paper tackles irony detection in English tweets for SemEval-2018 Task 3. 0.6135 precision. 3 references, cited by 2 papers here." authors: ["Usman Ahmed", "Lubna Zafar", "Faiza Qayyum", "Muhammad Arshad Islam"] year: 2018 venue: "Proceedings of the 12th International Workshop on Semantic Evaluation" source: "https://aclanthology.org/S18-1095/" doi: "10.18653/v1/s18-1095" licence: "CC BY 4.0 (ACL Anthology)" licence_url: "https://creativecommons.org/licenses/by/4.0/" summary_licence: "Elamur Cosmic License" canonical: "https://research.elamur.ai/free/en/papers/ahmed-2018-irony-detector-semeval-2018-task/" --- [Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/ahmed-2018-irony-detector-semeval-2018-task/) [home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2018](https://md.research.elamur.ai/free/en/papers/2018/page/5/) / Irony Detector at SemEval-2018 Task 3: Irony Detection in English Tweets using Word Graph # Irony Detector at SemEval-2018 Task 3: Irony Detection in English Tweets using Word Graph **Authors:** Usman Ahmed, Lubna Zafar, Faiza Qayyum and Muhammad Arshad Islam · **Year:** 2018 · **Venue:** [Proceedings of the 12th International Workshop on Semantic Evaluation](https://md.research.elamur.ai/free/en/venues/semeval/) ([SemEval 2018](https://md.research.elamur.ai/free/en/venues/semeval/2018/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/) **Links:** [Publisher's page](https://aclanthology.org/S18-1095/) · [PDF](https://aclanthology.org/S18-1095.pdf) · [DOI 10.18653/v1/s18-1095](https://doi.org/10.18653/v1/s18-1095) · [OpenAlex](https://openalex.org/W2805997936) **Citations:** 15 citations (OpenAlex, 2026-10-14) · 2 in our library, of the 141,840 papers whose bibliographies we read ## Summary This system paper tackles irony detection in English tweets for SemEval-2018 Task 3. It covers both the binary ironic/non-ironic subtask (A) and the four-way subtask (B): polarity-contrast verbal irony, other verbal irony, situational irony, and non-irony. Each tweet becomes a directed, unweighted word graph whose edges link words within a small sliding window. All training tweets of a class are combined into one class graph. A new tweet is scored against every class graph with several graph-similarity measures: containment and three maximum-common-subgraph variants. These scores, plus SenticNet polarity/subjectivity and LDA-based Hellinger distances to each class, form a compact feature vector. TPOT searches for the classifier pipeline, and SMOTE handles class imbalance. The system placed 23rd on subtask A (F1 0.5955) and 22nd on subtask B (F1 0.3101). The scores are modest, but the paper shows that comparing a text's graph with class-level prototype graphs yields usable, low-dimensional features for a pragmatic phenomenon. ## Key points - Data: 3,000 tweets gathered through #irony, #sarcasm and #not between Dec 2014 and Apr 2015, from 2,676 users. Annotators were three linguistics students using the brat tool, with kappa 0.72. The class counts are 1,728 polarity-contrast, 267 other verbal, 401 situational, and 604 non-ironic. - The organizers balanced the data with a background corpus to 4,792 tweets (2,396 ironic and 2,396 non-ironic). They split it 80/20 into 3,833 training and 958 test instances. - Graph construction uses a vicinity window of 2 with directed edges that keep word order. The similarity features are: - containment similarity (non-normalized), with shared edges divided by the size of the smaller graph; - MCS node similarity; - MCS undirected-edge similarity; - MCS directed-edge similarity. - TPOT chose a tuned XGBoost model for subtask A. For subtask B it chose a stack of XGBoost, extra trees and random forest, trained with SMOTE oversampling. - Results: - Subtask A: accuracy 0.6135, precision 0.5091, recall 0.7170, F1 0.5955. - Subtask B: accuracy 0.4158, precision 0.4055, recall 0.3526, F1 0.3101. ## Interesting ideas - Each class is summarized as one merged graph, and a text is described by its similarity to each class prototype, giving a tiny feature vector whose values can be traced to matched edges. - The same overlap is measured at three strictness levels: nodes only, edges ignoring direction, and edges with direction. The gaps between these levels show how much structure, not just vocabulary, is shared. - Containment divides shared edges by the smaller graph's size, so a short text is not penalized for being compared with a huge aggregate graph. - The task separates verbal irony that rests on a polarity contrast from other kinds, and the system adds sentence polarity scores as explicit features. ## For Mova How these ideas could serve Mova, our small open language model — its files, stages and tests: - **I1** (frames): an irony cue in prag could compare a sentence's UD dependency edges against prototype graphs built from ironic and literal examples, then report which shared edges drove the score. - **I2** (learn): when witness parsers disagree, scoring their trees at node, undirected-edge and directed-edge levels would separate attachment disagreements from head-direction flips before silver data is accepted. - **I3** (back): in text → graph → text round-trip checks, containment of the original meaning graph's edges in the regenerated one gives a size-fair preservation score. - **I4** (frames): prag could flag a polarity clash between a stated evaluation and the described event's valence as a named, explainable irony trigger, with non-contrast irony handled separately. *Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.* ## Institutions Also on the byline: [Capital University of Science and Technology](https://ror.org/004776246). 📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 9,086 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 8,889 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 9,296 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 9,206 works 🧭 Topics (2): [English](https://md.research.elamur.ai/free/en/topics/english/) · 1,373 works · [😊 Sentiment & emotion](https://md.research.elamur.ai/free/en/topics/sentiment-emotion/) · 608 works ## Cite Cite as: Usman Ahmed, Lubna Zafar, Faiza Qayyum and Muhammad Arshad Islam. 2018. Irony Detector at SemEval-2018 Task 3: Irony Detection in English Tweets using Word Graph. Proceedings of the 12th International Workshop on Semantic Evaluation. The paper itself, not this page. BibTeX from the library's record of the work. ```bibtex @inproceedings{ahmed2018irony, title = {{Irony Detector at SemEval-2018 Task 3: Irony Detection in English Tweets using Word Graph}}, author = {Usman Ahmed, Lubna Zafar, Faiza Qayyum and Muhammad Arshad Islam}, booktitle = {Proceedings of the 12th International Workshop on Semantic Evaluation}, year = {2018}, doi = {10.18653/v1/s18-1095}, url = {https://doi.org/10.18653/v1/s18-1095} } ``` - [← previous Improving Crowdsourcing-Based Annotation of Japanese Discourse Relations](https://md.research.elamur.ai/free/en/papers/kishimoto-2018-improving-crowdsourcing-based-annotation-japanese/) - [next → Linguistic Features of Sarcasm and Metaphor Production Quality](https://md.research.elamur.ai/free/en/papers/skalicky-2018-linguistic-features-sarcasm-metaphor-production/) - [↑ back to papers of 2018](https://md.research.elamur.ai/free/en/papers/2018/page/5/) ## References 3 references, 0 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet. 1. Rudolf Beran. 1977. Minimum hellinger distance estimates for parametric models. The annals of Statistics, pages 445–463. [↗](https://doi.org/10.1214/aos/1176343842) 2. Fotis Aisopos, George Papadakis, Konstantinos Tser-pes, and Theodora Varvarigou. 2012. Content vs. context for sentiment analysis: a comparative analysis over microblogs. In Proceedings of the 23rd ACM conference on Hypertext and social media, pages 187–196. ACM. [find](https://scholar.google.com/scholar?q=Fotis+Aisopos%2C+George+Papadakis%2C+Konstantinos+Tser%2Dpes%2C+and+Theodora+Varvarigou%2E+2012%2E+Content+vs%2E+context+for+sentiment+analysis%3A+a+comparative+analysis+over+microblogs%2E+In+Proceedings+of+the+23rd+AC) 3. Laura Alba-Juez and Salvatore Attardo. 2014. The evaluative palette of verbal irony. Evaluation in context, 242:93. Results Evaluation [↗](https://doi.org/10.1075/pbns.242.05alb) *Updated 24 October 2026*