--- title: "BFCAI at SemEval-2022 Task 6 — Ashraf 2022" description: "The paper presents the BFCAI team's entry to iSarcasm (SemEval-2022 Task 6), restricted to the Arabic binary subtask of labeling tweets as sarcastic or…" authors: ["Nsrin Ashraf", "Fathy Elkazzaz", "Mohamed Taha", "Hamada Nayel", "Tarek Elshishtawy"] year: 2022 venue: "Proceedings of the 16th International Workshop on Semantic Evaluation (SemEval-2022)" source: "https://aclanthology.org/2022.semeval-1.123/" doi: "10.18653/v1/2022.semeval-1.123" arxiv: "2205.08868" licence: "CC BY 4.0 (ACL Anthology)" licence_url: "https://creativecommons.org/licenses/by/4.0/" summary_licence: "Elamur Cosmic License" canonical: "https://research.elamur.ai/free/en/papers/ashraf-2022-bfcai-semeval-2022-task-6/" --- [Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/ashraf-2022-bfcai-semeval-2022-task-6/) [home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2022](https://md.research.elamur.ai/free/en/papers/2022/) / BFCAI at SemEval-2022 Task 6: Multi-Layer Perceptron for Sarcasm Detection in Arabic Texts # BFCAI at SemEval-2022 Task 6: Multi-Layer Perceptron for Sarcasm Detection in Arabic Texts **Authors:** Nsrin Ashraf, Fathy Elkazzaz, Mohamed Taha, [Hamada Nayel](https://md.research.elamur.ai/free/en/authors/hamada-nayel/), Tarek Elshishtawy · **Year:** 2022 · **Venue:** Proceedings of the 16th International Workshop on Semantic Evaluation (SemEval-2022) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/) **Links:** [Publisher's page](https://aclanthology.org/2022.semeval-1.123/) · [PDF](https://aclanthology.org/2022.semeval-1.123.pdf) · [DOI 10.18653/v1/2022.semeval-1.123](https://doi.org/10.18653/v1/2022.semeval-1.123) · [arXiv 2205.08868](https://arxiv.org/abs/2205.08868) · [OpenAlex](https://openalex.org/W4287889358) **Citations:** 1 citations (openalex-snapshot · 2026-10-14) ## Summary The paper presents the BFCAI team's entry to iSarcasm (SemEval-2022 Task 6), restricted to the Arabic binary subtask of labeling tweets as sarcastic or not. The task data is unusual in that the tweet authors labeled their own posts. Authors of sarcastic tweets also wrote a literal rephrasing, and linguists later assigned fine-grained ironic-speech categories. The system is deliberately lightweight and uses no external resources. Tweets are cleaned of symbols, punctuation, diacritics and non-Arabic text, and elongated letters are shortened. The cleaned text becomes unigram TF-IDF vectors, which feed several scikit-learn classifiers and a hard-voting ensemble. A small Multi-Layer Perceptron gave the best 5-fold cross-validation accuracy and was submitted. On the test set its accuracy placed fairly well. However, its F1 on the sarcastic class, the metric that mattered, was weak. The authors trace this gap mainly to choosing the model by accuracy during development. The work therefore serves as a simple baseline and as a warning about mismatched selection metrics on imbalanced data. ## Key points - Compared classifiers: SVM (linear and RBF kernels), Multinomial Naïve Bayes, SGD (hinge loss), MLP, Random Forest (300 trees), KNN, AdaBoost and a hard-voting ensemble. All use unigram TF-IDF features. - 5-fold cross-validation accuracy: MLP 83.6% (best), SVM-Linear 81.0%, Voting 80.4% and SGD 80.0%. AdaBoost was lowest at 75.2%. - The submitted MLP used one hidden layer of 20 nodes, logistic activation and the Adam solver. - Test results: - F1 on the sarcastic class: 0.3746 (rank 14). - Overall F-score: 0.6024 (rank 11). - Precision: 0.5968. Recall: 0.6608. - Accuracy: 0.7329 (rank 8). - Each Arabic example carries: - a dialect label; - an author-given sarcasm label; - for sarcastic tweets, the author's non-sarcastic rephrase. Expert labels follow the six Leggitt and Gibbs categories, which cover rhetorical questions, overstatement, understatement, satire, irony and sarcasm itself. ## Interesting ideas - The team picked its model by cross-validation accuracy, while the leaderboard scored F1 on the minority sarcastic class, so its strong accuracy did not carry over. Selecting by the target metric avoids this trap whenever classes are imbalanced. — For Mova (learn): Evaluation and repair-rule acceptance should be scored by per-class F1 on the rare phenomenon, such as irony or a rare dependency relation, rather than by overall accuracy. - The dataset pairs each sarcastic text with its author's literal rewording, which gives a ground-truth statement of the intended meaning. Such pairs could support evaluation beyond plain classification. — For Mova (frames): The irony module in prag could be tested by checking that the intended-meaning frame it derives from a sarcastic sentence matches the frame of the literal rephrase. - Labels come from the text's own author rather than outside annotators, which captures real intent instead of a reader's guess. This matters for phenomena where readers often disagree. — For Mova (learn): Author-intent labels could serve as a higher-trust witness tier for speech-act and implicature silver data, ranked above LLM or annotator judgments. - The ironic-speech taxonomy breaks a single sarcasm flag into six finer types: sarcasm proper, irony, satire, both under- and overstatement, and rhetorical questions. This gives a richer target than a yes/no label. — For Mova (frames): The prag irony component could output one of these types, each linked to a distinct, explainable inference about what the speaker actually means. - Runs of a repeated character are shortened to two characters instead of one, which normalizes spelling while keeping a trace of emphasis. — For Mova (tokens): The en tokenizer could map elongated words like "sooooo" to a canonical lemma plus an emphasis feature, so the signal can later reach the pragmatics stage. *Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.* Fields: [Linguistics & languages](https://md.research.elamur.ai/free/en/shelves/linguistics/) ## Cite The paper itself, not this page. BibTeX from the library's record of the work. ```bibtex @inproceedings{ashraf2022bfcai, title = {{BFCAI at SemEval-2022 Task 6: Multi-Layer Perceptron for Sarcasm Detection in Arabic Texts}}, author = {Ashraf, Nsrin and Elkazzaz, Fathy and Taha, Mohamed and Nayel, Hamada and Elshishtawy, Tarek}, booktitle = {Proceedings of the 16th International Workshop on Semantic Evaluation (SemEval-2022)}, year = {2022}, doi = {10.18653/v1/2022.semeval-1.123}, eprint = {2205.08868}, archivePrefix = {arXiv}, url = {https://doi.org/10.18653/v1/2022.semeval-1.123} } ``` - [← previous Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models](https://md.research.elamur.ai/free/en/papers/srivastava-2022-imitation-game-quantifying-extrapolating-capabilities/) - [next → Bibliography of Ethiopian Semitic, Cushitic, and Omotic Linguistics XXIV: 2020](https://md.research.elamur.ai/free/en/papers/maria-2022-bibliography-ethiopian-semitic-cushitic-omotic/) - [↑ back to papers of 2022](https://md.research.elamur.ai/free/en/papers/2022/) ## References 11 references, 1 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet. 1. Ibrahim Abu Farha, Silviu Oprea, Steven Wilson, and Walid Magdy. 2022. SemEval-2022 Task 6: iSarcas-mEval, Intended Sarcasm Detection in English and Arabic. In Proceedings of the 16th International Workshop on Semantic Evaluation (SemEval-2022). Association for Computational Linguistics. F1-score for the sarcastic class (F1-sarcastic) has been used for final evaluation. *Table 2: Results of MLP classifier for the test set* Results The cross validation accuracy of all training classifiers for the training set is given in Table 1. It is clear that MLP gives the best accuracy with moderate Standard Deviation (STD) for the five folds while development phase, so we decided to submit the output of this classifier. [find](https://scholar.google.com/scholar?q=Ibrahim+Abu+Farha%2C+Silviu+Oprea%2C+Steven+Wilson%2C+and+Walid+Magdy%2E+2022%2E+SemEval%2D2022+Task+6%3A+iSarcas%2DmEval%2C+Intended+Sarcasm+Detection+in+English+and+Arabic%2E+In+Proceedings+of+the+16th+International+Wo) 2. Abdulaziz M. Alayba, Vasile Palade, Matthew England, and Rahat Iqbal. 2018. Improving sentiment analysis in arabic using word representation. 2018 IEEE 2nd International Workshop on Arabic and Derived Script Analysis and Recognition (ASAR). [find](https://scholar.google.com/scholar?q=Abdulaziz+M%2E+Alayba%2C+Vasile+Palade%2C+Matthew+England%2C+and+Rahat+Iqbal%2E+2018%2E+Improving+sentiment+analysis+in+arabic+using+word+representation%2E+2018+IEEE+2nd+International+Workshop+on+Arabic+and+Derived) 3. REBECCA CLIFT. 1999. Irony in conversation. Language in Society, 28(4):523–553. [find](https://scholar.google.com/scholar?q=REBECCA+CLIFT%2E+1999%2E+Irony+in+conversation%2E+Language+in+Society%2C+28%284%29%3A523%E2%80%93553%2E) 4. Bilal Ghanem, Jihen Karoui, Farah Benamara, Paolo Rosso, and Véronique Moriceau. 2020. Irony detection in a multilingual context. In Advances in Information Retrieval, pages 141–149, Cham. Springer International Publishing. [find](https://scholar.google.com/scholar?q=Bilal+Ghanem%2C+Jihen+Karoui%2C+Farah+Benamara%2C+Paolo+Rosso%2C+and+V%C3%A9ronique+Moriceau%2E+2020%2E+Irony+detection+in+a+multilingual+context%2E+In+Advances+in+Information+Retrieval%2C+pages+141%E2%80%93149%2C+Cham%2E+Springer+In) 5. Jihen Karoui, Farah Banamara Zitoune, and Veronique Moriceau. 2017. Soukhria: Towards an irony detection system for arabic in social media. Procedia Computer Science, 117:161–168. [↗](https://doi.org/10.1016/j.procs.2017.10.105) 6. Muhammad Khalifa and Noura Hussein. 2019. Ensemble learning for irony detection in arabic tweets. In Working Notes of FIRE 2019 - Forum for Information Retrieval Evaluation, Kolkata, India, December 12-15, 2019, volume 2517 of CEUR Workshop Proceedings, pages 433–438. CEUR-WS.org. [find](https://scholar.google.com/scholar?q=Muhammad+Khalifa+and+Noura+Hussein%2E+2019%2E+Ensemble+learning+for+irony+detection+in+arabic+tweets%2E+In+Working+Notes+of+FIRE+2019+%2D+Forum+for+Information+Retrieval+Evaluation%2C+Kolkata%2C+India%2C+December+1) 7. Christopher J. Lee and Albert N. Katz. 1998. The differential role of ridicule in sarcasm and irony. Metaphor and Symbol, 13(1):1–15. [find](https://scholar.google.com/scholar?q=Christopher+J%2E+Lee+and+Albert+N%2E+Katz%2E+1998%2E+The+differential+role+of+ridicule+in+sarcasm+and+irony%2E+Metaphor+and+Symbol%2C+13%281%29%3A1%E2%80%9315%2E) 8. John S Leggitt and Raymond W Gibbs. 2000. Emotional reactions to verbal irony. Discourse processes, 29(1):1–24. [↗](https://doi.org/10.1207/s15326950dp2901_1) 9. Hamada Nayel, Eslam Amer, Aya Allam, and Hanya Abdallah. 2021. Machine learning-based model for sentiment and sarcasm detection. In Proceedings of the Sixth Arabic Natural Language Processing Workshop, pages 386–389, Kyiv, Ukraine (Virtual). Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Hamada+Nayel%2C+Eslam+Amer%2C+Aya+Allam%2C+and+Hanya+Abdallah%2E+2021%2E+Machine+learning%2Dbased+model+for+sentiment+and+sarcasm+detection%2E+In+Proceedings+of+the+Sixth+Arabic+Natural+Language+Processing+Workshop) [our card](https://md.research.elamur.ai/free/en/papers/nayel-2021-machine-learning-based-model-sentiment/) 10. Hamada A. Nayel, Walaa Medhat, and Metwally Rashad. 2019. Benha@idat: Improving irony detection in arabic tweets using ensemble approach. In Working Notes of FIRE 2019 - Forum for Information Retrieval Evaluation, Kolkata, India, December 12-15, 2019, volume 2517 of CEUR Workshop Proceedings, pages 401–408. CEUR-WS.org. [find](https://scholar.google.com/scholar?q=Hamada+A%2E+Nayel%2C+Walaa+Medhat%2C+and+Metwally+Rashad%2E+2019%2E+Benha%40idat%3A+Improving+irony+detection+in+arabic+tweets+using+ensemble+approach%2E+In+Working+Notes+of+FIRE+2019+%2D+Forum+for+Information+Retrieva) 11. Antonio Reyes, Paolo Rosso, and Tony Veale. 2013. A multidimensional approach for detecting irony in twitter. Lang. Resour. Eval., 47(1):239–268. [find](https://scholar.google.com/scholar?q=Antonio+Reyes%2C+Paolo+Rosso%2C+and+Tony+Veale%2E+2013%2E+A+multidimensional+approach+for+detecting+irony+in+twitter%2E+Lang%2E+Resour%2E+Eval%2E%2C+47%281%29%3A239%E2%80%93268%2E)