---
title: "YNU-HPCC at SemEval-2018 Task 3 — paper by Peng et al. (2018)"
description: "The paper describes the YNU-HPCC system for SemEval-2018 Task 3, which asks whether English tweets are… 0.655 F1. 13 references, cited by 2 papers here."
authors: ["Bo Peng", "Jin Wang", "Xuejie Zhang"]
year: 2018
venue: "Proceedings of the 12th International Workshop on Semantic Evaluation"
source: "https://aclanthology.org/S18-1101/"
doi: "10.18653/v1/s18-1101"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/peng-2018-ynu-hpcc-semeval-2018-task/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/peng-2018-ynu-hpcc-semeval-2018-task/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2018](https://md.research.elamur.ai/free/en/papers/2018/page/5/) / YNU-HPCC at SemEval-2018 Task 3: Ensemble Neural Network Models for Irony Detection on Twitter

# YNU-HPCC at SemEval-2018 Task 3: Ensemble Neural Network Models for Irony Detection on Twitter

**Authors:** Bo Peng, [Jin Wang](https://md.research.elamur.ai/free/en/authors/jin-wang/) and [Xuejie Zhang](https://md.research.elamur.ai/free/en/authors/xuejie-zhang/) · **Year:** 2018 · **Venue:** [Proceedings of the 12th International Workshop on Semantic Evaluation](https://md.research.elamur.ai/free/en/venues/semeval/) ([SemEval 2018](https://md.research.elamur.ai/free/en/venues/semeval/2018/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/S18-1101/) · [PDF](https://aclanthology.org/S18-1101.pdf) · [DOI 10.18653/v1/s18-1101](https://doi.org/10.18653/v1/s18-1101) · [OpenAlex](https://openalex.org/W2806239564)

**Citations:** 12 citations (OpenAlex, 2026-10-14) · [2](https://md.research.elamur.ai/free/en/papers/peng-2018-ynu-hpcc-semeval-2018-task/cited-by/) in our library, of the 141,840 papers whose bibliographies we read

## Summary
The paper describes the YNU-HPCC system for SemEval-2018 Task 3, which asks whether English tweets are ironic. Subtask A is a binary ironic/non-ironic decision. Subtask B uses four labels: irony through polarity clash, other verbal irony, situational irony, and non-irony. The organizers built the training set of 3834 tweets by searching for #irony, #sarcasm and #not.

The authors trained six recurrent classifiers: plain and bidirectional variants of both the LSTM and the GRU, plus a bidirectional LSTM and a bidirectional GRU each extended with an attention layer. Each one received 300-dimensional GloVe vectors trained on Wikipedia. A majority vote combined their predictions. Before training, the tweets were normalised: users, URLs, emoticons and hashtags were replaced with placeholders or words.

In cross-validation the ensemble gained only slightly over its best member. On the official test set, scores fell sharply, to a macro F1 of about 0.41 for A and 0.31 for B. The authors explain the weak ensemble gain by correlated errors: the six models are architecturally similar and fail on the same inputs. The work is mainly useful as a cautionary baseline for neural irony detection and for naive ensembling.

## Key points
- Training data: 3834 tweets, 1911 of them ironic, so subtask A is roughly balanced. Hashtags and emoticons were kept because they carry affect. The vocabulary had 9056 unique words, and 1266 of them had no GloVe vector, mostly numbers and user-coined words.
- Setup: Keras on TensorFlow, Adagrad, categorical cross-entropy, batch size 100, dropout 0.25 and 30 epochs. Early stopping (patience 3) informed the epoch count. There was no dev set, so cross-validation was used instead.
- Cross-validation results for the ensemble:
  - Subtask A: accuracy 0.660, macro F1 0.669; the best single model was attention BiGRU at 0.655 F1.
  - Subtask B: accuracy 0.754, macro F1 0.517.
- Official test results:
  - Subtask A: accuracy 0.509, F1 0.409.
  - Subtask B: accuracy 0.466, F1 0.313, after correcting a wrong initial submission.
  - Precision in B ranked 10th of 32 teams.
- Preprocessing details: positive emoticons became words such as "smile" or "happy", while negative ones became `<irony>`. In A, every hashtag became `<hashtag>`. In B, the hashtags #irony, #sarcasm and #not were turned into the word "irony", and other hashtags were masked.

## Interesting ideas
- Voting only pays off when the members make different mistakes; six near-identical RNNs mostly erred together, so the vote added little.
- The labels came from hashtag searches, so leaving those hashtags visible lets a model learn the collection method rather than irony; the authors masked them in A to avoid overfitting.
- The large drop from cross-validation to the hidden test set shows that in-sample validation on a small, distantly labelled corpus can badly overstate quality.
- Emoticons were mapped to explicit sentiment words or a special negative token, turning non-lexical signals into tokens the model can use.
- The authors suggest that irony detection should exploit the inconsistency inside ironic sentences, which matches the task's "polarity clash" category.

## For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

- **I1** (learn): When choosing witness parsers for silver data, pick structurally different ones (perceptron vs. gparse vs. rule-based) and measure their error overlap, since agreement among clones is weak evidence.
- **I2** (learn): Build negative controls that strip the cues used to construct a dataset, and check whether accuracy survives, to detect shortcut learning.
- **I3** (learn): Always report results on held-out data collected differently from the training material, alongside internal cross-validation.
- **I4** (tokens): The tokenizer could map emoji and emoticons to typed sentiment tokens with a lemma, so later stages can reason over them explicitly.
- **I5** (frames): The prag crate could flag irony when the literal sentiment of a frame contradicts the evaluated state of the situation in the world model, giving an explainable irony signal instead of a black-box score.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Yunnan University](https://md.research.elamur.ai/free/en/universities/yunnan-university/) — University, Kunming, China · 2 works · 23 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 9,086 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 8,889 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 9,296 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 9,206 works

🧭 Topics (2): [English](https://md.research.elamur.ai/free/en/topics/english/) · 1,373 works · [😊 Sentiment & emotion](https://md.research.elamur.ai/free/en/topics/sentiment-emotion/) · 608 works

## Cite

Cite as: Bo Peng, Jin Wang and Xuejie Zhang. 2018. YNU-HPCC at SemEval-2018 Task 3: Ensemble Neural Network Models for Irony Detection on Twitter. Proceedings of the 12th International Workshop on Semantic Evaluation.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{peng2018ynu,
  title = {{YNU-HPCC at SemEval-2018 Task 3: Ensemble Neural Network Models for Irony Detection on Twitter}},
  author = {Bo Peng, Jin Wang and Xuejie Zhang},
  booktitle = {Proceedings of the 12th International Workshop on Semantic Evaluation},
  year = {2018},
  doi = {10.18653/v1/s18-1101},
  url = {https://doi.org/10.18653/v1/s18-1101}
}
```

- [← previous Universal Dependencies and Quantitative Typological Trends. A Case Study on Word Order](https://md.research.elamur.ai/free/en/papers/alzetta-2018-universal-dependencies-quantitative-typological-trends/)
- [next → Integrating Predictions from Neural-Network Relation Classifiers into Coreference and Bridging Resolution](https://md.research.elamur.ai/free/en/papers/roesiger-2018-integrating-predictions-neural-network-relation/)
- [↑ back to papers of 2018](https://md.research.elamur.ai/free/en/papers/2018/page/5/)

## References

13 references, 6 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Silvio Amir, Byron C Wallace, Hao Lyu, and Paula Carvalho Mrio J Silva. 2016. Modelling context with user embeddings for sarcasm detection in social media. pages 167–177. [our card](https://md.research.elamur.ai/free/en/papers/amir-2016-modelling-context-user-embeddings-sarcasm/)
2. Mark J. Carman, Mark J. Carman, and Mark J. Carman. 2017. Automatic Sarcasm Detection: A Survey. ACM. [our card](https://md.research.elamur.ai/free/en/papers/joshi-2016-automatic-sarcasm-detection-survey/)
3. Kyunghyun Cho, Bart Van Merrienboer, Caglar Gulcehre, Dzmitry Bahdanau, Fethi Bougares, Holger Schwenk, and Yoshua Bengio. 2014. Learning phrase representations using rnn encoder-decoder for statistical machine translation. Computer Science. [↗](https://doi.org/10.48550/arxiv.1406.1078)
4. Mathieu Cliche. 2017. BB t wtr at semeval-2017 task 4: Twitter sentiment analysis with cnns and lstms. [find](https://scholar.google.com/scholar?q=Mathieu+Cliche%2E+2017%2E+BB+t+wtr+at+semeval%2D2017+task+4%3A+Twitter+sentiment+analysis+with+cnns+and+lstms%2E)
5. Els Lefever Cynthia Van Hee and Vronique Hoste. 2018. Semeval-2018 task 3: Irony detection in english tweets. In Proceedings of the 12th International Workshop on Semantic Evaluation (SemEval-2018). [our card](https://md.research.elamur.ai/free/en/papers/hee-2018-semeval-2018-task-3-irony/)
6. John Duchi, Elad Hazan, and Yoram Singer. 2011. Adaptive subgradient methods for online learning and stochastic optimization. Journal of Machine Learning Research, 12(7):257–269. [find](https://scholar.google.com/scholar?q=John+Duchi%2C+Elad+Hazan%2C+and+Yoram+Singer%2E+2011%2E+Adaptive+subgradient+methods+for+online+learning+and+stochastic+optimization%2E+Journal+of+Machine+Learning+Research%2C+12%287%29%3A257%E2%80%93269%2E)
7. Aniruddha Ghosh and Tony Veale. 2016. Fracking sarcasm using neural network. In The Workshop on Computational Approaches To Subjectivity. [↗](https://doi.org/10.13140/rg.2.2.16560.15363) [our card](https://md.research.elamur.ai/free/en/papers/ghosh-2016-fracking-sarcasm-neural-network/)
8. R. Maclin and D. Opitz. 1999. Popular ensemble methods: An empirical study. Journal of Artificial Intelligence Research, 11:169–198. [↗](https://doi.org/10.1613/jair.614)
9. Tomas Mikolov, Martin Karafit, Lukas Burget, Jan Cernocky, and Sanjeev Khudanpur. 2010. Recurrent neural network based language model. In IN-TERSPEECH 2010, Conference of the International Speech Communication Association, Makuhari, Chiba, Japan, September, pages 1045–1048. [find](https://scholar.google.com/scholar?q=Tomas+Mikolov%2C+Martin+Karafit%2C+Lukas+Burget%2C+Jan+Cernocky%2C+and+Sanjeev+Khudanpur%2E+2010%2E+Recurrent+neural+network+based+language+model%2E+In+IN%2DTERSPEECH+2010%2C+Conference+of+the+International+Speech+Comm)
10. Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg Corrado, and Jeffrey Dean. 2013. Distributed representations of words and phrases and their compositionality. 26:3111–3119. [↗](https://doi.org/10.48550/arxiv.1310.4546) [our card](https://md.research.elamur.ai/free/en/papers/mikolov-2013-distributed-representations-words-phrases-their/)
11. Jeffrey Pennington, Richard Socher, and Christopher Manning. 2014. Glove: Global vectors for word representation. In Conference on Empirical Methods in Natural Language Processing, pages 1532– 1543. [our card](https://md.research.elamur.ai/free/en/papers/pennington-2014-glove-global-vectors-word-representation/)
12. Ha?im Sak, Andrew Senior, and Fran?oise Beaufays. 2014. Long short-term memory based recurrent neural network architectures for large vocabulary speech recognition. Computer Science, pages 338– 342. [find](https://scholar.google.com/scholar?q=Ha%3Fim+Sak%2C+Andrew+Senior%2C+and+Fran%3Foise+Beaufays%2E+2014%2E+Long+short%2Dterm+memory+based+recurrent+neural+network+architectures+for+large+vocabulary+speech+recognition%2E+Computer+Science%2C+pages+338%E2%80%93+342%2E)
13. Mike Schuster and Kuldip K Paliwal. 1997. Bidirectional recurrent neural networks. IEEE Press. [↗](https://doi.org/10.1109/78.650093)

*Updated 24 October 2026*