---
title: "Application and Analysis of a — Cignarella 2018"
description: "Irony detection remains difficult for both people and NLP systems, and recent shared tasks increasingly call for fine-grained labels rather than a simple…"
authors: ["Alessandra Teresa Cignarella", "Cristina Bosco", "Viviana Patti", "Mirko Lai"]
year: 2018
venue: "Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018)"
source: "https://aclanthology.org/L18-1664/"
licence: "CC BY-NC 4.0 (ELRA, LREC proceedings)"
licence_url: "https://creativecommons.org/licenses/by-nc/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/cignarella-2018-application-analysis-multi-layered-scheme/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/cignarella-2018-application-analysis-multi-layered-scheme/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2018](https://md.research.elamur.ai/free/en/papers/2018/) / Application and Analysis of a Multi-layered Scheme for Irony on the Italian Twitter Corpus TWITTIRÒ

# Application and Analysis of a Multi-layered Scheme for Irony on the Italian Twitter Corpus TWITTIRÒ

**Authors:** [Alessandra Teresa Cignarella](https://md.research.elamur.ai/free/en/authors/alessandra-teresa-cignarella/), [Cristina Bosco](https://md.research.elamur.ai/free/en/authors/cristina-bosco/), [Viviana Patti](https://md.research.elamur.ai/free/en/authors/viviana-patti/), Mirko Lai · **Year:** 2018 · **Venue:** Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018) · **Licence:** [CC BY-NC 4.0 (ELRA, LREC proceedings)](https://creativecommons.org/licenses/by-nc/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/L18-1664/) · [PDF](https://aclanthology.org/L18-1664.pdf) · [OpenAlex](https://openalex.org/W2806455139)

**Citations:** 11 citations (openalex-snapshot · 2026-10-14) · [8](#cited-by) in our library, of the 140673 papers whose bibliographies we read

## Summary
Irony detection remains difficult for both people and NLP systems, and recent shared tasks increasingly call for fine-grained labels rather than a simple ironic/non-ironic decision. The authors build TWITTIRÒ, an Italian Twitter resource annotated with the multi-layered irony scheme of Karoui et al. (2017). They started from an existing 400-tweet Italian set already annotated with this scheme and added 1,200 ironic tweets drawn from three earlier corpora: TW-SPINO, TW-SENTIPOLC14 and TW-BS. They kept the original sub-corpus proportions. Trained annotators labelled each tweet for two things: whether the contradiction triggering irony is explicit or implicit, and which rhetorical device produces it. The paper analyses how labels are distributed and where annotators disagree, and it examines Italian cases the scheme handles poorly. The outcome is a gold standard of roughly 1,500 tweets with fine-grained irony labels. It is intended as reference data for EVALITA 2018 and is compatible with parallel French and English data annotated under the same scheme, which supports cross-lingual research.

## Key points
- **Sources:** The new tweets come from three sources with different styles.
  - TW-SPINO contains satirical posts from Spinoza.it written in a high register.
  - TW-SENTIPOLC14 contains colloquial political tweets about the Monti government (2011–2012).
  - TW-BS covers the "Buona Scuola" school reform debate and makes heavy use of ALL CAPS.
- **Annotation process:** Two annotators labelled all 1,200 new tweets and disagreed on 544. A third annotator resolved 410 of these, and the remaining 134 were discarded.
- **Level 2 (irony activation):** Each tweet is marked as EXPLICIT or IMPLICIT. Level 1 (ironic vs. not) was skipped because all collected tweets are ironic.
- **Level 3 (irony categories):** The categories are ANALOGY, EUPHEMISM, HYPERBOLE/EXAGGERATION, CONTEXT SHIFT, OXYMORON/PARADOX, FALSE ASSERTION, RHETORICAL QUESTION and OTHER. Some categories apply only to explicit or only to implicit irony.
- **Collection constraint:** Italian users have no stable self-marking hashtag like #irony or #sarcasm. As a result, the hashtag-based collection used for the French and English datasets could not be reused.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

Fields: [Linguistics & languages](https://md.research.elamur.ai/free/en/shelves/linguistics/)

## Cited by

Cited by

8 papers of our library cite it (of the 140673 whose bibliographies we read); 1 of them have a page here, 7 not yet.

- [Overview of the EVALITA 2018 Task on Irony Detection in Italian Tweets (IronITA)](https://md.research.elamur.ai/free/en/papers/cignarella-2018-overview-evalita-2018-task-irony/)

## Cite

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{cignarella2018application,
  title = {{Application and Analysis of a Multi-layered Scheme for Irony on the Italian Twitter Corpus TWITTIRÒ}},
  author = {Cignarella, Alessandra Teresa and Bosco, Cristina and Patti, Viviana and Lai, Mirko},
  booktitle = {Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018)},
  year = {2018},
  doi = {10.63317/582d597s9i68},
  url = {https://doi.org/10.63317/582d597s9i68}
}
```

- [← previous Annotation of Tense and Aspect Semantics for Sentential AMR](https://md.research.elamur.ai/free/en/papers/donatelli-2018-annotation-tense-aspect-semantics-sentential/)
- [next → Approximate Dynamic Oracle for Dependency Parsing with Reinforcement Learning](https://md.research.elamur.ai/free/en/papers/ud-2018-yu-approximate-dynamic-oracle-dependency-parsing/)
- [↑ back to papers of 2018](https://md.research.elamur.ai/free/en/papers/2018/)

## References

23 references, 3 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Barbieri, F., Basile, V., Croce, D., Nissim, M., Novielli, N., and Patti, V. (2016). Overview of the Evalita 2016 SEN-TIment POLarity Classification Task. In Proceedings of Third Italian Conference on Computational Linguistics (CLiC-it 2016) & Fifth Evaluation Campaign of Natural Language Processing… [find](https://scholar.google.com/scholar?q=Barbieri%2C+F%2E%2C+Basile%2C+V%2E%2C+Croce%2C+D%2E%2C+Nissim%2C+M%2E%2C+Novielli%2C+N%2E%2C+and+Patti%2C+V%2E+%282016%29%2E+Overview+of+the+Evalita+2016+SEN%2DTIment+POLarity+Classification+Task%2E+In+Proceedings+of+Third+Italian+Conference+on)
2. Final Workshop (EVALITA 2016), Napoli, Italy, December 5-7, 2016., volume 1749 of CEUR Workshop Proceedings. [find](https://scholar.google.com/scholar?q=Final+Workshop+%28EVALITA+2016%29%2C+Napoli%2C+Italy%2C+December+5%2D7%2C+2016%2E%2C+volume+1749+of+CEUR+Workshop+Proceedings%2E)
3. CEUR-WS.org. Basile, V., Bolioli, A., Nissim, M., Patti, V., and Rosso, P. (2014). Overview of the Evalita 2014 SENTIment POLarity Classification Task. In Proceedings of the 4th Evaluation Campaign of Natural Language Processing and Speech tools for Italian (EVALITA’14), pages 50–57, Pisa, Italy. Pi… [find](https://scholar.google.com/scholar?q=CEUR%2DWS%2Eorg%2E+Basile%2C+V%2E%2C+Bolioli%2C+A%2E%2C+Nissim%2C+M%2E%2C+Patti%2C+V%2E%2C+and+Rosso%2C+P%2E+%282014%29%2E+Overview+of+the+Evalita+2014+SENTIment+POLarity+Classification+Task%2E+In+Proceedings+of+the+4th+Evaluation+Campaign+of)
4. Bosco, C., Patti, V., and Bolioli, A. (2013). Developing Corpora for Sentiment Analysis: The Case of Irony and Senti-TUT. IEEE Intelligent Systems, 28(2):55–63. [find](https://scholar.google.com/scholar?q=Bosco%2C+C%2E%2C+Patti%2C+V%2E%2C+and+Bolioli%2C+A%2E+%282013%29%2E+Developing+Corpora+for+Sentiment+Analysis%3A+The+Case+of+Irony+and+Senti%2DTUT%2E+IEEE+Intelligent+Systems%2C+28%282%29%3A55%E2%80%9363%2E)
5. Cignarella, A. T., Bosco, C., and Patti, V. (2017). TWIT-TIRÒ: a Social Media Corpus with a Multi-layered Annotation for Irony. In Proceedings of the Fourth Italian Conference on Computational Linguistics (CLiC-it 2017), Rome, Italy, December 11-13, 2017, volume 2006 of CEUR Workshop Proceedings. C… [find](https://scholar.google.com/scholar?q=Cignarella%2C+A%2E+T%2E%2C+Bosco%2C+C%2E%2C+and+Patti%2C+V%2E+%282017%29%2E+TWIT%2DTIRO%CC%80%3A+a+Social+Media+Corpus+with+a+Multi%2Dlayered+Annotation+for+Irony%2E+In+Proceedings+of+the+Fourth+Italian+Conference+on+Computational+Lingui)
6. Fink, C. R., Chou, D. S., Kopecky, J. J., and Llorens, A. J. (2011). Coarse- and Fine-Grained Sentiment Analysis of Social Media Text. Johns Hopkins APL Technical Digest, 30(1):22–30. [find](https://scholar.google.com/scholar?q=Fink%2C+C%2E+R%2E%2C+Chou%2C+D%2E+S%2E%2C+Kopecky%2C+J%2E+J%2E%2C+and+Llorens%2C+A%2E+J%2E+%282011%29%2E+Coarse%2D+and+Fine%2DGrained+Sentiment+Analysis+of+Social+Media+Text%2E+Johns+Hopkins+APL+Technical+Digest%2C+30%281%29%3A22%E2%80%9330%2E)
7. Ghosh, A., Li, G., Veale, T., Rosso, P., Shutova, E., Barnden, J., and Reyes, A. (2015). Semeval-2015 task 11: Sentiment Analysis of Figurative Language in Twitter. In Proceedings of SemEval 2015, Co-located with NAACL, page 470–478. [find](https://scholar.google.com/scholar?q=Ghosh%2C+A%2E%2C+Li%2C+G%2E%2C+Veale%2C+T%2E%2C+Rosso%2C+P%2E%2C+Shutova%2C+E%2E%2C+Barnden%2C+J%2E%2C+and+Reyes%2C+A%2E+%282015%29%2E+Semeval%2D2015+task+11%3A+Sentiment+Analysis+of+Figurative+Language+in+Twitter%2E+In+Proceedings+of+SemEval+2015%2C+Co%2D)
8. ACL. Grice, P. H. (1975). Logic and Conversation. Syntax and Semantics 3: Speech Arts, pages 41–58. [find](https://scholar.google.com/scholar?q=ACL%2E+Grice%2C+P%2E+H%2E+%281975%29%2E+Logic+and+Conversation%2E+Syntax+and+Semantics+3%3A+Speech+Arts%2C+pages+41%E2%80%9358%2E)
9. Grice, P. H. (1978). Further Notes on Logic and Conversation. Pragmatics, 1:13–128. Hernández Farı́as, D. I., Patti, V., and Rosso, P. (2016). Irony Detection in Twitter: The Role of Affective Content. ACM Transactions on Internet Technologies, 16(3):19:1–19:24. [find](https://scholar.google.com/scholar?q=Grice%2C+P%2E+H%2E+%281978%29%2E+Further+Notes+on+Logic+and+Conversation%2E+Pragmatics%2C+1%3A13%E2%80%93128%2E+Herna%CC%81ndez+Far%C4%B1%CC%81as%2C+D%2E+I%2E%2C+Patti%2C+V%2E%2C+and+Rosso%2C+P%2E+%282016%29%2E+Irony+Detection+in+Twitter%3A+The+Role+of+Affective+Conten)
10. Karoui, J., Benamara, F., Moriceau, V., Aussenac-Gilles, N., and Belguith, L. H. (2015). Towards a contextual pragmatic model to detect irony in tweets. In 53rd Annual Meeting of the Association for Computational Linguistics (ACL 2015). [find](https://scholar.google.com/scholar?q=Karoui%2C+J%2E%2C+Benamara%2C+F%2E%2C+Moriceau%2C+V%2E%2C+Aussenac%2DGilles%2C+N%2E%2C+and+Belguith%2C+L%2E+H%2E+%282015%29%2E+Towards+a+contextual+pragmatic+model+to+detect+irony+in+tweets%2E+In+53rd+Annual+Meeting+of+the+Association+for+C) [our card](https://md.research.elamur.ai/free/en/papers/karoui-2015-contextual-pragmatic-model-detect-irony/)
11. Karoui, J., Benamara, F., Moriceau, V., Patti, V., and Bosco, C. (2017). Exploring the Impact of Pragmatic Phenomena on Irony Detection in Tweets: A Multilingual Corpus Study. In Proceedings of the European Chapter of the Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Karoui%2C+J%2E%2C+Benamara%2C+F%2E%2C+Moriceau%2C+V%2E%2C+Patti%2C+V%2E%2C+and+Bosco%2C+C%2E+%282017%29%2E+Exploring+the+Impact+of+Pragmatic+Phenomena+on+Irony+Detection+in+Tweets%3A+A+Multilingual+Corpus+Study%2E+In+Proceedings+of+the+Eu) [our card](https://md.research.elamur.ai/free/en/papers/karoui-2017-exploring-impact-pragmatic-phenomena-irony/)
12. Kouloumpis, E., Wilson, T., and Moore, J. D. (2011). Twitter sentiment analysis: The good the bad and the omg! In Proceedings of the ICWSM: International AAAI Conference on Web and Social Media. [find](https://scholar.google.com/scholar?q=Kouloumpis%2C+E%2E%2C+Wilson%2C+T%2E%2C+and+Moore%2C+J%2E+D%2E+%282011%29%2E+Twitter+sentiment+analysis%3A+The+good+the+bad+and+the+omg%21+In+Proceedings+of+the+ICWSM%3A+International+AAAI+Conference+on+Web+and+Social+Media%2E)
13. Landis, J. R. and Koch, G. G. (1977). The measurement of observer agreement for categorical data. biometrics, pages 159–174. [find](https://scholar.google.com/scholar?q=Landis%2C+J%2E+R%2E+and+Koch%2C+G%2E+G%2E+%281977%29%2E+The+measurement+of+observer+agreement+for+categorical+data%2E+biometrics%2C+pages+159%E2%80%93174%2E)
14. Maynard, D. and Funk, A. (2011). Automatic Detection of Political Opinions in Tweets. In Proceedings of the ESWC: Extended Semantic Web Conference. [find](https://scholar.google.com/scholar?q=Maynard%2C+D%2E+and+Funk%2C+A%2E+%282011%29%2E+Automatic+Detection+of+Political+Opinions+in+Tweets%2E+In+Proceedings+of+the+ESWC%3A+Extended+Semantic+Web+Conference%2E)
15. Mihalcea, R. and Pulman, S. (2007). Characterizing Humour: An Exploration of Features in Humorous Texts. In International Conference on Intelligent Text Processing and Computational Linguistics. [find](https://scholar.google.com/scholar?q=Mihalcea%2C+R%2E+and+Pulman%2C+S%2E+%282007%29%2E+Characterizing+Humour%3A+An+Exploration+of+Features+in+Humorous+Texts%2E+In+International+Conference+on+Intelligent+Text+Processing+and+Computational+Linguistics%2E)
16. Reyes, A., Rosso, P., and Buscaldi, D. (2010). Finding Humour in the Blogosphere: the Role of Wordnet Resources. In Proceedings of the 5th Global WordNet Conference. [find](https://scholar.google.com/scholar?q=Reyes%2C+A%2E%2C+Rosso%2C+P%2E%2C+and+Buscaldi%2C+D%2E+%282010%29%2E+Finding+Humour+in+the+Blogosphere%3A+the+Role+of+Wordnet+Resources%2E+In+Proceedings+of+the+5th+Global+WordNet+Conference%2E)
17. Reyes, A., Rosso, P., and Buscaldi, D. (2012). From Humor Recognition to Irony Detection: The Figurative Language of Social Media. Data & Knowledge Engineering, 74:1–12. [find](https://scholar.google.com/scholar?q=Reyes%2C+A%2E%2C+Rosso%2C+P%2E%2C+and+Buscaldi%2C+D%2E+%282012%29%2E+From+Humor+Recognition+to+Irony+Detection%3A+The+Figurative+Language+of+Social+Media%2E+Data+%26+Knowledge+Engineering%2C+74%3A1%E2%80%9312%2E)
18. Sanguinetti, M., Bosco, C., Mazzei, A., Lavelli, A., and Tamburini, F. (2017). Annotating Italian social media texts in Universal Dependencies. In Proceedings of the Fourth International Conference on Dependency Linguistics (Depling 2017), pages 229–239. Sanguinetti, M., Bosco, C., Lavelli, A., Mazz… [find](https://scholar.google.com/scholar?q=Sanguinetti%2C+M%2E%2C+Bosco%2C+C%2E%2C+Mazzei%2C+A%2E%2C+Lavelli%2C+A%2E%2C+and+Tamburini%2C+F%2E+%282017%29%2E+Annotating+Italian+social+media+texts+in+Universal+Dependencies%2E+In+Proceedings+of+the+Fourth+International+Conference+on)
19. Sperber, D. and Wilson, D. (1981). Irony and the Use-Mention Distinction. Philosophy, 3:143–184. Stranisci, M., Bosco, C., Farı́as, D. I. H., and Patti, V. (2016). Annotating sentiment and irony in the online italian political debate on \#labuonascuola. In Proceedings of the Tenth International Confe… [find](https://scholar.google.com/scholar?q=Sperber%2C+D%2E+and+Wilson%2C+D%2E+%281981%29%2E+Irony+and+the+Use%2DMention+Distinction%2E+Philosophy%2C+3%3A143%E2%80%93184%2E+Stranisci%2C+M%2E%2C+Bosco%2C+C%2E%2C+Far%C4%B1%CC%81as%2C+D%2E+I%2E+H%2E%2C+and+Patti%2C+V%2E+%282016%29%2E+Annotating+sentiment+and+irony+in+th)
20. Van Hee, C., Lefever, E., and Hoste, V. (2016). Exploring the realization of irony in Twitter data. In Nicoletta Calzolari, et al., editors, Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC 2016). European Language Resources Association (ELRA). Van Hee, C.… [find](https://scholar.google.com/scholar?q=Van+Hee%2C+C%2E%2C+Lefever%2C+E%2E%2C+and+Hoste%2C+V%2E+%282016%29%2E+Exploring+the+realization+of+irony+in+Twitter+data%2E+In+Nicoletta+Calzolari%2C+et+al%2E%2C+editors%2C+Proceedings+of+the+Tenth+International+Conference+on+Langua) [our card](https://md.research.elamur.ai/free/en/papers/hee-2016-exploring-realization-irony-twitter-data/)
21. New Orleans, LA, USA, June 2018. [find](https://scholar.google.com/scholar?q=New+Orleans%2C+LA%2C+USA%2C+June+2018%2E)
22. Van Hee, C. (2017). Can machines sense irony?: exploring automatic irony detection on social media. Ph.D. thesis, Ghent University. [find](https://scholar.google.com/scholar?q=Van+Hee%2C+C%2E+%282017%29%2E+Can+machines+sense+irony%3F%3A+exploring+automatic+irony+detection+on+social+media%2E+Ph%2ED%2E+thesis%2C+Ghent+University%2E)
23. Wilson, D. and Sperber, D. (2007). On verbal irony. Irony in language and thought, pages 35–56. [find](https://scholar.google.com/scholar?q=Wilson%2C+D%2E+and+Sperber%2C+D%2E+%282007%29%2E+On+verbal+irony%2E+Irony+in+language+and+thought%2C+pages+35%E2%80%9356%2E)