---
title: "ValenTo — paper by Farías et al. (2015) — summary and ideas"
description: "The paper presents the ValenTo team's entry to SemEval-2015 Task 11. 24 references, cited by 2 papers here."
authors: ["Delia Irazú Hernández Farías", "Emilio Sulis", "Viviana Patti", "Giancarlo Ruffo", "Cristina Bosco"]
year: 2015
venue: "Proceedings of the 9th International Workshop on Semantic Evaluation (SemEval 2015)"
source: "https://aclanthology.org/S15-2117/"
doi: "10.18653/v1/s15-2117"
licence: "CC BY-NC-SA 3.0 (ACL Anthology, before 2016)"
licence_url: "https://creativecommons.org/licenses/by-nc-sa/3.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/farias-2015-valento-sentiment-analysis-figurative-language/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/farias-2015-valento-sentiment-analysis-figurative-language/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2015](https://md.research.elamur.ai/free/en/papers/2015/page/2/) / ValenTo: Sentiment Analysis of Figurative Language Tweets with Irony and Sarcasm

# ValenTo: Sentiment Analysis of Figurative Language Tweets with Irony and Sarcasm

**Authors:** [Delia Irazú Hernández Farías](https://md.research.elamur.ai/free/en/authors/delia-irazu-hernandez-farias/), Emilio Sulis, [Viviana Patti](https://md.research.elamur.ai/free/en/authors/viviana-patti/), Giancarlo Ruffo and [Cristina Bosco](https://md.research.elamur.ai/free/en/authors/cristina-bosco/) · **Year:** 2015 · **Venue:** [Proceedings of the 9th International Workshop on Semantic Evaluation (SemEval 2015)](https://md.research.elamur.ai/free/en/venues/semeval/) ([SemEval 2015](https://md.research.elamur.ai/free/en/venues/semeval/2015/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY-NC-SA 3.0 (ACL Anthology, before 2016)](https://creativecommons.org/licenses/by-nc-sa/3.0/)

**Links:** [Publisher's page](https://aclanthology.org/S15-2117/) · [PDF](https://aclanthology.org/S15-2117.pdf) · [DOI 10.18653/v1/s15-2117](https://doi.org/10.18653/v1/s15-2117) · [OpenAlex](https://openalex.org/W2251295382)

**Citations:** 26 citations (OpenAlex, 2026-10-14) · [2](https://md.research.elamur.ai/free/en/papers/farias-2015-valento-sentiment-analysis-figurative-language/cited-by/) in our library, of the 141,840 papers whose bibliographies we read

## Summary
The paper presents the ValenTo team's entry to SemEval-2015 Task 11. The task asks systems to score the sentiment of tweets that contain irony, sarcasm or metaphor on an 11-point scale from −5 to +5. Such figurative devices often flip the literal polarity, and that misleads standard sentiment tools. The authors could still download 7,390 tweets of the roughly 9,000 released. From these they extracted two kinds of features:

- **Structural cues:** length, punctuation, POS counts, uppercase, URLs, emoticons, hashtags, mentions, retweets, and a polarity-flip flag for #sarcasm/#not.
- **Lexicon scores:** many affective, emotion and psycholinguistic resources, such as AFINN, ANEW, DAL, SenticNet, LIWC and NRC.

Classification worked poorly once the label set grew beyond three classes, so they switched to linear regression in Weka. The submitted system ranked 6th of 15 teams, with CS 0.634 and MSE 2.999. Its results on the ironic and sarcastic subsets came close to the winners'. The work suggests that word-level emotional and psycholinguistic knowledge helps with sarcasm and irony, but not much with metaphor.

## Key points
- **Data:** 7,390 tweets were recoverable for training plus trial. The polarity distribution leans strongly negative, with a median of −2.3 and a mean of −2.
- **Classification (10-fold CV on 6,928 tweets):**
  - With 3 classes, F-measure was about 0.79–0.83 (best: NB at 0.829).
  - With 4 classes, it fell to about 0.46.
  - With all 11 classes, it fell to about 0.30–0.32.
- **Regression:** Weka linear regression with feature set ft-conf2 reached cosine similarity of 0.8218, 0.8161 and 0.8199 on three random 70/30 splits.
- **Official per-category CS (ValenTo vs. best):**
  - sarcasm: 0.895 vs 0.904
  - irony: 0.901 vs 0.918
  - metaphor: 0.393 vs 0.655
  - other: 0.202 vs 0.612
- **Feature ranking:** RELIEF placed NRC emotion words and the AFINN and HL polarity lexicons at the top.

## Interesting ideas
- An explicit polarity-reversal feature fires when a tweet carries a self-labelling irony marker (#sarcasm, #not), so the author's own signal is treated as an operator on literal sentiment rather than as just another token.
- Several lexicons that measure different things (polarity, valence/arousal/dominance, pleasantness/imagery, discrete emotions) are kept as separate features instead of being merged, so each contributes its own dimension.
- Moving from coarse classes to fine-grained classes exposed a sharp accuracy drop, which motivated treating intensity as a continuous regression target.
- Breaking scores down by figurative type showed the method works for irony and sarcasm but fails on metaphor, which overall numbers would hide.
- A feature-relevance ranking (RELIEF) was used to see which knowledge sources actually drive predictions.

## For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

- **I1** (frames): The irony module could treat such explicit markers as a negation-like operator over the evaluated frame, recording why the reading was flipped.
- **I2** (world): Entity and state attitudes in the text world could store multi-dimensional affect values drawn from separate lexicons, keeping provenance for explanation.
- **I3** (answer): Graded judgements such as how strongly a character feels could be predicted as a scalar with a stated scale, not as many brittle classes.
- **I4** (learn): Evaluations should report per-phenomenon slices (irony, metaphor, literal) so weaknesses of a rule or model are visible rather than averaged away.
- **I5** (learn): A similar ranking could audit which compiled resources or repair rules matter, helping prune unused knowledge.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [University of Turin](https://md.research.elamur.ai/free/en/universities/university-of-turin/) — University, Turin, Italy · 27 works · 6,505 citations
- [Universitat Politècnica de València](https://md.research.elamur.ai/free/en/universities/universitat-politecnica-de-valencia/) — University, Valencia, Spain · 21 works · 3,275 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 9,086 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 8,889 works · [💭 Psychology](https://md.research.elamur.ai/free/en/shelves/psychology/) · 798 works · [💭 Experimental and Cognitive Psychology](https://md.research.elamur.ai/free/en/shelves/psychology/experimental-and-cognitive-psychology/) · 463 works

🧭 Topics (2): [😊 Sentiment & emotion](https://md.research.elamur.ai/free/en/topics/sentiment-emotion/) · 608 works · [🧷 Idioms & multiword expressions](https://md.research.elamur.ai/free/en/topics/multiword-expressions/) · 155 works

## Cite

Cite as: Delia Irazú Hernández Farías, Emilio Sulis, Viviana Patti, Giancarlo Ruffo and Cristina Bosco. 2015. ValenTo: Sentiment Analysis of Figurative Language Tweets with Irony and Sarcasm. Proceedings of the 9th International Workshop on Semantic Evaluation (SemEval 2015).

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{farias2015valento,
  title = {{ValenTo: Sentiment Analysis of Figurative Language Tweets with Irony and Sarcasm}},
  author = {Delia Irazú Hernández Farías, Emilio Sulis, Viviana Patti, Giancarlo Ruffo and Cristina Bosco},
  booktitle = {Proceedings of the 9th International Workshop on Semantic Evaluation (SemEval 2015)},
  year = {2015},
  doi = {10.18653/v1/s15-2117},
  url = {https://doi.org/10.18653/v1/s15-2117}
}
```

- [← previous World Englishes and English Language Teaching: A pragmatic and humanistic approach](https://md.research.elamur.ai/free/en/papers/bhowmik-2015-world-englishes-english-language-teaching/)
- [next → Statistical Parsing by Machine Learning from a Classical Arabic Treebank](https://md.research.elamur.ai/free/en/papers/dukes-2015-statistical-parsing-machine-learning-classical/)
- [↑ back to papers of 2015](https://md.research.elamur.ai/free/en/papers/2015/page/2/)

## References

24 references, 9 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Stefano Baccianella, Andrea Esuli, and Fabrizio Sebastiani. 2010. [Sentiwordnet 3.0: An enhanced lexical resource for sentiment analysis and opinion mining](https://md.research.elamur.ai/free/en/papers/baccianella-2010-sentiwordnet-3-0-enhanced-lexical/). In Proceedings of the Seventh International Conference on Language Resources and Evaluation (LREC’10).
2. Francesco Barbieri, Horacio Saggion, and Francesco Ronzano, 2014. Proceedings of the 5th Workshop on Computational Approaches to Subjectivity, Sentiment and Social Media Analysis, chapter [Modelling Sarcasm in Twitter, a Novel Approach](https://md.research.elamur.ai/free/en/papers/barbieri-2014-modelling-sarcasm-twitter-novel-approach/), pages 50–58.
3. Valerio Basile, Andrea Bolioli, Malvina Nissim, Viviana Patti, and Paolo Rosso. 2014. Overview of the Evalita 2014 SENTIment POLarity Classification Task. In Proceedings of the 4th evaluation campaign of Natural Language Processing and Speech tools for Italian (EVALITA’14), pages 50–57. [find](https://scholar.google.com/scholar?q=Valerio+Basile%2C+Andrea+Bolioli%2C+Malvina+Nissim%2C+Viviana+Patti%2C+and+Paolo+Rosso%2E+2014%2E+Overview+of+the+Evalita+2014+SENTIment+POLarity+Classification+Task%2E+In+Proceedings+of+the+4th+evaluation+campaign)
4. Cristina Bosco, Viviana Patti, and Andrea Bolioli. 2013. Developing corpora for sentiment analysis: The case of irony and senti-tut. IEEE Intelligent Systems, 28(2):55–63. [↗](https://doi.org/10.1109/mis.2013.28)
5. Margaret Bradley and Peter Lang. 1999. Affective norms for english words (ANEW): Instruction manual and affective ratings. Technical report, Citeseer. [find](https://scholar.google.com/scholar?q=Margaret+Bradley+and+Peter+Lang%2E+1999%2E+Affective+norms+for+english+words+%28ANEW%29%3A+Instruction+manual+and+affective+ratings%2E+Technical+report%2C+Citeseer%2E)
6. Konstantin Buschmeier, Philipp Cimiano, and Roman Klinger. 2014. [An impact analysis of features in a classification approach to irony detection in product reviews](https://md.research.elamur.ai/free/en/papers/buschmeier-2014-impact-analysis-features-classification-approach/). In Proceedings of the 5th Workshop on Computational Approaches to Subjectivity, Sentiment and Social Media Analysis, pages 42–49.
7. Erik Cambria, Catherine Havasi, and Amir Hussain. 2012. Senticnet 2: A semantic and affective resource for opinion mining and sentiment analysis. In AAAI FLAIRS Conference, pages 202–207. [find](https://scholar.google.com/scholar?q=Erik+Cambria%2C+Catherine+Havasi%2C+and+Amir+Hussain%2E+2012%2E+Senticnet+2%3A+A+semantic+and+affective+resource+for+opinion+mining+and+sentiment+analysis%2E+In+AAAI+FLAIRS+Conference%2C+pages+202%E2%80%93207%2E)
8. Erick Cambria, B. Schuller, Yunqing Xia, and C. Havasi. 2013. New avenues in opinion mining and sentiment analysis. Intelligent Systems, IEEE, 28(2):15– 21, March. [↗](https://doi.org/10.1109/mis.2013.30)
9. Dmitry Davidov, Oren Tsur, and Ari Rappoport. 2010. [Semi-supervised recognition of sarcastic sentences in twitter and amazon](https://md.research.elamur.ai/free/en/papers/davidov-2010-semi-supervised-recognition-sarcasm-twitter/). In Proceedings of the Fourteenth Conference on Computational Natural Language Learning, CoNLL ’10, pages 107–116.
10. A. Ghosh, G. Li, T. Veale, P. Rosso, E. Shutova, A. Reyes, and J. Barnden. 2015. [Semeval-2015 task 11: Sentiment analysis of figurative language in twitter](https://md.research.elamur.ai/free/en/papers/ghosh-2015-semeval-2015-task-11-sentiment/). In Proc. Int. Workshop on Semantic Evaluation (SemEval-2015), Co-located with NAACL and *SEM.
11. Roberto González-Ibáñez, Smaranda Muresan, and Nina Wacholder. 2011. [Identifying sarcasm in twitter: A closer look](https://md.research.elamur.ai/free/en/papers/ibanez-2011-identifying-sarcasm-twitter-closer-look/). In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies: Short Papers - Volume 2, HLT ’11, pages 581–586.
12. Minqing Hu and Bing Liu. 2004. Mining and summarizing customer reviews. In Proceedings of the Tenth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD ’04, pages 168–177. [find](https://scholar.google.com/scholar?q=Minqing+Hu+and+Bing+Liu%2E+2004%2E+Mining+and+summarizing+customer+reviews%2E+In+Proceedings+of+the+Tenth+ACM+SIGKDD+International+Conference+on+Knowledge+Discovery+and+Data+Mining%2C+KDD+%E2%80%9904%2C+pages+168%E2%80%93177%2E)
13. Bing Liu. 2010. Sentiment analysis and subjectivity. In Handbook of Natural Language Processing. [find](https://scholar.google.com/scholar?q=Bing+Liu%2E+2010%2E+Sentiment+analysis+and+subjectivity%2E+In+Handbook+of+Natural+Language+Processing%2E)
14. Diana Maynard and Mark Greenwood. 2014. [Who cares about sarcastic tweets? investigating the impact of sarcasm on sentiment analysis](https://md.research.elamur.ai/free/en/papers/maynard-2014-who-cares-about-sarcastic-tweets/). In Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC-2014).
15. Saif M. Mohammad and Peter D. Turney. 2013. Crowdsourcing a word–emotion association lexicon. Computational Intelligence, 29(3):436–465. [↗](https://doi.org/10.1111/j.1467-8640.2012.00460.x)
16. Finn Årup Nielsen. 2011. A new ANEW: evaluation of a word list for sentiment analysis in microblogs. [↗](https://doi.org/10.48550/arxiv.1103.2903)
17. Bo Pang and Lillian Lee. 2008. Opinion mining and sentiment analysis. Foundations and Trends in Information Retrieval, 2(12):1–135. [↗](https://doi.org/10.1561/9781601981516)
18. Antonio Reyes, Paolo Rosso, and Tony Veale. 2013. A multidimensional approach for detecting irony in twitter. Language Resources and Evaluation, 47(1):239– 268. [↗](https://doi.org/10.1007/s10579-012-9196-x)
19. Ellen Riloff, Ashequl Qadir, Prafulla Surve, Lalindra De Silva, Nathan Gilbert, and Ruihong Huang. 2013. [Sarcasm as contrast between a positive sentiment and negative situation](https://md.research.elamur.ai/free/en/papers/riloff-2013-sarcasm-as-contrast-between-positive/). In Proceedings of the EMNLP: Conference on Empirical Methods in Natural Language Processing, pages 704–714.
20. Marko Robnik-Sikonja and Igor Kononenko. 1997. An adaptation of relief for attribute estimation in regression. In Fourteenth International Conference on Machine Learning, pages 296–304. [find](https://scholar.google.com/scholar?q=Marko+Robnik%2DSikonja+and+Igor+Kononenko%2E+1997%2E+An+adaptation+of+relief+for+attribute+estimation+in+regression%2E+In+Fourteenth+International+Conference+on+Machine+Learning%2C+pages+296%E2%80%93304%2E)
21. Sara Rosenthal, Alan Ritter, Preslav Nakov, and Veselin Stoyanov. 2014. [Semeval-2014 task 9: Sentiment analysis in twitter](https://md.research.elamur.ai/free/en/papers/rosenthal-2014-semeval-2014-task-9-sentiment/). In Proceedings of the 8th International Workshop on Semantic Evaluation (SemEval 2014), pages 73–80, August.
22. Philip J. Stone and Earl B. Hunt. 1963. A computer approach to content analysis: Studies using the general inquirer system. In Proceedings of the May 21-23, 1963, AFIPS ’63 (Spring), pages 241–256. [find](https://scholar.google.com/scholar?q=Philip+J%2E+Stone+and+Earl+B%2E+Hunt%2E+1963%2E+A+computer+approach+to+content+analysis%3A+Studies+using+the+general+inquirer+system%2E+In+Proceedings+of+the+May+21%2D23%2C+1963%2C+AFIPS+%E2%80%9963+%28Spring%29%2C+pages+241%E2%80%93256%2E)
23. Tony Veale and Yanfen Hao. 2010. Detecting ironic intent in creative comparisons. In ECAI, volume 215, pages 765–770. [↗](https://doi.org/10.3233/978-1-60750-606-5-765)
24. Cynthia Whissell. 2009. Using the revised dictionary of affect in language to quantify the emotional undertones of samples of natural languages. In Psychological Reports. [find](https://scholar.google.com/scholar?q=Cynthia+Whissell%2E+2009%2E+Using+the+revised+dictionary+of+affect+in+language+to+quantify+the+emotional+undertones+of+samples+of+natural+languages%2E+In+Psychological+Reports%2E)

*Updated 24 October 2026*