--- title: "Determining the Sentiment of Opinions — Kim 2004" description: "Kim and Hovy address opinion analysis. Starting from a collection of documents and a subject of interest, the goal is to find who holds an opinion about…" authors: ["Soo-Min Kim", "Eduard Hovy"] year: 2004 venue: "COLING 2004: Proceedings of the 20th International Conference on Computational Linguistics" source: "https://aclanthology.org/C04-1200/" licence: "CC BY-NC-SA 3.0 (ACL Anthology, before 2016)" licence_url: "https://creativecommons.org/licenses/by-nc-sa/3.0/" summary_licence: "Elamur Cosmic License" canonical: "https://research.elamur.ai/free/en/papers/kim-2004-determining-sentiment-opinions/" --- [Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/kim-2004-determining-sentiment-opinions/) [home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2004](https://md.research.elamur.ai/free/en/papers/2004/) / Determining the Sentiment of Opinions # Determining the Sentiment of Opinions **Authors:** [Soo-Min Kim](https://md.research.elamur.ai/free/en/authors/soo-min-kim/), [Eduard Hovy](https://md.research.elamur.ai/free/en/authors/eduard-hovy/) · **Year:** 2004 · **Venue:** COLING 2004: Proceedings of the 20th International Conference on Computational Linguistics · **Licence:** [CC BY-NC-SA 3.0 (ACL Anthology, before 2016)](https://creativecommons.org/licenses/by-nc-sa/3.0/) **Links:** [Publisher's page](https://aclanthology.org/C04-1200/) · [PDF](https://aclanthology.org/C04-1200.pdf) · [OpenAlex](https://openalex.org/W2112422413) **Citations:** 1482 citations (openalex-snapshot · 2026-10-14) · [195](#cited-by) in our library, of the 140884 papers whose bibliographies we read ## Summary Kim and Hovy address opinion analysis. Starting from a collection of documents and a subject of interest, the goal is to find who holds an opinion about the topic and whether that opinion is positive, negative or neutral. They model an opinion as a tuple of Topic, Holder, Claim and Sentiment. The work proceeds in two stages. 1. **Word level.** Small hand-picked seed lists of positive and negative adjectives and verbs are expanded through WordNet synonyms and antonyms. Two Naive-Bayes-style models then score how strongly an unseen word leans each way, based on how many of its synonyms fall into each list. 2. **Sentence level.** IdentiFinder proposes candidate holders. A region of text around the holder is delimited, and the word sentiments inside it are combined with one of three models: sign product, harmonic mean or geometric mean. Sentence accuracy reached 81% when holders were given by annotators and 67% when holders were detected automatically. The study shows that a usable sentiment and holder system can be built from minimal manual seeding without annotated training corpora. ## Key points - **Seed lists:** 23 positive and 21 negative verbs, and 15 positive and 19 negative adjectives; nouns were added later. - **Expanded lexicon:** two rounds of WordNet expansion produced 5880 positive and 6233 negative adjectives, plus 2840 positive and 3239 negative verbs. Adjectives used synonyms and antonyms; verbs used only synonyms. - **Word-level agreement:** on 231 test adjectives, agreement with humans was about 68%, against about 58–59% for random selection. - **Holder detection:** BBN's IdentiFinder supplied PERSON and ORGANIZATION entities as holder candidates. When several appeared, the one closest to the topic phrase was chosen. - **Model comparison:** - The sign-only Model 0 gave the best single result. - With manual holders, the region from the holder to the end of the sentence (Window4) worked best. - Automatic holder detection caused roughly 7 more misclassified sentences, about 11%. ## Interesting ideas - **Lexicon from a thesaurus instead of a corpus.** Growing a polarity lexicon from a few seeds via synonym and antonym links avoids annotated data entirely. This could bootstrap sentiment resources for lower-resourced languages that have a WordNet-like resource. - **Scoring both polarities for each word.** Giving every word a positive and a negative strength exposes ambiguous items such as "great" or "get", which appear in both lists. A threshold can then filter them out instead of forcing a single label. - **Sweeping the context window.** Comparing several region definitions (full sentence, holder-to-topic span, padded span, holder to sentence end) is a cheap stand-in for parsing. It shows how much context the combination step actually needs. - **Testing with gold and with automatic upstream input.** Evaluating once with human-annotated holders and once with automatic ones isolates the error added by holder detection from the error of sentiment classification. This is a good practice for any pipeline evaluation. - **Trying the simple sign baseline first.** Multiplying polarity signs, with explicit negation flipping, beat strength-weighted averaging. Simple baselines deserve a test before more elaborate scoring. *Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.* Fields: [Natural language processing](https://md.research.elamur.ai/free/en/shelves/nlp/) ## Cited by Cited by 195 papers of our library cite it (of the 140884 whose bibliographies we read); 12 of them have a page here, 183 not yet. - [Sentiment Analysis of Twitter Data](https://md.research.elamur.ai/free/en/papers/agarwal-2011-sentiment-analysis-twitter-data/) - [Enhanced Sentiment Learning Using Twitter Hashtags and Smileys](https://md.research.elamur.ai/free/en/papers/davidov-2010-enhanced-sentiment-learning-twitter-hashtags/) - [Resolving Object and Attribute Coreference in Opinion Mining](https://md.research.elamur.ai/free/en/papers/ding-2010-resolving-object-attribute-coreference-opinion/) - [SENTIWORDNET: A Publicly Available Lexical Resource for Opinion Mining](https://md.research.elamur.ai/free/en/papers/esuli-2006-sentiwordnet-publicly-available-lexical-resource/) - [Analyses of the Association between Discourse Relation and Sentiment Polarity with a Chinese Human-Annotated Corpus](https://md.research.elamur.ai/free/en/papers/huang-2013-analyses-association-between-discourse-relation/) - [Extracting Opinions, Opinion Holders, and Topics Expressed in Online News Media Text](https://md.research.elamur.ai/free/en/papers/kim-2006-extracting-opinions-opinion-holders-topics/) - [Extracting Product Features and Opinions from Reviews](https://md.research.elamur.ai/free/en/papers/popescu-2005-extracting-product-features-opinions-reviews/) - [Opinion Word Expansion and Target Extraction through Double Propagation](https://md.research.elamur.ai/free/en/papers/qiu-2011-opinion-word-expansion-target-extraction/) - [Lexicon-Based Methods for Sentiment Analysis](https://md.research.elamur.ai/free/en/papers/taboada-2011-lexicon-based-methods-sentiment-analysis/) - [Co-Training for Cross-Lingual Sentiment Classification](https://md.research.elamur.ai/free/en/papers/wan-2009-co-training-cross-lingual-sentiment/) - [Recognizing Contextual Polarity in Phrase-Level Sentiment Analysis](https://md.research.elamur.ai/free/en/papers/wilson-2005-recognizing-contextual-polarity-phrase-level/) - [Recognizing Contextual Polarity: An Exploration of Features for Phrase-Level Sentiment Analysis](https://md.research.elamur.ai/free/en/papers/wilson-2009-recognizing-contextual-polarity-exploration-features/) ## Cite The paper itself, not this page. BibTeX from the library's record of the work. ```bibtex @inproceedings{kim2004determining, title = {{Determining the Sentiment of Opinions}}, author = {Kim, Soo-Min and Hovy, Eduard}, booktitle = {COLING 2004: Proceedings of the 20th International Conference on Computational Linguistics}, year = {2004}, doi = {10.3115/1220355.1220555}, url = {https://doi.org/10.3115/1220355.1220555} } ``` - [← previous Dependency Tree Kernels for Relation Extraction](https://md.research.elamur.ai/free/en/papers/culotta-2004-dependency-tree-kernels-relation-extraction/) - [next → Deterministic Dependency Parsing of English Text](https://md.research.elamur.ai/free/en/papers/aclbulk-2004-nivre-deterministic-dependency-parsing-english-text/) - [↑ back to papers of 2004](https://md.research.elamur.ai/free/en/papers/2004/) ## References 14 references, 4 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet. 1. Aristotle. The Rhetorics and Poetics (trans. W. Rhys Roberts), Modern Library, 1954. [find](https://scholar.google.com/scholar?q=Aristotle%2E+The+Rhetorics+and+Poetics+%28trans%2E+W%2E+Rhys+Roberts%29%2C+Modern+Library%2C+1954%2E) 2. Fellbaum, C., D. Gross, and K. Miller. 1993. Adjectives in WordNet. [http://www.cosgi.princeton.edu/~wn](http://www.cosgi.princeton.edu/~wn). [↗](http://www.cosgi.princeton.edu/~wn) 3. Hatzivassiloglou, V. and K. McKeown 1997. Predicting the Semantic Orientation of Adjectives. Proceedings of the 35 th ACL conference, 174–181. [find](https://scholar.google.com/scholar?q=Hatzivassiloglou%2C+V%2E+and+K%2E+McKeown+1997%2E+Predicting+the+Semantic+Orientation+of+Adjectives%2E+Proceedings+of+the+35+th+ACL+conference%2C+174%E2%80%93181%2E) 4. Miller, G.A., R. Beckwith, C. Fellbaum, D. Gross, and K. Miller. 1993. Introduction to WordNet: An On-Line Lexical Database. [http://www.cosgi.princeton.edu/~wn](http://www.cosgi.princeton.edu/~wn). [↗](http://www.cosgi.princeton.edu/~wn) 5. Pang, B. L. Lee, and S. Vaithyanathan, 2002. Thumbs up? Sentiment classification using Machine Learning Techniques. Proceedings of the EMNLP conference. [find](https://scholar.google.com/scholar?q=Pang%2C+B%2E+L%2E+Lee%2C+and+S%2E+Vaithyanathan%2C+2002%2E+Thumbs+up%3F+Sentiment+classification+using+Machine+Learning+Techniques%2E+Proceedings+of+the+EMNLP+conference%2E) [our card](https://md.research.elamur.ai/free/en/papers/pang-2002-thumbs-up-sentiment-classification-machine/) 6. Perelman, C. 1970. The New Rhetoric: A Theory of Practical Reasoning. In The Great Ideas Today. Chicago: Encyclopedia Britannica. [find](https://scholar.google.com/scholar?q=Perelman%2C+C%2E+1970%2E+The+New+Rhetoric%3A+A+Theory+of+Practical+Reasoning%2E+In+The+Great+Ideas+Today%2E+Chicago%3A+Encyclopedia+Britannica%2E) 7. Riloff, E., J. Wiebe, and T. Wilson 2003. Learning Subjective Nouns Using Extraction Pattern Bootstrapping. Proceedings of the CoNLL-03 conference. [find](https://scholar.google.com/scholar?q=Riloff%2C+E%2E%2C+J%2E+Wiebe%2C+and+T%2E+Wilson+2003%2E+Learning+Subjective+Nouns+Using+Extraction+Pattern+Bootstrapping%2E+Proceedings+of+the+CoNLL%2D03+conference%2E) [our card](https://md.research.elamur.ai/free/en/papers/riloff-2003-learning-subjective-nouns-extraction-pattern/) 8. Siegel, S. and N.J. Castellan Jr. 1988. Nonparametric Statistics for the Behavioral Sciences. McGraw-Hill. [find](https://scholar.google.com/scholar?q=Siegel%2C+S%2E+and+N%2EJ%2E+Castellan+Jr%2E+1988%2E+Nonparametric+Statistics+for+the+Behavioral+Sciences%2E+McGraw%2DHill%2E) 9. Toulmin, S.E., R. Rieke, and A. Janik. 1979. An Introduction to Reasoning. Macmillan, New York. [find](https://scholar.google.com/scholar?q=Toulmin%2C+S%2EE%2E%2C+R%2E+Rieke%2C+and+A%2E+Janik%2E+1979%2E+An+Introduction+to+Reasoning%2E+Macmillan%2C+New+York%2E) 10. Toulmin, S.E. 2003. The Uses of Argument. Cambridge University Press. [find](https://scholar.google.com/scholar?q=Toulmin%2C+S%2EE%2E+2003%2E+The+Uses+of+Argument%2E+Cambridge+University+Press%2E) 11. Turney, P. 2002. Thumbs Up or Thumbs Down? Semantic Orientation Applied to Unsupervised Classification of Reviews. Proceedings of the 40 th Annual Meeting of the ACL, Philadelphia, 417–424. [find](https://scholar.google.com/scholar?q=Turney%2C+P%2E+2002%2E+Thumbs+Up+or+Thumbs+Down%3F+Semantic+Orientation+Applied+to+Unsupervised+Classification+of+Reviews%2E+Proceedings+of+the+40+th+Annual+Meeting+of+the+ACL%2C+Philadelphia%2C+417%E2%80%93424%2E) [our card](https://md.research.elamur.ai/free/en/papers/turney-2002-thumbs-up-or-thumbs-down/) 12. Wallace, K. 1975. Topoi and the Problem of Invention. In W. Ross Winterowd (ed), Contemporary Rhetoric. Harcourt Brace Jovanovich. [find](https://scholar.google.com/scholar?q=Wallace%2C+K%2E+1975%2E+Topoi+and+the+Problem+of+Invention%2E+In+W%2E+Ross+Winterowd+%28ed%29%2C+Contemporary+Rhetoric%2E+Harcourt+Brace+Jovanovich%2E) 13. Wiebe, J. et al. 2002. NRRC summer study Jan Wiebe and group (University of Pittsburgh) on ‘subjective’ statements. [find](https://scholar.google.com/scholar?q=Wiebe%2C+J%2E+et+al%2E+2002%2E+NRRC+summer+study+Jan+Wiebe+and+group+%28University+of+Pittsburgh%29+on+%E2%80%98subjective%E2%80%99+statements%2E) 14. Yu, H. and V. Hatzivassiloglou. 2003. Towards Answering Opinion Questions: Separating Facts from Opinions and Identifying the Polarity of Opinion Sentences. Proceedings of the EMNLP conference. [find](https://scholar.google.com/scholar?q=Yu%2C+H%2E+and+V%2E+Hatzivassiloglou%2E+2003%2E+Towards+Answering+Opinion+Questions%3A+Separating+Facts+from+Opinions+and+Identifying+the+Polarity+of+Opinion+Sentences%2E+Proceedings+of+the+EMNLP+conference%2E) [our card](https://md.research.elamur.ai/free/en/papers/yu-2003-answering-opinion-questions-separating-facts/)