---
title: "Multilingual Subjectivity Analysis Using Machine Translation"
description: "MT therefore offers a cheap and workable route to subjectivity tools for… 4% F-measure. 21 references, cited by 62 papers here."
authors: ["Carmen Banea", "Rada Mihalcea", "Janyce Wiebe", "Samer Hassan"]
year: 2008
venue: "Proceedings of the 2008 Conference on Empirical Methods in Natural Language Processing"
source: "https://aclanthology.org/D08-1014/"
licence: "CC BY-NC-SA 3.0 (ACL Anthology, before 2016)"
licence_url: "https://creativecommons.org/licenses/by-nc-sa/3.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/banea-2008-multilingual-subjectivity-analysis-machine-translation/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/banea-2008-multilingual-subjectivity-analysis-machine-translation/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2008](https://md.research.elamur.ai/free/en/papers/2008/) / Multilingual Subjectivity Analysis Using Machine Translation

# Multilingual Subjectivity Analysis Using Machine Translation

**Authors:** [Carmen Banea](https://md.research.elamur.ai/free/en/authors/carmen-banea/), [Rada Mihalcea](https://md.research.elamur.ai/free/en/authors/rada-mihalcea/), [Janyce Wiebe](https://md.research.elamur.ai/free/en/authors/janyce-wiebe/) and Samer Hassan · **Year:** 2008 · **Venue:** [Proceedings of the 2008 Conference on Empirical Methods in Natural Language Processing](https://md.research.elamur.ai/free/en/venues/emnlp/) ([EMNLP 2008](https://md.research.elamur.ai/free/en/venues/emnlp/2008/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY-NC-SA 3.0 (ACL Anthology, before 2016)](https://creativecommons.org/licenses/by-nc-sa/3.0/)

**Links:** [Publisher's page](https://aclanthology.org/D08-1014/) · [PDF](https://aclanthology.org/D08-1014.pdf) · [OpenAlex](https://openalex.org/W2081795963)

**Citations:** 288 citations (OpenAlex, 2026-10-14) · [62](https://md.research.elamur.ai/free/en/papers/banea-2008-multilingual-subjectivity-analysis-machine-translation/cited-by/) in our library, of the 141,840 papers whose bibliographies we read

## Summary
The paper tackles the shortage of subjectivity analysis resources outside English. The authors test whether machine translation can carry English annotated data or tools into a new language and so produce training material for sentence-level subjective/objective classifiers. Three pipelines are compared. The first machine-translates the manually labelled MPQA corpus and keeps its labels. The second tags raw English SemCor text with OpinionFinder and then translates it into the target language. The third goes the other way: target-language text is translated into English, tagged by OpinionFinder, and the labels are mapped back. A fourth setup, which translates the test data itself, acts as an upper bound. Romanian is the main test case and Spanish serves as a check. No target-language knowledge is used, so any language with an MT system to and from English qualifies. Classifiers trained on these automatically built corpora come within about 4% F-measure of those trained on human-translated parallel data. MT therefore offers a cheap and workable route to subjectivity tools for under-resourced languages.

## Key points
- Source resources: MPQA, with 535 news articles and 9,700 sentences at sentence level (55% subjective), and OpinionFinder. On MPQA, OpinionFinder's high-precision classifier scores P 86.7 / R 32.6 / F 47.4, and its high-coverage classifier scores P 79.4 / R 70.6 / F 74.7.
- Raw-text experiments use a SemCor subset of 107 documents (about 11,000 sentences). SemCor has a manual Romanian translation, which allows comparison with human translation.
- Evaluation uses a gold standard of 504 Romanian sentences annotated by two native speakers, the same set used by Mihalcea et al. (2007). The Romanian-English MT engines were supplied by LanguageWeaver.
- Relabelling the training data with only the high-precision OpinionFinder output (3,700 sentences) gives F = 69.05% for Romanian and 66.47% for Spanish.
- About 20,000 automatically labelled sentences in Romanian and Spanish were released, together with the manually annotated test sets.

## Interesting ideas
- **Choosing the translation direction as a trade-off.** Labelling the original English and then translating gives cleaner labels but noisy target text. Translating native target text into English gives clean training text but noisier labels. Testing both directions is a useful template for any cross-lingual label-transfer task, such as NER, sentiment, or topic classification for Ukrainian.
- **Translating the test set as an upper bound.** Translating the test data into English and running the English tool directly gives a ceiling for what a classifier trained on projected labels could achieve. This makes it easy to see how much is lost at the learning step.
- **Controlled comparison of languages.** The same training sentences, test sentences and projected labels were used in English and in Romanian. Only the language differed, and Romanian scored higher (about 68–70 F vs. 60 for English). The authors link this to Romanian's rich inflection and verb moods (conditional, presumptive) acting as subjectivity cues. A morphologically rich language like Ukrainian could gain in a similar way.
- **Fewer but more precise labels.** Training on the smaller, precision-filtered automatic labels is a cheap way to check whether label noise or label quantity matters more when building silver-standard corpora.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [University of Pittsburgh](https://md.research.elamur.ai/free/en/universities/university-of-pittsburgh/) — University, Pittsburgh, United States · 38 works · 21,591 citations
- [University of North Texas](https://md.research.elamur.ai/free/en/universities/university-of-north-texas/) — University, Denton, United States · 18 works · 4,513 citations

📚 Shelves (6): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 9,086 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 8,889 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 9,296 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 9,206 works · [💭 Psychology](https://md.research.elamur.ai/free/en/shelves/psychology/) · 798 works · [💭 Experimental and Cognitive Psychology](https://md.research.elamur.ai/free/en/shelves/psychology/experimental-and-cognitive-psychology/) · 463 works

🧭 Topics (2): [🔁 Machine translation](https://md.research.elamur.ai/free/en/topics/machine-translation/) · 958 works · [🌍 Multilingual & cross-lingual NLP](https://md.research.elamur.ai/free/en/topics/multilingual-nlp/) · 467 works

## Cite

Cite as: Carmen Banea, Rada Mihalcea, Janyce Wiebe and Samer Hassan. 2008. Multilingual Subjectivity Analysis Using Machine Translation. Proceedings of the 2008 Conference on Empirical Methods in Natural Language Processing.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{banea2008multilingual,
  title = {{Multilingual Subjectivity Analysis Using Machine Translation}},
  author = {Carmen Banea, Rada Mihalcea, Janyce Wiebe and Samer Hassan},
  booktitle = {Proceedings of the 2008 Conference on Empirical Methods in Natural Language Processing},
  year = {2008},
  doi = {10.3115/1613715.1613734},
  url = {https://doi.org/10.3115/1613715.1613734}
}
```

- [← previous Understanding the Value of Features for Coreference Resolution](https://md.research.elamur.ai/free/en/papers/bengtson-2008-understanding-value-features-coreference-resolution/)
- [next → A Tale of Two Parsers: Investigating and Combining Graph-based and Transition-based Dependency Parsing](https://md.research.elamur.ai/free/en/papers/zhang-2008-tale-two-parsers-investigating-combining/)
- [↑ back to papers of 2008](https://md.research.elamur.ai/free/en/papers/2008/)

## References

21 references, 4 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. C. Ovesdotter Alm, D. Roth, and R. Sproat. 2005. Emotions from text: Machine learning for text-based emotion prediction. In Proceedings of the Human Language Technologies Conference/Conference on Empirical Methods in Natural Language Processing (HLT/EMNLP-2005), pages 347–354, Vancouver, Canada. [our card](https://md.research.elamur.ai/free/en/papers/alm-2005-emotions-text-machine-learning-text/)
2. K. Balog, G. Mishne, and M. de Rijke. 2006. Why are they excited? identifying and explaining spikes in blog mood levels. In Proceedings of the 11th Meeting of the European Chapter of the Association for Computational Linguistics (EACL-2006). [find](https://scholar.google.com/scholar?q=K%2E+Balog%2C+G%2E+Mishne%2C+and+M%2E+de+Rijke%2E+2006%2E+Why+are+they+excited%3F+identifying+and+explaining+spikes+in+blog+mood+levels%2E+In+Proceedings+of+the+11th+Meeting+of+the+European+Chapter+of+the+Association+f)
3. A. Esuli and F. Sebastiani. 2006. Determining term subjectivity and term orientation for opinion mining. In Proceedings the 11th Meeting of the European Chapter of the Association for Computational Linguistics (EACL-2006), pages 193–200, Trento, IT. [find](https://scholar.google.com/scholar?q=A%2E+Esuli+and+F%2E+Sebastiani%2E+2006%2E+Determining+term+subjectivity+and+term+orientation+for+opinion+mining%2E+In+Proceedings+the+11th+Meeting+of+the+European+Chapter+of+the+Association+for+Computational+Li)
4. R. Fan, P. Chen, and C. Lin. 2005. Working set selection using the second order information for training svm. Journal of Machine Learning Research, 6:1889–1918. [find](https://scholar.google.com/scholar?q=R%2E+Fan%2C+P%2E+Chen%2C+and+C%2E+Lin%2E+2005%2E+Working+set+selection+using+the+second+order+information+for+training+svm%2E+Journal+of+Machine+Learning+Research%2C+6%3A1889%E2%80%931918%2E)
5. M. Hu and B. Liu. 2004. Mining and summarizing customer reviews. In Proceedings of ACM SIGKDD Conference on Knowledge Discovery and Data Mining 2004 (KDD 2004), pages 168–177, Seattle, Washington. [find](https://scholar.google.com/scholar?q=M%2E+Hu+and+B%2E+Liu%2E+2004%2E+Mining+and+summarizing+customer+reviews%2E+In+Proceedings+of+ACM+SIGKDD+Conference+on+Knowledge+Discovery+and+Data+Mining+2004+%28KDD+2004%29%2C+pages+168%E2%80%93177%2C+Seattle%2C+Washington%2E)
6. Y. Hu, J. Duan, X. Chen, B. Pei, and R. Lu. 2005. A new method for sentiment classification in text retrieval. In IJCNLP, pages 1–9. [find](https://scholar.google.com/scholar?q=Y%2E+Hu%2C+J%2E+Duan%2C+X%2E+Chen%2C+B%2E+Pei%2C+and+R%2E+Lu%2E+2005%2E+A+new+method+for+sentiment+classification+in+text+retrieval%2E+In+IJCNLP%2C+pages+1%E2%80%939%2E)
7. T. Joachims. 1998. Text categorization with Support Vector Machines: learning with mny relevant features. In Proceedings of the European Conference on Machine Learning, pages 137–142. [find](https://scholar.google.com/scholar?q=T%2E+Joachims%2E+1998%2E+Text+categorization+with+Support+Vector+Machines%3A+learning+with+mny+relevant+features%2E+In+Proceedings+of+the+European+Conference+on+Machine+Learning%2C+pages+137%E2%80%93142%2E)
8. H. Kanayama and T. Nasukawa. 2006. Fully automatic lexicon expansion for domain-oriented sentiment analysis. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP-2006), pages 355–363, Sydney, Australia. [find](https://scholar.google.com/scholar?q=H%2E+Kanayama+and+T%2E+Nasukawa%2E+2006%2E+Fully+automatic+lexicon+expansion+for+domain%2Doriented+sentiment+analysis%2E+In+Proceedings+of+the+Conference+on+Empirical+Methods+in+Natural+Language+Processing+%28EMNLP)
9. N. Kando and D. Kirk Evans, editors. 2007. Proceedings of the Sixth NTCIR Workshop Meeting on Evaluation of Information Access Technologies: Information Retrieval, Question Answering, and Cross-Lingual Information Access, 2-1-2 Hitotsubashi, Chiyoda-ku, Tokyo 101-8430, Japan, May. National Institute… [find](https://scholar.google.com/scholar?q=N%2E+Kando+and+D%2E+Kirk+Evans%2C+editors%2E+2007%2E+Proceedings+of+the+Sixth+NTCIR+Workshop+Meeting+on+Evaluation+of+Information+Access+Technologies%3A+Information+Retrieval%2C+Question+Answering%2C+and+Cross%2DLingua)
10. S.-M. Kim and E. Hovy. 2006. Identifying and analyzing judgment opinions. In Proceedings of the Human Language Technology Conference of the NAACL, pages 200–207, New York, New York. [find](https://scholar.google.com/scholar?q=S%2E%2DM%2E+Kim+and+E%2E+Hovy%2E+2006%2E+Identifying+and+analyzing+judgment+opinions%2E+In+Proceedings+of+the+Human+Language+Technology+Conference+of+the+NAACL%2C+pages+200%E2%80%93207%2C+New+York%2C+New+York%2E)
11. N. Kobayashi, K. Inui, Y. Matsumoto, K. Tateishi, and T. Fukushima. 2004. Collecting evaluative expressions for opinion extraction. In Proceedings of the 1st International Joint Conference on Natural Language Processing (IJCNLP-04). [find](https://scholar.google.com/scholar?q=N%2E+Kobayashi%2C+K%2E+Inui%2C+Y%2E+Matsumoto%2C+K%2E+Tateishi%2C+and+T%2E+Fukushima%2E+2004%2E+Collecting+evaluative+expressions+for+opinion+extraction%2E+In+Proceedings+of+the+1st+International+Joint+Conference+on+Natural+)
12. L. Lloyd, D. Kechagias, and S. Skiena. 2005. Lydia: A system for large-scale news analysis. In String Processing and Information Retrieval (SPIRE 2005). [find](https://scholar.google.com/scholar?q=L%2E+Lloyd%2C+D%2E+Kechagias%2C+and+S%2E+Skiena%2E+2005%2E+Lydia%3A+A+system+for+large%2Dscale+news+analysis%2E+In+String+Processing+and+Information+Retrieval+%28SPIRE+2005%29%2E)
13. A. McCallum and K. Nigam. 1998. A comparison of event models for Naive Bayes text classification. In Proceedings of AAAI-98 Workshop on Learning for Text Categorization. [find](https://scholar.google.com/scholar?q=A%2E+McCallum+and+K%2E+Nigam%2E+1998%2E+A+comparison+of+event+models+for+Naive+Bayes+text+classification%2E+In+Proceedings+of+AAAI%2D98+Workshop+on+Learning+for+Text+Categorization%2E)
14. R. Mihalcea, C. Banea, and J. Wiebe. 2007. Learning multilingual subjective language via cross-lingual projections. In Proceedings of the Association for Computational Linguistics, Prague, Czech Republic. [our card](https://md.research.elamur.ai/free/en/papers/mihalcea-2007-learning-multilingual-subjective-language-cross/)
15. G. Miller, C. Leacock, T. Randee, and R. Bunker. 1993. A semantic concordance. In Proceedings of the 3rd DARPA Workshop on Human Language Technology, Plainsboro, New Jersey. [our card](https://md.research.elamur.ai/free/en/papers/miller-1993-semantic-concordance/)
16. Y. Suzuki, H. Takamura, and M. Okumura. 2006. Application of semi-supervised learning to evaluative expression classification. In Proceedings of the 7th International Conference on Intelligent Text Processing and Computational Linguistics (CICLing-2006), pages 502–513, Mexico City, Mexico. [find](https://scholar.google.com/scholar?q=Y%2E+Suzuki%2C+H%2E+Takamura%2C+and+M%2E+Okumura%2E+2006%2E+Application+of+semi%2Dsupervised+learning+to+evaluative+expression+classification%2E+In+Proceedings+of+the+7th+International+Conference+on+Intelligent+Text+Pr)
17. H. Takamura, T. Inui, and M. Okumura. 2006. Latent variable models for semantic orientations of phrases. In Proceedings of the 11th Meeting of the European Chapter of the Association for Computational Linguistics (EACL 2006), Trento, Italy. [find](https://scholar.google.com/scholar?q=H%2E+Takamura%2C+T%2E+Inui%2C+and+M%2E+Okumura%2E+2006%2E+Latent+variable+models+for+semantic+orientations+of+phrases%2E+In+Proceedings+of+the+11th+Meeting+of+the+European+Chapter+of+the+Association+for+Computational)
18. J. Wiebe and R. Mihalcea. 2006. Word sense and subjectivity. In Proceedings of COLING-ACL 2006. [find](https://scholar.google.com/scholar?q=J%2E+Wiebe+and+R%2E+Mihalcea%2E+2006%2E+Word+sense+and+subjectivity%2E+In+Proceedings+of+COLING%2DACL+2006%2E)
19. J. Wiebe and E. Riloff. 2005. Creating subjective and objective sentence classifiers from unannotated texts. In Proceedings of the 6th International Conference on Intelligent Text Processing and Computational Linguistics (CICLing-2005) ( invited paper), Mexico City, Mexico. [find](https://scholar.google.com/scholar?q=J%2E+Wiebe+and+E%2E+Riloff%2E+2005%2E+Creating+subjective+and+objective+sentence+classifiers+from+unannotated+texts%2E+In+Proceedings+of+the+6th+International+Conference+on+Intelligent+Text+Processing+and+Compu)
20. J. Wiebe, T. Wilson, and C. Cardie. 2005. Annotating expressions of opinions and emotions in language. Language Resources and Evaluation, 39(2-3):165–210. [find](https://scholar.google.com/scholar?q=J%2E+Wiebe%2C+T%2E+Wilson%2C+and+C%2E+Cardie%2E+2005%2E+Annotating+expressions+of+opinions+and+emotions+in+language%2E+Language+Resources+and+Evaluation%2C+39%282%2D3%29%3A165%E2%80%93210%2E)
21. H. Yu and V. Hatzivassiloglou. 2003. Towards answering opinion questions: Separating facts from opinions and identifying the polarity of opinion sentences. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP-2003), pages 129–136, Sapporo, Japan. [our card](https://md.research.elamur.ai/free/en/papers/yu-2003-answering-opinion-questions-separating-facts/)

*Updated 24 October 2026*