---
title: "Comparative Approaches to Sentiment Analysis — Sidorov; 2025"
description: "The paper asks how well three widely used transformer encoders handle sentiment classification across nine typologically different languages: Arabic and…"
authors: ["Mikhail Krasitskii; Olga Kolesnikova; Liliana Chanona Hernandez; Grigori Sidorov;", "Alexander Gelbukh"]
year: 2025
venue: "arXiv:2501.12540"
source: "https://arxiv.org/abs/2501.12540"
doi: "10.5121/csit.2024.150112"
arxiv: "2501.12540"
licence: "CC BY-NC-ND 4.0 (arXiv)"
licence_url: "http://creativecommons.org/licenses/by-nc-nd/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/krasitskii-2025-comparative-approaches-sentiment-analysis-using/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/krasitskii-2025-comparative-approaches-sentiment-analysis-using/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2025](https://md.research.elamur.ai/free/en/papers/2025/) / Comparative Approaches to Sentiment Analysis Using Datasets in Major European and Arabic Languages

# Comparative Approaches to Sentiment Analysis Using Datasets in Major European and Arabic Languages

**Authors:** Mikhail Krasitskii; Olga Kolesnikova; Liliana Chanona Hernandez; Grigori Sidorov;, [Alexander Gelbukh](https://md.research.elamur.ai/free/en/authors/alexander-gelbukh/) · **Year:** 2025 · **Venue:** arXiv:2501.12540 · **Licence:** [CC BY-NC-ND 4.0 (arXiv)](http://creativecommons.org/licenses/by-nc-nd/4.0/)

**Links:** [Publisher's page](https://doi.org/10.5121/csit.2024.150112) · [PDF](https://arxiv.org/pdf/2501.12540) · [DOI 10.5121/csit.2024.150112](https://doi.org/10.5121/csit.2024.150112) · [arXiv 2501.12540](https://arxiv.org/abs/2501.12540) · [OpenAlex](https://openalex.org/W4406774496)

**Citations:** 4 citations (openalex-snapshot · 2026-10-14) · 1 in our library, of the 140884 papers whose bibliographies we read

## Summary
The paper asks how well three widely used transformer encoders handle sentiment classification across nine typologically different languages: Arabic and Bulgarian, the Uralic pair Hungarian and Finnish, and five Western European languages (Spanish, English, Italian, German and French). The authors fine-tune BERT (an English-oriented baseline), mBERT and XLM-R separately on corpora for each language. They use subword tokenization, normalization and stopword removal, a cross-entropy objective and early stopping. Each language is scored on its own with accuracy, precision, recall and macro F1, followed by per-class error analysis. BERT is strongest on English. XLM-R leads on Italian, Finnish and Arabic. mBERT trails slightly almost everywhere. The authors attribute XLM-R's edge on morphologically rich languages to its broad multilingual pretraining, and they argue that language-specific fine-tuning matters most for dialectal and informal text. The paper reports a single number per model and language, with no variance and no significance testing, and its prose and tables sometimes disagree. Its value is a rough per-language baseline rather than a definitive ranking.

## Key points
- **Data:** ArSAS for Arabic, the FinnSentiment Corpus for Finnish, and the Twitter-based Open Dataset for Sentiment Analysis for EN/ES/FR/IT/DE. Sources for the Hungarian and Bulgarian data are not stated.
- **Training set sizes** range from 5,040,000 examples for English down to 12,000 for Finnish and 17,600 for Arabic. The validation and test splits are identical in size for every language.
- **Fine-tuning settings:** learning rate 3×10⁻⁵, batch size 32, 5 epochs, maximum sequence length 128.
- **Accuracy:**
  - BERT reaches 91.2% on English, against 90.0% for XLM-R and 89.5% for mBERT.
  - XLM-R reaches 89.0% on Italian, 88.0% on Finnish and 85.0% on Arabic.
  - The lowest score is mBERT's 82.0% on Bulgarian.
- **Inconsistency on Hungarian:** the conclusion calls XLM-R superior on Hungarian, but its own table gives XLM-R 83.0% there, below BERT (85.0%) and mBERT (84.0%). The "above 88%" claim in the abstract therefore holds only for some complex languages.

## Interesting ideas
- Running an English-centric model next to multilingual ones on every language gives a cheap reference point, so any gain from broader pretraining is measured against something concrete. — For Mova (learn): Mova's evaluation could always include a deliberately simple baseline reader, so that improvements claimed by the full tree-to-frames pipeline are measured against a fixed floor.
- Scoring each language separately, with macro averages and per-class error analysis, keeps a large language or a majority class from masking failures elsewhere. — For Mova (learn): Mova's test reports could be broken down by text genre and by question type, so that a weak category such as negation or irony questions cannot hide inside the overall accuracy.
- Pairing the corpus-size table with the results table exposes how strongly scores follow data volume, here roughly 5M versus 12k examples. — For Mova (tree): treebank sizes could be logged alongside per-dialect LAS in gparse experiments, to separate genuine parser weaknesses from plain data scarcity.
- The authors stress that subword segmentation matters most for agglutinative languages, where whole-word vocabularies fragment heavily. — For Mova (tokens): this supports measuring tagger and lemmatizer errors on rare, heavily inflected or compound English forms specifically, instead of only on the frequent vocabulary.
- The authors argue that fine-tuning on dialect-specific and informal data is what pushes performance on social-media text. — For Mova (frames): a separate slice of informal or colloquial input could be used to check whether irony and speech-act detection in prag hold up outside edited prose.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

Fields: [Linguistics & languages](https://md.research.elamur.ai/free/en/shelves/linguistics/)

## Cite

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@misc{sidorov2025comparative,
  title = {{Comparative Approaches to Sentiment Analysis Using Datasets in Major European and Arabic Languages}},
  author = {Sidorov;, Mikhail Krasitskii; Olga Kolesnikova; Liliana Chanona Hernandez; Grigori and Gelbukh, Alexander},
  year = {2025},
  doi = {10.5121/csit.2024.150112},
  eprint = {2501.12540},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.5121/csit.2024.150112}
}
```

- [← previous Cognitive Behaviors that Enable Self-Improving Reasoners, or, Four Habits of Highly Effective STaRs](https://md.research.elamur.ai/free/en/papers/gandhi-2025-cognitive-behaviors-that-enable-self/)
- [next → Comparative constructions in Bisha Colloquial Arabic: a case study](https://md.research.elamur.ai/free/en/papers/alharthy-2025-comparative-constructions-bisha-colloquial-arabic/)
- [↑ back to papers of 2025](https://md.research.elamur.ai/free/en/papers/2025/)

## References

27 references, 2 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. L. Zhu and A. Koivisto, ”XLM-R for Finnish Sentiment Analysis: Capturing Morphological Complexity,” Journal of Computational Linguistics, 2024. Available: https://doi.org/10.1111/finsen2024. [↗](https://doi.org/10.1111/finsen2024)
2. A. Virtanen and H. Paloma¨ki, ”Multilingual BERT and FinBERT for Finnish Social Media Analysis,” Journal of Natural Language Processing, 2019. Available: https://doi.org/10.1234/finbert2019. [↗](https://doi.org/10.1234/finbert2019)
3. K. Kettunen, ”Challenges of Sentiment Analysis in Agglutinative Languages: Case of Finnish,” Journal of Language Technology, 2021. Available: https://doi.org/10.5678/lantech2021. [↗](https://doi.org/10.5678/lantech2021)
4. S. Schweter, ”Fine-Tuning Multilingual BERT for German Sentiment Analysis,” International Journal of Computational Linguistics, 2020. Available: https://doi.org/10.1002/gerbert2020. [↗](https://doi.org/10.1002/gerbert2020)
5. L. Martin et al., ”CamemBERT: RoBERTa-Based Model for French,” Journal of French Language Processing, vol. 15, pp. 55-70, 2020. Available: https://doi.org/10.1093/jflp2020. [↗](https://doi.org/10.1093/jflp2020)
6. C. Guti´errez and J. Rivas, ”Optimizing BERT for Spanish Sentiment Analysis,” Arxiv P reprint, 2022. Available: https://arxiv.org/abs/2203.01567. [↗](https://arxiv.org/abs/2203.01567)
7. M. Polignano and P. Basile, ”AlBERTo: A Sentiment Analysis Model for Italian Social Media,” Pro - ceedings of the Italian NLP Conference, 2019. Available: https://doi.org/10.1002/alberto2019. [↗](https://doi.org/10.1002/alberto2019)
8. W. Antoun, R. Baly, and H. Hajj, ”AraBERT: Transformer-Based Model for Arabic Language Understanding,” Arxiv Preprint, 2020. Available: https://arxiv.org/abs/2003.00104. [↗](https://arxiv.org/abs/2003.00104) [our card](https://md.research.elamur.ai/free/en/papers/antoun-2020-arabert-transformer-based-model-arabic/)
9. M. Abdul-Mageed and E. M. Nagoudi, ”ARBERT and MARBERT: Deep Bidirectional Transformers for Arabic NLP,” Journal of Computational Arabic Linguistics, 2021. Available: https://doi.org/10.1002/arbmbert2021. [↗](https://doi.org/10.1002/arbmbert2021)
10. G. Farkas, ”Sentiment Analysis in Hungarian: Exploring Morphological Adaptations,” Journal of Eastern European Computational Linguistics, 2022. Available: https://doi.org/10.5678/hunsent2022. [↗](https://doi.org/10.5678/hunsent2022)
11. D. Ganchev, ”Exploring the Effectiveness of Multilingual Models for Bulgarian Sentiment Analysis,” Journal of Computational Linguistics, vol. 28, pp. 120-135, 2021. Available: https://doi.org/10.1001/jcl2021. [↗](https://doi.org/10.1001/jcl2021)
12. I. Ganchev, ”Adapting Multilingual BERT for Bulgarian Sentiment Classification,” Journal of Balkan Linguistics, 2023. Available: https://doi.org/10.1111/bulgsent2023. [↗](https://doi.org/10.1111/bulgsent2023)
13. A. Joshi et al., ”Unsupervised Learning for Low-resource Languages: Extending mBERT,” Journal of Low-resource Language Studies, vol. 25, pp. 35-50, 2022. Available: https://doi.org/10.1177/jlr2022. [↗](https://doi.org/10.1177/jlr2022)
14. J. Brown and T. Smith, ”BERT and Sentiment Analysis for English and Anal ytical Languages,” Language Processing Research, vol. 15, no. 3, pp. 225-240, 2023. Available: https://doi.org/10.1002/lpr2023. [↗](https://doi.org/10.1002/lpr2023)
15. X. Liu and R. Wang, ”Multilingual BERT in High Inflection Languages: A Comparative Study,” Journal of AI and Linguistics, 2022. Available: https://doi.org/10.1145/jail2022. [↗](https://doi.org/10.1145/jail2022)
16. A. Conneau et al., ”Unsupervised Cross-lingual Representation Learning at Scale,” Proceedings of the 58th ACL, pp. 8440-8451, 2020. Available: https://doi.org/10.18653/v1/2020.acl-main.747. [↗](https://doi.org/10.18653/v1/2020.acl-main.747)
17. C. Sun, X. Qiu, Y. Xu, ”How to Fine-Tune BERT for Text Classification?” CCF International Conference on NLP and Chinese Computing, 2019. Available: https://doi.org/10.1007/978-3-030-32381-3. [↗](https://doi.org/10.1007/978-3-030-32381-3) [our card](https://md.research.elamur.ai/free/en/papers/sun-2019-fine-tune-bert-text-classification/)
18. S. Kim, ”Sentiment Analysis for Agglutinative Languages: A Cross-linguistic Study,” International Journal of Language and Computing, vol. 29, pp. 65-80, 2023. Available: https://doi.org/10.1177/ijlc2023. [↗](https://doi.org/10.1177/ijlc2023)
19. M. Almutairi et al., ”AraBERT: Enhancing Sentiment Analysis for Arabic Dialects,” Computational Linguistics Journal, vol. 12, pp. 345-361, 2023. Available: https://doi.org/10.1056/arabert2023. [↗](https://doi.org/10.1056/arabert2023)
20. P. Gupta, ”Multilingual Transformers for Dialectal Sentiment Analysis,” ACM Transactions on Com - putational Logic, vol. 23, no. 4, 2022. Available: https://doi.org/10.1109/tcl2022. [↗](https://doi.org/10.1109/tcl2022)
21. L. Huang, ”Exploring mBERT in Underrepresented Languages: Performance and Challenges,” Linguistic Technology Review, vol. 18, pp. 112-130, 2024. Available: https://doi.org/10.1093/ltr2024. [↗](https://doi.org/10.1093/ltr2024)
22. H. Chen and M. Zhang, ”Multilingual Sentiment Classification Using Transformer Models,” Journal of Natural Language Engineering, vol. 32, pp. 102-118, 2023. Available: https://doi.org/10.1007/nle2023. [↗](https://doi.org/10.1007/nle2023)
23. D. Lee, ”Fine-Tuning Transformer Models for Language-Specific Sentiment Analysis,” Computational Intelligence Journal, vol. 29, pp. 57-70, 2023. Available: https://doi.org/10.1016/ci2023. [↗](https://doi.org/10.1016/ci2023)
24. K. Johnson, ”Cross-Lingual Sentiment Analysis Using XLM-R for Social Media Data,” Journal of Social Media Studies, vol. 11, pp. 92-107, 2022. Available: https://doi.org/10.1038/jsms2022. [↗](https://doi.org/10.1038/jsms2022)
25. A. dos Santos, ”Formal vs Informal Registers in Sentiment Analysis: A Study of French,” Language and Technology Review, vol. 17, pp. 95-110, 2021. Available: https://doi.org/10.1145/lt2021. [↗](https://doi.org/10.1145/lt2021)
26. D. Miller, ”Transfer Learning for Sentiment Analysis in Low-resource Languages,” Journal of Language and AI, vol. 14, pp. 90-105, 2023. Available: https://doi.org/10.1001/jla2023. [↗](https://doi.org/10.1001/jla2023)
27. J. Devlin et al., ”BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding,” Proceedings of the 2019 NAACL, pp. 4171-4186, 2019. Available: https://doi.org/10.18653/v1/N19-1423. [↗](https://doi.org/10.18653/v1/n19-1423)