---
title: "SentiMalti: A Maltese Sentiment Analysis Dataset and Models"
description: "Maltese is a low-resource language, and its existing sentiment data was too small to train strong models. 22 references."
authors: ["Ian Caruana", "Matthew Vella", "Fabio Zammit", "Kurt Micallef", "Claudia Borg"]
year: 2026
venue: "Proceedings of the Fifteenth Language Resources and Evaluation Conference"
source: "https://aclanthology.org/2026.lrec-1.630/"
doi: "10.63317/4kw8df57bza3"
licence: "CC BY-NC 4.0 (ELRA, LREC proceedings)"
licence_url: "https://creativecommons.org/licenses/by-nc/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/caruana-2026-sentimalti-maltese-sentiment-analysis-dataset/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/caruana-2026-sentimalti-maltese-sentiment-analysis-dataset/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/10/) / SentiMalti: A Maltese Sentiment Analysis Dataset and Models

# SentiMalti: A Maltese Sentiment Analysis Dataset and Models

**Authors:** Ian Caruana, Matthew Vella, Fabio Zammit, [Kurt Micallef](https://md.research.elamur.ai/free/en/authors/kurt-micallef/) and [Claudia Borg](https://md.research.elamur.ai/free/en/authors/claudia-borg/) · **Year:** 2026 · **Venue:** [Proceedings of the Fifteenth Language Resources and Evaluation Conference](https://md.research.elamur.ai/free/en/venues/lrec/) ([LREC 2026](https://md.research.elamur.ai/free/en/venues/lrec/2026/)) · **Publisher:** [European Language Resources Association (ELRA)](https://md.research.elamur.ai/free/en/publishers/european-language-resources-association-elra/) · **Licence:** [CC BY-NC 4.0 (ELRA, LREC proceedings)](https://creativecommons.org/licenses/by-nc/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2026.lrec-1.630/) · [PDF](https://aclanthology.org/2026.lrec-1.630.pdf) · [DOI 10.63317/4kw8df57bza3](https://doi.org/10.63317/4kw8df57bza3) · [OpenAlex](https://openalex.org/W7160012156)

**Citations:** 0 citations (OpenAlex, 2026-10-14)

## Summary
Maltese is a low-resource language, and its existing sentiment data was too small to train strong models. Before this work there were two datasets: a 900-comment microblog set and a 555-comment set about the 2018 budget, which is tied to one domain. Earlier fine-tuning of BERTu did worst on sentiment of all the tasks tried, and the limited data was a key reason. The authors collected comments from YouTube, Reddit and Facebook. They cleaned the text, anonymised it, split it into sentences and filtered by language, keeping Maltese sentences even when they mix in English. A custom crowdsourcing website then labelled the sentences with three-way polarity. The result is SentiMalti: 2,327 new sentences, which combine with the older data into a 3,772-item benchmark. The authors compared fine-tuned encoders against few-shot prompting of multilingual LLMs. A five-shot Aya-101 and a fine-tuned BERTu performed almost the same, at about 68 macro-F1. The two models made different kinds of errors. The data splits, code and the BERTu model are publicly released.

## Key points
- The new data has 2,327 sentences: 39% positive, 31% negative and 30% neutral. They come from 6,820 scraped sentences. Sources were YouTube podcast and music channels, the Facebook group RUBS and selected r/malta threads.
- The combined benchmark adds the earlier Microblogs and Budget 2018 sets, for 3,772 instances in total. It is released as MLRS/SentiMalti, and the model as MLRS/BERTu_SentiMalti.
- On the full test set, five-shot Aya-101 scores 68.65 macro-F1 and fine-tuned BERTu scores 68.36. The authors also tested Glot500, Gemma 2 Instruct 9B, mBERTu, mBERT, XLM-R and mmBERT.
- Training data matters for BERTu:
  - only the old data: 62.17;
  - only the new social media data: 66.06;
  - everything combined: 68.36.
- Training only on the new data hurts the Budget 2018 subset: BERTu drops to 49.10, compared with 56.93 when trained on the old data. Error analysis shows BERTu separates positive from negative better, while Aya-101 assigns the neutral label too often.

## Interesting ideas
- The preprocessing runs as a fixed chain of stages, and each stage's output can be checked before the next one starts. Cleaning comes first on purpose, so that the later regex anonymisation and sentence splitting see normalised text. Examples are collapsing repeated punctuation and removing video timestamps.
- The annotation website shows users the items with the fewest labels first and labels by majority vote. This spreads limited volunteer effort evenly instead of piling labels onto a few items.
- Models are trained on old-only, new-only and combined data, and every model is scored on each source subset separately. This shows domain shift directly, for example new-only training failing on the budget domain.
- The two approaches fail in different ways: the prompted LLM leans toward neutral, while the fine-tuned encoder separates the polar classes better. Looking at confusion patterns, not only the overall score, tells you which system to trust for which class.
- Sentence-level language filtering tolerates some code-switching instead of demanding pure-language text. This keeps realistic mixed input in the data.

## For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

- **I1** (tokens): Put an explicit, inspectable normalisation pass in front of the en tokenizer and sentence splitter, and log each stage's output so token-level errors can be traced back to their cause.
- **I2** (learn): When building silver data from witness parsers or LLM teacher tiers, send the next query to the sentences with the fewest or most conflicting votes.
- **I3** (learn): Evaluate each retraining on its own treebank or genre slices, so a gain on one genre cannot hide a loss on another.
- **I4** (learn): Keep confusion profiles for each LLM teacher tier, and discount a teacher's votes on label types it is known to over-predict.
- **I5** (tokens): Allow embedded foreign words in English sentences and tag them, rather than discarding such sentences, so the tagger learns to handle real mixed text.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 9,085 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 8,888 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 9,296 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 9,206 works

🧭 Topics (2): [😊 Sentiment & emotion](https://md.research.elamur.ai/free/en/topics/sentiment-emotion/) · 609 works · [Semitic languages](https://md.research.elamur.ai/free/en/topics/semitic-languages/) · 1,485 works

## Cite

Cite as: Ian Caruana, Matthew Vella, Fabio Zammit, Kurt Micallef and Claudia Borg. 2026. SentiMalti: A Maltese Sentiment Analysis Dataset and Models. Proceedings of the Fifteenth Language Resources and Evaluation Conference.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{caruana2026sentimalti,
  title = {{SentiMalti: A Maltese Sentiment Analysis Dataset and Models}},
  author = {Ian Caruana, Matthew Vella, Fabio Zammit, Kurt Micallef and Claudia Borg},
  booktitle = {Proceedings of the Fifteenth Language Resources and Evaluation Conference},
  year = {2026},
  doi = {10.63317/4kw8df57bza3},
  url = {https://doi.org/10.63317/4kw8df57bza3}
}
```

- [← previous SemToken: Semantic-Aware Tokenization for Efficient Long-Context Language Models](https://md.research.elamur.ai/free/en/papers/liu-2026-semtoken-semantic-aware-tokenization-efficient/)
- [next → Sentiment Analysis and Language Models for Kwanyama](https://md.research.elamur.ai/free/en/papers/nakashole-2026-sentiment-analysis-language-models-kwanyama/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/10/)

## References

22 references, 8 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Ibrahim Abu Farha and Walid Magdy. 2020. [From Arabic sentiment analysis to sarcasm detection: The ArSarcasm dataset](https://md.research.elamur.ai/free/en/papers/farha-2020-arabic-sentiment-analysis-sarcasm-detection/). In Proceedings of the 4th Workshop on Open-Source Arabic Corpora and Processing Tools, with a Shared Task on Offensive Language Detection, pages 32–39, Marseille, France. European Language Resource Association. (b) Aya-101 Five-Shot Prompting Alexis Conneau, Kartikay Khandelwal, Naman Goyal, Vishrav Chaudhary, Guillaume Wenzek, Francisco Guzmán, Edouard Grave, Myle Ott, Luke Zettlemoyer, and Veselin Stoyanov. 2020. Unsupervised cross-lingual representation learning at scale. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 8440–8451, Online. Association for Computational Linguistics.
2. Keith Cortis and Brian Davis. 2019. A social opinion gold standard for the Malta government budget 2018. In Proceedings of the 5th Workshop on Noisy User-generated Text (W-NUT 2019), pages 364–369, Hong Kong, China. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Keith+Cortis+and+Brian+Davis%2E+2019%2E+A+social+opinion+gold+standard+for+the+Malta+government+budget+2018%2E+In+Proceedings+of+the+5th+Workshop+on+Noisy+User%2Dgenerated+Text+%28W%2DNUT+2019%29%2C+pages+364%E2%80%93369%2C+Ho)
3. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. [BERT: Pre-training of deep bidirectional transformers for language understanding](https://md.research.elamur.ai/free/en/papers/devlin-2019-bert-pre-training-deep-bidirectional/). In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 4171–4186, Minneapolis, Minnesota. Association for Computational Linguistics. [↗](https://doi.org/10.18653/v1/n19-1423)
4. Alexei Dingli and Nicole Sant. 2016. Sentiment analysis on Maltese using machine learning. In The Tenth International Conference on Advances in Semantic Processing (SEMAPRO 2016), pages 21–25. [find](https://scholar.google.com/scholar?q=Alexei+Dingli+and+Nicole+Sant%2E+2016%2E+Sentiment+analysis+on+Maltese+using+machine+learning%2E+In+The+Tenth+International+Conference+on+Advances+in+Semantic+Processing+%28SEMAPRO+2016%29%2C+pages+21%E2%80%9325%2E)
5. Aleksandra Gabryszak and Philippe Thomas. 2022. MobASA: Corpus for aspect-based sentiment analysis and social inclusion in the mobility domain. In Proceedings of the First Computing Social Responsibility Workshop within the 13th Language Resources and Evaluation Conference, pages 35–39, Marseille, France. European Language Resources Association. [find](https://scholar.google.com/scholar?q=Aleksandra+Gabryszak+and+Philippe+Thomas%2E+2022%2E+MobASA%3A+Corpus+for+aspect%2Dbased+sentiment+analysis+and+social+inclusion+in+the+mobility+domain%2E+In+Proceedings+of+the+First+Computing+Social+Responsibil)
6. Leo Gao, Jonathan Tow, Baber Abbasi, Stella Biderman, Sid Black, Anthony DiPofi, Charles Foster, Laurence Golding, Jeffrey Hsu, Alain Le Noac’h, et al. 2024. The language model evaluation harness. [find](https://scholar.google.com/scholar?q=Leo+Gao%2C+Jonathan+Tow%2C+Baber+Abbasi%2C+Stella+Biderman%2C+Sid+Black%2C+Anthony+DiPofi%2C+Charles+Foster%2C+Laurence+Golding%2C+Jeffrey+Hsu%2C+Alain+Le+Noac%E2%80%99h%2C+et+al%2E+2024%2E+The+language+model+evaluation+harness%2E)
7. Gemma Team, Morgane Riviere, Shreya Pathak, Pier Giuseppe Sessa, Cassidy Hardin, Surya Bhupatiraju, Léonard Hussenot, Thomas Mesnard, Bobak Shahriari, Alexandre Ramé, et al. 2024. Gemma 2: Improving open language models at a practical size. [↗](https://doi.org/10.48550/arxiv.2408.00118)
8. Lucie Gianola, Ēriks Ajausks, Victoria Arranz, Chomicha Bendahman, Laurent Bié, Claudia Borg, Aleix Cerdà, Khalid Choukri, Montse Cuadros, Ona de Gibert, et al. 2020. Automatic Removal of Identifying Information in Official EU Languages for Public Administrations: The MAPA Project. In Legal Knowledge and Information Systems, volume 334 of Legal Knowledge and Information Systems, pages 223–226, Brno, Prague, Czech Republic. IOS Press. [↗](https://doi.org/10.3233/faia200869)
9. Ayyoob Imani, Peiqin Lin, Amir Hossein Kargaran, Silvia Severini, Masoud Jalili Sabet, Nora Kassner, Chunlan Ma, Helmut Schmid, André Martins, François Yvon, et al. 2023. Glot500: Scaling multilingual corpora and language models to 500 languages. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1082–1117, Toronto, Canada. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Ayyoob+Imani%2C+Peiqin+Lin%2C+Amir+Hossein+Kargaran%2C+Silvia+Severini%2C+Masoud+Jalili+Sabet%2C+Nora+Kassner%2C+Chunlan+Ma%2C+Helmut+Schmid%2C+Andr%C3%A9+Martins%2C+Fran%C3%A7ois+Yvon%2C+et+al%2E+2023%2E+Glot500%3A+Scaling+multilingual)
10. Marco Lui and Timothy Baldwin. 2012. [langid.py: An off-the-shelf language identification tool](https://md.research.elamur.ai/free/en/papers/lui-2012-langid-py-off-shelf-language/). In Proceedings of the ACL 2012 System Demonstrations, pages 25–30, Jeju Island, Korea. Association for Computational Linguistics.
11. Marc Marone, Orion Weller, William Fleshman, Eugene Yang, Dawn Lawrie, and Benjamin Van Durme. 2025. [mmBERT: A modern multilingual encoder with annealed language learning](https://md.research.elamur.ai/free/en/papers/marone-2025-mmbert-modern-multilingual-encoder-annealed/). [↗](https://doi.org/10.48550/arxiv.2509.06888)
12. Antonio Martínez-García, Toni Badia, and Jeremy Barnes. 2021. Evaluating morphological typology in zero-shot cross-lingual transfer. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 3136–3153, Online. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Antonio+Mart%C3%ADnez%2DGarc%C3%ADa%2C+Toni+Badia%2C+and+Jeremy+Barnes%2E+2021%2E+Evaluating+morphological+typology+in+zero%2Dshot+cross%2Dlingual+transfer%2E+In+Proceedings+of+the+59th+Annual+Meeting+of+the+Association+for+Co)
13. Kurt Micallef and Claudia Borg. 2025. MELABenchv1: Benchmarking large language models against smaller fine-tuned models for low-resource Maltese NLP. In Findings of the Association for Computational Linguistics: ACL 2025, pages 20505–20527, Vienna, Austria. Association for Computational Linguistics. [↗](https://doi.org/10.48448/2ccs-m780)
14. Kurt Micallef, Albert Gatt, Marc Tanti, Lonneke van der Plas, and Claudia Borg. 2022. Pre-training data quality and quantity for a low-resource language: New corpus and BERT models for Maltese. In Proceedings of the Third Workshop on Deep Learning for Low-Resource Natural Language Processing, pages 90–101, Hybrid. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Kurt+Micallef%2C+Albert+Gatt%2C+Marc+Tanti%2C+Lonneke+van+der+Plas%2C+and+Claudia+Borg%2E+2022%2E+Pre%2Dtraining+data+quality+and+quantity+for+a+low%2Dresource+language%3A+New+corpus+and+BERT+models+for+Maltese%2E+In+Pro)
15. Sara Rosenthal, Noura Farra, and Preslav Nakov. 2017. [SemEval-2017 task 4: Sentiment analysis in Twitter](https://md.research.elamur.ai/free/en/papers/rosenthal-2017-semeval-2017-task-4-sentiment/). In Proceedings of the 11th International Workshop on Semantic Evaluation (SemEval-2017), pages 502–518, Vancouver, Canada. Association for Computational Linguistics.
16. Marco Stranisci, Cristina Bosco, Delia Irazú Hernández Farías, and Viviana Patti. 2016. [Annotating sentiment and irony in the online Italian political debate on #labuonascuola](https://md.research.elamur.ai/free/en/papers/stranisci-2016-annotating-sentiment-irony-online-italian/). In Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC’16), pages 2892–2899, Portorož, Slovenia. European Language Resources Association (ELRA).
17. Ahmet Üstün, Viraat Aryabumi, Zheng Yong, Wei-Yin Ko, Daniel D’souza, Gbemileke Onilude, Neel Bhandari, Shivalika Singh, Hui-Lee Ooi, Amr Kayid, et al. 2024. [Aya model: An instruction finetuned open-access multilingual language model](https://md.research.elamur.ai/free/en/papers/ustun-2024-aya-model-instruction-finetuned-open/). In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 15894–15939, Bangkok, Thailand. Association for Computational Linguistics. [↗](https://doi.org/10.48550/arxiv.2402.07827)
18. Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Remi Louf, Morgan Funtowicz, et al. 2020. [Transformers: State-of-the-art natural language processing](https://md.research.elamur.ai/free/en/papers/wolf-2020-transformers-state-art-natural-language/). In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pages 38–45, Online. Association for Computational Linguistics.
19. Miaoran Zhang, Vagrant Gautam, Mingyang Wang, Jesujoba Alabi, Xiaoyu Shen, Dietrich Klakow, and Marius Mosbach. 2024. The impact of demonstrations on multilingual in-context learning: A multidimensional analysis. In Findings of the Association for Computational Linguistics: ACL 2024, pages 7342–7371, Bangkok, Thailand. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Miaoran+Zhang%2C+Vagrant+Gautam%2C+Mingyang+Wang%2C+Jesujoba+Alabi%2C+Xiaoyu+Shen%2C+Dietrich+Klakow%2C+and+Marius+Mosbach%2E+2024%2E+The+impact+of+demonstrations+on+multilingual+in%2Dcontext+learning%3A+A+multidimension)
20. Model Budget 2018 Microblogs Social Media All (SentiMalti) [find](https://scholar.google.com/scholar?q=Model+Budget+2018+Microblogs+Social+Media+All+%28SentiMalti%29)
21. Fine-Tuning with Previous Data Only (Budget 2018 + Microblogs) BERTu 56.93 ± 2.73 65.35 ± 2.75 60.63 ± 1.25 62.17 ± 0.32 mBERTu 61.07 ± 2.62 58.63 ± 1.22 54.66 ± 1.83 57.12 ± 1.31 mBERT 48.00 ± 3.31 45.07 ± 2.25 39.84 ± 2.63 42.97 ± 2.30 XLM-R 52.33 ± 1.87 43.52 ± 3.16 42.24 ± 1.69 44.79 ± 1.58 Glot500 61.33 ± 2.61 56.83 ± 1.46 51.82 ± 1.10 54.92 ± 1.27 mmBERT 55.81 ± 3.15 56.91 ± 4.57 47.31 ± 0.39 51.23 ± 1.26 Fine-Tuning with New Data Only (Social Media) BERTu 49.10 ± 3.05 65.55 ± 1.94 68.25 ± 0.99 66.06 ± 1.00 mBERTu 49.99 ± 2.74 60.23 ± 1.92 66.15 ± 0.97 63.59 ± 1.46 mBERT 37.42 ± 3.06 49.45 ± 1.81 52.63 ± 2.06 50.88 ± 1.58 XLM-R 38.58 ± 5.60 43.03 ± 1.62 55.33 ± 1.06 50.67 ± 0.89 Glot500 42.57 ± 4.32 52.65 ± 1.84 59.44 ± 1.93 56.55 ± 1.63 mmBERT 52.01 ± 2.55 56.16 ± 2.23 59.04 ± 1.47 58.71 ± 0.78 [find](https://scholar.google.com/scholar?q=Fine%2DTuning+with+Previous+Data+Only+%28Budget+2018+%2B+Microblogs%29+BERTu+56%2E93+%C2%B1+2%2E73+65%2E35+%C2%B1+2%2E75+60%2E63+%C2%B1+1%2E25+62%2E17+%C2%B1+0%2E32+mBERTu+61%2E07+%C2%B1+2%2E62+58%2E63+%C2%B1+1%2E22+54%2E66+%C2%B1+1%2E83+57%2E12+%C2%B1+1%2E31+mBERT+48%2E00+%C2%B1+3%2E31+4)
22. Fine-Tuning with All (SentiMalti) Data (Budget 2018 + Microblogs + Social Media) BERTu 57.65 ± 1.43 67.71 ± 2.64 68.88 ± 0.37 68.36 ± 0.62 mBERTu 55.40 ± 2.81 62.64 ± 0.69 67.31 ± 0.70 65.83 ± 0.27 mBERT 49.27 ± 1.07 50.23 ± 1.27 53.02 ± 1.56 53.48 ± 0.92 XLM-R 58.57 ± 3.65 47.39 ± 3.20 57.53 ± 1.77 57.43 ± 1.34 Glot500 59.42 ± 3.30 58.56 ± 2.30 61.67 ± 1.42 62.00 ± 1.52 mmBERT 60.73 ± 3.11 58.13 ± 1.86 60.14 ± 1.22 61.24 ± 1.18 Table 3: Results of all fine-tuned encoder-only (including the ones from Table 3) in terms of macro-averaged F1 (based on 5 runs with different random seeds). The best average score for each test set is shown in bold and underline. A. Results on additional encoder-only models [find](https://scholar.google.com/scholar?q=Fine%2DTuning+with+All+%28SentiMalti%29+Data+%28Budget+2018+%2B+Microblogs+%2B+Social+Media%29+BERTu+57%2E65+%C2%B1+1%2E43+67%2E71+%C2%B1+2%2E64+68%2E88+%C2%B1+0%2E37+68%2E36+%C2%B1+0%2E62+mBERTu+55%2E40+%C2%B1+2%2E81+62%2E64+%C2%B1+0%2E69+67%2E31+%C2%B1+0%2E70+65%2E83+%C2%B1+0%2E27+mB)

*Updated 24 October 2026*