---
title: "Seals-NLP at SemEval-2026 Task 9: A Comparative Study of"
description: "The authors compare three approaches: Fully fine-tuned encoders: BERT-base-uncased… 26 references, cited by 1 paper here."
authors: ["Minh Smith", "Cheryl Seals"]
year: 2026
venue: "Proceedings of the 20th International Workshop on Semantic Evaluation (2026)"
source: "https://aclanthology.org/2026.semeval-1.301/"
doi: "10.18653/v1/2026.semeval-1.301"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/smith-2026-seals-nlp-semeval-2026-task/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/smith-2026-seals-nlp-semeval-2026-task/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/17/) / Seals-NLP at SemEval-2026 Task 9: A Comparative Study of Transformer Architectures for Polarization Detection

# Seals-NLP at SemEval-2026 Task 9: A Comparative Study of Transformer Architectures for Polarization Detection

**Authors:** Minh Smith and [Cheryl Seals](https://md.research.elamur.ai/free/en/authors/cheryl-seals/) · **Year:** 2026 · **Venue:** [Proceedings of the 20th International Workshop on Semantic Evaluation (2026)](https://md.research.elamur.ai/free/en/venues/semeval/) ([SemEval 2026](https://md.research.elamur.ai/free/en/venues/semeval/2026/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2026.semeval-1.301/) · [PDF](https://aclanthology.org/2026.semeval-1.301.pdf) · [DOI 10.18653/v1/2026.semeval-1.301](https://doi.org/10.18653/v1/2026.semeval-1.301) · [OpenAlex](https://openalex.org/W7166877552)

**Citations:** 0 citations (OpenAlex, 2026-10-14) · [1](https://md.research.elamur.ai/free/en/papers/smith-2026-seals-nlp-semeval-2026-task/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
The paper addresses binary polarization detection in English social-media posts (SemEval-2026 Task 9, POLAR, Subtask 1). It asks which modeling family handles this task best, using only openly available models. The authors compare three approaches:

- **Fully fine-tuned encoders:** BERT-base-uncased, DeBERTa-v3-large and ModernBERT-large.
- **QLoRA-adapted LLMs:** Gemma-3-27b-it, Qwen3-32b and EXAONE-4.0-32b, loaded in 4-bit NF4.
- **Zero-shot prompting** of the same three LLMs.

All systems are trained only on the organizers' POLAR splits and evaluated with macro-F1. The comparison also records model storage size and per-input latency.

ModernBERT-large is the strongest and most economical option. It beats every LLM configuration while being orders of magnitude smaller and faster. The quantized fine-tuned LLMs mostly scored below their zero-shot versions.

An error analysis shows that all models react too readily to political keywords. They also tend to miss sarcastic or indirect vilification. For practical deployment, the study suggests that compact encoders remain preferable to large generative models for this focused classification task.

## Key points
- **Data:** English POLAR Subtask 1 has 3,222 train, 160 dev and 1,452 test instances. Roughly one third of each split is polarized. Texts come from X, Reddit, blogs, news comments and forums.
- **Dev-set macro-F1:** ModernBERT-large scored 0.7821 and BERT-base-uncased 0.7802. The best zero-shot LLM, Gemma-3-27b-it-zs, reached 0.7060. The QLoRA models scored only 0.37–0.47; EXAONE-4.0-32b-ft was the only one to beat its zero-shot counterpart.
- **Official leaderboard:** ModernBERT-large reached 0.7832 macro-F1, ranking 32nd of 44. The POLAR baseline scored 0.7802 and the top team 0.8252.
- **Efficiency:** ModernBERT-large is about 1.6 GB with roughly 20 ms latency per input. The LLMs take 55–65 GB and need about 9–51 s per input, with zero-shot slowest because it must generate text. Everything ran on a single RTX 5090.
- **Errors and limitations:**
  - The political subtype accounts for the most errors (186), followed by extreme language and vilification.
  - Religious targeting is handled comparatively well.
  - The evaluation is dev-only, uses a single run and tests a single zero-shot prompt.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Auburn University](https://md.research.elamur.ai/free/en/universities/auburn-university/) — University, Auburn, United States · 3 works · 155 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works

## Cite

Cite as: Minh Smith and Cheryl Seals. 2026. Seals-NLP at SemEval-2026 Task 9: A Comparative Study of Transformer Architectures for Polarization Detection. Proceedings of the 20th International Workshop on Semantic Evaluation (2026).

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{smith2026seals,
  title = {{Seals-NLP at SemEval-2026 Task 9: A Comparative Study of Transformer Architectures for Polarization Detection}},
  author = {Minh Smith and Cheryl Seals},
  booktitle = {Proceedings of the 20th International Workshop on Semantic Evaluation (2026)},
  year = {2026},
  doi = {10.18653/v1/2026.semeval-1.301},
  url = {https://doi.org/10.18653/v1/2026.semeval-1.301}
}
```

- [← previous Seahawk at MedGenVidQA 2026: LLM Segment-Range Selection for Medical Visual Answer Localization](https://md.research.elamur.ai/free/en/papers/tian-2026-seahawk-medgenvidqa-2026-llm-segment/)
- [next → SeCuRepair: Semantics-Aligned, Curriculum-Driven, and Reasoning-Enhanced Vulnerability Repair Framework](https://md.research.elamur.ai/free/en/papers/yang-2026-securepair-semantics-aligned-curriculum-driven/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/17/)

## References

26 references, 9 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Wissam Antoun, Benoît Sagot, and Djamé Seddah. 2025. ModernBERT or DeBERTaV3? Examining architecture and data influence on transformer encoder models performance. In Proceedings of the 14th International Joint Conference on Natural Language Processing and the 4th Conference of the Asia-Pacific Chapt… [find](https://scholar.google.com/scholar?q=Wissam+Antoun%2C+Beno%C3%AEt+Sagot%2C+and+Djam%C3%A9+Seddah%2E+2025%2E+ModernBERT+or+DeBERTaV3%3F+Examining+architecture+and+data+influence+on+transformer+encoder+models+performance%2E+In+Proceedings+of+the+14th+Internatio)
2. Jiashun Cheng, Aochuan Chen, Nuo Chen, Ziqi Gao, Yuhan Li, Jia Li, and Fugee Tsung. 2025. Revisiting LoRA through the lens of parameter redundancy: Spectral encoding helps. In Findings of the Association for Computational Linguistics: ACL 2025, pages 2701–2718, Vienna, Austria. Association for Compu… [find](https://scholar.google.com/scholar?q=Jiashun+Cheng%2C+Aochuan+Chen%2C+Nuo+Chen%2C+Ziqi+Gao%2C+Yuhan+Li%2C+Jia+Li%2C+and+Fugee+Tsung%2E+2025%2E+Revisiting+LoRA+through+the+lens+of+parameter+redundancy%3A+Spectral+encoding+helps%2E+In+Findings+of+the+Associat)
3. Michael D Conover, Jacob Ratkiewicz, Matthew Francisco, Bruno Gonçalves, Filippo Menczer, and Alessandro Flammini. 2011. Political polarization on twitter. In Proceedings of the 5th International AAAI Conference on Weblogs and Social Media (ICWSM), pages 89–96, Barcelona, Spain. AAAI Press. [find](https://scholar.google.com/scholar?q=Michael+D+Conover%2C+Jacob+Ratkiewicz%2C+Matthew+Francisco%2C+Bruno+Gon%C3%A7alves%2C+Filippo+Menczer%2C+and+Alessandro+Flammini%2E+2011%2E+Political+polarization+on+twitter%2E+In+Proceedings+of+the+5th+International+AAAI)
4. Tim Dettmers, Artidoro Pagnoni, Ari Holtzman, and Luke Zettlemoyer. 2023. QLoRA: Efficient fine-tuning of quantized large language models. In Advances in Neural Information Processing Systems (NeurIPS), volume 36, pages 10088–10115. Curran Associates, Inc. [find](https://scholar.google.com/scholar?q=Tim+Dettmers%2C+Artidoro+Pagnoni%2C+Ari+Holtzman%2C+and+Luke+Zettlemoyer%2E+2023%2E+QLoRA%3A+Efficient+fine%2Dtuning+of+quantized+large+language+models%2E+In+Advances+in+Neural+Information+Processing+Systems+%28NeurIPS)
5. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. [BERT: Pre-training of deep bidirectional transformers for language understanding](https://md.research.elamur.ai/free/en/papers/devlin-2019-bert-pre-training-deep-bidirectional/). In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies,…
6. Xudong Han, Junjie Yang, Tianyang Wang, Ziqian Bi, Xinyuan Song, Junfeng Hao, and Junhao Song. 2025. Towards alignment-centric paradigm: A survey of instruction tuning in large language models. Preprint, arXiv:2508.17184. — [arXiv:2508.17184](https://arxiv.org/abs/2508.17184) [find](https://scholar.google.com/scholar?q=Xudong+Han%2C+Junjie+Yang%2C+Tianyang+Wang%2C+Ziqian+Bi%2C+Xinyuan+Song%2C+Junfeng+Hao%2C+and+Junhao+Song%2E+2025%2E+Towards+alignment%2Dcentric+paradigm%3A+A+survey+of+instruction+tuning+in+large+language+models%2E+Prepri)
7. Pengcheng He, Jianfeng Gao, and Weizhu Chen. 2023. [DeBERTaV3: Improving DeBERTa using ELECTRA-style pre-training with gradient-disentangled embedding sharing](https://md.research.elamur.ai/free/en/papers/he-2021-debertav3-improving-deberta-using-electra/). In Proceedings of the 11th International Conference on Learning Representations (ICLR), Kigali, Rwanda. OpenReview.net.
8. Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Lu Wang, Shean Li, and Weizhu Chen. 2022. [LoRA: Low-rank adaptation of large language models](https://md.research.elamur.ai/free/en/papers/hu-2021-lora-low-rank-adaptation-large/). In Proceedings of the 10th International Conference on Learning Representations (ICLR), Virtual. OpenReview.net.
9. Julie Jiang, Xiang Ren, and Emilio Ferrara. 2023. Retweet-BERT: Political leaning detection using language features and information diffusion on social networks. In Proceedings of the 17th International AAAI Conference on Web and Social Media (ICWSM), pages 459–469, Limassol, Cyprus. [find](https://scholar.google.com/scholar?q=Julie+Jiang%2C+Xiang+Ren%2C+and+Emilio+Ferrara%2E+2023%2E+Retweet%2DBERT%3A+Political+leaning+detection+using+language+features+and+information+diffusion+on+social+networks%2E+In+Proceedings+of+the+17th+Internation)
10. LG AI Research. 2025. EXAONE 4.0: Unified large language models integrating non-reasoning and reasoning capabilities. Preprint, arXiv:2507.11407. — [arXiv:2507.11407](https://arxiv.org/abs/2507.11407) [find](https://scholar.google.com/scholar?q=LG+AI+Research%2E+2025%2E+EXAONE+4%2E0%3A+Unified+large+language+models+integrating+non%2Dreasoning+and+reasoning+capabilities%2E+Preprint%2C+arXiv%3A2507%2E11407%2E+%E2%80%94+%5BarXiv%3A2507%2E11407%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2507%2E11407%29)
11. Shih-Yang Liu, Chien-Yi Wang, Hongxu Yin, Pavlo Molchanov, Yu-Chiang Frank Wang, Kwang-Ting Cheng, and Min-Hung Chen. 2024. DoRA: Weight-decomposed low-rank adaptation. In Proceedings of the 41st International Conference on Machine Learning (ICML), volume 235 of Proceedings of Machine Learning Resea… [find](https://scholar.google.com/scholar?q=Shih%2DYang+Liu%2C+Chien%2DYi+Wang%2C+Hongxu+Yin%2C+Pavlo+Molchanov%2C+Yu%2DChiang+Frank+Wang%2C+Kwang%2DTing+Cheng%2C+and+Min%2DHung+Chen%2E+2024%2E+DoRA%3A+Weight%2Ddecomposed+low%2Drank+adaptation%2E+In+Proceedings+of+the+41st+Inte)
12. Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. [RoBERTa: A robustly optimized BERT pretraining approach](https://md.research.elamur.ai/free/en/papers/liu-2019-roberta-robustly-optimized-bert-pretraining/). Preprint, arXiv:1907.11692.
13. Kiran Garimella, Gianmarco De Francisci Morales, Aristides Gionis, and Michael Mathioudakis. 2018. Quantifying controversy on social media. ACM Transactions on Social Computing, 1(1):3:1–3:27. [find](https://scholar.google.com/scholar?q=Kiran+Garimella%2C+Gianmarco+De+Francisci+Morales%2C+Aristides+Gionis%2C+and+Michael+Mathioudakis%2E+2018%2E+Quantifying+controversy+on+social+media%2E+ACM+Transactions+on+Social+Computing%2C+1%281%29%3A3%3A1%E2%80%933%3A27%2E)
14. Renze Lou, Kai Zhang, and Wenpeng Yin. 2024. [Large language model instruction following: A survey of progresses and challenges](https://md.research.elamur.ai/free/en/papers/lou-2024-large-language-model-instruction-following/). Computational Linguistics, 50(3):1053–1095.
15. Gemma Team. 2025. Gemma 3 technical report. Preprint, arXiv:2503.19786. [find](https://scholar.google.com/scholar?q=Gemma+Team%2E+2025%2E+Gemma+3+technical+report%2E+Preprint%2C+arXiv%3A2503%2E19786%2E)
16. Fanxu Meng, Zhaohui Wang, and Muhan Zhang. 2024. PiSSA: Principal singular values and singular vectors adaptation of large language models. In Advances in Neural Information Processing Systems (NeurIPS), volume 37. [find](https://scholar.google.com/scholar?q=Fanxu+Meng%2C+Zhaohui+Wang%2C+and+Muhan+Zhang%2E+2024%2E+PiSSA%3A+Principal+singular+values+and+singular+vectors+adaptation+of+large+language+models%2E+In+Advances+in+Neural+Information+Processing+Systems+%28NeurIP)
17. Vahid Ghafouri. 2025. NLP-Driven Approaches to Measuring Online Polarization and Radicalization. Ph.D. thesis, IMDEA Networks Institute / Universidad Carlos III de Madrid, Madrid, Spain. [find](https://scholar.google.com/scholar?q=Vahid+Ghafouri%2E+2025%2E+NLP%2DDriven+Approaches+to+Measuring+Online+Polarization+and+Radicalization%2E+Ph%2ED%2E+thesis%2C+IMDEA+Networks+Institute+%2F+Universidad+Carlos+III+de+Madrid%2C+Madrid%2C+Spain%2E)
18. Daniela Gottesman, Alon Gilae-Dotan, Ido Cohen, Yoav Gur-Arieh, Marius Mosbach, Ori Yoran, and Mor Geva. 2025. LMEnt: A suite for analyzing knowledge in language models from pretraining data to representations. Preprint, arXiv:2509.03405. [find](https://scholar.google.com/scholar?q=Daniela+Gottesman%2C+Alon+Gilae%2DDotan%2C+Ido+Cohen%2C+Yoav+Gur%2DArieh%2C+Marius+Mosbach%2C+Ori+Yoran%2C+and+Mor+Geva%2E+2025%2E+LMEnt%3A+A+suite+for+analyzing+knowledge+in+language+models+from+pretraining+data+to+repres)
19. Saif Mohammad, Svetlana Kiritchenko, Parinaz Sobhani, Xiaodan Zhu, and Colin Cherry. 2016. [SemEval-2016 Task 6: Detecting stance in tweets](https://md.research.elamur.ai/free/en/papers/mohammad-2016-semeval-2016-task-6-detecting/). In Proceedings of the 10th International Workshop on Semantic Evaluation (SemEval-2016), pages 31– 41, San Diego, California. Association for Computational Ling…
20. Usman Naseem, Robert Geislinger, Juan Ren, Sarah Kohail, Rudy Garrido Veliz, P Sam Sahil, Yiran Zhang, Marco Antonio Stranisci, Idris Abdulmumin, Özge Alacam, Cengiz Acarürk, Aisha Jabr, Saba Anwar, Abinew Ali Ayele, Elena Tutubalina, Aung Kyaw Htet, Xintong Wang, Surendrabikram Thapa, Tanmoy Chakra… — [SemEval-2026 Task 9: Detecting Multilingual, Multicultural and Multievent Online Polarization](https://md.research.elamur.ai/free/en/papers/naseem-2026-semeval-2026-task-9-detecting/)
21. Usman Naseem, Robert Geislinger, Juan Ren, Sarah Kohail, Rudy Garrido Veliz, P Sam Sahil, Yiran Zhang, Marco Antonio Stranisci, Idris Abdulmumin, Özge Alacam, Cengiz Acarürk, Aisha Jabr, Saba Anwar, Abinew Ali Ayele, Simona Frenda, Alessandra Teresa Cignarella, Elena Tutubalina, Oleg Rogov, Aung Kya… — [POLAR: A Benchmark for Multilingual, Multicultural, and Multi-Event Online Polarization](https://md.research.elamur.ai/free/en/papers/naseem-2026-polar-benchmark-multilingual-multicultural-multi/)
22. Steve Nwaiwu. 2025. Parameter-efficient fine-tuning for low-resource text classification: A comparative study of LoRA, IA 3 , and ReFT. Frontiers in Big Data, 8:1677331. [find](https://scholar.google.com/scholar?q=Steve+Nwaiwu%2E+2025%2E+Parameter%2Defficient+fine%2Dtuning+for+low%2Dresource+text+classification%3A+A+comparative+study+of+LoRA%2C+IA+3+%2C+and+ReFT%2E+Frontiers+in+Big+Data%2C+8%3A1677331%2E)
23. Fabian Paischer, Lukas Hauzenberger, Thomas Schmied, Benedikt Alkin, Marc Peter Deisenroth, and Sepp Hochreiter. 2024. One initialization to rule them all: Fine-tuning via explained variance adaptation. In Advances in Neural Information Processing Systems (NeurIPS), volume 37. [find](https://scholar.google.com/scholar?q=Fabian+Paischer%2C+Lukas+Hauzenberger%2C+Thomas+Schmied%2C+Benedikt+Alkin%2C+Marc+Peter+Deisenroth%2C+and+Sepp+Hochreiter%2E+2024%2E+One+initialization+to+rule+them+all%3A+Fine%2Dtuning+via+explained+variance+adaptatio)
24. Marcos Zampieri, Shervin Malmasi, Preslav Nakov, Sara Rosenthal, Noura Farra, and Ritesh Kumar. 2019. [SemEval-2019 Task 6: Identifying and categorizing offensive language in social media (OffensEval)](https://md.research.elamur.ai/free/en/papers/zampieri-2019-semeval-2019-task-6-identifying/). In Proceedings of the 13th International Workshop on Semantic Evaluation (SemEval-2019), pages 75–8…
25. Junyan Zhang, Yiming Huang, Shuliang Liu, Yubo Gao, and Xuming Hu. 2025. Do BERT-like bidirectional models still perform better on text classification in the era of LLMs? In Findings of the Association for Computational Linguistics: EMNLP 2025, pages 18980–18989. Association for Computational Lingui… [find](https://scholar.google.com/scholar?q=Junyan+Zhang%2C+Yiming+Huang%2C+Shuliang+Liu%2C+Yubo+Gao%2C+and+Xuming+Hu%2E+2025%2E+Do+BERT%2Dlike+bidirectional+models+still+perform+better+on+text+classification+in+the+era+of+LLMs%3F+In+Findings+of+the+Associatio)
26. Shengyu Zhang, Linfeng Dong, Xiaoya Li, Sen Zhang, Xiaofei Sun, Shuhe Wang, Jiwei Li, Runyi Hu, Tianwei Zhang, Fei Wu, and Guoyin Wang. 2026. Instruction tuning for large language models: A survey. ACM Computing Surveys, 58(7):169:1–169:36. [find](https://scholar.google.com/scholar?q=Shengyu+Zhang%2C+Linfeng+Dong%2C+Xiaoya+Li%2C+Sen+Zhang%2C+Xiaofei+Sun%2C+Shuhe+Wang%2C+Jiwei+Li%2C+Runyi+Hu%2C+Tianwei+Zhang%2C+Fei+Wu%2C+and+Guoyin+Wang%2E+2026%2E+Instruction+tuning+for+large+language+models%3A+A+survey%2E+AC)

*Updated 31 October 2026*