---
title: "UMUTeam at SemEval-2026 Task 10: Transformer Ensembles"
description: "This paper describes UMUTeam's system for SemEval-2026 Task 10 (PsyCoMark). 0.7700 accuracy. 22 references, cited by 1 paper here."
authors: ["Jorge Gómez-Navalón", "Ronghao Pan", "Tomás Bernal-Beltrán", "José Antonio García-Díaz", "Rafael Valencia-Garcia"]
year: 2026
venue: "Proceedings of the 20th International Workshop on Semantic Evaluation (2026)"
source: "https://aclanthology.org/2026.semeval-1.69/"
doi: "10.18653/v1/2026.semeval-1.69"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/navalon-2026-umuteam-semeval-2026-task-10/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/navalon-2026-umuteam-semeval-2026-task-10/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/18/) / UMUTeam at SemEval-2026 Task 10: Transformer Ensembles for Conspiratorial Span Extraction and Detection

# UMUTeam at SemEval-2026 Task 10: Transformer Ensembles for Conspiratorial Span Extraction and Detection

**Authors:** Jorge Gómez-Navalón, [Ronghao Pan](https://md.research.elamur.ai/free/en/authors/ronghao-pan/), [Tomás Bernal-Beltrán](https://md.research.elamur.ai/free/en/authors/tomas-bernal-beltran/), José Antonio García-Díaz and [Rafael Valencia-Garcia](https://md.research.elamur.ai/free/en/authors/rafael-valencia-garcia/) · **Year:** 2026 · **Venue:** [Proceedings of the 20th International Workshop on Semantic Evaluation (2026)](https://md.research.elamur.ai/free/en/venues/semeval/) ([SemEval 2026](https://md.research.elamur.ai/free/en/venues/semeval/2026/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2026.semeval-1.69/) · [PDF](https://aclanthology.org/2026.semeval-1.69.pdf) · [DOI 10.18653/v1/2026.semeval-1.69](https://doi.org/10.18653/v1/2026.semeval-1.69) · [OpenAlex](https://openalex.org/W7166874275)

**Citations:** 0 citations (OpenAlex, 2026-10-14) · [1](https://md.research.elamur.ai/free/en/papers/navalon-2026-umuteam-semeval-2026-task-10/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
This paper describes UMUTeam's system for SemEval-2026 Task 10 (PsyCoMark). The task covers two problems in Reddit submission statements. The first requires finding text spans that signal any of five psycholinguistic categories: Actor, Victim, Action, Threat and Evidence. The second is a binary decision on whether a text is conspiratorial.

The authors fine-tuned four encoders: DeBERTa-v3-large, RoBERTa-large, EuroBERT-610M and ModernBERT-large. They trained each with cross-validation and combined the resulting models in ensembles.

- **Extraction:** markers were cast as BIOES token tagging. Predictions from all models and folds were merged by token-set voting. Extra specialist models targeted the unstable Evidence and Threat markers and were used to filter candidate spans.
- **Detection:** "Can't tell" items were dropped. The class probabilities from each model were averaged with manually tuned weights.

The detection ensemble reached a weighted F1 of 0.7688, placing 13th of 52 teams. Extraction scores were much lower, with an aggregate token F1 of 0.2551, but this placed 5th of 30.

The work shows that ensembles help on both levels and that fine-grained marker spans remain hard. Most extraction errors come from boundary mistakes and from confusing Action with Evidence.

## Key points
- **Data:** the PsyCoMark training split holds 1541 "Yes", 1990 "No" and 785 "Can't tell" instances, with character-level marker annotations.
- **Span voting:** a span is kept only if at least 3 model/fold predictions support it. Among overlapping candidates, the one with the most support wins.
- **Subtask 2:** the ensemble scored 0.7700 accuracy and 0.7688 weighted F1. The single DeBERTa-v3-large model scored 0.7403 accuracy and about 0.7273 weighted F1.
- **Subtask 1:** the final system scored 0.2551 aggregate F1 and 0.2411 macro F1. The single DeBERTa-v3-large model scored about 0.15 aggregate and 0.11 macro. Adding specialists gave the largest gain.
- **Per-marker F1:** Actor was easiest (0.3991), followed by Victim (0.2654), Threat (0.1891), Action (0.1783) and Evidence (0.1738). Sarcastic or implicit texts were the main source of classification errors.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Universidad de Murcia](https://md.research.elamur.ai/free/en/universities/universidad-de-murcia/) — University, Murcia, Spain · 5 works · 3 citations

📚 Shelves (6): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works · [🤝 Social Sciences](https://md.research.elamur.ai/free/en/shelves/social-sciences/) · 1,772 works · [🤝 Sociology and Political Science](https://md.research.elamur.ai/free/en/shelves/social-sciences/sociology-and-political-science/) · 577 works

## Cite

Cite as: Jorge Gómez-Navalón, Ronghao Pan, Tomás Bernal-Beltrán, José Antonio García-Díaz and Rafael Valencia-Garcia. 2026. UMUTeam at SemEval-2026 Task 10: Transformer Ensembles for Conspiratorial Span Extraction and Detection. Proceedings of the 20th International Workshop on Semantic Evaluation (2026).

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{gomeznavalon2026umuteam,
  title = {{UMUTeam at SemEval-2026 Task 10: Transformer Ensembles for Conspiratorial Span Extraction and Detection}},
  author = {Jorge Gómez-Navalón, Ronghao Pan, Tomás Bernal-Beltrán, José Antonio García-Díaz and Rafael Valencia-Garcia},
  booktitle = {Proceedings of the 20th International Workshop on Semantic Evaluation (2026)},
  year = {2026},
  doi = {10.18653/v1/2026.semeval-1.69},
  url = {https://doi.org/10.18653/v1/2026.semeval-1.69}
}
```

- [← previous UltraEval-Audio: A Unified Framework for Comprehensive Evaluation of Audio Foundation Models](https://md.research.elamur.ai/free/en/papers/shi-2026-ultraeval-audio-unified-framework-comprehensive/)
- [next → Uncertainty Quantification for Evaluating Gender Bias in Machine Translation](https://md.research.elamur.ai/free/en/papers/staliunaite-2026-uncertainty-quantification-evaluating-gender-bias/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/18/)

## References

22 references, 5 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Stephane J Baele. 2019. Conspiratorial narratives in violent political actors’ language. Journal of language and social psychology, 38(5-6):706–734. [find](https://scholar.google.com/scholar?q=Stephane+J+Baele%2E+2019%2E+Conspiratorial+narratives+in+violent+political+actors%E2%80%99+language%2E+Journal+of+language+and+social+psychology%2C+38%285%2D6%29%3A706%E2%80%93734%2E)
2. Nicolas Boizard, Hippolyte Gisserot-Boukhlef, Duarte M. Alves, André Martins, Ayoub Hammal, Caio Corro, Céline Hudelot, Emmanuel Malherbe, Etienne Malaboeuf, Fanny Jourdan, Gabriel Hautreux, João Alves, Kevin El-Haddad, Manuel Faysse, Maxime Peyrard, Nuno M. Guerreiro, Patrick Fernandes, Ricardo Rei… — [EuroBERT: Scaling Multilingual Encoders for European Languages](https://md.research.elamur.ai/free/en/papers/boizard-2025-eurobert-scaling-multilingual-encoders-european/)
3. Toby Bolsen, Risa Palm, and Justin T Kingsland. 2022. Effects of conspiracy rhetoric on views about the consequences of climate change and support for direct carbon capture. Environmental Communication, 16(2):209–224. [find](https://scholar.google.com/scholar?q=Toby+Bolsen%2C+Risa+Palm%2C+and+Justin+T+Kingsland%2E+2022%2E+Effects+of+conspiracy+rhetoric+on+views+about+the+consequences+of+climate+change+and+support+for+direct+carbon+capture%2E+Environmental+Communicatio)
4. J. Compton, S. Linden, J. Cook, and Melisa Basol. 2021. Inoculation theory in the post-truth era: Extant findings and new frontiers for contested science, misinformation, and conspiracy theories. Social and Personality Psychology Compass, 15. [find](https://scholar.google.com/scholar?q=J%2E+Compton%2C+S%2E+Linden%2C+J%2E+Cook%2C+and+Melisa+Basol%2E+2021%2E+Inoculation+theory+in+the+post%2Dtruth+era%3A+Extant+findings+and+new+frontiers+for+contested+science%2C+misinformation%2C+and+conspiracy+theories%2E+Soci)
5. Ahmad Diab, Rr Nefriana, and Yu-Ru Lin. 2024. Classifying conspiratorial narratives at scale: False alarms and erroneous connections. In Proceedings of the International AAAI Conference on Web and Social Media, volume 18, pages 340–353. [find](https://scholar.google.com/scholar?q=Ahmad+Diab%2C+Rr+Nefriana%2C+and+Yu%2DRu+Lin%2E+2024%2E+Classifying+conspiratorial+narratives+at+scale%3A+False+alarms+and+erroneous+connections%2E+In+Proceedings+of+the+International+AAAI+Conference+on+Web+and+Soc)
6. José Antonio García-Díaz, Ronghao Pan, and Rafael Valencia-García. 2023. Leveraging zero and few-shot learning for enhanced model generality in hate speech detection in spanish and english. Mathematics, 11(24):5004. [find](https://scholar.google.com/scholar?q=Jos%C3%A9+Antonio+Garc%C3%ADa%2DD%C3%ADaz%2C+Ronghao+Pan%2C+and+Rafael+Valencia%2DGarc%C3%ADa%2E+2023%2E+Leveraging+zero+and+few%2Dshot+learning+for+enhanced+model+generality+in+hate+speech+detection+in+spanish+and+english%2E+Mathematic)
7. Dax Gerts, Courtney D Shelley, Nidhi Parikh, Travis Pitts, Chrysm Watson Ross, Geoffrey Fairchild, Nidia Yadria Vaquera Chavez, and Ashlynn R Daughton. [find](https://scholar.google.com/scholar?q=Dax+Gerts%2C+Courtney+D+Shelley%2C+Nidhi+Parikh%2C+Travis+Pitts%2C+Chrysm+Watson+Ross%2C+Geoffrey+Fairchild%2C+Nidia+Yadria+Vaquera+Chavez%2C+and+Ashlynn+R+Daughton%2E)
8. Yubo Kou, Xinning Gui, Yunan Chen, and Kathleen H. Pine. 2017. Conspiracy talk on social media. Proceedings of the ACM on Human-Computer Interaction, 1:1 – 21. [find](https://scholar.google.com/scholar?q=Yubo+Kou%2C+Xinning+Gui%2C+Yunan+Chen%2C+and+Kathleen+H%2E+Pine%2E+2017%2E+Conspiracy+talk+on+social+media%2E+Proceedings+of+the+ACM+on+Human%2DComputer+Interaction%2C+1%3A1+%E2%80%93+21%2E)
9. Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. [Roberta: A robustly optimized bert pretraining approach](https://md.research.elamur.ai/free/en/papers/liu-2019-roberta-robustly-optimized-bert-pretraining/). arXiv preprint arXiv:1907.11692.
10. Zhiwei Liu, Boyang Liu, Paul Thompson, Kailai Yang, and Sophia Ananiadou. 2024. Conspemollm: Conspiracy theory detection using an emotion-based large language model. arXiv preprint arXiv:2403.06765. — [arXiv:2403.06765](https://arxiv.org/abs/2403.06765) [find](https://scholar.google.com/scholar?q=Zhiwei+Liu%2C+Boyang+Liu%2C+Paul+Thompson%2C+Kailai+Yang%2C+and+Sophia+Ananiadou%2E+2024%2E+Conspemollm%3A+Conspiracy+theory+detection+using+an+emotion%2Dbased+large+language+model%2E+arXiv+preprint+arXiv%3A2403%2E06765%2E+%E2%80%94)
11. Zhiwei Liu, Paul Thompson, Jiaqi Rong, and Sophia Ananiadou. 2025. Conspemollm-v2: A robust and stable model to detect sentiment-transformed conspiracy theories. arXiv preprint arXiv:2505.14917. — [arXiv:2505.14917](https://arxiv.org/abs/2505.14917) [find](https://scholar.google.com/scholar?q=Zhiwei+Liu%2C+Paul+Thompson%2C+Jiaqi+Rong%2C+and+Sophia+Ananiadou%2E+2025%2E+Conspemollm%2Dv2%3A+A+robust+and+stable+model+to+detect+sentiment%2Dtransformed+conspiracy+theories%2E+arXiv+preprint+arXiv%3A2505%2E14917%2E+%E2%80%94+%5Bar)
12. J. D. Moffitt, Catherine King, and K. Carley. 2021. Hunting conspiracy theories during the covid-19 pandemic. Social Media + Society, 7. [find](https://scholar.google.com/scholar?q=J%2E+D%2E+Moffitt%2C+Catherine+King%2C+and+K%2E+Carley%2E+2021%2E+Hunting+conspiracy+theories+during+the+covid%2D19+pandemic%2E+Social+Media+%2B+Society%2C+7%2E)
13. Cian O’Mahony, M. Brassil, G. Murphy, and Conor Linehan. 2023. The efficacy of interventions in reducing belief in conspiracy theories: A systematic review. PLOS ONE, 18. [find](https://scholar.google.com/scholar?q=Cian+O%E2%80%99Mahony%2C+M%2E+Brassil%2C+G%2E+Murphy%2C+and+Conor+Linehan%2E+2023%2E+The+efficacy+of+interventions+in+reducing+belief+in+conspiracy+theories%3A+A+systematic+review%2E+PLOS+ONE%2C+18%2E)
14. Eleftheria Papageorgiou, Christos Chronis, Iraklis Varlamis, and Yassine Himeur. 2024a. A survey on the use of large language models (llms) in fake news. Future Internet, 16:298. [find](https://scholar.google.com/scholar?q=Eleftheria+Papageorgiou%2C+Christos+Chronis%2C+Iraklis+Varlamis%2C+and+Yassine+Himeur%2E+2024a%2E+A+survey+on+the+use+of+large+language+models+%28llms%29+in+fake+news%2E+Future+Internet%2C+16%3A298%2E)
15. Eleftheria Papageorgiou, Christos Chronis, Iraklis Varlamis, and Yassine Himeur. 2024b. A survey on the use of large language models (llms) in fake news. Future Internet, 16(8):298. [find](https://scholar.google.com/scholar?q=Eleftheria+Papageorgiou%2C+Christos+Chronis%2C+Iraklis+Varlamis%2C+and+Yassine+Himeur%2E+2024b%2E+A+survey+on+the+use+of+large+language+models+%28llms%29+in+fake+news%2E+Future+Internet%2C+16%288%29%3A298%2E)
16. Youri Peskine, Giulio Alfarano, Ismail Harrando, Paolo Papotti, and Raphael Troncy. 2021. Detecting covid-19-related conspiracy theories in tweets. In MediaEval. [find](https://scholar.google.com/scholar?q=Youri+Peskine%2C+Giulio+Alfarano%2C+Ismail+Harrando%2C+Paolo+Papotti%2C+and+Raphael+Troncy%2E+2021%2E+Detecting+covid%2D19%2Drelated+conspiracy+theories+in+tweets%2E+In+MediaEval%2E)
17. Lotte Pummerer, Robert Böhm, Lau Lilleholt, Kevin Winter, Ingo Zettler, and K. Sassenberg. 2020. Conspiracy theories and their societal effects during the covid-19 pandemic. Social Psychological and Personality Science, 13:49 – 59. [find](https://scholar.google.com/scholar?q=Lotte+Pummerer%2C+Robert+B%C3%B6hm%2C+Lau+Lilleholt%2C+Kevin+Winter%2C+Ingo+Zettler%2C+and+K%2E+Sassenberg%2E+2020%2E+Conspiracy+theories+and+their+societal+effects+during+the+covid%2D19+pandemic%2E+Social+Psychological+and+P)
18. Mattia Samory, Felix Soldner, and Veronika Batzdorfer. 2026. [SemEval-2026 task 10: PsyCoMark – psycholinguistic conspiracy marker extraction and detection](https://md.research.elamur.ai/free/en/papers/samory-2026-semeval-2026-task-10-psycomark/). In Proceedings of the 20th International Workshop on Semantic Evaluation (SemEval-2026).
19. Benjamin Warner, Antoine Chaffin, Benjamin Clavié, Orion Weller, Oskar Hallström, Said Taghadouini, Alexis Gallagher, Raja Biswas, Faisal Ladhak, Tom Aarsen, Nathan Cooper, Griffin Adams, Jeremy Howard, and Iacopo Poli. 2024. [Smarter, better, faster, longer: A modern bidirectional encoder for fast,…](https://md.research.elamur.ai/free/en/papers/warner-2024-smarter-better-faster-longer-modern/)
20. Lu Xu, Yew Ken Chia, and Lidong Bing. 2021. [Learning span-level interactions for aspect sentiment triplet extraction](https://md.research.elamur.ai/free/en/papers/xu-2021-learning-span-level-interactions-aspect/). ArXiv, abs/2107.12214.
21. Jie Yu, Bin Ji, Shasha Li, Jun Ma, Huijun Liu, and Hao Xu. 2022. S-ner: A concise and efficient span-based model for named entity recognition. Sensors (Basel, Switzerland), 22. [find](https://scholar.google.com/scholar?q=Jie+Yu%2C+Bin+Ji%2C+Shasha+Li%2C+Jun+Ma%2C+Huijun+Liu%2C+and+Hao+Xu%2E+2022%2E+S%2Dner%3A+A+concise+and+efficient+span%2Dbased+model+for+named+entity+recognition%2E+Sensors+%28Basel%2C+Switzerland%29%2C+22%2E)
22. Zhengqiao Zeng, Zhongyuan Han, Jingyan Ye, Yaozu Tan, Haojie Cao, Zengyao Li, and Runjin Huang. 2024. A conspiracy theory text detection method based on roberta and xlm-roberta models. pages 3002–3006. [find](https://scholar.google.com/scholar?q=Zhengqiao+Zeng%2C+Zhongyuan+Han%2C+Jingyan+Ye%2C+Yaozu+Tan%2C+Haojie+Cao%2C+Zengyao+Li%2C+and+Runjin+Huang%2E+2024%2E+A+conspiracy+theory+text+detection+method+based+on+roberta+and+xlm%2Droberta+models%2E+pages+3002%E2%80%933006)

*Updated 1 November 2026*