---
title: "Team Macaroni at SemEval-2026 Task 10: PsyCoMark:"
description: "This SemEval-2026 Task 10 (PsyCoMark) submission addresses two problems on Reddit text: locating… 18 references, cited by 1 paper here."
authors: ["Rofaida Rabehi", "Nicolai Plenk", "Miriam Han"]
year: 2026
venue: "Proceedings of the 20th International Workshop on Semantic Evaluation (2026)"
source: "https://aclanthology.org/2026.semeval-1.50/"
doi: "10.18653/v1/2026.semeval-1.50"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/rabehi-2026-team-macaroni-semeval-2026-task/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/rabehi-2026-team-macaroni-semeval-2026-task/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/18/) / Team Macaroni at SemEval-2026 Task 10: PsyCoMark: Psycholinguistic Conspiracy Marker Extraction and Detection

# Team Macaroni at SemEval-2026 Task 10: PsyCoMark: Psycholinguistic Conspiracy Marker Extraction and Detection

**Authors:** Rofaida Rabehi, Nicolai Plenk and Miriam Han · **Year:** 2026 · **Venue:** [Proceedings of the 20th International Workshop on Semantic Evaluation (2026)](https://md.research.elamur.ai/free/en/venues/semeval/) ([SemEval 2026](https://md.research.elamur.ai/free/en/venues/semeval/2026/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2026.semeval-1.50/) · [PDF](https://aclanthology.org/2026.semeval-1.50.pdf) · [DOI 10.18653/v1/2026.semeval-1.50](https://doi.org/10.18653/v1/2026.semeval-1.50) · [OpenAlex](https://openalex.org/W7166829033)

**Citations:** 0 citations (OpenAlex, 2026-10-14) · [1](https://md.research.elamur.ai/free/en/papers/rabehi-2026-team-macaroni-semeval-2026-task/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
This SemEval-2026 Task 10 (PsyCoMark) submission addresses two problems on Reddit text: locating psycholinguistic conspiracy markers at the span level and deciding whether a whole comment is conspiratorial. The data are 4,361 Reddit submission statements. Each is labelled Yes, No or Can't tell for conspiracy content and annotated with five marker types: actor, action, effect, victim and evidence. For marker extraction, the team fine-tuned a range of encoders from the organisers' starter code, plus one generative model. They kept the binary token labelling scheme and varied the epochs, learning rate and batch size. For detection, they built a hybrid classifier that fuses the ModernBERT-large [CLS] embedding with eight surface and readability features. It was trained with stratified cross-validation and focal loss, and the fold models were ensembled using tuned decision thresholds. The submission placed 18th in extraction (macro F1 0.16) and 20th in detection (macro F1 0.76). The main lesson is that plain fine-tuning of a strong encoder performed as well as or better than larger models, generative ensembles and handcrafted features.

## Key points
- **Encoders tested for subtask 1:** DeBERTa-v3-base/large, RoBERTa-base/large, DistilBERT-base-cased/uncased and Flan-T5-base. Performance was roughly identical across model sizes. Flan-T5 brought no gain.
- **Per-marker test F1:** Actor was best (0.300) and Effect worst (0.072). Action (0.120) trailed Victim (0.209) despite having more training instances (3,601 vs 2,378). The authors attribute this to actions being diffuse within sentences, while actors and victims tend to be discrete noun phrases.
- **Subtask 2 system:**
  - ModernBERT-large combined with 8 textstat features (Flesch, Dale–Chall, TTR, counts and punctuation/case ratios), passed through a small MLP.
  - 5-fold stratified CV, Focal Loss (γ=2.0), AdamW at 1.5×10⁻⁵.
  - Softmax averaging across folds. A comment is labelled Yes if P(Yes) ≥ 0.35; otherwise Can't tell is down-weighted by 0.9 before comparison with No.
- **Development-phase gains:** DeBERTaV3-Large raised macro F1 from 0.678 (DeBERTa-base baseline) to 0.770. The final ModernBERT-large system reached 0.761 on test.
- **Discarded additions:** A Gemma-3-27B-IT ensemble scored only 0.387 macro F1. NLTK sentiment features and a conspiracy keyword lexicon also did not help.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [University of Tübingen](https://md.research.elamur.ai/free/en/universities/university-of-tubingen/) — University, Tübingen, Germany · 75 works · 41,081 citations

📚 Shelves (6): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works · [🤝 Social Sciences](https://md.research.elamur.ai/free/en/shelves/social-sciences/) · 1,772 works · [🤝 Sociology and Political Science](https://md.research.elamur.ai/free/en/shelves/social-sciences/sociology-and-political-science/) · 577 works

🧭 Topics (1): [🧒 Psycholinguistics & language acquisition](https://md.research.elamur.ai/free/en/topics/psycholinguistics/) · 102 works

## Cite

Cite as: Rofaida Rabehi, Nicolai Plenk and Miriam Han. 2026. Team Macaroni at SemEval-2026 Task 10: PsyCoMark: Psycholinguistic Conspiracy Marker Extraction and Detection. Proceedings of the 20th International Workshop on Semantic Evaluation (2026).

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{rabehi2026team,
  title = {{Team Macaroni at SemEval-2026 Task 10: PsyCoMark: Psycholinguistic Conspiracy Marker Extraction and Detection}},
  author = {Rofaida Rabehi, Nicolai Plenk and Miriam Han},
  booktitle = {Proceedings of the 20th International Workshop on Semantic Evaluation (2026)},
  year = {2026},
  doi = {10.18653/v1/2026.semeval-1.50},
  url = {https://doi.org/10.18653/v1/2026.semeval-1.50}
}
```

- [← previous Team JAT at SemEval-2026 Task 9: Enhancing Polarization Detection with Cross-Lingual Transfer and Feature Fusion](https://md.research.elamur.ai/free/en/papers/matkowska-2026-team-jat-semeval-2026-task/)
- [next → Team Oryu@DravidianLangTech 2026: A Multilingual Transformer Approach for Hope Speech Detection in Code-Mixed Tulu](https://md.research.elamur.ai/free/en/papers/moni-2026-team-oryu-dravidianlangtech-2026-multilingual/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/18/)

## References

18 references, 6 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Steven Bird and Edward Loper. 2004. [NLTK: The natural language toolkit](https://md.research.elamur.ai/free/en/papers/bird-2006-nltk-natural-language-toolkit/). In Proceedings of the ACL Interactive Poster and Demonstration Sessions, pages 214–217, Barcelona, Spain. Association for Computational Linguistics.
2. Hyung Won Chung, Le Hou, Shayne Longpre, Barret Zoph, Yi Tay, William Fedus, Yunxuan Li, Xuezhi Wang, Mostafa Dehghani, Siddhartha Brahma, Albert Webson, Shixiang Shane Gu, Zhuyun Dai, Mirac Suzgun, Xinyun Chen, Aakanksha Chowdhery, Alex Castro-Ros, Marie Pellat, Kevin Robinson, and 16 others. 2024.… [find](https://scholar.google.com/scholar?q=Hyung+Won+Chung%2C+Le+Hou%2C+Shayne+Longpre%2C+Barret+Zoph%2C+Yi+Tay%2C+William+Fedus%2C+Yunxuan+Li%2C+Xuezhi+Wang%2C+Mostafa+Dehghani%2C+Siddhartha+Brahma%2C+Albert+Webson%2C+Shixiang+Shane+Gu%2C+Zhuyun+Dai%2C+Mirac+Suzgun%2C+X)
3. Edgar Dale and Jeanne S. Chall. 1948. A formula for predicting readability. Educational Research Bulletin, 27:1–20, 37–54. [find](https://scholar.google.com/scholar?q=Edgar+Dale+and+Jeanne+S%2E+Chall%2E+1948%2E+A+formula+for+predicting+readability%2E+Educational+Research+Bulletin%2C+27%3A1%E2%80%9320%2C+37%E2%80%9354%2E)
4. Rudolf Flesch. 1948. A new readability yardstick. Journal of Applied Psychology, 32(3):221–233. [find](https://scholar.google.com/scholar?q=Rudolf+Flesch%2E+1948%2E+A+new+readability+yardstick%2E+Journal+of+Applied+Psychology%2C+32%283%29%3A221%E2%80%93233%2E)
5. Gemma Team. 2025. Gemma 3 technical report. Technical report, Google DeepMind. [find](https://scholar.google.com/scholar?q=Gemma+Team%2E+2025%2E+Gemma+3+technical+report%2E+Technical+report%2C+Google+DeepMind%2E)
6. Anastasia Giachanou, Bilal Ghanem, and Paolo Rosso. 2023. Detection of conspiracy propagators using psycho-linguistic characteristics. J. Inf. Sci., 49(1):3–17. [find](https://scholar.google.com/scholar?q=Anastasia+Giachanou%2C+Bilal+Ghanem%2C+and+Paolo+Rosso%2E+2023%2E+Detection+of+conspiracy+propagators+using+psycho%2Dlinguistic+characteristics%2E+J%2E+Inf%2E+Sci%2E%2C+49%281%29%3A3%E2%80%9317%2E)
7. Pengcheng He, Jianfeng Gao, and Weizhu Chen. 2021. [DeBERTaV3: Improving DeBERTa using ELECTRA-style pre-training with gradient-disentangled embedding sharing](https://md.research.elamur.ai/free/en/papers/he-2021-debertav3-improving-deberta-using-electra/). arXiv preprint arXiv:2111.09543.
8. Victor Sanh, Lysandre Debut, Julien Chaumond, and Thomas Wolf. 2019. DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter. arXiv preprint arXiv:1910.01108. [find](https://scholar.google.com/scholar?q=Victor+Sanh%2C+Lysandre+Debut%2C+Julien+Chaumond%2C+and+Thomas+Wolf%2E+2019%2E+DistilBERT%2C+a+distilled+version+of+BERT%3A+smaller%2C+faster%2C+cheaper+and+lighter%2E+arXiv+preprint+arXiv%3A1910%2E01108%2E)
9. Benjamin Warner, Antoine Chaffin, Benjamin Clavié, Orion Weller, Oskar Hallström, Said Taghadouini, Alexis Gallagher, Raja Biswas, Faisal Ladhak, Tom Aarsen, Nathan Cooper, Griffin Adams, Jeremy Howard, and Iacopo Poli. 2024. [Smarter, better, faster, longer: A modern bidirectional encoder for fast,…](https://md.research.elamur.ai/free/en/papers/warner-2024-smarter-better-faster-longer-modern/)
10. Leon Zrnić. 2024. Conspiracy theory detection using transformers with multi-task and multilingual approaches. In Working Notes of the Conference and Labs of the Evaluation Forum (CLEF 2024), volume 3740 of CEUR Workshop Proceedings, pages 3012– 3020. CEUR-WS.org. [find](https://scholar.google.com/scholar?q=Leon+Zrnic%CC%81%2E+2024%2E+Conspiracy+theory+detection+using+transformers+with+multi%2Dtask+and+multilingual+approaches%2E+In+Working+Notes+of+the+Conference+and+Labs+of+the+Evaluation+Forum+%28CLEF+2024%29%2C+volume+3)
11. Damien Lenoir. 2024. Authoritative and epistemic stance in the construction of conspiracy theories: A case study. ELAD-SILDA, (9). [find](https://scholar.google.com/scholar?q=Damien+Lenoir%2E+2024%2E+Authoritative+and+epistemic+stance+in+the+construction+of+conspiracy+theories%3A+A+case+study%2E+ELAD%2DSILDA%2C+%289%29%2E)
12. Tsung-Yi Lin, Priya Goyal, Ross Girshick, Kaiming He, and Piotr Dollár. 2017. [Focal loss for dense object detection](https://md.research.elamur.ai/free/en/papers/lin-2017-focal-loss-dense-object-detection/). In IEEE International Conference on Computer Vision (ICCV), pages 2980–2988.
13. Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. [RoBERTa: A robustly optimized BERT pretraining approach](https://md.research.elamur.ai/free/en/papers/liu-2019-roberta-robustly-optimized-bert-pretraining/). arXiv preprint arXiv:1907.11692.
14. Ilya Loshchilov and Frank Hutter. 2019. Decoupled weight decay regularization. In International Conference on Learning Representations (ICLR). [find](https://scholar.google.com/scholar?q=Ilya+Loshchilov+and+Frank+Hutter%2E+2019%2E+Decoupled+weight+decay+regularization%2E+In+International+Conference+on+Learning+Representations+%28ICLR%29%2E)
15. F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, M. Blondel, P. Prettenhofer, R. Weiss, V. Dubourg, J. Vanderplas, A. Passos, D. Cournapeau, M. Brucher, M. Perrot, and E. Duchesnay. 2011. Scikit-learn: Machine learning in Python. Journal of Machine Learning Research, 12:282… [find](https://scholar.google.com/scholar?q=F%2E+Pedregosa%2C+G%2E+Varoquaux%2C+A%2E+Gramfort%2C+V%2E+Michel%2C+B%2E+Thirion%2C+O%2E+Grisel%2C+M%2E+Blondel%2C+P%2E+Prettenhofer%2C+R%2E+Weiss%2C+V%2E+Dubourg%2C+J%2E+Vanderplas%2C+A%2E+Passos%2C+D%2E+Cournapeau%2C+M%2E+Brucher%2C+M%2E+Perrot%2C+and+E%2E+Duc)
16. Mattia Samory. 2025. SemEval-2026 Task 10 Starter Pack. [https://github.com/hide-ous/](https://github.com/hide-ous/) semeval26_task10_starter_pack. Scripts to facilitate participation in SemEval-2026 Task 10: Psy-CoMark. [↗](https://github.com/hide-ous/)
17. Mattia Samory, Felix Soldner, and Veronika Batzdorfer. 2025. PsyCoMark – Psycholinguistic Conspiracy Marker Dataset. [find](https://scholar.google.com/scholar?q=Mattia+Samory%2C+Felix+Soldner%2C+and+Veronika+Batzdorfer%2E+2025%2E+PsyCoMark+%E2%80%93+Psycholinguistic+Conspiracy+Marker+Dataset%2E)
18. Mattia Samory, Felix Soldner, and Veronika Batzdorfer. 2026. [SemEval-2026 task 10: PsyCoMark – psycholinguistic conspiracy marker extraction and detection](https://md.research.elamur.ai/free/en/papers/samory-2026-semeval-2026-task-10-psycomark/). In Proceedings of the 20th International Workshop on Semantic Evaluation (SemEval-2026).

*Updated 31 October 2026*