--- title: "TTLab at SemEval-2026 Task 10 — paper by Richter et al. (2026)" description: "Conspiracy content on social media is hard to detect from vocabulary alone. 0.835 F1. 9 references." authors: ["Samuel Richter", "Mounika Marreddy", "Alexander Mehler"] year: 2026 venue: "Proceedings of the 20th International Workshop on Semantic Evaluation (2026)" source: "https://aclanthology.org/2026.semeval-1.103/" doi: "10.18653/v1/2026.semeval-1.103" licence: "CC BY 4.0 (ACL Anthology)" licence_url: "https://creativecommons.org/licenses/by/4.0/" summary_licence: "Elamur Cosmic License" canonical: "https://research.elamur.ai/free/en/papers/richter-2026-ttlab-semeval-2026-task-10/" --- [Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/richter-2026-ttlab-semeval-2026-task-10/) [home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/46/) / TTLab at SemEval-2026 Task 10: Transformer-based Approaches for Psycholinguistic Conspiracy Detection in Social Media Discourse # TTLab at SemEval-2026 Task 10: Transformer-based Approaches for Psycholinguistic Conspiracy Detection in Social Media Discourse **Authors:** Samuel Richter, [Mounika Marreddy](https://md.research.elamur.ai/free/en/authors/mounika-marreddy/) and [Alexander Mehler](https://md.research.elamur.ai/free/en/authors/alexander-mehler/) · **Year:** 2026 · **Venue:** [Proceedings of the 20th International Workshop on Semantic Evaluation (2026)](https://md.research.elamur.ai/free/en/venues/semeval/) ([SemEval 2026](https://md.research.elamur.ai/free/en/venues/semeval/2026/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/) **Links:** [Publisher's page](https://aclanthology.org/2026.semeval-1.103/) · [PDF](https://aclanthology.org/2026.semeval-1.103.pdf) · [DOI 10.18653/v1/2026.semeval-1.103](https://doi.org/10.18653/v1/2026.semeval-1.103) · [OpenAlex](https://openalex.org/W7166886127) **Citations:** 0 citations (OpenAlex, 2026-10-14) ## Summary Conspiracy content on social media is hard to detect from vocabulary alone. It relies on psychological framing, such as blaming powerful agents or hinting at hidden motives. The TTLab team addressed this in SemEval-2026 Task 10 (PsyCoMark). The task's dataset contains Reddit submissions with document-level conspiracy labels and span annotations for five mechanisms: Actor, Action, Effect, Victim and Evidence. The authors dropped "can't tell" items to make the task binary. They fine-tuned BERT, DistilBERT, RoBERTa, XLNet and ELECTRA on three inputs: the raw post, a labeled list of only the marker spans, and both combined. They also stacked the five models under a logistic regression meta-learner. Marker inputs clearly beat raw text, raising the best macro F1 from 0.759 to 0.817 (both BERT). Adding text to markers mostly helped weaker models, and the ensemble scored 0.842. The test set had no gold markers, so the marker results are oracle upper bounds, and the leaderboard run used text only. The study shows narrative structure is a strong signal and motivates predicting markers automatically. ## Key points - PsyCoMark data: over 4,800 annotated Reddit submissions from more than 190 subreddits, labeled 1,715 conspiracy, 2,263 not conspiracy and 877 can't tell. Annotator agreement was moderate (Krippendorff's alpha 0.58). - Text-only macro F1 ranged from 0.727 (XLNet) to 0.759 (BERT). With markers only, BERT reached 0.817. - Combining text and markers lifted RoBERTa from 0.761 to 0.805 and XLNet from 0.782 to 0.806, while BERT stayed flat (0.816). - The stacking ensemble scored best (P 0.848, R 0.835, F1 0.842). Base models were trained on 80% of the data, and the meta-learner used the remaining 20%. - In error analysis, text models over-relied on trigger words like "vaccine" or "UFO". Marker models labeled any Actor–Action–Victim pattern as conspiracy. Activist political language was often misclassified as conspiratorial. *Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.* ## Institutions - [Goethe University Frankfurt](https://md.research.elamur.ai/free/en/universities/goethe-university-frankfurt/) — University, Frankfurt am Main, Germany · 18 works · 373 citations 📚 Shelves (6): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works · [🤝 Social Sciences](https://md.research.elamur.ai/free/en/shelves/social-sciences/) · 1,772 works · [🤝 Sociology and Political Science](https://md.research.elamur.ai/free/en/shelves/social-sciences/sociology-and-political-science/) · 577 works 🧭 Topics (1): [🧒 Psycholinguistics & language acquisition](https://md.research.elamur.ai/free/en/topics/psycholinguistics/) · 102 works ## Cite Cite as: Samuel Richter, Mounika Marreddy and Alexander Mehler. 2026. TTLab at SemEval-2026 Task 10: Transformer-based Approaches for Psycholinguistic Conspiracy Detection in Social Media Discourse. Proceedings of the 20th International Workshop on Semantic Evaluation (2026). The paper itself, not this page. BibTeX from the library's record of the work. ```bibtex @inproceedings{richter2026ttlab, title = {{TTLab at SemEval-2026 Task 10: Transformer-based Approaches for Psycholinguistic Conspiracy Detection in Social Media Discourse}}, author = {Samuel Richter, Mounika Marreddy and Alexander Mehler}, booktitle = {Proceedings of the 20th International Workshop on Semantic Evaluation (2026)}, year = {2026}, doi = {10.18653/v1/2026.semeval-1.103}, url = {https://doi.org/10.18653/v1/2026.semeval-1.103} } ``` - [← previous TTLab at Daleel 2026: STAR-Ar, Sequence Tagging for Argument Recognition in Arabic](https://md.research.elamur.ai/free/en/papers/verma-2026-ttlab-daleel-2026-star-ar/) - [next → TTLab at StanceEval-2026: A Cloze-Style Prompting Approach for Arabic-Language Stance Detection (CLASP-Ar)](https://md.research.elamur.ai/free/en/papers/verma-2026-ttlab-stanceeval-2026-cloze-style/) - [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/46/) ## References 9 references, 3 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet. 1. Kevin Clark, Minh-Thang Luong, Quoc V Le, and Christopher D Manning. 2020. Electra: Pre-training text encoders as discriminators rather than generators. arXiv preprint arXiv:2003.10555. [find](https://scholar.google.com/scholar?q=Kevin+Clark%2C+Minh%2DThang+Luong%2C+Quoc+V+Le%2C+and+Christopher+D+Manning%2E+2020%2E+Electra%3A+Pre%2Dtraining+text+encoders+as+discriminators+rather+than+generators%2E+arXiv+preprint+arXiv%3A2003%2E10555%2E) 2. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. [Bert: Pre-training of deep bidirectional transformers for language understanding](https://md.research.elamur.ai/free/en/papers/devlin-2019-bert-pre-training-deep-bidirectional/). In Proceedings of the 2019 conference of the North American chapter of the association for computational linguistics: human language technologies,… 3. Anastasia Giachanou, Bilal Ghanem, and Paolo Rosso. 2023. Detection of conspiracy propagators using psycho-linguistic characteristics. Journal of Information Science, 49:3–17. [find](https://scholar.google.com/scholar?q=Anastasia+Giachanou%2C+Bilal+Ghanem%2C+and+Paolo+Rosso%2E+2023%2E+Detection+of+conspiracy+propagators+using+psycho%2Dlinguistic+characteristics%2E+Journal+of+Information+Science%2C+49%3A3%E2%80%9317%2E) 4. Anastasia Giachanou, Bilal Ghanem, Esteban A. Ríssola, Paolo Rosso, Fabio Crestani, and Daniel Oberski. 2022. The impact of psycholinguistic patterns in discriminating between fake news spreaders and fact checkers. Data Knowledge Engineering, 138:101960. [find](https://scholar.google.com/scholar?q=Anastasia+Giachanou%2C+Bilal+Ghanem%2C+Esteban+A%2E+R%C3%ADssola%2C+Paolo+Rosso%2C+Fabio+Crestani%2C+and+Daniel+Oberski%2E+2022%2E+The+impact+of+psycholinguistic+patterns+in+discriminating+between+fake+news+spreaders+and+) 5. Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. [Roberta: A robustly optimized bert pretraining approach](https://md.research.elamur.ai/free/en/papers/liu-2019-roberta-robustly-optimized-bert-pretraining/). arXiv preprint arXiv:1907.11692. 6. Mattia Samory, Felix Soldner, and Veronika Batzdorfer. 2026. SemEval-2026 task 10: Psy-CoMark – psycholinguistic conspiracy marker extraction and detection. In Proceedings of the 20th International Workshop on Semantic Evaluation (SemEval-2026). [find](https://scholar.google.com/scholar?q=Mattia+Samory%2C+Felix+Soldner%2C+and+Veronika+Batzdorfer%2E+2026%2E+SemEval%2D2026+task+10%3A+Psy%2DCoMark+%E2%80%93+psycholinguistic+conspiracy+marker+extraction+and+detection%2E+In+Proceedings+of+the+20th+International+Wo) 7. Victor Sanh, Lysandre Debut, Julien Chaumond, and Thomas Wolf. 2019. Distilbert, a distilled version of bert: smaller, faster, cheaper and lighter. arXiv preprint arXiv:1910.01108. [find](https://scholar.google.com/scholar?q=Victor+Sanh%2C+Lysandre+Debut%2C+Julien+Chaumond%2C+and+Thomas+Wolf%2E+2019%2E+Distilbert%2C+a+distilled+version+of+bert%3A+smaller%2C+faster%2C+cheaper+and+lighter%2E+arXiv+preprint+arXiv%3A1910%2E01108%2E) 8. Marco Valentino, Leonardo Ranaldi, Giulia Pucci, Federico Ranaldi, and André Freitas. 2026. [Semeval-2026 task 11: Disentangling content and formal reasoning in large language models](https://md.research.elamur.ai/free/en/papers/valentino-2026-semeval-2026-task-11-disentangling/). In Proceedings of the 20th International Workshop on Semantic Evaluation (SemEval-2026). 9. Zhilin Yang, Zihang Dai, Yiming Yang, Jaime Carbonell, Russ R Salakhutdinov, and Quoc V Le. 2019. Xlnet: Generalized autoregressive pre-training for language understanding. Advances in neural information processing systems, 32. [find](https://scholar.google.com/scholar?q=Zhilin+Yang%2C+Zihang+Dai%2C+Yiming+Yang%2C+Jaime+Carbonell%2C+Russ+R+Salakhutdinov%2C+and+Quoc+V+Le%2E+2019%2E+Xlnet%3A+Generalized+autoregressive+pre%2Dtraining+for+language+understanding%2E+Advances+in+neural+informat) *Updated 1 November 2026*