--- title: "AAA at SemEval-2026 Task 10 — paper by Pan (2026)" description: "This system paper addresses SemEval-2026 Task 10, which has two parts. 0.849 accuracy. 13 references." authors: ["Xintong Pan"] year: 2026 venue: "Proceedings of the 20th International Workshop on Semantic Evaluation (2026)" source: "https://aclanthology.org/2026.semeval-1.250/" doi: "10.18653/v1/2026.semeval-1.250" licence: "CC BY 4.0 (ACL Anthology)" licence_url: "https://creativecommons.org/licenses/by/4.0/" summary_licence: "Elamur Cosmic License" canonical: "https://research.elamur.ai/free/en/papers/pan-2026-aaa-semeval-2026-task-10/" --- [Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/pan-2026-aaa-semeval-2026-task-10/) [home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/20/) / AAA at SemEval-2026 Task 10: Psycholinguistic Conspiracy Marker Extraction and Detection # AAA at SemEval-2026 Task 10: Psycholinguistic Conspiracy Marker Extraction and Detection **Authors:** Xintong Pan · **Year:** 2026 · **Venue:** [Proceedings of the 20th International Workshop on Semantic Evaluation (2026)](https://md.research.elamur.ai/free/en/venues/semeval/) ([SemEval 2026](https://md.research.elamur.ai/free/en/venues/semeval/2026/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/) **Links:** [Publisher's page](https://aclanthology.org/2026.semeval-1.250/) · [PDF](https://aclanthology.org/2026.semeval-1.250.pdf) · [DOI 10.18653/v1/2026.semeval-1.250](https://doi.org/10.18653/v1/2026.semeval-1.250) · [OpenAlex](https://openalex.org/W7166816033) **Citations:** 0 citations (OpenAlex, 2026-10-14) ## Summary This system paper addresses SemEval-2026 Task 10, which has two parts. The first is token-level tagging of psycholinguistic conspiracy markers: Actor, Action, Effect, Evidence and Victim. The second is binary detection of whether a social media text conveys a conspiracy narrative. Conspiratorial content often relies on insinuation and implied causal links rather than plainly false claims, so the author argues that surface semantics alone may miss it. For detection, the system pairs a DeBERTa-v3 encoder with ConspEmoLLM-v2, an emotion-aware LLM. The LLM writes an explanatory rationale for each input. Vectors from the original text and from that rationale are then fused and passed to a light supervised classifier. For extraction, standard pretrained encoders were simply compared as sequence labelers, with no architectural changes. Using the Reddit-based PsyCo-Mark data, the hybrid detector reached an F1 of 0.849, compared with about 0.75 for encoder-only baselines. This suggests that LLM-generated reasoning can help classifiers pick up implicit conspiratorial framing. Extraction results differed little across encoders and serve mainly as a reproducible baseline. ## Key points - **Dataset:** PsyCo-Mark contains more than 4,100 Reddit comments with roughly 4,800 annotations, drawn from over 190 subreddits. Samples labeled "Can't tell" were removed before training. - **Encoder comparison for detection:** DistilBERT, BERT-base, RoBERTa-base and DeBERTa-v3-base were trained for 15 epochs (batch size 16, learning rate 2×10⁻⁴, Adam). Their F1 scores fell between 0.745 and 0.755, with DeBERTa highest. - **Hybrid model results:** combining DeBERTa-v3 with ConspEmoLLM-v2 rationales gave F1 0.849, accuracy 0.845, precision 0.853 and recall 0.845. - **Data augmentation:** EDA-style synonym replacement using WordNet added about 2,000 samples but did not improve test performance. - **Extraction findings:** token accuracy was high for all models, but F1 was much lower. The author attributes this gap to class imbalance, since the many unlabeled tokens inflate accuracy. The paper lists two limitations: possible bias in the LLM-generated rationales, and no multilingual testing yet. *Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.* ## Institutions - [University of Tübingen](https://md.research.elamur.ai/free/en/universities/university-of-tubingen/) — University, Tübingen, Germany · 75 works · 41,081 citations 📚 Shelves (6): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works · [🤝 Social Sciences](https://md.research.elamur.ai/free/en/shelves/social-sciences/) · 1,772 works · [🤝 Sociology and Political Science](https://md.research.elamur.ai/free/en/shelves/social-sciences/sociology-and-political-science/) · 577 works 🧭 Topics (1): [🧒 Psycholinguistics & language acquisition](https://md.research.elamur.ai/free/en/topics/psycholinguistics/) · 102 works ## Cite Cite as: Xintong Pan. 2026. AAA at SemEval-2026 Task 10: Psycholinguistic Conspiracy Marker Extraction and Detection. Proceedings of the 20th International Workshop on Semantic Evaluation (2026). The paper itself, not this page. BibTeX from the library's record of the work. ```bibtex @inproceedings{pan2026aaa, title = {{AAA at SemEval-2026 Task 10: Psycholinguistic Conspiracy Marker Extraction and Detection}}, author = {Xintong Pan}, booktitle = {Proceedings of the 20th International Workshop on Semantic Evaluation (2026)}, year = {2026}, doi = {10.18653/v1/2026.semeval-1.250}, url = {https://doi.org/10.18653/v1/2026.semeval-1.250} } ``` - [← previous A3S@C-DAC at #SMM4H-HeaRD 2026: Reasoning Meets Evidence: LLMs for Interpretable Insomnia Detection with Evidence Extraction in Clinical Notes](https://md.research.elamur.ai/free/en/papers/maity-2026-a3s-c-dac-smm4h-heard/) - [next → AAbAAC: An Annotated Corpus for Autoimmunity Information Extraction](https://md.research.elamur.ai/free/en/papers/maury-2026-aabaac-annotated-corpus-autoimmunity-information/) - [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/20/) ## References 13 references, 6 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet. 1. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. [BERT: Pre-training of deep bidirectional transformers for language understanding](https://md.research.elamur.ai/free/en/papers/devlin-2019-bert-pre-training-deep-bidirectional/). In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies,… 2. Category Model Accuracy F1 Precision Recall Action RoBERTa DeBERTa BERT DistilBERT 0.936 0.938 0.934 0.931 0.368 0.372 0.383 0.346 0.480 0.491 0.480 0.443 0.298 0.299 0.319 0.287 Actor RoBERTa DeBERTa BERT DistilBERT 0.923 0.927 0.915 0.921 0.642 0.666 0.629 0.649 0.711 0.740 0.658 0.691 0.586 0.606… [find](https://scholar.google.com/scholar?q=Category+Model+Accuracy+F1+Precision+Recall+Action+RoBERTa+DeBERTa+BERT+DistilBERT+0%2E936+0%2E938+0%2E934+0%2E931+0%2E368+0%2E372+0%2E383+0%2E346+0%2E480+0%2E491+0%2E480+0%2E443+0%2E298+0%2E299+0%2E319+0%2E287+Actor+RoBERTa+DeBERTa) 3. Pengcheng He, Xiaodong Liu, Jianfeng Gao, and Wei Chen. 2021. [Deberta: Decoding-enhanced bert with disentangled attention](https://md.research.elamur.ai/free/en/papers/he-2020-deberta-decoding-enhanced-bert-disentangled/). In 2021 International Conference on Learning Representations. 4. Takeshi Kojima, Shixiang Shane Gu, Machel Reid, Yutaka Matsuo, and Yusuke Iwasawa. 2022. [Large language models are zero-shot reasoners](https://md.research.elamur.ai/free/en/papers/kojima-2022-large-language-models-zero-shot/). In ICML 2022 Workshop on Knowledge Retrieval and Language Models. 5. Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. [Roberta: A robustly optimized bert pretraining approach](https://md.research.elamur.ai/free/en/papers/liu-2019-roberta-robustly-optimized-bert-pretraining/). ArXiv, abs/1907.11692. 6. Zhiwei Liu, Boyang Liu, Paul Thompson, Kailai Yang, and Sophia Ananiadou. 2024. ConspEmoLLM: Conspiracy Theory Detection Using an Emotion-Based Large Language Model. IOS Press. [find](https://scholar.google.com/scholar?q=Zhiwei+Liu%2C+Boyang+Liu%2C+Paul+Thompson%2C+Kailai+Yang%2C+and+Sophia+Ananiadou%2E+2024%2E+ConspEmoLLM%3A+Conspiracy+Theory+Detection+Using+an+Emotion%2DBased+Large+Language+Model%2E+IOS+Press%2E) 7. Zhiwei Liu, Paul Thompson, Jiaqi Rong, and Sophia Ananiadou. 2025. ConspEmoLLM-v2: A Robust and Stable Model to Detect Sentiment-Transformed Conspiracy Theories. IOS Press. [find](https://scholar.google.com/scholar?q=Zhiwei+Liu%2C+Paul+Thompson%2C+Jiaqi+Rong%2C+and+Sophia+Ananiadou%2E+2025%2E+ConspEmoLLM%2Dv2%3A+A+Robust+and+Stable+Model+to+Detect+Sentiment%2DTransformed+Conspiracy+Theories%2E+IOS+Press%2E) 8. Milena Pustet, Elisabeth Steffen, and Helena Mihaljevic. 2024. Detection of conspiracy theories beyond keyword bias in german-language telegram using large language models. In Proceedings of the 8th Workshop on Online Abuse and Harms (WOAH 2024), page 13–27. Association for Computational Linguistics… [find](https://scholar.google.com/scholar?q=Milena+Pustet%2C+Elisabeth+Steffen%2C+and+Helena+Mihaljevic%2E+2024%2E+Detection+of+conspiracy+theories+beyond+keyword+bias+in+german%2Dlanguage+telegram+using+large+language+models%2E+In+Proceedings+of+the+8th+W) 9. Victor Sanh, Lysandre Debut, Julien Chaumond, and Thomas Wolf. 2019. Distilbert, a distilled version of bert: smaller, faster, cheaper and lighter. In NeurIPS 2019 Workshop on Energy Efficient Machine Learning and Cognitive Computing. [find](https://scholar.google.com/scholar?q=Victor+Sanh%2C+Lysandre+Debut%2C+Julien+Chaumond%2C+and+Thomas+Wolf%2E+2019%2E+Distilbert%2C+a+distilled+version+of+bert%3A+smaller%2C+faster%2C+cheaper+and+lighter%2E+In+NeurIPS+2019+Workshop+on+Energy+Efficient+Machine) 10. Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed H. Chi, Quoc V. Le, and Denny Zhou. 2022. [Chain-of-thought prompting elicits reasoning in large language models](https://md.research.elamur.ai/free/en/papers/wei-2022-chain-thought-prompting-elicits-reasoning/). In Proceedings of the 36th International Conference on Neural Information Processing Systems, NIPS ’22, R… 11. Jason Wei and Kai Zou. 2019. [EDA: Easy data augmentation techniques for boosting performance on text classification tasks](https://md.research.elamur.ai/free/en/papers/wei-2019-eda-easy-data-augmentation-techniques/). In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP… 12. Orion Weller, Kathryn Ricci, Marc Marone, Antoine Chaffin, Dawn Lawrie, and Benjamin Van Durme. 2025. Seq vs seq: An open suite of paired encoders and decoders. arXiv:2507.11412. — [arXiv:2507.11412](https://arxiv.org/abs/2507.11412) [find](https://scholar.google.com/scholar?q=Orion+Weller%2C+Kathryn+Ricci%2C+Marc+Marone%2C+Antoine+Chaffin%2C+Dawn+Lawrie%2C+and+Benjamin+Van+Durme%2E+2025%2E+Seq+vs+seq%3A+An+open+suite+of+paired+encoders+and+decoders%2E+arXiv%3A2507%2E11412%2E+%E2%80%94+%5BarXiv%3A2507%2E11412%5D%28) 13. Zecharias Zelalem and Peter Guest. 2021. Why facebook keeps failing in ethiopia. Accessed: 2026-02-27. [find](https://scholar.google.com/scholar?q=Zecharias+Zelalem+and+Peter+Guest%2E+2021%2E+Why+facebook+keeps+failing+in+ethiopia%2E+Accessed%3A+2026%2D02%2D27%2E) *Updated 1 November 2026*