--- title: "JIA at SemEval-2026 Task 10 — paper by Zhu (2026)" description: "This paper presents a system for SemEval-2026 Task 10, which covers two subtasks on Reddit comments… 1 reference, cited by 1 paper here." authors: ["Jiayue Zhu"] year: 2026 venue: "Proceedings of the 20th International Workshop on Semantic Evaluation (2026)" source: "https://aclanthology.org/2026.semeval-1.174/" doi: "10.18653/v1/2026.semeval-1.174" licence: "CC BY 4.0 (ACL Anthology)" licence_url: "https://creativecommons.org/licenses/by/4.0/" summary_licence: "Elamur Cosmic License" canonical: "https://research.elamur.ai/free/en/papers/zhu-2026-jia-semeval-2026-task-10/" --- [Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/zhu-2026-jia-semeval-2026-task-10/) [home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/14/) / JIA at SemEval-2026 Task 10: A Dual-Track System with BERT-based Encoders and LLMs for Conspiracy Analysis # JIA at SemEval-2026 Task 10: A Dual-Track System with BERT-based Encoders and LLMs for Conspiracy Analysis **Authors:** Jiayue Zhu · **Year:** 2026 · **Venue:** [Proceedings of the 20th International Workshop on Semantic Evaluation (2026)](https://md.research.elamur.ai/free/en/venues/semeval/) ([SemEval 2026](https://md.research.elamur.ai/free/en/venues/semeval/2026/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/) **Links:** [Publisher's page](https://aclanthology.org/2026.semeval-1.174/) · [PDF](https://aclanthology.org/2026.semeval-1.174.pdf) · [DOI 10.18653/v1/2026.semeval-1.174](https://doi.org/10.18653/v1/2026.semeval-1.174) · [OpenAlex](https://openalex.org/W7166887747) **Citations:** 0 citations (OpenAlex, 2026-10-14) · [1](https://md.research.elamur.ai/free/en/papers/zhu-2026-jia-semeval-2026-task-10/cited-by/) in our library, of the 149,439 papers whose bibliographies we read ## Summary This paper presents a system for SemEval-2026 Task 10, which covers two subtasks on Reddit comments: extracting psycholinguistic conspiracy markers (Subtask 1) and binary conspiracy detection (Subtask 2). The author uses the PsyCoMark dataset, which draws on more than 190 subreddits. After removing duplicates and conflicting annotations, 3,237 training texts remained. The system runs two tracks in parallel: - **Track A:** BERT-family encoders trained with multi-task learning. Text representations are combined with learned embeddings of metadata (subreddit, source ID, annotator) for classification, and BIO tagging handles the markers. - **Track B:** Qwen2.5-Instruct models (7B and 14B) prompted to return JSON. They were tested in zero-shot, five-shot and LoRA fine-tuned settings, and the generated marker spans were mapped back to positions in the original text. For detection, the fine-tuned Qwen2.5-14B scored best (Weighted F1 0.80), ahead of the strongest encoder, DeBERTa-V3-Large (0.76). Marker extraction proved much harder: all scores were low, and they fell sharply from the development set to the test set. The work shows that instruction tuning helps LLMs on narrative-level classification, while span-level extraction of conspiracy markers remains largely unsolved. ## Key points - **Data:** PsyCoMark contains 4,316 labeled training instances, plus 100 dev and 337 test instances without labels. Cleaning removed 645 duplicates and 434 conflicting samples. The remaining texts were labeled No (1,541), Yes (1,108) or Can't tell (588), and were annotated with five marker types: Actor, Action, Effect, Evidence and Victim. - **Models compared:** DistilBERT, BERT-Base, RoBERTa-Large and DeBERTa-V3-Large on the encoder side, and Qwen2.5-7B/14B-Instruct with PEFT/LoRA adapters in the attention projection layers on the generative side. - **Detection results:** - Qwen2.5-14B fine-tuned on the full set reached Acc/W-F1 of 0.80/0.80. - The zero-shot Qwen2.5-7B reached only 0.59 W-F1. - DistilBERT dropped from 0.80 on dev to 0.72 on test, which points to weak generalization. - **Marker extraction results:** Spans were scored with a token-level IoU threshold of 0.5. The fine-tuned Qwen2.5-7B led with an aggregate F1 of 0.094 and an Actor F1 of 0.144. RoBERTa-Large was best on Victim (0.085). Most zero-shot and few-shot LLM scores stayed below 0.041. - **Data analysis and future work:** About 92% of conspiracy texts contain Actor and Action markers. The author proposes adding external knowledge bases and synthetic data augmentation to address the remaining gaps. *Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.* ## Institutions - [University of Tübingen](https://md.research.elamur.ai/free/en/universities/university-of-tubingen/) — University, Tübingen, Germany · 78 works · 41,280 citations 📚 Shelves (6): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 18,614 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 17,965 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 17,450 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 17,296 works · [🤝 Social Sciences](https://md.research.elamur.ai/free/en/shelves/social-sciences/) · 1,954 works · [🤝 Sociology and Political Science](https://md.research.elamur.ai/free/en/shelves/social-sciences/sociology-and-political-science/) · 626 works 🧭 Topics (1): [🧠 Language models & pretraining](https://md.research.elamur.ai/free/en/topics/language-models/) · 3,894 works ## Cite Cite as: Jiayue Zhu. 2026. JIA at SemEval-2026 Task 10: A Dual-Track System with BERT-based Encoders and LLMs for Conspiracy Analysis. Proceedings of the 20th International Workshop on Semantic Evaluation (2026). The paper itself, not this page. BibTeX from the library's record of the work. ```bibtex @inproceedings{zhu2026jia, title = {{JIA at SemEval-2026 Task 10: A Dual-Track System with BERT-based Encoders and LLMs for Conspiracy Analysis}}, author = {Jiayue Zhu}, booktitle = {Proceedings of the 20th International Workshop on Semantic Evaluation (2026)}, year = {2026}, doi = {10.18653/v1/2026.semeval-1.174}, url = {https://doi.org/10.18653/v1/2026.semeval-1.174} } ``` - [← previous JCT at SemEval-2026 Task 4: A Multi-Method Approach to Narrative Story Similarity](https://md.research.elamur.ai/free/en/papers/rosenfeld-2026-jct-semeval-2026-task-4/) - [next → JobMatchAI - An Intelligent Job Matching Platform Using Knowledge Graphs, Semantic Search and Explainable AI](https://md.research.elamur.ai/free/en/papers/vyas-2026-jobmatchai-intelligent-job-matching-platform/) - [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/14/) ## References 1 references, 0 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet. 1. John A Banas and Gregory Miller. 2013. Inducing resistance to conspiracy theory propaganda: Testing inoculation and metainoculation strategies. Human communication research, 39(2):184–207. *Figure 2: Performance Comparison of Marker Extraction (F1 Score).* \#\# Conclusion [find](https://scholar.google.com/scholar?q=John+A+Banas+and+Gregory+Miller%2E+2013%2E+Inducing+resistance+to+conspiracy+theory+propaganda%3A+Testing+inoculation+and+metainoculation+strategies%2E+Human+communication+research%2C+39%282%29%3A184%E2%80%93207%2E+%2AFigure+2%3A+) *Updated 31 October 2026*