---
title: "mdok-style at SemEval-2026 Task 10 — paper by Macko (2026)"
description: "This system description tackles subtask 2 of SemEval-2026 Task 10 (PsyCoMark). 14 references, cited by 1 paper here."
authors: ["Dominik Macko"]
year: 2026
venue: "Proceedings of the 20th International Workshop on Semantic Evaluation (2026)"
source: "https://aclanthology.org/2026.semeval-1.64/"
doi: "10.18653/v1/2026.semeval-1.64"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/macko-2026-mdok-style-semeval-2026-task-2026-semeval-1-64/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/macko-2026-mdok-style-semeval-2026-task-2026-semeval-1-64/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/14/) / mdok-style at SemEval-2026 Task 10: Finetuning LLMs for Conspiracy Detection

# mdok-style at SemEval-2026 Task 10: Finetuning LLMs for Conspiracy Detection

**Authors:** [Dominik Macko](https://md.research.elamur.ai/free/en/authors/dominik-macko/) · **Year:** 2026 · **Venue:** [Proceedings of the 20th International Workshop on Semantic Evaluation (2026)](https://md.research.elamur.ai/free/en/venues/semeval/) ([SemEval 2026](https://md.research.elamur.ai/free/en/venues/semeval/2026/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2026.semeval-1.64/) · [PDF](https://aclanthology.org/2026.semeval-1.64.pdf) · [DOI 10.18653/v1/2026.semeval-1.64](https://doi.org/10.18653/v1/2026.semeval-1.64) · [OpenAlex](https://openalex.org/W7166829416)

**Citations:** 0 citations (OpenAlex, 2026-10-14) · [1](https://md.research.elamur.ai/free/en/papers/macko-2026-mdok-style-semeval-2026-task-2026-semeval-1-64/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
This system description tackles subtask 2 of SemEval-2026 Task 10 (PsyCoMark). The task is to decide whether an English Reddit comment conveys belief in a conspiracy theory. The main obstacle is the modest labeled set of roughly 4,000 comments, which is small for finetuning large models. The author adapts mdok, a KInIT detector of machine-generated text that won PAN@CLEF2025, to this new problem.

Copies of training texts are altered in four ways: anonymization of emails, mentions and phone numbers, lower-casing, upper-casing, and homoglyph substitution. A 10% sample from each technique is added, followed by de-duplication. A first Qwen3-32B model, finetuned with QLoRA, then pseudo-labels the unlabeled dev and test comments. Only its most confident predictions are kept as silver labels for retraining.

The submitted variant also shifts the decision threshold to 0.7. It reached a Macro F1 of 0.78, placing 8th of 52 entries. The work shows the mdok recipe transfers beyond its original domain. However, the far smaller DeBERTa-Large scored 0.75, which raises cost–benefit concerns.

## Key points
- Training data: 1,715 conspiracy and 2,263 non-conspiracy comments of 160–1,000 characters. After augmentation, de-duplication and a 100-per-class hold-out, 1,517/2,126 remained; adding silver labels raised this to 1,881/2,575.
- Silver labels came only from predictions with positive-class probability ≥0.99 or ≤0.01.
- Finetuning setup:
  - QLoRA with 4-bit quantization in the transformers library
  - paged AdamW optimizer with a cosine schedule at 2e-5
  - batch size 1, a single epoch
  - checkpoint chosen by Macro F1
- Qwen3 and Gemma-3 models up to 32B were compared. Qwen3-32B_ST_th0.7 scored best (0.78), against DeBERTa-Large at 0.75 and a random baseline at 0.50. The task winner reached 0.89.
- The code is public (kinit-sk/mdok-style-psycomark2026). Stated limitations: English only, few base models tested, and no external training data.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Kempelen Institute of Intelligent Technologies](https://md.research.elamur.ai/free/en/ngos/kempelen-institute-of-intelligent-technologies/) — Nonprofit, Bratislava, Slovakia · 11 works · 82 citations

📚 Shelves (6): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works · [🤝 Social Sciences](https://md.research.elamur.ai/free/en/shelves/social-sciences/) · 1,772 works · [🤝 Sociology and Political Science](https://md.research.elamur.ai/free/en/shelves/social-sciences/sociology-and-political-science/) · 577 works

## Cite

Cite as: Dominik Macko. 2026. mdok-style at SemEval-2026 Task 10: Finetuning LLMs for Conspiracy Detection. Proceedings of the 20th International Workshop on Semantic Evaluation (2026).

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{macko2026mdok,
  title = {{mdok-style at SemEval-2026 Task 10: Finetuning LLMs for Conspiracy Detection}},
  author = {Dominik Macko},
  booktitle = {Proceedings of the 20th International Workshop on Semantic Evaluation (2026)},
  year = {2026},
  doi = {10.18653/v1/2026.semeval-1.64},
  url = {https://doi.org/10.18653/v1/2026.semeval-1.64}
}
```

- [← previous MCP-Guard: A Multi-Stage Defense-in-Depth Framework for Securing Model Context Protocol in Agentic AI](https://md.research.elamur.ai/free/en/papers/xing-2026-mcp-guard-multi-stage-defense/)
- [next → mdok-style at SemEval-2026 Task 9: Finetuning LLMs for Multilingual Polarization Detection](https://md.research.elamur.ai/free/en/papers/macko-2026-mdok-style-semeval-2026-task/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/14/)

## References

14 references, 3 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Massih-Reza Amini, Vasilii Feofanov, Loïc Pauletto, Liès Hadjadj, Émilie Devijver, and Yury Maximov. 2025. Self-training: A survey. Neurocomputing, 616:128904. [find](https://scholar.google.com/scholar?q=Massih%2DReza+Amini%2C+Vasilii+Feofanov%2C+Lo%C3%AFc+Pauletto%2C+Li%C3%A8s+Hadjadj%2C+%C3%89milie+Devijver%2C+and+Yury+Maximov%2E+2025%2E+Self%2Dtraining%3A+A+survey%2E+Neurocomputing%2C+616%3A128904%2E)
2. Janek Bevendorff, Yuxia Wang, Jussi Karlgren, Matti Wiegmann, Maik Fröbe, Akim Tsivgun, Jinyan Su, Zhuohan Xie, Mervat T. Abassy, Jonibek Mansurov, Rui Xing, Minh Ngoc Ta, Kareem Ashraf Elozeiri, Tianle Gu, Raj Vardhan Tomar, Jiahui Geng, Ekaterina Artemova, Artem Shelmanov, Nizar Habash, and 5 othe… [find](https://scholar.google.com/scholar?q=Janek+Bevendorff%2C+Yuxia+Wang%2C+Jussi+Karlgren%2C+Matti+Wiegmann%2C+Maik+Fr%C3%B6be%2C+Akim+Tsivgun%2C+Jinyan+Su%2C+Zhuohan+Xie%2C+Mervat+T%2E+Abassy%2C+Jonibek+Mansurov%2C+Rui+Xing%2C+Minh+Ngoc+Ta%2C+Kareem+Ashraf+Elozeiri%2C+Tian)
3. Corinna Cortes and Vladimir Vapnik. 1995. [Support-vector networks](https://md.research.elamur.ai/free/en/papers/cortes-1995-support-vector-networks/). Machine learning, 20(3):273–297.
4. Tim Dettmers, Artidoro Pagnoni, Ari Holtzman, and Luke Zettlemoyer. 2023. QLoRA: Efficient fine-tuning of quantized LLMs. In Advances in Neural Information Processing Systems, volume 36, pages 10088–10115. Curran Associates, Inc. [find](https://scholar.google.com/scholar?q=Tim+Dettmers%2C+Artidoro+Pagnoni%2C+Ari+Holtzman%2C+and+Luke+Zettlemoyer%2E+2023%2E+QLoRA%3A+Efficient+fine%2Dtuning+of+quantized+LLMs%2E+In+Advances+in+Neural+Information+Processing+Systems%2C+volume+36%2C+pages+10088%E2%80%931)
5. Anastasia Giachanou, Bilal Ghanem, and Paolo Rosso. 2023. Detection of conspiracy propagators using psycho-linguistic characteristics. Journal of Information Science, 49(1):3–17. [find](https://scholar.google.com/scholar?q=Anastasia+Giachanou%2C+Bilal+Ghanem%2C+and+Paolo+Rosso%2E+2023%2E+Detection+of+conspiracy+propagators+using+psycho%2Dlinguistic+characteristics%2E+Journal+of+Information+Science%2C+49%281%29%3A3%E2%80%9317%2E)
6. Zhiwei Liu, Boyang Liu, Paul Thompson, Kailai Yang, and Sophia Ananiadou. 2024. Conspemollm: Conspiracy theory detection using an emotion-based large language model. arXiv preprint arXiv:2403.06765. — [arXiv:2403.06765](https://arxiv.org/abs/2403.06765) [find](https://scholar.google.com/scholar?q=Zhiwei+Liu%2C+Boyang+Liu%2C+Paul+Thompson%2C+Kailai+Yang%2C+and+Sophia+Ananiadou%2E+2024%2E+Conspemollm%3A+Conspiracy+theory+detection+using+an+emotion%2Dbased+large+language+model%2E+arXiv+preprint+arXiv%3A2403%2E06765%2E+%E2%80%94)
7. Dominik Macko. 2025. mdok of KInIT: Robustly fine-tuned LLM for binary and multiclass AI-generated text detection. In Working Notes of CLEF 2025 – Conference and Labs of the Evaluation Forum, CEUR-WS.org. [find](https://scholar.google.com/scholar?q=Dominik+Macko%2E+2025%2E+mdok+of+KInIT%3A+Robustly+fine%2Dtuned+LLM+for+binary+and+multiclass+AI%2Dgenerated+text+detection%2E+In+Working+Notes+of+CLEF+2025+%E2%80%93+Conference+and+Labs+of+the+Evaluation+Forum%2C+CEUR%2DWS%2E)
8. Dominik Macko, Robert Moro, and Ivan Srba. 2025. Increasing the robustness of the fine-tuned multilingual machine-generated text detectors. Preprint, arXiv:2503.15128. — [arXiv:2503.15128](https://arxiv.org/abs/2503.15128) [find](https://scholar.google.com/scholar?q=Dominik+Macko%2C+Robert+Moro%2C+and+Ivan+Srba%2E+2025%2E+Increasing+the+robustness+of+the+fine%2Dtuned+multilingual+machine%2Dgenerated+text+detectors%2E+Preprint%2C+arXiv%3A2503%2E15128%2E+%E2%80%94+%5BarXiv%3A2503%2E15128%5D%28https%3A%2F%2Farx)
9. Dominik Macko, Robert Moro, Adaku Uchendu, Ivan Srba, Jason S Lucas, Michiharu Yamashita, Nafis Irtiza Tripto, Dongwon Lee, Jakub Simko, and Maria Bielikova. 2024. Authorship obfuscation in multilingual machine-generated text detection. In Findings of the Association for Computational Linguistics: E… [find](https://scholar.google.com/scholar?q=Dominik+Macko%2C+Robert+Moro%2C+Adaku+Uchendu%2C+Ivan+Srba%2C+Jason+S+Lucas%2C+Michiharu+Yamashita%2C+Nafis+Irtiza+Tripto%2C+Dongwon+Lee%2C+Jakub+Simko%2C+and+Maria+Bielikova%2E+2024%2E+Authorship+obfuscation+in+multilingu)
10. Mattia Samory, Felix Soldner, and Veronika Batzdorfer. 2026. [SemEval-2026 task 10: PsyCoMark – psycholinguistic conspiracy marker extraction and detection](https://md.research.elamur.ai/free/en/papers/samory-2026-semeval-2026-task-10-psycomark/). In Proceedings of the 20th International Workshop on Semantic Evaluation (SemEval-2026).
11. Michal Spiegel and Dominik Macko. 2024a. IMGTB: A framework for machine-generated text detection benchmarking. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 3: System Demonstrations), pages 172–179, Bangkok, Thailand. Association for Computational… [find](https://scholar.google.com/scholar?q=Michal+Spiegel+and+Dominik+Macko%2E+2024a%2E+IMGTB%3A+A+framework+for+machine%2Dgenerated+text+detection+benchmarking%2E+In+Proceedings+of+the+62nd+Annual+Meeting+of+the+Association+for+Computational+Linguistic)
12. Michal Spiegel and Dominik Macko. 2024b. KInIT at SemEval-2024 task 8: Fine-tuned LLMs for multilingual machine-generated text detection. In Proceedings of the 18th International Workshop on Semantic Evaluation (SemEval-2024), pages 558–564, Mexico City, Mexico. Association for Computational Linguis… [find](https://scholar.google.com/scholar?q=Michal+Spiegel+and+Dominik+Macko%2E+2024b%2E+KInIT+at+SemEval%2D2024+task+8%3A+Fine%2Dtuned+LLMs+for+multilingual+machine%2Dgenerated+text+detection%2E+In+Proceedings+of+the+18th+International+Workshop+on+Semantic+)
13. Gemma Team, Aishwarya Kamath, Johan Ferret, Shreya Pathak, Nino Vieillard, Ramona Merhej, Sarah Perrin, Tatiana Matejovicova, Alexandre Ramé, Morgane Rivière, Louis Rouillard, Thomas Mesnard, Geoffrey Cideron, Jean bastien Grill, Sabela Ramos, Edouard Yvinec, Michelle Casbon, Etienne Pot, Ivo Penche… [find](https://scholar.google.com/scholar?q=Gemma+Team%2C+Aishwarya+Kamath%2C+Johan+Ferret%2C+Shreya+Pathak%2C+Nino+Vieillard%2C+Ramona+Merhej%2C+Sarah+Perrin%2C+Tatiana+Matejovicova%2C+Alexandre+Ram%C3%A9%2C+Morgane+Rivi%C3%A8re%2C+Louis+Rouillard%2C+Thomas+Mesnard%2C+Geoffrey)
14. An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, Chujie Zheng, Dayiheng Liu, Fan Zhou, Fei Huang, Feng Hu, Hao Ge, Haoran Wei, Huan Lin, Jialong Tang, and 41 others. 2025. [Qwen3 technical report](https://md.research.elamur.ai/free/en/papers/yang-2025-qwen3-technical-report/). Preprint, arXiv:2505.09388.

*Updated 31 October 2026*