---
title: "Fine-tuning with Hierarchical Prompting for Robust Propaganda"
description: "The authors instead propose an intent-based scheme that groups techniques by the strategic goal behind a message, such as… 29 references."
authors: ["Lukas Stähelin", "Veronika Solopova", "Max Upravitelev", "David Kaplan", "Premtim Sahitaj", "Ariana Sahitaj", "Charlott Jakob", "Sebastian Möller", "Vera Schmitt"]
year: 2026
venue: "Findings of the Association for Computational Linguistics: ACL 2026"
source: "https://aclanthology.org/2026.findings-acl.139/"
doi: "10.18653/v1/2026.findings-acl.139"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/stahelin-2026-fine-tuning-hierarchical-prompting-robust/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/stahelin-2026-fine-tuning-hierarchical-prompting-robust/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/30/) / Fine-tuning with Hierarchical Prompting for Robust Propaganda Classification Across Annotation Schemas

# Fine-tuning with Hierarchical Prompting for Robust Propaganda Classification Across Annotation Schemas

**Authors:** Lukas Stähelin, [Veronika Solopova](https://md.research.elamur.ai/free/en/authors/veronika-solopova/), [Max Upravitelev](https://md.research.elamur.ai/free/en/authors/max-upravitelev/), David Kaplan, [Premtim Sahitaj](https://md.research.elamur.ai/free/en/authors/premtim-sahitaj/), [Ariana Sahitaj](https://md.research.elamur.ai/free/en/authors/ariana-sahitaj/), [Charlott Jakob](https://md.research.elamur.ai/free/en/authors/charlott-jakob/), [Sebastian Möller](https://md.research.elamur.ai/free/en/authors/sebastian-moller/) and [Vera Schmitt](https://md.research.elamur.ai/free/en/authors/vera-schmitt/) · **Year:** 2026 · **Venue:** [Findings of the Association for Computational Linguistics: ACL 2026](https://md.research.elamur.ai/free/en/venues/findings/) ([Findings of the Association for Computational Linguistics 2026](https://md.research.elamur.ai/free/en/venues/findings/2026/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2026.findings-acl.139/) · [PDF](https://aclanthology.org/2026.findings-acl.139.pdf) · [DOI 10.18653/v1/2026.findings-acl.139](https://doi.org/10.18653/v1/2026.findings-acl.139) · [OpenAlex](https://openalex.org/W7166843282)

**Citations:** 0 citations (OpenAlex, 2026-10-14)

## Summary

Detecting propaganda in short social media posts is difficult. The texts are brief and noisy, and human annotators often disagree. Existing taxonomies list rhetorical devices such as loaded language. The authors instead propose an intent-based scheme that groups techniques by the strategic goal behind a message, such as shifting blame or sowing confusion.

They built the scheme by collecting free-text intent descriptions from annotators, clustering them with GPT-4 and refining them through group discussion. They then applied it to tweets from the HQP corpus and compared it with the higher-agreement taxonomy of Sahitaj et al. (2025). For the Sahitaj taxonomy, they derived new high-level groups through clustering.

Four LLMs were tested zero-shot and after fine-tuning. Each was asked to predict high-level labels either directly or through HiPP, a hierarchical method that first predicts fine-grained techniques and then aggregates them.

Fine-tuning proved essential, and HiPP helped most on the noisier intent schema. This suggests that structured inference pays off when annotations are imperfect. The work also offers a harder, more explanatory benchmark.

## Key points

- **New taxonomy:** 17 fine-grained intent labels organized under five broader groupings, for example *Shift Blame and Justify Aggression* and *Confuse and Distract*.
  - It was developed on 300 tweets and validated on 500.
  - The authors call it the first wholly new conceptual propaganda taxonomy since 2019.
- **Inter-annotator agreement:**
  - The intent schema reached only moderate agreement (Cohen's κ 0.309, Krippendorff's α 0.308).
  - The Sahitaj et al. schema scored far higher (κ = α = 0.594).
  - The clustered high-level Sahitaj labels, built from Jaccard co-occurrence and Dawid–Skene profiles, reached α_nom 0.78.
  - These clustered labels matched the human majority vote with macro-F1 0.93 and micro-F1 0.91.
- **Models:** GPT-4.1-nano, Phi-4 14B, Qwen2.5-14B and Qwen3-14B.
  - Phi-4 14B was fine-tuned with LoRA adapters; GPT-4.1-nano via the OpenAI API.
  - Each schema used a fixed 500-tweet split, including a balanced 210-tweet training set.
- **Results:**
  - The Qwen models performed best overall.
  - Phi-4 14B consistently beat GPT-4.1-nano.
  - HiPP (Main→High) gave the clearest gains after fine-tuning on the intent schema.
- **Resources and limitations:** Data, labels and code are released on GitHub, and the best fine-tuned models on Hugging Face. However, the results come from a single small split, so the authors present them as indicating trends rather than definitive performance.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [German Research Centre for Artificial Intelligence](https://md.research.elamur.ai/free/en/labs/german-research-centre-for-artificial-intelligence/) — Research institute, Kaiserslautern, Germany · 77 works · 29,479 citations
- [Technische Universität Berlin](https://md.research.elamur.ai/free/en/universities/technische-universitat-berlin/) — University, Berlin, Germany · 22 works · 957 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🤝 Social Sciences](https://md.research.elamur.ai/free/en/shelves/social-sciences/) · 1,772 works · [🤝 Sociology and Political Science](https://md.research.elamur.ai/free/en/shelves/social-sciences/sociology-and-political-science/) · 577 works

## Cite

Cite as: Lukas Stähelin, Veronika Solopova, Max Upravitelev, David Kaplan, Premtim Sahitaj, Ariana Sahitaj, Charlott Jakob, Sebastian Möller and Vera Schmitt. 2026. Fine-tuning with Hierarchical Prompting for Robust Propaganda Classification Across Annotation Schemas. Findings of the Association for Computational Linguistics: ACL 2026.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{stahelin2026fine,
  title = {{Fine-tuning with Hierarchical Prompting for Robust Propaganda Classification Across Annotation Schemas}},
  author = {Lukas Stähelin, Veronika Solopova, Max Upravitelev, David Kaplan, Premtim Sahitaj, Ariana Sahitaj, Charlott Jakob, Sebastian Möller and Vera Schmitt},
  booktitle = {Findings of the Association for Computational Linguistics: ACL 2026},
  year = {2026},
  doi = {10.18653/v1/2026.findings-acl.139},
  url = {https://doi.org/10.18653/v1/2026.findings-acl.139}
}
```

- [← previous Fine-tuning Whisper with Spontaneous Persian Speech (SPS)](https://md.research.elamur.ai/free/en/papers/namdarzadeh-2026-fine-tuning-whisper-spontaneous-persian/)
- [next → FINEST: Improving LLM Responses to Sensitive Topics Through Fine-Grained Evaluation](https://md.research.elamur.ai/free/en/papers/oh-2026-finest-improving-llm-responses-sensitive/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/30/)

## References

29 references, 7 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Marah Abdin, Jyoti Aneja, Harkirat Behl, Sébastien Bubeck, Ronen Eldan, Suriya Gunasekar, Michael Harrison, Russell J. Hewett, Mojan Javaheripi, Piero Kauffmann, James R. Lee, Yin Tat Lee, Yuanzhi Li, Weishung Liu, Caio C. T. Mendes, Anh Nguyen, Eric Price, Gustavo de Rosa, Olli Saarikivi, and 8 oth… — [Phi-4 Technical Report](https://md.research.elamur.ai/free/en/papers/abdin-2024-phi-4-technical-report/)
2. Malak Abdullah, Ola Altiti, and Rasha Obiedat. 2022. Detecting propaganda techniques in english news articles using pre-trained transformers. In 2022 13th International Conference on Information and Communication Systems (ICICS), pages 301–308. IEEE. [find](https://scholar.google.com/scholar?q=Malak+Abdullah%2C+Ola+Altiti%2C+and+Rasha+Obiedat%2E+2022%2E+Detecting+propaganda+techniques+in+english+news+articles+using+pre%2Dtrained+transformers%2E+In+2022+13th+International+Conference+on+Information+and+C)
3. Davide Bassi, Dimitar Iliyanov Dimitrov, Bernardo D’Auria, Firoj Alam, Maram Hasanain, Christian Moro, Luisa Orrù, Gian Piero Turchi, Preslav Nakov, and Giovanni Da San Martino. 2025. Annotating the annotators: Analysis, insights and modelling from an annotation campaign on persuasion techniques det… [find](https://scholar.google.com/scholar?q=Davide+Bassi%2C+Dimitar+Iliyanov+Dimitrov%2C+Bernardo+D%E2%80%99Auria%2C+Firoj+Alam%2C+Maram+Hasanain%2C+Christian+Moro%2C+Luisa+Orr%C3%B9%2C+Gian+Piero+Turchi%2C+Preslav+Nakov%2C+and+Giovanni+Da+San+Martino%2E+2025%2E+Annotating+the+a)
4. Judit Bayer, Bernd Holznagel, Katarzyna Lubianiec, Adela Pintea, Josephine B Schmitt, Judit Szakács, and Erik Uszkiewicz. 2021. Disinformation and propaganda: impact on the functioning of the rule of law and democratic processes in the eu and its member states. European Union. [find](https://scholar.google.com/scholar?q=Judit+Bayer%2C+Bernd+Holznagel%2C+Katarzyna+Lubianiec%2C+Adela+Pintea%2C+Josephine+B+Schmitt%2C+Judit+Szak%C3%A1cs%2C+and+Erik+Uszkiewicz%2E+2021%2E+Disinformation+and+propaganda%3A+impact+on+the+functioning+of+the+rule+of+)
5. Dallas Card, Amber E. Boydstun, Justin H. Gross, Philip Resnik, and Noah A. Smith. 2015. [The media frames corpus: Annotations of frames across issues](https://md.research.elamur.ai/free/en/papers/card-2015-media-frames-corpus-annotations-frames/). In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Lang…
6. Giovanni Da San Martino, Alberto Barrón-Cedeño, Henning Wachsmuth, Rostislav Petrov, and Preslav Nakov. 2020. [SemEval-2020 task 11: Detection of propaganda techniques in news articles](https://md.research.elamur.ai/free/en/papers/martino-2020-semeval-2020-task-11-detection/). In Proceedings of the Fourteenth Workshop on Semantic Evaluation, pages 1377–1414, Barcelona (online). Internationa…
7. Giovanni Da San Martino, Seunghak Yu, Alberto Barrón-Cedeño, Rostislav Petrov, and Preslav Nakov. 2019. [Fine-grained analysis of propaganda in news articles](https://md.research.elamur.ai/free/en/papers/martino-2019-fine-grained-analysis-propaganda-news/). In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natura…
8. A. P. Dawid and A. M. Skene. 1979. Maximum likelihood estimation of observer error-rates using the em algorithm. Journal of the Royal Statistical Society. Series C (Applied Statistics), 28(1):20–28. [find](https://scholar.google.com/scholar?q=A%2E+P%2E+Dawid+and+A%2E+M%2E+Skene%2E+1979%2E+Maximum+likelihood+estimation+of+observer+error%2Drates+using+the+em+algorithm%2E+Journal+of+the+Royal+Statistical+Society%2E+Series+C+%28Applied+Statistics%29%2C+28%281%29%3A20%E2%80%9328%2E)
9. Robert M. Entman. 1993. Framing: Toward clarification of a fractured paradigm. Journal of Communication, 43(4):51–58. [find](https://scholar.google.com/scholar?q=Robert+M%2E+Entman%2E+1993%2E+Framing%3A+Toward+clarification+of+a+fractured+paradigm%2E+Journal+of+Communication%2C+43%284%29%3A51%E2%80%9358%2E)
10. Maram Hasanain, Fatema Ahmad, and Firoj Alam. 2024. Can GPT-4 identify propaganda? annotation and detection of propaganda spans in news articles. In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024), pages 2724–… [find](https://scholar.google.com/scholar?q=Maram+Hasanain%2C+Fatema+Ahmad%2C+and+Firoj+Alam%2E+2024%2E+Can+GPT%2D4+identify+propaganda%3F+annotation+and+detection+of+propaganda+spans+in+news+articles%2E+In+Proceedings+of+the+2024+Joint+International+Confere)
11. Maram Hasanain, Fatema Ahmed, and Firoj Alam. 2023. Large language models for propaganda span annotation. arXiv preprint arXiv:2311.09812. [find](https://scholar.google.com/scholar?q=Maram+Hasanain%2C+Fatema+Ahmed%2C+and+Firoj+Alam%2E+2023%2E+Large+language+models+for+propaganda+span+annotation%2E+arXiv+preprint+arXiv%3A2311%2E09812%2E)
12. Maram Hasanain, Md Arid Hasan, Mohamed Bayan Kmainasi, Elisa Sartori, Ali Ezzat Shahroor, Giovanni Da San Martino, and Firoj Alam. 2025. PropXplain: Can LLMs enable explainable propaganda detection? In Findings of the Association for Computational Linguistics: EMNLP 2025, pages 23855– 23863, Suzhou,… [find](https://scholar.google.com/scholar?q=Maram+Hasanain%2C+Md+Arid+Hasan%2C+Mohamed+Bayan+Kmainasi%2C+Elisa+Sartori%2C+Ali+Ezzat+Shahroor%2C+Giovanni+Da+San+Martino%2C+and+Firoj+Alam%2E+2025%2E+PropXplain%3A+Can+LLMs+enable+explainable+propaganda+detection%3F+I)
13. Julia Jose and Rachel Greenstadt. 2025. Are large language models good at detecting propaganda? arXiv preprint arXiv:2505.13706. — [arXiv:2505.13706](https://arxiv.org/abs/2505.13706) [find](https://scholar.google.com/scholar?q=Julia+Jose+and+Rachel+Greenstadt%2E+2025%2E+Are+large+language+models+good+at+detecting+propaganda%3F+arXiv+preprint+arXiv%3A2505%2E13706%2E+%E2%80%94+%5BarXiv%3A2505%2E13706%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2505%2E13706%29)
14. Garth S Jowett and Victoria O’donnell. 2018. Propaganda & persuasion. Sage publications. [find](https://scholar.google.com/scholar?q=Garth+S+Jowett+and+Victoria+O%E2%80%99donnell%2E+2018%2E+Propaganda+%26+persuasion%2E+Sage+publications%2E)
15. Roman Kyslyi, Nataliia Romanyshyn, and Volodymyr Sydorskyi. 2025. The unlp 2025 shared task on detecting social media manipulation. In Proceedings of the Fourth Ukrainian Natural Language Processing Workshop (UNLP 2025), pages 105–111, Online. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Roman+Kyslyi%2C+Nataliia+Romanyshyn%2C+and+Volodymyr+Sydorskyi%2E+2025%2E+The+unlp+2025+shared+task+on+detecting+social+media+manipulation%2E+In+Proceedings+of+the+Fourth+Ukrainian+Natural+Language+Processing+W)
16. Jiateng Liu, Lin Ai, Zizhou Liu, Payam Karisani, Zheng Hui, Yi Fung, Preslav Nakov, Julia Hirschberg, and Heng Ji. 2025. PropaInsight: Toward deeper understanding of propaganda in terms of techniques, appeals, and intent. In Proceedings of the 31st International Conference on Computational Linguisti… [find](https://scholar.google.com/scholar?q=Jiateng+Liu%2C+Lin+Ai%2C+Zizhou+Liu%2C+Payam+Karisani%2C+Zheng+Hui%2C+Yi+Fung%2C+Preslav+Nakov%2C+Julia+Hirschberg%2C+and+Heng+Ji%2E+2025%2E+PropaInsight%3A+Toward+deeper+understanding+of+propaganda+in+terms+of+techniques%2C)
17. Abdurahman Maarouf, Dominik Bär, Dominique Geissler, and Stefan Feuerriegel. 2024. Hqp: A human-annotated dataset for detecting online propaganda. In Findings of the Association for Computational Linguistics ACL 2024, pages 6064–6089. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Abdurahman+Maarouf%2C+Dominik+B%C3%A4r%2C+Dominique+Geissler%2C+and+Stefan+Feuerriegel%2E+2024%2E+Hqp%3A+A+human%2Dannotated+dataset+for+detecting+online+propaganda%2E+In+Findings+of+the+Association+for+Computational+Ling)
18. G Martino, Alberto Barrón-Cedeno, Henning Wachsmuth, Rostislav Petrov, and Preslav Nakov. 2020. [Semeval-2020 task 11: Detection of propaganda techniques in news articles](https://md.research.elamur.ai/free/en/papers/martino-2020-semeval-2020-task-11-detection/). arXiv preprint arXiv:2009.02696.
19. OpenAI. 2023. [Gpt-4 technical report](https://md.research.elamur.ai/free/en/papers/openai-2023-gpt-4-technical-report/). arXiv preprint arXiv:2303.08774.
20. E Perez. 2022. Strategic disinformation: Russia, ukraine and crisis communication in digital era. [find](https://scholar.google.com/scholar?q=E+Perez%2E+2022%2E+Strategic+disinformation%3A+Russia%2C+ukraine+and+crisis+communication+in+digital+era%2E)
21. Jakub Piskorski, Nicolas Stefanovitch, Giovanni Da San Martino, and Preslav Nakov. 2023. SemEval-2023 task 3: Detecting the category, the framing, and the persuasion techniques in online news in a multilingual setup. In Proceedings of the 17th International Workshop on Semantic Evaluation (SemEval-2… [find](https://scholar.google.com/scholar?q=Jakub+Piskorski%2C+Nicolas+Stefanovitch%2C+Giovanni+Da+San+Martino%2C+and+Preslav+Nakov%2E+2023%2E+SemEval%2D2023+task+3%3A+Detecting+the+category%2C+the+framing%2C+and+the+persuasion+techniques+in+online+news+in+a+mul)
22. Ariana Sahitaj, Premtim Sahitaj, Veronika Solopova, Jiaao Li, Sebastian Möller, and Vera Schmitt. 2025. Hybrid annotation for propaganda detection: Integrating LLM preannotations with human intelligence. In Proceedings of the Fourth Workshop on NLP for Positive Impact (NLP4PI), pages 215–228, Vienna… [find](https://scholar.google.com/scholar?q=Ariana+Sahitaj%2C+Premtim+Sahitaj%2C+Veronika+Solopova%2C+Jiaao+Li%2C+Sebastian+M%C3%B6ller%2C+and+Vera+Schmitt%2E+2025%2E+Hybrid+annotation+for+propaganda+detection%3A+Integrating+LLM+preannotations+with+human+intelligen)
23. Veronika Solopova, Christoph Benzmüller, and Tim Landgraf. 2023. The evolution of pro-kremlin propaganda from a machine learning and linguistics perspective. In Proceedings of the Second Ukrainian Natural Language Processing Workshop (UNLP), pages 40–48, Dubrovnik, Croatia. Association for Computati… [find](https://scholar.google.com/scholar?q=Veronika+Solopova%2C+Christoph+Benzm%C3%BCller%2C+and+Tim+Landgraf%2E+2023%2E+The+evolution+of+pro%2Dkremlin+propaganda+from+a+machine+learning+and+linguistics+perspective%2E+In+Proceedings+of+the+Second+Ukrainian+Nat)
24. Veronika Solopova, Viktoriia Herman, Christoph Benzmüller, and Tim Landgraf. 2024. Check news in one click: NLP-empowered pro-kremlin propaganda detection. In Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics: System Demonstrations, pages 44–… [find](https://scholar.google.com/scholar?q=Veronika+Solopova%2C+Viktoriia+Herman%2C+Christoph+Benzm%C3%BCller%2C+and+Tim+Landgraf%2E+2024%2E+Check+news+in+one+click%3A+NLP%2Dempowered+pro%2Dkremlin+propaganda+detection%2E+In+Proceedings+of+the+18th+Conference+of+the)
25. Veronika Solopova, Robert Nickel, and Dorothea Kolossa. 2025. Lagging Behind: Challenges of Adapting Automated Propaganda Detection to the Chronological and Geographic Evolution of Language, chapter Chapter 9. [find](https://scholar.google.com/scholar?q=Veronika+Solopova%2C+Robert+Nickel%2C+and+Dorothea+Kolossa%2E+2025%2E+Lagging+Behind%3A+Challenges+of+Adapting+Automated+Propaganda+Detection+to+the+Chronological+and+Geographic+Evolution+of+Language%2C+chapter+C)
26. Claire Wardle and Hossein Derakhshan. 2017. Information disorder: Toward an interdisciplinary framework for research and policymaking, volume 27. Council of Europe Strasbourg. [find](https://scholar.google.com/scholar?q=Claire+Wardle+and+Hossein+Derakhshan%2E+2017%2E+Information+disorder%3A+Toward+an+interdisciplinary+framework+for+research+and+policymaking%2C+volume+27%2E+Council+of+Europe+Strasbourg%2E)
27. An Yang, Yuxiao Yang, Qingyao Ai, and 1 others. 2024. Qwen2.5 technical report. Preprint, arXiv:2412.15115. Describes the Qwen2.5 model family, including Qwen2.5-14B. [find](https://scholar.google.com/scholar?q=An+Yang%2C+Yuxiao+Yang%2C+Qingyao+Ai%2C+and+1+others%2E+2024%2E+Qwen2%2E5+technical+report%2E+Preprint%2C+arXiv%3A2412%2E15115%2E+Describes+the+Qwen2%2E5+model+family%2C+including+Qwen2%2E5%2D14B%2E)
28. An Yang, Yuxiao Yang, Qingyao Ai, and 1 others. 2025. [Qwen3 technical report](https://md.research.elamur.ai/free/en/papers/yang-2025-qwen3-technical-report/). Preprint, arXiv:2505.09388. Describes the Qwen3 model family, including Qwen3-14B.
29. Mariia Zhdanova and Dariya Orlova. 2017. Computational propaganda in ukraine: Caught between external threats and internal challenges. [find](https://scholar.google.com/scholar?q=Mariia+Zhdanova+and+Dariya+Orlova%2E+2017%2E+Computational+propaganda+in+ukraine%3A+Caught+between+external+threats+and+internal+challenges%2E)

*Updated 1 November 2026*