---
title: "Can Large Language Models Facilitate Qualitative Political"
description: "They propose a Human-LLM Co-Analysis approach for applying qualitative analysis to larger corpora. 25 references."
authors: ["Luke Stephens", "Clare Llewellyn", "Lauren Rogers", "Constantine Kyritsopoulos", "Arman Prangere", "Feiteng Long", "Peyton Snyder", "Laura Cram"]
year: 2026
venue: "Proceedings of the 3rd Workshop on Natural Language Processing for Political Sciences (PoliticalNLP 2026)"
source: "https://aclanthology.org/2026.politicalnlp-1.28/"
doi: "10.63317/4rwv2gsphhck"
licence: "CC BY-NC 4.0 (ELRA, LREC proceedings)"
licence_url: "https://creativecommons.org/licenses/by-nc/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/stephens-2026-can-large-language-models-facilitate/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/stephens-2026-can-large-language-models-facilitate/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/24/) / Can Large Language Models Facilitate Qualitative Political Narrative Analysis?

# Can Large Language Models Facilitate Qualitative Political Narrative Analysis?

**Authors:** Luke Stephens, Clare Llewellyn, Lauren Rogers, Constantine Kyritsopoulos, Arman Prangere, Feiteng Long, Peyton Snyder and Laura Cram · **Year:** 2026 · **Venue:** [Proceedings of the 3rd Workshop on Natural Language Processing for Political Sciences (PoliticalNLP 2026)](https://md.research.elamur.ai/free/en/venues/politicalnlp/) ([Workshop on Natural Language Processing for Political Sciences 2026](https://md.research.elamur.ai/free/en/venues/politicalnlp/2026/)) · **Publisher:** [European Language Resources Association (ELRA)](https://md.research.elamur.ai/free/en/publishers/european-language-resources-association-elra/) · **Licence:** [CC BY-NC 4.0 (ELRA, LREC proceedings)](https://creativecommons.org/licenses/by-nc/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2026.politicalnlp-1.28/) · [PDF](https://aclanthology.org/2026.politicalnlp-1.28.pdf) · [DOI 10.63317/4rwv2gsphhck](https://doi.org/10.63317/4rwv2gsphhck) · [OpenAlex](https://openalex.org/W7166547843)

**Citations:** 0 citations (OpenAlex, 2026-10-14)

## Summary
Political narrative research is mostly interpretive and labour-intensive. Earlier NLP work has largely pulled out isolated narrative components such as events, characters and entities, instead of treating narratives as coherent stories. This paper asks whether LLMs can serve as assistants in interpretive narrative analysis. The authors study European Union State of the Union (SOTEU) speeches from 2010–2023, focusing on two themes: migration and solidarity. Two expert narrative scholars independently wrote analyses of each speech and of the whole corpus. Mistral, Llama, ChatGPT-4o and DeepSeek received equivalent prompts that cast them as narrative scholars. The outputs were compared using lexical and semantic similarity metrics and checked through structured human validation of structural, factual and narrative aspects. The two scholars agreed closely in meaning while differing in wording. Model–human similarity was broadly on par with scholar–scholar similarity, often differing only marginally. However, validation revealed weaknesses that the metrics missed. The authors therefore argue that similarity scores alone cannot judge interpretive quality. They propose a Human-LLM Co-Analysis approach for applying qualitative analysis to larger corpora.

## Key points
- The corpus consists of English-language SOTEU addresses by three Commission Presidents: Barroso, Juncker and von der Leyen. The text describes 13 speeches of about 75,000 words, but Table 1 lists 12 addresses totalling 76,046 words.
- The task design had each scholar and model write a 300-word analysis per speech for each topic. Each also produced one 600-word analysis of how the topics evolve across the full corpus.
- Four models were benchmarked: Mistral, Llama, ChatGPT-4o and DeepSeek. Agreement between the two scholars served as the reference for interpretive consistency.
- Human validation found model-specific failures, including factual errors, overly structural or abstract readings, and weak treatment of less prominent narrative threads.
- The authors acknowledge a limitation: the human benchmark reflects a single interpretive community, and scholars from other traditions or national contexts might read the speeches differently.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [University of Edinburgh](https://md.research.elamur.ai/free/en/universities/university-of-edinburgh/) — University, Edinburgh, United Kingdom · 359 works · 77,352 citations

📚 Shelves (7): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works · [🤝 Social Sciences](https://md.research.elamur.ai/free/en/shelves/social-sciences/) · 1,772 works · [🤝 General Social Sciences](https://md.research.elamur.ai/free/en/shelves/social-sciences/general-social-sciences/) · 227 works · [🎓 Education](https://md.research.elamur.ai/free/en/shelves/social-sciences/education/) · 203 works

🧭 Topics (1): [🧠 Language models & pretraining](https://md.research.elamur.ai/free/en/topics/language-models/) · 3,320 works

## Cite

Cite as: Luke Stephens, Clare Llewellyn, Lauren Rogers, Constantine Kyritsopoulos, Arman Prangere, Feiteng Long, Peyton Snyder and Laura Cram. 2026. Can Large Language Models Facilitate Qualitative Political Narrative Analysis?. Proceedings of the 3rd Workshop on Natural Language Processing for Political Sciences (PoliticalNLP 2026).

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{stephens2026can,
  title = {{Can Large Language Models Facilitate Qualitative Political Narrative Analysis?}},
  author = {Luke Stephens, Clare Llewellyn, Lauren Rogers, Constantine Kyritsopoulos, Arman Prangere, Feiteng Long, Peyton Snyder and Laura Cram},
  booktitle = {Proceedings of the 3rd Workshop on Natural Language Processing for Political Sciences (PoliticalNLP 2026)},
  year = {2026},
  doi = {10.63317/4rwv2gsphhck},
  url = {https://doi.org/10.63317/4rwv2gsphhck}
}
```

- [← previous Can I guess where you are from? Modeling dialectal morphosyntactic similarities in Brazilian Portuguese](https://md.research.elamur.ai/free/en/papers/siqueira-2026-can-i-guess-where-you/)
- [next → Can Large Language Models Infer Human Actions and Motives? Evaluation in Social Prediction and Inspection Games](https://md.research.elamur.ai/free/en/papers/shrestha-2026-can-large-language-models-infer/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/24/)

## References

25 references, 1 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Muneera Bano, Didar Zowghi, and Jon Whittle. 2023. Exploring qualitative research using llms. [find](https://scholar.google.com/scholar?q=Muneera+Bano%2C+Didar+Zowghi%2C+and+Jon+Whittle%2E+2023%2E+Exploring+qualitative+research+using+llms%2E)
2. Kenneth Benoit, Scott De Marchi, Conor Laver, Michael Laver, and Jinshuai Ma. 2025. Using large language models to analyze political texts through natural language understanding. American Journal of Political Science. [find](https://scholar.google.com/scholar?q=Kenneth+Benoit%2C+Scott+De+Marchi%2C+Conor+Laver%2C+Michael+Laver%2C+and+Jinshuai+Ma%2E+2025%2E+Using+large+language+models+to+analyze+political+texts+through+natural+language+understanding%2E+American+Journal+of+P)
3. Natalia Chaban, Alister Miskimmon, and Ben O’Loughlin. 2019. Understanding eu crisis diplomacy in the european neighbourhood: strategic narratives and perceptions of the eu in ukraine, israel and palestine. European Security, 28(3):235–250. [find](https://scholar.google.com/scholar?q=Natalia+Chaban%2C+Alister+Miskimmon%2C+and+Ben+O%E2%80%99Loughlin%2E+2019%2E+Understanding+eu+crisis+diplomacy+in+the+european+neighbourhood%3A+strategic+narratives+and+perceptions+of+the+eu+in+ukraine%2C+israel+and+pale)
4. Thomas Colley and Carolijn {van Noort}. 2022. Strategic Narratives, Ontological Security and Global Policy: Responses to China’s Belt and Road Initiative. Palgrave Studies in International Relations. Palgrave Macmillan Ltd., United Kingdom. [find](https://scholar.google.com/scholar?q=Thomas+Colley+and+Carolijn+%7Bvan+Noort%7D%2E+2022%2E+Strategic+Narratives%2C+Ontological+Security+and+Global+Policy%3A+Responses+to+China%E2%80%99s+Belt+and+Road+Initiative%2E+Palgrave+Studies+in+International+Relations%2E+)
5. Elahe Rahimtoroghi, Jiaqi Wu, Ruimin Wang, Pranav Anand, and Marilyn Walker. 2017. Modelling protagonist goals and desires in first-person narrative. In Proceedings of the 18th Annual SIGdial Meeting on Discourse and Dialogue, pages 360–369. [find](https://scholar.google.com/scholar?q=Elahe+Rahimtoroghi%2C+Jiaqi+Wu%2C+Ruimin+Wang%2C+Pranav+Anand%2C+and+Marilyn+Walker%2E+2017%2E+Modelling+protagonist+goals+and+desires+in+first%2Dperson+narrative%2E+In+Proceedings+of+the+18th+Annual+SIGdial+Meeting+)
6. Niels Dekker, Tobias Kuhn, and Marieke van Erp. 2019. Evaluating named entity recognition tools for extracting social networks from novels. PeerJ Computer Science, 5:e189. [find](https://scholar.google.com/scholar?q=Niels+Dekker%2C+Tobias+Kuhn%2C+and+Marieke+van+Erp%2E+2019%2E+Evaluating+named+entity+recognition+tools+for+extracting+social+networks+from+novels%2E+PeerJ+Computer+Science%2C+5%3Ae189%2E)
7. Georg Rehm, Julian Moreno Schneider, Peter Bourgonje, Ankit Srivastava, Jan Nehring, Armin Berger, Luca König, Sören Räuchle, and Jens Gerth. 2017. Event detection and semantic storytelling: Generating a travelogue from a large collection of personal letters. In Proceedings of the Events and Stories… [find](https://scholar.google.com/scholar?q=Georg+Rehm%2C+Julian+Moreno+Schneider%2C+Peter+Bourgonje%2C+Ankit+Srivastava%2C+Jan+Nehring%2C+Armin+Berger%2C+Luca+K%C3%B6nig%2C+S%C3%B6ren+R%C3%A4uchle%2C+and+Jens+Gerth%2E+2017%2E+Event+detection+and+semantic+storytelling%3A+Generatin)
8. Zackary Okun Dunivin. 2025. Scaling hermeneutics: a guide to qualitative coding with llms for reflexive content analysis. EPJ Data Science, 14(1):28. [find](https://scholar.google.com/scholar?q=Zackary+Okun+Dunivin%2E+2025%2E+Scaling+hermeneutics%3A+a+guide+to+qualitative+coding+with+llms+for+reflexive+content+analysis%2E+EPJ+Data+Science%2C+14%281%29%3A28%2E)
9. Linus Hagström and Karl Gustafsson. 2019. Narrative power: how storytelling shapes east asian international politics. Cambridge Review of International Affairs, 32(4):387–406. [find](https://scholar.google.com/scholar?q=Linus+Hagstr%C3%B6m+and+Karl+Gustafsson%2E+2019%2E+Narrative+power%3A+how+storytelling+shapes+east+asian+international+politics%2E+Cambridge+Review+of+International+Affairs%2C+32%284%29%3A387%E2%80%93406%2E)
10. Alexandra Homolar and Oliver Turner. 2024. Narrative alliances: the discursive foundations of international order. International Affairs, 100(1):203– 220. [find](https://scholar.google.com/scholar?q=Alexandra+Homolar+and+Oliver+Turner%2E+2024%2E+Narrative+alliances%3A+the+discursive+foundations+of+international+order%2E+International+Affairs%2C+100%281%29%3A203%E2%80%93+220%2E)
11. Zhichao Hu and Marilyn Walker. 2017. Inferring narrative causality between event pairs in films. In Proceedings of the 18th annual SIGdial meeting on discourse and dialogue, pages 342–351. [find](https://scholar.google.com/scholar?q=Zhichao+Hu+and+Marilyn+Walker%2E+2017%2E+Inferring+narrative+causality+between+event+pairs+in+films%2E+In+Proceedings+of+the+18th+annual+SIGdial+meeting+on+discourse+and+dialogue%2C+pages+342%E2%80%93351%2E)
12. Muhammad Nihal Hussain, Kiran Kumar Bandeli, Samer Al-khateeb, and Nitin Agarwal. 2018. Analyzing shift in narratives regarding migrants in europe via blogosphere. Text2Story@ ECIR, 585:33–40. [find](https://scholar.google.com/scholar?q=Muhammad+Nihal+Hussain%2C+Kiran+Kumar+Bandeli%2C+Samer+Al%2Dkhateeb%2C+and+Nitin+Agarwal%2E+2018%2E+Analyzing+shift+in+narratives+regarding+migrants+in+europe+via+blogosphere%2E+Text2Story%40+ECIR%2C+585%3A33%E2%80%9340%2E)
13. Sarah Jenner, Dimitris Raidos, Emma Anderson, Stella Fleetwood, Ben Ainsworth, Kerry Fox, Jana Kreppner, and Mary Barker. 2025. Using large language models for narrative analysis: A novel application of generative ai. Methods in Psychology, 12:100183. [find](https://scholar.google.com/scholar?q=Sarah+Jenner%2C+Dimitris+Raidos%2C+Emma+Anderson%2C+Stella+Fleetwood%2C+Ben+Ainsworth%2C+Kerry+Fox%2C+Jana+Kreppner%2C+and+Mary+Barker%2E+2025%2E+Using+large+language+models+for+narrative+analysis%3A+A+novel+application+)
14. Alessandro Maisto. 2025. Collaborative storytelling and llm: A linguistic analysis of automatically-generated role-playing game sessions. arXiv preprint arXiv:2503.20623. — [arXiv:2503.20623](https://arxiv.org/abs/2503.20623) [find](https://scholar.google.com/scholar?q=Alessandro+Maisto%2E+2025%2E+Collaborative+storytelling+and+llm%3A+A+linguistic+analysis+of+automatically%2Dgenerated+role%2Dplaying+game+sessions%2E+arXiv+preprint+arXiv%3A2503%2E20623%2E+%E2%80%94+%5BarXiv%3A2503%2E20623%5D%28https%3A%2F%2F)
15. Sebastian Michelmann, Manoj Kumar, Kenneth A. Norman, and Mariya Toneva. 2023. Large language models can segment narrative events similarly to humans. [find](https://scholar.google.com/scholar?q=Sebastian+Michelmann%2C+Manoj+Kumar%2C+Kenneth+A%2E+Norman%2C+and+Mariya+Toneva%2E+2023%2E+Large+language+models+can+segment+narrative+events+similarly+to+humans%2E)
16. Andrew Piper and Sunyam Bagga. 2025. NarraDe-tect: An annotated dataset for the task of narrative detection. In Proceedings of the The 7th Workshop on Narrative Understanding, pages 1–7, Albuquerque, New Mexico. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Andrew+Piper+and+Sunyam+Bagga%2E+2025%2E+NarraDe%2Dtect%3A+An+annotated+dataset+for+the+task+of+narrative+detection%2E+In+Proceedings+of+the+The+7th+Workshop+on+Narrative+Understanding%2C+pages+1%E2%80%937%2C+Albuquerque%2C+)
17. Paul Ricoeur. 1984. Time and narrative, Volume 3, volume 3. University of Chicago press. [find](https://scholar.google.com/scholar?q=Paul+Ricoeur%2E+1984%2E+Time+and+narrative%2C+Volume+3%2C+volume+3%2E+University+of+Chicago+press%2E)
18. Benjamin M. Schmidt. 2015. Plot arceology: A vector-space model of narrative structure. In 2015 IEEE International Conference on Big Data (Big Data), pages 1667–1672. [find](https://scholar.google.com/scholar?q=Benjamin+M%2E+Schmidt%2E+2015%2E+Plot+arceology%3A+A+vector%2Dspace+model+of+narrative+structure%2E+In+2015+IEEE+International+Conference+on+Big+Data+%28Big+Data%29%2C+pages+1667%E2%80%931672%2E)
19. Margaret R. Somers. 1994. The narrative constitution of identity: A relational and network approach. Theory and Society, 23(5):605–649. [find](https://scholar.google.com/scholar?q=Margaret+R%2E+Somers%2E+1994%2E+The+narrative+constitution+of+identity%3A+A+relational+and+network+approach%2E+Theory+and+Society%2C+23%285%29%3A605%E2%80%93649%2E)
20. Evangelia Spiliopoulou, Eduard Hovy, and Teruko Mitamura. 2017. [Event detection using frame-semantic parser](https://md.research.elamur.ai/free/en/papers/spiliopoulou-2017-event-detection-frame-semantic-parser/). In Proceedings of the Events and Stories in the News Workshop, pages 15–20, Vancouver, Canada. Association for Computational Linguistics.
21. Dominik Stammbach, Maria Antoniak, and Elliott Ash. 2022. Heroes, villains, and victims, and GPT-3: Automated extraction of character roles without training data. In Proceedings of the 4th Workshop of Narrative Understanding (WNU2022), pages 47–56, Seattle, United States. Association for Computation… [find](https://scholar.google.com/scholar?q=Dominik+Stammbach%2C+Maria+Antoniak%2C+and+Elliott+Ash%2E+2022%2E+Heroes%2C+villains%2C+and+victims%2C+and+GPT%2D3%3A+Automated+extraction+of+character+roles+without+training+data%2E+In+Proceedings+of+the+4th+Workshop+of)
22. Brent J Steele. 2008. Ontological security in international relations: Self-identity and the IR state. Routledge. [find](https://scholar.google.com/scholar?q=Brent+J+Steele%2E+2008%2E+Ontological+security+in+international+relations%3A+Self%2Didentity+and+the+IR+state%2E+Routledge%2E)
23. Yidan Sun, Qin Chao, and Boyang Li. 2024. Event causality is key to computational story understanding. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), pages 3493–3511, Mexico Ci… [find](https://scholar.google.com/scholar?q=Yidan+Sun%2C+Qin+Chao%2C+and+Boyang+Li%2E+2024%2E+Event+causality+is+key+to+computational+story+understanding%2E+In+Proceedings+of+the+2024+Conference+of+the+North+American+Chapter+of+the+Association+for+Comput)
24. Nicholas Sloss Treynor and Joshua McCoy. 2025. A case study on user perception of parameterized llm-generated narratives. In 2025 IEEE Conference on Games (CoG), pages 1–7. IEEE. [find](https://scholar.google.com/scholar?q=Nicholas+Sloss+Treynor+and+Joshua+McCoy%2E+2025%2E+A+case+study+on+user+perception+of+parameterized+llm%2Dgenerated+narratives%2E+In+2025+IEEE+Conference+on+Games+%28CoG%29%2C+pages+1%E2%80%937%2E+IEEE%2E)
25. H. White. 1987. The Content of the Form: Narrative Discourse and Historical Representation. Johns Hopkins University Press. [find](https://scholar.google.com/scholar?q=H%2E+White%2E+1987%2E+The+Content+of+the+Form%3A+Narrative+Discourse+and+Historical+Representation%2E+Johns+Hopkins+University+Press%2E)

*Updated 1 November 2026*