---
title: "Prompting and Fine-Tuning Open-Sourced Large Language"
description: "Cruickshank and Ng test whether open-source large language models can do this job with little or no… 67 references, cited by 10 papers here."
authors: ["Iain J. Cruickshank", "Lynnette Hui Xian Ng"]
year: 2023
venue: "arXiv:2309.13734"
source: "https://arxiv.org/abs/2309.13734"
doi: "10.48550/arxiv.2309.13734"
arxiv: "2309.13734"
licence: "CC BY 4.0 (arXiv)"
licence_url: "http://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/cruickshank-2023-prompting-fine-tuning-open-sourced/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/cruickshank-2023-prompting-fine-tuning-open-sourced/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2023](https://md.research.elamur.ai/free/en/papers/2023/page/10/) / Prompting and Fine-Tuning Open-Sourced Large Language Models for Stance Classification

# Prompting and Fine-Tuning Open-Sourced Large Language Models for Stance Classification

**Authors:** Iain J. Cruickshank and [Lynnette Hui Xian Ng](https://md.research.elamur.ai/free/en/authors/lynnette-hui-xian-ng/) · **Year:** 2023 · **Venue:** arXiv:2309.13734 · **Licence:** [CC BY 4.0 (arXiv)](http://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://arxiv.org/abs/2309.13734) · [PDF](https://arxiv.org/pdf/2309.13734) · [DOI 10.48550/arxiv.2309.13734](https://doi.org/10.48550/arxiv.2309.13734) · [arXiv 2309.13734](https://arxiv.org/abs/2309.13734) · [OpenAlex](https://openalex.org/W4387075633)

**Citations:** 8 citations (OpenAlex, 2026-10-14) · [10](https://md.research.elamur.ai/free/en/papers/cruickshank-2023-prompting-fine-tuning-open-sourced/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
Stance classification identifies the position an author takes toward a given target. Most existing systems depend on costly hand-labelled training data and transfer poorly to new topics. Cruickshank and Ng test whether open-source large language models can do this job with little or no annotation. Their benchmark crosses 10 openly available LLMs with 7 prompting strategies, including few-shot prompting (FSP) and chain-of-thought (CoT). They evaluate these combinations on 6 Twitter datasets that already carry human stance labels. They also fine-tune the models to see whether task specialization helps.

The results are mixed. LLMs can match in-domain supervised classifiers and sometimes beat them, but only with particular prompts. Their scores vary widely across models, prompts and datasets. Fine-tuning brought no reliable gain and sometimes reduced out-of-domain accuracy. The authors argue that LLM developers should treat stance detection as a standard benchmark. For anyone labelling opinion in social media, the study cautions against using LLMs as a simple replacement for supervised models.

## Key points
- Scale of evaluation: 10 open-source LLMs × 7 prompting schemes × 6 manually annotated Twitter stance datasets. Decoding was greedy, and FSP examples were chosen by hand.
- LLMs are competitive with, but not consistently better than, smaller supervised baselines. They win only under specific prompt designs, so prompt wording matters a lot.
- Among outputs where the model produced a valid stance label, roughly half (50.8%) were correct.
- Fine-tuning did not guarantee improvement and could hurt out-of-domain performance. The authors therefore favor general models over specialized ones.
- Practical advice: encoder-decoder architectures combined with FSP or CoT prompts gave the best results. The code is published at github.com/ijcruic/LLM-Stance-Labeling.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [United States Army](https://md.research.elamur.ai/free/en/government/united-states-army/) — Government body, Arlington, United States · 2 works · 291 citations

Also on the byline: [Army Cyber Institute at West Point](https://ror.org/01b61x679).

📚 Shelves (6): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works · [🤝 Social Sciences](https://md.research.elamur.ai/free/en/shelves/social-sciences/) · 1,772 works · [🤝 General Social Sciences](https://md.research.elamur.ai/free/en/shelves/social-sciences/general-social-sciences/) · 227 works

🧭 Topics (1): [🧠 Language models & pretraining](https://md.research.elamur.ai/free/en/topics/language-models/) · 3,320 works

## Cite

Cite as: Iain J. Cruickshank and Lynnette Hui Xian Ng. 2023. Prompting and Fine-Tuning Open-Sourced Large Language Models for Stance Classification. arXiv:2309.13734.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@misc{cruickshank2023prompting,
  title = {{Prompting and Fine-Tuning Open-Sourced Large Language Models for Stance Classification}},
  author = {Iain J. Cruickshank and Lynnette Hui Xian Ng},
  year = {2023},
  doi = {10.48550/arxiv.2309.13734},
  eprint = {2309.13734},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2309.13734}
}
```

- [← previous Predict and Use: Harnessing Predicted Gaze to Improve Multimodal Sarcasm Detection](https://md.research.elamur.ai/free/en/papers/tiwari-2023-predict-use-harnessing-predicted-gaze/)
- [next → Violet: A Vision-Language Model for Arabic Image Captioning with Gemini Decoder](https://md.research.elamur.ai/free/en/papers/mohamed-2023-violet-vision-language-model-arabic/)
- [↑ back to papers of 2023](https://md.research.elamur.ai/free/en/papers/2023/page/10/)

## References

67 references, 24 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Rachith Aiyappa, Jisun An, Haewoon Kwak, and Yong-Yeol Ahn. 2023. [Can we trust the evaluation on ChatGPT?](https://md.research.elamur.ai/free/en/papers/aiyappa-2023-can-we-trust-evaluation-chatgpt/) arXiv preprint arXiv:2303.12767 (2023).
2. Abeer AlDayel and Walid Magdy. 2021. Stance detection on social media: State of the art and trends. Information Processing & Management 58, 4 (2021), 102597. [find](https://scholar.google.com/scholar?q=Abeer+AlDayel+and+Walid+Magdy%2E+2021%2E+Stance+detection+on+social+media%3A+State+of+the+art+and+trends%2E+Information+Processing+%26+Management+58%2C+4+%282021%29%2C+102597%2E)
3. Abeer Aldayel and Walid Magdy. 2022. Characterizing the role of bots’ in polarized stance on social media. Social Network Analysis and Mining 12, 1 (2022), 30. [find](https://scholar.google.com/scholar?q=Abeer+Aldayel+and+Walid+Magdy%2E+2022%2E+Characterizing+the+role+of+bots%E2%80%99+in+polarized+stance+on+social+media%2E+Social+Network+Analysis+and+Mining+12%2C+1+%282022%29%2C+30%2E)
4. Rabab Alkhalifa and Arkaitz Zubiaga. 2022. Capturing stance dynamics in social media: open challenges and research directions. International Journal of Digital Humanities 3, 1-3 (2022), 115–135. [find](https://scholar.google.com/scholar?q=Rabab+Alkhalifa+and+Arkaitz+Zubiaga%2E+2022%2E+Capturing+stance+dynamics+in+social+media%3A+open+challenges+and+research+directions%2E+International+Journal+of+Digital+Humanities+3%2C+1%2D3+%282022%29%2C+115%E2%80%93135%2E)
5. Emily Allaway and Kathleen McKeown. 2020. [Zero-shot stance detection: A dataset and model using generalized topic representations](https://md.research.elamur.ai/free/en/papers/allaway-2020-zero-shot-stance-detection-dataset/). arXiv preprint arXiv:2010.03640 (2020).
6. Emily Allaway and Kathleen McKeown. 2023. Zero-shot stance detection: Paradigms and challenges. Frontiers in Artificial Intelligence 5 (2023), 1070429. [find](https://scholar.google.com/scholar?q=Emily+Allaway+and+Kathleen+McKeown%2E+2023%2E+Zero%2Dshot+stance+detection%3A+Paradigms+and+challenges%2E+Frontiers+in+Artificial+Intelligence+5+%282023%29%2C+1070429%2E)
7. Nora Alturayeif, Hamzah Luqman, and Moataz Ahmed. 2023. A systematic review of machine learning techniques for stance detection and its applications. Neural Computing and Applications 35, 7 (2023), 5113–5144. [find](https://scholar.google.com/scholar?q=Nora+Alturayeif%2C+Hamzah+Luqman%2C+and+Moataz+Ahmed%2E+2023%2E+A+systematic+review+of+machine+learning+techniques+for+stance+detection+and+its+applications%2E+Neural+Computing+and+Applications+35%2C+7+%282023%29%2C+51)
8. Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. 2020. [Language models are few-shot learners](https://md.research.elamur.ai/free/en/papers/brown-2020-language-models-few-shot-learners/). Advances in neural information processing systems 33 (2020), 1877–1901.
9. Jiuhai Chen, Lichang Chen, Heng Huang, and Tianyi Zhou. 2023. When do you need Chain-of-Thought Prompting for ChatGPT? arXiv preprint arXiv:2304.03262 (2023). — [arXiv:2304.03262](https://arxiv.org/abs/2304.03262) [find](https://scholar.google.com/scholar?q=Jiuhai+Chen%2C+Lichang+Chen%2C+Heng+Huang%2C+and+Tianyi+Zhou%2E+2023%2E+When+do+you+need+Chain%2Dof%2DThought+Prompting+for+ChatGPT%3F+arXiv+preprint+arXiv%3A2304%2E03262+%282023%29%2E+%E2%80%94+%5BarXiv%3A2304%2E03262%5D%28https%3A%2F%2Farxiv%2Eorg%2Fab)
10. Hyung Won Chung, Le Hou, Shayne Longpre, Barret Zoph, Yi Tay, William Fedus, Eric Li, Xuezhi Wang, Mostafa Dehghani, Siddhartha Brahma, et al. 2022. Scaling instruction-finetuned language models. arXiv preprint arXiv:2210.11416 (2022). [find](https://scholar.google.com/scholar?q=Hyung+Won+Chung%2C+Le+Hou%2C+Shayne+Longpre%2C+Barret+Zoph%2C+Yi+Tay%2C+William+Fedus%2C+Eric+Li%2C+Xuezhi+Wang%2C+Mostafa+Dehghani%2C+Siddhartha+Brahma%2C+et+al%2E+2022%2E+Scaling+instruction%2Dfinetuned+language+models%2E+arXi)
11. Costanza Conforti, Jakob Berndt, Mohammad Taher Pilehvar, Chryssi Giannitsarou, Flavio Toxvaerd, and Nigel Collier. 2020. [Will-They-Won’t-They: A Very Large Dataset for Stance Detection on Twitter](https://md.research.elamur.ai/free/en/papers/conforti-2020-will-they-won-t-they/). In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. 1715–1724.
12. Kareem Darwish, Peter Stefanov, Michaël Aupetit, and Preslav Nakov. 2020. Unsupervised user stance detection on Twitter. In Proceedings of the International AAAI Conference on Web and Social Media, Vol. 14. 141–152. Manuscript submitted to ACM [find](https://scholar.google.com/scholar?q=Kareem+Darwish%2C+Peter+Stefanov%2C+Micha%C3%ABl+Aupetit%2C+and+Preslav+Nakov%2E+2020%2E+Unsupervised+user+stance+detection+on+Twitter%2E+In+Proceedings+of+the+International+AAAI+Conference+on+Web+and+Social+Media%2C+Vo)
13. Ning Ding, Yujia Qin, Guang Yang, Fuchao Wei, Zonghan Yang, Yusheng Su, Shengding Hu, Yulin Chen, Chi-Min Chan, Weize Chen, et al. 2023. [Parameter-efficient fine-tuning of large-scale pre-trained language models](https://md.research.elamur.ai/free/en/papers/ding-2023-parameter-efficient-fine-tuning-large/). Nature Machine Intelligence 5, 3 (2023), 220–235.
14. Heba Elfardy and Mona Diab. 2016. Cu-gwu perspective at semeval-2016 task 6: Ideological stance detection in informal text. In Proceedings of the 10th International Workshop on Semantic Evaluation (SemEval-2016). 434–439. [find](https://scholar.google.com/scholar?q=Heba+Elfardy+and+Mona+Diab%2E+2016%2E+Cu%2Dgwu+perspective+at+semeval%2D2016+task+6%3A+Ideological+stance+detection+in+informal+text%2E+In+Proceedings+of+the+10th+International+Workshop+on+Semantic+Evaluation+%28Se)
15. Hao Fei, Bobo Li, Qian Liu, Lidong Bing, Fei Li, and Tat-Seng Chua. 2023. Reasoning Implicit Sentiment with Chain-of-Thought Prompting. arXiv preprint arXiv:2305.11255 (2023). [find](https://scholar.google.com/scholar?q=Hao+Fei%2C+Bobo+Li%2C+Qian+Liu%2C+Lidong+Bing%2C+Fei+Li%2C+and+Tat%2DSeng+Chua%2E+2023%2E+Reasoning+Implicit+Sentiment+with+Chain%2Dof%2DThought+Prompting%2E+arXiv+preprint+arXiv%3A2305%2E11255+%282023%29%2E)
16. Suriya Gunasekar, Yi Zhang, Jyoti Aneja, Caio César Teodoro Mendes, Allie Del Giorno, Sivakanth Gopi, Mojan Javaheripi, Piero Kauffmann, Gustavo de Rosa, Olli Saarikivi, et al. 2023. Textbooks Are All You Need. arXiv preprint arXiv:2306.11644 (2023). [find](https://scholar.google.com/scholar?q=Suriya+Gunasekar%2C+Yi+Zhang%2C+Jyoti+Aneja%2C+Caio+C%C3%A9sar+Teodoro+Mendes%2C+Allie+Del+Giorno%2C+Sivakanth+Gopi%2C+Mojan+Javaheripi%2C+Piero+Kauffmann%2C+Gustavo+de+Rosa%2C+Olli+Saarikivi%2C+et+al%2E+2023%2E+Textbooks+Are+All)
17. Tamanna Hossain, Robert L. Logan IV, Arjuna Ugarte, Yoshitomo Matsubara, Sean Young, and Sameer Singh. 2020. [COVIDLies: Detecting COVID-19 Misinformation on Social Media](https://md.research.elamur.ai/free/en/papers/hossain-2020-covidlies-detecting-covid-19-misinformation/). In Proceedings of the 1st Workshop on NLP for COVID-19 (Part 2) at EMNLP 2020.
18. Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2021. [Lora: Low-rank adaptation of large language models](https://md.research.elamur.ai/free/en/papers/hu-2021-lora-low-rank-adaptation-large/). arXiv preprint arXiv:2106.09685 (2021).
19. Albert Q Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, et al. 2023. Mistral 7B. arXiv preprint arXiv:2310.06825 (2023). [find](https://scholar.google.com/scholar?q=Albert+Q+Jiang%2C+Alexandre+Sablayrolles%2C+Arthur+Mensch%2C+Chris+Bamford%2C+Devendra+Singh+Chaplot%2C+Diego+de+las+Casas%2C+Florian+Bressand%2C+Gianna+Lengyel%2C+Guillaume+Lample%2C+Lucile+Saulnier%2C+et+al%2E+2023%2E+Mist)
20. Kornraphop Kawintiranon and Lisa Singh. 2021. [Knowledge Enhanced Masked Language Model for Stance Detection](https://md.research.elamur.ai/free/en/papers/kawintiranon-2021-knowledge-enhanced-masked-language-model/). In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 4725–4735.
21. Kiana Kheiri and Hamid Karimi. 2023. [SentimentGPT: Exploiting GPT for Advanced Sentiment Analysis and its Departure from Current Machine Learning](https://md.research.elamur.ai/free/en/papers/kheiri-2023-sentimentgpt-exploiting-gpt-advanced-sentiment/). arXiv preprint arXiv:2307.10234 (2023).
22. Elena Kochkina, Maria Liakata, and Arkaitz Zubiaga. 2018. All-in-one: Multi-task Learning for Rumour Verification. In Proceedings of the 27th International Conference on Computational Linguistics. 3402–3413. [find](https://scholar.google.com/scholar?q=Elena+Kochkina%2C+Maria+Liakata%2C+and+Arkaitz+Zubiaga%2E+2018%2E+All%2Din%2Done%3A+Multi%2Dtask+Learning+for+Rumour+Verification%2E+In+Proceedings+of+the+27th+International+Conference+on+Computational+Linguistics%2E+340)
23. Ananya Kumar, Aditi Raghunathan, Robbie Jones, Tengyu Ma, and Percy Liang. 2022. Fine-tuning can distort pretrained features and underperform out-of-distribution. arXiv preprint arXiv:2202.10054 (2022). [find](https://scholar.google.com/scholar?q=Ananya+Kumar%2C+Aditi+Raghunathan%2C+Robbie+Jones%2C+Tengyu+Ma%2C+and+Percy+Liang%2E+2022%2E+Fine%2Dtuning+can+distort+pretrained+features+and+underperform+out%2Dof%2Ddistribution%2E+arXiv+preprint+arXiv%3A2202%2E10054+%282022)
24. Mirko Lai, Viviana Patti, Giancarlo Ruffo, and Paolo Rosso. 2018. Stance evolution and twitter interactions in an italian political debate. In International Conference on Applications of Natural Language to Information Systems. Springer, 15–27. [find](https://scholar.google.com/scholar?q=Mirko+Lai%2C+Viviana+Patti%2C+Giancarlo+Ruffo%2C+and+Paolo+Rosso%2E+2018%2E+Stance+evolution+and+twitter+interactions+in+an+italian+political+debate%2E+In+International+Conference+on+Applications+of+Natural+Langu)
25. Xiaochong Lan, Chen Gao, Depeng Jin, and Yong Li. 2023. Stance detection with collaborative role-infused llm-based agents. arXiv preprint arXiv:2310.10467 (2023). [find](https://scholar.google.com/scholar?q=Xiaochong+Lan%2C+Chen+Gao%2C+Depeng+Jin%2C+and+Yong+Li%2E+2023%2E+Stance+detection+with+collaborative+role%2Dinfused+llm%2Dbased+agents%2E+arXiv+preprint+arXiv%3A2310%2E10467+%282023%29%2E)
26. Ang Li, Bin Liang, Jingqian Zhao, Bowen Zhang, Min Yang, and Ruifeng Xu. 2023. [Stance Detection on Social Media with Background Knowledge](https://md.research.elamur.ai/free/en/papers/li-2023-stance-detection-social-media-background/). In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing. 15703–15717.
27. Chandreen Liyanage, Ravi Gokani, and Vijay Mago. 2023. GPT-4 as a Twitter Data Annotator: Unraveling Its Performance on a Stance Classification Task. Authorea Preprints (2023). [find](https://scholar.google.com/scholar?q=Chandreen+Liyanage%2C+Ravi+Gokani%2C+and+Vijay+Mago%2E+2023%2E+GPT%2D4+as+a+Twitter+Data+Annotator%3A+Unraveling+Its+Performance+on+a+Stance+Classification+Task%2E+Authorea+Preprints+%282023%29%2E)
28. Ilya Loshchilov and Frank Hutter. 2017. Decoupled weight decay regularization. arXiv preprint arXiv:1711.05101 (2017). [find](https://scholar.google.com/scholar?q=Ilya+Loshchilov+and+Frank+Hutter%2E+2017%2E+Decoupled+weight+decay+regularization%2E+arXiv+preprint+arXiv%3A1711%2E05101+%282017%29%2E)
29. Yao Lu, Max Bartolo, Alastair Moore, Sebastian Riedel, and Pontus Stenetorp. 2021. [Fantastically ordered prompts and where to find them: Overcoming few-shot prompt order sensitivity](https://md.research.elamur.ai/free/en/papers/lu-2022-fantastically-ordered-prompts-where-find/). arXiv preprint arXiv:2104.08786 (2021).
30. Mark Mets, Andres Karjus, Indrek Ibrus, and Maximilian Schich. 2023. [Automated stance detection in complex topics and small languages: the challenging case of immigration in polarizing news media](https://md.research.elamur.ai/free/en/papers/mets-2023-automated-stance-detection-complex-topics/). arXiv preprint arXiv:2305.13047 (2023).
31. Paulius Micikevicius, Sharan Narang, Jonah Alben, Gregory Diamos, Erich Elsen, David Garcia, Boris Ginsburg, Michael Houston, Oleksii Kuchaiev, Ganesh Venkatesh, et al. 2017. Mixed precision training. arXiv preprint arXiv:1710.03740 (2017). [find](https://scholar.google.com/scholar?q=Paulius+Micikevicius%2C+Sharan+Narang%2C+Jonah+Alben%2C+Gregory+Diamos%2C+Erich+Elsen%2C+David+Garcia%2C+Boris+Ginsburg%2C+Michael+Houston%2C+Oleksii+Kuchaiev%2C+Ganesh+Venkatesh%2C+et+al%2E+2017%2E+Mixed+precision+training%2E)
32. Saif Mohammad, Svetlana Kiritchenko, Parinaz Sobhani, Xiaodan Zhu, and Colin Cherry. 2016. [Semeval-2016 task 6: Detecting stance in tweets](https://md.research.elamur.ai/free/en/papers/mohammad-2016-semeval-2016-task-6-detecting/). In Proceedings of the 10th international workshop on semantic evaluation (SemEval-2016). 31–41.
33. Akiko Murakami and Rudy Raymond. 2010. Support or oppose? Classifying positions in online debates from reply activities and opinion expressions. In Coling 2010: Posters. 869–875. [find](https://scholar.google.com/scholar?q=Akiko+Murakami+and+Rudy+Raymond%2E+2010%2E+Support+or+oppose%3F+Classifying+positions+in+online+debates+from+reply+activities+and+opinion+expressions%2E+In+Coling+2010%3A+Posters%2E+869%E2%80%93875%2E)
34. Lynnette Hui Xian Ng and Kathleen M Carley. 2022. Is my stance the same as your stance? A cross validation study of stance detection datasets. Information Processing & Management 59, 6 (2022), 103070. [find](https://scholar.google.com/scholar?q=Lynnette+Hui+Xian+Ng+and+Kathleen+M+Carley%2E+2022%2E+Is+my+stance+the+same+as+your+stance%3F+A+cross+validation+study+of+stance+detection+datasets%2E+Information+Processing+%26+Management+59%2C+6+%282022%29%2C+103070%2E)
35. Lynnette Hui Xian Ng and Kathleen M Carley. 2022. Pro or Anti? a social influence model of online stance flipping. IEEE Transactions on Network Science and Engineering 10, 1 (2022), 3–19. [find](https://scholar.google.com/scholar?q=Lynnette+Hui+Xian+Ng+and+Kathleen+M+Carley%2E+2022%2E+Pro+or+Anti%3F+a+social+influence+model+of+online+stance+flipping%2E+IEEE+Transactions+on+Network+Science+and+Engineering+10%2C+1+%282022%29%2C+3%E2%80%9319%2E)
36. OpenAI. 2023. [GPT-4 Technical Report](https://md.research.elamur.ai/free/en/papers/openai-2023-gpt-4-technical-report/). [https://cdn.openai.com/papers/gpt-4.pdf](https://cdn.openai.com/papers/gpt-4.pdf). [Accessed 27-02-2024]. [↗](https://cdn.openai.com/papers/gpt-4.pdf)
37. Ron Korenblum Pick, Vladyslav Kozhukhov, Dan Vilenchik, and Oren Tsur. 2022. STEM: unsupervised structural embedding for stance detection. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 36. 11174–11182. [find](https://scholar.google.com/scholar?q=Ron+Korenblum+Pick%2C+Vladyslav+Kozhukhov%2C+Dan+Vilenchik%2C+and+Oren+Tsur%2E+2022%2E+STEM%3A+unsupervised+structural+embedding+for+stance+detection%2E+In+Proceedings+of+the+AAAI+Conference+on+Artificial+Intellige)
38. Sunil Ramlochan. 2023. What is Prompt Engineering? [https://www.promptengineering.org/what-is-prompt-engineering/](https://www.promptengineering.org/what-is-prompt-engineering/). Accessed: 2023-09-14. [↗](https://www.promptengineering.org/what-is-prompt-engineering/)
39. Alexander J Ratner, Christopher M De Sa, Sen Wu, Daniel Selsam, and Christopher Ré. 2016. Data programming: Creating large training sets, quickly. Advances in neural information processing systems 29 (2016). [find](https://scholar.google.com/scholar?q=Alexander+J+Ratner%2C+Christopher+M+De+Sa%2C+Sen+Wu%2C+Daniel+Selsam%2C+and+Christopher+R%C3%A9%2E+2016%2E+Data+programming%3A+Creating+large+training+sets%2C+quickly%2E+Advances+in+neural+information+processing+systems+29+)
40. Kevin Schaul, Szu Yu Chen, and Nitasha Tiku. 2023. Inside the Secret List of Websites that make AI like ChatGPT Sound Smart. [https://www.washingtonpost.com/technology/interactive/2023/ai-chatbot-learning/](https://www.washingtonpost.com/technology/interactive/2023/ai-chatbot-learning/) [↗](https://www.washingtonpost.com/technology/interactive/2023/ai-chatbot-learning/)
41. Douglas C Schmidt, Jesse Spencer-Smith, Quchen Fu, and Jules White. [n. d.]. Cataloging Prompt Patterns to Enhance the Discipline of Prompt Engineering. ([n. d.]). [find](https://scholar.google.com/scholar?q=Douglas+C+Schmidt%2C+Jesse+Spencer%2DSmith%2C+Quchen+Fu%2C+and+Jules+White%2E+%5Bn%2E+d%2E%5D%2E+Cataloging+Prompt+Patterns+to+Enhance+the+Discipline+of+Prompt+Engineering%2E+%28%5Bn%2E+d%2E%5D%29%2E)
42. Noam Shazeer. 2019. Fast transformer decoding: One write-head is all you need. arXiv preprint arXiv:1911.02150 (2019). Manuscript submitted to ACM [find](https://scholar.google.com/scholar?q=Noam+Shazeer%2E+2019%2E+Fast+transformer+decoding%3A+One+write%2Dhead+is+all+you+need%2E+arXiv+preprint+arXiv%3A1911%2E02150+%282019%29%2E+Manuscript+submitted+to+ACM)
43. Parinaz Sobhani, Diana Inkpen, and Xiaodan Zhu. 2017. [A dataset for multi-target stance detection](https://md.research.elamur.ai/free/en/papers/sobhani-2017-dataset-multi-target-stance-detection/). In Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 2, Short Papers. 551–557.
44. Marek Šuppa, Daniel Skala, Daniela Jašš, Samuel Sučík, Andrej Švec, and Peter Hraška. 2024. Bryndza at ClimateActivism 2024: Stance, Target and Hate Event Detection via Retrieval-Augmented GPT-4 and LLaMA. arXiv preprint arXiv:2402.06549 (2024). [find](https://scholar.google.com/scholar?q=Marek+%C5%A0uppa%2C+Daniel+Skala%2C+Daniela+Ja%C5%A1%C5%A1%2C+Samuel+Su%C4%8D%C3%ADk%2C+Andrej+%C5%A0vec%2C+and+Peter+Hra%C5%A1ka%2E+2024%2E+Bryndza+at+ClimateActivism+2024%3A+Stance%2C+Target+and+Hate+Event+Detection+via+Retrieval%2DAugmented+GPT%2D4+and+L)
45. Yi Tay, Mostafa Dehghani, Vinh Q Tran, Xavier Garcia, Dara Bahri, Tal Schuster, Huaixiu Steven Zheng, Neil Houlsby, and Donald Metzler. 2022. [Unifying language learning paradigms](https://md.research.elamur.ai/free/en/papers/tay-2022-ul2-unifying-language-learning-paradigms/). arXiv preprint arXiv:2205.05131 (2022).
46. Surendrabikram Thapa, Kritesh Rauniyar, Farhan Ahmad Jafri, Shuvam Shiwakoti, Hariram Veeramani, Raghav Jain, Guneet Singh Kohli, Ali Hürriyetoğlu, and Usman Naseem. 2024. Stance and Hate Event Detection in Tweets Related to Climate Activism - Shared Task at CASE 2024. In Proceedings of the 7th Work… [find](https://scholar.google.com/scholar?q=Surendrabikram+Thapa%2C+Kritesh+Rauniyar%2C+Farhan+Ahmad+Jafri%2C+Shuvam+Shiwakoti%2C+Hariram+Veeramani%2C+Raghav+Jain%2C+Guneet+Singh+Kohli%2C+Ali+H%C3%BCrriyeto%C4%9Flu%2C+and+Usman+Naseem%2E+2024%2E+Stance+and+Hate+Event+Detect)
47. Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. 2023. [Llama 2: Open foundation and fine-tuned chat models](https://md.research.elamur.ai/free/en/papers/touvron-2023-llama-2-open-foundation-fine/). arXiv preprint arXiv:2307.09288 (2023).
48. Apoorva Upadhyaya, Marco Fisichella, and Wolfgang Nejdl. 2023. A Multi-task Model for Sentiment Aided Stance Detection of Climate Change Tweets. In Proceedings of the International AAAI Conference on Web and Social Media, Vol. 17. 854–865. [find](https://scholar.google.com/scholar?q=Apoorva+Upadhyaya%2C+Marco+Fisichella%2C+and+Wolfgang+Nejdl%2E+2023%2E+A+Multi%2Dtask+Model+for+Sentiment+Aided+Stance+Detection+of+Climate+Change+Tweets%2E+In+Proceedings+of+the+International+AAAI+Conference+on+)
49. Ramon Villa-Cox, Sumeet Kumar, Matthew Babcock, and Kathleen M Carley. 2020. Stance in replies and quotes (srq): A new dataset for learning stance in twitter conversations. arXiv preprint arXiv:2006.00691 (2020). — [arXiv:2006.00691](https://arxiv.org/abs/2006.00691) [find](https://scholar.google.com/scholar?q=Ramon+Villa%2DCox%2C+Sumeet+Kumar%2C+Matthew+Babcock%2C+and+Kathleen+M+Carley%2E+2020%2E+Stance+in+replies+and+quotes+%28srq%29%3A+A+new+dataset+for+learning+stance+in+twitter+conversations%2E+arXiv+preprint+arXiv%3A2006%2E0)
50. Ingmar Weber, Venkata R. Kiran Garimella, and Alaa Batayneh. 2013. Secular vs. Islamist Polarization in Egypt on Twitter. In Proceedings of the 2013 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining (Niagara, Ontario, Canada) (ASONAM ’13). Association for Computing… — [doi:10.1145/2492517.2492557](https://doi.org/10.1145/2492517.2492557) [find](https://scholar.google.com/scholar?q=Ingmar+Weber%2C+Venkata+R%2E+Kiran+Garimella%2C+and+Alaa+Batayneh%2E+2013%2E+Secular+vs%2E+Islamist+Polarization+in+Egypt+on+Twitter%2E+In+Proceedings+of+the+2013+IEEE%2FACM+International+Conference+on+Advances+in+So)
51. Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. 2022. [Chain-of-thought prompting elicits reasoning in large language models](https://md.research.elamur.ai/free/en/papers/wei-2022-chain-thought-prompting-elicits-reasoning/). Advances in Neural Information Processing Systems 35 (2022), 24824–24837.
52. Jerry Wei, Jason Wei, Yi Tay, Dustin Tran, Albert Webson, Yifeng Lu, Xinyun Chen, Hanxiao Liu, Da Huang, Denny Zhou, et al. 2023. [Larger language models do in-context learning differently](https://md.research.elamur.ai/free/en/papers/wei-2023-larger-language-models-context-learning/). arXiv preprint arXiv:2303.03846 (2023).
53. Wan Wei, Xiao Zhang, Xuqin Liu, Wei Chen, and Tengjiao Wang. 2016. [pkudblab at semeval-2016 task 6: A specific convolutional neural network system for effective stance detection](https://md.research.elamur.ai/free/en/papers/wei-2016-pkudblab-semeval-2016-task-6/). In Proceedings of the 10th international workshop on semantic evaluation (SemEval-2016). 384–388.
54. Jules White, Quchen Fu, Sam Hays, Michael Sandborn, Carlos Olea, Henry Gilbert, Ashraf Elnashar, Jesse Spencer-Smith, and Douglas C Schmidt. 2023. [A prompt pattern catalog to enhance prompt engineering with chatgpt](https://md.research.elamur.ai/free/en/papers/white-2023-prompt-pattern-catalog-enhance-prompt/). arXiv preprint arXiv:2302.11382 (2023).
55. Junjielong Xu, Ziang Cui, Yuan Zhao, Xu Zhang, Shilin He, Pinjia He, Liqun Li, Yu Kang, Qingwei Lin, Yingnong Dang, et al. 2024. UniLog: Automatic Logging via LLM and In-Context Learning. In Proceedings of the 46th IEEE/ACM International Conference on Software Engineering. 1–12. [find](https://scholar.google.com/scholar?q=Junjielong+Xu%2C+Ziang+Cui%2C+Yuan+Zhao%2C+Xu+Zhang%2C+Shilin+He%2C+Pinjia+He%2C+Liqun+Li%2C+Yu+Kang%2C+Qingwei+Lin%2C+Yingnong+Dang%2C+et+al%2E+2024%2E+UniLog%3A+Automatic+Logging+via+LLM+and+In%2DContext+Learning%2E+In+Proceedin)
56. Ruoyao Yang, Wanying Xie, Chunhua Liu, and Dong Yu. 2019. BLCU_NLP at SemEval-2019 Task 7: An Inference Chain-based GPT Model for Rumour Evaluation. In Proceedings of the 13th International Workshop on Semantic Evaluation. Association for Computational Linguistics, Minneapolis, Minnesota, USA, 1090–… [find](https://scholar.google.com/scholar?q=Ruoyao+Yang%2C+Wanying+Xie%2C+Chunhua+Liu%2C+and+Dong+Yu%2E+2019%2E+BLCU%5FNLP+at+SemEval%2D2019+Task+7%3A+An+Inference+Chain%2Dbased+GPT+Model+for+Rumour+Evaluation%2E+In+Proceedings+of+the+13th+International+Workshop+o)
57. Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran, Thomas L Griffiths, Yuan Cao, and Karthik Narasimhan. 2023. Tree of thoughts: Deliberate problem solving with large language models. arXiv preprint arXiv:2305.10601 (2023). [find](https://scholar.google.com/scholar?q=Shunyu+Yao%2C+Dian+Yu%2C+Jeffrey+Zhao%2C+Izhak+Shafran%2C+Thomas+L+Griffiths%2C+Yuan+Cao%2C+and+Karthik+Narasimhan%2E+2023%2E+Tree+of+thoughts%3A+Deliberate+problem+solving+with+large+language+models%2E+arXiv+preprint+ar)
58. Zhiqiang Yuan, Junwei Liu, Qiancheng Zi, Mingwei Liu, Xin Peng, and Yiling Lou. 2023. Evaluating instruction-tuned large language models on code comprehension and generation. arXiv preprint arXiv:2308.01240 (2023). — [arXiv:2308.01240](https://arxiv.org/abs/2308.01240) [find](https://scholar.google.com/scholar?q=Zhiqiang+Yuan%2C+Junwei+Liu%2C+Qiancheng+Zi%2C+Mingwei+Liu%2C+Xin+Peng%2C+and+Yiling+Lou%2E+2023%2E+Evaluating+instruction%2Dtuned+large+language+models+on+code+comprehension+and+generation%2E+arXiv+preprint+arXiv%3A2308)
59. Guido Zarrella and Amy Marsh. 2016. [MITRE at SemEval-2016 Task 6: Transfer Learning for Stance Detection](https://md.research.elamur.ai/free/en/papers/zarrella-2016-mitre-semeval-2016-task-6/). In Proceedings of the 10th International Workshop on Semantic Evaluation (SemEval-2016). 458–463.
60. Bowen Zhang, Daijun Ding, and Liwen Jing. 2022. How would stance detection techniques evolve after the launch of chatgpt? arXiv preprint arXiv:2212.14548 (2022). [find](https://scholar.google.com/scholar?q=Bowen+Zhang%2C+Daijun+Ding%2C+and+Liwen+Jing%2E+2022%2E+How+would+stance+detection+techniques+evolve+after+the+launch+of+chatgpt%3F+arXiv+preprint+arXiv%3A2212%2E14548+%282022%29%2E)
61. Bowen Zhang, Daijun Ding, Liwen Jing, and Hu Huang. 2023. A Logically Consistent Chain-of-Thought Approach for Stance Detection. arXiv preprint arXiv:2312.16054 (2023). — [arXiv:2312.16054](https://arxiv.org/abs/2312.16054) [find](https://scholar.google.com/scholar?q=Bowen+Zhang%2C+Daijun+Ding%2C+Liwen+Jing%2C+and+Hu+Huang%2E+2023%2E+A+Logically+Consistent+Chain%2Dof%2DThought+Approach+for+Stance+Detection%2E+arXiv+preprint+arXiv%3A2312%2E16054+%282023%29%2E+%E2%80%94+%5BarXiv%3A2312%2E16054%5D%28https%3A%2F%2Far)
62. Bowen Zhang, Xianghua Fu, Daijun Ding, Hu Huang, Yangyang Li, and Liwen Jing. 2023. Investigating Chain-of-thought with ChatGPT for Stance Detection on Social Media. arXiv preprint arXiv:2304.03087 (2023). [find](https://scholar.google.com/scholar?q=Bowen+Zhang%2C+Xianghua+Fu%2C+Daijun+Ding%2C+Hu+Huang%2C+Yangyang+Li%2C+and+Liwen+Jing%2E+2023%2E+Investigating+Chain%2Dof%2Dthought+with+ChatGPT+for+Stance+Detection+on+Social+Media%2E+arXiv+preprint+arXiv%3A2304%2E03087+%282)
63. Biao Zhang, Zhongtao Liu, Colin Cherry, and Orhan Firat. 2023. When Scaling Meets LLM Finetuning: The Effect of Data, Model and Finetuning Method. In The Twelfth International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Biao+Zhang%2C+Zhongtao+Liu%2C+Colin+Cherry%2C+and+Orhan+Firat%2E+2023%2E+When+Scaling+Meets+LLM+Finetuning%3A+The+Effect+of+Data%2C+Model+and+Finetuning+Method%2E+In+The+Twelfth+International+Conference+on+Learning+R)
64. Chong Zhang, Zhenkun Zhou, Xingyu Peng, and Ke Xu. 2023. DoubleH: Twitter User Stance Detection via Bipartite Graph Neural Networks. arXiv preprint arXiv:2301.08774 (2023). — [arXiv:2301.08774](https://arxiv.org/abs/2301.08774) [find](https://scholar.google.com/scholar?q=Chong+Zhang%2C+Zhenkun+Zhou%2C+Xingyu+Peng%2C+and+Ke+Xu%2E+2023%2E+DoubleH%3A+Twitter+User+Stance+Detection+via+Bipartite+Graph+Neural+Networks%2E+arXiv+preprint+arXiv%3A2301%2E08774+%282023%29%2E+%E2%80%94+%5BarXiv%3A2301%2E08774%5D%28https%3A)
65. Guangzhen Zhao and Peng Yang. 2020. Pretrained embeddings for stance detection with hierarchical capsule network on social media. ACM Transactions on Information Systems (TOIS) 39, 1 (2020), 1–32. [find](https://scholar.google.com/scholar?q=Guangzhen+Zhao+and+Peng+Yang%2E+2020%2E+Pretrained+embeddings+for+stance+detection+with+hierarchical+capsule+network+on+social+media%2E+ACM+Transactions+on+Information+Systems+%28TOIS%29+39%2C+1+%282020%29%2C+1%E2%80%9332%2E)
66. Zihao Zhao, Eric Wallace, Shi Feng, Dan Klein, and Sameer Singh. 2021. [Calibrate before use: Improving few-shot performance of language models](https://md.research.elamur.ai/free/en/papers/zhao-2021-calibrate-before-use-improving-few/). In International Conference on Machine Learning. PMLR, 12697–12706.
67. Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric. P Xing, Hao Zhang, Joseph E. Gonzalez, and Ion Stoica. 2023. [Judging LLM-as-a-judge with MT-Bench and Chatbot Arena](https://md.research.elamur.ai/free/en/papers/zheng-2023-judging-llm-judge-mt-bench/). arXiv:2306.05685 [cs.CL] Manuscript submitted to ACM

*Updated 30 October 2026*