---
title: "Investigating Chain-of-thought with ChatGPT for Stance"
description: "Fine-tuning models with hundreds of billions of parameters is too expensive, so the authors test… 24 references, cited by 22 papers here."
authors: ["Bowen Zhang", "Xianghua Fu", "Daijun Ding", "Hu Huang", "Genan Dai", "Nan Yin", "Yangyang Li", "Liwen Jing"]
year: 2023
venue: "arXiv:2304.03087"
source: "https://arxiv.org/abs/2304.03087"
doi: "10.48550/arxiv.2304.03087"
arxiv: "2304.03087"
licence: "arXiv — non-exclusive licence to distribute"
licence_url: "http://arxiv.org/licenses/nonexclusive-distrib/1.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/zhang-2023-investigating-chain-thought-chatgpt-stance/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/zhang-2023-investigating-chain-thought-chatgpt-stance/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2023](https://md.research.elamur.ai/free/en/papers/2023/page/11/) / Investigating Chain-of-thought with ChatGPT for Stance Detection on Social Media

# Investigating Chain-of-thought with ChatGPT for Stance Detection on Social Media

**Authors:** [Bowen Zhang](https://md.research.elamur.ai/free/en/authors/bowen-zhang/), Xianghua Fu, [Daijun Ding](https://md.research.elamur.ai/free/en/authors/daijun-ding/), Hu Huang, [Genan Dai](https://md.research.elamur.ai/free/en/authors/genan-dai/), [Nan Yin](https://md.research.elamur.ai/free/en/authors/nan-yin/), Yangyang Li and [Liwen Jing](https://md.research.elamur.ai/free/en/authors/liwen-jing/) · **Year:** 2023 · **Venue:** arXiv:2304.03087 · **Licence:** [arXiv — non-exclusive licence to distribute](http://arxiv.org/licenses/nonexclusive-distrib/1.0/)

**Links:** [Publisher's page](https://arxiv.org/abs/2304.03087) · [PDF](https://arxiv.org/pdf/2304.03087) · [DOI 10.48550/arxiv.2304.03087](https://doi.org/10.48550/arxiv.2304.03087) · [arXiv 2304.03087](https://arxiv.org/abs/2304.03087) · [OpenAlex](https://openalex.org/W4362706813)

**Citations:** 17 citations (OpenAlex, 2026-10-14) · [22](https://md.research.elamur.ai/free/en/papers/zhang-2023-investigating-chain-thought-chatgpt-stance/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
This paper asks whether a very large language model can perform stance detection on social media without any training. Stance detection means labelling a tweet as favouring, opposing, or neutral toward a given target. Fine-tuning models with hundreds of billions of parameters is too expensive, so the authors test chain-of-thought (CoT) prompting with ChatGPT (GPT-3.5). They compare two strategies. DQA simply asks the model for the stance. StSQA first has the model explain one worked example, then supplies that question–answer pair (QAP) as a one-shot demonstration. The evaluation covers SemEval-2016, VAST and P-Stance. Prompted ChatGPT is compared with trained baselines in both zero-shot and in-domain settings, where the baselines learn from 80% of the data. StSQA performs best on most SemEval-2016 and P-Stance targets, but it trails the strongest baselines in some cases. The authors also identify three limitations: the model's own stance bias, sensitivity to how demonstrations are chosen, and targets that have several facets. The work matters because it shows training-free prompting can be a practical option for stance detection, and it marks out open problems.

## Key points
- **Datasets:** SemEval-2016 has 4,870 tweets across 6 targets. VAST is a zero-shot benchmark with 4,003 training samples. P-Stance has 21,574 political tweets about Donald Trump, Joe Biden and Bernie Sanders. The metrics are F avg and macro-F1 (F m).
- **Zero-shot on SemEval-2016:** StSQA reaches F avg of 78.9 (HC), 68.7 (FM), 61.5 (LA) and 71.6 (DT). The authors report an average gain of about 16.6% over PT-HCL. On VAST, however, StSQA scores 68.9 F m, below PT-HCL at 71.6.
- **In-domain on P-Stance:** StSQA beats Bert_Spc on every target, with F avg of 85.7 (Trump), 82.8 (Biden) and 80.8 (Bernie). On SemEval-2016, it falls well short of TPDG on the LA target (61.8 vs 74.7 F m).
- **Stance bias:** ChatGPT appears to support abortion legalization by default. As a result, it labels many neutral LA tweets as favourable, and the authors report a similar effect on VAST.
- **Demonstrations and task framing:** Adding more QAPs (tested from 1 to 5) can confuse the model's reasoning. Explanations at the semantic level work better than word-level ones. The authors also argue for fine-grained stance definitions for multi-faceted targets, for example Trump viewed as an actor versus as a politician.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [University of Science and Technology of China](https://md.research.elamur.ai/free/en/universities/university-of-science-and-technology-of-china/) — University, Hefei, China · 152 works · 114,627 citations
- [Mohamed bin Zayed University of Artificial Intelligence](https://md.research.elamur.ai/free/en/universities/mohamed-bin-zayed-university-of-artificial-intelligence/) — University, Abu Dhabi, United Arab Emirates · 145 works · 6,811 citations
- [Shenzhen Technology University](https://md.research.elamur.ai/free/en/universities/shenzhen-technology-university/) — University, Shenzhen, China · 5 works · 98 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 19,503 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 18,805 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 17,959 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 17,789 works

🧭 Topics (2): [🧮 Reasoning](https://md.research.elamur.ai/free/en/topics/reasoning/) · 1,948 works · [😊 Sentiment & emotion](https://md.research.elamur.ai/free/en/topics/sentiment-emotion/) · 1,037 works

## Cite

Cite as: Bowen Zhang, Xianghua Fu, Daijun Ding, Hu Huang, Genan Dai, Nan Yin, Yangyang Li and Liwen Jing. 2023. Investigating Chain-of-thought with ChatGPT for Stance Detection on Social Media. arXiv:2304.03087.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@misc{zhang2023investigating,
  title = {{Investigating Chain-of-thought with ChatGPT for Stance Detection on Social Media}},
  author = {Bowen Zhang, Xianghua Fu, Daijun Ding, Hu Huang, Genan Dai, Nan Yin, Yangyang Li and Liwen Jing},
  year = {2023},
  doi = {10.48550/arxiv.2304.03087},
  eprint = {2304.03087},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2304.03087}
}
```

- [← previous Findings of the Second WMT Shared Task on Sign Language Translation (WMT-SLT23)](https://md.research.elamur.ai/free/en/papers/muller-2023-findings-second-wmt-shared-task/)
- [next → Measuring Massive Multitask Chinese Understanding](https://md.research.elamur.ai/free/en/papers/zeng-2023-measuring-massive-multitask-chinese-understanding/)
- [↑ back to papers of 2023](https://md.research.elamur.ai/free/en/papers/2023/page/11/)

## References

24 references, 16 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Abeer AlDayel and Walid Magdy. 2021. [Stance detection on social media: State of the art and trends](https://md.research.elamur.ai/free/en/papers/aldayel-2020-stance-detection-social-media-state/). Information Processing & Management, 58(4):102597.
2. Emily Allaway and Kathleen Mckeown. 2020. [Zero-shot stance detection: A dataset and model using generalized topic representations](https://md.research.elamur.ai/free/en/papers/allaway-2020-zero-shot-stance-detection-dataset/). In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 8913– 8931.
3. Pranav Anand, Marilyn Walker, Rob Abbott, Jean E Fox Tree, Robeson Bowmani, and Michael Minor. 2011. [Cats rule and dogs drool!: Classifying stance in online debate](https://md.research.elamur.ai/free/en/papers/anand-2011-cats-rule-dogs-drool-classifying/). In Proceedings of the 2nd Workshop on Computational Approaches to Subjectivity and Sentiment Analysis (WASSA 2.011), pages 1–9.
4. I Augenstein, T Rocktaeschel, A Vlachos, and K Bontcheva. 2016. [Stance detection with bidirectional conditional encoding](https://md.research.elamur.ai/free/en/papers/augenstein-2016-stance-detection-bidirectional-conditional-encoding/). In Proceedings of the Conference on Empirical Methods in Natural Language Processing. Sheffield.
5. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. [BERT: pre-training of deep bidirectional transformers for language understanding](https://md.research.elamur.ai/free/en/papers/devlin-2019-bert-pre-training-deep-bidirectional/). In NAACL-HLT, pages 4171–4186. Association for Computational Linguistics.
6. Kuntal Dey, Ritvik Shrivastava, and Saroj Kaushik. 2018. Topical stance detection for twitter: A two-phase lstm model using attention. In European Conference on Information Retrieval, pages 529–536. Springer. [find](https://scholar.google.com/scholar?q=Kuntal+Dey%2C+Ritvik+Shrivastava%2C+and+Saroj+Kaushik%2E+2018%2E+Topical+stance+detection+for+twitter%3A+A+two%2Dphase+lstm+model+using+attention%2E+In+European+Conference+on+Information+Retrieval%2C+pages+529%E2%80%93536%2E+S)
7. Jiachen Du, Ruifeng Xu, Yulan He, and Lin Gui. 2017. Stance classification with target-specific neural attention networks. International Joint Conferences on Artificial Intelligence. [find](https://scholar.google.com/scholar?q=Jiachen+Du%2C+Ruifeng+Xu%2C+Yulan+He%2C+and+Lin+Gui%2E+2017%2E+Stance+classification+with+target%2Dspecific+neural+attention+networks%2E+International+Joint+Conferences+on+Artificial+Intelligence%2E)
8. Dheeru Dua, Shivanshu Gupta, Sameer Singh, and Matt Gardner. 2022. Successive prompting for decomposing complex questions. arXiv preprint arXiv:2212.04092. [find](https://scholar.google.com/scholar?q=Dheeru+Dua%2C+Shivanshu+Gupta%2C+Sameer+Singh%2C+and+Matt+Gardner%2E+2022%2E+Successive+prompting+for+decomposing+complex+questions%2E+arXiv+preprint+arXiv%3A2212%2E04092%2E)
9. Binxuan Huang, Yanglan Ou, and Kathleen M Carley. 2018. Aspect level sentiment classification with attention-over-attention neural networks. In International Conference on Social Computing, Behavioral-Cultural Modeling and Prediction and Behavior Representation in Modeling and Simulation, pages 197–… [find](https://scholar.google.com/scholar?q=Binxuan+Huang%2C+Yanglan+Ou%2C+and+Kathleen+M+Carley%2E+2018%2E+Aspect+level+sentiment+classification+with+attention%2Dover%2Dattention+neural+networks%2E+In+International+Conference+on+Social+Computing%2C+Behavioral)
10. Xinyu Jiang, Bowen Zhang, Yunming Ye, and Zhenhua Liu. 2019. A hierarchical model with recurrent convolutional neural networks for sequential sentence classification. In Natural Language Processing and Chinese Computing: 8th CCF International Conference, NLPCC 2019, Dunhuang, China, October 9–14, 20… [find](https://scholar.google.com/scholar?q=Xinyu+Jiang%2C+Bowen+Zhang%2C+Yunming+Ye%2C+and+Zhenhua+Liu%2E+2019%2E+A+hierarchical+model+with+recurrent+convolutional+neural+networks+for+sequential+sentence+classification%2E+In+Natural+Language+Processing+an)
11. Dilek Küçük and Fazli Can. 2020. Stance detection: A survey. ACM Computing Surveys (CSUR), 53(1):1– 37. [find](https://scholar.google.com/scholar?q=Dilek+K%C3%BC%C3%A7%C3%BCk+and+Fazli+Can%2E+2020%2E+Stance+detection%3A+A+survey%2E+ACM+Computing+Surveys+%28CSUR%29%2C+53%281%29%3A1%E2%80%93+37%2E)
12. Ang Li, Bin Liang, Jingqian Zhao, Bowen Zhang, Min Yang, and Ruifeng Xu. 2023. [Stance detection on social media with background knowledge](https://md.research.elamur.ai/free/en/papers/li-2023-stance-detection-social-media-background/). In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 15703–15717.
13. Yingjie Li, Tiberiu Sosea, Aditya Sawant, Ajith Jayaraman Nair, Diana Inkpen, and Cornelia Caragea. 2021. [P-stance: A large dataset for stance detection in political domain](https://md.research.elamur.ai/free/en/papers/li-2021-p-stance-large-dataset-stance/). In Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, pages 2355–2365.
14. Bin Liang, Zixiao Chen, Lin Gui, Yulan He, Min Yang, and Ruifeng Xu. 2022. Zero-shot stance detection via contrastive learning. In Proceedings of the ACM Web Conference 2022, pages 2738–2747. [find](https://scholar.google.com/scholar?q=Bin+Liang%2C+Zixiao+Chen%2C+Lin+Gui%2C+Yulan+He%2C+Min+Yang%2C+and+Ruifeng+Xu%2E+2022%2E+Zero%2Dshot+stance+detection+via+contrastive+learning%2E+In+Proceedings+of+the+ACM+Web+Conference+2022%2C+pages+2738%E2%80%932747%2E)
15. Yuxiao Lin, Yuxian Meng, Xiaofei Sun, Qinghong Han, Kun Kuang, Jiwei Li, and Fei Wu. 2021. BertGCN: Transductive text classification by combining GNN and BERT. In Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, pages 1456–1462, Online. Association for Computational Lingui… [find](https://scholar.google.com/scholar?q=Yuxiao+Lin%2C+Yuxian+Meng%2C+Xiaofei+Sun%2C+Qinghong+Han%2C+Kun+Kuang%2C+Jiwei+Li%2C+and+Fei+Wu%2E+2021%2E+BertGCN%3A+Transductive+text+classification+by+combining+GNN+and+BERT%2E+In+Findings+of+the+Association+for+Compu)
16. Saif M. Mohammad, Svetlana Kiritchenko, Parinaz Sobhani, Xiaodan Zhu, and Colin Cherry. 2016. [Semeval-2016 task 6: Detecting stance in tweets](https://md.research.elamur.ai/free/en/papers/mohammad-2016-semeval-2016-task-6-detecting/). In Proceedings of the International Workshop on Semantic Evaluation, SemEval ’16, San Diego, California.
17. Taylor Shin, Yasaman Razeghi, Robert L Logan IV, Eric Wallace, and Sameer Singh. 2020. [Autoprompt: Eliciting knowledge from language models with automatically generated prompts](https://md.research.elamur.ai/free/en/papers/shin-2020-autoprompt-eliciting-knowledge-language-models/). arXiv preprint arXiv:2010.15980.
18. Duyu Tang, Bing Qin, and Ting Liu. 2016. [Aspect level sentiment classification with deep memory network](https://md.research.elamur.ai/free/en/papers/tang-2016-aspect-level-sentiment-classification-deep/). In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, EMNLP 2016, Austin, Texas, USA, November 1-4,.
19. Marilyn Walker, Pranav Anand, Rob Abbott, and Ricky Grant. 2012. [Stance classification using dialogic properties of persuasion](https://md.research.elamur.ai/free/en/papers/walker-2012-stance-classification-dialogic-properties-persuasion/). In Proceedings of the 2012 conference of the North American chapter of the association for computational linguistics: Human language technologies, pages 592–596.
20. Jason Wei, Yi Tay, Rishi Bommasani, Colin Raffel, Barret Zoph, Sebastian Borgeaud, Dani Yogatama, Maarten Bosma, Denny Zhou, Donald Metzler, et al. 2022. [Emergent abilities of large language models](https://md.research.elamur.ai/free/en/papers/wei-2022-emergent-abilities-large-language-models/). arXiv preprint arXiv:2206.07682.
21. Chang Xu, Cecile Paris, Surya Nepal, and Ross Sparks. 2018. [Cross-target stance classification with self-attention networks](https://md.research.elamur.ai/free/en/papers/xu-2018-cross-target-stance-classification-self/). In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 778– 783.
22. Bowen Zhang, Min Yang, Xutao Li, Yunming Ye, Xiaofei Xu, and Kuai Dai. 2020. [Enhancing cross-target stance detection with transferable semantic-emotion knowledge](https://md.research.elamur.ai/free/en/papers/zhang-2020-enhancing-cross-target-stance-detection/). In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 3188–3197.
23. Chen Zhang, Qiuchi Li, and Dawei Song. 2019. [Aspect-based sentiment classification with aspect-specific graph convolutional networks](https://md.research.elamur.ai/free/en/papers/zhang-2019-aspect-based-sentiment-classification-aspect/). In EMNLP/IJCNLP (1).
24. Denny Zhou, Nathanael Schärli, Le Hou, Jason Wei, Nathan Scales, Xuezhi Wang, Dale Schuurmans, Olivier Bousquet, Quoc Le, and Ed Chi. 2022. [Least-to-most prompting enables complex reasoning in large language models](https://md.research.elamur.ai/free/en/papers/al-2022-least-most-prompting-enables-complex/). arXiv preprint arXiv:2205.10625.

*Updated 3 November 2026*