---
title: "Subjectivity Detection in English News using Large Language"
description: "The paper asks whether language models can reliably tell opinionated sentences from factual reporting… 38 references, cited by 3 papers here."
authors: ["Mohammad Shokri", "Vivek Sharma", "Elena Filatova", "Shweta Jain", "Sarah Levitan"]
year: 2024
venue: "Proceedings of the 14th Workshop on Computational Approaches to Subjectivity, Sentiment, & Social Media Analysis"
source: "https://aclanthology.org/2024.wassa-1.17/"
doi: "10.18653/v1/2024.wassa-1.17"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/shokri-2024-subjectivity-detection-english-news-large/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/shokri-2024-subjectivity-detection-english-news-large/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2024](https://md.research.elamur.ai/free/en/papers/2024/page/10/) / Subjectivity Detection in English News using Large Language Models

# Subjectivity Detection in English News using Large Language Models

**Authors:** [Mohammad Shokri](https://md.research.elamur.ai/free/en/authors/mohammad-shokri/), [Vivek Sharma](https://md.research.elamur.ai/free/en/authors/vivek-sharma/), [Elena Filatova](https://md.research.elamur.ai/free/en/authors/elena-filatova/), [Shweta Jain](https://md.research.elamur.ai/free/en/authors/shweta-jain/) and [Sarah Levitan](https://md.research.elamur.ai/free/en/authors/sarah-ita-levitan/) · **Year:** 2024 · **Venue:** [Proceedings of the 14th Workshop on Computational Approaches to Subjectivity, Sentiment, & Social Media Analysis](https://md.research.elamur.ai/free/en/venues/wassa/) ([Workshop on Computational Approaches to Subjectivity, Sentiment and Social Media Analysis 2024](https://md.research.elamur.ai/free/en/venues/wassa/2024/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2024.wassa-1.17/) · [PDF](https://aclanthology.org/2024.wassa-1.17.pdf) · [DOI 10.18653/v1/2024.wassa-1.17](https://doi.org/10.18653/v1/2024.wassa-1.17) · [OpenAlex](https://openalex.org/W4402669981)

**Citations:** 4 citations (OpenAlex, 2026-10-14) · [3](https://md.research.elamur.ai/free/en/papers/shokri-2024-subjectivity-detection-english-news-large/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
The paper asks whether language models can reliably tell opinionated sentences from factual reporting in news, and whether that ability survives a move to news from different sources and topics. The authors use three English sentence-level corpora: the news portion of MPQA, a political-news set (News-1), and a crime and COVID-19 set from UK outlets (News-2). These give a distribution shift that stays inside the news domain. They compare several approaches:

- a logistic-regression baseline on lexical and lexical-richness features;
- fine-tuned encoder models, each trained on one corpus and tested on the other two;
- an entailment reformulation using RoBERTa-large pre-trained on MNLI;
- GPT-3.5, GPT-4 and Gemini under standard in-context learning, chain-of-thought, and a simple prompt of their own.

Their prompt beats both standard ICL and CoT. Standard ICL proves fragile because results depend heavily on which demonstrations are chosen. An error analysis using sentiment scores and MPQA's nested opinion-holder annotations shows where the models fail. This matters for fact-checking, because many publishers do not label opinion pieces consistently.

## Key points
- MPQA v3 was restricted to news sentences of at least 5 words. This leaves 1,707 sentences: 954 subjective and 753 objective.
- News-1 (Antici et al., 2023) has 1,049 sentences from 23 articles across 8 political outlets: 638 objective and 411 subjective.
- News-2 (Savinova and Del Prado, 2023) keeps 2,973 news sentences (1,013 subjective, 1,960 objective) after Facebook posts were dropped.
  - Its labels are continuous in [0, 1], taken as the mean of 7-point human ratings.
  - Only part of it was hand-labeled; the remainder was labeled by a model.
- The feature baseline combines news-specific cues from Krüger et al. (2017) with 9 lexical richness measures (McCarthy and Jarvis, 2007). Fine-tuned models were evaluated strictly cross-corpus as OOD.
- In the error analysis:
  - GPT-4 recalls more subjective sentences than GPT-3.5 and Gemini, which lean toward precision. The authors give this difference as a reason an ensemble helps.
  - GPT-4's missed subjective sentences are mostly neutral in sentiment.

## Interesting ideas
- Train on one corpus and test on the others from the same domain, which exposes overfitting that an in-corpus split would hide.
- Treat subjectivity as graded rather than binary, using averaged annotator ratings, because the mid-range sentences are the ones people disagree on.
- Recast a classification task as premise–hypothesis entailment so an NLI-trained model can reuse its inference skills with few labels.
- Diagnose errors by slicing the misses along an independent signal (sentiment intensity), which revealed that models catch loud opinions but miss quiet, neutral-toned ones.
- Use nested opinion holders (writer → China → U.S. report) to explain why a sentence with no obvious opinion words is still subjective.

## For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

- **I1** (learn): Evaluate parsers and frame extractors trained on one treebank or genre against held-out genres (news vs. fables vs. school texts), and treat in-genre scores as optimistic.
- **I2** (frames): Speech-act and stance labels in prag could carry a confidence or degree, so the world model can mark mid-scale claims as uncertain opinions instead of forcing a fact/opinion choice.
- **I3** (learn): Silver labels for frame properties (e.g. "the writer judges X") could be checked by an entailment witness alongside the parsers, giving another independent vote.
- **I4** (learn): Slice evaluation failures by orthogonal features such as sentence length, negation or attachment depth, to find systematic blind spots rather than just an average score.
- **I5** (frames): Frames could record a chain of sources for each attitude, with the writer as the default outer source, so the world of the text attributes beliefs to the right holder and answers stay explainable.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [City University of New York](https://md.research.elamur.ai/free/en/universities/city-university-of-new-york/) — University, New York, United States · 17 works · 2,564 citations
- [Hunter College](https://md.research.elamur.ai/free/en/universities/hunter-college/) — University, New York, United States · 9 works · 1,245 citations
- [The Graduate Center, CUNY](https://md.research.elamur.ai/free/en/universities/the-graduate-center-cuny/) — University, New York, United States · 13 works · 207 citations

📚 Shelves (6): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works · [🤝 Social Sciences](https://md.research.elamur.ai/free/en/shelves/social-sciences/) · 1,772 works · [🤝 General Social Sciences](https://md.research.elamur.ai/free/en/shelves/social-sciences/general-social-sciences/) · 227 works

🧭 Topics (1): [English](https://md.research.elamur.ai/free/en/topics/english/) · 2,263 works

## Cite

Cite as: Mohammad Shokri, Vivek Sharma, Elena Filatova, Shweta Jain and Sarah Levitan. 2024. Subjectivity Detection in English News using Large Language Models. Proceedings of the 14th Workshop on Computational Approaches to Subjectivity, Sentiment, & Social Media Analysis.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{shokri2024subjectivity,
  title = {{Subjectivity Detection in English News using Large Language Models}},
  author = {Mohammad Shokri, Vivek Sharma, Elena Filatova, Shweta Jain and Sarah Levitan},
  booktitle = {Proceedings of the 14th Workshop on Computational Approaches to Subjectivity, Sentiment, \& Social Media Analysis},
  year = {2024},
  doi = {10.18653/v1/2024.wassa-1.17},
  url = {https://doi.org/10.18653/v1/2024.wassa-1.17}
}
```

- [← previous Shared Imagination: LLMs Hallucinate Alike](https://md.research.elamur.ai/free/en/papers/zhou-2024-shared-imagination-llms-hallucinate-alike/)
- [next → ThatiAR: Subjectivity Detection in Arabic News Sentences](https://md.research.elamur.ai/free/en/papers/suwaileh-2024-thatiar-subjectivity-detection-arabic-news/)
- [↑ back to papers of 2024](https://md.research.elamur.ai/free/en/papers/2024/page/10/)

## References

38 references, 22 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Francesco Antici, Andrea Galassi, Federico Ruggeri, Katerina Korre, Arianna Muti, Alessandra Bardi, Alice Fedotova, and Alberto Barrón-Cedeño. 2023. [A corpus for sentence-level subjectivity detection on english news articles](https://md.research.elamur.ai/free/en/papers/antici-2024-corpus-sentence-level-subjectivity-detection/). arXiv preprint arXiv:2305.18034. [↗](https://arxiv.org/abs/2305.18034)
2. Jonathan S Blake et al. 2019. News in a digital age: Comparing the presentation of news information over time and across media platforms. Rand Corporation. [find](https://scholar.google.com/scholar?q=Jonathan+S+Blake+et+al%2E+2019%2E+News+in+a+digital+age%3A+Comparing+the+presentation+of+news+information+over+time+and+across+media+platforms%2E+Rand+Corporation%2E)
3. Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. 2020. [Language models are few-shot learners](https://md.research.elamur.ai/free/en/papers/brown-2020-language-models-few-shot-learners/). Advances in neural information processing systems, 33:1877–1901.
4. Sébastien Bubeck, Varun Chandrasekaran, Ronen Eldan, Johannes Gehrke, Eric Horvitz, Ece Kamar, Peter Lee, Yin Tat Lee, Yuanzhi Li, Scott Lundberg, et al. 2023. [Sparks of artificial general intelligence: Early experiments with gpt-4](https://md.research.elamur.ai/free/en/papers/bubeck-2023-sparks-artificial-general-intelligence-early/). arXiv preprint arXiv:2303.12712. [↗](https://arxiv.org/abs/2303.12712)
5. Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung, Charles Sutton, Sebastian Gehrmann, et al. 2023. [Palm: Scaling language modeling with pathways](https://md.research.elamur.ai/free/en/papers/chowdhery-2022-palm-scaling-language-modeling-pathways/). Journal of Machine Learning Research, 24(240):1–113. [↗](https://doi.org/10.48550/arxiv.2204.02311)
6. Aida Mostafazadeh Davani, Mark Díaz, and Vinodkumar Prabhakaran. 2022. [Dealing with disagreements: Looking beyond the majority vote in subjective annotations](https://md.research.elamur.ai/free/en/papers/davani-2022-dealing-disagreements-looking-beyond-majority/). Transactions of the Association for Computational Linguistics, 10:92–110. In Proceedings of the 5th international conference on Soft computing as transdisciplinary science and technology, pages 211–217. [↗](https://doi.org/10.1162/tacl_a_00449)
7. Laurie Beth Harris. 2017. Helping readers tell the difference between news and opinion: 7 good questions with duke reporters’ lab’s rebecca iannucci. [find](https://scholar.google.com/scholar?q=Laurie+Beth+Harris%2E+2017%2E+Helping+readers+tell+the+difference+between+news+and+opinion%3A+7+good+questions+with+duke+reporters%E2%80%99+lab%E2%80%99s+rebecca+iannucci%2E)
8. Ozan Irsoy and Claire Cardie. 2014. Opinion mining with deep recurrent neural networks. In Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP), pages 720–728. [find](https://scholar.google.com/scholar?q=Ozan+Irsoy+and+Claire+Cardie%2E+2014%2E+Opinion+mining+with+deep+recurrent+neural+networks%2E+In+Proceedings+of+the+2014+conference+on+empirical+methods+in+natural+language+processing+%28EMNLP%29%2C+pages+720%E2%80%93728)
9. Caio Libanio Melo Jeronimo, Leandro Balby Marinho, Claudio EC Campelo, Adriano Veloso, and Allan Sales da Costa Melo. 2019. Fake news classification based on subjective language. In Proceedings of the 21st International Conference on Information Integration and Web-based Applications & Services, pages 15–24. [find](https://scholar.google.com/scholar?q=Caio+Libanio+Melo+Jeronimo%2C+Leandro+Balby+Marinho%2C+Claudio+EC+Campelo%2C+Adriano+Veloso%2C+and+Allan+Sales+da+Costa+Melo%2E+2019%2E+Fake+news+classification+based+on+subjective+language%2E+In+Proceedings+of+the)
10. Soo-Min Kim and Eduard Hovy. 2005. Automatic detection of opinion bearing words and sentences. In Companion Volume to the Proceedings of Conference including Posters/Demos and tutorial abstracts. [find](https://scholar.google.com/scholar?q=Soo%2DMin+Kim+and+Eduard+Hovy%2E+2005%2E+Automatic+detection+of+opinion+bearing+words+and+sentences%2E+In+Companion+Volume+to+the+Proceedings+of+Conference+including+Posters%2FDemos+and+tutorial+abstracts%2E)
11. Takeshi Kojima, Shixiang Shane Gu, Machel Reid, Yutaka Matsuo, and Yusuke Iwasawa. 2022. [Large language models are zero-shot reasoners](https://md.research.elamur.ai/free/en/papers/kojima-2022-large-language-models-zero-shot/). Advances in neural information processing systems, 35:22199– 22213. [↗](https://doi.org/10.52202/068431-1613)
12. Katarina R Krüger, Anna Lukowiak, Jonathan Sonntag, Saskia Warzecha, and Manfred Stede. 2017. Classifying news versus opinions in newspapers: Linguistic features for domain independence. Natural Language Engineering, 23(5):687–707. [↗](https://doi.org/10.1017/s1351324917000043)
13. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. [Bert: Pre-training of deep bidirectional transformers for language understanding](https://md.research.elamur.ai/free/en/papers/devlin-2019-bert-pre-training-deep-bidirectional/). arXiv preprint arXiv:1810.04805. [↗](https://arxiv.org/abs/1810.04805)
14. Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. [Roberta: A robustly optimized bert pretraining approach](https://md.research.elamur.ai/free/en/papers/liu-2019-roberta-robustly-optimized-bert-pretraining/). arXiv preprint arXiv:1907.11692. [↗](https://arxiv.org/abs/1907.11692)
15. Qingxiu Dong, Lei Li, Damai Dai, Ce Zheng, Zhiyong Wu, Baobao Chang, Xu Sun, Jingjing Xu, and Zhifang Sui. 2022. [A survey on in-context learning](https://md.research.elamur.ai/free/en/papers/dong-2023-survey-context-learning/). arXiv preprint arXiv:2301.00234. [↗](https://arxiv.org/abs/2301.00234)
16. Daniel Loureiro, Francesco Barbieri, Leonardo Neves, Luis Espinosa Anke, and Jose Camacho-Collados. 2022. [Timelms: Diachronic language models from twitter](https://md.research.elamur.ai/free/en/papers/loureiro-2022-timelms-diachronic-language-models-twitter/). arXiv preprint arXiv:2202.03829. [↗](https://arxiv.org/abs/2202.03829)
17. Andrew B Goldberg and Xiaojin Zhu. 2006. Seeing stars when there aren’t many stars: Graph-based semi-supervised learning for sentiment categorization. In Proceedings of TextGraphs: The first workshop on graph based methods for natural language processing, pages 45–52. [find](https://scholar.google.com/scholar?q=Andrew+B+Goldberg+and+Xiaojin+Zhu%2E+2006%2E+Seeing+stars+when+there+aren%E2%80%99t+many+stars%3A+Graph%2Dbased+semi%2Dsupervised+learning+for+sentiment+categorization%2E+In+Proceedings+of+TextGraphs%3A+The+first+workshop+)
18. Scott M Lundberg and Su-In Lee. 2017. [A unified approach to interpreting model predictions](https://md.research.elamur.ai/free/en/papers/lundberg-2017-unified-approach-interpreting-model-predictions/). Advances in neural information processing systems, 30. [↗](https://doi.org/10.48550/arxiv.1705.07874)
19. Andrew Gordon, Abe Kazemzadeh, Anish Nair, and Milena Petrova. 2003. [Recognizing expressions of commonsense psychology in english text](https://md.research.elamur.ai/free/en/papers/gordon-2003-recognizing-expressions-commonsense-psychology-english/). In Proceedings of the 41st annual meeting of the association for computational linguistics, pages 208–215.
20. Ali Harb, Michel Plantié, Gerard Dray, Mathieu Roche, François Trousset, and Pascal Poncelet. 2008. Web opinion mining: How to extract opinions from blogs? [find](https://scholar.google.com/scholar?q=Ali+Harb%2C+Michel+Planti%C3%A9%2C+Gerard+Dray%2C+Mathieu+Roche%2C+Fran%C3%A7ois+Trousset%2C+and+Pascal+Poncelet%2E+2008%2E+Web+opinion+mining%3A+How+to+extract+opinions+from+blogs%3F)
21. Yiwei Luo, Dallas Card, and Dan Jurafsky. 2020. Detecting stance in media on global warming. arXiv preprint arXiv:2010.15149. [↗](https://arxiv.org/abs/2010.15149)
22. Bryan McCann, Nitish Shirish Keskar, Caiming Xiong, and Richard Socher. 2018. [The natural language decathlon: Multitask learning as question answering](https://md.research.elamur.ai/free/en/papers/mccann-2018-natural-language-decathlon-multitask-learning/). arXiv preprint arXiv:1806.08730. [↗](https://arxiv.org/abs/1806.08730)
23. Philip M McCarthy and Scott Jarvis. 2007. vocd: A theoretical and empirical evaluation. Language Testing, 24(4):459–488. [↗](https://doi.org/10.1177/0265532207080767)
24. Sewon Min, Xinxi Lyu, Ari Holtzman, Mikel Artetxe, Mike Lewis, Hannaneh Hajishirzi, and Luke Zettlemoyer. 2022. [Rethinking the role of demonstrations: What makes in-context learning work?](https://md.research.elamur.ai/free/en/papers/min-2022-rethinking-role-demonstrations-what-makes/) arXiv preprint arXiv:2202.12837. [↗](https://arxiv.org/abs/2202.12837)
25. Alec Radford, Karthik Narasimhan, Tim Salimans, Ilya Sutskever, et al. 2018. Improving language understanding by generative pre-training. [find](https://scholar.google.com/scholar?q=Alec+Radford%2C+Karthik+Narasimhan%2C+Tim+Salimans%2C+Ilya+Sutskever%2C+et+al%2E+2018%2E+Improving+language+understanding+by+generative+pre%2Dtraining%2E)
26. Ellen Riloff and Janyce Wiebe. 2003. [Learning extraction patterns for subjective expressions](https://md.research.elamur.ai/free/en/papers/riloff-2003-learning-extraction-patterns-subjective-expressions/). In Proceedings of the 2003 conference on Empirical methods in natural language processing, pages 105–112.
27. Ellen Riloff, Janyce Wiebe, and William Phillips. 2005. Exploiting subjectivity classification to improve information extraction. In AAAI, pages 1106–1111. [find](https://scholar.google.com/scholar?q=Ellen+Riloff%2C+Janyce+Wiebe%2C+and+William+Phillips%2E+2005%2E+Exploiting+subjectivity+classification+to+improve+information+extraction%2E+In+AAAI%2C+pages+1106%E2%80%931111%2E)
28. Hong Yu and Vasileios Hatzivassiloglou. 2003. [Towards answering opinion questions: Separating facts from opinions and identifying the polarity of opinion sentences](https://md.research.elamur.ai/free/en/papers/yu-2003-answering-opinion-questions-separating-facts/). In Proceedings of the 2003 conference on Empirical methods in natural language processing, pages 129–136. [↗](https://doi.org/10.7916/d88w3nn0)
29. Wei Zhang, Clement Yu, and Weiyi Meng. 2007. Opinion retrieval from blogs. In Proceedings of the sixteenth ACM conference on Conference on information and knowledge management, pages 831–840. [find](https://scholar.google.com/scholar?q=Wei+Zhang%2C+Clement+Yu%2C+and+Weiyi+Meng%2E+2007%2E+Opinion+retrieval+from+blogs%2E+In+Proceedings+of+the+sixteenth+ACM+conference+on+Conference+on+information+and+knowledge+management%2C+pages+831%E2%80%93840%2E)
30. Zihao Zhao, Eric Wallace, Shi Feng, Dan Klein, and Sameer Singh. 2021. [Calibrate before use: Improving few-shot performance of language models](https://md.research.elamur.ai/free/en/papers/zhao-2021-calibrate-before-use-improving-few/). In International Conference on Machine Learning, pages 12697–12706. PMLR. [↗](https://doi.org/10.48550/arxiv.2102.09690)
31. Elena Savinova and Fermin Moscoso Del Prado. 2023. Analyzing subjectivity using a transformer-based regressor trained on naïve speakers’ judgements. In Proceedings of the 13th Workshop on Computational Approaches to Subjectivity, Sentiment, & Social Media Analysis, pages 305–314. [find](https://scholar.google.com/scholar?q=Elena+Savinova+and+Fermin+Moscoso+Del+Prado%2E+2023%2E+Analyzing+subjectivity+using+a+transformer%2Dbased+regressor+trained+on+na%C3%AFve+speakers%E2%80%99+judgements%2E+In+Proceedings+of+the+13th+Workshop+on+Computationa)
32. Gemini Team, Rohan Anil, Sebastian Borgeaud, Yonghui Wu, Jean-Baptiste Alayrac, Jiahui Yu, Radu Soricut, Johan Schalkwyk, Andrew M Dai, Anja Hauth, et al. 2023. [Gemini: a family of highly capable multimodal models](https://md.research.elamur.ai/free/en/papers/gemini-team-2023-gemini-family-highly-capable-multimodal/). arXiv preprint arXiv:2312.11805. [↗](https://arxiv.org/abs/2312.11805)
33. Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. 2023. [Llama 2: Open foundation and fine-tuned chat models](https://md.research.elamur.ai/free/en/papers/touvron-2023-llama-2-open-foundation-fine/). arXiv preprint arXiv:2307.09288. [↗](https://arxiv.org/abs/2307.09288)
34. Sinong Wang, Han Fang, Madian Khabsa, Hanzi Mao, and Hao Ma. 2021. Entailment as few-shot learner. arXiv preprint arXiv:2104.14690. [↗](https://arxiv.org/abs/2104.14690)
35. Xinyi Wang, Wanrong Zhu, and William Yang Wang. 2023. [Large language models are implicitly topic models: Explaining and finding good demonstrations for in-context learning](https://md.research.elamur.ai/free/en/papers/wang-2023-large-language-models-latent-variable/). arXiv preprint arXiv:2301.11916, page 3. [↗](https://arxiv.org/abs/2301.11916)
36. Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. 2022. [Chain-of-thought prompting elicits reasoning in large language models](https://md.research.elamur.ai/free/en/papers/wei-2022-chain-thought-prompting-elicits-reasoning/). Advances in Neural Information Processing Systems, 35:24824–24837.
37. Janyce Wiebe, Theresa Wilson, and Claire Cardie. 2005. Annotating expressions of opinions and emotions in language. Language resources and evaluation, 39:165–210. [↗](https://doi.org/10.1007/s10579-005-7880-9)
38. Adina Williams, Nikita Nangia, and Samuel R Bowman. 2017. [A broad-coverage challenge corpus for sentence understanding through inference](https://md.research.elamur.ai/free/en/papers/williams-2018-broad-coverage-challenge-corpus-sentence/). arXiv preprint arXiv:1704.05426. [↗](https://arxiv.org/abs/1704.05426)

*Updated 1 November 2026*