---
title: "Using Masked Language Model Probabilities of Connectives for"
description: "For each claim–premise pair, the authors build a sequence with a mask slot, in both orders (claim first or… 20 references, cited by 1 paper here."
authors: ["Regina Stodden", "Laura Kallmeyer", "Lea Kawaletz", "Heidrun Dorgeloh"]
year: 2023
venue: "Proceedings of the 10th Workshop on Argument Mining"
source: "https://aclanthology.org/2023.argmining-1.2/"
doi: "10.18653/v1/2023.argmining-1.2"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/stodden-2023-masked-language-model-probabilities-connectives/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/stodden-2023-masked-language-model-probabilities-connectives/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2023](https://md.research.elamur.ai/free/en/papers/2023/page/14/) / Using Masked Language Model Probabilities of Connectives for Stance Detection in English Discourse

# Using Masked Language Model Probabilities of Connectives for Stance Detection in English Discourse

**Authors:** Regina Stodden, [Laura Kallmeyer](https://md.research.elamur.ai/free/en/authors/laura-kallmeyer/), Lea Kawaletz and Heidrun Dorgeloh · **Year:** 2023 · **Venue:** [Proceedings of the 10th Workshop on Argument Mining](https://md.research.elamur.ai/free/en/venues/argmining/) ([Workshop on Argument Mining 2023](https://md.research.elamur.ai/free/en/venues/argmining/2023/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2023.argmining-1.2/) · [PDF](https://aclanthology.org/2023.argmining-1.2.pdf) · [DOI 10.18653/v1/2023.argmining-1.2](https://doi.org/10.18653/v1/2023.argmining-1.2) · [OpenAlex](https://openalex.org/W4389518190)

**Citations:** 0 citations (OpenAlex, 2026-10-14) · 1 in our library, of the 149,439 papers whose bibliographies we read

## Summary
The paper asks whether a premise backs or opposes a claim, judged only from the connectives a language model would place between the two. For each claim–premise pair, the authors build a sequence with a mask slot, in both orders (claim first or premise first). Several masked LMs (DistilBERT, BERT, RoBERTa, XLM-RoBERTa) then score how plausible each candidate connective is in that slot. The candidates are single-token connectives taken from DimLex-Eng whose PDTB relations point to support (causal/result) or attack (contrast/alternative). Their probabilities become a feature vector for a LightGBM classifier. The method is tested on ibmcs, perspectrum and argmin using macro F1. These features alone clearly beat the majority and random baselines and come close to fine-tuned SOTA systems. On argmin they even score higher than the reported SOTA. This suggests that an LM's implicit knowledge of discourse connectives carries real stance signal, which could complement text-embedding approaches.

## Key points
- Connective inventory:
  - The authors started from the 79 single-token connectives in DimLex-Eng, built from PDTB plus RST-SC entries.
  - They kept 12 support and 18 attack connectives (6 appear in both lists), giving 24 unique features in Group 1.
  - Connectives that the tokenizer splits into subwords (e.g., *however*) were dropped.
- Each pair is concatenated in two orders, C-LW-P and P-LW-C, because some connectives (e.g., *so*, *yet*) naturally put the premise first.
- LightGBM performed best among six classifiers tried, including SVM, random forest, MLP and XGBoost.
- Macro F1 results:
  - Combining features from all LMs (Group 1) gives 0.7467 on argmin, 0.7885 on ibmcs and 0.8314 on perspectrum.
  - The best SOTA comparison, MoLe, scores 0.6373, 0.7938 and 0.8527.
  - The majority baseline sits around 0.34.
  - RoBERTa-large is the strongest single LM.
- Ablation findings:
  - Smaller connective groups (12 or 5 features) lower the scores.
  - Removing attack connectives hurts more than removing support ones.
  - Random or "other" connective sets also perform worse, so both the number of connectives and the choice of connectives matter.

## Interesting ideas
- Use a masked LM as a cloze-style judge: compare how acceptable relation-signalling words (*because* vs *but*) are between two clauses, and treat the resulting scores as interpretable evidence for an unstated relation.
- Score each candidate in both argument orders, since connectives impose direction (*so* follows the reason, *because* introduces it), and keep both scores as separate features.
- Turn a lexicon's relation counts into a "predominance" percentage per connective, then group connectives into ambiguous vs. predominantly-one-relation tiers for analysis.
- Check that a feature set's value comes from its content rather than its size by comparing against random connective sets and an "other connectives" set of the same size.
- Drop candidates the LM would score only as subword fragments, rather than falling back to the first piece (e.g., *how* for *however*), so the probabilities stay meaningful.

## For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

- **I1** (frames): When two clauses have no explicit connective, Mova could probe candidate connectives and record which relation (cause, contrast) they support as an explainable witness for a discourse link.
- **I2** (world): When ordering events or causal links in the text world, testing both directions with directional connectives could help decide which state is cause and which is effect.
- **I3** (world): A compiled connective table with relation distributions from PDTB/DimLex-Eng would let Mova assign graded rather than all-or-nothing relation readings to *while*, *since* or *as*.
- **I4** (learn): This is exactly the kind of negative control to run when validating LM-teacher signals: same-size random probes show whether a probe set carries real information.
- **I5** (learn): When querying an LLM teacher for word-level judgments, restrict candidates to items scored as whole units to avoid misleading fragment probabilities in silver data.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Heinrich Heine University Düsseldorf](https://md.research.elamur.ai/free/en/universities/heinrich-heine-university-dusseldorf/) — University, Düsseldorf, Germany · 33 works · 1,487 citations

📚 Shelves (5): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works · [🤔 Philosophy](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/philosophy/) · 284 works

🧭 Topics (3): [English](https://md.research.elamur.ai/free/en/topics/english/) · 2,263 works · [😊 Sentiment & emotion](https://md.research.elamur.ai/free/en/topics/sentiment-emotion/) · 972 works · [🧵 Discourse analysis](https://md.research.elamur.ai/free/en/topics/discourse-analysis/) · 466 works

## Cite

Cite as: Regina Stodden, Laura Kallmeyer, Lea Kawaletz and Heidrun Dorgeloh. 2023. Using Masked Language Model Probabilities of Connectives for Stance Detection in English Discourse. Proceedings of the 10th Workshop on Argument Mining.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{stodden2023using,
  title = {{Using Masked Language Model Probabilities of Connectives for Stance Detection in English Discourse}},
  author = {Regina Stodden, Laura Kallmeyer, Lea Kawaletz and Heidrun Dorgeloh},
  booktitle = {Proceedings of the 10th Workshop on Argument Mining},
  year = {2023},
  doi = {10.18653/v1/2023.argmining-1.2},
  url = {https://doi.org/10.18653/v1/2023.argmining-1.2}
}
```

- [← previous Using Addressing Terms to Promote World-Englishes in Indonesia](https://md.research.elamur.ai/free/en/papers/wijayanti-2023-using-addressing-terms-promote-world/)
- [next → UWB at Arabic Reverse Dictionary shared task: Computing the meaning of a gloss](https://md.research.elamur.ai/free/en/papers/taylor-2023-uwb-arabic-reverse-dictionary-shared/)
- [↑ back to papers of 2023](https://md.research.elamur.ai/free/en/papers/2023/page/14/)

## References

20 references, 13 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Bethany Gray and Douglas Biber. 2014. Stance markers. In Karin Aijmer and Christoph Rühlemann, editors, Corpus Pragmatics: A Handbook, chapter 8, page 219–248. Cambridge University Press. [find](https://scholar.google.com/scholar?q=Bethany+Gray+and+Douglas+Biber%2E+2014%2E+Stance+markers%2E+In+Karin+Aijmer+and+Christoph+R%C3%BChlemann%2C+editors%2C+Corpus+Pragmatics%3A+A+Handbook%2C+chapter+8%2C+page+219%E2%80%93248%2E+Cambridge+University+Press%2E)
2. Takuya Akiba, Shotaro Sano, Toshihiko Yanase, Takeru Ohta, and Masanori Koyama. 2019. [Optuna: A next-generation hyperparameter optimization framework](https://md.research.elamur.ai/free/en/papers/akiba-2019-optuna-next-generation-hyperparameter-optimization/). In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, KDD ’19, page 2623–2631, New York, NY, USA. Association for Computing Machinery.
3. Momchil Hardalov, Arnav Arora, Preslav Nakov, and Isabelle Augenstein. 2021. [Cross-domain label-adaptive stance detection](https://md.research.elamur.ai/free/en/papers/hardalov-2021-cross-domain-label-adaptive-stance/). In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 9011–9028, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics.
4. Roy Bar-Haim, Indrajit Bhattacharya, Francesco Din-uzzo, Amrita Saha, and Noam Slonim. 2017. [Stance classification of context-dependent claims](https://md.research.elamur.ai/free/en/papers/haim-2017-stance-classification-context-dependent-claims/). In Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 1, Long Papers, pages 251–261, Valencia, Spain. Association for Computational Linguistics.
5. Freya Hewett, Roshan Prakash Rane, Nina Harlacher, and Manfred Stede. 2019. [The utility of discourse parsing features for predicting argumentation structure](https://md.research.elamur.ai/free/en/papers/hewett-2019-utility-discourse-parsing-features-predicting/). In Proceedings of the 6th Workshop on Argument Mining, pages 98–103, Florence, Italy. Association for Computational Linguistics.
6. Sihao Chen, Daniel Khashabi, Wenpeng Yin, Chris Callison-Burch, and Dan Roth. 2019. [Seeing things from a different angle:discovering diverse perspectives about claims](https://md.research.elamur.ai/free/en/papers/chen-2019-seeing-things-different-angle-discovering/). In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 542–557, Minneapolis, Minnesota. Association for Computational Linguistics. [↗](https://doi.org/10.48550/arxiv.1906.03538)
7. Yudai Kishimoto, Yugo Murawaki, and Sadao Kurohashi. 2020. [Adapting BERT to implicit discourse relation classification with a focus on discourse connectives](https://md.research.elamur.ai/free/en/papers/kishimoto-2020-adapting-bert-implicit-discourse-relation/). In Proceedings of the Twelfth Language Resources and Evaluation Conference, pages 1152– 1158, Marseille, France. European Language Resources Association.
8. Murathan Kurfalı and Robert Östling. 2021. [Let’s be explicit about that: Distant supervision for implicit discourse relation classification via connective prediction](https://md.research.elamur.ai/free/en/papers/kurfali-2021-let-s-be-explicit-about/). In Proceedings of the 1st Workshop on Understanding Implicit and Underspecified Language, pages 1–10, Online. Association for Computational Linguistics. [↗](https://doi.org/10.48448/h3vg-r297)
9. Alexis Conneau, Kartikay Khandelwal, Naman Goyal, Vishrav Chaudhary, Guillaume Wenzek, Francisco Guzmán, Edouard Grave, Myle Ott, Luke Zettlemoyer, and Veselin Stoyanov. 2020. Unsupervised cross-lingual representation learning at scale. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 8440– 8451, Online. Association for Computational Linguistics. [↗](https://doi.org/10.18653/v1/2020.acl-main.747)
10. Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. [Roberta: A robustly optimized BERT pretraining approach](https://md.research.elamur.ai/free/en/papers/liu-2019-roberta-robustly-optimized-bert-pretraining/). CoRR, abs/1907.11692. [↗](https://arxiv.org/abs/1907.11692)
11. Debopam Das, Tatjana Scheffler, Peter Bourgonje, and Manfred Stede. 2018. [Constructing a lexicon of English discourse connectives](https://md.research.elamur.ai/free/en/papers/das-2018-constructing-lexicon-english-discourse-connectives/). In Proceedings of the 19th Annual SIGdial Meeting on Discourse and Dialogue, pages 360–365, Melbourne, Australia. Association for Computational Linguistics. [↗](https://doi.org/10.18653/v1/w18-5042)
12. Rashmi Prasad, Nikhil Dinesh, Alan Lee, Eleni Milt-sakaki, Livio Robaldo, Aravind Joshi, and Bonnie Webber. 2008. [The Penn Discourse TreeBank 2.0](https://md.research.elamur.ai/free/en/papers/prasad-2008-penn-discourse-treebank-2-0/). In Proceedings of the Sixth International Conference on Language Resources and Evaluation (LREC’08), Marrakech, Morocco. European Language Resources Association (ELRA). [↗](https://doi.org/10.63317/55sr8v9kexr7)
13. Debopam Das and Maite Taboada. 2018. RST Signalling Corpus: A Corpus of Signals of Coherence Relations. Language Resources and Evaluation, 52(1):149–184. [↗](https://doi.org/10.1007/s10579-017-9383-x)
14. Ted J.M. Sanders, Vera Demberg, Jet Hoek, Merel C.J. Scholman, Fatemeh Torabi Asr, Sandrine Zufferey, and Jacqueline Evers-Vermeul. 2021. Unifying dimensions in coherence relations: How various annotation frameworks are related. Corpus Linguistics and Linguistic Theory, 17(1):1–71. [find](https://scholar.google.com/scholar?q=Ted+J%2EM%2E+Sanders%2C+Vera+Demberg%2C+Jet+Hoek%2C+Merel+C%2EJ%2E+Scholman%2C+Fatemeh+Torabi+Asr%2C+Sandrine+Zufferey%2C+and+Jacqueline+Evers%2DVermeul%2E+2021%2E+Unifying+dimensions+in+coherence+relations%3A+How+various+annota)
15. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. [BERT: Pre-training of deep bidirectional transformers for language understanding](https://md.research.elamur.ai/free/en/papers/devlin-2019-bert-pre-training-deep-bidirectional/). In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 4171–4186, Minneapolis, Minnesota. Association for Computational Linguistics. [↗](https://doi.org/10.18653/v1/n19-1423)
16. Victor Sanh, Lysandre Debut, Julien Chaumond, and Thomas Wolf. 2019. DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter. In 5th Workshop on Energy Efficient Machine Learning and Cognitive Computing @ NeurIPS 2019. [find](https://scholar.google.com/scholar?q=Victor+Sanh%2C+Lysandre+Debut%2C+Julien+Chaumond%2C+and+Thomas+Wolf%2E+2019%2E+DistilBERT%2C+a+distilled+version+of+BERT%3A+smaller%2C+faster%2C+cheaper+and+lighter%2E+In+5th+Workshop+on+Energy+Efficient+Machine+Learning)
17. Heidrun Dorgeloh and Anja Wanner. 2022. Discourse Syntax: English Grammar Beyond the Sentence. Cambridge University Press. [find](https://scholar.google.com/scholar?q=Heidrun+Dorgeloh+and+Anja+Wanner%2E+2022%2E+Discourse+Syntax%3A+English+Grammar+Beyond+the+Sentence%2E+Cambridge+University+Press%2E)
18. Benjamin Schiller, Johannes Daxenberger, and Iryna Gurevych. 2021. Stance detection benchmark: How robust is your stance detection? Intelligenz, 35(3-4):329–341. [↗](https://doi.org/10.1007/s13218-021-00714-w)
19. Hao Zhou, Man Lan, Yuanbin Wu, Yuefeng Chen, and Meirong Ma. 2022. [Prompt-based connective prediction method for fine-grained implicit discourse relation recognition](https://md.research.elamur.ai/free/en/papers/zhou-2022-prompt-based-connective-prediction-method/). In Findings of the Association for Computational Linguistics: EMNLP 2022, pages 3848–3858, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics. KI - Künstliche Christian Stab and Iryna Gurevych. 2017. Parsing Argumentation Structures in Persuasive Essays. Computational Linguistics, 43(3):619–659.
20. Christian Stab, Tristan Miller, Benjamin Schiller, Pranav Rai, and Iryna Gurevych. 2018. [Cross-topic argument mining from heterogeneous sources](https://md.research.elamur.ai/free/en/papers/stab-2018-cross-topic-argument-mining-heterogeneous/). In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 3664– 3674, Brussels, Belgium. Association for Computational Linguistics.

*Updated 1 November 2026*