---
title: "Classification and Clustering of Arguments — Reimers 2019"
description: "The paper addresses open-domain argument search: retrieving arguments about arbitrary, previously unseen topics from heterogeneous sources and then…"
authors: ["Nils Reimers", "Benjamin Schiller", "Tilman Beck", "Johannes Daxenberger", "Christian Stab", "Iryna Gurevych"]
year: 2019
venue: "Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics"
source: "https://aclanthology.org/P19-1054/"
doi: "10.18653/v1/p19-1054"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/reimers-2019-classification-clustering-arguments-contextualized-word/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/reimers-2019-classification-clustering-arguments-contextualized-word/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2019](https://md.research.elamur.ai/free/en/papers/2019/) / Classification and Clustering of Arguments with Contextualized Word Embeddings

# Classification and Clustering of Arguments with Contextualized Word Embeddings

**Authors:** [Nils Reimers](https://md.research.elamur.ai/free/en/authors/nils-reimers/), [Benjamin Schiller](https://md.research.elamur.ai/free/en/authors/benjamin-schiller/), [Tilman Beck](https://md.research.elamur.ai/free/en/authors/tilman-beck/), [Johannes Daxenberger](https://md.research.elamur.ai/free/en/authors/johannes-daxenberger/), [Christian Stab](https://md.research.elamur.ai/free/en/authors/christian-stab/), [Iryna Gurevych](https://md.research.elamur.ai/free/en/authors/iryna-gurevych/) · **Year:** 2019 · **Venue:** Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/P19-1054/) · [PDF](https://aclanthology.org/P19-1054.pdf) · [DOI 10.18653/v1/p19-1054](https://doi.org/10.18653/v1/p19-1054) · [OpenAlex](https://openalex.org/W2951281253)

**Citations:** 152 citations (openalex-snapshot · 2026-10-14) · [87](#cited-by) in our library, of the 140884 papers whose bibliographies we read

## Summary
The paper addresses open-domain argument search: retrieving arguments about arbitrary, previously unseen topics from heterogeneous sources and then grouping arguments that make the same point. The authors split this into two steps and evaluate each separately, because no end-to-end benchmark exists. The first step is topic-dependent sentence classification. The second is argument similarity and clustering. They test ELMo and BERT in cross-topic settings on four datasets. Fine-tuning BERT with the topic supplied as input substantially beats earlier systems on the UKP and IBM corpora. For clustering, they build a new resource, the UKP ASPECT Corpus. It consists of noisy argument pairs drawn from the ArgumenText search engine. A supervised BERT similarity model trained on little data clearly outperforms unsupervised embedding baselines, which perform poorly on this task. Adding agglomerative clustering lowers performance, because a single argument can cover several aspects and so does not fit a strict partition. The work shows that open-domain argument search is feasible. It also argues that overlapping clustering methods and more realistic datasets are needed. Code and models are released on GitHub.

## Key points
- **Classification results:** BERT improves the state of the art by 20.8 pp on the UKP Sentential Argument Mining Corpus (400 documents, 25,492 sentences, 8 topics, pro/con/no-argument labels). It improves by 7.4 pp on IBM Debater: Evidence Sentences (118 topics). For pro-arguments, F1 rises from .27 to .53. Feeding topic information into BERT accounts for about 13 pp of this gain.
- **UKP ASPECT Corpus:** This new resource contains 3,595 argument pairs over 28 technology and society topics, roughly 130 pairs per topic. Crowd workers labelled each pair with one of three similarity levels. About 16% of the pairs turned out to be noisy or off-topic.
- **Clustering results:**
  - On UKP ASPECT, fine-tuned BERT similarity beats strong baselines by 7.8 pp. Its score for dissimilar pairs matches human performance. Its F-score for similar pairs is .67, against .75 for humans.
  - On the AFS Corpus (6,000 pairs, 3 topics, 0–5 scale), BERT surpasses the SVR model of Misra et al. (2016). The abstract reports a gain of 12.3 pp, while the results section cites about 11 pp. In a cross-topic setup, the Spearman correlation falls to about .57.
- **Unsupervised baselines:** Averaged GloVe, ELMo, BERT, InferSent and Tf-Idf embeddings all perform weakly. Averaging ELMo or BERT embeddings gives no gain over GloVe.
- **Effect of clustering:** Combining the similarity scores with agglomerative clustering costs up to 7.64 pp. Transitivity is violated in 21.9% of cases (376 of 1,714), which argues for overlapping rather than strict-partition clustering.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

Fields: [Natural language processing](https://md.research.elamur.ai/free/en/shelves/nlp/)

## Cited by

Cited by

87 papers of our library cite it (of the 140884 whose bibliographies we read); 3 of them have a page here, 84 not yet.

- [SGPT: GPT Sentence Embeddings for Semantic Search](https://md.research.elamur.ai/free/en/papers/muennighoff-2022-sgpt-gpt-sentence-embeddings-semantic/)
- [Is Stance Detection Topic-Independent and Cross-topic Generalizable? - A Reproduction Study](https://md.research.elamur.ai/free/en/papers/reuver-2021-is-stance-detection-topic-independent/)
- [An Unsupervised Sentence Embedding Method by Mutual Information Maximization](https://md.research.elamur.ai/free/en/papers/zhang-2020-unsupervised-sentence-embedding-method-mutual/)

## Cite

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{reimers2019classification,
  title = {{Classification and Clustering of Arguments with Contextualized Word Embeddings}},
  author = {Reimers, Nils and Schiller, Benjamin and Beck, Tilman and Daxenberger, Johannes and Stab, Christian and Gurevych, Iryna},
  booktitle = {Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics},
  year = {2019},
  doi = {10.18653/v1/p19-1054},
  url = {https://doi.org/10.18653/v1/p19-1054}
}
```

- [← previous CITE: A Corpus of Image-Text Discourse Relations](https://md.research.elamur.ai/free/en/papers/alikhani-2019-cite-corpus-image-text-discourse/)
- [next → ClinicalBERT: Modeling Clinical Notes and Predicting Hospital Readmission](https://md.research.elamur.ai/free/en/papers/huang-2019-clinicalbert-modeling-clinical-notes-predicting/)
- [↑ back to papers of 2019](https://md.research.elamur.ai/free/en/papers/2019/)

## References

22 references, 8 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Filip Boltužić and Jan Šnajder. 2015. Identifying prominent arguments in online debates using semantic textual similarity. In Proceedings of the 2nd Workshop on Argumentation Mining, pages 110– 115. [find](https://scholar.google.com/scholar?q=Filip+Boltuz%CC%8Cic%CC%81+and+Jan+S%CC%8Cnajder%2E+2015%2E+Identifying+prominent+arguments+in+online+debates+using+semantic+textual+similarity%2E+In+Proceedings+of+the+2nd+Workshop+on+Argumentation+Mining%2C+pages+110%E2%80%93+115)
2. Elena Cabrio and Serena Villata. 2012. Natural Language Arguments: A Combined Approach. In Proceedings of the 20th European Conference on Artificial Intelligence, pages 205–210. [find](https://scholar.google.com/scholar?q=Elena+Cabrio+and+Serena+Villata%2E+2012%2E+Natural+Language+Arguments%3A+A+Combined+Approach%2E+In+Proceedings+of+the+20th+European+Conference+on+Artificial+Intelligence%2C+pages+205%E2%80%93210%2E)
3. Alexis Conneau, Douwe Kiela, Holger Schwenk, Loı̈c Barrault, and Antoine Bordes. 2017. Supervised learning of universal sentence representations from natural language inference data. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 670–680. [find](https://scholar.google.com/scholar?q=Alexis+Conneau%2C+Douwe+Kiela%2C+Holger+Schwenk%2C+Lo%C4%B1%CC%88c+Barrault%2C+and+Antoine+Bordes%2E+2017%2E+Supervised+learning+of+universal+sentence+representations+from+natural+language+inference+data%2E+In+Proceedings+of) [our card](https://md.research.elamur.ai/free/en/papers/conneau-2017-supervised-learning-universal-sentence-representations/)
4. William H. E. Day and Herbert Edelsbrunner. 1984. Efficient algorithms for agglomerative hierarchical clustering methods. Journal of Classification, 1(1):7–24. [find](https://scholar.google.com/scholar?q=William+H%2E+E%2E+Day+and+Herbert+Edelsbrunner%2E+1984%2E+Efficient+algorithms+for+agglomerative+hierarchical+clustering+methods%2E+Journal+of+Classification%2C+1%281%29%3A7%E2%80%9324%2E)
5. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. BERT: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805. [find](https://scholar.google.com/scholar?q=Jacob+Devlin%2C+Ming%2DWei+Chang%2C+Kenton+Lee%2C+and+Kristina+Toutanova%2E+2018%2E+BERT%3A+Pre%2Dtraining+of+deep+bidirectional+transformers+for+language+understanding%2E+arXiv+preprint+arXiv%3A1810%2E04805%2E)
6. Steffen Eger, Johannes Daxenberger, and Iryna Gurevych. 2017. Neural end-to-end learning for computational argumentation mining. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 11–22. [find](https://scholar.google.com/scholar?q=Steffen+Eger%2C+Johannes+Daxenberger%2C+and+Iryna+Gurevych%2E+2017%2E+Neural+end%2Dto%2Dend+learning+for+computational+argumentation+mining%2E+In+Proceedings+of+the+55th+Annual+Meeting+of+the+Association+for+Comput) [our card](https://md.research.elamur.ai/free/en/papers/eger-2017-neural-end-end-learning-computational/)
7. Dirk Hovy, Taylor Berg-Kirkpatrick, Ashish Vaswani, and Eduard Hovy. 2013. Learning whom to trust with MACE. In Proceedings of the 2013 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 1120–1130. [find](https://scholar.google.com/scholar?q=Dirk+Hovy%2C+Taylor+Berg%2DKirkpatrick%2C+Ashish+Vaswani%2C+and+Eduard+Hovy%2E+2013%2E+Learning+whom+to+trust+with+MACE%2E+In+Proceedings+of+the+2013+Conference+of+the+North+American+Chapter+of+the+Association+for+) [our card](https://md.research.elamur.ai/free/en/papers/hovy-2013-learning-whom-trust-mace/)
8. Ran Levy, Yonatan Bilu, Daniel Hershcovich, Ehud Aharoni, and Noam Slonim. 2014. Context dependent claim detection. In Proceedings of COLING 2014, the 25th International Conference on Computational Linguistics: Technical Papers, pages 1489– 1500. \#\# Acknowledgments The authors would like to sincerel… [find](https://scholar.google.com/scholar?q=Ran+Levy%2C+Yonatan+Bilu%2C+Daniel+Hershcovich%2C+Ehud+Aharoni%2C+and+Noam+Slonim%2E+2014%2E+Context+dependent+claim+detection%2E+In+Proceedings+of+COLING+2014%2C+the+25th+International+Conference+on+Computational+Li) [our card](https://md.research.elamur.ai/free/en/papers/levy-2014-context-dependent-claim-detection/)
9. Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean. 2013. Efficient Estimation of Word Representations in Vector Space. arXiv preprint arXiv:1301.3781. [find](https://scholar.google.com/scholar?q=Tomas+Mikolov%2C+Kai+Chen%2C+Greg+Corrado%2C+and+Jeffrey+Dean%2E+2013%2E+Efficient+Estimation+of+Word+Representations+in+Vector+Space%2E+arXiv+preprint+arXiv%3A1301%2E3781%2E)
10. Amine Trabelsi and Osmar R Zaı̈ane. 2015. Extraction and clustering of arguing expressions in contentious text. Data & Knowledge Engineering, 100:226– 239. [find](https://scholar.google.com/scholar?q=Amine+Trabelsi+and+Osmar+R+Za%C4%B1%CC%88ane%2E+2015%2E+Extraction+and+clustering+of+arguing+expressions+in+contentious+text%2E+Data+%26+Knowledge+Engineering%2C+100%3A226%E2%80%93+239%2E)
11. Amita Misra, Brian Ecker, and Marilyn A. Walker. 2016. Measuring the similarity of sentential arguments in dialogue. In Proceedings of the SIG-DIAL 2016 Conference, The 17th Annual Meeting of the Special Interest Group on Discourse and Dialogue, 13-15 September 2016, Los Angeles, CA, USA, pages 276–… [find](https://scholar.google.com/scholar?q=Amita+Misra%2C+Brian+Ecker%2C+and+Marilyn+A%2E+Walker%2E+2016%2E+Measuring+the+similarity+of+sentential+arguments+in+dialogue%2E+In+Proceedings+of+the+SIG%2DDIAL+2016+Conference%2C+The+17th+Annual+Meeting+of+the+Spec)
12. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Ł ukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett, editors, Advances in Neural Information Proces… [find](https://scholar.google.com/scholar?q=Ashish+Vaswani%2C+Noam+Shazeer%2C+Niki+Parmar%2C+Jakob+Uszkoreit%2C+Llion+Jones%2C+Aidan+N+Gomez%2C+%C5%81+ukasz+Kaiser%2C+and+Illia+Polosukhin%2E+2017%2E+Attention+is+all+you+need%2E+In+I%2E+Guyon%2C+U%2E+V%2E+Luxburg%2C+S%2E+Bengio%2C+H%2E)
13. Jeffrey Pennington, Richard Socher, and Christopher D. Manning. 2014. GloVe: Global Vectors for Word Representation. In Empirical Methods in Natural Language Processing (EMNLP), pages 1532– 1543. [find](https://scholar.google.com/scholar?q=Jeffrey+Pennington%2C+Richard+Socher%2C+and+Christopher+D%2E+Manning%2E+2014%2E+GloVe%3A+Global+Vectors+for+Word+Representation%2E+In+Empirical+Methods+in+Natural+Language+Processing+%28EMNLP%29%2C+pages+1532%E2%80%93+1543%2E)
14. Henning Wachsmuth, Nona Naderi, Ivan Habernal, Yufang Hou, Graeme Hirst, Iryna Gurevych, and Benno Stein. 2017. Argumentation quality assessment: Theory vs. practice. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 250–255. [find](https://scholar.google.com/scholar?q=Henning+Wachsmuth%2C+Nona+Naderi%2C+Ivan+Habernal%2C+Yufang+Hou%2C+Graeme+Hirst%2C+Iryna+Gurevych%2C+and+Benno+Stein%2E+2017%2E+Argumentation+quality+assessment%3A+Theory+vs%2E+practice%2E+In+Proceedings+of+the+55th+Annual)
15. Matthew Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke Zettlemoyer. 2018. Deep contextualized word representations. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technolog… [find](https://scholar.google.com/scholar?q=Matthew+Peters%2C+Mark+Neumann%2C+Mohit+Iyyer%2C+Matt+Gardner%2C+Christopher+Clark%2C+Kenton+Lee%2C+and+Luke+Zettlemoyer%2E+2018%2E+Deep+contextualized+word+representations%2E+In+Proceedings+of+the+2018+Conference+of+t)
16. Henning Wachsmuth, Shahbaz Syed, and Benno Stein. 2018. Retrieval of the best counterargument without prior topic knowledge. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 241–251. [find](https://scholar.google.com/scholar?q=Henning+Wachsmuth%2C+Shahbaz+Syed%2C+and+Benno+Stein%2E+2018%2E+Retrieval+of+the+best+counterargument+without+prior+topic+knowledge%2E+In+Proceedings+of+the+56th+Annual+Meeting+of+the+Association+for+Computatio) [our card](https://md.research.elamur.ai/free/en/papers/wachsmuth-2018-retrieval-best-counterargument-without-prior/)
17. Nils Reimers and Iryna Gurevych. 2018. Why comparing single performance scores does not allow to draw conclusions about machine learning approaches. arXiv preprint arXiv:1803.09578. — [arXiv:1803.09578](https://arxiv.org/abs/1803.09578) [find](https://scholar.google.com/scholar?q=Nils+Reimers+and+Iryna+Gurevych%2E+2018%2E+Why+comparing+single+performance+scores+does+not+allow+to+draw+conclusions+about+machine+learning+approaches%2E+arXiv+preprint+arXiv%3A1803%2E09578%2E+%E2%80%94+%5BarXiv%3A1803%2E0957)
18. Ruty Rinott, Lena Dankin, Carlos Alzate Perez, Mitesh M. Khapra, Ehud Aharoni, and Noam Slonim. 2015. Show me your evidence - an automatic method for context dependent evidence detection. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 440–450. [find](https://scholar.google.com/scholar?q=Ruty+Rinott%2C+Lena+Dankin%2C+Carlos+Alzate+Perez%2C+Mitesh+M%2E+Khapra%2C+Ehud+Aharoni%2C+and+Noam+Slonim%2E+2015%2E+Show+me+your+evidence+%2D+an+automatic+method+for+context+dependent+evidence+detection%2E+In+Proceedin) [our card](https://md.research.elamur.ai/free/en/papers/rinott-2015-show-me-your-evidence-automatic/)
19. Eyal Shnarch, Carlos Alzate, Lena Dankin, Martin Gleize, Yufang Hou, Leshem Choshen, Ranit Aharonov, and Noam Slonim. 2018. Will it Blend? Blending Weak and Strong Labeled Data in a Neural Network for Argumentation Mining. In Proceedings of the 56th Annual Meeting of the Association for Computationa… [find](https://scholar.google.com/scholar?q=Eyal+Shnarch%2C+Carlos+Alzate%2C+Lena+Dankin%2C+Martin+Gleize%2C+Yufang+Hou%2C+Leshem+Choshen%2C+Ranit+Aharonov%2C+and+Noam+Slonim%2E+2018%2E+Will+it+Blend%3F+Blending+Weak+and+Strong+Labeled+Data+in+a+Neural+Network+for)
20. Christian Stab, Johannes Daxenberger, Chris Stahlhut, Tristan Miller, Benjamin Schiller, Christopher Tauchmann, Steffen Eger, and Iryna Gurevych. 2018a. ArgumenText: Searching for Arguments in Heterogeneous Sources. In Proceedings of the 2018 Conference of the North American Chapter of the Associati… [find](https://scholar.google.com/scholar?q=Christian+Stab%2C+Johannes+Daxenberger%2C+Chris+Stahlhut%2C+Tristan+Miller%2C+Benjamin+Schiller%2C+Christopher+Tauchmann%2C+Steffen+Eger%2C+and+Iryna+Gurevych%2E+2018a%2E+ArgumenText%3A+Searching+for+Arguments+in+Heterog)
21. Christian Stab and Iryna Gurevych. 2014. Identifying Argumentative Discourse Structures in Persuasive Essays. Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 46–56. [find](https://scholar.google.com/scholar?q=Christian+Stab+and+Iryna+Gurevych%2E+2014%2E+Identifying+Argumentative+Discourse+Structures+in+Persuasive+Essays%2E+Proceedings+of+the+2014+Conference+on+Empirical+Methods+in+Natural+Language+Processing+%28EM) [our card](https://md.research.elamur.ai/free/en/papers/stab-2014-identifying-argumentative-discourse-structures-persuasive/)
22. Christian Stab, Tristan Miller, Benjamin Schiller, Pranav Rai, and Iryna Gurevych. 2018b. Cross-topic argument mining from heterogeneous sources. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, volume Long Papers, pages 3664–3674. [find](https://scholar.google.com/scholar?q=Christian+Stab%2C+Tristan+Miller%2C+Benjamin+Schiller%2C+Pranav+Rai%2C+and+Iryna+Gurevych%2E+2018b%2E+Cross%2Dtopic+argument+mining+from+heterogeneous+sources%2E+In+Proceedings+of+the+2018+Conference+on+Empirical+Met) [our card](https://md.research.elamur.ai/free/en/papers/stab-2018-cross-topic-argument-mining-heterogeneous/)