---
title: "A News Editorial Corpus for Mining Argumentation Strategies"
description: "Neither approach lets researchers study the overall persuasive strategy of a text. 30 references, cited by 61 papers here."
authors: ["Khalid Al-Khatib", "Henning Wachsmuth", "Johannes Kiesel", "Matthias Hagen", "Benno Stein"]
year: 2016
venue: "Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers"
source: "https://aclanthology.org/C16-1324/"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/khatib-2016-news-editorial-corpus-mining-argumentation/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/khatib-2016-news-editorial-corpus-mining-argumentation/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2016](https://md.research.elamur.ai/free/en/papers/2016/page/3/) / A News Editorial Corpus for Mining Argumentation Strategies

# A News Editorial Corpus for Mining Argumentation Strategies

**Authors:** [Khalid Al-Khatib](https://md.research.elamur.ai/free/en/authors/khalid-al-khatib/), [Henning Wachsmuth](https://md.research.elamur.ai/free/en/authors/henning-wachsmuth/), [Johannes Kiesel](https://md.research.elamur.ai/free/en/authors/johannes-kiesel/), Matthias Hagen and [Benno Stein](https://md.research.elamur.ai/free/en/authors/benno-stein/) · **Year:** 2016 · **Venue:** [Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers](https://md.research.elamur.ai/free/en/venues/coling/) ([COLING 2016](https://md.research.elamur.ai/free/en/venues/coling/2016/)) · **Publisher:** [International Committee on Computational Linguistics (ICCL)](https://md.research.elamur.ai/free/en/publishers/international-committee-on-computational-linguistics-iccl/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/C16-1324/) · [PDF](https://aclanthology.org/C16-1324.pdf)

**Citations:** [61](https://md.research.elamur.ai/free/en/papers/khatib-2016-news-editorial-corpus-mining-argumentation/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
Argument mining corpora up to 2016 label argumentative units either coarsely, for example as premise vs. conclusion, or only partially, for example evidence tied to given topics. Neither approach lets researchers study the overall persuasive strategy of a text. The authors fill this gap for news editorials, a genre with little existing coverage. They built Webis-Editorials-16, a corpus of 300 editorials drawn in equal shares from Al Jazeera, Fox News and The Guardian. Every argumentative discourse unit in every editorial received one of six content-based types:
- anecdote
- assumption
- common ground
- statistics
- testimony
- other

Three professional annotators hired via upwork.com labelled all units. Their agreement reached Fleiss' κ = 0.56, which the authors consider high given the number and difficulty of the categories. A first statistical analysis shows that the three portals use the types in noticeably different proportions, which points to different argumentation strategies. The resource matters because it supports fine-grained unit classification and the study of persuasion in monological opinion texts, and it is freely available.

## Key points
- The corpus contains 300 editorials (100 per portal) and 14,313 annotated units, about 47.7 units per editorial on average.
- Every unit carries exactly one of six types. Inter-annotator agreement is Fleiss' κ = 0.56, and the rare type common ground shows notably weaker agreement.
- Assumptions dominate, making up 68.4% of all units and 71.7% at The Guardian.
- The portals differ in other types:
  - Fox News uses the most testimony (8.7% of units; 4.55 per editorial vs. 2.53 at The Guardian) and the most common ground.
  - Al Jazeera leans on anecdotes (21.0% of units).
  - Statistics stay near 2.7–2.9% at all three portals.
- Relations between units, such as support and attack, were deliberately left out. Editorials rarely have a clear hierarchical argument structure and often contain enthymemes, which makes consistent relation annotation very hard.

## Interesting ideas
- **Label everything, not just the interesting parts.** Annotating every unit instead of only topic-relevant evidence turns each document into a complete sequence of types. Such sequences allow strategy or flow patterns to be modelled statistically, and the same approach could apply to Ukrainian news or political texts.
- **Force one label per unit with precedence rules.** Units can be hybrid, for example both testimony and statistics. The authors still assign a single type and give annotators guidelines on which type wins in which context. This keeps the classification task clean and is a practical trade-off for any multi-class annotation project.
- **Use the source as an analysis variable.** Sampling the same genre evenly from outlets with different profiles makes it possible to compare type distributions across sources. This is a cheap first test of whether an annotation scheme captures meaningful variation.
- **Look for disguised assertions.** Writers sometimes place phrases like "in fact" before an assumption so that it reads as shared knowledge. Detecting such framing could feed into persuasion or manipulation analysis.
- **Choose the right granularity for sequence analysis.** The authors suggest testing sequences at the whole-editorial level against the paragraph level, and abstracting them, for example to the majority type per paragraph. Long documents create too much variance for raw sequences, and this idea transfers to any discourse-flow study.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

📚 Shelves (2): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works

## Cite

Cite as: Khalid Al-Khatib, Henning Wachsmuth, Johannes Kiesel, Matthias Hagen and Benno Stein. 2016. A News Editorial Corpus for Mining Argumentation Strategies. Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{alkhatib2016news,
  title = {{A News Editorial Corpus for Mining Argumentation Strategies}},
  author = {Khalid Al-Khatib, Henning Wachsmuth, Johannes Kiesel, Matthias Hagen and Benno Stein},
  booktitle = {Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers},
  year = {2016},
  url = {https://aclanthology.org/C16-1324/}
}
```

- [← previous Squib: Expressive Power of Abstract Meaning Representations](https://md.research.elamur.ai/free/en/papers/bos-2016-squib-expressive-power-abstract-meaning/)
- [next → Learning a POS tagger for AAVE-like language](https://md.research.elamur.ai/free/en/papers/jorgensen-2016-learning-pos-tagger-aave-like/)
- [↑ back to papers of 2016](https://md.research.elamur.ai/free/en/papers/2016/page/3/)

## References

30 references, 9 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Ehud Aharoni, Anatoly Polnarov, Tamar Lavee, Daniel Hershcovich, Ran Levy, Ruty Rinott, Dan Gutfreund, and Noam Slonim. 2014. [A Benchmark Dataset for Automatic Detection of Claims and Evidence in the Context of Controversial Topics](https://md.research.elamur.ai/free/en/papers/aharoni-2014-benchmark-dataset-automatic-detection-claims/). In Proceedings of the First Workshop on Argumentation Mining, pages…
2. Khalid Al-Khatib, Henning Wachsmuth, Matthias Hagen, Jonas Köhler, and Benno Stein. 2016. Cross-Domain Mining of Argumentative Text through Distant Supervision. In 15th Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL 16),… [find](https://scholar.google.com/scholar?q=Khalid+Al%2DKhatib%2C+Henning+Wachsmuth%2C+Matthias+Hagen%2C+Jonas+K%C3%B6hler%2C+and+Benno+Stein%2E+2016%2E+Cross%2DDomain+Mining+of+Argumentative+Text+through+Distant+Supervision%2E+In+15th+Conference+of+the+North+America)
3. Aristotle. 2007. On Rhetoric: A Theory of Civic Discourse (George A. Kennedy, Translator). Clarendon Aristotle series. Oxford University Press. [find](https://scholar.google.com/scholar?q=Aristotle%2E+2007%2E+On+Rhetoric%3A+A+Theory+of+Civic+Discourse+%28George+A%2E+Kennedy%2C+Translator%29%2E+Clarendon+Aristotle+series%2E+Oxford+University+Press%2E)
4. Bal Krishna Bal and Patrick Saint-Dizier. 2009. Towards and Analysis of Argumentation Structure and the Strength of Arguments in News Editorials. In AISB Symposium on Persuasive Technologies, pages 55–63. [find](https://scholar.google.com/scholar?q=Bal+Krishna+Bal+and+Patrick+Saint%2DDizier%2E+2009%2E+Towards+and+Analysis+of+Argumentation+Structure+and+the+Strength+of+Arguments+in+News+Editorials%2E+In+AISB+Symposium+on+Persuasive+Technologies%2C+pages+55)
5. Bal Krishna Bal. 2009. Towards an analysis of opinions in news editorials: How positive was the year? In Proceedings of the Eighth International Conference on Computational Semantics, IWCS-8 ’09, pages 260–263, Stroudsburg, PA, USA. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Bal+Krishna+Bal%2E+2009%2E+Towards+an+analysis+of+opinions+in+news+editorials%3A+How+positive+was+the+year%3F+In+Proceedings+of+the+Eighth+International+Conference+on+Computational+Semantics%2C+IWCS%2D8+%E2%80%9909%2C+page)
6. Ann Bies, Mark Ferguson, Karen Katz, Robert MacIntyre, Victoria Tredinnick, Grace Kim, Mary Ann Marcinkiewicz, and Britta Schasberger. 1995. Bracketing Guidelines for Treebank II Style Penn Treebank Project. Technical report, University of Pennsylvania. [find](https://scholar.google.com/scholar?q=Ann+Bies%2C+Mark+Ferguson%2C+Karen+Katz%2C+Robert+MacIntyre%2C+Victoria+Tredinnick%2C+Grace+Kim%2C+Mary+Ann+Marcinkiewicz%2C+and+Britta+Schasberger%2E+1995%2E+Bracketing+Guidelines+for+Treebank+II+Style+Penn+Treebank+P)
7. Filip Boltužić and Jan Šnajder. 2014. Back up your Stance: Recognizing Arguments in Online Discussions. In Proceedings of the First Workshop on Argumentation Mining, pages 49–58, Baltimore, Maryland, June. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Filip+Boltu%C5%BEic%CC%81+and+Jan+%C5%A0najder%2E+2014%2E+Back+up+your+Stance%3A+Recognizing+Arguments+in+Online+Discussions%2E+In+Proceedings+of+the+First+Workshop+on+Argumentation+Mining%2C+pages+49%E2%80%9358%2C+Baltimore%2C+Maryland%2C)
8. Marisa Chow. 2016. Argument identification in chinese editorials. In NAACL Student Research Workshop 2016. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Marisa+Chow%2E+2016%2E+Argument+identification+in+chinese+editorials%2E+In+NAACL+Student+Research+Workshop+2016%2E+Association+for+Computational+Linguistics%2E)
9. Silvie Cinková, Martin Holub, and Vincent Kríž. 2012. Managing Uncertainty in Semantic Tagging. In Proceedings of the 13th Conference of the European Chapter of the Association for Computational Linguistics, EACL ’12, pages 840–850, Stroudsburg, PA, USA. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Silvie+Cinkov%C3%A1%2C+Martin+Holub%2C+and+Vincent+Kr%C3%AD%C5%BE%2E+2012%2E+Managing+Uncertainty+in+Semantic+Tagging%2E+In+Proceedings+of+the+13th+Conference+of+the+European+Chapter+of+the+Association+for+Computational+Lingu)
10. Debanjan Ghosh, Smaranda Muresan, Nina Wacholder, Mark Aakhus, and Matthew Mitsui. 2014. Analyzing Argumentative Discourse Units in Online Interactions. In Proceedings of the First Workshop on Argumentation Mining, pages 39–48, Baltimore, Maryland, June. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Debanjan+Ghosh%2C+Smaranda+Muresan%2C+Nina+Wacholder%2C+Mark+Aakhus%2C+and+Matthew+Mitsui%2E+2014%2E+Analyzing+Argumentative+Discourse+Units+in+Online+Interactions%2E+In+Proceedings+of+the+First+Workshop+on+Argumen)
11. Iryna Gurevych, Eduard H. Hovy, Noam Slonim, and Benno Stein. 2016. Debating Technologies (Dagstuhl Seminar 15512). Dagstuhl Reports, 5(12):18–46. [find](https://scholar.google.com/scholar?q=Iryna+Gurevych%2C+Eduard+H%2E+Hovy%2C+Noam+Slonim%2C+and+Benno+Stein%2E+2016%2E+Debating+Technologies+%28Dagstuhl+Seminar+15512%29%2E+Dagstuhl+Reports%2C+5%2812%29%3A18%E2%80%9346%2E)
12. Ivan Habernal and Iryna Gurevych. 2015. Exploiting Debate Portals for Semi-Supervised Argumentation Mining in User-Generated Web Discourse. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 2127–2137. [find](https://scholar.google.com/scholar?q=Ivan+Habernal+and+Iryna+Gurevych%2E+2015%2E+Exploiting+Debate+Portals+for+Semi%2DSupervised+Argumentation+Mining+in+User%2DGenerated+Web+Discourse%2E+In+Proceedings+of+the+2015+Conference+on+Empirical+Methods+i)
13. Ivan Habernal and Iryna Gurevych. 2016. [Argumentation mining in user-generated web discourse](https://md.research.elamur.ai/free/en/papers/habernal-2017-argumentation-mining-user-generated-web/). Computational Linguistics, page (in press). Submission received: 2 April 2015; revised version received: 20 April 2016; accepted for publication: 14 June 2016. Pre-print available at http://arxiv.org/abs/16… [↗](http://arxiv.org/abs/16…)
14. Gary G. Koch J. Richard Landis. 1977. The Measurement of Observer Agreement for Categorical Data. Biometrics, 33(1):159–174. [find](https://scholar.google.com/scholar?q=Gary+G%2E+Koch+J%2E+Richard+Landis%2E+1977%2E+The+Measurement+of+Observer+Agreement+for+Categorical+Data%2E+Biometrics%2C+33%281%29%3A159%E2%80%93174%2E)
15. Johannes Kiesel, Khalid Al-Khatib, Matthias Hagen, and Benno Stein. 2015. A Shared Task on Argumentation Mining in Newspaper Editorials. In Proc. of the 2nd Workshop on Argumentation Mining, pages 35–38. [find](https://scholar.google.com/scholar?q=Johannes+Kiesel%2C+Khalid+Al%2DKhatib%2C+Matthias+Hagen%2C+and+Benno+Stein%2E+2015%2E+A+Shared+Task+on+Argumentation+Mining+in+Newspaper+Editorials%2E+In+Proc%2E+of+the+2nd+Workshop+on+Argumentation+Mining%2C+pages+35%E2%80%93)
16. Christopher D. Manning, Mihai Surdeanu, John Bauer, Jenny Finkel, Steven J. Bethard, and David McClosky. 2014. [The Stanford CoreNLP natural language processing toolkit](https://md.research.elamur.ai/free/en/papers/manning-2014-stanford-corenlp-natural-language-processing/). In Association for Computational Linguistics (ACL) System Demonstrations, pages 55–60.
17. Joonsuk Park and Claire Cardie. 2014. [Identifying Appropriate Support for Propositions in Online User Comments](https://md.research.elamur.ai/free/en/papers/park-2014-identifying-appropriate-support-propositions-online/). In Proceedings of the First Workshop on Argumentation Mining, pages 29–38, Baltimore, Maryland, June. Association for Computational Linguistics.
18. Andreas Peldszus and Manfred Stede. 2015. [Joint prediction in mst-style discourse parsing for argumentation mining](https://md.research.elamur.ai/free/en/papers/peldszus-2015-joint-prediction-mst-style-discourse/). In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 938–948, Lisbon, Portugal, September. Association for Computational Linguistics.
19. Isaac Persing, Alan Davis, and Vincent Ng. 2010. [Modeling Organization in Student Essays](https://md.research.elamur.ai/free/en/papers/persing-2010-modeling-organization-student-essays/). In Proceedings of the 2010 Conference on Empirical Methods in Natural Language Processing, pages 229–239.
20. Chris Reed and Glenn Rowe. 2004. Araucaria: Software for Argument Analysis, Diagramming and Representation. International Journal on Artificial Intelligence Tools, 13. [find](https://scholar.google.com/scholar?q=Chris+Reed+and+Glenn+Rowe%2E+2004%2E+Araucaria%3A+Software+for+Argument+Analysis%2C+Diagramming+and+Representation%2E+International+Journal+on+Artificial+Intelligence+Tools%2C+13%2E)
21. Ruty Rinott, Lena Dankin, Carlos Alzate Perez, Mitesh M. Khapra, Ehud Aharoni, and Noam Slonim. 2015. [Show Me Your Evidence – An Automatic Method for Context Dependent Evidence Detection](https://md.research.elamur.ai/free/en/papers/rinott-2015-show-me-your-evidence-automatic/). In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 440–450…
22. Christian Stab and Iryna Gurevych. 2014. [Annotating Argument Components and Relations in Persuasive Essays](https://md.research.elamur.ai/free/en/papers/stab-2014-annotating-argument-components-relations-persuasive/). In Proceedings of the 25th International Conference on Computational Linguistics, COLING 2014, pages 1501– 1510.
23. Simone Teufel. 2014. Scientific Argumentation Detection as Limited-Domain Intention Recognition. In Proceedings of the Workshop on Frontiers and Connections between Argumentation Theory and Natural Language Processing, pages 101–109. [find](https://scholar.google.com/scholar?q=Simone+Teufel%2E+2014%2E+Scientific+Argumentation+Detection+as+Limited%2DDomain+Intention+Recognition%2E+In+Proceedings+of+the+Workshop+on+Frontiers+and+Connections+between+Argumentation+Theory+and+Natural+La)
24. Teun A. van Dijk. 1992. Racism and argumentation:”Race Rio” Rhetoric in Tabloib Editorials . In In van Emeren et al. (Eds.), Argumentation illuminated. [find](https://scholar.google.com/scholar?q=Teun+A%2E+van+Dijk%2E+1992%2E+Racism+and+argumentation%3A%E2%80%9DRace+Rio%E2%80%9D+Rhetoric+in+Tabloib+Editorials+%2E+In+In+van+Emeren+et+al%2E+%28Eds%2E%29%2C+Argumentation+illuminated%2E)
25. Teun A. van Dijk. 1995. Opinions and Ideologies in Editorials. In Proceedings of the 4th International Symposium of Critical Discourse Analysis, Language, Social Life and Critical Thought, Athens, June. [find](https://scholar.google.com/scholar?q=Teun+A%2E+van+Dijk%2E+1995%2E+Opinions+and+Ideologies+in+Editorials%2E+In+Proceedings+of+the+4th+International+Symposium+of+Critical+Discourse+Analysis%2C+Language%2C+Social+Life+and+Critical+Thought%2C+Athens%2C+Jun)
26. Henning Wachsmuth, Johannes Kiesel, and Benno Stein. 2015. Sentiment Flow – A General Model of Web Review Argumentation. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 601–611, Lisbon, Portugal. [find](https://scholar.google.com/scholar?q=Henning+Wachsmuth%2C+Johannes+Kiesel%2C+and+Benno+Stein%2E+2015%2E+Sentiment+Flow+%E2%80%93+A+General+Model+of+Web+Review+Argumentation%2E+In+Proceedings+of+the+2015+Conference+on+Empirical+Methods+in+Natural+Language+)
27. Henning Wachsmuth, Khalid Al-Khatib, and Benno Stein. 2016. Using Argument Mining to Assess the Argumentation Quality of Essays. In Proceedings of the 26th International Conference on Computational Linguistics. [find](https://scholar.google.com/scholar?q=Henning+Wachsmuth%2C+Khalid+Al%2DKhatib%2C+and+Benno+Stein%2E+2016%2E+Using+Argument+Mining+to+Assess+the+Argumentation+Quality+of+Essays%2E+In+Proceedings+of+the+26th+International+Conference+on+Computational+Li)
28. Douglas Walton, Christopher Reed, and Fabrizio Macagno. 2008. Argumentation schemes. Cambridge University Press. [find](https://scholar.google.com/scholar?q=Douglas+Walton%2C+Christopher+Reed%2C+and+Fabrizio+Macagno%2E+2008%2E+Argumentation+schemes%2E+Cambridge+University+Press%2E)
29. Theresa Wilson and Janyce Wiebe. 2003. Annotating opinions in the world press. In Proceedings of the Fourth SIGdial Workshop of Discourse and Dialogue. [find](https://scholar.google.com/scholar?q=Theresa+Wilson+and+Janyce+Wiebe%2E+2003%2E+Annotating+opinions+in+the+world+press%2E+In+Proceedings+of+the+Fourth+SIGdial+Workshop+of+Discourse+and+Dialogue%2E)
30. Hong Yu and Vasileios Hatzivassiloglou. 2003. [Towards answering opinion questions: Separating facts from opinions and identifying the polarity of opinion sentences](https://md.research.elamur.ai/free/en/papers/yu-2003-answering-opinion-questions-separating-facts/). In Proceedings of the 2003 Conference on Empirical Methods in Natural Language Processing, EMNLP ’03, pages 129–136, Stroudsburg, PA, U…

*Updated 31 October 2026*