---
title: "Discourse analysis — definition and key papers"
description: "Discourse analysis is the study of how stretches of language beyond the single sentence form coherent wholes."
canonical: "https://research.elamur.ai/free/en/concepts/discourse-analysis/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/concepts/discourse-analysis/)

[home](https://md.research.elamur.ai/free/en/) / [concepts](https://md.research.elamur.ai/free/en/concepts/) / Discourse analysis

# 🧵 Discourse analysis

Concept · 30 key papers · [all concepts](https://md.research.elamur.ai/free/en/concepts/)

Discourse analysis is the study of how stretches of language beyond the single sentence form coherent wholes. It covers the relations that link clauses and sentences (cause, contrast, elaboration), the structures those relations build, and the computational methods that recover them from text.

Also: discourse parsing, discourse structure analysis, rhetorical parsing, shallow discourse parsing, coherence analysis, RST parsing, PDTB-style parsing

## What it is

Discourse analysis asks what makes a sequence of sentences a text rather than a random list. A reader of "Max fell. John pushed him" understands that the pushing came first and caused the fall, even though nothing in the words says so. Discourse analysis tries to describe such links and to explain how readers infer them. The term has a broad life in linguistics and the social sciences, where it also covers conversation and ideology in talk. In computational linguistics it has a narrower, operational meaning: identifying the units of a text, the coherence relations among them, and the overall shape these relations form.

Three families of frameworks dominate the field:

| Framework | Unit and anchor | Structure | Main corpus |
| --- | --- | --- | --- |
| Rhetorical Structure Theory (RST) | clause-sized elementary discourse units (EDUs) | one tree per document, with nucleus/satellite roles | RST Discourse Treebank (RST-DT) |
| Penn Discourse TreeBank (PDTB) | connectives and their two arguments | flat, local relations with no global tree | PDTB on the Wall Street Journal |
| Segmented Discourse Representation Theory (SDRT) | EDUs and complex units | graphs, often non-tree | STAC dialogue corpus |

## The key idea

The core mechanism is the discourse relation. It is a typed link between two spans of text, each describing an abstract object such as an event, a state or a proposition. A connective may signal the relation overtly ("because", "however"), or the reader may have to infer it. Most automatic systems treat recognising a relation as choosing the most probable sense given its arguments:

```
ŝ = argmax over s in S of  P(s | Arg1, Arg2, c)
```

Here S is the inventory of relation senses, for example the four PDTB top-level classes Comparison, Contingency, Expansion and Temporal. Arg1 and Arg2 are the two text spans the relation joins. Arg2 is the span syntactically tied to the connective. The variable c is the connective, which may be empty when the relation is implicit, and ŝ is the predicted sense. In RST-style analysis, the same decision is applied recursively. Adjacent spans are merged into larger spans, and each merge receives a relation label and a nuclearity assignment, which marks which span is more central. The result is a tree covering the whole document.

The presence or absence of c is the field's central practical divide. Explicit connectives are only mildly ambiguous. A classifier that looks only at the connective labels explicit PDTB relations with 93.09% accuracy on the four top-level senses ([Pitler et al. 2008](https://md.research.elamur.ai/free/en/papers/pitler-2008-easily-identifiable-discourse-relations/)). Without a connective, the system must rely on lexical meaning, syntax, entities and context, and accuracy drops sharply. On the finer set of 11 second-level PDTB types, an early implicit classifier reached 40.2% accuracy, against a majority baseline of about 26.1% ([Lin et al. 2009](https://md.research.elamur.ai/free/en/papers/lin-2009-recognizing-implicit-discourse-relations-penn/)). The two label sets differ, so these figures are not directly comparable.

## History and founding papers

The phrase "discourse analysis" goes back to Zellig Harris (1952). Theoretical foundations for coherence relations came from Hobbs (1985) and from Grosz and Sidner (1986) on intentional structure. RST was developed by Mann and Thompson during the 1980s, and their 1988 paper is its standard statement. Early formal work showed that discourse relations could be inferred through default reasoning that combines world knowledge with pragmatic maxims. One example is narrating events in the order they happened unless causal knowledge overrides it ([Lascarides & Asher 1991](https://md.research.elamur.ai/free/en/papers/lascarides-1991-discourse-relations-defeasible-knowledge/)). RST quickly drew criticism. Moore and Pollack argued that text segments relate simultaneously on an informational level and an intentional level, and that a single relation per pair cannot capture both ([Moore & Pollack 1992](https://md.research.elamur.ai/free/en/papers/moore-1992-problem-rst-need-multi-level/)).

The empirical turn came with large annotated corpora. The RST Discourse Treebank (RST-DT, released as the RST Corpus) layered rhetorical trees over 385 Penn Treebank WSJ articles: more than 176,000 words segmented into 21,789 EDUs and labelled with 78 relations. Annotator agreement was highest for segmentation and declined higher up the tree ([Carlson et al. 2001](https://md.research.elamur.ai/free/en/papers/carlson-2001-building-discourse-tagged-corpus-framework/)). Taking a different approach, the PDTB anchored relations in lexical connectives and their arguments. It deliberately avoided committing to any one theory of global structure ([Miltsakaki et al. 2004](https://md.research.elamur.ai/free/en/papers/miltsakaki-2004-penn-discourse-treebank/)). Its 2.0 release covers about one million words and includes 18,459 explicit connective tokens. Annotators also inserted implicit connectives between adjacent sentences and recorded attribution ([Prasad et al. 2008](https://md.research.elamur.ai/free/en/papers/prasad-2008-penn-discourse-treebank-2-0/)). Webber and colleagues supplied a theoretical rationale for this lexical approach. They argued that discourse adverbials behave anaphorically, like pronouns, while conjunctions link structurally adjacent units ([Webber et al. 2003](https://md.research.elamur.ai/free/en/papers/webber-2003-anaphora-discourse-structure/)). Wolf and Gibson annotated 135 news texts with high agreement (Kappa, a chance-corrected agreement measure, above 0.8). They found frequent crossed dependencies and multi-parent nodes, which challenges the assumption that discourse is tree-shaped ([Wolf & Gibson 2005](https://md.research.elamur.ai/free/en/papers/wolf-2005-representing-discourse-coherence-corpus-based/)).

## How it is used today

Most current work is computational. The bulk of it follows the two large corpus traditions, RST and PDTB. SDRT work has centred on dialogue, where its graph structures matter most ([Asher et al. 2016](https://md.research.elamur.ai/free/en/papers/asher-2016-discourse-structure-dialogue-acts-multiparty/)).

- RST parsing. Soricut and Marcu showed that sentence-internal trees can largely be derived from syntax. Given gold syntax and gold segmentation, their parser came within about 1.5 F points of human agreement. Perfect segmentation helped more than perfect syntactic parses, which marks segmentation rather than syntactic parsing as the main bottleneck ([Soricut & Marcu 2003](https://md.research.elamur.ai/free/en/papers/soricut-2003-sentence-level-discourse-parsing-syntactic/)). Document-level parsing proved harder. Rich features help, but performance drops sharply for relations spanning sentences ([Feng & Hirst 2012](https://md.research.elamur.ai/free/en/papers/feng-2012-text-level-discourse-parsing-rich/)). Joty et al. combined intra- and multi-sentential CRFs with optimal bottom-up search ([Joty et al. 2013](https://md.research.elamur.ai/free/en/papers/joty-2013-combining-intra-multi-sentential-rhetorical/)). Representation learning followed. A shift-reduce parser over a learned latent space raised labelled relation F-measure on RST-DT by roughly 6 points over the prior best system, that of Joty et al. (2013) ([Ji & Eisenstein 2014](https://md.research.elamur.ai/free/en/papers/ji-2014-representation-learning-text-level-discourse/)). Recursive neural models scored 84.0 span, 70.8 nuclearity and 58.6 relation F, on par with contemporary parsers without leading on any single metric ([Li et al. 2014](https://md.research.elamur.ai/free/en/papers/li-2014-recursive-deep-models-discourse-parsing/)). A two-stage design that first builds an unlabelled tree and then labels relations scored 86.0 span, 72.4 nuclearity and 59.7 relation F on RST-DT ([Wang et al. 2017](https://md.research.elamur.ai/free/en/papers/wang-2017-two-stage-parsing-method-text/)). For comparison, reported human agreement is 88.70, 77.72 and 65.75 ([Joty et al. 2013](https://md.research.elamur.ai/free/en/papers/joty-2013-combining-intra-multi-sentential-rhetorical/)).
- Implicit relation recognition. Marcu and Echihabi generated training pairs by stripping cue phrases from about a billion words of text ([Marcu & Echihabi 2002](https://md.research.elamur.ai/free/en/papers/marcu-2002-unsupervised-approach-recognizing-discourse-relations/)). Later work showed that such synthetic examples transfer poorly to naturally implicit relations, and it evaluated on the PDTB's natural distribution instead ([Pitler et al. 2009](https://md.research.elamur.ai/free/en/papers/pitler-2009-automatic-sense-prediction-implicit-discourse/)). Successive gains came from: predicting the missing connective with a language model ([Zhou et al. 2010](https://md.research.elamur.ai/free/en/papers/zhou-2010-predicting-discourse-connectives-implicit-discourse/)); feature selection ([Park & Cardie 2012](https://md.research.elamur.ai/free/en/papers/park-2012-improving-implicit-discourse-relation-recognition/)); Brown cluster pairs and coreference patterns ([Rutherford & Xue 2014](https://md.research.elamur.ai/free/en/papers/rutherford-2014-discovering-implicit-discourse-relations-through/)); entity-augmented neural representations ([Ji & Eisenstein 2015](https://md.research.elamur.ai/free/en/papers/ji-2015-one-vector-is-not-enough/)); multi-task learning across PDTB, RST-DT and noisy news data ([Liu et al. 2016](https://md.research.elamur.ai/free/en/papers/liu-2016-implicit-discourse-relation-classification-multi/)).
- Shared benchmarks. The CoNLL-2015 shared task defined end-to-end shallow discourse parsing. Sixteen teams took part. The best system reached only 24.0% end-to-end parser F1, which scores connectives, arguments and senses together, on a blind Wikinews test set ([Xue et al. 2015](https://md.research.elamur.ai/free/en/papers/xue-2015-conll-2015-shared-task-shallow/)). The 2016 edition added Chinese and drew 23 teams ([Xue et al. 2016](https://md.research.elamur.ai/free/en/papers/xue-2016-conll-2016-shared-task-multilingual/)).
- Downstream applications. Discourse structure has served as a signal for several tasks: evaluating coherence ([Lin et al. 2011](https://md.research.elamur.ai/free/en/papers/lin-2011-automatically-evaluating-text-coherence-discourse/)); review summarization ([Gerani et al. 2014](https://md.research.elamur.ai/free/en/papers/gerani-2014-abstractive-summarization-product-reviews-discourse/)); sentiment analysis, where a recursive network over RST trees reached 84.1% accuracy on the Pang and Lee (2004) set of 2,000 reviews and 85.6% on a 50,000-review set, against 82.4% and 81.5% for a bag-of-words classifier ([Bhatia et al. 2015](https://md.research.elamur.ai/free/en/papers/bhatia-2015-better-document-level-sentiment-analysis/)); text categorization ([Ji & Smith 2017](https://md.research.elamur.ai/free/en/papers/ji-2017-neural-discourse-structure-text-categorization/)); argumentation mining ([Peldszus & Stede 2015](https://md.research.elamur.ai/free/en/papers/peldszus-2015-joint-prediction-mst-style-discourse/)).
- Dialogue. The STAC corpus annotates multi-party game chats with full SDRT graphs and links verbal moves to non-verbal game events ([Asher et al. 2016](https://md.research.elamur.ai/free/en/papers/asher-2016-discourse-structure-dialogue-acts-multiparty/)).

## Open questions

Several issues remain unsettled.

Structure is still contested: tree, flat relations or general graph. Corpus evidence of crossing dependencies ([Wolf & Gibson 2005](https://md.research.elamur.ai/free/en/papers/wolf-2005-representing-discourse-coherence-corpus-based/)) and dialogue structures that trees cannot represent ([Asher et al. 2016](https://md.research.elamur.ai/free/en/papers/asher-2016-discourse-structure-dialogue-acts-multiparty/)) argue for graphs. Anaphoric treatment of adverbials offers a way to keep a simpler backbone ([Webber et al. 2003](https://md.research.elamur.ai/free/en/papers/webber-2003-anaphora-discourse-structure/)).

Moore and Pollack's informational/intentional split implies that one label per pair of segments is not enough ([Moore & Pollack 1992](https://md.research.elamur.ai/free/en/papers/moore-1992-problem-rst-need-multi-level/)). The RST-style scheme described above still gives each tree node a single rhetorical relation ([Carlson et al. 2001](https://md.research.elamur.ai/free/en/papers/carlson-2001-building-discourse-tagged-corpus-framework/)).

Implicit relations remain the weakest component. Error analyses point to ambiguity between senses, the need for inference, wider context and world knowledge ([Lin et al. 2009](https://md.research.elamur.ai/free/en/papers/lin-2009-recognizing-implicit-discourse-relations-penn/)). Parser errors also limit the downstream uses that depend on implicit relations ([Lin et al. 2011](https://md.research.elamur.ai/free/en/papers/lin-2011-automatically-evaluating-text-coherence-discourse/)).

Data remain scarce, and for English the two largest corpora are both built on WSJ news. News-trained discourse parsers transfer poorly to other genres. Legislative bills are one example: there, a discourse-based text classifier failed to beat earlier systems, which its authors attributed to the parser's training domain ([Ji & Smith 2017](https://md.research.elamur.ai/free/en/papers/ji-2017-neural-discourse-structure-text-categorization/)). Errors also compound across pipeline stages ([Xue et al. 2015](https://md.research.elamur.ai/free/en/papers/xue-2015-conll-2015-shared-task-shallow/)).

Finally, cross-sentence relations are where document-level parsing falls furthest short ([Feng & Hirst 2012](https://md.research.elamur.ai/free/en/papers/feng-2012-text-level-discourse-parsing-rich/)). As of 2017, even the best parsers remained below the human agreement ceiling on relation labels ([Wang et al. 2017](https://md.research.elamur.ai/free/en/papers/wang-2017-two-stage-parsing-method-text/)).

## Key papers

The works this page cites, the most cited first; citations as counted by OpenAlex plus the papers of this library that cite the work, read 2026-10-23.

- [The Penn Discourse TreeBank 2.0.](https://md.research.elamur.ai/free/en/papers/prasad-2008-penn-discourse-treebank-2-0/) — Rashmi Prasad, Nikhil Dinesh, Alan Lee et al. · 2008 · 1,793 citations
- [Building a Discourse-Tagged Corpus in the Framework of Rhetorical Structure Theory](https://md.research.elamur.ai/free/en/papers/carlson-2001-building-discourse-tagged-corpus-framework/) — Lynn Carlson, Daniel Marcu, Mary Ellen Okurovsky · 2001 · 912 citations
- [Sentence Level Discourse Parsing using Syntactic and Lexical Information](https://md.research.elamur.ai/free/en/papers/soricut-2003-sentence-level-discourse-parsing-syntactic/) — Radu Soricut, Daniel Marcu · 2003 · 560 citations
- [An Unsupervised Approach to Recognizing Discourse Relations](https://md.research.elamur.ai/free/en/papers/marcu-2002-unsupervised-approach-recognizing-discourse-relations/) — Daniel Marcu, Abdessamad Echihabi · 2002 · 513 citations
- [Automatic sense prediction for implicit discourse relations in text](https://md.research.elamur.ai/free/en/papers/pitler-2009-automatic-sense-prediction-implicit-discourse/) — Emily Pitler, Annie Louis, Ani Nenkova · 2009 · 418 citations
- [The Penn Discourse Treebank](https://md.research.elamur.ai/free/en/papers/miltsakaki-2004-penn-discourse-treebank/) — Eleni Miltsakaki, Rashmi Prasad, Aravind Joshi et al. · 2004 · 415 citations
- [Recognizing Implicit Discourse Relations in the Penn Discourse Treebank](https://md.research.elamur.ai/free/en/papers/lin-2009-recognizing-implicit-discourse-relations-penn/) — Ziheng Lin, Min-Yen Kan, Hwee Tou Ng · 2009 · 412 citations
- [Representing Discourse Coherence: A Corpus-Based Study](https://md.research.elamur.ai/free/en/papers/wolf-2005-representing-discourse-coherence-corpus-based/) — Florian Wolf, Edward Gibson · 2005 · 377 citations
- [Representation Learning for Text-level Discourse Parsing](https://md.research.elamur.ai/free/en/papers/ji-2014-representation-learning-text-level-discourse/) — Yangfeng Ji, Jacob Eisenstein · 2014 · 344 citations
- [Anaphora and Discourse Structure](https://md.research.elamur.ai/free/en/papers/webber-2003-anaphora-discourse-structure/) — Bonnie Webber, Matthew Stone, Aravind Joshi et al. · 2003 · 315 citations
- [A Problem for RST: The Need for Multi-Level Discourse Analysis](https://md.research.elamur.ai/free/en/papers/moore-1992-problem-rst-need-multi-level/) — Johanna D. Moore, Martha E. Pollack · 1992 · 280 citations
- [Abstractive Summarization of Product Reviews Using Discourse Structure](https://md.research.elamur.ai/free/en/papers/gerani-2014-abstractive-summarization-product-reviews-discourse/) — Shima Gerani, Yashar Mehdad, Giuseppe Carenini et al. · 2014 · 260 citations
- [Better Document-level Sentiment Analysis from RST Discourse Parsing](https://md.research.elamur.ai/free/en/papers/bhatia-2015-better-document-level-sentiment-analysis/) — Parminder Bhatia, Yangfeng Ji, Jacob Eisenstein · 2015 · 235 citations
- [Joint prediction in MST-style discourse parsing for argumentation mining](https://md.research.elamur.ai/free/en/papers/peldszus-2015-joint-prediction-mst-style-discourse/) — Andreas Peldszus, Manfred Stede · 2015 · 219 citations
- [Automatically Evaluating Text Coherence Using Discourse Relations](https://md.research.elamur.ai/free/en/papers/lin-2011-automatically-evaluating-text-coherence-discourse/) — Ziheng Lin, Hwee Tou Ng, Min-Yen Kan · 2011 · 218 citations
- [Combining Intra- and Multi-sentential Rhetorical Parsing for Document-level Discourse Analysis](https://md.research.elamur.ai/free/en/papers/joty-2013-combining-intra-multi-sentential-rhetorical/) — Shafiq Joty, Giuseppe Carenini, Raymond Ng et al. · 2013 · 217 citations
- [The CoNLL-2015 Shared Task on Shallow Discourse Parsing](https://md.research.elamur.ai/free/en/papers/xue-2015-conll-2015-shared-task-shallow/) — Nianwen Xue, Hwee Tou Ng, Sameer Pradhan et al. · 2015 · 210 citations
- [Text-level Discourse Parsing with Rich Linguistic Features](https://md.research.elamur.ai/free/en/papers/feng-2012-text-level-discourse-parsing-rich/) — Vanessa Wei Feng, Graeme Hirst · 2012 · 208 citations
- [Recursive Deep Models for Discourse Parsing](https://md.research.elamur.ai/free/en/papers/li-2014-recursive-deep-models-discourse-parsing/) — Jiwei Li, Rumeng Li, Eduard Hovy · 2014 · 206 citations
- [Easily Identifiable Discourse Relations](https://md.research.elamur.ai/free/en/papers/pitler-2008-easily-identifiable-discourse-relations/) — Emily Pitler, Mridhula Raghupathy, Hena Mehta et al. · 2008 · 206 citations
- [Discourse Structure and Dialogue Acts in Multiparty Dialogue: the STAC Corpus](https://md.research.elamur.ai/free/en/papers/asher-2016-discourse-structure-dialogue-acts-multiparty/) — Nicholas Asher, Julie Hunter, Mathieu Morey et al. · 2016 · 202 citations
- [One Vector is Not Enough: Entity-Augmented Distributed Semantics for Discourse Relations](https://md.research.elamur.ai/free/en/papers/ji-2015-one-vector-is-not-enough/) — Yangfeng Ji, Jacob Eisenstein · 2015 · 198 citations
- [Discovering Implicit Discourse Relations Through Brown Cluster Pair Representation and Coreference Patterns](https://md.research.elamur.ai/free/en/papers/rutherford-2014-discovering-implicit-discourse-relations-through/) — Attapol Rutherford, Nianwen Xue · 2014 · 191 citations
- [CoNLL 2016 Shared Task on Multilingual Shallow Discourse Parsing](https://md.research.elamur.ai/free/en/papers/xue-2016-conll-2016-shared-task-multilingual/) — Nianwen Xue, Hwee Tou Ng, Sameer Pradhan et al. · 2016 · 187 citations
- [Neural Discourse Structure for Text Categorization](https://md.research.elamur.ai/free/en/papers/ji-2017-neural-discourse-structure-text-categorization/) — Yangfeng Ji, Noah A. Smith · 2017 · 180 citations
- [Discourse Relations and Defeasible Knowledge](https://md.research.elamur.ai/free/en/papers/lascarides-1991-discourse-relations-defeasible-knowledge/) — Alex Lascarides, Nicholas Asher · 1991 · 177 citations
- [Improving Implicit Discourse Relation Recognition Through Feature Set Optimization](https://md.research.elamur.ai/free/en/papers/park-2012-improving-implicit-discourse-relation-recognition/) — Joonsuk Park, Claire Cardie · 2012 · 165 citations
- [A Two-Stage Parsing Method for Text-Level Discourse Analysis](https://md.research.elamur.ai/free/en/papers/wang-2017-two-stage-parsing-method-text/) — Yizhong Wang, Sujian Li, Houfeng Wang · 2017 · 164 citations
- [Predicting Discourse Connectives for Implicit Discourse Relation Recognition](https://md.research.elamur.ai/free/en/papers/zhou-2010-predicting-discourse-connectives-implicit-discourse/) — Zhi-Min Zhou, Yu Xu, Zheng-Yu Niu et al. · 2010 · 163 citations
- [Implicit Discourse Relation Classification via Multi-Task Neural Networks](https://md.research.elamur.ai/free/en/papers/liu-2016-implicit-discourse-relation-classification-multi/) — Yang Liu, Sujian Li, Xiaodong Zhang et al. · 2016 · 162 citations

## Related concepts

- [Coreference resolution](https://md.research.elamur.ai/free/en/concepts/coreference-resolution/) — Coreference resolution is the task of finding the expressions in a text that refer to the same entity as one another, such as names…
- [Implicature](https://md.research.elamur.ai/free/en/concepts/implicature/) — Implicature is the part of what a speaker communicates that goes beyond the literal content of the words. Hearers infer it by assuming the…
- [Semantic role labeling](https://md.research.elamur.ai/free/en/concepts/semantic-role-labeling/) — Semantic role labeling is the task of finding, for each predicate in a sentence, the phrases that act as its participants and labeling each…
- [Corpus linguistics](https://md.research.elamur.ai/free/en/concepts/corpus-linguistics/) — Corpus linguistics is the empirical study of language through large, principled collections of naturally occurring text or transcribed…

## More papers

Topics: [🧵 Discourse analysis](https://md.research.elamur.ai/free/en/topics/discourse-analysis/) · [🔗 Coreference resolution](https://md.research.elamur.ai/free/en/topics/coreference-resolution/)

Shelves: [🗣️ Linguistics & languages](https://md.research.elamur.ai/free/en/shelves/linguistics/)

- [← previous concept Dependency parsing](https://md.research.elamur.ai/free/en/concepts/dependency-parsing/)
- [next concept → Distributional semantics](https://md.research.elamur.ai/free/en/concepts/distributional-semantics/)
- [↑ back to concepts](https://md.research.elamur.ai/free/en/concepts/)