---
title: "Annotating Discourse Connectives and Their — Miltsakaki 2004"
description: "The Penn Treebank and Propbank annotate structure only within sentences."
authors: ["Eleni Miltsakaki", "Aravind Joshi", "Rashmi Prasad", "Bonnie Webber"]
year: 2004
venue: "Proceedings of the Workshop Frontiers in Corpus Annotation at HLT-NAACL 2004"
source: "https://aclanthology.org/W04-2703/"
licence: "CC BY-NC-SA 3.0 (ACL Anthology, before 2016)"
licence_url: "https://creativecommons.org/licenses/by-nc-sa/3.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/miltsakaki-2004-annotating-discourse-connectives-their-arguments/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/miltsakaki-2004-annotating-discourse-connectives-their-arguments/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2004](https://md.research.elamur.ai/free/en/papers/2004/) / Annotating Discourse Connectives and Their Arguments

# Annotating Discourse Connectives and Their Arguments

**Authors:** [Eleni Miltsakaki](https://md.research.elamur.ai/free/en/authors/eleni-miltsakaki/), [Aravind Joshi](https://md.research.elamur.ai/free/en/authors/aravind-joshi/), [Rashmi Prasad](https://md.research.elamur.ai/free/en/authors/rashmi-prasad/), [Bonnie Webber](https://md.research.elamur.ai/free/en/authors/bonnie-webber/) · **Year:** 2004 · **Venue:** Proceedings of the Workshop Frontiers in Corpus Annotation at HLT-NAACL 2004 · **Licence:** [CC BY-NC-SA 3.0 (ACL Anthology, before 2016)](https://creativecommons.org/licenses/by-nc-sa/3.0/)

**Links:** [Publisher's page](https://aclanthology.org/W04-2703/) · [PDF](https://aclanthology.org/W04-2703.pdf) · [OpenAlex](https://openalex.org/W192533699)

**Citations:** 83 citations (OpenAlex, 2026-10-14) · [39](https://md.research.elamur.ai/free/en/papers/miltsakaki-2004-annotating-discourse-connectives-their-arguments/cited-by/) in our library, of the 140884 papers whose bibliographies we read

## Summary
The Penn Treebank and Propbank annotate structure only within sentences. Corpora built on abstract rhetorical relations, such as RST-style annotation, have proven hard to label reliably and do not record why a relation was chosen. This paper introduces the Penn Discourse TreeBank (PDTB), which takes a bottom-up, lexically anchored approach. Discourse connectives are treated as two-place predicates, analogous to verbs at sentence level, and annotators mark the spans of their two arguments.

The approach is motivated by DLTAG, but the guidelines are deliberately framework-neutral. Annotation is layered on the Penn Treebank and Propbank. It covers subordinating and coordinating conjunctions, adverbial connectives, and implicit connectives between adjacent sentences. For implicit cases, annotators also supply an explicit connective that captures the inferred relation.

Preliminary agreement studies show that argument spans can be annotated reliably. Most disagreements are partial overlaps in span boundaries, not genuine conflicts. This suggests the PDTB is a dependable resource for building and evaluating discourse-processing algorithms.

## Key points
- **Planned scope:** about 30,000 annotations, with roughly 20,000 for the 250 explicit connectives found in the corpus and 10,000 implicit ones. Release was expected by November 2005.
- **Data reported so far:** 10 explicit connectives totalling 2,717 annotations, plus 386 implicit-connective tokens. The explicit connectives are the adverbials 'therefore', 'as a result', 'instead', 'otherwise' and 'nevertheless', together with the subordinators 'so that', 'when', 'even though', 'although' and 'because'.
- **Annotation process:** four annotators per token, working in a modified version of the WordFreak tool and handling one connective at a time.
- **Agreement on implicit connectives:**
  - Arguments: 85.1% exact-match agreement between two annotators (657 of 772 ARG1/ARG2 tokens), rising to 92.6% under partial match.
  - Inferred relation type: 72% agreement across five coarse groups (addition, cause-effect, temporal, contrast, restatement).
- **Agreement on explicit connectives and sources of disagreement:** partial-match agreement reached about 94.5% overall. The main challenge is determining argument extent. Disagreements are classified as partial overlap (e.g., parentheticals, dependent clauses), no overlap, missing annotations, or unresolved guideline issues.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

Fields: [Linguistics & languages](https://md.research.elamur.ai/free/en/shelves/linguistics/)

Topics: [Discourse analysis](https://md.research.elamur.ai/free/en/topics/discourse-analysis/)

## Cite

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{miltsakaki2004annotating,
  title = {{Annotating Discourse Connectives and Their Arguments}},
  author = {Miltsakaki, Eleni and Joshi, Aravind and Prasad, Rashmi and Webber, Bonnie},
  booktitle = {Proceedings of the Workshop Frontiers in Corpus Annotation at HLT-NAACL 2004},
  year = {2004},
  url = {https://aclanthology.org/W04-2703/}
}
```

- [← previous The MATE/GNOME Proposals for Anaphoric Annotation, Revisited](https://md.research.elamur.ai/free/en/papers/poesio-2004-mate-gnome-proposals-anaphoric-annotation/)
- [next → Unsupervised Learning of Contextual Role Knowledge for Coreference Resolution](https://md.research.elamur.ai/free/en/papers/bean-2004-unsupervised-learning-contextual-role-knowledge/)
- [↑ back to papers of 2004](https://md.research.elamur.ai/free/en/papers/2004/)

## References

18 references, 5 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Nicholas Asher and Alex Lascarides. 1998. The semantics and pragmatics of presupposition. Journal of Semantics, 15(3):239–300. [find](https://scholar.google.com/scholar?q=Nicholas+Asher+and+Alex+Lascarides%2E+1998%2E+The+semantics+and+pragmatics+of+presupposition%2E+Journal+of+Semantics%2C+15%283%29%3A239%E2%80%93300%2E)
2. Jean Carletta. 1996. Assessing agreement on classification tasks. Computational Linguistics, 22:249–254. [find](https://scholar.google.com/scholar?q=Jean+Carletta%2E+1996%2E+Assessing+agreement+on+classification+tasks%2E+Computational+Linguistics%2C+22%3A249%E2%80%93254%2E)
3. Lynn Carlson, Daniel Marcu, and Mary Ellen Okurowski, 2003. Current Directions in Discourse and Dialogue, chapter Building a Discourse-Tagged Corpus in the Framework of Rhetorical Structure Theory. Kluwer Academic Publishers. [find](https://scholar.google.com/scholar?q=Lynn+Carlson%2C+Daniel+Marcu%2C+and+Mary+Ellen+Okurowski%2C+2003%2E+Current+Directions+in+Discourse+and+Dialogue%2C+chapter+Building+a+Discourse%2DTagged+Corpus+in+the+Framework+of+Rhetorical+Structure+Theory%2E+Kl) [our card](https://md.research.elamur.ai/free/en/papers/carlson-2001-building-discourse-tagged-corpus-framework/)
4. Kate Forbes. 2003. Discourse Semantics of S-Modifying Adverbials. Ph.D. thesis, Department of Linguistics, University of Pennsylvania. [find](https://scholar.google.com/scholar?q=Kate+Forbes%2E+2003%2E+Discourse+Semantics+of+S%2DModifying+Adverbials%2E+Ph%2ED%2E+thesis%2C+Department+of+Linguistics%2C+University+of+Pennsylvania%2E)
5. Claire Gardent. 1997. Discourse tree adjoining grammars. Claus 89, University of the Saarlandes, Saarbrucken. [find](https://scholar.google.com/scholar?q=Claire+Gardent%2E+1997%2E+Discourse+tree+adjoining+grammars%2E+Claus+89%2C+University+of+the+Saarlandes%2C+Saarbrucken%2E)
6. Paul Kingsbury and Martha Palmer. 2002. From Treebank to Propbank. In Third International Conference on Language Resources and Evaluation, LREC-02, Las Palmas, Canary Islands, Spain. [find](https://scholar.google.com/scholar?q=Paul+Kingsbury+and+Martha+Palmer%2E+2002%2E+From+Treebank+to+Propbank%2E+In+Third+International+Conference+on+Language+Resources+and+Evaluation%2C+LREC%2D02%2C+Las+Palmas%2C+Canary+Islands%2C+Spain%2E) [our card](https://md.research.elamur.ai/free/en/papers/kingsbury-2002-treebank-propbank/)
7. Alistair Knott. 1996. A Data-Driven Methodology for Motivating a Set of Coherence Relations. Ph.D. thesis, University of Edinburgh. [find](https://scholar.google.com/scholar?q=Alistair+Knott%2E+1996%2E+A+Data%2DDriven+Methodology+for+Motivating+a+Set+of+Coherence+Relations%2E+Ph%2ED%2E+thesis%2C+University+of+Edinburgh%2E)
8. William Mann and Sandra Thompson. 1988. Rhetorical structure theory. toward a functional theory of text organization. Text, 8(3):243–281. [find](https://scholar.google.com/scholar?q=William+Mann+and+Sandra+Thompson%2E+1988%2E+Rhetorical+structure+theory%2E+toward+a+functional+theory+of+text+organization%2E+Text%2C+8%283%29%3A243%E2%80%93281%2E) [our card](https://md.research.elamur.ai/free/en/papers/enud-1988-mann-rhetorical-structure-theory-functional/)
9. Mitch Marcus, Beatrice Santorini, and Mary Ann Marcinkiewicz. 1993. Building a large annotated corpus of english: the Penn Treebank. Computational Linguistics, 19:313–330. [find](https://scholar.google.com/scholar?q=Mitch+Marcus%2C+Beatrice+Santorini%2C+and+Mary+Ann+Marcinkiewicz%2E+1993%2E+Building+a+large+annotated+corpus+of+english%3A+the+Penn+Treebank%2E+Computational+Linguistics%2C+19%3A313%E2%80%93330%2E) [our card](https://md.research.elamur.ai/free/en/papers/bank-1993-marcus-building-large-annotated-corpus-english/)
10. Eleni Miltsakaki, Cassandre Creswell, Kate Forbes, Aravind Joshi, and Bonnie Webber. 2003. Anaphoric arguments of discourse connectives: Semantic properties of antecedents versus non-antecedents. In Proceedings of the Computational Treatment of Anaphora Workshop, EACL 2003, Budapest. [find](https://scholar.google.com/scholar?q=Eleni+Miltsakaki%2C+Cassandre+Creswell%2C+Kate+Forbes%2C+Aravind+Joshi%2C+and+Bonnie+Webber%2E+2003%2E+Anaphoric+arguments+of+discourse+connectives%3A+Semantic+properties+of+antecedents+versus+non%2Dantecedents%2E+In+P)
11. Livia Polanyi and Martin van den Berg. 1996. Discourse structure and discourse interpretation. In Proceedings of the Tenth Amsterdam Colloquium, University of Amsterdam, pages 113–131. [find](https://scholar.google.com/scholar?q=Livia+Polanyi+and+Martin+van+den+Berg%2E+1996%2E+Discourse+structure+and+discourse+interpretation%2E+In+Proceedings+of+the+Tenth+Amsterdam+Colloquium%2C+University+of+Amsterdam%2C+pages+113%E2%80%93131%2E)
12. Frank Schilder. 1997. Discourse tree grammar or how to get attached to a discourse? In Proceedings of the the second International Workshop on Computational Semantics (IWCS-II), Tilburg, The Netherlands, pages 261–273. [find](https://scholar.google.com/scholar?q=Frank+Schilder%2E+1997%2E+Discourse+tree+grammar+or+how+to+get+attached+to+a+discourse%3F+In+Proceedings+of+the+the+second+International+Workshop+on+Computational+Semantics+%28IWCS%2DII%29%2C+Tilburg%2C+The+Netherlan)
13. Sidney Siegel and N. J. Castellan. 1988. Nonparama-teric Statistics for the Behavioral Sciences. McGraw-Hill, 2nd edition. [find](https://scholar.google.com/scholar?q=Sidney+Siegel+and+N%2E+J%2E+Castellan%2E+1988%2E+Nonparama%2Dteric+Statistics+for+the+Behavioral+Sciences%2E+McGraw%2DHill%2C+2nd+edition%2E)
14. Bonnie Webber and Aravind Joshi. 1998. Anchoring a lexicalized tree adjoining grammar for discourse. In ACL/COLING Workshop on Discourse Relations and Discourse Markers, Montreal, pages 8–92. Montreal, Canada. [find](https://scholar.google.com/scholar?q=Bonnie+Webber+and+Aravind+Joshi%2E+1998%2E+Anchoring+a+lexicalized+tree+adjoining+grammar+for+discourse%2E+In+ACL%2FCOLING+Workshop+on+Discourse+Relations+and+Discourse+Markers%2C+Montreal%2C+pages+8%E2%80%9392%2E+Montreal)
15. Bonnie Webber, Alistair Knott, Matthew Stone, and Aravind Joshi. 1999a. Discourse relations: A structural and presuppositional account using lexicalized TAG. In Proceedings of the 37th Annual Meeting of the Association for Computational Linguistics, Maryland, pages 41–48. College Park MD. [find](https://scholar.google.com/scholar?q=Bonnie+Webber%2C+Alistair+Knott%2C+Matthew+Stone%2C+and+Aravind+Joshi%2E+1999a%2E+Discourse+relations%3A+A+structural+and+presuppositional+account+using+lexicalized+TAG%2E+In+Proceedings+of+the+37th+Annual+Meeting+)
16. Bonnie Webber, Alistair Knott, Matthew Stone, and Aravind Joshi. 1999b. What are little texts made of? A structural and presuppositional account using lexicalized TAG. In Proceedings of the International Workshop on Levels of Representation in Discourse (LORID ’99), Edinburgh, pages 145–149. [find](https://scholar.google.com/scholar?q=Bonnie+Webber%2C+Alistair+Knott%2C+Matthew+Stone%2C+and+Aravind+Joshi%2E+1999b%2E+What+are+little+texts+made+of%3F+A+structural+and+presuppositional+account+using+lexicalized+TAG%2E+In+Proceedings+of+the+Internatio)
17. Bonnie Webber, Alistair Knott, and Aravind Joshi. 2000. Multiple discourse connectives in a lexicalized grammar for discourse. In Proceedings of the Third International Workshop on Computational Semantics, Tilburg, The Netherlands. [find](https://scholar.google.com/scholar?q=Bonnie+Webber%2C+Alistair+Knott%2C+and+Aravind+Joshi%2E+2000%2E+Multiple+discourse+connectives+in+a+lexicalized+grammar+for+discourse%2E+In+Proceedings+of+the+Third+International+Workshop+on+Computational+Seman)
18. Bonnie Webber, Matthew Stone, Aravind Joshi, and Alistair Knott. 2003. Anaphora and discourse structure. Computational Linguistics, 29:545–587. [find](https://scholar.google.com/scholar?q=Bonnie+Webber%2C+Matthew+Stone%2C+Aravind+Joshi%2C+and+Alistair+Knott%2E+2003%2E+Anaphora+and+discourse+structure%2E+Computational+Linguistics%2C+29%3A545%E2%80%93587%2E) [our card](https://md.research.elamur.ai/free/en/papers/webber-2003-anaphora-discourse-structure/)