---
title: "Examples and Specifications that Prove a — Scholman 2017"
description: "Examples and specifications are among the most frequent discourse relations, but annotated corpora often disagree on how to label them."
authors: ["Merel C.J. Scholman", "Vera Demberg"]
year: 2017
venue: "Dialogue & Discourse"
source: "https://aclanthology.org/2017.dnd-8.8/"
doi: "10.5087/dad.2017.203"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/scholman-2017-examples-specifications-that-prove-point/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/scholman-2017-examples-specifications-that-prove-point/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2017](https://md.research.elamur.ai/free/en/papers/2017/page/2/) / Examples and Specifications that Prove a Point: Identifying Elaborative and Argumentative Discourse Relations

# Examples and Specifications that Prove a Point: Identifying Elaborative and Argumentative Discourse Relations

**Authors:** [Merel C.J. Scholman](https://md.research.elamur.ai/free/en/authors/merel-c-j-scholman/), [Vera Demberg](https://md.research.elamur.ai/free/en/authors/vera-demberg/) · **Year:** 2017 · **Venue:** Dialogue & Discourse · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2017.dnd-8.8/) · [PDF](https://aclanthology.org/2017.dnd-8.8.pdf) · [DOI 10.5087/dad.2017.203](https://doi.org/10.5087/dad.2017.203) · [OpenAlex](https://openalex.org/W2745399545)

**Citations:** 16 citations (openalex-snapshot · 2026-10-14) · [15](#cited-by) in our library, of the 140673 papers whose bibliographies we read

## Summary
Examples and specifications are among the most frequent discourse relations, but annotated corpora often disagree on how to label them. Using the WSJ-Aligned corpus, which maps Penn Discourse Treebank (PDTB) and RST-DT annotations of the same Wall Street Journal texts onto each other, Scholman and Demberg show that PDTB's INSTANTIATION and SPECIFICATION relations are often labelled as argumentative (causal) in RST-DT. The reverse pattern is much rarer.

To test how ordinary readers actually understand these relations, they ran a crowdsourcing study. Untrained participants inserted connectives, chosen from a small and fairly unambiguous predefined list, between the two segments. A replication without surrounding context gave highly similar results.

Readers frequently interpret these relations as argumentative. Examples and specifications can therefore illustrate a situation and support a claim at the same time. The authors argue that annotation schemes should capture such multiple simultaneous readings. They also see connective-insertion crowdsourcing as a cheap and replicable way to obtain distributions of relation senses, and potentially to build a corpus.

## Key points
- **Agreement between frameworks:** PDTB and RST-DT agree on about 60% of explicit relations but only about 35% of implicit ones. Agreement is just 32% for INSTANTIATION and 14% for SPECIFICATION.
- **Frequency:** INSTANTIATION and SPECIFICATION together make up 24% of implicit relations in the PDTB.
- **Asymmetry between frameworks:**
  - Among 306 PDTB INSTANTIATION relations, 25% receive causal labels (EXPLANATION-ARGUMENTATIVE, EVIDENCE) in RST-DT.
  - Among 426 PDTB SPECIFICATION relations, the figure is 21%.
  - In the other direction, only 7% of RST EXAMPLE and 12% of RST GENERAL-SPECIFIC relations get causal labels in the PDTB.
- **Use of the second connective:** Participants could insert a second connective but rarely did. The authors suggest making a second choice mandatory, with a "no other connective fits" option.
- **Domain knowledge:** Items with highly scattered answers likely reflect participants' limited knowledge of economics, which the authors note could also affect trained annotators.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

Fields: [Linguistics & languages](https://md.research.elamur.ai/free/en/shelves/linguistics/)

Topics: [Discourse analysis](https://md.research.elamur.ai/free/en/topics/discourse-analysis/)

## Cited by

Cited by

15 papers of our library cite it (of the 140673 whose bibliographies we read); 6 of them have a page here, 9 not yet.

- [Exploring the Potential of ChatGPT on Sentence Level Relations: A Focus on Temporal, Causal, and Discourse Relations](https://md.research.elamur.ai/free/en/papers/chan-2024-exploring-potential-chatgpt-sentence-level/)
- [Is there less annotator agreement when the discourse relation is underspecified?](https://md.research.elamur.ai/free/en/papers/hoek-2021-is-there-less-annotator-agreement/)
- [Extending Implicit Discourse Relation Recognition to the PDTB-3](https://md.research.elamur.ai/free/en/papers/liang-2020-extending-implicit-discourse-relation-recognition/)
- [Multi-Label Classification for Implicit Discourse Relation Recognition](https://md.research.elamur.ai/free/en/papers/long-2024-multi-label-classification-implicit-discourse/)
- [DiscoGeM: A Crowdsourced Corpus of Genre-Mixed Implicit Discourse Relations](https://md.research.elamur.ai/free/en/papers/scholman-2022-discogem-crowdsourced-corpus-genre-mixed/)
- [Crowdsourcing Discourse Relation Annotations by a Two-Step Connective Insertion Task](https://md.research.elamur.ai/free/en/papers/yung-2019-crowdsourcing-discourse-relation-annotations-two/)

## Cite

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{scholman2017examples,
  title = {{Examples and Specifications that Prove a Point: Identifying Elaborative and Argumentative Discourse Relations}},
  author = {Scholman, Merel C.J. and Demberg, Vera},
  booktitle = {Dialogue \& Discourse},
  year = {2017},
  doi = {10.5087/dad.2017.203},
  url = {https://doi.org/10.5087/dad.2017.203}
}
```

- [← previous Event Detection Using Frame-Semantic Parser](https://md.research.elamur.ai/free/en/papers/spiliopoulou-2017-event-detection-frame-semantic-parser/)
- [next → Explanation in Artificial Intelligence: Insights from the Social Sciences](https://md.research.elamur.ai/free/en/papers/miller-2017-explanation-artificial-intelligence-insights-social/)
- [↑ back to papers of 2017](https://md.research.elamur.ai/free/en/papers/2017/page/2/)

## References

54 references, 10 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Artstein, R., & Poesio, M. (2005). Bias decreases in proportion to the number of annotators. Proceedings of the Conference on Formal Grammar and Mathematics of Language (FG-MoL), (pp. 141–150). [find](https://scholar.google.com/scholar?q=Artstein%2C+R%2E%2C+%26+Poesio%2C+M%2E+%282005%29%2E+Bias+decreases+in+proportion+to+the+number+of+annotators%2E+Proceedings+of+the+Conference+on+Formal+Grammar+and+Mathematics+of+Language+%28FG%2DMoL%29%2C+%28pp%2E+141%E2%80%93150%29%2E)
2. Asr, F. T., & Demberg, V. (2013). On the information conveyed by discourse markers. In Proceedings of the Fourth Annual Workshop on Cognitive Modeling and Computational Linguistics (pp. 84–93). [find](https://scholar.google.com/scholar?q=Asr%2C+F%2E+T%2E%2C+%26+Demberg%2C+V%2E+%282013%29%2E+On+the+information+conveyed+by+discourse+markers%2E+In+Proceedings+of+the+Fourth+Annual+Workshop+on+Cognitive+Modeling+and+Computational+Linguistics+%28pp%2E+84%E2%80%9393%29%2E) [our card](https://md.research.elamur.ai/free/en/papers/asr-2013-information-conveyed-discourse-markers/)
3. Biran, O., & Rambow, O. (2011). Identifying justifications in written dialogs by classifying text as argumentative. International Journal of Semantic Computing, 5, 363–381. [find](https://scholar.google.com/scholar?q=Biran%2C+O%2E%2C+%26+Rambow%2C+O%2E+%282011%29%2E+Identifying+justifications+in+written+dialogs+by+classifying+text+as+argumentative%2E+International+Journal+of+Semantic+Computing%2C+5%2C+363%E2%80%93381%2E)
4. Blakemore, D. (1997). Restatement and exemplification: A relevance theoretic reassessment of elaboration. Pragmatics & Cognition, 5, 1–19. [find](https://scholar.google.com/scholar?q=Blakemore%2C+D%2E+%281997%29%2E+Restatement+and+exemplification%3A+A+relevance+theoretic+reassessment+of+elaboration%2E+Pragmatics+%26+Cognition%2C+5%2C+1%E2%80%9319%2E)
5. Carlson, L., & Marcu, D. (2001). Discourse tagging reference manual. ISI Technical Report ISI-TR-545, 54, 1–56. [find](https://scholar.google.com/scholar?q=Carlson%2C+L%2E%2C+%26+Marcu%2C+D%2E+%282001%29%2E+Discourse+tagging+reference+manual%2E+ISI+Technical+Report+ISI%2DTR%2D545%2C+54%2C+1%E2%80%9356%2E)
6. Carlson, L., Marcu, D., & Okurowski, M. E. (2003). Building a discourse-tagged corpus in the framework of Rhetorical Structure Theory. In Current and new directions in discourse and dialogue (pp. 85–112). Springer. [find](https://scholar.google.com/scholar?q=Carlson%2C+L%2E%2C+Marcu%2C+D%2E%2C+%26+Okurowski%2C+M%2E+E%2E+%282003%29%2E+Building+a+discourse%2Dtagged+corpus+in+the+framework+of+Rhetorical+Structure+Theory%2E+In+Current+and+new+directions+in+discourse+and+dialogue+%28pp%2E+85%E2%80%931) [our card](https://md.research.elamur.ai/free/en/papers/carlson-2001-building-discourse-tagged-corpus-framework/)
7. Carston, R. (1993). Conjunction, explanation and relevance. Lingua, 90, 27–48. [find](https://scholar.google.com/scholar?q=Carston%2C+R%2E+%281993%29%2E+Conjunction%2C+explanation+and+relevance%2E+Lingua%2C+90%2C+27%E2%80%9348%2E)
8. Cohen, R. (1987). Analyzing the structure of argumentative discourse. Computational Linguistics, 13, 11–24. [find](https://scholar.google.com/scholar?q=Cohen%2C+R%2E+%281987%29%2E+Analyzing+the+structure+of+argumentative+discourse%2E+Computational+Linguistics%2C+13%2C+11%E2%80%9324%2E)
9. Crible, L., & Degand, L. (in press). Reliability vs. granularity in discourse annotation: What is the trade-off? Corpus Linguistics and Linguistic Theory, . [find](https://scholar.google.com/scholar?q=Crible%2C+L%2E%2C+%26+Degand%2C+L%2E+%28in+press%29%2E+Reliability+vs%2E+granularity+in+discourse+annotation%3A+What+is+the+trade%2Doff%3F+Corpus+Linguistics+and+Linguistic+Theory%2C+%2E)
10. Cuenca, M.-J. (2003). Two ways to reformulate: a contrastive analysis of reformulation markers. Journal of Pragmatics, 35, 1069–1093. [find](https://scholar.google.com/scholar?q=Cuenca%2C+M%2E%2DJ%2E+%282003%29%2E+Two+ways+to+reformulate%3A+a+contrastive+analysis+of+reformulation+markers%2E+Journal+of+Pragmatics%2C+35%2C+1069%E2%80%931093%2E)
11. Cuenca, M.-J., & Marı́n, M.-J. (2009). Co-occurrence of discourse markers in Catalan and Spanish oral narrative. Journal of Pragmatics, 41, 899–914. [find](https://scholar.google.com/scholar?q=Cuenca%2C+M%2E%2DJ%2E%2C+%26+Mar%C4%B1%CC%81n%2C+M%2E%2DJ%2E+%282009%29%2E+Co%2Doccurrence+of+discourse+markers+in+Catalan+and+Spanish+oral+narrative%2E+Journal+of+Pragmatics%2C+41%2C+899%E2%80%93914%2E)
12. Degand, L. (1998). On classifying connectives and coherence relations. In Proceedings of the 1998 ACL Workshop on Discourse Relations and Discourse Markers (pp. 29–35). [find](https://scholar.google.com/scholar?q=Degand%2C+L%2E+%281998%29%2E+On+classifying+connectives+and+coherence+relations%2E+In+Proceedings+of+the+1998+ACL+Workshop+on+Discourse+Relations+and+Discourse+Markers+%28pp%2E+29%E2%80%9335%29%2E)
13. Demberg, V., Asr, F., & Scholman, M. (2017). How consistent are our discourse annotations? Insights from mapping RST-DT and PDTB annotations. ArXiv e-prints, April. arXiv:1704.08893. — [arXiv:1704.08893](https://arxiv.org/abs/1704.08893) [find](https://scholar.google.com/scholar?q=Demberg%2C+V%2E%2C+Asr%2C+F%2E%2C+%26+Scholman%2C+M%2E+%282017%29%2E+How+consistent+are+our+discourse+annotations%3F+Insights+from+mapping+RST%2DDT+and+PDTB+annotations%2E+ArXiv+e%2Dprints%2C+April%2E+arXiv%3A1704%2E08893%2E+%E2%80%94+%5BarXiv%3A1704%2E088)
14. González, M. (2005). Pragmatic markers and discourse coherence relations in English and Catalan oral narrative. Discourse Studies, 7, 53–86. [find](https://scholar.google.com/scholar?q=Gonza%CC%81lez%2C+M%2E+%282005%29%2E+Pragmatic+markers+and+discourse+coherence+relations+in+English+and+Catalan+oral+narrative%2E+Discourse+Studies%2C+7%2C+53%E2%80%9386%2E)
15. Halliday, M. A. (1994). Functional grammar. London: Edward Arnold, . [find](https://scholar.google.com/scholar?q=Halliday%2C+M%2E+A%2E+%281994%29%2E+Functional+grammar%2E+London%3A+Edward+Arnold%2C+%2E)
16. Hobbs, J. R. (1979). Coherence and coreference. Cognitive Science, 3, 67–90. [find](https://scholar.google.com/scholar?q=Hobbs%2C+J%2E+R%2E+%281979%29%2E+Coherence+and+coreference%2E+Cognitive+Science%2C+3%2C+67%E2%80%9390%2E)
17. Hovy, E. H. (1995). The multifunctionality of discourse markers. In Proceedings of the Workshop on Discourse Markers (pp. 1–11). Citeseer. [find](https://scholar.google.com/scholar?q=Hovy%2C+E%2E+H%2E+%281995%29%2E+The+multifunctionality+of+discourse+markers%2E+In+Proceedings+of+the+Workshop+on+Discourse+Markers+%28pp%2E+1%E2%80%9311%29%2E+Citeseer%2E)
18. Hovy, E. H., & Maier, E. (1995). Parsimonious or profligate: How many and which discourse structure relations. Unpublished manuscript, . [find](https://scholar.google.com/scholar?q=Hovy%2C+E%2E+H%2E%2C+%26+Maier%2C+E%2E+%281995%29%2E+Parsimonious+or+profligate%3A+How+many+and+which+discourse+structure+relations%2E+Unpublished+manuscript%2C+%2E)
19. Hyland, K. (2007). Applying a gloss: Exemplifying and reformulating in academic discourse. Applied Linguistics, 28, 266–285. [find](https://scholar.google.com/scholar?q=Hyland%2C+K%2E+%282007%29%2E+Applying+a+gloss%3A+Exemplifying+and+reformulating+in+academic+discourse%2E+Applied+Linguistics%2C+28%2C+266%E2%80%93285%2E)
20. Jasinskaja, K. (2013). Corrective elaboration. Lingua, 132, 51–66. [find](https://scholar.google.com/scholar?q=Jasinskaja%2C+K%2E+%282013%29%2E+Corrective+elaboration%2E+Lingua%2C+132%2C+51%E2%80%9366%2E)
21. Jasinskaja, K., & Karagjosova, E. (2011). Elaboration and explanation. Constraints in Discourse, 4. [find](https://scholar.google.com/scholar?q=Jasinskaja%2C+K%2E%2C+%26+Karagjosova%2C+E%2E+%282011%29%2E+Elaboration+and+explanation%2E+Constraints+in+Discourse%2C+4%2E)
22. Kawahara, D., Machida, Y., Shibata, T., Kurohashi, S., Kobayashi, H., & Sassano, M. (2014). Rapid development of a corpus with discourse annotations using two-stage crowdsourcing. In Proceedings of the International Conference on Computational Linguistics (pp. 269–278). [find](https://scholar.google.com/scholar?q=Kawahara%2C+D%2E%2C+Machida%2C+Y%2E%2C+Shibata%2C+T%2E%2C+Kurohashi%2C+S%2E%2C+Kobayashi%2C+H%2E%2C+%26+Sassano%2C+M%2E+%282014%29%2E+Rapid+development+of+a+corpus+with+discourse+annotations+using+two%2Dstage+crowdsourcing%2E+In+Proceedings+of+th)
23. Knott, A., & Dale, R. (1994). Using linguistic phenomena to motivate a set of coherence relations. Discourse Processes, 18, 35–62. [find](https://scholar.google.com/scholar?q=Knott%2C+A%2E%2C+%26+Dale%2C+R%2E+%281994%29%2E+Using+linguistic+phenomena+to+motivate+a+set+of+coherence+relations%2E+Discourse+Processes%2C+18%2C+35%E2%80%9362%2E)
24. Konig, E., & Siemund, P. (2000). Causal and concessive clauses: Formal and semantic relations. Topics in English Linguistics, 33, 341–360. [find](https://scholar.google.com/scholar?q=Konig%2C+E%2E%2C+%26+Siemund%2C+P%2E+%282000%29%2E+Causal+and+concessive+clauses%3A+Formal+and+semantic+relations%2E+Topics+in+English+Linguistics%2C+33%2C+341%E2%80%93360%2E)
25. Krippendorff, K. (1980). Content analysis: An introduction to its methodology. Sage. [find](https://scholar.google.com/scholar?q=Krippendorff%2C+K%2E+%281980%29%2E+Content+analysis%3A+An+introduction+to+its+methodology%2E+Sage%2E)
26. Li, J. J., & Nenkova, A. (2016). The Instantiation discourse relation: A corpus analysis of its properties and improved detection. In Proceedings of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) (pp. 1181–1186). [find](https://scholar.google.com/scholar?q=Li%2C+J%2E+J%2E%2C+%26+Nenkova%2C+A%2E+%282016%29%2E+The+Instantiation+discourse+relation%3A+A+corpus+analysis+of+its+properties+and+improved+detection%2E+In+Proceedings+of+the+North+American+Chapter+of+the+Association+for+C)
27. Mann, W. C., & Thompson, S. A. (1988). Rhetorical Structure Theory: Toward a functional theory of text organization. Text-Interdisciplinary Journal for the Study of Discourse, 8, 243–281. [find](https://scholar.google.com/scholar?q=Mann%2C+W%2E+C%2E%2C+%26+Thompson%2C+S%2E+A%2E+%281988%29%2E+Rhetorical+Structure+Theory%3A+Toward+a+functional+theory+of+text+organization%2E+Text%2DInterdisciplinary+Journal+for+the+Study+of+Discourse%2C+8%2C+243%E2%80%93281%2E) [our card](https://md.research.elamur.ai/free/en/papers/enud-1988-mann-rhetorical-structure-theory-functional/)
28. Martins, D., Kigiel, D., & Jhean-Larose, S. (2006). Influence of expertise, coherence, and causal connectives on comprehension and recall of an expository text. Current Psychology Letters. Behaviour, Brain & Cognition, 3. [find](https://scholar.google.com/scholar?q=Martins%2C+D%2E%2C+Kigiel%2C+D%2E%2C+%26+Jhean%2DLarose%2C+S%2E+%282006%29%2E+Influence+of+expertise%2C+coherence%2C+and+causal+connectives+on+comprehension+and+recall+of+an+expository+text%2E+Current+Psychology+Letters%2E+Behaviour%2C+)
29. Maschler, Y., & Schiffrin, D. (2015). Discourse markers: Language, meaning, and context. In D. S. D. Tannen, H.E. Hamilton (Ed.), The Handbook of Discourse Analysis, Second edition, Chichester, UK: John Wiley & Sons (pp. 189–221). [find](https://scholar.google.com/scholar?q=Maschler%2C+Y%2E%2C+%26+Schiffrin%2C+D%2E+%282015%29%2E+Discourse+markers%3A+Language%2C+meaning%2C+and+context%2E+In+D%2E+S%2E+D%2E+Tannen%2C+H%2EE%2E+Hamilton+%28Ed%2E%29%2C+The+Handbook+of+Discourse+Analysis%2C+Second+edition%2C+Chichester%2C+UK%3A+Jo)
30. McNamara, D. S., Kintsch, E., Songer, N. B., & Kintsch, W. (1996). Are good texts always better? Interactions of text coherence, background knowledge, and levels of understanding in learning from text. Cognition and Instruction, 14, 1–43. [find](https://scholar.google.com/scholar?q=McNamara%2C+D%2E+S%2E%2C+Kintsch%2C+E%2E%2C+Songer%2C+N%2E+B%2E%2C+%26+Kintsch%2C+W%2E+%281996%29%2E+Are+good+texts+always+better%3F+Interactions+of+text+coherence%2C+background+knowledge%2C+and+levels+of+understanding+in+learning+from+text)
31. Moore, J. D., & Pollack, M. E. (1992). A problem for RST: The need for multi-level discourse analysis. Computational Linguistics, 18, 537–544. [find](https://scholar.google.com/scholar?q=Moore%2C+J%2E+D%2E%2C+%26+Pollack%2C+M%2E+E%2E+%281992%29%2E+A+problem+for+RST%3A+The+need+for+multi%2Dlevel+discourse+analysis%2E+Computational+Linguistics%2C+18%2C+537%E2%80%93544%2E)
32. Peldszus, A., & Stede, M. (2013). From argument diagrams to argumentation mining in texts: A survey. International Journal of Cognitive Informatics and Natural Intelligence (IJCINI), 7, 1–31. [find](https://scholar.google.com/scholar?q=Peldszus%2C+A%2E%2C+%26+Stede%2C+M%2E+%282013%29%2E+From+argument+diagrams+to+argumentation+mining+in+texts%3A+A+survey%2E+International+Journal+of+Cognitive+Informatics+and+Natural+Intelligence+%28IJCINI%29%2C+7%2C+1%E2%80%9331%2E)
33. Prasad, R., Dinesh, N., Lee, A., Miltsakaki, E., Robaldo, L., Joshi, A. K., & Webber, B. (2008). The Penn Discourse TreeBank 2.0. In Proceedings of the International Conference on Language Resources and Evaluation (LREC). Citeseer. [find](https://scholar.google.com/scholar?q=Prasad%2C+R%2E%2C+Dinesh%2C+N%2E%2C+Lee%2C+A%2E%2C+Miltsakaki%2C+E%2E%2C+Robaldo%2C+L%2E%2C+Joshi%2C+A%2E+K%2E%2C+%26+Webber%2C+B%2E+%282008%29%2E+The+Penn+Discourse+TreeBank+2%2E0%2E+In+Proceedings+of+the+International+Conference+on+Language+Resources+a) [our card](https://md.research.elamur.ai/free/en/papers/disc-2008-prasad-penn-discourse-treebank-2-0/)
34. Prasad, R., Miltsakaki, E., Dinesh, N., Lee, A., Joshi, A., Robaldo, L., & Webber, B. (2007). The Penn Discourse Treebank 2.0 annotation manual. [find](https://scholar.google.com/scholar?q=Prasad%2C+R%2E%2C+Miltsakaki%2C+E%2E%2C+Dinesh%2C+N%2E%2C+Lee%2C+A%2E%2C+Joshi%2C+A%2E%2C+Robaldo%2C+L%2E%2C+%26+Webber%2C+B%2E+%282007%29%2E+The+Penn+Discourse+Treebank+2%2E0+annotation+manual%2E)
35. Pusse, F., Sayeed, A., & Demberg, V. (2016). Lingoturk: Managing crowdsourced tasks for psycholinguistics. In Proceedings of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT). [find](https://scholar.google.com/scholar?q=Pusse%2C+F%2E%2C+Sayeed%2C+A%2E%2C+%26+Demberg%2C+V%2E+%282016%29%2E+Lingoturk%3A+Managing+crowdsourced+tasks+for+psycholinguistics%2E+In+Proceedings+of+the+North+American+Chapter+of+the+Association+for+Computational+Linguistics)
36. Redeker, G. (1990). Ideational and pragmatic markers of discourse structure. Journal of Pragmatics, 14, 367–381. [find](https://scholar.google.com/scholar?q=Redeker%2C+G%2E+%281990%29%2E+Ideational+and+pragmatic+markers+of+discourse+structure%2E+Journal+of+Pragmatics%2C+14%2C+367%E2%80%93381%2E)
37. Riezler, S. (2014). On the problem of theoretical terms in empirical computational linguistics. Computational Linguistics, 40, 235–245. [find](https://scholar.google.com/scholar?q=Riezler%2C+S%2E+%282014%29%2E+On+the+problem+of+theoretical+terms+in+empirical+computational+linguistics%2E+Computational+Linguistics%2C+40%2C+235%E2%80%93245%2E)
38. Robaldo, L., & Miltsakaki, E. (2014). Corpus-driven semantics of concession: Where do expectations come from? Dialogue & Discourse, 5, 1–36. [find](https://scholar.google.com/scholar?q=Robaldo%2C+L%2E%2C+%26+Miltsakaki%2C+E%2E+%282014%29%2E+Corpus%2Ddriven+semantics+of+concession%3A+Where+do+expectations+come+from%3F+Dialogue+%26+Discourse%2C+5%2C+1%E2%80%9336%2E)
39. Rohde, H., Dickinson, A., Clark, C. N., Louis, A., & Webber, B. (2015). Recovering discourse relations: Varying influence of discourse adverbials. In Proceedings of the Workshop on Linking Models of Lexical, Sentential and Discourse-level Semantics (LSDSem) (pp. 1–22). [find](https://scholar.google.com/scholar?q=Rohde%2C+H%2E%2C+Dickinson%2C+A%2E%2C+Clark%2C+C%2E+N%2E%2C+Louis%2C+A%2E%2C+%26+Webber%2C+B%2E+%282015%29%2E+Recovering+discourse+relations%3A+Varying+influence+of+discourse+adverbials%2E+In+Proceedings+of+the+Workshop+on+Linking+Models+of+L) [our card](https://md.research.elamur.ai/free/en/papers/rohde-2015-recovering-discourse-relations-varying-influence/)
40. Rohde, H., Dickinson, A., Schneider, N., Clark, C. N., Louis, A., & Webber, B. (2016). Filling in the blanks in understanding discourse adverbials: Consistency, conflict, and context-dependence in a crowdsourced elicitation task. In Proceedings of the 10th Linguistic Annotation Workshop (LAW X) (pp.… [find](https://scholar.google.com/scholar?q=Rohde%2C+H%2E%2C+Dickinson%2C+A%2E%2C+Schneider%2C+N%2E%2C+Clark%2C+C%2E+N%2E%2C+Louis%2C+A%2E%2C+%26+Webber%2C+B%2E+%282016%29%2E+Filling+in+the+blanks+in+understanding+discourse+adverbials%3A+Consistency%2C+conflict%2C+and+context%2Ddependence+in+a+c)
41. Sanders, T. J., Demberg, V., Hoek, J., Scholman, M. C., Torabi Asr, F., Zufferey, S., & Evers-Vermuel, J. (Submitted). Unifying dimensions in discourse relations: How various annotation frameworks are related. Corpus Linguistics and Linguistic Theory, . [find](https://scholar.google.com/scholar?q=Sanders%2C+T%2E+J%2E%2C+Demberg%2C+V%2E%2C+Hoek%2C+J%2E%2C+Scholman%2C+M%2E+C%2E%2C+Torabi+Asr%2C+F%2E%2C+Zufferey%2C+S%2E%2C+%26+Evers%2DVermuel%2C+J%2E+%28Submitted%29%2E+Unifying+dimensions+in+discourse+relations%3A+How+various+annotation+frameworks+are)
42. Sanders, T. J., Demberg, V., Hoek, J., Scholman, M. C., Zufferey, S., & Evers-Vermuel, J. (2016). How can we relate various annotation schemes? unifying dimensions in discourse relations. In TextLink Second Action Conference (pp. 110–112). [find](https://scholar.google.com/scholar?q=Sanders%2C+T%2E+J%2E%2C+Demberg%2C+V%2E%2C+Hoek%2C+J%2E%2C+Scholman%2C+M%2E+C%2E%2C+Zufferey%2C+S%2E%2C+%26+Evers%2DVermuel%2C+J%2E+%282016%29%2E+How+can+we+relate+various+annotation+schemes%3F+unifying+dimensions+in+discourse+relations%2E+In+TextLink+)
43. Sanders, T. J., Spooren, W. P., & Noordman, L. G. (1992). Toward a taxonomy of coherence relations. Discourse Processes, 15, 1–35. [find](https://scholar.google.com/scholar?q=Sanders%2C+T%2E+J%2E%2C+Spooren%2C+W%2E+P%2E%2C+%26+Noordman%2C+L%2E+G%2E+%281992%29%2E+Toward+a+taxonomy+of+coherence+relations%2E+Discourse+Processes%2C+15%2C+1%E2%80%9335%2E)
44. Scholman, M. C., & Demberg, V. (2017). Crowdsourcing discourse interpretations: On the influence of context and the reliability of a connective insertion task. In Proceedings of the 11th Linguistic Annotation Workshop (LAW) (pp. 24–33). [find](https://scholar.google.com/scholar?q=Scholman%2C+M%2E+C%2E%2C+%26+Demberg%2C+V%2E+%282017%29%2E+Crowdsourcing+discourse+interpretations%3A+On+the+influence+of+context+and+the+reliability+of+a+connective+insertion+task%2E+In+Proceedings+of+the+11th+Linguistic+An)
45. Scholman, M. C., Evers-Vermeul, J., & Sanders, T. J. (2016). Categories of coherence relations in discourse annotation: Towards a reliable categorization of coherence relations. Dialogue & Discourse, 7, 1–28. [find](https://scholar.google.com/scholar?q=Scholman%2C+M%2E+C%2E%2C+Evers%2DVermeul%2C+J%2E%2C+%26+Sanders%2C+T%2E+J%2E+%282016%29%2E+Categories+of+coherence+relations+in+discourse+annotation%3A+Towards+a+reliable+categorization+of+coherence+relations%2E+Dialogue+%26+Discourse%2C+) [our card](https://md.research.elamur.ai/free/en/papers/scholman-2016-categories-coherence-relations-discourse-annotation/)
46. Snow, R., O’Connor, B., Jurafsky, D., & Ng, A. Y. (2008). Cheap and fast—but is it good?: Evaluating non-expert annotations for natural language tasks. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP) (pp. 254–263). Association for Computational Linguistic… [find](https://scholar.google.com/scholar?q=Snow%2C+R%2E%2C+O%E2%80%99Connor%2C+B%2E%2C+Jurafsky%2C+D%2E%2C+%26+Ng%2C+A%2E+Y%2E+%282008%29%2E+Cheap+and+fast%E2%80%94but+is+it+good%3F%3A+Evaluating+non%2Dexpert+annotations+for+natural+language+tasks%2E+In+Proceedings+of+the+Conference+on+Empirical+Me) [our card](https://md.research.elamur.ai/free/en/papers/snow-2008-cheap-fast-but-is-it/)
47. Stab, C., & Gurevych, I. (2014). Identifying argumentative discourse structures in persuasive essays. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP) (pp. 46–56). [find](https://scholar.google.com/scholar?q=Stab%2C+C%2E%2C+%26+Gurevych%2C+I%2E+%282014%29%2E+Identifying+argumentative+discourse+structures+in+persuasive+essays%2E+In+Proceedings+of+the+Conference+on+Empirical+Methods+in+Natural+Language+Processing+%28EMNLP%29+%28pp%2E+) [our card](https://md.research.elamur.ai/free/en/papers/stab-2014-identifying-argumentative-discourse-structures-persuasive/)
48. Taboada, M., & Das, D. (2013). Annotation upon annotation: Adding signalling information to a corpus of discourse relations. Dialogue & Discourse, 4, 249–281. [find](https://scholar.google.com/scholar?q=Taboada%2C+M%2E%2C+%26+Das%2C+D%2E+%282013%29%2E+Annotation+upon+annotation%3A+Adding+signalling+information+to+a+corpus+of+discourse+relations%2E+Dialogue+%26+Discourse%2C+4%2C+249%E2%80%93281%2E) [our card](https://md.research.elamur.ai/free/en/papers/taboada-2013-annotation-upon-annotation-adding-signalling/)
49. Vergez-Couret, M., & Adam, C. (2012). Signaling Elaboration: Combining french gerund clauses with lexical cohesion cues. Discours. Revue de linguistique, psycholinguistique et informatique. A journal of linguistics, psycholinguistics and computational linguistics, . [find](https://scholar.google.com/scholar?q=Vergez%2DCouret%2C+M%2E%2C+%26+Adam%2C+C%2E+%282012%29%2E+Signaling+Elaboration%3A+Combining+french+gerund+clauses+with+lexical+cohesion+cues%2E+Discours%2E+Revue+de+linguistique%2C+psycholinguistique+et+informatique%2E+A+journal+)
50. Versley, Y. (2011). Multilabel tagging of discourse relations in ambiguous temporal connectives. In International Conference on Recent Advances in Natural Language Processing (RANLP) (pp. 154–161). [find](https://scholar.google.com/scholar?q=Versley%2C+Y%2E+%282011%29%2E+Multilabel+tagging+of+discourse+relations+in+ambiguous+temporal+connectives%2E+In+International+Conference+on+Recent+Advances+in+Natural+Language+Processing+%28RANLP%29+%28pp%2E+154%E2%80%93161%29%2E)
51. Webber, B. (2013). What excludes an alternative in coherence relations. In Proceedings of the International Conference on Computational Semantics (IWCS). [find](https://scholar.google.com/scholar?q=Webber%2C+B%2E+%282013%29%2E+What+excludes+an+alternative+in+coherence+relations%2E+In+Proceedings+of+the+International+Conference+on+Computational+Semantics+%28IWCS%29%2E) [our card](https://md.research.elamur.ai/free/en/papers/webber-2013-what-excludes-alternative-coherence-relations/)
52. Webber, B., Knott, A., & Joshi, A. (2001). Multiple discourse connectives in a lexicalized grammar for discourse. In Computing Meaning (pp. 229–245). Springer. [find](https://scholar.google.com/scholar?q=Webber%2C+B%2E%2C+Knott%2C+A%2E%2C+%26+Joshi%2C+A%2E+%282001%29%2E+Multiple+discourse+connectives+in+a+lexicalized+grammar+for+discourse%2E+In+Computing+Meaning+%28pp%2E+229%E2%80%93245%29%2E+Springer%2E)
53. Webber, B., Prasad, R., Lee, A., & Joshi, A. (2016). A discourse-annotated corpus of conjoined VPs. In Proceedings of the 10th Linguistic Annotation Workshop (LAW) (pp. 22–31). [find](https://scholar.google.com/scholar?q=Webber%2C+B%2E%2C+Prasad%2C+R%2E%2C+Lee%2C+A%2E%2C+%26+Joshi%2C+A%2E+%282016%29%2E+A+discourse%2Dannotated+corpus+of+conjoined+VPs%2E+In+Proceedings+of+the+10th+Linguistic+Annotation+Workshop+%28LAW%29+%28pp%2E+22%E2%80%9331%29%2E)
54. Zufferey, S., & Degand, L. (2013). Annotating the meaning of discourse connectives in multilingual corpora. Corpus Linguistics and Linguistic Theory, 1, 1–24. [find](https://scholar.google.com/scholar?q=Zufferey%2C+S%2E%2C+%26+Degand%2C+L%2E+%282013%29%2E+Annotating+the+meaning+of+discourse+connectives+in+multilingual+corpora%2E+Corpus+Linguistics+and+Linguistic+Theory%2C+1%2C+1%E2%80%9324%2E)