---
title: "Combining Natural and Artificial Examples to — Braud 2014"
description: "This COLING 2014 paper presents the first experiments on recognizing implicit discourse relations in French, meaning relations not signalled by an…"
authors: ["Chloé Braud", "Pascal Denis"]
year: 2014
venue: "Proceedings of COLING 2014, the 25th International Conference on Computational Linguistics: Technical Papers"
source: "https://aclanthology.org/C14-1160/"
licence: "CC BY-NC-SA 3.0 (ACL Anthology, before 2016)"
licence_url: "https://creativecommons.org/licenses/by-nc-sa/3.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/papers/braud-2014-combining-natural-artificial-examples-improve/"
---
[Elamur Research](https://md.research.elamur.ai/) · [this page in HTML](https://research.elamur.ai/free/papers/braud-2014-combining-natural-artificial-examples-improve/)

[home](https://md.research.elamur.ai/) / [papers](https://md.research.elamur.ai/free/papers/) / [2014](https://md.research.elamur.ai/free/papers/2014/) / Combining Natural and Artificial Examples to Improve Implicit Discourse Relation Identification

# Combining Natural and Artificial Examples to Improve Implicit Discourse Relation Identification

**Authors:** [Chloé Braud](https://md.research.elamur.ai/free/authors/chloe-braud/), [Pascal Denis](https://md.research.elamur.ai/free/authors/pascal-denis/) · **Year:** 2014 · **Venue:** Proceedings of COLING 2014, the 25th International Conference on Computational Linguistics: Technical Papers · **Licence:** [CC BY-NC-SA 3.0 (ACL Anthology, before 2016)](https://creativecommons.org/licenses/by-nc-sa/3.0/)

**Links:** [Publisher's page](https://aclanthology.org/C14-1160/) · [PDF](https://aclanthology.org/C14-1160.pdf) · [OpenAlex](https://openalex.org/W4302321467)

**Citations:** 1 citations (openalex-snapshot · 2026-10-14) · [14](#cited-by) in our library, of the 140673 papers whose bibliographies we read

## Summary
This COLING 2014 paper presents the first experiments on recognizing implicit discourse relations in French, meaning relations not signalled by an explicit connective. Manually annotated data is scarce: the ANNODIS corpus has roughly 3,000 examples, compared with about 40,000 in the PDTB. French also lacks lexical resources such as polarity databases. To compensate, the authors follow Marcu and Echihabi (2002). They take explicit examples marked by unambiguous connectives, delete the connective, and use the result as artificially labelled implicit data. A classifier trained only on these artificial examples performs poorly on natural implicit relations, which confirms the earlier English finding of Sporleder and Lascarides (2008). The authors trace this to distribution mismatches between the two kinds of data, including prior probability shift and covariate shift. They then test simple combination schemes borrowed from domain adaptation, plus automatic filtering of noisy artificial examples. The best setup feeds the artificial model's predicted relation into the natural-data classifier as an extra feature. It reaches 41.7% accuracy, 4.4 points above a model trained only on manual data. The work shows that cheap artificial data helps only when integrated carefully.

## Key points
- Data: the French ANNODIS corpus, with four relations evaluated (contrast, result, continuation, explanation), plus artificial examples extracted from raw text using unambiguous connectives.
- Combination methods were inspired by domain adaptation (Daumé III and Marcu, 2006). They include training on the union of both datasets, weighting natural examples (NATW), subsampling artificial data (ARTSUB), linear interpolation of the two models (LININT), and adding the artificial model's prediction as a feature (ADDPRED).
- Automatic selection of artificial examples raised ARTONLY accuracy from 23.0 to 25.0 and UNION from 22.6 to 40.1.
- The best system, ADDPRED+SELEC, reached 41.7% accuracy. By class, it obtained an F1 of 20.8 on contrast, with result at 51.0, continuation at 31.2 and explanation at 53.9.
- Contrast and continuation gained most from artificial data; contrast's best F1 was 28.8, obtained with ARTONLY+SELEC. Heavy reliance on artificial data hurt result and explanation.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

Fields: [Linguistics & languages](https://md.research.elamur.ai/free/shelves/linguistics/)

Topics: [Discourse analysis](https://md.research.elamur.ai/free/topics/discourse-analysis/)

## Cited by

Cited by

14 papers of our library cite it (of the 140673 whose bibliographies we read); 6 of them have a page here, 8 not yet.

- [Comparing Word Representations for Implicit Discourse Relation Classification](https://md.research.elamur.ai/free/papers/braud-2015-comparing-word-representations-implicit-discourse/)
- [Let's be explicit about that: Distant supervision for implicit discourse relation classification via connective prediction](https://md.research.elamur.ai/free/papers/kurfal-2021-let-s-be-explicit-about/)
- [Implicit Discourse Relation Identification for Open-domain Dialogues](https://md.research.elamur.ai/free/papers/ma-2019-implicit-discourse-relation-identification-open/)
- [Parsing Argumentation Structures in Persuasive Essays](https://md.research.elamur.ai/free/papers/stab-2017-parsing-argumentation-structures-persuasive-essays/)
- [Shallow Convolutional Neural Network for Implicit Discourse Relation Recognition](https://md.research.elamur.ai/free/papers/zhang-2015-shallow-convolutional-neural-network-implicit/)
- [Variational Neural Discourse Relation Recognizer](https://md.research.elamur.ai/free/papers/zhang-2016-variational-neural-discourse-relation-recognizer/)

## Cite

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{braud2014combining,
  title = {{Combining Natural and Artificial Examples to Improve Implicit Discourse Relation Identification}},
  author = {Braud, Chloé and Denis, Pascal},
  booktitle = {Proceedings of COLING 2014, the 25th International Conference on Computational Linguistics: Technical Papers},
  year = {2014},
  url = {https://aclanthology.org/C14-1160/}
}
```

- [← previous CIDEr: Consensus-based Image Description Evaluation](https://md.research.elamur.ai/free/papers/vedantam-2014-cider-consensus-based-image-description/)
- [next → Comprehensive Annotation of Multiword Expressions in a Social Web Corpus](https://md.research.elamur.ai/free/papers/lingbulk-2014-schneider-comprehensive-annotation-multiword-expressions-social/)
- [↑ back to papers of 2014](https://md.research.elamur.ai/free/papers/2014/)

## References

35 references, 13 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Stergos Afantenos, Nicholas Asher, Farah Benamara, Myriam Bras, Cécile Fabre, Lydia-Mai Ho-Dac, Anne Le Draoulec, Philippe Muller, Marie-Paule Péry-Woodley, Laurent Prévot, Josette Rebeyrolles, Ludovic Tanguy, Marianne Vergez-Couret, and Laure Vieu. 2012. An empirical resource for discovering cognit… [find](https://scholar.google.com/scholar?q=Stergos+Afantenos%2C+Nicholas+Asher%2C+Farah+Benamara%2C+Myriam+Bras%2C+C%C3%A9cile+Fabre%2C+Lydia%2DMai+Ho%2DDac%2C+Anne+Le+Draoulec%2C+Philippe+Muller%2C+Marie%2DPaule+P%C3%A9ry%2DWoodley%2C+Laurent+Pr%C3%A9vot%2C+Josette+Rebeyrolles%2C+Ludovi)
2. Nicholas Asher and Alex Lascarides. 2003. Logics of Conversation. Cambridge University Press. [find](https://scholar.google.com/scholar?q=Nicholas+Asher+and+Alex+Lascarides%2E+2003%2E+Logics+of+Conversation%2E+Cambridge+University+Press%2E)
3. Sasha Blair-Goldensohn, Kathleen R. McKeown, and Owen C. Rambow. 2007. Building and refining rhetorical-semantic relation models. In Proceedings of NAACL HLT. [find](https://scholar.google.com/scholar?q=Sasha+Blair%2DGoldensohn%2C+Kathleen+R%2E+McKeown%2C+and+Owen+C%2E+Rambow%2E+2007%2E+Building+and+refining+rhetorical%2Dsemantic+relation+models%2E+In+Proceedings+of+NAACL+HLT%2E)
4. Marie Candito, Joakim Nivre, Pascal Denis, and Enrique Henestroza Anguiano. 2010. Benchmarking of statistical dependency parsers for french. In Proceedings of ICCL (posters). [find](https://scholar.google.com/scholar?q=Marie+Candito%2C+Joakim+Nivre%2C+Pascal+Denis%2C+and+Enrique+Henestroza+Anguiano%2E+2010%2E+Benchmarking+of+statistical+dependency+parsers+for+french%2E+In+Proceedings+of+ICCL+%28posters%29%2E)
5. Lynn Carlson, Daniel Marcu, and Mary Ellen Okurowski. 2001. Building a discourse-tagged corpus in the framework of rhetorical structure theory. In Proceedings of the Second SIGdial Workshop on Discourse and Dialogue. [find](https://scholar.google.com/scholar?q=Lynn+Carlson%2C+Daniel+Marcu%2C+and+Mary+Ellen+Okurowski%2E+2001%2E+Building+a+discourse%2Dtagged+corpus+in+the+framework+of+rhetorical+structure+theory%2E+In+Proceedings+of+the+Second+SIGdial+Workshop+on+Discour) [our card](https://md.research.elamur.ai/free/papers/carlson-2001-building-discourse-tagged-corpus-framework/)
6. Hal Daumé III and Daniel Marcu. 2006. Domain adaptation for statistical classifiers. Journal of Artificial Intelligence Research. [find](https://scholar.google.com/scholar?q=Hal+Daum%C3%A9+III+and+Daniel+Marcu%2E+2006%2E+Domain+adaptation+for+statistical+classifiers%2E+Journal+of+Artificial+Intelligence+Research%2E)
7. Hal Daumé III. 2007. Frustratingly easy domain adaptation. In Proceedings of ACL. [find](https://scholar.google.com/scholar?q=Hal+Daum%C3%A9+III%2E+2007%2E+Frustratingly+easy+domain+adaptation%2E+In+Proceedings+of+ACL%2E)
8. Joost C.F. de Winter. 2013. Using the student’s t-test with extremely small sample sizes. Practical Assessment, Research & Evaluation. [find](https://scholar.google.com/scholar?q=Joost+C%2EF%2E+de+Winter%2E+2013%2E+Using+the+student%E2%80%99s+t%2Dtest+with+extremely+small+sample+sizes%2E+Practical+Assessment%2C+Research+%26+Evaluation%2E)
9. Pascal Denis and Benoît Sagot. 2009. Coupling an annotated corpus and a morphosyntactic lexicon for state-ofthe-art POS tagging with less human effort. In Proceedings of PACLIC. [find](https://scholar.google.com/scholar?q=Pascal+Denis+and+Beno%C3%AEt+Sagot%2E+2009%2E+Coupling+an+annotated+corpus+and+a+morphosyntactic+lexicon+for+state%2Dofthe%2Dart+POS+tagging+with+less+human+effort%2E+In+Proceedings+of+PACLIC%2E)
10. Anna Gastel, Sabrina Schulze, Yannick Versley, and Erhard Hinrichs. 2011. Annotation of implicit discourse relations in the tüba-d/z treebank. GSCL. [find](https://scholar.google.com/scholar?q=Anna+Gastel%2C+Sabrina+Schulze%2C+Yannick+Versley%2C+and+Erhard+Hinrichs%2E+2011%2E+Annotation+of+implicit+discourse+relations+in+the+t%C3%BCba%2Dd%2Fz+treebank%2E+GSCL%2E)
11. David J. Hand. 2006. Classifier technology and the illusion of progress. Statistical Science. [find](https://scholar.google.com/scholar?q=David+J%2E+Hand%2E+2006%2E+Classifier+technology+and+the+illusion+of+progress%2E+Statistical+Science%2E)
12. Hugo Hernault, Helmut Prendinger, David A. duVerle, and Mitsuru Ishizuka. 2010. Hilda: A discourse parser using support vector machine classification. Dialogue and Discourse. [find](https://scholar.google.com/scholar?q=Hugo+Hernault%2C+Helmut+Prendinger%2C+David+A%2E+duVerle%2C+and+Mitsuru+Ishizuka%2E+2010%2E+Hilda%3A+A+discourse+parser+using+support+vector+machine+classification%2E+Dialogue+and+Discourse%2E) [our card](https://md.research.elamur.ai/free/papers/hernault-2010-hilda-discourse-parser-support-vector/)
13. Jing Jiang. 2008. A literature survey on domain adaptation of statistical classifiers. [http://sifaka.cs.uiuc.edu/jiang4/domain_adaptation/survey/da_survey.html](http://sifaka.cs.uiuc.edu/jiang4/domain_adaptation/survey/da_survey.html). [↗](http://sifaka.cs.uiuc.edu/jiang4/domain_adaptation/survey/da_survey.html)
14. Shafiq R. Joty, Giuseppe Carenini, Raymond T. Ng, and Yashar Mehdad. 2013. Combining intra- and multi-sentential rhetorical parsing for document-level discourse analysis. In Proceedings of ACL. [find](https://scholar.google.com/scholar?q=Shafiq+R%2E+Joty%2C+Giuseppe+Carenini%2C+Raymond+T%2E+Ng%2C+and+Yashar+Mehdad%2E+2013%2E+Combining+intra%2D+and+multi%2Dsentential+rhetorical+parsing+for+document%2Dlevel+discourse+analysis%2E+In+Proceedings+of+ACL%2E)
15. Ziheng Lin, Min-Yen Kan, and Hwee Tou Ng. 2009. Recognizing implicit discourse relations in the penn discourse treebank. In Proceedings of EMNLP. [find](https://scholar.google.com/scholar?q=Ziheng+Lin%2C+Min%2DYen+Kan%2C+and+Hwee+Tou+Ng%2E+2009%2E+Recognizing+implicit+discourse+relations+in+the+penn+discourse+treebank%2E+In+Proceedings+of+EMNLP%2E) [our card](https://md.research.elamur.ai/free/papers/lin-2009-recognizing-implicit-discourse-relations-penn/)
16. Ziheng Lin, Hwee Tou Ng, and Min-Yen Kan. 2010. A PDTB-styled end-to-end discourse parser. Technical report, National University of Singapore. [find](https://scholar.google.com/scholar?q=Ziheng+Lin%2C+Hwee+Tou+Ng%2C+and+Min%2DYen+Kan%2E+2010%2E+A+PDTB%2Dstyled+end%2Dto%2Dend+discourse+parser%2E+Technical+report%2C+National+University+of+Singapore%2E)
17. Yishay Mansour, Mehryar Mohri, and Afshin Rostamizadeh. 2009. Domain adaptation: Learning bounds and algorithms. In Proceedings of COLT. [find](https://scholar.google.com/scholar?q=Yishay+Mansour%2C+Mehryar+Mohri%2C+and+Afshin+Rostamizadeh%2E+2009%2E+Domain+adaptation%3A+Learning+bounds+and+algorithms%2E+In+Proceedings+of+COLT%2E)
18. Daniel Marcu and Abdessamad Echihabi. 2002. An unsupervised approach to recognizing discourse relations. In Proceedings of ACL. [find](https://scholar.google.com/scholar?q=Daniel+Marcu+and+Abdessamad+Echihabi%2E+2002%2E+An+unsupervised+approach+to+recognizing+discourse+relations%2E+In+Proceedings+of+ACL%2E) [our card](https://md.research.elamur.ai/free/papers/marcu-2002-unsupervised-approach-recognizing-discourse-relations/)
19. Ryan McDonald and Fernando Pereira. 2006. Online learning of approximate dependency parsing algorithms. In Proceedings of EACL. [find](https://scholar.google.com/scholar?q=Ryan+McDonald+and+Fernando+Pereira%2E+2006%2E+Online+learning+of+approximate+dependency+parsing+algorithms%2E+In+Proceedings+of+EACL%2E) [our card](https://md.research.elamur.ai/free/papers/continual-2006-mcdonald-online-learning-approximate-dependency-parsing/)
20. Jose G. Moreno-Torres, Troy Raeder, Rocío Alaiz-Rodríguez, Nitesh V. Chawla, and Francisco Herrera. 2012. A unifying view on dataset shift in classification. Pattern Recognition. [find](https://scholar.google.com/scholar?q=Jose+G%2E+Moreno%2DTorres%2C+Troy+Raeder%2C+Roc%C3%ADo+Alaiz%2DRodr%C3%ADguez%2C+Nitesh+V%2E+Chawla%2C+and+Francisco+Herrera%2E+2012%2E+A+unifying+view+on+dataset+shift+in+classification%2E+Pattern+Recognition%2E)
21. Joonsuk Park and Claire Cardie. 2012. Improving implicit discourse relation recognition through feature set optimization. In Proceedings of SIGDIAL. [find](https://scholar.google.com/scholar?q=Joonsuk+Park+and+Claire+Cardie%2E+2012%2E+Improving+implicit+discourse+relation+recognition+through+feature+set+optimization%2E+In+Proceedings+of+SIGDIAL%2E) [our card](https://md.research.elamur.ai/free/papers/park-2012-improving-implicit-discourse-relation-recognition/)
22. Emily Pitler and Ani Nenkova. 2009. Using syntax to disambiguate explicit discourse connectives in text. In Proceedings of the ACL-IJCNLP (Short Papers). [find](https://scholar.google.com/scholar?q=Emily+Pitler+and+Ani+Nenkova%2E+2009%2E+Using+syntax+to+disambiguate+explicit+discourse+connectives+in+text%2E+In+Proceedings+of+the+ACL%2DIJCNLP+%28Short+Papers%29%2E) [our card](https://md.research.elamur.ai/free/papers/pitler-2009-syntax-disambiguate-explicit-discourse-connectives/)
23. Emily Pitler, Annie Louis, and Ani Nenkova. 2009. Automatic sense prediction for implicit discourse relations in text. In Proceedings of ACL-IJCNLP. [find](https://scholar.google.com/scholar?q=Emily+Pitler%2C+Annie+Louis%2C+and+Ani+Nenkova%2E+2009%2E+Automatic+sense+prediction+for+implicit+discourse+relations+in+text%2E+In+Proceedings+of+ACL%2DIJCNLP%2E) [our card](https://md.research.elamur.ai/free/papers/disc-2009-pitler-automatic-sense-prediction-implicit-discourse/)
24. Rashmi Prasad, Nikhil Dinesh, Alan Lee, Eleni Miltsakaki, Livio Robaldo, Aravind Joshi, and Bonnie Webber. 2008. The penn discourse treebank 2.0. In Proceedings of LREC. [find](https://scholar.google.com/scholar?q=Rashmi+Prasad%2C+Nikhil+Dinesh%2C+Alan+Lee%2C+Eleni+Miltsakaki%2C+Livio+Robaldo%2C+Aravind+Joshi%2C+and+Bonnie+Webber%2E+2008%2E+The+penn+discourse+treebank+2%2E0%2E+In+Proceedings+of+LREC%2E) [our card](https://md.research.elamur.ai/free/papers/disc-2008-prasad-penn-discourse-treebank-2-0/)
25. Charlotte Roze, Laurence Danlos, and Philippe Muller. 2012. Lexconn: A french lexicon of discourse connectives. Discours. [find](https://scholar.google.com/scholar?q=Charlotte+Roze%2C+Laurence+Danlos%2C+and+Philippe+Muller%2E+2012%2E+Lexconn%3A+A+french+lexicon+of+discourse+connectives%2E+Discours%2E) [our card](https://md.research.elamur.ai/free/papers/roze-2012-lexconn-french-lexicon-discourse-connectives/)
26. Kenji Sagae. 2009. Analysis of discourse structure with syntactic dependencies and data-driven shift-reduce parsing. Proceedings of IWPT. [find](https://scholar.google.com/scholar?q=Kenji+Sagae%2E+2009%2E+Analysis+of+discourse+structure+with+syntactic+dependencies+and+data%2Ddriven+shift%2Dreduce+parsing%2E+Proceedings+of+IWPT%2E)
27. Available from: Tobias Scheffer. 1999. Error Estimation and Model Selection. Ph.D. thesis, Technischen Universitet Berlin, School of Computer Science. [find](https://scholar.google.com/scholar?q=Available+from%3A+Tobias+Scheffer%2E+1999%2E+Error+Estimation+and+Model+Selection%2E+Ph%2ED%2E+thesis%2C+Technischen+Universitet+Berlin%2C+School+of+Computer+Science%2E)
28. Anders Sogaard. 2013. Semi-supervised learning and domain adaptation in natural language processing. Morgan & Claypool. [find](https://scholar.google.com/scholar?q=Anders+Sogaard%2E+2013%2E+Semi%2Dsupervised+learning+and+domain+adaptation+in+natural+language+processing%2E+Morgan+%26+Claypool%2E)
29. Caroline Sporleder and Alex Lascarides. 2008. Using automatically labelled examples to classify rhetorical relations, an assessment. Natural Language Engineering. [find](https://scholar.google.com/scholar?q=Caroline+Sporleder+and+Alex+Lascarides%2E+2008%2E+Using+automatically+labelled+examples+to+classify+rhetorical+relations%2C+an+assessment%2E+Natural+Language+Engineering%2E)
30. Caroline Sporleder. 2008. Lexical models to identify unmarked discourse relations: Does Wordnet help? Lexical-Semantic Resources in Automated Discourse Analysis. [find](https://scholar.google.com/scholar?q=Caroline+Sporleder%2E+2008%2E+Lexical+models+to+identify+unmarked+discourse+relations%3A+Does+Wordnet+help%3F+Lexical%2DSemantic+Resources+in+Automated+Discourse+Analysis%2E)
31. Sudhir Varma and Richard Simon. 2006. Bias in error estimation when using cross-validation for model selection. BMC bioinformatics. [find](https://scholar.google.com/scholar?q=Sudhir+Varma+and+Richard+Simon%2E+2006%2E+Bias+in+error+estimation+when+using+cross%2Dvalidation+for+model+selection%2E+BMC+bioinformatics%2E)
32. Yannick Versley. 2013. Subgraph-based classification of explicit and implicit discourse relations. In Proceedings of IWCS. [find](https://scholar.google.com/scholar?q=Yannick+Versley%2E+2013%2E+Subgraph%2Dbased+classification+of+explicit+and+implicit+discourse+relations%2E+In+Proceedings+of+IWCS%2E)
33. WenTing Wang, Jian Su, and Chew Lim Tan. 2010. Kernel based discourse relation recognition with temporal ordering information. In Proceedings of ACL. [find](https://scholar.google.com/scholar?q=WenTing+Wang%2C+Jian+Su%2C+and+Chew+Lim+Tan%2E+2010%2E+Kernel+based+discourse+relation+recognition+with+temporal+ordering+information%2E+In+Proceedings+of+ACL%2E) [our card](https://md.research.elamur.ai/free/papers/wang-2010-kernel-based-discourse-relation-recognition/)
34. Xun Wang, Sujian Li, Jiwei Li, and Wenjie Li. 2012. Implicit discourse relation recognition by selecting typical training examples. In Proceedings of COLING (Technical Papers). [find](https://scholar.google.com/scholar?q=Xun+Wang%2C+Sujian+Li%2C+Jiwei+Li%2C+and+Wenjie+Li%2E+2012%2E+Implicit+discourse+relation+recognition+by+selecting+typical+training+examples%2E+In+Proceedings+of+COLING+%28Technical+Papers%29%2E) [our card](https://md.research.elamur.ai/free/papers/wang-2012-implicit-discourse-relation-recognition-selecting/)
35. Zhi-Min Zhou, Yu Xu, Zheng-Yu Niu, Man Lan, Jian Su, and Chew Lim Tan. 2010. Predicting discourse connectives for implicit discourse relation recognition. In Proceedings of COLING (Posters). [find](https://scholar.google.com/scholar?q=Zhi%2DMin+Zhou%2C+Yu+Xu%2C+Zheng%2DYu+Niu%2C+Man+Lan%2C+Jian+Su%2C+and+Chew+Lim+Tan%2E+2010%2E+Predicting+discourse+connectives+for+implicit+discourse+relation+recognition%2E+In+Proceedings+of+COLING+%28Posters%29%2E) [our card](https://md.research.elamur.ai/free/papers/zhou-2010-predicting-discourse-connectives-implicit-discourse/)