---
title: "Reducing Sparsity Improves the Recognition of — Li 2014"
description: "Li and Nenkova investigate the effect of feature sparsity on recognizing implicit relations between sentences, using the Penn Discourse Treebank (PDTB)…"
authors: ["Junyi Jessy Li", "Ani Nenkova"]
year: 2014
venue: "Proceedings of the 15th Annual Meeting of the Special Interest Group on Discourse and Dialogue (SIGDIAL)"
source: "https://aclanthology.org/W14-4327/"
doi: "10.3115/v1/w14-4327"
licence: "CC BY-NC-SA 3.0 (ACL Anthology, before 2016)"
licence_url: "https://creativecommons.org/licenses/by-nc-sa/3.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/li-2014-reducing-sparsity-improves-recognition-implicit/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/li-2014-reducing-sparsity-improves-recognition-implicit/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2014](https://md.research.elamur.ai/free/en/papers/2014/page/2/) / Reducing Sparsity Improves the Recognition of Implicit Discourse Relations

# Reducing Sparsity Improves the Recognition of Implicit Discourse Relations

**Authors:** [Junyi Jessy Li](https://md.research.elamur.ai/free/en/authors/junyi-jessy-li/), [Ani Nenkova](https://md.research.elamur.ai/free/en/authors/ani-nenkova/) · **Year:** 2014 · **Venue:** Proceedings of the 15th Annual Meeting of the Special Interest Group on Discourse and Dialogue (SIGDIAL) · **Licence:** [CC BY-NC-SA 3.0 (ACL Anthology, before 2016)](https://creativecommons.org/licenses/by-nc-sa/3.0/)

**Links:** [Publisher's page](https://aclanthology.org/W14-4327/) · [PDF](https://aclanthology.org/W14-4327.pdf) · [DOI 10.3115/v1/w14-4327](https://doi.org/10.3115/v1/w14-4327) · [OpenAlex](https://openalex.org/W2250965765)

**Citations:** 19 citations (openalex-snapshot · 2026-10-14) · [13](#cited-by) in our library, of the 140673 papers whose bibliographies we read

## Summary
Li and Nenkova investigate the effect of feature sparsity on recognizing implicit relations between sentences, using the Penn Discourse Treebank (PDTB) as their data. Implicit relations link adjacent sentences without an explicit connective. They focus on the two strongest feature families from earlier work: lexical word pairs and syntactic production rules. Using only lexical and syntactic information from gold Penn Treebank parses, they train binary SVM classifiers for seven relation classes.

They show that how features are encoded matters. Recording which words occur in each argument, instead of taking the cross-product of words across arguments, yields far fewer and denser features and better results. This explains part of the advantage earlier credited to syntax alone. They also propose a new, denser syntactic representation called production sticks, which significantly improves recognition. Removing lexical content from syntactic features lowers performance for most relations, so word-level information remains essential. Agreement analysis shows that different lexical encodings make quite different errors. This suggests that combining them in future discourse parsers could help.

## Key points
- **Setup:** Training uses PDTB sections 2–19 and testing uses the enlarged sections 20–24. The classifiers are SVMLight with a linear kernel, with a balanced (downsampled) negative class.
  - Seven relations are covered: Comparison, Contingency, Expansion, Temporal, and the Expansion subtypes Conjunction, Instantiation and Restatement.
- **Lexical features:** Word pairs produce 92,128 features, compared with 12,116 for binary-lexical. Moving to binary-lexical raises average F from 29.46 to 31.79 and accuracy from 57.22 to 60.42. Instantiation gains 7.7 F points.
- **Sparsity:** The average rate of feature occurrence is:
  - sticks: 0.00623
  - prodrules: 0.00374
  - binary-lexical: 0.00276
  - word-pairs: 0.00113
- **Lexicalization:** A purely syntactic variant of the sticks representation (sticks-nolex) performs worse on most relations. Lexical items also make up many of the most predictive features.
- **Classifier agreement:** Q-statistic analysis shows that word-pairs and binary-lexical disagree on at least about 25% of test instances. Sticks and prodrules, by contrast, agree closely.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

Fields: [Linguistics & languages](https://md.research.elamur.ai/free/en/shelves/linguistics/)

Topics: [Discourse analysis](https://md.research.elamur.ai/free/en/topics/discourse-analysis/)

## Cited by

Cited by

13 papers of our library cite it (of the 140673 whose bibliographies we read); 8 of them have a page here, 5 not yet.

- [PDTB Discourse Parsing as a Tagging Task: The Two Taggers Approach](https://md.research.elamur.ai/free/en/papers/biran-2015-pdtb-discourse-parsing-as-tagging/)
- [Comparing Word Representations for Implicit Discourse Relation Classification](https://md.research.elamur.ai/free/en/papers/braud-2015-comparing-word-representations-implicit-discourse/)
- [Learning Connective-based Word Representations for Implicit Discourse Relation Identification](https://md.research.elamur.ai/free/en/papers/braud-2016-learning-connective-based-word-representations/)
- [One Vector is Not Enough: Entity-Augmented Distributed Semantics for Discourse Relations](https://md.research.elamur.ai/free/en/papers/ji-2015-one-vector-is-not-enough/)
- [Implicit Discourse Relation Classification: We Need to Talk about Evaluation](https://md.research.elamur.ai/free/en/papers/kim-2020-implicit-discourse-relation-classification-we/)
- [Automatic Prediction of Discourse Connectives](https://md.research.elamur.ai/free/en/papers/malmi-2018-automatic-prediction-discourse-connectives/)
- [Implicit Discourse Relation Recognition with Context-aware Character-enhanced Embeddings](https://md.research.elamur.ai/free/en/papers/qin-2016-implicit-discourse-relation-recognition-context/)
- [Improving the Inference of Implicit Discourse Relations via Classifying Explicit Discourse Connectives](https://md.research.elamur.ai/free/en/papers/rutherford-2015-improving-inference-implicit-discourse-relations/)

## Cite

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{li2014reducing,
  title = {{Reducing Sparsity Improves the Recognition of Implicit Discourse Relations}},
  author = {Li, Junyi Jessy and Nenkova, Ani},
  booktitle = {Proceedings of the 15th Annual Meeting of the Special Interest Group on Discourse and Dialogue (SIGDIAL)},
  year = {2014},
  doi = {10.3115/v1/w14-4327},
  url = {https://doi.org/10.3115/v1/w14-4327}
}
```

- [← previous RED: A Reference Dependency Based MT Evaluation Metric](https://md.research.elamur.ai/free/en/papers/yu-2014-red-reference-dependency-based-mt/)
- [next → ReferItGame: Referring to Objects in Photographs of Natural Scenes](https://md.research.elamur.ai/free/en/papers/kazemzadeh-2014-referitgame-referring-objects-photographs-natural/)
- [↑ back to papers of 2014](https://md.research.elamur.ai/free/en/papers/2014/page/2/)

## References

14 references, 9 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Or Biran and Kathleen McKeown. 2013. Aggregated word pair features for implicit discourse relation disambiguation. In Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics (ACL): Short Papers, pages 69–73. [find](https://scholar.google.com/scholar?q=Or+Biran+and+Kathleen+McKeown%2E+2013%2E+Aggregated+word+pair+features+for+implicit+discourse+relation+disambiguation%2E+In+Proceedings+of+the+51st+Annual+Meeting+of+the+Association+for+Computational+Lingui) [our card](https://md.research.elamur.ai/free/en/papers/biran-2013-aggregated-word-pair-features-implicit/)
2. Sasha Blair-Goldensohn, Kathleen McKeown, and Owen Rambow. 2007. Building and refining rhetorical-semantic relation models. In Human Language Technologies: The Conference of the North American Chapter of the Association for Computational Linguistics (NAACL), pages 428–435. [find](https://scholar.google.com/scholar?q=Sasha+Blair%2DGoldensohn%2C+Kathleen+McKeown%2C+and+Owen+Rambow%2E+2007%2E+Building+and+refining+rhetorical%2Dsemantic+relation+models%2E+In+Human+Language+Technologies%3A+The+Conference+of+the+North+American+Chapter)
3. Hugo Hernault, Danushka Bollegala, and Mitsuru Ishizuka. 2010. A semi-supervised approach to improve classification of infrequent discourse relations using feature vector extension. In Proceedings of the 2010 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 399–409. [find](https://scholar.google.com/scholar?q=Hugo+Hernault%2C+Danushka+Bollegala%2C+and+Mitsuru+Ishizuka%2E+2010%2E+A+semi%2Dsupervised+approach+to+improve+classification+of+infrequent+discourse+relations+using+feature+vector+extension%2E+In+Proceedings+of+) [our card](https://md.research.elamur.ai/free/en/papers/hernault-2010-semi-supervised-approach-improve-classification/)
4. Thorsten Joachims. 1999. Making large-scale support vector machine learning practical. In Advances in kernel methods, pages 169–184. [find](https://scholar.google.com/scholar?q=Thorsten+Joachims%2E+1999%2E+Making+large%2Dscale+support+vector+machine+learning+practical%2E+In+Advances+in+kernel+methods%2C+pages+169%E2%80%93184%2E)
5. Ludmila I. Kuncheva and Christopher J. Whitaker. 2003. Measures of diversity in classifier ensembles and their relationship with the ensemble accuracy. Machine Learning, 51(2):181–207, May. [find](https://scholar.google.com/scholar?q=Ludmila+I%2E+Kuncheva+and+Christopher+J%2E+Whitaker%2E+2003%2E+Measures+of+diversity+in+classifier+ensembles+and+their+relationship+with+the+ensemble+accuracy%2E+Machine+Learning%2C+51%282%29%3A181%E2%80%93207%2C+May%2E)
6. Ziheng Lin, Min-Yen Kan, and Hwee Tou Ng. 2009. Recognizing implicit discourse relations in the Penn Discourse Treebank. In Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 343–351. [find](https://scholar.google.com/scholar?q=Ziheng+Lin%2C+Min%2DYen+Kan%2C+and+Hwee+Tou+Ng%2E+2009%2E+Recognizing+implicit+discourse+relations+in+the+Penn+Discourse+Treebank%2E+In+Proceedings+of+the+2009+Conference+on+Empirical+Methods+in+Natural+Language+) [our card](https://md.research.elamur.ai/free/en/papers/lin-2009-recognizing-implicit-discourse-relations-penn/)
7. Ziheng Lin, Hwee Tou Ng, and Min-Yen Kan. 2014. A PDTB-styled end-to-end discourse parser. Natural Language Engineering, 20:151–184, 4. [find](https://scholar.google.com/scholar?q=Ziheng+Lin%2C+Hwee+Tou+Ng%2C+and+Min%2DYen+Kan%2E+2014%2E+A+PDTB%2Dstyled+end%2Dto%2Dend+discourse+parser%2E+Natural+Language+Engineering%2C+20%3A151%E2%80%93184%2C+4%2E)
8. Daniel Marcu and Abdessamad Echihabi. 2002. An unsupervised approach to recognizing discourse relations. In Proceedings of the 40th Annual Meeting on Association for Computational Linguistics (ACL), pages 368–375. [find](https://scholar.google.com/scholar?q=Daniel+Marcu+and+Abdessamad+Echihabi%2E+2002%2E+An+unsupervised+approach+to+recognizing+discourse+relations%2E+In+Proceedings+of+the+40th+Annual+Meeting+on+Association+for+Computational+Linguistics+%28ACL%29%2C+p) [our card](https://md.research.elamur.ai/free/en/papers/marcu-2002-unsupervised-approach-recognizing-discourse-relations/)
9. Mitchell P. Marcus, Mary Ann Marcinkiewicz, and Beatrice Santorini. 1993. Building a large annotated corpus of English: The Penn Treebank. Computational Linguistics - Special issue on using large corpora, 19(2):313–330. [find](https://scholar.google.com/scholar?q=Mitchell+P%2E+Marcus%2C+Mary+Ann+Marcinkiewicz%2C+and+Beatrice+Santorini%2E+1993%2E+Building+a+large+annotated+corpus+of+English%3A+The+Penn+Treebank%2E+Computational+Linguistics+%2D+Special+issue+on+using+large+corp) [our card](https://md.research.elamur.ai/free/en/papers/bank-1993-marcus-building-large-annotated-corpus-english/)
10. Joonsuk Park and Claire Cardie. 2012. Improving implicit discourse relation recognition through feature set optimization. In Proceedings of the 13th Annual Meeting of the Special Interest Group on Discourse and Dialogue (SIGDIAL), pages 108–112. [find](https://scholar.google.com/scholar?q=Joonsuk+Park+and+Claire+Cardie%2E+2012%2E+Improving+implicit+discourse+relation+recognition+through+feature+set+optimization%2E+In+Proceedings+of+the+13th+Annual+Meeting+of+the+Special+Interest+Group+on+Dis) [our card](https://md.research.elamur.ai/free/en/papers/park-2012-improving-implicit-discourse-relation-recognition/)
11. Emily Pitler, Annie Louis, and Ani Nenkova. 2009. Automatic sense prediction for implicit discourse relations in text. In Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing of the AFNLP (ACL-IJCNLP), pag… [find](https://scholar.google.com/scholar?q=Emily+Pitler%2C+Annie+Louis%2C+and+Ani+Nenkova%2E+2009%2E+Automatic+sense+prediction+for+implicit+discourse+relations+in+text%2E+In+Proceedings+of+the+Joint+Conference+of+the+47th+Annual+Meeting+of+the+ACL+and+) [our card](https://md.research.elamur.ai/free/en/papers/disc-2009-pitler-automatic-sense-prediction-implicit-discourse/)
12. Rashmi Prasad, Nikhil Dinesh, Alan Lee, Eleni Miltsakaki, Livio Robaldo, Aravind Joshi, and Bonnie Webber. 2008. The Penn Discourse TreeBank 2.0. In Proceedings of the Sixth International Conference on Language Resources and Evaluation (LREC). [find](https://scholar.google.com/scholar?q=Rashmi+Prasad%2C+Nikhil+Dinesh%2C+Alan+Lee%2C+Eleni+Miltsakaki%2C+Livio+Robaldo%2C+Aravind+Joshi%2C+and+Bonnie+Webber%2E+2008%2E+The+Penn+Discourse+TreeBank+2%2E0%2E+In+Proceedings+of+the+Sixth+International+Conference+o) [our card](https://md.research.elamur.ai/free/en/papers/disc-2008-prasad-penn-discourse-treebank-2-0/)
13. Caroline Sporleder and Alex Lascarides. 2008. Using automatically labelled examples to classify rhetorical relations: An assessment. Natural Language Engineering, 14(3):369–416, July. [find](https://scholar.google.com/scholar?q=Caroline+Sporleder+and+Alex+Lascarides%2E+2008%2E+Using+automatically+labelled+examples+to+classify+rhetorical+relations%3A+An+assessment%2E+Natural+Language+Engineering%2C+14%283%29%3A369%E2%80%93416%2C+July%2E)
14. Zhi-Min Zhou, Yu Xu, Zheng-Yu Niu, Man Lan, Jian Su, and Chew Lim Tan. 2010. Predicting discourse connectives for implicit discourse relation recognition. In Proceedings of the 23rd International Conference on Computational Linguistics (COLING), pages 1507–1514. [find](https://scholar.google.com/scholar?q=Zhi%2DMin+Zhou%2C+Yu+Xu%2C+Zheng%2DYu+Niu%2C+Man+Lan%2C+Jian+Su%2C+and+Chew+Lim+Tan%2E+2010%2E+Predicting+discourse+connectives+for+implicit+discourse+relation+recognition%2E+In+Proceedings+of+the+23rd+International+Conf) [our card](https://md.research.elamur.ai/free/en/papers/zhou-2010-predicting-discourse-connectives-implicit-discourse/)