---
title: "Stack-propagation: Improved Representation Learning for Syntax"
description: "Zhang and Weiss propose \"stack-propagation\", which treats POS supervision as a regularizer rather… 25 references, cited by 43 papers here."
authors: ["Yuan Zhang", "David Weiss"]
year: 2016
venue: "Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)"
source: "https://aclanthology.org/P16-1147/"
doi: "10.18653/v1/p16-1147"
arxiv: "1603.06598"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/zhang-2016-stack-propagation-improved-representation-learning/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/zhang-2016-stack-propagation-improved-representation-learning/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2016](https://md.research.elamur.ai/free/en/papers/2016/page/3/) / Stack-propagation: Improved Representation Learning for Syntax

# Stack-propagation: Improved Representation Learning for Syntax

**Authors:** [Yuan Zhang](https://md.research.elamur.ai/free/en/authors/yuan-zhang/) and [David Weiss](https://md.research.elamur.ai/free/en/authors/david-weiss/) · **Year:** 2016 · **Venue:** [Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)](https://md.research.elamur.ai/free/en/venues/acl/) ([ACL 2016](https://md.research.elamur.ai/free/en/venues/acl/2016/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/P16-1147/) · [PDF](https://aclanthology.org/P16-1147.pdf) · [DOI 10.18653/v1/p16-1147](https://doi.org/10.18653/v1/p16-1147) · [arXiv 1603.06598](https://arxiv.org/abs/1603.06598) · [OpenAlex](https://openalex.org/W2311095070)

**Citations:** 62 citations (OpenAlex, 2026-10-14) · [43](https://md.research.elamur.ai/free/en/papers/zhang-2016-stack-propagation-improved-representation-learning/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
Neural dependency parsers usually take POS tags from a separately trained tagger. Tagging mistakes then carry over into the parse, and the tagger cannot use syntactic context. Zhang and Weiss propose "stack-propagation", which treats POS supervision as a regularizer rather than an input feature.

A window-based feed-forward tagger and a greedy transition-based feed-forward parser, both in the style of Chen and Manning (2014) and Weiss et al. (2015), are linked directly. The parser's feature templates pull the tagger's hidden-layer activations for tokens on the stack and buffer, instead of discrete tags. Training alternates stochastic updates between the tagging and parsing objectives, and parser gradients flow back into the tagger. At test time, the parser needs no predicted tags.

On 19 Universal Dependencies languages, the model improves labeled accuracy by 1.3% absolute over a strong graph-based parser and by 2.7% over a character-LSTM greedy parser. It is also smaller and faster. The work shows that upstream annotations can shape learned representations instead of acting as fragile pipeline inputs.

## Key points
- Evaluation on 19 UD treebanks: +1.3% absolute over the graph-based parser of Lei et al. (2014) and +2.7% over the greedy LSTM parser of Ballesteros et al. (2015).
- Cascaded errors drop sharply. On tokens the pipeline tagger mislabels, LAS rises from 34.1% to 45.0% (+10.9%), compared with a +1.8% gain on all other tokens.
- Because the parser and tagger share parameters, the combined model has roughly half the parameters of the pipeline baseline and runs about twice as fast.
- On the WSJ corpus, stackprop performs about as well with the coarse universal tagset as with fine tags, while the pipeline baseline degrades strongly. It beats earlier greedy parsers such as Dyer et al. (2015).
- Stackprop outperforms joint POS+parsing modeling with the transition system of Bohnet and Nivre (2012). Combining stackprop with the joint system improves tagging but slightly lowers parsing accuracy.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Massachusetts Institute of Technology](https://md.research.elamur.ai/free/en/universities/massachusetts-institute-of-technology/) — University, Cambridge, United States · 211 works · 69,000 citations
- [MIT Computer Science and Artificial Intelligence Laboratory](https://md.research.elamur.ai/free/en/labs/mit-computer-science-and-artificial-intelligence-laboratory/) — Research institute, Cambridge, United States · 69 works · 30,953 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works

🧭 Topics (1): [📐 Syntax & grammar formalisms](https://md.research.elamur.ai/free/en/topics/syntax-grammar/) · 214 works

## Cite

Cite as: Yuan Zhang and David Weiss. 2016. Stack-propagation: Improved Representation Learning for Syntax. Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{zhang2016stack,
  title = {{Stack-propagation: Improved Representation Learning for Syntax}},
  author = {Yuan Zhang and David Weiss},
  booktitle = {Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2016},
  doi = {10.18653/v1/p16-1147},
  eprint = {1603.06598},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.18653/v1/p16-1147}
}
```

- [← previous Analyzing lexical emergence in Modern American English online](https://md.research.elamur.ai/free/en/papers/grieve-2016-lexical-emergence-american-english/)
- [next → Very-large Scale Parsing and Normalization of Wiktionary Morphological Paradigms](https://md.research.elamur.ai/free/en/papers/kirov-2016-wiktionary-paradigms/)
- [↑ back to papers of 2016](https://md.research.elamur.ai/free/en/papers/2016/page/3/)

## References

25 references, 11 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Léon Bottou. 2010. Large-scale machine learning with stochastic gradient descent. In Proceedings of COMPSTAT’2010, pages 177–186. Springer. [find](https://scholar.google.com/scholar?q=Le%CC%81on+Bottou%2E+2010%2E+Large%2Dscale+machine+learning+with+stochastic+gradient+descent%2E+In+Proceedings+of+COMPSTAT%E2%80%992010%2C+pages+177%E2%80%93186%2E+Springer%2E)
2. Danqi Chen and Christopher D Manning. 2014. [A fast and accurate dependency parser using neural networks](https://md.research.elamur.ai/free/en/papers/chen-2014-fast-accurate-dependency-parser-neural/). In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), volume 1, pages 740–750.
3. Ronan Collobert, Jason Weston, Léon Bottou, Michael Karlen, Koray Kavukcuoglu, and Pavel P. Kuksa. 2011. [Natural language processing (almost) from scratch](https://md.research.elamur.ai/free/en/papers/collobert-2011-natural-language-processing-almost-scratch/). JMLR.
4. Marie-Catherine De Marneffe, Bill MacCartney, and Christopher D. Manning. 2006. [Generating typed dependency parses from phrase structure parses](https://md.research.elamur.ai/free/en/papers/marneffe-2006-generating-typed-dependency-parses-phrase/). In Proceedings of Fifth International Conference on Language Resources and Evaluation, pages 449– 454.
5. Greg Durrett and Dan Klein. 2015. Neural crf parsing. In Proceedings of the Association for Computational Linguistics. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Greg+Durrett+and+Dan+Klein%2E+2015%2E+Neural+crf+parsing%2E+In+Proceedings+of+the+Association+for+Computational+Linguistics%2E+Association+for+Computational+Linguistics%2E)
6. Chris Dyer, Miguel Ballesteros, Wang Ling, Austin Matthews, and Noah A Smith. 2015. [Transition-based dependency parsing with stack long short-term memory](https://md.research.elamur.ai/free/en/papers/dyer-2015-transition-based-dependency-parsing-stack/). ACL.
7. Nicholas FitzGerald, Oscar Tckstrm, Kuzman Ganchev, and Dipanjan Das. 2015. Semantic role labeling with neural network factors. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing (EMNLP ’15). [find](https://scholar.google.com/scholar?q=Nicholas+FitzGerald%2C+Oscar+Tckstrm%2C+Kuzman+Ganchev%2C+and+Dipanjan+Das%2E+2015%2E+Semantic+role+labeling+with+neural+network+factors%2E+In+Proceedings+of+the+2015+Conference+on+Empirical+Methods+in+Natural+La)
8. Jun Hatori, Takuya Matsuzaki, Yusuke Miyao, and Jun’ichi Tsujii. 2011. Incremental joint pos tagging and dependency parsing in chinese. In IJC-NLP, pages 1216–1224. Citeseer. [find](https://scholar.google.com/scholar?q=Jun+Hatori%2C+Takuya+Matsuzaki%2C+Yusuke+Miyao%2C+and+Jun%E2%80%99ichi+Tsujii%2E+2011%2E+Incremental+joint+pos+tagging+and+dependency+parsing+in+chinese%2E+In+IJC%2DNLP%2C+pages+1216%E2%80%931224%2E+Citeseer%2E)
9. Geoffrey E Hinton. 2012. A practical guide to training restricted boltzmann machines. In Neural Networks: Tricks of the Trade, pages 599–619. Springer. [find](https://scholar.google.com/scholar?q=Geoffrey+E+Hinton%2E+2012%2E+A+practical+guide+to+training+restricted+boltzmann+machines%2E+In+Neural+Networks%3A+Tricks+of+the+Trade%2C+pages+599%E2%80%93619%2E+Springer%2E)
10. Chris Alberti, David Weiss, Greg Coppola, and Slav Petrov. 2015. Improved transition-based parsing and tagging with neural networks. In Proceedings of EMNLP 2015. [find](https://scholar.google.com/scholar?q=Chris+Alberti%2C+David+Weiss%2C+Greg+Coppola%2C+and+Slav+Petrov%2E+2015%2E+Improved+transition%2Dbased+parsing+and+tagging+with+neural+networks%2E+In+Proceedings+of+EMNLP+2015%2E)
11. Terry Koo, Xavier Carreras Pérez, and Michael Collins. 2008. [Simple semi-supervised dependency parsing](https://md.research.elamur.ai/free/en/papers/koo-2008-simple-semi-supervised-dependency-parsing/). In 46th Annual Meeting of the Association for Computational Linguistics, pages 595–603.
12. Miguel Ballesteros, Chris Dyer, and Noah A. Smith. 2015. [Improved transition-based parsing by modeling characters instead of words with lstms](https://md.research.elamur.ai/free/en/papers/ballesteros-2015-improved-transition-based-parsing-modeling/). Proceddings of EMNLP.
13. Tao Lei, Yu Xin, Yuan Zhang, Regina Barzilay, and Tommi Jaakkola. 2014. Low-rank tensors for scoring dependency structures. In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics, volume 1, pages 1381–1391. [find](https://scholar.google.com/scholar?q=Tao+Lei%2C+Yu+Xin%2C+Yuan+Zhang%2C+Regina+Barzilay%2C+and+Tommi+Jaakkola%2E+2014%2E+Low%2Drank+tensors+for+scoring+dependency+structures%2E+In+Proceedings+of+the+52nd+Annual+Meeting+of+the+Association+for+Computation)
14. Bernd Bohnet and Joakim Nivre. 2012. [A transition-based system for joint part-of-speech tagging and labeled non-projective dependency parsing](https://md.research.elamur.ai/free/en/papers/bohnet-2012-transition-based-system-joint-part/). In Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning, pages 1455–1465.…
15. Zhenghua Li, Min Zhang, Wanxiang Che, Ting Liu, Wenliang Chen, and Haizhou Li. 2011. Joint models for chinese pos tagging and dependency parsing. In Proceedings of the Conference on Empirical Methods in Natural Language Processing, pages 1180–1191. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Zhenghua+Li%2C+Min+Zhang%2C+Wanxiang+Che%2C+Ting+Liu%2C+Wenliang+Chen%2C+and+Haizhou+Li%2E+2011%2E+Joint+models+for+chinese+pos+tagging+and+dependency+parsing%2E+In+Proceedings+of+the+Conference+on+Empirical+Methods+)
16. Zhenghua Li, Min Zhang, Wanxiang Che, Ting Liu, and Wenliang Chen. 2014. Joint optimization for chinese POS tagging and dependency parsing. IEEE/ACM Transactions on Audio, Speech & Language Processing, pages 274–286. [find](https://scholar.google.com/scholar?q=Zhenghua+Li%2C+Min+Zhang%2C+Wanxiang+Che%2C+Ting+Liu%2C+and+Wenliang+Chen%2E+2014%2E+Joint+optimization+for+chinese+POS+tagging+and+dependency+parsing%2E+IEEE%2FACM+Transactions+on+Audio%2C+Speech+%26+Language+Processing)
17. Mitchell P. Marcus, Beatrice Santorini, and Mary Ann Marcinkiewicz. 1993. [Building a large annotated corpus of English: The Penn Treebank](https://md.research.elamur.ai/free/en/papers/marcus-1993-building-large-annotated-corpus-english/). Computational Linguistics, 19(2):313–330.
18. Joakim Nivre. 2004. [Incrementality in deterministic dependency parsing](https://md.research.elamur.ai/free/en/papers/nivre-2004-incrementality-deterministic-dependency-parsing/). In Proceedings of the Workshop on Incremental Parsing: Bringing Engineering and Cognition Together, IncrementParsing ’04, pages 50–57, Stroudsburg, PA, USA. Association for Computational Linguistics.
19. Joakim Nivre. 2009. [Non-projective dependency parsing in expected linear time](https://md.research.elamur.ai/free/en/papers/nivre-2009-non-projective-dependency-parsing-expected/). In Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing of the AFNLP, pages 351–359.
20. Slav Petrov, Dipanjan Das, and Ryan McDonald. 2011. [A universal part-of-speech tagset](https://md.research.elamur.ai/free/en/papers/petrov-2012-universal-part-speech-tagset/). arXiv preprint arXiv:1104.2086.
21. Xian Qian and Yang Liu. 2012. Joint chinese word segmentation, pos tagging and parsing. In Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning, pages 501– 511. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Xian+Qian+and+Yang+Liu%2E+2012%2E+Joint+chinese+word+segmentation%2C+pos+tagging+and+parsing%2E+In+Proceedings+of+the+2012+Joint+Conference+on+Empirical+Methods+in+Natural+Language+Processing+and+Computationa)
22. Zhiguo Wang and Nianwen Xue. 2014. Joint pos tagging and transition-based constituent parsing in chinese with non-local features. In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics, pages 733–742. [find](https://scholar.google.com/scholar?q=Zhiguo+Wang+and+Nianwen+Xue%2E+2014%2E+Joint+pos+tagging+and+transition%2Dbased+constituent+parsing+in+chinese+with+non%2Dlocal+features%2E+In+Proceedings+of+the+52nd+Annual+Meeting+of+the+Association+for+Compu)
23. David Weiss, Chris Alberti, Michael Collins, and Slav Petrov. 2015. Structured training for neural network transition-based parsing. In Proceedings of ACL 2015, pages 323–333. [find](https://scholar.google.com/scholar?q=David+Weiss%2C+Chris+Alberti%2C+Michael+Collins%2C+and+Slav+Petrov%2E+2015%2E+Structured+training+for+neural+network+transition%2Dbased+parsing%2E+In+Proceedings+of+ACL+2015%2C+pages+323%E2%80%93333%2E)
24. David H Wolpert. 1992. Stacked generalization. Neural networks. [find](https://scholar.google.com/scholar?q=David+H+Wolpert%2E+1992%2E+Stacked+generalization%2E+Neural+networks%2E)
25. Yuan Zhang, Chengtao Li, Regina Barzilay, and Kareem Darwish. 2015. Randomized greedy inference for joint segmentation, POS tagging and dependency parsing. In Proceedings of the 2015 Conference of the North American Chapter of the Association for Computational Linguistics, pages 42–52. [find](https://scholar.google.com/scholar?q=Yuan+Zhang%2C+Chengtao+Li%2C+Regina+Barzilay%2C+and+Kareem+Darwish%2E+2015%2E+Randomized+greedy+inference+for+joint+segmentation%2C+POS+tagging+and+dependency+parsing%2E+In+Proceedings+of+the+2015+Conference+of+the)

*Updated 29 October 2026*