---
title: "A Fast and Accurate Dependency Parser using — Chen 2014"
description: "Almost all parsers of the time used millions of sparse indicator features built from hand-made templates, which generalise poorly and make feature…"
authors: ["Danqi Chen", "Christopher Manning"]
year: 2014
venue: "Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP)"
source: "https://aclanthology.org/D14-1082/"
doi: "10.3115/v1/d14-1082"
licence: "CC BY-NC-SA 3.0 (ACL Anthology, before 2016)"
licence_url: "https://creativecommons.org/licenses/by-nc-sa/3.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/missing-2014-chen-fast-accurate-dependency-parser-neural/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/missing-2014-chen-fast-accurate-dependency-parser-neural/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2014](https://md.research.elamur.ai/free/en/papers/2014/) / A Fast and Accurate Dependency Parser using Neural Networks

# A Fast and Accurate Dependency Parser using Neural Networks

**Authors:** Danqi Chen, Christopher D. Manning · **Year:** 2014 · **Venue:** Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) **Link:** [aclanthology.org/D14-1082](https://aclanthology.org/D14-1082/) · **Licence:** [CC BY-NC-SA 3.0](https://creativecommons.org/licenses/by-nc-sa/3.0/) — summary only

[**1795** citations](https://openalex.org/works?filter=cites:W2250861254) OpenAlex · 2026-10-14 · [**506** citations in our library](https://md.research.elamur.ai/free/en/papers/missing-2014-chen-fast-accurate-dependency-parser-neural/cited-by/) of the 140673 papers whose bibliographies we read · [Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP)](https://aclanthology.org/volumes/D14-1/) · [licence](https://creativecommons.org/licenses/by-nc-sa/3.0/)

- [Publisher's page](https://aclanthology.org/D14-1082/)
- [PDF](https://aclanthology.org/D14-1082.pdf)
- [DOI 10.3115/v1/d14-1082](https://doi.org/10.3115/v1/d14-1082)
- [OpenAlex](https://openalex.org/W2250861254)

**Authors (2)**

1. [Danqi Chen](https://openalex.org/A5051064208) [ORCID](https://orcid.org/0000-0002-6226-6838) · [Semantic Scholar](https://www.semanticscholar.org/author/50536468) · h 45 · 39918 citations · 236 works
2. [Christopher Manning](https://openalex.org/A5046006076) [ORCID](https://orcid.org/0000-0001-6155-649X) · [Semantic Scholar](https://www.semanticscholar.org/author/144783904) · h 139 · 180101 citations · 770 works

Citations and author metrics: OpenAlex, 2026-10-14; authors with the same name are not always told apart.

## What it is about

Almost all parsers of the time used millions of sparse indicator features built from hand-made templates, which generalise poorly and make feature extraction the main cost. This paper proposes a greedy arc-standard parser whose decisions come from a small feedforward network over dense, learned embeddings of words, POS tags and arc labels taken from the stack top, the buffer front and their children. The network uses only about 200 learned dense features, and a cube activation to capture interactions between them. It parses over 1000 sentences per second and reaches 92.2% UAS on the English Penn Treebank, with gains on Chinese too. The ablation study shows that lexicalised features and word-pair features of the top stack items carry most of the signal in the old sparse set.

## Ideas

- I1. en/src/parse.rs: add a from-scratch dense scorer (embeddings for the same stack and buffer slots, a small MLP with cube activation, trained inside the parser) as an extra score beside the perceptron, and measure LAS on EWT dev and sentences per second on CPU through `en parse-eval`.
- I2. en/src/parse.rs: ablate the stack and buffer slots one class at a time (words, tags, labels, children) and measure the LAS loss on EWT dev, to find which of the feature classes the perceptron actually needs.
- I3. en/src/parse.rs: compare the cube activation with tanh in the dense scorer of I1 on EWT dev, twice, to check that the choice of activation, not the size, makes the difference.

## For Mova

The ideas touch the scorer in `en/src/parse.rs` and its feature slots, and they respect the rule that the weights come from scratch on training data only. The chapters are `languages/en/10-word-classes` (the POS features) and `languages/en/20-syntax-overview`. Measure first I2, which uses only the current feature templates, and then I1 under the speed budget that this paper sets.

Topics: [Dependency parsing](https://md.research.elamur.ai/free/en/topics/dependency-parsing/)

## Cite

The paper itself, not this card. BibTeX from the library's record of the work. [DOI 10.3115/v1/d14-1082](https://doi.org/10.3115/v1/d14-1082)

```bibtex
@inproceedings{chen2014fast,
  title = {{A Fast and Accurate Dependency Parser using Neural Networks}},
  author = {Chen, Danqi and Manning, Christopher},
  booktitle = {Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP)},
  year = {2014},
  doi = {10.3115/v1/d14-1082},
  url = {https://doi.org/10.3115/v1/d14-1082}
}
```

- [← previous A Dependency Parser for Tweets](https://md.research.elamur.ai/free/en/papers/aclbulk-2014-kong-dependency-parser-tweets/)
- [next → A Generic Anaphora Resolution Engine for Indian Languages](https://md.research.elamur.ai/free/en/papers/devi-2014-generic-anaphora-resolution-engine-indian/)
- [↑ back to papers of 2014](https://md.research.elamur.ai/free/en/papers/2014/)

## References

29 references, 10 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Bernd Bohnet. 2010. Very high accuracy and fast dependency parsing is not a contradiction. In Coling. [find](https://scholar.google.com/scholar?q=Bernd+Bohnet%2E+2010%2E+Very+high+accuracy+and+fast+dependency+parsing+is+not+a+contradiction%2E+In+Coling%2E) [our card](https://md.research.elamur.ai/free/en/papers/bohnet-2010-top-accuracy-fast-dependency-parsing/)
2. Ronan Collobert, Jason Weston, Léon Bottou, Michael Karlen, Koray Kavukcuoglu, and Pavel Kuksa. 2011. Natural language processing (almost) from scratch. Journal of Machine Learning Research. [find](https://scholar.google.com/scholar?q=Ronan+Collobert%2C+Jason+Weston%2C+Le%CC%81on+Bottou%2C+Michael+Karlen%2C+Koray+Kavukcuoglu%2C+and+Pavel+Kuksa%2E+2011%2E+Natural+language+processing+%28almost%29+from+scratch%2E+Journal+of+Machine+Learning+Research%2E) [our card](https://md.research.elamur.ai/free/en/papers/collobert-2011-natural-language-processing-almost-scratch/)
3. Ronan Collobert. 2011. Deep learning for efficient discriminative parsing. In AISTATS. [find](https://scholar.google.com/scholar?q=Ronan+Collobert%2E+2011%2E+Deep+learning+for+efficient+discriminative+parsing%2E+In+AISTATS%2E)
4. Marie-Catherine de Marneffe, Bill MacCartney, and Christopher D. Manning. 2006. Generating typed dependency parses from phrase structure parses. In LREC. [find](https://scholar.google.com/scholar?q=Marie%2DCatherine+de+Marneffe%2C+Bill+MacCartney%2C+and+Christopher+D%2E+Manning%2E+2006%2E+Generating+typed+dependency+parses+from+phrase+structure+parses%2E+In+LREC%2E) [our card](https://md.research.elamur.ai/free/en/papers/map-2006-marneffe-generating-typed-dependency-parses-phrase/)
5. Jacob Devlin, Rabih Zbib, Zhongqiang Huang, Thomas Lamar, Richard Schwartz, and John Makhoul. 2014. Fast and robust neural network joint models for statistical machine translation. In ACL. [find](https://scholar.google.com/scholar?q=Jacob+Devlin%2C+Rabih+Zbib%2C+Zhongqiang+Huang%2C+Thomas+Lamar%2C+Richard+Schwartz%2C+and+John+Makhoul%2E+2014%2E+Fast+and+robust+neural+network+joint+models+for+statistical+machine+translation%2E+In+ACL%2E)
6. John Duchi, Elad Hazan, and Yoram Singer. 2011. Adaptive subgradient methods for online learning and stochastic optimization. The Journal of Machine Learning Research. [find](https://scholar.google.com/scholar?q=John+Duchi%2C+Elad+Hazan%2C+and+Yoram+Singer%2E+2011%2E+Adaptive+subgradient+methods+for+online+learning+and+stochastic+optimization%2E+The+Journal+of+Machine+Learning+Research%2E)
7. Rong-En Fan, Kai-Wei Chang, Cho-Jui Hsieh, Xiang-Rui Wang, and Chih-Jen Lin. 2008. Liblinear: A library for large linear classification. The Journal of Machine Learning Research. [find](https://scholar.google.com/scholar?q=Rong%2DEn+Fan%2C+Kai%2DWei+Chang%2C+Cho%2DJui+Hsieh%2C+Xiang%2DRui+Wang%2C+and+Chih%2DJen+Lin%2E+2008%2E+Liblinear%3A+A+library+for+large+linear+classification%2E+The+Journal+of+Machine+Learning+Research%2E)
8. Nikhil Garg and James Henderson. 2011. Temporal restricted boltzmann machines for dependency parsing. In ACL-HLT. [find](https://scholar.google.com/scholar?q=Nikhil+Garg+and+James+Henderson%2E+2011%2E+Temporal+restricted+boltzmann+machines+for+dependency+parsing%2E+In+ACL%2DHLT%2E)
9. He He, Hal Daumé III, and Jason Eisner. 2013. Dynamic feature selection for dependency parsing. In EMNLP. [find](https://scholar.google.com/scholar?q=He+He%2C+Hal+Daume%CC%81+III%2C+and+Jason+Eisner%2E+2013%2E+Dynamic+feature+selection+for+dependency+parsing%2E+In+EMNLP%2E)
10. James Henderson. 2004. Discriminative training of a neural network statistical parser. In ACL. [find](https://scholar.google.com/scholar?q=James+Henderson%2E+2004%2E+Discriminative+training+of+a+neural+network+statistical+parser%2E+In+ACL%2E)
11. Geoffrey E. Hinton, Nitish Srivastava, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. 2012. Improving neural networks by preventing co-adaptation of feature detectors. CoRR, abs/1207.0580. [find](https://scholar.google.com/scholar?q=Geoffrey+E%2E+Hinton%2C+Nitish+Srivastava%2C+Alex+Krizhevsky%2C+Ilya+Sutskever%2C+and+Ruslan+Salakhutdinov%2E+2012%2E+Improving+neural+networks+by+preventing+co%2Dadaptation+of+feature+detectors%2E+CoRR%2C+abs%2F1207%2E0580%2E)
12. Liang Huang, Wenbin Jiang, and Qun Liu. 2009. Bilingually-constrained (monolingual) shift-reduce parsing. In EMNLP. [find](https://scholar.google.com/scholar?q=Liang+Huang%2C+Wenbin+Jiang%2C+and+Qun+Liu%2E+2009%2E+Bilingually%2Dconstrained+%28monolingual%29+shift%2Dreduce+parsing%2E+In+EMNLP%2E)
13. Richard Johansson and Pierre Nugues. 2007. Extended constituent-to-dependency conversion for english. In Proceedings of NODALIDA, Tartu, Estonia. [find](https://scholar.google.com/scholar?q=Richard+Johansson+and+Pierre+Nugues%2E+2007%2E+Extended+constituent%2Dto%2Ddependency+conversion+for+english%2E+In+Proceedings+of+NODALIDA%2C+Tartu%2C+Estonia%2E) [our card](https://md.research.elamur.ai/free/en/papers/map-2007-johansson-extended-constituent-dependency-conversion-english/)
14. Lingpeng Kong and Noah A. Smith. 2014. An empirical comparison of parsing methods for Stanford dependencies. CoRR, abs/1404.4314. [find](https://scholar.google.com/scholar?q=Lingpeng+Kong+and+Noah+A%2E+Smith%2E+2014%2E+An+empirical+comparison+of+parsing+methods+for+Stanford+dependencies%2E+CoRR%2C+abs%2F1404%2E4314%2E) [our card](https://md.research.elamur.ai/free/en/papers/enud-2014-kong-empirical-comparison-parsing-methods-stanford/)
15. Terry Koo, Xavier Carreras, and Michael Collins. 2008. Simple semi-supervised dependency parsing. In ACL. [find](https://scholar.google.com/scholar?q=Terry+Koo%2C+Xavier+Carreras%2C+and+Michael+Collins%2E+2008%2E+Simple+semi%2Dsupervised+dependency+parsing%2E+In+ACL%2E) [our card](https://md.research.elamur.ai/free/en/papers/koo-2008-simple-semi-supervised-dependency-parsing/)
16. Sandra Kübler, Ryan McDonald, and Joakim Nivre. 2009. Dependency Parsing. Synthesis Lectures on Human Language Technologies. Morgan & Claypool. [find](https://scholar.google.com/scholar?q=Sandra+Ku%CC%88bler%2C+Ryan+McDonald%2C+and+Joakim+Nivre%2E+2009%2E+Dependency+Parsing%2E+Synthesis+Lectures+on+Human+Language+Technologies%2E+Morgan+%26+Claypool%2E)
17. Marshall R. Mayberry III and Risto Miikkulainen. 1999. Sardsrn: A neural network shift-reduce parser. In IJCAI. [find](https://scholar.google.com/scholar?q=Marshall+R%2E+Mayberry+III+and+Risto+Miikkulainen%2E+1999%2E+Sardsrn%3A+A+neural+network+shift%2Dreduce+parser%2E+In+IJCAI%2E)
18. Marshall R. Mayberry III and Risto Miikkulainen. 2005. Broad-coverage parsing with neural networks. Neural Processing Letters. [find](https://scholar.google.com/scholar?q=Marshall+R%2E+Mayberry+III+and+Risto+Miikkulainen%2E+2005%2E+Broad%2Dcoverage+parsing+with+neural+networks%2E+Neural+Processing+Letters%2E)
19. Ryan McDonald and Fernando Pereira. 2006. Online learning of approximate dependency parsing algorithms. In EACL. [find](https://scholar.google.com/scholar?q=Ryan+McDonald+and+Fernando+Pereira%2E+2006%2E+Online+learning+of+approximate+dependency+parsing+algorithms%2E+In+EACL%2E) [our card](https://md.research.elamur.ai/free/en/papers/continual-2006-mcdonald-online-learning-approximate-dependency-parsing/)
20. Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S Corrado, and Jeff Dean. 2013. Distributed representations of words and phrases and their compositionality. In NIPS. [find](https://scholar.google.com/scholar?q=Tomas+Mikolov%2C+Ilya+Sutskever%2C+Kai+Chen%2C+Greg+S+Corrado%2C+and+Jeff+Dean%2E+2013%2E+Distributed+representations+of+words+and+phrases+and+their+compositionality%2E+In+NIPS%2E)
21. Joakim Nivre, Johan Hall, and Jens Nilsson. 2006. Maltparser: A data-driven parser-generator for dependency parsing. In LREC. [find](https://scholar.google.com/scholar?q=Joakim+Nivre%2C+Johan+Hall%2C+and+Jens+Nilsson%2E+2006%2E+Maltparser%3A+A+data%2Ddriven+parser%2Dgenerator+for+dependency+parsing%2E+In+LREC%2E) [our card](https://md.research.elamur.ai/free/en/papers/nivre-2006-maltparser-data-driven-parser-generator/)
22. Richard Socher, John Bauer, Christopher D Manning, and Andrew Y Ng. 2013. Parsing with compositional vector grammars. In ACL. [find](https://scholar.google.com/scholar?q=Richard+Socher%2C+John+Bauer%2C+Christopher+D+Manning%2C+and+Andrew+Y+Ng%2E+2013%2E+Parsing+with+compositional+vector+grammars%2E+In+ACL%2E)
23. Richard Socher, Andrej Karpathy, Quoc V. Le, Christopher D. Manning, and Andrew Y. Ng. 2014. Grounded compositional semantics for finding and describing images with sentences. TACL. [find](https://scholar.google.com/scholar?q=Richard+Socher%2C+Andrej+Karpathy%2C+Quoc+V%2E+Le%2C+Christopher+D%2E+Manning%2C+and+Andrew+Y%2E+Ng%2E+2014%2E+Grounded+compositional+semantics+for+finding+and+describing+images+with+sentences%2E+TACL%2E) [our card](https://md.research.elamur.ai/free/en/papers/socher-2014-grounded-compositional-semantics-finding-describing/)
24. Pontus Stenetorp. 2013. Transition-based dependency parsing using recursive neural networks. In NIPS Workshop on Deep Learning. [find](https://scholar.google.com/scholar?q=Pontus+Stenetorp%2E+2013%2E+Transition%2Dbased+dependency+parsing+using+recursive+neural+networks%2E+In+NIPS+Workshop+on+Deep+Learning%2E)
25. Ivan Titov and James Henderson. 2007. Fast and robust multilingual dependency parsing with a generative latent variable model. In EMNLP-CoNLL. [find](https://scholar.google.com/scholar?q=Ivan+Titov+and+James+Henderson%2E+2007%2E+Fast+and+robust+multilingual+dependency+parsing+with+a+generative+latent+variable+model%2E+In+EMNLP%2DCoNLL%2E)
26. Kristina Toutanova, Dan Klein, Christopher D. Manning, and Yoram Singer. 2003. Feature-rich part-of-speech tagging with a cyclic dependency network. In NAACL. [find](https://scholar.google.com/scholar?q=Kristina+Toutanova%2C+Dan+Klein%2C+Christopher+D%2E+Manning%2C+and+Yoram+Singer%2E+2003%2E+Feature%2Drich+part%2Dof%2Dspeech+tagging+with+a+cyclic+dependency+network%2E+In+NAACL%2E)
27. Laurens van der Maaten and Geoffrey Hinton. 2008. Visualizing data using t-SNE. The Journal of Machine Learning Research. [find](https://scholar.google.com/scholar?q=Laurens+van+der+Maaten+and+Geoffrey+Hinton%2E+2008%2E+Visualizing+data+using+t%2DSNE%2E+The+Journal+of+Machine+Learning+Research%2E)
28. Yue Zhang and Stephen Clark. 2008. A tale of two parsers: Investigating and combining graph-based and transition-based dependency parsing using beam-search. In EMNLP. [find](https://scholar.google.com/scholar?q=Yue+Zhang+and+Stephen+Clark%2E+2008%2E+A+tale+of+two+parsers%3A+Investigating+and+combining+graph%2Dbased+and+transition%2Dbased+dependency+parsing+using+beam%2Dsearch%2E+In+EMNLP%2E) [our card](https://md.research.elamur.ai/free/en/papers/zhang-2008-tale-two-parsers-investigating-combining/)
29. Yue Zhang and Joakim Nivre. 2011. Transition-based dependency parsing with rich non-local features. In ACL. [find](https://scholar.google.com/scholar?q=Yue+Zhang+and+Joakim+Nivre%2E+2011%2E+Transition%2Dbased+dependency+parsing+with+rich+non%2Dlocal+features%2E+In+ACL%2E)