---
title: "Hierarchical Low-Rank Tensors for Multilingual Transfer Parsing"
description: "Transfer parsing builds a dependency parser for a target language that has little or no… 77.9 UAS. 31 references, cited by 42 papers here."
authors: ["Yuan Zhang", "Regina Barzilay"]
year: 2015
venue: "Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing"
source: "https://aclanthology.org/D15-1213/"
doi: "10.18653/v1/d15-1213"
licence: "CC BY-NC-SA 3.0 (ACL Anthology, before 2016)"
licence_url: "https://creativecommons.org/licenses/by-nc-sa/3.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/zhang-2015-hierarchical-low-rank-tensors-multilingual/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/zhang-2015-hierarchical-low-rank-tensors-multilingual/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2015](https://md.research.elamur.ai/free/en/papers/2015/page/2/) / Hierarchical Low-Rank Tensors for Multilingual Transfer Parsing

# Hierarchical Low-Rank Tensors for Multilingual Transfer Parsing

**Authors:** [Yuan Zhang](https://md.research.elamur.ai/free/en/authors/yuan-zhang/) and [Regina Barzilay](https://md.research.elamur.ai/free/en/authors/regina-barzilay/) · **Year:** 2015 · **Venue:** [Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing](https://md.research.elamur.ai/free/en/venues/emnlp/) ([EMNLP 2015](https://md.research.elamur.ai/free/en/venues/emnlp/2015/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY-NC-SA 3.0 (ACL Anthology, before 2016)](https://creativecommons.org/licenses/by-nc-sa/3.0/)

**Links:** [Publisher's page](https://aclanthology.org/D15-1213/) · [PDF](https://aclanthology.org/D15-1213.pdf) · [DOI 10.18653/v1/d15-1213](https://doi.org/10.18653/v1/d15-1213) · [OpenAlex](https://openalex.org/W2250397939)

**Citations:** 61 citations (OpenAlex, 2026-10-14) · [42](https://md.research.elamur.ai/free/en/papers/zhang-2015-hierarchical-low-rank-tensors-multilingual/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
Transfer parsing builds a dependency parser for a target language that has little or no annotation by reusing treebanks from other languages. The strongest such systems rely on hand-built feature templates. These templates join standard arc features with typological properties, such as WALS word-order codes. Low-rank tensor models can learn feature conjunctions automatically, but they combine all features freely. That includes meaningless pairings, like two different head POS values together with a verb-subject ordering feature.

Zhang and Barzilay propose a hierarchical low-rank tensor to fix this. It composes atomic features step by step into intermediate embeddings:
- head, modifier and direction/length first;
- then the arc label.

Expert typological features are added at the level whose atomic features they describe. They are summed rather than multiplied, so invalid interactions are excluded by design. Algebraically, the model is a sum of ordinary tensors with shared components, so standard online training still works.

The model was tested on the universal dependency treebank for 10 languages, with labeled parsing included. It outperforms the selective-sharing parser of Täckström et al. (2013), direct transfer and a plain multiway tensor. This holds in both unsupervised and semi-supervised transfer.

## Key points
- With source languages only, the model beats the baselines on both UAS and LAS for 7 of 10 languages. Its average UAS gain is 1.1% over Täckström et al. (2013) and 4.8% over direct transfer (McDonald et al., 2011).
- With 50 annotated target sentences added, the UAS gains rise to 1.7% over NT-Select and 4.5% over direct transfer. Average scores reach 77.9 UAS and 68.5 LAS.
- Compared with a traditional multiway tensor, which gives non-zero weights even to invalid combinations, the hierarchical version improves unsupervised UAS by 0.5% and LAS by 1.3%.
- Against RBGParser trained on full treebanks with partial lexicalization (82.9 UAS / 74.5 LAS), transfer closes about 30% of the gap left by training on 50 sentences alone.
- Decoding runs at about the speed of the multiway tensor (58.6 vs. 61.2 sentences/sec on English). Code is available at github.com/yuanzh/TensorTransfer.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Massachusetts Institute of Technology](https://md.research.elamur.ai/free/en/universities/massachusetts-institute-of-technology/) — University, Cambridge, United States · 211 works · 69,000 citations
- [MIT Computer Science and Artificial Intelligence Laboratory](https://md.research.elamur.ai/free/en/labs/mit-computer-science-and-artificial-intelligence-laboratory/) — Research institute, Cambridge, United States · 69 works · 30,953 citations

📚 Shelves (6): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works · [➗ Mathematics](https://md.research.elamur.ai/free/en/shelves/mathematics/) · 177 works · [➗ Computational Mathematics](https://md.research.elamur.ai/free/en/shelves/mathematics/computational-mathematics/) · 2 works

🧭 Topics (2): [🌍 Multilingual & cross-lingual NLP](https://md.research.elamur.ai/free/en/topics/multilingual-nlp/) · 828 works · [🪶 Efficient models & compression](https://md.research.elamur.ai/free/en/topics/efficient-models/) · 242 works

## Cite

Cite as: Yuan Zhang and Regina Barzilay. 2015. Hierarchical Low-Rank Tensors for Multilingual Transfer Parsing. Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{zhang2015hierarchical,
  title = {{Hierarchical Low-Rank Tensors for Multilingual Transfer Parsing}},
  author = {Yuan Zhang and Regina Barzilay},
  booktitle = {Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing},
  year = {2015},
  doi = {10.18653/v1/d15-1213},
  url = {https://doi.org/10.18653/v1/d15-1213}
}
```

- [← previous Parsing English into Abstract Meaning Representation Using Syntax-Based Machine Translation](https://md.research.elamur.ai/free/en/papers/pust-2015-parsing-english-into-abstract-meaning/)
- [next → Modeling Word Forms Using Latent Underlying Morphs and Phonology](https://md.research.elamur.ai/free/en/papers/cotterell-2015-modeling-word-forms-latent-underlying/)
- [↑ back to papers of 2015](https://md.research.elamur.ai/free/en/papers/2015/page/2/)

## References

31 references, 14 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Taylor Berg-Kirkpatrick and Dan Klein. 2010. Phylogenetic grammar induction. In Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics, pages 1288–1297. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Taylor+Berg%2DKirkpatrick+and+Dan+Klein%2E+2010%2E+Phylogenetic+grammar+induction%2E+In+Proceedings+of+the+48th+Annual+Meeting+of+the+Association+for+Computational+Linguistics%2C+pages+1288%E2%80%931297%2E+Association+fo)
2. Shay B Cohen and Noah A Smith. 2009. [Shared logistic normal distributions for soft parameter tying in unsupervised grammar induction](https://md.research.elamur.ai/free/en/papers/cohen-2009-shared-logistic-normal-distributions-soft/). In Proceedings of the Annual Conference of the North American Chapter of the Association for Computational Linguistics, pages 74–82. Association for Computational Lin…
3. Shay B Cohen, Dipanjan Das, and Noah A Smith. 2011. [Unsupervised structure prediction with non-parallel multilingual guidance](https://md.research.elamur.ai/free/en/papers/cohen-2011-unsupervised-structure-prediction-non-parallel/). In Proceedings of the Conference on Empirical Methods in Natural Language Processing, pages 50–61. Association for Computational Linguistics.
4. Koby Crammer, Ofer Dekel, Joseph Keshet, Shai Shalev-Shwartz, and Yoram Singer. 2006. Online passive-aggressive algorithms. The Journal of Machine Learning Research, 7:551–585. [find](https://scholar.google.com/scholar?q=Koby+Crammer%2C+Ofer+Dekel%2C+Joseph+Keshet%2C+Shai+Shalev%2DShwartz%2C+and+Yoram+Singer%2E+2006%2E+Online+passive%2Daggressive+algorithms%2E+The+Journal+of+Machine+Learning+Research%2C+7%3A551%E2%80%93585%2E)
5. Marie-Catherine De Marneffe and Christopher D Manning. 2008. [The stanford typed dependencies representation](https://md.research.elamur.ai/free/en/papers/marneffe-2008-stanford-typed-dependencies-representation/). In Coling 2008: Proceedings of the workshop on Cross-Framework and Cross-Domain Parser Evaluation, pages 1–8. Association for Computational Linguistics.
6. Matthew S Dryer, David Gil, Bernard Comrie, Hagen Jung, Claudia Schmidt, et al. 2005. The world atlas of language structures. [find](https://scholar.google.com/scholar?q=Matthew+S+Dryer%2C+David+Gil%2C+Bernard+Comrie%2C+Hagen+Jung%2C+Claudia+Schmidt%2C+et+al%2E+2005%2E+The+world+atlas+of+language+structures%2E)
7. Long Duong, Trevor Cohn, Steven Bird, and Paul Cook. 2015. Cross-lingual transfer for unsupervised dependency parsing without parallel data. Proceedings of the SIGNLL Conference on Computational Natural Language Learning, page 113. [find](https://scholar.google.com/scholar?q=Long+Duong%2C+Trevor+Cohn%2C+Steven+Bird%2C+and+Paul+Cook%2E+2015%2E+Cross%2Dlingual+transfer+for+unsupervised+dependency+parsing+without+parallel+data%2E+Proceedings+of+the+SIGNLL+Conference+on+Computational+Natur)
8. Greg Durrett, Adam Pauls, and Dan Klein. 2012. Syntactic transfer using a bilingual lexicon. In Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning, pages 1–11. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Greg+Durrett%2C+Adam+Pauls%2C+and+Dan+Klein%2E+2012%2E+Syntactic+transfer+using+a+bilingual+lexicon%2E+In+Proceedings+of+the+2012+Joint+Conference+on+Empirical+Methods+in+Natural+Language+Processing+and+Computa)
9. Jason M Eisner. 1996. [Three new probabilistic models for dependency parsing: An exploration](https://md.research.elamur.ai/free/en/papers/eisner-1996-three-new-probabilistic-models-dependency/). In Proceedings of the 16th conference on Computational linguistics-Volume 1, pages 340–345. Association for Computational Linguistics.
10. Manaal Faruqui and Chris Dyer. 2014. [Improving vector space word representations using multilingual correlation](https://md.research.elamur.ai/free/en/papers/faruqui-2014-improving-vector-space-word-representations/). In Proceedings of the Annual Conference of the European Chapter of the Association for Computational Linguistics., volume 2014.
11. Daniel Fried, Tamara Polajnar, and Stephen Clark. 2015. Low-rank tensors for verbs in compositional distributional semantics. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Daniel+Fried%2C+Tamara+Polajnar%2C+and+Stephen+Clark%2E+2015%2E+Low%2Drank+tensors+for+verbs+in+compositional+distributional+semantics%2E+In+Proceedings+of+the+53rd+Annual+Meeting+of+the+Association+for+Computati)
12. Rebecca Hwa, Philip Resnik, Amy Weinberg, Clara Cabezas, and Okan Kolak. 2005. Bootstrapping parsers via syntactic projection across parallel texts. Natural language engineering, 11(03):311–325. [find](https://scholar.google.com/scholar?q=Rebecca+Hwa%2C+Philip+Resnik%2C+Amy+Weinberg%2C+Clara+Cabezas%2C+and+Okan+Kolak%2E+2005%2E+Bootstrapping+parsers+via+syntactic+projection+across+parallel+texts%2E+Natural+language+engineering%2C+11%2803%29%3A311%E2%80%93325%2E)
13. Tao Lei, Yu Xin, Yuan Zhang, Regina Barzilay, and Tommi Jaakkola. 2014. Low-rank tensors for scoring dependency structures. In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics, volume 1, pages 1381–1391. [find](https://scholar.google.com/scholar?q=Tao+Lei%2C+Yu+Xin%2C+Yuan+Zhang%2C+Regina+Barzilay%2C+and+Tommi+Jaakkola%2E+2014%2E+Low%2Drank+tensors+for+scoring+dependency+structures%2E+In+Proceedings+of+the+52nd+Annual+Meeting+of+the+Association+for+Computation)
14. Tao Lei, Yuan Zhang, Regina Barzilay, Lluı́s Màrquez, and Alessandro Moschitti. 2015. High-order low-rank tensors for semantic role labeling. In Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Tao+Lei%2C+Yuan+Zhang%2C+Regina+Barzilay%2C+Llu%C4%B1%CC%81s+Ma%CC%80rquez%2C+and+Alessandro+Moschitti%2E+2015%2E+High%2Dorder+low%2Drank+tensors+for+semantic+role+labeling%2E+In+Proceedings+of+the+Conference+of+the+North+American+Ch)
15. Ryan McDonald, Koby Crammer, and Fernando Pereira. 2005. [Online large-margin training of dependency parsers](https://md.research.elamur.ai/free/en/papers/mcdonald-2005-online-large-margin-training-dependency/). In Proceedings of the 43rd Annual Meeting of the Association for Computational Linguistics, pages 91–98. Association for Computational Linguistics.
16. Ryan McDonald, Slav Petrov, and Keith Hall. 2011. [Multi-source transfer of delexicalized dependency parsers](https://md.research.elamur.ai/free/en/papers/mcdonald-2011-multi-source-transfer-delexicalized-dependency/). In Proceedings of the Conference on Empirical Methods in Natural Language Processing, pages 62–72. Association for Computational Linguistics.
17. Ryan T McDonald, Joakim Nivre, Yvonne Quirmbach-Brundage, Yoav Goldberg, Dipanjan Das, Kuzman Ganchev, Keith B Hall, Slav Petrov, Hao Zhang, Oscar Täckström, et al. 2013. [Universal dependency annotation for multilingual parsing](https://md.research.elamur.ai/free/en/papers/mcdonald-2013-universal-dependency-annotation-multilingual-parsing/). In Proceedings of the 51st Annual Meeting of the Association for Comp…
18. Tahira Naseem, Harr Chen, Regina Barzilay, and Mark Johnson. 2010. [Using universal linguistic knowledge to guide grammar induction](https://md.research.elamur.ai/free/en/papers/naseem-2010-universal-linguistic-knowledge-guide-grammar/). In Proceedings of the 2010 Conference on Empirical Methods in Natural Language Processing, pages 1234–1244. Association for Computational Linguistics.
19. Tahira Naseem, Regina Barzilay, and Amir Globerson. 2012. [Selective sharing for multilingual dependency parsing](https://md.research.elamur.ai/free/en/papers/naseem-2012-selective-sharing-multilingual-dependency-parsing/). In Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics: Long Papers-Volume 1, pages 629–637. Association for Computational Linguistics.
20. Slav Petrov, Dipanjan Das, and Ryan McDonald. 2011. [A universal part-of-speech tagset](https://md.research.elamur.ai/free/en/papers/petrov-2012-universal-part-speech-tagset/). arXiv preprint arXiv:1104.2086.
21. Audi Primadhanty, Xavier Carreras, and Ariadna Quattoni. 2015. Low-rank regularization for sparse conjunctive feature spaces: An application to named entity classification. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Audi+Primadhanty%2C+Xavier+Carreras%2C+and+Ariadna+Quattoni%2E+2015%2E+Low%2Drank+regularization+for+sparse+conjunctive+feature+spaces%3A+An+application+to+named+entity+classification%2E+In+Proceedings+of+the+53rd+)
22. Ariadna Quattoni, Borja Balle, Xavier Carreras, and Amir Globerson. 2014. Spectral regularization for max-margin sequence tagging. In Proceedings of the 31st International Conference on Machine Learning (ICML-14), pages 1710–1718. [find](https://scholar.google.com/scholar?q=Ariadna+Quattoni%2C+Borja+Balle%2C+Xavier+Carreras%2C+and+Amir+Globerson%2E+2014%2E+Spectral+regularization+for+max%2Dmargin+sequence+tagging%2E+In+Proceedings+of+the+31st+International+Conference+on+Machine+Learni)
23. Mohammad Sadegh Rasooli and Michael Collins. 2015. Density-driven cross-lingual transfer of dependency parsers. In Proceedings of the Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Mohammad+Sadegh+Rasooli+and+Michael+Collins%2E+2015%2E+Density%2Ddriven+cross%2Dlingual+transfer+of+dependency+parsers%2E+In+Proceedings+of+the+Conference+on+Empirical+Methods+in+Natural+Language+Processing%2E+As)
24. Sameer Singh, Tim Rocktaschel, and Sebastian Riedel. 2015. Towards combined matrix and tensor factorization for universal schema relation extraction. In NAACL Workshop on Vector Space Modeling for NLP. [find](https://scholar.google.com/scholar?q=Sameer+Singh%2C+Tim+Rocktaschel%2C+and+Sebastian+Riedel%2E+2015%2E+Towards+combined+matrix+and+tensor+factorization+for+universal+schema+relation+extraction%2E+In+NAACL+Workshop+on+Vector+Space+Modeling+for+NLP)
25. Richard Socher, Danqi Chen, Christopher D Manning, and Andrew Ng. 2013. [Reasoning with neural tensor networks for knowledge base completion](https://md.research.elamur.ai/free/en/papers/socher-2013-reasoning-neural-tensor-networks-knowledge/). In Proceedings of the Advances in Neural Information Processing Systems, pages 926–934.
26. Vivek Srikumar and Christopher D Manning. 2014. Learning distributed representations for structured output prediction. In Proceedings of the Advances in Neural Information Processing Systems, pages 3266–3274. [find](https://scholar.google.com/scholar?q=Vivek+Srikumar+and+Christopher+D+Manning%2E+2014%2E+Learning+distributed+representations+for+structured+output+prediction%2E+In+Proceedings+of+the+Advances+in+Neural+Information+Processing+Systems%2C+pages+32)
27. Oscar Täckström, Ryan McDonald, and Joakim Nivre. 2013. [Target language adaptation of discriminative transfer parsers](https://md.research.elamur.ai/free/en/papers/tackstrom-2013-target-language-adaptation-discriminative-transfer/). In Proceedings of the Annual Conference of the North American Chapter of the Association for Computational Linguistics.
28. Mo Yu and Mark Dredze. 2015. Learning composition models for phrase embeddings. Transactions of the Association for Computational Linguistics, 3:227– 242. [find](https://scholar.google.com/scholar?q=Mo+Yu+and+Mark+Dredze%2E+2015%2E+Learning+composition+models+for+phrase+embeddings%2E+Transactions+of+the+Association+for+Computational+Linguistics%2C+3%3A227%E2%80%93+242%2E)
29. Dong Yu, Li Deng, and Frank Seide. 2013a. The deep tensor neural network with applications to large vocabulary speech recognition. Audio, Speech, and Language Processing, IEEE Transactions on, 21(2):388–396. [find](https://scholar.google.com/scholar?q=Dong+Yu%2C+Li+Deng%2C+and+Frank+Seide%2E+2013a%2E+The+deep+tensor+neural+network+with+applications+to+large+vocabulary+speech+recognition%2E+Audio%2C+Speech%2C+and+Language+Processing%2C+IEEE+Transactions+on%2C+21%282%29%3A3)
30. Mo Yu, Tiejun Zhao, Yalong Bai, Hao Tian, and Dianhai Yu. 2013b. Cross-lingual projections between languages from different families. In Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics, pages 312–317. [find](https://scholar.google.com/scholar?q=Mo+Yu%2C+Tiejun+Zhao%2C+Yalong+Bai%2C+Hao+Tian%2C+and+Dianhai+Yu%2E+2013b%2E+Cross%2Dlingual+projections+between+languages+from+different+families%2E+In+Proceedings+of+the+51st+Annual+Meeting+of+the+Association+for+C)
31. Daniel Zeman and Philip Resnik. 2008. [Cross-language parser adaptation between related languages](https://md.research.elamur.ai/free/en/papers/zeman-2008-cross-language-parser-adaptation-between/). In Proceedings of the International Joint Conference on Natural Language Processing, pages 35–42.

*Updated 29 October 2026*