---
title: "Ontology-Aware Token Embeddings for Prepositional Phrase Attachment — Dasigi 2017"
description: "The paper builds a vector for each token as a weighted sum of WordNet concept vectors. 89.7% accuracy. 27 references, cited by 11 papers here."
authors: ["Pradeep Dasigi", "Waleed Ammar", "Chris Dyer", "Eduard Hovy"]
year: 2017
venue: "Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)"
source: "https://aclanthology.org/P17-1191/"
doi: "10.18653/v1/p17-1191"
arxiv: "1705.02925"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/dasigi-2017-ontology-aware-token-embeddings-prepositional/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/dasigi-2017-ontology-aware-token-embeddings-prepositional/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2017](https://md.research.elamur.ai/free/en/papers/2017/page/3/) / Ontology-Aware Token Embeddings for Prepositional Phrase Attachment

# Ontology-Aware Token Embeddings for Prepositional Phrase Attachment

**Authors:** [Pradeep Dasigi](https://md.research.elamur.ai/free/en/authors/pradeep-dasigi/), [Waleed Ammar](https://md.research.elamur.ai/free/en/authors/waleed-ammar/), [Chris Dyer](https://md.research.elamur.ai/free/en/authors/chris-dyer/) and [Eduard Hovy](https://md.research.elamur.ai/free/en/authors/eduard-hovy/) · **Year:** 2017 · **Venue:** Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) **Link:** [aclanthology.org/P17-1191](https://aclanthology.org/P17-1191/) · **Licence:** [CC BY 4.0](https://creativecommons.org/licenses/by/4.0/)

[**24** citations](https://openalex.org/works?filter=cites:W2612664992) OpenAlex, 2026-10-14 · [**11** citations in our library](https://md.research.elamur.ai/free/en/papers/dasigi-2017-ontology-aware-token-embeddings-prepositional/cited-by/) of the 140,884 papers whose bibliographies we read · [Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)](https://aclanthology.org/volumes/P17-1/) · [licence](https://creativecommons.org/licenses/by/4.0/)

- [Publisher's page](https://aclanthology.org/P17-1191/)
- [PDF](https://aclanthology.org/P17-1191.pdf)
- [DOI 10.18653/v1/p17-1191](https://doi.org/10.18653/v1/p17-1191)
- [arXiv 1705.02925](https://arxiv.org/abs/1705.02925)
- [OpenAlex](https://openalex.org/W2612664992)

**Authors (4)**

1. [Pradeep Dasigi](https://openalex.org/A5029074038) [ORCID](https://orcid.org/0000-0001-7127-1316) · [Semantic Scholar](https://www.semanticscholar.org/author/2697425) · h 20 · 3,171 citations · 90 works
2. [Waleed Ammar](https://openalex.org/A5102948988) [ORCID](https://orcid.org/0000-0003-3541-6981) · [Semantic Scholar](https://www.semanticscholar.org/author/145585097) · h 22 · 3,184 citations · 45 works
3. [Chris Dyer](https://openalex.org/A5111222692) [Semantic Scholar](https://www.semanticscholar.org/author/1745899) · h 73 · 36,419 citations · 271 works
4. [Eduard Hovy](https://openalex.org/A5060225743) [ORCID](https://orcid.org/0000-0002-3270-7903) · [Semantic Scholar](https://www.semanticscholar.org/author/144547315) · h 93 · 43,467 citations · 690 works

Citations and author metrics: OpenAlex, 2026-10-14; authors with the same name are not always told apart.

## What it is about

A type-level embedding gives one vector per word form, so "pool" as a place and "pool" as a game share a vector. The paper builds a vector for each token as a weighted sum of WordNet concept vectors. The weights come from a context-dependent distribution over the word's senses and their hypernyms, with a sense prior that decays with sense rank. The model is a bi-LSTM that picks the head of a prepositional phrase, trained and tested on the English data of Belinkov et al. (test set of 1951 examples). It reaches 89.7% accuracy, against 84.3% for plain GloVe vectors and 88.7% for the earlier HPCD system; the abstract reports a gain of 5.4 absolute points, or 34.4% fewer errors. Removing the sense priors gives 88.4% and removing attention gives 87.5%.

## Ideas

- I3. Ablation first: remove the sense-rank prior and the context weight in turn, as the paper does, and measure each drop on EWT before adding more.
- I4. Training-size curve: train the scorer on 2000, 5000 and all EWT sentences and report the gap with and without the hypernym feature. The paper finds the gap is largest on small training sets.

## For Mova

- I1. `onto/src/wsd.rs`: the sense prior is a rank-based weight, as in the paper. Use the same weight for the PP head candidates: score each candidate by the hypernym overlap between the head and the PP object, using the first senses and at most 5 hypernym levels (the paper's caps). Measure accuracy on EWT PP cases with and without the feature, in two identical runs.
- I2. `en/src/parse.rs`: add a deterministic feature for shared hypernyms of the head and the object of the preposition, drawn from OEWN, to the perceptron that chooses the PP head. Measure LAS on EWT test with `en parse-eval` and the change on PP arcs alone.

The chapters are [languages/en/22-phrases.md](https://mova.elamur.ai/free/en/docs/languages/en/22-phrases/) and `23-complementation.md`. The code is `onto/src/wsd.rs` for the sense weights and `en/src/parse.rs` for the PP choice. The paper uses WordNet 3.1 and an LSTM, so no number carries over: any figure for Mova must come from EWT test, with OEWN and the perceptron. The Belinkov test set is WSJ text from the LDC-licensed Penn Treebank, so it is for tests only.

## Institutions

[🎓**Carnegie Mellon University** Pittsburgh · United States 278 works · 138,810 citations](https://md.research.elamur.ai/free/en/universities/carnegie-mellon-university/) · [🏢**Google DeepMind** London · United Kingdom 29 works · 74,823 citations](https://md.research.elamur.ai/free/en/companies/google-deepmind/) · [🔬**Allen Institute for Artificial Intelligence** Seattle · United States 169 works · 65,355 citations](https://md.research.elamur.ai/free/en/labs/allen-institute-for-artificial-intelligence/)

Topics[Knowledge graphs & ontologies](https://md.research.elamur.ai/free/en/topics/knowledge-graphs/)

## Cite

Cite as Pradeep Dasigi, Waleed Ammar, Chris Dyer and Eduard Hovy. 2017. Ontology-Aware Token Embeddings for Prepositional Phrase Attachment. Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this card. BibTeX from the library's record of the work. [DOI 10.18653/v1/p17-1191](https://doi.org/10.18653/v1/p17-1191) · [arXiv 1705.02925](https://arxiv.org/abs/1705.02925)

```bibtex
@inproceedings{dasigi2017ontology,
  title = {{Ontology-Aware Token Embeddings for Prepositional Phrase Attachment}},
  author = {Pradeep Dasigi, Waleed Ammar, Chris Dyer and Eduard Hovy},
  booktitle = {Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2017},
  doi = {10.18653/v1/p17-1191},
  eprint = {1705.02925},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.18653/v1/p17-1191}
}
```

- [← previous No case for Case in locality: Case does not help interpretation when intervention blocks A-bar chains](https://md.research.elamur.ai/free/en/papers/friedmann-2017-no-case-case-locality-case/)
- [next → Bias in polar questions: Evidence from English and German production experiments](https://md.research.elamur.ai/free/en/papers/domaneschi-2017-bias-polar-questions-evidence-english/)
- [↑ back to papers of 2017](https://md.research.elamur.ai/free/en/papers/2017/page/3/)

## References

27 references, 14 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Eneko Agirre. 2008. Improving parsing and pp attachment performance with sense information. In ACL. Citeseer. [find](https://scholar.google.com/scholar?q=Eneko+Agirre%2E+2008%2E+Improving+parsing+and+pp+attachment+performance+with+sense+information%2E+In+ACL%2E+Citeseer%2E)
2. Sanjeev Arora, Yuanzhi Li, Yingyu Liang, Tengyu Ma, and Andrej Risteski. 2016. Linear algebraic structure of word senses, with applications to polysemy. arXiv preprint arXiv:1601.03764 . [↗](https://arxiv.org/abs/1601.03764) [our card](https://md.research.elamur.ai/free/en/papers/arora-2018-linear-algebraic-structure-word-senses/)
3. Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. 2014. Neural machine translation by jointly learning to align and translate. CoRR abs/1409.0473. [↗](https://arxiv.org/abs/1409.0473) [our card](https://md.research.elamur.ai/free/en/papers/bahdanau-2014-neural-machine-translation-jointly-learning/)
4. David Belanger and Sham M. Kakade. 2015. A linear dynamical system model for text. In ICML. [find](https://scholar.google.com/scholar?q=David+Belanger+and+Sham+M%2E+Kakade%2E+2015%2E+A+linear+dynamical+system+model+for+text%2E+In+ICML%2E)
5. Yonatan Belinkov, Tao Lei, Regina Barzilay, and Amir Globerson. 2014. Exploring compositional architectures and word vector representations for prepositional phrase attachment. Transactions of the Association for Computational Linguistics 2:561–572. [our card](https://md.research.elamur.ai/free/en/papers/belinkov-2014-exploring-compositional-architectures-word-vector/)
6. Eric Brill and Philip Resnik. 1994. A rule-based approach to prepositional phrase attachment disambiguation. In Proceedings of the 15th conference on Computational linguistics-Volume 2. Association for Computational Linguistics, pages 1198–1204. [find](https://scholar.google.com/scholar?q=Eric+Brill+and+Philip+Resnik%2E+1994%2E+A+rule%2Dbased+approach+to+prepositional+phrase+attachment+disambiguation%2E+In+Proceedings+of+the+15th+conference+on+Computational+linguistics%2DVolume+2%2E+Association+fo)
7. Danqi Chen and Christopher D Manning. 2014. A fast and accurate dependency parser using neural networks. In EMNLP. pages 740–750. [our card](https://md.research.elamur.ai/free/en/papers/chen-2014-fast-accurate-dependency-parser-neural/)
8. Xinxiong Chen, Zhiyuan Liu, and Maosong Sun. 2014. A unified model for word sense representation and disambiguation. In EMNLP. pages 1025–1035. [our card](https://md.research.elamur.ai/free/en/papers/chen-2014-unified-model-word-sense-representation/)
9. François Chollet. 2015. Keras. [https://github.com/fchollet/keras](https://github.com/fchollet/keras). [↗](https://github.com/fchollet/keras)
10. Manaal Faruqui, Jesse Dodge, Sujay Kumar Jauhar, Chris Dyer, Eduard H. Hovy, and Noah A. Smith. 2015. Retrofitting word vectors to semantic lexicons. In NAACL. [find](https://scholar.google.com/scholar?q=Manaal+Faruqui%2C+Jesse+Dodge%2C+Sujay+Kumar+Jauhar%2C+Chris+Dyer%2C+Eduard+H%2E+Hovy%2C+and+Noah+A%2E+Smith%2E+2015%2E+Retrofitting+word+vectors+to+semantic+lexicons%2E+In+NAACL%2E)
11. Eric H Huang, Richard Socher, Christopher D Manning, and Andrew Y Ng. 2012. Improving word representations via global context and multiple word prototypes. In Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics: Long Papers-Volume 1. Association for Computational Linguistics, pages 873–882. [our card](https://md.research.elamur.ai/free/en/papers/huang-2012-improving-word-representations-global-context/)
12. Sujay Kumar Jauhar, Chris Dyer, and Eduard H. Hovy. 2015. Ontologically grounded multi-sense representation learning for semantic vector space models. In NAACL. [find](https://scholar.google.com/scholar?q=Sujay+Kumar+Jauhar%2C+Chris+Dyer%2C+and+Eduard+H%2E+Hovy%2E+2015%2E+Ontologically+grounded+multi%2Dsense+representation+learning+for+semantic+vector+space+models%2E+In+NAACL%2E)
13. Richard Johansson and Luis Nieto Pina. 2015. Embedding a semantic network in a word space. In In Proceedings of the 2015 Conference of the North American Chapter of the Association for Computational Linguistics–Human Language Technologies. Citeseer. [find](https://scholar.google.com/scholar?q=Richard+Johansson+and+Luis+Nieto+Pina%2E+2015%2E+Embedding+a+semantic+network+in+a+word+space%2E+In+In+Proceedings+of+the+2015+Conference+of+the+North+American+Chapter+of+the+Association+for+Computational+L)
14. Karin Kipper, Anna Korhonen, Neville Ryant, and Martha Palmer. 2008. A large-scale classification of english verbs. Language Resources and Evaluation 42(1):21–40. [↗](https://doi.org/10.1007/s10579-007-9048-2)
15. Guillaume Lample, Miguel Ballesteros, Sandeep Subramanian, Kazuya Kawakami, and Chris Dyer. 2016. Neural architectures for named entity recognition. In NAACL. [our card](https://md.research.elamur.ai/free/en/papers/lample-2016-neural-architectures-named-entity-recognition/)
16. Tao Lei, Yuan Zhang, Regina Barzilay, and Tommi Jaakkola. 2014. Low-rank tensors for scoring dependency structures. In ACL. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Tao+Lei%2C+Yuan+Zhang%2C+Regina+Barzilay%2C+and+Tommi+Jaakkola%2E+2014%2E+Low%2Drank+tensors+for+scoring+dependency+structures%2E+In+ACL%2E+Association+for+Computational+Linguistics%2E)
17. Tomas Mikolov, Ilya Sutskever, Kai Chen, Gregory S. Corrado, and Jeffrey Dean. 2013. Distributed representations of words and phrases and their compositionality. CoRR abs/1310.4546. [↗](https://arxiv.org/abs/1310.4546) [our card](https://md.research.elamur.ai/free/en/papers/mikolov-2013-distributed-representations-words-phrases-their/)
18. George A Miller. 1995. Wordnet: a lexical database for english. Communications of the ACM 38(11):39– 41. [↗](https://doi.org/10.1145/219717.219748) [our card](https://md.research.elamur.ai/free/en/papers/miller-1994-wordnet/)
19. Arvind Neelakantan, Jeevan Shankar, Alexandre Passos, and Andrew McCallum. 2015. Efficient non-parametric estimation of multiple embeddings per word in vector space. arXiv preprint arXiv:1504.06654 . [↗](https://arxiv.org/abs/1504.06654) [our card](https://md.research.elamur.ai/free/en/papers/neelakantan-2014-efficient-non-parametric-estimation-multiple/)
20. Jeffrey Pennington, Richard Socher, and Christopher D Manning. 2014. Glove: Global vectors for word representation. In EMNLP. [our card](https://md.research.elamur.ai/free/en/papers/pennington-2014-glove-global-vectors-word-representation/)
21. Adwait Ratnaparkhi, Jeff Reynar, and Salim Roukos. 1994. A maximum entropy model for prepositional phrase attachment. In Proceedings of the workshop on Human Language Technology. [our card](https://md.research.elamur.ai/free/en/papers/ratnaparkhi-1994-maximum-entropy-model-prepositional-phrase/)
22. Joseph Reisinger and Raymond J Mooney. 2010. Multi-prototype vector-space models of word meaning. In HLT-ACL. [our card](https://md.research.elamur.ai/free/en/papers/reisinger-2010-multi-prototype-vector-space-models/)
23. Philip Resnik. 1993. Semantic classes and syntactic ambiguity. In Proceedings of the workshop on Human Language Technology. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Philip+Resnik%2E+1993%2E+Semantic+classes+and+syntactic+ambiguity%2E+In+Proceedings+of+the+workshop+on+Human+Language+Technology%2E+Association+for+Computational+Linguistics%2E)
24. Sascha Rothe and Hinrich Schütze. 2015. Autoextend: Extending word embeddings to embeddings for synsets and lexemes. In ACL. [find](https://scholar.google.com/scholar?q=Sascha+Rothe+and+Hinrich+Schu%CC%88tze%2E+2015%2E+Autoextend%3A+Extending+word+embeddings+to+embeddings+for+synsets+and+lexemes%2E+In+ACL%2E)
25. Luke Vilnis and Andrew McCallum. 2015. Word representations via gaussian embedding. In ICLR. [find](https://scholar.google.com/scholar?q=Luke+Vilnis+and+Andrew+McCallum%2E+2015%2E+Word+representations+via+gaussian+embedding%2E+In+ICLR%2E)
26. Mo Yu and Mark Dredze. 2014. Improving lexical embeddings with semantic knowledge. In ACL. [our card](https://md.research.elamur.ai/free/en/papers/yu-2014-improving-lexical-embeddings-semantic-knowledge/)
27. Benat Zapirain, Eneko Agirre, Lluis Marquez, and Mihai Surdeanu. 2013. Selectional preferences for semantic role classification. Computational Linguistics . [find](https://scholar.google.com/scholar?q=Benat+Zapirain%2C+Eneko+Agirre%2C+Lluis+Marquez%2C+and+Mihai+Surdeanu%2E+2013%2E+Selectional+preferences+for+semantic+role+classification%2E+Computational+Linguistics+%2E)