---
title: "A Neural Network Approach to Selectional Preference Acquisition — Tim 2014"
description: "The author proposes an unsupervised feed-forward neural network that learns embeddings for predicates and… 30 references, cited by 29 papers here."
authors: ["Van de Cruys, Tim"]
year: 2014
venue: "Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP)"
source: "https://aclanthology.org/D14-1004/"
doi: "10.3115/v1/d14-1004"
licence: "CC BY-NC-SA 3.0 (ACL Anthology, before 2016)"
licence_url: "https://creativecommons.org/licenses/by-nc-sa/3.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/cruys-2014-neural-network-approach-selectional-preference/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/cruys-2014-neural-network-approach-selectional-preference/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2014](https://md.research.elamur.ai/free/en/papers/2014/page/2/) / A Neural Network Approach to Selectional Preference Acquisition

# A Neural Network Approach to Selectional Preference Acquisition

**Authors:** Van de Cruys, Tim · **Year:** 2014 · **Venue:** Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · **Licence:** [CC BY-NC-SA 3.0 (ACL Anthology, before 2016)](https://creativecommons.org/licenses/by-nc-sa/3.0/)

**Links:** [Publisher's page](https://aclanthology.org/D14-1004/) · [PDF](https://aclanthology.org/D14-1004.pdf) · [DOI 10.3115/v1/d14-1004](https://doi.org/10.3115/v1/d14-1004) · [OpenAlex](https://openalex.org/W2128050560)

**Citations:** 50 citations (OpenAlex, 2026-10-14) · [29](https://md.research.elamur.ai/free/en/papers/cruys-2014-neural-network-approach-selectional-preference/cited-by/) in our library, of the 140,884 papers whose bibliographies we read

## Summary
This EMNLP 2014 paper by Tim Van de Cruys addresses the automatic acquisition of selectional preferences, which is the tendency of predicates to favour semantically fitting arguments. A verb like "sing" fits a vocalist singing a ballad but not a tomato being sung. Corpus counts are sparse because of Zipf's law, so a model must generalize to predicate–argument combinations it never observed.

The author proposes an unsupervised feed-forward neural network that learns embeddings for predicates and arguments directly from unannotated text. Attested tuples serve as positive examples, and randomly corrupted tuples serve as negatives. Training uses a margin-based ranking loss adapted from Collobert and Weston (2008). There are two variants:
- a two-way model for a verb and one argument;
- a three-way model that scores a verb together with its subject and direct object, using a newly modified training objective.

On a pseudo-disambiguation task, both variants reach state-of-the-art results. They are compared against EM-based latent clustering, distributional similarity, and tensor factorization. This makes the paper an early demonstration that neural embeddings suit selectional preference modelling.

## Key points
- **Architecture:** predicate and argument vectors are concatenated, passed through one tanh hidden layer, and mapped to a single preference score.
- **Training:** the score of an attested tuple must exceed that of a corrupted tuple by a margin of 1. Negatives are created by swapping in random words.
- **Baselines:** Rooth et al. (1999) EM latent-variable clustering, Erk (2007) distributional similarity, and an earlier tensor factorization approach by the same author (Van de Cruys, 2009) that handles preferences involving several arguments at once.
- **Separate argument slots:** the three-way model keeps distinct embedding matrices for subjects (S) and objects (O), so a word's meaning can depend on its position. For example, "virus" as a subject resembles "animal", while as an object it resembles "cell" or "device".
- **Future work:** initializing the embeddings from large neural language models (Mikolov et al., 2013), sharing representations across argument slots, and trying neural tensor networks.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

Fields: [AI & machine learning](https://md.research.elamur.ai/free/en/shelves/ai/)

## Cite

Cite as: Van de Cruys, Tim. 2014. A Neural Network Approach to Selectional Preference Acquisition. Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP).

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{tim2014neural,
  title = {{A Neural Network Approach to Selectional Preference Acquisition}},
  author = {Van {de Cruys, Tim}},
  booktitle = {Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP)},
  year = {2014},
  doi = {10.3115/v1/d14-1004},
  url = {https://doi.org/10.3115/v1/d14-1004}
}
```

- [← previous A New Entity Salience Task with Millions of Training Examples](https://md.research.elamur.ai/free/en/papers/dunietz-2014-new-entity-salience-task-millions/)
- [next → ParCor 1.0: A Parallel Pronoun-Coreference Corpus to Support Statistical MT](https://md.research.elamur.ai/free/en/papers/guillou-2014-parcor-1-0-parallel-pronoun/)
- [↑ back to papers of 2014](https://md.research.elamur.ai/free/en/papers/2014/page/2/)

## References

30 references, 8 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Marco Baroni, Silvia Bernardini, Adriano Ferraresi, and Eros Zanchetta. 2009. The wacky wide web: A collection of very large linguistically processed web-crawled corpora. Language Resources and Evaluation, 43(3):209–226. [find](https://scholar.google.com/scholar?q=Marco+Baroni%2C+Silvia+Bernardini%2C+Adriano+Ferraresi%2C+and+Eros+Zanchetta%2E+2009%2E+The+wacky+wide+web%3A+A+collection+of+very+large+linguistically+processed+web%2Dcrawled+corpora%2E+Language+Resources+and+Evalua)
2. Yoshua Bengio, Réjean Ducharme, Pascal Vincent, and Christian Jauvin. 2003. A neural probabilistic language model. Journal of Machine Learning Research, 3:1137–1155. [our card](https://md.research.elamur.ai/free/en/papers/bengio-2003-neural-probabilistic-language-model/)
3. Shane Bergsma, Dekang Lin, and Randy Goebel. 2008. Discriminative learning of selectional preference from unlabeled text. In Proceedings of the Conference on Empirical Methods in Natural Language Processing, pages 59–68. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Shane+Bergsma%2C+Dekang+Lin%2C+and+Randy+Goebel%2E+2008%2E+Discriminative+learning+of+selectional+preference+from+unlabeled+text%2E+In+Proceedings+of+the+Conference+on+Empirical+Methods+in+Natural+Language+Proc)
4. Kenneth Ward Church and Patrick Hanks. 1990. Word association norms, mutual information & lexicography. Computational Linguistics, 16(1):22–29. [find](https://scholar.google.com/scholar?q=Kenneth+Ward+Church+and+Patrick+Hanks%2E+1990%2E+Word+association+norms%2C+mutual+information+%26+lexicography%2E+Computational+Linguistics%2C+16%281%29%3A22%E2%80%9329%2E)
5. Stephen Clark and David Weir. 2001. Class-based probability estimation using a semantic hierarchy. In Proceedings of the second meeting of the North American Chapter of the Association for Computational Linguistics on Language technologies, pages 95–102. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Stephen+Clark+and+David+Weir%2E+2001%2E+Class%2Dbased+probability+estimation+using+a+semantic+hierarchy%2E+In+Proceedings+of+the+second+meeting+of+the+North+American+Chapter+of+the+Association+for+Computation)
6. Ronan Collobert and Jason Weston. 2008. A unified architecture for natural language processing: Deep neural networks with multitask learning. In Proceedings of the 25th international conference on Machine learning, pages 160–167. ACM. [find](https://scholar.google.com/scholar?q=Ronan+Collobert+and+Jason+Weston%2E+2008%2E+A+unified+architecture+for+natural+language+processing%3A+Deep+neural+networks+with+multitask+learning%2E+In+Proceedings+of+the+25th+international+conference+on+Mac)
7. Katrin Erk, Sebastian Padó, and Ulrike Padó. 2010. A flexible, corpus-driven model of regular and inverse selectional preferences. Computational Linguistics, 36(4):723–763. [find](https://scholar.google.com/scholar?q=Katrin+Erk%2C+Sebastian+Pado%CC%81%2C+and+Ulrike+Pado%CC%81%2E+2010%2E+A+flexible%2C+corpus%2Ddriven+model+of+regular+and+inverse+selectional+preferences%2E+Computational+Linguistics%2C+36%284%29%3A723%E2%80%93763%2E)
8. Katrin Erk. 2007. A simple, similarity-based model for selectional preferences. In Proceedings of the 45th Annual Meeting of the Association of Computational Linguistics, pages 216–223, Prague, Czech Republic, June. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Katrin+Erk%2E+2007%2E+A+simple%2C+similarity%2Dbased+model+for+selectional+preferences%2E+In+Proceedings+of+the+45th+Annual+Meeting+of+the+Association+of+Computational+Linguistics%2C+pages+216%E2%80%93223%2C+Prague%2C+Czech+)
9. Daniel Gildea and Daniel Jurafsky. 2002. Automatic labeling of semantic roles. Computational linguistics, 28(3):245–288. [our card](https://md.research.elamur.ai/free/en/papers/gildea-2002-automatic-labeling-semantic-roles/)
10. Eric H. Huang, Richard Socher, Christopher D. Manning, and Andrew Y. Ng. 2012. Improving word representations via global context and multiple word prototypes. In Annual Meeting of the Association for Computational Linguistics (ACL). [our card](https://md.research.elamur.ai/free/en/papers/huang-2012-improving-word-representations-global-context/)
11. Daniel D. Lee and H. Sebastian Seung. 2000. Algorithms for non-negative matrix factorization. In Advances in Neural Information Processing Systems 13, pages 556–562. [find](https://scholar.google.com/scholar?q=Daniel+D%2E+Lee+and+H%2E+Sebastian+Seung%2E+2000%2E+Algorithms+for+non%2Dnegative+matrix+factorization%2E+In+Advances+in+Neural+Information+Processing+Systems+13%2C+pages+556%E2%80%93562%2E)
12. Hang Li and Naoki Abe. 1998. Generalizing case frames using a thesaurus and the MDL principle. Computational linguistics, 24(2):217–244. [our card](https://md.research.elamur.ai/free/en/papers/li-1998-generalizing-case-frames-thesaurus-mdl/)
13. Tao Li and Chris Ding. 2006. The relationships among various nonnegative matrix factorization methods for clustering. In Data Mining, 2006. ICDM’06. Sixth International Conference on, pages 362–371. IEEE. [find](https://scholar.google.com/scholar?q=Tao+Li+and+Chris+Ding%2E+2006%2E+The+relationships+among+various+nonnegative+matrix+factorization+methods+for+clustering%2E+In+Data+Mining%2C+2006%2E+ICDM%E2%80%9906%2E+Sixth+International+Conference+on%2C+pages+362%E2%80%93371%2E+I)
14. Dong C. Liu and Jorge Nocedal. 1989. On the limited memory BFGS method for large scale optimization. Mathematical programming, 45(1-3):503–528. [find](https://scholar.google.com/scholar?q=Dong+C%2E+Liu+and+Jorge+Nocedal%2E+1989%2E+On+the+limited+memory+BFGS+method+for+large+scale+optimization%2E+Mathematical+programming%2C+45%281%2D3%29%3A503%E2%80%93528%2E)
15. Diana McCarthy and John Carroll. 2003. Disambiguating nouns, verbs, and adjectives using automatically acquired selectional preferences. Computational Linguistics, 29(4):639–654. [find](https://scholar.google.com/scholar?q=Diana+McCarthy+and+John+Carroll%2E+2003%2E+Disambiguating+nouns%2C+verbs%2C+and+adjectives+using+automatically+acquired+selectional+preferences%2E+Computational+Linguistics%2C+29%284%29%3A639%E2%80%93654%2E)
16. Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean. 2013. Efficient estimation of word representations in vector space. In ICLR 2013. [our card](https://md.research.elamur.ai/free/en/papers/mikolov-2013-efficient-estimation-word-representations-vector/)
17. Andriy Mnih and Geoffrey Hinton. 2007. Three new graphical models for statistical language modelling. In Proceedings of the 24th international conference on Machine learning, pages 641–648. ACM. [find](https://scholar.google.com/scholar?q=Andriy+Mnih+and+Geoffrey+Hinton%2E+2007%2E+Three+new+graphical+models+for+statistical+language+modelling%2E+In+Proceedings+of+the+24th+international+conference+on+Machine+learning%2C+pages+641%E2%80%93648%2E+ACM%2E)
18. Joakim Nivre, Johan Hall, and Jens Nilsson. 2006. Maltparser: A data-driven parser-generator for dependency parsing. In Proceedings of LREC-2006, pages 2216–2219. [our card](https://md.research.elamur.ai/free/en/papers/nivre-2006-maltparser-data-driven-parser-generator/)
19. Diarmuid Ó Séaghdha and Anna Korhonen. 2012. Modelling selectional preferences in a lexical hierarchy. In Proceedings of the First Joint Conference on Lexical and Computational Semantics-Volume 1: Proceedings of the main conference and the shared task, and Volume 2: Proceedings of the Sixth Intern… [find](https://scholar.google.com/scholar?q=Diarmuid+O%CC%81+Se%CC%81aghdha+and+Anna+Korhonen%2E+2012%2E+Modelling+selectional+preferences+in+a+lexical+hierarchy%2E+In+Proceedings+of+the+First+Joint+Conference+on+Lexical+and+Computational+Semantics%2DVolume+1%3A+P)
20. Diarmuid Ó Séaghdha. 2010. Latent variable models of selectional preference. In Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics, pages 435–444. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Diarmuid+O%CC%81+Se%CC%81aghdha%2E+2010%2E+Latent+variable+models+of+selectional+preference%2E+In+Proceedings+of+the+48th+Annual+Meeting+of+the+Association+for+Computational+Linguistics%2C+pages+435%E2%80%93444%2E+Association+fo)
21. Sebastian Padó, Ulrike Padó, and Katrin Erk. 2007. Flexible, corpus-based modelling of human plausibility judgements. In Proceedings of the 2007 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning (EMNLP-CoNLL), pages 400–409, Prague, Cz… [find](https://scholar.google.com/scholar?q=Sebastian+Pado%CC%81%2C+Ulrike+Pado%CC%81%2C+and+Katrin+Erk%2E+2007%2E+Flexible%2C+corpus%2Dbased+modelling+of+human+plausibility+judgements%2E+In+Proceedings+of+the+2007+Joint+Conference+on+Empirical+Methods+in+Natural+Lang)
22. Philip Resnik. 1996. Selectional constraints: An information-theoretic model and its computational realization. Cognition, 61:127–159, November. [find](https://scholar.google.com/scholar?q=Philip+Resnik%2E+1996%2E+Selectional+constraints%3A+An+information%2Dtheoretic+model+and+its+computational+realization%2E+Cognition%2C+61%3A127%E2%80%93159%2C+November%2E)
23. Alan Ritter, Mausam, and Oren Etzioni. 2010. A latent dirichlet allocation method for selectional preferences. In Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics, pages 424–434, Uppsala, Sweden, July. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Alan+Ritter%2C+Mausam%2C+and+Oren+Etzioni%2E+2010%2E+A+latent+dirichlet+allocation+method+for+selectional+preferences%2E+In+Proceedings+of+the+48th+Annual+Meeting+of+the+Association+for+Computational+Linguistic)
24. Mats Rooth, Stefan Riezler, Detlef Prescher, Glenn Carroll, and Franz Beil. 1999. Inducing a semantically annotated lexicon via em-based clustering. In Proceedings of the 37th annual meeting of the Association for Computational Linguistics on Computational Linguistics, pages 104–111. Association for… [find](https://scholar.google.com/scholar?q=Mats+Rooth%2C+Stefan+Riezler%2C+Detlef+Prescher%2C+Glenn+Carroll%2C+and+Franz+Beil%2E+1999%2E+Inducing+a+semantically+annotated+lexicon+via+em%2Dbased+clustering%2E+In+Proceedings+of+the+37th+annual+meeting+of+the+As)
25. Ekaterina Shutova, Simone Teufel, and Anna Korhonen. 2013. Statistical metaphor processing. Computational Linguistics, 39(2):301–353. [our card](https://md.research.elamur.ai/free/en/papers/shutova-2013-statistical-metaphor-processing/)
26. Kristina Toutanova, Dan Klein, Christopher Manning, and Yoram Singer. 2003. Feature-rich part-of-speech tagging with a cyclic dependency network. In Proceedings of HLT-NAACL 2003, pages 252– 259. [our card](https://md.research.elamur.ai/free/en/papers/toutanova-2003-feature-rich-part-speech-tagging/)
27. Masashi Tsubaki, Kevin Duh, Masashi Shimbo, and Yuji Matsumoto. 2013. Modeling and learning semantic co-compositionality through prototype projections and neural networks. In Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, pages 130–140, Seattle, Washington, U… [find](https://scholar.google.com/scholar?q=Masashi+Tsubaki%2C+Kevin+Duh%2C+Masashi+Shimbo%2C+and+Yuji+Matsumoto%2E+2013%2E+Modeling+and+learning+semantic+co%2Dcompositionality+through+prototype+projections+and+neural+networks%2E+In+Proceedings+of+the+2013+C)
28. Tim Van de Cruys. 2009. A non-negative tensor factorization model for selectional preference induction. [find](https://scholar.google.com/scholar?q=Tim+Van+de+Cruys%2E+2009%2E+A+non%2Dnegative+tensor+factorization+model+for+selectional+preference+induction%2E)
29. In Proceedings of the Workshop on Geometrical Models of Natural Language Semantics, pages 83– 90, Athens, Greece, March. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=In+Proceedings+of+the+Workshop+on+Geometrical+Models+of+Natural+Language+Semantics%2C+pages+83%E2%80%93+90%2C+Athens%2C+Greece%2C+March%2E+Association+for+Computational+Linguistics%2E)
30. Dong Yu, Li Deng, and Frank Seide. 2013. The deep tensor neural network with applications to large vocabulary speech recognition. IEEE Transactions on Audio, Speech, and Language Processing, 21(2):388–396. [find](https://scholar.google.com/scholar?q=Dong+Yu%2C+Li+Deng%2C+and+Frank+Seide%2E+2013%2E+The+deep+tensor+neural+network+with+applications+to+large+vocabulary+speech+recognition%2E+IEEE+Transactions+on+Audio%2C+Speech%2C+and+Language+Processing%2C+21%282%29%3A388)