--- title: "A Regression Model of Adjective-Noun Compositionality in Distributional Semantics — Guevara 2010" description: "The paper asks how distributional semantic models (DSMs) can build vectors for phrases, focusing on English… 6 references, cited by 76 papers here." authors: ["Emiliano Guevara"] year: 2010 venue: "Proceedings of the 2010 Workshop on GEometrical Models of Natural Language Semantics" source: "https://aclanthology.org/W10-2805/" licence: "CC BY-NC-SA 3.0 (ACL Anthology, before 2016)" licence_url: "https://creativecommons.org/licenses/by-nc-sa/3.0/" summary_licence: "Elamur Cosmic License" canonical: "https://research.elamur.ai/free/en/papers/guevara-2010-regression-model-adjective-noun-compositionality/" --- [Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/guevara-2010-regression-model-adjective-noun-compositionality/) [home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2010](https://md.research.elamur.ai/free/en/papers/2010/) / A Regression Model of Adjective-Noun Compositionality in Distributional Semantics # A Regression Model of Adjective-Noun Compositionality in Distributional Semantics **Authors:** Emiliano Guevara · **Year:** 2010 · **Venue:** Proceedings of the 2010 Workshop on GEometrical Models of Natural Language Semantics · **Licence:** [CC BY-NC-SA 3.0 (ACL Anthology, before 2016)](https://creativecommons.org/licenses/by-nc-sa/3.0/) **Links:** [Publisher's page](https://aclanthology.org/W10-2805/) · [PDF](https://aclanthology.org/W10-2805.pdf) · [OpenAlex](https://openalex.org/W2251471769) **Citations:** 127 citations (OpenAlex, 2026-10-14) · [76](https://md.research.elamur.ai/free/en/papers/guevara-2010-regression-model-adjective-noun-compositionality/cited-by/) in our library, of the 140,884 papers whose bibliographies we read ## Summary The paper asks how distributional semantic models (DSMs) can build vectors for phrases, focusing on English adjective–noun (A-N) bigrams. Standard approaches combine word vectors with a fixed operation, such as addition or component-wise multiplication. Guevara instead argues that composition depends on the syntactic relation involved, so it should be learned from data. A lemmatised, stop-word-filtered British National Corpus is processed so that frequent A-N pairs become single tokens. This yields observed phrase vectors that can serve as regression targets. Partial Least Squares Regression (PLSR), a method suited to many variables and few observations, then learns to map adjective and noun vectors to the phrase vector. It is compared with additive and multiplicative baselines using two distance-based evaluations. PLSR predictions lie clearly closest to the observed phrase vectors. However, when predictions are judged by how well their nearest neighbours match those of the real phrase, the additive model does best and PLSR comes second. The work presents composition as a supervised, relation-specific learning task. ## Key points - **Data and space** - Built with Infomap on the BNC, using a 5L–5R co-occurrence window. - 1,380 A-N pairs with frequency > 400 were extracted via [Det/Art–A–N] regex patterns. - The space covers 40,000 target tokens × 3,000 content-word dimensions, reduced by SVD to 500 latent dimensions. - **Setup** - Models were fitted in R with the `pls` package. - The split was 1,000 training and 380 test pairs, with 10-fold cross-validation during training. - **Euclidean distance to the observed phrase vector** (mean values) - ADD: 1.485 - MUL: 1.002 - PLSR: 0.866 - **Retrieval in a mixed space of observed and predicted vectors** - PLSR's prediction was the single nearest neighbour of the observed pair in 94 of 380 test items. - It appeared in the top 10 in 219 cases (57.6%). - The multiplicative model managed this in only 4 cases, and the additive model never did. - **Neighbour-overlap evaluation** - Under the strict setting (maximum score 3,800), only ADD scored, with 48 points. - Under the relaxed setting (maximum score 7,600), ADD reached 625, PLSR 263 and MUL 37. - Each model's predictions tended to cluster together; the neighbours of multiplicative predictions were all multiplicative. ## Interesting ideas - **Corpus-observed targets for supervision.** Merging frequent phrases into single tokens before building the space gives "real" vectors for those phrases. This turns composition into a supervised problem, and the trick can be reused for any construction type, such as noun compounds or verb–object pairs, in any language with a decent corpus. - **PLSR for wide, small datasets.** When there are many more dimensions than training examples, a latent-component regression like PLSR is a cheap, well-understood alternative to neural models. It could be useful for low-resource settings, where phrase data is scarce. - **Retrieval-based evaluation in a shared space.** Placing predictions and observations in one pool and asking whether the prediction ranks near its target tests usefulness for retrieval, not just raw error. Raw distance alone can hide whether a vector is actually findable. - **Auditing where neighbours come from.** Checking which model the neighbours of each prediction were generated by revealed that every model creates its own self-similar "cluster". Such a diagnostic is worth running whenever synthetic vectors are mixed with real ones, for example in embedding augmentation or OOV handling. - **One composition function per relation.** Learning a separate function for each syntactic relation, rather than one universal operation, is a design choice that could inform phrase representations in morphologically rich languages, where construction types differ strongly. *Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.* Fields: [Natural language processing](https://md.research.elamur.ai/free/en/shelves/nlp/) ## Institutions - [University of Oslo](https://md.research.elamur.ai/free/en/universities/university-of-oslo/) — University, Oslo, Norway · 24 works · 3,197 citations ## Cite Cite as: Emiliano Guevara. 2010. A Regression Model of Adjective-Noun Compositionality in Distributional Semantics. Proceedings of the 2010 Workshop on GEometrical Models of Natural Language Semantics. The paper itself, not this page. BibTeX from the library's record of the work. ```bibtex @inproceedings{guevara2010regression, title = {{A Regression Model of Adjective-Noun Compositionality in Distributional Semantics}}, author = {Emiliano Guevara}, booktitle = {Proceedings of the 2010 Workshop on GEometrical Models of Natural Language Semantics}, year = {2010}, url = {https://aclanthology.org/W10-2805/} } ``` - [← previous For the sake of simplicity: Unsupervised extraction of lexical simplifications from Wikipedia](https://md.research.elamur.ai/free/en/papers/yatskar-2010-sake-simplicity-unsupervised-extraction-lexical/) - [next → BabelNet: Building a Very Large Multilingual Semantic Network](https://md.research.elamur.ai/free/en/papers/navigli-2010-babelnet-building-very-large-multilingual/) - [↑ back to papers of 2010](https://md.research.elamur.ai/free/en/papers/2010/) ## References 6 references, 0 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet. 1. Marco Baroni and Alessandro Lenci. 2009. One semantic memory, many semantic tasks. In Proceedings GEMS 2009, 3–11. Athens: Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Marco+Baroni+and+Alessandro+Lenci%2E+2009%2E+One+semantic+memory%2C+many+semantic+tasks%2E+In+Proceedings+GEMS+2009%2C+3%E2%80%9311%2E+Athens%3A+Association+for+Computational+Linguistics%2E) 2. Eugenie Giesbrecht. 2009. In Search of Semantic Compositionality in Vector Spaces. In Proceedings of the 17th International Conference on Conceptual Structures, ICCS 2009, Moscow, Russia, pp. 173–184. Berlin: Springer. [find](https://scholar.google.com/scholar?q=Eugenie+Giesbrecht%2E+2009%2E+In+Search+of+Semantic+Compositionality+in+Vector+Spaces%2E+In+Proceedings+of+the+17th+International+Conference+on+Conceptual+Structures%2C+ICCS+2009%2C+Moscow%2C+Russia%2C+pp%2E+173%E2%80%93184%2E) 3. Zellig Harris. 1970 [1954]. Distributional structure. In Papers in structural and transformational linguistics, 775–794. Dordrecht: Reidel. [find](https://scholar.google.com/scholar?q=Zellig+Harris%2E+1970+%5B1954%5D%2E+Distributional+structure%2E+In+Papers+in+structural+and+transformational+linguistics%2C+775%E2%80%93794%2E+Dordrecht%3A+Reidel%2E) 4. Bjørn-Helge Mevik and Ron Wehrens. 2007. The pls package: principal component and partial least squares regression in R. Journal of Statistical Software, 18(2): 1–24. [find](https://scholar.google.com/scholar?q=Bj%C3%B8rn%2DHelge+Mevik+and+Ron+Wehrens%2E+2007%2E+The+pls+package%3A+principal+component+and+partial+least+squares+regression+in+R%2E+Journal+of+Statistical+Software%2C+18%282%29%3A+1%E2%80%9324%2E) 5. Jeff Mitchell and Mirella Lapata. 2008. Vector-based Models of Semantic Composition. In Proceedings of ACL-08: HLT, 236–244. Columbus, OH. [find](https://scholar.google.com/scholar?q=Jeff+Mitchell+and+Mirella+Lapata%2E+2008%2E+Vector%2Dbased+Models+of+Semantic+Composition%2E+In+Proceedings+of+ACL%2D08%3A+HLT%2C+236%E2%80%93244%2E+Columbus%2C+OH%2E) 6. Dominic Widdows. 2008. Semantic Vector Products: Some Initial Investigations. Second AAAI Symposium on Quantum Interaction, Oxford, 26th–28th March 2008. URL: [http://www.puttypeg.com/papers/](http://www.puttypeg.com/papers/) Magnus Sahlgren. 2006. The Word Space Model. Ph.D. dissertation, Stockholm University. [↗](http://www.puttypeg.com/papers/)