---
title: "Estimating Linear Models for Compositional Distributional Semantics — Zanzotto 2010"
description: "The paper addresses how to set the parameters of additive compositional distributional semantics (CDS) models… 25 references, cited by 66 papers here."
authors: ["Fabio Massimo Zanzotto", "Ioannis Korkontzelos", "Francesca Fallucchi", "Suresh Manandhar"]
year: 2010
venue: "Proceedings of the 23rd International Conference on Computational Linguistics (Coling 2010)"
source: "https://aclanthology.org/C10-1142/"
licence: "CC BY-NC-SA 3.0 (ACL Anthology, before 2016)"
licence_url: "https://creativecommons.org/licenses/by-nc-sa/3.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/zanzotto-2010-estimating-linear-models-compositional-distributional/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/zanzotto-2010-estimating-linear-models-compositional-distributional/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2010](https://md.research.elamur.ai/free/en/papers/2010/) / Estimating Linear Models for Compositional Distributional Semantics

# Estimating Linear Models for Compositional Distributional Semantics

**Authors:** Fabio Massimo Zanzotto, [Ioannis Korkontzelos](https://md.research.elamur.ai/free/en/authors/ioannis-korkontzelos/), Francesca Fallucchi and [Suresh Manandhar](https://md.research.elamur.ai/free/en/authors/suresh-manandhar/) · **Year:** 2010 · **Venue:** Proceedings of the 23rd International Conference on Computational Linguistics (Coling 2010) · **Licence:** [CC BY-NC-SA 3.0 (ACL Anthology, before 2016)](https://creativecommons.org/licenses/by-nc-sa/3.0/)

**Links:** [Publisher's page](https://aclanthology.org/C10-1142/) · [PDF](https://aclanthology.org/C10-1142.pdf) · [OpenAlex](https://openalex.org/W2100693535)

**Citations:** 121 citations (OpenAlex, 2026-10-14) · [66](https://md.research.elamur.ai/free/en/papers/zanzotto-2010-estimating-linear-models-compositional-distributional/cited-by/) in our library, of the 140,884 papers whose bibliographies we read

## Summary
The paper addresses how to set the parameters of additive compositional distributional semantics (CDS) models, which build a vector for a two-word phrase from the vectors of its words. Earlier work mostly used fixed or hand-tuned scalar weights. The authors generalise the additive model so that each word vector is multiplied by its own full square matrix (z = Ax + By). Learning these matrices is a regression problem with many output variables. They stack training triples into a linear system and solve it by least squares, using the Moore-Penrose pseudo-inverse computed through a truncated SVD. Training triples come from dictionaries: a WordNet headword whose gloss is a two-word phrase supplies the expected phrase vector, and Wiktionary supplies negative examples. Word vectors are built from the British National Corpus. Models are compared by how clearly they separate positive from negative pairs, measured with a t-test p-value. The estimated additive model (EAM) is the only one that reaches significance on both noun-phrase and verb-phrase data. This shows that learning richer composition parameters from data beats tuning simple weights.

## Key points
- **Models compared:** EAM against four baselines:
  - the basic additive model (BAM), with α = β = 1;
  - the basic multiplicative model (BMM);
  - the selectional-preference variants of Erk and Padó, BAM-SP and BMM-SP.
- **Data:** two sets, with WordNet positives and Wiktionary negatives, each split 50/50 into training and testing.
  - NN set: 1065 WordNet pairs and 377 Wiktionary pairs.
  - VN set: 161 WordNet pairs and 111 Wiktionary pairs.
- **Vectors:** derived from British National Corpus text, where each sentence serves as a context window and words act as features, giving 689,191 dimensions.
- **Main result (k = 20):**
  - EAM p-values are about 0.0043 on NN and 0.0045 on VN, both significant.
  - Each SP baseline reaches significance on only one set: BMM-SP on NN, BAM-SP on VN.
- **Tuning BAM is not enough:** no setting of α (with β = 1 − α) beats EAM once k is large. EAM becomes significant from k ≥ 10 on NN and at k = 20 on VN.

## Interesting ideas
- **Dictionary glosses as free supervision:** a word defined by a short phrase gives a natural target vector for composing that phrase. Glossaries and wordnets in other languages could supply training data for composition models in the same way.
- **Composition learned as multi-output linear regression:** fitting [A B] by least squares reduces learning the composition function to one closed-form pseudo-inverse. There is no iterative training, so the approach is easy to reuse for other vector-mapping tasks.
- **SVD rank k as a capacity knob:** truncating the pseudo-inverse to k singular values gives a family of models ranging from very constrained to expressive. This allows a controlled comparison with simpler baselines, since k = 1 has as many free parameters as BAM.
- **Evaluation by separability:** a model is ranked by the t-test p-value between similarity scores of correct and incorrect pairs. This requires no human similarity ratings, only positive and negative sets, so it is cheap to apply in low-resource settings.
- **Testing per syntactic relation:** running separate adjective/noun-noun and verb-noun sets checks whether a method's gains hold across construction types. This is a simple robustness check worth copying.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

Fields: [Natural language processing](https://md.research.elamur.ai/free/en/shelves/nlp/)

## Institutions

- [University of York](https://md.research.elamur.ai/free/en/universities/university-of-york/) — University, York, United Kingdom · 10 works · 6,464 citations
- [University of Rome Tor Vergata](https://md.research.elamur.ai/free/en/universities/university-of-rome-tor-vergata/) — University, Rome, Italy · 5 works · 1,207 citations

## Cite

Cite as: Fabio Massimo Zanzotto, Ioannis Korkontzelos, Francesca Fallucchi and Suresh Manandhar. 2010. Estimating Linear Models for Compositional Distributional Semantics. Proceedings of the 23rd International Conference on Computational Linguistics (Coling 2010).

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{zanzotto2010estimating,
  title = {{Estimating Linear Models for Compositional Distributional Semantics}},
  author = {Fabio Massimo Zanzotto, Ioannis Korkontzelos, Francesca Fallucchi and Suresh Manandhar},
  booktitle = {Proceedings of the 23rd International Conference on Computational Linguistics (Coling 2010)},
  year = {2010},
  url = {https://aclanthology.org/C10-1142/}
}
```

- [← previous Modelling pronominal anaphora in statistical machine translation](https://md.research.elamur.ai/free/en/papers/hardmeier-2010-modelling-pronominal-anaphora-statistical-machine/)
- [next → Learning to Translate with Source and Target Syntax](https://md.research.elamur.ai/free/en/papers/chiang-2010-learning-translate-source-target-syntax/)
- [↑ back to papers of 2010](https://md.research.elamur.ai/free/en/papers/2010/)

## References

25 references, 5 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Charniak, Eugene. 2000. A maximum-entropy-inspired parser. In proceedings of the 1st NAACL, pages 132–139, Seattle, Washington. [our card](https://md.research.elamur.ai/free/en/papers/charniak-2000-maximum-entropy-inspired-parser/)
2. Cook, Paul, Afsaneh Fazly, and Suzanne Stevenson. 2008. The VNC-Tokens Dataset. In proceedings of the LREC Workshop: Towards a Shared Task for Multiword Expressions (MWE 2008), Marrakech, Morocco. [find](https://scholar.google.com/scholar?q=Cook%2C+Paul%2C+Afsaneh+Fazly%2C+and+Suzanne+Stevenson%2E+2008%2E+The+VNC%2DTokens+Dataset%2E+In+proceedings+of+the+LREC+Workshop%3A+Towards+a+Shared+Task+for+Multiword+Expressions+%28MWE+2008%29%2C+Marrakech%2C+Morocco%2E)
3. Deerwester, Scott C., Susan T. Dumais, Thomas K. Landauer, George W. Furnas, and Richard A. Harshman. 1990. Indexing by latent semantic analysis. Journal of the American Society of Information Science, 41(6):391–407. [find](https://scholar.google.com/scholar?q=Deerwester%2C+Scott+C%2E%2C+Susan+T%2E+Dumais%2C+Thomas+K%2E+Landauer%2C+George+W%2E+Furnas%2C+and+Richard+A%2E+Harshman%2E+1990%2E+Indexing+by+latent+semantic+analysis%2E+Journal+of+the+American+Society+of+Information+Science)
4. Erk, Katrin and Sebastian Padó. 2008. A structured vector space model for word meaning in context. In proceedings of the Conference on Empirical Methods in Natural Language Processing, pages 897– 906. Association for Computational Linguistics. [our card](https://md.research.elamur.ai/free/en/papers/erk-2008-structured-vector-space-model-word/)
5. Erk, Katrin. 2007. A simple, similarity-based model for selectional preferences. In proceedings of ACL. Association for Computer Linguistics. [find](https://scholar.google.com/scholar?q=Erk%2C+Katrin%2E+2007%2E+A+simple%2C+similarity%2Dbased+model+for+selectional+preferences%2E+In+proceedings+of+ACL%2E+Association+for+Computer+Linguistics%2E)
6. Fallucchi, Francesca and Fabio Massimo Zanzotto. 2009. SVD feature selection for probabilistic taxonomy learning. In proceedings of the Workshop on Geometrical Models of Natural Language Semantics, pages 66–73. Association for Computational Linguistics, Athens, Greece. [find](https://scholar.google.com/scholar?q=Fallucchi%2C+Francesca+and+Fabio+Massimo+Zanzotto%2E+2009%2E+SVD+feature+selection+for+probabilistic+taxonomy+learning%2E+In+proceedings+of+the+Workshop+on+Geometrical+Models+of+Natural+Language+Semantics%2C+pa)
7. Firth, John R. 1957. Papers in Linguistics. Oxford University Press, London. [find](https://scholar.google.com/scholar?q=Firth%2C+John+R%2E+1957%2E+Papers+in+Linguistics%2E+Oxford+University+Press%2C+London%2E)
8. Golub, Gene and William Kahan. 1965. Calculating the singular values and pseudo-inverse of a matrix. Journal of the Society for Industrial and Applied Mathematics, Series B: Numerical Analysis, 2(2):205–224. [find](https://scholar.google.com/scholar?q=Golub%2C+Gene+and+William+Kahan%2E+1965%2E+Calculating+the+singular+values+and+pseudo%2Dinverse+of+a+matrix%2E+Journal+of+the+Society+for+Industrial+and+Applied+Mathematics%2C+Series+B%3A+Numerical+Analysis%2C+2%282%29%3A2)
9. Harris, Zellig. 1964. Distributional structure. In Katz, Jerrold J. and Jerry A. Fodor, editors, The Philosophy of Linguistics, New York. Oxford University Press. [find](https://scholar.google.com/scholar?q=Harris%2C+Zellig%2E+1964%2E+Distributional+structure%2E+In+Katz%2C+Jerrold+J%2E+and+Jerry+A%2E+Fodor%2C+editors%2C+The+Philosophy+of+Linguistics%2C+New+York%2E+Oxford+University+Press%2E)
10. Jones, Michael N. and Douglas J. K. Mewhort. 2007. Representing word meaning and order information in a composite holographic lexicon. Psychological Review, 114:1–37. [find](https://scholar.google.com/scholar?q=Jones%2C+Michael+N%2E+and+Douglas+J%2E+K%2E+Mewhort%2E+2007%2E+Representing+word+meaning+and+order+information+in+a+composite+holographic+lexicon%2E+Psychological+Review%2C+114%3A1%E2%80%9337%2E)
11. Kim, Su N. and Timothy Baldwin. 2008. Standardised evaluation of english noun compound interpretation. In proceedings of the LREC Workshop: Towards a Shared Task for Multiword Expressions (MWE 2008), pages 39–42, Marrakech, Morocco. [find](https://scholar.google.com/scholar?q=Kim%2C+Su+N%2E+and+Timothy+Baldwin%2E+2008%2E+Standardised+evaluation+of+english+noun+compound+interpretation%2E+In+proceedings+of+the+LREC+Workshop%3A+Towards+a+Shared+Task+for+Multiword+Expressions+%28MWE+2008%29%2C+)
12. Korkontzelos, Ioannis and Suresh Manandhar. 2009. Detecting compositionality in multi-word expressions. In proceedings of ACL-IJCNLP 2009, Singapore. [our card](https://md.research.elamur.ai/free/en/papers/korkontzelos-2009-detecting-compositionality-multi-word-expressions/)
13. Li, Ping, Curt Burgess, and Kevin Lund. 2000. The acquisition of word meaning through global lexical co-occurrences. In proceedings of the 31st Child Language Research Forum. [find](https://scholar.google.com/scholar?q=Li%2C+Ping%2C+Curt+Burgess%2C+and+Kevin+Lund%2E+2000%2E+The+acquisition+of+word+meaning+through+global+lexical+co%2Doccurrences%2E+In+proceedings+of+the+31st+Child+Language+Research+Forum%2E)
14. Lin, Dekang and Patrick Pantel. 2001. DIRT-discovery of inference rules from text. In Proceedings of the ACM Conference on Knowledge Discovery and Data Mining (KDD-01). San Francisco, CA. [find](https://scholar.google.com/scholar?q=Lin%2C+Dekang+and+Patrick+Pantel%2E+2001%2E+DIRT%2Ddiscovery+of+inference+rules+from+text%2E+In+Proceedings+of+the+ACM+Conference+on+Knowledge+Discovery+and+Data+Mining+%28KDD%2D01%29%2E+San+Francisco%2C+CA%2E)
15. Manning, Christopher D., Prabhakar Raghavan, and Hinrich Schütze. 2008. Introduction to Information Retrieval. Cambridge University Press, Cambridge, UK. [find](https://scholar.google.com/scholar?q=Manning%2C+Christopher+D%2E%2C+Prabhakar+Raghavan%2C+and+Hinrich+Schu%CC%88tze%2E+2008%2E+Introduction+to+Information+Retrieval%2E+Cambridge+University+Press%2C+Cambridge%2C+UK%2E)
16. McCarthy, Diana and John Carroll. 2003. Disambiguating nouns, verbs, and adjectives using automatically acquired selectional preferences. Computational Linguistics, 29(4):639–654. [find](https://scholar.google.com/scholar?q=McCarthy%2C+Diana+and+John+Carroll%2E+2003%2E+Disambiguating+nouns%2C+verbs%2C+and+adjectives+using+automatically+acquired+selectional+preferences%2E+Computational+Linguistics%2C+29%284%29%3A639%E2%80%93654%2E)
17. Miller, George A. and Walter G. Charles. 1991. Contextual correlates of semantic similarity. Language and Cognitive Processes, VI:1–28. [find](https://scholar.google.com/scholar?q=Miller%2C+George+A%2E+and+Walter+G%2E+Charles%2E+1991%2E+Contextual+correlates+of+semantic+similarity%2E+Language+and+Cognitive+Processes%2C+VI%3A1%E2%80%9328%2E)
18. Miller, George A. 1995. WordNet: A lexical database for English. Communications of the ACM, 38(11):39–41. [our card](https://md.research.elamur.ai/free/en/papers/miller-1994-wordnet/)
19. Mitchell, Jeff and Mirella Lapata. 2008. Vector-based models of semantic composition. In proceedings of ACL-08: HLT, pages 236–244, Columbus, Ohio. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Mitchell%2C+Jeff+and+Mirella+Lapata%2E+2008%2E+Vector%2Dbased+models+of+semantic+composition%2E+In+proceedings+of+ACL%2D08%3A+HLT%2C+pages+236%E2%80%93244%2C+Columbus%2C+Ohio%2E+Association+for+Computational+Linguistics%2E)
20. Nicholson, Jeremy and Timothy Baldwin. 2008. Interpreting compound nominalisations. In proceedings of the LREC Workshop: Towards a Shared Task for Multiword Expressions (MWE 2008), pages 43–45, Marrakech, Morocco. [find](https://scholar.google.com/scholar?q=Nicholson%2C+Jeremy+and+Timothy+Baldwin%2E+2008%2E+Interpreting+compound+nominalisations%2E+In+proceedings+of+the+LREC+Workshop%3A+Towards+a+Shared+Task+for+Multiword+Expressions+%28MWE+2008%29%2C+pages+43%E2%80%9345%2C+Marrak)
21. Pado, Sebastian and Mirella Lapata. 2007. Dependency-based construction of semantic space models. Computational Linguistics, 33(2):161– 199. [our card](https://md.research.elamur.ai/free/en/papers/pado-2007-dependency-based-construction-semantic-space/)
22. Penrose, Roger. 1955. A generalized inverse for matrices. In Proceedings of Cambridge Philosophical Society. [find](https://scholar.google.com/scholar?q=Penrose%2C+Roger%2E+1955%2E+A+generalized+inverse+for+matrices%2E+In+Proceedings+of+Cambridge+Philosophical+Society%2E)
23. Pollard, Carl J. and Ivan A. Sag. 1994. Head-driven Phrase Structured Grammar. Chicago CSLI, Stanford. [find](https://scholar.google.com/scholar?q=Pollard%2C+Carl+J%2E+and+Ivan+A%2E+Sag%2E+1994%2E+Head%2Ddriven+Phrase+Structured+Grammar%2E+Chicago+CSLI%2C+Stanford%2E)
24. Schone, Patrick and Daniel Jurafsky. 2001. Is knowledge-free induction of multiword unit dictionary headwords a solved problem? In Lee, Lillian and Donna Harman, editors, proceedings of the Conference on Empirical Methods in Natural Language Processing, pages 100–108. [find](https://scholar.google.com/scholar?q=Schone%2C+Patrick+and+Daniel+Jurafsky%2E+2001%2E+Is+knowledge%2Dfree+induction+of+multiword+unit+dictionary+headwords+a+solved+problem%3F+In+Lee%2C+Lillian+and+Donna+Harman%2C+editors%2C+proceedings+of+the+Conference)
25. Villavicencio, Aline. 2003. Verb-particle constructions and lexical resources. In proceedings of the ACL 2003 workshop on Multiword expressions, pages 57–64, Morristown, NJ, USA. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Villavicencio%2C+Aline%2E+2003%2E+Verb%2Dparticle+constructions+and+lexical+resources%2E+In+proceedings+of+the+ACL+2003+workshop+on+Multiword+expressions%2C+pages+57%E2%80%9364%2C+Morristown%2C+NJ%2C+USA%2E+Association+for+Compu)