---
title: "The Distributional Inclusion Hypotheses and — Geffet 2005"
description: "Classic distributional similarity tends to group words that are only loosely related (e.g., company–government)."
authors: ["Maayan Geffet", "Ido Dagan"]
year: 2005
venue: "Proceedings of the 43rd Annual Meeting of the Association for Computational Linguistics (ACL'05)"
source: "https://aclanthology.org/P05-1014/"
doi: "10.3115/1219840.1219854"
licence: "CC BY-NC-SA 3.0 (ACL Anthology, before 2016)"
licence_url: "https://creativecommons.org/licenses/by-nc-sa/3.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/geffet-2005-distributional-inclusion-hypotheses-lexical-entailment/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/geffet-2005-distributional-inclusion-hypotheses-lexical-entailment/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2005](https://md.research.elamur.ai/free/en/papers/2005/) / The Distributional Inclusion Hypotheses and Lexical Entailment

# The Distributional Inclusion Hypotheses and Lexical Entailment

**Authors:** Maayan Geffet, [Ido Dagan](https://md.research.elamur.ai/free/en/authors/ido-dagan/) · **Year:** 2005 · **Venue:** Proceedings of the 43rd Annual Meeting of the Association for Computational Linguistics (ACL'05) · **Licence:** [CC BY-NC-SA 3.0 (ACL Anthology, before 2016)](https://creativecommons.org/licenses/by-nc-sa/3.0/)

**Links:** [Publisher's page](https://aclanthology.org/P05-1014/) · [PDF](https://aclanthology.org/P05-1014.pdf) · [DOI 10.3115/1219840.1219854](https://doi.org/10.3115/1219840.1219854) · [OpenAlex](https://openalex.org/W2133413932)

**Citations:** 198 citations (openalex-snapshot · 2026-10-14) · [92](#cited-by) in our library, of the 140884 papers whose bibliographies we read

## Summary
Classic distributional similarity tends to group words that are only loosely related (e.g., company–government). It does not tell us whether one word can stand in for another while keeping the meaning inferable, which is what QA, IE and paraphrase systems need. Geffet and Dagan propose two Distributional Inclusion Hypotheses as a refinement of Harris' idea. The first says that if v entails w, the characteristic syntactic contexts of v should also appear with w. The second states the converse: such inclusion suggests entailment. To test these claims, they built an Inclusion Testing Algorithm (ITA). It takes Minipar dependency features from a corpus and checks whether they also occur with the other word, using web queries to offset data sparseness. A manual, sense-level analysis found that the first hypothesis holds almost completely, while the second holds only partly. Used as a filter on RFF-based similarity lists, ITA raised precision from 0.52 to 0.70–0.74 with only a modest loss in recall. This shows that asymmetric feature inclusion is a more useful signal for lexical entailment than symmetric overlap.

## Key points
- The evaluation data comes from the authors' earlier work: 1,200 noun pairs, built from the top-40 RFF similarity neighbours of 30 random nouns. Annotators judged each pair for lexical entailment, reaching Kappa 0.75 and 0.83. For this paper the authors also added the direction of entailment.
- RFF (Relative Feature Focus) re-weighting had already given about 10% higher precision than Lin's MI-based measure. Even so, over 47% of the top-40 pairs were not entailing. Fewer than 20% of the extracted entailments were found in WordNet.
- The filter was tested on a 200-pair sample, with precision / recall / F1 as follows:

  | Method | Precision | Recall | F1 |
  |---|---|---|---|
  | ITA-20 | 0.700 | 0.875 | 0.777 |
  | ITA-40 | 0.740 | 0.846 | 0.789 |
  | RFF-top-40 | 0.520 | 1.000 | 0.684 |
  | RFF-top-26 | 0.561 | 0.701 | 0.624 |

- ITA discarded about 60% of the incorrect pairs and about 35% of the output overall. Relative recall dropped by only 13%.
- After sense-level analysis, 27.4% of non-entailing pairs still passed the inclusion test. The causes were:
  - generic domain features, as in valley–town;
  - anaphoric, context-dependent entailment;
  - very broad, ambiguous words such as element.

## Interesting ideas
- **Directional inclusion instead of symmetric similarity.** The test asks whether one word's defining contexts are a subset of the other's contexts. This naturally captures asymmetric relations such as hyponymy, which cosine or Lin scores cannot express.
- **The web as a sparseness oracle.** Before treating a missing word–feature co-occurrence as real counter-evidence, the method checks it against a very large external source. This helps any pipeline built on small corpora, such as one for a lower-resource language.
- **Sample features at random rather than taking the top-weighted ones.** The top RFF features turned out to be shared across many similar words and were therefore less discriminative. In feature-based tests, the highest-weighted features are not always the most diagnostic.
- **Compare against an equal-size cutoff.** The authors set the filter against simply trimming the ranked list to the same size (top-26). This shows the gain comes from the filter itself, not just from producing shorter output.
- **Check errors at the sense level.** Re-examining failures by word sense separated real counterexamples from polysemy artifacts. This gives a fairer picture of whether a hypothesis actually holds.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

Fields: [Natural language processing](https://md.research.elamur.ai/free/en/shelves/nlp/)

## Cited by

Cited by

92 papers of our library cite it (of the 140884 whose bibliographies we read); 3 of them have a page here, 89 not yet.

- [Entailment above the word level in distributional semantics](https://md.research.elamur.ai/free/en/papers/baroni-2012-entailment-above-word-level-distributional/)
- [Learning Semantic Hierarchies via Word Embeddings](https://md.research.elamur.ai/free/en/papers/fu-2014-learning-semantic-hierarchies-word-embeddings/)
- [Espresso: Leveraging Generic Patterns for Automatically Harvesting Semantic Relations](https://md.research.elamur.ai/free/en/papers/pantel-2006-espresso-leveraging-generic-patterns-automatically/)

## Cite

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{geffet2005distributional,
  title = {{The Distributional Inclusion Hypotheses and Lexical Entailment}},
  author = {Geffet, Maayan and Dagan, Ido},
  booktitle = {Proceedings of the 43rd Annual Meeting of the Association for Computational Linguistics (ACL'05)},
  year = {2005},
  doi = {10.3115/1219840.1219854},
  url = {https://doi.org/10.3115/1219840.1219854}
}
```

- [← previous Syntactic Features for Evaluation of Machine Translation](https://md.research.elamur.ai/free/en/papers/liu-2005-syntactic-features-evaluation-machine-translation/)
- [next → The Proposition Bank: An Annotated Corpus of Semantic Roles](https://md.research.elamur.ai/free/en/papers/palmer-2005-proposition-bank-annotated-corpus-semantic/)
- [↑ back to papers of 2005](https://md.research.elamur.ai/free/en/papers/2005/)

## References

20 references, 4 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Chklovski, Timothy and Patrick Pantel. 2004. V ERB O CEAN : Mining the Web for Fine-Grained Semantic Verb Relations. In Proc. of EMNLP-04. Barcelona, Spain. [find](https://scholar.google.com/scholar?q=Chklovski%2C+Timothy+and+Patrick+Pantel%2E+2004%2E+V+ERB+O+CEAN+%3A+Mining+the+Web+for+Fine%2DGrained+Semantic+Verb+Relations%2E+In+Proc%2E+of+EMNLP%2D04%2E+Barcelona%2C+Spain%2E)
2. Church, Kenneth W. and Hanks Patrick. 1990. Word association norms, mutual information, and Lexicography. Computational Linguistics, 16(1), pp. 22–29. [find](https://scholar.google.com/scholar?q=Church%2C+Kenneth+W%2E+and+Hanks+Patrick%2E+1990%2E+Word+association+norms%2C+mutual+information%2C+and+Lexicography%2E+Computational+Linguistics%2C+16%281%29%2C+pp%2E+22%E2%80%9329%2E) [our card](https://md.research.elamur.ai/free/en/papers/church-1990-word-association-norms-mutual-information/)
3. Dagan, Ido. 2000. Contextual Word Similarity, in Rob Dale, Hermann Moisl and Harold Somers (Eds.), Handbook of Natural Language Processing, Marcel Dekker Inc, 2000, Chapter 19, pp. 459-476. [find](https://scholar.google.com/scholar?q=Dagan%2C+Ido%2E+2000%2E+Contextual+Word+Similarity%2C+in+Rob+Dale%2C+Hermann+Moisl+and+Harold+Somers+%28Eds%2E%29%2C+Handbook+of+Natural+Language+Processing%2C+Marcel+Dekker+Inc%2C+2000%2C+Chapter+19%2C+pp%2E+459%2D476%2E)
4. Dagan, Ido, Oren Glickman and Bernardo Magnini. 2005. The PASCAL Recognizing Textual Entailment Challenge. In Proc. of the PASCAL Challenges Workshop for Recognizing Textual Entailment. Southampton, U.K. [find](https://scholar.google.com/scholar?q=Dagan%2C+Ido%2C+Oren+Glickman+and+Bernardo+Magnini%2E+2005%2E+The+PASCAL+Recognizing+Textual+Entailment+Challenge%2E+In+Proc%2E+of+the+PASCAL+Challenges+Workshop+for+Recognizing+Textual+Entailment%2E+Southampton%2C+U)
5. Geffet, Maayan and Ido Dagan, 2004. Feature Vector Quality and Distributional Similarity. In Proc. of Coling-04. Geneva. Switzerland. [find](https://scholar.google.com/scholar?q=Geffet%2C+Maayan+and+Ido+Dagan%2C+2004%2E+Feature+Vector+Quality+and+Distributional+Similarity%2E+In+Proc%2E+of+Coling%2D04%2E+Geneva%2E+Switzerland%2E)
6. Grefenstette, Gregory. 1994. Exploration in Automatic Thesaurus Discovery. Kluwer Academic Publishers. [find](https://scholar.google.com/scholar?q=Grefenstette%2C+Gregory%2E+1994%2E+Exploration+in+Automatic+Thesaurus+Discovery%2E+Kluwer+Academic+Publishers%2E)
7. Harris, Zelig S. Mathematical structures of language. Wiley, 1968. [find](https://scholar.google.com/scholar?q=Harris%2C+Zelig+S%2E+Mathematical+structures+of+language%2E+Wiley%2C+1968%2E)
8. Hearst, Marti. 1992. Automatic acquisition of hyponyms from large text corpora. In Proc. of COLING-92. Nantes, France. [find](https://scholar.google.com/scholar?q=Hearst%2C+Marti%2E+1992%2E+Automatic+acquisition+of+hyponyms+from+large+text+corpora%2E+In+Proc%2E+of+COLING%2D92%2E+Nantes%2C+France%2E)
9. Keller, Frank, Maria Lapata, and Olga Ourioupina. 2002. Using the Web to Overcome Data Sparseness. In Jan Hajic and Yuji Matsumoto, eds., In Proc. of EMNLP-02. Philadelphia, PA. [find](https://scholar.google.com/scholar?q=Keller%2C+Frank%2C+Maria+Lapata%2C+and+Olga+Ourioupina%2E+2002%2E+Using+the+Web+to+Overcome+Data+Sparseness%2E+In+Jan+Hajic+and+Yuji+Matsumoto%2C+eds%2E%2C+In+Proc%2E+of+EMNLP%2D02%2E+Philadelphia%2C+PA%2E)
10. Lee, Lillian. 1997. Similarity-Based Approaches to Natural Language Processing. Ph.D. thesis, Harvard University, Cambridge, MA. [find](https://scholar.google.com/scholar?q=Lee%2C+Lillian%2E+1997%2E+Similarity%2DBased+Approaches+to+Natural+Language+Processing%2E+Ph%2ED%2E+thesis%2C+Harvard+University%2C+Cambridge%2C+MA%2E)
11. Lin, Dekang. 1993. Principle-Based Parsing without Overgeneration. In Proc. of ACL-93. Columbus, Ohio. . [find](https://scholar.google.com/scholar?q=Lin%2C+Dekang%2E+1993%2E+Principle%2DBased+Parsing+without+Overgeneration%2E+In+Proc%2E+of+ACL%2D93%2E+Columbus%2C+Ohio%2E+%2E)
12. Lin, Dekang. 1998. Automatic Retrieval and Clustering of Similar Words. In Proc. of COLING–ACL98, Montreal, Canada. [find](https://scholar.google.com/scholar?q=Lin%2C+Dekang%2E+1998%2E+Automatic+Retrieval+and+Clustering+of+Similar+Words%2E+In+Proc%2E+of+COLING%E2%80%93ACL98%2C+Montreal%2C+Canada%2E) [our card](https://md.research.elamur.ai/free/en/papers/lin-1998-automatic-retrieval-clustering-similar-words/)
13. Moldovan, Dan, Badulescu, A., Tatu, M., Antohe, D., and Girju, R. 2004. Models for the semantic classification of noun phrases. In Proc. of HLT/NAACL-2004 Workshop on Computational Lexical Semantics. Boston. [find](https://scholar.google.com/scholar?q=Moldovan%2C+Dan%2C+Badulescu%2C+A%2E%2C+Tatu%2C+M%2E%2C+Antohe%2C+D%2E%2C+and+Girju%2C+R%2E+2004%2E+Models+for+the+semantic+classification+of+noun+phrases%2E+In+Proc%2E+of+HLT%2FNAACL%2D2004+Workshop+on+Computational+Lexical+Semantics%2E+)
14. Pado, Sebastian and Mirella Lapata. 2003. Constructing semantic space models from parsed corpora. In Proc. of ACL-03, Sapporo, Japan. [find](https://scholar.google.com/scholar?q=Pado%2C+Sebastian+and+Mirella+Lapata%2E+2003%2E+Constructing+semantic+space+models+from+parsed+corpora%2E+In+Proc%2E+of+ACL%2D03%2C+Sapporo%2C+Japan%2E)
15. Pantel, Patrick and Dekang Lin. 2002. Discovering Word Senses from Text. In Proc. of ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD-02). Edmonton, Canada. [find](https://scholar.google.com/scholar?q=Pantel%2C+Patrick+and+Dekang+Lin%2E+2002%2E+Discovering+Word+Senses+from+Text%2E+In+Proc%2E+of+ACM+SIGKDD+Conference+on+Knowledge+Discovery+and+Data+Mining+%28KDD%2D02%29%2E+Edmonton%2C+Canada%2E)
16. Pereira, Fernando, Tishby Naftali, and Lee Lillian. 1993. Distributional clustering of English words. In Proc. of ACL-93. Columbus, Ohio. [find](https://scholar.google.com/scholar?q=Pereira%2C+Fernando%2C+Tishby+Naftali%2C+and+Lee+Lillian%2E+1993%2E+Distributional+clustering+of+English+words%2E+In+Proc%2E+of+ACL%2D93%2E+Columbus%2C+Ohio%2E) [our card](https://md.research.elamur.ai/free/en/papers/pereira-1993-distributional-clustering-english-words/)
17. Ravichandran, Deepak and Eduard Hovy. 2002. Learning Surface Text Patterns for a Question Answering System. In Proc. of ACL-02. Philadelphia, PA. [find](https://scholar.google.com/scholar?q=Ravichandran%2C+Deepak+and+Eduard+Hovy%2E+2002%2E+Learning+Surface+Text+Patterns+for+a+Question+Answering+System%2E+In+Proc%2E+of+ACL%2D02%2E+Philadelphia%2C+PA%2E) [our card](https://md.research.elamur.ai/free/en/papers/ravichandran-2002-learning-surface-text-patterns-question/)
18. Ruge, Gerda. 1992. Experiments on linguistically-based term associations. Information Processing & Management, 28(3), pp. 317–332. [find](https://scholar.google.com/scholar?q=Ruge%2C+Gerda%2E+1992%2E+Experiments+on+linguistically%2Dbased+term+associations%2E+Information+Processing+%26+Management%2C+28%283%29%2C+pp%2E+317%E2%80%93332%2E)
19. Weeds, Julie and David Weir. 2003. A General Framework for Distributional Similarity. In Proc. of EMNLP-03. Sapporo, Japan. [find](https://scholar.google.com/scholar?q=Weeds%2C+Julie+and+David+Weir%2E+2003%2E+A+General+Framework+for+Distributional+Similarity%2E+In+Proc%2E+of+EMNLP%2D03%2E+Sapporo%2C+Japan%2E)
20. Weeds, Julie, D. Weir, D. McCarthy. 2004. Characterizing Measures of Lexical Distributional Similarity. In Proc. of Coling-04. Geneva, Switzerland. [find](https://scholar.google.com/scholar?q=Weeds%2C+Julie%2C+D%2E+Weir%2C+D%2E+McCarthy%2E+2004%2E+Characterizing+Measures+of+Lexical+Distributional+Similarity%2E+In+Proc%2E+of+Coling%2D04%2E+Geneva%2C+Switzerland%2E)