---
title: "Inference Rules for Recognizing Textual Entailment — Dinu 2009"
description: "The paper asks why large automatically learned collections of inference rules, DIRT in particular, have… 15 references, cited by 1 paper here."
authors: ["Georgiana Dinu", "Rui Wang"]
year: 2009
venue: "Proceedings of the Eighth International Conference on Computational Semantics"
source: "https://aclanthology.org/W09-3710/"
licence: "CC BY-NC-SA 3.0 (ACL Anthology, before 2016)"
licence_url: "https://creativecommons.org/licenses/by-nc-sa/3.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/dinu-2009-inference-rules-recognizing-textual-entailment/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/dinu-2009-inference-rules-recognizing-textual-entailment/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2009](https://md.research.elamur.ai/free/en/papers/2009/page/2/) / Inference Rules for Recognizing Textual Entailment

# Inference Rules for Recognizing Textual Entailment

**Authors:** [Georgiana Dinu](https://md.research.elamur.ai/free/en/authors/georgiana-dinu/) and [Rui Wang](https://md.research.elamur.ai/free/en/authors/rui-wang/) · **Year:** 2009 · **Venue:** [Proceedings of the Eighth International Conference on Computational Semantics](https://md.research.elamur.ai/free/en/venues/iwcs/) ([International Conference on Computational Semantics 2009](https://md.research.elamur.ai/free/en/venues/iwcs/2009/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY-NC-SA 3.0 (ACL Anthology, before 2016)](https://creativecommons.org/licenses/by-nc-sa/3.0/)

**Links:** [Publisher's page](https://aclanthology.org/W09-3710/) · [PDF](https://aclanthology.org/W09-3710.pdf) · [OpenAlex](https://openalex.org/W2072735006)

**Citations:** 1 citations (OpenAlex, 2026-10-14) · 1 in our library, of the 149,439 papers whose bibliographies we read

## Summary
The paper asks why large automatically learned collections of inference rules, DIRT in particular, have added so little to recognizing textual entailment (RTE). It finds two causes. First, the needed rules are often missing. Second, rules that do exist are hard to apply in the right context.

For the first problem, the authors grow DIRT by swapping lexical items in its patterns for WordNet synonyms. WordNet also serves to filter out some bad rules. For the second, they reduce the dependency parses of T and H to a compact "tree skeleton" structure. This structure decides where a rule may fire, and anchors do not have to match exactly.

On RTE2 and RTE3 test data, the method fires on only a small share of pairs. Where it does fire, precision is fairly high. Layered on a word-overlap baseline, it gives a clear gain on the covered pairs but only a small gain on the full sets.

The work matters as a diagnosis. It measures how rarely distributional rules really apply. It shows that precision-oriented rule use can still be combined with other components. Its error analysis points to weak structural context, wrong rules, and phenomena such as embedding verbs.

## Key points
- The DIRT collection was built by running the DIRT algorithm over 1GB of news text. It relies on Minipar dependency paths and the Extended Distributional Hypothesis, and its accuracy has been reported at roughly 50%.
- Under strict rule matching, with identical anchors in T and H, only about 2% of RTE pairs contain a DIRT rule. About half of these are single-verb rules, and half of those are already in WordNet. If anchor values are ignored, rule patterns show up in about 25% of pairs.
- The extended method (Dirt+Id+WN) covers just 8% of RTE2 and 6% of RTE3 test pairs. Precision on RTE3 reaches about 69%. A stricter variant leaves no unmatched lexical items in the matched path fragment. It raises precision but loses roughly 30% of coverage.
- The combination with a BoW baseline works as follows:
  - On covered pairs, RTE2 (89 pairs) improves from 52.80% to 60.67%, and RTE3 (71 pairs) from 52.11% to 59.15%.
  - On the full 800-pair sets, the change is only 56.87%→57.75% and 61.12%→61.75%.
- The error analysis covers 30 misclassified RTE3 pairs:
  - 7 (23%) come from tree skeleton structure errors.
  - 9 (30%) come from incorrect or context-dependent rules, for example X generate Y ≈ X earn Y.
  - 14 (47%) are other phenomena, such as "imitation diamond", restrictors, and embedding verbs like manage, forget and attempt.

## Interesting ideas
- Before building on a knowledge resource, measure how often its rules could apply under two settings: strict anchor matching gives a lower bound and anchor-free pattern matching gives an upper bound; this reveals whether the bottleneck is missing knowledge or failure to apply it.
- Grow a sparse statistical rule set by replacing individual lexical items in its patterns with synonyms from a curated lexicon; this lets the corpus-derived structure and the hand-built lexical knowledge cover each other's gaps.
- Use a high-precision, low-coverage component that overrides a simple baseline only on the inputs it covers, and report accuracy on the covered subset and on the full set separately.
- Make rule application stricter by requiring that the matched fragment of the dependency path contains no leftover unmatched words; this gives a tunable trade between precision and coverage.
- A rule whose patterns match while its anchors mismatch, or stand in the wrong hypernym/hyponym relation, can serve as evidence for non-entailment or contradiction rather than being ignored.

## For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

- **I1** (learn): Run this two-bound count over the compiled verb-class and frame rules on our test corpora, so we know whether to invest in more rules or in better matching.
- **I2** (world): Expand paraphrase or entailment patterns in the knowledge layer with WordNet synonym slots, keeping a record of which substitution produced each derived rule so it stays explainable.
- **I3** (answer): Let a rule-based entailment checker decide only when it fires, falling back otherwise, and always report covered-set scores so that small overall gains do not hide real wins.
- **I4** (frames): When mapping a UD subtree to a frame via a paraphrase rule, add a strict mode that rejects matches leaving unexplained content words, and use it where confidence matters more than recall.
- **I5** (world): Feed such pattern-matched but anchor-conflicting cases into the world model as candidate contradictions between entity states, each with an explicit reason.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Saarland University](https://md.research.elamur.ai/free/en/universities/saarland-university/) — University, Saarbrücken, Germany · 148 works · 18,287 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works

🧭 Topics (1): [🧮 Reasoning](https://md.research.elamur.ai/free/en/topics/reasoning/) · 1,426 works

## Cite

Cite as: Georgiana Dinu and Rui Wang. 2009. Inference Rules for Recognizing Textual Entailment. Proceedings of the Eighth International Conference on Computational Semantics.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{dinu2009inference,
  title = {{Inference Rules for Recognizing Textual Entailment}},
  author = {Georgiana Dinu and Rui Wang},
  booktitle = {Proceedings of the Eighth International Conference on Computational Semantics},
  year = {2009},
  doi = {10.3115/1693756.1693768},
  url = {https://doi.org/10.3115/1693756.1693768}
}
```

- [← previous Improving Unsegmented Statistical Dialogue Act Labelling](https://md.research.elamur.ai/free/en/papers/tamarit-2009-improving-unsegmented-statistical-dialogue-act/)
- [next → Sobre as noções ‘nativo’/não nativo’: uma dicotomia perigosa para o inglês global? On the notions “native”/”nonnative”: a dangerous dichotomy for World Englishes?](https://md.research.elamur.ai/free/en/papers/schmitz-2009-sobre-nocoes-nativo-nao-nativo/)
- [↑ back to papers of 2009](https://md.research.elamur.ai/free/en/papers/2009/page/2/)

## References

15 references, 10 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Roy Bar-Haim, Ido Dagan, Iddo Greental, Idan Szpektor, and Moshe Friedman. [Semantic inference at the lexical-syntactic level for textual entailment recognition](https://md.research.elamur.ai/free/en/papers/haim-2007-semantic-inference-lexical-syntactic-level/). In Proceedings of the ACL-PASCAL Workshop on Textual Entailment and Paraphrasing, pages 131–136, Prague, June 2007.
2. Regina Barzilay and Kathleen R. McKeown. [Extracting paraphrases from a parallel corpus](https://md.research.elamur.ai/free/en/papers/barzilay-2001-extracting-paraphrases-parallel-corpus/). In Proceedings of 39th Annual Meeting of the Association for Computational Linguistics, pages 50–57, Toulouse, France, July 2001.
3. Peter Clark, Phil Harrison, John Thompson, William Murray, Jerry Hobbs, and Christiane Fellbaum. On the role of lexical and world knowledge in rte3. In Proceedings of the ACL-PASCAL Workshop on Textual Entailment and Paraphrasing, pages 54–59, June 2007. [find](https://scholar.google.com/scholar?q=Peter+Clark%2C+Phil+Harrison%2C+John+Thompson%2C+William+Murray%2C+Jerry+Hobbs%2C+and+Christiane+Fellbaum%2E+On+the+role+of+lexical+and+world+knowledge+in+rte3%2E+In+Proceedings+of+the+ACL%2DPASCAL+Workshop+on+Textua)
4. Ido Dagan, Oren Glickman, and Bernardo Magnini. The pascal recognising textual entailment challenge. In Lecture Notes in Computer Science, Vol. 3944, Springer, pages 177–190. Quionero-Candela, J.; Dagan, I.; Magnini, B.; d’Alch-Buc, F. Machine Learning Challenges, 2006. [find](https://scholar.google.com/scholar?q=Ido+Dagan%2C+Oren+Glickman%2C+and+Bernardo+Magnini%2E+The+pascal+recognising+textual+entailment+challenge%2E+In+Lecture+Notes+in+Computer+Science%2C+Vol%2E+3944%2C+Springer%2C+pages+177%E2%80%93190%2E+Quionero%2DCandela%2C+J%2E%3B+Dag)
5. Marie-Catherine de Marneffe, Anna N. Rafferty, and Christopher D. Manning. [Finding contradictions in text](https://md.research.elamur.ai/free/en/papers/marneffe-2008-finding-contradictions-text/). In Proceedings of ACL-08: HLT, pages 1039– 1047, Columbus, Ohio, June 2008.
6. Danilo Giampiccolo, Bernardo Magnini, Ido Dagan, and Bill Dolan. [The third pascal recognizing textual entailment challenge](https://md.research.elamur.ai/free/en/papers/giampiccolo-2007-third-pascal-recognizing-textual-entailment/). In Proceedings of the ACL-PASCAL Workshop on Textual Entailment and Paraphrasing, pages 1– 9, Prague, June 2007.
7. Sanda Harabagiu and Andrew Hickl. [Methods for using textual entailment in open-domain question answering](https://md.research.elamur.ai/free/en/papers/harabagiu-2006-methods-textual-entailment-open-domain/). In Proceedings of the 21st International Conference on Computational Linguistics and 44th Annual Meeting of the ACL, pages 905–912, Sydney, Australia, July 2006.
8. Adrian Iftene and Alexandra Balahur-Dobrescu. [Hypothesis transformation and semantic variability rules used in recognizing textual entailment](https://md.research.elamur.ai/free/en/papers/iftene-2007-hypothesis-transformation-semantic-variability-rules/). In Proceedings of the ACL-PASCAL Workshop on Textual Entailment and Paraphrasing, pages 125–130, Prague, June 2007.
9. Dekang Lin. Dependency-based evaluation of minipar. In Proc. Workshop on the Evaluation of Parsing Systems, Granada, 1998. [find](https://scholar.google.com/scholar?q=Dekang+Lin%2E+Dependency%2Dbased+evaluation+of+minipar%2E+In+Proc%2E+Workshop+on+the+Evaluation+of+Parsing+Systems%2C+Granada%2C+1998%2E)
10. Dekang Lin and Patrick Pantel. Dirt. discovery of inference rules from text. In KDD ’01: Proceedings of the seventh ACM SIGKDD international conference on Knowledge discovery and data mining, pages 323–328, New York, USA, 2001. [find](https://scholar.google.com/scholar?q=Dekang+Lin+and+Patrick+Pantel%2E+Dirt%2E+discovery+of+inference+rules+from+text%2E+In+KDD+%E2%80%9901%3A+Proceedings+of+the+seventh+ACM+SIGKDD+international+conference+on+Knowledge+discovery+and+data+mining%2C+pages+32)
11. Erwin Marsi, Emiel Krahmer, and Wauter Bosma. [Dependency-based paraphrasing for recognizing textual entailment](https://md.research.elamur.ai/free/en/papers/marsi-2007-dependency-based-paraphrasing-recognizing-textual/). In Proceedings of the ACL-PASCAL Workshop on Textual Entailment and Paraphrasing, pages 83–88, Prague, June 2007.
12. Bo Pang, Kevin Knight, and Daniel Marcu. [Syntax-based alignment of multiple translations: Extracting paraphrases and generating new sentences](https://md.research.elamur.ai/free/en/papers/pang-2003-syntax-based-alignment-multiple-translations/). In HLT-NAACL, pages 102–109, 2003.
13. Idan Szpektor, Eyal Shnarch, and Ido Dagan. Instance-based evaluation of entailment rule acquisition. In Proceedings of the 45th Annual Meeting of the Association of Computational Linguistics, pages 456–463, Prague, Czech Republic, June 2007. [find](https://scholar.google.com/scholar?q=Idan+Szpektor%2C+Eyal+Shnarch%2C+and+Ido+Dagan%2E+Instance%2Dbased+evaluation+of+entailment+rule+acquisition%2E+In+Proceedings+of+the+45th+Annual+Meeting+of+the+Association+of+Computational+Linguistics%2C+pages+4)
14. Idan Szpektor, Hristo Tanev, Ido Dagan, and Bonaventura Coppola. [Scaling web-based acquisition of entailment relations](https://md.research.elamur.ai/free/en/papers/szpektor-2004-scaling-web-based-acquisition-entailment/). In Proceedings of EMNLP, pages 41–48, 2004.
15. Rui Wang and Günter Neumann. [Recognizing textual entailment using sentence similarity based on dependency tree skeletons](https://md.research.elamur.ai/free/en/papers/wang-2007-recognizing-textual-entailment-sentence-similarity/). In Proceedings of the ACL-PASCAL Workshop on Textual Entailment and Paraphrasing, pages 36– 41, June 2007.

*Updated 30 October 2026*