---
title: "BIUTEE — paper by Stern and Dagan (2012)"
description: "The authors release an open-source, modular platform. 17 references, cited by 10 papers here."
authors: ["Asher Stern", "Ido Dagan"]
year: 2012
venue: "Proceedings of the ACL 2012 System Demonstrations"
source: "https://aclanthology.org/P12-3013/"
licence: "CC BY-NC-SA 3.0 (ACL Anthology, before 2016)"
licence_url: "https://creativecommons.org/licenses/by-nc-sa/3.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/stern-2012-biutee-modular-open-source-system/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/stern-2012-biutee-modular-open-source-system/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2012](https://md.research.elamur.ai/free/en/papers/2012/page/2/) / BIUTEE: A Modular Open-Source System for Recognizing Textual Entailment

# BIUTEE: A Modular Open-Source System for Recognizing Textual Entailment

**Authors:** Asher Stern and [Ido Dagan](https://md.research.elamur.ai/free/en/authors/ido-dagan/) · **Year:** 2012 · **Venue:** [Proceedings of the ACL 2012 System Demonstrations](https://md.research.elamur.ai/free/en/venues/acl/) ([ACL 2012](https://md.research.elamur.ai/free/en/venues/acl/2012/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY-NC-SA 3.0 (ACL Anthology, before 2016)](https://creativecommons.org/licenses/by-nc-sa/3.0/)

**Links:** [Publisher's page](https://aclanthology.org/P12-3013/) · [PDF](https://aclanthology.org/P12-3013.pdf) · [OpenAlex](https://openalex.org/W2168995019)

**Citations:** 21 citations (OpenAlex, 2026-10-14) · [10](https://md.research.elamur.ai/free/en/papers/stern-2012-biutee-modular-open-source-system/cited-by/) in our library, of the 141,840 papers whose bibliographies we read

## Summary
BIUTEE tackles a practical barrier in Recognizing Textual Entailment (RTE): building a full inference system from scratch takes a lot of work, and the few public systems (EDITS, NutCracker) could use only narrow kinds of knowledge. The authors release an open-source, modular platform. It decides whether a hypothesis H follows from a text T by rewriting T's parse tree into H through a chain of tree edits, called a proof. Edits include entailment rules (lexical, lexical-syntactic, syntactic), replacing a mention with a coreferent one, and "on-the-fly" operations with no backing resource, such as inserting nodes or moving subtrees. Each edit gets a cost: a learned weight vector multiplied by the edit's features. A search procedure looks for the cheapest proof, and the system reports entailment when the total cost falls below a learned threshold. Preprocessors, knowledge resources, learners and new edit types can all be swapped through interfaces or plug-ins. A visual debugger, Tracer, shows every step of the process. BIUTEE scored above the median on RTE-6 and RTE-7 and was the top system on RTE-6.

## Key points
- Preprocessing bundles the Easy-First parser, the Stanford NER, the ArkRef coreference resolver, a sentence splitter and number normalization; users can plug in their own tools.
- Proof cost is c(O) = w · Σ f(oᵢ). Both w and the threshold b are trained with Logistic Regression, and the final cost is mapped to a score in [0, 1].
- F1 results:
  - RTE-6: 49.09, against a median of 33.72 and a best competing score of 48.01.
  - RTE-7: 42.93, against a median of 39.89 and a best of 48.00.
- A plug-in implements interfaces that do three things: detect applicable transformations, apply them, and build their feature vectors. A plug-in can also add computations to a global pre-proof phase, such as marking negation and its scope.
- Tracer has two modes. Automatic mode replays the search's proof with per-step costs and before/after trees. Manual mode lists every matching transformation, including ones the search rejected, so the user can apply them one at a time.

## Interesting ideas
- Entailment is treated as a derivation: a sequence of tree rewrites whose cost is a sum of learned per-step costs, so every verdict comes with an itemized, inspectable justification.
- Edits that no knowledge resource supports are still allowed but priced by features such as word frequency from a unigram model and named-entity status, so a proof always exists and its weak spots are expensive and visible.
- One rule format (a left-hand and a right-hand subtree) covers lexical substitutions, predicate-argument paraphrases like accept → was offered, and syntactic alternations like active/passive.
- A human can build a proof by hand to locate the fault. If the hand-built proof is cheaper than the system's, the search is at fault. If it is linguistically better but costs more, the learned weights or the training data are at fault.
- The debugger shows the candidate operations that were rejected, not only the winning path.

## For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

- **I1** (answer): yes/no and "does the text say X" questions could be answered by rewriting the text's UD tree toward the question's tree, returning the costed step list as the explanation.
- **I2** (world): when linking a text to compiled knowledge, unsupported bridging assumptions could be admitted as explicitly costed steps instead of silently failing or silently succeeding.
- **I3** (frames): VerbNet-based role extraction could run after a pass of LHS/RHS subtree rules that normalize alternations, so each frame records which rules produced it.
- **I4** (learn): the same contrast could serve as an evaluation probe that tells search errors apart from model or data errors in the parser and in frame building.
- **I5** (tree): a trace view for the perceptron parser or gparse that lists competing arcs with their scores at each decision would make parse errors much easier to diagnose.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Bar-Ilan University](https://md.research.elamur.ai/free/en/universities/bar-ilan-university/) — University, Ramat Gan, Israel · 118 works · 21,077 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 9,086 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 8,889 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 9,296 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 9,206 works

🧭 Topics (1): [🧮 Reasoning](https://md.research.elamur.ai/free/en/topics/reasoning/) · 510 works

## Cite

Cite as: Asher Stern and Ido Dagan. 2012. BIUTEE: A Modular Open-Source System for Recognizing Textual Entailment. Proceedings of the ACL 2012 System Demonstrations.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{stern2012biutee,
  title = {{BIUTEE: A Modular Open-Source System for Recognizing Textual Entailment}},
  author = {Asher Stern and Ido Dagan},
  booktitle = {Proceedings of the ACL 2012 System Demonstrations},
  year = {2012},
  url = {https://aclanthology.org/P12-3013/}
}
```

- [← previous A Comparative Study of Root-Based and Stem-Based Approaches for Measuring the Similarity between Arabic Words](https://md.research.elamur.ai/free/en/papers/froud-2012-comparative-study-root-based-stem/)
- [next → Flexible finite-state lexical selection for rule-based machine translation](https://md.research.elamur.ai/free/en/papers/tyers-2012-flexible-finite-state-lexical-selection/)
- [↑ back to papers of 2012](https://md.research.elamur.ai/free/en/papers/2012/page/2/)

## References

17 references, 4 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Roy Bar-Haim, Ido Dagan, Iddo Greental, and Eyal Shnarch. 2007. Semantic inference at the lexical-syntactic level. In Proceedings of AAAI. [find](https://scholar.google.com/scholar?q=Roy+Bar%2DHaim%2C+Ido+Dagan%2C+Iddo+Greental%2C+and+Eyal+Shnarch%2E+2007%2E+Semantic+inference+at+the+lexical%2Dsyntactic+level%2E+In+Proceedings+of+AAAI%2E)
2. Luisa Bentivogli, Peter Clark, Ido Dagan, Hoa Dang, and Danilo Giampiccolo. 2010. The sixth pascal recognizing textual entailment challenge. In Proceedings of TAC. [find](https://scholar.google.com/scholar?q=Luisa+Bentivogli%2C+Peter+Clark%2C+Ido+Dagan%2C+Hoa+Dang%2C+and+Danilo+Giampiccolo%2E+2010%2E+The+sixth+pascal+recognizing+textual+entailment+challenge%2E+In+Proceedings+of+TAC%2E)
3. Luisa Bentivogli, Peter Clark, Ido Dagan, Hoa Dang, and Danilo Giampiccolo. 2011. The seventh pascal recognizing textual entailment challenge. In Proceedings of TAC. [find](https://scholar.google.com/scholar?q=Luisa+Bentivogli%2C+Peter+Clark%2C+Ido+Dagan%2C+Hoa+Dang%2C+and+Danilo+Giampiccolo%2E+2011%2E+The+seventh+pascal+recognizing+textual+entailment+challenge%2E+In+Proceedings+of+TAC%2E)
4. Johan Bos and Katja Markert. 2005. [Recognising textual entailment with logical inference](https://md.research.elamur.ai/free/en/papers/bos-2005-recognising-textual-entailment-logical-inference/). In Proceedings of EMNLP.
5. Peter Clark and Phil Harrison. 2010. Blue-lite: a knowledge-based lexical entailment system for rte6. In Proceedings of TAC. [find](https://scholar.google.com/scholar?q=Peter+Clark+and+Phil+Harrison%2E+2010%2E+Blue%2Dlite%3A+a+knowledge%2Dbased+lexical+entailment+system+for+rte6%2E+In+Proceedings+of+TAC%2E)
6. Ido Dagan, Oren Glickman, and Bernardo Magnini. 2006. The pascal recognising textual entailment challenge. In Quionero-Candela, J.; Dagan, I.; Magnini, B.; d’Alch-Buc, F. (Eds.) Machine Learning Challenges. Lecture Notes in Computer Science. [find](https://scholar.google.com/scholar?q=Ido+Dagan%2C+Oren+Glickman%2C+and+Bernardo+Magnini%2E+2006%2E+The+pascal+recognising+textual+entailment+challenge%2E+In+Quionero%2DCandela%2C+J%2E%3B+Dagan%2C+I%2E%3B+Magnini%2C+B%2E%3B+d%E2%80%99Alch%2DBuc%2C+F%2E+%28Eds%2E%29+Machine+Learning+Chall)
7. Rodrigo de Salvo Braz, Roxana Girju, Vasin Punyakanok, Dan Roth, and Mark Sammons. 2005. An inference model for semantic entailment in natural language. In Proceedings of AAAI. [find](https://scholar.google.com/scholar?q=Rodrigo+de+Salvo+Braz%2C+Roxana+Girju%2C+Vasin+Punyakanok%2C+Dan+Roth%2C+and+Mark+Sammons%2E+2005%2E+An+inference+model+for+semantic+entailment+in+natural+language%2E+In+Proceedings+of+AAAI%2E)
8. Christiane Fellbaum, editor. 1998. WordNet An Electronic Lexical Database. The MIT Press, May. [find](https://scholar.google.com/scholar?q=Christiane+Fellbaum%2C+editor%2E+1998%2E+WordNet+An+Electronic+Lexical+Database%2E+The+MIT+Press%2C+May%2E)
9. Jenny Rose Finkel, Trond Grenager, and Christopher Manning. 2005. [Incorporating non-local information into information extraction systems by gibbs sampling](https://md.research.elamur.ai/free/en/papers/finkel-2005-incorporating-non-local-information-into/). In Proceedings of ACL.
10. Yoav Goldberg and Michael Elhadad. 2010. An efficient algorithm for easy-first non-directional dependency parsing. In Proceedings of NAACL. [find](https://scholar.google.com/scholar?q=Yoav+Goldberg+and+Michael+Elhadad%2E+2010%2E+An+efficient+algorithm+for+easy%2Dfirst+non%2Ddirectional+dependency+parsing%2E+In+Proceedings+of+NAACL%2E)
11. Aria Haghighi and Dan Klein. 2009. [Simple coreference resolution with rich syntactic and semantic features](https://md.research.elamur.ai/free/en/papers/haghighi-2009-simple-coreference-resolution-rich-syntactic/). In Proceedings of EMNLP.
12. Adrian Iftene. 2008. Uaic participation at rte4. In Proceedings of TAC. [find](https://scholar.google.com/scholar?q=Adrian+Iftene%2E+2008%2E+Uaic+participation+at+rte4%2E+In+Proceedings+of+TAC%2E)
13. Philipp Koehn, Hieu Hoang, Alexandra Birch, Chris Callison-Burch, Marcello Federico, Nicola Bertoldi, Brooke Cowan, Wade Shen, Christine Moran, Richard Zens, Chris Dyer, Ondrej Bojar, Alexandra Constantin, and Evan Herbst. 2007. Moses: Open source toolkit for statistical machine translation. In Proceedings of ACL. [find](https://scholar.google.com/scholar?q=Philipp+Koehn%2C+Hieu+Hoang%2C+Alexandra+Birch%2C+Chris+Callison%2DBurch%2C+Marcello+Federico%2C+Nicola+Bertoldi%2C+Brooke+Cowan%2C+Wade+Shen%2C+Christine+Moran%2C+Richard+Zens%2C+Chris+Dyer%2C+Ondrej+Bojar%2C+Alexandra+Consta)
14. Milen Kouylekov and Matteo Negri. 2010. [An open-source package for recognizing textual entailment](https://md.research.elamur.ai/free/en/papers/kouylekov-2010-open-source-package-recognizing-textual/). In Proceedings of ACL Demo.
15. Andrew MacKinlay and Timothy Baldwin. 2009. A baseline approach to the rte5 search pilot. In Proceedings of TAC. [find](https://scholar.google.com/scholar?q=Andrew+MacKinlay+and+Timothy+Baldwin%2E+2009%2E+A+baseline+approach+to+the+rte5+search+pilot%2E+In+Proceedings+of+TAC%2E)
16. Asher Stern and Ido Dagan. 2011. A confidence model for syntactically-motivated entailment proofs. In Proceedings of RANLP. [find](https://scholar.google.com/scholar?q=Asher+Stern+and+Ido+Dagan%2E+2011%2E+A+confidence+model+for+syntactically%2Dmotivated+entailment+proofs%2E+In+Proceedings+of+RANLP%2E)
17. Asher Stern, Roni Stern, Ido Dagan, and Ariel Felner. 2012. Efficient search for transformation-based inference. In Proceedings of ACL. [find](https://scholar.google.com/scholar?q=Asher+Stern%2C+Roni+Stern%2C+Ido+Dagan%2C+and+Ariel+Felner%2E+2012%2E+Efficient+search+for+transformation%2Dbased+inference%2E+In+Proceedings+of+ACL%2E)

*Updated 24 October 2026*