---
title: "Multi-Level Alignments As An Extensible Representation Basis"
description: "The authors propose multi-level alignment as a common intermediate layer. 21 references, cited by 4 papers here."
authors: ["Tae-Gil Noh", "Sebastian Padó", "Vered Shwartz", "Ido Dagan", "Vivi Nastase", "Kathrin Eichler", "Lili Kotlerman", "Meni Adler"]
year: 2015
venue: "Proceedings of the Fourth Joint Conference on Lexical and Computational Semantics"
source: "https://aclanthology.org/S15-1022/"
doi: "10.18653/v1/s15-1022"
licence: "CC BY-NC-SA 3.0 (ACL Anthology, before 2016)"
licence_url: "https://creativecommons.org/licenses/by-nc-sa/3.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/noh-2015-multi-level-alignments-as-extensible/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/noh-2015-multi-level-alignments-as-extensible/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2015](https://md.research.elamur.ai/free/en/papers/2015/page/3/) / Multi-Level Alignments As An Extensible Representation Basis for Textual Entailment Algorithms

# Multi-Level Alignments As An Extensible Representation Basis for Textual Entailment Algorithms

**Authors:** Tae-Gil Noh, [Sebastian Padó](https://md.research.elamur.ai/free/en/authors/sebastian-pado/), [Vered Shwartz](https://md.research.elamur.ai/free/en/authors/vered-shwartz/), [Ido Dagan](https://md.research.elamur.ai/free/en/authors/ido-dagan/), [Vivi Nastase](https://md.research.elamur.ai/free/en/authors/vivi-nastase/), Kathrin Eichler, Lili Kotlerman and [Meni Adler](https://md.research.elamur.ai/free/en/authors/meni-adler/) · **Year:** 2015 · **Venue:** [Proceedings of the Fourth Joint Conference on Lexical and Computational Semantics](https://md.research.elamur.ai/free/en/venues/starsem/) ([\*SEM 2015](https://md.research.elamur.ai/free/en/venues/starsem/2015/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY-NC-SA 3.0 (ACL Anthology, before 2016)](https://creativecommons.org/licenses/by-nc-sa/3.0/)

**Links:** [Publisher's page](https://aclanthology.org/S15-1022/) · [PDF](https://aclanthology.org/S15-1022.pdf) · [DOI 10.18653/v1/s15-1022](https://doi.org/10.18653/v1/s15-1022) · [OpenAlex](https://openalex.org/W2113984822)

**Citations:** 13 citations (OpenAlex, 2026-10-14) · 4 in our library, of the 141,840 papers whose bibliographies we read

## Summary
Building Textual Entailment (TE) systems is expensive. Shared platforms already standardize preprocessing, annotation types and knowledge resources, but the algorithms themselves stay monolithic and hard to extend or port to new languages. The authors propose *multi-level alignment* as a common intermediate layer. Independent "alignment producers" add weighted, labeled links between any linguistic units of Text and Hypothesis, and these links may signal either match or mismatch. "Alignment consumers" then turn the pooled links into features for a supervised classifier. Unlike the earlier schema of Dagan et al., no step chooses a single best alignment, so evidence from different sources coexists. A pilot system, MultiAlign, was built inside the Excitement Open Platform with three aligners and four coverage features. On RTE-3 it matched or beat mature open-source engines (BIUTEE, TIE, EDITS) in English, German and Italian, and it performed similarly on real application data. The point is that a modular, language-independent design can reach state-of-the-art results with very little engineering.

## Key points
- The pilot uses three aligners:
  - a lexical aligner over lemmas, using WordNet and VerbOcean for EN, Italian WordNet for IT, and GermaNet plus DerivBase for DE;
  - a paraphrase aligner over token sequences, using METEOR paraphrase tables;
  - a resource-free lemma identity aligner, which is important for named entities.
- The four features all measure how much of H is covered by alignments: all words, content words, verbs and proper names. A word counts as covered if any aligner linked it.
- Accuracy on RTE-3 (800 training / 800 test pairs, translated into DE and IT):
  - EN: 67.0, tied with BIUTEE; TIE scored 65.2 and EDITS 63.6.
  - DE: 64.5, against 63.1 for TIE.
  - IT: 65.4, against 62.6 for EDITS.
- F1 on application data (EN 5300, IT 1700, DE 1274 balanced pairs from entailment graphs and e-mail categorization):
  - EN: 69.2, against 71.3 for BIUTEE.
  - DE: 72.4, tied with TIE.
  - IT: 69.5, against 65.6 for EDITS.
- The infrastructure combines TreeTagger pipelines, UIMA CAS with the DKPro type system, and a BRAT-based visualizer that displays each link with its source and score.

## Interesting ideas
- Skip alignment selection and keep every aligner's links side by side, positive and negative, so that interactions are handled only at feature-extraction time; adding a new evidence source then never breaks existing ones.
- Split each module into an aligner plus its own feature extractors behind one shared data structure, which keeps even complex pipelines to a few clean interfaces.
- Measure coverage of the hypothesis separately per word class (content words, verbs, names), because an entailment can survive weak overall coverage as long as the crucial words are matched.
- Make feature code language-independent by working only over a shared annotation type system, so the same extractor runs unchanged for three languages.
- Use a visualizer that shows every link with its origin and weight; it exposed a bug where the noun "judges" was misread as a verb and linked via VerbOcean.

## For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

- **I1** (world): The world builder could store competing entity or state matches from different modules as labeled, scored links and let the reasoning layer resolve conflicts explicitly instead of committing early.
- **I2** (answer): Readers for school tests or fable QA could each contribute alignment-style evidence between question and story graph, with answer scoring written as small, separable extractors.
- **I3** (answer): When checking whether a candidate answer follows from the text, report verb and entity coverage of the answer frame separately, which yields an explainable, non-bag-of-words justification.
- **I4** (back): Translation through the meta-world could check source and target meaning graphs with identical coverage features, since both live in one type system.
- **I5** (learn): Rendering cross-module links with their provenance would help detect systematic upstream errors, such as tagging or lemma mistakes, and turn them into repair rules or negative controls.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Bar-Ilan University](https://md.research.elamur.ai/free/en/universities/bar-ilan-university/) — University, Ramat Gan, Israel · 118 works · 21,077 citations
- [University of Stuttgart](https://md.research.elamur.ai/free/en/universities/university-of-stuttgart/) — University, Stuttgart, Germany · 75 works · 46,113 citations
- [German Research Centre for Artificial Intelligence](https://md.research.elamur.ai/free/en/labs/german-research-centre-for-artificial-intelligence/) — Research institute, Kaiserslautern, Germany · 42 works · 28,596 citations
- [Heidelberg University](https://md.research.elamur.ai/free/en/universities/heidelberg-university/) — University, Heidelberg, Germany · 27 works · 40,944 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 9,086 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 8,889 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 9,296 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 9,206 works

🧭 Topics (1): [🧮 Reasoning](https://md.research.elamur.ai/free/en/topics/reasoning/) · 510 works

## Cite

Cite as: Tae-Gil Noh, Sebastian Padó, Vered Shwartz, Ido Dagan, Vivi Nastase, Kathrin Eichler, Lili Kotlerman and Meni Adler. 2015. Multi-Level Alignments As An Extensible Representation Basis for Textual Entailment Algorithms. Proceedings of the Fourth Joint Conference on Lexical and Computational Semantics.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{noh2015multi,
  title = {{Multi-Level Alignments As An Extensible Representation Basis for Textual Entailment Algorithms}},
  author = {Tae-Gil Noh, Sebastian Padó, Vered Shwartz, Ido Dagan, Vivi Nastase, Kathrin Eichler, Lili Kotlerman and Meni Adler},
  booktitle = {Proceedings of the Fourth Joint Conference on Lexical and Computational Semantics},
  year = {2015},
  doi = {10.18653/v1/s15-1022},
  url = {https://doi.org/10.18653/v1/s15-1022}
}
```

- [← previous The DCU Discourse Parser for Connective, Argument Identification and Explicit Sense Classification](https://md.research.elamur.ai/free/en/papers/wang-2015-dcu-discourse-parser-connective-argument/)
- [next → Spectral Semi-Supervised Discourse Relation Classification](https://md.research.elamur.ai/free/en/papers/fisher-2015-spectral-semi-supervised-discourse-relation/)
- [↑ back to papers of 2015](https://md.research.elamur.ai/free/en/papers/2015/page/3/)

## References

21 references, 6 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Colin Bannard and Chris Callison-Burch. 2005. [Paraphrasing with bilingual parallel corpora](https://md.research.elamur.ai/free/en/papers/bannard-2005-paraphrasing-bilingual-parallel-corpora/). In Proceedings of the 43rd Annual Meeting of the Association for Computational Linguistics, pages 597–604, Ann Arbor, MI.
2. Jonathan Berant, Ido Dagan, and Jacob Goldberger. 2012. Learning entailment relations by global graph structure optimization. Computational Linguistics, 38(1):73– 111. [↗](https://doi.org/10.1162/coli_a_00085)
3. Ido Dagan and Oren Glickman. 2004. Probabilistic textual entailment: Generic applied modeling of language variability. In PASCAL workshop on Learning Methods for Text Understanding and Mining, Grenoble, France. [find](https://scholar.google.com/scholar?q=Ido+Dagan+and+Oren+Glickman%2E+2004%2E+Probabilistic+textual+entailment%3A+Generic+applied+modeling+of+language+variability%2E+In+PASCAL+workshop+on+Learning+Methods+for+Text+Understanding+and+Mining%2C+Grenobl)
4. Ido Dagan, Dan Roth, Mark Sammons, and Fabio Massimo Zanzotto. 2013. Recognizing Textual Entailment: Models and Applications. Number 17 in Synthesis Lectures on Human Language Technologies. Morgan & Claypool. [↗](https://doi.org/10.2200/s00509ed1v01y201305hlt023)
5. Richard Eckart de Castilho and Iryna Gurevych. 2014. [A broad-coverage collection of portable NLP components for building shareable analysis pipelines](https://md.research.elamur.ai/free/en/papers/castilho-2014-broad-coverage-collection-portable-nlp/). In Proceedings of the Workshop on Open Infrastructures and Analysis Frameworks for HLT at COLING 2014, pages 1–11, Dublin, Ireland, August.
6. Michael Denkowski and Alon Lavie. 2014. [Meteor universal: Language specific translation evaluation for any target language](https://md.research.elamur.ai/free/en/papers/denkowski-2014-meteor-universal-language-specific-translation/). In Proceedings of the Ninth Workshop on Statistical Machine Translation, pages 376–380, Baltimore, MD.
7. Georgiana Dinu and Rui Wang. 2009. Inference rules and their application to recognizing textual entailment. In Proceedings of the 12th Conference of the European Chapter of the ACL, pages 211–219, Athens, Greece. [find](https://scholar.google.com/scholar?q=Georgiana+Dinu+and+Rui+Wang%2E+2009%2E+Inference+rules+and+their+application+to+recognizing+textual+entailment%2E+In+Proceedings+of+the+12th+Conference+of+the+European+Chapter+of+the+ACL%2C+pages+211%E2%80%93219%2C+Ath)
8. Kathrin Eichler, Aleksandra Gabryszak, and Günter Neumann. 2014. An analysis of textual inference in German customer emails. In Proceedings of the Third Joint Conference on Lexical and Computational Semantics, pages 69–74, Dublin, Ireland. [find](https://scholar.google.com/scholar?q=Kathrin+Eichler%2C+Aleksandra+Gabryszak%2C+and+Gu%CC%88nter+Neumann%2E+2014%2E+An+analysis+of+textual+inference+in+German+customer+emails%2E+In+Proceedings+of+the+Third+Joint+Conference+on+Lexical+and+Computational+)
9. David Ferrucci and Adam Lally. 2004. UIMA: An architectural approach to unstructured information processing in the corporate research environment. Natural Language Engineering, 10(3–4):327–348. [↗](https://doi.org/10.1017/s1351324904003523)
10. Danilo Giampiccolo, Bernardo Magnini, Ido Dagan, and Bill Dolan. 2007. The third PASCAL recognising textual entailment challenge. In Proceedings of the ACL-PASCAL Workshop on Textual Entailment and Paraphrasing, pages 1–9, Prague, Czech Republic. [find](https://scholar.google.com/scholar?q=Danilo+Giampiccolo%2C+Bernardo+Magnini%2C+Ido+Dagan%2C+and+Bill+Dolan%2E+2007%2E+The+third+PASCAL+recognising+textual+entailment+challenge%2E+In+Proceedings+of+the+ACL%2DPASCAL+Workshop+on+Textual+Entailment+and+Pa)
11. Sanda Harabagiu and Andrew Hickl. 2006. [Methods for using textual entailment in open-domain question answering](https://md.research.elamur.ai/free/en/papers/harabagiu-2006-methods-textual-entailment-open-domain/). In Proceedings of the 21st International Conference on Computational Linguistics and 44th Annual Meeting of the Association for Computational Linguistics, pages 905–912, Sydney, Australia.
12. Lili Kotlerman, Ido Dagan, Bernardo Magnini, and Luisa Bentivogli. 2015. Textual entailment graphs. Natural Language Engineering. [↗](https://doi.org/10.1017/s1351324915000108)
13. Milen Kouylekov and Matteo Negri. 2010. [An open-source package for recognizing textual entailment](https://md.research.elamur.ai/free/en/papers/kouylekov-2010-open-source-package-recognizing-textual/). In Proceedings of the ACL 2010 System Demonstrations, pages 42–47, Uppsala, Sweden.
14. Bill MacCartney, Trond Grenager, Marie-Catherine de Marneffe, Daniel Cer, and Christopher D Manning. 2006. Learning to recognize features of valid textual entailments. In Proceedings of the Human Language Technology Conference of the NAACL, pages 41–48, New York City, USA. [find](https://scholar.google.com/scholar?q=Bill+MacCartney%2C+Trond+Grenager%2C+Marie%2DCatherine+de+Marneffe%2C+Daniel+Cer%2C+and+Christopher+D+Manning%2E+2006%2E+Learning+to+recognize+features+of+valid+textual+entailments%2E+In+Proceedings+of+the+Human+Lang)
15. Bill MacCartney, Michel Galley, and Christopher D Manning. 2008. A phrase-based alignment model for natural language inference. In Proceedings of the Conference on Empirical Methods in Natural Language Processing, pages 802–811, Honolulu, Hawaii. [find](https://scholar.google.com/scholar?q=Bill+MacCartney%2C+Michel+Galley%2C+and+Christopher+D+Manning%2E+2008%2E+A+phrase%2Dbased+alignment+model+for+natural+language+inference%2E+In+Proceedings+of+the+Conference+on+Empirical+Methods+in+Natural+Languag)
16. Bernardo Magnini, Roberto Zanoli, Ido Dagan, Kathrin Eichler, Günter Neumann, Tae-Gil Noh, Sebastian Padó, Asher Stern, and Omer Levy. 2014. The EX-CITEMENT open platform for textual inferences. In Proceedings of the ACL 2014 System Demonstrations, pages 43–48, Baltimore, MD. [find](https://scholar.google.com/scholar?q=Bernardo+Magnini%2C+Roberto+Zanoli%2C+Ido+Dagan%2C+Kathrin+Eichler%2C+Gu%CC%88nter+Neumann%2C+Tae%2DGil+Noh%2C+Sebastian+Pado%CC%81%2C+Asher+Stern%2C+and+Omer+Levy%2E+2014%2E+The+EX%2DCITEMENT+open+platform+for+textual+inferences%2E+In+)
17. Rodney D Nielsen, Wayne Ward, and James H Martin. 2009. Recognizing entailment in intelligent tutoring systems. Journal of Natural Language Engineering, 15(4):479–501. [↗](https://doi.org/10.1017/s135132490999012x)
18. Sebastian Padó, Tae-Gil Noh, Asher Stern, Rui Wang, and Roberto Zanoli. 2015. Design and realization of a modular architecture for textual entailment. Journal of Natural Language Engineering, 21(2):167–200. [find](https://scholar.google.com/scholar?q=Sebastian+Pado%CC%81%2C+Tae%2DGil+Noh%2C+Asher+Stern%2C+Rui+Wang%2C+and+Roberto+Zanoli%2E+2015%2E+Design+and+realization+of+a+modular+architecture+for+textual+entailment%2E+Journal+of+Natural+Language+Engineering%2C+21%282%29%3A1)
19. Asher Stern and Ido Dagan. 2012. [BIUTEE: A modular open-source system for recognizing textual entailment](https://md.research.elamur.ai/free/en/papers/stern-2012-biutee-modular-open-source-system/). In Proceedings of the ACL 2012 System Demonstrations, pages 73–78, Jeju Island, Korea.
20. Rui Wang and Yi Zhang. 2009. Recognizing textual relatedness with predicate-argument structures. In Proceedings of the Conference on Empirical Methods in Natural Language Processing, pages 784–792, Singapore. [find](https://scholar.google.com/scholar?q=Rui+Wang+and+Yi+Zhang%2E+2009%2E+Recognizing+textual+relatedness+with+predicate%2Dargument+structures%2E+In+Proceedings+of+the+Conference+on+Empirical+Methods+in+Natural+Language+Processing%2C+pages+784%E2%80%93792%2C+Si)
21. Britta Zeller, Jan Šnajder, and Sebastian Padó. 2013. DErivBase: Inducing and evaluating a derivational morphology resource for German. In Proceedings of ACL, pages 1201–1211, Sofia, Bulgaria. [find](https://scholar.google.com/scholar?q=Britta+Zeller%2C+Jan+S%CC%8Cnajder%2C+and+Sebastian+Pado%CC%81%2E+2013%2E+DErivBase%3A+Inducing+and+evaluating+a+derivational+morphology+resource+for+German%2E+In+Proceedings+of+ACL%2C+pages+1201%E2%80%931211%2C+Sofia%2C+Bulgaria%2E)

*Updated 24 October 2026*