---
title: "FBK-TR — paper by Vo et al. (2014) — summary and ideas"
description: "FBK-TR is the system the authors entered in SemEval 2014 Task #1. 75.401 accuracy. 15 references, cited by 1 paper here."
authors: ["Ngoc Phuoc An Vo", "Octavian Popescu", "Tommaso Caselli"]
year: 2014
venue: "Proceedings of the 8th International Workshop on Semantic Evaluation (SemEval 2014)"
source: "https://aclanthology.org/S14-2047/"
doi: "10.3115/v1/s14-2047"
licence: "CC BY-NC-SA 3.0 (ACL Anthology, before 2016)"
licence_url: "https://creativecommons.org/licenses/by-nc-sa/3.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/vo-2014-fbk-tr-svm-semantic-relatedeness/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/vo-2014-fbk-tr-svm-semantic-relatedeness/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2014](https://md.research.elamur.ai/free/en/papers/2014/page/3/) / FBK-TR: SVM for Semantic Relatedeness and Corpus Patterns for RTE

# FBK-TR: SVM for Semantic Relatedeness and Corpus Patterns for RTE

**Authors:** [Ngoc Phuoc An Vo](https://md.research.elamur.ai/free/en/authors/ngoc-phuoc-an-vo/), [Octavian Popescu](https://md.research.elamur.ai/free/en/authors/octavian-popescu/) and [Tommaso Caselli](https://md.research.elamur.ai/free/en/authors/tommaso-caselli/) · **Year:** 2014 · **Venue:** [Proceedings of the 8th International Workshop on Semantic Evaluation (SemEval 2014)](https://md.research.elamur.ai/free/en/venues/semeval/) ([SemEval 2014](https://md.research.elamur.ai/free/en/venues/semeval/2014/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY-NC-SA 3.0 (ACL Anthology, before 2016)](https://creativecommons.org/licenses/by-nc-sa/3.0/)

**Links:** [Publisher's page](https://aclanthology.org/S14-2047/) · [PDF](https://aclanthology.org/S14-2047.pdf) · [DOI 10.3115/v1/s14-2047](https://doi.org/10.3115/v1/s14-2047) · [OpenAlex](https://openalex.org/W2170502136)

**Citations:** 9 citations (OpenAlex, 2026-10-14) · 1 in our library, of the 141,840 papers whose bibliographies we read

## Summary
FBK-TR is the system the authors entered in SemEval 2014 Task #1. The task scores sentence pairs in two ways: graded semantic relatedness, and a three-way entailment label (ENTAILMENT, CONTRADICTION, NEUTRAL). The system does not use compositional distributional models.

**Relatedness.** An SVM regression model (LIBSVM) combines several kinds of features:
- word-level scores from WordNet (Lin measure), Wikipedia Miner and LSA, computed on best-matched noun and verb lemma pairs;
- LDA topic distributions compared with cosine, KL and JS divergence;
- string overlap around the Longest Common Substring.

**Entailment.** A baseline first derives labels from the relatedness score. Rule-based patterns then override it. These Sense Discriminative Patterns (SDPs) are verb frames mined from parsed BNC and Annotated English Gigaword, with slot fillers generalised to SUMO classes. The rules check hypernym, antonym and negation relations between slot fillers.

**Results.** Both submitted components ranked 11th. After a parser-output bug was fixed, entailment accuracy rose by about ten points, to slightly above the task winner. This suggests that explicit, frame-based rules can compete with statistical systems on RTE.

## Key points
- **Relatedness:** the best run (run3) reached Pearson 0.70892 and Spearman 0.64430, ranking 11th of 17. The top system, ECNU, scored 0.82795 / 0.76892.
- **Entailment:** the submitted run scored 75.401 accuracy (11th of 18), against 84.575 for the winner, Illinois-LH. The relatedness-only baseline gave 64.080, and the bug-fixed version gave 85.082.
- **Lexical resources:** TreeTagger for POS tagging and lemmas, WordNet::Similarity (Lin), Wikipedia Miner, and LSA and LDA (MALLET, 20–500 topics) trained on BNC and Wikipedia. Levenshtein distance is the fallback for words missing from WordNet.
- **SDP construction:**
  - The corpus is limited to sentences with at most two finite verbs, parsed with the Stanford parser.
  - Only these relations are kept: subject, object, prep and comp.
  - Words are mapped to SUMO attributes.
  - Patterns are clustered using OntoNotes and CPA.
- **Training data use:** the only component built from the task training data is a list of quasi-synonym phrases. It comes from ENTAILMENT pairs with relatedness 4–5.

## Interesting ideas
- The system starts from a cheap statistical guess (a relatedness score mapped to ENTAILMENT or NEUTRAL) and lets precise symbolic patterns override it whenever they fire, so the rules only need high precision rather than full coverage.
- Every frame match is taken to imply that each of its participants exists (the "Existential Assumptions"). An "there is no X" sentence then contradicts any frame containing X, while a plain existential statement is entailed by it.
- Entailment direction is decided by comparing fillers in the same syntactic slot of the same verb pattern. A hyponym entails its hypernym; an unrelated filler gives NEUTRAL; an antonymous verb gives CONTRADICTION.
- Quasi-synonym phrases are harvested by aligning high-relatedness entailment pairs slot by slot and keeping the parts that differ.
- When building the pattern corpus, the authors dropped Gigaword sentences whose Stanford parse disagreed with the corpus's own annotation, keeping only sentences where both analyses agree.

## For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

- **I1** (answer): a reader could fall back to a similarity-based default only when no frame-level rule decides the answer, and log which layer produced the result for explainability.
- **I2** (world): when a frame is asserted, the world model could add existence facts for its participants, and check existential negations against those facts to detect contradictions.
- **I3** (frames): role-aligned comparison of two frames could use WordNet hypernymy per role to produce explainable entailment verdicts, instead of comparing whole sentences.
- **I4** (learn): paraphrase pairs that agree at the frame level could be diffed by role to induce substitution lists automatically, which would then feed the repair or equivalence rules.
- **I5** (learn): this supports the witness-agreement filter for silver treebanks, i.e. keeping only sentences where independent parsers concur before mining frames from them.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 9,085 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 8,888 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 9,296 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 9,206 works

🧭 Topics (1): [🧮 Reasoning](https://md.research.elamur.ai/free/en/topics/reasoning/) · 510 works

## Cite

Cite as: Ngoc Phuoc An Vo, Octavian Popescu and Tommaso Caselli. 2014. FBK-TR: SVM for Semantic Relatedeness and Corpus Patterns for RTE. Proceedings of the 8th International Workshop on Semantic Evaluation (SemEval 2014).

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{vo2014fbk,
  title = {{FBK-TR: SVM for Semantic Relatedeness and Corpus Patterns for RTE}},
  author = {Ngoc Phuoc An Vo, Octavian Popescu and Tommaso Caselli},
  booktitle = {Proceedings of the 8th International Workshop on Semantic Evaluation (SemEval 2014)},
  year = {2014},
  doi = {10.3115/v1/s14-2047},
  url = {https://doi.org/10.3115/v1/s14-2047}
}
```

- [← previous Developing an interlingual translation lexicon using WordNets and Grammatical Framework](https://md.research.elamur.ai/free/en/papers/virk-2014-developing-interlingual-translation-lexicon-wordnets/)
- [next → Interoperability of Dialogue Corpora through ISO 24617-2-based Querying](https://md.research.elamur.ai/free/en/papers/petukhova-2014-interoperability-dialogue-corpora-through-iso/)
- [↑ back to papers of 2014](https://md.research.elamur.ai/free/en/papers/2014/page/3/)

## References

15 references, 3 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. David M Blei, Andrew Y Ng, and Michael I Jordan. 2003. [Latent Dirichlet Allocation](https://md.research.elamur.ai/free/en/papers/blei-2003-latent-dirichlet-allocation/). The Journal of Machine Learning research, 3:993–1022.
2. Chih-Chung Chang and Chih-Jen Lin. 2011. LIB-SVM: A Library for Support Vector Machines. ACM Transactions on Intelligent Systems and Technology (TIST), 2(3):27. [find](https://scholar.google.com/scholar?q=Chih%2DChung+Chang+and+Chih%2DJen+Lin%2E+2011%2E+LIB%2DSVM%3A+A+Library+for+Support+Vector+Machines%2E+ACM+Transactions+on+Intelligent+Systems+and+Technology+%28TIST%29%2C+2%283%29%3A27%2E)
3. Patrick Hanks. 2008. Mapping meaning onto use: a Pattern Dictionary of English Verbs. In Proceedings of the AACL 2008. [find](https://scholar.google.com/scholar?q=Patrick+Hanks%2E+2008%2E+Mapping+meaning+onto+use%3A+a+Pattern+Dictionary+of+English+Verbs%2E+In+Proceedings+of+the+AACL+2008%2E)
4. Eduard Hovy, Mitchell Marcus, Martha Palmer, Lance Ramshaw, and Ralph Weischedel. 2006. OntoNotes: The 90% Solution. In Proceedings of the human language technology conference of the NAACL, Companion Volume: Short Papers, pages 57–60. [find](https://scholar.google.com/scholar?q=Eduard+Hovy%2C+Mitchell+Marcus%2C+Martha+Palmer%2C+Lance+Ramshaw%2C+and+Ralph+Weischedel%2E+2006%2E+OntoNotes%3A+The+90%25+Solution%2E+In+Proceedings+of+the+human+language+technology+conference+of+the+NAACL%2C+Companion+)
5. Thomas K Landauer, Peter W Foltz, and Darrell Laham. 1998. An Introduction to Latent Semantic Analysis. Discourse processes, 25(2-3):259–284. [↗](https://doi.org/10.1080/01638539809545028)
6. Vladimir I Levenshtein. 1966. Binary Codes Capable of Correcting Deletions, Insertions and Reversals. In Soviet Physics Doklady, volume 10, page 707. [find](https://scholar.google.com/scholar?q=Vladimir+I+Levenshtein%2E+1966%2E+Binary+Codes+Capable+of+Correcting+Deletions%2C+Insertions+and+Reversals%2E+In+Soviet+Physics+Doklady%2C+volume+10%2C+page+707%2E)
7. M Marelli, L Bentivogli, M Baroni, R Bernardi, S Menini, and R Zamparelli. 2014a. [Semeval-2014 Task 1: Evaluation of compositional distributional semantic models on full sentences through semantic relatedness and textual entailment](https://md.research.elamur.ai/free/en/papers/marelli-2014-semeval-2014-task-1-evaluation/). In Proceedings of SemEval 2014: International Workshop on Semantic Evaluation, August 23-24, 2014, Dublin, Ireland.
8. M Marelli, S Menini, M Baroni, L Bentivogli, R Bernardi, and R Zamparelli. 2014b. [A SICK cure for the evaluation of compositional distributional semantic models](https://md.research.elamur.ai/free/en/papers/marelli-2014-sick-cure-evaluation-compositional-distributional/). In Proceedings of LREC 2014, Reykjavik (Iceland): ELRA.
9. Andrew Kachites McCallum. 2002. MALLET: A Machine Learning for Language Toolkit. [find](https://scholar.google.com/scholar?q=Andrew+Kachites+McCallum%2E+2002%2E+MALLET%3A+A+Machine+Learning+for+Language+Toolkit%2E)
10. David Milne and Ian H Witten. 2013. An Open-Source Toolkit for Mining Wikipedia. Artificial Intelligence, 194:222–239. [↗](https://doi.org/10.1016/j.artint.2012.06.007)
11. Ian Niles and Adam Pease. 2003. Mapping Word-Net to the SUMO Ontology. In Proceedings of the IEEE International Knowledge Engineering Conference, pages 23–26. [find](https://scholar.google.com/scholar?q=Ian+Niles+and+Adam+Pease%2E+2003%2E+Mapping+Word%2DNet+to+the+SUMO+Ontology%2E+In+Proceedings+of+the+IEEE+International+Knowledge+Engineering+Conference%2C+pages+23%E2%80%9326%2E)
12. Ted Pedersen, Patwardhan Siddharth, and Michelizzi Jason. 2004. Wordnet::Similarity: Measuring the Relatedness of Concepts. In Proceedings of the HLT-NAACL 2004. [find](https://scholar.google.com/scholar?q=Ted+Pedersen%2C+Patwardhan+Siddharth%2C+and+Michelizzi+Jason%2E+2004%2E+Wordnet%3A%3ASimilarity%3A+Measuring+the+Relatedness+of+Concepts%2E+In+Proceedings+of+the+HLT%2DNAACL+2004%2E)
13. Octavian Popescu, Elena Cabrio, and Bernardo Magnini. 2011. Textual Entailment Using Chain Clarifying Relationships. In Proceedings of the IJ-CAI Workshop Learning by Reasoning and its Applications in Intelligent Question-Answering. [find](https://scholar.google.com/scholar?q=Octavian+Popescu%2C+Elena+Cabrio%2C+and+Bernardo+Magnini%2E+2011%2E+Textual+Entailment+Using+Chain+Clarifying+Relationships%2E+In+Proceedings+of+the+IJ%2DCAI+Workshop+Learning+by+Reasoning+and+its+Applications+in)
14. Octavian Popescu. 2013. Learning Corpus Patterns Using Finite State Automata. In Proceedings of the ICSC 2013. [find](https://scholar.google.com/scholar?q=Octavian+Popescu%2E+2013%2E+Learning+Corpus+Patterns+Using+Finite+State+Automata%2E+In+Proceedings+of+the+ICSC+2013%2E)
15. Helmut Schmid. 1994. Probabilistic Part-of-Sspeech Tagging Using Decision Trees. In Proceedings of international conference on new methods in language processing, volume 12, pages 44–49. Manchester, UK. [find](https://scholar.google.com/scholar?q=Helmut+Schmid%2E+1994%2E+Probabilistic+Part%2Dof%2DSspeech+Tagging+Using+Decision+Trees%2E+In+Proceedings+of+international+conference+on+new+methods+in+language+processing%2C+volume+12%2C+pages+44%E2%80%9349%2E+Manchester%2C+U)

*Updated 24 October 2026*