---
title: "The “Meaning” system on the English all-words task (2004)"
description: "The paper describes the \"Meaning\" system, built within the Meaning… 67.5% precision on Senseval-2. 7 references, cited by 2 papers here."
authors: ["Luís Villarejo", "Lluis Màrquez", "Eneko Agirre", "David Martínez", "Bernardo Magnini", "Carlo Strapparava", "Diana McCarthy", "Andrés Montoyo", "Armando Suárez"]
year: 2004
venue: "Proceedings of SENSEVAL-3, the Third International Workshop on the Evaluation of Systems for the Semantic Analysis of Text"
source: "https://aclanthology.org/W04-0861/"
licence: "CC BY-NC-SA 3.0 (ACL Anthology, before 2016)"
licence_url: "https://creativecommons.org/licenses/by-nc-sa/3.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/villarejo-2004-meaning-system-english-all-words/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/villarejo-2004-meaning-system-english-all-words/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2004](https://md.research.elamur.ai/free/en/papers/2004/page/2/) / The “Meaning” system on the English all-words task

# The “Meaning” system on the English all-words task

**Authors:** Luís Villarejo, [Lluis Màrquez](https://md.research.elamur.ai/free/en/authors/lluis-marquez/), [Eneko Agirre](https://md.research.elamur.ai/free/en/authors/eneko-agirre/), David Martínez, [Bernardo Magnini](https://md.research.elamur.ai/free/en/authors/bernardo-magnini/), [Carlo Strapparava](https://md.research.elamur.ai/free/en/authors/carlo-strapparava/), [Diana McCarthy](https://md.research.elamur.ai/free/en/authors/diana-mccarthy/), Andrés Montoyo and Armando Suárez · **Year:** 2004 · **Venue:** [Proceedings of SENSEVAL-3, the Third International Workshop on the Evaluation of Systems for the Semantic Analysis of Text](https://md.research.elamur.ai/free/en/venues/senseval-3-the-international-workshop-on-the-evaluation-of-systems-for-the/) ([SENSEVAL-3, the International Workshop on the Evaluation of Systems for the Semantic Analysis of Text 2004](https://md.research.elamur.ai/free/en/venues/senseval-3-the-international-workshop-on-the-evaluation-of-systems-for-the/2004/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY-NC-SA 3.0 (ACL Anthology, before 2016)](https://creativecommons.org/licenses/by-nc-sa/3.0/)

**Links:** [Publisher's page](https://aclanthology.org/W04-0861/) · [PDF](https://aclanthology.org/W04-0861.pdf) · [OpenAlex](https://openalex.org/W337717459)

**Citations:** 4 citations (OpenAlex, 2026-10-14) · 2 in our library, of the 149,439 papers whose bibliographies we read

## Summary
The paper describes the "Meaning" system, built within the Meaning European project, for the Senseval-3 English all-words word sense disambiguation (WSD) task. It joins seven supervised classifiers with five knowledge-based modules. The supervised learners are SVM (two variants), AdaBoost, cosVSM, ME, NB and DL. They were trained only on SemCor-1.6 and share one rich feature set: local patterns, wider-context words, syntactic relations, and semantic features from the Multilingual Central Repository (MCR). The knowledge-based modules are two Domain Driven Disambiguation variants, DOM, autoPS and MFS. They cover words that have too few training examples, and they run as a cascade ordered by precision. All combination settings were tuned on the Senseval-2 all-words data. The best tuned ensemble uses weighted voting over three classifiers ranked per frequency band, and it reached 67.5% precision on Senseval-2. On the Senseval-3 test set the system dropped to about 62.5% precision, which the authors attribute mostly to differences between the training and test corpora. The work shows both the practical value and the limits of combining many WSD modules when training data is sparse.

## Key points
- A separate classifier was trained for each polysemous word with at least 10 SemCor-1.6 examples, giving 2,440 learning problems. WordNet-1.6 senses were then mapped automatically to WordNet-1.7 and 1.7.1.
- On covered polysemous Senseval-2 words, the individual supervised modules fell between 53.4% (DL) and 59.5% (SVM), with AdaBoost close behind at 59.1%.
- Among the unsupervised modules, the precision-oriented DDD reached 61.9% precision at only 48.8% coverage. The coverage-oriented DDD variant reached 50.2% at 99.6% coverage. autoPS (45.2%) was more precise than WordNet MFS (32.5%).
- On Senseval-2, the baselines scored 64.8%, 67.0% and 67.4%, and weighted voting with three classifiers scored 67.5% at 98% coverage. That result would have ranked second in the Senseval-2 all-words task.
- On Senseval-3, Meaning-c (cascade baseline) scored 61.1% precision and 61.0% recall. Meaning-wv (weighted voting) scored 62.5% precision and 62.3% recall, so the vote was more robust on unseen data.

## Interesting ideas
- Back-off modules are applied in a fixed order of measured precision, and each one may abstain so the next one decides, instead of mixing strong and weak modules in one vote. Mixing them hurt here, because the weak modules were far worse than the best one.
- The same method is offered in two tunings, one maximizing precision and one maximizing coverage, and they are stacked so the precise variant answers first and the broad variant fills the gaps.
- Classifiers are ranked separately for groups of words with similar amounts of training data, rather than by one global accuracy. This balances specialization against overfitting to a small tuning set.
- Pairwise agreement between systems is measured before ensembling, to estimate how independent their errors are and whether a vote can help at all.
- Two configurations that look equally good on the tuning set behaved differently on new data, and both lost about 5 points because the training and test corpora differ.

## For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

- **I1** (world): When choosing a WordNet sense or ontology node for an entity, Mova could try high-precision evidence first and fall back step by step, recording which tier made the decision so the choice stays explainable.
- **I2** (frames): Semantic role or frame assignment could pair a strict, high-precision rule set with a looser fallback, keeping the strict answers apart as more trustworthy.
- **I3** (learn): Witness parsers that build silver data could be weighted per frequency band of lemma or construction, because a parser that is best overall may not be best on rare items.
- **I4** (learn): Before trusting agreement among witness parsers as a silver label, Mova could check how often each pair agrees; witnesses that almost always agree add little independent evidence.
- **I5** (learn): Evaluations should keep a held-out set from a different domain alongside the tuning set, so that ties on the tuning set are broken by robustness rather than chance.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

📚 Shelves (8): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works · [💭 Psychology](https://md.research.elamur.ai/free/en/shelves/psychology/) · 1,552 works · [💭 Developmental and Educational Psychology](https://md.research.elamur.ai/free/en/shelves/psychology/developmental-and-educational-psychology/) · 499 works · [🧠 Neuroscience](https://md.research.elamur.ai/free/en/shelves/neuroscience/) · 165 works · [🧠 Cognitive Neuroscience](https://md.research.elamur.ai/free/en/shelves/neuroscience/cognitive-neuroscience/) · 164 works

🧭 Topics (1): [English](https://md.research.elamur.ai/free/en/topics/english/) · 2,263 works

## Cite

Cite as: Luís Villarejo, Lluis Màrquez, Eneko Agirre, David Martínez, Bernardo Magnini, Carlo Strapparava, Diana McCarthy, Andrés Montoyo and Armando Suárez. 2004. The “Meaning” system on the English all-words task. Proceedings of SENSEVAL-3, the Third International Workshop on the Evaluation of Systems for the Semantic Analysis of Text.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{villarejo2004meaning,
  title = {{The “Meaning” system on the English all-words task}},
  author = {Luís Villarejo, Lluis Màrquez, Eneko Agirre, David Martínez, Bernardo Magnini, Carlo Strapparava, Diana McCarthy, Andrés Montoyo and Armando Suárez},
  booktitle = {Proceedings of SENSEVAL-3, the Third International Workshop on the Evaluation of Systems for the Semantic Analysis of Text},
  year = {2004},
  url = {https://aclanthology.org/W04-0861/}
}
```

- [← previous Intentions, Implicatures and Processing of Complex Questions](https://md.research.elamur.ai/free/en/papers/harabagiu-2004-intentions-implicatures-processing-complex-questions/)
- [next → Abstracting a Dialog Act Tagset for Meeting Processing](https://md.research.elamur.ai/free/en/papers/belis-2004-abstracting-dialog-act-tagset-meeting/)
- [↑ back to papers of 2004](https://md.research.elamur.ai/free/en/papers/2004/page/2/)

## References

7 references, 2 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. J. Atserias, L. Villarejo, G. Rigau, E. Agirre, J. Carroll, B. Magnini, and P. Vossen. 2004. The Meaning multilingual central repository. In Proceedings of the Second International WordNet Conference. [find](https://scholar.google.com/scholar?q=J%2E+Atserias%2C+L%2E+Villarejo%2C+G%2E+Rigau%2C+E%2E+Agirre%2C+J%2E+Carroll%2C+B%2E+Magnini%2C+and+P%2E+Vossen%2E+2004%2E+The+Meaning+multilingual+central+repository%2E+In+Proceedings+of+the+Second+International+WordNet+Conference%2E)
2. G. Escudero, L. Màrquez, and G. Rigau. 2004. [TALP system for the english lexical sample task](https://md.research.elamur.ai/free/en/papers/escudero-2004-talp-system-english-lexical-sample/). In Proceedings of the Senseval-3 ACL Workshop, Barcelona, Spain.
3. T. Joachims. 1999. Making large–scale SVM learning practical. In B. Schölkopf, C. J. C. Burges, and A. J. Smola, editors, Advances in Kernel Methods — Support Vector Learning, pages 169– 184. MIT Press, Cambridge, MA. [↗](https://doi.org/10.17877/de290r-14262)
4. B. Magnini, C. Strapparava, G. Pezzulo, and A. Gliozzo. 2002. The role of domain information in word sense disambiguation. Natural Language Engineering, 8(4):359–373. [↗](https://doi.org/10.1017/s1351324902003029)
5. D. McCarthy, R. Koeling, J. Weeds, and J. Carroll. 2004. Using automatically acquired predominant senses for word sense disambiguation. In Proceedings of the Senseval-3 ACL Workshop, Barcelona, Spain. [find](https://scholar.google.com/scholar?q=D%2E+McCarthy%2C+R%2E+Koeling%2C+J%2E+Weeds%2C+and+J%2E+Carroll%2E+2004%2E+Using+automatically+acquired+predominant+senses+for+word+sense+disambiguation%2E+In+Proceedings+of+the+Senseval%2D3+ACL+Workshop%2C+Barcelona%2C+Spain%2E)
6. R. Schapire and Y. Singer. 1999. Improved boosting algorithms using confidence–rated predictions. Machine Learning, 37(3):297–336. [↗](https://doi.org/10.1023/a:1007614523901)
7. David Yarowsky. 1994. [Decision lists for lexical ambiguity resolution: Application to accent restoration in Spanish and French](https://md.research.elamur.ai/free/en/papers/yarowsky-1994-decision-lists-lexical-ambiguity-resolution/). In Proceedings of the 32nd Annual Meeting of the Association for Computational Linguistics, pages 88–95, Las Cruces, NM. [↗](https://doi.org/10.48550/arxiv.cmp-lg/9406034)

*Updated 1 November 2026*