--- title: "TALP system for the English lexical sample task — Escudero 2004" description: "The paper describes TALP, the system that Escudero, Màrquez and Rigau entered in the… 1.3 recall. 4 references, cited by 3 papers here." authors: ["Gerard Escudero", "Lluis Màrquez", "German Rigau"] year: 2004 venue: "Proceedings of SENSEVAL-3, the Third International Workshop on the Evaluation of Systems for the Semantic Analysis of Text" source: "https://aclanthology.org/W04-0828/" licence: "CC BY-NC-SA 3.0 (ACL Anthology, before 2016)" licence_url: "https://creativecommons.org/licenses/by-nc-sa/3.0/" summary_licence: "Elamur Cosmic License" canonical: "https://research.elamur.ai/free/en/papers/escudero-2004-talp-system-english-lexical-sample/" --- [Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/escudero-2004-talp-system-english-lexical-sample/) [home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2004](https://md.research.elamur.ai/free/en/papers/2004/page/2/) / TALP system for the English lexical sample task # TALP system for the English lexical sample task **Authors:** Gerard Escudero, [Lluis Màrquez](https://md.research.elamur.ai/free/en/authors/lluis-marquez/) and [German Rigau](https://md.research.elamur.ai/free/en/authors/german-rigau/) · **Year:** 2004 · **Venue:** [Proceedings of SENSEVAL-3, the Third International Workshop on the Evaluation of Systems for the Semantic Analysis of Text](https://md.research.elamur.ai/free/en/venues/senseval-3-the-international-workshop-on-the-evaluation-of-systems-for-the/) ([SENSEVAL-3, the International Workshop on the Evaluation of Systems for the Semantic Analysis of Text 2004](https://md.research.elamur.ai/free/en/venues/senseval-3-the-international-workshop-on-the-evaluation-of-systems-for-the/2004/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY-NC-SA 3.0 (ACL Anthology, before 2016)](https://creativecommons.org/licenses/by-nc-sa/3.0/) **Links:** [Publisher's page](https://aclanthology.org/W04-0828/) · [PDF](https://aclanthology.org/W04-0828.pdf) · [OpenAlex](https://openalex.org/W73807786) **Citations:** 12 citations (OpenAlex, 2026-10-14) · [3](https://md.research.elamur.ai/free/en/papers/escudero-2004-talp-system-english-lexical-sample/cited-by/) in our library, of the 149,439 papers whose bibliographies we read ## Summary The paper describes TALP, the system that Escudero, Màrquez and Rigau entered in the Senseval-3 English lexical sample task for word sense disambiguation (WSD). It is fully supervised and uses only the organisers' training data. External tools and ontologies are used only to build features. The learner is a linear SVM (SVM light). Two ways of turning the multiclass sense problem into binary SVM problems are compared: one-vs-all and constraint classification. Each example is described by a large, deliberately redundant feature set with four groups: local context, topical context, knowledge-based labels drawn from the Multilingual Central Repository (MCR), and syntactic relations from Minipar and Yarowsky's pattern extractor. Because every target word behaves differently, both the binarization scheme and the feature subset are chosen per word with 5-fold cross-validation. Feature selection adds about 3 points in cross-validation. On the official test set the full system is 1.3 recall points behind the best participant at both fine and coarse granularity. The work shows that per-word tuning of a rich feature space is competitive for WSD. ## Key points - **Learner setup:** SVM light with a linear kernel and C = 0.1, a value fixed from earlier Senseval-2 experiments. Non-linear kernels did not help. - **Feature groups:** - Local features come from a ±3-word window, with POS and lemmas from FreeLing. - Topical features use both a ±10-word window and the whole example. - Semantic features come from the MCR of the MEANING project: SUMO, domain labels, WordNet lexicographic files and the EuroWordNet Top Ontology. - Syntactic features come from Minipar and Yarowsky's dependency patterns. - **Per-word selection:** feature sets were first chosen per POS on Senseval-2. A single forward-addition / backward-deletion pass then refined them for each of the 57 Senseval-3 words. Running a second pass looked promising but was too costly. - **Cross-validation accuracy:** - Without feature selection: OVA 72.38%, CC 72.28%. - With feature selection: OVA 75.27%, CC 75.70%. - Full per-word combination (TALP): 76.02%. - None of these differences between variants is statistically significant. - **Test recall:** - Fine-grained: TALP 71.6, best system 72.9, median 65.1, MFS 55.2. - Coarse-grained: TALP 78.2, best 79.5. - Test accuracy fell more than 4 points below cross-validation. - The 'U' (unassignable) label was dropped from training. - Full model selection took about 12 hours for OVA and 5 days for CC. ## Interesting ideas - Multi-label training examples are encoded as pairwise ordering constraints (each correct sense must outrank each wrong one), which avoids assuming that every class can be cleanly separated from all the rest. - Model selection is layered: a default feature set is chosen per POS, and then each word is refined with one add-then-delete pass, which keeps search affordable and gives sensible starting points. - Semantic labels (SUMO, domain, lexicographic file) of all nouns in the context are weighted by how ambiguous each noun is and how frequent each label is in WordNet, and the strongest labels become features describing the topic of the passage. - The vague 'U' label was dropped from training rather than modelled as a regular class, because treating it as a class gave worse results. - The paper reports a gap of more than 4 points between cross-validation and test accuracy and treats it as a warning sign of possible overfitting from heavy per-item model selection. ## For Mova How these ideas could serve Mova, our small open language model — its files, stages and tests: - **I1** (frames): When a verb's frame or VerbNet class is legitimately ambiguous between several gold labels, ranking constraints could train the frame chooser without forcing one arbitrary "true" label. - **I2** (learn): Per-lemma repair rules or feature sets could inherit a per-POS default and be tuned locally only where held-out data shows a gain, keeping each choice explainable as "default plus these exceptions". - **I3** (world): The world builder could compute a weighted "dominant domain" profile of a text from its nouns' ontology labels and use it as an explicit, inspectable prior when choosing word senses or entity types. - **I4** (learn): When silver data from witness parsers includes "unsure" or "no agreement" markers, it may be better to exclude them from training than to learn them as a category. - **I5** (learn): Evaluation should always keep a held-out set untouched by any selection loop, and the selection-versus-held-out gap should be reported as a standard health metric. *Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.* 📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works 🧭 Topics (1): [English](https://md.research.elamur.ai/free/en/topics/english/) · 2,263 works ## Cite Cite as: Gerard Escudero, Lluis Màrquez and German Rigau. 2004. TALP system for the English lexical sample task. Proceedings of SENSEVAL-3, the Third International Workshop on the Evaluation of Systems for the Semantic Analysis of Text. The paper itself, not this page. BibTeX from the library's record of the work. ```bibtex @inproceedings{escudero2004talp, title = {{TALP system for the English lexical sample task}}, author = {Gerard Escudero, Lluis Màrquez and German Rigau}, booktitle = {Proceedings of SENSEVAL-3, the Third International Workshop on the Evaluation of Systems for the Semantic Analysis of Text}, year = {2004}, url = {https://aclanthology.org/W04-0828/} } ``` - [← previous Mining the Web for Discourse Markers](https://md.research.elamur.ai/free/en/papers/hutchinson-2004-mining-web-discourse-markers/) - [next → Application of the Double Metaphone Algorithm to Amharic Orthography](https://md.research.elamur.ai/free/en/papers/yacob-2004-application-double-metaphone-algorithm-amharic/) - [↑ back to papers of 2004](https://md.research.elamur.ai/free/en/papers/2004/page/2/) ## References 4 references, 0 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet. 1. J. Atserias, L. Villarejo, G. Rigau, E. Agirre, J. Carroll, B. Magnini, P. Vossen 2004. The MEAN-ING Multilingual Central Repository. In Proceedings of the Second International WordNet Conference. [find](https://scholar.google.com/scholar?q=J%2E+Atserias%2C+L%2E+Villarejo%2C+G%2E+Rigau%2C+E%2E+Agirre%2C+J%2E+Carroll%2C+B%2E+Magnini%2C+P%2E+Vossen+2004%2E+The+MEAN%2DING+Multilingual+Central+Repository%2E+In+Proceedings+of+the+Second+International+WordNet+Conference%2E) 2. N. Cristianini and J. Shawe-Taylor 2000. An Introduction to Support Vector Machines. Cambridge University Press. [find](https://scholar.google.com/scholar?q=N%2E+Cristianini+and+J%2E+Shawe%2DTaylor+2000%2E+An+Introduction+to+Support+Vector+Machines%2E+Cambridge+University+Press%2E) 3. T. Joachims 2002. Learning to Classify Text Using Support Vector Machines. Dissertation, Kluwer. [↗](https://doi.org/10.1007/978-1-4615-0907-3) 4. S. Har-Peled and D. Roth and D. Zimak 2002. Constraint Classification for Multiclass Classification and Ranking. In Proceedings of the 15th Workshop on Neural Information Processing Systems. [find](https://scholar.google.com/scholar?q=S%2E+Har%2DPeled+and+D%2E+Roth+and+D%2E+Zimak+2002%2E+Constraint+Classification+for+Multiclass+Classification+and+Ranking%2E+In+Proceedings+of+the+15th+Workshop+on+Neural+Information+Processing+Systems%2E) *Updated 1 November 2026*