---
title: "Shallow Semantic Parsing using Support Vector — Pradhan 2004"
description: "The paper addresses shallow semantic parsing, also called semantic role labeling: for each verb in a sentence, find the constituents that act as its…"
authors: ["Sameer S. Pradhan", "Wayne H. Ward", "Kadri Hacioglu", "James H. Martin", "Dan Jurafsky"]
year: 2004
venue: "Proceedings of the Human Language Technology Conference of the North American Chapter of the Association for Computational Linguistics: HLT-NAACL 2004"
source: "https://aclanthology.org/N04-1030/"
licence: "CC BY-NC-SA 3.0 (ACL Anthology, before 2016)"
licence_url: "https://creativecommons.org/licenses/by-nc-sa/3.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/pradhan-2004-shallow-semantic-parsing-support-vector/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/pradhan-2004-shallow-semantic-parsing-support-vector/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2004](https://md.research.elamur.ai/free/en/papers/2004/) / Shallow Semantic Parsing using Support Vector Machines

# Shallow Semantic Parsing using Support Vector Machines

**Authors:** [Sameer S. Pradhan](https://md.research.elamur.ai/free/en/authors/sameer-s-pradhan/), Wayne H. Ward, Kadri Hacioglu, [James H. Martin](https://md.research.elamur.ai/free/en/authors/james-h-martin/), [Dan Jurafsky](https://md.research.elamur.ai/free/en/authors/dan-jurafsky/) · **Year:** 2004 · **Venue:** Proceedings of the Human Language Technology Conference of the North American Chapter of the Association for Computational Linguistics: HLT-NAACL 2004 · **Licence:** [CC BY-NC-SA 3.0 (ACL Anthology, before 2016)](https://creativecommons.org/licenses/by-nc-sa/3.0/)

**Links:** [Publisher's page](https://aclanthology.org/N04-1030/) · [PDF](https://aclanthology.org/N04-1030.pdf) · [OpenAlex](https://openalex.org/W2885050925)

**Citations:** 367 citations (openalex-snapshot · 2026-10-14) · [130](#cited-by) in our library, of the 140884 papers whose bibliographies we read

## Summary
The paper addresses shallow semantic parsing, also called semantic role labeling: for each verb in a sentence, find the constituents that act as its arguments and assign them PropBank labels such as ARG0–ARG5 and ARGM adjuncts. The authors start from the Gildea and Jurafsky (2002) framework. They replace its statistical classifier with Support Vector Machines and then extend the feature set. Some new features come from earlier work (named entities, head word POS), and others are new (verb clusters, partial path, verb sense, prepositional-phrase head words). Training and testing use the WSJ-based PropBank with the standard section split, on both hand-corrected TreeBank parses and Charniak parser output. On the same features, SVMs clearly beat a decision tree and an interpolated probabilistic model. The full system improves on earlier published results for argument identification and labeling. The authors also annotated a small AQUAINT test set to measure how well the system carries over to other newswire. Performance drops substantially there, and they trace part of the drop to lower coverage of sparse lexical and path features.

## Key points
- **Data:** PropBank (July 2002 release), trained on WSJ Sections 02–21 and tested on Section 23.
  - Training: about 51,000 sentences and 132,000 arguments.
  - Feb 2004 release: about 85,000 training sentences and 250,000 arguments.
- **Setup:** one-vs-all SVMs built with TinySVM and YamCha, using a degree-2 polynomial kernel and C=1.
- **Classifier comparison:** with identical features, argument classification accuracy is 88% for the SVM, 79% for the decision tree of Surdeanu et al. (2003), and 77% for Gildea and Palmer (2002).
- **Baseline on hand-corrected parses (all arguments):**
  - Identification F1: 90.4.
  - Classification accuracy: 87.9%.
  - Identification + classification F1: 80.8, rising to 81.6 after removing overlapping arguments.
- **Final system, identification + classification:** F1 87 on hand-corrected parses and 79 on automatic parses.
- **AQUAINT test set** (400 PropBank-annotated sentences from NYT, AP and Xinhua): identification + classification F1 falls to 63.3, even though about 99% of predicates were seen in training.

## Interesting ideas
- **Two-stage training to handle NULL nodes.** About 80% of parse nodes are not arguments. The authors train one cheap NULL vs NON-NULL filter on all data and train the remaining one-vs-all classifiers only on the roughly 20% of nodes that pass the filter. This saves a lot of training time without hurting accuracy, and the same trick could help any heavily imbalanced candidate-classification task.
- **Enforcing output constraints with calibrated scores.** Raw SVM scores are converted to probabilities with Platt's sigmoid. When predicted arguments overlap, only the most confident one is kept. This is a simple, general way to apply structural constraints on top of independent local decisions.
- **Generalizing sparse features.** Two features trade detail for coverage:
  - "Partial path" keeps only the tree path up to the lowest common ancestor of the constituent and the predicate.
  - Verb clusters (64 classes) let the model handle predicates it has rarely or never seen.
  This kind of back-off is useful wherever rich features are too sparse.
- **Feature coverage as a diagnostic for domain shift.** The authors compare how often key feature combinations seen in training reappear in the in-domain and out-of-domain test sets. This gives a concrete, interpretable explanation for cross-domain performance loss and is easy to reproduce for other corpora.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

Fields: [Natural language processing](https://md.research.elamur.ai/free/en/shelves/nlp/)

Topics: [Semantic role labeling](https://md.research.elamur.ai/free/en/topics/semantic-role-labeling/)

## Cited by

Cited by

130 papers of our library cite it (of the 140884 whose bibliographies we read); 15 of them have a page here, 115 not yet.

- [SPICE: Semantic Propositional Image Caption Evaluation](https://md.research.elamur.ai/free/en/papers/anderson-2016-spice-semantic-propositional-image-caption/)
- [SemEval-2007 Task 19: Frame Semantic Structure Extraction](https://md.research.elamur.ai/free/en/papers/baker-2007-semeval-2007-task-19-frame/)
- [Natural Language Processing (almost) from Scratch](https://md.research.elamur.ai/free/en/papers/collobert-2011-natural-language-processing-almost-scratch/)
- [Frame-Semantic Parsing](https://md.research.elamur.ai/free/en/papers/das-2014-frame-semantic-parsing/)
- [MEANT: An inexpensive, high-accuracy, semi-automatic metric for evaluating translation utility based on semantic roles](https://md.research.elamur.ai/free/en/papers/lo-2011-meant-inexpensive-high-accuracy-semi/)
- [Exploiting Semantic Role Labeling, WordNet and Wikipedia for Coreference Resolution](https://md.research.elamur.ai/free/en/papers/ponzetto-2006-exploiting-semantic-role-labeling-wordnet/)
- [Semantic Role Labeling for Coreference Resolution](https://md.research.elamur.ai/free/en/papers/ponzetto-2006-semantic-role-labeling-coreference-resolution/)
- [Towards Robust Semantic Role Labeling](https://md.research.elamur.ai/free/en/papers/pradhan-2008-robust-semantic-role-labeling/)
- [The Importance of Syntactic Parsing and Inference in Semantic Role Labeling](https://md.research.elamur.ai/free/en/papers/punyakanok-2008-importance-syntactic-parsing-inference-semantic/)
- [Coreference Resolution with World Knowledge](https://md.research.elamur.ai/free/en/papers/rahman-2011-coreference-resolution-world-knowledge/)
- [Using Semantic Roles to Improve Question Answering](https://md.research.elamur.ai/free/en/papers/shen-2007-semantic-roles-improve-question-answering/)
- [Boosting Transition-based AMR Parsing with Refined Actions and Auxiliary Analyzers](https://md.research.elamur.ai/free/en/papers/wang-2015-boosting-transition-based-amr-parsing/)
- [CAMR at SemEval-2016 Task 8: An Extended Transition-based AMR Parser](https://md.research.elamur.ai/free/en/papers/wang-2016-camr-semeval-2016-task-8/)
- [Not an Interlingua, But Close: Comparison of English AMRs to Chinese and Czech](https://md.research.elamur.ai/free/en/papers/xue-2014-not-interlingua-but-close-comparison/)
- [Semantic Dependency Parsing of NomBank and PropBank: An Efficient Integrated Approach via a Large-scale Feature Selection](https://md.research.elamur.ai/free/en/papers/zhao-2009-semantic-dependency-parsing-nombank-propbank/)

## Cite

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{pradhan2004shallow,
  title = {{Shallow Semantic Parsing using Support Vector Machines}},
  author = {Pradhan, Sameer S. and Ward, Wayne H. and Hacioglu, Kadri and Martin, James H. and Jurafsky, Dan},
  booktitle = {Proceedings of the Human Language Technology Conference of the North American Chapter of the Association for Computational Linguistics: HLT-NAACL 2004},
  year = {2004},
  url = {https://aclanthology.org/N04-1030/}
}
```

- [← previous Robust VPE Detection using Automatically Parsed Text](https://md.research.elamur.ai/free/en/papers/aclbulk-2004-nielsen-robust-vpe-detection-automatically-parsed/)
- [next → SMOR: A German Computational Morphology Covering Derivation, Composition and Inflection](https://md.research.elamur.ai/free/en/papers/schmid-2004-smor-german-computational-morphology-covering/)
- [↑ back to papers of 2004](https://md.research.elamur.ai/free/en/papers/2004/)

## References

25 references, 7 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Daniel M. Bikel, Richard Schwartz, and Ralph M. Weischedel. 1999. An algorithm that learns what’s in a name. Machine Learning, 34:211– 231. [find](https://scholar.google.com/scholar?q=Daniel+M%2E+Bikel%2C+Richard+Schwartz%2C+and+Ralph+M%2E+Weischedel%2E+1999%2E+An+algorithm+that+learns+what%E2%80%99s+in+a+name%2E+Machine+Learning%2C+34%3A211%E2%80%93+231%2E)
2. Don Blaheta and Eugene Charniak. 2000. Assigning function tags to parsed text. In NAACL, pages 234–240. [find](https://scholar.google.com/scholar?q=Don+Blaheta+and+Eugene+Charniak%2E+2000%2E+Assigning+function+tags+to+parsed+text%2E+In+NAACL%2C+pages+234%E2%80%93240%2E)
3. Eugene Chaniak. 2001. Immediate-head parsing for language models. In ACL-01. [find](https://scholar.google.com/scholar?q=Eugene+Chaniak%2E+2001%2E+Immediate%2Dhead+parsing+for+language+models%2E+In+ACL%2D01%2E) [our card](https://md.research.elamur.ai/free/en/papers/charniak-2001-immediate-head-parsing-language-models/)
4. John Chen and Owen Rambow. 2003. Use of deep linguistics features for the recognition and labeling of semantic arguments. EMNLP-03. [find](https://scholar.google.com/scholar?q=John+Chen+and+Owen+Rambow%2E+2003%2E+Use+of+deep+linguistics+features+for+the+recognition+and+labeling+of+semantic+arguments%2E+EMNLP%2D03%2E)
5. Michael John Collins. 1999. Head-driven Statistical Models for Natural Language Parsing. Ph.D. thesis, University of Pennsylvania, Philadelphia. [find](https://scholar.google.com/scholar?q=Michael+John+Collins%2E+1999%2E+Head%2Ddriven+Statistical+Models+for+Natural+Language+Parsing%2E+Ph%2ED%2E+thesis%2C+University+of+Pennsylvania%2C+Philadelphia%2E)
6. K. Daniel, Y. Schabes, M. Zaidel, and D. Egedi. 1992. A freely available wide coverage morphological analyzer for English. In COLING-92. [find](https://scholar.google.com/scholar?q=K%2E+Daniel%2C+Y%2E+Schabes%2C+M%2E+Zaidel%2C+and+D%2E+Egedi%2E+1992%2E+A+freely+available+wide+coverage+morphological+analyzer+for+English%2E+In+COLING%2D92%2E)
7. Michael Fleischman and Eduard Hovy. 2003. A maximum entropy approach to framenet tagging. In HLT-03. [find](https://scholar.google.com/scholar?q=Michael+Fleischman+and+Eduard+Hovy%2E+2003%2E+A+maximum+entropy+approach+to+framenet+tagging%2E+In+HLT%2D03%2E)
8. Dan Gildea and Julia Hockenmaier. 2003. Identifying semantic roles using combinatory categorial grammar. In EMNLP-03. [find](https://scholar.google.com/scholar?q=Dan+Gildea+and+Julia+Hockenmaier%2E+2003%2E+Identifying+semantic+roles+using+combinatory+categorial+grammar%2E+In+EMNLP%2D03%2E)
9. Daniel Gildea and Daniel Jurafsky. 2000. Automatic labeling of semantic roles. In ACL-00, pages 512–520. [find](https://scholar.google.com/scholar?q=Daniel+Gildea+and+Daniel+Jurafsky%2E+2000%2E+Automatic+labeling+of+semantic+roles%2E+In+ACL%2D00%2C+pages+512%E2%80%93520%2E) [our card](https://md.research.elamur.ai/free/en/papers/gildea-2002-automatic-labeling-semantic-roles/)
10. Daniel Gildea and Daniel Jurafsky. 2002. Automatic labeling of semantic roles. Computational Linguistics, 28(3):245–288. [find](https://scholar.google.com/scholar?q=Daniel+Gildea+and+Daniel+Jurafsky%2E+2002%2E+Automatic+labeling+of+semantic+roles%2E+Computational+Linguistics%2C+28%283%29%3A245%E2%80%93288%2E) [our card](https://md.research.elamur.ai/free/en/papers/gildea-2002-automatic-labeling-semantic-roles/)
11. Daniel Gildea and Martha Palmer. 2002. The necessity of syntactic parsing for predicate argument recognition. In ACL-02. [find](https://scholar.google.com/scholar?q=Daniel+Gildea+and+Martha+Palmer%2E+2002%2E+The+necessity+of+syntactic+parsing+for+predicate+argument+recognition%2E+In+ACL%2D02%2E)
12. Kadri Hacioglu and Wayne Ward. 2003. Target word detection and semantic role chunking using support vector machines. In HLT-03. [find](https://scholar.google.com/scholar?q=Kadri+Hacioglu+and+Wayne+Ward%2E+2003%2E+Target+word+detection+and+semantic+role+chunking+using+support+vector+machines%2E+In+HLT%2D03%2E)
13. Kadri Hacioglu, Sameer Pradhan, Wayne Ward, James Martin, and Dan Jurafsky. 2003. Shallow semantic parsing using support vector machines. Technical Report TR-CSLR-2003-1, Center for Spoken Language Research, Boulder, Colorado. [find](https://scholar.google.com/scholar?q=Kadri+Hacioglu%2C+Sameer+Pradhan%2C+Wayne+Ward%2C+James+Martin%2C+and+Dan+Jurafsky%2E+2003%2E+Shallow+semantic+parsing+using+support+vector+machines%2E+Technical+Report+TR%2DCSLR%2D2003%2D1%2C+Center+for+Spoken+Language+Re)
14. Thomas Hofmann and Jan Puzicha. 1998. Statistical models for co-occurrence data. Memo, MIT AI Laboratory. [find](https://scholar.google.com/scholar?q=Thomas+Hofmann+and+Jan+Puzicha%2E+1998%2E+Statistical+models+for+co%2Doccurrence+data%2E+Memo%2C+MIT+AI+Laboratory%2E)
15. Paul Kingsbury, Martha Palmer, and Mitch Marcus. 2002. Adding semantic annotation to the Penn Treebank. In HLT-02. [find](https://scholar.google.com/scholar?q=Paul+Kingsbury%2C+Martha+Palmer%2C+and+Mitch+Marcus%2E+2002%2E+Adding+semantic+annotation+to+the+Penn+Treebank%2E+In+HLT%2D02%2E)
16. Taku Kudo and Yuji Matsumoto. 2000. Use of support vector learning for chunk identification. In CoNLL-00. [find](https://scholar.google.com/scholar?q=Taku+Kudo+and+Yuji+Matsumoto%2E+2000%2E+Use+of+support+vector+learning+for+chunk+identification%2E+In+CoNLL%2D00%2E)
17. Taku Kudo and Yuji Matsumoto. 2001. Chunking with support vector machines. In NAACL-01. [find](https://scholar.google.com/scholar?q=Taku+Kudo+and+Yuji+Matsumoto%2E+2001%2E+Chunking+with+support+vector+machines%2E+In+NAACL%2D01%2E) [our card](https://md.research.elamur.ai/free/en/papers/kudo-2001-chunking-support-vector-machines/)
18. LDC. 2002. The AQUAINT Corpus of English News Text, Catalog no. LDC2002t31. [find](https://scholar.google.com/scholar?q=LDC%2E+2002%2E+The+AQUAINT+Corpus+of+English+News+Text%2C+Catalog+no%2E+LDC2002t31%2E)
19. Dekang Lin. 1998. Automatic retrieval and clustering of similar words. In COLING-98. [find](https://scholar.google.com/scholar?q=Dekang+Lin%2E+1998%2E+Automatic+retrieval+and+clustering+of+similar+words%2E+In+COLING%2D98%2E) [our card](https://md.research.elamur.ai/free/en/papers/lin-1998-automatic-retrieval-clustering-similar-words/)
20. David Magerman. 1994. Natural Language Parsing as Statistical Pattern Recognition. Ph.D. thesis, Stanford University, CA. [find](https://scholar.google.com/scholar?q=David+Magerman%2E+1994%2E+Natural+Language+Parsing+as+Statistical+Pattern+Recognition%2E+Ph%2ED%2E+thesis%2C+Stanford+University%2C+CA%2E)
21. Mitchell Marcus, Grace Kim, Mary Ann Marcinkiewicz, Robert MacIntyre, Ann Bies, Mark Ferguson, Karen Katz, and Britta Schasberger. 1994. The Penn TreeBank: Annotating predicate argument structure. [find](https://scholar.google.com/scholar?q=Mitchell+Marcus%2C+Grace+Kim%2C+Mary+Ann+Marcinkiewicz%2C+Robert+MacIntyre%2C+Ann+Bies%2C+Mark+Ferguson%2C+Karen+Katz%2C+and+Britta+Schasberger%2E+1994%2E+The+Penn+TreeBank%3A+Annotating+predicate+argument+structure%2E) [our card](https://md.research.elamur.ai/free/en/papers/bank-1994-marcus-penn-treebank-annotating-predicate-argument/)
22. John Platt. 2000. Probabilities for support vector machines. In A. Smola, P. Bartlett, B. Scolkopf, and D. Schuurmans, editors, Advances in Large Margin Classifiers. MIT press. [find](https://scholar.google.com/scholar?q=John+Platt%2E+2000%2E+Probabilities+for+support+vector+machines%2E+In+A%2E+Smola%2C+P%2E+Bartlett%2C+B%2E+Scolkopf%2C+and+D%2E+Schuurmans%2C+editors%2C+Advances+in+Large+Margin+Classifiers%2E+MIT+press%2E)
23. Sameer Pradhan, Kadri Hacioglu, Wayne Ward, James Martin, and Dan Jurafsky. 2003. Semantic role parsing: Adding semantic structure to unstructured text. In ICDM-03. [find](https://scholar.google.com/scholar?q=Sameer+Pradhan%2C+Kadri+Hacioglu%2C+Wayne+Ward%2C+James+Martin%2C+and+Dan+Jurafsky%2E+2003%2E+Semantic+role+parsing%3A+Adding+semantic+structure+to+unstructured+text%2E+In+ICDM%2D03%2E)
24. Mihai Surdeanu, Sanda Harabagiu, John Williams, and Paul Aarseth. 2003. Using predicate-argument structures for information extraction. In ACL-03. [find](https://scholar.google.com/scholar?q=Mihai+Surdeanu%2C+Sanda+Harabagiu%2C+John+Williams%2C+and+Paul+Aarseth%2E+2003%2E+Using+predicate%2Dargument+structures+for+information+extraction%2E+In+ACL%2D03%2E) [our card](https://md.research.elamur.ai/free/en/papers/surdeanu-2003-predicate-argument-structures-information-extraction/)
25. Cynthia A. Thompson, Roger Levy, and Christopher D. Manning. 2003. A generative model for semantic role labeling. In ECML-03. [find](https://scholar.google.com/scholar?q=Cynthia+A%2E+Thompson%2C+Roger+Levy%2C+and+Christopher+D%2E+Manning%2E+2003%2E+A+generative+model+for+semantic+role+labeling%2E+In+ECML%2D03%2E)