--- title: "Text Chunking using Transformation-Based — Ramshaw 1995" description: "Ramshaw and Marcus tackle text chunking, which means splitting sentences into flat, non-overlapping segments as a step before full parsing." authors: ["Lance Ramshaw", "Mitch Marcus"] year: 1995 venue: "Third Workshop on Very Large Corpora" source: "https://aclanthology.org/W95-0107/" licence: "CC BY-NC-SA 3.0 (ACL Anthology, before 2016)" licence_url: "https://creativecommons.org/licenses/by-nc-sa/3.0/" summary_licence: "Elamur Cosmic License" canonical: "https://research.elamur.ai/free/en/papers/ramshaw-1995-text-chunking-transformation-based-learning/" --- [Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/ramshaw-1995-text-chunking-transformation-based-learning/) [home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [1995](https://md.research.elamur.ai/free/en/papers/1995/) / Text Chunking using Transformation-Based Learning # Text Chunking using Transformation-Based Learning **Authors:** [Lance Ramshaw](https://md.research.elamur.ai/free/en/authors/lance-ramshaw/), Mitch Marcus · **Year:** 1995 · **Venue:** Third Workshop on Very Large Corpora · **Licence:** [CC BY-NC-SA 3.0 (ACL Anthology, before 2016)](https://creativecommons.org/licenses/by-nc-sa/3.0/) **Links:** [Publisher's page](https://aclanthology.org/W95-0107/) · [PDF](https://aclanthology.org/W95-0107.pdf) **Citations:** [430](#cited-by) in our library, of the 140884 papers whose bibliographies we read ## Summary Ramshaw and Marcus tackle text chunking, which means splitting sentences into flat, non-overlapping segments as a step before full parsing. They aim at two targets: non-recursive "baseNP" chunks, and a full partition of each sentence into N-type and V-type chunks loosely modelled on Abney's proposal. Instead of inserting brackets, they assign each word a chunk tag. This turns chunking into tagging and lets them reuse Brill's transformation-based learning. Training and test data were generated automatically from the Wall Street Journal parses in the Penn Treebank, and the text was POS-tagged with Brill's tagger. A baseline gives each word a chunk tag based on its POS tag. The learner then builds an ordered list of rules that rewrite chunk tags using nearby words, POS tags and chunk tags. Scored automatically against the Treebank-derived answers, the system reaches roughly 92% recall and precision on baseNPs and about 88% on the partitioning chunks. The study matters because earlier chunkers used narrower chunk definitions and were mostly checked by hand. This work offers a reproducible, corpus-based way to evaluate chunking. ## Key points - **Data.** Chunk annotations were derived algorithmically from Penn Treebank WSJ parses. BaseNPs are roughly NPs that contain no nested NP. They include determiners but stop before postmodifying PPs or clauses. A possessive marker starts a new baseNP. - **Partitioning task.** Prepositions are grouped with their object NP inside an N chunk, so the system must tell prepositions apart from particles. Everything outside N chunks is labelled V-type, even non-verbal material such as adjective phrases. Tags such as BN, N, BV and V mark chunk starts and continuations. - **Results.** About 92% recall/precision on baseNPs and about 88% on N/V partitioning. - **Ablation without word-specific templates** (largest training size, 200K): - baseNP drops to 90.7% recall / 90.5% precision, a loss of about 1 point; - partitioning drops to 83.6% / 83.5%, a loss of about 5 points; - so lexical rules matter much more for partitioning. - **Error analysis.** VBG/VBN words that occur inside NPs and conjunctions (CC, commas) together make up about half of the sampled baseNP errors. Many of these cases appear to need semantic knowledge beyond the local context. ## Interesting ideas - **Chunks as per-word tags, not brackets.** Local rewrite rules can then never produce unbalanced bracketings. The same trick applies to NER, segmentation, or any span task handled by local models. - **Baseline first, then learned corrections.** Start from a trivial POS-based guess and learn an ordered sequence of readable fixes. This yields an interpretable model whose earliest rules show the most common systematic errors. - **Automatic gold data from a treebank.** Deriving chunk annotations from existing parses provides large training sets and fully automatic evaluation. This avoids the bias of judges checking a system's own output, and it is useful wherever a treebank exists but chunk data does not, for example deriving chunks from UD trees. - **Feature ablation as a diagnostic.** Rerunning without word-specific templates shows how much each task depends on lexical information versus POS patterns. It is a cheap way to decide whether a task needs a lexicalized model. - **Ideas for later work.** Rule templates could refer to chunk structure, such as the first word of the current chunk, rather than fixed offsets. Dependency-like tags such as "N+2" could encode attachment within the same tagging framework, extending chunking toward larger structures. *Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.* Fields: [Natural language processing](https://md.research.elamur.ai/free/en/shelves/nlp/) Topics: [Treebanks](https://md.research.elamur.ai/free/en/topics/treebanks/) · [Part-of-speech tagging](https://md.research.elamur.ai/free/en/topics/part-of-speech-tagging/) ## Cited by Cited by 430 papers of our library cite it (of the 140884 whose bibliographies we read); 27 of them have a page here, 403 not yet. - [Extracting Headless MWEs from Dependency Parse Trees: Parsing, Tagging, and Joint Modeling Approaches](https://md.research.elamur.ai/free/en/papers/acl-2020-shi-extracting-headless-mwes-dependency-parse/) - [Noun Phrase Chunking in Hebrew: Influence of Lexical and Morphological Features](https://md.research.elamur.ai/free/en/papers/aclbulk-2006-goldberg-noun-phrase-chunking-hebrew-influence/) - [SVM Model Tampering and Anchored Learning: A Case Study in Hebrew NP Chunking](https://md.research.elamur.ai/free/en/papers/aclbulk-2007-goldberg-svm-model-tampering-anchored-learning/) - [Publicly Available Clinical BERT Embeddings](https://md.research.elamur.ai/free/en/papers/alsentzer-2019-publicly-available-clinical-bert-embeddings/) - [Adding Noun Phrase Structure to the Penn Treebank](https://md.research.elamur.ai/free/en/papers/bank-2007-vadas-adding-noun-phrase-structure-penn/) - [Parsing Noun Phrases in the Penn Treebank](https://md.research.elamur.ai/free/en/papers/bank-2011-vadas-parsing-noun-phrases-penn-treebank/) - [Discriminative Training Methods for Hidden Markov Models: Theory and Experiments with Perceptron Algorithms](https://md.research.elamur.ai/free/en/papers/collins-2002-discriminative-training-methods-hidden-markov/) - [Survey: Multiword Expression Processing: A Survey](https://md.research.elamur.ai/free/en/papers/constant-2017-survey-multiword-expression-processing-survey/) - [A Lexicalized Tree Adjoining Grammar for English](https://md.research.elamur.ai/free/en/papers/group-1998-lexicalized-tree-adjoining-grammar-english/) - [Chunking with Support Vector Machines](https://md.research.elamur.ai/free/en/papers/kudo-2001-chunking-support-vector-machines/) - [Explainable Semantic Textual Similarity via Dissimilar Span Detection](https://md.research.elamur.ai/free/en/papers/lozano-2026-explainable-semantic-textual-similarity-dissimilar/) - [Generating Phrasal and Sentential Paraphrases: A Survey of Data-Driven Methods](https://md.research.elamur.ai/free/en/papers/madnani-2010-generating-phrasal-sentential-paraphrases-survey/) - [A Simple and Accurate Syntax-Agnostic Neural Model for Dependency-based Semantic Role Labeling](https://md.research.elamur.ai/free/en/papers/marcheggiani-2017-simple-accurate-syntax-agnostic-neural/) - [A Corpus and Model Integrating Multiword Expressions and Supersenses](https://md.research.elamur.ai/free/en/papers/missing-2015-schneider-corpus-model-integrating-multiword-expressions/) - [A Joint Framework for Coreference Resolution and Mention Head Detection](https://md.research.elamur.ai/free/en/papers/peng-2015-joint-framework-coreference-resolution-mention/) - [MaTESe: Machine Translation Evaluation as a Sequence Tagging Problem](https://md.research.elamur.ai/free/en/papers/perrella-2022-matese-machine-translation-evaluation-as/) - [Do LLMs Judge Distantly Supervised Named Entity Labels Well? Constructing the JudgeWEL Dataset](https://md.research.elamur.ai/free/en/papers/plum-2026-do-llms-judge-distantly-supervised/) - [Representing Text Chunks](https://md.research.elamur.ai/free/en/papers/sang-1999-representing-text-chunks/) - [Introduction to the CoNLL-2000 Shared Task Chunking](https://md.research.elamur.ai/free/en/papers/sang-2000-introduction-conll-2000-shared-task/) - [Introduction to the CoNLL-2002 Shared Task: Language-Independent Named Entity Recognition](https://md.research.elamur.ai/free/en/papers/sang-2002-introduction-conll-2002-shared-task/) - [Discriminative Lexical Semantic Segmentation with Gaps: Running the MWE Gamut](https://md.research.elamur.ai/free/en/papers/schneider-2014-discriminative-lexical-semantic-segmentation-gaps/) - [SemEval-2016 Task 10: Detecting Minimal Semantic Units and their Meanings (DiMSUM)](https://md.research.elamur.ai/free/en/papers/schneider-2016-semeval-2016-task-10-detecting/) - [Parsing Argumentation Structures in Persuasive Essays](https://md.research.elamur.ai/free/en/papers/stab-2017-parsing-argumentation-structures-persuasive-essays/) - [Investigating NP-Chunking with Universal Dependencies for English](https://md.research.elamur.ai/free/en/papers/ud-2018-lacroix-investigating-np-chunking-universal-dependencies/) - [Multimodal Sarcasm Target Identification in Tweets](https://md.research.elamur.ai/free/en/papers/wang-2022-multimodal-sarcasm-target-identification-tweets/) - [Chinese Word Segmentation as Character Tagging](https://md.research.elamur.ai/free/en/papers/xue-2003-chinese-word-segmentation-as-character/) - [TAT-QA: A Question Answering Benchmark on a Hybrid of Tabular and Textual Content in Finance](https://md.research.elamur.ai/free/en/papers/zhu-2021-tat-qa-question-answering-benchmark/) ## Cite The paper itself, not this page. BibTeX from the library's record of the work. ```bibtex @inproceedings{ramshaw1995text, title = {{Text Chunking using Transformation-Based Learning}}, author = {Ramshaw, Lance and Marcus, Mitch}, booktitle = {Third Workshop on Very Large Corpora}, year = {1995}, url = {https://aclanthology.org/W95-0107/} } ``` - [← previous Statistical Decision-Tree Models for Parsing](https://md.research.elamur.ai/free/en/papers/magerman-1995-statistical-decision-tree-models-parsing/) - [next → The Repair of Speech Act Misunderstandings by Abductive Inference](https://md.research.elamur.ai/free/en/papers/mcroy-1995-repair-speech-act-misunderstandings-abductive/) - [↑ back to papers of 1995](https://md.research.elamur.ai/free/en/papers/1995/) ## References 14 references, 1 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet. 1. Abney, Steven. 1991. Parsing by chunks. In Berwick, Abney, and Tenny, editors, Principle-Based Parsing. Kluwer Academic Publishers. [find](https://scholar.google.com/scholar?q=Abney%2C+Steven%2E+1991%2E+Parsing+by+chunks%2E+In+Berwick%2C+Abney%2C+and+Tenny%2C+editors%2C+Principle%2DBased+Parsing%2E+Kluwer+Academic+Publishers%2E) 2. Bourigault, D. 1992. Surface grammatical analysis for the extraction of terminological noun phrases. In Proceedings of the Fifteenth International Conference on Computational Linguistics, pages 977-981. [find](https://scholar.google.com/scholar?q=Bourigault%2C+D%2E+1992%2E+Surface+grammatical+analysis+for+the+extraction+of+terminological+noun+phrases%2E+In+Proceedings+of+the+Fifteenth+International+Conference+on+Computational+Linguistics%2C+pages+977%2D98) 3. Brill, Eric. 1993a. Automatic grammar induction and parsing free text: A transformation-based approach. In Proceedings of the DARPA Speech and Natural Language Workshop, 1993, pages 237-242. [find](https://scholar.google.com/scholar?q=Brill%2C+Eric%2E+1993a%2E+Automatic+grammar+induction+and+parsing+free+text%3A+A+transformation%2Dbased+approach%2E+In+Proceedings+of+the+DARPA+Speech+and+Natural+Language+Workshop%2C+1993%2C+pages+237%2D242%2E) 4. Brill, Eric. 1993b. A Corpus-Based Approach to Language Learning. Ph.D. thesis, University of Pennsylvania. [find](https://scholar.google.com/scholar?q=Brill%2C+Eric%2E+1993b%2E+A+Corpus%2DBased+Approach+to+Language+Learning%2E+Ph%2ED%2E+thesis%2C+University+of+Pennsylvania%2E) 5. Brill, Eric. 1993c. Rule based tagger, version 1.14. Available from ftp.cs.jhu.edu in the directory /pub/brill/programs/. [find](https://scholar.google.com/scholar?q=Brill%2C+Eric%2E+1993c%2E+Rule+based+tagger%2C+version+1%2E14%2E+Available+from+ftp%2Ecs%2Ejhu%2Eedu+in+the+directory+%2Fpub%2Fbrill%2Fprograms%2F%2E) 6. Brill, Eric. 1994. Some advances in transformation-based part of speech tagging. In Proceedings of the Twelfth National Conference on Artificial Intelligence, pages 722-727. (cmp-lg/9406010). [find](https://scholar.google.com/scholar?q=Brill%2C+Eric%2E+1994%2E+Some+advances+in+transformation%2Dbased+part+of+speech+tagging%2E+In+Proceedings+of+the+Twelfth+National+Conference+on+Artificial+Intelligence%2C+pages+722%2D727%2E+%28cmp%2Dlg%2F9406010%29%2E) 7. BriU, Eric and Philip Resnik. 1994. A rule-based approach to prepositional attachment disambiguation. In Proceedings of the Sixteenth International Conference on Computational Linguistics. (cmp-lg/9410026). [find](https://scholar.google.com/scholar?q=BriU%2C+Eric+and+Philip+Resnik%2E+1994%2E+A+rule%2Dbased+approach+to+prepositional+attachment+disambiguation%2E+In+Proceedings+of+the+Sixteenth+International+Conference+on+Computational+Linguistics%2E+%28cmp%2Dlg%2F941) 8. Church, Kenneth. 1988. A stochastic parts program and noun phrase parser for unrestricted text. In Second Conference on Applied Natural Language Processing. ACL. [find](https://scholar.google.com/scholar?q=Church%2C+Kenneth%2E+1988%2E+A+stochastic+parts+program+and+noun+phrase+parser+for+unrestricted+text%2E+In+Second+Conference+on+Applied+Natural+Language+Processing%2E+ACL%2E) [our card](https://md.research.elamur.ai/free/en/papers/church-1988-stochastic-parts-program-noun-phrase/) 9. Ejerhed, Eva I. 1988. Finding clauses in unrestricted text by finitary and stochastic methods. In Second Conference on Applied Natural Language Processing, pages 219-227. ACL. [find](https://scholar.google.com/scholar?q=Ejerhed%2C+Eva+I%2E+1988%2E+Finding+clauses+in+unrestricted+text+by+finitary+and+stochastic+methods%2E+In+Second+Conference+on+Applied+Natural+Language+Processing%2C+pages+219%2D227%2E+ACL%2E) 10. Gee, James Paul and Francois Grosjean. 1983. Performance structures: A psycholinguistic and linguistic appraisal. Cognitive Psychology, 15:411-458. [find](https://scholar.google.com/scholar?q=Gee%2C+James+Paul+and+Francois+Grosjean%2E+1983%2E+Performance+structures%3A+A+psycholinguistic+and+linguistic+appraisal%2E+Cognitive+Psychology%2C+15%3A411%2D458%2E) 11. Kupiec, Julian. 1993. An algorithm for finding noun phrase correspondences in bilingual corpora. In Proceedings of the 31st Annual Meeting of the ACL, pages 17-22. [find](https://scholar.google.com/scholar?q=Kupiec%2C+Julian%2E+1993%2E+An+algorithm+for+finding+noun+phrase+correspondences+in+bilingual+corpora%2E+In+Proceedings+of+the+31st+Annual+Meeting+of+the+ACL%2C+pages+17%2D22%2E) 12. Marcus, Mitchell, Grace Kim, Mary Ann Marcinkiewicz, Robert MacIntyre, Ann Bies, Mark Ferguson, Karen Katz, and Britta Schasberger. 1994. The Penn Treebank: A revised corpus design for extracting predicate argument structure. In Human Language Technology, ARPA March 1994 Workshop. Morgan Kaumann. [find](https://scholar.google.com/scholar?q=Marcus%2C+Mitchell%2C+Grace+Kim%2C+Mary+Ann+Marcinkiewicz%2C+Robert+MacIntyre%2C+Ann+Bies%2C+Mark+Ferguson%2C+Karen+Katz%2C+and+Britta+Schasberger%2E+1994%2E+The+Penn+Treebank%3A+A+revised+corpus+design+for+extracting+pred) 13. Ramshaw, Lance A. and Mitchell P. Marcus. 1994. Exploring the statistical derivation of transformational rule sequences for part-of-speech tagging. In Proceedings of the ACL Balancing Act Workshop on Combining Symbolic and Statistical Approaches to Language, pages 86-95. (cmp-lg/9406011). [find](https://scholar.google.com/scholar?q=Ramshaw%2C+Lance+A%2E+and+Mitchell+P%2E+Marcus%2E+1994%2E+Exploring+the+statistical+derivation+of+transformational+rule+sequences+for+part%2Dof%2Dspeech+tagging%2E+In+Proceedings+of+the+ACL+Balancing+Act+Workshop+on+) 14. Voutilainen, Atro. 1993. NPTool, a detector of English noun phrases. In Proceedings of the Workshop on Very Large Corpora, pages 48-57. ACL, June. (cmp-lg/9502010). [find](https://scholar.google.com/scholar?q=Voutilainen%2C+Atro%2E+1993%2E+NPTool%2C+a+detector+of+English+noun+phrases%2E+In+Proceedings+of+the+Workshop+on+Very+Large+Corpora%2C+pages+48%2D57%2E+ACL%2C+June%2E+%28cmp%2Dlg%2F9502010%29%2E)