--- title: "Unsupervised Models for Named Entity — Collins 1999" description: "The paper tackles named entity classification: assigning an already-segmented proper name to Person, Organization, or Location while using almost no…" authors: ["Michael Collins", "Yoram Singer"] year: 1999 venue: "1999 Joint SIGDAT Conference on Empirical Methods in Natural Language Processing and Very Large Corpora" source: "https://aclanthology.org/W99-0613/" licence: "CC BY-NC-SA 3.0 (ACL Anthology, before 2016)" licence_url: "https://creativecommons.org/licenses/by-nc-sa/3.0/" summary_licence: "Elamur Cosmic License" canonical: "https://research.elamur.ai/free/en/papers/collins-1999-unsupervised-models-named-entity-classification/" --- [Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/collins-1999-unsupervised-models-named-entity-classification/) [home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [1999](https://md.research.elamur.ai/free/en/papers/1999/) / Unsupervised Models for Named Entity Classification # Unsupervised Models for Named Entity Classification **Authors:** [Michael Collins](https://md.research.elamur.ai/free/en/authors/michael-collins/), [Yoram Singer](https://md.research.elamur.ai/free/en/authors/yoram-singer/) · **Year:** 1999 · **Venue:** 1999 Joint SIGDAT Conference on Empirical Methods in Natural Language Processing and Very Large Corpora · **Licence:** [CC BY-NC-SA 3.0 (ACL Anthology, before 2016)](https://creativecommons.org/licenses/by-nc-sa/3.0/) **Links:** [Publisher's page](https://aclanthology.org/W99-0613/) · [PDF](https://aclanthology.org/W99-0613.pdf) · [OpenAlex](https://openalex.org/W2785349534) **Citations:** 803 citations (openalex-snapshot · 2026-10-14) · [188](#cited-by) in our library, of the 140884 papers whose bibliographies we read ## Summary The paper tackles named entity classification: assigning an already-segmented proper name to Person, Organization, or Location while using almost no labeled data. Collins and Singer parsed 971,746 New York Times sentences with the Collins 96 parser. They kept only names found in two syntactic settings: with an appositive modifier, or as the complement of a preposition attached to a noun. Each example is split into two views. The *spelling* view covers the full string, the words it contains, capitalization and non-alphabetic patterns. The *context* view covers the appositive head or the preposition plus its noun. Seven seed rules provide the only supervision. Two algorithms exploit the fact that either view alone often reveals the type: - **DL-CoTrain** is a decision-list bootstrapper in the spirit of Yarowsky 95, adapted using ideas from Blum and Mitchell 98. - **CoBoost** is an AdaBoost-style method that directly minimizes a bound on how often the two view-specific classifiers disagree on unlabeled data. With about 90,000 unlabeled examples, the methods exceed 91% accuracy. This shows that redundancy in unlabeled text can stand in for large annotated sets. ## Key points - **Seed rules:** the only supervision is 7 rules. - New York, California and U.S. are locations. - Any name containing Mr. is a person. - Any name containing Incorporated is an organization. - Microsoft and I.B.M. are organizations. - **Data:** roughly 90,000 (spelling, context) pairs were extracted from 971,746 parsed NYT sentences. - **Test set:** 1,000 random examples were hand-labeled as 186 location, 289 person, 402 organization and 123 noise. Removing 38 day and month expressions left 962 items, 85 of them noise. - **Evaluation:** accuracy is reported twice, once counting noise items as errors and once excluding them. When CoBoost abstained (about 2% of the test set), the label defaulted to organization. - **Decision-list scoring:** rules are ranked by a smoothed estimate of label probability given a feature, with α = 0.1 and k = 3. In CoBoost's learning curves, coverage rises over iterations while the two classifiers keep agreeing on more than 94% of examples. ## Interesting ideas - **Two-view feature design:** each feature looks only at the spelling or only at the context, never both. This deliberately creates two independent views for co-training. The same design could apply wherever an item has an "internal form" and an "environment", such as place names, product names or trade entities in Ukrainian text. - **Agreement as a training signal:** when an unlabeled example carries strong cues in both views, it implicitly says those cues share a label. Collecting many such hints lets a few seeds spread across the corpus without manual annotation. - **Disagreement as an objective:** CoBoost turns "two classifiers should agree" into a smooth bound that can be optimized. This gives an otherwise heuristic bootstrapping process a principled target, and the idea is reusable for any multi-view semi-supervised task. - **Syntax as a precision filter:** extraction is restricted to two precise patterns (appositives and noun-modifying PPs), which yields clean context predictors cheaply. The trade-off is coverage, and the authors themselves list this limitation. - **Honest handling of out-of-scope items:** reporting accuracy both with and without noise items, and separating out trivially detectable cases such as dates, makes the evaluation of a closed-label classifier more transparent. *Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.* Fields: [Natural language processing](https://md.research.elamur.ai/free/en/shelves/nlp/) Topics: [Named entity recognition](https://md.research.elamur.ai/free/en/topics/named-entity-recognition/) ## Cited by Cited by 188 papers of our library cite it (of the 140884 whose bibliographies we read); 17 of them have a page here, 171 not yet. - [Supersense Tagging of Unknown Nouns in WordNet](https://md.research.elamur.ai/free/en/papers/aclbulk-2003-ciaramita-supersense-tagging-unknown-nouns-wordnet/) - [Extracting Paraphrases from a Parallel Corpus](https://md.research.elamur.ai/free/en/papers/barzilay-2001-extracting-paraphrases-parallel-corpus/) - [EM Works for Pronoun Anaphora Resolution](https://md.research.elamur.ai/free/en/papers/charniak-2009-em-works-pronoun-anaphora-resolution/) - [Coreference Resolution in a Modular, Entity-Centered Model](https://md.research.elamur.ai/free/en/papers/haghighi-2010-coreference-resolution-modular-entity-centered/) - [Knowledge Graphs](https://md.research.elamur.ai/free/en/papers/hogan-2020-knowledge-graphs/) - [Deterministic Coreference Resolution Based on Entity-Centric, Precision-Ranked Rules](https://md.research.elamur.ai/free/en/papers/lee-2013-deterministic-coreference-resolution-based-entity/) - [Alignment by Agreement](https://md.research.elamur.ai/free/en/papers/liang-2006-alignment-agreement/) - [Early results for Named Entity Recognition with Conditional Random Fields, Feature Induction and Web-Enhanced Lexicons](https://md.research.elamur.ai/free/en/papers/mccallum-2003-early-results-named-entity-recognition/) - [Weakly Supervised Learning for Hedge Classification in Scientific Literature](https://md.research.elamur.ai/free/en/papers/medlock-2007-weakly-supervised-learning-hedge-classification/) - [Name Tagging with Word Clusters and Discriminative Training](https://md.research.elamur.ai/free/en/papers/miller-2004-name-tagging-word-clusters-discriminative/) - [Bootstrapping Coreference Classifiers with Multiple Machine Learning Algorithms](https://md.research.elamur.ai/free/en/papers/ng-2003-bootstrapping-coreference-classifiers-multiple-machine/) - [Semantic Class Induction and Coreference Resolution](https://md.research.elamur.ai/free/en/papers/ng-2007-semantic-class-induction-coreference-resolution/) - [Named Entity Recognition for Chinese Social Media with Jointly Trained Embeddings](https://md.research.elamur.ai/free/en/papers/peng-2015-named-entity-recognition-chinese-social/) - [A Multi-Pass Sieve for Coreference Resolution](https://md.research.elamur.ai/free/en/papers/raghunathan-2010-multi-pass-sieve-coreference-resolution/) - [Named Entity Recognition in Tweets: An Experimental Study](https://md.research.elamur.ai/free/en/papers/ritter-2011-named-entity-recognition-tweets-experimental/) - [A Bootstrapping Method for Learning Semantic Lexicons using Extraction Pattern Contexts](https://md.research.elamur.ai/free/en/papers/thelen-2002-bootstrapping-method-learning-semantic-lexicons/) - [A Survey on Recent Advances in Named Entity Recognition from Deep Learning models](https://md.research.elamur.ai/free/en/papers/yadav-2018-survey-recent-advances-named-entity/) ## Cite The paper itself, not this page. BibTeX from the library's record of the work. ```bibtex @inproceedings{collins1999unsupervised, title = {{Unsupervised Models for Named Entity Classification}}, author = {Collins, Michael and Singer, Yoram}, booktitle = {1999 Joint SIGDAT Conference on Empirical Methods in Natural Language Processing and Very Large Corpora}, year = {1999}, url = {https://aclanthology.org/W99-0613/} } ``` - [← previous UNL-French deconversion as transfer & generation from an interlingua with possible quality enhancement through offline human interaction](https://md.research.elamur.ai/free/en/papers/serasset-1999-unl-french-deconversion-as-transfer/) - [next → Using Coreference Chains for Text Summarization](https://md.research.elamur.ai/free/en/papers/azzam-1999-coreference-chains-text-summarization/) - [↑ back to papers of 1999](https://md.research.elamur.ai/free/en/papers/1999/) ## References 20 references, 4 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet. 1. M. Berland and E. Charniak. 1999. Finding Parts in Very Large Corpora. In Proceedings of the the 37th Annual Meeting of the Association for Computational Linguistics (ACL-99). [find](https://scholar.google.com/scholar?q=M%2E+Berland+and+E%2E+Charniak%2E+1999%2E+Finding+Parts+in+Very+Large+Corpora%2E+In+Proceedings+of+the+the+37th+Annual+Meeting+of+the+Association+for+Computational+Linguistics+%28ACL%2D99%29%2E) [our card](https://md.research.elamur.ai/free/en/papers/berland-1999-finding-parts-very-large-corpora/) 2. D. M. Bikel, S. Miller, R. Schwartz, and R. Weischedel. 1997. Nymble: a High-Performance Learning Name-finder. In Proceedings of the Fifth Conference on Applied Natural Language Processing, pages 194-201. [find](https://scholar.google.com/scholar?q=D%2E+M%2E+Bikel%2C+S%2E+Miller%2C+R%2E+Schwartz%2C+and+R%2E+Weischedel%2E+1997%2E+Nymble%3A+a+High%2DPerformance+Learning+Name%2Dfinder%2E+In+Proceedings+of+the+Fifth+Conference+on+Applied+Natural+Language+Processing%2C+pages+194%2D) [our card](https://md.research.elamur.ai/free/en/papers/bikel-1997-nymble-high-performance-learning-name/) 3. A. Blum and T. Mitchell. 1998. Combining Labeled and Unlabeled Data with Co-Training. In Proceedings of the l lth Annual Conference on Computational Learning Theory (COLT-98). [find](https://scholar.google.com/scholar?q=A%2E+Blum+and+T%2E+Mitchell%2E+1998%2E+Combining+Labeled+and+Unlabeled+Data+with+Co%2DTraining%2E+In+Proceedings+of+the+l+lth+Annual+Conference+on+Computational+Learning+Theory+%28COLT%2D98%29%2E) [our card](https://md.research.elamur.ai/free/en/papers/blum-1998-combining-labeled-unlabeled-data-co-training/) 4. E. Brill. 1995. Unsupervised Learning of Disambiguation Rules for Part of Speech Tagging. In Proceedings of the " Third Workshop on Very Large Corpora. [find](https://scholar.google.com/scholar?q=E%2E+Brill%2E+1995%2E+Unsupervised+Learning+of+Disambiguation+Rules+for+Part+of+Speech+Tagging%2E+In+Proceedings+of+the+%22+Third+Workshop+on+Very+Large+Corpora%2E) 5. S. Brin. 1998. Extracting Patterns and Relations from the World Wide Web. In WebDB Wokshop at EDBT '98. [find](https://scholar.google.com/scholar?q=S%2E+Brin%2E+1998%2E+Extracting+Patterns+and+Relations+from+the+World+Wide+Web%2E+In+WebDB+Wokshop+at+EDBT+%2798%2E) 6. M. Collins. 1996. A New Statistical Parser Based on Bigram Lexical Dependencies. Proceedings of the 34th Annual ----* .... ..... • .... [find](https://scholar.google.com/scholar?q=M%2E+Collins%2E+1996%2E+A+New+Statistical+Parser+Based+on+Bigram+Lexical+Dependencies%2E+Proceedings+of+the+34th+Annual+%2D%2D%2D%2D%2A+%2E%2E%2E%2E+%2E%2E%2E%2E%2E+%E2%80%A2+%2E%2E%2E%2E) 7. Meeting of the Association for Computational Linguistics, pages 184-191. [find](https://scholar.google.com/scholar?q=Meeting+of+the+Association+for+Computational+Linguistics%2C+pages+184%2D191%2E) 8. A.P. Dempster, N.M. Laird, and D.B. Rubin, (1977). Maximum Likelihood from Incomplete Data Via the EM Algorithm, Journal of the Royal Statistical Society, Ser B, 39, 1-38. [find](https://scholar.google.com/scholar?q=A%2EP%2E+Dempster%2C+N%2EM%2E+Laird%2C+and+D%2EB%2E+Rubin%2C+%281977%29%2E+Maximum+Likelihood+from+Incomplete+Data+Via+the+EM+Algorithm%2C+Journal+of+the+Royal+Statistical+Society%2C+Ser+B%2C+39%2C+1%2D38%2E) 9. Y. Freund. Boosting a weak learning algorithm by majority. Information and Computation, 121 (2):256-285, 1995. [find](https://scholar.google.com/scholar?q=Y%2E+Freund%2E+Boosting+a+weak+learning+algorithm+by+majority%2E+Information+and+Computation%2C+121+%282%29%3A256%2D285%2C+1995%2E) 10. Y. Freund and R. E. Schapire. A decision-theoretic generalization of on-line learning and an application to boosting. Journal of Computer and System Sciences, 55( 1 ): I 19-139, 1997. [find](https://scholar.google.com/scholar?q=Y%2E+Freund+and+R%2E+E%2E+Schapire%2E+A+decision%2Dtheoretic+generalization+of+on%2Dline+learning+and+an+application+to+boosting%2E+Journal+of+Computer+and+System+Sciences%2C+55%28+1+%29%3A+I+19%2D139%2C+1997%2E) 11. M. Hearst. 1992. Automatic Acquisition of Hyponyms from Large Text Corpora. In Proceedings of the Fourteenth International Conference on Computational Linguistics. [find](https://scholar.google.com/scholar?q=M%2E+Hearst%2E+1992%2E+Automatic+Acquisition+of+Hyponyms+from+Large+Text+Corpora%2E+In+Proceedings+of+the+Fourteenth+International+Conference+on+Computational+Linguistics%2E) 12. Michael Kearns. Thoughts on hypothesis boosting. Unpublished manuscript, December 1988. [find](https://scholar.google.com/scholar?q=Michael+Kearns%2E+Thoughts+on+hypothesis+boosting%2E+Unpublished+manuscript%2C+December+1988%2E) 13. J. Lafferty. Additive Models, Boosting, and Inference for Generalized Divergences. In Proceedings of the Twelfth Annual Conference on Computational Learning Theory, 1999. [find](https://scholar.google.com/scholar?q=J%2E+Lafferty%2E+Additive+Models%2C+Boosting%2C+and+Inference+for+Generalized+Divergences%2E+In+Proceedings+of+the+Twelfth+Annual+Conference+on+Computational+Learning+Theory%2C+1999%2E) 14. Proceedings of the Sixth Message Understanding Conference (MUC-6). Morgan Kaufmann, San Mateo, CA. [find](https://scholar.google.com/scholar?q=Proceedings+of+the+Sixth+Message+Understanding+Conference+%28MUC%2D6%29%2E+Morgan+Kaufmann%2C+San+Mateo%2C+CA%2E) 15. E. Riloff and J. Shepherd. 1997. A Corpus-Based Approach for Building Semantic Lexicons. In Proceedings of the Second Conference on Empirical Methods in Natural Language Processing (EMNLP-2). [find](https://scholar.google.com/scholar?q=E%2E+Riloff+and+J%2E+Shepherd%2E+1997%2E+A+Corpus%2DBased+Approach+for+Building+Semantic+Lexicons%2E+In+Proceedings+of+the+Second+Conference+on+Empirical+Methods+in+Natural+Language+Processing+%28EMNLP%2D2%29%2E) 16. E. Riloff and R. Jones. 1999. Learning Dictionaries for Information Extraction by Multi-Level Bootstrapping. In Proceedings of the Sixteenth National Conference on Artificial Intelligence (AAAI-99). [find](https://scholar.google.com/scholar?q=E%2E+Riloff+and+R%2E+Jones%2E+1999%2E+Learning+Dictionaries+for+Information+Extraction+by+Multi%2DLevel+Bootstrapping%2E+In+Proceedings+of+the+Sixteenth+National+Conference+on+Artificial+Intelligence+%28AAAI%2D99%29%2E) 17. R. E. Schapire. The strength of weak learnability. Machine Learning, 5(2): 197-227, 1990. [find](https://scholar.google.com/scholar?q=R%2E+E%2E+Schapire%2E+The+strength+of+weak+learnability%2E+Machine+Learning%2C+5%282%29%3A+197%2D227%2C+1990%2E) 18. R. E. Schapire and Y. Singer. Improved boosting algorithms using confidence-rated predictions. In Proceedings of the Eleventh Annual Conference on Computational Learning Theory, pages 80-91, 1998. To appear, Machine Learning. [find](https://scholar.google.com/scholar?q=R%2E+E%2E+Schapire+and+Y%2E+Singer%2E+Improved+boosting+algorithms+using+confidence%2Drated+predictions%2E+In+Proceedings+of+the+Eleventh+Annual+Conference+on+Computational+Learning+Theory%2C+pages+80%2D91%2C+1998%2E+To+) 19. L. G. Valiant. A theory of the learnable. Communications of the ACM, 27(11): 1134-1142, November 1984. [find](https://scholar.google.com/scholar?q=L%2E+G%2E+Valiant%2E+A+theory+of+the+learnable%2E+Communications+of+the+ACM%2C+27%2811%29%3A+1134%2D1142%2C+November+1984%2E) 20. Yarowsky. 1995. Unsupervised Word Sense Disambiguation Rivaling Supervised Methods.In Proceedings of the 33rd [find](https://scholar.google.com/scholar?q=Yarowsky%2E+1995%2E+Unsupervised+Word+Sense+Disambiguation+Rivaling+Supervised+Methods%2EIn+Proceedings+of+the+33rd) [our card](https://md.research.elamur.ai/free/en/papers/yarowsky-1995-unsupervised-word-sense-disambiguation-rivaling/)