---
title: "Automatic Domain Adaptation for Parsing — McClosky 2010"
description: "McClosky, Charniak and Johnson define a new task, multiple source parser adaptation. 26 references, cited by 79 papers here."
authors: ["David McClosky", "Eugene Charniak", "Mark Johnson"]
year: 2010
venue: "Human Language Technologies: The 2010 Annual Conference of the North American Chapter of the Association for Computational Linguistics"
source: "https://aclanthology.org/N10-1004/"
licence: "CC BY-NC-SA 3.0 (ACL Anthology, before 2016)"
licence_url: "https://creativecommons.org/licenses/by-nc-sa/3.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/mcclosky-2010-automatic-domain-adaptation-parsing/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/mcclosky-2010-automatic-domain-adaptation-parsing/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2010](https://md.research.elamur.ai/free/en/papers/2010/) / Automatic Domain Adaptation for Parsing

# Automatic Domain Adaptation for Parsing

**Authors:** [David McClosky](https://md.research.elamur.ai/free/en/authors/david-mcclosky/), [Eugene Charniak](https://md.research.elamur.ai/free/en/authors/eugene-charniak/) and [Mark Johnson](https://md.research.elamur.ai/free/en/authors/mark-johnson/) · **Year:** 2010 · **Venue:** [Human Language Technologies: The 2010 Annual Conference of the North American Chapter of the Association for Computational Linguistics](https://md.research.elamur.ai/free/en/venues/naacl/) ([NAACL 2010](https://md.research.elamur.ai/free/en/venues/naacl/2010/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY-NC-SA 3.0 (ACL Anthology, before 2016)](https://creativecommons.org/licenses/by-nc-sa/3.0/)

**Links:** [Publisher's page](https://aclanthology.org/N10-1004/) · [PDF](https://aclanthology.org/N10-1004.pdf) · [OpenAlex](https://openalex.org/W98423455)

**Citations:** 149 citations (OpenAlex, 2026-10-14) · [79](https://md.research.elamur.ai/free/en/papers/mcclosky-2010-automatic-domain-adaptation-parsing/cited-by/) in our library, of the 141,840 papers whose bibliographies we read

## Summary
Statistical parsers usually do well only on text resembling their training data. When a collection mixes many genres, as the web does, it is unclear which parsing model suits each document. McClosky, Charniak and Johnson define a new task, multiple source parser adaptation. A system is trained on labeled and self-trained (automatically parsed) corpora from several domains. It must then parse a target text whose domain is not given.

Their approach builds candidate parsers by linearly interpolating the statistics of the Charniak (2000) parser trained on different source corpora. A linear regression model predicts the f-score each mixture would reach on the target text, using three features that compare the target to the sources. The mixture with the highest predicted score is then used for parsing.

Across six target domains, this beats every non-oracle baseline, including the single best domain-independent model, and comes close to the oracle. The result shows that tailoring the parsing model to each target text pays off and can be fully automated.

## Key points
- The six domains are BNC, GENIA, BROWN, SWBD, ETT and WSJ.
  - Self-trained corpora such as NANC, GUTENBERG and MEDLINE were also available as sources.
  - In the cross-domain table, the WSJ-trained model has the best average f-score (82.0).
- Feature selection kept three regression features:
  - COSINETOP50: cosine similarity over the 50 most frequent words.
  - UNKWORDS: the token-level share of source words never seen in the target.
  - ENTROPY: how evenly the mixture weights are spread across source domains.
- To train the regressor, the authors randomly sampled 1,040 source mixtures and evaluated each on all six targets, giving 6,240 data points. Roughly 200 samples were enough to capture most of the achievable oracle gain.
- Out-of-domain results (the target's own corpus excluded), as macro-averaged f-scores:
  - The model scored 84.0.
  - The best seen oracle scored 84.3.
  - SELF-TRAINED UNIFORM scored 83.4.
  - UNIFORM and FIXED SET: WSJ each stayed at about 81.
- In-domain results:
  - The model scored 86.9.
  - The best seen oracle scored 87.5.
  - The BEST OVERALL MODEL, the single mixture that is best across all targets, scored 86.2. It consists almost entirely of self-trained data: about 60% NANC and 26% MEDLINE.

## Interesting ideas
- **Choose among models by predicting their accuracy.** Instead of classifying which domain a text belongs to, the system regresses the expected score of each candidate model and picks the top one. This turns domain detection into a model-ranking problem and could apply to taggers, NER or MT systems trained on varied corpora.
- **Use cheap, complementary similarity signals.** Overlap among frequent words and the rate of unseen tokens capture different aspects of domain mismatch. Both can be computed from raw text alone, so the method suits languages without much annotated target data.
- **Scale features by mixture weight.** Each source's similarity feature is adjusted by that source's weight in the mixture, so a source with zero weight contributes nothing. This is a simple way to make pairwise features work for weighted ensembles.
- **Sample the configuration space randomly and check coverage.** The authors drew random mixtures (a few corpora, weights taken uniformly from a simplex) and plotted the cumulative oracle score to show the sample was large enough. This is a reusable sanity check for any search over model combinations.
- **Self-training as a domain-coverage tool.** Automatically parsed corpora greatly helped even the naive uniform baseline. Cheaply self-labeled data from many genres can patch sparsity in small treebanks, which is relevant for lower-resource languages such as Ukrainian.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Stanford University](https://md.research.elamur.ai/free/en/universities/stanford-university/) — University, Stanford, United States · 310 works · 284,480 citations
- [Brown University](https://md.research.elamur.ai/free/en/universities/brown-university/) — University, Providence, United States · 58 works · 25,310 citations
- [Macquarie University](https://md.research.elamur.ai/free/en/universities/macquarie-university/) — University, Sydney, Australia · 17 works · 4,827 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 9,085 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 8,888 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 9,296 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 9,206 works

## Cite

Cite as: David McClosky, Eugene Charniak and Mark Johnson. 2010. Automatic Domain Adaptation for Parsing. Human Language Technologies: The 2010 Annual Conference of the North American Chapter of the Association for Computational Linguistics.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{mcclosky2010automatic,
  title = {{Automatic Domain Adaptation for Parsing}},
  author = {David McClosky, Eugene Charniak and Mark Johnson},
  booktitle = {Human Language Technologies: The 2010 Annual Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2010},
  url = {https://aclanthology.org/N10-1004/}
}
```

- [← previous Better Arabic Parsing: Baselines, Evaluations, and Analysis](https://md.research.elamur.ai/free/en/papers/green-2010-better-arabic-parsing-baselines-evaluations/)
- [next → Large Scale Parallel Document Mining for Machine Translation](https://md.research.elamur.ai/free/en/papers/uszkoreit-2010-large-scale-parallel-document-mining/)
- [↑ back to papers of 2010](https://md.research.elamur.ai/free/en/papers/2010/)

## References

26 references, 10 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Michiel Bacchiani, Michael Riley, Brian Roark, and Richard Sproat. 2006. MAP adaptation of stochastic grammars. Computer Speech and Language, 20(1):41–68. [find](https://scholar.google.com/scholar?q=Michiel+Bacchiani%2C+Michael+Riley%2C+Brian+Roark%2C+and+Richard+Sproat%2E+2006%2E+MAP+adaptation+of+stochastic+grammars%2E+Computer+Speech+and+Language%2C+20%281%29%3A41%E2%80%9368%2E)
2. Ann Bies. 2007. GALE Phase 3 Release 1 - English Translation Treebank. Linguistic Data Consortium. LDC2007E105. [find](https://scholar.google.com/scholar?q=Ann+Bies%2E+2007%2E+GALE+Phase+3+Release+1+%2D+English+Translation+Treebank%2E+Linguistic+Data+Consortium%2E+LDC2007E105%2E)
3. John Blitzer, Mark Dredze, and Fernando Pereira. 2007. [Biographies, bollywood, boom-boxes and blenders: Domain adaptation for sentiment classification](https://md.research.elamur.ai/free/en/papers/blitzer-2007-biographies-bollywood-boom-boxes-blenders/). In Association for Computational Linguistics, Prague, Czech Republic.
4. Xavier Carreras, Michael Collins, and Terry Koo. 2008. TAG, dynamic programming, and the perceptron for efficient, feature-rich parsing. In Proceedings of CoNLL 2008, pages 9–16, Manchester, England, August. [find](https://scholar.google.com/scholar?q=Xavier+Carreras%2C+Michael+Collins%2C+and+Terry+Koo%2E+2008%2E+TAG%2C+dynamic+programming%2C+and+the+perceptron+for+efficient%2C+feature%2Drich+parsing%2E+In+Proceedings+of+CoNLL+2008%2C+pages+9%E2%80%9316%2C+Manchester%2C+England%2C+)
5. Eugene Charniak and Mark Johnson. 2005. [Coarse-to-fine n-best parsing and MaxEnt discriminative reranking](https://md.research.elamur.ai/free/en/papers/charniak-2005-coarse-fine-n-best-parsing/). In Proceedings of the ACL 2005, pages 173–180.
6. Eugene Charniak. 2000. [A maximum-entropy-inspired parser](https://md.research.elamur.ai/free/en/papers/charniak-2000-maximum-entropy-inspired-parser/). In Proceedings of the North American Chapter of the ACL (NAACL), pages 132–139.
7. Michael Collins. 1999. Head-driven Statistical Models for Natural Language Parsing. Ph.D. thesis, The University of Pennsylvania. [find](https://scholar.google.com/scholar?q=Michael+Collins%2E+1999%2E+Head%2Ddriven+Statistical+Models+for+Natural+Language+Parsing%2E+Ph%2ED%2E+thesis%2C+The+University+of+Pennsylvania%2E)
8. Hal Daumé III. 2007. [Frustratingly easy domain adaptation](https://md.research.elamur.ai/free/en/papers/iii-2007-frustratingly-easy-domain-adaptation/). In Proceedings of ACL 2007, Prague, Czech Republic.
9. Mark Dredze and Koby Crammer. 2008. Online methods for multi-domain learning and adaptation. In Proceedings of the EMNLP 2008, pages 689–697, Honolulu, Hawaii, October. [find](https://scholar.google.com/scholar?q=Mark+Dredze+and+Koby+Crammer%2E+2008%2E+Online+methods+for+multi%2Ddomain+learning+and+adaptation%2E+In+Proceedings+of+the+EMNLP+2008%2C+pages+689%E2%80%93697%2C+Honolulu%2C+Hawaii%2C+October%2E)
10. Jenny Rose Finkel and Christopher D. Manning. 2009. Hierarchical bayesian domain adaptation. In Proceedings of HLT-NAACL 2009, pages 602–610, Boulder, Colorado, June. [find](https://scholar.google.com/scholar?q=Jenny+Rose+Finkel+and+Christopher+D%2E+Manning%2E+2009%2E+Hierarchical+bayesian+domain+adaptation%2E+In+Proceedings+of+HLT%2DNAACL+2009%2C+pages+602%E2%80%93610%2C+Boulder%2C+Colorado%2C+June%2E)
11. Jennifer Foster and Josef van Genabith. 2008. [Parser evaluation and the bnc: Evaluating 4 constituency parsers with 3 metrics](https://md.research.elamur.ai/free/en/papers/foster-2008-parser-evaluation-bnc-evaluating-4/). In Proceedings LREC 2008, Marrakech, Morocco, May.
12. W. Nelson Francis and Henry Kučera. 1979. Manual of Information to accompany a Standard Corpus of Present-day Edited American English, for use with Digital Computers. Brown University, Providence, Rhode Island. [find](https://scholar.google.com/scholar?q=W%2E+Nelson+Francis+and+Henry+Kuc%CC%8Cera%2E+1979%2E+Manual+of+Information+to+accompany+a+Standard+Corpus+of+Present%2Dday+Edited+American+English%2C+for+use+with+Digital+Computers%2E+Brown+University%2C+Providence%2C+Rh)
13. Daniel Gildea. 2001. [Corpus variation and parser performance](https://md.research.elamur.ai/free/en/papers/gildea-2001-corpus-variation-parser-performance/). In Empirical Methods in Natural Language Processing (EMNLP), pages 167–202.
14. David Graff. 1995. North American News Text Corpus. Linguistic Data Consortium. LDC95T21. [find](https://scholar.google.com/scholar?q=David+Graff%2E+1995%2E+North+American+News+Text+Corpus%2E+Linguistic+Data+Consortium%2E+LDC95T21%2E)
15. Mark Johnson, Eugene Charniak, and Matthew Lease. 2004. An improved model for recognizing disfluencies in conversational speech. In Proc. of the Rich Text 2004 Fall Workshop (RT-04F). [find](https://scholar.google.com/scholar?q=Mark+Johnson%2C+Eugene+Charniak%2C+and+Matthew+Lease%2E+2004%2E+An+improved+model+for+recognizing+disfluencies+in+conversational+speech%2E+In+Proc%2E+of+the+Rich+Text+2004+Fall+Workshop+%28RT%2D04F%29%2E)
16. Daisuke Kawahara and Kiyotaka Uchimoto. 2008. Learning reliability of parses for domain adaptation of dependency parsing. In Third International Joint Conference on Natural Language Processing (IJCNLP ’08). [find](https://scholar.google.com/scholar?q=Daisuke+Kawahara+and+Kiyotaka+Uchimoto%2E+2008%2E+Learning+reliability+of+parses+for+domain+adaptation+of+dependency+parsing%2E+In+Third+International+Joint+Conference+on+Natural+Language+Processing+%28IJCNLP)
17. Mitchell P. Marcus, Beatrice Santorini, and Mary Ann Marcinkiewicz. 1993. [Building a large annotated corpus of English: The Penn Treebank](https://md.research.elamur.ai/free/en/papers/marcus-1993-building-large-annotated-corpus-english/). Comp. Linguistics, 19(2):313–330.
18. David McClosky, Eugene Charniak, and Mark Johnson. 2006a. [Effective self-training for parsing](https://md.research.elamur.ai/free/en/papers/mcclosky-2006-effective-self-training-parsing/). In Proceedings of HLT-NAACL 2006, pages 152–159.
19. David McClosky, Eugene Charniak, and Mark Johnson. 2006b. [Reranking and self-training for parser adaptation](https://md.research.elamur.ai/free/en/papers/mcclosky-2006-reranking-self-training-parser-adaptation/). In Proceedings of COLING-ACL 2006, pages 337–344, Sydney, Australia, July. Association for Computational Linguistics.
20. Slav Petrov and Dan Klein. 2007. [Improved inference for unlexicalized parsing](https://md.research.elamur.ai/free/en/papers/petrov-2007-improved-inference-unlexicalized-parsing/). In Human Language Technologies 2007: The Conference of the North American Chapter of the Association for Computational Linguistics; Proceedings of the Main Conference, pages 404–411, Rochester, New York, April. Associatio…
21. Barbara Plank and Khalil Sima’an. 2008. Subdomain sensitive statistical parsing using raw corpora. In Proceedings of the Sixth International Language Resources and Evaluation (LREC’08), Marrakech, Morocco, May. [find](https://scholar.google.com/scholar?q=Barbara+Plank+and+Khalil+Sima%E2%80%99an%2E+2008%2E+Subdomain+sensitive+statistical+parsing+using+raw+corpora%2E+In+Proceedings+of+the+Sixth+International+Language+Resources+and+Evaluation+%28LREC%E2%80%9908%29%2C+Marrakech%2C+Mor)
22. Sujith Ravi, Kevin Knight, and Radu Soricut. 2008. Automatic prediction of parser accuracy. In Proceedings of the 2008 Conference on Empirical Methods in Natural Language Processing, pages 887–896, Honolulu, Hawaii, October. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Sujith+Ravi%2C+Kevin+Knight%2C+and+Radu+Soricut%2E+2008%2E+Automatic+prediction+of+parser+accuracy%2E+In+Proceedings+of+the+2008+Conference+on+Empirical+Methods+in+Natural+Language+Processing%2C+pages+887%E2%80%93896%2C+Ho)
23. Hinrich Schütze. 1995. Distributional part-of-speech tagging. In Proceedings of the 7th conference of the EACL, pages 141–148. [find](https://scholar.google.com/scholar?q=Hinrich+Schu%CC%88tze%2E+1995%2E+Distributional+part%2Dof%2Dspeech+tagging%2E+In+Proceedings+of+the+7th+conference+of+the+EACL%2C+pages+141%E2%80%93148%2E)
24. Satoshi Sekine. 1997. The domain dependence of parsing. In Proc. Applied Natural Language Processing (ANLP), pages 96–102. [find](https://scholar.google.com/scholar?q=Satoshi+Sekine%2E+1997%2E+The+domain+dependence+of+parsing%2E+In+Proc%2E+Applied+Natural+Language+Processing+%28ANLP%29%2C+pages+96%E2%80%93102%2E)
25. Jun Suzuki, Hideki Isozaki, Xavier Carreras, and Michael Collins. 2009. An empirical study of semi-supervised structured conditional models for dependency parsing. In Proceedings EMNLP 2009, pages 551–560, Singapore, August. [find](https://scholar.google.com/scholar?q=Jun+Suzuki%2C+Hideki+Isozaki%2C+Xavier+Carreras%2C+and+Michael+Collins%2E+2009%2E+An+empirical+study+of+semi%2Dsupervised+structured+conditional+models+for+dependency+parsing%2E+In+Proceedings+EMNLP+2009%2C+pages+551)
26. Yuka Tateisi, Akane Yakushiji, Tomoko Ohta, and Jun’ichi Tsujii. 2005. Syntax Annotation for the GE-NIA corpus. Proceedings of IJCNLP 2005, Companion volume, pages 222–227. [find](https://scholar.google.com/scholar?q=Yuka+Tateisi%2C+Akane+Yakushiji%2C+Tomoko+Ohta%2C+and+Jun%E2%80%99ichi+Tsujii%2E+2005%2E+Syntax+Annotation+for+the+GE%2DNIA+corpus%2E+Proceedings+of+IJCNLP+2005%2C+Companion+volume%2C+pages+222%E2%80%93227%2E)

*Updated 24 October 2026*