---
title: "Domain Adaptation via Pseudo In-Domain Data — Axelrod 2011"
description: "Statistical machine translation for a narrow domain rarely has enough matching parallel data, while large general-domain corpora consist mostly of…"
authors: ["Amittai Axelrod", "Xiaodong He", "Jianfeng Gao"]
year: 2011
venue: "Proceedings of the 2011 Conference on Empirical Methods in Natural Language Processing"
source: "https://aclanthology.org/D11-1033/"
licence: "CC BY-NC-SA 3.0 (ACL Anthology, before 2016)"
licence_url: "https://creativecommons.org/licenses/by-nc-sa/3.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/axelrod-2011-domain-adaptation-pseudo-domain-data/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/axelrod-2011-domain-adaptation-pseudo-domain-data/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2011](https://md.research.elamur.ai/free/en/papers/2011/) / Domain Adaptation via Pseudo In-Domain Data Selection

# Domain Adaptation via Pseudo In-Domain Data Selection

**Authors:** [Amittai Axelrod](https://md.research.elamur.ai/free/en/authors/amittai-axelrod/), [Xiaodong He](https://md.research.elamur.ai/free/en/authors/xiaodong-he/), [Jianfeng Gao](https://md.research.elamur.ai/free/en/authors/jianfeng-gao/) · **Year:** 2011 · **Venue:** Proceedings of the 2011 Conference on Empirical Methods in Natural Language Processing · **Licence:** [CC BY-NC-SA 3.0 (ACL Anthology, before 2016)](https://creativecommons.org/licenses/by-nc-sa/3.0/)

**Links:** [Publisher's page](https://aclanthology.org/D11-1033/) · [PDF](https://aclanthology.org/D11-1033.pdf) · [OpenAlex](https://openalex.org/W1905522558)

**Citations:** 467 citations (openalex-snapshot · 2026-10-14) · [306](#cited-by) in our library, of the 140673 papers whose bibliographies we read

## Summary
Statistical machine translation for a narrow domain rarely has enough matching parallel data, while large general-domain corpora consist mostly of irrelevant text. Axelrod, He and Gao rank the sentences of a big general corpus by their similarity to the target domain and keep only the top slice, which they call a pseudo in-domain subcorpus. They compare three language-model-based scoring methods. The first is plain in-domain cross-entropy. The second is the Moore-Lewis cross-entropy difference, used here for the first time to select MT training data. The third is a new bilingual Moore-Lewis variant that also considers the target side of each sentence pair. The experiments use the IWSLT Chinese-to-English DIALOG travel task (about 30k in-domain pairs) and a general corpus of 12 million pairs, with Moses, GIZA++, MERT and SRILM 4-gram models. Systems trained on roughly 1% of the general data outperform the full general-domain system. Combining such a subcorpus model with the in-domain model gives the best results. The work shows that careful data selection can matter more than raw data volume, and it makes very large corpora usable on modest hardware.

## Key points
- Baselines on the IWSLT 2008 test set (504 sentences, 7 references each): the in-domain system reaches 37.17 BLEU, and the general-domain system (1.5 billion phrase-table entries) reaches 40.51.
- Keeping only 35k of the 12M sentences with the selection methods still adds about 1.8 BLEU over the general-domain baseline. The authors describe bilingual Moore-Lewis as the most stable of the three methods.
- For combining the in-domain and general-domain models, multiple decoding paths in Moses (42.50 test BLEU) beat linear interpolation of the phrase tables (41.28).
- Pairing the IWSLT model with a bilingual Moore-Lewis 150k subcorpus model yields 43.23 test BLEU. This system uses about 180k training sentences in total and still surpasses the two-model system built on all 12M.
- Target-side language model adaptation was deliberately left out, so the gains reflect translation-model adaptation alone.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

Fields: [Natural language processing](https://md.research.elamur.ai/free/en/shelves/nlp/)

Topics: [Machine translation](https://md.research.elamur.ai/free/en/topics/machine-translation/)

## Cited by

Cited by

306 papers of our library cite it (of the 140673 whose bibliographies we read); 22 of them have a page here, 284 not yet.

- [Challenging Language-Dependent Segmentation for Arabic: An Application to Machine Translation and Part-of-Speech Tagging](https://md.research.elamur.ai/free/en/papers/aclbulk-2017-sajjad-challenging-language-dependent-segmentation-arabic/)
- [Unsupervised Domain Clusters in Pretrained Language Models](https://md.research.elamur.ai/free/en/papers/aharoni-2020-unsupervised-domain-clusters-pretrained-language/)
- [Neural Machine Translation by Jointly Learning to Align and Translate](https://md.research.elamur.ai/free/en/papers/bahdanau-2014-neural-machine-translation-jointly-learning/)
- [Fill-up versus interpolation methods for phrase-based SMT adaptation](https://md.research.elamur.ai/free/en/papers/bisazza-2011-fill-up-versus-interpolation-methods/)
- [Effective Domain Mixing for Neural Machine Translation](https://md.research.elamur.ai/free/en/papers/britz-2017-effective-domain-mixing-neural-machine/)
- [Prompsit's submission to WMT 2018 Parallel Corpus Filtering shared task](https://md.research.elamur.ai/free/en/papers/cartagena-2018-prompsit-s-submission-wmt-2018/)
- [Tagged Back-Translation](https://md.research.elamur.ai/free/en/papers/caswell-2019-tagged-back-translation/)
- [On the Properties of Neural Machine Translation: Encoder-Decoder Approaches](https://md.research.elamur.ai/free/en/papers/cho-2014-properties-neural-machine-translation-encoder/)
- [A Survey of Domain Adaptation for Neural Machine Translation](https://md.research.elamur.ai/free/en/papers/chu-2018-survey-domain-adaptation-neural-machine/)
- [Dual Conditional Cross-Entropy Filtering of Noisy Parallel Corpora](https://md.research.elamur.ai/free/en/papers/dowmunt-2018-dual-conditional-cross-entropy-filtering/)
- [Domain-Specific Language Model Pretraining for Biomedical Natural Language Processing](https://md.research.elamur.ai/free/en/papers/gu-2020-domain-specific-language-model-pretraining/)
- [Achieving Human Parity on Automatic Chinese to English News Translation](https://md.research.elamur.ai/free/en/papers/hassan-2018-achieving-human-parity-automatic-chinese/)
- [Measuring Machine Translation Errors in New Domains](https://md.research.elamur.ai/free/en/papers/irvine-2013-measuring-machine-translation-errors-new/)
- [On the Impact of Various Types of Noise on Neural Machine Translation](https://md.research.elamur.ai/free/en/papers/khayrallah-2018-impact-various-types-noise-neural/)
- [Findings of the WMT 2018 Shared Task on Parallel Corpus Filtering](https://md.research.elamur.ai/free/en/papers/koehn-2018-findings-wmt-2018-shared-task/)
- [Findings of the WMT 2020 Shared Task on Parallel Corpus Filtering and Alignment](https://md.research.elamur.ai/free/en/papers/koehn-2020-findings-wmt-2020-shared-task/)
- [Quality at a Glance: An Audit of Web-Crawled Multilingual Datasets](https://md.research.elamur.ai/free/en/papers/kreutzer-2021-quality-glance-audit-web-crawled/)
- [Addressing the Rare Word Problem in Neural Machine Translation](https://md.research.elamur.ai/free/en/papers/luong-2015-addressing-rare-word-problem-neural/)
- [MTNT: A Testbed for Machine Translation of Noisy Text](https://md.research.elamur.ai/free/en/papers/michel-2018-mtnt-testbed-machine-translation-noisy/)
- [Using the Output Embedding to Improve Language Models](https://md.research.elamur.ai/free/en/papers/press-2017-output-embedding-improve-language-models/)
- [Neural Unsupervised Domain Adaptation in NLP–-A Survey](https://md.research.elamur.ai/free/en/papers/ramponi-2020-neural-unsupervised-domain-adaptation-nlp/)
- [Denoising Neural Machine Translation Training with Trusted Data and Online Data Selection](https://md.research.elamur.ai/free/en/papers/wang-2018-denoising-neural-machine-translation-training/)

## Cite

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{axelrod2011domain,
  title = {{Domain Adaptation via Pseudo In-Domain Data Selection}},
  author = {Axelrod, Amittai and He, Xiaodong and Gao, Jianfeng},
  booktitle = {Proceedings of the 2011 Conference on Empirical Methods in Natural Language Processing},
  year = {2011},
  url = {https://aclanthology.org/D11-1033/}
}
```

- [← previous Discriminating Gender on Twitter](https://md.research.elamur.ai/free/en/papers/burger-2011-discriminating-gender-twitter/)
- [next → Domain-Dependent Detection of Light Verb Constructions](https://md.research.elamur.ai/free/en/papers/aclbulk-2011-nagy-domain-dependent-detection-light-verb/)
- [↑ back to papers of 2011](https://md.research.elamur.ai/free/en/papers/2011/)

## References

17 references, 4 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Amittai Axelrod. 2006. Factored Language Models for Statistical Machine Translation. M.Sc. Thesis. University of Edinburgh, Scotland. [find](https://scholar.google.com/scholar?q=Amittai+Axelrod%2E+2006%2E+Factored+Language+Models+for+Statistical+Machine+Translation%2E+M%2ESc%2E+Thesis%2E+University+of+Edinburgh%2C+Scotland%2E)
2. Alexandra Birch, Miles Osborne and Philipp Koehn. 2007. CCG Supertags in Factored Translation Models. Workshop on Statistical Machine Translation, Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Alexandra+Birch%2C+Miles+Osborne+and+Philipp+Koehn%2E+2007%2E+CCG+Supertags+in+Factored+Translation+Models%2E+Workshop+on+Statistical+Machine+Translation%2C+Association+for+Computational+Linguistics%2E)
3. Stanley Chen and Joshua Goodman. 1998. An Empirical Study of Smoothing Techniques for Language Modeling. Technical Report 10-98, Computer Science Group, Harvard University. [find](https://scholar.google.com/scholar?q=Stanley+Chen+and+Joshua+Goodman%2E+1998%2E+An+Empirical+Study+of+Smoothing+Techniques+for+Language+Modeling%2E+Technical+Report+10%2D98%2C+Computer+Science+Group%2C+Harvard+University%2E) [our card](https://md.research.elamur.ai/free/en/papers/chen-1996-empirical-study-smoothing-techniques-language/)
4. Matthias Eck, Stephan Vogel, and Alex Waibel. 2004. Language Model Adaptation for Statistical Machine Translation based on Information Retrieval. Language Resources and Evaluation. [find](https://scholar.google.com/scholar?q=Matthias+Eck%2C+Stephan+Vogel%2C+and+Alex+Waibel%2E+2004%2E+Language+Model+Adaptation+for+Statistical+Machine+Translation+based+on+Information+Retrieval%2E+Language+Resources+and+Evaluation%2E)
5. George Foster and Roland Kuhn. 2007. Mixture-Model Adaptation for SMT. Workshop on Statistical Machine Translation, Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=George+Foster+and+Roland+Kuhn%2E+2007%2E+Mixture%2DModel+Adaptation+for+SMT%2E+Workshop+on+Statistical+Machine+Translation%2C+Association+for+Computational+Linguistics%2E) [our card](https://md.research.elamur.ai/free/en/papers/foster-2007-mixture-model-adaptation-smt/)
6. George Foster, Cyril Goutte, and Roland Kuhn. 2010. Discriminative Instatnce Weighting for Domain Adaptation in Statistical Machine Translation. Empirical Methods in Natural Language Processing. [find](https://scholar.google.com/scholar?q=George+Foster%2C+Cyril+Goutte%2C+and+Roland+Kuhn%2E+2010%2E+Discriminative+Instatnce+Weighting+for+Domain+Adaptation+in+Statistical+Machine+Translation%2E+Empirical+Methods+in+Natural+Language+Processing%2E)
7. Jianfeng Gao, Joshua Goodman, Mingjing Li, and Kai-Fu Lee. 2002. Toward a Unified Approach to Statistical Language Modeling for Chinese. ACM Transactions on Asian Language Information Processing. [find](https://scholar.google.com/scholar?q=Jianfeng+Gao%2C+Joshua+Goodman%2C+Mingjing+Li%2C+and+Kai%2DFu+Lee%2E+2002%2E+Toward+a+Unified+Approach+to+Statistical+Language+Modeling+for+Chinese%2E+ACM+Transactions+on+Asian+Language+Information+Processing%2E)
8. Xiaodong He. 2007. Using Word-Dependent Transition Models in HMM-based Word Alignment for Statistical Machine Translation. Workshop on Statistical Machine Translation, Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Xiaodong+He%2E+2007%2E+Using+Word%2DDependent+Transition+Models+in+HMM%2Dbased+Word+Alignment+for+Statistical+Machine+Translation%2E+Workshop+on+Statistical+Machine+Translation%2C+Association+for+Computational+Li)
9. Philipp Koehn, Hieu Hoang, Alexandra Birch, Chris Callison-Burch, Marcello Federico, Nicola Bertoldi, Brooke Cowan, Wade Shen, Christine Moran, Richard Zens, Chris Dyer, Ondrej Bojar, Alexandra Constantin, and Evan Herbst. 2003. Moses: Open Source Toolkit for Statistical Machine Translation. Demo Se… [find](https://scholar.google.com/scholar?q=Philipp+Koehn%2C+Hieu+Hoang%2C+Alexandra+Birch%2C+Chris+Callison%2DBurch%2C+Marcello+Federico%2C+Nicola+Bertoldi%2C+Brooke+Cowan%2C+Wade+Shen%2C+Christine+Moran%2C+Richard+Zens%2C+Chris+Dyer%2C+Ondrej+Bojar%2C+Alexandra+Consta)
10. Philipp Koehn and Josh Schroeder. 2007. Experiments in Domain Adaptation for Statistical Machine Translation. Workshop on Statistical Machine Translation, Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Philipp+Koehn+and+Josh+Schroeder%2E+2007%2E+Experiments+in+Domain+Adaptation+for+Statistical+Machine+Translation%2E+Workshop+on+Statistical+Machine+Translation%2C+Association+for+Computational+Linguistics%2E) [our card](https://md.research.elamur.ai/free/en/papers/koehn-2007-experiments-domain-adaptation-statistical-machine/)
11. Yajuan Lü, Jin Huang and Qun Liu. 2007. Improving Statistical Machine Translation Performance by Training Data Selection and Optimization. Empirical Methods in Natural Language Processing and Computational Natural Language Learning. [find](https://scholar.google.com/scholar?q=Yajuan+Lu%CC%88%2C+Jin+Huang+and+Qun+Liu%2E+2007%2E+Improving+Statistical+Machine+Translation+Performance+by+Training+Data+Selection+and+Optimization%2E+Empirical+Methods+in+Natural+Language+Processing+and+Computa)
12. Spyros Matsoukas, Antti-Veikko Rosti, Bing Zhang. 2009. Discriminative Corpus Weight Estimation for Machine Translation. Empirical Methods in Natural Language Processing. [find](https://scholar.google.com/scholar?q=Spyros+Matsoukas%2C+Antti%2DVeikko+Rosti%2C+Bing+Zhang%2E+2009%2E+Discriminative+Corpus+Weight+Estimation+for+Machine+Translation%2E+Empirical+Methods+in+Natural+Language+Processing%2E)
13. Robert Moore and William Lewis. 2010. Intelligent Selection of Language Model Training Data. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Robert+Moore+and+William+Lewis%2E+2010%2E+Intelligent+Selection+of+Language+Model+Training+Data%2E+Association+for+Computational+Linguistics%2E) [our card](https://md.research.elamur.ai/free/en/papers/moore-2010-intelligent-selection-language-model-training/)
14. Preslav Nakov. 2008. Improving English-Spanish Statistical Machine Translation: Experiments in Domain Adaptation, Sentence Paraphrasing, Tokenization, and Recasing. Workshop on Statistical Machine Translation, Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Preslav+Nakov%2E+2008%2E+Improving+English%2DSpanish+Statistical+Machine+Translation%3A+Experiments+in+Domain+Adaptation%2C+Sentence+Paraphrasing%2C+Tokenization%2C+and+Recasing%2E+Workshop+on+Statistical+Machine+Tra)
15. Franz Och and Hermann Ney. 2003. A Systematic Comparison of Various Statistical Alignment Models. [find](https://scholar.google.com/scholar?q=Franz+Och+and+Hermann+Ney%2E+2003%2E+A+Systematic+Comparison+of+Various+Statistical+Alignment+Models%2E)
16. Computational Linguistics Franz Och. 2003. Minimum Error Rate Training in Statistical Machine Translation. Association for Computational Linguistics Andreas Stolcke. 2002. SRILM – An Extensible Language Modeling Toolkit. Spoken Language Processing. [find](https://scholar.google.com/scholar?q=Computational+Linguistics+Franz+Och%2E+2003%2E+Minimum+Error+Rate+Training+in+Statistical+Machine+Translation%2E+Association+for+Computational+Linguistics+Andreas+Stolcke%2E+2002%2E+SRILM+%E2%80%93+An+Extensible+Langua)
17. Keiji Yasuda, Ruiqiang Zhang, Hirofumi Yamamoto, Eiichiro Sumita. 2008. Method of Selecting Training Data to Build a Compact and Efficient Translation Model. International Joint Conference on Natural Language Processing. [find](https://scholar.google.com/scholar?q=Keiji+Yasuda%2C+Ruiqiang+Zhang%2C+Hirofumi+Yamamoto%2C+Eiichiro+Sumita%2E+2008%2E+Method+of+Selecting+Training+Data+to+Build+a+Compact+and+Efficient+Translation+Model%2E+International+Joint+Conference+on+Natural+)