---
title: "Mixture-Model Adaptation for SMT — Foster 2007"
description: "SMT systems trained on parallel data from narrow sources, such as parliamentary proceedings, often perform poorly on other domains."
authors: ["George Foster", "Roland Kuhn"]
year: 2007
venue: "Proceedings of the Second Workshop on Statistical Machine Translation"
source: "https://aclanthology.org/W07-0717/"
licence: "CC BY-NC-SA 3.0 (ACL Anthology, before 2016)"
licence_url: "https://creativecommons.org/licenses/by-nc-sa/3.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/papers/foster-2007-mixture-model-adaptation-smt/"
---
[Elamur Research](https://md.research.elamur.ai/) · [this page in HTML](https://research.elamur.ai/free/papers/foster-2007-mixture-model-adaptation-smt/)

[home](https://md.research.elamur.ai/) / [papers](https://md.research.elamur.ai/free/papers/) / [2007](https://md.research.elamur.ai/free/papers/2007/) / Mixture-Model Adaptation for SMT

# Mixture-Model Adaptation for SMT

**Authors:** [George Foster](https://md.research.elamur.ai/free/authors/george-foster/), [Roland Kuhn](https://md.research.elamur.ai/free/authors/roland-kuhn/) · **Year:** 2007 · **Venue:** Proceedings of the Second Workshop on Statistical Machine Translation · **Licence:** [CC BY-NC-SA 3.0 (ACL Anthology, before 2016)](https://creativecommons.org/licenses/by-nc-sa/3.0/)

**Links:** [Publisher's page](https://aclanthology.org/W07-0717/) · [PDF](https://aclanthology.org/W07-0717.pdf) · [OpenAlex](https://openalex.org/W2132001515)

**Citations:** 251 citations (openalex-snapshot · 2026-10-14) · [146](#cited-by) in our library, of the 140673 papers whose bibliographies we read

## Summary
SMT systems trained on parallel data from narrow sources, such as parliamentary proceedings, often perform poorly on other domains. Foster and Kuhn address this with mixture modeling inside a phrase-based Chinese-to-English system. The training corpus is split by genre, which here roughly means by corpus source. Separate language models (LM) and translation models (TM) are trained on each part and then combined with weights reflecting how close each part is to the test domain.

The authors explore many design choices:
- cross-domain adaptation (a small in-domain dev set is available) versus dynamic adaptation (only the source text being translated is available);
- linear versus loglinear mixing;
- adapting the LM, the TM, or both;
- four proximity metrics (tf/idf, LSA, perplexity, EM);
- the unit adapted to: whole test set, genre, or single document.

The most effective recipe sets linear weights proportional to EM-estimated mixture weights of n-gram LMs on the source text. It gains about one BLEU point over a strong unadapted baseline. This matters because it reuses existing corpus divisions and needs no in-domain data at translation time.

## Key points
- Baseline: phrase-based SMT with a loglinear model tuned by Och's minimum-error training for BLEU, 4-gram Kneser-Ney LMs built with SRILM, and phrase pairs extracted from symmetrized IBM2 alignments.
- Word alignment comes from one global IBM2 model, which avoids weaker alignments on small components. Only phrase relative frequencies are made component-specific, and a large static global model is kept alongside the components.
- Cross-domain results (tuned on NIST04-nw): LM adaptation raises NIST05 from 30.3 to 31.2 BLEU. Adding an adapted TM gives no further gain, which suggests the two adapted models capture overlapping information.
- Dynamic adaptation: LM adaptation with source-side EM weights improves results by up to roughly one BLEU point, while TM adaptation is barely better than the baseline. Combining cross-domain global weights with dynamically computed mixture weights performs slightly worse than pure cross-domain adaptation.
- Granularity (Table 8): adapting per genre gives a slight edge, for example 31.1 versus 30.4 on NIST05 and 28.9 versus 27.6 on NIST06-nist. Differences among the file, genre and document levels are small.

## Interesting ideas
- **Learn the mapping from distance to weights instead of the weights themselves.** For unseen domains, the authors tune on a dev set mixing several domains and learn how weights should depend on distance. This lets the method generalize to domains absent from the dev data, a useful trick for any multi-source system that faces unpredictable inputs.
- **EM mixture weights as a proximity measure.** Fitting the weights of a word-level mixture of component LMs to the current text yields a ready-made relevance score for each component. LM-based metrics beat vector-space ones like tf/idf and LSA. The approach is cheap and could rank data sources for domain adaptation in other NLP tasks.
- **Soft weighting instead of hard data selection.** Every component contributes in proportion to its relevance, rather than relevant data being kept and the rest discarded. Texts with little surface overlap can still raise the probability of rare but useful words.
- **Use a global model for fragile substeps.** Word alignment is shared across components, and only the cheaper statistics are made component-specific. This prevents small subsets from degrading the quality of shared structure.
- **Prefer linear mixtures for tractability.** Loglinear mixing fits standard tuning but adds many global parameters and was not used for dynamic adaptation. Linear mixing was easier to control, which is a reasonable default when adaptation parameters must be set outside the main optimizer.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

Fields: [Linguistics & languages](https://md.research.elamur.ai/free/shelves/linguistics/) · [Machine translation](https://md.research.elamur.ai/free/shelves/machine-translation/)

Topics: [Machine translation](https://md.research.elamur.ai/free/topics/machine-translation/)

## Cited by

Cited by

146 papers of our library cite it (of the 140673 whose bibliographies we read); 6 of them have a page here, 140 not yet.

- [Domain Adaptation via Pseudo In-Domain Data Selection](https://md.research.elamur.ai/free/papers/axelrod-2011-domain-adaptation-pseudo-domain-data/)
- [Domain Adaptation for Statistical Machine Translation with Monolingual Resources](https://md.research.elamur.ai/free/papers/bertoldi-2009-domain-adaptation-statistical-machine-translation/)
- [Fill-up versus interpolation methods for phrase-based SMT adaptation](https://md.research.elamur.ai/free/papers/bisazza-2011-fill-up-versus-interpolation-methods/)
- [A Survey of Domain Adaptation for Neural Machine Translation](https://md.research.elamur.ai/free/papers/chu-2018-survey-domain-adaptation-neural-machine/)
- [Multi-Domain Neural Machine Translation through Unsupervised Adaptation](https://md.research.elamur.ai/free/papers/farajian-2017-multi-domain-neural-machine-translation/)
- [Measuring Machine Translation Errors in New Domains](https://md.research.elamur.ai/free/papers/irvine-2013-measuring-machine-translation-errors-new/)

## Cite

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{foster2007mixture,
  title = {{Mixture-Model Adaptation for SMT}},
  author = {Foster, George and Kuhn, Roland},
  booktitle = {Proceedings of the Second Workshop on Statistical Machine Translation},
  year = {2007},
  doi = {10.3115/1626355.1626372},
  url = {https://doi.org/10.3115/1626355.1626372}
}
```

- [← previous METEOR: An Automatic Metric for MT Evaluation with High Levels of Correlation with Human Judgments](https://md.research.elamur.ai/free/papers/lavie-2007-meteor-automatic-metric-mt-evaluation/)
- [next → Morphological Disambiguation of Hebrew: A Case Study in Classifier Combination](https://md.research.elamur.ai/free/papers/aclbulk-2007-shacham-morphological-disambiguation-hebrew-case-study/)
- [↑ back to papers of 2007](https://md.research.elamur.ai/free/papers/2007/)

## References

22 references, 2 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Peter F. Brown, Stephen A. Della Pietra, Vincent Della J. Pietra, and Robert L. Mercer. 1993. The mathematics of Machine Translation: Parameter estimation. Computational Linguistics, 19(2):263–312, June. [find](https://scholar.google.com/scholar?q=Peter+F%2E+Brown%2C+Stephen+A%2E+Della+Pietra%2C+Vincent+Della+J%2E+Pietra%2C+and+Robert+L%2E+Mercer%2E+1993%2E+The+mathematics+of+Machine+Translation%3A+Parameter+estimation%2E+Computational+Linguistics%2C+19%282%29%3A263%E2%80%93312%2C+Ju)
2. W. Byrne, S. Khudanpur, W. Kim, S. Kumar, P. Pecina, P. Virga, P. Xu, and D. Yarowsky. 2003. [find](https://scholar.google.com/scholar?q=W%2E+Byrne%2C+S%2E+Khudanpur%2C+W%2E+Kim%2C+S%2E+Kumar%2C+P%2E+Pecina%2C+P%2E+Virga%2C+P%2E+Xu%2C+and+D%2E+Yarowsky%2E+2003%2E)
3. The JHU 2003 Chinese-English Machine Translation System. In MT Summit IX, New Orleans, September. [find](https://scholar.google.com/scholar?q=The+JHU+2003+Chinese%2DEnglish+Machine+Translation+System%2E+In+MT+Summit+IX%2C+New+Orleans%2C+September%2E)
4. S. Deerwester, S. T. Dumais, G. W. Furnas, T. K. Landauer, and R. Harshman. 1990. Indexing by latent semantic analysis. JASIS, 41(6):391–407. [find](https://scholar.google.com/scholar?q=S%2E+Deerwester%2C+S%2E+T%2E+Dumais%2C+G%2E+W%2E+Furnas%2C+T%2E+K%2E+Landauer%2C+and+R%2E+Harshman%2E+1990%2E+Indexing+by+latent+semantic+analysis%2E+JASIS%2C+41%286%29%3A391%E2%80%93407%2E)
5. Radu Florian and David Yarowsky. 1999. Dynamic non-local language modeling via hierarchical topic-based adaptation. In ACL 1999, pages 167–174, College Park, Maryland, June. [find](https://scholar.google.com/scholar?q=Radu+Florian+and+David+Yarowsky%2E+1999%2E+Dynamic+non%2Dlocal+language+modeling+via+hierarchical+topic%2Dbased+adaptation%2E+In+ACL+1999%2C+pages+167%E2%80%93174%2C+College+Park%2C+Maryland%2C+June%2E)
6. George Foster, Roland Kuhn, and Howard Johnson. 2006. Phrasetable smoothing for statistical machine translation. In EMNLP 2006, Sydney, Australia. [find](https://scholar.google.com/scholar?q=George+Foster%2C+Roland+Kuhn%2C+and+Howard+Johnson%2E+2006%2E+Phrasetable+smoothing+for+statistical+machine+translation%2E+In+EMNLP+2006%2C+Sydney%2C+Australia%2E)
7. Trevor Hastie, Robert Tibshirani, and Jerome Friedman. 2001. The Elements of Statistical Learning. Springer. [find](https://scholar.google.com/scholar?q=Trevor+Hastie%2C+Robert+Tibshirani%2C+and+Jerome+Friedman%2E+2001%2E+The+Elements+of+Statistical+Learning%2E+Springer%2E)
8. Almut Silja Hildebrand, Matthias Eck, Stephan Vogel, and Alex Waibel. 1995. Adaptation of the translation model for statistical machine translation based on information retrieval. In EAMT 1995, Budapest, May. [find](https://scholar.google.com/scholar?q=Almut+Silja+Hildebrand%2C+Matthias+Eck%2C+Stephan+Vogel%2C+and+Alex+Waibel%2E+1995%2E+Adaptation+of+the+translation+model+for+statistical+machine+translation+based+on+information+retrieval%2E+In+EAMT+1995%2C+Budape)
9. R. Iyer and M. Ostendorf. 1999. Modeling long distance dependence in language: Topic mixtures vs. dynamic cache models. In IEEE Trans on Speech and Language Processing, 1999. [find](https://scholar.google.com/scholar?q=R%2E+Iyer+and+M%2E+Ostendorf%2E+1999%2E+Modeling+long+distance+dependence+in+language%3A+Topic+mixtures+vs%2E+dynamic+cache+models%2E+In+IEEE+Trans+on+Speech+and+Language+Processing%2C+1999%2E)
10. Frederick Jelinek. 1997. Statistical Methods for Speech Recognition. MIT Press. [find](https://scholar.google.com/scholar?q=Frederick+Jelinek%2E+1997%2E+Statistical+Methods+for+Speech+Recognition%2E+MIT+Press%2E)
11. Reinhard Kneser and Volker Steinbiss. 1993. On the dynamic adaptation of stochastic language models. In ICASSP 1993, pages 586–589, Minneapolis, Minnesota. IEEE. [find](https://scholar.google.com/scholar?q=Reinhard+Kneser+and+Volker+Steinbiss%2E+1993%2E+On+the+dynamic+adaptation+of+stochastic+language+models%2E+In+ICASSP+1993%2C+pages+586%E2%80%93589%2C+Minneapolis%2C+Minnesota%2E+IEEE%2E)
12. Philipp Koehn, Franz Josef Och, and Daniel Marcu. 2003. Statistical phrase-based translation. In NAACL 2003, pages 127–133. [find](https://scholar.google.com/scholar?q=Philipp+Koehn%2C+Franz+Josef+Och%2C+and+Daniel+Marcu%2E+2003%2E+Statistical+phrase%2Dbased+translation%2E+In+NAACL+2003%2C+pages+127%E2%80%93133%2E)
13. Roland Kuhn and Renato De Mori. 1990. A cache-based natural language model for speech recognition. IEEE Trans on PAMI, 12(6):570–583, June. [find](https://scholar.google.com/scholar?q=Roland+Kuhn+and+Renato+De+Mori%2E+1990%2E+A+cache%2Dbased+natural+language+model+for+speech+recognition%2E+IEEE+Trans+on+PAMI%2C+12%286%29%3A570%E2%80%93583%2C+June%2E)
14. Percy Liang, Alexandre Bouchard-Côté, Dan Klein, and Ben Taskar. 2006. An end-to-end discriminative approach to machine translation. In ACL 2006 [find](https://scholar.google.com/scholar?q=Percy+Liang%2C+Alexandre+Bouchard%2DCo%CC%82te%CC%81%2C+Dan+Klein%2C+and+Ben+Taskar%2E+2006%2E+An+end%2Dto%2Dend+discriminative+approach+to+machine+translation%2E+In+ACL+2006)
15. Franz Josef Och. 2003. Minimum error rate training for statistical machine translation. In ACL 2003, Sapporo, July. [find](https://scholar.google.com/scholar?q=Franz+Josef+Och%2E+2003%2E+Minimum+error+rate+training+for+statistical+machine+translation%2E+In+ACL+2003%2C+Sapporo%2C+July%2E)
16. Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2001. BLEU: A method for automatic evaluation of Machine Translation. Technical Report RC22176, IBM, September. [find](https://scholar.google.com/scholar?q=Kishore+Papineni%2C+Salim+Roukos%2C+Todd+Ward%2C+and+Wei%2DJing+Zhu%2E+2001%2E+BLEU%3A+A+method+for+automatic+evaluation+of+Machine+Translation%2E+Technical+Report+RC22176%2C+IBM%2C+September%2E) [our card](https://md.research.elamur.ai/free/papers/papineni-2002-bleu-method-automatic-evaluation-machine/)
17. William H. Press, Saul A. Teukolsky, William T. Vetterling, and Brian P. Flannery. 2002. Numerical Recipes in C++. Cambridge University Press, Cambridge, UK. [find](https://scholar.google.com/scholar?q=William+H%2E+Press%2C+Saul+A%2E+Teukolsky%2C+William+T%2E+Vetterling%2C+and+Brian+P%2E+Flannery%2E+2002%2E+Numerical+Recipes+in+C%2B%2B%2E+Cambridge+University+Press%2C+Cambridge%2C+UK%2E)
18. Christoph Tillmann and Tong Zhang. 2006. A discriminative global training algorithm for statistical MT. In ACL 2006. [find](https://scholar.google.com/scholar?q=Christoph+Tillmann+and+Tong+Zhang%2E+2006%2E+A+discriminative+global+training+algorithm+for+statistical+MT%2E+In+ACL+2006%2E)
19. Nicola Ueffing. 2006. Self-training for machine translation. In NIPS 2006 Workshop on MLIA, Whistler, B.C., December. [find](https://scholar.google.com/scholar?q=Nicola+Ueffing%2E+2006%2E+Self%2Dtraining+for+machine+translation%2E+In+NIPS+2006+Workshop+on+MLIA%2C+Whistler%2C+B%2EC%2E%2C+December%2E)
20. Richard Zens and Hermann Ney. 2004. Improvements in phrase-based statistical machine translation. In HLT/NAACL 2004, Boston, May. [find](https://scholar.google.com/scholar?q=Richard+Zens+and+Hermann+Ney%2E+2004%2E+Improvements+in+phrase%2Dbased+statistical+machine+translation%2E+In+HLT%2FNAACL+2004%2C+Boston%2C+May%2E) [our card](https://md.research.elamur.ai/free/papers/zens-2004-improvements-phrase-based-statistical-machine/)
21. R. Zhang, H. Yamamoto, M. Paul, H. Okuma, K. Yasuda, Y. Lepage, E. Denoual, D. Mochihashi, A. Finch, and E. Sumita. 2006. The NiCT-ATR statistical machine translation system for the IWSLT 2006 evaluation. In IWSLT 2006. [find](https://scholar.google.com/scholar?q=R%2E+Zhang%2C+H%2E+Yamamoto%2C+M%2E+Paul%2C+H%2E+Okuma%2C+K%2E+Yasuda%2C+Y%2E+Lepage%2C+E%2E+Denoual%2C+D%2E+Mochihashi%2C+A%2E+Finch%2C+and+E%2E+Sumita%2E+2006%2E+The+NiCT%2DATR+statistical+machine+translation+system+for+the+IWSLT+2006+evaluat)
22. Bing Zhao, Matthias Eck, and Stephan Vogel. 2004. Language model adaptation for statistical machine translation with structured query models. In COLING 2004, Geneva, August. [find](https://scholar.google.com/scholar?q=Bing+Zhao%2C+Matthias+Eck%2C+and+Stephan+Vogel%2E+2004%2E+Language+model+adaptation+for+statistical+machine+translation+with+structured+query+models%2E+In+COLING+2004%2C+Geneva%2C+August%2E)