---
title: "Measuring Machine Translation Errors in New — Irvine 2013"
description: "Statistical MT systems are often trained on large parliamentary corpora and then used on very different kinds of text, where quality drops sharply."
authors: ["Ann Irvine", "John Morgan", "Marine Carpuat", "Hal Daumé III", "Dragos Munteanu"]
year: 2013
venue: "Transactions of the Association for Computational Linguistics"
source: "https://aclanthology.org/Q13-1035/"
doi: "10.1162/tacl_a_00239"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/papers/irvine-2013-measuring-machine-translation-errors-new/"
---
[Elamur Research](https://md.research.elamur.ai/) · [this page in HTML](https://research.elamur.ai/free/papers/irvine-2013-measuring-machine-translation-errors-new/)

[home](https://md.research.elamur.ai/) / [papers](https://md.research.elamur.ai/free/papers/) / [2013](https://md.research.elamur.ai/free/papers/2013/) / Measuring Machine Translation Errors in New Domains

# Measuring Machine Translation Errors in New Domains

**Authors:** Ann Irvine, John Morgan, [Marine Carpuat](https://md.research.elamur.ai/free/authors/marine-carpuat/), [Hal Daumé III](https://md.research.elamur.ai/free/authors/hal-daume-iii/), Dragos Munteanu · **Year:** 2013 · **Venue:** Transactions of the Association for Computational Linguistics · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/Q13-1035/) · [PDF](https://aclanthology.org/Q13-1035.pdf) · [DOI 10.1162/tacl\_a\_00239](https://doi.org/10.1162/tacl_a_00239) · [OpenAlex](https://openalex.org/W2185701500)

**Citations:** 75 citations (openalex-snapshot · 2026-10-14) · [29](#cited-by) in our library, of the 140673 papers whose bibliographies we read

## Summary
Statistical MT systems are often trained on large parliamentary corpora and then used on very different kinds of text, where quality drops sharply. It is rarely clear what causes the drop. Irvine et al. propose a way to diagnose it.

Their S4 taxonomy divides lexical choice errors into four non-overlapping classes:
- **SEEN**: the source word never appeared in training.
- **SENSE**: the word is known, but the required translation was never observed.
- **SCORE**: the correct option existed but a wrong one ranked higher.
- **SEARCH**: the correct option was lost to beam pruning.

Two automatic methods measure these errors. WADE labels and counts errors word by word using word alignments. TETRA estimates how each error type affects corpus-level BLEU. The authors apply both to a French–English Moses system trained on Hansard and tested on four new domains. News behaves almost like the training domain. In the other domains, most of the loss comes from SEEN and SENSE errors. Adding candidate translations does not help unless they are also scored correctly. The work points adaptation research toward targeted remedies such as sense induction.

## Key points
- **Setup**: a phrase-based Moses system, French→English, with Hansard as the only old domain. The new domains are news, medical texts (EMEA), scientific abstracts and movie subtitles.
- **Limited adaptation**: the baseline is changed only through a new-domain English language model and a few thousand new-domain parallel sentences used for tuning.
- **Main finding**: news is a weak test case for domain adaptation because it has very few unseen-word errors. In the other domains, new senses cause roughly as much error as unseen words and are harder to translate correctly.
- **Robustness check**: a French speaker manually corrected the alignments for 955 EMEA test sentences. This produced fewer errors overall, but the errors were spread fairly evenly across categories, which supports WADE's conclusions.
- **Released materials**: data, code and system outputs are available at http://hal3.name/damt. The study also motivated follow-up work on detecting new senses and on learning translations from comparable corpora.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

Fields: [Machine translation](https://md.research.elamur.ai/free/shelves/machine-translation/) · [Natural language processing](https://md.research.elamur.ai/free/shelves/nlp/)

Topics: [Machine translation](https://md.research.elamur.ai/free/topics/machine-translation/)

## Cited by

Cited by

29 papers of our library cite it (of the 140673 whose bibliographies we read); 2 of them have a page here, 27 not yet.

- [Neural versus Phrase-Based Machine Translation Quality: a Case Study](https://md.research.elamur.ai/free/papers/bentivogli-2016-neural-versus-phrase-based-machine/)
- [A Survey of Domain Adaptation for Neural Machine Translation](https://md.research.elamur.ai/free/papers/chu-2018-survey-domain-adaptation-neural-machine/)

## Cite

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@article{irvine2013measuring,
  title = {{Measuring Machine Translation Errors in New Domains}},
  author = {Irvine, Ann and Morgan, John and Carpuat, Marine and III, Hal Daumé and Munteanu, Dragos},
  journal = {Transactions of the Association for Computational Linguistics},
  year = {2013},
  doi = {10.1162/tacl\_a\_00239},
  url = {https://doi.org/10.1162/tacl_a_00239}
}
```

- [← previous MCTest: A Challenge Dataset for the Open-Domain Machine Comprehension of Text](https://md.research.elamur.ai/free/papers/missing-2013-richardson-mctest-challenge-dataset-open-domain/)
- [next → More Constructions, More Genres: Extending Stanford Dependencies](https://md.research.elamur.ai/free/papers/bank-2013-marneffe-more-constructions-more-genres-extending/)
- [↑ back to papers of 2013](https://md.research.elamur.ai/free/papers/2013/)

## References

39 references, 8 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Sankaranarayanan Ananthakrishnan, Rohit Prasad, and Prem Natarajan. 2011. On-line language model biasing for statistical machine translation. In Proceedings of the Conference of the Association for Computational Linguistics (ACL). [find](https://scholar.google.com/scholar?q=Sankaranarayanan+Ananthakrishnan%2C+Rohit+Prasad%2C+and+Prem+Natarajan%2E+2011%2E+On%2Dline+language+model+biasing+for+statistical+machine+translation%2E+In+Proceedings+of+the+Conference+of+the+Association+for+Co)
2. Amittai Axelrod, Xiaodong He, and Jianfeng Gao. 2011. Domain adaptation via pseudo in-domain data selection. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP). [find](https://scholar.google.com/scholar?q=Amittai+Axelrod%2C+Xiaodong+He%2C+and+Jianfeng+Gao%2E+2011%2E+Domain+adaptation+via+pseudo+in%2Ddomain+data+selection%2E+In+Proceedings+of+the+Conference+on+Empirical+Methods+in+Natural+Language+Processing+%28EMNLP) [our card](https://md.research.elamur.ai/free/papers/axelrod-2011-domain-adaptation-pseudo-domain-data/)
3. Nguyen Bach, Fei Huang, and Yaser Al-Onaizan. 2011. Goodness: A method for measuring machine translation confidence. In Proceedings of the Conference of the Association for Computational Linguistics (ACL). [find](https://scholar.google.com/scholar?q=Nguyen+Bach%2C+Fei+Huang%2C+and+Yaser+Al%2DOnaizan%2E+2011%2E+Goodness%3A+A+method+for+measuring+machine+translation+confidence%2E+In+Proceedings+of+the+Conference+of+the+Association+for+Computational+Linguistics+%28)
4. Pratyush Banerjee, Sudip Kumar Naskar, Johann Roturier, Andy Way, and Josef van Genabith. 2012. Translation quality-based supplementary data selection by incremental update of translation models. In Proceedings of the International Conference on Computational Linguistics (COLING). [find](https://scholar.google.com/scholar?q=Pratyush+Banerjee%2C+Sudip+Kumar+Naskar%2C+Johann+Roturier%2C+Andy+Way%2C+and+Josef+van+Genabith%2E+2012%2E+Translation+quality%2Dbased+supplementary+data+selection+by+incremental+update+of+translation+models%2E+In+P)
5. Arianna Bisazza, Nick Ruiz, and Marcello Federico. 2011. Fill-up versus interpolation methods for phrase-based SMT adaptation. International Workshop on Spoken Language Translation (IWSLT). [find](https://scholar.google.com/scholar?q=Arianna+Bisazza%2C+Nick+Ruiz%2C+and+Marcello+Federico%2E+2011%2E+Fill%2Dup+versus+interpolation+methods+for+phrase%2Dbased+SMT+adaptation%2E+International+Workshop+on+Spoken+Language+Translation+%28IWSLT%29%2E) [our card](https://md.research.elamur.ai/free/papers/bisazza-2011-fill-up-versus-interpolation-methods/)
6. John Blatz, Erin Fitzgerald, George Foster, Simona Gandrabur, Cyril Goutte, Alex Kulesza, Alberto Sanchis, and Nicola Ueffing. 2004. Confidence estimation for machine translation. In Proceedings of the International Conference on Computational Linguistics (CO-LING). [find](https://scholar.google.com/scholar?q=John+Blatz%2C+Erin+Fitzgerald%2C+George+Foster%2C+Simona+Gandrabur%2C+Cyril+Goutte%2C+Alex+Kulesza%2C+Alberto+Sanchis%2C+and+Nicola+Ueffing%2E+2004%2E+Confidence+estimation+for+machine+translation%2E+In+Proceedings+of+th)
7. Marine Carpuat and Dekai Wu. 2007. Improving statistical machine translation using word sense disambiguation. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP). [find](https://scholar.google.com/scholar?q=Marine+Carpuat+and+Dekai+Wu%2E+2007%2E+Improving+statistical+machine+translation+using+word+sense+disambiguation%2E+In+Proceedings+of+the+Conference+on+Empirical+Methods+in+Natural+Language+Processing+%28EMNL)
8. Marine Carpuat, Hal Daumé III, Katharine Henry, Ann Irvine, Jagadeesh Jagarlamudi, and Rachel Rudinger. 2013. SenseSpotting: Never let your parallel data tie you to an old domain. In Proceedings of the Conference of the Association for Computational Linguistics (ACL). [find](https://scholar.google.com/scholar?q=Marine+Carpuat%2C+Hal+Daume%CC%81+III%2C+Katharine+Henry%2C+Ann+Irvine%2C+Jagadeesh+Jagarlamudi%2C+and+Rachel+Rudinger%2E+2013%2E+SenseSpotting%3A+Never+let+your+parallel+data+tie+you+to+an+old+domain%2E+In+Proceedings+of+t)
9. Yin-Wen Chang and Michael Collins. 2011. Exact decoding of phrase-based translation models through lagrangian relaxation. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP). [find](https://scholar.google.com/scholar?q=Yin%2DWen+Chang+and+Michael+Collins%2E+2011%2E+Exact+decoding+of+phrase%2Dbased+translation+models+through+lagrangian+relaxation%2E+In+Proceedings+of+the+Conference+on+Empirical+Methods+in+Natural+Language+Proc)
10. Colin Cherry and George Foster. 2012. Batch tuning strategies for statistical machine translation. In Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics (NAACL). [find](https://scholar.google.com/scholar?q=Colin+Cherry+and+George+Foster%2E+2012%2E+Batch+tuning+strategies+for+statistical+machine+translation%2E+In+Proceedings+of+the+Conference+of+the+North+American+Chapter+of+the+Association+for+Computational+L)
11. David Chiang. 2007. Hierarchical phrase-based translation. Computational Linguistics, 33(2):201–228. [↗](https://doi.org/10.1162/coli.2007.33.2.201) [our card](https://md.research.elamur.ai/free/papers/chiang-2007-hierarchical-phrase-based-translation/)
12. Hal Daumé III and Jagadeesh Jagarlamudi. 2011. Domain adaptation for machine translation by mining unseen words. In Proceedings of the Conference of the Association for Computational Linguistics (ACL). [find](https://scholar.google.com/scholar?q=Hal+Daume%CC%81+III+and+Jagadeesh+Jagarlamudi%2E+2011%2E+Domain+adaptation+for+machine+translation+by+mining+unseen+words%2E+In+Proceedings+of+the+Conference+of+the+Association+for+Computational+Linguistics+%28ACL)
13. Kevin Duh, Katsuhito Sudoh, and Hajime Tsukada. 2010. Analysis of translation model adaptation in statistical machine translation. In Proceedings of the International Workshop on Spoken Language Translation (IWSLT). [find](https://scholar.google.com/scholar?q=Kevin+Duh%2C+Katsuhito+Sudoh%2C+and+Hajime+Tsukada%2E+2010%2E+Analysis+of+translation+model+adaptation+in+statistical+machine+translation%2E+In+Proceedings+of+the+International+Workshop+on+Spoken+Language+Trans)
14. George Foster and Roland Kuhn. 2007. Mixture-model adaptation for SMT. In Proceedings of the Workshop on Statistical Machine Translation (WMT). [find](https://scholar.google.com/scholar?q=George+Foster+and+Roland+Kuhn%2E+2007%2E+Mixture%2Dmodel+adaptation+for+SMT%2E+In+Proceedings+of+the+Workshop+on+Statistical+Machine+Translation+%28WMT%29%2E) [our card](https://md.research.elamur.ai/free/papers/foster-2007-mixture-model-adaptation-smt/)
15. George Foster, Cyril Goutte, and Roland Kuhn. 2010. Discriminative instance weighting for domain adaptation in statistical machine translation. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP). [find](https://scholar.google.com/scholar?q=George+Foster%2C+Cyril+Goutte%2C+and+Roland+Kuhn%2E+2010%2E+Discriminative+instance+weighting+for+domain+adaptation+in+statistical+machine+translation%2E+In+Proceedings+of+the+Conference+on+Empirical+Methods+in)
16. Zhengxian Gong, Min Zhang, and Guodong Zhou. 2011. Cache-based document-level statistical machine translation. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP). [find](https://scholar.google.com/scholar?q=Zhengxian+Gong%2C+Min+Zhang%2C+and+Guodong+Zhou%2E+2011%2E+Cache%2Dbased+document%2Dlevel+statistical+machine+translation%2E+In+Proceedings+of+the+Conference+on+Empirical+Methods+in+Natural+Language+Processing+%28EMN)
17. Barry Haddow and Philipp Koehn. 2012. Analysing the effect of out-of-domain data on SMT systems. In Proceedings of the Workshop on Statistical Machine Translation (WMT). [find](https://scholar.google.com/scholar?q=Barry+Haddow+and+Philipp+Koehn%2E+2012%2E+Analysing+the+effect+of+out%2Dof%2Ddomain+data+on+SMT+systems%2E+In+Proceedings+of+the+Workshop+on+Statistical+Machine+Translation+%28WMT%29%2E)
18. Almut Silja Hildebrand, Matthias Eck, Stephan Vogel, and Alex Waibel. 2005. Adaptation of the translation model for statistical machine translation based on information retrieval. In Proceedings of the Conference of the European Association for Computational Linguistics (EACL). [↗](https://doi.org/10.5445/ir/1000182997) [our card](https://md.research.elamur.ai/free/papers/hildebrand-2005-adaptation-translation-model-statistical-machine/)
19. Ann Irvine, Chris Quirk, and Hal Daumé III. 2013. Monolingual marginal matching for translation model adaptation. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP). [find](https://scholar.google.com/scholar?q=Ann+Irvine%2C+Chris+Quirk%2C+and+Hal+Daume%CC%81+III%2E+2013%2E+Monolingual+marginal+matching+for+translation+model+adaptation%2E+In+Proceedings+of+the+Conference+on+Empirical+Methods+in+Natural+Language+Processing+)
20. Philipp Koehn and Josh Schroeder. 2007. Experiments in domain adaptation for statistical machine translation. In Proceedings of the Workshop on Statistical Machine Translation (WMT). [find](https://scholar.google.com/scholar?q=Philipp+Koehn+and+Josh+Schroeder%2E+2007%2E+Experiments+in+domain+adaptation+for+statistical+machine+translation%2E+In+Proceedings+of+the+Workshop+on+Statistical+Machine+Translation+%28WMT%29%2E) [our card](https://md.research.elamur.ai/free/papers/koehn-2007-experiments-domain-adaptation-statistical-machine/)
21. Philipp Koehn, Amittai Axelrod, Alexandra Birch, Chris Callison-Burch, Miles Osborne, David Talbot, and Michael White. 2005. Edinburgh system description for the 2005 IWSLT speech translation evaluation. In Proceedings of International Workshop on Spoken Language Translation. [find](https://scholar.google.com/scholar?q=Philipp+Koehn%2C+Amittai+Axelrod%2C+Alexandra+Birch%2C+Chris+Callison%2DBurch%2C+Miles+Osborne%2C+David+Talbot%2C+and+Michael+White%2E+2005%2E+Edinburgh+system+description+for+the+2005+IWSLT+speech+translation+evaluati) [our card](https://md.research.elamur.ai/free/papers/koehn-2005-edinburgh-system-description-2005-iwslt/)
22. Philipp Koehn, Hieu Hoang, Alexandra Birch, Chris Callison-Burch, Marcello Federico, Nicola Bertoldi, Brooke Cowan, Wade Shen, Christine Moran, Richard Zens, Chris Dyer, Ondrej Bojar, Alexandra Constantin, and Evan Herbst. 2007. Moses: Open source toolkit for statistical machine translation. In Proceedings of the Conference of the Association for Computational Linguistics (ACL). [find](https://scholar.google.com/scholar?q=Philipp+Koehn%2C+Hieu+Hoang%2C+Alexandra+Birch%2C+Chris+Callison%2DBurch%2C+Marcello+Federico%2C+Nicola+Bertoldi%2C+Brooke+Cowan%2C+Wade+Shen%2C+Christine+Moran%2C+Richard+Zens%2C+Chris+Dyer%2C+Ondrej+Bojar%2C+Alexandra+Consta)
23. Patrik Lambert, Holger Schwenk, and Frédéric Blain. 2012. Automatic translation of scientific documents in the HAL archive. In Proceedings of the International Conference on Language Resourcesand Evaluation (LREC). [find](https://scholar.google.com/scholar?q=Patrik+Lambert%2C+Holger+Schwenk%2C+and+Fre%CC%81de%CC%81ric+Blain%2E+2012%2E+Automatic+translation+of+scientific+documents+in+the+HAL+archive%2E+In+Proceedings+of+the+International+Conference+on+Language+Resourcesand+Ev)
24. Thomas Lavergne, Alexandre Allauzen, Hai-Son Le, and François Yvon. 2011. LIMSI’s experiments in domain adaptation for IWSLT11. In Proceedings of the International Workshop on Spoken Language Translation (IWSLT). [find](https://scholar.google.com/scholar?q=Thomas+Lavergne%2C+Alexandre+Allauzen%2C+Hai%2DSon+Le%2C+and+Franc%CC%A7ois+Yvon%2E+2011%2E+LIMSI%E2%80%99s+experiments+in+domain+adaptation+for+IWSLT11%2E+In+Proceedings+of+the+International+Workshop+on+Spoken+Language+Transla)
25. Yajuan Lu, Jin Huang, and Qun Liu. 2007. Improving statistical machine translation performance by training data selection and optimization. In Proceedings of the Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning (EMNLP-CoNLL). [find](https://scholar.google.com/scholar?q=Yajuan+Lu%2C+Jin+Huang%2C+and+Qun+Liu%2E+2007%2E+Improving+statistical+machine+translation+performance+by+training+data+selection+and+optimization%2E+In+Proceedings+of+the+Joint+Conference+on+Empirical+Methods+)
26. Saab Mansour, Joern Wuebker, and Hermann Ney. 2011. Combining translation and language model scoring for domain-specific data filtering. In Proceedings of the International Workshop on Spoken Language Translation (IWSLT). [find](https://scholar.google.com/scholar?q=Saab+Mansour%2C+Joern+Wuebker%2C+and+Hermann+Ney%2E+2011%2E+Combining+translation+and+language+model+scoring+for+domain%2Dspecific+data+filtering%2E+In+Proceedings+of+the+International+Workshop+on+Spoken+Language)
27. Spyros Matsoukas, Antti-Veikko I. Rosti, and Bing Zhang. 2009. Discriminative corpus weight estimation for machine translation. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP). [find](https://scholar.google.com/scholar?q=Spyros+Matsoukas%2C+Antti%2DVeikko+I%2E+Rosti%2C+and+Bing+Zhang%2E+2009%2E+Discriminative+corpus+weight+estimation+for+machine+translation%2E+In+Proceedings+of+the+Conference+on+Empirical+Methods+in+Natural+Languag)
28. Rada Mihalcea, Ravi Sinha, and Diana McCarthy. 2010. SemEval-2010 Task 2: Cross-Lingual Lexical Substitution. In Proceedings of the 5th International Workshop on Semantic Evaluation. [find](https://scholar.google.com/scholar?q=Rada+Mihalcea%2C+Ravi+Sinha%2C+and+Diana+McCarthy%2E+2010%2E+SemEval%2D2010+Task+2%3A+Cross%2DLingual+Lexical+Substitution%2E+In+Proceedings+of+the+5th+International+Workshop+on+Semantic+Evaluation%2E)
29. Jan Niehues and Alex Waibel. 2010. Domain adaptation in statistical machine translation using factored translation models. In Proceedings of the European Association for Machine Translation (EAMT). [↗](https://doi.org/10.5445/ir/1000030254)
30. Maja Popović and Hermann Ney. 2011. Towards automatic error analysis of machine translation output. Computational Linguistics, 37(4). [↗](https://doi.org/10.1162/coli_a_00072)
31. Majid Razmara, George Foster, Baskaran Sankaran, and Anoop Sarkar. 2012. Mixing multiple translation models in statistical machine translation. In Proceedings of the Conference of the Association for Computational Linguistics (ACL). [find](https://scholar.google.com/scholar?q=Majid+Razmara%2C+George+Foster%2C+Baskaran+Sankaran%2C+and+Anoop+Sarkar%2E+2012%2E+Mixing+multiple+translation+models+in+statistical+machine+translation%2E+In+Proceedings+of+the+Conference+of+the+Association+for+)
32. Rico Sennrich. 2012. Perplexity minimization for translation model domain adaptation in statistical machine translation. In Proceedings of the Conference of the European Association for Computational Linguistics (EACL). [find](https://scholar.google.com/scholar?q=Rico+Sennrich%2E+2012%2E+Perplexity+minimization+for+translation+model+domain+adaptation+in+statistical+machine+translation%2E+In+Proceedings+of+the+Conference+of+the+European+Association+for+Computational+)
33. Jörg Tiedemann. 2009. News from OPUS - A collection of multilingual parallel corpora with tools and interfaces. In N. Nicolov, K. Bontcheva, G. Angelova, and R. Mitkov, editors, Recent Advances in Natural Language Processing (RANLP). [↗](https://doi.org/10.1075/cilt.309.19tie)
34. Jörg Tiedemann. 2010. To cache or not to cache ? experiments with adaptive models in statistical machine translation. In Proceedings of the ACL Workshop on Statistical Machine Translation and Metrics (MATR). [find](https://scholar.google.com/scholar?q=Jo%CC%88rg+Tiedemann%2E+2010%2E+To+cache+or+not+to+cache+%3F+experiments+with+adaptive+models+in+statistical+machine+translation%2E+In+Proceedings+of+the+ACL+Workshop+on+Statistical+Machine+Translation+and+Metrics)
35. Kristina Toutanova, Dan Klein, Christopher Manning, and Yoram Singer. 2003. Feature-rich part-of-speech tagging with a cyclic dependency network. In NAACL. [find](https://scholar.google.com/scholar?q=Kristina+Toutanova%2C+Dan+Klein%2C+Christopher+Manning%2C+and+Yoram+Singer%2E+2003%2E+Feature%2Drich+part%2Dof%2Dspeech+tagging+with+a+cyclic+dependency+network%2E+In+NAACL%2E)
36. David Vilar, Jia Xu, Luis Fernando D’Haro, and Hermann Ney. 2006. Error analysis of statistical machine translation output. In Proceedings of the International Conference on Language Resourcesand Evaluation (LREC). [find](https://scholar.google.com/scholar?q=David+Vilar%2C+Jia+Xu%2C+Luis+Fernando+D%E2%80%99Haro%2C+and+Hermann+Ney%2E+2006%2E+Error+analysis+of+statistical+machine+translation+output%2E+In+Proceedings+of+the+International+Conference+on+Language+Resourcesand+Eval) [our card](https://md.research.elamur.ai/free/papers/vilar-2006-error-analysis-statistical-machine-translation/)
37. Stephan Vogel, Hermann Ney, and Christoph Tillmann. 1996. HMM-based word alignment in statistical translation. In Proceedings of the International Conference on Computational Linguistics (COLING). [find](https://scholar.google.com/scholar?q=Stephan+Vogel%2C+Hermann+Ney%2C+and+Christoph+Tillmann%2E+1996%2E+HMM%2Dbased+word+alignment+in+statistical+translation%2E+In+Proceedings+of+the+International+Conference+on+Computational+Linguistics+%28COLING%29%2E)
38. Guillaume Wisniewski, Alexandre Allauzen, and François Yvon. 2010. Assessing phrase-based translation models with oracle decoding. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP). [find](https://scholar.google.com/scholar?q=Guillaume+Wisniewski%2C+Alexandre+Allauzen%2C+and+Franc%CC%A7ois+Yvon%2E+2010%2E+Assessing+phrase%2Dbased+translation+models+with+oracle+decoding%2E+In+Proceedings+of+the+Conference+on+Empirical+Methods+in+Natural+Lan)
39. Bing Zhao, Matthias Eck, and Stephan Vogel. 2004. Language model adaptation for statistical machine translation with structured query models. In Proceedings of the International Conference on Computational Linguistics (COLING). [↗](https://doi.org/10.5445/ir/1000166436)