--- title: "Statistical Phrase-Based Post-Editing — Simard 2007" description: "The paper addresses the repetitive, costly manual correction of rule-based MT output." authors: ["Michel Simard", "Cyril Goutte", "Pierre Isabelle"] year: 2007 venue: "Human Language Technologies 2007: The Conference of the North American Chapter of the Association for Computational Linguistics; Proceedings of the Main Conference" source: "https://aclanthology.org/N07-1064/" licence: "CC BY-NC-SA 3.0 (ACL Anthology, before 2016)" licence_url: "https://creativecommons.org/licenses/by-nc-sa/3.0/" summary_licence: "Elamur Cosmic License" canonical: "https://research.elamur.ai/free/en/papers/simard-2007-statistical-phrase-based-post-editing/" --- [Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/simard-2007-statistical-phrase-based-post-editing/) [home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2007](https://md.research.elamur.ai/free/en/papers/2007/) / Statistical Phrase-Based Post-Editing # Statistical Phrase-Based Post-Editing **Authors:** [Michel Simard](https://md.research.elamur.ai/free/en/authors/michel-simard/), [Cyril Goutte](https://md.research.elamur.ai/free/en/authors/cyril-goutte/), [Pierre Isabelle](https://md.research.elamur.ai/free/en/authors/pierre-isabelle/) · **Year:** 2007 · **Venue:** Human Language Technologies 2007: The Conference of the North American Chapter of the Association for Computational Linguistics; Proceedings of the Main Conference · **Licence:** [CC BY-NC-SA 3.0 (ACL Anthology, before 2016)](https://creativecommons.org/licenses/by-nc-sa/3.0/) **Links:** [Publisher's page](https://aclanthology.org/N07-1064/) · [PDF](https://aclanthology.org/N07-1064.pdf) · [OpenAlex](https://openalex.org/W2141971526) **Citations:** 164 citations (openalex-snapshot · 2026-10-14) · [106](#cited-by) in our library, of the 140884 papers whose bibliographies we read ## Summary The paper addresses the repetitive, costly manual correction of rule-based MT output. The setting is Canada's Job Bank, where job ads must appear in both French and English. The authors treat post-editing as a translation problem. They train Portage, NRC's phrase-based statistical MT system, to map raw output from a commercial rule-based MT system onto the versions human post-editors actually published. The training data is a tri-parallel corpus from HRSDC of source text, rule-based MT and post-edited text, with fewer than one million words per language. TER is the main metric because it closely mirrors post-editing effort, and BLEU is also reported. The automatic post-editing (APE) system beats both the rule-based baseline and a standalone Portage system trained on the same Job Bank data. BLEU rises by up to 50% relative and TER falls by about one third. The result shows a simple, practical way to combine rule-based and statistical MT and to adapt an existing system to a new domain. Learning curves suggest, however, that direct phrase-based translation may eventually overtake APE as training data grows. ## Key points - **Data:** Job Bank receives over one million ads a year, about 180M words. The HRSDC sample contains blocks of three parts: source, rule-based MT and post-edited text. Blocks are very short, with a median of 9 source tokens (French-to-English) and 7 (English-to-French), so they were not split further. - **Baseline quality:** The rule-based output has a global TER of 58.77% (fr→en) and 53.33% (en→fr). Quality varies widely across blocks: roughly 12% exceed 100% TER, while over 10% need no edits at all. - **Portage setup:** The model is a log-linear combination of n-gram language models, phrase tables with Good-Turing smoothing, a distortion model and a length feature. Weights are tuned with Och's MERT on BLEU, followed by n-best rescoring. - **Main result:** With under 500k training words, standalone Portage already beats the rule-based system. Portage APE applied to the rule-based output beats both. - **Control experiments:** - Post-editing Portage output with an APE trained on the same Job Bank data brings no gain, and also no loss. - Portage trained on the Hansard (over 85M words) performs poorly on Job Bank. APE on top of it helps a lot but still falls short of the in-domain Portage system. - When Hansard and Job Bank components are merged in one model, the Hansard components receive negligible weights. ## Interesting ideas - **Post-editing as monolingual translation:** A standard SMT toolkit can learn to "translate" raw MT output into its corrected form. Any organisation that keeps logs of MT output alongside human corrections could reuse those logs this way to automate fixes for recurring errors. - **An APE module as a domain adapter:** A separate correction layer can be placed on top of a black-box or out-of-domain system that cannot itself be retrained. This makes it possible to customise commercial engines beyond what user dictionaries allow. - **Choosing the metric to fit the goal:** The authors use TER because it approximates human editing effort, rather than arguing about which metric best measures quality. They also note that block-level TER distributions expose quality differences that a single global score hides. Both practices are useful for evaluating systems that assist human workflows. - **Learning curves against baselines:** Plotting APE and direct translation against training size shows that the combined approach wins when data is scarce but grows more slowly. This kind of analysis is a valuable check before committing to a pipeline architecture. - **Giving the post-editor the source text:** Because the front-end MT loses information, the authors propose letting the APE also see the source-language input. Outputs from several MT systems could serve as secondary inputs, turning the post-editor into a system-combination model. *Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.* Fields: [Natural language processing](https://md.research.elamur.ai/free/en/shelves/nlp/) Topics: [Machine translation](https://md.research.elamur.ai/free/en/topics/machine-translation/) ## Cited by Cited by 106 papers of our library cite it (of the 140884 whose bibliographies we read); 8 of them have a page here, 98 not yet. - [Findings of the 2015 Workshop on Statistical Machine Translation](https://md.research.elamur.ai/free/en/papers/bojar-2015-findings-2015-workshop-statistical-machine/) - [Findings of the 2016 Conference on Machine Translation](https://md.research.elamur.ai/free/en/papers/bojar-2016-findings-2016-conference-machine-translation/) - [Findings of the 2017 Conference on Machine Translation (WMT17)](https://md.research.elamur.ai/free/en/papers/bojar-2017-findings-2017-conference-machine-translation/) - [Findings of the WMT 2019 Shared Task on Automatic Post-Editing](https://md.research.elamur.ai/free/en/papers/chatterjee-2019-findings-wmt-2019-shared-task/) - [Iterative Translation Refinement with Large Language Models](https://md.research.elamur.ai/free/en/papers/chen-2024-iterative-translation-refinement-large-language/) - [Log-linear Combinations of Monolingual and Bilingual Neural Machine Translation Models for Automatic Post-Editing](https://md.research.elamur.ai/free/en/papers/dowmunt-2016-log-linear-combinations-monolingual-bilingual/) - [Enhancing function word translation with syntax-based statistical post-editing](https://md.research.elamur.ai/free/en/papers/richardson-2015-enhancing-function-word-translation-syntax/) - [Diagnose, Then Repair: A Two-Stage MQM-Guided Post-Editing Framework for Domain-Specific Machine Translation](https://md.research.elamur.ai/free/en/papers/wang-2026-diagnose-then-repair-two-stage/) ## Cite The paper itself, not this page. BibTeX from the library's record of the work. ```bibtex @inproceedings{simard2007statistical, title = {{Statistical Phrase-Based Post-Editing}}, author = {Simard, Michel and Goutte, Cyril and Isabelle, Pierre}, booktitle = {Human Language Technologies 2007: The Conference of the North American Chapter of the Association for Computational Linguistics; Proceedings of the Main Conference}, year = {2007}, url = {https://aclanthology.org/N07-1064/} } ``` - [← previous Smoothing a Lexicon-based POS Tagger for Arabic and Hebrew](https://md.research.elamur.ai/free/en/papers/aclbulk-2007-manour-smoothing-lexicon-based-pos-tagger/) - [next → SVM Model Tampering and Anchored Learning: A Case Study in Hebrew NP Chunking](https://md.research.elamur.ai/free/en/papers/aclbulk-2007-goldberg-svm-model-tampering-anchored-learning/) - [↑ back to papers of 2007](https://md.research.elamur.ai/free/en/papers/2007/) ## References 12 references, 2 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet. 1. Jeffrey Allen and Christofer Hogan. 2000. Toward the development of a post-editing module for Machine Translation raw output: a new productivity tool for processing controlled language. In Third Inter-As a trivial example, imagine an MT system that “deletes” out-of-vocabulary words. national Control… [find](https://scholar.google.com/scholar?q=Jeffrey+Allen+and+Christofer+Hogan%2E+2000%2E+Toward+the+development+of+a+post%2Dediting+module+for+Machine+Translation+raw+output%3A+a+new+productivity+tool+for+processing+controlled+language%2E+In+Third+Inter) 2. Jeffrey Allen. 2004. Case study: Implementing MT for the translation of pre-sales marketing and post-sales software deployment documentation. In Proceedings of AMTA-2004, pages 1–6, Washington, USA. [find](https://scholar.google.com/scholar?q=Jeffrey+Allen%2E+2004%2E+Case+study%3A+Implementing+MT+for+the+translation+of+pre%2Dsales+marketing+and+post%2Dsales+software+deployment+documentation%2E+In+Proceedings+of+AMTA%2D2004%2C+pages+1%E2%80%936%2C+Washington%2C+USA%2E) 3. Peter F Brown, Stephen A Della Pietra, Vincent J Della Pietra, and Robert L Mercer. 1993. The Mathematics of Statistical Machine Translation: Parameter Estimation. Computational Linguistics, 19(2):263–311. [find](https://scholar.google.com/scholar?q=Peter+F+Brown%2C+Stephen+A+Della+Pietra%2C+Vincent+J+Della+Pietra%2C+and+Robert+L+Mercer%2E+1993%2E+The+Mathematics+of+Statistical+Machine+Translation%3A+Parameter+Estimation%2E+Computational+Linguistics%2C+19%282%29%3A263) 4. Jakob Elming. 2006. Transformation-based corrections of rule-based MT. In Proceedings of the EAMT 11th Annual Conference, Oslo, Norway. [find](https://scholar.google.com/scholar?q=Jakob+Elming%2E+2006%2E+Transformation%2Dbased+corrections+of+rule%2Dbased+MT%2E+In+Proceedings+of+the+EAMT+11th+Annual+Conference%2C+Oslo%2C+Norway%2E) 5. George Foster, Roland Kuhn, and Howard Johnson. 2006. Phrasetable Smoothing for Statistical Machine Translation. In Proceedings of EMNLP 2006, pages 53–61, Sydney, Australia. [find](https://scholar.google.com/scholar?q=George+Foster%2C+Roland+Kuhn%2C+and+Howard+Johnson%2E+2006%2E+Phrasetable+Smoothing+for+Statistical+Machine+Translation%2E+In+Proceedings+of+EMNLP+2006%2C+pages+53%E2%80%9361%2C+Sydney%2C+Australia%2E) 6. Kevin Knight and Ishwar Chander. 1994. Automated Postediting of Documents. In Proceedings of National Conference on Artificial Intelligence, pages 779–784, Seattle, USA. [find](https://scholar.google.com/scholar?q=Kevin+Knight+and+Ishwar+Chander%2E+1994%2E+Automated+Postediting+of+Documents%2E+In+Proceedings+of+National+Conference+on+Artificial+Intelligence%2C+pages+779%E2%80%93784%2C+Seattle%2C+USA%2E) 7. Philipp Koehn, Franz J. Och, and Daniel Marcu. 2003. Statistical Phrase-Based Translation. In Proceedings of HLT-NAACL 2003, pages 127–133, Edmonton, Canada. [find](https://scholar.google.com/scholar?q=Philipp+Koehn%2C+Franz+J%2E+Och%2C+and+Daniel+Marcu%2E+2003%2E+Statistical+Phrase%2DBased+Translation%2E+In+Proceedings+of+HLT%2DNAACL+2003%2C+pages+127%E2%80%93133%2C+Edmonton%2C+Canada%2E) 8. Philipp Koehn. 2004. Pharaoh: a Beam Search Decoder for Phrase-Based Statistical Machine Translation Models. In Proceedings of AMTA 2004, pages 115–124, Washington, USA. [find](https://scholar.google.com/scholar?q=Philipp+Koehn%2E+2004%2E+Pharaoh%3A+a+Beam+Search+Decoder+for+Phrase%2DBased+Statistical+Machine+Translation+Models%2E+In+Proceedings+of+AMTA+2004%2C+pages+115%E2%80%93124%2C+Washington%2C+USA%2E) 9. Daniel Marcu and William Wong. 2002. A Phrase-Based, Joint Probability Model for Statistical Machine Translation. In Proceedings of EMNLP 2002, Philadelphia, USA. [find](https://scholar.google.com/scholar?q=Daniel+Marcu+and+William+Wong%2E+2002%2E+A+Phrase%2DBased%2C+Joint+Probability+Model+for+Statistical+Machine+Translation%2E+In+Proceedings+of+EMNLP+2002%2C+Philadelphia%2C+USA%2E) [our card](https://md.research.elamur.ai/free/en/papers/marcu-2002-phrase-based-joint-probability-model/) 10. Franz Josef Och. 2003. Minimum error rate training in Statistical Machine Translation. In Proceedings of ACL-2003, pages 160–167, Sapporo, Japan. [find](https://scholar.google.com/scholar?q=Franz+Josef+Och%2E+2003%2E+Minimum+error+rate+training+in+Statistical+Machine+Translation%2E+In+Proceedings+of+ACL%2D2003%2C+pages+160%E2%80%93167%2C+Sapporo%2C+Japan%2E) 11. Fatiha Sadat, Howard Johnson, Akakpo Agbago, George Foster, Roland Kuhn, Joel Martin, and Aaron Tikuisis. 2005. PORTAGE: A Phrase-Based Machine Translation System. In Proceedings of the ACL Workshop on Building and Using Parallel Texts, pages 129–132, Ann Arbor, USA. [find](https://scholar.google.com/scholar?q=Fatiha+Sadat%2C+Howard+Johnson%2C+Akakpo+Agbago%2C+George+Foster%2C+Roland+Kuhn%2C+Joel+Martin%2C+and+Aaron+Tikuisis%2E+2005%2E+PORTAGE%3A+A+Phrase%2DBased+Machine+Translation+System%2E+In+Proceedings+of+the+ACL+Workshop+o) 12. Matthew Snover, Bonnie Dorr, Richard Schwartz, Linnea Micciulla, and John Makhoul. 2006. A Study of Translation Edit Rate with Targeted Human Annotation. In Proceedings of AMTA-2006, Cambridge, USA. [find](https://scholar.google.com/scholar?q=Matthew+Snover%2C+Bonnie+Dorr%2C+Richard+Schwartz%2C+Linnea+Micciulla%2C+and+John+Makhoul%2E+2006%2E+A+Study+of+Translation+Edit+Rate+with+Targeted+Human+Annotation%2E+In+Proceedings+of+AMTA%2D2006%2C+Cambridge%2C+USA%2E) [our card](https://md.research.elamur.ai/free/en/papers/snover-2006-study-translation-edit-rate-targeted/)