---
title: "A Joint Sequence Translation Model with Integrated Reordering"
description: "Durrani, Schmid and Fraser propose an alternative to phrase-based statistical MT. It describes… 22 references, cited by 72 papers here."
authors: ["Nadir Durrani", "Helmut Schmid", "Alexander Fraser"]
year: 2011
venue: "Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies"
source: "https://aclanthology.org/P11-1105/"
licence: "CC BY-NC-SA 3.0 (ACL Anthology, before 2016)"
licence_url: "https://creativecommons.org/licenses/by-nc-sa/3.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/durrani-2011-joint-sequence-translation-model-integrated/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/durrani-2011-joint-sequence-translation-model-integrated/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2011](https://md.research.elamur.ai/free/en/papers/2011/) / A Joint Sequence Translation Model with Integrated Reordering

# A Joint Sequence Translation Model with Integrated Reordering

**Authors:** [Nadir Durrani](https://md.research.elamur.ai/free/en/authors/nadir-durrani/), [Helmut Schmid](https://md.research.elamur.ai/free/en/authors/helmut-schmid/) and [Alexander Fraser](https://md.research.elamur.ai/free/en/authors/alexander-fraser/) · **Year:** 2011 · **Venue:** [Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies](https://md.research.elamur.ai/free/en/venues/acl/) ([ACL 2011](https://md.research.elamur.ai/free/en/venues/acl/2011/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY-NC-SA 3.0 (ACL Anthology, before 2016)](https://creativecommons.org/licenses/by-nc-sa/3.0/)

**Links:** [Publisher's page](https://aclanthology.org/P11-1105/) · [PDF](https://aclanthology.org/P11-1105.pdf) · [OpenAlex](https://openalex.org/W2116792345)

**Citations:** 103 citations (OpenAlex, 2026-10-14) · [72](https://md.research.elamur.ai/free/en/papers/durrani-2011-joint-sequence-translation-model-integrated/cited-by/) in our library, of the 141,840 papers whose bibliographies we read

## Summary
Durrani, Schmid and Fraser propose an alternative to phrase-based statistical MT. It describes translation as one left-to-right chain of operations that builds source and target sentences together. The operations do two jobs. Some generate words: they emit minimal bilingual units (cepts), continue a discontinuous source cept, or emit a source word with no translation. Others reorder: they insert a gap, jump back to an open gap, or jump forward. An n-gram model over this operation chain scores translation and reordering jointly, so each kind of decision can condition on the other. The model is first stated as a generative story and then used inside a discriminative framework. The work builds on N-gram SMT, but it explores all reorderings during search rather than a small precomputed set. It also relies only on lexical cues, with no POS tags. On German-, Spanish- and French-to-English, the system beats Moses significantly for German and Spanish and matches it for French. This shows that small translation units combined with explicit reordering operations can rival phrase-based MT, including on long-range verb movement.

## Key points
- The model uses 4 translation operations and 3 reordering operations (Insert Gap, Jump Back (W), Jump Forward). English is produced strictly in order, while German words are placed alongside their translations.
- The baselines are two Moses setups: one with no reordering limit and one with the default distortion limit of 6. Removing the limit costs Moses more than one BLEU point.
- The best system has no hard reordering limit. It gains significantly over both Moses baselines (p \< 0.05, pairwise bootstrap resampling with 1000 samples) for German- and Spanish-to-English.
- Gappy source tuples did not improve BLEU because they inflate the set of translation options. For German, the 10-best options numbered 965,515 with gaps versus 256,992 without. The future cost estimates for these tuples were also unreliable.
- Target-side discontinuities are removed from the training alignments by post-editing. The decoder and a binary of the corpus conversion tool were released.

## Interesting ideas
- **Reordering as explicit tokens in the sequence.** Gaps and jumps become ordinary symbols that a sequence model can predict. Lexical context, such as an auxiliary verb, can then trigger a reordering, and short and long movements are handled by one mechanism. This framing could carry over to other monotone-generation tasks that need occasional reordering.
- **Gaps as lexicalized placeholders.** An open gap behaves like a nonterminal in hierarchical models, yet no grammar is required. A learned pattern like "hat … gelesen – read" therefore generalizes to unseen fillers.
- **Minimal units with a unique segmentation.** Phrase-based MT lets many segmentations of the same sentence pair compete. Small units with exactly one segmentation per sentence pair avoid this spurious ambiguity. Frequent phrases can still be memorized as likely operation sequences.
- **Pruning composite units by a cost check.** A gappy tuple is discarded if its estimated cost is lower than the sum of its parts' best scores. This shrinks the search space and recovered most of the lost quality. A similar test could prune combinatorial candidate sets elsewhere.
- **Inspecting the search, not just the score.** When gappy units failed to help, the authors counted the translation options and examined future-cost errors to locate the cause. This is a useful habit whenever a richer representation does not improve results.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [University of Stuttgart](https://md.research.elamur.ai/free/en/universities/university-of-stuttgart/) — University, Stuttgart, Germany · 75 works · 46,113 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 9,086 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 8,889 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 9,296 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 9,206 works

🧭 Topics (1): [🔁 Machine translation](https://md.research.elamur.ai/free/en/topics/machine-translation/) · 958 works

## Cite

Cite as: Nadir Durrani, Helmut Schmid and Alexander Fraser. 2011. A Joint Sequence Translation Model with Integrated Reordering. Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{durrani2011joint,
  title = {{A Joint Sequence Translation Model with Integrated Reordering}},
  author = {Nadir Durrani, Helmut Schmid and Alexander Fraser},
  booktitle = {Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies},
  year = {2011},
  url = {https://aclanthology.org/P11-1105/}
}
```

- [← previous Analyzing and Integrating Dependency Parsers](https://md.research.elamur.ai/free/en/papers/mcdonald-2011-analyzing-integrating-dependency-parsers/)
- [next → Unsupervised Learning of Morphology](https://md.research.elamur.ai/free/en/papers/hammarstrom-2011-unsupervised-learning-morphology/)
- [↑ back to papers of 2011](https://md.research.elamur.ai/free/en/papers/2011/)

## References

22 references, 6 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Peter F. Brown, Stephen A. Della Pietra, Vincent J. Della Pietra, and R. L. Mercer. 1993. The mathematics of statistical machine translation: parameter estimation. Computational Linguistics, 19(2):263–311. [find](https://scholar.google.com/scholar?q=Peter+F%2E+Brown%2C+Stephen+A%2E+Della+Pietra%2C+Vincent+J%2E+Della+Pietra%2C+and+R%2E+L%2E+Mercer%2E+1993%2E+The+mathematics+of+statistical+machine+translation%3A+parameter+estimation%2E+Computational+Linguistics%2C+19%282%29%3A263)
2. David Chiang. 2007. [Hierarchical phrase-based translation](https://md.research.elamur.ai/free/en/papers/chiang-2007-hierarchical-phrase-based-translation/). Computational Linguistics, 33(2):201–228.
3. Josep Maria Crego and Franois Yvon. 2009. Gappy translation units under left-to-right smt decoding. In Proceedings of the meeting of the European Association for Machine Translation (EAMT), pages 66–73, Barcelona, Spain. [find](https://scholar.google.com/scholar?q=Josep+Maria+Crego+and+Franois+Yvon%2E+2009%2E+Gappy+translation+units+under+left%2Dto%2Dright+smt+decoding%2E+In+Proceedings+of+the+meeting+of+the+European+Association+for+Machine+Translation+%28EAMT%29%2C+pages+66%E2%80%937)
4. Josep Maria Crego and François Yvon. 2010. Improving reordering with linguistically informed bilingual n-grams. In Coling 2010: Posters, pages 197–205, Beijing, China, August. [find](https://scholar.google.com/scholar?q=Josep+Maria+Crego+and+Franc%CC%A7ois+Yvon%2E+2010%2E+Improving+reordering+with+linguistically+informed+bilingual+n%2Dgrams%2E+In+Coling+2010%3A+Posters%2C+pages+197%E2%80%93205%2C+Beijing%2C+China%2C+August%2E)
5. Coling 2010 Organizing Committee. [find](https://scholar.google.com/scholar?q=Coling+2010+Organizing+Committee%2E)
6. Josep M. Crego, Marta R. Costa-juss, Jos B. Mario, and Jos A. R. Fonollosa. 2005a. Ngram-based versus phrasebased statistical machine translation. In In Proceedings of the International Workshop on Spoken Language Technology (IWSLT05, pages 177–184. [find](https://scholar.google.com/scholar?q=Josep+M%2E+Crego%2C+Marta+R%2E+Costa%2Djuss%2C+Jos+B%2E+Mario%2C+and+Jos+A%2E+R%2E+Fonollosa%2E+2005a%2E+Ngram%2Dbased+versus+phrasebased+statistical+machine+translation%2E+In+In+Proceedings+of+the+International+Workshop+on+Sp)
7. Josep M. Crego, José B. Marin̂o, and Adrià de Gispert. 2005b. Reordered search and unfolding tuples for ngram-based SMT. In Proceedings of the 10th Machine Translation Summit (MT Summit X), pages 283– 289, Phuket, Thailand. [find](https://scholar.google.com/scholar?q=Josep+M%2E+Crego%2C+Jose%CC%81+B%2E+Marin%CC%82o%2C+and+Adria%CC%80+de+Gispert%2E+2005b%2E+Reordered+search+and+unfolding+tuples+for+ngram%2Dbased+SMT%2E+In+Proceedings+of+the+10th+Machine+Translation+Summit+%28MT+Summit+X%29%2C+pages+28)
8. Michel Galley and Christopher D. Manning. 2010. Accurate non-hierarchical phrase-based translation. In Human Language Technologies: The 2010 Annual Conference of the North American Chapter of the Association for Computational Linguistics, pages 966– 974, Los Angeles, California, June. Association fo… [find](https://scholar.google.com/scholar?q=Michel+Galley+and+Christopher+D%2E+Manning%2E+2010%2E+Accurate+non%2Dhierarchical+phrase%2Dbased+translation%2E+In+Human+Language+Technologies%3A+The+2010+Annual+Conference+of+the+North+American+Chapter+of+the+Asso)
9. Philipp Koehn and Barry Haddow. 2009. Edinburgh’s submission to all tracks of the WMT 2009 shared task with reordering and speed improvements to Moses. In Proceedings of the Fourth Workshop on Statistical Machine Translation, pages 160–164, Athens, Greece, March. Association for Computational Lingui… [find](https://scholar.google.com/scholar?q=Philipp+Koehn+and+Barry+Haddow%2E+2009%2E+Edinburgh%E2%80%99s+submission+to+all+tracks+of+the+WMT+2009+shared+task+with+reordering+and+speed+improvements+to+Moses%2E+In+Proceedings+of+the+Fourth+Workshop+on+Statist)
10. Philipp Koehn, Franz J. Och, and Daniel Marcu. 2003. Statistical phrase-based translation. In Proceedings of the Human Language Technology and North American Association for Computational Linguistics Conference, pages 127–133, Edmonton, Canada. [find](https://scholar.google.com/scholar?q=Philipp+Koehn%2C+Franz+J%2E+Och%2C+and+Daniel+Marcu%2E+2003%2E+Statistical+phrase%2Dbased+translation%2E+In+Proceedings+of+the+Human+Language+Technology+and+North+American+Association+for+Computational+Linguistics+)
11. Philipp Koehn, Amittai Axelrod, Alexandra Birch Mayne, Chris Callison-Burch, Miles Osborne, and David Talbot. 2005. [Edinburgh system description for the 2005 iwslt speech translation evaluation](https://md.research.elamur.ai/free/en/papers/koehn-2005-edinburgh-system-description-2005-iwslt/). In International Workshop on Spoken Language Translation 2005.
12. Philipp Koehn, Hieu Hoang, Alexandra Birch, Chris Callison-Burch, Marcello Federico, Nicola Bertoldi, Brooke Cowan, Wade Shen, Christine Moran, Richard Zens, Chris Dyer, Ondrej Bojar, Alexandra Constantin, and Evan Herbst. 2007. Moses: Open source toolkit for statistical machine translation. In Proc… [find](https://scholar.google.com/scholar?q=Philipp+Koehn%2C+Hieu+Hoang%2C+Alexandra+Birch%2C+Chris+Callison%2DBurch%2C+Marcello+Federico%2C+Nicola+Bertoldi%2C+Brooke+Cowan%2C+Wade+Shen%2C+Christine+Moran%2C+Richard+Zens%2C+Chris+Dyer%2C+Ondrej+Bojar%2C+Alexandra+Consta)
13. Philipp Koehn. 2004a. Pharaoh: A beam search decoder for phrase-based statistical machine translation models. In AMTA, pages 115–124. [find](https://scholar.google.com/scholar?q=Philipp+Koehn%2E+2004a%2E+Pharaoh%3A+A+beam+search+decoder+for+phrase%2Dbased+statistical+machine+translation+models%2E+In+AMTA%2C+pages+115%E2%80%93124%2E)
14. Philipp Koehn. 2004b. [Statistical significance tests for machine translation evaluation](https://md.research.elamur.ai/free/en/papers/koehn-2004-statistical-significance-tests-machine-translation/). In Dekang Lin and Dekai Wu, editors, Proceedings of EMNLP 2004, pages 388–395, Barcelona, Spain, July. Association for Computational Linguistics.
15. Zhifei Li, Chris Callison-burch, Chris Dyer, Juri Ganitkevitch, Sanjeev Khudanpur, Lane Schwartz, Wren N. G. Thornton, Jonathan Weese, and Omar F. Zaidan. 2009. [Joshua: An open source toolkit for parsing-based machine translation](https://md.research.elamur.ai/free/en/papers/li-2009-joshua-open-source-toolkit-parsing/).
16. J.B. Mariño, R.E. Banchs, J.M. Crego, A. de Gispert, P. Lambert, J.A.R. Fonollosa, and M.R. Costa-jussà. 2006. [N-gram-based machine translation](https://md.research.elamur.ai/free/en/papers/marino-2006-n-gram-based-machine-translation/). Computational Linguistics, 32(4):527–549.
17. I. Dan Melamed. 2004. Statistical machine translation by parsing. In Proceedings of the 42nd Annual Meeting of the Association for Computational Linguistics, Barcelona, Spain. [find](https://scholar.google.com/scholar?q=I%2E+Dan+Melamed%2E+2004%2E+Statistical+machine+translation+by+parsing%2E+In+Proceedings+of+the+42nd+Annual+Meeting+of+the+Association+for+Computational+Linguistics%2C+Barcelona%2C+Spain%2E)
18. Franz J. Och and Hermann Ney. 2003. A systematic comparison of various statistical alignment models. Computational Linguistics, 29(1):19–51. [find](https://scholar.google.com/scholar?q=Franz+J%2E+Och+and+Hermann+Ney%2E+2003%2E+A+systematic+comparison+of+various+statistical+alignment+models%2E+Computational+Linguistics%2C+29%281%29%3A19%E2%80%9351%2E)
19. Franz J. Och and Hermann Ney. 2004. The alignment template approach to statistical machine translation. Computational Linguistics, 30(1):417–449. [find](https://scholar.google.com/scholar?q=Franz+J%2E+Och+and+Hermann+Ney%2E+2004%2E+The+alignment+template+approach+to+statistical+machine+translation%2E+Computational+Linguistics%2C+30%281%29%3A417%E2%80%93449%2E)
20. Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. [Bleu: a method for automatic evaluation of machine translation](https://md.research.elamur.ai/free/en/papers/papineni-2002-bleu-method-automatic-evaluation-machine/). In Proceedings of the 40th Annual Meeting on Association for Computational Linguistics, ACL ’02, pages 311–318, Morristown, NJ, USA. Association for Computational Lingui…
21. Andreas Stolcke. 2002. SRILM - an extensible language modeling toolkit. In Intl. Conf. Spoken Language Processing, Denver, Colorado. [find](https://scholar.google.com/scholar?q=Andreas+Stolcke%2E+2002%2E+SRILM+%2D+an+extensible+language+modeling+toolkit%2E+In+Intl%2E+Conf%2E+Spoken+Language+Processing%2C+Denver%2C+Colorado%2E)
22. Omar F. Zaidan. 2009. Z-MERT: A fully configurable open source tool for minimum error rate training of machine translation systems. The Prague Bulletin of Mathematical Linguistics, 91:79–88. [find](https://scholar.google.com/scholar?q=Omar+F%2E+Zaidan%2E+2009%2E+Z%2DMERT%3A+A+fully+configurable+open+source+tool+for+minimum+error+rate+training+of+machine+translation+systems%2E+The+Prague+Bulletin+of+Mathematical+Linguistics%2C+91%3A79%E2%80%9388%2E)

*Updated 24 October 2026*