---
title: "A Discriminative Framework for Bilingual Word Alignment (2005)"
description: "Moore tackles bilingual word alignment, the step that underlies most statistical machine translation… 13 references, cited by 64 papers here."
authors: ["Robert C. Moore"]
year: 2005
venue: "Proceedings of Human Language Technology Conference and Conference on Empirical Methods in Natural Language Processing"
source: "https://aclanthology.org/H05-1011/"
licence: "CC BY-NC-SA 3.0 (ACL Anthology, before 2016)"
licence_url: "https://creativecommons.org/licenses/by-nc-sa/3.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/moore-2005-discriminative-framework-bilingual-word-alignment/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/moore-2005-discriminative-framework-bilingual-word-alignment/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2005](https://md.research.elamur.ai/free/en/papers/2005/) / A Discriminative Framework for Bilingual Word Alignment

# A Discriminative Framework for Bilingual Word Alignment

**Authors:** [Robert C. Moore](https://md.research.elamur.ai/free/en/authors/robert-c-moore/) · **Year:** 2005 · **Venue:** [Proceedings of Human Language Technology Conference and Conference on Empirical Methods in Natural Language Processing](https://md.research.elamur.ai/free/en/venues/hlt/) ([HLT 2005](https://md.research.elamur.ai/free/en/venues/hlt/2005/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY-NC-SA 3.0 (ACL Anthology, before 2016)](https://creativecommons.org/licenses/by-nc-sa/3.0/)

**Links:** [Publisher's page](https://aclanthology.org/H05-1011/) · [PDF](https://aclanthology.org/H05-1011.pdf) · [OpenAlex](https://openalex.org/W2124810114)

**Citations:** 142 citations (OpenAlex, 2026-10-14) · [64](https://md.research.elamur.ai/free/en/papers/moore-2005-discriminative-framework-bilingual-word-alignment/cited-by/) in our library, of the 141,840 papers whose bibliographies we read

## Summary
Moore tackles bilingual word alignment, the step that underlies most statistical machine translation pipelines. The dominant approach relies on the generative IBM and HMM models, which have two drawbacks. Their free parameters are expensive to tune, and new features are hard to add because each one needs a fitting generative story. Instead, the paper scores each candidate alignment as a weighted sum of a few features and picks the highest-scoring one. The weights are learned with a modified averaged perceptron (following Collins, 2002) on a small annotated set.

Two model families are built:
- an LLR-based model using log-likelihood-ratio association scores;
- CLP-based models using discounted conditional link probabilities taken from a simpler aligner's output.

Both add features for reordering (nonmonotonicity), unlinked words and, in the LLR model, one-to-many links. On an English–French corpus of 500,000 sentence pairs, the best CLP model slightly beats the strongest IBM Model 4 combination from Giza++. The point is that a simple, easily extensible model tuned on a few hundred annotated pairs can match much more complex generative systems.

## Key points
- Weights were tuned on a development set of only 224 annotated sentence pairs. Each perceptron pass took under two minutes, convergence came within six passes, and total tuning time per model was about an hour. Aligning the full corpus with the Perl implementation still took more than a day.
- Test-set AER:
  - LLR model: 0.160
  - CLP 1 (link probabilities from the LLR model): 0.086
  - CLP 2 (link probabilities from Moore's 2005 heuristic aligner): 0.075
- For comparison, IBM Model 4 via Giza++ scored 0.079 AER with the "refined" combination of both directions and 0.097 with intersection.
- A discount of 0.4 in the conditional link probability estimate worked best on the development set. CLP training used N-best = 20.
- The author attributes CLP 2's edge to coverage: it received 403,195 link probabilities, against 144,051 for CLP 1.

## Interesting ideas
- **Map features onto the old model's components.** Each feature has a counterpart in the generative models: association scores correspond to translation probabilities, reordering to distortion, unlinked words to null links, and one-to-many links to fertility. This is a practical recipe for turning any generative pipeline into a discriminative one without losing what the original modeled.
- **Count reordering in two ways at once.** Rather than choosing between the number of backward jumps and their total size, both are used as features and the learner decides how to weigh them. This sidesteps a design decision and is cheap to apply to other ordering or structure measures.
- **Stack a simple model under a better one.** Link counts from a simpler aligner become discounted conditional probabilities for a second model. Subtracting a fixed discount keeps rarely co-occurring pairs from getting inflated scores, which suits any statistic estimated from sparse counts.
- **Prune the search space with an unscored feature.** The many-to-many count is never weighted in the model. Any alignment where it is nonzero is simply discarded, so a feature doubles as a hard filter that keeps search tractable.
- **Make the learned weights the last stage.** Because the weights come at the end of model building, tuning only reruns a fast online pass over a small development set, not the full expensive training. This matters wherever annotation is scarce, for example in lower-resource language pairs.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Microsoft](https://md.research.elamur.ai/free/en/companies/microsoft/) — Company, Redmond, United States · 240 works · 123,537 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 9,085 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 8,888 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 9,296 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 9,206 works

## Cite

Cite as: Robert C. Moore. 2005. A Discriminative Framework for Bilingual Word Alignment. Proceedings of Human Language Technology Conference and Conference on Empirical Methods in Natural Language Processing.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{moore2005discriminative,
  title = {{A Discriminative Framework for Bilingual Word Alignment}},
  author = {Robert C. Moore},
  booktitle = {Proceedings of Human Language Technology Conference and Conference on Empirical Methods in Natural Language Processing},
  year = {2005},
  doi = {10.3115/1220575.1220586},
  url = {https://doi.org/10.3115/1220575.1220586}
}
```

- [← previous A Classifier-Based Parser with Linear Run-Time Complexity](https://md.research.elamur.ai/free/en/papers/sagae-2005-classifier-based-parser-linear-run/)
- [next → Co-occurrence Retrieval: A Flexible Framework for Lexical Distributional Similarity](https://md.research.elamur.ai/free/en/papers/weeds-2005-co-occurrence-retrieval-flexible-framework/)
- [↑ back to papers of 2005](https://md.research.elamur.ai/free/en/papers/2005/)

## References

13 references, 2 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Peter F. Brown, Stephen A. Della Pietra, Vincent J. Della Pietra, and Robert L. Mercer. 1993. The Mathematics of Statistical Machine Translation: Parameter Estimation. Computational Linguistics, 19(2):263–311. [find](https://scholar.google.com/scholar?q=Peter+F%2E+Brown%2C+Stephen+A%2E+Della+Pietra%2C+Vincent+J%2E+Della+Pietra%2C+and+Robert+L%2E+Mercer%2E+1993%2E+The+Mathematics+of+Statistical+Machine+Translation%3A+Parameter+Estimation%2E+Computational+Linguistics%2C+19%282%29)
2. Chris Callison-Burch, David Talbot, and Miles Osborne. 2005. Statistical Marchine Translation with Word- and Sentences-Aligned Parallel Corpora. In Proceedings of the 42nd Annual Meeting of the ACL, pp. 176–183, Barcelona, Spain. [find](https://scholar.google.com/scholar?q=Chris+Callison%2DBurch%2C+David+Talbot%2C+and+Miles+Osborne%2E+2005%2E+Statistical+Marchine+Translation+with+Word%2D+and+Sentences%2DAligned+Parallel+Corpora%2E+In+Proceedings+of+the+42nd+Annual+Meeting+of+the+ACL%2C+p)
3. Michael Collins. 2002. [Discriminative Training Methods for Hidden Markov Models: Theory and Experiments with Perceptron Algorithms](https://md.research.elamur.ai/free/en/papers/collins-2002-discriminative-training-methods-hidden-markov/). In Proceedings of the Conference on Empirical Methods in Natural Language Processing, pp. 1–8, Philadelphia, Pennsylvania.
4. Alexander Fraser and Daniel Marcu. 2005. ISI’s Participation in the Romanian-English Alignment Task. In Proceedings of the ACL Workshop on Building and Using Parallel Texts, pp. 91–94, Ann Arbor, Michigan. [find](https://scholar.google.com/scholar?q=Alexander+Fraser+and+Daniel+Marcu%2E+2005%2E+ISI%E2%80%99s+Participation+in+the+Romanian%2DEnglish+Alignment+Task%2E+In+Proceedings+of+the+ACL+Workshop+on+Building+and+Using+Parallel+Texts%2C+pp%2E+91%E2%80%9394%2C+Ann+Arbor%2C+Mich)
5. Yang Liu, Qun Liu, and Shouxun Lin. 2005. Log-linear Models for Word Alignment. In Proceedings of the 43rd Annual Meeting of the ACL, pp. 459–466, Ann Arbor, Michigan. [find](https://scholar.google.com/scholar?q=Yang+Liu%2C+Qun+Liu%2C+and+Shouxun+Lin%2E+2005%2E+Log%2Dlinear+Models+for+Word+Alignment%2E+In+Proceedings+of+the+43rd+Annual+Meeting+of+the+ACL%2C+pp%2E+459%E2%80%93466%2C+Ann+Arbor%2C+Michigan%2E)
6. Dan Klein and Ben Taskar. 2005. Max-Margin Methods for NLP: Estimation, Structure, and Applications. Tutorial presented at ACL 2005, Ann Arbor, Michigan. [find](https://scholar.google.com/scholar?q=Dan+Klein+and+Ben+Taskar%2E+2005%2E+Max%2DMargin+Methods+for+NLP%3A+Estimation%2C+Structure%2C+and+Applications%2E+Tutorial+presented+at+ACL+2005%2C+Ann+Arbor%2C+Michigan%2E)
7. I. Dan Melamed. 2000. Models of Translational Equivalence. Computational Linguistics, 26(2):221–249. [find](https://scholar.google.com/scholar?q=I%2E+Dan+Melamed%2E+2000%2E+Models+of+Translational+Equivalence%2E+Computational+Linguistics%2C+26%282%29%3A221%E2%80%93249%2E)
8. Rada Mihalcea and Ted Pedersen. 2003. [An Evaluation Exercise for Word Alignment](https://md.research.elamur.ai/free/en/papers/mihalcea-2003-evaluation-exercise-word-alignment/). In Proceedings of the HLT-NAACL 2003 Workshop, Building and Using Parallel Texts: Data Driven Machine Translation and Beyond, pp. 1–6, Edmonton, Alberta, Canada.
9. Robert C. Moore. 2004. On Log-Likelihood-Ratios and the Significance of Rare Events. In Proceedings of the 2004 Conference on Empirical Methods in Natural Language Processing, pp. 333– 340, Barcelona, Spain. [find](https://scholar.google.com/scholar?q=Robert+C%2E+Moore%2E+2004%2E+On+Log%2DLikelihood%2DRatios+and+the+Significance+of+Rare+Events%2E+In+Proceedings+of+the+2004+Conference+on+Empirical+Methods+in+Natural+Language+Processing%2C+pp%2E+333%E2%80%93+340%2C+Barcelona%2C)
10. Robert C. Moore. 2005. Association-Based Bilingual Word Alignment. In Proceedings of the ACL Workshop on Building and Using Parallel Texts, pp. 1–8, Ann Arbor, Michigan. [find](https://scholar.google.com/scholar?q=Robert+C%2E+Moore%2E+2005%2E+Association%2DBased+Bilingual+Word+Alignment%2E+In+Proceedings+of+the+ACL+Workshop+on+Building+and+Using+Parallel+Texts%2C+pp%2E+1%E2%80%938%2C+Ann+Arbor%2C+Michigan%2E)
11. Franz Joseph Och and Hermann Ney. 2003. A Systematic Comparison of Various Statistical Alignment Models. Computational Linguistics, 29(1):19–51. [find](https://scholar.google.com/scholar?q=Franz+Joseph+Och+and+Hermann+Ney%2E+2003%2E+A+Systematic+Comparison+of+Various+Statistical+Alignment+Models%2E+Computational+Linguistics%2C+29%281%29%3A19%E2%80%9351%2E)
12. Franz Joseph Och and Hermann Ney. 2004. The Alignment Template Approach to Statistical Machine Translation. Computational Linguistics, 30(4):417–449. [find](https://scholar.google.com/scholar?q=Franz+Joseph+Och+and+Hermann+Ney%2E+2004%2E+The+Alignment+Template+Approach+to+Statistical+Machine+Translation%2E+Computational+Linguistics%2C+30%284%29%3A417%E2%80%93449%2E)
13. William H. Press, Saul A. Teukolsky, William T. Vetterling, and Brian P. Flannery. 1992. Numerical Recipies in C: The Art of Scientific Computing, Second Edition. Cambridge University Press, Cambridge, England. [find](https://scholar.google.com/scholar?q=William+H%2E+Press%2C+Saul+A%2E+Teukolsky%2C+William+T%2E+Vetterling%2C+and+Brian+P%2E+Flannery%2E+1992%2E+Numerical+Recipies+in+C%3A+The+Art+of+Scientific+Computing%2C+Second+Edition%2E+Cambridge+University+Press%2C+Cambridge)

*Updated 24 October 2026*