---
title: "Extracting Parallel Sub-Sentential Fragments from Non-Parallel"
description: "Most methods for mining comparable corpora look for whole parallel sentences. 28 references, cited by 75 papers here."
authors: ["Dragos Stefan Munteanu", "Daniel Marcu"]
year: 2006
venue: "Proceedings of the 21st International Conference on Computational Linguistics and 44th Annual Meeting of the Association for Computational Linguistics"
source: "https://aclanthology.org/P06-1011/"
doi: "10.3115/1220175.1220186"
licence: "CC BY-NC-SA 3.0 (ACL Anthology, before 2016)"
licence_url: "https://creativecommons.org/licenses/by-nc-sa/3.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/munteanu-2006-extracting-parallel-sub-sentential-fragments/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/munteanu-2006-extracting-parallel-sub-sentential-fragments/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2006](https://md.research.elamur.ai/free/en/papers/2006/) / Extracting Parallel Sub-Sentential Fragments from Non-Parallel Corpora

# Extracting Parallel Sub-Sentential Fragments from Non-Parallel Corpora

**Authors:** [Dragos Stefan Munteanu](https://md.research.elamur.ai/free/en/authors/dragos-stefan-munteanu/) and [Daniel Marcu](https://md.research.elamur.ai/free/en/authors/daniel-marcu/) · **Year:** 2006 · **Venue:** [Proceedings of the 21st International Conference on Computational Linguistics and 44th Annual Meeting of the Association for Computational Linguistics](https://md.research.elamur.ai/free/en/venues/acl/) ([ACL 2006](https://md.research.elamur.ai/free/en/venues/acl/2006/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY-NC-SA 3.0 (ACL Anthology, before 2016)](https://creativecommons.org/licenses/by-nc-sa/3.0/)

**Links:** [Publisher's page](https://aclanthology.org/P06-1011/) · [PDF](https://aclanthology.org/P06-1011.pdf) · [DOI 10.3115/1220175.1220186](https://doi.org/10.3115/1220175.1220186) · [OpenAlex](https://openalex.org/W2140903445)

**Citations:** 163 citations (OpenAlex, 2026-10-14) · [75](https://md.research.elamur.ai/free/en/papers/munteanu-2006-extracting-parallel-sub-sentential-fragments/cited-by/) in our library, of the 141,840 papers whose bibliographies we read

## Summary
Most methods for mining comparable corpora look for whole parallel sentences. Truly non-parallel collections, however, may hold almost no such pairs, and their translated content survives only as pieces inside otherwise unrelated sentences. Munteanu and Marcu target this case, using English–Romanian BBC news as the running example: two reports on the anniversary of Ukraine's Orange Revolution with no parallel sentences between them.

Their pipeline has three stages:
1. **Document pairing.** Lemur retrieves likely matching documents using word-for-word translated queries.
2. **Candidate filtering.** A coarse filter drops sentence pairs that share few translated words.
3. **Fragment detection.** This step is inspired by signal processing. Each word becomes a positive or negative value depending on whether it seems to have a translation. The sequence is smoothed, and stretches that stay positive are kept.

The precise final stage relies on a new lexicon, LLR-Lex. It is derived from log-likelihood-ratio scores over word-aligned data and yields both translation and non-translation probabilities. Adding the extracted fragments to the training data improves a state-of-the-art SMT system. This matters because it lets even very noisy comparable data serve as MT training material.

## Key points
- The pipeline is Lemur document retrieval (top 20 target documents per source document), then a candidate sentence-pair filter, then sub-sentential fragment detection.
- Two lexicons are built from the same initial parallel corpus:
  - **GIZA-Lex** comes from GIZA++/IBM models and averages about 12 translations per source word. It is used for the permissive early stages.
  - **LLR-Lex** averages about 5 translations per source word. It is used for the precision-critical final stage.
- LLR-Lex counts two words as cooccurring only when they are linked in a word alignment, not merely present in aligned sentences. GIZA++ is run in both directions and symmetrized with the refined heuristic. Positive and negative associations are separated and normalized per source word.
- Ablation results:
  - Fragments extracted with GIZA-Lex in the final stage (Fragment-noLLR) gave no BLEU gain.
  - With a small initial corpus (1M words) and the noisy BBC data, those fragments even lowered BLEU.
  - On a corpus that is mostly sentence-parallel, adding fragments to extracted sentences did not help further.
- Known limitations:
  - The smoothing filter is crude and needs an extra rule requiring fragments of at least 3 words.
  - Source and target sides are filtered separately rather than jointly.
  - Most remaining errors come from false links involving punctuation and frequent function words.

## Interesting ideas
- **Treat alignment as a 1-D signal.** Each word gets a signed score for whether it is translated, and the sequence is smoothed to find contiguous positive regions. This turns noisy word-level evidence into span-level decisions and could carry over to other "which parts match" problems, such as near-duplicate detection or locating quoted passages.
- **Match lexicon recall to each pipeline stage.** A broad, noisy lexicon keeps early filters permissive, while a cleaner lexicon serves the final precise step. Staged pipelines elsewhere could likewise use different resources for coarse recall and fine precision.
- **Use negative association as explicit evidence.** Splitting LLR scores into positive and negative associations gives a probability that two words are *not* translations. That lets the system actively penalize untranslated content instead of only rewarding matches.
- **Stricter cooccurrence via word-alignment links.** Counting pairs only when the aligner links them, rather than whenever they appear in the same sentence pair, reuses the aligner's knowledge to produce a cleaner lexicon. This could help build bilingual dictionaries for low-resource pairs such as Ukrainian–X.
- **Evaluate extracted data by its downstream effect.** Judging the mined data by the MT improvement it brings, and ablating the lexicon component, shows which piece actually drives quality.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [University of Southern California](https://md.research.elamur.ai/free/en/universities/university-of-southern-california/) — University, Los Angeles, United States · 104 works · 33,876 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 9,086 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 8,889 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 9,296 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 9,206 works

## Cite

Cite as: Dragos Stefan Munteanu and Daniel Marcu. 2006. Extracting Parallel Sub-Sentential Fragments from Non-Parallel Corpora. Proceedings of the 21st International Conference on Computational Linguistics and 44th Annual Meeting of the Association for Computational Linguistics.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{munteanu2006extracting,
  title = {{Extracting Parallel Sub-Sentential Fragments from Non-Parallel Corpora}},
  author = {Dragos Stefan Munteanu and Daniel Marcu},
  booktitle = {Proceedings of the 21st International Conference on Computational Linguistics and 44th Annual Meeting of the Association for Computational Linguistics},
  year = {2006},
  doi = {10.3115/1220175.1220186},
  url = {https://doi.org/10.3115/1220175.1220186}
}
```

- [← previous Experiments with a Multilanguage Non-Projective Dependency Parser](https://md.research.elamur.ai/free/en/papers/attardi-2006-experiments-multilanguage-non-projective-dependency/)
- [next → Lexical Markup Framework (LMF)](https://md.research.elamur.ai/free/en/papers/francopoulo-2006-lexical-markup-framework-lmf/)
- [↑ back to papers of 2006](https://md.research.elamur.ai/free/en/papers/2006/)

## References

28 references, 7 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Peter F. Brown, Stephen A. Della Pietra, Vincent J. Della Pietra, and Robert L. Mercer. 1993. The mathematics of machine translation: Parameter estimation. Computational Linguistics, 19(2):263–311. [find](https://scholar.google.com/scholar?q=Peter+F%2E+Brown%2C+Stephen+A%2E+Della+Pietra%2C+Vincent+J%2E+Della+Pietra%2C+and+Robert+L%2E+Mercer%2E+1993%2E+The+mathematics+of+machine+translation%3A+Parameter+estimation%2E+Computational+Linguistics%2C+19%282%29%3A263%E2%80%93311%2E)
2. Percy Cheung and Pascale Fung. 2004. Sentence alignment in parallel, comparable, and quasi-comparable corpora. In LREC2004 Workshop. [find](https://scholar.google.com/scholar?q=Percy+Cheung+and+Pascale+Fung%2E+2004%2E+Sentence+alignment+in+parallel%2C+comparable%2C+and+quasi%2Dcomparable+corpora%2E+In+LREC2004+Workshop%2E)
3. Yonggang Deng, Shankar Kumar, and William Byrne. 2006. Segmentation and alignment of parallel text for statistical machine translation. Journal of Natural Language Engineering. to appear. [find](https://scholar.google.com/scholar?q=Yonggang+Deng%2C+Shankar+Kumar%2C+and+William+Byrne%2E+2006%2E+Segmentation+and+alignment+of+parallel+text+for+statistical+machine+translation%2E+Journal+of+Natural+Language+Engineering%2E+to+appear%2E)
4. Mona Diab and Steve Finch. 2000. A statistical word-level translation model for comparable corpora. In RIAO 2000. [find](https://scholar.google.com/scholar?q=Mona+Diab+and+Steve+Finch%2E+2000%2E+A+statistical+word%2Dlevel+translation+model+for+comparable+corpora%2E+In+RIAO+2000%2E)
5. Ted Dunning. 1993. [Accurate methods for the statistics of surprise and coincidence](https://md.research.elamur.ai/free/en/papers/dunning-1993-accurate-methods-statistics-surprise-coincidence/). Computational Linguistics, 19(1):61–74.
6. Pascale Fung and Percy Cheung. 2004a. Mining very non-parallel corpora: Parallel sentence and lexicon extraction vie bootstrapping and EM. In EMNLP 2004, pages 57–63. [find](https://scholar.google.com/scholar?q=Pascale+Fung+and+Percy+Cheung%2E+2004a%2E+Mining+very+non%2Dparallel+corpora%3A+Parallel+sentence+and+lexicon+extraction+vie+bootstrapping+and+EM%2E+In+EMNLP+2004%2C+pages+57%E2%80%9363%2E)
7. Pascale Fung and Percy Cheung. 2004b. Multi-level bootstrapping for extracting parallel sentences from a quasi-comparable corpus. In COLING 2004, pages 1051–1057. [find](https://scholar.google.com/scholar?q=Pascale+Fung+and+Percy+Cheung%2E+2004b%2E+Multi%2Dlevel+bootstrapping+for+extracting+parallel+sentences+from+a+quasi%2Dcomparable+corpus%2E+In+COLING+2004%2C+pages+1051%E2%80%931057%2E)
8. Pascale Fung and Lo Yuen Yee. 1998. [An IR approach for translating new words from nonparallel, comparable texts](https://md.research.elamur.ai/free/en/papers/fung-1998-ir-approach-translating-new-words/). In ACL 1998, pages 414–420.
9. Eric Gaussier, Jean-Michel Renders, Irina Matveeva, Cyril Goutte, and Herve Dejean. 2004. A geometric view on bilingual lexicon extraction from comparable corpora. In ACL 2004, pages 527–534. [find](https://scholar.google.com/scholar?q=Eric+Gaussier%2C+Jean%2DMichel+Renders%2C+Irina+Matveeva%2C+Cyril+Goutte%2C+and+Herve+Dejean%2E+2004%2E+A+geometric+view+on+bilingual+lexicon+extraction+from+comparable+corpora%2E+In+ACL+2004%2C+pages+527%E2%80%93534%2E)
10. Philipp Koehn and Kevin Knight. 2000. Estimating word translation probabilities from unrelated monolingual corpora using the EM algorithm. In National Conference on Artificial Intelligence, pages 711–715. [find](https://scholar.google.com/scholar?q=Philipp+Koehn+and+Kevin+Knight%2E+2000%2E+Estimating+word+translation+probabilities+from+unrelated+monolingual+corpora+using+the+EM+algorithm%2E+In+National+Conference+on+Artificial+Intelligence%2C+pages+711%E2%80%93)
11. Philipp Koehn. 2004. [Statistical significance tests for machine translation evaluation](https://md.research.elamur.ai/free/en/papers/koehn-2004-statistical-significance-tests-machine-translation/). In EMNLP 2004, pages 388–395.
12. I. Dan Melamed. 2000. Models of translational equivalence among words. Computational Linguistics, 26(2):221–249. [find](https://scholar.google.com/scholar?q=I%2E+Dan+Melamed%2E+2000%2E+Models+of+translational+equivalence+among+words%2E+Computational+Linguistics%2C+26%282%29%3A221%E2%80%93249%2E)
13. Robert C. Moore. 2004a. Improving IBM word-alignment model 1. In ACL 2004, pages 519–526. [find](https://scholar.google.com/scholar?q=Robert+C%2E+Moore%2E+2004a%2E+Improving+IBM+word%2Dalignment+model+1%2E+In+ACL+2004%2C+pages+519%E2%80%93526%2E)
14. Robert C. Moore. 2004b. On log-likelihood-ratios and the significance of rare events. In EMNLP 2004, pages 333–340. [find](https://scholar.google.com/scholar?q=Robert+C%2E+Moore%2E+2004b%2E+On+log%2Dlikelihood%2Dratios+and+the+significance+of+rare+events%2E+In+EMNLP+2004%2C+pages+333%E2%80%93340%2E)
15. Dragos Stefan Munteanu and Daniel Marcu. 2005. [Improving machine translation performance by exploiting non-parallel corpora](https://md.research.elamur.ai/free/en/papers/munteanu-2005-improving-machine-translation-performance-exploiting/). Computational Linguistics, 31(4).
16. Franz Joseph Och and Hermann Ney. 2003. A systematic comparison of various statistical alignment models. Computational Linguistics, 29(1):19–51. [find](https://scholar.google.com/scholar?q=Franz+Joseph+Och+and+Hermann+Ney%2E+2003%2E+A+systematic+comparison+of+various+statistical+alignment+models%2E+Computational+Linguistics%2C+29%281%29%3A19%E2%80%9351%2E)
17. Franz Joseph Och and Hermann Ney. 2004. The alignment template approach to statistical machine translation. Computational Linguistics, 30(4):417–450. [find](https://scholar.google.com/scholar?q=Franz+Joseph+Och+and+Hermann+Ney%2E+2004%2E+The+alignment+template+approach+to+statistical+machine+translation%2E+Computational+Linguistics%2C+30%284%29%3A417%E2%80%93450%2E)
18. P. Ogilvie and J. Callan. 2001. Experiments using the Lemur toolkit. In TREC 2001, pages 103–108. [find](https://scholar.google.com/scholar?q=P%2E+Ogilvie+and+J%2E+Callan%2E+2001%2E+Experiments+using+the+Lemur+toolkit%2E+In+TREC+2001%2C+pages+103%E2%80%93108%2E)
19. Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. [BLEU: a method for automatic evaluation of machine translation](https://md.research.elamur.ai/free/en/papers/papineni-2002-bleu-method-automatic-evaluation-machine/). In ACL 2002, pages 311–318.
20. Reinhard Rapp. 1999. [Automatic identification of word translations from unrelated English and German corpora](https://md.research.elamur.ai/free/en/papers/rapp-1999-automatic-identification-word-translations-unrelated/). In ACL 1999, pages 519–526.
21. Philip Resnik and Noah A. Smith. 2003. [The web as a parallel corpus](https://md.research.elamur.ai/free/en/papers/resnik-2003-web-as-parallel-corpus/). Computational Linguistics, 29(3):349–380.
22. Philip Resnik, Douglas Oard, and Gina Lewow. 2001. Improved cross-language retrieval using backoff translation. In HLT 2001. [find](https://scholar.google.com/scholar?q=Philip+Resnik%2C+Douglas+Oard%2C+and+Gina+Lewow%2E+2001%2E+Improved+cross%2Dlanguage+retrieval+using+backoff+translation%2E+In+HLT+2001%2E)
23. Li Shao and Hwee Tou Ng. 2004. Mining new word translations from comparable corpora. In COLING 2004, pages 618–624. [find](https://scholar.google.com/scholar?q=Li+Shao+and+Hwee+Tou+Ng%2E+2004%2E+Mining+new+word+translations+from+comparable+corpora%2E+In+COLING+2004%2C+pages+618%E2%80%93624%2E)
24. Yusuke Shinyama and Satoshi Sekine. 2004. Named entity discovery using comparable news articles. In COLING 2004, pages 848–853. [find](https://scholar.google.com/scholar?q=Yusuke+Shinyama+and+Satoshi+Sekine%2E+2004%2E+Named+entity+discovery+using+comparable+news+articles%2E+In+COLING+2004%2C+pages+848%E2%80%93853%2E)
25. Masao Utiyama and Hitoshi Isahara. 2003. Reliable measures for aligning Japanese-English news articles and sentences. In ACL 2003, pages 72–79. [find](https://scholar.google.com/scholar?q=Masao+Utiyama+and+Hitoshi+Isahara%2E+2003%2E+Reliable+measures+for+aligning+Japanese%2DEnglish+news+articles+and+sentences%2E+In+ACL+2003%2C+pages+72%E2%80%9379%2E)
26. Stephan Vogel. 2003. Using noisy bilingual data for statistical machine translation. In EACL 2003, pages 175–178. [find](https://scholar.google.com/scholar?q=Stephan+Vogel%2E+2003%2E+Using+noisy+bilingual+data+for+statistical+machine+translation%2E+In+EACL+2003%2C+pages+175%E2%80%93178%2E)
27. Dekai Wu and Pascale Fung. 2005. Inversion transduction grammar constraints for mining parallel sentences from quasi-comparable corpora. In IJCNLP 2005, pages 257–268. [find](https://scholar.google.com/scholar?q=Dekai+Wu+and+Pascale+Fung%2E+2005%2E+Inversion+transduction+grammar+constraints+for+mining+parallel+sentences+from+quasi%2Dcomparable+corpora%2E+In+IJCNLP+2005%2C+pages+257%E2%80%93268%2E)
28. Bing Zhao and Stephan Vogel. 2002. Adaptive parallel sentences mining from web bilingual news collection. In 2002 IEEE Int. Conf. on Data Mining, pages 745–748. [find](https://scholar.google.com/scholar?q=Bing+Zhao+and+Stephan+Vogel%2E+2002%2E+Adaptive+parallel+sentences+mining+from+web+bilingual+news+collection%2E+In+2002+IEEE+Int%2E+Conf%2E+on+Data+Mining%2C+pages+745%E2%80%93748%2E)

*Updated 24 October 2026*