--- title: "Sentence Alignment for Monolingual — Barzilay 2003" description: "The paper tackles sentence alignment between monolingual comparable texts." authors: ["Regina Barzilay", "Noemie Elhadad"] year: 2003 venue: "Proceedings of the 2003 Conference on Empirical Methods in Natural Language Processing" source: "https://aclanthology.org/W03-1004/" licence: "CC BY-NC-SA 3.0 (ACL Anthology, before 2016)" licence_url: "https://creativecommons.org/licenses/by-nc-sa/3.0/" summary_licence: "Elamur Cosmic License" canonical: "https://research.elamur.ai/free/en/papers/barzilay-2003-sentence-alignment-monolingual-comparable-corpora/" --- [Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/barzilay-2003-sentence-alignment-monolingual-comparable-corpora/) [home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2003](https://md.research.elamur.ai/free/en/papers/2003/) / Sentence Alignment for Monolingual Comparable Corpora # Sentence Alignment for Monolingual Comparable Corpora **Authors:** [Regina Barzilay](https://md.research.elamur.ai/free/en/authors/regina-barzilay/), [Noemie Elhadad](https://md.research.elamur.ai/free/en/authors/noemie-elhadad/) · **Year:** 2003 · **Venue:** Proceedings of the 2003 Conference on Empirical Methods in Natural Language Processing · **Licence:** [CC BY-NC-SA 3.0 (ACL Anthology, before 2016)](https://creativecommons.org/licenses/by-nc-sa/3.0/) **Links:** [Publisher's page](https://aclanthology.org/W03-1004/) · [PDF](https://aclanthology.org/W03-1004.pdf) · [OpenAlex](https://openalex.org/W2061235289) **Citations:** 194 citations (openalex-snapshot · 2026-10-14) · [86](#cited-by) in our library, of the 140884 papers whose bibliographies we read ## Summary The paper tackles sentence alignment between monolingual comparable texts. These are documents in one language that cover overlapping content but differ heavily in what they include, how they order it, and how they phrase it. The goal is to automatically collect sentence pairs that can serve as training data for learning text-to-text rewriting rules, for example for simplification or summarization. The method keeps the sentence similarity measure deliberately simple (cosine word overlap) and adds context at the paragraph level in two stages: - **Macro alignment:** paragraphs in each collection are clustered into domain topics without supervision, and rules for mapping topics between the two collections are learned from manually aligned text pairs. - **Micro alignment:** sentences are aligned locally inside each mapped paragraph pair. The evaluation uses encyclopedia-style texts in elementary and comprehensive versions. The approach beats the Cosine baseline, Decomposition (an HMM) and SimFinder, and it recovers matches even when two sentences share few words. Even a fully unsupervised variant that uses only paragraph-level lexical similarity outperforms systems built on richer sentence-similarity functions. ## Key points - In one example text pair, only 35 of the 31 × 270 possible sentence pairs are true matches, and their order varies widely between the texts. This shows why MT-style monotonic alignment assumptions break down for comparable corpora. - Topics are induced by hierarchical complete-link clustering of paragraphs, with cosine similarity over content words. Proper names, dates and numbers are replaced by generic tags before clustering. - Adding the context-based search on top of Cosine yields a 19% performance improvement at 55.8% Recall. - Ablations at 55.8% Recall: - Removing local alignment drops Precision to 73.3%, which is 3% below the full method. - Replacing automatic clusters with manually derived "ideal" clusters (8 for the elementary version, 11 for the comprehensive one) adds only 4% Precision. - For sentence pairs with low lexical overlap (0–40%), the full method reaches 50% Precision, compared with 34% for Decomposition and 23% for Cosine. ## Interesting ideas - **Lean on context, not a heavy similarity function.** Pairing a cheap lexical score with a search over context can outperform a sophisticated sentence-level scorer. This is a useful default when building alignment or paraphrase resources for languages with few semantic tools. - **Strip text-specific details before clustering.** Masking names, dates and numbers groups paragraphs by the kind of information they carry, such as "climate" or "symptoms", rather than by the specific entity described. The same trick could help induce document schemas in other genres, like news or trade reports. - **Align coarse-to-fine.** First decide which paragraphs could plausibly correspond, then align sentences only inside those pairs. This sharply shrinks the search space and suppresses spurious matches in noisy, reordered document pairs. - **Let neighboring pairs support weak matches.** Local alignment lets a lexically weak sentence pair be accepted when the pairs around it match well. This idea could transfer to aligning loosely related bilingual or multi-version documents. - **Test sensitivity to cluster quality with ideal clusters.** Using existing section headings to build "ideal" clusters gives a cheap upper bound on how much better clustering would help. The same evaluation trick works for any pipeline that depends on an induced intermediate structure. *Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.* Fields: [Natural language processing](https://md.research.elamur.ai/free/en/shelves/nlp/) ## Cited by Cited by 86 papers of our library cite it (of the 140884 whose bibliographies we read); 16 of them have a page here, 70 not yet. - [A Survey of Paraphrasing and Textual Entailment Methods](https://md.research.elamur.ai/free/en/papers/androutsopoulos-2009-survey-paraphrasing-textual-entailment-methods/) - [Modeling Local Coherence: An Entity-Based Approach](https://md.research.elamur.ai/free/en/papers/barzilay-2008-modeling-local-coherence-entity-based/) - [Putting it Simply: a Context-Aware Approach to Lexical Simplification](https://md.research.elamur.ai/free/en/papers/biran-2011-putting-it-simply-context-aware/) - [Constructing Corpora for the Development and Evaluation of Paraphrase Systems](https://md.research.elamur.ai/free/en/papers/cohn-2008-constructing-corpora-development-evaluation-paraphrase/) - [Learning to Simplify Sentences Using Wikipedia](https://md.research.elamur.ai/free/en/papers/coster-2011-learning-simplify-sentences-wikipedia/) - [Simple English Wikipedia: A New Text Simplification Task](https://md.research.elamur.ai/free/en/papers/coster-2011-simple-english-wikipedia-new-text/) - [Neural CRF Model for Sentence Alignment in Text Simplification](https://md.research.elamur.ai/free/en/papers/jiang-2020-neural-crf-model-sentence-alignment/) - [A Continuously Growing Dataset of Sentential Paraphrases](https://md.research.elamur.ai/free/en/papers/lan-2017-continuously-growing-dataset-sentential-paraphrases/) - [Learning How to Simplify From Explicit Labeling of Complex-Simplified Text Pairs](https://md.research.elamur.ai/free/en/papers/manchego-2017-learning-how-simplify-explicit-labeling/) - [Data-Driven Sentence Simplification: Survey and Benchmark](https://md.research.elamur.ai/free/en/papers/manchego-2020-data-driven-sentence-simplification-survey/) - [Improving Machine Translation Performance by Exploiting Non-Parallel Corpora](https://md.research.elamur.ai/free/en/papers/munteanu-2005-improving-machine-translation-performance-exploiting/) - [Hybrid Simplification using Deep Semantics and Machine Translation](https://md.research.elamur.ai/free/en/papers/narayan-2014-hybrid-simplification-deep-semantics-machine/) - [Reading Level Assessment Using Support Vector Machines and Statistical Language Models](https://md.research.elamur.ai/free/en/papers/schwarm-2005-reading-level-assessment-support-vector/) - [Semantic Structural Evaluation for Text Simplification](https://md.research.elamur.ai/free/en/papers/sulem-2018-semantic-structural-evaluation-text-simplification/) - [Learning to Simplify Sentences with Quasi-Synchronous Grammar and Integer Programming](https://md.research.elamur.ai/free/en/papers/woodsend-2011-learning-simplify-sentences-quasi-synchronous/) - [A Monolingual Tree-based Translation Model for Sentence Simplification](https://md.research.elamur.ai/free/en/papers/zhu-2010-monolingual-tree-based-translation-model/) ## Cite The paper itself, not this page. BibTeX from the library's record of the work. ```bibtex @inproceedings{barzilay2003sentence, title = {{Sentence Alignment for Monolingual Comparable Corpora}}, author = {Barzilay, Regina and Elhadad, Noemie}, booktitle = {Proceedings of the 2003 Conference on Empirical Methods in Natural Language Processing}, year = {2003}, doi = {10.3115/1119355.1119359}, url = {https://doi.org/10.3115/1119355.1119359} } ``` - [← previous Roots & patterns vs. stems plus grammar-lexis specifications: on what basis should a multilingual database centred on Arabic be built?](https://md.research.elamur.ai/free/en/papers/lingbulk-2003-dichy-roots-patterns-vs-stems-plus/) - [next → Sentence Level Discourse Parsing using Syntactic and Lexical Information](https://md.research.elamur.ai/free/en/papers/soricut-2003-sentence-level-discourse-parsing-syntactic/) - [↑ back to papers of 2003](https://md.research.elamur.ai/free/en/papers/2003/) ## References 18 references, 3 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet. 1. Regina Barzilay, Noemie Elhadad, and Kathleen McKeown. 2002. Inferring strategies for sentence ordering in multi-document news summarization. Journal of Artificial Intelligence Research, 17:35–55. [find](https://scholar.google.com/scholar?q=Regina+Barzilay%2C+Noemie+Elhadad%2C+and+Kathleen+McKeown%2E+2002%2E+Inferring+strategies+for+sentence+ordering+in+multi%2Ddocument+news+summarization%2E+Journal+of+Artificial+Intelligence+Research%2C+17%3A35%E2%80%9355%2E) 2. John Carroll, Guido Minnen, Darren Pearce, Yvonne Canning, Siobhan Devlin, and John Tait. 1999. Simplifying text for language-impaired readers. In Proceedings of EACL. [find](https://scholar.google.com/scholar?q=John+Carroll%2C+Guido+Minnen%2C+Darren+Pearce%2C+Yvonne+Canning%2C+Siobhan+Devlin%2C+and+John+Tait%2E+1999%2E+Simplifying+text+for+language%2Dimpaired+readers%2E+In+Proceedings+of+EACL%2E) [our card](https://md.research.elamur.ai/free/en/papers/carroll-1999-simplifying-text-language-impaired-readers/) 3. Raman Chandrasekar and Srinivas Bangalore. 1997. Automatic Induction of Rules for Text Simplification. Knowledge-Based Systems, 10(3):183–190. [find](https://scholar.google.com/scholar?q=Raman+Chandrasekar+and+Srinivas+Bangalore%2E+1997%2E+Automatic+Induction+of+Rules+for+Text+Simplification%2E+Knowledge%2DBased+Systems%2C+10%283%29%3A183%E2%80%93190%2E) 4. Pascale Fung and Lo Yuen Yee. 1998. An IR approach for translating new words from nonparallel, comparable texts. In Proceedings of ACL. [find](https://scholar.google.com/scholar?q=Pascale+Fung+and+Lo+Yuen+Yee%2E+1998%2E+An+IR+approach+for+translating+new+words+from+nonparallel%2C+comparable+texts%2E+In+Proceedings+of+ACL%2E) [our card](https://md.research.elamur.ai/free/en/papers/fung-1998-ir-approach-translating-new-words/) 5. William Gale and Kenneth Church. 1991. A program for aligning sentences in bilingual corpora. In Proceedings of ACL. [find](https://scholar.google.com/scholar?q=William+Gale+and+Kenneth+Church%2E+1991%2E+A+program+for+aligning+sentences+in+bilingual+corpora%2E+In+Proceedings+of+ACL%2E) [our card](https://md.research.elamur.ai/free/en/papers/gale-1993-program-aligning-sentences-bilingual-corpora/) 6. Vasileios Hatzivassiloglou, Judith Klavans, and Eleazar Eskin. 1999. Detecting text similarity over short passages: Exploring linguistic feature combinations via machine learning. In Proceedings of EMNLP. [find](https://scholar.google.com/scholar?q=Vasileios+Hatzivassiloglou%2C+Judith+Klavans%2C+and+Eleazar+Eskin%2E+1999%2E+Detecting+text+similarity+over+short+passages%3A+Exploring+linguistic+feature+combinations+via+machine+learning%2E+In+Proceedings+of+EM) 7. Marti Hearst. 1994. Multi-paragraph segmentation of expository text. In Proceedings of ACL. [find](https://scholar.google.com/scholar?q=Marti+Hearst%2E+1994%2E+Multi%2Dparagraph+segmentation+of+expository+text%2E+In+Proceedings+of+ACL%2E) 8. Hongyan Jing and Kathleen McKeown. 2000. Cut and paste based summarization. In Proceedings of NAACL. [find](https://scholar.google.com/scholar?q=Hongyan+Jing+and+Kathleen+McKeown%2E+2000%2E+Cut+and+paste+based+summarization%2E+In+Proceedings+of+NAACL%2E) 9. Hongyan Jing. 2002. Using hidden Markov modeling to decompose human-written summaries. Computational Linguistics, 28(4):527–543. [find](https://scholar.google.com/scholar?q=Hongyan+Jing%2E+2002%2E+Using+hidden+Markov+modeling+to+decompose+human%2Dwritten+summaries%2E+Computational+Linguistics%2C+28%284%29%3A527%E2%80%93543%2E) 10. Richard Kittredge, Tanya Korelsky, and Owen Rambow. 1991. On the need for domain communication knowledge. Computational Intelligence, 7(4):305–314. [find](https://scholar.google.com/scholar?q=Richard+Kittredge%2C+Tanya+Korelsky%2C+and+Owen+Rambow%2E+1991%2E+On+the+need+for+domain+communication+knowledge%2E+Computational+Intelligence%2C+7%284%29%3A305%E2%80%93314%2E) 11. Kevin Knight and Daniel Marcu. 2000. Statistics-based summarization - step one: Sentence compression. In Proceedings of AAAI. [find](https://scholar.google.com/scholar?q=Kevin+Knight+and+Daniel+Marcu%2E+2000%2E+Statistics%2Dbased+summarization+%2D+step+one%3A+Sentence+compression%2E+In+Proceedings+of+AAAI%2E) 12. William Mann and Sandra Thompson. 1987. Rhetorical structure theory: description and construction of text structures. In G. Kempen, editor, Natural Language Generation: Recent Advances in Artificial Intelligence, Psychology, and Linguistics. Kluwer Academic. [find](https://scholar.google.com/scholar?q=William+Mann+and+Sandra+Thompson%2E+1987%2E+Rhetorical+structure+theory%3A+description+and+construction+of+text+structures%2E+In+G%2E+Kempen%2C+editor%2C+Natural+Language+Generation%3A+Recent+Advances+in+Artificial+I) 13. Daniel Marcu. 1999. The automatic construction of large-scale corpora for summarization research. In Proceedings of SI-GIR. [find](https://scholar.google.com/scholar?q=Daniel+Marcu%2E+1999%2E+The+automatic+construction+of+large%2Dscale+corpora+for+summarization+research%2E+In+Proceedings+of+SI%2DGIR%2E) 14. Kathleen McKeown. 1985. Text Generation. Cambridge University Press. [find](https://scholar.google.com/scholar?q=Kathleen+McKeown%2E+1985%2E+Text+Generation%2E+Cambridge+University+Press%2E) 15. Dan Melamed. 1999. Bitext maps and alignment via pattern recognition. Computational Linguistics, 25(1):107–130. [find](https://scholar.google.com/scholar?q=Dan+Melamed%2E+1999%2E+Bitext+maps+and+alignment+via+pattern+recognition%2E+Computational+Linguistics%2C+25%281%29%3A107%E2%80%93130%2E) 16. Cecile Paris. 1993. User Modelling in Text Generation. Frances Pinter Publishers. [find](https://scholar.google.com/scholar?q=Cecile+Paris%2E+1993%2E+User+Modelling+in+Text+Generation%2E+Frances+Pinter+Publishers%2E) 17. Hadar Shemtov. 1993. Text alignment in a tool for translating revised documents. In Proceedings of EACL. [find](https://scholar.google.com/scholar?q=Hadar+Shemtov%2E+1993%2E+Text+alignment+in+a+tool+for+translating+revised+documents%2E+In+Proceedings+of+EACL%2E) 18. Yoram Singer and Robert Schapire. 1998. Improved boosting algorithms using confidence-rated predictions. In Proceedings of COLT. [find](https://scholar.google.com/scholar?q=Yoram+Singer+and+Robert+Schapire%2E+1998%2E+Improved+boosting+algorithms+using+confidence%2Drated+predictions%2E+In+Proceedings+of+COLT%2E)