---
title: "Multi-Document Summarization By Sentence Extraction (2000)"
description: "The authors propose an extraction-based system that selects sentences using fast… 33 references, cited by 68 papers here."
authors: ["Jade Goldstein", "Vibhu Mittal", "Jaime Carbonell", "Mark Kantrowitz"]
year: 2000
venue: "NAACL-ANLP 2000 Workshop: Automatic Summarization"
source: "https://aclanthology.org/W00-0405/"
licence: "CC BY-NC-SA 3.0 (ACL Anthology, before 2016)"
licence_url: "https://creativecommons.org/licenses/by-nc-sa/3.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/goldstein-2000-multi-document-summarization-sentence-extraction/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/goldstein-2000-multi-document-summarization-sentence-extraction/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2000](https://md.research.elamur.ai/free/en/papers/2000/) / Multi-Document Summarization By Sentence Extraction

# Multi-Document Summarization By Sentence Extraction

**Authors:** Jade Goldstein, [Vibhu Mittal](https://md.research.elamur.ai/free/en/authors/vibhu-mittal/), [Jaime Carbonell](https://md.research.elamur.ai/free/en/authors/jaime-carbonell/) and Mark Kantrowitz · **Year:** 2000 · **Venue:** [NAACL-ANLP 2000 Workshop: Automatic Summarization](https://md.research.elamur.ai/free/en/venues/naacl-anlp-workshop-automatic-summarization/) ([NAACL-ANLP Workshop: Automatic Summarization 2000](https://md.research.elamur.ai/free/en/venues/naacl-anlp-workshop-automatic-summarization/2000/)) · **Licence:** [CC BY-NC-SA 3.0 (ACL Anthology, before 2016)](https://creativecommons.org/licenses/by-nc-sa/3.0/)

**Links:** [Publisher's page](https://aclanthology.org/W00-0405/) · [PDF](https://aclanthology.org/W00-0405.pdf) · [OpenAlex](https://openalex.org/W1980813323)

**Citations:** 353 citations (OpenAlex, 2026-10-14) · [68](https://md.research.elamur.ai/free/en/papers/goldstein-2000-multi-document-summarization-sentence-extraction/cited-by/) in our library, of the 141,840 papers whose bibliographies we read

## Summary
The paper looks at how to summarize a whole set of related documents, such as hundreds of news stories returned by one search query. It argues that multi-document summarization is not just single-document summarization repeated. Related articles share a lot of background, news streams change over time, coreference across sources is harder, and the summary must be far shorter relative to the input. The authors propose an extraction-based system that selects sentences using fast, domain-independent statistical processing. It adds a metric that penalizes redundancy and rewards diversity among the chosen passages. The framework is modular, so it can be tuned to different genres, corpus properties and user needs. The paper lists requirements for such systems and several strategies for building summaries, and it shows a sample summary ordered by time. Evaluation was still being set up: 10-document sets with human gold-standard summaries and user ratings. The work matters because it frames redundancy control and extreme compression as the central problems in summarizing large document clusters.

## Key points
- The paper names four differences from single-document summarization: higher redundancy across documents, a temporal dimension where later reports may override earlier ones, much stronger compression, and harder coreference.
- SUMMAC tested summaries at 10% compression. For 200-document clusters, the authors needed compression of roughly 1% or even 0.1%.
- In an informal test, three people summarized 10 articles about the Microsoft Trial. Each produced a very different summary: one on courtroom details, one on the day's gist, and one on the trial's goals and outcome.
- Six extraction strategies are listed, including common sections only, a centroid document, and centroid or latest document plus outliers. The proposed strategy combines common and unique sections with time weighting.
- The planned evaluation has two parts:
  - Users sort sentences into clusters and build their own summaries, which serve as a gold standard.
  - Users rate system sentences as good, okay or bad, and suggest better replacements for weak ones.
  - Summarizer #6 is compared with #1, which takes common portions with no anti-redundancy, and with #3, a centroid-document summary.

## Interesting ideas
- **Treat redundancy as a first-class objective.** When choosing passages, score them against what has already been selected, not only against the query. This is useful wherever outputs overlap heavily, such as search results, news aggregation or retrieval for RAG.
- **Make the presentation order a configurable module.** Passages can be ordered by document rank, by importance first, by topic cluster or along a time line. A deliberate ordering choice can change how readable the same set of sentences is.
- **Centroid plus outliers.** Summarize the most typical document, then add a little material from atypical ones. This is a cheap way to combine the core story with coverage of side information, and it carries over to clustering of any text collection.
- **Ask raters for replacements, not just scores.** When a sentence is judged okay or bad, the rater picks a better sentence from the source. This produces richer feedback that can also be used for training.
- **Plan for update summaries and inconsistency detection.** New summaries should account for earlier ones, and conflicting figures across sources should be flagged. Both matter for monitoring news, trade or other data streams over time.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Carnegie Mellon University](https://md.research.elamur.ai/free/en/universities/carnegie-mellon-university/) — University, Pittsburgh, United States · 365 works · 158,048 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 9,085 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 8,888 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 9,296 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 9,206 works

🧭 Topics (1): [📝 Summarization](https://md.research.elamur.ai/free/en/topics/summarization/) · 159 works

## Cite

Cite as: Jade Goldstein, Vibhu Mittal, Jaime Carbonell and Mark Kantrowitz. 2000. Multi-Document Summarization By Sentence Extraction. NAACL-ANLP 2000 Workshop: Automatic Summarization.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{goldstein2000multi,
  title = {{Multi-Document Summarization By Sentence Extraction}},
  author = {Jade Goldstein, Vibhu Mittal, Jaime Carbonell and Mark Kantrowitz},
  booktitle = {NAACL-ANLP 2000 Workshop: Automatic Summarization},
  year = {2000},
  doi = {10.3115/1117575.1117580},
  url = {https://doi.org/10.3115/1117575.1117580}
}
```

- [← previous Integrating Subject Field Codes into WordNet](https://md.research.elamur.ai/free/en/papers/magnini-2000-integrating-subject-field-codes-into/)
- [next → An Evaluation Tool for Machine Translation: Fast Evaluation for MT Research](https://md.research.elamur.ai/free/en/papers/niessen-2000-evaluation-tool-machine-translation-fast/)
- [↑ back to papers of 2000](https://md.research.elamur.ai/free/en/papers/2000/)

## References

33 references, 2 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. James Allan, Jaime Carbonell, George Doddington,, Jonathan Yamron, and Yiming Yang. 1998. Topic detection and tracking pilot study: Final report. In Proceedings of the DARPA Broadcast News Transcription and Understanding Workshop. [find](https://scholar.google.com/scholar?q=James+Allan%2C+Jaime+Carbonell%2C+George+Doddington%2C%2C+Jonathan+Yamron%2C+and+Yiming+Yang%2E+1998%2E+Topic+detection+and+tracking+pilot+study%3A+Final+report%2E+In+Proceedings+of+the+DARPA+Broadcast+News+Transcripti)
2. Chinatsu Aone, M. E. Okurowski, J. Gorlinsky, and B. Larsen. 1997. A scalable summarization system using robust NLP. In Proceedings of the ACL'97/EACL'97 Workshop on Intelligent Scalable Text Summarization, pages 66-73, Madrid, Spain. [find](https://scholar.google.com/scholar?q=Chinatsu+Aone%2C+M%2E+E%2E+Okurowski%2C+J%2E+Gorlinsky%2C+and+B%2E+Larsen%2E+1997%2E+A+scalable+summarization+system+using+robust+NLP%2E+In+Proceedings+of+the+ACL%2797%2FEACL%2797+Workshop+on+Intelligent+Scalable+Text+Summariz)
3. Breck Baldwin and Thomas S. Morton. 1998. [Dynamic coreference-based summarization](https://md.research.elamur.ai/free/en/papers/baldwin-1998-dynamic-coreference-based-summarization/). In Proceedings of the Third Conference on Empirical Methods in Natural Language Processing (EMNLP-3), Granada, Spain, June.
4. Regina Barzilay and Michael Elhadad. 1997. [Using lexical chains for text summarization](https://md.research.elamur.ai/free/en/papers/barzilay-1997-lexical-chains-text-summarization/). In Proceedings of the ACL'97/EACL'97 Workshop on Intelligent Scalable Text Summarization, pages 10-17, Madrid, Spain.
5. Branimir Boguraev and Chris Kennedy. 1997. Salience based content characterization of text documents. In Proceedings of the ACL'97/EACL'97 Workshop on Intelligent Scalable Text Summarization, pages 2-9,. Madrid, Spain. [find](https://scholar.google.com/scholar?q=Branimir+Boguraev+and+Chris+Kennedy%2E+1997%2E+Salience+based+content+characterization+of+text+documents%2E+In+Proceedings+of+the+ACL%2797%2FEACL%2797+Workshop+on+Intelligent+Scalable+Text+Summarization%2C+pages+2%2D)
6. Chris Buckley. 1985. Implementation of the SMART information retrieval system. Technical Report TR 85-686, Cornell University. [find](https://scholar.google.com/scholar?q=Chris+Buckley%2E+1985%2E+Implementation+of+the+SMART+information+retrieval+system%2E+Technical+Report+TR+85%2D686%2C+Cornell+University%2E)
7. Jaime G. Carbonell and Jade Goldstein. 1998. The use of MMR, diversity-based reranking for reordering documents and producing summaries. In Proceedings of SIGIR-98, Melbourne, Australia, August. [find](https://scholar.google.com/scholar?q=Jaime+G%2E+Carbonell+and+Jade+Goldstein%2E+1998%2E+The+use+of+MMR%2C+diversity%2Dbased+reranking+for+reordering+documents+and+producing+summaries%2E+In+Proceedings+of+SIGIR%2D98%2C+Melbourne%2C+Australia%2C+August%2E)
8. Jade Goldstein and Jaime Carbonell. 1998. The use of mmr and diversity-based reranking in document reranking and summarization. In Proceedings of the 14th Twente Workshop on Language Technology in Multimedia Information Retrieval, pages 152-166, Enschede, the Netherlands, December. [find](https://scholar.google.com/scholar?q=Jade+Goldstein+and+Jaime+Carbonell%2E+1998%2E+The+use+of+mmr+and+diversity%2Dbased+reranking+in+document+reranking+and+summarization%2E+In+Proceedings+of+the+14th+Twente+Workshop+on+Language+Technology+in+Mul)
9. Jade Goldstein, Mark Kantrowitz, Vibhu O. Mittal, and • Jaime G. Carbonell. 1999. Summarizing Text Documents: Sentence Selection and Evaluation Metrics. Irf Proceedings of the 22nd International ACM SIGIR I i I I ! [find](https://scholar.google.com/scholar?q=Jade+Goldstein%2C+Mark+Kantrowitz%2C+Vibhu+O%2E+Mittal%2C+and+%E2%80%A2+Jaime+G%2E+Carbonell%2E+1999%2E+Summarizing+Text+Documents%3A+Sentence+Selection+and+Evaluation+Metrics%2E+Irf+Proceedings+of+the+22nd+International+ACM+S)
10. I ! I I ! I I Conference on Research and Development in Information Retrieval (S1G1R-99), pages 121-128, Berkeley, CA. [find](https://scholar.google.com/scholar?q=I+%21+I+I+%21+I+I+Conference+on+Research+and+Development+in+Information+Retrieval+%28S1G1R%2D99%29%2C+pages+121%2D128%2C+Berkeley%2C+CA%2E)
11. Eduard Hovy and Chin-Yew Lin. 1997. Automated text summarization in SUMMARIST. In ACUEACL-97 Workshop on Intelligent Scalable Text Summarization, pages 18-24, Madrid, Spain, July. [find](https://scholar.google.com/scholar?q=Eduard+Hovy+and+Chin%2DYew+Lin%2E+1997%2E+Automated+text+summarization+in+SUMMARIST%2E+In+ACUEACL%2D97+Workshop+on+Intelligent+Scalable+Text+Summarization%2C+pages+18%2D24%2C+Madrid%2C+Spain%2C+July%2E)
12. Judith L. Klavans and James Shaw. 1995. Lexical semantics in summarization. In Proceedings of the First Annual Workshop of the IFIP Working Group FOR NLP and KR, Nantes, France, April. [find](https://scholar.google.com/scholar?q=Judith+L%2E+Klavans+and+James+Shaw%2E+1995%2E+Lexical+semantics+in+summarization%2E+In+Proceedings+of+the+First+Annual+Workshop+of+the+IFIP+Working+Group+FOR+NLP+and+KR%2C+Nantes%2C+France%2C+April%2E)
13. Julian M. Kupiec, Jan Pedersen, and Francine Chen. 1995. A trainable document summarizer. In Proceedings of the 18th Annual Int. ACM/SIG1R Coaference on Research and Development in IR, pages 68-73, Seattle, WA, July. [find](https://scholar.google.com/scholar?q=Julian+M%2E+Kupiec%2C+Jan+Pedersen%2C+and+Francine+Chen%2E+1995%2E+A+trainable+document+summarizer%2E+In+Proceedings+of+the+18th+Annual+Int%2E+ACM%2FSIG1R+Coaference+on+Research+and+Development+in+IR%2C+pages+68%2D73%2C+Se)
14. P. H. Luhn. 1958. Automatic creation of literature abstracts. IBM Journal, pages 159-165. [find](https://scholar.google.com/scholar?q=P%2E+H%2E+Luhn%2E+1958%2E+Automatic+creation+of+literature+abstracts%2E+IBM+Journal%2C+pages+159%2D165%2E)
15. Inderjeet Mani and Eric Bloedern. 1997. Multi-document summarization by graph search and merging. In Proceedings of AAA1-97, pages 622--628. AAAI. [find](https://scholar.google.com/scholar?q=Inderjeet+Mani+and+Eric+Bloedern%2E+1997%2E+Multi%2Ddocument+summarization+by+graph+search+and+merging%2E+In+Proceedings+of+AAA1%2D97%2C+pages+622%2D%2D628%2E+AAAI%2E)
16. Inderjeet Mani and Eric Bloedom. 1999. Summarizing similarities and differences among related documents. Information Retrieval, 1:35-67. [find](https://scholar.google.com/scholar?q=Inderjeet+Mani+and+Eric+Bloedom%2E+1999%2E+Summarizing+similarities+and+differences+among+related+documents%2E+Information+Retrieval%2C+1%3A35%2D67%2E)
17. Daniel'Marcu. 1997. From discourse structures to text summaries. In Proceedings of the ACL'97/EACL'97 Workshop on Intelligent Scalable Text Summarization, pages 82-88, Madrid, Spain. [find](https://scholar.google.com/scholar?q=Daniel%27Marcu%2E+1997%2E+From+discourse+structures+to+text+summaries%2E+In+Proceedings+of+the+ACL%2797%2FEACL%2797+Workshop+on+Intelligent+Scalable+Text+Summarization%2C+pages+82%2D88%2C+Madrid%2C+Spain%2E)
18. Kathleen McKeown, Jacques Robin, and Karen Kukich. 1995. Designing and evaluating a new revision-based model for summary generation. Info. Proc. and Management, 31 (5). [find](https://scholar.google.com/scholar?q=Kathleen+McKeown%2C+Jacques+Robin%2C+and+Karen+Kukich%2E+1995%2E+Designing+and+evaluating+a+new+revision%2Dbased+model+for+summary+generation%2E+Info%2E+Proc%2E+and+Management%2C+31+%285%29%2E)
19. Kathleen McKeown, Judith Klavans, Vasileios Hatzivassiloglou, Regina Barzilay, and Eleazar Eskin. 1999. Towards Multidocument Summarization by Reformulation: Progress and Prospects. In Proceedings of AAAI-99, pages 453--460, Orlando, FL, July. [find](https://scholar.google.com/scholar?q=Kathleen+McKeown%2C+Judith+Klavans%2C+Vasileios+Hatzivassiloglou%2C+Regina+Barzilay%2C+and+Eleazar+Eskin%2E+1999%2E+Towards+Multidocument+Summarization+by+Reformulation%3A+Progress+and+Prospects%2E+In+Proceedings+of+)
20. Mandar Mitra, Amit Singhal, and Chris Buckley. 1997. Automatic text summarization by paragraph extraction. In ACL/EACL-97 Workshop on Intelligent Scalable Text Summarization, pages 31-36, Madrid, Spain, July. [find](https://scholar.google.com/scholar?q=Mandar+Mitra%2C+Amit+Singhal%2C+and+Chris+Buckley%2E+1997%2E+Automatic+text+summarization+by+paragraph+extraction%2E+In+ACL%2FEACL%2D97+Workshop+on+Intelligent+Scalable+Text+Summarization%2C+pages+31%2D36%2C+Madrid%2C+Spai)
21. Chris D. Paice. 1990. Constructing literature abstracts by computer: Techniques and prospects. Info. Proc. and Management, 26:171-186. [find](https://scholar.google.com/scholar?q=Chris+D%2E+Paice%2E+1990%2E+Constructing+literature+abstracts+by+computer%3A+Techniques+and+prospects%2E+Info%2E+Proc%2E+and+Management%2C+26%3A171%2D186%2E)
22. Dragomir Radev and Kathy McKeown. 1998. Generating natural language summaries from multiple online sources. Compuutational Linguistics. [find](https://scholar.google.com/scholar?q=Dragomir+Radev+and+Kathy+McKeown%2E+1998%2E+Generating+natural+language+summaries+from+multiple+online+sources%2E+Compuutational+Linguistics%2E)
23. Gerald Salttm. 1970. Automatic processing of foreign language docuemnts. Journal of American Society for Information Sciences, 21:187-194. [find](https://scholar.google.com/scholar?q=Gerald+Salttm%2E+1970%2E+Automatic+processing+of+foreign+language+docuemnts%2E+Journal+of+American+Society+for+Information+Sciences%2C+21%3A187%2D194%2E)
24. Gerald Salton. 1989. Automatic Text Processing: The Transformation, Analysis, and Retrieval of Information by Computer. Addison-Wesley. [find](https://scholar.google.com/scholar?q=Gerald+Salton%2E+1989%2E+Automatic+Text+Processing%3A+The+Transformation%2C+Analysis%2C+and+Retrieval+of+Information+by+Computer%2E+Addison%2DWesley%2E)
25. James Shaw. 1995. Conciseness through aggregation in text generation. In Proceedings of 33rd Association for Computational Linguistics, pages 329-331. [find](https://scholar.google.com/scholar?q=James+Shaw%2E+1995%2E+Conciseness+through+aggregation+in+text+generation%2E+In+Proceedings+of+33rd+Association+for+Computational+Linguistics%2C+pages+329%2D331%2E)
26. Gees C. Stein, Tomek Strzalkowski, and G. Bowden Wise. 1999. Summarizing Multiple Documents Using Text Extraction and Interactive Clustering. In Proceedings of PacLing-99: The Pacific Rim Conference on Computational Linguistics, pages 200-208, Waterloo, Canada. [find](https://scholar.google.com/scholar?q=Gees+C%2E+Stein%2C+Tomek+Strzalkowski%2C+and+G%2E+Bowden+Wise%2E+1999%2E+Summarizing+Multiple+Documents+Using+Text+Extraction+and+Interactive+Clustering%2E+In+Proceedings+of+PacLing%2D99%3A+The+Pacific+Rim+Conference+o)
27. Tomek Strzalkowski, Jin Wang, and Bowden Wise. 1998. A robust practical text summarization system. In AAAI Intelligent Text Summarization Workshop, pages 26-30, Stanford, CA, March. [find](https://scholar.google.com/scholar?q=Tomek+Strzalkowski%2C+Jin+Wang%2C+and+Bowden+Wise%2E+1998%2E+A+robust+practical+text+summarization+system%2E+In+AAAI+Intelligent+Text+Summarization+Workshop%2C+pages+26%2D30%2C+Stanford%2C+CA%2C+March%2E)
28. J. I. Tait. 1983. Automatic Summarizing of English Texts. Ph.D. thesis, University of Cambridge, Cambridge, UK. [find](https://scholar.google.com/scholar?q=J%2E+I%2E+Tait%2E+1983%2E+Automatic+Summarizing+of+English+Texts%2E+Ph%2ED%2E+thesis%2C+University+of+Cambridge%2C+Cambridge%2C+UK%2E)
29. Simone Teufel and Marc Moens. 1997. Sentence extraction as a classification task. In ACL/EACL-97 Workshop on Intelligent Scalable Text Summarization, pages 58-65, Madrid, Spain, July. [find](https://scholar.google.com/scholar?q=Simone+Teufel+and+Marc+Moens%2E+1997%2E+Sentence+extraction+as+a+classification+task%2E+In+ACL%2FEACL%2D97+Workshop+on+Intelligent+Scalable+Text+Summarization%2C+pages+58%2D65%2C+Madrid%2C+Spain%2C+July%2E)
30. TIPSTER. 1998a. Tipster text phase III 18-month workshop notes, May. [find](https://scholar.google.com/scholar?q=TIPSTER%2E+1998a%2E+Tipster+text+phase+III+18%2Dmonth+workshop+notes%2C+May%2E)
31. Fairfax, VA. TIPSTER. 1998b. Tipster text phase III 24-month workshop notes, October. [find](https://scholar.google.com/scholar?q=Fairfax%2C+VA%2E+TIPSTER%2E+1998b%2E+Tipster+text+phase+III+24%2Dmonth+workshop+notes%2C+October%2E)
32. Baltimore, MD. Charles J. van Rijsbergen. 1979. Information Retrieval. Butterworths, London. Yiming Yang, Tom Pierce, and Jaime 13. [find](https://scholar.google.com/scholar?q=Baltimore%2C+MD%2E+Charles+J%2E+van+Rijsbergen%2E+1979%2E+Information+Retrieval%2E+Butterworths%2C+London%2E+Yiming+Yang%2C+Tom+Pierce%2C+and+Jaime+13%2E)
33. Carbonell. 1998. A study on retrospective and on-line event detection. In Proceedings of the 21th Ann lnt ACM SI-G1R Conference on Research and Development in Information Retrieval ( SIGIR'98), pages 28-36. :Yiming Yang, Jaime G. Carbonell, Ralf D. Brown, Tom Pierce, Brian T. Archibald, and Xin Liu.… [find](https://scholar.google.com/scholar?q=Carbonell%2E+1998%2E+A+study+on+retrospective+and+on%2Dline+event+detection%2E+In+Proceedings+of+the+21th+Ann+lnt+ACM+SI%2DG1R+Conference+on+Research+and+Development+in+Information+Retrieval+%28+SIGIR%2798%29%2C+pages+)

*Updated 24 October 2026*