---
title: "Retrieve, Rerank and Rewrite — paper by Cao et al. (2018)"
description: "Standard seq2seq models for abstractive sentence summarization generate output from the source… 21 references, cited by 71 papers here."
authors: ["Ziqiang Cao", "Wenjie Li", "Sujian Li", "Furu Wei"]
year: 2018
venue: "Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)"
source: "https://aclanthology.org/P18-1015/"
doi: "10.18653/v1/p18-1015"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/cao-2018-retrieve-rerank-rewrite-soft-template/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/cao-2018-retrieve-rerank-rewrite-soft-template/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2018](https://md.research.elamur.ai/free/en/papers/2018/page/2/) / Retrieve, Rerank and Rewrite: Soft Template Based Neural Summarization

# Retrieve, Rerank and Rewrite: Soft Template Based Neural Summarization

**Authors:** [Ziqiang Cao](https://md.research.elamur.ai/free/en/authors/ziqiang-cao/), [Wenjie Li](https://md.research.elamur.ai/free/en/authors/wenjie-li/), [Sujian Li](https://md.research.elamur.ai/free/en/authors/sujian-li/) and [Furu Wei](https://md.research.elamur.ai/free/en/authors/furu-wei/) · **Year:** 2018 · **Venue:** [Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)](https://md.research.elamur.ai/free/en/venues/acl/) ([ACL 2018](https://md.research.elamur.ai/free/en/venues/acl/2018/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/P18-1015/) · [PDF](https://aclanthology.org/P18-1015.pdf) · [DOI 10.18653/v1/p18-1015](https://doi.org/10.18653/v1/p18-1015) · [OpenAlex](https://openalex.org/W2798542795)

**Citations:** 198 citations (OpenAlex, 2026-10-14) · [71](https://md.research.elamur.ai/free/en/papers/cao-2018-retrieve-rerank-rewrite-soft-template/cited-by/) in our library, of the 141,840 papers whose bibliographies we read

## Summary
Standard seq2seq models for abstractive sentence summarization generate output from the source sentence alone, and their behaviour can be erratic. The authors report very short outputs, words repeated dozens of times, and output that largely copies the source in order. Re3Sum addresses this by combining classic template-based summarization with neural generation. Instead of handcrafted rule templates, it treats existing human-written summaries of similar sentences as "soft templates."

The system works in three steps:
- **Retrieve:** Lucene pulls candidate templates from the training corpus.
- **Rerank:** a shared BiRNN encoder and a bilinear scorer estimate how salient each candidate is, measured by ROUGE against the reference.
- **Rewrite:** an attentional decoder writes the final summary using both the source and the chosen template.

The model is trained jointly on English Gigaword. It significantly beats state-of-the-art seq2seq baselines on ROUGE, and the selected templates alone are already competitive. External summaries also make the outputs more stable and readable. This matters because it shows that retrieval can steer neural generation without manual template engineering.

## Key points
- **Problem with baselines:** in the authors' analysis, about 3% of baseline seq2seq summaries had fewer than 3 words, and 4 outputs repeated a single word up to 99 times.
- **Retrieval:** Lucene with default settings (TextField, EnglishAnalyzer) indexes the over-3M-pair corpus and returns the top 30 hits per input as candidate templates.
- **Joint model:**
  - Source and template share one BiRNN encoder.
  - Saliency is predicted with a sigmoid bilinear layer and trained with cross-entropy against summed ROUGE-1 and ROUGE-2 scores.
  - Generation is trained with NLL, and the two losses are summed.
- **Data and training:** English Gigaword with 3.8M train / 189k dev / 1951 test pairs; SGD with batch size 64, dropout 0.3, learning rate 1 halved on plateau.
- **Robustness:** even with random, irrelevant templates, Re3Sum still produces acceptable summaries, which suggests it learns to ignore untrustworthy templates. Code and results were released by the authors.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Peking University](https://md.research.elamur.ai/free/en/universities/peking-university/) — University, Beijing, China · 100 works · 35,618 citations
- [Microsoft Research Asia](https://md.research.elamur.ai/free/en/companies/microsoft-research-asia/) — Company, Beijing, China · 54 works · 59,921 citations
- [Hong Kong Polytechnic University](https://md.research.elamur.ai/free/en/universities/hong-kong-polytechnic-university/) — University, Hong Kong, China · 34 works · 4,476 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 9,085 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 8,888 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 9,296 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 9,206 works

🧭 Topics (1): [📝 Summarization](https://md.research.elamur.ai/free/en/topics/summarization/) · 159 works

## Cite

Cite as: Ziqiang Cao, Wenjie Li, Sujian Li and Furu Wei. 2018. Retrieve, Rerank and Rewrite: Soft Template Based Neural Summarization. Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{cao2018retrieve,
  title = {{Retrieve, Rerank and Rewrite: Soft Template Based Neural Summarization}},
  author = {Ziqiang Cao, Wenjie Li, Sujian Li and Furu Wei},
  booktitle = {Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2018},
  doi = {10.18653/v1/p18-1015},
  url = {https://doi.org/10.18653/v1/p18-1015}
}
```

- [← previous Simpler but More Accurate Semantic Dependency Parsing](https://md.research.elamur.ai/free/en/papers/dozat-2018-simpler-but-more-accurate-semantic/)
- [next → Summarizing Opinions: Aspect Extraction Meets Sentiment Prediction and They Are Both Weakly Supervised](https://md.research.elamur.ai/free/en/papers/angelidis-2018-summarizing-opinions-aspect-extraction-meets/)
- [↑ back to papers of 2018](https://md.research.elamur.ai/free/en/papers/2018/page/2/)

## References

21 references, 7 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Shiqi Shen Ayana, Zhiyuan Liu, and Maosong Sun. 2016. Neural headline generation with minimum risk training. arXiv preprint arXiv:1604.01904. — [arXiv:1604.01904](https://arxiv.org/abs/1604.01904) [find](https://scholar.google.com/scholar?q=Shiqi+Shen+Ayana%2C+Zhiyuan+Liu%2C+and+Maosong+Sun%2E+2016%2E+Neural+headline+generation+with+minimum+risk+training%2E+arXiv+preprint+arXiv%3A1604%2E01904%2E+%E2%80%94+%5BarXiv%3A1604%2E01904%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F1604%2E01904%29)
2. Philipp Koehn and Rebecca Knowles. 2017. Six challenges for neural machine translation. arXiv preprint arXiv:1706.03872. [find](https://scholar.google.com/scholar?q=Philipp+Koehn+and+Rebecca+Knowles%2E+2017%2E+Six+challenges+for+neural+machine+translation%2E+arXiv+preprint+arXiv%3A1706%2E03872%2E)
3. Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. 2014. [Neural machine translation by jointly learning to align and translate](https://md.research.elamur.ai/free/en/papers/bahdanau-2014-neural-machine-translation-jointly-learning/). arXiv preprint arXiv:1409.0473.
4. Chin-Yew Lin. 2004. [Rouge: A package for automatic evaluation of summaries](https://md.research.elamur.ai/free/en/papers/lin-2004-rouge-package-automatic-evaluation-summaries/). In Proceedings of the ACL Workshop, pages 74–81.
5. Minh-Thang Luong, Hieu Pham, and Christopher D Manning. 2015. [Effective approaches to attention-based neural machine translation](https://md.research.elamur.ai/free/en/papers/luong-2015-effective-approaches-attention-based-neural/). arXiv preprint arXiv:1508.04025.
6. Michele Banko, Vibhu O Mittal, and Michael J Witbrock. 2000. Headline generation based on statistical translation. In Proceedings of the 38th Annual Meeting on Association for Computational Linguistics, pages 318–325. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Michele+Banko%2C+Vibhu+O+Mittal%2C+and+Michael+J+Witbrock%2E+2000%2E+Headline+generation+based+on+statistical+translation%2E+In+Proceedings+of+the+38th+Annual+Meeting+on+Association+for+Computational+Linguistic)
7. Christopher D. Manning, Mihai Surdeanu, John Bauer, Jenny Finkel, Steven J. Bethard, and David McClosky. 2014. [The Stanford CoreNLP natural language processing toolkit](https://md.research.elamur.ai/free/en/papers/manning-2014-stanford-corenlp-natural-language-processing/). In Proceedings of ACL: System Demonstrations, pages 55–60.
8. Ziqiang Cao, Chuwei Luo, Wenjie Li, and Sujian Li. 2017a. Joint copying and restricted generation for paraphrase. In AAAI, pages 3152–3158. [find](https://scholar.google.com/scholar?q=Ziqiang+Cao%2C+Chuwei+Luo%2C+Wenjie+Li%2C+and+Sujian+Li%2E+2017a%2E+Joint+copying+and+restricted+generation+for+paraphrase%2E+In+AAAI%2C+pages+3152%E2%80%933158%2E)
9. Ramesh Nallapati, Bowen Zhou, Caglar Gulcehre, Bing Xiang, et al. 2016. Abstractive text summarization using sequence-to-sequence rnns and beyond. arXiv preprint arXiv:1602.06023. [find](https://scholar.google.com/scholar?q=Ramesh+Nallapati%2C+Bowen+Zhou%2C+Caglar+Gulcehre%2C+Bing+Xiang%2C+et+al%2E+2016%2E+Abstractive+text+summarization+using+sequence%2Dto%2Dsequence+rnns+and+beyond%2E+arXiv+preprint+arXiv%3A1602%2E06023%2E)
10. Ziqiang Cao, Furu Wei, Wenjie Li, and Sujian Li. 2017b. Faithful to the original: Fact aware neural abstractive summarization. arXiv preprint arXiv:1711.04434. — [arXiv:1711.04434](https://arxiv.org/abs/1711.04434) [find](https://scholar.google.com/scholar?q=Ziqiang+Cao%2C+Furu+Wei%2C+Wenjie+Li%2C+and+Sujian+Li%2E+2017b%2E+Faithful+to+the+original%3A+Fact+aware+neural+abstractive+summarization%2E+arXiv+preprint+arXiv%3A1711%2E04434%2E+%E2%80%94+%5BarXiv%3A1711%2E04434%5D%28https%3A%2F%2Farxiv%2Eorg%2Fa)
11. Romain Paulus, Caiming Xiong, and Richard Socher. 2017. [A deep reinforced model for abstractive summarization](https://md.research.elamur.ai/free/en/papers/paulus-2017-deep-reinforced-model-abstractive-summarization/). arXiv preprint arXiv:1705.04304.
12. Danqi Chen, Jason Bolton, and Christopher D Manning. 2016. [A thorough examination of the cnn/daily mail reading comprehension task](https://md.research.elamur.ai/free/en/papers/chen-2016-thorough-examination-cnn-daily-mail/). arXiv preprint arXiv:1606.02858.
13. Alexander M Rush, Sumit Chopra, and Jason Weston. 2015a. A neural attention model for abstractive sentence summarization. arXiv preprint arXiv:1509.00685. [find](https://scholar.google.com/scholar?q=Alexander+M+Rush%2C+Sumit+Chopra%2C+and+Jason+Weston%2E+2015a%2E+A+neural+attention+model+for+abstractive+sentence+summarization%2E+arXiv+preprint+arXiv%3A1509%2E00685%2E)
14. Kyunghyun Cho, Bart Van Merriënboer, Caglar Gulcehre, Dzmitry Bahdanau, Fethi Bougares, Holger Schwenk, and Yoshua Bengio. 2014. Learning phrase representations using rnn encoder-decoder for statistical machine translation. arXiv preprint arXiv:1406.1078. [find](https://scholar.google.com/scholar?q=Kyunghyun+Cho%2C+Bart+Van+Merrie%CC%88nboer%2C+Caglar+Gulcehre%2C+Dzmitry+Bahdanau%2C+Fethi+Bougares%2C+Holger+Schwenk%2C+and+Yoshua+Bengio%2E+2014%2E+Learning+phrase+representations+using+rnn+encoder%2Ddecoder+for+statisti)
15. Alexander M. Rush, Sumit Chopra, and Jason Weston. 2015b. A neural attention model for abstractive sentence summarization. In Proceedings of EMNLP, pages 379–389. [find](https://scholar.google.com/scholar?q=Alexander+M%2E+Rush%2C+Sumit+Chopra%2C+and+Jason+Weston%2E+2015b%2E+A+neural+attention+model+for+abstractive+sentence+summarization%2E+In+Proceedings+of+EMNLP%2C+pages+379%E2%80%93389%2E)
16. Sumit Chopra, Michael Auli, Alexander M Rush, and SEAS Harvard. 2016. [Abstractive sentence summarization with attentive recurrent neural networks](https://md.research.elamur.ai/free/en/papers/chopra-2016-abstractive-sentence-summarization-attentive-recurrent/). Proceedings of NAACL-HLT16, pages 93–98.
17. Horacio Saggion. 2017. Automatic text simplification. Synthesis Lectures on Human Language Technologies, 10(1):1–137. [find](https://scholar.google.com/scholar?q=Horacio+Saggion%2E+2017%2E+Automatic+text+simplification%2E+Synthesis+Lectures+on+Human+Language+Technologies%2C+10%281%29%3A1%E2%80%93137%2E)
18. Abigail See, Peter J Liu, and Christopher D Manning. 2017. Get to the point: Summarization with pointer-generator networks. arXiv preprint arXiv:1704.04368. [find](https://scholar.google.com/scholar?q=Abigail+See%2C+Peter+J+Liu%2C+and+Christopher+D+Manning%2E+2017%2E+Get+to+the+point%3A+Summarization+with+pointer%2Dgenerator+networks%2E+arXiv+preprint+arXiv%3A1704%2E04368%2E)
19. Nitish Srivastava, Geoffrey E Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. 2014. Dropout: a simple way to prevent neural networks from overfitting. Journal of Machine Learning Research, 15(1):1929–1958. [find](https://scholar.google.com/scholar?q=Nitish+Srivastava%2C+Geoffrey+E+Hinton%2C+Alex+Krizhevsky%2C+Ilya+Sutskever%2C+and+Ruslan+Salakhutdinov%2E+2014%2E+Dropout%3A+a+simple+way+to+prevent+neural+networks+from+overfitting%2E+Journal+of+Machine+Learning+Re)
20. Yonghui Wu, Mike Schuster, Zhifeng Chen, Quoc V Le, Mohammad Norouzi, Wolfgang Macherey, Maxim Krikun, Yuan Cao, Qin Gao, Klaus Macherey, et al. 2016. Google’s neural machine translation system: Bridging the gap between human and machine translation. arXiv preprint arXiv:1609.08144. [find](https://scholar.google.com/scholar?q=Yonghui+Wu%2C+Mike+Schuster%2C+Zhifeng+Chen%2C+Quoc+V+Le%2C+Mohammad+Norouzi%2C+Wolfgang+Macherey%2C+Maxim+Krikun%2C+Yuan+Cao%2C+Qin+Gao%2C+Klaus+Macherey%2C+et+al%2E+2016%2E+Google%E2%80%99s+neural+machine+translation+system%3A+Bridg)
21. Liang Zhou and Eduard Hovy. 2004. Template-filtered headline summarization. Text Summarization Branches Out. [find](https://scholar.google.com/scholar?q=Liang+Zhou+and+Eduard+Hovy%2E+2004%2E+Template%2Dfiltered+headline+summarization%2E+Text+Summarization+Branches+Out%2E)

*Updated 24 October 2026*