---
title: "Searching for Effective Neural Extractive Summarization: What"
description: "Zhong et al. build a training–testing testbed on CNN/DailyMail to take these systems apart. 37 references, cited by 61 papers here."
authors: ["Ming Zhong", "Pengfei Liu", "Danqing Wang", "Xipeng Qiu", "Xuanjing Huang"]
year: 2019
venue: "Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics"
source: "https://aclanthology.org/P19-1100/"
doi: "10.18653/v1/p19-1100"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/zhong-2019-searching-effective-neural-extractive-summarization/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/zhong-2019-searching-effective-neural-extractive-summarization/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2019](https://md.research.elamur.ai/free/en/papers/2019/page/3/) / Searching for Effective Neural Extractive Summarization: What Works and What's Next

# Searching for Effective Neural Extractive Summarization: What Works and What's Next

**Authors:** [Ming Zhong](https://md.research.elamur.ai/free/en/authors/ming-zhong/), [Pengfei Liu](https://md.research.elamur.ai/free/en/authors/pengfei-liu/), [Danqing Wang](https://md.research.elamur.ai/free/en/authors/danqing-wang/), [Xipeng Qiu](https://md.research.elamur.ai/free/en/authors/xipeng-qiu/) and [Xuanjing Huang](https://md.research.elamur.ai/free/en/authors/xuanjing-huang/) · **Year:** 2019 · **Venue:** [Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics](https://md.research.elamur.ai/free/en/venues/acl/) ([ACL 2019](https://md.research.elamur.ai/free/en/venues/acl/2019/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/P19-1100/) · [PDF](https://aclanthology.org/P19-1100.pdf) · [DOI 10.18653/v1/p19-1100](https://doi.org/10.18653/v1/p19-1100) · [OpenAlex](https://openalex.org/W2964028111)

**Citations:** 164 citations (OpenAlex, 2026-10-14) · [61](https://md.research.elamur.ai/free/en/papers/zhong-2019-searching-effective-neural-extractive-summarization/cited-by/) in our library, of the 141,840 papers whose bibliographies we read

## Summary
Neural extractive summarizers have improved quickly, but it is unclear which design choices actually drive their gains. Zhong et al. build a training–testing testbed on CNN/DailyMail to take these systems apart. They keep a CNN sentence encoder fixed and vary four things:
- the document encoder (LSTM vs. Transformer)
- the decoder (non auto-regressive sequence labeling, SeqLab, vs. an auto-regressive Pointer network)
- the transferable knowledge (GloVe, BERT, or supervised pre-training on NEWSROOM)
- the learning schema (supervised vs. reinforcement learning)

They probe the models with cross-domain evaluation, sentence-shuffling tests, and analyses of repetition, sentence length and position bias. Auto-regressive decoding generally helps. LSTM overfits to its architecture more readily than Transformer. On CNN/DailyMail, success depends heavily on sentence position. Unsupervised BERT knowledge transfers better than NEWSROOM pre-training, which is hurt by domain shift. An LSTM+Pointer model with BERT, refined with RL, sets a new state of the art. The work guides future architecture choices and warns that benchmark gains may partly reflect positional bias.

## Key points
- On CNN/DailyMail, ROUGE scores (R-1 / R-2 / R-L) rise step by step from the LSTM + Pointer baseline:
  - baseline: 41.56 / 18.77 / 37.83
  - with BERT: 42.39 / 19.51 / 38.69
  - with BERT and RL: 42.69 / 19.60 / 38.85
  - strongest prior system listed: about 41.6 R-1
- In a disentangling test, a Transformer given only positional embeddings, without sentence content, still reached 40.08 R-1. Removing positional information caused a large drop.
- Encoding sentences one at a time with BERT gave about 41.7 R-1. Encoding the full article and mean-pooling tokens gave about 42.3 R-1. This suggests BERT's benefit comes mainly from document-level context, including relations between sentence positions.
- Context-independent word vectors (GloVe, word2vec) added little. NEWSROOM pre-training usually helped slightly but hurt CNN-LSTM-Pointer, which is the configuration most sensitive to sentence order.
- Code was released via the fastNLP repository on GitHub.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Fudan University](https://md.research.elamur.ai/free/en/universities/fudan-university/) — University, Shanghai, China · 65 works · 19,402 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 9,085 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 8,888 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 9,296 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 9,206 works

🧭 Topics (1): [📝 Summarization](https://md.research.elamur.ai/free/en/topics/summarization/) · 159 works

## Cite

Cite as: Ming Zhong, Pengfei Liu, Danqing Wang, Xipeng Qiu and Xuanjing Huang. 2019. Searching for Effective Neural Extractive Summarization: What Works and What's Next. Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{zhong2019searching,
  title = {{Searching for Effective Neural Extractive Summarization: What Works and What's Next}},
  author = {Ming Zhong, Pengfei Liu, Danqing Wang, Xipeng Qiu and Xuanjing Huang},
  booktitle = {Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics},
  year = {2019},
  doi = {10.18653/v1/p19-1100},
  url = {https://doi.org/10.18653/v1/p19-1100}
}
```

- [← previous SParC: Cross-Domain Semantic Parsing in Context](https://md.research.elamur.ai/free/en/papers/yu-2019-sparc-cross-domain-semantic-parsing/)
- [next → A Hierarchically-Labeled Portuguese Hate Speech Dataset](https://md.research.elamur.ai/free/en/papers/fortuna-2019-hierarchically-labeled-portuguese-hate-speech/)
- [↑ back to papers of 2019](https://md.research.elamur.ai/free/en/papers/2019/page/3/)

## References

37 references, 20 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Kristjan Arumae and Fei Liu. 2018. Reinforced extractive summarization with question-focused rewards. In Proceedings of ACL 2018, Student Research Workshop. pages 105–111. [find](https://scholar.google.com/scholar?q=Kristjan+Arumae+and+Fei+Liu%2E+2018%2E+Reinforced+extractive+summarization+with+question%2Dfocused+rewards%2E+In+Proceedings+of+ACL+2018%2C+Student+Research+Workshop%2E+pages+105%E2%80%93111%2E)
2. Asli Celikyilmaz, Antoine Bosselut, Xiaodong He, and Yejin Choi. 2018. [Deep communicating agents for abstractive summarization](https://md.research.elamur.ai/free/en/papers/celikyilmaz-2018-deep-communicating-agents-abstractive-summarization/). In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers). v…
3. Yen-Chun Chen and Mohit Bansal. 2018. [Fast abstractive summarization with reinforce-selected sentence rewriting](https://md.research.elamur.ai/free/en/papers/chen-2018-fast-abstractive-summarization-reinforce-selected/). In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). volume 1, pages 675–686.
4. Jianpeng Cheng and Mirella Lapata. 2016. [Neural summarization by extracting sentences and words](https://md.research.elamur.ai/free/en/papers/cheng-2016-neural-summarization-extracting-sentences-words/).
5. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). volume 1, pages 484–494. [find](https://scholar.google.com/scholar?q=In+Proceedings+of+the+54th+Annual+Meeting+of+the+Association+for+Computational+Linguistics+%28Volume+1%3A+Long+Papers%29%2E+volume+1%2C+pages+484%E2%80%93494%2E)
6. Zihang Dai, Zhilin Yang, Yiming Yang, William W Cohen, Jaime Carbonell, Quoc V Le, and Ruslan Salakhutdinov. 2018. Transformer-xl: Language modeling with longer-term dependency . [find](https://scholar.google.com/scholar?q=Zihang+Dai%2C+Zhilin+Yang%2C+Yiming+Yang%2C+William+W+Cohen%2C+Jaime+Carbonell%2C+Quoc+V+Le%2C+and+Ruslan+Salakhutdinov%2E+2018%2E+Transformer%2Dxl%3A+Language+modeling+with+longer%2Dterm+dependency+%2E)
7. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. [Bert: Pre-training of deep bidirectional transformers for language understanding](https://md.research.elamur.ai/free/en/papers/devlin-2019-bert-pre-training-deep-bidirectional/). arXiv preprint arXiv:1810.04805 .
8. Yue Dong, Yikang Shen, Eric Crawford, Herke van Hoof, and Jackie Chi Kit Cheung. 2018. Banditsum: Extractive summarization as a contextual bandit. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing. pages 3739–3748. [find](https://scholar.google.com/scholar?q=Yue+Dong%2C+Yikang+Shen%2C+Eric+Crawford%2C+Herke+van+Hoof%2C+and+Jackie+Chi+Kit+Cheung%2E+2018%2E+Banditsum%3A+Extractive+summarization+as+a+contextual+bandit%2E+In+Proceedings+of+the+2018+Conference+on+Empirical+Me)
9. Sebastian Gehrmann, Yuntian Deng, and Alexander Rush. 2018. [Bottom-up abstractive summarization](https://md.research.elamur.ai/free/en/papers/gehrmann-2018-bottom-up-abstractive-summarization/). In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing. pages 4098–4109.
10. Max Grusky, Mor Naaman, and Yoav Artzi. 2018. [Newsroom: A dataset of 1.3 million summaries with diverse extractive strategies](https://md.research.elamur.ai/free/en/papers/grusky-2018-newsroom-dataset-1-3-million/). In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers). vo…
11. Karl Moritz Hermann, Tomas Kocisky, Edward Grefenstette, Lasse Espeholt, Will Kay, Mustafa Suleyman, and Phil Blunsom. 2015. [Teaching machines to read and comprehend](https://md.research.elamur.ai/free/en/papers/hermann-2015-teaching-machines-read-comprehend/). In Advances in Neural Information Processing Systems. pages 1693– 1701.
12. Sepp Hochreiter and Jürgen Schmidhuber. 1997. Long short-term memory. Neural computation 9(8):1735–1780. [find](https://scholar.google.com/scholar?q=Sepp+Hochreiter+and+Ju%CC%88rgen+Schmidhuber%2E+1997%2E+Long+short%2Dterm+memory%2E+Neural+computation+9%288%29%3A1735%E2%80%931780%2E)
13. Masaru Isonuma, Toru Fujino, Junichiro Mori, Yutaka Matsuo, and Ichiro Sakata. 2017. Extractive summarization using multi-task learning with document classification. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing. pages 2101–2110. [find](https://scholar.google.com/scholar?q=Masaru+Isonuma%2C+Toru+Fujino%2C+Junichiro+Mori%2C+Yutaka+Matsuo%2C+and+Ichiro+Sakata%2E+2017%2E+Extractive+summarization+using+multi%2Dtask+learning+with+document+classification%2E+In+Proceedings+of+the+2017+Confere)
14. Aishwarya Jadhav and Vaibhav Rajan. 2018. Extractive summarization with swap-net: Sentences and words from alternating pointer networks. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). volume 1, pages 142–151. [find](https://scholar.google.com/scholar?q=Aishwarya+Jadhav+and+Vaibhav+Rajan%2E+2018%2E+Extractive+summarization+with+swap%2Dnet%3A+Sentences+and+words+from+alternating+pointer+networks%2E+In+Proceedings+of+the+56th+Annual+Meeting+of+the+Association+fo)
15. Chris Kedzie, Kathleen McKeown, and Hal Daume III. 2018. [Content selection in deep learning models of summarization](https://md.research.elamur.ai/free/en/papers/kedzie-2018-content-selection-deep-learning-models/). In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing. pages 1818–1828.
16. Technologies, Volume 1 (Long Papers). volume 1, pages 2227–2237. [find](https://scholar.google.com/scholar?q=Technologies%2C+Volume+1+%28Long+Papers%29%2E+volume+1%2C+pages+2227%E2%80%932237%2E)
17. Yoon Kim. 2014. [Convolutional neural networks for sentence classification](https://md.research.elamur.ai/free/en/papers/kim-2014-convolutional-neural-networks-sentence-classification/). arXiv preprint arXiv:1408.5882 .
18. Chin-Yew Lin. 2004. [Rouge: A package for automatic evaluation of summaries](https://md.research.elamur.ai/free/en/papers/lin-2004-rouge-package-automatic-evaluation-summaries/). Text Summarization Branches Out .
19. Pengfei Liu, Xipeng Qiu, Jifan Chen, and Xuanjing Huang. 2016a. Deep fusion lstms for text semantic matching. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). volume 1, pages 1034–1043. [find](https://scholar.google.com/scholar?q=Pengfei+Liu%2C+Xipeng+Qiu%2C+Jifan+Chen%2C+and+Xuanjing+Huang%2E+2016a%2E+Deep+fusion+lstms+for+text+semantic+matching%2E+In+Proceedings+of+the+54th+Annual+Meeting+of+the+Association+for+Computational+Linguistics)
20. Pengfei Liu, Xipeng Qiu, and Xuanjing Huang. 2016b. Recurrent neural network for text classification with multi-task learning. In Proceedings of IJCAI. pages 2873–2879. [find](https://scholar.google.com/scholar?q=Pengfei+Liu%2C+Xipeng+Qiu%2C+and+Xuanjing+Huang%2E+2016b%2E+Recurrent+neural+network+for+text+classification+with+multi%2Dtask+learning%2E+In+Proceedings+of+IJCAI%2E+pages+2873%E2%80%932879%2E)
21. Pengfei Liu, Xipeng Qiu, and Xuanjing Huang. 2017. [Adversarial multi-task learning for text classification](https://md.research.elamur.ai/free/en/papers/liu-2017-adversarial-multi-task-learning-text/). In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). volume 1, pages 1–10.
22. Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean. 2013. [Efficient estimation of word representations in vector space](https://md.research.elamur.ai/free/en/papers/mikolov-2013-efficient-estimation-word-representations-vector/). arXiv preprint arXiv:1301.3781 .
23. Ramesh Nallapati, Feifei Zhai, and Bowen Zhou. 2017. [Summarunner: A recurrent neural network based sequence model for extractive summarization of documents](https://md.research.elamur.ai/free/en/papers/nallapati-2016-summarunner-recurrent-neural-network-based/). In Thirty-First AAAI Conference on Artificial Intelligence.
24. Ramesh Nallapati, Bowen Zhou, Cicero dos Santos, Ça glar Gulçehre, and Bing Xiang. 2016. Abstractive text summarization using sequence-to-sequence rnns and beyond. [find](https://scholar.google.com/scholar?q=Ramesh+Nallapati%2C+Bowen+Zhou%2C+Cicero+dos+Santos%2C+C%CC%A7a+glar+Gulc%CC%A7ehre%2C+and+Bing+Xiang%2E+2016%2E+Abstractive+text+summarization+using+sequence%2Dto%2Dsequence+rnns+and+beyond%2E)
25. Shashi Narayan, Shay B Cohen, and Mirella Lapata. 2018. [Ranking sentences for extractive summarization with reinforcement learning](https://md.research.elamur.ai/free/en/papers/narayan-2018-ranking-sentences-extractive-summarization-reinforcement/). In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers…
26. Romain Paulus, Caiming Xiong, and Richard Socher. 2017. [A deep reinforced model for abstractive summarization](https://md.research.elamur.ai/free/en/papers/paulus-2017-deep-reinforced-model-abstractive-summarization/). arXiv preprint arXiv:1705.04304 .
27. Jeffrey Pennington, Richard Socher, and Christopher Manning. 2014. [Glove: Global vectors for word representation](https://md.research.elamur.ai/free/en/papers/pennington-2014-glove-global-vectors-word-representation/). In Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP). pages 1532–1543.
28. Matthew Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke Zettlemoyer. 2018a. [Deep contextualized word representations](https://md.research.elamur.ai/free/en/papers/peters-2018-deep-contextualized-word-representations/). In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Matthew…
29. Tim Rocktäschel, Edward Grefenstette, Karl Moritz Hermann, Tomáš Kočiskỳ, and Phil Blunsom. 2015. [Reasoning about entailment with neural attention](https://md.research.elamur.ai/free/en/papers/rocktaschel-2015-reasoning-about-entailment-neural-attention/). arXiv preprint arXiv:1509.06664 .
30. Alexander M Rush, Sumit Chopra, and Jason Weston. 2015. A neural attention model for abstractive sentence summarization. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing. pages 379–389. [find](https://scholar.google.com/scholar?q=Alexander+M+Rush%2C+Sumit+Chopra%2C+and+Jason+Weston%2E+2015%2E+A+neural+attention+model+for+abstractive+sentence+summarization%2E+In+Proceedings+of+the+2015+Conference+on+Empirical+Methods+in+Natural+Language+)
31. Abigail See, Peter J Liu, and Christopher D Manning. 2017. Get to the point: Summarization with pointer-generator networks. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). volume 1, pages 1073–1083. [find](https://scholar.google.com/scholar?q=Abigail+See%2C+Peter+J+Liu%2C+and+Christopher+D+Manning%2E+2017%2E+Get+to+the+point%3A+Summarization+with+pointer%2Dgenerator+networks%2E+In+Proceedings+of+the+55th+Annual+Meeting+of+the+Association+for+Computation)
32. Ilya Sutskever, Oriol Vinyals, and Quoc VV Le. 2014. Sequence to sequence learning with neural networks. In Advances in Neural Information Processing Systems. pages 3104–3112. [find](https://scholar.google.com/scholar?q=Ilya+Sutskever%2C+Oriol+Vinyals%2C+and+Quoc+VV+Le%2E+2014%2E+Sequence+to+sequence+learning+with+neural+networks%2E+In+Advances+in+Neural+Information+Processing+Systems%2E+pages+3104%E2%80%933112%2E)
33. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. [Attention is all you need](https://md.research.elamur.ai/free/en/papers/vaswani-2017-attention-all-need/). In Advances in Neural Information Processing Systems. pages 5998–6008.
34. Oriol Vinyals, Samy Bengio, and Manjunath Kudlur. 2015. Order matters: Sequence to sequence for sets. arXiv preprint arXiv:1511.06391 . [find](https://scholar.google.com/scholar?q=Oriol+Vinyals%2C+Samy+Bengio%2C+and+Manjunath+Kudlur%2E+2015%2E+Order+matters%3A+Sequence+to+sequence+for+sets%2E+arXiv+preprint+arXiv%3A1511%2E06391+%2E)
35. Xiaolong Wang, Ross Girshick, Abhinav Gupta, and Kaiming He. 2018. Non-local neural networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. pages 7794–7803. [find](https://scholar.google.com/scholar?q=Xiaolong+Wang%2C+Ross+Girshick%2C+Abhinav+Gupta%2C+and+Kaiming+He%2E+2018%2E+Non%2Dlocal+neural+networks%2E+In+Proceedings+of+the+IEEE+Conference+on+Computer+Vision+and+Pattern+Recognition%2E+pages+7794%E2%80%937803%2E)
36. Yuxiang Wu and Baotian Hu. 2018. Learning to extract coherent summary via deep reinforcement learning. In Thirty-Second AAAI Conference on Artificial Intelligence. [find](https://scholar.google.com/scholar?q=Yuxiang+Wu+and+Baotian+Hu%2E+2018%2E+Learning+to+extract+coherent+summary+via+deep+reinforcement+learning%2E+In+Thirty%2DSecond+AAAI+Conference+on+Artificial+Intelligence%2E)
37. Qingyu Zhou, Nan Yang, Furu Wei, Shaohan Huang, Ming Zhou, and Tiejun Zhao. 2018. [Neural document summarization by jointly learning to score and select sentences](https://md.research.elamur.ai/free/en/papers/zhou-2018-neural-document-summarization-jointly-learning/). In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). volume 1, pages 654–6…

*Updated 24 October 2026*