--- title: "A Unified Model for Extractive and Abstractive — Hsu 2018" description: "The paper addresses a trade-off in neural summarization." authors: ["Wan-Ting Hsu", "Chieh-Kai Lin", "Ming-Ying Lee", "Kerui Min", "Jing Tang", "Min Sun"] year: 2018 venue: "Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)" source: "https://aclanthology.org/P18-1013/" doi: "10.18653/v1/p18-1013" licence: "CC BY 4.0 (ACL Anthology)" licence_url: "https://creativecommons.org/licenses/by/4.0/" summary_licence: "Elamur Cosmic License" canonical: "https://research.elamur.ai/free/en/papers/hsu-2018-unified-model-extractive-abstractive-summarization/" --- [Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/hsu-2018-unified-model-extractive-abstractive-summarization/) [home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2018](https://md.research.elamur.ai/free/en/papers/2018/) / A Unified Model for Extractive and Abstractive Summarization using Inconsistency Loss # A Unified Model for Extractive and Abstractive Summarization using Inconsistency Loss **Authors:** Wan-Ting Hsu, Chieh-Kai Lin, Ming-Ying Lee, Kerui Min, Jing Tang, Min Sun · **Year:** 2018 · **Venue:** Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/) **Links:** [Publisher's page](https://aclanthology.org/P18-1013/) · [PDF](https://aclanthology.org/P18-1013.pdf) · [DOI 10.18653/v1/p18-1013](https://doi.org/10.18653/v1/p18-1013) · [OpenAlex](https://openalex.org/W2963545005) **Citations:** 252 citations (openalex-snapshot · 2026-10-14) · [83](#cited-by) in our library, of the 140884 papers whose bibliographies we read ## Summary The paper addresses a trade-off in neural summarization. Extractive systems pick whole sentences and score well on ROUGE, but their output reads poorly. Abstractive systems produce fluent text, yet they can drop or distort facts. The authors join both in one end-to-end network. A sentence extractor following Nallapati et al. (2017) gives each sentence a selection probability. This probability is treated as a fixed sentence-level attention and multiplied into the word-level attention of a pointer-generator abstracter (See et al., 2017), then renormalized. As a result, words from low-scoring sentences become unlikely to be generated. A new inconsistency loss penalizes cases where a word gets strong attention while its sentence gets weak attention, and it needs no extra annotation. The system is trained jointly with the original extractive and abstractive objectives on CNN/Daily Mail. It reports state-of-the-art ROUGE, beats the lead-3 baseline, and is judged most informative and readable in human evaluation. This shows that the two paradigms can reinforce each other rather than compete. ## Key points - **Architecture:** - The extractor is a hierarchical GRU model with word-level and sentence-level RNNs. - The abstracter is the pointer-generator. - The two are combined by scaling and renormalizing word attention with sentence probabilities. - Unlike the dynamic sentence attention in Nallapati et al. (2016b), here sentence attention stays constant for every generated word. - **Inconsistency loss:** It discourages high word attention inside sentences the extractor considers unimportant. Attention visualizations show inconsistent fragments appearing when the model is trained without this loss. - **Automatic evaluation:** On the CNN/Daily Mail dataset, the best configuration (end-to-end training with inconsistency loss) reaches top ROUGE scores and surpasses the strong lead-3 baseline. - **Human evaluation setup:** - Run on Amazon Mechanical Turk, rating informativity, conciseness and readability on a 1–5 scale. - Each example received at least 3 ratings, with 5 requested. - A trap random summary was used to filter out careless raters. - **Human evaluation results:** - The model outperformed pointer-generator, a GAN-based model (Liu et al., 2017) and DeepRL (Paulus et al., 2017). - It scored slightly above the reference summaries on informativity and readability. - The reference summaries remained the most concise. *Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.* Fields: [Natural language processing](https://md.research.elamur.ai/free/en/shelves/nlp/) ## Cited by Cited by 83 papers of our library cite it (of the 140884 whose bibliographies we read); 6 of them have a page here, 77 not yet. - [Bottom-Up Abstractive Summarization](https://md.research.elamur.ai/free/en/papers/gehrmann-2018-bottom-up-abstractive-summarization/) - [Neural Text Summarization: A Critical Evaluation](https://md.research.elamur.ai/free/en/papers/kryscinski-2019-neural-text-summarization-critical-evaluation/) - [Comparing Approaches to Automatic Summarization in Less-Resourced Languages](https://md.research.elamur.ai/free/en/papers/michel-2026-comparing-approaches-automatic-summarization-less/) - [On Extractive and Abstractive Neural Document Summarization with Transformer Language Models](https://md.research.elamur.ai/free/en/papers/pilault-2020-extractive-abstractive-neural-document-summarization/) - [Understanding Factual Errors in Summarization: Errors, Summarizers, Datasets, Error Detectors](https://md.research.elamur.ai/free/en/papers/tang-2023-understanding-factual-errors-summarization-errors/) - [HIBERT: Document Level Pre-training of Hierarchical Bidirectional Transformers for Document Summarization](https://md.research.elamur.ai/free/en/papers/zhang-2019-hibert-document-level-pre-training/) ## Cite The paper itself, not this page. BibTeX from the library's record of the work. ```bibtex @inproceedings{hsu2018unified, title = {{A Unified Model for Extractive and Abstractive Summarization using Inconsistency Loss}}, author = {Hsu, Wan-Ting and Lin, Chieh-Kai and Lee, Ming-Ying and Min, Kerui and Tang, Jing and Sun, Min}, booktitle = {Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)}, year = {2018}, doi = {10.18653/v1/p18-1013}, url = {https://doi.org/10.18653/v1/p18-1013} } ``` - [← previous A Survey on Recent Advances in Named Entity Recognition from Deep Learning models](https://md.research.elamur.ai/free/en/papers/yadav-2018-survey-recent-advances-named-entity/) - [next → Abstract Meaning Representation for Multi-Document Summarization](https://md.research.elamur.ai/free/en/papers/liao-2018-abstract-meaning-representation-multi-document/) - [↑ back to papers of 2018](https://md.research.elamur.ai/free/en/papers/2018/) ## References 2 references, 1 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet. 1. the-art methods in Fig. 6. More examples (5 using CNN/Daily Mail news articles and 3 using non-news articles as inputs) are provided in the supplementary material. [find](https://scholar.google.com/scholar?q=the%2Dart+methods+in+Fig%2E+6%2E+More+examples+%285+using+CNN%2FDaily+Mail+news+articles+and+3+using+non%2Dnews+articles+as+inputs%29+are+provided+in+the+supplementary+material%2E) 2. Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. 2014. Neural machine translation by jointly learning to align and translate. In Proceedings of the 2015 International Conference on Learning Representations. \#\# Conclusion Qian Chen, Xiaodan Zhu, Zhenhua Ling, Si Wei, and Hui Jiang. 2016. Distracti… [find](https://scholar.google.com/scholar?q=Dzmitry+Bahdanau%2C+Kyunghyun+Cho%2C+and+Yoshua+Bengio%2E+2014%2E+Neural+machine+translation+by+jointly+learning+to+align+and+translate%2E+In+Proceedings+of+the+2015+International+Conference+on+Learning+Represe) [our card](https://md.research.elamur.ai/free/en/papers/bahdanau-2014-neural-machine-translation-jointly-learning/)