---
title: "SAMSum Corpus — paper by Gliwa et al. (2019)"
description: "The paper addresses abstractive dialogue summarization, specifically of short messenger… 23 references, cited by 380 papers here."
authors: ["Bogdan Gliwa", "Iwona Mochol", "Maciej Biesek", "Aleksander Wawer"]
year: 2019
venue: "Proceedings of the 2nd Workshop on New Frontiers in Summarization"
source: "https://aclanthology.org/D19-5409/"
doi: "10.18653/v1/d19-5409"
arxiv: "1911.12237"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/gliwa-2019-samsum-corpus-human-annotated-dialogue/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/gliwa-2019-samsum-corpus-human-annotated-dialogue/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2019](https://md.research.elamur.ai/free/en/papers/2019/) / SAMSum Corpus: A Human-annotated Dialogue Dataset for Abstractive Summarization

# SAMSum Corpus: A Human-annotated Dialogue Dataset for Abstractive Summarization

**Authors:** Bogdan Gliwa, Iwona Mochol, Maciej Biesek and Aleksander Wawer · **Year:** 2019 · **Venue:** [Proceedings of the 2nd Workshop on New Frontiers in Summarization](https://md.research.elamur.ai/free/en/venues/workshop-on-new-frontiers-in-summarization/) ([Workshop on New Frontiers in Summarization 2019](https://md.research.elamur.ai/free/en/venues/workshop-on-new-frontiers-in-summarization/2019/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/D19-5409/) · [PDF](https://aclanthology.org/D19-5409.pdf) · [DOI 10.18653/v1/d19-5409](https://doi.org/10.18653/v1/d19-5409) · [arXiv 1911.12237](https://arxiv.org/abs/1911.12237) · [OpenAlex](https://openalex.org/W2989743967)

**Citations:** 335 citations (OpenAlex, 2026-10-14) · [380](https://md.research.elamur.ai/free/en/papers/gliwa-2019-samsum-corpus-human-annotated-dialogue/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary

The paper addresses abstractive dialogue summarization, specifically of short messenger conversations. Before it, there was no large, high-quality dataset for this task. Existing resources (AMI, Argumentative Dialogue Summary Corpus) contain very few examples, while ready-made data of other types (IRC, SMS, movie subtitles, meeting transcripts) bear little resemblance to ordinary chat. The authors from Samsung R&D Institute Poland therefore asked linguists fluent in English to write dialogues similar to their own everyday conversations. Language experts then wrote a concise third-person summary for each dialogue, mentioning the participants' names. This produced the SAMSum Corpus. The authors cleaned the data semi-automatically and checked how realistic the dialogues are. On the corpus, they compared simple heuristic baselines and neural models (Pointer-Generator, Fast Abs RL, Transformer, LightConv/DynamicConv), including with the addition of CNN/Daily Mail data. The main finding is that models obtain higher ROUGE scores on dialogues than on news, but human evaluators rate these summaries as worse. Hence, dialogues require dedicated models and new quality metrics.

## Key points

- **Corpus size:** SAMSum contains 16,369 dialogues. Split: 14,732 for training, 818 for validation, 819 for testing. About 75% of the conversations are between two people. By length, the dialogues are evenly distributed across four groups: 3–6, 7–12, 13–18, and 19–30 utterances.
- **Realism check:** on a sample of 50 dialogues, 94% were judged to resemble real chats. For ready-made dialogue datasets, this figure was only 28%.
- **Baselines:** the authors proposed the heuristics MIDDLE-n, LONGEST-n, LONGER-THAN-n, and MOST-ACTIVE-PERSON. The best was LONGEST-3 (R-1 ≈ 32.5, R-2 ≈ 10.3), and even Lead-3 performed better than MIDDLE.
- **Data preparation for models:** utterance boundaries were marked with a special token (`<EOU>` or `|`). The Fast Abs RL Enhanced variant appends the names of the other interlocutors to each utterance so that the model names the participants correctly. Among the strongest models is DynamicConv with GPT-2 embeddings.
- **The ROUGE problem:** this metric poorly reflects the quality of dialogue summaries. For example, a factually incorrect summary received a higher ROUGE score than a correct one. The authors propose developing new metrics, as well as more challenging corpora with longer, multi-topic conversations.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Samsung](https://md.research.elamur.ai/free/en/companies/samsung-pl/) — Company, Warsaw, Poland · 2 works · 716 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works

🧭 Topics (2): [🗨️ Dialogue systems](https://md.research.elamur.ai/free/en/topics/dialogue-systems/) · 624 works · [📝 Summarization](https://md.research.elamur.ai/free/en/topics/summarization/) · 232 works

💡 Concepts (1): [💡 Text summarization](https://md.research.elamur.ai/free/en/concepts/text-summarization/) · 49 works

## Cite

Cite as: Bogdan Gliwa, Iwona Mochol, Maciej Biesek and Aleksander Wawer. 2019. SAMSum Corpus: A Human-annotated Dialogue Dataset for Abstractive Summarization. Proceedings of the 2nd Workshop on New Frontiers in Summarization.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{gliwa2019samsum,
  title = {{SAMSum Corpus: A Human-annotated Dialogue Dataset for Abstractive Summarization}},
  author = {Bogdan Gliwa, Iwona Mochol, Maciej Biesek and Aleksander Wawer},
  booktitle = {Proceedings of the 2nd Workshop on New Frontiers in Summarization},
  year = {2019},
  doi = {10.18653/v1/d19-5409},
  eprint = {1911.12237},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.18653/v1/d19-5409}
}
```

- [← previous A Multiscale Visualization of Attention in the Transformer Model](https://md.research.elamur.ai/free/en/papers/vig-2019-multiscale-visualization-attention-transformer-model/)
- [next → Transfer Learning in Natural Language Processing](https://md.research.elamur.ai/free/en/papers/ruder-2019-transfer-learning-natural-language-processing/)
- [↑ back to papers of 2019](https://md.research.elamur.ai/free/en/papers/2019/)

## References

23 references, 11 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Yashar Mehdad, Giuseppe Carenini, and Raymond T. Ng. 2014. Abstractive summarization of spoken and written conversations based on phrasal queries. In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics, volume 1, pages 1220–1230. [find](https://scholar.google.com/scholar?q=Yashar+Mehdad%2C+Giuseppe+Carenini%2C+and+Raymond+T%2E+Ng%2E+2014%2E+Abstractive+summarization+of+spoken+and+written+conversations+based+on+phrasal+queries%2E+In+Proceedings+of+the+52nd+Annual+Meeting+of+the+Asso)
2. Siddhartha Banerjee, Prasenjit Mitra, and Kazunari Sugiyama. 2015. Abstractive meeting summarization using dependency graph fusion. In Proceedings of the 24th International Conference on World Wide Web, pages 5–6. [find](https://scholar.google.com/scholar?q=Siddhartha+Banerjee%2C+Prasenjit+Mitra%2C+and+Kazunari+Sugiyama%2E+2015%2E+Abstractive+meeting+summarization+using+dependency+graph+fusion%2E+In+Proceedings+of+the+24th+International+Conference+on+World+Wide+We)
3. Amita Misra, Pranav Anand, Jean Fox Tree, and Marilyn Walker. 2015. Using summarization to discover argument facets in online idealogical dialog. In The North American Chapter of the Association for Computational Linguistics (NAACL). [find](https://scholar.google.com/scholar?q=Amita+Misra%2C+Pranav+Anand%2C+Jean+Fox+Tree%2C+and+Marilyn+Walker%2E+2015%2E+Using+summarization+to+discover+argument+facets+in+online+idealogical+dialog%2E+In+The+North+American+Chapter+of+the+Association+for+C)
4. Yen-Chun Chen and Mohit Bansal. 2018. [Fast abstractive summarization with reinforce-selected sentence rewriting](https://md.research.elamur.ai/free/en/papers/chen-2018-fast-abstractive-summarization-reinforce-selected/). In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics, pages 675–686.
5. Ramesh Nallapati, Bowen Zhou, Cicero Nogueira dos Santos, Caglar Gulcehre, and Bing Xiang. 2016. [Abstractive text summarization using sequence-to-sequence rnns and beyond](https://md.research.elamur.ai/free/en/papers/nallapati-2016-abstractive-text-summarization-sequence-sequence/). In Computational Natural Language Learning.
6. Sumit Chopra, Michael Auli, and Alexander M. Rush. 2016. [Abstractive sentence summarization with attentive recurrent neural networks](https://md.research.elamur.ai/free/en/papers/chopra-2016-abstractive-sentence-summarization-attentive-recurrent/). In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 93–âĂŞ98.
7. Nikola Nikolov, Michael Pfeiffer, and Richard Hahnloser. 2018. Data-driven summarization of scientific articles. In Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018). [find](https://scholar.google.com/scholar?q=Nikola+Nikolov%2C+Michael+Pfeiffer%2C+and+Richard+Hahnloser%2E+2018%2E+Data%2Ddriven+summarization+of+scientific+articles%2E+In+Proceedings+of+the+Eleventh+International+Conference+on+Language+Resources+and+Evalu)
8. Prakhar Ganesh and Saket Dingliwal. 2019. Abstractive summarization of spoken and written conversation. arXiv:1902.01615. — [arXiv:1902.01615](https://arxiv.org/abs/1902.01615) [find](https://scholar.google.com/scholar?q=Prakhar+Ganesh+and+Saket+Dingliwal%2E+2019%2E+Abstractive+summarization+of+spoken+and+written+conversation%2E+arXiv%3A1902%2E01615%2E+%E2%80%94+%5BarXiv%3A1902%2E01615%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F1902%2E01615%29)
9. Myle Ott, Sergey Edunov, Alexei Baevski, Angela Fan, Sam Gross, Nathan Ng, David Grangier, and Michael Auli. 2019. [fairseq: A fast, extensible toolkit for sequence modeling](https://md.research.elamur.ai/free/en/papers/ott-2019-fairseq-fast-extensible-toolkit-sequence/). In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics (Demonstr…
10. Sebastian Gehrmann, Yuntian Deng, and Alexander Rush. 2018. [Bottom-up abstractive summarization](https://md.research.elamur.ai/free/en/papers/gehrmann-2018-bottom-up-abstractive-summarization/). In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 4098–4109.
11. Alec Radford, Jeff Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. 2019. Language models are unsupervised multitask learners. [find](https://scholar.google.com/scholar?q=Alec+Radford%2C+Jeff+Wu%2C+Rewon+Child%2C+David+Luan%2C+Dario+Amodei%2C+and+Ilya+Sutskever%2E+2019%2E+Language+models+are+unsupervised+multitask+learners%2E)
12. Chih-Wen Goo and Yun-Nung Chen. 2018. Abstractive dialogue summarization with sentence-gated modeling optimized by dialogue acts. 2018 IEEE Spoken Language Technology Workshop (SLT), pages 735–742. [find](https://scholar.google.com/scholar?q=Chih%2DWen+Goo+and+Yun%2DNung+Chen%2E+2018%2E+Abstractive+dialogue+summarization+with+sentence%2Dgated+modeling+optimized+by+dialogue+acts%2E+2018+IEEE+Spoken+Language+Technology+Workshop+%28SLT%29%2C+pages+735%E2%80%93742%2E)
13. Alexander M. Rush, Sumit Chopra, and Jason Weston. 2015. [A neural attention model for abstractive sentence summarization](https://md.research.elamur.ai/free/en/papers/rush-2015-neural-attention-model-abstractive-sentence/). In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 379–389.
14. Karl M. Hermann, Tomà as Kociská, Edward Grefenstette, Lasse Espeholt, Will Kay, Mustafa Suleyman, and Phil Blunsom. 2015. [Teaching machines to read and comprehend](https://md.research.elamur.ai/free/en/papers/hermann-2015-teaching-machines-read-comprehend/). CoRR, abs/1506.03340.
15. Abigail See, Peter J. Liu, and Christopher D. Manning. 2017. Get to the point: Summarization with pointer-generator networks. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, volume 1, pages 1073–1083. [find](https://scholar.google.com/scholar?q=Abigail+See%2C+Peter+J%2E+Liu%2C+and+Christopher+D%2E+Manning%2E+2017%2E+Get+to+the+point%3A+Summarization+with+pointer%2Dgenerator+networks%2E+In+Proceedings+of+the+55th+Annual+Meeting+of+the+Association+for+Computati)
16. Urvashi Khandelwal, Kevin Clark, Dan Jurafsky, and Lukasz Kaiser. 2019. Sample efficient text summarization using a single pre-trained transformer. CoRR, abs/1905.08836. [find](https://scholar.google.com/scholar?q=Urvashi+Khandelwal%2C+Kevin+Clark%2C+Dan+Jurafsky%2C+and+Lukasz+Kaiser%2E+2019%2E+Sample+efficient+text+summarization+using+a+single+pre%2Dtrained+transformer%2E+CoRR%2C+abs%2F1905%2E08836%2E)
17. Rico Sennrich, Barry Haddow, and Alexandra Birch. 2015. [Neural machine translation of rare words with subword units](https://md.research.elamur.ai/free/en/papers/sennrich-2016-neural-machine-translation-rare-words/). CoRR, abs/1508.07909.
18. Chin-Yew Lin. 2004. [ROUGE: A package for automatic evaluation of summaries](https://md.research.elamur.ai/free/en/papers/lin-2004-rouge-package-automatic-evaluation-summaries/). In Text Summarization Branches Out, pages 74–81, Barcelona, Spain. Association for Computational Linguistics.
19. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. [Attention is all you need](https://md.research.elamur.ai/free/en/papers/vaswani-2017-attention-all-need/). In Advances in Neural Information Processing Systems 30, pages 5998–6008.
20. I. McCowan, J. Carletta, W. Kraaij, S. Ashby, S. Bourban, M. Flynn, M. Guillemot, T. Hain, J. Kadlec, V. Karaiskos, M. Kronenthal, G. Lathoud, M. Lincoln, A. Lisowska, W. Post, Dennis Reidsma, and P. Wellner. 2005. The ami meeting corpus. In Proceedings of Measuring Behavior 2005, 5th International… [find](https://scholar.google.com/scholar?q=I%2E+McCowan%2C+J%2E+Carletta%2C+W%2E+Kraaij%2C+S%2E+Ashby%2C+S%2E+Bourban%2C+M%2E+Flynn%2C+M%2E+Guillemot%2C+T%2E+Hain%2C+J%2E+Kadlec%2C+V%2E+Karaiskos%2C+M%2E+Kronenthal%2C+G%2E+Lathoud%2C+M%2E+Lincoln%2C+A%2E+Lisowska%2C+W%2E+Post%2C+Dennis+Reidsma%2C+and+P%2E+)
21. Felix Wu, Angela Fan, Alexei Baevski, Yann Dauphin, and Michael Auli. 2019. [Pay less attention with lightweight and dynamic convolutions](https://md.research.elamur.ai/free/en/papers/wu-2019-pay-less-attention-lightweight-dynamic/). In International Conference on Learning Representations.
22. Haoyu Zhang, Jianjun Xu, and Ji Wang. 2019. Pretraining-based natural language generation for text summarization. CoRR, abs/1902.09243. [find](https://scholar.google.com/scholar?q=Haoyu+Zhang%2C+Jianjun+Xu%2C+and+Ji+Wang%2E+2019%2E+Pretraining%2Dbased+natural+language+generation+for+text+summarization%2E+CoRR%2C+abs%2F1902%2E09243%2E)
23. Mary L. McHugh. 2012. Interrater reliability: the kappa statistic. Biochemia medica, 22(3):276–282. [find](https://scholar.google.com/scholar?q=Mary+L%2E+McHugh%2E+2012%2E+Interrater+reliability%3A+the+kappa+statistic%2E+Biochemia+medica%2C+22%283%29%3A276%E2%80%93282%2E)

*Updated 31 October 2026*