--- title: "SCURank — paper by Wang et al. (2026) — summary and ideas" description: "The authors propose SCURank, which scores each candidate by the information it contains rather than asking an LLM for a… 39 references." authors: ["Bo-Jyun Wang", "Ying-Jia Lin", "Hung-Yu Kao"] year: 2026 venue: "Findings of the Association for Computational Linguistics: ACL 2026" source: "https://aclanthology.org/2026.findings-acl.1941/" doi: "10.18653/v1/2026.findings-acl.1941" licence: "CC BY 4.0 (ACL Anthology)" licence_url: "https://creativecommons.org/licenses/by/4.0/" summary_licence: "Elamur Cosmic License" canonical: "https://research.elamur.ai/free/en/papers/wang-2026-scurank-ranking-multiple-candidate-summaries/" --- [Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/wang-2026-scurank-ranking-multiple-candidate-summaries/) [home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/42/) / SCURank: Ranking Multiple Candidate Summaries with Summary Content Units for Enhanced Summarization # SCURank: Ranking Multiple Candidate Summaries with Summary Content Units for Enhanced Summarization **Authors:** Bo-Jyun Wang, Ying-Jia Lin and Hung-Yu Kao · **Year:** 2026 · **Venue:** [Findings of the Association for Computational Linguistics: ACL 2026](https://md.research.elamur.ai/free/en/venues/findings/) ([Findings of the Association for Computational Linguistics 2026](https://md.research.elamur.ai/free/en/venues/findings/2026/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/) **Links:** [Publisher's page](https://aclanthology.org/2026.findings-acl.1941/) · [PDF](https://aclanthology.org/2026.findings-acl.1941.pdf) · [DOI 10.18653/v1/2026.findings-acl.1941](https://doi.org/10.18653/v1/2026.findings-acl.1941) · [OpenAlex](https://openalex.org/W7166831547) **Citations:** 0 citations (OpenAlex, 2026-10-14) ## Summary The paper tackles how to choose the best summaries when distilling large language models into a compact summarizer. In contrastive training such as BRIO, candidate summaries must be ranked. ROUGE captures only surface overlap, and LLM-based rankers like GPTRank produce inconsistent, position-biased judgments. The authors propose SCURank, which scores each candidate by the information it contains rather than asking an LLM for a direct ranking. It works in three steps: - gpt-4o-mini breaks every candidate into Summary Content Units, following the SGU approach of Nawrath et al. - The units are embedded with all-mpnet-base-v2 and clustered with HDBSCAN. - Each unit is weighted by how many candidates share its cluster, and a summary's score is the length-normalized sum of its unit weights. The ranked candidates, drawn from several different LLMs, are used to train BART with BRIO. On CNN/DailyMail and XSum, SCURank-trained models beat ROUGE- and LLM-ranked baselines. Using diverse LLM sources also made outputs more abstractive. The work shows that a stable, information-focused ranker improves small distilled summarizers. ## Key points - **Pipeline:** gpt-4o-mini (1-shot, REALSumm example) extracts SCUs, all-mpnet-base-v2 embeds them, HDBSCAN clusters them, and cluster size sets each unit's importance. The LLM is used only for extraction, never for comparing summaries. - **Baselines and data:** compared against ROUGE, BERTScore, BLANC and GPTRank on CNN/DailyMail and XSum, using a BART student trained with BRIO. The setup mirrors GPTRank: one reference plus eight candidates. - **Human evaluation:** 30 articles per dataset were each rated by 3 MTurk annotators. SCURank models scored higher on Faithfulness, Conciseness and Completeness, with Completeness gains significant at p \< 0.01 on both datasets. - **LLM judges:** Grok-4, Claude-4-sonnet and Gemini-2.5-pro each compared outputs twice with the order swapped. All three preferred SCURank-trained models over GPTRank-trained ones. - **Abstractiveness:** on CNN/DailyMail, the SCURank model trained on nine-LLM data (LLMs-9) cut density from 10.34 to 6.73 versus BASE. This narrows the gap to human-written summaries (2.07) and LLM-generated ones (2.70). Code: github.com/IKMLab/SCURank. *Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.* ## Institutions - [National Tsing Hua University](https://md.research.elamur.ai/free/en/universities/national-tsing-hua-university/) — University, Hsinchu, Taiwan · 8 works · 834 citations - [National Cheng Kung University](https://md.research.elamur.ai/free/en/universities/national-cheng-kung-university/) — University, Tainan, Taiwan · 5 works · 615 citations Also on the byline: [Chang Gung University](https://ror.org/00d80zx46). 📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works 🧭 Topics (1): [📝 Summarization](https://md.research.elamur.ai/free/en/topics/summarization/) · 232 works ## Cite Cite as: Bo-Jyun Wang, Ying-Jia Lin and Hung-Yu Kao. 2026. SCURank: Ranking Multiple Candidate Summaries with Summary Content Units for Enhanced Summarization. Findings of the Association for Computational Linguistics: ACL 2026. The paper itself, not this page. BibTeX from the library's record of the work. ```bibtex @inproceedings{wang2026scurank, title = {{SCURank: Ranking Multiple Candidate Summaries with Summary Content Units for Enhanced Summarization}}, author = {Bo-Jyun Wang, Ying-Jia Lin and Hung-Yu Kao}, booktitle = {Findings of the Association for Computational Linguistics: ACL 2026}, year = {2026}, doi = {10.18653/v1/2026.findings-acl.1941}, url = {https://doi.org/10.18653/v1/2026.findings-acl.1941} } ``` - [← previous Scripting History: A Diachronic Urdu Text and Image Corpus from the 18Th to 19Th Centuries](https://md.research.elamur.ai/free/en/papers/shams-2026-scripting-history-diachronic-urdu-text/) - [next → SD-E2: Semantic Exploration for Reasoning Under Token Budgets](https://md.research.elamur.ai/free/en/papers/mishra-2026-sd-e2-semantic-exploration-reasoning/) - [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/42/) ## References 39 references, 21 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet. 1. Anthropic. 2024. Claude 3.5 sonnet: Latest model release. [https://www.anthropic.com/](https://www.anthropic.com/) news/claude-3-5-sonnet. Accessed: 2025-02-01. [↗](https://www.anthropic.com/) 2. Anthropic. 2025. Claude 4: Latest model release. [https://www.anthropic.com/news/](https://www.anthropic.com/news/) claude-4. Accessed: 2025-08-01. [↗](https://www.anthropic.com/news/) 3. Manik Bhandari, Pranav Narayan Gour, Atabak Ashfaq, Pengfei Liu, and Graham Neubig. 2020. [Re-evaluating evaluation in text summarization](https://md.research.elamur.ai/free/en/papers/bhandari-2020-re-evaluating-evaluation-text-summarization/). In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 9347–9359, Online. Association for Computational Linguist… 4. Ricardo J. G. B. Campello, Davoud Moulavi, and Joerg Sander. 2013. Density-based clustering based on hierarchical density estimates. In Advances in Knowledge Discovery and Data Mining, pages 160– 172, Berlin, Heidelberg. Springer Berlin Heidelberg. [find](https://scholar.google.com/scholar?q=Ricardo+J%2E+G%2E+B%2E+Campello%2C+Davoud+Moulavi%2C+and+Joerg+Sander%2E+2013%2E+Density%2Dbased+clustering+based+on+hierarchical+density+estimates%2E+In+Advances+in+Knowledge+Discovery+and+Data+Mining%2C+pages+160%E2%80%93+172%2C) 5. Arman Cohan and Nazli Goharian. 2016. Revisiting summarization evaluation for scientific articles. In Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC’16), pages 806–813, Portorož, Slovenia. European Language Resources Association (ELRA). [find](https://scholar.google.com/scholar?q=Arman+Cohan+and+Nazli+Goharian%2E+2016%2E+Revisiting+summarization+evaluation+for+scientific+articles%2E+In+Proceedings+of+the+Tenth+International+Conference+on+Language+Resources+and+Evaluation+%28LREC%E2%80%9916%29%2C+) 6. Martin Ester, Hans-Peter Kriegel, Jörg Sander, Xiaowei Xu, and 1 others. 1996. A density-based algorithm for discovering clusters in large spatial databases with noise. In kdd, volume 96, pages 226–231. [find](https://scholar.google.com/scholar?q=Martin+Ester%2C+Hans%2DPeter+Kriegel%2C+J%C3%B6rg+Sander%2C+Xiaowei+Xu%2C+and+1+others%2E+1996%2E+A+density%2Dbased+algorithm+for+discovering+clusters+in+large+spatial+databases+with+noise%2E+In+kdd%2C+volume+96%2C+pages+226%E2%80%9323) 7. Jinlan Fu, See-Kiong Ng, Zhengbao Jiang, and Pengfei Liu. 2024. GPTScore: Evaluate as you desire. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), pages 6556–6576, Mexico City, M… [find](https://scholar.google.com/scholar?q=Jinlan+Fu%2C+See%2DKiong+Ng%2C+Zhengbao+Jiang%2C+and+Pengfei+Liu%2E+2024%2E+GPTScore%3A+Evaluate+as+you+desire%2E+In+Proceedings+of+the+2024+Conference+of+the+North+American+Chapter+of+the+Association+for+Computation) 8. Google. 2024. Gemini models. [https://deepmind.google/technologies/gemini/](https://deepmind.google/technologies/gemini/). Accessed: 2025-02-01. [↗](https://deepmind.google/technologies/gemini/) 9. Max Grusky, Mor Naaman, and Yoav Artzi. 2018. [Newsroom: A dataset of 1.3 million summaries with diverse extractive strategies](https://md.research.elamur.ai/free/en/papers/grusky-2018-newsroom-dataset-1-3-million/). In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), pa… 10. Geoffrey Hinton, Oriol Vinyals, and Jeff Dean. 2015. [Distilling the knowledge in a neural network](https://md.research.elamur.ai/free/en/papers/hinton-2015-distilling-knowledge-neural-network/). Preprint, arXiv:1503.02531. 11. Cheng-Yu Hsieh, Chun-Liang Li, Chih-kuan Yeh, Hootan Nakhost, Yasuhisa Fujii, Alex Ratner, Ranjay Krishna, Chen-Yu Lee, and Tomas Pfister. 2023. [Distilling step-by-step! outperforming larger language models with less training data and smaller model sizes](https://md.research.elamur.ai/free/en/papers/hsieh-2023-distilling-step-step-outperforming-larger/). In Findings of the Association for Computati… 12. Pengcheng Jiang, Cao Xiao, Zifeng Wang, Parminder Bhatia, Jimeng Sun, and Jiawei Han. 2024. TriSum: Learning summarization ability from large language models with structured rationale. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistic… [find](https://scholar.google.com/scholar?q=Pengcheng+Jiang%2C+Cao+Xiao%2C+Zifeng+Wang%2C+Parminder+Bhatia%2C+Jimeng+Sun%2C+and+Jiawei+Han%2E+2024%2E+TriSum%3A+Learning+summarization+ability+from+large+language+models+with+structured+rationale%2E+In+Proceedings+) 13. Daniel Kang and Tatsunori B. Hashimoto. 2020. [Improved natural language generation via loss truncation](https://md.research.elamur.ai/free/en/papers/kang-2020-improved-natural-language-generation-loss/). In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 718–731, Online. Association for Computational Linguistics. 14. Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Veselin Stoyanov, and Luke Zettlemoyer. 2020. [BART: Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension](https://md.research.elamur.ai/free/en/papers/lewis-2020-bart-denoising-sequence-sequence-pre/). In Proceedings of the 58th Annual Meeting of the… 15. Chin-Yew Lin. 2004. [ROUGE: A package for automatic evaluation of summaries](https://md.research.elamur.ai/free/en/papers/lin-2004-rouge-package-automatic-evaluation-summaries/). In Text Summarization Branches Out, pages 74–81, Barcelona, Spain. Association for Computational Linguistics. 16. Yang Liu, Dan Iter, Yichong Xu, Shuohang Wang, Ruochen Xu, and Chenguang Zhu. 2023. [G-eval: NLG evaluation using gpt-4 with better human alignment](https://md.research.elamur.ai/free/en/papers/liu-2023-g-eval-nlg-evaluation-gpt/). In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 2511–2522, Singapore. Association for Computational Lin… 17. Yixin Liu and Pengfei Liu. 2021. [SimCLS: A simple framework for contrastive learning of abstractive summarization](https://md.research.elamur.ai/free/en/papers/liu-2021-simcls-simple-framework-contrastive-learning/). In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 2: Short Pa… 18. Yixin Liu, Pengfei Liu, Dragomir Radev, and Graham Neubig. 2022. [BRIO: Bringing order to abstractive summarization](https://md.research.elamur.ai/free/en/papers/liu-2022-brio-bringing-order-abstractive-summarization/). In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 2890–2903, Dublin, Ireland. Association for Computational Ling… 19. Yixin Liu, Kejian Shi, Katherine He, Longtian Ye, Alexander Fabbri, Pengfei Liu, Dragomir Radev, and Arman Cohan. 2024. [On learning to summarize with large language models as references](https://md.research.elamur.ai/free/en/papers/liu-2024-learning-summarize-large-language-models/). In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguist… 20. Joshua Maynez, Shashi Narayan, Bernd Bohnet, and Ryan McDonald. 2020. [On faithfulness and factuality in abstractive summarization](https://md.research.elamur.ai/free/en/papers/maynez-2020-faithfulness-factuality-abstractive-summarization/). In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 1906–1919, Online. Association for Computational Linguistics. 21. MistralAI. 2024. Mistral large 2: A new generation of frontier ai. [https://mistral.ai/news/](https://mistral.ai/news/) mistral-large-2407/. Accessed: 2025-02-01. [↗](https://mistral.ai/news/) 22. Ramesh Nallapati, Bowen Zhou, Cicero dos Santos, Çağlar Gu˙lçehre, and Bing Xiang. 2016. [Abstractive text summarization using sequence-to-sequence RNNs and beyond](https://md.research.elamur.ai/free/en/papers/nallapati-2016-abstractive-text-summarization-sequence-sequence/). In Proceedings of the 20th SIGNLL Conference on Computational Natural Language Learning, pages 280–290, Berlin, Germany. Association fo… 23. Shashi Narayan, Shay B. Cohen, and Mirella Lapata. 2018. [Don’t give me the details, just the summary!](https://md.research.elamur.ai/free/en/papers/narayan-2018-don-t-give-me-details/) topic-aware convolutional neural networks for extreme summarization. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 1797–1807, Brussels, Belgium. As… 24. Marcel Nawrath, Agnieszka Nowak, Tristan Ratz, Danilo Walenta, Juri Opitz, Leonardo Ribeiro, João Sedoc, Daniel Deutsch, Simon Mille, Yixin Liu, Sebastian Gehrmann, Lining Zhang, Saad Mahamood, Miruna Clinciu, Khyathi Chandu, and Yufang Hou. 2024. On the role of summary content units in text summari… [find](https://scholar.google.com/scholar?q=Marcel+Nawrath%2C+Agnieszka+Nowak%2C+Tristan+Ratz%2C+Danilo+Walenta%2C+Juri+Opitz%2C+Leonardo+Ribeiro%2C+Jo%C3%A3o+Sedoc%2C+Daniel+Deutsch%2C+Simon+Mille%2C+Yixin+Liu%2C+Sebastian+Gehrmann%2C+Lining+Zhang%2C+Saad+Mahamood%2C+Miruna) 25. Ani Nenkova and Rebecca Passonneau. 2004. Evaluating content selection in summarization: The pyramid method. In Proceedings of the Human Language Technology Conference of the North American Chapter of the Association for Computational Linguistics: HLT-NAACL 2004, pages 145–152, Boston, Massachusetts… [find](https://scholar.google.com/scholar?q=Ani+Nenkova+and+Rebecca+Passonneau%2E+2004%2E+Evaluating+content+selection+in+summarization%3A+The+pyramid+method%2E+In+Proceedings+of+the+Human+Language+Technology+Conference+of+the+North+American+Chapter+of) 26. OpenAI. 2024a. [Gpt-4 technical report](https://md.research.elamur.ai/free/en/papers/openai-2023-gpt-4-technical-report/). Preprint, arXiv:2303.08774. 27. Nils Reimers and Iryna Gurevych. 2019. [Sentence-BERT: Sentence embeddings using Siamese BERT-networks](https://md.research.elamur.ai/free/en/papers/reimers-2019-sentence-bert-sentence-embeddings-siamese/). In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 3982–3992,… 28. Thibault Sellam, Dipanjan Das, and Ankur Parikh. 2020. [BLEURT: Learning robust metrics for text generation](https://md.research.elamur.ai/free/en/papers/sellam-2020-bleurt-learning-robust-metrics-text/). In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 7881–7892, Online. Association for Computational Linguistics. 29. Ori Shapira, David Gabay, Yang Gao, Hadar Ronen, Ramakanth Pasunuru, Mohit Bansal, Yael Amsterdamer, and Ido Dagan. 2019. Crowdsourcing lightweight pyramids for manual summary evaluation. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguis… [find](https://scholar.google.com/scholar?q=Ori+Shapira%2C+David+Gabay%2C+Yang+Gao%2C+Hadar+Ronen%2C+Ramakanth+Pasunuru%2C+Mohit+Bansal%2C+Yael+Amsterdamer%2C+and+Ido+Dagan%2E+2019%2E+Crowdsourcing+lightweight+pyramids+for+manual+summary+evaluation%2E+In+Proceedin) 30. Chenhui Shen, Liying Cheng, Xuan-Phi Nguyen, Yang You, and Lidong Bing. 2023. [Large language models are not yet human-level evaluators for abstractive summarization](https://md.research.elamur.ai/free/en/papers/shen-2023-large-language-models-are-not/). In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 4215–4233, Singapore. Association for Computational Li… 31. Oleg Vasilyev, Vedant Dharnidharka, and John Bohannon. 2020. [Fill in the BLANC: Human-free quality estimation of document summaries](https://md.research.elamur.ai/free/en/papers/vasilyev-2020-fill-blanc-human-free-quality/). In Proceedings of the First Workshop on Evaluation and Comparison of NLP Systems, pages 11–20, Online. Association for Computational Linguistics. 32. Ashwin Vijayakumar, Michael Cogswell, Ramprasaath Selvaraju, Qing Sun, Stefan Lee, David Crandall, and Dhruv Batra. 2018. Diverse beam search for improved description of complex scenes. Proceedings of the AAAI Conference on Artificial Intelligence, 32(1). [find](https://scholar.google.com/scholar?q=Ashwin+Vijayakumar%2C+Michael+Cogswell%2C+Ramprasaath+Selvaraju%2C+Qing+Sun%2C+Stefan+Lee%2C+David+Crandall%2C+and+Dhruv+Batra%2E+2018%2E+Diverse+beam+search+for+improved+description+of+complex+scenes%2E+Proceedings+of) 33. OpenAI. 2024b. Gpt-4o mini. [https://openai.com/index/](https://openai.com/index/) Peiyi Wang, Lei Li, Liang Chen, Zefan Cai, Dawei gpt-4o-mini-advancing-cost-efficient-intelligence/. Zhu, Binghuai Lin, Yunbo Cao, Lingpeng Kong, Accessed: 2025-02-01. [↗](https://openai.com/index/) 34. Qi Liu, Tianyu Liu, and Zhifang Sui. 2024. Large language models are not fair evaluators. In Proceedings OpenAI. 2024c. [Gpt-4o system card](https://md.research.elamur.ai/free/en/papers/openai-2024-gpt-4o-system-card/). Preprint, of the 62nd Annual Meeting of the Association for arXiv:2410.21276. Computational Linguistics (Volume 1: Long Papers), pages 9440–9450, Bangkok, Thail… 35. Paul F Christiano, Jan Leike, and Ryan Lowe. 2022. Training language models to follow instructions with Weizhe Yuan, Graham Neubig, and Pengfei Liu. 2021. human feedback. In Advances in Neural Information BARTScore: Evaluating generated text as text gener-Processing Systems, volume 35, pages 27730–2… [find](https://scholar.google.com/scholar?q=Paul+F+Christiano%2C+Jan+Leike%2C+and+Ryan+Lowe%2E+2022%2E+Training+language+models+to+follow+instructions+with+Weizhe+Yuan%2C+Graham+Neubig%2C+and+Pengfei+Liu%2E+2021%2E+human+feedback%2E+In+Advances+in+Neural+Informa) 36. Shiyue Zhang and Mohit Bansal. 2021. Finding a balanced degree of automation for summary evaluation. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 6617–6632, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Shiyue+Zhang+and+Mohit+Bansal%2E+2021%2E+Finding+a+balanced+degree+of+automation+for+summary+evaluation%2E+In+Proceedings+of+the+2021+Conference+on+Empirical+Methods+in+Natural+Language+Processing%2C+pages+66) 37. You split the provided input in small sentences separated by an \#. The split sentences represent subsentences of the original sentences. Example inputs: Anuradha Koirala and 425 young women and girls have been sleeping outdoors because of aftershocks. Pushpa Basnet and 45 children she cares for were… [find](https://scholar.google.com/scholar?q=You+split+the+provided+input+in+small+sentences+separated+by+an+%5C%23%2E+The+split+sentences+represent+subsentences+of+the+original+sentences%2E+Example+inputs%3A+Anuradha+Koirala+and+425+young+women+and+girls) 38. Tianyi Zhang*, Varsha Kishore*, Felix Wu*, Kilian Q. Weinberger, and Yoav Artzi. 2020. Bertscore: Evaluating text generation with bert. In International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Tianyi+Zhang%2A%2C+Varsha+Kishore%2A%2C+Felix+Wu%2A%2C+Kilian+Q%2E+Weinberger%2C+and+Yoav+Artzi%2E+2020%2E+Bertscore%3A+Evaluating+text+generation+with+bert%2E+In+International+Conference+on+Learning+Representations%2E) 39. Tianyi Zhang, Faisal Ladhak, Esin Durmus, Percy Liang, Kathleen McKeown, and Tatsunori B. Hashimoto. 2024. [Benchmarking large language models for news summarization](https://md.research.elamur.ai/free/en/papers/zhang-2024-benchmarking-large-language-models-news/). Transactions of the Association for Computational Linguistics, 12:39–57. *Updated 1 November 2026*