---
title: "Yesterday's News — paper by Verhoeven et al. (2026)"
description: "The authors present misinfo-general, a benchmark that estimates robustness to these shifts. 24 references."
authors: ["Ivo Verhoeven", "Pushkar Mishra", "Ekaterina Shutova"]
year: 2026
venue: "Computational Linguistics"
source: "https://aclanthology.org/2026.cl-2.6/"
doi: "10.1162/coli.a.585"
licence: "CC BY-NC-ND 4.0 (Computational Linguistics, MIT Press)"
licence_url: "https://creativecommons.org/licenses/by-nc-nd/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/verhoeven-2026-yesterday-s-news-benchmarking-multi/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/verhoeven-2026-yesterday-s-news-benchmarking-multi/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/48/) / Yesterday's News: Benchmarking Multi-Dimensional Out-of-Distribution Generalization of Misinformation Detection Models

# Yesterday's News: Benchmarking Multi-Dimensional Out-of-Distribution Generalization of Misinformation Detection Models

**Authors:** Ivo Verhoeven, [Pushkar Mishra](https://md.research.elamur.ai/free/en/authors/pushkar-mishra/) and [Ekaterina Shutova](https://md.research.elamur.ai/free/en/authors/ekaterina-shutova/) · **Year:** 2026 · **Venue:** [Computational Linguistics](https://md.research.elamur.ai/free/en/venues/cl/) ([Computational Linguistics 2026](https://md.research.elamur.ai/free/en/venues/cl/2026/)) · **Publisher:** [The MIT Press](https://md.research.elamur.ai/free/en/publishers/the-mit-press/) · **Licence:** [CC BY-NC-ND 4.0 (Computational Linguistics, MIT Press)](https://creativecommons.org/licenses/by-nc-nd/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2026.cl-2.6/) · [PDF](https://aclanthology.org/2026.cl-2.6.pdf) · [DOI 10.1162/coli.a.585](https://doi.org/10.1162/coli.a.585) · [OpenAlex](https://openalex.org/W4404306290)

**Citations:** 0 citations (OpenAlex, 2026-10-14)

## Summary
Misinformation detectors are usually trained on dated, narrowly focused datasets. In deployment, however, they meet content whose events, publishers and topics have shifted, and current models handle such out-of-distribution (OoD) data poorly. The authors present misinfo-general, a benchmark that estimates robustness to these shifts. It applies distant, publisher-level labels to articles from the NELA corpora, which were originally built for publisher reliability estimation, and adds rich metadata such as MBFC political-bias and factuality ratings. To limit label noise, they apply extensive pre-processing and run post-hoc checks for modelling shortcuts. The benchmark defines six axes of generalization, covering shifts across time, events and topics, as well as differences in publisher, in political leaning, and in the kind of misinformation involved. A common baseline classifier is evaluated on each axis. Transfer to unseen classes of publishers proves hardest. Year-to-year drift within the same publishers matters less than expected. A metadata-driven regression analysis reveals flaws that aggregate scores hide: infrequent publishers are classified worse, and political leanings are treated unequally. The public dataset and code enable more realistic evaluation of article-level detectors.

## Key points
- **Six OoD axes:** time, specific events, topics, publisher style, political bias and misinformation type. Each axis is operationalized as its own train/test split.
- **Labels:** distant labels at the publisher level from NELA plus MBFC. The categories include Reliable, Conspiracy-PseudoScience, Questionable Source and Satire.
- **Publisher frequency:** in the publisher-level logistic model, log10 of the training count has β = 0.65 (odds ×1.91). A publisher with ten times more training articles nearly doubles its odds of correct classification.
- **Topic and factuality effects:**
  - Topic distance to the training set has only a weak positive effect (β = 0.15, odds ×1.16).
  - Publishers with middling factuality scores tend to fall nearest the decision boundary.
- **Availability:** the dataset and code are released at github.com/ioverho/misinfo-general. The paper is published in Computational Linguistics (2026) under CC BY-NC-ND 4.0.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [University of Amsterdam](https://md.research.elamur.ai/free/en/universities/university-of-amsterdam/) — University, Amsterdam, Netherlands · 144 works · 173,906 citations
- [Meta (United Kingdom)](https://md.research.elamur.ai/free/en/companies/meta-united-kingdom/) — Company, London, United Kingdom · 46 works · 18,033 citations

📚 Shelves (5): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🤝 Social Sciences](https://md.research.elamur.ai/free/en/shelves/social-sciences/) · 1,772 works · [🗄️ Information Systems](https://md.research.elamur.ai/free/en/shelves/computer-science/information-systems/) · 1,437 works · [🤝 Sociology and Political Science](https://md.research.elamur.ai/free/en/shelves/social-sciences/sociology-and-political-science/) · 577 works

## Cite

Cite as: Ivo Verhoeven, Pushkar Mishra and Ekaterina Shutova. 2026. Yesterday's News: Benchmarking Multi-Dimensional Out-of-Distribution Generalization of Misinformation Detection Models. Computational Linguistics.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@article{verhoeven2026yesterday,
  title = {{Yesterday's News: Benchmarking Multi-Dimensional Out-of-Distribution Generalization of Misinformation Detection Models}},
  author = {Ivo Verhoeven, Pushkar Mishra and Ekaterina Shutova},
  journal = {Computational Linguistics},
  year = {2026},
  doi = {10.1162/coli.a.585},
  url = {https://doi.org/10.1162/coli.a.585}
}
```

- [← previous YallaMorph: A Benchmark for Evaluating Arabic Morphological Generation in Large Language Models](https://md.research.elamur.ai/free/en/papers/reda-2026-yallamorph-benchmark-evaluating-arabic-morphological/)
- [next → Yeswa-Stories: A Three-Way Parallel Dataset of Female African Figures in Low-Web Data Languages](https://md.research.elamur.ai/free/en/papers/mamo-2026-yeswa-stories-three-way-parallel/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/48/)

## References

24 references, 9 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Agresti, Alan and Brent A. Coull. 1998. Approximate is better than “exact” for interval estimation of binomial proportions. The American Statistician, 52(2):119–126. https://doi.org/10.1080/00031305.1998 .10480550 Ahmed, Hadeer, Issa Traore, Sherif Saad, and Mohammad Mamun. 2024. Effect of text aug… — [doi:10.1080/00031305.1998](https://doi.org/10.1080/00031305.1998) [↗](https://doi.org/10.1080/00031305.1998)
2. Alba-Juez, Laura and J Lachlan Mackenzie. 2019. Emotion, lies, and “bullshit” in journalistic discourse: The case of fake news. Iberica, (39). [find](https://scholar.google.com/scholar?q=Alba%2DJuez%2C+Laura+and+J+Lachlan+Mackenzie%2E+2019%2E+Emotion%2C+lies%2C+and+%E2%80%9Cbullshit%E2%80%9D+in+journalistic+discourse%3A+The+case+of+fake+news%2E+Iberica%2C+%2839%29%2E)
3. Allcott, Hunt and Matthew Gentzkow. 2017. [Social media and fake news in the 2016 election](https://md.research.elamur.ai/free/en/papers/baly-2020-we-can-detect-your-bias/). Journal of Economic Perspectives, 31(2):211–236. https://doi.org/10 .1257/jep.31.2.211 Altay, Sacha, Manon Berriche, Hendrik Heuer, Johan Farkas, and Steven Rathje. 2023. A survey of expert views on misinforma…
4. Barrón-Cedeño, Alberto, Israa Jaradat, Giovanni Da San Martino, and Preslav Nakov. 2019. Proppy: Organizing the news based on their propagandistic content. Information Processing & Management, 56(5):1849–1864. https://doi.org/10.1016/j.ipm.2019.03.005 Bianchi, John, Manuel Pratelli, Marinella Pet… — [doi:10.1016/j.ipm.2019.03.005](https://doi.org/10.1016/j.ipm.2019.03.005) [↗](https://doi.org/10.1016/j.ipm.2019.03.005)
5. Bojic, Ljubisa, Nikola Prodanovic, and Agariadne Dwinggo Samala. 2024. Maintaining journalistic integrity in the digital age: A comprehensive NLP framework for evaluating online news content. International Journal for Quality Research, 20(1):201–218. https://doi.org/10.24874/IJQR20.01-13 Bostan, La… — [doi:10.24874/ijqr20.01-13](https://doi.org/10.24874/ijqr20.01-13) [↗](https://doi.org/10.24874/IJQR20.01-13)
6. Bozarth, Lia and Ceren Budak. 2020. Toward a better performance evaluation framework for fake news classification. Proceedings of the International AAAI Conference on Web and Social Media, 14:60–71. https://doi.org/10.1609/icwsm.v14i1.7279 Bradd, Sam. 2024. Infodemic. https://www.who.int/health-to… [↗](https://doi.org/10.1609/icwsm.v14i1.7279)
7. Dai, Enyan, Yiwei Sun, and Suhang Wang. 2020. [Ginger cannot cure cancer: Battling fake health news with a comprehensive data repository](https://md.research.elamur.ai/free/en/papers/deepseek-ai-2025-deepseek-r1-incentivizing-reasoning-capability/). Proceedings of the International AAAI Conference on Web and Social Media, 14:853–862. https://doi.org/10.1609/icwsm.v14i1.7350 DeepSeek-AI. 2025. DeepSeek-R1: Inc… [↗](https://doi.org/10.1609/icwsm.v14i1.7350)
8. Gelfert, Axel. 2018. Fake news: A definition. Informal Logic, 38(1):84–117. https://doi.org/10.22329/il.v38i1.5068 Gemini-2.5-team. 2025. [Gemini 2.5: Pushing the frontier with advanced reasoning, multimodality, long context, and next generation agentic capabilities](https://md.research.elamur.ai/free/en/papers/comanici-2025-gemini-2-5-pushing-frontier/). arXiv preprint arXiv:2507.06261.… [↗](https://doi.org/10.22329/il.v38i1.5068)
9. Holtzman, Ari, Peter West, Vered Shwartz, Yejin Choi, and Luke Zettlemoyer. 2021. [Surface form competition: Why the highest probability answer isn’t always right](https://md.research.elamur.ai/free/en/papers/holtzman-2021-surface-form-competition-why-highest/). In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 7038–7051. https://doi.org/10.18653/v1 /…
10. Jenkins, Thomas, Sameerah Talafha, and Adam Goodwin. 2023. [An ordered sample consensus (ORSAC) method for data cleaning inspired by RANSAC: Identifying probable mislabeled data](https://md.research.elamur.ai/free/en/papers/kiesel-2019-semeval-2019-task-4-hyperpartisan/). https://doi.org/10.36227/techrxiv.23511453.v1 Jeronimo, Caio Libanio Melo, Leandro Balby Marinho, Claudio E. C. Campelo,… [↗](https://doi.org/10.36227/techrxiv.23511453.v1)
11. Kochkina, Elena, Tamanna Hossain, Robert L. Logan, Miguel Arana-Catania, Rob Procter, Arkaitz Zubiaga, Sameer Singh, Yulan He, and Maria Liakata. 2023. [Evaluating the generalisability of neural rumour verification models](https://md.research.elamur.ai/free/en/papers/lampinen-2022-can-language-models-learn-explanations/). Information Processing & Management, 60(1):103116. https://doi.org/10.1016 /j.…
12. Liu, Zhiwei, Tianlin Zhang, Kailai Yang, Paul Thompson, Zeping Yu, and Sophia Ananiadou. 2024. Emotion detection for misinformation: A review. Information Fusion, 107:102300. https://doi.org/10.1016/j.inffus.2024.102300 Llama Team, AI @ Meta. 2024. [The Llama 3 herd of models](https://md.research.elamur.ai/free/en/papers/grattafiori-2024-llama-3-herd-models/). arXiv preprint arXiv:2… [↗](https://doi.org/10.1016/j.inffus.2024.102300)
13. Lucas, Jason, Adaku Uchendu, Michiharu Yamashita, Jooyoung Lee, Shaurya Rohatgi, and Dongwon Lee. 2023. Fighting fire with fire: The dual role of LLMs in crafting and detecting elusive disinformation. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 14… [find](https://scholar.google.com/scholar?q=Lucas%2C+Jason%2C+Adaku+Uchendu%2C+Michiharu+Yamashita%2C+Jooyoung+Lee%2C+Shaurya+Rohatgi%2C+and+Dongwon+Lee%2E+2023%2E+Fighting+fire+with+fire%3A+The+dual+role+of+LLMs+in+crafting+and+detecting+elusive+disinformation%2E)
14. Martel, Cameron, Gordon Pennycook, and David G. Rand. 2020. Reliance on emotion promotes belief in fake news. Cognitive Research: Principles and Implications, 5(1):47. https://doi.org/10.1186/s41235-020 -00252-3, PubMed: 33026546 Mihalcea, Rada and Carlo Strapparava. 2009. The lie detector: Explorat… [↗](https://doi.org/10.1186/s41235-020)
15. Nakov, Preslav, Jisun An, Haewoon Kwak, Muhammad Arslan Manzoor, Zain Muhammad Mujahid, and Husrev Taha Sencar. 2024. A survey on predicting the factuality and the bias of news media. In Findings of the Association for Computational Linguistics: ACL 2024, pages 15947–15962. https://doi .org/10.18653… [find](https://scholar.google.com/scholar?q=Nakov%2C+Preslav%2C+Jisun+An%2C+Haewoon+Kwak%2C+Muhammad+Arslan+Manzoor%2C+Zain+Muhammad+Mujahid%2C+and+Husrev+Taha+Sencar%2E+2024%2E+A+survey+on+predicting+the+factuality+and+the+bias+of+news+media%2E+In+Findings+of+t)
16. Ozcelik, Oguzhan, Arda Sarp Yenicesu, Onur Yildirim, Dilruba Sultan Haliloglu, Erdem Ege Eroglu, and Fazli Can. 2023. [find](https://scholar.google.com/scholar?q=Ozcelik%2C+Oguzhan%2C+Arda+Sarp+Yenicesu%2C+Onur+Yildirim%2C+Dilruba+Sultan+Haliloglu%2C+Erdem+Ege+Eroglu%2C+and+Fazli+Can%2E+2023%2E)
17. Cross-lingual transfer learning for misinformation detection: investigating performance across multiple languages. In Proceedings of the 4th Conference on Language, Data and Knowledge, pages 549–558. Pathak, Archita and Rohini Srihari. 2019. BREAKING! Presenting fake news corpus for automated fact c… [find](https://scholar.google.com/scholar?q=Cross%2Dlingual+transfer+learning+for+misinformation+detection%3A+investigating+performance+across+multiple+languages%2E+In+Proceedings+of+the+4th+Conference+on+Language%2C+Data+and+Knowledge%2C+pages+549%E2%80%93558%2E+)
18. Phillips, Samantha Cavanaugh, Sze Yuh Nina Wang, Kathleen M. Carley, David Gertler Rand, and Gordon Pennycook. 2024. Emotional language reduces belief in false claims. Judgment and Decision Making. 20:e43. https://doi.org/10.1017/jdm.2025.10019 Pleiss, Geoff, Tianyi Zhang, Kilian Q Weinberger, and… — [doi:10.1017/jdm.2025.10019](https://doi.org/10.1017/jdm.2025.10019) [↗](https://doi.org/10.1017/jdm.2025.10019)
19. Plutchik, Robert. 1980, [Chapter 1 - A general psychoevolutionary theory of emotion](https://md.research.elamur.ai/free/en/papers/potthast-2018-stylometric-inquiry-into-hyperpartisan-fake/). In Robert Plutchik and Henry Kellerman, editors, Theories of Emotion. Academic Press, pages 3–33. https://doi.org/10.1016/B978-0-12-558701-3 .50007-7 Poldvere, Nele, Zia Uddin, and Aleena Thomas. 2023. The PolitiFac… [↗](https://doi.org/10.1016/B978-0-12-558701-3)
20. Shahi, Gautam Kishore and Durgesh Nandini. 2020. [FakeCovid – A multilingual cross-domain fact check news dataset for COVID-19](https://md.research.elamur.ai/free/en/papers/shokri-2024-subjectivity-detection-english-news-large/). arXiv preprint arXiv:2006.11343. https://doi.org/10.48550/arXiv.2006.11343 Shokri, Mohammad, Vivek Sharma, Elena Filatova, Shweta Jain, and Sarah Levitan. 2024. Subjectivit… [↗](https://doi.org/10.48550/arXiv.2006.11343)
21. Swayamdipta, Swabha, Roy Schwartz, Nicholas Lourie, Yizhong Wang, Hannaneh Hajishirzi, Noah A. Smith, and Yejin Choi. 2020. Dataset cartography: Mapping and diagnosing datasets with training dynamics. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP),… [find](https://scholar.google.com/scholar?q=Swayamdipta%2C+Swabha%2C+Roy+Schwartz%2C+Nicholas+Lourie%2C+Yizhong+Wang%2C+Hannaneh+Hajishirzi%2C+Noah+A%2E+Smith%2C+and+Yejin+Choi%2E+2020%2E+Dataset+cartography%3A+Mapping+and+diagnosing+datasets+with+training+dynamics%2E)
22. Tobi, Abraham. 2024. Towards an epistemic compass for online content moderation. Philosophy & Technology, 37(3):109. https://doi.org/10.1007/s13347 -024-00791-3 Verhoeven, Ivo, Pushkar Mishra, Rahel Beloch, Helen Yannakoudakis, and Ekaterina Shutova. 2024. A (more) realistic evaluation setup for gen… [↗](https://doi.org/10.1007/s13347)
23. ACM SIGKDD Explorations Newsletter, 21(2):80–90. https://doi.org/10.1145/3373464.3373475 Xiao, Madelyne and Jonathan Mayer. 2024. The challenges of machine learning for trust and safety: A case study on misinformation detection. https://doi.org/10.48550/arXiv.2308 .12215 Yang, Kai Cheng and Filippo… [↗](https://doi.org/10.1145/3373464.3373475)
24. Zubiaga, Arkaitz, Maria Liakata, and Rob Procter. 2017. Exploiting context for rumour detection in social media. In Social Informatics, pages 109–123. https://doi.org/10.1007/978-3-319 -67217-5 8 — [doi:10.1007/978-3-319](https://doi.org/10.1007/978-3-319) [↗](https://doi.org/10.1007/978-3-319)

*Updated 1 November 2026*