---
title: "HistoryBankQA — Mandal 2026"
description: "Large language models are increasingly expected to reason about time, yet existing benchmarks for this skill tend to be small, English-only, synthetic…"
authors: ["Biswadip Mandal", "Anant Khandelwal", "Manish Gupta"]
year: 2026
venue: "Proceedings of the 15th Joint Conference on Lexical and Computational Semantics (*SEM 2026)"
source: "https://aclanthology.org/2026.starsem-conference.33/"
doi: "10.18653/v1/2026.starsem-conference.33"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/papers/mandal-2026-historybankqa-multilingual-temporal-question-answering/"
---
[Elamur Research](https://md.research.elamur.ai/) · [this page in HTML](https://research.elamur.ai/free/papers/mandal-2026-historybankqa-multilingual-temporal-question-answering/)

[home](https://md.research.elamur.ai/) / [papers](https://md.research.elamur.ai/free/papers/) / [2026](https://md.research.elamur.ai/free/papers/2026/page/3/) / HistoryBankQA: Multilingual Temporal Question Answering on Historical Events

# HistoryBankQA: Multilingual Temporal Question Answering on Historical Events

**Authors:** Biswadip Mandal, Anant Khandelwal, Manish Gupta · **Year:** 2026 · **Venue:** Proceedings of the 15th Joint Conference on Lexical and Computational Semantics (\*SEM 2026) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2026.starsem-conference.33/) · [PDF](https://aclanthology.org/2026.starsem-conference.33.pdf) · [DOI 10.18653/v1/2026.starsem-conference.33](https://doi.org/10.18653/v1/2026.starsem-conference.33) · [OpenAlex](https://openalex.org/W4415316406)

**Citations:** 0 citations (openalex-snapshot · 2026-10-14)

## Summary
Large language models are increasingly expected to reason about time, yet existing benchmarks for this skill tend to be small, English-only, synthetic, or focused on recent happenings. This paper addresses that gap for historical events. The authors automatically build HistoryBank, a database of more than 10 million dated events. These events are drawn from Wikipedia's "On This Day" date pages and from article infoboxes across ten language editions: six European ones (Spanish, Russian, Portuguese, Italian, German and French), two South Asian ones (Hindi and Bengali), plus Indonesian and English. From this database they generate HistoryBankQA, a benchmark of six question families. These cover date recall, chronological sorting, interval arithmetic, relations between event spans, counting events within a period, and finding the previous or next edition of a recurring event. The authors test LLaMA-3-8B, Mistral-7B, Gemma-2-9B, Qwen3-8B and GPT-4o, mainly zero-shot. GPT-4o comes out strongest, Gemma-2 leads the smaller models, and scores vary considerably by task and language. The resource helps researchers judge whether models truly reason about time or rely on recalled facts and English-specific cues. Code and data are publicly released.

## Key points
- **Scale of HistoryBank:** 10M+ events in 10 languages. English dominates with about 8.2M events from roughly 4.26M articles, and German is next with about 1.52M events.
- **Six HistoryBankQA tasks:** FactQA, SequenceQA, DurationQA, RelationQA (with subtypes such as Overlap, Inclusion, Gap Between, Order Span Start/End), CountQA and RecurrenceQA.
- **Event quality:** In a quality check, 84% of extracted events were rated interesting. The rest were mostly low-salience infobox entries such as census or administrative records, not generation errors.
- **GPT-4o error analysis (English test set):** Three recurring failure modes appear:
  - missing or wrong temporal knowledge;
  - correct knowledge but faulty reasoning, such as mishandled century boundaries or inclusive/exclusive ranges;
  - final answers that contradict otherwise correct reasoning.
- **Acknowledged limitations:**
  - Western-centric Wikipedia coverage;
  - LLM-generated questions;
  - a narrow evaluation setup;
  - no human evaluation of model answers.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

Fields: [Linguistics & languages](https://md.research.elamur.ai/free/shelves/linguistics/)

## Cite

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{mandal2026historybankqa,
  title = {{HistoryBankQA: Multilingual Temporal Question Answering on Historical Events}},
  author = {Mandal, Biswadip and Khandelwal, Anant and Gupta, Manish},
  booktitle = {Proceedings of the 15th Joint Conference on Lexical and Computational Semantics (*SEM 2026)},
  year = {2026},
  doi = {10.18653/v1/2026.starsem-conference.33},
  url = {https://doi.org/10.18653/v1/2026.starsem-conference.33}
}
```

- [← previous HistoriQA-ThirdRepublic: Multi-Hop Question Answering Corpus for Historical Research, Parliamentary Debates from the French Third Republic (1870-1940)](https://md.research.elamur.ai/free/papers/pellet-2026-historiqa-thirdrepublic-multi-hop-question/)
- [next → How far back in antiquity can we trace the oral tradition underlying Tiberian Hebrew?](https://md.research.elamur.ai/free/papers/hbo-2026-joosten-far-back-antiquity-trace-oral/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/papers/2026/page/3/)

## References

29 references, 2 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Abdelrahman Abdallah, Mohammed Ali, Muhammad Abdul-Mageed, and Adam Jatowt. 2026. Tempo: A realistic multi-domain benchmark for temporal reasoning-intensive retrieval. arXiv preprint arXiv:2601.09523. — [arXiv:2601.09523](https://arxiv.org/abs/2601.09523) [find](https://scholar.google.com/scholar?q=Abdelrahman+Abdallah%2C+Mohammed+Ali%2C+Muhammad+Abdul%2DMageed%2C+and+Adam+Jatowt%2E+2026%2E+Tempo%3A+A+realistic+multi%2Ddomain+benchmark+for+temporal+reasoning%2Dintensive+retrieval%2E+arXiv+preprint+arXiv%3A2601%2E09523%2E)
2. James F Allen. 1983. Maintaining knowledge about temporal intervals. Communications of the ACM, 26(11):832–843. [find](https://scholar.google.com/scholar?q=James+F+Allen%2E+1983%2E+Maintaining+knowledge+about+temporal+intervals%2E+Communications+of+the+ACM%2C+26%2811%29%3A832%E2%80%93843%2E)
3. Sören Auer, Christian Bizer, Georgi Kobilarov, Jens Lehmann, Richard Cyganiak, and Zachary Ives. 2007. Dbpedia: A nucleus for a web of open data. In international semantic web conference, pages 722– 735. Springer. [find](https://scholar.google.com/scholar?q=S%C3%B6ren+Auer%2C+Christian+Bizer%2C+Georgi+Kobilarov%2C+Jens+Lehmann%2C+Richard+Cyganiak%2C+and+Zachary+Ives%2E+2007%2E+Dbpedia%3A+A+nucleus+for+a+web+of+open+data%2E+In+international+semantic+web+conference%2C+pages+722%E2%80%93+7)
4. Claire Barale, Leslie Barrett, Vikram Sunil Bajaj, and Michael Rovatsos. 2025. Lextime: A benchmark for temporal ordering of legal events. arXiv preprint arXiv:2506.04041. [find](https://scholar.google.com/scholar?q=Claire+Barale%2C+Leslie+Barrett%2C+Vikram+Sunil+Bajaj%2C+and+Michael+Rovatsos%2E+2025%2E+Lextime%3A+A+benchmark+for+temporal+ordering+of+legal+events%2E+arXiv+preprint+arXiv%3A2506%2E04041%2E)
5. Nadav Borenstein, Natalia da Silva Perez, and Isabelle Augenstein. 2023. Multilingual event extraction from historical newspaper adverts. arXiv preprint arXiv:2305.10928. [find](https://scholar.google.com/scholar?q=Nadav+Borenstein%2C+Natalia+da+Silva+Perez%2C+and+Isabelle+Augenstein%2E+2023%2E+Multilingual+event+extraction+from+historical+newspaper+adverts%2E+arXiv+preprint+arXiv%3A2305%2E10928%2E)
6. Aleksandar Botev, Soham De, Samuel L Smith, Anushan Fernando, George-Cristian Muraru, Ruba Haroun, Leonard Berrada, Razvan Pascanu, Pier Giuseppe Sessa, Robert Dadashi, and 1 others. 2024. Recurrentgemma: Moving past transformers for efficient open language models. arXiv preprint arXiv:2404.07839. — [arXiv:2404.07839](https://arxiv.org/abs/2404.07839) [find](https://scholar.google.com/scholar?q=Aleksandar+Botev%2C+Soham+De%2C+Samuel+L+Smith%2C+Anushan+Fernando%2C+George%2DCristian+Muraru%2C+Ruba+Haroun%2C+Leonard+Berrada%2C+Razvan+Pascanu%2C+Pier+Giuseppe+Sessa%2C+Robert+Dadashi%2C+and+1+others%2E+2024%2E+Recurrentge)
7. Chia-Hui Chang, Yuan-Hao Lin, and Hsiu-Min Chuang. 2020. Eventgo! exploring event dynamics from social-media posts. In 2020 International Computer Symposium (ICS), pages 548–552. IEEE. [find](https://scholar.google.com/scholar?q=Chia%2DHui+Chang%2C+Yuan%2DHao+Lin%2C+and+Hsiu%2DMin+Chuang%2E+2020%2E+Eventgo%21+exploring+event+dynamics+from+social%2Dmedia+posts%2E+In+2020+International+Computer+Symposium+%28ICS%29%2C+pages+548%E2%80%93552%2E+IEEE%2E)
8. Hyung Won Chung, Le Hou, Shayne Longpre, Barret Zoph, Yi Tay, William Fedus, Yunxuan Li, Xuezhi Wang, Mostafa Dehghani, Siddhartha Brahma, and 1 others. 2024. Scaling instruction-finetuned language models. Journal of Machine Learning Research, 25(70):1–53. [find](https://scholar.google.com/scholar?q=Hyung+Won+Chung%2C+Le+Hou%2C+Shayne+Longpre%2C+Barret+Zoph%2C+Yi+Tay%2C+William+Fedus%2C+Yunxuan+Li%2C+Xuezhi+Wang%2C+Mostafa+Dehghani%2C+Siddhartha+Brahma%2C+and+1+others%2E+2024%2E+Scaling+instruction%2Dfinetuned+language+mo)
9. Simon Gottschalk and Elena Demidova. 2018. Eventkg: A multilingual event-centric temporal knowledge graph. In European semantic web conference, pages 272–287. Springer. [find](https://scholar.google.com/scholar?q=Simon+Gottschalk+and+Elena+Demidova%2E+2018%2E+Eventkg%3A+A+multilingual+event%2Dcentric+temporal+knowledge+graph%2E+In+European+semantic+web+conference%2C+pages+272%E2%80%93287%2E+Springer%2E)
10. Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, and et al. 2024. The llama 3 herd of models. arXiv preprint arXiv:2407.21783. [find](https://scholar.google.com/scholar?q=Aaron+Grattafiori%2C+Abhimanyu+Dubey%2C+Abhinav+Jauhri%2C+and+et+al%2E+2024%2E+The+llama+3+herd+of+models%2E+arXiv+preprint+arXiv%3A2407%2E21783%2E)
11. Raphael Gruber, Abdelrahman Abdallah, Michael Färber, and Adam Jatowt. 2025. Complextempqa: A 100m dataset for complex temporal question answering. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pages 9111–9123. [find](https://scholar.google.com/scholar?q=Raphael+Gruber%2C+Abdelrahman+Abdallah%2C+Michael+F%C3%A4rber%2C+and+Adam+Jatowt%2E+2025%2E+Complextempqa%3A+A+100m+dataset+for+complex+temporal+question+answering%2E+In+Proceedings+of+the+2025+Conference+on+Empirical+M)
12. Johannes Hoffart, Fabian M Suchanek, Klaus Berberich, and Gerhard Weikum. 2013. Yago2: A spatially and temporally enhanced knowledge base from wikipedia. Artificial intelligence, 194:28–61. [find](https://scholar.google.com/scholar?q=Johannes+Hoffart%2C+Fabian+M+Suchanek%2C+Klaus+Berberich%2C+and+Gerhard+Weikum%2E+2013%2E+Yago2%3A+A+spatially+and+temporally+enhanced+knowledge+base+from+wikipedia%2E+Artificial+intelligence%2C+194%3A28%E2%80%9361%2E)
13. Duygu Sezen Islakoglu and Jan-Christoph Kalo. 2025. Chronosense: Exploring temporal understanding in large language models with time intervals of events. arXiv preprint arXiv:2501.03040. [find](https://scholar.google.com/scholar?q=Duygu+Sezen+Islakoglu+and+Jan%2DChristoph+Kalo%2E+2025%2E+Chronosense%3A+Exploring+temporal+understanding+in+large+language+models+with+time+intervals+of+events%2E+arXiv+preprint+arXiv%3A2501%2E03040%2E)
14. Albert Q. Jiang, Alexandre Sablayrolles, Arthur Mensch, and et al. 2023. Mistral 7b. arXiv preprint arXiv:2310.06825. [find](https://scholar.google.com/scholar?q=Albert+Q%2E+Jiang%2C+Alexandre+Sablayrolles%2C+Arthur+Mensch%2C+and+et+al%2E+2023%2E+Mistral+7b%2E+arXiv+preprint+arXiv%3A2310%2E06825%2E)
15. Kalev Leetaru and Philip A Schrodt. 2013. Gdelt: Global data on events, location, and tone, 1979–2012. In ISA annual convention, volume 2, pages 1–49. Citeseer. [find](https://scholar.google.com/scholar?q=Kalev+Leetaru+and+Philip+A+Schrodt%2E+2013%2E+Gdelt%3A+Global+data+on+events%2C+location%2C+and+tone%2C+1979%E2%80%932012%2E+In+ISA+annual+convention%2C+volume+2%2C+pages+1%E2%80%9349%2E+Citeseer%2E)
16. Zhuoyuan Liu and Yilin Luo. 2024. Document-level event extraction with definition-driven icl. arXiv preprint arXiv:2408.05566. — [arXiv:2408.05566](https://arxiv.org/abs/2408.05566) [find](https://scholar.google.com/scholar?q=Zhuoyuan+Liu+and+Yilin+Luo%2E+2024%2E+Document%2Dlevel+event+extraction+with+definition%2Ddriven+icl%2E+arXiv+preprint+arXiv%3A2408%2E05566%2E+%E2%80%94+%5BarXiv%3A2408%2E05566%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2408%2E05566%29)
17. Jiayu Song, Mahmud Akhter, Dana Atzil Slonim, and Maria Liakata. 2024. Temporal reasoning for timeline summarisation in social media. arXiv preprint arXiv:2501.00152. [find](https://scholar.google.com/scholar?q=Jiayu+Song%2C+Mahmud+Akhter%2C+Dana+Atzil+Slonim%2C+and+Maria+Liakata%2E+2024%2E+Temporal+reasoning+for+timeline+summarisation+in+social+media%2E+arXiv+preprint+arXiv%3A2501%2E00152%2E)
18. Qingyu Tan, Hwee Tou Ng, and Lidong Bing. 2023. Towards benchmarking and improving the temporal reasoning capability of large language models. arXiv preprint arXiv:2306.08952. [find](https://scholar.google.com/scholar?q=Qingyu+Tan%2C+Hwee+Tou+Ng%2C+and+Lidong+Bing%2E+2023%2E+Towards+benchmarking+and+improving+the+temporal+reasoning+capability+of+large+language+models%2E+arXiv+preprint+arXiv%3A2306%2E08952%2E)
19. Gemma Team, Aishwarya Kamath, Johan Ferret, Shreya Pathak, Nino Vieillard, Ramona Merhej, Sarah Perrin, Tatiana Matejovicova, Alexandre Ramé, Morgane Rivière, and 1 others. 2025. Gemma 3 technical report. arXiv preprint arXiv:2503.19786. [find](https://scholar.google.com/scholar?q=Gemma+Team%2C+Aishwarya+Kamath%2C+Johan+Ferret%2C+Shreya+Pathak%2C+Nino+Vieillard%2C+Ramona+Merhej%2C+Sarah+Perrin%2C+Tatiana+Matejovicova%2C+Alexandre+Ram%C3%A9%2C+Morgane+Rivi%C3%A8re%2C+and+1+others%2E+2025%2E+Gemma+3+technical+rep)
20. Denny Vrandečić and Markus Krötzsch. 2014. Wikidata: a free collaborative knowledgebase. Communications of the ACM, 57(10):78–85. [find](https://scholar.google.com/scholar?q=Denny+Vrandec%CC%8Cic%CC%81+and+Markus+Kr%C3%B6tzsch%2E+2014%2E+Wikidata%3A+a+free+collaborative+knowledgebase%2E+Communications+of+the+ACM%2C+57%2810%29%3A78%E2%80%9385%2E)
21. Qifan Wang, Bhargav Kanagal, Vijay Garg, and D Sivakumar. 2019. Constructing a comprehensive events database from the web. In Proceedings of the 28th ACM International Conference on Information and Knowledge Management, pages 229–238. [find](https://scholar.google.com/scholar?q=Qifan+Wang%2C+Bhargav+Kanagal%2C+Vijay+Garg%2C+and+D+Sivakumar%2E+2019%2E+Constructing+a+comprehensive+events+database+from+the+web%2E+In+Proceedings+of+the+28th+ACM+International+Conference+on+Information+and+Kn)
22. Yuqing Wang and Yun Zhao. 2024. TRAM: Benchmarking temporal reasoning for large language models. In Findings of the Association for Computational Linguistics: ACL 2024, pages 6389–6415, Bangkok, Thailand. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Yuqing+Wang+and+Yun+Zhao%2E+2024%2E+TRAM%3A+Benchmarking+temporal+reasoning+for+large+language+models%2E+In+Findings+of+the+Association+for+Computational+Linguistics%3A+ACL+2024%2C+pages+6389%E2%80%936415%2C+Bangkok%2C+Thail)
23. Siheng Xiong, Ali Payani, Ramana Kompella, and Faramarz Fekri. 2024. Large language models can learn temporal reasoning. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 10452– 10470, Bangkok, Thailand. Association for Computat… [find](https://scholar.google.com/scholar?q=Siheng+Xiong%2C+Ali+Payani%2C+Ramana+Kompella%2C+and+Faramarz+Fekri%2E+2024%2E+Large+language+models+can+learn+temporal+reasoning%2E+In+Proceedings+of+the+62nd+Annual+Meeting+of+the+Association+for+Computational+)
24. An Yang, Anfeng Li, Baosong Yang, and et al. 2025. Qwen3 technical report. arXiv preprint arXiv:2505.09388. [find](https://scholar.google.com/scholar?q=An+Yang%2C+Anfeng+Li%2C+Baosong+Yang%2C+and+et+al%2E+2025%2E+Qwen3+technical+report%2E+arXiv+preprint+arXiv%3A2505%2E09388%2E) [our card](https://md.research.elamur.ai/free/papers/yang-2025-qwen3-technical-report/)
25. Ben Zhou, Daniel Khashabi, Qiang Ning, and Dan Roth. 2019. “going on a vacation” takes longer than “going for a walk”: A study of temporal commonsense understanding. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference o… [find](https://scholar.google.com/scholar?q=Ben+Zhou%2C+Daniel+Khashabi%2C+Qiang+Ning%2C+and+Dan+Roth%2E+2019%2E+%E2%80%9Cgoing+on+a+vacation%E2%80%9D+takes+longer+than+%E2%80%9Cgoing+for+a+walk%E2%80%9D%3A+A+study+of+temporal+commonsense+understanding%2E+In+Proceedings+of+the+2019+Confere)
26. Overview of Appendices • Appendix A: Related Work. [find](https://scholar.google.com/scholar?q=Overview+of+Appendices+%E2%80%A2+Appendix+A%3A+Related+Work%2E)
27. OpenAI, Aaron Hurst, Adam Lerer, and et al. 2024. Gpt-4o system card. arXiv preprint arXiv:2410.21276. [find](https://scholar.google.com/scholar?q=OpenAI%2C+Aaron+Hurst%2C+Adam+Lerer%2C+and+et+al%2E+2024%2E+Gpt%2D4o+system+card%2E+arXiv+preprint+arXiv%3A2410%2E21276%2E) [our card](https://md.research.elamur.ai/free/papers/openai-2024-gpt-4o-system-card/)
28. Gemma Team: Morgane Riviere, Shreya Pathak, Pier Giuseppe Sessa, and et al. 2024. Gemma 2: Improving open language models at a practical size. arXiv preprint arXiv:2408.00118. [find](https://scholar.google.com/scholar?q=Gemma+Team%3A+Morgane+Riviere%2C+Shreya+Pathak%2C+Pier+Giuseppe+Sessa%2C+and+et+al%2E+2024%2E+Gemma+2%3A+Improving+open+language+models+at+a+practical+size%2E+arXiv+preprint+arXiv%3A2408%2E00118%2E)
29. Appendix B: Detailed Dataset Analysis. • Appendix C: Prompt for Event Extraction. • Appendix D: Prompt for FactQA question generation. [find](https://scholar.google.com/scholar?q=Appendix+B%3A+Detailed+Dataset+Analysis%2E+%E2%80%A2+Appendix+C%3A+Prompt+for+Event+Extraction%2E+%E2%80%A2+Appendix+D%3A+Prompt+for+FactQA+question+generation%2E)