---
title: "An Overview Of Temporal Commonsense Reasoning and"
description: "This survey covers temporal commonsense (TCS) reasoning: a model's ability to infer unstated… 90 references, cited by 11 papers here."
authors: ["Georg Wenzel", "Adam Jatowt"]
year: 2023
venue: "arXiv:2308.00002"
source: "https://arxiv.org/abs/2308.00002"
doi: "10.48550/arxiv.2308.00002"
arxiv: "2308.00002"
licence: "CC BY 4.0 (arXiv)"
licence_url: "http://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/wenzel-2023-overview-temporal-commonsense-reasoning-acquisition/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/wenzel-2023-overview-temporal-commonsense-reasoning-acquisition/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2023](https://md.research.elamur.ai/free/en/papers/2023/page/14/) / An Overview Of Temporal Commonsense Reasoning and Acquisition

# An Overview Of Temporal Commonsense Reasoning and Acquisition

**Authors:** Georg Wenzel and [Adam Jatowt](https://md.research.elamur.ai/free/en/authors/adam-jatowt/) · **Year:** 2023 · **Venue:** arXiv:2308.00002 · **Licence:** [CC BY 4.0 (arXiv)](http://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://arxiv.org/abs/2308.00002) · [PDF](https://arxiv.org/pdf/2308.00002) · [DOI 10.48550/arxiv.2308.00002](https://doi.org/10.48550/arxiv.2308.00002) · [arXiv 2308.00002](https://arxiv.org/abs/2308.00002) · [OpenAlex](https://openalex.org/W4385507432)

**Citations:** 3 citations (OpenAlex, 2026-10-14) · [11](https://md.research.elamur.ai/free/en/papers/wenzel-2023-overview-temporal-commonsense-reasoning-acquisition/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
This survey covers temporal commonsense (TCS) reasoning: a model's ability to infer unstated temporal properties of events, such as their typical order, duration, or time of occurrence. For example, a broken leg usually comes before a hospital visit. The authors place TCS between commonsense and temporal reasoning. They exclude temporal factual knowledge and commonsense causality from scope. They gathered transformer-era literature from 2018 to 2023 and reviewed TCS benchmark datasets, evaluation metrics, and the ways researchers have tried to strengthen language models on these tasks.

Their main finding is that transformers beat earlier methods by a wide margin, yet they remain far below human performance when stricter metrics are used. Models are vulnerable to mispriming, reporting bias, small input changes, and overlooked negations. Much of their apparent success seems to come from pattern matching rather than real reasoning.

The survey calls for carefully built datasets and robust metrics so that results are not overstated. It also points to neuro-symbolic augmentation as a promising direction. Progress here would benefit downstream tasks including temporal NLI, timeline summarization, and temporal question answering.

## Key points
- **Literature search:** The authors searched Google Scholar, Semantic Scholar, and dblp for "temporal commonsense", limited to 2018–2023. They examined all 21 dblp hits and the top 50 results from each of the other two sources. They kept only English papers that mention TCS in the abstract.
- **Related models and resources:** Time-aware models such as TempoBERT and BiTimeBERT temporalize embeddings using document creation time or explicit time expressions. Knowledge graphs used in this area include ConceptNet, ATOMIC 2020 (with the COMET model), and ASER.
- **Model improvements:** Most work focuses on engineering the inputs and outputs around the transformer, not changing its architecture. Approaches include:
  - adding TCS training data from crowdsourcing, weakly supervised web extraction, or knowledge graphs;
  - ensembling several models;
  - adding symbolic logic, for example probabilistic soft logic.
- **Explicit vs. implicit knowledge:** Injecting explicit symbolic knowledge about duration, frequency, and distance often appears to work better than teaching it implicitly through auxiliary training objectives. Pre-training choices also matter, as the accuracy gap between BERT and RoBERTa shows.
- **GPT-4:** Early results for GPT-4 are strong, but its TCS ability may still be limited. Its size and closed weights make local use impractical, so research on smaller, portable models remains relevant.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Universität Innsbruck](https://md.research.elamur.ai/free/en/universities/universitat-innsbruck/) — University, Innsbruck, Austria · 23 works · 1,475 citations

📚 Shelves (5): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 19,503 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 18,805 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 17,959 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 17,789 works · [👁️ Computer Vision and Pattern Recognition](https://md.research.elamur.ai/free/en/shelves/computer-science/computer-vision-and-pattern-recognition/) · 3,621 works

🧭 Topics (1): [🧮 Reasoning](https://md.research.elamur.ai/free/en/topics/reasoning/) · 1,948 works

## Cite

Cite as: Georg Wenzel and Adam Jatowt. 2023. An Overview Of Temporal Commonsense Reasoning and Acquisition. arXiv:2308.00002.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@misc{wenzel2023overview,
  title = {{An Overview Of Temporal Commonsense Reasoning and Acquisition}},
  author = {Georg Wenzel and Adam Jatowt},
  year = {2023},
  doi = {10.48550/arxiv.2308.00002},
  eprint = {2308.00002},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2308.00002}
}
```

- [← previous AmbiCoref: Evaluating Human and Model Sensitivity to Ambiguous Coreference](https://md.research.elamur.ai/free/en/papers/yuan-2023-ambicoref-evaluating-human-model-sensitivity/)
- [next → Analyzing Multilingual Competency of LLMs in Multi-Turn Instruction Following: A Case Study of Arabic](https://md.research.elamur.ai/free/en/papers/boughorbel-2023-analyzing-multilingual-competency-llms-multi/)
- [↑ back to papers of 2023](https://md.research.elamur.ai/free/en/papers/2023/page/14/)

## References

90 references, 24 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Abend O, Rappoport A (2013) [Universal conceptual cognitive annotation (ucca)](https://md.research.elamur.ai/free/en/papers/abend-2013-universal-conceptual-cognitive-annotation-ucca/). Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
2. Agrawal H, Chandrasekaran A, Batra D, et al (2016) Sort story: Sorting jumbled images and captions into stories. Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, 925–931 [find](https://scholar.google.com/scholar?q=Agrawal+H%2C+Chandrasekaran+A%2C+Batra+D%2C+et+al+%282016%29+Sort+story%3A+Sorting+jumbled+images+and+captions+into+stories%2E+Proceedings+of+the+2016+Conference+on+Empirical+Methods+in+Natural+Language+Processing%2C)
3. Allen JF (1983) Maintaining knowledge about temporal intervals. Communications of the ACM 26(11):832–843. https://doi.org/10.1016/b978-1-4832-1447-4.50033-x [↗](https://doi.org/10.1016/b978-1-4832-1447-4.50033-x)
4. Almquist A, Jatowt A (2019) Towards content expiry date determination: predicting validity periods of sentences. Advances in Information Retrieval: 41st European Conference on IR Research, ECIR 2019, Cologne, Germany, April 14–18, 2019, Proceedings, Part I 41, 86–101 [find](https://scholar.google.com/scholar?q=Almquist+A%2C+Jatowt+A+%282019%29+Towards+content+expiry+date+determination%3A+predicting+validity+periods+of+sentences%2E+Advances+in+Information+Retrieval%3A+41st+European+Conference+on+IR+Research%2C+ECIR+2019%2C+)
5. Banarescu L, Bonial C, Cai S, et al (2013) [Abstract meaning representation for sembanking](https://md.research.elamur.ai/free/en/papers/banarescu-2013-abstract-meaning-representation-sembanking/). Proceedings of the 7th linguistic annotation workshop and interoperability with discourse, 178–186
6. Bethard S, Savova G, Chen WT, et al (2016) [Semeval-2016 task 12: Clinical tempeval](https://md.research.elamur.ai/free/en/papers/bethard-2016-semeval-2016-task-12-clinical/). Proceedings of the 10th international workshop on semantic evaluation (SemEval-2016), 1052–1062
7. Bhargava P, Ng V (2022) Commonsense knowledge reasoning and generation with pre-trained language models: A survey. Proceedings of the AAAI Conference on Artificial Intelligence, 12317–12325 [find](https://scholar.google.com/scholar?q=Bhargava+P%2C+Ng+V+%282022%29+Commonsense+knowledge+reasoning+and+generation+with+pre%2Dtrained+language+models%3A+A+survey%2E+Proceedings+of+the+AAAI+Conference+on+Artificial+Intelligence%2C+12317%E2%80%9312325)
8. Bian N, Han X, Sun L, et al (2023) [Chatgpt is a knowledgeable but inexperienced solver: An investigation of commonsense problem in large language models](https://md.research.elamur.ai/free/en/papers/bian-2024-chatgpt-is-knowledgeable-but-inexperienced/). Preprint at https://arxiv.org/abs/2303.16421 [↗](https://arxiv.org/abs/2303.16421)
9. Brown T, Mann B, Ryder N, et al (2020) [Language models are few-shot learners](https://md.research.elamur.ai/free/en/papers/brown-2020-language-models-few-shot-learners/). Advances in neural information processing systems 33:1877–1901
10. Bulian J, Buck C, Gajewski W, et al (2022) [Tomayto, tomahto. beyond token-level answer equivalence for question answering evaluation](https://md.research.elamur.ai/free/en/papers/bulian-2022-tomayto-tomahto-beyond-token-level/). Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, 291–305
11. Cai B, Ding X, Chen B, et al (2022) Mitigating reporting bias in semi-supervised temporal commonsense inference with probabilistic soft logic. Proceedings of the AAAI Conference on Artificial Intelligence, 10454–10462 [find](https://scholar.google.com/scholar?q=Cai+B%2C+Ding+X%2C+Chen+B%2C+et+al+%282022%29+Mitigating+reporting+bias+in+semi%2Dsupervised+temporal+commonsense+inference+with+probabilistic+soft+logic%2E+Proceedings+of+the+AAAI+Conference+on+Artificial+Intellig)
12. Cai B, Ding X, Sun Z, et al (2023) Self-supervised logic induction for explainable fuzzy temporal commonsense reasoning. Proceedings of the AAAI Conference on Artificial Intelligence, 12580–12588 [find](https://scholar.google.com/scholar?q=Cai+B%2C+Ding+X%2C+Sun+Z%2C+et+al+%282023%29+Self%2Dsupervised+logic+induction+for+explainable+fuzzy+temporal+commonsense+reasoning%2E+Proceedings+of+the+AAAI+Conference+on+Artificial+Intelligence%2C+12580%E2%80%9312588)
13. Campos R, Dias G, Jorge AM, et al (2014) Survey of temporal information retrieval and related applications. ACM Computing Surveys (CSUR) 47(2):1–41. https://doi.org/10.1145/2619088 — [doi:10.1145/2619088](https://doi.org/10.1145/2619088) [↗](https://doi.org/10.1145/2619088)
14. Cao S, Wang L (2022) Time-aware prompting for text generation. Findings of the Association for Computational Linguistics: EMNLP 2022, 7231–7246 [find](https://scholar.google.com/scholar?q=Cao+S%2C+Wang+L+%282022%29+Time%2Daware+prompting+for+text+generation%2E+Findings+of+the+Association+for+Computational+Linguistics%3A+EMNLP+2022%2C+7231%E2%80%937246)
15. Chen W, Wang X, Wang WY (2021) A dataset for answering time-sensitive questions. Thirty-fifth Conference on Neural Information Processing Systems Datasets and Benchmarks Track (Round 2) [find](https://scholar.google.com/scholar?q=Chen+W%2C+Wang+X%2C+Wang+WY+%282021%29+A+dataset+for+answering+time%2Dsensitive+questions%2E+Thirty%2Dfifth+Conference+on+Neural+Information+Processing+Systems+Datasets+and+Benchmarks+Track+%28Round+2%29)
16. Davis E (2023) Benchmarks for automated commonsense reasoning: A survey. Preprint at https://arxiv.org/abs/2302.04752 [↗](https://arxiv.org/abs/2302.04752)
17. DeJong G (1982) An overview of the frump system. Strategies for natural language processing 113:149–176 [find](https://scholar.google.com/scholar?q=DeJong+G+%281982%29+An+overview+of+the+frump+system%2E+Strategies+for+natural+language+processing+113%3A149%E2%80%93176)
18. Dhingra B, Cole JR, Eisenschlos JM, et al (2022) [Time-aware language models as temporal knowledge bases](https://md.research.elamur.ai/free/en/papers/dhingra-2022-time-aware-language-models-as/). Transactions of the Association for Computational Linguistics 10:257–273. https://doi.org/10.1162/tacl a 00459 [↗](https://doi.org/10.1162/tacl)
19. Ferro L, Mani I, Sundheim B, et al (2001) Tides temporal annotation guidelines version 1.0. 2. [find](https://scholar.google.com/scholar?q=Ferro+L%2C+Mani+I%2C+Sundheim+B%2C+et+al+%282001%29+Tides+temporal+annotation+guidelines+version+1%2E0%2E+2%2E)
20. The MITRE Corporation, McLean-VG-USA Fries J (2016) Recurrent neural networks vs. joint inference for clinical temporal information extraction. Proceedings of the 10th International Workshop on Semantic Evaluation (SemEval-2016), 1274–9 [find](https://scholar.google.com/scholar?q=The+MITRE+Corporation%2C+McLean%2DVG%2DUSA+Fries+J+%282016%29+Recurrent+neural+networks+vs%2E+joint+inference+for+clinical+temporal+information+extraction%2E+Proceedings+of+the+10th+International+Workshop+on+Semant)
21. Gardner M, Artzi Y, Basmov V, et al (2020) [Evaluating models’ local decision boundaries via contrast sets](https://md.research.elamur.ai/free/en/papers/gardner-2020-evaluating-models-local-decision-boundaries/). Findings of the Association for Computational Linguistics: EMNLP 2020, 1307–1323
22. Ghosal D, Hong P, Shen S, et al (2021a) Cider: Commonsense inference for dialogue explanation and reasoning. Proceedings of the 22nd Annual Meeting of the Special Interest Group on Discourse and Dialogue, 301–313 [find](https://scholar.google.com/scholar?q=Ghosal+D%2C+Hong+P%2C+Shen+S%2C+et+al+%282021a%29+Cider%3A+Commonsense+inference+for+dialogue+explanation+and+reasoning%2E+Proceedings+of+the+22nd+Annual+Meeting+of+the+Special+Interest+Group+on+Discourse+and+Dialo)
23. Ghosal D, Majumder N, Mihalcea R, et al (2021b) Stack: Sentence ordering with temporal commonsense knowledge. Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, 8676–8686 [find](https://scholar.google.com/scholar?q=Ghosal+D%2C+Majumder+N%2C+Mihalcea+R%2C+et+al+%282021b%29+Stack%3A+Sentence+ordering+with+temporal+commonsense+knowledge%2E+Proceedings+of+the+2021+Conference+on+Empirical+Methods+in+Natural+Language+Processing%2C+86)
24. Gordon J, Van Durme B (2013) Reporting bias and knowledge extraction. Preprint at [https://openreview.net/pdf?id=AzxEzvpdE3Wcy](https://openreview.net/pdf?id=AzxEzvpdE3Wcy) [↗](https://openreview.net/pdf?id=AzxEzvpdE3Wcy)
25. Grover C, Tobin R, Alex B, et al (2010) Edinburgh-ltg: Tempeval-2 system description. Proceedings of the 5th International Workshop on Semantic Evaluation, 333–336 [find](https://scholar.google.com/scholar?q=Grover+C%2C+Tobin+R%2C+Alex+B%2C+et+al+%282010%29+Edinburgh%2Dltg%3A+Tempeval%2D2+system+description%2E+Proceedings+of+the+5th+International+Workshop+on+Semantic+Evaluation%2C+333%E2%80%93336)
26. Guan J, Huang F, Zhao Z, et al (2020) A knowledge-enhanced pretraining model for commonsense story generation. Transactions of the Association for Computational Linguistics 8:93–108. https://doi.org/10.1162/tacl a 00302 [↗](https://doi.org/10.1162/tacl)
27. Han R, Liang M, Alhafni B, et al (2019) Contextualized word embeddings enhanced event temporal relation extraction for story understanding. ArXiv preprint arXiv:1904.11942 — [arXiv:1904.11942](https://arxiv.org/abs/1904.11942) [find](https://scholar.google.com/scholar?q=Han+R%2C+Liang+M%2C+Alhafni+B%2C+et+al+%282019%29+Contextualized+word+embeddings+enhanced+event+temporal+relation+extraction+for+story+understanding%2E+ArXiv+preprint+arXiv%3A1904%2E11942+%E2%80%94+%5BarXiv%3A1904%2E11942%5D%28https%3A%2F)
28. Helwe C, Clavel C, Suchanek FM (2021) Reasoning with transformer-based models: Deep learning, but shallow reasoning. 3rd conference on automated knowledge base construction Hosokawa T, Jatowt A, Sugiyama K (2023) Temporal natural language inference: Evidence-based evaluation of temporal text validit… [find](https://scholar.google.com/scholar?q=Helwe+C%2C+Clavel+C%2C+Suchanek+FM+%282021%29+Reasoning+with+transformer%2Dbased+models%3A+Deep+learning%2C+but+shallow+reasoning%2E+3rd+conference+on+automated+knowledge+base+construction+Hosokawa+T%2C+Jatowt+A%2C+Sugiy)
29. Huang L, Le Bras R, Bhagavatula C, et al (2019) [Cosmos qa: Machine reading comprehension with contextual commonsense reasoning](https://md.research.elamur.ai/free/en/papers/huang-2019-cosmos-qa-machine-reading-comprehension/). Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCN…
30. Hwang JD, Bhagavatula C, Le Bras R, et al (2021) [(comet-) atomic 2020: on symbolic and neural commonsense knowledge graphs](https://md.research.elamur.ai/free/en/papers/hwang-2020-comet-atomic-2020-symbolic-neural/). Proceedings of the AAAI Conference on Artificial Intelligence, 6384–6392
31. Jain R, Sojitra D, Acharya A, et al (2023) [Do language models have a common sense regarding time? revisiting temporal commonsense reasoning in the era of large language models](https://md.research.elamur.ai/free/en/papers/jain-2023-do-language-models-have-common/). Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing (EMNLP 2023)
32. Ji S, Pan S, Cambria E, et al (2021) A survey on knowledge graphs: Representation, acquisition, and applications. IEEE transactions on neural networks and learning systems 33(2):494–514. https://doi.org/10.1109/tnnls.2021.3070843 [↗](https://doi.org/10.1109/tnnls.2021.3070843)
33. Kenton JDMWC, Toutanova LK (2019) [Bert: Pre-training of deep bidirectional transformers for language understanding](https://md.research.elamur.ai/free/en/papers/devlin-2019-bert-pre-training-deep-bidirectional/). Proceedings of NAACL-HLT, 4171–4186
34. Kimura M, Pereira LK, Kobayashi I (2021) Towards a language model for temporal commonsense reasoning. Proceedings of the Student Research Workshop Associated with RANLP 2021, 78–84 [find](https://scholar.google.com/scholar?q=Kimura+M%2C+Pereira+LK%2C+Kobayashi+I+%282021%29+Towards+a+language+model+for+temporal+commonsense+reasoning%2E+Proceedings+of+the+Student+Research+Workshop+Associated+with+RANLP+2021%2C+78%E2%80%9384)
35. Levesque H, Davis E, Morgenstern L (2012) The winograd schema challenge. Thirteenth international conference on the principles of knowledge representation and reasoning Liu Y, Ott M, Goyal N, et al (2019) [Roberta: A robustly optimized bert pretraining approach](https://md.research.elamur.ai/free/en/papers/liu-2019-roberta-robustly-optimized-bert-pretraining/). Preprint at https://arxiv.org/abs/1907… [↗](https://arxiv.org/abs/1907…)
36. Llorens H, Saquete E, Navarro B (2010) Tipsem (english and spanish): Evaluating crfs and semantic roles in tempeval-2. Proceedings of the 5th International Workshop on Semantic Evaluation, 284–291 [find](https://scholar.google.com/scholar?q=Llorens+H%2C+Saquete+E%2C+Navarro+B+%282010%29+Tipsem+%28english+and+spanish%29%3A+Evaluating+crfs+and+semantic+roles+in+tempeval%2D2%2E+Proceedings+of+the+5th+International+Workshop+on+Semantic+Evaluation%2C+284%E2%80%93291)
37. Lymperaiou M, Stamou G (2022) A survey on knowledge-enhanced multimodal learning. Preprint at https://arxiv.org/abs/2211.12328 — [arXiv:2211.12328](https://arxiv.org/abs/2211.12328) [↗](https://arxiv.org/abs/2211.12328)
38. Lynden S, Heilemariam M, Kim KS, et al (2023) Commonsense temporal action knowledge (cotak) dataset. Proceedings of the 32nd ACM International Conference on Information and Knowledge Management (CIKM 2023) [find](https://scholar.google.com/scholar?q=Lynden+S%2C+Heilemariam+M%2C+Kim+KS%2C+et+al+%282023%29+Commonsense+temporal+action+knowledge+%28cotak%29+dataset%2E+Proceedings+of+the+32nd+ACM+International+Conference+on+Information+and+Knowledge+Management+%28CIKM+)
39. Mikolov T, Chen K, Corrado G, et al (2013) [Efficient estimation of word representations in vector space](https://md.research.elamur.ai/free/en/papers/mikolov-2013-efficient-estimation-word-representations-vector/). Preprint at https://arxiv.org/abs/1301.3781 [↗](https://arxiv.org/abs/1301.3781)
40. Mostafazadeh N, Chambers N, He X, et al (2016a) A corpus and cloze evaluation for deeper understanding of commonsense stories. Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 839–849 [find](https://scholar.google.com/scholar?q=Mostafazadeh+N%2C+Chambers+N%2C+He+X%2C+et+al+%282016a%29+A+corpus+and+cloze+evaluation+for+deeper+understanding+of+commonsense+stories%2E+Proceedings+of+the+2016+Conference+of+the+North+American+Chapter+of+the+A)
41. Mostafazadeh N, Grealish A, Chambers N, et al (2016b) Caters: Causal and temporal relation scheme for semantic annotation of event structures. Proceedings of the Fourth Workshop on Events, 51–61 [find](https://scholar.google.com/scholar?q=Mostafazadeh+N%2C+Grealish+A%2C+Chambers+N%2C+et+al+%282016b%29+Caters%3A+Causal+and+temporal+relation+scheme+for+semantic+annotation+of+event+structures%2E+Proceedings+of+the+Fourth+Workshop+on+Events%2C+51%E2%80%9361)
42. Mostafazadeh N, Kalyanpur A, Moon L, et al (2020) [Glucose: Generalized and contextualized story explanations](https://md.research.elamur.ai/free/en/papers/mostafazadeh-2020-glucose-generalized-contextualized-story-explanations/). Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), 4569–4586
43. Ning Q, Wu H, Peng H, et al (2018a) Improving temporal relation extraction with a globally acquired statistical resource. Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), 841–851 [find](https://scholar.google.com/scholar?q=Ning+Q%2C+Wu+H%2C+Peng+H%2C+et+al+%282018a%29+Improving+temporal+relation+extraction+with+a+globally+acquired+statistical+resource%2E+Proceedings+of+the+2018+Conference+of+the+North+American+Chapter+of+the+Associ)
44. Ning Q, Wu H, Roth D (2018b) A multi-axis annotation scheme for event temporal relations. Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 1318–1328 [find](https://scholar.google.com/scholar?q=Ning+Q%2C+Wu+H%2C+Roth+D+%282018b%29+A+multi%2Daxis+annotation+scheme+for+event+temporal+relations%2E+Proceedings+of+the+56th+Annual+Meeting+of+the+Association+for+Computational+Linguistics+%28Volume+1%3A+Long+Papers)
45. Ning Q, Wu H, Dasigi P, et al (2020a) Easy, reproducible and quality-controlled data collection with crowdaq. Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, 127–134 [find](https://scholar.google.com/scholar?q=Ning+Q%2C+Wu+H%2C+Dasigi+P%2C+et+al+%282020a%29+Easy%2C+reproducible+and+quality%2Dcontrolled+data+collection+with+crowdaq%2E+Proceedings+of+the+2020+Conference+on+Empirical+Methods+in+Natural+Language+Processing%3A+Sy)
46. Ning Q, Wu H, Han R, et al (2020b) [Torque: A reading comprehension dataset of temporal ordering questions](https://md.research.elamur.ai/free/en/papers/ning-2020-torque-reading-comprehension-dataset-temporal/). Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), 1158–1172
47. O’Gorman T, Wright-Bettner K, Palmer M (2016) [Richer event description: Integrating event coreference with temporal, causal and bridging annotation](https://md.research.elamur.ai/free/en/papers/gorman-2016-richer-event-description-integrating-event/). Proceedings of the 2nd workshop on computing news storylines (CNS 2016), 47–56
48. Pan F, Mulkar-Mehta R, Hobbs JR (2006) Extending timeml with typical durations of events. Proceedings of the Workshop on Annotating and Reasoning about Time and Events, 38–45 [find](https://scholar.google.com/scholar?q=Pan+F%2C+Mulkar%2DMehta+R%2C+Hobbs+JR+%282006%29+Extending+timeml+with+typical+durations+of+events%2E+Proceedings+of+the+Workshop+on+Annotating+and+Reasoning+about+Time+and+Events%2C+38%E2%80%9345)
49. Pasquali A, Campos R, Ribeiro A, et al (2021) Tls-covid19: A new annotated corpus for timeline summarization. European Conference on Information Retrieval, 497–512 [find](https://scholar.google.com/scholar?q=Pasquali+A%2C+Campos+R%2C+Ribeiro+A%2C+et+al+%282021%29+Tls%2Dcovid19%3A+A+new+annotated+corpus+for+timeline+summarization%2E+European+Conference+on+Information+Retrieval%2C+497%E2%80%93512)
50. Pennington J, Socher R, Manning CD (2014) [Glove: Global vectors for word representation](https://md.research.elamur.ai/free/en/papers/pennington-2014-glove-global-vectors-word-representation/). Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP), 1532–1543
51. Pereira L, Liu X, Cheng F, et al (2020) Adversarial training for commonsense inference. Proceedings of the 5th Workshop on Representation Learning for NLP (RepL4NLP-2020), 55–60 [find](https://scholar.google.com/scholar?q=Pereira+L%2C+Liu+X%2C+Cheng+F%2C+et+al+%282020%29+Adversarial+training+for+commonsense+inference%2E+Proceedings+of+the+5th+Workshop+on+Representation+Learning+for+NLP+%28RepL4NLP%2D2020%29%2C+55%E2%80%9360)
52. Pereira L, Cheng F, Asahara M, et al (2021) Alice++: Adversarial training for robust and effective temporal reasoning. Proceedings of the 35th Pacific Asia Conference on Language, Information and Computation, 373–382 [find](https://scholar.google.com/scholar?q=Pereira+L%2C+Cheng+F%2C+Asahara+M%2C+et+al+%282021%29+Alice%2B%2B%3A+Adversarial+training+for+robust+and+effective+temporal+reasoning%2E+Proceedings+of+the+35th+Pacific+Asia+Conference+on+Language%2C+Information+and+Comp)
53. Pustejovsky J (2003) The timebank corpus. Proceedings of Corpus Linguistics 2003, 647–656 [find](https://scholar.google.com/scholar?q=Pustejovsky+J+%282003%29+The+timebank+corpus%2E+Proceedings+of+Corpus+Linguistics+2003%2C+647%E2%80%93656)
54. Pustejovsky J, Castano JM, Ingria R, et al (2003) Timeml: Robust specification of event and temporal expressions in text. New directions in question answering 3:28– 34 [find](https://scholar.google.com/scholar?q=Pustejovsky+J%2C+Castano+JM%2C+Ingria+R%2C+et+al+%282003%29+Timeml%3A+Robust+specification+of+event+and+temporal+expressions+in+text%2E+New+directions+in+question+answering+3%3A28%E2%80%93+34)
55. Pustejovsky J, Littman J, Saurı́ R, et al (2006) Timebank 1.2 documentation. Event London, no. April, 6–11 [find](https://scholar.google.com/scholar?q=Pustejovsky+J%2C+Littman+J%2C+Saur%C4%B1%CC%81+R%2C+et+al+%282006%29+Timebank+1%2E2+documentation%2E+Event+London%2C+no%2E+April%2C+6%E2%80%9311)
56. Qin C, Zhang A, Zhang Z, et al (2023) [Is chatgpt a general-purpose natural language processing task solver?](https://md.research.elamur.ai/free/en/papers/qin-2023-chatgpt-general-purpose-natural-language/) Preprint at https://arxiv.org/abs/2302.06476 [↗](https://arxiv.org/abs/2302.06476)
57. Qin L, Gupta A, Upadhyay S, et al (2021) Timedial: Temporal commonsense reasoning in dialog. Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), 7066–7076 [find](https://scholar.google.com/scholar?q=Qin+L%2C+Gupta+A%2C+Upadhyay+S%2C+et+al+%282021%29+Timedial%3A+Temporal+commonsense+reasoning+in+dialog%2E+Proceedings+of+the+59th+Annual+Meeting+of+the+Association+for+Computational+Linguistics+and+the+11th+Intern)
58. Radford A, Narasimhan K, Salimans T, et al (2018) [Improving language understanding by generative pre-training Radford A, Wu J, Child R, et al (2019) Language models are unsupervised multitask learners](https://md.research.elamur.ai/free/en/papers/raffel-2019-exploring-limits-transfer-learning-unified/). OpenAI blog Raffel C, Shazeer N, Roberts A, et al (2020) Exploring the limits of transfer learning…
59. Richardson M, Burges CJ, Renshaw E (2013) [Mctest: A challenge dataset for the open-domain machine comprehension of text](https://md.research.elamur.ai/free/en/papers/richardson-2013-mctest-challenge-dataset-open-domain/). Proceedings of the 2013 conference on empirical methods in natural language processing, 193–203
60. Rosin G, Radinsky K (2022) Temporal attention for language models. Findings of the Association for Computational Linguistics: NAACL 2022, 1498–1508 [find](https://scholar.google.com/scholar?q=Rosin+G%2C+Radinsky+K+%282022%29+Temporal+attention+for+language+models%2E+Findings+of+the+Association+for+Computational+Linguistics%3A+NAACL+2022%2C+1498%E2%80%931508)
61. Rosin GD, Guy I, Radinsky K (2022) Time masking for temporal language models. Proceedings of the Fifteenth ACM International Conference on Web Search and Data Mining, 833–841 [find](https://scholar.google.com/scholar?q=Rosin+GD%2C+Guy+I%2C+Radinsky+K+%282022%29+Time+masking+for+temporal+language+models%2E+Proceedings+of+the+Fifteenth+ACM+International+Conference+on+Web+Search+and+Data+Mining%2C+833%E2%80%93841)
62. Sakaguchi K, Bras RL, Bhagavatula C, et al (2021) Winogrande: An adversarial winograd schema challenge at scale. Communications of the ACM 64(9):99–106. https://doi.org/10.1145/3474381 [↗](https://doi.org/10.1145/3474381)
63. Sauréi R, Littman J, Knippen B, et al (2005) Timeml annotation guidelines. [https://timeml.github.io/site/publications/timeMLdocs/annguide](https://timeml.github.io/site/publications/timeMLdocs/annguide) 1.2.pdf Shwartz V, Choi Y (2020) Do neural language models overcome reporting bias? Proceedings of the 28th International Conference on Computational Linguistic… [↗](https://timeml.github.io/site/publications/timeMLdocs/annguide)
64. Singh S, Wen N, Hou Y, et al (2021) Com2sense: A commonsense reasoning benchmark with complementary sentences. Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, 883–898 [find](https://scholar.google.com/scholar?q=Singh+S%2C+Wen+N%2C+Hou+Y%2C+et+al+%282021%29+Com2sense%3A+A+commonsense+reasoning+benchmark+with+complementary+sentences%2E+Findings+of+the+Association+for+Computational+Linguistics%3A+ACL%2DIJCNLP+2021%2C+883%E2%80%93898)
65. Sousa H, Jorge A, Campos R (2023) tieval: An evaluation framework for temporal information extraction systems. Preprint at https://arxivorg/abs/230104643 [find](https://scholar.google.com/scholar?q=Sousa+H%2C+Jorge+A%2C+Campos+R+%282023%29+tieval%3A+An+evaluation+framework+for+temporal+information+extraction+systems%2E+Preprint+at+https%3A%2F%2Farxivorg%2Fabs%2F230104643)
66. Speer R, Chin J, Havasi C (2017) [Conceptnet 5.5: An open multilingual graph of general knowledge](https://md.research.elamur.ai/free/en/papers/storks-2019-recent-advances-natural-language-inference/). Proceedings of the AAAI conference on artificial intelligence Storks S, Gao Q, Chai JY (2019) Commonsense reasoning for natural language understanding: A survey of benchmarks, resources, and approaches.…
67. Strötgen J, Gertz M (2010) Heideltime: High quality rule-based extraction and normalization of temporal expressions. Proceedings of the 5th international workshop on semantic evaluation, 321–324 [find](https://scholar.google.com/scholar?q=Stro%CC%88tgen+J%2C+Gertz+M+%282010%29+Heideltime%3A+High+quality+rule%2Dbased+extraction+and+normalization+of+temporal+expressions%2E+Proceedings+of+the+5th+international+workshop+on+semantic+evaluation%2C+321%E2%80%93324)
68. Sun Z, Fan C, Han Q, et al (2020) Self-explaining structures improve nlp models. Preprint at https://arxiv.org/abs/2012.01786 — [arXiv:2012.01786](https://arxiv.org/abs/2012.01786) [↗](https://arxiv.org/abs/2012.01786)
69. Tan Q, Ng HT, Bing L (2023) Towards benchmarking and improving the temporal reasoning capability of large language models Toosi A, Bottino AG, Saboury B, et al (2021) A brief history of ai: how to prevent another winter (a critical review). PET clinics 16(4):449–469. https://doi.org/10.1016/j.cpet.2… [↗](https://doi.org/10.1016/j.cpet.2…)
70. UzZaman N, Llorens H, Derczynski L, et al (2013) Semeval-2013 task 1: Tempeval-3: Evaluating time expressions, events, and temporal relations. Second Joint Conference on Lexical and Computational Semantics (* SEM), Volume 2: Proceedings of the Seventh International Workshop on Semantic Evaluation (S… [find](https://scholar.google.com/scholar?q=UzZaman+N%2C+Llorens+H%2C+Derczynski+L%2C+et+al+%282013%29+Semeval%2D2013+task+1%3A+Tempeval%2D3%3A+Evaluating+time+expressions%2C+events%2C+and+temporal+relations%2E+Second+Joint+Conference+on+Lexical+and+Computational+Sema)
71. Vashishtha S, Poliak A, Lal YK, et al (2020) [Temporal reasoning in natural language inference](https://md.research.elamur.ai/free/en/papers/vashishtha-2020-temporal-reasoning-natural-language-inference/). Findings of the Association for Computational Linguistics: EMNLP 2020, 4070–4078
72. Vaswani A, Shazeer N, Parmar N, et al (2017) [Attention is all you need](https://md.research.elamur.ai/free/en/papers/vaswani-2017-attention-all-need/). Advances in neural information processing systems 30
73. Verhagen M, Gaizauskas R, Schilder F, et al (2007) Semeval-2007 task 15: Tempeval temporal relation identification. Proceedings of the fourth international workshop on semantic evaluations (SemEval-2007), 75–80 [find](https://scholar.google.com/scholar?q=Verhagen+M%2C+Gaizauskas+R%2C+Schilder+F%2C+et+al+%282007%29+Semeval%2D2007+task+15%3A+Tempeval+temporal+relation+identification%2E+Proceedings+of+the+fourth+international+workshop+on+semantic+evaluations+%28SemEval%2D20)
74. Verhagen M, Sauri R, Caselli T, et al (2010) Semeval-2010 task 13: Tempeval-2. Proceedings of the 5th international workshop on semantic evaluation, 57–62 [find](https://scholar.google.com/scholar?q=Verhagen+M%2C+Sauri+R%2C+Caselli+T%2C+et+al+%282010%29+Semeval%2D2010+task+13%3A+Tempeval%2D2%2E+Proceedings+of+the+5th+international+workshop+on+semantic+evaluation%2C+57%E2%80%9362)
75. Vila L (1994) A survey on temporal reasoning in artificial intelligence. Ai Communications 7(1):4–28. https://doi.org/10.3233/aic-1994-7102 — [doi:10.3233/aic-1994-7102](https://doi.org/10.3233/aic-1994-7102) [↗](https://doi.org/10.3233/aic-1994-7102)
76. Wang J, Jatowt A, Färber M, et al (2020) Answering event-related questions over long-term news article archives. Advances in Information Retrieval: 42nd European Conference on IR Research, ECIR 2020, Lisbon, Portugal, April 14–17, 2020, Proceedings, Part I 42, 774–789 [find](https://scholar.google.com/scholar?q=Wang+J%2C+Jatowt+A%2C+Fa%CC%88rber+M%2C+et+al+%282020%29+Answering+event%2Drelated+questions+over+long%2Dterm+news+article+archives%2E+Advances+in+Information+Retrieval%3A+42nd+European+Conference+on+IR+Research%2C+ECIR+2020%2C)
77. Wang J, Jatowt A, Yoshikawa M (2022) Bitimebert: Extending pre-trained language representations with bi-temporal information. Preprint at https://arxiv.org/abs/ 2204.13032 [↗](https://arxiv.org/abs/)
78. Wang X, Jiang M (2020) Precise temporal slot filling via truth finding with data-driven commonsense. Knowledge and Information Systems 62(10):4113–4139. https://doi.org/10.1007/s10115-020-01493-w Wolff P, Song G (2003) Models of causation and the semantics of causal verbs. Cognitive psychology 47(3… — [doi:10.1007/s10115-020-01493-w](https://doi.org/10.1007/s10115-020-01493-w) [↗](https://doi.org/10.1007/s10115-020-01493-w)
79. Yang Z, Du X, Rush AM, et al (2020) Improving event duration prediction via time-aware pre-training. Findings of the Association for Computational Linguistics: EMNLP 2020 [find](https://scholar.google.com/scholar?q=Yang+Z%2C+Du+X%2C+Rush+AM%2C+et+al+%282020%29+Improving+event+duration+prediction+via+time%2Daware+pre%2Dtraining%2E+Findings+of+the+Association+for+Computational+Linguistics%3A+EMNLP+2020)
80. Yin D, Dong L, Cheng H, et al (2022) A survey of knowledge-intensive nlp with pre-trained language models. Preprint at https://arxiv.org/abs/2202.08772 — [arXiv:2202.08772](https://arxiv.org/abs/2202.08772) [↗](https://arxiv.org/abs/2202.08772)
81. Yu C, Zhang H, Song Y, et al (2022) Cocolm: Complex commonsense enhanced language model with discourse relations. Findings of the Association for Computational Linguistics: ACL 2022, 1175–1187 [find](https://scholar.google.com/scholar?q=Yu+C%2C+Zhang+H%2C+Song+Y%2C+et+al+%282022%29+Cocolm%3A+Complex+commonsense+enhanced+language+model+with+discourse+relations%2E+Findings+of+the+Association+for+Computational+Linguistics%3A+ACL+2022%2C+1175%E2%80%931187)
82. Zhang H, Liu X, Pan H, et al (2020a) Aser: A large-scale eventuality knowledge graph. Proceedings of the web conference 2020, 201–211 [find](https://scholar.google.com/scholar?q=Zhang+H%2C+Liu+X%2C+Pan+H%2C+et+al+%282020a%29+Aser%3A+A+large%2Dscale+eventuality+knowledge+graph%2E+Proceedings+of+the+web+conference+2020%2C+201%E2%80%93211)
83. Zhang J, Zhang H, Su W, et al (2022) Rock: Causal inference principles for reasoning about commonsense causality. International Conference on Machine Learning, 26750–26771 [find](https://scholar.google.com/scholar?q=Zhang+J%2C+Zhang+H%2C+Su+W%2C+et+al+%282022%29+Rock%3A+Causal+inference+principles+for+reasoning+about+commonsense+causality%2E+International+Conference+on+Machine+Learning%2C+26750%E2%80%9326771)
84. Zhang L, Lyu Q, Callison-Burch C (2020b) Reasoning about goals, steps, and temporal ordering with wikihow. Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), 4630–4639 [find](https://scholar.google.com/scholar?q=Zhang+L%2C+Lyu+Q%2C+Callison%2DBurch+C+%282020b%29+Reasoning+about+goals%2C+steps%2C+and+temporal+ordering+with+wikihow%2E+Proceedings+of+the+2020+Conference+on+Empirical+Methods+in+Natural+Language+Processing+%28EMNLP)
85. Zhang Z, Zhou Z, Tang H, et al (2021) Enriching ontology with temporal commonsense for low-resource audio tagging. Proceedings of the 30th ACM International Conference on Information & Knowledge Management, 3652–3656 [find](https://scholar.google.com/scholar?q=Zhang+Z%2C+Zhou+Z%2C+Tang+H%2C+et+al+%282021%29+Enriching+ontology+with+temporal+commonsense+for+low%2Dresource+audio+tagging%2E+Proceedings+of+the+30th+ACM+International+Conference+on+Information+%26+Knowledge+Manag)
86. Zhou B, Khashabi D, Ning Q, et al (2019) “going on a vacation” takes longer than “going for a walk”: A study of temporal commonsense understanding. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Pr… [find](https://scholar.google.com/scholar?q=Zhou+B%2C+Khashabi+D%2C+Ning+Q%2C+et+al+%282019%29+%E2%80%9Cgoing+on+a+vacation%E2%80%9D+takes+longer+than+%E2%80%9Cgoing+for+a+walk%E2%80%9D%3A+A+study+of+temporal+commonsense+understanding%2E+Proceedings+of+the+2019+Conference+on+Empirical+Meth)
87. Zhou B, Ning Q, Khashabi D, et al (2020a) Temporal common sense acquisition with minimal supervision. Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, 7579–7589 [find](https://scholar.google.com/scholar?q=Zhou+B%2C+Ning+Q%2C+Khashabi+D%2C+et+al+%282020a%29+Temporal+common+sense+acquisition+with+minimal+supervision%2E+Proceedings+of+the+58th+Annual+Meeting+of+the+Association+for+Computational+Linguistics%2C+7579%E2%80%937589)
88. Zhou B, Richardson K, Ning Q, et al (2021) Temporal reasoning on implicit events from distant supervision. Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 1361–1371 [find](https://scholar.google.com/scholar?q=Zhou+B%2C+Richardson+K%2C+Ning+Q%2C+et+al+%282021%29+Temporal+reasoning+on+implicit+events+from+distant+supervision%2E+Proceedings+of+the+2021+Conference+of+the+North+American+Chapter+of+the+Association+for+Compu)
89. Zhou B, Chen Y, Liu K, et al (2022) Generating temporally-ordered event sequences via event optimal transport. Proceedings of the 29th International Conference on Computational Linguistics, 1875–1884 [find](https://scholar.google.com/scholar?q=Zhou+B%2C+Chen+Y%2C+Liu+K%2C+et+al+%282022%29+Generating+temporally%2Dordered+event+sequences+via+event+optimal+transport%2E+Proceedings+of+the+29th+International+Conference+on+Computational+Linguistics%2C+1875%E2%80%931884)
90. Zhou Y, Zhao T, Jiang M (2020b) A probabilistic model with commonsense constraints for pattern-based temporal fact extraction. Proceedings of the Third Workshop on Fact Extraction and VERification (FEVER), 18–25 [find](https://scholar.google.com/scholar?q=Zhou+Y%2C+Zhao+T%2C+Jiang+M+%282020b%29+A+probabilistic+model+with+commonsense+constraints+for+pattern%2Dbased+temporal+fact+extraction%2E+Proceedings+of+the+Third+Workshop+on+Fact+Extraction+and+VERification+%28FE)

*Updated 3 November 2026*