---
title: "Critic Rule Induction — paper by Ning et al. (2026)"
description: "The authors propose CRI (Critic-Guided Rule Induction), which treats every rule as a hypothesis to be tested and splits the… 21 references."
authors: ["Yingsong Ning", "Fu Zhang", "Jingwei Cheng", "Jiashun Peng", "Xiaoke Wang"]
year: 2026
venue: "Findings of the Association for Computational Linguistics: ACL 2026"
source: "https://aclanthology.org/2026.findings-acl.1471/"
doi: "10.18653/v1/2026.findings-acl.1471"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/ning-2026-critic-rule-induction-improving-temporal/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/ning-2026-critic-rule-induction-improving-temporal/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/26/) / Critic Rule Induction: Improving Temporal Knowledge Graph Forecasting with Generator-Critic Language Models

# Critic Rule Induction: Improving Temporal Knowledge Graph Forecasting with Generator-Critic Language Models

**Authors:** [Yingsong Ning](https://md.research.elamur.ai/free/en/authors/yingsong-ning/), [Fu Zhang](https://md.research.elamur.ai/free/en/authors/fu-zhang/), [Jingwei Cheng](https://md.research.elamur.ai/free/en/authors/jingwei-cheng/), [Jiashun Peng](https://md.research.elamur.ai/free/en/authors/jiashun-peng/) and [Xiaoke Wang](https://md.research.elamur.ai/free/en/authors/xiaoke-wang/) · **Year:** 2026 · **Venue:** [Findings of the Association for Computational Linguistics: ACL 2026](https://md.research.elamur.ai/free/en/venues/findings/) ([Findings of the Association for Computational Linguistics 2026](https://md.research.elamur.ai/free/en/venues/findings/2026/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2026.findings-acl.1471/) · [PDF](https://aclanthology.org/2026.findings-acl.1471.pdf) · [DOI 10.18653/v1/2026.findings-acl.1471](https://doi.org/10.18653/v1/2026.findings-acl.1471) · [OpenAlex](https://openalex.org/W7166871261)

**Citations:** 0 citations (OpenAlex, 2026-10-14)

## Summary
The paper addresses forecasting future events in temporal knowledge graphs (TKGs) using explicit logical rules. Classic rule miners such as TLogic count co-occurrences along enumerated paths. They yield few rules that miss many queries. LLM-based approaches like LLM-DA can propose more rules, but these may sound reasonable while being poorly supported by the graph or breaking time ordering. The authors propose CRI (Critic-Guided Rule Induction), which treats every rule as a hypothesis to be tested and splits the work into generation and discrimination stages:

1. Time-constrained random walks collect seed paths, which an LLM generalizes into candidate rules.
2. A Fact-Grounded Rule Evaluator scores each candidate on temporal causality, statistical support, semantic consistency and LLM-judged logic, then calibrates and filters the results.
3. Symbolic reasoning over the surviving rules ranks candidate entities, with an explicit evidence trail for each prediction.

On three benchmarks, CRI reports state-of-the-art results against embedding, rule and LLM baselines. Its largest gains come on rare relations. This shows that LLM generalization can improve rule coverage without sacrificing interpretability.

## Key points
- **Pipeline components:**
  - Markovian random walks with a temporal constraint filter.
  - Relation pruning via Sentence-BERT similarity (top-k).
  - An LLM generator for raw rule hypotheses.
  - An evaluator combining S_temp, S_conf, S_sem and S_llm, calibrated with a Beta CDF and thresholded by τ.
  - Final scoring weighted by temporal recency.
- **ICEWS14 results:** Hits@10 = 0.702 and Hits@1 = 0.381. The gap is attributed mainly to difficulty separating similar candidate entities.
- **Sparse relations:** ICEWS14 relations were split by frequency (\<50, 50–500, >500 instances). Sparse relations make up about 62% of the total. On this group, CRI reportedly beats TLogic by about +107% in Hits@10 and also beats RE-GCN.
- **Ablation:** Dropping the temporal causality check is highlighted as especially damaging. Statistical and semantic scores serve as complementary filters.
- **Limitations:**
  - LLM API costs, although rule induction runs offline.
  - Dependence on the quality of the backbone LLM.
  - Weak handling of long-range or global patterns.

  Future work targets distilling the rules into smaller models.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Northeastern University](https://md.research.elamur.ai/free/en/universities/northeastern-university-cn/) — University, Shenyang, China · 42 works · 1,940 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works

🧭 Topics (2): [🧠 Language models & pretraining](https://md.research.elamur.ai/free/en/topics/language-models/) · 3,320 works · [🕸️ Knowledge graphs & ontologies](https://md.research.elamur.ai/free/en/topics/knowledge-graphs/) · 192 works

## Cite

Cite as: Yingsong Ning, Fu Zhang, Jingwei Cheng, Jiashun Peng and Xiaoke Wang. 2026. Critic Rule Induction: Improving Temporal Knowledge Graph Forecasting with Generator-Critic Language Models. Findings of the Association for Computational Linguistics: ACL 2026.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{ning2026critic,
  title = {{Critic Rule Induction: Improving Temporal Knowledge Graph Forecasting with Generator-Critic Language Models}},
  author = {Yingsong Ning, Fu Zhang, Jingwei Cheng, Jiashun Peng and Xiaoke Wang},
  booktitle = {Findings of the Association for Computational Linguistics: ACL 2026},
  year = {2026},
  doi = {10.18653/v1/2026.findings-acl.1471},
  url = {https://doi.org/10.18653/v1/2026.findings-acl.1471}
}
```

- [← previous Criterial Features in German: Towards Interpretable NLP in Readability Assessment](https://md.research.elamur.ai/free/en/papers/loefflad-2026-criterial-features-german-interpretable-nlp/)
- [next → Critical-CoT: A Robust Defense Framework against Reasoning-Level Backdoor Attacks in Large Language Models](https://md.research.elamur.ai/free/en/papers/truong-2026-critical-cot-robust-defense-framework/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/26/)

## References

21 references, 2 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Rakesh Agrawal, Tomasz Imieliński, and Arun Swami. 1993. Mining association rules between sets of items in large databases. In Proceedings of the 1993 ACM SIGMOD international conference on Management of data, pages 207–216. [find](https://scholar.google.com/scholar?q=Rakesh+Agrawal%2C+Tomasz+Imielin%CC%81ski%2C+and+Arun+Swami%2E+1993%2E+Mining+association+rules+between+sets+of+items+in+large+databases%2E+In+Proceedings+of+the+1993+ACM+SIGMOD+international+conference+on+Managemen)
2. Long Bai, Zixuan Li, Xiaolong Jin, Jiafeng Guo, Xueqi Cheng, and Tat-Seng Chua. 2025. G2S: A general-to-specific learning framework for temporal knowledge graph forecasting with large language models. In Findings of the Association for Computational Linguistics: ACL 2025, pages 20927–20938, Vienna,… [find](https://scholar.google.com/scholar?q=Long+Bai%2C+Zixuan+Li%2C+Xiaolong+Jin%2C+Jiafeng+Guo%2C+Xueqi+Cheng%2C+and+Tat%2DSeng+Chua%2E+2025%2E+G2S%3A+A+general%2Dto%2Dspecific+learning+framework+for+temporal+knowledge+graph+forecasting+with+large+language+models%2E)
3. Borui Cai, Yong Xiang, Longxiang Gao, He Zhang, Yunfeng Li, and Jianxin Li. 2023. Temporal knowledge graph completion: a survey. In Proceedings of the Thirty-Second International Joint Conference on Artificial Intelligence, pages 6545–6553. [find](https://scholar.google.com/scholar?q=Borui+Cai%2C+Yong+Xiang%2C+Longxiang+Gao%2C+He+Zhang%2C+Yunfeng+Li%2C+and+Jianxin+Li%2E+2023%2E+Temporal+knowledge+graph+completion%3A+a+survey%2E+In+Proceedings+of+the+Thirty%2DSecond+International+Joint+Conference+on+A)
4. Alberto García-Durán, Sebastijan Dumančić, and Mathias Niepert. 2018. Learning sequence encoders for temporal knowledge graph completion. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 4816–4821, Brussels, Belgium. Association for Computational Lin… [find](https://scholar.google.com/scholar?q=Alberto+Garc%C3%ADa%2DDur%C3%A1n%2C+Sebastijan+Dumanc%CC%8Cic%CC%81%2C+and+Mathias+Niepert%2E+2018%2E+Learning+sequence+encoders+for+temporal+knowledge+graph+completion%2E+In+Proceedings+of+the+2018+Conference+on+Empirical+Methods+i)
5. Zhen Han, Peng Chen, Yunpu Ma, and Volker Tresp. 2020. Explainable subgraph reasoning for forecasting on temporal knowledge graphs. In International conference on learning representations. [find](https://scholar.google.com/scholar?q=Zhen+Han%2C+Peng+Chen%2C+Yunpu+Ma%2C+and+Volker+Tresp%2E+2020%2E+Explainable+subgraph+reasoning+for+forecasting+on+temporal+knowledge+graphs%2E+In+International+conference+on+learning+representations%2E)
6. Woojeong Jin, Meng Qu, Xisen Jin, and Xiang Ren. 2020. Recurrent event network: Autoregressive structure inferenceover temporal knowledge graphs. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 6669–6683, Online. Association for Computational… [find](https://scholar.google.com/scholar?q=Woojeong+Jin%2C+Meng+Qu%2C+Xisen+Jin%2C+and+Xiang+Ren%2E+2020%2E+Recurrent+event+network%3A+Autoregressive+structure+inferenceover+temporal+knowledge+graphs%2E+In+Proceedings+of+the+2020+Conference+on+Empirical+Met)
7. Meelis Kull, Telmo Silva Filho, and Peter Flach. 2017. Beta calibration: a well-founded and easily implemented improvement on logistic calibration for binary classifiers. In Artificial intelligence and statistics, pages 623–631. PMLR. [find](https://scholar.google.com/scholar?q=Meelis+Kull%2C+Telmo+Silva+Filho%2C+and+Peter+Flach%2E+2017%2E+Beta+calibration%3A+a+well%2Dfounded+and+easily+implemented+improvement+on+logistic+calibration+for+binary+classifiers%2E+In+Artificial+intelligence+an)
8. Dong-Ho Lee, Kian Ahrabian, Woojeong Jin, Fred Morstatter, and Jay Pujara. 2023. Temporal knowledge graph forecasting without knowledge using in-context learning. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 544–557, Singapore. Association for Comp… [find](https://scholar.google.com/scholar?q=Dong%2DHo+Lee%2C+Kian+Ahrabian%2C+Woojeong+Jin%2C+Fred+Morstatter%2C+and+Jay+Pujara%2E+2023%2E+Temporal+knowledge+graph+forecasting+without+knowledge+using+in%2Dcontext+learning%2E+In+Proceedings+of+the+2023+Conference)
9. Ningyuan Li, E Haihong, Shi Li, Mingzhi Sun, Tianyu Yao, Meina Song, Yong Wang, and Haoran Luo. 2023. Tr-rules: Rule-based model for link forecasting on temporal knowledge graph considering temporal redundancy. In Findings of the association for computational linguistics: EMNLP 2023, pages 7885–7894… [find](https://scholar.google.com/scholar?q=Ningyuan+Li%2C+E+Haihong%2C+Shi+Li%2C+Mingzhi+Sun%2C+Tianyu+Yao%2C+Meina+Song%2C+Yong+Wang%2C+and+Haoran+Luo%2E+2023%2E+Tr%2Drules%3A+Rule%2Dbased+model+for+link+forecasting+on+temporal+knowledge+graph+considering+temporal+r)
10. Yujia Li, Shiliang Sun, and Jing Zhao. 2022. Tirgn: Time-guided recurrent graph network with local-global historical patterns for temporal knowledge graph reasoning. In Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence, IJCAI-22, pages 2152–2158. International… [find](https://scholar.google.com/scholar?q=Yujia+Li%2C+Shiliang+Sun%2C+and+Jing+Zhao%2E+2022%2E+Tirgn%3A+Time%2Dguided+recurrent+graph+network+with+local%2Dglobal+historical+patterns+for+temporal+knowledge+graph+reasoning%2E+In+Proceedings+of+the+Thirty%2DFirst)
11. Zixuan Li, Xiaolong Jin, Wei Li, Saiping Guan, Jiafeng Guo, Huawei Shen, Yuanzhuo Wang, and Xueqi Cheng. 2021. Temporal knowledge graph reasoning based on evolutional representation learning. In Proceedings of the 44th international ACM SIGIR conference on research and development in information ret… [find](https://scholar.google.com/scholar?q=Zixuan+Li%2C+Xiaolong+Jin%2C+Wei+Li%2C+Saiping+Guan%2C+Jiafeng+Guo%2C+Huawei+Shen%2C+Yuanzhuo+Wang%2C+and+Xueqi+Cheng%2E+2021%2E+Temporal+knowledge+graph+reasoning+based+on+evolutional+representation+learning%2E+In+Proce)
12. Ruotong Liao, Xu Jia, Yangzhe Li, Yunpu Ma, and Volker Tresp. 2024. GenTKG: Generative forecasting on temporal knowledge graph with large language models. In Findings of the Association for Computational Linguistics: NAACL 2024, pages 4303–4317, Mexico City, Mexico. Association for Computational Lin… [find](https://scholar.google.com/scholar?q=Ruotong+Liao%2C+Xu+Jia%2C+Yangzhe+Li%2C+Yunpu+Ma%2C+and+Volker+Tresp%2E+2024%2E+GenTKG%3A+Generative+forecasting+on+temporal+knowledge+graph+with+large+language+models%2E+In+Findings+of+the+Association+for+Computatio)
13. Yushan Liu, Yunpu Ma, Marcel Hildebrandt, Mitchell Joblin, and Volker Tresp. 2022. Tlogic: Temporal logical rules for explainable link forecasting on temporal knowledge graphs. In Proceedings of the AAAI conference on artificial intelligence, volume 36, pages 4120–4127. [find](https://scholar.google.com/scholar?q=Yushan+Liu%2C+Yunpu+Ma%2C+Marcel+Hildebrandt%2C+Mitchell+Joblin%2C+and+Volker+Tresp%2E+2022%2E+Tlogic%3A+Temporal+logical+rules+for+explainable+link+forecasting+on+temporal+knowledge+graphs%2E+In+Proceedings+of+the+A)
14. Linhao Luo, Jiaxin Ju, Bo Xiong, Yuan-Fang Li, Gholamreza Haffari, and Shirui Pan. 2025. Chatrule: Mining logical rules with large language models for knowledge graph reasoning. In Pacific-asia conference on knowledge discovery and data mining, pages 314–325. Springer. [find](https://scholar.google.com/scholar?q=Linhao+Luo%2C+Jiaxin+Ju%2C+Bo+Xiong%2C+Yuan%2DFang+Li%2C+Gholamreza+Haffari%2C+and+Shirui+Pan%2E+2025%2E+Chatrule%3A+Mining+logical+rules+with+large+language+models+for+knowledge+graph+reasoning%2E+In+Pacific%2Dasia+confer)
15. Nils Reimers and Iryna Gurevych. 2019. [Sentence-BERT: Sentence embeddings using Siamese BERT-networks](https://md.research.elamur.ai/free/en/papers/reimers-2019-sentence-bert-sentence-embeddings-siamese/). In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 3982–3992,…
16. Guo Tang, Zheng Chu, Wenxiang Zheng, Junjia Xiang, Yizhuo Li, Weihao Zhang, Ming Liu, and Bing Qin. 2025. AnRe: Analogical replay for temporal knowledge graph forecasting. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 4632–4… [find](https://scholar.google.com/scholar?q=Guo+Tang%2C+Zheng+Chu%2C+Wenxiang+Zheng%2C+Junjia+Xiang%2C+Yizhuo+Li%2C+Weihao+Zhang%2C+Ming+Liu%2C+and+Bing+Qin%2E+2025%2E+AnRe%3A+Analogical+replay+for+temporal+knowledge+graph+forecasting%2E+In+Proceedings+of+the+63rd+A)
17. Jiapu Wang, Kai Sun, Linhao Luo, Wei Wei, Yongli Hu, Alan W Liew, Shirui Pan, and Baocai Yin. 2024. Large language models-guided dynamic adaptation for temporal knowledge graph reasoning. Advances in Neural Information Processing Systems, 37:8384– 8410. [find](https://scholar.google.com/scholar?q=Jiapu+Wang%2C+Kai+Sun%2C+Linhao+Luo%2C+Wei+Wei%2C+Yongli+Hu%2C+Alan+W+Liew%2C+Shirui+Pan%2C+and+Baocai+Yin%2E+2024%2E+Large+language+models%2Dguided+dynamic+adaptation+for+temporal+knowledge+graph+reasoning%2E+Advances+in+)
18. Yuwei Xia, Ding Wang, Qiang Liu, Liang Wang, Shu Wu, and Xiaoyu Zhang. 2024. Chain-of-history reasoning for temporal knowledge graph forecasting. Preprint, arXiv:2402.14382. [find](https://scholar.google.com/scholar?q=Yuwei+Xia%2C+Ding+Wang%2C+Qiang+Liu%2C+Liang+Wang%2C+Shu+Wu%2C+and+Xiaoyu+Zhang%2E+2024%2E+Chain%2Dof%2Dhistory+reasoning+for+temporal+knowledge+graph+forecasting%2E+Preprint%2C+arXiv%3A2402%2E14382%2E)
19. Wenjie Xu, Ben Liu, Miao Peng, Xu Jia, and Min Peng. 2023. Pre-trained language model with prompts for temporal knowledge graph completion. In Findings of the Association for Computational Linguistics: ACL 2023, pages 7790–7803, Toronto, Canada. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Wenjie+Xu%2C+Ben+Liu%2C+Miao+Peng%2C+Xu+Jia%2C+and+Min+Peng%2E+2023%2E+Pre%2Dtrained+language+model+with+prompts+for+temporal+knowledge+graph+completion%2E+In+Findings+of+the+Association+for+Computational+Linguistics)
20. Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric Xing, and 1 others. 2023. [Judging llm-as-a-judge with mt-bench and chatbot arena](https://md.research.elamur.ai/free/en/papers/zheng-2023-judging-llm-judge-mt-bench/). Advances in neural information processing systems, 36:46595–46623.
21. Cunchao Zhu, Muhao Chen, Changjun Fan, Guangquan Cheng, and Yan Zhang. 2021. Learning from history: Modeling temporal knowledge graphs with sequential copy-generation networks. In Proceedings of the AAAI conference on artificial intelligence, volume 35, pages 4732–4740. [find](https://scholar.google.com/scholar?q=Cunchao+Zhu%2C+Muhao+Chen%2C+Changjun+Fan%2C+Guangquan+Cheng%2C+and+Yan+Zhang%2E+2021%2E+Learning+from+history%3A+Modeling+temporal+knowledge+graphs+with+sequential+copy%2Dgeneration+networks%2E+In+Proceedings+of+the+A)

*Updated 1 November 2026*