---
title: "Chain of History — paper by Luo et al. (2024)"
description: "Their pipeline builds each query's history in two steps. 54 references, cited by 17 papers here."
authors: ["Ruilin Luo", "Tianle Gu", "Haoling Li", "Junzhe Li", "Zicheng Lin", "Jiayi Li", "Yujiu Yang"]
year: 2024
venue: "arXiv:2401.06072"
source: "https://arxiv.org/abs/2401.06072"
doi: "10.48550/arxiv.2401.06072"
arxiv: "2401.06072"
licence: "CC BY 4.0 (arXiv)"
licence_url: "http://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/luo-2024-chain-history-learning-forecasting-llms/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/luo-2024-chain-history-learning-forecasting-llms/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2024](https://md.research.elamur.ai/free/en/papers/2024/page/11/) / Chain of History: Learning and Forecasting with LLMs for Temporal Knowledge Graph Completion

# Chain of History: Learning and Forecasting with LLMs for Temporal Knowledge Graph Completion

**Authors:** Ruilin Luo, [Tianle Gu](https://md.research.elamur.ai/free/en/authors/tianle-gu/), Haoling Li, Junzhe Li, Zicheng Lin, [Jiayi Li](https://md.research.elamur.ai/free/en/authors/jiayi-li/) and [Yujiu Yang](https://md.research.elamur.ai/free/en/authors/yujiu-yang/) · **Year:** 2024 · **Venue:** arXiv:2401.06072 · **Licence:** [CC BY 4.0 (arXiv)](http://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://arxiv.org/abs/2401.06072) · [PDF](https://arxiv.org/pdf/2401.06072) · [DOI 10.48550/arxiv.2401.06072](https://doi.org/10.48550/arxiv.2401.06072) · [arXiv 2401.06072](https://arxiv.org/abs/2401.06072) · [OpenAlex](https://openalex.org/W4390833674)

**Citations:** 8 citations (OpenAlex, 2026-10-14) · [17](https://md.research.elamur.ai/free/en/papers/luo-2024-chain-history-learning-forecasting-llms/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
The paper addresses extrapolative Temporal Knowledge Graph Completion (TKGC). Given time-stamped quadruples observed up to some point, the goal is to predict the missing entity in events at later timestamps. Earlier approaches based on GNNs, rules or reinforcement learning make little use of textual information and struggle when links are sparse. The authors instead treat the task as text generation: a large language model receives a chronological chain of past facts and outputs the answer.

Their pipeline builds each query's history in two steps. It first takes facts that share the query's subject and relation. If the history is too short, it adds neighbouring facts about the entity and then about the relation, favouring recent timestamps. To counter the reversal curse, the training data also includes inverse queries, tested in several prompt formats. Llama-2-7b is adapted with parameter-efficient fine-tuning, producing Llama-2-7b-CoH.

On ICEWS14, ICEWS05-15, ICEWS18 and YAGO, the model matches or outperforms prior methods on Hits@n. The work suggests that careful data design helps LLMs reason over temporal graphs more than enlarging the model does.

## Key points
- History has three sources: schema-matching history, entity-augmented history and relation-augmented history. Inverse queries are encoded with three prompt strategies: Ordinary, Text-aware and Position-aware.
- Results are reported as Hits@n under both raw and time-aware filtered settings. 8-shot ICL baselines from open-source models are included for reference.
- Model size has little effect. Llama-2-13b and Vicuna-33b reach Hits@1 close to the 7b model, and on ICEWS18 they score 3.7% and 0.9% lower, respectively.
- Commercial LLMs were tested with 8-shot ICL on 1000 random test queries per dataset, using exact match (raw Hits@1).
  - Qwen-72B-Chat roughly equals or exceeds GPT-4.
  - On ICEWS14, overall scores are 0.248 for Qwen-72B-Chat, 0.266 for GPT-4 and 0.209 for GPT-3.5-turbo.
  - All three trail Llama-2-7b-CoH on the ICEWS datasets but are close to it on YAGO, which is biased toward common knowledge.
- The average schema-matching history length per test query is 30.05 on ICEWS14 and 56.95 on ICEWS05-15. Requiring histories much longer than this adds padding facts that can distract the model.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Tsinghua University](https://md.research.elamur.ai/free/en/universities/tsinghua-university/) — University, Beijing, China · 439 works · 94,783 citations
- [Peking University](https://md.research.elamur.ai/free/en/universities/peking-university/) — University, Beijing, China · 335 works · 43,758 citations
- [Baidu](https://md.research.elamur.ai/free/en/companies/baidu/) — Company, Beijing, China · 87 works · 17,840 citations
- [Tsinghua Shenzhen International Graduate School](https://md.research.elamur.ai/free/en/universities/tsinghua-shenzhen-international-graduate-school/) — University, Shenzhen, China · 30 works · 3,060 citations

📚 Shelves (5): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 19,503 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 18,805 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 17,959 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 17,789 works · [🗄️ Information Systems](https://md.research.elamur.ai/free/en/shelves/computer-science/information-systems/) · 1,888 works

🧭 Topics (1): [🕸️ Knowledge graphs & ontologies](https://md.research.elamur.ai/free/en/topics/knowledge-graphs/) · 218 works

## Cite

Cite as: Ruilin Luo, Tianle Gu, Haoling Li, Junzhe Li, Zicheng Lin, Jiayi Li and Yujiu Yang. 2024. Chain of History: Learning and Forecasting with LLMs for Temporal Knowledge Graph Completion. arXiv:2401.06072.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@misc{luo2024chain,
  title = {{Chain of History: Learning and Forecasting with LLMs for Temporal Knowledge Graph Completion}},
  author = {Ruilin Luo, Tianle Gu, Haoling Li, Junzhe Li, Zicheng Lin, Jiayi Li and Yujiu Yang},
  year = {2024},
  doi = {10.48550/arxiv.2401.06072},
  eprint = {2401.06072},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2401.06072}
}
```

- [← previous What Makes Math Word Problems Challenging for LLMs?](https://md.research.elamur.ai/free/en/papers/srivatsa-2024-what-makes-math-word-problems/)
- [next → CiteME: Can Language Models Accurately Cite Scientific Claims?](https://md.research.elamur.ai/free/en/papers/press-2024-citeme-language-models-accurately/)
- [↑ back to papers of 2024](https://md.research.elamur.ai/free/en/papers/2024/page/11/)

## References

54 references, 16 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Jiayan Guo, Lun Du, and Hengyu Liu. 2023a. [Gpt4graph: Can large language models understand graph structured data ? an empirical evaluation and benchmarking](https://md.research.elamur.ai/free/en/papers/guo-2023-gpt4graph-large-language-models-understand/). CoRR, abs/2305.15066.
2. Jinze Bai, Shuai Bai, Yunfei Chu, Zeyu Cui, Kai Dang, Xiaodong Deng, Yang Fan, Wenbin Ge, Yu Han, Fei Huang, Binyuan Hui, Luo Ji, Mei Li, Junyang Lin, Runji Lin, Dayiheng Liu, Gao Liu, Chengqiang Lu, Keming Lu, Jianxin Ma, Rui Men, Xingzhang Ren, Xuancheng Ren, Chuanqi Tan, Sinan Tan, Jianhong Tu, P… — [Qwen Technical Report](https://md.research.elamur.ai/free/en/papers/bai-2023-qwen-technical-report/)
3. Ankur Bapna, Naveen Arivazhagan, and Orhan Firat. 2019. [Simple, scalable adaptation for neural machine translation](https://md.research.elamur.ai/free/en/papers/bapna-2019-simple-scalable-adaptation-neural-machine/).
4. Lukas Berglund, Meg Tong, Max Kaufmann, Mikita Balesni, Asa Cooper Stickland, Tomasz Korbak, and Owain Evans. 2023. The reversal curse: Llms trained on "a is b" fail to learn "b is a". CoRR, abs/2309.12288. [find](https://scholar.google.com/scholar?q=Lukas+Berglund%2C+Meg+Tong%2C+Max+Kaufmann%2C+Mikita+Balesni%2C+Asa+Cooper+Stickland%2C+Tomasz+Korbak%2C+and+Owain+Evans%2E+2023%2E+The+reversal+curse%3A+Llms+trained+on+%22a+is+b%22+fail+to+learn+%22b+is+a%22%2E+CoRR%2C+abs%2F2309%2E)
5. Sid Black, Stella Biderman, Eric Hallahan, Quentin Anthony, Leo Gao, Laurence Golding, Horace He, Connor Leahy, Kyle McDonell, Jason Phang, Michael Pieler, USVSN Sai Prashanth, Shivanshu Purohit, Laria Reynolds, Jonathan Tow, Ben Wang, and Samuel Weinbach. 2022. [Gpt-neox-20b: An open-source autoregr…](https://md.research.elamur.ai/free/en/papers/black-2022-gpt-neox-20b-open-source/)
6. Ziwei Chai, Tianjie Zhang, Liang Wu, Kaiqiao Han, Xiaohai Hu, Xuanwen Huang, and Yang Yang. 2023. Graphllm: Boosting graph reasoning ability of large language model. [find](https://scholar.google.com/scholar?q=Ziwei+Chai%2C+Tianjie+Zhang%2C+Liang+Wu%2C+Kaiqiao+Han%2C+Xiaohai+Hu%2C+Xuanwen+Huang%2C+and+Yang+Yang%2E+2023%2E+Graphllm%3A+Boosting+graph+reasoning+ability+of+large+language+model%2E)
7. Linlin Chao, Jianshan He, Taifeng Wang, and Wei Chu. 2021. Pairre: Knowledge graph embeddings via paired relation vectors. In ACL/IJCNLP(1), pages 4360–4369. [find](https://scholar.google.com/scholar?q=Linlin+Chao%2C+Jianshan+He%2C+Taifeng+Wang%2C+and+Wei+Chu%2E+2021%2E+Pairre%3A+Knowledge+graph+embeddings+via+paired+relation+vectors%2E+In+ACL%2FIJCNLP%281%29%2C+pages+4360%E2%80%934369%2E)
8. Junyoung Chung, Çaglar Gülçehre, KyungHyun Cho, and Yoshua Bengio. 2014. Empirical evaluation of gated recurrent neural networks on sequence modeling. CoRR, abs/1412.3555. [find](https://scholar.google.com/scholar?q=Junyoung+Chung%2C+%C3%87aglar+G%C3%BCl%C3%A7ehre%2C+KyungHyun+Cho%2C+and+Yoshua+Bengio%2E+2014%2E+Empirical+evaluation+of+gated+recurrent+neural+networks+on+sequence+modeling%2E+CoRR%2C+abs%2F1412%2E3555%2E)
9. Zifeng Ding, Zhen Han, Yunpu Ma, and Volker Tresp. 2021. Temporal knowledge graph forecasting with neural ODE. CoRR, abs/2101.05151. — [arXiv:2101.05151](https://arxiv.org/abs/2101.05151) [find](https://scholar.google.com/scholar?q=Zifeng+Ding%2C+Zhen+Han%2C+Yunpu+Ma%2C+and+Volker+Tresp%2E+2021%2E+Temporal+knowledge+graph+forecasting+with+neural+ODE%2E+CoRR%2C+abs%2F2101%2E05151%2E+%E2%80%94+%5BarXiv%3A2101%2E05151%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2101%2E05151%29)
10. Chao Feng, Xinyu Zhang, and Zichu Fei. 2023. Knowledge solver: Teaching llms to search for domain knowledge from knowledge graphs. CoRR, abs/2309.03118. — [arXiv:2309.03118](https://arxiv.org/abs/2309.03118) [find](https://scholar.google.com/scholar?q=Chao+Feng%2C+Xinyu+Zhang%2C+and+Zichu+Fei%2E+2023%2E+Knowledge+solver%3A+Teaching+llms+to+search+for+domain+knowledge+from+knowledge+graphs%2E+CoRR%2C+abs%2F2309%2E03118%2E+%E2%80%94+%5BarXiv%3A2309%2E03118%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2309)
11. Jiayan Guo, Lun Du, Hengyu Liu, Mengyu Zhou, Xinyi He, and Shi Han. 2023b. [Gpt4graph: Can large language models understand graph structured data ? an empirical evaluation and benchmarking](https://md.research.elamur.ai/free/en/papers/guo-2023-gpt4graph-large-language-models-understand/).
12. Zhen Han, Peng Chen, Yunpu Ma, and Volker Tresp. 2021a. Explainable subgraph reasoning for forecasting on temporal knowledge graphs. In 9th International Conference on Learning Representations, ICLR 2021, Virtual Event, Austria, May 3-7, 2021. OpenReview.net. [find](https://scholar.google.com/scholar?q=Zhen+Han%2C+Peng+Chen%2C+Yunpu+Ma%2C+and+Volker+Tresp%2E+2021a%2E+Explainable+subgraph+reasoning+for+forecasting+on+temporal+knowledge+graphs%2E+In+9th+International+Conference+on+Learning+Representations%2C+ICLR+2)
13. Zhen Han, Zifeng Ding, Yunpu Ma, Yujia Gu, and Volker Tresp. 2021b. Learning neural ordinary equations for forecasting future links on temporal knowledge graphs. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, EMNLP 2021, Virtual Event / Punta Cana, Dominic… [find](https://scholar.google.com/scholar?q=Zhen+Han%2C+Zifeng+Ding%2C+Yunpu+Ma%2C+Yujia+Gu%2C+and+Volker+Tresp%2E+2021b%2E+Learning+neural+ordinary+equations+for+forecasting+future+links+on+temporal+knowledge+graphs%2E+In+Proceedings+of+the+2021+Conference+)
14. Junxian He, Chunting Zhou, Xuezhe Ma, Taylor Berg-Kirkpatrick, and Graham Neubig. 2022. [Towards a unified view of parameter-efficient transfer learning](https://md.research.elamur.ai/free/en/papers/he-2021-towards-unified-view-parameter-efficient/).
15. Neil Houlsby, Andrei Giurgiu, Stanislaw Jastrzebski, Bruna Morrone, Quentin de Laroussilhe, Andrea Gesmundo, Mona Attariyan, and Sylvain Gelly. 2019. Parameter-efficient transfer learning for nlp. [find](https://scholar.google.com/scholar?q=Neil+Houlsby%2C+Andrei+Giurgiu%2C+Stanislaw+Jastrzebski%2C+Bruna+Morrone%2C+Quentin+de+Laroussilhe%2C+Andrea+Gesmundo%2C+Mona+Attariyan%2C+and+Sylvain+Gelly%2E+2019%2E+Parameter%2Defficient+transfer+learning+for+nlp%2E)
16. Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2021. [Lora: Low-rank adaptation of large language models](https://md.research.elamur.ai/free/en/papers/hu-2021-lora-low-rank-adaptation-large/).
17. Prachi Jain, Sushant Rathi, Mausam, and Soumen Chakrabarti. 2020. Temporal knowledge base completion: New algorithms and evaluation protocols. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, EMNLP 2020, Online, November 16-20, 2020, pages 3733– 3747. Associ… [find](https://scholar.google.com/scholar?q=Prachi+Jain%2C+Sushant+Rathi%2C+Mausam%2C+and+Soumen+Chakrabarti%2E+2020%2E+Temporal+knowledge+base+completion%3A+New+algorithms+and+evaluation+protocols%2E+In+Proceedings+of+the+2020+Conference+on+Empirical+Method)
18. Woojeong Jin, Meng Qu, Xisen Jin, and Xiang Ren. 2020. Recurrent event network: Autoregressive structure inferenceover temporal knowledge graphs. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, EMNLP 2020, Online, November 16-20, 2020, pages 6669– 6683. Ass… [find](https://scholar.google.com/scholar?q=Woojeong+Jin%2C+Meng+Qu%2C+Xisen+Jin%2C+and+Xiang+Ren%2E+2020%2E+Recurrent+event+network%3A+Autoregressive+structure+inferenceover+temporal+knowledge+graphs%2E+In+Proceedings+of+the+2020+Conference+on+Empirical+Met)
19. Timothée Lacroix, Guillaume Obozinski, and Nicolas Usunier. 2020. Tensor decompositions for temporal knowledge base completion. In 8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020. OpenReview.net. [find](https://scholar.google.com/scholar?q=Timoth%C3%A9e+Lacroix%2C+Guillaume+Obozinski%2C+and+Nicolas+Usunier%2E+2020%2E+Tensor+decompositions+for+temporal+knowledge+base+completion%2E+In+8th+International+Conference+on+Learning+Representations%2C+ICLR+2020%2C+)
20. Julien Leblay and Melisachew Wudage Chekol. 2018. Deriving validity time in knowledge graph. In Companion Proceedings of the The Web Conference 2018, pages 1771–1776. [find](https://scholar.google.com/scholar?q=Julien+Leblay+and+Melisachew+Wudage+Chekol%2E+2018%2E+Deriving+validity+time+in+knowledge+graph%2E+In+Companion+Proceedings+of+the+The+Web+Conference+2018%2C+pages+1771%E2%80%931776%2E)
21. Dong-Ho Lee, Kian Ahrabian, Woojeong Jin, Fred Morstatter, and Jay Pujara. 2023a. Temporal knowledge graph forecasting without knowledge using in-context learning. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, EMNLP 2023, Singapore, December 6-10, 2023, p… [find](https://scholar.google.com/scholar?q=Dong%2DHo+Lee%2C+Kian+Ahrabian%2C+Woojeong+Jin%2C+Fred+Morstatter%2C+and+Jay+Pujara%2E+2023a%2E+Temporal+knowledge+graph+forecasting+without+knowledge+using+in%2Dcontext+learning%2E+In+Proceedings+of+the+2023+Conferenc)
22. Dong-Ho Lee, Kian Ahrabian, Woojeong Jin, Fred Morstatter, and Jay Pujara. 2023b. Temporal knowledge graph forecasting without knowledge using in-context learning. arXiv preprint arXiv:2305.10613. [find](https://scholar.google.com/scholar?q=Dong%2DHo+Lee%2C+Kian+Ahrabian%2C+Woojeong+Jin%2C+Fred+Morstatter%2C+and+Jay+Pujara%2E+2023b%2E+Temporal+knowledge+graph+forecasting+without+knowledge+using+in%2Dcontext+learning%2E+arXiv+preprint+arXiv%3A2305%2E10613%2E)
23. Brian Lester, Rami Al-Rfou, and Noah Constant. 2021. [The power of scale for parameter-efficient prompt tuning](https://md.research.elamur.ai/free/en/papers/lester-2021-power-scale-parameter-efficient-prompt/).
24. Xiang Lisa Li and Percy Liang. 2021. Prefix-tuning: Optimizing continuous prompts for generation. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 4582–… [find](https://scholar.google.com/scholar?q=Xiang+Lisa+Li+and+Percy+Liang%2E+2021%2E+Prefix%2Dtuning%3A+Optimizing+continuous+prompts+for+generation%2E+In+Proceedings+of+the+59th+Annual+Meeting+of+the+Association+for+Computational+Linguistics+and+the+11t)
25. Yujia Li, Shiliang Sun, and Jing Zhao. 2022. Tirgn: Time-guided recurrent graph network with local-global historical patterns for temporal knowledge graph reasoning. In Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence, IJCAI 2022, Vienna, Austria, 23-29 July… [find](https://scholar.google.com/scholar?q=Yujia+Li%2C+Shiliang+Sun%2C+and+Jing+Zhao%2E+2022%2E+Tirgn%3A+Time%2Dguided+recurrent+graph+network+with+local%2Dglobal+historical+patterns+for+temporal+knowledge+graph+reasoning%2E+In+Proceedings+of+the+Thirty%2DFirst)
26. Ruotong Liao, Xu Jia, Yunpu Ma, and Volker Tresp. 2023. Gentkg: Generative forecasting on temporal knowledge graph. CoRR, abs/2310.07793. — [arXiv:2310.07793](https://arxiv.org/abs/2310.07793) [find](https://scholar.google.com/scholar?q=Ruotong+Liao%2C+Xu+Jia%2C+Yunpu+Ma%2C+and+Volker+Tresp%2E+2023%2E+Gentkg%3A+Generative+forecasting+on+temporal+knowledge+graph%2E+CoRR%2C+abs%2F2310%2E07793%2E+%E2%80%94+%5BarXiv%3A2310%2E07793%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2310%2E07793%29)
27. Hao Liu, Jiarui Feng, Lecheng Kong, Ningyue Liang, Dacheng Tao, Yixin Chen, and Muhan Zhang. 2023a. One for all: Towards training one graph model for all classification tasks. CoRR, abs/2310.00149. — [arXiv:2310.00149](https://arxiv.org/abs/2310.00149) [find](https://scholar.google.com/scholar?q=Hao+Liu%2C+Jiarui+Feng%2C+Lecheng+Kong%2C+Ningyue+Liang%2C+Dacheng+Tao%2C+Yixin+Chen%2C+and+Muhan+Zhang%2E+2023a%2E+One+for+all%3A+Towards+training+one+graph+model+for+all+classification+tasks%2E+CoRR%2C+abs%2F2310%2E00149%2E+%E2%80%94+)
28. Pei Liu, Bing Qian, Qi Sun, and Longgang Zhao. 2023b. Prompt-wnqa: A prompt-based complex question answering for wireless network over knowledge graph. Comput. Networks, 236:110014. [find](https://scholar.google.com/scholar?q=Pei+Liu%2C+Bing+Qian%2C+Qi+Sun%2C+and+Longgang+Zhao%2E+2023b%2E+Prompt%2Dwnqa%3A+A+prompt%2Dbased+complex+question+answering+for+wireless+network+over+knowledge+graph%2E+Comput%2E+Networks%2C+236%3A110014%2E)
29. Yushan Liu, Yunpu Ma, Marcel Hildebrandt, Mitchell Joblin, and Volker Tresp. 2022. Tlogic: Temporal logical rules for explainable link forecasting on temporal knowledge graphs. In Thirty-Sixth AAAI Conference on Artificial Intelligence, AAAI 2022, Thirty-Fourth Conference on Innovative Applications… [find](https://scholar.google.com/scholar?q=Yushan+Liu%2C+Yunpu+Ma%2C+Marcel+Hildebrandt%2C+Mitchell+Joblin%2C+and+Volker+Tresp%2E+2022%2E+Tlogic%3A+Temporal+logical+rules+for+explainable+link+forecasting+on+temporal+knowledge+graphs%2E+In+Thirty%2DSixth+AAAI+Co)
30. Linhao Luo, Yuan-Fang Li, Gholamreza Haffari, and Shirui Pan. 2023. Reasoning on graphs: Faithful and interpretable large language model reasoning. CoRR, abs/2310.01061. [find](https://scholar.google.com/scholar?q=Linhao+Luo%2C+Yuan%2DFang+Li%2C+Gholamreza+Haffari%2C+and+Shirui+Pan%2E+2023%2E+Reasoning+on+graphs%3A+Faithful+and+interpretable+large+language+model+reasoning%2E+CoRR%2C+abs%2F2310%2E01061%2E)
31. Ang Lv, Kaiyi Zhang, Shufang Xie, Quan Tu, Yuhan Chen, Ji-Rong Wen, and Rui Yan. 2023. Are we falling in a middle-intelligence trap? an analysis and mitigation of the reversal curse. CoRR, abs/2311.07468. [find](https://scholar.google.com/scholar?q=Ang+Lv%2C+Kaiyi+Zhang%2C+Shufang+Xie%2C+Quan+Tu%2C+Yuhan+Chen%2C+Ji%2DRong+Wen%2C+and+Rui+Yan%2E+2023%2E+Are+we+falling+in+a+middle%2Dintelligence+trap%3F+an+analysis+and+mitigation+of+the+reversal+curse%2E+CoRR%2C+abs%2F2311%2E07)
32. Farzaneh Mahdisoltani, Joanna Biega, and Fabian M. Suchanek. 2015. [YAGO3: A knowledge base from multilingual wikipedias](https://md.research.elamur.ai/free/en/papers/mahdisoltani-2015-yago3-knowledge-base-multilingual-wikipedias/). In Seventh Biennial Conference on Innovative Data Systems Research, CIDR 2015, Asilomar, CA, USA, January 4-7, 2015, Online Proceedings. www.cidrdb.org.
33. OpenAI. 2023. [GPT-4 technical report](https://md.research.elamur.ai/free/en/papers/openai-2023-gpt-4-technical-report/). abs/2303.08774.
34. CoRR, Zixuan Li, Xiaolong Jin, Wei Li, Saiping Guan, Jiafeng Guo, Huawei Shen, Yuanzhuo Wang, and Xueqi Cheng. 2021a. Temporal knowledge graph reasoning based on evolutional representation learning. In SIGIR ’21: The 44th International ACM SIGIR Conference on Research and Development in Information… [find](https://scholar.google.com/scholar?q=CoRR%2C+Zixuan+Li%2C+Xiaolong+Jin%2C+Wei+Li%2C+Saiping+Guan%2C+Jiafeng+Guo%2C+Huawei+Shen%2C+Yuanzhuo+Wang%2C+and+Xueqi+Cheng%2E+2021a%2E+Temporal+knowledge+graph+reasoning+based+on+evolutional+representation+learning%2E+I)
35. Long Ouyang, Jeff Wu, Xu Jiang, Diogo Almeida, Carroll L. Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, John Schulman, Jacob Hilton, Fraser Kelton, Luke Miller, Maddie Simens, Amanda Askell, Peter Welinder, Paul Christiano, Jan Leike, and Ryan Lowe. 2022. [Train…](https://md.research.elamur.ai/free/en/papers/ouyang-2022-training-language-models-follow-instructions/)
36. Zixuan Li, Xiaolong Jin, Wei Li, Saiping Guan, Jiafeng Guo, Huawei Shen, Yuanzhuo Wang, and Xueqi Cheng. 2021b. Temporal knowledge graph reasoning based on evolutional representation learning. In SIGIR ’21: The 44th International ACM SIGIR Conference on Research and Development in Information Retrie… [find](https://scholar.google.com/scholar?q=Zixuan+Li%2C+Xiaolong+Jin%2C+Wei+Li%2C+Saiping+Guan%2C+Jiafeng+Guo%2C+Huawei+Shen%2C+Yuanzhuo+Wang%2C+and+Xueqi+Cheng%2E+2021b%2E+Temporal+knowledge+graph+reasoning+based+on+evolutional+representation+learning%2E+In+SIGI)
37. Chengwen Qi, Bowen Li, Binyuan Hui, Bailin Wang, Jinyang Li, Jinwang Wu, and Yuanjun Laili. 2023. An investigation of llms’ inefficacy in understanding converse relations. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, EMNLP 2023, Singapore, December 6-10,… [find](https://scholar.google.com/scholar?q=Chengwen+Qi%2C+Bowen+Li%2C+Binyuan+Hui%2C+Bailin+Wang%2C+Jinyang+Li%2C+Jinwang+Wu%2C+and+Yuanjun+Laili%2E+2023%2E+An+investigation+of+llms%E2%80%99+inefficacy+in+understanding+converse+relations%2E+In+Proceedings+of+the+2023+C)
38. Yijian Qin, Xin Wang, Ziwei Zhang, and Wenwu Zhu. 2023. Disentangled representation learning with large language models for text-attributed graphs. CoRR, abs/2310.18152. — [arXiv:2310.18152](https://arxiv.org/abs/2310.18152) [find](https://scholar.google.com/scholar?q=Yijian+Qin%2C+Xin+Wang%2C+Ziwei+Zhang%2C+and+Wenwu+Zhu%2E+2023%2E+Disentangled+representation+learning+with+large+language+models+for+text%2Dattributed+graphs%2E+CoRR%2C+abs%2F2310%2E18152%2E+%E2%80%94+%5BarXiv%3A2310%2E18152%5D%28https%3A%2F%2Fa)
39. Yingrong Qin, Chen Gao, Shuangqing Wei, Yue Wang, Depeng Jin, Jian Yuan, Lin Zhang, Dong Li, Jianye Hao, and Yong Li. 2024. Learning from hierarchical structure of knowledge graph for recommendation. ACM Trans. Inf. Syst., 42(1):18:1–18:24. [find](https://scholar.google.com/scholar?q=Yingrong+Qin%2C+Chen+Gao%2C+Shuangqing+Wei%2C+Yue+Wang%2C+Depeng+Jin%2C+Jian+Yuan%2C+Lin+Zhang%2C+Dong+Li%2C+Jianye+Hao%2C+and+Yong+Li%2E+2024%2E+Learning+from+hierarchical+structure+of+knowledge+graph+for+recommendation%2E+)
40. Sylvestre-Alvise Rebuffi, Hakan Bilen, and Andrea Vedaldi. 2017. [Learning multiple visual domains with residual adapters](https://md.research.elamur.ai/free/en/papers/rebuffi-2017-learning-multiple-visual-domains-residual/).
41. Pengpeng Shao, Dawei Zhang, Guohua Yang, Jianhua Tao, Feihu Che, and Tong Liu. 2022. Tucker decomposition-based temporal knowledge graph completion. Knowl. Based Syst., 238:107841. [find](https://scholar.google.com/scholar?q=Pengpeng+Shao%2C+Dawei+Zhang%2C+Guohua+Yang%2C+Jianhua+Tao%2C+Feihu+Che%2C+and+Tong+Liu%2E+2022%2E+Tucker+decomposition%2Dbased+temporal+knowledge+graph+completion%2E+Knowl%2E+Based+Syst%2E%2C+238%3A107841%2E)
42. Haohai Sun, Jialun Zhong, Yunpu Ma, Zhen Han, and Kun He. 2021. Timetraveler: Reinforcement learning for temporal knowledge graph forecasting. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, EMNLP 2021, Virtual Event / Punta Cana, Dominican Republic, 7-11 N… [find](https://scholar.google.com/scholar?q=Haohai+Sun%2C+Jialun+Zhong%2C+Yunpu+Ma%2C+Zhen+Han%2C+and+Kun+He%2E+2021%2E+Timetraveler%3A+Reinforcement+learning+for+temporal+knowledge+graph+forecasting%2E+In+Proceedings+of+the+2021+Conference+on+Empirical+Method)
43. Jiashuo Sun, Chengjin Xu, Lumingyuan Tang, Saizhuo Wang, Chen Lin, Yeyun Gong, Heung-Yeung Shum, and Jian Guo. 2023. Think-on-graph: Deep and responsible reasoning of large language model with knowledge graph. CoRR, abs/2307.07697. [find](https://scholar.google.com/scholar?q=Jiashuo+Sun%2C+Chengjin+Xu%2C+Lumingyuan+Tang%2C+Saizhuo+Wang%2C+Chen+Lin%2C+Yeyun+Gong%2C+Heung%2DYeung+Shum%2C+and+Jian+Guo%2E+2023%2E+Think%2Don%2Dgraph%3A+Deep+and+responsible+reasoning+of+large+language+model+with+knowled)
44. Zhiqing Sun, Zhi-Hong Deng, Jian-Yun Nie, and Jian Tang. 2019. [Rotate: Knowledge graph embedding by relational rotation in complex space](https://md.research.elamur.ai/free/en/papers/sun-2019-rotate-knowledge-graph-embedding-relational/). In ICLR.
45. Jiabin Tang, Yuhao Yang, Wei Wei, Lei Shi, Lixin Su, Suqi Cheng, Dawei Yin, and Chao Huang. 2023. [Graphgpt: Graph instruction tuning for large language models](https://md.research.elamur.ai/free/en/papers/tang-2023-graphgpt-graph-instruction-tuning-large/). CoRR, abs/2310.13023.
46. Yijun Tian, Huan Song, Zichen Wang, Haozhu Wang, Ziqing Hu, Fang Wang, Nitesh V. Chawla, and Panpan Xu. 2023. Graph neural prompting with large language models. CoRR, abs/2309.15427. — [arXiv:2309.15427](https://arxiv.org/abs/2309.15427) [find](https://scholar.google.com/scholar?q=Yijun+Tian%2C+Huan+Song%2C+Zichen+Wang%2C+Haozhu+Wang%2C+Ziqing+Hu%2C+Fang+Wang%2C+Nitesh+V%2E+Chawla%2C+and+Panpan+Xu%2E+2023%2E+Graph+neural+prompting+with+large+language+models%2E+CoRR%2C+abs%2F2309%2E15427%2E+%E2%80%94+%5BarXiv%3A2309%2E154)
47. Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, Aurélien Rodriguez, Armand Joulin, Edouard Grave, and Guillaume Lample. 2023. [Llama: Open and efficient foundation language models](https://md.research.elamur.ai/free/en/papers/touvron-2023-llama-open-efficient-foundation-language/). CoRR, ab…
48. Rakshit Trivedi, Hanjun Dai, Yichen Wang, and Le Song. 2017. Know-evolve: Deep temporal reasoning for dynamic knowledge graphs. In Proceedings of the 34th International Conference on Machine Learning, ICML 2017, Sydney, NSW, Australia, 6-11 August 2017, volume 70 of Proceedings of Machine Learning R… [find](https://scholar.google.com/scholar?q=Rakshit+Trivedi%2C+Hanjun+Dai%2C+Yichen+Wang%2C+and+Le+Song%2E+2017%2E+Know%2Devolve%3A+Deep+temporal+reasoning+for+dynamic+knowledge+graphs%2E+In+Proceedings+of+the+34th+International+Conference+on+Machine+Learning%2C)
49. Vicuna. 2023. Vicuna: An open-source chatbot impressing gpt-4 with 90%* chatgpt quality. [https://vicuna.lmsys.org/](https://vicuna.lmsys.org/). [↗](https://vicuna.lmsys.org/)
50. Jason Wei, Maarten Bosma, Vincent Y Zhao, Kelvin Guu, Adams Wei Yu, Brian Lester, Nan Du, Andrew M Dai, and Quoc V Le. 2021. [Finetuned language models are zero-shot learners](https://md.research.elamur.ai/free/en/papers/wei-2021-finetuned-language-models-zero-shot/). arXiv preprint arXiv:2109.01652.
51. Wenhan Xiong, Mo Yu, Shiyu Chang, Xiaoxiao Guo, and William Yang Wang. 2018. One-shot relational learning for knowledge graphs. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, Brussels, Belgium, October 31 - November 4, 2018, pages 1980–1990. Association fo… [find](https://scholar.google.com/scholar?q=Wenhan+Xiong%2C+Mo+Yu%2C+Shiyu+Chang%2C+Xiaoxiao+Guo%2C+and+William+Yang+Wang%2E+2018%2E+One%2Dshot+relational+learning+for+knowledge+graphs%2E+In+Proceedings+of+the+2018+Conference+on+Empirical+Methods+in+Natural+La)
52. Long Yu, Zhicong Luo, Huanyong Liu, Deng Lin, Hongzhu Li, and Yafeng Deng. 2022. Triplere: Knowledge graph embeddings via tripled relation vectors. CoRR, abs/2209.08271. — [arXiv:2209.08271](https://arxiv.org/abs/2209.08271) [find](https://scholar.google.com/scholar?q=Long+Yu%2C+Zhicong+Luo%2C+Huanyong+Liu%2C+Deng+Lin%2C+Hongzhu+Li%2C+and+Yafeng+Deng%2E+2022%2E+Triplere%3A+Knowledge+graph+embeddings+via+tripled+relation+vectors%2E+CoRR%2C+abs%2F2209%2E08271%2E+%E2%80%94+%5BarXiv%3A2209%2E08271%5D%28https%3A%2F%2Fa)
53. Mengqi Zhang, Yuwei Xia, Qiang Liu, Shu Wu, and Liang Wang. 2023. Learning long-and short-term representations for temporal knowledge graph reasoning. In Proceedings of the ACM Web Conference 2023, pages 2412–2422. [find](https://scholar.google.com/scholar?q=Mengqi+Zhang%2C+Yuwei+Xia%2C+Qiang+Liu%2C+Shu+Wu%2C+and+Liang+Wang%2E+2023%2E+Learning+long%2Dand+short%2Dterm+representations+for+temporal+knowledge+graph+reasoning%2E+In+Proceedings+of+the+ACM+Web+Conference+2023%2C+pa)
54. Cunchao Zhu, Muhao Chen, Changjun Fan, Guangquan Cheng, and Yan Zhang. 2021. Learning from history: Modeling temporal knowledge graphs with sequential copy-generation networks. In Thirty-Fifth AAAI Conference on Artificial Intelligence, AAAI 2021, Thirty-Third Conference on Innovative Applications o… [find](https://scholar.google.com/scholar?q=Cunchao+Zhu%2C+Muhao+Chen%2C+Changjun+Fan%2C+Guangquan+Cheng%2C+and+Yan+Zhang%2E+2021%2E+Learning+from+history%3A+Modeling+temporal+knowledge+graphs+with+sequential+copy%2Dgeneration+networks%2E+In+Thirty%2DFifth+AAAI+Co)

*Updated 3 November 2026*