---
title: "Exploring Graph Learning Tasks with Pure LLMs: A"
description: "The authors build a standardized benchmark for node classification and link prediction on text-attributed graphs. 98 references."
authors: ["Yuxiang Wang", "Xinnan Dai", "Wenqi Fan", "Yao Ma"]
year: 2026
venue: "Findings of the Association for Computational Linguistics: ACL 2026"
source: "https://aclanthology.org/2026.findings-acl.389/"
doi: "10.18653/v1/2026.findings-acl.389"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/wang-2026-exploring-graph-learning-tasks-pure/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/wang-2026-exploring-graph-learning-tasks-pure/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/12/) / Exploring Graph Learning Tasks with Pure LLMs: A Comprehensive Benchmark and Investigation

# Exploring Graph Learning Tasks with Pure LLMs: A Comprehensive Benchmark and Investigation

**Authors:** [Yuxiang Wang](https://md.research.elamur.ai/free/en/authors/yuxiang-wang/), Xinnan Dai, [Wenqi Fan](https://md.research.elamur.ai/free/en/authors/wenqi-fan/) and [Yao Ma](https://md.research.elamur.ai/free/en/authors/yao-ma/) · **Year:** 2026 · **Venue:** [Findings of the Association for Computational Linguistics: ACL 2026](https://md.research.elamur.ai/free/en/venues/findings/) ([Findings of the Association for Computational Linguistics 2026](https://md.research.elamur.ai/free/en/venues/findings/2026/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2026.findings-acl.389/) · [PDF](https://aclanthology.org/2026.findings-acl.389.pdf) · [DOI 10.18653/v1/2026.findings-acl.389](https://doi.org/10.18653/v1/2026.findings-acl.389) · [OpenAlex](https://openalex.org/W4416043369)

**Citations:** 1 citations (OpenAlex, 2026-10-14)

## Summary
LLM-based graph learning studies use inconsistent datasets, preprocessing and splits, so it remains unclear how pure LLMs compare with dedicated graph models. The authors build a standardized benchmark for node classification and link prediction on text-attributed graphs. Graphs are serialized into textual prompts that contain a target node's attributes and, optionally, its 1-hop or 2-hop neighbours, with or without neighbour labels. They test off-the-shelf models, ranging from Llama3B and Llama8B to Qwen-max, GPT-4o, Deepseek V3 and Gemini2.5 Pro, and LoRA-tuned Llama variants against 16 graph learning baselines. Beyond accuracy, the study examines few-shot learning, cross-domain transfer, structural perturbations, data leakage, compute cost and algorithmic graph reasoning. Larger LLMs match or beat most baselines. Instruction tuning lets even small Llama models reach or exceed the strongest graph models, and it is especially useful when labels are scarce. Continuous pre-training adds further gains. The study gives a fair reference point for deciding when LLMs are a practical option for graph tasks.

## Key points
- **Baselines:** the 16 graph learning models span GNNs, graph SSL, graph transformers, LM-augmented graph models and graph prompt methods. Code is available in the JensenYX/LLM-benchmarking GitHub repository.
- **Prompts and tuning:**
  - Node classification uses five prompt formats: ego, plus 1-hop and 2-hop, each with or without labels.
  - Link prediction uses two formats, 1-hop and 2-hop.
  - Llama3B and Llama8B are fine-tuned with LoRA for a single epoch.
- **Data leakage check:** LLaMA-2-13B was compared on ogbn-arxiv and the newer arxiv-2023. The tuned Llama13B scores 76.51 and 75.82 with the 2-hop w/o label prompt, while GCN scores 69.90 and 65.33. This suggests leakage has little effect.
- **Graph reasoning (NLGraph prompts, random NetworkX graphs of 10–20 nodes):**
  - Tuned Llama8B is best on connectivity (86.40), cycle (88.42) and shortest path (56.98).
  - Maximum flow stays hard; the top score is 16.38, from Gemini2.5 Pro.
  - Classical algorithms remain more reliable for these tasks.
- **Limitations:** compute costs are high, and LLM context windows restrict how large a neighbourhood can be serialized. Graph-level tasks and graphs without text features are explored less.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Hong Kong Polytechnic University](https://md.research.elamur.ai/free/en/universities/hong-kong-polytechnic-university/) — University, Hong Kong, China · 84 works · 4,956 citations
- [Rensselaer Polytechnic Institute](https://md.research.elamur.ai/free/en/universities/rensselaer-polytechnic-institute/) — University, Troy, United States · 21 works · 4,933 citations
- [Chinese University of Hong Kong, Shenzhen](https://md.research.elamur.ai/free/en/universities/chinese-university-of-hong-kong-shenzhen/) — University, Shenzhen, China · 47 works · 2,596 citations
- [Michigan State University](https://md.research.elamur.ai/free/en/universities/michigan-state-university/) — University, East Lansing, United States · 23 works · 1,077 citations

📚 Shelves (5): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works · [👁️ Computer Vision and Pattern Recognition](https://md.research.elamur.ai/free/en/shelves/computer-science/computer-vision-and-pattern-recognition/) · 2,869 works

🧭 Topics (2): [🧠 Language models & pretraining](https://md.research.elamur.ai/free/en/topics/language-models/) · 3,320 works · [📏 Evaluation & benchmarks](https://md.research.elamur.ai/free/en/topics/evaluation-benchmarks/) · 1,220 works

## Cite

Cite as: Yuxiang Wang, Xinnan Dai, Wenqi Fan and Yao Ma. 2026. Exploring Graph Learning Tasks with Pure LLMs: A Comprehensive Benchmark and Investigation. Findings of the Association for Computational Linguistics: ACL 2026.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{wang2026exploring,
  title = {{Exploring Graph Learning Tasks with Pure LLMs: A Comprehensive Benchmark and Investigation}},
  author = {Yuxiang Wang, Xinnan Dai, Wenqi Fan and Yao Ma},
  booktitle = {Findings of the Association for Computational Linguistics: ACL 2026},
  year = {2026},
  doi = {10.18653/v1/2026.findings-acl.389},
  url = {https://doi.org/10.18653/v1/2026.findings-acl.389}
}
```

- [← previous Exploration of Perceptual Speech Features for Clinical Decision-Support in Mental Health Care](https://md.research.elamur.ai/free/en/papers/lyberatos-2026-exploration-perceptual-speech-features-clinical/)
- [next → EyeMulator: Improving Code Language Models by Mimicking Human Visual Attention](https://md.research.elamur.ai/free/en/papers/zhang-2026-eyemulator-improving-code-language-models/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/12/)

## References

98 references, 22 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Sami Abu-El-Haija, Bryan Perozzi, Amol Kapoor, Nazanin Alipourfard, Kristina Lerman, Hrayr Harutyunyan, Greg Ver Steeg, and Aram Galstyan. 2019. Mixhop: Higher-order graph convolutional architectures via sparsified neighborhood mixing. In international conference on machine learning, pages 21–29. PM… [find](https://scholar.google.com/scholar?q=Sami+Abu%2DEl%2DHaija%2C+Bryan+Perozzi%2C+Amol+Kapoor%2C+Nazanin+Alipourfard%2C+Kristina+Lerman%2C+Hrayr+Harutyunyan%2C+Greg+Ver+Steeg%2C+and+Aram+Galstyan%2E+2019%2E+Mixhop%3A+Higher%2Dorder+graph+convolutional+architectures+)
2. Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, and 1 others. 2023. [Gpt-4 technical report](https://md.research.elamur.ai/free/en/papers/openai-2023-gpt-4-technical-report/). arXiv preprint arXiv:2303.08774.
3. Jinze Bai, Shuai Bai, Yunfei Chu, Zeyu Cui, Kai Dang, Xiaodong Deng, Yang Fan, Wenbin Ge, Yu Han, Fei Huang, and 1 others. 2023. [Qwen technical report](https://md.research.elamur.ai/free/en/papers/bai-2023-qwen-technical-report/). arXiv preprint arXiv:2309.16609.
4. Yukun Cao, Shuo Han, Zengyi Gao, Zezhong Ding, Xike Xie, and S Kevin Zhou. 2024. Graphinsight: Unlocking insights in large language models for graph structure understanding. arXiv preprint arXiv:2409.03258. [find](https://scholar.google.com/scholar?q=Yukun+Cao%2C+Shuo+Han%2C+Zengyi+Gao%2C+Zezhong+Ding%2C+Xike+Xie%2C+and+S+Kevin+Zhou%2E+2024%2E+Graphinsight%3A+Unlocking+insights+in+large+language+models+for+graph+structure+understanding%2E+arXiv+preprint+arXiv%3A2409%2E)
5. Ziwei Chai, Tianjie Zhang, Liang Wu, Kaiqiao Han, Xiaohai Hu, Xuanwen Huang, and Yang Yang. 2023. Graphllm: Boosting graph reasoning ability of large language model. arXiv preprint arXiv:2310.05845. — [arXiv:2310.05845](https://arxiv.org/abs/2310.05845) [find](https://scholar.google.com/scholar?q=Ziwei+Chai%2C+Tianjie+Zhang%2C+Liang+Wu%2C+Kaiqiao+Han%2C+Xiaohai+Hu%2C+Xuanwen+Huang%2C+and+Yang+Yang%2E+2023%2E+Graphllm%3A+Boosting+graph+reasoning+ability+of+large+language+model%2E+arXiv+preprint+arXiv%3A2310%2E05845%2E+%E2%80%94)
6. Nuo Chen, Yuhan Li, Jianheng Tang, and Jia Li. 2024a. Graphwiz: An instruction-following language model for graph computational problems. In Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pages 353–364. [find](https://scholar.google.com/scholar?q=Nuo+Chen%2C+Yuhan+Li%2C+Jianheng+Tang%2C+and+Jia+Li%2E+2024a%2E+Graphwiz%3A+An+instruction%2Dfollowing+language+model+for+graph+computational+problems%2E+In+Proceedings+of+the+30th+ACM+SIGKDD+Conference+on+Knowledge+)
7. Runjin Chen, Tong Zhao, Ajay Jaiswal, Neil Shah, and Zhangyang Wang. 2024b. Llaga: Large language and graph assistant. arXiv preprint arXiv:2402.08170. — [arXiv:2402.08170](https://arxiv.org/abs/2402.08170) [find](https://scholar.google.com/scholar?q=Runjin+Chen%2C+Tong+Zhao%2C+Ajay+Jaiswal%2C+Neil+Shah%2C+and+Zhangyang+Wang%2E+2024b%2E+Llaga%3A+Large+language+and+graph+assistant%2E+arXiv+preprint+arXiv%3A2402%2E08170%2E+%E2%80%94+%5BarXiv%3A2402%2E08170%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2402%2E)
8. Zhikai Chen, Haitao Mao, Hang Li, Wei Jin, Hongzhi Wen, Xiaochi Wei, Shuaiqiang Wang, Dawei Yin, Wenqi Fan, Hui Liu, and 1 others. 2024c. Exploring the potential of large language models (llms) in learning on graphs. ACM SIGKDD Explorations Newsletter, 25(2):42–61. [find](https://scholar.google.com/scholar?q=Zhikai+Chen%2C+Haitao+Mao%2C+Hang+Li%2C+Wei+Jin%2C+Hongzhi+Wen%2C+Xiaochi+Wei%2C+Shuaiqiang+Wang%2C+Dawei+Yin%2C+Wenqi+Fan%2C+Hui+Liu%2C+and+1+others%2E+2024c%2E+Exploring+the+potential+of+large+language+models+%28llms%29+in+lea)
9. Zhikai Chen, Haitao Mao, Jingzhe Liu, Yu Song, Bingheng Li, Wei Jin, Bahare Fatemi, Anton Tsitsulin, Bryan Perozzi, Hui Liu, and 1 others. 2024d. Text-space graph foundation models: Comprehensive benchmarks and new insights. arXiv preprint arXiv:2406.10727. — [arXiv:2406.10727](https://arxiv.org/abs/2406.10727) [find](https://scholar.google.com/scholar?q=Zhikai+Chen%2C+Haitao+Mao%2C+Jingzhe+Liu%2C+Yu+Song%2C+Bingheng+Li%2C+Wei+Jin%2C+Bahare+Fatemi%2C+Anton+Tsitsulin%2C+Bryan+Perozzi%2C+Hui+Liu%2C+and+1+others%2E+2024d%2E+Text%2Dspace+graph+foundation+models%3A+Comprehensive+benc)
10. Zhikai Chen, Haitao Mao, Hongzhi Wen, Haoyu Han, Wei Jin, Haiyang Zhang, Hui Liu, and Jiliang Tang. 2023. Label-free node classification on graphs with large language models (llms). arXiv preprint arXiv:2310.04668. — [arXiv:2310.04668](https://arxiv.org/abs/2310.04668) [find](https://scholar.google.com/scholar?q=Zhikai+Chen%2C+Haitao+Mao%2C+Hongzhi+Wen%2C+Haoyu+Han%2C+Wei+Jin%2C+Haiyang+Zhang%2C+Hui+Liu%2C+and+Jiliang+Tang%2E+2023%2E+Label%2Dfree+node+classification+on+graphs+with+large+language+models+%28llms%29%2E+arXiv+preprint+arX)
11. Yao Cheng, Yige Zhao, Jianxiang Yu, and Xiang Li. 2024. Boosting graph foundation model from structural perspective. arXiv preprint arXiv:2407.19941. — [arXiv:2407.19941](https://arxiv.org/abs/2407.19941) [find](https://scholar.google.com/scholar?q=Yao+Cheng%2C+Yige+Zhao%2C+Jianxiang+Yu%2C+and+Xiang+Li%2E+2024%2E+Boosting+graph+foundation+model+from+structural+perspective%2E+arXiv+preprint+arXiv%3A2407%2E19941%2E+%E2%80%94+%5BarXiv%3A2407%2E19941%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2407%2E19)
12. Wei-Lin Chiang, Xuanqing Liu, Si Si, Yang Li, Samy Bengio, and Cho-Jui Hsieh. 2019. Cluster-gcn: An efficient algorithm for training deep and large graph convolutional networks. In Proceedings of the 25th ACM SIGKDD international conference on knowledge discovery & data mining, pages 257–266. [find](https://scholar.google.com/scholar?q=Wei%2DLin+Chiang%2C+Xuanqing+Liu%2C+Si+Si%2C+Yang+Li%2C+Samy+Bengio%2C+and+Cho%2DJui+Hsieh%2E+2019%2E+Cluster%2Dgcn%3A+An+efficient+algorithm+for+training+deep+and+large+graph+convolutional+networks%2E+In+Proceedings+of+the+)
13. Eli Chien, Wei-Cheng Chang, Cho-Jui Hsieh, Hsiang-Fu Yu, Jiong Zhang, Olgica Milenkovic, and Inderjit S Dhillon. 2021. Node feature extraction by self-supervised multi-scale neighborhood prediction. arXiv preprint arXiv:2111.00064. — [arXiv:2111.00064](https://arxiv.org/abs/2111.00064) [find](https://scholar.google.com/scholar?q=Eli+Chien%2C+Wei%2DCheng+Chang%2C+Cho%2DJui+Hsieh%2C+Hsiang%2DFu+Yu%2C+Jiong+Zhang%2C+Olgica+Milenkovic%2C+and+Inderjit+S+Dhillon%2E+2021%2E+Node+feature+extraction+by+self%2Dsupervised+multi%2Dscale+neighborhood+prediction%2E+a)
14. Gheorghe Comanici, Eric Bieber, Mike Schaekermann, Ice Pasupat, Noveen Sachdeva, Inderjit Dhillon, Marcel Blistein, Ori Ram, Dan Zhang, Evan Rosen, and 1 others. 2025. [Gemini 2.5: Pushing the frontier with advanced reasoning, multimodality, long context, and next generation agentic capabilities](https://md.research.elamur.ai/free/en/papers/comanici-2025-gemini-2-5-pushing-frontier/). arX…
15. Xinnan Dai, Haohao Qu, Yifen Shen, Bohang Zhang, Qihao Wen, Wenqi Fan, Dongsheng Li, Jiliang Tang, and Caihua Shan. 2024a. How do large language models understand graph patterns? a benchmark for graph pattern comprehension. arXiv preprint arXiv:2410.05298. — [arXiv:2410.05298](https://arxiv.org/abs/2410.05298) [find](https://scholar.google.com/scholar?q=Xinnan+Dai%2C+Haohao+Qu%2C+Yifen+Shen%2C+Bohang+Zhang%2C+Qihao+Wen%2C+Wenqi+Fan%2C+Dongsheng+Li%2C+Jiliang+Tang%2C+and+Caihua+Shan%2E+2024a%2E+How+do+large+language+models+understand+graph+patterns%3F+a+benchmark+for+graph)
16. Xinnan Dai, Qihao Wen, Yifei Shen, Hongzhi Wen, Dongsheng Li, Jiliang Tang, and Caihua Shan. 2024b. Revisiting the graph reasoning ability of large language models: Case studies in translation, connectivity and shortest path. arXiv preprint arXiv:2408.09529. — [arXiv:2408.09529](https://arxiv.org/abs/2408.09529) [find](https://scholar.google.com/scholar?q=Xinnan+Dai%2C+Qihao+Wen%2C+Yifei+Shen%2C+Hongzhi+Wen%2C+Dongsheng+Li%2C+Jiliang+Tang%2C+and+Caihua+Shan%2E+2024b%2E+Revisiting+the+graph+reasoning+ability+of+large+language+models%3A+Case+studies+in+translation%2C+connec)
17. Jacob Devlin. 2018. [Bert: Pre-training of deep bidirectional transformers for language understanding](https://md.research.elamur.ai/free/en/papers/devlin-2019-bert-pre-training-deep-bidirectional/). arXiv preprint arXiv:1810.04805.
18. Edsger W Dijkstra. 2022. A note on two problems in connexion with graphs. In Edsger Wybe Dijkstra: his life, work, and legacy, pages 287–290. [find](https://scholar.google.com/scholar?q=Edsger+W+Dijkstra%2E+2022%2E+A+note+on+two+problems+in+connexion+with+graphs%2E+In+Edsger+Wybe+Dijkstra%3A+his+life%2C+work%2C+and+legacy%2C+pages+287%E2%80%93290%2E)
19. Wenyu Du, Shuang Cheng, Tongxu Luo, Zihan Qiu, Zeyu Huang, Ka Chun Cheung, Reynold Cheng, and Jie Fu. 2024. Unlocking continual learning abilities in language models. arXiv preprint arXiv:2406.17245. [find](https://scholar.google.com/scholar?q=Wenyu+Du%2C+Shuang+Cheng%2C+Tongxu+Luo%2C+Zihan+Qiu%2C+Zeyu+Huang%2C+Ka+Chun+Cheung%2C+Reynold+Cheng%2C+and+Jie+Fu%2E+2024%2E+Unlocking+continual+learning+abilities+in+language+models%2E+arXiv+preprint+arXiv%3A2406%2E17245%2E)
20. Taoran Fang, Yunchao Zhang, Yang Yang, Chunping Wang, and Lei Chen. 2024. Universal prompt tuning for graph neural networks. Advances in Neural Information Processing Systems, 36. [find](https://scholar.google.com/scholar?q=Taoran+Fang%2C+Yunchao+Zhang%2C+Yang+Yang%2C+Chunping+Wang%2C+and+Lei+Chen%2E+2024%2E+Universal+prompt+tuning+for+graph+neural+networks%2E+Advances+in+Neural+Information+Processing+Systems%2C+36%2E)
21. Yi Fang, Bowen Jin, Jiacheng Shen, Sirui Ding, Qiaoyu Tan, and Jiawei Han. 2025. Graphgpt-o: Synergistic multimodal comprehension and generation on graphs. Preprint, arXiv:2502.11925. — [arXiv:2502.11925](https://arxiv.org/abs/2502.11925) [find](https://scholar.google.com/scholar?q=Yi+Fang%2C+Bowen+Jin%2C+Jiacheng+Shen%2C+Sirui+Ding%2C+Qiaoyu+Tan%2C+and+Jiawei+Han%2E+2025%2E+Graphgpt%2Do%3A+Synergistic+multimodal+comprehension+and+generation+on+graphs%2E+Preprint%2C+arXiv%3A2502%2E11925%2E+%E2%80%94+%5BarXiv%3A2502%2E11)
22. Bahare Fatemi, Jonathan Halcrow, and Bryan Perozzi. 2023. Talk like a graph: Encoding graphs for large language models. arXiv preprint arXiv:2310.04560. [find](https://scholar.google.com/scholar?q=Bahare+Fatemi%2C+Jonathan+Halcrow%2C+and+Bryan+Perozzi%2E+2023%2E+Talk+like+a+graph%3A+Encoding+graphs+for+large+language+models%2E+arXiv+preprint+arXiv%3A2310%2E04560%2E)
23. Aric Hagberg, Pieter J Swart, and Daniel A Schult. 2008. Exploring network structure, dynamics, and function using networkx. Technical report, Los Alamos National Laboratory (LANL), Los Alamos, NM (United States). [find](https://scholar.google.com/scholar?q=Aric+Hagberg%2C+Pieter+J+Swart%2C+and+Daniel+A+Schult%2E+2008%2E+Exploring+network+structure%2C+dynamics%2C+and+function+using+networkx%2E+Technical+report%2C+Los+Alamos+National+Laboratory+%28LANL%29%2C+Los+Alamos%2C+NM+%28Un)
24. Xiaoxin He, Xavier Bresson, Thomas Laurent, Bryan Hooi, and 1 others. 2023. Explanations as features: Llm-based features for text-attributed graphs. arXiv preprint arXiv:2305.19523, 2(4):8. [find](https://scholar.google.com/scholar?q=Xiaoxin+He%2C+Xavier+Bresson%2C+Thomas+Laurent%2C+Bryan+Hooi%2C+and+1+others%2E+2023%2E+Explanations+as+features%3A+Llm%2Dbased+features+for+text%2Dattributed+graphs%2E+arXiv+preprint+arXiv%3A2305%2E19523%2C+2%284%29%3A8%2E)
25. Yufei He and Bryan Hooi. 2024. Unigraph: Learning a cross-domain graph foundation model from natural language. arXiv preprint arXiv:2402.13630. — [arXiv:2402.13630](https://arxiv.org/abs/2402.13630) [find](https://scholar.google.com/scholar?q=Yufei+He+and+Bryan+Hooi%2E+2024%2E+Unigraph%3A+Learning+a+cross%2Ddomain+graph+foundation+model+from+natural+language%2E+arXiv+preprint+arXiv%3A2402%2E13630%2E+%E2%80%94+%5BarXiv%3A2402%2E13630%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2402%2E13630%29)
26. Zhenyu Hou, Haozhan Li, Yukuo Cen, Jie Tang, and Yuxiao Dong. 2024. Graphalign: Pretraining one graph neural network on multiple graphs via feature alignment. arXiv preprint arXiv:2406.02953. — [arXiv:2406.02953](https://arxiv.org/abs/2406.02953) [find](https://scholar.google.com/scholar?q=Zhenyu+Hou%2C+Haozhan+Li%2C+Yukuo+Cen%2C+Jie+Tang%2C+and+Yuxiao+Dong%2E+2024%2E+Graphalign%3A+Pretraining+one+graph+neural+network+on+multiple+graphs+via+feature+alignment%2E+arXiv+preprint+arXiv%3A2406%2E02953%2E+%E2%80%94+%5BarXiv)
27. Zhenyu Hou, Xiao Liu, Yukuo Cen, Yuxiao Dong, Hongxia Yang, Chunjie Wang, and Jie Tang. 2022. Graphmae: Self-supervised masked graph autoencoders. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pages 594–604. [find](https://scholar.google.com/scholar?q=Zhenyu+Hou%2C+Xiao+Liu%2C+Yukuo+Cen%2C+Yuxiao+Dong%2C+Hongxia+Yang%2C+Chunjie+Wang%2C+and+Jie+Tang%2E+2022%2E+Graphmae%3A+Self%2Dsupervised+masked+graph+autoencoders%2E+In+Proceedings+of+the+28th+ACM+SIGKDD+Conference+on+K)
28. Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2021a. [Lora: Low-rank adaptation of large language models](https://md.research.elamur.ai/free/en/papers/hu-2021-lora-low-rank-adaptation-large/). arXiv preprint arXiv:2106.09685.
29. Weihua Hu, Matthias Fey, Marinka Zitnik, Yuxiao Dong, Hongyu Ren, Bowen Liu, Michele Catasta, and Jure Leskovec. 2020. Open graph benchmark: Datasets for machine learning on graphs. Advances in neural information processing systems, 33:22118–22133. [find](https://scholar.google.com/scholar?q=Weihua+Hu%2C+Matthias+Fey%2C+Marinka+Zitnik%2C+Yuxiao+Dong%2C+Hongyu+Ren%2C+Bowen+Liu%2C+Michele+Catasta%2C+and+Jure+Leskovec%2E+2020%2E+Open+graph+benchmark%3A+Datasets+for+machine+learning+on+graphs%2E+Advances+in+neural)
30. Yang Hu, Haoxuan You, Zhecan Wang, Zhicheng Wang, Erjin Zhou, and Yue Gao. 2021b. Graph-mlp: Node classification without message passing in graph. arXiv preprint arXiv:2106.04051. — [arXiv:2106.04051](https://arxiv.org/abs/2106.04051) [find](https://scholar.google.com/scholar?q=Yang+Hu%2C+Haoxuan+You%2C+Zhecan+Wang%2C+Zhicheng+Wang%2C+Erjin+Zhou%2C+and+Yue+Gao%2E+2021b%2E+Graph%2Dmlp%3A+Node+classification+without+message+passing+in+graph%2E+arXiv+preprint+arXiv%3A2106%2E04051%2E+%E2%80%94+%5BarXiv%3A2106%2E04051%5D)
31. Jin Huang, Xingjian Zhang, Qiaozhu Mei, and Jiaqi Ma. 2023. Can llms effectively leverage graph structural information: when and why. arXiv preprint arXiv:2309.16595. — [arXiv:2309.16595](https://arxiv.org/abs/2309.16595) [find](https://scholar.google.com/scholar?q=Jin+Huang%2C+Xingjian+Zhang%2C+Qiaozhu+Mei%2C+and+Jiaqi+Ma%2E+2023%2E+Can+llms+effectively+leverage+graph+structural+information%3A+when+and+why%2E+arXiv+preprint+arXiv%3A2309%2E16595%2E+%E2%80%94+%5BarXiv%3A2309%2E16595%5D%28https%3A%2F%2Farxi)
32. Qian Huang, Hongyu Ren, Peng Chen, Gregor Kržmanc, Daniel Zeng, Percy S Liang, and Jure Leskovec. 2024a. Prodigy: Enabling in-context learning over graphs. Advances in Neural Information Processing Systems, 36. [find](https://scholar.google.com/scholar?q=Qian+Huang%2C+Hongyu+Ren%2C+Peng+Chen%2C+Gregor+Kr%C5%BEmanc%2C+Daniel+Zeng%2C+Percy+S+Liang%2C+and+Jure+Leskovec%2E+2024a%2E+Prodigy%3A+Enabling+in%2Dcontext+learning+over+graphs%2E+Advances+in+Neural+Information+Processing+Sy)
33. Xuanwen Huang, Kaiqiao Han, Yang Yang, Dezheng Bao, Quanjin Tao, Ziwei Chai, and Qi Zhu. 2024b. Can gnn be good adapter for llms? In Proceedings of the ACM Web Conference 2024, pages 893–904. [find](https://scholar.google.com/scholar?q=Xuanwen+Huang%2C+Kaiqiao+Han%2C+Yang+Yang%2C+Dezheng+Bao%2C+Quanjin+Tao%2C+Ziwei+Chai%2C+and+Qi+Zhu%2E+2024b%2E+Can+gnn+be+good+adapter+for+llms%3F+In+Proceedings+of+the+ACM+Web+Conference+2024%2C+pages+893%E2%80%93904%2E)
34. Will Hamilton, Zhitao Ying, and Jure Leskovec. 2017. [Inductive representation learning on large graphs](https://md.research.elamur.ai/free/en/papers/hamilton-2017-inductive-representation-learning-large-graphs/). Advances in neural information processing systems, 30.
35. Jinhao Jiang, Kun Zhou, Zican Dong, Keming Ye, Wayne Xin Zhao, and Ji-Rong Wen. 2023. Structgpt: A general framework for large language model to reason over structured data. arXiv preprint arXiv:2305.09645. [find](https://scholar.google.com/scholar?q=Jinhao+Jiang%2C+Kun+Zhou%2C+Zican+Dong%2C+Keming+Ye%2C+Wayne+Xin+Zhao%2C+and+Ji%2DRong+Wen%2E+2023%2E+Structgpt%3A+A+general+framework+for+large+language+model+to+reason+over+structured+data%2E+arXiv+preprint+arXiv%3A2305%2E)
36. Pengcheng He, Xiaodong Liu, Jianfeng Gao, and Weizhu Chen. 2020. [Deberta: Decoding-enhanced bert with disentangled attention](https://md.research.elamur.ai/free/en/papers/he-2020-deberta-decoding-enhanced-bert-disentangled/). arXiv preprint arXiv:2006.03654.
37. Zixuan Ke, Yijia Shao, Haowei Lin, Tatsuya Konishi, Gyuhak Kim, and Bing Liu. 2023. Continual pre-training of language models. arXiv preprint arXiv:2302.03241. [find](https://scholar.google.com/scholar?q=Zixuan+Ke%2C+Yijia+Shao%2C+Haowei+Lin%2C+Tatsuya+Konishi%2C+Gyuhak+Kim%2C+and+Bing+Liu%2E+2023%2E+Continual+pre%2Dtraining+of+language+models%2E+arXiv+preprint+arXiv%3A2302%2E03241%2E)
38. Shima Khoshraftar, Niaz Abedini, and Amir Hajian. 2025. Graphit: Efficient node classification on text-attributed graphs with prompt optimized llms. arXiv preprint arXiv:2502.10522. — [arXiv:2502.10522](https://arxiv.org/abs/2502.10522) [find](https://scholar.google.com/scholar?q=Shima+Khoshraftar%2C+Niaz+Abedini%2C+and+Amir+Hajian%2E+2025%2E+Graphit%3A+Efficient+node+classification+on+text%2Dattributed+graphs+with+prompt+optimized+llms%2E+arXiv+preprint+arXiv%3A2502%2E10522%2E+%E2%80%94+%5BarXiv%3A2502%2E1052)
39. Jiho Kim, Yeonsu Kwon, Yohan Jo, and Edward Choi. 2023. Kg-gpt: A general framework for reasoning on knowledge graphs using large language models. arXiv preprint arXiv:2310.11220. [find](https://scholar.google.com/scholar?q=Jiho+Kim%2C+Yeonsu+Kwon%2C+Yohan+Jo%2C+and+Edward+Choi%2E+2023%2E+Kg%2Dgpt%3A+A+general+framework+for+reasoning+on+knowledge+graphs+using+large+language+models%2E+arXiv+preprint+arXiv%3A2310%2E11220%2E)
40. Thomas N Kipf and Max Welling. 2016. [Semi-supervised classification with graph convolutional networks](https://md.research.elamur.ai/free/en/papers/kipf-2016-semi-supervised-classification-graph-convolutional/). arXiv preprint arXiv:1609.02907.
41. Yuhan Li, Peisong Wang, Zhixun Li, Jeffrey Xu Yu, and Jia Li. 2024a. Zerog: Investigating cross-dataset zero-shot transferability in graphs. In Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pages 1725–1735. [find](https://scholar.google.com/scholar?q=Yuhan+Li%2C+Peisong+Wang%2C+Zhixun+Li%2C+Jeffrey+Xu+Yu%2C+and+Jia+Li%2E+2024a%2E+Zerog%3A+Investigating+cross%2Ddataset+zero%2Dshot+transferability+in+graphs%2E+In+Proceedings+of+the+30th+ACM+SIGKDD+Conference+on+Knowled)
42. Yuhan Li, Peisong Wang, Xiao Zhu, Aochuan Chen, Haiyun Jiang, Deng Cai, Victor Wai Kin Chan, and Jia Li. 2024b. Glbench: A comprehensive benchmark for graph with large language models. arXiv preprint arXiv:2407.07457. — [arXiv:2407.07457](https://arxiv.org/abs/2407.07457) [find](https://scholar.google.com/scholar?q=Yuhan+Li%2C+Peisong+Wang%2C+Xiao+Zhu%2C+Aochuan+Chen%2C+Haiyun+Jiang%2C+Deng+Cai%2C+Victor+Wai+Kin+Chan%2C+and+Jia+Li%2E+2024b%2E+Glbench%3A+A+comprehensive+benchmark+for+graph+with+large+language+models%2E+arXiv+preprint+)
43. Chanuk Lim, Kyong-Ha Lee, Hyun Ji Jeong, and Sungsu Lim. 2025. Grail: Graph retrieval-augmented in-context learning for node classification in real-world textual-attributed graphs. [find](https://scholar.google.com/scholar?q=Chanuk+Lim%2C+Kyong%2DHa+Lee%2C+Hyun+Ji+Jeong%2C+and+Sungsu+Lim%2E+2025%2E+Grail%3A+Graph+retrieval%2Daugmented+in%2Dcontext+learning+for+node+classification+in+real%2Dworld+textual%2Dattributed+graphs%2E)
44. Aixin Liu, Bei Feng, Bing Xue, Bingxuan Wang, Bochao Wu, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, and 1 others. 2024. Deepseek-v3 technical report. arXiv preprint arXiv:2412.19437. [find](https://scholar.google.com/scholar?q=Aixin+Liu%2C+Bei+Feng%2C+Bing+Xue%2C+Bingxuan+Wang%2C+Bochao+Wu%2C+Chengda+Lu%2C+Chenggang+Zhao%2C+Chengqi+Deng%2C+Chenyu+Zhang%2C+Chong+Ruan%2C+and+1+others%2E+2024%2E+Deepseek%2Dv3+technical+report%2E+arXiv+preprint+arXiv%3A2412)
45. Hao Liu, Jiarui Feng, Lecheng Kong, Ningyue Liang, Dacheng Tao, Yixin Chen, and Muhan Zhang. 2023a. One for all: Towards training one graph model for all classification tasks. arXiv preprint arXiv:2310.00149. — [arXiv:2310.00149](https://arxiv.org/abs/2310.00149) [find](https://scholar.google.com/scholar?q=Hao+Liu%2C+Jiarui+Feng%2C+Lecheng+Kong%2C+Ningyue+Liang%2C+Dacheng+Tao%2C+Yixin+Chen%2C+and+Muhan+Zhang%2E+2023a%2E+One+for+all%3A+Towards+training+one+graph+model+for+all+classification+tasks%2E+arXiv+preprint+arXiv%3A231)
46. Zemin Liu, Xingtong Yu, Yuan Fang, and Xinming Zhang. 2023b. Graphprompt: Unifying pre-training and downstream tasks for graph neural networks. In Proceedings of the ACM Web Conference 2023, pages 417–428. [find](https://scholar.google.com/scholar?q=Zemin+Liu%2C+Xingtong+Yu%2C+Yuan+Fang%2C+and+Xinming+Zhang%2E+2023b%2E+Graphprompt%3A+Unifying+pre%2Dtraining+and+downstream+tasks+for+graph+neural+networks%2E+In+Proceedings+of+the+ACM+Web+Conference+2023%2C+pages+417)
47. Donald Loveland, Jiong Zhu, Mark Heimann, Benjamin Fish, Michael T Schaub, and Danai Koutra. 2024. On performance discrepancies across local homophily levels in graph neural networks. In Learning on Graphs Conference, pages 6–1. PMLR. [find](https://scholar.google.com/scholar?q=Donald+Loveland%2C+Jiong+Zhu%2C+Mark+Heimann%2C+Benjamin+Fish%2C+Michael+T+Schaub%2C+and+Danai+Koutra%2E+2024%2E+On+performance+discrepancies+across+local+homophily+levels+in+graph+neural+networks%2E+In+Learning+on+G)
48. Linhao Luo, Yuan-Fang Li, Gholamreza Haffari, and Shirui Pan. 2024a. Reasoning on graphs: Faithful and interpretable large language model reasoning. Preprint, arXiv:2310.01061. [find](https://scholar.google.com/scholar?q=Linhao+Luo%2C+Yuan%2DFang+Li%2C+Gholamreza+Haffari%2C+and+Shirui+Pan%2E+2024a%2E+Reasoning+on+graphs%3A+Faithful+and+interpretable+large+language+model+reasoning%2E+Preprint%2C+arXiv%3A2310%2E01061%2E)
49. Yuankai Luo, Lei Shi, and Xiao-Ming Wu. 2024b. Classic gnns are strong baselines: Reassessing gnns for node classification. arXiv preprint arXiv:2406.08993. — [arXiv:2406.08993](https://arxiv.org/abs/2406.08993) [find](https://scholar.google.com/scholar?q=Yuankai+Luo%2C+Lei+Shi%2C+and+Xiao%2DMing+Wu%2E+2024b%2E+Classic+gnns+are+strong+baselines%3A+Reassessing+gnns+for+node+classification%2E+arXiv+preprint+arXiv%3A2406%2E08993%2E+%E2%80%94+%5BarXiv%3A2406%2E08993%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F)
50. Shengjie Ma, Chengjin Xu, Xuhui Jiang, Muzhi Li, Huaren Qu, Cehao Yang, Jiaxin Mao, and Jian Guo. 2025. Think-on-graph 2.0: Deep and faithful large language model reasoning with knowledge-guided retrieval augmented generation. Preprint, arXiv:2407.10805. — [arXiv:2407.10805](https://arxiv.org/abs/2407.10805) [find](https://scholar.google.com/scholar?q=Shengjie+Ma%2C+Chengjin+Xu%2C+Xuhui+Jiang%2C+Muzhi+Li%2C+Huaren+Qu%2C+Cehao+Yang%2C+Jiaxin+Mao%2C+and+Jian+Guo%2E+2025%2E+Think%2Don%2Dgraph+2%2E0%3A+Deep+and+faithful+large+language+model+reasoning+with+knowledge%2Dguided+retri)
51. Andrew Kachites McCallum, Kamal Nigam, Jason Rennie, and Kristie Seymore. 2000. Automating the construction of internet portals with machine learning. Information Retrieval, 3:127–163. [find](https://scholar.google.com/scholar?q=Andrew+Kachites+McCallum%2C+Kamal+Nigam%2C+Jason+Rennie%2C+and+Kristie+Seymore%2E+2000%2E+Automating+the+construction+of+internet+portals+with+machine+learning%2E+Information+Retrieval%2C+3%3A127%E2%80%93163%2E)
52. Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S Corrado, and Jeff Dean. 2013. [Distributed representations of words and phrases and their compositionality](https://md.research.elamur.ai/free/en/papers/mikolov-2013-distributed-representations-words-phrases-their/). Advances in neural information processing systems, 26.
53. Jianmo Ni, Jiacheng Li, and Julian McAuley. 2019. [Justifying recommendations using distantly-labeled reviews and fine-grained aspects](https://md.research.elamur.ai/free/en/papers/ni-2019-justifying-recommendations-distantly-labeled-reviews/). In Proceedings of the 2019 conference on empirical methods in natural language processing and the 9th international joint conference on natural language processing (…
54. Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, and 1 others. 2022. [Training language models to follow instructions with human feedback](https://md.research.elamur.ai/free/en/papers/ouyang-2022-training-language-models-follow-instructions/). Advances in neural information processing systems, 35:27730–27744.
55. Ethan Perez, Sam Ringer, Kamilė Lukošiūtė, Karina Nguyen, Edwin Chen, Scott Heiner, Craig Pettit, Catherine Olsson, Sandipan Kundu, Saurav Kadavath, and 1 others. 2022. [Discovering language model behaviors with model-written evaluations](https://md.research.elamur.ai/free/en/papers/perez-2022-discovering-language-model-behaviors-model/). arXiv preprint arXiv:2212.09251.
56. Bryan Perozzi, Bahare Fatemi, Dustin Zelle, Anton Tsitsulin, Mehran Kazemi, Rami Al-Rfou, and Jonathan Halcrow. 2024. Let your graph do the talking: Encoding structured data for llms. arXiv preprint arXiv:2402.05862. — [arXiv:2402.05862](https://arxiv.org/abs/2402.05862) [find](https://scholar.google.com/scholar?q=Bryan+Perozzi%2C+Bahare+Fatemi%2C+Dustin+Zelle%2C+Anton+Tsitsulin%2C+Mehran+Kazemi%2C+Rami+Al%2DRfou%2C+and+Jonathan+Halcrow%2E+2024%2E+Let+your+graph+do+the+talking%3A+Encoding+structured+data+for+llms%2E+arXiv+preprint+a)
57. Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu. 2020. [Exploring the limits of transfer learning with a unified text-to-text transformer](https://md.research.elamur.ai/free/en/papers/raffel-2019-exploring-limits-transfer-learning-unified/). Journal of machine learning research, 21(140):1–67.
58. Yide Ran, Zhaozhuo Xu, Yuhang Yao, Zijian Hu, Shanshan Han, Han Jin, Alay Dilipbhai Shah, Jipeng Zhang, Dimitris Stripelis, Tong Zhang, and 1 others. 2024. Alopex: A computational framework for enabling on-device function calls with llms. arXiv preprint arXiv:2411.05209. — [arXiv:2411.05209](https://arxiv.org/abs/2411.05209) [find](https://scholar.google.com/scholar?q=Yide+Ran%2C+Zhaozhuo+Xu%2C+Yuhang+Yao%2C+Zijian+Hu%2C+Shanshan+Han%2C+Han+Jin%2C+Alay+Dilipbhai+Shah%2C+Jipeng+Zhang%2C+Dimitris+Stripelis%2C+Tong+Zhang%2C+and+1+others%2E+2024%2E+Alopex%3A+A+computational+framework+for+enabli)
59. N Reimers. 2019. [Sentence-bert: Sentence embeddings using siamese bert-networks](https://md.research.elamur.ai/free/en/papers/reimers-2019-sentence-bert-sentence-embeddings-siamese/). arXiv preprint arXiv:1908.10084.
60. Victor Sanh, Albert Webson, Colin Raffel, Stephen H Bach, Lintang Sutawika, Zaid Alyafeai, Antoine Chaffin, Arnaud Stiegler, Teven Le Scao, Arun Raja, and 1 others. 2021. [Multitask prompted training enables zero-shot task generalization](https://md.research.elamur.ai/free/en/papers/sanh-2021-multitask-prompted-training-enables-zero/). arXiv preprint arXiv:2110.08207.
61. Prithviraj Sen, Galileo Namata, Mustafa Bilgic, Lise Getoor, Brian Galligher, and Tina Eliassi-Rad. 2008. Collective classification in network data. AI magazine, 29(3):93–93. [find](https://scholar.google.com/scholar?q=Prithviraj+Sen%2C+Galileo+Namata%2C+Mustafa+Bilgic%2C+Lise+Getoor%2C+Brian+Galligher%2C+and+Tina+Eliassi%2DRad%2E+2008%2E+Collective+classification+in+network+data%2E+AI+magazine%2C+29%283%29%3A93%E2%80%9393%2E)
62. Yuan Sui, Yufei He, Nian Liu, Xiaoxin He, Kun Wang, and Bryan Hooi. 2025. Fidelis: Faithful reasoning in large language models for knowledge graph question answering. In Findings of the Association for Computational Linguistics: ACL 2025, pages 8315–8330. [find](https://scholar.google.com/scholar?q=Yuan+Sui%2C+Yufei+He%2C+Nian+Liu%2C+Xiaoxin+He%2C+Kun+Wang%2C+and+Bryan+Hooi%2E+2025%2E+Fidelis%3A+Faithful+reasoning+in+large+language+models+for+knowledge+graph+question+answering%2E+In+Findings+of+the+Association+fo)
63. Xiangguo Sun, Hong Cheng, Jia Li, Bo Liu, and Jihong Guan. 2023. All in one: Multi-task prompting for graph neural networks. In Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pages 2120–2131. [find](https://scholar.google.com/scholar?q=Xiangguo+Sun%2C+Hong+Cheng%2C+Jia+Li%2C+Bo+Liu%2C+and+Jihong+Guan%2E+2023%2E+All+in+one%3A+Multi%2Dtask+prompting+for+graph+neural+networks%2E+In+Proceedings+of+the+29th+ACM+SIGKDD+Conference+on+Knowledge+Discovery+and)
64. Yuanfu Sun, Zhengnan Ma, Yi Fang, Jing Ma, and Qiaoyu Tan. 2025. Graphicl: Unlocking graph learning potential in llms through structured prompt design. arXiv preprint arXiv:2501.15755. [find](https://scholar.google.com/scholar?q=Yuanfu+Sun%2C+Zhengnan+Ma%2C+Yi+Fang%2C+Jing+Ma%2C+and+Qiaoyu+Tan%2E+2025%2E+Graphicl%3A+Unlocking+graph+learning+potential+in+llms+through+structured+prompt+design%2E+arXiv+preprint+arXiv%3A2501%2E15755%2E)
65. Jiabin Tang, Yuhao Yang, Wei Wei, Lei Shi, Lixin Su, Suqi Cheng, Dawei Yin, and Chao Huang. 2024a. Graphgpt: Graph instruction tuning for large language models. In Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 491–500. [find](https://scholar.google.com/scholar?q=Jiabin+Tang%2C+Yuhao+Yang%2C+Wei+Wei%2C+Lei+Shi%2C+Lixin+Su%2C+Suqi+Cheng%2C+Dawei+Yin%2C+and+Chao+Huang%2E+2024a%2E+Graphgpt%3A+Graph+instruction+tuning+for+large+language+models%2E+In+Proceedings+of+the+47th+Internationa)
66. Jianheng Tang, Qifan Zhang, Yuhan Li, and Jia Li. 2024b. Grapharena: Benchmarking large language models on graph computational problems. arXiv preprint arXiv:2407.00379. — [arXiv:2407.00379](https://arxiv.org/abs/2407.00379) [find](https://scholar.google.com/scholar?q=Jianheng+Tang%2C+Qifan+Zhang%2C+Yuhan+Li%2C+and+Jia+Li%2E+2024b%2E+Grapharena%3A+Benchmarking+large+language+models+on+graph+computational+problems%2E+arXiv+preprint+arXiv%3A2407%2E00379%2E+%E2%80%94+%5BarXiv%3A2407%2E00379%5D%28https%3A%2F%2Fa)
67. Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, and 1 others. 2023. [Llama: Open and efficient foundation language models](https://md.research.elamur.ai/free/en/papers/touvron-2023-llama-open-efficient-foundation-language/). arXiv preprint arXiv:2302.13971.
68. Gido M Van de Ven and Andreas S Tolias. 2019. Three scenarios for continual learning. arXiv preprint arXiv:1904.07734. [find](https://scholar.google.com/scholar?q=Gido+M+Van+de+Ven+and+Andreas+S+Tolias%2E+2019%2E+Three+scenarios+for+continual+learning%2E+arXiv+preprint+arXiv%3A1904%2E07734%2E)
69. Petar Veličković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Lio, and Yoshua Bengio. 2017. [Graph attention networks](https://md.research.elamur.ai/free/en/papers/veli-vckovic-2017-graph-attention-networks/). arXiv preprint arXiv:1710.10903.
70. Jianing Wang, Junda Wu, Yupeng Hou, Yao Liu, Ming Gao, and Julian McAuley. 2024b. Instructgraph: Boosting large language models via graph-centric instruction tuning and preference alignment. arXiv preprint arXiv:2402.08785. [find](https://scholar.google.com/scholar?q=Jianing+Wang%2C+Junda+Wu%2C+Yupeng+Hou%2C+Yao+Liu%2C+Ming+Gao%2C+and+Julian+McAuley%2E+2024b%2E+Instructgraph%3A+Boosting+large+language+models+via+graph%2Dcentric+instruction+tuning+and+preference+alignment%2E+arXiv+pre)
71. Liang Wang, Nan Yang, Xiaolong Huang, Binxing Jiao, Linjun Yang, Daxin Jiang, Rangan Majumder, and Furu Wei. 2022. [Text embeddings by weakly-supervised contrastive pre-training](https://md.research.elamur.ai/free/en/papers/wang-2022-text-embeddings-weakly-supervised-contrastive/). arXiv preprint arXiv:2212.03533.
72. Liyuan Wang, Xingxing Zhang, Hang Su, and Jun Zhu. 2024c. A comprehensive survey of continual learning: Theory, method and application. IEEE Transactions on Pattern Analysis and Machine Intelligence. [find](https://scholar.google.com/scholar?q=Liyuan+Wang%2C+Xingxing+Zhang%2C+Hang+Su%2C+and+Jun+Zhu%2E+2024c%2E+A+comprehensive+survey+of+continual+learning%3A+Theory%2C+method+and+application%2E+IEEE+Transactions+on+Pattern+Analysis+and+Machine+Intelligence%2E)
73. Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, and 1 others. 2022. [Chain-of-thought prompting elicits reasoning in large language models](https://md.research.elamur.ai/free/en/papers/wei-2022-chain-thought-prompting-elicits-reasoning/). Advances in neural information processing systems, 35:24824– 24837.
74. Xixi Wu, Yifei Shen, Fangzhou Ge, Caihua Shan, Yizhu Jiao, Xiangguo Sun, and Hong Cheng. 2025. A comprehensive analysis on llm-based node classification algorithms. arXiv preprint arXiv:2502.00829. — [arXiv:2502.00829](https://arxiv.org/abs/2502.00829) [find](https://scholar.google.com/scholar?q=Xixi+Wu%2C+Yifei+Shen%2C+Fangzhou+Ge%2C+Caihua+Shan%2C+Yizhu+Jiao%2C+Xiangguo+Sun%2C+and+Hong+Cheng%2E+2025%2E+A+comprehensive+analysis+on+llm%2Dbased+node+classification+algorithms%2E+arXiv+preprint+arXiv%3A2502%2E00829%2E+%E2%80%94+)
75. Lianghao Xia, Ben Kao, and Chao Huang. 2024. Opengraph: Towards open graph foundation models. arXiv preprint arXiv:2403.01121. [find](https://scholar.google.com/scholar?q=Lianghao+Xia%2C+Ben+Kao%2C+and+Chao+Huang%2E+2024%2E+Opengraph%3A+Towards+open+graph+foundation+models%2E+arXiv+preprint+arXiv%3A2403%2E01121%2E)
76. Keyulu Xu, Weihua Hu, Jure Leskovec, and Stefanie Jegelka. 2018. [How powerful are graph neural networks?](https://md.research.elamur.ai/free/en/papers/xu-2018-powerful-graph-neural-networks/) arXiv preprint arXiv:1810.00826.
77. Hao Yan, Chaozhuo Li, Ruosong Long, Chao Yan, Jianan Zhao, Wenwen Zhuang, Jun Yin, Peiyan Zhang, Weihao Han, Hao Sun, and 1 others. 2023. A comprehensive study on text-attributed graphs: Benchmarking and rethinking. Advances in Neural Information Processing Systems, 36:17238–17264. [find](https://scholar.google.com/scholar?q=Hao+Yan%2C+Chaozhuo+Li%2C+Ruosong+Long%2C+Chao+Yan%2C+Jianan+Zhao%2C+Wenwen+Zhuang%2C+Jun+Yin%2C+Peiyan+Zhang%2C+Weihao+Han%2C+Hao+Sun%2C+and+1+others%2E+2023%2E+A+comprehensive+study+on+text%2Dattributed+graphs%3A+Benchmarking+)
78. An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, and 1 others. 2025. [Qwen3 technical report](https://md.research.elamur.ai/free/en/papers/yang-2025-qwen3-technical-report/). arXiv preprint arXiv:2505.09388.
79. Z Yang. 2019. [Xlnet: Generalized autoregressive pre-training for language understanding](https://md.research.elamur.ai/free/en/papers/yang-2019-xlnet-generalized-autoregressive-pretraining-language/). arXiv preprint arXiv:1906.08237.
80. Petar Velickovic, William Fedus, William L Hamilton, Pietro Liò, Yoshua Bengio, and R Devon Hjelm. 2019. Deep graph infomax. ICLR (Poster), 2(3):4. [find](https://scholar.google.com/scholar?q=Petar+Velickovic%2C+William+Fedus%2C+William+L+Hamilton%2C+Pietro+Li%C3%B2%2C+Yoshua+Bengio%2C+and+R+Devon+Hjelm%2E+2019%2E+Deep+graph+infomax%2E+ICLR+%28Poster%29%2C+2%283%29%3A4%2E)
81. Ruosong Ye, Caiqi Zhang, Runhui Wang, Shuyuan Xu, Yongfeng Zhang, and 1 others. 2023. Natural language is all a graph needs. arXiv preprint arXiv:2308.07134, 4(5):7. [find](https://scholar.google.com/scholar?q=Ruosong+Ye%2C+Caiqi+Zhang%2C+Runhui+Wang%2C+Shuyuan+Xu%2C+Yongfeng+Zhang%2C+and+1+others%2E+2023%2E+Natural+language+is+all+a+graph+needs%2E+arXiv+preprint+arXiv%3A2308%2E07134%2C+4%285%29%3A7%2E)
82. Heng Wang, Shangbin Feng, Tianxing He, Zhaoxuan Tan, Xiaochuang Han, and Yulia Tsvetkov. 2024a. Can language models solve graph problems in natural language? Advances in Neural Information Processing Systems, 36. [find](https://scholar.google.com/scholar?q=Heng+Wang%2C+Shangbin+Feng%2C+Tianxing+He%2C+Zhaoxuan+Tan%2C+Xiaochuang+Han%2C+and+Yulia+Tsvetkov%2E+2024a%2E+Can+language+models+solve+graph+problems+in+natural+language%3F+Advances+in+Neural+Information+Processing+)
83. Çağatay Yıldız, Nishaanth Kanna Ravichandran, Pr-ishruit Punia, Matthias Bethge, and Beyza Ermis. 2024. Investigating continual pretraining in large language models: Insights and implications. arXiv preprint arXiv:2402.17400. [find](https://scholar.google.com/scholar?q=%C3%87ag%CC%86atay+Y%C4%B1ld%C4%B1z%2C+Nishaanth+Kanna+Ravichandran%2C+Pr%2Dishruit+Punia%2C+Matthias+Bethge%2C+and+Beyza+Ermis%2E+2024%2E+Investigating+continual+pretraining+in+large+language+models%3A+Insights+and+implications%2E+arXiv+)
84. Chengxuan Ying, Tianle Cai, Shengjie Luo, Shuxin Zheng, Guolin Ke, Di He, Yanming Shen, and Tie-Yan Liu. 2021. Do transformers really perform badly for graph representation? Advances in neural information processing systems, 34:28877–28888. [find](https://scholar.google.com/scholar?q=Chengxuan+Ying%2C+Tianle+Cai%2C+Shengjie+Luo%2C+Shuxin+Zheng%2C+Guolin+Ke%2C+Di+He%2C+Yanming+Shen%2C+and+Tie%2DYan+Liu%2E+2021%2E+Do+transformers+really+perform+badly+for+graph+representation%3F+Advances+in+neural+informa)
85. Yuning You, Tianlong Chen, Yongduo Sui, Ting Chen, Zhangyang Wang, and Yang Shen. 2020. Graph contrastive learning with augmentations. Advances in neural information processing systems, 33:5812– 5823. [find](https://scholar.google.com/scholar?q=Yuning+You%2C+Tianlong+Chen%2C+Yongduo+Sui%2C+Ting+Chen%2C+Zhangyang+Wang%2C+and+Yang+Shen%2E+2020%2E+Graph+contrastive+learning+with+augmentations%2E+Advances+in+neural+information+processing+systems%2C+33%3A5812%E2%80%93+5823%2E)
86. Xingtong Yu, Yuan Fang, Zemin Liu, Yuxia Wu, Zhihao Wen, Jianyuan Bo, Xinming Zhang, and Steven CH Hoi. 2024. A survey of few-shot learning on graphs: from meta-learning to pre-training and prompt learning. arXiv preprint arXiv:2402.01440. — [arXiv:2402.01440](https://arxiv.org/abs/2402.01440) [find](https://scholar.google.com/scholar?q=Xingtong+Yu%2C+Yuan+Fang%2C+Zemin+Liu%2C+Yuxia+Wu%2C+Zhihao+Wen%2C+Jianyuan+Bo%2C+Xinming+Zhang%2C+and+Steven+CH+Hoi%2E+2024%2E+A+survey+of+few%2Dshot+learning+on+graphs%3A+from+meta%2Dlearning+to+pre%2Dtraining+and+prompt+lea)
87. Jiawei Zhang. 2023. Graph-toolformer: To empower llms with graph reasoning ability via prompt augmented by chatgpt. arXiv preprint arXiv:2304.11116. [find](https://scholar.google.com/scholar?q=Jiawei+Zhang%2E+2023%2E+Graph%2Dtoolformer%3A+To+empower+llms+with+graph+reasoning+ability+via+prompt+augmented+by+chatgpt%2E+arXiv+preprint+arXiv%3A2304%2E11116%2E)
88. Jiawei Zhang, Haopeng Zhang, Congying Xia, and Li Sun. 2020. Graph-bert: Only attention is needed for learning graph representations. arXiv preprint arXiv:2001.05140. — [arXiv:2001.05140](https://arxiv.org/abs/2001.05140) [find](https://scholar.google.com/scholar?q=Jiawei+Zhang%2C+Haopeng+Zhang%2C+Congying+Xia%2C+and+Li+Sun%2E+2020%2E+Graph%2Dbert%3A+Only+attention+is+needed+for+learning+graph+representations%2E+arXiv+preprint+arXiv%3A2001%2E05140%2E+%E2%80%94+%5BarXiv%3A2001%2E05140%5D%28https%3A%2F%2Farxi)
89. Jiong Zhang, Wei-Cheng Chang, Hsiang-Fu Yu, and Inderjit Dhillon. 2021. Fast multi-resolution transformer fine-tuning for extreme multi-label text classification. Advances in Neural Information Processing Systems, 34:7267–7280. [find](https://scholar.google.com/scholar?q=Jiong+Zhang%2C+Wei%2DCheng+Chang%2C+Hsiang%2DFu+Yu%2C+and+Inderjit+Dhillon%2E+2021%2E+Fast+multi%2Dresolution+transformer+fine%2Dtuning+for+extreme+multi%2Dlabel+text+classification%2E+Advances+in+Neural+Information+Proces)
90. Siwei Zhang, Yun Xiong, Yateng Tang, Xi Chen, Zian Jia, Zehao Gu, Jiarong Xu, and Jiawei Zhang. 2025a. Unifying text semantics and graph structures for temporal text-attributed graphs with large language models. arXiv preprint arXiv:2503.14411. — [arXiv:2503.14411](https://arxiv.org/abs/2503.14411) [find](https://scholar.google.com/scholar?q=Siwei+Zhang%2C+Yun+Xiong%2C+Yateng+Tang%2C+Xi+Chen%2C+Zian+Jia%2C+Zehao+Gu%2C+Jiarong+Xu%2C+and+Jiawei+Zhang%2E+2025a%2E+Unifying+text+semantics+and+graph+structures+for+temporal+text%2Dattributed+graphs+with+large+langu)
91. Yizhuo Zhang, Heng Wang, Shangbin Feng, Zhaoxuan Tan, Xiaochuang Han, Tianxing He, and Yulia Tsvetkov. 2024a. Can llm graph reasoning generalize beyond pattern memorization? arXiv preprint arXiv:2406.15992. [find](https://scholar.google.com/scholar?q=Yizhuo+Zhang%2C+Heng+Wang%2C+Shangbin+Feng%2C+Zhaoxuan+Tan%2C+Xiaochuang+Han%2C+Tianxing+He%2C+and+Yulia+Tsvetkov%2E+2024a%2E+Can+llm+graph+reasoning+generalize+beyond+pattern+memorization%3F+arXiv+preprint+arXiv%3A2406%2E)
92. Zeyang Zhang, Xin Wang, Ziwei Zhang, Haoyang Li, Yijian Qin, and Wenwu Zhu. 2024b. Llm4dyg: can large language models solve spatial-temporal problems on dynamic graphs? In Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pages 4350–4361. [find](https://scholar.google.com/scholar?q=Zeyang+Zhang%2C+Xin+Wang%2C+Ziwei+Zhang%2C+Haoyang+Li%2C+Yijian+Qin%2C+and+Wenwu+Zhu%2E+2024b%2E+Llm4dyg%3A+can+large+language+models+solve+spatial%2Dtemporal+problems+on+dynamic+graphs%3F+In+Proceedings+of+the+30th+ACM+)
93. Zhihan Zhang, Xunkai Li, Guang Zeng, Hongchao Qin, Ronghua Li, and Guoren Wang. 2025b. Rethinking graph structure learning in the era of llms. arXiv preprint arXiv:2503.21223. — [arXiv:2503.21223](https://arxiv.org/abs/2503.21223) [find](https://scholar.google.com/scholar?q=Zhihan+Zhang%2C+Xunkai+Li%2C+Guang+Zeng%2C+Hongchao+Qin%2C+Ronghua+Li%2C+and+Guoren+Wang%2E+2025b%2E+Rethinking+graph+structure+learning+in+the+era+of+llms%2E+arXiv+preprint+arXiv%3A2503%2E21223%2E+%E2%80%94+%5BarXiv%3A2503%2E21223%5D%28htt)
94. Huanjing Zhao, Beining Yang, Yukuo Cen, Junyu Ren, Chenhui Zhang, Yuxiao Dong, Evgeny Kharlamov, Shu Zhao, and Jie Tang. 2024. Pre-training and prompting for few-shot node classification on text-attributed graphs. In Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Minin… [find](https://scholar.google.com/scholar?q=Huanjing+Zhao%2C+Beining+Yang%2C+Yukuo+Cen%2C+Junyu+Ren%2C+Chenhui+Zhang%2C+Yuxiao+Dong%2C+Evgeny+Kharlamov%2C+Shu+Zhao%2C+and+Jie+Tang%2E+2024%2E+Pre%2Dtraining+and+prompting+for+few%2Dshot+node+classification+on+text%2Dattri)
95. Jianan Zhao, Le Zhuo, Yikang Shen, Meng Qu, Kai Liu, Michael Bronstein, Zhaocheng Zhu, and Jian Tang. 2023. Graphtext: Graph reasoning in text space. arXiv preprint arXiv:2310.01089. — [arXiv:2310.01089](https://arxiv.org/abs/2310.01089) [find](https://scholar.google.com/scholar?q=Jianan+Zhao%2C+Le+Zhuo%2C+Yikang+Shen%2C+Meng+Qu%2C+Kai+Liu%2C+Michael+Bronstein%2C+Zhaocheng+Zhu%2C+and+Jian+Tang%2E+2023%2E+Graphtext%3A+Graph+reasoning+in+text+space%2E+arXiv+preprint+arXiv%3A2310%2E01089%2E+%E2%80%94+%5BarXiv%3A2310%2E010)
96. Huachi Zhou, Jiahe Du, Chuang Zhou, Chang Yang, Yilin Xiao, Yuxuan Xie, and Xiao Huang. 2025. Each graph is a new language: Graph learning with llms. In Findings of the Association for Computational Linguistics: ACL 2025, pages 17548–17559. [find](https://scholar.google.com/scholar?q=Huachi+Zhou%2C+Jiahe+Du%2C+Chuang+Zhou%2C+Chang+Yang%2C+Yilin+Xiao%2C+Yuxuan+Xie%2C+and+Xiao+Huang%2E+2025%2E+Each+graph+is+a+new+language%3A+Graph+learning+with+llms%2E+In+Findings+of+the+Association+for+Computational+L)
97. Kerui Zhu, Bo-Wei Huang, Bowen Jin, Yizhu Jiao, Ming Zhong, Kevin Chang, Shou-De Lin, and Jiawei Han. 2024. Investigating instruction tuning large language models on graphs. arXiv preprint arXiv:2408.05457. — [arXiv:2408.05457](https://arxiv.org/abs/2408.05457) [find](https://scholar.google.com/scholar?q=Kerui+Zhu%2C+Bo%2DWei+Huang%2C+Bowen+Jin%2C+Yizhu+Jiao%2C+Ming+Zhong%2C+Kevin+Chang%2C+Shou%2DDe+Lin%2C+and+Jiawei+Han%2E+2024%2E+Investigating+instruction+tuning+large+language+models+on+graphs%2E+arXiv+preprint+arXiv%3A2408%2E)
98. Chenyi Zi, Haihong Zhao, Xiangguo Sun, Yiqing Lin, Hong Cheng, and Jia Li. 2024. Prog: A graph prompt learning benchmark. the Thirty-Eighth Advances in Neural Information Processing Systems (NeurIPS 2024). [find](https://scholar.google.com/scholar?q=Chenyi+Zi%2C+Haihong+Zhao%2C+Xiangguo+Sun%2C+Yiqing+Lin%2C+Hong+Cheng%2C+and+Jia+Li%2E+2024%2E+Prog%3A+A+graph+prompt+learning+benchmark%2E+the+Thirty%2DEighth+Advances+in+Neural+Information+Processing+Systems+%28NeurIPS+2)

*Updated 31 October 2026*