---
title: "Extracting Low-/High- Frequency Knowledge from Graph Neural"
description: "To counter this, they propose FF-G2M (Full-Frequency GNN-to-MLP). 47 references, cited by 4 papers here."
authors: ["Lirong Wu; Haitao Lin; Yufei Huang; Tianyu Fan;", "Stan Z. Li"]
year: 2023
venue: "arXiv:2305.10758"
source: "https://arxiv.org/abs/2305.10758"
doi: "10.48550/arxiv.2305.10758"
arxiv: "2305.10758"
licence: "CC BY 4.0 (arXiv)"
licence_url: "http://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/wu-2023-extracting-low-high-frequency-knowledge/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/wu-2023-extracting-low-high-frequency-knowledge/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2023](https://md.research.elamur.ai/free/en/papers/2023/page/12/) / Extracting Low-/High- Frequency Knowledge from Graph Neural Networks and Injecting it into MLPs: An Effective GNN-to-MLP Distillation Framework

# Extracting Low-/High- Frequency Knowledge from Graph Neural Networks and Injecting it into MLPs: An Effective GNN-to-MLP Distillation Framework

**Authors:** Lirong Wu; Haitao Lin; Yufei Huang; Tianyu Fan; and [Stan Z. Li](https://md.research.elamur.ai/free/en/authors/stan-ziqing-li/) · **Year:** 2023 · **Venue:** [arXiv:2305.10758](https://md.research.elamur.ai/free/en/venues/proceedings-of-the-aaai-conference-on-artificial-intelligence/) ([Proceedings of the AAAI Conference on Artificial Intelligence 2023](https://md.research.elamur.ai/free/en/venues/proceedings-of-the-aaai-conference-on-artificial-intelligence/2023/)) · **Publisher:** [Association for the Advancement of Artificial Intelligence](https://md.research.elamur.ai/free/en/publishers/association-for-the-advancement-of-artificial-intelligence/) · **Licence:** [CC BY 4.0 (arXiv)](http://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://arxiv.org/abs/2305.10758) · [PDF](https://arxiv.org/pdf/2305.10758) · [DOI 10.48550/arxiv.2305.10758](https://doi.org/10.48550/arxiv.2305.10758) · [arXiv 2305.10758](https://arxiv.org/abs/2305.10758) · [OpenAlex](https://openalex.org/W4382202907)

**Citations:** 31 citations (OpenAlex, 2026-10-14) · 4 in our library, of the 149,439 papers whose bibliographies we read

## Summary
Graph Neural Networks (GNNs) classify nodes well, but their reliance on neighbourhood message passing makes inference slow. MLPs are fast but less accurate. GNN-to-MLP distillation tries to bridge this gap, and the authors ask which parts of a teacher GNN's knowledge actually reach the student MLP.

Using graph signal processing, they split GNN knowledge into a low-frequency part (smoothness among neighbours) and a high-frequency part (differences between connected nodes). They then link these spectral components to spatial interpretations. Their analysis reveals an "information drowning" effect: in standard node-to-node distillation such as GLNN, the dominant low-frequency signal tends to swamp the high-frequency signal.

To counter this, they propose FF-G2M (Full-Frequency GNN-to-MLP). It pairs a topology-aware low-frequency distillation loss with a high-frequency loss that keeps teacher-student pairwise node relations consistent.

Across six benchmark graphs and three GNN backbones, the distilled MLPs exceed both plain MLPs and their own teachers. The result shows that fast, graph-free inference need not cost accuracy.

## Key points
- Average gains: +12.6% over vanilla MLPs and +2.6% over the corresponding teacher GNNs, computed across six datasets and three GNN architectures.
- Datasets: Cora, Citeseer, Pubmed, Amazon-Photo, Coauthor-CS, Coauthor-Phy.
- With a GCN teacher on Cora, accuracy is 84.3% for FF-G2M, compared with 82.8% for GLNN, 82.2% for GCN and 59.7% for the vanilla MLP.
- In ablations, low-frequency distillation (L_LFD) carries most of the benefit and high-frequency distillation (L_HFD) plays a supporting role. Combining the two works better than using either alone.
- The paper appeared at AAAI 2023, and the code is released at github.com/LirongWu/FF-G2M.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Zhejiang University](https://md.research.elamur.ai/free/en/universities/zhejiang-university/) — University, Hangzhou, China · 236 works · 14,072 citations
- [Westlake University](https://md.research.elamur.ai/free/en/universities/westlake-university/) — University, Hangzhou, China · 60 works · 8,829 citations

📚 Shelves (3): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 19,503 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 18,805 works · [👁️ Computer Vision and Pattern Recognition](https://md.research.elamur.ai/free/en/shelves/computer-science/computer-vision-and-pattern-recognition/) · 3,621 works

🧭 Topics (1): [🪶 Efficient models & compression](https://md.research.elamur.ai/free/en/topics/efficient-models/) · 328 works

## Cite

Cite as: Lirong Wu; Haitao Lin; Yufei Huang; Tianyu Fan; and Stan Z. Li. 2023. Extracting Low-/High- Frequency Knowledge from Graph Neural Networks and Injecting it into MLPs: An Effective GNN-to-MLP Distillation Framework. arXiv:2305.10758.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@misc{fan2023extracting,
  title = {{Extracting Low-/High- Frequency Knowledge from Graph Neural Networks and Injecting it into MLPs: An Effective GNN-to-MLP Distillation Framework}},
  author = {Lirong Wu; Haitao Lin; Yufei Huang; Tianyu Fan; and Stan Z. Li},
  year = {2023},
  doi = {10.48550/arxiv.2305.10758},
  eprint = {2305.10758},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2305.10758}
}
```

- [← previous Efficient Online Data Mixing For Language Model Pre-Training](https://md.research.elamur.ai/free/en/papers/albalak-2023-efficient-online-data-mixing-language/)
- [next → Knowledge Card: Filling LLMs' Knowledge Gaps with Plug-in Specialized Language Models](https://md.research.elamur.ai/free/en/papers/feng-2023-knowledge-card-filling-llms-knowledge/)
- [↑ back to papers of 2023](https://md.research.elamur.ai/free/en/papers/2023/page/12/)

## References

47 references, 8 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Bo, D.; Wang, X.; Shi, C.; and Shen, H. 2021a. Beyond Low-frequency Information in Graph Convolutional Networks. arXiv preprint arXiv:2101.00797. [find](https://scholar.google.com/scholar?q=Bo%2C+D%2E%3B+Wang%2C+X%2E%3B+Shi%2C+C%2E%3B+and+Shen%2C+H%2E+2021a%2E+Beyond+Low%2Dfrequency+Information+in+Graph+Convolutional+Networks%2E+arXiv+preprint+arXiv%3A2101%2E00797%2E)
2. Bo, D.; Wang, X.; Shi, C.; and Shen, H. 2021b. Beyond Low-frequency Information in Graph Convolutional Networks. Association for the Advance of Artificial Intelligence (AAAI). [find](https://scholar.google.com/scholar?q=Bo%2C+D%2E%3B+Wang%2C+X%2E%3B+Shi%2C+C%2E%3B+and+Shen%2C+H%2E+2021b%2E+Beyond+Low%2Dfrequency+Information+in+Graph+Convolutional+Networks%2E+Association+for+the+Advance+of+Artificial+Intelligence+%28AAAI%29%2E)
3. Bruna, J.; Zaremba, W.; Szlam, A.; and LeCun, Y. 2013. Spectral networks and locally connected networks on graphs. arXiv preprint arXiv:1312.6203. [find](https://scholar.google.com/scholar?q=Bruna%2C+J%2E%3B+Zaremba%2C+W%2E%3B+Szlam%2C+A%2E%3B+and+LeCun%2C+Y%2E+2013%2E+Spectral+networks+and+locally+connected+networks+on+graphs%2E+arXiv+preprint+arXiv%3A1312%2E6203%2E)
4. Chien, E.; Peng, J.; Li, P.; and Milenkovic, O. 2021. Adaptive Universal Generalized PageRank Graph Neural Network. International Conference on Learning Representations (ICLR). [find](https://scholar.google.com/scholar?q=Chien%2C+E%2E%3B+Peng%2C+J%2E%3B+Li%2C+P%2E%3B+and+Milenkovic%2C+O%2E+2021%2E+Adaptive+Universal+Generalized+PageRank+Graph+Neural+Network%2E+International+Conference+on+Learning+Representations+%28ICLR%29%2E)
5. Chung, F. R.; and Graham, F. C. 1997. Spectral graph theory. [find](https://scholar.google.com/scholar?q=Chung%2C+F%2E+R%2E%3B+and+Graham%2C+F%2E+C%2E+1997%2E+Spectral+graph+theory%2E)
6. American Mathematical Soc. Defferrard, M.; Bresson, X.; and Vandergheynst, P. 2016. [Convolutional neural networks on graphs with fast localized spectral filtering](https://md.research.elamur.ai/free/en/papers/defferrard-2016-convolutional-neural-networks-graphs-fast/). arXiv preprint arXiv:1606.09375.
7. Errica, F.; Podda, M.; Bacciu, D.; and Micheli, A. 2019. A fair comparison of graph neural networks for graph classification. arXiv preprint arXiv:1912.09893. — [arXiv:1912.09893](https://arxiv.org/abs/1912.09893) [find](https://scholar.google.com/scholar?q=Errica%2C+F%2E%3B+Podda%2C+M%2E%3B+Bacciu%2C+D%2E%3B+and+Micheli%2C+A%2E+2019%2E+A+fair+comparison+of+graph+neural+networks+for+graph+classification%2E+arXiv+preprint+arXiv%3A1912%2E09893%2E+%E2%80%94+%5BarXiv%3A1912%2E09893%5D%28https%3A%2F%2Farxiv%2Eorg%2Fab)
8. Fan, W.; Ma, Y.; Li, Q.; He, Y.; Zhao, E.; Tang, J.; and Yin, D. 2019. Graph neural networks for social recommendation. In The world wide web conference, 417–426. [find](https://scholar.google.com/scholar?q=Fan%2C+W%2E%3B+Ma%2C+Y%2E%3B+Li%2C+Q%2E%3B+He%2C+Y%2E%3B+Zhao%2C+E%2E%3B+Tang%2C+J%2E%3B+and+Yin%2C+D%2E+2019%2E+Graph+neural+networks+for+social+recommendation%2E+In+The+world+wide+web+conference%2C+417%E2%80%93426%2E)
9. Ghorbani, M.; Bahrami, M.; Kazi, A.; Soleymani Baghshah, M.; Rabiee, H. R.; and Navab, N. 2021. GKD: Semi-supervised Graph Knowledge Distillation for Graph-Independent Inference. In International Conference on Medical Image Computing and Computer-Assisted Intervention, 709–718. [find](https://scholar.google.com/scholar?q=Ghorbani%2C+M%2E%3B+Bahrami%2C+M%2E%3B+Kazi%2C+A%2E%3B+Soleymani+Baghshah%2C+M%2E%3B+Rabiee%2C+H%2E+R%2E%3B+and+Navab%2C+N%2E+2021%2E+GKD%3A+Semi%2Dsupervised+Graph+Knowledge+Distillation+for+Graph%2DIndependent+Inference%2E+In+International+Conf)
10. Springer. Giles, C. L.; Bollacker, K. D.; and Lawrence, S. 1998. Cite-Seer: An automatic citation indexing system. In Proceedings of the third ACM conference on Digital libraries, 89–98. [find](https://scholar.google.com/scholar?q=Springer%2E+Giles%2C+C%2E+L%2E%3B+Bollacker%2C+K%2E+D%2E%3B+and+Lawrence%2C+S%2E+1998%2E+Cite%2DSeer%3A+An+automatic+citation+indexing+system%2E+In+Proceedings+of+the+third+ACM+conference+on+Digital+libraries%2C+89%E2%80%9398%2E)
11. Gou, J.; Yu, B.; Maybank, S. J.; and Tao, D. 2021. Knowledge distillation: A survey. International Journal of Computer Vision, 129(6): 1789–1819. [find](https://scholar.google.com/scholar?q=Gou%2C+J%2E%3B+Yu%2C+B%2E%3B+Maybank%2C+S%2E+J%2E%3B+and+Tao%2C+D%2E+2021%2E+Knowledge+distillation%3A+A+survey%2E+International+Journal+of+Computer+Vision%2C+129%286%29%3A+1789%E2%80%931819%2E)
12. Hamilton, W.; Ying, Z.; and Leskovec, J. 2017. [Inductive representation learning on large graphs](https://md.research.elamur.ai/free/en/papers/hamilton-2017-inductive-representation-learning-large-graphs/). In Advances in neural information processing systems, 1024–1034.
13. Hinton, G.; Vinyals, O.; Dean, J.; et al. 2015. [Distilling the knowledge in a neural network](https://md.research.elamur.ai/free/en/papers/hinton-2015-distilling-knowledge-neural-network/). arXiv preprint arXiv:1503.02531, 2(7).
14. Iscen, A.; Tolias, G.; Avrithis, Y.; and Chum, O. 2019. Label propagation for deep semi-supervised learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 5070–5079. [find](https://scholar.google.com/scholar?q=Iscen%2C+A%2E%3B+Tolias%2C+G%2E%3B+Avrithis%2C+Y%2E%3B+and+Chum%2C+O%2E+2019%2E+Label+propagation+for+deep+semi%2Dsupervised+learning%2E+In+Proceedings+of+the+IEEE%2FCVF+Conference+on+Computer+Vision+and+Pattern+Recognition%2C+5070%E2%80%93)
15. Jia, Z.; Lin, S.; Ying, R.; You, J.; Leskovec, J.; and Aiken, A. 2020. Redundancy-Free Computation for Graph Neural Networks. In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, 997–1005. [find](https://scholar.google.com/scholar?q=Jia%2C+Z%2E%3B+Lin%2C+S%2E%3B+Ying%2C+R%2E%3B+You%2C+J%2E%3B+Leskovec%2C+J%2E%3B+and+Aiken%2C+A%2E+2020%2E+Redundancy%2DFree+Computation+for+Graph+Neural+Networks%2E+In+Proceedings+of+the+26th+ACM+SIGKDD+International+Conference+on+Knowledg)
16. Joshi, C. K.; Liu, F.; Xun, X.; Lin, J.; and Foo, C.-S. 2021. On Representation Knowledge Distillation for Graph Neural Networks. arXiv preprint arXiv:2111.04964. — [arXiv:2111.04964](https://arxiv.org/abs/2111.04964) [find](https://scholar.google.com/scholar?q=Joshi%2C+C%2E+K%2E%3B+Liu%2C+F%2E%3B+Xun%2C+X%2E%3B+Lin%2C+J%2E%3B+and+Foo%2C+C%2E%2DS%2E+2021%2E+On+Representation+Knowledge+Distillation+for+Graph+Neural+Networks%2E+arXiv+preprint+arXiv%3A2111%2E04964%2E+%E2%80%94+%5BarXiv%3A2111%2E04964%5D%28https%3A%2F%2Farxiv%2Eor)
17. Kim, T.; Oh, J.; Kim, N.; Cho, S.; and Yun, S.-Y. 2021. Comparing kullback-leibler divergence and mean squared error loss in knowledge distillation. arXiv preprint arXiv:2105.08919. — [arXiv:2105.08919](https://arxiv.org/abs/2105.08919) [find](https://scholar.google.com/scholar?q=Kim%2C+T%2E%3B+Oh%2C+J%2E%3B+Kim%2C+N%2E%3B+Cho%2C+S%2E%3B+and+Yun%2C+S%2E%2DY%2E+2021%2E+Comparing+kullback%2Dleibler+divergence+and+mean+squared+error+loss+in+knowledge+distillation%2E+arXiv+preprint+arXiv%3A2105%2E08919%2E+%E2%80%94+%5BarXiv%3A2105%2E0891)
18. Kipf, T. N.; and Welling, M. 2016. [Semi-supervised classification with graph convolutional networks](https://md.research.elamur.ai/free/en/papers/kipf-2016-semi-supervised-classification-graph-convolutional/). arXiv preprint arXiv:1609.02907.
19. Lassance, C.; Bontonou, M.; Hacene, G. B.; Gripon, V.; Tang, J.; and Ortega, A. 2020. Deep geometric knowledge distillation with graphs. In ICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 8484–8488. IEEE. [find](https://scholar.google.com/scholar?q=Lassance%2C+C%2E%3B+Bontonou%2C+M%2E%3B+Hacene%2C+G%2E+B%2E%3B+Gripon%2C+V%2E%3B+Tang%2C+J%2E%3B+and+Ortega%2C+A%2E+2020%2E+Deep+geometric+knowledge+distillation+with+graphs%2E+In+ICASSP+2020%2D2020+IEEE+International+Conference+on+Acoustics%2C)
20. Liu, M.; Gao, H.; and Ji, S. 2020. Towards deeper graph neural networks. In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, 338–348. [find](https://scholar.google.com/scholar?q=Liu%2C+M%2E%3B+Gao%2C+H%2E%3B+and+Ji%2C+S%2E+2020%2E+Towards+deeper+graph+neural+networks%2E+In+Proceedings+of+the+26th+ACM+SIGKDD+International+Conference+on+Knowledge+Discovery+%26+Data+Mining%2C+338%E2%80%93348%2E)
21. Liu, Z.; Luo, Y.; Wu, L.; Li, S.; Liu, Z.; and Li, S. Z. 2022. Are Gradients on Graph Structure Reliable in Gray-box Attacks? arXiv preprint arXiv:2208.05514. — [arXiv:2208.05514](https://arxiv.org/abs/2208.05514) [find](https://scholar.google.com/scholar?q=Liu%2C+Z%2E%3B+Luo%2C+Y%2E%3B+Wu%2C+L%2E%3B+Li%2C+S%2E%3B+Liu%2C+Z%2E%3B+and+Li%2C+S%2E+Z%2E+2022%2E+Are+Gradients+on+Graph+Structure+Reliable+in+Gray%2Dbox+Attacks%3F+arXiv+preprint+arXiv%3A2208%2E05514%2E+%E2%80%94+%5BarXiv%3A2208%2E05514%5D%28https%3A%2F%2Farxiv%2Eorg%2Fab)
22. Luo, Y.; Chen, A.; Yan, K.; and Tian, L. 2021. Distilling Self-Knowledge From Contrastive Links to Classify Graph Nodes Without Passing Messages. arXiv preprint arXiv:2106.08541. — [arXiv:2106.08541](https://arxiv.org/abs/2106.08541) [find](https://scholar.google.com/scholar?q=Luo%2C+Y%2E%3B+Chen%2C+A%2E%3B+Yan%2C+K%2E%3B+and+Tian%2C+L%2E+2021%2E+Distilling+Self%2DKnowledge+From+Contrastive+Links+to+Classify+Graph+Nodes+Without+Passing+Messages%2E+arXiv+preprint+arXiv%3A2106%2E08541%2E+%E2%80%94+%5BarXiv%3A2106%2E08541%5D%28)
23. McCallum, A. K.; Nigam, K.; Rennie, J.; and Seymore, K. 2000. Automating the construction of internet portals with machine learning. Information Retrieval, 3(2): 127–163. [find](https://scholar.google.com/scholar?q=McCallum%2C+A%2E+K%2E%3B+Nigam%2C+K%2E%3B+Rennie%2C+J%2E%3B+and+Seymore%2C+K%2E+2000%2E+Automating+the+construction+of+internet+portals+with+machine+learning%2E+Information+Retrieval%2C+3%282%29%3A+127%E2%80%93163%2E)
24. McInnes, L.; Healy, J.; and Melville, J. 2018. [Umap: Uniform manifold approximation and projection for dimension reduction](https://md.research.elamur.ai/free/en/papers/mcinnes-2018-umap-uniform-manifold-approximation-projection/). arXiv preprint arXiv:1802.03426.
25. Pei, H.; Wei, B.; Chang, K. C.-C.; Lei, Y.; and Yang, B. 2020. Geom-gcn: Geometric graph convolutional networks. arXiv preprint arXiv:2002.05287. — [arXiv:2002.05287](https://arxiv.org/abs/2002.05287) [find](https://scholar.google.com/scholar?q=Pei%2C+H%2E%3B+Wei%2C+B%2E%3B+Chang%2C+K%2E+C%2E%2DC%2E%3B+Lei%2C+Y%2E%3B+and+Yang%2C+B%2E+2020%2E+Geom%2Dgcn%3A+Geometric+graph+convolutional+networks%2E+arXiv+preprint+arXiv%3A2002%2E05287%2E+%E2%80%94+%5BarXiv%3A2002%2E05287%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2002%2E05287%29)
26. Ren, Y.; Ji, J.; Niu, L.; and Lei, M. 2021. Multi-task Self-distillation for Graph-based Semi-Supervised Learning. arXiv preprint arXiv:2112.01174. — [arXiv:2112.01174](https://arxiv.org/abs/2112.01174) [find](https://scholar.google.com/scholar?q=Ren%2C+Y%2E%3B+Ji%2C+J%2E%3B+Niu%2C+L%2E%3B+and+Lei%2C+M%2E+2021%2E+Multi%2Dtask+Self%2Ddistillation+for+Graph%2Dbased+Semi%2DSupervised+Learning%2E+arXiv+preprint+arXiv%3A2112%2E01174%2E+%E2%80%94+%5BarXiv%3A2112%2E01174%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2112%2E0117)
27. Ricaud, B.; Borgnat, P.; Tremblay, N.; Gonçalves, P.; and Vandergheynst, P. 2019. Fourier could be a data scientist: From graph Fourier transform to signal processing on graphs. Comptes Rendus Physique, 20(5): 474–488. [find](https://scholar.google.com/scholar?q=Ricaud%2C+B%2E%3B+Borgnat%2C+P%2E%3B+Tremblay%2C+N%2E%3B+Gonc%CC%A7alves%2C+P%2E%3B+and+Vandergheynst%2C+P%2E+2019%2E+Fourier+could+be+a+data+scientist%3A+From+graph+Fourier+transform+to+signal+processing+on+graphs%2E+Comptes+Rendus+Physiq)
28. Sandryhaila, A.; and Moura, J. M. 2013. Discrete signal processing on graphs: Graph fourier transform. In 2013 IEEE International Conference on Acoustics, Speech and Signal Processing, 6167–6170. IEEE. [find](https://scholar.google.com/scholar?q=Sandryhaila%2C+A%2E%3B+and+Moura%2C+J%2E+M%2E+2013%2E+Discrete+signal+processing+on+graphs%3A+Graph+fourier+transform%2E+In+2013+IEEE+International+Conference+on+Acoustics%2C+Speech+and+Signal+Processing%2C+6167%E2%80%936170%2E+IEEE)
29. Sen, P.; Namata, G.; Bilgic, M.; Getoor, L.; Galligher, B.; and Eliassi-Rad, T. 2008. [Collective classification in network data](https://md.research.elamur.ai/free/en/papers/sen-2008-collective-classification-network-data/). AI magazine, 29(3): 93–93.
30. Shchur, O.; Mumme, M.; Bojchevski, A.; and Günnemann, S. 2018. Pitfalls of graph neural network evaluation. arXiv preprint arXiv:1811.05868. — [arXiv:1811.05868](https://arxiv.org/abs/1811.05868) [find](https://scholar.google.com/scholar?q=Shchur%2C+O%2E%3B+Mumme%2C+M%2E%3B+Bojchevski%2C+A%2E%3B+and+Gu%CC%88nnemann%2C+S%2E+2018%2E+Pitfalls+of+graph+neural+network+evaluation%2E+arXiv+preprint+arXiv%3A1811%2E05868%2E+%E2%80%94+%5BarXiv%3A1811%2E05868%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F1811%2E05868%29)
31. Shuman, D. I.; Narang, S. K.; Frossard, P.; Ortega, A.; and Vandergheynst, P. 2013. [The emerging field of signal processing on graphs: Extending high-dimensional data analysis to networks and other irregular domains](https://md.research.elamur.ai/free/en/papers/veli-vckovic-2017-graph-attention-networks/). IEEE signal processing magazine, 30(3): 83–98. Veličković, P.; Cucurull, G.; Casa…
32. Wang, M.; Zheng, D.; Ye, Z.; Gan, Q.; Li, M.; Song, X.; Zhou, J.; Ma, C.; Yu, L.; Gai, Y.; He, T.; Karypis, G.; Li, J.; and Zhang, Z. 2019. Deep Graph Library: A Graph-Centric, Highly-Performant Package for Graph Neural Networks. arXiv preprint arXiv:1909.01315. — [arXiv:1909.01315](https://arxiv.org/abs/1909.01315) [find](https://scholar.google.com/scholar?q=Wang%2C+M%2E%3B+Zheng%2C+D%2E%3B+Ye%2C+Z%2E%3B+Gan%2C+Q%2E%3B+Li%2C+M%2E%3B+Song%2C+X%2E%3B+Zhou%2C+J%2E%3B+Ma%2C+C%2E%3B+Yu%2C+L%2E%3B+Gai%2C+Y%2E%3B+He%2C+T%2E%3B+Karypis%2C+G%2E%3B+Li%2C+J%2E%3B+and+Zhang%2C+Z%2E+2019%2E+Deep+Graph+Library%3A+A+Graph%2DCentric%2C+Highly%2DPerformant+Packa)
33. Wu, F.; Souza, A.; Zhang, T.; Fifty, C.; Yu, T.; and Weinberger, K. 2019. Simplifying graph convolutional networks. In International conference on machine learning, 6861– 6871. [find](https://scholar.google.com/scholar?q=Wu%2C+F%2E%3B+Souza%2C+A%2E%3B+Zhang%2C+T%2E%3B+Fifty%2C+C%2E%3B+Yu%2C+T%2E%3B+and+Weinberger%2C+K%2E+2019%2E+Simplifying+graph+convolutional+networks%2E+In+International+conference+on+machine+learning%2C+6861%E2%80%93+6871%2E)
34. PMLR. Wu, L.; Lin, H.; Gao, Z.; Tan, C.; Li, S.; et al. 2021a. GraphMixup: Improving Class-Imbalanced Node Classification on Graphs by Self-supervised Context Prediction. arXiv preprint arXiv:2106.11133. — [arXiv:2106.11133](https://arxiv.org/abs/2106.11133) [find](https://scholar.google.com/scholar?q=PMLR%2E+Wu%2C+L%2E%3B+Lin%2C+H%2E%3B+Gao%2C+Z%2E%3B+Tan%2C+C%2E%3B+Li%2C+S%2E%3B+et+al%2E+2021a%2E+GraphMixup%3A+Improving+Class%2DImbalanced+Node+Classification+on+Graphs+by+Self%2Dsupervised+Context+Prediction%2E+arXiv+preprint+arXiv%3A2106%2E111)
35. Wu, L.; Lin, H.; Gao, Z.; Tan, C.; Li, S.; et al. 2021b. Self-supervised on Graphs: Contrastive, Generative, or Predictive. arXiv preprint arXiv:2105.07342. — [arXiv:2105.07342](https://arxiv.org/abs/2105.07342) [find](https://scholar.google.com/scholar?q=Wu%2C+L%2E%3B+Lin%2C+H%2E%3B+Gao%2C+Z%2E%3B+Tan%2C+C%2E%3B+Li%2C+S%2E%3B+et+al%2E+2021b%2E+Self%2Dsupervised+on+Graphs%3A+Contrastive%2C+Generative%2C+or+Predictive%2E+arXiv+preprint+arXiv%3A2105%2E07342%2E+%E2%80%94+%5BarXiv%3A2105%2E07342%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F)
36. Wu, L.; Lin, H.; Huang, Y.; and Li, S. Z. 2022a. Knowledge Distillation Improves Graph Structure Augmentation for Graph Neural Networks. In Oh, A. H.; Agarwal, A.; Belgrave, D.; and Cho, K., eds., Advances in Neural Information Processing Systems. [find](https://scholar.google.com/scholar?q=Wu%2C+L%2E%3B+Lin%2C+H%2E%3B+Huang%2C+Y%2E%3B+and+Li%2C+S%2E+Z%2E+2022a%2E+Knowledge+Distillation+Improves+Graph+Structure+Augmentation+for+Graph+Neural+Networks%2E+In+Oh%2C+A%2E+H%2E%3B+Agarwal%2C+A%2E%3B+Belgrave%2C+D%2E%3B+and+Cho%2C+K%2E%2C+eds%2E%2C+Adv)
37. Wu, L.; Lin, H.; Xia, J.; Tan, C.; and Li, S. Z. 2022b. Multi-level disentanglement graph neural network. Neural Computing and Applications, 1–15. [find](https://scholar.google.com/scholar?q=Wu%2C+L%2E%3B+Lin%2C+H%2E%3B+Xia%2C+J%2E%3B+Tan%2C+C%2E%3B+and+Li%2C+S%2E+Z%2E+2022b%2E+Multi%2Dlevel+disentanglement+graph+neural+network%2E+Neural+Computing+and+Applications%2C+1%E2%80%9315%2E)
38. Wu, T.; Zhao, Z.; Wang, J.; Bai, X.; Wang, L.; Wong, N.; and Yang, Y. 2023. Edge-free but Structure-aware: Prototype-Guided Knowledge Distillation from GNNs to MLPs. arXiv preprint arXiv:2303.13763. [find](https://scholar.google.com/scholar?q=Wu%2C+T%2E%3B+Zhao%2C+Z%2E%3B+Wang%2C+J%2E%3B+Bai%2C+X%2E%3B+Wang%2C+L%2E%3B+Wong%2C+N%2E%3B+and+Yang%2C+Y%2E+2023%2E+Edge%2Dfree+but+Structure%2Daware%3A+Prototype%2DGuided+Knowledge+Distillation+from+GNNs+to+MLPs%2E+arXiv+preprint+arXiv%3A2303%2E13763%2E)
39. Wu, Z.; Pan, S.; Chen, F.; Long, G.; Zhang, C.; and Philip, S. Y. 2020. [A comprehensive survey on graph neural networks](https://md.research.elamur.ai/free/en/papers/wu-2019-comprehensive-survey-graph-neural-networks/). IEEE transactions on neural networks and learning systems.
40. Xia, J.; Wu, L.; Chen, J.; Hu, B.; and Li, S. Z. 2022. SimGRACE: A Simple Framework for Graph Contrastive Learning without Data Augmentation. In Proceedings of the ACM Web Conference 2022, 1070–1079. [find](https://scholar.google.com/scholar?q=Xia%2C+J%2E%3B+Wu%2C+L%2E%3B+Chen%2C+J%2E%3B+Hu%2C+B%2E%3B+and+Li%2C+S%2E+Z%2E+2022%2E+SimGRACE%3A+A+Simple+Framework+for+Graph+Contrastive+Learning+without+Data+Augmentation%2E+In+Proceedings+of+the+ACM+Web+Conference+2022%2C+1070%E2%80%931079%2E)
41. Yan, B.; Wang, C.; Guo, G.; and Lou, Y. 2020. TinyGNN: Learning Efficient Graph Neural Networks. In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, 1848–1856. [find](https://scholar.google.com/scholar?q=Yan%2C+B%2E%3B+Wang%2C+C%2E%3B+Guo%2C+G%2E%3B+and+Lou%2C+Y%2E+2020%2E+TinyGNN%3A+Learning+Efficient+Graph+Neural+Networks%2E+In+Proceedings+of+the+26th+ACM+SIGKDD+International+Conference+on+Knowledge+Discovery+%26+Data+Mining%2C+18)
42. Yang, C.; Liu, J.; and Shi, C. 2021. Extract the Knowledge of Graph Neural Networks and Go Beyond it: An Effective Knowledge Distillation Framework. In Proceedings of the Web Conference 2021, 1227–1237. [find](https://scholar.google.com/scholar?q=Yang%2C+C%2E%3B+Liu%2C+J%2E%3B+and+Shi%2C+C%2E+2021%2E+Extract+the+Knowledge+of+Graph+Neural+Networks+and+Go+Beyond+it%3A+An+Effective+Knowledge+Distillation+Framework%2E+In+Proceedings+of+the+Web+Conference+2021%2C+1227%E2%80%93123)
43. Zhang, M.; and Chen, Y. 2018. Link prediction based on graph neural networks. arXiv preprint arXiv:1802.09691. — [arXiv:1802.09691](https://arxiv.org/abs/1802.09691) [find](https://scholar.google.com/scholar?q=Zhang%2C+M%2E%3B+and+Chen%2C+Y%2E+2018%2E+Link+prediction+based+on+graph+neural+networks%2E+arXiv+preprint+arXiv%3A1802%2E09691%2E+%E2%80%94+%5BarXiv%3A1802%2E09691%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F1802%2E09691%29)
44. Zhang, S.; Liu, Y.; Sun, Y.; and Shah, N. 2021. Graph-less Neural Networks: Teaching Old MLPs New Tricks via Distillation. arXiv preprint arXiv:2110.08727. — [arXiv:2110.08727](https://arxiv.org/abs/2110.08727) [find](https://scholar.google.com/scholar?q=Zhang%2C+S%2E%3B+Liu%2C+Y%2E%3B+Sun%2C+Y%2E%3B+and+Shah%2C+N%2E+2021%2E+Graph%2Dless+Neural+Networks%3A+Teaching+Old+MLPs+New+Tricks+via+Distillation%2E+arXiv+preprint+arXiv%3A2110%2E08727%2E+%E2%80%94+%5BarXiv%3A2110%2E08727%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2)
45. Zhang, W.; Miao, X.; Shao, Y.; Jiang, J.; Chen, L.; Ruas, O.; and Cui, B. 2020. Reliable data distillation on graph convolutional network. In Proceedings of the 2020 ACM SIGMOD International Conference on Management of Data, 1399– 1414. [find](https://scholar.google.com/scholar?q=Zhang%2C+W%2E%3B+Miao%2C+X%2E%3B+Shao%2C+Y%2E%3B+Jiang%2C+J%2E%3B+Chen%2C+L%2E%3B+Ruas%2C+O%2E%3B+and+Cui%2C+B%2E+2020%2E+Reliable+data+distillation+on+graph+convolutional+network%2E+In+Proceedings+of+the+2020+ACM+SIGMOD+International+Conferenc)
46. Zhou, J.; Cui, G.; Hu, S.; Zhang, Z.; Yang, C.; Liu, Z.; Wang, L.; Li, C.; and Sun, M. 2020. Graph neural networks: A review of methods and applications. AI Open, 1: 57–81. [find](https://scholar.google.com/scholar?q=Zhou%2C+J%2E%3B+Cui%2C+G%2E%3B+Hu%2C+S%2E%3B+Zhang%2C+Z%2E%3B+Yang%2C+C%2E%3B+Liu%2C+Z%2E%3B+Wang%2C+L%2E%3B+Li%2C+C%2E%3B+and+Sun%2C+M%2E+2020%2E+Graph+neural+networks%3A+A+review+of+methods+and+applications%2E+AI+Open%2C+1%3A+57%E2%80%9381%2E)
47. Zhu, J.; Yan, Y.; Zhao, L.; Heimann, M.; Akoglu, L.; and Koutra, D. 2020. Beyond Homophily in Graph Neural Networks: Current Limitations and Effective Designs. Advances in Neural Information Processing Systems, 33. [find](https://scholar.google.com/scholar?q=Zhu%2C+J%2E%3B+Yan%2C+Y%2E%3B+Zhao%2C+L%2E%3B+Heimann%2C+M%2E%3B+Akoglu%2C+L%2E%3B+and+Koutra%2C+D%2E+2020%2E+Beyond+Homophily+in+Graph+Neural+Networks%3A+Current+Limitations+and+Effective+Designs%2E+Advances+in+Neural+Information+Processin)

*Updated 3 November 2026*