AUGKD: Adaptive Universal Graph Knowledge Distillation

Yu Xie

¹Key Laboratory of Computational Intelligence and Chinese Information Processing of Ministry of Education, Shanxi University, Taiyuan 030006, China

Graph data distillation frameworks have shown great potential in compressing graph representations and improving learning efficiency. Nevertheless, their robustness and generalization remain limited when applied to different graph neural network architectures. To address these limitations, we propose Adaptive Universal Graph Knowledge Distillation Framework (AUGKD). AUGKD adaptively enhances node representations by adapting to dataset heterogeneity and designing an importance-based node feature enhancement strategy. Then we introduce a label propagation–guided feature extraction module within the multilayer perceptron to mitigate over-smoothing, and employs a weight-adaptive mechanism that enables robust generalization to heterophilic graphs through adaptive and negative propagation coefficients. Experiments on multiple benchmark datasets demonstrate that AUGKD achieves robust and effective performance on both homophilic and heterophilic graphs benchmark datasets, significantly outperforming the baselines.

Graph Data Distillation, Heterophilic Graphs, Adaptive Mechanisms, Graph Neural Networks

Junqi Liu, Yu Xie, Tianjun Ma, Wei Zheng, Jiangjiang Zhang (2026). AUGKD: Adaptive Universal Graph Knowledge Distillation. Journal of Networking and Network Applications, Volume 6, Issue 1, pp. 32–40. https://doi.org/10.33969/J-NaNA.2026.060104.

[1] M. Lin, X. Hong, W. Li, and S. Lu, “Unified graph neural networks pre-training for multi-domain graphs,” in Proc. AAAI Conf. Artif. Intell., vol. 39, pp. 12165–12173, 2025.

[2] M. Freund, R. Dorsch, S. Schmid, T. Wehr, and A. Harth, “Enriching RDF data with LLM-based named entity recognition and linking,” in Int. Knowl. Graph Semantic Web Conf., Springer, pp. 109–122, 2024.

[3] S. M. Oo and O. Hartig, “An algebraic foundation for knowledge graph construction,” in Eur. Semantic Web Conf. (ESWC), Springer, pp. 3–22, 2025.

[4] Z. Sheng, W. Guo, Y. Shao, W. Zhang, and B. Cui, “LLMs are noisy oracles: Noise-aware graph active learning,” in Proc. ACM SIGKDD, pp. 2526–2537, 2025.

[5] Y. Zhao, H. Lin, S. Wen, J. Shen, and B. Hua, “SMA-GNN: A symbol-aware GNN for signed link prediction,” in Proc. ACM SIGKDD, pp. 3957–3967, 2025.

[6] N. Firouzkouhi et al., “Generalized fuzzy hypergraph for link prediction and identification of influencers in dynamic social media networks,” Expert Syst. Appl., vol. 238, p. 121736, 2024.

[7] J. Wang et al., “Multimodal fusion framework based on knowledge graph for personalized recommendation,” Expert Syst. Appl., vol. 268,

p. 126308, 2025.

[8] Y. Ding, Z. Zhang, and B. Wang, “Category-integrated dual-task graph neural networks for session-based recommendation,” Expert Syst. Appl., vol. 263, p. 125784, 2025.

[9] X. Song, B. Zhou, Y. Wang, and W. Liu, “Dynamic graph structure evolution for node classification with missing attributes,” Sci. Rep., vol. 15, no. 1, p. 25687, 2025.

[10] T. N. Kipf and M. Welling, “Semi-supervised classification with graph convolutional networks,” in Proc. Int. Conf. Learn. Represent. (ICLR), 2017.

[11] P. Velickovic et al., “Graph attention networks,” in Proc. Int. Conf. Learn. Represent. (ICLR), 2018.

[12] R. Zeng, J. Fang, S. Liu, Z. Meng, and S. Liang, “Enhancing graph neural networks via memorized global information,” ACM Trans. Web, vol. 18, no. 4, pp. 1–34, 2024.

[13] E. Hssayni, A. Boufssasse, N.-E. Joudar, and M. Ettaouil, “Novel dropout approach for mitigating over-smoothing in graph neural net-works,” Appl. Intell., vol. 55, no. 5, p. 354, 2025.

[14] F. Qian et al., “Exploring the over-smoothing problem of graph neural networks for graph classification: An entropy-based viewpoint,” in Proc. Int. Joint Conf. Artif. Intell. (IJCAI), pp. 3235–3244, 2025.

[15] J. Zhu et al., “On the impact of feature heterophily on link prediction with graph neural networks,” in Adv. Neural Inf. Process. Syst. (NeurIPS), vol. 37, pp. 65823–65851, 2024.

[16] L. Duan et al., “Ligand-receptor dynamics in heterophily-aware graph neural networks for enhanced cell type prediction from single-cell RNA-seq data,” Front. Mol. Biosci., vol. 12, Art. no. 1547231, 2025.

[17] S. Liu et al., “Integrating co-training with edge discrimination to enhance graph neural networks under heterophily,” in Proc. AAAI Conf. Artif. Intell., vol. 39, pp. 18960–18968, 2025.

[18] R. Duan et al., “Unifying homophily and heterophily for spectral graph neural networks via triple filter ensembles,” in Adv. Neural Inf. Process. Syst. (NeurIPS), vol. 37, pp. 93540–93567, 2024.

[19] D. Lopez-Paz et al., “Unifying distillation and privileged information,” in Adv. Neural Inf. Process. Syst. (NeurIPS), vol. 28, 2015.

[20] Y. Yang et al., “Distilling knowledge from graph convolutional net-works,” in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (CVPR), pp. 9294–9302, 2019.

[21] J. Liu et al., “Ddk: Distilling domain knowledge for efficient large language models,” in Adv. Neural Inf. Process. Syst. (NeurIPS), vol. 37, pp. 98297–98319, 2024.

[22] M. A. Khan et al., “HYDRA-FL: Hybrid knowledge distillation for robust and accurate federated learning,” in Adv. Neural Inf. Process. Syst.(NeurIPS), vol. 37, pp. 50469–50493, 2024.

[23] W. Xu et al., “Speculative knowledge distillation: Bridging the teacher-student gap through interleaved sampling,” in Proc. Int. Conf. Learn. Represent. (ICLR), 2025.

[24] C. He et al., “DA-KD: Difficulty-aware knowledge distillation for efficient large language models,” in Proc. Int. Conf. Mach. Learn. (ICML), 2025.

[25] Y. Tian et al., “Knowledge distillation on graphs: A survey,” ACM Comput. Surv., vol. 57, no. 8, pp. 1–16, 2025.

[26] Y. You et al., “Graph contrastive learning with augmentations,” in Adv. Neural Inf. Process. Syst. (NeurIPS), vol. 33, pp. 5812–5823, 2020.

[27] Y. Zhu et al., “Graph contrastive learning with adaptive augmentation,” in Proc. Web Conf. (WWW), pp. 2069–2080, 2021.

[28] J. Shen et al., “GTAT: Empowering graph neural networks with cross attention,” Sci. Rep., vol. 15, no. 1, p. 4760, 2025.

[29] Y. Yang et al., “Implicit vs unfolded graph neural networks,” J. Mach. Learn. Res., vol. 26, no. 82, pp. 1–46, 2025.

[30] C. Yang, J. Liu, and C. Shi, “Extract the knowledge of graph neural networks and go beyond it,” in Proc. Web Conf. (WWW), pp. 1227–1237, 2021.

[31] J. Klicpera et al., “Predict then propagate: Graph neural networks meet personalized PageRank,” in Proc. Int. Conf. Learn. Represent. (ICLR), 2019.

[32] D. Bo et al., “Beyond low-frequency information in graph convolutional networks,” in Proc. AAAI Conf. Artif. Intell., vol. 35, pp. 3950–3957, 2021.

[33] E. Chien et al., “Adaptive universal generalized PageRank graph neural network,” in Proc. Int. Conf. Learn. Represent. (ICLR), 2021.

[34] M. He et al., “BernNet: Learning arbitrary graph spectral filters via Bernstein approximation,” in Adv. Neural Inf. Process. Syst. (NeurIPS), vol. 34, pp. 14239–14251, 2021.

[35] Q. Ma et al., “Adaptive propagation network based on multi-scale information fusion,” in Int. Conf. Artif. Neural Netw. (ICANN), Springer, pp. 51–62, 2023.