MEMORY-EFFICIENT PROBABILISTIC GRAPH ATTENTION NETWORKS FOR LARGE-SCALE NODE CLASSIFICATION

Abstract

Graph neural networks (GNNs) have demonstrated exceptional performance in various graph-structured learning tasks; however, they frequently encounter scalability challenges due to substantial computational and memory requirements. This work presents ME-MS-PGAN, a memory-efficient multi-scale progressive graph attention network that disassembles large graphs into smaller, more manageable pieces while keeping the full-graph contextual dependencies intact. The proposed model greatly lowers the attention memory complexity from  to  and the computational cost from  to , where  is much smaller than .

Numerous experiments on benchmark citation networks show that ME-MS-PGAN works well. The model has 610,411 parameters and gets 0.4630 accuracy on Cora (2,708 nodes), 0.3960 on CiteSeer (3,327 nodes), and 0.4310 on PubMed (19,717 nodes). It also needs considerably fewer processors than regular GATs. The probabilistic framework also gives accurate uncertainty quantification at all phases, resulting in predictions easier for individuals to comprehend and more reliable. In general, ME-MS-PGAN is a big step forward for scalable GNN design because it allows you look at big graphs on regular computers. Its combination of attention that doesn't use a lot of memory and probabilistic reasoning is very helpful in healthcare and science, where it's important to know the probability something is to happen.