Graph neural network: Difference between revisions

Revision as of 10:56, 29 January 2025

Revision as of 05:01, 2 February 2025

Line 128:

The self-attention pooling layer can be seen as an extension of the top-k pooling layer. Differently from top-k pooling, the self-attention scores computed in self-attention pooling account both for the graph features and the graph topology.

= Heterophilic Graph Learning =

[[Homophily]] principle, i.e., nodes with the same labels or similar attributes are more likely to be connected, has been commonly believed to be the main reason for the superiority of Graph Neural Networks (GNNs) over traditional Neural Networks (NNs) on graph-structured data, especially on node-level tasks<ref name=":0">{{Citation |last=Luan |first=Sitao |title=The Heterophilic Graph Learning Handbook: Benchmarks, Models, Theoretical Analysis, Applications and Challenges |date=2024-07-12 |url=https://arxiv.org/abs/2407.09618 |access-date=2025-02-02 |publisher=arXiv |doi=10.48550/arXiv.2407.09618 |id=arXiv:2407.09618 |last2=Hua |first2=Chenqing |last3=Lu |first3=Qincheng |last4=Ma |first4=Liheng |last5=Wu |first5=Lirong |last6=Wang |first6=Xinyu |last7=Xu |first7=Minkai |last8=Chang |first8=Xiao-Wen |last9=Precup |first9=Doina}}</ref>. However, recent work has identified a non-trivial set of datasets where GNN’s performance compared to the NN’s is not satisfactory<ref>{{Cite journal |last=Luan |first=Sitao |last2=Hua |first2=Chenqing |last3=Lu |first3=Qincheng |last4=Zhu |first4=Jiaqi |last5=Chang |first5=Xiao-Wen |last6=Precup |first6=Doina |date=2024 |editor-last=Cherifi |editor-first=Hocine |editor2-last=Rocha |editor2-first=Luis M. |editor3-last=Cherifi |editor3-first=Chantal |editor4-last=Donduran |editor4-first=Murat |title=When Do We Need Graph Neural Networks for Node Classification? |url=https://link.springer.com/chapter/10.1007/978-3-031-53468-3_4 |journal=Complex Networks & Their Applications XII |language=en |___location=Cham |publisher=Springer Nature Switzerland |pages=37–48}}</ref>. [[Heterophily]], i.e., low homophily, has been considered the main cause of this empirical observation<ref name=":1">{{Cite journal |last=Luan |first=Sitao |last2=Hua |first2=Chenqing |last3=Lu |first3=Qincheng |last4=Zhu |first4=Jiaqi |last5=Zhao |first5=Mingde |last6=Zhang |first6=Shuyuan |last7=Chang |first7=Xiao-Wen |last8=Precup |first8=Doina |date=2022-12-06 |title=Revisiting Heterophily For Graph Neural Networks |url=https://proceedings.neurips.cc/paper_files/paper/2022/hash/092359ce5cf60a80e882378944bf1be4-Abstract-Conference.html |journal=Advances in Neural Information Processing Systems |language=en |volume=35 |pages=1362–1375}}</ref>. People have begun to revisit and re-evaluate most existing graph models in the heterophily scenario across various kinds of graphs, e.g., [[Heterogeneous network|heterogeneous graphs]], [[Temporal network|temporal graphs]] and [[Hypergraph|hypergraphs]]. Moreover, numerous graph-related applications are found to be closely related to the heterophily problem, e.g. [[Fraud detection|graph fraud/anomaly detection]], [[Adversarial attack|graph adversarial attacks and robustness]], privacy, [[federated learning]] and [[Point cloud|point cloud segmentation]], [[Clustering|graph clustering]], [[Recommender system|recommender systems]], [[Generative model|generative models]], [[link prediction]], [[Graph isomorphism|graph classification]] and [[Graph coloring|coloring]], etc. In the past few years, considerable effort has been devoted to studying and addressing the heterophily issue in graph learning<ref name=":0" /><ref name=":1" /><ref>{{Cite journal |last=Luan |first=Sitao |last2=Hua |first2=Chenqing |last3=Xu |first3=Minkai |last4=Lu |first4=Qincheng |last5=Zhu |first5=Jiaqi |last6=Chang |first6=Xiao-Wen |last7=Fu |first7=Jie |last8=Leskovec |first8=Jure |last9=Precup |first9=Doina |date=2023-12-15 |title=When Do Graph Neural Networks Help with Node Classification? Investigating the Homophily Principle on Node Distinguishability |url=https://proceedings.neurips.cc/paper_files/paper/2023/hash/5ba11de4c74548071899cf41dec078bf-Abstract-Conference.html |journal=Advances in Neural Information Processing Systems |language=en |volume=36 |pages=28748–28760}}</ref>.

== Applications ==