Downloaded 23 times
More Related Content
![[기초개념] Graph Convolutional Network (GCN)](https://cdn.slidesharecdn.com/ss_thumbnails/agistdkimgcn190507-190507153736-thumbnail.jpg?width=640&height=640&fit=bounds)
Similar to Introduction to Graph neural networks @ Vienna Deep Learning meetup
Introduction to Graph neural networks @ Vienna Deep Learning meetup
- 1. Graph Neural Networks LiadMagen March October 2020
- 2.
- 3. Prof. Dr. StefanThurner
- 4.
- 5. Seven Bridges ofKönigsberg → →
- 6. Location & Streetrepresentation
- 7.
- 8.
- 9. Paper citations asa graph
- 10. Mesh & Pointcloud as a graph
- 11.
- 14. Process other (non-Euclidean)formats of data
- 15.
- 16.
- 19. Graph Neural Networks ICLRSubmissions Source: Twitter - @prlz77
- 20.
- 21. Graphs, Mathematically G =(Vetrices, Edges) V = {v1 … vn) E = {(vi, vj), … }
- 22. Graph - algebraicrepresentation G = (Vetrices, Edges) Adjacency matrix 1 - if {vi, vj} ∈ E && i != j Aij 0 - Otherwise
- 23. Graph - algebraicrepresentation G = (Vetrices, Edges) Laplacian matrix * L = D-A Degree Adjacency Laplacian * For undirected graphs
- 24.
- 25. GCN GCN Layer: Y =ReLU(AXW) ...but with message passing
- 26. Message Passing For eachnode : 1. Aggregate neighbor nodes into an intermediate representation 2. Transform the aggregated representation with a linear projection followed by a non-linearity (ReLU) Mathematically: H →Network Layer W→Network Weights A →Adjacency Matrix D →Degree Matrix
- 27. Model types ● Graphclassification ○ Chemical properties of a molecule ○ Comparing user preferences / activities ● Node classification - node label prediction ○ Malicious users in a social network ○ Visually inferred Named Entity Recognition (NER) ○ Node clustering ● Edge prediction ○ Recommendation system ○ Protein-protein interaction ○ “Friend” suggestion
- 28.
- 29. NetworkX ● Store andmutate Graphs ● Graph algorithms (Shortest path - Dijkstra, TreeWidth, clustering, centrality) ● Network analysis ● Node / edge data ● Visualization tools
- 30. DGL ● Building blocks ●Great tutorials ● Generative graphs ● Great for research and complicated tasks
- 31. ● An extensionlibrary for pyTorch ● Officially part of the pyTorch ecosystem ● Easily extensible ● Papers are implemented directly in it ● Looooooooooooooong list of ready-to-use methods and algorithms: ○ TransformerConf (2020) ○ GCN2Conv (2020) ○ DeeperGCN (2020) ○ Top-K Pooling ○ PairNorm PyTorch Geometric
- 32. Model types ● Graphclassification ○ Chemical properties of a molecule ○ Comparing user preferences / activities ● Node classification - node label prediction ○ Malicious users in a social network ○ Visually inferred Named Entity Recognition (NER) ○ Node clustering ● Edge prediction ○ Recommendation system ○ Protein-protein interaction ○ “Friend” suggestion
- 33.
- 34. PyTorch Geometric Example ●DataLoader ● GCN Layers
- 35. Potential Pitfalls whengoing ‘deep’ ● Vanishing Gradient ● Overfitting ● Over-smoothing Node-vectors become too similar ● Bottleneck (Over-Squashing) A single node vector contains data of too many nodes https://arxiv.org/abs/2006.07107 - Effective Training Strategies for Deep Graph Neural Networks
- 36. Scaling ● GraphSAGE https://arxiv.org/abs/1706.02216 ● ClusterGCN https://arxiv.org/abs/1905.07953 ●Graph-BERT https://arxiv.org/abs/2001.05140 https://github.com/jwzhanggy/Graph-Bert Additional reading: https://towardsdatascience.com/simple-scalable-graph-neural-networks-7eb04f366d07
- 37. Model Evaluation https://ogb.stanford.edu/ ● Datasets+ Data Loaders (DGL / pyTorch Geometric) ● Node pred ● Edge pred ● Graph pred
- 38. Please feel freeto ask questions ;)
- 39.