This presentation explains basic ideas of graph neural networks (GNNs) and their common applications. Primary target audiences are students, engineers and researchers who are new to GNNs but interested in using GNNs for their projects. This is a modified version of the course material for a special lecture on Data Science at Nara Institute of Science and Technology (NAIST), given by Preferred Networks researcher Katsuhiko Ishiguro, PhD.
Slides for a talk about Graph Neural Networks architectures, overview taken from very good paper by Zonghan Wu et al. (https://arxiv.org/pdf/1901.00596.pdf)
Introduction to Graph neural networks @ Vienna Deep Learning meetupLiad Magen
Graphs are useful data structures that can be used to model various sorts of data: from molecular protein structures to social networks, pandemic spreading models, and visually rich content such as websites & invoices. In the recent few years, graph neural networks have done a huge leap forward. It is a powerful tool that every data scientist should know. In this talk, we will review their basic structure, show some example usages, and explore the existing (python) tools.
https://telecombcn-dl.github.io/2017-dlcv/
Deep learning technologies are at the core of the current revolution in artificial intelligence for multimedia data analysis. The convergence of large-scale annotated datasets and affordable GPU hardware has allowed the training of neural networks for data analysis tasks which were previously addressed with hand-crafted features. Architectures such as convolutional neural networks, recurrent neural networks and Q-nets for reinforcement learning have shaped a brand new scenario in signal processing. This course will cover the basic principles and applications of deep learning to computer vision problems, such as image classification, object detection or image captioning.
Machine learning on graphs is an important and ubiquitous task with applications ranging from drug design to friendship recommendation in social networks. The primary challenge in this domain is finding a way to represent, or encode, graph structure so that it can be easily exploited by machine learning models. However, traditionally machine learning approaches relied on user-defined heuristics to extract features encoding structural information about a graph. In this talk I will discuss methods that automatically learn to encode graph structure into low-dimensional embeddings, using techniques based on deep learning and nonlinear dimensionality reduction. I will provide a conceptual review of key advancements in this area of representation learning on graphs, including random-walk based algorithms, and graph convolutional networks.
Slides for a talk about Graph Neural Networks architectures, overview taken from very good paper by Zonghan Wu et al. (https://arxiv.org/pdf/1901.00596.pdf)
Introduction to Graph neural networks @ Vienna Deep Learning meetupLiad Magen
Graphs are useful data structures that can be used to model various sorts of data: from molecular protein structures to social networks, pandemic spreading models, and visually rich content such as websites & invoices. In the recent few years, graph neural networks have done a huge leap forward. It is a powerful tool that every data scientist should know. In this talk, we will review their basic structure, show some example usages, and explore the existing (python) tools.
https://telecombcn-dl.github.io/2017-dlcv/
Deep learning technologies are at the core of the current revolution in artificial intelligence for multimedia data analysis. The convergence of large-scale annotated datasets and affordable GPU hardware has allowed the training of neural networks for data analysis tasks which were previously addressed with hand-crafted features. Architectures such as convolutional neural networks, recurrent neural networks and Q-nets for reinforcement learning have shaped a brand new scenario in signal processing. This course will cover the basic principles and applications of deep learning to computer vision problems, such as image classification, object detection or image captioning.
Machine learning on graphs is an important and ubiquitous task with applications ranging from drug design to friendship recommendation in social networks. The primary challenge in this domain is finding a way to represent, or encode, graph structure so that it can be easily exploited by machine learning models. However, traditionally machine learning approaches relied on user-defined heuristics to extract features encoding structural information about a graph. In this talk I will discuss methods that automatically learn to encode graph structure into low-dimensional embeddings, using techniques based on deep learning and nonlinear dimensionality reduction. I will provide a conceptual review of key advancements in this area of representation learning on graphs, including random-walk based algorithms, and graph convolutional networks.
A comprehensive tutorial on Convolutional Neural Networks (CNN) which talks about the motivation behind CNNs and Deep Learning in general, followed by a description of the various components involved in a typical CNN layer. It explains the theory involved with the different variants used in practice and also, gives a big picture of the whole network by putting everything together.
Next, there's a discussion of the various state-of-the-art frameworks being used to implement CNNs to tackle real-world classification and regression problems.
Finally, the implementation of the CNNs is demonstrated by implementing the paper 'Age ang Gender Classification Using Convolutional Neural Networks' by Hassner (2015).
https://telecombcn-dl.github.io/2017-dlcv/
Deep learning technologies are at the core of the current revolution in artificial intelligence for multimedia data analysis. The convergence of large-scale annotated datasets and affordable GPU hardware has allowed the training of neural networks for data analysis tasks which were previously addressed with hand-crafted features. Architectures such as convolutional neural networks, recurrent neural networks and Q-nets for reinforcement learning have shaped a brand new scenario in signal processing. This course will cover the basic principles and applications of deep learning to computer vision problems, such as image classification, object detection or image captioning.
The next phase of Smart Network Convergence could be putting Deep Learning systems on the Internet. Deep Learning and Blockchain Technology might be combined in the smart networks of the future for automated identification (deep learning) and automated transaction (blockchain). Large scale future-class problems might be addressed with Blockchain Deep Learning nets as an advanced computational infrastructure, challenges such as million-member genome banks, energy storage markets, global financial risk assessment, real-time voting, and asteroid mining.
Blockchain Deep Learning nets and Smart Networks more generally are computing networks with intelligence built in such that identification and transfer is performed by the network itself through sophisticated protocols that automatically identify (deep learning), and validate, confirm, and route transactions (blockchain) within the network.
Transformer modality is an established architecture in natural language processing that utilizes a framework of self-attention with a deep learning approach.
This presentation was delivered under the mentorship of Mr. Mukunthan Tharmakulasingam (University of Surrey, UK), as a part of the ScholarX program from Sustainable Education Foundation.
We present Graph Convolutional Networks that, unlike classic DL models, allow supervised learning by exploiting both the single node features and its relationships with the others within the network.
In machine learning, a convolutional neural network is a class of deep, feed-forward artificial neural networks that have successfully been applied fpr analyzing visual imagery.
Deep learning (also known as deep structured learning or hierarchical learning) is the application of artificial neural networks (ANNs) to learning tasks that contain more than one hidden layer. Deep learning is part of a broader family of machine learning methods based on learning data representations, as opposed to task-specific algorithms. Learning can be supervised, partially supervised or unsupervised.
A comprehensive tutorial on Convolutional Neural Networks (CNN) which talks about the motivation behind CNNs and Deep Learning in general, followed by a description of the various components involved in a typical CNN layer. It explains the theory involved with the different variants used in practice and also, gives a big picture of the whole network by putting everything together.
Next, there's a discussion of the various state-of-the-art frameworks being used to implement CNNs to tackle real-world classification and regression problems.
Finally, the implementation of the CNNs is demonstrated by implementing the paper 'Age ang Gender Classification Using Convolutional Neural Networks' by Hassner (2015).
https://telecombcn-dl.github.io/2017-dlcv/
Deep learning technologies are at the core of the current revolution in artificial intelligence for multimedia data analysis. The convergence of large-scale annotated datasets and affordable GPU hardware has allowed the training of neural networks for data analysis tasks which were previously addressed with hand-crafted features. Architectures such as convolutional neural networks, recurrent neural networks and Q-nets for reinforcement learning have shaped a brand new scenario in signal processing. This course will cover the basic principles and applications of deep learning to computer vision problems, such as image classification, object detection or image captioning.
The next phase of Smart Network Convergence could be putting Deep Learning systems on the Internet. Deep Learning and Blockchain Technology might be combined in the smart networks of the future for automated identification (deep learning) and automated transaction (blockchain). Large scale future-class problems might be addressed with Blockchain Deep Learning nets as an advanced computational infrastructure, challenges such as million-member genome banks, energy storage markets, global financial risk assessment, real-time voting, and asteroid mining.
Blockchain Deep Learning nets and Smart Networks more generally are computing networks with intelligence built in such that identification and transfer is performed by the network itself through sophisticated protocols that automatically identify (deep learning), and validate, confirm, and route transactions (blockchain) within the network.
Transformer modality is an established architecture in natural language processing that utilizes a framework of self-attention with a deep learning approach.
This presentation was delivered under the mentorship of Mr. Mukunthan Tharmakulasingam (University of Surrey, UK), as a part of the ScholarX program from Sustainable Education Foundation.
We present Graph Convolutional Networks that, unlike classic DL models, allow supervised learning by exploiting both the single node features and its relationships with the others within the network.
In machine learning, a convolutional neural network is a class of deep, feed-forward artificial neural networks that have successfully been applied fpr analyzing visual imagery.
Deep learning (also known as deep structured learning or hierarchical learning) is the application of artificial neural networks (ANNs) to learning tasks that contain more than one hidden layer. Deep learning is part of a broader family of machine learning methods based on learning data representations, as opposed to task-specific algorithms. Learning can be supervised, partially supervised or unsupervised.
DAOR - Bridging the Gap between Community and Node Representations: Graph Emb...Artem Lutov
Slides of the presentation given at BigData'19, special session on Information Granulation in Data Science and Scalable Computing.
The fully automatic (i.e., without any manual tuning) graph embedding (i.e., network representation learning, unsupervised feature extraction) performed in near-linear time is presented. The resulting embeddings are interpretable, preserve both low- and high-order structural proximity of the graph nodes, computed (i.e., learned) by orders of magnitude faster and perform competitively to the manually tuned best state-of-the-art embedding techniques evaluated on diverse tasks of graph analysis.
https://telecombcn-dl.github.io/2017-dlcv/
Deep learning technologies are at the core of the current revolution in artificial intelligence for multimedia data analysis. The convergence of large-scale annotated datasets and affordable GPU hardware has allowed the training of neural networks for data analysis tasks which were previously addressed with hand-crafted features. Architectures such as convolutional neural networks, recurrent neural networks and Q-nets for reinforcement learning have shaped a brand new scenario in signal processing. This course will cover the basic principles and applications of deep learning to computer vision problems, such as image classification, object detection or image captioning.
Hanjun Dai, PhD Student, School of Computational Science and Engineering, Geo...MLconf
Graph Representation Learning with Deep Embedding Approach:
Graphs are commonly used data structure for representing the real-world relationships, e.g., molecular structure, knowledge graphs, social and communication networks. The effective encoding of graphical information is essential to the success of such applications. In this talk I’ll first describe a general deep learning framework, namely structure2vec, for end to end graph feature representation learning. Then I’ll present the direct application of this model on graph problems on different scales, including community detection and molecule graph classification/regression. We then extend the embedding idea to temporal evolving user-product interaction graph for recommendation. Finally I’ll present our latest work on leveraging the reinforcement learning technique for graph combinatorial optimization, including vertex cover problem for social influence maximization and traveling salesman problem for scheduling management.
In this presentation we consider several main methods for contruction regular QC-LDPC codes using algebraic approach. Consider existance of non broken by circulant permutation matrix cycles (short balanced cycles). Using Vontobel approach illustrate way to estimate girth bound and it influence on error-floor properties of QC-LDPC codes
Similar to Introduction to Graph Neural Networks: Basics and Applications - Katsuhiko Ishiguro, PhD (20)
PFN福田圭祐による東大大学院「融合情報学特別講義Ⅲ」(2022年10月19日)の講義資料です。
・Introduction to Preferred Networks
・Our developments to date
・Our research & platform
・Simulation ✕ AI
JMeter webinar - integration with InfluxDB and GrafanaRTTS
Watch this recorded webinar about real-time monitoring of application performance. See how to integrate Apache JMeter, the open-source leader in performance testing, with InfluxDB, the open-source time-series database, and Grafana, the open-source analytics and visualization application.
In this webinar, we will review the benefits of leveraging InfluxDB and Grafana when executing load tests and demonstrate how these tools are used to visualize performance metrics.
Length: 30 minutes
Session Overview
-------------------------------------------
During this webinar, we will cover the following topics while demonstrating the integrations of JMeter, InfluxDB and Grafana:
- What out-of-the-box solutions are available for real-time monitoring JMeter tests?
- What are the benefits of integrating InfluxDB and Grafana into the load testing stack?
- Which features are provided by Grafana?
- Demonstration of InfluxDB and Grafana using a practice web application
To view the webinar recording, go to:
https://www.rttsweb.com/jmeter-integration-webinar
State of ICS and IoT Cyber Threat Landscape Report 2024 previewPrayukth K V
The IoT and OT threat landscape report has been prepared by the Threat Research Team at Sectrio using data from Sectrio, cyber threat intelligence farming facilities spread across over 85 cities around the world. In addition, Sectrio also runs AI-based advanced threat and payload engagement facilities that serve as sinks to attract and engage sophisticated threat actors, and newer malware including new variants and latent threats that are at an earlier stage of development.
The latest edition of the OT/ICS and IoT security Threat Landscape Report 2024 also covers:
State of global ICS asset and network exposure
Sectoral targets and attacks as well as the cost of ransom
Global APT activity, AI usage, actor and tactic profiles, and implications
Rise in volumes of AI-powered cyberattacks
Major cyber events in 2024
Malware and malicious payload trends
Cyberattack types and targets
Vulnerability exploit attempts on CVEs
Attacks on counties – USA
Expansion of bot farms – how, where, and why
In-depth analysis of the cyber threat landscape across North America, South America, Europe, APAC, and the Middle East
Why are attacks on smart factories rising?
Cyber risk predictions
Axis of attacks – Europe
Systemic attacks in the Middle East
Download the full report from here:
https://sectrio.com/resources/ot-threat-landscape-reports/sectrio-releases-ot-ics-and-iot-security-threat-landscape-report-2024/
Slack (or Teams) Automation for Bonterra Impact Management (fka Social Soluti...Jeffrey Haguewood
Sidekick Solutions uses Bonterra Impact Management (fka Social Solutions Apricot) and automation solutions to integrate data for business workflows.
We believe integration and automation are essential to user experience and the promise of efficient work through technology. Automation is the critical ingredient to realizing that full vision. We develop integration products and services for Bonterra Case Management software to support the deployment of automations for a variety of use cases.
This video focuses on the notifications, alerts, and approval requests using Slack for Bonterra Impact Management. The solutions covered in this webinar can also be deployed for Microsoft Teams.
Interested in deploying notification automations for Bonterra Impact Management? Contact us at sales@sidekicksolutionsllc.com to discuss next steps.
Dev Dives: Train smarter, not harder – active learning and UiPath LLMs for do...UiPathCommunity
💥 Speed, accuracy, and scaling – discover the superpowers of GenAI in action with UiPath Document Understanding and Communications Mining™:
See how to accelerate model training and optimize model performance with active learning
Learn about the latest enhancements to out-of-the-box document processing – with little to no training required
Get an exclusive demo of the new family of UiPath LLMs – GenAI models specialized for processing different types of documents and messages
This is a hands-on session specifically designed for automation developers and AI enthusiasts seeking to enhance their knowledge in leveraging the latest intelligent document processing capabilities offered by UiPath.
Speakers:
👨🏫 Andras Palfi, Senior Product Manager, UiPath
👩🏫 Lenka Dulovicova, Product Program Manager, UiPath
Let's dive deeper into the world of ODC! Ricardo Alves (OutSystems) will join us to tell all about the new Data Fabric. After that, Sezen de Bruijn (OutSystems) will get into the details on how to best design a sturdy architecture within ODC.
Connector Corner: Automate dynamic content and events by pushing a buttonDianaGray10
Here is something new! In our next Connector Corner webinar, we will demonstrate how you can use a single workflow to:
Create a campaign using Mailchimp with merge tags/fields
Send an interactive Slack channel message (using buttons)
Have the message received by managers and peers along with a test email for review
But there’s more:
In a second workflow supporting the same use case, you’ll see:
Your campaign sent to target colleagues for approval
If the “Approve” button is clicked, a Jira/Zendesk ticket is created for the marketing design team
But—if the “Reject” button is pushed, colleagues will be alerted via Slack message
Join us to learn more about this new, human-in-the-loop capability, brought to you by Integration Service connectors.
And...
Speakers:
Akshay Agnihotri, Product Manager
Charlie Greenberg, Host
Search and Society: Reimagining Information Access for Radical FuturesBhaskar Mitra
The field of Information retrieval (IR) is currently undergoing a transformative shift, at least partly due to the emerging applications of generative AI to information access. In this talk, we will deliberate on the sociotechnical implications of generative AI for information access. We will argue that there is both a critical necessity and an exciting opportunity for the IR community to re-center our research agendas on societal needs while dismantling the artificial separation between the work on fairness, accountability, transparency, and ethics in IR and the rest of IR research. Instead of adopting a reactionary strategy of trying to mitigate potential social harms from emerging technologies, the community should aim to proactively set the research agenda for the kinds of systems we should build inspired by diverse explicitly stated sociotechnical imaginaries. The sociotechnical imaginaries that underpin the design and development of information access technologies needs to be explicitly articulated, and we need to develop theories of change in context of these diverse perspectives. Our guiding future imaginaries must be informed by other academic fields, such as democratic theory and critical theory, and should be co-developed with social science scholars, legal scholars, civil rights and social justice activists, and artists, among others.
Epistemic Interaction - tuning interfaces to provide information for AI supportAlan Dix
Paper presented at SYNERGY workshop at AVI 2024, Genoa, Italy. 3rd June 2024
https://alandix.com/academic/papers/synergy2024-epistemic/
As machine learning integrates deeper into human-computer interactions, the concept of epistemic interaction emerges, aiming to refine these interactions to enhance system adaptability. This approach encourages minor, intentional adjustments in user behaviour to enrich the data available for system learning. This paper introduces epistemic interaction within the context of human-system communication, illustrating how deliberate interaction design can improve system understanding and adaptation. Through concrete examples, we demonstrate the potential of epistemic interaction to significantly advance human-computer interaction by leveraging intuitive human communication strategies to inform system design and functionality, offering a novel pathway for enriching user-system engagements.
Accelerate your Kubernetes clusters with Varnish CachingThijs Feryn
A presentation about the usage and availability of Varnish on Kubernetes. This talk explores the capabilities of Varnish caching and shows how to use the Varnish Helm chart to deploy it to Kubernetes.
This presentation was delivered at K8SUG Singapore. See https://feryn.eu/presentations/accelerate-your-kubernetes-clusters-with-varnish-caching-k8sug-singapore-28-2024 for more details.
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...James Anderson
Effective Application Security in Software Delivery lifecycle using Deployment Firewall and DBOM
The modern software delivery process (or the CI/CD process) includes many tools, distributed teams, open-source code, and cloud platforms. Constant focus on speed to release software to market, along with the traditional slow and manual security checks has caused gaps in continuous security as an important piece in the software supply chain. Today organizations feel more susceptible to external and internal cyber threats due to the vast attack surface in their applications supply chain and the lack of end-to-end governance and risk management.
The software team must secure its software delivery process to avoid vulnerability and security breaches. This needs to be achieved with existing tool chains and without extensive rework of the delivery processes. This talk will present strategies and techniques for providing visibility into the true risk of the existing vulnerabilities, preventing the introduction of security issues in the software, resolving vulnerabilities in production environments quickly, and capturing the deployment bill of materials (DBOM).
Speakers:
Bob Boule
Robert Boule is a technology enthusiast with PASSION for technology and making things work along with a knack for helping others understand how things work. He comes with around 20 years of solution engineering experience in application security, software continuous delivery, and SaaS platforms. He is known for his dynamic presentations in CI/CD and application security integrated in software delivery lifecycle.
Gopinath Rebala
Gopinath Rebala is the CTO of OpsMx, where he has overall responsibility for the machine learning and data processing architectures for Secure Software Delivery. Gopi also has a strong connection with our customers, leading design and architecture for strategic implementations. Gopi is a frequent speaker and well-known leader in continuous delivery and integrating security into software delivery.
Transcript: Selling digital books in 2024: Insights from industry leaders - T...BookNet Canada
The publishing industry has been selling digital audiobooks and ebooks for over a decade and has found its groove. What’s changed? What has stayed the same? Where do we go from here? Join a group of leading sales peers from across the industry for a conversation about the lessons learned since the popularization of digital books, best practices, digital book supply chain management, and more.
Link to video recording: https://bnctechforum.ca/sessions/selling-digital-books-in-2024-insights-from-industry-leaders/
Presented by BookNet Canada on May 28, 2024, with support from the Department of Canadian Heritage.
Neuro-symbolic is not enough, we need neuro-*semantic*Frank van Harmelen
Neuro-symbolic (NeSy) AI is on the rise. However, simply machine learning on just any symbolic structure is not sufficient to really harvest the gains of NeSy. These will only be gained when the symbolic structures have an actual semantics. I give an operational definition of semantics as “predictable inference”.
All of this illustrated with link prediction over knowledge graphs, but the argument is general.
How world-class product teams are winning in the AI era by CEO and Founder, P...
Introduction to Graph Neural Networks: Basics and Applications - Katsuhiko Ishiguro, PhD
1. An Introduction to Graph Neural Networks:
basics and applications
Katsuhiko ISHIGURO, Ph. D (Preferred Networks, Inc.)
Oct. 23, 2020
1
Modified from the course material of:
Nara Institute of Science and Technology
Data Science Special Lecture
2. Take home message
• Graph Neural Networks (GNNs): Neural Networks (NNs) to
compute nodes’ representation of graph-structured data
• Practical applications in industry, hard competitions in academia
• Model: The fundamental is approximated Graph Convolution
• Applications: applicable in several tasks in different domains
2
3. Table of Contents (1/2)
• Graph Neural Networks (GNN)s; what is it
– Graph structure data
– Overview of GNN: function, application
• The basic and the de-facto standard: GCN
– Graph Convolutions and the GCN model
– Exemplar task: semi-supervised node classification with GCN
3
4. Table of Contents (2/2)
• Application in Chemo-Informatics: Protein Interface Prediction
– A straightforward application of GNN as I/O for graphs
• Application in Computer Vision: Scene Graph generation
– Task-tailored GNN model
• Theoretical issues of GNN
– Deep GNN does not work well
– Representation power of GNN is upper-bounded
• Conclusion and Materials
4
5. Acknowledgements
Many thanks for helpful comments and provided
materials!!
Daisuke Okanohara, Shin-ichi Maeda, Kentaro Minami,
Yuta Tsuboi, Sousuke Kobayashi, Kenta Oono, Hiroshi
Maruyama (Preferred Networks)
5
6. Graph: for relational data
• Graph
• Set of nodes (vertices): :
• Set of edges:
– directional/non-directional
• Especially interested in Attributed graph
– Nodes and/or edges have some features (label,
numbers, vectors) 6
7. Matrix representation of Attributed graph
• Suitable for formulations/implementations of GNNs
7
Feature Matrix
Adjacency Matrix
N nodes, D dim. features
0 1 1 0 0
1 0 0 1 1
1 0 0 0 1
0 1 0 0 0
0 1 1 0 0
Edge existence between
N X N node pairs
Symmetric A <-- non-directional edges
9. Interface for intelligent processes on graphs
• H is informative and easy-to-handle “data” for machine-
learning (ML) models/tools
9
L-layered
GNN
Informative features
ML-friendly structure (set of vectors)
Classification
Regression
Interpretation
Generative
models
10. Application Domains of GNNs: concrete
applications
Chemical Molecular Graph
[Jin18JTVAE, Laugier18CGCNN]
[http://konect.uni-
koblenz.de/networks/ucidata-
zachary]
Social Network
Scene Graph
[Qi19AttentiveRN]
Pointcloud [Landrieu_Boussaha19PCS]
Physical Simulator [Sanchez-Gonzalez20GNS]
Knowledge Graph
[DeCao19QA2]
10
11. GNN in Machine Learning Researches
11
0
10
20
30
40
50
60
70
80
90
ICML ICLR NeurIPS
Numbers of accepted papers: “title: Graph”
2014 2015 2016 2017 2018 2019 2020 [by the lecturer]
~2016: “graphical models”, “graph kernels”, and pure graph theory works
2017~: GNN works add up the counts
13. Convolution
• Modify a signal f by a filter g, accumulate over ``shift’’ t
on the coordinate (axes) x
13
abcpedia.acoustics.jp
1D signal example
f: Sound signal
g: Impulse response
Hall resonated sound
(reverb)
f g f*g
14. E.g. ConvNet in computer vision [Krizhevsky12ConvNet]
14 [http://image-net.org/challenges/LSVRC/2012/ilsvrc2012.pdf]
CnvNet (Alexnet)
[Krizhevsky12ConvNet]
10% margin
Former (Non-DNN) top runners:
> 1% difference
2012 ILSVRC competition
15. E.g. ConvNet in computer vision [Krizhevsky12ConvNet]
• L- iterative conv. Operations for image pixels
15
Layer l
Layer l-1
Image signal (feature) at pixel (x,y) of layer l
shift signal filter
pixel (x, y)
[Kipf18talk]
16. How can we define Conv. in graphs?
• No “axes” in graphs unlike sound signals (time) and
image signals (x, y)
• What is “coordinate”? What is “shift”?
16 [いらすとや]
Spectral approach on Graphs
17. Fourier Transform and Convolution
Convolutions in the signal (spatial) domain = Multiplication
in the spectral (frequency) domain
K-th frequency’sFourier base function
17
1D case:
For Graphs, what is the signal? How the FT defined??
20. Graph Fourier Transform =
Multiply Eigenvectors of (Normalized) Graph
Laplacians
20
i-th eigenvalue
= frequency of i-th base
i-th eigenvevtor
= i-th graph Fourier base
Graph signal in spectral (freq.)
space
Graph Fourier Transform(GFT)
Inner product between the graph
signal (x) and Fourier bases (U)
22. Graph Convolution via GFT
• Convolution is a multiplication in Fourier space
and GFT as well
22
GFT of the graph signal
Convolution = simple
multiplication in spectral
IGFT to get conv-filtered graph signal
23. Naïve Graph Convolution
23
FixedTwo parameters:
Eig.Decompose of (normalized) graph Laplacians O(N^3) for each graph
Trainable
(amplify each spectral coefficients)
Simple linear algebra for a generic Graph Conv.
(eigenvectors of the Laplacian = Fourier bases over freq.)
24. • Model N trainable param. with K (<<N) order
Chebychev Polynomials
Approximation: ChebyNet [Defferrard16ChebNet]
Chebychev
Polynomial
24
25. 25
Graph Laplacian is sparse in nature
Polynomial L is lightweight << O(N^3)
No U-s = No Eig. Decomp.
26. GCN [Kipf_Welling17GCN]: much simpler Convolution
26
Set K = 1, and replace l_N with 2 (the theoretical maximum value)
Assume w = -w1 = (w0-w1)
Very simple GraphConv
• No Eig. Decomp.
• No Matrix Polynomials
• single tunable parameter (w)
27. spatial interpretation of GCN’s convolution
• Weighted sum of neighbor nodes (shift) + self-link
• Number of neighbor nodes vary unlike image Cnvnet
27
Neighboring nodesSelf-link
29. Once trained, run anywhere (on different
topologies)
• The sole trainable parameter W does not care the
adjacency
29
いらすとや
https://www.irasutoya.com
GCN
Trained GCN
Applicable for
Different Graphs
30. The simplest GCN… it works quite well
30
[Kipf_Welling17GCN]
Perform better than
the original ChebNet
31. ? ?
Exemplar task: semi-supervised node
classification with GCN
31
Number of Nodes: N (many) Set of labeled nodes (supervisions)
Set of unlabeled nodes
Goal: predict the labels of unlabeled nodes yj
using a trained model (GCN and classifier)
Train: tune parameters of the model to recover yi
32. Forward path
Classifier computes the label prob. distribution based on the final H
Apply L-layered GCN to update the latent node representations, H
C = { , }
32
The initial node vector
= node attributes
q: trainable param of
a classifier
33. Objective Function and Training
33
``Labeled’’ Objective function: cross entropy
≒ accuracy of label prediction on Labeled Nodes
Training: find the best parameters W(GCN) and q(classifier)
to minimize the negative of the objective function L
34. Prediction using the Trained Model
34
``unlabeled’’ Predict yj (a label of node j) in unlabeled set,
perform forward path from the observed xj
36. Interacting proteins
• Where is a binding site between two proteins?
– Biding site: where a ligand and a receptor interacts
36
Ligand
Receptor
human organ, cell, ….
Drug A
Drug A’
Binding sites:
interface for connection
37. Predict binding site nodes by GNN [Fout17Interface]
• Predict the binding site regions of ligand-receptor
proteins
– Possible binding = possible drug discovery and
improvement
• Protein: a graph of inter-connected amino acids
• Use GNN for the binding prediction
37
38. Attributed Protein Graphs
38
[Sanyal20PGCN]
Node (residue): a sub-structure
consists of amino acids
Ligand protein graph
Receptor protein graph
Binding sites: specific nodes interact between ligand and receptor
39. Training/Test of Interaction Prediction [Fout17Interface]
• Training: optimize parameters to predict the interaction
label between a node pair ( , ) (for all training
sampls)
• Test: predict interactions of node pairs in unseen
Ligand-Receptor pairs
– “Train once, run any graphs (with different topology)”
39
minimize
41. GCN formulation choices
41
Considering nodes
features only
Node features plus
Edge features
With ordering of
nearest node orders
Neighbor nodes j are ordered by distances from the node i
[Fout17Interface]
45. Scene Graph: summary graph of image contents
• Useful representation of images’ understanding
• Want to generate a scene graph from image input
45
[Qi19AttentiveRN]
In a scene graph of an image,
node: object region in an image
edge: relation between object regions
46. Graph R-CNN for scene-graph generation
[Yang18GRCNN]
• Consists of three inner components
– (a) (b) Off-the-shelf object detector [Ren15RCNN] (omit)
– (b) (c) RePN to prune unnecessary branches (omit)
– (c) (d) Attentional GCN to predict labels of nodes and edges
[Yang18GRCNN]46
47. Graph
Expansion
Expand Graph with Relation Nodes
47
[Yang18GRCNN]
Objects detected
Original input graph
(sparsified by RePN):
node = object
edge = relation Object node
Obj-Obj edge
Relation node
Obj-Rel node
One GNN can infer
all objects and relations
48. Attentional GCN [Yang18GRCNN]
• GCN update with Attention [Bahdanau15Attention] -based variable
connection
48
GCN
Attentional GCN
Adjacency is fixed
Object node’s
latent vector
Relation node’s
latent vector
Parameter f can switch based on node types
Attention conncection
49. Ground Truth Scene GraphExpanded Graph
[Yang18GRCNN]
Inferring labels of objects/relations
49
Object node’s latent vector
Relation node’s latent vector
53. Theoretical Topics of GNN
• “Deep” GNNs do no work well
– “Oversmoothing”
– Current solution: normalizer + residual connection
• The theoretical limit of representation powers of GNNs
– Graph Isomorphism test
– Invariance/Equivalence
53
54. ``Deep” GNNs do not work well
54
[Kipf_Welling17GCN]
Better
Quite different from the successful deep Conv models in Computer vision
55. Oversmoothing Problem [Li18Oversmoothing]
• Latent node vectors get closer to each other as the
GCN layers go deeper (higher)
• Difficult to distinguish nodes in deeper GCNs
55
Latent node vectors of ``Karate club’’ social network [Li18Oversmoothing]
56. Oversmoothing is theoretically proven
• Deeper GCN converges to a solution where
connected nodes will have similar latent vectors
[Li18Oversmoothing, NT_Maehara19revisit]
• Such convergence in GCN proceeds very quickly
(exponential to the depth), regardless of the initial
node vectors [Oono_Suzuki20Exponential]
– Similar conclusion also holds for a generic GNN
56
57. A good workaround [Zhou20Deep, Chen20SimpleDeep, Li20DeeperGCN]
• Combining a proper normalizer and a residual term
– Normalizing the latent node vectors keep distant
from each other [Zhao_Akoglu20PairNorm]
– Residual terms keep the norm of loss gradients in
a moderate scale [Kipf_Welling17GCN]
57
Residual term:
Add the current layer ‘’as it is’’
Not a workaround for “deeper layers, stronger GNNs” (like image recognition)
58. The theoretical limit of representation powers of
GNNs
• Surprisingly, the limitation of GNNs is already known in
the problem of “graph isomorphism” test
• The theory does NOT directly give limitations of GNN’s
power for other tasks (i.e. node classification), but it is
loosely related [Chen19RGNN]
58
59. Graph isomorphism problem (for theory of GNNs)
• Classify whether two given graphs have an edge-
preserving bijective map of node sets
– the same topology in terms of the edges?
59
[ https://ja.wikipedia.org/wiki/グラフ同型 ]
The same
graph?
60. Weisfeiler-Lehman (WL) test algorithm [Weisfeiler_Lehman68WL]
• A popular heuristics to test the isomorphism
• Idea: concatenate the neighbour nodes’ labels to check
the edge topology
– Used in graph kernels [Shervashidze11WLKernel, Togninalli18WWL] and
GNNs [Jin17WLorg, Morris19kGNN]
60
62. Upper limit of the GNN’s representation power
• In terms of the Graph Isomorphism problem,
a generic GNN ≦ WL test algorithm [Xu19GIN, Morris19WL]
– There could be a case where WL test can decide
isomorphism but GNN can not.
62
63. Graph Isomorphism Network (GIN) [Xu19GIN]
• Proposed a specific GNN architecture that attain the
same graph isomorphism detection power
63
Each layer update must be as powerful as MLP
A layer update of
typical GNNs
One non-linear activation function
A layer update of
the proposed GIN
64. Higher-order WL/GNN
• k-dimensional WL (k-WL) test
– labels the k-tuple of nodes
– powerful than the originalWL
– K-GNN [Morris19kGNN] is as good
as k-WL test
64
[Maron19NeurIPS]
65. More powerful GNNs [Sato20Survey]
• K-order graph network
– consists of all linear functions that are invariant or
equivariant to node permutations [Maron19ICLR]
– as powerful as k-GNN, memory-efficient [Maron19NeurIPS]
• CPNGNN introduces the local node ordering, and is
strictly powerful than GIN [Sato19CPNGNN]
65
67. Take home message (Revisited)
• Graph Neural Networks (GNNs): Neural Networks (NNs) to
compute nodes’ representation of graph-structured data
• Practical applications in industry, hard competitions in academia
• Model: The fundamental is approximated Graph Convolution
• Applications: applicable in several tasks in different domains
67
69. Surveys and documents for studying GNN
• Survey papers on GNN
– [Wu19survey] [Zhou18survey]
– For experts: [Battaglia18survey]
• Tutorials, slideshare, blogs
– English: [Kipf18talk], [Ma20AAAI]
– 日本語: [Honda19GNN]
69
70. The most famous dataset resources
• Prof. Leskovec (stanford)
– Stanford Large Network Dataset collections
http://snap.stanford.edu/data/index.html
• UC Santa Cruz LINQS group
– https://linqs.soe.ucsc.edu/data
– train/valid/test split:
https://github.com/kimiyoung/planetoid
70
71. Finally: GNN for your use
• Say good-bye for “D = {vector x_i, label y_i}” framework
with GNNs
– Applicable for generic structured data = graph
– The standard GCN is strong enough
• Many diverse application fields
– chemo, pharmacy, materials, computer vision, social
networks, …
71
72. Dataset for specific domains
• MoleculeNet [Wu18MoleculeNet]
– Molecular graph datasets from several chemical field.
TensorFlow implmenetaions attached
• Scene Graphs
– Visual Genome[Krishna17VG]: 108K images
• Pointcloud
– S3DIS [Armeni16S3DIS]: inside office
72
73. Programing Libraries
• Recommended: PyTorch Geometric
https://github.com/rusty1s/pytorch_geometric
• Deep Graph Library https://www.dgl.ai/
• Chainer Chemistry https://github.com/pfnet-
research/chainer-chemistry
– Tailored for molecular graphs, but also applicable for
other domains
73
74. References A-G
[Anderson19Cormorant] Anderson+, “Comorant: Covariant Molecular Neural Networks”, NeurIPS, 2019.
[Armeni16S3DIS] Armeni+, “3D Semantics Parsing of Large-scale Indoor Spaces”, CVPR, 2016.
[Bahdanau15Attention] Bahdanau+, “Neural Machine Translation by Jointly Learning to Align and Translate”, ICLR,
2015.
[Battaglia18survey] Battaglia+, “Relational Inductive Biases, Deep Learning, and Graph Networks”, arXiv:
1806.01261v3 [cs.LG], 2018.
[Bruna14Spectral] Bruna+, “Spectral Networks and Locally Connected Networks on Graphs”, ICLR, 2014.
[Chem19RGNN] Chen+, “On the Equivalence between Graph Isofmorphism Testing and Function Approximaion with
GNNs”, NeurIPS 2019.
[Chen20SimpleDeep] Chen+, “Simple and Deep Graph Convolutional Networks”, ICML 2020.
[DeCao19QA2] De Cao+, “Question Answering by Reasoning Across Documents with Graph Convolutional Networks“,
ICML Workshop, 2019.
[Defferrard16ChebNet] Defferrard+, “Convoluional Neural Networks on Graphs with Fast Localized Spectral Filtering”,
NIPS, 2016.
[Fout17Interface] Fout+, “Protein Interface Prediction using Graph Convolutional Networks”, NIPS, 2017.
[Gilmer17MPNN] Gilmer+, “Neural Message Passing for Quantum Chemistry”, ICML, 2017.
75. References H-K
[Hamilton17GraphSAGE] Hamilton+, “Inductive Representation Learning on Large Graphs”, NIPS 2017.
[Honda19GNN] Honda, “GNNまとめ(1-3)”, 2019. https://qiita.com/shionhonda/items/d27b8f13f7e9232a4ae5
https://qiita.com/shionhonda/items/0d747b00fe6ddaff26e2
https://qiita.com/shionhonda/items/e11a9cf4699878723844
[Hu20RandLaNet] Hu+, “RandLA-Net: Efficient Semantic Segmentation of Large-Scale Point Clouds”, CVPR 2020.
[Jin17WLorg] Jin+, “Predicting Organic Reaction Outcomes with Weisfeiler-Lehman Network”, NIPS 2017.
[Jin18JTVAE] Jin+, “Junction Tree Variational Autoencoder for Molecular Graph Generation”, ICML 2018.
[Kipf18talk] Kipf, “Structured Deep Models: Deep learning on graphs and beyond”, 2018.
http://tkipf.github.io/misc/SlidesCambridge.pdf
[Kipf_Welling17GCN] Kipf and Welling, “Semi-supervised Classification with Graph Convolutional Networks”, ICLR
2017.
[Klicepra20DimeNet] Klicepra+, “Directional Message Passing for Molecular Graphs”, ICLR 2020.
[Krizhevsky12ConvNet] Krizhevsky+, “ImageNet Classification with Deep Convolutional Neural Networks”, NIPS 2012.
[Krishna17VG] Krishna+, “Visual genome: Connecting language and vision using crowdsourced dense image
annotations”, ICCV, 2017. 75
76. References L
[Landrieu_Boussaha19PCS] Landrieu and Boussaha, “Point Cloud Oversegmentation with Graph-Structured Deep
Metric Learning”, CVPR, 2019.
[Laugier18CGCNN] Laugier+, “Predicting thermoelectric properties from crystal graphs and material descriptors - first
application for functional materials”, NeurIPS Workshop, 2018.
[Li16GGNN] Li+, “Gated Graph Sequence Neural Networks”, ICLR, 2016.
[Li18Oversmoothing] Li+, “Deeper Insights into Graph Convolutional Networks for Semi-Supervised Learning”, arXiv:
1801.07606 [cs.LG], 2018.
[Li19DeepGCNs] Li+, “DeepGCNs: can GCNs go as deep as CNNs?”, ICCV 2019.
[Li20DeperGCN] Li+, “DeeperGCN: ALL You Need to Train Deeper GCNs”, arXiv: 2006.07739, 2020
[Lu16VRD] Lu+,”Visual Relationship Detection with Language Priors”, ECCV, 2016.
[Liu18CGVAE] Liu+, “Constrained Graph Variational Autoencoders for Molecule Design”, NeurIPS 2018.
76
77. References M-P
[Ma20tutorial] Ma+, “Graph Neural Networks: Models and Applications”, AAAI, 2020.
http://cse.msu.edu/~mayao4/tutorials/aaai2020/
[Madhawa19GNVP] Madhawa+, “GraphNVP: An Invertible Flow Model for Generating Molecular Graphs”, arXiv:
1905.11600, 2019.
[Marrn19NeurIPS] Marron+, “Provably Powerful Graph Networks”, NeurIPS, 2019.
[Maron19ICLR] Marron+, “Invariant and Equivariant Graph Networks”, ICLR 2019.
[Morris19kGNN] Morris+, “Weisfeiler and Lehman Go Neural: Higher-order Graph Neural Networks”, AAAI, 2019.
[Ngueyn_Maehara20Homomorph] Nguyen and Maehara, “Graph Homomorphism Network”, ICML, 2020.
[NT_Maehara19revisit] NT and Maehara, “Revisiting Graph Neural Networks: All We Have is Low-pass Filters”, arXiv:
1905.09550, 2019.
[Oono_Suzuki20Exponential] Oono and Suzuki, “Graph Neural Networks Exponentially Lose Expressive Power for
Node Classification”, ICLR 2020.
[Pei20GeomGCN] Pei+, “Geom-GCN: Geometric Graph Convolutional Networks”, ICLR, 2020.
77
78. References Q-V
[Qi19AttentiveRN] Qi+, “Attentive Relataional Networks for Mapping Images to Scene Graphs”, CVPR, 2019.
[Sanchez-Gonzalez20GNS] Sanchez-Gonzalez+, “Learning to Simulate Complex Physics with Graph Networks”,
arXiv: 2002.09405. 2020.
[Sanyal20PGCN] Sanyal+, “Protein GCN: Protein model quality assessment using graph convolutional networks”,
bioRxiv, 2020.
[Sato19CPNGNN] Sato+, “Approximation Ratios of Graph Neural Networks for Combinatorial Problems”, NeurIPS,
2019.
[Sato20Survey] Sato, “A Survey on the Expressive Power of Graph Neural Networks”, arXiv: 2003.04078.
[Scarselli08GNN] Scarselli+, “The Graph Neural Network Model”, IEEE Trans. Neural Networks, 20(1), pp. 61-80,
2008
[Schlichtkrull18RGCN] Modeling Relational Data with Graph Convolutional Networks, ESWC 2018.
[Shervashidze11WLKernel: Shervashidez+, “Weisfeiler-Lehman Graph Kernels”, JMLR, 12, pp.2539-2661, 2011.
[Shuman13Graph] Shuman+, “The emerging field of signal processing on graphs: extending high-dimensional data
analysis to networks and other irregular domains”, IEEE Signal Processing Magazine, 30(3), pp.83-98, 2013
[Togninalli18WWL] Togninalli+, “Wasserstein Weisfeiler-Lehman Graph Kernels”, NeurIPS 2018.
[Veličković18GAT] Veličković+, “Graph Attention Networks”, ICLR 2018.
79. References W-Z
[Wang18NLNN] Wang+, “Non-local Neural Networks”, CVPR 2018.
[Weisfeiler_Lehman68WL] Weisfeiler and Lehman, “A Reduction of a Graph to a Canonical Form and an Algebra
Arising during this Reduction”, Nauchno-Technicheskaya Informatsia, Ser.2(9), pp.12-16, 1968.
[Wu18MoleculeNet] Wu+, “MoleculeNet: a benchmark for molecular machine learning”, Chemical Science, 9(513),
2018.
[Wu19SGC] Wu+, “Simplifying Graph Convolutional Networks”, in Proc. ICML, 2019.
[Wu19survey] Wu+, “A Comprehensive Survey on Graph Neural Networks”, arXiv:1901.00596v1 [cs.LG], 2019.
[Xu19GIN] Xu+, “How powerful are Graph Neural Networks?”, in Proc. ICLR, 2019.
[Yang18GRCNN] Yang+, “Graph R-CNN for Scene Graph Generation”, ECCV, 2018.
[You18GCPN] You+, “Graph Convolutional Policy Network for Goal-Directed Molecular Graph Generation”, NeurIPS
2018.
[Zhao_Akoglu20PairNorm] Zhao and Akoglu, “PairNorm: Tackling Oversmoothing in GNNs”, ICLR, 2020.
[Zhou20Deep] Zhou+, “Effective Training Strategies for Deep Graph Neural Networks”, arXiv: 2006.07107, 2020.
[Zhou18survey] Zhou+, “Graph Neural Networks: A Review of Methods and Applications”, arXiv: 1812.08434v2
[cs.LG], 2018. 79
80. GNN research history
80
Original GNN
(2008)
Spectral GNN
(ICLR14)
ChebNet
(NIPS16)
GCN
(ICLR17)
GAT
(ICLR18)
MPNN
(ICML18)
GraphSAGE
(NIPS17)
Birth of GNN The Door Opener
NLNN
(CVPR18)
GIN
(ICLR19)
k-GNN
(AAAI19)
CPNGNN
(NeurIPS19)
DimeNet
(ICLR20)
Model Research
Explosion
GNN extremes
Deeper theory
/ applications
Homomorphis
m (ICML20)
SGC
(ICML19)
Original GNN [Scarteslli08GNN] Spectral GNN [Bruna14Spectral] ChebNet[Defferrard16Cheb]
GGNN[Li16GGNN] GCN [Kipf_Welling17GCN] GraphSAGE [Hamilton17GraphSAGE] GAT [Veličković18GAT]
MPNN [Gilmer17MPNN] NLNN [Wang18NLNN] GIN [Xu19GIN] K-GNN [Morris19kGNN] SGC [Wu19SGC]
CPNGNN [Sato19CPNGNN] Graph Homomorphism Convolution [Ngueyn_Maehara20Homomorph] DimeNet [Klcepra20DimeNet]
81. Def: Graph signals
• Suppose the 1D node attribute case: N-array
81
Low frequency graph signal
High frequency graph signal
[Ma20tutorial]