SlideShare a Scribd company logo
1 of 16
Towards Predicting Molecular
Property by Graph Neural
Networks
Shion HONDA
The Graduate School of Information Science
and Technology,
The University of Tokyo
@ National Taiwan University
Contents
• Basics of molecular property prediction
• Basics of graph theory
• Introduction to graph convolution
• Graph convolutional networks
• Recent advancements
2019/3/20 Shion HONDA 2
Who Am I?
• Studying cheminformatics, or ML application
to drug discovery
• My interests: NLP, CV, GANs, RL etc
• SNS
2019/3/20 Shion HONDA 3
@shion_honda
@shionhonda
Why Graphs?
• Many kinds of data can be represented as
graphs
• Web
• Traffic network
• Social network
• Citation network
• Neuronal network
• Molecules
2019/3/20 Shion HONDA 4
Zachary’s karate club
Salicylic acid
Molecular Property Prediction
• When predicting molecular property by some
model, molecular fingerprints are useful
• Hand-crafted fingerprints
• MACCS Keys
• Circular fingerprint (most popular)
• Binary sparse vector
• Learned fingerprints
• SMILES-based
• Graph neural networks
2019/3/20 Shion HONDA 5
c1ccccc1C(=O)O
Graphs Are Difficult
• Unaligned structure
• Graphs are not aligned like images/texts
• Different structures and different tasks
• Directed vs undirected
• Weighted vs unweighted etc.
• Link prediction vs graph embedding
• Scalability
• Some graphs (e.g., web, SNS) are huge
• Domain knowledge
• In the case of molecules, link degrees are up to 5
• In the case of SNS, links are unevenly distributed
2019/3/20 Shion HONDA 6
Preliminaries And Definition
2019/3/20 Shion HONDA 7
Symbol Meaning
V, E Set of nodes, edges
N, M Number of nodes, edges
G=(V,E) Graph
𝐹 𝑉
, 𝐹 𝐸 Feature vector of nodes, edges
A Adjacency matrix
D Degree matrix
L=D-A Laplacian matrix
H Hidden vectors
Normalized Laplacian matrix
Learnable parameters
k-hop neighbors from node i
Graph Fourier Transform
• Eigen decomposition of normalized Laplacian
• U is a unitary matrix as L is a real symmetric
(Hermitian) matrix
• Graph Fourier transform of a signal on a
graph is defined as:
• Inverse operation
2019/3/20 Shion HONDA 8
Graph Convolution
• Convolution theorem
• Convolution of a filter
where
• Graph convolution is defined as a product of
matrices
2019/3/20 Shion HONDA 9
Naïve GCN
• is a filter
->Analogy of CNNs for images
• Arbitrary network can be
constructed by repeating
this operation
• Problems
• Eigen-decomposition has at least O(N^2) time
complexity
• Parameters cannot be shared over graphs of
different sizes
2019/3/20 Shion HONDA 10
ChebNet
• Eigen-decomposition can be avoided by
Chebyshev polynomials
• Learnable parameters are:
• Then graph convolution is defined as:
2019/3/20 Shion HONDA 11
GCN (Kipf & Welling)
• ChebNet when
• By assuming , graph conv. is
• By replacing and
stacking over channels to
where
2019/3/20 Shion HONDA 12
Independent of N
GCN Variants
• Neural Fingerprint
• PATCHY-SAN
• Message Passing Neural Network
• Graph Network
• Relational GCN
• Jumping Knowledge Network
2019/3/20 Shion HONDA 13
Recent Advances
• Autoencoder/VAE
• Decode in a non-parametric manner
• GAN + reinforcement learning
• Generate drug candidates
• e.g., MolGAN, GCPN
• Dynamic graphs
• In most of the real problems, graphs are dynamic
• e.g., web, electricity, SNS
• GCN+LSTM
2019/3/20 Shion HONDA 14
Summary
• Graph convolution can be defined with graph
Fourier transform
• Chebyshev polynomials approximation is
used to avoid computationally-expensive
eigen-decomposition
• Several other GCNs have been proposed
• Next direction: Generative models and
dynamic graphs
2019/3/20 Shion HONDA 15
References
[1] Z. Zhang et al., Deep Learning on Graphs: A Survey, arXiv, 2018.
[2] J. Zhou et al., Graph Neural Networks: A Review of Methods and
Applications, arXiv, 2018.
[3] Z. Wu et al., A Comprehensive Survey on Graph Neural Networks,
arXiv, 2019.
[4] D. I. Shuman et al., The Emerging Field of Signal Processing on
Graphs: Extending High-Dimensional Data Analysis to Networks and
Other Irregular Domains, IEEE Signal Processing Magazine, 2013.
[5]グラフラプラシアン - 初級Mathマニアの寝言
2019/3/20 Shion HONDA 16

More Related Content

Similar to Towards Predicting Molecular Property by Graph Neural Networks

[20240318_LabSeminar_Huy]GSTNet: Global Spatial-Temporal Network for Traffic ...
[20240318_LabSeminar_Huy]GSTNet: Global Spatial-Temporal Network for Traffic ...[20240318_LabSeminar_Huy]GSTNet: Global Spatial-Temporal Network for Traffic ...
[20240318_LabSeminar_Huy]GSTNet: Global Spatial-Temporal Network for Traffic ...thanhdowork
 
Emerging Technologies in On-Chip and Off-Chip Interconnection Networks
Emerging Technologies in On-Chip and Off-Chip Interconnection NetworksEmerging Technologies in On-Chip and Off-Chip Interconnection Networks
Emerging Technologies in On-Chip and Off-Chip Interconnection NetworksAshif Sikder
 
2017 09-ohkawa-MCSoC2017-presen
2017 09-ohkawa-MCSoC2017-presen2017 09-ohkawa-MCSoC2017-presen
2017 09-ohkawa-MCSoC2017-presenTakeshi Ohkawa
 
Kharita: Robust Road Map Inference Through Network Alignment of Trajectories
Kharita: Robust Road Map Inference Through Network Alignment of TrajectoriesKharita: Robust Road Map Inference Through Network Alignment of Trajectories
Kharita: Robust Road Map Inference Through Network Alignment of Trajectoriesvipyoung
 
2019 3 testing and verification of vlsi design_sta
2019 3 testing and verification of vlsi design_sta2019 3 testing and verification of vlsi design_sta
2019 3 testing and verification of vlsi design_staUsha Mehta
 
[20240311_LabSeminar_Huy]LINE: Large-scale Information Network Embedding.pptx
[20240311_LabSeminar_Huy]LINE: Large-scale Information Network Embedding.pptx[20240311_LabSeminar_Huy]LINE: Large-scale Information Network Embedding.pptx
[20240311_LabSeminar_Huy]LINE: Large-scale Information Network Embedding.pptxthanhdowork
 
Camera-Based Road Lane Detection by Deep Learning II
Camera-Based Road Lane Detection by Deep Learning IICamera-Based Road Lane Detection by Deep Learning II
Camera-Based Road Lane Detection by Deep Learning IIYu Huang
 
PacNOG 31: Internet Exchange Points
PacNOG 31: Internet Exchange PointsPacNOG 31: Internet Exchange Points
PacNOG 31: Internet Exchange PointsAPNIC
 
PITA 27th AGM & Business Forum Expo 23: Internet Exchange Points
PITA 27th AGM & Business Forum Expo 23: Internet Exchange PointsPITA 27th AGM & Business Forum Expo 23: Internet Exchange Points
PITA 27th AGM & Business Forum Expo 23: Internet Exchange PointsAPNIC
 
3D SLAM introcution& current status
3D SLAM introcution& current status3D SLAM introcution& current status
3D SLAM introcution& current statuse8xu
 
[20240408_LabSeminar_Huy]PivotalSTGNN.pptx
[20240408_LabSeminar_Huy]PivotalSTGNN.pptx[20240408_LabSeminar_Huy]PivotalSTGNN.pptx
[20240408_LabSeminar_Huy]PivotalSTGNN.pptxthanhdowork
 
Crash course on data streaming (with examples using Apache Flink)
Crash course on data streaming (with examples using Apache Flink)Crash course on data streaming (with examples using Apache Flink)
Crash course on data streaming (with examples using Apache Flink)Vincenzo Gulisano
 
Aalto University Mobile Management in SDN
Aalto University Mobile Management in SDNAalto University Mobile Management in SDN
Aalto University Mobile Management in SDNHector Fuentes
 
FrrbaseViz-A Tool for Exploring Freebase Using Query-Driven Visualisation
FrrbaseViz-A Tool for Exploring Freebase Using Query-Driven VisualisationFrrbaseViz-A Tool for Exploring Freebase Using Query-Driven Visualisation
FrrbaseViz-A Tool for Exploring Freebase Using Query-Driven VisualisationMahmoud Elbattah
 
Efficient architecture for arithmetic designs using perpendicular NanoMagneti...
Efficient architecture for arithmetic designs using perpendicular NanoMagneti...Efficient architecture for arithmetic designs using perpendicular NanoMagneti...
Efficient architecture for arithmetic designs using perpendicular NanoMagneti...VIT-AP University
 

Similar to Towards Predicting Molecular Property by Graph Neural Networks (20)

PCL (Point Cloud Library)
PCL (Point Cloud Library)PCL (Point Cloud Library)
PCL (Point Cloud Library)
 
Vlsi td introduction
Vlsi td introductionVlsi td introduction
Vlsi td introduction
 
lecture_16_jiajun.pdf
lecture_16_jiajun.pdflecture_16_jiajun.pdf
lecture_16_jiajun.pdf
 
[20240318_LabSeminar_Huy]GSTNet: Global Spatial-Temporal Network for Traffic ...
[20240318_LabSeminar_Huy]GSTNet: Global Spatial-Temporal Network for Traffic ...[20240318_LabSeminar_Huy]GSTNet: Global Spatial-Temporal Network for Traffic ...
[20240318_LabSeminar_Huy]GSTNet: Global Spatial-Temporal Network for Traffic ...
 
Multicore architectures
Multicore architecturesMulticore architectures
Multicore architectures
 
Emerging Technologies in On-Chip and Off-Chip Interconnection Networks
Emerging Technologies in On-Chip and Off-Chip Interconnection NetworksEmerging Technologies in On-Chip and Off-Chip Interconnection Networks
Emerging Technologies in On-Chip and Off-Chip Interconnection Networks
 
2017 09-ohkawa-MCSoC2017-presen
2017 09-ohkawa-MCSoC2017-presen2017 09-ohkawa-MCSoC2017-presen
2017 09-ohkawa-MCSoC2017-presen
 
Kharita: Robust Road Map Inference Through Network Alignment of Trajectories
Kharita: Robust Road Map Inference Through Network Alignment of TrajectoriesKharita: Robust Road Map Inference Through Network Alignment of Trajectories
Kharita: Robust Road Map Inference Through Network Alignment of Trajectories
 
2019 3 testing and verification of vlsi design_sta
2019 3 testing and verification of vlsi design_sta2019 3 testing and verification of vlsi design_sta
2019 3 testing and verification of vlsi design_sta
 
[20240311_LabSeminar_Huy]LINE: Large-scale Information Network Embedding.pptx
[20240311_LabSeminar_Huy]LINE: Large-scale Information Network Embedding.pptx[20240311_LabSeminar_Huy]LINE: Large-scale Information Network Embedding.pptx
[20240311_LabSeminar_Huy]LINE: Large-scale Information Network Embedding.pptx
 
Presentation
PresentationPresentation
Presentation
 
Camera-Based Road Lane Detection by Deep Learning II
Camera-Based Road Lane Detection by Deep Learning IICamera-Based Road Lane Detection by Deep Learning II
Camera-Based Road Lane Detection by Deep Learning II
 
PacNOG 31: Internet Exchange Points
PacNOG 31: Internet Exchange PointsPacNOG 31: Internet Exchange Points
PacNOG 31: Internet Exchange Points
 
PITA 27th AGM & Business Forum Expo 23: Internet Exchange Points
PITA 27th AGM & Business Forum Expo 23: Internet Exchange PointsPITA 27th AGM & Business Forum Expo 23: Internet Exchange Points
PITA 27th AGM & Business Forum Expo 23: Internet Exchange Points
 
3D SLAM introcution& current status
3D SLAM introcution& current status3D SLAM introcution& current status
3D SLAM introcution& current status
 
[20240408_LabSeminar_Huy]PivotalSTGNN.pptx
[20240408_LabSeminar_Huy]PivotalSTGNN.pptx[20240408_LabSeminar_Huy]PivotalSTGNN.pptx
[20240408_LabSeminar_Huy]PivotalSTGNN.pptx
 
Crash course on data streaming (with examples using Apache Flink)
Crash course on data streaming (with examples using Apache Flink)Crash course on data streaming (with examples using Apache Flink)
Crash course on data streaming (with examples using Apache Flink)
 
Aalto University Mobile Management in SDN
Aalto University Mobile Management in SDNAalto University Mobile Management in SDN
Aalto University Mobile Management in SDN
 
FrrbaseViz-A Tool for Exploring Freebase Using Query-Driven Visualisation
FrrbaseViz-A Tool for Exploring Freebase Using Query-Driven VisualisationFrrbaseViz-A Tool for Exploring Freebase Using Query-Driven Visualisation
FrrbaseViz-A Tool for Exploring Freebase Using Query-Driven Visualisation
 
Efficient architecture for arithmetic designs using perpendicular NanoMagneti...
Efficient architecture for arithmetic designs using perpendicular NanoMagneti...Efficient architecture for arithmetic designs using perpendicular NanoMagneti...
Efficient architecture for arithmetic designs using perpendicular NanoMagneti...
 

More from Shion Honda

BERTをブラウザで動かしたい! ―MobileBERTとTensorFlow.js―
BERTをブラウザで動かしたい!―MobileBERTとTensorFlow.js―BERTをブラウザで動かしたい!―MobileBERTとTensorFlow.js―
BERTをブラウザで動かしたい! ―MobileBERTとTensorFlow.js―Shion Honda
 
Bridging between Vision and Language
Bridging between Vision and LanguageBridging between Vision and Language
Bridging between Vision and LanguageShion Honda
 
Deep Learning Chap. 12: Applications
Deep Learning Chap. 12: ApplicationsDeep Learning Chap. 12: Applications
Deep Learning Chap. 12: ApplicationsShion Honda
 
Deep Learning Chap. 6: Deep Feedforward Networks
Deep Learning Chap. 6: Deep Feedforward NetworksDeep Learning Chap. 6: Deep Feedforward Networks
Deep Learning Chap. 6: Deep Feedforward NetworksShion Honda
 
画像認識 第9章 さらなる話題
画像認識 第9章 さらなる話題画像認識 第9章 さらなる話題
画像認識 第9章 さらなる話題Shion Honda
 
画像認識 6.3-6.6 畳込みニューラル ネットワーク
画像認識 6.3-6.6 畳込みニューラルネットワーク画像認識 6.3-6.6 畳込みニューラルネットワーク
画像認識 6.3-6.6 畳込みニューラル ネットワークShion Honda
 
深層学習による自然言語処理 第2章 ニューラルネットの基礎
深層学習による自然言語処理 第2章 ニューラルネットの基礎深層学習による自然言語処理 第2章 ニューラルネットの基礎
深層学習による自然言語処理 第2章 ニューラルネットの基礎Shion Honda
 
BERT: Pre-training of Deep Bidirectional Transformers for Language Understand...
BERT: Pre-training of Deep Bidirectional Transformers for Language Understand...BERT: Pre-training of Deep Bidirectional Transformers for Language Understand...
BERT: Pre-training of Deep Bidirectional Transformers for Language Understand...Shion Honda
 
IaGo: an Othello AI inspired by AlphaGo
IaGo: an Othello AI inspired by AlphaGoIaGo: an Othello AI inspired by AlphaGo
IaGo: an Othello AI inspired by AlphaGoShion Honda
 
Planning chemical syntheses with deep neural networks and symbolic AI
Planning chemical syntheses with deep neural networks and symbolic AIPlanning chemical syntheses with deep neural networks and symbolic AI
Planning chemical syntheses with deep neural networks and symbolic AIShion Honda
 

More from Shion Honda (11)

BERTをブラウザで動かしたい! ―MobileBERTとTensorFlow.js―
BERTをブラウザで動かしたい!―MobileBERTとTensorFlow.js―BERTをブラウザで動かしたい!―MobileBERTとTensorFlow.js―
BERTをブラウザで動かしたい! ―MobileBERTとTensorFlow.js―
 
Bridging between Vision and Language
Bridging between Vision and LanguageBridging between Vision and Language
Bridging between Vision and Language
 
Graph U-Nets
Graph U-NetsGraph U-Nets
Graph U-Nets
 
Deep Learning Chap. 12: Applications
Deep Learning Chap. 12: ApplicationsDeep Learning Chap. 12: Applications
Deep Learning Chap. 12: Applications
 
Deep Learning Chap. 6: Deep Feedforward Networks
Deep Learning Chap. 6: Deep Feedforward NetworksDeep Learning Chap. 6: Deep Feedforward Networks
Deep Learning Chap. 6: Deep Feedforward Networks
 
画像認識 第9章 さらなる話題
画像認識 第9章 さらなる話題画像認識 第9章 さらなる話題
画像認識 第9章 さらなる話題
 
画像認識 6.3-6.6 畳込みニューラル ネットワーク
画像認識 6.3-6.6 畳込みニューラルネットワーク画像認識 6.3-6.6 畳込みニューラルネットワーク
画像認識 6.3-6.6 畳込みニューラル ネットワーク
 
深層学習による自然言語処理 第2章 ニューラルネットの基礎
深層学習による自然言語処理 第2章 ニューラルネットの基礎深層学習による自然言語処理 第2章 ニューラルネットの基礎
深層学習による自然言語処理 第2章 ニューラルネットの基礎
 
BERT: Pre-training of Deep Bidirectional Transformers for Language Understand...
BERT: Pre-training of Deep Bidirectional Transformers for Language Understand...BERT: Pre-training of Deep Bidirectional Transformers for Language Understand...
BERT: Pre-training of Deep Bidirectional Transformers for Language Understand...
 
IaGo: an Othello AI inspired by AlphaGo
IaGo: an Othello AI inspired by AlphaGoIaGo: an Othello AI inspired by AlphaGo
IaGo: an Othello AI inspired by AlphaGo
 
Planning chemical syntheses with deep neural networks and symbolic AI
Planning chemical syntheses with deep neural networks and symbolic AIPlanning chemical syntheses with deep neural networks and symbolic AI
Planning chemical syntheses with deep neural networks and symbolic AI
 

Recently uploaded

一比一原版阿德莱德大学毕业证成绩单如何办理
一比一原版阿德莱德大学毕业证成绩单如何办理一比一原版阿德莱德大学毕业证成绩单如何办理
一比一原版阿德莱德大学毕业证成绩单如何办理pyhepag
 
Easy and simple project file on mp online
Easy and simple project file on mp onlineEasy and simple project file on mp online
Easy and simple project file on mp onlinebalibahu1313
 
一比一原版纽卡斯尔大学毕业证成绩单如何办理
一比一原版纽卡斯尔大学毕业证成绩单如何办理一比一原版纽卡斯尔大学毕业证成绩单如何办理
一比一原版纽卡斯尔大学毕业证成绩单如何办理cyebo
 
Artificial_General_Intelligence__storm_gen_article.pdf
Artificial_General_Intelligence__storm_gen_article.pdfArtificial_General_Intelligence__storm_gen_article.pdf
Artificial_General_Intelligence__storm_gen_article.pdfscitechtalktv
 
Pre-ProductionImproveddsfjgndflghtgg.pptx
Pre-ProductionImproveddsfjgndflghtgg.pptxPre-ProductionImproveddsfjgndflghtgg.pptx
Pre-ProductionImproveddsfjgndflghtgg.pptxStephen266013
 
一比一原版加利福尼亚大学尔湾分校毕业证成绩单如何办理
一比一原版加利福尼亚大学尔湾分校毕业证成绩单如何办理一比一原版加利福尼亚大学尔湾分校毕业证成绩单如何办理
一比一原版加利福尼亚大学尔湾分校毕业证成绩单如何办理pyhepag
 
Supply chain analytics to combat the effects of Ukraine-Russia-conflict
Supply chain analytics to combat the effects of Ukraine-Russia-conflictSupply chain analytics to combat the effects of Ukraine-Russia-conflict
Supply chain analytics to combat the effects of Ukraine-Russia-conflictJack Cole
 
一比一原版西悉尼大学毕业证成绩单如何办理
一比一原版西悉尼大学毕业证成绩单如何办理一比一原版西悉尼大学毕业证成绩单如何办理
一比一原版西悉尼大学毕业证成绩单如何办理pyhepag
 
The Significance of Transliteration Enhancing
The Significance of Transliteration EnhancingThe Significance of Transliteration Enhancing
The Significance of Transliteration Enhancingmohamed Elzalabany
 
Exploratory Data Analysis - Dilip S.pptx
Exploratory Data Analysis - Dilip S.pptxExploratory Data Analysis - Dilip S.pptx
Exploratory Data Analysis - Dilip S.pptxDilipVasan
 
basics of data science with application areas.pdf
basics of data science with application areas.pdfbasics of data science with application areas.pdf
basics of data science with application areas.pdfvyankatesh1
 
Data Visualization Exploring and Explaining with Data 1st Edition by Camm sol...
Data Visualization Exploring and Explaining with Data 1st Edition by Camm sol...Data Visualization Exploring and Explaining with Data 1st Edition by Camm sol...
Data Visualization Exploring and Explaining with Data 1st Edition by Camm sol...ssuserf63bd7
 
社内勉強会資料  Mamba - A new era or ephemeral
社内勉強会資料   Mamba - A new era or ephemeral社内勉強会資料   Mamba - A new era or ephemeral
社内勉強会資料  Mamba - A new era or ephemeralNABLAS株式会社
 
Formulas dax para power bI de microsoft.pdf
Formulas dax para power bI de microsoft.pdfFormulas dax para power bI de microsoft.pdf
Formulas dax para power bI de microsoft.pdfRobertoOcampo24
 
一比一原版(Monash毕业证书)莫纳什大学毕业证成绩单如何办理
一比一原版(Monash毕业证书)莫纳什大学毕业证成绩单如何办理一比一原版(Monash毕业证书)莫纳什大学毕业证成绩单如何办理
一比一原版(Monash毕业证书)莫纳什大学毕业证成绩单如何办理pyhepag
 
2024 Q2 Orange County (CA) Tableau User Group Meeting
2024 Q2 Orange County (CA) Tableau User Group Meeting2024 Q2 Orange County (CA) Tableau User Group Meeting
2024 Q2 Orange County (CA) Tableau User Group MeetingAlison Pitt
 
AI Imagen for data-storytelling Infographics.pdf
AI Imagen for data-storytelling Infographics.pdfAI Imagen for data-storytelling Infographics.pdf
AI Imagen for data-storytelling Infographics.pdfMichaelSenkow
 
Fuzzy Sets decision making under information of uncertainty
Fuzzy Sets decision making under information of uncertaintyFuzzy Sets decision making under information of uncertainty
Fuzzy Sets decision making under information of uncertaintyRafigAliyev2
 

Recently uploaded (20)

一比一原版阿德莱德大学毕业证成绩单如何办理
一比一原版阿德莱德大学毕业证成绩单如何办理一比一原版阿德莱德大学毕业证成绩单如何办理
一比一原版阿德莱德大学毕业证成绩单如何办理
 
Easy and simple project file on mp online
Easy and simple project file on mp onlineEasy and simple project file on mp online
Easy and simple project file on mp online
 
一比一原版纽卡斯尔大学毕业证成绩单如何办理
一比一原版纽卡斯尔大学毕业证成绩单如何办理一比一原版纽卡斯尔大学毕业证成绩单如何办理
一比一原版纽卡斯尔大学毕业证成绩单如何办理
 
Artificial_General_Intelligence__storm_gen_article.pdf
Artificial_General_Intelligence__storm_gen_article.pdfArtificial_General_Intelligence__storm_gen_article.pdf
Artificial_General_Intelligence__storm_gen_article.pdf
 
Slip-and-fall Injuries: Top Workers' Comp Claims
Slip-and-fall Injuries: Top Workers' Comp ClaimsSlip-and-fall Injuries: Top Workers' Comp Claims
Slip-and-fall Injuries: Top Workers' Comp Claims
 
Pre-ProductionImproveddsfjgndflghtgg.pptx
Pre-ProductionImproveddsfjgndflghtgg.pptxPre-ProductionImproveddsfjgndflghtgg.pptx
Pre-ProductionImproveddsfjgndflghtgg.pptx
 
一比一原版加利福尼亚大学尔湾分校毕业证成绩单如何办理
一比一原版加利福尼亚大学尔湾分校毕业证成绩单如何办理一比一原版加利福尼亚大学尔湾分校毕业证成绩单如何办理
一比一原版加利福尼亚大学尔湾分校毕业证成绩单如何办理
 
Supply chain analytics to combat the effects of Ukraine-Russia-conflict
Supply chain analytics to combat the effects of Ukraine-Russia-conflictSupply chain analytics to combat the effects of Ukraine-Russia-conflict
Supply chain analytics to combat the effects of Ukraine-Russia-conflict
 
一比一原版西悉尼大学毕业证成绩单如何办理
一比一原版西悉尼大学毕业证成绩单如何办理一比一原版西悉尼大学毕业证成绩单如何办理
一比一原版西悉尼大学毕业证成绩单如何办理
 
The Significance of Transliteration Enhancing
The Significance of Transliteration EnhancingThe Significance of Transliteration Enhancing
The Significance of Transliteration Enhancing
 
Exploratory Data Analysis - Dilip S.pptx
Exploratory Data Analysis - Dilip S.pptxExploratory Data Analysis - Dilip S.pptx
Exploratory Data Analysis - Dilip S.pptx
 
basics of data science with application areas.pdf
basics of data science with application areas.pdfbasics of data science with application areas.pdf
basics of data science with application areas.pdf
 
Data Visualization Exploring and Explaining with Data 1st Edition by Camm sol...
Data Visualization Exploring and Explaining with Data 1st Edition by Camm sol...Data Visualization Exploring and Explaining with Data 1st Edition by Camm sol...
Data Visualization Exploring and Explaining with Data 1st Edition by Camm sol...
 
社内勉強会資料  Mamba - A new era or ephemeral
社内勉強会資料   Mamba - A new era or ephemeral社内勉強会資料   Mamba - A new era or ephemeral
社内勉強会資料  Mamba - A new era or ephemeral
 
Formulas dax para power bI de microsoft.pdf
Formulas dax para power bI de microsoft.pdfFormulas dax para power bI de microsoft.pdf
Formulas dax para power bI de microsoft.pdf
 
一比一原版(Monash毕业证书)莫纳什大学毕业证成绩单如何办理
一比一原版(Monash毕业证书)莫纳什大学毕业证成绩单如何办理一比一原版(Monash毕业证书)莫纳什大学毕业证成绩单如何办理
一比一原版(Monash毕业证书)莫纳什大学毕业证成绩单如何办理
 
Abortion pills in Dammam Saudi Arabia// +966572737505 // buy cytotec
Abortion pills in Dammam Saudi Arabia// +966572737505 // buy cytotecAbortion pills in Dammam Saudi Arabia// +966572737505 // buy cytotec
Abortion pills in Dammam Saudi Arabia// +966572737505 // buy cytotec
 
2024 Q2 Orange County (CA) Tableau User Group Meeting
2024 Q2 Orange County (CA) Tableau User Group Meeting2024 Q2 Orange County (CA) Tableau User Group Meeting
2024 Q2 Orange County (CA) Tableau User Group Meeting
 
AI Imagen for data-storytelling Infographics.pdf
AI Imagen for data-storytelling Infographics.pdfAI Imagen for data-storytelling Infographics.pdf
AI Imagen for data-storytelling Infographics.pdf
 
Fuzzy Sets decision making under information of uncertainty
Fuzzy Sets decision making under information of uncertaintyFuzzy Sets decision making under information of uncertainty
Fuzzy Sets decision making under information of uncertainty
 

Towards Predicting Molecular Property by Graph Neural Networks

  • 1. Towards Predicting Molecular Property by Graph Neural Networks Shion HONDA The Graduate School of Information Science and Technology, The University of Tokyo @ National Taiwan University
  • 2. Contents • Basics of molecular property prediction • Basics of graph theory • Introduction to graph convolution • Graph convolutional networks • Recent advancements 2019/3/20 Shion HONDA 2
  • 3. Who Am I? • Studying cheminformatics, or ML application to drug discovery • My interests: NLP, CV, GANs, RL etc • SNS 2019/3/20 Shion HONDA 3 @shion_honda @shionhonda
  • 4. Why Graphs? • Many kinds of data can be represented as graphs • Web • Traffic network • Social network • Citation network • Neuronal network • Molecules 2019/3/20 Shion HONDA 4 Zachary’s karate club Salicylic acid
  • 5. Molecular Property Prediction • When predicting molecular property by some model, molecular fingerprints are useful • Hand-crafted fingerprints • MACCS Keys • Circular fingerprint (most popular) • Binary sparse vector • Learned fingerprints • SMILES-based • Graph neural networks 2019/3/20 Shion HONDA 5 c1ccccc1C(=O)O
  • 6. Graphs Are Difficult • Unaligned structure • Graphs are not aligned like images/texts • Different structures and different tasks • Directed vs undirected • Weighted vs unweighted etc. • Link prediction vs graph embedding • Scalability • Some graphs (e.g., web, SNS) are huge • Domain knowledge • In the case of molecules, link degrees are up to 5 • In the case of SNS, links are unevenly distributed 2019/3/20 Shion HONDA 6
  • 7. Preliminaries And Definition 2019/3/20 Shion HONDA 7 Symbol Meaning V, E Set of nodes, edges N, M Number of nodes, edges G=(V,E) Graph 𝐹 𝑉 , 𝐹 𝐸 Feature vector of nodes, edges A Adjacency matrix D Degree matrix L=D-A Laplacian matrix H Hidden vectors Normalized Laplacian matrix Learnable parameters k-hop neighbors from node i
  • 8. Graph Fourier Transform • Eigen decomposition of normalized Laplacian • U is a unitary matrix as L is a real symmetric (Hermitian) matrix • Graph Fourier transform of a signal on a graph is defined as: • Inverse operation 2019/3/20 Shion HONDA 8
  • 9. Graph Convolution • Convolution theorem • Convolution of a filter where • Graph convolution is defined as a product of matrices 2019/3/20 Shion HONDA 9
  • 10. Naïve GCN • is a filter ->Analogy of CNNs for images • Arbitrary network can be constructed by repeating this operation • Problems • Eigen-decomposition has at least O(N^2) time complexity • Parameters cannot be shared over graphs of different sizes 2019/3/20 Shion HONDA 10
  • 11. ChebNet • Eigen-decomposition can be avoided by Chebyshev polynomials • Learnable parameters are: • Then graph convolution is defined as: 2019/3/20 Shion HONDA 11
  • 12. GCN (Kipf & Welling) • ChebNet when • By assuming , graph conv. is • By replacing and stacking over channels to where 2019/3/20 Shion HONDA 12 Independent of N
  • 13. GCN Variants • Neural Fingerprint • PATCHY-SAN • Message Passing Neural Network • Graph Network • Relational GCN • Jumping Knowledge Network 2019/3/20 Shion HONDA 13
  • 14. Recent Advances • Autoencoder/VAE • Decode in a non-parametric manner • GAN + reinforcement learning • Generate drug candidates • e.g., MolGAN, GCPN • Dynamic graphs • In most of the real problems, graphs are dynamic • e.g., web, electricity, SNS • GCN+LSTM 2019/3/20 Shion HONDA 14
  • 15. Summary • Graph convolution can be defined with graph Fourier transform • Chebyshev polynomials approximation is used to avoid computationally-expensive eigen-decomposition • Several other GCNs have been proposed • Next direction: Generative models and dynamic graphs 2019/3/20 Shion HONDA 15
  • 16. References [1] Z. Zhang et al., Deep Learning on Graphs: A Survey, arXiv, 2018. [2] J. Zhou et al., Graph Neural Networks: A Review of Methods and Applications, arXiv, 2018. [3] Z. Wu et al., A Comprehensive Survey on Graph Neural Networks, arXiv, 2019. [4] D. I. Shuman et al., The Emerging Field of Signal Processing on Graphs: Extending High-Dimensional Data Analysis to Networks and Other Irregular Domains, IEEE Signal Processing Magazine, 2013. [5]グラフラプラシアン - 初級Mathマニアの寝言 2019/3/20 Shion HONDA 16

Editor's Notes

  1. But first of all, let me introduce myself briefly. I’m a master student, studying cheminformatics, or ML application to drug discovery. My interests lies over ML: NLP, CV, GANs, RL etc.
  2. Let’s move on to the topic. Why are graph neural networks important? Why do they attract many people? Because, many kinds of data can be represented as graphs and they have not been benefitted from DL so much as NLP and CV. One of the applications is chemistry. molecular property prediction, molecule generation, and so on. And I’m working on these stuff.
  3. I’ll now briefly tell you about the backgrounds of Molecular Property Prediction. When predicting molecular property by some model, vector representations, or embeddings, are useful. This feature vectors are called fingerprints in chemistry. Fingerprints have been hand-crafted for a long time. For example, MACCS Keys is a binary vector that represents the existence of pre-defined substructures. However, hand-crafted fingerprints are sparse and do not perform well for ML models. Recent researches try to learn better fingerprints applying seq2seq on the character representation called SMILES. Or, simply applying GNNs. Today I focus on the GNN approaches.
  4. Generally speaking, graphs are more complicated than texts and images. This is firstly due to their unaligned structure. There is no order in nodes, so even judging whether two given graphs are the same or not is difficult. Second, there are several types of graphs such as directed vs undirected… Scalability and Domain knowledge are also problems.
  5. Before moving on to graph convolution, let me give you definitions. To keep up with the following talk, You need to remember adjacency matrix, degree matrix. Laplacian matrix, and normalized Laplacian matrix.
  6. Graph convolution is defined through graph fourier transform. Consider Eigen decomposition of normalized Laplacian. Here, U is a unitary matrix as L is a real symmetric matrix. Now, Graph Fourier transform of a signal on a graph x (blue pillars) is defined by multiplying transposed U. Inverse operation is simple.
  7. Convolution theorem holds just as convolution on images and time-series data. I mean, convolution in space domain is equal to the element-wise product in the Fourier domain. Let theta as the diagonal matrix like this, then convolution of a filter F can be written as: Then, Graph convolution is defined as a product of matrices There is a beautiful theory behind the formula, so if you are interested, please check it out afterwards.
  8. Now, “naive” graph convolutional network can be designed by naively imitating CNN for images. You can compose arbitrary network by repeating this operation. However, there are 2 major problems: First, Eigen-decomposition has O(N^2) time complexity Second, Parameters cannot be shared over graphs of different sizes because theta depends on the size of U, the number of nodes
  9. But don’t worry. Chebyshev save the world! ChebNet avoided Eigen-decomposition by using k-degree Chebyshev polynomials! They set learnable parameters in this formula. Then graph convolution is defined without eigen decomposition.
  10. Kipf and Welling futrher approximate by considering K=1 case. By assuming thetas are the same, graph convolution is: And lastly, by replacing and stacking over channels, theta is independent of graph size N.
  11. The following researches proposed variants of GCNs. Here are some examples.
  12. I didn’t have enough time to step into recent topics such as autoencoder/VAE, GANs, RL, and dynamic graphs. If you are interested, I can give you some relevant literature.
  13. To sum up,