Training Graph Convolutional Neural Networks in Graph Database

Training Graph Convolutional Neural
Networks in Graph Database
Changran Liu

| GRAPHAIWORLD.COM | #GRAPHAIWORLD |
Outline
● Graph Convolutional Networks
(GCN) for Node Classification
● Motivations of Training GCN In
Graph Database
● Demo: Paper Classification in a
Citation Graph

Paper Classiﬁcation
Neural Network
We examine the
quantum confinement
in the photoemission
ionization energy in
air and optical band
gap of carbon
nanoparticles (CNPs)
...
air 0
aromatic 0
band gap 1
carbon 1
nanoparticle 1
orbital 0
Abstract Term
Bag of words vector
Topic
C.Liu, PNAS, (2019)
Phys 0.9
Bio 0.1
CS 0
Econ 0
Reference
[1] ...
[2] …

Citation Graph

Graph Convolutional Network

horizontal propagation

horizontal propagation
vertical propagation

Pros & Cons
● Semi-supervised approach
● High accuracy can be achieved
with a low labeling rate
● Prediction requires graph
traversal
● Size of A and X scale with number
of edges and vertices

Traditional Model Training Pipeline
training data
model
Database:
● Paper contents table
● Citation relation table
● data update
● preprocess data
Machine learning platform:
● Build feature matrix X
● Build adjacency matrix A
● model training
● model validation

In-Database Model Training
training data
model
Database:
● Citation graph
● data update
● preprocess data
● Model training
Machine learning platform:
● Build adjacency matrix A
● Build feature matrix X
● model training
● model validation
● The adjacency matrix is stored as a
graph in the database.
● Prediction and training can be
done by running queries.
● Better support continuous model
training over evolving data
● Support distributed model training

Distributed Model Training in Graph Database
ŷ(2)
ŷ(1)
a(1,3)
a (1,4)
a(1,2)
a(2,5)
x(1)
x(2)
x(3)
x(4)
x(5)
W(0)
, W(1)
● Each vertex collects the features
from its neighbors and combines
them with its own feature to form
z(1)

ŷ(2)
ŷ(1)
a(1,3)
a (1,4)
a(1,2)
a(2,5)
z(1)
z(2)
z(3)
z(4)
z(5)
W(0)
, W(1)
● Each vertex collects the features
from its neighbors and combines
them with its own feature to form
z(1)
● Propagate z(1)
through W(0)
to the
hidden layer

ŷ(2)
ŷ(1)
a(1,3)
a (1,4)
a(1,2)
a(2,5)
𝜎
(1)
=ReLU(z(1)
W(0)
)
𝜎(2)
𝜎(3)
𝜎(4)
𝜎(5)
W(0)
, W(1)
● Compute the activation on the
hidden layer, 𝜎, using ReLU
function

ŷ(2)
ŷ(1)
a(1,3)
a (1,4)
a(1,2)
a(2,5)
𝜎(1)
𝜎(2)
𝜎(3)
𝜎(4)
𝜎(5)
W(0)
, W(1)
function
● Repeat the first two steps to
compute the output layer

ŷ(2)
ŷ(1)
a(1,3)
a (1,4)
a(1,2)
a(2,5)
y(1)
y(2)
y(3)
y(4)
y(5)
W(0)
, W(1)
function
● Repeat the first two steps to
compute the output layer
● Compute the prediction using
softmax function

ŷ(2)
ŷ(1)
a(1,3)
a (1,4)
a(1,2)
a(2,5)
𝛿(1)
= ŷ(1)
- y(1)
𝛿(2)
= ŷ(2)
-
y(2)
y(3)
y(4)
y(5)
𝜕J/ 𝜕W(0)
W(0)
=W(0)
- ⍺ ( 𝜕J/ 𝜕W(0)
)
● Aggregate the prediction error
𝛿 and use gradient descent to
update the weight matrix.

Demo (GCN for Node Classiﬁcation)
Data set
•Cora, Citation network (undirected)
•2708 nodes, 5,429 edges, 7 classes
•Sparse bag-of-words feature vectors (dim:1433)
Model
•Y=softmax(AReLU(AXW(0)
) W(1)
)
•One hidden layer with 16 hidden features
•Training: 140, validation: 500, testing: 1000
•Loss: softmax_cross_entropy_with_logits
•Batch gradient descent
•Dropout: 0.5, L2 regularization (5e-4) for first layer
Data: Sen et al., AI magazine (2008)
Model: Thomas N. Kipf and Max Welling, ICLR (2017)

Demo (GCN for Node Classiﬁcation)

Training Graph Convolutional Neural Networks in Graph Database

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to Training Graph Convolutional Neural Networks in Graph Database

Similar to Training Graph Convolutional Neural Networks in Graph Database (20)

More from TigerGraph

More from TigerGraph (20)

Recently uploaded

Recently uploaded (20)

Training Graph Convolutional Neural Networks in Graph Database