The document discusses a novel approach to link prediction through subgraph embedding and convex matrix completion, referred to as SEMAC. This method includes creating a new graph based on breadth-first search, learning subgraph embeddings, and using nuclear norm minimization for better accuracy in predicting connections. Results from various experiments on social networks, including Facebook and Wikipedia, indicate that SEMAC achieves state-of-the-art performance in link prediction tasks.

1. Link Prediction via
Subgraph Embedding-Based
Convex Matrix Completion
Link Prediction via
Subgraph Embedding-Based
Convex Matrix Completion
Zhu Cao
Tsinghua University
Linlin Wang
Tsinghua University
Gerard de Melo
http://gerard.demelo.org
Rutgers University

2. Social NetworksSocial Networks
Image: CC-BY-SA by Tanja Cappell. https://www.flickr.com/photos/frauhoelle/8464661409

3. Link PredictionLink Prediction

4. Link PredictionLink Prediction
?

5. Traditional
Neighbour Overlap Methods
Traditional
Neighbour Overlap Methods
?
Examples:
● Common Neighbours
● Adamic/Adar
(frequency weighted)
● Jaccard coefficient
based on neighbours

6. Traditional
Neighbour Overlap Methods
Traditional
Neighbour Overlap Methods
?
Examples:
● Common Neighbours
● Adamic/Adar
(frequency weighted)
● Jaccard coefficient
based on neighbours

7. Previous Work:
Connectivity-Based Methods
Previous Work:
Connectivity-Based Methods
?
Examples:
● PageRank
● SimRank
● Hitting Time
● Commute Time
(roundtrip)
● Katz

8. Learning RepresentationsLearning Representations
Image: Adapted from Perozzi et al. 2014. DeepWalk: Online Learning of Social Representations

9. SVDSVD
E.g. Graph's
Adjacency Matrix
Image: Adapted from Tim Roughgarden & Gregory Valiant

10. SVDSVD
Left Singular Vectors Right Singular Vectors
Singular Values
E.g. Graph's
Adjacency Matrix
Image: Adapted from Tim Roughgarden & Gregory Valiant

11. Low-Rank Approximation
via SVD
Low-Rank Approximation
via SVD
≈
k
Left Singular Vectors
k
Right Singular Vectors
k Singular
ValuesE.g. Graph's
Adjacency Matrix
Image: Adapted from Tim Roughgarden & Gregory Valiant
k k k

12. Word Vector Representations:
word2vec
Word Vector Representations:
word2vec
0.02
0.12
0.04
...
0.03
0.08
Prediction
…
the
local
population
of
sparrows
...
Large-Scale
Text
sparrows
population
local
...
word2vec Skip-Gram Model

13. Previous Work:
DeepWalk
Previous Work:
DeepWalk
0.02
0.12
0.04
...
0.03
0.08
Prediction
…
n15
n382
n49
n729
n23
...
Social
Network
n49
n729
n382
...
word2vec Skip-Gram Model
Random Walk
Perozzi et al. 2014. DeepWalk: Online Learning of Social Representations

14. Previous Work:
node2vec
Previous Work:
node2vec
Grover & Leskovec (2016). node2vec: Scalable Feature Learning for Networks.
Biased Random Walks

15. Our Approach:
SEMAC
Our Approach:
SEMAC
Subgraph
Embedding
+
Matrix Completion

16. Our Approach:
SEMAC
Our Approach:
SEMAC

17. Our Approach:
SEMAC
Our Approach:
SEMAC
Step 1:
Run Breadth-First Search for different depths d
from each node v
Step 1:
Run Breadth-First Search for different depths d
from each node v

18. Our Approach:
SEMAC
Our Approach:
SEMAC

19. Our Approach:
SEMAC
Our Approach:
SEMAC

20. Our Approach:
SEMAC
Our Approach:
SEMAC

21. Our Approach:
SEMAC
Our Approach:
SEMAC

22. Our Approach:
SEMAC
Our Approach:
SEMAC
Step 2:
Create a new graph G'
with edges between
subgraphs
Step 2:
Create a new graph G'
with edges between
subgraphs

23. Our Approach:
SEMAC
Our Approach:
SEMAC
Step 2:
Create a new graph G'
with edges between
subgraphs
Step 2:
Create a new graph G'
with edges between
subgraphs
Edges:
a) same node,
depth ± 1
b) same depth
neighbour node

24. Our Approach:
SEMAC
Our Approach:
SEMAC
Edges:
a) same node,
depth ± 1
b) same depth,
neighbour node
Step 2:
Create a new graph G'
with edges between
subgraphs
Step 2:
Create a new graph G'
with edges between
subgraphs

25. Our Approach:
SEMAC
Our Approach:
SEMAC
Edges:
a) same node,
depth ± 1
b) same depth,
neighbour node
Step 2:
Create a new graph G'
with edges between
subgraphs
Step 2:
Create a new graph G'
with edges between
subgraphs

26. Our Approach:
SEMAC
Our Approach:
SEMAC
Step 3:
Learn subgraph embeddings
using G'
Step 3:
Learn subgraph embeddings
using G'

27. Our Approach:
SEMAC
Our Approach:
SEMAC
Step 3:
Learn subgraph embeddings
using G'
Step 3:
Learn subgraph embeddings
using G'
Nuclear Norm
Nuclear Norm Minimization
Find W that
minimizes

28. Our Approach:
SEMAC
Our Approach:
SEMAC
Step 3:
Learn subgraph embeddings
using G'
Step 3:
Learn subgraph embeddings
using G'
Nuclear Norm
Nuclear Norm Minimization
Find W that
minimizes
Frobenius NormCompare only
non-zero (observed)
entries (unlike SVD)

29. Our Approach:
SEMAC
Our Approach:
SEMAC
Result of Step 3:
Embedding for
every subgraph
Result of Step 3:
Embedding for
every subgraph
0.32
...
0.27
0.81
...
0.12

30. Our Approach:
SEMAC
Our Approach:
SEMAC
Step 4:
Create node embeddings
Concatenate embeddings
of subgraphs for different
depths d
Step 4:
Create node embeddings
Concatenate embeddings
of subgraphs for different
depths d
0.32
...
0.27
0.81
...
0.12
...
0.32
...
0.27
0.81
...
0.12

31. Link PredictionLink Prediction
?
0.32
0.14
0.03
...
0.18
0.09
0.28
0.11
0.08
...
0.24
0.13
Step 5:
Link prediction
via Vector Cosine
Step 5:
Link prediction
via Vector Cosine

32. ExperimentsExperiments
Image: CC-BY by Marc Smith with NodeXL. https://www.flickr.com/photos/marc_smith/6871711979

33. ExperimentsExperiments
Facebook
(McAuley &
Leskovec 2012)

34. ExperimentsExperiments
Facebook
(McAuley &
Leskovec 2012)
Small connected
component
subsets of Facebook

35. ExperimentsExperiments
Wikipedia Coauthorship
(Leskovec & Krevl 2014)

36. ExperimentsExperiments
Wikipedia Coauthorship
Protein-Protein Interactions
(Breitkreutz et al. 2008)
(Leskovec & Krevl 2014)

37. ExperimentsExperiments
AUC based on 5-fold Cross-Validation

38. SummarySummarySummarySummary
Goal: State-of-the-Art
Link Prediction
Consider Subgraphs
► Different depths
► Graph of subgraphs with links
to related subgraphs
Create Representations
► Nuclear-Norm Minimization
to better account for
unobserved links
► Concatenate, Cosine
Get in Touch!
http://gerard.demelo.org
gdm@demelo.org
Get in Touch!
http://gerard.demelo.org
gdm@demelo.org
Thank
you!
0.32
0.14
...
0.09
0.28
0.11
...
0.13

39. AcknowledgmentsAcknowledgmentsAcknowledgmentsAcknowledgments
User Icons by
Freepik (CC-BY)
https://www.freepik.com
Title Image: CC-BY by
Chris Potter
https://www.flickr.com/photos/865
Thank
you!
0.32
0.14
...
0.09
0.28
0.11
...
0.13