Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

SEMAC Graph Node Embeddings for Link Prediction

75 views

Published on

We present a new graph representation learning approach called SEMAC that jointly exploits fine-grained node features as well as the overall graph topology. In contrast to the SGNS or SVD methods espoused in previous representation-based studies, our model represents nodes in terms of subgraph embeddings acquired via a form of convex matrix completion to iteratively reduce the rank, and thereby, more effectively eliminate noise in the representation. Thus, subgraph embeddings and convex matrix completion are elegantly integrated into a novel link prediction framework.

Published in: Technology
  • Be the first to comment

  • Be the first to like this

SEMAC Graph Node Embeddings for Link Prediction

  1. 1. Link Prediction via Subgraph Embedding-Based Convex Matrix Completion Link Prediction via Subgraph Embedding-Based Convex Matrix Completion Zhu Cao Tsinghua University Linlin Wang Tsinghua University Gerard de Melo http://gerard.demelo.org Rutgers University
  2. 2. Social NetworksSocial Networks Image: CC-BY-SA by Tanja Cappell. https://www.flickr.com/photos/frauhoelle/8464661409
  3. 3. Link PredictionLink Prediction
  4. 4. Link PredictionLink Prediction ?
  5. 5. Traditional Neighbour Overlap Methods Traditional Neighbour Overlap Methods ? Examples: ● Common Neighbours ● Adamic/Adar (frequency weighted) ● Jaccard coefficient based on neighbours
  6. 6. Traditional Neighbour Overlap Methods Traditional Neighbour Overlap Methods ? Examples: ● Common Neighbours ● Adamic/Adar (frequency weighted) ● Jaccard coefficient based on neighbours
  7. 7. Previous Work: Connectivity-Based Methods Previous Work: Connectivity-Based Methods ? Examples: ● PageRank ● SimRank ● Hitting Time ● Commute Time (roundtrip) ● Katz
  8. 8. Learning RepresentationsLearning Representations Image: Adapted from Perozzi et al. 2014. DeepWalk: Online Learning of Social Representations
  9. 9. SVDSVD E.g. Graph's Adjacency Matrix Image: Adapted from Tim Roughgarden & Gregory Valiant
  10. 10. SVDSVD Left Singular Vectors Right Singular Vectors Singular Values E.g. Graph's Adjacency Matrix Image: Adapted from Tim Roughgarden & Gregory Valiant
  11. 11. Low-Rank Approximation via SVD Low-Rank Approximation via SVD ≈ k Left Singular Vectors k Right Singular Vectors k Singular ValuesE.g. Graph's Adjacency Matrix Image: Adapted from Tim Roughgarden & Gregory Valiant k k k
  12. 12. Word Vector Representations: word2vec Word Vector Representations: word2vec 0.02 0.12 0.04 ... 0.03 0.08 Prediction … the local population of sparrows ... Large-Scale Text sparrows population local ... word2vec Skip-Gram Model
  13. 13. Previous Work: DeepWalk Previous Work: DeepWalk 0.02 0.12 0.04 ... 0.03 0.08 Prediction … n15 n382 n49 n729 n23 ... Social Network n49 n729 n382 ... word2vec Skip-Gram Model Random Walk Perozzi et al. 2014. DeepWalk: Online Learning of Social Representations
  14. 14. Previous Work: node2vec Previous Work: node2vec Grover & Leskovec (2016). node2vec: Scalable Feature Learning for Networks. Biased Random Walks
  15. 15. Our Approach: SEMAC Our Approach: SEMAC Subgraph Embedding + Matrix Completion
  16. 16. Our Approach: SEMAC Our Approach: SEMAC
  17. 17. Our Approach: SEMAC Our Approach: SEMAC Step 1: Run Breadth-First Search for different depths d from each node v Step 1: Run Breadth-First Search for different depths d from each node v
  18. 18. Our Approach: SEMAC Our Approach: SEMAC
  19. 19. Our Approach: SEMAC Our Approach: SEMAC
  20. 20. Our Approach: SEMAC Our Approach: SEMAC
  21. 21. Our Approach: SEMAC Our Approach: SEMAC
  22. 22. Our Approach: SEMAC Our Approach: SEMAC Step 2: Create a new graph G' with edges between subgraphs Step 2: Create a new graph G' with edges between subgraphs
  23. 23. Our Approach: SEMAC Our Approach: SEMAC Step 2: Create a new graph G' with edges between subgraphs Step 2: Create a new graph G' with edges between subgraphs Edges: a) same node, depth ± 1 b) same depth neighbour node
  24. 24. Our Approach: SEMAC Our Approach: SEMAC Edges: a) same node, depth ± 1 b) same depth, neighbour node Step 2: Create a new graph G' with edges between subgraphs Step 2: Create a new graph G' with edges between subgraphs
  25. 25. Our Approach: SEMAC Our Approach: SEMAC Edges: a) same node, depth ± 1 b) same depth, neighbour node Step 2: Create a new graph G' with edges between subgraphs Step 2: Create a new graph G' with edges between subgraphs
  26. 26. Our Approach: SEMAC Our Approach: SEMAC Step 3: Learn subgraph embeddings using G' Step 3: Learn subgraph embeddings using G'
  27. 27. Our Approach: SEMAC Our Approach: SEMAC Step 3: Learn subgraph embeddings using G' Step 3: Learn subgraph embeddings using G' Nuclear Norm Nuclear Norm Minimization Find W that minimizes
  28. 28. Our Approach: SEMAC Our Approach: SEMAC Step 3: Learn subgraph embeddings using G' Step 3: Learn subgraph embeddings using G' Nuclear Norm Nuclear Norm Minimization Find W that minimizes Frobenius NormCompare only non-zero (observed) entries (unlike SVD)
  29. 29. Our Approach: SEMAC Our Approach: SEMAC Result of Step 3: Embedding for every subgraph Result of Step 3: Embedding for every subgraph 0.32 ... 0.27 0.81 ... 0.12
  30. 30. Our Approach: SEMAC Our Approach: SEMAC Step 4: Create node embeddings Concatenate embeddings of subgraphs for different depths d Step 4: Create node embeddings Concatenate embeddings of subgraphs for different depths d 0.32 ... 0.27 0.81 ... 0.12 ... 0.32 ... 0.27 0.81 ... 0.12
  31. 31. Link PredictionLink Prediction ? 0.32 0.14 0.03 ... 0.18 0.09 0.28 0.11 0.08 ... 0.24 0.13 Step 5: Link prediction via Vector Cosine Step 5: Link prediction via Vector Cosine
  32. 32. ExperimentsExperiments Image: CC-BY by Marc Smith with NodeXL. https://www.flickr.com/photos/marc_smith/6871711979
  33. 33. ExperimentsExperiments Facebook (McAuley & Leskovec 2012)
  34. 34. ExperimentsExperiments Facebook (McAuley & Leskovec 2012) Small connected component subsets of Facebook
  35. 35. ExperimentsExperiments Wikipedia Coauthorship (Leskovec & Krevl 2014)
  36. 36. ExperimentsExperiments Wikipedia Coauthorship Protein-Protein Interactions (Breitkreutz et al. 2008) (Leskovec & Krevl 2014)
  37. 37. ExperimentsExperiments AUC based on 5-fold Cross-Validation
  38. 38. SummarySummarySummarySummary Goal: State-of-the-Art Link Prediction Consider Subgraphs ► Different depths ► Graph of subgraphs with links to related subgraphs Create Representations ► Nuclear-Norm Minimization to better account for unobserved links ► Concatenate, Cosine Get in Touch! http://gerard.demelo.org gdm@demelo.org Get in Touch! http://gerard.demelo.org gdm@demelo.org Thank you! 0.32 0.14 ... 0.09 0.28 0.11 ... 0.13
  39. 39. AcknowledgmentsAcknowledgmentsAcknowledgmentsAcknowledgments User Icons by Freepik (CC-BY) https://www.freepik.com Title Image: CC-BY by Chris Potter https://www.flickr.com/photos/865 Thank you! 0.32 0.14 ... 0.09 0.28 0.11 ... 0.13

×