Uploaded byssuser3a8f33
PDF, PPTX114 views

Pinsage Stanford slides.pdf

This document summarizes a lecture on graph neural network models for recommendation systems. It discusses three graph neural network models: PinSage for item recommendation on Pinterest, Decagon for heterogeneous graphs, and GCPN for goal-directed generation. PinSage is described in detail, including how it uses graph convolutions to generate embeddings for items, importance sampling to select neighbors, training on positive and hard negative pairs, and its performance gains over baseline methods for related item recommendation.

Embed presentation

Download as PDF, PPTX
CS224W: Analysis of Networks Jure Leskovec, R.Ying and J.You, Stanford University http://cs224w.stanford.edu
Three topics for today: 1. GNN recommendation (PinSage) 2. Heterogeneous GNN (Decagon) 3. Goal-directed generation (GCPN) 12/5/19 Jure Leskovec, Stanford CS224W: Machine Learning with Graphs, http://cs224w.stanford.edu 2
Jure Leskovec, Stanford CS224W: Machine Learning with Graphs, http://cs224w.stanford.edu 4 Items Users ¡ Users interacts with items § Watch movies, buy merchandise, listen to music ¡ Goal: Recommend items users might like § Customer X buys Metallica and Megadeth CDs § Customer Y buys Megadeth, the recommender system suggests Metallica as well 12/5/19 Interactions “You might also like”
Goal: Learn what items are related ¡ For a given query item(s) Q, return a set of similar items that we recommend to the user Idea: ¡ User interacts with a set of items ¡ Formulate a query Q ¡ Search the items and return recommendations 12/5/19 Jure Leskovec, Stanford CS224W: Machine Learning with Graphs, http://cs224w.stanford.edu 5 Items Query Recommendations Products, web sites, movies, posts, ads, …
Query: 12/5/19 Jure Leskovec, Stanford CS224W: Machine Learning with Graphs, http://cs224w.stanford.edu 6
Query: Recommendations: 12/5/19 Jure Leskovec, Stanford CS224W: Machine Learning with Graphs, http://cs224w.stanford.edu 7
Query: 12/5/19 Jure Leskovec, Stanford CS224W: Machine Learning with Graphs, http://cs224w.stanford.edu 8
Query: Recommendations: 12/5/19 Jure Leskovec, Stanford CS224W: Machine Learning with Graphs, http://cs224w.stanford.edu 9
Having a universal similarity function allows for many applications: 12/5/19 Jure Leskovec, Stanford CS224W: Machine Learning with Graphs, http://cs224w.stanford.edu 10 Homefeed (endless feed of recommendations) Related pins (find most similar/related pins) Ads and shopping (use organic for the query and search the ads database)
Question: How do we define similarity? ¡ 1) Content-based: User and item features, in the form of images, text, categories, etc. ¡ 2) Graph-based: User-item interactions, in the form of graph/network structure § This is called collaborative filtering: § For a given user X, find others who liked similar items § Estimate what X will like based on what similar others like 12/5/19 Jure Leskovec, Stanford CS224W: Machine Learning with Graphs, http://cs224w.stanford.edu 11
How do we define similarity: ¡ (1) Gathering “known” similarities § How to collect the data about what users like ¡ (2) Extrapolating unknown similarities from the known ones § Mainly interested in high unknown similarities § We are not interested in knowing what you don’t like but what you like ¡ (3) Evaluating methods § How to measure success/performance of recommendation methods 12/5/19 12 Jure Leskovec, Stanford CS224W: Machine Learning with Graphs, http://cs224w.stanford.edu
¡ 300M users ¡ 4+B pins, 2+B boards Jure Leskovec, Stanford CS224W: Machine Learning with Graphs, http://cs224w.stanford.edu 13 12/5/19
12/5/19 Jure Leskovec, Stanford CS224W: Machine Learning with Graphs, http://cs224w.stanford.edu 14 Pinterest: Human curated collection of pins Pin:Avisualbookmarksomeonehas savedfromtheinternettoaboard they’vecreated. Pin:Image,text,link Board: A collection of ideas (pins having something in common)
Two sources of signal: Features: ¡ Image and text of each pin Graph: ¡ Graph is dynamic: Need to apply to new nodes without model retraining 12/5/19 Jure Leskovec, Stanford CS224W: Machine Learning with Graphs, http://cs224w.stanford.edu 15 Q
Goal: Learn embeddings for items ¡ Related Pins Query: Which pin to recommend when a user interacts with a pin 𝑣"? ¡ Answer: Find the closest embedding (𝑣#) to 𝑣" by nearest neighbor. Recommend it. Jure Leskovec, Stanford CS224W: Machine Learning with Graphs, http://cs224w.stanford.edu 16 𝑣$ 𝑣% 𝑣" 𝑣# 12/5/19 Item embeddings Previously pinned Query pin Related pin recommendation
¡ Goal 1: Efficiently learn embeddings for billions of pins (items, nodes) using neural networks ¡ Goal 2: Perform nearest neighbor query to recommend items in real-time 12/5/19 Jure Leskovec, Stanford CS224W: Machine Learning with Graphs, http://cs224w.stanford.edu 17 Query pin “Predicted” related pin Embed Embedding space The closer the embeddings are, the more similar the pins are
Task: Recommend related pins to users Query pin 8 Predict whether two nodes in a graph are related Task: Learn node embeddings 𝑧' such that 𝑑 𝑧)*+,$, 𝑧)*+,% < 𝑑(𝑧)*+,$, 𝑧01,*2,3) Jure Leskovec, Stanford CS224W: Machine Learning with Graphs, http://cs224w.stanford.edu 𝑧$ 𝑧% 𝑑(𝑧$, 𝑧%) 12/5/19
Approach: ¡ Pins have embeddings at each layer ¡ Layer-0 embedding of a node are its features: § Text, image, … pin board ... Aggregator ... ... ... Agg. Agg. Agg. Predict whether two nodes in a graph are related 12/5/19 Jure Leskovec, Stanford CS224W: Machine Learning with Graphs, http://cs224w.stanford.edu 19
¡ PinSage graph convolutional network: § Goal: Generate embeddings for nodes (e.g., pins) in the Pinterest graph containing billions of objects § Key Idea: Borrow information from nearby nodes § E.g., bed rail Pin might look like a garden fence, but gates and beds are rarely adjacent in the graph § Pin embeddings are essential to many different tasks. Aside from the “Related Pins” task, it can also be used in: § Recommend related ads § Homefeed recommendation § Cluster users by their interest 12/5/19 Jure Leskovec, Stanford CS224W: Machine Learning with Graphs, http://cs224w.stanford.edu 20 [Ying et al.,WWW 2018] sum - multiset > mean - distribution max - set > Input Figure 2: Ranking by expressive power for sum, mean and max-pooling aggregators over a multiset. Left panel shows the input multiset and the three panels illustrate the aspects of the multiset a given aggregator is able to capture: sum captures the full multiset, mean captures the proportion/distribution of elements of a given type, and the max aggregator ignores multiplicities (reduces the multiset to a simple set). vs. (a) Mean and Max both fail vs. (b) Max fails vs. (c) Mean and Max both fail Figure 3: Examples of simple graph structures that mean and max-pooling aggregators fail to distinguish. Figure 2 gives reasoning about how different aggregators “compress” different graph structures/multisets. existing GNNs instead use a 1-layer perceptron W (Duvenaud et al., 2015; Kipf & Welling, 2017; Zhang et al., 2018), a linear mapping followed by a non-linear activation function such as a ReLU. A
1. Collect billions of training pairs from logs. § Positive pair: Two pins that are consecutively saved into the same board within a time interval (1 hour) § Negative pair: A random pair of 2 pins § With high probability the pins are not on the same board 12/5/19 Jure Leskovec, Stanford CS224W: Machine Learning with Graphs, http://cs224w.stanford.edu 21
1. Collect billions of training pairs from logs. § Positive pair: Two pins that are consecutively saved into the same board within a time interval (1 hour) § Negative pair: A random pair of 2 pins § With high probability the pins are not on the same board 2. Train GNN to generate similar embeddings for training pairs 3. Inference: Generate embeddings for all pins 4. Nearest neighbor search in embedding space to make recommendations. 12/5/19 Jure Leskovec, Stanford CS224W: Machine Learning with Graphs, http://cs224w.stanford.edu 22
¡ Train so that pins that are consecutively pinned have similar embeddings ¡ Max-margin loss: 12/5/19 Jure Leskovec, Stanford CS224W: Machine Learning with Graphs, http://cs224w.stanford.edu 23 L = X (u,v)2D max(0, z> u zv + z> u zn + ) set of training pairs from user logs “positive”/true training pair “negative” example “margin” (i.e., how much larger positive pair similarity should be compared to negative)
¡ Four key innovations: 1. On-the-fly graph convolutions § Sample the neighborhood around a node and dynamically construct a computation graph 12/5/19 Jure Leskovec, Stanford CS224W: Machine Learning with Graphs, http://cs224w.stanford.edu 24 Minibatch of neighborhoods
¡ Four key innovations: 1. On-the-fly graph convolutions § Perform a localized graph convolution around a particular node § Does not need the entire graph during training 12/5/19 Jure Leskovec, Stanford CS224W: Machine Learning with Graphs, http://cs224w.stanford.edu 25 At every iteration, only source node embeddings are computed
¡ Four key innovations: 2. Selecting neighbors via random walks § Performing aggregation on all neighbors is infeasible: § How to select the set of neighbors of a node to convolve over? § Personalized PageRank can help! § Define Importance pooling: Define importance-based neighborhoods by simulating random walks and selecting the neighbors with the highest visit counts 12/5/19 Jure Leskovec, Stanford CS224W: Machine Learning with Graphs, http://cs224w.stanford.edu 26
¡ Proximity to query node(s) Q PixieRandomWalk 5 5 5 5 5 5 14 9 16 7 8 8 8 8 1 1 1 Strawberries Smoothies Yummm Smoothie Madness!•! Q Jure Leskovec, Stanford CS224W: Machine Learning with Graphs, http://cs224w.stanford.edu 27 12/5/19
¡ Proximity to query node(s) Q ¡ Importance pooling § Choose nodes with top K visit counts § Pool over the chosen nodes § The chosen nodes are not necessarily neighbors PixieRandomWalk 5 5 5 5 5 5 14 9 16 7 8 8 8 8 1 1 1 Strawberries Smoothies Yummm Smoothie Madness!•!• Q Jure Leskovec, Stanford CS224W: Machine Learning with Graphs, http://cs224w.stanford.edu 28 12/5/19
¡ Example: suppose 𝐾=5 ¡ Rank nodes based on Random Walk visit counts ¡ Pick top 𝑲 nodes and normalize counts 16 55 , 14 55 , 9 55 , 8 55 , 8 55 ¡ Aggregate messages from the top 𝐾 nodes Jure Leskovec, Stanford CS224W: Machine Learning with Graphs, http://cs224w.stanford.edu 29 PixieRandomWalk 5 5 5 5 5 5 14 9 16 7 8 8 8 8 1 1 1 Strawberries Smoothies Yummm Smoothie Madness!•!• Q Top 𝑲 nodes 12/5/19
¡ Pick top K nodes and normalize counts 16 55 , 14 55 , 9 55 , 8 55 , 8 55 ¡ GraphSAGE mean pooling § Average the messages from direct neighbors ¡ PinSAGE Importance pooling § Use the normalized counts as weights for weighted mean of messages from the top K nodes ¡ PinSAGE uses 𝐾 = 50 § Negligible performance gain for 𝐾 > 50 Jure Leskovec, Stanford CS224W: Machine Learning with Graphs, http://cs224w.stanford.edu 30 12/5/19
Four key innovations: 3. Efficient MapReduce inference § Problem: Many repeated computation if using localized graph convolution at inference step § Need to avoid repeated computation 12/5/19 Jure Leskovec, Stanford CS224W: Machine Learning with Graphs, http://cs224w.stanford.edu 31 Repeated computation
¡ Recall how we obtain negative examples Jure Leskovec, Stanford CS224W: Machine Learning with Graphs, http://cs224w.stanford.edu 33 L = X (u,v)2D max(0, z> u zv + z> u zn + ) set of training pairs from logs “positive”/true example “negative” example “margin” (i.e., how much larger positive pair similarity should be compared to negative) 12/5/19
Goal: Identify target pin among 3B pins ¡ Issue: Need to learn with resolution of 100 vs. 3B ¡ Massive size: 3 billion nodes, 20 billion edges ¡ Idea: Use harder and harder negative samples Jure Leskovec, Stanford CS224W: Machine Learning with Graphs, http://cs224w.stanford.edu 34 L = X (u,v)2D max(0, z> u zv + z> u zn + ) “positive”/true example negative examples “margin” (i.e., how much larger positive pair similarity should be compared to negative) set of training pairs from logs Force model to learn subtle distinctions between pins 12/5/19
¡ Hard negative examples improve performance ¡ How to obtain hard negatives: Use random walks: § Use nodes with visit counts ranked at 1000-5000 as hard negatives § Have something in common, but are not too similar 12/5/19 Jure Leskovec, Stanford CS224W: Machine Learning with Graphs, http://cs224w.stanford.edu 35 Harder to distinguish from the positive pair Positive pair
¡ Hard negative examples improve performance ¡ Curriculum training on hard negatives § Start with random negative examples § Provide harder negative examples over time 12/5/19 Jure Leskovec, Stanford CS224W: Machine Learning with Graphs, http://cs224w.stanford.edu 36 Harder to distinguish from the positive pair Positive pair
Related Pin recommendations ¡ Given a user just saved pin Q, predict what pin X are they going to save next ¡ Setup: Embed 3B pins, find nearest neighbors of Q ¡ Baseline embeddings: § Visual: VGG visual embeddings Annotation: Word2vec embeddings § Combined: Concatenate embeddings 12/5/19 Jure Leskovec, Stanford CS224W: Machine Learning with Graphs, http://cs224w.stanford.edu 37 MRR: Mean reciprocal rank of the positive example X w.r.t Q Hit rate: Fraction of times the positive example X is among top K closest to Q
Jure Leskovec, Stanford CS224W: Machine Learning with Graphs, http://cs224w.stanford.edu 12/5/19 38 Pixie (graph-based): the method of simulating random walks starting at query Pin using the Pixie algorithm in class. Items with top scores are retrieved as recommendations Visual, Annot. (feature-based): nearest neighbor recommendation using visual (CNN) and annotation features of pins
Pixie Graph- SAGE Query PinSAGE 12/5/19 Jure Leskovec, Stanford CS224W: Machine Learning with Graphs, http://cs224w.stanford.edu 39
Pixie Graph- SAGE Query PinSAGE 12/5/19 Jure Leskovec, Stanford CS224W: Machine Learning with Graphs, http://cs224w.stanford.edu 40
1. GNN recommendation (PinSage) 2. Heterogeneous GNN (Decagon) 3. Goal-directed generation (GCPN) 12/5/19 Jure Leskovec, Stanford CS224W: Machine Learning with Graphs, http://cs224w.stanford.edu 41
¡ So far we only applied GNNs to simple graphs § GNNs do not explicitly use node and edge type information ¡ Real networks are often heterogeneous ¡ How to use GNN for heterogeneous graphs? Jure Leskovec, Stanford CS224W: Machine Learning with Graphs, http://cs224w.stanford.edu 43 12/5/19
12/5/19 Jure Leskovec, Stanford CS224W: Machine Learning with Graphs, http://cs224w.stanford.edu 44 , Patient’s side effects Patient’s medications Polypharmacy side effect Drug combination Polypharmacy: use multiple drugs for a disease
¡ Polypharmacy is common to treat complex diseases and co-existing conditions ¡ High risk of side effects due to interactions ¡ 15% of the U.S. population affected ¡ Annual costs exceed $177 billion ¡ Difficult to identify manually: § Rare, occur only in a subset of patients § Not observed in clinical testing 12/5/19 Jure Leskovec, Stanford CS224W: Machine Learning with Graphs, http://cs224w.stanford.edu 45
¡ Systematic experimental screening of drug interactions is challenging ¡ Idea: Computationally screen/predict polypharmacy side effects § Use molecular, pharmacological and patient population data § Guide translational strategies for combination treatments in patients 12/5/19 Jure Leskovec, Stanford CS224W: Machine Learning with Graphs, http://cs224w.stanford.edu 46
, . . . . . . How likely with a pair of drugs 𝑐, 𝑑 lead to side effect 𝑟? Model and predict side effects of drug pairs 𝑐 𝑑 𝑟 Jure Leskovec, Stanford CS224W: Machine Learning with Graphs, http://cs224w.stanford.edu 12/5/19 47
¡ Heterogeneous (multimodal) graphs: graphs with different node types and/or edge types 12/5/19 Jure Leskovec, Stanford CS224W: Machine Learning with Graphs, http://cs224w.stanford.edu 48 2 node types edge types
Goal: Given a partially observed graph, predict labeled edges between drug nodes Ciprofloxacin r1 r2 Simvastatin Mupirocin r2 Doxycycline S C M D Query: Given a drug pair 𝑐, 𝑑, how likely does an edge (𝑐, 𝑟%, 𝑑) exist? Co-prescribed drugs 𝑐 and 𝑑 lead to side effect 𝑟% Jure Leskovec, Stanford CS224W: Machine Learning with Graphs, http://cs224w.stanford.edu 12/5/19 49
¡ Predict labeled edges between drugs nodes § i.e., predict the likelihood that an edge (𝑐, 𝑟%, 𝑠) exists between drug nodes 𝑐 and 𝑠 § Meaning: Drug combination (𝑐, 𝑠) leads to polypharmacy side effect 𝑟% 12/5/19 Jure Leskovec, Stanford CS224W: Machine Learning with Graphs, http://cs224w.stanford.edu 50 Predictions:
¡ Key Insight: Compute GNN messages from each edge type, then aggregate across different edge types 12/5/19 Jure Leskovec, Stanford CS224W: Machine Learning with Graphs, http://cs224w.stanford.edu 51 § Input: heterogenous graph § Output: node embeddings One layer of Heterogeneous GNN GNN for Edge type: 𝒓𝟐 GNN for Edge type: drug-target Sum GNN for Edge type: 𝒓𝟏
12/5/19 Jure Leskovec, Stanford CS224W: Machine Learning with Graphs, http://cs224w.stanford.edu 52 § Input: Node embeddings of query drug pairs § Output: predicted edges ¡ Key Insight: Use pair of computed node embeddings to make edge predictions Predict possible edges with NN Neural Network
v v v p – probability Jure Leskovec, Stanford CS224W: Machine Learning with Graphs, http://cs224w.stanford.edu 12/5/19 53
¡ Data: § Graph over Molecules: protein-protein interaction and drug target relationships § Graph over Population: Side effects of individual drugs, polypharmacy side effects of drug combinations ¡ Setup: § Construct a heterogeneous graph of all the data § Train: Fit a model to predict known associations of drug pairs and polypharmacy side effects § Test: Given a query drug pair, predict candidate polypharmacy side effects 12/5/19 Jure Leskovec, Stanford CS224W: Machine Learning with Graphs, http://cs224w.stanford.edu 54
¡ Up to 54% improvement over baselines ¡ First opportunity to computationally flag polypharmacy side effects for follow-up analyses 12/5/19 Jure Leskovec, Stanford CS224W: Machine Learning with Graphs, http://cs224w.stanford.edu 55 AUROC AUPRC AP@50 Decagon (3-layer) 0.834 0.776 0.731 Decagon (2-layer) 0.809 0.762 0.713 RESCAL 0.693 0.613 0.476 Node2vec 0.725 0.708 0.643 Drug features 0.736 0.722 0.679
Drug c Drug d Jure Leskovec, Stanford CS224W: Machine Learning with Graphs, http://cs224w.stanford.edu 12/5/19 56
Evidence found Drug c Drug d Jure Leskovec, Stanford CS224W: Machine Learning with Graphs, http://cs224w.stanford.edu 12/5/19 57
1. GNN recommendation (PinSage) 2. Heterogeneous GNN (Decagon) 3. Goal-directed generation (GCPN) 12/5/19 Jure Leskovec, Stanford CS224W: Machine Learning with Graphs, http://cs224w.stanford.edu 58
¡ Given: Graphs sampled from 𝑝G*2*(𝐺) ¡ Goal: §Learn the distribution 𝑝IJG,K(𝐺) §Sample from 𝑝IJG,K(𝐺) 12/5/19 Jure Leskovec, Stanford CS224W: Machine Learning with Graphs, http://cs224w.stanford.edu 60 𝑝G*2*(𝐺) 𝑝IJG,K(𝐺) Learn & Sample
Generating graphs via sequentially adding nodes and edges 12/5/19 Jure Leskovec, Stanford CS224W: Machine Learning with Graphs, http://cs224w.stanford.edu 61 [You et al., ICML 2018] 1 1 2 1 2 3 1 2 4 3 1 2 4 3 5 1 2 4 3 5 Graph 𝐺 Generation process 𝑆M
Quick Summary of GraphRNN: § Generate a graph by generating a two level sequence § Use RNN to generate the sequences 12/5/19 Jure Leskovec, Stanford CS224W: Machine Learning with Graphs, http://cs224w.stanford.edu 62 0 1 1 0 0 1 0 0 1 0 1 0 0 1 1 0 1 1 0 1 0 0 1 1 0 1 2 4 3 5 Graph 𝐺 Node-level RNN Edge-level RNN Adjacency matrix ⇔
12/5/19 Jure Leskovec, Stanford CS224W: Machine Learning with Graphs, http://cs224w.stanford.edu 63
12/5/19 Jure Leskovec, Stanford CS224W: Machine Learning with Graphs, http://cs224w.stanford.edu 64 Can we do more than imitating given graphs?
Question: Can we learn a model that can generate valid and realistic molecules with high value of a given chemical property? 12/5/19 Jure Leskovec, Stanford CS224W: Machine Learning with Graphs, http://cs224w.stanford.edu 65 Model Property output that optimizes e.g., drug_likeness=0.95 [You et al., NeurIPS 2018] Graph Convolutional Policy Network for Goal-Directed Molecular Graph Generation. J. You, B. Liu, R. Ying, V. Pande, J. Leskovec. Neural Information Processing Systems (NeurIPS), 2018.
12/5/19 Jure Leskovec, Stanford CS224W: Machine Learning with Graphs, http://cs224w.stanford.edu 66 § Node types: C, N, O, … § Edge types: single bond, double bond, … § Note: “H”s can be automatically inferred via chemical validity rules, thus are ignored in molecular graphs C N C C N C C Nodes Edges N
Generating graphs that: ¡ Optimize a given objective (High scores) § e.g., drug-likeness ¡ Obey underlying rules (Valid) § e.g., chemical validity rules ¡ Are learned from examples (Realistic) § e.g., Imitating a molecule graph dataset 12/5/19 Jure Leskovec, Stanford CS224W: Machine Learning with Graphs, http://cs224w.stanford.edu 67 Graph Convolutional Policy Network for Goal-Directed Molecular Graph Generation. J. You, B. Liu, R. Ying, V. Pande, J. Leskovec. Neural Information Processing Systems (NeurIPS), 2018.
Generating graphs that: ¡ Optimize a given objective (High scores) § e.g., drug-likeness ¡ Obey underlying rules (Valid) § e.g., chemical validity rules 12/5/19 Jure Leskovec, Stanford CS224W: Machine Learning with Graphs, http://cs224w.stanford.edu 68 Graph Convolutional Policy Network for Goal-Directed Molecular Graph Generation. J. You, B. Liu, R. Ying, V. Pande, J. Leskovec. Neural Information Processing Systems (NeurIPS), 2018. Including “Black-box” in ML: Objectives like drug-likeness are governed by physical law, which are assumed to be unknown to us!
¡ A ML agent observes the environment, takes an action to interact with the environment, and receives positive or negative reward ¡ The agent then learns from this loop ¡ Key: Environment is a blackbox to the agent 12/5/19 Jure Leskovec, Stanford CS224W: Machine Learning with Graphs, http://cs224w.stanford.edu 69 ML Agent Action Environment Observation, Reward
¡ Policy: Agent behavior, which maps observation to action ¡ Policy-based RL: An agent directly learns an optimal policy from data 12/5/19 Jure Leskovec, Stanford CS224W: Machine Learning with Graphs, http://cs224w.stanford.edu 70 Agent Policy Action Environment Observation, Reward
Graph Convolutional Policy Network combines graph representation + RL: ¡ Graph Neural Network captures complex structural information, and enables validity check in each state transition (Valid) ¡ Reinforcement learning optimizes intermediate/final rewards (High scores) ¡ Adversarial training imitates examples in given datasets (Realistic) 12/5/19 Jure Leskovec, Stanford CS224W: Machine Learning with Graphs, http://cs224w.stanford.edu 71
¡ (a) Insert nodes/scaffolds ¡ (b) Compute state via GCN ¡ (c) Sample next action ¡ (d) Take action (check chemical validity) ¡ (e, f) Compute reward Jure Leskovec, Stanford CS224W: Machine Learning with Graphs, http://cs224w.stanford.edu 72 12/5/19
¡ Learn to take valid action § At each step, assign small positive reward for valid action ¡ Optimize desired properties § At the end, assign positive reward for high desired property ¡ Generate realistic graphs § At the end, adversarially train a GCN discriminator, compute adversarial rewards that encourage realistic molecule graphs Jure Leskovec, Stanford CS224W: Machine Learning with Graphs, http://cs224w.stanford.edu 73 12/5/19
Reward: 𝑟2 = Final reward + Step reward ¡ Final reward = Domain-specific reward ¡ Step rewards = Step-wise validity reward 12/5/19 Jure Leskovec, Stanford CS224W: Machine Learning with Graphs, http://cs224w.stanford.edu 74
¡ Two parts: ¡ (1) Supervised training: Train policy by imitating the action given by real observed graphs. Use gradient. ¡ (2) RL training: Train policy to optimize rewards. Use standard policy gradient algorithm (refer to any RL course, e.g., CS234). 12/5/19 Jure Leskovec, Stanford CS224W: Machine Learning with Graphs, http://cs224w.stanford.edu 75
12/5/19 Jure Leskovec, Stanford CS224W: Machine Learning with Graphs, http://cs224w.stanford.edu 76 GCPN C N N C C C 0.1 1 Step reward Final reward Environment Run until stop Run one step 0.6 Cross entropy loss Graph !" Generated graph !"#$ Generated graph !% ∗ Real graph !"#$ ∗ Policy gradient C N C C C C C C C C C Query dataset Gradient Supervised Training RL Training
Jure Leskovec, Stanford CS224W: Machine Learning with Graphs, http://cs224w.stanford.edu 77 Validity Score Realistic 12/5/19
¡ Property optimization § Generate molecules with high specified property score ¡ Property targeting § Generate molecules whose specified property score falls within given range ¡ Constrained property optimization § Edit a given molecule for a few steps to achieve higher specified property score Jure Leskovec, Stanford CS224W: Machine Learning with Graphs, http://cs224w.stanford.edu 78 12/5/19
¡ ZINC250k dataset § 250,000 drug like molecules whose maximum atom number is 38 ¡ Baselines: § ORGAN: String representation + RL [Guimaraes et al., 2017] § JT-VAE: VAE-based vector representation + Bayesian optimization [Jin et al., 2018] Jure Leskovec, Stanford CS224W: Machine Learning with Graphs, http://cs224w.stanford.edu 79 12/5/19
Property optimization ¡ +60% higher property scores Jure Leskovec, Stanford CS224W: Machine Learning with Graphs, http://cs224w.stanford.edu 80 logP: octanol-water partition coef., indicates solubility QED: indicator of drug-likeness 12/5/19
Property targeting ¡ 7x higher success rate than JT-VAE, 10% less diversity Jure Leskovec, Stanford CS224W: Machine Learning with Graphs, http://cs224w.stanford.edu 81 logP: octanol-water partition coef., indicates solubility MW: molecular weight an indicator of drug-likeness Diversity: avg. pairwise Tanimoto distance between Morgan fingerprints of molecules 12/5/19
Constrained property optimization ¡ +180% higher scores than JT-VAE Jure Leskovec, Stanford CS224W: Machine Learning with Graphs, http://cs224w.stanford.edu 82 12/5/19
Visualization of GCPN graphs: Property optimization Jure Leskovec, Stanford CS224W: Machine Learning with Graphs, http://cs224w.stanford.edu 83 12/5/19
Visualization of GCPN graphs: Constrained optimization Jure Leskovec, Stanford CS224W: Machine Learning with Graphs, http://cs224w.stanford.edu 84 Starting structure Finished structure 12/5/19
¡ Complex graphs can be successfully generated via sequential generation ¡ Each step a decision is made based on hidden state, which can be § Explicit: intermediate generated graphs, decode with GCN § Implicit: vector representation, decode with RNN ¡ Possible tasks: § Imitating a set of given graphs § Optimizing graphs towards given goals 12/5/19 Jure Leskovec, Stanford CS224W: Machine Learning with Graphs, http://cs224w.stanford.edu 85
PinSage: ¡ Graph convolutional neural networks for web-scale recommender systems. R. Ying, R. He, K. Chen, P. Eksombatchai, W. Hamilton, J. Leskovec. KDD 2018. Decagon: ¡ Modeling polypharmacy side effects with graph convolutional networks. Z., Marinka, M. Agrawal, J. Leskovec. Bioinformatics 2018. ¡ Website: http://snap.stanford.edu/decagon/ GCPN: ¡ Graph Convolutional Policy Network for Goal-Directed Molecular Graph Generation. J. You, B. Liu, R. Ying, V. Pande, J. Leskovec. NeurIPS 2018. ¡ Code: https://github.com/bowenliu16/rl_graph_generation 12/5/19 Jure Leskovec, Stanford CS224W: Machine Learning with Graphs, http://cs224w.stanford.edu 86
¡ Project write-ups: § Tue Dec 10 (11:59PM) Pacific Time § 1 team member uploads PDF to Gradescope § Don’t forget to tag your other team members! ¡ Poster session: § Thu Dec 12, 12:15 – 3:15 pm in Huang Foyer § All groups with at least one non-SCPD member must present § There should be 1 person at the poster at all times § Prepare a 2-minute elevator pitch of your poster § More instructions on Piazza 12/5/19 Jure Leskovec, Stanford CS224W: Machine Learning with Graphs, http://cs224w.stanford.edu 87 No late days!
¡ CS246: Mining Massive Datasets (Winter 2020) § Data Mining & Machine Learning for Big Data § (big==doesn’t fit in memory/single machine), SPARK ¡ CS341: Project in Data Mining (Spring 2020) § Groups do a research project on Big Data § We provide interesting data, projects and access to the Google Cloud infrastructure § Nice way to finish up CS224W project & publish it! 12/5/19 Jure Leskovec, Stanford CS224W: Machine Learning with Graphs, http://cs224w.stanford.edu 88
¡ Conferences / Journals: § KDD: Conf. on Knowledge Discovery & Data Mining § ICML: Intl. Conf. on Machine Learning § NeurIPS: Neural Information Processing Systems § ICLR: Intl. Conf. on Learning Representations § WWW: ACM World Wide Web Conference § WSDM: ACM Web search and Data Mining § ICWSM: AAAI Int. Conf. on Web-blogs & Social Media § Journal of Network Science § Journal of Complex Networks 12/5/19 Jure Leskovec, Stanford CS224W: Machine Learning with Graphs, http://cs224w.stanford.edu 89
¡ Other relevant courses: § CS229: Machine Learning § CS230: Deep Learning § MSE231: Computational Social Science § MSE334: The Structure of Social Data § CS276: Information Retrieval and Web Search § CS245: Database System Principles § CS347: Transaction Processing & Databases 12/5/19 Jure Leskovec, Stanford CS224W: Machine Learning with Graphs, http://cs224w.stanford.edu 90
Thank You 12/5/19 Jure Leskovec, Stanford CS224W: Machine Learning with Graphs 91
¡ You Have Done a Lot!!! ¡ And (hopefully) learned a lot!!! § Answered questions and proved many interesting results § Implemented a number of methods § And are doing excellently on the class project! 12/5/19 Jure Leskovec, Stanford CS224W: Machine Learning with Graphs, http://cs224w.stanford.edu 92 ThankYou for the Hard Work!!!
93 Jure Leskovec, Stanford CS224W: Social and Information Network Analysis, http://cs224w.stanford.edu 12/5/19

More Related Content

PPTX
Machine learning with graph
byDing Li
 
PDF
02-tradition-ml.pdf
byKarasuLee
 
PPTX
Graph Representation Learning
byJure Leskovec
 
PDF
Gnn overview
byLouis (Yufeng) Wang
 
PDF
Graph Neural Networks for Recommendations
byWQ Fan
 
PPTX
Colloquium.pptx
byMythili680896
 
PDF
COMP4222-Lecture 6-Graph-Neural-Networks-1-Basic.pdf
bylingchang140
 
PPTX
Sun_MAPL_GNN.pptx
byssuser1760c0
 
Machine learning with graph
byDing Li
 
02-tradition-ml.pdf
byKarasuLee
 
Graph Representation Learning
byJure Leskovec
 
Gnn overview
byLouis (Yufeng) Wang
 
Graph Neural Networks for Recommendations
byWQ Fan
 
Colloquium.pptx
byMythili680896
 
COMP4222-Lecture 6-Graph-Neural-Networks-1-Basic.pdf
bylingchang140
 
Sun_MAPL_GNN.pptx
byssuser1760c0
 

Similar to Pinsage Stanford slides.pdf

PDF
Representation Learning in Large Attributed Graphs
byNesreen K. Ahmed
 
PDF
COMP4222-Lecture 9-Graph-Isomorphic-Network.pdf
bylingchang140
 
PDF
kdd_talk.pdf
byssuser6d9950
 
PDF
kdd_talk.pdf
byssuser6d9950
 
PPTX
Introduction to Graph Neural Networks: Basics and Applications - Katsuhiko Is...
byPreferred Networks
 
PDF
Graph Neural Networks.pdf
bymahyamk
 
PDF
Convex optimization analysis in todays world scenario.pdf
bymahizxy
 
PDF
Graph neural networks overview
byRodion Kiryukhin
 
PDF
Memory Efficient Graph Convolutional Network based Distributed Link Prediction
bymiyurud
 
PDF
Scalable Graph Convolutional Network Based Link Prediction on a Distributed G...
bymiyurud
 
PPTX
240715_Thuy_Labseminar[SeedGNN: Graph Neural Network for Supervised Seeded Gr...
bythanhdowork
 
PPTX
Learning to Classify Users in Online Interaction Networks
bySymeon Papadopoulos
 
PDF
An experimental evaluation of similarity-based and embedding-based link predi...
byIJDKP
 
PDF
Deep Learning for Graphs
byDeepLearningBlr
 
PDF
High-Performance Graph Analysis and Modeling
byNesreen K. Ahmed
 
PDF
Grl book
byHibaRamadan4
 
PDF
Creating Community at WeWork through Graph Embeddings with node2vec - Karry Lu
byRising Media Ltd.
 
PPTX
Chapter 4 better.pptx
byAbanobZakaria1
 
PPTX
20191107 deeplearningapproachesfornetworks
bytm1966
 
PPTX
Graph R-CNN for Scene Graph Generation
bySangmin Woo
 
Representation Learning in Large Attributed Graphs
byNesreen K. Ahmed
 
COMP4222-Lecture 9-Graph-Isomorphic-Network.pdf
bylingchang140
 
kdd_talk.pdf
byssuser6d9950
 
kdd_talk.pdf
byssuser6d9950
 
Introduction to Graph Neural Networks: Basics and Applications - Katsuhiko Is...
byPreferred Networks
 
Graph Neural Networks.pdf
bymahyamk
 
Convex optimization analysis in todays world scenario.pdf
bymahizxy
 
Graph neural networks overview
byRodion Kiryukhin
 
Memory Efficient Graph Convolutional Network based Distributed Link Prediction
bymiyurud
 
Scalable Graph Convolutional Network Based Link Prediction on a Distributed G...
bymiyurud
 
240715_Thuy_Labseminar[SeedGNN: Graph Neural Network for Supervised Seeded Gr...
bythanhdowork
 
Learning to Classify Users in Online Interaction Networks
bySymeon Papadopoulos
 
An experimental evaluation of similarity-based and embedding-based link predi...
byIJDKP
 
Deep Learning for Graphs
byDeepLearningBlr
 
High-Performance Graph Analysis and Modeling
byNesreen K. Ahmed
 
Grl book
byHibaRamadan4
 
Creating Community at WeWork through Graph Embeddings with node2vec - Karry Lu
byRising Media Ltd.
 
Chapter 4 better.pptx
byAbanobZakaria1
 
20191107 deeplearningapproachesfornetworks
bytm1966
 
Graph R-CNN for Scene Graph Generation
bySangmin Woo
 

Recently uploaded

PPTX
Software Analysis &Design ethiopia chap-2.pptx
byAbbas484771
 
PPTX
Building Cyber Resilience for 2026: Best Practices for a Secure, AI-Driven Bu...
byYasir Naveed Riaz
 
DOCX
iRobot Post‑Mortem and Alternative Paths - Discussion Document for Boards and...
byDave Litwiller
 
PDF
WCAG Analyzer Tools and Techniques for User-Friendly Websites
byAccessibility Analyzer
 
PDF
Day 5 - Red Team + Blue Team in the Cloud - 2nd Sight Lab Cloud Security Class
by2nd Sight Lab
 
PPTX
How to make Ai agents using Google Gemini AI.pptx
bypkluffy111
 
PPTX
From Backup to Resilience: How MSPs Are Preparing for 2026
byMSP360
 
PPTX
Emancipatory Information Retrieval: Radically Reorienting Information Retriev...
byBhaskar Mitra
 
PPTX
Top _HR Technology Trends _for 2026_.pptx
byAutomationEdge Technologies
 
PDF
HCSP-Presales-Campus Network Planning and Design V2.0
byVeronica916344
 
PDF
Six Shifts For 2026 (And The Next Six Years)
byDavid Armano
 
PDF
Exam Prep Plan Overview: Amazon Web Services (AWS) Certified
byVICTOR MAESTRE RAMIREZ
 
PDF
Data Virtualization in Action: Scaling APIs and Apps with FME
bySafe Software
 
PDF
Our Digital Tribe_ Cultivating Connection and Growth in Our Slack Community 🌿...
bysanjeetmishra30
 
PDF
AI Powered Document Processing and Data Extraction
byRobert McDermott
 
PDF
DevFest El Jadida 2025 - Product Thinking
byElmehdi AMLOU
 
PDF
Dev Dives: AI that builds with you - UiPath Autopilot for effortless RPA & AP...
byUiPathCommunity
 
PDF
TrustArc Webinar - Looking Ahead: The 2026 Privacy Landscape
byTrustArc
 
PPTX
Session 2 - Solving Unstructured & Complex Documents with UiPath IXP
byDianaGray10
 
PDF
Internet_of_Things_IoT_for_Next_Generation_Smart_Systems_Utilizing.pdf
byRoshanSyed12
 
Software Analysis &Design ethiopia chap-2.pptx
byAbbas484771
 
Building Cyber Resilience for 2026: Best Practices for a Secure, AI-Driven Bu...
byYasir Naveed Riaz
 
iRobot Post‑Mortem and Alternative Paths - Discussion Document for Boards and...
byDave Litwiller
 
WCAG Analyzer Tools and Techniques for User-Friendly Websites
byAccessibility Analyzer
 
Day 5 - Red Team + Blue Team in the Cloud - 2nd Sight Lab Cloud Security Class
by2nd Sight Lab
 
How to make Ai agents using Google Gemini AI.pptx
bypkluffy111
 
From Backup to Resilience: How MSPs Are Preparing for 2026
byMSP360
 
Emancipatory Information Retrieval: Radically Reorienting Information Retriev...
byBhaskar Mitra
 
Top _HR Technology Trends _for 2026_.pptx
byAutomationEdge Technologies
 
HCSP-Presales-Campus Network Planning and Design V2.0
byVeronica916344
 
Six Shifts For 2026 (And The Next Six Years)
byDavid Armano
 
Exam Prep Plan Overview: Amazon Web Services (AWS) Certified
byVICTOR MAESTRE RAMIREZ
 
Data Virtualization in Action: Scaling APIs and Apps with FME
bySafe Software
 
Our Digital Tribe_ Cultivating Connection and Growth in Our Slack Community 🌿...
bysanjeetmishra30
 
AI Powered Document Processing and Data Extraction
byRobert McDermott
 
DevFest El Jadida 2025 - Product Thinking
byElmehdi AMLOU
 
Dev Dives: AI that builds with you - UiPath Autopilot for effortless RPA & AP...
byUiPathCommunity
 
TrustArc Webinar - Looking Ahead: The 2026 Privacy Landscape
byTrustArc
 
Session 2 - Solving Unstructured & Complex Documents with UiPath IXP
byDianaGray10
 
Internet_of_Things_IoT_for_Next_Generation_Smart_Systems_Utilizing.pdf
byRoshanSyed12
 

Pinsage Stanford slides.pdf

  • 1.
    CS224W: Analysis ofNetworks Jure Leskovec, R.Ying and J.You, Stanford University http://cs224w.stanford.edu
  • 2.
    Three topics fortoday: 1. GNN recommendation (PinSage) 2. Heterogeneous GNN (Decagon) 3. Goal-directed generation (GCPN) 12/5/19 Jure Leskovec, Stanford CS224W: Machine Learning with Graphs, http://cs224w.stanford.edu 2
  • 4.
    Jure Leskovec, StanfordCS224W: Machine Learning with Graphs, http://cs224w.stanford.edu 4 Items Users ¡ Users interacts with items § Watch movies, buy merchandise, listen to music ¡ Goal: Recommend items users might like § Customer X buys Metallica and Megadeth CDs § Customer Y buys Megadeth, the recommender system suggests Metallica as well 12/5/19 Interactions “You might also like”
  • 5.
    Goal: Learn whatitems are related ¡ For a given query item(s) Q, return a set of similar items that we recommend to the user Idea: ¡ User interacts with a set of items ¡ Formulate a query Q ¡ Search the items and return recommendations 12/5/19 Jure Leskovec, Stanford CS224W: Machine Learning with Graphs, http://cs224w.stanford.edu 5 Items Query Recommendations Products, web sites, movies, posts, ads, …
  • 6.
    Query: 12/5/19 Jure Leskovec,Stanford CS224W: Machine Learning with Graphs, http://cs224w.stanford.edu 6
  • 7.
    Query: Recommendations: 12/5/19 JureLeskovec, Stanford CS224W: Machine Learning with Graphs, http://cs224w.stanford.edu 7
  • 8.
    Query: 12/5/19 Jure Leskovec,Stanford CS224W: Machine Learning with Graphs, http://cs224w.stanford.edu 8
  • 9.
    Query: Recommendations: 12/5/19 JureLeskovec, Stanford CS224W: Machine Learning with Graphs, http://cs224w.stanford.edu 9
  • 10.
    Having a universalsimilarity function allows for many applications: 12/5/19 Jure Leskovec, Stanford CS224W: Machine Learning with Graphs, http://cs224w.stanford.edu 10 Homefeed (endless feed of recommendations) Related pins (find most similar/related pins) Ads and shopping (use organic for the query and search the ads database)
  • 11.
    Question: How dowe define similarity? ¡ 1) Content-based: User and item features, in the form of images, text, categories, etc. ¡ 2) Graph-based: User-item interactions, in the form of graph/network structure § This is called collaborative filtering: § For a given user X, find others who liked similar items § Estimate what X will like based on what similar others like 12/5/19 Jure Leskovec, Stanford CS224W: Machine Learning with Graphs, http://cs224w.stanford.edu 11
  • 12.
    How do wedefine similarity: ¡ (1) Gathering “known” similarities § How to collect the data about what users like ¡ (2) Extrapolating unknown similarities from the known ones § Mainly interested in high unknown similarities § We are not interested in knowing what you don’t like but what you like ¡ (3) Evaluating methods § How to measure success/performance of recommendation methods 12/5/19 12 Jure Leskovec, Stanford CS224W: Machine Learning with Graphs, http://cs224w.stanford.edu
  • 13.
    ¡ 300M users ¡4+B pins, 2+B boards Jure Leskovec, Stanford CS224W: Machine Learning with Graphs, http://cs224w.stanford.edu 13 12/5/19
  • 14.
    12/5/19 Jure Leskovec,Stanford CS224W: Machine Learning with Graphs, http://cs224w.stanford.edu 14 Pinterest: Human curated collection of pins Pin:Avisualbookmarksomeonehas savedfromtheinternettoaboard they’vecreated. Pin:Image,text,link Board: A collection of ideas (pins having something in common)
  • 15.
    Two sources ofsignal: Features: ¡ Image and text of each pin Graph: ¡ Graph is dynamic: Need to apply to new nodes without model retraining 12/5/19 Jure Leskovec, Stanford CS224W: Machine Learning with Graphs, http://cs224w.stanford.edu 15 Q
  • 16.
    Goal: Learn embeddingsfor items ¡ Related Pins Query: Which pin to recommend when a user interacts with a pin 𝑣"? ¡ Answer: Find the closest embedding (𝑣#) to 𝑣" by nearest neighbor. Recommend it. Jure Leskovec, Stanford CS224W: Machine Learning with Graphs, http://cs224w.stanford.edu 16 𝑣$ 𝑣% 𝑣" 𝑣# 12/5/19 Item embeddings Previously pinned Query pin Related pin recommendation
  • 17.
    ¡ Goal 1:Efficiently learn embeddings for billions of pins (items, nodes) using neural networks ¡ Goal 2: Perform nearest neighbor query to recommend items in real-time 12/5/19 Jure Leskovec, Stanford CS224W: Machine Learning with Graphs, http://cs224w.stanford.edu 17 Query pin “Predicted” related pin Embed Embedding space The closer the embeddings are, the more similar the pins are
  • 18.
    Task: Recommend relatedpins to users Query pin 8 Predict whether two nodes in a graph are related Task: Learn node embeddings 𝑧' such that 𝑑 𝑧)*+,$, 𝑧)*+,% < 𝑑(𝑧)*+,$, 𝑧01,*2,3) Jure Leskovec, Stanford CS224W: Machine Learning with Graphs, http://cs224w.stanford.edu 𝑧$ 𝑧% 𝑑(𝑧$, 𝑧%) 12/5/19
  • 19.
    Approach: ¡ Pins haveembeddings at each layer ¡ Layer-0 embedding of a node are its features: § Text, image, … pin board ... Aggregator ... ... ... Agg. Agg. Agg. Predict whether two nodes in a graph are related 12/5/19 Jure Leskovec, Stanford CS224W: Machine Learning with Graphs, http://cs224w.stanford.edu 19
  • 20.
    ¡ PinSage graphconvolutional network: § Goal: Generate embeddings for nodes (e.g., pins) in the Pinterest graph containing billions of objects § Key Idea: Borrow information from nearby nodes § E.g., bed rail Pin might look like a garden fence, but gates and beds are rarely adjacent in the graph § Pin embeddings are essential to many different tasks. Aside from the “Related Pins” task, it can also be used in: § Recommend related ads § Homefeed recommendation § Cluster users by their interest 12/5/19 Jure Leskovec, Stanford CS224W: Machine Learning with Graphs, http://cs224w.stanford.edu 20 [Ying et al.,WWW 2018] sum - multiset > mean - distribution max - set > Input Figure 2: Ranking by expressive power for sum, mean and max-pooling aggregators over a multiset. Left panel shows the input multiset and the three panels illustrate the aspects of the multiset a given aggregator is able to capture: sum captures the full multiset, mean captures the proportion/distribution of elements of a given type, and the max aggregator ignores multiplicities (reduces the multiset to a simple set). vs. (a) Mean and Max both fail vs. (b) Max fails vs. (c) Mean and Max both fail Figure 3: Examples of simple graph structures that mean and max-pooling aggregators fail to distinguish. Figure 2 gives reasoning about how different aggregators “compress” different graph structures/multisets. existing GNNs instead use a 1-layer perceptron W (Duvenaud et al., 2015; Kipf & Welling, 2017; Zhang et al., 2018), a linear mapping followed by a non-linear activation function such as a ReLU. A
  • 21.
    1. Collect billionsof training pairs from logs. § Positive pair: Two pins that are consecutively saved into the same board within a time interval (1 hour) § Negative pair: A random pair of 2 pins § With high probability the pins are not on the same board 12/5/19 Jure Leskovec, Stanford CS224W: Machine Learning with Graphs, http://cs224w.stanford.edu 21
  • 22.
    1. Collect billionsof training pairs from logs. § Positive pair: Two pins that are consecutively saved into the same board within a time interval (1 hour) § Negative pair: A random pair of 2 pins § With high probability the pins are not on the same board 2. Train GNN to generate similar embeddings for training pairs 3. Inference: Generate embeddings for all pins 4. Nearest neighbor search in embedding space to make recommendations. 12/5/19 Jure Leskovec, Stanford CS224W: Machine Learning with Graphs, http://cs224w.stanford.edu 22
  • 23.
    ¡ Train sothat pins that are consecutively pinned have similar embeddings ¡ Max-margin loss: 12/5/19 Jure Leskovec, Stanford CS224W: Machine Learning with Graphs, http://cs224w.stanford.edu 23 L = X (u,v)2D max(0, z> u zv + z> u zn + ) set of training pairs from user logs “positive”/true training pair “negative” example “margin” (i.e., how much larger positive pair similarity should be compared to negative)
  • 24.
    ¡ Four keyinnovations: 1. On-the-fly graph convolutions § Sample the neighborhood around a node and dynamically construct a computation graph 12/5/19 Jure Leskovec, Stanford CS224W: Machine Learning with Graphs, http://cs224w.stanford.edu 24 Minibatch of neighborhoods
  • 25.
    ¡ Four keyinnovations: 1. On-the-fly graph convolutions § Perform a localized graph convolution around a particular node § Does not need the entire graph during training 12/5/19 Jure Leskovec, Stanford CS224W: Machine Learning with Graphs, http://cs224w.stanford.edu 25 At every iteration, only source node embeddings are computed
  • 26.
    ¡ Four keyinnovations: 2. Selecting neighbors via random walks § Performing aggregation on all neighbors is infeasible: § How to select the set of neighbors of a node to convolve over? § Personalized PageRank can help! § Define Importance pooling: Define importance-based neighborhoods by simulating random walks and selecting the neighbors with the highest visit counts 12/5/19 Jure Leskovec, Stanford CS224W: Machine Learning with Graphs, http://cs224w.stanford.edu 26
  • 27.
    ¡ Proximity toquery node(s) Q PixieRandomWalk 5 5 5 5 5 5 14 9 16 7 8 8 8 8 1 1 1 Strawberries Smoothies Yummm Smoothie Madness!•! Q Jure Leskovec, Stanford CS224W: Machine Learning with Graphs, http://cs224w.stanford.edu 27 12/5/19
  • 28.
    ¡ Proximity toquery node(s) Q ¡ Importance pooling § Choose nodes with top K visit counts § Pool over the chosen nodes § The chosen nodes are not necessarily neighbors PixieRandomWalk 5 5 5 5 5 5 14 9 16 7 8 8 8 8 1 1 1 Strawberries Smoothies Yummm Smoothie Madness!•!• Q Jure Leskovec, Stanford CS224W: Machine Learning with Graphs, http://cs224w.stanford.edu 28 12/5/19
  • 29.
    ¡ Example: suppose𝐾=5 ¡ Rank nodes based on Random Walk visit counts ¡ Pick top 𝑲 nodes and normalize counts 16 55 , 14 55 , 9 55 , 8 55 , 8 55 ¡ Aggregate messages from the top 𝐾 nodes Jure Leskovec, Stanford CS224W: Machine Learning with Graphs, http://cs224w.stanford.edu 29 PixieRandomWalk 5 5 5 5 5 5 14 9 16 7 8 8 8 8 1 1 1 Strawberries Smoothies Yummm Smoothie Madness!•!• Q Top 𝑲 nodes 12/5/19
  • 30.
    ¡ Pick topK nodes and normalize counts 16 55 , 14 55 , 9 55 , 8 55 , 8 55 ¡ GraphSAGE mean pooling § Average the messages from direct neighbors ¡ PinSAGE Importance pooling § Use the normalized counts as weights for weighted mean of messages from the top K nodes ¡ PinSAGE uses 𝐾 = 50 § Negligible performance gain for 𝐾 > 50 Jure Leskovec, Stanford CS224W: Machine Learning with Graphs, http://cs224w.stanford.edu 30 12/5/19
  • 31.
    Four key innovations: 3.Efficient MapReduce inference § Problem: Many repeated computation if using localized graph convolution at inference step § Need to avoid repeated computation 12/5/19 Jure Leskovec, Stanford CS224W: Machine Learning with Graphs, http://cs224w.stanford.edu 31 Repeated computation
  • 32.
    ¡ Recall howwe obtain negative examples Jure Leskovec, Stanford CS224W: Machine Learning with Graphs, http://cs224w.stanford.edu 33 L = X (u,v)2D max(0, z> u zv + z> u zn + ) set of training pairs from logs “positive”/true example “negative” example “margin” (i.e., how much larger positive pair similarity should be compared to negative) 12/5/19
  • 33.
    Goal: Identify targetpin among 3B pins ¡ Issue: Need to learn with resolution of 100 vs. 3B ¡ Massive size: 3 billion nodes, 20 billion edges ¡ Idea: Use harder and harder negative samples Jure Leskovec, Stanford CS224W: Machine Learning with Graphs, http://cs224w.stanford.edu 34 L = X (u,v)2D max(0, z> u zv + z> u zn + ) “positive”/true example negative examples “margin” (i.e., how much larger positive pair similarity should be compared to negative) set of training pairs from logs Force model to learn subtle distinctions between pins 12/5/19
  • 34.
    ¡ Hard negativeexamples improve performance ¡ How to obtain hard negatives: Use random walks: § Use nodes with visit counts ranked at 1000-5000 as hard negatives § Have something in common, but are not too similar 12/5/19 Jure Leskovec, Stanford CS224W: Machine Learning with Graphs, http://cs224w.stanford.edu 35 Harder to distinguish from the positive pair Positive pair
  • 35.
    ¡ Hard negativeexamples improve performance ¡ Curriculum training on hard negatives § Start with random negative examples § Provide harder negative examples over time 12/5/19 Jure Leskovec, Stanford CS224W: Machine Learning with Graphs, http://cs224w.stanford.edu 36 Harder to distinguish from the positive pair Positive pair
  • 36.
    Related Pin recommendations ¡Given a user just saved pin Q, predict what pin X are they going to save next ¡ Setup: Embed 3B pins, find nearest neighbors of Q ¡ Baseline embeddings: § Visual: VGG visual embeddings Annotation: Word2vec embeddings § Combined: Concatenate embeddings 12/5/19 Jure Leskovec, Stanford CS224W: Machine Learning with Graphs, http://cs224w.stanford.edu 37 MRR: Mean reciprocal rank of the positive example X w.r.t Q Hit rate: Fraction of times the positive example X is among top K closest to Q
  • 37.
    Jure Leskovec, StanfordCS224W: Machine Learning with Graphs, http://cs224w.stanford.edu 12/5/19 38 Pixie (graph-based): the method of simulating random walks starting at query Pin using the Pixie algorithm in class. Items with top scores are retrieved as recommendations Visual, Annot. (feature-based): nearest neighbor recommendation using visual (CNN) and annotation features of pins
  • 38.
    Pixie Graph- SAGE Query PinSAGE 12/5/19 Jure Leskovec,Stanford CS224W: Machine Learning with Graphs, http://cs224w.stanford.edu 39
  • 39.
    Pixie Graph- SAGE Query PinSAGE 12/5/19 Jure Leskovec,Stanford CS224W: Machine Learning with Graphs, http://cs224w.stanford.edu 40
  • 40.
    1. GNN recommendation(PinSage) 2. Heterogeneous GNN (Decagon) 3. Goal-directed generation (GCPN) 12/5/19 Jure Leskovec, Stanford CS224W: Machine Learning with Graphs, http://cs224w.stanford.edu 41
  • 42.
    ¡ So farwe only applied GNNs to simple graphs § GNNs do not explicitly use node and edge type information ¡ Real networks are often heterogeneous ¡ How to use GNN for heterogeneous graphs? Jure Leskovec, Stanford CS224W: Machine Learning with Graphs, http://cs224w.stanford.edu 43 12/5/19
  • 43.
    12/5/19 Jure Leskovec,Stanford CS224W: Machine Learning with Graphs, http://cs224w.stanford.edu 44 , Patient’s side effects Patient’s medications Polypharmacy side effect Drug combination Polypharmacy: use multiple drugs for a disease
  • 44.
    ¡ Polypharmacy iscommon to treat complex diseases and co-existing conditions ¡ High risk of side effects due to interactions ¡ 15% of the U.S. population affected ¡ Annual costs exceed $177 billion ¡ Difficult to identify manually: § Rare, occur only in a subset of patients § Not observed in clinical testing 12/5/19 Jure Leskovec, Stanford CS224W: Machine Learning with Graphs, http://cs224w.stanford.edu 45
  • 45.
    ¡ Systematic experimentalscreening of drug interactions is challenging ¡ Idea: Computationally screen/predict polypharmacy side effects § Use molecular, pharmacological and patient population data § Guide translational strategies for combination treatments in patients 12/5/19 Jure Leskovec, Stanford CS224W: Machine Learning with Graphs, http://cs224w.stanford.edu 46
  • 46.
    , . . . . . . How likely witha pair of drugs 𝑐, 𝑑 lead to side effect 𝑟? Model and predict side effects of drug pairs 𝑐 𝑑 𝑟 Jure Leskovec, Stanford CS224W: Machine Learning with Graphs, http://cs224w.stanford.edu 12/5/19 47
  • 47.
    ¡ Heterogeneous (multimodal)graphs: graphs with different node types and/or edge types 12/5/19 Jure Leskovec, Stanford CS224W: Machine Learning with Graphs, http://cs224w.stanford.edu 48 2 node types edge types
  • 48.
    Goal: Given apartially observed graph, predict labeled edges between drug nodes Ciprofloxacin r1 r2 Simvastatin Mupirocin r2 Doxycycline S C M D Query: Given a drug pair 𝑐, 𝑑, how likely does an edge (𝑐, 𝑟%, 𝑑) exist? Co-prescribed drugs 𝑐 and 𝑑 lead to side effect 𝑟% Jure Leskovec, Stanford CS224W: Machine Learning with Graphs, http://cs224w.stanford.edu 12/5/19 49
  • 49.
    ¡ Predict labelededges between drugs nodes § i.e., predict the likelihood that an edge (𝑐, 𝑟%, 𝑠) exists between drug nodes 𝑐 and 𝑠 § Meaning: Drug combination (𝑐, 𝑠) leads to polypharmacy side effect 𝑟% 12/5/19 Jure Leskovec, Stanford CS224W: Machine Learning with Graphs, http://cs224w.stanford.edu 50 Predictions:
  • 50.
    ¡ Key Insight:Compute GNN messages from each edge type, then aggregate across different edge types 12/5/19 Jure Leskovec, Stanford CS224W: Machine Learning with Graphs, http://cs224w.stanford.edu 51 § Input: heterogenous graph § Output: node embeddings One layer of Heterogeneous GNN GNN for Edge type: 𝒓𝟐 GNN for Edge type: drug-target Sum GNN for Edge type: 𝒓𝟏
  • 51.
    12/5/19 Jure Leskovec,Stanford CS224W: Machine Learning with Graphs, http://cs224w.stanford.edu 52 § Input: Node embeddings of query drug pairs § Output: predicted edges ¡ Key Insight: Use pair of computed node embeddings to make edge predictions Predict possible edges with NN Neural Network
  • 52.
    v v v p – probability JureLeskovec, Stanford CS224W: Machine Learning with Graphs, http://cs224w.stanford.edu 12/5/19 53
  • 53.
    ¡ Data: § Graphover Molecules: protein-protein interaction and drug target relationships § Graph over Population: Side effects of individual drugs, polypharmacy side effects of drug combinations ¡ Setup: § Construct a heterogeneous graph of all the data § Train: Fit a model to predict known associations of drug pairs and polypharmacy side effects § Test: Given a query drug pair, predict candidate polypharmacy side effects 12/5/19 Jure Leskovec, Stanford CS224W: Machine Learning with Graphs, http://cs224w.stanford.edu 54
  • 54.
    ¡ Up to54% improvement over baselines ¡ First opportunity to computationally flag polypharmacy side effects for follow-up analyses 12/5/19 Jure Leskovec, Stanford CS224W: Machine Learning with Graphs, http://cs224w.stanford.edu 55 AUROC AUPRC AP@50 Decagon (3-layer) 0.834 0.776 0.731 Decagon (2-layer) 0.809 0.762 0.713 RESCAL 0.693 0.613 0.476 Node2vec 0.725 0.708 0.643 Drug features 0.736 0.722 0.679
  • 55.
    Drug c Drugd Jure Leskovec, Stanford CS224W: Machine Learning with Graphs, http://cs224w.stanford.edu 12/5/19 56
  • 56.
    Evidence found Drug cDrug d Jure Leskovec, Stanford CS224W: Machine Learning with Graphs, http://cs224w.stanford.edu 12/5/19 57
  • 57.
    1. GNN recommendation(PinSage) 2. Heterogeneous GNN (Decagon) 3. Goal-directed generation (GCPN) 12/5/19 Jure Leskovec, Stanford CS224W: Machine Learning with Graphs, http://cs224w.stanford.edu 58
  • 59.
    ¡ Given: Graphssampled from 𝑝G*2*(𝐺) ¡ Goal: §Learn the distribution 𝑝IJG,K(𝐺) §Sample from 𝑝IJG,K(𝐺) 12/5/19 Jure Leskovec, Stanford CS224W: Machine Learning with Graphs, http://cs224w.stanford.edu 60 𝑝G*2*(𝐺) 𝑝IJG,K(𝐺) Learn & Sample
  • 60.
    Generating graphs viasequentially adding nodes and edges 12/5/19 Jure Leskovec, Stanford CS224W: Machine Learning with Graphs, http://cs224w.stanford.edu 61 [You et al., ICML 2018] 1 1 2 1 2 3 1 2 4 3 1 2 4 3 5 1 2 4 3 5 Graph 𝐺 Generation process 𝑆M
  • 61.
    Quick Summary ofGraphRNN: § Generate a graph by generating a two level sequence § Use RNN to generate the sequences 12/5/19 Jure Leskovec, Stanford CS224W: Machine Learning with Graphs, http://cs224w.stanford.edu 62 0 1 1 0 0 1 0 0 1 0 1 0 0 1 1 0 1 1 0 1 0 0 1 1 0 1 2 4 3 5 Graph 𝐺 Node-level RNN Edge-level RNN Adjacency matrix ⇔
  • 62.
    12/5/19 Jure Leskovec,Stanford CS224W: Machine Learning with Graphs, http://cs224w.stanford.edu 63
  • 63.
    12/5/19 Jure Leskovec,Stanford CS224W: Machine Learning with Graphs, http://cs224w.stanford.edu 64 Can we do more than imitating given graphs?
  • 64.
    Question: Can welearn a model that can generate valid and realistic molecules with high value of a given chemical property? 12/5/19 Jure Leskovec, Stanford CS224W: Machine Learning with Graphs, http://cs224w.stanford.edu 65 Model Property output that optimizes e.g., drug_likeness=0.95 [You et al., NeurIPS 2018] Graph Convolutional Policy Network for Goal-Directed Molecular Graph Generation. J. You, B. Liu, R. Ying, V. Pande, J. Leskovec. Neural Information Processing Systems (NeurIPS), 2018.
  • 65.
    12/5/19 Jure Leskovec,Stanford CS224W: Machine Learning with Graphs, http://cs224w.stanford.edu 66 § Node types: C, N, O, … § Edge types: single bond, double bond, … § Note: “H”s can be automatically inferred via chemical validity rules, thus are ignored in molecular graphs C N C C N C C Nodes Edges N
  • 66.
    Generating graphs that: ¡Optimize a given objective (High scores) § e.g., drug-likeness ¡ Obey underlying rules (Valid) § e.g., chemical validity rules ¡ Are learned from examples (Realistic) § e.g., Imitating a molecule graph dataset 12/5/19 Jure Leskovec, Stanford CS224W: Machine Learning with Graphs, http://cs224w.stanford.edu 67 Graph Convolutional Policy Network for Goal-Directed Molecular Graph Generation. J. You, B. Liu, R. Ying, V. Pande, J. Leskovec. Neural Information Processing Systems (NeurIPS), 2018.
  • 67.
    Generating graphs that: ¡Optimize a given objective (High scores) § e.g., drug-likeness ¡ Obey underlying rules (Valid) § e.g., chemical validity rules 12/5/19 Jure Leskovec, Stanford CS224W: Machine Learning with Graphs, http://cs224w.stanford.edu 68 Graph Convolutional Policy Network for Goal-Directed Molecular Graph Generation. J. You, B. Liu, R. Ying, V. Pande, J. Leskovec. Neural Information Processing Systems (NeurIPS), 2018. Including “Black-box” in ML: Objectives like drug-likeness are governed by physical law, which are assumed to be unknown to us!
  • 68.
    ¡ A MLagent observes the environment, takes an action to interact with the environment, and receives positive or negative reward ¡ The agent then learns from this loop ¡ Key: Environment is a blackbox to the agent 12/5/19 Jure Leskovec, Stanford CS224W: Machine Learning with Graphs, http://cs224w.stanford.edu 69 ML Agent Action Environment Observation, Reward
  • 69.
    ¡ Policy: Agentbehavior, which maps observation to action ¡ Policy-based RL: An agent directly learns an optimal policy from data 12/5/19 Jure Leskovec, Stanford CS224W: Machine Learning with Graphs, http://cs224w.stanford.edu 70 Agent Policy Action Environment Observation, Reward
  • 70.
    Graph Convolutional PolicyNetwork combines graph representation + RL: ¡ Graph Neural Network captures complex structural information, and enables validity check in each state transition (Valid) ¡ Reinforcement learning optimizes intermediate/final rewards (High scores) ¡ Adversarial training imitates examples in given datasets (Realistic) 12/5/19 Jure Leskovec, Stanford CS224W: Machine Learning with Graphs, http://cs224w.stanford.edu 71
  • 71.
    ¡ (a) Insertnodes/scaffolds ¡ (b) Compute state via GCN ¡ (c) Sample next action ¡ (d) Take action (check chemical validity) ¡ (e, f) Compute reward Jure Leskovec, Stanford CS224W: Machine Learning with Graphs, http://cs224w.stanford.edu 72 12/5/19
  • 72.
    ¡ Learn totake valid action § At each step, assign small positive reward for valid action ¡ Optimize desired properties § At the end, assign positive reward for high desired property ¡ Generate realistic graphs § At the end, adversarially train a GCN discriminator, compute adversarial rewards that encourage realistic molecule graphs Jure Leskovec, Stanford CS224W: Machine Learning with Graphs, http://cs224w.stanford.edu 73 12/5/19
  • 73.
    Reward: 𝑟2 =Final reward + Step reward ¡ Final reward = Domain-specific reward ¡ Step rewards = Step-wise validity reward 12/5/19 Jure Leskovec, Stanford CS224W: Machine Learning with Graphs, http://cs224w.stanford.edu 74
  • 74.
    ¡ Two parts: ¡(1) Supervised training: Train policy by imitating the action given by real observed graphs. Use gradient. ¡ (2) RL training: Train policy to optimize rewards. Use standard policy gradient algorithm (refer to any RL course, e.g., CS234). 12/5/19 Jure Leskovec, Stanford CS224W: Machine Learning with Graphs, http://cs224w.stanford.edu 75
  • 75.
    12/5/19 Jure Leskovec,Stanford CS224W: Machine Learning with Graphs, http://cs224w.stanford.edu 76 GCPN C N N C C C 0.1 1 Step reward Final reward Environment Run until stop Run one step 0.6 Cross entropy loss Graph !" Generated graph !"#$ Generated graph !% ∗ Real graph !"#$ ∗ Policy gradient C N C C C C C C C C C Query dataset Gradient Supervised Training RL Training
  • 76.
    Jure Leskovec, StanfordCS224W: Machine Learning with Graphs, http://cs224w.stanford.edu 77 Validity Score Realistic 12/5/19
  • 77.
    ¡ Property optimization §Generate molecules with high specified property score ¡ Property targeting § Generate molecules whose specified property score falls within given range ¡ Constrained property optimization § Edit a given molecule for a few steps to achieve higher specified property score Jure Leskovec, Stanford CS224W: Machine Learning with Graphs, http://cs224w.stanford.edu 78 12/5/19
  • 78.
    ¡ ZINC250k dataset §250,000 drug like molecules whose maximum atom number is 38 ¡ Baselines: § ORGAN: String representation + RL [Guimaraes et al., 2017] § JT-VAE: VAE-based vector representation + Bayesian optimization [Jin et al., 2018] Jure Leskovec, Stanford CS224W: Machine Learning with Graphs, http://cs224w.stanford.edu 79 12/5/19
  • 79.
    Property optimization ¡ +60%higher property scores Jure Leskovec, Stanford CS224W: Machine Learning with Graphs, http://cs224w.stanford.edu 80 logP: octanol-water partition coef., indicates solubility QED: indicator of drug-likeness 12/5/19
  • 80.
    Property targeting ¡ 7xhigher success rate than JT-VAE, 10% less diversity Jure Leskovec, Stanford CS224W: Machine Learning with Graphs, http://cs224w.stanford.edu 81 logP: octanol-water partition coef., indicates solubility MW: molecular weight an indicator of drug-likeness Diversity: avg. pairwise Tanimoto distance between Morgan fingerprints of molecules 12/5/19
  • 81.
    Constrained property optimization ¡+180% higher scores than JT-VAE Jure Leskovec, Stanford CS224W: Machine Learning with Graphs, http://cs224w.stanford.edu 82 12/5/19
  • 82.
    Visualization of GCPNgraphs: Property optimization Jure Leskovec, Stanford CS224W: Machine Learning with Graphs, http://cs224w.stanford.edu 83 12/5/19
  • 83.
    Visualization of GCPNgraphs: Constrained optimization Jure Leskovec, Stanford CS224W: Machine Learning with Graphs, http://cs224w.stanford.edu 84 Starting structure Finished structure 12/5/19
  • 84.
    ¡ Complex graphscan be successfully generated via sequential generation ¡ Each step a decision is made based on hidden state, which can be § Explicit: intermediate generated graphs, decode with GCN § Implicit: vector representation, decode with RNN ¡ Possible tasks: § Imitating a set of given graphs § Optimizing graphs towards given goals 12/5/19 Jure Leskovec, Stanford CS224W: Machine Learning with Graphs, http://cs224w.stanford.edu 85
  • 85.
    PinSage: ¡ Graph convolutionalneural networks for web-scale recommender systems. R. Ying, R. He, K. Chen, P. Eksombatchai, W. Hamilton, J. Leskovec. KDD 2018. Decagon: ¡ Modeling polypharmacy side effects with graph convolutional networks. Z., Marinka, M. Agrawal, J. Leskovec. Bioinformatics 2018. ¡ Website: http://snap.stanford.edu/decagon/ GCPN: ¡ Graph Convolutional Policy Network for Goal-Directed Molecular Graph Generation. J. You, B. Liu, R. Ying, V. Pande, J. Leskovec. NeurIPS 2018. ¡ Code: https://github.com/bowenliu16/rl_graph_generation 12/5/19 Jure Leskovec, Stanford CS224W: Machine Learning with Graphs, http://cs224w.stanford.edu 86
  • 86.
    ¡ Project write-ups: §Tue Dec 10 (11:59PM) Pacific Time § 1 team member uploads PDF to Gradescope § Don’t forget to tag your other team members! ¡ Poster session: § Thu Dec 12, 12:15 – 3:15 pm in Huang Foyer § All groups with at least one non-SCPD member must present § There should be 1 person at the poster at all times § Prepare a 2-minute elevator pitch of your poster § More instructions on Piazza 12/5/19 Jure Leskovec, Stanford CS224W: Machine Learning with Graphs, http://cs224w.stanford.edu 87 No late days!
  • 87.
    ¡ CS246: MiningMassive Datasets (Winter 2020) § Data Mining & Machine Learning for Big Data § (big==doesn’t fit in memory/single machine), SPARK ¡ CS341: Project in Data Mining (Spring 2020) § Groups do a research project on Big Data § We provide interesting data, projects and access to the Google Cloud infrastructure § Nice way to finish up CS224W project & publish it! 12/5/19 Jure Leskovec, Stanford CS224W: Machine Learning with Graphs, http://cs224w.stanford.edu 88
  • 88.
    ¡ Conferences /Journals: § KDD: Conf. on Knowledge Discovery & Data Mining § ICML: Intl. Conf. on Machine Learning § NeurIPS: Neural Information Processing Systems § ICLR: Intl. Conf. on Learning Representations § WWW: ACM World Wide Web Conference § WSDM: ACM Web search and Data Mining § ICWSM: AAAI Int. Conf. on Web-blogs & Social Media § Journal of Network Science § Journal of Complex Networks 12/5/19 Jure Leskovec, Stanford CS224W: Machine Learning with Graphs, http://cs224w.stanford.edu 89
  • 89.
    ¡ Other relevantcourses: § CS229: Machine Learning § CS230: Deep Learning § MSE231: Computational Social Science § MSE334: The Structure of Social Data § CS276: Information Retrieval and Web Search § CS245: Database System Principles § CS347: Transaction Processing & Databases 12/5/19 Jure Leskovec, Stanford CS224W: Machine Learning with Graphs, http://cs224w.stanford.edu 90
  • 90.
    Thank You 12/5/19 JureLeskovec, Stanford CS224W: Machine Learning with Graphs 91
  • 91.
    ¡ You HaveDone a Lot!!! ¡ And (hopefully) learned a lot!!! § Answered questions and proved many interesting results § Implemented a number of methods § And are doing excellently on the class project! 12/5/19 Jure Leskovec, Stanford CS224W: Machine Learning with Graphs, http://cs224w.stanford.edu 92 ThankYou for the Hard Work!!!
  • 92.
    93 Jure Leskovec, StanfordCS224W: Social and Information Network Analysis, http://cs224w.stanford.edu 12/5/19