2. About me
● Applied Scientist @ Amazon
○ Machine Learning for Recommendations & Search
● Previously: VP, Data Science @ Lazada
○ Led the team on e-commerce ML systems
○ Facilitated integration with Alibaba (acquired 2016)
● Previously: Data Science Lead @ Healthtech startup
○ Early detection of preventable diseases
○ Healthcare resource allocation
● More at eugeneyan.com
3. Topics
● Context: Batch vs. Real-time (i.e., on-demand)
● System design
○ Offline vs. Online environments
○ Candidate retrieval vs. Ranking
● Industry Examples
○ Alibaba, FaceBook, DoorDash
● How to build an MVP
○ If there’s time...
13. Online
Offline
● Host batch processes such as
training, index/graph building
● Load data into feature stores
14. Online
Offline
● Host batch processes such as
training, index/graph building
● Load data into feature stores
● Uses artifacts from offline
environment to serve requests
● Candidate retrieval and ranking
16. Retrieval Ranking
● Fast but coarse
● Searches millions of items to
get hundreds of candidates
● Approx NN, Graphs, etc.
17. Retrieval Ranking
● Fast but coarse
● Searches millions of items to
get hundreds of candidates
● Approx NN, Graphs, etc.
● Slower but more precise
● Ranks hundreds of candidates
● Adds more features
● Classification or learning-to-rank
19. Online
Offline
Retrieval Ranking
Train embedding
model
Train ranking
model
Embed items
from catalog
Build Approx.
NN index
Embed input
item or query
Retrieve top k
candidates
Build feature store
(e.g., item, user)
Add features to
candidates
Rank top k
candidates
22. Online
Offline
Retrieval Ranking
Train embedding
model
Training data Training data
Train ranking
model
Embed items
from catalog
Build Approx.
NN index
Item data
Embedding
model
ANN
Index
Build feature store
(e.g., item, user)
Item & user data
Feature
Store
Ranking
model
24. Online
Offline
Retrieval Ranking
Embed input
item or query
Retrieve top k
candidates
Request
Embedding
model
ANN
Index
Add features to
candidates
Rank top k
candidates
Feature
Store
Ranking
model
Results
25. Online
Offline
Retrieval Ranking
Train embedding
model
Training data Training data
Train ranking
model
Embed items
from catalog
Build Approx.
NN index
Item data
Embed input
item or query
Retrieve top k
candidates
Request
Embedding
model
ANN
Index
Build feature store
(e.g., item, user)
Item & user data
Add features to
candidates
Rank top k
candidates
Feature
Store
Ranking
model
Results
30. Building a graph network for ranking (Alibaba)
Source: https:/
/arxiv.org/abs/2005.12002
31. Building a graph network for ranking (Alibaba)
Source: https:/
/arxiv.org/abs/2005.12002
32. Building a graph network for ranking (Alibaba)
Source: https:/
/arxiv.org/abs/2005.12002
33. Building a graph network for ranking (Alibaba)
Source: https:/
/arxiv.org/abs/2005.12002
Ranker
Candidate
Retrieval
Train
Ranker
Feature
Stores
Graph
34. Building embeddings for retrieval in search (Facebook)
Source: https:/
/arxiv.org/abs/2006.11632
35. Building embeddings for retrieval in search (Facebook)
Source: https:/
/arxiv.org/abs/2006.11632
Train
Embedding
model
36. Building embeddings for retrieval in search (Facebook)
Source: https:/
/arxiv.org/abs/2006.11632
Train
Embedding
model
37. Building embeddings for retrieval in search (Facebook)
Source: https:/
/arxiv.org/abs/2006.11632
Train
Embedding
model
38. Building embeddings for retrieval in search (Facebook)
Source: https:/
/arxiv.org/abs/2006.11632
Train
Embedding
model
39. Building embeddings for retrieval in search (Facebook)
Source: https:/
/arxiv.org/abs/2006.11632
Train
Embedding
model
Offline
Online
40. Building embeddings for retrieval in search (Facebook)
Source: https:/
/arxiv.org/abs/2006.11632
Train
Embedding
model
Retrieval
Ranker
Feature
Store
Offline
Online
41. Building graphs for query expansion & retrieval (DoorDash)
Source: https:/
/doordash.engineering/2020/12/15/understanding-search-intent-with-better-recall/
42. Building graphs for query expansion & retrieval (DoorDash)
Source: https:/
/doordash.engineering/2020/12/15/understanding-search-intent-with-better-recall/
43. Building graphs for query expansion & retrieval (DoorDash)
Source: https:/
/doordash.engineering/2020/12/15/understanding-search-intent-with-better-recall/
44. Building graphs for query expansion & retrieval (DoorDash)
Source: https:/
/doordash.engineering/2020/12/15/understanding-search-intent-with-better-recall/
45. Building graphs for query expansion & retrieval (DoorDash)
Source: https:/
/doordash.engineering/2020/12/15/understanding-search-intent-with-better-recall/
46. Building graphs for query expansion & retrieval (DoorDash)
Source: https:/
/doordash.engineering/2020/12/15/understanding-search-intent-with-better-recall/
Knowledge
Graph
Retrieval
Ranker
52. Ranking: Logistic regression
Source: https:/
/eugeneyan.com/writing/real-time-recommendations/
• • •
User
features
Item
features
Probability of
interaction