Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Maja Kabiljo & Aleksandar Ilic,
Facebook
Large scale item recommendations
with Apache Giraph
Conclusion
04
01
02
05
03
Motivation and challenges
Apache Giraph
Distributing Collaborative Filtering
Social signals and ...
Collaborative
Filtering
Collaborative Filtering
Predict user’s interests based on many other users
Disney Roller coasters Disneyland Six Flags
?
? ?
? ?
?
? ? ?
Matrix factorization
4 4 1 3
5 3 1
1 2 4
5 3 4 5
2 3
...
. . .
...
U1
U2
U3
U4
users
...
U5
. . .I1 I2 I...
Basic form
Objective function
Two iterative approaches:
•Stochastic Gradient Descent
•Alternating Least Squares
Challenges
Scale
•100s of billions of (user, item) pairs
•Over billion users
•Tens of millions of items
Performance
•Train...
Training / testing metrics exampleRMSE
0
0.2
0.4
0.6
0.8
Iterations
0 4 8 12 16 20 24 28 32 36 40 44
Train f=8
Test f=8
Tr...
Apache Giraph
Iterative and graph processing on massive datasets
Billion vertices, trillion edges
Data mapped to a graph
•Vertex ids and...
What is Apache Giraph?
Runs on top of Hadoop
Map only jobs
Keeps data in memory
Mappers communicate through network
Giraph workflow
Worker 1
Worker 2
Worker 3
Distributing
Collaborative Filtering
Common approach
A bipartite graph:
•Users and items are vertices
•Known ratings are edges
•Feature vectors sent through ed...
Our solution - rotational approach
Worker 1
Worker 2
Worker 3
item
set 3
item
set 1
item
set 2
•Network traffic?
•Memory?
•...
Performance
Comparison with Spark MLlib
Spark MLlib ALS CF
•On scaled copies of Amazon reviews dataset
We can handle over 100 billion ...
Hybrid - common + rotational
Choose how to update item based on its degree
Network traffic per item:
•Common: #features * i...
Extensions
Slower connections can be a bottleneck
Solution: in every step send items between all workers
Rotating items
(#workers - 1) item sets on each worker
Decomposing complete graph into edge disjoint Hamilton cycles
Construction using L...
Social signals
Incorporate social network information - social regularization
User’s latent features should be similar to ...
Social signals
Easy to add in Giraph model
Additional complexity #friendships * #features
Solves cold start problem
Additional features
Tracking rmse, average rank and auc
Combining SGD & ALS
Different objective functions
•Implicit feedba...
Calculate item similarities based on:
•Common users
•Global item properties
Adjustable formulas for easy experimentation
I...
Applications
Use user and item embeddings in ranking models
Get user to item score in realtime
Direct user recommendations...
Conclusion
Conclusion
Scalable implementation of Collaborative Filtering
On top of Apache Giraph
Highly performant (100s of billion r...
Thank you!
tinyurl.com/fb-mf-cf
tinyurl.com/giraph-vldb-2015
Questions?
Upcoming SlideShare
Loading in …5
×

Maja Kabiljo, Software Engineer, Facebook Inc. at MLconf ATL - 9/18/15

890 views

Published on

Large-scale item recommendations with Apache Giraph – This is a joint work with Aleksandar Ilic, Facebook Inc.: Recommendation systems try to make personalized item recommendations to users based on available historical information. One of the well-known recommendation techniques is Collaborative Filtering – which is often solved with matrix factorization of a sparse user-item matrix of known ratings. In this talk, we will describe our scalable implementation of SGD and ALS methods for Collaborative Filtering on top of Apache Giraph (an iterative graph processing system built for high scalability on big data).

In order to scale our implementation to over a billion users and tens of millions of items, we developed novel methods for distributing the problem and added several extensions to the Giraph framework. Experiments show that our implementation is up to 10x faster than some of the leading open source implementations in this domain (e.g. Spark MLlib) on the Amazon benchmark data while maintaining the same output quality.

We will describe several additional techniques for handling Facebook’s data (e.g. implicit and skewed item data, different offline metrics) that are required in page and group recommendations. To complete our comprehensive approach for computing recommendations at Facebook, we also implemented an efficient method for finding top-k recommendations per user and item-based recommendations with pairwise item similarities that is easily extendable with different formulas.

Published in: Technology
  • Be the first to comment

Maja Kabiljo, Software Engineer, Facebook Inc. at MLconf ATL - 9/18/15

  1. 1. Maja Kabiljo & Aleksandar Ilic, Facebook Large scale item recommendations with Apache Giraph
  2. 2. Conclusion 04 01 02 05 03 Motivation and challenges Apache Giraph Distributing Collaborative Filtering Social signals and Applications
  3. 3. Collaborative Filtering
  4. 4. Collaborative Filtering Predict user’s interests based on many other users Disney Roller coasters Disneyland Six Flags
  5. 5. ? ? ? ? ? ? ? ? ? Matrix factorization 4 4 1 3 5 3 1 1 2 4 5 3 4 5 2 3 ... . . . ... U1 U2 U3 U4 users ... U5 . . .I1 I2 I3 I4 items I5 ?
  6. 6. Basic form Objective function Two iterative approaches: •Stochastic Gradient Descent •Alternating Least Squares
  7. 7. Challenges Scale •100s of billions of (user, item) pairs •Over billion users •Tens of millions of items Performance •Train models and iterate quickly •Use more features
  8. 8. Training / testing metrics exampleRMSE 0 0.2 0.4 0.6 0.8 Iterations 0 4 8 12 16 20 24 28 32 36 40 44 Train f=8 Test f=8 Train f=128 Test f=128
  9. 9. Apache Giraph
  10. 10. Iterative and graph processing on massive datasets Billion vertices, trillion edges Data mapped to a graph •Vertex ids and values •Edges and edge values What is Apache Giraph? 10 5 1 3 Neural networks Logistic regression Neural networks Boosted decision
  11. 11. What is Apache Giraph? Runs on top of Hadoop Map only jobs Keeps data in memory Mappers communicate through network
  12. 12. Giraph workflow Worker 1 Worker 2 Worker 3
  13. 13. Distributing Collaborative Filtering
  14. 14. Common approach A bipartite graph: •Users and items are vertices •Known ratings are edges •Feature vectors sent through edges Problems: •Data sent per iteration: #knownRatings * #features •Memory •Large degree items •SGD modifications are different than in the sequential solution Worker 1 Worker 2 Worker 3 I2 I1 I3 I4
  15. 15. Our solution - rotational approach Worker 1 Worker 2 Worker 3 item set 3 item set 1 item set 2 •Network traffic? •Memory? •Skewed item degrees? •SGD calculation? Users are vertices, items are worker data
  16. 16. Performance
  17. 17. Comparison with Spark MLlib Spark MLlib ALS CF •On scaled copies of Amazon reviews dataset We can handle over 100 billion ratings Cpuminutes 0 150 300 450 600 Millions examples 0 300 600 900 1200 Common approach (in Spark) Rotational (in Giraph)
  18. 18. Hybrid - common + rotational Choose how to update item based on its degree Network traffic per item: •Common: #features * itemDegree •Rotational SGD: #features * #workers •Rotational ALS: #features * #features * #workers
  19. 19. Extensions
  20. 20. Slower connections can be a bottleneck Solution: in every step send items between all workers Rotating items
  21. 21. (#workers - 1) item sets on each worker Decomposing complete graph into edge disjoint Hamilton cycles Construction using Latin squares Rotating items
  22. 22. Social signals Incorporate social network information - social regularization User’s latent features should be similar to his/her friends
  23. 23. Social signals Easy to add in Giraph model Additional complexity #friendships * #features Solves cold start problem
  24. 24. Additional features Tracking rmse, average rank and auc Combining SGD & ALS Different objective functions •Implicit feedback •Degree based regularization Incremental training Fast top K recommendations
  25. 25. Calculate item similarities based on: •Common users •Global item properties Adjustable formulas for easy experimentation Item similarities ? u u u u u u I1 I2 150M users 15M items 4B ratings 1.3B users 35M items 15B ratings 2.4B users 8M items 220B ratings Hive CPU hours 10 227 963 Giraph CPU hours 3 16 87 Sample datasets
  26. 26. Applications Use user and item embeddings in ranking models Get user to item score in realtime Direct user recommendations Context based recommendations
  27. 27. Conclusion
  28. 28. Conclusion Scalable implementation of Collaborative Filtering On top of Apache Giraph Highly performant (100s of billion ratings) Utilizing social signals and item similarities Many use cases at Facebook
  29. 29. Thank you! tinyurl.com/fb-mf-cf tinyurl.com/giraph-vldb-2015 Questions?

×