Cetas Presentation on Real-time Recommendation Systems

2,674 views

Published on

Real-time Recommender systems

Published in: Technology

Cetas Presentation on Real-time Recommendation Systems

  1. 1. Real-Time Recommender SystemsBay Area Search Meetup at eBayApril 25, 2012 Balu Rajagopal
  2. 2. Goal of Recommenders INSTANT INTELLIGENCE 1. Increase number of items sold 2. Cross-Sell, Up-Sell diverse items 3. Increase Customer Satisfaction 4. Build Loyalty 5. Improve User ExperienceCetas Software Inc. – Copyright © 2012– CONFIDENTIAL – DO NOT DISTRIBUTE 2
  3. 3. Recommendations INSTANT INTELLIGENCE USERS Search Recommendations Products Web sites Social networks ITEMS Blogs News ….Cetas Software Inc. – Copyright © 2012– CONFIDENTIAL – DO NOT DISTRIBUTE 3
  4. 4. Two Challenges INSTANT INTELLIGENCE  Make a Personalized Recommendation – Multi-Dimensional Data – Streams: Social, Activity, Apps, Tweets, Actions, … – Demographic – Temporal, Spatial  Do it in real-time – Query to Analysis to Visualization – User Experience (UX) – System Constraints – Network, Capacity, SLACetas Software Inc. – Copyright © 2012– CONFIDENTIAL – DO NOT DISTRIBUTE 4
  5. 5. Problem Space INSTANT INTELLIGENCE Cetas Instant Intelligence Framework Secs or Less Large RESPONSE TIME TO USERDATA DIMENSIONS Minutes Medium Hours Small Gigabytes Terabytes Petabytes ANALYSIS VOLUME Cetas Software Inc. – Copyright © 2012– CONFIDENTIAL – DO NOT DISTRIBUTE 5
  6. 6. Real-time Recommender System INSTANT INTELLIGENCE Inputs Terabytes of Multi-Dimensional data Preprocessing Reduction @ Scale @ Speed Analysis Classifying, Clustering Output Prediction, RecommendationCetas Software Inc. – Copyright © 2012– CONFIDENTIAL – DO NOT DISTRIBUTE 6
  7. 7. Real-time Recommender System INSTANT INTELLIGENCE • Spatial Inputs • Temporal • Demographic • Personal • Psychographic • Behavioral Preprocessing Reduction Analysis Classifying, Clustering Output Predictions, Recommendations, PatternsCetas Software Inc. – Copyright © 2012– CONFIDENTIAL – DO NOT DISTRIBUTE 7
  8. 8. Real-time Recommender System INSTANT INTELLIGENCE • Spatial Inputs • Temporal • Demographic • Personal • Psychographic • Behavioral • Distance Measures Preprocessing • Sampling • PCA • Dimensionality Reduction • SVD Analysis Classifying, Clustering Output Predictions, Recommendations, PatternsCetas Software Inc. – Copyright © 2012– CONFIDENTIAL – DO NOT DISTRIBUTE 8
  9. 9. Real-time Recommender System INSTANT INTELLIGENCE • Spatial Inputs • Temporal • Demographic • Personal • Psychographic • Behavioral • Distance Measures Preprocessing • Sampling • PCA • Dimensionality Reduction • SVD • Predictors • Classification Analysis • Descriptors • Association • Clustering OutputCetas Software Inc. – Copyright © 2012– CONFIDENTIAL – DO NOT DISTRIBUTE 9
  10. 10. Real-time Recommender System INSTANT INTELLIGENCE • Spatial Inputs • Temporal • Demographic • Personal • Psychographic • Behavioral • Distance Measures Preprocessing • Sampling • PCA • Dimensionality Reduction • SVD • Predictors • Classification Analysis • Descriptors • Association • Clustering • Predictions Output • Recommendations • PatternsCetas Software Inc. – Copyright © 2012– CONFIDENTIAL – DO NOT DISTRIBUTE 10
  11. 11. Big Data Analytics – eCommerce INSTANT INTELLIGENCE Input data Clustering Closed-loop Action User transactions live stream Product placement decision Demographics data stream Category, sub- category sorting Online app events stream New product Ad placement offering stream Other streams … Other actions …Cetas Software Inc. – Copyright © 2012– CONFIDENTIAL – DO NOT DISTRIBUTE 11
  12. 12. Real-time Stream Processing INSTANT INTELLIGENCE Billions of Events I n d e x CEP RAM Cache Joins RAM Disk Aggregates HBase HDFSCetas Software Inc. – Copyright © 2012– CONFIDENTIAL – DO NOT DISTRIBUTE 12
  13. 13. Wrap-up INSTANT INTELLIGENCE  Personalized Recommendation Engine – Non-trivial – Focus on Specific Use Case  Real-time – Distributed Indexing – Pre-computation – Compact store (in memory, on disk) – ParallelizationCetas Software Inc. – Copyright © 2012– CONFIDENTIAL – DO NOT DISTRIBUTE 13
  14. 14. References INSTANT INTELLIGENCE  Mining Massive Datasets – Free eBook – Anand Rajaraman, Jeff Ullman – cs246.stanford.edu  Introduction to Data Mining – Tan, Steinback, Kumar  Introduction to Recommender Systems Handbook – Ricci, Rokach, ShapiraCetas Software Inc. – Copyright © 2012– CONFIDENTIAL – DO NOT DISTRIBUTE 14
  15. 15. INSTANT INTELLIGENCECetas Software Inc. – Copyright © 2012– CONFIDENTIAL – DO NOT DISTRIBUTE 15

×