Cetas Presentation on Real-time Recommendation Systems

Real-Time Recommender SystemsBay Area Search Meetup at eBayApril 25, 2012 Balu Rajagopal

Goal of Recommenders INSTANT INTELLIGENCE 1. Increase number of items sold 2. Cross-Sell, Up-Sell diverse items 3. Increase Customer Satisfaction 4. Build Loyalty 5. Improve User ExperienceCetas Software Inc. – Copyright © 2012– CONFIDENTIAL – DO NOT DISTRIBUTE 2

Two Challenges INSTANT INTELLIGENCE  Make a Personalized Recommendation – Multi-Dimensional Data – Streams: Social, Activity, Apps, Tweets, Actions, … – Demographic – Temporal, Spatial  Do it in real-time – Query to Analysis to Visualization – User Experience (UX) – System Constraints – Network, Capacity, SLACetas Software Inc. – Copyright © 2012– CONFIDENTIAL – DO NOT DISTRIBUTE 4

Problem Space INSTANT INTELLIGENCE Cetas Instant Intelligence Framework Secs or Less Large RESPONSE TIME TO USERDATA DIMENSIONS Minutes Medium Hours Small Gigabytes Terabytes Petabytes ANALYSIS VOLUME Cetas Software Inc. – Copyright © 2012– CONFIDENTIAL – DO NOT DISTRIBUTE 5

Real-time Recommender System INSTANT INTELLIGENCE Inputs Terabytes of Multi-Dimensional data Preprocessing Reduction @ Scale @ Speed Analysis Classifying, Clustering Output Prediction, RecommendationCetas Software Inc. – Copyright © 2012– CONFIDENTIAL – DO NOT DISTRIBUTE 6

Real-time Recommender System INSTANT INTELLIGENCE • Spatial Inputs • Temporal • Demographic • Personal • Psychographic • Behavioral Preprocessing Reduction Analysis Classifying, Clustering Output Predictions, Recommendations, PatternsCetas Software Inc. – Copyright © 2012– CONFIDENTIAL – DO NOT DISTRIBUTE 7

Real-time Recommender System INSTANT INTELLIGENCE • Spatial Inputs • Temporal • Demographic • Personal • Psychographic • Behavioral • Distance Measures Preprocessing • Sampling • PCA • Dimensionality Reduction • SVD Analysis Classifying, Clustering Output Predictions, Recommendations, PatternsCetas Software Inc. – Copyright © 2012– CONFIDENTIAL – DO NOT DISTRIBUTE 8

Real-time Recommender System INSTANT INTELLIGENCE • Spatial Inputs • Temporal • Demographic • Personal • Psychographic • Behavioral • Distance Measures Preprocessing • Sampling • PCA • Dimensionality Reduction • SVD • Predictors • Classification Analysis • Descriptors • Association • Clustering OutputCetas Software Inc. – Copyright © 2012– CONFIDENTIAL – DO NOT DISTRIBUTE 9

Real-time Recommender System INSTANT INTELLIGENCE • Spatial Inputs • Temporal • Demographic • Personal • Psychographic • Behavioral • Distance Measures Preprocessing • Sampling • PCA • Dimensionality Reduction • SVD • Predictors • Classification Analysis • Descriptors • Association • Clustering • Predictions Output • Recommendations • PatternsCetas Software Inc. – Copyright © 2012– CONFIDENTIAL – DO NOT DISTRIBUTE 10

Big Data Analytics – eCommerce INSTANT INTELLIGENCE Input data Clustering Closed-loop Action User transactions live stream Product placement decision Demographics data stream Category, sub- category sorting Online app events stream New product Ad placement offering stream Other streams … Other actions …Cetas Software Inc. – Copyright © 2012– CONFIDENTIAL – DO NOT DISTRIBUTE 11

Real-time Stream Processing INSTANT INTELLIGENCE Billions of Events I n d e x CEP RAM Cache Joins RAM Disk Aggregates HBase HDFSCetas Software Inc. – Copyright © 2012– CONFIDENTIAL – DO NOT DISTRIBUTE 12

Wrap-up INSTANT INTELLIGENCE  Personalized Recommendation Engine – Non-trivial – Focus on Specific Use Case  Real-time – Distributed Indexing – Pre-computation – Compact store (in memory, on disk) – ParallelizationCetas Software Inc. – Copyright © 2012– CONFIDENTIAL – DO NOT DISTRIBUTE 13

References INSTANT INTELLIGENCE  Mining Massive Datasets – Free eBook – Anand Rajaraman, Jeff Ullman – cs246.stanford.edu  Introduction to Data Mining – Tan, Steinback, Kumar  Introduction to Recommender Systems Handbook – Ricci, Rokach, ShapiraCetas Software Inc. – Copyright © 2012– CONFIDENTIAL – DO NOT DISTRIBUTE 14

http://www.scoop.it	36
http://www.linkedin.com	7
https://twitter.com	2

Cetas Presentation on Real-time Recommendation Systems

by Pivotal Analytics (Cetas Analytics)

Accessibility

Categories

Upload Details

Usage Rights

3 Embeds 45

Statistics