By Mr. Abhinit Kumar Ambastha, Adjunct Lecturer
Predict what your customer wants before they even know it! AI is a powerful game changer. Companies like IBM, Facebook, Microsoft and now Google are investing heavily in AI. Pattern recognition and visual search are becoming new norms in interacting with customers. Recommendation systems provide intuition-based recommendations that can influence and boost customer engagement in ways not previously possible. You’ll learn how to leverage customer profiles based on Facebook and Twitter data to generate recommendations using the IBM Watson.
12. #ISSlearn#ISSlearn
Machine learning and AI
❏ AI gives computers the ability to learn
without explicit programming or
teaching
❏ Data is equivalent to experience
❏ Programming is deterministic,
machine learning is non-deterministic
❏ Knowledge is stored in connection
weights
❏ Teaching is hard!
12
18. #ISSlearn#ISSlearn
❏ Purpose?
❏ Automation
❏ Scaling
❏ Better performance
❏ Aid and assistance
❏ Google pagerank
❏ Facebook friends and ads
❏ Amazon!
❏ Taobao
Who’s doing it? And why?
18
19. #ISSlearn#ISSlearn
❏ Scarcity to abundance
❏ There are a few items which are
extremely desirable but a majority
of items are sold in small
quantities
❏ Long tail concept!
❏ Netflix challenge
❏ Amazon recommendations
❏ LinkedIn
❏ Facebook friends
❏ Youtube ads
❏ The idea of discovering an
underdog!
Recommendation systems
and businesses
19
20. #ISSlearn#ISSlearn
❏ Large catalog of items
❏ A system which provides you options based on your preferences
❏ It’s an alternate to the search methodology
Recommendation systems
Items...
Search
R
ecom
m
end
20
21. #ISSlearn#ISSlearn
Exempli gratia!
❏ Probably one of the largest
recommendation systems!
❏ Data: facebook profiles, Instagram
pictures, demographics, preferences, chat
conversations, hopes, dreams, your soul
❏ Goal: Recommend possible date(s),
disappointing relationships, lower
self-esteem
❏ Methods:
❏ Clustering
❏ Semi-supervised
❏ Reinforcement learning
21
23. #ISSlearn#ISSlearn
❏ Rock star recommendations
❏ Bestseller
❏ Top movie of the month, etc.
❏ Curated lists
❏ Top 50 songs worldwide on Spotify
❏ Top 10 movie lists
❏ Mostly hand curated or from user activity
❏ Personalised recommendations
❏ Amazon, Netflix, Facebook, Spotify
Generating recommendation
23
24. #ISSlearn#ISSlearn
❏ C = Set of customers
❏ P = Set of products
❏ R = Rating of products for the user
❏ u = Utility table which shows the rating for every product for
each user
Representation
P1
P2
P3
P4
C1
R11
...
C2
...
24
25. #ISSlearn#ISSlearn
❏ Simplest method for proposing recommendations
❏ Classify an item as “like” or “dislike” for a given user
❏ Requires a lot of data for each user
❏ Not practical
Classification based recommendation
25
26. #ISSlearn#ISSlearn
❏ Works based on item to item similarity
❏ Create a product profile for every product based on its attributes:
Content-based filtering
Actor 1 1
Actor 2 1
Genre 1 0
Action rating 3.5
or [1,1,0,3.5]
❏ User profile = Average of product profile values
❏ We calculate the distance between a user and product
❏ Usually cosine distance
26
27. #ISSlearn#ISSlearn
❏ Pros:
❏ No need of data of other users
❏ Able to recommend users with unique taste
❏ Able to recommend products with no previous ratings
❏ Explanations for recommended items!
❏ Cons:
❏ Hand crafted attributes! (images? Music? movies?)
❏ Only recommends items inside user’s content profile
Content-based filtering
27
29. #ISSlearn#ISSlearn
Collaborative filtering
❏ Works based on user to user similarity
❏ Create a user profile based on ratings, user details, etc.
❏ Recommend movies preferred by similar users
1 2 3 5 2
4 3 5
5 4 ?
29
30. #ISSlearn#ISSlearn
❏ Latent semantic analysis
❏ User similarity measure can be derived from external sources
❏ Product interest can be registered to make better associations
Collaborative filtering
LDA
30
31. #ISSlearn#ISSlearn
❏ Pros:
❏ Captures user similarities, which can capture some latent attributes
❏ Can recommend new products outside user’s content profile
❏ Cons:
❏ Requires a user base and user activity base
❏ Doesn’t work well for new users
Collaborative filtering
31
32. #ISSlearn#ISSlearn
❏ Possible for image based
recommendations
❏ Salient associations between
items
❏ Items need not be
morphologically similar
❏ Products which are
mentioned together, can be
considered associated
Association mining
32
33. #ISSlearn#ISSlearn
❏ Siamese networks to derive associations
❏ Automated profiling or feature selection
❏ Latent associations can be realized
❏ Mostly limited to visual associations
❏ Still in academics!
Deep learning
33
34. #ISSlearn#ISSlearn
❏ Root mean square error metric (RMSE): Absolute distance between
the predicted and actual similarity rating
❏ Hidden ratings to create test set
❏ Can only be verified in a supervised manner!
❏ True performance evaluation can be captured only by testing
Evaluation of recommendation
systems
34
36. #ISSlearn#ISSlearn
❏ Highly sparse dataset
❏ Large amounts of offline calculations
❏ Product profile creation
❏ Lot of stress on engineering section
Large databases, bigger problems
36
37. #ISSlearn#ISSlearn
❏ Use multiple algorithms:
❏ Singular vector
decomposition (SVD)
❏ Deep neural nets
❏ Logistic and linear
regression
❏ ...
❏ Offline processing to train
models
❏ Nearline to use new data to
improve offline models
❏ Online computation to generate
recommendations based on user
events
Netflix recommendation systems
37
38. #ISSlearn#ISSlearn
❏ Offline stage uses complex
algorithms
❏ Nearline uses comparatively
simple algorithms
❏ A/B testing on subset!
❏ Netflix prize: 1 million $ for
10% improvement in their
recommendation system
❏ Never used the algorithm!
Netflix recommendation systems
38
41. #ISSlearn#ISSlearn
Linkedin recommendation system
❏ Has structured user data
❏ Multiple events used to calculate co-interests
❏ Co-view
❏ Skill rating
❏ Connections distance
❏ Bag of models vs. blended models
❏ Engineering driven!
41
43. #ISSlearn#ISSlearn
Pricing strategy using AI
43
❏ Right pricing = more customers
❏ Uber pricing for routes
❏ Insurance companies like AXA
and Aviva
❏ Static pricing vs. variable
pricing?
❏ E-commerce bargaining?
45. #ISSlearn#ISSlearn
Pricing strategy using AI
45
❏ Modelling customer
purchasing behaviour
❏ Modelling Product flow
behaviour
❏ Price suggestions!
Customer behaviors,
influencing factors
Data science Machine learning Dashboard
46. #ISSlearn#ISSlearn
Pricing strategy using AI
46
❏ Dynamic pricing per user
❏ Multiple products testing
Customer behaviors,
influencing factors
Data science Machine learning Dashboard
49. #ISSlearn#ISSlearn
Product development using AI
49
❏ Product design for different
mediums
❏ Product design for more
engagement
❏ Low hanging fruits:
❏ Poster designs
❏ UX / UI designs
❏ Tough cookie:
❏ Product trends
❏ Design principles
50. #ISSlearn#ISSlearn
Product development using AI
50
❏ Web design being automated
❏ Automated A/B testing and analysis
❏ UI placements and image content aware positioning
❏ Deep learning and complex data
❏ AI and architecture
54. #ISSlearn#ISSlearn
Think data
54
❏ Play with libraries (Python):
❏ Sci-kit learn
❏ Pandas
❏ Numpy
❏ “An average data scientist can hire an excellent data scientist”
