Mark Graus gave a presentation on recommender systems and the importance of considering the human factor. He discussed how recommender systems use machine learning algorithms like collaborative filtering and content-based filtering to make predictions, but that machine learning alone is not enough. User behavior data from recommender systems is limited and does not capture things like user preferences, privacy concerns, or choice overload. It is important to conduct A/B testing and user experience evaluations to understand how users actually interact with and feel about recommendation systems. The key takeaway is that recommender systems require both strong machine learning and a focus on the human perspective.

1. RecommenderSystems
and the Human Factor
Mark Graus
Netherlands Machine Learning Meetup
2016/03/16

2. I’m
50% Machine
Learner
50%
Psychologist

3. I’ve been
working with
‘recommender
systems’ since
2009
 Movie Recommender Systems
 Website
 Personalization
 App
 Personalization
200920112016

4. Content
 WhatAre Recommender Systems
 Why Machine Learning is not Enough

5. What are
Recommender
Systems?

7. The Machine
Learning
Behind
Recommender
Systems
 We use historical item-user data to predict
unobserved item-user data
 Typically big datasets
 i.e. billions of observations
 millions of users
 tons of items
 Numerous Specifically Designed Algorithms

8. How I see
Recommender
Algorithms
Implicit Feedback Explicit Feedback
Collaborative
Content-Based

9. Distinction 1:
Implicit versus
Explicit
Feedback
Implicit
My actual behavior
 watching
 skipping/stopping
Explicit
The feedback I give
 star rating

10. Distinction 1:
Considerations
“Oh no! MyTiVo thinks I’m gay”
Jeffrey Zaslow,TheWall Street Journal, December 2002
What I Like versus What I Say I Like
Solution: Use a bit of both implicit and explicit

11. Distinction 2:
Content-Based
versus
Collaborative
Filtering
 Supervised learning
 Features are extracted from ‘metadata’
 Target variable is rating (explicit) or whether the movie will be
watched (implicit)
Genre Director Main
Actor
Year Rating
The Usual
Suspects
Crime Bryan
Singer
Kevin
Spacey
1995
Titanic Drama James
Cameron
Leonardo
DiCaprio
1997
Die Hard Action John
McTiernan
Bruce
Willis
1988
?

12. Distinction 2:
Content-Based
versus
Collaborative
Filtering
KNN,SlopeOne
?
?

13. Matrix
Factorization
butalso FunkSVD,
SVD+
UsualSuspects
Titanic
DieHard
TheGodfather
Jack
Dylan
Olivia
Mark
?
?
?
? ? ?
?
 Dimensionality Reduction

14. Matrix
Factorization
butalso FunkSVD,
SVD+
Jack
Mark
Olivia Dylan

15. Content-Based
versus
Collaborative
Considerations
 Metadata availability
 Need for explaining

16. MyApproach
 Start with Open Source Software
 Lenskit (Java)
 MyMediaLite (C#)
 Mahout (Python)
 Learn about Recommender Systems and User Base
 Scale Up
 Cassandra
 Akka

17. State-of-the-
Art
 We can do predictions really well
 Challenges
 Cold Start Problem
 Context-Aware Recommendations
 Social Recommendations
 “Merged accounts”

18. Why Machine Learning
is Not Enough

19. Recommender
System Data is
Observable
Behavior
Recommendations
Behavior
Recommender
System
User
Experience

20. Examples of
Things Data
CannotTellUs
 Do I feel my privacy invaded?
 Am I happy to have American Pie 2 recommended?
 Why do people react to recommendations the way they do?
 Presentation?
 Bad Recommendations?
 Choice Overload?

21. We need to do
A/B testing
andUX
measurement
System A System B

22. What did we
learn from
surveys?
 Satisfaction =
Recommendation Set
Attractiveness - Choice
Difficulty
 More views != Satisfaction
 Diversity influences
Satisfaction
 Long Lists = Difficult to
Choose
 Short Lists = Easier to
Choose, but not enough
choice
 Right Balance = Short
Lists of Diverse Items

23. Take Home
Message
 The Machine Learning is just the beginning of Recommender
Systems

24. Thank you for
listening!
Some Pointers
 Recommender Algorithms
 Yehuda Koren, Google
 Introduction to Recommender Systems,Coursera/GroupLens
 Infrastructure
 NetflixTech Blog
 A/BTesting
 Ron Kohavi, Microsoft Research
 User Experience Evaluation in Recommender Systems
 Bart Knijnenburg, Clemson University