Slides for Shuo Chang's Recsys talk on the paper "Crowd-Based Personalized Natural Language Explanations for Recommendations".

1. Crowd-Based Personalized
Natural Language Explanations
for Recommendations
Shuo (Steven) Chang
Maxwell Harper
Loren Terveen
GroupLens
University of Minnesota
1

2. 2Background Introduction Method Experiment Result Discussion
How do recommenders explain
themselves?

3. 3Background Introduction Method Experiment Result Discussion

4. 4
Herlocker et. al. (2000) Explaining collaborative filtering recommendations.
Background Introduction Method Experiment Result Discussion

5. 5
Can we generate personalized natural
language explanations at scale?
Background Introduction Method Experiment Result Discussion

6. 6
Tintarev et. al. (2012) Evaluating the effectiveness of explanations
for recommender systems.
Vig. et. al (2008) Tagsplanations.
Background Introduction Method Experiment Result Discussion

7. 7
Can we do better than formulaic
explanations?
Background Introduction Method Experiment Result Discussion

8. 8Background Introduction Method Experiment Result Discussion

9. 9Background Introduction Method Experiment Result Discussion

10. 10
From your MovieLens profile it seems that you
prefer movies tagged as visual, Gravity is unlike
what we have seen on a cinema screen before and
arguably it has one of the best uses of 3D in a movie.
Background Introduction Method Experiment Result Discussion

11. 11
From your MovieLens profile it seems that you
prefer movies tagged as intense, the movie is a
pretty intense ninety minutes, with Bullock's
character constantly battling one catastrophe after
another, and all of it is amazing to see.
Background Introduction Method Experiment Result Discussion

12. Process Overview
12Background Introduction Method Experiment Result Discussion

13. 13Background Introduction Method Experiment Result Discussion

14. 14
From your MovieLens profile it seems that you prefer movies
tagged as [Topical Dimension], [Natural Language Explanation].
1. Model topics of items
2. Generate explanations
3. Model user interest and present matching
explanation
Background Introduction Method Experiment Result Discussion

15. 15
Model topics of items
1
Background Introduction Method Experiment Result Discussion
Crowd recruited from Amazon Mechanical Turk

16. 16
Top 20 most relevant tags
Background Introduction Method Experiment Result Discussion
mafia, gangster, gangsters,
mob, crime, mentor
violent, narrated, violence, stylish,
visceral, stylized, bloody, brutality
masterpiece, storytelling, drama,
dialogue
interesting, original

17. 17
Build semantic similarity
graph (word2vec trained on
IMDB reviews)
Clustering
(Affinity
Propagation)
Input to crowd
Background Introduction Method Experiment Result Discussion

18. 18
Crowds refine clusters
Background Introduction Method Experiment Result Discussion

19. 19
1.Filter tags
2.Pick representative tag
Background Introduction Method Experiment Result Discussion

20. 20Background Introduction Method Experiment Result Discussion
mafia, gangster, gangsters,
mob, crime, mentor
violent, narrated, violence, stylish,
visceral, stylized, bloody, brutality
masterpiece, storytelling, drama,
dialogue
interesting, original

21. 21
mafia, gangster, gangsters,
mob, crime, mentor
violent, narrated, violence, stylish,
visceral, stylized, bloody, brutality
masterpiece, storytelling, drama,
dialogue
interesting, original
Background Introduction Method Experiment Result Discussion

22. 22
Generate explanations
2
Background Introduction Method Experiment Result Discussion

23. 23
Quotes from
IMDB reviews
mafia, gangster, gangsters,
mob, crime, mentor
violent, narrated, violence, stylish,
visceral, stylized, bloody, brutality
masterpiece, storytelling, drama,
dialogue
interesting, original
Background Introduction Method Experiment Result Discussion

24. 24
Map
Background Introduction Method Experiment Result Discussion

25. 25
Reduce
Background Introduction Method Experiment Result Discussion

26. 26
Model user interest and present
matching explanation
3
Background Introduction Method Experiment Result Discussion

27. User Experiment
27Background Introduction Method Experiment Result Discussion

28. 28
• Generate natural language explanations
for 100 movies with $3.90/movie
• Survey 216 MovieLens users
Background Introduction Method Experiment Result Discussion

29. 29
• Within subject design
• Two random unseen movies
• One with baseline and another with
natural language explanation
Background Introduction Method Experiment Result Discussion

30. 30
• Baseline: user preferred relevant topics
We recommend the movie because you like the
following features: [tag1, ..., tag5]
Vig. et. al (2008) Tagsplanations.
Background Introduction Method Experiment Result Discussion

31. Result
31Background Introduction Method Experiment Result Discussion

32. 32
Users are more satisfied with
natural language explanations
1
Background Introduction Method Experiment Result Discussion

33. 33
15%
25%
66%
51%
19%
25%
13%
22%
76%
61%
12%
17%
19%
26%
68%
51%
13%
23%
I wish MovieLens included explanations like this.
The explanation is easy to understand.
The explanation is useful.
CROWD
TAG
CROWD
TAG
CROWD
TAG
100 50 0 50 100
Percentage
Strongly disagree Disagree Neutral Agree Strongly agree
Background Introduction Method Experiment Result Discussion

34. 34
Users are have more trust in
natural language explanations
2
Background Introduction Method Experiment Result Discussion

35. 35
15%
17%
66%
57%
19%
26%
24%
22%
48%
42%
28%
36%
I trust the explanation.
The explanation reflects my preferences about this movie.
CROWD
TAG
CROWD
TAG
100 50 0 50 100
Percentage
Strongly disagree Disagree Neutral Agree Strongly agree
Background Introduction Method Experiment Result Discussion

36. 36
Users perceive natural language
explanations to contain more
appropriate amount of information
3
Background Introduction Method Experiment Result Discussion

37. 37
34%
55%
45%
19%
21%
26%
The explanation contains right amount of information.
CROWD
TAG
100 50 0 50 100
Percentage
Strongly disagree Disagree Neutral Agree Strongly agree
4%
6%
63%
55%
32%
39%
Changes in response regarding knowledge about a movie
CROWD
TAG
100 50 0 50 100
Percentage
Response −2 −1 0 1 2 3 4
Background Introduction Method Experiment Result Discussion

38. 38
4
Little difference in helping users
make decision
Background Introduction Method Experiment Result Discussion

39. 39
Discussion
Background Introduction Method Experiment Result Discussion

40. 40Background Introduction Method Experiment Result Discussion
Mixed computation

41. 41
https://www.google.com/url?sa=i&rct=j&q=&esrc=s&source=images&cd=&cad=rja&uact=8&ved=0ahUKEwi34q-j6bTOAhUE2mMKHdnfBcEQjRwIBw&url=https%3A%2F%2Fwww.engadget.com%2F2014%2F09%2
recognition-tech%2F&bvm=bv.129389765,d.cGc&psig=AFQjCNEWUMTMw249fiU1MrdLKiztXM_KOA&ust=1470848879786441
Background Introduction Method Experiment Result Discussion

42. 42Background Introduction Method Experiment Result Discussion

43. 43Background Introduction Method Experiment Result Discussion

44. 44
Takeaways
Background Introduction Method Experiment Result Discussion

45. 45
• Mixed computation approach
• Human effort to
- Refine item topic clusters
- Synthesize review quotes
• Better user experience than tag based
explanations
Background Introduction Method Experiment Result Discussion

46. 46
Questions?
Shuo (Steven) Chang @ Quora
schang@cs.umn.edu
http://www-users.cs.umn.edu/~schang
Crowd-Based Personalized Natural Language Explanations for
Recommendations

47. 47Intro New user Model topics Recommend Explain Discussion
mafia, gangster, gangsters,
mob, crime, mentor
violent, narrated, violence, stylish,
visceral, stylized, bloody, brutality
masterpiece, storytelling, drama,
dialogue
interesting, original
Quotes about
drama masterpiece, story-
telling, dialogue :
As much as the true events of Henry’s life have more
than likely been dramatised and glamourised to a
certain extent, the essence of this film IMO is that it is
still a brilliantly damning portrayal of the characters
and lifestyle of mobsters.
The consistently fine acting by the large ensemble cast
(both known and unknown), the cinematography,
editing, dialogue, brilliant use of music, it’s all
breathtaking.
The dialogue is incredible.
Storytelling with impeccable pacing, this is what it’s
like when a master composer conducts his masterpiece.
If ever the word ‘masterpiece’was meant to be used, it
was for this film. ‘Goodfellas’is a masterpiece, pure
and simple.

48. 48Background Motivation Expt_1 Expt_2 Discussion