Your SlideShare is downloading. ×
Affect in recommender systems
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

Affect in recommender systems

527
views

Published on

Published in: Technology

0 Comments
2 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
527
On Slideshare
0
From Embeds
0
Number of Embeds
4
Actions
Shares
0
Downloads
14
Comments
0
Likes
2
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. Affect in recommender systems Marko Tkalčič University of LjubljanaUniverza v Ljubljani ..: Fakulteta za elektrotehniko:..[LDOS] ..: Laboratorij za digitalno obdelavo signalov, slik in videa:..
  • 2. Univerza v Ljubljani ..: Fakulteta za elektrotehniko:.. [LDOS] ..: Laboratorij za digitalno obdelavo signalov, slik in videa:.. Presentation overview I: LDOS presentation & Motivation II: What are emotions? III: Emotion in recsys – related work IV: Role of emotions in the MM consumption chain V: Affect in the decision-making stage Conclusions Note: some material is not ours ... Fair use ...
  • 3. Univerza v Ljubljani ..: Fakulteta za elektrotehniko:.. [LDOS] ..: Laboratorij za digitalno obdelavo signalov, slik in videa:..Part I: LDOS group at UL FE and underlying assumption
  • 4. Univerza v Ljubljani ..: Fakulteta za elektrotehniko:.. [LDOS] ..: Laboratorij za digitalno obdelavo signalov, slik in videa:.. LDOS group at UL FE University of Ljubljana – Faculty of electrical engineering • LDOS (Digital signal processing laboratory) – Approx 15 members Relevant people Head: prof. Jurij Tasič Andrej Košir Marko Tkalčič Ante Odić Matevž Kunaver Tomaž Požrl
  • 5. Univerza v Ljubljani ..: Fakulteta za elektrotehniko:.. [LDOS] ..: Laboratorij za digitalno obdelavo signalov, slik in videa:.. LDOS work on recommender systems 2002-2009: public movie datasets – CBR Basic RecSys – CF 2009-2012 – Emotions Affective – Context Computing 2012 – Affective – Decision making (affective + cognitive attributes) RecSys • Ajzen model • Kahneman/Tversky model • ... Decision making Modeling in RecSys
  • 6. Univerza v Ljubljani ..: Fakulteta za elektrotehniko:.. [LDOS] ..: Laboratorij za digitalno obdelavo signalov, slik in videa:.. Underlying presumption of our work Recommender System = predictor of users‘ decision making Decision making: EMOTIONS DO INFLUENCE (c) Dilbert.com
  • 7. Univerza v Ljubljani ..: Fakulteta za elektrotehniko:.. [LDOS] ..: Laboratorij za digitalno obdelavo signalov, slik in videa:..PART II : What is affect/emotions/mood/personality
  • 8. Univerza v Ljubljani ..: Fakulteta za elektrotehniko:.. [LDOS] ..: Laboratorij za digitalno obdelavo signalov, slik in videa:.. PART II : What are affect/emotions/mood/personality Not well defined (wikipedia): Psychophysiological expressions – Emotion = subjective, conscious experience Biological reactions – Affect = experience of emotion (interchangable) Mental states – Emotion vs. Mood: • Emotion = high arousal, short term • Mood = low arousal, long term – Personality = accounts for the individual differences in the users’ emotional, interpersonal, experiential, attitudinal and motivational styles [John and Srivastava, 1999] CHANGES FIXED emotion mood personality Time (duration)
  • 9. Univerza v Ljubljani ..: Fakulteta za elektrotehniko:.. [LDOS] ..: Laboratorij za digitalno obdelavo signalov, slik in videa:.. Overview of emotions/moods Several definitions We take simple models, easy to incorporate in computers: – Basic emotions – Dimensional model – Circumplex model
  • 10. Univerza v Ljubljani ..: Fakulteta za elektrotehniko:.. [LDOS] ..: Laboratorij za digitalno obdelavo signalov, slik in videa:.. Basic emotions Discrete classes model Different sets Charles Darwin: Expression of emotions in man and animal Paul Ekman definition (6 + neutral): – Happiness – Anger – Fear – Sadness – Disgust – Surprise (c) Paul Ekman
  • 11. Univerza v Ljubljani ..: Fakulteta za elektrotehniko:.. [LDOS] ..: Laboratorij za digitalno obdelavo signalov, slik in videa:.. Basic emotions Discrete classes model Different sets Charles Darwin: Expression of emotions in man and animal Paul Ekman definition (6 + neutral): – Happiness – Anger – Fear – Sadness – Disgust – Surprise (c) Paul Ekman
  • 12. Univerza v Ljubljani ..: Fakulteta za elektrotehniko:.. [LDOS] ..: Laboratorij za digitalno obdelavo signalov, slik in videa:.. Dimensional model Three dimensions – Valence (positive vs. Negative) – Arousal (high vs. Low) – Dominance (power(less) over emotions)
  • 13. Univerza v Ljubljani ..: Fakulteta za elektrotehniko:.. [LDOS] ..: Laboratorij za digitalno obdelavo signalov, slik in videa:.. Dimensional model Three dimensions – Valence – Arousal – Dominance – (c) Lang, P. J. (1980) Each emotive state is a point in the VAD space
  • 14. Univerza v Ljubljani ..: Fakulteta za elektrotehniko:.. [LDOS] ..: Laboratorij za digitalno obdelavo signalov, slik in videa:.. Circumplex model Maps basic emotions dimensional model (Posner et al.) Arousal high joy anger surprise disgust fear Valence neutral negative positive sadness low
  • 15. Univerza v Ljubljani ..: Fakulteta za elektrotehniko:.. [LDOS] ..: Laboratorij za digitalno obdelavo signalov, slik in videa:.. How to detect emotions Emotions are characterized: – psychophysiological expressions, – biological reactions – mental states SENSORS !!!
  • 16. Univerza v Ljubljani ..: Fakulteta za elektrotehniko:.. [LDOS] ..: Laboratorij za digitalno obdelavo signalov, slik in videa:.. How to detect emotions? Explicit vs. Implicit Explicit – Questionnaires (SAM)
  • 17. Univerza v Ljubljani ..: Fakulteta za elektrotehniko:.. [LDOS] ..: Laboratorij za digitalno obdelavo signalov, slik in videa:.. How to detect emotions? Explicit vs. Implicit Explicit – Questionnaires (SAM) Implicit: – Work done in the affective computing community – Different modalities (sources): • Facial actions (video) • Physiological signals ( GSR, EEG) • Voice • Posture • ... – ML techniques • Classification (basic emotions) • Regression (dimensional model)
  • 18. Univerza v Ljubljani ..: Fakulteta za elektrotehniko:.. [LDOS] ..: Laboratorij za digitalno obdelavo signalov, slik in videa:.. Lie To Me Part1.avi (c) 20th Century Fox Main Character Cal Lightman = Paul Ekman Defined the FACS (Facial Action Coding System)
  • 19. Univerza v Ljubljani ..: Fakulteta za elektrotehniko:.. [LDOS] ..: Laboratorij za digitalno obdelavo signalov, slik in videa:.. LDOS Experiment 2 datasets: – Posed (Cohn-Kanade dataset) – Spontaneous (LDOS-PerAff-1 dataset) Input: Video streams of facial expressions as responses to visual stimuli Output: emotive states as distinct classes Gabor features kNN Emotive state
  • 20. Univerza v Ljubljani ..: Fakulteta za elektrotehniko:.. [LDOS] ..: Laboratorij za digitalno obdelavo signalov, slik in videa:.. Results and conclusions Posed dataset: accuracy = 92 % Spontaneous dataset: accuracy = 62% Reasons for bad results: – Weak learning supervision – Non optimal video acquisition (face rotation, occlusions, changing lightning ...) – Non extreme facial expressions Upcoming paper: IEEE Transactions on Multimedia
  • 21. Univerza v Ljubljani ..: Fakulteta za elektrotehniko:.. [LDOS] ..: Laboratorij za digitalno obdelavo signalov, slik in videa:.. Personality Definition:  User personality accounts for the individual differences in the users’ emotional, interpersonal, experiential, attitudinal and motivational styles [John and Srivastava, 1999]  Ever-lasting Several models  Five Factor Model (FFM or Big5):  Openness (inventive/curious vs. consistent/cautious)  Conscientiousness (efficient/organized vs. easy-going/careless)  Extraversion (outgoing/energetic vs. solitary/reserved)  Agreeableness (friendly/compassionate vs. cold/unkind)  Neuroticism (sensitive/nervous vs. secure/confident) How to measure?  Questionnaires:  International Personality Item Pool ( http://ipip.ori.org/ )
  • 22. Univerza v Ljubljani ..: Fakulteta za elektrotehniko:.. [LDOS] ..: Laboratorij za digitalno obdelavo signalov, slik in videa:.. LDOS PerAff-1 dataset Emotive responses Ratings Personality data Videos of facial expressions 50 users, 70 items, sparsity=0 http://slavnik.fe.uni-lj.si/markot/Main/LDOS-PerAff-1
  • 23. Univerza v Ljubljani ..: Fakulteta za elektrotehniko:.. [LDOS] ..: Laboratorij za digitalno obdelavo signalov, slik in videa:..PART III : Related work on emotions in recsys
  • 24. Univerza v Ljubljani ..: Fakulteta za elektrotehniko:.. [LDOS] ..: Laboratorij za digitalno obdelavo signalov, slik in videa:.. PART III : Related work on emotions in recsys Emotions and personality Scattered work
  • 25. Univerza v Ljubljani ..: Fakulteta za elektrotehniko:.. [LDOS] ..: Laboratorij za digitalno obdelavo signalov, slik in videa:.. Emotions in recsysGonzalez, 2007 ? Emotions as context in recsys? User affective feedback from Arapakis et al., 2009 automatic facial expression analysis
  • 26. Univerza v Ljubljani ..: Fakulteta za elektrotehniko:.. [LDOS] ..: Laboratorij za digitalno obdelavo signalov, slik in videa:.. Emotions in recsysGonzalez, 2007 ? Emotions as context in recsys? User affective feedback from Arapakis, 2009 automatic facial expression analysis Tkalčič et al., 2010 Affective user model
  • 27. Univerza v Ljubljani ..: Fakulteta za elektrotehniko:.. [LDOS] ..: Laboratorij za digitalno obdelavo signalov, slik in videa:.. Emotions in recsysGonzalez, 2007 ? Emotions as context in recsys? User affective feedback from Arapakis, 2009 automatic facial expression analysis Tkalčič et al., 2010 Affective user model find an appropriate musical score that Kaminskas, Ricci would reinforce the affective state induced 2011 by the touristic attraction.
  • 28. Univerza v Ljubljani ..: Fakulteta za elektrotehniko:.. [LDOS] ..: Laboratorij za digitalno obdelavo signalov, slik in videa:.. Emotions in recsysGonzalez, 2007 ? Emotions as context in recsys? User affective feedback from Arapakis, 2009 automatic facial expression analysis Tkalčič et al., 2010 Affective user model find an appropriate musical score that Kaminskas, Ricci would reinforce the affective state induced 2011 by the touristic attraction. Lops et al, 2012 Ongoing work: emotion detection in the phase of presentation of the recommendations for generating unexpected and seredipitous recommendations
  • 29. Univerza v Ljubljani ..: Fakulteta za elektrotehniko:..[LDOS] ..: Laboratorij za digitalno obdelavo signalov, slik in videa:..Personality in recsys Nunes et al., 2007 Personality as ??? Personality-based user similarity measure Tkalčič et al., 2009 For the cold start problem Rong Hu and Personality-based user similarity measure Pearl Pu, For the cold start problem Dennis and Masthoff, Adapting persuasive (learning) 2012 Technologies to personality traits
  • 30. Univerza v Ljubljani ..: Fakulteta za elektrotehniko:.. [LDOS] ..: Laboratorij za digitalno obdelavo signalov, slik in videa:..PART IV: Emotions in the MM consumption chain
  • 31. Univerza v Ljubljani ..: Fakulteta za elektrotehniko:.. [LDOS] ..: Laboratorij za digitalno obdelavo signalov, slik in videa:.. PART IV: Emotions in the MM consumption chain Scattered work on emotions in RecSys Unifying framework (too ambitious?)
  • 32. Univerza v Ljubljani ..: Fakulteta za elektrotehniko:..[LDOS] ..: Laboratorij za digitalno obdelavo signalov, slik in videa:..The proposed framework - 1 time choice Give Give recommendations content Content applicationEntry stage Consumption stage Exit stage
  • 33. Univerza v Ljubljani ..: Fakulteta za elektrotehniko:.. [LDOS] ..: Laboratorij za digitalno obdelavo signalov, slik in videa:.. The proposed framework - 2 time Entry mood Exit mood choice Detect Give Give entry recommendations content mood Content application• Context• Decision making• Influence• Diversification Entry stage Consumption stage Exit stage
  • 34. Univerza v Ljubljani ..: Fakulteta za elektrotehniko:.. [LDOS] ..: Laboratorij za digitalno obdelavo signalov, slik in videa:.. The proposed framework - 3 timeEntry mood Content-induced affective state choice Detect Give Give entry Observe user recommendations content mood Content application • Affective tagging • Affective user profiles Entry stage Consumption stage Exit stage
  • 35. Univerza v Ljubljani ..: Fakulteta za elektrotehniko:.. [LDOS] ..: Laboratorij za digitalno obdelavo signalov, slik in videa:.. The proposed framework - 3 timeEntry mood Content-induced affective state Exit mood choice Detect Detect Give Give entry Observe user exit recommendations content mood mood Content application • Implicit feedback • Evaluation metrics Entry stage Consumption stage Exit stage
  • 36. Univerza v Ljubljani ..: Fakulteta za elektrotehniko:.. [LDOS] ..: Laboratorij za digitalno obdelavo signalov, slik in videa:.. The proposed framework - 3 time Entry mood Content-induced affective state Exit mood choice Detect Detect Give Give entry Observe user exit recommendations content mood mood Content application• Context• Decision making • Affective tagging • Affective user profiles • Implicit feedback• Influence • Evaluation metrics• Diversification Entry stage Consumption stage Exit stage
  • 37. Univerza v Ljubljani ..: Fakulteta za elektrotehniko:.. [LDOS] ..: Laboratorij za digitalno obdelavo signalov, slik in videa:..PART V: Affect in the decision making step
  • 38. Univerza v Ljubljani ..: Fakulteta za elektrotehniko:.. [LDOS] ..: Laboratorij za digitalno obdelavo signalov, slik in videa:.. PART V: Affect in the decision making step Stage 2 and 3 are straightforward Stage 1 is interesting = new research avenues time Entry mood Content-induced affective state Exit mood choice Detect Detect Give Give entry Observe user exit recommendations content mood mood Content application • Context • Decision making • Affective tagging • Affective user profiles • Implicit feedback • Influence • Evaluation metrics • Diversification Entry stage Consumption stage Exit stage
  • 39. Univerza v Ljubljani ..: Fakulteta za elektrotehniko:.. [LDOS] ..: Laboratorij za digitalno obdelavo signalov, slik in videa:.. From data-centric to user-centric The community is problem-solving oriented – „The existing datasets are real, why building synthetic ones?“ (??, RecSys 2011) The data-centric approach is still rooted in the research community: – „It‘s about music, not about recommenders“ (?? at RecSys 2011) Solving existing problems is only a part of research ... ... the other part is generating new knowledge (on how the world works) ... ... which in turn generates new problems ... ... which in turn opens new publishing/funding/citing possibilities
  • 40. Univerza v Ljubljani ..: Fakulteta za elektrotehniko:.. [LDOS] ..: Laboratorij za digitalno obdelavo signalov, slik in videa:.. General user modeling framework Data-centric = uses data that – Is available (genres, actors, directors ...) – Easy to acquire (rating, „liking“ ...) But NOT necessarily data that carry information Controlled variables USER MODEL Prediction accuracy ? Uncontrolled variables
  • 41. Univerza v Ljubljani ..: Fakulteta za elektrotehniko:.. [LDOS] ..: Laboratorij za digitalno obdelavo signalov, slik in videa:.. LET‘S MOVE FORWARD Try new models! Generate new kind of data! Find out how the world really works! Model DECISION MAKING: – Ajzen model (Andrej‘s talk) – Kahneman model
  • 42. Univerza v Ljubljani ..: Fakulteta za elektrotehniko:.. [LDOS] ..: Laboratorij za digitalno obdelavo signalov, slik in videa:.. System 1 / System 2 (c) Kahneman, 2003
  • 43. Univerza v Ljubljani ..: Fakulteta za elektrotehniko:.. [LDOS] ..: Laboratorij za digitalno obdelavo signalov, slik in videa:.. Decision Making Modeling in RecSys Personality Content detection Affective stimuli metadata detectionEmotiondetection System 1 model System 2 model Aggregation Decision prediction
  • 44. Univerza v Ljubljani ..: Fakulteta za elektrotehniko:.. [LDOS] ..: Laboratorij za digitalno obdelavo signalov, slik in videa:.. SoA Modeling in RecSys Personality Content detection Affective stimuli metadata detectionEmotiondetection System 1 model System 2 model Aggregation Decision prediction
  • 45. Univerza v Ljubljani ..: Fakulteta za elektrotehniko:.. [LDOS] ..: Laboratorij za digitalno obdelavo signalov, slik in videa:.. Conclusions RecSys = decision making predictor Assumption = emotions do influence Scattered work  Unifying framework Our wish = Focus on stage 1: decision making: – System 1 / System 2 modeling
  • 46. Univerza v Ljubljani ..: Fakulteta za elektrotehniko:..[LDOS] ..: Laboratorij za digitalno obdelavo signalov, slik in videa:..Future work Improve models Generate dataset Validate

×