!"#$%&(!$")"*)+$&,-(.%)/+!."+./How Different Users Call for Different Interaction                   Methods     EACH TO
PEOPLE DIFFER IN WHAT      THEY LIKE
PEOPLE DIFFER IN WHAT          THEY LIKE  recommender system: computing     .......likes blue                             ...
MORE DIFFERENCES...Recommendations are tailored......but the interaction is the same for every userPeople also differ in h...
TAILORED INTERFACESWe take a closer look at thesedifferences...On what characteristics do users differ?Do they use differe...
DECISION STRATEGIESA decision strategyis a procedure for making decisionsWeigh the attribute valuesPick the very first item...
PERSONAL DIFFERENCESThe decision strategy selected by theuser depends on her personalcharacteristicsBettman, Luce & Payne,...
IN SHORT...   Different user Different decisionDifferent interaction
!"#$%&(!$")"*)+$&,-(.%)/+!."+./EXPERIMENT
SETUP OF EXPERIMENT
SETUP OF EXPERIMENT     pre         post        ?           ?        ?           ?        ?           ?
THE SYSTEM
THE SYSTEM
THE SYSTEM
THE SYSTEM
FIVE INTERACTION METHODSTopN: the 10 most popular measuresBaseline condition; not personalized, virtually nointeractionGet...
FIVE INTERACTION METHODSSort: sort the measures by any attributeImplements the lexicographic strategy: ‣ Select the most i...
FIVE INTERACTION METHODSExplicit: a typical MAUT recommenderImplements the weighted adding strategy ‣   Normalize the attr...
FIVE INTERACTION METHODSImplicit: system decides on the weightsUser behavior is analyzed to update the weightsUpdate rules...
FIVE INTERACTION METHODSHybrid: Explicit + ImplicitBoth the system and the user can update theweights
PARTICIPANTS147 participants(158 at first, 11 removed due to very shortinteraction time)Recruited by an external company79 ...
PERSONAL CHARACTERISTICSDomain knowledge: Experts vs. Novices ‣ 7 items, e.g. “I understand the difference between energy ...
USER EXPERIENCEControl: Does it support my strategy? ‣ 7 items, e.g. “I had full control over the system”Understandability...
USER EXPERIENCEQUIS: Is the user interface usable? ‣ 5 items, summed 9-point scalePerceived system effectiveness: Is ituse...
!"#$%&(!$")"*)+$&,-(.%)/+!."+./RESULTS
DOMAINDifferences between experts and novices                             !"#$%&(!$")"*)+$&,-(.%)/+!."+./
DOMAIN KNOWLEDGENovices may like TopN,Sort and Implicitbecause they lack attributeknowledgeImplicit may be more confusingE...
DOMAIN KNOWLEDGENovices like                                2!the TopN                        *!system                    ...
DOMAIN KNOWLEDGENovices like                                    2!the TopN                            *!system            ...
DOMAIN KNOWLEDGENovices like                                                                       2!                     ...
DOMAIN KNOWLEDGENovices like                                                                       2!                     ...
DOMAIN KNOWLEDGENovices likethe TopNsystemThey perceivemore control inthis system thanexpertsThey find it farmore effective
DOMAIN KNOWLEDGEExperts like the                                     2!Hybrid system                   Understandability! ...
DOMAIN KNOWLEDGEExperts like the                                       2!Hybrid systemThey understand it   Understandabili...
DOMAIN KNOWLEDGEExperts like the                                                  45!Hybrid system                        ...
DOMAIN KNOWLEDGEExperts like the                                               ***!                                       ...
DOMAIN KNOWLEDGEExperts like the                                               ***!                                       ...
DOMAIN KNOWLEDGEExperts like theHybrid systemThey understand itThey are moresatisfied with its UIThey find it moreeffectiveC...
DOMAIN KNOWLEDGEChoices
DOMAIN KNOWLEDGEChoices                                                       2!                                          ...
DOMAIN KNOWLEDGEChoices                                                       2!                                          ...
DOMAIN KNOWLEDGEChoicesExperts make betterdecisions withExplicit, Implicitand Hybrid ‣   But only Hybrid is     better ove...
TRUSTINGDifferences between people who trust the                 system                             !"#$%&(!$")"*)+$&,-(.%...
TRUSTING PROPENSITYTrust is necessary toaccept therecommendationsA lack of trust can cause reactanceUsers have an initial ...
TRUSTING PROPENSITYDistrusting                                                    45!users dislike                        ...
TRUSTING PROPENSITYDistrusting                                                       45!users dislike                     ...
TRUSTING PROPENSITYDistrusting                                                           2!users dislikeExplicit,         ...
TRUSTING PROPENSITYDistrustingusers dislikeExplicit,Implicit, TopNThey are notsatisfied with theUIThey do not findthese syst...
PERSISTENCEDifferences between satisficers and            maximizers                          !"#$%&(!$")"*)+$&,-(.%)/+!."+./
PERSISTENCESatisficers may like ImplicitThe system updates therecommendations to providesimilar itemsMaximizers may likeImp...
PERSISTENCEChoices
PERSISTENCE                                                           2!Choices                                           ...
PERSISTENCE                                                           2!Choices                                           ...
PERSISTENCE                                                           2!Choices                                           ...
PERSISTENCEChoicesMaximizers aremore satisfied withtheir choices!Maximizers liketheir choices inTopNSatisficers liketheir ch...
!"#$%&(!$")"*)+$&,-(.%)/+!."+./Implications for real-world recommender                 systemsCONCLUSIO
FIRST SOME RESERVATIONS...Small sample of users28-33 participants per condition; low powerDomain encourages multiple decis...
CONCLUSIONSHybrid is better than Explicit andImplicitFor experts: tweak preferences: convenience andcontrolFor distrusting...
HOW TO COMBINE TOP-N AND        HYBRID?Spatially separate themIn different sections of the interfaceTemporally separate th...
MORE IN GENERAL...Each to his ownThe best interaction method depends on usercharacteristicsTaking these into account may r...
THANKS TO...Niels Reijmer   Martijn Willemsen
!"#$%&(!$")"*)+$&,-(.%)/+!."+./bart.k@uci.edu   @usabart   www.usabart.nlTHANK YOU
Upcoming SlideShare
Loading in …5
×

Recsys2011 presentation "Each to his own - How Different Users Call for Different Interaction Methods"

927 views

Published on

Thanks to my little brother for the awesome cartoon characters!

If you want to download the slides, I can send you a PDF.

Published in: Technology, Education
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
927
On SlideShare
0
From Embeds
0
Number of Embeds
8
Actions
Shares
0
Downloads
0
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • DON’T explain them here :-)\n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • This is what people here usually see as a baseline (aside from random, the worst possible)\n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • We did not expect this for explicit and TopN\nNo differences in control or understandability\n
  • We did not expect this for explicit and TopN\nNo differences in control or understandability\n
  • We did not expect this for explicit and TopN\nNo differences in control or understandability\n
  • We did not expect this for explicit and TopN\nNo differences in control or understandability\n
  • We did not expect this for explicit and TopN\nNo differences in control or understandability\n
  • We did not expect this for explicit and TopN\nNo differences in control or understandability\n
  • We did not expect this for explicit and TopN\nNo differences in control or understandability\n
  • We did not expect this for explicit and TopN\nNo differences in control or understandability\n
  • \n
  • \n
  • System gives them overview\n
  • System gives them overview\n
  • System gives them overview\n
  • System gives them overview\n
  • System gives them overview\n
  • System gives them overview\n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • Recsys2011 presentation "Each to his own - How Different Users Call for Different Interaction Methods"

    1. 1. !"#$%&(!$")"*)+$&,-(.%)/+!."+./How Different Users Call for Different Interaction Methods EACH TO
    2. 2. PEOPLE DIFFER IN WHAT THEY LIKE
    3. 3. PEOPLE DIFFER IN WHAT THEY LIKE recommender system: computing .......likes blue likes purple
    4. 4. MORE DIFFERENCES...Recommendations are tailored......but the interaction is the same for every userPeople also differ in how they makedecisions!They may need different ways of interacting withthe system
    5. 5. TAILORED INTERFACESWe take a closer look at thesedifferences...On what characteristics do users differ?Do they use different decision strategies?...and tailor the system to thesedifferencesWhich interaction methods support thesestrategies?
    6. 6. DECISION STRATEGIESA decision strategyis a procedure for making decisionsWeigh the attribute valuesPick the very first item you seeDifferent interaction methods maysupport these strategies to a differentextent
    7. 7. PERSONAL DIFFERENCESThe decision strategy selected by theuser depends on her personalcharacteristicsBettman, Luce & Payne, 1998Typical characteristics:Experts vs. Novices (Alba & Hutchinson, 1987; Coupey et al.,1998)Distrusting vs. Trusting (Vries, 2004; Wang & Benbasat,2007)
    8. 8. IN SHORT... Different user Different decisionDifferent interaction
    9. 9. !"#$%&(!$")"*)+$&,-(.%)/+!."+./EXPERIMENT
    10. 10. SETUP OF EXPERIMENT
    11. 11. SETUP OF EXPERIMENT pre post ? ? ? ? ? ?
    12. 12. THE SYSTEM
    13. 13. THE SYSTEM
    14. 14. THE SYSTEM
    15. 15. THE SYSTEM
    16. 16. FIVE INTERACTION METHODSTopN: the 10 most popular measuresBaseline condition; not personalized, virtually nointeractionGet more recommendations by classifying measures
    17. 17. FIVE INTERACTION METHODSSort: sort the measures by any attributeImplements the lexicographic strategy: ‣ Select the most important attribute ‣ Choose the item with the highest value on this attributeUsers can (re-)sort by clicking on table headers
    18. 18. FIVE INTERACTION METHODSExplicit: a typical MAUT recommenderImplements the weighted adding strategy ‣ Normalize the attribute values: vij ‣ Assign weights to the attributes: wj ‣ Multiply and sum to get a utility: Ui = ∑vij* wj ‣ Choose item with highest utility
    19. 19. FIVE INTERACTION METHODSImplicit: system decides on the weightsUser behavior is analyzed to update the weightsUpdate rules based on previous versions of thesystemWeights are not shown
    20. 20. FIVE INTERACTION METHODSHybrid: Explicit + ImplicitBoth the system and the user can update theweights
    21. 21. PARTICIPANTS147 participants(158 at first, 11 removed due to very shortinteraction time)Recruited by an external company79 male, 68 femaleAverage age: 40 (sd: 15.9)29 students, 93 working, 25 retired23 high school, 24 intermediate degree, 53 college,47 grad
    22. 22. PERSONAL CHARACTERISTICSDomain knowledge: Experts vs. Novices ‣ 7 items, e.g. “I understand the difference between energy saving measures”Trusting propensity: Distrusting vs.Trusting ‣ 6 items, e.g. “In general, most folks keep their promises”Persistence: Satisficers vs. Maximizers ‣ 4 items, e.g. “I am willing to examine the product attributes very carefully in order to make sure that the
    23. 23. USER EXPERIENCEControl: Does it support my strategy? ‣ 7 items, e.g. “I had full control over the system”Understandability: Is it confusing? ‣ 8 items, e.g. “I understand the system”Trust in the system: Is it fair to me? ‣ 4 items, “The system is not biased”
    24. 24. USER EXPERIENCEQUIS: Is the user interface usable? ‣ 5 items, summed 9-point scalePerceived system effectiveness: Is ituseful? ‣ 5 items, “I make better choices with this system”Choice satisfaction: Do I like what Ichose? ‣ 4 items, “I think I chose the best measures”
    25. 25. !"#$%&(!$")"*)+$&,-(.%)/+!."+./RESULTS
    26. 26. DOMAINDifferences between experts and novices !"#$%&(!$")"*)+$&,-(.%)/+!."+./
    27. 27. DOMAIN KNOWLEDGENovices may like TopN,Sort and Implicitbecause they lack attributeknowledgeImplicit may be more confusingExperts may like Explicitand Hybridbecause they can leverage theirattribute knowledge, and because
    28. 28. DOMAIN KNOWLEDGENovices like 2!the TopN *!system 1! TopN! Control! Sort! 0! -2! -1! 0! 1! 2! Explicit! Implicit! Hybrid! -1! -2! Domain Knowledge!
    29. 29. DOMAIN KNOWLEDGENovices like 2!the TopN *!system 1! TopN!They perceive Control! Sort!more control in -2! -1! 0! 0! 1! 2! Explicit!this system than Implicit!experts -1! Hybrid! -2! Domain Knowledge!
    30. 30. DOMAIN KNOWLEDGENovices like 2! ***! Perceived system effectiveness!the TopNsystem 1! 1! **! TopN!They perceive Sort!more control in -2! -1! 0! 0! 1! 2! Explicit!this system than Implicit!experts Hybrid! -1!They find it farmore effective -2! Domain Knowledge!
    31. 31. DOMAIN KNOWLEDGENovices like 2! ***! Perceived system effectiveness!the TopNsystem 1! 1! **! TopN!They perceive Sort!more control in -2! -1! 0! 0! 1! 2! Explicit!this system than Implicit!experts Hybrid! -1!They find it farmore effective -2! Domain Knowledge!
    32. 32. DOMAIN KNOWLEDGENovices likethe TopNsystemThey perceivemore control inthis system thanexpertsThey find it farmore effective
    33. 33. DOMAIN KNOWLEDGEExperts like the 2!Hybrid system Understandability! 1! *! TopN! Sort! 0! -2! -1! 0! 1! 2! Explicit! Implicit! Hybrid! -1! -2! Domain Knowledge!
    34. 34. DOMAIN KNOWLEDGEExperts like the 2!Hybrid systemThey understand it Understandability! 1! *! TopN! Sort! 0! -2! -1! 0! 1! 2! Explicit! Implicit! Hybrid! -1! -2! Domain Knowledge!
    35. 35. DOMAIN KNOWLEDGEExperts like the 45!Hybrid system 40! User interface satisfaction! 35!They understand it 30! ***!They are more 25! TopN! Sort!satisfied with its UI Explicit! 20! **! Implicit! 15! Hybrid! 10! 5! -2! -1! 0! 1! 2! Domain Knowledge!
    36. 36. DOMAIN KNOWLEDGEExperts like the ***! 2! Perceived system effectiveness!Hybrid systemThey understand it 1! 1! **!They are more TopN! Sort!satisfied with its UI -2! -1! 0! 0! 1! 2! Explicit!They find it more Implicit! Hybrid!effective -1! -2! Domain Knowledge!
    37. 37. DOMAIN KNOWLEDGEExperts like the ***! 2! Perceived system effectiveness!Hybrid systemThey understand it 1! 1! **!They are more TopN! Sort!satisfied with its UI -2! -1! 0! 0! 1! 2! Explicit!They find it more Implicit! Hybrid!effective -1!Control of Explicit -2!and convenience of Domain Knowledge!
    38. 38. DOMAIN KNOWLEDGEExperts like theHybrid systemThey understand itThey are moresatisfied with its UIThey find it moreeffectiveControl of Explicitand convenience of
    39. 39. DOMAIN KNOWLEDGEChoices
    40. 40. DOMAIN KNOWLEDGEChoices 2! *! 1!Experts make betterdecisions with Choice satisfaction! 1! *!Explicit, Implicit TopN!and Hybrid 0! Sort! ‣ But only Hybrid is -2! -1! 0! 1! 2! Explicit! Implicit! better overall Hybrid! -1! -2! Domain Knowledge!
    41. 41. DOMAIN KNOWLEDGEChoices 2! *! 1!Experts make betterdecisions with Choice satisfaction! 1! *!Explicit, Implicit TopN!and Hybrid 0! Sort! ‣ But only Hybrid is -2! -1! 0! 1! 2! Explicit! Implicit! better overall Hybrid!Novices make better -1!decisions withTopN* -2! ‣ Unable to leverage Domain Knowledge!
    42. 42. DOMAIN KNOWLEDGEChoicesExperts make betterdecisions withExplicit, Implicitand Hybrid ‣ But only Hybrid is better overallNovices make betterdecisions withTopN* ‣ Unable to leverage
    43. 43. TRUSTINGDifferences between people who trust the system !"#$%&(!$")"*)+$&,-(.%)/+!."+./
    44. 44. TRUSTING PROPENSITYTrust is necessary toaccept therecommendationsA lack of trust can cause reactanceUsers have an initial trustingpropensityDistrusting users may notlike Implicitbecause they need a system that is
    45. 45. TRUSTING PROPENSITYDistrusting 45!users dislike 40! User interface satisfaction! 35!Explicit, 1!Implicit, TopN 30! *! TopN! 25! Sort! Explicit! 20! Implicit! ****! 15! Hybrid! 10! 5! -2! -1! 0! 1! 2! Trusting propensity!
    46. 46. TRUSTING PROPENSITYDistrusting 45!users dislike 40! User interface satisfaction! 35!Explicit, 1!Implicit, TopN 30! *! TopN! 25! Sort!They are not 20! Explicit!satisfied with the Implicit!UI ****! 15! Hybrid! 10! 5! -2! -1! 0! 1! 2! Trusting propensity!
    47. 47. TRUSTING PROPENSITYDistrusting 2!users dislikeExplicit, Perceived system effectiveness! 1!Implicit, TopN TopN! Sort! 0!They are not -2! -1! 0! 1! 2! Explicit! 1!satisfied with the Implicit! Hybrid!UI -1!They do not find *!these systems *! -2!effective Trusting propensity!
    48. 48. TRUSTING PROPENSITYDistrustingusers dislikeExplicit,Implicit, TopNThey are notsatisfied with theUIThey do not findthese systemseffective
    49. 49. PERSISTENCEDifferences between satisficers and maximizers !"#$%&(!$")"*)+$&,-(.%)/+!."+./
    50. 50. PERSISTENCESatisficers may like ImplicitThe system updates therecommendations to providesimilar itemsMaximizers may likeImplicit or TopNMore counterfactual thinking, moreanticipated post-decision regretThis is aggravated in systems with
    51. 51. PERSISTENCEChoices
    52. 52. PERSISTENCE 2!Choices **!Maximizers are 1!more satisfied with Choice satisfaction!their choices! TopN! Sort! 0! -2! -1! 0! 1! 2! Explicit! Implicit! Hybrid! -1! -2! Persistence!
    53. 53. PERSISTENCE 2!Choices **!Maximizers are 1!more satisfied with Choice satisfaction!their choices! TopN! Sort!Maximizers like -2! -1! 0! 0! 1! 2! Explicit!their choices in Implicit!TopN -1! Hybrid! -2! Persistence!
    54. 54. PERSISTENCE 2!Choices **!Maximizers are 1!more satisfied with Choice satisfaction!their choices! TopN! Sort!Maximizers like -2! -1! 0! 0! 1! 2! Explicit!their choices in Implicit!TopN -1! Hybrid!Satisficers liketheir choices in -2!Implicit* Persistence!
    55. 55. PERSISTENCEChoicesMaximizers aremore satisfied withtheir choices!Maximizers liketheir choices inTopNSatisficers liketheir choices inImplicit*
    56. 56. !"#$%&(!$")"*)+$&,-(.%)/+!."+./Implications for real-world recommender systemsCONCLUSIO
    57. 57. FIRST SOME RESERVATIONS...Small sample of users28-33 participants per condition; low powerDomain encourages multiple decisions; dampensthe effectsResults pertain to attribute-basedsystemsDoes not apply to collaborative filtering
    58. 58. CONCLUSIONSHybrid is better than Explicit andImplicitFor experts: tweak preferences: convenience andcontrolFor distrusting users: negative reactions to othersystemsHowever, TopN may be better in somecasesFor novices: no knowledge to exploit the benefits
    59. 59. HOW TO COMBINE TOP-N AND HYBRID?Spatially separate themIn different sections of the interfaceTemporally separate themStart with the TopN, carefully introduce implicitrecommendations, then introduce explicit controlsAssign the correct method to each userDiscover the user’s characteristics,then tailor the interface to her specific needs
    60. 60. MORE IN GENERAL...Each to his ownThe best interaction method depends on usercharacteristicsTaking these into account may result in significantlybetter recommender systems
    61. 61. THANKS TO...Niels Reijmer Martijn Willemsen
    62. 62. !"#$%&(!$")"*)+$&,-(.%)/+!."+./bart.k@uci.edu @usabart www.usabart.nlTHANK YOU

    ×