Your SlideShare is downloading. ×
0
Measuring users' experience - or, the memory of them?
Measuring users' experience - or, the memory of them?
Measuring users' experience - or, the memory of them?
Measuring users' experience - or, the memory of them?
Measuring users' experience - or, the memory of them?
Measuring users' experience - or, the memory of them?
Measuring users' experience - or, the memory of them?
Measuring users' experience - or, the memory of them?
Measuring users' experience - or, the memory of them?
Measuring users' experience - or, the memory of them?
Measuring users' experience - or, the memory of them?
Measuring users' experience - or, the memory of them?
Measuring users' experience - or, the memory of them?
Measuring users' experience - or, the memory of them?
Measuring users' experience - or, the memory of them?
Measuring users' experience - or, the memory of them?
Measuring users' experience - or, the memory of them?
Measuring users' experience - or, the memory of them?
Measuring users' experience - or, the memory of them?
Measuring users' experience - or, the memory of them?
Measuring users' experience - or, the memory of them?
Measuring users' experience - or, the memory of them?
Measuring users' experience - or, the memory of them?
Measuring users' experience - or, the memory of them?
Measuring users' experience - or, the memory of them?
Measuring users' experience - or, the memory of them?
Measuring users' experience - or, the memory of them?
Measuring users' experience - or, the memory of them?
Measuring users' experience - or, the memory of them?
Measuring users' experience - or, the memory of them?
Measuring users' experience - or, the memory of them?
Measuring users' experience - or, the memory of them?
Measuring users' experience - or, the memory of them?
Measuring users' experience - or, the memory of them?
Measuring users' experience - or, the memory of them?
Measuring users' experience - or, the memory of them?
Measuring users' experience - or, the memory of them?
Measuring users' experience - or, the memory of them?
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

Measuring users' experience - or, the memory of them?

2,083

Published on

Presentation given at Cul

Presentation given at Cul

Published in: Technology, Business
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
2,083
On Slideshare
0
From Embeds
0
Number of Embeds
3
Actions
Shares
0
Downloads
21
Comments
0
Likes
1
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  1. Measuring users’ experiences or, the memory of them?Trajectory reminders EmoSnaps Footprint tracker Evangelos Karapanos Newcastle, 8 March 2011
  2. Nuno  Nunes Vassilis  Kostakos Monchu  Chen Laura Rodríguez Gonçalo  Gouveia Néstor  Catano• 20  faculty Pedro  Campos Paulo  Sampaio Eduardo  Fermé – 14  countries,  8  languages• Areas: Larry  ConstanHne Jos  van  Leeuwen Barbara  Pizzileo – 11  CS,  2  physics/electronics,   2  psychology,  2  architecture,   2  design,  2  art,  2  other Ian  Oakley Luis  Gomes Ron  Salden Leonel  Nóbrega ValenHna  Nisi Evangelos  Karapanos David  Aveiro Luis  Gomes Yoram  Chisik
  3. MSc HCI & Entertainment Technology
  4. Industry Involvement
  5. My  backgroundBSc Physics / microelectronics, U Patras, Greece (2004)Thesis: Model based design and evaluation of walk-up-and-use interfaces (HCIGroup, ECE department)MSc HCI / UCL Interaction Centre, UK (2005)Thesis: User acceptance of nomadic user interfaces (Philips Research,Eindhoven)PhD HCI / TU Eindhoven, NL (2010)Title: Quantifying diversity in user experience
  6. Some  of  my  current  work   (that  I  will  not  discuss  today) Socially translucent eco-feedback technologies How do eco-feedback technologies: a) raise mutual awareness of family members’ consumption behaviors b) induce feelings of accountability on individuals regarding their consumption behaviors.! Citizen participation on the go How can we motivate citizen participation through mobile technologies? •Public transit: The role of psychological empowerment: self-efficacy, sense of community, and causal importance •Inclucity: The role of visual and location cues on users’ ability to reconstruct the context, and form an empathic understanding of the experience of disabled individuals Location-aware narratives: Does locality matter? Does the coupling between physical and virtual space result to increased immersion in the narrative world?
  7. OutlineUser experience over timeWhat makes for positive experiences in the long run?iScale - longitudinal data through memoryCan an online survey tool assist users in recollecting their experienceswith a product?Technology Assisted ReconstructionCan mobile sensors assist participants in reconstructing their dailyexperiences and whereabouts?
  8. Soft Reliability48% of returned products are not attributed to a violation of product specifications
  9. problems rooted earlyin (concept) design phase
  10. failure to truly incorporate it in one’s life
  11. User  experience  over  Hme An  exploratory  study How  do  users  form  overall  evalua/ve  judgments   about  interac/ve  products? Hassenzahl,  2004Karapanos, E., Hassenzahl, M., & Martens, J.-B. (2008). User experience over time. CHI ’08 extended abstracts on Human factors incomputing systems (pp. 3561-3566). Florence, Italy: ACM.
  12. User  experience  over  Hme An  exploratory  study Items  close  together  are  highly  correlated.  Lines  represent  clusters.Karapanos, E., Hassenzahl, M., & Martens, J.-B. (2008). User experience over time. CHI ’08 extended abstracts on Human factors incomputing systems (pp. 3561-3566). Florence, Italy: ACM.
  13. What makes for positive experiences in the long run?Karapanos, E., Zimmerman, J., Forlizzi, J., & Martens, J. (2009). User experience over time: an initial framework. In CHI 09:Proceedings of the 27th international conference on Human factors in computing systems (pp. 729-738). New York, NY, USA: ACM.
  14. Now  think  of  the  three  experiences  that  were  for  you  personally  most   sa.sfying  or  unsa.sfying  experiences  of  today.  Please,  use  your   own  feeling  or  a  defini.on  of  what  “sa.sfying”  and   “unsa.sfying  experience”  means.  Take  a  couple  of  minutes  to  be   sure  to  come  up  with  three  most  crucial  experiences;  you  may  also  want  to  write   them  down  for  yourself.  We  want  you  to  be  open  as  to  which  experiences  to   report.Karapanos, E., Zimmerman, J., Forlizzi, J., & Martens, J. (2009). User experience over time: an initial framework. In CHI 09:Proceedings of the 27th international conference on Human factors in computing systems (pp. 729-738). New York, NY, USA: ACM.
  15. Karapanos, E., Zimmerman, J., Forlizzi, J., & Martens, J. (2009). User experience over time: an initial framework. In CHI 09:Proceedings of the 27th international conference on Human factors in computing systems (pp. 729-738). New York, NY, USA: ACM.
  16. Cross-­‐sec8onal Repeated  sampling “Longitudinal”  paradigms  in  HCI Longitudinal Retrospec8ve
  17. Karapanos, E., Martens, J.-B., Hassenzahl, M. (2009) Reconstructing Experiences through Sketching. Arxiv preprint, arXiv:0912.5343.
  18. Remembering  is  an  act  of  reconstruc.on   rather  than  reproduc.on Barlea  (1932)
  19. How do we recall experiences?• Validity? i.e. do memories reflect what we really experienced?• Reliability? i.e. in a second trial, will we recall the same experiences? Can we assist people in recalling - more reliably - their experiences with a product?
  20. How do we recall emotional experiences?Two schools of thought - The Constructive approach - The Value-Account approach
  21. The Constructive approachThe emotional experience can neither be Design principlesstored nor retrieved. It is reconstructed 1. Feed-forward sketching, as each recalled eventfrom recalled contextual details will cue more events, eventually resulting to richer episodic memories from which to infer emotionsRobinson & Clore (2002) 2. Concurrency between sketching and reporting, as reporting will positively contribute Type of Knowledge Source of information Type of Self-Report towards the recall of episodic cues Experiential Online emotion, e.g. Episodic Knowledge Experience Sampling Retrospective, e.g. Episodic Episodic memory Day Reconstruction Situation-specific Semantic Exit questionnaires belief Identity-related Semantic Exit questionnaires belief
  22. The Value-Account approach People may recall an overall evaluation of an Design principles event even when they fail to recall contextual 1. Top-down sketching (i.e., split completed line in details - “I like it but I don’t know why” parts), as participants are expected to have direct (Betsch et al., 2001) access to this value-charged information 2. Non-concurrency between sketching and reporting, as reporting (i.e., thinking about concreteThis type of memory, Value-Account: episodic details) might hinder or bias the recall of value-charged information- is more accessible than episodic memory- can be used to cue the reconstruction from episodic memory- is better retained over time
  23. Constructive Value-Account Control (no-graphing)Constructive iScale, but not the Value-Account, performed better thancontrol condition•More experience reports•With more details (references to temporal information, discrete events)•Higher test-retest consistency of time estimation (i.e., when did an experience take place)•Higher test-retest consistency of graphed patterns (over Value- Account) 24
  24. Technology Assisted ReconstructionTrajectory reminders Emosnaps Footprint tracker Newcastle, 8 March 2011
  25. Experience Sampling Method What variables do ES studies measure? We distinguish below between self-reported measures of behavior and experience (see Table 1). This distinction is sampling method. Two of these provided no justifications for their choice. Analyzing the remaining 19 papers resulted to a total of 11 reasons for choosing alternative methods to relevant as their reconstruction follows a different process ESM (see table 2). whereas behavioral information may be directly accessible through episodic memory while experiential information Table 2. Reasons for not selecting the Experience SamplingPrompts at random, or computationally has to be further inferred from recalled episodic cues [63]. Method along with frequency of occurence (No of papers). to self-report on ongoingestimated times,measures of of ES studies eliciting or both. Table 1. Number behavior, experience, self-reported Reason Nobehaviors and experiences. Disrupts the activity 6 Type of measures that studies elicit No Imposes high burden to participants 3– Where areSelf-reported measures of behavior you? 5 Requires high effort from researchers 3– What are you doing? of experience Self-reported measures 22– How far isSelf-reported measures phone?& experience your mobile of behavior 18 Inappropriate for eliciting rich qualitative data 3– How do you feel? Misses rare and brief events 3 Behavioral measures related most frequently to the activity a few relevant venues would still miss a substantial number method while following a user-initiated diary approach. of studies. We instead used the ACM Guide towas engaged with prior to of the remaining 49 user should be reported (n=18) that the participant Computing The analysis being The studies is being in control of when, what and 2 Literature querying for the term “experience duration (n=2) (e.g. [35]), the interrupted (e.g. [30]), its sampling” below. how often to report No of papers referring to without constraining to particular venues. This query participant’s current physical location (n=15) length, sampling frequency, and response rate returned 284 papers, published in more than fifty venues. Study (e.g. [17]) Experience Sampling and the social context (n=10), e.g. the number or nature of of the studies had a duration of several The majority (80%) Limits sample size 2 60 relationship of people that are in close proximity or days up to one month with 14 studies (34%) lasting participate with in a conversation (e.g. [34]). Other seven days (see figure 3). Only two between four and Depends on participants’ ability to articulate 2 45 measures of behavior related to mode studies had a duration of more than a month. of transit [25], ongoing experience participants’ current physical engagement [18] and mode of 15 convrersation (e.g. f2f, fixed/mobile phone etc.) [32]. 30 Poses privacy concerns 2 Experiential measures related to: 15 10 • Attitudes towards behaviors or events (n=15) such as Limits number of measured variables 1 0 being interrupted (e.g. [55]), disclosing information to 2001 2003 2005 2007 2009 2011being video recorded (e.g. relevant others (e.g. [17]), or Technology limitations 1 5 [58]). Figure 2. 243 papers referring to experience sampling over a
  26. Day Reconstruction Method Kahneman et al. (2004) Can a retrospective method help participants in recalling more accurately their experiences? ry attempt to Type of Knowledge Source of information Type of Self-Reportntation of thean unfamiliar Episodic Experiential Knowledge Online emotion, e.g. Experience Sampling called stories tails, altering in applying Retrospective, e.g. Episodic Episodic memory riginal story. Day Reconstruction h repeated Situation-specific Semantic Exit questionnaireseconstruction belief e support. At ion between Identity-relatedhile episodic Semantic belief Exit questionnaires om the past,lar event but Robinson & Clore (2002)
  27. Technology  Assisted  ReconstrucHon Can mobile sensors assist participants in reconstructing their daily experiences and whereabouts? Trajectory reminders in location-based preferences Do trajectory reminders (locations visited before and after) increase the test-retest reliability of the reconstruction process? Emosnaps - inferring emotion from self-face pics Can self-face pictures assist in recalling momentary emotions? If so, is it through a recognition or a reconstruction process? Footprint tracker How do visual cues (i.e., Sensecam), location cues, and context cues (SMS and calls made or received) assist in reconstructing daily behaviors and experiences?
  28. Technology  Assisted  ReconstrucHon Can mobile sensors assist participants in reconstructing their daily experiences and whereabouts? Trajectory reminders in location-based preferences Do trajectory reminders (locations visited before and after) increase the test-retest reliability of the reconstruction process? Emosnaps - inferring emotion from self-face pics Can self-face pictures assist in recalling momentary emotions? If so, is it through a recognition or a reconstruction process? Footprint tracker How do visual cues (i.e., Sensecam), location cues, and context cues (SMS and calls made or received) assist in reconstructing daily behaviors and experiences?
  29. Control condition With trajectory reminders ! !When recalling with trajectory reminders, participants were more consistent over two repeated recalls separated by 1 week
  30. Technology  Assisted  ReconstrucHon Can mobile sensors assist participants in reconstructing their daily experiences and whereabouts? Trajectory reminders in location-based preferences Do trajectory reminders (locations visited before and after) increase the test-retest reliability of the reconstruction process? Emosnaps - inferring emotion from self-face pics Can self-face pictures assist in recalling momentary emotions? If so, is it through a recognition or a reconstruction process? Footprint tracker How do visual cues (i.e., Sensecam), location cues, and context cues (SMS and calls made or received) assist in reconstructing daily behaviors and experiences?
  31. Measuring  emoHons  in  mobile  contexts Self-Cam ’06 Wearable EMG interface ’10Teeters, Kaliouby & Picard Gruebler & Suzuki Can we develop a tool that is truly unobtrusive to daily life and can be employed in long-term field studies? Emosnaps - inferring emotion from self-face pics Can self-face pictures assist in recalling momentary emotions? If so, is it through a recognition or a reconstruction process?
  32. Experience Sampling (Ground truth) 78% of pictures could be used forinferring emotions Time-Day Photo-day Photo-week control condition Emotion reconstruction Emotion recognition
  33. Technology  Assisted  ReconstrucHon Can mobile sensors assist participants in reconstructing their daily experiences and whereabouts? Trajectory reminders in location-based preferences Do trajectory reminders (locations visited before and after) increase the test-retest reliability of the reconstruction process? Emosnaps - inferring emotion from self-face pics Can self-face pictures assist in recalling momentary emotions? If so, is it through a recognition or a reconstruction process? Footprint tracker How do visual cues (i.e., Sensecam), location cues, and context cues (SMS and calls made or received) assist in reconstructing daily behaviors and experiences?
  34. 1. Sensecam 2. Location logging 3. Context logging(SMS/calls made or received)
  35. Thank youEvangelos Karapanos ekarapanos.com User experience over time What makes for positive experiences in the long run? iScale Can an online survey tool assist users in recollecting their experiences with a product? Tech. Assist. Reconstruction Can mobile sensors assist participants in reconstructing their daily experiences and whereabouts?

×