Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Fast, Cheap andIn Control:               Using an Online Diary/Focus Group Methodology to Gauge Meaningful Experiences <br...
Q: How?<br />Short on $ budget and time<br />Single session with UI not enough<br />You want to assess more than just ‘usa...
Is there a “budget” UI test that will gauge meaningful experiences? <br />
Case Study<br />
Who would like to be happier? <br />
No problem<br />
“There’s an app for that”<br />
Signal Patterns “Live Happy” for the iPhone – a mobile happiness building application<br />
Why is this woman smiling?<br />Sonja Lyubormirsky<br />
<ul><li>What makes people happy?
Benefits of happiness?
How can we make people happier?</li></li></ul><li>And, she wrote a book about it<br />
<ul><li>Savor
Restore and Cultivate Relationships
Goal setting
Be grateful
Cultivate optimism</li></li></ul><li>Timeline: record your progress<br />
<ul><li>Personality survey
Happiness and Mood survey
 Activity “Fit” Survey </li></li></ul><li>Q: Sounds great -- but how do you put …<br />?<br />
A: Partner with a company that knows about technology …<br />
The Project<br />
Client Objectives: <br /><ul><li>Assess “proof of concept”
Assess “usability”
Assess preference of guided vs. unguided navigation
Assess price sensitivity/value
Would a donation help?  </li></li></ul><li>And, by the way …<br /><ul><li>Highly specialized audience
Requires at least a week of use in order to realize its benefits
Short time window prior to launch
Difficult to distribute the app to users
Limited budget for research</li></li></ul><li>“Can we get a copy of the app and take it for a spin?”<br />
Not quite ready for prime-time<br />Please stand by….<br />Photo: Diane Diederich<br />
Reframe Objectives to include 3 additional questions re: meaning:<br />Does the application provide personal significance ...
Recommendation: <br />Qualitative assessment<br />
Methodology: <br />Diary:<br /><ul><li>Sync to iPhone/iPod Touch
Use Live Happy prototype for 7-10 days
Provide detailed feedback via Google Docs every day during the test period</li></ul>Focus Group:<br /><ul><li>Participate ...
Include in-depth discussion and instant polling</li></li></ul><li>Why not remote UI study?<br /><ul><li>Limited research b...
Remote testing limited to a few tasks
Too much focus on “errors”
Need detailed feedback about experiences</li></li></ul><li>Recruiting tips<br /><ul><li>Keep sample small (N=10-12)
Use client resources: user or prospect email lists, etc.
Use online social networks (e.g.,  Facebook) and related user pages (e.g., ipod lounge)
Ask qualifying questions relating to specific behaviors/attitudes (e.g., price sensitivity)
Only include those with some interest in overall concept</li></li></ul><li>$ Incentive tips<br /><ul><li>Base $ amount on ...
Cash is king: coupons or merch seldom work
Don’t be afraid to overpay to insure participation – you’ve only got one shot
Spread out payments: e.g.,
 50% after complete diary entries
50% after complete focus group</li></li></ul><li>The Diary Study<br />
Upcoming SlideShare
Loading in …5
×

Fast, Cheap and In Control: Using an Online Diary/Focus Group Methodology to Gauge Meaningful Experiences

2,393 views

Published on

Your management/client is demanding that you do interface testing with actual users but you don’t have the time or money to conduct a full blown usability study. And, here’s another catch: it takes more than a single session in front of the interface for users to truly understand and appreciate – and evaluate -- its features and experience. What do you do?

This presentation uses an iPhone app case study to demonstrate tools and tips for conducting an effective online diary study, combined with an online focus group session.

We take you through all of the steps of this methodology that we’ve used successfully to test iPhone applications (for companies like Franklin Covey and Deepak Chopra) and other online interfaces that require a deeper user involvement in the interface.

Published in: Technology
  • Be the first to comment

Fast, Cheap and In Control: Using an Online Diary/Focus Group Methodology to Gauge Meaningful Experiences

  1. 1. Fast, Cheap andIn Control: Using an Online Diary/Focus Group Methodology to Gauge Meaningful Experiences <br />David Kozatch and Susanna Kirk<br />DIG<br />
  2. 2. Q: How?<br />Short on $ budget and time<br />Single session with UI not enough<br />You want to assess more than just ‘usability”<br />
  3. 3.
  4. 4. Is there a “budget” UI test that will gauge meaningful experiences? <br />
  5. 5. Case Study<br />
  6. 6. Who would like to be happier? <br />
  7. 7. No problem<br />
  8. 8. “There’s an app for that”<br />
  9. 9. Signal Patterns “Live Happy” for the iPhone – a mobile happiness building application<br />
  10. 10. Why is this woman smiling?<br />Sonja Lyubormirsky<br />
  11. 11. <ul><li>What makes people happy?
  12. 12. Benefits of happiness?
  13. 13. How can we make people happier?</li></li></ul><li>And, she wrote a book about it<br />
  14. 14. <ul><li>Savor
  15. 15. Restore and Cultivate Relationships
  16. 16. Goal setting
  17. 17. Be grateful
  18. 18. Cultivate optimism</li></li></ul><li>Timeline: record your progress<br />
  19. 19. <ul><li>Personality survey
  20. 20. Happiness and Mood survey
  21. 21. Activity “Fit” Survey </li></li></ul><li>Q: Sounds great -- but how do you put …<br />?<br />
  22. 22. A: Partner with a company that knows about technology …<br />
  23. 23.
  24. 24.
  25. 25. The Project<br />
  26. 26. Client Objectives: <br /><ul><li>Assess “proof of concept”
  27. 27. Assess “usability”
  28. 28. Assess preference of guided vs. unguided navigation
  29. 29. Assess price sensitivity/value
  30. 30. Would a donation help? </li></li></ul><li>And, by the way …<br /><ul><li>Highly specialized audience
  31. 31. Requires at least a week of use in order to realize its benefits
  32. 32. Short time window prior to launch
  33. 33. Difficult to distribute the app to users
  34. 34. Limited budget for research</li></li></ul><li>“Can we get a copy of the app and take it for a spin?”<br />
  35. 35. Not quite ready for prime-time<br />Please stand by….<br />Photo: Diane Diederich<br />
  36. 36. Reframe Objectives to include 3 additional questions re: meaning:<br />Does the application provide personal significance for users?<br />Is the interface helpful? Respectful? Interested? Forgiving?<br />Does the app take full advantage of the personalized features and interface provided by the iPhone?<br />
  37. 37. Recommendation: <br />Qualitative assessment<br />
  38. 38. Methodology: <br />Diary:<br /><ul><li>Sync to iPhone/iPod Touch
  39. 39. Use Live Happy prototype for 7-10 days
  40. 40. Provide detailed feedback via Google Docs every day during the test period</li></ul>Focus Group:<br /><ul><li>Participate in follow-up discussion online via WebEx and telephone
  41. 41. Include in-depth discussion and instant polling</li></li></ul><li>Why not remote UI study?<br /><ul><li>Limited research budget: requires greater time commitment
  42. 42. Remote testing limited to a few tasks
  43. 43. Too much focus on “errors”
  44. 44. Need detailed feedback about experiences</li></li></ul><li>Recruiting tips<br /><ul><li>Keep sample small (N=10-12)
  45. 45. Use client resources: user or prospect email lists, etc.
  46. 46. Use online social networks (e.g., Facebook) and related user pages (e.g., ipod lounge)
  47. 47. Ask qualifying questions relating to specific behaviors/attitudes (e.g., price sensitivity)
  48. 48. Only include those with some interest in overall concept</li></li></ul><li>$ Incentive tips<br /><ul><li>Base $ amount on difficulty of participation – not just audience type
  49. 49. Cash is king: coupons or merch seldom work
  50. 50. Don’t be afraid to overpay to insure participation – you’ve only got one shot
  51. 51. Spread out payments: e.g.,
  52. 52. 50% after complete diary entries
  53. 53. 50% after complete focus group</li></li></ul><li>The Diary Study<br />
  54. 54. Lose benefits of “direct observation” but gain …<br /><ul><li>UI issues in a “real world” environment
  55. 55. Users choose their own pace
  56. 56. “Unfiltered” responses
  57. 57. Adjust questions during test
  58. 58. Easy, continuous client involvement</li></li></ul><li>
  59. 59. <ul><li>Flexible and Free
  60. 60. Can reveal a small portion of the diary questions at a time via skip patterns
  61. 61. Responses automatically populate Google spreadsheet for easy review and sharing
  62. 62. All responses are dated
  63. 63. Users can upload links to pictures
  64. 64. Users trust Google</li></ul>Why we like<br />
  65. 65. Google Docs diary tips<br /><ul><li>Keep questions simple and short
  66. 66. Use mix of closed and open ends
  67. 67. Give user the ability to decide which questions they’d like to answer
  68. 68. Provide final screen text that invites user to go back and answer more q’s
  69. 69. For picture uploads, have respondents use http://imageshack.us
  70. 70. Free
  71. 71. No registration required
  72. 72. Don’t have to share own flickr photos</li></li></ul><li>
  73. 73. More diary tips<br /><ul><li>Stay in touch!
  74. 74. Use friendly language
  75. 75. Praise and cajole
  76. 76. Foster competition, if possible
  77. 77. Remind them that they are getting paid</li></li></ul><li>
  78. 78. Focus Group<br />
  79. 79. <ul><li>Allows users to share experiences
  80. 80. Allows client to hear direct feedback and probe deeper
  81. 81. Adds more depth re: userexperiences beyond initial diary entries
  82. 82. More opportunities to assess emotionallanguage
  83. 83. Can test additional interface ideas/fixes
  84. 84. Provides confirmation of hypothesis learned from diaries</li></ul>Why bother doing a focus group?<br />
  85. 85. And …<br />… can better assess meaning and user engagement<br />
  86. 86. Q:<br /><ul><li>Was the (interface) helpful?
  87. 87. Define value of the (interface) in terms of how it made you feel
  88. 88. Did you get a sense that the (interface) was listening/responding to your needs as you completed each task?
  89. 89. How does this (application) fit with how you perform related activities in your life?</li></li></ul><li><ul><li>Relatively low cost
  90. 90. Has video capability
  91. 91. Can switch from PPT to live Web easily if needed
  92. 92. Private chat feature allows clients to participate silently
  93. 93. Has polling feature with instant display of results
  94. 94. Can record session and stream to client or save as .wmv file</li></ul>Why we like<br />
  95. 95. Focus Group moderating tips<br /><ul><li>Length: 1 ½ - 2 hours
  96. 96. Keep text on PPT slides simple and uncluttered
  97. 97. Make use of screen shots/images
  98. 98. Use opportunity to follow up on individual diary posts
  99. 99. Use polling sparingly
  100. 100. Open poll prior to group discussion to nail down individual responses</li></li></ul><li>Conclusions<br />Users liked the Live Happyprototype …<br /><ul><li>Well designed and inviting
  101. 101. Can choose your own path or follow a sequence
  102. 102. Takes advantage of personalized iPhone features
  103. 103. Fast learning curve</li></li></ul><li>But initial prototype didn’t support higher price point desired by the client.<br />Why?<br />
  104. 104. Research showed that users wanted a more meaningful experience …<br /><ul><li>Respond to my activities with affirmations/positive comments
  105. 105. Clearer indications that suggestions are targeted and unique
  106. 106. Suggestions across various media
  107. 107. Integrate LiveHappy with an online account for data entry and backup
  108. 108. Add more content from Sonja's book that is inquiry based
  109. 109. Track my change for me</li></li></ul><li>Isn’t that what we all want? <br />
  110. 110. How do we get there?<br />
  111. 111. “Budget” solution<br />
  112. 112. <ul><li>Speed (“Fast”)
  113. 113. Value (“Cheap”)</li></li></ul><li>Performance<br />
  114. 114. Contact Information<br />David Kozatch, Principal<br />Ph 646.416.6784<br />david@digsmarter.com<br />Susanna Kirk, Senior Analyst<br />Ph 617.475.5105<br />susanna@digsmarter.com<br />Web: www.digsmarter.com<br />Blog: blog.digsmarter.com<br />

×