Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Growing pains: Launching our first longitudinal study

1,254 views

Published on

In our ongoing quest to discover how our users think, our UX group conducted a four-month-long study. As we would soon discover, this was no mean feat, as there was very little precedent from the UX community on the nuts and bolts of longitudinal studies. We learned the hard way how to actually set up a longitudinal study, how to maintain it, how to collect and analyze data, and perhaps most importantly, how to get real insights that can go into design solutions. In this session, I'll present a case study of our methodology, our insights, what mistakes we made, and what you can do to make qualitative research a success in your organization.

Published in: Business, Technology
  • Be the first to comment

  • Be the first to like this

Growing pains: Launching our first longitudinal study

  1. 1. Growing pains<br />Leanna Gingras - @leegoesplaces<br />Internet User Experience 2011<br />
  2. 2. Hi. <br />IUE October 2011 @leegoesplaces<br />
  3. 3. IUE October 2011 @leegoesplaces<br />
  4. 4. IUE October 2011 @leegoesplaces<br />
  5. 5. IUE October 2011 @leegoesplaces<br />
  6. 6. IUE October 2011 @leegoesplaces<br />
  7. 7. IUE October 2011 @leegoesplaces<br />
  8. 8. Grab ‘n Go Gus<br />“this looks good enough.”<br />“that’s not it…that’s not it either…”<br />Savvy Sally<br />IUE October 2011 @leegoesplaces<br />
  9. 9. IUE October 2011 @leegoesplaces<br />
  10. 10. 1. Our research need<br />IUE October 2011 @leegoesplaces<br />http://www.flickr.com/photos/lwr/2446626352/sizes/l/in/photostream/<br />
  11. 11. “this looks good enough.”<br />Grab ‘n Go Gus<br />
  12. 12. Selecting our tools<br />IUE October 2011 @leegoesplaces<br />
  13. 13. “A longitudinal study is a correlational research study that involves repeated observations of the same variables over long periods of time.”<br />--Wikipedia<br />IUE October 2011 @leegoesplaces<br />
  14. 14. UX resources are sparse<br />http://dux.typepad.com/dux/2009/05/usability-over-time-longitudinal-research-studies.html<br />“I participated in the "Best Practices in Longitudinal Research" workshop at CHI 2009 with a small group of academics and practitioners from five countries. … Here's a brief sampling of a few topics from the day's discussions.”<br />User experience over time: An initial framework<br />“This paper presents an in-depth, five-week ethnographic study that followed 6 individuals during an actual purchase of the Apple iPhone". We found prolonged use to be motivated by different qualities than the ones that provided positive initial experiences.”<br />(and many abstracts and workshops)<br />IUE October 2011 @leegoesplaces<br />
  15. 15. IUE October 2011 @leegoesplaces<br />
  16. 16. Psychology resources galore<br />To learn how to retain participants: Minimizing respondent attrition in longitudinal research: Practical implications from a cohort study of adolescent drinking<br />More on retaining participants: Retaning Participants in Longitudinal Community Research: A Comprehensive Protocol<br />Examples of how much to pay: Psychology Department at the University of Vermont<br />IUE October 2011 @leegoesplaces<br />
  17. 17. IUE October 2011 @leegoesplaces<br />
  18. 18. 2. Planning<br />http://www.flickr.com/photos/dacotahsgirl/299198494/sizes/o/in/photostream/<br />IUE October 2011 @leegoesplaces<br />
  19. 19. Decide the research question<br />“The goal of this study is to learn how a novice researcher’s research patterns and strategies change over time.”<br />IUE October 2011 @leegoesplaces<br />
  20. 20. Decide who to study<br />“The goal of this study is to learn how a novice researcher’s research patterns and strategies change over time.”<br />IUE October 2011 @leegoesplaces<br />
  21. 21. Decide what data to collect<br />“All of it.”<br />IUE October 2011 @leegoesplaces<br />
  22. 22. Decide what data to collect<br />“We will be collecting data from usability tests, interviews, Twitter entries, diary entries, and focus group activities.”<br />IUE October 2011 @leegoesplaces<br />
  23. 23. Plan touchpoints and activities<br />1 - Initial interview & usability test<br />1 - Contact the recruits for longitudinal study<br />1 - Orientation, <br />1 – Follow-up email <br />3 - check-in emails<br />1 - Mid-semester interview based on tweets<br />1 - End-semester interview<br />1 - Focus group<br />IUE October 2011 @leegoesplaces<br />
  24. 24. Plan compensation strategy<br />IUE October 2011 @leegoesplaces<br />
  25. 25. Plan retention strategy<br />IUE October 2011 @leegoesplaces<br />
  26. 26. 3. Execution<br />http://www.flickr.com/photos/goincase/2720494170/sizes/o/in/photostream/<br />IUE October 2011 @leegoesplaces<br />
  27. 27. The actual study<br />1 usability test<br />10-30 tweets<br />At least 3 diary entries<br />3 in-person interviews<br />Focus group<br />IUE October 2011 @leegoesplaces<br />
  28. 28. Screener usability tests<br />IUE October 2011 @leegoesplaces<br />
  29. 29. Screener usability tests<br />IUE October 2011 @leegoesplaces<br />
  30. 30. Kickoff meeting<br />Agenda:<br /><ul><li>Explain study expectations and compensation
  31. 31. Plan dates for touchpoints throughout semester</li></ul>IUE October 2011 @leegoesplaces<br />
  32. 32. Twitter guidelines<br />IUE October 2011 @leegoesplaces<br />
  33. 33. Diary due dates<br />IUE October 2011 @leegoesplaces<br />
  34. 34. Diary entries<br />IUE October 2011 @leegoesplaces<br />
  35. 35. Interviews<br />Standard set of questions:<br />“Tell me about your toughest moment.”<br />Observations from Twitter/diary to build interview from:<br />“Does she understand where to find scholarly stuff?”<br />Exercise:<br />“Let’s revisit one of your earlier Tweets. Walk me through finding resources on Faust. Is there anything you’d do differently?”<br />IUE October 2011 @leegoesplaces<br />
  36. 36. End of semester focus group<br />“so how did your research change over the semester?”<br />IUE October 2011 @leegoesplaces<br />
  37. 37. End of semester focus group<br />“so how did your research change over the semester?”<br />IUE October 2011 @leegoesplaces<br />
  38. 38. End of semester workshop<br />“so how did your research change over the semester?”<br />IUE October 2011 @leegoesplaces<br />
  39. 39. IUE October 2011 @leegoesplaces<br />
  40. 40. Building activities<br />IUE October 2011 @leegoesplaces<br />
  41. 41. Articulating activities<br />IUE October 2011 @leegoesplaces<br />
  42. 42. YAY! We’re done, right? <br />http://www.flickr.com/photos/markfive/4121088992/<br />IUE October 2011 @leegoesplaces<br />
  43. 43. 4. Analysis<br />IUE October 2011 @leegoesplaces<br />
  44. 44. Analyze as we go<br /><ul><li>Does she understand where to get academic content?
  45. 45. What keywords did she try? </li></ul>http://www.island94.org/2009/09/boston-subway-in-vector-format-svg/<br />IUE October 2011 @leegoesplaces<br />
  46. 46. Asked specific questions of the data<br />“The goal of this study is to learn how a novice researcher’s research patterns and strategies change over time.”<br />What do they learn? What tools and hacks are useful to them and why? What do they hate and why? <br />IUE October 2011 @leegoesplaces<br />
  47. 47. “What do they hate and why?”<br />IUE October 2011 @leegoesplaces<br />
  48. 48. Archived everything<br />IUE October 2011 @leegoesplaces<br />
  49. 49. What’d we find out?<br />IUE October 2011 @leegoesplaces<br />
  50. 50. Finding: Emotions run high<br />IUE October 2011 @leegoesplaces<br />
  51. 51. IUE October 2011 @leegoesplaces<br />
  52. 52. Finding: Wikipedia is a broad resource<br />IUE October 2011 @leegoesplaces<br />
  53. 53. Finding: Resources drive research<br />IUE October 2011 @leegoesplaces<br />
  54. 54. Yeah, but what about the deliverables? <br />IUE October 2011 @leegoesplaces<br />
  55. 55. Cross-unit brainstorming<br />IUE October 2011 @leegoesplaces<br />
  56. 56. Persona-building<br />
  57. 57. Project steering<br />IUE October 2011 @leegoesplaces<br />
  58. 58. Recycle, recycle, recycle<br />IUE October 2011 @leegoesplaces<br />
  59. 59. 5. Lessons learned<br />IUE October 2011 @leegoesplaces<br />
  60. 60. Study management takes time<br />IUE October 2011 @leegoesplaces<br />
  61. 61. Retain ‘em<br />IUE October 2011 @leegoesplaces<br />
  62. 62. Nag, nag, nag<br />IUE October 2011 @leegoesplaces<br />
  63. 63. Make it easy for them<br />IUE October 2011 @leegoesplaces<br />
  64. 64. Adapt.<br />IUE October 2011 @leegoesplaces<br />
  65. 65. Define research need<br />Planning <br />Execution<br />Analysis and findings<br />Lessons learned<br />IUE October 2011 @leegoesplaces<br />
  66. 66. Questions?<br />http://www.flickr.com/photos/picken/5694078937/sizes/o/in/photostream/<br />IUE October 2011 @leegoesplaces<br />
  67. 67. Thanks<br />Leanna Gingras<br />@leegoesplaces<br />usabilityfail.com<br />

×