Collaborative Research | uxlx 2014

3,197 views

Published on

Slides from a half-day workshop I gave in Lisbon, on June 5, 2014.

Published in: Design, Technology

Collaborative Research | uxlx 2014

  1. 1. Collaborative Research with Erika Hall (@mulegirl) UXLX
  2. 2. Hello!
  3. 3. I have a question...
  4. 4. Do you enjoy being right?
  5. 5. You are correct!
  6. 6. YESSS!
  7. 7. p0wned!
  8. 8. No.
  9. 9. ?
  10. 10. >? !
  11. 11. Dogma
  12. 12. Flickr/Chris Voll
  13. 13. Ego!
  14. 14. Why?
  15. 15. People!
  16. 16. Collaboration!
  17. 17. Text Agenda
  18. 18. Text Agenda 9–10:30 10:30–11 11–12:30 Introduction. The research process Questions and activities Understanding your organization Break User research Analysis Models and reports Finale
  19. 19. Barriers
  20. 20. Overcoming Objections
  21. 21. Laziness
  22. 22. Laziness Fear
  23. 23. Laziness Fear Lunch
  24. 24. Laziness Fear Lunch Following
  25. 25. Laziness Fear Lunch Following Losing Control
  26. 26. Laziness Fear Lunch Following Losing Control Sharing
  27. 27. Shared Understanding
  28. 28. We don’t have the time.
  29. 29. We don’t have the money.
  30. 30. We don’t have the expertise.
  31. 31. We’re already A/B testing
  32. 32. Everyone wants better products, faster.
  33. 33. No one wants to read a report.
  34. 34. What is your experience?
  35. 35. Research + Collaboration
  36. 36. A design project is a series of decisions.
  37. 37. Data doesn’t change minds.
  38. 38. What is What ought to be
  39. 39. Design-Led Research-Led Expert Mindset Participatory Mindset Users seen as subjects Users seen as partners Design-led with expert mindset Design-led with participatory mindset Research-led with expert mindset Research-led with participatory mindset Dubberly Design Office
  40. 40. Goal Driven Skeptical Mindset Increase chance of success Reduce risk Willing to question the value of any approach
  41. 41. Team + Goal + Reality = Good
  42. 42. One Simple Process
  43. 43. Form Questions Analyze Data Gather Data
  44. 44. Form Questions Analyze Data Think Critically
  45. 45. Form Questions Analyze Data Observe
  46. 46. Form Questions Analyze Data Interview
  47. 47. Form Questions Analyze Data Read
  48. 48. Form Questions Analyze Data Read Experiment Interview Observe Think
  49. 49. Personal View Personal View Personal View
  50. 50. Shared Reality
  51. 51. Research is a Craft
  52. 52. Form Questions Analyze Data Gather Data
  53. 53. Questions determine results.
  54. 54. Questions give research meaning.
  55. 55. Research high-priority questions.
  56. 56. Good Questions Specific Actionable Practical
  57. 57. A Bad Question “What do people think about pets?”
  58. 58. A Better Question “How do single urban adults choose and acquire a pet?”
  59. 59. A Bad Question “What do people do around here all day?”
  60. 60. A Better Question “How do editors and designers work together?”
  61. 61. The Best Question The unknown with the most risk.
  62. 62. Bias
  63. 63. Bias: Something that causes an influence or prejudice
  64. 64. Confirmation Bias: You selectively weight the information that confirms what you already believe.
  65. 65. Sampling Bias: Your sample of research subjects isn’t sufficiently representative.
  66. 66. Interviewer Bias: You insert your opinion into interviews.
  67. 67. Social Desirability Bias People don’t say the true things that they worry will make them look bad.
  68. 68. Ease Clear Display Related Primed Idea Good Mood Feels True Feels Familiar Feels Good Feels Effortless Daniel Kahneman
  69. 69. Feeling confident? It’s not a good sign.
  70. 70. You might have a bad case of Dunning-Kruger.
  71. 71. Critical Thinking
  72. 72. Critical Thinking Disciplined Self-correcting Clear Logical
  73. 73. Uncritical Thinking “I hate yellow, so a yellow website won’t succeed.”
  74. 74. Critical Thinking “I hate yellow, but based on the evidence, it might work for our audience.”
  75. 75. Critical Thinking “I don’t know.”
  76. 76. Activities!
  77. 77. Form Questions Analyze Data Gather Data
  78. 78. Questions About Users ProductOrg Competition InterviewsInterviews Usability Testing A/B Testing Contextual Inquiry Literature Review SWOT Analysis Brand Audit Usability Testing Competitive Analysis Heuristic Analysis Descriptive Evaluative Evaluative Evaluative Analytic Analytic Generative Descriptive
  79. 79. Research Activity Topic Purpose Time Money
  80. 80. Phone Interviews What do we need to know about? What kind of decision will it inform? How long do we have? What is our budget? Contextual Inquiry In-Person Interviews Usability Testing Competitive Analysis
  81. 81. Why not just make a prototype?
  82. 82. 20 If we only test bottle openers, we may never realize customers prefer screw-top bottles. – Victor Lombardi, Why We Fail
  83. 83. Topics
  84. 84. Organizational Research
  85. 85. Stakeholders
  86. 86. Stakeholders
  87. 87. Executives
  88. 88. Sales People
  89. 89. Customer Service
  90. 90. Editors
  91. 91. Production Team
  92. 92. Organizational research helps you with: Requirements Politics Workflow Capabilities Goodwill
  93. 93. Requirements What are the top business priorities for this project/ product?
  94. 94. Politics What does success mean to the individual stakeholders?
  95. 95. Workflow Do we have to change how people are working together to be successful?
  96. 96. Workflow How do we have to change how people are working together to be successful?
  97. 97. Workflow How can we possibly change how people are working together?
  98. 98. Capabilities What are the strengths and weaknesses of our team?
  99. 99. Capabilities Where is the internal expertise?
  100. 100. Goodwill How can this project make your job easier (or harder)?
  101. 101. Get them alone
  102. 102. Basic Stakeholder Questions What is your title? How long have you been in this role? What are your essential duties and responsibilities? What does a typical day look like? Who are the people you work most closely with? How is that going? What does success mean from your perspective, what will have changed for the better once this project is complete? Do you have any concerns about this project? What do you think the greatest challenges to success are? Internal and external?
  103. 103. For each stakeholder, note the following: What’s their general attitude toward this project? What’s the goal as they describe it? To what extent are this person’s incentives aligned with the project’s success? How much and what type of influence do they have? Who else do they communicate with on a regular basis? To what extent does this stakeholder need to participate throughout the project, and in which role? Is what you heard in harmony or in conflict with what you’ve heard from others throughout the organization?
  104. 104. Stakeholder power moves “Why are you asking me this?” “I don’t understand that question. It doesn’t make any sense.” “I don’t feel comfortable talking to you about that.” “No one pays attention to anything I have to say, so I don’t know why I should bother talking to you.” “How much more time is this going to take?”
  105. 105. Practice!
  106. 106. 10 minutes practice. Find a partner. Take turns. What is your title? How long have you had this job? What are your essential duties and responsibilities? What is a typical day like? Who are the people you work most closely with? How is that going? What do you think the greatest challenges to your success are? Internal and external?
  107. 107. Empathy
  108. 108. Break!
  109. 109. To Review Team + Goal + Shared Reality = Good Research is a simple process you can apply to however you work. You shouldn’t be dogmatic. Even though this sounds obvious, some people will resist this because questions can feel threatening. Facts will not change the minds of people who are threatened. You need to appeal to what you know is important to them, and fit your facts into their story. So, understanding what is important to your stakeholders is necessary for design and research to succeed.
  110. 110. User Research
  111. 111. Photo: Flickr/theloushe Ethnography
  112. 112. How to do bad user research: Ask people what they want.
  113. 113. How to do bad user research: Ask people what they like.
  114. 114. Never ask users what they want or like.
  115. 115. The Four Ds of Design Ethnography
  116. 116. Deep Dive Daily Life Data Analysis Drama
  117. 117. “...true ethnography reveals not just what people say they do, but what they actually do.” –PARC
  118. 118. Photo: Flickr/lintmachine
  119. 119. The Art of The Interview
  120. 120. Interviewing is not talking.
  121. 121. Interviewing is listening.
  122. 122. Good Interviewers: Know Your Question Warm Up Shut Up
  123. 123. Interview Structure: Introduction Body Conclusion
  124. 124. Introduction: Smile Express gratitude Describe the process Ask to record Warm up questions
  125. 125. Body: Ask open-ended questions Probe for more Allow silence Use questions as checklist
  126. 126. Conclusion: Transition to wrap-up Ask if there is anything else Thank for time
  127. 127. You are the host You are the student
  128. 128. Out of your comfort zone, and into theirs.
  129. 129. Interview Checklist Create a welcoming atmosphere to make participants feel at ease. Always listen more than you speak. Take responsibility to accurately convey the thoughts and behaviors of the people you are studying. Start each interview with a general description of the goal, but be careful of focusing responses too narrowly. Avoid leading questions and closed yes/no questions. Ask follow-up questions. Prepare an outline of your interview questions in advance, but don’t be afraid to stray from it. Also note the exact phrases and vocabulary that participants use.
  130. 130. Look for Goals Priorities Tasks Motivators Barriers Habits Relationships Tools Environment
  131. 131. Roles Interviewer Notetaker Observer
  132. 132. Practice!
  133. 133. Interview Scenario You work for an e-commerce site that wants to develop a new service to help people give gifts. The goal of the research is to identify unmet needs people might have with regard to giving gifts.
  134. 134. Interview Practice Break into groups of 3-4 people 1 interviewee, 1 interviewer , 1 notetaker, 1 observer (optional), Switch in 15 minutes 2 rounds
  135. 135. Look for Goals Priorities Tasks Motivators Barriers Habits Relationships Tools Environment
  136. 136. How did that go?
  137. 137. How about a focus group?
  138. 138. 14 “Even when the subjects are well selected, focus groups are supposed to be merely the source of ideas that need to be researched.” –Robert K. Merton, Sociologist, invented focus groups
  139. 139. Everybody Lies
  140. 140. Competitive Research
  141. 141. How else might your target customer solve the same problem?
  142. 142. Competitive Review What do they say they offer? Who is their customer? How is this the same or different from your target audience or users? What are the key differentiators—the factors that make them uniquely valuable to their target market, if any? How do the user needs or wants they’re serving overlap or differ from those that you’re serving or desire to serve? What do you notice that they’re doing particularly well or badly? Based on this assessment, where do you see emerging or established conventions in how they do things, opportunities to offer something clearly superior, or good practices you’ll need to adopt or take into consideration to compete with them?
  143. 143. Your target customers have to love you more than they hate change.
  144. 144. (Usability) Testing
  145. 145. A good research activity: •Answers a key question •Addresses identified assumptions •Informs specific decisions •Involves your team •Fits your level of expertise •Fits your schedule and budget
  146. 146. •Fundamentally research is a simple process •There are many activities and definitions •No pressure! •Select the methods that inform decisions •Begin by understanding your organization •Never ask what people like •People are lazy, forgetful creatures of habit •Keep each other honest •Practice and learn
  147. 147. Research and Collaboration Working together across disciplines and making decisions based on evidence shouldn’t be hard, but they can be. Done right, research and working collaboratively reinforce each other through a shared understanding of reality. Start with your goal in mind, not with any process or buzzword. Asking questions and cutting across traditional roles can both be threatening to the established order. Commit to clear communication and critical thinking. Research questions follow from goals, assumptions, and risk. Always have a framework and a plan.
  148. 148. Creating Meaning From Data
  149. 149. 1. Compile data 2. Analyze 3. Identify Insights 4. Create Model
  150. 150. Basic Analysis Closely review the notes. Look for interesting behaviors, emotions, actions, and verbatim quotes. Write what you observed on a sticky note (coded to the source, the actual user, so you can trace it back). Group the notes. Watch the patterns emerge. Rearrange the notes as you continue to assess the patterns.
  151. 151. Observation Observation Observation Observation Observation Observation
  152. 152. Observation Observation Observation Observation Observation Observation Observation
  153. 153. Collaborates on purchases Observation Observation Observation Observation Observation Observation Observation
  154. 154. Collaborates on purchases Uses several devices Observation Observation Observation Observation Observation Observation Observation
  155. 155. Collaborates on purchases Uses several devices Needs affirmation Observation Observation Observation Observation Observation Observation Observation
  156. 156. Ground rules Acknowledge that the goal of this exercise is to better understand the context and needs of the user. Focus solely on that goal. Respect the structure of the session. Refrain from identifying larger patterns before you’ve gone through the data. Clearly differentiate observations from interpretations (what happened versus what it means). No specific solutions until after you’ve gone through insights and principles. Solutions come next.
  157. 157. Practice!
  158. 158. 20 minutes analysis. Break into groups of 6-8 people Each group work together to fill out one diagram with the strongest patterns. Negotiate and advocate for your perspective.
  159. 159. Look for Goals Priorities Tasks Motivators Barriers Habits Relationships Tools Environment
  160. 160. 20 minutes analysis. Break into groups of 6-8 people Each group work together to fill out one diagram with the strongest patterns. Negotiate and advocate for your perspective.
  161. 161. How did that go?
  162. 162. Models
  163. 163. Reporting
  164. 164. You are collaborating with your future selves.
  165. 165. Research ReportStudy Title Date Completed Research Goal Activities Related Decisions Key Insights Supporting Observations Recommended Actions
  166. 166. A useful report supports Clear goals Shared values Access to information Clear decision-making
  167. 167. You decide if it’s important for the report to be Informing? Inspiring? Focusing? Remembering? Recording? Deciding?
  168. 168. Finale
  169. 169. In summary Research creates a shared understanding of reality. Asking questions is uncomfortable. Embrace that feeling. A truly collaborative approach and environment is necessary for research to be effective, and it also makes it more fun. Clear goals and good questions are required. Choose only the research activities that answer real questions and inform your top priority design and development decisions. Practice! Observe and listen every day. Document! Report! Share! It’s easy to lose what you learn.
  170. 170. Any questions?
  171. 171. Additional sources: Designing Together by Dan A. Brown http://www.designingtogetherbook.com/ LeanUX by Jeff Gothelf http://www.leanuxbook.com/ Remote Research by Nate Bolt & Tony Tony Tulathimutte http://rosenfeldmedia.com/books/remote-research/ Interviewing Users by Steve Portigal http://rosenfeldmedia.com/books/interviewing-users/ Google Ventures Library | Design http://www.gv.com/library/design/ Pacific Standard Magazine http://www.psmag.com/ Helsinki Design Lab (closed, but excellent publications still available) http://www.helsinkidesignlab.org/pages/publications
  172. 172. Brief books for people who make websites No. 9 JUSTENOUGH RESEARCH Erika Hall You might enjoy the book. www.abookapart.com For 15% off, use code: UXLXJER14

×