Collaborative Research The Conference by Media Evolution Malmö


Published on

Slides from half-day Collaborative Research workshop presented in Malmö on August 21, 2014.

Published in: Design

Collaborative Research The Conference by Media Evolution Malmö

  1. 1. Collaborative Research with Erika Hall (@mulegirl) The Conference | Malmö
  2. 2. Hello!
  3. 3. I have a question...
  4. 4. Do you enjoy being right?
  5. 5. You are correct!
  6. 6. YESSS!
  7. 7. p0wned!
  8. 8. No.
  9. 9. ?
  10. 10. ? > !
  11. 11. Dogma
  12. 12. Flickr/Chris Voll
  13. 13. Ego!
  14. 14. Why?
  15. 15. People!
  16. 16. Collaboration!
  17. 17. 9–10:30 AgeTexnt da 10:30–10:40 10:40–12:00 Introduction. The research process Questions and activities Understanding your organization Break User research Analysis Models and reports Finale
  18. 18. Barriers
  19. 19. Overcoming Objections
  20. 20. Laziness Fear Lunch Following Losing Control Sharing
  21. 21. Shared Understanding
  22. 22. We don’t have the time.
  23. 23. We don’t have the money.
  24. 24. We don’t have the expertise.
  25. 25. We’re already A/B testing
  26. 26. Everyone wants better products, faster.
  27. 27. No one wants to read a report.
  28. 28. What is your experience?
  29. 29. Research + Collaboration
  30. 30. A design project is a series of decisions.
  31. 31. Data doesn’t change minds.
  32. 32. What is What ought to be
  33. 33. Design-Led Research-Led Expert Mindset participatory mindset Participatory Mindset Design-led with expert mindset Design-led with Users seen as subjects Users seen as partners Research-led with expert mindset Research-led with participatory mindset Dubberly Design Office
  34. 34. Goal Driven Increase chance of success Reduce risk Skeptical Mindset Willing to question the value of any approach
  35. 35. Team + Goal + Reality = Good
  36. 36. One Simple Process
  37. 37. Form Questions Analyze Data Gather Data
  38. 38. Form Questions Analyze Data Think Critically
  39. 39. Form Questions Analyze Data Observe
  40. 40. Form Questions Analyze Data Interview
  41. 41. Form Questions Analyze Data Read
  42. 42. Form Questions Analyze Data Think Observe Interview Experiment Read
  43. 43. Personal View Personal View Personal View
  44. 44. Shared Reality
  45. 45. Research is a Craft
  46. 46. Form Questions Analyze Data Gather Data
  47. 47. Questions determine results.
  48. 48. Questions give research meaning.
  49. 49. Research high-priority questions.
  50. 50. Good Questions Specific Actionable Practical
  51. 51. A Bad Question “What do people think about pets?”
  52. 52. A Better Question “How do single urban adults choose and acquire a pet?”
  53. 53. A Bad Question “What do people do around here all day?”
  54. 54. A Better Question “How do editors and designers work together?”
  55. 55. The Best Question The unknown with the most risk.
  56. 56. Bias
  57. 57. Bias: Something that causes an influence or prejudice
  58. 58. Confirmation Bias: You selectively weight the information that confirms what you already believe.
  59. 59. Sampling Bias: Your sample of research subjects isn’t sufficiently representative.
  60. 60. Interviewer Bias: You insert your opinion into interviews.
  61. 61. Social Desirability Bias People don’t say the true things that they worry will make them look bad.
  62. 62. Ease Related Clear Display Primed Idea Good Mood Feels Familiar Feels True Feels Good Feels Effortless Daniel Kahneman
  63. 63. Feeling confident? It’s not a good sign.
  64. 64. You might have a bad case of Dunning-Kruger.
  65. 65. Critical Thinking
  66. 66. Critical Thinking Disciplined Self-correcting Clear Logical
  67. 67. Uncritical Thinking “I hate yellow, so a yellow website won’t succeed.”
  68. 68. Critical Thinking “I hate yellow, but based on the evidence, it might work for our audience.”
  69. 69. Critical Thinking “I don’t know.”
  70. 70. Activities!
  71. 71. Form Questions Analyze Data Gather Data
  72. 72. Users Interviews Interviews Descriptive Descriptive Questions About Evaluative Org Product Evaluative Competition Usability Testing A/B Testing Contextual Inquiry Literature Review SWOT Analysis Analytic Brand Audit Usability Testing Competitive Analysis Heuristic Analysis Evaluative Analytic Generative
  73. 73. Research Activity Topic Purpose Time Money
  74. 74. Phone Interviews What do we need to know about? What kind of decision will it inform? How long do we have? What is our budget? In-Person Interviews Contextual Inquiry Usability Testing Competitive Analysis
  75. 75. Why not just make a prototype?
  76. 76. If we only test bottle openers, we may never realize customers prefer screw-top bottles. – Victor Lombardi, Why We Fail 20
  77. 77. Topics
  78. 78. Organizational Research
  79. 79. Stakeholders
  80. 80. Stakeholders
  81. 81. Executives
  82. 82. Sales People
  83. 83. Customer Service
  84. 84. Editors
  85. 85. Production Team
  86. 86. Organizational research helps you with: Requirements Politics Workflow Capabilities Goodwill
  87. 87. Requirements What are the top business priorities for this project/ product?
  88. 88. Politics What does success mean to the individual stakeholders?
  89. 89. Workflow Do we have to change how people are working together to be successful?
  90. 90. Workflow How do we have to change how people are working together to be successful?
  91. 91. Workflow How can we possibly change how people are working together?
  92. 92. Capabilities What are the strengths and weaknesses of our team?
  93. 93. Capabilities Where is the internal expertise?
  94. 94. Goodwill How can this project make your job easier (or harder)?
  95. 95. Get them alone
  96. 96. Basic Stakeholder Questions What is your title? How long have you been in this role? What are your essential duties and responsibilities? What does a typical day look like? Who are the people you work most closely with? How is that going? What does success mean from your perspective, what will have changed for the better once this project is complete? Do you have any concerns about this project? What do you think the greatest challenges to success are? Internal and external?
  97. 97. For each stakeholder, note the following: What’s their general attitude toward this project? What’s the goal as they describe it? To what extent are this person’s incentives aligned with the project’s success? How much and what type of influence do they have? Who else do they communicate with on a regular basis? To what extent does this stakeholder need to participate throughout the project, and in which role? Is what you heard in harmony or in conflict with what you’ve heard from others throughout the organization?
  98. 98. Stakeholder power moves “Why are you asking me this?” “I don’t understand that question. It doesn’t make any sense.” “I don’t feel comfortable talking to you about that.” “No one pays attention to anything I have to say, so I don’t know why I should bother talking to you.” “How much more time is this going to take?”
  99. 99. Practice!
  100. 100. 10 minutes practice. Find a partner. Take turns. What is your title? How long have you had this job? What are your essential duties and responsibilities? What is a typical day like? Who are the people you work most closely with? How is that going? What do you think the greatest challenges to your success are? Internal and external?
  101. 101. Empathy
  102. 102. Break!
  103. 103. To Review Team + Goal + Shared Reality = Good Research is a simple process you can apply to however you work. You shouldn’t be dogmatic. Even though this sounds obvious, some people will resist this because questions can feel threatening. Facts will not change the minds of people who are threatened. You need to appeal to what you know is important to them, and fit your facts into their story. So, understanding what is important to your stakeholders is necessary for design and research to succeed.
  104. 104. User Research
  105. 105. Photo: Flickr/theloushe Ethnography
  106. 106. How to do bad user research: Ask people what they want.
  107. 107. How to do bad user research: Ask people what they like.
  108. 108. Never ask users what they want or like.
  109. 109. The Four Ds of Design Ethnography
  110. 110. Deep Dive Daily Life Data Analysis Drama
  111. 111. “...true ethnography reveals not just what people say they do, but what they actually do.” –PARC
  112. 112. Photo: Flickr/lintmachine
  113. 113. The Art of The Interview
  114. 114. Interviewing is not talking.
  115. 115. Interviewing is listening.
  116. 116. Good Interviewers: Know Your Question Warm Up Shut Up
  117. 117. Interview Structure: Introduction Body Conclusion
  118. 118. Introduction: Smile Express gratitude Describe the process Ask to record Warm up questions
  119. 119. Body: Ask open-ended questions Probe for more Allow silence Use questions as checklist
  120. 120. Conclusion: Transition to wrap-up Ask if there is anything else Thank for time
  121. 121. You are the host You are the student
  122. 122. Out of your comfort zone, and into theirs.
  123. 123. Interview Checklist Create a welcoming atmosphere to make participants feel at ease. Always listen more than you speak. Take responsibility to accurately convey the thoughts and behaviors of the people you are studying. Start each interview with a general description of the goal, but be careful of focusing responses too narrowly. Avoid leading questions and closed yes/no questions. Ask follow-up questions. Prepare an outline of your interview questions in advance, but don’t be afraid to stray from it. Also note the exact phrases and vocabulary that participants use.
  124. 124. Look for Goals Priorities Tasks Motivators Barriers Habits Relationships Tools Environment
  125. 125. Roles Interviewer Notetaker Observer
  126. 126. Practice!
  127. 127. Interview Scenario You want to develop a new service to help people give gifts. The goal of the research is to identify unmet needs people might have with regard to giving gifts.
  128. 128. Interview Practice Break into groups of 3-4 people 1 interviewee, 1 interviewer , 1 notetaker, 1 observer (optional), Switch in 15 minutes 2 rounds
  129. 129. Look for Goals Priorities Tasks Motivators Barriers Habits Relationships Tools Environment
  130. 130. Switch!
  131. 131. Get ready to finish.
  132. 132. Time’s Up!
  133. 133. How did that go?
  134. 134. How about a focus group?
  135. 135. “Even when the subjects are well selected, focus groups are supposed to be merely the source of ideas that need to be researched.” 14 –Robert K. Merton, Sociologist, invented focus groups
  136. 136. Everybody Lies
  137. 137. Competitive Research
  138. 138. How else might your target customer solve the same problem?
  139. 139. Competitive Review What do they say they offer? Who is their customer? How is this the same or different from your target audience or users? What are the key differentiators—the factors that make them uniquely valuable to their target market, if any? How do the user needs or wants they’re serving overlap or differ from those that you’re serving or desire to serve? What do you notice that they’re doing particularly well or badly? Based on this assessment, where do you see emerging or established conventions in how they do things, opportunities to offer something clearly superior, or good practices you’ll need to adopt or take into consideration to compete with them?
  140. 140. Your target customers have to love you more than they hate change.
  141. 141. (Usability) Testing
  142. 142. A good research activity: •Answers a key question •Addresses identified assumptions •Informs specific decisions •Involves your team •Fits your level of expertise •Fits your schedule and budget
  143. 143. •Fundamentally research is a simple process •There are many activities and definitions •No pressure! •Select the methods that inform decisions •Begin by understanding your organization •Never ask what people like •People are lazy, forgetful creatures of habit •Keep each other honest •Practice and learn
  144. 144. Research and Collaboration Working together across disciplines and making decisions based on evidence shouldn’t be hard, but they can be. Done right, research and working collaboratively reinforce each other through a shared understanding of reality. Start with your goal in mind, not with any process or buzzword. Asking questions and cutting across traditional roles can both be threatening to the established order. Commit to clear communication and critical thinking. Research questions follow from goals, assumptions, and risk. Always have a framework and a plan.
  145. 145. Creating Meaning From Data
  146. 146. 1. Compile data 2. Analyze 3. Identify Insights 4. Create Model
  147. 147. Basic Analysis Closely review the notes. Look for interesting behaviors, emotions, actions, and verbatim quotes. Write what you observed on a sticky note (coded to the source, the actual user, so you can trace it back). Group the notes. Watch the patterns emerge. Rearrange the notes as you continue to assess the patterns.
  148. 148. Observation Observation Observation Observation Observation Observation
  149. 149. Collaborates on purchases Uses several devices Needs affirmation Observation Observation Observation Observation Observation Observation Observation
  150. 150. Ground rules Acknowledge that the goal of this exercise is to better understand the context and needs of the user. Focus solely on that goal. Respect the structure of the session. Refrain from identifying larger patterns before you’ve gone through the data. Clearly differentiate observations from interpretations (what happened versus what it means). No specific solutions until after you’ve gone through insights and principles. Solutions come next.
  151. 151. Practice!
  152. 152. 15 minutes analysis. Break into groups of 6-8 people Each group work together to fill out one diagram with the strongest patterns. Negotiate and advocate for your perspective.
  153. 153. Look for Goals Priorities Tasks Motivators Barriers Habits Relationships Tools Environment
  154. 154. 15 minutes analysis. Break into groups of 3-4 people Each group work together to fill out one diagram with the strongest patterns. Negotiate and advocate for your perspective.
  155. 155. Get ready to finish.
  156. 156. Time’s Up!
  157. 157. How did that go?
  158. 158. Models
  159. 159. Reporting
  160. 160. You are collaborating with your future selves.
  161. 161. Study Title Research Report Date Completed Research Goal Related Decisions Activities Key Insights Supporting Observations Recommended Actions Questions for Further Study
  162. 162. A useful report supports Clear goals Shared values Access to information Clear decision-making
  163. 163. You decide if it’s important for the report to be Informing? Inspiring? Focusing? Remembering? Recording? Deciding?
  164. 164. Finale
  165. 165. In summary Research creates a shared understanding of reality. Asking questions is uncomfortable. Embrace that feeling. A truly collaborative approach and environment is necessary for research to be effective, and it also makes it more fun. Clear goals and good questions are required. Choose only the research activities that answer real questions and inform your top priority design and development decisions. Practice! Observe and listen every day. Document! Report! Share! It’s easy to lose what you learn.
  166. 166. Any questions?
  167. 167. Additional sources: Designing Together by Dan A. Brown LeanUX by Jeff Gothelf Remote Research by Nate Bolt & Tony Tony Tulathimutte Interviewing Users by Steve Portigal Google Ventures Library | Design Pacific Standard Magazine Helsinki Design Lab (closed, but excellent publications still available)
  168. 168. Brief books for people who make websites No. 9 Erika Hall JUST ENOUGH RESEARCH You might enjoy the book. For 15% off, use code: HALL-THE14 (Good through November 2014)